--access-key=xxx and --secret-key=yyy options

These will be used by all of the commands, as an optional alternative to the boto3 default (see https://github.com/simonw/s3-credentials/issues/1#issuecomment-958135375)

Example code is here: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html#passing-credentials-as-parameters
client = boto3.client(
    's3',
    aws_access_key_id=ACCESS_KEY,
    aws_secret_access_key=SECRET_KEY,
    aws_session_token=SESSION_TOKEN
)

Since access_key_id and secret_access_key are a bit verbose (and the _id suffix always confuses me) I'm going with --access-key and --secret-key instead.

Being able to configure the endpoint_url would be useful too - that way this could be used against compatible systems such as Wasabi.

I'm going to add four common options:

--access-key
--secret-key
--session-token
--endpoint-url

Quick test, using credentials created for a --read-only user:
% s3-credentials create static.niche-museums.com --read-only
Created user: s3.read-only.static.niche-museums.com with permissions boundary: arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess
Attached policy s3.read-only.static.niche-museums.com to user s3.read-only.static.niche-museums.com
Created access key for user: s3.read-only.static.niche-museums.com
{
    "UserName": "s3.read-only.static.niche-museums.com",
    "AccessKeyId": "AKIAWXFXAIOZOPREKLM6",
    "Status": "Active",
    "SecretAccessKey": "...",
    "CreateDate": "2021-11-03 18:53:05+00:00"
}
% s3-credentials whoami --access-key AKIAWXFXAIOZOPREKLM6 --secret-key ...
Traceback (most recent call last):
  ...
botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the GetUser operation: User: arn:aws:iam::462092780466:user/s3.read-only.static.niche-museums.com is not authorized to perform: iam:GetUser on resource: user s3.read-only.static.niche-museums.com
% s3-credentials delete-user s3.read-only.static.niche-museums.com
User: s3.read-only.static.niche-museums.com
  Deleted policy: s3.read-only.static.niche-museums.com
  Deleted access key: AKIAWXFXAIOZOPREKLM6
  Deleted user
% s3-credentials whoami --access-key AKIAWXFXAIOZOPREKLM6 --secret-key ...
Traceback (most recent call last):
  ...
botocore.exceptions.ClientError: An error occurred (InvalidClientTokenId) when calling the GetUser operation: The security token included in the request is invalid.

This looks good to me.

Help text looks like this now:
% s3-credentials create --help
Usage: s3-credentials create [OPTIONS] BUCKETS...

  Create and return new AWS credentials for specified S3 buckets

Options:
  --username TEXT                 Username to create or existing user to use
  -c, --create-bucket             Create buckets if they do not already exist
  --read-only                     Only allow reading from the bucket
  --write-only                    Only allow writing to the bucket
  --policy POLICY                 Path to a policy.json file, or literal JSON
                                  string - $!BUCKET_NAME!$ will be replaced
                                  with the name of the bucket
  --bucket-region TEXT            Region in which to create buckets
  --silent                        Don't show performed steps
  --user-permissions-boundary TEXT
                                  Custom permissions boundary to use for
                                  created users, or 'none' to create without.
                                  Defaults to limiting to S3 based on --read-
                                  only and --write-only options.
  --access-key TEXT               AWS access key ID
  --secret-key TEXT               AWS secret access key
  --session-token TEXT            AWS session token
  --endpoint-url TEXT             Custom endpoint URL
  --help                          Show this message and exit.
% s3-credentials whoami --help
Usage: s3-credentials whoami [OPTIONS]

  Identify currently authenticated user

Options:
  --access-key TEXT     AWS access key ID
  --secret-key TEXT     AWS secret access key
  --session-token TEXT  AWS session token
  --endpoint-url TEXT   Custom endpoint URL
  --help                Show this message and exit.

Documentation:

https://github.com/simonw/s3-credentials/blob/41bcc8c5100dc508de60f1f5af46551c60d92a93/README.md#configuration
https://github.com/simonw/s3-credentials/blob/41bcc8c5100dc508de60f1f5af46551c60d92a93/README.md#common-options


Make docs easier for beginners

@vladinator1000 and I thought it would be easier for beginners if the docs worked out of the box.
Should be enough to close #304

This was fun! We made sure to test that it runs using the example audio files, and it does. 😍


Dependency ws1.1.5 has security issue, should be upgraded

🐛 Bug Report

React native 0.58.6
Dependency ws1.1.5 has security issue.
Affected version of ws (0.2.6--3.3.0) are vulnerable to A specially crafted value of the Sec-WebSocket-Extensions header that used Object.prototype property names as extension or parameter names could be used to make a ws server crash.
To Reproduce

Yarn and check the yarn.lock file
Expected Behavior

Should update ws to more than 3.3.1
Code Example

<EMAIL_ADDRESS>version "0.58.6"
resolved: ...
dependencies:
ws "^1.1.0"
...
Environment

React Native Environment Info:
System:
  OS: macOS High Sierra 10.13.6
  CPU: (8) x64 Intel(R) Core(TM) i7-4770HQ CPU @ 2.20GHz
  Memory: 1.32 GB / 16.00 GB
  Shell: 5.3 - /bin/zsh
Binaries:
  Node: 11.10.1 - /usr/local/bin/node
  Yarn: 1.13.0 - /usr/local/bin/yarn
  npm: 6.7.0 - /usr/local/bin/npm
  Watchman: 4.9.0 - /usr/local/bin/watchman
SDKs:
  iOS SDK:
    Platforms: iOS 12.1, macOS 10.14, tvOS 12.1, watchOS 5.1
  Android SDK:
    API Levels: 24, 25, 26, 27, 28
    Build Tools: 24.0.2, 26.0.2, 28.0.1, 28.0.2, 28.0.3
    System Images: android-26 | Google APIs Intel x86 Atom, android-27 | Google APIs Intel x86 Atom, android-28 | Google APIs Intel x86 Atom, android-28 | Google APIs Intel x86 Atom_64
IDEs:
  Android Studio: 3.2 AI-181.55<IP_ADDRESS>56338
  Xcode: 10.1/10B61 - /usr/bin/xcodebuild
npmPackages:
  react: ^16.5.1 => 16.8.4
  react-native: 0.58.6 => 0.58.6
npmGlobalPackages:
  create-react-native-app: 2.0.2
  react-native-cli: 2.0.1
  react-native-git-upgrade: 0.2.7

👋 there - version 1.1.5 is a special release that is not affected by that security issue.


Fix --jvm-debug not working on Windows

Patch for sbt/sbt issue#5223

@XenonAbe Thanks for the contribution! Could you add a test here please? https://github.com/sbt/sbt-launcher-package/blob/master/integration-test/src/test/scala/RunnerTest.scala

I added a test and tested on...

Windows 10 with Java 8
macOS Catalina with Java 8


New component FloatingPanel

Feature request 🚀
Expected:

No breaking changes

Examples:

Programme:
Others:

@chouchouji


Let Actor to kill and resurrect itself

This is to provide an option to take down an actor during the simulation.

this has been achieved via failure injection


Missing native library dependency for dbus service

I'm using this through signal-cli, I initially opened an issue there (AsamK/signal-cli#321) and the maintainer rightfully redirected me here.
I'm having an error message with one architecture: aarch64 (rpi4)
Missing native library dependency for dbus service: could not load FFI provider jnr.ffi.provider.jffi.Provider
Is this something path related that I can fix myself or a know issue?

We definitely do ship support for Linux aarch64, so perhaps signal-cli is depending on an older version of the JNR projects?

If you could provide the full error output, that would also be helpful.
I'm not sure if this is the version of dbus-java that's being used, but it appears to be using the most recent jnr-unixsocket. It may not be the version that's getting used in your case, though.
https://github.com/hypfvieh/dbus-java/blob/master/pom.xml#L201-L205

The aarch64.so file is present in the -native.jar that is shipped with signal-cli.
I opened multiple issues because I did not know where this problem could come from. I had some answers here: https://github.com/jnr/jnr-ffi/issues/206.
Should I close this issue in favor of https://github.com/jnr/jnr-ffi/issues/206 ?

Sorry for the delay in responding here. If you can show me how to reproduce this maybe it is a simple fix. I have access to some limited aarch64 environments.

Based on jnr/jnr-ffi#206 it seems like this may simply have been a build issue in signal-cli that omitted the required jffi native jar. I am optimistically closing this as invalid, but if it turns out we are doing something wrong please let us know.


Is it possible to render TOC markdown as HTML?

Has HTML rendering been considered in mtoc's design? If not, do we have any plans to make it integrate better with the Preview plugin?
Consider following markdown with TOC:

On converting to HTML with Preview plugin it looks like this:

Hi @Kshitiz-Sharma, sorry but I can't understand what you mean...(I'm poor at English 😵 )
Would you explain more detail? or in another way?
I'm not using  Preview plugin.

Is this your problem ? (red area in image below)

Is it possible to render TOC markdown as HTML?

It's not possible.
This plugin does only 'making toc'. HTML rendering feature is out of this plugin's purpose so far.

@Kshitiz-Sharma
I found a way.  Use bracket="round" then the sublimetext-markdown-preview plugin render correctly.
<!-- MarkdownTOC autolink="true" bracket="round" -->

- [test](#test)

<!-- /MarkdownTOC -->

# test

Maybe the sublimetext-markdown-preview plugin doesn't support "Markdown standard reference-style links". See also this section of our document.


[v1.x] [MKLDNN] 2 Conv Tests Failed with MKLDNN + ACL

I was trying to build MXNet 1.x with MKLDNN with ACL (Arm Compute Library) integration on an Arm instance. I used this cmake config file to integrate MKLDNN with ACL. The build was very performant and would surely benefit MXNet users hugely. I got a 3-4X boost with (16/64, 3, 512, 512) on Resnet compared to MKLDNN with no integration. However two operator unit test failed on this build:
test_operator.test_convolution_grouping
test_operator.test_convolution_independent_gradients

I tried multiple mkldnn versions (v1.x now points to mkldnn release 2.0 beta 10, I also tried release 2.1 and 2.2) and ACl versions (20.08 and 21.2, 20.08 is Aug 2020 and 21.2 is the latest) and the failures persisted.
This got me suspect that there is some integration issue with MXNet-MKLDNN (more likely) or MKLDNN-ACL. Would someone from the Intel team help share some insights on this?
Would you help triage?@szha @leezu
CC @josephevans @mseth10 @waytrue17 @sandeep-krishnamurthy

Here is the source code of the first test case test_operator.test_convolution_grouping
https://github.com/apache/incubator-mxnet/blob/master/tests/python/unittest/test_operator.py#L1674-L1703
@with_seed()
def test_convolution_grouping():
    for dim in [1, 2, 3]:
        num_filter = 4
        for num_group in [1, 2]:
            kernel = (3,) * dim
            shape = (1, 4) + (9,) * dim

The test will only fail when dim=2 and num_group=2, i.e. when kernel=(3, 3) and shape=(1, 4, 9, 9).
So the test case will check if the Conv op will produce the same output and gradients (for x, w, b) when num_group=2 as those of when we do the "grouping" manually. So it looks like for all of output, x gradient, w gradient, and b gradient the first halves of then tensors are the same, but the second halves differ.
Mismatched elements: 98 / 196 (50%)
Max absolute difference: 16.347755
Max relative difference: 23.329699
 x: array([[[[ 1.597207e-01,  1.540166e+00,  6.755731e+00, -5.728347e+00,
           1.296894e+00, -1.326614e+00,  6.375989e+00],
         [ 1.170041e+00,  3.974693e-01,  5.603638e+00, -1.179677e+00,...
 y: array([[[[ 1.597207e-01,  1.540166e+00,  6.755731e+00, -5.728347e+00,
           1.296894e+00, -1.326614e+00,  6.375989e+00],
         [ 1.170040e+00,  3.974691e-01,  5.603638e+00, -1.179677e+00,...

Here is the source code of the second test case https://github.com/apache/incubator-mxnet/blob/master/tests/python/unittest/test_operator.py#L1750-L1829
@with_seed()
def test_convolution_independent_gradients():
    ctx = mx.cpu()
    atol = 1.0e-3
    rtol = 1.0e-3
    reqs = ["null", "write", "add"]
    var_names = ["x", "w", "b"]
    dims = [2]
    num_bases = [1, 8]
    kernel_xs = [3, 5]
    stride_xs = [1, 2]
    pad_xs = [0, 1]
    in_sizes = [7, 32]
    no_biases = [True, False]

This test will only fail when dim=2. However, if at the same time we set reqs = ["write", "add"] rather than reqs = ["null", "write", "add"], the test will also run fine.
failure:
======================================================================
FAIL: test_operator.test_convolution_independent_gradients
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/ubuntu/.local/lib/python3.6/site-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/home/ubuntu/incubator-mxnet/tests/python/unittest/common.py", line 218, in test_new
    orig_test(*args, **kwargs)
  File "/home/ubuntu/incubator-mxnet/tests/python/unittest/test_operator.py", line 2242, in test_convolution_independent_gradients
    exe2.outputs[0].asnumpy(), rtol=rtol, atol=atol)
  File "/home/ubuntu/.local/lib/python3.6/site-packages/numpy/testing/_private/utils.py", line 1528, in assert_allclose
    verbose=verbose, header=header, equal_nan=equal_nan)
  File "/home/ubuntu/.local/lib/python3.6/site-packages/numpy/testing/_private/utils.py", line 840, in assert_array_compare
    raise AssertionError(msg)
AssertionError:
Not equal to tolerance rtol=0.001, atol=0.001

Mismatched elements: 50 / 50 (100%)
Max absolute difference: 7.249808
Max relative difference: 37.313366
 x: array([[[[-3.723791,  4.639688, -0.084135,  1.286607,  5.300301],
         [ 2.492521, -4.99145 , -2.470633,  0.893196, -1.087579],
         [-0.640068, -1.888586, -1.97117 , -0.046645, -2.467756],...
 y: array([[[[ 2.861494, -1.560436,  2.371599,  6.006711,  1.072686],
         [-3.419539, -2.064956,  2.353943,  0.867886,  0.583497],
         [-3.662952,  0.055854, -2.707001, -3.451663,  2.091815],...
-------------------- >> begin captured logging << --------------------

command to run test:
nosetests-3.4 --with-coverage --cover-inclusive --cover-xml --cover-branches --cover-package=mxnet --with-timer --timer-ok 1 --timer-warning 15 --timer-filter warning,error --with-xunit --xunit-file nosetests_unittest.xml --verbose tests/python/unittest/test_operator.py:test_convolution_independent_gradients

@szha @anko-intel @josephevans I have collected more info about the two test failures. I've also prepared a clean ec2 instance with this specific mxnet build so that others can quickly get access. Please let me know about anything to try or if I can add your secret to the instance. Help is much appreciated!

Hi @Zha0q1, could you provide this container and how to build it.

Hi @Zha0q1 , can you confirm that gemm:acl is being called for the tests on grouped convolutions?
In my logs I am seeing it fallback to reference kernels (gemm:ref) when it performs the following check:https://github.com/oneapi-src/oneDNN/blob/83ebc40d86bc54f0f23e947235e53570eeacf254/src/cpu/aarch64/acl_gemm_convolution_utils.cpp#L72
    // Compute Library unsupported shape scenarios
    if (one_of(true, is_3d, is_1d, with_groups)) {
        return status::unimplemented;
    }

Hi @cfRod Thanks for taking a look at this! Yeah I will prepare a docker image for you. I am not sure about which kernel it was using -- our observation was that (MXNet + OneDNN on aarch64) had no issues whatsoever, and (MXNet + OneDNN + ACL) would fail the two tests, so it was related to the ACL integration

Hi @Zha0q1 ,
I can confirm that I've built MxNET with oneDNN and ACL and the behaviour I observed is that while running this test it fallbacks to reference kernels and the tests pass. The fallback to onednn reference is because of the check in the ACL integration layer (as I linked above).
ldd libmxnet.so
  1         linux-vdso.so.1 (0x0000ffffa7a60000)
  2         libdl.so.2 => /lib64/libdl.so.2 (0x0000ffffa1f30000)
  3         libdnnl.so.2 => /home/ubuntu/mxnet/mxnet/build/3rdparty/mkldnn/src/libdnnl.so.2 (0x0000ffffa1600000)
  4         libpthread.so.0 => /lib64/libpthread.so.0 (0x0000ffffa15c0000)
  5         libarm_compute.so => /home/ubuntu/mxnet/ComputeLibrary/build/libarm_compute.so (0x0000ffffa1230000)
  6         libarm_compute_graph.so => /home/ubuntu/mxnet/ComputeLibrary/build/libarm_compute_graph.so (0x0000ffffa1130000)
  7         libarm_compute_core.so => /home/ubuntu/mxnet/ComputeLibrary/build/libarm_compute_core.so (0x0000ffffa08b0000)
  8         libopenblas.so.0 => /home/ubuntu/mxnet/openblas/0.3.9/lib/libopenblas.so.0 (0x0000ffff9fe70000)

Yeah I will prepare a docker image for you today.
Cheers that would be great.

Hi @cfRod thanks for you reply! Hmm that's weird, anyways I am creating a docker now, will share it shortly

@cfRod I was able to reproduce the test failure within a docker container. I uploaded the image to here public.ecr.aws/y2l8i4q9/acl_debug:latest. Please let me know if you are able to pull & reproduce, thanks!
test_operator.test_convolution_grouping
test_operator.test_convolution_independent_gradients

p.s. The second test would cause a crash for some reason, but this was not an issue in my non-docker ubuntu enviroment. I think for now we can start with the first test.
/usr/local/gcc-8.5.0/include/c++/8.5.0/bits/stl_vector.h:950: std::vector<_Tp, _Alloc>::const_reference std::vector<_Tp, _Alloc>::operator[](std::vector<_Tp, _Alloc>::size_type) const [with _Tp = mxnet::OpReqType; _Alloc = std::allocator<mxnet::OpReqType>; std::vector<_Tp, _Alloc>::const_reference = const mxnet::OpReqType&; std::vector<_Tp, _Alloc>::size_type = long unsigned int]: Assertion '__builtin_expect(__n < this->size(), true)' failed.
Aborted (core dumped)

@Zha0q1, Thanks I was able to pull and reproduce your issue.
Update on my build: I was also able to reproduce it on my version of the build. I will look into this.

Hi @Zha0q1 ,
This is my WIP analysis so far (see verbose logs to support this)
The convolution with grouping is calling into gemm reference kernels, which is expected as grouped convolutions is unsupported by Compute library at the moment.
The case of manually slicing the input and running the convolution operation on the two slices of input and weights, here it is calling gemm:acl and the first half of the output is matches the output from the reference kernel but the second half doesn't. My initial analysis is that the primitive is created once and reused twice with different inputs and weights (see logs below). However Compute library assumes weights are immutable, which is because it does some preprocessing internally of the weights once for every new primitive creation and but in this case the second time around the new sets of weights are not used by ACL.
/home/ubuntu/mxnet/mxnet/src/executor/graph_executor.cc:1992: Subgraph backend MKLDNN is activated.
[14:52:42] /home/ubuntu/mxnet/mxnet/src/executor/graph_executor.cc:1992: Subgraph backend MKLDNN is activated.
dnnl_verbose,info,oneDNN v2.0.0 (commit 83ebc40d86bc54f0f23e947235e53570eeacf254)
dnnl_verbose,info,cpu,runtime:OpenMP
dnnl_verbose,info,cpu,isa:Generic
dnnl_verbose,info,gpu,runtime:none

# THIS IS GEMM REFFERENCE

dnnl_verbose,create:cache_miss,cpu,convolution,gemm:ref,forward_training,src_f32::blocked:abcd:f0 wei_f32::blocked:abcde:f0 bia_f32::blocked:a:f0 dst_f32::blocked:abcd:f0,,alg:convolution_direct,mb1_g2ic4oc4_ih9oh7kh3sh1dh0ph0_iw9ow7kw3sw1dw0pw0,0.130859
dnnl_verbose,exec,cpu,convolution,gemm:ref,forward_training,src_f32::blocked:abcd:f0 wei_f32::blocked:abcde:f0 bia_f32::blocked:a:f0 dst_f32::blocked:abcd:f0,,alg:convolution_direct,mb1_g2ic4oc4_ih9oh7kh3sh1dh0ph0_iw9ow7kw3sw1dw0pw0,19.759

# THIS IS GEMM ACL with sliced inputs
dnnl_verbose,create:cache_miss,cpu,convolution,gemm:acl,forward_training,src_f32::blocked:abcd:f0 wei_f32::blocked:abcd:f0 bia_f32::blocked:a:f0 dst_f32::blocked:abcd:f0,,alg:convolution_direct,mb1_ic2oc2_ih9oh7kh3sh1dh0ph0_iw9ow7kw3sw1dw0pw0,0.0969238
dnnl_verbose,exec,cpu,convolution,gemm:acl,forward_training,src_f32::blocked:abcd:f0 wei_f32::blocked:abcd:f0 bia_f32::blocked:a:f0 dst_f32::blocked:abcd:f0,,alg:convolution_direct,mb1_ic2oc2_ih9oh7kh3sh1dh0ph0_iw9ow7kw3sw1dw0pw0,0.415039
dnnl_verbose,exec,cpu,convolution,gemm:acl,forward_training,src_f32::blocked:abcd:f0 wei_f32::blocked:abcd:f0 bia_f32::blocked:a:f0 dst_f32::blocked:abcd:f0,,alg:convolution_direct,mb1_ic2oc2_ih9oh7kh3sh1dh0ph0_iw9ow7kw3sw1dw0pw0,0.321045

# THIS IS THE CONCAT STAGE
dnnl_verbose,create:cache_miss,cpu,concat,simple:any,undef,src_f32::blocked:abcd:f0 src_f32::blocked:abcd:f0 dst_f32::blocked:abcd:f0,,axis:1,1x2x7x7:1x2x7x7 1x4x7x7,0.0700684
dnnl_verbose,exec,cpu,concat,simple:any,undef,src_f32::blocked:abcd:f0 src_f32::blocked:abcd:f0 dst_f32::blocked:abcd:f0,,axis:1,1x2x7x7:1x2x7x7 1x4x7x7,0.468994

We've seen a similar issue with primitive caching in TensorFlow and I am looking to confirm the hypothesis, possibly by modifying the test.

@cfRod, thanks for the detailed analysis, this makes a lot of sense! Just to understand it better, is this a OneDNN & ACL integration issue (rather than MXNet & OneDNN or TensorFlow & OneDNN integration issue)? Another quick questions is that do you work with the OneDNN or Arm? I am trying to think if we need to loop in more people to fix this : )

Hi @Zha0q1, in this case, it is an issue with oneDNN and ACL rather than a framework specific issue.

do you work with the OneDNN or Arm

Yes. I am part of the team that is working on this.

Hi @Zha0q1, in this case, it is an issue with oneDNN and ACL rather than a framework specific issue.

do you work with the OneDNN or Arm

Yes. I am part of the team that is working on this.

Thanks  👍 👍

Hi @cfRod I have been tracking your discuss on the other thread, thanks for working on this! Meanwhile were you able to reproduce the other test failure test_operator.test_convolution_independent_gradients? This test "fails fine" on my native env but crashes in the docker. I am not sure why, but I just wanted to know if you could reproduce, thanks!

Would appreciate your input @cfRod :)

Yes, this seems to be an issue which shows up with -D_GLIBCXX_ASSERTIONS turned on.
In mxnet this is turned on with -DCMAKE_BUILD_TYPE=Debug.
Is your non-docker mxnet built with -DCMAKE_BUILD_TYPE=Release?

Right I think the non-docker build is RELEASE. So you were able to reproduce test_operator.test_convolution_independent_gradients?

Yes, I was able to reproduce the issue outside docker with a -DCMAKE_BUILD_TYPE=Debug build.

Hi @Zha0q1,
Could you point me to all the tests that I need to run for submitting a PR on AArch64? Is it nosetests --verbose tests/python/unittest?

Hi @Zha0q1,
Could you point me to all the tests that I need to run for submitting a PR on AArch64? Is it nosetests --verbose tests/python/unittest?

Yeah I think that's exactly right :)

Related question I raised here: https://github.com/apache/incubator-mxnet/discussions/20457

Thanks @cfRod for adding the workaround for test_convolution_grouping.
Can you also update on the status of test_convolution_independent_gradients? From your comments, I gather that you were able to reproduce it with -DCMAKE_BUILD_TYPE=Debug build flag.

Hi mseth10, I was able to reproduce the crash due to GLIBC turned on but I have not looked further on the second issue.


Be more specific about privacy by default

Privacy (Is it private?) / Overreach - Collecting Too Much Data / Privacy by default
For procedures here, aside from looking at what's disclosed, it may be worth monitoring what kind of network traffic a product is generating.
For the indicator regarding user settings, maybe specify precisely that common things like telemetry or error reporting should be opt-in.

Thanks for your procedure suggestion. In our current procedure, we capture all network traffic from all IoT devices in test. So, we know which endpoint the test device connected to by default.


feat: port Order.WithBot

SHA: 4674d16c9f219334f07b9220ea5f315838bbb573
Finished port
The main change was introducing WithBot.some and WithTop.some and defining the coercion as WithBot.some instead of Option.some to avoid elaboration issues with using Option.some.

bors merge

bors merge


some click on GitHub point to links that are currently 404

Code of Conduct

[X] I have read and agree to the GitHub Docs project's Code of Conduct

What article on docs.github.com is affected?
some click on GitHub point to links that are currently 404.
When entering /action/new path of any repo, as shown in the image below, click on the left side pointing to the link:
https://docs.github.com/actions/getting-started-with-github-actions, which is 404.

It seems that this requires modifying the click from GitHub, or restoring the 404 link in the docs to the previous webpage?
What part(s) of the article would you like to see updated?
When clicked, jump to a right link.
Additional information
No response

@heygsc Thank you for flagging the broken link! ✨ It looks like that article no longer exists and was likely intended to be replaced by newer Actions article(s).
As we will need to fix this internally, I'l flag this for our team 💛 thank you again for submitting an issue!


Remove --accept-arin-rpa and routinator-init as they are no longer required

https://www.arin.net/announcements/20220926/

Some things to do:

Update the installation instructions, initialisation page, building page and manual page at RtD.
Update the man page.
Update the README.
Update the README at Docker Hub.
Pre-init Routinator inside the Docker image including all TALs?
Remove use of --accept-arin-rpa in GitHub Actions pkg workflow, specifically in pkg/test-scripts/test-routinator.sh.

After installation with apt, immediately start running with the default settings en have systemd enable start at boot.

Removing routinator-init and changing the systemd service to be enabled and to start automatically requires additional changes in the following places:

Remove the pkg/common/routinator-init script.
Cargo.toml should no longer bundle the routinator-init script for DEB and RPM packages.
More updates to the installation and initialisation docs pages.
Remove the invocation of routinator-init entirely from pkg/test-scripts/test-routinator.sh and update it to verify that the service is started automatically, rather than starting it via systemctl start.

Should we leave routinator-init but replace it by a dummy This command no longer needs to be run or similar for one release?

For upgrade of an existing deployment, maybe the systemd service shouldn't be started if it isn't already running?

@AlexanderBand: Surely not the entirety of it as routinator init will still exist, e.g. to list TALs and choose to do custom TAL setups etc?

So, the proposed plan for this is: Get rid of installing TAL files entirely.
By default, Routinator will use the five RIR TALs that are baked into the binary. If you want to install your own TAL files, you need to explicitly specify a --tal-dir and Routinator will use the files in that directory in addition to the built-in TALs. If you don’t want the built-in TALs, you can say so via a new --no-rir-tals options.
How does that sound?

Should we leave routinator-init but replace it by a dummy This command no longer needs to be run warning or similar for one release?

We discussed this and concluded that the script would be removed entirely.

For upgrade of an existing deployment, maybe the systemd service shouldn't be started if it isn't already running?

The implementation starts the service on the basis that normal usage previously would be to do init then start the service and if the user hadn't done that already they still would have intended to do so, and normal behaviour now on initial install is to start so this makes the upgrade and initial install behaviour the same.


Use django-env to move secrets out of settings.py

Secrets and database settings should not be stored directly in settings.
Using environment variables to store configuration likely to change
between environments allows these settings to be easily updated in a
deployed application (see https://www.12factor.net/config).
For local development, these variables can be stored in a .env file
(which is excluded from version control). And example .env.example
file is included for reference.
Resolves #4

This is great! Will we eventually need to juggle multiple secret keys or database settings? Maybe there should be subdirectories for the environment files (or some other way to not have the copy and replace environment files)? Maybe a settings file where we specify the environment file?

Not sure I fully answered your question in the meeting. The idea is that each environment (your workstation, GitHub Actions server, dev server, production server) will have it's own (possibly unique) settings. The .env file is excluded from the repository. For a developer, it's probably simplest to just create a .env and fill in any settings you with to use there. For GitHub actions, it's easier to just set the environment variables before running tests. For dev and production servers they can have their own .env files with the appropriate values and/or we could use environment variables there as well. Does that help to clarify?

This is great! Will we eventually need to juggle multiple secret keys or database settings? Maybe there should be subdirectories for the environment files (or some other way to not have the copy and replace environment files)? Maybe a settings file where we specify the environment file?

Not sure I fully answered your question in the meeting. The idea is that each environment (your workstation, GitHub Actions server, dev server, production server) will have it's own (possibly unique) settings. The .env file is excluded from the repository. For a developer, it's probably simplest to just create a .env and fill in any settings you with to use there. For GitHub actions, it's easier to just set the environment variables before running tests. For dev and production servers they can have their own .env files with the appropriate values and/or we could use environment variables there as well. Does that help to clarify?

I mean, I pretty much knew all that.  What I think I was unclear on was whether we ever expected a context where we might switch back and forth between different .env files during development (and if so - make that easy to change/support where we wouldn't have to shift file names and backups of .env files around).  It appears the answer is "rarely, if ever", in which case, yes, my question is answered.


F0170 reverse chromosome

Awesome new reverseChromsomeClickEvent and a better colour gradient for link identities is ready to merge with the master branch.

Coverage remained the same at 100.0% when pulling ec5e390f66731dbe0e3a9e5fa505cd17dd58e5af on F0170_reverseChromosome into 6f4d996d8aaf2c09dabd527ae014751cab234947 on master.

Coverage remained the same at 100.0% when pulling b3ae339e98ac449391070639076167aa2145cac4 on F0170_reverseChromosome into 6f4d996d8aaf2c09dabd527ae014751cab234947 on master.


improved error message when models do not match

Priority
Low. This PR simply provides a more helpful error message when there is a misalignment defined amongst models.
What Changed & Why
Previously when the JSON API type attribute didn't match the specified Devour model the error message did not provide much indication as to where the problem lay. It read: TypeError: Cannot read property 'attributes' of undefined.
Now, when the Devour model and JSON API type attribute do not align, Devour displays a more helpful error message that reads: The JSON API response had a type of "match-result" but Devour expected the type to be "match-set"..

I love this and countless future developers love this as well.

This error will also be thrown if there is no model defined.


Add pytest-cov as test dependency

Codecov Report

:exclamation: No coverage uploaded for pull request base (main@48a4abd). Click here to learn what that means.
Patch has no changes to coverable lines.

Additional details and impacted files
@@           Coverage Diff            @@
##             main       #12   +/-   ##
========================================
  Coverage        ?   100.00%
========================================
  Files           ?         1
  Lines           ?         7
  Branches        ?         0
========================================
  Hits            ?         7
  Misses          ?         0
  Partials        ?         0

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Avoid possible divide by zero

Backport of commit 624efd2

This pull request is ready to merge

Merged to 3.1 and 3.0 branches. Thank you.


add CloudFormation Custom Resource

I updated one of our CloudFormation custom resources to use this draft implementation: https://github.com/Dwolla/postgresql-init-custom-resource/pull/6

@bpholt Apologies I wasn't so clear in my comments above, I think I'm imagining an API like this without CfnSetup, hopefully this makes a bit more sense?
abstract class CloudFormationCustomResourceHandler[Input: Decoder, Output: Encoder]
  extends IOLambda[CloudFormationCustomResourceRequest[Input], Unit] {

  // Users can override if they wish
  def client: Resource[IO, Client[IO]] = EmberClientBuilder.default[IO].build

  // client is the only shared setup, users can setup anything else they want in this resource
  def handler(client: Client[IO]): Resource[CloudFormationCustomResource[IO, Input, Output]

  // Now we combine the above to get everything we need in Setup
  type Setup = (Client[IO], CloudFormationCustomResource[IO, Input, Output])
  override final def setup: Resource[IO, Setup] = client.flatMap(client => (client, handler(client)))

  // ...
}

Yes please, a snapshot would be great

Sorry, just saw this. Do you have keys you can install that can publish under org.typelevel? Otherwise I can publish under my own group.


download: fix outdated use of page.obtain_release_key

Ironically, @harding's comment in https://github.com/bitcoin-core/bitcoincore.org/pull/803#discussion_r712468502 was spot on. This fixes a lingering reference to $(BUILDER_KEYS_TXT_URL) in the macOS and Linux instructions.

Tested ACK 578b7d606b6a52b763ca263113951a8e52c41f8d . Fixes a broken link and adds a useful test.  Thanks!


Adding Go-fmt tools and fixing go-fmt errors

part of #302

@poothia thanks for doing this. There is a basic fmt target written in root Makefile, but it does not exhaustively include all project files. It would be a good idea to integrate hack/update-gofmt.sh into that target.
Makefile has some additional targets, for example make vet, invoking go vet, which might be useful as a CI check.
We have been intending to update the CI checks invoking make rather then calling various scripts.
If that might interest you, please feel free to create an issue and claim it.

/ok-to-test

@irfanurrehman @gyliu513 thanks for reviewing this.
@irfanurrehman i have created an issue #364 regarding CI and makefile integration. i'll investigate and try to fix this.

@poothia Thank you for doing this.  In future, please separate code additions from mechanical changes in separate commits.
/lgtm
/approve


fix several issues with the autodraining lambda

This patch makes the following changes that address several issues
and cleans up the existing Lambda function:

Previously, the old Lambda function was included as a separate 8.1 MB
ZIP file that needed to be stored at S3 and managed separately from
the rest of your ECS cluster. Python code in AWS Lambda no longer
needs to bundle all of its dependencies . With all of the refactoring
above, the new Python code is small enough that it is embedded
directly in the CloudFormation template to reduce external
dependencies. This will make it easy to make changes to this code on
a branch and test it against a single ECS cluster.
The AWS code discovers the ECS cluster name by parsing the user data.
This will cause issues if the EC2 instances need to be bootstrapped
with other items through the user data script.
The AWS code can post messages to the wrong SNS topic when retrying.
It looks for the first SNS topic in the account that has a lambda
function subscribed to it and posts the retry message to that topic.
The AWS code does not do any kind of pagination against the ECS API
when reading the list of EC2 instances. So if it couldn't find the
instance ID that was about to be terminated on the first page, then
the instance was not set to DRAINING and the end users would see 50X
messages when the operation timed out and autoscaling killed the
instance.
The retry logic did not put in any kind of delay in place when
retrying. The Lambda function would be invoked about 5-10 times a
second, and each Lambda function invocation would probably
make close to a dozen AWS API calls. A 5 second delay between each
retry was introduced.
There was a large amount of unused code and variables in the in the
AWS implementation.
Converted the code from Python 2 to 3.

Totally legit, great work, thanks @masneyb 👍

Thanks! Here is a blog post that describes how we are using ECS at my employer: https://techblog.realtor.com/a-better-ecs/. It includes a CloudFormation template with the corrected autodraining Lambda.
Brian

That is correct. The sleep time is to avoid the API throttling errors
that we were encountering in our accounts (even beyond the autodraining
lambdas).
According to the AWS pricing page, the 128MB Lambdas with 128MB of RAM
get 3,200,000 free seconds per month. Beyond that, each 5 second pause
will cost cost $0.000010400. If I recall correctly, the EC2 scale down
timeout is set to 15 minutes in the template.
Brian
On Wed, Jul 11, 2018 at 11:28:31AM -0700, Leonid wrote:

Calling time.sleep(5) does not stop the execution environment. You will still pay for the duration of the execution environment, from the time the function is invoked until the time the function exits.
--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/aws-samples/ecs-cid-sample/pull/23#issuecomment-404266798

Merging post testing. Thank you for the changes in PR.


Serialize classes in case of init-based parsing

We can parse class if it has init with annotations. Probably it is ok to serialize it as well. All schema features must be included

Maybe vars(object) enough?

Need support all serialising features


C# codegen commandline options (no_server and no_client)

In #6284, new C# codegen options were merged into the code base - but I'm having some issues getting this working. I'm not familiar with the compiler side of protobuf here so I'm a bit on thin ground debugging the issue.
As far as I can tell, the intent here is for these options to be passed in as protoc --csharp_out=path --csharp_opt=no_server, but invoking the compiler with any of these new options (no_server or no_client) yields the following error message:
--csharp_out: test.proto: Unknown generator option: no_client

The internal_access flag added in the same merge seems to work as intended though, adding that as csharp_opt flags causes models to generated as internal instead of public.
Am I using this wrong? I'm on OSX 10.12.1, invoking the plugin tools/macosx_x64/grpc_csharp_plugin from the 1.0.1 version of Grpc.Tools from NuGet.

the no_server and no_client options are part of the grpc plugin ("csharp" plugin is what generates code for C# protobuf messages and is embedded in the protoc binary), so you need to rather do --grpc_opt I believe.

--grpc_opt is not a valid flag as far as I can tell, at least protoc is spitting out an Unknown flag: --grpc_opt error when I try to invoke it here.

Alright, so the right syntax to use for plugins  is --grpc_out=no_client:YOUR_OUTPUT_PATH

You can specify the options for given plugin as a prefix for the *_out path.  option=val:PATH or option:PATH. Just verified that it works.

Ah, allright - then feel free to close this issue :)
Thanks for the help!


Add Location Service and Tests

In this PR, we add a service that wraps the Location API from
https://nominatim.openstreetmap.org/search/{location}?format=json";
and add a test for this service.
Closes #8

LGTM!
The code runs well locally and looks good.


Updated copyright year

in README.md
in layouts.views/layout.erb

@despo  are you still maintaining this?


Add some Jenkins jobs

Add some Jenkins jobs to Jenkins in AWS.
Basic ones such as deploy_puppet and deploy_app.

Looks good, merge when ready.


ERR_MODULE_NOT_FOUND error while running migration in nextjs

Hey
While I am running the migration in a nextjs ts project, I am getting the below error
Error [ERR_MODULE_NOT_FOUND]: Cannot find package 'src' imported from /<path-to-project>/node_modules/kysely/dist/esm/migration/file-migration-provider.js
    at new NodeError (node:internal/errors:372:5)
    at packageResolve (node:internal/modules/esm/resolve:954:9)
    at moduleResolve (node:internal/modules/esm/resolve:1003:20)
    at defaultResolve (node:internal/modules/esm/resolve:1218:11)
    at ESMLoader.resolve (node:internal/modules/esm/loader:580:30)
    at ESMLoader.getModuleJob (node:internal/modules/esm/loader:294:18)
    at ESMLoader.import (node:internal/modules/esm/loader:380:22)
    at importModuleDynamically (node:internal/modules/esm/translators:106:35)
    at importModuleDynamicallyCallback (node:internal/process/esm_loader:35:14)
    at FileMigrationProvider.getMigrations (file:///<path-to-project>/node_modules/kysely/dist/esm/migration/file-migration-provider.js:33:60) {
  code: 'ERR_MODULE_NOT_FOUND'
}

I am using the PlanetScale Serverless Driver for the prject. But for the migrations I am using the inbuilt MysqlDialect. Below is the migration script.
import * as path from "path"
import { promises as fs } from "fs"
import { Kysely, Migrator, FileMigrationProvider, MysqlDialect } from "kysely"
import { createPool } from "mysql2"
import { config } from "dotenv"

config()

async function migrateToLatest() {
	const db = new Kysely<any>({
		dialect: new MysqlDialect({
			pool: createPool({
				host: process.env.DATABASE_HOST,
				user: process.env.DATABASE_USERNAME,
				password: process.env.DATABASE_PASSWORD,
				database: process.env.DATABASE,
				ssl: {},
			}),
		}),
	})

	const migrator = new Migrator({
		db,
		provider: new FileMigrationProvider({
			fs,
			path,
			migrationFolder: "./src/server/migrations",
		}),
	})

	const { error, results } = await migrator.migrateToLatest()

	results?.forEach((it) => {
		if (it.status === "Success") {
			console.log(`migration "${it.migrationName}" was executed successfully`)
		} else if (it.status === "Error") {
			console.error(`failed to execute migration "${it.migrationName}"`)
		}
	})

	if (error) {
		console.error("failed to migrate")
		console.error(error)
		process.exit(1)
	}

	await db.destroy()
}

migrateToLatest()

I build this file with tsc -t ES2015 --moduleResolution node scripts/migrate.ts

Hey 👋
Can you provide a repository that can reproduce this?

Hi @igalklebanov,
Here is the repo link nextjs-kysely-migration

You need to use an absolute path to the built javascript migrations. You are using a relative path to the typescript migrations. Node can't run typescript.


segment_web_sessions__initial drops data on incremental runs

Describe the bug

During incremental runs, the segment_web_sessions__initial model excludes page views from segment_web_page_views__sessionized that fall outside of the bounds of the segment_sessionization_trailing_window.
The result is that any session calculations downstream of here will be based on incomplete data.
Steps to reproduce

Run:
dbt run -m  +segment_web_sessions --full-refresh
dbt run -m  +segment_web_sessions
Expected results

This depends on your pages data, but this should return 0. (i.e. every page should be included in it's aggregated session.)
select
    count(*)
from segment_web_page_views__sessionized pages
left join segment_web_sessions sessions on pages.session_id = sessions.session_id
where pages.tstamp < sessions.session_start_tstamp

Actual results

Some page views aren't included in the aggregated session data.
Screenshots and log output

Example:
select
    pages.tstamp,
    sessions.session_start_tstamp,
    sessions.session_end_tstamp
from segment_web_page_views__sessionized pages
left join segment_web_sessions sessions on pages.session_id = sessions.session_id
where pages.tstamp < sessions.session_start_tstamp

TSTAMP
SESSION_START_TSTAMP
SESSION_END_TSTAMP

2020-11-03 12:13:29.179
2020-11-03 12:15:18.752
2020-11-03 12:15:24.659

2020-11-03 12:13:51.331
2020-11-03 12:15:18.752
2020-11-03 12:15:24.659

2020-11-03 12:13:52.435
2020-11-03 12:15:18.752
2020-11-03 12:15:24.659

The page views are at 12:13, but the session isn't said to "start" until 12:15! 🤯
System information
The contents of your packages.yml file:
packages:
  - package: fishtown-analytics/dbt_utils
    version: 0.6.2
  - package: fishtown-analytics/segment
    version: 0.5.1

Which database are you using dbt with?

[ ] postgres
[ ] redshift
[ ] bigquery
[x] snowflake
[ ] other (specify: ____________)

The output of dbt --version:
installed version: 0.18.1
   latest version: 0.18.1

Up to date!

Plugins:
  - bigquery: 0.18.1
  - snowflake: 0.18.1
  - redshift: 0.18.1
  - postgres: 0.18.1

The operating system you're using:
macOS Catalina
The output of python --version:
Python 2.7.16
Additional context

The issue seems to be arising from here: https://github.com/fishtown-analytics/segment/blob/master/models/sessionization/segment_web_sessions__initial.sql#L45
Rather than just selecting page views that occur in that window, we should select all page views for any sessions that intersect with the window.
Are you interested in contributing the fix?

PR: https://github.com/fishtown-analytics/segment/pull/64

Hey @samkessaram, thanks for the detailed writeup, and for the corresponding PRs.
If you haven't already, I'd encourage you to read this discourse post about the various trade-offs involved in incremental modeling.
This package takes the "close enough & performant" approach, rather than "always correct & slow." Based on some empirical research, a 72-hour trailing window tends to be sufficient to incorporate 99.9% of events, at the cost of missing 0.01% of late-arriving events. We're intentionally trading off absolute correctness for orders-of-magnitude better performance during incremental runs, by scanning the last 3 days of data instead of the entire table. We "catch up" to absolute correctness with a --full-refresh run once per week.
That's especially relevant on databases like BigQuery, where the cost is directly related to the amount of data scanned, but it's also a big help on databases like Redshift and Snowflake. If we need to filter on both the date (new page views) and the session_id (old page views from the same session), should we sort/cluster by the date (to speed up filter) or by session_id (to speed up implicit join criteria)?
At the same time, we know that some users will want to look back further—or they may not care much about late-arriving events at all. For that reason, segment_sessionization_trailing_window is a configurable var that can be overridden in the user's project or on the CLI.
Check out this related PR from the Snowplow package: https://github.com/fishtown-analytics/snowplow/pull/71. Previously, we had used an "always correct & slow" approach, but it just wasn't worth it. By default, it was preferable to have a speedier incremental build, with a configurable look-back window, that could be 99.9% right at scale and catch up once a week.
I think there may still be code we could improve here. For instance, we could check to see if an event is late-arriving beyond its acceptable window, such that we should exclude it from the incremental build entirely and prevent it from overwriting the existing page views and sessions.
All of that said, if in your own project, you need to implement an "always correct & slow" lookback to capture every page view for every session, no matter how late-arriving its component events, you absolutely can by disabling the package's version of segment_web_sessions__initial and adding your own model by the same name in its place:
models:
  segment:
    sessionization:
      segment_web_sessions__initial:
        +enabled: false

@jtcohen6 Thanks for the detailed response, what you're saying make sense, especially in the broader context. I had read that discourse post a while ago, but the refresher was definitely helpful. I think your periodic --full-refresh strategy is a good way to go for us. Thanks again, much appreciated. 👍


Functions that start with a _ are private

https://godotengine.org/qa/14264/gd-script-_-conventions

Closing, as this style guide should now be considered deprecated (see the official GDScript style guide).


Could not find dependency with some gradle configuration

Hello,
I started to work on the integration of the Google Plays Games in my game but I got quickly an error with the gradle configuration.
After some researches, I have exactly the same problem than this topic on the LibGdx's forum but the "solution" is weird.
It seems that, sometimes, some LibGdx projects can't use external dependencies and the solution is to create a new LibGdx project (bad solution!).

How is it a libgdx issue that you don't have the correct Google artifacts?

https://github.com/libgdx/libgdx/blob/master/CONTRIBUTING.md#using-the-issue-tracker


narrow: Add support for anchoring messages by date.

This adds a new anchor_date parameter which needs to be used in conjunction with the anchor parameter set to date value. It allows to anchor the view to the messages that are closest to the specified searched date (provided via anchor_date).
Fixes: #25436.

Self-review checklist

[x] Self-reviewed the changes for clarity and maintainability
(variable names, code reuse, readability, etc.).

Communicate decisions, questions, and potential concerns.

[x] Explains differences from previous plans (e.g., issue description).
[x] Highlights technical choices and bugs encountered.
[x] Calls out remaining decisions and concerns.
[x] Automated tests verify logic where appropriate.

Individual commits are ready for review (see commit discipline).

[x] Each commit is a coherent idea.
[x] Commit message(s) explain reasoning and motivation for changes.

Completed manual review and testing of the following:

[ ] Visual appearance of the changes.
[ ] Responsiveness and internationalization.
[ ] Strings and tooltips.
[ ] End-to-end functionality of buttons, interactions and flows.
[x] Corner cases, error conditions, and easily imagined bugs.

These are the backend changes will add the frontend changes in the next PR.

Thanks for the work on this @akshatdalton! I posted a batch of initial comments. I think the main structural thing that we need to decide in "api design" is how we want to handle the timezone/UTC issue.

@timabbott I've updated the PR addressing the comments.

@zulipbot add "buddy review"
@zulipbot add "mentor review"

@akshatdalton do you have time to rebase this PR and get the backend tests passing? I'd like to try to integrate the API part of this next.

Sure, will do it!


Incorrect order in categories

Categories are sorted by count ASC, they should be sorted by count DESC. Categories with more results should be first in the list.

Fixed, thanks rochoa.


AddApplicationInsights() calls aren't available

In the doc, this code is shown:
            .AddApplicationInsights()
            .AddApplicationInsightsLogger();

however said code does not compile in a vanilla isolated .NET fxn:

Until you add a preview nupkg:

which is not mentioned in the docs (and kinda bad given it's not GA)

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 6ad5f362-cbfb-0621-4389-6378f50359e4
Version Independent ID: ebf1a8f3-ab9d-d295-e3a4-a982ed9df1e9
Content: Guide for running C# Azure Functions in an isolated process
Content Source: articles/azure-functions/dotnet-isolated-process-guide.md
Service: azure-functions
GitHub Login: @ggailey777
Microsoft Alias: glenga

@brandonh-msft Thanks for the feedback. We will update as appropriate.


Add read-only access to ux0:/app and ur0:/appmeta

This could be a lot useful for homebrews/plugins that need to interact with games to grab title, icon0 and other useful metadata.

Thanks for the suggestion, but it seems to be a lot of work. With the release of taiHEN, we invite developers to write their own CFW features if they so desire.


generate longer answers with Phi2 model

This PR

updates the phi2 tutorial
updates instruct prompt for the phi2 model. appending \n to instruct prompt makes phi2 generate longer answers.

With "Instruct:{prompt}\nOutput:"
>> Prompt: Write a detailed analogy between mathematics and a lighthouse.
>> Reply:  Mathematics is like a lighthouse. Mathematics provides a method to guide us through the sometimes chaotic and confusing waters of life.
It provides a structured approach to problems which can help us find our way and provide direction.
Just as a lighthouse keeps watch over the sea, mathematics can provide us with the tools to try and make sense of the world.
Furthermore, just as a lighthouse keeps a watchful eye on the horizon, mathematics can help us reach our goals by showing us the way.

With "Instruct:{prompt}\nOutput:\n"
>> Prompt: Write a detailed analogy between mathematics and a lighthouse.

>> Reply: Mathematics is like a lighthouse, guiding and illuminating the path to understanding and knowledge. Just as a lighthouse emits a focused beam of light to guide ships safely to shore, mathematics provides a clear and structured framework for logical reasoning and problem-solving.

Similar to how a lighthouse uses its beam to reveal hidden obstacles and dangers to ships, mathematics allows us to uncover patterns, relationships, and hidden truths. It provides a structured language and tools, such as numbers, equations, and formulas, that help us navigate through complex problems and make sense of the world.

Furthermore, a lighthouse serves as a beacon of hope and direction in times of darkness and uncertainty. Similarly, mathematics offers a sense of certainty and reliability in our pursuit of knowledge. It provides a solid foundation and framework upon which other subjects and disciplines can build, much like how a lighthouse supports and guides ships on their journey.

Moreover, both mathematics and a lighthouse require careful planning, precision, and attention to detail. Just as a lighthouse needs to be strategically placed and calibrated to ensure its beam reaches the desired destination, mathematics requires careful calculation and logical reasoning to reach the correct solution.

In conclusion, mathematics and a lighthouse share the qualities of guiding and illuminating the way, uncovering hidden truths, providing direction, and requiring precision and attention to detail.

agree, its more about prompt engineering than core functionality. I will close this PR and open new one with only tutorial update


64-bit Python

Please add 64-bit Python binary for phones with 64-bit CPU.

it runs on my oneplus 5t with snapdragon 835.That is 64 bit


MSE incorrect when using fit_generator and max_queue_size != 1

I am getting unexpected behaviour when using mean_squared_error loss.
My original issue was that fit_generator was reporting an incorrect validation loss. I was unable to reproduce that bug in a gist that I can make public, but I was able to demonstrate an incorrect training loss.
Bafflingly, if I change max_queue_size to 1 instead of the default 10 in fit_generator, the issue is resolved.
https://gist.github.com/alekseynp/1bb6cbe4ee7fb6e8ea46d68c675b934d

Not sure what you really mean (you don't provide much info), but note that the docs say that using fit_generator with multiple workers and a Python generator that isn't a Sequence instance, can result in duplicated data. Maybe that's what you are observing.
To avoid such data duplication issues, you can either use a single worker, or implement a Sequence instead of a generator to produce the data to train your model.

Sorry for not providing more info. I hoped that the gist would be demonstrative. I thought this was an MSE-related error, but that seems to not be the case now. I'll admit it seems kind of crazy that it would be MSE-specific.
I tried implementing a Sequence and was successful. This led me to review my code that has the issue, so here's some more analysis.
fit_generator by default uses max_queue_size=10, workers=1, and use_multiprocessing=False which leads me to believe that I do not need to take special steps to make my code "safe" for multiprocessing. However, this is not the case.
When I use the following generator and the default settings for fit_generator, I have problems which I see manifest in incorrectly reported training_loss and validation_loss.
def my_generator(batch_size):
    inputs = np.zeros((batch_size,10), dtype=np.float32)
    outputs = np.zeros((batch_size,1), dtype=np.float32)

    while True:
        for i in range(batch_size):
            inputs[i,:] = np.random.random(10)
            outputs[i,0] = np.sum(np.arange(0,10) * inputs[i,:])

        yield inputs, outputs

If I change max_queue_size to 1, my problems are resolved.
When I move the array creation inside the while True loop, my issue is also resolved.
def my_generator(batch_size):
    while True:
        inputs = np.zeros((batch_size,10), dtype=np.float32)
        outputs = np.zeros((batch_size,1), dtype=np.float32)

        for i in range(batch_size):
            inputs[i,:] = np.random.random(10)
            outputs[i,0] = np.sum(np.arange(0,10) * inputs[i,:])

        yield inputs, outputs

So obviously for resolving my own problem, the key is to not re-use the same array in my generator. And I would hope that other people with issues will find this and find the same resolution. From a Keras perspective, this seems like a gotcha for users falsely thinking that they are insulated from problems related to multi-processing.


Open webui cant use o1-model

Bug Report
Installation Method
I installed Open WebUI using Docker(GPU version) by pulling the official Docker image and running it according to the provided documentation.
Environment

Open WebUI Version: v0.3.11
Operating System: Ubuntu 22.04 LTS
Browser (if applicable): Google Chrome 112.0.5615.49

Confirmation:

[x] I have read and followed all the instructions provided in the README.md.
[x] I am on the latest version of Open WebUI.
[x] I have included the browser console logs.
[x] I have included the Docker container logs.
[x] I have provided the exact steps to reproduce the bug in the "Steps to Reproduce" section below.

Expected Behavior
I expect to be able to use the o1-model within Open WebUI seamlessly, allowing for model loading and inference without any errors.
Actual Behavior
When attempting to use the o1-model in Open WebUI, the system fails to recognize the model and displays an error message. The model fails to load, rendering the related functionalities unusable.
Description
Bug Summary:
Unable to use the o1-model in Open WebUI as the system does not recognize or load the model, causing related features to fail.
Reproduction Details
Steps to Reproduce:

Pull and run the Open WebUI Docker image using the following commands:
docker pull openwebui/openwebui:latest
docker run -d -p 8080:8080 openwebui/openwebui:latest

Place the o1-model in the designated models directory within the Docker container.

Navigate to the models management page in Open WebUI.

Attempt to add the o1-model.

Observe that the system displays an error message and fails to load the model.

Logs and Screenshots
Browser Console Logs:
Docker Container Logs:
Screenshots/Screen Recordings (if applicable):

Additional Information
I have verified that the o1-model files are correctly placed in the models directory and that other models are functioning correctly within Open WebUI. The issue only occurs when attempting to use the o1-model. I suspect that the problem might be related to the model file format or version incompatibility.

Likewise. Same issue.

Yes, I am too

Unsupported value: 'stream' does not support true with this model. Only the default (false) value is supported.

Unsupported value: 'stream' does not support true with this model. Only the default (false) value is supported.
https://platform.openai.com/docs/guides/reasoning/beta-limitations

I see so can we shut down the stream mode manually?

Unsupported value: 'stream' does not support true with this model. Only the default (false) value is supported.
https://platform.openai.com/docs/guides/reasoning/beta-limitations

I see so can we shut down the stream mode manually?

The same need

Stream Chat Response toggle option has been added to Settings > Interface in dev!

what version you are
my show below and i try close the most likey one but still not work 'Fluidly stream large external response chunks'

what version you are my show below and i try close the most likey one but still not work 'Fluidly stream large external response chunks'

His should be a development version, not yet release

ok but is can i think add this to workplace will be better fulfill way


Leaking kubeadmconfigtemplates, openstackmachinetemplates ...

/kind bug
What steps did you take and what happened:
A management cluster (kind) running in an SCS-2V-4 VM for 3 months (mostly idle) became unusable.
After some debugging, it was found that the kube-apiserver's memory usage had exploded to > 2GiB RSS.
This caused the machine to aggressively discard memory (kswapd0) just to hit major page faults resulting in the memory to be paged back in. System load > 50 (on a 2vCPU server), >>10k major page faults/s and >500MB/s reading from disk.
What did you expect to happen:
4GiB should be sufficient RAM for a not too busy management host.
Anything else you would like to add:
I was assuming that the CSO/CSPO are causing the kube-apiserver memory usage by storing too many objects.
I thus far found kubeadmconfigtemplates and clusterclasses to exist in excessive numbers.
Environment:

kind v0.20.0 go1.20.4 linux/amd64
Ubuntu 22.04 VM on an SCS-2V-4 flavor (2vCPU, 4GiB RAM, x86-64)
CSO/CSPO as of 93d ago (let me know how I can report this better)

13683 kubeadmconfigtemplates:
cluster2    capi-openstack-alpha-1-28                      93d
cluster4    capi-openstack-alpha-1-28                      93d
cluster4    cs-cluster4-capi-openstack-alpha-1-28-ljnkh    93d
cluster4    cs-cluster4a-capi-openstack-alpha-1-28-222ck   55d
cluster4    cs-cluster4a-capi-openstack-alpha-1-28-224lk   14d
cluster4    cs-cluster4a-capi-openstack-alpha-1-28-225pj   68d
[...]

15646 openstackmachinetemplates:
cluster2    capi-openstack-alpha-1-28                      94d
cluster2    capi-openstack-alpha-1-28-control-plane        94d
cluster4    capi-openstack-alpha-1-28                      93d
cluster4    capi-openstack-alpha-1-28-control-plane        93d
cluster4    cs-cluster4-capi-openstack-alpha-1-28-mmjrw    93d
cluster4    cs-cluster4-xlh9r                              93d
cluster4    cs-cluster4a-capi-openstack-alpha-1-28-226gt   87d
cluster4    cs-cluster4a-capi-openstack-alpha-1-28-226qq   76m
cluster4    cs-cluster4a-capi-openstack-alpha-1-28-2275d   92d
cluster4    cs-cluster4a-capi-openstack-alpha-1-28-228jx   88d
cluster4    cs-cluster4a-capi-openstack-alpha-1-28-229r2   89d
cluster4    cs-cluster4a-capi-openstack-alpha-1-28-229vc   79d
cluster4    cs-cluster4a-capi-openstack-alpha-1-28-22b2t   30d
cluster4    cs-cluster4a-capi-openstack-alpha-1-28-22bm4   73d
cluster4    cs-cluster4a-capi-openstack-alpha-1-28-22cl4   84d
[...]

kubectl delete -n cluster4 kubeadmtemplate <LIST OF 13000 names> takes more than an hour, but seems to help memory usage. Same for openstackmachinetemplate. I also did compacting and defragmenting on etcd to recover.


How to add this project to my app?

Sorry to waste u guys time with such a simple question but I can't figure out how to add this to my project. I have added compile 'com.tom_roush:pdfbox-android:<IP_ADDRESS>' to my build.grade. When I try to use some of the methods it still doesn't reckognise them?
Maybe I have to add the libraries SpongyCastle core, prov, and pkix? How can I do this?

Show your build.gradle. You probably have two of them so show both.

Take a look at the sample app if you're not sure if your build.gradle is correct. Which methods in particular are you having trouble with? You shouldn't need to add the SpongyCastle libraries, Gradle will will automatically download those for you.

Checked sample build gradle against mine, thanks.


when a map is allowed to wrap around the world, the heatmap is not rendered

In the example http://www.patrick-wied.at/static/heatmapjs/example-heatmap-leaflet.html
if you keep on moving the map to the right or left so that you get the United States again, then you don't see the heapmap.

never mind the map library I am using (leaflet) has worldCopyJump flag to copy the marks and what not to the repeated map tiles.


Package upgrades

Routine package upgrades: all dev dependencies (lint, storybook, etc), except typescript and @griffel/react.

Change files needed but otherwise good to go


Implement Translation/Multi-Language Functionality

At the request of Brightway resident linguist @tngTUDOR work is to commence to add multi-language functionality to the Brightway documentation.
As luck would have it, I recently came across this PR in the pydata-sphinx-theme repo related to a project that uses translations heavily:

As you support heavy translation I would be happy to promote it to featured afterward, please
open an issue if I forget.

https://github.com/pydata/pydata-sphinx-theme/pull/1217
https://decentralchain.io

Perhaps we could take some inspiration from their setup.

If I might be so bold, @tngTUDOR : What is the real case for this? My main concerns:

[ ] The API will be in English in any case. Is this just about translating some of the "explanations" pages (Introduction, LCA, FAQ)?
[ ] Browser translation might work better even if we set multi-lang support perfectly, without the need for any overhead on our side.

yes, I was thinking specifically about the "explanations" pages themselves. @michaelweinold can you point me to what you mean by "browser translation" ?

can you point me to what you mean by "browser translation"?

Chrome (native)
Safari (native)
Firefox (plugin)

...maybe as additional food for thought:
How would we ensure that contributions such as eg. https://github.com/brightway-lca/brightway-documentation/pull/118 are propagated to the other-than-English branches?
Would we auto-translate those (in which case, 👀 browser translation) or would we need to set up a strategy that assigns language experts (from the Brightway developers?) to these PRs?


Specify designated LB external IP for an OCP cluster

Allow a user to specify a designated IP for LB.

Completed, user can specify LB IP optionally.


feat: Add chainlink node run

ACK I was planning to hook it up to CI but I'll wait for you to merge the other PR and add a run.sh here 👍


server can not run ,deep on your step

server can not run ,deep on your step ,could you help me

@bravekingzhang Hey! were you able to resolve this? I facing the same issue

I don't know why, At last  I create a new server using php.

Got it!!!!!
had to go to this url http://localhost:3000/users and generate a new user!
App is working as expected now 🙂


binary for amd64 in linux arm64 image

Is there an existing issue for this?

[X] I have searched the existing issues

What happened?
exec /usr/bin/tetragon-operator: exec format error
Tetragon Version
quay.io/cilium/tetragon-operator:v0.9.0
Kernel Version
Linux x 5.15.0-75-generic #82-Ubuntu SMP Mon Jun 12 06:02:05 UTC 2023 aarch64 aarch64 aarch64 GNU/Linux
Kubernetes Version
1.27
Bugtool
No response
Relevant log output
No response
Anything else?
No response
Code of Conduct

[X] I agree to follow this project's Code of Conduct

Hey indeed, v0.9.0 has this issue and it was already fixed by https://github.com/cilium/tetragon/pull/880 but we did not make any further releases yet. If you want to run Tetragon on arm64, I would suggest using the images that are built on top of main or using the v0.10.0-pre tags https://quay.io/repository/cilium/tetragon-operator?tab=tags&tag=latest, with the according Tetragon image.

As this specific bug has already been fixed, I'll close this issue. Please comment again or reopen if you don't agree.


Cert copy from wrong host

The component pack install playbook runs against component_pack_master. https://github.com/HCL-TECH-SOFTWARE/connections-automation/blob/6284e97dc630da1d84ba873d066be366655e8694/playbooks/hcl/harbor/setup-component-pack-only.yml#L8
but the metrics enable role copies from k8s_master[0],
https://github.com/HCL-TECH-SOFTWARE/connections-automation/blob/6284e97dc630da1d84ba873d066be366655e8694/roles/hcl/component-pack-harbor/tasks/enable_es_metrics.yml#L231
So, when you install from a different machine than the k8s_master, then the copy fails. Replacing with component_pack_master instead of k8s_master makes the role more portable.

Thank you @stoeps13. #282 merged.


Integrate MobileSAM into the pipeline for lightweight and faster inference

Reference: https://github.com/ChaoningZhang/MobileSAM
Our project performs on par with the original SAM and keeps exactly the same pipeline as the original SAM except for a change on the image encode, therefore, it is easy to Integrate into any project.
MobileSAM is around 60 times smaller and around 50 times faster than original SAM, and it is around 7 times smaller and around 5 times faster than the concurrent FastSAM. The comparison of the whole pipeline is summarzed as follows:

Yes, this would be awesome!

Hi @qiaoyu1002, I tried to use MobileSAM by replacing the SAM_checkpoint in app.py (line 377), but I got the following errors. Do you have any idea to make it work?
Error:
size mismatch for image_encoder.neck.0.weight: copying a param with shape torch.Size([256, 320, 1, 1]) from checkpoint, the shape in current model is torch.Size([256, 1280, 1, 1]).

Hello，
Thank you for your effort.
Except for replacing the SAM_checkpoint, still need to add some code in the
official SAM code.
Could you please refer to this pull request?
https://github.com/ChaoningZhang/MobileSAM/pull/26/files
On Tue, Jul 4, 2023 at 2:53 AM K-Maehashi @.***> wrote:

Hi @qiaoyu1002 https://github.com/qiaoyu1002, I tried to use MobileSAM
by replacing the SAM_checkpoint in app.py (line 377), but I got the
following errors. Do you have any idea to make it work?
Error:
size mismatch for image_encoder.neck.0.weight: copying a param with shape torch.Size([256, 320, 1, 1]) from checkpoint, the shape in current model is torch.Size([256, 1280, 1, 1]).
—
Reply to this email directly, view it on GitHub
https://github.com/gaomingqi/Track-Anything/issues/97#issuecomment-1618947139,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AYPVVX5UTKS5PJIUGN3VWA3XOMBK7ANCNFSM6AAAAAAZWPQY4E
.
You are receiving this because you were mentioned.Message ID:
@.***>


Call to enable_result_type_inference is not safe

Calling enable_result_type_inference on OperationBuilder may trigger undefined behavior if types cannot be inferred, since mlirOperationCreate returns a nullptr in this case. OperationBuilder::build does not check for nullity, and as such an invalid Operation may be constructed, causing undefined behavior when it is accessed or dropped.
Code to reproduce:
{
    let _ = OperationBuilder::new("test", location)
        .enable_result_type_inference()
        .build();
}

Suggested fix: check for nullity before calling Operation::from_raw in OperationBuilder::build and return a Result<Operation, _> or simply panic.

Thanks for the research!

Sorry, I closed by mistake!


Tesla PW2 All entities unavailable

Is there an existing issue for this?

[X] I have searched both the existing open issues & recently closed issues and did not find a duplicate of this issue.

Version of the Tesla component
v3.21.0
Version of the Powerwall Gateway software
24.4.0
Model
2
Current Behavior
None of the sensors are available in HA.  They apear to have gone offline at ~1am (uk time 10th May).
The standard tesla intergration is still reporting the state of the entities.
Expected Behavior
I'm expecting to view the state of the entities.
Debug logs
This error originated from a custom integration.

Logger: py.warnings
Source: custom_components/tesla_custom/__init__.py:462
integration: Tesla Custom Integration (documentation, issues)
First occurred: 9 May 2024 at 01:06:26 (95 occurrences)
Last logged: 07:31:14

/usr/local/lib/python3.12/asyncio/base_events.py:783: RuntimeWarning: coroutine 'Controller.update.<locals>._get_and_process_site_summary' was never awaited def call_soon(self, callback, *args, context=None):
/usr/local/lib/python3.12/site-packages/httpcore/_async/http11.py:220: RuntimeWarning: coroutine 'Controller.update.<locals>._get_and_process_site_summary' was never awaited with map_exceptions({h11.RemoteProtocolError: RemoteProtocolError}):
/usr/local/lib/python3.12/ipaddress.py:1886: RuntimeWarning: coroutine 'Controller.update.<locals>._get_and_process_site_summary' was never awaited def __init__(self, address):
/usr/local/lib/python3.12/weakref.py:415: RuntimeWarning: coroutine 'Controller.update.<locals>._get_and_process_site_summary' was never awaited return self.data[ref(key)]
/usr/local/lib/python3.12/_weakrefset.py:88: RuntimeWarning: coroutine 'Controller.update.<locals>._get_and_process_site_summary' was never awaited self.data.add(ref(item, self._remove))

Home Assistant Core
2024-05-08 19:57:38.106 WARNING (SyncWorker_2) [homeassistant.loader] We found a custom integration tesla_custom which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2024-05-08 20:00:10.534 WARNING (Recorder) [homeassistant.components.sensor.recorder] The unit of sensor.tesla_solar_generation_daily (kWh) cannot be converted to the unit of previously compiled statistics (None). Generation of long term statistics will be suppressed unless the unit changes back to None or a compatible unit. Go to https://my.home-assistant.io/redirect/developer_statistics to fix this
2024-05-08 23:56:38.467 ERROR (MainThread) [custom_components.tesla_custom] Timeout fetching tesla_custom data
2024-05-09 02:32:26.865 ERROR (MainThread) [custom_components.tesla_custom] Unexpected error fetching tesla_custom data
File "/config/custom_components/tesla_custom/__init__.py", line 462, in _async_update_data
File "/usr/local/lib/python3.12/site-packages/teslajsonpy/controller.py", line 843, in update
File "/usr/local/lib/python3.12/site-packages/teslajsonpy/controller.py", line 686, in _get_and_process_site_data
File "/usr/local/lib/python3.12/site-packages/teslajsonpy/controller.py", line 352, in get_site_data
File "/usr/local/lib/python3.12/site-packages/teslajsonpy/controller.py", line 1375, in api
File "/usr/local/lib/python3.12/site-packages/teslajsonpy/controller.py", line 1407, in __post_with_retries_except_unavailable
File "/usr/local/lib/python3.12/site-packages/teslajsonpy/connection.py", line 130, in post
File "/usr/local/lib/python3.12/site-packages/teslajsonpy/connection.py", line 572, in refresh_access_token
2024-05-09 02:32:55.662 ERROR (MainThread) [custom_components.tesla_custom] Unexpected error fetching tesla_custom data
File "/config/custom_components/tesla_custom/__init__.py", line 462, in _async_update_data
File "/usr/local/lib/python3.12/site-packages/teslajsonpy/controller.py", line 843, in update
File "/usr/local/lib/python3.12/site-packages/teslajsonpy/controller.py", line 686, in _get_and_process_site_data
File "/usr/local/lib/python3.12/site-packages/teslajsonpy/controller.py", line 352, in get_site_data
File "/usr/local/lib/python3.12/site-packages/teslajsonpy/controller.py", line 1375, in api
File "/usr/local/lib/python3.12/site-packages/teslajsonpy/controller.py", line 1407, in __post_with_retries_except_unavailable
File "/usr/local/lib/python3.12/site-packages/teslajsonpy/connection.py", line 130, in post
File "/usr/local/lib/python3.12/site-packages/teslajsonpy/connection.py", line 572, in refresh_access_token
2024-05-10 06:29:15.010 DEBUG (MainThread) [teslajsonpy.connection] Token expiration in -2 days, 17:26:50
2024-05-10 06:29:15.010 DEBUG (MainThread) [teslajsonpy.connection] Oauth expiration detected
2024-05-10 06:29:15.010 DEBUG (MainThread) [teslajsonpy.connection] Refreshing access token with refresh_token
2024-05-10 06:29:15.286 DEBUG (MainThread) [teslajsonpy.connection] Unable to refresh sso oauth token
2024-05-10 06:29:15.286 DEBUG (MainThread) [teslajsonpy.connection] Auth returned {'error': 'invalid_request', 'error_description': "The 'client_id' parameter is missing. Include client_id in the request.", 'referenceID': '0a66c94e-8068-4881-b32a-845caa5d2a03-1715322555179. For more information, visit: https://developer.tesla.com/docs/fleet-api#help-and-support'}
2024-05-10 06:29:20.468 DEBUG (MainThread) [teslajsonpy.connection] Token expiration in -2 days, 17:26:45
2024-05-10 06:29:20.468 DEBUG (MainThread) [teslajsonpy.connection] Oauth expiration detected
2024-05-10 06:29:20.468 DEBUG (MainThread) [teslajsonpy.connection] Refreshing access token with refresh_token
2024-05-10 06:29:20.720 DEBUG (MainThread) [teslajsonpy.connection] Unable to refresh sso oauth token
2024-05-10 06:29:20.720 DEBUG (MainThread) [teslajsonpy.connection] Auth returned {'error': 'invalid_request', 'error_description': "The 'client_id' parameter is missing. Include client_id in the request.", 'referenceID': 'e43b6813-9b91-47f9-933e-2932652b989d-1715322560627. For more information, visit: https://developer.tesla.com/docs/fleet-api#help-and-support'}
2024-05-10 06:29:20.720 DEBUG (MainThread) [teslajsonpy.controller] Updating SITE_CONFIG for energysite:<PHONE_NUMBER>347213
2024-05-10 06:29:20.720 DEBUG (MainThread) [teslajsonpy.connection] Token expiration in -2 days, 17:26:45
2024-05-10 06:29:20.720 DEBUG (MainThread) [teslajsonpy.connection] Oauth expiration detected
2024-05-10 06:29:20.720 DEBUG (MainThread) [teslajsonpy.connection] Refreshing access token with refresh_token
2024-05-10 06:29:21.013 DEBUG (MainThread) [teslajsonpy.connection] Unable to refresh sso oauth token
2024-05-10 06:29:21.013 DEBUG (MainThread) [teslajsonpy.connection] Auth returned {'error': 'invalid_request', 'error_description': "The 'client_id' parameter is missing. Include client_id in the request.", 'referenceID': 'c582bd4e-4b83-4035-816a-fa82c7158dd4-1715322560904. For more information, visit: https://developer.tesla.com/docs/fleet-api#help-and-support'}
2024-05-10 06:29:22.881 DEBUG (MainThread) [teslajsonpy.connection] Token expiration in -2 days, 17:26:43
2024-05-10 06:29:22.881 DEBUG (MainThread) [teslajsonpy.connection] Oauth expiration detected
2024-05-10 06:29:22.882 DEBUG (MainThread) [teslajsonpy.connection] Refreshing access token with refresh_token
2024-05-10 06:29:23.122 DEBUG (MainThread) [teslajsonpy.connection] Unable to refresh sso oauth token
2024-05-10 06:29:23.122 DEBUG (MainThread) [teslajsonpy.connection] Auth returned {'error': 'invalid_request', 'error_description': "The 'client_id' parameter is missing. Include client_id in the request.", 'referenceID': 'ee351e94-b427-4cdb-bc14-85645212a96b-1715322563033. For more information, visit: https://developer.tesla.com/docs/fleet-api#help-and-support'}
2024-05-10 06:29:25.797 DEBUG (MainThread) [teslajsonpy.connection] Token expiration in -2 days, 17:26:40
2024-05-10 06:29:25.797 DEBUG (MainThread) [teslajsonpy.connection] Oauth expiration detected
2024-05-10 06:29:25.798 DEBUG (MainThread) [teslajsonpy.connection] Refreshing access token with refresh_token
2024-05-10 06:29:26.073 DEBUG (MainThread) [teslajsonpy.connection] Unable to refresh sso oauth token
2024-05-10 06:29:26.073 DEBUG (MainThread) [teslajsonpy.connection] Auth returned {'error': 'invalid_request', 'error_description': "The 'client_id' parameter is missing. Include client_id in the request.", 'referenceID': '77ea2160-ca9d-4113-8b2a-4191cf897b74-1715322565968. For more information, visit: https://developer.tesla.com/docs/fleet-api#help-and-support'}

Anything else?
No response

In the log there is an entry Unable to refresh sso oauth token so implies you have an authentication issue

Thank you.
I've looked and I can't see anywhere to re add my credentials, only if I add the intergration again?
maybe i'm not looking in the right place.
Any advice?

Ahh, restart HA and then the intergration prompted me to put a new auth token in.
thank you for the advice.


Missing inputGroup and outputGroup on IntermediateExchange

OS: Linux
Python version: 3.10
Project version: latest
I was using pyecospold to read intermediate exchange in ecoSpold file, but didn't see inputGroup or outputGroup as attributes of IntermediateExchange class, even though there were such child elements of intermediateExchanges in the ecoSpold file.
ElementaryExchange seems to have inputGroup and outputGroup attributes but not IntermediateExchange:
https://github.com/sami-m-g/pyecospold/blob/main/pyecospold/model_v2.py#L1221

@demianruban thanks for raising this up. This is now fixed by a0badf5 in v3.1.0 now available on PyPi as well!


test,docs: Vagrant misc updates for net-next.

This PR:

Prevents Vagrant from printing to stdout about running on net-next.
Adds a note about running ubuntu-next on vbox which vsn != 5.2.0.

After merging this PR, we can close #7255 without merging it.

This change is

test-me-please

test-docs-please

test-docs-please

test-me-please

Coverage decreased (-0.03%) to 43.136% when pulling e001c2106993994745ec0bde52476c0bf6c322c0 on pr/brb/test-vagrant-misc into 6e4bd810ae14f46628bad47e31c955d2480a9424 on master.

Test failed due to unrelated to my changes flake.

@brb in future would you mind looking for a corresponding issue and linking to it, or creating a new issue for it? If we don't document flaky issues, then there's no central place to co-ordinate it and it's difficult to track how often the same issue is being hit.
For this issue I've filed it here:
https://github.com/cilium/cilium/issues/7358

@be right back  per comments by @aanm in https://github.com/cilium/cilium/pull/7401 , removing needs-backport/1.3 label.


Why does the package have peer dependencies of both date-fns and @date-io/date-fns, and why different versions?

https://github.com/dohomi/react-hook-form-mui/blob/5b0597f80b72566f86e02605d52ad65011cb706e/package.json#L34
https://github.com/dohomi/react-hook-form-mui/blob/5b0597f80b72566f86e02605d52ad65011cb706e/package.json#L39
Is this needed for some reason?
npm seems to have trouble resolving these dependencies for me because of conflicting versions and I keep having to use --force.

@nspaeth yes this is required if you use a datepicker in your forms. if not you can simply ignore it. Instead of using --force you can always use --legacy-peer-deps The v4 of Material-UI depends on this 2 date libraries. In the future in v5 this might be handled differently


[PDI-20138] Make SpoonPluginType honor plugins base folder settings

This change allows "SpoonPluginType" to be loaded from user specified paths with the -DKETTLE_PLUGIN_BASE_FOLDERS option, which also allows this type of plugin to be loaded from ~/.kettle/plugins (user home directory plugins that cross versions).

 Note:

Frogbot also supports Contextual Analysis, Secret Detection, IaC and SAST Vulnerabilities Scanning. This features are included as part of the JFrog Advanced Security package, which isn't enabled on your system.

🐸 JFrog Frogbot

:white_check_mark: Build finished in 32m 42s
Build command:
mvn clean verify -B -e -Daudit -Djs.no.sandbox -pl ui

:ok_hand: All tests passed!
Tests run: 357, Failures: 0, Skipped: 0    Test Results

:information_source:  This is an automatic message

 Note:

Frogbot also supports Contextual Analysis, Secret Detection, IaC and SAST Vulnerabilities Scanning. This features are included as part of the JFrog Advanced Security package, which isn't enabled on your system.

🐸 JFrog Frogbot

:white_check_mark: Build finished in 29m 22s
Build command:
mvn clean verify -B -e -Daudit -Djs.no.sandbox -pl core,ui

:ok_hand: All tests passed!
Tests run: 1676, Failures: 0, Skipped: 1    Test Results

:information_source:  This is an automatic message

 Note:

Frogbot also supports Contextual Analysis, Secret Detection, IaC and SAST Vulnerabilities Scanning. This features are included as part of the JFrog Advanced Security package, which isn't enabled on your system.

🐸 JFrog Frogbot

:white_check_mark: Build finished in 17m 31s
Build command:
mvn clean verify -B -e -Daudit -Djs.no.sandbox -pl core,ui

:ok_hand: All tests passed!
Tests run: 1676, Failures: 0, Skipped: 1    Test Results

:information_source:  This is an automatic message


Custom dataset

Hello, I would like to run your model on a custom dataset.
I am not quite sure besides the 6 RGB images and can bus could affect the result of your algo.
Do you think that just the domain shift in the video could impact the result that bad?
I would simply like to run it on multi-view images (I have all transformations and imu data).
Thank you!!

Unfortunately is a private custom dataset with no annotation, so it might be quite difficult to get these annotations.
I wanted to double-check if 6-views images and can_bus data (inputs of the model) can affect the results.
Or do you think also other inputs might influence the results of the algorithm?
Thank you!!

Or this issue could be caused by images only? Because i have seen that lidar2img transformation matrix are used to construct some anchor points. But I am using the same lidar2img matrices in another algorithm and apparently those are correct.
So I really do not know which could be the problem.

Are you meeting some problems when deploying the pretrained weights? There will be many factors to affect the performance. For example, 1) how is the image data normalized and preprocessed? 2) is the can_bus-like information aligned well with our processing (eg. the same units)? 3) is the distribution shift of your custom dataset from nuScenes severe? etc.

I have tested your algo with can_bus data all equal to zeros and it works the same (I am using the simple version of the model). Moreover i am using the same preprocessing technique of yours, I didn't change anything.


require.paths is removed

The "sys" module is now called "util". It should have a similar interface.
node.js:201
throw e; // process.nextTick error, or 'error' event on first tick
^
Error: require.paths is removed. Use node_modules folders, or the NODE_PATH environment variable instead.
at Function. (module.js:376:11)
at Object. (/usr/local/lib/node_modules/less/bin/lessc:7:8)
at Module._compile (module.js:432:26)
at Object..js (module.js:450:10)
at Module.load (module.js:351:31)
at Function._load (module.js:310:12)
at Array.0 (module.js:470:10)
at EventEmitter._tickCallback (node.js:192:40)
node v0.6.0

you're kind of late to the party - did you look through the open pull requests?

Lol, no.  I just want less to work :(

for those that saw this bug first like I did, #320 is the relevant issue.

Well now that 0.6.0 is out and considered stable it raises the priority of this issue.


Fixed & Upgraded ChatBot

Hello @codervivek5 ,
I have fixed the ChatBot issue using Advance JS and API,
in future you can access info for the products directly from GPT and this is 100% working.
Video -
Screencast from 26-06-23 08:01:26 PM IST.webm
It is ready to Merge 👍🏻

@codervivek5 ,
He is just a spammer, who wanted to lose my ranking also he is not even a part of GSSoC 23

@codervivek5 ,
He is just a spammer, who wanted to lose my ranking also he is not even a part of GSSoC 23.
Ask his details before any actions.
btw same goes to you Killer2OP

You have a problem.


SciPy 2024 conference Jupyter project update

In recent years, the SciPy Conference has had a Tools Plenary that gives short (~3 minute) talk slots to a bunch of the projects in the ecosystem. Jupyter was again invited to participate this year, and the SciPy conference runs next week, with the Tools Plenary running on each of the three days of talks  (July 10th-12)
Last year, @willingc  and I (@ivanov) gave the update ( video ), and the year prior I believe it was @jasongrout  and @marthacryan (video)
Who is coming to SciPy this year and would like to give this update? I will send an email out to get this issue some extra visibility.
Please leave a comment if you'd like to help put together the slides and/or present, but if you just want to indicate that you'll be at SciPy, you can leave a :rocket: reaction. <3

I will be at SciPy and would love to help out with any announcements about the inclusion of JupyterBook / MyST in the Jupyter Project. We will also (hopefully) be giving a lightning talk about the proceedings, which is running on Curvenote / MyST.

There are some other MyST/Jupyter Book folks joining as well, and I encourage jupyter people to give them all a friendly hello as newcomers to the jupyter community!

@agoose77
@fwkoch
@stevejpurves

I'll also be at SciPy! Definitely want to let others step up but I'll be around to help if needed :)

I'll be at SciPy and am happy to help. I'm willing to present but also content not to.

@gabalafou did you end up dragging this into "Done" as it was discussed at the SSC working meeting, and did anyone at the meeting want to pitch in to make this presentation happen?
I haven't heard from anyone other than those who chimed in on this issue. I started a presentation document in the Jupyter Governance shared drive, but others will have to populate the content, as I have other responsibilities to take care of this week.

When I was working on this 2 years ago, I remember the toughest part was just getting a concise update from all of the subprojects. Maybe we can tag all of the subprojects and ask them to comment a few bullet points about the last year / what's coming up for them in this thread?

A thought: maybe the SSC reps can drive the updates from each subproject? (Not that they have to do it themselves, but they can ping people and make sure it gets done?)

I like that idea! Anyone tagged below: Can you comment a short update on recent updates for the subproject you represent in the SSC?
@gabalafou (Accessibility), @ivanov (Foundations and Standards), @JohanMabille (Kernels), @jtpio (Frontends), @marthacryan (DEI), @martinRenou (Voilà), @minrk (JupyterHub and Binder), @rpwagner (Security), @SylvainCorlay (Widgets), @vidartf (Server)

I also just sent out another email, this time to the union of councils seeking updates. Anyone who doesn't have access to the presentation yet can just add a comment for what they think should go in the updates here.

An update: we will need slides ready by end of day Thursday, and our talk slot will be on Friday. Those that expressed interest and are the conference have been invited to a private channel on the SciPy2024 conference slack.

Thanks for coordinating! I added a few slides of content from the EC directly into the deck. The team who is presenting should feel oki to move that content around as the deck starts to get built. Note that the section for technical updates is not meant to be comprehensive, simply a starting point for folks to add highlights from the subprojects.

Excellent, thanks @Ruv7!

I'm at SciPy, and while I don't have corporate approval from my employer to present this update, I can help prepare it. Met up with @ivanov and @marthacryan, happy to help!

Hi all, here is a short summary from the jupyter server subproject. Due to the time-sensitive nature here, I wasn't able to run this by the server team. I kept it high level, so it shouldn't been raise any issues from folks. No pressure to use this if you feel it's too late.

Jupyter Server is thriving!

We meet on Thursdays, at 8am Pacific on Zoom. Come join us and meet the team! Everyone and anyone is welcome to join.
This year has been focused on building the server-side architecture for real-time collaboration (as an extension)! Try it out!

We've got a few points from JupyterHub on https://github.com/jupyterhub/team-compass/issues/731
feel free to pick/choose/edit if you haven't already

one of the big things that I think should be talked about is the proposed move away from NumFOCUS and to Linux Foundation as a fiscal sponsor, along with a creation of a 501c6 industry tradegroup - which is a vote that's currently under way: https://github.com/jupyter/governance/pull/226

Thank you everyone who pitched and helped us out. Update delivered!
Here's a photos of us at SciPy!


[kwokctl] Parallel Up/Down cluster with binary runtime

for later: maybe we can think about doing the ForkExecKill of each component in parallel rather than serial (same for bring up)
Originally posted by @yibozhuang in https://github.com/kubernetes-sigs/kwok/pull/5#discussion_r939808671

/kind feature

/assign


Usage? With UnrealIRCD

Hello, would it be possible to explain the usage with the WebIRC {} Block on UnrealIRCD? And the available settings/commands to your module.
It's the only one of it's kind and i am interested in using it.

Hi,
Sorry for the incredibly late reply I only just saw this
The block for unrealircd would be
webirc {         mask IP;         type webirc;         password "password"; };
The only setting is the webirc password setting and it can be found in the global module settings in znc. The module is only really designed for bncs connecting to a single network but I'm sure it can be modified to be able to work for different networks
I'm not actively using this anymore but if you have any issues just let me know


Enable kubefed2 join support adding labels to a joining cluster

With PR https://github.com/kubernetes-sigs/federation-v2/pull/484 , fed v2 now support placement policy with clusterSelector. But when joining a cluster to the federation controller plane, we cannot set labels for the joining cluster, it is better to enable kubefed2 join support adding labels.
My proposal here is adding a new option like label to kubefed2 like kubectl label, and also enable kubefed2 join support --label option.
Setting label when joining a cluster:
./bin/kubefed2 join cluster1 --cluster-context cluster1 --host-cluster-context cluster1 --add-to-registry --v=2 --label foo=bar,foo1=bar1

Adding label to an existing federatedclusters:
kubefed2 label federatedclusters cluster1 foo=bar

Deleting label from an existing federatedclusters:
kubefed2 label federatedclusters cluster1 foo-

Currently, we can use kubectl label federatedclusters to set labels for federated clusters, so not sure if we still need kubefed2 label, but I think we need enable kubefed2 join support adding a federated cluster with some labels.
/cc @marun @kubernetes-sigs/federation-wg

I don't think this should be the responsibility of kubefed2.  Labels are a part of every single kube resource, and there is a wealth of existing tooling and commands that allow them to be edited.  The best code is the code we don't have to write.

@marun so for now, if an end user want to add labels for the federated clusters, then two steps are needed:

Join the cluster
Label the cluster via kubectl label federatedclusters

Shall we update the document a bit by clarifying this?

Maybe add a pointer to the kube docs on labeling?
Also, just because label selection is supported doesn't mean that clusters need to be labeled.  I think the base use case is distributing resources to all clusters, and an empty selector will ensure that regardless of whether clusters are labeled or not.


EXT_mesh_gpu_instancing: Use thin instances when possible

If the node we must create instances for is an InstancedMesh, thin instances can't be used because the features are mutually exclusive (and InstancedMesh subclass AbstractMesh whereas thin instances pertain to Mesh, anyway).
That's why there are several checks if (!(babylonMesh as Mesh).thinInstanceSetBuffer) ... in the code, to deal with both cases.

I performed some changes, let me know if those make sense.

Also of note: I create static buffers for thin instances for performance sake, not dynamic ones. I don't know if it's the right thing to do (?)

The "GLTF Extension EXT_mesh_gpu_instancing" test fails, but it does work on my computer...

This is what was rendered:

Yep.
Maybe it is linked with the problem that the code is not put in the webpack packages.
Trying to push a change, but I have problems with github...


scons: Use pkgconfig to get correct Protobuf dependency

Latest protobuf library depends on abseil libraries. We should rely on pkgconfig to give us correct dependency. We still keep the old check as fallback.
Change-Id: I529ea1f61e5bbc16b2520ab1badff3d8264f1c33

Hi Jason,
It seems the failed test are all related to RISCV. Do you know if there's WIP change to fix this?
Test: asm-riscv-rv64ui-ps-blt-atomic-ALL-x86_64-opt Failed
Test: asm-riscv-rv64ui-ps-bltu-atomic-ALL-x86_64-opt Failed
Test: asm-riscv-rv64ui-ps-bne-atomic-ALL-x86_64-opt Failed
Test: asm-riscv-rv64ui-ps-fence_i-atomic-ALL-x86_64-opt Failed
Test: asm-riscv-rv64ui-ps-jal-atomic-ALL-x86_64-opt Failed
Test: asm-riscv-rv64ui-ps-jalr-atomic-ALL-x86_64-opt Failed
Test: asm-riscv-rv64ui-ps-lb-atomic-ALL-x86_64-opt Failed

Thanks.

@powerjg sorry I forgot to use @ to generate the notification.

We're having problems with downloading some resources when running the CI tests. We're looking into it.

@powerjg Thanks for help. It seems I need to wait for maintainers to merge my change to develop branch for now. I am figuring out the github flow :)

We're all figuring it out :)

@powerjg Would you like to help the change again? I'm not sure why it needs another approval...

Sorry about that. Once this is committed, then it shouldn't ask again. We have it set up such that "first time contributors" need the approval to run the CI tests since the tests run on our local infrastructure.


How to test on pictures or videos

How to test on pictures or videos, follow the steps to get the final result of a txt file.Thanks!

To perform inference:
> python main.py --mode inference --inference_file proj/debug_exp/inference_file.txt --experiment_path proj/debug_exp

mode: Mode to run the main file.
inference_file: Text file specifying images to perform inference. A row is: 'full_path_of_image x1 y1 x2 y2', where (x1,y1) and (x2,y2) specify the bounding box. Refer sample_inference_list.txt.
experiment_path: Path of the experiment directory. Models stored in the the directory are used for inference.