Need a synchronization method for batches of tasks

It frequently occurs that we have loops of the kind
for (int i=0; i<large_number; ++i) {
    hpx::async(task, stuff);
}
wait_here_for_tasks_to_complete();
// access some shared structure that is being modified by the tasks

Examples of this can be seen in tests like the future overhead performance test, the network storage tests and various other places in the code.
https://github.com/STEllAR-GROUP/hpx/blob/6df436bf5b260716aad5663c943c2f7f5d76857e/tests/performance/local/future_overhead.cpp
In some ways this is related to fork_join parallelism since we have to do N lots of work and block on the result. In the future_overhead test we have a wait_each, wait_all and a thread counter implementation. The wait_all version is most well suited to the hpx way of doing things, but is expensive since we must store futures and wait on them. the thread counter implementation is intrusive since it queries the pool at start and at end to ensure no other tasks are running. Another version in my own local tree uses a sliding semaphore, but this requires the user to decorate their task with a decrement of the counter. In other tests, we have used a simepl atomic counter and this also requires modification of the user's task to update the value.
Ideally, we should be able to add tasks (to a thread pool for example) and supply a latch so that we can be signalled when the last one has completed.
const in num_tasks = 1000000;
hpx::lco::latch my_latch(num_tasks);
for  (int i=0; i<num_tasks; ++i) {
    hpx::async(my_latch, task, args);
}
my_latch.wait();

An alternative that uses the existing executor style API
const in num_tasks = 1000000;
hpx::lco::latch my_latch(num_tasks);
for  (int i=0; i<num_tasks; ++i) {
    hpx::async(an_executor.with(my_latch);, task, args);
}
my_latch.wait();

When tasks complete, the existing internal plumbing would decrement the latch.
I believe something of this kind would be a very useful mechanism to have in our code. I welcome comments on what the best API might be. It could be used on some of the parallel::algorithms to remove some wait_all uses.

This is also connected to the idea of N-ary tasks. In that case, promoting the callable object (as suggested in #3348) to be created earlier and storing things like counters in a task structure. I didn't want to write this in the issue descriptions, but just tag it here as a random thought.

Note that in the examples I pasted in above, it would actually be nice to have this functionality with hpx::apply since we don't need to return the futures and wait on them.

I'm not against this idea at all, but I think for example the future_overhead test is somewhat artificial because there we're really just interested in testing task spawning. In practice one wants to avoid spawning that many tasks and would instead use for_loop or for_each:
for_loop(par, 0, num_tasks, [](const int /*don't care about the index*/) { task(args); });

This essentially does what you want but with chunking. It removes the syntactic overhead of (e.g.) having to use a latch but adds some with the lambda (is there a cleaner way to write that if task doesn't take an index as it's first argument?). Is there an argument that for_loop is not the semantically correct thing to use?

Internally for_loop is creating tasks and then waiting on the futures. I could use the for_loop in my example (and I will for the time being), but I wanted to address the general case where you create a large number of tasks and want to wait on them without storing a vector of futures and using wait_all or something similar.
the idea of the latch is simply to simplify the user experience and provide a hook that links the scheduler back to the user's code.
A for loop becomes much harder when we wish to mix some futures and throw away tasks as follows (which is my main intention in the long run).
for (int i=0; i<large_number; ++i) {
    if (i%10==0) {
        my_futures.emplace_back(std::move(hpx::async(latch, blah)));
    }
    else {
        hpx::async(latch, task, stuff);
    }
    // also might spawn tasks without a latch in same loop as we don't need them to be counted
}
latch.wait_here_for_tasks_to_complete();
for (auto && f : my_futures) {
    // do things
}

Using a latch here for some tasks, and no latch for others, saving some futures and not saving others. This pattern keeps cropping up.

@biddisco FWIW, we do have a latch (see here: https://github.com/STEllAR-GROUP/hpx/blob/master/hpx/lcos/local/latch.hpp). Also, if you don't need the future returned by hpx::async, just use hpx::apply (i.e. fire and forget) - that saves you creating the future in the first place.

Yes. I am aware of the existence of the latch - this is what I want to use. My question is about how we could nicely fire off tasks and use a latch to keep track of them, without the user having to do anything to their actual tasks.

My question is about how we could nicely fire off tasks and use a latch to keep track of them, without the user having to do anything to their actual tasks.

I surmise that hpx::wait_all is not what you want? Would a task_block work for you?

As I wrote above. I would like to mix tasks that are 'important' and should be counted in the latch, with tasks that are not important and can just run as they want, but synchronize cheaply on the completion.
A task_block would not allow me to mix between the two.

It seems I didn't really explain the problem very well - suppose I do the following
auto f = hpx::parallel::for_loop(
         hpx::parallel::execution::par_task, 0, N,
          [&](int i) {
            hpx::async(task, stuff);
        });
}
f.get();

This only waits on the loop, it does not wait on the tasks generated inside the loop. The loop will be chunked and there will be C chunks and C futures waited on, but the N async tasks will not complete until later. What I want to do be using a construct of this kind
const in N = 1000000;
hpx::lco::latch my_latch(N);
for (int i=0; i<N; ++i) {
    hpx::async(my_latch, task, args);
}
my_latch.wait();

is wait on the N tasks so that I know when they have all completed without having to have a vector<future> of size N that I then wait on. The specific reason I want it is to implement one of the loops in DCA++, but I have had to use vectors of futures in numerous tests previously.

Is the code you posted above with the latch your ideal way of writing that or would go further if you had anything available?

Well it seems like a very simple way of doing it and could be implemented using a custom applier that is instantiated by an executor that is 'aware' of it. The other obvious ways involve modifying the task itself to signal, but that is intrusive. It is also related to the use of a sliding semaphore in other tests to limit the number of tasks, in those, we have to modify the tasks to signal, but with this approach, it would be possible to do it via the executor/scheduler and leave the tasks untouched - I believe it would be a cleaner way of handling it.

the most consistent and, IMHO, leanest way to accomplish this is via an executor. On the executor side of things, you can block until all tasks are completed, the implementation can then be completely hidden inside that executor

I agree. This new executor could even be a wrapper around any other (already existing) executor, just adding the ability to wait for all executed tasks. Also, some of our executors already block on destruction. That could be a good starting point.

I am imagining that we have an executor that creates an applier - with an additional function object that is called when the task completes - instead of (or as well as) setting a future, it can decrement a counter/latch/semaphore or suchlike. This could be used for apply or for async - (I didn't look at the code for apply recently, so I'm not sure if it uses an applier, or something else).
We woould need to add a template param to applier, and have the usual executors pass a default param to these that would be the function that sets the future. The latching_executor or whatever, would decorate this function with it's extra features, or replace the future set with other operations.

Issues #3348 and #3553 should not be closed as they are under consideration at the time of writing.


Overzealous code 64 check

Context: https://neurostars.org/t/inconsistent-errors-with-bids-validator/2419/3
File naming conventions inside stimuli do not need to be checked.

@olgn do you think this could be related to changes in https://github.com/INCF/bids-validator/pull/501?

I see. Could you look or @bGrass look into it nonetheless?  Thanks in advance.

@chrisfilo would you verify that https://1526-37161308-gh.circle-artifacts.com/0/root/web_version/index.html resolves this issue?

Yup! https://neurostars.org/t/inconsistent-errors-with-bids-validator/2419/5

splendid


feat: Draft implementation of "Move to X" button for the posts on the Kanban Board

This is a draft implementation of the low-hanging fruit solution described in #46.

Check it on testnet: https://test.near.social/#/gigs.frol14.testnet/widget/gigs-board.pages.Boards?selectedBoardId=nearsocial
TODO:

[ ] Do not display the button to those users who don't have permission to edit the labels
[ ] Improve the design of the button
[ ] Consider if there should be a button to move back (e.g. if it was moved by mistake)
[ ] Consider making a special prop flag for a board where this "Move to" is applicable, as it does not make sense to move a post from a "widget" to "integration".
[ ] Notify the involved parties about the update - it is not a blocker, but worth to think about

After syncing with Ori, we decided that this implementation will only make a marginal improvement as it is common to also edit the description of the post to provide extra details (e.g. link to a transaction). Thus, I will not pursue this direction.


kibana Unable to revive connection elasticsearch

Hi,
elasticsearch 5.3.0 has  installed Search Guard 5 and Kibana 5.3.2 has installed Search Guard 5
My certificates use the TLS generator service
es cluster node 1 elasticsearch.yml configed:
searchguard.ssl.transport.enabled: true
searchguard.ssl.transport.keystore_type: JKS
searchguard.ssl.transport.keystore_filepath: <IP_ADDRESS>-keystore.jks
searchguard.ssl.transport.truststore_filepath: truststore.jks
searchguard.ssl.transport.enforce_hostname_verification: false
searchguard.ssl.transport.keystore_password: "da4816291ba"
searchguard.ssl.transport.truststore_password: "3e2a441a72f"
searchguard.ssl.http.enabled: true
searchguard.ssl.http.keystore_filepath: <IP_ADDRESS>-keystore.jks
searchguard.ssl.http.truststore_filepath: truststore.jks
searchguard.ssl.http.keystore_password: "da4816291ba"
searchguard.ssl.http.truststore_password: "3e2a441a72f"
searchguard.ssl.http.clientauth_mode: OPTIONAL
searchguard.authcz.admin_dn:

CN=sgadmin

es cluster node 2 elasticsearch.yml configured:
searchguard.ssl.transport.enabled: true
searchguard.ssl.transport.keystore_type: JKS
searchguard.ssl.transport.keystore_filepath: <IP_ADDRESS>-keystore.jks
searchguard.ssl.transport.truststore_filepath: truststore.jks
searchguard.ssl.transport.enforce_hostname_verification: false
searchguard.ssl.transport.keystore_password: "da4816291ba"
searchguard.ssl.transport.truststore_password: "3e2a441a72f"
searchguard.ssl.http.enabled: true
searchguard.ssl.http.keystore_filepath: <IP_ADDRESS>-keystore.jks
searchguard.ssl.http.truststore_filepath: truststore.jks
searchguard.ssl.http.keystore_password: "da4816291ba"
searchguard.ssl.http.truststore_password: "3e2a441a72f"
searchguard.ssl.http.clientauth_mode: OPTIONAL
searchguard.authcz.admin_dn:

CN=sgadmin

kibana's kibana.yml configured:
elasticsearch.url: "https://<IP_ADDRESS>:9200"
elasticsearch.username: "kibanaserver"
elasticsearch.password: "kibanaserver"
elasticsearch.ssl.verificationMode: none
elasticsearch.ssl.verify: true
searchguard.basicauth.enabled: true
searchguard.cookie.secure: false
searchguard.cookie.name: 'searchguard_authentication'
searchguard.cookie.password: 'searchguard_cookie_default_password'
after start es cluster all nodes and  kibana ,I see the following warn in kibana log:
{"type":"log","@timestamp":"2017-06-02T06:51:11Z","tags":["warning","elasticsearch","admin"],"pid":33426,"message":"No living connections"}
{"type":"log","@timestamp":"2017-06-02T06:51:14Z","tags":["warning","elasticsearch","admin"],"pid":33426,"message":"Unable to revive connection: https://<IP_ADDRESS>:9200/"}
{"type":"log","@timestamp":"2017-06-02T06:51:14Z","tags":["warning","elasticsearch","admin"],"pid":33426,"message":"No living connections"}
{"type":"log","@timestamp":"2017-06-02T06:51:17Z","tags":["warning","elasticsearch","admin"],"pid":33426,"message":"Unable to revive connection: https://<IP_ADDRESS>:9200/"}
{"type":"log","@timestamp":"2017-06-02T06:51:17Z","tags":["warning","elasticsearch","admin"],"pid":33426,"message":"No living connections"}
how can i do ?

Please do not ask questions here.
For that refer to https://groups.google.com/forum/#!forum/search-guard
Github is a place only for reporting bugs or feature requests.
If you report a bug please include always the following informations:

Search Guard and Elasticsearch version
JVM version and operating system version
Number of nodes in your cluster
Description of the bug

These informations are optional but are very valueable and
typically would lead to get bugs fixed faster:

Steps to reproduce
Logfiles on DEBUG level
A failing unittest


Added Physics Python interface

Signed-off-by: ahcorde<EMAIL_ADDRESS>🎉 New feature
This PR is part of this mete ticket https://github.com/ignitionrobotics/sdformat/issues/931
Summary
Added Physics Python interface
Test it
from sdformat import Physics

physics = Physics()

Checklist

[x] Signed all commits for DCO
[x] Added tests
[ ] Added example and/or tutorial
[ ] Updated documentation (as needed)
[ ] Updated migration guide (as needed)
[x] codecheck passed (See contributing)
[x] All tests passed (See test coverage)
[x] While waiting for a review on your PR, please help review another open pull request to support the maintainers

Note to maintainers: Remember to use Squash-Merge and edit the commit message to match the pull request summary while retaining Signed-off-by messages.

pyWorld_TEST is failing. And it looks like Jenkins is not capturing test failures from python tests. There are a bunch of failures on homebrew, but the test results shows 0 failures. Is this a known issue?


Fix PHPUnit Configuration warning

Prevent the following warning being thrown by PHPUnit during testing:
  Warning - The configuration file did not pass validation!
  The following problems have been detected:

  Line 6:
  - Element 'testsuite': The attribute 'name' is required but missing.

  Test results may not be as expected.

Thanks!


addresssing comments on CreateOption

@eisber,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot

@cynthn Please review these proposed changes. Thanks.

#sign-off @eisber thank you!


An issue in the description of the WayPoint

Challenge Waypoint: Make Instances of Objects with a Constructor Function has an issue.
User Agent is: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.86 Safari/537.36.
Please describe how to reproduce this issue, and include links to screenshots if possible.
The description text on the left-side of the page has an error in it. I'll mention the text here:
Now, once the myCar instance is created it can be used like any other object and can have its properties accessed and modified the same way you would usually. For example:

var myCar.turboType = "twin";     // This line has an extra keyword "var", which causes an error.

Our myCar variable now has a property turboType with a value of "twin".

Closing as a duplicate of #4324.
Next time please Search for your issue on github before filing a bug.
Thanks and happy coding!


⚠ Jenkins Plugins Artifactory has degraded performance

In c2eee72, Jenkins Plugins Artifactory (https://get.jenkins.io/plugins/artifactory/) experienced degraded performance:

HTTP code: 200
Response time: 1179 ms

Resolved: Jenkins Plugins Artifactory performance has improved in 0a5993d after 21 minutes.


add wasm and watch the whole project burn

Adds wasm-bindgen and also doesn't work :)

https://github.com/rustwasm/wasm-bindgen/pull/2631
https://github.com/rustwasm/wasm-bindgen/issues/2407
Relevant issues and PR for possible solutions to errors

I recommend working off the dev branch if possible. It's got a lot of changes and improvements. I plan to merge it into main eventually.
Besides that, since serde support is already implemented there, I don't think you actually need to decorate every struct with #[wasm_bindgen] and can instead just let serde_wasm_bindgen do it for you.
Example:
use js_sys::Uint8Array;
use serde_wasm_bindgen::{from_value, to_value};
use tes3::esp::Plugin;
use wasm_bindgen::prelude::*;

#[wasm_bindgen]
pub fn load_objects(array: Uint8Array) -> Result<JsValue, JsValue> {
    let mut plugin = Plugin::new();
    plugin
        .load_bytes(array.to_vec().as_ref())
        .map_err(|e| JsValue::from(e.to_string()))?;

    let value = to_value(&plugin.objects)?;

    Ok(value)
}

#[wasm_bindgen]
pub fn save_objects(value: JsValue) -> Result<Uint8Array, JsValue> {
    let mut plugin = Plugin {
        objects: from_value(value)?,
    };

    let bytes = plugin
        .save_bytes()
        .map_err(|e| JsValue::from(e.to_string()))?;

    let length = u32::try_from(bytes.len()) //
        .map_err(|e| JsValue::from(e.to_string()))?;

    let array = Uint8Array::new_with_length(length);
    array.copy_from(&bytes);

    Ok(array)
}


chore(doc): Update Percona installation

Update Percona installation using Percona operator

Signed-off-by: Ranjith R<EMAIL_ADDRESS>

@ranjithwingrider - lgtm. Can we also get @nareshdesh to check this one, please.

@ranjithwingrider - lgtm. Can we also get @nareshdesh to check this one, please.


Google Mobile Ads 7.0 iOS8 incompatible

I don't know why, but it's not working together in iOS8 ;(
popup just not showing up, without any error

Ok, i found solution. If somebody need it just ask! ;)

I am getting same issue after adding  Google Mobile Ads 7.0
Please help me out how to solve this issue.
Regards
Alok

Hi,
@anclefeudor
Can you Please provide the solution for the same.
Regards
Alok

i have same issue :(
@anclefeudor , @alokfuterox please help.
Thanks advance.

I think Google used same upper view for default popup, so we need to define custom root layer for popup.
As a quick fix i just create a new method:
KLCPopup.h
- (void)showWithRoot:(UIView*)view;
KLCPopup.m
- (void)showWithRoot:(UIView*)view {
    NSDictionary* parameters = @{@"layout" : [NSValue valueWithKLCPopupLayout:KLCPopupLayoutCenter],
                                 @"duration" : @"0.0",
                                 @"root" : view};
    [self showWithParameters:parameters];
}

and replace
if(!self.superview){
        NSEnumerator *frontToBackWindows = [[[UIApplication sharedApplication] windows] reverseObjectEnumerator];

        for (UIWindow *window in frontToBackWindows) {
          if (window.windowLevel == UIWindowLevelNormal) {
            [window addSubview:self];

            break;
          }
        }
      }

with this
if(!self.superview){
          UIView* rootView = [parameters valueForKey:@"root"];
          if(rootView == nil){
              NSEnumerator *frontToBackWindows = [[[UIApplication sharedApplication] windows] reverseObjectEnumerator];

              for (UIWindow *window in frontToBackWindows) {
                  if (window.windowLevel == UIWindowLevelNormal) {
                      [window addSubview:self];
                      break;
                  }
              }
          }else{
              [rootView addSubview:self];
          }
      }

And of course you need to use this method for show popup.
Just create some empty view and use it as root for popup.
Actually it must be fixed - (void)showAtCenter:(CGPoint)center inView:(UIView*)view method, but i'm lazy, sorry...

@anclefeudor thanks for your reply, I've done it :tada:
Again, Thank you so much!

@anclefeudor
Thank a lot.
Regards
Alok

Awesome workaround. Thanks a bunch!

Thank you for fast fix!

It is still not working for me. I tried it on iOS 7.1 and it's working fine. But on iOS 8.1 and 8.4, nothing is showing up.

it's only for 7.1! i'm not tested 8.x yet

Sorry, in 8.4.1 my fix is work.

Do you know other libraries conflicting with this one? Aside from the Google Mobile Ads? I tried to run it in a separate project and it worked. But when I integrate it to my app, it fails. I can't figure out why. Thanks!

I made it work by making self.view of the viewController as rootview.

Thank you  @anclefeudor

How about this fix?

@@ -541,7 +541,7 @@ - (void)showWithParameters:(NSDictionary*)parameters {
NSEnumerator *frontToBackWindows = [[[UIApplication sharedApplication] windows] reverseObjectEnumerator];

                 for (UIWindow *window in frontToBackWindows) {

                   if (window.windowLevel == UIWindowLevelNormal) {

                   if (window.windowLevel == UIWindowLevelNormal && !window.isHidden) {
                       [window addSubview:self];

                       break;


az extension add -n ml failed with Pip failed with status code 1

Describe the bug
When using azure-cli from the official image (sha256:88e032ed2623fa2ce4e5cba8cba5eb095147a9d4929c477a4e2a5aa53d41600d, the installation of the ml extension failed with
An error occurred. Pip failed with status code 1. Use --debug for more information.

Related command
docker run -it --rm mcr.microsoft.com/azure-cli:latest /bin/bash
az extension add -n ml

Errors
Default enabled including preview versions for extension installation now. Disabled in May 2024. Use '--allow-preview true' to enable it specifically if needed. Use '--allow-preview false' to install stable version only.
An error occurred. Pip failed with status code 1. Use --debug for more information.

Issue script & Debug output
running build_ext
  building 'psutil._psutil_linux' extension
  creating build/temp.linux-x86_64-cpython-311
  creating build/temp.linux-x86_64-cpython-311/psutil
  creating build/temp.linux-x86_64-cpython-311/psutil/arch
  creating build/temp.linux-x86_64-cpython-311/psutil/arch/linux
  gcc -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_SIZEOF_PID_T=4 -DPSUTIL_VERSION=598 -DPy_LIMITED_API=0x03060000 -DPSUTIL_ETHTOOL_MISSING_TYPES=1 -DPSUTIL_LINUX=1 -I/usr/local/include/python3.11 -c psutil/_psutil_common.c -o build/temp.linux-x86_64-cpython-311/psutil/_psutil_common.o
  psutil could not be installed from sources because gcc is not installed. Try running:
    sudo apk add gcc python3-dev musl-dev linux-headers
  error: command 'gcc' failed: No such file or directory
  error: subprocess-exited-with-error

  × Building wheel for psutil (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /usr/local/bin/python /usr/local/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py build_wheel /tmp/tmpmtbwlall
  cwd: /tmp/pip-install-1b1ouzfe/psutil_dca6a20dba644533a2cf9d19e2da704d
  Building wheel for psutil (pyproject.toml): finished with status 'error'
  ERROR: Failed building wheel for psutil
Failed to build psutil
ERROR: Could not build wheels for psutil, which is required to install pyproject.toml-based projects
Exception information:
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/pip/_internal/cli/base_command.py", line 180, in exc_logging_wrapper
    status = run_func(*args)
             ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pip/_internal/cli/req_command.py", line 248, in wrapper
    return func(self, options, args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pip/_internal/commands/install.py", line 429, in run
    raise InstallationError(
pip._internal.exceptions.InstallationError: Could not build wheels for psutil, which is required to install pyproject.toml-based projects
Removed build tracker: '/tmp/pip-build-tracker-t3vv_e7w'

cli.azure.cli.core.extension.operations: Command '['/usr/local/bin/python', '-m', 'pip', 'install', '--target', '/root/.azure/cliextensions/ml', '/tmp/tmplkup8dgc/ml-2.22.0-py3-none-any.whl', '-vv', '--disable-pip-version-check', '--no-cache-dir']' returned non-zero exit status 1.
cli.azure.cli.core.extension.operations: Pip failed so deleting anything we might have installed at /root/.azure/cliextensions/ml
cli.azure.cli.core.azclierror: Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/knack/cli.py", line 233, in invoke
    cmd_result = self.invocation.execute(args)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py", line 663, in execute
    raise ex
  File "/usr/local/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py", line 726, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py", line 697, in _run_job
    result = cmd_copy(params)
             ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py", line 333, in __call__
    return self.handler(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/azure/cli/core/commands/command_operation.py", line 121, in handler
    return op(**command_args)
           ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/azure/cli/command_modules/extension/custom.py", line 16, in add_extension_cmd
    return add_extension(cli_ctx=cmd.cli_ctx, source=source, extension_name=extension_name, index_url=index_url,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/azure/cli/core/extension/operations.py", line 349, in add_extension
    extension_name = _add_whl_ext(cli_ctx=cmd_cli_ctx, source=source, ext_sha256=ext_sha256,
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/azure/cli/core/extension/operations.py", line 172, in _add_whl_ext
    raise CLIError('An error occurred. Pip failed with status code {}. '
knack.util.CLIError: An error occurred. Pip failed with status code 1. Use --debug for more information.

cli.azure.cli.core.azclierror: An error occurred. Pip failed with status code 1. Use --debug for more information.
az_command_data_logger: An error occurred. Pip failed with status code 1. Use --debug for more information.
cli.knack.cli: Event: Cli.PostExecute [<function AzCliLogging.deinit_cmd_metadata_logging at 0x7fbb895e0400>]
az_command_data_logger: exit code: 1
cli.__main__: Command ran in 26.585 seconds (init: 0.293, invoke: 26.292)
telemetry.main: Begin splitting cli events and extra events, total events: 1
telemetry.client: Accumulated 0 events. Flush the clients.
telemetry.main: Finish splitting cli events and extra events, cli events: 1
telemetry.save: Save telemetry record of length 3398 in cache
telemetry.main: Begin creating telemetry upload process.
telemetry.process: Creating upload process: "/usr/local/bin/python /usr/local/lib/python3.11/site-packages/azure/cli/telemetry/__init__.py /root/.azure"
telemetry.process: Return from creating process
telemetry.main: Finish creating telemetry upload process.

Expected behavior
ml extension installed.
Environment Summary
{
"azure-cli": "2.56.0",
"azure-cli-core": "2.56.0",
"azure-cli-telemetry": "1.1.0",
"extensions": {}
}
Additional context
using the docker image

Thank you for opening this issue, we will look into it.

@yonzhan Thanks for your quick reply. I also tested different tags with it, the problems occur also on 2.54.0, 2.55.0 and 2.56.0. It starts to disapper on 2.53.1.

This is related to this pr: https://github.com/Azure/azure-cli/pull/27567
The psutil does not provide wheel for Alpine, you need to run apk add gcc musl-dev linux-headers libffi-dev before install ml extension.

This is related to this pr: #27567 The psutil does not provide wheel for Alpine, you need to run apk add gcc musl-dev linux-headers libffi-dev before installing ml extension.

@bebound Thanks for your reply. Would this be fixed in new published images, or these steps are suggested to be done everytime we install the extension?

You need to run these steps manually.

No plan to revert it. You need to run these steps manually.

Any reason why there is no plan to solve this issue? IMO it makes the usage of azure-cli image in danger when one wants to use ml extension.

It's explained in the https://github.com/Azure/azure-cli/pull/27567 PR description.


Issue with builds and maven

the pom has the version as 1.3.5-${build.version} which makes the version in the maven repo literally that. This means AFKPGC can't be added as a maven dependency because maven doesn't read 1.3.5-${build.version} as a literal string and can't find the version in the repo

The build server read that as null anyway, no idea why it was there. Might be an obscure leftover from the original AFKPGC code. Anyway I fixed it https://github.com/Civcraft/AFK-Player-GC/commit/9418057649b4d03c09407f42833a76a0dd75c348


Improvements for coordinate decompression

Adds:

Separate secp256k1_ge_set_xquad_var which constructs a point with a given X coordinate, guaranteeing a Y coordinate that is a quadratic residue.
Implement secp256k1_ge_set_xo_var in function of the above.
Documentation to explain this.
Unit tests for all of the above.

utACK.   If you wanted to elaborate the comments a bit you could point out why the _sqr() power ladder always results in the result which is itself a quadratic residue.

Done.

ACK.


Present sheet with Clear Background

I needed to present a modal sheet with clear background. I'm using now this workaround:
extension UIViewController {
    func present<Content: View>(presentationStyle: UIModalPresentationStyle = .automatic, transitionStyle: UIModalTransitionStyle = .coverVertical, clearBackground: Bool = false, animated flag: Bool = true, completion: @escaping () -> Void = {}, @ViewBuilder builder: () -> Content) {
        let toPresent = UIHostingController(rootView: AnyView(EmptyView()))
        toPresent.modalPresentationStyle = presentationStyle
        toPresent.modalTransitionStyle = transitionStyle
        toPresent.rootView = AnyView(
            builder()
//                .environment(\.viewController, ViewControllerHolder(toPresent))
        )
        if clearBackground  {
            toPresent.view.backgroundColor = .clear
        }
        self.present(toPresent, animated: flag, completion: completion)
    }
}

and using it with:
UIApplication.shared.windows.first?.rootViewController?.present(...)
There is the possibility to implement this like for View+navigate, (View+present)?

I'm actually tackling this and a bunch of other problems in a separate project, Coordinator. Stay tuned there for more details.


Revert "Copy test data to local to save runtime (#786)"

This reverts commit 813217d6291d695e09db0aab0231ec8398c4490b.

/kind improvement

/lgtm


Default timeouts are quite low.

(0.9.5) I've been using Caddy as a reverse proxy to provide tls, logging, and gzip to back-end services. I was seeing seemingly random file upload failures with code 502 and an error message client disconnected in the log. Eventually I figured out that the requests were timing out.
I think the solution is to have either higher timeout values by default or an explicit log message when a timeout occurs.
Thanks,
JohnnyLee

@johnnylee Was there any timeout message in your -log destination?  Can you post here?

The default timeouts are set at what they are because, in my local testing with a slowloris tool, they were the values that mitigated the effects of the attacks the best. Raising them will make servers more vulnerable to resource depletion (depending on exact system configuration and specs).
I hope that people are reading the release notes before they upgrade.
Timeout events should be logged, as @tobya mentioned. I see log messages mentioning "timeout".
And like I've said before, this seems to be frustrating to a number of people, but I don't know what's worse: being frustrated because it's secure by default or being frustrated because your server is attacked with slowloris. I'm not sure that slowloris attacks are common, though, so I'm not sure what to do. Secure by default is kind of the Caddy philosophy...

@tobya - There wasn't any message in my -log file. The only message was from Caddy itself, which was the client disconnected message:
Feb 20 14:26:39 ip-172-30-1-19 caddy[9448]: 20/Feb/2017:14:26:39 +0000 [ERROR 502 /u/398994285254414337/] client disconnected

I think if it were logged, then it wouldn't have been an issue.

I don't know what's worse: being frustrated because it's secure by default or being frustrated because your server is attacked with slowloris. I'm not sure that slowloris attacks are common, though, so I'm not sure what to do.

It's a tough one. Particularly if you're on the receiving end of a problem and it's not blindingly obvious what is wrong (see #1422).
It seems a tough tradeoff. As a visitor of the internet, I have a 4G mobile connection and I work for a couple of hours from the train every day. Will this timeout prevent me from downloading large files from caddy servers when my internet connection is running at 10kiB/sec for a few minutes before returning to 100-1000kiB/sec? If so, that's quite annoying.

@johnnylee "The only message was from caddy itself", you mean, caddy was proxying to {another app}, and {another app} sees caddy disconnect. But caddy itself doesn't make it clear that the client timed out? Yeah, I hit that too over in #1422.

@pwaller I mean the the application didn't log anything at all. As far as I can tell the request never reached the handler of the application. The message was in my /var/daemon.log file (systemd).

Please ignore me. I missed that that this was happening during uploads.

@johnnylee So Caddy logged "client disconnected" but the application did not log anything?
The "client disconnected" message is emitted when an HTTP/2 connection closes: https://golang.org/src/net/http/h2_bundle.go?h=http2errClientDisconnected

@mholt That's correct. I put a log message at the top of the application's handler to see if it was being called, and it didn't show up in the log. I would guess it has to do with some buffering occurring during the proxying process.

@pwaller Your point is still valid, the timeouts are for both reads and writes, uploads and downloads.
It's difficult to distinguish between genuinely slow networks and slowloris clients.
The majority of me thinks that Caddy should not have timeouts by default, and assume that clients are benevolent, and that slow connections are just slow networks. In a sense, that serves a greater population that does not have access to high speed internet (even in urban, developed areas). Slowloris attacks are probably much less common than slow networks.
The smaller part of my thinking says that Caddy is secure by default and should keep the timeouts.
But I think I might disable them by default.

One possibility in the future of "default timeouts" would be to have a "smart timeout"

I suppose the thing about Slowloris is that it allows one attacker to starve a server with just one measily low bandwidth net connection. If that's the attack you want to defend against, you could also do so by limiting the number of long ongoing connections from one source. This needs to be done with care because sometime entire countries can be behind an IPv4 address. So one needs to think carefully about what it is they are trying to defend against and how that might harmfully impact harmless use-cases. But maybe there is a useful metric which distinguishes slowloris-style problems.
Another style of thing that could be done is to make the timeout dependent on the resource utilization. For instance, if we only have a handful of connections (let's say, 100), that's no problem, we can sustain those. But if we have 10,000, maybe that's time to say "sorry, right now we're not handling longer requests".
A problem with such an approach is that it is somewhat magical and could lead to things breaking in a way that is very hard to diagnose.

Is there a possibility of a set of presets such as php for fast cgi that sets particular values
E.g.
Timeout secure
Or
Timeout strict
Which would set values to prevent slowloris attacks, this would allow for a simple alternative to defaults

@pwaller I agree, I like simple better. I'd have to give it a lot more thought before anything else gets implemented there.
@tobya I guess, but saying timeouts 10s is basically the same thing and less obscure.

I've disabled default timeouts. in that commit. Also because there is some room for improvement in how the standard lib does timeouts (as pointed out above), let's give it some time before we look into enabling them again: at least the issues above should be fixed (Go 1.9 maybe) first.

@mholt thanks. I'm a little confused by the way about which Go1.9 maybe issues you are referring to. I couldn't find them mentioned in this issue.

@pwaller Oops, they're linked over in https://github.com/mholt/caddy/issues/1422 - I refer to https://github.com/golang/go/issues/18437 and also https://github.com/golang/go/issues/16100.


Update htmx.js

This may match the spec: https://html.spec.whatwg.org/multipage/server-sent-events.html#server-sent-events-intro
The default event type is "message".

And then a few lines later:
https://github.com/benpate/htmx/blob/40474943536ca18887470b9663de9038c29369db/src/htmx.js#L824
you can have a for loop for all values: value[1],value[2],value[3]. Etc.
I think this will match expectations with what is listened to and all of these would be pooled together in a single,  but does no do global opportunistic pooling.


Add Azure Queue Storage support

Contributing to the Azure SDK
Please see our CONTRIBUTING.md if you are not familiar with contributing to this repository or have questions.
For specific information about pull request etiquette and best practices, see this section.

Can we please shift the source path to $/sdk/wcf to better conform to the patterns used by the repository?

//fyi:  @KrzysztofCwalina, @tg-msft, @schaabs

Can we please shift the source path to $/sdk/wcf to better conform to the patterns used by the repository?

Sure, Have updated the path to sdk/wcf


Searching in compact is disconnected from search in regular view

This doesn't seem to be an issue any more.


SRTP Context can now handle multiple SSRCes

Improve the SRTP context so it can be sent multiple SSRCes.
This refactor simplifies our network code, and also enables us to
implement RTCP before we would only negotiate a context on RTP traffic
and RTCP breaks this pattern
Resolves #143

Pull Request Test Coverage Report for Build 818

144 of 188 (76.6%)  changed or added relevant lines in 4 files are covered.
11 unchanged lines in 2 files lost coverage.
Overall coverage increased (+0.2%) to 44.085%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

internal/network/port-send.go
0
5
0.0%

internal/srtp/context.go
135
153
88.24%

internal/network/port-receive.go
0
21
0.0%

Files with Coverage Reduction
New Missed Lines
%

internal/dtls/dtls.go
1
18.25%

internal/network/port-receive.go
10
14.55%

Totals

Change from base Build 810:
0.2%

Covered Lines:
2687

Relevant Lines:
6095

💛  - Coveralls


Allow tagging suggestions

Do you see Bill Poucher on a picture, but he's not properly tagged? We want you to tag that person!
Is there a way for us to make a "suggest change" button that will send suggested coordinates and person somewhere for a confirmation?
Or if it's complicated, can we build a local application / website version that will show a custom gallery version with ability to suggest changes and give a list of suggested changes as a file? We can then set up a tagging station onsite and ask people to tag their friends

I have one possible solution: create button with mailto and coordinates in title and body. This will work like this:
Submit changes

This is actually brilliant! Can you calculate coordinates on click or something?
As in:
Click "edit"
Click on a face in the photo
Select person from a list or type a new person
Submit it into email

This is actually brilliant! Can you calculate coordinates on click or something? As in: Click "edit" Click on a face in the photo Select person from a list or type a new person Submit it into email

I will investigate this possibility and write to the ticket. Most likely it can be done with small forces.

Works great for now, minor possible improvements:
[] in email put each URL/string of tags on a separate line for easier copy-paste
[] editing mode of photo information panel is quite high -- it often covers the tagging overlay and i have to scale the page to type a name. Need to design that interface part in a more compact manner, or maybe move it out of the photo area altogether, TBD
[] if editing information panel has no university tagged the width of university tag is too small for comfort
[] it's easy to forget to press Enter after typing a name, have no idea how to fix it


Flutter install failed in ubuntu

Hi, hope you can help me.
When running the flutter install task I got the following error:
##[error]TypeError: Cannot read property 'hash' of undefined
Thanks

Provide the log of what the error is, what is your input config. I can't help you without any information

The error message you provided is not very specific, but it sounds like it could be related to a problem with the Flutter installation process. Here are some steps you can try to resolve the issue:

Check that you have the latest version of Flutter installed. You can run flutter upgrade to make sure you have the latest version.

Check that your system meets the minimum requirements for running Flutter. You can find the requirements on the Flutter website.

If you recently updated Flutter, try running flutter clean to clean the build artifacts and then running flutter pub get to get the latest packages.

If the error persists, try uninstalling and reinstalling Flutter.

If none of the above steps work, try creating a new Flutter project and see if the error still occurs. If the error occurs in a new project, it could be a problem with your system or installation. If the error does not occur in a new project, it could be a problem with your project code.

If you are still having trouble after trying these steps, you may need to provide more information about your environment, Flutter version, and the specific command you are running.

Hi, is this issue fixed. It will get closed soon as stale-inactive.


Set up to connect two instances

Acceptance Criterion:

[x] One game instance is able to ping the other over a P2P connection.

Resolved by c3ea963.


"Added notebooks that are colab accessible, they should both be runnable from within colab. "

To run,

Open colab
paste in the link to the notebooks (the github links should work, I think)
Press run all, or modify the inputs and run the notebooks to play with them

Good feedback :)

We will need to add these to the notebooks README.md and explain how to run them in Colab. We can provide a direct link to open these notebooks in Colab. ColabFold uses https://colab.research.google.com/github/sokrypton/ColabFold/blob/main/AlphaFold2.ipynb for example. We should document that the T4 GPU runtime is sufficient to run both notebooks. Is it necessary to connect the runtime in Colab before starting?

From my understanding I think you have to connect to it manually if you want to use the GPU. I think it has something to do with them not trying to automatically start billing you if you are allocating like A100's or something

Will we keep the original finetuning.ipynb as well? We probably can as long as we explain their different purposes.

Yea. I think we should keep them separate. If you are running the normal one, you don't need the notebook cloning metl again (it's assumed it should be cloned already) and force-downloading all of the dependencies every time you try to run it

I made another pass through the notebooks. Some of the orginal comments are still unresolved, including some small typos corrections. I'd like to add these to the readme table before merging.
@samgelman do you want to test these in Colab before we merge?

I made another pass through the notebooks. Some of the orginal comments are still unresolved, including some small typos corrections. I'd like to add these to the readme table before merging.
@samgelman do you want to test these in Colab before we merge?

Were the typo's the links or were there other things that you saw? I can't seem to find anything else. I thought I got all of the typo corrections you put out on the previous round of comments

Were the typo's the links or were there other things that you saw? I can't seem to find anything else.

I think a lot of my small comments on the pretraining notebook were masked by GitHub as outdated but weren't incorporated. Try searching "To zero based helper functoin" as one example.

Were the typo's the links or were there other things that you saw? I can't seem to find anything else.

I think a lot of my small comments on the pretraining notebook were masked by GitHub as outdated but weren't incorporated. Try searching "To zero based helper functoin" as one example.

Should've gotten them now. There are a few comments that you can answer though in the unresolved comments above

I've reviewed both notebooks, and they are working great!
I have a few minor comments, for the prediction notebook:

For the "Variant text input" cell, specify whether the example variants are 0-based or 1-based indexing
For the "Variant file upload" cell, how should the JSON file be formatted? It might be simpler for biologists to be able to provide a list of variants in a text file, one on each line, rather than formatting a JSON file.
For the "METL predicting loop" cell, the output predictions are labeled as logits, but I'm not sure if that is the correct way to describe them. I usually see logits in the context of classifier networks, and the term refers to the outputs from the final layer of the network, before the softmax / activation. I think I've also seen the term to describe intermediate network layer outputs. For us, I think "predictions" or simply "outputs" would be more accurate.

For the finetuning notebook, there are some broken links. It looks like there may have been a find and replace, but the replacement text erroneously added "data/dms_data/avgfp" to the path.

https://github.com/gitter-lab/metl/tree/main/data/dms_data/avgfp/data/dms_data should be https://github.com/gitter-lab/metl/tree/main/data/dms_data
https://github.com/gitter-lab/metl/tree/main/data/dms_data/avgfp/pretrained_models should be https://github.com/gitter-lab/metl/tree/main/pretrained_models
https://github.com/gitter-lab/metl/tree/main/data/dms_data/avgfp/data/dms_data/avgfp should be https://github.com/gitter-lab/metl/tree/main/data/dms_data/avgfp
https://github.com/gitter-lab/metl/tree/main/data/dms_data/avgfp/code/models.py should be https://github.com/gitter-lab/metl/tree/main/code/models.py
https://github.com/gitter-lab/metl/tree/main/data/dms_data/avgfp/args/pretrain_avgfp_local.txt should be https://github.com/gitter-lab/metl/blob/main/args/pretrain_avgfp_local.txt
and so on.

I suggest fixing the broken links, and then other changes can be discussed / added later in a separate PR (especially the JSON vs. text file upload, I don't think that's a critical issue that should block the merge).
Great job!

I reran both notebooks and made a few small changes in<PHONE_NUMBER>ebe1246a2e35a92488e424d7ae571b. They both look excellent.


from multiprocessing import Process not found in autocomplete

Environment data
VS Code version:  1.19.2
Python Extension version:  0.9.1
Python Version:   3.6.2
OS and version:  Windows 10 64 educational v1709
Actual behavior
No auto complete for from multiprocessing import Process
Expected  behavior
auto complete
Steps to reproduce:

try and import from a core python module

I have reported this upstream.

VS engine appears to be fine

The problem is still there. from multiprocessing.context import Pool can only help 'Procesbut notPoolsince raiseImportError: cannot import name 'Pool'`. The first letter should be upper.
vscode: 1.26.1
ms-python.python: 2018.7.1 (23 July 2018)
python: 3.5.5
OS: Ubuntu 16.04.1
Thanks


org setting: Change add_emoji_by_only_admin setting

Fixes #7617

@rishig Changed the third setting here.
Please review when you got time, I Want to know if this is right approach.

Great, thanks! (And thanks for checking in part way :)).
My thought is to change only the frontend for these renamings, but curious for @showell's opinion.

I think it's fine to change the backend here, as the migrations should run super fast, and it's kind of yucky for the backend and frontend to have different names.
@rishig  Have you tested this?  I feel like it's merge-worthy right now, or I can wait for other changes.

Yeah, first I had change only frontend part but then, it was confusing so I change setting in backend.
One more thing which is confusing is that, every setting in Organisation setting section is starting from Prevent user ... I think we should maintain one flow, for better user experience either with Allow user.. or Prevent user..
thoughts @rishig @showell ?

Also we have to update translations for different languages accordingly.

@YJDave I noticed on this pass that we don't seem to have data migrations.  If we (essentially) rename a field to have the opposite boolean value, we need a way to set the values for all existing users to the correct value.

@showell Okay agree, so should I remove data migrations in case of only "stream setting" or both "emoji setting" and "stream setting"?

I guess that's part of the reason why I'm not sure we should change the backend; it's a lot of churn for settings names that could easily change again in a few months. And there are already lots of fields in Realm that should probably be renamed.
I agree that changing everything to "Allow .." on that page is the best final state (though again was a bit hesitant to do the work given that it could change again in a few months).
Don't have strong feelings here, though.

@YJDave Given that you're almost there, it might make sense to just write the data migration.  While it's a bit of churn, it's a good thing to learn, and it's a fast migration to run, since the relevant table is small.
I'm also fine with just changing the frontend, so you'd need to back out the migrations plus any other backend changes that would no longer apply.

@showell @rishig I tried to make changes in only frontend, but it will be more confusing. Cause every setting have relative name in frontend and backend respectively(like if Realm have create_emoji_by_admin_only field then frontend has id_realm_create_emoji_by_admin_only, and default functions directly save/restore values accordingly. If we include setting which have different names in frontend and backend then we have to add another function which will be confusing and temporary solution.
What if we first rename Realm properties as Rishi mentioned and create one migration. And then do further changes.
I am okay with both, if we want to only change frontend or both.

@rishig We've done similar migrations to this before.  The migrations should happen while the server is down, and these are fast.  There's a brief window when old clients may send requests to the server using an outdated API, but these usually just fail and force a reload.  I'm not saying glitches are impossible, but they're usually manageable, and users can usually work around them (just re-set the setting after reload).
Having said all that, I think going back to the drawing board might make sense here.

Ok, @showell @rishig So here I have only change label for second setting because it was confusing to me also and update user docs accordingly.
Let me know your suggestions!!

How about:
[ ] Prevent users less than [input] days old from creating streams
    [ ] Only admins can create streams

The second should be greyed out if the first is unchecked.
I'm not sure if this is translatable in transifex, but maybe ask in #development help or #translations to see if anyone has any ideas?

@rishig That setting will require one more variable to be stored in model, for Prevent new user for creating stream or not. Should I implement this?

Ready for review!!
What are the changes?

Change the label for Waiting period for stream creation (in days) to Prevent new users from creating stream for (in days)
Link settings realm_create_stream_by_admins_only and id_realm_waiting_period_threshold. If first one setting is checked then second setting should be disabled.

@YJDave Thanks for working on this.  I did a preliminary review, and like how the UI works, and the code is nice and clean, including the well-written tests.
@rishig I think the next steps here are to clean up #7617 with our final decisions on how this should work (and probably split out the emoji stuff to its own ticket) and then re-connect with @YJDave for the final pass.

I think this is still going to confuse people a bit. How about we make a radio button
o Only admins can create streams
o Only users at least [input] days old can create streams
o All users can create streams

and style it similar to the bottom of http://localhost:9991/#settings/display-settings?
The [input] should be 3 by default, and be greyed out unless the middle button is selected.
Is this something that is easy to put through transifex?

Let's hold on this for a bit, until the design is settled.

okay, I think the whole checkboxes thing was probably not the right design in the first place; other products have dropdowns like
Who can create streams: [dropdown with "Admins only", "Users at least 3 days old", and "Everyone"]
Possibly we should also just not have made "3" configurable.
Changing this is a big project for another day though; for now let's just do the following:

Change the wordings to:

Only admins can create streams
New user waiting period for stream creation (in days)
Only admins can add custom emoji

Make "New user waiting period .." an explicit checkbox.

Link the two streams setting together

@rishig making it explicit checkbox will require to add one more property/migrations in Realm model.
I like the idea of radio buttons though.

Radio buttons will also require another property in Realm.
This has turned into a much harder design question than expected, but thanks for sticking with it.
For now let's just do the first commit which will fix the worst of it, and then wait for folks to get back and comment on a higher level plan. I opened #7889 with some thoughts.

Ok, @rishig I change setting wording for now.


Polynetwork update: Pause EthCrossChainManager

Closes L2B-5505

did you know that we have a pausable field in the contracts section? try it out :p


Can not swap mouse button

Is there a solution for left-handed people?
thanks!

Hi @liang16
Could you add a lot more context please. What are you trying to do, and what doesn't work exactly?

Hi @matt335672

My pc is Ubuntu 22.04 x64.
Change the primary mouse button on in "setting".
It's work ok on direct desktop, but not work on xrdp.

thinks.

I've had a good look into this, and it doesn't seem to be possible at the moment.
The main problem is that the xrdp mouse driver doesn't support the libinput "Left Handed Enabled" property. If I use xinput list-props on an evdev device I get this:-

Device 'spice vdagent tablet':
	Device Enabled (124):	1
	Coordinate Transformation Matrix (126):	1.000000, 0.000000, 0.000000, 0.000000, 1.000000, 0.000000, 0.000000, 0.000000, 1.000000
	libinput Natural Scrolling Enabled (264):	0
	libinput Natural Scrolling Enabled Default (265):	0
	libinput Middle Emulation Enabled (266):	0
	libinput Middle Emulation Enabled Default (267):	0
	libinput Calibration Matrix (268):	1.000000, 0.000000, 0.000000, 0.000000, 1.000000, 0.000000, 0.000000, 0.000000, 1.000000
	libinput Calibration Matrix Default (269):	1.000000, 0.000000, 0.000000, 0.000000, 1.000000, 0.000000, 0.000000, 0.000000, 1.000000
        libinput Left Handed Enabled (270):	0
	libinput Left Handed Enabled Default (271):	0
	libinput Send Events Modes Available (247):	1, 0
	libinput Send Events Mode Enabled (248):	0, 0
	libinput Send Events Mode Enabled Default (249):	0, 0
	Device Node (250):	"/dev/input/event4"
	Device Product ID (251):	0, 0
	libinput Drag Lock Buttons (272):
	libinput Horizontal Scroll Enabled (273):	1
	libinput High Resolution Wheel Scroll Enabled (274):	1

Toggling the GNOME control above changes libinput Left Handed Enabled to 1.
Using get_props on the xrdp mouse driver I get this:-
Device 'xrdpMouse':
	Device Enabled (113):	1
	Coordinate Transformation Matrix (115):	1.000000, 0.000000, 0.000000, 0.000000, 1.000000, 0.000000, 0.000000, 0.000000, 1.000000
	Device Accel Profile (232):	0
	Device Accel Constant Deceleration (233):	1.000000
	Device Accel Adaptive Deceleration (234):	1.000000
	Device Accel Velocity Scaling (235):	10.000000

I've tried messing about with xinput set-button-map, but I can only set the first two buttons for some reason. Similarly, xmodmap has similar issues.
The same happens with the VNC backend.
The RDP protocol itself has no way to communicate the user's preference from the RDP client that I can find. So I don't think a solution at the xrdp level is right here either.
I'm going to mark this as a 'feature request'. At the moment I can't see the best way to implement this. Anyone have any suggestions?

As it happens, I've made some progress on this.
The issues I had with xmodmap yesterday aren't explained but they don't happen just after the X server is started. The old way of swapping buttons in the xmodmap manpage seems OK.
Create etc/X11/Xsession.d/90xrdp-gnome-set-handed with the following contents to set the X server handedness at login time from the GNOME setting:-
# -*- sh -*-
# Xsession.d script to set handedness for a GNOME XRDP session mouse
#
# This file is sourced by Xsession(5), not executed.

if [ -n "$XRDP_SESSION" ]; then
    case "`/usr/bin/dconf read \
           /org/gnome/desktop/peripherals/mouse/left-handed`" in
        true)   /usr/bin/xmodmap -e "pointer = 3 2 1" ;;
        *)      /usr/bin/xmodmap -e "pointer = 1 2 3"
    esac
fi

The required utilities are installed with sudo apt install dconf-cli x11-server-utils.
@liang16 - do you want to try this and report back?
In the longer term, we could go down the route of emulating the libinput-specific device parameters for the xorgxrdp driver. These are text strings defined by the xorg-xf86-input-libinput driver in include/libinput-properties.h. This might work - I haven't tried it yet. Disadvantage of this technique is it only works for the xorgxrdp backend.

@liang16 - did you manage to try this?

@matt335672 I've tested your above config and it works correctly, with the expected caveat that a full log-off/log-on is required to toggle the setting.
FWIW The x11-server-utils package was not found on my instance of debian stable.

Thanks for reporting back @BruceJL
The missing package looks like it's called x11-xserver-utils in Bullseye. My mistake!
I'll move this to a feature request on xorgxrdp.


reliable zarr v3 python package

I need a reliable package to produce test data.  I thought I did something wrong in the code, but after inspecting the data directly in python, clearly it's wrong from the start. I can't test properly if I can't generate data that follows the proposed standard correctly...

Actually, I'm going to generate the test files "on-the-fly" via the rust zarrs package, that will make tests much easier to manage, and much easier to read.


difference-of-squares: make short blurb consistent with full description

Er? I thought you did the reverse at #546. Did the long description change back again?

Oh, no, you did the same in #546. Did someone change it back, then?

Oh, I get it. #546 didn't target master. OK then.

#546 is currently the only PR in the history of this entire repo that did not target master:
https://github.com/exercism/x-common/pulls?utf8=✓&q=is%3Apr is%3Aclosed -base%3Amaster

In that case, since this is the first time this commit will appear on master, I recommend the same commit message from #546 be reused. The fact that it is present in #546 will not be visible from someone who just clones the repo and uses git log without looking at github, so it is useful to have it.

@petertseng yeah, not sure what I was doing with that original branch! 😖  Updated the commit description to match the original!

Thanks again @stkent!


Example site doesn't come with sidebar

Please modify the example site to include a sidebar, or include instructions on how to use partials.

https://github.com/hjiawei/hugo-future-imperfect/ is the current version of Future Imperfect due to lack of activity here on the original.

Issue resolved with #48


OAuthPrompt timeout applies to all auth related events and invokes.

Fixes https://github.com/microsoft/botbuilder-dotnet/issues/3627
This PR also prioritizes the timeout check before RecognizeTokenAsync, since RecognizeTokenAsync has side affects.

@carlosscastro We beleive this is good to merge. Tom has looked at it, and Eric confident. Just because OAuth and SSO are such nuanced areas, can you take a quick peek before we merge?


feat!: implement the referrers route in the v2 package

Implement the referrers route in the registry/api/v2 package. Path with filtering on ArtifactType is also implemented. Unit tests are included. The next pr will implement referrersHandler but probably will not implement filtering yet.
Part 5 of #21
Signed-off-by: wangxiaoxuan273<EMAIL_ADDRESS>

Codecov Report

Merging #32 (000665f) into main (0fce2c9) will increase coverage by 0.00%.
The diff coverage is 62.50%.

:exclamation: Current head 000665f differs from pull request most recent head 6486bcc. Consider uploading reports for the commit 6486bcc to get more accurate results

@@           Coverage Diff           @@
##             main      #32   +/-   ##
=======================================
  Coverage   57.17%   57.18%
=======================================
  Files         106      106
  Lines       10892    10900    +8
=======================================
+ Hits         6228     6233    +5
- Misses       3972     3974    +2
- Partials      692      693    +1

Impacted Files
Coverage Δ

registry/api/v2/descriptors.go
100.00% <ø> (ø)

registry/api/v2/routes.go
100.00% <ø> (ø)

registry/api/v2/urls.go
75.30% <62.50%> (-0.67%)
:arrow_down:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Recursion

Your code doesn't support recursion. I just stubbed it out like this:
case 'r' :
case 'R' :

  $reader->readUntil( ';' );
  $val = '**RECURSION**';

  break;

If you have an example of a data structure we can push through as a test, we'll get his included.

Some details on the reference types (r and R) here:
http://www.phpinternalsbook.com/classes_objects/serialization.html
Example object:
$obj = new stdClass;
$obj->p1 = 'abc';
$obj->p2 = $obj;
$obj->p3 =& $obj->p1;

echo serialize($obj);

// O:8:"stdClass":3:{s:2:"p1";s:3:"abc";s:2:"p2";r:1;s:2:"p3";R:2;}

p3 is a reference to the second value in the structure. p1 is a reference to the first value (which is the whole thing). I think r is used for object references and R for explicit =& references.
I think this parser should not just declare "recursion" and walk away. So long as it can keep track of each value it encounters, then it should be able to link the references properly, so the final parsed structure will have its own proper recursion in it.

Indexing a reference to each value is easy enough. The difficulty comes in the order in which they are parsed. If a reference points forward to an element that has not yet been parsed, then we need to keep it until later to link it up. So creating back-references can be done immediately, but forward references would be kept to reference when possible (or left until the end). A two-pass parsing could also work, but is unnecessary IMO.

I'm working on this in the background. The approach I'm taking is:

As each data item is parsed, add it to to a numbered list. The path to each item (a list of object property or array key names that point at the value) is kept for each data item. It is key that the list is built in the correct order, the same order as they are encountered in the serialized string.
Encountering a reference (R or r) will result in the reference number being stored as an intermediate (temporary) value and the path to the reference being stored in a list like the values. The list does not need to be ordered in that case - just a stack.
Once the full data structure has been parsed and built, the references can be put in. Given the two lists built up, we know where the references are and what they point to numerically. The reference number can be looked up in the first ordered list to get the path, which is then navigated to and a reference can replace the numeric value. The reference will be either a full reference (=& $scalar) or a reference-link pointer to an object (= $object).

Notes:

The reference linking can only be done at the end because of forward-references. Value number 2 may reference value number 30 that has not been parsed at that point.
The final structure is built recursively, so it is not put together as the final full structure until the very last step. So setting references early, even if the destinations are known, will end up pointing to intermediate data fragments that will be copied in the final construction. I would like to avoid just making the whole process a pass-by-reference as a workaround though as it does not get around the first note.

Been playing with the recursion over the weekend, and it seems that the way PHP serializes it is rather bizarre. Take this as an example:
$arr = [
    'a' => 'one',
];

$arr['b'] =& $arr['a'];

var_dump($arr);
echo serialize($arr);

/*
array(2) {
  ["a"]=>
  &string(3) "one"
  ["b"]=>
  &string(3) "one"
}
a:2:{s:1:"a";s:3:"one";s:1:"b";R:2;}
*/

Here element b references element a. The var_dump shows the value of element b since that is what it contains (a and b share the same source data). The serialize does something different - it provides the value for a - the first time the shared data is encountered. The second time it is encountered, it is shown as a hard reference to data item number 2 (the whole array is item number 1, and a is item number 2). This is exactly what I would expect, and that's easy enough to parse.

If a is itself an array, then it works just the same:
$arr = [
    'a' => ['x' => 'ten', 'y' => 'eleven'],
];

$arr['b'] =& $arr['a'];

/*
array(2) {
  ["a"]=>
  &array(2) {
    ["x"]=>
    string(3) "ten"
    ["y"]=>
    string(6) "eleven"
  }
  ["b"]=>
  &array(2) {
    ["x"]=>
    string(3) "ten"
    ["y"]=>
    string(6) "eleven"
  }
}
a:2:{s:1:"a";a:2:{s:1:"x";s:3:"ten";s:1:"y";s:6:"eleven";}s:1:"b";R:2;}
*/

Note that the "ten" and "eleven" elements are shown in the var_dump() for convenience, but only appear once in the serialization. Again, simple to parse.

Now this is where is starts to get crazy with the recursion. If a references the root array rather than the a element, this is what happens:
$arr = [
    'a' => ['x' => 'ten', 'y' => 'eleven'],
];

$arr['b'] =& $arr;

/*
array(2) {
  ["a"]=>
  array(2) {
    ["x"]=>
    string(3) "ten"
    ["y"]=>
    string(6) "eleven"
  }
  ["b"]=>
  &array(2) {
    ["a"]=>
    array(2) {
      ["x"]=>
      string(3) "ten"
      ["y"]=>
      string(6) "eleven"
    }
    ["b"]=>
    *RECURSION*
  }
}
a:2:{s:1:"a";a:2:{s:1:"x";s:3:"ten";s:1:"y";s:6:"eleven";}s:1:"b";a:2:{s:1:"a";a:2:{s:1:"x";s:3:"ten";s:1:"y";s:6:"eleven";}s:1:"b";R:5;}}
*/

Again, the var_dump() shows the value that is referenced, and recognises where recursion occurs and labels it appropriately. That's good and consistent with the previous examples.
But now look at the serialized string. Suddenly the source (root) array is being replicated - it is NOT a reference any more. The b in that duplication is a reference though, but to the copy of the root array (data item number 5 and not data item number 2.
If I add an extra element to the a array then I see it appear twice, so internally the data is a reference:
$arr['a']['z'] = 'twelve';

/*
array(2) {
  ["a"]=>
  array(3) {
    ["x"]=>
    string(3) "ten"
    ["y"]=>
    string(6) "eleven"
    ["z"]=>
    string(6) "twelve"
  }
  ["b"]=>
  &array(2) {
    ["a"]=>
    array(3) {
      ["x"]=>
      string(3) "ten"
      ["y"]=>
      string(6) "eleven"
      ["z"]=>
      string(6) "twelve"
    }
    ["b"]=>
    *RECURSION*
  }
}
a:2:{s:1:"a";a:3:{s:1:"x";s:3:"ten";s:1:"y";s:6:"eleven";s:1:"z";s:6:"twelve";}s:1:"b";a:2:{s:1:"a";a:3:{s:1:"x";s:3:"ten";s:1:"y";s:6:"eleven";s:1:"z";s:6:"twelve";}s:1:"b";R:6;}}
*/

It just looks like the serialization is wrong.

So is this serialization wrong? Is it just that I do not know how to parse it? In theory, I should be able to unserialize a serialized array and get back what I started with. So just before I add twelve, lets take it through that cycle:
$arr = [
    'a' => ['x' => 'ten', 'y' => 'eleven'],
];

$arr['b'] =& $arr;

$arr = unserialize(serialize($arr));

$arr['a']['z'] = 'twelve';

/*
array(2) {
  ["a"]=>
  array(3) {
    ["x"]=>
    string(3) "ten"
    ["y"]=>
    string(6) "eleven"
    ["z"]=>
    string(6) "twelve"
  }
  ["b"]=>
  &array(2) {
    ["a"]=>
    array(2) {
      ["x"]=>
      string(3) "ten"
      ["y"]=>
      string(6) "eleven"
    }
    ["b"]=>
    *RECURSION*
  }
}
a:2:{s:1:"a";a:3:{s:1:"x";s:3:"ten";s:1:"y";s:6:"eleven";s:1:"z";s:6:"twelve";}s:1:"b";a:2:{s:1:"a";a:2:{s:1:"x";s:3:"ten";s:1:"y";s:6:"eleven";}s:1:"b";R:6;}}
*/

Oh, whoops, where has the second twelve go to? It looks to me like a PHP bug. The serialize is not handling the reference correctly, and so the original array CANNOT be reconstructed from the serialized array. So we have not got a hope in hell's chance of correctly parsing it, since PHP itself can't parse it.
Dhoh. Grrr.

It seems to be a problem only when the reference points to the root array. This behaves entirely as expected:
$arr = [
    'a' => ['x' => 'ten', 'y' => 'eleven'],
];

$arr['a']['b'] =& $arr['a'];

/*
array(1) {
  ["a"]=>
  &array(3) {
    ["x"]=>
    string(3) "ten"
    ["y"]=>
    string(6) "eleven"
    ["b"]=>
    *RECURSION*
  }
}
a:1:{s:1:"a";a:3:{s:1:"x";s:3:"ten";s:1:"y";s:6:"eleven";s:1:"b";R:2;}}
*/

The PHP manual says this:

You can even serialize() arrays that contain references to itself. Circular references inside the array/object you are serializing will also be stored. Any other reference will be lost.

I suspect that referencing the root of the array, it thinks it is an external variable and does not realise it is referencing itself. So, it treats it as an external variable and destroys the reference, turning it into a duplication instead.

I've not tried this with objects (something for another day) but have found an older reference to this bug from 2004, though I'm not sure if it ever got officially reported.


Date Time Picker Config

My users would like to be able to type or paste a date in MM/DD/YYYY format into the form then tab to the next line. When they try this now the calendar automatically opens as soon as the field is tabbed to and instantly typing does not clear the date. If the field is cleared and a date entered manually as MM/DD/YYYY the picker sets the date incorrectly. Example manually entered 8/20/2015 and the picker changed it to January 8, 0020 8:15 AM.
I would like to see an option to configure the date time picker in the following ways:

be able to set-up how the date is displayed
allow entry without using the calendar control.

hey @gardneg the way the date is currently displayed depends on the user's locale. So someone from another country get's the format appropriate for that country.
I'm adding formats to allow entering dates as you proposed. It doesn't seem to let me allow time though.
96fd026adbbf2b794a47bd912d98ae6dd72f18a6


Update meeting-policies-participants-and-guests

Added section explaining how to enable these meeting policies, including PowerShell and time to propagate
#Signoff

@JohanFreelancer9 Copy editing is needed for this PR. Thanks.

@JohanFreelancer9 & @scanum - thank you. I've completed commits to changes.


Project Creation Failed - [ErrNo 13] Permission denied u'c:\oauth2service.json'

The issue tracker is for reporting product deficiencies. "How do I?" questions should be posted to the discussion forum at https://groups.google.com/group/google-apps-manager. When in doubt, start at the discussion forum and return here only when instructed to do so.
Please confirm the following:

I have upgraded to the latest GAM release from https://git.io/gamreleases and I still have this issue.
I am typing the command as described in the GAM Wiki at https://github.com/jay0lee/gam/wiki

Full steps to reproduce the issue:

ran MSI 64 bit installer for latest release
entered admin email address
last message in cmd window was "creating service account"

Expected outcome (what are you trying to do?):
setup GAM to add classroom courses in bluk from a CSV file
Actual outcome (what errors or bad behavior do you see instead?):
Project Creation Failed - [ErrNo 13] Permission denied u'c:\oauth2service.json'

Anyone knows how to solve this error?
I had this same problem today.
The error message was
ERROR: {'name': 'operations/cp.8844208477150365015', 'metadata': {'@type': 'type.googleapis.com/google.cloudresourcemanager.v1.ProjectCreationStatus', 'ready': True}, 'done': True, 'error': {'code': 7, 'message': "Permission 'resourcemanager.projects.create' denied on parent resource 'organizations/5555555555555'.", 'details': [{'@type': 'type.googleapis.com/google.rpc.ResourceInfo', 'resourceType': 'project', 'resourceName': 'projects/gam-project-s60-4r4-wdj'}]}}

This indicates the user you authorized as does not have permissions to create GCP projects in your organization. Do you know who does?

Having this issue today.
ERROR: [Errno 13] Permission denied: 'C:\Program Files\GYB\oauth2service.json'
It's a personal Gmail account, so I should have all rights(?). I did navigate into one of the GCP projects that was created and and clicked "Enable" on the Secret Manager API and got the message "Billing required". Do I need a paid GCP account? I searched the GYB docs and didn't see a mention of that. Thanks for your help.

You installed GYB to a folder where your user does not have write
permissions. Recommend moving GYB to c:\GYB and making sure the windows
user you are running as has rights to write to the folder.
Jay Lee
On Thu, Nov 5, 2020 at 2:29 PM Monty<EMAIL_ADDRESS>wrote:

Having this issue today.
ERROR: [Errno 13] Permission denied: 'C:\Program
Files\GYB\oauth2service.json'
It's a personal Gmail account, so I should have all rights(?). I did
navigate into one of the GCP projects that was created and and clicked
"Enable" on the Secret Manager API and got the message "Billing required".
Do I need a paid GCP account? I searched the GYB docs and didn't see a
mention of that. Thanks for your help.
[image: image]
https://user-images.githubusercontent.com/32070390/98286639-4b994200-1f72-11eb-9733-070bc7b39c23.png
[image: image]
https://user-images.githubusercontent.com/32070390/98287024-c82c2080-1f72-11eb-8c32-56a133430372.png
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/jay0lee/GAM/issues/782#issuecomment-722592048, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ABDIZMEJN3QWPG24VIWJHWTSOL4IZANCNFSM4FQ7YQFA
.


office 365  for everyone

for linux office 365 for everyone

@Scorpionkingeg1 What is your feedback and which document you are referring to. Thanks


Requesting to Host it on Streamlit

Use streamlit and host it on the community cloud, too hassle to install and experience it :) .

Yes, it was already hosted on Streamlit. Please follow the demo(hyperlink) link available in the Readme.md to experience it. Let me know if there any suggestions.
Thanks!

Pu it in about me for better visibility 🤖

@sharanreddy799 Done, thanks! Yes it makes more sense to show in the 'about' section. Please experience and feel free to suggest any improvements.


GUI Upgrade

What does your feature do?
Allow the size of the gui quest journal to be changed. from values between 9-54
Allow the empty space to be defined so instead of it being blank I could do BLACK_STAINED_GLASS
Allow exiting the quest journal with ESC, E, etc instead of requiring hitting the exit button
How would people use it?
This would allow people to customize the quest journal to their server more. For me it is an amazing addition but doesn't fit the rest of my menu's so it feels out of place.

This issue tracker is for the Quests plugin only. Please direct any and all QuestsGUI bugs/suggestions to the discussion thread at:
https://www.spigotmc.org/threads/questsgui-paid.399346/


Collision events are not reliably fired

I've forked the Jenga game and I'm trying to detect collisions between the table and the blocks, I've forked the game, and then added the following:
table.addEventListener('collision', function( other_object ) {
    console.log('table collision', other_object);
});

Dragging blocks off the tower, and letting them fall onto the table, I expect to see a collision event fired, but I usually don't. I notice that the blocks towards the bottom of the stack behave differently than the ones towards the top, but still not really what I expected: drop a block onto the table, get a collision event.
https://timdix.github.io/Physijs/examples/jenga.html
I'm very new to both Three.js and Physijs, any suggestions are greatly welcome.

http://screencast.com/t/fJzfSETVbQI

It appears that it might be related to the issues this guy was having:
https://japhr.blogspot.com/2012/08/more-physijs-collisions.html
"But it seems that subsequent collision events are not fired if any object is mid-collision with any object."
Maybe because of the stacking of the blocks, the events aren't getting fired? Any suggestions at all would be greatly appreciated to work around this issue, even hacks ;)

Hi, I was also not able to get collisions working in my project, but I saw that they worked on this page:  http://chandlerprall.github.io/Physijs/examples/collisions.html so I figured that the problem must be on my side.  I copied that example site exactly to my local machine and pointed my local web server at it as the root.  What I found was that it would not work if I accessed it by http://localhost/collisions.html (gave no errors in Chrome debugger either) , but it worked perfectly if I used http://<IP_ADDRESS>/collisions.html (or the hostname on my local domain).  Pretty strange, but hopefully this work-around solves the problem for you until the root cause can be identified and fixed.

Ok, so after a bit more investigation, it seems like changing the hostname is just a way of forcing a cache miss of some sort.  The real problem seems to be with a newer version of three.js or physi.js.  I'm looking at this hard and I hope to figure it out soon.

Ok, so after finding the "disable caching" checkbox in the Chrome developer window settings..... I figured out the issue is with ammo.js.  The latest version of ammo.js silently breaks collision detection.  Reverting to the version of ammo.js included with the example http://chandlerprall.github.io/Physijs/examples/collisions.html solves the issue.  I am continuing to dig into this because I think the newer version of ammo.js is faster and I'd like to use it...


I will fork this project to update it with more resources!

This is an awesome project! I would love to contribute with my own resources and information. Is there anything that you are missing in the list? (for example, electronic notebooks)

Hi @AleixMT. We are happy for any contributions. :-) Since also this area is evolving and we have only a limited view on things.
Electronic lab notebooks is a particular blind spot currently, so it would be very nice if you add some you would recommend, or that you know people are using in your field.
Just create a merge request over a fork, or write them in this issue then I will add them.  Cheers!


Wrong counting of Activation Limit

Hi, I currently have the problem that my Activation Limit is not counted correctly.
I'm running Porcupine on a Linux Desktop Computer in a Docker container and after a few restarts of the container/pc I'm running into the PorcupineActivationLimitError:
|   File "/Jaco-Satellite/wake-word/action-ww.py", line 42, in load_keyword_listener
|     pcpn = pvporcupine.create(
|   File "/usr/local/lib/python3.8/dist-packages/pvporcupine/_factory.py", line 69, in create
|     return Porcupine(
|   File "/usr/local/lib/python3.8/dist-packages/pvporcupine/_porcupine.py", line 198, in __init__
|     raise self._PICOVOICE_STATUS_TO_EXCEPTION[status](
| pvporcupine._porcupine.PorcupineActivationLimitError: Initialization failed:
|   [0] Picovoice Error (code `00000136`)
|   [1] Picovoice Error (code `00000136`)
|   [2] Picovoice Error (code `00000136`)

I think that the issue could be that every time I restart the docker container Porcupine thinks it's running on a new device.
And then after a few times, I can't use Porcupine anymore.
Is there any option how I can solve this?

Duplicate of https://github.com/Picovoice/picovoice/issues/552.


[Thor] Namespaced the vendored version of Thor and update to the v0.19.1...

... release
Closes https://trello.com/c/MJdUqrh3/59-namespace-vendored-thor.

Great work!
I'm just wondering if those cosmetic changes are really needed

@simi I made no changes to source -- it's grabbing directly from the Thor release and just namespacing -- see the rake task I added

Ahh, I missed that. Sorry.

Can you lowercase the directory name in vendor? It's traditional in ruby-land.

Can you lowercase the directory name in vendor? It's traditional in ruby-land.

done

Thanks!


fix(redshift): Fix Redshift User Secret Multi-User Rotation

Fixes Redshift User Secret Multi-User Rotation for new Users by including masterarn in the Secret's Serialized JSON Object Text.
Note: This doesn't affect existing users (nor fixes roation for them) since the secret string template is only used when the secret is first created. For those existing secrets, the secret text will need to be updated to include masterarn using the GetSecretValue and UpdateSecret SecretManager APIs.
Closes #28852

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

Created by mistake


Changing error to println to prevent misinterpretation

To prevent this message being interpreted as cause for failure in and of itself, I'm changing this to a println.
Tested here.
Note: I'm also hoping this prevents the pipeline job from interpreting an unstable job as a cause for overall failure. Since this doesn't break anything, I advocate putting this change in anyway and running a big pipeline to test it in anger.

run tests

@adamfarley Looks like some failures for jdk20u on Windows and the pipeline-build-check

@adamfarley Looks like some failures for jdk20u on Windows and the pipeline-build-check

Looks like the failure on Windows wasn't anywhere near my code, so I'm going to merge this check anyway. My code executes prior to the build, and the failure appears to occur after the build stopped outputting for 40-ish minutes and @steelhead31 aborted it.
Also, I believe the pipeline-build-check is the superset of the test builds, including Windows jdk20u. FYI.


The digsite quest

on the step where you need to grab a specimen jar and rock pick, the guide jumps over the part where you need to grab it and only wants you to grab a rock pick, only that grabbing a rock pick does not update the quest until the player himself looks up that he need to grab a specimen jar from the south cupboard whereupon the quest will update.
TL:dr the digsite guide jumps over the step of grabbing the specimen jar.

Should be fixed with https://github.com/Zoinkwiz/quest-helper/commit/af395fbc8fd9a74b6aa989de688e0afe194279b4, thanks.


Tensorflow js models?

Hi Stefan, et al.
You've done great work here, it's really inspiring! I wanted to ask if there wsa any prospect of releasing the latest models in a tensorflowjs format, so they could be run in a web browser? I've seen this repo: https://github.com/kahst/BirdNET-Electron but it was last updated about 4 yewars ago.

We don't have a workflow to convert models right now, but you can try for yourself. There is a guide here.

Thamks for getting back to me @Josef-Haupt ,
I did try the conversion, this is the command I used:
tensorflowjs_converter --input_format=tf_saved_model --output_node_names='CLASS_DENSE_LAYER' --saved_model_tags=serve --signature_name='basic' ./checkpoints/V2.4/BirdNET_GLOBAL_6K_V2.4_Model/ ./js_model
On execution, I get the expected whinging about gradients, but converstion fails with:
ValueError: Unsupported Ops in the model after optimization _FusedBatchNormEx
I wondered if you had implemented custom operations in the model? That would be one reason I would get this error.

It's been a while and I just redoscovered this issue: We do have a TFJS version of BirdNET ready to use, including an example on how to run it. You can find the model and the code here: https://github.com/kahst/BirdNET-Analyzer/tree/main/checkpoints/V2.4/BirdNET_GLOBAL_6K_V2.4_Model_TFJS
I would love to develop something like birdnet.js or BirdNET as npm package, yet, we do lack the skills to do so. If you're interested in attempting that, let us know :)

@kahst Sorry - I didn't see your reply until this morning. Thanks for sharing, that's awesome!
I've had a little play around, it's looking really great :smile:
One suggestion: if you make the following small changes to call method of the MelSpecLayerSimple class in your example, it will support batch predictions, which gives a decent performance boost (up to c. 40% faster inference):
class MelSpecLayerSimple extends tf.layers.Layer {
    constructor(config) {
        super(config);

        // Initialize parameters
        this.sampleRate = config.sampleRate;
        this.specShape = config.specShape;
        this.frameStep = config.frameStep;
        this.frameLength = config.frameLength;
        this.fmin = config.fmin;
        this.fmax = config.fmax;
        this.melFilterbank = tf.tensor2d(config.melFilterbank);
    }

    build(inputShape) {
        // Initialize trainable weights, for example:
        this.magScale = this.addWeight(
            'magnitude_scaling',
            [],
            'float32',
            tf.initializers.constant({ value: 1.23 })
        );

        super.build(inputShape);
    }

    // Compute the output shape of the layer
    computeOutputShape(inputShape) {
        return [inputShape[0], this.specShape[0], this.specShape[1], 1];
    }

    // Define the layer's forward pass
    call(inputs) {
        return tf.tidy(() => {
            // inputs is a tensor representing the input data
            inputs = inputs[0];
            const inputList = tf.split(inputs, inputs.shape[0])
            const specBatch = inputList.map(input =>{
                input = input.squeeze();
                // Normalize values between -1 and 1
                input = tf.sub(input, tf.min(input, -1, true));
                input = tf.div(input, tf.max(input, -1, true).add(0.000001));
                input = tf.sub(input, 0.5);
                input = tf.mul(input, 2.0);

                // Perform STFT
                let spec = tf.signal.stft(
                    input,
                    this.frameLength,
                    this.frameStep,
                    this.frameLength,
                    tf.signal.hannWindow,
                );

                // Cast from complex to float
                spec = tf.cast(spec, 'float32');

                // Apply mel filter bank
                spec = tf.matMul(spec, this.melFilterbank);

                // Convert to power spectrogram
                spec = spec.pow(2.0);

                // Apply nonlinearity
                spec = spec.pow(tf.div(1.0, tf.add(1.0, tf.exp(this.magScale.read()))));

                // Flip the spectrogram
                spec = tf.reverse(spec, -1);

                // Swap axes to fit input shape
                spec = tf.transpose(spec)

                // Adding the channel dimension
                spec = spec.expandDims(-1);

                // Adding batch dimension
                //spec = spec.expandDims(0);

                return spec;
            })
            inputList.map(x => x.dispose())
            return tf.stack(specBatch)
        });
    }

    // Optionally, include the `className` method to provide a machine-readable name for the layer
    static get className() {
        return 'MelSpecLayerSimple';
    }
}

 // Register the custom layer with TensorFlow.js
 tf.serialization.registerClass(MelSpecLayerSimple);

I can create a PR if you like. But I ran out of time today! Meanwhile, I have an Electron App which now has your BirdNet model as a backend option, taking advantage of the metadata model's 'species location' filtering. Happy days!

Awesome! Yes, please create a PR. Also, let us know more about your Electron app, we're always looking for cool projects to feature in our showcase. Thanks!

@kahst @Josef-Haupt  - hey guys, sorry to bother you again. I have a question about your TFJS conversion, motivated by the finding that a comparison shows the results from the TFJS model are similar, but far from identical to those given by BirdNET-Analyser-GUI.exe. A link to the TFJS model output from the example soundscape.wav file from this repo is here:
TFJS_detections.csv
And for convenience, the python version output:
BirdNET-Analyser-GUI.csv
For the avoidance of doubt, I have used these settings in the TFJS analysis:

no location filters
confidence 0.1
no sigmoid
(the TFJS model  has a trainable sigmoid activation head so already outputs probabilities - and this architectural difference will explain some of the difference - it may even be the answer?)

And in the GUI

sigmoid 1 (default)
all species (i.e. no filter)
confidence 0.1 (default)

In both cases batchsize and threads settings make zero difference, as you would expect.
I know there could be myriad reasons for this - there are many possibilities at my end. However, before I spend too much time comparing the outputs of the individual libraries involved, I wanted to see if  you'd used any aggressive quantization in the TFJS conversion? Like UINT8 in some activation layers for example? Using Netron, I noticed some of the layers have "QUANT"  and others 'NO_QUANT' appended to their names, so I thought it would be a good thing to check.
It's worth saying, I don't see this as a  blocker - both models make good predictions, but if am to release an App which claims to offer the BirdNET 2.4 model for predictions, I would like a handle on why the results are this different.
Thanks again for your input, I really appreciate it! The app I've been developing was initially inspired by BirdNET-Electron. You may find some of the design elements familiar Stefan, although it's come a long way since. You can read about, even download it, here: https://chirpity.mattkirkland.co.uk
(When I release a version with the new BirdNET model, the app will auto-update)

Hmmm, there shouldn't be a significant difference between models. We do test our models after conversion to make sure the scores align with the original Keras version - even after quantization. The TFJS model is not quantized and should yield scores that are very similar to the TFLite 32bit model. There might be something off with the GUI though. We'll investigate and keep you posted.

Hi @kahst , thanks again for getting back to me. I just ran the same analysis of the soundscape.wav file through the CLI and the results are identical to the GUI results posted above.
I think next steps would be for me to build/run a minimal implementation for  the TFJS model such that it can accept the soundscape.wav, chunk it up into 3 second clips and output predictions. If the results are still different I'll share that code back with you, otherwise I'll know it's an issue at my end.

@Mattk70 I bet difference comes from differences in sigmoid sensitivity implementation you mentioned

@kahst @tphakala OK. I have done a validation of the TFJS model using the flask app and sample.wav
Analyzer.py output for ths file:
Start (s),End (s),Scientific name,Common name,Confidence
0,3.0,Acanthis cabaret,Lesser Redpoll,0.9444
0,3.0,Acanthis flammea,Common Redpoll,0.7997
0,3.0,Acanthis hornemanni,Hoary Redpoll,0.0590
TFJS via the flask app in this repo* produces the following:
Results:
Acanthis cabaret_Lesser Redpoll: 0.9447994828224182
Acanthis flammea_Common Redpoll: 0.7999176383018494
Acanthis hornemanni_Hoary Redpoll: 0.05872755125164986
Whilst these are different they are minimally so, which is what I'd have expected.
Sadly, my implementation produces:
Start (s) | End (s) | Scientific name | Common name | Confidence
0 | 3 | Acanthis cabaret | Lesser Redpoll | 0.96
0 | 3 | Acanthis flammea | Common Redpoll | 0.84
0 | 3 | Acanthis hornemanni | Hoary Redpoll | 0.06
So, it's an issue with my code.
The Flask app has a couple of issues I needed to correct before I could get it to work.

If you create a new AudioContext with no parameters, its default behaviour is to resample audio to 44100Hz - consequently, the resulting tensor is too short. I added {sampleRate: 48000}
If you try to create the audioContext onload, it breaks default browser policies for CORS requests around auto playing media, and errors out. I added a button to generate the required user interaction, which is disabled until the page has loaded

I'll do a PR with these changes. And then start looking into the dang problem with my code. At least, if there's an upside, I'd never have spotted this bug if I hadn't integrated BirdNET ;-)

Nailed it! My native model uses 24000Hz sample rate, and there was one place in the code I'd missed, which was switching the audioContext in the audio pipeline to this with BirdNET, before then upsampling the audioBuffer back to 48k. Now that's fixed, the correlation in the soundscape.wav between the predictions of the TFJS and python models is perfect to 3 decimal places.
Fixed comparison.csv