Implement Swedish Bankid in flutter

Hi,
Anyone knows how to implement bankid login in flutter please help

Please use stackoverflow for such questions. This is not the right place.

Hi @wifirevaithi
Please see https://flutter.dev/community for resources and asking questions like this,
you may also get some help if you post it on Stack Overflow.
Closing, as this isn't an issue with Flutter itself,
if you disagree please write in the comments and I will reopen it
Thank you


Invalid SKU on ubuntu after firmware upgrade in windows

Required Info

Camera Model
D455

Firmware Version
not available after recommended upg in windows

Operating System & Version
Linux Ubuntu 18.04 LTS / Windows10

Kernel Version (Linux Only)
4.15.0-136-generic

Platform
PC

SDK Version
2.0

Language
C/C++/Python/Opencv

Segment
Deep Learning

Issue Description
I've upgraded a D455 on my windows system following the recommendation provided by the realviewer. Unfortunately the realsense version was not updated and something get wrong. After upgrading, the camera is not recognized both on windows and linux.
I've investigated on this behavoiour and after having verified that everything it is fine with an other D455, not upgraded, (so no cable/driver/software/... problem) I found that this seems due to the upgrade with a wrong version (noticeably it was the one recommended!).
On windows, plugging-in the upgraded camera the message is: "The last device you connected to this computer system malfunctioned and Windows does not recognize it". Quite hermetic...
In Linux the message is more useful in my opinion (/var/log/syslog):
Feb 25 09:10:58 DEEP-ONE kernel: [  284.426565] usb 2-1: new SuperSpeed USB device number 39 using xhci_hcd
Feb 25 09:10:58 DEEP-ONE kernel: [  284.446915] usb 2-1: config 1 interface 0 altsetting 0 has 0 endpoint descriptors, different from the interface descriptor's value: 2
Feb 25 09:10:58 DEEP-ONE kernel: [  284.447111] usb 2-1: New USB device found, idVendor=8086, idProduct=0af5
Feb 25 09:10:58 DEEP-ONE kernel: [  284.447116] usb 2-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
Feb 25 09:10:58 DEEP-ONE kernel: [  284.447120] usb 2-1: Product: Intel(R) RealSense(TM) Invalid SKU
Feb 25 09:10:58 DEEP-ONE kernel: [  284.447124] usb 2-1: Manufacturer: Intel(R) RealSense(TM) Invalid SKU
Feb 25 09:10:58 DEEP-ONE kernel: [  284.447128] usb 2-1: SerialNumber:<PHONE_NUMBER>86
Feb 25 09:11:03 DEEP-ONE kernel: [  289.666754] usb 2-1: Enable of device-initiated U2 failed.
Feb 25 09:11:03 DEEP-ONE mtp-probe: checking bus 2, device 39: "/sys/devices/pci0000:00/0000:00:14.0/usb2/2-1"
Feb 25 09:11:03 DEEP-ONE mtp-probe: bus: 2, device: 39 was not an MTP device
Comparing the log obtained from a functioning D455:
Feb 25 09:22:08 DEEP-ONE kernel: [  954.150124] usb 2-1: new SuperSpeed USB device number 83 using xhci_hcd
Feb 25 09:22:08 DEEP-ONE kernel: [  954.170711] usb 2-1: New USB device found, idVendor=8086, idProduct=0b5c
Feb 25 09:22:08 DEEP-ONE kernel: [  954.170717] usb 2-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
Feb 25 09:22:08 DEEP-ONE kernel: [  954.170721] usb 2-1: Product: Intel(R) RealSense(TM) Depth Camera 455
Feb 25 09:22:08 DEEP-ONE kernel: [  954.170725] usb 2-1: Manufacturer: Intel(R) RealSense(TM) Depth Camera 455
Feb 25 09:22:08 DEEP-ONE kernel: [  954.170728] usb 2-1: SerialNumber:<PHONE_NUMBER>34
Feb 25 09:22:08 DEEP-ONE kernel: [  954.171828] uvcvideo: Found UVC 1.50 device Intel(R) RealSense(TM) Depth Camera 455  (8086:0b5c)
Feb 25 09:22:08 DEEP-ONE kernel: [  954.173341] uvcvideo: Unable to create debugfs 2-83 directory.
Feb 25 09:22:08 DEEP-ONE kernel: [  954.173443] uvcvideo 2-1:1.0: Entity type for entity Intel(R) RealSense(TM) Depth Ca was not initialized!
Feb 25 09:22:08 DEEP-ONE kernel: [  954.173448] uvcvideo 2-1:1.0: Entity type for entity Processing 2 was not initialized!
Feb 25 09:22:08 DEEP-ONE kernel: [  954.173452] uvcvideo 2-1:1.0: Entity type for entity Camera 1 was not initialized!
Feb 25 09:22:08 DEEP-ONE kernel: [  954.173605] input: Intel(R) RealSense(TM) Depth Ca as /devices/pci0000:00/0000:00:14.0/usb2/2-1/2-1:1.0/input/input13
Feb 25 09:22:08 DEEP-ONE kernel: [  954.174287] uvcvideo: Found UVC 1.50 device Intel(R) RealSense(TM) Depth Camera 455  (8086:0b5c)
...
the problem seems to be the fact that the upgraded camera pretend to have a IdProduct equal to 0af5 and I assume that this is due to the wrong upgrade.
Unfortunately even following the instructions provided for similar issues  don't work and all the tools cannot see the upgraded camera because it is not binded to a correct driver => no device connected.
So the question is pretty simple: how can I override the Product ID in order to trick (bypass?) the system to accept the wrong PID as a D435 PID and allowing a correct firmware upgrade with the realsense tools?
Or should I consider the camera absolutely dead?
I think that a tool for recovering from a similar wrong state could be really useful

The PID=0x0af5 means the RGB ISP is under FW updating. This PID is supposed to stay shortly then change to 0x0b5c. Is the camera now always showing PID=0af5?
If the D455 is unable to be recognized on USB port, that could be "dead" in FW. You probably did FW update with some old one which is not supported on D455. Do you remember which old FW version was used and caused the issue? Let me try  reproducing the problem and see if there is any solution to recover it.
Thanks

Thank you for your answer.
It is still showing PID 0af5.
Unfortunately I don't remember the version used by the recommended upgrade but I can say that the realsense version on windows was 2.34 and the original version on d455 was the one provided with the camera. Hopefully this allow you to reproduce the problem.

@emaffione, could you please try the following on Linux OS?

Install Python3.7-64
Extract ForceDFU.zip
Type: python3.7 ForceDFU.py
Type: lsusb
Check whether the PID has changed.

Thank @icarpis
I tryed (several times) but it doesn't work:
$ python3.7 ForceDFU.py
No RealSense Camera has found!
I think that the problen is due to the fact that linux continously reassign a new id to the not recognized device.
Launching lsusb more times I get different Device Id.
$ lsusb
Bus 002 Device 012: ID 8086:0af5 Intel Corp.
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 002: ID 046d:c31c Logitech, Inc. Keyboard K120
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
$ lsusb
Bus 002 Device 013: ID 8086:0af5 Intel Corp.
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 002: ID 046d:c31c Logitech, Inc. Keyboard K120
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
I don't know if I can stop this behaviour...

@emaffione, thank you for trying my suggestion.
Yes you right, it looks that Linux continuously reassign new device id.
Maybe we have a short time window for sending the DFU mode command.
Could you please try again the above with the following new Python script?
ForceDFU2.zip
Also, please try to run with sudo, since the 0xaf5 PID is not added to the Udev Rules file.

Thanks for you new script.
I've launched the command as sudo and then lsusb. The PID is always 0xaf5. I think I should trick  the usb driver in order to stop reassigning a new id and then use the script.
$ sudo python3.7 ForceDFU2.py
Try to enter DFU mode for 7 seconds...
Done!
$ lsusb
Bus 002 Device 017: ID 8086:0af5 Intel Corp.
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 002: ID 046d:c31c Logitech, Inc. Keyboard K120
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

I did a quick test on Viewer 2.34.0 by using Recommended FW which is 5.12.3, but the D455 is working. However one issue is found that there is no option to stream Left IR. But anyway, the Viewer is too old to D455. The latest Viewer doesn't have such issue.
Thanks
Attachment(s):
D455_RS2.34.0.PNG - https://realsensesupport.intel.com/attachments/token/eAqpPrGsIQlFAuXj5pkmtJpFl/?name=D455_RS2.34.0.PNG

I have the same issue with my device. Last week I got the notification on Windows 10 that a Firmware Upgrade was available for the D455, which I installed.
The device is now bricked. Upon connecting it to a Windows 10 PC, the message "USB device has not been recognized" pops up.
Entering the following command:
rs-fw-update -l
results in:
No RealSense Camera has found!
Does this mean it's not possible to revert this firmware "upgrade"?

Hi fgldmn,
Can you please tell us which RealSense Viewer version you were using for the FW update?
If it's unable to be recognized on USB port, there is no SW tool can recover it. Please return it or leave your email address here (format: i.e. ). I will contact you via Intel email address and tell you how to return it.
Thanks

Thanks for getting back to me so quickly.
I installed the SDK on 31st of August 2020, the filename of the installer was Intel.RealSense.SDK-WIN10-<IP_ADDRESS>3.exe.
I cannot tell you further details because I installed the newest SDK (2.42.0) yesterday in an attempt to resolve the issue.
my email is florian[dot]goldmann[at]iis[dot]fraunhofer[dot]de

Hi "emaffione",
I think it's better to return this D455 and we can recover it from "dead" firmware. Please leave your email address here (format: i.e. ), I will contact you via Intel email address and tell you how to return it.
Thanks

Thanks for your support. I'll be glad to help you in finding out a software solution, if possibile.
My email is eugenio[at]geoinference[dot]com

I'm also facing the same issue where the realsense D457 is not detected when connected to system (windows and linux). The issue is because the firmware has not installed properly. And currently I'm getting "Invalid SKU" when listing all usb devices.
Please assist in fixing the issue.

Hi @Sujeetdas07  On the D457 model there is a hidden mode switch in the casing that needs to be moved sidewards into the 'USB' position in order to use the camera with a USB cable.  Have you moved the switch across to 'USB' in the way described at the link below, please?
https://support.intelrealsense.com/hc/en-us/community/posts/14840675121043/comments/14843232242835

Yes @MartyG-RealSense, the switch was always on the left side as earlier it was working fine, currently the camera is not detected in linux and getting 'Invalid SKU' as the firmware was not installed properly in windows. I tried updating the firmware using rs-fw-update, but it shows no device connected.

If you have an Invalid SKU error resulting from a failed firmware update then it cannot be fixed with software, unfortunately.  The only solution is to mail your camera to Intel to have it recovered.
If you would like to proceed with that option then please send me an email at the address below with the name of your country included in it and I can provide further details.
<EMAIL_ADDRESS>

Sure, I will send an email right away, also just want to clarify the statement mentioned in one thread, where firmware can be updated by opening up the realsense camera and disconnecting the vision processor D4 board. So, can you confirm if this method will work for sure (if tried) as we have a tight deadline ahead. Please let me know, but at the same let's proceed ahead in returning the camera and get to know the overall time required it to fix it from your side.

@Sujeetdas07  Thanks very much.  I have replied to your email and provided in it an estimate for the time required for a fix after receiving a returned camera.
You can restore the camera yourself by replacing the Vision Processor D4 board.  The V5 version of the D4 board must be used with a D457.

I just have to follow the steps mentioned in the below link right?
https://support.intelrealsense.com/hc/en-us/community/posts/16647570577683-D457-Camera-no-longer-detected-after-a-failed-change-of-firmware

You can update the firmware either with rs-fw-update as described in the above link, or do it in the RealSense Viewer tool.

Sure, thanks for the info. will decide and revert back to you. Thanks for the assistance


RDS Schema Migrations - Error: Could not resolve "pg-native"

Hello,
I'm attempting to use the new RDS construct w/ Kysely to run migrations for Aurora Serverless.
I'm seeing the following error when I click the "Migrations" button in the SST console (while running locally w/ sst start):
7dfa4346-d7a0-44a8-9c34-744bbf50573c REQUEST aurora-api-db-dbrdsauroraMigrationFunction97E-vOGZsozm21aw [index.handler] invoked
✘ [ERROR] Could not resolve "pg-native"

    node_modules/pg/lib/native/client.js:4:21:
      4 │ var Native = require('pg-native')
        ╵                      ~~~~~~~~~~~

  You can mark the path "pg-native" as external to exclude it from the bundle, which will remove
  this error. You can also surround this "require" call with a try/catch block to handle this
  failure at run-time instead of bundle-time.

7dfa4346-d7a0-44a8-9c34-744bbf50573c ERROR build_failure: The function index.handler failed to build

Any idea what could be happening here? Thanks.

Hey @bbeckrest, are you using Prisma or any ORM client?

Yeah, I was confused too. The migrator does not import pg. Btw, are you configuring any function props using app.setDefaultFunctionProps()?

I am, but just this though:
app.setDefaultFunctionProps({
  runtime: 'nodejs14.x',
});

Idk how this is related, but here's my API construct that references pg-native. I think this is the only place I reference it.
   this.api = new sst.Api(this, 'Api', {
      // ...
      defaultFunctionProps: {
        environment: {
          DB_NAME: process.env.DB_NAME,
          DB_CLUSTER_ARN: dbCluster.clusterArn,
          DB_SECRET_ARN: dbCluster.secretArn,
        },
        permissions: [dbCluster],
        bundle: {
          esbuildConfig: {
            keepNames: true,
          },
          externalModules: ['pg-native', 'pg-query-stream'],
          nodeModules: ['oso'],
          copyFiles: [{ from: 'src/oso/main.polar', to: 'src/oso/main.polar' }],
        },
        // ...
      });

So once I deleted & uninstalled knex and pg from my package.json, the error disappeared and I can apply migrations successfully.
Not sure why that was affecting the migrator though.

Hmm.. that's weird.
Are you ready to remove knex and pg from your app and switch to myself? If you have to put them back, would it be possible for u to setup a sample repo that can reproduce the issue? I can give it a try on my end.


tf.data.experimental.CsvDataset iterates forever

I have a csv file that has number of rows equal to 1 batch but when I run my code it loops that batch forever and never throws an exception. How can I make the iterator stop after it reaches the end of the dataset? Is this a bug?
train_dataset_1 = tf.data.experimental.make_csv_dataset([os.path.join(FLAGS.data_path, "small_interpolated_2_4_6_ONE.csv")],
                                                       batch_size=BATCH_SIZE*SEQ_LEN,
                                                       select_columns=[5,6],
                                                       label_name='angle',
                                                       shuffle=False,
                                                       column_defaults=[tf.string, tf.float32])

iterator_1 = Iterator.from_structure(train_dataset_1.output_types, train_dataset_1.output_shapes)

train_iter_init_op_1 = iterator_1.make_initializer(train_dataset_1)
next_batch_op_1 = iterator_1.get_next()

config = tf.ConfigProto(gpu_options=tf.GPUOptions(allow_growth = True),
                        allow_soft_placement=True,
                        log_device_placement=False)
sess = tf.Session(config=config)
sess.run(tf.global_variables_initializer())
sess.run(train_iter_init_op_1)

while True:
    try:
        img_batch_dict, label_batch = sess.run(next_batch_op_1)

        for k in img_batch_dict["filename"]:
            print(k)

        print()
    except tf.errors.OutOfRangeError:
        print("Epoch ended<PHONE_NUMBER>1111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111")
        break

Can you provide all necessary environment information required by issue template?

It looks like you haven't used a template to create this issue. Please resubmit your issue using a template from here. We ask users to use the template because it reduces overall time to resolve a new issue by avoiding extra communication to get to the root of the issue. We will close this issue in lieu of the new one you will create from the template. Thank you for your cooperation.

Is this issue resolved? @chris12766


JMX - Refactor the Attributes Component to include tabular form of multiple beans

The old Attributes Component includes the ability to display the children of a group category, eg. Contexts, as a table with the bean attributes as the columns and mbeans as the rows:

Old Attributes component for camel contexts ^

@joshiraez You can work on this too.

I'm picking this up then and using @phantomjinx approach in camel as a base. I'll do it after the current camel help page

I spoke with Paul and I'll be picking this up to continue the work on the JMX plugin. From the mentioned Camel issue for the attributes view it seems they should be similar so I'll tr to take that as a base

@tadayosi Just to be sure, this extends #10 right? The chart component should be used for this


drivers: sensors: add veaa_x_3 proportional pressure regulator sensor

Add driver for Festo VEAA-X-3 series proportional pressure regulator.
The driver assumes that the maximum ADC value matches the maximum output from the device, and that the maximum DAC value matches the maximum input value for the device. External hardware is probably required between the ADC/DAC and the device.

I pushed some commits by a mistake to this branch and have removed them again.
@zephyrbot added new reviewers due to this mistake. Sorry about that!

Needs to be rebased

Needs to be rebased

Thanks. Done :)

@MaureenHelm @teburd I rebased to resolve conflicts and removed the unnecessary casts.

Would you be willing to maintain this driver? If so please add yourself (in another PR) as maintainer of this driver alongside @MaureenHelm as co-maintainer.
We are currently in the process of working, ideally, on moving away from fetch+get to read+decode, new drivers are great but I'd love to see them maintained by the authors if possible.
I can't possibly update every driver in tree, and I'd like us to move that way.

Makes sense. Created https://github.com/zephyrproject-rtos/zephyr/pull/72962.
Thanks for the re-reviews :)


First Test is Wrong

In first test it says will launch the default browser and navigates to https://localhost:/swagger/index.html
It does not it goes to https://localhost:44351/weatherforecast.
Swagger does not appear to be installed at all. (No Swagger Nuget package or other references) ???
This tutorial needs to be tested !!!!!!!!!

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 583ae3e7-5915-f22b-e54c-6c100cf2a7da
Version Independent ID: 95f2549e-29f9-e31a-b4d0-432b971643ff
Content: Tutorial: Create a web API with ASP.NET Core
Content Source: aspnetcore/tutorials/first-web-api.md
Product: aspnet-core
Technology: aspnetcore-tutorials
GitHub Login: @Rick-Anderson
Microsoft Alias: riande

That's for the 5.0 version.


there is no way to declare public final fields

The compiler always create private fields:
https://github.com/mirah/mirah/blob/master/lib/mirah/jvm/compiler.rb#L593-597
In some cases, constants for instance, public and final fields are useful.

Hm. I think since we're emulating Ruby's behavior it might make sense to default fields to protected. We could allow using @syntax to refer to private and public fields that are not constants.
Maybe for that we'd want to make a syntax for declarations in cases where you want to change the visibility.


Moving Reveal protocol out of sort

Reveal is also used in a variety of other protocols, including modulus conversion.

Looks good! This should have been outside from the start :/ also, similar to multiplication having multiply_all for entire vector, we would also have reveal_all which might be useful for other protocols.


[Bug] addRows replace heading row when there are hidden columns

Below is the code I'm currently using:
const serviceAccountAuth = new JWT({
  email: process.env.GOOGLE_SERVICE_ACCOUNT_EMAIL,
  key: process.env.GOOGLE_PRIVATE_KEY,
  scopes: ['https://www.googleapis.com/auth/spreadsheets'],
});

const doc = new GoogleSpreadsheet('<some sheet ID>', serviceAccountAuth);
await doc.loadInfo();

const data = getRowData();

if (data.length > 0) {
  const sheet = doc.sheetsByTitle['some title'];
  await sheet.clear();
  await sheet.setHeaderRow(Object.keys(data[0]));

  // This replaces the heading row if there are hidden columns
  await sheet.addRows(data);
}

This works fine when there are no hidden columns. However, if I were to hide some columns using Google Sheet's GUI interface, the heading gets replaced by the first row when using sheet.addRows(data)
By hidden columns, I mean this:

hello, @nktnet1  have you found any workaround for this issue?

Not really - other than manually un-hiding the columns, run my script, then hide them again.


config.yml  for segnet

I want to train segnet, but i don't know how to set the config. Can anyone help me? Thank you so much.
What's more, how should i process the dataset if i use pascal 2012?


Users expected to select from a list of department and organizations

Although government employees would be comfortable with a long dropdown list, it's recommended to look deeper into this issue to see if there is a more user-friendly pattern that can be applied; the design system might be able to help here.
The design for this is pretty straightforward, but more needs to be understood on who should own and maintain this pattern.

The CRM team maintains a list of departments and has implemented a typeahead department selection component with Notify.

This would be a good one for UI / usability improvement @dsamojlenko @Abi-Nada

Beth (she/her)
10:09 AM
https://github.com/cds-snc/gc-organisations
https://github.com/cds-snc/gc-organisations-qa
Pat (in core) and Stephen M (on leave?) both worked on this (edited)
10:10
Also we had asked Abdul if he would build a reusable one, not sure if that ever started... might be worth checking with that team to see how you might save duplication?

Link to Figma design: https://www.figma.com/file/gkmd5aQpcnjuPeAT51oIFs/v2.-Add-modal?type=design&node-id=2349%3A3299&mode=design&t=GSPBpJn4mRFdy57H-1

FIGMA

https://github.com/cds-snc/platform-forms-client/pull/3139

Work has been merged into Staging. We'll close this card and create new issues for any bugs.


FIX: AWS Deployment name was changed due to aws resource name restrictions

https://docs.aws.amazon.com/ARG/latest/userguide/gettingstarted-query.html

A resource group name can have a maximum of 128 characters, including letters, numbers, hyphens, periods, and underscores. The name cannot start with AWS or aws. These are reserved. A resource group name must be unique in the current region in your account.

Ah! missed that. Thanks for fixing this.


How configuration would work in .NET

.NET configuration is a very common and convenient way to configure all sorts of .net apps.
It supports multiple configuration sources, but merges them in one common configuration object.
The common pattern is to define configuration for all components in a single file - appsettings.json and add/override things via other sources.
Given that it's the idiomatic way to do things in this ecosystem:

users will be surprised that they need to keep their otel config in a different file than the rest of their configuration.
they would likely be surprised they need to configure the file with the env var

The ideal outcome as I see it is that configuration story should leave a room for SDKs to decide how onboarding happens and whether it's done within a separate file or as a part of existing configuration system.
I assume there could be a similar ask to integrate otel configuration with spring config, microprofile config or something else.
In .NET it might look like a otel-specific section in the appsettings.yaml and no env var to opt into - this is pretty much how it could happen today, but I assume the schema will need to change to match new one defined in this repo.

I believe that is already the case. At least that is how I planned it for Erlang/Elixir. We have similar in Erlang config. A Release contains at least 1 sys.config that configures the Applications within it. Today users configure OpenTelemetry with this file so it picks it up on boot and no user code is necessary to to start the SDK (allowing it to start before all other Applications in the Release).
But if the user passes a configuration file the plan was to ignore per-Application configuration on config since any sort of merging may be too painful and confusing.

Thanks for the clarification, @tsloughter
Reading https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/configuration/sdk-configuration.md#other-mechanisms, I get an impression that 'other mechanisms' are unregulated by the spec.
I.e. the schema and configuration API are part of file configuration and don't apply to other mechanisms.
Also, the OTEP says that SDK MUST provide file option, which would probably be a rare case for .NET users.
It'd be great to have more clarity eventually.

Right, the other mechanisms are unregulated.

Also, the OTEP says that SDK MUST provide file option, which would probably be a rare case for .NET users.

Oh, were you saying .NET didn't want to provide support for the file at all?

We've had some talks in .NET but haven't made any decisions yet. We definitely want to support the config file. We also would like to support the standard .NET way of configuring things.
Here are my personal thoughts...
Today we support .NET's IConfiguration. The IConfiguration given to the app is composited from any number of sources. Typically, envvars, command-line, appSettings.json files. But you can add in other things like cloud config repositories. One simple thing we could do is pick up the open telemetry config file and just treat it as just another config source (transposing it into IConfiguration style). In theory, users who want to use the config file will be happy and users who want to stick to appSettings.json (or whatever) will also be able to do everything they could using the config file.
Now the config file itself has some dependency requirements. Like you can turn on OtlpExporter via the config. The .NET SDK alone can't do that without a dependency to the OtlpExporter assembly. So we might have to ship all of this as an add-on meta package (or something) which brings in all the assemblies for what is possible to configure. Or we try to dynamically load assemblies from the SDK for what is requested and blow up on startup for missing dependencies of things requested via config.
Anyway I feel we have options to explore and we should be able to do something to support the config file. Likely needs some prototyping.

So we might have to ship all of this as an add-on meta package (or something) which brings in all the assemblies for what is possible to configure.

I think this is a problem for every implementation. Esp when third party propagators, id generators, exporters and eventually instrumentation libraries are taken into account. But a separate issue to this discussion.
But I will ask, shouldn't it already be an issue? What happens today if the user defines OTEL_TRACES_EXPORTER to OTLP?

At the moment .NET doesn't support OTEL_TRACES_EXPORTER. The only component-configuring key we support is OTEL_TRACES_SAMPLER / OTEL_TRACES_SAMPLER_ARG because the SDK happens to have the samplers on hand. That was just added in the last week or so coincidentally.


Simple Code to connect to server, spawn a vehicle with a camera and retrieve frames

Could you provide a sample code for connecting to the world, spawning a vehicle with a camera sensor and running the world in a loop.
There isnt a proper documentation of whether sensor.listen needs to happen in every loop update, or only once. Also if the world has to be updated?

Give this a good read:
https://carla.readthedocs.io/en/latest/getting_started/
https://carla.readthedocs.io/en/latest/python_api_tutorial/
https://carla.readthedocs.io/en/latest/python_cookbook/
https://carla.readthedocs.io/en/latest/python_api/
Then starting from either manual_control.py or automatic_control.py you can work your way through this.

Actually, the saving image code in manual_control.py is rather advanced in my opinion with the controller classes. The code in PythonAPI/examples/tutorial.py is way simpler and I would suggest to start there for a simple code example.
Also read up on synchronized mode, that may also help with your questions.
But basically, you give sensor.listen() a callback function that get's called every simulation frame. You only need a "loop to run the simulation" in sync mode. Not sure what you mean with "updating the world". It's not a low level 3D engine where you need to set transforms to animate things. It's a simulator. It does those things for you.

@cassfalg so every sensor has its listen function. To drive a car using the information from the camera, should I interpret the rgb data and set the control in the listen function? where do people control the car using only the sensor data?


Fixed-Admin Create Task - End Date Validation

closes https://github.com/HTBox/allReady/issues/1312

@jnSWPC thanks for your contribution.
@tonysurma  or @digitaldrummerj :shipit:


Add a 'Subscribe' button to the  mailing list signup?

I figured I could subscribe by hitting 'enter', but some people may not think about that. It may keep them from signing up. Thoughts?

Fixed :)


added all changes for audio feature input

Code Pull Requests
separate add audio feature pull request with correct author.
integration test is included.
Please provide the following:

a clear explanation of what your code dose
if applicable, a reference to an issue
a reproducible test for your PR (code, model definition and data sample)

Documentation Pull Requests
Note that the documentation HTML files are in docs/ while the Markdown sources are in mkdocs/docs.
If you are proposing a modification to the documentation you should change only the Markdown files, then recreate the documentation as explained in the mkdocs/README.md file with mkdocs build, which will create the HTML files automatically, and only after this create a commit.
api.md is automatically generated from the docstrings in the code, so if you want to change something in that file, first modify ludwig/api.py docstring, then run mkdocs/code_docs_autogen.py, which will create mkdocs/docs/api.md and then finally run mkdocs build which will generate the HTML in docs/.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.1 out of 2 committers have signed the CLA.:white_check_mark: patrickvonplaten:x: pvonplatenpvonplaten seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


release drafter and pypi

Codecov Report

Merging #63 (4fcf52a) into master (17625ba) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #63   +/-   ##
=======================================
  Coverage   82.91%   82.91%
=======================================
  Files          11       11
  Lines         597      597
=======================================
  Hits          495      495
  Misses        102      102

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 17625ba...4fcf52a. Read the comment docs.

@chayim build is failing


Should default style be "short" instead of "long"?

The slide presented in TC39 2021 Oct meeting meeting
https://ryzokuken.dev/slides/2021-10-df#/6/0/2
mentioned "There is a base style. This is "short" by default."
and in https://github.com/tc39/notes/blob/HEAD/meetings/2021-10/oct-25.md#durationformat
"...it would by default give you the short style..." and
"This is short by default, but you can..."
But the current spec text in step 13 of
https://tc39.es/proposal-intl-duration-format/#sec-Intl.DurationFormat
use "long" as default instead.
@ryzokuken @sffc @anba @michaelficarra @rbuckton @rkirsling

Discussion: https://github.com/tc39/ecma402/blob/master/meetings/notes-2022-09-01.md#should-default-style-be-short-instead-of-long-116
Conclusion: Use "short" as the default for non-numeric digital.

#121  Closes this one


Array#remove is slow and description is not very clear.

The splice and concat in a loop introduce O(N^2) solution.
Also, the current implementation has tricky side-effects. I'll address both things in a PR.

I'm closing the issue, as we can talk about thins under #196. Thanks for taking the time to contribute!


Any chance to make pty.js compatible with Node 0.11.x ?

I manage to modify and install pty.js but it doesn't work with Node.js 0.11.13.
Is there any plan or any patch that could make it work with newer node version ?
Thanks

Activity in that node ticket, asking if this is still a problem.

It has been fixed a while ago by @chjj I don't remember the exact commit.
I guess we can close this :)


Fails in webpack 4 with UglifyJsPlugin

Error: webpack.optimize.UglifyJsPlugin has been removed, please use config.optimization.minimize instead.
node_modules/serviceworker-webpack-plugin/lib/index.js:198:61

Work in progress, feel free to - temporarily - use oliviertassinari/serviceworker-webpack-plugin#next in your package.json. A webpack 4 compatible release will be released to npm asap.

@kontrollanten Looking into that. I just recently joined this project, so bear with us.

I see! Thanks! Keep up the good work!

I just pushed out the first alpha with webpack 4 support. You can install this as `serviceworker-webpack-plugin@next'.
If you find any more issues, please do open an issue marked for the 1.0.0 milestone


GlobalFilters are not executing when RedisRateLimiter blocks a request

I have implemented a global prefilter and a global postfilter to log the request events. I also have used a RedisRateLimiter filter to avoid requests peaks that might overwhelm the service.
I want to use my pre and post filter to log two events.

A request from apiKey X has been noticed

The request has been either accepted or rejected by the rate limiter.

When the rate limiter accepts a request and forwards it, there is no problem, the pre and post filter execute and I can get to log the data I need. The problem is that when the rate limiter blocks a request, neither the pre nor the postfilter are being executed so I cannot log or retrieve this event for metrics.
Even though spring cloud gateway works through a chain of filters I think that the problem above described may be caused by the isAllowed method from the RequestRateLimiterFactory class. This method sets completed the request when it is rejected instead of passing the exchange to the next filters. But I have been reasearching, and I did nothing useful to solve this problem.
return limiter.isAllowed(routeId, key).flatMap(response -> {

				for (Map.Entry<String, String> header : response.getHeaders().entrySet()) {
					exchange.getResponse().getHeaders().add(header.getKey(), header.getValue());
				}

				if (response.isAllowed()) {
					return chain.filter(exchange);
				}

				setResponseStatus(exchange, config.getStatusCode());
				return exchange.getResponse().setComplete();
			});

@JavierBrRuiz When one request bolcked ServerWebExchange should not be transfer to next GatewayFilter or GlobalFilter,otherwise rate limiter useless.You can do log work in WebHandler instead.

@ctlove0523
First of all, thank you for answering.
It makes sense to me since I am picturing in mind the basic spring cloud schema where the web handler takes care of the exchange before it is passed to the filters. (basic scheme example)
Anyway I do not know how to gain access to the webHandler, I have been reading the docs but I did not find nothing about a custom implementation.
What I mean is that for example, for predicates or filters I found resources such us baeldung where I could see examples of implementatios that had access to the exchange. I do understand that the webhandler will have access to the exchange but I do not know how to implement the code to access the webHandler.
Could you provide some code example or some documentation about this, where I could learn how to access the webHandler and therefore, log events in there?

@JavierBrRuiz
First,define your WebHandler and do what you want in handle(ServerWebExchange exchange) method：
public class LogWebHandler implements WebHandler {
    private static final Logger log = LoggerFactory.getLogger(LogWebHandler.class);

    private final DispatcherHandler defaultHandler;

    public LogWebHandler(DispatcherHandler defaultHandler) {
        this.defaultHandler = defaultHandler;
    }

    @Override
    public Mono<Void> handle(ServerWebExchange exchange) {
        log.info("before dispatch");
        defaultHandler.handle(exchange);
        log.info("after dispatch");
        return Mono.empty();
    }
}

Second,replace the default WebHandler use LogWebHandler：
@Component
public class LogWebHandlerPostProcessor implements BeanPostProcessor {
    private static final Logger log = LoggerFactory.getLogger(LogWebHandlerPostProcessor.class);

    @Override
    public Object postProcessBeforeInitialization(@NotNull Object bean, @NotNull String beanName) throws BeansException {
        return bean;
    }

    @Override
    public Object postProcessAfterInitialization(@NotNull Object bean, @NotNull String beanName) throws BeansException {
        if ("webHandler".equals(beanName) && bean instanceof DispatcherHandler) {
            log.info("begin to create log dispatcher handler");
            return new LogWebHandler((DispatcherHandler) bean);
        }
        return bean;
    }
}

Thank you very much for the information!, I will get hands on with the implementation. If everything works as expected I will close the issue, else, I will post here the problems.
Thank you again for the help.

@ctlove0523
I have implemented the classes you provided, it builds and runs, but it is not logging nothing on the console when I make a request.
My project is very simple so here you have a replication in case you want to try and replicate the problem.
application.yml
spring:
  redis:
    host: localhost
    port: 6379
  cloud:
    gateway:
      routes:
        - id: basic
          uri: http://localhost:8080/api
          predicates:
            - Path=/api
            - Header=Authorization, aut0
          filters:
            - name: RequestRateLimiter
              args:
                redis-rate-limiter.replenishRate: 10
                redis-rate-limiter.burstCapacity: 20

        - id: standard
          uri: http://localhost:8080/api
          predicates:
            - Path=/api
            - Header=Authorization, aut1
          filters:
            - name: RequestRateLimiter
              args:
                redis-rate-limiter.replenishRate: 1
                redis-rate-limiter.burstCapacity: 0

And the rest is the classes you provided.
When I do a request to aut0 or aut1 redisratelimiter is working as expected but in neither case is logging with the webHandler.
> Task :compileJava UP-TO-DATE
> Task :processResources UP-TO-DATE
> Task :classes UP-TO-DATE
> Task :bootRunMainClassName

> Task :bootRun
2021-11-30 10:39:33.496  INFO 9032 --- [           main] o.s.boot.SpringApplication               :
  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::                (v2.5.6)

2021-11-30 10:39:33.582  INFO 9032 --- [           main] c.d.s.g.GatewayModuleApplication         : Starting GatewayModuleApplication using Java 1.8.0_181 on DMS0815 with PID 9032 (C:\Users\JABR\SD\gateway_module\build\classes\java\main started by jabr in C:\Users\JABR\SD\gateway_module)
2021-11-30 10:39:33.582  INFO 9032 --- [           main] c.d.s.g.GatewayModuleApplication         : The following profiles are active: @activeProfiles@
2021-11-30 10:39:34.403  INFO 9032 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Multiple Spring Data modules found, entering strict repository configuration mode!
2021-11-30 10:39:34.406  INFO 9032 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data Redis repositories in DEFAULT mode.
2021-11-30 10:39:34.434  INFO 9032 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 9 ms. Found 0 Redis repository interfaces.
2021-11-30 10:39:34.621  INFO 9032 --- [           main] o.s.cloud.context.scope.GenericScope     : BeanFactory id=a82e580d-a2d2-3a6a-bbf2-6a7b0a57f540
2021-11-30 10:39:36.646  INFO 9032 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port(s): 8081 (http)
2021-11-30 10:39:36.657  INFO 9032 --- [           main] o.apache.catalina.core.StandardService   : Starting service [Tomcat]
2021-11-30 10:39:36.657  INFO 9032 --- [           main] org.apache.catalina.core.StandardEngine  : Starting Servlet engine: [Apache Tomcat/9.0.54]
2021-11-30 10:39:43.743  INFO 9032 --- [           main] o.s.c.g.r.RouteDefinitionRouteLocator    : Loaded RoutePredicateFactory [After]
2021-11-30 10:39:43.743  INFO 9032 --- [           main] o.s.c.g.r.RouteDefinitionRouteLocator    : Loaded RoutePredicateFactory [Before]
2021-11-30 10:39:43.743  INFO 9032 --- [           main] o.s.c.g.r.RouteDefinitionRouteLocator    : Loaded RoutePredicateFactory [Between]
2021-11-30 10:39:43.743  INFO 9032 --- [           main] o.s.c.g.r.RouteDefinitionRouteLocator    : Loaded RoutePredicateFactory [Cookie]
2021-11-30 10:39:43.743  INFO 9032 --- [           main] o.s.c.g.r.RouteDefinitionRouteLocator    : Loaded RoutePredicateFactory [Header]
2021-11-30 10:39:43.743  INFO 9032 --- [           main] o.s.c.g.r.RouteDefinitionRouteLocator    : Loaded RoutePredicateFactory [Host]
2021-11-30 10:39:43.743  INFO 9032 --- [           main] o.s.c.g.r.RouteDefinitionRouteLocator    : Loaded RoutePredicateFactory [Method]
2021-11-30 10:39:43.743  INFO 9032 --- [           main] o.s.c.g.r.RouteDefinitionRouteLocator    : Loaded RoutePredicateFactory [Path]
2021-11-30 10:39:43.743  INFO 9032 --- [           main] o.s.c.g.r.RouteDefinitionRouteLocator    : Loaded RoutePredicateFactory [Query]
2021-11-30 10:39:43.743  INFO 9032 --- [           main] o.s.c.g.r.RouteDefinitionRouteLocator    : Loaded RoutePredicateFactory [ReadBody]
2021-11-30 10:39:43.743  INFO 9032 --- [           main] o.s.c.g.r.RouteDefinitionRouteLocator    : Loaded RoutePredicateFactory [RemoteAddr]
2021-11-30 10:39:43.743  INFO 9032 --- [           main] o.s.c.g.r.RouteDefinitionRouteLocator    : Loaded RoutePredicateFactory [Weight]
2021-11-30 10:39:43.744  INFO 9032 --- [           main] o.s.c.g.r.RouteDefinitionRouteLocator    : Loaded RoutePredicateFactory [CloudFoundryRouteService]
2021-11-30 10:39:46.176  INFO 9032 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 8081 (http) with context path ''
2021-11-30 10:39:50.447  INFO 9032 --- [           main] c.d.s.g.GatewayModuleApplication         : Started GatewayModuleApplication in 20.042 seconds (JVM running for 20.673)

@JavierBrRuiz can your post your code into your repository so i can write and reproduce this issue.

Hi @ctlove0523, Here you have.
https://github.com/JavierBrRuiz/spring-cloud-gateway-rate-limiter

@JavierBrRuiz I crate a PR for your repository, can you try again?

@ctlove0523 Now it logs before dispatch and after dispatch, the problem now is that when it uses the custom webHandler you provided, Redis Rate Limiter is not working.
Without custom Handler
If I delete the ComponentScan annotation, it does not use the custom Handler and when I do a request, RedisRateLimiter works adding its headers and all its functionality.

With custom Handler
If I add ComponentScan, to use the custom Handler, it logs on console and the request is not being forwarded, I get a 200 status on postman but no rate limiter headers and not body responses.

@ctlove0523 Thank you for the last PR, it solved my last comment. Anyway the originator purpose of the issue was to log accepted and rejected responses. You said this could be done by the web Handler. I tried to log request headers/httpstatus pre-dispatch and response-headers/httpstatus after dispatch, being able to log total requests, accepted and rejected by the rate limiter.
The thing is that post-dispatch logging is showing empty response headers even though I got a response from the gateway because the rate limit accepts the request.
I updated this in the repository so you can check what I mean.
Also if instead of trying to retrieve response headers I try to retrieve status code (also valid to note if the request was accepted or rejected by the rate limiter) I always get 200 OK even if the rateLimiter rejects the request and return 429 TOO-MANY-REQUESTS.

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.

Closing due to lack of requested feedback. If you would like us to look at this issue, please provide the requested information and we will re-open the issue.


Support for Autolinking

Is your feature request related to a problem? Please describe.
I have a build error:
error React Native CLI uses autolinking for native dependencies, but the following modules are linked manually:
 -  react-native-cardview (to unlink run: "react-native unlink react-native-cardview")
Describe the solution you'd like
Update to support autolinking in react-native 0.60+

This seems to be working after unlinked. Closing this.


Memory limit during sdm.spatialdm_local

Hello, I'm using stereo-seq with lower bin size, and so I've got a few ten thousands of pixels, when I try to run spatialdm_local it tells me my memory is not enough. I want to ask if I can reduce the n_permutation to save memory? What would be an ok level of n_perm if I don't set it to 1000? How would that affect my result? Also, if there any other ways to get around this memory limit...?
Thank you!

Thanks Joy for raising the issue. @jackywangtj66 is working on more tutorials with the z-score and API functions for both the spatial weight matrix and Moran's R. Please keep tuned.


Would it be possible to use the built-in python module 'getpass' as a somewhat OS agnostic way to get the username?

As opposed to 'pwd' which is Unix specific and/or needing to litter the Python code with conditional statements about the OS.

As for the test for Admin or root, I haven't found an OS agnostic way yet. The closest I could find was this answer from grigoryvp on this question on Stack Overflow: https://stackoverflow.com/questions/1026431/cross-platform-way-to-check-admin-rights-in-a-python-script-under-windows
I'm still looking.

Well, the purpose of checking if binwalk is running as root/administrator, why not theow away the idea that this should be done based on a username and instead use pseido-ish code like:
def rootdir():
    if windows: return “c:\”
    return “/“
if can_write(os.path.join(rootdir(), “testfile”)):
    raise RuntimeError(“too dangerous”)

Checking for the username “Administrator” or “root” is not really a sufficient check to determine safety anyway
I’m not much of a Windows user, but I can speak to Linux and say that there are ways to allow any user to bypass filesystem access controls so (e.g. CAP_* capabilities that override kernel filesystem permission checking logic)
Having logic like to actually test if the dangerous behavior is permitted with a real test can simplify the check and make it more comprehensive on all of Windows, Linux and MacOS.


Create analysis scripts

This PR closes #30 by creating .py files of functions that will produce the analysis we've been using, and demonstrating them with the baseline once-through scenario.
This PR does not update the other analysis notebooks in the repo, a separate issue has been created to make sure this does not fall to the wayside. Issue

Hello @nsryan2! Thanks for opening this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file scripts/deployment.py:

Line 11:80: E501 line too long (82 > 79 characters)
Line 28:80: E501 line too long (85 > 79 characters)
Line 40:80: E501 line too long (111 > 79 characters)
Line 46:80: E501 line too long (88 > 79 characters)
Line 53:80: E501 line too long (89 > 79 characters)
Line 65:80: E501 line too long (96 > 79 characters)
Line 67:35: W292 no newline at end of file

In the file scripts/fuel_transactions.py:

Line 12:80: E501 line too long (129 > 79 characters)
Line 27:80: E501 line too long (129 > 79 characters)
Line 39:1: W293 blank line contains whitespace
Line 42:80: E501 line too long (80 > 79 characters)
Line 43:80: E501 line too long (80 > 79 characters)

In the file scripts/waste.py:

Line 10:22: E128 continuation line under-indented for visual indent
Line 39:1: E302 expected 2 blank lines, found 1
Line 89:1: E302 expected 2 blank lines, found 1
Line 98:45: W292 no newline at end of file


luci-app-firewall: Fix typo in forwards redirect

When creating a forwarding rule with protocol set to other, a user is
forwarded to the configuration page. The URL for the configuration page
contained a typo - the user was forwarded to
admin/network/firewall/redirect/cfg... and not
admin/network/firewall/forwards/cfg..., leading to a 404.
Signed-off-by: Kristian Evensen<EMAIL_ADDRESS>

Applied in d3aa12b8e - thanks!


Alter regex to make it compatible with API

Sort the API output

Kp index is now published with 2 decimal numbers in NOAA's report, REGEX was changed to match it. Returned Kp is still integer (JSON string) to not break the API;

Regex changed to ignore the geomagnetic scale, which comes after the Kp index inside parenthesis, for example: (G3) (link to the geomagnetic scale: https://www.swpc.noaa.gov/noaa-scales-explanation)

NOAA's report example:
:Product: 3-Day Forecast
:Issued: 2023 Apr 24 1230 UTC
A. NOAA Geomagnetic Activity Observation and Forecast

The greatest observed 3 hr Kp over the past 24 hours was 8 (NOAA Scale G4).
The greatest expected 3 hr Kp for Apr 24-Apr 26 2023 is 8.00 (NOAA Scale G4).

NOAA Kp index breakdown Apr 24-Apr 26 2023

             Apr 24       Apr 25       Apr 26
00-03UT       7.00 (G3)    5.33 (G1)    2.33
03-06UT       8.00 (G4)    4.33         2.67
06-09UT       6.67 (G3)    4.00         2.33
09-12UT       6.33 (G2)    3.00         2.00
12-15UT       5.67 (G2)    3.00         1.33
15-18UT       4.67 (G1)    2.33         1.00
18-21UT       3.00         2.00         1.33
21-00UT       4.00         2.67         2.33

Rationale: The geomagnetic field is expected to be at G1-G2 (Minor-Moderate) levels through the rest of 24 Apr with G1 (Minor) levels likely on 25 Apr as CME effects slowly diminish.

B. NOAA Solar Radiation Activity Observation and Forecast

Solar radiation, as observed by NOAA GOES-16 over the past 24 hours, was above S-scale storm level thresholds.

Solar Radiation Storm Forecast for Apr 24-Apr 26 2023

              Apr 24  Apr 25  Apr 26
S1 or greater    1%      1%      1%

Rationale: No S1 (Minor) or greater solar radiation storms are expected. No significant active region activity favorable for radiation storm production is forecast.

C. NOAA Radio Blackout Activity and Forecast

No radio blackouts were observed over the past 24 hours.

Radio Blackout Forecast for Apr 24-Apr 26 2023

              Apr 24        Apr 25        Apr 26
R1-R2           25%           25%           10%
R3 or greater    1%            1%            1%

Rationale: There is a chance for R1-R2 (Minor-Moderate) radio blackouts on 24-25 Apr.

Sort the API output
Just sort by date to easy reading.

Thank you @alinebellozo!!


unable LinkingIOS: undeclared identifier 'RCTLinkingManager'

/AppDelegate.m: Use of undeclared identifier 'RCTLinkingManager'

What's your React-Native version?

Sorry, I forgot to import it

Never mind.

I import it but it is showing 'RCTLinkingManager.h' file not found

I was getting RCTLinkingManager.h' file not found as well, but this solved for me:
#import <React/RCTLinkingManager.h>

Kindly make sure that you place the
`#import <React/RCTLinkingManager.h>`

in the Appdelegate.m file above the
 #ifdef FB_SONARKIT_ENABLED
it worked for us.

alreadt have it does not work for me.


Restrict tag names

Fixes #162

Coverage decreased (-0.4%) to 45.943% when pulling adc7772cf8672da350fefffd2479da6315cfcc66 on restrict-tag-names into 9c3bc0579092cb4b7ddea3e5dc007aab4878a16d on master.

Coverage increased (+0.4%) to 46.743% when pulling ba1e98af67572d736a77c4cb287e4e27caafcfff on restrict-tag-names into 9c3bc0579092cb4b7ddea3e5dc007aab4878a16d on master.

Coverage increased (+0.7%) to 47.045% when pulling 4e26a72b3abaa19cac89f1fd001dfcbd62a0f8a3 on restrict-tag-names into 9c3bc0579092cb4b7ddea3e5dc007aab4878a16d on master.


[install_mac_os] Enable systemwide LaunchDaemon installation of agent

What does this PR do?
Implements functionality to install the agent as systemwide LaunchDaemon instead of a user LaunchAgent. Some notes on the solution:

The systemwide installation is triggered by specifying DD_SYSTEMDAEMON_INSTALL=true.
It requires specifying the user+group under which the agent will run, e.g. DD_SYSTEMDAEMON_USER_GROUP=123:345 or DD_SYSTEMDAEMON_USER_GROUP=username:groupname.
When agent is install as systemwide LaunchDaemon, the script makes it impossible to also install it as user LaunchAgent.
The system tray app doesn't start when agent is set up as LaunchDaemon, because it wouldn't work with it.

Motivation
This makes it possible to set up the agent to start on system startup instead when a specific user logs in.
Additional Notes

Possible Drawbacks / Trade-offs

Describe how to test/QA your changes

With a clean system (no existing agent installation), try installing the agent and verify it's correctly submitting metrics:

Without specifying DD_SYSTEMDAEMON_INSTALL - should install the agent as LaunchAgent for the user, open a dialog asking for permissions and then launch the systemtray app.
Test the following scenario both locally and also through SSH on a user account that's not logged in via GUI:

While specifying DD_SYSTEMDAEMON_INSTALL=true (and DD_SYSTEMDAEMON_USER_GROUP) - should install the agent as LaunchDaemon, not open any dialog and not launch the systemtray app. (If testing this on a system that previously had agent installed, you need to run sudo tccutil reset AppleEvents com.datadoghq.agent before uninstalling the previous agent to ensure the permissions get removed and we actually properly test the agent doesn't ask again when installing systemwide).

With a system that already has agent installed as user LaunchAgent:

Without specifying DD_SYSTEMDAEMON_INSTALL - should reinstall the agent.
While specifying DD_SYSTEMDAEMON_INSTALL=true (and DD_SYSTEMDAEMON_USER_GROUP) - should reinstall the agent as systemwide LaunchDaemon.

With a system that already has agent installed as systemwide LaunchDaemon:

Without specifying DD_SYSTEMDAEMON_INSTALL - should fail, saying it's not possible to install the agent as user LaunchAgent when systemwide installation is present.
While specifying DD_SYSTEMDAEMON_INSTALL=true (and DD_SYSTEMDAEMON_USER_GROUP) - should reinstall the agent (and keep it running as systemwide LaunchDaemon).

Reviewer's Checklist

[ ] If known, an appropriate milestone has been selected; otherwise the Triage milestone is set.
[ ] The appropriate team/.. label has been applied, if known.
[ ] Use the major_change label if your change either has a major impact on the code base, is impacting multiple teams or is changing important well-established internals of the Agent. This label will be use during QA to make sure each team pay extra attention to the changed behavior. For any customer facing change use a releasenote.
[ ] A release note has been added or the changelog/no-changelog label has been applied.
[ ] Changed code has automated tests for its functionality.
[ ] Adequate QA/testing plan information is provided if the qa/skip-qa label is not applied.
[ ] If applicable, docs team has been notified or an issue has been opened on the documentation repo.
[ ] If applicable, the need-change/operator and need-change/helm labels have been applied.
[ ] If applicable, the config template has been updated.

Note to myself before merging: it seems that UID/GID don't work (anymore?). We'll have to only accept UserName/GroupName for the plist file (or transform UID/GID to UserName/GroupName ourselves).


[presenter] no way to start a presentation

currently no apparent way to start a presentation in staging (0.4.8 candidate) - presenter plugin is on but no "present item" action on a stack

Closed by #6052


GRPC Package Name

Hello all.
First of all, thank you for your wonderful work on this project!
I have a quick question regarding the package name of the generated protobuf specifications. I am running openapi2proto on the command line, and it appears that the package name is being generated from the 'title' under the 'info' message.
From the tests, it appears that one can configure this with fixtures programmatically. But, is there a way I can configure this via the command line?
Many thanks,
Prashanth

Howdy @forwardmeasure!
At this point in time, the tool is only set to use the title of the spec or, if provided, an x-global-option can be added to declare the package names for specific languages. An example is in the README: https://github.com/NYTimes/openapi2proto#global-options

Hey mate,
Thank you much for your prompt response. I actually took a different approach - I ended up writing a Swagger Code generator as an extension of the swagger-codegen project. As of now, I'm able to get a fairly complex OpeanAPI spec to produce a nice-looking (semantically correct :-) set of .protos and GRPC service definitions. But, it wouldn't have been possible if you hadn't put this project together, so thank you!
I shall be cleaning the code up and submitting a PR to the swagger repo soon!
Cheers,
Prashanth


Onboarding Request: AgeingError

Active Development
Yes
Move source code
No
Tool name
AgeingError
Tool abbreviation
NA
Author(s)
Andre E. Punt, Kelli F. Johnson, James T. Thorson, Ian G. Taylor, Paul Burch
Contributor(s)
Ian J. Stewart, Melissa A. Haltuch
Tool logo
No response
NOAA Internal
Yes
Maintainer Name
Ian Taylor
Maintainer Email
<EMAIL_ADDRESS>Background Text
The AgeingError package estimates uncertainty in age estimates double-reads of hard structures (e.g., otoliths) based on the method of Punt et al. (2008) that account for both ageing bias and imprecision. The output can be used in Stock Synthesis or other assessment platforms that incorporate ageing error.
Citation
Users should cite both the R package and the associated 2008 manuscript, both described at  https://pfmc-assessments.github.io/AgeingError/authors.html#citation
References
https://doi.org/10.1139/F08-111
Associated Tools
No response
User Organizations
https://ror.org/05r7z1k40
Online App Link
No response
Executable Link
No response
Website Link
https://pfmc-assessments.github.io/AgeingError/
Documentation Link
https://pfmc-assessments.github.io/AgeingError/articles/getting_started.html
Source Code Link
https://github.com/pfmc-assessments/AgeingError
Software Badges
No response
Installation instructions
In R, install from GitHub:
# using {remotes}
remotes::install_github("pfmc-assessments/AgeingError")
# using {pak}
pak::pak("pfmc-assessments/AgeingError")

Toolbox Drawers

[ ] Ecosystems
[X] Fish and Fisheries
[ ] Human Dimensions
[ ] Protected Species
[ ] General Modeling Tools
[ ] Utility Tools

Keywords

[ ] ADMB
[ ] Age Structured Model
[ ] Bayesian Modeling
[ ] Bycatch
[ ] C++
[ ] Calculator
[X] Data Exploration
[ ] Data Limited Method
[ ] Data Visualization
[ ] Decision Support Tool
[ ] Developer Utilities
[ ] Economics
[ ] Ecosystem
[ ] Ensemble Modeling
[ ] Fish Biology
[X] Fisheries
[ ] Gradient Solver
[ ] Guided User Interface
[ ] Individual Based Model
[ ] Legacy Tool
[ ] Length Structured Model
[ ] Management Strategy Evaluation
[ ] Marine Mammals
[ ] Maximum Likelihood Estimation
[ ] Model Builder
[ ] Model Comparision
[ ] Multispecies Model
[ ] Natural Mortality Model
[ ] NOAA Branding
[ ] North Pacific
[ ] Output Visualization
[ ] Projection Modeling
[X] R Package
[ ] Reference Points Model
[ ] Risk Assessment Tool
[ ] Salmon
[ ] Sampling Decision Tool
[ ] Shiny
[ ] Simulation
[ ] Size Spectra Model
[ ] Size Structured Model
[ ] Socioeconomics
[ ] Spatial Model
[ ] Species Distribution Model
[ ] Standardized Reporting
[ ] Stock Assessment Model
[ ] Stock Assessment Projections
[ ] Stock Assessment Tool
[ ] Surplus Production Model
[ ] Tag Data Model
[ ] Teaching Tool
[ ] Time Series
[X] TMB
[ ] User Utilities
[ ] Virtual Population Analysis
[ ] Visualization
[ ] Web App

Uses Github Releases
Yes
Static version number
No response
Comments
Latest release is out of date so new github release will be soon (after resolving a few lingering issues).
Code of Conduct

[X] I agree to follow the FIT's Code of Conduct

Automatically generated JSON for the onboarding request:
{
  "active_development": true,
  "tool_name": "AgeingError",
  "tool_abbreviation": "NA",
  "authors": "Andre E. Punt, Kelli F. Johnson, James T. Thorson, Ian G. Taylor, Paul Burch",
  "contributors": "Ian J. Stewart, Melissa A. Haltuch",
  "noaa_internal": true,
  "maintainer_name": "Ian Taylor",
  "maintainer_email"<EMAIL_ADDRESS>  "background_text": "The AgeingError package estimates uncertainty in age estimates double-reads of hard structures (e.g., otoliths) based on the method of Punt et al. (2008) that account for both ageing bias and imprecision. The output can be used in Stock Synthesis or other assessment platforms that incorporate ageing error.",
  "citation": "Users should cite both the R package and the associated 2008 manuscript, both described at  https://pfmc-assessments.github.io/AgeingError/authors.html#citation",
  "references": ["https://doi.org/10.1139/F08-111"],
  "user_organizations": ["https://ror.org/05r7z1k40"],
  "website_link": "https://pfmc-assessments.github.io/AgeingError/",
  "documentation_link": "https://pfmc-assessments.github.io/AgeingError/articles/getting_started.html",
  "source_code_link": "https://github.com/pfmc-assessments/AgeingError",
  "installation_instructions": "In R, install from GitHub:\n```\n# using {remotes}\nremotes::install_github(\"pfmc-assessments/AgeingError\")\n# using {pak}\npak::pak(\"pfmc-assessments/AgeingError\")\n```",
  "toolbox_drawers": ["Fish and Fisheries"],
  "keywords": ["Data Exploration", "Fisheries", "R Package", "TMB"],
  "uses_github_releases": true
}

Thanks @iantaylor-NOAA, how does this preview look?
I also just added AgeingError to the FIT R universe, so let me know if the install instructions should include how to install from R universe, too (benefits: precompiled packages don't require users to install the toolchain to build R packages locally, so provides "cran like" experience)

Looks good to me, thanks for dealing with this so quickly.
I'm happy to have R universe instructions included. I should add those to the AgeingError README and vignette as well, but I can just copy whatever you put into the FIT page.

Great, I added the R universe install instructions, too.
Here's the link to the R universe page: https://noaa-fisheries-integrated-toolbox.r-universe.dev/AgeingError
I'll post once more when the Ageing Error page has been pushed to production!

Now onboarded! https://noaa-fisheries-integrated-toolbox.github.io/AgeingError


Referencing/representing 'extra' varinat info as metadata within an annotation

One issue that has come up is how our model for a given VA type might enable presentation of information from other annotations types about the same variant - in cases where the annotation provider thinks it represents useful context/metadata for the consumer to have.
For example, in Javier's notes on the 'Molecular Consequence' VA type he suggests that the particular exon affected by a variant is important additional information for consumers of a Molecular Consequence annotation. This information represents 'extra' metadata about the variant - as it does not qualify/refine the meaning of the primary molecular consequence statement, and it is  not evidence information used to support the primary statement.
How can we ensure our model is able to support provision of such extra metadata in a consistent way?  A few initial/naïve ideas (framed around this particular example):

Allow nesting of the related 'Affected Feature' annotation object about impacted exons within the Molecular Consequence annotation
Provide a reference/pointer to the Affected Feature object about impacted exons that lives within the same message/document.
Create a new short hand representation that captures the main statement of an Affected Feature annotation and can succinctly be packaged as metadata in the Molecular Consequence annotation?

For the item 3 above, should we generally consider providing direct/simple ways to capture some of the types of info that we also provide VA types for? This would be used in cases where we simply want to capture this info as an attribute of a variant to provide metadata (or a qualifier) relevant to the primary statement made in the context of another annotation.
We could consider mirroring each 'VA type' with an attribute that can be used to capture the same type of info as metadata about a variant in cases where this is all that is required (i.e. when we don’t care about  tracking the evidence/provenance for  this info as a primary statement in an annotation)
Here, we could use these attributes in the variant object, or in a separate 'metadata' object within the target annotation that holds metadata about the subject or descriptor separately from the objects representing the subject and descriptor themselves. (e.g. a 'subject metadata' object/element, and a 'descriptor metadata' object/element).


fix /undefine token and add test for indent level

Changes
Fixes #310
Checklist

[x] have tested my change
[n/a?] updated relevant documentation
[x] Remove any/all console.logs I added
[ ] eslint is not complaining
[ ] have added myself to the contributors' list in the README
[ ] for feature PRs: PR only includes one feature enhancement.

PS here's a link to the documentation - https://www.ibm.com/docs/en/i/7.5?topic=concepts-compiler-directives

Thanks for your PR, @brandonp42!


Potential infinite re-rendering when using unstable-identity selector

What version of React, ReactDOM/React Native, Redux, and React Redux are you using?

React: v18.2.0
ReactDOM:  v18.2.0
Redux: @reduxjs/toolkit v1.9.3
React Redux: v8.0.5

What is the current behavior?
I encountered an infinite React re-rendering loop bug when upgrading an existing code repo from react-redux v7 -> v8
Reproduction sandbox: https://codesandbox.io/s/redux-v8-issue-vr25b1?file=/src/features/counter/Counter.js
Code sample:
  const [boolValue, setBoolValue] = useState(false);

  const _count = useSelector((state) => {
    // NOTE: we return new array here
    return [];
  });

  useEffect(() => {
    // NOTE: we always set to `true` here
    setBoolValue(true);
  }, [_count]); // effect depends on an unstable-idenity value `_count`

The above code sandbox has a few notable points:

React version v18
the selector callback function used in useSelector(callback) does NOT have a stable-identity (Object.is equals)
the selector callback returns an selectedValue object which does NOT have stable-identity
there is an useEffect() relying on the above selectedValue in its dependencies list
the component's state should not change after 2nd render because setBoolValue(true); always

In the code sandbox, we observe that the component keeps re-rendering itself.
Without specific short-circuit code:
    // Stop infinite loop after certain render count
    if (window.renderCount >= STOP_RENDER) return;

the component will re-render infinitely.
What is the expected behavior?
I am not sure if this is a bug / new expected behaviour in react-redux v8
Granted the existing old code was not ideal:

the selector callback function used in useSelector(callback) does NOT have stable-identity (Object.is equals)
the selector callback returns an selectedValue object which does NOT have stable-identity

However, when reading the code, we do not expect the component to re-render infinitely:

eventually, the component will settle because:

there are no new update from the state store
the component's state will stop changing because of setBoolValue(true);  always

the code was working in previous react-redux v7

I am not too familiar with internal of react-redux, could this behaviour is a by product of this useSyncExternalStore behaviour ?
I tested a few things using the above codesandbox, and these help preventing the infinite loop:

downgrade React to v17
or downgrade react-redux to v7
or ensure the selector callback function has stable-identity
or ensure the returned selectedValue object has stable-identity

Which browser and OS are affected by this issue?
Chrome Version 114.0.5735.45 (Official Build) beta (x86_64)
Did this work in previous versions of React Redux?

[x] Yes

useSelector, useEffect, and setState() have known defined behaviors based on object references.  useSelector re-renders the component if you return a new reference.  useEffect executes if a value in the deps array is a new reference.  setState() forces a re-render if you pass in a new reference as the value.
Note that we do explicitly describe useSelector's behavior in the docs and tell you not to write selectors that always return a new reference:

https://react-redux.js.org/api/hooks#equality-comparisons-and-updates

Now, given the example code you showed, I would not expect that component to keep re-rendering.  React should see that you're passing in a value that is the same as the existing state value, and bail out:

https://blog.isquaredsoftware.com/2020/05/blogged-answers-a-mostly-complete-guide-to-react-rendering-behavior/#immutability-and-rerendering

So I don't immediately know why this component keeps re-rendering multiple times.
But in general: both React and React-Redux expect that you only create new references when values actually change, and rendering behavior does rely on you following those rules correctly.


Training on Arabic language

Hello,
Is there any document or guide on how to train on Arabic ?
Is this possible ? if yes what are the requirements ?
Thanks in advance,

Hello @lecidhugo  !
You can create the resources for a new language with https://github.com/kermitt2/grisp
The readme describes the process. It's an Hadoop process that is going to take a few hours.
Once done, you can start an environment for Arabic with entity-fishing, the knowledge base will be automatically build. Then you need to train a ranker and a selector model as described here -> https://nerd.readthedocs.io/en/latest/train.html#training-with-wikipedia
Loading the markupFull is the DB that is time consuming, it stores all the article text content.
You don't need to create embeddings if I remember well, it should work without them. However it improves a bit the disambiguation. This is also quite time consuming (it should be half day for Arabic given the number of articles).
There are 1,080,907 articles in Arabic, so it's a pretty big number, it should be doable and provide decent results.

Thank you very much for your kind reply!

Note that Arabic is now supported by default, with already trained models and KB resources available, see the documentation.


Sync client stops process exit when running in a Thread if not explicitly closed

Environment
Python: 3.11.4 (though the version doesn't seem to make a difference)
Websockets: 12.0
OS: Ubuntu in WSL2 (though this also happens in MacOS on my work environment)
Description
When running a sync websocket inside a Thread that starts itself, the program is blocked from exiting at the normal exit point if the websocket is not explicitly closed. This seems to be due to a race condition on a lock acquisition.
Recreation Code
This simple program will demonstrate the issue:
from threading import Thread
from websockets.sync import client

class ThreadedClient(Thread):
    def __init__(self):
        super().__init__()
        self.daemon = True
        self.ws = client.connect("wss://echo.websocket.org")
        self.start()

    def run(self):
        for msg in self.ws:
            print(msg)

ws_client = ThreadedClient()

print("This will be printed, proving we don't block on Thread creation...")

# We should exit immediately here as we did not call a blocking method on the thread
# In fact, let's call an explicit exit to prove the point:

exit()

Running this, the program never exits. In fact, if we send it a KeyboardInterruption, then you can see that the stack trace seems to be suggesting that the standard thread library is blocked obtaining a thread lock on exit, for some reason:
Traceback (most recent call last):
  File "/home/simon/.pyenv/versions/3.11.4/lib/python3.11/threading.py", line 1583, in _shutdown
    lock.acquire()
KeyboardInterrupt:

More strangely, running the same code, but with an async client inside an asycio event loop, works fine and exits cleanly (which is actually the workaround I have employed for this issue in my own code for now).
If we explicitly close the socket before exit, then the program exits cleanly. However, this is not a great requirement for a pure background thread that only communicates with the caller via callbacks. Having the caller having to hold a reference to the thread and then explicitly call a close method before exiting is not very ergonomic. Therefore, I am hoping that if the websocket self-closed on __del__ then this would solve the problem with blocking the exit.

Interestingly, rewriting the above code as below does work as expected. So it only seems to be an issue when you run a websocket inside a class that is a subclass of Thread, rather than in a Thread itself, necessarily:
from websockets.sync import client
from threading import Thread

def ws():
    ws = client.connect("wss://echo.websocket.org")

    for msg in ws:
        print(msg)

t = Thread(target=ws)
t.daemon = True
t.start()

print("Thread started. Now trying to exit")

exit()

Actually, with a few more tests this morning, I narrowed down the issue a bit more. Seems to be if the client is passed into a Thread that runs the read loop, then that prevents process exit, even if the Thread is a daemon. So, taking the code above that exits successfully, if we simply move the instantiation of the client and pass it into the Thread then the process now cannot exit as long as the client connection is open:
from websockets.sync import client
from threading import Thread

def ws(sock):
    for msg in sock:
        print(msg)

c = client.connect("wss://echo.websocket.org")
t = Thread(target=ws, args=(c,))
t.daemon = True
t.start()

print("Thread started. Now trying to exit")

exit()
# Never exits

This is a pain as it means that we cannot have a continuous read in a daemon thread, whilst keeping the reference to the client so we can asyncronously send. Unless, as I have done in my own code to get around this, we run an async client in a daemon thread (which the documentation states is not supported).

I was looking into this and the reason this is happening is because the client.connect() call constructs the ClientConnection object, which starts the recv_events_thread. It does so without specifying the daemon parameter to the thread creation: https://github.com/python-websockets/websockets/blob/main/src/websockets/sync/connection.py#L86
This means this thread inherits the daemon status from its parent. So if you want the WebSocket connection's "background" thread to truly be a daemon thread, it must be created from within a thread that is itself a daemon – and the main thread of the Python program is not.
This certainly would be helpful to have noted in the docs of the sync client.

Perhaps the background thread should be a daemon thread? If the main thread as well as any other thread managed by the user of the library is done, there's no reason to prevent the program from exiting.

we run an async client in a daemon thread (which the documentation states is not supported).

It works. I'm discouraging it because:

there's little benefit to running an event loop just for one client connection;
many users of websockets don't have a clear mental model of what an event loop is and, as a consequence, struggle to start it in the right thread and use call_soon_threadsafe properly.

@mpetazzoni I just released version 13.0 which includes this change.
(Yes this library has an irregular release schedule. It's a side project.)


Emit multiple data chunks from output Streams (was: Proper streaming)

It would be very cool if sharp could support proper streaming. As it is now, it will always emit exactly one data event which is the whole image. It kind of defeats the purposes of streams.
As I see it there is two ways to implement this.

The hard, but robust way.
Every time that libvips have data that should be written, it should call a function that we have defined. We then use this data to generate a data-event on the stream. It would be really cool if libvips could provide a state-machine that has a synchronous function that gives the next chunk. That way pause and unpause would work perfectly.

The quick and easy fix.
We can also create a named pipe and tell libvips to write it to that file via vips_image_write_to_file. To the user we would then return a ReadStream as returned by fs.createReadStream(path).

If you've not already done so, have a look at the Stream discussion in #30.
At the time, @jcupitt created a load-from-stream branch with some initial ideas, mainly around the creation of a VipsInputStream/VipsOutputStream.
On a related note, I spotted this recent change in io.js that looks like it could make handing JavaScript Stream objects in C++ easier.

My final thought back then was in this comment: https://github.com/lovell/sharp/issues/30#issuecomment-46960443
Thinking about it again, it might be possible to do a hybrid of buffering and streaming. You could read and store data off the input until there was enough there that you could get all the metadata you need to be able to decide how to handle the file.
Once you've picked an approach, close the image and start again. This time you feed the reader the stored data from the first pass and only start reading off the input again when that runs out. At that point (presumably) you are decoding actual pixels.
There are still some awful problems. What if you send 10 jpg images down a pipe to a resizer, but image 4 is corrupt. How will the reader recover from the error?

I'm sorry, I missed out the very vital part that is that I'm talking about output, not input.
I'm not that interested in being able to pipe more than one image and for that I think it would be more reasonable to use object-streams (objectMode = true) for. Anyhow, that's another discussion.
My use case is to stream the output image data to S3 (or to any other service for that matter...).
My ideal usage would be like this:
var out = sharp(bufferObject).resize(1080, 1080).min().toStream();
S3.upload({ Body: out, Key: 'my-image.jpg' }, cb);

This could be done as a said in number 2 by letting libvips write it to a named pipe and then createReadStream() on that one. That should give me data events as the data is normally written to disk, e.g. when fwrite(...) is called in C.
Actually, even more usual is probably piping directly back to the browser on http request.
server.on('request', function (req, res) {
  Sharp(myBuffer).resize(1080, 1080).min().toStream().pipe(res);
});

Why I like to have .toStream is because I think it makes it more explicit on what I want to do. In my humble opinion that is better than letting the sharp object always be a stream, which sometimes emits data.

Ah OK, that makes sense. I agree, this would be an nice cleanup to libvips. The load-from-stream branch does this already, I'll fix it up a bit and add it.

@jcupitt So what's the resolution on this issue? It looks like the load-from-stream branch was closed.

I spent a bit of time looking at the branch, but master has moved and merging it would have taken quite a lot of work.
Another alternative would be GIO. It has a thing called GFile which does more or less the same thing as VipsInputStream / VipsOutputStream, but better: a POSIX-y IO API which can have memory, files, sockets, URIs, or whatever you like as the backend:
https://developer.gnome.org/gio/stable/GFile.html
We could rework the png/jpg/webp read and write operations to use GFile. We'd then be able to open images directly over http or ftp, for example:
vips copy ftp://a.b.c/d.jpg x.jpg

No idea if that would be useful.
On the other hand ... It would mean adding yet another required dependency. And VIpsInputStream / VipsOutputStream were small and easy to understand and extend.

@jcupitt great work on the streams branch, a shame it could never be merged. Just adding my support for any work on true streams, which would allow significantly less memory usage since right now sharp needs to load the source image into memory, and then hold the whole output buffer in memory, before writing out.

I'm trying to find a conclusion to this, did anyone succeed in getting sharp to write to a readableStream that could as suggested above be piped to a browser.

libvips 8.9 (due rsn) has true streaming:
https://libvips.github.io/libvips/2019/11/29/True-streaming-for-libvips.html
eg.
aws s3 cp s3://mybucket/input.jpg - | \
    vips thumbnail_stream [descriptor=0] .jpg 128 | \
    aws s3 cp - s3://mybucket/output.jpg

Obviously you could send to a browser instead.
It would need some (unknown?) amount of work to integrate this in sharp. I think Lovell is planning to look at this when 8.9 finally ships.

Yes, after there's a version of sharp that depends on the future libvips v8.9.0 we can look at the best way of handling its new Stream API.
A possible first implementation could involve accepting file descriptors as input/output, although that is likely to have to be via new API and not the standard, idiomatic Node.js Stream API (which will still be supported using the existing implementation).
A more complete solution might be to create custom libvips Stream objects and use these for passing data chunks to/from Node.js, but there are multi-threading considerations e.g. libvips will be running on a different thread to V8.
I'm planning to migrate the internals of sharp to use Node.js's newish N-API first - see #1282 - sadly I don't see anything in N-API for Node.js Stream objects, plus this migration will remove access to things like Node.js' StreamBase class.

Huh that sounds a little frustrating. Do let me know if there's anything I can do to help.

Thanks for the explanation - I'm not up enough on the details of the APIs to make intelligent comments on the choices made, but I think keeping a mind on the goal is important - i.e. to be able to interface with external (non-sharp) code.
In my case .... I had a Node stream with the image rather than a file, (being pulled from the net or a cache, or a P2P process) and needed to pass it back to the browser request.  There are probably other requirements for other people but I'm guessing these node streams are the common theme to most of them rather than file descriptors.
Also ... can I suggest a minor change to the documentation to explain how to work with streams, it only took me a few minutes to find that sharp claimed to support streams, it took me a couple of hours - searching through git issues etc - to figure out how to plug it into streams, and I almost gave up when this thread implied (to me) that it wasn't supported (rather than what it actually says, which is that its supported but with bad memory handling).
I'd suggest an edit to https://sharp.pixelplumbing.com/en/stable/api-constructor/  maybe one of the examples, best I could figure out was the following code.
# Where s is the stream; and res is a browser Response.
cost sharpie = sharp();
s.pipe(sharpie);
sharpie.extract({left,top,width,height})
    .pipe(res);

This works (subject to the memory issues) but wasn't intuitive to figure this out from the docs and I'm not at all sure its the right way to do it. The instinct was to look at "toFile", it took a while to realize that wasnt the right way.

@mitra42 Glad you got it working; is this for use with the Internet Archive?
The second example in https://sharp.pixelplumbing.com/en/stable/api-constructor/#examples which is generated from https://github.com/lovell/sharp/blob/master/lib/constructor.js#L53 demonstrates Stream-based I/O and I'm always happy to accept PRs with more examples and improvements to the JSDocs if you're able.

@lovell Oops - sorry I missed that example - not sure how.
Yes -this is for the Internet Archive - for the Offline Archive, a version of the archive running in devices as small as Raspberry Pi's in particular to support interaction between that box and a local copy of mediawiki running an extension built by David Kamholz for transcribing palm leaf books, it was using IIIF on the archive to zoom in on the full scan, and I needed to port that support to Javascript.  I tried using IIIF via @jonallo 's  iiif-image-server-node but the installer for that is broken (it fails to install sharp) and once I'd got sharp installed separately I realized sharp could probably be used better on its own than going through iiif-image-server-node > iiif-image

Yes, after there's a version of sharp that depends on the future libvips v8.9.0 we can look at the best way of handling its new Stream API.
A possible first implementation could involve accepting file descriptors as input/output, although that is likely to have to be via new API and not the standard, idiomatic Node.js Stream API (which will still be supported using the existing implementation).
A more complete solution might be to create custom libvips Stream objects and use these for passing data chunks to/from Node.js, but there are multi-threading considerations e.g. libvips will be running on a different thread to V8.
I'm planning to migrate the internals of sharp to use Node.js's newish N-API first - see #1282 - sadly I don't see anything in N-API for Node.js Stream objects, plus this migration will remove access to things like Node.js' StreamBase class.

@lovell is this still on the roadmap given sharp is now using libvips 8.9?

Do I understand correctly? The underlying cli command vips does supports streams and won't load entire file to start processing. But the Sharp library can't do it? So in Node.js I have to do all in Linux Shell land to achieve it? Maybe I could run vips shell command as child_process.spawn and connect the child process to Node streams via its stdin/stdout?

As https://github.com/libvips/libvips/pull/1443 has been merged is there any progress to make streams fully work in sharp as well? 😇

This improvement will greatly enhance the end user experience.
In our current process, we download the source image and pass it to Sharp for resizing. Then, we send the resized image to an S3 bucket and simultaneously provide it as a chunked response. However, there is currently a delay because Sharp only emits one large chunk. By implementing proper stream emission, the final image will be delivered to end users more quickly.

I'm currently working with large images, and true streaming would be ideal. I’ll start working on this unless someone else is already on it.

I got streaming working, unfortunately I run into an issue that will require dropping Napi::AsyncWorker since code running in the Execute method uses libuv worker threads which cause the following code to deadlock because fs also uses libuv threads.
it('concurrency', async () => {
    const concurrency = 10; // any number equal or greater to UV_THREADPOOL_SIZE
    const promises = Array(concurrency).fill().map(async() => {
      const file = fixtures.inputJpg320x240;
      const input = fs.createReadStream(file);
      const stream = sharp().rotate();

      input.pipe(stream);
      return buffer(stream);
    });
    const result = await Promise.all(promises);
    // ...
  });

Given that I rarely code in C++ and need to become more familiar with N-API, it might take me a bit.


fix(django): Delay patching of DRF

Fix #397

BTW I think we can soonish remove support for Django 1.6. Need to check stats in snuba.


How to use the precarious_dataset & synthetic_dataset datasets to train model?

@huangshiyu13 Hi, thank you for sharing the code. I have dowloaded the two datasets. I can train the model with the following data path settings:
imageLoadDir = Dataset/precarious_dataset/train/
anoLoadDir = precarious_dataset/trainAno/.
If I also want to use the synthetic_dataset dataset, how should I set the data path?

I think one way to solve this problem is that you merge these two datasets by hand.


Is it possible to calculate anchor automatically without sides setting?

Is it possible to calculate anchor automatically without sides setting?
Thank you in advance!
Since sometimes it is hard to specify the value of sides.

I was thinking about adding a shell script which calls this functions with sane parameters.
Usually, the --sides parameters is needed because we don't know the strides the user might have in their network, so we need a way to filter the bounding boxes according to their size, and that is network architecture dependent.
I usually call the program like this:
./compute_anchors --size 640 \
    --clusters 3 --sides 8 48  \   # for 8×8 stride
    --clusters 3 --sides 32 192 \  # for 16×16 stride
    --clusters 3 --sides 128 768 \ # for 32×32 stride, basically, the rest
    --dataset /path/to/dataset.xml

It doesn't matter in which order you put them, it will sort them and use the first appearance of clusters with the first appearance of sides and so on.
Note that, in the way the program is written right now, it doesn't use the minimum value for the sides option... I think we don't need it, but I did when I was writting the program, however it is not used.
Maybe a better option would be to just have a strides option, and we should automatically group the bounding boxes that make sense for that stride...
Honestly, I implemented this pretty quickly, so I didn't do a lot of research on how to do it properly. I am open to suggestions.

For the COCO 2017 train dataset, that command outputs:
group #0: 8 - 48
group #1: 32 - 192
group #2: 128 - 768
total number of boxes: 860001
Computing anchors for 272821 samples
  {{13, 14}, {18, 35}, {36, 28}}
  Sample Fraction: 0.317233
  Average IoU:     0.586747
Computing anchors for 377191 samples
  {{54, 58}, {70, 137}, {141, 102}}
  Sample Fraction: 0.438594
  Average IoU:     0.566556
Computing anchors for 209989 samples
  {{293, 156}, {159, 302}, {498, 376}}
  Sample Fraction: 0.244173
  Average IoU:     0.589815

Total Coverage: 1

Which are a bit different from the ones used in YOLOv4 and YOLOv7, but still they seem reasonable:
anchors:
  - [12,16, 19,36, 40,28]  # P3/8
  - [36,75, 76,55, 72,146]  # P4/16
  - [142,110, 192,243, 459,401]  # P5/32

You could also call it like this:
./compute_anchors --size 640 --clusters 9 --sides 8 768 --dataset data/coco_train2017.xml

That is, put all boxes in the same group and find 9 clusters automatically, that would output:
group #0: 8 - 768
total number of boxes: 860001
Computing anchors for 860001 samples
  {{26, 30}, {77, 81}, {98, 184}, {216, 126}, {164, 327}, {334, 271}, {526, 172}, {322, 496}, {576, 415}}
  Sample Fraction: 1
  Average IoU:     0.525404

Total Coverage: 1

And that's basically what the YOLO example program does. However, the problem is that now you don't know which anchors to put in which stride. I would say that {98, 184} is too large for an 8×8 stride...

Sincere thanks for your kind help! I got it.


Documentation suggestion

Moved common parameter help documentation to a single location. This eliminates dozens of help files that otherwise need to be independently kept in sync.
Leveraged stapler's include function to render common parameters like cluster URL/time/verbose, etc. This should improve consistency and decrease boilerplace / maintenance throughout.
Found and fixed a few missing symlinks in the DSL config.jellys.

@gabemontero
fwiw, I think this pattern could also help @oatmealraisin . The time control needs only to be implemented once in verbose-wait.jelly. The cluster test button would only need to be implemented once in cluster.jelly.

Looks good!  Thanks!!!
I'll merge following the test runs.

:tada:

@gabemontero Forgot to mention that I changed the authkey to a password field. Just wanted to make sure you were ok with that before  a merge.

Nope, can't do that (migration issues).  Got to switch it back to the old
value.
On Thu, Oct 6, 2016 at 6:34 PM, Justin Pierce<EMAIL_ADDRESS>wrote:

@gabemontero https://github.com/gabemontero Forgot to mention that I
changed the authkey to a password field. Just wanted to make sure you were
ok with that before a merge.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/openshift/jenkins-plugin/pull/77#issuecomment-252107684,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADbadF2HU8C6zytG4idf3zsJB6gp6Dq1ks5qxXeCgaJpZM4KQZFT
.

Sorry, misuderstood.  You didn't change the field name.  You changed the jelly field type.  My bad - that is fine.