[Feature Request] Selectable date range to render

Existing Component
Yes
Component Name
DatePicker
Description
The DatePicker component renders dates years from now and backward, with the ability to disable certain dates.
I would like to be able to select a date range for the DatePicker to render and only be able to navigate to those dates.
Example: 30 September 2018 - 30 December 2018
Only the dates within that range will be rendered on the DatePicker, not allowing users to even see additional dates (as opposed to disabling them)
The reason for this feature request is that running a disabled check with the "disabledDate" picker options can become slow with computationally expensive operations, as the operations are performed on multiple years worth of days, instead of say, 4 months worth of days.

How expensive, do you have measurements?
I don't see why 4 months of days (or 10 years) will be a problem on popular devices.

Well. Then you need rethink about the disabledDate implementation.
My recommendation is: do not perform complex computations in disabledDate, do them beforehand.
From my experience, a correctly implemented disabledDate function called over 10 year of dates is quite fast. The most preferred disabledDate implementation should only perform numerical comparisons, such as:
MIN_TIME = do_whatever_UTC_conversion(server_params)

disabledDate(date) { return MIN_TIME <= date.getTime() }

Also, a word of caution. Native JS date do not have timezone support, and element can cause edge cases during timezone shift (i.e. the moment when daylight saving is turned on/off).
I personally won't do string manipulations when dealing with timezones. Though, date-picker and JS date does come with a few guarantees / alternatives:

User always sees local time (browser's timezone)
When you get timestamp (getDate), it will be normalized to UTC unix epoch. See: MDN
You can use value-format to get string style date (which will be local time), and combine it with a time-zone selector.

I will reconsider the way I implement this and try to find a better solution with your recommendations.
We are aware of the time zone issues with Native JS, and are therefore using moment.js to deal with all date-time related properties. However I must be messing up somewhere, will look into it and leave a comment on the findings.
In regards to my final paragraph, what are your thoughts on the UX of showing many disabled fields vs showing only fields that are available? Would this be an option to consider adding to the component?

I am not entirely clear about your meaning of hiding disabled fields. You can make a screenshot to illustrate your point.
As for date picker, I don't think showing disabled dates cause problem to users.
Assume user can only choose dates in the past. Normally, they choose the available ones. Unless they are really boring and start navigating to future years.
Another reason for showing disabled dates is to maintain the shape of date table. I assume most users want to see something like a calendar. Hiding disabled dates breaks this mental model somehow.

My suggestion for the date picker to have the option to hide selected dates are as follows:

The shape of the table will not change at all, it will still look like a calendar.
As an example, user can select 5 dates in a row into the past, all others will be disabled.

With that being said, all the dates not currently in view (dates in other months) do nothing, hence I want to not render those extra days to boost performance a little and so that users cannot scroll to months where all dates are disabled.
Screenshot of this example

My suggestion would be to select a range of months that will be rendered, for example, August.
All months outside this range will not be rendered, thus the arrows in the black boxes I have drawn will not be visible either, as there is only one month selected.
Does this make sense? I can try to clarify more if needed.

I am unsure if my previous comment made it clear as to what I mean with hiding dates away from users that are unselectable.
Here is an example of how Vuetify's date picker performs the action that I was referring to, where the arrows are disabled because the dates outside of this month are all disabled.
Codepen example
I hope this clears up any confusion.

Any news on this?


Adding Local Storage

Added vuex-persist to handle persisting state
Store tooldata and page number to localstorage refactored store.ts to make it a bit simpler
Some data is now being transformed on mutation instead of as getters working on simplifying calculations
Moved Building file from store to saveload component
Moving tooldata and plaindata to state

Closes #29

Works perfectly, also supports switching languages.
Will need a "Start over" option to manually clear local storage.
Submitting new issue.


Proposal: Provide an additional TabView "skin" that is based on the WCT TabView

Proposal: Provide an additional TabView "skin" that is based on the WCT look

Summary

A lot of people coming from the Windows Community Toolkit TabView prefer the look of the WCT TabView compared to the WinUI TabView. To make the switch to the WinUI TabView easier, we should have a WinUI TabView style which is based on the look of the WCT TabView.
Rationale

Since the WCT TabView is deprecated now, and will be removed at some point, people would either need to move the WinUI TabView despite prefering the WCT look or fork the WCT TabView and maintain it themselves. By providing a style that uses the WinUI TabView and looks like the WCT TabView, we would allow people to use the style they prefer.
This is also something some people already asked about (to some degree):
Request to have the WinUI TabView styled the same as the WCT
Wish to have Acrylic and Reveal in WinUI TabView

Important Notes

Possible API:
<!-- WinUI Fluent TabView -->
<muxc:TabView />

<!-- WinUI TabView with look based on WCT-->
<muxc:TabView Style="{StaticResource BoxTabViewStyle}" />

Comparison
WinUI TabView

WCT TabView

Open Questions

Where should this new style be hosted? Inside WinUI since it is a WinUI control or maybe somewhere else?
What should the new "skin" be called?

@chingucoding could you provide pictures as well?

@chingucoding could you provide pictures as well?

Sure thing @StephenLPeters, I've updated the issue to include gifs and pictures showing both versions.

You say Skin, do you mean bundling a Style for the control in the WinUI toolkit that, when applied, will give it the WCT/Legacy Edge look?
Or do you see it as being a more fundamental property like TabVisual.WCT | TabVisual.Edge?

I would expect this be a named style as opposed to an API on TabView. @chigy Is there a precedent/pattern for providing multiple styles for the same control ?

I would expect this be a named style as opposed to an API on TabView. @chigy Is there a precedent/pattern for providing multiple styles for the same control ?

There is the AccentButton Style, as well as an Icon Button and Text Button

Yes the idea is to provide a separate style for the WinUI TabView (as shown in the small example API snippet).

Ok so ResourceDictionary then.

I think it'd literally be a Style, similar to AccentButtonStyle.

Posting this comment here as it may be related or relevant...
So as Edge allows for Tabs with PWA windows, we see a new style for the Tab bar, using a chosen colour.
Does TabView need to offer a style option to allow for an Accent colour to be used.

Terminal may benefit from any steps in this direction, as it hopes to add XAML themeing options for the user - or to allow for the user's choice of a coloured TitleBar to work with TabViews extending into the title bar area.

@mdtauk - thanks for the input on accent color, but I'm going to mark this particular thread as "off topic" since it isn't relevant to the proposal here about supporting a Windows Community Toolkit-style skin.
Note also that the TabView already does support accent color background as an app-provided choice, such as found in the Xaml Controls Gallery:

While investigating this, I noticed an issue for this that is a complete show stopper: there is no way to override the default TabViewItem style. Unlike some other controls, the TabViewItem does not follow the pattern of:
<Style TargetType="ControlName" BasedOn="DefaultControlNameStyle"/>

<Style x:Key="DefaultControlNameStyle" TargetType="ControlName">
...
</Style>

but rather just exposes the complete template as the default style leaving users with no choice of easily overriding the default style. However that is something that we would need for this feature to be easily usable. @ranjeshj @stmoy @StephenLPeters Is that something we can change to make that customization easier?

How about a tab like skin for Pivot? Behavior of Pivot is closer to TabControl in WPF.

The main point of this proposal is to make the migration from the WCT TabView to the WinUI easier for people who preferred the WCT look of TabView.

make the migration from the WCT TabView to the WinUI

I did migration from WCT TabView to WinUI Pivot - I do need the pivot-like behavior instead of TabView. I remember that TabView didn't satisfy me.

make the migration from the WCT TabView to the WinUI

I did migration from WCT TabView to WinUI Pivot - I do need the pivot-like behavior instead of TabView. I remember that TabView didn't satisfy me.

No reason to not keep using the Pivot then

If you need Pivot behavior, neither WCT TabView nor the WinUI TabView would be a good choice. WinUI TabView has replaced the WCT TabView and with feature parity of WinUI and WCT TabView, WCT TabView was deprecated. That's why I was proposing making that upgrade path easier.

While investigating this, I noticed an issue for this that is a complete show stopper: there is no way to override the default TabViewItem style. Unlike some other controls, the TabViewItem does not follow the pattern...

Yeah, I've noticed this in a few places, I just called it out on the new Expander PR. There's a lot of general improvements to the templates that should be done for this across multiple controls in WinUI. Probably deserves its own general issue.

How about a tab like skin for Pivot? Behavior of Pivot is closer to TabControl in WPF.

Pivot was very much meant for touch-first experiences, it behaves oddly if you have scrolling content within it. It's not really like the TabControl in WPF at all. We had gone down the path at first of customizing Pivot to implement TabView in the Toolkit before we realized it should be its own control.
The new SwitchPresenter we're adding in the Toolkit should make basic scenarios work well (as we did for the new ColorPicker) if you don't need the swipe behavior and need heavy customization of a few static items.


Comments missing in files

Perhaps the code is self-explanatory and comments are not needed.

Maybe comments should be added. What's the consensus on this?


Presentation: Do not initialize UpdatesTracker if IPC host is not available

In some backends iModels are getting modified even when IPC host is not available. Ensure that doesn't cause issues for PresentationManager.

@mergifyio backport release/2.17.x

@mergifyio backport release/2.16.x


⚠ Bofrost.fr has degraded performance

In b962879, Bofrost.fr (https://www.bofrost.fr) experienced degraded performance:

HTTP code: 200
Response time: 1574 ms

Resolved: Bofrost.fr performance has improved in 6903166.


Using dropdown with form elements causing the whole app to freeze

Hi, I have implemented the library in my project and when I use the dropdown alone, it works fine. But if I use the dropdown component with different form elements, it freezes the whole app. I have to kill the app and restart it to make it work again. This is always happening.
Bug:
https://github.com/hoaphantn7604/react-native-element-dropdown/assets/40450695/5540db81-f3cb-4a60-a1b5-5759be939ade
There is no infinite loop in the code as I checked the console.

Hi @iarmankhan did you find a solution to this?

Hi, I have implemented the library in my project and when I use the dropdown alone, it works fine. But if I use the dropdown component with different form elements, it freezes the whole app. I have to kill the app and restart it to make it work again. This is always happening.
Bug: https://github.com/hoaphantn7604/react-native-element-dropdown/assets/40450695/5540db81-f3cb-4a60-a1b5-5759be939ade
There is no infinite loop in the code as I checked the console.

you are using dropdown in the custom component it is freezing on data props because you are using like props.data
but we have to use like this.
export default function CustomDropDownPicker({ data}: any) {
<Dropdown
style={[styles.dropdown, dropDownCont]}
placeholderStyle={styles.placeholderStyle}
data={data}
}


CFFL

CFFL

Linter check found the following problems:
The following problems have been found:
ERROR: /tmp/abs_17pc5kwnwq/clone/recipe/meta.yaml:16: pip_install_args: pip install should be run with --no-deps.
ERROR: /tmp/abs_17pc5kwnwq/clone/recipe/meta.yaml:21: missing_wheel: For pypi packages, wheel should be present in the host section
ERROR: /tmp/abs_17pc5kwnwq/clone/recipe/meta.yaml:40: missing_documentation: The recipe is missing a doc_url or doc_source_url
ERROR: /tmp/abs_17pc5kwnwq/clone/recipe/meta.yaml:40: missing_license_family: The recipe is missing the about/license_family key.
WARNING: /tmp/abs_17pc5kwnwq/clone/recipe/meta.yaml:40: missing_description: The recipe is missing a description
Errors were found


Update pullImage() method for the mock manager (Tests may be fail…

…ing because of this)

@JonathanRosado run make test (pointing to rethinkdb container) to make sure that it does fix things.

@JonathanRosado thanks for testing. This is a go :)


Failed to call generateOrRecoverAess on the test net using BrowserProvider

Failed to call generateOrRecoverAess on the test net using BrowserProvider,.
My tx is https://testnet.cotiscan.io/transaction/0xa8a7f7fc903cc05d1f168d5fcb57bda40990ea49276c0b1c75faa4e3eaa64b27.
My code:
export async function setupAccount(address: string) {
const ethereumProvider = window.ethereum as Eip1193Provider;
const provider = new BrowserProvider(ethereumProvider);
const singer: JsonRpcSigner = await provider.getSigner(address);
await singer.generateOrRecoverAes()
return singer
}

@myishuohuo can you please add details of the error ? what's the failure ?

Please use the latest Devnet version (v0.1.1-b) in the meantime while we work on fixing the issue.

Thank you very much, the version I used before was Testnet version (v1.0.1-b) and this problem occurred, now I have tried Devnet version (v0.0.1-b) and it is OK.

@myishuohuo appreciate if you could confirm that issue is resolved so we can close this issue


webserver_1 | NameError: name '_mysql' is not defined

webserver_1 | NameError: name '_mysql' is not defined

webserver_1 | NameError: name '_mysql' is not defined

Fixed it. copied this from the other thread.
version: '3'
services:
postgres:
image: postgres:9.6
environment:
- POSTGRES_USER=airflow
- POSTGRES_PASSWORD=airflow
- POSTGRES_DB=airflow
ports:
- "5432:5432"
webserver:
image: puckel/docker-airflow:1.10.9
build:
context: https://github.com/puckel/docker-airflow.git#1.10.9
dockerfile: Dockerfile
args:
AIRFLOW_DEPS: gcp_api,s3
PYTHON_DEPS: sqlalchemy==1.3.0
restart: always
depends_on:
- postgres
environment:
- LOAD_EX=n
- EXECUTOR=Local
- FERNET_KEY=jsDPRErfv8Z_eVTnGfF8ywd19j4pyqE3NpdUBA_oRTo=
volumes:
- ./examples/intro-example/dags:/usr/local/airflow/dags
# Uncomment to include custom plugins
# - ./plugins:/usr/local/airflow/plugins
ports:
- "8080:8080"
command: webserver
healthcheck:
test: ["CMD-SHELL", "[ -f /usr/local/airflow/airflow-webserver.pid ]"]
interval: 30s
timeout: 30s
retries: 3

I'm still having this issue even though I have made the following updates:
image: puckel/docker-airflow:1.10.9
   PYTHON_DEPS: sqlalchemy==1.3.0

I'm also facing the same issue ...I have used the docker-compose-yml file in this thread and followed the instructions under "Getting Started" . Has anyone managed to solve it

?

Still not solved, I'm having the same problem.

I was having the same _mysql is not defined error as above so I updated the docker-compose.yml file as mentioned above. It fixed the mysql issue but another one came up...ImportError: cannot import name 'soft_unicode' from 'markupsafe', I noticed the python version running is now 3.7 instead of python3.6 from before the .yml file update. Did anyone encounter this as well and how did you fix the issue?
I have the following:
Docker version 20.10.12, build e91ed57
docker-compose version 1.29.2, build 5becea4c
Airflow 2.2.3

For fix this issue: please add:
PYTHON_DEPS: sqlalchemy==1.3.0 markupsafe==2.0.1 wtforms==2.2
Works for me.

I was having the same _mysql is not defined error as above so I updated the docker-compose.yml file as mentioned above. It fixed the mysql issue but another one came up...ImportError: cannot import name 'soft_unicode' from 'markupsafe', I noticed the python version running is now 3.7 instead of python3.6 from before the .yml file update. Did anyone encounter this as well and how did you fix the issue?
I have the following: Docker version 20.10.12, build e91ed57 docker-compose version 1.29.2, build 5becea4c Airflow 2.2.3

Fo fix this issue, please add:
PYTHON_DEPS: sqlalchemy==1.3.0 markupsafe==2.0.1 wtforms==2.2
Works for me.
All the best!

I can confirm this!
Be careful to change 1.10.1 to 1.10.9 at both positions
[...]
webserver:
    image: puckel/docker-airflow:1.10.9
    build:
      context: https://github.com/puckel/docker-airflow.git#1.10.9
      dockerfile: Dockerfile
      args:
        AIRFLOW_DEPS: gcp_api,s3
        PYTHON_DEPS: sqlalchemy==1.3.0 markupsafe==2.0.1 wtforms==2.2
[...]

Docker version 20.10.7, docker-compose version 1.29.2

Help! I'm still having a soft_unicode import error from markupsafe with Jinja2.
Did a downgrade to markupsafe 2.0.1, updated the yml file but still facing the same issue.

I am getting the same error:
ImportError: cannot import name 'soft_unicode' from 'markupsafe' (/usr/local/lib/python3.7/site-packages/markupsafe/init.py)
Traceback (most recent call last):
File "/usr/local/bin/airflow", line 25, in
from airflow.configuration import conf
File "/usr/local/lib/python3.7/site-packages/airflow/init.py", line 42, in
from airflow.models import DAG
File "/usr/local/lib/python3.7/site-packages/airflow/models/init.py", line 21, in
from airflow.models.baseoperator import BaseOperator, BaseOperatorLink  # noqa: F401
File "/usr/local/lib/python3.7/site-packages/airflow/models/baseoperator.py", line 36, in
import jinja2
File "/usr/local/lib/python3.7/site-packages/jinja2/init.py", line 33, in
from jinja2.environment import Environment, Template
File "/usr/local/lib/python3.7/site-packages/jinja2/environment.py", line 15, in
from jinja2 import nodes
File "/usr/local/lib/python3.7/site-packages/jinja2/nodes.py", line 19, in
from jinja2.utils import Markup
File "/usr/local/lib/python3.7/site-packages/jinja2/utils.py", line 642, in
from markupsafe import Markup, escape, soft_unicode
ImportError: cannot import name 'soft_unicode' from 'markupsafe' (/usr/local/lib/python3.7/site-packages/markupsafe/init.py)

Hi, thanks for the recommendations. I have combined two cases to solve this problem after reading your instructions, you can just follow them here.
If you encountered the _mysql problem, please don't forget to download apache-airflow using the keywords "pip install apache-airflow" or depending on the terminal you're using.
After that update the docker-compose.yml file from the cloned repository.
version: '3'
services:
postgres:
image: postgres:9.6
environment:
- POSTGRES_USER=airflow
- POSTGRES_PASSWORD=airflow
- POSTGRES_DB=airflow
ports:
- "5432:5432"
webserver:
image: puckel/docker-airflow:1.10.9
build:
context: https://github.com/puckel/docker-airflow.git#1.10.9
dockerfile: Dockerfile
args:
AIRFLOW_DEPS: gcp_api,s3
PYTHON_DEPS: sqlalchemy==1.3.0 markupsafe==2.0.1 wtforms==2.2
restart: always
depends_on:
- postgres
environment:
- LOAD_EX=n
- EXECUTOR=Local
- FERNET_KEY=jsDPRErfv8Z_eVTnGfF8ywd19j4pyqE3NpdUBA_oRTo=
volumes:
- ./examples/intro-example/dags:/usr/local/airflow/dags
# Uncomment to include custom plugins
# - ./plugins:/usr/local/airflow/plugins
ports:
- "8080:8080"
command: webserver
healthcheck:
test: ["CMD-SHELL", "[ -f /usr/local/airflow/airflow-webserver.pid ]"]
interval: 30s
timeout: 30s
retries: 3
After this, don't forget to close the terminal first, save the .yml file and open the terminal again. Just to be sure HAHA.
Then go to the folder again in the terminal. Use the command "docker-compose up -d --build" first before you use the command "docker-compose up"
It works 100%

Hi, thanks for the recommendations. I have combined two cases to solve this problem after reading your instructions, you can just follow them here.
If you encountered the _mysql problem, please don't forget to download apache-airflow using the keywords "pip install apache-airflow" or depending on the terminal you're using.
After that update the docker-compose.yml file from the cloned repository.
version: '3' services: postgres: image: postgres:9.6 environment: - POSTGRES_USER=airflow - POSTGRES_PASSWORD=airflow - POSTGRES_DB=airflow ports: - "5432:5432"
webserver: image: puckel/docker-airflow:1.10.9 build: context: https://github.com/puckel/docker-airflow.git#1.10.9 dockerfile: Dockerfile args: AIRFLOW_DEPS: gcp_api,s3 PYTHON_DEPS: sqlalchemy==1.3.0 markupsafe==2.0.1 wtforms==2.2 restart: always depends_on: - postgres environment: - LOAD_EX=n - EXECUTOR=Local - FERNET_KEY=jsDPRErfv8Z_eVTnGfF8ywd19j4pyqE3NpdUBA_oRTo= volumes: - ./examples/intro-example/dags:/usr/local/airflow/dags # Uncomment to include custom plugins # - ./plugins:/usr/local/airflow/plugins ports: - "8080:8080" command: webserver healthcheck: test: ["CMD-SHELL", "[ -f /usr/local/airflow/airflow-webserver.pid ]"] interval: 30s timeout: 30s retries: 3
After this, don't forget to close the terminal first, save the .yml file and open the terminal again. Just to be sure HAHA. Then go to the folder again in the terminal. Use the command "docker-compose up -d --build" first before you use the command "docker-compose up"
It works 100%

Worked for me, thanks


Bar Chart : Labels are displayed with decimal values when the highest series data is less than 5

I am using Apex Chart's Distributed Column Chart
This is the data value I provided in series:
series: [{       data: [4, 1]     }],
From this data, I am expecting the labels on Y-axis to be rendered as  0, 1, 2, 3, 4 but that doesn't seem to happen. I tried putting the highest value to 5 in the data array and it rendered correctly.
So the problem here is that I am not able to handle the case when highest value is less than 5 as in the series above for [4, 1].
I tried formatter callback for labels where I used Math.floor() but that didn't work either as it displayed 0, 0, 1, 2, 3, 4
Thanks

I stumble upon the same issue

@junedchhipa is there any way to remove these decimal points in y axis as well? I tried
formatter: function(val) {
return val.toFixed(0);
}
}
seems like not working.


Reconstruction with RealSense camera fail

Platform: Jetson Orin Nano & Ubuntu 22.04 & Isaac ROS 3.0.0
Run example:
ros2 launch nvblox_examples_bringup realsense_example.launch.py \ mode:=dynamic
The data in output is empty, width & height & origin are also invaid.
$ ros2 topic echo /nvblox_node/dynamic_map_slice
header:
  stamp:
    sec:<PHONE_NUMBER>
    nanosec: 223499342
  frame_id: odom
resolution: 0.05000000074505806
width: 0
height: 0
origin:
  x: 3.4028234663852886e+38
  y: 3.4028234663852886e+38
  z: 0.0
unknown_value: 1000.0
data: []

Run this example with rosbag is correct:
$ ros2 topic echo /nvblox_node/dynamic_map_slice
header:
  stamp:
    sec:<PHONE_NUMBER>
    nanosec: 790565322
  frame_id: odom
resolution: 0.05000000074505806
width: 256
height: 184
origin:
  x: -8.0
  y: -9.199999809265137
  z: 0.09000000357627869
unknown_value: 1000.0
data:
- 1000.0
- 1000.0
- 1000.0
- 1000.0
- 1000.0
- 1000.0
- 1000.0
- 1000.0
- '...'

The log maybe helpful:
[component_container_mt-1] -----------
[component_container_mt-1]
[component_container_mt-1] [INFO] [1718268590.584968756] [nvblox_node]: Rates statistics:
[component_container_mt-1]
[component_container_mt-1] NVBlox Rates (in Hz)
[component_container_mt-1] namespace/tag - NumSamples (Window Length) - Mean
[component_container_mt-1] -----------
[component_container_mt-1] ros/color_image_callback     100     15.1
[component_container_mt-1] ros/depth_image_callback     100     7.6
[component_container_mt-1] ros/update_esdf              100     9.5
[component_container_mt-1] mapper/stream_mesh           91      4.9
[component_container_mt-1] ros/tick                     100     101.0
[component_container_mt-1] -----------
[component_container_mt-1]
[component_container_mt-1] [INFO] [1718268590.585014902] [nvblox_node]: Delay statistics:
[component_container_mt-1]
[component_container_mt-1] NVBlox Delays
[component_container_mt-1] namespace/tag - NumSamples (Window Length) - Mean Delay (seconds)
[component_container_mt-1] -----------
[component_container_mt-1] ros/color_image_callback     100     0.069
[component_container_mt-1] ros/depth_image_callback     100     0.013
[component_container_mt-1] -----------
[component_container_mt-1]
[component_container_mt-1] [INFO] [1718268591.423968351] [nvblox_node]: Dropped an item from: color_queue. Size of queue: 9. Total number of dropped messages is: 367
[component_container_mt-1] [INFO] [1718268592.424187846] [nvblox_node]: Dropped an item from: color_queue. Size of queue: 9. Total number of dropped messages is: 382
[component_container_mt-1] [INFO] [1718268593.426054636] [nvblox_node]: Dropped an item from: color_queue. Size of queue: 9. Total number of dropped messages is: 397
[component_container_mt-1] [INFO] [1718268594.426266499] [nvblox_node]: Dropped an item from: color_queue. Size of queue: 9. Total number of dropped messages is: 412
[component_container_mt-1] [INFO] [1718268595.427498494] [nvblox_node]: Dropped an item from: color_queue. Size of queue: 9. Total number of dropped messages is: 427
[component_container_mt-1] [INFO] [1718268595.961308811] [nvblox_node]: Color Statistics:
[component_container_mt-1] avg=66.706161, min=54.031483, max=81.477836, std_dev=2.645830, count=444
[component_container_mt-1] [INFO] [1718268596.438006748] [nvblox_node]: Dropped an item from: depth_queue. Size of queue: 9. Total number of dropped messages is: 220
[component_container_mt-1] [INFO] [1718268597.495052617] [nvblox_node]: Dropped an item from: color_queue. Size of queue: 9. Total number of dropped messages is: 458
[component_container_mt-1] [INFO] [1718268598.495971999] [nvblox_node]: Dropped an item from: color_queue. Size of queue: 9. Total number of dropped messages is: 473
[component_container_mt-1] [INFO] [1718268599.496557407] [nvblox_node]: Dropped an item from: color_queue. Size of queue: 9. Total number of dropped messages is: 488
[component_container_mt-1] [INFO] [1718268600.497162073] [nvblox_node]: Dropped an item from: color_queue. Size of queue: 9. Total number of dropped messages is: 503


fixing problem with palette names

@makella I've just discovered we were using old names. It's very important that we don't change those or backwards compatibility will fail.
CR @javisantana

using where?

but we'd need to update cartocolors or something, right?

yeah totally i knew this would break things but we didn't have a chance to update deep insights yet. i'm sorry!

@saleiva one other thing the palettes deep insights was using were from April! So there were updates that needed to happen.

ok, can you comment on those updates on tickets?

In the future? Or would you like me to do that here now?
I opened the question back in May in Carto Color repo... probably the wrong place.
Sorry!!
https://github.com/CartoDB/CartoColor/issues/5


Feature: switch visibility with update_repo_settings #2537

This PR enhances the update_repo_settings method to provide more control over repository settings, and deprecates the update_repo_visibility method:
Key changes:

Added a private parameter to set repositories as public or private.
Made the gated and private parameters optional, allowing updates to either setting independently.
Added checks to ensure at least one of gated or private is provided.
Payload Optimization: gated and private are only included in the request payload if they have non-None values.
Updated the docstring for update_repo_settings and replaced all references to update_repo_visibility in the documentation.
Added the @_deprecate_method decorator to update_repo_visibility, marking it for removal in v0.29.x.

Hey @Wauplin, the changes in this PR are implemented as per the TO-DOs in issue #2537.

Hey @hanouticelina :), changes were added in the recent commit.

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

I just realized that update_repo_visibility() is used in other tests in tests/test_file_download.py and tests/test_snapshot_download.py. Could you replace update_repo_visibility() with update_repo_settings() in those files as well?

For deprecated methods, we have a expect_deprecation decorator that can be used in tests (see here). The goal is to continue to have the method tested since it's still officially supported and we are also testing that the method is correctly deprecated (i.e. correctly triggers warning).
So in the case of the deprecate use of it here, you can do
    @expect_deprecation("update_repo_visibility")
    def test_download_private_model(self):
        self.api.update_repo_visibility(repo_id=self.repo_id, private=True)

(+ adapt the import)
(sorry it's the kind of undocumentated things...)

@Wauplin do we still need this test ? it tests that there is no warnings when passing repo_id as positional argument into some HfApi functions (create_repo, delete_repo, and update_repo_visibility).

Clearly not needed anymore indeed! Let's just delete it instead of trying to fix the warning capture thingy (we could extend expect_deprecation to handle multiple inputs but really not worth it)

HfApi functions (create_repo, delete_repo, and update_repo_visibility).

Hey @Wauplin and @hanouticelina, so do I remove these HfApi functions (create_repo, delete_repo, and update_repo_visibility) completely or extend those functions with expect_deprecation ??

No no, the only thing to do is to remove the test test_repo_id_no_warning. It's an outdated and legacy one. The methods create_repo and delete_repo are far from been deprecated!

Hey @Wauplin I removed the test test_repo_id_no_warning completely and iterated on @hanouticelina minor comments in the recent commit.

(I've switched the PR as "ready for review". In general @WizKnight you can switch a PR to "ready" as soon as you request someone to review it)

CI's green, let's merge :tada: Thanks again @WizKnight :hugs:

Hey @Wauplin & @hanouticelina , I really appreciate the opportunity!! Thankyou once again🤗


Excessive TCP connections to nodes

I've observed that using the ElastiCacheClusterConfig component seems to lead to an excessive number of TCP connections to nodes.
For example, when I run NETSTAT with the default Enyim configuration, I find that each node has 10 connections. When I run NETSTAT with the ElastiCacheClusterConfig, I can see connections in the ESTABLISHED state that exist in order of multiples of 10. I've seen instances of up to 140 connections per node.
Is this expected?

That is not expected, I'll take a look and see what I can find.

More detail -
It seems to increase gradually with time, in increments of 10. It also seems to affect a single node more than others. I had thought that it could be because this is the node selected as the configuration endpoint, but my tests weren't conclusive and could be just co-incidental. I find that one node will seem to have a large number, whereas other nodes may have 10 or 20, but never >100. I left an app running overnight and it reached 340 connections for a single node.
My setup is 3 nodes in a cluster. However, when I add a node to the cluster, this seems to 'reset' all the connections back to a nominal level (10 per node) after around a minute or so, although they soon build up again. Removing a node has the same effect.
The code I'm using is:
MemcachedClient client = new MemcachedClient(new ElastiCacheClusterConfig(
    ConfigurationManager.AppSettings["ConfigEndpointHost"], 11211));

while (true)
{
    client.Store(StoreMode.Set, "Test", "Test");
    Console.WriteLine(client.Get("Test").ToString());
    Thread.Sleep(60000);
}

I believe I have found the cause in polling logic. I need to do a bit more testing before pushing a fix though.

I pushed out version <IP_ADDRESS> to nuget that fixes the problem. Thanks for pointing out the issue.

Confirmed new build fixes the issue. Thanks for speedy resolution!


Revert allowing timeouts for commands and downloads

[ ] Have you followed the guidelines in our Contributing document?
[ ] Have you checked to ensure there aren't other open Pull Requests for the same change?
[ ] Have you added an explanation of what your changes do and why you'd like us to include them?
[ ] Have you written new tests for your changes? Here's an example.
[ ] Have you successfully run brew style with your changes locally?
[ ] Have you successfully run brew typecheck with your changes locally?
[ ] Have you successfully run brew tests with your changes locally?

Reverts #10864 and #10898
Fixes #10913 (for real this time, I hope)

Review period skipped due to critical label.


Decouple the dashboard and dcp from the core Aspire.Hosting APIs (DistributedApplicationBuilder etc)

We need to separate the core DistirbutedApplication* APIs and the dashboard. The end experience should be the same but we need lower level APIs that don't couple everything together to make the experience we have today work.

@mitchdenny it seems we already had this issue 😄

Closing this issue as too broad in favor of https://github.com/dotnet/aspire/issues/797


Add text to images

Please add a tool to overlay some simple text.
This is a feature most commonly used in online toolboxes.
I wonder that it is missing.

Thanks for the request, it's planned 👀

🥰

Added


[Content] Move caption block into previous block during import

https://main--blog--adobe.hlx3.page/en/publish/2021/10/26/new-creative-cloud-releases-enable-creative-collaboration-drive-innovation-empower-creative-careers
While image captions are displayed correctly, the others (video, animations, embed) are still italic text under the assets.

No caption block, the captions should be moved into the previous blocks during import.

Fixed in importer.


[INFER] Replace "fused_act_bias_wrapper" with "fused_bias_act" api

PR types
Others
PR changes
Others
Description
replace "fused_act_bias_wrapper" with "fused_bias_act" api from paddle

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Env Viz: Avoid trivial step at the end

Example:

Steps 7 and 8 are the same. One of them is redundant.

This issue seems to be solved already. @dickongwd @vansh284 can you check and close the issue?

Closed by https://github.com/source-academy/js-slang/pull/1436


e2e/cypress:fixed authenticaton test cases

Summary
authentication_1_spec.ts fixed
authentication_4_spec.ts fixed
hide_create_account_spec.ts fixed
auth_sso/authentication_spec.js fixed
existing_email_adress_spec.js fixed
login_page_link_account_creation_spec.js
link_customization_cloud_spec.js fixed
ink_customization_e20_1_spec.js fixed
link_customization_e20_2_spec.js fixed
keyboard_shortcuts_2_spec.js demoted
Release Note
NONE

/e2e-test

Successfully triggered e2e testing!
https://git.internal.mattermost.com/qa/cypress-ui-automation/-/pipelines/259325

/e2e-test

A pipeline with the same environment variables is already running.
Please cancel it first with /e2e-cancel, or specify different environment variables.

/e2e-test

Successfully triggered e2e testing!
https://git.internal.mattermost.com/qa/cypress-ui-automation/-/pipelines/259403

/e2e-test

A pipeline with the same environment variables is already running.
Please cancel it first with /e2e-cancel, or specify different environment variables.

/e2e-cancel

Successfully canceled pipelines:
https://git.internal.mattermost.com/qa/cypress-ui-automation/-/pipelines/259403

/e2e-test

Successfully triggered e2e testing!
https://git.internal.mattermost.com/qa/cypress-ui-automation/-/pipelines/259405

/e2e-cancel

Successfully canceled pipelines:
https://git.internal.mattermost.com/qa/cypress-ui-automation/-/pipelines/259405

/e2e-test

Successfully triggered e2e testing!
https://git.internal.mattermost.com/qa/cypress-ui-automation/-/pipelines/259406

/e2e-test

Successfully triggered e2e testing!
https://git.internal.mattermost.com/qa/cypress-ui-automation/-/pipelines/259434

/e2e-test

Successfully triggered e2e testing!
https://git.internal.mattermost.com/qa/cypress-ui-automation/-/pipelines/259788

/e2e-cancel

Successfully canceled pipelines:
https://git.internal.mattermost.com/qa/cypress-ui-automation/-/pipelines/259788

/e2e-run

Successfully triggered e2e testing!
https://git.internal.mattermost.com/qa/cypress-ui-automation/-/pipelines/259790

/e2e-test

Successfully triggered e2e testing!
https://git.internal.mattermost.com/qa/cypress-ui-automation/-/pipelines/259809

/e2e-test

Successfully triggered e2e testing!
https://git.internal.mattermost.com/qa/cypress-ui-automation/-/pipelines/259829


Are some operations impossible via the Viewer.update() function?

This code works how I expect: the screen updates with random "snow".
import numpy as np
import napari
import time

with napari.gui_qt():
    viewer = napari.Viewer()
    layer = viewer.add_image(np.random.random((512, 512)))

    def update(update_period):
        while True:
            time.sleep(update_period)
            layer.data = np.random.random((512, 512))

    viewer.update(update, update_period=0.05)

...but this code does not; it triggers several warnings like this:
WARNING: QObject::setParent: Cannot set parent, new parent is in a different thread
and also seems to break and undock the layer controls panel.
import numpy as np
import napari
import time

with napari.gui_qt():
    viewer = napari.Viewer()

    def update(update_period):
        layer = viewer.add_image(np.random.random((512, 512)))
        while True:
            time.sleep(update_period)
            layer.data = np.random.random((512, 512))

    viewer.update(update, update_period=0.05)

I guess Viewer.update() creates a QRunnable, which I guess I should think of as a thread?
https://github.com/napari/napari/blob/63b0a939b88e6b6ffaa76a026c69d2168afa87e2/napari/viewer.py#L111-L114
https://github.com/napari/napari/blob/63b0a939b88e6b6ffaa76a026c69d2168afa87e2/napari/_qt/qt_update_ui.py#L4
It seems to me that it's not possible to create a layer in any thread except the main thread, which suggests that I can't create layers interactively via the update function. Is that accurate? If so, I assume this applies to other napari operations, and maybe we should focus on a different approach (for example, scheduling interaction via a QTimer)

Yes it is basically a new thread @AndrewGYork . These are run and forget type of threads to handle some side computation/data manipulation.
The way to interact with parent thread's objects would be using signals and slots from another thread. Creating a slot in parent thread, passing connected signal to closure can do what you want to achieve here. Maybe wrapping some viewer methods as slots can help, or do you think that will not be enough for your use?

Just to echo what @AhmetCanSolak said here...  The update function here is very nice for updating the data of a specific layer in the viewer, but for a lot of other interactive things you might want to do with the viewer, it's not going to be flexible enough for you, and you'll need to get a little more familiar with the event loop(s) running in napari.  I definitely think it's worth getting familiar with Qt signals and slots, but there are some other built-in features that you might also want to try using.
If you want something to happen following some event that happens within napari, then trick becomes knowing which signals any given napari object provides for you to "connect" to.  If, on the other hand, you want something to happen following your own event, then you basically need to hook into one of the (confusingly) multiple event loops that are running in napari.
One option is to use keybindings, that will listen for keypresses and then call some callback whenever pressed (providing that function the viewer instance).  These are not well-documented, but basically work like this (see also custom_key_bindings example):
import numpy as np
import napari

with napari.gui_qt():
    viewer = napari.Viewer()

    @viewer.bind_key('i')
    def add_layer(viewer):
        viewer.add_image(np.random.random((512, 512)))

    @viewer.bind_key('k')
    def delete_layer(viewer):
        try:
            viewer.layers.pop(0)
        except IndexError:
            pass

You can also use mouse events, such as a click/drag event (see also custom_mouse_functions example):
import numpy as np
import napari

with napari.gui_qt():
    viewer = napari.Viewer()
    layer = viewer.add_image(np.random.random((512, 512)))

    @layer.mouse_drag_callbacks.append
    def update_layer(layer, event):
        layer.data = np.random.random((512, 512))

If you want GUI elements you will either need to learn a little Qt, or try out magicgui.  For instance to add a Qt button that does something (note, the buttons will be on top of each other here unless you also use a QLayout):
import numpy as np
import napari

# we use qtpy as an abstraction layer on most of the Qt goodies
from qtpy.QtWidgets import QPushButton

with napari.gui_qt():
    viewer = napari.Viewer()
    layer = viewer.add_image(np.random.random((512, 512)))

    def add_layer():
        viewer.add_image(np.random.random((512, 512)))

    def randomize_last_layer():
        try:
            layer = viewer.layers[-1]
            layer.data = np.random.random((512, 512))
        except IndexError:
            pass

    # create buttons
    add_button = QPushButton("add layer")
    update_button = QPushButton("update layer")

    # hook up the "clicked" event to a callback:
    # https://doc.qt.io/qt-5/qabstractbutton.html#clicked
    add_button.clicked.connect(add_layer)
    update_button.clicked.connect(randomize_last_layer)

    # show the buttons
    add_button.show()
    update_button.show()

lastly, here's an example of magicgui, which is more focused on providing an interface for a specific function, and doing something with the result when that function is called:
import numpy as np
import napari
from magicgui import magicgui

with napari.gui_qt():
    viewer = napari.Viewer()
    layer = viewer.add_image(np.random.normal(10, 1, (256, 256)))

    @magicgui(
        auto_call=True,  # call function whenever a param is changed
        width={'maximum': 512},
        height={'maximum': 512},
        noise_type={'choices': ['normal', 'gumbel', 'laplace', 'logistic']},
    )
    def new_image(
        width=256, height=256, loc=10, scale=1.0, noise_type='normal'
    ):
        """generate new random image with some type of noise"""
        function = getattr(np.random, noise_type)
        return function(loc, scale, (width, height))

    def update_layer(result):
        layer.data = result
        # layer.reset_contrast_limits()

    # create the widget
    new_image_widget = new_image.Gui(show=True)
    # connect its "called" signal to your callback function
    new_image.called.connect(update_layer)
    # optionally, add it as a docked widget (and inherit styles) on viewer
    viewer.window.add_dock_widget(new_image_widget)

in all cases though, we're hooking into some event loop that listens for an event to occur, and then performs our desired action.  So that's key here... and it really comes down to that: what event are you interested in responding to (e.g. a new frame from the camera, or a user interaction), and what result to you want to happen...

Good information! Thanks, @AhmetCanSolak and @tlambert03. The event I'm (most) interested in responding to is an object becoming available in a multiprocessing Connection. Depending on the context, we typically either block:
while True:
    command = conn.recv() # This blocks until an object is available
    # Handle the command

or poll:
while True:
    if conn.poll(): # This returns immediately
        command = conn.recv() # We know an object is available so this returns immediately
        # Handle the command
    # Do other things in the event loop

We played with using a QTimer in the main process to trigger polling, which seems to work. The main drawback is this can lead to "busy-waiting". Is there a smarter/standard way to do this? I'll look at slots and signals.

Yeah, for heavy polling like that, you should put the listener in a different thread.  While  viewer.update(func) is designed as a convenience to create a simple thread based on a function, it doesn't provide ways to connect to the main thread.
For more complicated event handling, you might consider looking at napari._qt.utils.new_worker_qthread which I wrote to wrap more complicated worker processes that need to communicate with the main thread.  The docstring of that function provides a decent overview, but I'll show it here too.   The gist is this: You create a generic QObject subclass that (minimally) defines a work() method which starts the worker (this could create your Connection object), and it should also define at least a finished signal that you will emit when the thread is done.  It can also emit any other signals that you need.
class ConnectionWorker(QObject):

    finished = Signal()
    command_received = Signal(Command)  # not sure what type command is

    def __init__(self, handle, readable=True, writable=True):
        super().__init__()
        self.handle = handle
        self.readable = readable
        self.writable = writable

    @Slot()
    def work(self):
        conn = Connection(self.handle)
        while True:
            command = conn.recv()
            command_received.emit(command)
            if some_exit_condition:
                break
        self.finished.emit()

Then hook that up in the main thread, optionally starting it immediately:
from napari._qt.utils import new_worker_qthread

def handle_connection(command):
    # callback that does something with command

worker, thread = new_worker_qthread(
    ConnectionWorker,
    connection_handle,
    start=True,
    # here's where you connect the signals emitted by your custom
    # worker to callbacks in the main thread.
    connections={'command_received': handle_connection},
)

If you want a real world example, you can look at the AnimationWorker object in napari, which runs the timer for the play button (e.g. for time lapse animations) in a separate thread.  The AnimationWorker is defined here, and it is created with new_worker_qthread here.

I made a mock example of how we might use napari to display live images. Does this make any sense to the napari team? If so, would it make sense to put it in the examples directory?
This is an extremely simplified version of how I might use napari as a live display for a microscope. It's way more complicated than many of the current examples, but I think it might allow the kind of fancy interactive functionality that others might be looking for too.
#!/usr/bin/python3
import multiprocessing as mp
import numpy as np
from qtpy.QtCore import QTimer
import napari

"""
An example of how to use Napari in a child process, sending it images
via shared memory, and commands via a pipe. You could build on this
example to use Napari as the live display for your microscope, for
example.
"""

# Image dimensions
nz, ny, nx = 32, 256, 256

def main():
    shared_memory = mp.RawArray('B', nz*ny*nx*2)
    display = NapariDisplay(shared_memory)
    camera = FakeCamera(shared_memory)
    print("\nTo quit, close Napari, then type Ctrl-C in this terminal.")
    while True:
        try:
            camera.record_to_shared_memory()
            display.show_image_from_shared_memory()
        except KeyboardInterrupt:
            break # Closing gracefully is tricky; this is a lazy workaround

class NapariDisplay: # This puts Napari in a child process
    def __init__(self, shared_memory):
        self.shared_memory = shared_memory
        self.command_pipe, self.child_pipe = mp.Pipe()
        self.process = mp.Process(target=display_process, daemon=True,
                                  args=(self.shared_memory, self.child_pipe))
        self.process.start()

    def show_image_from_shared_memory(self):
        self.command_pipe.send('show')
        assert self.command_pipe.recv() == 'done'

def display_process(shared_memory, command_pipe):
    np_array = np.frombuffer(shared_memory, dtype='uint16').reshape(nz, ny, nx)
    with napari.gui_qt():
        viewer = napari.Viewer()
        def check_for_command(): # This will get called regularly by Qt
            if command_pipe.poll():
                command = command_pipe.recv()
                if command == 'show':
                    if len(viewer.layers) == 0:
                        viewer.add_image(np_array)
                    else: # Assume only one layer
                        viewer.layers[0].data = np_array
                command_pipe.send('done')
        timer = QTimer()
        timer.timeout.connect(check_for_command)
        timer.start(1) # 1 millisecond

class FakeCamera: # Fake but fancy images
    def __init__(self, shared_memory):
        self.shared_numpy_array = np.frombuffer(shared_memory, dtype='uint16'
                                                ).reshape(nz, ny, nx)
        self.noiseless = np.array([
            fractal(c) for c in np.linspace(0.7, 1.2, nz) + 0.27015j])
        self.noisy = np.array([
            np.random.poisson(self.noiseless) for i in range(30)])
        self.t = 0

    def record_to_shared_memory(self):
        self.shared_numpy_array[:] = self.noisy[self.t, :, :, :]
        self.t = (self.t + 1) % self.noisy.shape[0]

# A compact way to make pretty pictures to display:
def fractal(c): # https://codereview.stackexchange.com/a/210445
    num_iterations, threshold = 500, 2
    z = (1  * np.linspace(-1, 1, nx).reshape(1, nx) +
         1j * np.linspace(-1, 1, ny).reshape(ny, 1)).flatten()
    live, = np.indices(z.shape)
    iterations = np.empty_like(z, dtype=np.uint16)
    for i in range(num_iterations):
        z_live = z[live] = (z[live])**2 - c
        escaped = z_live.real**2 + z_live.imag**2 > threshold**2
        iterations[live[escaped]] = i
        live = live[~escaped]
    iterations[live] = num_iterations - 1
    return (200 / iterations.max()) * iterations.reshape(ny, nx)

# A simple incantation to make multiprocessing vary less across OS's:
if mp.get_start_method(allow_none=True) != 'spawn':
    mp.set_start_method('spawn')

if __name__ == '__main__':
    main()

Thanks @AndrewGYork, this is great to see.  Definitely "honing in" on the use case here, and a neat low-level example of mp.Process.
I think, from the perspective of napari, my main concern with this is that it adds yet another event loop (display_process) to what is already becoming an "embarrassment" of event loops.  napari already uses both the Qt event loop, and also borrows the vispy event system for it's internal event handling.  And it's not entirely clear to me why we need to put napari in a child process at all, rather than taking advantage of the existing loops we already have going on (other than, perhaps, more familiarity with mp.Process and mp.Pipes?) This approach also puts the burden on the developer to create new APIs for communicating between threads, which is something that these other event loops already offer (event objects in Vispy and Qt signals/slots).
As a complementary example, here's how I would do this with napari in the main thread, receiving signals from the FrameGrabber thread.  I also added some examples of cross thread communication here, with a button at the bottom that both starts the FrameGrabber thread, and also pauses/restarts the camera stream.  In turn, the frame grabber sends back its status to update the text of the button with the pause_changed signal, and updates the image with the frame_ready signal.  Lastly, we can clean up everything nicely when the user closes the viewer (rather than requiring the KeyboardInterrupt) by sending an abort signal to the framegrabber and waiting for its graceful finish.
#!/usr/bin/python3
import multiprocessing as mp
import time

import numpy as np
from qtpy.QtCore import QObject, Signal
from qtpy.QtWidgets import QPushButton

import napari
from napari._qt.utils import new_worker_qthread

# Image dimensions
nz, ny, nx = 32, 256, 256

# A compact way to make pretty pictures to display:
def fractal(c):  # https://codereview.stackexchange.com/a/210445
    num_iterations, threshold = 500, 2
    z = (
        1 * np.linspace(-1, 1, nx).reshape(1, nx)
        + 1j * np.linspace(-1, 1, ny).reshape(ny, 1)
    ).flatten()
    (live,) = np.indices(z.shape)
    iterations = np.empty_like(z, dtype=np.uint16)
    for i in range(num_iterations):
        z_live = z[live] = (z[live]) ** 2 - c
        escaped = z_live.real ** 2 + z_live.imag ** 2 > threshold ** 2
        iterations[live[escaped]] = i
        live = live[~escaped]
    iterations[live] = num_iterations - 1
    return (200 / iterations.max()) * iterations.reshape(ny, nx)

class FrameGrabber(QObject):

    finished = Signal()
    frame_ready = Signal()
    pause_changed = Signal(bool)

    def __init__(self, shared_memory):
        super().__init__()
        self.shared_numpy_array = np.frombuffer(
            shared_memory, dtype='uint16'
        ).reshape(nz, ny, nx)
        self.noiseless = np.array(
            [fractal(c) for c in np.linspace(0.7, 1.2, nz) + 0.27015j]
        )
        self.noisy = np.array(
            [np.random.poisson(self.noiseless) for i in range(15)]
        )
        self.t = 0
        self._aborted = False
        self._paused = False

    def work(self):
        while not self._aborted:
            if not self._paused:
                self.record_to_shared_memory()
                self.frame_ready.emit()
            time.sleep(0.01)
        self.finished.emit()

    def record_to_shared_memory(self):
        self.shared_numpy_array[:] = self.noisy[self.t]
        self.t = (self.t + 1) % self.noisy.shape[0]

    def abort(self):
        self._aborted = True

    def toggle(self):
        self._paused = not self._paused
        self.pause_changed.emit(self._paused)

def main():
    shared_memory = mp.RawArray('B', nz * ny * nx * 2)

    with napari.gui_qt():
        viewer = napari.Viewer()
        button = QPushButton('start')
        viewer.window.add_dock_widget(button)

        def update_image_layer():
            img_name = f"shared {id(shared_memory)}"
            arr = np.frombuffer(shared_memory, dtype='uint16').reshape(
                nz, ny, nx
            )
            try:
                viewer.layers[img_name].data = arr
            except KeyError:
                viewer.add_image(arr, name=img_name)

        def on_pause_change(paused):
            button.setText("stream" if paused else "pause")

        worker, thread = new_worker_qthread(
            FrameGrabber,
            shared_memory,
            connections={
                'frame_ready': update_image_layer,
                'pause_changed': on_pause_change,
            },
        )

        def handle_click():
            if not thread.isRunning():
                thread.start()
            else:
                worker.toggle()

        button.clicked.connect(handle_click)

        # this logic could easily be added to napari so user is not responsible
        def cleanClose(event):
            worker.abort()
            thread.quit()
            thread.wait()
            return viewer.window._qt_window.close()

        viewer.window._qt_window.closeEvent = cleanClose

if __name__ == '__main__':
    main()

I like the approach you show, though I think it targets a slightly different use case (napari as master, vs napari as a module)
Some advantages of napari-as-master:

A reduced number of event loops (this appeals to me too)
Only one GUI (commands to the napari window can easily control microscope hardware)
Control logic (concurrency etc) is written in Qt; you don't have to learn another set of tools (python threading, multiprocessing)

Some advantages of napari-as-module:

Separates the hardware-control code from the display code (this is important for us; we typically run the same microscope in both GUI-controlled and script-controlled mode, but either way we want live image display)
Modular (you can switch out napari for another display module, e.g. pyglet, without rewriting much code)
A separate process, not a separate thread (Our camera often goes 10,000 fps, with no tolerance for pausing. Can Qt threads use multiple CPU cores?)
If done right, concurrent code can be written in standard-library python (threading, multiprocessing), and downstream coders don't have to learn any Qt (my whole team knows python but will probably never learn Qt).

I could keep going, but I suspect we agree there's room/need for both approaches.
I'm sure napari-as-module can be done better than the example I gave. The code I pasted above prioritized being self-contained and conceptually simple; I agree this gives a  super-clunky API for the main process. My current plan is to use napari as a proxy object (standard library version or our custom version) to give a relatively clean API for downstream coders. I really like the idea of my team being able to write code like:
from pco import Camera
from napari_as_module import Viewer

camera = Camera()
display = Viewer()

while True:
    image = camera.snap()
    display.show(image)

They don't have to know anything about napari implementation or internals, they can use python standard-library threading for concurrency without weird crashes, it's still (potentially) high-performance, and they can rapidly/easily re-configure their control code as they reconfigure their instrument, e.g.:
from photometrics import Camera
from pyglet_as_module import Viewer

camera = Camera()
display = Viewer()

while True:
    image = camera.snap()
    display.show(image)

Separates the hardware-control code from the display code

while my example had all this together, I don't see this as a requirement... but see my last comment.

Modular (you can switch out napari for another display module, e.g. pyglet, without rewriting much code)

for just viewing the image, sure.  And if all you even want from your display module is to show an image, sure.  But if you intend eventually to support interactivity with a GUI, then the API-specific things that need to be rewritten grow.  I do agree though, if all you ever want from napari is just to view an image, and have no future plans of building a gui on top of it, then I would keep it as loosely coupled as possible.

A separate process, not a separate thread (Our camera often goes 10,000 fps, with no tolerance for pausing. Can Qt threads use multiple CPU cores?)

there is a QProcess as well https://doc.qt.io/qt-5/qprocess.html

If done right, concurrent code can be written in standard-library python (threading, multiprocessing), and downstream coders don't have to learn any Qt (my whole team knows python but will probably never learn Qt).

this is probably the most important point... and also probably where the "York Lab" approach and the "napari" approach probably have to fork a bit.  If we're talking about "downstream coders in the York lab", then we are restricting ourselves to a very specific use case, and only you know whether your lab will use the additional features offered by a library.  If we're talking about "downstream coders" as anyone in the world or anyone who might stumble on napari, then the possible applications ("but how do I...") explode.
As much as I personally like knowing stdlib approaches, from the perspective of a new user learning "goal-directed" python, I don't know that I think it's always better for someone to learn how to do it with standard library stuff, rather than depend on a library (otherwise we wouldn't have scipy, dask, matplotlib, etc...).  I agree that there some a cognitive load in learning something like Qt, but it's also been around for decades and has learned a lot of tricks for dealing with event loops and concurrency like this and offers a lot of built-in power.  A standard library approach is definitely awesome (though it too comes with some cognitive load to grok), but if your feature needs grow, you will inevitably start making a more and more complicated API, and eventually may find that you've reinvented event loop application logic.  (I mean, this is why people use packages in the first place).
I guess what I'm trying to say is this:
Everyone has to do some calculus to estimate the feature set they are eventually going to need.  If capturing frames and putting them on a monitor is 90% of what your lab is going to need to do, or if you feel like you'll never need the additional features offered by an application library like Qt, then it makes a lot of sense for your lab to keep the coupling as loose as possible.
If you expect your feature needs to grow to include GUIs, interactivity, layouts, cross-thread/process communication, then that's a lot of stuff to have to add to your lab's code base, and then the added time coding those features has to be weighed against the time spent learning existing libraries.  I really do like your approach here!  i think think it's worth pointing out that you are definitely building a new independent API that has lots of similarities to existing solutions.

It's a good point, who else would benefit from a napari-as-module interface, just using it as a viewer rather than a master GUI. I think most folks building microscopes, but that's <100 people. Brainstorming below.
Maybe scikit image? I think they're trying to ditch their own viewers and jump onto y'alls.
https://github.com/scikit-image/scikit-image/issues/4559#issuecomment-610213652
I've seen a lot of scripts that open a matplotlib imshow window, without blocking execution of the underlying script. Napari is way better for interactive image viewing (gui is better for pan/zoom/contrast/cmap), I bet it could eat that market, if it didn't force blocking in the underlying script.
Maybe another example is the napari ipython console itself. I know it's implemented internally with Qt as master and ipython called periodically from the Qt event loop, but to the user, a console that lets you call methods of a live napari window seems a lot like a script that lets you call methods of a live napari window.
Maybe the right approach is for me to try to make a napari-as-module interface that I like, and if y'all like it too, take it in.
In the mean time, do you think the example I posted has any place in the examples directory? Either way, I should probably close this issue soon.

just to be clear, I'm not suggesting that anyone who wants to use napari as a viewer must also use it as a master GUI.  we definitely don't want that!  I was just saying that for what you're trying to achieve in your example there, there already exist fully-featured libraries, one of which happens to be at the root of napari as well... so, inasmuch as you want to use them together, you might want to know about it.

I think [scikit is] trying to ditch their own viewers and jump onto y'alls.

you probably know this, but @jni, core dev at scikit-image, is a co-creator of napari.

I've seen a lot of scripts that open a matplotlib imshow window, without blocking... Napari is way better for interactive image viewing if it didn't force blocking in the underlying script.

I still think there might be some misunderstandings here about the underpinnings of blocking and interactive viewing in napari.  matplotlib.pyplot. imshow is also blocking (non-interactive mode) by default, but of course offers convenience functions like plt.show(block=False) or plt.ion() that start the mainloop of whatever the current backend is (one of which is Qt!).  That's very similar to what napari.gui_qt does.  So if you're feeling like there's something that you fundamentally can't do with napari that you can do with matplotlib imshow, then perhaps there's a convenience API we can offer that would improve that?  show me the syntax in matplotlib that you use to achieve what you want.

but to the user, a console that lets you call methods of a live napari window seems a lot like a script that lets you call methods of a live napari window.

this is how I, and I think @jni use napari all the time (as a window that is seperate from the console we're actually using where we can call methods of the active viewer).  it's also possible. but, as mentioned before, you just can't use napari.gui_qt(). here's a gif, is this what you mean?

Maybe the right approach is for me to try to make a napari-as-module interface that I like, and if y'all like it too, take it in.

by all means!!  it can only come to eventual good to attack this from all angles :)
I also don't want to "monopolize" the napari response here, so will step back and let @jni, @sofroniewn, @royerloic and other @napari/core-devs weigh in with any thoughts.

@AndrewGYork very exciting to see the progress you're making here!
For the napari as master vs module discussion, I think given where we are now it's great that you are exploring all the different options. As we get further along I think we want to make more and more possible through our plugin interface - which is more along the lines of "napari as master" to enable broader reuse, but there might still be some advanced applications like yours where the "as module" makes more sense, so exploring both now seems highly valuable

In the mean time, do you think the example I posted has any place in the examples directory? Either way, I should probably close this issue soon.

I think this is probably too advanced for our examples, but I think it would make a nice how-to-guide in the spirit of https://napari.org/tutorials/applications/dask, including the gifs / and motivating text, not just the code script. I think we'd welcome a PR to the applications section our tutorials repo with this content. As you get further and try new things it will also be easier to update the tutorial. How does this sound?

@sofroniewn sounds like a great path forward! I'll close the issue, look at the other tutorials, and think about what I might write. It's also fine if this doesn't belong in tutorials either, I don't want to push anything that doesn't fit. (maybe ping me if it's a bad fit though, to save me the writing time)
@tlambert03, the workflow you show is awesome, a perfect parallel to how I often work. The main difference is I usually edit-and-run a script rather than type into a REPL. If napari calls worked in a script like they work in ipython, that's how I'd use napari 100% of the time. I (think I) understand why it currently doesn't work that way, and I don't see a simple/perfect way to change that.  If you/y'all every come up with a simple/perfect way to get napari calls to work in a "normal" script like they work in ipython/jupyter, I'll be psyched, but I won't expect it to be a priority.
Thanks everybody for engaging with me, and for making napari.

If you ever come up with a simple/perfect way to get napari calls to work in a "normal" script like they work in ipython/jupyter, I'll be psyched, but I won't expect it to be a priority.

i still think this is already solvable, we're not that different than something like matplotlib... and I'd like to see the syntax you're looking for that you can get matplotlib.imshow if you get a moment!

Dig it, here goes.
When I'm developing an algorithm, I use matplotlib windows (and numpy->tif->imagej) a lot like how I use print statements. Oversimplified pseudocode might look like:
data = load('data.tif')
print(data.shape, data.dtype)

# Potentially buggy, long-running operation:
processed_data_1 = fancy_algorithm_1(data) # Some algorithm I'm developing
# Visualization:
plot(processed_data_1[:, 100]) # Leave this window open for interactive zooming etc
save(processed_data_1, 'processed_data_1.tif')
open_in_imagej('processed_data_1.tif') # Leave this imagej window open for pan/zoom etc

# Another potentially buggy, long-running operation, that builds on the previous one:
processed_data_2 = fancy_algorithm_2(processed_data_1)
# Visualization:
plot(processed_data_2[:, 100]) # Leave this window open for interactive zooming etc
save(processed_data_2, 'processed_data_2.tif')
open_in_imagej('processed_data_2.tif') # Leave this imagej window open for pan/zoom etc

# And so on, for many more lines...

It's nice to have the visualization windows open and interactive while the rest of the script is running (just like it's nice to see the print statements), so I can monitor progress and sanity. It's nice to do this in a script instead of a REPL so I can guarantee initial state and execution order. (Since debugging is most of the work, this is even more important than it sounds)
For concreteness, here's matplotlib syntax I might use:
import matplotlib.pyplot as plt
fig = plt.figure()
plt.plot([1, 2, 3])
fig.show() # Doesn't block, at least on some platforms/versions

I've found matplotlib interactive window behavior to be pretty platform dependent, so I try not to put this in code I share, but I still do it a lot during development.
plt.imshow() is awesome, but so different from what I want in an image inspector, that I often end up defining a code snippet like:
def open_in_imagej(filename):
    imagej_path = config.get('ImageJ', 'path')
    cmd = """run("TIFF Virtual Stack...", "open=%s");"""%(str(filename).replace('\\', '\\\\'))
    subprocess.Popen([imagej_path, "-eval", cmd])

This works great, but of course it's fragile and not suitable for code I distribute.
Napari would be a perfect (non-fragile?) replacement for imagej in this case, if it didn't block in scripts (like how it already works in ipython/jupyter).
Image acquisition/live display is another topic, but the punchlines are similar and this is already pretty long.

thank you for taking the time to write this up!  and just to clarify, you put this all in some __main__ block and then run the script from the command line as python path/to/script.py?  and the result is that you now have N windows open for each time you called plot()?  Is that what you're after, and are you under the impression that you can only have a single napari window open at a time?  such that you have to close it for your script to proceed?

In this case, just run it directly, from the command line, yeah. Yeah, one window for each plot(). I know you can have multiple napari windows open, but yeah I am under the impression that the napari window doesn't open until you hit the __exit__ part of the context manager, and the script blocks at that line until the napari window(s) closes. I'd be psyched to be wrong about that of course :)

sorry to keep asking for details, but matplotlib offers a lot of different APIs and does a lot behind the scenes to start and stop event loops, so I'm just trying to figure out which you are using, cause I'm confident we can do something similar.   For me, if I run script that you had it doesn't show anything:
import matplotlib.pyplot as plt
fig = plt.figure()
plt.plot([1, 2, 3])
fig.show()

if I change the fig.show() method to plt.show() it does show figures, but they block and don't show later figures until I close them.
if you run this exact script, can you tell me A) what backend and platform you're using and B) whether you get three images that come up once per second?
import time
import sys
import matplotlib
import matplotlib.pyplot as plt

print(sys.platform)
print(matplotlib.get_backend())

fig1 = plt.figure()
plt.plot([1, 2, 3])
fig1.show()

time.sleep(1)
fig2 = plt.figure()
plt.plot([1, 2, 3])
fig2.show()

time.sleep(1)
fig2 = plt.figure()
plt.plot([1, 2, 3])
fig2.show()

regarding ImageJ, since you're just starting subprocesses... you can do that with napari like this:
import numpy as np
import subprocess
import tifffile as tf
import time

data = np.random.rand(256, 256)
tf.imsave('test1.tif', data)
subprocess.Popen(['napari', 'test1.tif'])

time.sleep(3)
data = np.random.rand(256, 256)
tf.imsave('test2.tif', data)
subprocess.Popen(['napari', 'test2.tif'])

time.sleep(3)
data = np.random.rand(256, 256)
tf.imsave('test3.tif', data)
subprocess.Popen(['napari', 'test3.tif'])

Good approach, specifying the script. Here's a screenshot:

Text version of the output:
Python 3.5.3 (default, May 11 2017, 09:10:41)
[GCC 6.3.1 20161221 (Red Hat 6.3.1-1)] on linux
Type "copyright", "credits" or "license()" for more information.
>>>
============ RESTART: /home/user/Desktop/test_matplotlib/test2.py ============
linux
TkAgg
1.5.2rc2
>>>

The figures pop up ~1s apart, like you said, from both the "normal" terminal shell, and from IDLE (kind of a weirdo shell).
I've been using this pattern successfully for about a decade on linux and windows, I don't know if it works on mac. I think some time around 2012 it didn't work on a colleague's mac, so I tried to stop using this pattern in code I distribute.

ok!  super helpful, and some interesting things learned:

first off, I was able to repeat what you saw using linux ... but (and this partially explains my confusion) I had to downgrade to matplotlib  to<2.1.0 (2.5 years old) in order to get it to work like you see it.  Current versions of matplotlib block until the end with this script, at which point all the windows show up.

second, even though the window shows up with matplotlib v 1.5.2 ... it's not actually interactive (at least on my machine) in the sense that it cannot be zoomed and resized.  if you run this script, that waits a long time after the first figure, can you actually resize the first figure while it sleeps (without putting the sleep in a second thread)?
import matplotlib.pyplot as plt
import time
fig = plt.figure()
plt.plot([1,2,3])
fig.show()

time.sleep(10)
fig2 = plt.figure()
plt.plot([2,3,4])
fig2.show()

lastly, if you want to emulate that behavior with napari, as mentioned before, you can't use the context manager, and you just need to tell the Qt app to process events in between napari calls (this is similar to what's happening behind the scenes in matplotlib, and assuming we can nail down the desired usage/API, it might be possible to provide some conveniences to do this for you in a less platform-version-dependent way to what you're observing):

from napari import view_image
import numpy as np
import time
from qtpy.QtWidgets import QApplication

app = QApplication([])

v1 = view_image(np.random.rand(512, 512))
app.processEvents()  # this is the trick

time.sleep(2)
v2 = view_image(np.random.rand(512, 512))

app.exec_()

however, in this script, just as I observed with matplotlib above, though the images show immediately as they're going, you can't interact with them until the end... so if you can interact with yours in example number 2 (unlike me) then we need to know exactly how you are invoking the script (I'm running python script_name.py from the command line)
I hope it's becoming clearer here that a lot of this has less to do with napari vs matplotlib or any other viewer, but rather with the nitty gritty details of your specific environment and exactly how you're using it.  It's great if you've found something that works for you, and we can likely mimic it, but we need to figure out exactly why what you're doing ends up working for you.

I finally got some time to dig into this. You're right, matplotlib sure has changed over the years since I first learned it! Even the old version (1.5.2rc2) I was using above doesn't work how I remember it; as you say, I can see the plots, but can't interact with them during time.sleep().
I upgraded to a newer matplotlib (3.2.0, Python 3.7.6, Fedora 31), and found simple (different) commands that give interactive, live behavior, run from 'vanilla' python launched with python myscript.py:
import time
import matplotlib
matplotlib.use('TkAgg') # Doesn't work with Qt
import matplotlib.pyplot as plt

plt.ion()
for i in range(3):
    input('Press Enter to spawn figure %i'%(i+1)) # No spawn in IDLE til end
    fig = plt.figure()
    plt.plot([1, 2, 3])
input("Press Enter to make plots nonresponsive for five seconds...")
time.sleep(5)
input("Plots should be responsive again; press enter to finish.")

At each input() I can interact with all the open plots. It's pretty fragile though; if I switch backends to Qt4Agg or Qt5Agg, the windows come up blank. If I run in IDLE instead of launch from python myscript.py, no windows appear until the end. During a long time.sleep, the plots become unresponsive. If I remove the input, no windows appear until the end.
Tkinter doesn't mind running in a non-main thread, so I can get responsive, nonblocking behavior via threading:
import threading
import time
import matplotlib
matplotlib.use('TkAgg')
import matplotlib.pyplot as plt

def make_figure():
    fig = plt.figure()
    plt.plot([1, 2, 3])
    plt.show()

threading.Thread(target=make_figure).start() # You can only do this once!
print("Pausing for five seconds, but the plot window should stay responsive.")
time.sleep(5)
input("Close the plot window and press Enter to trigger a RuntimeError...")

...but Tkinter hates being called from two threads, so this approach crashes on exit, as the main thread garbage-collects the Tkinter objects in the child.
If I force garbage collection in the child thread, though, this works great!
import threading
import gc
import time
import matplotlib
matplotlib.use('TkAgg')
import matplotlib.pyplot as plt

def make_figure():
    def f():
        fig = plt.figure()
        plt.plot([1, 2, 3])
        plt.show()
    f()
    gc.collect()

threading.Thread(target=make_figure).start() # You can only do this once!
print("Pausing for five seconds, but the plot window should stay responsive.")
time.sleep(5)
input("Close the plot window and press Enter to finish cleanly.")

...unless, of course, I want to plot more than once in a single script :(
Which brings me to the following ugly but functional code:
import threading
import gc
import time
import matplotlib
matplotlib.use('TkAgg')
import matplotlib.pyplot as plt

def plt_loop():
    while True:
        if len(plt_commands) > 0:
            cmd = plt_commands.pop()
            if cmd == 'quit':
                break
            cmd()
            plt.show(block=False)
        try:
            plt._pylab_helpers.Gcf.get_active().canvas.start_event_loop(0.001)
        except AttributeError:
            time.sleep(0.001)
    if plt._pylab_helpers.Gcf.get_active() is not None:
        print("Close the matplotlib windows to finish...")
        plt.show()
    gc.collect()

def make_figure():
    fig = plt.figure()
    plt.plot([1, 2, 3])

plt_commands = []
plt_thread = threading.Thread(target=plt_loop).start()
for i in range(3):
    input('Press Enter to spawn figure %i'%(i+1))
    plt_commands.append(make_figure)
input("Press Enter to sleep for five seconds. Plots should remain responsive.")
time.sleep(5)
plt_commands.append('quit')

I could wrap most of this ugliness in an importable class. (Or I could accept that matplotlib doesn't want to allow this use case anymore ;) ).
Interestingly, (and maybe napari-relevantly), this approach almost works if I change TkAgg to Qt5Agg. It doesn't fail until the end, and the reason it raises an exception seems to be because Qt5Agg tries to use the signal module, which only works from the main thread.


Can't play again

I an using this for an ionic app.    The mp3 file can only play once .
this is the console.log messages:

play
Loading track http:///APP/DonkeyDonkey.mp3
Playing audio sample 'http:///APP/DonkeyDonkey.mp3'
Playing stream with AVPlayer & default rate
Playing track http:///APP/DonkeyDonkey.mp3
Finished playback
Track finished
Media with id 1e10e9b5-6b8e-b325-379f-e45b8377ff7d released
Released track http:///APP/DonkeyDonkey.mp3

play again:
Loading track http:///APP/DonkeyDonkey.mp3
Will use resource 'http:///APP/DonkeyDonkey.mp3' from the Internet.
Audio error => track http:///APP/DonkeyDonkey.mp3 [object Object]
Loading track http:///APP/DonkeyDonkey.mp3
Audio error => track http://***/APP/DonkeyDonkey.mp3 [object Object]

config:
from.tracks = [{
src: 'http://***' + encodeURI(from.arr_people.audio),
artist: '',
title: '',
art: '',
preload: 'metadata'
}];

HTML :
<audio-track #audio *ngFor="let track of tracks" [track]="track" (onFinish)="onTrackFinished($event)">
<ion-item style="background-color:rgb(117, 165, 236)" [hidden]='!playing'>

<audio-track-play dark [audioTrack]="audio">

<audio-track-progress-bar dark duration progress [audioTrack]="audio" [ngStyle]="{visibility: audio.completed > 0 ? 'visible' : 'hidden'}">

Are you using the cordova plugin Media?

"cordova-plugin-media": "^3.0.1",
"cordova-plugin-streaming-media": "^1.0.2",
I installed those two .

I  use  WebAudioProvider now.
It can play again, but can't display  pause and play button, it always refreshing...

Hmm it is hard to tell, it worked fine when I tested it back then. I will try to look into it whenever I find some spare time.

@codinronan That makes sense and explains why so many people are reporting the same issues. There are so many different use cases that it is hard to find one approach that works for all.

@codinronan Can you let me know how can I recreate the audio track after the track is released by the plugin. I mean which method to use to re-create it again?


Handle SS tiers

By /u/iDervyi
Here's the SS formula, which you could use to automate the tiers in your program:
=(ROUNDDOWN((ROUNDDOWN((1-ROUNDDOWN(((SS-354) * (0.13/858)),3)) * cast_time,3)) * (1-SSbuff),2))
You can use it to check all of the tiers of 2.5, 3 and 3.5s casts (for nin/mnk you'll want to ensure to use huton/gl3), then use the remove duplicates function to see all the tiers and the corresponding SS values.
Alternatively, you can use this: https://docs.google.com/spreadsheets/d/1UC503iaohJnz6JnnTtUEWe0z9nqWm9f8SpObE_xenXY/edit#gid=403936574
And go to the "grand sheet" tab. There, you'll see all the SS tiers. The tiers repeat from 361 -> 493 SS. So you can either take a mathematical approach, or a manual approach where your sim will read from a chart with all the SS tiers listed.
If you're 1 SS V meld away from hitting a tier, that should take priority over any other meld. When it comes to relic, once you manage to fit that in, you can use the Relic to spec the required SS to hit the next tier by sacrificing Crit, rather than using melds.
Any leftover SS after you've hit a tier is "dead weight", and should not be counted.
As for Crit Tiers, no evidence to support it yet. Anyone claiming there are Crit tiers has 0 proof, as far as I'm concerned.

OK so:
I'm going to add a setting castTimes to the job configs, possibly with weights.
These are used to generate a step function for mapping real to effective SS, which then replaces SS in the objective.

This is fixed in eb648fdec16d2d8935765ece73be455f9d432c36. I need someone to edit config.yaml (see #20).


validate downloads

I guess it would be good if we could somehow validate the downloaded files to make sure they are what we expect (like MD5 or similar). maybe we can autogenerate such a checksum in the CLI when creating the dataloaders.
I.e. CLI would ask: povide the path to the file this dataloader will be loading...and then the cli computs them mds of that file and adds it as an attribute to the dataloader.
or is this overcomplication thins too much @davidsebfischer @Zethson

+1 for the idea in general
But what if the file that the dataloader is supposed to load is stored somewhere in the cloud?
You cannot compute the md5 of such a file, right?

even that would be possible I think: https://stackoverflow.com/questions/54483868/python-compare-local-and-remote-file-md5-hash
however, I'm not sure if this would be necessary for a file hosted by eg. cellxgene. we could just trust that it's fine.

even that would be possible I think: https://stackoverflow.com/questions/54483868/python-compare-local-and-remote-file-md5-hash
however, I'm not sure if this would be necessary for a file hosted by eg. cellxgene. we could just trust that it's fine.

Cool, was not aware of this.
Then yes, we could do that.

We could host a separate file of dataset IDs and MD5s in sfaira that can be used to test downloads (or even files on remote before download). We would then just ignore this check if the MD5 is not given.

how would such a file be populated though?


feat: serialize RouteList for Trips

Modify SerializedTrip and TripSerializer to add RouteList field that takes in a list of SerializedLatLng. This data will allow the sample app to display polyline with traffic condition.

Nit: lets add a bit more context to the PR title, example: 'feat: serialize RouteList for Trips/Vehicles'
And also let's remove the part that mentions 'CurrentRouteSegmentTraffic' from the description.

Nit: just confirming, did we test that existing single destination trip scenario works with this change?

Nit: just confirming, did we test that existing single destination trip scenario works with this change?

Yes. In trip response, "routeList" will be added, and the corresponding value is a list SerializedLatLng routes with fields "latitude" and "longitude". The route displays correctly with this change


subscribe notification with xpath-filter is throwing error at netopeer-server

Hi Team,
On netopeer2-cli, subscribing for notifications with following xpath filter is throwing error at netopeer-server. Though netopeer2-cli outputs "OK" but no notification comes on edit-config. unable to find where "instance-identifier" type is defined and its supported pattern. Please help in how to subscribe for specific xpath edit-config notifications.
sysrepo : 2.2.7
netopeer2 : 2.1.41
libyang : 2.0.263
libnetconf2 : 2.1.25

subscribe --filter-xpath /ietf-netconf-notifications:netconf-config-change/edit[target="/urn:hb-mgmt:1.0:hb"]
OK

server side logs:
2023-07-27T11:52:20.186501793Z netopeer2-server|INFO "netopeer2-server-log" [INF]: SR: Published event "rpc" "/notifications:create-subscription" with ID 1 priority 0 for 1 subscribers.
2023-07-27T11:52:20.186521542Z netopeer2-server|INFO "netopeer2-server-log" [INF]: SR: Processing "/notifications:create-subscription" "rpc" event with ID 1 priority 0 (remaining 1 subscribers).
2023-07-27T11:52:20.186526198Z netopeer2-server|ERROR "netopeer2-server-log" [ERR]: LY: Invalid character ':'[13] of expression '/urn:hb-mgmt:1.0:hb'. (Schema location "/ietf-netconf-notifications:netconf-config-change/edit/target".)
2023-07-27T11:52:20.186530416Z netopeer2-server|WARNING "netopeer2-server-log" [WRN]: LY: Invalid value "/urn:hb-mgmt:1.0:hb" which does not fit the type (Invalid instance-identifier "/urn:hb-mgmt:1.0:hb" value - syntax error.).
2023-07-27T11:52:20.186534670Z netopeer2-server|WARNING "netopeer2-server-log" [WRN]: LY: Previous warning generated by XPath subexpression[55] "target="/urn:hb-mgmt:1.0:hb"" with context node "/".
Tried below as well but same error.

subscribe --filter-xpath /ietf-netconf-notifications:netconf-config-change/edit[target="hb-mgmt:hb"]

Server side log:
2023-07-27T12:06:12.444928449Z netopeer2-server|INFO "netopeer2-server-log" [INF]: SR: Published event "rpc" "/notifications:create-subscription" with ID 4 priority 0 for 1 subscribers.
2023-07-27T12:06:12.444943313Z netopeer2-server|INFO "netopeer2-server-log" [INF]: SR: Processing "/notifications:create-subscription" "rpc" event with ID 4 priority 0 (remaining 1 subscribers).
2023-07-27T12:06:12.444947634Z netopeer2-server|ERROR "netopeer2-server-log" [ERR]: LY: Unexpected XPath token "NameTest" ("hb-mgmt:hb"), expected "Operator(Path)". (Schema location "/ietf-netconf-notifications:netconf-config-change/edit/target".)
2023-07-27T12:06:12.444957681Z netopeer2-server|WARNING "netopeer2-server-log" [WRN]: LY: Invalid value "hb-mgmt:hb" which does not fit the type (Invalid instance-identifier "hb-mgmt:hb" value - syntax error.).2023-07-27T12:06:12.444962887Z netopeer2-server|WARNING "netopeer2-server-log" [WRN]: LY: Previous warning generated by XPath subexpression[55] "target="hb-mgmt:hb"" with context node "/".
sample edit-xml
<hb xmlns="urn:hb-mgmt:1.0">         <hbperiod>15</hbperiod> </hb>
Attached yang for reference:
hb-mgmt.zip

Use the module name instead of the namespace or prefix.

Thank you. it worked. you can close the ticket.


[system-probe] Automatically download kernel headers (1)

What does this PR do?
Leverages the nikos library to allow the system-probe to automatically download kernel headers in the event that
a) runtime compilation is enabled, and
b) kernel headers are not found on the host machine.
To make reviewing a little easier, this PR contains only the changes needed for downloading kernel headers on non-RPM distributions. I will open a follow-up PR which will contain the changes necessary for building nikos with support for RPM distributions (using omnibus-nikos).
Motivation
What inspired you to submit this pull request?
Additional Notes
Anything else we should know when reviewing?
Describe your test plan
Write there any instructions and details that should be tested during the QA.

@ISauve It looks like you need to rebase and update the licenses file with inv generate-licenses.


Prevent ipc from crashing sway

Fixes https://github.com/SirCmpwn/sway/issues/458

Just for reference SIGPIPE is triggered here: https://github.com/SirCmpwn/sway/blob/master/sway/ipc-server.c#L458 because sway is trying to send a binding event signal to the socket of the terminated swaybar.

Thanks!


Add shape functions and ignore un-consumed outputs

This PR contains two commits:
[graph] ignore nodes whose outputs are not consumed

If the input graph contains a node (e.g. an initializer) whose output is
not consumed by any node, then the graph initialization fails.  More
specifically, in looking up the list of consumer nodes, the code indexes
into the `consumedby` dictionary, but the key (i.e. the output
name) does not exist in the dictionary since the output is unused.

This patch makes a quick fix by first checking whether the output name
is present in the dictionary before looking up the value in it.

and
[shape] add shape functions for GridSample and DepthToSpace nodes

This patch registers shape functions for the GridSample and DepthToSpace
nodes based on the ONNX operator spec.

To test the shape functions for GridSample and DepthToSpace nodes, use the following scripts to generate the ONNX files:
import onnx

from onnxscript import opset19, script, FLOAT

@script()
def this_script(X: FLOAT[1, 1, 4, 4], G: FLOAT[1, 6, 6, 2]) -> FLOAT[1, 1, 6, 6]:
    return opset19.GridSample(X, G)

onnx_model = this_script.to_model_proto()
onnx.save(onnx_model, "/tmp/a.onnx")

import onnx

from onnxscript import opset19, script, FLOAT

@script()
def this_script(X: FLOAT[1, 8, 2, 3]) -> FLOAT[1, 2, 4, 6]:
    return opset19.DepthToSpace(X, blocksize=2)

onnx_model = this_script.to_model_proto()
onnx.save(onnx_model, "/tmp/b.onnx")

And then reproduce the error using the following commands:
$ python3 -m onnx_tool --in /tmp/a.onnx --out /dev/null
$ python3 -m onnx_tool --in /tmp/b.onnx --out /dev/null

Thanks for your contribution! For personal reseason , your code review will be delayed for 3 days. Thanks again!
Best Regards

From: Ashay Rane @.>
Sent: Wednesday, October 4, 2023 5:59:36 AM
To: ThanatosShinji/onnx-tool @.>
Cc: Subscribed @.***>
Subject: Re: [ThanatosShinji/onnx-tool] Add shape functions and ignore un-consumed outputs (PR #53)
To test the shape functions for GridSample and DepthToSpace nodes, use the following scripts to generate the ONNX files:
import onnx
from onnxscript import opset19, script, FLOAT
@script()
def this_script(X: FLOAT[1, 1, 4, 4], G: FLOAT[1, 6, 6, 2]) -> FLOAT[1, 1, 6, 6]:
return opset19.GridSample(X, G)
onnx_model = this_script.to_model_proto()
onnx.save(onnx_model, "/tmp/a.onnx")
import onnx
from onnxscript import opset19, script, FLOAT
@script()
def this_script(X: FLOAT[1, 8, 2, 3]) -> FLOAT[1, 2, 4, 6]:
return opset19.DepthToSpace(X, blocksize=2)
onnx_model = this_script.to_model_proto()
onnx.save(onnx_model, "/tmp/b.onnx")
And then reproduce the error using the following commands:
$ python3 -m onnx_tool --in /tmp/a.onnx --out /dev/null
$ python3 -m onnx_tool --in /tmp/b.onnx --out /dev/null
—
Reply to this email directly, view it on GitHubhttps://github.com/ThanatosShinji/onnx-tool/pull/53#issuecomment-1745792396, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AZZIQRUK25WV5BS7IAVL5RLX5SDERAVCNFSM6AAAAAA5RUP27OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONBVG44TEMZZGY.
You are receiving this because you are subscribed to this thread.Message ID: @.***>

No worries, thanks for agreeing to review the PR!

Good work!


[INTEL MKL] Enabled MIN_FIRST support and primitive caching for MKL-DNN Quantize OP

This PR will enable

Min first mode support in the MKL-DNN quantize Op and
Primitive Caching for the same Op

Code changes involve

Rewriting the rules in the mkl_layout_pass.cc to allow mkl-dnn quantize function being called for MIN_FIRST mode as well
Appropriate code added in mkl_quantize_op.cc to support MIN_FIRST mode and primitive caching
Added 2 more tests in mkl_quantize_op_test.cc specific to MIN_FIRST mode.

@rgomathi Can you please resolve conflicts? Thanks!

Hi @penpornk , Thanks for the comments, will resolve them and get back to you ASAP

Hi @penpornk , I'm extremely Sorry for the delay. Made the changes recommended. Please review and let me know your comments. Thanks.

@rgomathi Can you please resolve conflicts? Thanks!

Hi @penpornk , Thanks for the comments and I changed the code accordingly. Please review and let me know. Thanks

Fixed them....

@rgomathi Could you please address Ubuntu Sanity errors? Thanks!

@rgomathi Never mind. Ubuntu Sanity build successful after re-trigger tests. Thanks!

Thank you @penpornk


Error when running npm start

I followed the steps in the README to clone the repo and install the dependencies. When I run npm start, I get the following error log:
0 info it worked if it ends with ok
1 verbose cli [ '/usr/local/bin/node', '/usr/local/bin/npm', 'start' ]
2 info using<EMAIL_ADDRESS>3 info using<EMAIL_ADDRESS>4 verbose run-script [ 'prestart', 'start', 'poststart' ]
5 info lifecycle<EMAIL_ADDRESS><EMAIL_ADDRESS>6 silly lifecycle<EMAIL_ADDRESS>no script for prestart, continuing
7 info lifecycle<EMAIL_ADDRESS><EMAIL_ADDRESS>8 verbose lifecycle<EMAIL_ADDRESS>unsafe-perm in lifecycle true
9 verbose lifecycle<EMAIL_ADDRESS>PATH: /usr/local/lib/node_modules/npm/bin/node-gyp-bin:/Users/anthony/Development/Test/get-started-node/node_modules/.bin:/Users/anthony/.swiftenv/shims:/Library/Developer/Toolchains/swift-latest.xctoolchain/usr/bin:/Users/anthony/Development/SDK/android-sdk-macosx/platform-tools/:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/usr/local/MacGPG2/bin
10 verbose lifecycle<EMAIL_ADDRESS>CWD: /Users/anthony/Development/Test/get-started-node
11 silly lifecycle<EMAIL_ADDRESS>Args: [ '-c', 'node server.js' ]
12 silly lifecycle<EMAIL_ADDRESS>Returned: code: 1  signal: null
13 info lifecycle<EMAIL_ADDRESS>Failed to exec start script
14 verbose stack Error<EMAIL_ADDRESS>start: `node server.js`
14 verbose stack Exit status 1
14 verbose stack     at EventEmitter.<anonymous> (/usr/local/lib/node_modules/npm/lib/utils/lifecycle.js:279:16)
14 verbose stack     at emitTwo (events.js:106:13)
14 verbose stack     at EventEmitter.emit (events.js:194:7)
14 verbose stack     at ChildProcess.<anonymous> (/usr/local/lib/node_modules/npm/lib/utils/spawn.js:40:14)
14 verbose stack     at emitTwo (events.js:106:13)
14 verbose stack     at ChildProcess.emit (events.js:194:7)
14 verbose stack     at maybeClose (internal/child_process.js:899:16)
14 verbose stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:226:5)
15 verbose pkgid<EMAIL_ADDRESS>16 verbose cwd /Users/anthony/Development/Test/get-started-node
17 verbose Darwin 16.5.0
18 verbose argv "/usr/local/bin/node" "/usr/local/bin/npm" "start"
19 verbose node v7.10.0
20 verbose npm  v4.5.0
21 error code ELIFECYCLE
22 error errno 1
23 error<EMAIL_ADDRESS>start: `node server.js`
23 error Exit status 1
24 error Failed at the<EMAIL_ADDRESS>start script 'node server.js'.
24 error Make sure you have the latest version of node.js and npm installed.
24 error If you do, this is most likely a problem with the get-started-node package,
24 error not with npm itself.
24 error Tell the author that this fails on your system:
24 error     node server.js
24 error You can get information on how to open an issue for this project with:
24 error     npm bugs get-started-node
24 error Or if that isn't available, you can get their info via:
24 error     npm owner ls get-started-node
24 error There is likely additional logging output above.
25 verbose exit [ 1, true ]

I updated node and npm versions to the latest, as suggested in the logs, but I got the same result.

Nevermind, this was my own issue. Turns out my machine was running something else on the port 3000.