Water Leak Sensor. Detected, but state does not update and automations don't trigger.

Device normal info

NEO
Unknown
Device is recognized. Water leak sensors and battery levels are recognized, but they don't change when the alarm is triggered. Tuya app triggers notification immediately. I've tried setting an automation using trigger type: Device, State and Numeric state. The issue is that the state of the sensor is always Dry for HomeAssistant and does not update. I can see state changes in the logs, however.

https://i.ibb.co/nnYywfs/Screenshot-20210930-173952.png
Device info (please complete the following information, which can be found in log):
like this:
{
"active_time":<PHONE_NUMBER>,
"biz_type": 0,
"category": "sj",
"create_time":<PHONE_NUMBER>,
"icon": "smart/icon/1528364839bqwi2byfbhv_0.png",
"id": "84445000807d3a00bb62",
"ip": "***",
"lat": "***",
"local_key": "***",
"lon": "***",
"name": "SensorAguaTerraza",
"online": true,
"owner_id": "33667113",
"product_id": "SmFKLTOcGHbPQvmh",
"product_name": "Water sensor",
"status": [
{
"code": "watersensor_state",
"value": "2"
},
{
"code": "battery_state",
"value": "high"
}
],
"sub": false,
"time_zone": "+01:00",
"uid": "***",
"update_time":<PHONE_NUMBER>,
"uuid": "84445000807d3xxxxxxx"
},

Sensor changes state when triggered, but the state change never reaches HA and hence automation is not triggered.
2021-09-30 17:26:46 DEBUG (Thread-4) [tuya iot] mq _on_device_report-> [{'101': True, 'code': 'watersensor_state', 't': '1633015606', 'value': '1'}]
2021-09-30 17:26:46 DEBUG (Thread-4) [custom_components.tuya_v2] _update--><custom_components.tuya_v2._init_tuya_sdk.<locals>.DeviceListener object at 0x7f01117d27f0>;->>{'watersensor_state': '1', 'battery_state': 'middle'}
2021-09-30 17:26:46 DEBUG (Thread-4) [custom_components.tuya_v2] _update--><custom_components.tuya_v2._init_tuya_sdk.<locals>.DeviceListener object at 0x7f01117d27f0>;->>{'watersensor_state': '1', 'battery_state': 'middle'}
2021-09-30 17:26:53 DEBUG (Thread-4) [tuya iot] _on_log: Received PUBLISH (d0, q0, r0, m0), 'cloud/token/in/7xxxxxxxxxx7f53689ed96b42', ... (372 bytes)
2021-09-30 17:26:53 DEBUG (Thread-4) [tuya iot] payload-> b'{"data":"ARvxxxxxxxxxbuKDpN/S91iyhuAihvDFIwxxxxxxxxxxxxxxxHhUGM3t79XEiob7aPtHD3ml9PjbOtlnmOgvv7ThTvqyZpZIUYmdZniclo3ohBkzElpErr+Z3XZyKoeEAuunbb89Z1O5yeFxxxxxxxxxxxxxxxxxxxxxpGERKC9HaamTsyxxxxxxxxxxxxxJSmQ==","protocol":4,"pv":"2.0","sign":"d9xxxxxxxxxxxxxxbbd6a77be0","t":1633015613}'
2021-09-30 17:26:53 DEBUG (Thread-4) [tuya iot] on_message: {'data': {'dataId': 'e4e3a47e-126f-49f7-868e-3bcaea85eea7', 'devId': '844450008xxxxx', 'productKey': 'SmFKxxxxx', 'status': [{'102': '0x00', 'code': 'battery_state', 't': '1633015613', 'value': 'high'}]}, 'protocol': 4, 'pv': '2.0', 'sign': 'd9bcabf878b8ed8690073dbbd6a77be0', 't':<PHONE_NUMBER>}
2021-09-30 17:26:53 DEBUG (Thread-4) [tuya iot] mq receive-> {'data': {'dataId': 'e4e3a47e-126f-49f7-868e-3bcaea85eea7', 'devId': '8444500080xxxxx', 'productKey': 'SmFKxxxxxxvmh', 'status': [{'102': '0x00', 'code': 'battery_state', 't': '1633015613', 'value': 'high'}]}, 'protocol': 4, 'pv': '2.0', 'sign': 'd9bcabf878b8ed8690073dbbd6a77be0', 't':<PHONE_NUMBER>}
2021-09-30 17:26:53 DEBUG (Thread-4) [tuya iot] mq _on_device_report-> [{'102': '0x00', 'code': 'battery_state', 't': '1633015613', 'value': 'high'}]
2021-09-30 17:26:53 DEBUG (Thread-4) [custom_components.tuya_v2] _update--><custom_components.tuya_v2._init_tuya_sdk.<locals>.DeviceListener object at 0x7f01117d27f0>;->>{'watersensor_state': '1', 'battery_state': 'high'}
2021-09-30 17:26:53 DEBUG (Thread-4) [custom_components.tuya_v2] _update--><custom_components.tuya_v2._init_tuya_sdk.<locals>.DeviceListener object at 0x7f01117d27f0>;->>{'watersensor_state': '1', 'battery_state': 'high'}
2021-09-30 17:26:53 DEBUG (Thread-4) [tuya iot] _on_log: Received PUBLISH (d0, q0, r0, m0), 'cloud/token/in/721b9a9c795e8c58c07f53689ed96b42', ... (372 bytes)
2021-09-30 17:26:53 DEBUG (Thread-4) [tuya iot] payload-> b'{"data":"RjvJrTFuOBz+brh5e4/FhD5qIpLO0OaDoYfu80fDEKvq9Q7R+ADR4ce4+QldVWeGxCSbirnBovsSb48MWrFhoS8zo0HbXknQPlPiz6YcGNz2MaE+LBQIIHhUGMxxxxxxxxxxxxxxxxxxxxxxxxx/vBarJ7DbYKALwSf5Z4wP8snMaEqMz0e2amTsyJxT5ruKhxnRIIJSmQ==","protocol":4,"pv":"2.0","sign":"07c1bc3a416d2f4c71e0d04a8eb785ac","t":1633015613}'
2021-09-30 17:26:53 DEBUG (Thread-4) [tuya iot] on_message: {'data': {'dataId': '8f1cb9a4-946a-4ce2-9600-d737d59bbf2c', 'devId': '8xxx', 'productKey': 'SmFxxxxmh', 'status': [{'101': False, 'code': 'watersensor_state', 't': '1633015613', 'value': '2'}]}, 'protocol': 4, 'pv': '2.0', 'sign': '07c1bc3a416d2f4c71e0d04a8eb785ac', 't':<PHONE_NUMBER>}
2021-09-30 17:26:53 DEBUG (Thread-4) [tuya iot] mq receive-> {'data': {'dataId': '8f1cb9a4-946a-4ce2-9600-d737d59bbf2c', 'devId': '8444xx', 'productKey': 'SmFKxxxQvmh', 'status': [{'101': False, 'code': 'watersensor_state', 't': '1633015613', 'value': '2'}]}, 'protocol': 4, 'pv': '2.0', 'sign': '07c1bc3a416d2f4c71e0d04a8eb785ac', 't':<PHONE_NUMBER>}
2021-09-30 17:26:53 DEBUG (Thread-4) [tuya iot] mq _on_device_report-> [{'101': False, 'code': 'watersensor_state', 't': '1633015613', 'value': '2'}]
2021-09-30 17:26:53 DEBUG (Thread-4) [custom_components.tuya_v2] _update--><custom_components.tuya_v2._init_tuya_sdk.<locals>.DeviceListener object at 0x7f01117d27f0>;->>{'watersensor_state': '2', 'battery_state': 'high'}
2021-09-30 17:26:53 DEBUG (Thread-4) [custom_components.tuya_v2] _update--><custom_components.tuya_v2._init_tuya_sdk.<locals>.DeviceListener object at 0x7f01117d27f0>;->>{'watersensor_state': '2', 'battery_state': 'high'}

I have the same issue. I have six STITCH by Monoprice Smart Water Leak/Flood Sensor (Product # 33124) I did  not like MonoPrice Stitch app because it kept disconnecting me when  not used for sometime. It means when there is a leak, I wont get notification.   So I moved them to SmartLife app so I can use Tuya integration with HA.
HA sees its status and it's battery but when you test it, HA does not update but SmartLife App immediately notify you of water leak.
I decided to purchaseTuya Water Leakage Detector from Aliexpress to see if their unit behave differently.
I found that the new unit shows up as "Flood sensor" in HA and works as expected.
Mono Price Flood sensor shows up as "Water sensor" and does not work correctly. No update in HA when it is under water.
I wonder if Tuya integration is not handling the case for "Water sensor" which explains why other brands are not working in HA but works in the SmartLife App. Hope this gives some clue.


Add cargo release metadata to mc-sgx-panic-abort

Motivation

Future Work

Current dependencies on/for this PR:

main

PR #18   👈

This comment was auto-generated by Graphite.

Codecov Report

Merging #18 (357fe89) into main (336cf09) will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff          @@
##            main     #18   +/-   ##
=====================================
  Coverage   9.09%   9.09%
=====================================
  Files          2       2
  Lines         11      11
=====================================
  Hits           1       1
  Misses        10      10

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


output directory set to target branch name

I think the output directory is set to /{branch-name} when target-branch is specified in dependabot.yml, a single npm dependency is updated, and that dependency is scoped (has two segments).
dependabot.yml with target branch name: https://github.com/npm/agent/blob/21c19874834fb00c7ab37268b385fb84deb2df04/.github/dependabot.yml
Example Dependabot PR branch names
The first PR updates just @npmcli/template-oss. The second updates multiple dependencies including @npmcli/template-oss.

PR
branch name
branch name segments
outputs.directory

https://github.com/npm/agent/pull/110
dependabot/npm_and_yarn/main/npmcli/template-oss-4.23.0
5
/main

https://github.com/npm/statusboard/pull/877
dependabot/npm_and_yarn/main/dependency-updates-4de8b5bfcf
4
/

npm/agent output:
Run dependabot/fetch-metadata@v1
Parsing Dependabot metadata
Outputting metadata for 1 updated dependency
  outputs.dependency-names: @npmcli/template-oss
  outputs.dependency-type: direct:development
  outputs.update-type: version-update:semver-minor
  outputs.directory: /main
  outputs.package-ecosystem: npm_and_yarn
  outputs.target-branch: main
  outputs.previous-version: 4.22.0
  outputs.new-version: 4.23.0
  outputs.compatibility-score: 0
  outputs.maintainer-changes: false
  outputs.dependency-group:
  outputs.alert-state:
  outputs.ghsa-id:
  outputs.cvss: 0

npm/statusboard output:
Run dependabot/fetch-metadata@v1
Parsing Dependabot metadata
Outputting metadata for 3 updated dependencies
  outputs.dependency-names: @npmcli/template-oss, @octokit/rest, semver
  outputs.dependency-type: direct:production
  outputs.update-type: version-update:semver-minor
  outputs.directory: /
  outputs.package-ecosystem: npm_and_yarn
  outputs.target-branch: main
  outputs.previous-version:
  outputs.new-version:
  outputs.compatibility-score: 0
  outputs.maintainer-changes: false
  outputs.dependency-group: dependency-updates
  outputs.alert-state:
  outputs.ghsa-id:
  outputs.cvss: 0

Issue is still present in v2 release, and the main branch:
https://github.com/npm/agent/actions/runs/9976724284/job/27569631669
Run dependabot/fetch-metadata@main

Parsing Dependabot metadata
Outputting metadata for 1 updated dependency
  outputs.dependency-names: @npmcli/template-oss
  outputs.dependency-type: direct:development
  outputs.update-type: version-update:semver-minor
  outputs.directory: /main
  outputs.package-ecosystem: npm_and_yarn
  outputs.target-branch: main
  outputs.previous-version: 4.22.0
  outputs.new-version: 4.23.0
  outputs.compatibility-score: 0
  outputs.maintainer-changes: false
  outputs.dependency-group:
  outputs.alert-state:
  outputs.ghsa-id:
  outputs.cvss: 0

I'm sorry, I don't understand the issue as currently described.
What is your desired/expected behavior? And what is the current behavior?

The branch name is incorrectly parsed by this action under some conditions
Expected behavior
The directory output is set to /
Actual behavior
The directory output is set to /main. main being the target branch name.


Feature request: Add asn_lookup hyperlink for "Remaining IPs" and formatting changes

Love the new updates. Great idea having added the ability to just click on a hop for a new asn_lookup.
Can we do the same for the remaining IPs that get "ignored"? To avoid ambiguity, in this screenshot <IP_ADDRESS>

While I love this new feature, it does make the hop output a bit busy.
Can we remove the unscores from the hyperlinks and open the links in a new window (style="text-decoration: none;" and target="_blank" or something)?
I'm not sure aha supports that out of the box, so maybe a stylesheet?
My OCD is kicking in now, and the links to APIs seem inconsistent. Currently in the detailed ASN lookup top section we have
WHOIS,  host.io and ipinfo.io
Can we change that to
whois, host and ipinfo (and without the chain link icon thing)
I don't mind doing this but as it's really esthetics and not functional, do feel free to tell me to naff off and get a life.
ASN number lookup output hurts my brain at the moment, but it's super rich.
Not sure how much sense it makes to do traceroutes to subnets, but maybe I'm missing something. I will spend more time playing with it soon.
Thanks again for this great tool

First things first:

do feel free to tell me to naff off and get a life

I never would, I welcome your (just as anybody's) feedback, and am thankful that someone is investing their precious time to help improve the tool. I always try to make the tool as user-friendly and functional as possible, but I am a network engineer (who is inherently rather inept at web designing :), therefore fresh and unbiased external opinions are something I really appreciate.

Can we do the same for the remaining IPs that get "ignored"? To avoid ambiguity, in this screenshot <IP_ADDRESS>

Sure thing, will add that next. Good idea.

Can we remove the unscores from the hyperlinks and open the links in a new window (style="text-decoration: none;" and target="_blank" or something)?

Good point about the underlines, it didn't feel entirely right to me as well. A good compromise would be to make the underline appear only on hover. I disagree about the target=_blank part though: that's perfect for external links (and it already works like that), but placing it everywhere can become a popup/new tab nightmare real fast. After all, a middle click can work around that so that you get the best of both worlds.

Can we change that to
whois, host and ipinfo (and without the chain link icon thing)

I'd like to retain a hyperlink distinction between the whois popup and external links, and the chain link icon was the quickest thing I could come up with without resorting to fontawesome and the likes. Any suggestions on how to distinguish those, and an alternative formatting style (I'm not entirely convinced with the looks either)?

Not sure how much sense it makes to do traceroutes to subnets

Not sure about which part you're referring to, but a traceroute to a subnet (in regard to how asn performs it) is really just a traceroute to the first IP in the subnet, which is consistent with traceroutes to any other IP in the same CIDR block (therefore giving you insight to what the AS path to the target is).
Thanks for your feedback!

Ah, by the way:

ASN number lookup output hurts my brain at the moment

if you refer to the uneven spacing between the lines, that's due to a known bug in aha v0.5, which was fixed in v0.5.1, but some distros still package the older aha version.
If that's the case (and you're using aha v0.5), please try upgrading it, that should fix the issue. I was planning to mention this in the README, but eventually forgot to.

Some time has passed, and we'll stick to the nineties for the time being, I guess. Feel free to reopen if anything comes to mind :-)


[DOCS] Fix formatting in README

Description
The below formatting:

can be changed to:

It decreases nesting of li items and makes the section look more logical.
Record

[X] I agree to follow this project's Code of Conduct
[X] I have checked the existing issues
[X] I'm a GSSoC'23 contributor
[X] I want to work on this issue

Description
The below formatting:

can be changed to:

It decreases nesting of li items and makes the section look more logical.
Record

[x]  I agree to follow this project's Code of Conduct
[x]  I have checked the existing issues
[x]  I'm a GSSoC'23 contributor
[x]  I want to work on this issue

Hi @Sriparno08 this seems like a level 0 issue, can you add more small issue to this to upgrade this to level1 ?

Sure, @narayan954.
We can bold the titles of this section to make them distinct and separate them from their descriptions.

Sure, @narayan954.
We can bold the titles of this section to make them distinct and separate them from their descriptions.

alright, you can go ahead!


When uninstalling hooks, the .committee file is deleted

I think the file shouldn't be deleted but maybe moved to .committee.backup.exs, we can add an option to committee.uninstall like --delete to delete the hooks file.
I did lost my hooks while testing them because before committing it I wanting to just deactivate committee by running mix committee.uninstall.
From my point of you, a config file shouldn't be deleted while uninstalling/removing, instead it depends of the user to do it or not.
Maybe it's just me ^^
Thanks

The idea of uninstall is to fully uninstall committee for the user, leaving a .backup.exs is going to be against that. If anything, a --backup flag to uninstall would be probably more realistic.
But, as for whether or not to remove a config file upon uninstallation, I think that is still up for discussion, though I don't really see the benefits of one vs the other.
However, when you said this:

I did lost my hooks while testing them because before committing it I wanting to just deactivate committee by running mix committee.uninstall.

Can you elaborate what you mean? Do you just want to skip the hook for a certain actions? In that case you can just pass in --no-verify, that should work. The docs could probably use some update in this regard.


respect user provided search limit argument

Scope & Features

Since commit 6685969 we haven't been respecting the { limit } argument in io3d.furniture.search

Design

the limit parameter now uses either the argument provided by the user or falls back to the default of 50

How to use

io3d.furniture.search('chair', { limit: 5 })

How to test

compare io3d.furniture.search('chair', { limit: 5 }) in this branch compared to master

To better explain...
Before this PR:
io3d.furniture.search('office', { limit: 5 })
 // 203 results

io3d.furniture.search('', { limit: 1000 })
 // 500 results

After this PR:
io3d.furniture.search('office', { limit: 5 })
// 5 results

io3d.furniture.search('', { limit: 1000 })
// 957 results

@bnjm @m-ke thx for taking care of this :) 👍
do we have any blockers from also adding the offset param?
means: outcomenting

https://github.com/archilogic-com/3dio-js/blob/master/src/furniture/search.js#L13
https://github.com/archilogic-com/3dio-js/blob/master/src/furniture/search.js#L26

I was just looking at it as well... I don't think there are any blockers there. I'd suggest we coordinate the release tomorrow afternoon.


[Rancher2]  Update AKS docs for AKSv2

Edit 5/12 (cmurphy): config reference should include which fields can be changed after the cluster is created
Epic GitHub issue: https://github.com/rancher/rancher/issues/31045
Points of contact: Michal Jura and Colleen

[ ] Includes AKS permissions needed to create and update a cluster

AKS cluster might take time (hours) to show up in Cluster To register list, based on region. We should mention this in our docs.

@mjura confirmed that the minimum AKS permissions are the same as what was required in v2.5, so no additional docs changes are needed.

@catherineluse we added a new section for EKSv2 / GKEv2 right, we would need to do the same here, permissions were just an additional thing to check.
https://rancher.com/docs/rancher/v2.x/en/cluster-provisioning/hosted-kubernetes-clusters/eks/#changes-in-rancher-v2-5

@cbron The new section is here https://staging.rancher.com/docs/rancher/v2.6/en/cluster-admin/editing-clusters/aks-config-reference/#changes-in-rancher-v2-6
It's just on a different page than before because we figured it would be better to reorganize the docs by putting the cluster configuration references under cluster administration instead of cluster provisioning. The cluster provisioning page still links to the same configuration docs.


training yolo3 with only one GTX 1080

is that possible to train yolo3 with only one GTX 1080 (8G) on my own dataset? likely this is not possible only one GPU 8G , right?

@zeynali No problem at all. You can train in smaller batches or just reduce the min_input_size and max_in_size. Use google collaborator for free K80.

Struggling with GTX 1060 with batch_size = 4.........
So you can do it !!!

@yueyangming , very good , so you've trained yolo3 with 1060 ? what input_size?

@zeynali  I didn't change that input size, just decrease bacth_size, if will work when batch_size is 7 or 6.
My 1060 has 6G memory, i am using ubuntu 16.04, and driver version is 384.111.
Enjoy training : )

@yueyangming , are you result of detection good after training ?

@yueyangming @zeynali  "gpus":                 "0,1", Do you write cofig like this?,because I am not sure my config is right.

If you get an out of memory error try allowing memory growth when you create the model, after the clear_session
import tensorflow as tf
from keras.backend.tensorflow_backend import set_session,get_session
config = tf.ConfigProto()
config.gpu_options.allow_growth = True  # dynamically grow the memory used on the GPU
sess = tf.Session(config=config)
set_session(sess)  # set this TensorFlow session as the default session for Keras

This helped me many times on windows. (and sometime make it worst)


Store endpoint state transitions

[ ] Replace the current per endpoint logs section with a state transition system that keep tracks of state transitions with their cause.
[ ] Move this state transition log to a new endpoint logs command to declutter endpoint get

@tgraf, @aanm and I  discussed this in person. We have a general direction but details will depend on my implementation (they should also correct these notes if they're wrong).
Our goal is to:

Include a health field for an endpoint under cilium endpoint get that contains:

- health: {OK, Bootstrap, Pending, Warning, Failure}
- build: {OK, Bootstrap, Pending, Warning, Failure}
- policy: {OK, Bootstrap, Pending, Warning, Failure}
- connected: {yes, no}

These fields are intended to provide a user with a quick representation of the endpoint's state. They are not states themselves, but should provide a definitive status. For example: connected: no means the endpoint is not connected. This could be because of a variety of things, but a user should be able to check this first while diagnosing a connectivity problem.

We will interleave the current endpoint status logs and state transitions (with transition reason) into a single linear log that you can see with cilium endpoint logs. The API itself will export JSON but CLI will default to a grepable 1-line scheme. This log is a debugging resource and should contain enough information to understand how an endpoint arrived at a certain condition.

Solved via #2060


Please add the thing in tachij2k

Describe your suggested feature
Where you cant outzoom the manga and only zoom in, you know like it does not goes out from the panel.
Other details
No response
Acknowledgements

[X] I have searched the existing issues and this is a new ticket, NOT a duplicate or related to another open or closed issue.
[X] I have written a short but informative title.
[X] If this is an issue with an official extension, I should be opening an issue in the extensions repository.
[X] I have updated the app to version 0.16.0.
[X] I will fill out all of the requested information in this form.

Sorry i was writing this in mihon preview and just notice that


Deprecate singleton usage

Now that instances of Shortcode can be created singleton usage should be deprecated.
@toddnestor are you happy all of the cases that need warnings have been covered by this commit https://github.com/kernow/shortcode/commit/8f539c8110d25d50e331c57fa4dca24a274dd16c

Yeah, it looks good to me. Thanks for doing that and for getting a new release out! I can update the CMS at my company now!

I am getting a [DEPRECATION] on each Rails start, I think it is thrown by Shortcode itself:
https://github.com/kernow/shortcode/blob/master/lib/shortcode.rb#L68

@krisdigital yes it will show a deprecation warning if you have upgraded to the latest version.  You could force the previous version in your Gemfile if you don't want to see the deprecation warnings.
The deprecation warnings are there to let you know the singleton usage of the Shortcode gem is being deprecated in 2.0 and so if you upgrade to that version without changing your code it will stop working for you.

@krisdigital @toddnestor I'm going to try and work on the 2.0 release over the next week or 2. If you follow the guide in the readme on usage (by creating an instance of Shortcode) then you should not have any issues upgrading to the 2.0 release.

@toddnestor ok, but why do I get warnings if I do not use the singleton anymore? warning everybody even if he has already updated seems a bit overkill to me..

I feel like there should be some way to turn off the deprecation message if my code isn't actually using the singleton. I just searched through my codebase trying to find where I was doing so just to realize the deprecation message shows regardless of any usage.

I've released 2.0.0.pre today which completely removes the singleton code so you won't get any deprecation warnings. If you could test it out and check it is working for you that would be great.

@kernow just pulled it down, and it is working for me, and I'm no longer getting the deprecation warning 👍  Would you consider this version safe for production or should I wait for 2.0.0 to come along?

@stevehanson glad it's working for you. Re: production ready, it depends on you're attitude to risk, I'm not aware of any issues and in terms of code changes you can see that the main 2.0 change is removing the singleton support. I don't see why you would experience any problem but I wanted to have a prerelease period incase anyone has issues upgrading.

Thanks, @kernow! I'll probably upgrade in the next couple weeks. 👍


feat: Uri activation with path

allows to activate the app via uri and specify a path.
files-uwp:///C:/windows

The drive letter must be uppercase
forward slashes / must be used.

@lampenlampen Seems like your patch get function "Open in new windows" broken. Please, check it.

Don't we already support it using files?

@tsvietOK Will have a look. Thanks 👍
@yaichenbaum We only support the commandline with files.exe -Directory "" and to start the app with the uri files-uwp:. This adds support to open the app in a folder via uri files-uwp:///C:/users.
This is useful for other uwp apps, because they can then start files with Launcher.LaunchUriAsync(), because starting another process is prohibited by default.

Okay now i learned, we already support this 🤣.


Question: do you notify contributors when their contribution first takes flight?

I'm thinking of finding something simple to fix for fun, and this would be a huge motivation 😄

Good question/suggestion!  I see if we can make something like that happen, at least for missions we know about!


Placeholder for empty cells

Traditionally, I'd rather show a specific placeholder in empty cells to distinguish between absent and a zero value. However, using a string in a column that's been designated as float obviously derails precision management. The only nonfloat supported in those columns is None. Is there a way of telling Tabulate to render a certain character/string when it encounters a None, instead of an empty string?

Bump.

Moved to https://github.com/astanin/python-tabulate/issues/311


Support ephemeral-storage requests and limits on projects, namespaces and workloads

What kind of request is this (question/bug/enhancement/feature request):
Enhancement
Description:
I want to be able to use Rancher to set default ephemeral-storage request and limits on my workloads.
Running Rancher 2.4.3 it is not possible to assign ephemeral-storage requests/limits to projects, namespaces or workloads. I would like to be able to do this in a similar fashion as CPU, Memory and NVIDIA GPUs (all of which are already supported).
Ephemeral-storage has been in beta since at least Kubernetes 1.10: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#local-ephemeral-storage

Still not possible with rancher 2.5.7.


Refine Winery user guides

Signed-off-by: Ana Cristina Franco da Silva<EMAIL_ADDRESS>The following changes were done:

content of 'modeling guide' was merged into the 'quickstart'

a 'quicksetup' guide was added, which contains now the configuration information previously existent in the 'quickstart'

FAQ was expanded

documentation to Implementation Artifact generation was added

[x] Ensure that you followed http://eclipse.github.io/winery/dev/ToolChain#github---prepare-pull-request. Especially, we require a single commit

[ ] Ensure that the commit message is a good commit message

[ ] Ensure to use auto format in all files

[x] Ensure that you appear in NOTICE at Copyright Holders

[ ] Tests created for changes

[x] Documentation updated (if needed)

[ ] Screenshots added (for UI changes)

Can one of the admins verify this patch?


Mainly to handle non-ASCII user names, also some tidy-ups / unused code removed

Pull Request Test Coverage Report for Build 124

1 of 53 (1.89%)  changed or added relevant lines in 2 files are covered.
5 unchanged lines in 2 files lost coverage.
Overall coverage decreased (-1.2%) to 68.578%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

client_converter_callback.py
0
8
0.0%

webhook.py
1
45
2.22%

Files with Coverage Reduction
New Missed Lines
%

aws_tools/dynamodb_handler.py
2
77.67%

webhook.py
3
13.29%

Totals

Change from base Build 119:
-1.2%

Covered Lines:
2440

Relevant Lines:
3558

💛  - Coveralls


Add support for HiP-CT

Dear BIDS community
Hierarchical Phase-Contrast Tomography (HiP-CT) is an ex vivo X-ray technique that leverages high-energy X-ray sources -- specifically, the ESRF's Extremely Brilliant Source (EBS).  HiP-CT is non-destructive, reaches resolutions of 1 to 10 um, and allows specific ROIs to be imaged within the sample without physical sectioning. Although the modality is obviously not widely available, the UCL group has a relatively streamlined acquisition process and we will be collecting data on multiple brains in the coming years.
We would like to add support for HiP-CT in BIDS, and we think it most naturally fits as a new modality suffix within the microscopy extension. This makes sense because micro-CT is already supported as a microscopy suffix. Also, the microscopy extension already includes tags that are required by HiP-CT data (for example, the existence of multiple images of the same samples, potentially with different resolutions and different field-of-views, labeled as chunk-). We are therefore proposing to add the hipCT suffix to the microscopy extension.
We have uploaded an exampled dataset that follows the proposed convention to DANDI:
https://dandiarchive.org/dandiset/000026/draft/files?location=sub-I58%2Fses-Hip-CT%2Fmicr&page=1
We also provide three neuroglancer links that allow each volume to be visualized online, directly streaming the dandi data:

sub-I58_sample-01_chunk-01_hipCT.ome.zarr : left hemisphere @ 15.13 um
sub-I58_sample-01_chunk-02_hipCT.ome.zarr : Broca's area @ 6.21 um
sub-I58_sample-01_chunk-03_hipCT.ome.zarr : Brainstem @ 6.21 um

Let me also include a typical metadata file provided by the UCL group:
############# DOI ###################

Title		= "U.N."
Abstract	= "U.N."
Users		= "U.N."

############ proposal ###############

proposalTitle   ="U.N."
proposalNumber  ="md190"
proposers       ="U.N."
experimentType  ="U.N."
technique       ="U.N."
beamlineName    ="BM18"
instrumentName  ="U.N."

############ patient ################

PatientInstitue ="U.N."
PatientNumber   ="I58-LH"
PatientAge      ="U.N. years"
PatientSex      ="U.N."
PatientWeight   ="U.N. kg"
PatientSize     ="U.N. cm"
PatientMedInfo  ="U.N."

########### sample ################

SampleName  ="I58-LH_brain"
SampleInfo  ="complete heart from the body donor program of the Laboratoire d'Anatomie des Alpes Francaise (LADAF)"
SamplePrep  ="formalin fixed, progressive transfer to ethanol 70%. Vacuum degassing at each transfer to new ethanol bath, mounted with agar crushed gel at 70% ethanol"

########### scan parameters ###########

StartTime       ="20/01/2023"
SRCurrent       ="200mA mA"
IDName          ="BM18"
ScanRadix       ="HA-2300_15.13um_I58-LH_brain_C"
XStep           ="U.N."
XStages         ="U.N."
YStep           ="U.N."
YStages         ="U.N."
ZStep           ="5"
ZStages         ="32"
Projections     ="15000"
RefN            ="U.N."
DarkN           ="U.N."
RefOn           ="U.N."
ScanningMode    ="U.N."
ExposureTime    ="0.014 s"
AccExposureTime ="0.07 s"
AccFramesCount  ="5"
PixelSize       ="15.13 um"
PropDistance    ="15000 mm"
Filters         ="Sapphire 10mm SiO2 15mm"
Det_Avg_Energy  ="90 keV"
ScanGeometry    ="half-acquisition"
ScanRange       ="360 deg"
SensorName      ="IRIS15"
SensorMode      ="U.N."
SensorPixelSize ="4.25 um"
Magnification   ="0.3"
XPixelNum       ="5046"
YPixelNum       ="400"
OpticsType      ="Dzoom"
Scintillator    ="LuAG 2000 reflective"
SurfDoseRate    ="26.9 Gy/s"
ScanTime        ="20"
SeriesTime      ="10.7"

######### processing ############

RefApproach     ="U.N."
VolumeX         ="9413"
VolumeY         ="9413"
VolumeZ         ="10656"
32to16BitsMin   ="-0.17"
32to16BitsMax   ="0.31"
Jp2ComprRatio   ="10"

And the corresponding BIDS json file
{
	"IntendedFor":"sub-I58_sample-01_chunk-01_hipCT.ome.json",
	"Manufacturer":"ESRF",
	"StationName": "BM18",
	"InstitutionName":"European Synchrotron Radiation Facility",
	"InstitutionAddress":"71 Av. des Martyrs, 38000 Grenoble, France",
	"PixelSize":"[15.13, 15.13, 15.13]",
	"PixelSizeUnits":"um",
	"OtherAcquisitionParameters": "{\"StartTime\": \"20/01/2023\", \"SRCurrent\": \"200mA mA\", \"IDName\": \"BM18\", \"ScanRadix\": \"HA-2300_15.13um_I58-LH_brain_C\", \"ZStep\": \"5\", \"ZStages\": \"32\", \"Projections\": \"15000\", \"ExposureTime\": \"0.014 s\", \"AccExposureTime\": \"0.07 s\", \"AccFramesCount\": \"5\", \"PixelSize\": \"15.13 um\", \"PropDistance\": \"15000 mm\", \"Filters\": \"Sapphire 10mm SiO2 15mm\", \"Det_Avg_Energy\": \"90 keV\", \"ScanGeometry\": \"half-acquisition\", \"ScanRange\": \"360 deg\", \"SensorName\": \"IRIS15\", \"SensorPixelSize\": \"4.25 um\", \"Magnification\": \"0.3\", \"XPixelNum\": \"5046\", \"YPixelNum\": \"400\", \"OpticsType\": \"Dzoom\", \"Scintillator\": \"LuAG 2000 reflective\", \"SurfDoseRate\": \"26.9 Gy/s\", \"ScanTime\": \"20\", \"SeriesTime\": \"10.7\"}",
	"BodyPart":"Brain",
	"BodyPartDetails":"Left Hemisphere",
	"SampleEnviornment":"ex vivo",
	"SampleEmbedding":"agar crushed gel at 70% ethanol",
	"SampleFixation":"formalin",
	"SampleExtractionInstitution":"Massachusetts General Hospital"
}

Currently, the HiP-CT acquisition parameters are stored in the "OtherAcquisitionParameters" field, as a serialized JSON dictionary. But it may be better to move some of these HiP-CT-specific tags into the BIDS spec, and we would welcome any feedback from users of the other microscopy modalities.
@satra @Kevancic @dstansby
Thanks
Yael

out of curiosity would it not be an issue if we one day wanted to integrate CT in BIDS:
see https://bids.neuroimaging.io/bep024 ?

This is a good question.
I am a bit wary of having HiP-CT as a special case of CT for several reasons:

It is a phase-based contrast -- not the typical absorption-based contrast used in CT
It can only be acquired on ex vivo tissue, which makes it closer to a microscopy technique than an in vivo modality
Its very high resolution and ginormous size means that we must use the zarr format, which is only accepted in the standard under micr/, as far as I understand

That said, I am neither a CT nor a HiP-CT expert, let me try to get someone who knows what they are talking about to chime in.

Hi all, thought I'd jump in here to add some extra info regarding HiP-CT. @balbasty is correct for all the above. I think to summarise and add some of my own  reasons to separate HiP-CT from CT:

HiP-CT is ex vivo only
It replies of propagation based phase-contrast
It has a hierarchical resolution format, i.e. the whole organ is imaged at an overview resolution and then specific areas can be imaged at higher resolution creating multi-scale datasets of single organ
As mentioed datasets are large, some of our largest aquired brain data sets are now at 14TB for one sample.
HiP-CT is a synchrotron technique, meaning that the meta-data fields for both acquisition and also reconstruction will be quite different and varied compared with a clinical or even a lab based CT system.

Also there is more detail on the technique in this paper , and you can see some video that give a clearer idea of the technique here.

OK it feels pretty clear from what you are saying that this is quite different from clinical CT.
From what I understand it looks like adding a microscopy suffix is needed to start supporting this.

Currently, the HiP-CT acquisition parameters are stored in the "OtherAcquisitionParameters" field, as a serialized JSON dictionary.

In many cases BIDS lets you add extra metadata, so no need to to have it serialized however better in this case to start listing the metadata that will be REQUIRED for downstream analysis and add those to the bids spec

@balbasty - do you want to start a PR to the spec for this?

I would not mind also seeing an example added to the bids example repo: https://github.com/bids-standard/bids-examples#contributing

looks like adding a microscopy suffix

you mean under the micr prefix. the suffix is just hipCT as proposed right?

seeing an example added to the bids example

perhaps a downscaled version for the moment could work. may be just at the coarsest levels. one of the the images that @balbasty linked to has 1 million objects :)

Sure I can draft a PR.
The one thing I am not sure yet is which metadata should be added (if any). @ucbpclw do you think any of the metadata should be REQUIRED or RECOMMENDED to store in the sidecar json file?
And don't worry I won't push terabytes of data to the example repo! Something like the 1mm resolution level would make sense I'd say.

I don't think we inspect the contents of zarr directories. You could drop everything but any metadata files inside that may need to be inspected to ensure consistency between OME and BIDS.

Do you mean which of metadata fields we sent in our example file are essential?
C

From: Yaël Balbastre @.>
Sent: 08 November 2023 16:57
To: bids-standard/bids-specification @.>
Cc: Walsh, Claire @.>; Mention @.>
Subject: Re: [bids-standard/bids-specification] Add support for HiP-CT (Issue #1646)
⚠ Caution: External sender
Sure I can draft a PR.
The one thing I am not sure yet is which metadata should be added (if any). @ucbpclwhttps://github.com/ucbpclw do you think any of the metadata should be REQUIRED or RECOMMENDED to store in the sidecar json file?
And don't worry I won't push terabytes of data to the example repo! Something like the 1mm resolution level would make sense I'd say.
—
Reply to this email directly, view it on GitHubhttps://github.com/bids-standard/bids-specification/issues/1646#issuecomment-1802291066, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ADEWAE6FHY2WTDKDIKYSERLYDO2V3AVCNFSM6AAAAAA6WEWN66VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMBSGI4TCMBWGY.
You are receiving this because you were mentioned.Message ID: @.***>

to illustrate what @effigies said, here are the content of the microscopy examples:

@ucbpclw

Do you mean which of metadata fields we sent in our example file are essential?

Yes, exactly

Hi
I am reopening this thread. After further discussions with "synchrotron-CT" experts (@chourroutm, @dstansby, @ucbpclw), it appeared that hipCT might not be the most adequate suffix. The "H" means hierarchical and really relates to the acqusition of multiple images in the same sample. Each individual image is a (very fancy) phase-contrast CT.
They have suggested "PC" for phase-contrast, although in the current spec, "PC" relates to optical phase-contrast, not xray phase-contrast.
It might be good to fix this suffix before the next releast of the spec.
@satra @yendiki @effigies @yarikoptic

XPC - X-ray Phase Contrast?

Adding the X does make it sound cooler.

From: Chris Markiewicz @.>
Sent: Friday, April 26, 2024 7:53 AM
To: bids-standard/bids-specification @.>
Cc: ayendiki @.>; Mention @.>
Subject: Re: [bids-standard/bids-specification] Add support for HiP-CT (Issue #1646)
    External Email - Use Caution

XPC - X-ray Phase Contrast?
—
Reply to this email directly, view it on GitHubhttps://secure-web.cisco.com/1RJ4VJxNzwdOx__6K5a5aSKoHEBWWcoRNu31bi88izNWrHHlE3pEACHrvA_Imz4Pkx1iLs-17wyg7mQvMFnJWhnLtvMqlO41J4qyb4pkww2b_4FTUR0H9rG14DE4L4JpbPXj26a7I6zAohXT7LwH_OB3GjIc9VCcTsX2c-s3GXOW2nIADNpeBGa3dEWYGrzkut9SxcD3TCXPTPzTZSqkQbRx723V9_nLpGsqNIU6WMmz7tFfqyVv_J4DH-QqDwwCBj3xTXzmNHbTRy4pg2BSCkL-Xiu7MJy5-RTOiUFYVZ8vbrBnyqH8EBAwOuGkC4R1sfupd-z0-utxIOivmv7eMpQ/https%3A%2F%2Fgithub.com%2Fbids-standard%2Fbids-specification%2Fissues%2F1646%23issuecomment-2079237706, or unsubscribehttps://secure-web.cisco.com/1oGFgqJFwUAtlKAwMqRdsiqqCI3yVq4UA5x61nHovlUGc0-Kg19k0ttBr4m13I2Ox1ck8HvWzRv-vopF23Ij0-Ne9WFr9hFYbDXzSPsyL_wExaY0_l9pyZubBJFltOqdTJp1V-hOn4wsL8xq_rWdaTMqoJr3Dh0hprk7CxcZ1CZNd9juHILZV_xFy8mDlxU_mU_PJDnwBSVOaKSlQOY947ZXM-860g0tO_4aw-m6-l6lKKiTan9p-3fhJ__llZGvpxIvNkNECNhDEeY_p0hlHAZ9sEJU2FSZaqWQt620w07GIbfP7T6wIOxM3Z7RVW9AWJpBy0_ymvfOdoixmwiOAzg/https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FADU34VYLIELDK3ILVD5V3ETY7I52XAVCNFSM6AAAAAA6WEWN66VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANZZGIZTONZQGY.
You are receiving this because you were mentioned.Message ID: @.***>
[ { @.": "http://schema.org", @.": "EmailMessage", "potentialAction": { @.": "ViewAction", "target": "https://github.com/bids-standard/bids-specification/issues/1646#issuecomment-2079237706", "url": "https://github.com/bids-standard/bids-specification/issues/1646#issuecomment-2079237706", "name": "View Issue" }, "description": "View this Issue on GitHub", "publisher": { @.": "Organization", "name": "GitHub", "url": "https://github.com" } } ]
The information in this e-mail is intended only for the person to whom it is addressed.  If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at https://www.massgeneralbrigham.org/complianceline https://www.massgeneralbrigham.org/complianceline .
Please note that this e-mail is not secure (encrypted).  If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately.  Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail.

Agree that it's worth dropping the "Hi" of "HiP-CT" - as mentioned above stands for Hierarchical which is about how mutiple datasets/scans relate to each other (basically some are higher resolution VOIs of others).
XPC sounds good. Does the suffix need to indicate it's CT data though? I note there's already a microCT suffix in the spec. So how about something like xPCCT? Or just PCCT for phase contrast computed tomography?

XPC sounds good. Does the suffix need to indicate it's CT data though?

The only hard and fast rule is that it needs to be alphanumeric. Otherwise, if the affected community agrees that it's precise enough, and people outside the community don't see it conflicting with something familiar to them, then it's all good.

I note there's already a microCT suffix in the spec. So how about something like xPCCT? Or just PCCT for phase contrast computed tomography?

I don't love xPCCT, but XPC and PCCT seem equally fine to me. But I don't work with these data, so I'm happy to leave it to those who do.

If there's no particular standard/rules for what has to be in the suffix, apart from understandable by the community, then maybe it's worth keeping this as hipCT?

@dstansby I'd rather go for a broader name, as we're the only group/consortium doing HiP-CT (at the moment) but the community using the same underlying technique --- without the hierarchical component, which indeed rather corresponds to a dataset rather than a sole volume --- is already quite developed .
The different names I am aware of are:

X-ray phase-contrast tomography / XPCT, doi:10.1016/j.neuroimage.2018.09.044
X-ray phase-contrast computed tomography / XPCCT, doi:10.1088/1361-6560/acd48d
X-ray phase microtomography, doi:10.1038/srep00826
Synchrotron X-ray computed tomography / SXCT, doi:10.1038/s41598-022-21741-z
Phase-contrast X-ray imaging / PCXI, doi:10.1038/s41598-018-29841-5
X-ray phase-contrast imaging / XPCI, doi:10.3390/app12199539
Phase-contrast X-ray synchrotron microtomography / PCXSμT, doi:10.1371/journal.pone.0229140
X-ray tomographic imaging / XTI, doi:10.1007/s00418-020-01868-8

As there is no real consensus of the naming within the community, we will not please everyone, of course. It seems to me that XPC might be sufficient as a suffix to the microscopy keyword. Otherwise, to convey the 3D, maybe XPCT is the one to go.

What's the opinion of the others? @ucbpclw @balbasty @satra
My former collaborators were also interested in such an addition, I can include them in the decision-making.
Then, what is the course of events? Should I draft a PR with one of the two suffices?
I suppose we can prepare in the following months (once the validator recognizes the suffix) a truncated version of the open brain dataset from the Human Organ Atlas: https://human-organ-atlas.esrf.eu/datasets/572252538

@chourroutm  I vote XPCT, but yes it'd be great to have a validation from other groups, so that it lives up to the test of time :)
Once it's ok'd, you (or I) should revert the changes from PR #1686 and replace all occurences of hipCT by XPCT.

It would be good to get an example in https://github.com/bids-standard/bids-examples as well, so that we can update and test the validator. See the contributing guide.

I would vote XPCT as well, it is the wording we are generally using when writing grants as it captures all the essential components of the technique and is specific enough to differentiate it from other X-ray or tomography examples.

@gobej and @embrun do you want to weigh in?

@effigies I've tailored an example dataset in chourroutm/bids-xpct-example/tree/master/micr_XPCTzarr that we'll amend before I draft a PR to bids-standard/bids-examples.
@ucbpclw @dstansby and eventually @balbasty can I have your inputs on the dataset? This is the file structure of the example dataset:
micr_XPCTzarr
|   dataset_description.json
|   participants.json
|   participants.tsv
|   README.md
|   samples.json
|   samples.tsv
|
+---code
|       data_retrieval.py
|       data_retrieval_reqs.txt
|       metadata_formatter.py
|       metadata_formatter_reqs.txt
|       truncate_data.sh
|
\---sub-LADAF-2020-31
    |   sub-LADAF-2020-31_sessions.json
    |   sub-LADAF-2020-31_sessions.tsv
    |
    \---ses-01
        \---micr
            |   sub-LADAF-2020-31_ses-01_sample-brain_photo.jpg
            |   sub-LADAF-2020-31_ses-01_sample-brain_photo.json
            |   sub-LADAF-2020-31_ses-01_sample-brain_XPCT.json
            |
            \---sub-LADAF-2020-31_ses-01_sample-brain_XPCT.ome.zarr
                |   .zattrs
                |   .zgroup
                |
                +---0
                |   |   .zarray
                |   [...]
                [...]
                |
                +---4
                    |
                    [...]
                    |
                    \---9
                        +---0
                        |       0
                        |       1
                        |       2
                        |
                        +---1
                        |       0
                        |       1
                        |       2
                        |
                        \---2
                                0
                                1
                                2

Layout looks good to me (FWIW I sucessfully adapted a HiP-CT dataset to BIDS to upload to LINC), but I think that's beyond the scope of this issue (which is just about the prefix)?

#1822 changing hipCT to XPCT has been open a week with no objections. I would ask someone else to leave an approving review and encourage anybody who would object to do so as soon as possible.

I am also in favour of "XPCT".

XPCT adopted in #1822.


How I can increase sample rate?

Hello, thanks for this project it works very well with a 1-second sample rate...
I do this by adding a delay(1000) I am monitoring 10 channels and I got 10 channels plotted.
When I set delay(100) I am getting channels duplicated by 10 and the plotter starts drawing 100 channels I guess each channel 10 times.
Is it possible to change the sample rate of the plotter?

I guess I am having exactly the same issue mentioned here, I am running this on Windows 11
https://github.com/yhur/arduplot/issues/4

Hi,
As you may know, this uses the animation library from matplotlib.
So the possible minimal plotting granularity would depend on the host computer’s performance regarding on the python and the associated libraries like the matplotlib.animation. And it would add another time constraints if you run it through PIO since the additional TCP communication layer as well as the PIO would add some more latency.


[Content]: Missing Analytics Rust doc

What article on segment.com/docs is affected?
Analytics-rust
What part(s) of the article would you like to see updated?
There is no documentation for analytics-rust even though the github points to our docs twice.

https://segment.com/docs/connections/sources/#server
https://segment.com/docs/libraries/rust/

Additional information
No response

@stayseesong Can you check with the libraries team to see if there is any plan for these libraries? We have documentation for Rust, but it's hidden and unpublished.
We should either update and make public, or remove the link from the GitHub repo.

@markzegarelli the team said Rust is not officially supported, and while the library exists, it's not actively maintained. I got the ok to just remove the links from the repo. I removed https://segment.com/docs/libraries/rust/ and submitted a PR to update the readme to remove the other docs link

Sweet, thanks for the quick turnaround @stayseesong. I will close this one out.


Op 7611

We prefer small, well tested pull requests.
Please refer to Contributing to Spinnaker.
When filling out a pull request, please consider the following:

Follow the commit message conventions found here.
Provide a descriptive summary for your changes.
If it fixes a bug or resolves a feature request, be sure to link to that issue.
Add inline code comments to changes that might not be obvious.
Squash your commits as you keep adding changes.
Add a comment to @spinnaker/reviewers for review if your issue has been outstanding for more than 3 days.

Note that we are unlikely to accept pull requests that add features without prior discussion. The best way to propose a feature is to open an issue first and discuss your ideas there before implementing them.
https://devopsmx.atlassian.net/browse/OP-7611

This PR  triggered this build in this execution, please merge to master only if this succeeds:  https://spin.prod.opsmx.com/#/applications/gitprtrigger/executions/details/01FKJBVC2YH9QTA572P1GSCP90


Variable Already defined error

I'm using 2.9.4
I’m trying to run the wsdl generation tool against a particular few wsdls and I’m getting a strange error.  Any way you might be able to shed some light on this.  This is the command I'm trying to
C:\WsdlToPhp2>c:\php_7.1\php.exe ./wsdltophp.phar generate:package
--urlorpath="http://website:8113/change/xverification?wsdl" --destination="C:\serverpath\misc_wsdls\2018.00"  --composer-name="myproject/mypackage" --namespace="Verify" --composer-settings="config.disable-tls:true" --validation=false --force

Here is the output:
Start at 2018-11-01 22:22:28
PHP Fatal error:  SOAP-ERROR: Parsing Schema: attribute 'http://www.change.generated.domain.com/v1/common/dto/common:ProcessorUser' already defined in
phar://C:/serverpath/WsdlToPhp2/wsdltophp.phar/vendor/wsdltophp/packagebase/src/AbstractSoapClientBase.php on line 81
PHP Fatal error:  SOAP-ERROR: Parsing Schema: attribute 'http://www.change.generated.domain.com/v1/common/dto/common:ProcessorUser' already defined in
phar://C:/serverpath/WsdlToPhp2/wsdltophp.phar/vendor/wsdltophp/packagebase/src/AbstractSoapClientBase.php on line 81

In GeneratorSoapClient.php line 33:

  Unable to load WSDL at "http://domain:8113/change/xverification?wsdl"!

In AbstractSoapClientBase.php line 81:

  SOAP-ERROR: Parsing Schema: attribute 'http://www.change.generated.domain.com/v1/common/dto/common:ProcessorUser' already defined

Any clue as to what is causing this issue?
Thanks!

Having the WSDL would be easier to be able to debug this but I currently think that this is due to the WSDL that is not well formated (as you certrainly suspected).
Otherwise, it is a feature that is not handled correctly by the native PHP SoapClient class for which I can't really help.
My advice would be to download the WSDL locally, then remove the duplicated attribute in order to try to generate the package with the fix.
Let me know then,

I did just that late last night and finally got them to generate.
What's weird is the SOAPUI tool was able to open them up successfully but PHP barfs completely.  I assume that SOAPUI is allowing the WSDL to have a few nuances and still be ok.  Frustrating though.
After regenerating the files, things seem a bit broken for the code we have in place, so I'm working on it today and making sure it's not something else related to this.   I'll report back once I know more.

Hmmm.... Well, although I was able to download the WSDL and make the correction that fixed the issue so the tool would work.  Now each time I setup a soap connections using:
  $get = new \Account\ServiceType\Get($options);
  $create = new \Account\ServiceType\Create($options);

Each of these seems to be taking 3.5 seconds to load and it's the same URL path.
I'm certain it didn't take that long before.  Does it no longer cache the WSDL ?  Or do you think something else might be going on?

Strangely, I copied and pasted the old set of WSDLS to php classes from a backup before I generated new files yesterday (as these services got an upgrade from the vendor), and they work very fast again for the services that were unaffected by the original problem above, while of course missing new methods.  Is there something that needs to be configured in a newer version of your tool that could be preventing the soap client from using a wsdl cache?  That's pretty much what it feels like is happening (and I see the cache being created regardless, but it seems like it's not actually using it), or  can you think of any other thing that could make a new version be so much slower.

It depends on the schemas associated to the WSDL if there is any and if the network is easily reachable. There is not difference concerning the SoapClient usage except that the WSDL_CACHE is working well since versions 2.0.1 and 1.0.12 of PackageBase project.

What is odd is that the last time I went to build new wsdls, we fixed an issue and I ran the wsdls with version 2.5.0.
The latest version seems to be doing something different.
Do you have the WSDL_CACHE as a setting that I should be setting somewhere, or is it defaulted on?

It uses the default settings from the AbstractSoapClientBase.php Line 132 to Line 155

Well, looks like in the past I never had to set the WSDL_CACHE in the options for this to work.  I must have generated this particular large webservice earlier than I thought, prior to you adding that option.
I found that you are manually setting it to 'not cache' as the default, so after setting it to WSDL_CACHE_BOTH on all of my SOAP calls, it is now responding fast again.  Thanks for pointing me to that code.  That was most helpful.
So, back to the original issue with the poorly modified schema ... while downloading the wsdl and manually tweaking the code in it and generating the WSDLtoPhp classes completed successfully, when I try to go to use the generated WSDLs now the only issue I'm having is that even though your tool is working, when PHP seems to still be trying to interface with the WSDL as it gets the same error your tool was giving me earlier.
SOAP-ERROR: Parsing Schema: attribute 'http://www.change.generated.domain.com/v1/common/dto/common:ProcessorUser' already defined...

Is there a way, to make PHP bypass the WSDL at the endpoint, and just use the classes method in the project and send and receive the SOAP message until I can convince the vendor to fix their schema?

Try the non-WSDL mode maybe? If it can be done and works, let me know as I recently answered to another issue that it would not be possible :disappointed:

From the php website, it looks like the non-wsdl mode requires you to set a location and a uri as optional parameters, and the wsdl parameter in the soapClient as a null.  But the current project doesn't allow for the uri to be set, but just the location.  I think that may be why it won't currently work.  Otherwise, maybe it could.

Is that something you plan to add?

I did not plan to add it no until you surfaced it. Can you then create the issue for the PackageBase proiject as it's not related to this project anymore?
Thx

Can you point me to the PackageBase project ?  I'm not sure where that exists?

This is https://github.com/WsdlToPhp/PackageBase


unix timestamp to datetime

use system api DateTime.fromMicrosecondsSinceEpoch Not accurate enough
version flutter_windows_v1.8.3-beta

Please specify the issue, what do you mean by 'Not accurate enough'?

请说明问题，“不够准确”是什么意思？

请说明问题，“不够准确”是什么意思？

Please specify the issue, what do you mean by 'Not accurate enough'?

Beijing time is different after formatting date


the Cache

why do we need three different policies compiled into the code when only one is going to be called?

updates:

1 policy
cache tests enabled
tests dependent on cache and block size


chore: 👷 Add Husky

Each commits get autoformatted with prettier,
could be used for other things.
https://typicode.github.io/husky

I sneaked 6d2b3daf644738d4ef9c5f2dd38c46840eb0220b in, it's not really related but it was to avoid another PR

@melMass i pulled in pretty-quick, this made it a bit easier to use the same file pattern in the hook as in the npm script.
i also changed the npm script from npm run prettier to npm run format, think that's a more convenient name and added .scss to the list of supported files.

Awesome! Very nice!
I think we can merge?


Italian 3.23

Uploaded italian global.ini provided by https://github.com/xGattoMattox/StarCitizen-Localization/commit/d7ae9eb168fc1814133dcc6251b74878c6e39ed2#commitcomment-141715096
Fixed duplicate entries

I need to fix a lot of placeholders. Progressed a lot but as my Italian is not that great I still may need some time. Hopefully finish later today

@xGattoMatto, I had to update a few entries to pass validation and also removed some duplicates, which was quite a task given my limited Italian skills! Nonetheless, I truly appreciate your dedication to the project. If you'd like further assistance with GitHub, feel free to join our Discord server here https://discord.gg/Gbvz9fTmZU. Looking forward to seeing you there!

@xGattoMattox, I had to update a few entries to pass validation and also removed some duplicates, which was quite a task given my limited Italian skills! Nonetheless, I truly appreciate your dedication to the project. If you'd like further assistance with GitHub, feel free to join our Discord server here https://discord.gg/Gbvz9fTmZU. Looking forward to seeing you there!


Can't dynamically add a node inside another node

I don't know how to add node inside another node.
would you like to let me know about it?
thanks

I have tried as following.
addNode(node, index) {
node.data.children.push({
hasChildren:false,
name: "node3.3",
subTitle: "new and improved"
});
}

Same problem on my side.


Broken Link

Your link to your code came up as an error due to text entry. Hopefully this will link it properly.

Hope this clears up issues


Consider expanding SyntaxItemList<T> and SeparatedSyntaxItemList<TElement, TSeparator> API surface

We can at least expose Span and FullSpan properties, as well as ToString() and ToFullString() methods.
This begs the question, though: Should we also expose GetText() and GetFullText()? That would require a Parent property. But then that might signal that the list is a node in the tree, which is not actually the case. We'd then also have to consider tree traversal methods, at which point these lists start to look an awful lot like SyntaxItems in their own right...
Need to think on this one.

Somewhat related: LeadingTrivia and TrailingTrivia on SyntaxToken should probably be changed to SyntaxItemList<SyntaxTrivia>.

This begs the question, though: Should we also expose GetText() and GetFullText()? That would require a Parent property. But then that might signal that the list is a node in the tree, which is not actually the case. We'd then also have to consider tree traversal methods, at which point these lists start to look an awful lot like SyntaxItems in their own right...

Decision: Tree property, Parent property, Span/FullSpan properties, ToString()/ToFullString() methods, GetText()/GetFullText() methods. No tree traversal methods. In the future, we could consider extracting these into an interface if there's a use case for that.
(I think this will also allow us to reduce the amount of SetParent() calls we emit in generated node classes.)


fuzz test: Prevent codec_impl mock from closing stream twice.

Commit Message: fuzz test: Prevent codec_impl mock from closing stream twice.
Additional Description:
codec_impl_fuzz_test tried to close a request stream that had already been
closed. Now taking the stream's state of the response into account before
closing.
Signed-off-by: Andre Vehreschild<EMAIL_ADDRESS>Risk Level: low
Testing: n/a
Docs Changes: n/a
Release Notes: n/a
Platform Specific Features: n/a

CC @adisuissa @KBaichoo @pradeepcrao


downgrade minimum perl version and set Type::Params minimum version

all tests pass on perl 5.24.1 after updating Type::Params from 1.002001 to 1.004004, former version failed with:
$ PERL5LIB=../Type-Tiny-1.002001/lib TEST_METHOD=tie perl -MCarp::Always -It/unit t/run_unit_some
1..2
ok 1 - use Google::RestApi::SheetsApi4::Worksheet;
not ok 2 - worksheet (for test method 'tie') died (Can't locate object method "parents" via package "post_process" (perhaps you forgot to load "post_process"?) at ../Type-Tiny-1.002001/lib/Type/Params.pm line 307.

Thank you for the testing!


Tutorials: Could not create sample application with rust

Hello!
i'm trying to create rust application like it described in your tutorial
https://sdk.dfinity.org/docs/quickstart/network-quickstart.html
so, i have started local network with dfx start
i have updated application created with rust_hello command like it described in guide
but after running dfx deploy command i received next error
crawter@PC:~/rust_hello/src/rust_hello$ dfx deploy
Deploying all canisters.
Creating canisters...
Creating canister "rust_hello"...
Creating the canister using the wallet canister...
Creating a wallet canister on the local network.
The wallet canister on the "local" network for user "default" is "rwlgt-iiaaa-aaaaa-aaaaa-cai"
"rust_hello" canister created with canister id: "rrkah-fqaaa-aaaaa-aaaaq-cai"
Building canisters...
Executing 'cargo build --target wasm32-unknown-unknown --package rust_hello'
thread 'main' panicked at 'Could not run custom tool.: Os { code: 2, kind: NotFound, message: "No such file or directory" }', src/dfx/src/lib/builders/custom.rs:181:18
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

would  you kindly tell me what could be wrong?
my dfx.json
{
  "canisters": {
    "rust_hello": {
      "build": "cargo build --target wasm32-unknown-unknown --package rust_hello",
      "candid": "src/rust_hello/rust_hello.did",
      "wasm": "target/wasm32-unknown-unknown/debug/rust_hello.wasm",
      "type": "custom"
    }
  },
  "defaults": {
    "build": {
      "packtool": ""
    }
  },
  "dfx": "0.6.25",
  "networks": {
    "local": {
      "bind": "<IP_ADDRESS>:8000",
      "type": "ephemeral"
    },
    "ic": {
      "providers": [
        "https://gw.dfinity.network"
      ],
      "type": "persistent"
    }
  },
  "version": 1
}

rust_hello/src/rust_hello/Cargo.toml
[package]
name = "rust_hello"
version = "0.1.0"
authors = ["Your Name<EMAIL_ADDRESS>edition = "2018"

[lib]
path = "main.rs"
crate-type = ["cdylib"]

[dependencies]
ic-cdk = { path = "../../../cdk-rs/src/ic-cdk", version = "0.1.0" }
ic-cdk-macros = { path = "../../../cdk-rs/src/ic-cdk-macros", version = "0.1.0" }

cdk-rs and rust_hello are situated in the same folder

Seems like you don't have cargo installed?

@crawter, I think the problem you found is a result of our documentation getting a bit stale and it needs to be [updated.](url)
If you install the latest version of the SDK (0.7.0), you can try out the instructions in the PDF to see if they work for you.

rust-quickstart.pdf

Fixed with the merged PR.


Text Overlapping Issue

This worked pretty awesome but then randomly noticed text overlapping issue. Not sure how it the scenario is but somehow it appeared multiple times. Can we fix it please.

Hi ch-adnan. i had the same issue and i solve it by change spawnAmount value in line 178 in marquee.js file from 3 to 1 and it's work like a charm for me.
spawnAmount = 1;
try it and i hope it work fine with you.


high resource usage in the housekeeper

hi, we're using moka in a query cache in the Apollo router, and we are seeing high CPU usage in the housekeeper code, as shown in that flamegraph of a benchmark that uses one core and does not insert or get anything from the moka cache(the large tower on the left is the housekeeper):

With 4 cores, It is much lower, but still 2.45% of sampled time, for a cache that is doing nothing:

do you have an idea why it would cost that much to run the housekeeper?

Hi. Thank you for reporting the issue.
Which OS and CPU architecture did you use? Perhaps Linux x86_64? And you were using a multi-core machine, right?
When moka cache is not busy (including idle), its housekeeper task will run every 0.3 seconds. If cache is idle, it will spend very little time to run because there is no work to do. I see it takes only 0.05% of the space in the flamegraph.

I do not have answer yet, but I found the followings:

As you said, the large tower on the left is the housekeeper, taking up 29.34% of the space of the sigle-core flamegraph.
In the tower:

Running actual housekeeper task takes up only 0.05%.
futex_wait in a system call takes up ~22.20%. (12.51% + 7.61% + 1.44% + 0.64%)
futex_wait was called by Condvar::wait_until of parking_lot_core crate, which will put the calling thread to sleep until the given duration is reached.

If I understand correctly, the system spent lots of time (?) in futex_wait calls. And futex_wait is used for sleeping a thread for given duration.
Maybe native_write_msr on the top of the stack is the key. I am not sure yet, but it might be used to record Linux perf data?

Why does native_write_msr dominate my profiling result?
What does native_write_msr in kernel do?

If so, I could try different platforms without perf; e.g. macOS or FreeBSD with dtrace.

It's on linux x86_64 yes

A quick update:
I am currently investigating why the system spent lots of time in Linux futex_wait while all worker threads in the thread pool were mostly sleeping.
I created a testing environment at Amazon EC2, but have not had time to write a test program to reproduce the issue:

Ubuntu Server 22.04, 4 vCPUs
In the env, I confirmed that I can generate flamegraphs including kernel stacks by using [cargo-]framegraph with Linux perf.

I read relevant codes in scheduled-thread-pool, parking_lot and parking_lot_core crates. I also read some blog articles to learn very basic of Linux futex.
My current theory (not verified) is that spurious wakeups are occurring in futex_wait.

I see in the original single-core flamegraph, futex_wait was calling schedule to return the CPU core to the kernel.

This would mean that the worker threads were woken up by the kernel, but not ready to proceed.

Maybe the kernel was waking up the threads because there were not enough tasks to be scheduled to make the CPU cores busy?

maybe it's a function of how busy the CPUs are yes. In my initial tests I limit the number of cores used by the router like this: https://github.com/apollographql/router/blob/main/apollo-router/src/executable.rs#L139-L146
(it's necessary because it's hard to saturate the router with traffic if it has access to all cores)

the cache-tools branch is about factoring the various cache implementations in the router (there were 5 different ones when I started that work :sweat_smile: ) around one cache interface that can have multiple levels, in memory and external (redis, memcached).
We have not decided to move away from moka yet, we're evaluating cache libraries that we can fit in that model. If we find a good library that handles the multi level for us we might go for it.
If there's one point that would motivate me to move away from moka, it's that it feels too complicated for me to go and fix quickly during an incident. But that's not an inconvenient of the library itself, you chose to cover a lot of ground in one coherent implementation. An example here: with the get_or_insert_with, moka can return a value or perform the work to retrieve a new one, while coalescing concurrent queries, and this affects multiple parts of the cache. In the router we handle query coalescing ourselves (because we can fit it better with the graphql query model), so we mainly need a synchronized KV store.

Hi. Think you very much for sharing your thoughts. I am not aware of any library that handles the multi level (tiers) of caches in Rust, but I hope you will find one. I have not got idea how the tiered cache in the cache-tool branch would work. I will read the code closely and try to figure out.
In the latest version of Moka (v0.9.0), I added support for eviction listener callback function. That feature was requested by multiple users. I wondered if it will be useful when implementing a tiered cache. The listener can be used to move an entry evicted from Moka cache to an external cache.
So I wrote an example tiered cache implementation here, which uses Moka as the primary cache and Redis as the secondary cache:

https://gitlab.com/moka-labs/tiered-cache-example

I am not sure if it can fit your needs; I guess not. I also found some issues (limitations) in the current API and internal design. I listed them up in the "Limitations" section of the README. I hope we can solve them in the future.
I agree that Moka cache's implementation is complicated. I chose to go in the direction of providing convenient features requested by users. I also put my knowledge and ten-year experience in distributed key-value and object stores to Moka cache to make it more performant. But this also makes it more complicated. My hope is that it becomes mature enough so that nobody have to go into and quickly fix it.
On the other hand, a few users have asked for a much simpler version of cache; like a HashMap with size constraints and entry expiration. We have experimental concurrent cache implementation called moka::dash::Cache for that. It has no advanced features, so its implementation is a bit simpler.
I am thinking to move it to a separate crate to make it clear that it is a simpler version. Maybe I will make it even simpler than now. I hope you might be interested in it.

I will add an option to remove the thread pools from moka::sync::Cache. If there is no thread pool, there will be no CondVar with Linux futex and the problem will not exist in the first place.

I am working on it here: https://github.com/moka-rs/moka/pull/165
You can try it by doing the followings:
Modify Cargo.toml:
[dependencies]
moka = { git = "https://github.com/moka-rs/moka", branch = "disable-housekeeper-threads" }

When creating a cache instance, set thread_pool_enabled to false using the cache builder.
use moka::sync::Cache;

let cache = Cache::builder()
    .max_capacity(MAX_CAPACITY)
    .thread_pool_enabled(false)
    .build();

For backward compatibility, thread_pool_enabled is set to true by default. We will change the default to false in the future (v0.10.0?).

I will add an option to remove the thread pools from moka::sync::Cache. If there is no thread pool, there will be no CondVar with Linux futex and the problem will not exist in the first place.

I have published Moka v0.9.3 with this feature. (doc)
I will keep this issue open to investigate the original issue. It will do it when I have more time.

Closing as complete.

In v0.9.3 (released in last summer, August 4th, 2022), we added an option to the cache builder of the sync caches not to start and use the global thread pools for housekeeping tasks.

#165

In v0.12.0 (currently beta), we removed the thread pool completely from all cache types.

#316
#294

v0.12.0 has major breaking changes. Please read the MIGRATION-GUIDE.md for details.


Incorrect type hint when hovering over object field

let obj = {
  "someField": 42,
}

Hovering over "someField" gives
(~someField: int) => {"someField": int}

Cannot reproduce with current master, closing.


JSON friendly logger statements

Make almost all lodestar monorepo logger statements JSON friendly. Fixes https://github.com/ChainSafe/lodestar/issues/1249
Extends the API of the logger to force the error to always be the third argument. This way the error is identified in the JSON renderization and simplifies the logic
type LogHandler = (message: string, context?: Context, error?: Error) => void

Some eth2RequestDecode are failing but I've only modified logger statements. Do those tests depend on stuff being logged out or what can be happening?
@chainsafe/lodestar:   1) request encoders
@chainsafe/lodestar:        eth2RequestDecode - request validation
@chainsafe/lodestar:          should yield {isValid: false} if it takes more than 10 bytes for varint:
@chainsafe/lodestar:       AssertionError: expected false to be true
@chainsafe/lodestar:       + expected - actual
@chainsafe/lodestar:       -false
@chainsafe/lodestar:       +true
@chainsafe/lodestar:
@chainsafe/lodestar:       at Context.<anonymous> (test/unit/network/encoders/request.test.ts:123:57)

https://github.com/ChainSafe/lodestar/pull/1837/checks?check_run_id=1507855943#step:9:1138

Some eth2RequestDecode are failing but I've only modified logger statements. Do those tests depend on stuff being logged out or what can be happening?
@chainsafe/lodestar:   1) request encoders
@chainsafe/lodestar:        eth2RequestDecode - request validation
@chainsafe/lodestar:          should yield {isValid: false} if it takes more than 10 bytes for varint:
@chainsafe/lodestar:       AssertionError: expected false to be true
@chainsafe/lodestar:       + expected - actual
@chainsafe/lodestar:       -false
@chainsafe/lodestar:       +true
@chainsafe/lodestar:
@chainsafe/lodestar:       at Context.<anonymous> (test/unit/network/encoders/request.test.ts:123:57)

https://github.com/ChainSafe/lodestar/pull/1837/checks?check_run_id=1507855943#step:9:1138

yup, just change message for extra credit each validation check could be utility function which can be stubbed and spied. But then each of those utility function should have own unit tests I guess.

yup, just change message or for extra credit each validation check could be utility function which can be stubbed and spied. But then each of those utility function should have own unit tests I guess.

Asserting a logged statement as a test success sounds really crazy. Not in the scope on this PR, but couldn't this be done more cleanly with custom errors or something?

yup, just change message or for extra credit each validation check could be utility function which can be stubbed and spied. But then each of those utility function should have own unit tests I guess.

Asserting a logged statement as a test success sounds really crazy. Not in the scope on this PR, but couldn't this be done more cleanly with custom errors or something?

Possibly, but it's almost the same thing. You change error name/enum and you have to correct tests as well? :slightly_smiling_face:

Possibly, but it's almost the same thing. You change error name/enum and you have to correct tests as well?

Tests are still failing, could you take a look?


No category description.

[Enter feedback here]
There is no description on each category
it is impossible for common of mortal to know the difference between
File scale targets and Azure file share scale targets
you should add a description for each particular category.
For example, Microsoft DSE explains to us that the file share target apply to the file share global
and file target apply per single file. That should be explained on the page

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 991338c0-1a61-99e1-41ea-45fd5d5fd63d
Version Independent ID: f2dd8b70-29f2-bb22-38f5-097c5eb6b797
Content: Azure Files scalability and performance targets
Content Source: articles/storage/files/storage-files-scale-targets.md
Service: storage
Sub-service: files
GitHub Login: @khdownie
Microsoft Alias: kendownie

Thank you for your detailed feedback.  We are looking into the issue, and we will respond when we have more information.

@christgelinas Apologies for the delay response! I have assigned the issue to the content author to investigate further and update the document as appropriate.

@christgelinas Apologies for the delay response! I have assigned the issue to the content author to investigate further and update the document as appropriate.

Thank you, the documentation on this page missing category description.  There is no description that explain what is file share target and global file share target. Each category should have a description.


Field of data type long shows up with Long.MIN value during TTL of documents

Describe the bug
We’re running into an issue with one of our Vespa schemas for which TTL of documents is enabled - the documents are TTLed 36 hours after ingestion after one version of the dataset is ingested. We’re observing a strange behaviour while the TTL is in progress where one of our fields snapshot_version which is of type long shows up as Long.MIN. We know for a fact that our ingestion code doesn’t ever set this value to a negative number as this is ingested as a timestamp value from upstream. When we run a YQL like this, where x is one of the snapshot_version values present in the records -
select * from my_schema where snapshot_version = x limit 0 | all(group(snapshot_version) each(output(count())))

we see that Vespa returns this negative value in one of the groups, while the others are expected values.
Also, one other observation is that even when the YQL filters on one of the expected snapshot_version values, the values returned include those from the Long.MIN snapshot version. Due to this behaviour, we’re seeing inconsistent values for other fields from the result of our grouping/aggregation queries.
To Reproduce
Setup garbage collection on schema:
<documents garbage-collection="true" >
          <document type="my_schema" mode="index" selection="my_schema.timestamp > now() - 129600" />
          <document-processing cluster="feed" />
</documents>

Define schema as below:
field my_schema type long {
            indexing: attribute
        }

        field version type string {
        }

        field snapshot_version type long {
            indexing: attribute
            attribute: fast-search
            rank: filter
        }

        field timestamp type long {
            indexing: attribute
            attribute: fast-access
        }

Run YQL as TTL happens
select * from my_schema where snapshot_version = x limit 0 | all(group(snapshot_version) each(output(count())))

While TTL happens, it's observed that a snapshot_version of Long.MIN also shows as one of the grouping results even though it's not provided in the filter.
Expected behavior
The grouping result with snapshot_version = Long.MIN shouldn't appear in the results as it was not provided in the YQL filter clause.
Environment (please complete the following information):
OS: Linux
Infrastructure: Kubernetes
Vespa version
8.128.22
Additional context
I have confirmed that we don’t ingest/have documents where this attribute misses a value as we ingest this value from upstream computed as the current timestamp. Also, pls note that this behaviour happens only during ingestion and works perfectly fine otherwise.
I wanted to add a couple of observations while documents are getting TTL’ed. Upon running our YQL, I can see results where snapshot_version is Long.MIN but those documents vanish away (that’s when they’re really evicted from memory?) before new ones show up. This behaviour continues until all the documents are TTL’ed.

One possible explanation is that a grouping query is not one atomic read operation, and some state is preserved between t1 and t3 in the stateless container.

t1 select * from my_schema where snapshot_version = x limit 0  finds the documents
t2 Some documents found by query deleted
t3 all(group(snapshot_version) each(output(count()))) returns default value for deleted documents found at t1

Plausible @bjorncs @baldersheim ?

@jobergum but the documents that were deleted belong to another snapshot_version y older than x. So shouldn't they have been filtered out in the first place at t1?

How can you be sure about that since your expire is not on snapshot_version?

I guess this happens during what we call lid space compaction.
Internally in the back each document is assigned a local document id when it is ingested.
This lid is an integer number in the range[1.....current_lid_limit]. When a document is removed, its lid is placed on a freelist after all live references to it has gone.
A new document is then given a lid that is on the freelist, or assigned the next integer after current_lid_limit, which is then increased.
Having a large lid freelist has a cost, both extra memory and more cpu spent query time. We call this waste lidbloat.
Once the lidbloat exceed a threshold, a background process, lid-space-compaction kicks in. Documents with the highest lids are moved to a lower vacant lid. The document is first copied to its new location, and then deleted from its old location.
I assume what happens is that the first the query is executed and a document with a high lid is a match.
But then when grouping is executed afterwards this document has now completed its move to a new lower lid, and its data has been deleted from it older higher lid.
The scenario you have with generational feed and TTL for removal is more likely to observe this.
In this use case lid-space-compaction is rather an anti-feature.
1 - We will add configuration to disabling lid-space-compaction.
2 - If my theory is correct we will investigate ways to improve it. Most likely by deferring removal of data at old high lid until all references are gone.

Thanks @baldersheim. So, if I understand it correctly, since we're seeing -ve values, it means that a few documents which were originally selected (before grouping) were since deleted by the time grouping was executed. So, is it safe to say that these documents which should've been considered to return a value after grouping aren't actually considered because of compaction?

@baldersheim @jobergum thank you so much for the quick turnaround on this issue. My original thought was to do a grouping on our snapshot_version field and filter out the group with Long.MIN, but after @baldersheim's comment, I understand that is incorrect.
Since this is affecting our production systems, I was hoping to get an ETA on the release with the fixes for this issue or any recommendations wrt what we can do to workaround this problem. Please let me know. Thank you!

1 - Configuration control of lid-space compaction will come next week.
2 - Will know ETA next week for better handling of lid-space compaction.

PR for item 1 is in #27223

@baldersheim thank you! what would be the impact of disabling lid-space compaction? as i understand, it's bound to increase the memory footprint, so would you suggest it's something we should do when it's available?

It will not increase the maximum memory usage. memory usage will stay at its peak.
Let us say that you at some point in time have 20M docs on your node consuming, for simplicity 20G memory.
If you then remove 10M docs you will then use 10G memory afterwards once lidspace has been compacted.
If you disable lidspace compaction, your memory usage will somewhere between 10G and 20G. Your schema decides if you will be closer to 10G or closer to 20G.
If your corpus again grows to 20M docs you will not grow beyond 20G as that bloat will be reused.
So in your usecase I would disable lidspace compaction. You will avoid wasting cpu and I/O to keep lidspace compact.

Documentation add in https://github.com/vespa-engine/documentation/pull/2690

thanks @baldersheim, that makes sense. once the update is available with the config option to disable lid space compaction, we will use it. pls let us know, thanks!

Tuning lidspace max-bloat-factor is part of Vespa 8.171.14.

0.95 sounds high enough. It depends how much your corpus varies in size.
If you use a generational approach and have 2 generations a value of 0.6 would be fine.

@baldersheim by setting the value to 0.6, we're effectively not disabling lid-space compaction entirely but limiting when it occurs (when 60% of the documents have been TTLed for example). so, if i'm not wrong the issue of getting -ve values with our generational workload approach will still happen, although less severely. is this understanding correct?
If we want to entirely disable compaction by setting the value to 1.0, we will have to live with the bloat (and thus a memory peak) until the next version is written when the lid values are reused. If this is a reasonable compromise for us, do you suggest setting the value to 1.0 to disable compaction as this will ensure correctness of the vespa results?

Your understanding is correct.
If you have a huge variation in number of documents concurrently on a node you should set it higher.

thank you @baldersheim. we'll set the value to 1.0 on our test env and observe the behaviour. will get back.

hi @baldersheim @jobergum we deployed version 8.174.17 of Vespa with the config parameter for max-bloat-factor set to 1. As per my understanding, this translates to never triggering the lid-space compaction unless all the documents in our schema are deleted. However, even when we have one snapshot_version of the data in the schema, the issue with the Long.MIN group is still reproducible. Maybe I'm missing something, so was wondering if I could get some help in this regard, thanks.

I think we need to establish state first:
1 - Did you observe any difference at all when you turned off lid-space compaction ?
2 - In your last comment you are stating "one snapshot_version" ? Does that mean that all documents in corpus have the same version, so the query actually touches matches all documents.
If your query matches documents that have reached the TTL you still have the chance of hitting https://github.com/vespa-engine/vespa/issues/27177#issuecomment-1558648675. That problem does not have any solution. The probability of observing this issue is linear to the time between T1 and T3, which is linear to the number of documents matching the query.

@baldersheim pls find replies -

not really - with the value of max-bloat-factor set to 1.0, found no change in behaviour.
we always have more than one snapshot_version during the time this issue is reproducible. The documents with the older snapshot_version is the dataset that gets TTL'ed during this time.

We always query with the newest snapshot_version - the one that's not getting TTL'ed at any moment, so our query never matches with the docs that have reached the TTL.

hi @baldersheim a correction on 1. At some point, when one snapshot_version is getting TTL'ed (the older version) we have another version (the newer version) which is active and not being TTL'ed. When we set the max-bloat-factor to 1.0, we observe that querying with a filter on the new snapshot version doesn't result in the -ve version. However, it does appear when we query with a filter on the version that's getting TTL'ed.
Since we've effectively disabled compaction by setting the value to 1.0, documents from the newer version aren't changing their lid values and that's the reason this issue is mitigated. Can you confirm if this is the correct understanding?


Is it an expected behavior that onEnter triggered both on server and client?

Hi, guys. There are some data-fetching logics in onEnter in my SSR case. But onEnter is triggered both on server and client. Data will be fetched twice. Is it an expected behavior for react-router-redux?

That is not an issue with this library, it's something with React Router.
But yes, onEnter hooks are expected to be triggered in all environments. It's core to the routing engine.


Turning on window border can break focus following

It looks like the extra time it takes to render the border can interfere with focus updates. More noticeable with larger border_width.
Reproduction:

Turn on window_border
Using the mouse, move from one window to another, then quickly switch back.
Note that occasionally focus will remain on the second window, rather than returning.
(Moving the mouse another time will correctly update focus.)

nb: Aside from the above issue, I'd really like to use the window border feature to better know which app I have focused during keyboard navigation, but the draw overhead it adds makes switching windows feel a bit sluggish. Not sure if this can be improved at all, but it would be a huge usability boost!

So I don't think this has anything to do particularly with window borders being enabled. There is a check in place to only respond to a mouse moved event if it has been at least 25ms since the last movement. This is to reduce the amount of cpu usage during heavy mouse activity. I don't at this time have a better solution than rate limiting.

window borders are no longer a thing, so I'm closing this.


Fix link to CONTRIBUTING.md in README

Hi folks! 👋
The documentation section on contributing was moved to a separate file in https://github.com/alexreisner/geocoder/commit/a81e028cf21d1fd31f3990ed22c466962f138a21, but the link in the main README still used anchors to the README itself for both the 'Contributing' and 'Reporting Issues' links.
This updates those links so they point to the contributing file on GitHub. Even though it's now a small document, both links point to the heading for the relevant section using anchors.

Good catch! Thanks!


Error: nix: command error: nix --extra-experimental-features ca-derivations --option experimental-features 'nix-command flakes fetch-closure' build --impure --no-link 'github:NixOS/nixpkgs/1f37660f64a850233baab512c9b9bd83fb72be57#mongodb': exit code 1

What happened?
After upgrading to 0.13.3 and start services up then I got error message Error: nix: command error: nix --extra-experimental-features ca-derivations --option experimental-features 'nix-command flakes fetch-closure' build --impure --no-link 'github:NixOS/nixpkgs/1f37660f64a850233baab512c9b9bd83fb72be57#mongodb': exit code 1 after running devbox services up
Steps to reproduce
the devbox.json
{
  "packages": [
    "nats-server@latest",
    "ripgrep@latest",
    "overmind@latest",
    "postgresql_12@latest",
    "tldr@latest",
    "natscli@latest",
    "redis@latest",
    "nats-top@latest",
    "mongosh@latest",
    "mongodb@latest"
  ],
  "shell": {
    "init_hook": [
      ""
    ],
    "scripts": {
      "test": [
        "echo \"Error: no test specified\" && exit 1"
      ],
      "setup_db": [
        "initdb --encoding=UTF8"
      ],
      "add_users": [
        "psql -f add-users.sql"
      ]
    }
  },
  "include": [
    "github:jetpack-io/devbox-plugins?dir=mongodb"
  ]
}

Command
No response
devbox.json
No response
Devbox version
0.13.3
Nix version
No response
What system does this bug occur on?
macOS (Apple Silicon)
Debug logs
Info: Ensuring packages are installed.
Info: Installing the following packages to the nix store: mongodb@latest
error: the user '_nixbld1' in the group 'nixbld' does not exist
Error: nix: command error: nix --extra-experimental-features ca-derivations --option experimental-features 'nix-command flakes fetch-closure' build --impure --no-link 'github:NixOS/nixpkgs/1f37660f64a850233baab512c9b9bd83fb72be57#mongodb': exit code 1
Error: There was an internal error. Run with DEVBOX_DEBUG=1 for a detailed error message, and consider reporting it at https://github.com/jetify-com/devbox/issues

error: the user '_nixbld1' in the group 'nixbld' does not exist
Error: nix: command error: nix --extra-experimental-features ca-derivations --option experimental-features 'nix-command

I think this is related to an issue with Nixpkgs on macOS Sequoia. Newer versions of macOS overwrote some of the UIDs that Nix uses to build it's packages.
To fix this, you can try running the repair script here:
curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix/tag/v0.26.0 | sh -s -- repair sequoia --move-existing-users

For more details, check out the issue here: https://github.com/NixOS/nix/issues/10892

I get a similar error on Linux (popOS) . Devbox version 0.13.4.

devbox update
or
devbox add<EMAIL_ADDRESS>
{
  "$schema":  "https://raw.githubusercontent.com/jetify-com/devbox/0.13.3/.schema/devbox.schema.json",
  "packages"<EMAIL_ADDRESS>  "shell": {
    "init_hook": [
      ". $VENV_DIR/bin/activate"
    ],
    "scripts": {
      "test": [
        "echo \"Error: no test specified\" && exit 1"
      ]
    }
  }
}

I get
Error: nix: command error: nix --extra-experimental-features ca-derivations --option experimental-features 'nix-command flakes fetch-closure' print-dev-env --json path:/home/papo/dev/python-etf/.devbox/gen/flake: attribute 'python313' missing: exit code 1

Error: There was an internal error. Run with DEVBOX_DEBUG=1 for a detailed error message, and consider reporting it at https://github.com/jetify-com/devbox/issues

Same problem with devbox 0.13.4

error: the user '_nixbld1' in the group 'nixbld' does not exist
Error: nix: command error: nix --extra-experimental-features ca-derivations --option experimental-features 'nix-command

I think this is related to an issue with Nixpkgs on macOS Sequoia. Newer versions of macOS overwrote some of the UIDs that Nix uses to build it's packages.
To fix this, you can try running the repair script here:
curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix/tag/v0.26.0 | sh -s -- repair sequoia --move-existing-users

For more details, check out the issue here: NixOS/nix#10892

This fixed my issue!


Changing build target from debug to release makes Visual Studio hang

Environment
Windows build number: Win32NT             10.0.18362.0 Microsoft Windows NT 10.0.18362.0
Windows Terminal version 35d7d20a079e78323ed510e5535bb71fdb68712a

Any other software?

Visual Studio Community 2019 16.3.4

Steps to reproduce
git clone https://github.com/microsoft/Terminal.git
cd Terminal
git submodule update --init --recursive
nuget restore OpenConsole.sln
Open solution in VS
Change debug to release

Expected behavior
VS Changes debug to release and proceeds

Actual behavior
VS hangs at 100% cpu then drops to 0% still hung.

Other Info
Other projects still work OK, previous WindowsTerminal trees can still be built either release or debug target.

VS seems to have trouble with certain solutions that contain more than, like, ten projects. It might be worth reporting an issue at the VS developer community if you can reliably reproduce this issue. We've seen it sporadically, on all sorts of projects, but never reliably enough to report.


Untick paper-plugin.yml in paper project setup guide

Made in response to https://github.com/PaperMC/docs/pull/444#issuecomment-2231115386.
Basically just fixes an error which I did, but the original PR sadly already closed

wait no
wrong theme

do not merge yet

Okay there, fixed