Unknown variant lifetime_predicate

Hello,
I was trying to get out a api diff in the branch: https://github.com/rust-skia/rust-skia/pull/993, but
running
cargo public-api -p skia-safe -s --features=all-macos diff latest -- >skia-safe-api.diff

lead to
unknown variant `lifetime_predicate`, expected one of `bound_predicate`, `region_predicate`, `eq_predicate` at line 1 column 2310045

❯ cargo public-api --version
cargo-public-api 0.36.0

stable-aarch64-apple-darwin unchanged - rustc 1.79.0 (129f3b996 2024-06-10)
nightly-aarch64-apple-darwin unchanged - rustc 1.81.0-nightly (5affbb171 2024-07-18)

Thanks for reporting. The JSON format changed recently but apparently our regression tests did not detect it.
Fixed by https://github.com/Enselic/cargo-public-api/pull/595


Add A. Rosien to contributors.yml

DAFFODIL-2693

@arosien, it might be good for you to merge this PR to verify that your merge permissions work. I'm not sure if you've merged anything yet?


offline docs deployment are dependent with some CDN resources

Hi all. The deployment cannot be more simple, and thank you. As we deploy this docs for our client, totally offline. We need to download and customize many offline packages.
We make a success to deploy this doc with a lot efforts, but unfortunately we find it dependent with many online CDN resources, like js and css files. This makes the deployment fail to serve the client. I wonder, if there's a quick way to fix this?

Hi @wiwengweng not sure what the issue is here, can you be more specific?

Hi, Adam. I mean can we deploy and use this document totally offline. Now there are many online resources need to be fetched from CDN. And my client want to deploy this doc, and use this doc without Internet. Thank you.

Hi, there is no offline option, but there is a Local Development option: https://github.com/mendix/docs#local-development

In the future we hope to make downloadable PDFs available. Until then, there is no offline version of our documentation site. You can choose to download selected content in Markdown: https://github.com/mendix/docs/tree/development/content

@Adam-Dupaski Thank you Adam. I suggest if we can add a 'export PDF' in the official document website, that's more convenient.


Update the documentation: Install on Ubuntu/Mint needs libssl-dev

When building Xephem 4.1.0 on Linux Mint 20.2, make received fatal error "openssl/sha.h: No such file or directory"
This was fixed by installing libssl-dev
sudo apt-get install libssl-dev
Please update the INSTALL readme file to indicate the need to install this library on Debian/Ubuntu distributions.

Happily, the INSTALL file was recently updated:
https://github.com/XEphem/XEphem/blob/main/INSTALL#L18
The new text should appear in the next release of XEphem.


BottomNavigationBar breaks when increasing number of items it contains

I believe that the cause is that the bar's state allocates animation controllers for each item when the state is first constructed, but it doesn't create new controllers when the state updates with more items.
Steps to Reproduce
Create a nav bar with 3 items.
Increase the number of items to 4.
RangeError(index): invalid value: not in range 0..2, inclusive: 3.

Logs
I/flutter (27323): ══╡ EXCEPTION CAUGHT BY WIDGETS LIBRARY ╞═══════════════════════════════════════════════════════════
I/flutter (27323): The following RangeError was thrown building BottomNavigationBar(dirty, state:
I/flutter (27323): _BottomNavigationBarState#68e58(tickers: tracking 3 tickers)):
I/flutter (27323): RangeError (index): Invalid value: Not in range 0..2, inclusive: 3
I/flutter (27323):
I/flutter (27323): When the exception was thrown, this was the stack:
I/flutter (27323): #0      List.[] (dart:core-patch/dart:core/growable_array.dart:141)
I/flutter (27323): #1      _BottomNavigationBarState._createTiles (package:flutter/src/material/bottom_navigation_bar.dart:411:26)
I/flutter (27323): #2      _BottomNavigationBarState.build (package:flutter/src/material/bottom_navigation_bar.dart:477:45)
I/flutter (27323): #3      StatefulElement.build (package:flutter/src/widgets/framework.dart:3690:27)
I/flutter (27323): #4      ComponentElement.performRebuild (package:flutter/src/widgets/framework.dart:3602:15)
I/flutter (27323): #5      Element.rebuild (package:flutter/src/widgets/framework.dart:3463:5)
I/flutter (27323): #6      StatefulElement.update (package:flutter/src/widgets/framework.dart:3737:5)
I/flutter (27323): #7      Element.updateChild (package:flutter/src/widgets/framework.dart:2667:15)
I/flutter (27323): #8      ComponentElement.performRebuild (package:flutter/src/widgets/framework.dart:3613:16)
I/flutter (27323): #9      Element.rebuild (package:flutter/src/widgets/framework.dart:3463:5)
I/flutter (27323): #10     ProxyElement.update (package:flutter/src/widgets/framework.dart:3847:5)
I/flutter (27323): #11     Element.updateChild (package:flutter/src/widgets/framework.dart:2667:15)
I/flutter (27323): #12     ComponentElement.performRebuild (package:flutter/src/widgets/framework.dart:3613:16)
I/flutter (27323): #13     Element.rebuild (package:flutter/src/widgets/framework.dart:3463:5)
I/flutter (27323): #14     ProxyElement.update (package:flutter/src/widgets/framework.dart:3847:5)
I/flutter (27323): #15     Element.updateChild (package:flutter/src/widgets/framework.dart:2667:15)
I/flutter (27323): #16     RenderObjectElement.updateChildren (package:flutter/src/widgets/framework.dart:4317:32)
I/flutter (27323): #17     MultiChildRenderObjectElement.update (package:flutter/src/widgets/framework.dart:4703:17)
I/flutter (27323): #18     Element.updateChild (package:flutter/src/widgets/framework.dart:2667:15)
I/flutter (27323): #19     ComponentElement.performRebuild (package:flutter/src/widgets/framework.dart:3613:16)
I/flutter (27323): #20     Element.rebuild (package:flutter/src/widgets/framework.dart:3463:5)
I/flutter (27323): #21     ProxyElement.update (package:flutter/src/widgets/framework.dart:3847:5)
I/flutter (27323): #22     Element.updateChild (package:flutter/src/widgets/framework.dart:2667:15)
I/flutter (27323): #23     ComponentElement.performRebuild (package:flutter/src/widgets/framework.dart:3613:16)
I/flutter (27323): #24     Element.rebuild (package:flutter/src/widgets/framework.dart:3463:5)
I/flutter (27323): #25     StatefulElement.update (package:flutter/src/widgets/framework.dart:3737:5)
I/flutter (27323): #26     Element.updateChild (package:flutter/src/widgets/framework.dart:2667:15)
I/flutter (27323): #27     SingleChildRenderObjectElement.update (package:flutter/src/widgets/framework.dart:4595:14)
I/flutter (27323): #28     Element.updateChild (package:flutter/src/widgets/framework.dart:2667:15)
I/flutter (27323): #29     ComponentElement.performRebuild (package:flutter/src/widgets/framework.dart:3613:16)
I/flutter (27323): #30     Element.rebuild (package:flutter/src/widgets/framework.dart:3463:5)
I/flutter (27323): #31     StatelessElement.update (package:flutter/src/widgets/framework.dart:3662:5)
I/flutter (27323): #32     Element.updateChild (package:flutter/src/widgets/framework.dart:2667:15)
I/flutter (27323): #33     SingleChildRenderObjectElement.update (package:flutter/src/widgets/framework.dart:4595:14)
I/flutter (27323): #34     Element.updateChild (package:flutter/src/widgets/framework.dart:2667:15)
I/flutter (27323): #35     ComponentElement.performRebuild (package:flutter/src/widgets/framework.dart:3613:16)
I/flutter (27323): #36     Element.rebuild (package:flutter/src/widgets/framework.dart:3463:5)
I/flutter (27323): #37     StatefulElement.update (package:flutter/src/widgets/framework.dart:3737:5)
I/flutter (27323): #38     Element.updateChild (package:flutter/src/widgets/framework.dart:2667:15)
I/flutter (27323): #39     ComponentElement.performRebuild (package:flutter/src/widgets/framework.dart:3613:16)
I/flutter (27323): #40     Element.rebuild (package:flutter/src/widgets/framework.dart:3463:5)
I/flutter (27323): #41     StatefulElement.update (package:flutter/src/widgets/framework.dart:3737:5)
I/flutter (27323): #42     Element.updateChild (package:flutter/src/widgets/framework.dart:2667:15)
I/flutter (27323): #43     ComponentElement.performRebuild (package:flutter/src/widgets/framework.dart:3613:16)
I/flutter (27323): #44     Element.rebuild (package:flutter/src/widgets/framework.dart:3463:5)
I/flutter (27323): #45     ProxyElement.update (package:flutter/src/widgets/framework.dart:3847:5)
I/flutter (27323): #46     Element.updateChild (package:flutter/src/widgets/framework.dart:2667:15)
I/flutter (27323): #47     ComponentElement.performRebuild (package:flutter/src/widgets/framework.dart:3613:16)
I/flutter (27323): #48     Element.rebuild (package:flutter/src/widgets/framework.dart:3463:5)
I/flutter (27323): #49     ProxyElement.update (package:flutter/src/widgets/framework.dart:3847:5)
I/flutter (27323): #50     Element.updateChild (package:flutter/src/widgets/framework.dart:2667:15)
I/flutter (27323): #51     ComponentElement.performRebuild (package:flutter/src/widgets/framework.dart:3613:16)
I/flutter (27323): #52     Element.rebuild (package:flutter/src/widgets/framework.dart:3463:5)
I/flutter (27323): #53     StatefulElement.update (package:flutter/src/widgets/framework.dart:3737:5)
I/flutter (27323): #54     Element.updateChild (package:flutter/src/widgets/framework.dart:2667:15)
I/flutter (27323): #55     ComponentElement.performRebuild (package:flutter/src/widgets/framework.dart:3613:16)
I/flutter (27323): #56     Element.rebuild (package:flutter/src/widgets/framework.dart:3463:5)
I/flutter (27323): #57     BuildOwner.buildScope (package:flutter/src/widgets/framework.dart:2210:33)
I/flutter (27323): #58     BindingBase&GestureBinding&ServicesBinding&SchedulerBinding&PaintingBinding&RendererBinding&WidgetsBinding.drawFrame (package:flutter/src/widgets/binding.dart:621:20)
I/flutter (27323): #59     BindingBase&GestureBinding&ServicesBinding&SchedulerBinding&PaintingBinding&RendererBinding._handlePersistentFrameCallback (package:flutter/src/rendering/binding.dart:208:5)
I/flutter (27323): #60     BindingBase&GestureBinding&ServicesBinding&SchedulerBinding._invokeFrameCallback (package:flutter/src/scheduler/binding.dart:990:15)
I/flutter (27323): #61     BindingBase&GestureBinding&ServicesBinding&SchedulerBinding.handleDrawFrame (package:flutter/src/scheduler/binding.dart:930:9)
I/flutter (27323): #62     BindingBase&GestureBinding&ServicesBinding&SchedulerBinding._handleDrawFrame (package:flutter/src/scheduler/binding.dart:842:5)
I/flutter (27323): #63     _invoke (file:///b/build/slave/Linux_Engine/build/src/flutter/lib/ui/hooks.dart:120)
I/flutter (27323): #64     _drawFrame (file:///b/build/slave/Linux_Engine/build/src/flutter/lib/ui/hooks.dart:109)
I/flutter (27323): ════════════════════════════════════════════════════════════════════════════════════════════════════
Doctor
Version of Flutter is the current g3 version.

try to call setState trigger rebuild

i have the same issue

I get the same error when creating a BottomNavigationBar with 'items' being a list of four BottomNavigationBarItems.

I have the same error with three items.

Same error with four items in hot reload. A full restart solved.

Duplicate of #10322


Fixes to equality and hashing

Haven't implemented hashing on pdop etc, since that's more work.
I expect this to fix some LGTM warnings.

Now that gh-197 is merged, this change must go in, else the ga != ga lines in that PR will fail on python 2.7


How do you use the credits.txt (0.5 Prerelease)

What is your question?
I'm just wondering I saw in the files that credits.txt are a thing and in the 0.5 prerelease notes on github and Im wondering how to use it? It's not important since I'm just wondering so you don't need to reply If you don't want to

check the wiki FAQ

Oh thanks


tag doesn't work with sanitizeHtml

PLEASE NOTE: make sure the bug exists in the latest patch level of the project. For instance, if you are running a 2.x version of Apostrophe, you should use the latest in that major version to confirm the bug.
To Reproduce
  const formattedTitle = 'My Company®';
  console.log(sanitizeHtml(formattedTitle));

The superscript tag doesn't work, even though it is in the default tags
Expected behavior

A clear and concise description of what you expected to happen.

The symbol should be a superscript
Describe the bug

A clear and concise description of what the bug is.

The superscript tag doesn't work, even though it is in the default tags
Details
The superscript tag doesn't work, even though it is in the default tags
Version of Node.js:
PLEASE NOTE: Only stable LTS versions (10.x and 12.x) are fully supported but we will do our best with newer versions.
Node 16
Server Operating System:
The server (which might be your dev laptop) on which Apostrophe is running. Linux? MacOS X? Windows? Is Docker involved?
Mac OS X
Additional context:

Add any other context about the problem here. If the problem is specific to a browser, OS or mobile device, specify which.

Screenshots
If applicable, add screenshots to help explain your problem.

Please provide a unit test that fails (see the tests folder). If this does
not fail when written as a unit test then something is happening in your
larger application that is unrelated to sanitize-html itself.
On Fri, Dec 16, 2022 at 4:11 PM Sami Lieberman @.***>
wrote:

PLEASE NOTE: make sure the bug exists in the latest patch level of the
project. For instance, if you are running a 2.x version of Apostrophe, you
should use the latest in that major version to confirm the bug.
To Reproduce
const formattedTitle = 'My Company®';
console.log(sanitizeHtml(formattedTitle));
[image: Screen Shot 2022-12-16 at 4 10 27 PM]
https://user-images.githubusercontent.com/15946916/208189541-baf52788-da32-4989-848c-823950d6b46a.png
The superscript tag doesn't work, even though it is in the default tags
Expected behavior
A clear and concise description of what you expected to happen.
The symbol should be a superscript
Describe the bug
A clear and concise description of what the bug is.
Details
Version of Node.js:
PLEASE NOTE: Only stable LTS versions (10.x and 12.x) are fully supported
but we will do our best with newer versions.
Server Operating System:
The server (which might be your dev laptop) on which Apostrophe is
running. Linux? MacOS X? Windows? Is Docker involved?
Additional context:
Add any other context about the problem here. If the problem is specific
to a browser, OS or mobile device, specify which.
Screenshots
If applicable, add screenshots to help explain your problem.
—
Reply to this email directly, view it on GitHub
https://github.com/apostrophecms/sanitize-html/issues/595, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAH27JYD6EUDDKOF4MVNN3WNTLIRANCNFSM6AAAAAATBQIKOQ
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

--
THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER
APOSTROPHECMS | apostrophecms.com | he/him/his

I attempted to reproduce the reported issue but couldn't confirm it. The provided test, which checks for the removal of  tags, did not yield the expected result. The  tags were retained, including the registered trademark symbol. Based on my findings, I believe this issue can be closed.
it('should not remove sup tags as they are in default allowed list', function() {   assert.equal(sanitizeHtml('My Company®'), 'My Company®'); });
Please consider reviewing the test and closing the issue accordingly.

@boutell  - We are using another library as a workaround because this is still not working for us. I don't have the bandwidth to look into this at the moment, but my colleague Ganesh is going to pick it up. Thanks!


Questition: How does this project differ from homebridge-homeassistant?

I'm new to homebridge/homeassistant so I apologize for the stupid question, but is the purpose of this project roughly the same as https://github.com/home-assistant/homebridge-homeassistant ?

Well, the goal is to be the same 😉
This is a prepackaged add-on/docker container for Hass.io type installations.
It gives users a single click installation experience, instead of doing it all manually.


Not able to connect on non-standard SSH port.

My server listens on port 2222 instead of the standard 22.
mbtamuli@nyc-512mb ~ » sudo ufw status numbered
Status: active

     To                         Action      From
     --                         ------      ----
[ 1] 22                         DENY IN     Anywhere
[ 2] 2222                       ALLOW IN    Anywhere
[ 3] 2022                       ALLOW IN    Anywhere
[ 4] 22 (v6)                    DENY IN     Anywhere (v6)
[ 5] 2222 (v6)                  ALLOW IN    Anywhere (v6)
[ 6] 2022 (v6)                  ALLOW IN    Anywhere (v6)

I opened the 2022 port for et. Still I am not able to connect. (I already have open 2222)
$ et mbtamuli@example.com:2222
ssh: connect to host techfreak.ga port 22: Connection timed out
SSH handshake failed.

I can confirm it's working perfectly fine for my other server which is listening on the standard SSH port 22

Et and ssh run on different ports. To change the ssh port, you need to pass
a flag to ssh itself using the -s command.
On Sun, Jul 2, 2017, 9:46 PM Mriyam Tamuli<EMAIL_ADDRESS>wrote:

My server listens on port 2222 instead of the standard 22.
mbtamuli@nyc-512mb ~ » sudo ufw status numbered
Status: active
 To                         Action      From
 --                         ------      ----

[ 1] 22                         DENY IN     Anywhere
[ 2] 2222                       ALLOW IN    Anywhere
[ 3] 2022                       ALLOW IN    Anywhere
[ 4] 22 (v6)                    DENY IN     Anywhere (v6)
[ 5] 2222 (v6)                  ALLOW IN    Anywhere (v6)
[ 6] 2022 (v6)                  ALLOW IN    Anywhere (v6)
I opened the 2022 port for et. Still I am not able to connect. (I already
have open 2222)
$ et mbtamuli@example.com:2222
ssh: connect to host techfreak.ga port 22: Connection timed out
SSH handshake failed.
I can confirm it's working perfectly fine for my other server which is
listening on the standard SSH port 22
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/MisterTea/EternalTCP/issues/48, or mute the thread
https://github.com/notifications/unsubscribe-auth/AASuvytcrfZ8dxiVlGenyPfhOnExb47oks5sKHIIgaJpZM4OL1GO
.

Can you please describe how to do that? It doesn't seem to be documented. I tried reading the launcher to figure it out, but from what I could see, the et bash launcher sends the port specified at the end of the whole command to the. The :2222 at the end(As per line #79). I may be wrong.
I've tried the following commands.
$ et -s="mbtamuli@example.com:2222"<EMAIL_ADDRESS>ssh: Could not resolve hostname example.com:2222: Name or service not known
SSH handshake failed.
$ et -s="ssh -p 2222<EMAIL_ADDRESS><EMAIL_ADDRESS>ssh: Could not resolve hostname ssh: Name or service not known
SSH handshake failed.
$ et -s="ssh -p 2222"<EMAIL_ADDRESS>ssh: Could not resolve hostname ssh: Name or service not known
SSH handshake failed.
$ et -s="-p 2222"<EMAIL_ADDRESS>ssh: Could not resolve hostname /bin/bash: Name or service not known
SSH handshake failed.
$ et -s="2222"<EMAIL_ADDRESS>ssh: connect to host 2222 port 22: Invalid argument
SSH handshake failed.

You circumnavigated the right answer, but here's how to do it.  This works for me (I run ET on port 8080):
et -s="-p 22 example.com" example.com:8080

What would make this easier?  Maybe I should have people also put the 'ssh' in the beginning and provide an example in the help?
Would you be willing to edit this file and add some relevant documentation in a pull request once you get it working?  https://github.com/MisterTea/EternalTCP/blob/master/launcher/et
Thanks!

I was thinking about adding example in the Usage Section

That's a good idea.  Here's what you need to edit to change the usage section: https://github.com/MisterTea/EternalTCP/blob/gh-pages/usermanual.md
I think an example in the launcher help would be good as well.  Thanks a lot!

@MisterTea If you want the examples to be worded differently, do tell.

Thanks!!!


web camera scan qr

Hello.
Could you provide some code examples, how to scan qr code using web camera live view.
For ex. via video html tag
Thank you.

Does the note on webcams or demo from the readme provide what you're looking for?

oops. sorry for flood


Obtaining dist and prob maps directly

Hi Martin and Uwe!
I've trained a 3D model using 32x32x32 voxels patches - and so prediction must occur on patches of that size as well(?). But of course my actual 3D images are much larger. I would like to run the model in parallel on all 32x32x32 patches from my full image with a stride of say 24 voxels along each dimension (so 25% overlap), and then stitch the results back together. It seems like the best objects to stitch would be the probability maps and distance predictions, since they are continuous variables, and then do the NMS afterwards on the entire stitched volume. Does this sound like a reasonable approach? Can you guys provide guidance on modifying the source to obtain these objects directly, rather than just the segments and details, from running predict_instances? Or are they available via another function?

Hi @GFleishman,

Or are they available via another function?

Provided that you have enough RAM (~100x input image), tiled prediction already is already provided via model.predict_instances(..., n_tiles = (2,4,4)) which will chop the input images in the respective number of tiles (taking care of overlap etc). You only need to make sure, that the number of tiles is large enough such that a single one fits into GPU memory.

But of course my actual 3D images are much larger

How large is "large"? If the above doesn't work (as your image is e.g. >2 GB), we currently are implementing tiled predictions for large stacks (where not only prob and dist, but the labels itself are stitched). But this is WIP.
Hope that helps,
Martin

Thanks Martin! n_tiles looks like exactly what I need. My image isn't that big ~1024x1014x512 voxels. I've noticed the RAM limitation (running on the CPU on my laptop - I'm happy to say though that with like 10 minutes of training on my CPU I already get pretty darn good results!), especially because I used 256 rays. I may stick with the 256 rays but use lots of tiles at prediction time?

n_tiles looks like exactly what I need. My image isn't that big ~1024x1014x512 voxels.

Ok, for that size that indeed should be the way to go!

(running on the CPU on my laptop - I'm happy to say though that with like 10 minutes of training on my CPU I already get pretty darn good results!

Happy to hear that! Although 10mins on a CPU is like 1 epoch? I would never expect that to work :)

I may stick with the 256 rays but use lots of tiles at prediction time?

Yes, that'd be an option. Alternatively you could use less rays (I practically never use more than ~96). And again, in the future the RAM limitations should be taken care of by the label tiling  (there is already a branch big for that).
Cheers and thanks for the feedback!

Maybe it's just my parameterization(?) but with ~10 minutes on my CPU I'm doing 100 epochs at 5 steps_per_epoch; my training patches are 32^3 (so pretty small?), and otherwise my network is just the default resnet I think. It seems like it converges with that limited training as well:
tensorboard_results_train.pdf
tensorboard_results_validate.pdf
I went with 256 because the plots of IOU over number of rays really saturated there. 128 wasn't too far off, but 64 was at least 0.1 points lower. Maybe part of that is that I'm using a smaller step size for the polygon reconstruction (divided the dx/dy/dz by 4) so maybe it's being more finicky about exactly matching the boundary?

32^3 (so pretty small?),

Indeed, that might be the reason for the fast training

smaller step size for the polygon reconstruction (divided the dx/dy/dz by 4)

You mean grid=4?  Together with that small of input size its interesting that it works at all - are your cells that small?

tensorboard_results_validate.pdf

prob_loss looks still pretty bad though - do the results look fine?

Not grid=4; I was referencing this previous discussion: https://github.com/mpicbg-csbd/stardist/issues/33#issuecomment-583421466
Yeah, 32^3 with grid=4 would be 2^3 patches right?! Definitely too small :)
Qualitatively the results look ok - there are definitely some nuclei missing and very few cells that are just too large. The shape of nuclei is fine, they're all roughly elliptical, but exact boundary matching isn't important to me, if I get 0.6 IOU or better for the majority of cells than I will be very happy.
I don't really know what magnitude of prob_loss would be considered good, I was just looking at the trend. It's so jumpy, it seemed that I had reached the best I could with the data I have - which is rather noisy and was even quite difficult to segment by hand in many places. Qualitatively things look ok (here's a 128x128x128 patch, z-voxels are 2x bigger than x/y voxels hence z dimension looking larger in viewer):

I only manually annotated a 128x128x32 patch in a different part of the brain shown above, so it at least generalizes ok to different ROIs of the acquisition and different z-depths. Would you consider manually lowering the learning rate after so many epochs? Or any other recommendations for getting the prob_loss even lower?

Also - here is what the prob and dists look like for my validation images after 100 epochs of training:

Not grid=4; I was referencing this previous discussion:

Ah, I see :)

Qualitatively the results look ok - there are definitely some nuclei missing and very few cells that are just too large.

Yep, it's pretty good for that the image have a rather bad SNR and are pretty wild along z! Nice!

I only manually annotated a 128x128x32 patch

I see. I would try to set train_patch_size to the largest possible size (e.g. (32,128,128)) to avoid boundary effects. Maybe that will help with the probability. But othe rthan that, your network outputs look reasonable!

Would you consider manually lowering the learning rate after so many epochs?

That will be done automatically during taring (if the loss saturates), so I wouldn't do that.

But if I set train_patch_size to (32, 128, 128) doesn't that mean that I only have a single training patch? Right now I'm cutting it up into 32x32x32 patches (randomly choosing the corner of the crop window), and then randomly applying a little augmentation, just 90 degree rotations and axis flips. That way I can go up to like 200 distinct training patches (hopefully covers all the manually annotated region).

doesn't that mean that I only have a single training patch?

Yes, that would be the case. Although for dense image segmentation it is not really the number of images that matters, but the number of labels and they scale with the image size (as every pixel is practically a single label). One does therefore not really gain much by subdividing the image (apart from having effectively a smaller batchsize) but might loos something due to the border effect. I would thus suggest using the single full stack and do the flip augmentation on it. Additionally I would shift the intensity globally and add noise for augmentation (to increase robustness). Btw, you can do every augmentation on the fly by providing augmenter=aug_func to model.train (where x2,y2 = aug_func(x,y) transforms single image/mask pairs).
Hope that helps!
M

Yep - good suggestions - thanks Martin!

Hi again - I need to revisit this discussion. I've now moved to my cluster and am trying to run on the full image:
pred, det = model.predict_instances(image_norm, n_tiles=(2, 16, 8))
The image is not huge: ~900x1600x300 voxels; but there are many cells, something on the order of 10^5. Whichever part of predict_instances that has the tqdm progress bar finishes in a reasonable amount of time, about 30 minutes, but whatever is after is taking several hours. Due to some network connectivity issues I haven't been able to get through a complete run yet.
I'm guessing the loop monitored by tqdm is model prediction over the tiles, and what comes after is NMS and "polyhedron_to_label"? I guess NMS would scale with the number of voxels, but polyhedron_to_label would scale with the number of labels? Do you have any estimate for how long something like my dataset might take?

I'm guessing the loop monitored by tqdm is model prediction over the tiles, and what comes after is NMS and "polyhedron_to_label"?

Correct, tqdm only monitors the candidate generation via the CNN, afterwards those candidates get filter by the NMS. For a stack of your size, it should not take that long though. You can check how many candidates are proposed and what the nms progress is by using predict_instances(...verbose=True) which should print some info to stdout (if you run it from a notebook it will print it to the calling terminal). Could you check if anything is wrong there? (Eg if there are >>10^6 candidates).
If that's not good enough, you can test the big branch, where improvements for exactly these situations were added (stardist.big.predict_big)

Ok this is running now - had to upgrade to v0.4.1 to get predict_instances with verbose parameter. Just double checking, the NMS and polyhedron_to_label functions do not use anything on the GPU?

It seems like it is finding a very large number of candidates:
predicting instances with prob_thresh = 0.5430663590930008 and nms_thresh = 0.4
found 7046882 candidates
non-maximum suppression...
Non Maximum Suppression (3D) ++++
NMS: n_polys  = 7046882
NMS: n_rays   = 128
NMS: n_faces  = 252
NMS: thresh   = 0.400
NMS: use_bbox = 1
NMS: using OpenMP with 64 thread(s)
NMS: precompute volumes, bounding boxes, etc
precompute done
NMS: calculated anisotropy: 1.05 	 1.00 	 1.37
NMS: starting suppression loop
According to what I've heard informally about this organism, I only expect about 100,000 nuclei in the entire brain. Would you consider increasing the prob_thresh?

According to what I've heard informally about this organism, I only expect about 100,000 nuclei in the entire brain. Would you consider increasing the prob_thresh?

Ok, given that your cells are rather small, it could actually be that 7Mio candidates is reasonable and NMS might take longer. Which system are you on? Does NMS uses all available cores?

Scientific Linux and the scheduler is LSF. I believe the job is using all cores (requested 32; on our system each core comes with a fixed amount of RAM and that's what I needed to accommodate the peak usage), since LSF gives a "Job Efficiency" percentage, which I believe is like a CPU utilization percentage, and it considers all available cores. So I think the job is executing with 64 threads (cores support hyper threading). If there is a better way to check this then I'm happy to.

The current run has been going for about 3.3 hours and is 38% complete - so I'm hoping for a total runtime of less than 12 hours?

Mhhhm, that sounds a bit excessive. Could you predict on a subset (e.g. 1/2 or 1/4 along each dim) and attach the stdout of NMS (verbose=True)?

Yep - this is 1/2 size along each axis. Total run time (including data normalization and network evaluation) was about 18 minutes.
Output:
fleishmang@h10u02:test> python ../test_test.py ../../../data/20200116_hucH2BGC7f_hucGal4_UASrgeco_6dpf_green_full_crop_90x110x10.nrrd ./models ./half_crop_pred.nrrd ./half_crop.nrrd
/groups/scicompsoft/home/fleishmang/Python-3.6.4/lib/python3.6/site-packages/h5py/init.py:36: FutureWarning: Conversion of the second argument of issubdtype from float to np.floating is deprecated. In future, it will be treated as np.float64 == np.dtype(float).type.
from ._conv import register_converters as _register_converters
Using TensorFlow backend.
NORMALIZING
PREDICING
WARNING:tensorflow:From /groups/scicompsoft/home/fleishmang/Python-3.6.4/lib/python3.6/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.init (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
Loading network weights from 'weights_best.h5'.
2020-03-04 10:03:46.625620: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /misc/lsf/10.1/linux3.10-glibc2.17-x86_64/lib:/groups/scicompsoft/home/fleishmang/bin/py2714/lib:/groups/scicompsoft/home/fleishmang/bin/py2714/lib
2020-03-04 10:03:46.625656: E tensorflow/stream_executor/cuda/cuda_driver.cc:318] failed call to cuInit: UNKNOWN ERROR (303)
2020-03-04 10:03:46.625682: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (h10u02.int.janelia.org): /proc/driver/nvidia/version does not exist
2020-03-04 10:03:46.625954: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-03-04 10:03:46.636682: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency:<PHONE_NUMBER> Hz
2020-03-04 10:03:46.640146: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x5af1090 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-03-04 10:03:46.640168: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
Loading thresholds from 'thresholds.json'.
Using default values: prob_thresh=0.543066, nms_thresh=0.4.
image shape:  (430, 838, 150) 	 tiles:  (4, 7, 2)
WARNING:tensorflow:From /groups/scicompsoft/home/fleishmang/Python-3.6.4/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py:422: The name tf.global_variables is deprecated. Please use tf.compat.v1.global_variables instead.
100%|█████████████████████████████████████████████████████████████████████████████████████████| 56/56 [02:38<00:00,  2.58s/it]
predicting instances with prob_thresh = 0.5430663590930008 and nms_thresh = 0.4
found 1044074 candidates
non-maximum suppression...
Non Maximum Suppression (3D) ++++
NMS: n_polys  = 1044074
NMS: n_rays   = 128
NMS: n_faces  = 252
NMS: thresh   = 0.400
NMS: use_bbox = 1
NMS: using OpenMP with 64 thread(s)
NMS: precompute volumes, bounding boxes, etc
precompute done
NMS: calculated anisotropy: 1.21 	 1.00 	 1.36
NMS: starting suppression loop
|#                                                 | [3 % suppressed]
Qhull output at end
Qhull precision warning: repartition point p252 from f308 as a outside point above a hidden facet f306 dist 1.1e-05 nearest vertices 0.011
|############                                      | [24 % suppressed]
Qhull output at end
Qhull precision warning: repartition point p360 from f420 as a outside point above a hidden facet f334 dist 0.0013 nearest vertices 0.23
|#######################                           | [46 % suppressed]
Qhull output at end
Qhull precision warning: repartition point p328 from f239 as a outside point above a hidden facet f259 dist 0.0017 nearest vertices 0.049
|#################################                 | [66 % suppressed]
Qhull output at end
Qhull precision warning: repartition point p204 from f43 as a outside point above a hidden facet f90 dist 0.0095 nearest vertices 0.22
|###########################################       | [86 % suppressed]
Qhull output at end
Qhull precision warning: repartition point p153 from f65 as a outside point above a hidden facet f271 dist 0.00068 nearest vertices 0.092
|##############################################    | [93 % suppressed]
Qhull output at end
Qhull precision warning: repartition point p41 from f94 as a outside point above a hidden facet f96 dist 0.01 nearest vertices 0.08
|################################################  | [97 % suppressed]
NMS: Function calls:
NMS: ~ bbox+out:<PHONE_NUMBER>
NMS: ~ inner:     1380013
NMS: ~ kernel:    1379302
NMS: ~ convex:     444185
NMS: ~ render:     153342
NMS: Excluded intersection:
NMS: + pretest:<PHONE_NUMBER>
NMS: + convex:     290843
NMS: Function calls timing:
NMS: / kernel:   6487.08 s  (4.70 ms per call)
NMS: / convex:   2277.00 s  (5.13 ms per call)
NMS: / render:   1567.44 s  (10.22 ms per call)
NMS: Suppressed polyhedra:
NMS: # inner:         711 / 1044074  (0.07 %)
NMS: # kernel:     935117 / 1044074  (89.56 %)
NMS: # render:      87564 / 1044074  (8.39 %)
NMS: # total:     1023392 / 1044074  (98.02 %)
NMS: # keeping      20682 / 1044074  polyhedra (1.98 %)
NMS took 727.1174 s
keeping 20682/1044074 polyhedra
render polygons...
+++++++++++++++ polyhedra to label +++++++++++++++
n_polys           = 20682
n_rays            = 128
n_faces           = 252
nz, ny, nx        = 430 838 150
use_overlap_label = 0
overlap_label     = 0
Since this is 1/8th of the dataset, of the NMS scaled linearly than a runtime of 20mins * 8 = ~2.5 hours would be perfectly fine. The number of polygons does seem to scale linearly, about 1M here and 7.5M for the full image. So NMS must scale super-linearly with the number of polygons, and/or there is something sub-optimal happening on the software side?

I notice that the first axis of the array is labeled as z - but my data is arranged xyz - x and y have the same voxel sampling rate (0.5189um) and z has larger sampling(1.0um). Is z treated differently in any way?

Did you guys consider any kind of tree structures for increasing the NMS performance?

In 2D, we have an optimization that helps for the (typical) case where the largest object candidate is much smaller in size than the image. We don't have something like that in 3D at the moment.
Using smart data structures would certainly help, but typically makes everything more complicated. So far, it was not the most pressing issue.
As Martin mentioned before, we are close to finishing a new feature that allows predicting on very large images (see predict_big, work in progress), doing the entire pipeline (including NMS) on overlapping blocks and stitching the results back together.

Based on your previous info

pred, det = model.predict_instances(image_norm, n_tiles=(2, 16, 8))
The image is not huge: ~900x1600x300 voxels;

try this (increase n_tiles if you get GPU memory errors):
labels, polys, probs = predict_big(model, image_norm, axes='ZYX', tile_size=(256, 256, 256), min_overlap=32, context=64, n_tiles=(2, 2, 2))

Am I doing something wrong?

That assertion is triggered when tiles overlap not only with their direct neighbors. As mentioned earlier, this is not done and doesn't have proper errors message or documentation.

Thanks Uwe, this is definitely an improvement, mostly w.r.t. memory usage - it seems like I can now get down to below 15GB of RAM (though I haven't run a job to completion yet). Based on the tqdm estimates, the time will still be quite long, ~8 hours, but using only a single cpu (accessible on my cluster now due to lower RAM requirement) will significantly bring down the cost. Is there any interest in running the tiles in a cluster environment with DASK (or similar)?

Based on the tqdm estimates, the time will still be quite long, ~8 hours, but using only a single cpu (accessible on my cluster now due to lower RAM requirement) will significantly bring down the cost.

Why are you only using a single CPU now? The non-maximum suppression will be much faster with more CPU/cores available.

Is there any interest in running the tiles in a cluster environment with DASK (or similar)?

We first want to roll out this functionality with a simple API for single-machine use.
Running this in a distributed environment is certainly appealing, but makes everything much more complicated, hence I can't promise you when/if we'll do it. If you have suggestions on how to do this, please tell us.


Fixed errors in word_adapter and added fuzzing target

I tested our WordAdapter and found and fixed these errors:

The zero extension of the partial read of the last few bytes was implementation-dependent.
The original code was:

#[inline(always)]
fn read_word(&mut self) -> Result<W, std::io::Error> {
    let mut res: W::Bytes = Default::default();
    match self.backend.read_exact(res.as_mut()) {
        Ok(()) => Ok(W::from_ne_bytes(res)),
        // We must guarantee zero-extension
        Err(e) if e.kind() == ErrorKind::UnexpectedEof => Ok(W::from_ne_bytes(res)),
        Err(e) => Err(e),
    }
}

But the documentation of read_exact says:

If this function encounters an “end of file” before completely filling the buffer, it returns an error of the kind ErrorKind::UnexpectedEof. The contents of buf are unspecified in this case.

Indeed, when used with a std::io::Cursor, the res buffer would be empty and not filled with a partial read.

The write_word function would silently drop data.

let _ = self.backend.write(word.to_ne_bytes().as_ref())?;

Here, we need write_all instead of write.
As I explained in the documentation, we have a corner case on the last few bytes because we can read but not write them. Consider:
use dsi_bitstream::prelude::*;
use std::io::Cursor;
let mut data = [0x01, 0x02, 0x03];
let mut adapter = WordAdapter::<u32, _>::new(Cursor::new(data.as_mut()));
// we can read these bytes but they will be zero extended
assert_eq!(adapter.read_word().unwrap(), u32::from_ne_bytes([0x01, 0x02, 0x03, 0x00]));
// Reset the adapter to the start
adapter.set_word_pos(0);
// the write has to fail as we would try to write 4 bytes but there is only
// space for 3
assert!(adapter.write_word(u32::from_ne_bytes([0x02, 0x03, 0x04, 0x05])).is_err());

That's a lot of additional testing in a very critical hotspot. Did you run some benchmarks before and after the change?

The core issue is that using read_exact doesn't give us partial reads  Rust Playground so the code would silently return zero instead of the last few bytes:
use std::io::Read;

fn main() {
    let data = [1, 2, 3];
    let mut buffer = [0; 4];
    assert!(data.as_slice().read_exact(&mut buffer).is_err());
    assert_eq!(buffer, [0; 4]);
}

This is because the read_exact implementation for &[u8] doesn't fill the buffer if the read
is too long:
    #[inline]
    fn read_exact(&mut self, buf: &mut [u8]) -> io::Result<()> {
        if buf.len() > self.len() {
            // `read_exact` makes no promise about the content of `buf` if it
            // fails so don't bother about that.
            *self = &self[self.len()..];
            return Err(io::Error::READ_EXACT_EOF);
        }
        let (a, b) = self.split_at(buf.len());

        // First check if the amount of bytes we want to read is small:
        // `copy_from_slice` will generally expand to a call to `memcpy`, and
        //For a single byte, the overhead is significant.
        if buf.len() == 1 {
            buf[0] = a[0];
        } else {
            buf.copy_from_slice(a);
        }

        *self = b;
        Ok(())
    }

Did you run some benchmarks before and after the change?

Not yet.
In most cases, it shouldn't change anything as the default implementation of Read::read_exact is:
    fn read_exact(&mut self, buf: &mut [u8]) -> Result<()> {
        default_read_exact(self, buf)
    }

Where default_read_exact is:
pub(crate) fn default_read_exact<R: Read + ?Sized>(this: &mut R, mut buf: &mut [u8]) -> Result<()> {
    while !buf.is_empty() {
        match this.read(buf) {
            Ok(0) => break,
            Ok(n) => {
                buf = &mut buf[n..];
            }
            Err(ref e) if e.is_interrupted() => {}
            Err(e) => return Err(e),
        }
    }
    if !buf.is_empty() { Err(Error::READ_EXACT_EOF) } else { Ok(()) }
}

std::fs::File, std::sys::net::Socket, and std::io::Cursor uses the default implementation, but std::io::BufReader has an optimized read_exact:
    // Small read_exacts from a BufReader are extremely common when used with a deserializer.
    // The default implementation calls read in a loop, which results in surprisingly poor code
    // generation for the common path where the buffer has enough bytes to fill the passed-in
    // buffer.
    fn read_exact(&mut self, buf: &mut [u8]) -> io::Result<()> {
        if self.buf.consume_with(buf.len(), |claimed| buf.copy_from_slice(claimed)) {
            return Ok(());
        }

        crate::io::default_read_exact(self, buf)
    }

Where crate::io::default_read_exact is the above implementation.
So my code should be correct, but we will lose that optimization.
I don't know how to make the code correct and optimized.
I thought first to try read_exact and, if that fails, fallback on the new implementation like:
#[inline(always)]
fn read_word(&mut self) -> Result<W, std::io::Error> {
    let mut res: W::Bytes = Default::default();
    // first try read_exact for the optimized case
    match self.backend.read_exact(res.as_mut()) {
        Ok(()) => Ok(W::from_ne_bytes(res)),
        // We must guarantee zero-extension
        Err(e) if e.kind() == ErrorKind::UnexpectedEof => {
            // fallback to the manual impl for correctness
            // as read_exact might not fill res.
            let mut buf = res.as_mut();
            while !buf.is_empty() {
                match self.backend.read(buf) {
                    Ok(0) => break,
                    Ok(n) => {
                        buf = &mut buf[n..];
                    }
                    Err(ref e) if e.kind() == ErrorKind::Interrupted => {}
                    Err(e) => return Err(e),
                }
            }
            Ok(W::from_ne_bytes(res))
        },
        Err(e) => Err(e),
    }
}

Which might be fast for the most common case, and use the extra logic only for the last read.
The problem is that read_exact doesn't specify if the stream position will change if it fails. So the only way to do that would be to require Seek, save the stream position before the read_exact and seek back or infer how much was read. But it adds even more overhead.

I know—it's just that I'm totally puzzled as I recall clearly to have explicitly checked the docs of read_exact to be sure the code was correct. Maybe you're right—there's no impact practice—but let's test this before merging.
Ok, I see what happened: I read
If this function encounters an “end of file” before completely filling the buffer, it returns an error of the kind ErrorKind::UnexpectedEof. The contents of buf are unspecified in this case.
My interpretation was that the remaining content of buf is unspecified. You're telling me that the entire content is unspecified, even in case of a partial read. I was confused by the expression “before completely filling the buffer”, which should be, more precisely, “before reading enough bytes to completely fill the buffer”. The current wording suggests that the buffer is being filled, partially, anyway.


Replace InsertLane format with TernaryImm8

The InsertLane format has an ordering (value().imm().value()) and immediate name ("lane") that make it awkward to use for other instructions. This changes the ordering (value().value().imm()) and uses the default name ("imm") throughout the codebase.

I am also thinking of doing the same for ExtractLane--it's a bit too specific--but that can be a separate PR.

Closing this one in favor of #1770 after chatting offline with @iximeow.


Add Task Status

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@12197fb). Click here to learn what that means.
The diff coverage is n/a.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
@@            Coverage Diff            @@
##             master      #95   +/-   ##
=========================================
  Coverage          ?   57.08%
  Complexity        ?      594
=========================================
  Files             ?      112
  Lines             ?     2323
  Branches          ?      267
=========================================
  Hits              ?     1326
  Misses            ?      939
  Partials          ?       58

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


FontAwesome

Laravel Version: 5.7.x
PHP Version: 7.2.x
Laravel-admin: 1.6.1

Description:
How to remove font awesome 4? I want to replace it with font awesome 5.

Just replace the files under public/vendor/laravel-admin/font-awesome with font awesome 5.

It would not work as the class name changed a lot, even the icon in filed coming with the admin package wouldn't display properly

@RobinHoo1973 there is a polyfill.
@z-song Yeah, I know! I would like to overload Has Assets.
vendor/encore/laravel-admin/src/Traits/HasAssets.php:27

@rez1dent3 when you tried, you'll found its quite headache. I did and rollback few days ago.

yeah. font awesome 5 is needed


Data read fails.

Example request that constantly fail with EOFError:
HTTP.accept(:json).get("http://github.com/").to_s
HTTP.get("https://www.googleapis.com/oauth2/v1/userinfo?alt=json&access_token=foobar").to_s

Failure happens after headers were read. Thus to_s is essential here.
Regression was introduced by: https://github.com/httprb/http/commit/65900ad36e2ec47b1e1c5dce2e801fdf21637a8b

As far as I can see ruby can raise EOFError even if you passed exception: false, e.g.: https://github.com/ruby/ruby/blob/ruby_2_1/ext/openssl/lib/openssl/buffering.rb#L189

:|

I'll fix it in http.rb but I want to get a PR into openssl too and actually fix it properly

Actually wait no, that's not a regression from that commit, it's a regression from timeouts.
That openssl code looks like it could never actually trigger.

Well, probably I have pointed out wrong commit, indeed, it was just lack of that rescue block so I said so :D I could be wrong, indeed. Was also fixing that, but was slower on pushing the fix ;)) My patch looked (rejected in favor of your fix) like this:
diff --git a/lib/http/connection.rb b/lib/http/connection.rb
index 7daeed3..a01785f 100644
--- a/lib/http/connection.rb
+++ b/lib/http/connection.rb
@@ -207,12 +207,15 @@ module HTTP
     def read_more(size)
       return if @parser.finished?

-      value = @socket.readpartial(size)
-      if value == :eof
-        :eof
-      elsif value
-        @parser << value
+      begin
+        value = @socket.readpartial(size)
+      rescue EOFError
+        return :eof
       end
+
+      return :eof if value == :eof
+
+      @parser << value
     end
   end
 end


Refine the setup of http status codes returned by nitro

May be obsolete after the useQuery fix in https://github.com/acdh-oeaw/vicav-vue3/pull/103/commits/dc968ef959989820bddf2e954dc000798e558878


Evaluation result is lower than reported in paper

Dear author,
Thanks for the great work. I trained hstu and sasrec with your code on ml-1m, but the results of last epoch are significantly lower than the result you reported.
Do the reported results are the best evaluation result during training? If so, how do you select the best result.
Best regards

Hi, please apply early stopping on validation set to determine which epoch to use for reading (and please check https://github.com/facebookresearch/generative-recommenders/issues/8 for more details re code used here).


Restore backward compatibility with v0.22.1 and earlier

what

Restore backward compatibility with v0.22.1 and earlier
Allow setting of label_key_case and label_value_case by vars, not just by context attributes.

why

Allow interoperability of old and new modules
Normally root modules make settings via individual variables

/test all

/test all


Update pdf-converter-master to 6.0.0

After making all changes to the cask:

[x] brew cask audit --download {{cask_file}} is error-free.
[x] brew cask style --fix {{cask_file}} left no offenses.
[x] The commit message includes the cask’s name and version.

https://github.com/caskroom/homebrew-cask/pull/45460


EntrustSourcePlugin get_certificates not needed

In file /lemur/plugins/lemur_entrust/plugin.py, the EntrustSourcePlugin is defined.  However its method get_certificates is not implemented, and the comment is "Not needed for ENTRUST".
I don't understand why this method is not needed.  How does this source plugin work then?

@sirferl
Hi there.  I can see you've added this comment.  Can you give some details, please?

Hi @yiluzhu, the source plugin is not implemented for Entrust, but the issuer plugin is implemented. This means you can issue certificates via Entrust
https://github.com/Netflix/lemur/blob/master/lemur/plugins/lemur_entrust/plugin.py#L206-L256
However, you cannot sync already issued certs in Entrust to Lemur.
Is that a functionality you need?

Hi Hossein, thanks for your reply.  Yes we do need this functionality.
You said "you cannot sync already issued certs in Entrust to Lemur", did you mean the code is not implemented?  Or Entrust does not allow Lemur to import its certificates?

hi @yiluzhu it is not implemented at the moment. If there is a demand, I can work in it.

Thank you very much, @sirferl .  If you could work on it that would be really appreciated.

@sirferl @hosseinsh how are you doing, guys?  Have you got a chance to start the source plugin for Entrust?

Hi @yiluzhu, I don't have it planned on my roadmap to add the source plugin for Entrust, but happy to review it if anyone raises a PR.

Hello, I just added a pull request with this functionality.
Sorry that I am not as responsive as I hope to be at the moment,
But there is a lot of other work in my company at the moment to do.
regards

Hi @sirferl , thank you very much for your help! That is much quicker than I expected!

@yiluzhu you're welcome. Please Test it before putting into production. I could just make a quick rundown. If you encounter any errors I will fix them ASAP!

Yes I will test it. Thanks.

@yiluzhu this the PR, if you would like to do some local testing, provide your feedback:
https://github.com/Netflix/lemur/pull/3280

@yiluzhu added a few more fields from the entrust information

@yiluzhu Does this work for you? If yes, we should close the issue.

The testing team is still testing it, and so far so good.  I think we can close it for now.
Thanks again for your help.


Method to bypass required reviewers

We have a config that uses mergeMethod: merge
version: '1'
rules:
  - base: master
    upstream: release
    mergeMethod: merge

Pull requests from pull are automatically merged after someone clicks approve.
Is there any way to get the app to merge automatically (bypassing reviewers?)

Please check "Require pull request reviews before merging" under branch protection.

At this time because of API limitations pull has to follow Github settings when operating on PRs. This includes following the branch protection rules setup under settings.

No way to mark the bot as an admin?

I don't think admins can bypass the protection rules either. This is a Github limitation (aka business decision).

@wei I'm a project maintainer and admins do have the ability / option to bypass protection rules, unless they have specified / checked that also admins need to follow the rules.

@Remzi1993  thanks! I see that Github added the following option:
☑ Include administrators
Enforce all configured restrictions for administrators.
Pull app is using the installation token which cannot be made an admin.

@wei Okay, thanks for the explanation.


renaming config to rescript

Tiny fix. making the settings in vscode be "ReScript" instead of "Example configuration"

@BlueHotDog Good catch! Thanks


Registration

Ahead of saving and loading workflows, we need the saved nodes to be able to know what node package they came from, so that workflows can register these on load, re-create the nodes, then the nodes can load their state from the data. (Note that Macros don't have the same problem, because they know how to create all their children and do so at initialization, so they can simply read off their saved states -- there is an edge case where a macro has replaced one or more of it's children, but we can deal with that later).
In principle each node package should have a unique identifier, but in practice I'm still just using the import module, e.g. "pyiron_workflow.node_library.standard". This "identifier" now gets stored on the node classes when they are registered as a package, so they'll be available to save later.
Otherwise there's a bunch of refactoring, including moving more responsibility onto NodePackage and making the (for now) equivalence of the package identifier and module name more clear.
There's also a bit of extra syntactic sugar so that you can register stuff to a deeply nested domain from the get-go, e.g. Workflow.create.register("my.deep.domain", "pyiron_workflow.node_library.standard"); Workflow.create.my.deep.domain.Add().

Notebook failure appears to just be a versioning inconsistency with pyiron/atomistics; I'll fix it before merging but it doesn't impact the meat here

Original notebook error seems to have been a caching error and outdated version of pyiron/atomistics. Now pyiron_atomistics is breaking in a strange (and still seemingly unrelated) way:
File /usr/share/miniconda3/envs/my-env/lib/python3.11/site-packages/pyiron_base/jobs/job/runfunction.py:700, in handle_failed_job(job, error)
    698     if job.server.run_mode.non_modal:
    699         state.database.close_connection()
--> 700     raise RuntimeError("Job aborted")
    701 else:
    702     return True, out

RuntimeError: Job aborted

One problem was that updates to the extras_require in setup.py are not being automatically propagated to .ci_support/notebooks-env.yml. This is a CI bug, and I opened an issue for it: #148

Looking for: ['jupyter', 'papermill', 'coveralls', 'coverage', 'bidict=0.22.1', 'cloudpickle=3.0.0', 'graphviz=8.1.0', 'matplotlib=3.8.2', 'pympipool=0.7.9', 'python-graphviz=0.20.1', 'toposort=1.10', 'typeguard=4.1.5', 'ase=3.22.1', 'atomistics=0.1.20', 'lammps', 'phonopy=2.21.0', 'pyiron_atomistics=0.4.1', 'pyiron-data=0.0.24', 'numpy=1.26.2']
Could not solve for environment specs
The following packages are incompatible
├─ atomistics 0.1.20**  is requested and can be installed;
└─ pyiron_atomistics 0.4.1**  is uninstallable because it requires
└─ atomistics >=0.1.12,<=0.1.18 , which conflicts with any installable versions previously reported.
Error: Process completed with exit code 1.

Right, the dependabot upgrade of `atomistics` to 0.1.19 got through the tests before because it only impacts the notebooks test, but changes to setup.py by dependabot were not getting propagated to the actual notebooks env so the conflict never arose. I'm going to go patch this in a separate PR.

Closes #152


feat(category_theory): yoneda lemma

This defines the yoneda functor, shows that it is an embedding, and proves the yoneda lemma.

Are we going to implement the silly version?


[Form] (CInputRadio-) Documentation

Hi, I'm having a hard time going through the documentation regarding the form components.
Currently I'm trying to work with the CInputRadio component. I couldn't get them to work properly, but then I found the name prop. Yes, I know the prop is displayed in the example, but the prop is not documented anywhere on the page.
The CInputRadio component uses the label and value prop for the label and value of the respective radio button. But I would like to set a label on the form group. For me it would make more sense to use the options prop to set the radio's labels and values, and use the label prop to set the label for the entire group. But this would require a CInputRadioGroup component I guess.
But trying to figure out a solution for now, in the code I saw there was a CFormGroup component. This component is not documented anywhere on the website. I can use the description prop to achieve my goal, however if I wrap the CInputRadio with the CFormGroup, the CInputRadio disappears from the html source.
Minor: the clicks + Shared props and slots are not referring to the correct anchor.

Hi!
Ad.1 name is not a prop, but attribute, in all components passing when attributes, they have default HTML behavior. Nonetheless, I agree that using HTML attributes should be better documented.
Ad.2
Concerning 'label' prop, it works the same in all components for consistency - creating natural label for the element. You cannot have 'options' unless you have component that renders multiple CRadioInputs. We will consider creating such a component.
For now, you can do it like this:
<CRow form class="form-group">
    <CCol sm="3">
      Radio group name
    </CCol>
    <CCol sm="9" class="form-inline">
      <CInputRadio
        v-for="(option, optKey) in radioOptions"
        :key="option + optKey"
        :label="option"
        name="radioGroupName"
        :value="option"
        :checked="checkedRadioOption === option"
        @update:checked="() => checkedRadioOption = option"
      />
    </CCol>
</CRow>

Doesn't it look clean? ;) And we have full flexibility persisted.
Ad.3
CFormGroup is not documented because its internal component used to render other form components. In fact, it would be better to name this component differently...
Ad.4
Thank you for the bug report.

The documentation is now updated

CInputRadioGroup introduced in af899f5a0f46c96aa7804f7b5711fdd59dc9d943.
Now the application would look like this:
<CRow form>
  <CCol>
    Label
  </CCol>
  <CCol>
    <CInputRadioGroup
      :options="[
        'option 1',
        { value: 'option 2', label: 'label', props: { description: 'description' }}
      ]"
      checked="option 1"
      custom
    />
  </CCol>
</CRow>


Update/further upadation image

updation #7
updation are made now we see clearly text on screen
#7
here screenshots

These bright images makes text hard to read, please add another


Make IdentifierFactory configurable similarly to other Axon Components

For discussion:
having looked at the IdentifierFactory during #635, it's a bit odd that this component cannot be configured like the other Axon components. It seems to be the only one using ServiceLoader. From a developers' perspective i would expect these options:

A method Configuration.configureIdentifierFactory()
DefaultConfigurer defaults to the current DefaultIdentifierFactory
SpringAxonAutoConfigurer registers IdentifierFactoryBean if present
AxonAutoConfiguration providing @ConditionalOnMissingBean the DefaultIdentifierFactory

Introducing this into core however might not be easily done keeping the current IdentifierFactory with the static INSTANCE accessor. So perhaps a new interface 'IdGenerator' or something is in order. Thoughts?

I think your assumption is fair @jorgheymans, hence I've taken the liberty to add it for 3.4.
This might change after further discussion of course.

The risk of introducing a mutable singleton state is simply not worth the (in our eyes) small advantage of having a more easily configurable identifier generator.


Curses Error

~/hakkuframework# ./hakku
Traceback (most recent call last):
File "./hakku", line 55, in
from core import shell
File "/root/hakkuframework/core/shell.py", line 9, in
from core import command_handler
File "/root/hakkuframework/core/command_handler.py", line 6, in
from core import cmethods
File "/root/hakkuframework/core/cmethods.py", line 9, in
import curses
File "/usr/local/lib/python3.6/curses/init.py", line 13, in
from _curses import *
ModuleNotFoundError: No module named '_curses'

Chmod 7777 hakku && sudo python hakku
Problem in root run this tool as root user.

Hello,
Sorry but i dont know what are you talking about. Can u elaborate it please?
As i didn't sent any error conplain for any tool.
On Feb 5, 2019 14:58, "weedguy532"<EMAIL_ADDRESS>wrote:

What problem you have recently faced in this tool as o faced when i run
the program.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/4shadoww/hakkuframework/issues/3#issuecomment-460568451,
or mute the thread
https://github.com/notifications/unsubscribe-auth/Aeh6e7qcF6lmYbjEW_FErUNbFzXEdJZVks5vKU61gaJpZM4QWJvQ
.

Can you explain me how stuff this tool works ?
On Tue 5 Feb, 2019, 3:08 PM kalash99<EMAIL_ADDRESS>wrote:

Hello,
Sorry but i dont know what are you talking about. Can u elaborate it
please?
As i didn't sent any error conplain for any tool.
On Feb 5, 2019 14:58, "weedguy532"<EMAIL_ADDRESS>wrote:

What problem you have recently faced in this tool as o faced when i run
the program.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<
https://github.com/4shadoww/hakkuframework/issues/3#issuecomment-460568451
,
or mute the thread
<
https://github.com/notifications/unsubscribe-auth/Aeh6e7qcF6lmYbjEW_FErUNbFzXEdJZVks5vKU61gaJpZM4QWJvQ
.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/4shadoww/hakkuframework/issues/3#issuecomment-460571855,
or mute the thread
https://github.com/notifications/unsubscribe-auth/Asb2nwu3xx0VpvSRENK57SAeB3JZGp7Zks5vKVErgaJpZM4QWJvQ
.

Traceback (most recent call last):
File "/home/haider/Desktop/hakkuframework/./hakku", line 56, in
from core import scriptio
File "/home/haider/Desktop/hakkuframework/core/scriptio.py", line 4, in
import readline
ModuleNotFoundError: No module named 'readline'


Sorting for Linked list and Doubly linked list

currently, the algorithms in SortingAlgorithms.py are not compatible with Linked list and Doubly linked list

I was trying the linked list and doubly linked list objects on the current algorithms, and they work for in place or mutating versions, but not for the nonmutating versions. Even the change of key works, so the functions that need to be changed are non-mutataing quick and merge sort.


Implemented #194.

Send multiple messages in queue_ all at once.
Size limit is not implemented yet.

Performance checking now...

Removing from queue_ requires to change.

The default value of max_queue_send_count_ is 1. That means the same as the original behavior. async_write() calls one by one messages.
User can set the count using the following API.
     /**
     * @brief Set maximum number of queued message sending.
     *        When async message sending function called during asynchronous
     *        processing, the message is enqueued. When current asynchronous
     *        message is processed, then concatenate queued messages and
     *        send it.
     *        This value limits the maximum number of concatenating messages.
     *        The default value is 1.
     *
     * @param count maximum number of queued message sending. 0 means infinity.
     *
     */
    void set_max_queue_send_count(std::size_t count) {
        max_queue_send_count_ = count;
    }

When I set the count to 0 (infinity), then I got some performance problem. I need to analyze it before merging. I can't user the performance testing environment this week. I will restart the checking next week.

I've checked the performance. When I set the count to 1, then there is no observable difference from the original code. That means we can ignore the additional code cost by the PR.
When I set the count to 512, and tested with very high load e.g. 50000 publish/sec, I observed the behavior change, but no significant throughput improvement is observed.
I think that it depends on environment.
Anyway, if we use with the default setting, there is no difference from the original behavior. If users can tuning the behavior with setting the count and/or size.
I merge the PR.


Style modules are occasionally not loaded in Chrome

I’ve heard a couple of users report this at the office many weeks ago, and now it happened to me as well. I rebooted my laptop, and opened Chrome again, and pasted the URL in. Here’s what I got:

Both of the previous reports also happened on the “Badges” page.

What Chrome version was this?


e2e tests

Issue #, if available:
N/A
Description of changes:

Add e2e tests for primitive filters

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Codecov Report

Merging #41 into master will increase coverage by 0.18%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master      #41      +/-   ##
==========================================
+ Coverage   88.90%   89.09%   +0.18%
==========================================
  Files           9        9
  Lines        1055     1055
==========================================
+ Hits          938      940       +2
+ Misses         81       80       -1
+ Partials       36       35       -1

Impacted Files
Coverage Δ

pkg/selector/selector.go
85.39% <0.00%> (+0.88%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 80cb0b3...568c52b. Read the comment docs.


operators [CI] gitlab-runner-operator (1.6.0)

New Submissions

[ ] Are you familiar with our contribution guidelines?
[ ] Have you packaged and deployed your Operator for Operator Framework?

[ ] Have you tested your Operator in all supported installation modes?
[ ] Have you considered whether you want use semantic versioning order?
[ ] Is your submission signed?
[ ] Is operator icon set?

Updates to existing Operators

[ ] Did you create a ci.yaml file according to the update instructions?
[ ] Is your new CSV pointing to the previous version with the replaces property if you chose replaces-mode via the updateGraph property in ci.yaml?
[ ] Is your new CSV referenced in the appropriate channel defined in the package.yaml or annotations.yaml ?
[ ] Have you tested an update to your Operator when deployed via OLM?
[ ] Is your submission signed?

Your submission should not

[ ] Modify more than one operator
[ ] Modify an Operator you don't own
[ ] Rename an operator - please remove and add with a different name instead
[ ] Modify any files outside the above mentioned folders
[ ] Contain more than one commit. Please squash your commits.

Operator Description must contain (in order)

[ ] Description about the managed Application and where to find more information
[ ] Features and capabilities of your Operator and how to use it
[ ] Any manual steps about potential pre-requisites for using your Operator

Operator Metadata should contain

[ ] Human readable name and 1-liner description about your Operator
[ ] Valid category name1
[ ] One of the pre-defined capability levels2
[ ] Links to the maintainer, source code and documentation
[ ] Example templates for all Custom Resource Definitions intended to be used
[ ] A quadratic logo

Remember that you can preview your CSV here.
--
1 If you feel your Operator does not fit any of the pre-defined categories, file an issue against this repo and explain your need
2 For more information see here

/merge possible

/merge possible

@J0zi @mvalarh This should be ready to merge it seems. I included myself in the ci.yaml file so I should be able to do it myself in the future. Not sure why the DCO check fails as I have included the signoff trailer in my commit.

@ggeorgiev-gitlab please follow instructions and modify according your case.

/merge possible

/merge possible

@J0zi Sorry for bugging you again, the DCO job is passing but the github-actions bot doesn't agree. Not sure how to approach this.

/merge possible


[QST] - Multi-GPU Support w/ Naive model.fit()

❓ Questions & Help
Details
Hello! My team is using T4Rec w/ a BinaryClassificationTask Head. We are utilizing the recommended class method model.fit() rather than the trainer class.
We are currently running experiments on various AWS instances (4xV100 / 8xV100) & it appears that the model is training on a single gpu.
Should the naive model.fit() have distributed training out of the box?
Thanks in advance!
CLI output below during training:
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.57.02    Driver Version: 470.57.02    CUDA Version: 11.4     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Tesla V100-SXM2...  On   | 00000000:00:1B.0 Off |                    0 |
| N/A   62C    P0    88W / 300W |   2132MiB / 16160MiB |      7%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   1  Tesla V100-SXM2...  On   | 00000000:00:1C.0 Off |                    0 |
| N/A   45C    P0    39W / 300W |      3MiB / 16160MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   2  Tesla V100-SXM2...  On   | 00000000:00:1D.0 Off |                    0 |
| N/A   42C    P0    44W / 300W |      3MiB / 16160MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   3  Tesla V100-SXM2...  On   | 00000000:00:1E.0 Off |                    0 |
| N/A   45C    P0    40W / 300W |      3MiB / 16160MiB |      0%      Default |
|                               |                      |                  N/A

Hi @jacobdineen. We have currently two ways of training models with Transformers4Rec PyTorch API:

Trainer.train()
Model.fit()

The 2nd approach was created to make the PyTorch API aligns with the TF API (like Keras model.fit()). But for PyTorch API it is just a simple train loop, as you can see here, which does not support multi-GPU OOTB.
The trainer.train() is the most comprehensive training method, as we inherit our Trainer from the HuggingFace's Trainer and use its train() method, which leverages Torch's DataParallel and DistributedDataParallel options for multi-GPU training, as mentioned in this doc.
So for multi-GPU data parallel training I suggest either using our Trainer.train() or creating you custom train loop based on that method, wrapping the model with DataParallel or DistributedDataParallel.

Thank you, @gabrielspmoreira!
You can close this issue. My team and I will try to write a custom train loop for the keras-style PyTorch API.

@gabrielspmoreira Is there any chance that this will be added as a feature in later releases?

Hi @jacobdineen . Can you be a little bit more specific on what you are looking for. Do you need multi-GPU training but prefer a simpler training loop (like we have for Model.fit()) rather than using the Trainer class we inherit from HF Trainer?

Apologies for not being clear. I'll send over some additional context tomorrow.

@jacobdineen thanks for more contexts. I guess your answer to Gabriel's question Do you need multi-GPU training but prefer a simpler training loop (like we have for Model.fit()) rather than using the Trainer class we inherit from HF Trainer? is a Yes, isnt it?

@rnyak Correct, ideally the simpler training loop would support multi-gpu distributed training.

My team and I will try to write a custom train loop for the keras-style PyTorch API.

@jacobdineen what's the current status with your custom work? I understand,  DistributedDataParallel did not work. Did you test DataParallel and gain any training run time perf improvement with it ? Thanks.

@rnyak
A member of the team is going to test out the solutions from this issue in the next day or so.
As for what we have tried - Assume the following setup:
# type: transformers4rec.torch.model.base.Model
model = transformer_config.to_torch_model(input_module, prediction_task)
model = model.cuda()

train_path = glob.glob(os.path.join(output_dir, f"1/train.parquet"))
train_loader = next(iter(get_dataloader(schema, train_path)))

The workflow we currently follow is incremental (as the tutorials are):
for time_index in range(start_time_window_index, final_time_window_index):
    # fetch paths and dataloaders here
   ...........
    model.fit(train_loader,
              optimizer=optimizer,
              num_epochs=num_epochs,
              amp=False,
              verbose = False)

   # custom model evaluate code here

The issue with torch.nn.DataParallel is that we can not simply wrap the model and call fit, e.g., :
import torch.nn as nn
wrapped_model = nn.DataParallel(model)
wrapped_model.fit(data)

Throws an attribute error that the now DataParallel object has no attribute fit. As discussed above, the torch model class inherits from torch.nn.module, so wrapping the object in DataParallel works for evaluation as it invokes torch.nn.Module.call on the forward pass
So something like this still works for flowing forward:
train_loader = next(iter(get_dataloader(schema, train_path)))
x, y = data[0]

wrapped_model = nn.DataParallel(model)
wrapped_model(x) - computes probas

But to my understanding (correct me if I'm wrong), accessing the t4rec model's class methods via wrapped_model.module.fit() doesn't invoke DataParallel at all.

This is a pretty good example of the issue

this issue

thanks for more context.

This is a pretty good example of the issue

Thanks for more context.

This issue is going to be addressed by task #487

@rnyak , please close this issue after tagging the PRs.


Can pdf-frame-vue use Chinese or other language?

I have tried using xxx.ttc of Chinese, but sadly it does not work. May someone give me some advices?

@PoppingaLi, I see an issue with loading collection fonts (.ttc and .dfont) in the current version. Use .ttf/.woff/.otf files if possible. We will address this issue in the next version soon.
Chinese Font example:  https://stackblitz.com/edit/pdf-frame-vuejs-xyoztz?file=src%2FApp.vue

<i-text
:x="30"
:y="150"
:width="540"
text="你好！"
:style="{
fill: '#ffffff',
textAlign: 'center',
font: '15px Chinese',
}"
“你好” means Hello

@PoppingaLi
PDF-Frame internally uses pdfkit for pdf rendering. By default it supports just ASCII characters. For  non-Ascii characters, You need to use the right font file. I have created an example to showcase multi language support:
https://www.pdf-frame.org/?title=PDF:+Language+Fonts
let me know, if any questions.

@nswamy14 Great job! The font problem is solved. I have a slight question about "[Vue warn]: Failed to resolve component: i-path". Is there some steps about using package which I added not correctly?

@PoppingaLi , looks like you missed this config.
https://github.com/I2Djs/pdf-frame/wiki/pdf‐frame‐vue#config

Issue fixed as part of v2.3.0 version


kubectl get all does not return custom resources on first use (works subsequently)

What happened:
The first time (see below) that I run kubectl get all, it returns "No resource found in {namespace} namespace.
kubectl get all
No resources found in {namespace} namespace.

If I kubectl get {resource} knowing that there are custom resources in the namespace, this command returns correctly:
kubectl get customers
NAME              NAME              DOMAIN
42a3              Foo               foo.com
872f              Bar               bar.com

NOTE I realized that I've duplicated the column name NAME but think this should not the problem, because...

Then (!) rerunning kubectl get all will subsequently correctly (!) return all the custom resources in the namespace:
kubectl get all
NAME              NAME              DOMAIN
42a3              Foo               foo.com
872f              Bar               bar.com

...

NOTE There are multiple resources types returned.

What you expected to happen:
I expect kubectl get all to return all the customer resources in the namespace consistently (first time too).
How to reproduce it (as minimally and precisely as possible):

Every day (!), I deploy a new Google Kubernetes Engine and deploy self-developed CRDs and an operator to the cluster.
For the past few weeks (unfortunately I don't have an audit log of software updates), when debugging, I've noticed the observed behavior (every day the first time I try the command).
Before this, kubectl get all returned the list of custom resources correctly on first use.
Anything else we need to know?:
There are only custom resources in the namespace. There are no Pods etc.
It should not affect the behavior but, I consistently (tomorrow I won't to see whether it makes a difference) kubectl config set-context before I try kubectl get all:
kubectl config set-context --current --namespace={namespace}

It should not affect the behavior but, I'm also forcing the use of gke-gcloud-auth-plugin with an updated ${HOME}/.bashrc:
export USE_GKE_GCLOUD_AUTH_PLUGIN=True

Environment:

Kubernetes client and server versions (use kubectl version):

kubectl version --output=yaml
clientVersion:
  buildDate: "2022-05-24T12:26:19Z"
  compiler: gc
  gitCommit: 3ddd0f45aa91e2f30c70734b175631bec5b5825a
  gitTreeState: clean
  gitVersion: v1.24.1
  goVersion: go1.18.2
  major: "1"
  minor: "24"
  platform: linux/amd64
kustomizeVersion: v4.5.4
serverVersion:
  buildDate: "2022-06-03T03:28:59Z"
  compiler: gc
  gitCommit: c47264b4fe4c0eee76c51b69f1bfcc167fc40c7b
  gitTreeState: clean
  gitVersion: v1.24.0-gke.1801
  goVersion: go1.18.1b7
  major: "1"
  minor: "24"
  platform: linux/amd64

Cloud provider or hardware configuration:

Google Cloud
Kubernetes Engine
Version: 1.24.0-gke.1801

OS (e.g: cat /etc/os-release):

After applying new CRDs, it takes some time for them to be aggregated in discovery endpoint(if there are lots of CRDs, it takes more). It is possible that kubectl get all does not return in first request(or does it get timeout?, kubectl get all -v=9 shows what is happening).
After calling kubectl get {specific_CRDs}, you are forcing discovery caching for these specific CRDs and probably that's why, subsequent kubectl get all returns these.

Today, I left the cluster running (w/ CRDs deployed and custom resources created) for 3 hours before attempting to enumerate the custom resources.
I used the --namespace flag rather than kubectl config set-context ...
The behavior is unchanged from what I described.
kubectl get all --namespace={namespace}
No resources found in {namespace}

kubectl get all --namespace={namespace}
No resources found in {namespace}.

kubectl get customers --namespace={namespace}
NAME              NAME              DOMAIN
42a3              Foo               foo.com
872f              Bar               bar.com

kubectl get all --namespace={namespace}
NAME              NAME              DOMAIN
42a3              Foo               foo.com
872f              Bar               bar.com

I forgot to add --v=9 (will try that tomorrow) but -- IIUC -- that I'm getting No resources found suggests that the command is working correctly.

Hmmmm 🤔
Using -v=9

kubectl get all --namespace={namespace} -v=9

loader.go:372] Config loaded from file:  [REDACTED]
round_trippers.go:466] curl -v -XGET  -H "Accept: application/json;as=Table;v=v1;g=meta.k8s.io,application/json;as=Table;v=v1beta1;g=meta.k8s.io,application/json" -H "User-Agent: kubectl/v1.24.2 (linux/amd64) kubernetes/f66044f" 'https://<IP_ADDRESS>/api/v1/namespaces/[REDACTED]/pods?limit=500'
round_trippers.go:510] HTTP Trace: Dial to tcp:<IP_ADDRESS>:443 succeed
round_trippers.go:553] GET https://<IP_ADDRESS>/api/v1/namespaces/[REDACTED]/pods?limit=500 200 OK in 188 milliseconds

It repeats GETS for the non-custom resources (replicationcontrollers, services etc.) but doesn't appear to check for non-custom resources.

kubectl get {customer-resource} --namespace={namespace} -v=9

loader.go:372] Config loaded from file:  [REDACTED]
discovery.go:214] Invalidating discovery information
round_trippers.go:466] curl -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubectl/v1.24.2 (linux/amd64) kubernetes/f66044f" 'https://<IP_ADDRESS>/api?timeout=32s'
round_trippers.go:510] HTTP Trace: Dial to tcp:<IP_ADDRESS>:443 succeed
round_trippers.go:553] GET https://<IP_ADDRESS>/api?timeout=32s 200 OK in 171 milliseconds
round_trippers.go:570] HTTP Statistics: DNSLookup 0 ms Dial 48 ms TLSHandshake 55 ms ServerProcessing 63 ms Duration 171 ms
round_trippers.go:577] Response Headers:
round_trippers.go:580]     Audit-Id: 456aff2c-84eb-4343-8b13-0eb4ee45ac21
round_trippers.go:580]     Cache-Control: no-cache, private
round_trippers.go:580]     Content-Type: application/json
round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: f9f235c8-7631-4cbc-86ee-08b8880d0902
round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: 8588b28e-012e-4c56-b921-d9a2f30e4523
round_trippers.go:580]     Content-Length: 132
round_trippers.go:580]     Date: Fri, 24 Jun 2022 16:48:24 GMT
request.go:1073] Response Body: {"kind":"APIVersions","versions":["v1"],"serverAddressByClientCIDRs":[{"clientCIDR":"<IP_ADDRESS>/0","serverAddress":"<IP_ADDRESS>:443"}]}
round_trippers.go:466] curl -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubectl/v1.24.2 (linux/amd64) kubernetes/f66044f" 'https://<IP_ADDRESS>/apis?timeout=32s'
round_trippers.go:553] GET https://<IP_ADDRESS>/apis?timeout=32s 200 OK in 50 milliseconds
round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 50 ms Duration 50 ms
round_trippers.go:577] Response Headers:
round_trippers.go:580]     Date: Fri, 24 Jun 2022 16:48:24 GMT
round_trippers.go:580]     Audit-Id: aa0f4f2a-8b9d-472d-9f94-62365006788b
round_trippers.go:580]     Cache-Control: no-cache, private
round_trippers.go:580]     Content-Type: application/json
round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: f9f235c8-7631-4cbc-86ee-08b8880d0902
round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: 8588b28e-012e-4c56-b921-d9a2f30e4523
request.go:1073] Response Body: {"kind":"APIGroupList","apiVersion":"v1","groups":[REDACTED]}

NOTE

Invalidating discovery information
Response Body enumerates all resources including my custom resources

kubectl get all --namespace={namespace} -v=9

loader.go:372] Config loaded from file:  [REDACTED]
discovery.go:214] Invalidating discovery information
round_trippers.go:466] curl -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubectl/v1.24.2 (linux/amd64) kubernetes/f66044f" 'https://<IP_ADDRESS>/api?timeout=32s'
round_trippers.go:510] HTTP Trace: Dial to tcp:<IP_ADDRESS>:443 succeed
round_trippers.go:553] GET https://<IP_ADDRESS>/api?timeout=32s 200 OK in 189 milliseconds
round_trippers.go:570] HTTP Statistics: DNSLookup 0 ms Dial 47 ms TLSHandshake 48 ms ServerProcessing 88 ms Duration 189 ms
round_trippers.go:577] Response Headers:
round_trippers.go:580]     Audit-Id: 8b0f694c-0ffd-4213-8cb4-b1fb22489ec8
round_trippers.go:580]     Cache-Control: no-cache, private
round_trippers.go:580]     Content-Type: application/json
round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: f9f235c8-7631-4cbc-86ee-08b8880d0902
round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: 8588b28e-012e-4c56-b921-d9a2f30e4523
round_trippers.go:580]     Content-Length: 132
round_trippers.go:580]     Date: Fri, 24 Jun 2022 16:50:21 GMT
request.go:1073] Response Body: {"kind":"APIVersions","versions":["v1"],"serverAddressByClientCIDRs":[{"clientCIDR":"<IP_ADDRESS>/0","serverAddress":"<IP_ADDRESS>:443"}]}
round_trippers.go:466] curl -v -XGET  -H "User-Agent: kubectl/v1.24.2 (linux/amd64) kubernetes/f66044f" -H "Accept: application/json, */*" 'https://<IP_ADDRESS>/apis?timeout=32s'
round_trippers.go:553] GET https://<IP_ADDRESS>/apis?timeout=32s 200 OK in 47 milliseconds
round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 47 ms Duration 47 ms
round_trippers.go:577] Response Headers:
round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: 8588b28e-012e-4c56-b921-d9a2f30e4523
round_trippers.go:580]     Date: Fri, 24 Jun 2022 16:50:21 GMT
round_trippers.go:580]     Audit-Id: 912c67cc-b865-411f-9a26-e115ea401fe2
round_trippers.go:580]     Cache-Control: no-cache, private
round_trippers.go:580]     Content-Type: application/json
round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: f9f235c8-7631-4cbc-86ee-08b8880d0902
request.go:1073] Response Body: {"kind":"APIGroupList","apiVersion":"v1","groups":[REDACTED]}
...
round_trippers.go:466] curl -v -XGET  -H "Accept: application/json;as=Table;v=v1;g=meta.k8s.io,application/json;as=Table;v=v1beta1;g=meta.k8s.io,application/json" -H "User-Agent: kubectl/v1.24.2 (linux/amd64) kubernetes/f66044f" 'https://<IP_ADDRESS>/apis/{api}/{version}/namespaces/{namespace}/{resource}?limit=500'
...

NOTE

Once again it invalidates discovery information
Response Body enumerates all resources including my custom resources
GET's all the custom resources

I've retained the logs but would prefer not to share them publicly.

We think this is a known issue with a PR in flight.
/assing @soltysh

https://github.com/kubernetes/kubernetes/pull/96771

It might be similar to https://github.com/kubernetes/kubernetes/pull/96771, but I think the category expander works differently. I'll have to play with it a bit more to get to the root cause.

One last note regarding this issue;
I tested same steps on latest version and it worked as expected.


Plugin crashes on iOS7 devices

Commenting the line below fixes cause of the bug:
https://github.com/chemerisuk/cordova-plugin-firebase/commit/1506f25011116629522918a7101922b8f9642f40
Not sure why you call registerForRemoteNotifications at applicationDidFinishLaunching. May be better to do it in grantPermission?
Code below should work on all devices, but I don't have a chanse to do appropriate testing...
#if __IPHONE_OS_VERSION_MAX_ALLOWED >= 80000
  if ([[UIApplication sharedApplication]respondsToSelector:@selector(registerUserNotificationSettings:)]) {
    UIUserNotificationSettings *settings = [UIUserNotificationSettings settingsForTypes:UserNotificationTypes categories:nil];
    [[UIApplication sharedApplication] registerUserNotificationSettings:settings];
    [[UIApplication sharedApplication] registerForRemoteNotifications];
  } else {
    [[UIApplication sharedApplication] registerForRemoteNotificationTypes:notificationTypes];
  }
#else
    [[UIApplication sharedApplication] registerForRemoteNotificationTypes:notificationTypes];
#endif

excellent. 31c5e9c05d9708949cdd937fdecddfb06ed461e9

integrated in 0.1.11


Bug:

Sleep command sends my pc into a deep sleep / hibernate instead of sleep
When sending pc to sleep manually, it goes into a regular sleep
To Reproduce
Send sleep command to computer from home assistant
Expected behavior
The computer goes into sleep mode
Misc info (please complete the following information):

Windows 11 Build 25151.1010
Windows' UI language: English
HASS.Agent version: 2022.12.0

Please check what's applicable (multiple answers possible):

[x] Installed via installer
[ ] Installed manually
[x] Problem occurs in HASS.Agent
[ ] Problem occurs in Satellite Service

Did you read this part of the description:


Delete CODEOWNERS in main branch

Summary of the pull request
References and relevant issues
Detailed description of the pull request / Additional comments
Validation steps performed
PR checklist

[ ] Closes #xxx
[ ] Tests added/passed
[ ] Documentation updated

Closing this PR and opening a new one to try testing the new PR validation pipeline


Moved application configuration to client configuration

Most configuration options need to vary by clients.

Coverage remained the same at 100.0% when pulling a58825f97fa48edb4a47b5b5c97c5adcb2dc4b3c on feature/deprecate_application_config into 0485526e10eed0f561ecf6c89dc190dac84bb937 on develop.


Invalid declarations emitted when using ZodObject from zod as return type of generic function

Bug Report
🔎 Search Terms
zod, ZodObject, declaration, invalid declaration emit, extends infer T, union distribution, Type 'k' cannot be used to index type 'addQuestionMarks', skipLibCheck, error 2536, generic function return type
🕗 Version & Regression Information
Observed in all versions starting from 4.7. Tested on 4.7.4, 4.8.4, 4.9.5, 5.0.4, 5.1.3, 5.2.0-dev20230607. Works correctly on 4.6.4.

This changed between versions 4.6.4 and 4.7.4
This is the behavior in every version I tried, and I reviewed the full FAQ

⏯ Playground Link
The playground contains relevant simplified type definitions from zod library in an attempt to produce a minimal reproducible example.
Playground link with relevant code
💻 Code
export const buildSchema = <V extends string>(version: V):
  objectOutputType<{ version: ZodLiteral<V>; }> =>
    ({}) as any;

Ignore ({}) as any, it is just to avoid writing function implementation.
🙁 Actual behavior
Code gets compiled into the following declaration:
export declare const buildSchema: <V extends string>(version: V) => addQuestionMarks<baseObjectOutputType<{
    version: ZodLiteral<V>;
}>, undefined extends V ? never : "version"> extends infer T ? { [k in keyof T]: addQuestionMarks<baseObjectOutputType<{
    version: ZodLiteral<V>;
}>, undefined extends V ? never : "version">[k]; } : never;

This declaration produces the following type error when copy-pasted back into TS playground:
Type 'k' cannot be used to index type 'addQuestionMarks<baseObjectOutputType<{ version: ZodLiteral<V>; }>, undefined extends V ? never : "version">'.(2536)

🙂 Expected behavior
The generated declaration should be valid.
For example, this would be a valid declaration:
export declare const buildSchema: <V extends string>(version: V) => addQuestionMarks<baseObjectOutputType<{
    version: ZodLiteral<V>;
}>, undefined extends V ? never : "version"> extends infer T ? { [k in keyof T]: T[k]; } : never;

4.6.4 produces declaration without extends infer T which is also valid:
export declare const buildSchema: <V extends string>(version: V) => { [k in keyof addQuestionMarks<baseObjectOutputType<{
    version: ZodLiteral<V>;
}>, undefined extends V ? never : "version">]: addQuestionMarks<baseObjectOutputType<{
    version: ZodLiteral<V>;
}>, undefined extends V ? never : "version">[k]; };

Related issues:

https://github.com/microsoft/TypeScript/issues/49572
https://github.com/microsoft/TypeScript/issues/49591
https://github.com/colinhacks/zod/issues/2260
https://github.com/colinhacks/zod/issues/1986

Special thanks to @KaoSDlanor for posting an issue in the zod repo than inspired me to write this one and @pivstone for helping me with the investigation

Is this still planned to be worked on for 5.7.0?


Directly re-exec LXD when requesting to remove self from a cluster

Addresses the EOF error in #9464
This is the full fix after investigation for #9468
When a node requests to remove itself from the database, the request is forwarded to the leader, which removes the node from dqlite and then sends another request to the to-be-removed node to disable clustering on itself and re-exec its daemon. This happens via the following in lxd/api_cluster.go
put := api.ClusterPut{}
put.Enabled = false
_, err = client.UpdateCluster(put, "")

This made it so that flushing the response in clusterPutDelete was rendering to the leader node, even though we were initiating the request from the to-be-removed node. The daemon on the requesting node would restart before the response could be sent back.
Instead of asking the leader to request the to-be-removed node to disable clustering, this commit makes the original requesting node immediately disable clustering on itself and re-exec its daemon if the request goes through, and prevents the leader from sending the update request if the request came from the node to be removed.

@masnax the test suite is red across the board I'm afraid. There's at least 2 different issues, one is code style in the tests and the other is an actual functionality issue.

Taking a look at this.

@masnax the test suite is red across the board I'm afraid. There's at least 2 different issues, one is code style in the tests and the other is an actual functionality issue.

That's what I get for a late-night PR :)
Pretty simple fixes here. Just some formatting on the tests, and I'm checking the wrong variable r.Host. It should be r.RemoteAddr and the port should be cut off.

Thanks, in my tests as well as the EOF error which hopefully is fixed now, I also see that LXD hangs when restarting.

I'll push this branch down fresh and try again.

Missed one line in the test. Static analysis never complains for me locally :(

@stgraber @masnax added systemd socket activation detection and exits in that case rather than replacing the LXD process, as otherwise we were losing the socket FDs and environment variables and were incorrectly trying to connect to ourselves to check if running already, causing a hang.

I'm going with a slightly different approach that doesn't depend on remote address comparison logic, as that has proven unreliable in tests, and we can do it a different way internally without having to rely on remote address logic.


Update toml_edit crate

Current dependencies on/for this PR:

main

PR #234

PR #235   👈

PR #236

This comment was auto-generated by Graphite.

Merge Activity

Sep 19, 8:38 AM: @zurawiki merged this pull request with Graphite.