LL-9022 - Improve legibility of app sizes in the manager for LNS+

Type
Feature improvement
Context
https://ledgerhq.atlassian.net/browse/LL-9022
1 - New device has more precise values data about storage (apps, firmware allocation, free space, etc...). We don't really need it on UI side for now. So we have to control the precision and the values to show.
2 - We wanted to better handle the rounding of data being less optimistic. When we are showing data about used space, we want to ceil the value and when it's about free space we will floor it to always unsure user will not be surprised thinking he could install something but when I do it, it fails cause missing few bytes on nano device.
Parts of the app affected / Test plan
1 - Use a nano SP and go to the manager
2 - Check it works with other devices

Please try adding some description, a pretty screenshot, and how to test this easily @tlaforge-ledger it goes a long way for others to know what this is about without having to read a, potentially long, Jira task.

Like I asked for, two months ago, please update the PR with context, description and screenshots. @tlaforge-ledger
Since the PR is now very outdated, rebase (or reset like on LLD) and request a review again. Ping me when it's ready for review

I updated the Jira task since we split LLD and LLM onces

I can't merge but it's in QA OK @gre


Info type is annoying to access

having to import Info like this:
from strawberry.types.info import Info

feels a bit annoying
I wonder if we can (and should) allow this:
strawberry.Info[...]

I think that would be very nice!

In my experience you usually have to override it to bind it with your specific context class
But having Info accessible in strawberry module would be nice
@dataclasses.dataclass
class Context:
    ...

Info = StrawberryInfo[Context, None]

from something.something import Info


Mac OS installation problem with Microsoft Visual Studio Code

The major problem comes when I downloaded a package from the official webspace and as I tried to run it, it cracks up and pops up message that it is not uptodate thing. Well I downloaded it few minutes ago what should I drop I also tried with home-brew and Curl codes still they aren't working

/duplicate - see https://github.com/microsoft/vscode/issues/91593#issuecomment-591682821 for latest advice


0.17.1-lite: Container not starting

Running frooodle/s-pdf:0.17.1-lite via Docker on Debian 12.4. The container is not able to start with this error message:
Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "/scripts/init.sh": stat /scripts/init.sh: no such file or directory: unknown
The non-lite version works as expected.
If needed, here is my compose:
version: '3.3'

services:
  stirling-pdf:
    container_name: pdf_mydomain_tld
    image: frooodle/s-pdf:0.17.1-lite
    ports:
      - ':8080'
    volumes:
      - /home/user/bind-mounts/stirling-pdf/trainingData:/usr/share/tesseract-ocr/5/tessdata
      - /home/user/bind-mounts/stirling-pdf/extraConfigs:/configs
      - /home/user/bind-mounts/stirling-pdf/customFiles:/customFiles/
    environment:
      - VIRTUAL_HOST=pdf.mydomain.tld
      - LETSENCRYPT_HOST=pdf.mydomain.tld
      - DOCKER_ENABLE_SECURITY=true
      - SECURITY_ENABLE_LOGIN=true
      - SYSTEM_DEFAULT_LOCALE=de-DE
      - SYSTEM_GOOGLE_VISIBILITY=false
    networks:
      - net
    restart: always

networks:
  net:
    external: true

Can replicate. Latest working version is 0.16.1-lite.
Regression was brought in 78473e9. Affects images 0.17.0-lite and later fail to start /scripts/init.sh: no such file or directory
Also related 206cf40. Images 0.17.0-ultra-lite and later fail to start /scripts/init.sh: no such file or directory

My bad will fix this today


[SYM-4204] Allow the Runtime IAM role to assume itself

Due to recent changes by AWS, this behavior must be explicitly defined in the policy.
While it's undesirable for Sym to need self-referential IAM role assumption permission long term, this will unblock any new users for now.

Current dependencies on/for this PR:

main

PR #3   👈

This comment was auto-generated by Graphite.


[TD-968]: Implementing releng bundle using gromit policy with multiple version support.

This PR will

Translate the m4 templates to go templates for direct rendering of releng code with gromit policy.
Will add support for multiple versions(from within the templates)

This approach is fine. We'll have to fix the tests though. I'll work on applying auth from TD-1025 in this PR.

All the files managed by tyk-ci except the goreleaser-el7.yml has been converted to text/template, and is generated for gateway.
Generated test PR is here: https://github.com/tyklabs/tyk/pull/8.

test PR is here: tyklabs/tyk#8
Please create a PR on TykTechnologies/tyk for testing.

Please create a PR on TykTechnologies/tyk for testing.

Created one here: https://github.com/TykTechnologies/tyk/pull/4148

https://github.com/TykTechnologies/tyk/pull/4157
Release successful in this PR, after the fix above.

Please update this PR to update Dockerfile.std as suggested by @titpetric in https://github.com/TykTechnologies/tyk/pull/4157
This PR can be merged after https://github.com/TykTechnologies/tyk/pull/4157 is merged.

@alephnull I've also added the el7 changes in this PR itself, since it's only a few files, will additionally add the latest dockerfile changes too.

Sounds good @asutosh. The plan is,

make https://github.com/TykTechnologies/tyk/pull/4157 merge-able to master
merge https://github.com/TykTechnologies/tyk/pull/4157
wait 3-4 days for real world testing
make any needed changes to this PR
merge this PR

@asutosh Lets setup a pairing session to fix these tests. Since we are going to use gromit in the short-term, these tests need to be fixed.


[Bug] Kyverno Block Stale images not working, its wrongly calculating the image age

Kyverno Version
1.10.3
Description
re:f: https://kyverno.io/policies/other/b-d/block-stale-images/block-stale-images/
we have setup a kyverno version: 1.10.3 & 1.7.5, in both we are facing the block-stale-images isn't working as expected. its wrong calculating, we are using AWS ECR for image registry.
Bug:
I0911 11:58:48.789958       1 evaluate.go:24] EngineValidate "msg"="a condition failed in 'all' block" "kind"="Deployment" "name"="xyz" "namespace"="xyz" "policy"="block-stale-images" "rule"="autogen-block-stale-images" "condition"={"key":"94h35m32.961287512s","operator":"GreaterThan","value":"4380h"}
I0911 11:58:48.973298       1 validate_audit.go:200] ValidateAuditHandler "msg"="validation failed" "action"="validate" "gvk"="apps/v1, Kind=Deployment" "operation"="UPDATE" "resource"="xyz/Deployment/xyz" "failed rules"=["autogen-block-stale-images"] "policy"="block-stale-images"
Please let me know how to move forward.
Slack discussion
No response
Troubleshooting

[X] I have read and followed the documentation AND the troubleshooting guide.
[X] I have searched other issues in this repository and mine is not recorded.

Transferred to kyverno/policies.
{"key":"94h35m32.961287512s","operator":"GreaterThan","value":"4380h"}

94 hours is, in fact, NOT greater than 4,380 hours. What about this is wrong?

thanks @chipzoller for your reply, thats where the confusion begins, its correct if you see the next line, the validation failed. would you pl help this ?

The log is just saying the condition evaluated to a false result. For a deny to occur, the condition must evaluate to a true.

we have used the same ref: https://kyverno.io/policies/other/b-d/block-stale-images/block-stale-images/ do we need to change anything from the policy side, pl share your recommendations. thanks.

Does it work or not work? What's the problem here?

itsnt working, its giving false validation. for the image which was build a week or 3-4 days it showing failed as stale block image

Please show how you're testing this and the exact result.

@chipzoller - pl find the below log
I0911 11:58:48.973298 1 validate_audit.go:200] ValidateAuditHandler "msg"="validation failed" "action"="validate" "gvk"="apps/v1, Kind=Deployment" "operation"="UPDATE" "resource"="xyz/Deployment/xyz" "failed rules"=["autogen-block-stale-images"] "policy"="block-stale-images"

Forget the log. Does the policy work or not?

@chipzoller  - Again. Policy not working as expected, its giving false values. we are also using kyverno-policy reporter UI. Pl see the below for one of the deployments.

@chipzoller  - its same behavior in both 1.10.3 & 1.7.5 versions. Could you pl share how the image age is calculated by kyverno,  we are using AWS ECR. I need to figure why/how kyverno is wrong calculating, is there a way we can deep dive and see ? this policy is giving false positive for latest images which was deployed 3-4 days before.

I think maybe you might not understand how this policy works. This policy checks the created date of the image and NOT when it was deployed in a Pod. If an image was built more than 6 months ago, it will fail this policy.
You can check with the crane tool (install with homebrew using brew install crane) on the created field.
crane config ghcr.io/chipzoller/zulu:v0.0.14 | jq .created
"2022-10-06T02:15:30Z"

Please use code blocks and not screenshots for text and terminal output. We can't search it. No need to keep mentioning me, I see your responses.
Do you get a failed result for a fresh deployment of a new Pod?

sure, It failed for fresh and recently deployed ones also.

Please show the full policy you have (sanitize if you need) as well as the Policy Report where it's showing the failure. I know you're using Policy Reporter but I want to see the Policy Report YAML manifest itself.

Thanks @chipzoller for sharing the details, but pl check the created date.

Here is the code output of the command
crane config XXXXXXXXXX.dkr.ecr.eu-west-2.amazonaws.com/XXXXX:v0.66.0 | jq .created "2023-08-23T09:39:48.342672082Z"
Here is the YAML policy.
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  annotations:
    kyverno.io/kubernetes-version: "1.23"
    kyverno.io/kyverno-version: 1.10.3
    pod-policies.kyverno.io/autogen-controllers: DaemonSet,Deployment,Job,StatefulSet,CronJob
    policies.kyverno.io/category: Other
    policies.kyverno.io/description: 'Images that are old usually have some open security
      vulnerabilities which are not patched. This policy checks the contents of every
      container image and inspects them for the create time. If it finds any image
      which was built more than 6 months ago this policy blocks the deployment.      '
    policies.kyverno.io/minversion: 1.6.0
    policies.kyverno.io/severity: medium
    policies.kyverno.io/subject: Pod
    policies.kyverno.io/title: Block Stale Images
  creationTimestamp: "2023-03-23T13:20:17Z"
  generation: 306326
  labels:
    kustomize.toolkit.fluxcd.io/name: kyverno-policies-kustomize
    kustomize.toolkit.fluxcd.io/namespace: kyverno
  name: block-stale-images
  resourceVersion: "921818965"
  uid: XXXXX
spec:
  background: true
  failurePolicy: Fail
  rules:
  - exclude:
      resources: {}
    generate:
      clone: {}
    match:
      any:
      - resources:
          kinds:
          - Pod
      resources: {}
    mutate: {}
    name: block-stale-images
    validate:
      foreach:
      - context:
        - imageRegistry:
            reference: '{{ element.image }}'
          name: imageData
        deny:
          conditions:
            all:
            - key: '{{ time_since('''', ''{{ imageData.configData.created }}'', '''')
                }}'
              operator: GreaterThan
              value: 4380h
        list: request.object.spec.containers
      message: Images built more than 6 months ago are prohibited.
  - exclude:
      resources: {}
    generate:
      clone: {}
    match:
      any:
      - resources:
          kinds:
          - DaemonSet
          - Deployment
          - Job
          - StatefulSet
      resources: {}
    mutate: {}
    name: autogen-block-stale-images
    validate:
      foreach:
      - context:
        - imageRegistry:
            reference: '{{ element.image }}'
          name: imageData
        deny:
          conditions:
            all:
            - key: '{{ time_since('''', ''{{ imageData.configData.created }}'', '''')
                }}'
              operator: GreaterThan
              value: 4380h
        list: request.object.spec.template.spec.containers
      message: Images built more than 6 months ago are prohibited.
  - exclude:
      resources: {}
    generate:
      clone: {}
    match:
      any:
      - resources:
          kinds:
          - CronJob
      resources: {}
    mutate: {}
    name: autogen-cronjob-block-stale-images
    validate:
      foreach:
      - context:
        - imageRegistry:
            reference: '{{ element.image }}'
          name: imageData
        deny:
          conditions:
            all:
            - key: '{{ time_since('''', ''{{ imageData.configData.created }}'', '''')
                }}'
              operator: GreaterThan
              value: 4380h
        list: request.object.spec.jobTemplate.spec.template.spec.containers
      message: Images built more than 6 months ago are prohibited.
  validationFailureAction: audit

Here is the policy report
kubectl get polr cpol-block-stale-images -n XXXX -o jsonpath='{.results[?(@.result=="fail")]}' | yq -p json -
category: Other
message: 'validation failure: Images built more than 6 months ago are prohibited.'
policy: block-stale-images
resources:
  - apiVersion: apps/v1
    kind: Deployment
    name: myDeployment
    namespace: xxxx
    uid: xxx
result: fail
rule: autogen-block-stale-images
scored: true
severity: medium
source: kyverno
timestamp:
  nanos: 0
  seconds: 1.694463715e+09
---
category: Other
message: 'validation failure: Images built more than 6 months ago are prohibited.'
policy: block-stale-images
resources:
  - apiVersion: apps/v1
    kind: Deployment
    name: mydeployment
    namespace: xxxx
    uid: XXXX
result: fail
rule: autogen-block-stale-images
scored: true
severity: medium
source: kyverno
timestamp:
  nanos: 0
  seconds: 1.694463042e+09
---
category: Other
message: 'validation failure: Images built more than 6 months ago are prohibited.'
policy: block-stale-images
resources:
  - apiVersion: v1
    kind: Pod
    name: mypod
    namespace: XXXX
    uid: XXXX
result: fail
rule: block-stale-images
scored: true
severity: medium
source: kyverno
timestamp:
  nanos: 0
  seconds: 1.694463638e+09
---
category: Other
message: 'validation failure: Images built more than 6 months ago are prohibited.'
policy: block-stale-images
resources:
  - apiVersion: v1
    kind: Pod
    name: mypod
    namespace: XXXX
    uid: XXXX
result: fail
rule: block-stale-images
scored: true
severity: medium
source: kyverno
timestamp:
  nanos: 0
  seconds: 1.694463066e+09
---
category: Other
message: 'validation failure: Images built more than 6 months ago are prohibited.'
policy: block-stale-images
resources:
  - apiVersion: v1
    kind: Pod
    name: mypod
    namespace: XXXX
    uid: XXXX
result: fail
rule: block-stale-images
scored: true
severity: medium
source: kyverno
timestamp:
  nanos: 0
  seconds: 1.694463633e+09
---
category: Other
message: 'validation failure: Images built more than 6 months ago are prohibited.'
policy: block-stale-images
resources:
  - apiVersion: v1
    kind: Pod
    name: mypod
    namespace: XXXX
    uid: XXXX
result: fail
rule: block-stale-images
scored: true
severity: medium
source: kyverno
timestamp:
  nanos: 0
  seconds: 1.694463063e+09

Please let us know if you need anything more. thanks

What I'm curious to know is can you reproduce this with a fresh Deployment using this image, right now, by creating a net new one? Take this image, leave your policy alone, and use it to create a new Deployment called foo. Does foo show up as failed in the same Policy Report even though the image, as reported by crane, is less than 4,380 hours old?

Looks like the container passes the policy validation when used in a different deployment. I tested in the default namespace and it worked/passed
I deleted the deployments that have the issue and let Flux create them again, the policy is still failing for these after they have been recreated
Evidence below of the test
kubectl get deployment foo-kyverno -n default
NAME          READY   UP-TO-DATE   AVAILABLE   AGE
foo-kyverno   0/1     1            0           22m

kubectl describe deployment foo-kyverno -n default
Name:                   foo-kyverno
Namespace:              default
CreationTimestamp:      Mon, 11 Sep 2023 22:01:37 +0100
Labels:                 app=foo-kyverno
Annotations:            deployment.kubernetes.io/revision: 1
Selector:               app=foo-kyverno
Replicas:               1 desired | 1 updated | 1 total | 1 available | 0 unavailable
StrategyType:           RollingUpdate
MinReadySeconds:        0
RollingUpdateStrategy:  25% max unavailable, 25% max surge
Pod Template:
  Labels:  app=foo-kyverno
  Containers:
   foo-kyverno:
    Image:        xxxx.dkr.ecr.eu-west-2.amazonaws.com/xxxx:xxxx
    Port:         8080/TCP
    Host Port:    0/TCP
    Environment:  <none>
    Mounts:       <none>
  Volumes:        <none>
Conditions:
  Type           Status  Reason
  ----           ------  ------
  Progressing    True    NewReplicaSetAvailable
  Available      True    MinimumReplicasAvailable
OldReplicaSets:  <none>
NewReplicaSet:   foo-kyverno-7585564466 (1/1 replicas created)
Events:
  Type    Reason             Age   From                   Message
  ----    ------             ----  ----                   -------
  Normal  ScalingReplicaSet  23m   deployment-controller  Scaled up replica set foo-kyverno-7585564466 to 1

The kyverno policy passes without failing, the failure below is on the echo-deployment, the foo-kyverno passes
kubectl get polr cpol-block-stale-images -n default -o jsonpath='{.results[?(@.result=="fail")]}' | yq -p json -
category: Other
message: 'validation failure: Images built more than 6 months ago are prohibited.'
policy: block-stale-images
resources:
  - apiVersion: apps/v1
    kind: Deployment
    name: echo-deployment
    namespace: default
    uid: 26756c09-0d14-401b-8a70-7c720641c8f0
result: fail
rule: autogen-block-stale-images
scored: true
severity: medium
source: kyverno
timestamp:
  nanos: 0
  seconds: 1.694466132e+09

Update - looks like the pod contained 2 container images but kyverno does not tell you which one is failing, it just tells you the whole pod fails the policy, when we looked closer it was the init container that was over 6 months. Is there any way to get kyverno to report in the container image within the pod that is has found to violate the policy?

Ok, that explains it.

Is there any way to get kyverno to report in the container image within the pod that is has found to violate the policy?

Yes, you can use the new message field in the conditions block to add a custom message along with a variable for the image. See here.

Issue is confirmed not a bug but glad to keep discussing.


Should Rails/Output Skip migrations and generators by default?

The Rails/Output cop is flagging p and puts inside migrations and generators saying I should use the logger instead. Rails.logger does not appear to be a drop in replacement for them in these contexts. Rails.logger.info goes to log/dev.log which is incorrect, it should be displayed to the screen like the rest of the generator or migration output.
Should Rails/Output be disabled by default for generators and migrations? Or does Rails/Output need extra documentation about the right thing to do inside generators and migrations?

For migrations, you can use say and say_with_time:
https://guides.rubyonrails.org/active_record_migrations.html#changing-the-output-of-running-migrations
For generators, the readme method can output the contents of a file:
https://guides.rubyonrails.org/generators.html#readme
Does that cover your use cases?

If this issue is the same problem as #934 will be solved by #965.


Move search implementation to q-server

This PR moves the search implementation to the q-server
There is now an endpoint which takes requests in the following form:

GET /search?
tool=imageslider&
<EMAIL_ADDRESS>department=Storytelling%20/%20Infografik&
publication=nzz&
searchString=Bundesrat&
limit=18

This will be in the next release.


Added links scraping functionality to parser

I downloaded this package looking for something that could parse links within the main article, but noticed the functionality wasn't included, so I added some small changes so the Article object also collects a list of links when parse() is called.
Users will now by able to access a list of links in the article with article.links after calling article.parse().

Looks like an issue with the article.text being truncated:
======================================================================

FAIL: test_thai_fulltext_extract (__main__.MultiLanguageTestCase)

----------------------------------------------------------------------

Traceback (most recent call last):

  File "tests/unit_tests.py", line 42, in run

    method(*args, **kw)

  File "tests/unit_tests.py", line 741, in test_thai_fulltext_extract

    self.assertEqual(text, article.text)

AssertionError: "21 ม[1272 chars]0 วัน\n\nเรื่องที่เกี่ยวข้อง\n\nลืออุ้ม อ.สุรช[108 chars]่าน'" != "21 ม[1272 chars]0 วัน"

Diff is 1955 characters long. Set self.maxDiff to None to see it.

----------------------------------------------------------------------

Kind of stumped by this one, any ideas?


Preserve order or elements in DynamicSetProperty and DynamicMapProperty

NetflixOSS » archaius » archaius-pull-requests #201 FAILURE
Looks like there's a problem with this pull request

archaius-pull-requests #369 FAILURE
Looks like there's a problem with this pull request

archaius-pull-requests #370 SUCCESS
This pull request looks good

NetflixOSS » archaius » archaius-pull-requests #202 FAILURE
Looks like there's a problem with this pull request


[docs] Clarify non-persistent topics guaranties part

It was unclear.
"Consumers" looks like chain of persistent-topic-A -> function -> non-persistent-topic-B -> function -> persistent-topic-C will be zero loss on restart of brokers, but unfortunately it's seems not to be true:)

run java8 tests

run java8 tests

run java8 tests

run java8 tests


How do I make something wait?

Describe your problem here.

I was making a note where if you hit it, the window will float away offscreen. It works, but it doesnt work.
Whenever I tried using a wait function in order to make the two tweens seperate, it didnt work.
Thats when i learned something, THERE IS NO WAIT FUNCTION. Not in this version of lua, anyway.
All of the other things said "use os.time" or something like that but you can't use os commands in funkinlua because of security risks?
Im confused. help.
Are you modding a build from source or with Lua?
Lua
What is your build target?
Windows x64
Did you edit anything in this build? If so, mention or summarize your changes.
No response

https://github.com/ShadowMario/FNF-PsychEngine/assets/49921877/c592894a-62d8-4e13-b373-c99f72513101
heres whats happening by the way

Use timers, you can run one from runHaxeCode or use Lua functions

Use timers, you can run one from runHaxeCode or use Lua functions

Thank you.


Fix #883

Skip overriding variant IDs for externally managed products (printful)
Add a script to fix already broken variant IDs

@franckc Merging this now. Will run the script later today on prod to fix that shop's files. Let me know if you have any other comments


Kayla

SuggestedJourneys version 1
Right now I have it working like a button that when pressed brings you to another page with a predetermined journey. Theoretically, it should work for an arbitrary number of journeys.
I tested this with 2 potential journeys (rt 128 -> back bay && blue hill ave -> south station).
Below is what the button should look like:

Right now I like how the button looks when it is pressed (the animation? I'm not entire sure what to call it), but that can also change if no one else likes it.
There are a few more things I would like to change:

[ ] The layout of the button, it looks pretty basic
[ ] The color depending on what time of day
[ ] size? I'm not exactly sure how big this button is going to turn out. I'm worried that its massive.
[ ] anything else you want me to change!


Accordion variation with CTA- Margin Issue

Hi Team,
I have noticed an issue in the accordion variation with CTA, there is a margin issue. Please look into the below snap and do the needful.

@Kummu007 - Please validate.

Working On it!!

Hi @ssahoo1996
Thanks for the update, the issue is almost corrected but can you just make a padding correction between the expand all radio button and the accordion. Please look into the below snap.

@Kummu007 - Please validate and close.

@ssahoo1996, this issue still persists and Exapand all radio button is missing. Please look into the below snap.
@pardeepgera23 FYI

@Kummu007 - Please validate

Hi Team,
This is working as expected, hence closing the ticket.


Add keywords(apk, pip, pip3) to bashStatement

Add alpine package management tool apk and python's pip pip3

Thanks


Fix #531

The bug is caused as the video's preload attribute is set to metadata while its src is filled with a magnet link.
I've made some modifications which work as the following statement describes.

If type='webtorrent', set video.preload to none
When the torrent id is set, change video.src to ""
Then set video.preload to metadata

For the player, this line causes a wrong error popup notification:
this.tran && this.notice && this.type !== 'webtorrent' & this.notice(this.tran('Video load failed'), -1);

the last & doesn't make any sense in the context, maybe referring to &&, so I make it.

And in fact the callback of the WebTorrent File's renderTo api is only triggered after the first play event. //WTF


Issue 62

fixe #62

Closing due to inactivity. Feel free to reopen, but keep code in separate PRs where possible to make things easier to review.


Validate Azure Container Registry name/resource ID before running container release create

Updated!

After CI passes, I'll merge.


Mismatched coordinate scales

Nice package, and easy to use!
I come across some weird behavior, though, when the x and y coordinates of the grid points have different scales.
I wonder if this could be easily fixed by having a normalization step before constructing the interpolant, which would scale the given points to an e.g. unit box (then, when the interpolant is called, the coordinates used for the new interpolated data can be transformed with the same normalization)?
An example of the current problem below, and let me know if I've missed something obvious in the docs that solves this 😅

When the y indices have a scale that is 50-times smaller than the x indices:
   using NaturalNeighbours
    using CairoMakie
    N = 50
    f = (xy) -> sinc(sqrt((xy[1])^2 + (xy[2] * N)^2) .+ 0.1 * randn())
    x = Float64.(-6:0.1:6)
    y = Float64.((-6:0.1:6) ./ N)
    xy = Iterators.product(x, y)
    z = f.(xy)

    itp = interpolate(first.(xy)[:], last.(xy)[:], z[:])
    _x = range(-6, 6, length=100)
    _y = range(-6 / N, 6 / N, length=100)
    _xy = Iterators.product(_x, _y)
    _z = itp(first.(_xy)[:], last.(_xy)[:])
    Z = reshape(_z, length.([_x, _y])...)

    f = Figure(size=(600, 300))
    ax = Axis(f[1, 1]; title="Input")
    p = heatmap!(ax, x, y, z)
    ax1 = Axis(f[1, 2]; title="Output")
    heatmap!(ax1, _x, _y, Z)
    display(f)

It works fine for N = 1 though (matched scales):

Interesting. I tried this for some other functions and, it of course is fine for e.g. f(x, y) = x^2 + y^2, but I can find it for others like e.g.
Nx = 1
Ny = 50
f = (x, y) -> tanh(x * Nx + y * Ny - (y * Ny)^2)
xg = collect(-6:0.1:6) ./ Nx
yg = collect(-6:0.1:6) ./ Ny
x = [x for x in xg, y in yg] |> vec
y = [y for x in xg, y in yg] |> vec
z = f.(x, y)
method = Hiyoshi(2)
itp = interpolate(x, y, z, method=method)
xvg = collect(-6:0.04:6.0) ./ Nx
yvg = collect(-6:0.04:6.0) ./ Ny
xv = [x for x in xvg, y in yvg] |> vec
yv = [y for x in xvg, y in yvg] |> vec
zv = itp(xv, yv)
f = Figure(size=(600, 300))
ax = Axis(f[1, 1]; title="Input")
p = heatmap!(ax, x, y, z)
ax1 = Axis(f[1, 2]; title="Output")
heatmap!(ax1, xv, yv, zv)
display(f)

I tried looking online for other such issues but can't really find any discussion of it. I suppose your idea would probably work, Here's an idea:
struct ScaledInterpolant{I}
    itp::I
    xmin::Float64
    xmax::Float64
    ymin::Float64
    ymax::Float64
end
function scaled_interpolant(x, y, z; kwargs...)
    xmin, xmax = extrema(x)
    ymin, ymax = extrema(y)
    x̂ = (x .- xmin) ./ (xmax - xmin)
    ŷ = (y .- ymin) ./ (ymax - ymin)
    itp = interpolate(x̂, ŷ, z; kwargs...)
    return ScaledInterpolant(itp, xmin, xmax, ymin, ymax)
end
function (itp::ScaledInterpolant)(x, y; kwargs...)
    x̂ = (x .- itp.xmin) ./ (itp.xmax - itp.xmin)
    ŷ = (y .- itp.ymin) ./ (itp.ymax - itp.ymin)
    return itp.itp(x̂, ŷ; kwargs...)
end

Nx = 1
Ny = 50
f = (x, y) -> sinc(sqrt((x * Nx)^2 + (y * Ny)^2) .+ 0.1 * randn())
xg = collect(-6:0.1:6) ./ Nx
yg = collect(-6:0.1:6) ./ Ny
x = [x for x in xg, y in yg] |> vec
y = [y for x in xg, y in yg] |> vec
z = f.(x, y)
method = Hiyoshi(2)
itp = interpolate(x, y, z, method=method)
xvg = collect(-6:0.04:6.0) ./ Nx
yvg = collect(-6:0.04:6.0) ./ Ny
xv = [x for x in xvg, y in yvg] |> vec
yv = [y for x in xvg, y in yvg] |> vec
zv = itp(xv, yv)
f = Figure(size=(600, 300))
ax = Axis(f[1, 1]; title="Input")
p = heatmap!(ax, x, y, z)
ax1 = Axis(f[1, 2]; title="Output w/o Scaling")
heatmap!(ax1, xv, yv, zv)
ax2 = Axis(f[1, 3]; title="Output w/ Scaling")
itp = scaled_interpolant(x, y, z, method=method)
zv = itp(xv, yv)
heatmap!(ax2, xv, yv, zv)
display(f)

What do you think?
I won't be able to dedicate the time to work on this (in fact, I'll be putting a notice on this package soon that I unfortunately won't have the time to push new content into this package), but if you are interested I would be happy to assist with any PRs / review. Care would have to be taken for how we differentiate this interpolant, of course.

I also note that, I don't want this to be a step that's done automatically within the interpolator since it could cause more problems / won't be obvious to the user. I reckon either having it as a keyword or as its own struct (the former idea, having it as a keyword, would just redirect into a struct like the one I defined above) is more appropriate.

I tried looking online for other such issues but can't really find any discussion of it

My first guess was that this was a triangulation issue, and actually it is (if you don't know, the interpolant is pretty reliant on the quality of the Delaunay triangulation of the points). I found some other discussion about it: https://blogs.mathworks.com/loren/2019/02/04/data-scaling-for-scattered-interpolation/. The normalisation in this blog post might be more appropriate.

Ah, it makes sense that the triangulation isn't normalized, and in general it shouldn't be; it could be desirable for the quality of the meshing could depend on the absolute scale of the in each dimension, or, as in my case, it could not. I guess it's not too much work to normalize the data before interpolating, but that means keeping track of the parameters externally which could be inconvenient when working with the interpolant on the fly.
I'll see if I can put together a small PR from your example above over the weekend. Thanks for your help!

In the end I went with pre-normalizing the data, as below:
using NaturalNeighbours, Normalization, TimeseriesTools
function interpolate(X::TimeseriesTools.AbstractTimeSeries; kwargs...)
    xy = [Float64.(x) for x in lookup(X)]
    N = fit.([MinMax], xy) # Could just be (x-minimum(x))/(maximum(x) - minimum(x))
    points = Iterators.product((xy .|> N)...) |> collect |> vec
    itp = interpolate(Float64.(first.(points)), Float64.(last.(points)), X.data[:];
                      kwargs...)
    return itp, N
end

MinMax normalization worked well, but you could also use any of the others (z-score, sigmoid, etc.) from Normalization.jl
Yeah I think having something in the docs to flag when and how you might need to pre-normalize sounds like a good idea

Oh nice, glad that works. I will add that to the documentation soon, thanks!


Bringing C# into WF

Bringing C# into WF library

Created all equivalent classes for C# to be supported into WF, same as VB
Refactored code that resulted as duplicate into C# into more generic classes
For compatibility reasons, VisualBasicSettings and VisualBasicImportReference do not have a C# equivalent, as agreed with Bogdan Toma
VBValue and VBReference will now be serialized more explicitly into xaml, the same as its C# equivalent always did.

TODO:

[ ] unit tests

WF designer pull request & studio one (in order to test) will come shortly

Changes here are better reflected in #132

Changes here are better reflected in #132


Soundcloud downloads are wav, even if mp3 is specified

For instance:
youtube-dl --audio-format mp3 https://soundcloud.com/mycelium_music/spore-session-vol-13-emog-12-3
[soundcloud] mycelium_music/spore-session-vol-13-emog-12-3: Resolving id
[soundcloud] mycelium_music/spore-session-vol-13-emog-12-3: Downloading info JSON
[soundcloud] 123028713: Downloading track url
[soundcloud] 123028713: Checking download video format URL
[soundcloud] 123028713: Checking http_mp3_128_url video format URL
WARNING: Your copy of avconv is outdated and unable to properly mux separate video and audio files, youtube-dl will download single file media. Update avconv to version 10-0 or newer to fix this.
[download] Destination: Spore Session Vol. 13 - Emog [12-3-2013]-123028713.wav

I'm not sure whether this is because there isn't an mp3 available? If not, rather than downloading the wav, youtube-dl should probably suggest an alternative format. The user probably doesn't want to hammer soundcloud with a giant unecessary wav download!

youtube-dl downloads best available quality by default. You have to explicitly tell it not to download wav if you don't want it.

@dstftw if you look at the command line I used, I actually attempted to do that with the --audio-format flag, which was the first thing I found by searching for "mp3" in the man page.  Why doesn't --audio-format x imply --format x for audio-only downloads?

Because these are different options. --audio-format a postprocessing option that is only used with --extract-audio to specify settings of audio stream to be extracted from downloaded format (wav in this case). --format is an option to specify formats to download.


[AIRFLOW-369] Allow setting default DAG orientation in configuration file

Dear Airflow Maintainers,
Please accept this PR that addresses the following issues:
AIRFLOW-369
No tests infrastructure for webserver exists - verified manually.
Thanks.

@spektom Looks pretty good. A few suggestions:

You've added the new config element in 2 of the 3 customary places in configuration.py. You excluded it from TEST_CONFIG, probably because it it not important to test. However, please add it for completeness.
Follow the items mentioned in https://cwiki.apache.org/confluence/display/AIRFLOW/Contributors'+Guide, such as sticking to a 50 char subject line (you have 73) and squashing your commits into a single one.

Current coverage is 64.54% (diff: 100%)

Merging #1713 into master will increase coverage by <.01%

@@             master      #1713   diff @@
==========================================
  Files           127        127
  Lines          9450       9451     +1
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
+ Hits           6099       6100     +1
  Misses         3351       3351
  Partials          0          0

Powered by Codecov. Last update 9aa7da5...2c96d93

@spektom Looks pretty good, but please address the code quality issues.
Specifically, it looks like one of your format lines has extraneous nesting -- e.g. format((...)) when it should be format(..)

@r39132 Can you point me to a line? I don't see any format() lines that I've added in this PR.

Landscape seems out of its mind on this one.
+1 merging
-s


Question: Create build script to make installation easier?

Currently, users have to follow 6 steps to install and run 2048. Though these are small and can be done quickly, it would be much more convenient if we have a bash script that does this for us.
The bash script will do the following:

Clone the repository in the home folder.
mkdir build, cd build to create the build directory.
cmake .. and make to create the final, executable program.

These are the 3 main steps that the script will follow. After this, we should add the 2048 program to the PATH so that users can access the game from anywhere in their terminal.
Either adding to the PATH permanently or adding an alias to the game in their .bashrc or .zshrc files.
With the bash script we can also now handle OS specific actions, as adding to the PATH will be quite different in all the three major OSs.

Humm, normally I only "run" programs listed in the releases section of a github project.

One possibility is a statically-compiled program and then release it via the release section.
Another possibility (which is a bit more verifiable) is using openSUSE Build Service to make the release for you [1].

One possibility is a statically-compiled program and then release it via the release section.

Wouldn't there have to be a different compiled binary for each OS, platform and system architecture? If yes, then it's a bit too much of work IMO


Develop - Merge to master

@kiritigowda This seems to be a rhel-alone issue for all HIP.
Tests are passing on ubuntu and sles.


updated deploymate (1.3.0)

Closes #17605.

Technically, PRs from the script should be made from the user (your) fork, rather than caskroom/homebrew-cask.

I’m still testing it out in the real world (made it today), but yes, there are a few reasons to do it this way.
There are two scripts: find_outdated_appcasts and fix_outdated_appcasts. This PR is a result of the latter (uses cask-repair for the updates, but grabs all relevant issues from their label). For it to be easily transferable to other maintainers, it needs to be done this way for now, otherwise there will be issues regarding names of remotes, as well as problems when a maintainer doesn’t have all official repos tapped.
It’s not out of the question to fix those limitations, but they require a lot more work, and the priority was to see if the scripts could work. So far, working wonderfully.

Merged as f094a01e7ada49b7ff925922dc82550ae97e4930.


Security Requirements Object Contradiction

In the Swagger Object descriptions and the Operation Descriptions, the field security states that "The list of values describes alternative security schemes that can be used (that is, there is a logical OR between the security requirements)."
The Security Requirements Object however, states that "Lists the required security schemes to execute this operation. The object can have multiple security schemes declared in it which are all required (that is, there is a logical AND between the schemes)."
where security requirements object is:
{name}: [ ]
by the sound of it, I think it would work like:
primary: ["api_key"]
two_factor: ["password", "pin"]
where "api_key", "password", and "pin" reference security schemes.
However, the examples and descriptions go on to say that name should map to the security scheme and that the array should be used for oauth scopes only, everyone else should have an empty array.
So this seems a bit contradictory to me.  Given the above, I don't understand how one could support:

a security requirement that allows "api_key" OR "password"
a security requirement that supports "password" AND "pin"

Additionally (but somewhat unrelated), I'm not sure I understand why security types are fixed to "basic", "api key" and "oauth".   This seems needlessly limiting (especially since other fields ^x- can be added to support additional configurations of other mechanisms).  How would one suggest something like "hmac" with a nonce?   where additional parameters and algorithms need to be communicated.  I suppose this could be wedged into api key with optional parameters for the nonce/etc. on all operations/paths.  But that doesn't seem appropriate.    Likewise, if we wanted to limit certain security types (like api_key and oauth) to https calls but allow something like "hmac" for unsecure channels (like passing an authorized secure/one-time-user/expiring link through an intermediary) to be less restrictive, it doesn't seem possible.
Just starting to wrap my head around it so apologies if I missed something.  Thanks in advance.

i may have answered one of my own questions.  I see now that the swagger/operation security field is an array of security requirement objects, and that the security requirement object may have more than one key.  so to answer my question directly:
the OR case:
security: [ { "api_key":[ ] }, {"password": [ ]} ]
the AND case
security: [ { "pin":[ ] , "password: []" } ]
and combo ((password & pin) OR oauth):
security: [ { "pin":[ ] , "password: []" }, {"oauth": ["read", "write"]} ]
Is my understanding now correct?
I'm still interested in hearing the rationale behind limiting the security types to basic/api_key/oauth.  (were API key is a little misleading, it's more analogous to passing a credential/identifier in a "specified header").

@mikestanley - Hi Mike, thanks for taking the time to write this up.
Regarding the AND/OR case, you got it right in your second comment, so I hope that's OK.
As for the security types - I believe this is more about tooling and the ability to automate support for security types.
Before I get into that I'll say that the name api_key may be a bit misleading and it's basically any key/value combination that tends to appear either as a query parameter or a header (granted, cookies may be applicable as well). You can argue that basicis a specific case here, which is true, but the usage is probably common enough (not getting into whether that's good or not) to simplify documentation.
The specific types allow the supporting tools to automate the support for the security. If you look at swagger-ui, it has internal support for (most) OAuth2 flows, and we plan on integrating better support for basic/api_key and the possible combinations of them. The same goes for generated code using the code generator and I assume (and hope) that other tools out there take advantage of it as well.
There's a fine balance between extensibility and support for anything in such a spec, and security is not different in that sense. We could allow extension points in the various tools so that people could add support for all kinds of variances, but I think that Swagger 2.0 was mostly based on feedback gathered over the years since the initial spec was released and as part of the process it's not always easy to decide what to include and what not.
That said, we're always looking into the future trying to see what and where we can improve. Since this issue started as a collection of questions (which can most certainly continue to discuss), I'd invite you to open a 'clean' issue regarding the security definition restrictions and we could mark that as Swagger.Next for discussion.

@webron  thanks for the response.
I understand the benefits to tooling the restricted security options provide.  After I understood the AND/OR case, most (if not all) my current use cases are satisfied without issue, so the current options do strike a nice balance.
I do think it's worth time considering, at the very least, how an hmac like security protocol could be achieved (with support for expiration and nonce parameters, as well as specification of a crypto algorithm).  this could be something that can be generalized and supported by tools as well.   It may be an edge case though so not sure it's worth the carry cost (making the case for more open extensions mechanism).
I will mark this ticket as resolved and revisit opening a ticket about the restrictions on the security definitions.   I can happily live with those restrictions though for now.

@mikestanley - Glad it's clearer now. Please make sure to open a ticket once you're ready. This is how the spec moves forward.


aws sdk v3

Updates aws sdk to v3 to work with other fluent plugins that have moved to aws sdk v3.
Resolves #77

Is there an updated container image that includes this update.  I'm working from quay.io/coreos/fluentd-kubernetes:v0.12-cloudwatch.


Clarification on the density scaling

Hi!
After having a deeper look into the code while debugging problems related with #346,
I came across the HU->density transformation.
Since I do not have a strong medical background, it is a bit hard for me to make sense of the code here. I would have expected a simple linear transformation to get into the 0,1 range (potentially with special consideration regarding the -1024 fill values).
If you don't mind clarifying:

Why would you set the data in the "soft tissue range" to the lowest observed value? I guess it increases the bone contrast a bit, but that is not how a real x-ray would work, right? If the improved contrast is the goal that's fine, but I would have assumed you want to keep it as close to realistic as possible?
The .min() and .max() scalings do seem to be suboptimal in general. Isn't one huge advantage of the hounsfield scale that you can define reasonable ranges without looking at the particular data?

Granted, the second point is not really an issue with full CT volumes, BUT I ran into somewhat weird behaviour using (again) masks: The ranges are calculated on the full volume and only later the mask is applied. If, on the other hand, I use a masked volume directly (meaning my volume data only contains the region of interest), the scaling might be wildly different.
Maybe that is a super niche edge-case as the real CT volumes observed in the wild will not be "pre-segmented", but I still don't think it is intentional.

I think there's a slight clarification for point 1: data in the air range is set to the min of soft_tissue. By default (i.e., bone_attenuation_multiplier = 1.0), soft_tissue and bone still obey the same linear relationship. Increasing bone_attenuation_multiplier just serves to make bones brighter, which while not exactly how X-ray works, provides a useful data augmentation mechanism for when training neural networks (see here). Please let me know if I misunderstood your question.
For point 2, you're right that that minmax scaling is a hacky way of implementing this conversion. There are some more principled ways of doing this (e.g., see #200), but haven't had time to change them. But ofc, happy to accept PRs to change this behavior!

Apologies, you are of course right: It is the air range, that is set to the minimum of the soft tissue range. The way you do it, air and the low end of the soft tissue range both end up at 0.
That will probably affect the visual output way less than I thought (if at all)...
I was just worried about the resulting nonlinearities in the data scaling, but that mostly originated from my misunderstanding.
I am fine with minmax scaling, but I argue that the range should not be calculated on the specific sample (at least once masks are introduced). Instead I would personally hardcode the range (as the CT values are always on the Hounsfield scale) to make sure the scaling does not depend on the input.
That way different CT files with and without masks would be scaled in the same way and one can more easily compare the image values of different x-ray images.
That is debatable though, there are advantages to both approaches.

If I'm not mistaken, I think a given CT scan with and without a mask is scaled by the same way.
Before masking the volume, the density is computed using the global min/max of the entire CT:
https://github.com/eigenvivek/DiffDRR/blob/6a486b64dde2cec0ab6e88068d62b41e06f30cf0/diffdrr/data.py#L80-L82
Only after this is the mask applied:
https://github.com/eigenvivek/DiffDRR/blob/6a486b64dde2cec0ab6e88068d62b41e06f30cf0/diffdrr/data.py#L134-L141

You are right again (and I need to be precise with what I write...).
Using the labelmap argument to read a mask alongside the full volume, everything is fine.
My issue arises only if you do not use a full CT + mask, but instead create a new file, that only contains the masked region or whatever your model predicts as the bone of interest and everything else is zero / -1024.
In that case, the values might not reflect what is inside a full CT. For example, there might be almost no soft tissue present.
Like I said: Might be niche use and I am totally fine with closing this after it turned out I simply got confused about Point 1.


Combine all images into one

[x] Combination of images code
[x] Optimization of filesize/CPU usage
[x] Serverless alternative to combine endpoint(?)

Live at c.vxtwitter.com; Still might need optimizations to lower processing time


Content in non expanded sections

Hey, I've found that when a topic h1 is within a section which is not expanded, the toc does not show the topic. I guess it's because the h1 mark is not visible.
But could it be possible to take the section name as a topic? It's aswell in a marked div and should be detectable. What do you think?
Kind regards,
Constantin

In the code a section looks like this, and the Name of the section is also classified as an h2 element. so it should be possible to detect it.

I guess the problem is that currently the queryselector is only looking for classes with the cke_editable tag, which a section does not have. I've not fully understood it, but couldn't we simple add the collapsible class to include those ?

Hi Constantin. As mentioned in the readme file, I modified this code from another author, and I expect collapsible sections didn’t exist when it was originally written (to be honest, I didn’t even know they existed until you mentioned them!) From what you have posted, it looks like it should be a simple fix. I don’t have time to look at it right now as I’m super busy, but I’ll try and look at it over the holiday period (if I remember). Thank you for bringing it to my attention.

Hi @RedEchidnaUK ,
i've figured it out :-) just add a:
var textBoxClassSection = '.Collapsible';
and an additional line unter the equivalent:
queryParts.push(textBoxClassSection + " " + TableofContents.h2Tag);
queryParts.push(textBoxClassSection + " " + TableofContents.h3Tag);
queryParts.push(textBoxClassSection + " " + TableofContents.h4Tag);
and now the sections will be aswell inventoried as (per default) h2Tag :-)

Hi, that’s great. That’s exactly what I thought I needed adding. I’ll make the change as soon as I have time and get it uploaded. Thanks for your help and support.

Sounds nice. I'll send you a pull request. Added aswell a german language file ;-)
What i aswell saw is that the ToC background is fix set to #FFFFFF so for example if you have a section wich has a blue background the ToC will have a seperated background in white, which does not look good.
I've not yet found where it is specified, may you can have a look at this :-)
The Theme Colors could be used like this, if I am right: learn.microsoft.com/en-us/sharepoint/dev/spfx/use-theme-colors-in-your-customizations
Thanks so far, and have a nice weekend


[Question]: The Tensors class should not implement an implicit cast to Tensor

Description
The Tensors class implements an implicit cast to Tensor. It simply returns the first Tensor in its list. There is a potential risk of information loss in this case. While casting from float to double does not pose any problems, casting from double to float requires an explicit conversion.
Alternatives
I looked into the impact of an explicit cast. I made the necessary corrections to pass the tests. However, I had to modify the unit tests to ensure proper casting of Tensors. It doesn't feel clean to do that...
I went a bit further and realized that almost all implementations of the layers use a Tensor as input and output. I got overwhelmed by the task of revising everything quickly. Then, I reached the RNN section, which seems to be under development. At that point, I couldn't make the necessary corrections or add a comment to guide the changes. I gave up on completing the task when I realized the extent of the impact. I was in the process of making a modification that would affect everything derived from Loss or Layer, and with generics (to avoid having to declare typed arguments and hiding inherited members, etc.).

@Oceania2018 any ideas about this?

There is nothing insurmountable, I believe. Classes working with a tensor and returning a tensor should have a common abstract class inheriting from Layer. It would virtually and naively implement the Call method for each tensor input and associate it with its output.
There are few layers that really need to implement tensors as inputs. They could keep their own implementations.
My main concern is that the modification will still change a lot of things in the keras.layer directory of the TensorFlow.Keras project.
I'm not an expert in TensorFlow or Keras, so forgive me if I'm mistaken.

Thank you a lot for telling us this concern. I agree that the implicit cast from Tensors to Tensor is risky. To me this proposal hits a key problem and sounds very reasonable. I approve removing this implicit conversion in the future. Here're some of my thoughts of the design of Tensors and welcome to discuss it together.

The Apply of the layer, which is corresponding to __call__ in python, should keep using Tensors because it will make APIs consistent. Otherwise users may need to figure out whether the Apply with Tensors or the one with Tensor should be used for some layers.
Originally I wanted to delay this change to v1.0.0 to avoid break change in minor version. However when developing RNN recently, I found it better to change some behaviors of Tensors. Therefore I wonder if making this modification in v0.100.6 will impact much on users.
Also when developing RNN, I found sometimes the output of a layer is nested tensors, for example, [t1, [t2, t3]], and it's expected to be used as var (value, states) = tensors. This pushed me to re-consider the design of Tensors. Supporting nested structure in Tensors may introduce complexities to users, while letting users deconstruct a Tensors themselves may also be annoying. If the first one is taken as the solution finally, it's a good time to modify it together with the implicit conversion.

I fully agree your first point. From user side, everything should work with tensors as input or ouput. Tensorflow.Net need to take it into account internally, from Tensorflow.Net developpers pov.
For the second point, most impact should be on contributor. If some one activelly work on an impacted surface by the change, delaying seem to be a good.
For the 3 point I also see some code with strange using of Tensors. But as I said, without Keras and tensor flow expertise I can see if it's functional or "experimental".
You're welcom to discuss. I think I need to meet some one on discord or other.
Verry, I just want to have Model that support Tensors as y. And I need to understand a lot of thing within the code to do this change if no one do it ^^'.

Thank you for your suggestions. I'm working on the refactoring of Tensors now together with RNN. I'll let you review the code after I complete it.

Verry, I just want to have Model that support Tensors as y.

Sorry that I didn't get that point. Could you please give me an example?

I have a model that have one Tensor input, but two tensor output. Something like:
            var inputs = keras.Input(7, dtype: TF_DataType.TF_FLOAT);             var outputs_pred = tf.keras.layers.Dense(3, activation: "relu").Apply(inputs);             var outputs_pred2 = tf.keras.layers.Dense(3, activation: "relu").Apply(inputs);             var model = keras.Model(inputs, (outputs_pred, outputs_pred2));
model.summary() and model.predict() seem to work as expected. But I've find no way to use model.evaluate() nor model.fit().
Because I don't know how to do, I read the code and try to have this to be fixed :p. And when reading I see a lot a litle thing that I want refactor: Class architecture, adding some generic, using of static reaonly string instead of constant or literal, add a string enumeration class to have "typed string" (on the back end at last), remove empty array creation, and so on and so forth ^^'...

Yes, currently the solution for multi-outputs is outputting an Tensors or Tensor[]. I agree sometimes that may not be enough.
tensorflow.net is far from perfect and is progressing. Since v1.0.0 could include break changes, it'll be okay to refactor some parts. We'll be sincerely glad If you'd like to improve it with us together. If you would like so, what about inviting you to our discord dev channel? (though it's newly opened) :)


"Go to Test" to go to selected parameters in parametrized tests.

This is blocked on pytest and unittest providing us the line numbers for the parameters.

Thanks for the feature request! We are going to give the community 60 days from when this issue was created to provide 7 👍 upvotes on the opening comment to gauge general interest in this idea. If there's enough upvotes then we will consider this feature request in our future planning. If there's unfortunately not enough upvotes then we will close this issue.

Thank you for submitting your feature request and everyone who considered it! Unfortunately, this issue did not receive enough votes over the allotted time, and so we are closing the issue.


IndexError when calling emg3d.construct_mesh

Hi all,
I just start to learn emg3d following the example gallery.
I got an index error when I run the script in the tutorial.
Here's the code:
import emg3d
import numpy as np
import matplotlib.pyplot as plt
plt.style.use('ggplot')

print(emg3d.Report())
###############################################################################
# Survey
# ------

src = [0, 0, -950, 0, 0]    # x-dir. source at the origin, 50 m above seafloor
off = np.arange(5, 81)*100  # Offsets
rec = [off, off*0, -1000]   # In-line receivers on the seafloor
res = [1e10, 0.3, 1]        # 1D resistivities (Ohm.m): [air, water, backgr.]
freq = 1.0                  # Frequency (Hz)

###############################################################################
# Mesh
# ----
#
# We create quite a coarse grid (100 m minimum cell width), to have reasonable
# fast computation times.
grid = emg3d.construct_mesh(
    frequency=freq,
    min_width_limits=100.0,
    properties=[res[1], 100., 2, 100],
    center=(src[0], src[1], -1000),
    seasurface=0.0,
    domain=([-100, 8100], [-500, 500], [-2500, 0]),
    verb=2,
)
print(grid)

The output information:

--------------------------------------------------------------------------------
  Date: Wed Nov 18 14:57:21 2020 China Standard Time

                OS : Windows
            CPU(s) : 8
           Machine : AMD64
      Architecture : 64bit
               RAM : 63.9 GB
       Environment : Python

  Python 3.7.9 (default, Aug 31 2020, 17:10:11) [MSC v.1916 64 bit (AMD64)]

             numpy : 1.19.2
             scipy : 1.5.4
             numba : 0.51.2
             emg3d : 0.14.1
           empymod : 2.0.3
            xarray : 0.16.1
        discretize : 0.5.1
              h5py : 2.10.0
        matplotlib : 3.3.2
           IPython : 7.13.0

  Intel(R) Math Kernel Library Version 2019.0.0 Product Build 20180829 for
  Intel(R) 64 architecture applications
--------------------------------------------------------------------------------
         == GRIDDING IN X ==
Skin depth          [m] : 276 / 5033 / 5033  [corresponding to `properties`]
Survey domain DS    [m] : -100 - 8100
Comp. domain DC     [m] : -31723 - 39723
Final extent        [m] : -33701 - 41701
Cell widths         [m] : 100 / 100 / 5465  [min(DS) / max(DS) / max(DC)]
Number of cells         : 128 (82 / 46 / 0)  [Total (DS/DC/remain)]
Max stretching          : 1.000 (1.000) / 1.190  [DS (seasurface) / DC]

         == GRIDDING IN Y ==
Skin depth          [m] : 276 / 5033 / 5033  [corresponding to `properties`]
Survey domain DS    [m] : -500 - 500
Comp. domain DC     [m] : -32123 - 32123
Final extent        [m] : -34890 - 34890
Cell widths         [m] : 100 / 100 / 9016  [min(DS) / max(DS) / max(DC)]
Number of cells         : 40 (10 / 30 / 0)  [Total (DS/DC/remain)]
Max stretching          : 1.000 (1.000) / 1.350  [DS (seasurface) / DC]

         == GRIDDING IN Z ==
Traceback (most recent call last):
  File "c:/Users/JingXu/Desktop/code/start_emg3d/model_property_mapping/mapping.py", line 30, in <module>
    verb=2,
  File "D:\Anaconda3\envs\tdem\lib\site-packages\emg3d\meshes.py", line 462, in construct_mesh
    z0, hz = get_origin_widths(**kwargs, **zparams)
  File "D:\Anaconda3\envs\tdem\lib\site-packages\emg3d\meshes.py", line 626, in get_origin_widths
    thxr = hx[-1]*sa**np.arange(nx)
IndexError: index -1 is out of bounds for axis 0 with size 0

Thanks for reporting @jingxu94 . The entire construct_mesh()-functionality is very new, so unfortunately it is possible that there are bugs, even though I tried to avoid them ;-)
As I explained in our earlier discussion, I can currently not reproduce the problem, so I cannot do debugging myself. I will therefore ask some questions, it would be great if you could help me with that. Hopefully we will get to the bottom of it.
Could you try to run the following and post the output?
import emg3d
z0, hz = emg3d.meshes.get_origin_widths(
    frequency=1.0,
    min_width_limits=100.0,
    properties=[0.3, 2, 100],
    center=-1000,
    seasurface=0.0,
    domain=[-2500, 0],
    verb=2,
)

Just to ensure if the issue is in construct_mesh or in get_origin_widths.

Similar error as follows:
Traceback (most recent call last):
  File "c:/Users/JingXu/Desktop/code/start_emg3d/model_property_mapping/test_issue.py", line 9, in <module>
    verb=2,
  File "D:\Anaconda3\envs\tdem\lib\site-packages\emg3d\meshes.py", line 626, in get_origin_widths
    thxr = hx[-1]*sa**np.arange(nx)
IndexError: index -1 is out of bounds for axis 0 with size 0

That is really strange, I get the following without any issues:
Skin depth          [m] : 276 / 712 / 5033  [corresponding to `properties`]
Survey domain DS    [m] : -2500 - 0
Comp. domain DC     [m] : -6974 - 31623
Final extent        [m] : -8729 - 32628
Cell widths         [m] : 100 / 100 / 9085  [min(DS) / max(DS) / max(DC)]
Number of cells         : 48 (25 / 23 / 0)  [Total (DS/DC/remain)]
Max stretching          : 1.000 (1.000) / 1.380  [DS (seasurface) / DC]

I even recreated an environment fixing all the package versions to match yours, but I do not encounter the problem. I also tried it with pip and with conda, both work fine. The only difference that remains between your system and mine is Windows/Linux.
I will try and get hold of a Windows machine to test it.

OK, I found the problem, it is a Windows issue. Int's default to 64 on Linux/Mac, but on Windows they default to 32. This caused an overflow problem in emg3d.meshes.good_mg_cell_nr.
If you change line 780 (emg3d v0.14.1) from
    primes = np.array([2, 3, 5, 7, 11, 13, 17, 19])

to
    primes = np.array([2, 3, 5, 7, 11, 13, 17, 19], dtype=np.int64)

the problem will be fixed.
I will push an update shortly.
Thanks for reporting!

Thinks, it works. That's a helpful tip for programming!

(With #144 I will hopefully start to also test on Windows and Mac machines, which should avoid these sort of issues in the future, hopefully.)


FIX repr_html of Info add all channel types

Fixes #9707

I've merged with main branch to attempt to fix CIs @vagechirkov

Azure failure is an unrelated / spurious caching error after successful tests. @vagechirkov can you add a latest.inc entry for this fix? Then I'll merge!

Pushed a commit to move the change to below those of new contributors, then will merge once CIs come back happy. Thanks in advance @vagechirkov !

thx @vagechirkov and @larsoner !


Include persistent damage in Phantom Pain spell damage

The issue with this being part of the spell damage is the success state doesn't take the persistent damage at all, but does take the initial damage.


Increase max_semi_space

See, https://github.com/nodejs/node/issues/42511.

Was discussed on TSC meeting and we would love to have a PR to discuss.

I'll like to take a look at this. Does anybody have any extra knowledge on this, in case some help is needed?

sure, I actually did run some tests, seems like 64 mb is the sweet spot actually
for 128mb
debadreechatterjee@Debadree-TagMango-MacBook-Pro web-tooling-benchmark % /Users/debadreechatterjee/Documents/Personal/node/node-perf dist/cli.js
Running Web Tooling Benchmark v0.5.3…
-------------------------------------
         acorn: 14.11 runs/s
         babel: 10.97 runs/s
  babel-minify: 16.76 runs/s
       babylon: 20.19 runs/s
         buble:  6.60 runs/s
          chai: 18.82 runs/s
  coffeescript: 10.61 runs/s
        espree:  5.38 runs/s
       esprima: 16.88 runs/s
        jshint: 15.18 runs/s
         lebab: 15.31 runs/s
       postcss:  9.26 runs/s
       prepack:  9.91 runs/s
      prettier: 10.38 runs/s
    source-map: 10.87 runs/s
        terser: 29.18 runs/s
    typescript: 15.18 runs/s
     uglify-js:  7.73 runs/s
-------------------------------------
Geometric mean: 12.45 runs/s

and for 64mb
debadreechatterjee@Debadree-TagMango-MacBook-Pro web-tooling-benchmark % /Users/debadreechatterjee/Documents/Personal/node/node-perf --max-semi-space-size=64 dist/cli.js
Running Web Tooling Benchmark v0.5.3…
-------------------------------------
         acorn: 16.04 runs/s
         babel: 11.32 runs/s
  babel-minify: 16.83 runs/s
       babylon: 18.43 runs/s
         buble:  7.23 runs/s
          chai: 19.44 runs/s
  coffeescript: 11.17 runs/s
        espree:  5.46 runs/s
       esprima: 19.62 runs/s
        jshint: 17.77 runs/s
         lebab: 16.95 runs/s
       postcss:  8.80 runs/s
       prepack: 11.63 runs/s
      prettier: 10.41 runs/s
    source-map: 11.36 runs/s
        terser: 29.36 runs/s
    typescript: 14.97 runs/s
     uglify-js:  8.74 runs/s
-------------------------------------
Geometric mean: 13.12 runs/s

Just needed to know if that was the right place to put this change since am new to the cpp part of the codebase 😅

I'm not sure that is safe to do:
I was trying to use --max-semi-space-size=64 for one of the services. After ~ week it just blowup during high traffic:
Mem
P1:

yellow requested "new memory"
green used "new memory"

P2:

orange requested "new memory"
blue used "new memory"

As we can see, at some point v8 started requesting 128MiB (rather than 64MiB) of memory for the "new" stuff, while not being able to handle and use it.
Processes started to use multiple cpus/cores (3 cores at 100% each):

yellow P1
green P1

The funny fact is that these services were still able to handle the traffic without any issues.

I think presently the flag --max-semi-space-size=64 just directly sets semi-space to 64mb I think from the basic stuff I understood from reading that part of v8 code we need to change the partition between young gen and old gen space (and semi-space as a proportion of young space, directly setting it causes OOM error, in the PR also https://github.com/nodejs/node/pull/47277 the CI failures are all OOM related tests

The node docs already explicitly mention setting max-old-space-size, and this is essential for efficient use of memory in single-process server applications. The reason this setting isn't baked in is because of the qualifications, such as single-process, and because YMMV.
IRC, max-semi-space-size will be derived from max-old-space-size.
If anything is done wrt max-semi-space-size, I suggest first adding a recommendation to the docs to accompany (and complement) the max-old-space-size recommendation.

In the version of v8 (11.3) used in Node 20, the default max-semi-space-size is no longer a constant, and instead scales with the memory limit.
In addition to the documentation now being incorrect:
https://github.com/nodejs/node/issues/55487
This can also have an undesirable performance impact when upgrading from Node 18 to Node 20:
https://blog.ztec.fr/en/2024/post/node.js-20-upgrade-journey-though-unexpected-heap-issues-with-kubernetes/


feat!: replace lazy_static with once_cell

Extended commitment generators are currently produced using the lazy_static macro. Unfortunately, lazy_static is no longer actively maintained. Other repositories in the ecosystem are moving to once_cell, which is actively maintained.
This PR migrates from lazy_static to once_cell. In the process, it cleans up the construction. Specifically, it moves from using word-based numbering (e.g. ONE, TWO) in hash inputs to numerals, which simplifies the design. It also adds a useful test that asserts the number of generators matches the size of ExtensionDegree to avoid panics.
Closes #67.
BREAKING CHANGE: Modifies the construction of commitment generators.

There are still some assumptions that this code makes about ExtensionDegree that I don't particularly like:

The enum will always contain only incrementing numerical values
The MINIMUM and MAXIMUM constants will always refer to the lowest and highest valid enum values, respectively
The TOTAL constant (defined in terms of the other constants) will always represent the total number of valid enum values

However, these assumptions are carefully documented, should be easy to check by inspection, are reasonably tested, and I don't see a good way to remove them without overhauling the ExtensionDegree design.


Removed RenderProxyBoxMixin class

This issue is fixed https://github.com/dart-lang/sdk/issues/31543
Pre-launch Checklist

[x] I read the Contributor Guide and followed the process outlined there for submitting PRs.
[x] I read the Tree Hygiene wiki page, which explains my responsibilities.
[x] I read and followed the Flutter Style Guide, including Features we expect every widget to implement.
[x] I signed the CLA.
[x] I listed at least one issue that this PR fixes in the description above.
[x] I updated/added relevant documentation (doc comments with ///).
[ ] I added new tests to check the change I am making or feature I am adding, or Hixie said the PR is test-exempt.
[ ] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel on Discord.

Hello @Hixie
can you please review this PR?
Thanks

Gold has detected about 7 untriaged digest(s) on patchset 1.
View them at https://flutter-gold.skia.org/cl/github/78774

Gold has detected about 5 untriaged digest(s) on patchset 2.
View them at https://flutter-gold.skia.org/cl/github/78774

test-exempt because this is a code refactor that should have no effect on semantics.

@Abhishek01039 Looks like this is causing many existing tests to fail. Can you take a look at those?

Gold has detected about 4 untriaged digest(s) on patchset 3.
View them at https://flutter-gold.skia.org/cl/github/78774

@goderbauer The issue of CI is not related to this PR.

At least the analyzer failures do look related to this PR: https://cirrus-ci.com/task/4668905705701376

@Abhishek01039 Do you still have plans to come back to this?

Hello @goderbauer
I am still not able to understand what is the exact error is!
can you please guide me on this?
Thanks

Gold has detected about 9 untriaged digest(s) on patchset 5.
View them at https://flutter-gold.skia.org/cl/github/78774

Gold has detected about 14 untriaged digest(s) on patchset 6.
View them at https://flutter-gold.skia.org/cl/github/78774


docs: update to the used by section of the readme

Please check if the PR fulfills these requirements

[ ] It's submitted to right branch according to our branching model
[x] It's right issue type on title
[ ] When resolving a specific issue, it's referenced in the PR's title (e.g. fix #xxx[,#xxx], where "xxx" is the issue number)
[ ] The commit message follows our guidelines
[ ] Tests for the changes have been added (for bug fixes/features)
[ ] Docs have been added/updated (for bug fixes/features)
[ ] It does not introduce a breaking change or has description for the breaking change

Description
README의 used by 섹션에 회사명을 추가합니다

Thank you for your contribution to TOAST UI product. 🎉 😘 ✨

Can one of the admins verify this patch?

We will wait for other TOAST UI Applications result. So, please wait. Thank you.


Check isNaN on alMax & use today as birthdate max

Finish the code done in https://github.com/SuffolkLITLab/docassemble-ALToolbox/pull/245.
@plocket, is this the direction we want to take? I tried a version using if/elses like you suggested in https://github.com/SuffolkLITLab/docassemble-ALToolbox/pull/245#discussion_r1453649722, but there are more cases than you mentioned there. It was much harder to read IMO, so I didn't continue that.
This does treat invalid alMax / alMin dates as nothing instead of as a NaN Date, which will always be false when compared to other dates with <= and >=. I think means user's wouldn't be able to submit? (part of the testing I need to still do). It's a behavior change, but probably the safest one?
Tested with:

[x] no max date
[x] invalid max date
[x] valid max date
[x] no min date
[x] invalid min date
[x] valid min date
[x] no max birthdate
[x] invalid max birthdate
[x] valid max birthdate (before and after today)

Testing is done! Made a few last fixes for issues I found, but it's ready for review / merging

@plocket for you to review when you have a chance i think! You know this code much better


Add torch and cuda args to dockerfile

Fixed default versions gives problems, acutal dockerfile crushes since it install a not compatible mmcv version:
AssertionError: MMCV==1.1.5 is used but incompatible. Please install mmcv>=1.2.4, <=1.3.

This might not completely fix the problem, since docker CUDA version depends on tensor rt base image (see https://docs.nvidia.com/deeplearning/tensorrt/container-release-notes/index.html).

Thanks for the contribution!
I did want to update the base image to a new version. But some conversions failed when I am trying to update tensorrt:20.09-py3. I will try the latest one this weekend.

This worked for me with CUDA 11.2, just tested and run the output .engine with latest deepstream version.
Thanks to you for this amazing work!

cub has been include in cuda11, which is conflict with the cub in amirstan_plugin. Two stage model conversion failed in  tensorrt:20.09-py3(cuda11). This PR will be merged after I fix this.

So what is the fix? Except 11.2, does it owrk for any version (from 10.0 to 11.1)? I get the same error AssertionError: MMCV==1.1.5 is used but incompatible. Please install mmcv>=1.2.4, <=1.3.

You can use this branch and set cuda and pytorch versions in docker build, it should work (it worked for me).

You can use this branch and set cuda and pytorch versions in docker build, it should work (it worked for me).

I followed your suggested changes yet I'm stick with some errors. First, while building docker, mmpycocotools failed to build and also mmlvis. But the build still completed and upon running the command: sudo docker run --gpus all -it --rm -v ${your_data_path}:${bind_path} mmdet2trt_docker:v1.0 ${bind_path}/config.py ${bind_path}/checkpoint.pth ${bind_path}/output.trt, it gave me an error: pkg_resources.DistributionNotFound: The 'mmdet2trt' distribution was not found and is required by the application
Any ideas?

You can use this branch and set cuda and pytorch versions in docker build, it should work (it worked for me).

I followed your suggested changes yet I'm stuck with some errors. First, while building docker, mmpycocotools failed to build and also mmlvis. But the build still completed and upon running the command: sudo docker run --gpus all -it --rm -v ${your_data_path}:${bind_path} mmdet2trt_docker:v1.0 ${bind_path}/config.py ${bind_path}/checkpoint.pth ${bind_path}/output.trt, it gave me an error: pkg_resources.DistributionNotFound: The 'mmdet2trt' distribution was not found and is required by the application
Any ideas?

It seems there was some problem installing mmdet2trt... I cannot tell you much from that information.  Check docker build output after the step:
### install mmdetection-to-tensorrt
RUN cd /root/space/mmdetection-to-tensorrt &&\
    python3 setup.py develop

Or get into docker container and try to figure out if mmdet2trt was properly installed.

Any news about this?

Sorry for the late reply
As MMCV only provides packages with cuda<=11.1, could you downgrade the base image and update the default config?


revert changes to click decorators after click/mypy conflicts resolved

While updating dependencies, PR https://github.com/MITLibraries/transmogrifier/pull/91 also reworked the order of click decorators to address a new, external conflict between click==8.1.3 and mypy==1.4.1.
This underlying conflict was discovered after the code change, and it's assumed a fix will be rolled out to click in the near future that will allow the previous decorator order while also aligning with mypy linting, as many others have encountered this as well.
Proposing to revert the click decorators to match other projects (with @click.command as the top-most decorator, with others following) once it's been determined that click and mypy are aligned.
Current:
@click.option(
    "-i",
    "--input-file",
    required=True,
    help="Filepath for harvested input records to transform",
)
@click.option(
    "-o",
    "--output-file",
    required=True,
    help="Filepath to write output TIMDEX JSON records to",
)
@click.option(
    "-s",
    "--source",
    required=True,
    type=click.Choice(list(SOURCES.keys()), case_sensitive=False),
    help="Source records were harvested from, must choose from list of options",
)
@click.option(
    "-v", "--verbose", is_flag=True, help="Pass to log at debug level instead of info"
)
@click.command()
def main(source, input_file, output_file, verbose):
...

Proposed reverted ordering, matching convention in other projects:
@click.command()
@click.option(
    "-i",
    "--input-file",
    required=True,
    help="Filepath for harvested input records to transform",
)
@click.option(
    "-o",
    "--output-file",
    required=True,
    help="Filepath to write output TIMDEX JSON records to",
)
@click.option(
    "-s",
    "--source",
    required=True,
    type=click.Choice(list(SOURCES.keys()), case_sensitive=False),
    help="Source records were harvested from, must choose from list of options",
)
@click.option(
    "-v", "--verbose", is_flag=True, help="Pass to log at debug level instead of info"
)
def main(source, input_file, output_file, verbose):
...

https://github.com/MITLibraries/alma-sapinvoices is also affected by this issue and the click version will need to be unpinned when the incompatibility is resolved


Config Flow could not be loaded TrueNAS

Describe the bug
On adding the integration in Config->Integrations->Add Integration it loads for about 5 minutes and then errors out in "Config Flow could not be loaded"

I am running Home Assistant Core on TrueNAS (That's FreeBSD as host OS) 12.0-U4 with Home Assistant 2021.5.4.
This seems to be a similar issue to  #183  (but the issue turned around from FreeNAS to OMV5, because the topic changed.)
I have tried to install the mentioned "build-essentials", but these don't seem to exist on FreeBSD, with a "build" package not helping.
Version of the integration
If you are unsure about the version check the github release here.

1.5.0 (newest)

Logs

Logger: homeassistant.util.package
Source: util/package.py:96
First occurred: 10:19:20 AM (2 occurrences)
Last logged: 10:27:40 AM

Unable to install package glocaltokens==0.3.1: ERROR: Command errored out with exit status 1: command:
/usr/local/share/homeassistant/bin/python3.8 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-finegkt_/grpcio-tools/setup.py'"'"'; __file__='"'"'/tmp/pip-install-finegkt_/grpcio-tools/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-azr81ghu cwd: /tmp/pip-install-finegkt_/grpcio-tools/ Complete output (2051 lines): running bdist_wheel running build running build_py creating build creating build/lib.freebsd-12.2-RELEASE-p6-amd64-3.8 creating build/lib.freebsd-12.2-RELEASE-p6-amd64-3.8/grpc_tools copying grpc_tools/command.py -> build/lib.freebsd-12.2-RELEASE-p6-amd64-3.8/grpc_tools copying grpc_tools/__init__.py -> build/lib.freebsd-12.2-RELEASE-p6-amd64-3.8/grpc_tools copying grpc_tools/protoc.py -> build/lib.freebsd-12.2-RELEASE-p6-amd64-3.8/grpc_tools creating build/lib.freebsd-12.2-RELEASE-p6-amd64-3.8/grpc_tools/_proto creating build/lib.freebsd-12.2-RELEASE-p6-amd64-3.8/grpc_tools/_proto/google creating build/lib.freebsd-12.2-RELEASE-p6-amd64-3.8/grpc_tools/_proto/google/protobuf copying grpc_tools/_proto/google/protobuf/wrappers.proto -> build/lib.freebsd-12.2-RELEASE-p6-amd64-3.8/grpc_tools/_proto/google/protobuf copying grpc_tools/_proto/google/protobuf/type.proto -> build/lib.freebsd-12.2-RELEASE-p6-amd64-3.8/grpc_tools/_proto/google/protobuf copying grpc_tools/_proto/google/protobuf/timestamp.proto -> build/lib.freebsd-12.2-RELEASE-p6-amd64-3.8/grpc_tools/_proto/google/protobuf copying grpc_tools/_proto/google/protobuf/struct.proto -> build/lib.freebsd-12.2-RELEASE-p6-amd64-3.8/grpc_tools/_proto/google/protobuf copying grpc_tools/_proto/google/protobuf/source_context.proto -> build/lib.freebsd-12.2-RELEASE-p6-amd64-3.8/grpc_tools/_proto/google/protobuf copying grpc_tools/_proto/google/protobuf/field_mask.proto -> build/lib.freebsd-12.2-RELEASE-p6-amd64-3.8/grpc_tools/_proto/google/protobuf copying grpc_tools/_proto/google/protobuf/empty.proto -> build/lib.freebsd-12.2-RELEASE-p6-amd64-3.8/grpc_tools/_proto/google/protobuf copying grpc_tools/_proto/google/protobuf/duration.proto -> build/lib.freebsd-12.2-RELEASE-p6-amd64-3.8/grpc_tools/_proto/google/protobuf copying grpc_tools/_proto/google/protobuf/descriptor.proto -> build/lib.freebsd-12.2-RELEASE-p6-amd64-3.8/grpc_tools/_proto/google/protobuf creating build/lib.freebsd-12.2-RELEASE-p6-amd64-3.8/grpc_tools/_proto/google/protobuf/compiler copying grpc_tools/_proto/google/protobuf/compiler/plugin.proto -> build/lib.freebsd-12.2-RELEASE-p6-amd64-3.8/grpc_tools/_proto/google/protobuf/compiler

Looks like GRPC didn't compile successfully on your host.

How would I go around compiling it?

Try to run pip install grpcio-tools in your HA venv. You may be missing build-essentials alternative for FreeBSD or some dependencies.

Installed pip and installed home again. Here are some logs:
Hope they are helpful
log1.log
log2.log
log3.log
log4.log

@UraniumDonut did you manage to fix your issue?

Not yet. (Tho I haven't tried since a month) I will try again and reply

@leikoilja I have tried to reinstall with the newest version and can confirm that the issue is still there.

Update:
The issue is still there, but it quickly creates multiple processes of CC which consume a lot of CPU
└─ /usr/bin/cc -pthread -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -O2 -pipe -fstack-protector-strong -fno-strict-aliasing -fPIC -I. -Igrpc_root -Igrpc_root/include -Ithird_party/protobuf/src -I/usr/local/share/homeassistant/include -I/usr/local/include/python3.9 -c third_party/protobuf/src/google/protobuf/message_lite.cc -o build/temp.freebsd-12.2-RELEASE-p11-amd64-3.9/third_party/protobuf/src/google/protobuf/message_lite.o -std=c++11

It tries to compile the protobuf and I guess it fails at some point.
You need to figure out why grpc/protobuf doesn't compile successfully on your host.

I'm facing the same issue - homeassistant 2022.2.1 (but was also present on 2021.12). FreeBSD 12.2-RELEASE-P9 (TrueNAS-12.0-U5.1).
Python 3.9
Cython     0.29.27
essentials 1.1.4
pip        22.0.3
protobuf   3.19.4
setuptools 60.7.1
six        1.16.0
sqlite3    0.0.0
wheel      0.37.1

I m on TrueNAS-12.0-U8, but was really unhappy with the jail HomeAssistant plugin, so I ended up setting a VM with HomeAssitant OS. Works perfectly for the past 1-2 months and with the latest build of ha-google-home the integration works perfectly smooth :)

I got it working!!
I let it run pinning all 24 threads at 100% for about 4 minutes, and I was about to kill it when all of a sudden I saw one thread drop in utilization, and then another and finally after about 5 minutes total, it finished installing glocaltokens!
I went through the rest of the setup without a hitch and everything appears to be working correctly!
 I have the Google cast integration set up so I get devices from there as well as from here :(
Is there any way to pull the same/ more identifiers so that devices aren't duplicated? I can start a new issue if you prefer.

I got it working!! I let it run pinning all 24 threads at 100% for about 4 minutes, and I was about to kill it when all of a sudden I saw one thread drop in utilization, and then another and finally after about 5 minutes total, it finished installing glocaltokens! I went through the rest of the setup without a hitch and everything appears to be working correctly!  I have the Google cast integration set up so I get devices from there as well as from here :(
Is there any way to pull the same/ more identifiers so that devices aren't duplicated? I can start a new issue if you prefer.

Hey, I have a couple questions:

did you change anything with your installation (like installing grpcio-tools or build essentials) or did it just work after waiting for a long time?
What are the specs of your server, so I can guess how long to wait?

The latest version of the integration should use grpcio 1.43.0 which should be installed from wheels.

The latest version of the integration should use grpcio 1.43.0 which should be installed from wheels.

I swear when I originally (and also a few time afterwards when I was struggling to get it to work) that it was trying to install version 1.31.0... I'll try to just update everything and see what happens

Closing due to inactivity and potentially being solved.
Feel free to provide more info and reopen if relevant


0404271d-d9c7-4f1c-9a0c-837b0c55a4d3

this is for automated testing

c66d8f8c-bb19-4fb6-9d2d-1610dd4b8ce6


[fcl] Update to 0.7.0 #20917 does not compile

It seems that the doxygen usage is the problem, as it is generated in the source path.
There ist no flag to diable it in the cmake file.
See error below:
`[1/2] "C:/vcpkg/downloads/tools/cmake-3.21.1-windows/cmake-3.21.1-windows-i386/bin/cmake.exe" -S "C:/vcpkg/buildtrees/fcl/src/0.7.0-1e57a52a06.clean" -B .. "-DFCL_STATIC_LIBRARY=1" "-DFCL_BUILD_TESTS=OFF" "-DFCL_USE_X64_SSE=ON" "-DBUILD_SHARED_LIBS=OFF" "-DVCPKG_CHAINLOAD_TOOLCHAIN_FILE=C:/vcpkg/scripts/toolchains/windows.cmake" "-DVCPKG_TARGET_TRIPLET=x64-windows" "-DVCPKG_SET_CHARSET_FLAG=ON" "-DVCPKG_PLATFORM_TOOLSET=v142" "-DCMAKE_EXPORT_NO_PACKAGE_REGISTRY=ON" "-DCMAKE_FIND_PACKAGE_NO_PACKAGE_REGISTRY=ON" "-DCMAKE_FIND_PACKAGE_NO_SYSTEM_PACKAGE_REGISTRY=ON" "-DCMAKE_INSTALL_SYSTEM_RUNTIME_LIBS_SKIP=TRUE" "-DCMAKE_VERBOSE_MAKEFILE=ON" "-DVCPKG_APPLOCAL_DEPS=OFF" "-DCMAKE_TOOLCHAIN_FILE=C:/vcpkg/scripts/buildsystems/vcpkg.cmake" "-DCMAKE_ERROR_ON_ABSOLUTE_INSTALL_DESTINATION=ON" "-DVCPKG_CXX_FLAGS=" "-DVCPKG_CXX_FLAGS_RELEASE=" "-DVCPKG_CXX_FLAGS_DEBUG=" "-DVCPKG_C_FLAGS=" "-DVCPKG_C_FLAGS_RELEASE=" "-DVCPKG_C_FLAGS_DEBUG=" "-DVCPKG_CRT_LINKAGE=dynamic" "-DVCPKG_LINKER_FLAGS=" "-DVCPKG_LINKER_FLAGS_RELEASE=" "-DVCPKG_LINKER_FLAGS_DEBUG=" "-DVCPKG_TARGET_ARCHITECTURE=x64" "-DCMAKE_INSTALL_LIBDIR:STRING=lib" "-DCMAKE_INSTALL_BINDIR:STRING=bin" "-D_VCPKG_ROOT_DIR=C:/vcpkg" "-D_VCPKG_INSTALLED_DIR=C:/vcpkg/installed" "-DVCPKG_MANIFEST_INSTALL=OFF" "-DCMAKE_DISABLE_SOURCE_CHANGES=ON" -G "NMake Makefiles" "-DCMAKE_BUILD_TYPE=Release" "-DCMAKE_INSTALL_PREFIX=C:/vcpkg/packages/fcl_x64-windows"
FAILED: ../CMakeCache.txt
"C:/vcpkg/downloads/tools/cmake-3.21.1-windows/cmake-3.21.1-windows-i386/bin/cmake.exe" -S "C:/vcpkg/buildtrees/fcl/src/0.7.0-1e57a52a06.clean" -B .. "-DFCL_STATIC_LIBRARY=1" "-DFCL_BUILD_TESTS=OFF" "-DFCL_USE_X64_SSE=ON" "-DBUILD_SHARED_LIBS=OFF" "-DVCPKG_CHAINLOAD_TOOLCHAIN_FILE=C:/vcpkg/scripts/toolchains/windows.cmake" "-DVCPKG_TARGET_TRIPLET=x64-windows" "-DVCPKG_SET_CHARSET_FLAG=ON" "-DVCPKG_PLATFORM_TOOLSET=v142" "-DCMAKE_EXPORT_NO_PACKAGE_REGISTRY=ON" "-DCMAKE_FIND_PACKAGE_NO_PACKAGE_REGISTRY=ON" "-DCMAKE_FIND_PACKAGE_NO_SYSTEM_PACKAGE_REGISTRY=ON" "-DCMAKE_INSTALL_SYSTEM_RUNTIME_LIBS_SKIP=TRUE" "-DCMAKE_VERBOSE_MAKEFILE=ON" "-DVCPKG_APPLOCAL_DEPS=OFF" "-DCMAKE_TOOLCHAIN_FILE=C:/vcpkg/scripts/buildsystems/vcpkg.cmake" "-DCMAKE_ERROR_ON_ABSOLUTE_INSTALL_DESTINATION=ON" "-DVCPKG_CXX_FLAGS=" "-DVCPKG_CXX_FLAGS_RELEASE=" "-DVCPKG_CXX_FLAGS_DEBUG=" "-DVCPKG_C_FLAGS=" "-DVCPKG_C_FLAGS_RELEASE=" "-DVCPKG_C_FLAGS_DEBUG=" "-DVCPKG_CRT_LINKAGE=dynamic" "-DVCPKG_LINKER_FLAGS=" "-DVCPKG_LINKER_FLAGS_RELEASE=" "-DVCPKG_LINKER_FLAGS_DEBUG=" "-DVCPKG_TARGET_ARCHITECTURE=x64" "-DCMAKE_INSTALL_LIBDIR:STRING=lib" "-DCMAKE_INSTALL_BINDIR:STRING=bin" "-D_VCPKG_ROOT_DIR=C:/vcpkg" "-D_VCPKG_INSTALLED_DIR=C:/vcpkg/installed" "-DVCPKG_MANIFEST_INSTALL=OFF" "-DCMAKE_DISABLE_SOURCE_CHANGES=ON" -G "NMake Makefiles" "-DCMAKE_BUILD_TYPE=Release" "-DCMAKE_INSTALL_PREFIX=C:/vcpkg/packages/fcl_x64-windows"
-- The CXX compiler identification is MSVC 19.29.30133.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Set RPATH explicitly to 'C:/vcpkg/packages/fcl_x64-windows/lib'
-- Performing Test _compiler_supports_sse
-- Performing Test _compiler_supports_sse - Success
-- FCL uses OctoMap
-- Performing Test COMPILER_HAS_DEPRECATED_ATTR
-- Performing Test COMPILER_HAS_DEPRECATED_ATTR - Failed
-- Performing Test COMPILER_HAS_DEPRECATED
-- Performing Test COMPILER_HAS_DEPRECATED - Success
CMake Deprecation Warning at CMakeLists.txt:371 (message):
FCL_BUILD_TESTS is deprecated; use BUILD_TESTING instead.
-- Configuring done
CMake Error at CMakeLists.txt:405 (add_custom_command):
OUTPUT path
C:/vcpkg/buildtrees/fcl/src/0.7.0-1e57a52a06.clean/doc/doxygen/index.html

in a source directory as an output of custom command.
-- Generating done
CMake Warning:
Manually-specified variables were not used by the project:
_VCPKG_ROOT_DIR

CMake Generate step failed.  Build files cannot be regenerated correctly.
[2/2] "C:/vcpkg/downloads/tools/cmake-3.21.1-windows/cmake-3.21.1-windows-i386/bin/cmake.exe" -S "C:/vcpkg/buildtrees/fcl/src/0.7.0-1e57a52a06.clean" -B "../../x64-windows-dbg" "-DFCL_STATIC_LIBRARY=1" "-DFCL_BUILD_TESTS=OFF" "-DFCL_USE_X64_SSE=ON" "-DBUILD_SHARED_LIBS=OFF" "-DVCPKG_CHAINLOAD_TOOLCHAIN_FILE=C:/vcpkg/scripts/toolchains/windows.cmake" "-DVCPKG_TARGET_TRIPLET=x64-windows" "-DVCPKG_SET_CHARSET_FLAG=ON" "-DVCPKG_PLATFORM_TOOLSET=v142" "-DCMAKE_EXPORT_NO_PACKAGE_REGISTRY=ON" "-DCMAKE_FIND_PACKAGE_NO_PACKAGE_REGISTRY=ON" "-DCMAKE_FIND_PACKAGE_NO_SYSTEM_PACKAGE_REGISTRY=ON" "-DCMAKE_INSTALL_SYSTEM_RUNTIME_LIBS_SKIP=TRUE" "-DCMAKE_VERBOSE_MAKEFILE=ON" "-DVCPKG_APPLOCAL_DEPS=OFF" "-DCMAKE_TOOLCHAIN_FILE=C:/vcpkg/scripts/buildsystems/vcpkg.cmake" "-DCMAKE_ERROR_ON_ABSOLUTE_INSTALL_DESTINATION=ON" "-DVCPKG_CXX_FLAGS=" "-DVCPKG_CXX_FLAGS_RELEASE=" "-DVCPKG_CXX_FLAGS_DEBUG=" "-DVCPKG_C_FLAGS=" "-DVCPKG_C_FLAGS_RELEASE=" "-DVCPKG_C_FLAGS_DEBUG=" "-DVCPKG_CRT_LINKAGE=dynamic" "-DVCPKG_LINKER_FLAGS=" "-DVCPKG_LINKER_FLAGS_RELEASE=" "-DVCPKG_LINKER_FLAGS_DEBUG=" "-DVCPKG_TARGET_ARCHITECTURE=x64" "-DCMAKE_INSTALL_LIBDIR:STRING=lib" "-DCMAKE_INSTALL_BINDIR:STRING=bin" "-D_VCPKG_ROOT_DIR=C:/vcpkg" "-D_VCPKG_INSTALLED_DIR=C:/vcpkg/installed" "-DVCPKG_MANIFEST_INSTALL=OFF" "-DCMAKE_DISABLE_SOURCE_CHANGES=ON" -G "NMake Makefiles" "-DCMAKE_BUILD_TYPE=Debug" "-DCMAKE_INSTALL_PREFIX=C:/vcpkg/packages/fcl_x64-windows/debug"
FAILED: ../../x64-windows-dbg/CMakeCache.txt
"C:/vcpkg/downloads/tools/cmake-3.21.1-windows/cmake-3.21.1-windows-i386/bin/cmake.exe" -S "C:/vcpkg/buildtrees/fcl/src/0.7.0-1e57a52a06.clean" -B "../../x64-windows-dbg" "-DFCL_STATIC_LIBRARY=1" "-DFCL_BUILD_TESTS=OFF" "-DFCL_USE_X64_SSE=ON" "-DBUILD_SHARED_LIBS=OFF" "-DVCPKG_CHAINLOAD_TOOLCHAIN_FILE=C:/vcpkg/scripts/toolchains/windows.cmake" "-DVCPKG_TARGET_TRIPLET=x64-windows" "-DVCPKG_SET_CHARSET_FLAG=ON" "-DVCPKG_PLATFORM_TOOLSET=v142" "-DCMAKE_EXPORT_NO_PACKAGE_REGISTRY=ON" "-DCMAKE_FIND_PACKAGE_NO_PACKAGE_REGISTRY=ON" "-DCMAKE_FIND_PACKAGE_NO_SYSTEM_PACKAGE_REGISTRY=ON" "-DCMAKE_INSTALL_SYSTEM_RUNTIME_LIBS_SKIP=TRUE" "-DCMAKE_VERBOSE_MAKEFILE=ON" "-DVCPKG_APPLOCAL_DEPS=OFF" "-DCMAKE_TOOLCHAIN_FILE=C:/vcpkg/scripts/buildsystems/vcpkg.cmake" "-DCMAKE_ERROR_ON_ABSOLUTE_INSTALL_DESTINATION=ON" "-DVCPKG_CXX_FLAGS=" "-DVCPKG_CXX_FLAGS_RELEASE=" "-DVCPKG_CXX_FLAGS_DEBUG=" "-DVCPKG_C_FLAGS=" "-DVCPKG_C_FLAGS_RELEASE=" "-DVCPKG_C_FLAGS_DEBUG=" "-DVCPKG_CRT_LINKAGE=dynamic" "-DVCPKG_LINKER_FLAGS=" "-DVCPKG_LINKER_FLAGS_RELEASE=" "-DVCPKG_LINKER_FLAGS_DEBUG=" "-DVCPKG_TARGET_ARCHITECTURE=x64" "-DCMAKE_INSTALL_LIBDIR:STRING=lib" "-DCMAKE_INSTALL_BINDIR:STRING=bin" "-D_VCPKG_ROOT_DIR=C:/vcpkg" "-D_VCPKG_INSTALLED_DIR=C:/vcpkg/installed" "-DVCPKG_MANIFEST_INSTALL=OFF" "-DCMAKE_DISABLE_SOURCE_CHANGES=ON" -G "NMake Makefiles" "-DCMAKE_BUILD_TYPE=Debug" "-DCMAKE_INSTALL_PREFIX=C:/vcpkg/packages/fcl_x64-windows/debug"
-- The CXX compiler identification is MSVC 19.29.30133.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Set RPATH explicitly to 'C:/vcpkg/packages/fcl_x64-windows/debug/lib'
-- Performing Test _compiler_supports_sse
-- Performing Test _compiler_supports_sse - Success
-- FCL uses OctoMap
-- Performing Test COMPILER_HAS_DEPRECATED_ATTR
-- Performing Test COMPILER_HAS_DEPRECATED_ATTR - Failed
-- Performing Test COMPILER_HAS_DEPRECATED
-- Performing Test COMPILER_HAS_DEPRECATED - Success
CMake Deprecation Warning at CMakeLists.txt:371 (message):
FCL_BUILD_TESTS is deprecated; use BUILD_TESTING instead.
-- Configuring done
CMake Error at CMakeLists.txt:405 (add_custom_command):
OUTPUT path
C:/vcpkg/buildtrees/fcl/src/0.7.0-1e57a52a06.clean/doc/doxygen/index.html

in a source directory as an output of custom command.
-- Generating done
CMake Warning:
Manually-specified variables were not used by the project:
_VCPKG_ROOT_DIR

CMake Generate step failed.  Build files cannot be regenerated correctly.
ninja: build stopped: subcommand failed.
`
Originally posted by @CaeruleusAqua in https://github.com/microsoft/vcpkg/issues/20917#issuecomment-957642129

I created a quick patch:
diff --git a/CMakeLists.txt b/CMakeLists.txt
index 6e5bbb8..5a1523b 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -389,6 +389,8 @@ endif()
 #   http://www.cmake.org/pipermail/cmake/2007-February/012796.html
 #===============================================================================
 # Doxygen
+
+if(DOXYGEN_ENABLED)
 find_package(Doxygen QUIET)

 if(DOXYGEN_FOUND)
@@ -420,3 +422,4 @@ if(DOXYGEN_FOUND)
   )

 endif()
+endif()
--

There ist no flag to diable it in the cmake file.

-DCMAKE_DISABLE_FIND_PACKAGE_Doxygen=1

There ist no flag to diable it in the cmake file.

-DCMAKE_DISABLE_FIND_PACKAGE_Doxygen=1

Good to know ;-)


ForceTorque sensor stores wrenches/IMU sensor improvement

ForceTorque sensor stores and gives wrenches (in addition to force/torque vectors). One note: force/torque getters make copies now..

IMU sensor returns the angular position. For now this is the ground truth value; in subsequent versions we need to do this closer to what real sensors do.

OK for merging asap.

OK for merging asap.

I'll wait for the CIs to pass..


docs: Reorganize content

Give the content some structure similar to the nstream backend docs

Closed with #135


rplidar_ros: 1.5.3-0 in 'jade/distribution.yaml' [bloom]

Increasing version of package(s) in repository rplidar_ros to 1.5.3-0:

upstream repository: https://github.com/robopeak/rplidar_ros.git
release repository: https://github.com/robopeak/rplidar_ros-release.git
distro file: jade/distribution.yaml
bloom version: 0.5.21
previous version for package: null

rplidar_ros
* fixed the changeLog and delete tracks.yaml
* change supporter email
* Initial tracks.yaml
* Contributors: kint.zhao

@wjwwood   why my package can not pass jenkins build .  can you tell me some method  to solve it with web link.
email report
Invoking 'git clone --branch debian/ros-jade-rplidar-ros_1.5.3-0_vivid --depth 1 --no-single-branch https://github.com/robopeak/rplidar_ros-release.git /tmp/sourcedeb/source'
dpkg-source: error: syntax error in source/debian/control at line 14: block lacks the 'Package' field  .
but i check it,  and it is ok  in my release github .

yhzhao@yhzhao:~$ apt-cache show ros-jade-rplidar-ros
N: Unable to locate package ros-jade-rplidar-ros
E: No packages found

This is still failing to build the source deb. I'll revert it.
I don't know why but you have extra empty lines which seem to causing the error you pasted here. You cannot use apt-cache show because the source deb is never generated.

@wjwwood  I   delete the empty lines  using git edit button,  but it still build fail in jenkins .  How can i check it detail , can you give me some tutorial  about jenkins with ros ?


Markdown Sample🐛

What version of Integration has the issue?
2.6.1
System Health details
Not needed
Checklist

[X] I have enabled debug logging for my installation.
[X] This issue only contains 1 issue (if you have multiple issues, open one issue for each issue).

Describe the issue
I tried the Markdown Sample, only changed the sensor ro Match my one but Markdown stays empty
An 30 Seconds Automation to read the Messages is also created
Reproduction steps

Copy Paste Markdown Sample,
edit Sensor
Created Automation to read Messages

Debug logs
None

@bluenazgul
Have you tried a restart?

I tried several restarts, sadly without a changing result
I See the Message Sensor changing, once the watch sends s Message, but thats all

By the way, have also empty markdown...

@RestOp
@bluenazgul
Check: https://github.com/Ludy87/xplora_watch/issues/185


add dynamic rendering, new icons and update readme

Hi, I'm Agilso! First off, thanks for creating this package.
I've recently forked the repository and made a few improvements that I hope you'll find useful.

The first update is adding new icons from the Lucide collection.
For the second update, I introduced dynamic rendering. This allows for rendering icons directly from a variable and/or creating a custom component. It’s worth mentioning that HeroIcon uses a similar approach.
Lastly, I made some minor tweaks to the readme.md.

I hope these changes help. Thanks again!
Agilso!

Looks great! Thanks for the enhancements and improved documentation.