Added osramlightify groups.

Allows you to make use of the build in osram lightify groups. Group states get
handeled similar as in the case of phillips hue. A lightify group shows up as
light in the homeassistant webinterface. If one light of the
group is on, the complete group is considered to be on.
To use this feature, first define some groups within your lighrify bridge, then
add allow_lightify_groups=true to you osramlightify config.
Example entry for configuration.yaml (if applicable):
- platform: osramlightify
  host: IP-ADDRES
  allow_lightify_groups: true

Checklist:
If user exposed functionality or configuration variables are added/changed:

[ ] Documentation added/updated in home-assistant.github.io

If the code communicates with devices, web services, or third-party tools:

[ ] Local tests with tox run successfully. Your PR cannot be merged unless tests pass
[ ] New dependencies have been added to the REQUIREMENTS variable (example).
[ ] New dependencies are only imported inside functions that use them (example).
[ ] New dependencies have been added to requirements_all.txt by running script/gen_requirements_all.py.
[ ] New files were added to .coveragerc.

If the code does not interact with devices:

[ ] Local tests with tox run successfully. Your PR cannot be merged unless tests pass
[ ] Tests have been added to verify that the new code works.

Can you also remove all the debug statements in properties ? It is slowing down Hass a lot and it's unnecessary.

This is ok to merge when the comment change has been reversed and linting has been fixed 🎉

I tried to rebase it, but I coudnt rebase anything before the merge. Shoud I make a new PR or is it okay the way it is?

You should revert the change to the doc string as @balloob asked, before we can merge. See comment above.

Awesome, this was a Feature Request I really wanted! Thank you! <3

:) I wanted it as well, and so I made it.
chimpy<EMAIL_ADDRESS>writes:

Awesome, this was a Feature Request I really wanted! Thank you! <3

This change breaks my Home Assistant setup.  Each time I start HA the value of self._light_ids[0] seems to be incorrect but the error does not always happen with the same light.  Maybe there's a discrepancy between the bridge and the states in HA.  I've rolled back to 0.43.2 for now and the problem went away.
2017-05-10 13:31:46 ERROR (MainThread) [homeassistant.components.light] Error while setting up platform osramlightify Traceback (most recent call last):   File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/helpers/entity_component.py", line 155, in _async_setup_platform     entity_platform.schedule_add_entities, discovery_info   File "/opt/rh/rh-python35/root/usr/lib64/python3.5/asyncio/futures.py", line 358, in __iter__     yield self  # This tells Task to wait for completion.   File "/opt/rh/rh-python35/root/usr/lib64/python3.5/asyncio/tasks.py", line 290, in _wakeup     future.result()   File "/opt/rh/rh-python35/root/usr/lib64/python3.5/asyncio/futures.py", line 274, in result     raise self._exception   File "/opt/rh/rh-python35/root/usr/lib64/python3.5/concurrent/futures/thread.py", line 55, in run     result = self.fn(*self.args, **self.kwargs)   File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/components/light/osramlightify.py", line 60, in setup_platform     setup_bridge(bridge, add_devices, add_groups)   File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/components/light/osramlightify.py", line 106, in setup_bridge     update_lights()   File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/util/__init__.py", line 303, in wrapper     result = method(*args, **kwargs)   File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/util/__init__.py", line 303, in wrapper     result = method(*args, **kwargs)   File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/components/light/osramlightify.py", line 97, in update_lights     update_lights)   File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/components/light/osramlightify.py", line 268, in __init__     super().__init__(group, update_lights)   File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/components/light/osramlightify.py", line 121, in __init__     self.update()   File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/components/light/osramlightify.py", line 282, in update     light = self._bridge.lights()[self._light_ids[0]] KeyError:<PHONE_NUMBER>890067646

Hey Ian
Would you be willing to help me fix it? I cant reproduce it, so I need
somebody to test a fix and probably give me some debug output.
Cheers.
Ian Young<EMAIL_ADDRESS>writes:

This change breaks my Home Assistant setup.  Each time I start HA the value of self._light_ids[0] seems to be incorrect but the error does not always happen with the same light.  Maybe there's a discrepancy between the bridge and the states in HA.  I've rolled back to 0.43.2 for now and the problem went away.
2017-05-10 13:31:46 ERROR (MainThread) [homeassistant.components.light] Error while setting up platform osramlightify Traceback (most recent call last):   File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/helpers/entity_component.py", line 155, in _async_setup_platform     entity_platform.schedule_add_entities, discovery_info   File "/opt/rh/rh-python35/root/usr/lib64/python3.5/asyncio/futures.py", line 358, in __iter__     yield self  # This tells Task to wait for completion.   File "/opt/rh/rh-python35/root/usr/lib64/python3.5/asyncio/tasks.py", line 290, in _wakeup     future.result()   File "/opt/rh/rh-python35/root/usr/lib64/python3.5/asyncio/futures.py", line 274, in result     raise self._exception   File "/opt/rh/rh-python35/root/usr/lib64/python3.5/concurrent/futures/thread.py", line 55, in run     result = self.fn(*self.args, **self.kwargs)   File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/components/light/osramlightify.py", line 60, in setup_platform     setup_bridge(bridge, add_devices, add_groups)   File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/components/light/osramlightify.py", line 106, in setup_bridge     update_lights()   File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/util/__init__.py", line 303, in wrapper     result = method(*args, **kwargs)   File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/util/__init__.py", line 303, in wrapper     result = method(*args, **kwargs)   File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/components/light/osramlightify.py", line 97, in update_lights     update_lights)   File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/components/light/osramlightify.py", line 268, in __init__     super().__init__(group, update_lights)   File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/components/light/osramlightify.py", line 121, in __init__     self.update()   File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/components/light/osramlightify.py", line 282, in update     light = self._bridge.lights()[self._light_ids[0]] KeyError:<PHONE_NUMBER>890067646

I won't be available until Wednesday but I can help you after then.

Please open an issue and discuss a potential fix there.


Switch over to PyTorch testing parametrization

from our custom one

Done in https://github.com/facebookresearch/functorch/commit/e1811d6a81f513b9b694a88da22ce0b0794f7c24


e2e: make tests divided into smaller parts (DLB, DSA, IAA, QAT)

Restructure e2e tests from a single It() as follows:
Describe("device")
*BeforeEach("deploys plugin")
**Context("When device resources are available")
***BeforeEach("checks if resources are available")
****It("runs a pod requesting resources")
****It("runs another pod requesting resources if there is")
Signed-off-by: Hyeongju Johannes Lee<EMAIL_ADDRESS>

Codecov Report

Merging #1263 (70bc243) into main (dfc8918) will not change coverage.
The diff coverage is n/a.

:exclamation: Current head 70bc243 differs from pull request most recent head 8752bfc. Consider uploading reports for the commit 8752bfc to get more accurate results

@@           Coverage Diff           @@
##             main    #1263   +/-   ##
=======================================
  Coverage   50.84%   50.84%
=======================================
  Files          41       41
  Lines        4608     4608
=======================================
  Hits         2343     2343
  Misses       2131     2131
  Partials      134      134

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

Additionally, fix grammar mistakes and typos

these can go to a separate PR

@mythi Sure, i will divide them.!

Updates:
FPGA has some problem, so I thought it would be good to do first the 4 device plugins that are used in SPR (DLB, DSA, IAA, QAT).
To do (maybe in another PR):
QAT4 - cy device is recognized as generic inside a VM.
DLB - pf device is recognized as vf inside a VM.
We need to deal with this matter.

Could you review? :)

@mythi @ozhuraki Could you review? :)

Is there a possibility this breaks if Ginkgo randomizes things differently? Now the test cases have a dependency and it might be that the demo app gets deployed before the plugin?

@mythi yes, I thought about that matter,, and there is 'ginkgo. Order' that makes everything run in order. Following ones are skipped in case previous It() fails.
I thought I put them already since no test got failed. Let me put it.

there is 'ginkgo. Order'

randomisation is on purpose to ensure the tests are correctly written so I don't think we want to disable it

@mythi Umm, did I understand what you meant correctly? What 'randomization' do we have for now? We have only one It()!

@mythi Umm, did I understand what you meant correctly? What 'randomization' do we have for now? We have only one It()!

I was reading this before asking my question https://onsi.github.io/ginkgo/#spec-randomization

I then suggest this way:
Describe ("device")

BeforeEach("deploys plugin")

Context("When device resources are available")

BeforeEach("checks if resources are available")
It("runs a pod requesting resources")
It("runs another pod requesting resources if there is")

In this way, we can randomize specs and does not cause any problem no matter how many It()s exists.

In this way, we can randomize specs and does not cause any problem no matter how many It()s exists.

The proposal looks much better and closer to what I originally had in mind with #1143

@mythi Thanks! :) I guess all ci passed.! Please review the code! :)
For the matter of GPU and FPGA, can I do in another PR later?

@mythi Thanks! :) I guess all ci passed.! Please review the code! :) For the matter of GPU and FPGA, can I do in another PR later?

It's fine. For this PR, I'd suggest we change things to one commit per device.

@mythi Done.! What else would this pr you think need? :)

@ozhuraki Could you review?

After this get merged, let me also make a pr for the improvement of documentations for e2e tests.!


installation error in laravel 5.7

i am getting this error while trying to install version 3.0.0 which is compatible with laravel 5.7
   Problem 1     - Installation request for pbmedia/laravel-ffmpeg 3.0.0 -> satisfiable by pbmedia/laravel-ffmpeg[3.0.0].     - pbmedia/laravel-ffmpeg 3.0.0 requires php-ffmpeg/php-ffmpeg dev-master#c11b79ab5b0174aa1a56c54c67491169e78a4c17 -> satisfiable by php-ffmpeg/php-ffmpeg[dev-master] but these conflict with your requirements or minimum-stability.
can you please solve this issue for me
this is my project other packages
 "php": ">=7.2.19",         "alaouy/youtube": "^2.2",         "barryvdh/laravel-cors": "^0.11.1",         "consoletvs/charts": "^6.5",         "cyrildewit/eloquent-viewable": "^3.0",         "devmarketer/easynav": "^1.0",         "doctrine/dbal": "^2.6",         "fideloper/proxy": "^4.0",         "geoip2/geoip2": "^2.8",         "intervention/image": "^2.4",         "laravel/cashier": "~7.0",         "laravel/cashier-braintree": "^3.1",         "laravel/framework": "5.7.*",         "laravel/passport": "^7.5",         "laravel/socialite": "3.0.9",         "laravel/tinker": "~1.0",         "laravelcollective/html": "^5.5",         "paypal/rest-api-sdk-php": "*",         "pragmarx/tracker": "^3.2",         "socialiteproviders/gitlab": "^3.1.0",         "spatie/laravel-newsletter": "^4.2",         "spatie/laravel-permission": "^2.31",         "spatie/laravel-translatable": "^2.2",         "stripe/stripe-php": "^5.8",         "tzsk/payu": "^2.0",         "pbmedia/laravel-ffmpeg":"3.0.0",         "unicodeveloper/laravel-paystack": "^1.0",         "yajra/laravel-datatables-oracle": "~5.0"

Please try using 3.0.2, there was a dependency issue with 3.0.0. It might be useful to upgrade your Laravel version as well, 5.7 is not supported anymore.


Question: How to set a persistent iptables rule

Hello,
I would like to setup a persistent iptables rule on each of my flatcar VMs. How would I do this?
First, I have just set it manually and I thought one of the systemd services would persist it automatically before a controlled reboot/shutdown.

Hi @shibumi,
You just need two things and they can be done trough ignition:

add your rule to /var/lib/iptables/rules-save
enable the systemd service: iptables-restore.service

With this both condition, you should be able to persist your iptables rule across the reboot. Let us know how it goes :)

Hi @tormath1,
first of all: Wow! That answer was fast. I did not expect that :)
I will check this out. Thanks for the help

@shibumi : here's a simple example with a basic iptable rule: https://gist.github.com/tormath1/1184ffb73bc393c6413d94bd1317b486

@tormath1 is the "ct" tool still recommended? Somewhere, I heard that it's deprecated and nobody should use it. If it's not deprecated, can you point me to the github repository? I would like to package it for Arch Linux :)

@shibumi ct is still maintained by Flatcar - we even released 0.9.1 in June 2021.
We're currently working on adding support for Ignition version 3 specs into Flatcar, ct upgrade will follow to generate Ignition version 3.

I would like to package it for Arch Linux :)

That would be awesome for the community! Do you want add it to the AUR? Don't hesitate to ping in this issue or in the Matrix channel if you need a hand. Thanks a lot!

ct is still maintained by Flatcar - we even released 0.9.1 in June 2021.

Oh okay. Looks like I visited the wrong repository in the past. I think a few months ago the kinvolk documencation has linked to the containerOS repository instead of the flatcar-linux repo.

That would be awesome for the community! Do you want add it to the AUR? Don't hesitate to ping in this issue or in the Matrix channel if you need a hand. Thanks a lot!

No. I am package maintainer. I would like to push it to the official repositories :) I maintain CNCF-related packages there (kubernetes, istio, helm, etc)..

@tormath1 https://archlinux.org/packages/community/x86_64/ct/ done

@shibumi thanks a lot, this is awesome !
Just a nit detail, the version should be passed as a ldflag, so we currently have:
$ ct --version
ct was not built properly

We might need to update the PKGBUILD with something like:
22c22
<   go build -ldflags="-w -s -linkmode=external" -o ct
---
>   go build -ldflags="-w -s -linkmode=external -X github.com/flatcar-linux/container-linux-config-transpiler/internal/version.Raw=${pkgver}" -o ct

To finally have the version:
$ ct --version
ct v0.9.1

Hi @shibumi , we just released a new version of ct: https://github.com/flatcar-linux/container-linux-config-transpiler/releases/tag/v0.9.2. (I don't know if you susbcribed to the github "release" notification)

@tormath1  I forgot subscribing. Thanks for the reminder! :)

Hi @shibumi,
A new version of ct (0.9.4) is available - it's mainly to follow the recent renaming of flatcar-linux org to flatcar, here's the patch to apply on the PKGBUILD:
--- PKGBUILD	2022-09-14 11:35:38.512089045 +0200
+++ PKGBUILD.new	2022-09-14 11:39:16.874898786 +0200
@@ -4,13 +4,13 @@
 pkgrel=1
 pkgdesc='config transpiler for Flatcar Container Linux'
 arch=('x86_64' 'aarch64')
-url='https://github.com/flatcar-linux/container-linux-config-transpiler'
+url='https://github.com/flatcar/container-linux-config-transpiler'
 license=('Apache')
 depends=('kubectl')
 makedepends=('git' 'go')
-source=("${pkgname}-${pkgver}.tar.gz::https://github.com/flatcar-linux/container-linux-config-transpiler/archive/refs/tags/v${pkgver}.tar.gz")
-sha512sums=('b31000fa24607c9480e6e4df67f779e7052e2da160ce0c08c5a0fbb9666d750003b5b75995af08ccc442abaccb059f0f6e7fe8fdfaadad90c4accdd490082078')
-b2sums=('23c43d103d960deca8075b92545841c34bc553b28212500846c4f2770b31bc9c803d894a2e20f8d67e6a3f4bebfc55aaa1d9a7442008e407af7a6c94353f2e5f')
+source=("${pkgname}-${pkgver}.tar.gz::https://github.com/flatcar/container-linux-config-transpiler/archive/refs/tags/v${pkgver}.tar.gz")
+sha512sums=('a00e3a1ab401630a7c38a18b4f0ffb43d433134353e774ceae633338f32926989db75f18b2d58bcf7ed2103412472da8a8700990136c9197c0211680128edd75')
+b2sums=('90d0c875d523e5223b1015fc2433c389f02437d129ac4cd683699a3d59fe50ec4465e7389cfd7f7636c233bf63b7091a35781fe1a54bbc4ce6fe62e867c00300')

 build() {
   cd "container-linux-config-transpiler-${pkgver}/internal"
@@ -19,7 +19,7 @@
   export CGO_CPPFLAGS="$CPPFLAGS"
   export CGO_CXXFLAGS="$CXXFLAGS"
   export GOFLAGS='-buildmode=pie -trimpath -modcacherw -mod=readonly'
-  go build -ldflags="-w -s -linkmode=external -X github.com/flatcar-linux/container-linux-config-transpiler/internal/version.Raw=${pkgver}" -o ct
+  go build -ldflags="-w -s -linkmode=external -X github.com/flatcar/container-linux-config-transpiler/internal/version.Raw=${pkgver}" -o ct
 }

 #check() {

I can submit the patch myself if you're busy at the moment. Thanks again for maintaining this on Arch!

@tormath1  Sorry for the delay :D let me work on it.

I released 0.9.4. :)
sorry for the delay again, I recently switched the job and the first few weeks were a little bit overwhelming.


i386 installer

Is there any i386 .deb for atom? I only found amd64

Thanks for reaching out!
Atom only has 64-bit installers for Linux.
x-ref: https://github.com/atom/atom/issues/6252

Because we treat our issues list as the Atom team's backlog, we close issues that are questions since they don't represent a task needing to be completed. For most questions about Atom there are a lot of options, check out:

User Documentation — The Atom Flight Manual
Developer Documentation — Atom API Documentation
FAQ — The Atom FAQ on Discuss
Message Board — Discuss, the official Atom and Electron message board
Chat — Join the Atom Slack team

On Discuss and in the Atom Slack team, there are a bunch of helpful community members that should be willing to point you in the right direction.


Block builder interrupt, incremental block building option

Description
As a validator, I want besu to produce a block within a specified duration so that I do not miss block proposals.
In some cases besu might take a too long to build a full block, and when the consensus client asks for the prepared block, only an empty block is available.  Conversely, if besu builds a block quickly (with few or low value transactions) and susequent high value transaction arrive after the block is constructed, besu will produce a suboptimal block.
Both of these situations can be mitigated by having besu's block building process continue to iterate on building the highest value block until it reaches a timeout threshold and/or it is interrupted via engine_getPayload.
Acceptance Criteria

implement an interruptible block builder that will save a partial block on interruption
engine_getPayload should interrupt block creation, causing the current partial block to be saved (and if it is the highest value block returned to the calling consensus layer)

relates to #4400


Use GCM encryption for configuration cache

Configuration cache encryption currently uses the Electronic Codebook (ECB) mode of AES. This mode is generally regarded as insecure. See:
https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation#Electronic_codebook_(ECB)
In general, for any algorithm to be secure, it must include a random initialization vector -- a string of bytes that must never be reused as an IV with the same key. This ensures that the same plaintext never maps to the same ciphertext when encrypted more than once.
There are a number of AES modes that satisfy this constraint. However, CTR and GCM modes can be parallelized. Most well-known modes, such as CBC, CFB, and OFB, rely on some form of serial chaining of blocks during encryption and decryption.
This commit migrates CC from the AES/ECB/PKCS5PADDING transformation to the AES/GCM/NoPadding transformation.
Reviewing cheatsheet
Before merging the PR, comments starting with

❌ ❓must be fixed
🤔 💅 should be fixed
💭 may be fixed
🎉 celebrate happy things

@bot-gradle test apt

I've triggered the following builds for you. Click here to see all build failures.

AllPerformanceTestsReadyForRelease build

@bot-gradle test apt

I've triggered the following builds for you. Click here to see all build failures.

AllPerformanceTestsReadyForRelease build

@bot-gradle test apt

I've triggered the following builds for you. Click here to see all build failures.

AllPerformanceTestsReadyForRelease build

@bot-gradle test apt

I've triggered the following builds for you. Click here to see all build failures.

AllPerformanceTestsReadyForRelease build

@bot-gradle test rfn

I've triggered the following builds for you. Click here to see all build failures.

ReadyForNightly build

@bot-gradle test PT

I've triggered the following builds for you. Click here to see all build failures.

AllPerformanceTestsReadyForNightly build


🐞[Bug]:  Missing Icons in some places

Is there an existing issue for this?

[X] I have searched the existing issues

Describe the bug
The social media icons are missing in some pages , present in some places these will lead to bug , so I recommend to change that situation.
Asign me this isssue @Anishkagupta04 to contribute this project
Expected behavior
Missing of social media icons lead to make user thought ,they are not have those accounts for the organization.
Add ScreenShots
Here all present
Main Page

Here some are missing
Doctors listing page

In this page also
Contact Page

What browsers are you seeing the problem on?
No response
Record

[X] I have read the Contributing Guidelines
[X] I'm a GSSOC'24 contributor
[X] I have starred the repository

assigning


I need to send options as arguments to puppeteer

You have
...restOptions,
         html: tmpHtmlPath,
         scriptFn: conversionFactory.getScriptFn()

I want to add:
headless: true,
         slowMo: 100,
         args: amIRoot() ? ['--no-sandbox', '--disable-setuid-sandbox'] : undefined,

But I need a few options to be passed to puppeteer. Please direct me to where in the lib to make the adjustment, or let know if there's another way to do this.
Just started using this package...pretty good!

I ended up modifying the lib/eval.js file in chrome-page-eval
But I don't wanna do that. I tried adding args to restOptions, but it was ignored.

you can pass the options when creating the instance of chrome-page-eval
const chromeEval = require('chrome-page-eval')({
  puppeteer,
  launchOptions: {
    slowMo: 100,
    args: amIRoot() ? ['--no-sandbox', '--disable-setuid-sandbox'] : undefined
  }
})


Change sender and preview text for collective update emails

Currently the update emails say they come from "Hello". This adds to the problem of it not being clear which collective a message is from, and looks a bit odd in today's polished email marketing environment. I suggest changing it to "Update from [Collective Name]".
Also, the first few words that show in the preview are always "view online [collective name] logo", which isn't very engaging or helpful. I've seen lots of other emails that begin with a brand image that don't show that kind of text, so there must be a way to start the preview with the actual text content? Or, if not, can we add something generic to show there like "Message for backers of the [collective name] Collective"?

good point, fixing this


GEODE-8870: Removes GFE_70.

The PR is intentionally broken into multiple commits. The work to just remove the version is in the first commit.
The followup commits are to cleanup all the compiler and static analyzer warnings in the changed files.
Thank you for submitting a contribution to Apache Geode.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
For all changes:

[ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message?

[ ] Has your PR been rebased against the latest commit within the target branch (typically develop)?

[ ] Is your initial contribution a single, squashed commit?

[ ] Does gradlew build run cleanly?

[ ] Have you written or updated unit tests to verify your changes?

[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?

Note:
Please ensure that once the PR is submitted, check Concourse for build issues and
submit an update to your PR as soon as possible. If you need help, please send an
email to<EMAIL_ADDRESS>

I see lot more refactoring changes than the fix for actual issue. I agree and believe in incremental refactoring as we come across the opportunity; but here I see lot more refactoring changes; even in non-related files (where the actual change is not expected); it will be nice to keep those changes minimal and focused on the issue being fixed...just my thoughts and I am fine with other way too...

I see lot more refactoring changes than the fix for actual issue. I agree and believe in incremental refactoring as we come across the opportunity; but here I see lot more refactoring changes; even in non-related files (where the actual change is not expected); it will be nice to keep those changes minimal and focused on the issue being fixed...just my thoughts and I am fine with other way too...

The fix and the cleanup on on two separate commits to make reviewing easier. The other cleaned files are files affected by the correction of issues in a dependent class.

Also, I would request refactorings to be kept in separate PRs.

The caption JIRA mentions that this GEODE-8870 ticket is for removal of obsolete GFE_70 code but has GEODE-6588 along with it which has other major refactorings.
Keeping the CL shorter will have faster approval / thorough review as the reviewer list smaller
We view the final set of files changed as it is more convenient than looking at each and every commit individually, as the same file may have been modified in multiple commits.

Also, I would request refactorings to be kept in separate PRs.

The caption JIRA mentions that this GEODE-8870 ticket is for removal of obsolete GFE_70 code but has GEODE-6588 along with it which has other major refactorings.
Keeping the CL shorter will have faster approval / thorough review as the reviewer list smaller
We view the final set of files changed as it is more convenient than looking at each and every commit individually, as the same file may have been modified in multiple commits.

I think you should take this preference to a discussion on dev@geode. We have had similar discussions without an clear consensus. The closest we got was around not forcing squashes on PRs that include explicitly distinguished fix and cleanup commits, like this one.
https://markmail.org/message/hflkinvcxjg6stbo

Sure but GEODE-6588 could have been another PRs and limit this PR to GEODE-8870

Sure but GEODE-6588 could have been another PRs and limit this PR to GEODE-8870

Is the issue that there is cleanup of files touched by the fix in this PR as separate commits or that those commits have a different JIRA issue associated with them?


updated middleware.py to make it proxy proof

The middleware is now proxy proof. IP addresses will be displayed correctly.
Instead of the Proxy servers's IP addresses the X_FORWARDED_FOR ip_address will now be displayed.
CloudFlare proof.

We can't rely on that header to be correctly set by http middleware. See also #12, #20, #21, #35 and #70.


fromJS reviver should have access to the full key path

I've come to the conclusion that the fromJS reviver should have access to the full key path.
If that was provided then we can easily setup a map to do custom things at certain paths when converting nested objects and lists into Immutable.js structures.
For instance, given a plain JS structure
const struct1 = {
  profile: {
    firstName: 'John',
    lasName: 'Smith',
    emails: [
      { type: 'home', name: 'J Smith', email<EMAIL_ADDRESS>},
      { type: 'work', name: 'John Smith', email<EMAIL_ADDRESS>}
    ]
  }
};

If we want to convert structures like this into Records and Lists, we might create a reviver that maps the root, profile, emails, and emails entries to records.
const recordMap = {
  '': RootRecord,
  'profile': ProfileRecord,
  'profile.emails[x]': EmailRecord
}

Then in our reviver, we can apply the appropriate record based on the full path of the key.
Currently as of today, we only get the local key, so there is no good way to setup a proper mapping. That local key might be used in many places so we really need a full path key. Also with the list items, the local key will just be a number, so this doesn't tell us which Record to use assuming there could be many lists in the structure.
I see that your v4 subRecords feature might help but even then I'm not sure they go deep enough and can reach into lists.
But besides the use of Records if you wanted to do any specific tailoring to structures when converting with fromJS, it really makes sense to have the full key path available so you can know what to do. You can't always just assume things based on the data, it would be better to also have the full key path so you know if you are in the right part of the structure.
I thought I might be able to figure out the full key path using the this object which provides the parent, but as far as I can tell that doesn't help any.
If there is another way to get the full path in the reviver, please let me know.
Otherwise, I'm thinking it would be good to add another parameter which is the full key path to the reviver. For consistency with the rest of the API it would probably be an array like ['profile', 'emails', 1].
So the new signature for the fromJS reviver would be
fromJS(json: any, reviver?: (k: any, v: Iterable<any, any>, path: Array<string>) => any): any

Thoughts or alternative ideas?

Awesome! Thanks @leebyron


Deprecate dollar sign during translation

This pull request addresses #86 by replacing the occurrences of dollar signs with double underscores.

The signal name with N levels of hierarchy will be 5 characters longer than before if we use double underscore. Is that a problem? I guess it's not a big problem?

The signal name with N levels of hierarchy will be 5 characters longer than before if we use double underscore. Is that a problem? I guess it's not a big problem?

This in most cases is not a problem. However if it’s the component name, Verilator might fail due to excessively long names as #75 indicates, and I plan to address that in another pull request.


Max data packet length and callback timing on NimBLE vs. Arduino standard BLE library

Hi,
I'm porting some file transfer code to NimBLE that previously used the Arduino standard BLE library but are running into some issues. I wonder if I can get some help sorting this out as I really like (and need) the exceptionally low footprint of NimBLE.
Firstly, pls note that the file transfer works fine with the standard Arduino BLE library. However the write callback, when using NimBLE, is not called (at all it seems) if the data package sent from the client is longer than 512 bytes.
Secondly, when limiting data package length to 512 or lower on the client side, NimBLE seems to drop data packages when I perform something slightly time consuming inside the callback (like writing to a SD card, even if I buffer before writing to the card). In fact I always randomly loose between 2 to 40 bytes. This works without any issue with the Arduino standard BLE library.
Here's some details:

My ESP32 is running an BLE server with a characteristic (WRITE_NR) used for receiving files from an iOS app on an iPhone X.

My iOS app report a possible MTU of 514 bytes after connection to the ESP32. I use this length to chunk up the transfer file into chunks of 514 bytes. These are transmitted to the ESP32 with "write without response" (WRITE_NR). The iOS app checks if the the ESP32 is ready before sending another chunk. It works reasonably well, for my application, as I achieve 16-18kB/s. As mentioned above a chunk size of 514 only works with Arduino standard BLE library. For NimBLE I need to lower it to 512 or less for the callback to register the data package.

On the ESP32 side I use a 16 kB buffer in the callback which, when full is offloaded to an SD card and thereafter continues until all packages are received.

Heap memory seems fine but the stack HWM runs much lower on NimBLE than Arduino BLE.

I would really appreciate any help I could get :-)
And some code which I use for testing:
/*
  #include <BLEDevice.h>
  #include <BLEServer.h>
  #include <BLEUtils.h>
  #include <BLE2902.h>
*/
#include "NimBLEDevice.h"

#include <SPI.h>
#include "FS.h"
#include "FFat.h"
#include "SD.h"

File uploadFile;

BLEServer* pServer = NULL;
BLECharacteristic * pTransferCharacteristic;

bool      deviceConnected = false;
bool      oldDeviceConnected = false;
bool      sdCardOk      = false;

#define SERVICE_UUID                  "4FAFC201-1FB5-459E-8FCC-C5C9C331914B"
#define CHARACTERISTIC_TRANSFER_UUID  "62ec0272-3ec5-11eb-b378-0242ac130004"

bool downloadFlag = false;
int bufferIndex = 0;
int bufferCount = 0;
char dataBuffer[16448];

class MyServerCallbacks: public BLEServerCallbacks {
    void onConnect(BLEServer* pServer) {
      deviceConnected = true;
      Serial.println(F("*** Client connected"));
      Serial.print(F("Max heap size: "));
      Serial.print(ESP.getFreeHeap());
      Serial.print(F(" bytes, largest continous size: "));
      Serial.print(ESP.getMinFreeHeap());
      Serial.print(F(" bytes, "));
      Serial.printf("Stack HWM: %i \n", uxTaskGetStackHighWaterMark(NULL));
    }

    void onDisconnect(BLEServer* pServer) {
      deviceConnected = false;
      Serial.println(F("*** Client disconnected"));
    }
};

class transferCallback: public BLECharacteristicCallbacks
{
    void onWrite(BLECharacteristic *pCharacteristic)
    {
      std::string rxData = pCharacteristic->getValue();
      totBytesReceived += rxData.length();
      if (rxData.length() < 512)
      {
        Serial.print(F("Finished"));
      }
    }
};

class transferCallbackSD: public BLECharacteristicCallbacks
{
    void onWrite(BLECharacteristic *pCharacteristic)
    {
      std::string rxData = pCharacteristic->getValue();
      for (int i = 0; i < rxData.length(); i++)
      {
        dataBuffer[bufferIndex++] = rxData[i];
      }
      bufferCount++;
      if (!downloadFlag)
      {
        if (SD.exists("/temp.dat"))
        {
          Serial.println("File already exist");
          if (!SD.remove("/temp.dat"))
          {
            Serial.println("Failed to remove file");
          } else {
            Serial.println("File removed");
          }
        }
      }
      downloadFlag = true;
      if (bufferCount > 31 || rxData.length() < 512)
      {
        uploadFile = SD.open("/temp.dat", FILE_APPEND);
        if (uploadFile.write((uint8_t *) dataBuffer, bufferIndex))
        {
          Serial.println(bufferIndex);
          /*
            Serial.print(F("Max heap size: "));
            Serial.print(ESP.getFreeHeap());
            Serial.print(F(" bytes, largest continous size: "));
            Serial.print(ESP.getMinFreeHeap());
            Serial.print(F(" bytes, "));
            Serial.printf("Stack HWM: %i \n", uxTaskGetStackHighWaterMark(NULL));
          */
        } else {
          Serial.println(F("Write buffer to SD failed"));
        }
        uploadFile.flush();
        bufferIndex = 0;
        bufferCount = 0;
      }
      if (rxData.length() < 512)
      {
        Serial.print("Finished: ");
        Serial.println(uploadFile.size());
        downloadFlag = false;
        uploadFile.close();
      }
    }
};

void setup() {
  Serial.begin(115200);
  // SD card setup
  SPIClass SDSPI(VSPI);
  SDSPI.begin(SD_SCLK, SD_MISO, SD_MOSI, SD_CS);
  SD.begin();
  if (!SD.begin(SD_CS, SDSPI))
  {
    Serial.println(F("SDCard Mount Failed"));
    sdCardOk = false;
  } else {
    Serial.println(F("SDCard Mount PASS"));
    sdCardOk = true;
  }

  // 1. Create the BLE Device
  BLEDevice::init("BlipZero");
  BLEDevice::setMTU(517);

  // 2. Create the BLE server
  pServer = BLEDevice::createServer();
  pServer->setCallbacks(new MyServerCallbacks());

  // 3. Create BLE Service
  BLEService *pService = pServer->createService(SERVICE_UUID);

  // 4. Create BLE Characteristics inside the service(s)
  pTransferCharacteristic = pService->createCharacteristic(CHARACTERISTIC_TRANSFER_UUID,
                            NIMBLE_PROPERTY::WRITE_NR);
  //BLECharacteristic::PROPERTY_WRITE_NR);
  pTransferCharacteristic->setCallbacks(new transferCallbackSD());

  // 5. Start the service(s)
  pService->start();

  // 6. Start advertising
  pServer->getAdvertising()->addServiceUUID(pService->getUUID());
  pServer->getAdvertising()->start();
  BLEDevice::startAdvertising();
  Serial.println(F("Waiting a client connection to notify..."));
}

void loop() {
  if (!deviceConnected && oldDeviceConnected) {
    delay(100);
    pServer->startAdvertising();
    Serial.println(F("start advertising"));
    oldDeviceConnected = deviceConnected;
  }
  if (deviceConnected && !oldDeviceConnected) {
    oldDeviceConnected = deviceConnected;
  }
}

Hi @ClaesClaes, I'm not sure what bluedroid may be doing differently but as per the bluetooth spec:

The maximum length of an attribute value shall be 512 octets.

So that is the max length of the characteristic value.
The MTU is a different thing since a 3 byte header is prepended to the data (5 bytes for long write). That being said, if your phone has a max MTU of 514 then your max value data length would be 511 bytes, this should not be an issue for NimBLE as it will only reject the data if longer than 512.
My guess then is, if you are sending 514 (MTU) bytes from the phone, it might be trying to send via long write (since it needs room for the header) but your characteristic does not have NIMBLE::PROPERTY_WRITE which would be required for that to work. In that case the suggestion would be to reduce the chunk size sent from the phone to 511 to avoid this or, if possible, set the phone MTU to 515 and then you will be able to send 512bytes.

Heap memory seems fine but the stack HWM runs much lower on NimBLE than Arduino BLE.

NimBLE task stack only has 4096 bytes, not sure about bluedroid. If you're running into issues overflowing you can adjust the task stack size in nimconfig.h.

Hi @ClaesClaes, I'm not sure what bluedroid may be doing differently but as per the bluetooth spec:

The maximum length of an attribute value shall be 512 octets.

So that is the max length of the characteristic value.
The MTU is a different thing since a 3 byte header is prepended to the data (5 bytes for long write). That being said, if your phone has a max MTU of 514 then your max value data length would be 511 bytes, this should not be an issue for NimBLE as it will only reject the data if longer than 512.
My guess then is, if you are sending 514 (MTU) bytes from the phone, it might be trying to send via long write (since it needs room for the header) but your characteristic does not have NIMBLE::PROPERTY_WRITE which would be required for that to work. In that case the suggestion would be to reduce the chunk size sent from the phone to 511 to avoid this or, if possible, set the phone MTU to 515 and then you will be able to send 512bytes.

Heap memory seems fine but the stack HWM runs much lower on NimBLE than Arduino BLE.

NimBLE task stack only has 4096 bytes, not sure about bluedroid. If you're running into issues overflowing you can adjust the task stack size in nimconfig.h.

Thanks @h2zero, I really appreciate you quick reply. Thanks for the hint. I revisited the BLE specs (<IP_ADDRESS>) and indeed ATT MTU-3 it is. So for "write without response" I'm limited to 511 bytes payload with an MTU of 514. Thanks for pointing that out. Still I've no idea how Arduino BLE library does it, but it's solid every time I send out +100kB files using a 514 (or less) byte WRITE_NR transfer. Regardless it's not a big deal with the extra 3 bytes so I'm happy if I can at least get 511 bytes.
However, while 511 bytes did the trick (mostly) I still occasionally see dropped data using NimBLE (511 bytes payload, WRITE_NR), while Arduino BLE library never drops any data. How can I troubleshoot this? My guess would be that the peripheral respons to a WRITE_NR is sent out to the client before the callback has finished and therefore are not ready to receive a new package. Could this be the case?
The phones MTU is as far as I know not adjustable. It depends on the inner works of iOS CoreBluetooth. So I'm just a taker there. For writes with respons it gives me 512 and 514 for writes without response.
When using "WRITE" with response, NimBLE performs flawless, however the transfer rate drops as expected to half at around 7kB/s which is a tad too low for me.
Again grateful for any help I could get to get WRITE_NR to work stably.

Thanks @h2zero, I really appreciate you quick reply. Thanks for the hint. I revisited the BLE specs (<IP_ADDRESS>) and indeed ATT MTU-3 it is. So for "write without response" I'm limited to 511 bytes payload with an MTU of 514. Thanks for pointing that out. Still I've no idea how Arduino BLE library does it, but it's solid every time I send out +100kB files using a 514 (or less) byte WRITE_NR transfer. Regardless it's not a big deal with the extra 3 bytes so I'm happy if I can at least get 511 bytes.
However, while 511 bytes did the trick (mostly) I still occasionally see dropped data using NimBLE (511 bytes payload, WRITE_NR), while Arduino BLE library never drops any data. How can I troubleshoot this? My guess would be that the peripheral respons to a WRITE_NR is sent out to the client before the callback has finished and therefore are not ready to receive a new package. Could this be the case?
The phones MTU is as far as I know not adjustable. It depends on the inner works of iOS CoreBluetooth. So I'm just a taker there. For writes with respons it gives me 512 and 514 for writes without response.
When using "WRITE" with response, NimBLE performs flawless, however the transfer rate drops as expected to half at around 7kB/s which is a tad too low for me.
Again grateful for any help I could get to get WRITE_NR to work stably.

You're welocme @ClaesClaes .

My guess would be that the peripheral respons to a WRITE_NR is sent out to the client before the callback has finished and therefore are not ready to receive a new package. Could this be the case?

This is certainly possible, the write request is always responded to on the link layer which will tell your phone app that it's complete but that does no mean the peripheral is done processing the data. Since you are writing to SPIFFS in your callback that could be the cause, as that is a time consuming process. I would think the best approach would be a message queue and a separate task that does the SPIFFS write while the callback just copies the data into a buffer and sends a queue message.

You're welocme @ClaesClaes .

My guess would be that the peripheral respons to a WRITE_NR is sent out to the client before the callback has finished and therefore are not ready to receive a new package. Could this be the case?

This is certainly possible, the write request is always responded to on the link layer which will tell your phone app that it's complete but that does no mean the peripheral is done processing the data. Since you are writing to SPIFFS in your callback that could be the cause, as that is a time consuming process. I would think the best approach would be a message queue and a separate task that does the SPIFFS write while the callback just copies the data into a buffer and sends a queue message.

Thanks @h2zero,
I think I 've been thrown off the track by the fact that the same code produce different results depending upon which Arduino Library I use. Hence I thought there could be an issue on how callbacks are implemented.
However the real issue seems to be how I have interpreted the iOS CoreBluetooth framework function that supposedly should give a "go a head" signal when the peripheral is ready to receive another "write without response" data package.
Apple's documentation does not reveal any details but I now suspect that the function just monitors the internal iOS peripheral managers transmit buffer and calculates, based on the connection interval time, when the server "should" be ready. But that does not take into account the time the callback takes to complete.
I've read on a Nordic Dev forum that their own solution on how to speed up OTA DFU traffic using write without respons involves a function that sends a (n) number of packages then waits for package recept notification sent by the server after (n) packages been received/stored at the end of the callback. Sounds like a promising way forward.
Thanks for your time and patience :-)

I think I 've been thrown off the track by the fact that the same code produce different results depending upon which Arduino Library I use. Hence I thought there could be an issue on how callbacks are implemented.

This is possible due to internal BLE core handling of these packets. It's quite likely that NimBLE just responds to them sooner.
I suspect that IOS responds with a ready signal when the link layer packet confirmation and or packet sent confirmation message is received. As the nature of the write without response is "does not matter if it was received correctly" then you very well may send data before it the previous data was processed.

I've read on a Nordic Dev forum that their own solution on how to speed up OTA DFU traffic using write without respons involves a function that sends a (n) number of packages then waits for package recept notification sent by the server after (n) packages been received/stored at the end of the callback. Sounds like a promising way forward.

That seems like a reasonable way to do things, let me know what you come up with.

Finally, stability established. All packages arrives safely and with twice the speed (average 32kB/s) using a loop approach with WRITE_NR packages and leveraging iOS CoreBluetooth framework functions.
The stability comes from sending, in my case the sweet spot is 16 x 511 byte, WRITE_NR packages from my phone in a loop. Inside the loop I use the iOS CoreBluetooth functions to make sure the peripheral is ready. The iOS app then waits for the BLE server to send a specific ACK on a separate Characteristic which then triggers my iOS app to loop another batch of WRITE_NR packages until all packages are transfered.
The speed increase was mostly achieved by lowering the max connection intervall to 15 ms (lowest allowed by current iOS CoreBluetooth framework for iPhones). I've totally missed to set this in the BLE server as I assumed it was set automatically to the lowest at the time of the connection.

Nice, I'm glad it's working out! Thanks for reporting your solution.

I've totally missed to set this in the BLE server as I assumed it was set automatically to the lowest at the time of the connection.

The client controls the connection, no need to set/change anything on the server side.

The client controls the connection, no need to set/change anything on the server side.

Sorry I'm a bit confused I thought that the server (peripheral), not the client, set the boundaries for the connection intervall length?  I've set the max connection interval to 15 ms (12 x 1.25 ms) on the server after connection. This gives me 32kB/s using this:
(pServer->updateConnParams(desc->conn_handle, 9, 12, 0, 50);
If I omit this setting and use the library defaults, speed drops to 13-16 kB/s. What's the default min/max connection intervall?

...or are there no defaults and the client (the phone in my case) proposes the min/max intervalls during the connection interval, which the server accepts? If so I assume that iOS starts with a higher (probably 30 ms) connection interval and only goes down to the lowest (15 ms) after a request from the server (peripheral).

The intervals should be specified specifically by the application and if not I would assume iOS uses some default values. I have no idea what those would be, however, whatever they are the peripheral has no choice but to accept them or disconnect and NimBLE will accept any value provided if it is a legitimate value.

Ok, that might be it. -> iOS proposes something and in this case I tell it (via the server side setting) that I'd rather prefer a much shorter (15 ms) max interval. Anyhow all is well and good so a super thanks for your help in guiding me to a solution, it was super helpful of you.

My pleasure, quite an interesting topic as I have not done any iOS BLE applications. This will be a good reference for the others as well. Let me know if you have any other issues/questions.

Will do, thanks again.


var vs let?

I noticed this library is using var instead of let/const in some places. Is this intentional and how future code should be written as well?

totally intentional for performance reason

totally intentional for performance reason

Oh, I was not aware this was still a thing. Is it due to Node 10 support or is this also relevant for more recent V8 versions?

How does this performance reason look like under the hood?

I had not time to do a check lately, but I see very little reason that are worth the switch. const is fine to use.


JSpecify: initial checks for generic type compatibility at assignments

This pull request adds the identical type parameter nullability checks for both sides of an assignment.
for example
NullableTypeParam<@Nullable String> nullableTypeParam = new NullableTypeParam<String>(); should generate an error as the type parameters of both sides of the assignment do not have the same Nullability annotations.
This PR covers the following cases:

checks for the variable instantiation A<String> a = new A<@Nullable String>();
checks for the assignments
A<@Nullable String> t1 = new A<@Nullable String>(); A<String> t2; t2 = t1;
LHS and RHS with the exact same types
A<String> a = new A<@Nullable String>();
RHS being a subtype of LHS

class Test {
  class D<P extends @Nullable Object> {}
  class B<P extends @Nullable Object> extends D<P>{}
  void test1() {
      // BUG: Diagnostic contains: Cannot assign from type
      D<String> f = new B<@Nullable String>();
  }
}

nested generics
A<B<String>, String> a = new A<B<@Nullable String>>();

The PR does not handle pseudo-assignments due to parameter passing / returns; that support will come in a follow-up PR.  Also note that this PR only changes NullAway behavior when JSpecifyMode is enabled.

@lazaroclapp I have reviewed these changes separately before @NikitaAware opened this PR.  I think they are ready for you to take a look.

Question: Does this cause any issues with lambdas? For example:
Function<@Nullable Object,String> p = o -> (o != null) ? o.toString() : "default";

I don't expect it to run inference over the lambda expression at this point, but we should make sure it at least doesn't crash. We should probably add a test for that 😉

Good point!  I don't know what happens in these cases 🙂 @NikitaAware can we add tests for the case @lazaroclapp gives above.  Let's also do a method reference, like:
class Test {
  static String foo(@Nullable Object o) {
    return o != null ? o.toString() : "default";
  }
  static void bar() {
    Function<@Nullable Object,String> p = Test::foo;
  }
}

And also let's just test a simple diamond operator:
NullableTypeParam<@Nullable Object> t = new NullableTypeParam<>();

We obviously don't need to get the "right" answer for these cases yet, but we should not crash, and the test can document our current behavior.  I expect to see some false positive reports, and we'll have to think about what to do about those short of implementing full inference.

Question: Does this cause any issues with lambdas? For example:
Function<@Nullable Object,String> p = o -> (o != null) ? o.toString() : "default";

I don't expect it to run inference over the lambda expression at this point, but we should make sure it at least doesn't crash. We should probably add a test for that 😉

Good point! I don't know what happens in these cases 🙂 @NikitaAware can we add tests for the case @lazaroclapp gives above. Let's also do a method reference, like:
class Test {
  static String foo(@Nullable Object o) {
    return o != null ? o.toString() : "default";
  }
  static void bar() {
    Function<@Nullable Object,String> p = Test::foo;
  }
}

And also let's just test a simple diamond operator:
NullableTypeParam<@Nullable Object> t = new NullableTypeParam<>();

We obviously don't need to get the "right" answer for these cases yet, but we should not crash, and the test can document our current behavior. I expect to see some false positive reports, and we'll have to think about what to do about those short of implementing full inference.

Added a test but that does not generate any error. Please have a look at the test.

Small update: I chatted with @NikitaAware and she is going to re-structure the test code to more clearly show positive and negative tests for each scenario.  Once she finishes that, I will do a commit to fix formatting issues in the tests.  After that I will re-request a review @lazaroclapp

Update: I auto-formatted the generics test code with the help of the ErrorProneTestHelperSourceFormat check.  Unfortunately this change also re-formatted some of the existing test code so the change is not as clear anymore.  Basically, the new tests start from line 209 and changes before that line in the file are just formatting changes.  @lazaroclapp does this seem ok to you?  In terms of other things, I think @NikitaAware is still addressing other comments

Update: I auto-formatted the generics test code with the help of the ErrorProneTestHelperSourceFormat check. Unfortunately this change also re-formatted some of the existing test code so the change is not as clear anymore. Basically, the new tests start from line 209 and changes before that line in the file are just formatting changes. @lazaroclapp does this seem ok to you? In terms of other things, I think @NikitaAware is still addressing other comments

This is fine by me 🙂 As long as it's just that one test suite for now and not our other files. If it ever becomes an issue, we could easily make a single PR auto-formatting some/all tests and rebase this on that one, but for now let's just leave stuff as is in that regard.

I may tweak the lambda and diamond operator test cases to be a bit clearer

This is now done.  To summarize, we don't seem to crash for lambdas / method refs / diamond operators, but (unsurprisingly) we miss some bugs.

Cool! Should land automatically once CI passes. Again, thanks for all the great work @NikitaAware  🎉 !


DeltaAggregatedResources gRPC config stream to xds_cluster closed: 13,

Description:
Hi all.
Every 20-30 minutes my pods of envoy recreated. I saw logs
[2024-06-06 14:22:33.055][1][warning][main] [source/server/[server.cc:881](https://server.cc:881/)] caught ENVOY_SIGTERM
[2024-06-06 14:22:33.059][1][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:155] DeltaAggregatedResources gRPC config stream to xds_cluster closed: 13,

in istio we have keepaliveMaxServerConnectionAge parametrs. I do not find same in envoy gateway.

No, in events i saw that envoy gateway operator make rollout it
5m35s       Normal    EnsuringLoadBalancer     service/envoy-public-efa1f375                  Ensuring load balancer
5m38s       Normal    Scheduled                pod/envoy-public-efa1f375-76fcc4b64f-l6rtw     Successfully assigned envoy-gateway-system/envoy-public-efa1f375-76fcc4b64f-l6rtw to infra-eu-north1-a-upyc
5m38s       Normal    SuccessfulCreate         replicaset/envoy-public-efa1f375-76fcc4b64f    Created pod: envoy-public-efa1f375-76fcc4b64f-l6rtw
5m38s       Normal    ScalingReplicaSet        deployment/envoy-public-efa1f375               Scaled up replica set envoy-public-efa1f375-76fcc4b64f to 1 from 0
5m37s       Normal    Pulled                   pod/envoy-public-efa1f375-76fcc4b64f-l6rtw     Container image "cr.nemax.nebius.cloud/mirror/io/docker/envoyproxy/envoy:distroless-v1.29.2" already present on machine
5m37s       Normal    Created                  pod/envoy-public-efa1f375-76fcc4b64f-l6rtw     Created container envoy
5m37s       Normal    Started                  pod/envoy-public-efa1f375-76fcc4b64f-l6rtw     Started container envoy
5m37s       Normal    Pulled                   pod/envoy-public-efa1f375-76fcc4b64f-l6rtw     Container image "envoyproxy/gateway-dev:62ff3e7" already present on machine
5m37s       Normal    Created                  pod/envoy-public-efa1f375-76fcc4b64f-l6rtw     Created container shutdown-manager
5m37s       Normal    Started                  pod/envoy-public-efa1f375-76fcc4b64f-l6rtw     Started container shutdown-manager
5m36s       Normal    ScalingReplicaSet        deployment/envoy-public-efa1f375               Scaled down replica set envoy-public-efa1f375-76fcc4b64f to 0 from 1
5m36s       Normal    Created                  pod/envoy-public-efa1f375-7b499b6b85-xf642     Created container envoy
5m36s       Normal    Scheduled                pod/envoy-public-efa1f375-7b499b6b85-xf642     Successfully assigned envoy-gateway-system/envoy-public-efa1f375-7b499b6b85-xf642 to infra-eu-north1-c-afiw
5m36s       Normal    SuccessfulDelete         replicaset/envoy-public-efa1f375-76fcc4b64f    Deleted pod: envoy-public-efa1f375-76fcc4b64f-l6rtw
5m36s       Normal    ScalingReplicaSet        deployment/envoy-public-efa1f375               Scaled up replica set envoy-public-efa1f375-7b499b6b85 to 1 from 0
5m36s       Normal    Killing                  pod/envoy-public-efa1f375-76fcc4b64f-l6rtw     Stopping container shutdown-manager
5m36s       Warning   FailedToUpdateEndpoint   endpoints/envoy-public-efa1f375                Failed to update endpoint envoy-gateway-system/envoy-public-efa1f375: Operation cannot be fulfilled on endpoints "envoy-public-efa1f375": the object has been modified; please apply your changes to the latest version and try again
5m33s       Normal    EnsuredLoadBalancer      service/envoy-public-efa1f375                  Ensured load balancer
5m36s       Normal    ScalingReplicaSet        deployment/envoy-public-efa1f375               Scaled down replica set envoy-public-efa1f375-7b499b6b85 to 0 from 1
5m36s       Normal    SuccessfulDelete         replicaset/envoy-public-efa1f375-7b499b6b85    Deleted pod: envoy-public-efa1f375-7b499b6b85-xf642
5m36s       Normal    SuccessfulCreate         replicaset/envoy-public-efa1f375-7b499b6b85    Created pod: envoy-public-efa1f375-7b499b6b85-xf642
5m36s       Normal    Pulled                   pod/envoy-public-efa1f375-7b499b6b85-xf642     Container image "cr.nemax.nebius.cloud/mirror/io/docker/envoyproxy/envoy:distroless-v1.29.2" already present on machine
5m36s       Normal    Killing                  pod/envoy-public-efa1f375-76fcc4b64f-l6rtw     Stopping container envoy
5m35s       Normal    Pulled                   pod/envoy-public-efa1f375-7b499b6b85-xf642     Container image "envoyproxy/gateway-dev:62ff3e7" already present on machine
5m35s       Normal    Started                  pod/envoy-public-efa1f375-7b499b6b85-xf642     Started container envoy
5m35s       Normal    Killing                  pod/envoy-public-efa1f375-7b499b6b85-xf642     Stopping container shutdown-manager
5m35s       Normal    Started                  pod/envoy-public-efa1f375-7b499b6b85-xf642     Started container shutdown-manager
5m35s       Normal    Created                  pod/envoy-public-efa1f375-7b499b6b85-xf642     Created container shutdown-manager
5m35s       Normal    Killing                  pod/envoy-public-efa1f375-7b499b6b85-xf642     Stopping container envoy
2m7s        Normal    UpdatedLoadBalancer      service/envoy-public-efa1f375                  Updated load balancer with new hosts

envoy-public-efa1f375-549c4f4775    2         2         2       7d16h
envoy-public-efa1f375-557c79cd68    0         0         0       9h
envoy-public-efa1f375-76d66b5f58    0         0         0       2d9h
envoy-public-efa1f375-76fcc4b64f    0         0         0       14h
envoy-public-efa1f375-77f6977fc9    0         0         0       7d12h
envoy-public-efa1f375-7945d9b799    0         0         0       16h
envoy-public-efa1f375-7b499b6b85    0         0         0       7d23h
envoy-public-efa1f375-84656cc87b    0         0         0       7d23h
envoy-public-efa1f375-8478d6956c    0         0         0       33h
envoy-public-efa1f375-85b46d786c    0         0         0       7d12h
envoy-public-efa1f375-cb7f6954f     0         0         0       28h

but i did not find reason in logs envoy operator

looks like you are getting caught ENVOY_SIGTERM, are the liveness etc probes failing ?

it is strange situation
i have main gateway with configuration
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: public
spec:
  gatewayClassName: public
  listeners:
  - allowedRoutes:
      namespaces:
        from: All
    name: http
    port: 80
    protocol: HTTP
  - allowedRoutes:
      namespaces:
        from: All
    name: https
    port: 443
    protocol: TLS
    tls:
      mode: Passthroughh

and a lot tof gateway like this
spec:
  gatewayClassName: public
  listeners:
  - allowedRoutes:
      namespaces:
        from: All
    hostname: tracking.mlflow-test.cloud
    name:racking.mlflow-test.cloud
    port: 443
    protocol: HTTPS
    tls:
      certificateRefs:
      - group: ""
        kind: Secret
        name: cert-tracking-secret
      mode: Terminate

and saw how periodical;y changing rolling upgrade and changed ports
        - containerPort: 10443
          name: mlflow-cf75fb44
          protocol: TCP

        - containerPort: 10443
          name: envoy-e3314cb2
          protocol: TCP

kind: EnvoyProxy have option mergeGateways: true

hey @Uburro you're hitting https://github.com/envoyproxy/gateway/pull/3130, this should be part of the next patch release out soon

hey @Uburro you're hitting #3130, this should be part of the next patch release out soon

Thank you!

Should be fixed by #3130, close this one now. Reopen if got any issues.


fix: change from onUnmount to onScopeDispose hook

Replacing onUnmounted with  onScopeDispose hook will allow better integration and composability with 3rd party libraries that are creating a custom effect scope, that is not tied to the component lifetime (e.g. Pinia)
If you need more info, let me know - or check out the docs
I've marked this change as a fix, as this should be perfectly backwards compatible. Let me know if we also need to include a special test or sample for this functionality, and how would you want it to look like.

This seems to fail with nuxt2 - codesandbox.io/s/damianosipiuk-vue-query-jwysrg

@DamianOsipiuk Hello again - oops, I've tested only the basic example - Good catch :)
I've fixed the issues - hopefully :) .
I was able to run the nuxt2 exmaple you've provided when targetting the latest commit.
Also I've rolled back some changes that are not that important to reduce some overhead.
I've also illustrated the issue here The posts are reactive unless you hide/show them again, then they lose the reactivity (due to unmounted hook being called when they are hidden)
You cam switch the import to target this PR's version to see the difference

Hmm i think there are few things to notice

scope dispose work with vue2, but not with nuxt2 (nuxt2 composition API is lacking some features?). So detecting Vue version should be good enought.
you do not need pinia to use vue-query. Your state will be shared between components and requests deduplicated without additional boilerplate, just use useQuery. Pinia can still be used for global state. This is due to vue-query having its own state cache.

Good point with vue2 - I was investigating further and it seems the problem with not finding the onScopeDispose hook lies
 which was integrated in version 1.1.2. I've bumped the versions, and reverted some code to use the onScopeDispose exclusively, to not pollute the code unnecessarily.
Regarding the points mentioned later, I really appreciate the explanation.
I am aware that my example haven't actually described the reason why I am solving the issue with a centralized store for the sake of simplicity, but if interested, I'm very happy to share the problem and reasoning behind my will to use it like this.
I see vue-query (as well as react-query for that matter) to be a "reactive proxy of a remote state" - even though I am ware that you can use any fetcher, sync or async. The pinia or any other store implementation, tries to achieve a centralized single source of truth for an application state with an ability for a nice API (with computed properties and actions).
So, in my head, these two are solving different concerns (even-though, vue-query can make a lot of store-centric code obsolete), I am more interested in the behavior and the single-shared state which can "drive the queries".
If I need to have a single app-wide accessible state, using a store like pinia is a no-brainer, as it is pretty lightweight and provides an excellent dev-tools experience (as opposed of exposing home-brewed state, or using a custom injectable context)
Similarly, for async state, vue-query came as a huge thing (I've used with react-query before), especially with the dev tools as well.
Now for the example - it is kinda long, so tl;dr; I have a special case that would benefit from using vue-query within the store:
Imagine that I have a single state, that I want to be able to access through the whole app (to avoid prop drilling, etc...), that consists of multiple queries driven by a user input. some queries are dependent, some not, but in the end, all results are mapped into different getters (computed properties). This store also provides couple of behaviors that can invalidate multiple queries, and contain some business logic, Currently I am solving this issues by splitting it into the sync state (only user inputted values in my case) and a lot of custom hooks that are doing the "magic" - some of them are the aforementioned getters, some of them contain the business and mutation logic (there is also a possibility for one wrapping function, which would essentially give me the same interface as I am expecting from the store).
This solution, however, has a couple of issues:

It's a bit harder to find and reason about the actual state, as now it is split between vue-query, custom hooks (without dev tools insight), store and individual components - you see this can get overwhelming quickly
when I combine the results of a number of queries, the resulting computed state can get a little too big and can cause performance issues when used in a huge number of components (while the vue-query results are coming from the cache, the projections are always recreated).

You see, how both of these issues can be easily solved by embedding the queries in the store itself.
I am also aware that I can actually do multiple requests and the projections in the fetcher, and let the cache address the "performance" issues, but, then I would not be able to use vue-query for the individual queries - unless I will do some custom work with the query client.
Anyway, I am really thankful that you are considering this suggestion.

Hi, sorry for the delay 🙄
Thanks for the contribution 🎉

No worries, I understand that this is an open source project and our time is limited. Thank you as well - if you find any issues - let me know :)


Bump Dependencies #8868445967

Bumps dependencies for product-is. Link : https://github.com/wso2/product-is/actions/runs/8868445967

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Fix typos in Spanish translation

Fix typos I've found in the Spanish version of the course 👀

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

Thanks for the review, done @lewtun !


Appstudio update test-component-pac-sulw

Pipelines as Code configuration proposal

Pipelines as Code CI/test-component-pac-sulw-on-pull-request has successfully validated your commit.

StatusDurationName

✅ Succeeded
8 seconds
init

✅ Succeeded
20 seconds
clone-repository

✅ Succeeded
1 minute
build-container

✅ Succeeded
22 seconds
sanity-inspect-image

✅ Succeeded
23 seconds
deprecated-base-image-check

✅ Succeeded
47 seconds
clamav-scan

✅ Succeeded
3 minutes
clair-scan

✅ Succeeded
18 seconds
sbom-json-check

✅ Succeeded
14 seconds
sanity-optional-label-check

✅ Succeeded
14 seconds
sanity-label-check

✅ Succeeded
9 seconds
show-summary

Pipelines as Code CI/test-component-pac-sulw-on-pull-request has successfully validated your commit.

StatusDurationName

✅ Succeeded
7 seconds
init

✅ Succeeded
16 seconds
clone-repository

✅ Succeeded
31 seconds
build-container

✅ Succeeded
11 seconds
deprecated-base-image-check

✅ Succeeded
11 seconds
sanity-inspect-image

✅ Succeeded
10 seconds
clair-scan

✅ Succeeded
31 seconds
clamav-scan

✅ Succeeded
9 seconds
sbom-json-check

✅ Succeeded
9 seconds
sanity-optional-label-check

✅ Succeeded
9 seconds
sanity-label-check

✅ Succeeded
5 seconds
show-summary


Undocumented not allowed parameter for functions

Describe the bug

If you write a function that accepts as a query parameter code it will throw a 500 http error.
This error can't be reproduced local so I think it's an issue in the static web apps repo. It only happens after deployment.
You can see the issue and the fix in this Commit.
To Reproduce
Steps to reproduce the behavior:

Create with dotnet a new function project (I used dotnet 6)
In the default template of a http function there is requesting a name with the query parameter name
change this parameter to code
deploy
try to call the function with the query parameter code
Get an 500 HTTP error

Expected behavior
The would be no error or it would be at least documented. It would be also nice if the server error would return a message like: it's not allowed to use a query parameter ``code``

Repeat of this longstanding issue, please use that issue for tracking: https://github.com/Azure/static-web-apps/issues/165


[unit-testing] Prevent autostubbing for some packages

We want to use some package to define classes we want to unit-test with jasmine.
 MyService = Class.extend({....})

Die Class function is coming from some external package, which during tests is stubbed.
Is there a way to exclude some package from being auto-stubbed?

In the latest release there is an option. See last bullet point of https://github.com/Sanjo/meteor-jasmine/releases/tag/0.12.0.


convert modelchain.ipynb to rst, remove nbsphinx

[x] Closes #567
[x] I am familiar with the contributing guidelines.
[ ] Fully tested. Added and/or modified tests to ensure correct behavior for all reasonable inputs. Tests (usually) must pass on the TravisCI and Appveyor testing services.
[x] Updates entries to docs/sphinx/source/api.rst for API changes.
[ ] Adds description and name entries in the appropriate docs/sphinx/source/whatsnew file for all changes.
[ ] Code quality and style is sufficient. Passes LGTM and SticklerCI checks.
[x] New code is fully documented. Includes sphinx/numpydoc compliant docstrings and comments in the code where necessary.
[x] Pull request is nearly complete and ready for detailed review.

New documentation using the new modelchain.rst file:
https://wholmgren-pvlib-python-new.readthedocs.io/en/mcdocclean/modelchain.html
Last successful build on pvlib/master using the old modelchain.ipynb file:
https://pvlib-python.readthedocs.io/en/latest/modelchain.html

Does this also drop the jupyter requirement too? If so it will be a huge relief for this of us running headless, no gui, eg as in Windows Subsystem for Linux, two less requirements to install that come with a lot of overhead, do thanks! :smile:

@mikofski I believe we can drop jupyter. The docs are now mostly building without jupyter present. For some reason the ipython directive no longer executes the code in the pvsystem.rst and timetimezones.rst files. I think it's unrelated but not 100% sure.
I am a bit confused as to why you're concerned about a jupyter requirement for the doc builds but also in favor of adding scipy to the base requirements. Feels like I'm missing something.

Well scipy can be used for many things other than pvlib, whereas Jupyter is mainly used as a graphical user interface, which is pointless in a headless environment. But it doesn't really matter, it's more just the principle I guess.

I cannot reliably reproduce the ipython execution failures locally or on rtd. I'll merge and hope for the best. At least the docs should build again.

Aaaaand the docs build but fail to execute the ipython directive blocks in pvsystem.rst and timetimezones.rst. :(


Library loader for mac doesn't always work

MachOLibraryLoader.find_library in lib.py has the following line:
if '.' not in libname:
Loading of ffmpeg library right now involves passing 'avbin' and 'avbin.56' to the load library function, which calls this find_library. If the avbin.56 is passed, we don't properly search for 'libavbin.56.dylib'.
I think the line needs to be:
if '.dylib' not in libname:

Hi Paul,
I had a look at the code, and I think your fix should be fine. The intention is just to catch "name.dynlib" files, but the original check did not account for names like "avbin.56".

Thanks Paul, I merged in your PR.


chore: getEnrollmentsByCourse is useless

[ ] Replace call to getEnrollmentsByCourse with call to getEnrollmentsWithActivity.
[ ] Delete getEnrollmentsByCourse
[ ] Rename getEnrollmentsWithActivity to getEnrollmentsByCourse (double check that this makes sense.)

This does nothing:
// getEnrollmentsByCourse returns all enrollments for a course that match the given enrollment request.
func (s *QuickFeedService) getEnrollmentsByCourse(request *qf.EnrollmentRequest) ([]*qf.Enrollment, error) {
	enrollments, err := s.getEnrollmentsWithActivity(request.GetCourseID())
	if err != nil {
		return nil, err
	}
	return enrollments, nil
}

@JosteinLindhom I looked into this. The logic performed by getEnrollmentsWithActivity is a bit sophisticated in that

fetch all submissions for the course;
finds all enrollments and inspects each submission that belongs to that enrollee (user)
based on the submissions, update the enrollment's TotalApproved and LastActivityDate
fetch all pending enrollments
the returned list of enrollments includes those with activity (submissions) and those with pending status

The above is based only on the courseID from the frontend, because of this:
	case *qf.EnrollmentRequest_CourseID:
		enrollments, err = s.getEnrollmentsWithActivity(in.Msg.GetCourseID())

(that is the EnrollmentRequest.Statuses field is ignored.)
However, the frontend makes calls like this:
            const statuses = isStudent(enrollment) ? [Enrollment_UserStatus.STUDENT, Enrollment_UserStatus.TEACHER] : []
            results.push(actions.getEnrollmentsByCourse({ courseID, statuses }))

        await actions.getEnrollmentsByCourse({ courseID: state.activeCourse, statuses: [Enrollment_UserStatus.PENDING] })

Thus, these "statuses" fields are ignored by the server.


PhotoInfo.export should default to original filename, not current filename

This would mirror the behavior of Photos and, with Photos 5 name mangling, likely be what the user expects

What would be required to add support for this? I was looking to use this in my own backup script; perhaps I can contribute.

I think all that's needed is just to replace self.filename with self.original_filename in these lines.  The test suite will likely need to be updated too --  running pytest will identify which tests, if any, are testing for the default exported file name.  This is a really easy fix, just haven't gotten to it as I already work around in the osxphotos command line and have been busy on other fixes.
https://github.com/RhetTbull/osxphotos/blob/595307a003c8ae5d3bee3ad161bb880d884b3cc3/osxphotos/photoinfo/_photoinfo_export.py#L616-L618

Thanks @AaronVanGeffen for implementing this!


Ask for prompt to generate cartoon characters of videomamba

Dear authors,
I would like to ask a question other than the model, do you remember what your prompt was when generating videomamba cartoon characters? I think the image of videomamba is very cute, thank you!

HaHa! Thank you so much :)


remove js_settop from mujs.h because it has no implemtation.

I found js_settop has no implemtation, use js_pop instead of js_settop，so remove js_settop from mujs.h?

Yeah. I never found myself needing it, so removing it is fine.


Package fails on Julia 1.0.5 (LTS)

with error ERROR: LoadError: UndefVarError: isnothing not defined

When checking benchmarks with 1.0.5 I noticed the problem and thought I'd better let you know.


Starting up stops at "listening on: <IP_ADDRESS>:8080"

I'm following the instructions for the setup and it starts up fine but it just stops at listening on: <IP_ADDRESS>:8080. The relay is not available probably because the other expected logs are not popping up and it's in fact not listening/ready yet.  There is nothing in the logs (posted them below). The same thing happens when I build from source and deploy without containers.
root@cluster:~/nostr-rs-relay# podman run -it --rm -p 7000:8080   --user=100:100   -v $(pwd)/data:/usr/src/app/db:Z   -v $(pwd)/config.toml:/usr/src/app/config.toml:ro,Z   --name nostr-relay nostr-rs-relay:latest
Jan 28 18:41:50.244  INFO nostr_rs_relay: Starting up from main
Jan 28 18:41:50.245  INFO nostr_rs_relay::server: listening on: <IP_ADDRESS>:8080

Jan 28 19:20:43.032  INFO nostr_rs_relay: Starting up from main
Jan 28 19:20:43.033  INFO nostr_rs_relay::server: listening on: <IP_ADDRESS>:8080
Jan 28 19:20:43.036 DEBUG nostr_rs_relay::repo::sqlite: Database reader pool is waiting on the database to be created...
Jan 28 19:20:43.536 DEBUG nostr_rs_relay::repo::sqlite: Database reader pool is waiting on the database to be created...
Jan 28 19:20:44.036 DEBUG nostr_rs_relay::repo::sqlite: Database reader pool is waiting on the database to be created...
Jan 28 19:20:44.536 DEBUG nostr_rs_relay::repo::sqlite: Database reader pool is waiting on the database to be created...

That message comes over and over

If the database exists but cannot be written, the relay will error out. If the database file is not present, and the writer fails to create it, you'll see this string of messages.
Looks like the root cause is the inability to create the database.  I'm going to work on a better error message when the database cannot be created.
What operating system / container runtime are you using?

I've tried on linux and MacOS. Not sure what you mean with container runtime. I'm using podman as suggested.

Just pushed commit bdd4e43, which changes the order of how database pools are created.  This should fix the problem you found.  Looks like it was introduced pretty recently with the reshuffling of database code.
Thanks for the report; let me know if the latest revision fixes this issue for you.

Tried the new commit. Now I'm getting this error:
Jan 28 20:29:50.596  INFO nostr_rs_relay: Starting up from main
Jan 28 20:29:50.598  INFO nostr_rs_relay::server: listening on: <IP_ADDRESS>:8080
Jan 28 20:29:50.602 ERROR r2d2: unable to open database file: /usr/src/app/db/nostr.db
Jan 28 20:29:51.002 ERROR r2d2: unable to open database file: /usr/src/app/db/nostr.db
Jan 28 20:29:51.802 ERROR r2d2: unable to open database file: /usr/src/app/db/nostr.db

ok, I was just able to duplicate this problem while running as root.  Works as a normal user though (using unshare).

Hmm ok. So I should try using a different user then. Doing that, I again run into this issue though Error: please use unshare with rootless

I'd recommend running as a normal user, instead of root.  You can add the option "--user=0:0" to podman run, and then it should be able to write to the local directory root-owned directory you are mapping into the container.
Or, you can use some other user account (instead of UID zero).  Make a new user, ensure they own the data directory and config.toml, and then pass that account's UID/GID (--user=U:G).

You get the "user unshare with rootless" error when you are running as a non-root user?

yup, I also can't start a container because podman thinks my user doesn't own the directory for some reason. Looks like i have some debugging to do. I will try to setup the vm from scratch again and see what happens. I think something with the user creation was not done properly on my side.
I got it working as root now so I know the process now. Thanks very much for your help, much appreciated.

Good to hear, thanks for opening the issue, that was a good bug to squash early.


Serialization error on lock history

having a bit of trouble with the get_lock_history function
for example, trying the following:
eh = extensions.ExtensionsHandler()
eh.load_defined(lock.extensions)
response, data = chaster_api.get_lock_history(lock._id, eh.wheel_of_fortunes[0]._id)
outputs: {'results': [], 'hasMore': False, 'count': 0}
despite having spun the wheel twice
when I instead try: response, data = chaster_api.get_lock_history(lock._id)
it will error with: AttributeError 'types.SimpleNamespace' object has no attribute 'user'

duplicate


[Recipe Request]: Padlet Integration

Preflight Checklist

[X] I have read the Contributing Guidelines for this project.
[X] I agree to follow the Code of Conduct that this project adheres to.
[X] I have searched the issue tracker for a feature request that matches the one I want to file, without success.

Name of the service
Padlet
URL of the service
https://padlet.com/
GitHub URL to a recipe that has already been created by someone else or you
https://github.com/bfsinger/ferdiumpadlet
Features you want implemented
Hi all,
I'm trying to create a Padlet service for Ferdium. Padlet is a cool app that is very flexible, allowing the user to create 'boards' where anyone can post information to. Ideally, I'd like to use Padlet in Ferdium and receive notifications when someone posts to one of the Padlet boards.
I haven't been able to find a recipe for Padlet+Franz on Github. Based on the instructions, I've created a Padlet recipe for Ferdium (hopefully I've uploaded it to Github correctly). The app seems to load correctly in Ferdium. What is not working are notifications of posts to Padlet boards. I'm not sure I have the correct code in the webview.js file.
Also to note, my university has a special Padlet link (https://uofsussex.padlet.org/). Instead, I used https://padlet.com/ in the Github upload, to make the code more general.
Any help is appreciated regarding the notifications; thanks!

@bfsinger - please create a PR as opposed to an issue for this. Thanks!


How to use Dragonfly with CRI-O?

Question

I want to use Dragonfly in my k8s cluster,which used CRI-O  container engine:
NAME      STATUS   ROLES    AGE   VERSION   INTERNAL-IP     EXTERNAL-IP   OS-IMAGE                KERNEL-VERSION              CONTAINER-RUNTIME
node1      Ready    <none>   21h   v1.16.0      xxx        <none>        CentOS Linux 7 (Core)   5.7.7-1.el7.elrepo.x86_64   cri-o://1.16.6

I tried to modify some configurations, but when I pulled the image, it did not use dragonfly.
Where is the problem？
I want to pull images from harbor and docker.io,quay.mirrors.ustc.edu.cn,hub-mirror.c.163.com.etc
dfdaemon.yml
apiVersion: apps/v1
kind: DaemonSet
metadata:
  labels:
    k8s-app: dfdaemon
  name: dfdaemon
  namespace: kube-system
spec:
  selector:
    matchLabels:
      app: dfdaemon
  template:
    metadata:
      labels:
        app: dfdaemon
    spec:
      containers:
      - name: dfdaemon
        image: dragonflyoss/dfclient:1.0.3
        imagePullPolicy: IfNotPresent
        resources:
          requests:
            cpu: 250m
        volumeMounts:
        - mountPath: /etc/dragonfly/dfdaemon.yml
          subPath: dfdaemon.yml
          name: dfconf
        - mountPath: /root/.small-dragonfly
          name: data
        - mountPath: /certs
          name: certs
        - mountPath: /etc/localtime
          name: localtime
      hostNetwork: true
      dnsPolicy: ClusterFirstWithHostNet
      restartPolicy: Always
      tolerations:
      - effect: NoExecute
        operator: Exists
      - effect: NoSchedule
        operator: Exists
      volumes:
      - name: dfconf
        configMap:
          name: dfdaemon-conf
          defaultMode: 384
      - name: data
        hostPath:
          path: /home/eecdn/.small-dragonfly
          type: DirectoryOrCreate
      - name: certs
        hostPath:
           path: /data/dfcert
           type: DirectoryOrCreate
      - name: localtime
        hostPath:
          path: /etc/localtime

---
apiVersion: v1
kind: ConfigMap
metadata:
  labels:
    k8s-app: dfdaemon
  name: dfdaemon-conf
  namespace: kube-system
data:
  dfdaemon.yml: |-
    # proxy all http image layer download requests with dfget
    proxies:
      - regx: blobs/sha256.*
    hijack_https:
      cert: /certs/df.crt
      key: /certs/df.key
      hosts:
        - regx: myharbor.test.com:1443
          insecure: true
    supernodes:
      - supernode-ip

/etc/containers/registries.conf : I added [registries.insecure] and [registries.mirror] as following
(referring this issue: https://github.com/cri-o/cri-o/issues/372)
unqualified-search-registries = ['myharbor.test.com:1443','hub-mirror.c.163.com','docker.io','quay.mirrors.ustc.edu.cn', 'registry.access.redhat.com', 'registry.fedoraproject.org', 'quay.io', 'registry.centos.org']

[[registry]]
prefix = "myharbor.test.com:1443"
insecure = true
blocked = false
location = "localhost:65001"

[[registry]]
prefix = "docker.io/library"
insecure = true
blocked = false
location = "localhost:65001/library"

[[registry.mirror]]
location = "localhost:65001"
insecure = true

/usr/lib/systemd/system/crio.service : I added
Environment="HTTP_PROXY=http://<IP_ADDRESS>:65001"
Environment="HTTPS_PROXY=http://<IP_ADDRESS>:65001"

When I execute "crictl pull busybox",I can see  that there are packets passing <IP_ADDRESS>:65001,which means  http_proxy is working:
[ecf@node1 ~]$ sudo tcpdump -i lo -nn -A port 65001
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on lo, link-type EN10MB (Ethernet), capture size 262144 bytes
17:06:22.484033 IP <IP_ADDRESS>.50746 > <IP_ADDRESS>.65001: Flags [S], seq 239256780, win 65495, options [mss 65495,sackOK,TS val<PHONE_NUMBER> ecr 0,nop,wscale 7], length 0
E..<.>@.@.#|.........:...B...........0.........
..Y.........
17:06:22.484058 IP <IP_ADDRESS>.65001 > <IP_ADDRESS>.50746: Flags [S.], seq<PHONE_NUMBER>, ack 239256781, win 65483, options [mss 65495,sackOK,TS val<PHONE_NUMBER> ecr<PHONE_NUMBER>,nop,wscale 7], length 0
E..<..@.@.<............:~....B.......0.........
..Y...Y.....
17:06:22.484079 IP <IP_ADDRESS>.50746 > <IP_ADDRESS>.65001: Flags [.], ack 1, win 512, options [nop,nop,TS val<PHONE_NUMBER> ecr<PHONE_NUMBER>], length 0
E..4.?@.@.#..........:...B..~........(.....
..Y...Y.
17:06:22.484220 IP <IP_ADDRESS>.50746 > <IP_ADDRESS>.65001: Flags [P.], seq 1:110, ack 1, win 512, options [nop,nop,TS val<PHONE_NUMBER> ecr<PHONE_NUMBER>], length 109
E....@@.@.#..........:...B..~..............
..Y...Y.CONNECT hub-mirror.c.163.com:443 HTTP/1.1
Host: hub-mirror.c.163.com:443
User-Agent: Go-http-client/1.1

Please, exec into dfclient container, cat /etc/dragonfly/dfdaemon.yml

Please, exec into dfclient container, cat /etc/dragonfly/dfdaemon.yml

dfdaemon.yml: |-
# proxy all http image layer download requests with dfget
proxies:
- regx: blobs/sha256.*
hijack_https:
cert: /certs/df.crt
key: /certs/df.key
hosts:
- regx: myharbor.test.com:1443
insecure: true
supernodes:
- supernode-ip

Enable verbose like this, and pull image again
proxies:
- regx: blobs/sha256.*
dfget_flags:
- "--node"
- "<IP_ADDRESS>"
# add this line
verbose: true


Cannot re-use permalinks of existing/archived pages

Describe the Bug
In https://admin.integreat-app.de/bochum/pages/de/1238/edit/ I try to change /informationen-zum-krieg-in-der-ukraine-2/ to /informationen-zum-krieg-in-der-ukraine/
It is not possible and always jumps back to /informationen-zum-krieg-in-der-ukraine-2/ even i have renamed the other page which was orginally named /informationen-zum-krieg-in-der-ukraine/

This is the expected behavior of a permalink. The old link continues to direct to the old page (and this "blocks" the link for other pages), even after the link is changed.
If you don't want this behavior, we can change it, but still this would be a feature request and no bug. Keep in mind that this would imply that all of the old links won't continue to work when you edit them.
Please discuss this in the service team if you are really sure.

I don't think that a permalink should be changeable. It would not be a permalink anymore.

And to answer Daniel's question: you can change the link when the old page is being removed and the permalink is "free" again.

Another possibility would be to keep the permalink behavior as it is, but in the page form next to the link we provide a button "delete permalink history" to free a specific link for other pages without deleting the whole page...
Not sure though if this creates more problems than it solves... but maybe for expert users or staff members it might be helpful?

We will discuss in on Monday in our serviceteam call.
After duplicating the deutschlandvorlage or another city, permalink changes are necessary.

Due to the fact that i cannot reopen issues here, i would create a new one if we come to conclusion that it is necessary

Due to the fact that i cannot reopen issues here, i would create a new one if we come to conclusion that it is necessary

Deliberately creating duplicate issues is always a bad idea. Please just leave a comment with your conclusion and we can reopen the ticket.
However, I also invited you to the digitalfabrik orga, this hopefully gives you permission to reopen issues.

It is possible to change perma links. But you're not allowed to use already used permalinks.
Also there is the option that we do change permalinks in the database as application admins.

Service team answer: Removing the "unprofessional characters" from the permalinks is more important than the permalink history. Question in that case: Can we remove the permalink history our does that also affect internal links in cases of permalink-changes?

You can change the permalink to anything that does not collide with existing permalinks. For example you could simply change the slug to informationen-krieg-in-der-ukraine or information-zum-krieg-in-der-ukraine which does not really take anything away from from the readability. This just needs a little creativity instead of just removing the last characters.
You can simply delete the page that causes the collision. This will allow you to re-use the slug.

The permalink history is the problem here in combination with the fact that the uniquess is now defined by only one part of the slug (not the whole slug as it was before). WordPress was completly fine. The requirements haven't changed. It wasn't improved with Django, it's now worse.
"Collide with existing links" is problematic because an existing link is everything what has existed in past at single pages that are not deleted.
From the experience of the first 3 month it is not intuitive to our users. Some cities asked us how to change it but it was even to complex to explain for us (although we don't want to explain at all)

"To improve the functionality, we could add a message that explains which page has a colliding slug and maybe even link to it." would be a good idea if the permalink history can also be looked up or is explained somewhere. We have dropped of the trainings for the system because the plan of the migration was that it all becomes easier and more self-explaining.

I do think there are some misconceptions about the idea behind the permalinks. The cause for creating permalinks is that they do not change, because websites etc link to the pages. Humans are usually bad with maintaining these links over a long time. Breaking links are bad for at least 2 reasons: a) It impacts search engine scores because they at least somewhat depend on the number of links to a page. b) Users cannot follow links.
Now in our case there are quite often huge changes in structure and therefore changes in the page path. This ensures that links still work after changing the structure. The rest is only for readability
I'm also not convinced that the trailing number is actually a problem. There are tons of pages that by default without collisions add some non-readable parts to the end of the path, for example heise.de or theregister.co.uk. Also, the Google link guide is also totally aligned with this behavior (https://developers.google.com/search/docs/advanced/guidelines/url-structure).

would be a good idea if the permalink history can also be looked up or is explained somewhere. We have dropped of the trainings for the system because the plan of the migration was that it all becomes easier and more self-explaining.

Well, IMHO it is. The permalinks work quite well. The system will automatically correct them if they're already used.

After changing the scope of the issue (only inform about colliding permalinks instead of allowing the user to change them), this is now a duplicate of #1521.


Contracts v3

This epic collects issues scheduled for contracts v3.

Done. Waiting for tests to come back positive. All tests will be done on 24.08.2023 but the L1 invalidation tests will be done on 15.08.2023.


Remove test keys as defaults

Change-type: patch
Signed-off-by: Josh Bowling<EMAIL_ADDRESS>

@balena-ci I self certify!


NPM High vulnerability issue

Hi,
I run this community-server version in one of my university servers and the security team always flags the NPM high vulnerability issue which arises during the NPM audit. Is there any fix available?

I have updated the main branch to fix one vulnerability. All other vulnerabilities we can't fix though since they are not direct dependencies. They all are a result of the manual-git-changelog dependency. So what you can do is remove that dependency from the package.json (since it's not needed to run the server anyway). If you do an install then there should be 0 vulnerabilities.

Ah cool! That fixed the problem. Thanks!!


'guard' is never reassigned. Use 'const' instead.

Proposed changes
This pull request addresses the prefer-const issue identified in our codebase. The change involves modifying the guard variable declaration from let to const in the authentication loop, as this variable is never reassigned. This change aligns with the global standard "prefer-const" of using const for variables that will not be mutated later.
Types of changes
What types of changes does your code introduce?
Put an x in the boxes that apply

[x] Bugfix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist
Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code.

[x] I have read the CONTRIBUTING doc
[x] Lint and unit tests pass locally with my changes
[x] I have added tests that prove my fix is effective or that my feature works.
[x] I have added necessary documentation (if appropriate)

Further comments
The prefer-const rule is a widely accepted best practice in JavaScript and TypeScript development. It helps to prevent unintentional mutation of variables and makes the code easier to reason about. By making this change, we ensure that our code adheres to this best practice, making it more robust and maintainable.

Presets are at https://github.com/adonisjs/presets


Fix off-by-one error in #scan

We noticed that #scan was

returning duplicate values where a chunk's endpoint index is the same as the next chunk's starting index point, and
When determining to end or continue the chunking, matches weren't accounted for; instead the size check was always on the entire dataset instead of the match value.

rebased in #170


Support for Time Picker

Here is a suggestion to please add support for time picker. In some situations we might want user to enter date and time so instead of having two different plugins, what if  could also add optional dropdowns for time selection?

This is definitely on the radar - once the datepicker component is solidified I'll be looking at adding timepicker functionality as well as date range selection