PostgreSQL replication with repmgr: hot_standby mode not enabled

Describe the bug
Hot standby mode is not enabled on a PostgreSQL replica on RedHat environemnts despite running a task that should enable it.
To Reproduce
Steps to reproduce the behavior:

Deploy a cluster with 2 PostgreSQL vms

Expected behavior
Hot standby mode is enabled on a PostgreSQL replica node.
Actual behavior
The log file looks good but there is no effect in the configuration file.
2020-05-14T07:26:58.9085515Z 07:26:58 INFO cli.engine.ansible.AnsibleCommand - TASK [postgresql : Turn on hot_standby] ****************************************
2020-05-14T07:26:58.9316048Z 07:26:58 INFO cli.engine.ansible.AnsibleCommand - skipping: [ec2-xx-xx-xx-xx.eu-west-3.compute.amazonaws.com]
2020-05-14T07:26:59.4780903Z 07:26:59 INFO cli.engine.ansible.AnsibleCommand - changed: [ec2-yy-yy-yy-yy.eu-west-3.compute.amazonaws.com]

[root@ec2-yy-yy-yy-yy ~]# cat /var/lib/pgsql/10/data/postgresql.conf | grep hot
#old_snapshot_threshold = -1            # 1min-60d; -1 disables; 0 is immediate
#hot_standby = on                       # "off" disallows queries during recovery
#hot_standby_feedback = off             # send info from standby to prevent

Config files
Configuration that should be included in the yaml file:
...
    postgresql:
      count: 2
      machine: postgresql-machine-rhel
...
---
kind: infrastructure/virtual-machine
name: postgresql-machine-rhel
provider: aws
based_on: postgresql-machine
specification:
  os_full_name: RHEL-7.8_HVM_GA-20200225-x86_64-1-Hourly2-GP2
---
kind: configuration/postgresql
name: default
provider: aws
specification:
  extensions:
    pgaudit:
      enabled: yes
    pgbouncer:
      enabled: yes
    replication:
      enabled: yes
      use_repmgr: yes
title: Postgresql

OS (please complete the following information):

OS: [RHEL]

Cloud Environment (please complete the following information):

Cloud Provider [all]

I just validate the described issue, and it looks that hot_standby option is working as expected.
I suppose, that above grep command executed on PostgreSQL config file, was performed on master server, and this behavior is valid, as hot_standby option should be activated only on slave host. Master host should have this option disabled.
The hot_standby mode support replication in ReadOnly mode, what means, that regular users are not able to Write anything into slave host.
Please check bellow example which prove that hot_standby option is working as expected:
**MASTER HOST:**
[postgres@pll-gdajukk-postgresql-vm-0 ~]$ grep hot_stand /etc/postgresql/10/main/postgresql.conf
#hot_standby = on			# "off" disallows queries during recovery
#hot_standby_feedback = off		# send info from standby to prevent
[postgres@pll-gdajukk-postgresql-vm-0 ~]$
[postgres@pll-gdajukk-postgresql-vm-0 ~]$ repmgr cluster show -f /etc/postgresql/10/main/repmgr.conf
 ID | Name                        | Role    | Status    | Upstream                    | Location | Connection string
----+-----------------------------+---------+-----------+-----------------------------+----------+-------------------------------------------------------------------
 1  | pll-gdajukk-postgresql-vm-0 | primary | * running |                             | default  | host=<IP_ADDRESS> user=epi_repmgr dbname=epi_repmgr connect_timeout=2
 2  | pll-gdajukk-postgresql-vm-1 | standby |   running | pll-gdajukk-postgresql-vm-0 | default  | host=<IP_ADDRESS> user=epi_repmgr dbname=epi_repmgr connect_timeout=2
[postgres@pll-gdajukk-postgresql-vm-0 ~]$
[postgres@pll-gdajukk-postgresql-vm-0 ~]$ psql -c "select pg_is_in_recovery();"
 pg_is_in_recovery
-------------------
 f
(1 row)

[postgres@pll-gdajukk-postgresql-vm-0 ~]$

**SLAVE HOST:**
[postgres@pll-gdajukk-postgresql-vm-1 ~]$ grep hot_stand /etc/postgresql/10/main/postgresql.conf
hot_standby = on			# "off" disallows queries during recovery
#hot_standby_feedback = off		# send info from standby to prevent
[postgres@pll-gdajukk-postgresql-vm-1 ~]$ repmgr cluster show -f /etc/postgresql/10/main/repmgr.conf
 ID | Name                        | Role    | Status    | Upstream                    | Location | Connection string
----+-----------------------------+---------+-----------+-----------------------------+----------+-------------------------------------------------------------------
 1  | pll-gdajukk-postgresql-vm-0 | primary | * running |                             | default  | host=<IP_ADDRESS> user=epi_repmgr dbname=epi_repmgr connect_timeout=2
 2  | pll-gdajukk-postgresql-vm-1 | standby |   running | pll-gdajukk-postgresql-vm-0 | default  | host=<IP_ADDRESS> user=epi_repmgr dbname=epi_repmgr connect_timeout=2
[postgres@pll-gdajukk-postgresql-vm-1 ~]$ psql -c "select pg_is_in_recovery();"
 pg_is_in_recovery
-------------------
 t
(1 row)

[postgres@pll-gdajukk-postgresql-vm-1 ~]$

@mum0l check on RedHat, not Ubuntu

Tested configurations:
:white_check_mark: epicli apply: AWS/Azure x RedHat/Ubuntu - 1 PostgreSQL vm
:white_check_mark: epicli apply: AWS/Azure x RedHat/Ubuntu - 2 PostgreSQL vms clustered
No problems encountered :heavy_check_mark:


Add FAQs from Synapse 1P Migration Office Hours

Incorporating FAQs from Synapse 1P Migration weekly office hours.

@nick-shapiro : Thanks for your contribution! The author(s) have been notified to review your proposed change.

@saveenr
Can you review this PR?
IMPORTANT: When this content is ready to merge, you must add #sign-off in a comment or the approval may get overlooked.
#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team

@nick-shapiro : Thanks for your contribution! The author(s) have been notified to review your proposed change.

#sign-off

@Jak-MS : Thanks for your contribution! The author(s) have been notified to review your proposed change.


URIError: malformed URI sequence

When you reload a page with a URL having the form '?q=%'  an error occurs URIError: malformed URI sequence
Solution:
        try {             val = val === undefined ? null : decodeURIComponent(val);         } catch (e) {             val = null;         }
Is it correct?

See the discussion at #97

Thanks

Fixed in 5.0.0.


Correc byte ordering to convert numbers to Ipv4 addresses

This commit changes the byte order when converting IPv4 Addresses, as
the order was reversed.
The test program used to validate it works was sending a multicast
address to join_multicast_v4 which was not accepted by either Windows
or Linux. Now the test program compiles and run without issues.
use socket2::{Domain, Protocol, Socket, Type};
use std::net::Ipv4Addr;

fn main() {
    let socket = Socket::new(Domain::IPV4, Type::DGRAM, Some(Protocol::UDP))
        .expect("could not create socket");
    socket
        .join_multicast_v4(&Ipv4Addr::new(224, 0, 0, 251), &Ipv4Addr::UNSPECIFIED)
        .expect("could not join multicast group");
}

Fixes https://github.com/alexcrichton/socket2-rs/issues/87

Thanks @bltavares.
This does show we need additional testing.


New-PartnerAccessToken does not returns refresh token

Steps to reproduce

What steps can reproduce the defect?
Please share the setup, commandline for vstest.console, sample project, target
framework etc.

Refresh Token was issued with commands in v 1.5 of the module. It follows the standard way of creating SP, creation impersonation with the SP of a user that has global administrator and admin agent rights. Nothing out of the ordinary.
$p=New-PartnerAccessToken -ApplicationId "<app id>" -Credential (Get-Credential) -RefreshToken $token -Scopes 'https://api.partnercenter.microsoft.com/user_impersonation' -ServicePrincipal -Tenant <tenant id>
$p.RefreshToken

$p.RefreshToken is empty. Only access token is filled.
Expected behavior

Share the expected output
Expected output should contain Refresh token

Actual behavior

What is the behavior observed?

No refresh token is returned, only access one.
Diagnostic logs

Please share test platform diagnostics logs.
The logs may contain test assembly paths, kindly review and mask those before sharing.

Environment

Please share additional details about your environment.
Version

Both 5.1 and 6.2.3

@slavizh thank you for opening this new issue. When you generate a new access token using any refresh token, that refresh token should be injected into the cache. After that is accomplished everything should function as excepted. There is a possibility that what you stated in the other thread is correct, but that definitely is not the expected behavior. So, we will need to dig into this further to see how best to reproduce the issue.

If I try to generate new refresh token I get error on the web page after entering credentials:
AADSTS50011: The reply url specified in the request does not match the reply urls configured for the application:
I am using the following command assuming it is the right way to issue the refresh token for the first time.
$t=New-PartnerAccessToken -ApplicationId '<app id>' -Credential (Get-Credential) -Scopes 'https://api.partnercenter.microsoft.com/user_impersonation' -ServicePrincipal -Tenant '<app id>' -UseAuthorizationCode

BTW when problems happen in the broswer you can never cancel the command on the PowerShell window. You should have some time out or ability to cancel the command otherwise I have to close the whole window.

I have not managed to test it yet since the new release but I bank my money @slavizh is correct. Plus the issue of not being able to cancel is so annoying it is beyond belief... especially that the new module is so broken that you end up testing all of it<PHONE_NUMBER>00 times a day - needed to restart everything each time...

Also I did mention in my last issue that Refresh Tokens are not being returned... I do not know C# well enough but when I read the code it does not appear to have the right property assigned to it.
            string key = GetTokenCacheKey(authResult);

            AuthResult result = new AuthResult(
                authResult.AccessToken,
                authResult.IsExtendedLifeTimeToken,
                authResult.UniqueId,
                authResult.ExpiresOn,
                authResult.ExtendedExpiresOn,
                authResult.TenantId,
                authResult.Account,
                authResult.IdToken,
                authResult.Scopes);

            if (tokens.ContainsKey(key))
            {
                result.RefreshToken = tokens[key].Secret;
            }

            WriteObject(result);
        }

Surely authResult.RefreshToken should exist somewhere here?

I do not know if this will help with reply url but here is mine. This is what was suggested when we first started this authentication model. As far as I can see the redirect url in the module is using localhost. Could that also cause the issue where the refresh token is not returned?

If I put reply url https://localhost as well I can do interactive authentication. Seems you have changed the reply URL. Can you set it back to urn:ietf:wg:oauth:2.0:oob so we do not have to change our apps?
I still cannot get refresh token even when I have made this change,

It has got nothing to do with what is setup in your App but everything to do with the code inside the command.
When I run the "fixed" code and check in fiddler:

So when we run this:
New-PartnerAccessToken -ApplicationId $SPNClientId -Credential $Credentials -RefreshToken $SPNRefreshToken -Scopes 'https://api.partnercenter.microsoft.com/user_impersonation' -ServicePrincipal -Tenant $TenantId

We get:
RefreshToken            :
AccessToken             : eyJ0eXAiOiJxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
                          sSD-mWE2TO-ksI8fA
IsExtendedLifeTimeToken : False
UniqueId                : xxxxx-xxxx-xxx-xxxx-xxxxx
ExpiresOn               : 9/16/2019 8:20:18 AM +00:00
ExtendedExpiresOn       : 9/16/2019 8:20:18 AM +00:00
TenantId                : xxxxx-xxxx-xxx-xxxx-xxxxx
Account                 : Account username<EMAIL_ADDRESS>environment login.windows.net home account id: AccountId:
                          xxxxx-xxxx-xxx-xxxx-xxxxx.xxxxx-xxxx-xxx-xxxx-xxxxx
IdToken                 : eyJ0eXxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Scopes                  : {https://api.partnercenter.microsoft.com/user_impersonation}
User                    :

But API is returning the Refresh Token correctly (see Fiddler) but the command is written in a way so it does not.

@cblackuk Thanks for running this trough Fiddler. Do you get also reply URL issue if you try to issue completely new refresh token. I would assume yes if you have setup the app as me.

The code above should issue a new token and it does but it does not get returned... until they fix it... no point testing anything else really :-)

I was wondering if you get the same AAD error when you use -UseAuthorizationCode parameter but anyway. I hope they will resolve that problem as well.

Will test it tomorrow mate :)

You were 150% right mate:

Thanks for testing it.

2.0.1909.3 does not fixes refresh token not returned. :(

Why am I not surprised... I guess the reply URL is also broken still right?

Haven't tested that but it is useless to test it if refresh token does not work.

Just tested both.
No refresh token and reply URL is broken still.

Same here, issue still persists.

@isaiahwilliams Several days have passed and the only update was merged that didn't fixed the issue. Is the issue is investigated? Do we need to provide more information? When we can expect a fix?

Refresh tokens are valid for 90 days, so while this issue is being debugged you should not encounter any issue with using the current value. Also, I just released version 2.0.1909.4 that adds the ability to get debug statements from MSAL. Can you update and run the command with and without the debug flag please?

@isaiahwilliams We have automation that renews tokens every day. We cannot migrate if we are not able to run it. ran the new version. refresh token is still not available. Debug is below:
DEBUG: MSAL Info False (False) MSAL <IP_ADDRESS> MSAL.CoreCLR N/A [09/20/2019 15:45:13 - bf5e5a32-3ed1-47da-b3ca-3fad629b5e7d] MSAL MSAL.CoreCLR with assembly version '<IP_ADDRESS>', file version '<IP_ADDRESS>' and informational version '3.0.8+5db620ee347d54948b2775699985ae7a4d2bef1a'.  TelemetryCorrelationId(bf5e5a32-3ed1-47da-b3ca-3fad629b5e7d)                                                                                Confirm                                                                                Continue with this operation?                                                                                [Y] Yes  [A] Yes to All  [H] Halt Command  [S] Suspend  [?] Help (default is "Y"): a                                                                                DEBUG: MSAL Info False (False) MSAL <IP_ADDRESS> MSAL.CoreCLR N/A [09/20/2019 15:45:13 - bf5e5a32-3ed1-47da-b3ca-3fad629b5e7d] Using 1 scopes for acquire token by refresh token request
DEBUG: MSAL Info False (False) MSAL <IP_ADDRESS> MSAL.CoreCLR N/A [09/20/2019 15:45:13 - bf5e5a32-3ed1-47da-b3ca-3fad629b5e7d]                                                                                === Request Data ===
Authority Provided? - True
Scopes - https://api.partnercenter.microsoft.com/user_impersonation
Extra Query Params Keys (space separated) -
DEBUG: MSAL Info False (False) MSAL <IP_ADDRESS> MSAL.CoreCLR N/A [09/20/2019 15:45:13 - bf5e5a32-3ed1-47da-b3ca-3fad629b5e7d] === Token Acquisition (ByRefreshTokenRequest) started:
 Cache Provided: True
 Authority Host: login.microsoftonline.com
DEBUG: MSAL Info False (False) MSAL <IP_ADDRESS> MSAL.CoreCLR N/A [09/20/2019 15:45:13 - bf5e5a32-3ed1-47da-b3ca-3fad629b5e7d] Resolving authority endpoints... Already resolved? - TRUE                                                                                DEBUG: MSAL Info False (False) MSAL <IP_ADDRESS> MSAL.CoreCLR N/A [09/20/2019 15:45:14 - bf5e5a32-3ed1-47da-b3ca-3fad629b5e7d] Checking client info returned from the server..                                                                                DEBUG: MSAL Info False (False) MSAL <IP_ADDRESS> MSAL.CoreCLR N/A [09/20/2019 15:45:14 - bf5e5a32-3ed1-47da-b3ca-3fad629b5e7d] Saving Token Response to cache..                                                                                DEBUG: MSAL Info False (False) MSAL <IP_ADDRESS> MSAL.CoreCLR N/A [09/20/2019 15:45:14 - bf5e5a32-3ed1-47da-b3ca-3fad629b5e7d] Looking for scopes for the authority in the cache which intersect with https://api.partnercenter.microsoft.com/user_impersonation                     DEBUG: MSAL Info False (False) MSAL <IP_ADDRESS> MSAL.CoreCLR N/A [09/20/2019 15:45:14 - bf5e5a32-3ed1-47da-b3ca-3fad629b5e7d] Intersecting scope entries count - 0
DEBUG: MSAL Info False (False) MSAL <IP_ADDRESS> MSAL.CoreCLR N/A [09/20/2019 15:45:14 - bf5e5a32-3ed1-47da-b3ca-3fad629b5e7d] Matching entries after filtering by user - 0                                                                                DEBUG: MSAL Info False (False) MSAL <IP_ADDRESS> MSAL.CoreCLR N/A [09/20/2019 15:45:14 - bf5e5a32-3ed1-47da-b3ca-3fad629b5e7d] Saving RT in cache...                                                                                DEBUG: MSAL Info False (False) MSAL <IP_ADDRESS> MSAL.CoreCLR N/A [09/20/2019 15:45:14] Serializing token cache with 1 items.                                                                                DEBUG: MSAL Info False (False) MSAL <IP_ADDRESS> MSAL.CoreCLR N/A [09/20/2019 15:45:14 - bf5e5a32-3ed1-47da-b3ca-3fad629b5e7d] === Token Acquisition finished successfully. An access token was returned with Expiration Time: 09/20/2019 16:45:13 +00:00 ===

@slavizh thank you for sharing this information. Just to confirm the refresh token value was still missing when you ran the command with this version correct?

@isaiahwilliams  yes

@slavizh that is interesting because the debug logs state the refresh token returned was cached. Can you confirm that C:\Users\<username>\AppData\Local\.IdentityService is empty as well?

@isaiahwilliams

@slavizh thank you for sharing that. There should be a file named msal.cache in that directory. I am not sure what is happening just yet, but that is the reason the refresh token value is missing. The token cache is not persisting to storage. I was able to reproduce the issue and then I got it corrected in my environment. Let me keep digging to see what is causing this inconsistent behavior.

Good luck and fingers crossed :) @isaiahwilliams

@slavizh the root cause for this issue has been identified and corrected. Can you update to version 2.0.1909.5 and let me know if the value is returning as expected now?

@isaiahwilliams I have verified that version 2.0.1909.5 produces token. Thank you for fixing this!

Awesome that is great news. It was a rather interesting issue, but the good news it is resolved now. Thank you for bearing with me while I worked to get everything sorted.

Trying to generate a new token but receiving the following error

AADSTS50011: The reply url specified in the request does not match the reply urls configured for the application

I'm running 2.0.1909.5. This occurs right after the consent in the browser on a fresh setup. Reply URI set correctly on app to "urn:ietf:wg:oauth:2.0:oob"
Anyone else running into this issue?

@prysmcorp https://github.com/microsoft/Partner-Center-PowerShell/issues/158
Add https://localhost - job done

That was definitely it. Thank you, sir!

hello I know this is closed but I've updated the module to the "2.0.1909.5 " version but I as well don't have a refreshcode.
The localhost solution did fixed my reply url.
@slavizh I was wondering what you mean by automating to refresh the token.
This is my code.

$appId = 'hello'
$appSecret = 'world' | ConvertTo-SecureString -AsPlainText -Force
$tenantId = 'tenant'

#Perform this code if the refreshtoken is expired
$credential = New-Object System.Management.Automation.PSCredential $appId, $appSecret
$credential = Get-Credential
$token = New-PartnerAccessToken -ApplicationId $appId -Scopes 'https://api.partnercenter.microsoft.com/user_impersonation' -ServicePrincipal -Credential $credential -Tenant $tenantId -UseAuthorizationCode
$token.RefreshToken  | Out-File "C:\CSP\Refreshtoken.txt"

$refreshToken = "the token that was found inside C:\CSP\Refreshtoken.txt" "

# This will create a new accesstoken based on the refreshtoken so that the script will run automatically without any user interaction
$Newtoken = New-PartnerAccessToken -RefreshToken $refreshToken -Credential $credential -Scopes 'https://api.partnercenter.microsoft.com/user_impersonation' -ApplicationId $appId -ServicePrincipal -Tenant $tenantId -ErrorVariable ExpiredToken
Connect-PartnerCenter -AccessToken $Newtoken.AccessToken

@bilalAchahbar the root cause for this issue was related to how the module was being published. Since you have updated to a version where this has been corrected and your are still having issues please open a new issue. If you do open a new issue please include details about what exactly is missing.

I am unable to obtain refresh token with PowerShell 7.0.3 in ubuntu 18.04. It does provide refresh token if i try in windows powershell 5.1.19041.1
Command to reproduce the issue - New-PartnerAccessToken -UseDeviceAuthentication -ApplicationId 'a0c73c16-a7e3-4564-9a95-2bdf47383716' -Scopes 'https://outlook.office365.com/.default';

We have the same issue.
Powershell module version is "3.0.10 PartnerCenter" and Refresh Token is not included in the response. All other properties are presented, only the Refresh token is missing. Also, it works on my machine (Refresh token is returned), but when I deploy it on prod server, it missing...
Any additional ideas or suggestions?


Use yowasp_yosys instead of system-installed yosys

Following the discussion in #40, change to use the wasm-bundled version of yosys as in yowasp_yosys instead of the system-installed yosys (symbiflow projects use conda to install it, but this might not be the case for other users).
One benefit of this is that it allows us to control the version of yosys used by the user. This is important because there might be breaking changes in yosys, such as #13, and it wouldn't be ideal to cater for multiple versions of yosys.
One small downside is this takes about twice the duration compared to native yosys but I don't think this really matters.
Signed-off-by: Daniel Lim Wee Soong<EMAIL_ADDRESS>

Can we make this a configuration option?

Yup sure, so a setting in conf.py?

Yeap! Sounds good.

Now we support both types of yosys installations. @mithro Do you think we need to implement tests for both use cases? I can't think of any straightforward way of doing so without setting up a whole sphinx project for each of the two cases.

@daniellimws - Can we just set up travis to run in the two configurations?

@mithro I'm not good with Travis so if you got any ideas you could try it first

I believe that the mechanism for tests provided in the #52 can be useful here.

@rw1nkler -- Can you take a look at this PR? If @daniellimws doesn't currently have the time, can you take it over and get it merged?

Hi, yes I currently am quite busy with school work. I might be free next week though. Feel free to take over if you want :).


change field-campaigner-data as module

This fix #79

change field-campaigner-data as module to this repo : https://github.com/hotosm/field-campaigner-data
update code regarding this changes

hi @cchristelis @smit1678
travis failed when submodule is executed. we need to add travis in the field-campaigner-data?

@meomancer Done. Travis should now be good to go.

@meomancer, let's not hard code the data source.

@smit1678 @cchristelis
updated:

added DATA_SOURCE as a path to data source path on local
handling if data source is not found or valid.

hi @cchristelis found issue.
if we don't harcoded, i'm not sure how to implement it.
because production is using docker, we need to mount the folder first before able to used.
and this mounting needs to be implemented in compose level
do you have suggestion @cchristelis ?

@meomancer, let's add a step in the make receipe. If this is not specified, let's promt the user.

hi @cchristelis
updated

on top of makefile, there is CAMPAIGN_DATA variable
now when we run the container, if the CAMPAIGN_DATA is empty, it will ask where the folder of data

Found that git isn't working on container

install git
setup git global to container

hi @cchristelis
it seem that it is quite hard to do git automatically (or maybe we could just create commit and let user push manually)
because some keys that is needed to be done first

need to update git config
need to copy ssh_keys from local to container

what do you suggest, @cchristelis ? should we copy the ssh_keys?

@meomancer, we could add the keys to allowed keys on the repository and allow push on that account as well:

hi @cchristelis
updated:
when creating container, it will regenerate ssh keys and after registered to deploy keys, git is working

now it is using root user. maybe need to update it?

great!

@meomancer, is this ready to merge? I'd like to get this in so that we don't keep having campaign changes muddy up PRs...


Fix tests

Fix build ... and apparently I can't spell https 🤦♂

Ah, rebased them myself just a little bit ago. ^_^ Thanks!


[DO NOT MERGE] POC: autocomplete open on focus

Description

[!TIP]
Add a TL;DR when the description is longer than 500 words or extremely technical (helps the content, marketing, and DevRel team).
Please also include relevant motivation and context. List any dependencies that are required for this change. Add links to Notion, Figma or any other documents that might be relevant to the PR.

Fixes #34921
or
Fixes Issue URL

[!WARNING]
If no issue exists, please create an issue first, and check with the maintainers if the issue is valid.

Automation
/ok-to-test tags=""
:mag: Cypress test results

[!CAUTION]
If you modify the content in this section, you are likely to disrupt the CI result for your PR.

Communication
Should the DevRel and Marketing teams inform users about this change?

[ ] Yes
[ ] No

Summary by CodeRabbit

New Features

Introduced a new property to configure the autocomplete menu to display by default in various widgets.
Added new utility functions to enhance cursor context detection and autocomplete hint logic.
Integrated an AI assistance window in the CodeEditor for improved contextual help.
Enhanced hint display logic in the editor to provide more contextual hints based on cursor position.

Bug Fixes

Updated logic in multiple components to eliminate deprecated features and simplify user interactions.

Style

Cleaned up code by removing unused components and commented-out code, enhancing readability and maintainability.

/build-deploy-preview skip-tests=true

/build-deploy-preview skip-tests=true

/build-deploy-preview skip-tests=true

Its merged to the featured branch, hence closing this.


Missing right channel in new SARDARA format

Is it a problem with SDT, or a problem with SARDARA?

SARDARA. Only one channel is listed in the SECTION TABLE


Update sayit to 4.0.1

https://github.com/ioBroker/ioBroker.sayit/issues/317

ON HOLD due to issues


fixed JVM's config name corresponding to MaxWaitTime

Fixed JVM config name for MaxWaitTime config in docs.
Incorrect JVM config: fetch.wait.max.ms <-- docs mention this.
Correct JVM config: fetch.max.wait.ms

@abhipranay thanks, this is a good spot and worth correcting.
One small clarification is that fetch.wait.max.ms was previously correct and was used in the original Consumer implementation in Apache Kafka (which was eventually deleted in 2.1.0) that Sarama implemented in its first incarnation in the early days. When Kafka implemented Consumer Groups in 0.8.2 they renamed the property to fetch.max.wait.ms


Does not work on Raspberry Pi

This is the script I'm using to test BreakfastSerial
########################################
from BreakfastSerial import Arduino, Servo
from time import sleep
board = Arduino()
servo = Servo(board, 10)
import code
code.InteractiveConsole(locals=globals()).interact()
########################################
When running this on an Ubuntu Virtual Machine, it works fine. When running this with Raspberry Pi, this consistently happens:
First run (tries to connect to ttyACM0):
python servo_test.py
Connecting to /dev/ttyACM0
Python 2.7.3 (default, Jan 13 2013, 11:20:46)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)

Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 552, in __bootstrap_inner
self.run()
File "/usr/local/lib/python2.7/dist-packages/BreakfastSerial/BreakfastSerial.py", line 80, in run
while self.board.bytes_available():
File "/usr/local/lib/python2.7/dist-packages/pyfirmata/pyfirmata.py", line 225, in bytes_available
return self.sp.inWaiting()
File "/usr/local/lib/python2.7/dist-packages/serial/serialposix.py", line 435, in inWaiting
s = fcntl.ioctl(self.fd, TIOCINQ, TIOCM_zero_str)
IOError: [Errno 5] Input/output error

exit()

Second run (Tries to connect to ttyACM1):
python servo_test.py
Connecting to /dev/ttyACM1
Python 2.7.3 (default, Jan 13 2013, 11:20:46)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)

servo.center() #This call worked properly before the exception below
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 552, in __bootstrap_inner
self.run()
File "/usr/local/lib/python2.7/dist-packages/BreakfastSerial/BreakfastSerial.py", line 80, in run
while self.board.bytes_available():
File "/usr/local/lib/python2.7/dist-packages/pyfirmata/pyfirmata.py", line 225, in bytes_available
return self.sp.inWaiting()
File "/usr/local/lib/python2.7/dist-packages/serial/serialposix.py", line 435, in inWaiting
s = fcntl.ioctl(self.fd, TIOCINQ, TIOCM_zero_str)
IOError: [Errno 5] Input/output error
exit()

Every subsequent run(tries connecting to ttyACM0)
python servo_test.py
Connecting to /dev/ttyACM0
Traceback (most recent call last):
File "servo_test.py", line 5, in
board = Arduino()
File "/usr/local/lib/python2.7/dist-packages/BreakfastSerial/BreakfastSerial.py", line 31, in init
raise FirmataNotOnBoardException
BreakfastSerial.BreakfastSerial.FirmataNotOnBoardException

Hi
I had same problem. It seems strange but you probably connect servo to Arduino  without special power source for Arduino you power Arduino from Pi same as me. Try to  connect supplemental source only for servo.
I hope it helps you, at least this solution helped myself.
Kind regards Daniel

Closing since this hasn't gotten attention in 2.5 years. If someone wants to reopen feel freel


Out of memory after 7396000 records loaded

I think I've found memory leak.
The test case is following:

create graph db on local filesystem
keep loading some random data via graph API
after some (rather long) time (cca 30 minutes - 7396000 records loaded in my case) heap memory is exhausted and my application crashes with following exception:

Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded
	at java.lang.Long.toUnsignedString(Long.java:242)
	at java.lang.Long.toHexString(Long.java:177)
	at java.util.UUID.digits(UUID.java:385)
	at java.util.UUID.toString(UUID.java:375)
	at com.tinkerpop.blueprints.impls.orient.OrientElement.setProperties(OrientElement.java:139)
	at com.tinkerpop.blueprints.impls.orient.OrientBaseGraph.addVertex(OrientBaseGraph.java:474)
	at com.github.maiklos.orientdb_sandbox.Run.start(Run.java:43)
	at com.github.maiklos.orientdb_sandbox.Run.main(Run.java:19)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)

demonstration is available here: https://github.com/maiklos/orientdb_sandbox/commit/ef3f7c2cfc52db9cb9c6f0a59e54b7093c92d260
I run it with -Xmx64M. Also the resulting database size is 5.3G so enough space is needed on filesystem.
jvisualvm snapshot is here:

Seem like it fixed :-) I close it.

Please I have a problem with OrientDB and Java I have a program with this query to calculate the shortest path to 300000 but the program is very slow someone can help me how I do the memory configuration I read this post but I don't understand how I do this changes @laa and @maiklos please help me

Hi Lorena,  check http://orientdb.com/docs/last/Performance-Tuning.html for sample jvm options, and http://orientdb.com/docs/last/Configuration.html for xml properties too.  For an appropriate Xmx value, you could monitor gc activity with additional options in your server.sh and java client (e.g for server: -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -XX:+PrintTenuringDistribution -Xloggc:$ORIENTDB_HOME/log/gc_%p_%t.log)   You can check at startup current jvm settings with +XX:+PrintFlagsFinal and current orientdb properties with -Denvironment.dumpCfgAtStartup=true  You can set appropriate values for direct memory and buffers, e.g: -XX:MaxDirectMemorySize=12g -Dstorage.diskCache.bufferSize=12288 (or buffer size in xml, in mb, with buffer size <= direct memory). Claudio


about the KEGG_decoder.py

Hello! I think the KEGG-Decoder is very useful. But it seem that there is a typo in the script KEGG_decoder.py.
line265-266
#citryl-CoA synthetase AND citryl-CoA lyase
	if ('K15232' in ko_match and 'K15232' in ko_match and 'K15234' in ko_match):

One of the K15232 should be K15233. And, in my opinion, it is better to define this enzyme as:
 if (('K15232' in ko_match or 'K15233' in ko_match) and 'K15234' in ko_match):
What do you think about it?
Best regards,
Cheng

Great find, @caris1988!
I have made the correction. But I did not change the enzyme definition as you suggested. I don't think the OR statement to check for K15232 or K15233 is appropriate, as they are subunits of the same enzyme. I am intentionally trying to be conservative in calling reverse TCA, just because of the low threshold it takes to go from heterotroph to autotroph in that instance.
If you have any info. that would suggest K15233 isn't necessary for a functioning citryl-CoA synthetase please send it may way and I'll adjust accordingly.

Great find, @caris1988!
I have made the correction. But I did not change the enzyme definition as you suggested. I don't think the OR statement to check for K15232 or K15233 is appropriate, as they are subunits of the same enzyme. I am intentionally trying to be conservative in calling reverse TCA, just because of the low threshold it takes to go from heterotroph to autotroph in that instance.
If you have any info. that would suggest K15233 isn't necessary for a functioning citryl-CoA synthetase please send it may way and I'll adjust accordingly.


List resources should be expanded

Currently (khelm 2.0.0-beta.3) there is no special handling for resources of kind List which stops khelm from applying filters and transformations to items of such a resource and, when using kustomize output, writes all items into a list.yaml file.
This can be reproduced using the traefik example shown in #13.
Therefore List resources should be expanded.

This is fixed now in khelm 2.0.0-beta.4


Unauthorized error seen in logs

I've deployed prometheus and prometheus-adapter with their respective helm charts in the same namespace.

I'm seeing these as well and would like to understand why before changing anything.
POST https://<IP_ADDRESS>:443/apis/authorization.k8s.io/v1beta1/subjectaccessreviews

I'm seeing some calls logged as the above succeed and some calls return 403 Forbidden:
I1007 21:40:05.707821       1 request.go:897] Request Body: {"kind":"SubjectAccessReview","apiVersion":"authorization.k8s.io/v1beta1","metadata":{"creationTimestamp":null},"spec":{"nonResourceAttributes":{"path":"/healthz","verb":"get"},"user":"system:anonymous","group":["system:unauthenticated"]},"status":{"allowed":false}}
I1007 21:40:05.709716       1 round_trippers.go:386] curl -k -v -XPOST  -H "Accept: application/json, */*" -H "Content-Type: application/json" -H "User-Agent: adapter/v0.0.0 (linux/amd64) kubernetes/$Format" -H "Authorization: Bearer ..."
I1007 21:40:05.785410       1 request.go:897] Request Body: {"kind":"SubjectAccessReview","apiVersion":"authorization.k8s.io/v1beta1","metadata":{"creationTimestamp":null},"spec":{"nonResourceAttributes":{"path":"/apis/metrics.k8s.io/v1beta1","verb":"get"},"user":"system:serviceaccount:kube-system:resourcequota-controller","group":["system:serviceaccounts","system:serviceaccounts:kube-system","system:authenticated"]},"status":{"allowed":false}}
I1007 21:40:05.785575       1 round_trippers.go:386] curl -k -v -XPOST  -H "Authorization: Bearer ..."  'https://<IP_ADDRESS>:443/apis/authorization.k8s.io/v1beta1/subjectaccessreviews'
I1007 21:40:06.005960       1 round_trippers.go:405] POST https://<IP_ADDRESS>:443/apis/authorization.k8s.io/v1beta1/subjectaccessreviews 201 Created in 1 milliseconds
I1007 21:40:06.005987       1 round_trippers.go:411] Response Headers:
I1007 21:40:06.005993       1 round_trippers.go:414]     Audit-Id: d773b903-001f-4eaf-918d-af29328657a0
I1007 21:40:06.005998       1 round_trippers.go:414]     Content-Type: application/json
I1007 21:40:06.006003       1 round_trippers.go:414]     Content-Length: 260
I1007 21:40:06.006008       1 round_trippers.go:414]     Date: Mon, 07 Oct 2019 21:40:06 GMT
I1007 21:40:06.006031       1 request.go:897] Response Body: {"kind":"SubjectAccessReview","apiVersion":"authorization.k8s.io/v1beta1","metadata":{"creationTimestamp":null},"spec":{"nonResourceAttributes":{"path":"/","verb":"get"},"user":"system:anonymous","group":["system:unauthenticated"]},"status":{"allowed":false}}
I1007 21:40:06.006112       1 authorization.go:73] Forbidden: "/", Reason: ""
I1007 21:40:06.006199       1 wrap.go:42] GET /: (2.474734ms) 403 [[Go-http-client/2.0] <IP_ADDRESS>:52772]

These logs seem to be from incoming calls to the prometheus adapter, not outgoing calls from the prometheus adapter; one call that succeeds as user system:anonymous seems to be for the healthcheck to url path /healthz.
@gokhandincer @s-urbaniak so what would be making incoming calls as user system:anonymous to root url path /, is it a valid url path, and is it really safe to enable system:anonymous user access?

I've fixed the issue by setting proxy-client parameters for api server:
see:
https://kubernetes.io/docs/tasks/access-kubernetes-api/configure-aggregation-layer/#kubernetes-apiserver-client-authentication
btw your anonymous user is apiservice discovery/health check that is continuously executed by api server.

This solved my issue
`apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
labels:
name: system:non-resource-urls
rules:

nonResourceURLs: [ "*" ]
verbs: [ "get" ]

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: cluster-system-anonymous
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: system:non-resource-urls
subjects:

apiGroup: rbac.authorization.k8s.io
kind: User
name: system:anonymous`

This solved my issue
`
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
labels:
name: system:non-resource-urls
rules:

nonResourceURLs: [ "*" ]
verbs: [ "get" ]
`

`
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: cluster-system-anonymous
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: system:non-resource-urls
subjects:

apiGroup: rbac.authorization.k8s.io
kind: User
name: system:anonymous
`

i have exactly the same problem since month and i cannot use metrics. I just updated to latest kube-prometheus project code and i am in the same state:
I1212 07:59:14.128026       1 round_trippers.go:405] POST https://<IP_ADDRESS>:443/apis/authorization.k8s.io/v1beta1/subjectaccessreviews 201 Created in 6 milliseconds
I1212 07:59:14.128623       1 round_trippers.go:411] Response Headers:
I1212 07:59:14.128671       1 round_trippers.go:414]     Date: Thu, 12 Dec 2019 07:59:14 GMT
I1212 07:59:14.128710       1 round_trippers.go:414]     Content-Type: application/json
I1212 07:59:14.128824       1 round_trippers.go:414]     Content-Length: 287
I1212 07:59:14.128994       1 request.go:897] Response Body: {"kind":"SubjectAccessReview","apiVersion":"authorization.k8s.io/v1beta1","metadata":{"creationTimestamp":null},"spec":{"nonResourceAttributes":{"path":"/apis/metrics.k8s.io/v1beta1","verb":"get"},"user":"system:anonymous","group":["system:unauthenticated"]},"status":{"allowed":false}}
I1212 07:59:14.129126       1 authorization.go:73] Forbidden: "/apis/metrics.k8s.io/v1beta1", Reason: ""
I1212 07:59:24.640047       1 wrap.go:42] GET /apis/metrics.k8s.io/v1beta1: (5.083206ms) 403 [[Go-http-client/2.0] <IP_ADDRESS>:60276]
I1212 07:59:26.331904       1 authorization.go:73] Forbidden: "/apis/metrics.k8s.io/v1beta1", Reason: ""

Is there any way to ensure we will not be not authed on this healthcheck ?

@sandeeplamb i tried your "fix", it works but i'm doing it more grain because i don't want anonymous to access all my APIs
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
  name: system:prometheus-k8s:metrics-service
rules:
- nonResourceURLs:
    - "/apis/metrics.k8s.io/v1beta1"
    - "/openapi/v2"
  verbs: [ "get" ]
- apiGroups:
  - "metrics.k8s.io"
  resources:
  - nodes
  - pods
  verbs:
  - get
  - list
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: system:prometheus-k8s:metrics-service
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: system:prometheus-k8s:metrics-service
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: User
  name: system:anonymous

@s-urbaniak @gokhandincer is it okay to just leave it as it is.
It logs unauthorized every now & then but apart from that it seems to work, so instead of granting additional permissions to system:anonymous, how about we just ignore the log statement?

@jaydp17 in my case it didn't work unless I give following permission to system:anonymous
- nonResourceURLs:
    - "/apis/custom.metrics.k8s.io/v1beta1"
  verbs: [ "get" ]

we don't need to add openapi/v2 anyway it is 404, but this is just workaround. as suggested by @Ludek2 , setting proxy-client parameters would be the ideal solution.


fix: tcc mode run failed

ref: https://github.com/seata-golang/seata-golang/issues/
Ⅰ. Describe what this PR did
Ⅱ. Does this pull request fix one issue?
Ⅲ. Why don't you add test cases (unit test/integration test)?
Ⅳ. Describe how to verify it
Ⅴ. Special notes for reviews

Codecov Report

Merging #117 (3ef7b75) into v2 (ae2e91f) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##               v2     #117   +/-   ##
=======================================
  Coverage   25.46%   25.46%
=======================================
  Files           4        4
  Lines         805      805
=======================================
  Hits          205      205
  Misses        593      593
  Partials        7        7

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ae2e91f...3ef7b75. Read the comment docs.


Delete manifests/g/GitHub/cli/1.4.0 directory

[ ] Have you signed the Contributor License Agreement?
[ ] Have you checked that there aren't other open pull requests for the same manifest update/change?
[ ] Have you validated your manifest locally with winget validate --manifest <path>?
[ ] Have you tested your manifest locally with winget install --manifest <path>?
[ ] Does your manifest conform to the 1.2 schema?

Note: <path> is the name of the directory containing the manifest you're submitting.

Close with reason: Will not be removing versions just because of CVE;


HTML to MarkDown in hello-hybrid-aadj-sso-cert.md

Description:
This is a combined effort to alleviate a translation bug as well as improving the MarkDown codestyle in this document, both for the English (en-us) version of the document as well as the translated versions.
Proposed changes:

Remove remnants of HTML code (<strong> </strong>) and use only MarkDown code for the Github document source.
Add double asterisks around the intended words to be highlighted in bold , thereby removing the reported "orphans".

Issue ticket reference or closure:
This change should in theory close the issue tickets #3451 and #3453 after the scripted translation process has been re-run on this document.
The solution is based on a user discussion in issue ticket #4589 .

It looks good to me. Perhaps if things get slow this week, I can spend some time proactively hunting for other troublesome HTML tags.


Cannot resolve imported module variables/functions

First of all, I love this plugin!
Question... in my xquery modules/files where I'm referencing an imported module's variables/functions I'm getting a "cannot resolve variable ...".
I'm using MarkLogic 8 as my database and IntelliJ Ultimate 2016.1.
I have configured the plugin with the appropriate login information and I'm pointing to a WebDav database. Is this correct?
I have also tried configuring the plugin to connect to one of my HTTP servers, but still have the same issue.
Thanks...

Hey,
Thanks for kind words :) 👍
Can you please show a screenshot of which modules/variables are getting "cannot resolve variable"? Just a few examples.
Also, can you please let me know if the sources of those imported modules are part of the project? Are they part of standard MarkLogic API (e.g. available in https://docs.marklogic.com/ )?
Currently plugin does not support fetching/browsing modules in WebDav. Please raise a separate issue with the description of the expected functionality if you'd be interested in such feature.
What you have configured is a data source used for running XQuery code (for MarkLogic it generally works only when all of the used [imported] modules are either standard api or uploaded to server). It requires adding XDBC server and specifying its port in the data source configuration.
Can you please also let me know what you have selected in File -> Settings -> Other settings -> XQuery -> Other options -> XQuery flavour?

Hi and thanks for the quick response.
As for my settings I have MarkLogic 8 selected as the XQuery flavor.
In my XQuery Data Sources, I have the following settings:

Name: MarkLogic-local
Host: localhost
Port: 8041 (references XDBC connection)
Database name: sof-sdl-modules (references Modules/Content databases
for my project where code/modules for project reside)
Username: login user name
Password: Password for DB
NOTE: The XDBC server is set up to use Digest authentication

Here is an example of the plugin not seeing imported module
methods/variables
[image: Screen Shot 2016-07-18 at 7.57.22 AM.png]
Module imports above.
Then, further down in the same module...
[image: Screen Shot 2016-07-18 at 7.58.12 AM.png]
Here you can see the imported modules for "req", "status-lib", "cp", etc
being imported and then later on where they are referenced you can see the
plugin highlighting them in red with "mouse-over" hints stating "Cannot
resolve variable XXX".
Thanks,
Bill
On Sun, Jul 17, 2016 at 10:08 AM ligasgr<EMAIL_ADDRESS>wrote:

Hey,
Thanks for kind words :) 👍
Can you please show a screenshot of which modules/variables are getting
"cannot resolve variable"? Just a few examples.
Also, can you please let me know if the sources of those imported modules
are part of the project? Are they part of standard MarkLogic API (e.g.
available in https://docs.marklogic.com/ )?
Currently plugin does not support fetching/browsing modules in WebDav.
Please raise a separate issue with the description of the expected
functionality if you'd be interested in such feature.
What you have configured is a data source used for running XQuery code
(for MarkLogic it generally works only when all of the used [imported]
modules are either standard api or uploaded to server). It requires adding
XDBC server and specifying its port in the data source configuration.
Can you please also let me know what you have selected in File -> Settings
-> Other settings -> XQuery -> Other options -> XQuery flavour?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/ligasgr/intellij-xquery/issues/203#issuecomment-233183759,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AF3fjivS1yU7AIHN1RvVaOVz5CCKyFt5ks5qWjdXgaJpZM4JLVJB
.

Hey,
Thanks for more details.
Unfortunately you'd have to respond on github portal and not via email for attachments to be visible to me.

Sorry,
Didn't realize the images would not be visible in the email.

Hey,
please raise a separate issue for WebDav module access for MarkLogic.
This would be to allow browsing the modules currently deployed onto the server and allow referencing them in the code in the IDE.
Current functionality does not allow that. To get rid of these warnings you could copy the modules that get these highlights into the workspace into one of the source directories.
I know that it's not ideal but currently that is the only way.

Ok, thanks
On Tue, Jul 19, 2016 at 8:35 AM ligasgr<EMAIL_ADDRESS>wrote:

Hey,
please raise a separate issue for WebDav module access for MarkLogic.
This would be to allow browsing the modules currently deployed onto the
server and allow referencing them in the code in the IDE.
Current functionality does not allow that. To get rid of these warnings
you could copy the modules that get these highlights into the workspace
into one of the source directories.
I know that it's not ideal but currently that is the only way.
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/ligasgr/intellij-xquery/issues/203#issuecomment-233618310,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AF3fjnxqNO07wAeXF472uW19CP3qvW67ks5qXMSRgaJpZM4JLVJB
.


Fixes issue #779

Description
I have made the changes to the code so that the newsletter gives appropriate message to the user .
Related Issues
[Cite any related issue(s) this pull request addresses. If none, simply state “None”]

Closes #779

Type of PR

[x] Bug fix
[ ] Feature enhancement
[ ] Documentation update
[ ] Other (specify): _______________

Screenshots / videos (if applicable)
for entering correct email format :

this is message .
for incorrect email :

Checklist

[x] I have gone through the contributing guide
[x] I have updated my branch and synced it with project main branch before making this PR
[x] I have performed a self-review of my code
[x] I have tested the changes thoroughly before submitting this pull request.
[x] I have provided relevant issue numbers, screenshots, and videos after making the changes.
[x] I have commented my code, particularly in hard-to-understand areas.

Additional context:
[Include any additional information or context that might be helpful for reviewers.]

Please review my PR.

@sanmarg  Please review my PR.

@varshith257 Please review my PR .


[EasyCodingStandard] Incompatibility with YamlFileLoader

Bug I get:
Warning: Undefined array key "serviceKeywords" in /var/www/project/vendor/symplify/easy-coding-standard/src/Yaml/CheckerServiceParametersShifter.php on line 67
Solution:
Change:
        /** @var string[] $serviceKeywordsProperty */
        $serviceKeywordsProperty = (new ReflectionClass(YamlFileLoader::class))
            ->getStaticProperties()['serviceKeywords'];

To
        /** @var string[] $serviceKeywordsProperty */
        $serviceKeywordsProperty = (new ReflectionClass(YamlFileLoader::class))
            ->getConstant('SERVICE_KEYWORDS');

Hi, thanks for report.
The PR is welcomed :+1:

Hi Tomas,
I forgot to mention version I use: v7.3.18 - I am not able to create PR to this version :)

Ah, that is way deprecated :) be sure to upgrade to ECS 0.9.46
It was fixed there: https://github.com/symplify/symplify/pull/2874

@TomasVotruba is there a quick way to upgrade ECS to 9.x? Like an automation of transforming yaml config file to php one?

Good question! There is a tool that automates it:
https://tomasvotruba.com/blog/2020/07/27/how-to-switch-from-yaml-xml-configs-to-php-today-with-migrify/

@TomasVotruba I am trying to convert my configs but without success. The error I get:
vendor/bin/config-transformer switch-format -i yaml -o php ecs.yaml

 [ERROR] Cannot autowire service "Migrify\ConfigTransformer\Converter\ConfigFormatConverter": argument
         "$yamlToPhpConverter" of method "__construct()" references class "Migrify\PhpConfigPrinter\YamlToPhpConverter"
         but no such service exists.
         ```

I am using ECS version 8.0 because I have to use Symfony in LTS version 4.4. What is the right way to fix it? Is there prefixed/dockerized version of config transformer?

Nevermind, I have just found this cool plugin: https://github.com/bamarni/composer-bin-plugin

That's wierd, probably some version conflict in Symplify.
If you solved it in another way, then good enough 👍
We should fix it otherwise


master.source.test.do/dist/theme.js doesnt exist

So I am trying to install this theme and the URL for the theme.js does not exist which then throws errors in Chrome. is there a new URL for the theme.js file?

master.source.test.do??? I have no idea what you are trying to do. The correct url is http://afonsof.com/jenkins-material-theme/dist/material-teal.css. You can also replace teal by a color of your choice. Take a look at http://afonsof.com/jenkins-material-theme/

Install the plugin. Go to settings page and there are two URLS one for the CSS as you point out and one for the theme.js file. By default they both point to a weird URL as what I reported. When inspecting a page load there are numerous references to said js file that fail... well because it doesn't exist.


The question about change the freq to 1min in .yaml file

❓ Questions and Help
你好，我想使用benchmark里的TRA模型跑一下cn_data_1min的分钟级别数据。我在data_handler_config中设置了freq为1min，结果运行的时候一直出现以下error:
File "/home/anaconda3/envs/Qlib/lib/python3.8/site-packages/pyqlib-<IP_ADDRESS>-py3.8-linux-x86_64.egg/qlib/data/data.py", line 90, in calendar
_calendar, _calendar_index = self._get_calendar(freq, future)
File "/home/anaconda3/envs/Qlib/lib/python3.8/site-packages/pyqlib-<IP_ADDRESS>-py3.8-linux-x86_64.egg/qlib/data/data.py", line 173, in _get_calendar
_calendar = np.array(self.load_calendar(freq, future))
File "/home/anaconda3/envs/Qlib/lib/python3.8/site-packages/pyqlib-<IP_ADDRESS>-py3.8-linux-x86_64.egg/qlib/data/data.py", line 672, in load_calendar
return [pd.Timestamp(x) for x in backend_obj]
File "/home/anaconda3/envs/Qlib/lib/python3.8/site-packages/pyqlib-<IP_ADDRESS>-py3.8-linux-x86_64.egg/qlib/data/data.py", line 672, in
return [pd.Timestamp(x) for x in backend_obj]
File "pandas/_libs/tslibs/timestamps.pyx", line 1399, in pandas._libs.tslibs.timestamps.Timestamp.new
File "pandas/_libs/tslibs/conversion.pyx", line 408, in pandas._libs.tslibs.conversion.convert_to_tsobject
File "pandas/_libs/tslibs/conversion.pyx", line 652, in pandas._libs.tslibs.conversion._convert_str_to_tsobject
ValueError: could not convert string to Timestamp
我想请问一下如何修改yaml里的freq才能正确的跑cn_data_1min这个数据集？
We sincerely suggest you to carefully read the documentation of our library as well as the official paper. After that, if you still feel puzzled, please describe the question clearly under this issue.

Currently, Qlib only provides a very limited length of range of 1min data


KNN - JavaScript Implementation

Implement KNN from scratch in

JavaScript

Create new issue according to the template if you would like to implement a different algorithm (or) in a different language.

Hey, I'd be interested in working on this, could you assign this issue to me?


docs: remove extraneous path from vscode usage

This is no longer necessary as of https://github.com/microsoft/vscode-js-debug/pull/1985, which will be released with VS Code 1.89 in 1-2 days. You may want to wait to merge until then.
Happened to look at the docs today and saw this :)

Beautiful! Thanks for this and your work on VSCode @connor4312 !

:tada: This issue has been resolved in v4.9.2
If you appreciate this project, please consider supporting this project by sponsoring :heart: :pray:


validate: accept lowercase true and false as boolean values

This resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1638325

@VasishtaShastry Would you mind adding a Signed-off-by to your commits?

@andrewschoen I've squashed commits and added Signed-off-by

jenkins test pipeline

@andrewschoen next time please use the backport-stable-3.2 label. Thanks.

jenkins test pipeline

We have no functional coverage this actually works. osd_auto_discovery is not working on ceph-volume and we stopped running ceph-disk based tests. At least this hasn't broken anything.


Binary to Gray code converter

Student, group: Yantimirov Timur, 3530901/90203
URL: https://github.com/yantimirov-timur/BinaryCodeConverter
Git approach: git-flow

`+


doc/storage: no need to escape underscore in bash examples

Just a minor fix. In the bash examples you should not escape the underscores.

This pull request didn't trigger Jenkins as its author isn't in the whitelist.
An organization member must perform one of the following:

To have this branch tested by Jenkins, use the "ok to test" command.
To have a one time test done, use the "test this please" command.

Those commands are simple Github comments of the format: "jenkins: COMMAND"

@keestux thanks! Please can you sign off your commit and then I can merge.

Like this?

@keestux perfect thanks


NTP sync

HI.
I added NTP sync with time zone management and daylight saving time.
It use my localPTZtime MicroPython module, and the definition of the zone must be entered in Posix Time Zone notation, like "CET-1CEST,M3.5.0,M10.5.0/3" for Rome or "EST5EDT,M3.2.0,M11.1.0" for New York.
Now the update occurs only at startup; one could think of repeating it every hour...

That is great thanks!

Thank you.
I would like to add some more changes (when I have time):
A problem with the network connection shouldn't be blocking - but I'd like to be notified.
It would be useful to perform time synchronization cyclically.
I'd like to add also some changes (first of all "Added 'day of week' setting") by @klaymens, who forkd a few months ago, but I don't know how to do it or how to contact him...

Agreed, I am going to get my hands on a Pico WH shortly so I can actually implement the wireless stuff and test it, had no means of doing so.
I think I can bring the changes he has made, will give it a go now.


Display the git commit hash and/or git tag for each version

It would be helpful if the soldeer.xyz website displayed the git commit hash and git tag (if available) for a specific version.
For example, version 6.7.0 is listed for solmate, but the actual solmate repo does not have a tag or release version for 6.7.0, so I am unable to determine what the version represents.

hey, unfortunately solmate is pushed using the npmjs, so i am using the npmjs versioning, will look to check if npmjs does have any commit info

My comment was more general, with solmate as an example. Sorry for the confusion!


Build up of containers eventually chokes system resources

After running several lab tasks throughout the day, the containers started by DevLab build up over time.  These eventually turn my fans into rocket ships and start choking my system:

I find I have to restart docker periodically to reclaim system performance and settle my fans.  It would be grand if we could find a way to reliably kill off these containers after using them.
If we know they are running, can we spawn a child process to kill them instead of logging that they are still running?

To help illustrate the prevalence of the issue, these are now the containers that have accumulated in the 15m since I posted the issue:

The previous versions of Devlab did not do a "headless" shutdown. After the task completed the stop and rm tasks were run before the parent process exited. We moved this to a forked process because with multiple services this shutdown process can take in upwards of 10s with some of the containers we were using.
I'm more than happy to restore the original in-process shutdown of service containers, but would like to get some other thoughts before pulling the trigger.

cc: @TomFrost

Is it possible to also kick off lab --cleanup in another process if running containers are found?  In contrast to only listing them and giving the user a command to clean them up.
⚠ These containers did not exit: ...
⚠ You can remove them with `lab --cleanup`, we'll try that now.

This would just kick off lab --cleanup along with the task that was executed.

After thinking about this for a couple days I feel like the best option is to return to in-process shutdown of service containers. It adds a few seconds but was very consistent previously.
We could go with your suggestion above, however, it's really just deferring the issue and it seems like detached processes under heavy load (which is what we're seeing) is hit or miss.

Might be the best option.  My worry with it is the same as the original issue, how long it will take.
Timing it:

2 stray containers lab --cleanup 21s
20 stray containers --lab cleanup 1m 42s

On this particular project properly pronounced with a "p", there are 5 service containers.  With it taking 5-10s per container to clean up and ~5s to start them, it seems there would be ~30s - 1m of overhead time involved in starting/stopping tasks.
If this is the best option, and truly the only option, I think it is better to leave the resource hogs for now.  My reasoning is that I'd rather stop for a couple minutes to restart docker when it is convenient for me than add a full minute to every task run.
Thoughts?

I think the addressing the serial shutdown process could be a good way to address this.
The approach previously has been to join stop and rm commands with &&, then chain those. I've tried manual startup and shutdowns on numerous containers and even on some of the "heavier" containers, run alone, the shutdown time is 3-4s.
I'm thinking if I reworked the shutdown script to run these in parallel we could 1. stop all service containers quickly and 2. be able to report failures immediately, allowing users to see them before running another process.

To preemptively address the follow-up question of why not continue to do this in a forked process?
The consistency here seems to be an increase in relics when the host system is under load. I don't see any reason to have the system create additional "threads" if I can prove spawned processes under the parent process to be efficient. If this proves itself I'll go this route, if there's a dramatic increase in time (upwards of 10s) then it may be prudent to go the route of headless forks.
Considering that services not shutting down only contributes to additional load, and thus would mean an increased chance of future services not shutting down, it seem prudent to nip this in the bud during the running process and give the user an indication of an issue before running another task.

I think this is all very sane-- I'm still curious why this is happening at all, though. It happened with Stephen yesterday as well, and I've noticed that both he and Levi use Ctrl+C to kill the lab script a lot when I pair with them, often times while the supporting containers are still starting up. Perhaps there's a window during that startup in which they can send SIGINT without triggering the shutdown process?

I thought it might be that, but even running a few back-to-back locally on my Mac I got the same thing (just running env tasks). I agree early kills should during service start should be handled, but can confirm it's not the cause.
I got the parallel shutdown process mocked out and it's fast. Does add a couple seconds but not bad on most scenarios, about as fast as (if not faster than) the startup. The best news is that it has prevented any failures in my tests thus far.
I did discover one thing though - a lot of time-suck of shutdown could be circumvented by finding better/faster containers. The Elasticsearch image on one particularly problematic project was a big pain and (alone) takes about 12-13s to shut down just using standalone docker stop.


Add examples to Channel Configuration Timestamping section

I noticed a conversation on our internal Slack workspace suggesting that the team would like to see the documentation for timestamps in the Channel Configuration docs improved:

https://github.com/real-logic/aeron/wiki/Channel-Configuration#timestamps

The team generally had to resort to looking at thee regression tests to understand how things worked:

https://github.com/real-logic/aeron/search?q=channel-rcv-ts-offset

There was a suggestion to have at least an example or two so we don't have to reverse engineer things.
The docs do have a few improvements since our team mentioned this, but no examples and I bet the subject could do with even more explanation.

Thanks, @stackedsax. We will take a look and see what can be improved.

I've added a code sample to that section of the documentation.


Enable Google Optimize on homepage

We want to run an experiment on the homepage; enable Google Optimize so the script is included.

@ethax-ross Can we close this PR now?

@gemmadallmandfe Ramone was just asking me about this and it sounded like we're going ahead with the landing home page test, so will we not want this still?

@ethax-ross was thinking this was the blur-specific version but of course this PR is just adding Optimize to the page
Yes, we are still going ahead with a home page test (just a different one than before!)

Closing this in favour of #3224


Default export prevents HMR

The default export which serves as a configuration object causes Vite to not fast refresh on changes in the story.
export function Story() {
  return
story

}
export default {
  title: 'Alternative title'
}

Whenever any code is adjusted in this file Vite will warn in the console and reload the page:
11:07:42 [vite] hmr invalidate<EMAIL_ADDRESS>Could not Fast Refresh. Learn more at https://github.com/vitejs/vite-plugin-react/tree/main/packages/plugin-react#consistent-components-exports

I'm not really sure if there's a proper workaround.

That's how react fast refresh operates, it's their limitation. Workarounds:

You don't need to define export default unless you want to customize the title.
HMR still works in other modules (e.g. components that you import into this story) - so try to keep the bulk of code outside of the story file itself.

Yeah, I figured this is unavoidable. I'll make do with the workarounds, thanks.


RiverML: update the plots less frequently

Instead of using hvplot I've directly used holoviews and DynamicMaps and Pipes. I've partitioned the observations and clusters and concatenated them to send them only every X events. Running the script with viz=True I get a refresh rate of about 50 Hz for a cadence of 100 Hz, while I get an output at about 70 Hz without any viz (same cadence).

Thanks very much, good to see! We should make this example more prominent, right?

Yes why not! It's an interesting and powerful application of streamz+holoviews. The question is, where? @jbednar any recommendation? This is what the example looks like (running the river_kmeans.py script with viz=True):

Maybe link from examples.pyviz?

Maybe this should just be a proper example on examples.pyviz.org
@martindurant would you be interested in writing a notebook that introduces streamz and its support of RiverML, and the example (online k-means clustering)? If so, I could take care of polishing the plot and publishing the example on examples.pyviz.org (creating the anaconda-project and deploying it on AE5).


Feature/update test

I think master needs to be merged into the develop branch.

could you add our linter, and make it so that only test/model.js is linted for now?


App changes made using App studio doesn't appear on other users MS teams client

I followed this document to register a Ms teams App using App studio for my exiting BOT.
Then added a Menu items as mentioner here
Now everything seems to be on my computer or I should say MS Teams client that I am using. Like I am able to see the Menu items and able to interact with the new App.
When I ask other users , who are using same Bot in their personal scope / 1:1 chat they do not see any updates or Menu items in this case.
Note: I just clicked Install Button from following options after making changes.
Does that mean it will install to only my Teams App and doesn't actually make changes to Existing BOT which is being used my many people in their personal chat ?

Here is what it looks like in App studio where scope is personal for Commands.

@rjgmail88 - bot menus are part of manifest, you need to install the bot again with updated manifest file to see bot menu. Other users will see the bot menu only after re-installing updated manifest file.

Hi @Wajeed-msft , But, other users are not able to see anything ? like no Menu items for them. Is that because I clicked Install from my App studio ? and it install it only for me ?

@rjgmail88 - Yes, you are correct. Personal scope app needs to be installed by individual. For team scope, installing updated package in team will reflect the changes for all the team members. Please download the manifest and ask another user to install in his/her personal scope to see bot menu.

@Wajeed-msft , I am confused between definition of scope here. As per my knowledge there are 2 scope personal and team , and those represents how BOT will be added/accessible in Ms teams.
ex:- I have added a bot to ms teams by using deep link:-  https://teams.microsoft.com/l/chat/0/0?users=28:{BotId}
that is called Bot in my personal scope or 1:1 chat.
Now if I would like to use it for scopes like team, groupchat I would have to create an App using studio which will involve manifest etc. Then users can add it to respective teams channel after publishing to store.
However,  when you mentioned about personal scope app needs to be installed by individual. What do they need to do ? I was able to do it because I am a developer so I use app studio and I clicked on Install button.
We need to roll this to 25K plus user from our organization so I am trying to find best way to roll this out.
Any thought on using this https://docs.microsoft.com/en-us/microsoftteams/platform/get-started/get-started-app-studio

You can use the Microsoft Teams Tenant Apps Catalog to test and distribute line-of-business applications to your organization. Please have a look at documentation on how to Publish apps to the Microsoft Teams Tenant Apps Catalog.

@Wajeed-msft , I followed above document  to distribute an App by using Apps Catalog to test.  Here are the steps:

Used appV1.0.zip to upload & publish using Tenant  catalog

User searched for appV1 in store, and downloaded it to his Ms teams web app (personal scope )

Developer sent new version of App to tenant admin appV2.0.zip that has minor changes on the functionality.

Tenant admin did reinstalled it.

User from step 2 don't see any changes reflected.

Is this expected ?
@billbliss it will be nice to have your thoughts on this as well.

@rjgmail88 - Currently, only way to get the updated app is by restarting desktop client or for web browser reloading the browser tab.


No sound with AKPanner / AKMixer / AKReverb

This may or may not be an issue, since I found a resolution, but I would appreciate some feedback on what's going on. I'm also putting it here to make it available to search engines.
I tried a very simple program to pan a single note,
func startPlaying() {

    let note = AKOscillator()
    note.start()

    let leftPan = AKPanner(note, pan: -1)

    AudioKit.output = leftPan
    AudioKit.start()

}

but strangely got no sound. The same was true if the panner was replaced by a mixer or reverb node. The solution, inspired by #755, was to move the note declaration outside of the function:
let note = AKOscillator()

func startPlaying() {

    note.start()

    let leftPan = AKPanner(note, pan: -1)

    AudioKit.output = leftPan
    AudioKit.start()

}

Why is it necessary to have the declaration outside of the function? Does this have to do with how Swift works or is it an issue for AudioKit?
And thanks for providing this terrific library!

Yeah, its a new thing where all the nodes need to be instance variables or else they're kind of forgotten.  Not sure what caused it, if it was something we did or a change in swift.

Members of the AudioKit team think that this is now the correct behavior.  Before, there was memory leaks, but now that we have that cleared up you need to have an explicit ownership of the nodes.  So I guess its a feature!? :)

Thanks for the response. Shouldn't this information be added to the documentation for AKNode?


Broken link to code of conduct

The CODE_OF_CONDUCT.md file is referenced from a badge in README.md, which expects the file to be in the repository root, but the file is actually in the other directory. So the link is broken.
Either the link is wrong or the location is wrong.  react-testing-library puts this file in the repo root.
Suggested solution:
Move CODE_OF_CONDUCT.md from other to the repo root?
(I was about to submit a PR for this, thinking the file was absent, but then I found it in other so I backed off and raised this issue instead.)

Yeah, I think the file should be moved.


[WFLY-19932] Use Testcontainers to start Kafka rather than running em…

…bedded"
https://issues.redhat.com/browse/WFLY-19932

/retest

Thanks @rhusar  rebased


Fix VectorImageLayer options types

VectorImageLayer supports mostly the same options as VectorLayer, because they both extend from BaseVectorLayer.
declutter, style (were too narrow) and background (was missing) have been synced from the base class.
Resolves #15703

@ahocevar The declutter option description I copied from the base layer is not quite accurate after all. I only tested it superficially (i.e. looks about right and doesn't crash). But now I noticed that multiple VectorImageLayers won't declutter together like VectorLayers do, when given the same declutter value. It only declutters inside the layer.
Is that a bug in VectorImageLayer or is that how it's supposed to work?

That's because a VectorImageLayer creates its own frameState for the vector renderer. I think the behavior could be changed to do what the docs say after your change without too much effort. So if you want to give it a try, please go ahead. Otherwise, please submit another pull request to fix the docs.


Bug?

00:00:34.701 --> 00:00:37.704 line:20%
[music turns dramatic]

I think it is bug if after timestamp format like "line:20%"

it parsed to:
{
  entries: [
    { id: '', from: 34701, to: 37704, text: '[music turns dramatic]' }
  ]
}

The parser does not support inline style.


Create a mock Client for testing

Create a mockable Client that allows us to perform fake Api calls, intercept the output, and send fake responses to it.
Why?:

Allows us (maintainers) to unit test kube::Api and kube_runtime
Allows us (maintainers) to unit test Api -> Service interactions (like the proposed default_namespaced in #209)
Allows users to unit test their abstractions on top of kube::Api and kube_runtime by faking a few responses

Asked for in #426.
This is something that is theoretically possible through something like tower_test, but it brings with it complications:

Client would need to take two types of Service's (one of which intercepts calls and allows shoehorning in responses)
Multiple Client types => generic parameter required in Api (since it houses the Client) - discussed in #406
The latter is a huge breaking change (though not necessarily against it unless it can be avoided)
Actually shoehorning in responses to a MockService seems hairy based on tower_test::Mock

This is a more important (and concrete) want than what is nebulously outlined in #406 - so we probably close that in favour of this testing issue.

One point on the potential breaking change, if we are to do it, we should try our best to ensure the ergonomics of it stay as close to "specify the service generic once, have it be inferred everywhere else" when modifying Api.

kube::Service was added to avoid adding a type parameter to Client. It should be possible to create a mock service (its new takes any tower Service as long as request and response type matches) and create a Client with it.
Maybe I missed something? I’m on my phone right now, so I’ll look into it later.

If it's possible to use kube::Service directly, then that would be great!
I only did some small experiments that led me to believe that we would have to add a lot to kube::Service to be able to support both a mockable mode and a normal mode, but maybe I am wrong.

The following works:
#[cfg(test)]
mod tests {
    use kube::{Api, Client, Service};

    use futures::pin_mut;
    use http::{Request, Response};
    use hyper::Body;
    use k8s_openapi::api::core::v1::Pod;
    use tower_test::mock;

    #[tokio::test]
    async fn test_mock() {
        let (mock_service, handle) = mock::pair::<Request<Body>, Response<Body>>();
        let spawned = tokio::spawn(async move {
            // Receive a request for pod and respond with it
            pin_mut!(handle);
            let (request, send) = handle.next_request().await.expect("service not called");
            assert_eq!(request.method(), http::Method::GET);
            println!("{:?}", request);
            let pod: Pod = serde_json::from_value(serde_json::json!({
                "apiVersion": "v1",
                "kind": "Pod",
                "metadata": { "name": "test" },
                "spec": {
                    "containers": [{
                        "name": "test",
                        "image": "test-image"
                    }],
                }
            }))
            .unwrap();
            send.send_response(
                Response::builder()
                    .body(Body::from(serde_json::to_vec(&pod).unwrap()))
                    .unwrap(),
            );
        });

        // `kube::Service` takes `tower::Service` with request/response of `hyper::Body`
        let service = Service::new(mock_service);
        // `kube::Client` takes `kube::Service` to avoid additional type parameter
        let client = Client::new(service);
        let pods: Api<Pod> = Api::namespaced(client, "default");
        // Make a request
        let _pod = pods.get("test").await.unwrap();
        // Asssertions ...
        spawned.await.unwrap();
    }
}

I'd like to provide something more convenient, though.

Ah, that is not bad for out of the box!
I assumed we had to build something on top of our Service, but I had approached it the annoying way of trying to get all our service layers into the mock, which probably isn't that useful for end-users, but somewhat useful for us when unit testing. Not sure if that's worth pursuing. We can test auth at a unit test level within service at the very least - and that's the layer that probably is most test-worthy.

Yeah, I'd test layers separately like I did for AuthLayer.

I had approached it the annoying way of trying to get all our service layers into the mock, which probably isn't that useful for end-users, but somewhat useful for us when unit testing.

If you want, you should be able to define a function that returns the default stack, then apply that layer, something like mock_service.layer(default_stack()) (might need to use ServiceBuilder).

Thanks, it works!
For what it's worth this is what I came up with. The code is not self-contained (also probably far from optimal), but I guess it may help somebody looking for more examples of the syntax.
#[tokio::test(flavor="current_thread")]
async fn test_k8s() {

  use std::{fs::File, io::BufReader};
  use http::{Request, Response};
  use hyper::Body;
  use kube::{Client, Service, api::{Api, ListParams, ObjectList}};
  use tower_test::{mock, mock::Handle};
  use k8s_openapi::api::core::v1::{Node, Pod};
  use super::{AsgInstance, eks::{ClientInfo, NodeInfo, PodInfo}};

  async fn mock_k8s(handle: &mut Handle<Request<Body>, Response<Body>>) {

    use kube::api::ListMeta;
    use serde::{Serialize, Deserialize};

    #[derive(Serialize, Deserialize)]
    struct ObjectListDef<T> where T: Clone,
    {
        pub metadata: ListMeta,
        pub items: Vec<T>,
    }

    let (request, send) = handle.next_request().await.expect("Service not called");

    let body = match request.uri().to_string().as_str() {
      "/api/v1/nodes?" => {
        let file = File::open("sample/nodes.json").unwrap();
        let reader = BufReader::new(file);
        let nodes: ObjectListDef<Node> = serde_json::from_reader(reader).unwrap();
        serde_json::to_vec(&nodes).unwrap()
      },
      "/api/v1/namespaces/kube-system/pods?" => {
        let file = File::open("sample/pods.json").unwrap();
        let reader = BufReader::new(file);
        let pods: ObjectListDef<Pod> = serde_json::from_reader(reader).unwrap();
        serde_json::to_vec(&pods).unwrap()
      },
      _ => panic!("Unexpected API request {:?}", request),
    };

    send.send_response(
      Response::builder()
        .body(Body::from(body))
        .unwrap());
  }

  let (mock_service, mut handle) = mock::pair::<Request<Body>, Response<Body>>();
  let service = Service::new(mock_service);
  let client = Client::new(service);

  let c = client.clone();
  let h = mock_k8s(&mut handle);
  let (result, _) = futures::join!(c.get_count(), h);
  let count = result.unwrap();

  assert_eq!(count, 1);

  let c = client.clone();
  let h = mock_k8s(&mut handle);
  let inst = AsgInstance{ id: String::from("i-06a31794b0771a1a9"), zone: String::from("us-east-1a") };
  let (result, _) = futures::join!(c.get_node(&inst), h);
  let node = result.unwrap();

  assert_eq!(node.unwrap().name(), "ip-10-169-187-100.ec2.internal");

  let c = client.clone();
  let h = mock_k8s(&mut handle);
  let n = Api::namespaced(c, "kube-system");
  let p = ListParams::default();
  let (result, _) = futures::join!(n.list(&p), h);
  let pods: ObjectList<Pod> = result.unwrap();
  let pod = pods.iter().next().unwrap().to_owned();

  assert_eq!(pod.name(), "aws-node-g8m5d");
  assert_eq!(pod.owner(), Some("DaemonSet"));
  assert_eq!(pod.phase(), Some("Running"));
  assert_eq!(pod.can_evict(), false);
}

Using tower_test::mock::Handle (like an example provided by a user):
async fn mock_get_pod(handle: &mut Handle<Request<Body>, Response<Body>>) {
    let (request, send) = handle.next_request().await.expect("Service not called");

    let body = match (request.method().as_str(), request.uri().to_string().as_str()) {
        ("GET", "/api/v1/namespaces/default/pods/test") => {
            // Can also use recorded resources by reading from a file.
            // Or create entire mock from some file mapping routes to resources.
            let pod: Pod = serde_json::from_value(serde_json::json!({
                "apiVersion": "v1",
                "kind": "Pod",
                "metadata": {
                    "name": "test",
                    "annotations": { "kube-rs": "test" },
                },
                "spec": {
                    "containers": [{ "name": "test", "image": "test-image" }],
                }
            }))
            .unwrap();
            serde_json::to_vec(&pod).unwrap()
        }
        _ => panic!("Unexpected API request {:?}", request),
    };

    send.send_response(Response::builder().body(Body::from(body)).unwrap());
}

Used like:
#[tokio::test]
async fn test_mock_handle() {
    let (mock_service, mut handle) = mock::pair::<Request<Body>, Response<Body>>();
    let spawned = tokio::spawn(async move {
        mock_get_pod(&mut handle).await;
    });
    let pods: Api<Pod> = Api::default_namespaced(Client::new(mock_service));
    let pod = pods.get("test").await.unwrap();
    assert_eq!(pod.annotations().get("kube-rs").unwrap(), "test");
    spawned.await.unwrap();
}

Now with kube::Service removed, it's pretty straightforward. We can add some helpers and come up with conventions to make this more convenient, but it's usable already.

@clux I think we don't need anything else on Client now that it can use mock Service directly. I'll continue playing with tower_test and see if I find anything better. Maybe this can be closed, and new issues can be opened for adding test helpers (not sure what yet).

Yeah, I agree. The situation is pretty good now, and improvement is likely to come from helper functions.
Side note; It feels really hard to present a nice interface for testing controllers well. So many potential flows to keep track of, even the basic test case would be something like: 1. watchevent input -> 2. reconcile start -> 3. reconcile asks something of the api 4. api mock returns canned response, 5. reconcile decides on action, 6. we verify final request.
But yeah, let's close. The core issue (and many other things this issue was originally concerned with) are dealt with :+1:

Yeah, this will be useful for small unit tests, but I don't think it's realistic to test controller's complex behavior like this. It should be possible to make stateful mocks (like POST and then GET, or change response after some condition), but it'll be painful to manage for complex scenarios.
For that, I think we should add a test helper to start a cluster for each test (#382), and some test macros for convenience.


Update grunt-contrib-jasmine to version 0.9.1

Since grunt-contrib-jasmine version 0.6 jasmine 2.0 is available.

Thanks for the pull request @Kienz


Fork mod

This is a change of functionality.
Added the ability to split DBC files on distinct option of configuration file.
New option UseFilesPerDBC

I close this, and set up a new one with the recent changes.


Performance - Nginx Stream SSL Server Vs Nginx HTTPS Server

We have observed that the performance is very low in the Nginx Stream SSL Server compared to the Nginx HTTPS Server. Why is the Stream SSL Server giving low performance? Please find the Nginx configuration below and can anyone suggest ways to boost the performance of the Stream SSL Server?
**benchmark tool: Apache benchmark
events {
use epoll;
worker_connections 65536;
multi_accept on;
accept_mutex off;
}
Stream Server
stream {
upstream https_server {
server <IP_ADDRESS>:443;
}
server {
listen 1500 ssl ;
ssl_certificate cert/server.crt.pem;
ssl_certificate_key cert/server.key.pem;
ssl_client_certificate cert/RootCA.crt.pem;
ssl_asynch on;
proxy_pass https_server;
proxy_ssl on;
proxy_ssl_session_reuse off;
proxy_ssl_asynch on;
}
}
Backend Server
http {
keepalive_timeout 50s;
connection_pool_size 1024;
keepalive_requests 10000;
access_log off;
server {
listen 443 ssl default_server;
ssl_protocols TLSv1.2;
ssl_certificate cert/server.crt.pem;
ssl_certificate_key cert/server.key.pem;
ssl_client_certificate cert/rootca.crt.pem;
ssl_session_cache off;
root /html;
index index.html index.htm index.nginx-debian.html;
server_name _;
location / {
try_files $uri $uri/ =404;
}
}
}**

Can you define how you measure performance?

I have used the Apache Benchmark (ab) tool to measure performance with the following command:
taskset -c 4 ab -s 30 -c 80 -n 10000 -Z -f TLS1.2 https://<IP_ADDRESS>:1500/test/file_64KB.html.


Type hints using custom classes ignored by pydantic

Discussed in https://github.com/samuelcolvin/pydantic/discussions/3711
Using a custom class as the type hint is ignored, and pydantic will instantiate its parent class.
Here's some example code that demonstrates what I'm seeing:
sample_data = [
    {'addr': '<IP_ADDRESS>/29'},
    {'addr': '<IP_ADDRESS>/29'}
]

class CustomNetwork(ipaddress.IPv4Network):
    def __init__(self, address, strict=False) -> None:
        super().__init__(address, strict)

class Network(pydantic.BaseModel):
    addr: CustomNetwork

def describe(name, obj):
    print(f"{name} is a {type(obj)} and contains {obj!r}")

for item in sample_data:
    x = CustomNetwork(item['addr'])
    describe("x", x)
    try:
        y = ipaddress.IPv4Network(item['addr'])
        describe("y", y)
    except ValueError as e:
        print(f"Failed to create y: {e}")
    try:
        z = Network(**item)
        describe("z", z)
    except pydantic.error_wrappers.ValidationError as e:
        print(f"Failed to create z: {e}")

The first piece of sample data works with both the original address.IPv4Network class and my subclass.  The second piece of sample data fails creation with address.IPv4Network unless strict=False is passed in the constructor, so it works only with my subclass and not the parent.  When the Network pydantic dataclass is instantiated, it uses the constructor for address.IPv4Network instead of CustomNetwork.
I expect every assignment except the second assignment to y to work in this code, however the second assignment to z also fails, despite using the custom class with the altered __init__() method.
x is a <class '__main__.CustomNetwork'> and contains CustomNetwork('<IP_ADDRESS>/29')
y is a <class 'ipaddress.IPv4Network'> and contains IPv4Network('<IP_ADDRESS>/29')
z is a <class '__main__.Network'> and contains Network(addr=IPv4Network('<IP_ADDRESS>/29'))
x is a <class '__main__.CustomNetwork'> and contains CustomNetwork('<IP_ADDRESS>/29')
Failed to create y: <IP_ADDRESS>/29 has host bits set
Failed to create z: 1 validation error for Network
addr
  value is not a valid IPv4 network (type=value_error.ipv4network)

Notably, z.addr is an address.IPv4Network object in the first test case, not a CustomNetwork object.  Pydantic bypasses my custom class and creates an instance of its parent instead.

Not really a response to what you're reporting, but a solution to your problem:
You can use https://docs.python.org/3/library/ipaddress.html#ipaddress.IPv4Interface if you want to have strict=False.


Tuya version 1.5.21 missing?

Hi,
Just got a new GU10 RGBW LED spot from MOES WB-TD5-RWW-GU10-MS. Read it used to run tuya version 1.3.21 but mine is 1.5.21 which is not in the list of tuya cloudcutter. Tried a couple of other versions without success. Are there plans to add support for this version or can I use another version from the list? Just keeping on trying would take a lot of time and potentially not lead to success. Thanks for any tips or infos!

1.5.21 is on the [known patched firmware](https://github.com/tuya-cloudcutter/tuya-cloudcutter/wiki/Known-Patched-Firmware list, and is not exploitable by CloudCutter.  Serial flashing will be your only option.

Thanks for the super quick answer! Too bad but guess my luck had to end at some point, flashed 20 tuya devices with cloudcutter and this is the second in a row where I need to resort to serial flashing.

Any BKM on how to flash ESPHome firmware using serial?

https://docs.libretiny.eu/docs/flashing/esphome/


Timesync after ReportScore error

This is another way to go, it can sleep for less than 1 sec (down to zero).
The idea is, first inside ExecuteRequest(), to compute the "missing" time to reach 110 seconds (for the servers), using the error message.
Then return that value to the main loop.
Then subtract the $SkippedLagTime.
Here my "skipped lag" is often above 1sec, so I added some extra comparisons.
It works ok, giving output like this:
!! ITerritoryControlMinigameService/ReportScore failed - EResult: 2 - {"response":{}}
-- Report score failed, trying again in 0.500 seconds...

From my observations the calculated delay already works in pretty much all cases, does it not?

Yes or no... It depends of the lag.
With my modification, it often sleeps for less than 1 sec, and it works also.
I'm still having issues when the lag is more than the time difference -- just look at the 3rd example below : the zone fight lasted 107 seconds only for the server's clock ; since the lag is more than 3*2, my code does the same as yours, but it could have been 4.5 seconds less.
To be really good, the code would need to differentiate between sending and receiving lags, which may in turn be overkill !

This round lasted 110.000 seconds including 0.900 seconds lag.
!! ITerritoryControlMinigameService/ReportScore failed - EResult: 93 - {"response":{}}
!! API failed - x-error_message: User joined zone 17462 at<PHONE_NUMBER>, and now it's<PHONE_NUMBER>, which is too soon
-- Report score failed, trying again in 0.550 seconds...

This round lasted 110.000 seconds including 0.700 seconds lag.
!! ITerritoryControlMinigameService/ReportScore failed - EResult: 93 - {"response":{}}
!! API failed - x-error_message: User joined zone 17398 at<PHONE_NUMBER>, and now it's<PHONE_NUMBER>, which is too soon
-- Report score failed, trying again in 0.650 seconds...

This round lasted 110.001 seconds including 7.500 seconds lag.
!! ITerritoryControlMinigameService/ReportScore failed - EResult: 93 - {"response":{}}
!! API failed - x-error_message: User joined zone 17430 at<PHONE_NUMBER>, and now it's<PHONE_NUMBER>, which is too soon
-- Report score failed, trying again in 7.500 seconds...

This round lasted 110.000 seconds including 0.800 seconds lag.
!! ITerritoryControlMinigameService/ReportScore failed - EResult: 93 - {"response":{}}
!! API failed - x-error_message: User joined zone 17453 at<PHONE_NUMBER>, and now it's<PHONE_NUMBER>, which is too soon
-- Report score failed, trying again in 0.600 seconds...

 I don't know if it's intended or not ? On the other hand, I'm not getting the inventory items (hats and armors) anymore since the 1st boss !

Closing as repository will be archived.


Update Validator after addition of trip-to-trip and route-to-route transfers

In Q4 2021, trip-to-trip and route-to-route transfers were added to the specification (PR#278).
The validator needs to be updated with 4 additional optional fields in transfers.txt:

from_trip_id - Foreign key referencing trips.trip_id
to_trip_id -  Foreign key referencing trips.trip_id
from_route_id -  Foreign key referencing routes.route_id
to_route_id - Foreign key referencing routes.route_id

No additional rules need to be added.
Definition of done:

the ForeignKeyViolationNotice is triggered if these fields are referencing a value that doesn't exist.
the UnknownColumnNotice is not triggered by these four fields.

@isabelle-dr please let me know if I should be updating the status of things on this board. I didn't see a way for me to set that status from Current Sprint to Under Review.

@isabelle-dr please let me know if I should be updating the status of things on this board. I didn't see a way for me to set that status from Current Sprint to Under Review.

I just updated it :) Thank you @KClough !