IPS rules not auto downloading

After me forgetting to set one of the offloading that needed to be disabled I did a factory reset from the web GUI to start the setup fresh after running this for a few days as only got my firewall Thursday last week
https://www.applianceshop.eu/security-appliances/security-appliances-desktop-and-wallmountable/pfsense-based-desktop/opnsense-a10-quad-core-ssd-desktop.html
It will not download the rules automatically. This is also the same in my OPNsense VM where that was setup correctly from the start. Running both on 16.1.7 LibreSSL

@L1ghtn1ng can you try to run the following in a shell for me?
/usr/local/opnsense/scripts/suricata/rule-updater.py
ls -asl /usr/local/etc/suricata/rules/
cat /var/cron/tabs/root

To check if the manual operation functions normally and verify if the cron job is installed correctly.

Will do that when I get home from work. But if I login to the web GUI and manually download the rules it does download them. Just does not do it automatically. But will still try what you have asked

@AdSchellevis this is the output from running the following commands you asked.
root@OPNsense:~ # /usr/local/opnsense/scripts/suricata/rule-updater.py
root@OPNsense:~ # ls -asl /usr/local/etc/suricata/rules/
total 2072
  2 drwxr-xr-x  2 root  wheel    1024 Mar 20 19:15 .
  2 drwxr-xr-x  4 root  wheel     512 Mar 17 23:17 ..
  2 -rw-r-----  1 root  wheel      98 Mar 20 19:18 OPNsense.rules
156 -rw-r-----  1 root  wheel  158395 Mar 22 22:48 abuse.ch.feodotracker.rules
304 -rw-r-----  1 root  wheel  283314 Mar 22 22:48 abuse.ch.sslblacklist.rules
 24 -rw-r-----  1 root  wheel   23261 Mar 22 22:48 abuse.ch.sslipblacklist.rules
  2 -rw-r--r--  1 root  wheel    1295 Mar 16 06:55 app-layer-events.rules
 54 -rw-r-----  1 root  wheel   54563 Mar 22 22:48 botcc.rules
  2 -rw-r--r--  1 root  wheel    1498 Mar 16 06:55 dns-events.rules
  4 -rw-r-----  1 root  wheel    2513 Mar 22 22:48 dshield.rules
192 -rw-r-----  1 root  wheel  196535 Mar 22 22:48 emerging-exploit.rules
432 -rw-r-----  1 root  wheel  413569 Mar 22 22:48 emerging-malware.rules
 70 -rw-r-----  1 root  wheel   70194 Mar 22 22:48 emerging-mobile_malware.rules
100 -rw-r-----  1 root  wheel  100867 Mar 22 22:48 emerging-scan.rules
  2 -rw-r--r--  1 root  wheel    1763 Mar 16 06:55 modbus-events.rules
  2 -rw-r-----  1 root  wheel    1963 Mar 22 22:48 rbn-malvertisers.rules
  2 -rw-r-----  1 root  wheel    1934 Mar 22 22:48 rbn.rules
720 -rw-r-----  1 root  wheel  715776 Mar 18 19:55 rules.sqlite
  0 -rw-r-----  1 root  wheel       0 Mar 18 19:55 rules.sqlite.LCK
root@OPNsense:~ # cat /var/cron/tabs/root
# DO NOT EDIT THIS FILE - OPNsense auto generated file.
# (Cron version -- $FreeBSD$)
SHELL=/bin/sh
PATH=/etc:/bin:/sbin:/usr/bin:/usr/sbin
# Order of crontab fields
#   minute      hour        mday        month       wday        command
# Origin/Description: IDS/ids rule updates
0    4    *    *    *   /usr/local/sbin/configctl ids update

@L1ghtn1ng ok, that looks normal. it kind of looks like cron is not running, can you try this:
ps fax | grep cron

There should be a "/usr/sbin/cron -s" in the list.

@AdSchellevis Here is the output
root@OPNsense:~ # ps fax | grep cron
30180  -  Is       0:00.00 minicron 240 /var/run/ping_hosts.pid /usr/local/sbin
30379  -  I        0:00.10 minicron: helper /usr/local/sbin/ping_hosts.sh  (min
30556  -  Is       0:00.00 minicron 3600 /var/run/expire_accounts.pid /usr/loca
30717  -  I        0:00.01 minicron: helper /usr/local/etc/rc.expireaccounts  (
31190  -  Is       0:00.00 minicron 86400 /var/run/update_alias_url_data.pid /u
31381  -  I        0:00.00 minicron: helper /usr/local/etc/rc.update_alias_url_
69358  -  Is       0:00.53 /usr/sbin/cron -s
88042  0  S+       0:00.00 grep cron

I also decided to check the rules and here is the output for that
root@OPNsense:~ # ls -asl /usr/local/etc/suricata/rules/
total 2072
  2 drwxr-xr-x  2 root  wheel    1024 Mar 20 19:15 .
  2 drwxr-xr-x  4 root  wheel     512 Mar 17 23:17 ..
  2 -rw-r-----  1 root  wheel      98 Mar 23 00:47 OPNsense.rules
156 -rw-r-----  1 root  wheel  158395 Mar 23 04:00 abuse.ch.feodotracker.rules
304 -rw-r-----  1 root  wheel  283314 Mar 23 04:00 abuse.ch.sslblacklist.rules
 24 -rw-r-----  1 root  wheel   23261 Mar 23 04:00 abuse.ch.sslipblacklist.rules
  2 -rw-r--r--  1 root  wheel    1295 Mar 16 06:55 app-layer-events.rules
 54 -rw-r-----  1 root  wheel   54563 Mar 23 04:00 botcc.rules
  2 -rw-r--r--  1 root  wheel    1498 Mar 16 06:55 dns-events.rules
  4 -rw-r-----  1 root  wheel    2513 Mar 23 04:00 dshield.rules
192 -rw-r-----  1 root  wheel  196535 Mar 23 04:00 emerging-exploit.rules
432 -rw-r-----  1 root  wheel  413569 Mar 23 04:00 emerging-malware.rules
 70 -rw-r-----  1 root  wheel   70194 Mar 23 04:00 emerging-mobile_malware.rules
100 -rw-r-----  1 root  wheel  100867 Mar 23 04:00 emerging-scan.rules
  2 -rw-r--r--  1 root  wheel    1763 Mar 16 06:55 modbus-events.rules
  2 -rw-r-----  1 root  wheel    1963 Mar 23 04:00 rbn-malvertisers.rules
  2 -rw-r-----  1 root  wheel    1934 Mar 23 04:00 rbn.rules
720 -rw-r-----  1 root  wheel  715776 Mar 18 19:55 rules.sqlite
  0 -rw-r-----  1 root  wheel       0 Mar 18 19:55 rules.sqlite.LCK

@L1ghtn1ng cron seems to be running fine and your files also seem to be updated at the right time, are you sure you have an issue? (all seemed to be updated at 04:00)

@AdSchellevis It should be updating every 4 hours and it is not. As if you look at my previous report of the settings for cron you will see that I have set it for every 4 hours.

@thelinuxfr ok, I see your issue, its configuration. its now set to update every day at 4:00 (0    4    *    *    *). What report are you referring to?

@L1ghtn1ng I see your issue, its configuration. its now set to update every day at 4:00 (0    4    *    *    *). What report are you referring to?

@L1ghtn1ng if you want to update every 4 hours, you could set this in the hour box  : 0,4,8,12,16,20

@AdSchellevis Thanks for that my bad. One more thing regarding the interface that the IPS listens on would that be enough to block everything just listening on the wan interface our will I need to enable on the lan interface as well?

@L1ghtn1ng  no problem, I just misunderstood the first time ;)
It should normally be enough to only enable the wan site to block all offending packets.
Only thing to take into account is that if threads are coming from the internal network you can't tell exactly from which internal machine they come (at the wan side all is transmitted from the wan ip).

@AdSchellevis Okay thanks for that. I have found another issue that I think it a bug but not related to this one. Do you want me to open a new issue for it?

@L1ghtn1ng yes please, I will close this one then.


Handle different combinations of package purity

(I'm working on this, just documenting what I'm working on)
There are three types of Python packages that we have, each requiring slightly different build steps. The types of packages and build steps required are:

Pure Python packages (no compiled extensions) with no platform-/python-version-specific dependencies.

single sdist
single pure wheel
single noarch conda package

Pure Python packages, with platform/python-version-specific dependencies

single sdist
single pure wheel
{pythons} × {win-32, win-64, linux-64, osx-64} conda packages

Packages with compiled extensions:

single sdist
{pythons} × {win-32, win-64, osx-64} wheels compiled on the latest OS versions
{pythons} × {linux-64} wheels compiled using the manylinux docker image
{pythons} × {win-32, win-64, linux-64, osx-64} conda packages

I've done type 3. in desktop-app, so the other variants are a simpler kind of it.
I'm going to test if we can define global hard-coded switches in workflows that say which jobs to run (I think we probably can with job-level if conditions based on workflow-level environment variables).
If we can, then we can make a single workflow file differing between projects only in these switches (and I guess hard-coded repository names), containing all the steps even if they are not in use. Barring that we can just include the jobs in a repository that are actually in use, but verbatim rather than combining them where possible (e.g not combining the sdist and wheel jobs even for the case of pure packages), to ease keeping any changes in sync between repositories.
So the steps we'll need are:

a. sdist
b. pure wheel
c. impure mac and windows wheels
d. impure manylinux wheels
e. single noarch conda package
f. impure conda packages

Package type 1.  has a, b, e. Type 2 has a, b, f. Type 3 has a, c, d, f.

No beans on job-level if conditionals using the workflow-level env context.

Ok, so since it's not possible to block out whole jobs and it's a lot of duplication to split into multiple jobs, I'll try to make it just 2 build jobs - one for manylinux and one for sdist, other wheels and conda, with the job matrix being the configuration you need to change for different 'purity classes'.

Doesn't the last post in the linked thread say you can use the env context in if conditionals, just not secrets?

Thanks for documenting this, @chrisjbillington. And well done getting type 3 working.

Ok, so since it's not possible to block out whole jobs

You mean using environment variables, per your earlier post? That's been my experience.

Doesn't the last post in the linked thread say you can use the env context in if conditionals, just not secrets?

This comment implies you cant use the env context in a job-level if. Though I tried it and got no luck.

This is done!


Coinbase platform Invaid Config issue Hassio 0.64 upgrade

After I upgraded from Hassio from 0.63.3 to 0.64 getting "Invalid Config" and below error message in home-assistant.log, please help. Thanks.
2018-02-26 22:13:09 ERROR (MainThread) [homeassistant.setup] Error during setup of component coinbase
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/homeassistant/setup.py", line 148, in _async_setup_component
    component.setup, hass, processed_config)
  File "/usr/lib/python3.6/asyncio/futures.py", line 332, in __iter__
    yield self  # This tells Task to wait for completion.
  File "/usr/lib/python3.6/asyncio/tasks.py", line 250, in _wakeup
    future.result()
  File "/usr/lib/python3.6/asyncio/futures.py", line 245, in result
    raise self._exception
  File "/usr/lib/python3.6/concurrent/futures/thread.py", line 56, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/lib/python3.6/site-packages/homeassistant/components/coinbase.py", line 51, in setup
    api_key, api_secret)
  File "/usr/lib/python3.6/site-packages/homeassistant/components/coinbase.py", line 77, in __init__
    from coinbase.wallet.client import Client
  File "/usr/lib/python3.6/site-packages/coinbase/wallet/client.py", line 38, in <module>
    from Crypto.Hash import SHA256
  File "/usr/lib/python3.6/site-packages/Crypto/Hash/SHA256.py", line 23, in <module>
    from Crypto.Util._raw_api import (load_pycryptodome_raw_lib,
  File "/usr/lib/python3.6/site-packages/Crypto/Util/_raw_api.py", line 32, in <module>
    from Crypto.Util.py3compat import byte_string
ImportError: cannot import name 'byte_string'
2018-02-26 22:13:39 WARNING (SyncWorker_7) [soco.data_structures_entry] DATA STRUCTURE UPGRADE FAIL. Unable to upgrade music library data structure to music service data structure because an entry is missing for DidlFavorite in DIDL_NAME_TO_QUALIFIED_MS_NAME. This should be reported as a bug.

That is fix with 0.64.1


Fixing the PostgreSQLImageOpenShiftProvisioner application secret management

Description
Fixing the PostgreSQLImageOpenShiftProvisioner so that the application secret is moved to the application interface and used by the provisioning workflow as expected (i.e. implementing the HasSecrets interface).
The changes add the test class to the ProvisionerCleanupTestCase tests, which would be failing without these fixes, due to the fact that the secret isn't cleaned up properly.
Fix #74
Type of change

[x] Bug fix (non-breaking change which fixes an issue)
[ ] Feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change
[x] Chore (non-breaking change which doesn't affect codebase;
test, version modification, documentation, etc.)

Checklist

[x] Code contributed by me aligns with current project license: Apache 2.0
[x] Pull Request contains a description of the changes
[x] Pull Request does not include fixes for multiple issues/topics
[x] Code is self-descriptive and/or documented
[ ] I have implemented unit tests to cover my changes
[x] I tested my code in OpenShift

/test intersmash-e2e

Thank you for this MR! Merging.


WIP - MTA-2850: Installing and Configurating OCP Local

JIRA

MTA-2850

PREVIEW

MTA OCP Local

Structure and content are great. A few minor suggestions, but overall, readable and straightforward.


Main dev

close s3 connection in objectstore sitemover, to avoid too many open connections
download events before start athenaMP

Great, thanks!


Check point dimension on tell

Fixes #487
Added check for dimension of point and space.
Check is added in utils.check_x_in_space(), could also be added in contains function of space, but then Exception message in check_x_in_space should be adapted.

Codecov Report

Merging #501 into master will increase coverage by 0.03%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master     #501      +/-   ##
==========================================
+ Coverage   86.24%   86.28%   +0.03%
==========================================
  Files          22       22
  Lines        1658     1662       +4
==========================================
+ Hits         1430     1434       +4
  Misses        228      228

Impacted Files
Coverage Δ

skopt/utils.py
97.29% <100%> (+0.07%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2f9eda4...20f4adb. Read the comment docs.

Looks good to me. Thanks! What do others think?

Thanks for this @mikesmyk! Congratulations on your first contribution to scikit-optimize and your first ever PR on GitHub! 🎆🍻


MouseEvent is not defined in SSR

My map is loading and working fine in normal mode but when i use server side rendering and serving i am getting this error __metadata("design:paramtypes", [MouseEvent]), MouseEvent is not defined.
Has anyone else faced this issue.
Steps to reproduce and a minimal demo of the problem
After adding map and drag to chose co-ordinate option, I added angular-universal for SSR. After that
npm run build:ssr - everything ran fine. Next
npm run serve:ssr gave error.
_Use stackblitz.com or similar -- try this template as a starting point: https://stackblitz.com/edit/angular-google-maps-demo
What steps should we try in your demo to see the problem?
angular-google-maps, Angular, & any other relevant dependency versions
Angular 7
"@agm/core": "^1.0.0",

Probably a bug that was already fixed, but not yet deployed. Any workaround to make it work?


spelling corrected

I want to merge this change because...

Screenshots

Pull Request Checklist

[ ] Privileged views and APIs are guarded by proper permission checks.
[ ] All visible strings are translated with proper context.
[ ] All data-formatting is locale-aware (dates, numbers, and so on).
[ ] Database queries are optimized and the number of queries is constant.
[ ] Database migration files are up to date.
[ ] The changes are tested.
[x] The code is documented (docstrings, project documentation).
[ ] GraphQL schema and type definitions are up to date.
[ ] Changes are mentioned in the changelog.

Nice catch! Thanks for contributing to Saleor!


Memory Error

I am facing memory issue when train the system about 14 classes of images and i need to train the system for 345 classes of the image. Please suggest me something effective.

Ohh thats a lot ..!! try reducing the batch size of the training step.
But the accuracy will decrease. 345 classes of how many images in each class.

All classes of quick draw dataset.
On 2 April 2018 at 16:22, Chandra Kanth<EMAIL_ADDRESS>wrote:

Ohh thats a lot ..!! try reducing the batch size of the training step.
But the accuracy will decrease. 345 classes of how many images in each
class.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/ck090/Google_Quick_Draw/issues/2#issuecomment-377910313,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AVyrSas395OhxvfF_NOCEh6ohGbpc73yks5tkgn8gaJpZM4TC5RQ
.

Well than it's hard to train that many using a normal CPU, you might have to use a GPU enabled system that is going to be capable of training on that many classes.
Otherwise there is no other option but try something along the lines of Transfer learning!


Fix horde end time

I noticed that the bot reports the horde is rampaging for two hours longer than it should be.  The default is for the horde to be from 22:00 until 04:00 on the following morning, but the bot thinks the horde runs until 06:00.
It might be better to read the end hour for the night from the config file similar to how it reads the blood moon frequency, but that's a bigger change I don't have the time to make or test right now.

Thanks for the correction, LGTM.


Add support for .NET Trimming

Resolves #39.
Work-in-progress. It looks about right to me but I haven't put it through its paces just yet to see if it (a) still works, and (b) trims the appropriate stuff.

is it worth screwing around with MergeWith or should I just upload the coverage for the latest target (in this PR, net5.0)?

Latest target is fine by me. The reason I ommited version from folder is so that actions don't need to be updated when target is changed.

now I just need to figure out what the actual filepaths are 😅

home/runner/work/ValveKeyValue/ValveKeyValue/ValveKeyValue/ValveKeyValue/bin/Release/netstandard2.1/ValveKeyValue.dll
You want to add Release/net5.0

405 not allowed? 😕

It's broken in master too 🤷

that's better

Release workflow probably needs path update too.
Merge whenever you want.

Shall we target net6 with this instead? It's LTS.

Sure.
I'm thinking it might also be worth redoing this from scratch with the latest linker and analyzers.


Relay join request forwarding table limitations

The implementation of the FilterListReq MAC command rejects rules that have length greater than zero for the rule index 0:
https://github.com/Lora-net/SWL2001/blob/c4f386ac54b3e35f45b8ba7502207faa6f634921/smtc_modem_core/lr1mac/src/relay/relay_rx/relay_rx_mac_parser.c#L363-L367
However, the specification is more nuanced than this, specifically:

The way I interpret these rules is that:

It is legal to actually use index 0 to filter a prefix (i.e. length > 0) - this is due to the third point.
There is a special meaning for using index 0 and length 0 - it alters the default forwarding behavior (such that it could become filter instead of forward when nothing matches), but it is not a rule change in itself.

Section 8.6 also hints at this possibility:

My interpretation is that for index 0 and length 0 a 'default action' slot is updated via a request, while a length > 0 request updates the 16 entries table. I would argue that this interpretation allows for all 16 slots to be used with /16 prefixes, but I am open to alternative interpretations.

This interpretation is wrong, due to rule 2.


ci: Require cluster-wide connectivity before running tests

This PR introduces two new features to the cilium-health subsystem.
The first set of commits ensures that cilium-health properly reports connectivity failures in dual-stack setups. Before this PR, cilium-health would probe the connectivity of secondary endpoint addresses, but not report any failures in the health API and thus also not display the failures in cilium status or cilium-health status. To fix this, the API is extended to expose the status of the secondary endpoint addresses probes, and the formatting and output logic is improved to better display and report these kind of failures.
The second set of commits introduces a new undocumented agent flag called cluster-health-required-for-readiness flag. It inhibits agent readiness until cluster connectivity has been established. It is used in CI to ensure that no tests are executed before Cilium has established cluster-wide connectivity. Without this flag, Cilium will declare readiness even though connectivity to other nodes might not even have been set up yet (e.g. one example that caused flaky tests is delayed IPv6 route setup in direct routing mode).
Fixes: #17895
Fixes: #18014

/test

/test

/test

/test

/test

/test

net-next failed likely due to the k8s version change https://jenkins.cilium.io/job/Cilium-PR-K8s-1.23-kernel-net-next/5/ - it's known to be broken at the moment.

Rebasing on master to pull in the net-next fixes, to ensure this doesn't break anything on the new pipeline either.

Travis failed with a new issue. Filed https://github.com/cilium/cilium/issues/18204

/test-runtime

Addressed two nits, rebased and removed the commit which un-qurantines the test. Actual diff:
diff --git a/pkg/health/client/client.go b/pkg/health/client/client.go
index ab2b2f088466..fe93c9834d55 100644
--- a/pkg/health/client/client.go
+++ b/pkg/health/client/client.go
@@ -274,7 +274,7 @@ func GetAllHostAddresses(node *models.NodeStatus) []*models.PathStatus {
 }

 // GetEndpointPrimaryAddress returns the PrimaryAddress for the health endpoint
-// within node. If node.Host is nil, returns nil.
+// within node. If node.HealthEndpoint is nil, returns nil.
 func GetEndpointPrimaryAddress(node *models.NodeStatus) *models.PathStatus {
        if node.HealthEndpoint == nil {
                return nil
@@ -283,7 +283,8 @@ func GetEndpointPrimaryAddress(node *models.NodeStatus) *models.PathStatus {
        return node.HealthEndpoint.PrimaryAddress
 }

-// GetEndpointSecondaryAddresses returns the secondary host addresses (if any)
+// GetEndpointSecondaryAddresses returns the secondary health endpoint addresses
+// (if any)
 func GetEndpointSecondaryAddresses(node *models.NodeStatus) []*models.PathStatus {
        if node.HealthEndpoint == nil {
                return nil

/test

/mlh new-flake Cilium-PR-K8s-GKE

/test-gke

/ci-multicluster


Unable to access /userinfo endpoint when access token does not contain PII

In this scenario, I dont want the access token to contain any PII (personally identifiable information). If required, I want the resource server to get PII using  /userinfo endpoint.
As suggested by @fhanik (see discussion), this has been done using UAA configuration option
  token:
    claims:
      exclude:
      - user_name
      - email

But, when passing the access token without PII to the /userinfo endpoint, I get a 500 Internal Server Error.
Stacktrace indicates a null pointer exception while getting principal
: Caused by: java.lang.NullPointerException
2016-09-14T15:51:34.16+0200 [APP/0]      OUT [2016-09-14 13:51:34.159] cloudfoundry-identity-server - ???? [http-nio-8080-exec-4] .... ERROR --- HomeController:        at org.cloudfoundry.identity.uaa.account.UserInfoEndpoint.extractUaaPrincipal(UserInfoEndpoint.java:55)
2016-09-14T15:51:34.16+0200 [APP/0]      OUT [2016-09-14 13:51:34.159] cloudfoundry-identity-server - ???? [http-nio-8080-exec-4] .... ERROR --- HomeController:        at org.cloudfoundry.identity.uaa.account.UserInfoEndpoint.loginInfo(UserInfoEndpoint.java:50)

Looking at UaaTokenServices
[...]
        Authentication userAuthentication = null;
        // Is this a user token?
        if (claims.containsKey(EMAIL)) {
            UaaUser user = userDatabase.retrieveUserById((String)claims.get(USER_ID));
            UaaPrincipal principal = new UaaPrincipal(user);
            userAuthentication = new UaaAuthentication(principal, UaaAuthority.USER_AUTHORITIES, null);
        } else {
            authorizationRequest.setAuthorities(authorities);
        }

        OAuth2Authentication authentication = new UaaOauth2Authentication(accessToken,
                                                                          IdentityZoneHolder.get().getId(),
                                                                          authorizationRequest.createOAuth2Request(),
                                                                          userAuthentication);
        authentication.setAuthenticated(true);
        return authentication;
    }
[...]

it seems that if email (ie PII) is not in the access token, then no userAuthentication/principal is set, which whould explain the exception.
Is there a reason why userAuthentication is only set if email (ie PII) is in the token ?
Thanks

Hello @fhanik . Do you have any advice about how to deal with this issue ?

@s-bortolussi it's a bug. email is a required claim, I'm fixing it right now.

Finished via e59544a60a2df62874e07a086bda1cd63d2d789e

Thanks @fhanik for the fix.
However, I am wondering if it would not make more sense to use a more standard claim such as "sub" to represent user id instead of the non standard "user_id" claim.

@s-bortolussi sub is the subject. in case of client credentials, this should be the client_id.
we use the presence of user_id to  tell us if it is a user token or not.

Thanks @fhanik for your response


preventing apps using previous version to crash

Prevent this crash :
com.facebook.react.bridge.UnexpectedNativeTypeException: TypeError: expected dynamic type `int64', but had type `null'
        at com.facebook.react.bridge.ReadableNativeMap.getInt(Native Method)

when using one of the latest release.

Yes, you are right. Even better I used :
PropTypes.oneOfType([
      PropTypes.object,
      PropTypes.number,
    ])

To support both object and Stylesheet


Adding globals

Hi, I am coming across the following error: myVar is not defined.
Here myVar is a global variable. Is there a way to indicate this in the grunt file?
The relevant part of my gruntfile:
mocha_istanbul: {
      coverage: {
        src: 'build/**/*.js',
          options: {
            mask: '**/*.js',
            coverage: true,
            coverageFolder: "test/coverage",
            print: "detail",
            reportFormats: ['lcovonly']
          }
        }
      },

Thanks.

that's a mocha issue, you should try adding:
{
options:{
mochaOptions: ['--globals', 'myVar']
}
}

and see if it works for you.

Do you mean change it to:
mocha_istanbul: {
      options: {
        mochaOptions: ['--globals', 'myVar']
      },
      coverage: {
        src: 'build/**/*.js',
          options: {
            mask: '**/*.js',
            coverage: true,
            coverageFolder: "test/coverage",
            print: "detail",
            reportFormats: ['lcovonly']
          }
        }
      },

If so, I still get the same issue.
Any other suggestions?

are your test files in build/**/*.js? you should be adding only the tests files in src. also mask option overrides the src.

Ah I found my error, the build directory was not complete due to an error elsewhere.
Thanks for the help!


How do I "install" flowplotter

Hi, do I just extract the master.zip in a suitable directory or in the silk directory?

Any directory is fine. Then simply cd into the flowplotter directory and you should be able to run through some of the examples.


Update workflow for new diagnostics

Description
Timings and model-space diagnostic execution signature are updated corresponding to https://github.com/JCSDA-internal/mpas-jedi/pull/704.
The primary change is that DiagnoseModelStatistics.py replaces writediagstats_modelspace.py.  The ensemble obs-space verification has a higher wall-time per member than before, while the single-member obs-space verification has lower wall-time.  That is caused by the new code spending more time in the IODA file reading step for more than one member, while the actual binning is parallelized, reducing masking and computation wall-time.
Tests completed

[ ] 3dvar_OIE120km_WarmStart
[ ] 3denvar_OIE120km_WarmStart
[ ] 3dvar_OIE120km_ColdStart
[ ] 3denvar_O30kmIE60km_WarmStart
[ ] eda_OIE120km_WarmStart

Tests are in progress.  Will update PR radio boxes if/when they complete.


[New Plugin] - Alchemist Lab - v1.0.0 - Sololegends

Alchemist Lab
Helper for the Alchemist Lab minigame
Features

Shows colored orbs next to each order denoting ingredients
Highlights mix levers in order to create the potion
Highlights proper post-processing tool

Hey, let's collaborate over https://github.com/hex-agon/mastering-mixology

I'm fairly confident that mine at present is more feature rich. I've started working on automatic balancing of points and prioritization for certain point type as well.


help message

In help message,
--modes
should be
--mode
clipkit --version
clipkit 1.1.3

clipkit --help

optional arguments:

  -m, --modes <smart-gap,                     trimming mode

Hi @haruosuz,
First, thank you very much for your interest in ClipKIT.
In version 1.1.4, we have fixed the typo. Thank you for pointing this out to us.
All the best,
Jacob


Revert "Reverts #473. (#508)"

This reverts commit 49bc7d7fc25fa566db4cc66f08a0dc023ca8d1a0.

This change is

@rocketraman,
Thanks for your contribution.
To ensure that the project team has proper rights to use your work, please complete the Contribution License Agreement at https://cla.microsoft.com.
It will cover your contributions to all Microsoft-managed open source projects.
Thanks,
Microsoft Pull Request Bot

@rocketraman, thanks for signing the contribution license agreement. We will now validate the agreement and then the pull request.
Thanks, Microsoft Pull Request Bot

@rocketraman please describe how you fixed and confirmed regression #493.  Also we will need a unit test for to ensure the regression is fixed.

@rocketraman  This snippet in apimodel.json should do it.
{
  "apiVersion": "vlabs",
  "properties": {
    "orchestratorProfile": {
      "orchestratorType": "Kubernetes",
      "kubernetesConfig": {
        "networkPolicy": "azure"
      }
    }
  }
}

@rocketraman We have made changes to the acs-engine repo, where we have removed test comparison with the expected files. Please pull the latest changes and resubmit.

@rocketraman we will be closing this PR as there has been no response from you

@shrutir25 I have rebased the pull request. I'd recommend a close review -- I didn't have time to validate or test my rebased changes.

It looks like it needs another rebase...

So that I'm not chasing a moving target, if I rebase this again, will someone actually look at it within a reasonable timeframe?

@rocketraman Part of the issue is you haven't answered @anhowe 's comment:

@rocketraman please describe how you fixed and confirmed regression #493. Also we will need a unit test for to ensure the regression is fixed.

@rocketraman Part of the issue is you haven't answered @anhowe 's comment:

@rocketraman please describe how you fixed and confirmed regression #493. Also we will need a unit test for to ensure the regression is fixed.

#493 has already been fixed by pull #632. Regarding the test, as I mentioned above I don't have time to create a test for it right now, unfortunately. Part of the issue is, the way the testing is done has completely changed and its not obvious how to even run them, let alone add to them.

Since #632 has merged we should be able to move forward to try to take this now. Can you please rebase? It would be nice to add an example cluster definition that uses both custom vnet and azure CNI plugin. If that would be possible. We could then reference that example in our test sweet to check for regressions.

@rocketraman @JackQuincy any thoughts on doing this rebase + example configs as requested? I'd be happy to take over this PR to get it merged -- it has an impact on my current deployment.

@rocketraman @JackQuincy any thoughts on doing this rebase + example configs as requested? I'd be happy to take over this PR to get it merged -- it has an impact on my current deployment.

Sorry for the delay on this -- I'm going to try a rebase this morning.

@JackQuincy @dmccaffery I rebased this. However, I tried to deploy a cluster using the updated code and got a failure like this:
VM has reported a failure when processing extension 'cse0'. Error message: \"Enable failed: failed to execute command: command terminated with exit status=127\"

I see that the deployment has two masters -- one with name k8s-master-36147952-0/cse0 (which failed) and one with name k8s-master-36147952-0 (which succeeded). This doesn't seem related to this pull, but I wanted to report it here just in case.

@rocketraman I would be very surprised if it wasn't related to this pull.  that is our setup script that runs on the master. defined at src\github.com\Azure\acs-engine\parts\kubernetesmastercustomscript.sh. Not sure what 127 status code means but you should be able to find more info if you ssh onto the box and look in these locations:
/var/log/azure/custom-script/handler.log
/var/log/azure/Microsoft.Azure.Extensions.CustomScript/2.0.3/CommandExecution.log
/var/log/azure/cluster-provision.log
sudo journalctl -u kubelet

@JackQuincy
I'm not sure what I'm seeing here...
In /var/log/azure/custom-script/handler.log:
time=2017-06-05T16:38:19Z version=v2.0.3/git@f21ef81-clean operation=enable seq=0 event="executing command" output=/var/lib/waagent/custom-script/download/0
time=2017-06-05T16:38:19Z version=v2.0.3/git@f21ef81-clean operation=enable seq=0 event="failed to execute command" error="command terminated with exit status=127" output=/var/lib/waagent/custom-script/download/0
time=2017-06-05T16:38:19Z version=v2.0.3/git@f21ef81-clean operation=enable seq=0 event="enable failed"
time=2017-06-05T16:38:19Z version=v2.0.3/git@f21ef81-clean operation=enable seq=0 event="failed to handle" error="failed to execute command: command terminated with exit status=127"

No errors in  /var/log/azure/Microsoft.Azure.Extensions.CustomScript/2.0.3/CommandExecution.log.
In /var/log/azure/cluster-provision.log:
/bin/bash: /opt/azure/containers/provision.sh: No such file or directory

In  journalctl -u kubelet:
-- No entries --

@JackQuincy That fixed it, thanks. I was able to deploy a cluster using the custom VNET process. Note that I did not try a default install, but hopefully this gets us 95% (or 100) of the way there!

So I tried to verify integration with azure-cni plugin and I wasn't able to get a second node up and running while doing azure cni plugin and the changes here. I had one node working flawlessly but the second node wouldn't come up. It is highly possible this is due to my lack of expertise in networking in general. Can you deploy a cluster with

"networkPolicy": "azure"

in the kubernetes config and see if everything is working. This is why we pulled it out in the first place.

@JackQuincy You mean with a custom subnet, or just using the default settings?

@rocketraman I mean using the functionality this PR enables.

@JackQuincy It doesn't even deploy correctly for me:
{"code":"DeploymentFailed","message":"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-debug for usage details.","details":[{"code":"BadRequest","message":"{\r\n \"error\": {\r\n \"code\": \"PrivateIPAddressIsAllocated\",\r\n \"message\": \"IP configuration /subscriptions/x-x-x-x-x/resourceGroups/foo-Test-Kube/providers/Microsoft.Network/networkInterfaces/k8s-master-22043857-nic-0/ipConfigurations/ipconfig1 is using the private IP address <IP_ADDRESS> which is already allocated to resource /subscriptions/x-x-x-x-x/resourceGroups/foo-Test-Kube/providers/Microsoft.Network/networkInterfaces/k8s-agentpool1-22043857-nic-1.\",\r\n \"details\": []\r\n }\r\n}"}]}

It seems to be trying to allocate the same IP address to both the master and an agent.
My config:
      "kubernetesConfig": {
        "networkPolicy": "azure",
        "clusterSubnet": "<IP_ADDRESS>/17",
        "serviceCidr": "<IP_ADDRESS>/18",
        "dnsServiceIP": "<IP_ADDRESS>"
      }

I don't have any additional insight.

@JackQuincy @rocketraman
I looked into this, and the problem is fundamentally the implementation of getFirstConsecutiveStaticIPAddress in defaults.go: https://github.com/rocketraman/acs-engine/blob/eda452529c1a18a636066afce304b333a02d90ef/pkg/acsengine/defaults.go#L276
I created a test implementation using your parameters to show the issue:
https://play.golang.org/p/0UilZ8QG62
This resolves to <IP_ADDRESS>, which is then used to statically allocate an IP to the master.
Since the masters spin up AFTER the agents, and the CNI allocates 128 IPs per agent NIC (by default) dynamically to be used for pod IPs, <IP_ADDRESS> ends up dynamically allocated before the master.
Somehow the static IPs need to be allocated first, so that the agent IPs aren't dynamically allocated over the same range, or, as a stop gap, the getFirstConsecutiveStaticIPAddress needs to be updated to use the end of the cluster subnet range and hope that dynamic allocations don't reach it.

As a side note; this works just fine in the event that a custom vnet is used and the FirstConsecutiveStaticIP is set to near the end of the cluster subnet range.

@jackquincy @ofiliz @rocketraman what do we need to move this forward? I can attempt to fix getFirstConsecutiveStaticIPAddress myself in a separate PR. Adding @ofiliz since he was the original author of the getFirstConsecutiveStaticIPAddress method that is causing issues here.

@dmccaffery what is needed is for the original reason we reverted it, namely it broke the azure cni network plugin, to be fixed. It sounds like you have figured out how to do that though I haven't been able to verify. And I'm not sure if that is what you were saying fixing getFirstConsecutiveStaticIPAddress does that or something else.  If fixing getFirstConsecutiveStaticIPAddress fixes the integration with the azure cni network plugin then we could move forward with this. Side note the team has been heads down on an internal goal and we are getting close to finishing that and should be able to give more time to the PRs here soon.

@dmccaffery I think I understand the problem you are describing, but I don't see how this is in any way related to PR #632 or getFirstConsecutiveStaticIPAddress. All that function does is to compute dynamically what used to be a static constant. The value of that statically allocated IP address did not change. The ordering of IP allocation for masters vs. agents also did not change. So it seems to me, if this bug reproduces (I can't repro it), it should be a bug that predates #632 changes.
Having said that, I'll look into how we can fix this. Is there anything special you do to reproduce the problem? Do you create clusters with multiple masters, or custom cluster subnets? Sorry if I missed if you already described this. I've read the entire thread but this issue seems to have been re-used to track multiple separate issues.

@dmccaffery Never mind, I think I now understand what you meant. I somehow missed the significance of <IP_ADDRESS>/17 in the example.

@dmccaffery This is a race condition that is more likely to reproduce for smaller subnets. It is not really caused by getFirstConsecutiveStaticIPAddress, as the template shouldn't take a dependency on what the static IP addresses can be, but the fact that that function puts static IP addresses in the first few IP addresses of the subnet if the subnet prefix length is >16 makes the race condition much more likely to reproduce.
This commit should fix the race condition:
https://github.com/ofiliz/acs-engine/commit/283d61203f4f77effb27677bed12f919682c3204
@anhowe We need to talk about how we want to allocate static IP addresses for the scale-up scenario to work. Once we agree, I can send a PR that includes that logic plus the above commit.

Folks, it's not clear what the PR provides and multiple issues are being discussed here, which makes it impossible understand the scope of the work and map into our release roadmap.
I'm closing this and ask you follow this process when contributing:

Open an issue to file a bug or feature request—this is really important as it provides a method to collaborate before committing to it.
If an ACS maintainer agrees with the proposal, open a PR that clearly states what the PR provides and what GitHub issue(s) it addresses.

We now have GitHub templates in place for both issue filing and pull requests, which should help make clear what information we need.

Re-opening

@rocketraman,
Thanks for having already signed the Contribution License Agreement. Your agreement was validated by Microsoft. We will now review your pull request.
Thanks,
Microsoft Pull Request Bot

@ofiliz  If your race condition patch fixes the CNI plugin issue, why not just push it to this branch (https://github.com/rocketraman/acs-engine/tree/rg/cidr-revert-revert)? That way @JackQuincy can verify that everything works correctly, and this can be merged. Alternatively, with your permission, I'm fine with cherry-picking your commit into this branch, if that is what is needed to move this forward.

@seanknox : this is the other PR that was important; if aM moves to Calico it won't be.

Just rebased this again.
@seanknox @JackQuincy @colemickens how do you guys want to move this forward? @dmccaffery seems to have identified the problem and @ofiliz appears to have fixed it, but someone who has a setup that can replicate the issue needs to confirm @ofiliz's solution.

I tried to set it up once in the past and I'm not sure I did it right. @anhowe @dmitsh  If someone has time can they set this up and check if it works now?

I agree with @seanknox that this issue has been overloaded with multiple unrelated asks. So I created #882 to clearly identify the deployment failure that @dmccaffery analyzed above. I submitted a PR that addresses that issue specifically. Please test and let me know whether it works for you.

@ofiliz Your fix is based on master. Does that mean that master has the same issue with the CNI plugin? Why then was #473 reverted in the first place?

@rocketraman I'm not sure if I understand your question correctly. But yes, CNI plugin is on master. The original PR caused a regression when CNI is enabled, so it was reverted. I resubmitted the clusterSubnet part of it, plus the proper changes necessary to support CNI a month ago.

@ofiliz So it sounds like we have strong reason to believe this will work with the CNI plugin. I should have some time today to try it out. If that works I'd be happy to get this merged

@acs-bot test this please

@acs-bot test this please

@acs-bot test this please

Going to hold on this for now.

Can you provide a bit more context? Are other changes happening that will make this obsolete? Is there too much uncertainty around the change? Something else?

I need this badly!

Understood. I don't have time to verify this isn't breaking the CNI plugin right now. Can someone try out specifying both? and make sure pod to pod communication still works/the pods start up. Before this broke the cluster completely if both were specified and that is why we pulled it out.

@ofiliz updated the cni so it should work I just want someone to verify with this PR.

So as I need this I rebased the branch onto master this morning and did some tests.
It seems to work all right with networkPolicy: none (what I'm currently using for my deployments).
But as far as I understand from the previous messages the problem lies with networkPolicy: azure so I tried it and it failed :
sylvain@ubuntu-1604-dev:~/gopath/src/github.com/Azure/acs-engine[cidr-revert-revert]$ az group deployment create --name "${RGName}-deployment" --resource-group "${RGName}" --template-file "_output/${env}/azuredeploy.json" --parameters "_output/${env}/azuredeploy.parameters.json"
Deployment failed. {
  "error": {
    "code": "SubnetIsFull",
    "message": "Subnet /subscriptions/xxxxxxxxxxxxxxxxxxxxxxxxxx/resourceGroups/xxxxxxx-xxxxxxx-dev-RG/providers/Microsoft.Network/virtualNetworks/xxxxxxx-EUW-xxxxxx-VNet/subnets/k8s2-master-EUW-xxxx-xxxxxxxxxxxxxxwith address prefix <IP_ADDRESS>/26 is already full.",
    "details": []
  }
}

Rebased branch can be found at sylr/acs-engine@b10ea23f0e2fedf44b995bc72a252d65cb53710f

@sylr It seems you specified a /26 subnet, which is not big enough to create a cluster. I thought I had placed a validation check for this. Are you running the latest acs-engine? Please specify at least a /23 for a minimum one worker node cluster. Or just leave it blank for defaults.

I rebased @rocketraman branch on top of this morning's master.
Why does it require a /23 subnet ? I don't plan on having 510 masters in my cluster, 3 would suffice, the rest of available IPs is more than enough for cluster IP (services).

@sylr the azure cni plugin pre allocates ips for containers on the nodes. So you will need at least 100, as that is the default of what k8s allows on a node. I'm not sure if you need that many per master as well be definitely per agent.

@rocketraman can you update the description of what this PR accomplishes? I think it enables customization of pod and service networks, but want to make sure.

@rocketraman can you update the description of what this PR accomplishes? I think it enables customization of pod and service networks, but want to make sure.

Done.

Rebased again...

@seanknox are you still requesting changes here?
@rocketraman  there are some build breaks in this last rebase. Can you fix them?

@rocketraman there are some build breaks in this last rebase. Can you fix them?

Done.

Rebased again. Ping. @seanknox

Could we please stop merging things into master that would make this conflict until this is resolved ?

... and rebased again.

... and rebased again.  Either close this pull request, merge it, or provide feedback. Right now, as far as I can tell, the blocker is @seanknox .

Hi @rocketraman, apologies for the delay.  @seanknox has been heads down and super busy, so we will discuss in our scrum tomorrow, and prioritize this PR with @jackfrancis.

@lachie83 Thanks for the review -- I believe I've addressed all of your comments.

Review status: 0 of 17 files reviewed at latest revision, 13 unresolved discussions.

pkg/acsengine/defaults.go, line 298 at r5 (raw file):
Previously, lachie83 (Lachlan Evenson) wrote…
Does this work when the boundary isn't a 0? For example <IP_ADDRESS>/27

Fixed.

Comments from Reviewable

Review status: 0 of 17 files reviewed at latest revision, 13 unresolved discussions.

pkg/api/vlabs/validate.go, line 409 at r5 (raw file):
Previously, lachie83 (Lachlan Evenson) wrote…
Should we validate to confirm that the DnsServiceIP is Not the first IP of ServiceCidr as I believe that will cause an issue in this configuration.

Fixed

Comments from Reviewable

LGTM - @rocketraman thank you so much for this!

Awesome! Just for posterity: most of the credit goes to @ashb and @alxlchnr who did the bulk of the work on this pull request. For the most part, I just kept dumbly rebasing it.
I do have one follow-up pull request here to add a missing test into validate_test.go.


Error: Path </kit-ids/undefined/data.json> not found.

I got this error today after a fresh installation of the project.

could you detail all the steps?


Adding more / better tests

One thing I notice is that functions do not always do what you expect them to do.
For instance, the is_in_scope() function should return true or false based on the fact if an element from the open_elements stack is present or not. This should probably be unit-tested, as there is currently no guarantee that this function works properly (and apparently, it does not).

This is an on-goimg issue so i close this for now


Update Webhooks Rotate Secret path

The Rotate Secret call should webhooks/rotate-secret/id not webhooks/id/rotate-secret
https://nylas.atlassian.net/browse/TW-2580
License
I confirm that this contribution is made under the terms of the MIT license and that I have the authority necessary to make this contribution on behalf of its copyright owner.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (d5fb9f8) 78.53% compared to head (1945c49) 78.53%.

Additional details and impacted files
@@            Coverage Diff            @@
##               main     #212   +/-   ##
=========================================
  Coverage     78.53%   78.53%
  Complexity      199      199
=========================================
  Files            29       29
  Lines           722      722
  Branches         69       69
=========================================
  Hits            567      567
  Misses          117      117
  Partials         38       38

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Is there a way to close a connection after an elapsed time of idling/sleeping

I have found during load testings (2000 concurrent users) that  mysqljs /mysql connection pool uses tons of connections (more than 3000)  and those connections are not really closed on client side after the run is over, even I call connection.release() after each query. They are just put into sleep mode.  It relies on mysql server to destroy those connections after a certain idling time (by default 24 hours on Aurora).
In most applications I worked on, the client is responsible for disconnecting the connection after idling/stuck/sleep, not by the db server. So to me, this library behaves differently. There is a node mysql-connection-pool-manager module  created to make up this gap by periodically destroying the long idling connections. However, I still want to use mysqljs/mysql as my codebase is written on this.
My question is why mysqljs/mysql does not offer this option to do the same cleaning up on client side? The pool.end() does this but this should be only invoked when the application terminates.
 I am afraid if I call destroy, then the pool won't be able to reuse the connection and cause performance issue.
Also, regarding the high connection usages, does the connection.release() a synchronous call? because I notice one user sometimes needs 2 connections even I execute my queries sequentially (excluding the connection.release()).
Please help.
Thanks.
Richard

Someone already created a PR for this: #2218

Waiting some news about this feature and the PR, I have implemented something like this
TS
  const getConnectionFromPool = (
    pool: mysql.Pool
  ): Promise<mysql.PoolConnection> => {
    return new Promise((resolve, reject) => {
      pool.getConnection(
        (err: mysql.MysqlError, connection: mysql.PoolConnection) => {
          if (err) {
            debug(`Cant get connection from pool`)
            return reject(err)
          }

          // This is a provissory solution waiting this pull request https://github.com/mysqljs/mysql/pull/2218
          const connectionIdleTimer = (connection as any).__idleCloseTimer
          if (connectionIdleTimer) {
            clearTimeout(connectionIdleTimer)
          }

          ;(connection as any).__idleCloseTimer = setTimeout(() => {
            debug('close connection due inactivity')
            connection.destroy()
          }, 30 * 1000)

          return resolve(connection)
        }
      )
    })
  }

JS:
  const getConnectionFromPool = pool => {
    return new Promise((resolve, reject) => {
      pool.getConnection(
        (err, connection) => {
          if (err) {
            debug(`Cant get connection from pool`)
            return reject(err)
          }

          // This is a provissory solution waiting this pull request https://github.com/mysqljs/mysql/pull/2218
          const connectionIdleTimer = connection.__idleCloseTimer
          if (connectionIdleTimer) {
            clearTimeout(connectionIdleTimer)
          }

          connection.__idleCloseTimer = setTimeout(() => {
            debug('close connection due inactivity')
            connection.destroy()
          }, 30 * 1000)

          return resolve(connection)
        }
      )
    })
  }


Editable examples

It woulld be really nice if the code in the examples were editable in the browser. Are there plans for implementing such feature?

I think it will be hard since Stencil is more of a compiler than a framework/library. However, I think I have seen some discussions in the Stencil slack channel about running the compiler in the browser, and that would probably make this easier :slightly_smiling_face:

@jgroth I stumbled across storybook.js in the docs of several webcomponent libraries, have you seen that? https://storybook.js.org/
Examples where you can play with the components using the "Knobs" tab below the canvas:

https://esri.github.io/calcite-app-components/?path=/story/overview-home--page
https://bulmil.netlify.app/

Yes, I've played around with it several times but I never got it working in a satisfying way with Stencil :disappointed: Maybe it's working better these days? Last time I tried it was at the start of 2020 I think and then I gave up and started on this instead :smile:

@jgroth I'm currently on a slow wifi but every time I try use Storybook apps they are dead slow so I'm not for it. Although I liked the knobs stuff. That also opens up for exploratory testing of a component, even though i really like the way the "knobs" are on this for the Lime Elements button http://lime-documentation-lime-elements.s3-website-eu-west-1.amazonaws.com/versions/29.1.0/#/component/limel-button
I guess together with components like <limel-example-value/> you could build a utility component that can reflect over a component and bind a "form" presenting all the available "knobs" of a component? Then it's more like Storybook knobs, even though not "editable" like codepen or similar, but still really useful for playing around! :)

Yeah that's a good idea. It would be quite easy to iterate all the props and generate som fields allowing a user to change them 🙂 All the metadata is available

At least for primitive types, lists and objects will be harder 🤔


 Can we just make it redirect to the index? I'm sure I'm not the only one with the old address cached in his browser.

That works too! I'll make a JS redirect to preserve the in-page anchors


add popup window for submitting approval while clicking on mail option under detailed page of debit note with draft status

issue closed


Release Google.Cloud.RecaptchaEnterprise.V1Beta1 version 2.0.0-beta07

Changes in this release:
New features

Add Fraud Prevention settings field (commit d54c222)

Created release for Google.Cloud.RecaptchaEnterprise.V1Beta1-2.0.0-beta07


.cache/... in git(hub) diffs

Just one example:
https://github.com/puemos/hls-downloader-chrome-extension/commit/9c04ca1c93d427e94241239003412f4c4ef5ca22
https://github.com/puemos/hls-downloader-chrome-extension/blob/9c04ca1c93d427e94241239003412f4c4ef5ca22/.cache/03/21c8bd04e39c80d057a33d0b7bacb3.json
Notice enormous amount of trash it generated.
.cache/... should never, ever be present on a git server

Absolutely! it was an unfortunate mistake, was fixed in a419b4ff92fee020594d1962d4011f67d6fb1607


Tools: Send web joystick axes via WebSocket instead of XHR

https://github.com/commaai/openpilot/issues/25996

Why not add websockets as a pipenv dependency?

Why not add websockets as a pipenv dependency?

I couldn't find where python packages get installed on device, was guessing it's in Agnos somewhere

Oh, it would be in https://github.com/commaai/agnos-builder/blob/master/userspace.

I forgot this might have been good in the new AGNOS update


velocity_from_position: Handle n-d arrays

Previous discussion in #68.
This issue is to discuss whether and how should velocity_from_position handle n-d arrays. Some specific questions:

Is 2-d sufficient for most use cases or is n-d also useful? The second obvious dimension here are trajectories.
If not, what are the use cases for n-d?

One can imagine any other dimension such as ensemble, depth, etc. To me it seems that 2d is just a special case of n-d.

I think the function can be modified in a few small ways to accept n-d arrays.

Overall we should follow the philosophy of the relevant functions in jLab that accept 2-d arrays (but by expecting time to be in the column direction) as well as cell arrays which is the data structure used to handle Lagrangian data.

see https://github.com/Cloud-Drift/clouddrift/issues/68#issuecomment-1381062954


🔥 remove Cirrus CI jobs

Description
cda-tum/mqt-core#553 transitioned the reusable MQT workflows from Cirrus CI back to GitHub Actions.
As such, the remaining Cirrus CI workflows can be removed.
Checklist:

[x] The pull request only contains commits that are related to it.
[x] I have added appropriate tests and documentation.
[x] I have made sure that all CI jobs on GitHub pass.
[x] The pull request introduces no new warnings and follows the project's style guidelines.

A corresponding CD run can be found here: https://github.com/cda-tum/mqt-qcec/actions/runs/7991187722


Add custom FileType

How can I add a custom file type?
I tried the following but without succes.
static MyClass()
{
    MimeAnalyzers.PrimaryAnalyzer.Insert(EPS);
}

static void Check()
{
    var type = uploadedFile.InputStream.GetFileType(); // returns null but should return the EPS file.
}

// https://www.garykessler.net/library/file_sigs.html
private static readonly FileType EPS = new FileType(
    new byte?[] { 0x25, 0x21, 0x50, 0x53, 0x2D, 0x41, 0x64, 0x6F,
        0x62, 0x65, 0x2D, 0x33, 0x2E, 0x30, 0x20, 0x45,
        0x50, 0x53, 0x46, 0x2D, 0x33, 0x20, 0x30 },
    "eps",
    "application/postscript");

I checked the file with an hex viewer and the magic bytes are correctly configured above.
Anything I'm missing?

Nevermind everything works as expected, was misconfiguration on my part.


Use with Webpack?

Is this the correct way to employ your plugin with webpack?
const cucumber = require( 'cypress-cucumber-preprocessor' ).default;
const webpack = require( '@cypress/webpack-preprocessor' );

module.exports = ( on /* config */ ) => {
  on( 'file:preprocessor', webpack());
  on( 'file:preprocessor', cucumber());
};

@leegee not sure, as I've never used it with webpack. Do you get an error? Could you give us more information? :)
A repository with a reproduction would be fantastic

You can use
on('file:preprocessor', (file) => {
    if (file.filePath.match(/\.(js|jsx)/g)) {
      return webpack(options)(file)
    } else {
      return cucumber()(file)
    }
})

Thanks!

Very cool, thanks for helping @lobor :-)


Fix the hardcoded logic in list method check, and pageable item check

Now this change is not correct. The paging mechanism should only work on the "serialized names". We should not change that part. Adding pageable extension is a workaround to make something happen using existing logic. If we have to tweak the code to make it work, we should consider a more proper, a more official way to do it, instead of patching things up on a workaround

Good point, revert the change in common pard, and only change the mgmt IsListMethod func.


Use a responsive Wear layout

New devices on the market are at 233dp, already larger than the Large Round emulator.  So switch to responsive layout, instead of hardcoded for 192 & 227 dp.

Nice!


Plugin breaks when markdown contains < or >

The plugin does not transform < and > to legal JSX.
Please see:
https://codesandbox.io/p/sandbox/throbbing-water-zcxqs6?file=%2Fsrc%2Ftest.md%3A6%2C32

I have fixed it, and you can pass the latest version 0.2.10.
https://github.com/geekris1/vite-plugin-react-markdown/commit/2267d95b99454b816952c3588c7c8b811884ad37


Cannot start the GUI

Describe the bug
Hi excellent developers, I found this software would be greatly helpful for me. However, after I downloaded its Windows version I cannot start it successfully due to the following error message:
FileNotFoundError: [Errno 2] No such file or directory: 'C:\Users\bzhao92\AppData\Local\Programs\peptdeep\_internal\alphabase\constants\const_files\common_constants.yaml'
I looked up my file explorer and cannot find the alphabase folder.
To Reproduce
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error

Expected behavior
A clear and concise description of what you expected to happen.
Logs
Please provide the log (see the peptdeep terminal on where to find it).
Screenshots
If applicable, add screenshots to help explain your problem.
Version (please complete the following information):

Installation Type [e.g. One-Click Installer / Pip / Developer]
If no log is available, provide the following:

Platform information

system    [e.g. Darwin]
release   [e.g. 19.6.0]
version   [e.g. 10.15.7]
machine   [e.g. x86_64]
processor [e.g. i386]
cpu count [e.g. 8]

Python information:

peptdeep version [e.g. 0.1.2]
[other packages]

Additional context
Add any other context about the problem here. Attached log files or upload data files if possible.

Thanks for reporting, and apologies for the delayed response. We have found a solution and will release it in the next couple of days.

should be fixed by release 1.3.1: https://github.com/MannLabs/alphapeptdeep/releases/tag/v1.3.1


[ADP-2342] miniature TxSet database

[x] add DBTxSet
[x] implement update
[x] implement select
[x] test getTxById

Comments

Issue Number

ADP-2342

bors try

bors r+

bors r+

bors merge


fix: Catchable should be the right generics, not ().

Thanks for providing this PR!  Will take a look - but any chance you could you provide me with a code snippet of the failing code before this PR that this PR fixes?  Thank you very much

Thanks for providing this PR! Will take a look - but any chance you could you provide me with a code snippet of the failing code before this PR that this PR fixes? Thank you very much

like this.
attempt {
    // do something
    return 1  // here will be an error: Cannot convert value of type 'Int' to closure result type '()'
}.catch { error in
    print(error)
}


Unify chart downsampling behaviour with android app

Base discussion:
https://github.com/ruuvi/com.ruuvi.station/issues/369

Created on the wrong repo. :/  Closed.


[Question] Direction of movement

Hello excellent work, You know an algorithm or technique to get the direction of movement from the matrix? Simple direction like up, down, left or right?
thanks

@amsolucionesweb  Thanks! Great question. I've been thinking about this myself also.
Not as of right now. But I'm thinking about including this in 2.x - I'll keep you posted once I roll that release out.

ok @maniart, for my part if I find a solution, I will share it here.

as of right now, I'm thinking something along the lines of finding the center point in a "cluster" of "changed" values in the matrix, and compare that to the previous center point and calculate the direction of a vector between the two. Planning to mess with this idea soon @amsolucionesweb

@maniart I think it is a good approach, I think that we should look for an optimal dimension of the matrix to minimize noise and maybe just take as a reference the coordinates whose value is closest to zero.


Test expired AND called cover

We want to be very sure this is correctly handled as it seems like an error-prone corner case.  Keep in mind collection classes (and maybe LevelDB) may have problems with concurrent modification of iterators.  I think we may need to referentially specify cover orders in this case.

Please note that all LevelDB iterators are const iterators. Do not use them for modification.

The problem is we need two iterators to walk covers by call price and expiration.  This ticket is just to remind me that if we don't explicitly handle the behavior (and write a test to make sure that the handling is correct), quite conceivably an order that's both margin called and expired simultaneously could end up either being filled twice, or yanked out of a collection that has a live iterator (the documentation of most languages' collection frameworks has big red BEWARE UNDEFINED BEHAVIOR warnings when it comes to modifying collections that have active iterators).

I'm not familiar with the new market engine, but note that it should never modify any LevelDB, and all state changes (including index updates) need to be staged to the pending state.
Relatedly, note that @bytemaster's implementation of the new short indexes in https://github.com/BitShares/bitshares/commit/d076871b8e74efae4e69a535b96da392f8ea73f3 and https://github.com/BitShares/bitshares/commit/b37671d64f03c4993eff4f20604c5267f458d381 is incorrect, because it is modifying the chain state directly instead of the pending state--making it so those changes cannot be rewound. I have left that as-is for now while you guys develop, until we have time to sit down and think about how to do it correctly.

Thanks for bringing this up.  I'll fix it.

Actually, my implementation is not incorrect.   The feed records are all managed by the pending state so when ever you "undo" a store feed record it should revert the indexes.   You will note that the index values are ONLY changing within chain database.   So it doesn't need to be fixed.

@theoreticalbts @bytemaster Upon review I think the original indexing is not as problematic as I thought. It is viable to modify indexes on store like that if they are not needed until the next block, which always the case here since the market runs before feed updates are evaluated. However, there still seem to be some issues:

The market_status db is directly modified in an unrewindable way
There are not corresponding index updates for remove<feed_record>
get_active_feed_price is expensive, and this indexing looks generally expensive as well. I would wager that feed updates are by far the most common operation on the network, so the indexing should be done once at the end of applying the block for all dirty assets, rather than for every store that happens.

if they are "fully" reindexed always, then they should work for arbitrary stores

Rewinding the main order table as needed and rebuilding all order indexes in every block is a workable approach, but horribly inefficient.

the indexing should be done once at the end of applying the block for all dirty assets

A feed movement can cause a clean market to become dirty -- e.g. relative order moving into territory where it overlaps another order, a margin call being triggered when there's already an overlapping order that can fill it, or a short at the feed moving into an existing order.
I definitely agree that all feed updates in a block should be queued and executed in a batch at a single point in the processing of the block.  Furthermore we only need to reindex / dirty markets when a feed update actually results in changing the median value.
But hold off on this for another day or so, I'm pretty deep into patching it now.

A feed movement can cause a clean market to become dirty

Yes I should have been more clear--I indeed meant any assets which are now "dirty" due to feed updates, not the currently existing "dirty markets" notion that the code already uses for deciding when to execute the market engine.

MC test has it.


PrettifyTreeprocessor AttributeError

This code section near the bottom of markdown.treeprocessors raises AttributeError if pre[0].text is None.
# Clean up extra empty lines at end of code blocks.
pres = root.iter('pre')
for pre in pres:
    if len(pre) and pre[0].tag == 'code':
        pre[0].text = util.AtomicString(pre[0].text.rstrip() + '\n')

The check should be if len(pre) and pre[0].tag == 'code' and pre[0].text is not None or isinstance(pre[0].text, str), but that may be too strict.

I encountered this with a custom  Block/Inline processor. The band-aid fix was just to ensure these elements had text.
for pre in root.iter("pre"):
    if pre[0].text is None:
        pre[0].text = ""

Oh, I got it. pre[0].text.rstrip() raises an error is pre[0].text is None.
I believe this has not been an issue thus far as internally we always assign a string value to the text attribute of an element. The question is: do we want to require all extensions to do the same or not? If not, then we should apply the suggested fix. if so, then we can leave it alone.
Finally, the relevant code is here for future reference.

By default ET.Element(tag).text is None. But you could add a requirement that it is a string. Thanks for linking to the code.

I was curious how the serializer handles a None value so I checked the code:
https://github.com/Python-Markdown/markdown/blob/a2e4788f8d1850f59699ce0ab500688031ea1571/markdown/serializers.py#L160-L168
When text is None , if text evaluates to False  and the serializer simply skips writing out the text content and moves on to child elements (if any) and the closing tag. It makes sense for an element with text of None to simply be empty (no different that text being an empty string). Therefore, there is no reason to require an element's text attribute to be of type str.
Finally, the question remains whether the prettifier needs to take any action on an empty code block. In other words, do we want

or

If the former, then the suggested fix will work fine. If the latter, then we need a slightly more sophisticated fix. Checking Balemark (compare this with this) is looks like the first option is acceptable.
Given the above, a PR is welcome adding and pre[0].text is not None to the check.

Thanks, I'll make a PR later :)

Thanks for your help! :+1:
Minor note, this also affects non-empty blocks, i.e. blocks with children but no text, such as
x
. But I don't think that changes anything here.


Restrict Domain Templates to a User

This is a request, not an issue.  We have PowerDNS-Admin working without issue, but would like to restrict Domain Templates to a specific user (i.e. - User1 is assigned template CorpDNSTemplate1, and User2 will not be able to see this template in their list.)
I wasn't able to find any information in prior issues/requests or in the configuration to be able to this.  Can this be added to the application?
Best,
Bobby

Accounts or Users would be fine either way, just need a way to restrict templates so that all are not seen by all users/accounts.
Bobby

So if I was to put a bounty on this feature of let say $100 USD, would that help to get this going?
Bobby

Khanh,
What would it take to get this feature built into an upcoming release?  I really need this feature, and I'm willing to increase the bounty (within reason) to make this happen.
Bobby

If I was to increase a bounty to $250 USD, would that get some traction on this request?
Bobby

Hello,
I can put 100$ more on this feature request. It is very important for us too.
Thank you

Hello,
This request is stale for too long. Regarding our policy to cleanup this project, we are closing it. As this is a Feature Request, it will be listed in #1178
Regards
PowerDNS Admin Team
See #1153 for the policy


Climate entity error: this entity is unavailable

Hi everyone,
In the last days I'm having some problems with this integration. The climate entity is unavailable but the new sensors seems to work fine.
I have a Ravelli stove, I'm running v0.2.4 of Micronova Agua IOT integration and I tried to remove and add the pellet stove again and again a lot of times.
Could the cause of the problem be that the stove is off and I haven't turned it on yet since I reinstalled the integration a few days ago? It seems strange to me because the climate entity has always been available even during the summer when the stove was off.
Here's the logs:
2023-10-19 00:35:38.171 ERROR (MainThread) [homeassistant.components.climate] Error adding entities for domain climate with platform aguaiot
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 507, in async_add_entities
    await asyncio.gather(*tasks)
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 752, in _async_add_entity
    await entity.add_to_platform_finish()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1057, in add_to_platform_finish
    self.async_write_ha_state()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 779, in async_write_ha_state
    self._async_write_ha_state()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 879, in _async_write_ha_state
    state, attr = self._async_generate_attributes()
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 822, in _async_generate_attributes
    attr.update(self.state_attributes or {})
                ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/climate/__init__.py", line 316, in state_attributes
    hass, self.current_temperature, temperature_unit, precision
          ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/aguaiot/climate.py", line 208, in current_temperature
    return self._device.get_register_value(self._temperature_get_key)
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'AguaIOTHeatingDevice' object has no attribute '_temperature_get_key'
2023-10-19 00:35:38.192 ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved

@vincentwolsink here's the diagnostic file: Home Assistant Config Entry.json.txt. Honestly I don't know why in the file the aguaiot version is 0.2.1 when on HACS I see that I don't have updates and the last installed version is 0.2.4.


Fix: Don't try to round when mean is nil

Oh, we already have a test suite, so we should definitely add a test here if doable and ship it :)


basic documentation for repeat properties

quick documentation for Button.repeatDelay and Button.repeatInterval
I didn't link it in too many places as it's javascript only currently

Add a configuration section in the controller page (at the bottom).

Sounds good, just a list of links? I don't think the cards used for the other sections work with js only functions / properties

Right, let’s do links for now.
Get Outlook for iOShttps://aka.ms/o0ukef

From: Joey Wunderlich<EMAIL_ADDRESS>Sent: Friday, February 22, 2019 1:24:24 PM
To: Microsoft/pxt-common-packages
Cc: Peli de Halleux; Comment
Subject: Re: [Microsoft/pxt-common-packages] basic documentation for repeat properties (#711)
Sounds good, just a list of links? I don't think the cards used for the other sections work with js only functions / properties
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoft%2Fpxt-common-packages%2Fpull%2F711%23issuecomment-466516185&data=02|01|jhalleux%40microsoft.com|5753870003ea4483660c08d698fb5ab9|72f988bf86f141af91ab2d7cd011db47|1|0|636864602658454508&sdata=z7AwicuYUUN9zsdvhKUxFAMPZfJul4RGJa109wV0nVg%3D&reserved=0, or mute the threadhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAD-4KThfeleA9LqjwAcwHCvJkp2sz-fNks5vQEPogaJpZM4bKOWl&data=02|01|jhalleux%40microsoft.com|5753870003ea4483660c08d698fb5ab9|72f988bf86f141af91ab2d7cd011db47|1|0|636864602658454508&sdata=t6LR6MwH0ZX1jHUSIZoD4Ful5Y%2FWPabb%2Bw0%2Bo5i2iOg%3D&reserved=0.


Thinking about plugins

How to you guys think about the entire plugin story cordova is currently offering?
As discussed with @subhagpo offline, creating cordova plugins is hard for people who have just started doing cordova based development. When creating plugins there are plenty of pitfalls and that entire area is a kind of undocumented.
In my opinion taco should support a generator to scaffold a graceful plugin as standalone 'project'. Thinking of somthing like
taco create com.foo.myawesomeplugin --plugin

Can you elaborate more? you can use "plugman create" to create a very basic plugin. Are you looking for an advanced generator which can also generate some native code?

sorry, was on my way back from Redmond to Germany.
When you talk about adding a new plugin to an existing project plugman is okay, but normally when creating cross platform plugins, I would love to see a top-level generator which is able to create a plugin within a dedicated project.
each plugin we're creating is tracked as it's own repo. (which makes publishing to npm also easier) that's why I thought about having an option for the create command.

Have you taken a look at the link provided by @nisheetjain? If you scroll down specifically to the  create section, I believe you may find what you are looking for. You can also take a look at the following tutorial on MSDN. Some sections of the tutorial appear incomplete, but it should still give a good idea of how you can use plugman create to generate a basic scaffolding for your new plugin.
If you already know about the plugman create command and it still isn't good enough for your needs, please elaborate on what you would want that isn't provided by plugman create, and we will see if there is something we could add to TACO to help in that area.

Hi @ThorstenHans! We really appreciate you taking the time to provide suggestions. We were just wondering if you had had the chance to take a look at the plugman create documentation for plugin scaffolding. If you get the chance to take a look, please let us know whether that's the kind of scaffolding you were looking for!

Hey @ThorstenHans, we haven't heard back from you on this matter, so I will close this for now. Please reopen if your needs aren't covered by plugman create!


For Instagram rounded button  class is  only Instagram and For Square button class is Instagram-rounded this should be changed

Describe the bug
Expected behavior of what should have happened (if applicable)
Screenshots (if applicable)
Additional notes

Can I fix this bug

#272

I agree with what you are saying, but I think the better solution is to keep it consistent throughout the website. Usually, we just added the class rounded-btn to any button type to make it rounded instead of adding a new class relating to that button (for example, instagram-rounded)
I think what we need to do is remove all the "rounded" classes for the social buttons (instagram-rounded, facebook-rounded, etc....) and just add to these buttons rounded-btn. This will also help us display the buttons in the same manner as previous buttons.
Let me know if you can work on this (in a different PR than the one you submitted).

I can work if you want


Setting --weight flag when running new services fails

Describe the bug
FATA[0000] no services found when setting the --weight flag on rio run
To Reproduce

rio run --weight=100 -p 80 ibuildthecloud/demo:v1

Expected behavior
It should create the service with a spec weight of 100
Type:
Rio version: rio version v0.6.0-alpha.5 (ac4bbfee)

#755 stole the change from #759

Fixed in v0.6.0-rc1


Running a project for Mobile Blazor Bindings failed for Windows app

I tried to follow the 'Get Started' for Creating an app with Experimental Mobile Blazor Bindings
https://docs.microsoft.com/fr-fr/mobile-blazor-bindings/walkthroughs/build-first-hybrid-app
So I create my project by running this command:
dotnet new blazorhybrid -o FirstBlazorHybridApp
I can open the solution in Visual Studio 2019 and compile without problems.
I can even run the project inside the Android simulator. In Solution Explorer, right-click on the Android project and select Set as StartUp Project.

Unfortunately running the project for a Windows app failed.

I have no idea what may be the problem.

Hi @Bronzato1 , add this package reference to the MyProjectName.Windows.csproj project:
    <PackageReference Include="Microsoft.Web.WebView2" Version="1.0.961.33" />

And the problem should go away. This will be fixed in the next update!

It works. Thanks!


How to set & reset timeout for data?

I am just curious to know how I can set and reset timeout for data.
For example, I want to set timeout to 15 minutes so after 15 minutes the data will be wiped out automatically.
I tried to find that in documentation but not able to figure out looking at the code or docs.
For example, something like following.
hash_client.set(key, value, timeout=90000)
...

hash_client.reset(key, timeout=90000)

@martinnj any comments on this?

@mr-py-007
You're looking for the touch memcache command.
It's implemented in both the base- and hash- client.
Give it a key and an expire argument and it will reset the key if it exists.

@martinnj
Can I set that expire argument on set command?

expire is the name of the timeout argument you're describing in your original question.
set(self, key, value, expire=0, noreply=None, flags=None)
From https://github.com/pinterest/pymemcache/blob/master/pymemcache/client/base.py#L440

Thank you, @martinnj
I got it!


rstudio: 1.2.1335 -> 1.2.5033

Motivation for this change
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[x] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Doing random review.
I was able to successfully build the package using nixpkgs-review, and run the resulting RStudio binary.


Update libstarlark patches and remove maven warnings

This PR does two things:

Fix maven-compiler-plugin warning to stop using <forceJavacCompilerUse> in favor of <forceLegacyJavacApi>
Update libstarlark patches from upstream@ad261614c49 to take into account recent enhancements made by @Iapetus999

Description of changes in release / Impact of release:
N/A
Documentation
How I generated the diffs (for future reference):
$ pwd
starlarky

$ dirs=("src/main" "src/test"); for target_dir ("$dirs[@]"); do  rsync -avz --progress --exclude=BUILD .tmp/bazel/${target_dir}/java/net/ libstarlark/${target_dir}/java/net/ --delete; done

$ git diff -R >! ./bin/$(date +%Y-%m-%d)-differences-libstarlark-$(pushd .tmp/bazel/src; git log --pretty=tformat:"%h" -n1 .; popd).patch

$ git apply ./bin/2024-04-18-differences-libstarlark-$(pushd .tmp/bazel/src; git log --pretty=tformat:"%h" -n1 .; popd).patch

Risks of this release
None
Is this a breaking change?

[ ] Yes
[X] No

Is there a way to disable the change?

[x] Use previous release

Closing in favor of #491


Allow for uniform arrays

fixes #85

bors r+

Oops, force-pushed the wrong branch... commit is the same tho

bors r+


chore: adding linux questions

Checklist:

[x] I have read freeCodeCamp's contribution guidelines.
[x] My pull request has a descriptive title (not a vague title like Update index.md)

For issue #16

thanks for all the help.


Fix Issues with Extracts

There were a few bugs with extracts that this PR aims to fix:

Add extracts lines back to the subject (erroneously removed in #158)
Fix several issues with adjusting frames with sub-keys (ex: frame0.0, frame0.1, etc.)

Allow lines to be added
Allow lines to be deleted
Allow lines to be reordered

Merging this now so we can begin testing


Can't show variable value by p in gdb (DW_FORM_strp)

when I build envoy by "-c dbg", then run by gdb and try to check variable value and get below  info
**p socket_->isOpen())**
DW_FORM_strp pointing outside of .debug_str section [in module /build/.cache/bazel/_bazel_envoybuild/b570b5ccd0454dc9af9f65ab1833764d/execroot/envoy/bazel-out/k8-dbg/bin/source/exe/envoy-static]
**p read_end_stream_**
DW_FORM_strp pointing outside of .debug_str section [in module /build/.cache/bazel/_bazel_envoybuild/b570b5ccd0454dc9af9f65ab1833764d/execroot/envoy/bazel-out/k8-dbg/bin/source/exe/envoy-static]

anything or path missed? and check below for details:

build
cd /source; export PATH  bazel build -c dbg    //source/exe:envoy
run
gdb /build/.cache/bazel/_bazel_envoybuild/b570b5ccd0454dc9af9f65ab1833764d/execroot/envoy/ba
zel-out/k8-dbg/bin/source/exe/envoy-static
b ConnectionImpl::onRead
run -l trace --concurrency 1 -c /tmp/sesDySecLCnoDEnvoy.yaml --log-path /tmp/envoy.log
trigger message
check variable
p this->read_end_stream_
DW_FORM_strp pointing outside of .debug_str section [in module /build/.cache/bazel/_bazel_envoybuild/b570b5ccd0454dc9af9f65ab1833764d/execroot/envoy/bazel-out/k8-dbg/bin/source/exe/envoy-static]

thanks
Cliff

@KBaichoo has been looking at debug symbol related things recently and may have some insight.

Hi @KBaichoo ,
Would you share some idea on this?
Thanks in advance!
Cliff

The major concern is that it can't show the content of *this

See my PR here: https://github.com/envoyproxy/envoy/pull/33942 for a brief overview on some debug symbols related stuff.
I'd check that the section being referenced exists.
Also what version of clang / gcc are you using? Likewise with gdb ?
I've ran into some issues of using newer toolchains with older gdb versions.

hi @KBaichoo ,
Really Appreciated your reply! in our build container(ubuntu), I didn't find clang and see below for gdb and gcc version:
gdb --version
GNU gdb (Ubuntu 10.2-0ubuntu1~20.04~1) 10.2
Copyright (C) 2021 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
gcc --version
gcc (Ubuntu 11.4.0-2ubuntu1~20.04) 11.4.0
Copyright (C) 2021 Free Software Foundation, Inc.
BTW, which version of gdb and gcc works for you?
I mean that it can read *this in gdb for envoy built by "-c dbg"
thanks,
Cliff

neither of those versions look too old.
I actually build with clang 14 (which is what CI is using) and use gdb GNU gdb (GDB) 14.2. I know some other folks have tried use lldb  when debugging and run into issues using clang.
FWIW I'm able to get the value printed as expected following your experiment:
Thread 8 "wrk:worker_0" hit Breakpoint 1, Envoy::Network::ConnectionImpl::onRead (this=0x26a0bd3f6000, read_buffer_size=79)
    at source/common/network/connection_impl.cc:357
357       ASSERT(dispatcher_.isThreadSafe());
(gdb) p this->read_end_stream_
$1 = false

And these are the segments I have from in the debug build:
[11:16:27] kbaichoo:envoy:0 $size --format=SysV bazel-bin/source/exe/envoy-static
bazel-bin/source/exe/envoy-static  :
section                     size        addr
.interp                       28         736
.note.ABI-tag                 32         764
.note.gnu.build-id            36         796
.dynsym                    14088         832
.gnu.version                1174       14920
.gnu.version_r               560       16096
.gnu.hash                   1396       16656
.dynstr                     7286       18052
.rela.dyn                7981248       25344
.rela.plt                   9600     8006592
.rodata                  9880792     8019968
.gcc_except_table        2210936    17900760
protodesc_cold            581552    20111696
flags_help_cold             5452    20693248
.eh_frame_hdr            8443644    20698700
.eh_frame               34644932    29142344
.text                   97679155    63791424
.init                         27   161470580
.fini                         13   161470608
google_malloc             815665   161470656
malloc_hook<PHONE_NUMBER>22
.plt                        6416   162286880
.tdata<PHONE_NUMBER>08
.tbss                       9581   162297600
.fini_array                    8   162297600
.init_array                17320   162297608
.data.rel.ro             2895800   162314928
.dynamic<PHONE_NUMBER>28
.got<PHONE_NUMBER>40
.got.plt                    3224   165211536
.data                     381560   165218880
.tm_clone_table                0   165600440
__rseq_cs                   2624   165600448
__rseq_cs_ptr_array<PHONE_NUMBER>72
.bss                    10189248   165603776
.comment                      97           0
.debug_abbrev             245791           0
.debug_info              2565679           0
.debug_str_offsets        580120           0
.debug_str                668320           0
.debug_addr             60972824           0
.debug_line            363996157           0
.debug_line_str           497779           0
.debug_rnglists         27153314           0
.debug_loclists           445938           0
.debug_aranges              1056           0
.debug_frame                  96           0
.gdb_index             382380659           0
Total<PHONE_NUMBER>

In bazel-bin I have the following:
[11:18:37] kbaichoo:bazel-bin:0 $find . -name *.dwp
[11:18:41] kbaichoo:bazel-bin:0 $find . -name *.dwo
./external/com_github_alibaba_hessian2_codec/hessian2/_objs/codec_impl_lib/codec.pic.dwo
./external/com_github_alibaba_hessian2_codec/hessian2/_objs/reader_lib/reader.pic.dwo
./external/com_github_alibaba_hessian2_codec/hessian2/basic_codec/_objs/byte_codec_lib/byte_codec.pic.dwo
...

No dwp files but many dwo.

Congratulations!
I still can't print *this
thanks
Cliff


RNLocalSearch is undefined

hey, currently trying to use this lib, i have RNLocalSearch.xcodeproj in my project's Libraries directory, importing the lib via import RNLocalSearch from 'react-native-localsearch'.
console.log(RNLocalSearch); gives undefined. could it be version compatibility? i notice the project hasn't had any updates in a while.

sorted, i needed to manually add the libRNLocalSearch.a file to the project after including the directories. this is probably because i have no idea what i'm doing when it comes to xcode and native iOS.


LSP document diagnostics not showing anymore

When I use the shortcut for LSP document diagnostics nothing pops up anymore. Anyone else?

Hello @partounian ,
Thanks for opening this issue. The LSP mappings were a little messed up. I did some rework.
However, if you are still facing issues, please run the commands behind the shortcuts directly.

Trouble symbols toggle (if Trouble.nvim is enabled in your config)
Telescope lsp_document_symbols (if Trouble.nvim is not enabled

If there are no diagnostics in the current document, Telescope will not pop up but show a warning when directly called via the command.


Update backend group name with a prefix

Proposed changes
Write a clear and concise description that helps reviewers understand the purpose and impact of your changes. Use the
following format:
Problem: User want to configure split client upstream in a namespace starting with a digit without running into any issues.
When a user was configuring split clients in a namespace starting with a digit, it led to proxy_pass variable being http://$9c_network__some_route_name_rule0$request_uri , which led to issues with nginx since variable names cannot start with a digit, leading to a 502 Bad Gateway error.  The above problem existed for cases when a variable name is mentioned for using split weightage of upstreams.
Solution:  Updated backend group name with a prefix "group_". Hence, we update split client variable names, distribution names and upstream names.
Testing: Testing with cafe and traffic splitting example.
Config with cafe example
location = /coffee {
       proxy_pass http://9test_coffee_80$request_uri;
    }
    location = /tea {
       proxy_pass http://9test_tea_80$request_uri;
    }

upstream 9test_coffee_80 {
    random two least_conn;
    zone 9test_coffee_80 512k;

    server <IP_ADDRESS>:8080;
}

upstream 9test_tea_80 {
    random two least_conn;
    zone 9test_tea_80 512k;

    server <IP_ADDRESS>:8080;
}

Config with trafic-splitting example
location /coffee/ {
        proxy_pass http://$group_9test__cafe_route_rule0$request_uri;
    }

location = /coffee {
       proxy_pass http://$group_9test__cafe_route_rule0$request_uri;
 }

upstream 9test_coffee-v1_80 {
    random two least_conn;
    zone 9test_coffee-v1_80 512k;

    server <IP_ADDRESS>:8080;
}

upstream 9test_coffee-v2_80 {
    random two least_conn;
    zone 9test_coffee-v2_80 512k;

    server <IP_ADDRESS>:8080;
}

split_clients $request_id $group_9test__cafe_route_rule0 {
    80.00% 9test_coffee-v1_80;
    20.00% 9test_coffee-v2_80;
}

Please focus on (optional): If you any specific areas where you would like reviewers to focus their attention or provide
specific feedback, add them here.
Closes #2606
Checklist
Before creating a PR, run through this checklist and mark each as complete.

[x] I have read the CONTRIBUTING doc
[x] I have added tests that prove my fix is effective or that my feature works
[x] I have checked that all unit tests pass after adding my changes
[x] I have updated necessary documentation
[x] I have rebased my branch onto main
[x] I will ensure my PR is targeting the main branch and pulling from my branch from my own fork

Release notes
If this PR introduces a change that affects users and needs to be mentioned in the release notes,
please add a brief note that summarizes the change.

Be sure to use your fork in the future, this doesn't need to be a branch on the main repo.

Thank you for noticing that ! Need to fix my setup, probably messed it up !


[NIFI-1794] Update .dialog-content to wrap text

Thanks @scottyaslan. This has been merged to master.


Add new properties to organization model

Feature Request
Is your feature request related to a problem or unsupported use case? Please describe.
In the organization model we need new properties in order to use them when generated the contract in PDF.
Describe the solution you'd like
Add following properties on Organization model :

address (see if it's possible to create a relation with the existing Address model)
siret (a string)
activity_number
representative_profession
signatory_representative (optional)
signatory_representative_profession (required when signatory_representative filled)
contact_phone
contact_mail
dpo_mail

All these new properties must be managed in the admin application

To solve this issue :

PR that adds new properties for the Organization model.
PR that adds new fields into the AdminOrganizationSerializer for the backoffice.


Optimize reverse proxy

Do some profiling and get the latency as low as possible, possible replace Go's reverse proxy with https://github.com/valyala/fasthttp, test unix domain sockets in Lambda as well, check heap and see what can be pooled. It's also making some assumptions about having concurrency of 1 at the moment, since that's the case with Lambda.
Make sure it's not decompressing gzip and recompressing either, not sure what the proxy does but I know net/http's client does this transparently, test with some larger responses.

[ ] profile cpu
[ ] profile mem
[ ] test larger responses
[ ] implement any quick wins
[x] either remove Accept-Encoding or ignore when already compressed

I've been using fasthttp as I make my way thru learning Go. It's been treating me really well so far & I think it'd be a good fit here!

need to do some better measuring but it looks liek latency is already < 1ms so it might not be a big deal, especially with each Lambda container having a concurrency of 1 haha

FWIW The product I'm launching on up will be running fasthttp at its core 🎉

would it be reasonable to add opting out of the reverse proxy as an option?

@StoneCypher It's mandatory for now, since it relays events to the app you deploy. Honestly though it only introduces ~500μ, so it's nothing to worry about

Looks like this has been deprioritized, but wanted to add some thoughts in case anyone revisits.
Careful with unix domain sockets, you need to always spawn a random one to use - since containers are reused (and frozen), you have to always choose a different unix domain socket to prevent reuse errors.
I think switching to TCP / Unix domain sockets is a huge win over stdin / stdout simply because it eliminates oddness with accidentally logging to stdout.  Eliminating that gotcha is worth way more valuable than speed IMO - plus TCP stacks are pretty dang fast these days.
Last point - if you're looking to tighten up speed, my spidey-sense tells me that the big win will be in eliminating the extra marshal / unmarshal pass that happens in the node.js shim (over stdout).
And this might be sad, but I believe aws-lambda-go-shim uses a shared memory technique which is going to blow all of them out of the water.

@skabbes FWIW that stdout weirdness only really applies to the up proxy, at least no one else has to be aware of it (thank god haha).
Latency seems fine as-is but I wouldn't be too surprised if the serialization for big requests would be worth getting rid of, seems negligible for average stuff.
Reinvent is soon too, I'm banking on a whole bunch of new features/runtimes :D

@tj Ahh, ya I was confusing with go-apex, since I think you do have to be aware of stdout there.
FWIW, as a user of up in production, I love that I have the ability to read all the code and understand exactly what is going on.  External reverse proxies, or things getting too magic would turn me off a bit.

installing pkgs takes longer than uploading a binary

I'd imagine that they'll just load your compiled code as a plugin, and put the compile requirements on you, so you get the same kind of speed.  Pretty much, I think they'd just eliminate the need for the shim entirely.  I'd even more love for them to add streaming upload and download support to APIGateway.

Oh man yeah, I'm sure that'll be comforting for a lot of people, and not relying on some little PaaS startup is a nice benefit too.
Trueee good point I forgot about plugins, that's a great use-case for it, seems like it'd slim down quite a bit:
   λ up (master): make size
2.2M	net/http.a
1.7M	runtime.a
1.1M	net.a
972K	reflect.a
860K	github.com/apex/up/vendor/github.com/aws/aws-sdk-go/aws/endpoints.a
648K	crypto/tls.a
496K	math/big.a
476K	text/template.a
404K	encoding/xml.a
388K	html/template.a
380K	text/template/parse.a
380K	syscall.a
380K	crypto/x509.a
376K	encoding/json.a
...


Fix ContextValue Annotation for Event Listeners

SpongeAPI | SpongeCommon
Long story short: Since the Cause and Context PR merge, this annotation which replaced @Named, we've been unable to reproduce the feature set due to the language requirements that annotations are utilizing constants at compile time and not runtime. As such, it's not feasible to use for example:
@Listener
public void onChange(ChangeBlockEvent.Place event, @ContextValue(EventContextKeys.OWNER.getName()) Player player) {
 // foo
}

This will result in a compile time error due to the lack of constants in this case.
This adds the constants to be used and some fuzzy string matching on the context to perform the job as expected.

Thinking further not sure why the new method on EventContext is even needed, is basically just a helper over getting the key from the registry, but without the caching that the proper way of doing this provided. Plugins should just be getting the key from the registry by id if they wish, allowing them to handle any error from the key not existing there. This method provides nothing new while simultaneously lowering the performance and masking any error from the key not existing.