route "stdout" contains reference to undefined or misconfigured template "raw-json"."

Description

What did you expect to happen?
The installation succeeded. We expected to see results of the scan in the container log.
What happened instead?
We received error from the logs regarding the template.
Output of run with POSTEE_DEBUG=true:
#################################################
ERROR MESSAGE
#################################################
{
insertId: "q35i4olmi0ay1o75"
labels: {5}
logName: "projects/t-2004107-aquaops-0000/logs/stderr"
receiveTimestamp: "2022-11-16T10:55:12.840010372Z"
resource: {2}
severity: "ERROR"
textPayload: "2022/11/16 10:55:08 route "stdout" contains reference to undefined or misconfigured template "raw-json"."
timestamp: "2022-11-16T10:55:08.364027297Z"
}
(paste your output here)

Additional details (environment setup, networking info...):
We copied the yaml file from the GitHub page and checked it several times. There are no syntax errors inside. Also the config map after the deployment exactly resembles the documentation.

@AndreyLevchenko can you take a look at this?

Hello @gsoeldner
Thanks for your report!
I can reproduce similar case:
2022/11/18 14:14:09 route "stdout" contains reference to undefined or misconfigured template "raw-json\".".

Looks like your config file contains extra suffix in template name: raw-json".
Can you check this?

We copied the yaml file from the GitHub page and checked it several times.

Can you also send link to GitHub where you get this example yaml file?
Regards, Dmitriy

Hi Dmitry,
we are using 2.9.0...we used the values.yaml as and removed all the unnecessary stuff:
https://github.com/aquasecurity/postee/blob/main/deploy/helm/postee/values.yaml
Best
Guido

Here is the config:
####################################################
Installation: helm install app -n postee aqua/postee --values values.yaml
#################################################### values.yaml
image:
repository: xxx:5200/aquasec/postee
pullPolicy: Always
tag: "2.8.4-amd64"
imageInit:
repository: xxx:5200/busybox
pullPolicy: IfNotPresent
tag: "1.34"
posteUi:
port: 8000
user: ""
pass: ""
image: xxx:5200/aquasec/postee-ui
tag: "2.8.4-amd64"
Default values for postee.
This is a YAML-formatted file.
Declare variables to be passed into your templates.
replicaCount: 1
posteeConfig: |
The configuration file contains a general settings section,
routes, templates and actions sections.
name: aqua-test           #  The tenant name
aqua-server: xxx
max-db-size: 1000MB       #  Max size of DB.  pattern is used, such as "300MB" or "1GB". If empty or 0 then unlimited
db-verify-interval: 1   #  How often to check the DB size. By default, Postee checks every 1 hour
Routes are used to define how to handle an incoming message
routes:

name: stdout
actions: [ splunk ]
template: raw-json

Templates are used to format a message
templates:

name: raw-json                        # route message "As Is" to external webhook
rego-package: postee.rawmessage.json

Actions are target services that should consume the messages
actions:

name: splunk
type: splunk
enable: true
url: xxx
token: xxx          # Mandatory. a HTTP Event Collector Token
size-limit: 10000          # Optional. Maximum scan length, in bytes. Default: 10000

Hello @gsoeldner
Can you check config log(start of Postee)?
From Starting Router.... to Starting WebServer....
We need to check that template has been configured.
my configuration example:
2022/11/21 10:12:29 Starting Router....
2022/11/21 10:12:29 Loading alerts configuration file ./test_config/cfg.yaml ....
2022/11/21 10:12:29 Configuring template raw-json
2022/11/21 10:12:29 No aggregation package configured!!!
2022/11/21 10:12:29 Configured with Rego package postee.rawmessage.json
2022/11/21 10:12:29 Action stdout is configured
2022/11/21 10:12:29 Starting ServiceNow action "my-servicenow"....
2022/11/21 10:12:29 Your ServiceNow Table is "incident" on 'xxx'
2022/11/21 10:12:29 Action my-servicenow is configured
2022/11/21 10:12:29 Starting WebServer....

Hi,
here is the thconfig log:
2022/11/21 14:18:12 Starting Router....
2022/11/21 14:18:12 Loading alerts configuration file /data/cfg.yaml ....
2022/11/21 14:18:12 Configuring template raw-json
2022/11/21 14:18:12 Can not initialize template raw-json: 1 error occurred during loading: open rego-templates/custom/lost+found: permission denied
2022/11/21 14:18:12 &router.ActionSettings{Name:"splunk", Type:"splunk", RunsOn:"", Enable:true, Url:"", User:"", Password:"", TlsVerify:false, ProjectKey:"", IssueType:"", BoardName:"", Priority:"", Assignee:[]string(nil), Summary:"", FixVersions:[]string(nil), AffectsVersions:[]string(nil), Labels:[]string(nil), Sprint:"", Unknowns:map[string]string(nil), Host:"", Port:0, Recipients:[]string(nil), Sender:"", Token:"dfc15c07-8440-4b0e-b809-69c7257ee3c9", UseMX:false, InstanceName:"", SizeLimit:10000, InputFile:"", ExecScript:"", Env:[]string(nil), BodyFile:"", BodyContent:"", Method:"", Timeout:"", Headers:map[string][]string(nil), OrganizationId:"", KubeConfigFile:"", KubeLabelSelector:"", KubeActions:map[string]map[string]string(nil), KubeNamespace:"", DockerImageName:"", DockerNetwork:"", DockerCmd:[]string(nil), DockerVolumes:map[string]string(nil), DockerEnv:[]string(nil), Tags:[]string(nil), Alias:"", Entity:""}
2022/11/21 14:18:12 Starting Action "splunk": "splunk"
2022/11/21 14:18:12 Starting Splunk action "splunk"....
2022/11/21 14:18:12 Action splunk is configured
2022/11/21 14:18:12 Starting WebServer....
2022/11/21 14:18:13 Listening for HTTPS on <IP_ADDRESS>:8445
2022/11/21 14:18:13 Listening for HTTP on <IP_ADDRESS>:8082

Hello!
Thanks a lot.
Looks like there is problem with access to templates.
can you try to get list of templates in /server/rego-templates folder?
example commands(on my PC):
➜  kubectl get pods -n postee2
NAME                            READY   STATUS    RESTARTS   AGE
app-postee-0                    1/1     Running   0          6m30s
app-posteeui-68955bc58c-9775c   1/1     Running   0          6m30s
➜  kubectl exec -it --namespace postee2 app-postee-0 /bin/sh

kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
Defaulted container "postee" out of: postee, setting-db (init), setting-cfg (init)
/server $ ls -hl /server/rego-templates/
total 60K
drwxr-xr-x    1 postee   postee      4.0K Sep 14 23:26 common
drwxrwxrwx    2 root     root        4.0K Nov 22 05:33 custom
drwxr-xr-x    1 postee   postee      4.0K Sep 14 23:26 example
-rw-r--r--    1 postee   postee       199 Sep 14 23:26 raw-message-html.rego
-rw-r--r--    1 postee   postee        93 Sep 14 23:26 raw-message-json.rego

Hi,
here is the output:
/server/rego-templates $ ls -hl /server/rego-templates/
total 60K
drwxr-xr-x    1 postee   postee      4.0K Sep 14 23:26 common
drwxr-xr-x    3 root     root        4.0K Sep 16 08:41 custom
drwxr-xr-x    1 postee   postee      4.0K Sep 14 23:26 example
-rw-r--r--    1 postee   postee       199 Sep 14 23:26 raw-message-html.rego
-rw-r--r--    1 postee   postee        93 Sep 14 23:26 raw-message-json.rego
-rw-r--r--    1 postee   postee       416 Sep 14 23:26 tracee-html.rego
-rw-r--r--    1 postee   postee       654 Sep 14 23:26 tracee-slack.rego
-rw-r--r--    1 postee   postee      2.6K Sep 14 23:26 vuls-cyclonedx.rego
-rw-r--r--    1 postee   postee       467 Sep 14 23:26 vuls-html-aggregation.rego
-rw-r--r--    1 postee   postee      5.9K Sep 14 23:26 vuls-html.rego
-rw-r--r--    1 postee   postee       154 Sep 14 23:26 vuls-opsgenie.rego
-rw-r--r--    1 postee   postee       517 Sep 14 23:26 vuls-slack-aggregation.rego
-rw-r--r--    1 postee   postee      7.9K Sep 14 23:26 vuls-slack.rego

I was able to get a similar error:
2022/11/24 03:57:11 Configuring template raw-html
2022/11/24 03:57:11 Can not initialize template raw-html: 1 error occurred during loading: open rego-templates/custom/123: permission denied

can you check /server/rego-templates/custom/ folder?
Looks like it contains file without permission.
Remove /server/rego-templates/custom folder and rerun postee.

Hi,
how can we delete the folder. We are not root.
When using su, we have the error:
su: must be suid to work properly

Ok that helps...but next Problem:
Error while sending event: Post "https://hecendpoint:8088/services/collector": x509: certificate signed by unknown authority
How can we ignore the warning?

Looks like you need to install and update ca-certificates.
Postee currently has insecure flag only for jira.

Well that is pretty complicated without being root.
The Helm Chart also does not allow to mount a volume from a config map with the certifcates.
How are custom certificates to designed to be uploaded?

You mean this here:
--insecure-skip-tls-verify                   skip tls certificate checks for the chart download
We can use the chart, however we do not know how to import our ca-certificate into the app-postee-0 pod.

if i understand correctly - you can add certificate using ca-file( flag https://stackoverflow.com/a/59966896).

No, the installation is not the problem. We need to bring our certificates into the Pod in order to talk to Splunk....see here:
https://paraspatidar.medium.com/add-self-signed-or-ca-root-certificate-in-kubernetes-pod-ca-root-certificate-store-cb7863cb3f87
Two solutions:

you change the implementation for the Splunk Action (allow ssl-non-verify)
You change the kubernetes deployment / yaml file to upload the certificates via a config map.

hm... you are right. It makes sense.
I will add insecure skip verify for `slack.

We need it for Splunk not for the slack action

oh... my mistake...
Thank you for finding this.
I created #505. We will try to merge it as soon as possible.

Hello All,
I have something to clarify related to the logs collected by postee. I would like to know if there is a way where I can configure to turn off the logs but postee sends the scan result to the Roxy only. The logs is actually making some issue with the splunk in my current project.
please help me out guys. i am new ro aqua topics.

Hello @linthoisana
I thinks it is different issue.
Can you create new issue and add some examples into new issue?
I asked for examples because postee doesn't include logs in send message.

thank you @DmitriyLewen . i have created an issue #507

@DmitriyLewen do you have any updates to share regarding this issue?

Hello @tectime
We merged #505
Next release will include this changes and you will be able to гыу tls-verify option in splunk.

Hi we are using Postee and run into the same issue where lost+found objects are confusing the template scanner. I've tested the fix posted in this thread (move mount point to /server/tests/custom). But we are still interested in a solution for it. Any idea how we could fix it? Can someone guide me where in the code the scanner stumbles upon this lost+found item?

Hello, +1.
I have the same issue as posted before.
Are there some ideas on how to fix the "lost+found" issue?

Hello @smangels , @stanislavdip
Thanks for you reports and sorry for long wait.
I created filter for rego-templates by lost+found name - #556
Can you check this changes?
Also I have 1 more request - can you send me permissions for lost + found in your cases?
It might be better to filter files by this value.
Regards, Dmitriy

Hello

DmitriyLewen
Postee deployed in Kubernetes cluster via ArgoCD, helm chart:
version: v2.10.2
repository: https://aquasecurity.github.io/helm-charts/

Here are the permissions for the files:
/server $ ls -la /server/rego-templates/custom/
total 28
drwxr-xr-x    3 root     root          4096 Apr 27 07:48 .
drwxr-xr-x    1 postee   postee        4096 Apr 27 09:29 ..
drwx------    2 root     root         16384 Apr 27 07:48 lost+found
/server $ ls -la /server/rego-filters/custom/
total 28
drwxr-xr-x    3 root     root          4096 Apr 27 07:48 .
drwxr-xr-x    1 postee   postee        4096 Apr 27 09:29 ..
drwx------    2 root     root         16384 Apr 27 07:48 lost+found

Hello @g-soeldner , @smangels , @stanislavdip
We merged #556.
Fix will be include to next release.
Regards, Dmtriyi


Bump API Version

I pushed a new version (0.4.0) of the myicomfort api to PyPI and it is being tested.  Once all is well I need to bump this integration to use the new version.  This is here to remind that this is pending.

API is tested and working.  Bumped the required version to 0.4.1.


Implementation classes under models should be renamed as impl

These should be moved into an 'implementation' sub package #647

Available in master.


client should periodically reload InClusterConfig token

See https://github.com/kubernetes/kubernetes/issues/70679

/lifecycle frozen

/cc @brendandburns: this issue can be closed now. thanks.

@zshihang Thanks!


Typedef must go

The usage of typedefs has proven to be of very limited value. Because the typesystem allows for easy creation of own primitive / collection types type aliasing is not really required. Aliasing interface types like structs and classes is too rare to justify the feature.

Side question, is there native support for overriding cast operations?

No. Cast operations are currently hard coded in the type system. Casts between primitive types are performed by the cx_convert routine. The cx_type_castable function determines whether types can cast.
Composite types can only cast to each other when there is an inheritance relation. A sub class can always be assigned without casts to a base. The reverse (base to a subclass) requires an explicit cast.
The array, sequence and list collection types can be casted to each other provided they share compatible element types. Restrictions apply to array: since it is of fixed length sequence/list cannot be casted to an array where the reverse is possible.


Another internal error

Environment Info
kivy: 1.9.0
pygments: 2.0.2
docutils: 0.12
watchdog: 0.8.3
jedi: 0.9.0
gitpython: 1.0.1
six: 1.10.0
kivy-garden: 0.1.4
Platform: Linux-4.2.0-18-generic-x86_64-with-Ubuntu-15.10-wily
Python: 2.7.10
Traceback
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/kivy/core/window/window_sdl2.py", line 525, in mainloop
self._mainloop()
File "/usr/local/lib/python2.7/dist-packages/kivy/core/window/window_sdl2.py", line 290, in _mainloop
EventLoop.idle()
File "/usr/local/lib/python2.7/dist-packages/kivy/base.py", line 327, in idle
Clock.tick()
File "/usr/local/lib/python2.7/dist-packages/kivy/clock.py", line 483, in tick
self._process_events()
File "/usr/local/lib/python2.7/dist-packages/kivy/clock.py", line 615, in _process_events
event.tick(self._last_tick, remove)
File "/usr/local/lib/python2.7/dist-packages/kivy/clock.py", line 374, in tick
ret = callback(self._dt)
File "/home/kuldeep/Downloads/kivy-designer-master_main/designer/project_loader.py", line 535, in perform_auto_save
os.path.basename(self.root_rule.kv_file))
File "/usr/lib/python2.7/posixpath.py", line 114, in basename
i = p.rfind('/') + 1
AttributeError: 'NoneType' object has no attribute 'rfind'

---
Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/28749737-another-internal-error?utm_campaign=plugin&utm_content=tracker%2F77147&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F77147&utm_medium=issues&utm_source=github).

project_loader will be replaced in the next commits

Ok!, So no more issues related to project_loader until the next commit.


attaching custom new security list to existing subnet with terraform

Hi, I have a question to which I cant find any terraform resources or cases in google:
I create a new security list using terraform-oci-tdf-network-security module, which I want to attach to a workers subnet  autocreated with terraform-oci-oke module.  As per terraform-oci-oke module configuration, I cant find a way to configure security lists, seems this option does not exist there, only default lists with default rules are created for each subnet it creates.  On the other hand, terraform-oci-tdf-network-security module can also only create security lists, but not attach it to an existing subnet.
Do you know how can one attach an existing security list to an existing subnet?  I cant also find any default terraform resources for this..  It seems to me like a feature unimplemented.. ?

Hi.
Security list has always been independent of subnets i.e. you can always create them on their own. However, security lists were required for subnets. They are now optional.
Adding an existing security list to a subnet can be done by modifying the security_list_ids parameter which takes an array: see here: https://registry.terraform.io/providers/hashicorp/oci/latest/docs/resources/core_subnet. It's part of the core subnet resource itself.
This is exactly what we are doing: https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/master/modules/okenetwork/subnets.tf#L11 , https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/master/modules/okenetwork/subnets.tf#L22 and https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/master/modules/okenetwork/subnets.tf#L35.
The security lists we are creating are those that are required by OKE docs.
Please elaborate on your use case and what you think we have missed. We'll definitely consider it.

So I understand that currently the only way to add security list to subnet is via subnets creation resource  oci_core_subnet. But thats not good enough, as theoretically (as you mentioned, security lists are separate from subnets) - they cant be created standalone, so there must also be an option to assign a security list to an existing subnet (subnets can have more than just one single security list assigned). My use case here is that once I create my workers and other subnets cia your oci-oke  modules, I get default subnets provided by this module with their default security lists. Yes, that works as a default, but now that I need some new functionality (in my case I need some special access from operator to worker subnet on specific ports which dont go by default), I can do that manually via GUI by creating new security list with new rules and assigning it to an existing subnet,   BUT, the very same thing I cannot do via terraform, because there is a resource to create security list with its rules, but there is no resource to assign this new security list to any possible subnet.  This is my issue

The security_list_ids parameter takes a list. In the repo, we are passing a list of 1 sec list.
You can create other sec lists and include their ids in the aray.
How you create the sec lists is up to you. Since you mentioned you want some ports opened, you can check how we adding optional ssh access from bastion to the worker nodes i.e. include the ports in the existing sec lists. Another option would be to create your sec list alongside the defauly worker sec lists in the okenetwork module.
Since you also mentioned using the tdf seclist module, you can use this too. You'll need to either create it as a subrepo in your git ok or use it as a remote module. You'll also need to output the seclist ids it created and pass them as a parameters to the okenetwork module and add them to the security_list_ids parameter.
Hope that hells.

@bieksaz have we answered your question?

Hi,
I'm closing this issue. If this is still a problem, please reopen.

@bieksaz how did you work around this?

I think the challenge the OP was trying to accomplish is something I just realized I have to deal with.
Example, you might have two stacks ...

VCN, subnets, routing tables, security lists, etc.
A stack for something else that creates a new security list that needs to be "attached" to the subnet.

The only solution is to follow a process of:

Apply the VCN stack with subnets, etc.
Apply the something-else-stack with the new security list.
Go BACK to the VCN stack and modify whatever variable has the security lists to attach to the subnet.

In other providers is the concept of individual sub-resources ... in this case, something like a oci_subnet_security_list_attachment that would accept a subnet_id and a security_list_id. (I presume we don't have that because the OCI API doesn't support it, I'm just saying.) If that attachment resource existed, we could pass in the subnet id (or find the convention or however you like) into the something-else-stack and then just attach the new security list to it.
And while I'm getting cooked up over this (my original reason for dealing with this), the OCI Redis Cluster will automatically create a security list and attach it to the subnet you provide and allow ALL the CIDR space in the VCN to connect to the Redis cluster just created.
So now you create a VCN stack, go and create your redis cluster, and your VCN stack is immediately drifted from your defined stack. You have to go find the secret security list (because it is not an attribute of the cluster) and go back to your VCN stack to add it to a variable just so your drift detection isn't sending alarms.


Pseudo label

Issue #, if available:
Description of changes:
Incorporated pseudolabeling into Autogluon. The changes entail the following:

Adding extra X_pseudo and y_pseudo args into aux_kwargs and core_kwargs at the fit_extra() function
Adding pseudo data validation
Incorporating pseudo data into training at the bagged ensemble model level and single model level

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Job PR-1323-1 is done.
Docs are uploaded to http://autogluon-staging.s3-website-us-west-2.amazonaws.com/PR-1323/1/index.html

Job PR-1323-2 is done.
Docs are uploaded to http://autogluon-staging.s3-website-us-west-2.amazonaws.com/PR-1323/2/index.html

Job PR-1323-3 is done.
Docs are uploaded to http://autogluon-staging.s3-website-us-west-2.amazonaws.com/PR-1323/3/index.html

Job PR-1323-4 is done.
Docs are uploaded to http://autogluon-staging.s3-website-us-west-2.amazonaws.com/PR-1323/4/index.html

Job PR-1323-17 is done.
Docs are uploaded to http://autogluon-staging.s3-website-us-west-2.amazonaws.com/PR-1323/17/index.html

Job PR-1323-18 is done.
Docs are uploaded to http://autogluon-staging.s3-website-us-west-2.amazonaws.com/PR-1323/18/index.html

Job PR-1323-23 is done.
Docs are uploaded to http://autogluon-staging.s3-website-us-west-2.amazonaws.com/PR-1323/23/index.html

Job PR-1323-26 is done.
Docs are uploaded to http://autogluon-staging.s3-website-us-west-2.amazonaws.com/PR-1323/26/index.html

Job PR-1323-28 is done.
Docs are uploaded to http://autogluon-staging.s3-website-us-west-2.amazonaws.com/PR-1323/28/index.html

Job PR-1323-30 is done.
Docs are uploaded to http://autogluon-staging.s3-website-us-west-2.amazonaws.com/PR-1323/30/index.html

Job PR-1323-31 is done.
Docs are uploaded to http://autogluon-staging.s3-website-us-west-2.amazonaws.com/PR-1323/31/index.html

Job PR-1323-50 is done.
Docs are uploaded to http://autogluon-staging.s3-website-us-west-2.amazonaws.com/PR-1323/50/index.html


Added optional trialResearchSystem column to ClinicalTrialInformationExtractor

Summary
ClinicalTrialInformationExtractor can now take an optional column for trialResearchSystem
New behavior
ClinicalTrialInformationExtractor can now take an extra column for trialResearchSystem in it's CSV input. If the column is omitted from the CSV, the "system" field in the resulting study resource will also be omitted. The value can also be blank in any individual row of the CSV and the "system" field in the study resource will also be omitted
Code changes

Changed CSVClinicalTrialInformationExtractor.js to take the new field, defaulting to no value if it isn't provided. Since the field is optional, it won't throw an error if it is missing
Changed ResearchStudyTemplate.js and ResearchSubjectTemplate.js to take in trialReseachSystem as a parameter, previously they just had hard coded default values
Changed CSVClinicalTrialInformationExtractor.test.js and associated files to reflect changes described above

Testing guidance
I've changed the sample data csv (clinical-trial-information.csv) to have values for trialResearchSystem for each sample patient, but as mentioned above, any row can have a blank value for this or the entire column can be omitted.

DP1 - One thing I want to point out (shoutout JA and MG for the help) is that these changes should probably result in an update to the E-MEF. Would you be okay making an identically-named branch in that repo and update that repo's ClinicalTrialInformationExtractor to parse off the trialResearchSystem in it's joinAndReformatData function? There may be some fixtures that need updating as well, but hopefully there won't be a ton of work involved in that branch.

Since that's done I think I'm all set to approve here. 👍

Sometime tomorrow @dmendelowitz you can rebase and then merge this branch, when you are ready. After your code has been rebased and merged here, you can run npm update mcode-extraction-framework from the dependent repo to update the version of the MEF we use to include this newly merged code. After that, you should observe a change to the commit hash in the package-lock.json (importantly: we don't expect a change in package.json). If everything looks good there, you can push the updated commit hash for the MEF. This will allow our CI tests to run appropriately in the dependent repo.
Congrats on your first PR! 🎉 🎉 🎉


Please help add PyOpenGL based viewer

Please help add my PyOpenGL implementation of Gaussian Viewer at https://github.com/limacv/GaussianSplattingViewer.git. Thanks~

Done. Thx!


xla_gup not compatible to newest tensorflow version

the error is:
ValueError: To call multi_gpu_modelwithgpus=2, we expect the following devices to be available: ['/cpu:0', '/gpu:0', '/gpu:1']. However this machine only has: ['/cpu:0', '/xla_gpu:0', '/xla_gpu:1', '/xla_cpu:0']. Try reducing gpus.
Others on github says it is a bug of newest tf, and the flowing version will solve the above issue:
Tensorflow 2.0
CUDA 10.0
cuDNN 7.6.4 (described as dedicated for CUDA 10.0)
However, when I degrade the tf version to 2.0, the issue still exists.
Could you make you model running on these version?
Could you tell me which version should I use or update the requirements?

Hi,
This is likely because the mpunet still relies on the old tf.keras multi_gpu_model function for distributing models across multiple GPUs. I will soon (within the next few days) update to use the proper TensorFlow 2 approach (tf.distribute.MirroredStrategy), which I believe will fix your error.

Hi,
Could you try and checkout on branch 'v.0.2.5' and see if it that works for you?


cannot flash a Sonoff Mini

when i run your most excellent script it just sits there:
./sonoff-ota-flash.sh -i <IP_ADDRESS> tasmota-9.2.0-lite.bin
Checking new firmware file exists
OK
Looking up sha256sum for firmware
OK
Getting Module Info...
{
"seq":2,
"error":0,
"data":
{
"switch":"off",
"startup":"off",
"pulse":"off",
"pulseWidth":500,
"ssid":"koala-iot",
"otaUnlock":false,
"fwVersion":"3.6.0",
"deviceid":"1000a05bf6",
"bssid":"46:d9:e7:49:83:a8",
"signalStrength":-43
}
}
Unlocking for OTA flashing...
and then after a couple of minutes it says:
curl: (52) Empty reply from server
Error posting to: http://<IP_ADDRESS>:8081/zeroconf/ota_unlock
i have been trying to post ota_unlock to the mini for several hours now.
am i doing something wrong?

Hm! The modules aren't very good at proving feedback about what is going on.
I have experienced "Empty reply from server" but I have had some problems with the virus / web protection software on my laptop, so I wasn't sure if it was related to that (it intercepts all TCP/HTTP connections). However resetting the module/starting again did fix it. The OTA Unlock step is very fast - takes less than a second - so you shouldn't have to wait.
Have you put the module into DIY mode?

Wire up the module to mains power. The blue LED will flash once every 2 seconds.
Hold down the button for 5 seconds. The blue LED then started flashing quickly 3 times every second.
I then had to hold down the button again for 5 seconds and it then flashed on and off quickly.
Connect to the Wifi network called ITEAD-xxxxxx using your computer. The password is 12345678.
Open http://<IP_ADDRESS>/ in your browser and enter your main Wifi network name and password. This is only used during the flashing process.
The module will then try and connect to your Wifi network. The blue LED will then flash twice per second. My router reported that the hostname of the device is "ESP_XXXXXX".

There are a couple of things I should do:

I haven't tried flashing a module with firmware 3.6.0 (only version 3.5.0). So I should try upgrading my test module and then try my script on it.

I should document the full beginning-to-end steps for OTA flashing and put instructions in the README

Hm! The modules aren't very good at proving feedback about what is going on.
I have experienced "Empty reply from server" but I have had some problems with the virus / web protection software on my laptop, so I wasn't sure if it was related to that (it intercepts all TCP/HTTP connections). However resetting the module/starting again did fix it. The OTA Unlock step is very fast - takes less than a second - so you shouldn't have to wait.
Have you put the module into DIY mode?

Wire up the module to mains power. The blue LED will flash once every 2 seconds.
Hold down the button for 5 seconds. The blue LED then started flashing quickly 3 times every second.
I then had to hold down the button again for 5 seconds and it then flashed on and off quickly.
Connect to the Wifi network called ITEAD-xxxxxx using your computer. The password is 12345678.
Open http://<IP_ADDRESS>/ in your browser and enter your main Wifi network name and password. This is only used during the flashing process.
The module will then try and connect to your Wifi network. The blue LED will then flash twice per second. My router reported that the hostname of the device is "ESP_XXXXXX".

There are a couple of things I should do:

I haven't tried flashing a module with firmware 3.6.0 (only version 3.5.0). So I should try upgrading my test module and then try my script on it.

I should document the full beginning-to-end steps for OTA flashing and put instructions in the README

thank you for your quick response. i really regret updating the firmware to 3.6.0. the device had been sitting in the box for about a year. my bad.
what was/is not clear to me was:

i put it into diy mode (once). does the device fall out of DIY mode once power has been removed? the fact that a) i could make the web service calls and b) the itead app says that my device is not online sort of led me to believe that it remained in diy mode after power off.

can you do the 5 seconds/5 seconds shuffle more that once? i was not able to.

does the device make any calls to itead as part of the ota_unlock web service. i have seen some reports that this is the case. i modified my dhcp server so that it uses <IP_ADDRESS> for dns (i normally use pi-hole). that did not make any difference (apart from a awful of adverts on web pages while i was doing this).

thank you for your quick response. i really regret updating the firmware to 3.6.0. the device had been sitting in the box for about a year. my bad.
what was/is not clear to me was:

i put it into diy mode (once). does the device fall out of DIY mode once power has been removed? the fact that a) i could make the web service calls and b) the itead app says that my device is not online sort of led me to believe that it remained in diy mode after power off.

can you do the 5 seconds/5 seconds shuffle more that once? i was not able to.

does the device make any calls to itead as part of the ota_unlock web service. i have seen some reports that this is the case. i modified my dhcp server so that it uses <IP_ADDRESS> for dns (i normally use pi-hole). that did not make any difference (apart from a awful of adverts on web pages while i was doing this).

I think I need to go through this again to double-check, but I think once you have configured the module with your home Wifi, it stays in DIY mode.
I did have to keep having to reset it while I was experimenting/writing the script. Some HTTP requests seem to cause it to crash and I had to power-cycle it to get it back. Pressing the button for 5 seconds is described as "Factory Reset"- so I would have thought it should be possible to do it multiple times.
I am not sure if it calls home or not during the ota_unlock request. When I try doing it again, I will do a packet capture and see if there are any requests out to the Internet.
I have just tried installing the eWeLink software on my phone for the first time but I can't get it to setup my module :-(
It is possible I have done something to to the module so that original firmware won't work on it anymore 😞

I think the backup of my Sonoff Mini is from just after I configured it to access my home Wifi. When I do a restore, it goes back to that state just before I called ota_unlock.
I managed to do a packet capture during unlocking and it is indeed calling home:

This is what the HTTP request looks like:
POST /api/device/otaFlash HTTP/1.1
Host: api.coolkit.cn:8081
Content-Type: application/json
Authorization: Sign 8a62a48e34cfe4e4f5f0a8810267b2af34574396f227b575839310b14ec9a67d
Connection: close
Content-Length: 52

{"deviceid":"1000e4c17c","sequence":"4058031471168"}

HTTP/1.1 200 OK
Server: openresty
Date: Tue, 02 Feb 2021 00:31:26 GMT
Content-Type: application/json
Content-Length: 13
Connection: close

{"error":422}

Weirdly unlocking worked fine, despite the error code in the response.
When I lookup api.coolkit.cn using my local resolver in the UK:
$ host api.coolkit.cn
api.coolkit.cn is an alias for api.coolkit.cc.
api.coolkit.cc is an alias for eu-api.coolkit.cc.
eu-api.coolkit.cc has address <IP_ADDRESS>

I get the same result when resolving using using <IP_ADDRESS> and <IP_ADDRESS>.
It looks like Coolkit is the people who make eWeLink.

Good news, I have successfully managed to OTA a flash a Sonoff Mini running firmware version 3.6.0.
Although I had real trouble getting it into DIY mode.
I quit everything on my Mac apart from Safari. Went to Wifi settings and told it to forget about the ITEAD wireless network.
Then after holding down the button for 5 seconds to factory reset, then holding it down for 5 seconds again to put it into "Compatibility AP" mode, I was able to connect to the Wifi network, then connect to <IP_ADDRESS> and set my home Wifi network name and password. It then successfully joined my home network.
Once in DIY mode, it seemed to stay in DIY mode between power cycles.
Running the bash script then worked as expected:
./sonoff-ota-flash.sh
Checking new firmware file exists
OK

Looking up sha256sum for firmware
OK

Searching for Sonoff module on network...
Found module on network.
Hostname: eWeLink_1000e4c17c
IPv4 Address: <IP_ADDRESS>

Getting Module Info...
 {
    "seq":3,
    "error":0,
    "data":
    {
        "switch":"off",
        "startup":"off",
        "pulse":"off",
        "pulseWidth":500,
        "ssid":"test",
        "otaUnlock":true,
        "fwVersion":"3.6.0",
        "deviceid":"1000e4c17c",
        "bssid":"b4:fb:e4:77:7f:aa",
        "signalStrength":-47
    }
 }

Unlocking for OTA flashing...
 {
    "seq":3,
    "error":0
 }

Proceed with flashing? [N/y] y
Requesting OTA flashing...
 {
    "seq":4,
    "error":0
 }

Please wait for your device to finish flashing.

Packet capture on my router showed it phoning home again.
Again it got an 422 error response back in the JSON endpoint, which it didn't seem to mind.

@simonckenyon did you have any success with this?

I can just say I have managed to flash 5 Sonoff Minis flawlessly.

Just for the record I've managed to flash Sonoff Basic R3 (newer version without jumper for DIY mode) with firmware v3.6.0.
The only issue was that avahi was not able to find the IP of the device but I put it with the -i parameter.
Thanks a lot for this script.

I recently bought some more Sonoff Mini R2s, that came with firmware v3.6.0.
All worked fine, so closing this issue now.

I keep getting "connection reset by peer". Any idea why?


Fix: LCP metric no longer reported on initially hidden pages

Thank you for submitting a pull request. This code is leveraged to monitor critical services. Before contributing, please read our contributing guidelines and code of conduct.

Overview
Previously, pages opened in the background and had a visibilityState == 'hidden' would continue to report the LCP metric, and an inflated one at that.
Now, in such cases the LCP will not be reported (or even buffered at all).
Related Github Issue
Fixes #170
Testing

New html added to tests/assets/
Test subcase appended to existing case in tests/functional/timings.test.js

In addition to manually pasting the Browser Agent, follow the test case in the related issue to add a basic performance observer and/or core-web-vitals' getLCP for comparison. Example:

It may be helpful to programmatically include this in the lcpObserver code right before the handle("lcp", payload) line:
console.log("NR LCP Observer - handling lcp payload", payload);

Now on your test server, test the workflow of opening a new in-focus tab and background tab.

For the foreground tab, the regression test should show the LCP output in console, with the data reported to NR1:

While the background tab should show the output from NR agent (and CWV) missing, with just the performance observer LCP output.

More details on running your tests locally can be found in the
CONTRIBUTING and README docs.

Reopening accidental closure of #231


Can't import arrays from csv to postgres db

Is this supported? Even when I manually change the mapping type, I'm getting "org.jkiss.dbeaver.model.exec.DBCException: String data type 'VARCHAR' not supported by database"

No, it is not supported yet. I'll make this ticket feature request.

Problems with creating arrays in postgres https://github.com/dbeaver/dbeaver/issues/7819

already implemented


Use notification_vpn_enabled always

Task/Issue URL: https://app.asana.com/0/488551667048375/1205432938087991/f
Description
Use always the same layout in VPN notifications, to potentially avoid issues as they are remote views
Steps to test this PR
QA optional
Smoke tests AppTP/NetP notifications

Current dependencies on/for this PR:

develop

PR #3519   👈

This comment was auto-generated by Graphite.


Provide searchable API docs

With the mass of ndarray extensions it's hard to find the functions you need. I would be happy about some simple document (can be in the wiki or just a Markdown file somewhere) which lists all functions from all extensions so that I immediately know what exists and where to find it.

http://scijs.net/packages/ :)

Ha! Ok, this has to be linked prominently from the ndarray repo.

Updated the site, so same content except with more packages and searchable by package name:
http://scijs.net/packages
Yeah, should maybe link all the repos to their doc page or something...

Linked at the top. Closing this, although I don't think it'd be a bad idea to link every scijs repo to the scijs doc site.


cargo install cargo-wasi build failure

Hi, I am getting a build failure on Windows:
C:\Users\mylogin>cargo install cargo-wasi
   ...
   Compiling toml v0.5.5
   Compiling cookie_store v0.7.0
   Compiling reqwest v0.9.24
   Compiling cargo-wasi v0.1.18
error[E0433]: failed to resolve: use of undeclared type or module `cargo_wasi_src`
  --> C:\Users\mylogin\.cargo\registry\src\github.com-1ecc6299db9ec823\cargo-wasi-0.1.18\src\main.rs:43:32
   |
43 |         fn cargo_wasi_main() { cargo_wasi_src::main() }
   |                                ^^^^^^^^^^^^^^ use of undeclared type or module `cargo_wasi_src`

error: aborting due to previous error

I am trying to follow this instruction:
https://github.com/bytecodealliance/wasmtime-demos/tree/master/dotnet
The only preceding step was to install Rust from: https://win.rustup.rs/
Am I missing something or is there a problem in cargo-wasi?

I retried this on a clean machine and it went through without problems.
I am not sure what is the root-cause on my primary machine, but I am okay to close this issue.


Infinite recursion in normal representation of type (Z1K1)?

I’m trying to parse the normal JSON serialization, as described in the specification, and realized I don’t know what the type of an object is supposed to look like in that serialization.
In the JSON files shipped with eneyj, which are in canonical representation, the value of the Z1K1 key (the type) is a string literal like "Z2", which IIUC means that it’s a reference – if it was a string value, it would have to be represented as { "Z1K1": "Z6", "Z6K1": "Z2" }. And in the normal representation, Z1K1 is not a special key (only Z1K2 and Z6K1 are), so the reference (Z9) must be an object. Presumably, the reference ID (Z9K1) of that reference must be an object representing a string value… but what does the type of the reference look like?
{
  "Z1K1": {
    "Z1K1": {
      "Z1K1": {
        // ... infinite representation of a reference to Z9?
      },
      "Z9K1": {
        "Z1K1": /* reference to Z6 */,
        "Z6K1": "Z9"
      }
    },
    "Z9K1": {
      "Z1K1": /* reference to Z6 */,
      "Z6K1": "Z2" // actual type name goes here
    },
  // other keys of Z2 or other type go here
}

I suspect the specification needs to be amended to break this cycle; probably make Z1K1 another key that is serialized as a string literal.
I might be misunderstanding something, though. I tried to see if eneyj has a facility to show the normal serialization of a value, but couldn’t find it.

expand in src/utils/json.js should create the normal representation, and canonicalize the same place the canonical one. There is a script to do that, src/scripts/expander.js but that doesn't work for me right now.
(this is all IIRC, I haven't looked into the code myself for a while!)
I wrote the specification as a wishlist, it doesn't fit with the current kernel implementation. (That's obviously a bug)
The stable keys are currently   'Z1K1', 'Z1K2', 'Z6K1', 'Z11K2', 'Z16K1', 'Z16K2',
'Z18K1', 'Z60K1', 'Z70K1', 'Z180K2', none of these gets processed as described by the normalization / canonicalization. This list should become shorter - I think only Z1K1, Z1K2, and Z6K1 really need to be there.
But yes, as you say, Z1K1 indeed has to be there - that's an error in the Spec for exactly the reason you state and needs to be fixed.

Just FYI and as a comment, the reason why I didn't put Z1K1 there is because I haven't decided yet on how to deal with parametrised types. Should it just be a Z7 that creates a type on the fly from a given argument? Or something else? But this leads to the question what the identity of type is (and that's the reason for Z4K1 which is kinda weird), and down the whole rabbithole of type checking etc.
But really, that needs a proper solution, and until then I Z1K1 should probably just be treated as a stable key.

I fixed the expander script: fba8a4ba1f68345860c0a4a7ae4ddc63aabc42da
That at least should work now.

Fixed the specification to take that into account.

Thanks, and also thanks for explaining the x and * in the specification :)


github-actions: Add production build and install jobs.

With this all the jobs from Circle CI expect xenial-legacy will be migrated. The last commit may need adjustments because I am not sure I got all directories list correct for which we should run the production suite. For example, do we need requirements?

@alexmv @arpit551 @amanagr @andersk can you folks help review this?

Looks good to me!

Merged as 98d1c215def3caef27a44351fc8eea3a5478e267, after s/unit_tests/tests/ in the last commit, since some of our test suites are definitely not "unit test" suites.  Thanks @priyank-p!
I suppose we'll get some experience over the next week or two with how GitHub Actions and CircleCI compare in terms of reliability and speed, fix reliability bugs, and then hopefully we'll feel good about switching to just running GitHub.
Thanks @priyank-p!

Probably the next project we should do is reading the logs with ts to look for performance bugs (like caching not working).  @amanagr can help with this if needed.


Server error

HI,
My component was working but stopped so I removed and reinstalled it but now get this error when trying to start

Can you enable debug in your YAML and post logs. You might want to obscure email address before you do.

The only related lines I can find in the logs is
2023-12-15 10:54:17.189 ERROR (SyncWorker_6) [homeassistant.util.package] Unable to install package kingspan-connect-sensor==3.0.2: ERROR: Cannot install kingspan-connect-sensor==3.0.2 because these package versions have conflicting dependencies.
ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts
[notice] A new release of pip is available: 23.2.1 -> 23.3.1
[notice] To update, run: pip install --upgrade pip
2023-12-15 10:54:17.189 INFO (SyncWorker_6) [homeassistant.util.package] Attempting install of kingspan-connect-sensor==3.0.2
2023-12-15 10:54:17.189 DEBUG (SyncWorker_6) [homeassistant.util.package] Running pip command: args=['/usr/local/bin/python3', '-m', 'pip', 'install', '--quiet', 'kingspan-connect-sensor==3.0.2', '--timeout', '60', '--no-cache-dir', '--upgrade', '--constraint', '/usr/src/homeassistant/homeassistant/package_constraints.txt', '--find-links', 'https://wheels.home-assistant.io/musllinux/', '--prefer-binary']
homeassistant.requirements.RequirementsNotFound: Requirements for kingspan_watchman_sensit not found: ['kingspan-connect-sensor==3.0.2'].
Don’t know if this helps?
Thanks
From: Jon Connell @.>
Sent: 15 December 2023 10:35
To: masaccio/ha-kingspan-watchman-sensit @.>
Cc: tyep10trev @.>; Author @.>
Subject: Re: [masaccio/ha-kingspan-watchman-sensit] Server error (Issue #16)
Can you enable debug in your YAML and post logs. You might want to obscure email address before you do.
—
Reply to this email directly, view it on GitHubhttps://github.com/masaccio/ha-kingspan-watchman-sensit/issues/16#issuecomment-1857645223, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BC7MNQKI5U2W7MACA2CDF73YJQRURAVCNFSM6AAAAABAWHHUR6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNJXGY2DKMRSGM.
You are receiving this because you authored the thread.Message ID<EMAIL_ADDRESS>

Reinstallation of Home Assistant has fixed it


Default WSL version to 2 when wsl --import <distro> <dir> <file>

Windows Version
Microsoft Windows [Version 10.0.19044.1466]
WSL Version
No such command option.
Are you using WSL 1 or WSL 2?

[X] WSL 2
[ ] WSL 1

Kernel Version
5.10.16
Distro Version
Ubuntu 22.04.2 LTS
Other Software
No response
Repro Steps
wsl --import    # import any wsl2 backup
Expected Behavior
Getting a WSL2 distro by default
Actual Behavior
Getting legacy WSL1 instead if WSL version is not specified when importing.
Diagnostic Logs
n.a.

wsl.exe --set-default-version 2
wsl.exe --import <distro> <dir> <file> --version 2

wsl.exe --set-default-version 2

Shouldn't version 2 be the default out of the box?
If it is not the default on a brand new machine after WSL installation, please re-open the issue and make it default. @benhillis

Okay, if you think

that this issue is only about the setting the default version command, then I'll open a new issue raise the fact that the default import version before doing --set-default-version 2 is version 1
that having version 1 as the out-of-the-box import default is your desired behavior, then I'd open a new issue to argue why this is bad.

Please let me know what's in your head, @benhillis. Thanks.

Okay, if you think

that this issue is only about the setting the default version command, then I'll open a new issue to raise the fact that the default import version before doing --set-default-version 2 is version 1
that having version 1 as the out-of-the-box import default is your desired behavior, then I'd open a new issue to argue why this is bad.

Please let me know what's in your head, @benhillis. Thanks.

I've assumed 1 and opened https://github.com/microsoft/WSL/issues/10158.
If it is 2, let me know, @benhillis, PLEASE.


Support for Internationalization (i18n) in FEEL Editor

Hi there,
Awesome library, thank you for building this. I wanted to ask if it currently supports (i18n)?
If not, are there any plans to add this feature? Alternatively, how can we implement it manually?
Adding i18n support would enhance accessibility and improve the user experience
Thanks,
Pandika

Hi,
There is no built-in API for translations in feel-editor. However, as it is built on CodeMirror, I think we could add it quite easily with the phrases extension: https://codemirror.net/docs/ref/#state.EditorState^phrases. I imagine that we'd also need to add the extension to other dependencies, e.g. feel-lint.
Would you like to contribute it to the editor?


gist:sameTimeAs should be transitive

The gist:sameTimeAs property should be transitive.  If X sameTimeAs Y and Y sameTimeAs Z, then it is true that X sameTimeAs Z.

Its also reflexive.

Deferring because we may do away with TimeInstants based on issue #499.


add tests for supported services

At the moment there are no tests for most of the services, having tests for supported services will help greatly to keep the app stable.
Telegram
Slack
Pushover
Pushbullet
Ms Teams
RocketChat
Discord
Twillio

Is the work still relevant? I would like to do this


feature(engine): Using pydantic dataclasses for json-schema/validation

As a first step to a potential FastAPI migration,
this PR replaces the usage of dataclasses_jsonschema obsolete library
by the broadly adoptedpydantic.
BREAKING CHANGES

python dataclasses.dataclass is no longer supported in @dataobjects annotated classes. Pydantic @dataclasses (and in the future BaseModel should be used). Recommended way is to import from dataobjects module:

from hopeit.dataobjects import dataclass, dataobject, field

where dataclass and field are synomyms from pydantic.dataclasses.dataclass and pydantic.Field

@pcanto-hopeit this needs only one more iteration for release (in 0.25.0beta1 at least):

Remove unusued / commented-out code

For beta2 or maybe 0.26 I will rework dataframes plugin and remove some magic from the @dataframeobject class to make it more explicit. For now is not working as expected, but since the plugin is beta that should not stop the release.

@pcanto-hopeit : cleanup done, ready for review & test (please test simple-example and client-example apps)


OpenLineage should more actively track releases of projects we have integrations for

There are couple of instances of OpenLineage integrations erroring when external projects released new versions, and we only learnt it when we saw people having problems on Slack or in GH issues.
We should track releases more actively. As this is inherently manual process, "someone" should do it. Now, what would be in scope of this:

once or twice a week take a look at releases of supported integrations: Airflow, Spark, dbt, Dagster (...Flink, etc).
if new release is added, and we have automatic unit or integration tests, new version should be added to test matrix if possible. TBD if minor versions should be also treated this way.
if automatic testing does not exist, or is impossible for some reason (ex. project does not release docker images till later date), running a manual test should be ok.
if manual test is too time consuming - for example - person doing this does not have account on particular cloud provider or technical knowledge how to test - at least issue should be opened in OL GitHub.

Now, I did not specify who this "someone" should be. I propose volunteer rotation of OpenLineage committers. One rotation shouldn't take more than two weeks, and take more than 15 minutes time in those two weeks if there were no releases, or hour if there were.

I'd like to proposal a work that I'm thinking related to this issue:

Create a page in the repo (maybe in readme.md) which contains a table that lists each integrations, and specify:

link to its release page (usually github)
current supported(or tested version) version - and its last date
current latest version of the target integration


[BUG] Dropdown not working in all browsers

Will be fixed with Post CSS.

Fixed without postcss!! 😎 (But we will implement post anyway)


Will monitor can be the next feature? Like hook

Hi i got  an issue that when excelsizing there is no monitor to track how long the process. I use Elastic apm agent and got the gap in the tracer.
This is the tracer looks like

Hi @sangianpatrick sorry, I didn't understand what you mean.

@xuri hmm yah i mean, is it possible to add feature to watch the process when excelizing the data, because it takes 50 -100 ms untill it is done (20 rows, 7 column)
maybe like monitor function in golang mongdb or hook in go redis
imagine you want to add a span of excelize in new relic, data dog or elastic apm
refers to the picture above, fuel-direct-order.find is a span in golang mongodb monitor and it was traced in a single request
and if excelize has monitor function, we can add span of excelize in Excelize Gap

I think maybe this feature is not in the scope of the library, you can inject tracing code in your API call, I close this issue and welcome left comments if you have any more questions.


Used encapsulated system properties for AWS layer

PR for #230
AWS system properties are abstractized for easier testing. Tests do not have to set any aws system property anymore. See this post: http://www.amihaiemil.com/2017/02/24/each-system-property-a-class.html

Coverage increased (+1.8%) to 78.125% when pulling eae1dc8e1c3bd664da8ddfa29a09fa1b89e5f626 on 230 into 90fd3e57c62b85b0866c4ed5216d13421cef918b on master.

@rultor please merge this

@rultor please merge this

@amihaiemil OK, I'll try to merge now. You can check the progress of the merge here

@rultor please merge this

@amihaiemil Done! FYI, the full log is here (took me 2min)


Is this DbUnitTest Support Mybatis ?

Out Team use Mybaits plus and Koltin. I read the README.MD don't found mybatis caes .So wanna to ask support that ?
Thanks, I like this project.

Hi, Sorry for the late reply. I'm not familiar with Mybatis, would that still be using Spring Boot?

Having read through some docs, as long as you are using mybatis-spring this library should work fine
Just @Autowire in your repository / mapper class into your test in the place of JdbcTemplate in the examples

Having read through some docs, as long as you are using mybatis-spring this library should work fine
Just @Autowire in your repository / mapper class into your test in the place of JdbcTemplate in the examples

no problem, thanks friend.


Release 0.12.0

<!-- Make sure you have read http://napalm.readthedocs.io/en/latest/contributing/index.html --!>

Coverage decreased (-5.06%) to 78.556% when pulling ea10eddc7e4e43167aa8237cdfe9fb6926ac1b9c on develop into 5a3721a9f682b5e0ec70cc72ff92ca3471557825 on master.

Coverage decreased (-4.9%) to 78.759% when pulling 29026c1a53423bcd2a17d14c90d7da6d7269c381 on develop into 5a3721a9f682b5e0ec70cc72ff92ca3471557825 on master.

Coverage decreased (-4.9%) to 78.759% when pulling 29026c1a53423bcd2a17d14c90d7da6d7269c381 on develop into 5a3721a9f682b5e0ec70cc72ff92ca3471557825 on master.

Coverage decreased (-4.9%) to 78.759% when pulling 29026c1a53423bcd2a17d14c90d7da6d7269c381 on develop into 5a3721a9f682b5e0ec70cc72ff92ca3471557825 on master.


Refactor activity.js and activity-works.js to Remove Redundant Code and Improve Code Quality

Overview
This pull request refactors the activity.js and activity-works.js file to eliminate redundant code, enhance readability, and improve overall code quality. By organizing the code into a DrawingApp class and removing unnecessary repetitions, I made the codebase more maintainable and accessible for current and future contributor
Changes Made:-

Removed Redundant Code by Consolidating similar Functions
Wrapped related variables and functions within the DrawingApp class to encapsulate the this activities state and behaviour. Improved the modularity of the code.
Simplified the way event handlers are attached to objects, reducing complexity. And Ensured consistent use of this by leveraging closures and binding methods where appropriate.
Applied consistent code formatting for better readability.

[ ] commit message does not follow our conventions; https://github.com/sugarlabs/sugar-docs/blob/master/src/contributing.md#making-commits

Your changes would have been better done in more than one commit, separating formatting changes from refactoring changes.  It would have made it easier to review.

@quozl i have changes commit message to a better suitable one. For review you can just test all the features i have created a drawing app class to encapsulate all the working and removed duplicate code from activity-works.js. like most of the code was redundant there for some reason...

@quozl can you review this pr and if there are changes you can let me know....

The diff is unreadable due to length (276 + 528 lines) and change complexity, as if you have made edits while testing, then made one commit at the end.  Please break the commit up into smaller commits that show the changes in structure separate to changes in naming.
You suggest that review can reduce to testing of features.  What are the features you mention?  I've looked at README.md and run the activity and it is not clear what you mean by features.
@chimosky, you were involved in the previous pull request, have you a chance to review this one?  Thanks.

The diff is unreadable due to length (276 + 528 lines) and change complexity, as if you have made edits while testing, then made one commit at the end. Please break the commit up into smaller commits that show the changes in structure separate to changes in naming.
You suggest that review can reduce to testing of features. What are the features you mention? I've looked at README.md and run the activity and it is not clear what you mean by features.
@chimosky, you were involved in the previous pull request, have you a chance to review this one? Thanks.

@quozl I meant working of this activity by features like drawing, numbers swapning, and clearing canvas ets.. sure i will break this PR into smaller commits and add one more PR with these changes....

Thank you.  Looking forward to your changes.  You can and should replace the commit on this branch with a series of commits, rather than create a new pull request.  A new pull request displaces the discussion and confuses later readers.  But if you are lacking in time to do this, then a link back to the old pull request is at least a minimum courtesy.

@quozl I had updated it with my local branch in which i refactored the code can you review this one. As merging this PR will make this code much more readable for contributors....


Issues/kp 362 new search mobile view

In the latest change, I refactored the way of rendering different types of filters to make it easier to clear filters in different types of search. this approach will help us to add newer search easier in future as well.


Add Openstack Capacity to StackHPC Kayobe Config

This patch adds Openstack Capacity https://github.com/stackhpc/os-capacity to StackHPC Kayobe Config.

I think we need two more things.

Either some documentation on what the playbook is an how to deploy it OR a hook to run it automatically
A release note

Current CI failures are just from a lack of space on SMS. As long as one of them passes, I think we an force push these changes through

I would vote for some documentation, especially we require defined variables in secrets

I would vote for some documentation, especially we require defined variables in secrets

Updated the docs to include some secrets information.

Couple of tox docs errors but otherwise looks good

Couple of tox docs errors but otherwise looks good

Classic mistake :sweat_smile:

We should merge the PR on smslab that is pending first and ensure all comments from there are addressed to avoid having two open PR's with different discussions @assumptionsandg

Removed the playbook hook from service deploy to leave for manual deployment, as this exporter requires application credentials to be defined prior to deploying the exporter.

We should merge the PR on smslab that is pending first and ensure all comments from there are addressed to avoid having two open PR's with different discussions @assumptionsandg

I agree that we should resolve all the conversations, but I think it would be best to merge this one first, then sync smslab/yoga with skc/yoga so we don't end up with any merge weirdness. Or we could cherry-pick this commit to the SMS PR so the hash is the same which would have the same effect

Something funky seems to have happened to your rebase. Might need a bit of git untangling

Something funky seems to have happened to your rebase. Might need a bit of git untangling

Not sure what happened with that rebase, should be good now though! :smile:


Probably wrong and not working annotation in Fluid.Utilities

Reported by hubertus on 26 Aug 2014 21:14 UTC
The annotation for the derivative function to regRoot is at least weird, and probably not conform to the spec. It does not work in at least one Modelica tool.
l 59 in Utilities.mo is:
annotation(derivative(zeroDerivative=delta)=Utilities.regRoot_der,
and should be
annotation(derivative(zeroDerivative=delta)=regRoot_der,
Note that the regRoot_der function is defined in the same scope, so the Utilities prefix is at least superfluous. Lookup rules may even allow this,I am not sure, but it does not work in at least 1 popular tool.
I am not sure if there is any systematic test for derivative functions, and smoothOrder annotations and the like either.

Migrated-From: https://trac.modelica.org/Modelica/ticket/1559

Comment by hubertus on 27 Aug 2014 15:49 UTC
Fixed in trunk in 2b9b1ca9fa70ae06bd6881884e642ec25b544074. I'll close this, we should probably open a ticket for a broader check for derivatives

Modified by beutlich on 23 Oct 2015 07:50 UTC

Changelog modified by beutlich on 23 Oct 2015 07:50 UTC
Fixed derivative annotation of Modelica.Fluid.Utilities.regRoot


add a file with one single line (and no newline at end of file)

Trivial: Add a file that does not have a newline at the end of file to test the functionality introduced in #28 and #32

Thanks.


RuntimeError: No points provided

I am trying to run the det_sam2_RT.py script with the following code:
if __name__ == '__main__':
    sam2_output_frame_dir = './temp_output/video_frames'
    sam2_checkpoint_path = '../checkpoints/sam2.1_hiera_base_plus.pt'
    sam2_config_path = './configs/sam2.1/sam2.1_hiera_b+.yaml'
    detect_model_weights = '/app/det_weights/train_referee12_960.pt'
    output_video_dir = './pipeline_output'
    video_path = '/app/det_sam2_inference/example/crowd.mp4'
    save_inference_state_path = '/app/autodl-tmp/output_inference_state/inference_state_10frames.pkl'
    processor = VideoProcessor(
        output_dir=output_video_dir,
        sam2_checkpoint=sam2_checkpoint_path,
        model_cfg=sam2_config_path,
        detect_model_weights=detect_model_weights,
    )
    processor.run(
        video_path=video_path,
    )

Do I understand correctly that it should be self-prompted and I don't need to specify the points? However, I encounter the following error:
---最大累积帧数/frame_buffer_size:30,检测间隔/detect_interval:30,最大传播长度/max_frame_num_to_track:60,推理保留帧数/max_inference_state_frames:60,预加载内存库/load_inference_state_path:None,
Initializing: offload_video_to_cpu:True,offload_state_to_cpu:False
Traceback (most recent call last):
  File "/app/det_sam2_inference/det_sam2_runtime.py", line 715, in <module>
    processor.run(
  File "/app/det_sam2_inference/det_sam2_runtime.py", line 611, in run
    self.inference_state = self.process_frame(self.pre_frames + frame_idx, frame_rgb)
  File "/app/det_sam2_inference/det_sam2_runtime.py", line 441, in process_frame
    self.Detect_and_SAM2_inference(frame_idx)
  File "/app/det_sam2_inference/det_sam2_runtime.py", line 394, in Detect_and_SAM2_inference
    for out_frame_idx, out_obj_ids, out_mask_logits in self.predictor.propagate_in_video(
  File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 36, in generator_context
    response = gen.send(None)
  File "/app/sam2/sam2_video_predictor.py", line 1119, in propagate_in_video
    raise RuntimeError("No points provided; please add points first / 没有提供点；请先添加点")
RuntimeError: No points provided; please add points first / 没有提供点；请先添加点

What could be the cause of the error?

Make sure that the first frame in the video is a conditional frame—that is, the first frame in the video can be detected by the detection model.
For example, if you use the detection model I provided (in the billiards scenario), then the video should contain any billiard object at the beginning for SAM2 to reason properly.
You can uncomment in the det_sam2_RT.py VideoProcessor.detect_predict() function:
print(f"Frame index (excluding pre-loaded): {absolute_indices[i]-self.pre_frames}: Coordinates: {coords}, Class: {cls}, Confidence: {conf}")
to confirm that the detection model captured any objects in the first frame

Am I correct in understanding that your model does not perform online tracking since the number of object IDs is limited by the number of classes (as in your case, where each ball has its own class)? In other words, there is no mechanism for assigning new IDs (as in a classic online tracker), correct?

Yes. Detection model detects categories, from the category concept to the gap of the object concept of SAM2 I can't think of any good way to solve it.

So, another point to be aware of is to make sure that only one object appears in each category detected by the detection model.
The detection model in Det-SAM2 wants to play a role in prompting correction, so when two identical detection categories appear at the same time, the detection model cannot tell the correspondence of objects in the same category before and after SAM2

We can't easily assign IDs to the same categories of detection models. Because we need to make sure that the prompt input of SAM2 is absolutely correct.


lambda requires IAM configuration

I've set:
export INITIAL_NO_AUTH_ACTION_COUNT=100

and then running moto server for testing lambdas:
moto_server lambda -p 5555

when trying to create a lambda function:
aws lambda --region=us-east-1 --endpoint-url=http://localhost:5555 create-function --function-name gsoc20 --zip-file fileb://file.zip --handler index.handler --runtime python --role arn:aws:iam::123456789012:role/lambda-ex

i get the following error:
An error occurred (400) when calling the CreateFunction operation: The role defined for the function cannot be assumed by Lambda.

Is it possible to define a lambda function without defining a role first?

Hi @yuvalif!

Is it possible to define a lambda function without defining a role first?

That's not possible, no. Moto implements some basic checks to verify that the role exists, mimicking the AWS behaviour around this feature.
Creating the role is possible, as long the moto_server is started without any service parameter:
moto_server -p 5555

This will start a server for every service, so that both IAM and AWSLambda can talk to the same endpoint.
The unmodified IAM call will now succeed:
aws iam --region=us-east-1 --endpoint-url=http://localhost:5555 create-role --role-name "lambda-ex" --assume-role-policy-document "some policy" --path "/my-path/"

The AWSLambda call needs a different role-arn-parameter, as the path is also part of the ARN:
aws lambda --region=us-east-1 --endpoint-url=http://localhost:5555 create-function --function-name gsoc20 --zip-file fileb://file.zip --handler index.handler --runtime python --role arn:aws:iam::123456789012:role/my-path/lambda-ex

@bblommers thank you for the reply!
This fixed the role and lambda creation issue.

Happy to hear that! Will close this, but let us know if you run into any other issues.


Implings plugin: Add Crystal implings

Should be added to #9459

These look like they are a lower level than dragon imps


HHH-14408 Fix wrong property type for integrator provided user types applied 5.4

Backport of https://github.com/hibernate/hibernate-orm/pull/3712 for 5.4

I'm experimenting with a patch on the Quarkus side which might make this less brittle ( and make your patch work)

The problem is that the validation requires property types which might not be available yet because integrators can contribute them. I deferred the validation until after the integrators ran so that this is not an issue.

Right, so I think the problem is the Integrator contract: integrators are passed a SessionFactory, but in its very nature the SessionFactory should be immutable - so it's too late.
Like the javadoc on SessionFactoryImpl states:

This class must appear immutable to clients, even if it does all kinds of caching and pooling under the covers.

IMO it's not legal to expect Integrator (s) to be allowed to mutate its state other than enlist listeners, but here it seems it's used also to register user types?
Not sure how to crack this w/o changing the SPIs.  @sebersole WDYT, is this something we could improve on in 6?
I'll try workaround this in Quarkus but it's a big challenge to not being able to rely on the metadata and metamodel being "fully complete" yet when we're constructing the SessionFactory. IMO if there's a need for intermediate steps, we should represent them with an intermediate object to hold the intermediate state.

IMO it's not legal to expect Integrator (s) to be allowed to mutate its state other than enlist listeners,

To elaborate more on that; I wonder if even the "enlist liteners" aspect is a bit questionable: I mentioned it to be ok only because it's perhaps less problematic and I tend to be pragmatic about such extension needs.
But thinking in terms of better design, pehaps we should just expose the Metadata and ServiceRegisty; this would allow modifying EventListenerRegistry, etc.. and all other components in a previous phase.
I suspect that the SessionFactory is handed over because a reference to it might be useful? We should look at the use cases and see if we can defer providing the reference.

User types are registered through the org.hibernate.boot.spi.MetadataImplementor. The SPI even says that

The session factory being created

i.e. it suggests that it is in the process of creation. IMO it's fine how it is, but we could certainly try to move integrators out. The only issue then is that we have to move certain other parts out as well and pass a different context object to the integrators so that they can do changes. Maybe we can try to split the validation into two steps instead to do whatever is causing problems for Quarkus earlier?

yes the SPI is being clear, but "session factory being created" is a bit vague - I guess intentionally :)
I'd prefer to have the integration-with-integrators moved out, but we can't do that today as the Integrator SPI expects to be passed a reference  to the SessionFactory instance. Which doesn't exist before we invoke its constructor ..
Regarding validating in two steps.. maybe that's a compromise, but I'm not sure how far that makes sense a I would prefer to being able to run all validations (including the second step) as part of the build. But of course while that's ideal, if we hit a roadblock we'll only validate what's possible.
Anyhow, I managed to workaround this change in Quarkus now, and it's not looking too bad so we could merge this. Just be aware that (in Quarkus only) this validation phase will still run earlier. We also don't support such integrators yet, so we can face that problem when we get there.

Workaround how? What do you mean by we don't support such integrators yet?
The proper way for contributing types in Hibernate is IMO to use a org.hibernate.boot.spi.MetadataContributor and do it there as that is called while the metadata is being built. We could tell users to just switch to this, but I don't know if there are other purposes for integrators that could interfere with the metadata.

IMO the problem is that we pass the Metadata which allows changes. Maybe for 6.0 we can drop that parameter?

Workaround how?

Not very polished yet but this works:

https://github.com/quarkusio/quarkus/commit/9dbb5c5566d454e14c39617bf0835eeb867c26c7

As dicussed, but I'll repeat it here as note to share with others, the drawback of my Quarkus workaround is that the validation() invocation happens before the last-minute Integrator (s) are applied. This gets mitigated by the fact that I don't really know of Integrator implementations which are supposed to work, we could improve that bommit by scanning for them and logging a warning about how to do the better integration (as discussed below).

What do you mean by we don't support such integrators yet?

Just that in Quarkus most implementations of Integrator are unlikely to work OOB; they might work with some luck but I'd prefer adding explicit extensions for the most popular ones.
And once we have extensions, we could make them bind over the org.hibernate.boot.spi.MetadataContributor like you suggest: I agree that's the better way forward for this use case.
I'll merge this next; hopefully we can improve on this soonish but that's not something orthogonal to this PR. Thanks!


example loading popup only some columns

Hi! this plugin is great,
the only thing i need to show only certain columns on the popup:
function popUp(f,l){
    var out = [];
    if (f.properties){
        for(key in f.properties){
            out.push(key+": "+f.properties[key]);
        }
        l.bindPopup(out.join("<br />"));
    }
}

i need to only bring 2 specific columns from the geojson:
column names are: 'name_depto' and 'postal'
thanks!!!

this works identically to the default leaflet feature collection so instead of the loop just grab those specific columns

:) thanks!


Error Getting a list of running containers

Output of docker version:
Client:
 Version:      1.12.1
 API version:  1.24
 Go version:   go1.6.3
 Git commit:   f1e1b83
 Built:
 OS/Arch:      linux/amd64

Server:
 Version:      1.12.1
 API version:  1.24
 Go version:   go1.6.3
 Git commit:   f1e1b83
 Built:
 OS/Arch:      linux/amd64

Output of docker info:
Containers: 2
 Running: 1
 Paused: 0
 Stopped: 1
Images: 2
Server Version: 1.12.1
Storage Driver: overlay
 Backing Filesystem: extfs
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge overlay null host
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Security Options: selinux
Kernel Version: 4.7.1-coreos
Operating System: CoreOS 1151.0.0 (MoreOS)
OSType: linux
Architecture: x86_64
CPUs: 1
Total Memory: 7.801 GiB
Name: core-01
ID: 3D66:DYWJ:CZIL:ESFB:CW3S:UTSG:VFZY:YBLT:E234:H7L2:OIOT:Y7ZL
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Username: easywebdeploy
Registry: https://index.docker.io/v1/
Insecure Registries:
 <IP_ADDRESS>/8

Additional environment details (AWS, VirtualBox, physical, etc.):
Running CoreOS Alpha 1151.0.0 On Vagrant 1.8.5 Ubuntu 16.04
Steps to reproduce the issue:

Run a simple container and make sure its running
Run "curl -v --unix-socket /var/run/docker.sock http:/containers/json" to get a list of Containers

Describe the results you received:
*   Trying /var/run/docker.sock...
* Connected to containers (/var/run/docker.sock) port 80 (#0)
> GET /json HTTP/1.1
> Host: containers
> User-Agent: curl/7.50.1
> Accept: */*
>
< HTTP/1.1 404 Not Found
< Content-Type: application/json
< Date: Mon, 29 Aug 2016 08:48:02 GMT
< Content-Length: 29
<
{"message":"page not found"}
* Connection #0 to host containers left intact

Describe the results you expected:
Expected is the JSON list of all running containers
Additional information you deem important (e.g. issue happens only occasionally):
Was working correctly on CoreOS with Docker version till 1.11.2

@mindnuts hi, I get the exact same output
GET /json HTTP/1.1
Host: containers

when I use 2 / in  http://
It was to be only one, are you sure you didn't do a typo ? see:
$ curl -v --unix-socket /var/run/docker.sock http://containers/json
*   Trying /var/run/docker.sock...
* Connected to containers (/Users/victorvieux/Library/Containers/com.doc) port 80 (#0)
> GET /json HTTP/1.1
> Host: containers
> User-Agent: curl/7.43.0
> Accept: */*
>
< HTTP/1.1 404 Not Found
< Content-Length: 29
< Content-Type: application/json
< Date: Mon, 29 Aug 2016 18:14:56 GMT
<
{"message":"page not found"}
* Connection #0 to host containers left intact

$ curl -v --unix-socket /var/run/docker.sock http:/containers/json
*   Trying /var/run/docker.sock...
* Connected to http (/Users/victorvieux/Library/Containers/com.doc) port 80 (#0)
> GET /containers/json HTTP/1.1
> Host: http
> User-Agent: curl/7.43.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Content-Length: 3
< Content-Type: application/json
< Date: Mon, 29 Aug 2016 18:15:04 GMT
< Server: Docker/1.12.1 (linux)
<
[]
* Connection #0 to host http left intact

@vieux No dice
core@core-01 ~ $ curl -v --unix-socket /var/run/docker.sock http://containers/json
*   Trying /var/run/docker.sock...
* Connected to containers (/var/run/docker.sock) port 80 (#0)
> GET /json HTTP/1.1
> Host: containers
> User-Agent: curl/7.50.1
> Accept: */*
>
< HTTP/1.1 404 Not Found
< Content-Type: text/plain; charset=utf-8
< X-Content-Type-Options: nosniff
< Date: Tue, 30 Aug 2016 06:54:04 GMT
< Content-Length: 19
<
404 page not found
* Connection #0 to host containers left intact

Did you use the same Docker version?:
core@core-01 ~ $ docker version
Client:
 Version:      1.11.2
 API version:  1.23
 Go version:   go1.5.4
 Git commit:   4a6e2b1
 Built:
 OS/Arch:      linux/amd64

Server:
 Version:      1.11.2
 API version:  1.23
 Go version:   go1.5.4
 Git commit:   4a6e2b1
 Built:
 OS/Arch:      linux/amd64

@mindnuts no I didn't, I used 1.12
what's weird to me is that part:
GET /json HTTP/1.1
Host: containers

@mindnuts yeah I said you have to use one / only.

GET /json HTTP/1.1

This url is wrong, the correct url is /containers/json. The issue is how curl is interpreting the url.
I had to use: curl -v --unix-socket /var/run/docker.sock http://localhost/containers/json to get it to work (on alpine:3.4).  If this command is documented somewhere in the docker documentation, please link it here, and we will make sure to correct the documentation.
Closing this issue since there is nothing to be done in docker/docker.

@dnephin Here you go, link to Docker documentation

Still not fixed in the docs, why the issue is closed?

The url in the docs worked for @vieux above. I'm not sure which version of curl supports which form.

Mine works only with curl -v --unix-socket /var/run/docker.sock http://localhost/containers/json. curl version is:
➜  ~ curl --version
curl 7.50.1 (x86_64-pc-linux-gnu) libcurl/7.50.1 OpenSSL/1.0.1f zlib/1.2.8 libidn/1.28 librtmp/2.3
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp smb smbs smtp smtps telnet tftp
Features: IDN IPv6 Largefile NTLM NTLM_WB SSL libz TLS-SRP UnixSockets

This issue seems to have started since version 7.47.1 (so version higher than 7.47.1).
On 7.47.1:
 curl -v --unix-socket /var/run/docker.sock http:/containers/json
*   Trying /var/run/docker.sock...
* Connected to http (/var/run/docker.sock) port 80 (#0)
> GET /containers/json HTTP/1.1
> Host: http
> User-Agent: curl/7.47.1
> Accept: */*
>
< HTTP/1.1 200 OK
< Content-Type: application/json
< Server: Docker/1.12.1 (linux)
< Date: Tue, 06 Sep 2016 16:24:29 GMT
< Content-Length: 884
<
[{"Id":"c6ccd147723dd414ca5fa582b8326752df8eb4ac9fe22e6ee8b743743366e669","Names":["/sleepy_franklin"],"Image":"alpine","ImageID":"sha256:4e38e38c8ce0b8d9041a9c4fefe786631d1416225e13b0bfe8cfa2321aec4bba","Command":"sh","Created":1473178577,"Ports":[],"Labels":{},"State":"running","Status":"Up 8 minutes","HostConfig":{"NetworkMode":"default"},"NetworkSettings":{"Networks":{"bridge":{"IPAMConfig":null,"Links":null,"Aliases":null,"NetworkID":"2cb5d0f6d1cf11f788919f66f5a30d55404897c88a0e4fb387c3acde4533ade4","EndpointID":"d6a2cba4167f43fa023cfbf97f59e9e26695658d5b3c93d26ffb2e1688194c64","Gateway":"<IP_ADDRESS>","IPAddress":"<IP_ADDRESS>","IPPrefixLen":16,"IPv6Gateway":"","GlobalIPv6Address":"","GlobalIPv6PrefixLen":0,"MacAddress":"02:42:ac:11:00:02"}}},"Mounts":[{"Source":"/var/run/docker.sock","Destination":"/var/run/docker.sock","Mode":"","RW":true,"Propagation":"rprivate"}]}]

On 7.50.1:
curl -v --unix-socket /var/run/docker.sock http:/containers/json
* Unwillingly accepted illegal URL using 1 slash!
*   Trying /var/run/docker.sock...
* Connected to containers (/var/run/docker.sock) port 80 (#0)
> GET /json HTTP/1.1
> Host: containers
> User-Agent: curl/7.50.1
> Accept: */*
>
< HTTP/1.1 404 Not Found
< Content-Type: application/json
< Date: Tue, 06 Sep 2016 16:24:35 GMT
< Content-Length: 29
<
{"message":"page not found"}
* Connection #0 to host containers left intact

The command which works on both versions is: curl --unix-socket /var/run/docker.sock http://localhost/containers/json so I suggest changing the documentation to this command.

@gerwim thanks for confirming and finding the version.  Would you be interested in creating a PR to fix this in the docs? I believe the file is docs/reference/api/docker_remote_api.md

LGTM. @sfsmithcha PTAL

I think this doc change is not enough to describe the issue for everybody. Double slash is not enough to fix this problem in all cases, as is pointed out earlier in this thread. Are we sure we are not breaking with the HTTP spec with this? Anyway, I think the doc should atleast reflect the possible issues that recent curl version can bring. Currently I see three working formats, only one of which works on version 7.50.3:

"Old way", 'http:/info' Gets accepted but corrected into 'http://info' by curl with warning Unwillingly accepted illegal URL using 1 slash!
The version we are now correcting the docs to http://info, gets accepted by curl, but it interprets it as a "proper" url, making "info" the host and "/" the path
My current hack, inserting anything in before the first slash, makes curl resolve with a hostname, but it gets ignored due to the unix socket, e.g. curl --unix-socket /var/run/docker.sock http://john/info

The way I read the HTTP spec for urls, I  think solution 3 is the only way to comply with it

docs were updated including both examples for 7.4x / 7.5x, see https://github.com/docker/docker/pull/27640
I'll go ahead and close this issue, but @cstpdk please check the linked PR if there's still something not right

@thaJeztah
Just so you know, on this page https://docs.docker.com/engine/reference/api/docker_remote_api/
It still says curl --unix-socket /var/run/docker.sock http:/containers/json. I had to come here to figure it out.

@CarltonSemple Thanks for reporting. I think these changes are not yet synced to the new docs repository, so not published yet. I'll see if we can get them published soon


Exchange Extended Protection & Netscaler Content Switching

Can someone please clarify the known issue "SSL offloading or SSL termination via Layer 7 load balancing" in combination with Netscaler Content Switching?
https://microsoft.github.io/CSS-Exchange/Security/Extended-Protection/
I read a lot about this and wording is always a little bit different which makes understanding the impact not easy. We use Citrix Netscaler Content Switching without SSL Offloading. We use the same certificate on Netscaler and Exchange. I also talked to our network guy and he is also not sure if Content Switching blocks us from using Extended Protection.
Would it be possible for someone to be a little bit more specific about content switching?
If I enable it and it is not working with our setup - can I go back without any problem with the script?
Thanks a lot for your help!

I'll take a stab at this. I read the Content Switching doc I found here: https://docs.citrix.com/en-us/citrix-adc/current-release/content-switching.html
Note the doc states the following: "Content switching can be used with HTTP, HTTPS, TCP, and UDP connections. For HTTPS, you must enable SSL Offload." So, I'm a bit confused about the statement that it is being used without SSL Offload.
Since Content Switching involves looking at the URL on the request, that means TLS must be terminating at the Netscaler, because it must open up the TLS stream and read the HTTP method and such. That means that the Channel Binding Token is invalid when the request is sent on to the Exchange server, even if a new TLS connection is established from the Netscaler to Exchange using the same cert. This is exactly the type of man-in-the-middle scenario that Extended Protection is intended to protect against. Thus, I would expect that Extended Protection cannot be used in this scenario.
If there is indeed a way to use Content Switching without terminating the TLS session in the middle - for example, if Content Switching can act purely based on source IP or something else exposed at the TCP layer and leave the TLS session untouched - then it's possible Extended Protection could work in that type of configuration. However, the docs imply it's not possible to do that.
@tweekerz @lusassl-msft Can I get a sanity check on the above? Does this sound right?

Thanks for your reply! You exactly hit what I am thinking about...
My understanding for SSL Offloading is, that the connection is terminated on Netscaler and the backend connection from Netscaler to Exchange is HTTP. This is not the case in our setup! And also the official site has this understanding I think:
https://microsoft.github.io/CSS-Exchange/Security/Extended-Protection/

Here is a guide on how to setup Content Switching with Netscaler. We didn´t followed this specific guide, but it shows how the cofiguration works.
https://citrixguyblog.com/2017/07/22/citrix-netscaler-loadbalancing-exchange-20132016-walkthrough-guide/
My understanding of what I wrote so far would be, that it should work. But when I start the script, I get this warning:
Known Issues: Following scenarios will not work when Extended Protection is enabled.
SSL offloading or SSL termination via Layer 7 load balancing
Our Network guy said Content Switching is kind of Layer 7 as we check the URL and redirect the request to the correct backend as cou can see in the setup guide i linked above. And we also terminate the SSL connection on Netscaler, but the backend connection is again SSL.
After reading and thinking about all this information, I am really not sure if this would work or not. Unfortunately I have no test environment and just because the documentation states going back is easy, it must not be the cas in this scenario.
Thanks a lot for your help!

@bill-long I read the same as you regarding SSL offloading. It will be performed on the device in the middle and not on the Exchange server. My understanding of this is that it can be considered as SSL bridging, which should work if the same certificate is used on the device in the middle (Netscaler for example) and Exchange Server. The channel binding token (CBT) is a hash of the certificate which is used and therefore it must be the same on all devices, otherwise Exchange would reject the connection. But let me double-check internally.

@lusassl-msft @Deas-h Based on a discussion in an email thread, the consensus seems to be that Extended Protection should actually work as long as the Exchange server and the device have the same certificate.

@bill-long Thanks for clarifying this! I will do the change next week in a maintenance window and update this thread about the result.
@All - thanks a lot for your help!!!

I can confirm now, that Citrix Netscaler LB with Exchange Content Switching configuration is working with Extended Protection. In the meantime I was also in contact with Frank Carius from MSXFAQ and he said, that he implemented Extended Protection successfully with KEMP. He also confirmed what @lusassl-msft said - the important part is that the certificate used MUST be the same on the load balancer and exchange. With this prerequisite it should work on all load balancers.
I think it would be helpful for others if you could use this information to update the documentation about SSL bridging and "SSL termination via Layer 7 load balancing"
Thanks again for your help!!!

Assigned to @tweekerz to update the documentation

@Deas-h we have the following wording in the EP documentation. Please let us know if this is sufficient:
SSL Bridging supported scenarios section:
Extended Protection is supported in environments that use SSL Bridging under certain conditions. To enable Extended Protection in your Exchange environment using SSL Bridging, you must use the same SSL certificate on Exchange and your Load Balancers. If not this will cause Extended Protection to fail.
Q&A section:
Q: While we understand that preventing MitM attacks is important, can we have our own devices in the middle with our own certificates?
A: If the device uses the same certificate as the Exchange Server, they can be used.

Thanks for the wording - I think this explains everything and makes clear what the important part is!
Thanks again for your help!!!

Okay, great. Thanks for your feedback and feel free to reach out to us if you have any further feedback.

@Deas-h - I am looking at implementing this, and wanted to confirm your results: If the CSVS and LBVS's all have their protocol set to 'SSL', and the same certificate is loaded in the NetScaler and Exchange servers, then you were able to enable Extended Protection successfully?

My LBVS ist SSL, my CSVS is SSL and they all use the same certificate. YES, Extended Protection is working without a problem in this setup since more than one year.
I already confirmed this in a post above:
https://github.com/microsoft/CSS-Exchange/issues/1463#issuecomment-1409911092

@Deas-h , just for clarity CS vs bound cert needs to matches with exchange right?  nothing to do with ssl profile/cert on service groups?

LBVS, CSVS and Exchange have alle the same certificate bound to it. Service Groups don´t have a cert bound to them.

Thanks @Deas-h

Thanks @Deas-h - appreciate your response.


SSR; statics

I noticed that none of the x in navigator statements are guarded, so they will all break when rendering server-side. I propose adding typeof navigator !== 'undefined' in front of all those.
Furthermore, with the exception of the networking, the hooks return static data and even use useState to store the static data, causing extra work and memory use for something that's basically a single static value that can be determined at module load time. This confuses me, I created #1 to explain what I mean.

I noticed that none of the x in navigator statements are guarded, so they will all break when rendering server-side. I propose adding typeof navigator !== 'undefined' in front of all those.

I'm supportive of us introducing a guard to avoid breakage during server-side rendering. This is a good call and is something we discussed but didn't circle back on. I'd be open to a PR with this change.

Furthermore, with the exception of the networking, the hooks return static data and even use useState to store the static data

Shifting to a model where we treat all hooks short of network as providing static values likely makes sense. We started off our implementation looking at additional APIs that fired change events, but given the current scoped set doesn't heavily do this, a change here sgtm as long as tests pass.

@addyosmani so your preferred API would still require calling an ersatz hook that returns the static values, in case these values become dynamic in the future?
I think the tests can pass by messing with require.cache, I'll take a look.

ok, the static hooks keep their API and the tests pass in #1

Thanks once again for #1, @wmertens. I believe with the guards in place and shift away from using useState we should be good on the SSR side here. Just wanted to check there isn't anything remaining to be done for this issue before I close it?

No, I can't think of anything else 👍

@wmertens @addyosmani Wouldn't this cause hydration to fail? Something like useNetworkStatus would return different results server side and client side, and I'm not sure if react would handle it correctly in all cases.
Usually what is done is to move rendering to three phases

Once Server Side
Once client side during hydration, where the value is same as the server side
Update a flag on componentDidMount / useEffect, which causes the value to be set correctly

Right, but in this case, either you probably want hydration to fail, so
that the conservative estimate that the server did is replaced with the
correct estimate on the client side. React will only fail to hydrate the
subtree, so it should work optimally.

@wmertens I don't think react will fail. I have created a sample repository demonstrating the issue, and moved this into a new issue over here: https://github.com/GoogleChromeLabs/react-adaptive-hooks/issues/18


Create 404 page

It's just for now... Design better one later!

404 page looks good, but I don't believe this will fix the routing behaviour I'm referring to. For example, just click this link https://lifeline.techstartucalgary.com/app/ and you'll get a 404.

I mean cus this PR hasn't merged yet, so it's not on production yet and apparently that link will still give 404...
In my local environment it's working good, you should swicth to the branch and test locally.

I mean cus this PR hasn't merged yet, so it's not on production yet and apparently that link will still give 404...
In my local environment it's working good, you should swicth to the branch and test locally.

Going directly to http://localhost:3000/app work on main and this branch, but not in build (and consequently production).
On this branch when built, the only way to get this 404 page is to go to localhost/index.html ironically.

I mean cus this PR hasn't merged yet, so it's not on production yet and apparently that link will still give 404...
In my local environment it's working good, you should swicth to the branch and test locally.

Going directly to http://localhost:3000/app work on main and this branch, but not in build (and consequently production).
On this branch when built, the only way to get this 404 page is to go to localhost/index.html ironically.

Not sure how it doesn't work for you. On my laptop I'm running the vite preview command and it works perfectly, and this approach worked for my previous projects.
https://user-images.githubusercontent.com/20573623/218286731-6c60eab7-d69a-4302-8e37-7d05f5479ddc.mov
If you have to access a page from index.html then that means your env is not routing all urls to the app and that's not expected.

Okay, we'll just try it and if it doesn't work we'll fix it

😂 Github pages simply is not routing urls to react app, if you type in any route, eg https://lifeline.techstartucalgary.com/app, it'll always be 404.

Yep... This is what I was trying to explain
On Sun, Feb 12, 2023, 12:34 AM Yunfan Yang @.***> wrote:

😂 Github pages simply is not routing urls to react app, if you type in
any route, eg https://lifeline.techstartucalgary.com/app, it'll always be
404.
—
Reply to this email directly, view it on GitHub
https://github.com/techstartucalgary/lifeline/pull/120#issuecomment-1426962895,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AQZPOEWC26NBNUQIB6LEVRTWXCHBVANCNFSM6AAAAAAUYO4TKU
.
You are receiving this because you modified the open/close state.Message
ID: @.***>

Oh this PR provides a solution to display our customized 404 page when a page is not found and all urls are properly routed to react app as a pre-condition, otherwise even if all urls are routed properly, it's still gonna show Github pages 404 page.
Regarding the routing issue it's likely caused by vite migratio, similar to this PR, so... it's out of the scope of PR, open a new one! 🤣


Admin endpoint

Our next target is to create endpoint to enable admin carry put the following functionalities.
1 Add, Edit, Delete of coin form the database
2 Add, Edit, Delete of exchanges from the database
3 Assign coin to exchanges
4 Add, Edit, Delete of blog post
@Wisdom132 will handle -- 1
@NoorAhmd  will handle -- 2
@melodyogonna  will handle --- 3
@Emibrown will handle -- 4
We have to complete all the above task within 24hours from now.

But first @Wisdom132 you have to setup authentication for the Admin.

But first @Wisdom132 you have to setup authentication for the Admin.

@Emibrown do we have exchange collection ready in DB?

@Emibrown do we have exchange collection ready in DB?

@NoorAhmd yes we have the exchange collection setup already https://github.com/coinratecap/Backend/blob/main/src/modules/models/Exchanges.js

@NoorAhmd yes we have the exchange collection setup already https://github.com/coinratecap/Backend/blob/main/src/modules/models/Exchanges.js

How does assigning coins to exchanges work?

How does assigning coins to exchanges work?

The exchange schema is referencing an array of coin, all you need to do is create an endpoint that accept multiple or single coin by _id and push them to the coin field on the exchange schema.

The exchange schema is referencing an array of coin, all you need to do is create an endpoint that accept multiple or single coin by _id and push them to the coin field on the exchange schema.

Oh, cool.

Oh, cool.

@Emibrown I can't find the coin field in the exchanges schema

@Emibrown I can't find the coin field in the exchanges schema

@melodyogonna please create one and make it an array of Coin ID
 coin: [{             type: mongoose.Schema.Types.ObjectId,             ref: 'Coin'         }]

@melodyogonna please create one and make it an array of Coin ID
 coin: [{             type: mongoose.Schema.Types.ObjectId,             ref: 'Coin'         }]


Atelier Sophie ~The Alchemist of the Mysterious Book~ [PCSE00892]

Game summary

Game name: Atelier Sophie -The Alchemist of the Mysterious Book-
Game serial: PCSE00892
Game version: 01.00

Vita3K summary

Version: v0.1
Build number: 1013-54f39b4
Commit hash: https://github.com/vita3k/vita3k/commit/54f39b4

Test environment summary

Tested by: HorrorTroll
OS: Windows 10
CPU: Intel Core i5-4200H
GPU: Nvidia Geforce GTX 950M
RAM: 6 GB DDR3

Issues

Unicorn crash and unsupported texture format!
Screenshots

Log
vita3k.log

Game plays without issues