What should the schema look like for results?

This project doesn't have a formal process for enhancements, so let's use an issue.
The goal of this issue is to understand what we need to write to persistent storage, and come up with a schema. Once we have agreement, we can port those schemas to the applicable open issues.

[ ] Define result schema
[ ] Define profile schema
[x] Define an assessment schema
[ ] Determine how we should handle deletes (e.g., SET NULL or DELETE)
[x] Define subject schema
[x] Update subject schema to allow for hierarchical relationships
[x] Define metadata schema
[x] Determine how we want to store dates
[x] Determine a consistent format for identifiers

Terminology
result is the outcome of a compliance scan. It includes information like the outcome of the scan, the control the rule may be associated to, along with general information about the rule.
subject is the target of a scan, or the piece of infrastructure that was assessed.
profile is the baseline that results or rules refer to (e.g., NIST 800-53).

I added the initial schema for 'subjects' in https://github.com/rhmdnd/compserv/commit/3b2d322b536975cbbfea7de3b966b87fd36242d8.
I'd like to keep it general, which is why there is only three columns (id, name, and type). An example would be 02b26437-0fb3-4d2a-b788-8c2b1cc640de, https://api.example..com, cluster).
From there, we would have a foreign key to other tables that store the results (allowing us to query results by the cluster).
Thoughts?

For results, I think we could start with id, name, control, outcome, and description.
id
Required: True
Type: UUID
Exposed publicly: True
An RFC 4122 UUID. We can use an existing go module to generate them, specifically UUID4 (which are randomly generated).
Should we expose an ID for each resource?
Is it safe to assume the gRPC API is going to expose a resource ID so that clients could call compservClient.GetResult(resultId)? If we allow clients to fetch all results, then allowing them to ask for a specific result feels completely reasonable. I think this makes a valid argument for exposing a unique identifier.
Related reading.
name
Required: True
Type: VARCHAR(255)
Exposed publicly: True
Must be the name of the rule (at least within the context of ComplianceAsCode/content rules). But it could be generalized to a unique name for any type of evaluation.
control
Required: True
Type: VARCHAR(50) or a foreign key to a table dedicated to controls (UUID)
Exposed publicly: True
This could be something as simple as NIST-AC02 or cis@ocp4: 4.2.5. In free form, this isn't very useful and it doesn't contain additional metadata about the control. Something else would need to fill in the gaps from a separate source of truth.
A more robust solution would to be to have a UUID that references the control, stored in a standard format (e.g., OSCAL) that would fill in the gaps.
The main point here, I think, would be to only include the control for the result. The result wouldn't contain any more information about the control than a reference to the control(s) it's related to.
outcome
Required: True
Type: VARCHAR(50)
Exposed publicly: True
This seems like a good use case for a boolean, but the outcome could be one of several types (passed, failed, not-applicable, manual, etc.).
Is VARCHAR(50) appropriate for this? Is there a case where this type won't work for outcomes?
description
Required: False
Type: VARCHAR(255)
Exposed publicly: True
This could be added on to the schema after, since it's not a required piece of information. The usage of this would really be up to the data ingestion process and what information it may need to set on the result. I'd be OK punting on this column until we have a clear use case for including it.

Subject schema
A subject is a generalized term for anything that can be scanned for compliance (a server, a piece of networking gear, a Kubernetes cluster, a private cloud, etc.)
id
Required: True
Type: UUID
Exposed publicly: True
name
Required: True
Type: VARCHAR(255)
Exposed publicly: True
A unique name for the subject. This could be a server host name or cluster name. We need some more thought here to see what makes this unique.
type
Required: True
Type: VARCHAR(50)
Exposed publicly: True
Denotes the type of subject this is. This could be almost anything, but initially it would be a server or cluster.
metadata_id
Required: False
Type: UUID
Exposed publicly: False
This would be a foreign key to metadata.id table/column, used to link additional, common attributes to subjects (like created at and updated at times).

@rhmdnd should we make the subject be hierarchical? e.g. add a parent_subject column that may point to another subject? that would allow us to build sub-subjects. e.g.
Cluster:
{id: 1, name: "cluster-a", type: "cluster", parent_id: null, ...}
Control Plane Machine Pool:
{id: 2, name: "control-plane", type: "machine-pool", parent_id: 1, ...}
Worker Machine Pool:
{id: 3, name: "workers", type: "machine-pool", parent_id: 1, ...}
Node 1:
{id: 4, name: "cluster-a-node-1", type: "node", parent_id: 2, ...}
...
this way, it would be a lot easier to specify where stuff belongs to.

@rhmdnd I'm not so sure about the control being a varchar(50) ... as more compliance standards get evaluated, the number of references will grow. It's a matter of seeing how big the reference list already is in CaC. I also am not very keen on keeping a table just to reference different compliance standards... seems to me like a lot of duplication. So... I'd like to see if other folks have other ideas. @jhrozek anything on your side?

Thanks for taking a look!

@rhmdnd I'm not so sure about the control being a varchar(50) ... as more compliance standards get evaluated, the number of references will grow. It's a matter of seeing how big the reference list already is in CaC.

I'm not opposed to making this larger, just as long as it accommodates the CaC rule with the most usage across profiles, right?

I also am not very keen on keeping a table just to reference different compliance standards... seems to me like a lot of duplication. So... I'd like to see if other folks have other ideas. @jhrozek anything on your side?

Are you referring to the profile table?

The hierarchical subject idea is interesting.

this way, it would be a lot easier to specify where stuff belongs to. An alternative is to make the parent_id be part of the metadata.

Can you elaborate on this a little more? For example, are you suggesting that we would have a subject for the cluster, which would have child subjects for nodes?

At what depth do you anticipate the hierarchy? It's easy to think of a simple case where a cluster is the parent and each node is a child, resulting in tree with the height of 1 (assuming 0 is the first layer).
Do you anticipate an immediate need for a layer in between the nodes and the cluster (e.g., organized by service)?

Do you anticipate an immediate need for a layer in between the nodes and the cluster (e.g., organized by service)?

Answered in the comment above showing the example of a hierarchical relationship between the cluster, node pool, and nodes.

@JAORMX proposed a general idea to add a FK for the subject table so that we can build hierarchies. Let me know what you think.
Also, one thing that came up in discussion is how strict we should valid the results. It's expected that if someone were to try to send results with a field longer than what the database could accommodate, the service should fail to ingest that result, right?

We could also move the description to the metadata table, since it could be used for other things outside results.
Another thing to think about is that currently results from the Compliance Operator have descriptions and instructions. We might consider adding another column to the results schema to account for this.

@JAORMX proposed a general idea to add a FK for the subject table so that we can build hierarchies. Let me know what you think.
Also, one thing that came up in discussion is how strict we should valid the results. It's expected that if someone were to try to send results with a field longer than what the database could accommodate, the service should fail to ingest that result, right?

It is expected and we should fail, but we should accommodate for storing these results somehow. I suggest failing for now but making these failures visible in such a way we can detect this (metrics perhaps?). We certainly don't want to loose information and want to evaluate cases as they arise.

We could also move the description to the metadata table, since it could be used for other things outside results.
Another thing to think about is that currently results from the Compliance Operator have descriptions and instructions. We might consider adding another column to the results schema to account for this.

I agree. It's very relevant for auditors and operators to know what to do with failing results, how to evaluate the issue and how to address/fix the issue.

It is expected and we should fail, but we should accommodate for storing these results somehow. I suggest failing for now but making these failures visible in such a way we can detect this (metrics perhaps?). We certainly don't want to loose information and want to evaluate cases as they arise.

To clarify, this would require some alerting machinery inside the service, correct?

I agree. It's very relevant for auditors and operators to know what to do with failing results, how to evaluate the issue and how to address/fix the issue.

I addressed this in the updated schemas.

It is expected and we should fail, but we should accommodate for storing these results somehow. I suggest failing for now but making these failures visible in such a way we can detect this (metrics perhaps?). We certainly don't want to loose information and want to evaluate cases as they arise.

To clarify, this would require some alerting machinery inside the service, correct?

that's right, we'd need to start supporting metrics (probably as a prometheus-friendly endpoint)

that's right, we'd need to start supporting metrics (probably as a prometheus-friendly endpoint)

I pulled this into its own issue https://github.com/rhmdnd/compserv/issues/107

Since I just proposed patches that include foreign keys. How do we want to handle foreign key relationships when an entity is deleted?
For example, say we have a foreign key between results.subject_id and subject.id. If or when the subject is deleted, should all results associated to that subject be deleted, too?
Since we're dealing with compliance data, I'm hesitant to delete data since the service may track data for long periods of time (longer than the lifecycle of a cluster).
Thoughts on how we should handle this?

I agree with your concern. We should not delete data, but have the means to display subsets instead. I recall we had something similar in barbican, where we'd "disable" an entry

I agree with your concern. We should not delete data, but have the means to display subsets instead. I recall we had something similar in barbican, where we'd "disable" an entry

Some projects treated "disable" like a soft delete. Should we support soft-deletes?

I added a separate patch to call out how we should handle common things like UUIDs and datetimes. It's only a documentation patch but it makes doing some work here more consistent.
https://github.com/rhmdnd/compserv/pull/111

Opened a separate issue to document the need for a table dedicated to assessments or scans https://github.com/rhmdnd/compserv/issues/118.
I lean towards the term assessment since it's more inline with terminology used by NIST.
https://pages.nist.gov/OSCAL/concepts/layer/assessment/

Assessment schema
An assessment is a collection of results. From the compliance-operator perspective, this is a scan. I'm leaning towards using the term assessment instead to be consistent with NIST terminology:
https://pages.nist.gov/OSCAL/concepts/layer/assessment/#assessment-results-model
id
Required: True
Type: UUID
Exposed publicly: True
name
Required: True
Type: VARCHAR(255)
Exposed publicly: True
A name for the assessment, but it doesn't have to be unique (fedramp-high-scan).
metadata_id
Required: False
Type: UUID
Exposed publicly: False
This would be a foreign key to metadata.id table/column, used to link additional, common attributes to assessments (like created at and updated at times).

@JAORMX proposed a general idea to add a FK for the subject table so that we can build hierarchies. Let me know what you think.
Also, one thing that came up in discussion is how strict we should valid the results. It's expected that if someone were to try to send results with a field longer than what the database could accommodate, the service should fail to ingest that result, right?

It is expected and we should fail, but we should accommodate for storing these results somehow. I suggest failing for now but making these failures visible in such a way we can detect this (metrics perhaps?). We certainly don't want to loose information and want to evaluate cases as they arise.

Isn't it better to persist the result with the attribute truncated to the length we allow AND issue a warning?

Assessments schema
An assessment is a collection of results. From the compliance-operator perspective, this is a scan. I'm leaning towards using the term assessment instead to be consistent with NIST terminology:

This is useful, but how does an assessment link to the result?

Assessments schema
An assessment is a collection of results. From the compliance-operator perspective, this is a scan. I'm leaning towards using the term assessment instead to be consistent with NIST terminology:

This is useful, but how does an assessment link to the result?

Nevermind, I just looked at PR #109 and noticed that the result has an assessment ID. This looks good!
I wonder if you think that the results should link to the subjects directly or through an assessment? (Genuine q., I don't have a lot of experience designing SQL tables). Naively it feels like you would always "assess a subject".

instruction
Required: False Type: TEXT Exposed publicly: True
Useful to storing additional instructions for a particular rule. Many of the existing rules in the ComplinaceAsCode content store instructions that are used to communicate what actions should be done based on a finding. Aggregating this information is important for consumers of this service.
An alternative name for this could be rationale.

So is this:

why are we doing the check
what are we checking
how to remediate the finding

Instructions sounds like "what are we checking", but the alternative name you proposed (rationale) sounds like "why are we checking" and the text ("what actions should be done based on a finding") sounds like the last option.
Or were you thinking about combining them all?

If you want to include all references, then it's not going to be enough, for very popular rules (like package_audit_installed) the list of references is ~1500 chars. I guess 2048 or 4096 to be on the safe side would be enough.

Ok - that's a good point.
If we increase the length, are we assuming that we would be persisting a list? If so, we should consider separating those values out into a different table so we don't violate first normal form.

Isn't it better to persist the result with the attribute truncated to the length we allow AND issue a warning?

Ok - maybe truncation is the devil we have to deal with then in this case if we need to persist something.

This is useful, but how does an assessment link to the result?

The way I'm thinking about this is that an assessment as a one to many relationship with results. Each result must link to an assessment through a foreign key relationship. The assessment would store information like scan time and ID, which are applicable to all results from the report.

On Wed, Aug 03, 2022 at 06:38:44AM -0700, Lance Bragstad wrote:

If you want to include all references, then it's not going to be enough, for very popular rules (like package_audit_installed) the list of references is ~1500 chars. I guess 2048 or 4096 to be on the safe side would be enough.

Ok - that's a good point.
If we increase the length, are we assuming that we would be persisting a list? If so, we should consider separating those values out into a different table so we don't violate first normal form.

Hmm, I think it is concievable that someone would want to query results
that pertain to a single control. I think I remember ACM had mentioned
this use-case. So splitting the references to a separate table would
probably at least give us enough room to design something better than a
blob.

Nevermind, I just looked at PR #109 and noticed that the result has an assessment ID. This looks good!
I wonder if you think that the results should link to the subjects directly or through an assessment? (Genuine q., I don't have a lot of experience designing SQL tables). Naively it feels like you would always "assess a subject".

Good question. I think we could do either.
Linking the assessment to the subject reduces duplication across all results rows related to that assessment. Linking the result to the subject gives us the flexibility to have subjects differ, or link to different components of the system.

On Wed, Aug 03, 2022 at 06:42:34AM -0700, Lance Bragstad wrote:

Isn't it better to persist the result with the attribute truncated to the length we allow AND issue a warning?

Ok - maybe truncation is the devil we have to deal with then in this case if we need to persist something.

Sorry, maybe I was unclear, I was trying to say that from a user
perspective, it seems preferable to know that my cluster is out of
compliance even if the instructions would be incomplete than having no
record of the issue peristing.

So is this:
* why are we doing the check

* what are we checking

* how to remediate the finding

Instructions sounds like "what are we checking", but the alternative name you proposed (rationale) sounds like "why are we checking" and the text ("what actions should be done based on a finding") sounds like the last option.
Or were you thinking about combining them all?

Good point. Initially, I only wanted to create schemas for what we would use, but it sounds like it's useful (based on your definitions) to have instructions, rationale, and description since they all fill a unique purpose. Additionally, putting rationale in the instructions will probably complicate things for callers. For example, if someone asked for the result instructions, they would have to parse the different portions of it into rationale and instructions (which arguably violates first normal form).
I can propose a separate, null-able column for rationale. It's available for us if we need it and the compliance service can use it.

On Wed, Aug 03, 2022 at 06:53:31AM -0700, Lance Bragstad wrote:

So is this:
* why are we doing the check

* what are we checking

* how to remediate the finding

Instructions sounds like "what are we checking", but the alternative name you proposed (rationale) sounds like "why are we checking" and the text ("what actions should be done based on a finding") sounds like the last option.
Or were you thinking about combining them all?

Good point. Initially, I only wanted to create schemas for what we would use, but it sounds like it's useful (based on your definitions) to have instructions, rationale, and description since they all fill a unique purpose. Additionally, putting rationale in the instructions will probably complicate things for callers. For example, if someone asked for the result instructions, they would have to parse the different portions of it into rationale and instructions (which arguably violates first normal form).
I can propose a separate, null-able column for rationale. It's available for us if we need it and the compliance service can use it.

This sounds good!

Control schema
A control would be a reference to a control as written by a standards body (NIST or CIS).
id
Required: True
Type: UUID
Exposed publicly: True
name
Required: True
Type: VARCHAR(255)
Exposed publicly: True
A name for the control.
metadata_id
Required: False
Type: UUID
Exposed publicly: False
This would be a foreign key to metadata.id table/column, used to link additional, common attributes (like description, created at, and updated at times).
profile_id
Required: True
Type: UUID
Exposed publicly: True
This would be a foreign key to profile.id table/column, used to reference the profile or baseline for the control.
severity
Required: True
Type: VARCHAR(50)
Exposed publicly: True
This is an optional parameter that denotes the severity of a particular control (e.g., high, medium, low).

Profile schema
A profile references a benchmark containing controls (CIS benchmarks for OpenShift, FedRAMP High).
id
Required: True
Type: UUID
Exposed publicly: True
name
Required: True
Type: VARCHAR(255)
Exposed publicly: True
A name for the control.
metadata_id
Required: False
Type: UUID
Exposed publicly: False
This would be a foreign key to metadata.id table/column, used to link additional, common attributes (like description, created at, and updated at times).

Not sure if I should ask here or in PR#128, but can you give me an example of a row in the controls table? I'm mostly wondering about what's in the name, is it the plain english "title" of the control (e.g. auditing must be enabled) or the control's ID (std.1.2.3)?

Actually - if we just reference the control group, the name is redundant (assuming we can derive it from the actual benchmark or profile).
But, an example with the name would look like:
control.id = "158d705f-dcc6-451a-afff-68497253a27b"
control.name = "Ensure that the --authorization-mode argument is not set to AlwaysAllow"
control.group = "4.2.2"
control.metadata_id = "994f01b1-db54-4066-b0a3-bffe2ea7afed"
control.profile_id = "c98d799e-1b29-4897-b26a-58dcfa33f1fb"

Alternatively, if we completely omit the name, we would have:
control.id = "158d705f-dcc6-451a-afff-68497253a27b"
control.group = "4.2.2"
control.metadata_id = "994f01b1-db54-4066-b0a3-bffe2ea7afed"
control.profile_id = "c98d799e-1b29-4897-b26a-58dcfa33f1fb"

Then, we would need something to derive the name from the benchmark, which we will probably need anyway to derive other pieces of information.

Catalog schema
A catalog is a pristine copy of a benchmark, containing controls, as written by the benchmark author (NIST, CIS).
id
Required: True
Type: UUID
Exposed publicly: True
name
Required: True
Type: VARCHAR(255)
Exposed publicly: True
A name for the catalog (e.g., NIST SP 800-53).
metadata_id
Required: False
Type: UUID
Exposed publicly: False
This would be a foreign key to metadata.id table/column, used to link additional, common attributes (like description, created at, and updated at times).
content
Required: True
Type: TEXT
Exposed publicly: True
This would be a free-from field for storing catalogs, as provided by a standards body. An example of the NIST SP 800-53 catalog would be:
https://github.com/usnistgov/oscal-content/tree/main/nist.gov/SP800-53/rev5


Build basic modelling tool

As an RPA developer, I'd like to able to visit a modeling site where I can create BPMN models, in order to document my customer's process.

:thought_balloon: Hints & Thoughts
Check out https://bpmn.io/

Simple install, should only be able to use it.
Do #2 first or in parallel

:question: Additional Questions to Answer / Consider

:white_check_mark: Conditions of satisfaction

[ ] I can move BPMN activities, events, gateways into the drawing area
[ ] I can connect the BPMN artifacts
[ ] I can move the artifacts in the drawing area

@lukashueller @WolfgangDaniel Maybe this could help you: https://github.com/bpmn-io/react-bpmn

Inspired by https://github.com/Varooneh/reactbpmn

Time spent: 3SP


[Feature Request] Add @babel/plugin-transform-runtime

Are you submitting a bug report or a feature request?
feature request
Other information
https://babeljs.io/docs/en/next/babel-plugin-transform-runtime.html
Remove trash from builds for decrease package size, e.g. es/createField.js:
Before:

After:

Published fix in v8.1.0.


No mockito.version property set in POM.

I couldn't find any property names mockito.version set. It seems to be a bug?

Sorry, it seems to inherited from its parent. Close this.


Missing source code for v0.4.0

There is a version v0.4.0-rc.0 on hex.pm, but not here.

@linjunpop Yes, I'm rewriting the API and the manual, will release by the end of week.

@linjunpop I'm writing a thin API wrapper to make it support multiple clients and most of the APIs, mostly inspired by https://github.com/edgurgel/tentacat. I wonder whether to publish a new break version or create a new project, what do you think?
If there is anyone using this project in production, please leave a comment. If there is few, I will choose to publish a new break version.
In the new API wrapper, I plan to support most common APIs, leaving a request function to help users easily implement uncommon APIs if they need.

@goofansu A wrapper for all WeChat's APIs? That sounds cool.

@linjunpop Thanks.
It includes the most commonly used APIs, including two parts:

Wechat public account APIs (微信公众号API)
Endpoint: https://api.weixin.qq.com/cgi-bin/
Reference: https://github.com/Eric-Guo/wechat

Wechat mini-program APIs (微信小程序API)
Endpoints:

https://api.weixin.qq.com/wxa/
https://api.weixin.qq.com/sns/

But it doesn't include APIs like https://github.com/goofansu/wechat_elixir/blob/master/lib/wechat/datacube.ex, as there are few users using it directly.
It will only be an API thin wrapper, plugs will be moved into another repo.

plugs will be moved into another repo.

That's what I'm hesitating about whether there should be two packages for wechat_pay or a unified one. 😸

@linjunpop wechat_pay seems OK as it looks not too heavy.
wechat_elixir needs to be splitted into three parts as they are no direct relations.

APIs
Plugs
Message responder DSL (I see this has been implemented, for example, https://github.com/h1u2i3/ex_wechat)

There are some benefits I can see:

Wraps only the APIs users want to use:

defmodule Example.WechatAPI do
  def client do
    appid = System.get_env("WECHAT_APPID")
    secret = System.get_env("WECHAT_SECRET")
    Wechat.Client.new(appid, secret)
  end

  def users do
    Wechat.User.users(client())
  end
end

Try multiple accounts in iex more easily

iex> client = Wechat.Client.new("appid", "secret")
...> Wechat.User.users(client)

iex> client = Wechat.Client.new("appid2", "secret2")
...> Wechat.User.users(client)

I will merge https://github.com/goofansu/orca into this repo.


Upgrade Storage Blob sdk from T1 to T2

1.Upgrade Storage Blob sdk from T1 to T2
2.Upgrade files:

CleanTrigger1/clean.py
CleanTrigger2/clean.py
README.md
Reconcile/clean.py
Reconcile/fetch_blob.py
azure-deploy-linux-app-plan.json
blob_to_smart_contract/clean.py
blob_to_smart_contract/fetch_blob.py
requirements.txt

@lmazuel, @johanste, @kyle-patterson, @kurtzeborn, @annatisch for notification.

@lmazuel, @johanste, @kyle-patterson, @kurtzeborn, @annatisch Could you help review this PR? Thanks a lot.

@lmazuel, @johanste, @kyle-patterson, @kurtzeborn, @annatisch Could you help review this PR? Thanks a lot.

@lmazuel, @johanste, @kyle-patterson, @kurtzeborn, @annatisch Could you help review this PR? Thanks a lot.

@johanste We have updated by your comments, please re-review this PR. Thanks a lot.

@johanste We have updated by your comments, please re-review this PR. Thanks a lot.

@annatisch We have updated by your comments, please re-review this PR.

@annatisch We have updated by your comments, please re-review this PR.

@annatisch We have updated by your comments, please re-review this PR.

@annatisch We have updated by your comments, please re-review this PR.


Kong is unable to detect ingress resource

NOTE: GitHub issues are reserved for bug reports only.
For anything else, please join the conversation
in Kong Nation https://discuss.konghq.com/c/kubernetes.

Summary
The Kong ingress controller does not detect the ingress resource for a service.
Kong Ingress controller version
0.14.0
Kubernetes version
Kubernetes v1.9.5
Environment

Cloud provider or hardware configuration: Local setup using kubespray
OS (e.g. from /etc/os-release): CentOS Linux 7
Kernel (e.g. uname -a): Linux node1 3.10.0-862.9.1.el7.x86_64 #1 SMP Mon Jul 16 16:29:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Install tools:  helm install --name kongssl stable/kong
Others:

What happened
I have a grafana service running in my k8s cluster and am able to access it without any issues using NodePort.
I installed kong using helm as follows

helm install --name kongssl stable/kong

And added the following ingress

----
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: grafana-ingress
  namespace: ns
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  tls:
  - hosts:
    - foo.bar
    secretName: foo-cert
  rules:
  - host: foo.bar
    http:
      paths:
      - path: /
        backend:
          serviceName: grafana
          servicePort: 3000

The ingress gets added without issues however Kong does not pick up the ingress and add it to the list of it's services

curl -k https://ADMIN_IP:ADMIN_PORT/apis
{"total":0,"data":[]}

curl -k https://ADMIN_IP:ADMIN_PORT/services
{"next":null,"data":[]}

Expected behvaior
Kong should have picked up the new ingress and added the services for proxying the traffic to it
Steps To Reproduce
Same as what happened

Hello @vishpat,
stable/kong Helm chart deploys Kong as an application on k8s and not as an Ingress controller, so it won't pick up ingress resources.
Please follow our getting started
 guide to understand how to setup Kong as an Ingress controller.
There is an kong-ingress chart in works and will be released soon!

Many thanks @hbagdi


generate images with specific size

I am using the save to PNG feature to make an overview of a 60 000 × 60 000 big world (the world is also getting bigger very fast). I started it yesterday but my PC crashed in the middle of the night and it was only at 30 % of generating the image.
This could have multiple reasons:

Too many regions to load
Too large image file
My PC sucks

At least for the second potential cause I have a fix idea that would also be useful in general:

Make it able to decide the size of the output image (this is similar to Issue#97: Increase zoom. So if this ever gets implemented it could be used for both)
Let the user decide which areas should be in the output image (basically instead of putting every single chunk into the output image you can tell it two coordinates and all chunks between them are made to an image)

Just a note, an image 60000 * 60000 blocks would need at least 9 GiB of memory (either RAM, or swap space), that could be the reason it crashed. For such a large map, I think a better solution would be to have a batch output mode which would output images, say 1024px * 1024px, that could be displayed next to each other (just like google maps).

It is possible to define the output size! Just read the [documentation about exportsize].(https://github.com/mrkite/minutor/wiki/Command-Line)
And with an exported image you can do whatever zoom or postprocessing you want.
Regarding your really large world and export problems:

In principle it should work!
Minutor uses spezial code to export Chunk by Chunk to write PNGs with arbitrary size.
The PNG is written row wise and only one Chunk at the time has to be in memory
Maybe you hit PNGs maximum size?
I suggest to export "smaller" parts. E.g. 16x16 regions

It could be possible he's running out of disk space, as 30% of a 60kx60k image would be quite big.


Wrong Heading Date when Using as Time Picker Only

There seems to be no way to set the heading date when using the widget as only a time picker:
Screenshot:
http://screencast.com/t/jJsGChLPAUyp
I tried sorting through the jquery to add hooks to remove this header - but couldn't figure out that.
Is there a setting I can apply?

FWIW this is affecting me, too.

I have a workaround (that comes from a comment made against #416): set startView and maxView to 1. You still get a header with the date on, but the date is today's date and the UI is actually better, I think?
Here is my javascript:
$('.tmpicker').datetimepicker({
    startView:   1,
    maxView:     1,
    minuteStep:  15,
    format:     "hh:ii",
    forceParse: true,
    autoclose:  true
  });

~

Thank you. But I couldn't get it to work for me. Still says December 1889

I have the same problem. Date changes to 1899-12-31 after changing the time...

Yep, you're both correct.  Works when entering a new time for me, but not when editing an existing one.  Ah well, better than nothing I suppose.

@newscloud / @ChrisLamprecht / @andy-twosticks - See the reference PR above ☝. Feel free to check this out locally and confirm if this works, and we'll see about having it reviewed by @AuspeXeu.

FYI you can also set the title of the datetimepicker by providing title: 'Some Title' as part of options to the datetimepicker.

@davidpatters0n thanks for the replies. What's the right way to apply the patch if I'm using composer? Am I only able to test it locally before composer overwrites it?

@newscloud - The does seem correct the version that yii2-date-time-picker-widget is using is 2.3.1, and in order to use my patch you'd need to update the composer file to use my forked repo and the associated branch. So correct me if I am wrong but you may be able to do:
"require": {
        "yiisoft/yii2": "~2.0",
        "yiisoft/yii2-bootstrap": "~2.0.0",
        "bower-asset/bootstrap-datetimepicker": "bug/timepicker-parsing"
    },

Alternatively you can check out my forked repo and test it locally.

@davidpatters0n thanks so much for trying to help me with my lack of composer knowledge.
I got this when I tried your suggestion:
[UnexpectedValueException]
Could not parse version constraint bug/timepicker-parsing: Invalid version string "bug/timepicker-parsing"
Any idea?

@newscloud apologies that did not work. I do not have a particular why. But the PR for this issue has been merged and is available in 2.4.1. So if you just update the version to 2.4.1 you should be good 👍

Thanks again. Not working. Is there anyway to install 2.4.1 even though that plugin req 2.3? I've asked them to update. Some days composer is a pain :)


fixed issue #307

Link to issue description

When can this get merged in master?

Can it be merged? As an extension developer, I have the same issue.


Adds random password generation capability to demo config install scripts

Description
As a follow to changes made in demo config script, this PR adds a script to generate a random 16 character password if the flag to generate random password -g is passed. It will only generate and set admin password when following conditions are true:

initialAdminPassword variable is not set as env variable
initialAdminPassword.txt doesn't exist
-g flag is passed to the script

If -g is not passed and other two conditions are false, the install demo config script would fail.
Related to: https://github.com/opensearch-project/security/issues/1576
Testing
Manual testing:

Windows install logs:
**************************************************************************
** This tool will be deprecated in the next major release of OpenSearch **
** https://github.com/opensearch-project/security/issues/1755           **
**************************************************************************

OpenSearch Security Demo Installer
 ** Warning: Do not use on production or public reachable systems **

Basedir: C:\Users\Administrator\Documents\localDistro\opensearch-3.0.0-SNAPSHOT\
OpenSearch install type: .zip on Windows_NT
OpenSearch config dir: C:\Users\Administrator\Documents\localDistro\opensearch-3.0.0-SNAPSHOT\config\
OpenSearch config file: C:\Users\Administrator\Documents\localDistro\opensearch-3.0.0-SNAPSHOT\config\opensearch.yml
OpenSearch bin dir: C:\Users\Administrator\Documents\localDistro\opensearch-3.0.0-SNAPSHOT\bin\
OpenSearch plugins dir: C:\Users\Administrator\Documents\localDistro\opensearch-3.0.0-SNAPSHOT\plugins\
OpenSearch lib dir: C:\Users\Administrator\Documents\localDistro\opensearch-3.0.0-SNAPSHOT\lib\
Detected OpenSearch Version: 3.0.0
Detected OpenSearch Security Version: <IP_ADDRESS>
"what is in the config directory"
 Volume in drive C has no label.
 Volume Serial Number is E61A-B1DE

 Directory of C:\Users\Administrator\Documents\localDistro\opensearch-3.0.0-SNAPSHOT\config

10/05/2023  06:48 PM    <DIR>          .
10/05/2023  06:33 PM    <DIR>          ..
10/05/2023  06:48 PM             1,707 esnode-key.pem
10/05/2023  06:48 PM             1,532 esnode.pem
08/31/2023  03:27 PM             2,747 jvm.options
08/31/2023  03:27 PM    <DIR>          jvm.options.d
10/05/2023  06:48 PM             1,707 kirk-key.pem
10/05/2023  06:48 PM             1,658 kirk.pem
08/31/2023  03:27 PM            14,808 log4j2.properties
08/31/2023  05:37 PM    <DIR>          opensearch-security
08/31/2023  05:33 PM               196 opensearch.keystore
10/05/2023  06:48 PM             6,416 opensearch.yml
10/05/2023  06:48 PM             1,719 root-ca.pem
               9 File(s)         32,490 bytes
               4 Dir(s)  169,389,215,744 bytes free
"what is in the password file"
"   ***************************************************"
"   ***   ADMIN PASSWORD SET TO: BoXO6rWKjVJUqpUC  ***"
"   ***************************************************"
### Success
### Execute this script now on all your nodes and then start all nodes
### OpenSearch Security will be automatically initialized.
### If you like to change the runtime configuration
### change the files in ../../../config/opensearch-security and execute:

C:\Users\Administrator\Documents\localDistro\opensearch-3.0.0-SNAPSHOT\plugins\opensearch-security\tools\securityadmin.bat -cd C:\Users\Administrator\Documents\localDistro\opensearch-3.0.0-SNAPSHOT\config\opensearch-security -icl -key C:\Users\Administrator\Documents\localDistro\opensearch-3.0.0-SNAPSHOT\config\kirk-key.pem -cert C:\Users\Administrator\Documents\localDistro\opensearch-3.0.0-SNAPSHOT\config\kirk.pem -cacert C:\Users\Administrator\Documents\localDistro\opensearch-3.0.0-SNAPSHOT\config\root-ca.pem -nhnv
### or run ./securityadmin_demo.bat
### To use the Security Plugin ConfigurationGUI
### [Ignore the SSL certificate warning because we installed self-signed demo certificates]

Windows curl ouput after password change:
C:\Users\Administrator>curl -XGET https://localhost:9200/ -k -u admin:admin
Unauthorized
C:\Users\Administrator>curl -XGET https://localhost:9200/ -k -u admin:BoXO6rWKjVJUqpUC
{
  "name" : "EC2AMAZ-SPV3HQH",
  "cluster_name" : "opensearch",
  "cluster_uuid" : "QhnbMcigR6KKTj-XZtr21w",
  "version" : {
    "distribution" : "opensearch",
    "number" : "3.0.0-SNAPSHOT",
    "build_type" : "zip",
    "build_hash" : "bb38ed4836496ac70258c2472668325a012ea3ed",
    "build_date" : "2023-08-31T15:25:32.930646600Z",
    "build_snapshot" : true,
    "lucene_version" : "9.8.0",
    "minimum_wire_compatibility_version" : "2.10.0",
    "minimum_index_compatibility_version" : "2.0.0"
  },
  "tagline" : "The OpenSearch Project: https://opensearch.org/"
}

Mac install_demo_config log:
➜  opensearch-security ./tools/install_demo_configuration.sh -y -i -s -g
./tools/install_demo_configuration.sh: line 3: ./generate-password.sh: No such file or directory
**************************************************************************
** This tool will be deprecated in the next major release of OpenSearch **
** https://github.com/opensearch-project/security/issues/1755           **
**************************************************************************
OpenSearch Security Demo Installer
 ** Warning: Do not use on production or public reachable systems **
Basedir: /Users/dchanp/Documents/amazon/testing-playground/test/binaries/opensearch-3.0.0-SNAPSHOT
OpenSearch install type: .tar.gz on
OpenSearch config dir: /Users/dchanp/Documents/amazon/testing-playground/test/binaries/opensearch-3.0.0-SNAPSHOT/config
OpenSearch config file: /Users/dchanp/Documents/amazon/testing-playground/test/binaries/opensearch-3.0.0-SNAPSHOT/config/opensearch.yml
OpenSearch bin dir: /Users/dchanp/Documents/amazon/testing-playground/test/binaries/opensearch-3.0.0-SNAPSHOT/bin
OpenSearch plugins dir: /Users/dchanp/Documents/amazon/testing-playground/test/binaries/opensearch-3.0.0-SNAPSHOT/plugins
OpenSearch lib dir: /Users/dchanp/Documents/amazon/testing-playground/test/binaries/opensearch-3.0.0-SNAPSHOT/lib
Detected OpenSearch Version: x-content-3.0.0-SNAPSHOT
Detected OpenSearch Security Version: <IP_ADDRESS>-SNAPSHOT

   ***************************************************
   ***   ADMIN PASSWORD SET TO: Mari4UfLQajoASVo    ***
   ***************************************************
**************************************************************************
** This tool will be deprecated in the next major release of OpenSearch **
** https://github.com/opensearch-project/security/issues/1755           **
**************************************************************************
### Success
### Execute this script now on all your nodes and then start all nodes
### OpenSearch Security will be automatically initialized.
### If you like to change the runtime configuration
### change the files in ../../../config/opensearch-security and execute:
"/Users/dchanp/Documents/amazon/testing-playground/test/binaries/opensearch-3.0.0-SNAPSHOT/plugins/opensearch-security/tools/securityadmin.sh" -cd "/Users/dchanp/Documents/amazon/testing-playground/test/binaries/opensearch-3.0.0-SNAPSHOT/config/opensearch-security" -icl -key "/Users/dchanp/Documents/amazon/testing-playground/test/binaries/opensearch-3.0.0-SNAPSHOT/config/kirk-key.pem" -cert "/Users/dchanp/Documents/amazon/testing-playground/test/binaries/opensearch-3.0.0-SNAPSHOT/config/kirk.pem" -cacert "/Users/dchanp/Documents/amazon/testing-playground/test/binaries/opensearch-3.0.0-SNAPSHOT/config/root-ca.pem" -nhnv
### or run ./securityadmin_demo.sh
### To use the Security Plugin ConfigurationGUI
### To access your secured cluster open https://<hostname>:<HTTP port> and log in with admin/admin.
### (Ignore the SSL certificate warning because we installed self-signed demo certificates)

Mac curl output after password change:
➜  ~ curl -XGET 'https://[::1]:9200/' -k -u admin:admin

Unauthorized%
➜  ~ curl -XGET 'https://[::1]:9200/' -k -u admin:Mari4UfLQajoASVo

{
  "name" : "3c06300b34da.ant.amazon.com",
  "cluster_name" : "opensearch",
  "cluster_uuid" : "Zw9pjloBTiiqzfusz2nyLQ",
  "version" : {
    "distribution" : "opensearch",
    "number" : "3.0.0-SNAPSHOT",
    "build_type" : "tar",
    "build_hash" : "d656e3db592f29466d35452867caa241f5429485",
    "build_date" : "2023-09-28T15:29:27.870375Z",
    "build_snapshot" : true,
    "lucene_version" : "9.8.0",
    "minimum_wire_compatibility_version" : "2.11.0",
    "minimum_index_compatibility_version" : "2.0.0"
  },
  "tagline" : "The OpenSearch Project: https://opensearch.org/"
}

Check List

[x] New functionality includes testing
~- [ ] New functionality has been documented~
[x] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Closing this for now. Will re-open once demo configuration changes are finalized.


Find books by publisher

This reverts commit 4ff28933
Summary of change
added to schema and find by publisher
Related issue
Closes #57
sorry, still learning how to use github so i deleted last branch, i thing it might be ok now.

Thanks for this.
I noticed you took out the pull request checklist. In your next pull request, could you please keep it in?

Created a branch that has a descriptive name (what your branch is for in a few words and includes the issue number at the end, e.g. test-reading-goal-123

This is one of the checkboxes. Note that you'll need to include the issue number in your branch name. This should be find-books-by-publisher-57. You can rename your branch with git branch -m <new-branch-name>. You will have to create another pull request, however.
If you need a hand with this or anything else in the process, feel free to ask by creating a new GitHub Q&A discussion or on our Slack #help channel.
Pull request checklist
Please keep this checklist in & ensure you have done the following:

[ ] Read, understood and adhered to our contributing document.

[ ] Ensure that you were first assigned to a relevant issue before creating this pull request
[ ] Ensure code changes pass all tests

[ ] Read, understood and adhered to our style guide. A lot of our code reviews are spent on ensuring compliance with our style guide, so it would save a lot of time if this was adhered to from the outset.

[ ] Filled in the summary, context (if applicable) and related issue section. Replace the square brackets and its placeholder content with your contents. For an example, see any merged in pull request

[ ] Created a branch that has a descriptive name (what your branch is for in a few words and includes the issue number at the end, e.g. test-reading-goal-123

[ ] Set this pull request to 'draft' if you are still working on it

[ ] Resolved any merge conflicts

For any of the optional checkboxes (e.g. the screenshots one), still check it if it does not apply.
If in doubt, get in touch with us via our Slack workspace or by creating a new Q&A discussion on GitHub


build(deps-dev): bump undici from 5.23.0 to 5.28.4

Bumps undici from 5.23.0 to 5.28.4.

Release notes
Commits

updated-dependencies:

dependency-name: undici dependency-type: direct:development ...

@dependabot recreate


[WIP] Add typed OpenAI clients

Current focus: get mypy to pass
Working example code:
import asyncio
from typing import Callable, Coroutine
from openai import AsyncOpenAI
from pydantic import BaseModel
from instructor import patch

class UserDetail(BaseModel):
    name: str
    age: int

async def main():
    client = patch(AsyncOpenAI())
    user = await client.chat.completions.create(
        model="gpt-3.5-turbo",
        response_model=UserDetail,
        messages=[
            {"role": "user", "content": "Extract Jason is 25 years old"},
        ],
    )

    assert isinstance(user, UserDetail)
    assert user.name == "Jason"
    assert user.age == 25

    print(user)

asyncio.run(main())

make sure the lib is now 1.0 :D

Also this pull request is in a different github repo, I'm so confused

Hahaha it's in my fork. This is from pre-contributor era. I'll move it in


updated BTE default token contract address

from bitcoineum (BTE) contract address to BitSerial (BTE) contract address. BitSerial has twice as many transactions.

If there is confusion between token symbols, a clarifier should be added to the token symbol, not replaced. e.g. BTE (Bitcoineum) and BTE (BitSerial)
Original PR: https://github.com/kvhnuke/etherwallet/pull/1036
Will need support email for new BitSerial Token regardless (https://myetherwallet.github.io/knowledge-base/tokens/token-creators-add-your-token-to-myetherwallet.html)


How did you convert an image path into string array

The image need to be a vector image. Just open the SVG in a text editor to get the path data. Useful tool: http://inloop.github.io/svg2android/


Make superset-logo-horiz.png bigger

I would like to enlarge the superset-logo-horiz.png logo on the Superset UI's frontend.
I understand that the superset_config.py file does not recognize CUSTOM_CSS = "/static/assets/".
Does anyone know the correct way to achieve this?

@Dosu I dont have InjectCustomCss because I have installed superset with pip install apache-superset and not with superset git repository. So how I do that with pip installed superset.


Fix sampler slot extraction on gather4_po_c when debugging dxbc shader

gather4_po_c has 6 operands and offset of sample should be 4 instead of 3.

Thanks for the fix! This looks good.
Can you rebase the PR onto the v1.x branch? v0.x is soft-locked so I'm only accepting PRs against v1.x. A safe fix like this can still be backported onto v0.x but I'll do that manually afterwards.

https://github.com/baldurk/renderdoc/pull/812
It's done. And format fixed too.


add LICENSE

Thanks, but we already have a LICENSE over in LICENSE.md.

Oops, made a mistake in a commit URL, sorry :/
On Saturday, July 18, 2015, Josh Nichols<EMAIL_ADDRESS>wrote:

Thanks, but we already have a LICENSE over in LICENSE.md.
—
Reply to this email directly or view it on GitHub
https://github.com/github/hubot/pull/1000#issuecomment-122395027.

--


[unifi] Add jvmInitHeapSize and jvmMaxHeapSize configuration

Add jvmInitHeapSize and jvmMaxHeapSize which are passed on
to the container as env vars

Bump version, no change in default behaviour

Update README.md with new configuration parameters

Signed-off-by: Thomas Ingvarsson<EMAIL_ADDRESS>Special notes for your reviewer:
Checklist
[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

[x] Chart Version bumped
[x] Variables are documented in the README.md
[x] Title of the PR starts with chart name (e.g. [radarr])

@tingvarsson thanks for the contribution! Do you think those options could be set with the existing extraJvmOpts option?

@onedr0p looked some at it before the PR, jvmInitHeapSize (or rather -xms) is not set by upstream docker image by default and could be set via extraJvmOpts.
But jvmMaxHeapSize (-xmx) is set explicitly to a default value by upstream docker image, so setting it via extraJvmOpts would be overridden (not explicitly but by the ordering of the ARG string built up by the entrypoint script).
But a thought with the PR as is was to mimic the available config/env params as upstream docker image in this case.

Thanks for the explanation, LGTM!


ringtone -> notification panel

How to stop ringtone without click event?
How to stop click on notification panel?

currently not supported yet


Fill immutable data classes

Hey, it looks like the filling of immutable data classes is not currently available as:
RandomData.java

if ((f.getModifiers() & Modifier.FINAL) != 0)
         continue;

Perhaps setting field to non-final would work there?

This is intentional.
Immutable data classes can only be generated. The sample app contains an example of an immutable data class being generated using a constructor with parameter annotations.
Final fields are not modified by RandomData because plenty of them are used for optimized constants and implicit references to enclosing classes' instances, which shouldn't be modified. I don't think that there's a way to reliably tell, which final field is safe to modify, so I decided to leave them alone.

Ok, I've found the root of my issue, when Kotlin's default value is provided for the constructor field it creates second constructor which does not inherit RandomValue annotations, so the data is not populated properly.
Some generators work anyway as they don't check annotations in getDefaultMatcher().


java.lang.IllegalArgumentException: Input to reshape is a tensor with 3072 values, but the requested shape has 9437184

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 18.04
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: NA
TensorFlow installed from (source or binary): source
TensorFlow version (use command below): 1.8
Python version:3.6
Bazel version (if compiling from source):0.16.1
GCC/Compiler version (if compiling from source):NA
CUDA/cuDNN version:NA
GPU model and memory:NA
Exact command to reproduce:NA

Describe the problem
I have built a custom classifier model using keras with tensorflow backend and I am trying to inference the model to use it in model. I have successfully created protobuf file which contains inference model. When I am trying to feed the input to the model in android. I am getting this error
java.lang.IllegalArgumentException: Input to reshape is a tensor with 3072 values, but the requested shape has 9437184
    	 [[{{node reshape_1/Reshape}} = Reshape[T=DT_FLOAT, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_reshape_1_input_0_0, reshape_1/Reshape/shape)]]

Source code / logs
09-24 18:24:52.780 2480-3594/com.appa.iocsv2 E/TensorFlowInferenceInterface: Failed to run TensorFlow inference with inputs:[reshape_1_input], outputs:[dense_2/Softmax]

    --------- beginning of crash
09-24 18:24:52.780 2480-3594/com.appa.iocsv2 E/AndroidRuntime: FATAL EXCEPTION: IntentService[Prediction Service]
    Process: com.appa.iocsv2, PID: 2480
    java.lang.IllegalArgumentException: Input to reshape is a tensor with 3072 values, but the requested shape has 9437184
    	 [[{{node reshape_1/Reshape/eightbit}} = QuantizedReshape[T=DT_QUINT8, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/device:CPU:0"](reshape_1/Reshape_eightbit/reshape_1_input/quantize, reshape_1/Reshape/shape, reshape_1/Reshape_eightbit/reshape_1_input/quantize:1, reshape_1/Reshape_eightbit/reshape_1_input/quantize:2)]]
        at org.tensorflow.Session.run(Native Method)
        at org.tensorflow.Session.access$100(Session.java:48)
        at org.tensorflow.Session$Runner.runHelper(Session.java:314)
        at org.tensorflow.Session$Runner.run(Session.java:264)
        at org.tensorflow.contrib.android.TensorFlowInferenceInterface.run(TensorFlowInferenceInterface.java:228)
        at org.tensorflow.contrib.android.TensorFlowInferenceInterface.run(TensorFlowInferenceInterface.java:197)
        at org.tensorflow.contrib.android.TensorFlowInferenceInterface.run(TensorFlowInferenceInterface.java:187)
        at com.appa.iocsv2.analyze.PredictionService.predict(PredictionService.java:151)
        at com.appa.iocsv2.analyze.PredictionService.onHandleIntent(PredictionService.java:103)
        at android.app.IntentService$ServiceHandler.handleMessage(IntentService.java:68)
        at android.os.Handler.dispatchMessage(Handler.java:102)
        at android.os.Looper.loop(Looper.java:154)
        at android.os.HandlerThread.run(HandlerThread.java:61)

@maitreya2954 Hi, could you please share a code snippet to reproduce this error.

Hey @harshini-gadige
Here is the code for which i am facing the issue
Keras model
model = Sequential()
model.add(Reshape((image_dim, image_dim, 3), input_shape=(image_dim * image_dim * 3,)))
model.add(Conv2D(32, (5, 5), padding='same', kernel_regularizer=l2(reg_strength)))
model.add(LeakyReLU(alpha=alpha_LReLU))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Dropout(dropout_rate))
model.add(Flatten())
model.add(Dense(512, kernel_constraint=maxnorm(1), kernel_regularizer=l2(reg_strength)))
model.add(LeakyReLU(alpha=alpha_LReLU))
model.add(Dropout(dropout_rate))
model.add(Dense(64, kernel_constraint=maxnorm(1), kernel_regularizer=l2(reg_strength)))
model.add(LeakyReLU(alpha=alpha_LReLU))
model.add(Dropout(dropout_rate))
model.add(Dense(num_of_classes, activation='softmax', kernel_regularizer=l2(reg_strength)))

Inferencing into protobuf file
from tensorflow.python.framework import graph_util
from tensorflow.python.framework import graph_io
if args.quantize:
    from tensorflow.tools.graph_transforms import TransformGraph
    transforms = ["quantize_weights", "quantize_nodes"]
    transformed_graph_def = TransformGraph(sess.graph.as_graph_def(), [], pred_node_names, transforms)
    constant_graph = graph_util.convert_variables_to_constants(sess, transformed_graph_def, pred_node_names)
else:
    constant_graph = graph_util.convert_variables_to_constants(sess, sess.graph.as_graph_def(), pred_node_names)
graph_io.write_graph(constant_graph, output_fld, args.output_model_file, as_text=False)

Android
inferenceInterface = new TensorFlowInferenceInterface(getAssets(), "CNNMODEL.pb");                Log.d("GETREP", "Tensorflow model loaded...Input size: " + String.valueOf(input.length));
//dimensions of input is (1, 3072)
float[] output = predict(input);

private float[] predict(float[] input){
        // model has only 2 output neuron
        float output[] = new float[2];
        inferenceInterface.feed("conv2d_1_input", input, 1, input.length);
        Log.d("GETREP", "Feed completed");
        inferenceInterface.run(new String[]{"dense_2/Softmax"});
        Log.d("GETREP", "Run completed");
        inferenceInterface.fetch("dense_2/Softmax", output);
        // return prediction
        return output;
    }

@maitreya2954 You may want to check whether the input is 1D when feeding the graph. Consider reshaping the input to match TFLite model requirements.

@wt-huang  Currently I am initializing input as

float[] input = new float[3072];
So input is 1D

@maitreya2954 In your code snippet it appears that your image size is 32x32, note that input.length should equal input size. If you are using (1,3072) as your input, then system is expecting image size of 3072x3072, which I doubt that is what you want.

@wt-huang The system is throwing an error The array of length xxxx cannot be wrapped into tensor of length 3072 if size of the array is anything else but 3072.
I have grep'ed the error result and found out it was in https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/kernels/reshape_op.h line 78
OP_REQUIRES(context, shape.num_elements() == input.NumElements(),
                errors::InvalidArgument("Input to reshape is a tensor with ",
                                        input.NumElements(),
                                        " values, but the requested shape has ",
                                        shape.num_elements()));

@maitreya2954 Please post your full code also attach .pb file.

Closing this for now, feel free to reopen if any errors come up.


Issue with Menu dropdown in the navbar

hi,
Thanks for the template its fantatic. I need a little help with the dropdown in navbar. It looks like this html is not working:

          Dropdown

          Action
          Another action

          Something else here

Here is what I have done to kind of accommodate a work around:

                        Homer Simpson

                              {{link.name}}

any thoughts how I can make this better?

I resolved it. For anyone that is having this issues. Make sure that you "import 'bootstrap';" in app.js.
Please also keep in mind that if you are not using entire bootstrap for your application, you can import only the components that you need.
Example:
import 'bootstrap/js/dist/util';
import 'bootstrap/js/dist/dropdown';


feature: host on a dedicated domain

If the editor was hosted on a dedicated domain rather than on a path in my personal domain, it would be easier to remember and more trustworthy.

planckscratchpad.xyz is available for $1.99/year.

Done! Yayyyyy.


Leverage new Python API to pass python.exe path for notebook to Pylance

Contributes to https://github.com/microsoft/pyrx/issues/2343
When Pylance's LSP notebooks experiment is enabled, the Python extension will be responsible for starting Pylance and therefore also needs to be able to provide the python.exe path. This change leverages a new Python API called registerJupyterPythonPathFunction to hook into Pylance's config setting requests and provide the python.exe path for a notebook's kernel.
Jupyter's default behavior is unchanged. It only uses the new behavior when the experiment is enabled via the python.pylanceLspNotebooksEnabled setting.
The related Pylance changes are in this PR: https://github.com/microsoft/pyrx/pull/2382
The related Python changes will be coming soon. They are currently blocked on upgrading Python's LSP version.
@rchiodo
Happy to add unit tests if you think they would be valuable.
I didn't see any mention of Pylance in the test plan, so I didn't make any changes there.

[X] Pull request represents a single change (i.e. not fixing disparate/unrelated things in a single PR).
[X] Title summarizes what is changing.
[X] ~Has a news entry file (remember to thank yourself!).~
[X] Appropriate comments and documentation strings in the code.
[X] Has sufficient logging.
[X] ~Has telemetry for enhancements.~
[X] ~Unit tests & system/integration tests are added/updated.~
[X] ~Test plan is updated as appropriate.~
[X] package-lock.json has been regenerated by running npm install (if dependencies have changed).

Codecov Report

Merging #9842 (64c36dd) into main (26c193c) will increase coverage by 0%.
The diff coverage is 100%.

@@         Coverage Diff          @@
##           main   #9842   +/-   ##
====================================
  Coverage    63%     63%
====================================
  Files       203     203
  Lines      9810    9812    +2
  Branches   1556    1556
====================================
+ Hits       6214    6218    +4
+ Misses     3094    3093    -1
+ Partials    502     501    -1

Impacted Files
Coverage Δ

src/platform/api/types.ts
100% <ø> (ø)

src/platform/common/types.ts
100% <ø> (ø)

src/platform/common/configSettings.ts
84% <100%> (+<1%)
:arrow_up:

src/platform/errors/errorHandler.ts
67% <0%> (+1%)
:arrow_up:

Test failure is due to https://github.com/microsoft/vscode-jupyter/issues/10046


Does it support react？

React is the general trend. Maybe it should support react!!   <Anime ...props  />

I haven't tried it out with react myself but I've seen this repo which has a pre-made react component for working with anime.

Yes react-anime is probably what you're looking for, but note that it still use anime v1.x.

An example of updating a javascript object in state inside a class component instead of directly altering the DOM element ref:
const style = this.state.style

animejs({
  targets: style,
  scale: 1.005,
  boxShadow: 8,
  update: a => this.setState({ style })
})

Is it impossible to use 'ref' as the 'targets'?
In my case it was okay.

I think altering the ref node directly is considered antipattern because it bypasses the whole virtual dom magic. I try to avoid it and keep things declarative.
https://reactjs.org/docs/refs-and-the-dom.htm


BUG: copy wheel files from the Meson install directory

The first commit is the interesting one, the next two are related to installing directories, which can be fixed now that we are copying the files from the installation directory, and the last commit is a refactoring of _install_path that extracts the RPATH changing logic into an external function.
I can split the PR if required. I just did all here because the changes depend on the previous ones, and I thought that they were simple enough to not complicate the review much.

@rgommers, thanks. I should have opened this as a draft PR. I mostly wanted to trigger the CI to see if anything was failing, instead of having to spin up all my local machines :sweat_smile:
I'll address the comments and mark as ready to review later.

I updated test_rpath to check all the RPATH values, it now fails on main, but passes with this PR.
$ git diff
diff --git a/tests/test_wheel.py b/tests/test_wheel.py
index fc91faf..f41236a 100644
--- a/tests/test_wheel.py
+++ b/tests/test_wheel.py
@@ -191,12 +191,15 @@ def test_rpath(wheel_link_against_local_lib, tmp_path):
     artifact = wheel.wheelfile.WheelFile(wheel_link_against_local_lib)
     artifact.extractall(tmp_path)

+    module = tmp_path / f'example{EXT_SUFFIX}'
     if platform.system() == 'Linux':
-        elf = mesonpy._elf.ELF(tmp_path / f'example{EXT_SUFFIX}')
-        assert '$ORIGIN/.link_against_local_lib.mesonpy.libs' in elf.rpath
-    else:  # 'Darwin'
-        dylib = mesonpy._dylib.Dylib(tmp_path / f'example{EXT_SUFFIX}')
-        assert<EMAIL_ADDRESS>in dylib.rpath
+        binary = mesonpy._elf.ELF(module)
+        libdir_origin = '$ORIGIN'
+    elif platform.system() == 'Darwin':
+        binary = mesonpy._dylib.Dylib(module)
+        libdir_origin = '@loader_path'
+
+    assert binary.rpath == {f'{libdir_origin}/.link_against_local_lib.mesonpy.libs'}

 @pytest.mark.skipif(platform.system() not in ['Linux', 'Darwin'], reason='Unsupported on this platform for now')

E       AssertionError: assert frozenset({'$ORIGIN/.link_against_local_lib.mesonpy.libs', '$ORIGIN/'}) == {'$ORIGIN/.link_against_local_lib.mesonpy.libs'}
E         Extra items in the left set:
E         '$ORIGIN/'
E         Full diff:
E         - {'$ORIGIN/.link_against_local_lib.mesonpy.libs'}
E         + frozenset({'$ORIGIN/.link_against_local_lib.mesonpy.libs', '$ORIGIN/'})
E         ? ++++++++++                                               ++++++++++++ +

macOS is failing and I was able to confirm it was due to the commit that copies the files from the install directory.
After some debugging, I suspect that our RPATH handling code on macOS is broken. I don't think install_name_tool -add_rpath is enough.

Here's the difference before (good) and after (bad):
--- test-rpath/bad/otool-l.txt	2023-03-17 10:10:26
+++ test-rpath/good/otool-l.txt	2023-03-17 10:10:19
@@ -176,8 +176,8 @@
   version 0.0
 Load command 11
           cmd LC_LOAD_DYLIB
-      cmdsize 120
-         name<EMAIL_ADDRESS>(offset 24)
+      cmdsize 48
+         name @rpath/libexample.dylib (offset 24)
    time stamp 2 Thu Jan  1 01:00:02 1970
       current version 0.0.0
 compatibility version 0.0.0
@@ -197,6 +197,10 @@
   dataoff 49648
  datasize 560
 Load command 15
+          cmd LC_RPATH
+      cmdsize 32
+         path @loader_path/ (offset 12)
+Load command 16
           cmd LC_RPATH
       cmdsize 64
          path @loader_path/.link_against_local_lib.mesonpy.libs (offset 12)

With the working one:
$ test-rpath/venv/bin/python -c 'import example; print(example.example_sum(1, 2))'
3
$ install_name_tool -delete_rpath @loader_path/ test-rpath/venv/lib/python3.11/site-packages/example.cpython-311-darwin.so
$ test-rpath/venv/bin/python -c 'import example; print(example.example_sum(1, 2))'
3

So it still works after removing @loader_path/ from RPATH. Leaving our otool diff to:
--- test-rpath/bad/otool-l.txt	2023-03-17 10:10:26
+++ test-rpath/artificial/otool-l.txt	2023-03-17 10:18:08
@@ -176,8 +176,8 @@
   version 0.0
 Load command 11
           cmd LC_LOAD_DYLIB
-      cmdsize 120
-         name<EMAIL_ADDRESS>(offset 24)
+      cmdsize 48
+         name @rpath/libexample.dylib (offset 24)
    time stamp 2 Thu Jan  1 01:00:02 1970
       current version 0.0.0
 compatibility version 0.0.0

Meaning we really just need to replace the library install path with @rpath in the LC_LOAD_DYLIB entry.

The macOS CI is happy now, but Windows is still broken.
FAILED tests/test_wheel.py::test_install_subdir_exclude - AssertionError: assert {'subdir2/bar/file1', 'install_subdir_exclude-1.0.0.dist-info/RECORD', 'install_subdir_exclude-1.0.0.dist-info/WHEEL', 'subdir1/foo/file2', 'subdir4/file1', 'exclude/subdir1/file1', 'exclude/subdir1/file2', 'install_subdir_exclude-1.0.0.dist-info/METADATA', 'exclude/file1', 'exclude/subdir1/file3', 'subdir3/bar/file1', 'subdir1/foo/bar/file1', 'subdir2/file2'} == {'subdir2/bar/file1', 'install_subdir_exclude-1.0.0.dist-info/RECORD', 'install_subdir_exclude-1.0.0.dist-info/WHEEL', 'subdir1/foo/file2', 'subdir4/file1', 'exclude/subdir1/file1', 'install_subdir_exclude-1.0.0.dist-info/METADATA', 'exclude/file1', 'subdir3/bar/file1', 'subdir1/foo/bar/file1', 'subdir2/file2'}
  Extra items in the left set:
  'exclude/subdir1/file3'
  'exclude/subdir1/file2'
  Full diff:
    {
     'exclude/file1',
     'exclude/subdir1/file1',
  +  'exclude/subdir1/file2',
  +  'exclude/subdir1/file3',
     'install_subdir_exclude-1.0.0.dist-info/METADATA',
     'install_subdir_exclude-1.0.0.dist-info/RECORD',
     'install_subdir_exclude-1.0.0.dist-info/WHEEL',
     'subdir1/foo/bar/file1',
     'subdir1/foo/file2',
     'subdir2/bar/file1',
     'subdir2/file2',
     'subdir3/bar/file1',
     'subdir4/file1',
    }

I suspect Meson simply isn't doing the path separator translation in the exclude_files field.
...
install_subdir(
  'exclude',
  install_dir: purelib,
  exclude_directories: ['subdir2', 'subdir3'],
  exclude_files: ['file2', 'file3', 'subdir1/file2', 'subdir1/file3'],
)

@eli-schwartz is this expected?

@rgommers this is now ready for review. There's just the Windows failure to deal with, but that's fairly simple, I just want to get some confirmation from the Meson maintainers, and then we can just adjust the test to work around the issue.

@eli-schwartz is this expected?

Looking at the debug commit, this isn't producing any values at all, anywhere. Looking at the point release, this change wasn't released in 1.0.1...

Looking at the debug commit, this isn't producing any values at all, anywhere. Looking at the point release, this change wasn't released in 1.0.1...

Just to be clear, you mean the metadata, right? Anyway, it's super weird the behavior is different on Linux+macOS and Windows.
I am gonna mark this test as XFAIL on Windows to unblock this PR and open a new issue on the bug tracker to track this issue.

It would be very useful for review to not mix unneeded code moves like MAINT: refactor WheelBuilder._install_path  with bug fixes and test changes in the future @FFY00. It makes the diff way larger than it has to be, and checking commit-by-commit is a pain.

Gotcha. I guess this depends on everyone's review approach :sweat_smile:. I'll add XXX comments just to note the possible improvements instead, to avoid mixing changes :+1:

Maybe it's possible to just land those separately then? :)

They conflict with current PR, which is why I did not send them as separate PRs in the first place.

I don't understand why the commits are organized the way they are. The PR would be much easier to review if the fixup commits (and the fixups of the fixups and the reverts of the fixups, and so on) were squashed to the commits they are fixing.

I was asked to not force-push to existing commits to make review easier. This should be an easy enough thing for you to do locally, though.

Both the install_subdir() issue and the rpath issue has been an issue for as long as meson-python exist. I don't think there is anything that requires urgent attention and delaying a release here. install_subdir() with excludes keeps being broken in some configurations on Windows because a bug in Meson.

Closing this as the PR is out of date, probably needs to be reconsidered facing the work done since it was opened, and I am not currently actively contributing this project anymore.


cloudflare_page_rule priority management

Hello,
(terraform 13.0 , cloudflare procider : 2.13.2)
On cloudflare_page_rule we try to manage priority but after apply it's alway propose change because when terraform set priority to 100 for exemple cloudflare recalculate thie priority in function of the number of page rules.
It's cause problem because it's very hard to maintain the page rules priority and when we insert a page rule we need to change all the other priority.
We need than when terraform set priority to 100 cloudflare maintain this priority on 100 and don't change it to be indempotent.
Thanks

Please see https://github.com/cloudflare/terraform-provider-cloudflare/issues/187#issuecomment-450987683 where I go into details on the cause and workaround for this one.

Thanks but it's not a very pleasant way to manage our rule..
imagine i have 100 pages rule i want to insert one more i need to renumber all depend_on and all priority ...
is there a hope that cloudflare change their API to allow a fix priority ?

Being a customer myself using 115-120 of 125 page rules for at least 15 zones, I can definitely sympathise with you :)
I don't know of anything concrete to address it however the engineers and PMs are definitely well across the issues.

Ok i found a better workarround for me
i change priority for make a priority +1 on each ressource
like this i add the depend_on and the priority
`resource "cloudflare_page_rule" "www_js" {
zone_id  = data.cloudflare_zones.cf_zones.zones[0].id
target   = "https://www${local.suffix}.${var.domain}/js/*"
priority = cloudflare_page_rule.www_css.priority + 1
status   = var.page_rules_conf.www_js.status
actions {
browser_cache_ttl = 60
cache_level       = "basic"
edge_cache_ttl    = 7200
cache_ttl_by_status {
codes = "300-599"
ttl   = 0
}
}
}
resource "cloudflare_page_rule" "www_css" {
zone_id  = data.cloudflare_zones.cf_zones.zones[0].id
target   = "https://www${local.suffix}.${var.domain}/css/*"
priority =  1
status   = var.page_rules_conf.www_css.status
actions {
browser_cache_ttl = 60
cache_level       = "basic"
edge_cache_ttl    = 7200
cache_ttl_by_status {
codes = "300-599"
ttl   = 0
}
}
}`
nb : i have different environnement so i get the starting priority from an env from a terraform remote state


fix: don't panic in semantics due to cfg_attr disrupting offsets

Reduces the panic in https://github.com/rust-analyzer/rust-analyzer/issues/11298 to an early return, that means we won't resolve these cases again for now, but this is better than constantly panicking in highlighting and hovering.
bors r+

bors r+


If there are any duplicate Confluence page names, we should catch them before publishing rather than during

At the moment the behaviour is that the plugin deletes all the existing published specs and then republishes, suddenly aborting when it encounters a duplicate page name.  This leaves the specs overall in a half-published state.  Better would be to do a dry run where it walks the tree and checks for duplicates first, so that it fails fast without deleting existing specs or publishing just some of the specs.  This dry run could also be a flag that can be run separately.   This could be a useful check to run on CI/CD pipeline merge requests for instance, to ensure that duplicates are always caught on feature branches before merging to trunk.

Fixed by #37 and #38


Add sni-proxy support to CCM

So the different drivers can be test their new
implementations of sni-proxy
for using it, need to start the cluster from the commandline
like this:
❯ ccm start --sni-proxy
sni_proxy listening on: <IP_ADDRESS>:443

using it from python code, would be a bit diffrent:
nodes_info = get_cluster_info(self.cluster.get_path(),
                              address=self.cluster.nodelist()[0].address(),
                              port=9142)
docker_id, listen_address, listen_port = \
  start_sni_proxy(self.cluster.get_path(), nodes_info=nodes_info)

Ref: https://github.com/scylladb/gocql/pull/97

I really appreciate you writing this feature so quickly and in the coming days I'll try to write an example test in Java Driver that uses this.
Maybe I am too ambitious but it'd be great if this supported topology changes and multi-DC configurations. The second one is probably trivial to implement, but topology changes (adding nodes) may be more difficult (it would require restarting the proxy?).

When running on Fedora 34, I get this error:
$ ccm start --sni-proxy --config-dir=/tmp/1655464854196-0
generating keystore.jks in [/tmp/1655464854196-0/ccm_1]

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore /tmp/1655464854196-0/ccm_1/keystore.jks -destkeystore /tmp/1655464854196-0/ccm_1/keystore.jks -deststoretype pkcs12".
exporting cert from keystore.jks in [/tmp/1655464854196-0/ccm_1]
Certificate stored in file </tmp/1655464854196-0/ccm_1/ccm_node.cer>

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore /tmp/1655464854196-0/ccm_1/keystore.jks -destkeystore /tmp/1655464854196-0/ccm_1/keystore.jks -deststoretype pkcs12".
importing cert into truststore.jks in [/tmp/1655464854196-0/ccm_1]
Certificate was added to keystore
exporting cert to pks12 from keystore.jks in [/tmp/1655464854196-0/ccm_1]
Importing keystore /tmp/1655464854196-0/ccm_1/keystore.jks to /tmp/1655464854196-0/ccm_1/ccm_node.p12...
Using openssl to split pks12 in [/tmp/1655464854196-0/ccm_1] to pem format
pkcs12: Unrecognized flag legacy
pkcs12: Use -help for summary.

I will investigate it myself.

Removing -legacy flag leads to another error:
[piotrgrabowski@localhost scylla-ccm]$ ccm create ccm_1 -i 127.0.1.  -n 1  --scylla  -v release:4.6.3 --config-dir=/tmp/konfig
Current cluster is now: ccm_1
[piotrgrabowski@localhost scylla-ccm]$ ccm start --sni-proxy --config-dir=/tmp/konfig
generating keystore.jks in [/tmp/konfig/ccm_1]

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore /tmp/konfig/ccm_1/keystore.jks -destkeystore /tmp/konfig/ccm_1/keystore.jks -deststoretype pkcs12".
exporting cert from keystore.jks in [/tmp/konfig/ccm_1]
Certificate stored in file </tmp/konfig/ccm_1/ccm_node.cer>

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore /tmp/konfig/ccm_1/keystore.jks -destkeystore /tmp/konfig/ccm_1/keystore.jks -deststoretype pkcs12".
importing cert into truststore.jks in [/tmp/konfig/ccm_1]
Certificate was added to keystore
exporting cert to pks12 from keystore.jks in [/tmp/konfig/ccm_1]
Importing keystore /tmp/konfig/ccm_1/keystore.jks to /tmp/konfig/ccm_1/ccm_node.p12...
Using openssl to split pks12 in [/tmp/konfig/ccm_1] to pem format
writing RSA key
(EE)  Connection error: ('Unable to connect to any servers', {'<IP_ADDRESS>': error(111, "Tried connecting to [('<IP_ADDRESS>', 9042)]. Last error: Connection refused")})(EE)  (EE)  Connection error: ('Unable to connect to any servers', {'<IP_ADDRESS>': error(111, "Tried connecting to [('<IP_ADDRESS>', 9042)]. Last error: Connection refused")})(EE)  generating keystore.jks in [/tmp/tmpv4kzgxjl]

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore /tmp/tmpv4kzgxjl/keystore.jks -destkeystore /tmp/tmpv4kzgxjl/keystore.jks -deststoretype pkcs12".
exporting cert from keystore.jks in [/tmp/tmpv4kzgxjl]
Certificate stored in file </tmp/tmpv4kzgxjl/ccm_node.cer>

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore /tmp/tmpv4kzgxjl/keystore.jks -destkeystore /tmp/tmpv4kzgxjl/keystore.jks -deststoretype pkcs12".
importing cert into truststore.jks in [/tmp/tmpv4kzgxjl]
Certificate was added to keystore
exporting cert to pks12 from keystore.jks in [/tmp/tmpv4kzgxjl]
Importing keystore /tmp/tmpv4kzgxjl/keystore.jks to /tmp/tmpv4kzgxjl/ccm_node.p12...
Using openssl to split pks12 in [/tmp/tmpv4kzgxjl] to pem format
writing RSA key
Traceback (most recent call last):
  File "/usr/local/bin/ccm", line 4, in <module>
    __import__('pkg_resources').run_script('ccm==2.0.5', 'ccm')
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 651, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 1448, in run_script
    exec(code, namespace, namespace)
  File "/usr/local/lib/python3.9/site-packages/ccm-2.0.5-py3.9.egg/EGG-INFO/scripts/ccm", line 75, in <module>
    cmd.run()
  File "/usr/local/lib/python3.9/site-packages/ccm-2.0.5-py3.9.egg/ccmlib/cmds/cluster_cmds.py", line 680, in run
    start_sni_proxy(self.cluster.get_path(), nodes_info=nodes_info)
  File "/usr/local/lib/python3.9/site-packages/ccm-2.0.5-py3.9.egg/ccmlib/utils/sni_proxy.py", line 120, in start_sni_proxy
    address, _, _ = list(nodes_info)[0]
IndexError: list index out of range

When running on Fedora 34, I get this error:
$ ccm start --sni-proxy --config-dir=/tmp/1655464854196-0
generating keystore.jks in [/tmp/1655464854196-0/ccm_1]

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore /tmp/1655464854196-0/ccm_1/keystore.jks -destkeystore /tmp/1655464854196-0/ccm_1/keystore.jks -deststoretype pkcs12".
exporting cert from keystore.jks in [/tmp/1655464854196-0/ccm_1]
Certificate stored in file </tmp/1655464854196-0/ccm_1/ccm_node.cer>

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore /tmp/1655464854196-0/ccm_1/keystore.jks -destkeystore /tmp/1655464854196-0/ccm_1/keystore.jks -deststoretype pkcs12".
importing cert into truststore.jks in [/tmp/1655464854196-0/ccm_1]
Certificate was added to keystore
exporting cert to pks12 from keystore.jks in [/tmp/1655464854196-0/ccm_1]
Importing keystore /tmp/1655464854196-0/ccm_1/keystore.jks to /tmp/1655464854196-0/ccm_1/ccm_node.p12...
Using openssl to split pks12 in [/tmp/1655464854196-0/ccm_1] to pem format
pkcs12: Unrecognized flag legacy
pkcs12: Use -help for summary.
Traceback (most recent call last):
  File "/usr/local/bin/ccm", line 4, in <module>
    __import__('pkg_resources').run_script('ccm==2.0.5', 'ccm')
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 651, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 1448, in run_script
    exec(code, namespace, namespace)
  File "/usr/local/lib/python3.9/site-packages/ccm-2.0.5-py3.9.egg/EGG-INFO/scripts/ccm", line 75, in <module>
    cmd.run()
  File "/usr/local/lib/python3.9/site-packages/ccm-2.0.5-py3.9.egg/ccmlib/cmds/cluster_cmds.py", line 654, in run
    generate_ssl_stores(self.cluster.get_path())
  File "/usr/local/lib/python3.9/site-packages/ccm-2.0.5-py3.9.egg/ccmlib/utils/ssl_utils.py", line 48, in generate_ssl_stores
    subprocess.check_call(['openssl', 'pkcs12', '-legacy', '-in', os.path.join(base_dir, 'ccm_node.p12'),
  File "/usr/lib64/python3.9/subprocess.py", line 373, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['openssl', 'pkcs12', '-legacy', '-in', '/tmp/1655464854196-0/ccm_1/ccm_node.p12', '-passin', 'pass:cassandra', '-nokeys', '-out', '/tmp/1655464854196-0/ccm_1/ccm_node.pem']' returned non-zero exit status 1.

I will investigate it myself.

I've fixed the -legacy flag, I was using locally openssl 3.1, and I see fedora has openssl 1.1, which doesn't have this flag.

Removing -legacy flag leads to another error:
[piotrgrabowski@localhost scylla-ccm]$ ccm create ccm_1 -i 127.0.1.  -n 1  --scylla  -v release:4.6.3 --config-dir=/tmp/konfig
Current cluster is now: ccm_1
[piotrgrabowski@localhost scylla-ccm]$ ccm start --sni-proxy --config-dir=/tmp/konfig
generating keystore.jks in [/tmp/konfig/ccm_1]

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore /tmp/konfig/ccm_1/keystore.jks -destkeystore /tmp/konfig/ccm_1/keystore.jks -deststoretype pkcs12".
exporting cert from keystore.jks in [/tmp/konfig/ccm_1]
Certificate stored in file </tmp/konfig/ccm_1/ccm_node.cer>

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore /tmp/konfig/ccm_1/keystore.jks -destkeystore /tmp/konfig/ccm_1/keystore.jks -deststoretype pkcs12".
importing cert into truststore.jks in [/tmp/konfig/ccm_1]
Certificate was added to keystore
exporting cert to pks12 from keystore.jks in [/tmp/konfig/ccm_1]
Importing keystore /tmp/konfig/ccm_1/keystore.jks to /tmp/konfig/ccm_1/ccm_node.p12...
Using openssl to split pks12 in [/tmp/konfig/ccm_1] to pem format
writing RSA key
(EE)  Connection error: ('Unable to connect to any servers', {'<IP_ADDRESS>': error(111, "Tried connecting to [('<IP_ADDRESS>', 9042)]. Last error: Connection refused")})(EE)  (EE)  Connection error: ('Unable to connect to any servers', {'<IP_ADDRESS>': error(111, "Tried connecting to [('<IP_ADDRESS>', 9042)]. Last error: Connection refused")})(EE)  generating keystore.jks in [/tmp/tmpv4kzgxjl]

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore /tmp/tmpv4kzgxjl/keystore.jks -destkeystore /tmp/tmpv4kzgxjl/keystore.jks -deststoretype pkcs12".
exporting cert from keystore.jks in [/tmp/tmpv4kzgxjl]
Certificate stored in file </tmp/tmpv4kzgxjl/ccm_node.cer>

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore /tmp/tmpv4kzgxjl/keystore.jks -destkeystore /tmp/tmpv4kzgxjl/keystore.jks -deststoretype pkcs12".
importing cert into truststore.jks in [/tmp/tmpv4kzgxjl]
Certificate was added to keystore
exporting cert to pks12 from keystore.jks in [/tmp/tmpv4kzgxjl]
Importing keystore /tmp/tmpv4kzgxjl/keystore.jks to /tmp/tmpv4kzgxjl/ccm_node.p12...
Using openssl to split pks12 in [/tmp/tmpv4kzgxjl] to pem format
writing RSA key
Traceback (most recent call last):
  File "/usr/local/bin/ccm", line 4, in <module>
    __import__('pkg_resources').run_script('ccm==2.0.5', 'ccm')
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 651, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 1448, in run_script
    exec(code, namespace, namespace)
  File "/usr/local/lib/python3.9/site-packages/ccm-2.0.5-py3.9.egg/EGG-INFO/scripts/ccm", line 75, in <module>
    cmd.run()
  File "/usr/local/lib/python3.9/site-packages/ccm-2.0.5-py3.9.egg/ccmlib/cmds/cluster_cmds.py", line 680, in run
    start_sni_proxy(self.cluster.get_path(), nodes_info=nodes_info)
  File "/usr/local/lib/python3.9/site-packages/ccm-2.0.5-py3.9.egg/ccmlib/utils/sni_proxy.py", line 120, in start_sni_proxy
    address, _, _ = list(nodes_info)[0]
IndexError: list index out of range

I've tested with multiple node (i.e. 3) and seem like the code doesn't wait for CQL to be open, before moving forward, I'll fix that.
(EE)  Connection error: ('Unable to connect to any servers', {'<IP_ADDRESS>': error(111, "Tried connecting to [('<IP_ADDRESS>', 9042)]. Last error: Connection refused")})(EE)  (EE)  Connection error: ('Unable to connect to any servers', {'<IP_ADDRESS>': error(111, "Tried connecting to [('<IP_ADDRESS>', 9042)]. Last error: Connection refused")})(EE)  generating keystore.jks in [/tmp/tmpv4kzgxjl]

On the latest version of this PR with some manual configuration (on POC driver side) I have successfully started it and connected with my POC driver.
There are a few changes that would be welcome:

Writing create_cloud_config() to file so that it can be read by client.
Configurable sniproxy port, DNS name (not cluster-id.scylla.com). Scylla Java Driver integration tests maintain a pool of started CCM clusters, so there could be a few running at the same time (and they should have different ports).
Currently the same certificate is used for both client, server - those should be separate
(Continuing previous point) Preferably it should be possible to turn off CCM key generation and provide your own keys.
To properly test with my POC driver, I had to add any.cluster-id.scylla.com to /etc/hosts - the driver tries to connect to any.cluster-id.scylla.com. Should CCM add it automatically to /etc/hosts (and clean up it afterwards)? Or maybe it is possible to do it without modifying /etc/hosts?
When adding new nodes, sniproxy container is stopped and restarted. It would be better to update its configuration without restarting (don't know if its possible with that sniproxy program - maybe something more advanced would be needed). The reason is that because proxy is stopped, the control connection is dropped and started again, not properly testing whether the control node got the notification.

I'd be happy to help you with doing some of those if you'd like.


Example 4 help needed

Hi,
i just followed : https://matthijskamstra.github.io/haxephp/04webapi/example.html
and compiled it successfully however i am getting :

The problem is that i am not using root of the Localhost: i have a folder : TESST/
and the url is always going to the root [ localhost/contact ] instead of [ localhost/TESST/contact ]
i tried to add that to :
Dispatch.run("TESST/"+_webURI, Web.getParams(), new Routes());
but no matter what it is going to the root.

Hi @Gamezpedia,
thx for taking the time to follow the Haxe php instructions.
The example is a very simple one if you want to create more routes you should check out this gist https://gist.github.com/jasononeil/5667079
it might help you on your way

Oh for I forget
I don't use the Haxe target PHP any more. So the documentation might become more and more obsolete.
I know that the new Haxe (4) version only supports php7.
Read more about that here: https://haxe.org/download/version/4.0.0-preview.2/
That page also explains that the dispatch (in Haxe 4) is moving to an other lib: https://github.com/HaxeFoundation/hx3compat/tree/master/std/haxe/web
So that changes as well with the new version

ohh i see , it looks like i am late to the party :D any ways thanks for these tutorials i am not a php guy but they are a kick starter for me instead of writing ugly php i wrote Haxe :D   👍