NIFI-13420 Persist value of maxNonHeapBytes as -1 for a cluster's system diagnostics when at least node has this value - for 1.x

Summary
This is a request for changes made for
NIFI-13420 for 2.0 in 8983 be backported for 1.x as well.
Tracking
Please complete the following tracking steps prior to pull request creation.
Issue Tracking

[x] Apache NiFi Jira issue created

Pull Request Tracking

[x] Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
[x] Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000

Pull Request Formatting

[ ] Pull Request based on current revision of the main branch
[x] Pull Request refers to a feature branch with one commit containing changes

Verification
Please indicate the verification steps performed prior to pull request creation.
Build

[ ] Build completed using mvn clean install -P contrib-check

[ ] JDK 21

Licensing

[x] New dependencies are compatible with the Apache License 2.0 according to the License Policy
[x] New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

[x] Documentation formatting appears as expected in rendered files

Merged to support branch in https://github.com/apache/nifi/commit/34b5297c93543a085fcfe98f7f325e728986af48


[BSD] Server listens on IPv6 only

Looks like oxy binds on  [::], which is

dual-stack by default (actually, depending on configuration, might not be!) on Linux
v6-only by default on FreeBSD (interestingly, mio API offers set_only_v6 that you can call... after the bind?!)
v6-only always on OpenBSD and others

https://community.openvpn.net/openvpn/ticket/937?cversion=1&cnum_hist=5
I think two sockets should be used.

Yeah, mio::TcpListener::set_only_v6 is literally useless, because there's no way to do it before bind() and it doesn't work after the bind. It's inherited from the Rust stdlib: https://github.com/rust-lang/rust/issues/33052 ... it's a heck of a thing.
We'd basically just have to use the ::nix socket calls to make the socket and then ::mio::TcpListener::from_raw_socket.
I started out using two sockets, but gave up because set_only_v6 doesn't work.
I'll bust it out to two sockets eventually, or merge your PR. :+1:

I've pushed an implementation in dev: https://github.com/oxy-secure/oxy/commit/562e42780d526cc9bfa925153abc5bae8c044195
Instead of actually dropping down to ::nix, I think I can get away with just yoloing it and letting the IPv4 bind fail if the IPv6 bind is shadowing it? Please test it out and let me know what you think.

Yeah, that works for me :) looks like a potential annoying warning on Linux though?

Oh yeah, those should all be debug!() for sure. I'll bump 'em down.

No, wait, are you talking about the "Failed to register" warn!()s? Those should only fire in the weird case that bind() actually succeeds but then epoll_ctl() is what fails. If it fails on bind() (where it should) it just moves along silently.

Oh, right, I can't read apparently lol


Add compatibility with Julia LTS

Currently the Julia interface only supports Julia versions 1.8 and newer because of its use of const annotation for 3 fields of StanModel. This Julia bound will propagate to all dependants of this package at Pkg resolve time, so e.g. if StanSample adds BridgeStan as a required dependency, they will need to drop support for Julia v1.7 and earlier.
Currently the LTS is v1.6, and most Julia packages support at least the LTS unless they have critical functionality that requires newer Julia features. In my opinion the utility of making these 3 fields constant is not worth the limitations it imposes on dependents, but in this PR, I demonstrate how a macro can be used to conditionally annotate those fields only on v1.8 and newer. Locally, the test suite passes now for at least v1.6.
Thanks to @MasonProtter for writing the macro!

Merging #70 caused a merge conflict here, but if you can fix this it looks phenomenal!

Done! I also checked whether older versions of Julia could be supported, but no, Libc.Libdl was introduced on v1.6. This is fine; many packages are bumping their lower bounds to v1.6, so I don't think this will block others from depending on BridgeStan.


[bug] and_return_result_of doesn't correctly handle async methods

    def return_value(self, *args, **kwargs):
        """Extracts the real value to be returned from the wrapping callable.

        :return: The value the double should return when called.
        """

        self._called()
>       return self._return_value(*args, **kwargs)
E       TypeError: 'coroutine' object is not callable

if you create an allowance on an async method and use and_return_result_of you get the above error. I think this specifically will only happen when the fake is expecting arguments as the default usage that and_return does hides the error by wrapping argument-less _return_value in a lambda

I found the issue, working on a solution.


Database Development

Supplied:
Code should be added to: https://github.com/nimh-dsst/sharestats-leo-notfork/tree/dev_2024
100 PDFs will be provided
Metadata for populating provenance

Create Documents, Works, and provenance table schema (sqlalchemy models) from the tables outlined here: https://docs.google.com/spreadsheets/u/6/d/10RpZ0DqPvWx4Et_-K7OtQ2Guqoby6-od3F9sux6p5XY/edit .
Write a simple reusable script to create a postgres database (local docker container using compose)
Write a script to do a SQL dump to back up DB state, eventually will store on s3. Should be able to target DB that the backup is restored to.
Write a script to upload the 100 pdfs to s3 and populate the appropriate tables: document (for each pdf), work (representing a distinct publication), and provenance (representing history of each table record)
Database creation, population, backup, and restoration should be reproducible on another computer
Pytest should be started and expanded for utility scripts, database integrity, and DB models
Deploy to a AWS postgres and add github actions for preliminary management

I'm working on this issue, at PR: https://github.com/nimh-dsst/sharestats-leo-notfork/pull/7

I'm working on this issue, at PR: #7

From @leej3's message on slack. The tables should be filled in with the following information:

Documents Table

hash: This is just a hash of the PDF bytes/file.
created_at: Autogenerated during script execution (or using a default factory).
s3uri: The URI where the PDF file is stored for later use (AWS credentials will be provided to upload each PDF).
work_id: (Previously publication_id) A reference to an entry in the "Works" table. A "work" represents a distinct academic publication (it may be accessible via different URLs, such as at the Nature Neuroscience journal's website and in PubMed Central).
provenance_id: A reference to a record in the provenance table. This will be very sparsely populated.

Works Table

id: Auto-generated (by database upon creation).
created_at: Auto-generated (default factory).
modified_at: Auto-generated.
primary_document_hash: For now, this will be the same as the initial document ID.
provenance_id: Another reference to an entry in the provenance table, also very sparsely populated.

Provenance Table

id: Auto-generated.
pipeline_name: Possible values include "work initialization" and "document upload" for the two records.
version: Currently arbitrary; dsst_etl._version is likely the most appropriate.
compute: See code reference.
personnel: Use the host's username for now.
comment: Optional argument in the CLI.

The code for this issue is complete and the tests for database creation, record creation, and record removal after testing all pass in my hands. I am closing this issue


[FLINK-15656][k8s] Support pod template for native kubernetes integration

What is the purpose of the change
Now the native K8s integration is almost production ready. We already have the common features for deploying a Flink cluster on K8s cluster natively. However, compared with the current supported features[1] in flink-on-k8s-operator and the feedback from users(ML and JIRAs), I am afraid that we still need some advanced features.

Init containers[2]
Sidecar containers[3]
Arbitrary volume(PVC, hostpath, emptyDir, ConfigMap) mount[4]
Set owner reference for the JobManager deployment[5]
Support customizing of containers for native kubernetes setup[6]
Pod Affinity and Anti-affinity
Pod Security Context
Pod Environment Variables Reference

So this PR tries to introduce a pod template[7] for all the use cases. Users could configure a local yaml file for JobManager/TaskManager. This will let Flink create the JobManager/TaskManager pods with the template first, not the empty. Of course, the Flink config options for Kubernetes[8] have high priority and could overwrite these values.
Pod template is is a typical pod yaml. You could set arbitrary fields that is supported by K8s directly. The following is a simple example, which add the init container, sidecar container and volume mount.
apiVersion: v1
kind: Pod
metadata:
  name: pod-template
spec:
  initContainers:
  - name: artifacts-fetcher
    image: busybox
    imagePullPolicy: IfNotPresent
    # Use wget or other tools to get user jars from remote storage
    command: ['wget', 'https://flink-debug-yiqi.oss-cn-beijing.aliyuncs.com/StateMachineExample.jar', '-O', '/flink-artifact/myjob.jar']
    volumeMounts:
    - mountPath: /flink-artifact
      name: flink-artifact
  containers:
    # Do not change the main container name
  - name: flink-job-manager
    volumeMounts:
    - mountPath: /opt/flink/volumes/hostpath
      name: flink-volume-hostpath
    - mountPath: /opt/flink/usrlib
      name: flink-artifact
    - mountPath: /opt/flink/log
      name: flink-logs
    # Use sidecar container to push logs to remote storage or do some other debugging things
  - name: sidecar-log-collector
    image: busybox
    imagePullPolicy: IfNotPresent
    args:
      - tail
      - -F
      - /flink-log/jobmanager.log
    volumeMounts:
    - mountPath: /flink-log
      name: flink-logs
  volumes:
  - name: flink-volume-hostpath
    hostPath:
      path: /tmp
      type: Directory
  - name: flink-artifact
    emptyDir: {}
  - name: flink-logs
    emptyDir: {}

[1]. https://github.com/GoogleCloudPlatform/flink-on-k8s-operator#features
[2]. https://issues.apache.org/jira/browse/FLINK-15641
[3]. https://issues.apache.org/jira/browse/FLINK-15871
[4]. https://issues.apache.org/jira/browse/FLINK-15649
[5]. https://issues.apache.org/jira/browse/FLINK-20359
[6]. https://issues.apache.org/jira/browse/FLINK-20324
[7]. https://issues.apache.org/jira/browse/FLINK-15656
[8]. https://ci.apache.org/projects/flink/flink-docs-master/deployment/config.html#kubernetes
Brief change log

Introduce the pod template
Add documentation and example

Verifying this change

All the new introduced functionalities are covered by new added unit test
Manually testing

Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (yes / no)
The public API, i.e., is any changed class annotated with @Public(Evolving): (yes / no)
The serializers: (yes / no / don't know)
The runtime per-record code paths (performance sensitive): (yes / no / don't know)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: (yes / no / don't know)
The S3 file system connector: (yes / no / don't know)

Documentation

Does this pull request introduce a new feature? (yes / no)
If yes, how is the feature documented? (doc)

Thanks @wangyang0918,
This works great, I tested it on my app, and its fine. I have 2 questions:

With this in place, you now have 2 ways for setting annotations, labels, env, resources and tolerations, etc. You can set them using dedicated decorators or through port template yaml., for example:

apiVersion: v1
kind: Pod
metadata:
annotations:
prometheus.io/port: "3000"
prometheus.io/scrape: "true"
labels:
deploy: test
version: alphav1
name: pod-template
spec:
containers:
- name: flink-job-manager
env:
- name: "variable1"
value: "value1"
resources:
limits:
cpu: "2"
memory: 4Gi
requests:
cpu: "2"
memory: 4Gi
volumeMounts:
- mountPath: /opt/flink/conf
name: flink-config-volume
- mountPath: /mnt/flink/storage
name: pvc-checkpointing
- mountPath: /opt/pvcclaim/test/path1
name: pvc-mount2
subPath: path1
- mountPath: /opt/pvcclaim/test/path2
name: pvc-mount2
subPath: path2
- mountPath: /emptydirclaim
name: edir-mount
- mountPath: /var/local/hp
name: hp-mount
- mountPath: /opt/secret
name: secret-mount
- mountPath: /opt/configmap
name: test-configmap
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: cloudflowflink-token-dbnvh
readOnly: true
tolerations:
- effect: NoExecute
key: node.kubernetes.io/not-ready
operator: Exists
tolerationSeconds: 300
- effect: NoExecute
key: node.kubernetes.io/unreachable
operator: Exists
tolerationSeconds: 300
volumes:
- configMap:
defaultMode: 420
items:
- key: logback-console.xml
path: logback-console.xml
- key: log4j-console.properties
path: log4j-console.properties
- key: flink-conf.yaml
path: flink-conf.yaml
name: flink-config-testapp
name: flink-config-volume
- name: pvc-checkpointing
persistentVolumeClaim:
claimName: testclaim
- name: pvc-mount2
persistentVolumeClaim:
claimName: testclaim1
- emptyDir: {}
name: edir-mount
- hostPath:
path: /var/local/hp
type: DirectoryOrCreate
name: hp-mount
- name: secret-mount
secret:
defaultMode: 420
items:
- key: token
path: value
secretName: test-secret2
- configMap:
defaultMode: 420
items:
- key: key1
path: "1"
name: test-configmap
name: test-configmap
- name: cloudflowflink-token-dbnvh
secret:
defaultMode: 420
secretName: cloudflowflink-token-dbnvh
This template is setting a lot of parameters, that can be used to set a lot of things. Does it make sense to use template as a main configuration approach and remove other options.

Both this and owner manager PRs are still not merged. - do you have a time frame for merging/releasing them?

Thanks @wangyang0918,
This works great, I tested it on my app, and its fine. I have 2 questions:

With this in place, you now have 2 ways for setting annotations, labels, env, resources and tolerations, etc. You can set them using dedicated decorators or through port template yaml., for example:

apiVersion: v1
kind: Pod
metadata:
annotations:
prometheus.io/port: "3000"
prometheus.io/scrape: "true"
labels:
deploy: test
version: alphav1
name: pod-template
spec:
containers:
- name: flink-job-manager
env:
- name: "variable1"
value: "value1"
resources:
limits:
cpu: "2"
memory: 4Gi
requests:
cpu: "2"
memory: 4Gi
volumeMounts:
- mountPath: /opt/flink/conf
name: flink-config-volume
- mountPath: /mnt/flink/storage
name: pvc-checkpointing
- mountPath: /opt/pvcclaim/test/path1
name: pvc-mount2
subPath: path1
- mountPath: /opt/pvcclaim/test/path2
name: pvc-mount2
subPath: path2
- mountPath: /emptydirclaim
name: edir-mount
- mountPath: /var/local/hp
name: hp-mount
- mountPath: /opt/secret
name: secret-mount
- mountPath: /opt/configmap
name: test-configmap
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: cloudflowflink-token-dbnvh
readOnly: true
tolerations:
- effect: NoExecute
key: node.kubernetes.io/not-ready
operator: Exists
tolerationSeconds: 300
- effect: NoExecute
key: node.kubernetes.io/unreachable
operator: Exists
tolerationSeconds: 300
volumes:
- configMap:
defaultMode: 420
items:
- key: logback-console.xml
path: logback-console.xml
- key: log4j-console.properties
path: log4j-console.properties
- key: flink-conf.yaml
path: flink-conf.yaml
name: flink-config-testapp
name: flink-config-volume
- name: pvc-checkpointing
persistentVolumeClaim:
claimName: testclaim
- name: pvc-mount2
persistentVolumeClaim:
claimName: testclaim1
- emptyDir: {}
name: edir-mount
- hostPath:
path: /var/local/hp
type: DirectoryOrCreate
name: hp-mount
- name: secret-mount
secret:
defaultMode: 420
items:
- key: token
path: value
secretName: test-secret2
- configMap:
defaultMode: 420
items:
- key: key1
path: "1"
name: test-configmap
name: test-configmap
- name: cloudflowflink-token-dbnvh
secret:
defaultMode: 420
secretName: cloudflowflink-token-dbnvh
This template is setting a lot of parameters, that can be used to set a lot of things. Does it make sense to use template as a main configuration approach and remove other options.

Both this and owner manager PRs are still not merged. - do you have a time frame for merging/releasing them?

@blublinsky Thanks for your testing on the preview version and sharing your thoughts.

The priority between pod template and Flink config options

This is a very good and important concern. When I try to introduce the pod template, I always have a rule in my mind, "Flink config options have a high priority than pod template". If you could enable this feature(e.g. labels, annotations, tolerations, etc.) via config options, then you should do it.
Flink allows users to define the JobManager and TaskManager via pod template file with the non-common features, which are not supported by Flink Kubernetes config options directly. Please use kubernetes.jobmanager.pod-template-file
or kubernetes.taskmanager.pod-template-file to specify a local file that contains pod definition. It will be used to initialize the JobManager/TaskManager instead of an empty pod. Please note that some fields of the pod template will always be overridden by Flink. Refer to the following tables for the full list of pod fields that will be overwritten by Flink. I am now working to provide such tables in this PR.
I know some guys may argue for that. In the pod template design, it is a supplement for some un-common features and provide enough flexibility to use new K8s features. Also I believe for Flink users, they prefer the config option if they could.

Both this and owner manager PRs are still not merged. - do you have a time frame for merging/releasing them?

I am trying to refine this PR and also provide a thorough documentation so that users could easily use it. For owner reference PR, I already left some comments few days ago, could you please address them before merging?

@blublinsky Thanks for your testing on the preview version and sharing your thoughts.

The priority between pod template and Flink config options

This is a very good and important concern. When I try to introduce the pod template, I always have a rule in my mind, "Flink config options have a high priority than pod template". If you could enable this feature(e.g. labels, annotations, tolerations, etc.) via config options, then you should do it.
Flink allows users to define the JobManager and TaskManager via pod template file with the non-common features, which are not supported by Flink Kubernetes config options directly. Please use kubernetes.jobmanager.pod-template-file
or kubernetes.taskmanager.pod-template-file to specify a local file that contains pod definition. It will be used to initialize the JobManager/TaskManager instead of an empty pod. Please note that some fields of the pod template will always be overridden by Flink. Refer to the following tables for the full list of pod fields that will be overwritten by Flink. I am now working to provide such tables in this PR.
I know some guys may argue for that. In the pod template design, it is a supplement for some un-common features and provide enough flexibility to use new K8s features. Also I believe for Flink users, they prefer the config option if they could.

Both this and owner manager PRs are still not merged. - do you have a time frame for merging/releasing them?

I am trying to refine this PR and also provide a thorough documentation so that users could easily use it. For owner reference PR, I already left some comments few days ago, could you please address them before merging?

Thanks @wangyang0918,
I did fix owner reference PR
As for this one, its still up to the user to decide how he wants to set certain parameters - via template or via additional parameters. I think the best that we can do is to create a table (that you are working on) to specify what can be set via different mechanisms and the precedence rules, for example, if I set a label foo with the value bar in template and then set the same label with the value foobar in configuration, the value of the label will be foobar, etc.

Thanks @wangyang0918,
I did fix owner reference PR
As for this one, its still up to the user to decide how he wants to set certain parameters - via template or via additional parameters. I think the best that we can do is to create a table (that you are working on) to specify what can be set via different mechanisms and the precedence rules, for example, if I set a label foo with the value bar in template and then set the same label with the value foobar in configuration, the value of the label will be foobar, etc.

I believe your suggestions make sense. For labels, annotations, node-selectors, tolerations and other similar features, Flink config option will add additional results to such fields. They could be combined easily. However, for other Flink internal generated fields(e.g. namespace, podName, restartPolicy, mem, cpu, etc.), the values from pod template will be always overwritten.
This should be declared clearly in the documentation with few tables.

I believe your suggestions make sense. For labels, annotations, node-selectors, tolerations and other similar features, Flink config option will add additional results to such fields. They could be combined easily. However, for other Flink internal generated fields(e.g. namespace, podName, restartPolicy, mem, cpu, etc.), the values from pod template will be always overwritten.
This should be declared clearly in the documentation with few tables.

@wangyang0918
Did some more experiments with this PR and took more careful look at the code.
In the InitJob/TaskManagerDecorator, editOrNewMetadata() overwrites all of the information from the pod template. As a result, it looks like any metadata settings in the template will be wiped out

@wangyang0918
Did some more experiments with this PR and took more careful look at the code.
In the InitJob/TaskManagerDecorator, editOrNewMetadata() overwrites all of the information from the pod template. As a result, it looks like any metadata settings in the template will be wiped out

Moreover, you can't use pod template for anything, that is set by Flink decorators

Moreover, you can't use pod template for anything, that is set by Flink decorators

For labels, annotations, node-selectors, tolerations and other similar features, Flink config option will add additional results to such fields. They could be combined easily. However, for other Flink internal generated fields(e.g. namespace, podName, restartPolicy, mem, cpu, etc.), the values from pod template will be always overwritten.

As a result, it looks like any metadata settings in the template will be wiped out

@blublinsky Yes, this is what I am changing to get your suggestion integrated.

For labels, annotations, node-selectors, tolerations and other similar features, Flink config option will add additional results to such fields. They could be combined easily. However, for other Flink internal generated fields(e.g. namespace, podName, restartPolicy, mem, cpu, etc.), the values from pod template will be always overwritten.

As a result, it looks like any metadata settings in the template will be wiped out

@blublinsky Yes, this is what I am changing to get your suggestion integrated.

Thank you for your work
Following the discussion on the user ML at the beginning of the week, I tried your PR against my workload.
My goal was to set the ephemeral storage requests and limits, but it seems the output of the ResourceRequirementsBuilder takes precedence over the template, so my config is ignored.
However I was able to set a priorityClassName without trouble.
As discussed in this PR, I think that while options set using flink options should have higher precedence, they should be merged whenever possible with the template.

@MiLk Thanks for your testing. Could you please share how do you set the request and limit for ephemeral storage, like the following?
  volumes:
  - name: flink-artifact
    emptyDir:
      sizeLimit: "1Gi"
  - name: flink-logs
    emptyDir:
      sizeLimit: "1Gi"

No, I'm doing the following:
    apiVersion: v1
    kind: Pod
    metadata:
      name: pod-template
    spec:
      containers:
        - name: flink-task-manager
          resources:
            requests:
              ephemeral-storage: 256Mi
            limits:
              ephemeral-storage: 256Mi

cc @blublinsky @MiLk I have pushed another commit to support merging Flink config options and pod template. Maybe you could have a try on this and share your feedback. I really appreciate for that.
The documentation is on the way. And I believe this PR will be ready for review in this week.

@wangyang0918, I did not have a chance to run the code, but looking at the changes, it looks really good

I could confirm that I could set both priority class and ephemeral storage requests and limits 🎉
It seems to behave as expected, thank you for the hard work.

@MiLk Thanks for your feedback.
I have added a new commit for the documentation. Now it is ready for review.

cc @xintongsong @tillrohrmann Could you please help with reviewing this PR? The functionality has already be verified by at least three community users and our internal use case.

Thanks @wangyang0918. I'll take a look asap.

cc @xintongsong I have resolved the conflicts and update the documentation based on hugo. Please help with the review.

@xintongsong Thanks for the review. Address your concerns inline and update the PR.

It seems the current implementation cannot support setting the same template file for both JM / TM, due to the different main container names. Maybe it's not necessary to have different main container names for JM/TM.

Yes, it does not support sharing the pod template at my first implementation. Because I used to believe only the JobManager needs init container. And this will make JobManager and TaskManager pod templates always different. Inspired by you and after more consideration, I think it makes sense to let them share a same pod template since it is easier to use and harmless. I have unified the main container name and introduced a common config option kubernetes.pod-template-file in this update.

It's not clear to me what kind of compatibility guarantees we provide for the pod templates. Would it be possible that something user defined in the template works well in the current Flink version and is overwritten in a new version? I understand it's hard to guarantee that all templates are compatible across versions, but maybe we can define a set of commonly used functions that is guaranteed (guarded by tests) compatible across versions.

I think it is possible that a pod template works well in current Flink version and does not work in a new Flink version. This only happens when we introduce a new config option and overwrite(not merge) the corresponding Kubernetes field. This should rarely happen unless we have no other choices. Then we will document them and publish in the release note.
We could guarantee to our users that a white list fields(e.g. annotations, labels, envs, init container, sidecar container, volumes) in pod template that will always be respected. And I have enriched the tests in KubernetesJobManagerFactoryWithPodTemplateTest and KubernetesTaskManagerFactoryWithPodTemplateTest to guide them.

@xintongsong Thanks for the review. I have addressed the comments and force pushed.

@xintongsong Thanks for reviewing the documentation. I have integrated your comments. Please have another look.

cc @tillrohrmann could you please have a final check and help with merging?

cc @tillrohrmann I have addressed the comments and force pushed.

@tillrohrmann Thanks for your comments and suggestions. I have updated this PR. The main changes are list as followings.

Add a separate decorator for PodTemplateMountDecorator which is only used for KubernetesJobManagerFactory. And then we could still keep the file existence check.
Introduce new mechanism for resolving effective field values. "Defined by Flink", "Defined by the user" and "Merged with Flink". The documentation is updated accordingly.

For your posted two problems, the first one is a bug and has been fixed. For the second one, I think it is an expected behavior. We have deferred the sanity check to the Fabric8 Kubernetes client, which will call the Kubernetes APIServer for the verification. I admin that the exception is not friendly, but you still could find the root cause spec.template.spec.containers[0].image: Required value. I guess you do not set the image for the non-main container.

cc @tillrohrmann comments addressed.

cc @tillrohrmann comments addressed. Please have a look at your convenience.

cc @tillrohrmann Comments addressed. Please have a look at your convenience.
I have also changed a bit about the service account because of following reasons.

The Kubernetes provides two ways to set the service account(withServiceAccount and withServiceAccountName)
When checking whether config options are explicitly set, the fallback config option should also be checked. So I am introducing the withFallbackKeys.

cc @tillrohrmann Comments addressed. Please have a look at your convenience.
I have also changed a bit about the service account because of the following reasons.

The Kubernetes provides two ways to set the service account(withServiceAccount and withServiceAccountName). We should set them both.
When checking whether config options are explicitly set, the fallback config option should also be checked. So I am introducing the withFallbackKeys.

Thanks for updating the PR @wangyang0918. The changes look good to me. Merging this PR now.

One thing we should probably do as follow up it to add an end-to-end test for the pod template. Maybe we can adapt one of the existing K8s e2e tests. That way we ensure that we don't accidentally break this feature w/o us noticing it (e.g. calling the file existence check for the specified file in the pods). What do you think @wangyang0918?

Make sense to me. I will create another ticket and PR to update the current application e2e test to also verify pod template.


CIV-6861 Add Unless Order Scheduler Task

Before creating a pull request make sure that:

[x] commit messages are meaningful and follow good commit message guidelines
[x] README and other documentation has been updated / added (if needed)
[x] tests have been updated / new tests has been added (if needed)

Please remove this line and everything above and fill the following sections:
JIRA link (if applicable)
https://tools.hmcts.net/jira/browse/CIV-6861
Change description
Add Unless order WA Task
Does this PR introduce a breaking change? (check one with "x")
[ ] Yes
[ ] No


swap first and last global in hcopt / hc tc path

improved hlb cifar / stable diffusion / resnet default but not quite to pre-lowerer level yet


fix: disable button

The button is no longer clickable(its oket) but I suggest that in this case the button says "Out of Stock".


Research on deduplicate method for pretraining dataset

Description: Research algorithms to do deduplicating the data.
Possible definitions of deduplicated data:

Exact match
Near-duplicate => Only a few words different.
Semantic duplicate => Same meaning

Interesing papers:

Dedup Google: https://arxiv.org/abs/2107.06499 (Note: this paper assumes whitespace as a tokenization technique)
Dedup Facebook: https://arxiv.org/abs/2303.09540 (Note: this paper use fully trained LM to duplicate then train partial LM, so be aware)
Simhash: https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/33026.pdf

สนใจค่ะ

Updated
Input: Huggingface Dataset
Output: Deduplicated Huggingface Dataset


popmenu / dropmenu / scrolllist

been checking old extended objects for inspiration of a dropdown/popup GUI. The ones from extended are actually "popup menus" and we have examples from flatgui/popup and "tof", namelu tof/menubutton and tof/pmenu.
flatgui/popup and tof/menubutton both use the "menubutton" widget from tcl/tk, which looks nice but I'm finding it very problematic. The tof/menu is interesting because it is not a GUI object, without an actual interface, and it does create a popup menu wherever the mouse position is. Hence, it bypasses the "menubutton" mechanism.
MAX has the [umenu] object, which is described as a pop-up menu. Maybe I can design something like that from scratch in tcl/tk and use the popup mechanism from [pmenu].
An actual drop down menu would be more of a scrollist mechanism from unauthorized/scrolllist, where clicking on the object drops down a menu that you can choose an item and select, and even have a scroll bar.
The "menubutton" approach seems like the easiest and more convenient solution right now, as the damn thing is ready as a tcl/tk widget. And I'm working on a rather advanced prototype with it. I may go on with it until the "end". Then I may work on a new one from scratch that is more like MAX's [umenu] to replace it.
By the way, I am also finding some minimal trouble with the popup mechanism from [pmenu] as after selecting, the patch window doesn't get well focused back. As for the "menubutton" approach, it seems to need its own window and that is always on top and hard to manage clicks. And it gets disabled and active in a way that is hard to control and manage in Pd.
An actual dropmenu could be a 2nd option that I can design latter and I think that would be the most sophisticated and less problematic way to do things in Pd. It's gonna take a lot of time and hassle, but I wanna better understand how designing Pd GUIs in tcl/tk work.
BTW, this is a nice look for something designed from scratch in both cases.

[ui.menu] from ceammc seems like a perfect dropdown menu, but I don't understand all the wickery they do with "CICM Wrapper". It's what I would like to try and build from scratch in plain tcl/tk


feat: Hive tests fix

Related: #1489

@fkrause98 can you please allow members of the repo to be pushing to your PRs?^^

@gakonst
Done, please let us know if anyone else needs push permission.

@Rjected can you please take this over so that we can get the hive genesis smoke test passing?
@fkrause98 As a process point: this is part of the blocker PRs for Hive, so if you're opening it as a Draft, ideally we'd like to figure out a way to move it forward. Here it seems like it's NBD as it's just a small bug, but if you guys take on blocker issues let's try to have some more active coordination on how to not let them stagnate.

Sounds good, I'll take this one over the line.

Ok!
And @Rjected, I've been debugging this for a bit, I've found out that the state root is being returned as 0xbdf2d9a7f05e669cacafa199298ab1ee68ec2bb457c516897f22c5971b434591 whereas geth returns 0x9a6049ac535e3dc7436c189eaa81c73f35abd7f282ab67c32944ff0301d63360.

fixed parsing here:
https://github.com/gakonst/ethers-rs/pull/2205

working on another issue with computing the state root after fixing parsing, so there were bugs in both areas!

ethers-rs#2205 is merged @Rjected

@gakonst rebased and updated ethers

Codecov Report

Merging #1530 (f052f45) into main (530a4a0) will decrease coverage by 30.80%.
The diff coverage is 4.54%.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
@@             Coverage Diff             @@
##             main    #1530       +/-   ##
===========================================
- Coverage   75.03%   44.23%   -30.80%
===========================================
  Files         372      372
  Lines       43610    43692       +82
===========================================
- Hits        32723    19329    -13394
- Misses      10887    24363    +13476

Flag
Coverage Δ

integration-tests
17.87% <0.00%> (-3.58%)
:arrow_down:

unit-tests
37.30% <4.54%> (-32.29%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

 Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Does this close https://github.com/paradigmxyz/reth/issues/1489? If so, please mark it as such by typing "Closes #1489" or similar in the PR description

Does this close https://github.com/paradigmxyz/reth/issues/1489? If so, please mark it as such by typing "Closes https://github.com/paradigmxyz/reth/issues/1489" or similar in the PR description

Yep, just added


:tada: dynamic heatmap background

No need for configuration, the map for the background of the heatmap is adjusted from the coordinates in the TCX or GPX file.

solves #28


bug: pnpm dev -F tauri-app Not work!

Describe the bug
Hi Again Arash :)
i hope you doing well
i read your project new document and also read tauri document to install pre requirements of running application i install all step and also run pnpm install and install all package.
but when run command to launch development mode, it not run and i got this error,
how to solve this and run application

To reproduce
No response
Additional information
No response

Hi, Thanks!
Looks like you don't have tauri-cli installed in cargo.
Just simply run this command in the root directory of the project:
cargo install tauri-cli --version "1.5.10" --locked


make error tips optional

Per popular request this design may get in the way of people's working flow. Making it optional.

May you publish a new version to Package Control as well? I cloned the repo, but maybe not all users will do it.


add disabled property to input

Hi, I've found an issues about the possibility to add a disabled property to the input.
Please make further test and if you like, merge in the main branch.
Regards

Codecov Report

Merging #21 (73c72f1) into master (9fd2efa) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #21   +/-   ##
=======================================
  Coverage   98.05%   98.05%
=======================================
  Files           4        4
  Lines         154      154
  Branches       42       42
=======================================
  Hits          151      151
  Misses          3        3

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

I think there are some things that need to be considered:

Styling to make the field look disabled
Not letting the date picker be open while it's disabled
Maybe disabling the DatePicker too, not just the DateInput


Using matplotlib on WSLg

Environment
Windows build number: Microsoft Windows [Version 10.0.22499.1010]
Your Distribution version: Ubuntu 20.04
Your WSLg version: 1.0.27

Steps to reproduce
Try to show a plot using matplotlib
Expected behavior
A window opens showing the graph.
Actual behavior
A windows opens but you can see it only in the taskbar.

Can you provide a command or example to use that program?

First install the matplotlib python module using pip install matplotlib and then run this script.
Running this script on a normal environmnet (like on windows or a native linux installation) should show the window in the attached image.
Said script: (plt.show() is the command that should trigger the window)
import matplotlib.pyplot as plt
import numpy as np
t = np.arange(0.0, 2.0, 0.01)
s = 1 + np.sin(2*np.pi*t)
plt.plot(t, s)
plt.show()

In my case, it works after installing python3-tk. At first, there was a warning like this:
UserWarning: Matplotlib is currently using agg, which is a non-GUI backend, so cannot show the figure.

This answer from StackOverflow helped https://stackoverflow.com/a/56675620/8928481. I installed python3-tk package and it works. BTW, I am using Windows 10 and the WSLg from Windows Store Preview app.
$ cat /mnt/wsl/versions.txt
WSLg ( x86_64 ): 1.0.29+Branch.main.Sha.2e87e194b507f0ced0b08f715096a35cf9f51498
Mariner: VERSION="1.0.20210224"
FreeRDP: b05321cd4e6a862aef76163a69db4e1910245736
weston: b49e1942ac6242c9dd2118fbd7072e1692f49208
pulseaudio: 2f0f0b8c3872780f15e275fc12899f4564f01bd5
mesa:


Is there any manual for using the library with google?

I tried to use the library with google, but I din not find any manul and the readme has few information and poor documentation. Do someone have any more detailed manual? How does it work and how can I customize authentication?

@eliasadriano2010 did you find an answer to this?

A solution was presented in following link: https://www.toptal.com/django/integrate-oauth-2-into-django-drf-back-end. But, I got it like the presented in the original manual using bearer header also. You need to set the parameter in Authentication Backends. Next config presents how to use google, facebook and github:
AUTHENTICATION_BACKENDS = (
'social_core.backends.google.GoogleOAuth2',
'social_core.backends.facebook.FacebookOAuth2',
'social_core.backends.github.GithubOAuth2',
'django.contrib.auth.backends.ModelBackend',
)
Do not forget setting the permission classes:
REST_FRAMEWORK = {
#enables authentication in all endpoints
'DEFAULT_PERMISSION_CLASSES': (
'rest_framework.permissions.IsAuthenticated',
),
# previous config is enough, but I am also using native token auth
'DEFAULT_AUTHENTICATION_CLASSES': (
'rest_framework.authentication.TokenAuthentication',
),
}
Best regards!

do you have complete example ?

do you have complete example ?

In the link I've sent has an example. You only need to  follow the tutorial.


Improved code to getEyeHeight

Description
This PR's updates the current logic to get eye height, to match the correct values.
The entities width, height and eye height were validated against Minecraft Wiki and Mojang Source code (net.minecraft.world.entity.EntityType).
Checklist
The following items should be checked before the pull request can be merged.

[X] Code follows existing style.
[X] Unit tests added (if applicable).

Sorry for the PR size, but there's a lot of entities in the project.
I tried to split de work in commit to make it easier to review.

I know you put a lot of work filling in those json files (I wasted one day on the initial commit), would it be possible to add this to metaminer?

I will try to add it there, but i will need some time to get used to this.
Is it possible to access Mojang classes directly from there? To do this i mined in paper project and had to access mojang EntityType.
Is it possible to do it in MetaMiner? If so, i can implement it, i still have the code for it.

I will try to add it there, but i will need some time to get used to this. Is it possible to access Mojang classes directly from there? To do this i mined in paper project and had to access mojang EntityType.
Is it possible to do it in MetaMiner? If so, i can implement it, i still have the code for it.

It's possible, i did a quick test to copy and paste the code from paper directly to it and it worked.
I'll improve the code and open a PR for it.

At the moment it have it working for the default state only. It's really hard to automate this steps for other sub-types such as baby, small, medium, big, ...
The main problem is that this values are not stored in the entity type. They are stored in the entity, in BABY_DIMENSIONS and BABY_SCALE. I tried to use reflection to retrieve this information, but mapping the entity type to the real entity in the Mojang has been a difficult task.
Should I open a PR as draft containing the current code?

It's really hard to automate this steps for other sub-types such as baby, small, medium, big

Ah, I can give it a go when I have the time then. Could probably be done using reflection

Should I open a PR as draft containing the current code?

Sure, do that


feat(test): fix syntax error in compatibility tests

Description
Fixing syntax error (typo) in compatibility tests.
Checklist

[ ] Investigated and inspected CI test results
[ ] Unit test and regression tests added
[ ] Evaluated and added CHANGELOG entry if required
[ ] Determined and documented upgrade steps
[ ] Documented user facing changes (create PR based on openshift/openshift-docs and merge into rhacs-docs)

If any of these don't apply, please comment below.
Testing Performed
Here I tell how I validated my change

[ ] Compatibility tests passing here

Reminder for reviewers
In addition to reviewing code here, reviewers must also review testing and request further testing in case the
performed one does not seem sufficient. As a reviewer, you must not approve the change until you understand the
performed testing and you are satisfied with it.

/test ?

/test gke-version-compatibility-tests


Improve packaging

Currently only raw binaries are distributed. This is not ideal especially for MacOS and Linux. It would be nice to package the app properly.

Was a mistake in my derivation.
If I were to contribute flake.nix to the repo, would such a PR be accepted? I’m willing to support it.
Originally posted by @GoldsteinE in https://github.com/Kl4rry/simp/issues/11#issuecomment-1231498023
I happily accept help with package/platform/distro related stuff on other platforms then the ones I use as long as I don't have to maintain them as it is impossible for me to do so.

#13 Adds a flake for Nix (developer environment + package).

Adding a desktop entry for linux would be nice.

Perhaps cargo-binstall could be an option: https://github.com/cargo-bins/cargo-binstall/blob/main/SUPPORT.md

It is effectively cargo install for binaries, yes. Given the amount of dependencies for this project though does mean that you save some time and power installing it rather than having to compile it. I'd like more projects, especially the bigger ones, to support cargo-binstall.

Yes I agree being able to install binaries would be nice considering both the long compile times and the problems with dependancies we have had.

I can recommend cargo-bundle to build a MacOS app bundle (I am building a simple image viewer with rudimentary mac support).
 Here is my info.plist:
https://github.com/woelper/oculante/blob/master/res/info.plist

I do not own a Mac computer and therefore is unable to debug bundling for Mac.


From/to unified pair name/exchange specific name

I'd like in unified API having two functions which map from unified names (e.g. BTC/EUR) to market specific name and back (e.g. BTCEUR for kraken IIRC).
This will help me test your library further and detect subtle errors.
I recommend names:
to_market_api_name(self, symbol)
to_unified_name(self, symbol)

They should throw exception of not found in the exchange load products. It should use cached load_products()
This functionality is implemented somewhere anyhow, it's only about exposing it.

Furthemore, this should simplify unit tests for pair name conversion (since I think there are still issues with this)

Hey! Thanks again for feedback, it's very constructive. Sure, I will expose the mapping.
In fact one half of that mapping (symbol → product id → symbol) is already there:
symbol = 'BTC/USD'
print (market.products[symbol]['id']) # print product id by symbol
print (market.productId (symbol)) # same as previous line
product = market.product (symbol) get a full product dictionary by symbol
print (product['id']) # same as previous output

I will expose the other half, sure! Will upload this soon.

Though, the existing mapping does not throw errors, but returns the input string untouched. I will update that part of the interface anyway.

This feature will soon be ready, btw, Product Ids And Symbols. The Manual is frequently updated, I'm adding more and more details into it, so it's worth revisiting from time to time. I will also update it and add the two methods proposed here very soon.

I've uploaded the interface to products, product symbols and product ids.
Here's how to use it:
import ccxt
k = ccxt.kraken ()
k.load_products ()

# symbol → product
k.products['BTC/USD']

# id → product
k.products_by_id['XXRPZUSD']

# symbol → id
k.products['BTC/USD']['id']

# id → symbol
k.products_by_id['XXRPZUSD']['symbol']

No functions, basic dictionaries indexed by appropriate fields. The are cached and can be force-reloaded if necessary.
You will get a user-land exception if you access non-existent keys in these dicts.
I hope the above solution simplifies the usage, closing this issue down now. Will soon add that to the Manual.

Update: It's in The Manual now.


IngressRoute that references a secret which has not been delegated is deemed valid

What steps did you take and what happened:

Install contour:

kubectl apply -f https://raw.githubusercontent.com/heptio/contour/v0.14.2/examples/render/deployment-rbac.yaml

Create a TLS secret in the heptio-contour namespace

Create an IngressRoute outside of the heptio-contour namespace. In its TLS block, the IngressRoute should reference the secret that was created in the heptio-contour namespace. For example:

apiVersion: contour.heptio.com/v1beta1
kind: IngressRoute
metadata:
  name: app-with-tls-delegation
  namespace: app-with-tls-delegation
spec:
  virtualhost:
    fqdn: app-with-tls-delegation.<IP_ADDRESS>.nip.io
    tls:
      secretName: heptio-contour/ssl-cert
  routes:
  - match: '/'
    services:
    - name: app-with-tls-delegation
      port: 80

Observe IngressRoute status:

$ kubectl get ingressroute
NAME                      FQDN                                       TLS SECRET                FIRST ROUTE   STATUS   STATUS DESCRIPTION
app-with-tls-delegation   app-with-tls-delegation.<IP_ADDRESS>.nip.io   heptio-contour/ssl-cert   /             valid    valid IngressRoute

What did you expect to happen:
The IngressRoute should be deemed invalid because the referenced TLS certificate is in another namespace, and there is no TLSCertificateDelegation resource granting access to the cert.
Anything else you would like to add:
By inspecting the DAG, it seems like the HTTPS listener is not setup:
digraph DAG {
rankdir="LR"
"0xc0006671a0" [shape=record, label="{listener|:80}"]
"0xc0006671c0" [shape=record, label="{http://app-with-tls-delegation.<IP_ADDRESS>.nip.io}"]
"0xc0000afc70" [shape=record, label="{prefix|/}"]
"0xc000500dc0" [shape=record, label="{cluster|{app-with-tls-delegation/app-with-ef7df1/80/da39a3ee5e|weight 0}}"]
"0xc00012d9d0" [shape=record, label="{httpservice|app-with-tls-delegation/app-with-tls-delegation:80}"]
"0xc000500dc0" -> "0xc00012d9d0"
"0xc0000afc70" -> "0xc000500dc0"
"0xc0006671c0" -> "0xc0000afc70"
"0xc0006671a0" -> "0xc0006671c0"
}

Once the correct TLSCertificateDelegation resource is created, the HTTPS listener is added:
digraph DAG {
rankdir="LR"
"0xc00065afa0" [shape=record, label="{listener|:443}"]
"0xc0007bafc0" [shape=record, label="{https://app-with-tls-delegation.<IP_ADDRESS>.nip.io}"]
"0xc00015cf00" [shape=record, label="{prefix|/}"]
"0xc000500900" [shape=record, label="{cluster|{app-with-tls-delegation/app-with-ef7df1/80/da39a3ee5e|weight 0}}"]
"0xc0002d8a80" [shape=record, label="{httpservice|app-with-tls-delegation/app-with-tls-delegation:80}"]
"0xc000500900" -> "0xc0002d8a80"
"0xc00015cf00" -> "0xc000500900"
"0xc0007bafc0" -> "0xc00015cf00"
"0xc0001600c8" [shape=record, label="{secret|heptio-contour/ssl-cert}"]
"0xc0007bafc0" -> "0xc0001600c8"
"0xc00065afa0" -> "0xc0007bafc0"
"0xc00065afe0" [shape=record, label="{listener|:80}"]
"0xc00065b020" [shape=record, label="{http://app-with-tls-delegation.<IP_ADDRESS>.nip.io}"]
"0xc00065b020" -> "0xc00015cf00"
"0xc00065afe0" -> "0xc00065b020"
}

Environment:

Contour version: v0.14.2
Kubernetes version: (use kubectl version):

Client Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.0", GitCommit:"641856db18352033a0d96dbc99153fa3b27298e5", GitTreeState:"clean", BuildDate:"2019-03-26T00:04:52Z", GoVersion:"go1.12.1", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.0", GitCommit:"e8462b5b5dc2584fdcd18e6bcfe9f1e4d970a529", GitTreeState:"clean", BuildDate:"2019-06-25T23:41:27Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"}

@alexbrand could you please try to reproduce this with 0.15. I was unable to reproduce the fault
(~/src/github.com/heptio/contour) % k -n app-with-tls-delegation describe ingressroute
Name:         app-with-tls-delegation
Namespace:    app-with-tls-delegation
Labels:       <none>
Annotations:  kubectl.kubernetes.io/last-applied-configuration:
                {"apiVersion":"contour.heptio.com/v1beta1","kind":"IngressRoute","metadata":{"annotations":{},"name":"app-with-tls-delegation","namespace"...
API Version:  contour.heptio.com/v1beta1
Kind:         IngressRoute
Metadata:
  Creation Timestamp:  2019-08-30T02:04:50Z
  Generation:          1
  Resource Version:    174455827
  Self Link:           /apis/contour.heptio.com/v1beta1/namespaces/app-with-tls-delegation/ingressroutes/app-with-tls-delegation
  UID:                 8d09b6ee-caca-11e9-a775-42010a980092
Spec:
  Routes:
    Match:  /
    Services:
      Name:  app-with-tls-delegation
      Port:  80
  Virtualhost:
    Fqdn:  app-with-tls-delegation.<IP_ADDRESS>.nip.io
    Tls:
      Secret Name:  heptio-contour/ssl-cert
Status:
  Current Status:  invalid
  Description:     TLS Secret [heptio-contour/ssl-cert] not found or is malformed
Events:            <none>
(~/src/github.com/heptio/contour) % k -n app-with-tls-delegation get ingressroute
NAME                      FQDN                                       TLS SECRET                FIRST ROUTE   STATUS    STATUS DESCRIPTION
app-with-tls-delegation   app-with-tls-delegation.<IP_ADDRESS>.nip.io   heptio-contour/ssl-cert   /             invalid   TLS Secret [heptio-contour/ssl-cert] not found or is malformed

Did you create the heptio-contour/ssl-cert? I am still able to repro with 0.15.
Looking through the e2e tests, it seems like this is the expected behavior though.
I am specifically referring to this assertion: https://github.com/heptio/contour/blob/master/internal/e2e/lds_test.go#L1525, which expects there to be an HTTP listener but not an HTTPS listener when there is no TLS Delegation.
I would argue that if the IngressRoute is trying to use a TLS cert that has not been delegated, neither an HTTP nor an HTTPS listener should be created. Thoughts?
I have a commit locally that fixes this issue. Happy to open a PR if we agree this should be the desired behavior.
// internal/dag/builder.go
@@ -401,12 +401,21 @@ func (b *Builder) computeIngressRoutes() {
                        // attach secrets to TLS enabled vhosts
                        m := splitSecret(tls.SecretName, ir.Namespace)
                        sec := b.lookupSecret(m, validSecret)
-                       if sec != nil && b.delegationPermitted(m, ir.Namespace) {
+                       permittedByDelegation := b.delegationPermitted(m, ir.Namespace)
+
+                       // ensure the ingressroute is allowed to use the given secret
+                       if sec != nil && !permittedByDelegation {
+                               b.setStatus(Status{Object: ir, Status: StatusInvalid, Description: fmt.Sprintf("TLS Secret [%s] is not accessible from this namespace", tls.SecretName)})
+                               continue
+                       }
+
+                       if sec != nil && permittedByDelegation {
                                svhost := b.lookupSecureVirtualHost(host)
                                svhost.Secret = sec
                                svhost.MinProtoVersion = MinProtoVersion(ir.Spec.VirtualHost.TLS.MinimumProtocolVersion)
                                enforceTLS = true
                        }


WIP - keycloak

in order to add ueberauth_oidcc I had to update telemetry_poller which triggered updating phoenix.
This then tries to rip out cognito and add in keycloak. WIP
I realised I think we should use a unique role name for this app so that there is no chance of accidentally giving access to users of different apps here because they have a role with the same name in keycloak
the implementation is broadly working just need to fix tests now.

@Adzz  I made some changes here, mostly to fix some warnings that came up failing the build and test step and making sure it integrates well with Keycloak DEV (looks like it worked well for me). Last steps here are to fix up the tests happy to work on that today

Hey no worry the tests are a nightmare so I can crack that out now, thanks for the contribution though!

I'm still reviewing some other stuff in this PR, but it looks like its failing and I have one important thing that I think will need to be done to successfully work in deployed environments: can you add in ca-certificates as a dependency in this project's dockerfile?
See:
https://github.com/mbta/alerts_concierge/blob/d7b0b431bb2e25864e008bb17dbaaa3367387bf6/Dockerfile#L59
And the error that happens if you do not do this: https://www.notion.so/mbta-downtown-crossing/Implementing-a-Keycloak-integration-ebf669208d464986b4989294891b1d8e?pvs=4#49283e6e8bc0478e965a416f8cc5e8d8
This happened while I was testing APA and suspect it would happen here too.


Secure Helm: separate ansible tasks

First step to #938 is creating ansible tasks and templates for Helm

Wish we could get CI to trigger on forked merges.
Can you potentially change the base branch to master-merge-from-forks, and then we can merge it in from there?


Helm vs breaking change migration

I got strange issue and can't find good solution.
I have two chart versions:

0.1:

Deployment with gRPC service listen port 8443
Service with ClusterIP map port 8443 to 443

0.2:

Deployment with gRPC service listen port 443
Service without ClusterIP (headless) for port 443
Service used outside chart by name.

In this way there are no way to create migration without downtime: ClusterIP field is immutable and I must recreate Service.
On upgrade 0.1 -> 0.2 I have:

Deployment is upgraded successfully and listen port 443;
Service upgrade failed and try to use container port 8443.
As result I have downtime until somebody resolve broken update.

In this situation I want to have way to fail on 0.1 -> 0.2 upgrade with some human friendly message after chart render, but before some changes happens.

Your assumption is correct. There is no way to upgrade from 0.1 to 0.2 in this case, as Kubernetes prevents upgrades to immutable fields such as the clusterIP field in a Service.
Your only recourse is to delete the service prior to the upgrade.

In this situation I want to have way to fail on 0.1 -> 0.2 upgrade with some human friendly message after chart render, but before some changes happens.

You could use the fail function to fail early.
https://helm.sh/docs/chart_template_guide/function_list/#fail

Your assumption is correct. There is no way to upgrade from 0.1 to 0.2 in this case, as Kubernetes prevents upgrades to immutable fields such as the clusterIP field in a Service.
Your only recourse is to delete the service prior to the upgrade.

In this situation I want to have way to fail on 0.1 -> 0.2 upgrade with some human friendly message after chart render, but before some changes happens.

You could use the fail function to fail early.
https://helm.sh/docs/chart_template_guide/function_list/#fail

I can't use fail function, because I can't get previously installed version number.
So I can't tell upgrade 0.1 -> 0.2 from upgrade 0.2 -> 0.2.

I can't use fail function, because I can't get previously installed version number.
So I can't tell upgrade 0.1 -> 0.2 from upgrade 0.2 -> 0.2.

For your information: --force flag can't apply clusterIP chage by recreate Service.

For your information: --force flag can't apply clusterIP chage by recreate Service.


Add filters to memo

Scalar expressions added to memo along with scalar properties, expression ids, filter closures. Goal here is equivalent behavior to before, just with filters represented differently. Filter organization mostly mirrors the plan package, except scalar and relational expressions are both represented as expression groups here. Done in a rush, still back and forth on whether there should be an interface there.
Additionally:
scalar expressions added to memo along with scalar properties, expression id
rewrites join planning and costing to use bitset representations of filters
refactors codegen so definition files are yaml, source is compiled independently from target code
The organization is bit wonky b/c this should be using my name resolution symbol tables, and the entire tree should be memoized not just the join tree (used temporary solutions for the problems created by both of these).
Re: https://github.com/dolthub/dolt/issues/5993

see https://github.com/dolthub/go-mysql-server/pull/1797 for comments


[e2e] [FEATURE] Support Volume Clone

What's the test to develop? Please describe
A clear and concise description of what the test you want to develop.
Prerequisite and dependency of test
Any prerequisite environment and pre-condition required for this test.
Provide test case dependency here if any.
Describe the items of the test development (DoD, definition of done) you'd like

Please use a task list for items on a separate line with a clickable checkbox https://docs.github.com/en/issues/tracking-your-work-with-issues/about-task-lists

[ ] item 1

Additional context
Add any other context or screenshots about the test request here.
related issue: https://github.com/harvester/harvester/issues/2293

Please update the project in the ticket.

close as the test cases implemented and work as expected in daily testing for days.


Please, instruct in the docs that executable should be installed in the PATH

I encountered a:
ghc: could not execute: tasty-discover

build reports.
tasty-discover is in the *.cabal dependencies.
tasty-discover gets downloaded and build by Cabal during project build.
But it was throwing ghc: could not execute: tasty-discover error, websearch pointed-out there is a number of reports to different projects of this, and all what I read about it - is that it is a "project internal issue".
Because I do not have prior experience with tasty-discover I looked at the docs - and they indicated to me that nothing is wrong, that there is no need to install anything.
Only when a fellow member of a team suggested that tasty-discover should be installed before the build with something like cabal v2-install tasty-discover - only by help and trial and error that it needs installation was discovered.
This was encountered while doing CI/tests for the project where I first encountered tasty-discover test suite (https://github.com/haskell-nix/hnix-store/pull/88).
Please, add information that people need to install the executable, where it would be appropriate to instruct.

It was pointed-out to me that such cases are solved by:
  build-tool-depends:
    tasty-discover:tasty-discover

That is how it works properly.
So, please, add this to the documentation.

Yeah, I just hit this as well. The tl;dr is that the docs should definitely mention build-tool-depends - which is preferable to installing globally as cabal will manage the version properly.

Yes, as was written
  build-tool-depends:
    tasty-discover:tasty-discover

That would also allow for .cabal description to be converted into Nix language properly. To find a proper solution I arrived up to the main Nixpkgs Haskell maintainer giving me the proper answer, while that just should be in docs.

Thanks for the suggestion.

@newhoggy
Thank you from the community.
I was rambling here a little, that is the way I am.
Thank you again, now setup flow for people would be smooth.


Make cloudpickle as default pickler

TEST PR for making cloudpickle the default pickler

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

Combiner tests that are failing.
FAILED apache_beam/transforms/combinefn_lifecycle_test.py::LocalCombineFnLifecycleTest_0::test_combine
FAILED apache_beam/transforms/combinefn_lifecycle_test.py::LocalCombineFnLifecycleTest_1::test_combine


feat: require and then remove karpenter startup taint

Fixes https://github.com/kubernetes-sigs/karpenter/issues/1049
Description
Based on implementation discussion, it is required that userdata scripts for starting kubelet include karpenter.sh/unregistered=true:NoExecute as part of the --register-with-taints flag.
How was this change tested?
Unit testing and local aws/karpenter-provider-aws deployment. This will be ship with https://github.com/aws/karpenter-provider-aws/pull/6388/
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

/reopen

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

25 of 29 (86.21%)  changed or added relevant lines in 4 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.05%) to 78.358%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

pkg/controllers/nodeclaim/lifecycle/registration.go
19
23
82.61%

Totals

Change from base Build<PHONE_NUMBER>:
0.05%

Covered Lines:
8740

Relevant Lines:
11154

💛  - Coveralls

/ok-to-test

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Warning: This coverage report may be inaccurate.
This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

For more information on this, see Tracking coverage changes with pull request builds.
To avoid this issue with future PRs, see these Recommended CI Configurations.
For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

18 of 22 (81.82%)  changed or added relevant lines in 3 files are covered.
174 unchanged lines in 11 files lost coverage.
Overall coverage increased (+0.3%) to 78.646%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

pkg/controllers/nodeclaim/lifecycle/registration.go
9
13
69.23%

Files with Coverage Reduction
New Missed Lines
%

pkg/controllers/nodeclaim/lifecycle/registration.go
1
82.54%

pkg/test/cachesyncingclient.go
2
82.29%

pkg/controllers/provisioning/scheduling/nodeclaim.go
2
89.13%

pkg/apis/v1/nodepool_validation.go
2
89.47%

pkg/controllers/node/termination/terminator/eviction.go
2
89.09%

pkg/controllers/nodeclaim/disruption/controller.go
3
54.17%

pkg/apis/v1/nodeclaim_validation.go
7
88.64%

pkg/operator/operator.go
15
4.41%

pkg/apis/v1/nodepool.go
20
67.86%

pkg/controllers/state/cluster.go
28
91.77%

Totals

Change from base Build<PHONE_NUMBER>:
0.3%

Covered Lines:
8596

Relevant Lines:
10930

💛  - Coveralls

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

18 of 22 (81.82%)  changed or added relevant lines in 3 files are covered.
8 unchanged lines in 4 files lost coverage.
Overall coverage increased (+0.01%) to 78.628%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

pkg/controllers/nodeclaim/lifecycle/registration.go
9
13
69.23%

Files with Coverage Reduction
New Missed Lines
%

pkg/controllers/nodeclaim/lifecycle/registration.go
1
82.54%

pkg/test/cachesyncingclient.go
2
82.29%

pkg/controllers/node/termination/terminator/eviction.go
2
89.09%

pkg/controllers/nodeclaim/disruption/controller.go
3
54.17%

Totals

Change from base Build<PHONE_NUMBER>:
0.01%

Covered Lines:
8594

Relevant Lines:
10930

💛  - Coveralls

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

19 of 23 (82.61%)  changed or added relevant lines in 4 files are covered.
10 unchanged lines in 5 files lost coverage.
Overall coverage decreased (-0.07%) to 78.545%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

pkg/controllers/nodeclaim/lifecycle/registration.go
9
13
69.23%

Files with Coverage Reduction
New Missed Lines
%

pkg/controllers/nodeclaim/lifecycle/registration.go
1
82.54%

pkg/test/expectations/expectations.go
2
93.69%

pkg/test/cachesyncingclient.go
2
82.29%

pkg/controllers/node/termination/terminator/eviction.go
2
89.09%

pkg/controllers/nodeclaim/disruption/controller.go
3
54.17%

Totals

Change from base Build<PHONE_NUMBER>:
-0.07%

Covered Lines:
8585

Relevant Lines:
10930

💛  - Coveralls

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

19 of 23 (82.61%)  changed or added relevant lines in 4 files are covered.
8 unchanged lines in 4 files lost coverage.
Overall coverage increased (+0.03%) to 78.646%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

pkg/controllers/nodeclaim/lifecycle/registration.go
9
13
69.23%

Files with Coverage Reduction
New Missed Lines
%

pkg/controllers/nodeclaim/lifecycle/registration.go
1
82.54%

pkg/test/cachesyncingclient.go
2
82.29%

pkg/controllers/node/termination/terminator/eviction.go
2
89.09%

pkg/controllers/nodeclaim/disruption/controller.go
3
54.17%

Totals

Change from base Build<PHONE_NUMBER>:
0.03%

Covered Lines:
8596

Relevant Lines:
10930

💛  - Coveralls

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

19 of 23 (82.61%)  changed or added relevant lines in 4 files are covered.
8 unchanged lines in 4 files lost coverage.
Overall coverage increased (+0.05%) to 78.664%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

pkg/controllers/nodeclaim/lifecycle/registration.go
9
13
69.23%

Files with Coverage Reduction
New Missed Lines
%

pkg/controllers/nodeclaim/lifecycle/registration.go
1
82.54%

pkg/test/cachesyncingclient.go
2
82.29%

pkg/controllers/provisioning/scheduling/nodeclaim.go
2
89.13%

pkg/controllers/nodeclaim/disruption/controller.go
3
54.17%

Totals

Change from base Build<PHONE_NUMBER>:
0.05%

Covered Lines:
8598

Relevant Lines:
10930

💛  - Coveralls

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

19 of 23 (82.61%)  changed or added relevant lines in 4 files are covered.
6 unchanged lines in 3 files lost coverage.
Overall coverage increased (+0.05%) to 78.664%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

pkg/controllers/nodeclaim/lifecycle/registration.go
9
13
69.23%

Files with Coverage Reduction
New Missed Lines
%

pkg/controllers/nodeclaim/lifecycle/registration.go
1
82.54%

pkg/test/cachesyncingclient.go
2
82.29%

pkg/controllers/nodeclaim/disruption/controller.go
3
54.17%

Totals

Change from base Build<PHONE_NUMBER>:
0.05%

Covered Lines:
8598

Relevant Lines:
10930

💛  - Coveralls

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Warning: This coverage report may be inaccurate.
This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

For more information on this, see Tracking coverage changes with pull request builds.
To avoid this issue with future PRs, see these Recommended CI Configurations.
For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

16 of 20 (80.0%)  changed or added relevant lines in 4 files are covered.
38 unchanged lines in 4 files lost coverage.
Overall coverage increased (+0.07%) to 78.68%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

pkg/controllers/nodeclaim/lifecycle/registration.go
8
12
66.67%

Files with Coverage Reduction
New Missed Lines
%

pkg/controllers/nodeclaim/lifecycle/registration.go
1
82.26%

pkg/test/cachesyncingclient.go
2
82.29%

pkg/controllers/nodeclaim/disruption/controller.go
3
54.17%

pkg/controllers/provisioning/provisioner.go
32
79.25%

Totals

Change from base Build<PHONE_NUMBER>:
0.07%

Covered Lines:
8606

Relevant Lines:
10938

💛  - Coveralls

/lgtm

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Warning: This coverage report may be inaccurate.
This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

For more information on this, see Tracking coverage changes with pull request builds.
To avoid this issue with future PRs, see these Recommended CI Configurations.
For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

17 of 19 (89.47%)  changed or added relevant lines in 4 files are covered.
100 unchanged lines in 9 files lost coverage.
Overall coverage increased (+0.1%) to 78.754%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

pkg/controllers/nodeclaim/lifecycle/registration.go
11
13
84.62%

Files with Coverage Reduction
New Missed Lines
%

pkg/utils/disruption/disruption.go
1
93.75%

pkg/controllers/nodeclaim/lifecycle/registration.go
1
85.71%

pkg/test/expectations/expectations.go
2
93.7%

pkg/scheduling/requirements.go
2
98.01%

pkg/controllers/metrics/pod/controller.go
15
77.42%

pkg/apis/v1/nodepool.go
15
67.86%

pkg/controllers/nodepool/counter/controller.go
15
62.07%

pkg/controllers/nodeclaim/termination/controller.go
17
65.0%

pkg/controllers/provisioning/provisioner.go
32
79.25%

Totals

Change from base Build<PHONE_NUMBER>:
0.1%

Covered Lines:
8611

Relevant Lines:
10934

💛  - Coveralls


Space_X_Travellers

What does this PR do?
This PR is raised to implement the following features: 👍 :

[x] Add the functionality of the two reducers of the rockets and missions 🪄
[x] Fetch the data from the API for the rockets and missions.👌
[x] Use the react-router dom for navigating from one page to another.💯
[x] Render the reversed rockets from the state.⤵
[x] Filter the rockets in the state👌

Description of the task to be completed 👎

[x] Render the reversed rockets from the state when the user clicks the button the reversed will go to the state💯
[x] Use the basic CSS style for styling the components.⤵
[x] Use useSelector for fetching the state and displaying it.
[x] Filter the rockets in the state and display it in the profile👌

How should it be tested
Clone the repository
git clone https://github.com/osamaashraf6/Space-X-React-Group

Navigate to the folder
cd Space-X-React-Group

Checkout the branch

git checkout dev

#### Thanks for reviewing my PR 😊

Thanks @jerryowusu 🙏 for reviewing my pull request I appreciate your review 💯

Thanks again for reviewing my PR  and have a good day😊

Thanks @omarsalem7 🙏 for reviewing my pull request I appreciate your review 💯

Thanks again for reviewing my PR  and have a good day😊


Store size when call store_bytes

For ckb, these two things are usually done together.

I totally understand that moving fast is a requirement, and we want to merge this asap so as not to block work.
But when you have a time, please fill in tests that actually assert that the addresses for holding size are actually updated. Right now this change only ensures compiling works, but do not assert the actual behavior.


When I use recording rules, the dat would miss sometimes.

What did you do?
I predefined a rule in Prometheus, like this.
sum(increase(remote_qps{status='success'}[5m])) by (method, region) / sum(increase(remote_qps[5m])) by (method, region)
What did you expect to see?
I think I should get continuous graphics when I use graph function in the Prometheus web page
What did you see instead? Under which circumstances?
But the data flow is Intermittent.

It is normal when I use the raw query in the Prometheus web page.


How to use neptune-gremlin-client via gradle

Is neptune-gremlin-client available to use via Gradle as a dependency?
I looked up and could not find it in   https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk-neptune

We're also looking for this @iansrobinson?

I could do that, but it is a catch-up game, new versions need to be always copied and tracked.

Would it be useful to have them published to Maven?

Yes, they would definitely be useful if available on maven.

@beebs-systap Yes that would be super useful for us!

Now published to Maven: https://search.maven.org/artifact/software.amazon.neptune/gremlin-client


THORN-2117 upgrade wildfly-nosql to 1.0.0.Final

https://issues.jboss.org/browse/THORN-2117

Windows failure doesn't look related to this pr:
org.eclipse.microprofile.config.tck.ConfigProviderTest.testEnvironmentConfigSource (from TestSuite)
Failing for the past 5 builds (Since Unstable#539 )
Took 81 ms.
Error Message

The property MY_STRING_PROPERTY did not have the expected value. expected [haha] but found [woohoo]

Stacktrace

java.lang.AssertionError: The property MY_STRING_PROPERTY did not have the expected value. expected [haha] but found [woohoo]
	at org.eclipse.microprofile.config.tck.ConfigProviderTest.testEnvironmentConfigSource(ConfigProviderTest.java:98)

retest this please.


MNIST traning on TFoS takes much longer time than on native TensorFlow

I have a Spark cluster, each of them has 64 cores and 256 G RAM, I tried run MNIST training with TFoS and TF only, the results was out of my expectation.
The training on TFoS took took 25m53s with 100 epochs, here is the script:
spark-submit
--master yarn
--deploy-mode cluster
--num-executors 3
--executor-memory 32G
--py-files TensorFlowOnSpark/tfspark.zip,TensorFlowOnSpark/examples/mnist/spark/mnist_dist.py
--conf spark.dynamicAllocation.enabled=false
--conf spark.yarn.maxAppAttempts=1
--archives hdfs:///user/spark/Python.zip#Python
TensorFlowOnSpark/examples/mnist/spark/mnist_spark.py
--images mnist/csv/train/images
--labels mnist/csv/train/labels
--mode train
--model mnist_model
--epochs 10

However it only took 834s for training with TF only with 1000 epochs.
I didn't have chance to investigate to TFoS, is the performance related to training data size and some other elements?

@leewyang , thanks for your reply. Actually the non-distributed TensorFlow running took only 834s.
I checked the logic of TFoS and see it is implemented base on TensorFlow distributed https://www.tensorflow.org/deploy/distributed. It parallellizes the input data to be more partitions according to the number of executors, on each of the executor it creates  PS process or WORKER. The training and inference process should be the same as TensorFlow distributed via gRPC.
One of my colleague is testing with TensorFlow distributed, it has better performance than non-distributed TF. So I don't think it has so much trade-off to run MNIST training even though it is a tiny dataset, because it runs much the same as TensorFlow distributed.
Did you TFoS team ever run performance testing against Non-Distributed TensorFlow node and TensorFlow distributed on different dataset? Performance concerns much for me to choose the TF distribution solution.
Thanks a lot.

@zxsimple our focus is mainly to enable running distributed TensorFlow on Hadoop/Spark clusters.  If you're interested in comparing distributed TensorFlow against non-distributed TensorFlow, I think that would be something that you can take up with the TensorFlow team.

@leewyang Maybe I didn't state clearly, actually I compared TFoS with non-distributed TensorFlow, not with distributed TensorFlow . The simply ran with MNIST resulted big performance diff,  834s on non-distributed TensorFlow(1000 epochs) and 25m53s on TFoS(100 epochs). I read the code see you implemented TFoS based on TensorFlow distributed, it doesn't make sense it runs so poor performance on TFoS.

Hi @zxsimple, I'm basically trying to say that benchmarking a small program on a single machine against a distributed version of the same small program will almost always show that the single machine is faster.  An exaggerated example of this would be something like printing the numbers 1 to 1000.  If I run this on a single machine, it will probably finish in microseconds to milliseconds.  But if I write a distributed Spark program to print these numbers using 10 executors, you will incur all the overhead of starting Spark on distributed nodes along w/ various costs of communicating RDDs over the network.
So to re-iterate, the distributed mnist example code was not provided to demonstrate any performance gains, but rather to demonstrate the mechanics of distributing a simple program (e.g. like "printing 1 to 1000" in a distributed cluster).
So anyhow, I'd recommend two things:

Try to benchmark with a larger DNN.  Note that we demonstrated in our original blog post that the TF0.12 version of imagenet/inception scaled roughly linearly with the number of nodes (at least for the small scale clusters that we tested with).
Try to benchmark non-distributed TensorFlow against distributed TensorFlow (w/o TFoS, just to eliminate the variables).  Note that in our experiments, running with TFoS may add some slight overhead, since we're also running Spark/Hadoop/HDFS on the same nodes (vs. running distributed TF by itself).

And finally, one other thing to consider is that IF you are benchmarking on a shared Hadoop grid, that grid may be running other jobs on the same nodes as well...
Hope that explains my points better...

@leewyang Thanks, that sounds be the root cause why small dataset runs much slower on TFoS. I'll test with larger dataset and see.
Thanks.


Introduce auto-start menu

Is your feature request related to a problem? Please describe.
Currently when I want to start Jandi as mac starts, I should go to login Items in settings manually.
Describe the solution you'd like
I think introducing new menu that toggles to auto-launch the app when start mac.
Describe alternatives you've considered
I don't know much about Apple's policy for auto-lauching app, but I saw that some apps just introduce to login items menu rather than directly make app auto-launchable.

@simp7  Thank you for good suggestion!, We will support this feature. 😄

https://github.com/sindresorhus/LaunchAtLogin
It might be helpful.

@junho85 Thanks for good suggestion!

@simp7 @junho85  Thank you for your contribution. Released to 1.7 version. 🎉


Add Batching to Mistral 7B

Problem description
Customer team is interested in implementing batching for the mistral 7B model but there is an issue with context leakage for same users across different batches.
What's changed

Added batching to the mistral 7b demo by modulo wrapping input prompts of batch size 8 (batch size of up to 5 are supported in order to see issue occur)
option to print output and save to file (set to false by default)

Issue
Refer to following issue: https://github.com/tenstorrent/tt-metal/issues/11061

Started all relevant tests. When they all pass we can merge this PR 👍
[ ] - All post commit
[ ] - Nightly tests
[ ] - Demo tests
[ ] - Perf tests

Re-running Demo and perf tests after pushing some minor fixes.
I'm slightly concerned that there seems to be a new regression in main causing the mistral test with 4k KV-cache to hang 🤔


[6play] Error 404: no video formats found

Checklist

[x] I'm reporting a broken site support
[x] I've verified that I'm running youtube-dl version 2020.03.01
[x] I've checked that all provided URLs are alive and playable in a browser
[x] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[x] I've searched the bugtracker for similar issues including closed ones

Verbose log

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['https://www.rtlmost.hu/batrak-foldje-p_15803/batrak-foldje-45-resz-c_12591431', '--verbose']
[debug] Encodings: locale cp1250, fs mbcs, out cp852, pref cp1250
[debug] youtube-dl version 2020.03.01
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.18362
[debug] exe versions: none
[debug] Proxy map: {}
[6play] 12591431: Downloading JSON metadata
[6play] 12591431: Downloading webpage
[6play] 12591431: Downloading m3u8 information
WARNING: Failed to download m3u8 information: HTTP Error 404: Not Found
[6play] 12591431: Downloading MPD manifest
WARNING: Failed to download MPD manifest: HTTP Error 404: Not Found
[6play] 12591431: Downloading m3u8 information
WARNING: Failed to download m3u8 information: HTTP Error 404: Not Found
[6play] 12591431: Downloading MPD manifest
WARNING: Failed to download MPD manifest: HTTP Error 404: Not Found
[6play] 12591431: Downloading m3u8 information
WARNING: Failed to download m3u8 information: HTTP Error 404: Not Found
[6play] 12591431: Downloading MPD manifest
WARNING: Failed to download MPD manifest: HTTP Error 404: Not Found
ERROR: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpavop71lg\build\youtube_dl\YoutubeDL.py", line 797, in extract_info
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpavop71lg\build\youtube_dl\extractor\common.py", line 530, in extract
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpavop71lg\build\youtube_dl\extractor\sixplay.py", line 113, in _real_extract
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpavop71lg\build\youtube_dl\extractor\common.py", line 1327, in _sort_formats
youtube_dl.utils.ExtractorError: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

Description

Downloading newer videos from rtlmost.hu stopped working this week. Some older links still work. Tested in Sorozatok (series) menu.
Videos that are not marked with a lock icon (paid) are watchable in browser after login (bejelentkezés in top right corner<EMAIL_ADDRESS>pw: Mosttest2020

Only DRM protected playlists are served for rtlmost from 2020.03.01.


Parse requirements properly with pip

To support requirements formats like "-r" in requirements files, I replace the current parsing with pip's parse_requirements.
This will change existing terrarium keys.

If we could get a testcase on nested requirements files, this will be ready to merge in.

Hi, I am right in thinking that the functionality of the PR has been added to terrarium master and therefore this should be closed?

This functionality is available in the 2.0.0-alpha release and will be available for the full 2.0.0 release.


Linking error in example1

I'm getting this:
➜  dlangui git:(master) ✗ dub run dlangui:example1
Building package dlangui:example1 in /home/katafrakt/dev/github/dlangui/examples/example1/
WARNING: DUB package names may only contain alphanumeric characters, as well as '-' and '_', please modify the "name" field in /home/katafrakt/dev/github/dlangui/examples/example1/dub.json accordingly. You can use {"targetName": "dlangui:example1"} to keep the current executable name.
Target gl3n 1.0.1 is up to date. Use --force to rebuild.
Target dlib ~master is up to date. Use --force to rebuild.
Target derelict-util 1.9.1 is up to date. Use --force to rebuild.
Target derelict-ft 1.0.2 is up to date. Use --force to rebuild.
Target derelict-sdl2 1.9.4 is up to date. Use --force to rebuild.
Target derelict-gl3 1.0.12 is up to date. Use --force to rebuild.
Building dlangui 0.6.10 configuration "default", build type debug.
Running dmd...
Building dlangui:example1 0.6.10 configuration "application", build type debug.
Compiling using dmd...
Linking...
examples/example1/.dub/build/application-debug-linux.posix-x86_64-dmd_2067-4299F752DF5821F0F9DFF585463D2EB2/example1.o: In function `_Dmain':
/home/katafrakt/dev/github/dlangui/src/dlangui/platforms/common/platform.d:1395: undefined reference to `_D7dlangui9platforms6common8platform11DLANGUImainFAAyaZi'
collect2: error: ld returned 1 exit status
--- errorlevel 1
FAIL examples/example1/.dub/build/application-debug-linux.posix-x86_64-dmd_2067-4299F752DF5821F0F9DFF585463D2EB2/ example1 executable
Error executing command run:
dmd failed with exit code 1.

I'd really like to provide you with more details, but I'm new to all this so I don't know what information is needed ;) I'm running Arch Linux x64 with dmd 2.067. I narrowed it down to changes in commit d97cec952239cedebd055de383546f6b3a23993a. After resetting to e592462f6b0532f0af95d4d69e05162cc7484177 (one before) everything works fine.

Fixed in v0.6.15


Migration of farmers and laborers

TypicalRain
What I meant though was that when the population decline started happening that the Tuvans were almost wiped out not the Russians. :eek: Both populations in the state were going down but since only non Tuvans were immigrating into the state they were reduced to a few thousand. This also happened to local Minorities like Komi and Yakuts. Is that merely a by product of me educating those populations to much?
simsulla
Maybe it is because Tuva is a state/region capital in Russia, and thus that province will attract migrants. I will try and fix the Tuvan migrating by making framers/laborers much less likely to migrate. For very small minorities (Komi, Yakuts) dilution is hard to avoid.

I'm planning to add cultures of republics and autonomies of Russia as accepted. But it will completely stop their assimilation, which is non-realistic too.

Nice idea, a scenario of a more Federated Russia. But surely if for example Tatarstan gets real autonomy (becoming a vassal) would halt Russification. It seems realistic.


Cache pip

[Current coverage][cc-pull] is 92.52%

Merging [#179][cc-pull] into [master][cc-base-branch] will not change coverage

@@             master       #179   diff @@
==========================================
  Files            70         70
  Lines          4184       4184
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
  Hits           3871       3871
  Misses          313        313
  Partials          0          0

[][cc-pull]

Powered by Codecov. Last updated by [b3099a3...27ed30a][cc-compare]
[cc-base-branch]: https://codecov.io/gh/ORNL-CEES/Cap/branch/master?src=pr
[cc-compare]: https://codecov.io/gh/ORNL-CEES/Cap/compare/b3099a3f59330e455614baed48da510d9104f5d5...27ed30abc586eb6aa42966749be09634aeefcc6a
[cc-pull]: https://codecov.io/gh/ORNL-CEES/Cap/pull/179?src=pr


@-mention autocomplete should display more than 5 results

In case of several matches, the @-mention autocomplete should display either all of them or there should be an upper limit which can be configured by users. The list of results can be scrollable.
This is needed because there is a possibility that more than 5 users have same initial name, hence the desired person's name can be left out of the results which can be annoying.

This seems reasonable. So perhaps we should have five unique names be shown by the popover?

Hmm, but what does a "unique name" mean?  Unique first part of name before whitespace?  Maybe OK as long as we're not being used in a context where everyone's name has a title in front (probably a fairly safe assumption?).
I think we should do a quick investigation into what's easy to do from a tooling perspective, but I'd say the common case where you really want this is where you have 12 people whose names starts with "Alex" (or whatever), in which case we probably want to show all of them (but keep the thing 5 high with a scrollbar to find the rest).  We don't want to put 500 things into a scrollable typeahead, though, for perf reasons.

I sorta feel we should focus on #4249 first and see if that resolves the actual user-facing problem of finding the right person in typeahead efficiently.

I think #4249 has effectively solved this problem.


drop menu

i cannot open the category dropdown list menu either on the browser or mobile. can you please help

You should provide more details, eventually a link to your demo maybe.

hi is there any way to get this navbar to work on mobile app right now its only compatible for broswer.i like your style especailly dropdown style  in mobile app.
thank you


Added the page for emailing

Added a page documenting the configuration for sending email verification messages.
I also edited some stuff a little
Make sure to close https://github.com/kitsune-soc/docs/issues/10

Are the changes I made to your PR acceptable or should they be revised again?

Tis' fine


Unbind events

Hi, thanks for this awesome lib.
I solved a memory leak you had (you didn't unregister the scroll and touch events).

Thanks for the pull request! I'm getting a few javascript errors with this commit though. It looks like you may have copy pasted the bind logic from the parallax background attribute to the parallax attribute without changing out some values.

Hi, thanks for the review.
Let me know if it works better now.

It looks like most of the correct directive values have been switched out, but I'm getting more errors while testing these changes. It looks like the parallax and parallaxBackground directives initially had a different attrs variable in the link function signature. I fixed the JS error by changing out the attrs variable with $attrs in the parallax directive, but that started throwing more errors in the unbind function.


Some width problems at repository title

As you can see at the screenshot there are some width issues with the repository title.

One option is to move the repo description down near the owner (either above or below). Not sure how that would look.

isn't there a way to set a limit for the width?

I think the location of the description is good. A solution could be to set a padding of like 10 to 15 pixels on both sides and expand the text view down?

@RolfKoenders :+1: Yep that'd work nicely! Is there a character limit for repository descriptions?

@andrewda After trying it with different characters, there is no actual limit on how much you can insert/save. Github is only showing 347 characters before it cuts it of "Loru..."

Alright, then does it make sense to have the text view expand dynamically?

I think we should decide on a limit. 1/3 of my iPhone 6(s) Plus screen is filled with the repository description with 347 characters.. Thats to much i think?

Yep I agree (though I don't think there are many repos with huge descriptions). Maybe add an ellipsis after the word before the text reaches ~150 characters?

So we can definitely add some padding on the sides (and we probably should) but super long descriptions can still look funky.
In the list screen long descriptions look fine in my opinion:

But in the repo screen this can be a problem:

So I think we should:

Add a little padding (or width) so long descriptions aren't so close to the left and right edges of the screen
Definitely cut out descriptions after a certain point with ellipsis (~150 characters sounds fine)

Looks good for me, and cutting after some kind of characters seems fine.

@housseindjirdeh That sounds like a decent solution!

Hi there, I've made some changes to the description style in repo screen, mainly according to what @housseindjirdeh suggested. Looking forward to hearing your feedback 😉

@Antondomashnev I haven't checked it yet, but does your PR fixes this issue? It looks like it, except the 150 character limit.

@RolfKoenders yeah, instead of 150 characters limit I introduces the number of lines limit, I think it's  better, so if we change the font size of something there will be still 5 lines and maybe less than 150 characters.

Thanks for your awesome fix @Antondomashnev - this is good to close :)


Add an html repr for ASTs nodes.

🤔

Yeah, the following is not helpful:

Would be nice if you could collapse/expand...

Yeah, are you thinking a nested

details / summary tag
That expands to show more nodes?

or something more like graphviz representation

Yeah, are you thinking a nested
details / summary tag

Yes.
AST is interesting... but ast will be a tree anyway, so likely easier to represent explicitely.

+1
For an easier starting point, maybe add a text repr that pretty prints it? I use an ast pretty printer which I stole off someone's blog:
https://bitbucket.org/takluyver/greentreesnakes/src/96d6e70210789c0b743eabbbb57d9282b7e41d8c/astpp.py?at=default
http://alexleone.blogspot.co.uk/2010/01/python-ast-pretty-printer.html
It doesn't have an explicit license, so we'd need to contact the author to clarify that before using it in IPython.


Cache busting for glory

Easy way to get multiple hiera declarations working! :smile:

Thanks @dcarley and @madAndroid for the help! :smile:

@bobtfish can I get a merge? :+1:

@bobtfish can you look at this? I could use it. Thank you!


Release v0.8.0

env vars

SECRET_KEY_BASE
GITHUB_API_TOKEN

Codecov Report

:exclamation: No coverage uploaded for pull request base (main@4a02b51). Click here to learn what that means.
The diff coverage is 100.00%.

@@           Coverage Diff            @@
##             main       #58   +/-   ##
========================================
  Coverage        ?   100.00%
========================================
  Files           ?         8
  Lines           ?       100
  Branches        ?         0
========================================
  Hits            ?       100
  Misses          ?         0
  Partials        ?         0

Impacted Files
Coverage Δ

apps/publishing/lib/publishing/helper.ex
100.00% <100.00%> (ø)

apps/publishing/lib/publishing/integration.ex
100.00% <100.00%> (ø)

...ps/publishing/lib/publishing/integration/github.ex
100.00% <100.00%> (ø)

apps/publishing/lib/publishing/manage.ex
100.00% <100.00%> (ø)

apps/publishing/lib/publishing/manage/article.ex
100.00% <100.00%> (ø)

apps/publishing/lib/publishing/manage/blog.ex
100.00% <100.00%> (ø)

apps/publishing/lib/publishing/manage/platform.ex
100.00% <100.00%> (ø)

apps/publishing/lib/publishing/markdown.ex
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4a02b51...0b9aec7. Read the comment docs.