WorkflowTemplate CRD

I implemented WorkflowTemplate CRD which holds template data outside of Workflow which was discussed in #927.
You can refer to another template from any template by either local template name of template or template resource reference of templateRef. When the referenced template is loaded, template level arguments of arguments are applied.
Here's the example which describes the feature well.
apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
  name: workflow-template-whalesay-template
spec:
  templates:
  - name: whalesay-template
    inputs:
      parameters:
      - name: message
    container:
      image: docker/whalesay
      command: [cowsay]
      args: ["{{inputs.parameters.message}}"]
---
apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
  name: workflow-template-nested-template
spec:
  templates:
  - name: whalesay-inner-template
    templateRef:
      name: workflow-template-whalesay-template
      template: whalesay-template
    inputs:
      parameters:
      - name: message
    arguments:
      parameters:
      - name: message
        value: "{{inputs.parameters.message}} hello"
  - name: whalesay-template
    template: whalesay-inner-template
    inputs:
      parameters:
      - name: message
    arguments:
      parameters:
      - name: message
        value: "{{inputs.parameters.message}} hello"
---
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: workflow-template-nested-
spec:
  entrypoint: whalesay
  templates:
  - name: whalesay
    templateRef:
      name: workflow-template-nested-template
      template: whalesay-template
    arguments:
      parameters:
      - name: message
        value: "hello"

Here's the guide to read the example above.

It starts from the entry point whalesay of the workflow workflow-template-nested-xxxx
The template refers to the template whalesay-template in the workflow template workflow-template-nested-template.
The referred template refers to the template whalesay-inner-template of the context of the workflow template workflow-template-nested-template.
The referred template refers to the template whalesay-template of the context of the workflow template workflow-template-whalesay-template.
The template is a concrete template so returns this template.

After resolving input parameters of referred templates by arguments of referring templates at each step, this workflow outputs hello hello hello in the log.
Workflow templates must be submitted with argo template submit command before workflows using them are submitted unless you set the runtimeResolution option.

I'm still adding e2e tests and unit tests as the PR is quite huge.

I'll also check if ArgoUI works with this template well.

Here's the list of tasks I'm working on for the PR.

Add e2e and unit tests
Use shared indexer to get templates
Resolve templates on validate
Make sure ArgoUI still works with templates

Do you check templateRef is already load or not during workflow submit?

Yes! I will validate resolved templates when a workflow is submitted.

I added tests and template resolution on valdate. I didn't implement shared informer support to reduce workflow template get requests because it takes more time to implement it but the current implementation may be changed after reviews. I'll make another PR for it after this PR gets merged.
Workflows with WorkflowTemplate can be displayed in ArgoUI although there's no way to check the template contents on UI. I'll add template resolution on UI after this PR gets merged too.

@jessesuen @alexmt Could you review the PR?

Thanks for your work! 🎉
I'm trying to run the example from the PR description. For now I've encountered two issues:

I can't submit the full example to argo, I have to first submit workflowTemplate CRDs first then submit the workflow, otherwise I get:

2019/04/23 09:53:48 Failed to submit workflow: spec.entrypoint workflow template workflow-template-nested-template not found

If I create the WorkflowTemplate CRDs prior to submitting the workflow, the execution fails afterwards and argo get shows:

argo get workflow-template-nested-pgmcs
Name:                workflow-template-nested-pgmcs
Namespace:           default
ServiceAccount:      default
Status:              Failed
Message:             invalid spec: template type unspecified. choose one of: container, steps, script, resource, dag, suspend
Created:             Tue Apr 23 10:01:51 +0000 (10 seconds ago)
Started:             Tue Apr 23 10:01:51 +0000 (10 seconds ago)
Finished:            Tue Apr 23 10:01:51 +0000 (10 seconds ago)
Duration:            0 seconds

I've tried with both v2.3.0-rc1 and a local build from the latest version of this branch.

@xlucas Thank you for checking the function! Are you using minikube? If so, you need to build the image with minikue env.

@dtaniwaki I'm using hyperkube but that was a pretty similar issue with old images being pulled. This looks fine now!

@xlucas I actually want to have feedback about the CLI for the workflow template feature. I wanted to handle workflow templates in the same sub command for workflows, but I was not sure the best way to print workflow templates with workflows. Maybe, just print two tables for workflows and workflow templates? If we can handle workflow templates like this, we don’t need extra su command like argo template submit.

@dtaniwaki I do believe this a good think to have separate subcommands for separate objects.
Even though it may be confusing for newcomers not to be able to submit an all-in-one "kubernetes-style" manifest, I think it's not a problem as long as it's properly documented. Otherwise, operating templates would be tightly coupled to submitting specific workflows whilst they serve the opposite purpose. WDYT?

I realized I need to use the default service account of workflow’s namespace to refer workflow templates and maybe need to add something like workflow template service account option. I will implement it.

Hey. This looks great - is it planned to be merged soon?

@hnykda I was busy the past month and I still need to consider the accessibility of the WorkflowTemplate CRD. I'll tackle it soon!

@dtaniwaki This is a really useful feature to make templates reusable! Would love to see it merged and try it out.

I realized I need to use the default service account of workflow’s namespace to refer workflow templates, otherwise, anyone can access to any workflow templates. Maybe we need to add something like workflow template service account option. I will implement it.

After thinking of this issue a while, I concluded that we don't have to care the accessibility of workflow templates as long as the templates exist in the same namespace of a workflow which refers to them like the same way as other resources like ConfigMap are handled.

I refactored the code a little bit and I think it's pretty much of the basic function of this feature.
Here's the output of argo cli when I submitted workflows w/ templates.
$ ./dist/argo template submit -n argo examples/workflow-template/*.yaml
WARN[0000] Workflow is not a workflow template
WARN[0000] Workflow is not a workflow template
WARN[0000] Workflow is not a workflow template
WARN[0000] Workflow is not a workflow template
WARN[0000] Workflow is not a workflow template
WARN[0000] Workflow is not a workflow template
Name:                workflow-template-whalesay-template
Namespace:           argo
Created:             Mon Jul 15 01:08:13 +0900 (1 second ago)
Name:                workflow-template-nested-template
Namespace:           argo
Created:             Mon Jul 15 01:08:14 +0900 (now)
Name:                workflow-template-random-fail-template
Namespace:           argo
Created:             Mon Jul 15 01:08:14 +0900 (now)
$ ./dist/argo submit -n argo examples/workflow-template/*.yaml
WARN[0000] WorkflowTemplate is not a workflow
WARN[0000] WorkflowTemplate is not a workflow
WARN[0000] WorkflowTemplate is not a workflow
Name:                workflow-template-dag-diamond-4q2rh
Namespace:           argo
ServiceAccount:      default
Status:              Pending
Created:             Mon Jul 15 01:08:17 +0900 (now)
Name:                workflow-template-hello-world-ptcvr
Namespace:           argo
ServiceAccount:      default
Status:              Pending
Created:             Mon Jul 15 01:08:18 +0900 (now)
Name:                hello-world-zk7qr
Namespace:           argo
ServiceAccount:      default
Status:              Pending
Created:             Mon Jul 15 01:08:18 +0900 (now)
Name:                workflow-template-nested-2mdjk
Namespace:           argo
ServiceAccount:      default
Status:              Pending
Created:             Mon Jul 15 01:08:20 +0900 (now)
Name:                workflow-template-retry-with-steps-6542b
Namespace:           argo
ServiceAccount:      default
Status:              Pending
Created:             Mon Jul 15 01:08:21 +0900 (now)
Name:                workflow-template-steps-g6llt
Namespace:           argo
ServiceAccount:      default
Status:              Pending
Created:             Mon Jul 15 01:08:23 +0900 (now)
$ argo list -n argo
NAME                                       STATUS      AGE   DURATION   PRIORITY
workflow-template-dag-diamond-4q2rh        Succeeded   1m    1m         0
workflow-template-retry-with-steps-6542b   Succeeded   1m    56s        0
workflow-template-steps-g6llt              Succeeded   1m    50s        0
workflow-template-nested-2mdjk             Succeeded   1m    15s        0
workflow-template-local-ref-zk7qr                          Succeeded   1m    14s        0
workflow-template-hello-world-ptcvr        Succeeded   1m    11s        0
$ ./dist/argo get -n argo workflow-template-retry-with-steps-6542b
Name:                workflow-template-retry-with-steps-6542b
Namespace:           argo
ServiceAccount:      default
Status:              Succeeded
Created:             Mon Jul 15 01:08:21 +0900 (2 minutes ago)
Started:             Mon Jul 15 01:08:21 +0900 (2 minutes ago)
Finished:            Mon Jul 15 01:09:17 +0900 (1 minute ago)
Duration:            56 seconds

STEP                                                                               PODNAME                                              DURATION  MESSAGE
 ✔ workflow-template-retry-with-steps-6542b (retry-with-steps)
 ├---✔ hello1 (workflow-template-random-fail-template/random-fail-template)
 |   ├-✖ hello1(0) (workflow-template-random-fail-template/random-fail-template)   workflow-template-retry-with-steps-6542b-3456092177  33s       failed with exit code 1
 |   └-✔ hello1(1) (workflow-template-random-fail-template/random-fail-template)   workflow-template-retry-with-steps-6542b-4060233556  4s
 └-·-✔ hello2a(0) (workflow-template-random-fail-template/random-fail-template)    workflow-template-retry-with-steps-6542b-3526014558  8s
   └-✔ hello2b (workflow-template-random-fail-template/random-fail-template)
     ├-✖ hello2b(0) (workflow-template-random-fail-template/random-fail-template)  workflow-template-retry-with-steps-6542b-3678971621  8s        failed with exit code 1
     ├-✖ hello2b(1) (workflow-template-random-fail-template/random-fail-template)  workflow-template-retry-with-steps-6542b-3477787288  3s        failed with exit code 1
     └-✔ hello2b(2) (workflow-template-random-fail-template/random-fail-template)  workflow-template-retry-with-steps-6542b-1329810771  3s

I updated the steps and DAG example so we can check how nested steps/DAG work.
$ ./dist/argo get -n argo workflow-template-steps-dlqqb
Name:                workflow-template-steps-dlqqb
Namespace:           argo
ServiceAccount:      default
Status:              Succeeded
Created:             Mon Jul 15 01:28:25 +0900 (27 seconds ago)
Started:             Mon Jul 15 01:28:25 +0900 (27 seconds ago)
Finished:            Mon Jul 15 01:28:48 +0900 (4 seconds ago)
Duration:            23 seconds

STEP                                                                              PODNAME                                   DURATION  MESSAGE
 ✔ workflow-template-steps-dlqqb (hello-hello-hello)
 ├---✔ hello1 (workflow-template-whalesay-template/whalesay-template)             workflow-template-steps-dlqqb-66550121    6s
 └-·-✔ hello2a (workflow-template-inner-steps/inner-steps)
   | ├---✔ inner-hello1 (workflow-template-whalesay-template/whalesay-template)   workflow-template-steps-dlqqb-1238316130  6s
   | └-·-✔ inner-hello2a (workflow-template-whalesay-template/whalesay-template)  workflow-template-steps-dlqqb-2511940447  7s
   |   └-✔ inner-hello2b (whalesay-template)                                      workflow-template-steps-dlqqb-2528718066  10s
   └-✔ hello2b (workflow-template-whalesay-template/whalesay-template)            workflow-template-steps-dlqqb-769280991   9s

$ ./dist/argo get -n argo workflow-template-dag-diamond-v9m47
Name:                workflow-template-dag-diamond-v9m47
Namespace:           argo
ServiceAccount:      default
Status:              Succeeded
Created:             Mon Jul 15 01:32:59 +0900 (43 seconds ago)
Started:             Mon Jul 15 01:32:59 +0900 (43 seconds ago)
Finished:            Mon Jul 15 01:33:39 +0900 (3 seconds ago)
Duration:            40 seconds

STEP                                                                    PODNAME                                         DURATION  MESSAGE
 ✔ workflow-template-dag-diamond-v9m47 (diamond)
 ├-✔ A (workflow-template-whalesay-template/whalesay-template)          workflow-template-dag-diamond-v9m47-3420111096  5s
 ├-✔ B (workflow-template-whalesay-template/whalesay-template)          workflow-template-dag-diamond-v9m47-3470443953  10s
 ├-✔ C (workflow-template-inner-dag/inner-diamond)
 | ├-✔ inner-A (workflow-template-whalesay-template/whalesay-template)  workflow-template-dag-diamond-v9m47-1233185554  7s
 | ├-✔ inner-B (whalesay-template)                                      workflow-template-dag-diamond-v9m47-1216407935  9s
 | ├-✔ inner-C (whalesay-template)                                      workflow-template-dag-diamond-v9m47-1199630316  6s
 | └-✔ inner-D (workflow-template-whalesay-template/whalesay-template)  workflow-template-dag-diamond-v9m47-1182852697  6s
 └-✔ D (workflow-template-whalesay-template/whalesay-template)          workflow-template-dag-diamond-v9m47-3503999191  6s

I updated the steps and DAG example so we can check how nested steps/DAG work.
$ ./dist/argo get -n argo workflow-template-steps-dlqqb
Name:                workflow-template-steps-dlqqb
Namespace:           argo
ServiceAccount:      default
Status:              Succeeded
Created:             Mon Jul 15 01:28:25 +0900 (27 seconds ago)
Started:             Mon Jul 15 01:28:25 +0900 (27 seconds ago)
Finished:            Mon Jul 15 01:28:48 +0900 (4 seconds ago)
Duration:            23 seconds

STEP                                                                              PODNAME                                   DURATION  MESSAGE
 ✔ workflow-template-steps-dlqqb (hello-hello-hello)
 ├---✔ hello1 (workflow-template-whalesay-template/whalesay-template)             workflow-template-steps-dlqqb-66550121    6s
 └-·-✔ hello2a (workflow-template-inner-steps/inner-steps)
   | ├---✔ inner-hello1 (workflow-template-whalesay-template/whalesay-template)   workflow-template-steps-dlqqb-1238316130  6s
   | └-·-✔ inner-hello2a (workflow-template-whalesay-template/whalesay-template)  workflow-template-steps-dlqqb-2511940447  7s
   |   └-✔ inner-hello2b (whalesay-template)                                      workflow-template-steps-dlqqb-2528718066  10s
   └-✔ hello2b (workflow-template-whalesay-template/whalesay-template)            workflow-template-steps-dlqqb-769280991   9s

$ ./dist/argo get -n argo workflow-template-dag-diamond-v9m47
Name:                workflow-template-dag-diamond-v9m47
Namespace:           argo
ServiceAccount:      default
Status:              Succeeded
Created:             Mon Jul 15 01:32:59 +0900 (43 seconds ago)
Started:             Mon Jul 15 01:32:59 +0900 (43 seconds ago)
Finished:            Mon Jul 15 01:33:39 +0900 (3 seconds ago)
Duration:            40 seconds

STEP                                                                    PODNAME                                         DURATION  MESSAGE
 ✔ workflow-template-dag-diamond-v9m47 (diamond)
 ├-✔ A (workflow-template-whalesay-template/whalesay-template)          workflow-template-dag-diamond-v9m47-3420111096  5s
 ├-✔ B (workflow-template-whalesay-template/whalesay-template)          workflow-template-dag-diamond-v9m47-3470443953  10s
 ├-✔ C (workflow-template-inner-dag/inner-diamond)
 | ├-✔ inner-A (workflow-template-whalesay-template/whalesay-template)  workflow-template-dag-diamond-v9m47-1233185554  7s
 | ├-✔ inner-B (whalesay-template)                                      workflow-template-dag-diamond-v9m47-1216407935  9s
 | ├-✔ inner-C (whalesay-template)                                      workflow-template-dag-diamond-v9m47-1199630316  6s
 | └-✔ inner-D (workflow-template-whalesay-template/whalesay-template)  workflow-template-dag-diamond-v9m47-1182852697  6s
 └-✔ D (workflow-template-whalesay-template/whalesay-template)          workflow-template-dag-diamond-v9m47-3503999191  6s

Awesome work on this @dtaniwaki
Once you've created a WorkflowTemplate is there a way to update it without deleting it first?

Once you've created a WorkflowTemplate is there a way to update it without deleting it first?

@mark9white Updating existing WorkflowTemplate is very dangerous because it affects the result of referring workflows. I think this is the discussion point with everyone. Do you expect WorkflowTemplates immutable or not?

One idea I came up is we can record template resolution in node status, then completed workflows don't care template modifications.

@dtaniwaki That sounds a great idea ...

Could they be versioned to address the issue with updates breaking running workflows?

Or copied to workflow during workflow execution - so if tpl would change later, it won't affect existing workflows, which makes sense as they're already done.

@dtaniwaki A usability question looking at the example above:
Does it make sense for templateRef.template to be an optional field with reasonable defaults to avoid repetition/verbosity? I suspect that in practice, either a WorkflowTemplate is used for reusing a single template (e.g. workflow-template-whalesay-template) OR it contains multiple templates (e.g. workflow-template-nested-template) and the the default can be specified in WorkflowTemplate itself instead of by each reference.

@frankfarzan I agree that we want to avoid repetition and verbosity, but I'm not sure what reasonable defaults are. What do you think of having a field which specifies the default template in WorkflowTemplate? I haven't come up with a good name for the field yet, though.

I see parallels between this field and spec.entrypoint in Workflow, so calling this entrypoint may be intuitive to users? We can also make the name more descriptive like defaultTemplate or something along those lines.

@frankfarzan Thanks for your idea about the field name. I think it's better to make that improvement separately after the basic function of this PR gets merged to avoid extra complexity for reviewers.

@jessesuen I fixed the code based on your review and also added the lint command.
By the way, do you have any idea the ideal behavior of updating templates?
https://github.com/argoproj/argo/pull/1312#issuecomment-511662757
https://github.com/argoproj/argo/pull/1312#issuecomment-511665396
I think we can record resolved template contents in node status so we won't lose what was executed even after updating or deleting templates. We may need to wait for the Workflow persistence to store massively expanded contents in node status.

By the way, do you have any idea the ideal behavior of updating templates?
#1312 (comment)
#1312 (comment)

One idea I came up is we can record template resolution in node status, then we can keep executed workflow step information, and it's possible to draw it in UI, even if referred templates got modified or deleted later.

I had the same thought about storing resolved templates in the status upon submission. So when workflow initially starts, we would resolve all templates, store it in status, which would allow templates objects to change even during in-flight workflows.
I think we could that as a separate PR, I want to get this in faster.

Minor comment about processAggregateNodeOutputs and resolve types.go conflict, and it should be good to go.

Filed https://github.com/argoproj/argo/issues/1525 to resolve workflow templates into status.

#1462 significantly conflicts with this PR. I'm resolving it...

@jessesuen I resolved the conflicts and found a test failure. Please check the commits just in case.

Is somewhere a link to the documentation for this or is it just a wishful thinking? :grin:

When defining multiple volumeMounts in a template linting / submitting fails with :
templates.foo templates.foo.container.volumeMounts[1].mountPath 'placeholder' already mounted in container.volumeMounts.placeholder
Linting succeeds when I comment out the second volumeMount.
Seems like all volume mounts are being mapped to the same placeholder ?

This change is very weird and confusing from the design perspective.
Argo has two major types - Template and Task.
Task ~= template + arguments
It would have been natural to add templateRef to Task so that you can have
Task ~= templateRef + arguments
Instead this PR does something really weird by adding templateRef to Teamplate
Now template can reference template which can reference template etc. Meanwhile each template in that infinite stack has separate inputs, outputs, dag, etc. This just seems very wrong from API or DB architecture perspective.
I also see that this PR has added arguments to Template.
So, previously Task ~= template + arguments.
Now Template also ~= Template + arguments
What does that mean?

This could have been a small nice design...
Just
apiVersion: argoproj.io/v1alpha1
kind: Workflow
spec:
  entrypoint: whalesay
  templates:
  - name: whalesay
    templateRef:
      url: https://..... # Can point to any template YAML - container, DAG, etc
    arguments:
      parameters:
      - name: message
        value: "hello"

Similar to https://github.com/kubeflow/pipelines/blob/fd6c756dd2a510581e52557cee881ec0c2cbc445/sdk/python/tests/components/test_data/retail_product_stockout_prediction_pipeline.component.yaml#L40


Issue 116 - Add gauge tests and tests for a second gas. Also finish off documentation.

https://github.com/Code-and-Response/Prometeo/issues/116
Add gauge tests and tests for a second gas. Also finish off documentation.
@lidderupk or @narengan - would one of you be OK to review this? Thanks!

Thanks @JSegrave-IBM . Awesome work!

Thanks @lidderupk !


Considering rename oxc_vscode into oxc_ls

Currently, the oxc language server is named oxc_vscode https://github.com/oxc-project/oxc/blob/117d95f0aedcc3fe5d92030e1195ec05057b0741/editors/vscode/server/Cargo.toml#L2-L2,  but as a generic language server, it can be used in any editor that supports language server protocol. So I think oxc_ls aka oxc language server should be a better name for our language server.

cc @Boshen

part of https://github.com/oxc-project/oxc/issues/1217#issuecomment-1811740007

biome calls it biome_lsp I think? The naming doesn't really matter as long as it works for IDEs ...

No, if we want to support more editor clients, it matters, oxc_vscode sounds like it is vscode only, but it is a generic language server, It may affect adoption.

I think we should rename it ASAP since the language server is not widely used yet.

I think we should rename it ASAP since the language server is not widely used yet.

Sure, just do it. It's currently published to vscode market place as oxc, the underlying crate is not published.

Alright, lsp technically means language server protocal, I don't think  biome_lsp is a good name for a language server, so I am sticked to use oxc_ls aka oxc language server

I always try to avoid abbreviations when possible, to avoid confusion for new contributors, how about oxc_language_server?

I always try to avoid abbreviations when possible, to avoid confusion for new contributors, how about oxc_language_server?

That's alright, but ls is commonly used,
ls

language_server

I always try to avoid abbreviations when possible, to avoid confusion for new contributors, how about oxc_language_server?

I used name oxc_language_server in pull request.


Fix missing assembly error on 2019.3

An errant, unused using statement was causing a missing assembly error on Unity 2019.3.
This change also adds in a missing #if !UNITY_2019_3_OR_NEWER in the configuration dialog as it was attempting to access a dictionary key that will not exist.

/azp run

Tested and confirmed this fixes the error


can we support conversationId and parentMessageId?

To maintain a conversation, it is helpful to use consistent prompts throughout. Is this currently being done? If not, could you please add the conversationId and parentMessageId so that we can achieve this?

@lzhgus i add id field in new version.


I don't see the "Secured with IWA" option when running AGO onsite

Is this really an issue?
Thanks

I'm not sure I understand...could you provide some more detail? Did you download the app and host it locally and that's where you're not seeing the checkbox for IWA?

Yes I am hosting the app locally.
My portal is using IWA.
When I tried to login in from https://ago-assistant.esri.com/ there was a check box option
I can login in locally if I provide my username and password though.

Based on the other issues you're experiencing I think you may have a broken or dated version of the app. Can you try downloading the latest version and test again?

Correct!
I grabbed the "latest" version from https://github.com/Esri/ago-assistant/releases and it isn't :)

You're right, I need to start tagging and pushing all updates (no matter how minor) into the releases page.


Add support for https check with selfsigned certificate

It seems new feature from consul that was implemented by PR https://github.com/hashicorp/consul/pull/1984 and is available in 0.7.1.
I created PR for consul-api https://github.com/Ecwid/consul-api/pull/83 to support it

+1. In my project we are planning to use consul as discovery service and do register services with https health checks and self signed certificate and this merge is the key for us to move forward.

@vivekasingh this is depending on the consul-api PR, nothing I can do here yet.


Can not use the library in strict mode.

Can we create a new release for this fix ? I see that we fixed this, but we didn't create a new release.

Please use the fix locally for now. I will update this with the next release. Thankyou for letting me know about the issue. :+1:

Could you please let me know all the details? How are you facing the issue? Thanks. @duc01226

Hi @zeeshanhyder . Thanks for your reply.

In my application, I used "ng-strict-di". Reference : https://docs.angularjs.org/api/ng/directive/ngApp
In ng-right-click.js, we have this part which is not allowed in strict mode :

We need to use explicit function annotation, like in your current source code, which is not included in a new release :

Hi @zeeshanhyder ,
Could you please check to create a new release to fix this issue ?

@duc01226 Sure. I'll release new version in a while.
Thanks.

Fixed by commit https://github.com/zeeshanhyder/angular-right-click/commit/2257a49164ca46bca46521b2332534820dbcd29d


ZSTD or XZ compression for server

Describe your idea
Configurable ZSTD and/or XZ compression options would be very useful for this mod, as the inefficiency and CPU usage for creating very large ZIP backups can cause significant server lag and massive world backup files.
Using xz on Linux creates a world backup anywhere from ~35% to ~65% smaller than zip in an equivalent time, and for server admins who frequently create backups (like me!) the reduced disk I/O and compression time would come in handy.

a lot more can saved by using diffs (like github)
only saving what is changed in files

I started looking into adding another compression method. However, only saving diffs is already possible. (But this feature needs a bit more improvement I guess)

I looked a bit deeper into it and had a good system to compress using ZSTD and XZ. However, this was only theoretically and not able to compress in game since it couldn't find the correct classes. As reference how I tried implementing it, I leave this Gist here: https://gist.github.com/MelanX/6b3924bbcf7157f4a200d8309f89dffd
Maybe someone else can use it and add it in a pull request.

@MelanX Sorry for necroing this old issue, but I think if your problems were finding classes, you might just have needed to include dependencies. For example, I know Distant Horizons has both LZMA (xz) and LZ44 compression. They use the shadow Gradle plugin to shadow the dependencies, which can be found here: https://github.com/johnrengelman/shadow
With that, you should be able to properly include the dependencies in the mod. HOWEVER...
I know shadowing can cause some issues if multiple copies of the same jar are included, for example, if you also decided to add org.tukaani:xz in a different version than Distant Horizons, so maybe there's a better way to go about this.
I'm also interested in a slightly better compression than ZIP (albeit XZ tends to be CPU intensive), so I'm willing to help if I can (however I have zero knowledge specific to Minecraft modding)

Thank you very much for trying to help :)
I really want to implement that one, too!
As you can see here, I already added it as dependency using the forge intern feature Jar-in-Jar: https://github.com/ChaoticTrials/SimpleBackups/blob/feature/xz-backup/build.gradle#L11
I’m not sure anymore what the exact problem for me was now, but I think it was that the file size was nearly identical to zip and it took even longer. Here my way to implement it: https://github.com/ChaoticTrials/SimpleBackups/blob/feature/xz-backup/src/main/java/de/melanx/simplebackups/compression
If I’m doing this wrong and you (or anyone else) can help with this, I’d really love it!

I'm actually trying to set it up right now! I was willing to test Zstd, since it has a native Java implementation too, and could be a better middle ground than XZ and the others.
 This guide gets me an error:
> Failed to apply plugin 'org.moddingx.modgradle.mapping'.
   > Cannot add extension with name 'modgradle', as there is an extension already registered with that name.

Very interesting. I decided to do some basic tests, since I wasn't getting the plugin to work and you said that XZ was very similar to zip.
I used 7zips LZMA2 implementation, just because that's what I had installed. The results are quite shocking:

Format
Size in bytes

Raw
161.501.184

7z Ultra
113.864.704

7z Normal
114.577.408

Zip Normal
118.218.752

Zip Ultra
117.891.072

I'm actually shocked at how poorly LZMA2 did. I'm thinking Minecraft's data just doesn't compress well, and whatever gains are mostly from archiving the files together and thus creating some "repeating information".
I'll... Give up on this for now. It simply looks like it's not worth the trouble for such small gains. Thank you for the great mod!


BUG: Fix index error when volume roll occurs at end of window.

When a volume roll makes the back contract the primary at the end of a window,
the next auto close to use in the roll finder loop (i.e. the auto close of the
front contract on the last session of the window) is greater than the last date
of the window. This caused an index error because the auto close would
searchsorted
Fix by changing the start date when entering the loop to calculate rolls so that
it takes the floor of the auto close and the next date to consider while working
backwards.
Also, use get_loc instead of searchsorted since exact dates are used.

Coverage decreased (-0.04%) to 87.079% when pulling e4a3689d11b4b7e286f0793676e5df32795355d5 on roll-index-error into f3a36fe97f7f504688619b1f44fbe0c2009624a1 on master.

Closing, bounds errors were fixed via https://github.com/quantopian/zipline/pull/1600


Cannot find a way to configure Eureka client with Docker swarm mode

When using Eureka client with Docker swarm mode, I didn't find a way to correctly configure the client to notify the correct hostname / ip. With Docker swarm mode you have at least two networks (if the container exposes a port to the public network), the ingress and the overlay network used for internal cluster communication (the correct one to use for Eureka). The problem is that, in each container, you have two interfaces eth0 and eth1; these are assigned randomically to the ingress or overlay ip thus is not possible to benefit from spring.cloud.inetutils.ignoredInterfaces property. So what happens is that randomically Eureka clients advertise the correct host configuration. Did you have any similar case to this or is there a specific Eureka configuration that can help me with that?

You have to have some way to know what address to advertise. I've not used swarm before, so I can't help there.

I have found two solutions to this problem:

Specifying a custom network value in Docker swarm and using spring.cloud.inetutils.preferredNetworks property
Using eureka.instance.hostName: service-name and eureka.instance.preferIpAddress: false, where service-name is the same service-name specified in the Docker compose file

What do you think about them?

I think either of those things sound reasonable if you've found them to work.

Excelent! using eureka.instance.hostName: service-name and eureka.instance.preferIpAddress: false worked for me!

The problem with option 2 (eureka.instance.hostName: service-name and eureka.instance.preferIpAddress: false ) is that I think load balancing is being made in docker swarm. Look the following code:
discoveryClient.getServices().forEach(serviceName -> {
discoveryClient.getInstances(serviceName).forEach(instance->{
services.add( String.format("%s:%s",serviceName,instance.getUri()));
});
});
These are the results:
[
"organizationservice:http://organizationservice:8085",
"organizationservice:http://organizationservice:8085",
"organizationservice:http://organizationservice:8085",
"organizationservice:http://organizationservice:8085",
"licensingservice:http://licensingservice:8080"
]
So load balancing is not on client side.
But using option 1 instead,such as:
spring:
cloud
inetutils:
preferredNetworks:
- 192.168
- 10.0
I could see is working appropriately,these are the results:
[
"licensingservice:http://<IP_ADDRESS>:8080",
"organizationservice:http://<IP_ADDRESS>:8085",
"organizationservice:http://<IP_ADDRESS>:8085",
"organizationservice:http://<IP_ADDRESS>:8085",
"organizationservice:http://<IP_ADDRESS>:8085"
]

Does this issue need to be open anymore?  Sounds like there is a solution using option 2 above

In think option 1 is more adequate. With option 2 load balancing is being made in docker swarm and not in the client, so we would not have client load balancing which is one of the key advantages of using Eureka plus Ribbon.

Either way, there is a configuration that works. I don't see the need for us to change anything.

Hello everyone ✋ . Same problem with my spring-cloud (Dalston.SR3) system in docker swarm.
Option 2 (preferIpAddress: false & service-name) really breaks the client-side load balancing (eureka + ribbon). That's why I am trying to use option 1 (preferIpAddress: true & preferredNetworks).
When we run a service in docker swarm, service without public ports registers in only one overlay network (by default it's <IP_ADDRESS>/24, but we can create own docker network create with another mask, gateway, etc). When we register a service with public ports, docker swarm add this service to overlay (<IP_ADDRESS>/24 or your own) and ingress (<IP_ADDRESS>/16) networks. Overlay lets services to communicate and ingress lets to swarm's built-in load balancing between service replicas.
This is my configuration v1:
spring:
  cloud:
    inetutils:
      preferred-networks:
        - 10.0

eureka:
  instance:
    preferIpAddress: true

With v1 configuration Eureka show this:

Look at instance-id in last column. By default instance-id create by pattern:
${spring.cloud.client.hostname}:${spring.application.name}:${spring.application.instance_id:${server.port}}}

This means that spring.cloud.client.hostname is getting from ingress network while overlay is not ready. Eureka uses instance-id to distinct between different instances of the same application.  For now the most important feature is scaling.
One more time. Look at hostName and ipAddr of every instances of both services. They are equal 😣. Eureka has 3 instances of my 1th service with same ip <IP_ADDRESS> which was used on 1th registration. Let's check out our network docker network inspect stack_default:
[
    {
        "Name": "stack_default",
        "Id": "fusf35hkd98se12cj83w60idn",
        "Created": "2017-11-22T08:14:29.363971Z",
        "Scope": "swarm",
        "Driver": "overlay",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": null,
            "Config": [
                {
                    "Subnet": "<IP_ADDRESS>/24",
                    "Gateway": "<IP_ADDRESS>"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {
            "0f42cbc6278af4a2d7b5dcf886f3946183f8f28569f13a6f7f081b469964d34f": {
                "Name": "stack_template-service-2.2.1mk1nm3k7lkxulq3hu82ytcf6",
                "EndpointID": "a0d09bf5458eaf281867c039897056cc5b290963d6d8f4e7944780b4bc019411",
                "MacAddress": "02:42:0a:00:00:17",
                "IPv4Address": "<IP_ADDRESS>/24",
                "IPv6Address": ""
            },
            "34d01b2927a24637c6e5b7410216fb24bf725f3ac49dc2fab0e6dcf3eecde953": {
                "Name": "stack_viz.h8fm2kw19gujlslk952iwtzac.yhv4kws5cf1qo26ba69w87e2h",
                "EndpointID": "5e9fce24189119727dc29dd4d4eb936f3f75f0b16e43b53944b108b021d1f762",
                "MacAddress": "02:42:0a:00:00:0c",
                "IPv4Address": "<IP_ADDRESS>/24",
                "IPv6Address": ""
            },
            "3eddd0f792238c031a9f0da77fb4ffa39b6d341f315e037d1599d8468f56ffe5": {
                "Name": "stack_portainer.h8fm2kw19gujlslk952iwtzac.tw9ph1a7e23lec87oelxgsqhs",
                "EndpointID": "84598bd4876b0494b250c3b56953b9dc21134954292dca0a6baf1490e1ba9c26",
                "MacAddress": "02:42:0a:00:00:03",
                "IPv4Address": "<IP_ADDRESS>/24",
                "IPv6Address": ""
            },
            "519586a0855dc64d041e76e3e8d647177f689b72552ee4db86fd6c5caa60e058": {
                "Name": "stack_template-service-1.3.r6p74h0lv0825pyt4j9r9rshn",
                "EndpointID": "f2da49faa39dead6a98c5b18388aed58980cc281a3dc74ca046feda765fd8af8",
                "MacAddress": "02:42:0a:00:00:1b",
                "IPv4Address": "<IP_ADDRESS>/24",
                "IPv6Address": ""
            },
            "596154a80c95f21152130099b3738259e2d87f6c66580c78e2422a339cb33135": {
                "Name": "stack_config.1.nc2kmq8no8nha00u37dgxf84k",
                "EndpointID": "34f00a19458a098d0536ecf3ec73c972294befbcda89d441e9930420e745532b",
                "MacAddress": "02:42:0a:00:00:0e",
                "IPv4Address": "<IP_ADDRESS>/24",
                "IPv6Address": ""
            },
            "6e05edd58d8b300bad4407efa3462a30e0bc678c15cb93041a516ec88419c4ac": {
                "Name": "stack_rabbitmq.1.n3nio3q50zfmhxawzac0wlpkm",
                "EndpointID": "46b7f2c24927fed77bb66625a338caf8af9581dd1161802eab80e204fb22e3c2",
                "MacAddress": "02:42:0a:00:00:10",
                "IPv4Address": "<IP_ADDRESS>/24",
                "IPv6Address": ""
            },
            "7c1f26e5237488e131e500de15c8656d949e32b591d5d0bf943b04dbd01d2bfa": {
                "Name": "stack_monitor.1.lly269pqrq61229hr8qsc9yx7",
                "EndpointID": "02845c9e40d65d021a78967d5cef443915c9b15e213ee67d13584854cbe7805b",
                "MacAddress": "02:42:0a:00:00:14",
                "IPv4Address": "<IP_ADDRESS>/24",
                "IPv6Address": ""
            },
            "ab62abccad3c50745d1ddfd5b44e5d320c7915e0e880caf6df94dc699138e814": {
                "Name": "stack_template-service-1.1.pe9kmnzhord5e7cqcf6asstvw",
                "EndpointID": "70bb8a08b6d1a2da1fb228e52ec1acbddb7b371b41a73c55af74115e36642201",
                "MacAddress": "02:42:0a:00:00:09",
                "IPv4Address": "<IP_ADDRESS>/24",
                "IPv6Address": ""
            },
            "bfe93d45cb68dda8225fc0ef655420bf69c4f3c03fed12b17b4c511ab3f95f64": {
                "Name": "stack_dvizz.h8fm2kw19gujlslk952iwtzac.b5n2hfo54qbv650wrcjwbqic3",
                "EndpointID": "d14ca4f296411155ed4fa7868ef63db411532d44f11f2083c01b10295b2dc420",
                "MacAddress": "02:42:0a:00:00:05",
                "IPv4Address": "<IP_ADDRESS>/24",
                "IPv6Address": ""
            },
            "da06a28ab31cefb030cd68729589f45ae82fdfbd5545b431f2cf97a8f0af7e30": {
                "Name": "stack_registry.1.if6gdn48et01zpgyssa7nfq8e",
                "EndpointID": "9dfd032f61bfe8d096e8f7413e690cd02e06d47fc88cf2a9e4477be18a96aee9",
                "MacAddress": "02:42:0a:00:00:12",
                "IPv4Address": "<IP_ADDRESS>/24",
                "IPv6Address": ""
            },
            "e788ac5083cdb8e6f33264e5cb2de58e5c9f73810286feb593a2f51ae993ea11": {
                "Name": "stack_template-service-1.2.qnlor4tcx6kt9fypadxtt7lgp",
                "EndpointID": "7591b226918a51d1f4dbe6c8b66edb14f104ca658d6c20f298e8f0bfb79b8eab",
                "MacAddress": "02:42:0a:00:00:0a",
                "IPv4Address": "<IP_ADDRESS>/24",
                "IPv6Address": ""
            },
            "ea22b9903c95c62adbf6dd916890e858fce79b7b1cc5241c78d78051aad8cc22": {
                "Name": "stack_gateway.1.y42ihbkq93f5t5spwj321amkz",
                "EndpointID": "62fb9e93c23b3c45e3fc2ee4b83410bca9e70b3ca5c27f4d39a7cfac64f522a4",
                "MacAddress": "02:42:0a:00:00:1a",
                "IPv4Address": "<IP_ADDRESS>/24",
                "IPv6Address": ""
            },
            "f694dc0cf21f774490413293a328cbf400f26f6cd6739804d71aa08c65b7a0ca": {
                "Name": "stack_auth.1.xp8tb6h1zam0luh53ucnxsus0",
                "EndpointID": "06021ee2004052bd40a5969c551f217e3db99603a0b09479e34b12d4f39a82a9",
                "MacAddress": "02:42:0a:00:00:07",
                "IPv4Address": "<IP_ADDRESS>/24",
                "IPv6Address": ""
            }
        },
        "Options": {
            "com.docker.network.driver.overlay.vxlanid_list": "4097"
        },
        "Labels": {
            "com.docker.stack.namespace": "stack"
        },
        "Peers": [
            {
                "Name": "moby-33d8cf6a608e",
                "IP": "<IP_ADDRESS>"
            }
        ]
    }
]

There are 3 replicas of template-service-1:

"Name": "stack_template-service-1.1.pe9kmnzhord5e7cqcf6asstvw", "IPv4Address": "<IP_ADDRESS>/24",
"Name": "stack_template-service-1.2.qnlor4tcx6kt9fypadxtt7lgp", "IPv4Address": "<IP_ADDRESS>/24"
"Name": "stack_template-service-1.3.r6p74h0lv0825pyt4j9r9rshn", "IPv4Address": "<IP_ADDRESS>/24"

Why Eureka registered service's replicas with same ip <IP_ADDRESS>? 🤕 This IP address is not using anymore in overlay network, it was using for 1th replica before I changed scale value. This means that Eureka remember 1th IP address of 1th replica of any service and continue to use it for every other repicas.
What do you think, guys? How can I fix that?

UPDATE4:
Day 2⃣. I added code below in all my spring-boot apps in a cluster.
@Configuration
public class EurekaClientConfig {

    @Bean
    @Autowired
    @Profile("docker")
    public EurekaInstanceConfigBean eurekaInstanceConfig(final InetUtils inetUtils) throws IOException {
        final String hostName = System.getenv("HOSTNAME");
        String hostAddress = null;

        final Enumeration<NetworkInterface> networkInterfaces = NetworkInterface.getNetworkInterfaces();
        for (NetworkInterface netInt : Collections.list(networkInterfaces)) {
            for (InetAddress inetAddress : Collections.list(netInt.getInetAddresses())) {
                if (hostName.equals(inetAddress.getHostName())) {
                    hostAddress = inetAddress.getHostAddress();
                }

                System.out.printf("%s: %s / %s\n", netInt.getName(),  inetAddress.getHostName(), inetAddress.getHostAddress());
            }
        }

        if (hostAddress == null) {
            throw new UnknownHostException("Cannot find ip address for hostname: " + hostName);
        }

        final EurekaInstanceConfigBean config = new EurekaInstanceConfigBean(inetUtils);
        config.setHostname(hostName);
        config.setIpAddress(hostAddress);
        return config;
    }
}

Lets check config and registry services.
Config:
eth2: 979c83c891fa / <IP_ADDRESS>
eth1: <IP_ADDRESS> / <IP_ADDRESS>
eth0: <IP_ADDRESS> / <IP_ADDRESS>
lo: <IP_ADDRESS> / <IP_ADDRESS>
lo: <IP_ADDRESS> / <IP_ADDRESS>
lo: localhost / <IP_ADDRESS>

<application>
  <name>CONFIG</name>
  <instance>
    <instanceId>config:b9701dfcc9f70001937fc53e9c3dbebb</instanceId>
    <hostName><IP_ADDRESS></hostName>
    <app>CONFIG</app>
    <ipAddr><IP_ADDRESS></ipAddr>
    <status>UP</status>
    <...>
  </instance>
</application>

Registry:
eth2: b2c8b40d16de / <IP_ADDRESS>
eth1: <IP_ADDRESS> / <IP_ADDRESS>
eth0: <IP_ADDRESS> / <IP_ADDRESS>
lo: <IP_ADDRESS> / <IP_ADDRESS>
lo: <IP_ADDRESS> / <IP_ADDRESS>
lo: localhost / <IP_ADDRESS>

<application>
  <name>REGISTRY</name>
  <instance>
    <instanceId>registry:07942e8b1a5cb63062d0caa7da9529f9</instanceId>
    <hostName><IP_ADDRESS></hostName>
    <app>REGISTRY</app>
    <ipAddr><IP_ADDRESS></ipAddr>
    <status>UP</status>
    <...>
  </instance>
</application>

Overlay network
[
    {
        "Name": "stack_default",
        "Id": "mnln7pi1y53chqcqfl5njmap3",
        "Created": "2017-11-23T05:37:47.3809552Z",
        "Scope": "swarm",
        "Driver": "overlay",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": null,
            "Config": [
                {
                    "Subnet": "<IP_ADDRESS>/24",
                    "Gateway": "<IP_ADDRESS>"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {
            "5210e982c5dac5f60295fe8b838c0125ef57af59ef674109aaf311976f48ea90": {
                "Name": "stack_rabbitmq.1.9ai08boa6ts8v6aanwd3fpjbv",
                "EndpointID": "32d9066c6daaee194ebd771ea959b8783d012744fd564ca791776aa43faf2d96",
                "MacAddress": "02:42:0a:00:00:07",
                "IPv4Address": "<IP_ADDRESS>/24",
                "IPv6Address": ""
            },
            "979c83c891fa39b4d1ed3c2c95de9209eba15217afde8a5f115ab68c609381fa": {
                "Name": "stack_config.1.84it2lq7j67vmxeibtvp0q7eq",
                "EndpointID": "aee5c133b60f4b21c92f4b6196ef1a29ec4803414348b2363b41e240fbf788d8",
                "MacAddress": "02:42:0a:00:00:03",
                "IPv4Address": "<IP_ADDRESS>/24",
                "IPv6Address": ""
            },
            "b2c8b40d16de987d87d6ec6cd44acb036246249c1316daf0511b7a07ad41236d": {
                "Name": "stack_registry.1.x7j9gzh4gu1id44j60d0ilx8e",
                "EndpointID": "0126dd6894d961bfe4a6f7788dc1f5a6d9b20bc53021798a5e4c41126f4053b1",
                "MacAddress": "02:42:0a:00:00:05",
                "IPv4Address": "<IP_ADDRESS>/24",
                "IPv6Address": ""
            }
        },
        "Options": {
            "com.docker.network.driver.overlay.vxlanid_list": "4097"
        },
        "Labels": {
            "com.docker.stack.namespace": "stack"
        },
        "Peers": [
            {
                "Name": "moby-e6a48a47d001",
                "IP": "<IP_ADDRESS>"
            }
        ]
    }
]

Remember, I asked about <IP_ADDRESS> / <IP_ADDRESS> or <IP_ADDRESS> / <IP_ADDRESS> which Eureka uses by default. Network interfaces show this one as Loopback. Actually every service in Docker Swarm has 3 loobpack and 3 eth interfaces. When we set to prefer 10.0 addresses in yaml configuration, we force to use this loopback, because the real eth hostname in hex format (979c83c891fa, b2c8b40d16de, etc).

Also if u don't wanna fight with Eureka, client-side load balancing and other things in Docker Swarm, it can work with other discovery backeds ☺  https://docs.docker.com/swarm/discovery

tank you @binakot for you sollutuion. I have one extension, if you want use eureka in HA and then provide eureka clinets all overlay ipadreses you can get it from dns request tasks.{{swarmEurekaServiceName}}. Eureka in HA need eureka.client.defaultZone.serviceUrl of another replicas, so this can by done something like this :
`
@Bean
@Primary
EurekaClientConfigBean eurekaClientConfigBean() {
EurekaClientConfigBean client = new EurekaClientConfigBean();
if ("bootstrap".equals(propertyResolver.getProperty("spring.config.name"))) {
// We don't register during bootstrap by default, but there will be another
// chance later.
client.setRegisterWithEureka(false);
} else {
String serviceName = propertyResolver.getProperty("eureka.client.swarmServiceName");
String eurekaPort = propertyResolver.getProperty("eureka.client.servicePort");
String myHostAddress = findHostAddress();
        List<String> eurekaIps = getDnsIps("tasks." + serviceName);
        //remove my ip maybe Iam eureka app
        eurekaIps.remove(myHostAddress);

        Map<String, String> eurekaServiceUrls = new HashMap<>();
        for (String eurekaServiceUrl : eurekaIps) {
            String url = "http://" + eurekaServiceUrl + ":" + eurekaPort + "/eureka/";
            eurekaServiceUrls.put("defaultZone", url);
            log.info("eureka service url" + url);
        }
        client.setServiceUrl(eurekaServiceUrls);
    }
    return client;
}

private static List<String> getDnsIps(String host) {
    List<String> ips = new ArrayList<>();
    try {
        for (InetAddress inetAddress : InetAddress.getAllByName(host)) {
            String hostAddress = inetAddress.getHostAddress();
            ips.add(hostAddress);
        }
    } catch (UnknownHostException e) {
        throw new RuntimeException("error", e);
    }
    return ips;
}

private static String getHostName() {
    try {
        return InetAddress.getLocalHost().getHostName();
    } catch (UnknownHostException e) {
        e.printStackTrace();
        throw new RuntimeException("error", e);
    }
}

private String findHostAddress() {
    String hostAddress = null;
    String hostName = getHostName();

    try {
        final Enumeration<NetworkInterface> networkInterfaces;
        networkInterfaces = NetworkInterface.getNetworkInterfaces();
        for (NetworkInterface netInt : Collections.list(networkInterfaces)) {
            for (InetAddress inetAddress : Collections.list(netInt.getInetAddresses())) {
                if (hostName.equals(inetAddress.getHostName())) {
                    hostAddress = inetAddress.getHostAddress();
                }
                log.info("inet {} : {} / {}", netInt.getName(), inetAddress.getHostName(), inetAddress.getHostAddress());
            }
        }
        if (hostAddress == null) {
            throw new IllegalStateException("Cannot find ip address for hostname: " + hostName);
        }
    } catch (SocketException e) {
        e.printStackTrace();
        throw new RuntimeException(e);
    }
    return hostAddress;
}

`

@meatfly Thank you 👍. You are right. At this moment I'm using Eureka with standalone mode in testing environment. The HA Eureka configuration will be my next step for the stage/production environment. I'll try to use your code snippets. When it will work, I write here my solution and experience.

try run service with --endpoint-mode dnsrr, it works for me. otherwise, i user a user define overlay network for all my services

If you configure a service to use DNS round-robin (DNSRR) service discovery, there is not a single virtual IP. Instead, Docker sets up DNS entries for the service such that a DNS query for the service name returns a list of IP addresses, and the client connects directly to one of these.

DNS round-robin is useful in cases where you want to use your own load balancer, such as HAProxy. To configure a service to use DNSRR, use the flag --endpoint-mode dnsrr when creating a new service or updating an existing one.

I use docker stack to deploy my services with user define overlay too. But how DNSRR can help to Eureka registration? With DNS ribbon's client-side load balancing won't work.

Service discovery is the mechanism Docker uses to route a request from your service’s external clients to an individual swarm node, without the client needing to know how many nodes are participating in the service or their IP addresses or ports.

The problem is in internal communications, when one service inside the cluster call the other one in the cluster. Server-side and Docker Swarm built-in load balancing doesn't work for internal commucations.

I haven't figure out why does this is work, but at first I had the same problem to yours, I deploy a service with 3 replacs but eureka only have one, then I add flag --endpoint-mode dnsrr , eureka has all the three, you may have try and tell me the results

After a few weeks trying to solve this situation, I finally get it to work! I don't know if this is the best way to handle, but worked for me.
I solve the issue deploying the Eureka Server like this:
version: '3.4'
services:
  eureka-1:
    image: xxxxx
    networks:
      my-net:
        aliases:
          - eureka
  eureka-2:
    image: xxxxx
    networks:
      my-net:
        aliases:
          - eureka
  eureka-3:
    image: xxxxx
    networks:
      my-net:
        aliases:
          - eureka
networks:
  my-net:
    driver: overlay

The aliases force an alternative host name to bind the ip address to the my-net network. So eureka server will resolve to 3 different IPs (using round robbin) and that way I could mark the correct interface (I've multiple interfaces in containers: db, elk, kafka, etc).
Now we need the eureka clients to use the same network to "talk to" eureka server:
version: '3.4'
services:
  my-client:
    image: xxxxx
    networks:
      - my-net
    environment:
      eureka.client.serviceUrl.defaultZone: http://eureka:8761/eureka

Note that my-net network is the same we bind the eureka alias before. Also, the service url points to the alias eureka (RR) instead of directly to the host.
To identify the correct interface, I'm using the SubnetUtils class from commons-net to check when the IP address from eureka client matches the IP address from eureka server. When both IPs has been in the same subnet, we found the correct IP address and hostname to register:
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.net.util.SubnetUtils;
import org.springframework.cloud.commons.util.InetUtils;
import org.springframework.cloud.netflix.eureka.EurekaClientAutoConfiguration;
import org.springframework.cloud.netflix.eureka.EurekaInstanceConfigBean;
import org.springframework.cloud.netflix.eureka.metadata.ManagementMetadataProvider;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;
import org.springframework.context.annotation.Profile;
import org.springframework.core.env.ConfigurableEnvironment;

import java.net.*;
import java.util.Collections;
import java.util.Enumeration;
import java.util.List;

@Profile("docker")
@Configuration
@Slf4j
public class DockerEurekaClientConfiguration extends EurekaClientAutoConfiguration {

	public DockerEurekaClientConfiguration(ConfigurableEnvironment env) {
		super(env);
	}

	@SuppressWarnings("SpringJavaInjectionPointsAutowiringInspection")
	@Bean
	@Primary
	public EurekaInstanceConfigBean eurekaInstanceConfigBean(
			InetUtils inetUtils,
			ManagementMetadataProvider managementMetadataProvider) throws MalformedURLException {
		EurekaInstanceConfigBean defaultResult = super.eurekaInstanceConfigBean(inetUtils, managementMetadataProvider);
		EurekaInstanceConfigBean result = null;
		try {
			List<String> servers = eurekaClientConfigBean().getEurekaServerServiceUrls(null);
			Enumeration<NetworkInterface> networkInterfaces = NetworkInterface.getNetworkInterfaces();
			external_loop:
			for (NetworkInterface networkInterface : Collections.list(networkInterfaces)) {
				if (networkInterface.getName().startsWith("lo")) {
					continue;
				}
				for (InterfaceAddress interfaceAddress : networkInterface.getInterfaceAddresses()) {
					if (interfaceAddress.getAddress() instanceof Inet4Address) {
						log.info(
								"Interface {}: {}/{}",
								networkInterface.getName(),
								interfaceAddress.getAddress(),
								interfaceAddress.getNetworkPrefixLength()
						);
						SubnetUtils subnet = new SubnetUtils(
								interfaceAddress.getAddress().getHostAddress() +
										"/" + interfaceAddress.getNetworkPrefixLength()
						);
						for (String server : servers) {
							URL serverUrl = new URL(server);
							InetAddress eurekaServerAddress = InetAddress.getByName(serverUrl.getHost());
							boolean matches = subnet.getInfo().isInRange(eurekaServerAddress.getHostAddress());
							log.info("Testing server {} ({}): {}", server, eurekaServerAddress.getHostAddress(), matches);
							if (matches) {
								log.info(
										"Found Interface {}: {} ({})",
										networkInterface.getName(),
										interfaceAddress.getAddress().getHostName(),
										interfaceAddress.getAddress().getHostAddress()
								);
								result = createEurekaInstanceConfigBean(inetUtils, defaultResult, interfaceAddress);
								break external_loop;
							}
						}
					} else {
						log.info(
								"Skipping IPv6 from Interface {}: {}/{}",
								networkInterface.getName(),
								interfaceAddress.getAddress(),
								interfaceAddress.getNetworkPrefixLength()
						);
					}
				}
			}
		} catch (Exception e) {
			log.error("Error while detecting eureka client address", e);
		}
		return result == null ? defaultResult : result;
	}

	private EurekaInstanceConfigBean createEurekaInstanceConfigBean(InetUtils inetUtils, EurekaInstanceConfigBean defaultResult, InterfaceAddress interfaceAddress) {
		EurekaInstanceConfigBean result;
		result = new EurekaInstanceConfigBean(inetUtils);
		result.setPreferIpAddress(true);
		result.setHostname(interfaceAddress.getAddress().getHostName());
		result.setIpAddress(interfaceAddress.getAddress().getHostAddress());
		result.setSecurePortEnabled(defaultResult.isSecurePortEnabled());
		result.setSecurePort(defaultResult.getSecurePort());
		result.setNonSecurePortEnabled(defaultResult.isNonSecurePortEnabled());
		result.setNonSecurePort(defaultResult.getNonSecurePort());
		result.setMetadataMap(defaultResult.getMetadataMap());
		if (result.isSecurePortEnabled()) {
			result.setInstanceId(result.getIpAddress() + ":" + result.getHostname() + ":" +
					result.getSecurePort());
			result.setSecureHealthCheckUrl("https://" + result.getIpAddress() + ":" +
					result.getSecurePort() + defaultResult.getHealthCheckUrlPath());
		} else {
			result.setInstanceId(result.getIpAddress() + ":" + result.getHostname() + ":" +
					result.getNonSecurePort());
			result.setHealthCheckUrl("http://" + result.getIpAddress() + ":" +
					result.getNonSecurePort() + defaultResult.getHealthCheckUrlPath());
			result.setStatusPageUrl("http://" + result.getIpAddress() + ":" +
					result.getNonSecurePort() + defaultResult.getStatusPageUrlPath());
		}
		return result;
	}
}

I hope this could help who got stuck!
Tested with:

Spring Boot 1.5.9.RELEASE
Spring Cloud Edgware.SR1
Docker 17.12.0-ce (Swarm Mode)

I was having a similar problem (don't know how similar the problem is, but reading the some of the above makes me think it is similar, so I am posting :P)
Basically I followed
https://spring.io/guides/gs/service-registration-and-discovery/
to create a eureka server and a eureka client. Played with them by launching few clients manually (without swarm) and server and all looked fine. Then wanted to start playing with swarm. My docker-compose was this:
version: "3.3"
services:
  eureka-server:
    image: eureka-server
    deploy:
      replicas: 1
    ports:
      - "8761:8761"

  eureka-client:
    image: eureka-client
    deploy:
      replicas: 3
    environment:
      - EUREKA_SERVER=eureka-server

nothing special, the images are created corresponding to the service in the tutorial and the client defined the eureka URL like the following, and hence the ENV var, EUREKA_SERVER
eureka.client.serviceUrl.defaultZone: http://${EUREKA_SERVER:localhost}:8761/eureka

launched the above with
docker stack deploy -c docker-compose.yml eureka

but all three client replicas were registering with same IP, in my case <IP_ADDRESS>. Eureka of course counted them as one, so only one client was found.
looked around the web (including this thread) like a headless chicken and tried few things (including setting the property, eureka.instance.hostName but no luck. Thereafter, found the endpoint_mode on docker:
https://docs.docker.com/compose/compose-file/#endpoint_mode
and setting to endpoint_mode: dnsrr does the trick. I won't pretend to know what DNS round robin does to the eureka client, but it allowed each of my client to register with unique IP (the container-id):
version: "3.3"
services:
  eureka-server:
    image: nexus.psenterprise.com:18079/eureka-server
    deploy:
      replicas: 1
    ports:
      - "8761:8761"

  eureka-client:
    image: nexus.psenterprise.com:18079/eureka-client
    deploy:
      replicas: 3
      endpoint_mode: dnsrr
    environment:
      - EUREKA_SERVER=eureka-server

hostname should be fine by

I am using Rancher, my stack are all SpringBoot netflix 1.5.10-RELEASE

Eureka
-Config Service
-API gateway with zuul.
-etc
since containers can communicate with each other and across hosts, I created a load balancer in rancher and connect it to Eureka, and also the API Gateway.
In all my services, I have the following settings which is pulled from the config server.
spring.cloud.inetutils.preferred-networks=10.0
eureka.instance.preferIpAddress=true
When my service register with Eureka, they all register with the docker container IP and since my Eureka and API Gateway services are also containerized, when a request goes thru the API Gateway, it grabs the service/ip from Eureka and forward the request to the corresponding container/service, it is able to route it via the Rancher/Kubernets network. The only thing I need to do is expose port 9002 as a public accessible port on every host, this also has the side effect of not having to open all the ports on the hosts that my services binds to. In Rancher, you can leave the public (host) port and container port blank and Rancher will assign a random container port, however, this is not really needed by our service but allows us to scale our service to more instances than hosts without running into port conflicts on the host. Additionally, I set the server.port on my config service to 0, this allows spring boot to assign a random port to my service which is what is registered with Eureka. So if the container IP assigned is <IP_ADDRESS> and the service port is 9999, In Eureka, you will see the containerid:servicename:port, if you hover over the service name, you will see the containerIP and not the containerId, this is how the API Gateway is able to route the request to the applicable container.
I also have a generic host file which maven uses when building the docker image:

Dockerfile
FROM openjdk:8-jdk-alpine
MAINTAINER Paul Gobin<EMAIL_ADDRESS>VOLUME /tmp
ARG JAR_FILE
ADD target/${JAR_FILE} app.jar
ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-jar","/app.jar"]
This seem to work really well for me and does not require any special Eureka registration bean.

Hi @SaeedMasoumi. This is my application.yml from Eureka:
server:
  port: 8761

eureka:
  client:
    registerWithEureka: false
    fetchRegistry: false
  instance:
    preferIpAddress: false
  server:
    waitTimeInMsWhenSyncEmpty: 5
    enableSelfPreservation: false


UI Issue when changing default values, validation passes, success message but new version is not added

Issue: Changing default value in UI, validates but doesn't save in db.
Version: 0.5.1
Steps to reproduce:

Add in UI a new Schema with Backwards Compatibility
e.g.

{"namespace": "com.data_platform.events",
 "type": "record",
 "name": "GenericEvent",
 "doc": "https://github.com/model",
 "fields": [
 {
   "name": "version_with_default",
   "type": [
    "string",
    "null"
   ],
   "default": "v.1"
  }
 ]
}

Update Schema with new default value: e.g. Change v.1 to v.2

{"namespace": "com.data_platform.events",
 "type": "record",
 "name": "GenericEvent",
 "doc": "https://github.com/model",
 "fields": [
 {
   "name": "version_with_default",
   "type": [
    "string",
    "null"
   ],
   "default": "v.2"
  }
 ]
}

Press Validate. It validates fine:

"Schema is valid"

Press Save. It saves fine and the confirmation message appears:

"Version updated successfully"

However, no new version is created.
So either validation should not pass or backend should save new schema version to db correctly.

Got that issue too.


Dropping a component in a gap inserts it at the start/end of its parent

In a gap between siblings, the mouse pointer is over the parent element, so the actual insertion point will be either the start or the end of the parent, whichever is closer. If you want to insert something into a gap, you have to drop it NOT on the gap, but on one of the adjacent siblings.
This obviously isn't intuitive, and the larger the gap is, the worse it gets.

Even worse, there's actually no way to insert anything between two adjacent containers. For example, create a Page with two empty Rows, and try to insert something between them. When you hover over a row, it would insert into the row, and when you hover over the gap, it would insert at the top or bottom of the page.


[BEAM-10825] Fix null pointer for non-Jenkins build.

R: @udim
cc: @pulasthi

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Choose reviewer(s) and mention them in a comment (R: @username).
[ ] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

Thanks for the fix, this also broke the linkage checker.


Expose the serialization API

What kind of change does this PR introduce?
feature
Did you add tests for your changes?
no
Does this PR introduce a breaking change?
no
What needs to be documented once your changes are merged?
webpack.util.serialization is now exposed.

Thanks


Master fix#7

Fix for issue #7 and fix compatibility with latest boostrap.
Closes #7.

Looks great, thanks!

May I ask you when next release is planned?

@YuraDubensky My vague plan is "soon". I want get your other merge request in, and fix #12 before releasing a new one.
Regarding this change; I had not pulled it to run it locally until after it was merged, but there seems to be a few issues: the linter died on a tabs vs spaces issue (easy fix), but then most of the dropdowns on the demo page stopped working. (The ones appending to body continue to work.) Is there a chance you could look at that? It seems to have been introduced by your change; I'm hesitant to make a release until that's resolved.

@Morgul Give me some time to setup demo page and try it out. But don't think my fix broke it, since it is working fine in my project with those changes. But I will verify and revert

@Morgul Are using 2 spaces indentation in sources?

...yes. (I'm not happy about it either, this is a fork of the original, and I haven't had time to redo their build system, so we're stuck with the original formatting decisions.)
My recommendation is to run grunt before doing a merge request. That will run all unit tests, and linting.

Also, @YuraDubensky, I don't want a revert; your change is a very good one. It's entirely possible the dropdowns are wrong on the demo page. I just want the make sure the demo page is working, and there wasn't any unintended side effects. :)

@Morgul I have fixed eslint errors for my 2 fixes and created a new pull request
https://github.com/Morgul/ui-bootstrap4/pull/16


[RUNTIME] Add device specific timers

This PR adds device specific timers for use in profiling. I've included CPU (std::steady_clock), CUDA (cudaEvent), and ROCM timers (rocmEvent). On devices that do no have a timer supplied, we fall back to using the CPU timer with TVMSychronize.
@areusch @hlu1 @jwfromm @icemelon9

I think in the future i'd like to see us move to a log-based approach where each event of interest and timer used gets a unique id assigned to it at compile time. but I think this is a good change in the meantime, as that system is way more complicated and would need an RFC or two.

I've switch from a function based timing approach to an object based one. This means that the api is substantially different. @areusch and @tqchen you will probably have to re-review. Sorry about that.

cc @junrushao1994@comaniac @hzfan @d-smirnov please help to review this PR

Thanks @tkonolige @areusch @hzfan !


Fix/201 tidy up reviewers assignment

Let's merge after #211 with the corresponding tweaks

:shipit:


fix: release job image

What
Updates release job image to one that works with AWS CLI without issue (ref: AWS CLI calls broken during change from 2.169.1 to 2.262.1)

:tada: This PR is included in version 0.5.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


[AJ-591] abstract (some) ajax calls out of DataTable and DataTableActions

There should be zero user-visible changes in this PR; it is purely a refactor. This refactor is in prep for the #3429 feature branch, which adds a bunch of WDS support. I'm separating the refactor out into this PR to make the #3429 branch smaller and more manageable.
At a high level, we want the UI's data tables to connect to both the current Rawls Entity Service backend or a WDS backend. But, the DataTable and DataTableActions components were tightly coupled to Entity Service.
In this PR:

create a TypeScript DataProvider definition, which includes model classes and method signatures for backend API calls
create a TypeScript EntityServiceDataProvider, which implements DataProvider and extracts the Ajax calls for querying data table pages and deleting tables out of DataTable and DataTableActions
modify DataTable and DataTableActions to call the methods on a supplied DataProvider. In this case the supplied provider is EntityServiceDataProvider.
make some UI controls conditionally display based on whether or not the current DataProvider supports the feature. In this PR, EntityServiceDataProvider supports all features, so there is no functional change.

Coming up in a future PR (probably #3429): allow DataTable and DataTableActions to use either EntityServiceDataProvider or WDSDataProvider.

converting to draft - the test failure is real, I will fix and then put this back into ready

test fixed; ready for review.

Thanks @davidangb, some nice cleanup/simplification in those new commits. Just one question about eslint, but otherwise LGTM.


Make MacOS.language less opinionated.

[x] Have you followed the guidelines in our Contributing document?
[x] Have you checked to ensure there aren't other open Pull Requests for the same change?
[x] Have you added an explanation of what your changes do and why you'd like us to include them?
[ ] Have you written new tests for your changes? Here's an example.
[x] Have you successfully run brew tests with your changes locally?

This adds a languages method which returns an array, which in turn is called by language.

👍 thanks!

Seems like the Mavericks test-bot is using zh-Hans as language. So the test in the regex should also match http://www.unicode.org/iso15924/iso15924-codes.html.

@Homebrew/cask, could someone look over this?

Current coverage is 83.72% (diff: 100%)

Merging #906 into master will increase coverage by <.01%

@@             master       #906   diff @@
==========================================
  Files           340        340
  Lines         13948      13950     +2
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
+ Hits          11677      11679     +2
  Misses         2271       2271
  Partials          0          0

Powered by Codecov. Last update a54c968...fda249e

I'm trying to work on this, but I'm having a ton of trouble getting things working. The develop_brew_cask command isn't working, and I can't run the tests because of missing environment variables. I'm on Gitter if anyone can help me.

Never used the develop_brew_cask command, but this will probably need to be rewritten or be removed, as developing only Cask will not work as before anyways.

Current coverage is 83.72% (diff: 100%)

Merging #906 into master will increase coverage by 8.21%

@@             master       #906   diff @@
==========================================
  Files           360        340     -20
  Lines         15469      13952   -1517
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
+ Hits          11680      11681      +1
+ Misses         3789       2271   -1518
  Partials          0          0

Powered by Codecov. Last update c419f92...fda249e

Current coverage is 83.59% (diff: 100%)

Merging #906 into master will increase coverage by 8.09%

@@             master       #906   diff @@
==========================================
  Files           360        340     -20
  Lines         15469      13981   -1488
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
+ Hits          11680      11688      +8
+ Misses         3789       2293   -1496
  Partials          0          0

Powered by Codecov. Last update c419f92...fdcf8b8

@reitermarkus 🚢?

Would still like to get some feedback from @Homebrew/cask.

I'd love to help, but I can't get things running locally. I'd also love to have some tests around this. I also don't like that we have to make :default the last declaration.

I also don't like that we have to make :default the last declaration.

This is not the case anymore. This is done using the language_eval method now.

:-1:

@reitermarkus I made a slightly different DSL: https://github.com/reitermarkus/brew/compare/os-language...mwean:mw/language-dsl I wanted to avoid having a separate default block, and I wanted to make it easy to make the language name different from the pattern.

@mwean, I like the idea of only allowing strings in the DSL, so I have added a Locale class to enable matching locales by string only instead of regexes and strings. Still needs to be integrated into the language DSL.

I think this is pretty much ready. @mwean, I changed it mostly to your DSL, except for the name: parameter, as I think simply using the return value of the block is more flexible, and more in line with url do. @jawshooah, please also have a final look.


False positive "UndefinedVariable" on array destructure (list syntax)

PHPMD version: 2.8.2
PHP Version: 7.3.0
Installation type: composer
Operating System / Distribution & Version: Alpine:latest

Current Behavior
PHPMD falsely identifies list syntax as "undefined variables".
Given the code:
[$example] = [10];
echo $example; // prints 10

It gives the following violation
| VIOLATION | Avoid using undefined variables such as $example which will lead to PHP notices.

Expected Behavior
No violations on list syntax
Steps To Reproduce:
See current behavior
Checks before submitting

[x] Be sure that there isn't already an issue about this. See: Issues list
[x] Be sure that there isn't already a pull request about this. See: Pull requests
[x] I have added every step to reproduce the bug.
[x] If possible I added relevant code examples.
[x] This issue is about 1 bug and nothing more.
[x] The issue has a descriptive title. For example:  "JSON rendering failed on Windows for filenames with space".

@jannes-io thanks for your report. We just release version 2.9.0 do you have the possibility to check it with that version so we are sure it isn't already fixed?

@tvbeek issue fixed in 2.9.0, excellent!

@jannes-io thanks for checking and reporting back 😃


Error 403 (Forbidden)!!1 for google scholar

When searching https://scholar.google.com, I'll get error messages as below.

Your client does not have permission to get URL / from this server. (Client IP address: x.x.x.x)
Please see Google's Terms of Service posted at http://www.google.com/terms_of_service.html

When searching https://scholar.google.com, I'll get error messages as below.

Your client does not have permission to get URL / from this server. (Client IP address: x.x.x.x)
Please see Google's Terms of Service posted at http://www.google.com/terms_of_service.html

@rusterx 请问您解决了这个问题吗？貌似最近被谷歌学术禁的ip有点多。

仍然是这样的，如果vps支持ipv6，改一下hosts文件即可，但是我用的不支持，所以无奈了。
------------------ 原始邮件 ------------------
发件人<EMAIL_ADDRESS>发送时间: 2020年3月31日(星期二) 下午3:04
收件人<EMAIL_ADDRESS>抄送<EMAIL_ADDRESS><EMAIL_ADDRESS>主题: Re: [v2ray/v2ray-core] Error 403 (Forbidden)!!1 for google scholar (#2395)
@rusterx 请问您解决了这个问题吗？貌似最近被谷歌学术禁的ip有点多。
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.

现在有解决办法了吗，怎么搞呢。急哭了

请问如何解决？

换别的VPS，你的服务器的IP被Google Scholar列黑名单了。
On Fri, Apr 24, 2020 at 10:39 AM lihanyu1204<EMAIL_ADDRESS>wrote:

请问如何解决？
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/v2ray/v2ray-core/issues/2395#issuecomment-618769601,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABPLTYIHWEMJ4EBPYH4KR2TROD3VDANCNFSM4LWLZSRQ
.

换别的VPS，你的服务器的IP被Google Scholar列黑名单了。
…
On Fri, Apr 24, 2020 at 10:39 AM lihanyu1204 @.***> wrote: 请问如何解决？ — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#2395 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABPLTYIHWEMJ4EBPYH4KR2TROD3VDANCNFSM4LWLZSRQ .

怎么换vps，重新换个代理吗

一直vps都是自用的，结果就出现了被谷歌学术封禁的现象。不知道是因为被别的ip连带封禁的，还是因为用的233boy安装脚本有后门，自己安装又比较嫌麻烦。

换一个search engine吧


Migrate from guava to jdk

Codecov Report

Merging #5255 into master will decrease coverage by 0.11%.
The diff coverage is 83.87%.

@@             Coverage Diff              @@
##             master    #5255      +/-   ##
============================================
- Coverage      82.6%   82.48%   -0.12%
+ Complexity    11046    11027      -19
============================================
  Files          1330     1330
  Lines         44968    44964       -4
  Branches       5626     5628       +2
============================================
- Hits          37146    37090      -56
- Misses         5214     5250      +36
- Partials       2608     2624      +16

Impacted Files
Coverage Δ
Complexity Δ

...rc/main/java/io/crate/blob/BlobTransferTarget.java
86.75% <100%> (-0.67%)
25 <0> (-1)

...src/main/java/io/crate/module/CrateCoreModule.java
93.75% <77.77%> (-0.19%)
2 <0> (ø)

...i/src/main/java/io/crate/module/AdminUIModule.java
92.85% <81.81%> (-0.48%)
2 <1> (ø)

...pp/src/main/java/io/crate/plugin/PluginLoader.java
53.29% <88.88%> (+0.29%)
26 <10> (ø)
:arrow_down:

...n/java/io/crate/exceptions/JobKilledException.java
0% <0%> (-66.67%)
0% <0%> (-1%)

.../main/java/io/crate/planner/PositionalOrderBy.java
41.02% <0%> (-41.03%)
9% <0%> (-4%)

...o/crate/analyze/SafeExpressionToStringVisitor.java
83.33% <0%> (-16.67%)
7% <0%> (-1%)

...rc/main/java/io/crate/analyze/UnboundAnalyzer.java
70% <0%> (-15%)
2% <0%> (ø)

...crate/executor/transport/TransportShardAction.java
77.5% <0%> (-12.5%)
8% <0%> (-1%)

...ration/collect/collectors/OrderedDocCollector.java
90% <0%> (-10%)
5% <0%> (-1%)

... and 15 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3be7420...3f7e4a5. Read the comment docs.


Add check for medvram-sdxl memory optimization

this is unsafe as only a1111 has that option and system info is used on different webui ports.


AWS RDS Cluster has undefined outputs

What happened?
The outputs of the RDS cluster instance are undefined, even though the typescript declarations mark them as defined string outputs:

https://github.com/pulumi/pulumi-aws/blob/58cb20764dbd721d91c53cd932b4f495de0aefa5/sdk/nodejs/rds/cluster.ts#L251
https://github.com/pulumi/pulumi-aws/blob/58cb20764dbd721d91c53cd932b4f495de0aefa5/sdk/nodejs/rds/cluster.ts#L339

Steps to reproduce
const masterUsername = "TestUser";

const cluster = new aws.rds.Cluster(
      "mycluster",
      {
        clusterIdentifier,
        backupRetentionPeriod,
        skipFinalSnapshot: true,
        engine: "aurora-postgresql",
        engineMode: "provisioned",
        engineVersion: dbEngineVersion,
        masterUsername,
        masterPassword: this.masterPassword.result,
        port,
        storageEncrypted: true
      },
      {
        deleteBeforeReplace: true,
      }
    );

pulumi.interpolate`Your database name is '${cluster.databaseName}' and your master user is '${cluster.masterUsername}'`.apply(msg => console.log(msg));

Expected Behavior
Outputs in log:

Your database name is 'postgres' and your master user is 'TestUser'

Actual Behavior
Outputs in log with undefined values:

Your database name is 'undefined' and your master user is 'undefined'

Output of pulumi about
CLI
Version      3.38.0
Go Version   go1.19
Go Compiler  gc
Plugins
NAME    VERSION
nodejs  unknown
Host
OS       darwin
Version  12.5
Arch     arm64
This project is written in nodejs: executable='/Users/gsuess/.nvm/versions/node/v18.7.0/bin/node' version='v18.7.0'
Additional context
This is a trivial bug, so I did not try to isolate it, but let me know if you have trouble reproducing this. I can then then isolating the issue.
Contributing
Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

Hey @gsuess
I will have a look at the upstream provider and see what I can find - it could be due to the fact that the TF resource is used for many different scenarios :/
Paul

CLI
Version      3.130.0
Go Version   go1.22.6
Go Compiler  gc

Plugins
KIND      NAME    VERSION
resource  aws     6.52.0
resource  awsx    2.15.0
resource  docker  4.5.5
resource  docker  3.6.1
language  nodejs  unknown

Host
OS       darwin
Version  14.6.1
Arch     arm64

This project is written in nodejs: executable='/Users/anton/bin/node' version='v18.18.2'

Current Stack: anton-pulumi-corp/aws-2106/dev

TYPE                     URN
pulumi:pulumi:Stack      urn:pulumi:dev::aws-2106::pulumi:pulumi:Stack::aws-2106-dev
pulumi:providers:aws     urn:pulumi:dev::aws-2106::pulumi:providers:aws::default_6_52_0
aws:rds/cluster:Cluster  urn:pulumi:dev::aws-2106::aws:rds/cluster:Cluster::mycluster4

Found no pending operations associated with dev

Backend
Name           pulumi.com
URL            https://app.pulumi.com/anton-pulumi-corp
User           anton-pulumi-corp
Organizations  anton-pulumi-corp, moolumi, demo, pulumi
Token type     personal

Dependencies:
NAME            VERSION
@types/node     18.19.50
typescript      5.6.2
@pulumi/aws     6.52.0
@pulumi/awsx    2.15.0
@pulumi/pulumi  3.134.0

Pulumi locates its logs in /var/folders/gd/3ncjb1lj5ljgk8xl5ssn_gvc0000gn/T/com.apple.shortcuts.mac-helper// by default

As of today this seems to be working right:
import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";
import * as awsx from "@pulumi/awsx";

const masterUsername = "TestUser";

const cluster = new aws.rds.Cluster(
      "mycluster4",
      {
        clusterIdentifier: "example4",
        // backupRetentionPeriod,
        skipFinalSnapshot: true,
        engine: "aurora-postgresql",
        engineMode: "provisioned",
        engineVersion: "13.12",
        databaseName: "test",
        masterUsername,
        masterPassword: "passsword",
          // port,
        storageEncrypted: true
      },
      {
        deleteBeforeReplace: true,
      }
    );

pulumi.interpolate`Your database name is '${cluster.databaseName}' and your master user is '${cluster.masterUsername}'`.apply(msg => console.log(msg));

Gets:
Diagnostics:
  pulumi:pulumi:Stack (aws-2106-dev):
    Your database name is 'test' and your master user is 'TestUser'


Parameter basename in function routerForBrowser does not work if it ends with a slash

Hi! It is probably just a matter of making the documentation more explicit. I'll let you qualify this issue.
Anyway I lost a fair amount of time figuring out why my routing was not working properly. My basename was the result of an expression evaluating to "/".

I have had this issue as well. It should at least be mentioned in the docs.


Output window

Add an output window to show details about Monokle background tasks, such as executed terminal commands
ex: git commands, helm commands.
Users can easily inspect output messages from within Monokle

Avoid adding messages that require acknowledgement:

Show background tasks in the interface (i.e. Calibre showing there is something happening in the background)
Add it to the events, without requiring ack from users


Single quote error in Cypher

We are having error in cypher property match for single quote.
cypher = "MATCH (h:Hub {_id: '#{self.id}'})-[:NODES]->(n:Node {name: '#{name}'}) RETURN n"
result = Neo4j::Session.query(cypher)

throws error:
(n:Node {name: 'Festival d'Automne'})`
Neo4j::Server::CypherResponse::ResponseError (Invalid input 'u': expected 'n/N

But whe we escape the single quote it works fine:
name = name.gsub("'", %q(\\\'))
cypher = "MATCH (h:Hub {_id: '#{self.id}'})-[:NODES]->(n:Node {name: '#{name}'}) RETURN n"

Cypher doc says the String Literals can be escaped. Should the neo4jrb escape these by default?
http://neo4j.com/docs/stable/cypher-expressions.html

It's possible that the escape process isn't evaluating strings given to the
'query' method. Try using 'query.match()', see if it's any different. I
know that if you use parameters, which you absolutely should, quotes will
be escaped, so that could be a workaround for the time being.
On Thursday, July 16, 2015, Burak Arikan<EMAIL_ADDRESS>wrote:

We are having error in cypher property match for single quote.
cypher = "MATCH (h:Hub {_id: '#{self.id}'})-[:NODES]->(n:Node {name: '#{name}'}) RETURN n"
result = Neo4j::Session.query(cypher)
throws error:
(n:Node {name: 'Festival d'Automne'})`
Neo4j::Server::CypherResponse::ResponseError (Invalid input 'u': expected 'n/N
But whe we escape the single quote it works fine:
name = name.gsub("'", %q(\'))
cypher = "MATCH (h:Hub {_id: '#{self.id}'})-[:NODES]->(n:Node {name: '#{name}'}) RETURN n"
However Cypher docs says the String Literals are escaped by default, we
should not need to escape them. Would this be a bug in the neo4jrb library.
http://neo4j.com/docs/stable/cypher-expressions.html
—
Reply to this email directly or view it on GitHub
https://github.com/neo4jrb/neo4j/issues/873.

But yes, it does sound like a bug and I will take a look sometime today.
On Thursday, July 16, 2015, Chris Grigg<EMAIL_ADDRESS>wrote:

It's possible that the escape process isn't evaluating strings given to
the 'query' method. Try using 'query.match()', see if it's any different. I
know that if you use parameters, which you absolutely should, quotes will
be escaped, so that could be a workaround for the time being.
On Thursday, July 16, 2015, Burak Arikan<EMAIL_ADDRESS><javascript:_e(%7B%7D,'cvml','notifications@github.com');>> wrote:

We are having error in cypher property match for single quote.
cypher = "MATCH (h:Hub {_id: '#{self.id}'})-[:NODES]->(n:Node {name: '#{name}'}) RETURN n"
result = Neo4j::Session.query(cypher)
throws error:
(n:Node {name: 'Festival d'Automne'})`
Neo4j::Server::CypherResponse::ResponseError (Invalid input 'u': expected 'n/N
But whe we escape the single quote it works fine:
name = name.gsub("'", %q(\'))
cypher = "MATCH (h:Hub {_id: '#{self.id}'})-[:NODES]->(n:Node {name: '#{name}'}) RETURN n"
However Cypher docs says the String Literals are escaped by default, we
should not need to escape them. Would this be a bug in the neo4jrb library.
http://neo4j.com/docs/stable/cypher-expressions.html
—
Reply to this email directly or view it on GitHub
https://github.com/neo4jrb/neo4j/issues/873.

I'm not sure how this is a bug.  We can't take arbitrary query strings and escape the string literals that we find.  You should definitely use parameters.  They are faster, more secure if this data is coming from users, and easier because there is no escaping to worry about.
Also if you use the Query method chaining API parameters will be used for you.  Like this:
cypher = "MATCH (h:Hub {_id: '#{self.id}'})-[:NODES]->(n:Node {name: '#{name}'}) RETURN n"
result = Neo4j::Session.query.match("(h:Hub)-[:NODES]->(n:Node)").where(h: {_id: self.id}, n: {name: name}).return(:n).exec

Or:
result = Neo4j::Session.query.match(h: {Hub: {_id: self.id}}, n: {Node: {name: name}}).match("h-[:NODES]->n").return(:n).exec

You can also replace the return(:n).exec with pluck(:n)

Thoughts?  I say we close, but I could be misunderstanding the issue

Closing for now.  Please let me know if I'm misunderstanding, though


Questions about the results of your experiment.

hi. I was wondering about the results of your experiments. In section 3.1, in the codebook size ablation study, you show that the usage increases from 75%->97% when the size increases from 8192 to 16384. It's very interesting to see that at 4096 size, the usage is 100%, at 8192 size, the usage drops to 75%, and then rises to 97% at 16384 size. Do you guys have any insights or conclusions on this result?

Hi~
We are now re-running this set of ablation study to see whether these results are reproducing. We will update here soon, please stay tuned.

Hi. I was also wondering the experiments about code book dimension. I have trained the VQGAN with the downsamlping of 4, however, I do not get the similar result. Do you have the similar results when the downsample is 8 or 4? Thanks.


Enable Bazel build with clang

Brief feature description
Enable using clang building for bazel. This is nice to have.
Detailed information
The commits in this branch: https://github.com/phongt/iceoryx/tree/bazel-build-clang-rules. Should I raise a PR for reviewing?

@phongt Yes please, this is great!

@elfenpiff  Raised PR https://github.com/eclipse-iceoryx/iceoryx/pull/1999


ci: included github plugin

:tada: This PR is included in version 1.1.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Missing Tinq icon

The icon for the TinQ fuel station is missing --> https://www.tinq.be

It seems to be an encoding issue:
IMG src from the carbu.com site: https://carbucomstatic-5141.kxcdn.com/brandLogo/326_Capture d’écran 2021-09-27 à 11.11.24.png
IMG src in HA: https://carbucomstatic-5141.kxcdn.com/brandLogo/326_Capture dâécran 2021-09-27 à 11.11.24.png
Why they didn't properly rename the file rather than using a screenshot name from MacOS...

I have tried many different solutions to encode this url, but still it was not working. For now, I have hard-coded the tinq logo, I didn't find any other solution so far and lost a lot of time on it...


plot_circle requires a profile to assign color; problem for JuMP data

Attempting to plot ShapeJump interactions in a circular layout, rather than as an Arc Plot. Since shapejump data does not generate a profile, the function can't find the color and fails. See below:

AttributeError                            Traceback (most recent call last)
Cell In [23], line 2
1 #Plotting WTT-RT1 as a circ; subtracted background
----> 2 plot_c1 = WT_RT1_norm.plot_circle(
3     ct="ct",
4     interactions="shapejump",
5     interactions_filter={"Percentile_ge":0.95})
File ~/Documents/Jupyter/RNAvigate/rnavigate/rnavigate.py:488, in Sample.plot_circle(self, **kwargs)
487 def plot_circle(self, **kwargs):
--> 488     return plot_circle_multisample([self], **kwargs)
File ~/Documents/Jupyter/RNAvigate/rnavigate/rnavigate.py:736, in plot_circle_multisample(samples, ct, comp, interactions, interactions_filter, interactions2, interactions2_filter, profile, label, **kwargs)
733     if interactions2 is not None:
734         sample.filter_interactions(interactions2, ct,
735                                    **interactions2_filter)
--> 736     plot.add_sample(sample, ct=ct, comp=comp, interactions=interactions,
737                     interactions2=interactions2, profile=profile,
738                     label=label, **kwargs)
739 return plot
File ~/Documents/Jupyter/RNAvigate/rnavigate/plots/plots.py:36, in Plot.add_sample(self, sample, **kwargs)
34     if key not in self.pass_through:
35         kwargs[key] = sample.get_data_list(kwargs[key])
---> 36 self.plot_data(**kwargs)
File ~/Documents/Jupyter/RNAvigate/rnavigate/plots/circle.py:47, in Circle.plot_data(self, ct, comp, interactions, interactions2, profile, label)
44 # self.add_sequence(ax, ct.sequence, yvalue=0.5)
45 # self.plot_profile(ax, profile, ct)
46 ax.set_title(label)
---> 47 nuc_colors = profile.get_colors("profile", profile=profile)
48 ax.scatter(self.x, self.y, marker='o', c=nuc_colors)
49 self.i += 1
AttributeError: 'NoneType' object has no attribute 'get_colors'

I wrote this part badly. I will prioritize fixing this. For now though, the only way around it is to give your sample a fake profile.
WT_RT1_norm.set_data(
    name="profile",
    instantiator=rnav.data.Profile,
    seq_source="shapejump",
    colors=['black']*WT1_RT1_norm.data["shapejump"].length)

Let me know if this works.

This almost works
NameError: name ‘rnav’ is not defined
Swapping rnav for MaP (imported rnavigate as MaP) provides a different error:
[cid:B7D475F8-6D35-451A-B666-B5135C3FAF82]
On Oct 25, 2022, at 3:25 PM, Patrick Irving<EMAIL_ADDRESS>wrote:
I wrote this part badly. I will prioritize fixing this. For now though, the only way around it is to give your sample a fake profile.
WT_RT1_norm.set_data(
name="profile",
instantiator=rnav.data.Profile,
seq_source="shapejump",
colors=['black']*WT1_RT1_norm.data["shapejump"].length)
Let me know if this works.
—
Reply to this email directly, view it on GitHubhttps://github.com/Weeks-UNC/RNAvigate/issues/12#issuecomment-1291033597, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ATSBUDIQQBVNDNMDMCRBVSDWFAX4JANCNFSM6AAAAAAROGABHA.
You are receiving this because you authored the thread.Message ID: @.***>

Yeah, that was close.
The first error: NameError: name ‘rnav’ is not defined likely happened because you forgot to run the cell containing import rnavigate as rnav.
The second error: We're trying to create a Profile using no data and the bare minimum arguments, but I forgot to add cmap. Try this:
WT_RT1_norm.set_data(
    name="profile",
    instantiator=rnav.data.Profile,
    seq_source="shapejump",
    cmap="black"
    colors=["black"]*WT1_RT1_norm.data["shapejump"].length)

Added some of your requested features in the latest commit. Let me know how it works for you. These were all very useful suggestions, and RNAvigate is better for it.

The primers annotation, which is basically the same as spans, but they are directional. This works with other plot types as well.
Circle plots can display annotations.
Circle plots accept the argument seq_source. This makes it even more flexible than other plot types. I'll probably be adding this feature to all plots in the future. Including the ability to put in any sequence you want here (e.g. aligning the data to the sequence with or without introns).
Circle plots use the same color options as secondary structure diagrams: colors defines the nucleotide coloring scheme. This was previously taken from profile, so you don't need a profile anymore. apply_color_to can be "sequence" or "background".

import rnavigate as rnav
example = rnav.Sample(
    sample="example",
    shapejump={
        "filepath": "data/example-rnasep-deletions.txt",
        "fasta": "data/RNaseP-noSC.fasta"},
    primers={
        "seq_source": "shapejump",
        "primer_list": [[1,15], [240, 225]],
        "color": "red",
    },
    spans={
        "seq_source": "shapejump",
        "span_list": [[40, 80], [120, 160]],
        "color": "red"})

plot = example.plot_circle(
    seq_source='shapejump',
    interactions="shapejump",
    interactions_filter={
        "Percentile_ge":0.95
    },
    colorbar=False,
    annotations=["primers", "spans"])
plot.save('circle-test.svg')


Remove sudo from makefile and test files

Describe the bug
Currently we use sudo inside the makefile, and some of the tests in order to create/modify/delete files under /etc:
https://github.com/project-zot/zot/blob/main/Makefile#L51
Also the dedupe tests added in:
https://github.com/project-zot/zot/commit/53b5fa6493af0e8510001d8b07a9efe2cf7db584
This means we cannot run these make targets / tests unless we have sudo installed and the user has permissions to sudo.
On some operating systems there may not even be a sudo command available.
To Reproduce
Steps to reproduce the behavior:
Execute the make target called test.
Expected behavior
We would be able to execute most of the tests without needing the sudo command.
Desktop (please complete the following information):

OS: Linux / MacOS

Proposed solution
Separate the the tests needing sudo from the tests not needing sudo in separate make targets, and execute the make command using sudo if necessary.
To separate the tests we could add a new build tag for go tests requiring sudo.

For Catalin Hornar.