make cluster-sync not working.

Is this a BUG REPORT or FEATURE REQUEST?:
/kind bug
What happened:
after make cluster-up
make cluster-sync fails .
The images are not build and hence while pushing images to local registry it fails.
What you expected to happen:
Images should be build and visible in host machine docker images output
How to reproduce it (as minimally and precisely as possible):
Anything else we need to know?:
Log
$ make cluster-sync
PUSH_LOG_FILE= ./hack/cluster-build.sh
Building ...
[+] Building 0.2s (9/9) FINISHED
 => [internal] load .dockerignore                                                                                0.0s
 => => transferring context: 2B                                                                                0.0s
 => [internal] load build definition from Dockerfile                                                                                0.0s
 => => transferring dockerfile: 38B                                                                                0.0s
 => [internal] load metadata for docker.io/kubevirt/builder@sha256:08b4db91e5f6272004892c6174129172c1bef5c6906bde9f8414cb945a588e12                                                       0.0s
 => [1/4] FROM docker.io/kubevirt/builder@sha256:08b4db91e5f6272004892c6174129172c1bef5c6906bde9f8414cb945a588e12                                                                         0.0s
 => => resolve docker.io/kubevirt/builder@sha256:08b4db91e5f6272004892c6174129172c1bef5c6906bde9f8414cb945a588e12                                                                         0.0s
 => [internal] load build context                                                                                0.0s
 => => transferring context: 33B                                                                                0.0s
 => CACHED [2/4] RUN     mkdir -p /go &&     source /etc/profile.d/gimme.sh &&     go get github.com/mattn/goveralls &&     go get -u github.com/golang/mock/gomock &&     go get -u git  0.0s
 => CACHED [3/4] RUN pip install j2cli && pip3 install operator-courier==1.3.0                                                                                0.0s
 => CACHED [4/4] COPY rsyncd.conf /etc/rsyncd.conf                                                                                0.0s
 => exporting to image                                                                                0.0s
 => => exporting layers                                                                                0.0s
 => => writing image sha256:6e512c03eb7b69fd0cca43aa2266799c72fead3ffa535c0390cfd7ae493d0ac8                                                                                0.0s
 => => naming to docker.io/library/kubevirt                                                                                0.0s
go version go1.11.5 linux/amd64

go version go1.11.5 linux/amd64
INFO: Build option --define has changed, discarding analysis cache.
INFO: Analysed 54 targets (10 packages loaded, 13248 targets configured).
INFO: Found 54 targets...
INFO: Elapsed time: 2.877s, Critical Path: 1.27s
INFO: 0 processes.
INFO: Build completed successfully, 1 total action
INFO: Build option --platforms has changed, discarding analysis cache.
INFO: Analysed target //:build-virtctl (0 packages loaded, 8939 targets configured).
INFO: Found 1 target...
Target //:build-virtctl up-to-date:
  bazel-genfiles/virtctl-copier
INFO: Elapsed time: 0.936s, Critical Path: 0.10s
INFO: 1 process: 1 processwrapper-sandbox.
INFO: Build completed successfully, 2 total actions
INFO: Build completed successfully, 2 total actions
INFO: Build option --platforms has changed, discarding analysis cache.
INFO: Analysed target //:build-virtctl (0 packages loaded, 8939 targets configured).
INFO: Found 1 target...
Target //:build-virtctl up-to-date:
  bazel-genfiles/virtctl-copier
INFO: Elapsed time: 0.854s, Critical Path: 0.09s
INFO: 0 processes.
INFO: Build completed successfully, 1 total action
INFO: Build completed successfully, 1 total action
INFO: Build option --platforms has changed, discarding analysis cache.
INFO: Analysed target //:build-virtctl (0 packages loaded, 8941 targets configured).
INFO: Found 1 target...
Target //:build-virtctl up-to-date:
  bazel-genfiles/virtctl-copier
INFO: Elapsed time: 0.906s, Critical Path: 0.11s
INFO: 1 process: 1 processwrapper-sandbox.
INFO: Build completed successfully, 2 total actions
INFO: Build completed successfully, 2 total actions
INFO: Build option --platforms has changed, discarding analysis cache.
INFO: Analysed target //:build-virtctl (0 packages loaded, 8307 targets configured).
INFO: Found 1 target...
Target //:build-virtctl up-to-date:
  bazel-genfiles/virtctl-copier
INFO: Elapsed time: 0.723s, Critical Path: 0.11s
INFO: 1 process: 1 processwrapper-sandbox.
INFO: Build completed successfully, 2 total actions
INFO: Build completed successfully, 2 total actions
[+] Building 0.1s (9/9) FINISHED
 => [internal] load build definition from Dockerfile                                                                                0.0s
 => => transferring dockerfile: 38B                                                                                0.0s
 => [internal] load .dockerignore                                                                                0.0s
 => => transferring context: 2B
 => [internal] load metadata for docker.io/kubevirt/builder@sha256:08b4db91e5f6272004892c6174129172c1bef5c6906bde9f8414cb945a588e12                                                       0.0s
 => [1/4] FROM docker.io/kubevirt/builder@sha256:08b4db91e5f6272004892c6174129172c1bef5c6906bde9f8414cb945a588e12                                                                         0.0s
 => => resolve docker.io/kubevirt/builder@sha256:08b4db91e5f6272004892c6174129172c1bef5c6906bde9f8414cb945a588e12                                                                         0.0s
 => [internal] load build context                                                                                0.0s
 => => transferring context: 33B                                                                                0.0s
 => CACHED [2/4] RUN     mkdir -p /go &&     source /etc/profile.d/gimme.sh &&     go get github.com/mattn/goveralls &&     go get -u github.com/golang/mock/gomock &&     go get -u git  0.0s
 => CACHED [3/4] RUN pip install j2cli && pip3 install operator-courier==1.3.0                                                                                0.0s
 => CACHED [4/4] COPY rsyncd.conf /etc/rsyncd.conf                                                                                0.0s
 => exporting to image                                                                                0.0s
 => => exporting layers                                                                                0.0s
 => => writing image sha256:6e512c03eb7b69fd0cca43aa2266799c72fead3ffa535c0390cfd7ae493d0ac8                                                                                0.0s
 => => naming to docker.io/library/kubevirt                                                                                0.0s
go version go1.11.5 linux/amd64

go version go1.11.5 linux/amd64
devel devel_alt
devel
INFO: Build options --define and --platforms have changed, discarding analysis cache.
INFO: Analysed target //:push-images (0 packages loaded, 13125 targets configured).
INFO: Found 1 target...
Target //:push-images up-to-date:
  bazel-bin/push-images
INFO: Elapsed time: 1.678s, Critical Path: 0.32s
INFO: 0 processes.

Environment:

KubeVirt version (use virtctl version):
Kubernetes version (use kubectl version):
VM or VMI specifications:
Cloud provider or hardware configuration:
OS (e.g. from /etc/os-release):
Kernel (e.g. uname -a):
Install tools:
Others:

Master git commit: f680f4122bfa8da5bfb27cba8a51d5692e2df8b6
$ go version
go version go1.12.4 linux/amd64

updates
OS : Linux HP-workstation 4.15.0-54-generic #58-Ubuntu
Docker version:  19.03.1
KUBEVIRT_PROVIDER = k8s-1.13.3

Hi @kunalkushwaha, thanks for reporting this issue.  Please, which OS are you running? What docker version are you using? Also do you have any nod default setup on you machine - eg virtualization solution.
Thanks a lot.

also, which KUBEVIRT_PROVIDER do you use? I just tried it with the k8s-1.13.3 default provider and it worked (on current master branch, commit 292310f82a96e727e69c7cb7b5e692a46dd7f446)

OS : Linux HP-workstation 4.15.0-54-generic #58-Ubuntu
Docker version:  18.09.7
KUBEVIRT_PROVIDER = k8s-1.13.3

@slintes @petrkotas I just updated the required information.
Also, I tried to do with make clean, but still images are not visible in docker images output.

$ make bazel-build-images
.
.
7978 blocks
Verifying packages...
Preparing packages...
qemu-img-2:2.11.2-5.fc28.x86_64
Verifying packages...
Preparing packages...
qemu-guest-agent-2:2.11.2-5.fc28.x86_64
Verifying packages...
Preparing packages...
stress-1.0.4-20.fc28.x86_64
Verifying packages...
Preparing packages...
libstdc++-8.3.1-2.fc28.x86_64
Verifying packages...
Preparing packages...
capstone-3.0.5-1.fc28.x86_64
Verifying packages...
Preparing packages...
libaio-0.3.110-11.fc28.x86_64
Verifying packages...
Preparing packages...
e2fsprogs-1.44.2-0.fc28.x86_64
Target //:build-images up-to-date (nothing to build)   <-- Is this expected?
INFO: Elapsed time: 271.148s, Critical Path: 48.95s
INFO: 1653 processes: 1653 processwrapper-sandbox.
INFO: Build completed successfully, 1739 total actions

even after make clean,  make bazel-build-images skips building images and push to local registry.
I see this line
Target //:build-images up-to-date (nothing to build)

Is this correct behaviour?
Is there any way, I can skip the cache check, and bazel forcefully build without cache?

@kunalkushwaha I am honestly not sure what is happening. I have not encountered this error before. Maybe @cynepco3hahue may help, since he build the bazel tooling.

even after make clean, make bazel-build-images skips building images and push to local registry.

That the images aren't rebuild after a clean is strange indeed. That no push happens is expected, because bazel-build-images does not push at all. That's done by bazel-push-images
The bazel server is running in docker, and all cached files are in a docker volume. You can try do delete the latter and retry:
$ docker kill kubevirt-bazel-server && docker rm kubevirt-bazel-server
$ docker volume rm kubevirt

I tried deleting the kubevirt-bazel-server container , but never deleted volume. Maybe this will fix.
I am trying to rebuild again after deleting volume.

@kunalkushwaha Can you please provide the output of docker ps command on the environment when do you run make cluster-up, probably F0726 08:20:35.171421   24317 fast_pusher_.py:194] Error publishing localhost:32803/kubevirt/virt-handler:devel: (503, 'Service Unavailable') happens because it refers wrong registry port.

The ports have changed, as I created cluster again.
$ docker ps -a
CONTAINER ID        IMAGE                   COMMAND                  CREATED             STATUS              PORTS                                                                                NAMES
80f21b0ec665        kubevirtci/k8s-1.13.3   "/bin/bash -c '/vm.s…"   55 seconds ago      Up 54 seconds                                                                                k8s-1.13.3-node01
e48f5b847a0c        registry:2.7.1          "/entrypoint.sh /etc…"   55 seconds ago      Up 54 seconds                                                                                k8s-1.13.3-registry
2080ec374838        kubevirtci/k8s-1.13.3   "/bin/bash -c /dnsma…"   59 seconds ago      Up 58 seconds       <IP_ADDRESS>:8443->8443/tcp, <IP_ADDRESS>:32843->2201/tcp, <IP_ADDRESS>:32842->5000/tcp, <IP_ADDRESS>:32841->5901/tcp, <IP_ADDRESS>:32840->6443/tcp   k8s-1.13.3-dnsmasq
3a45e8c0c776        kubevirt                "/entrypoint.sh hack…"   19 minutes ago      Up 19 minutes                                                                                kubevirt-bazel-server
22eb02e05003        kubevirt                "/entrypoint.sh /usr…"   19 minutes ago      Up 19 minutes       <IP_ADDRESS>:32839->873/tcp                                                                                suspicious_heisenberg

But I had checked that and I was able to push other images into registry on same port and pull it in kubevirt k8s VM from registry.
The problem is images e.g (localhost:32803/kubevirt/virt-handler:devel) should be visible after build images on host with docker images. Its not visible.
even after I removed the docker volume  kubvevirt and build again.

nope it is not, currently we are building via bazel, so all images exist under the bazel cache and not under the local registry

The error has changed now.
Target //:build-virtctl up-to-date:
  bazel-genfiles/virtctl-copier
INFO: Elapsed time: 26.367s, Critical Path: 22.15s
INFO: 279 processes: 279 processwrapper-sandbox.
INFO: Build completed successfully, 280 total actions
INFO: Build completed successfully, 280 total actions
[+] Building 0.2s (9/9) FINISHED
 => [internal] load build definition from Dockerfile                                                                                0.0s
 => => transferring dockerfile: 38B                                                                                0.0s
 => [internal] load .dockerignore                                                                                0.0s
 => => transferring context: 2B                                                                                0.0s
 => [internal] load metadata for docker.io/kubevirt/builder@sha256:08b4db91e5f6272004892c6174129172c1bef5c6906bde9f8414cb945a588e12                                                       0.0s
 => [1/4] FROM docker.io/kubevirt/builder@sha256:08b4db91e5f6272004892c6174129172c1bef5c6906bde9f8414cb945a588e12                                                                         0.0s
 => => resolve docker.io/kubevirt/builder@sha256:08b4db91e5f6272004892c6174129172c1bef5c6906bde9f8414cb945a588e12                                                                         0.0s
 => [internal] load build context                                                                                0.0s
 => => transferring context: 33B                                                                                0.0s
 => CACHED [2/4] RUN     mkdir -p /go &&     source /etc/profile.d/gimme.sh &&     go get github.com/mattn/goveralls &&     go get -u github.com/golang/mock/gomock &&     go get -u git  0.0s
 => CACHED [3/4] RUN pip install j2cli && pip3 install operator-courier==1.3.0                                                                                0.0s
 => CACHED [4/4] COPY rsyncd.conf /etc/rsyncd.conf                                                                                0.0s
 => exporting to image                                                                                0.0s
 => => exporting layers                                                                                0.0s
 => => writing image sha256:6e512c03eb7b69fd0cca43aa2266799c72fead3ffa535c0390cfd7ae493d0ac8                                                                                0.0s
 => => naming to docker.io/library/kubevirt                                                                                0.0s
go version go1.11.5 linux/amd64

go version go1.11.5 linux/amd64
devel devel_alt
devel
[bazel FATAL src/main/cpp/blaze.cc:1325] Unknown startup option: '-dfs'.
  For more info, run 'bazel help startup_options'.
devel_alt
[bazel FATAL src/main/cpp/blaze.cc:1325] Unknown startup option: '-dfs'.
  For more info, run 'bazel help startup_options'.
[+] Building 0.1s (9/9) FINISHED

I updated the code to master ( earlier code was of last week)
removed the docker volume kubevirt

complete log https://gist.github.com/kunalkushwaha/b85f45ccc086023afaac72120ff425cb

@cynepco3hahue got it. Please check here https://github.com/kubevirt/kubevirt/issues/2532#issuecomment-515902657 error is changed now.

@kunalkushwaha hm, what exactly command do you run?

$ make bazel-build-images
$ make cluster-sync

error happens on make cluster-sync

@kunalkushwaha Give me some time will check it locally.

@kunalkushwaha hm I check it locally after the rebase, and all works as expected :worried:

Can proxy be issues?
Though I am able to push image to registry form host and pull it from
KubeVirt VM.
Can you give me pointer to debug the issue? Like check in cache if images
are build, and place where I need to look for pushing images
On Tue, Jul 30, 2019, 7:34 PM Artyom Lukianov<EMAIL_ADDRESS>wrote:

@kunalkushwaha https://github.com/kunalkushwaha hm I check it locally
after the rebase, and all works as expected 😟
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/kubevirt/kubevirt/issues/2532?email_source=notifications&email_token=AAVBOIOP4MN2HU2P3RZHP7TQCAKMRA5CNFSM4IHBVRE2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3DRMXY#issuecomment-516363871,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAVBOIJDSI4RSTRB5TACKJ3QCAKMRANCNFSM4IHBVREQ
.

Mh, the push is trying to push to localhost:32803
F0726 08:20:35.171421   24317 fast_pusher_.py:194] Error publishing localhost:32803/kubevirt/virt-handler:devel: (503, 'Service Unavailable')
But I don't see that port open:
2080ec374838        kubevirtci/k8s-1.13.3   "/bin/bash -c /dnsma…"   59 seconds ago      Up 58 seconds       <IP_ADDRESS>:8443->8443/tcp, <IP_ADDRESS>:32843->2201/tcp, <IP_ADDRESS>:32842->5000/tcp, <IP_ADDRESS>:32841->5901/tcp, <IP_ADDRESS>:32840->6443/tcp   k8s-1.13.3-dnsmasq
The correct port should be 32842, the one which is being forwared to 5000.
What value do you have for docker_prefix in _ci-configs/k8s-1.13.3/config-provider-k8s-1.13.3.sh? Can you delete _ci-configs and retry? That should recreate that file with the correct value hopefully :thinking:

I just came into a similar situation and noticed that make push ignores KUBEVIRT_PROVIDER and always uses settings of k8s-1.13.3, which does not work if you started another cluster. make cluster-sync should work though.

the make push issue mentioned in the last comment will be fixed by https://github.com/kubevirt/kubevirt/pull/2549

The correct port should be 32842, the one which is being forwared to 5000

The output is from different run, so ports changed.
I feel this error is due to proxy settings
I have one question:
F0731 00:55:18.670257    4551 fast_pusher_.py:194] Error publishing localhost:32849/kubevirt/virt-handler:devel: (503, 'Service Unavailable')
this line i.e. push logic executes on host or within container?

@slintes @cynepco3hahue @petrkotas It got resolved :)
bazel push script understand proxy with HTTP_PROXY & HTTPS_PROXY setting not in small cases i.e. http(s)_proxy
Thanks for all help :)

glad that you sorted this out! :slightly_smiling_face:

@kunalkushwaha thanks for sharing the how-to solve this issue. And I am glad that this issue is sorted out.


Expose new :from_machine attribute on machine_image

@cwebberOps has a use case where he has an existing machine that he wants to turn into a machine_image (EG, an ami in amazon land).  This machine AMI needs to be pristine - IE, it needs to NOT have Chef installed on it.
This can be accomplished on a linux or windows machine by running a machine resource (to create the base machine and converge it) then using a machine_execute resource to uninstall Chef.  The problem with this approach is that there is no way to tell machine_image to use the existing machine as the base to create the image.
This could also be accomplished by updating the machine_image resource with some pre_converge_execute and post_converge_execute hooks.
\cc @jkeiser what do you think?

An update - cwebber was able to do this by monkey patching the allocate_image method to not call machine.action_destroy

Lets go with from_machine

\cc @ryancragun

In this case, machine_image will converge by checking for the named AMI before trying to create it.

There is an example of a hack that roughly does what this does is intended to do in the Corp Delivery instance.. CIA/training-images.


[FLINK-25846][FLINK-25848] Async Sink does not gracefully shutdown on Cancel, KDS Sink does not fast fail when invalid configuration supplied

What is the purpose of the change
Bugfixes:
Async Sink does not gracefully shutdown on Cancel
KDS Sink does not fast fail when invalid configuration supplied
Brief change log

Added more cases to the isRetryable method in the KDSSinkWriter to handle permanently failing cases.

 Thank you.

|| @dannycranmer
|| Why did you split out to separate HTTP clients?
I will revert to the previous, where it is using a single http client. Please continue to consider this PR as the other commits  relate to necessary improvements to the sinks.

@flinkbot run azure

Here is the code I've used during debugging, it could provide some more context for understanding the issue better. https://gist.github.com/dmvk/6f3fbf5a6432becd076a9f7989ab8bdd

Thank you. I have understood the issue. I have now changed my usage of Netty to the Apache synchronous http client. I have also removed the @Ignore flag for the integration test case.

@flinkbot run azure

Also I would rather avoid switching the client implementations now, because that might require another investigation whether there is some kind of shared thread pool underneath (which might be a pattern in the SDK).
Only think we have to do is to construct a separate ELG group (of reasonable size) for the test suite and for each writer.

Now the problem is that the Netty thread inherits a context classloader from the thread that has created it

I re-read the excellent deep dive, thanks @dmvk. From what I understand there would be no issue for Flink jobs submitted to a cluster, since they will use the FlinkUserClassloader. However, as we saw with the tests, when using the mini cluster this issue could resurface. Therefore I agree we should fix in AWSGeneralUtil.

One thing to note that there is a separate FlinkUserClassloader per-job (probably even per TM-slot, I'm not sure). So if there would be multiple jobs running on the same TM and one of them terminates (and closes it's classloader), we might IMO see this on a real cluster as well.

@dmvk Thank you for the in-depth view of the issue. I have created a PR#18733 relating to this change.

Since, @CrynetLogistics has isolated the Integration test fix in #18733 ,
I have rebased this PR now to address issues in [FLINK-25846] and [FLINK-25848] only.
@dannycranmer I am trying to find related comments but they seem to be addressed, could you please re-review?!

@flinkbot run azure

@flinkbot run azure

@flinkbot run azure


add fishman-moore pseudo random number generator

The checks were failing on this PR and it was still merged.

Apparently the problem is unrelated to this PR, something about code coverage.


🙏 Sorry, I'm really busy at the moment

Hi, you might have noticed that support on pydantic and particularly reviewing and merging pull requests is a bit slow at the moment.
@PrettyWood and others have been amazing in their support recently, but I have been noticeably absent, and I know that slows things down or stops them.
I'm really sorry about that, I'm currently flat out with work at my new role with Nell Health and have reduced time to spend on pydantic.
I'm still committed to pydantic, I use it most days, I rely on it in production with multiple projects. I want the new features currently languishing in PRs and issue descriptions as much as you. I also have big ideas about v2 and how we can make pydantic both faster and easier to use at the same time.
I'll do my best to review all open pull requests and issues over the month.
You can help!
If you're waiting on new features, please consider helping out with the pydantic project, you could:

check open issues you've created, check if they're still relevant and either add more information or close them
look through recent open issues and see if you can solve their problem, give feedback on a feature request or confirm a bug
review, try and give feedback on open pull requests

I really appreciate this library. It has simplified an absolute ton of workflows for me. I know what it's like to be absolutely slammed at work. Try your best to delegate, maintain balance, and most of all....Keep up the awesome work!

That is wonderful library and letting it be affected by bit rot it would be a big loss for python community.
How about moving it under a github organization that can assure its future does not depend on a single person? It could be either one of the existing ones focused on python or a new one. The key here is to find few others that you can trust as not doing anything bad and give them access, so they can help when needed.
That is why none of the libraries I created of maintained with the goal of sharing them are under my personal github account, as I did not want to eventually become the bottleneck.

That is wonderful library and letting it be affected by bit rot it would be a big loss for python community.
How about moving it under a github organization that can assure its future does not depend on a single person? It could be either one of the existing ones focused on python or a new one. The key here is to find few others that you can trust as not doing anything bad and give them access, so they can help when needed.
That is why none of the libraries I created of maintained with the goal of sharing them are under my personal github account, as I did not want to eventually become the bottleneck.

I did not want to imply that pydantic would be into a bad place. I only wanted to do a risk-mitigation measures. Do you have a designated successor configured on https://github.com/settings/admin ? -- that was the kind of issue I wanted to highlight: assuring that everything is set in place in case something bad happens with you, the current pandemy made us more aware of risks.
There is nothing stopping you from having an organization solely for pydantic if you want, they do not cost money.
Regarding use of other organizations, do not see it as "giving away" you project. Some of them were created with the main goal of providing help when needed, not to take away original author control over the project. I am aware of at least three such organizations @sphinx-contrib @pytest-dev and @pycontribs (i bet there are lots more). AFAIK, all of these assure that the author owner keeps his rights, including possibility to transfer it back at any moment. The org admins would step-in to help only when needed, as a fallback. Usually those orgs have an meta repository used for dealing with stuff like this.
There is also the @psf organization, but I am afraid that that one may not suit the project due to its restricted goal and see 2/4 as not being met for it, likely first of them never will.
BTW, could you please enable discussions option on this repository? It would be great if we could make use of the embedded forum and avoid cluttering the issue tracker with non-issues.

I did not want to imply that pydantic would be into a bad place. I only wanted to do a risk-mitigation measures. Do you have a designated successor configured on https://github.com/settings/admin ? -- that was the kind of issue I wanted to highlight: assuring that everything is set in place in case something bad happens with you, the current pandemy made us more aware of risks.
There is nothing stopping you from having an organization solely for pydantic if you want, they do not cost money.
Regarding use of other organizations, do not see it as "giving away" you project. Some of them were created with the main goal of providing help when needed, not to take away original author control over the project. I am aware of at least three such organizations @sphinx-contrib @pytest-dev and @pycontribs (i bet there are lots more). AFAIK, all of these assure that the author owner keeps his rights, including possibility to transfer it back at any moment. The org admins would step-in to help only when needed, as a fallback. Usually those orgs have an meta repository used for dealing with stuff like this.
There is also the @psf organization, but I am afraid that that one may not suit the project due to its restricted goal and see 2/4 as not being met for it, likely first of them never will.
BTW, could you please enable discussions option on this repository? It would be great if we could make use of the embedded forum and avoid cluttering the issue tracker with non-issues.

Moved the conversation about hosting/managing pydantic to #2267.

Moved the conversation about hosting/managing pydantic to #2267.


Minor bug in FtpStream::get

Line 298 in FtpStream in the get function is :
self.read_response(status::ABOUT_TO_SEND).await?;

but should be
self.read_response_in(&[status::ABOUT_TO_SEND, status::ALREADY_OPEN]).await?;

(same as in the FtpStream::retr function)
Can this be fixed in the next release?

Sounds reasonable, can you make a PR changing it?

Yeah will do.

Thanks! How often do updates get pushed to crates?

Usually as soon as there are some changes, 4.0.3 is up now with your PR included.

Thank you so much! This was very helpful


Log key when rate limit has been exceeded.

Why?
I'd like to know the key for which the rate limit has been exceeded.
Test
I tested both for distributed: {} and without it.
Caddy JSON rate limit setup
{
  "handler": "rate_limit",
  "rate_limits": {
    "dynamic_example": {
      "key": "{http.request.host}",
      "window": "5m",
      "max_events": 2
    }
  },
  "storage": {
    "module": "redis",
    "address": [
      "localhost:6379"
    ],
    "username": "",
    "password": "",
    "db": 0,
    "timeout": "5",
    "key_prefix": "caddy-rate-limit",
    "encryption_key": "",
    "compression": false,
    "tls_enabled": false,
    "tls_insecure": true
  },
  "distributed": {}
}

Resulting logs (prettified)
{
  "level": "info",
  "ts": "2024-06-28T12:54:58.067Z",
  "logger": "http.handlers.rate_limit",
  "msg": "rate limit exceeded",
  "zone": "dynamic_example",
  "key": "localhost",
  "wait":<PHONE_NUMBER>9,
  "remote_ip": "<IP_ADDRESS>"
}

I'm not an expert in Go so please let me know if this is good enough to merge.
Cheers!

The change looks good, nice work. Although, my only concern is that it's possible for the key to contain sensitive information.
Caddy does have config options to redact certain fields, but its HTTP server, for example, redacts certain values by default (sensitive header fields for instance). I wonder if we should make this an opt-in thing. :thinking:

Good thinking @mholt.
I updated the code, squashed it into a single commit and pushed.
Also I run the tests with log_key not set, set to false and true. Works as expected.
Let me know what you think.

Cool! Do I need to do anything else in order for this to get merged?

Nope. :)

Amazing, my first code contribution to Caddy 🤠
Cheers!

Congrats!


provide task_attempt table

This reworks how task executions are tracked by providing a new table, task_attempt which is a log of each dequeue of a given task from the queue. Put another way, each time a task is executed, a row is inserted into the attempts table and the result of that execution is stored on that row.
As such, the task table is now a container for the fixed metadata of a specific enqueue of a task. This means that metadata associated with an attempt now lives on the task_attempt table. For example, error messages are stored on the attempts table and likewise creation and completion timestamps also live on this table.
Importantly some fields of the task row have been removed as they are now derived from the attempts log. For instance, task_attempt provides an updated_at column which allows for e.g. determining succeeded_at or failed_at timestamps of the task enqueue. Similarly retry_count is a function of the number of attempt rows associated to a given task.
Furthermore, the retry policy is now embedded as a JSONB column on the task row.
Related discussion: https://github.com/maxcountryman/underway/discussions/17

This is ready for review.
Testing could still be improved but I've added cases for the state transitions that check attempt rows are being created as we'd expect.

it looks good to me

Maybe I'll have more comments once I start using it

@kirillsalykin thanks for all your feedback thus far: very much appreciate it. 🙏


Package Formly LumX=angular-formly-templates-lumx not found

I'm not sure if this is an issue with bower or angular-formly-templates-lumx, but I thought i'd report in here first.
I've installed the package with bower install angular-formly-templates-lumx which works fine.  The next time I try to use bower, I get this error message:
bower ENOTFOUND     Package Formly LumX=angular-formly-templates-lumx not found
I've got no idea how to fix this, I've resorted to pulling it from bower.json and including it statically for the time being.

Let's see if it's an issue with Bower first.
Try bower cache clean then re-install again with bower install --save.

I gave that a shot: It still seems to be occurring however.
 It seems I had changed the name inside bower.json, everything should work once the registry is updated.

Just tried it & looks to be working properly. Thanks for addressing so quickly!


Can't find variable: Gmaps

Rails beginner.
Following the #555, after website is loaded, I am trying to load my first maps on rails, but I get the following error:
Can't find variable: Gmaps
on the line:
handler = Gmaps.build('Google');
I have simply copied and pasted the example snipped code from point 5 of the Requirements page "5) Javascript code"

Maybe my mistake in adding and installing the gem.