[TASK] Add mamba as an alternate to conda.

Issue Description
Is your feature request related to a problem? Please describe.
mamba can be faster for installation than conda
Describe the solution you'd like
Add mamba as a option for the install script

For Change Control Board: Issue Review
This review should occur before any development is performed as a response to this issue.

[x] 1. Is it tagged with a type: defect or task?
[x] 2. Is it tagged with a priority: critical, normal or minor?
[x] 3. If it will impact requirements or requirements tests, is it tagged with requirements?
[x] 4. If it is a defect, can it cause wrong results for users? If so an email needs to be sent to the users.
[x] 5. Is a rationale provided? (Such as explaining why the improvement is needed or why current code is wrong.)

For Change Control Board: Issue Closure
This review should occur when the issue is imminently going to be closed.

[x] 1. If the issue is a defect, is the defect fixed?
[x] 2. If the issue is a defect, is the defect tested for in the regression test system?  (If not explain why not.)
[x] 3. If the issue can impact users, has an email to the users group been written (the email should specify if the defect impacts stable or master)?
[x] 4. If the issue is a defect, does it impact the latest release branch? If yes, is there any issue tagged with release (create if needed)?
[x] 5. If the issue is being closed without a pull request, has an explanation of why it is being closed been provided?

issue closure checklist is satisfied.


A shared library should have exactly one target with the suffix

Hi.
I have problem during running the final code for openwsn-fw. The code is
scons board=python toolchain=gcc oos_openwsn
The error is:-
scons: *** A shared library should have exactly one target with the suffix:  Target(s) are:bsp_bothtimers File "xxx\yyy\zzz\openwsn-fw\SConscript", line 1280, in project_finder
The line 1280 in SConscript is:-
targetAction = localEnv.SharedLibrary(target,source,LIBS=libs,SHLIBPREFIX='',SHLIBSUFFIX=pylib_ext,)

I already solved this issue by following exactly the versions that you suggest to install.
I thought by installing the latest version will cause no problems. But, it is not.. everything must be installed as per advised.


Bump active_utils to v3.1.0

Bump active_utils dependency to v3.1.0
Diff for active_utils https://github.com/Shopify/active_utils/compare/v3.0.0...v3.1.0
Review @kmcphillips @RichardBlair

Hmm looks like that test failure has been broken for over a week or at least since https://github.com/Shopify/active_shipping/pull/319 since it failed in that PR too.

Diff looks good, but yeah, broken test. Sorry that got through.

I would just skip the test. Remote tests are super flaky


The response was not an image.

hi,
thanks for this project.
i got following error:
$ unsplash-wallpaper -r
Request https://source.unsplash.com/random
✗ The response was not an image.
anything i can provide?

Same problem here.
It seems source has been deleted after being deprecated since November 2021: https://changelog.unsplash.com/deprecations/2021/11/25/source-deprecation.html

i've created for me a simple script:
#!/bin/bash
# export DBUS_SESSION_BUS_ADDRESS environment variable
PID=$(pgrep -o gnome-shell)
export DBUS_SESSION_BUS_ADDRESS=$(grep -z DBUS_SESSION_BUS_ADDRESS /proc/$PID/environ|cut -d= -f2-)

/usr/bin/gsettings set org.gnome.desktop.background picture-uri `/usr/bin/curl -s 'https://api.unsplash.com/photos/random?client_id=xxx&query=wallpaper&orientation=landscape&count=1' | /usr/bin/jq '.[].urls.full'`

just request an client_id from usplash
https://unsplash.com/developers


Note additional configuration might be required to mount volumes with Docker Toolbox

Mention this section of Docker Manual in the Using TestCafe Docker Image section: https://docs.docker.com/toolbox/toolbox_install_windows/#optional-add-shared-directories
By default, Toolbox only has access to the C:\Users directory and mounts it into the VMs at /c/Users.

Note: Within the VM path, c is lowercase and the Users is capitalized.

If your project lives elsewhere or needs access to other directories on the host filesystem, you can add them, using the VirtualBox UI.

1. Open the VirtualBox UI.

2. Click the Settings gear, then go to Shared Folders.

3. Select any existing listing under Machine Folders, then click the + icon.

    * Choose the Folder Path on the host, enter the Folder Name for within the VM (or take the default, which is the same name as on the host), and configure any additional options you need.

    * Choose Auto-mount if you want the folder to automatically be mounted into the VM, and choose Make Permanent for it to be considered a permanently shared folder.

4. Click OK to add the new folder to the Shared Folders list.

5. Click OK again to save your changes and exit the Settings dialog.

When does a user needs to add shared directories to TestCafe Docker image? Is it required to map the test directory if tests are located outside of C:\Users? Or probably some other cases?

Yes, you have to do this if you want to mount a directory outside of C:\Users.


Flake8 rule 'E901' reported an invalid point

Flake8 reported an invalid point for the rule E901, with the messge SyntaxError: invalid syntax.
#!/usr/bin/env python
import os

for var in os.environ:

Seems like leaving the for without content raises this.
Debug information:
Atom version: 1.16.0
Flake8 version: 2.5.5 (pep8: 1.6.2, pyflakes: 1.2.3, mccabe: 0.2.1) CPython 2.7.13 on Linux

Please update your flake8 version (pip install -U flake8). This particular error should be fixed in flake8>=3.2.1.

Just as a thought, it might be worth putting some additional message about upgrading in there, around the E901-catching mechanism, to save time & nerves @Arcanemagus :) I actually thought I was being helpful when reporting the bug.

The problem with that is that it's a generic "flake8 gave an invalid point" catch, some "smarts" could be put in there though where if the flake8 version is below v3.2.1 it tells the user to upgrade first.

Closing due to a lack of response, if you are still having issues after updating flake8 please provide code to reproduce this!


Improve loading performance significantly

Take the bithon-server itself as an example
Data  1
agent disabled
2021-12-26 12:02:18.673  INFO 68220 --- [  restartedMain] .e.DevToolsPropertyDefaultsPostProcessor : Devtools pr
...
2021-12-26 12:02:23.941  INFO 68220 --- [on(2)-<IP_ADDRESS>] o.s.web.servlet.DispatcherServlet        : Completed initialization in 5 ms

Loading time is about 5 seconds
Before optimization
2021-12-26 12:06:18.642 [Agent INFO] - [main:AgentStarter] Found lib agent-controller-1.0-SNAPSHOT.jar
...
2021-12-26 12:06:45.138  INFO 68602 --- [on(5)-<IP_ADDRESS>] o.s.web.servlet.DispatcherServlet        : [bTxId:, bSpanId:] Completed initialization in 23 ms

Loading time is about 17 seconds
After optimization
2021-12-26 11:59:17.118 [Agent INFO] - [main:AgentStarter] Found lib agent-controller-1.0-SNAPSHOT.jar
....
2021-12-26 11:59:27.700  INFO 67942 --- [on(1)-<IP_ADDRESS>] o.s.web.servlet.DispatcherServlet        : [bTxId:, bSpanId:] Completed initialization in 8 ms

loading time is about 10 second
Data 2
Before optimization
2021-12-26 04:10:24.446 [Agent INFO] - [main:AgentStarter] Found lib agent-controller-1.0-SNAPSHOT.jar
...
2021-12-26 04:11:16.253 [Agent INFO] - [main:Dispatcher] Application port updated to 8091, metric will soon be at work

Loading time is about 61 seconds.
After optimization
2021-12-26 04:16:24.059 [Agent INFO] - [main:AgentStarter] Found lib agent-controller-1.0-SNAPSHOT.jar
...
2021-12-26 04:16:41.902 [Agent INFO] - [main:Dispatcher] Application port updated to 8091, metric will soon be at work

Loading time is about 17 seconds.

Before optimization
2021-12-26 14:46:33.686 [Agent INFO] - [restartedMain:BeanMethodAopInstaller$InstallerTask] Setup AOP for class [org.springframework.context.event.EventListenerMethodProcessor]
...
2021-12-26 14:46:36.815 [Agent INFO] - [metric-sink-thread-2-jvm-gc-metrics:BeanMethodAopInstaller$InstallerTask] Setup AOP for class [org.springframework.jdbc.support.SQLErrorCodes]

After optimization
It took 2 seconds to finish the installation.
2021-12-26 14:42:04.662 [Agent INFO] - [restartedMain:BeanMethodAopInstaller] Setup AOP for Bean class [org.bithon.server.metric.sink.LocalSchemaMetricSink]
...
2021-12-26 14:42:06.891 [Agent INFO] - [restartedMain:BeanMethodAopInstaller] Setup AOP for Bean class [com.sun.jmx.mbeanserver.JmxMBeanServer]


GoScript - A frontend framework for fullstack Golang

Proposal Details
Proposing a Golang frontend framework very very similar to typescript [for less learning curve] and svelte like features.
focusing on composable frontend framework and AI prompt based generative layout capabilities will make GoScript inevitable.
wordpress may switch to GoScript/Golang
please consider and do the needful.
https://www.tiobe.com/tiobe-index/
Golang is climbing but needs a frontend framework desperately

This seems out of scope and there's no real proposal to evaluate here


DeadCScroll dead link

The original link is dead, and @BlitterObjectBob hasn't replied yet to my pings for a couple days.
Apparently, the only surviving version is a fork of mine from when I was making PRs. Even gb-archive doesn't have it.
What should we do?

Even gb-archive doesn't have it.

This is quite strange, as gb-archive should have updated forks of every git repository mentioned in the awesome-gb list..
Also, the gb-archive script should parse Pan Docs for links too..


#4281: Add autofocus when opening preview

[ ] Breaking change? (if so, please describe the impact and migration path for existing application instances)

What changes did you make? (Give an overview)
Add autofocus to the field when selecting a preview
Is there anything you'd like reviewers to focus on?
How Has This Been Tested? (put an "x" (case-sensitive!) next to an item)

[ ] No need to
[x ] Manually (please, describe, if necessary)
[ ] Unit checks
[ ] Integration checks
[ ] Covered by existing automation

Checklist (put an "x" (case-sensitive!) next to all the items, otherwise the build will fail)

[ ] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation (e.g. ENVIRONMENT VARIABLES)
[ ] My changes generate no new warnings (e.g. Sonar is happy)
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes
[ ] Any dependent changes have been merged

Check out Contributing and Code of Conduct
A picture of a cute animal (not mandatory but encouraged)

hey @ZuevKirill95, this repo is not maintained (#4255), but we'll be happy to see your PRs here from now on: https://github.com/kafbat/kafka-ui


Cloudflare APO Cache Purge not working on the latest version

Confirmation

[X] My issue isn't already found on the issue tracker.
[X] I have replicated my issue using the latest version of the plugin and it is still present.

WordPress version
6.0
Cloudflare-WordPress version
4.10.0
PHP version
8.1
Expected result
The cache should be purged from the Cloudflare servers when we update the site content.
Actual result
While using the latest version of the plugin and APO, if we update the site content, the Cloudflare Cache isn't purged and it still shows the old content.
Steps to reproduce

Setup the latest Cloudflare Plugin on WordPress.
Enable APO settings.
Add a Page or article and get it cached on the Cloudflare Servers.
Update the article content from the WordPress dashboard and then open the URL in a private window of your browser.
You will notice that cache isn't purged from Cloudflare and it still serving the old content.

Additional factoids
APO Cache Purging works fine when we switch the plugin to version 4.9.1
References
It happened in the past too: https://github.com/cloudflare/Cloudflare-WordPress/issues/456

@nsgoyat can you provide an example of URLs that should be purged but are not being purged

the output of WP_DEBUG would also be helpful here to diagnose whether this is intentional.

just a note from #456, nothing changed functionality wise. the updated release was a fix for the user agent.

Hey Folks,
I Have captured the screencast showing the issue. With the latest version, the cache isn't getting purged. When I roll back to the older version, cache purge works fine.
https://photos.app.goo.gl/DiNG6wy9MadsfmJw5

thanks but we need the URLs and debug information here.

The concerns raised by this poster is correct. How do I know? I own a news website today.ng and updated the plugin to 4.10.0 about two days ago. I noticed since yesterday that my homepage and other pages were not updating immediately until after a few hours (about six hours or more). I spent most of the time trying to figure out the problem until I stumbled upon this link (https://community.cloudflare.com/t/latest-cloudflare-plugin-update-cache-issues/388638) in the community. It led me to think page and I downgraded as suggested by the poster. Everything is now back to normal. I think the problem is with the latest plugin and it should be looked into asap.

@ediong are you able to provide debug logs from the plugin as well as a list of URLs that should have been purged but were not? is https://www.today.ng/ one of them? I currently see bypass as the cache status for your home page so it wouldn't be cached to begin with:

Same here.... Cache not cleared when updating a post... Come on, it's the very reason we have this plugin and pay for APO.
The last few versions of the plugin have all had issues with APO, not clearing the homepage on new posts, etc.
And now this....

@ediong are you able to provide debug logs from the plugin as well as a list of URLs that should have been purged but were not? is https://www.today.ng/ one of them?

Yes! https://www.today.ng is the main website with the issue. After downgrading, the purge now work as described. There's a problem with the latest version. In truth, you need to look into it. You can even stage a dummy site and test the plugin.

closed by #485

Thanks, appreciate the quick fix!


Don't print contents of secret values with '\n' characters.

We found that one of our secrets was showing immediately in the logs. :(
The secret is a private key. It is a JSON value that contains \n in the string.
This tweak means that the command that sed processes is limited to the first line of the 'raw' string.
Problematic output:

@72636c Are you still looking after this repo? Does the above make sense?


Install Go manually in the GitStream container

The setup-go action does not work on Alpine Linux, the base image for the GitStream container.
Install Go manually and leverage caching to avoid downloading at every run.

The setup-go action does not work on Alpine Linux, the base image for the GitStream container.

Why do we care about Alpine? Aren't github runner running Ubuntu?

@ybettan We are running the container in the action to avoid building the image on every run: https://github.com/rh-ecosystem-edge/kernel-module-management/blob/main/.github/workflows/gitstream.yml#L20-L21

/lgtm


Code snippets in documentation matches background color

Steps to reproduce:

Visit http://twitter.github.io/heron/docs/operators/heron-tracker-api/#metricsquery
See that code in snippet is invisible

Expected outcome:
To be able to see the text in the code snippet.

Looks like this may have been fixed but not deployed? Serving the docs off of master locally gives the expected outcome.

is this out of master?

this was fixed in https://github.com/twitter/heron/pull/1331

closing the issue.

@kramasamy @objmagic is there a plan to deploy the change? Looks like it was fixed 6 days ago but is still broken in production

@ajorgensen will build the website now

@ajorgensen actually, we are going to release 0.14.3 soon. I will build website after 0.14.3.

Ok sounds good. Thanks!
On Thursday, September 8, 2016, Mark Li<EMAIL_ADDRESS>wrote:

@ajorgensen https://github.com/ajorgensen actually, we are going to
release 0.14.3 soon. I will build website after 0.14.3.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/twitter/heron/issues/1358#issuecomment-245756155, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAfHU_JZBDW3YFyaETxq0yI4Stm4xk3cks5qoIYzgaJpZM4J4SXu
.

--
Andrew Jorgensen
@ajorgensen


Rename peer connection events

'addstream' => 'pc:track'
_onTrickle 'error' => 'pc:error'

@zazabe please review

[Current coverage][cc-pull] is 94.26%

Merging [#39][cc-pull] into [master][cc-base-branch] will not change coverage

@@             master        #39   diff @@
==========================================
  Files            12         12
  Lines           488        488
  Methods         124        124
  Messages          0          0
  Branches         61         61
==========================================
  Hits            460        460
  Misses           28         28
  Partials          0          0

Powered by Codecov. Last updated by [11de8db...fb32ea2][cc-compare]
[cc-base-branch]: https://codecov.io/gh/cargomedia/janus-gateway-js/branch/master?src=pr
[cc-compare]: https://codecov.io/gh/cargomedia/janus-gateway-js/compare/11de8db1292c276cec46f2e0dd8768464fb355f9...fb32ea2a7e9f42d70a08fd235edab001b1650f1f
[cc-pull]: https://codecov.io/gh/cargomedia/janus-gateway-js/pull/39?src=pr

It seems that RTCTrackEvent provide a FrozenArray<MediaStream> but we addstream gives  a MediaStream, not sure if it's good to mix them into the same pc:track event, wdyt?

I agree. I just read that onaddstream is deprecated and ontrack is a replacement for it. I supposed that ontrack will preserve the previous syntax. My fault.
@zazabe please look

lgtm!

🎉


apulse can't find AppImage in $PATH.

Firstly, thank you very much for apulse. I am experiencing a problem trying to use apulse with an AppImage program in $PATH. MuseScore-2.1-x86_64.AppImage resides in ~/bin, which is in $PATH:
$ echo $PATH /home/username/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
However, when I run apulse ./MuseScore-2.1-x86_64.AppImage, I get:
$ apulse ./MuseScore-2.1-x86_64.AppImage /usr/bin/apulse: 5: exec: ./MuseScore-2.1-x86_64.AppImage: not found
I am trying to run this on a fresh, minimal, vanilla Devuan Jessie install with i3wm as the window manager. It of course is using Alsa and apulse.
Optimally, the idea is to run the above command through an entry in the i3 config file, such as:
bindsym $mod+n exec apulse ./MuseScore-2.1-x86_64.AppImage
I know that the system recognizes MuseScore in $PATH as it appears in dmenu.

Just a guess, what happens if you copy or symlink MuseScore-2.1-x86_64.AppImage to /usr/local/bin/musescore?

Thanks probonopd. Nice to hear from you again. That solved the issue. Please explain the reason for this. Is this an AppImage thing or an apulse thing?

This is an apulse thing. Looks like it searches for musecore on the $PATH.

$ apulse ./MuseScore-2.1-x86_64.AppImage

What you are trying to do here is to run MuseScore-2.1-x86_64.AppImage binary from current directory. That's what ./ at the beginning do. Shell doesn't try paths in PATH, since you are pointing to the exact location. For example, when you try to run ls, shell searches for ls, and eventually invokes /bin/ls. But when you run ./ls, it fails to run /bin/ls, unless you are currently in /bin.
Try:
$ apulse MuseScore-2.1-x86_64.AppImage

Ahh. Confusion averted. Thanks.
$ apulse MuseScore-2.1-x86_64.AppImage
works fine :)


How to debug cpp examples in IDE?

I have success with running my ortools program via:
make build SOURCE=example.cc
make run SOURCE=example.cc
but how it is possible to debug this source in IDE such as Visual Studio or CLion?
I want to make breakpoints and watch memory in runtime step by step.
What build tools configuration do I need to perform?

See #1447


Make a list of projects that may be interested in this project

Description
This repo aims to create a mechanism for efficient management and distribution of SNARK artifacts. Since the main goal is to standardize such processes within the PSE team, it makes sense to outline an initial list of relevant projects.
Possible list here: https://ceremony.pse.dev.

Short term goal: cover internal PSE projects like RLN, Unirep, Maci...?

Hello, what is the details about this Issue?

Hello, what is the details about this Issue?

This issue is about capturing (listing) projects that rely on groth16 zkSNARK and may be interested in the distribution mechanisms that we offer for their applications.
So that we can create new packages for their artifacts, publish them to npm, add them to our server so that they become available for download (e.g on https://snark-artifacts.pse.dev)


Support vpc peering for azure

@jerrychenhf Hi haifeng, I've finished and validated this PR. Thanks.

@jerrychenhf Hi haifeng, I've refined and validated the latest code. Thank!

Great! Thank you for the contribution. I will commit


Why do we need nfs-server running on Docs servers?

This enables and starts nfs server on conversion, docs and viewer servers, but all shares are created on the nfs servere configured in inventory. As this sometimes fails and there is nothing in /etc/exports, why do we need the nfs server here?
https://github.com/HCL-TECH-SOFTWARE/connections-automation/blob/c478e6f05b6942bba71d3a04852b853e6ace594e/roles/third_party/nfs-install/tasks/main.yml#L12

It's meant to setup NFS client in conversion_servers, docs_servers and viewer_servers but I can see configure_clients.yml starts nfs-server instead of nfs-client hence your point here.  Need to look into this.


header age not critical to HSTS header test result.

Age 0 or age 1 causes autest to fail randomly.

Cherry-picked to 8.0.x


Create email-validation template

Context:
In order to send messages from the webapi it is needed to implement a communication module
To do:

TDD: create tests
Create template for e-mail email-validation in modules/communication/0.domain/templates

Definition of done:

email-validation template created
All tests passing

I created pull request at  #28

Hey @yazaldefilimonepinto !
Actually the To do is a little confusing, I'm sorry 😅
What is needed here is to create a template to be used to send the confirmation email when the user creates a new account.
email-validation: the e-mail sent to confirm that the e-mail used by the user is valid (will be used here: src/modules/communication/1.application/use-cases/send-email-validation-email-use-case.ts:31)
Thanks for the contribution, anyway 😉


[BUG] A assert fail problem occurred in iot_mqtt_api

esp32 connected to AWS IoT platform.
MQTT PINGRESP is generated.
Network connection closed by PINGRESP
A assert fail problem occurred in iot_mqtt_api.

I used commit id #176636b9 in master branch

==============================================================================
[Wed Apr 28 08:18:07.161 2021]
[Wed Apr 28 08:18:07.161 2021] 13721 4761169 [_scheduleIntern] [INFO ][HDC][47611690] +-hdc_timer_mqtt_telemetry_start(time=60000)[1]
[Wed Apr 28 08:18:07.161 2021]
[Wed Apr 28 08:18:07.161 2021] 13722 4761169 [_scheduleIntern] [WARN ][MQTT][47611690] (MQTT connection 0x3f826994) Reference count changed from 3 to 2.
[Wed Apr 28 08:18:07.163 2021]
[Wed Apr 28 08:19:05.407 2021] 13723 4766996 [_scheduleIntern] [ERROR][MQTT][47669960] (MQTT connection 0x3f826994) Failed to receive PINGRESP within 10000 ms.
[Wed Apr 28 08:19:05.407 2021]
[Wed Apr 28 08:19:06.168 2021] 13724 4767068 [_scheduleIntern] [INFO ][MQTT][47670680] (MQTT connection 0x3f826994) Network connection closed.
[Wed Apr 28 08:19:06.168 2021]
[Wed Apr 28 08:19:06.168 2021] 13725 4767068 [_scheduleIntern] [INFO ][HDC][47670680] Shadow Ne../libraries/c_sdk/standard/mqtt/src/iot_mqtt_api.c:823 (_IotMqtt_DecrementConnectionReferences)- assert failed!
[Wed Apr 28 08:19:06.168 2021]
[Wed Apr 28 08:19:06.168 2021] abort() was called at PC 0x400fa263 on core 0
[Wed Apr 28 08:19:06.168 2021]
[Wed Apr 28 08:19:06.168 2021] Backtrace:0x40090b2f:0x3fff4570 0x4009116d:0x3fff4590 0x40095a1a:0x3fff45b0 0x400fa263:0x3fff4620 0x400fba65:0x3fff4650 0x40196a89:0x3fff4690 0x40197837:0x3fff46b0 0x400911dd:0x3fff46d0
[Wed Apr 28 08:19:06.178 2021]
[Wed Apr 28 08:19:06.178 2021]
[Wed Apr 28 08:19:06.178 2021] ELF file SHA256: 4ad6762f5ceeaded
[Wed Apr 28 08:19:06.178 2021]
[Wed Apr 28 08:19:06.178 2021] Rebooting...
[Wed Apr 28 08:19:06.202 2021] ets Jul 29 2019 12:21:46
[Wed Apr 28 08:19:06.202 2021]
[Wed Apr 28 08:19:06.202 2021] rst:0xc (SW_CPU_RESET),boot:0x17 (SPI_FAST_FLASH_BOOT)
[Wed Apr 28 08:19:06.202 2021] configsip: 0, SPIWP:0xee
[Wed Apr 28 08:19:06.202 2021] clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
[Wed Apr 28 08:19:06.202 2021] mode:DIO, clock div:2
[Wed Apr 28 08:19:06.202 2021] load:0x3fff0030,len:4
[Wed Apr 28 08:19:06.202 2021] load:0x3fff0034,len:7700
[Wed Apr 28 08:19:06.202 2021] load:0x40078000,len:13724
[Wed Apr 28 08:19:06.202 2021] load:0x40080400,len:4628
[Wed Apr 28 08:19:06.202 2021] entry 0x40080704
[Wed Apr 28 08:19:06.248 2021] I (29) boot: ESP-IDF v4.2-15-g6abb2ca33-dirty 2nd stage bootloader
[Wed Apr 28 08:19:06.248 2021] I (29) boot: compile time 16:58:07

Hi @strawberry3papa , We will investigate further and get back to you as soon as we are able to solve the problem. Also, I highly recommend that you have your application use coreMQTT instead if possible.

Hi @strawberry3papa,
This deprecated version of the MQTT library is no longer in the main branch of this repository. I would recommend migrating to the latest release 202107.00, which uses the coreMQTT and coreMQTT-Agent libraries. You can find various examples for using the coreMQTT-Agent library here and here.


Feature/add tree information to profiler

This PR adds id/parent id information for each entry in the profiler report. This allows for future visualizers to display the report in a tree-like structure, where inclusive/exclusive times can be seen more clearly.
Additionally, it fixes a bug where entries representing the same element were not merged properly.

This PR actually breaks the binary logger format. But since the last version update happened a couple of days ago, it might be fine. Please let me know, otherwise I'll bump the version and add the appropriate checks for back-compat.

@smera there's been a release in the meantime. Can you please bump up the version and branch in the reader? :)

Updated, new commit on the way

In reply to: 352068502 [](ancestors = 352068502)

<EMAIL_ADDRESS>Pushed, just in case you want to take a look

In reply to: 352148433 [](ancestors = 352148433,352068502)

@dotnet-bot test this please

@dotnet-bot test this please

@dotnet-bot test Windows_NT Build for CoreCLR please

@dotnet-bot test Windows_NT Build for CoreCLR please

@dotnet-bot test Windows_NT Build for CoreCLR please


MQTT on esp32c6 (and probably esp32c3) different behaviour than esp32

This is more of a note for other that stumble into this and hopefully saves them a few hours of debugging. (If there is a better place for this, I will gladly move it)
Testing around with the ESP32C6 I noticed that the mqtt client behaves way different.
First of all a (currently) non recoverable panic in the main loop happens, if you try to subscribe to any topic, before the connected callback is received (I do not know if this was ever valid, but it worked):
let mut client =  EspMqttClient::new_cb(&config.mqtt_url, &mqtt_client_config, move |event| {yadda yadda}
client.subscribe(stay_alive_topic.as_str(), ExactlyOnce)?;

If you do this you get a:
Guru Meditation Error: Core  0 panic'ed (Load access fault). Exception was unhandled.

Core  0 register dump:
MEPC    : 0x42072eca  RA      : 0x42072ec6  SP      : 0x40839a50  GP      : 0x408173e4
0x42072eca - run_event_loop
    at /home/empire/workspace/PlantCtrl/rust/.embuild/espressif/esp-idf/v5.2.1/components/mqtt/esp-mqtt/mqtt_client.c:1539
0x42072ec6 - run_event_loop
    at /home/empire/workspace/PlantCtrl/rust/.embuild/espressif/esp-idf/v5.2.1/components/mqtt/esp-mqtt/mqtt_client.c:1539
0x408173e4 - __global_pointer$
    at ??:??

the same code works on a esp32 with the xtensa toolchain, so apparently there is some difference in the blocking/async behaviour of the idf core here.
While I would prefer the subscribe return a error result, I'm not sure if this is even possible (or a good idea for that) without storing the connection state from the callbacks somewhere in the idf wrapper.
As a workaround waiting for the connected event in the callback, and only subscribing after it works fine on the C6.
Additionally, since the naming is confusing. The boolean connected in the embedded_svc::mqtt::client::EventPayload::Connected(connected)
actually represents session_present. This flag serves as an indicator for the client, informing it whether a previously established session is still available on the broker.  -> Does it make sense to rename this to session_present? If so I would offer to create a pull request for it.

Additionally, since the naming is confusing. The boolean connected in the embedded_svc::mqtt::client::EventPayload::Connected(connected) actually represents session_present. This flag serves as an indicator for the client, informing it whether a previously established session is still available on the broker. -> Does it make sense to rename this to session_present? If so I would offer to create a pull request for it.

Please do.

This is more of a note for other that stumble into this and hopefully saves them a few hours of debugging. (If there is a better place for this, I will gladly move it)
Testing around with the ESP32C6 I noticed that the mqtt client behaves way different.
First of all a (currently) non recoverable panic in the main loop happens, if you try to subscribe to any topic, before the connected callback is received (I do not know if this was ever valid, but it worked):
let mut client =  EspMqttClient::new_cb(&config.mqtt_url, &mqtt_client_config, move |event| {yadda yadda}
client.subscribe(stay_alive_topic.as_str(), ExactlyOnce)?;

If you do this you get a:
Guru Meditation Error: Core  0 panic'ed (Load access fault). Exception was unhandled.

Core  0 register dump:
MEPC    : 0x42072eca  RA      : 0x42072ec6  SP      : 0x40839a50  GP      : 0x408173e4
0x42072eca - run_event_loop
    at /home/empire/workspace/PlantCtrl/rust/.embuild/espressif/esp-idf/v5.2.1/components/mqtt/esp-mqtt/mqtt_client.c:1539
0x42072ec6 - run_event_loop
    at /home/empire/workspace/PlantCtrl/rust/.embuild/espressif/esp-idf/v5.2.1/components/mqtt/esp-mqtt/mqtt_client.c:1539
0x408173e4 - __global_pointer$
    at ??:??

the same code works on a esp32 with the xtensa toolchain, so apparently there is some difference in the blocking/async behaviour of the idf core here. While I would prefer the subscribe return a error result, I'm not sure if this is even possible (or a good idea for that) without storing the connection state from the callbacks somewhere in the idf wrapper.

We have to - as annoying as it is - first understand the root cause. Perhaps by looking at the C code. I don't think it is normal for the MQTT client to panic.

Thanks for reporting it!
Can you also please give additional context for the riscv case:
What rust compiler version  esp-idf version, optimization level where you using?


feat: image upload

This PR adds support for attaching images to the threads and posts.
I resigned from other types of files since no one reasonably would download a ZIP file from a stranger. Larger projects can be hosted on GitHub or Gist.
I took the approach of making all the files publicly available. Therefore there is no option to restrict access to the files. I find it reasonable since no one should send any sensitive data, and requiring to log in to see the images was never a standard in our forum.
More info can be found in images.md file.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

72 of 138 (52.17%)  changed or added relevant lines in 7 files are covered.
1 unchanged line in 1 file lost coverage.
Overall coverage decreased (-3.0%) to 79.234%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

app/Concerns/HasImages.php
34
35
97.14%

app/Concerns/HasTags.php
1
3
33.33%

app/Commands/CleanupImages.php
0
22
0.0%

app/Controllers/Discussions/ImageController.php
0
41
0.0%

Files with Coverage Reduction
New Missed Lines
%

app/Models/PostModel.php
1
92.0%

Totals

Change from base Build<PHONE_NUMBER>:
-3.0%

Covered Lines:
1076

Relevant Lines:
1358

💛  - Coveralls


can't download ice

When trying to download from https://scottrice.github.io/Ice/
I get the message 'cant connect to server'
cant open the page consolegrid.com/ice.zip

Also, the downloadable zip is hosted here on Github as well. Just go to the releases page of this repository.

As downloading grid images for roms is not working anymore i implemented a new scraper which uses thegamesdb.net images.
#493
You can download the binary from my fork:
https://github.com/SonicBoom89/Ice/releases/tag/1.0.1

I used your verison of Ice. When using Ice it add's the ROM's and then I'm stuck at "updating grid images for U:1200457".

What is the last entry of the log file?

consolegrid.com does not load


rancher.md: update default frontend rule

Updates the rancher docs to the actual current behavior.

Thanks for your interest in Traefik 😃
The frontend rule in Rancher it's not the same as Docker:
https://github.com/containous/traefik/blob/f6520727a34dea4322cd43a0dcb424af49876c75/provider/rancher/config.go#L112-L113
Your change is invalid.

Thanks. Better?

I have fixed the problem in my PR #2601.
Thanks for the pointer 👍


Finished per countries dataframes

Multiple warnings due to copied variables need to be revised.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


problem with publishing ports in swarm mode - works (sometimes) only on worker nodes, not on manager, no load balancing

3 nodes (node02 - master, node01 & node03 - workers)
on all of them docker 1.12
root@node02:~# docker node ls
ID                           HOSTNAME  MEMBERSHIP  STATUS  AVAILABILITY  MANAGER STATUS
24l3ymmyj28oagu8egrbds74d *  node02    Accepted    Ready   Active        Leader
2ojam2qis4hmxd833lsmr0655    node01    Accepted    Ready   Active
9l21flvdxqs9dftc61trl95eo    node03    Accepted    Ready   Active
root@node02:~# docker version
Client:
Version:      1.12.0-dev
API version:  1.25
Go version:   go1.6.2
Git commit:   2358d54
Built:        Fri Jul  1 03:39:09 2016
OS/Arch:      linux/amd64
Server:
Version:      1.12.0-dev
API version:  1.25
Go version:   go1.6.2
Git commit:   2358d54
Built:        Fri Jul  1 03:39:09 2016
OS/Arch:      linux/amd64
root@node02:~#
my Dockerfile:
FROM ubuntu:14.04
EXPOSE 80
RUN apt-get -y update && apt-get -y install apache2
CMD apachectl start && hostname > /var/www/html/index.html && ping some_host_on_internet.com
from this Dockerfile I built new image with name "www_server"
When I create new service with name "www-service" via command:
root@node02:~# docker service create --replicas 2 --name www-service --publish 80:80/tcp  www_server
my swarm put 2 copies on master-node :
root@node02:~# docker service tasks www-service
ID                         NAME           SERVICE      IMAGE       LAST STATE          DESIRED STATE  NODE
4xupw6dqolrqeymturu77ycgn  www-service.1  www-service  www_server  Starting 3 seconds  Running        node02
bxmug188but65xp5yax12hj4j  www-service.2  www-service  www_server  Starting 3 seconds  Running        node02
but none of 3 hosts open port 80 (tested by curl <IP_ADDRESS>:80 & telnet on <IP_ADDRESS> port 80)
when I rescale my service to 3 replicas:
root@node02:~# docker service scale www-service=3
www-service scaled to 3
swarm put new (3rd) container) to worker-node (node03)
root@node02:~# docker service tasks www-service
ID                         NAME           SERVICE      IMAGE       LAST STATE              DESIRED STATE  NODE
4xupw6dqolrqeymturu77ycgn  www-service.1  www-service  www_server  Running About a minute  Running        node02
bxmug188but65xp5yax12hj4j  www-service.2  www-service  www_server  Running About a minute  Running        node02
6hhlq6wtasgmuvqdotk9v1qt2  www-service.3  www-service  www_server  Starting 3 seconds      Running        node03
root@node02:~#
and on node03 swarm opened port 80, but not on node02 and not on node01
lets scale to = 4
root@node02:~# docker service scale www-service=4
www-service scaled to 4
root@node02:~# docker service tasks www-service
ID                         NAME           SERVICE      IMAGE       LAST STATE           DESIRED STATE  NODE
4xupw6dqolrqeymturu77ycgn  www-service.1  www-service  www_server  Running 2 minutes    Running        node02
bxmug188but65xp5yax12hj4j  www-service.2  www-service  www_server  Running 2 minutes    Running        node02
6hhlq6wtasgmuvqdotk9v1qt2  www-service.3  www-service  www_server  Running 56 seconds   Running        node03
90qz3jmnsf1jq28s9fuwljtyz  www-service.4  www-service  www_server  Preparing 1 seconds  Running        node01
And ONLY NOW swarm opens port 80 on 1st worker node (node01) , but still not open this port on node02 (even if there are 2 containers with my app)
And worse - there is no load-balancing - curl on node03 returns only 1st container
root@node03:/tmp# curl  <IP_ADDRESS> 80
8c605fa61d10
root@node03:/tmp# curl  <IP_ADDRESS> 80
8c605fa61d10
root@node03:/tmp# curl  <IP_ADDRESS> 80
8c605fa61d10

I've got something very similar with rc3 - if i kick things long enough, then some of the other nodes start to respond with html - but not all, and most requests are served by one web container.

I am experiencing the same problem.  The services created with swarm don't bind to random ports, only to the matching ports of the host machine and only one container can be attached to the hosts ports at a time.
I create this service:
docker service create --replicas 4 --name foobar --publish 8080:80 nginx
It launches successfully:
myUsername@consul0-3439:~$ docker service tasks foobar
ID                         NAME      SERVICE  IMAGE  LAST STATE           DESIRED STATE  NODE
8x4i0fqwh62aivlqx1ldbr6a9  foobar.1  foobar   nginx  Running 9 minutes    Running        client1-6803
4h6ui726y0woq80uepgdz1m98  foobar.2  foobar   nginx  Running 4 seconds    Running        client2-9869
7gq49n3xfburutveis4rylbe8  foobar.3  foobar   nginx  Preparing 4 seconds  Running        client0-8805
8gjmlielpunllq08pps5byd5r  foobar.4  foobar   nginx  Running 4 seconds    Running        client1-6803

However it does not publish itself to port 8080 on the host machine as I specified:
myUsername@consul0-3439:~$ docker -H :4000 ps
CONTAINER ID        IMAGE                                           COMMAND                  CREATED             STATUS              PORTS               NAMES
34bc67d15430        nginx:latest                                    "nginx -g 'daemon off"   4 minutes ago       Up 4 minutes        80/tcp, 443/tcp     client0-8805/foobar.3.7gq49n3xfburutveis4rylbe8
271682bc1566        nginx:latest                                    "nginx -g 'daemon off"   4 minutes ago       Up 4 minutes        80/tcp, 443/tcp     client1-6803/foobar.4.8gjmlielpunllq08pps5byd5r
f3a801188aa6        nginx:latest                                    "nginx -g 'daemon off"   4 minutes ago       Up 4 minutes        80/tcp, 443/tcp     client2-9869/foobar.2.4h6ui726y0woq80uepgdz1m98
3596fee1b2da        nginx:latest                                    "nginx -g 'daemon off"   14 minutes ago      Up 14 minutes       80/tcp, 443/tcp     client1-6803/foobar.1.8x4i0fqwh62aivlqx1ldbr6a9

Whats very annoying is that this was not an issue a week ago when I began experimenting with docker 1.12, so this is a brand new bug.

ping @mrjana

@slawekgh Do you use virtual box to create these nodes or does your nodes have more than one interface? If so, and if you did not provide a proper --listen-addr option then that might be your issue. We are trying to fix this in the next rc (by forcing users to provide a valid --listen-addr if auto-detection cannot be done reliable) but until then make sure you have a valid  listen-addr option.
If you do not have more than one valid IP addresses in your host please provide more information on how many nodes you have and a simple repro to triggering this problem.
Also if you are using a cloud provide like AWS you have to open up certain ports and you have to do that before you create the swarm cluster.

@SvenDowideit The above comments apply to your case as well. Further than that if you are using a browser to access your service, please keep in mind that the browser opens multiple connection requests towards the service and that will make it look the service is not getting load balanced. When in doubt always use something like curl to make sure that it is getting load balanced.

@cohenaj194 You seem to have asked for a specific PublishedPort for you service in your options with -p 8080:80. Because of that host binds to port 8080 as requested. Please keep in mind that the way port publishing works for services is a little different than containers. With a service a container doesn't get bind to that port. But a special port binding happens outside of the containers and that will take care of effectively load balancing between tasks of the same service. I don't quite get from your description what was working a week ago which stopped working now. Can you please elaborate? Because the port exposing behavior hasn't changed from rc2 to rc3

@mrjana I actually looked into the issue of why my swarm containers stopped publishing ports about a week ago and it turns out the change was due to a health check of mine (I had recently removed) that removes containers that fail a specific curl check and remakes them locally with regular docker run commands.
With this "special port binding that happens outside of the containers"  where does it bind itself to?  It is not binding itself to port 8080 and load balancing the service containers on a node.  However there is some type of strange port behavior happening, here is the output of some sudo netstat -tnlp commands (with irrelevant output removed):
Here is a change caused by my service:
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp6       0      0 :::8080                 :::*                    LISTEN      4038/dockerd

Here is a change caused by running the same type of docker container locally with the -P flag:
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp6       0      0 :::32769                :::*                    LISTEN      9133/docker-proxy

After removing the locally run container this last item was removed, however after removing the service port 8080 remained open and active in the output of sudo netstat -tnlp.  Honestly I just want to have my service containers grab a random port and work exactly as they would if I was running them locally.  Is there any way to do that?

@mrjana I'm using a set of ec2 boxes - @justincormack has access to them

@mavenugo can you assign a priority label to this issue, if needed?

@mrjana, I use virtualbox for my 3 machines, there is only one network interface on each of them
I have 3 nodes (node02 - master, node01 & node03 - workers) - on all of them docker 1.12
root@node02:~# docker node ls
ID                           HOSTNAME  MEMBERSHIP  STATUS  AVAILABILITY  MANAGER STATUS
24l3ymmyj28oagu8egrbds74d *  node02    Accepted    Ready   Active        Leader
2ojam2qis4hmxd833lsmr0655    node01    Accepted    Ready   Active
9l21flvdxqs9dftc61trl95eo    node03    Accepted    Ready   Active
Command I use to create service:
docker service create --replicas 2 --name www-service --publish 80:80/tcp www_server

@mrjana I found the solution to my issue and possibly yours too, docker services require a docker network to be reachable.  Follow these two steps and the service nodes will work, heres how it would work with your service:
docker network create -d overlay mynet
docker service create --replicas 2 --name www-service --network mynet --publish 80:80/tcp www_server

Afterwards all your service containers that are on any one node will all be automatically be load balanced between port 80 on that one node.

ping @mrjana @mavenugo @mgoelzer PTAL

from the further playing i've done - I expected docker-compose bundle to make everything "just work", and my compose file didn't have a network defined - it wasn't necessary for things to work in non-swarm mode.

I am having the same exactly problem. The other frustrating thing is that when I access a container that is actually listening on port 80 (PublishedPort) it seems to only half way work. TCP connections can be made, but the service takes so long to respond that browsers time out. This same container works fine stand alone mind you.
I get the same results with a dedicated network or not.

I have a similar issue: 2 masters and 2 workers on Azure, only workers behave as expected with a strict round robin trough tasks inside a service. managers fails every now and then.
I posted more informations on Docker forums

Make sure to check to docker.log file and ensure that you have these enabled in the kernel:
CONFIG_IP_VS
CONFIG_IP_VS_RR
CONFIG_IP_VS_PROTO_TCP
CONFIG_IP_VS_PROTO_UDP
CONFIG_IP_VS_NFCT
Maybe these should be added into https://github.com/docker/docker/blob/master/contrib/check-config.sh (if I am right that these are minimum requirement but it fixed the problem for me).

I am having the exact same issue as OP. It's like mesh routing/load balancing is not working at all.

Actually, switching from Ubuntu 16.04 to RancherOS did the trick. Mesh routing works all of a sudden.

@gnomff Confirmed that port publishing (seems like mesh routing / load balancing completely) is broken for me on Ubuntu 16.04. I've downgraded to 14.04 and publishing behaves as described here.

Looks like there are a few reports of people having issues with ubuntu 16.04. I will try to recreate the problem in 16.04 and see if there are any specific there in that distro.

@dperny will attempt to reproduce the issue with 16.04 and report here

After further investigation, I found that while this issue addresses a few of the possible symptoms I was facing in more vague terms, the actual problem with my swarm is described in #26530. Because removing services does not clean up the ports adequately, I've run into all kinds of funky behavior, especially on a cluster that has been up for a while. The routing mesh seems to exhibit intermittent outages when, in fact, ports aren't being allocated properly at all.

Cannot reproduce original issue on Ubuntu 16.04.1
@aluzzardi i think you can close this now.

Using docker for mac and still hit the same issue, service ports not mapped to host, however, one could manually publish the port once the service is up, as shown below:
$ docker service update --publish-add 80:80 my_service

Ubuntu 16.04, 3 nodes swarm cluster, docker 1.12.1
LoadBalancing works perfectly
root@node6:/tmp# cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04 LTS"

root@node6:/tmp# docker version
Client:
 Version:      1.12.1
 API version:  1.24
 Go version:   go1.6.3
 Git commit:   23cf638
 Built:        Thu Aug 18 05:33:38 2016
 OS/Arch:      linux/amd64

Server:
 Version:      1.12.1
 API version:  1.24
 Go version:   go1.6.3
 Git commit:   23cf638
 Built:        Thu Aug 18 05:33:38 2016
 OS/Arch:      linux/amd64

root@node6:/tmp# docker node ls
ID                           HOSTNAME  STATUS  AVAILABILITY  MANAGER STATUS
1bzwyjhnrr6tba49nbtnzzfsy    node7     Ready   Active
3nxypglehaeagmlzq963tcv2m *  node6     Ready   Active        Leader
5z18cmk87nfs9794sj8ugwevp    node8     Ready   Active

@dperny What do you think about those latest comments?

I'm having what I think is a similar error on 1.12.1. 6 nodes exposing 4 different ports for 4 different services, and getting a lot of connection refused errors.
The errors are intermittent; sometimes the connection works, and sometimes it doesn't. I've got HAProxy in front of the swarm, so it's handling the servers going up and down OK, but it's still surprising.
flapping.txt

However, I don't see a difference between master and worker nodes. Both flap for me.

I ran into issues where I was not able to access an exposed port on a service from other nodes and finally realized that, when I set up the cluster nodes from a single cloned image, I forgot to update the hostname for each node. After updating /etc/hostname and /etc/hosts and restarting the nodes, I had to kill the existing Swarm to get the hostname updates to be incorporated. Everything works as expected after the updates.

@slawekgh are you saying everything works now?

on ubuntu 16.04 (3 node cluster) everything is ok

on centos 7.2 (3 node cluster) doesnt work - it servers only local container www-content and load-balancing doesnt work

of course I opened all required ports from this document:
https://forums.docker.com/t/routing-mesh-for-docker-1-12-not-working/17279
and this :
https://docs.docker.com/engine/swarm/swarm-tutorial/#/open-ports-between-the-hosts
after I disabled firewalld on all 3 nodes it started working but only on 2nd and 3rd node and after migarting service by swarm is not stable - sometimes it serves content sometime not
seems like there is a huge problem with iptables/firewalld on centos ?

@dperny Can you share the broken setup with @mrjana ?

@dperny Can you share details on the problem that you are seeing?

@slawekgh #26235 is tracking the issue with CentOS & firewalld.
Yes, with firewalld accessing the published port using that node's IP doesn't work. firewalld is adding two REJECT rules in the filter table FORWARD and INPUT chains that seem to be blocking the overlay traffic. I also noticed that --add-port didn't help.

+1

@slawekgh Do I understand it right that you are able to publish port 80 on Ubuntu? In my case it's showing as published in docker service inspect but I cannot reach the container. This is happening on Ubuntu 16.04 on Azure (port 80 is open in firewall).

@maxfrigge I have the same problem with Debian Jessie after server reboot.

@vingrad Just figured that my issue is docker/dockercloud-haproxy it doesn't seem to work with swarm mode.

If I understand correctly:

This problem is not showing up on 1.12.1
There is a problem on Centos with 1.12.1 tracked as #26235

I'm going to close this one so we move the conversation to #26235 - please re-open if my understanding is wrong.

@aluzzardi why this got closed?  it seems u didn't understand that correctly!
Same problem still happened on ubuntu 14.04, docker 1.12.3 with ufw service disabled, just very simple setup, docker swarm init -> manger join -> worker join, then docker service create, port totally didn't get published to physical host!
lsb_release -a
leonard@docker01:~$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 14.04.5 LTS
Release:	14.04
Codename:	trusty

docker info:
Containers: 11
 Running: 11
 Paused: 0
 Stopped: 0
Images: 24
Server Version: 1.12.3
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 77
 Dirperm1 Supported: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: overlay host bridge null
Swarm: active
 NodeID: 1k3nh1kfpdq76yui6u8i96xf7
 Is Manager: true
 ClusterID: 29th4cjgz641vmjstnl3im2le
 Managers: 2
 Nodes: 4
 Orchestration:
  Task History Retention Limit: 5
 Raft:
  Snapshot Interval: 10000
  Heartbeat Tick: 1
  Election Tick: 3
 Dispatcher:
  Heartbeat Period: 5 seconds
 CA Configuration:
  Expiry Duration: 3 months
 Node Address: <IP_ADDRESS>
Runtimes: runc
Default Runtime: runc
Security Options: apparmor
Kernel Version: 3.13.0-91-generic
Operating System: Ubuntu 14.04.5 LTS
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 3.859 GiB
Name: docker01.sgdev.vcube.com
ID: LZ7J:32FB:MURU:Q6ZP:5LD3:LYUY:7KVW:CTKV:RNTS:RSQW:OZKP:6MN7
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Insecure Registries:
 <IP_ADDRESS>:7777
 <IP_ADDRESS>/8

docker node ls
ID                           HOSTNAME                  STATUS  AVAILABILITY  MANAGER STATUS
1k3nh1kfpdq76yui6u8i96xf7 *  docker01.sgdev.vcube.com  Ready   Active        Leader
3j581urv8k7lstx459vyil5c9    micro03.sgdev.vcube.com   Ready   Active
5vxw5j7hyd9xsmk85mk05g8cn    micro01.sgdev.vcube.com   Ready   Active        Reachable
9inye85f67s1ca6bof9etv71x    micro02.sgdev.vcube.com   Ready   Active

docker service create:
docker service create \
	--name ms-auth-jwt \
	-p 34348:53547/tcp \
	--env SERVICE_53547_ID=auth@<IP_ADDRESS>:34348 \
	--env SERVICE_53547_NAME=auth@<IP_ADDRESS>:34348 \
	--env SERVICE_TAGS=auth,micro02.sgdev.vcube.com \
	--constraint 'node.hostname == micro02.sgdev.vcube.com' \
	--restart-condition any \
	ms-auth-jwt:0.0.1

then on the target node, docker ps:
CONTAINER ID        IMAGE                           COMMAND                  CREATED             STATUS              PORTS               NAMES
ba75780de487        ms-auth-jwt:0.0.1               "node /app/app.js"       9 minutes ago       Up 9 minutes        53547/tcp           ms-auth-jwt.1.esn3zohugxoxc470mgeujv95s

ufw service status:
ufw stop/waiting

the port totally didn't get exposed to physical host.

@leonard-sky published ports for services in swarm mode go through the network mesh, and are not visible in docker ps. Try docker service inspect.

@thaJeztah thanks, checked the docker service inspect, i can see that exposed port, but registrator was complaining that: port XXX was not published to host, and removing the ability to publish port to host is kind of wried.

removing the ability to publish port to host is kind of wried.

The option to publish a container's port to the host is still present (i.e. docker run -p 8080:80 nginx still works as usual). The difference here is that services are not containers; when deploying a service, the service can (and in most cases will be) backed by multiple containers. When publishing a service port, that port is exposed through the routing mesh, and will be available on every node in the Swarm. The routing mesh automatically routes traffic to that port to the containers backing the service through a load balancer. For that reason, the container itself is not directly exposed.
Registrator (currently) is not updated to take the service concept into account (i.e., it still expects individual containers, and not services).
Note that for docker 1.13, we are considering adding a "mode=host" publish option (see https://github.com/docker/docker/pull/27917), which would publish service ports directly to the host a service's task ends on, instead of going through the routing mesh. However, this may also implicate that multiple tasks of a service cannot be deployed on the same node.

anybody successful publishing port ,I tried docker 1.12.2 but not able to publish port to host

@thaJeztah  got it now, thank u very much for this very clear and detailed explanation

@prparmar77  Please read the above thaJeztah's very clear explanation first.

Looks like mine is simple setup and I was using Oracle Linux 6.6 ,the service creation was filling docker logs with following information ;
time="2016-11-08T04:10:25-05:00" level=error msg="setting up rule failed, [-t nat -A POSTROUTING -m ipvs --ipvs -d <IP_ADDRESS>/16 -j SNAT --to-source <IP_ADDRESS>]:  (iptables failed: iptables -t nat -A POSTROUTING -m ipvs --ipvs -d <IP_ADDRESS>/16 -j SNAT --to-source <IP_ADDRESS>: iptables v1.4.7: Couldn't load match ipvs':/lib64/xtables/libipt_ipvs.so: cannot open shared object file: No such file or directory\n\nTry iptables -h' or 'iptables --help' for more information.\n (exit status 2))"
time="2016-11-08T04:10:25.620384456-05:00" level=error msg="Failed to add firewall mark rule in sbox /var/run/docker/netns/ingress_sbox: reexec failed: exit status 5"
After bit of investigation realised the issue is with iptables-1.4.7 that was on my linux. Unfortunately , the oel6.6 provides only the 1.4.7 version of it.
Later I tried this on Oracle Linux 7.2 and it worked fine on my worker node ,i.e I was able to publish the port from container to host via docker create service command with -p flag .Hope it helps somebody.

@prparmar77 can you open a separate issue for docker failing to work correctly on Oracle Linux 6?

ok created new issue #28190

@prparmar77 docker 1.12 requires uek-kernel 4.1 on oracle Linux 7.  If you run OEL 6.6, what is the docker version?

It is possible to expose more than one port with docker service?
If I run docker service create --name web -p 80:80 -p 443:443 nginx I only see the port 80 opened.

@Sispheor yes, you can publish multiple ports;
docker service create --name web -p 80:80 -p 443:443 nginx

And docker service inspect will show that both are published;
docker service inspect --pretty  web

ID:		0klkfxpzp0tpd14odhkoejtsb
Name:		web
Mode:		Replicated
 Replicas:	1
Placement:
UpdateConfig:
 Parallelism:	1
 On failure:	pause
ContainerSpec:
 Image:		nginx
Resources:
Ports:
 Protocol = tcp
 TargetPort = 80
 PublishedPort = 80
 Protocol = tcp
 TargetPort = 443
 PublishedPort = 443

However the problem in this case is that the default configuration of nginx only enables port 80, not 443;
docker run --rm nginx /bin/sh -c "cat /etc/nginx/conf.d/default.conf"

server {
    listen       80;
    server_name  localhost;

    #charset koi8-r;
    #access_log  /var/log/nginx/log/host.access.log  main;

    location / {
        root   /usr/share/nginx/html;
        index  index.html index.htm;
    }

    #error_page  404              /404.html;

    # redirect server error pages to the static page /50x.html
    #
    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root   /usr/share/nginx/html;
    }

    # proxy the PHP scripts to Apache listening on <IP_ADDRESS>:80
    #
    #location ~ \.php$ {
    #    proxy_pass   http://<IP_ADDRESS>;
    #}

    # pass the PHP scripts to FastCGI server listening on <IP_ADDRESS>:9000
    #
    #location ~ \.php$ {
    #    root           html;
    #    fastcgi_pass   <IP_ADDRESS>:9000;
    #    fastcgi_index  index.php;
    #    fastcgi_param  SCRIPT_FILENAME  /scripts$fastcgi_script_name;
    #    include        fastcgi_params;
    #}

    # deny access to .htaccess files, if Apache's document root
    # concurs with nginx's one
    #
    #location ~ /\.ht {
    #    deny  all;
    #}
}

So even though docker maps/publishes the ports, there's nothing listening inside the container on that port.
To make nginx listen on TLS connections (port 443), you need to provide it a custom configuration file, and add certificates (or mount them through a volume) (see the "complex configuration" section in the readme of the image: https://hub.docker.com/_/nginx/).
Note that docker 1.13 will come with a "secrets" option for services, which would be a good choice to provide the certificates to the service. (see https://github.com/docker/docker/pull/27794)
Having said the above, I think the official nginx image should not EXPOSE 443, because in the default configuration nothing is listening on that port. I'll open a pull-request / proposal for that in the nginx repository (https://github.com/nginxinc/docker-nginx)

I opened PR https://github.com/nginxinc/docker-nginx/pull/130 for discussion

Hi, thank you very much for your answer. Actually the command I wrote was an example and not the real one.
This is the one I use (base on httpd image)
docker service create \
--name web \
-p 80:80 \
-p 443:443 \
--mount type=bind,source=/home/tester/vhost.conf,target=/etc/httpd/conf.d/vhost.conf \
--mount type=bind,source=/home/tester/ssl,target=/etc/httpd/ssl/ \
--network isolated_swarm_nw \
localhost:5000/centos7-httpd

And then when I check on the node.
 docker ps -a
CONTAINER ID        IMAGE                                 COMMAND                CREATED             STATUS                  PORTS               NAMES
ce527cb6ec10        localhost:5000/centos7-httpd:latest   "/run-httpd.sh"        13 seconds ago      Up 12 seconds           80/tcp              web.1.2e8lxiz2tanatye4riz0g2ql8

The docker service command is just a translation of this docker run command
docker run -it \
--name web \
-p 80:80 \
-p 443:443 \
--network=isolated_nw \
-v /home/tester/vhost.conf:/etc/httpd/conf.d/vhost.conf \
-v /home/tester/ssl:/etc/httpd/ssl/ \
-d centos7-httpd

When I run this last, everything went weel, both port are open
47823682e1bc        centos7-httpd           "/run-httpd.sh"          6 seconds ago       Up 5 seconds        <IP_ADDRESS>:80->80/tcp, <IP_ADDRESS>:443->443/tcp   web

in fact you'll see the docker ps does not show any port is published (it only shows that according to the container, port 80 is exposed; 80/tcp) if the port was published, it would show (for example) <IP_ADDRESS>:80->80/tcp.
Containers that are backing a service are not individually published. When publishing a service's ports, the published ports go through the routing mesh, and use the internal load balancer (so when you request a port, the load balancer makes sure that traffic is routed to all containers backing the service).
To check if ports are published, you need to check the service not the containers (docker service inspect). Basically docker ps should not be used for managing services; containers backing a service are managed by Swarm, and in most cases you don't have to interact with them directly.

It's ok. The iptables config was full of wrong lines. Certainly due to a lot of service rm and start. After a clean of iptables and a restart of docker everything works fine.
Thanks for your support @thaJeztah !

rhel 7.2 + docker 1.12.2 + firewalld disabled
over 400 tests , everything works great

Thanks @slawekgh! May be worth updating to 1.12.3 if possible, there's various fixes in that one, including one security related (https://github.com/docker/docker/releases/tag/v1.12.3)

I have test-infra with 1.12.2 only :-(

Still cannot access port 80 on 1.13_rc2

@hnaparst this discussion is getting lengthy; could you open a new issue with details?

It is the same as everyone else's details.  With just one node, the master,
ec2-user@manager1 ~ $ docker node ls
ID                           HOSTNAME  STATUS  AVAILABILITY  MANAGER STATUS
qow9qn0jop7k0h6na140g63tg *  manager1  Ready   Active        Leader
docker network create -d overlay mynet
docker service create --replicas 2 --name network --network mynet --publish 80:80/tcp nginx
wget localhost
--2016-12-03 01:17:18--  http://localhost/
Resolving localhost... ::1, <IP_ADDRESS>
Connecting to localhost|::1|:80... connected.
HTTP request sent, awaiting response... ^

@hnaparst does it work with wget -4 localhost? Looks like it resolves to the IPv6 localhost address

Maybe I am missing some kernel options.
ec2-user@manager1 ~ $ wget -4 localhost
--2016-12-03 01:24:54--  http://localhost/
Resolving localhost... <IP_ADDRESS>, <IP_ADDRESS>
Connecting to localhost|<IP_ADDRESS>|:80... failed: Connection refused.
Connecting to localhost|<IP_ADDRESS>|:80... failed: Connection refused.

@hnaparst if it's just from localhost, it may not be related to the issue here, could be issue with the userland-proxy. If it's not publishing the port at all, could also be https://github.com/docker/docker/issues/26235. there's many possible options with networking, related to setup, configuration, which was my reason for asking to open a separate issue with details

It was something with my kernel options.  It works with Rancher 0.7.1, kernel 4.4.24.  Sorry for my lameness.  Thanks for your help and patience.

I'm seeing the same problem: The routing mesh doesn't work at all. However, the netstat output on the worker node is a bit interesting. It seems that 7946 is only open to IPv6 traffic on the worker:
root@swrmwrkr-01 [ ~ ]# netstat -tulpen
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       User       Inode       PID/Program name
tcp        0      0 <IP_ADDRESS>:22              <IP_ADDRESS>:*               LISTEN      0          14604       1017/sshd
tcp        0      0 <IP_ADDRESS>:7001          <IP_ADDRESS>:*               LISTEN      0          12886       612/etcd
tcp        0      0 <IP_ADDRESS>:4001          <IP_ADDRESS>:*               LISTEN      0          12892       612/etcd
tcp        0      0 <IP_ADDRESS>:2379          <IP_ADDRESS>:*               LISTEN      0          12889       612/etcd
tcp        0      0 <IP_ADDRESS>:5355            <IP_ADDRESS>:*               LISTEN      77         12785       599/systemd-resolve
tcp        0      0 <IP_ADDRESS>:2380          <IP_ADDRESS>:*               LISTEN      0          12883       612/etcd
tcp        0      0 <IP_ADDRESS>:111             <IP_ADDRESS>:*               LISTEN      0          12078       562/rpcbind
tcp6       0      0 :::22                   :::*                    LISTEN      0          14622       1017/sshd
tcp6       0      0 :::2181                 :::*                    LISTEN      996        13125       649/java
tcp6       0      0 :::7946                 :::*                    LISTEN      0          1319140     36752/dockerd
tcp6       0      0 :::5355                 :::*                    LISTEN      77         12787       599/systemd-resolve
tcp6       0      0 :::43853                :::*                    LISTEN      996        13086       649/java
tcp6       0      0 :::111                  :::*                    LISTEN      0          12099       562/rpcbind
tcp6       0      0 :::8080                 :::*                    LISTEN      0          1350547     36752/dockerd
tcp6       0      0 :::80                   :::*                    LISTEN      0          12865       664/httpd
udp        0      0 <IP_ADDRESS>:68      <IP_ADDRESS>:*                           76         1350850     2060/systemd-networ
udp        0      0 <IP_ADDRESS>:111             <IP_ADDRESS>:*                           0          12075       562/rpcbind
udp        0      0 <IP_ADDRESS>:123          <IP_ADDRESS>:*                           87         1320361     633/ntpd
udp        0      0 <IP_ADDRESS>:123     <IP_ADDRESS>:*                           87         1111825     633/ntpd
udp        0      0 <IP_ADDRESS>:123           <IP_ADDRESS>:*                           0          12814       633/ntpd
udp        0      0 <IP_ADDRESS>:123             <IP_ADDRESS>:*                           0          12810       633/ntpd
udp        0      0 <IP_ADDRESS>:4789            <IP_ADDRESS>:*                           0          1320134     -
udp        0      0 <IP_ADDRESS>:693             <IP_ADDRESS>:*                           0          12077       562/rpcbind
udp        0      0 <IP_ADDRESS>:5355            <IP_ADDRESS>:*                           77         12726       599/systemd-resolve
udp6       0      0 :::69                   :::*                                0          11531       1/systemd
udp6       0      0 :::111                  :::*                                0          12097       562/rpcbind
udp6       0      0 fe80::d8b3:75ff:fec:123 :::*                                87         1320391     633/ntpd
udp6       0      0 fe80::42:72ff:fe48::123 :::*                                87         1320365     633/ntpd
udp6       0      0 fe80::20c:29ff:feb6:123 :::*                                87         1111828     633/ntpd
udp6       0      0 ::1:123                 :::*                                0          12818       633/ntpd
udp6       0      0 :::123                  :::*                                0          12807       633/ntpd
udp6       0      0 :::693                  :::*                                0          12098       562/rpcbind
udp6       0      0 :::5355                 :::*                                77         12786       599/systemd-resolve
udp6       0      0 :::7946                 :::*                                0          1319141     36752/dockerd

However, it looks like the manager node is trying to contact it over IPv4:
Feb 16 21:40:11 swrmmgr-01 docker[51324]: time="2017-02-16T21:40:11.333925196Z" level=warning msg="2017/02/16 21:40:11 [ERR] memberlist: Push/Pull with swrmwrkr-01-16835daa070d failed: dial tcp <IP_ADDRESS>:7946: i/o timeout\n"

Is there a way to force all nodes to listen to IPv4?

@prydin could you're  open a new issue with details (as requested in the issue template), steps to reproduce? This issue was closed, and there's a number of different issues related to networking being reported here (not all related); having a separate issue makes it easier to follow, and to confirm it's the same issue / different issue.
I'm locking the conversation on this issue to prevent it from collecting possibly unrelated issues; if you're having issues on an up-to-date version of docker, and suspect there's a bug at hand, please open a new issue with details


Windows binding Raw Socket with TCP protocol immediately kills process

no errors captured. If I simply try to send and not bind I get An invalid argument was supplied. (os error 10022)
 use socket2::{Domain, Protocol, SockAddr, Socket, Type};
use std::{env, net::SocketAddr, process::{Command, Stdio}, thread, time::Duration, io::{BufReader, BufRead}};

fn main() -> std::io::Result<()> {
    let mut args: Vec<String> = std::env::args().collect();
    args.remove(0);

    if args.len() == 0 {
        let file_path = env::current_exe().unwrap();

        let file_path = file_path.to_str().unwrap();

        let (command, cmd_args) = (
            "powershell.exe",
            vec![
                String::from("-NoProfile"),
                String::from("-NoExit"),
                String::from("-Command"),
                String::from("Start-Process"),
                String::from(file_path),
                String::from("-Verb"),
                String::from("RunAs"),
                String::from("-ArgumentList"),
                String::from("'bind'"),
            ],
        );

        let mut cmd = Command::new(command)
            .args(cmd_args)
            .stderr(Stdio::piped())
            .spawn()
            .unwrap();

        let err = cmd.stderr.take().unwrap();
        let reader = BufReader::new(err);

        for line in reader.lines() {
            println!("{}", line.unwrap());
        }

        println!("DONE");
    } else {
        thread::sleep(Duration::from_secs(5));

        let address = SockAddr::from(SocketAddr::new([0, 0, 0, 0].into(), 0));

        let send_socket = Socket::new(Domain::IPV4, Type::RAW, Some(Protocol::TCP))?;

        send_socket.bind(&address)?;

        loop {}
    }

    Ok(())
}

Can you reduce the code example? It has a lot of stuff in it unrelated to socket2 such as running a command and an infinite loop.

The reason for doing so is to provide the privileges needed. Given privileges, the else statement is sufficient. The loop is there simply to demonstrate the failure - the process immediately dies. Im on Win 11. According to the docs
On Windows 7, Windows Vista, Windows XP with Service Pack 2 (SP2), and Windows XP with Service Pack 3 (SP3), the ability to send traffic over raw sockets has been restricted in several ways:
TCP data cannot be sent over raw sockets.

its not clear if these docs are stale or not...

According to  Nmap
Nmap only supports ethernet interfaces (including most 802.11 wireless cards and many VPN clients) for raw packet scans. Unless you use the -sT -Pn options, RAS connections (such as PPP dialups) and certain VPN clients are not supported. This support was dropped when Microsoft removed raw TCP/IP socket support in Windows XP SP2. Now Nmap must send lower-level ethernet frames instead.
if you read the MS docs above TCP Raw sockets are treated as datagram sockets... and if I use a DGRAM / TCP socket then the process fails just like a RAW / TCP socket....


can not supported one build everywhere run

what happened？i found some jni method，i don't know detail，but i want compose-jb can one build everywhere run.
eg: we can try some thing to deal with it when we build a native app in macos , i want the app can run everywhere( x86_64 / arm ).
but now i need build in m1 ,build in intel to save Diffrent CPU Core. at now, i'm writing a SerialCom program, i package all native component in a module, when i use it, it can found system detail to choose current component.

Regretfully I can't understand your question :( Could you try to reformulate it please?

I think he targets a container that is compiled on a platform X (e.g. MacOS) and runs also on other platforms X, Y, Z, (e.g. also Windows, Linux, MacOS on M1, ....). This was possible with JVM -- but I think Skija adds binary dependencies that are platform-dependent.
Technically this would mean we would have to

add also Skija binaries for all platforms in the resulting container.
have a platform-independent container-format -- or a number of different containers, one for each platform.
maybe some adaptions in the compose plugin to deal with 1 and 2 (?)

add all binary dependencies in the resulting container is a good idea

Each platform has it's own distribution format. You can't install Mac package on Windows and vice versa. It is not a matter of Compose, but a matter of different formats.
For a specific OS you, in theory, could package different Skia binaries into one package - just replace
implementation(compose.desktop.currentOs)
with a desktop-jvm-macos-arm64 and desktop-jvm-macos-x64 dependencies (for Mac) for example. This will make sure both binaries are packed into the distribution. But I'm not sure this will work - never saw bundles that works on both OSes

Please check the following ticket on YouTrack for follow-ups to this issue. GitHub issues will be closed in the coming weeks.


Build zip with Xcode 13.3.1

diff looks reasonable, so I'll merge. Some highlights:
diff -r /Users/paulbeusterien/zips/9.0.0/Firebase-release-zip-zip/9_0_0/Firebase/README.md Firebase/README.md
13c13
< Xcode 13.2.1 or newer is required.
---
> Xcode 13.3.1 or newer is required.
263c263
< GTMSessionFetcher                      | 1.7.1
---
> GTMSessionFetcher                      | 1.7.2

diff -r /Users/paulbeusterien/zips/9.0.0/Firebase-release-zip-zip/9_0_0/Firebase/FirebaseStorage/FirebaseStorage.xcframework/tvos-arm64_x86_64-simulator/FirebaseStorage.framework/Modules/FirebaseStorage.swiftmodule/arm64-apple-tvos-simulator.swiftinterface Firebase/FirebaseStorage/FirebaseStorage.xcframework/tvos-arm64_x86_64-simulator/FirebaseStorage.framework/Modules/FirebaseStorage.swiftmodule/arm64-apple-tvos-simulator.swiftinterface
2c2
< // swift-compiler-version: Apple Swift version 5.5.2 (swiftlang-13<IP_ADDRESS> clang-13<IP_ADDRESS>)
---
> // swift-compiler-version: Apple Swift version 5.6 (swiftlang-<IP_ADDRESS>3.62 clang-13<IP_ADDRESS>)


Question about sharing a client

I'm implementing a web service and I need to connect to S3 in my endpoints. What's the best way to tackle the rusoto S3 client?
Should I create one single client and share it for the whole application or should I create one new client per request?

I would take this snippet as a reference:
https://github.com/wezm/upload-to-s3/blob/master/src/main.rs
To your question, I would definitely use a single client to start with.

I've used lazy_static to create a new client and then used that across tons of threads without any issues that I'm aware of.

I would love to see data on this: creating and destroying a service client per request is probably less efficient than reusing a single client, but I don't know for sure.
The lazy_static approach seems like it would work out nicely. 👍

Hi!
Let me try to add some more info to this discussion:
Creating a client is generally relatively expensive operation, as it will:

Create a hyper client from scratch, which will spawn a threadpool for blocking tasks such as DNS lookups.

Start with an empty credentials cache, meaning the first request will have to load credentials. This is less problematic if the credentials are passed through environment variables, but otherwise will either do a filesystem read for the credentials file, or worse ping the AWS metadata endpoint to retrieve instance credentials.

Especially (2) is something to be aware of. I was involved in incidents in the past where we would accidentally create new clients for every request, and the volume of requests lead AWS to throttle calls to the instance metadata endpoint, resulting in failing calls to AWS resources.
While creating a client can be quite costly, cloning an existing client is a lightweight operation which will simply share the underlying resources (both hyper client as well as credentials cache).
If the lazy_static approach works for you, that's great and will avoid the pitfalls I mentioned. Clients are both Send and Sync so no worries sharing and accessing them across threads.
My personal preference for sharing clients though would be to create a single client at a high level in your app, and then pass clones of this client down into the parts of your app that need it. This avoids having a global singleton with all the usual drawbacks this may have.
Hope this gives you a bit more data to decide on :)

Thank you for the information. I decided to share the client in the end :)


Add "entrypoint" interface to the core logic

Problem
The user's input shall be handled by the code logic according to the diagram below.

Proposed Solution

The Handler interface with the following public methods:

GenerateDiagram(string) (ResponseDiagram, error)

The Handler is configured with ClientPlantUML and ClientOpenAI

Acceptance Criteria

The logic is implemented in handler.go
Test coverage is at least 80%

It was decided to squash the task. The logic shall be implemented at a higher, interface, level for the sake of simplicity - no need to proxy two functions calls through an additional interface as this stage.


Make pid option on platforms other than linux

Closes #895
Signed-off-by: Michael Crosby<EMAIL_ADDRESS>

Do you want to pick these up too?

no, i'm not changing it to a pointer to an int

I removed it in the schema code

no, i'm not changing it to a pointer to an int

Then are you at least setting omitempty?  Also, how is this different from timeout, where you asked for a pointer?

@wking lets not discuss pointers vs non-pointers here.  we have went round in circles before and i'm not going to start again

@mrunalp @vbatts PTAL

LGTM

LGTM


Update Composer dependencies (2021-11-12-00-09)

Visual regression test passed!
View the visual regression test report


Ensures that EVP encryption & decryption operations check the encrypt flag on the context.

I'm new to this codebase, so sorry if I made any style mistakes. Also, I have a couple questions:

do we need to set EVPerr before returning 0;
should we unittest this case?

I believe EVP should make it hard to shoot yourself in the foot, so this change ensures that a user cannot accidentally decrypt data with an encryption context or vice-versa. For example, without the check, if an encryption context is used to decrypt EVP_aes_256_gcm encrypted data, the code will fail to validate the TAG.
Example code availabe at: http://quaxio.com/wtf/openssl_wtf.html

I like the concept, but I think returning 0 without setting an error code isn't a good idea.

Feel free to modify the PR and merge it.

It's unfortunate that this hasn't been looked at for so long.  The code looks correct (apart from the nit I picked), but it needs a rebase to a fresher master.

This PR is from pre-reformat times. Either we should bring it up to date and fix the nit, or abandon it. @levitte?

Like I said, "it needs a rebase to a fresher master"

Ok. So I suggest unless @alokmenghrajani feels like bringing this up to date, we should close this PR.

Agreed

Not psyched to go over this ~2 years later.

See for instance the question on SO in the next URL to see that this causes issues.
https://stackoverflow.com/questions/53690728/openssl-evp-decryption-fails-for-ecb-and-cbc-but-works-for-ofb/53691697#comment94239271_53691697
The routines are currently ignoring the fail fast and least surprise design principles. Please fix.

Feel free to create an other pull request to add those checks.

@owlstead re "Please don't close issues if they still persist.", this wasn't an issue but a PR. By my books, it's fine to close PRs if no progress is being made.
@levitte Appreciate that you eventually got to fix this issue.

@alokmenghrajani Thank you for initially proposing the code. But your PR consists of code to fix an issue, correct? It doesn't offer any other functionality but to correctly report an error if the API is misused.

@owlstead thank you for eventually applying the PR and not keeping the author from the original commit. Sure. Happy?

Hmm, not about the author part, but you're clearly the initiator and it is fixed so yeah - very happy :)


StoreOptions.path being ignored

Hi -
I'm getting uploads working full round-trip, but having one small issue.  The path option seems to be ignored when I upload a file.  When I inspect the blob object in the delegate method
- (void)fsPicker:(FSPickerController *)picker didFinishedPickingMediaWithBlobs:(NSArray<FSBlob *> *)blobs
I also see that path is nil.  The file successfully uploads to s3, but always uploads to the root of my s3 container.
Here is the relevant pieces of code.
    FSConfig *config = [[FSConfig alloc] initWithApiKey:@"key_removed"];

    FSStoreOptions *storeOptions = [[FSStoreOptions alloc] init];
    storeOptions.location = FSStoreLocationS3;
    storeOptions.path = @"uploads/";

    config.storeOptions = storeOptions;
    config.selectMultiple = NO;
    config.sources = @[FSSourceCameraRoll, FSSourceCamera];

    FSTheme *theme = [FSTheme filestackTheme];

    FSPickerController *fsPickerController = [[FSPickerController alloc] initWithConfig:config theme:theme];
    fsPickerController.fsDelegate = self;

    [self presentViewController:fsPickerController animated:YES completion:nil];

Thanks -
John

Hi @JohnnyCon please run pod update, it should update Filestack (not FSPicker) to version 0.3.1 and let me know if it is working for you.

Confirmed that path is now being honored.  Thanks!


SP-1691 - Backport of PDI-13367 - ETL Metada Injection does not actualize the "Source Step" in the "Inject Metadata" tab when renaming the cited step (5.3 Suite)

@mattyb149 Please, look it.

@mattyb149 The fix was updated.


Support PyO3

Feature Request

[x] runtime schema (in progress)

https://github.com/tonbo-io/tonbo/pull/165

[x] PyO3

It is complete by #181


Crash after running for a while "overflow when subtracting durations"

Several times I've had ytop left running and came back to find it crashed with this backtrace:
Backtrace (most recent call first):
  File "src/libcore/option.rs", line 1185, in core::option::expect_failed
  File "src/libcore/option.rs", line 345, in core::option::Option<T>::expect
  File "src/libcore/time.rs", line 744, in <core::time::Duration as core::ops::arith::Sub>::sub
  File "<unknown>", line 0, in psutil::process::process::Process::cpu_percent
  File "<unknown>", line 0, in <core::iter::adapters::Map<I,F> as core::iter::traits::iterator::Iterator>::try_fold
  File "<unknown>", line 0, in <alloc::vec::Vec<T> as alloc::vec::SpecExtend<T,I>>::from_iter
  File "<unknown>", line 0, in <ytop::widgets::proc::ProcWidget as ytop::update::UpdatableWidget>::update
  File "<unknown>", line 0, in ytop::update::update_widgets
  File "<unknown>", line 0, in ytop::main
  File "<unknown>", line 0, in std::rt::lang_start::{{closure}}
  File "src/libstd/rt.rs", line 48, in std::rt::lang_start_internal::{{closure}}
  File "src/libstd/panicking.rs", line 287, in std::panicking::try::do_call
  File "src/libpanic_unwind/lib.rs", line 78, in __rust_maybe_catch_panic
  File "src/libstd/panicking.rs", line 265, in std::panicking::try
  File "src/libstd/panic.rs", line 396, in std::panic::catch_unwind
  File "src/libstd/rt.rs", line 47, in std::rt::lang_start_internal
  File "<unknown>", line 0, in main
  File "<unknown>", line 0, in __libc_start_main
  File "<unknown>", line 0, in <unknown>
  File "<unknown>", line 0, in <unknown>

The application panicked (crashed).
  overflow when subtracting durations
in src/libcore/option.rs, line 1185
thread: main

In case it helps, I managed to repro (wait long enough) with full debuginfo:
Backtrace (most recent call first):
  File "src/libcore/option.rs", line 1191, in core::option::expect_failed
  File "src/libcore/option.rs", line 347, in core::option::Option<T>::expect
  File "src/libcore/time.rs", line 753, in <core::time::Duration as core::ops::arith::Sub>::sub
  File "/home/matthias/.cargo/git/checkouts/rust-psutil-b7c53c3e828a6c66/d16e3f7/src/utils.rs", line 40, in psutil::utils::calculate_cpu_percent
    (((second - first).as_nanos() as f64 / total_diff.as_nanos() as f64) * 100.0) as f32
  File "/home/matthias/.cargo/git/checkouts/rust-psutil-b7c53c3e828a6c66/d16e3f7/src/process/process.rs", line 130, in psutil::process::process::Process::cpu_percent
    let percent = calculate_cpu_percent(self.busy, busy, instant - self.instant);
  File "src/widgets/proc.rs", line 232, in <ytop::widgets::proc::ProcWidget as ytop::update::UpdatableWidget>::update::{{closure}}
    cpu: process.cpu_percent()? / cpu_count,
  File "rust:src/libcore/iter/adapters/mod.rs", line 779, in core::iter::adapters::map_try_fold::{{closure}}
  File "rust:src/libcore/iter/traits/iterator.rs", line 1876, in core::iter::traits::iterator::Iterator::try_fold
  File "rust:src/libcore/iter/adapters/mod.rs", line 805, in <core::iter::adapters::Map<I,F> as core::iter::traits::iterator::Iterator>::try_fold
  File "rust:src/libcore/iter/traits/iterator.rs", line 2204, in core::iter::traits::iterator::Iterator::find_map
  File "rust:src/libcore/iter/adapters/mod.rs", line 1070, in <core::iter::adapters::FilterMap<I,F> as core::iter::traits::iterator::Iterator>::next
  File "rust:src/liballoc/vec.rs", line 2131, in alloc::vec::Vec<T>::extend_desugared
  File "rust:src/liballoc/vec.rs", line 2024, in <alloc::vec::Vec<T> as alloc::vec::SpecExtend<T,I>>::spec_extend
  File "rust:src/liballoc/vec.rs", line 2019, in <alloc::vec::Vec<T> as alloc::vec::SpecExtend<T,I>>::from_iter
  File "rust:src/liballoc/vec.rs", line 1919, in <alloc::vec::Vec<T> as core::iter::traits::collect::FromIterator<T>>::from_iter
  File "rust:src/libcore/iter/traits/iterator.rs", line 1658, in core::iter::traits::iterator::Iterator::collect
  File "src/widgets/proc.rs", line 218, in <ytop::widgets::proc::ProcWidget as ytop::update::UpdatableWidget>::update
    self.procs = self
  File "src/update.rs", line 30, in ytop::update::update_widgets
    widget.update();
  File "src/main.rs", line 147, in ytop::main
    update_widgets(&mut app.widgets, update_seconds);

The application panicked (crashed).
  overflow when subtracting durations
in src/libcore/time.rs, line 753
thread: main

@cjbassi Leave ytop running and hibernate computer for at least half a day. (I left it before going to sleep, and when I came back in the morning, it had crashed with this stacktrace.)

All right, it should be fixed in master and in the latest release. If not, let me know and I'll reopen this.

Up updated to v0.4.1 and it crashed again with the same issue:
Backtrace (most recent call first):
  File "src/libcore/option.rs", line 1188, in core::option::expect_failed
  File "src/libcore/option.rs", line 348, in core::option::Option<T>::expect
  File "src/libcore/time.rs", line 756, in <core::time::Duration as core::ops::arith::Sub>::sub
  File "<unknown>", line 0, in psutil::cpu::cpu_times_percent::calculate_cpu_times_percent
  File "<unknown>", line 0, in <alloc::vec::Vec<T> as alloc::vec::SpecExtend<T,I>>::from_iter
  File "<unknown>", line 0, in psutil::cpu::cpu_times_percent::CpuTimesPercentCollector::cpu_times_percent_percpu
  File "<unknown>", line 0, in psutil::cpu::cpu_percent::CpuPercentCollector::cpu_percent_percpu
  File "<unknown>", line 0, in <ytop::widgets::cpu::CpuWidget as ytop::update::UpdatableWidget>::update
  File "<unknown>", line 0, in ytop::update::update_widgets
  File "<unknown>", line 0, in ytop::main
  File "<unknown>", line 0, in std::rt::lang_start::{{closure}}
  File "src/libstd/rt.rs", line 52, in std::rt::lang_start_internal::{{closure}}
  File "src/libstd/panicking.rs", line 292, in std::panicking::try::do_call
  File "src/libpanic_unwind/lib.rs", line 78, in __rust_maybe_catch_panic
  File "src/libstd/panicking.rs", line 270, in std::panicking::try
  File "src/libstd/panic.rs", line 394, in std::panic::catch_unwind
  File "src/libstd/rt.rs", line 51, in std::rt::lang_start_internal
  File "<unknown>", line 0, in main
  File "<unknown>", line 0, in __libc_start_main
  File "<unknown>", line 0, in <unknown>
  File "<unknown>", line 0, in <unknown>

The application panicked (crashed).
  overflow when subtracting durations
in src/libcore/option.rs, line 1188
thread: main

Hmm, that's interesting. We can just reopen this one for now. I might have had the wrong idea about the original issue and this is just the original issue popping up in a different area.

@trolley Is it crashing after waking from hibernation or from suspension too?
So it seems like for some reason Linux is reporting lower CPU times at some point when ytop is running. I'm not really sure why it would be doing that.

No, this is on my desktop workstation that's running 24/7. I have tmux running with ytop in one pane and I'll switch to a different window then come back to it crashed.

All right, it should be fixed in master now. Of course, if theres still issues, let me know and I'll reopen this :sweat_smile:

@cjbassi Thanks. I'm installing using Homebrew so I'll wait for a new release an try it again :)

Ah, I just tried this and it lasted longer than it did before the change, but it still crashed later:
Backtrace (most recent call first):
  File "src/libcore/option.rs", line 1187, in core::option::expect_failed
  File "src/libcore/option.rs", line 347, in core::option::Option<T>::expect
  File "src/libcore/time.rs", line 744, in <core::time::Duration as core::ops::arith::Sub>::sub
  File "<unknown>", line 0, in psutil::cpu::cpu_times_percent::calculate_cpu_times_percent
  File "<unknown>", line 0, in <core::iter::adapters::Map<I,F> as core::iter::traits::iterator::Iterator>::fold
  File "<unknown>", line 0, in psutil::cpu::cpu_times_percent::CpuTimesPercentCollector::cpu_times_percent_percpu
  File "<unknown>", line 0, in psutil::cpu::cpu_percent::CpuPercentCollector::cpu_percent_percpu
  File "<unknown>", line 0, in <ytop::widgets::cpu::CpuWidget as ytop::update::UpdatableWidget>::update
  File "<unknown>", line 0, in ytop::update::update_widgets
  File "<unknown>", line 0, in ytop::main
  File "<unknown>", line 0, in std::rt::lang_start::{{closure}}
  File "src/libstd/rt.rs", line 48, in std::rt::lang_start_internal::{{closure}}
  File "src/libstd/panicking.rs", line 287, in std::panicking::try::do_call
  File "src/libpanic_unwind/lib.rs", line 81, in __rust_maybe_catch_panic
  File "src/libstd/panicking.rs", line 265, in std::panicking::try
  File "src/libstd/panic.rs", line 395, in std::panic::catch_unwind
  File "src/libstd/rt.rs", line 47, in std::rt::lang_start_internal
  File "<unknown>", line 0, in main
  File "<unknown>", line 0, in __libc_start_main
  File "<unknown>", line 0, in _start
  File "<unknown>", line 0, in <unknown>

The application panicked (crashed).
  overflow when subtracting durations
in src/libcore/option.rs, line 1187
thread: main

PS: Thank you for pointing me to the better_panic crate. ;) Now I'm using it in my project instead of colored-backtrace. Those are very nice panic messages.

Are you running any vm's by any chance?

Yep I've got an Ubuntu Vagrant machine running.

I have a Windows VM running in VMWare Workstation.

All right I just pushed another fix if someone would be so kind as to test it out :)

Sure thing. In the last couple days I've been using it to monitor my machine that I frequently max the memory out on. I'll let you know how it goes. :+1:

I think it worked. I've been running it almost all day. Earlier I would have gotten about two crashes in that time. :slightly_smiling_face:

Cool! Closing this, hopefully for the last time.


Scoped alternative to World API with granular dynamic borrow checking

This PR implements a scoped API that mirrors much of the World API, but does component-level borrow checking at runtime instead of relying on static borrow checking. See ErgoScope.
Still have a few improvements to do, and I hope to expand the API to include queries, but hopefully I can get some feedback on the design and implementation.
The primary use for this is more straight-forward gameplay code by avoiding mutable borrows on World, but also

Supporting arbitrarily deep callbacks that can modify any world state, while having ComponentRefs active
Supporting interacting with World from languages that don't support compile time borrow checking

I recently realized that with the indirection for data access that exists, it'd be possible to keep component references around between scopes, and just patch up the data pointer on scope exit/enter. Of course, actual access when a scope isn't active would panic. I imagine this could be nice as a way of storing persistent component references in scripts for example. What do you think @Ralith ?

It's an intersecting prospect. Off the cuff, I'm concerned that it would be very easy to capture pseudo-references that might be unpredictably invalidated while the scope is inactive. For example, if you have a script/task that gets a component, then waits, then manipulates that component, it would be prone to panicing if the associated entity was despawned for any reason. Would this be too large a footgun?

I think we can provide try_read/try_write functions for borrowing, so that errors can be propagated within the scripting environment as opposed to panicking. This would be equivalent to a null reference exception in Unity's C# for example, where the behaviour is similar to what you're talking about. Game objects & components can be destroyed between wait points of coroutines, in which case a null reference exception is thrown on access.

I wrote a small simulation to test how it is to use the API
https://github.com/kabergstrom/cogito/blob/master/src/main.rs

I think the next step for this feature is to support storing references in the World itself. The current implementation assumes that the number of referenced entities is small at any point in time (allocating a component reference is O(n) to the number of active refs). This has nice cache and memory usage for working with World on the call stack, but will fall apart when used for references between components in the World state. The implementation also has a restriction where it panics if a reference escapes the ErgoScope lifetime. This restriction needs to be lifted in order to be able to store references persistently.
My goal is to make persistent references as ergonomic as the transient references. I will need to implement some form of pointer patching to handle moving and removing components. I think the current indirection-block design is strong, but I think we will need memory compaction to make it more scalable and reliable. Compaction means pointer patching, and at that point we may as well do GC. And that's probably what I'll do - reflection-driven GC.


add the Köhler dataset

This adds the Köhler dataset, a popular benchmark dataset for blind deblurring.
What's slightly unusual about the Köhler dataset is that although it consists of blurry and sharp images, each blurry image is not associated with a single sharp image, but actually with about 200 of them. Measuring the deblurring performance using all sharp frames is considered to be more reliable than using a single one per blurry shot but it takes much longer to compute and users may often want to avoid that. For this reason, the implementation gives a single sharp frame per blurry shot using the standard dataset methods while allowing easy retrieval of all sharp frames associated to a given blurry shot using non-standard methods if needed.
Moreover, each of the 48 blurry shots is associated to one of 4 printed out images, and to one of 12 trajectories (blur settings). This makes indexing the dataset by a single number slightly awkward and for this reason the implementation allows retrieval by printout and trajectory using the non-standard method get_item while providing a sensible lexicographic ordering for the standard method __getitem__.
Checks to be done before submitting your PR

[ ] python3 -m pytest tests/ runs successfully.
[x] black . runs successfully.
[ ] make html runs successfully (in the docs/ directory).
[x] Updated docstrings related to the changes (as applicable).
[x] Added an entry to the CHANGELOG.rst.

The code should be good but the CI pipeline fails and I'm not sure why. Maybe it's because the dataset is heavy. 🤷

yes, you are actually using all of the disk space

https://github.com/deepinv/deepinv/blob/main/deepinv/datasets/flickr2k.py
for an example of not using doctest (>>>) but instead, a simple paragraph of codes

I've made the following changes

make sure that compiling the documentation does not download the dataset
add an annotation for skipping the single test that downloads the dataset

@MaximeSongIdris It should be ready for merging. Let me know if anything else needs to be done!

Thanks for the review!

This has been merged in PR #368