fix(controller): Roll back scale on failure

fix(controller): roll back if scale fails
fixes #3605
Log_event method was implemented in scheduler and app.id is passed to scheduler record logs to app logs. Rollback happens if any of the container state is not in created.
$ deis scale cmd=7
Scaling processes... but first, coffee!
503 Service Unavailable
Detail:
aborting, failed to create some containers rolling back
reshut2-82-75-dhcp:example-dockerfile-http smothiki$ deis logs
2015-05-14T23:13:45UTC deis[api]: smothiki created initial release
2015-05-14T23:14:34UTC deis[api]: smothiki deployed 6578692
2015-05-14T23:14:35UTC deis[api]: smothiki scaled containers cmd=1
2015-05-14T23:15:54UTC deis[api]: smothiki changed limits for memory
2015-05-14T23:45:46UTC deis[api]: smothiki scaled containers cmd=5
2015-05-14T23:46:38UTC deis[api]: smothiki scaled containers cmd=6
2015-05-14T23:46:47UTC deis[api]: smothiki scaled containers cmd=7
2015-05-14T23:46:47UTC deis[api]: 500 Server Error: Internal Server Error  no resources available to schedule container
2015-05-14T23:46:47UTC deis[api]: aborting, failed to create some containers rolling back
2015-05-14T23:46:47UTC deis[api]: smothiki scaled containers cmd=7
2015-05-14T23:46:47UTC deis[api]: 500 Server Error: Internal Server Error  no resources available to schedule container
2015-05-14T23:46:47UTC deis[api]: aborting, failed to create some containers rolling back
2015-05-14T23:46:47UTC deis[api]: smothiki scaled containers cmd=7
2015-05-14T23:46:47UTC deis[api]: 500 Server Error: Internal Server Error  no resources available to schedule container
2015-05-14T23:46:47UTC deis[api]: aborting, failed to create some containers rolling back
reshut2-82-75-dhcp:example-dockerfile-http smothiki$ deis info
=== luxury-neckwear Application
{
  "updated": "2015-05-14T23:46:38UTC",
  "uuid": "a77c23e2-da7c-4077-b201-ef4bf7032097",
  "created": "2015-05-14T23:13:45UTC",
  "url": "luxury-neckwear.smothiki.io",
  "owner": "smothiki",
  "id": "luxury-neckwear",
  "structure": {
    "cmd": 6
  }
}

=== luxury-neckwear Processes

--- cmd:
cmd.1 up (v3)
cmd.2 up (v3)
cmd.3 up (v3)
cmd.4 up (v3)
cmd.5 up (v3)
cmd.6 up (v3)

=== luxury-neckwear Domains
No domains

This adds a new parameter (appid or app--not consistent) to each scheduler class init call, but only the swarm scheduler does anything with it. If logging messages are added to swarm, shouldn't we be logging similar messages from fleet (and maybe mock and chaos)?

@bacongobbler @mboersma I have updated the code with docker exceptions. Please take a look

Removed appid argument from schedulers. catching docker exceptions instead of logging events

@smothiki Let's see what we can do to get this merged soon. I fear letting it hang about will lead to a lot of refactoring and rebasing.

 Adjusted blank lines as suggested by @mboersma
$ deis scale cmd=9
Scaling processes... but first, coffee!
 o..
  !    WARNING: Client and server API versions do not match. Please consider upgrading.
  !    Client version: 1.3
  !    Server version: 1.5

503 SERVICE UNAVAILABLE
Detail:
aborting, failed to create some containers
$ deis logs
2015-06-16T21:16:48UTC deis[api]: sm scaled containers cmd=5
2015-06-16T21:17:08UTC deis[api]: sm scaled containers cmd=4
2015-06-16T21:17:44UTC deis[api]: sm scaled containers cmd=9
2015-06-16T21:17:44UTC deis[api]: docile-caffeine_v3.cmd.9 (create): __init__() takes at least 3 arguments (2 given)
2015-06-16T21:17:44UTC deis[api]: aborting, failed to create some containers
2015-06-16T21:17:44UTC deis[api]: docile-caffeine_v3.cmd.9 (destroy): 404 Client Error: Not Found ("No such container: docile-caffeine_v3.cmd.9")
2015-06-16T21:38:53UTC deis[api]: sm scaled containers cmd=9
2015-06-16T21:38:53UTC deis[api]: docile-caffeine_v3.cmd.9 (create): 500 Server Error: Internal Server Error ("no resources available to schedule container")
2015-06-16T21:38:54UTC deis[api]: aborting, failed to create some containers
2015-06-16T21:38:54UTC deis[api]: docile-caffeine_v3.cmd.9 (destroy): 404 Client Error: Not Found ("No such container: docile-caffeine_v3.cmd.9")
$ deis info

  !    WARNING: Client and server API versions do not match. Please consider upgrading.
  !    Client version: 1.3
  !    Server version: 1.5

=== docile-caffeine Application
{
  "updated": "2015-06-16T21:17:18UTC",
  "uuid": "e1ad2367-06ca-4a44-a6c8-10e2fa9a3325",
  "created": "2015-06-16T21:13:19UTC",
  "url": "docile-caffeine.local3.deisapp.com",
  "owner": "sm",
  "id": "docile-caffeine",
  "structure": {
    "cmd": 4
  }
}

  !    WARNING: Client and server API versions do not match. Please consider upgrading.
  !    Client version: 1.3
  !        WARNING: Client and server API versions do not match. Please consider upgrading.
  !    Client version: 1.3
  !    Server version: 1.5

=== docile-caffeine Domains
No domains

Code LGTM.

Code LGTM.


fix: handle the error kindly when a certificate request fails

Thank you @akkuman for the change, this looks good to me.


Adds netstat -tulpn

Whoops, accidentally obliterated your space alias. I'll add it back manually.

Rip
On Sun, Jan 29, 2017, 1:26 PM Ben Roytenberg<EMAIL_ADDRESS>wrote:

Whoops, accidentally obliterated your space alias. I'll add it back
manually.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/roytenberg/brilliant-bash/pull/8#issuecomment-275934561,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AFu9Ryiq7-Cbte6SqtIyZHxZqxl_tksHks5rXNm3gaJpZM4Lw3Dq
.


Add custom script

I would like to add custom javascript to my slides for adding other functionalities, for example, call Reveal.js API directly, add Reveal.js plugins, add other rich content renderers like in http://mdp.tylingsoft.com/ .
I've tried adding script tags in PITCHME.md, but it doesn't work, probably due to the markdown parser.
And I also noticed that reveal-md has provided such function as "Custom Scripts".
Can you add this as an option just like the custom CSS? Or if I implement it, would this feature be accepted, or is this the preferable way?

@gitpitch ping?

Hi @utensil I have been giving this some thought. I am not yet sure of the practical value of your proposal but I'm open to explore it a little further. I have a few questions regarding some of your ideas which is probably a good place to start this conversation.
1. Call Reveal.js API
Can you provide one or two examples of when and how exactly you might use this functionality? I'm looking for concrete use cases for how it would improve/benefit a presentation experience, for either the presentation author or the audience.
2. Add Reveal.js Plugins
Allowing users to add custom JS would typically not suffice to activate plugins. Additional configuration at Reveal initialization would also be needed. Anyway, can you provide examples of specific plugins that you would like to see added? What are your use cases for those plugins?
3. Support MDP
How do you see something like MDP being used within a GitPitch presentation? GitPitch is itself a markdown rendered. As before, I'm interested in concrete use cases so I can evaluate the utility of the suggestion.
As you can see here, in all cases I'm trying to understand what practical utility, if any, would be provided by considering this feature request.

1. Call Reveal.js API
One of the use cases are:
Currently, not all Reveal.js options are exposed to PITCHME.yaml, so I need to manually call Reveal.configure({previewLinks:true}) etc.
2. Add Reveal.js Plugins
One of the use cases are:
zoom.js is not working(Zoom in and out with Alt+click), and I've noticed some plugins that I'd like to use, e.g. anything, chalkboard, spreadsheet and LaTeX Beamer-like progress indicator .
3. Render Rich Content
Not directly use mdp, but render rich contents such as Task list, mermaid charts, Definition list and much more. As long as I can use custom js, anything can be implemented, that's the spirit.

The spirit of what you are asking for sounds reasonable. However, the mechanism you are asking for is tantamount to formalizing support for cross-site scripting XSS-attacks-as-a-feature. Obviously this is not going to happen.
My advice is if there are features you would like to see added to GitPitch open an RFE. Those features will then become candidates for inclusion. Closing this RFE as will-not-implement.

I think the lack of scripting should be in the official documentation, with rationale - i.e., a specific example of an XSS attack that would be possible if scripts were enabled.
GitPitch's target market right now is primarily developers / technical doc writers (the cross section of markdown users with source control users). These are power users - ones who expect to have more control. So, I expect custom scripts to be a common request.
To me, a GitPitch presentation is a presentation of data - a fancy web page. And web pages allow scripting, so why not GitPitch?

I think the lack of scripting should be in the official documentation

I will look into adding a note to the docs - most likely alongside the custom css section - to spell out that custom scripts are not permitted.

with rationale -
i.e., a specific example of an XSS attack that would be possible if scripts were enabled.

The rationale is simple: bad actors exist and bad actors will exploit weaknesses for malicious purposes.
IF CUSTOM SCRIPTING was supported a malicious presentation author could bundle a malicious script and deliver that script to unsuspecting end users that visit the presentation. The end user’s browser has no way of knowing that the script should not be trusted, and will therefore execute the script. Damage done.
GitPitch end users must be confident that gitpitch.com is a trusted site. The way I guarantee that for my end users is (1) I control all executable code on every page and (2) I do that by preventing malicious code being bundled by presentation authors either accidentally or intentionally.
Here are two simple attacks that could easily be activated if custom scripts were permitted:
1. Keylogging Attack
IF CUSTOM SCRIPTING was supported a malicious presentation author could add custom code to register a keyboard event listener and then send all of the user's keystrokes to his own server, potentially recording sensitive information.
2. Phishing Attack
IF CUSTOM SCRIPTING was supported a malicious presentation author could insert a fake login form into a slide - using DOM manipulation - and then direct the form's action to target his own server, again tricking the user into submitting sensitive information.
I'm sure any XSS expert could provide many more sophisticated examples that would highlight the risks.

To me, a GitPitch presentation is a presentation of data - a fancy web page. And web
pages allow scripting, so why not GitPitch?

I can see the superficial similarities. But GitPitch is a slideshow presentation service - not a web hosting service. I try my very best to deliver a slideshow presentation service that delivers powerful features but also a service that can be trusted by presentation authors and their end users alike.
So if you really need to enhance your presentations with custom scripting then GitPitch is not for you. My best recommendation for you is to work with the reveal.js library directly and then to host the presentation on your own servers.

I recommend you consult with an XSS expert. Neither of the attack examples are XSS attacks, and I still don't see a good reason to disallow scripting.
System-wide keyloggers cannot be installed from JavaScript within a browser. There are multiple levels of security that prevent this. The only thing an in-browser event listener would be able to do is to listen to keystrokes entered into that presentation - and I can do the same (pretty much useless) thing on my blog or any other website.
The phishing attack doesn't require scripting at all. A malicious presentation author could just write an HTML form to begin with and style it with custom css.

Thanks for your additional feedback Stephen. The examples I provided were off the top of my head - and perhaps trivial. But I will take your suggestion and consult with experts to gain a more full understanding of the risks - real or imagined. When I have concrete data I will update this thread.

Personally I've chosen to use reveal-md for its support for custom scripting and markdown preprocessing, while maintaining compatibility with Gitpitch (e.g. using Gitpitch grammar for customizing similar features and not to show something weird on Gitpitch) and consider Gitpitch as a online fallback (missing some interactions etc.) while my audience would not bother to install reveal-md.
For security reasons, I understand it's easier to disable custom scripting than actively preventing XSS from the backend (due to the possible new ways to exploits it). And I also understand the degrees of freedom I asked comes with a lot of maintenance efforts regarding security and potential risks. That's why I finally used a method combining local (more freedom and less security concerns ) and online (less freedom and easier access).
But there is supposed to be more in this issue than simply discussing security. I've given three concrete use cases which could lead to other ways to satisfy, like supporting more plugins and more rich contents with community efforts. A direct action of closing this issue as will-not-implement without any further exploration on the use cases made me disappointed back then and found reveal-md. I apologize for not following up this issue and come back to share my thoughts on it.


I need to add something like placeholder

Im trying to set up a placeholder for my TextInputMask, but how ?

Its very easy, You just set the initial value on constructor then it will be the placeholder :), something like this :
constructor(props){
super(props);
this.state = {
initialValue: 'My Placeholder'
}
}
And then in the  TextInputMask, use the attribute :
value={this.state.textInput2}

Oh No !!
What if I set up type={'money'}, but I want to make my placeholder as string like "I want this to be placeholder" ??

Hi =),
Fixed at 1.3.4
Thanks for reporting =)


Staging to Production

Hello,
If I have a staging server and wanted to update pages/content, then push changes to production, would this script be what I'm looking for?
I have October on a staging server, clients will want to control their content etc but I do not want them doing this on the live site. Once they're done, they can push the changes to the master branch using a push to production button?
I have Bitbucket hooked up to Forge so that would trigger the deployment anyway for the last hurdle.
Thanks!

This plugin doesn't provide any functionality like that for you, it just gives you a way to connect shell scripts to webhooks. With that said, you should be able to write a script that does your deployment for you, and then wire it up to whatever event you want to trigger it,


No Services Listed

I started Bamboo on Mesos  & DockerHUb's  milk/bamboo:latest-port across our mesos slaves.

But when I try to list services using
curl -i http://X.X.X.X:8000/api/services
I get an empty JSON response
this is what the query produces. I'm running marathon 0.13.0 and mesos 0.25
curl -i http://<IP_ADDRESS>:8000/api/services
HTTP/1.1 200 OK
Content-Type: application/json
Date: Wed, 09 Dec 2015 18:17:07 GMT
Content-Length: 2
{}

this is what the query produces. I'm running marathon 0.13.0 and mesos 0.25
curl -i http://<IP_ADDRESS>:8000/api/services
HTTP/1.1 200 OK
Content-Type: application/json
Date: Wed, 09 Dec 2015 18:17:07 GMT
Content-Length: 2
{}

Does the marathon has any tasks deployed?

On 9 Dec 2015, at 18:23, Matt Parker<EMAIL_ADDRESS>wrote:
this is what the query produces. I'm running marathon 0.13.0 and mesos 0.25
curl -i http://<IP_ADDRESS>:8000/api/services
HTTP/1.1 200 OK
Content-Type: application/json
Date: Wed, 09 Dec 2015 18:17:07 GMT
Content-Length: 2
{}
—
Reply to this email directly or view it on GitHub.

116 instances

any logs from Bamboo being able to identity changes?
Can bamboo communicate with Marathon?

On 9 Dec 2015, at 19:10, Matt Parker<EMAIL_ADDRESS>wrote:
116 instances
—
Reply to this email directly or view it on GitHub.

I'm assuming so since the bamboo interfaces lists every instance (see graphic in first post)

core@r105u21 ~ $ docker logs bamboo
/usr/lib/python2.7/dist-packages/supervisor/options.py:295: UserWarning: Supervisord is running as root and it is searching for its configuration file in default locations (including its current working directory); you probably want to specify a "-c" argument specifying an absolute path to a configuration file for improved security.
  'Supervisord is running as root and it is searching '
2015-12-09 18:39:10,333 CRIT Supervisor running as root (no user in config file)
2015-12-09 18:39:10,333 WARN Included extra file "/etc/supervisor/conf.d/supervisord.conf" during parsing
2015-12-09 18:39:10,442 INFO RPC interface 'supervisor' initialized
2015-12-09 18:39:10,442 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2015-12-09 18:39:10,442 INFO supervisord started with pid 10
2015-12-09 18:39:11,446 INFO spawned: 'bamboo' with pid 13
2015-12-09 18:39:11,573 DEBG 'bamboo' stdout output:
2015/12/09 18:39:11 Using environment override MARATHON_ENDPOINT=http://<IP_ADDRESS>,http://<IP_ADDRESS>,http://<IP_ADDRESS>,http://<IP_ADDRESS>,http://<IP_ADDRESS>
2015/12/09 18:39:11 Using environment override BAMBOO_ENDPOINT=http://bamboo:8000
2015/12/09 18:39:11 Using environment override BAMBOO_ZK_HOST=zk://<IP_ADDRESS>:2181,<IP_ADDRESS>:2181,<IP_ADDRESS>:2181,<IP_ADDRESS>:2181,<IP_ADDRESS>:2181
2015/12/09 18:39:11 Using environment override BAMBOO_ZK_PATH=/bamboo
2015/12/09 18:39:11 Environment variable not set: STATSD_ENABLED
2015/12/09 18:39:11 Starting update loop

2015-12-09 18:39:11,635 DEBG 'bamboo' stdout output:
2015/12/09 18:39:11 bamboo_startup => 2015-12-09T18:39:11Z
2015/12/09 18:39:11 Queuing an haproxy update.

2015-12-09 18:39:11,678 DEBG 'bamboo' stdout output:
2015/12/09 18:39:11 Exec cmd: PIDS=`pidof haproxy`; haproxy -f /etc/haproxy/haproxy.cfg -p /var/run/haproxy.pid -sf $PIDS && while ps -p $PIDS; do sleep 0.2; done

2015-12-09 18:39:11,899 DEBG 'bamboo' stdout output:
[martini] listening on :8000 (development)

2015-12-09 18:39:12,273 DEBG 'bamboo' stdout output:
2015/12/09 18:39:12 HAProxy: Configuration updated

2015-12-09 18:39:13,274 INFO success: bamboo entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)

Does each service need to be configured individually to make it available through bamboo/haproxy? Or should they be available by default?

Depends which template you use. We mostly use host based routing with DNS. But if you use path based, the default template support it already without adding new entry in Bamboo API or UI.
We don't often use path based because some applications may need special treatment to render properly (some apps can't deal with paths correctly)

On 9 Dec 2015, at 19:27, Matt Parker<EMAIL_ADDRESS>wrote:
Does each service need to be configured individually to make it available through bamboo/haproxy? Or should they be available by default?
—
Reply to this email directly or view it on GitHub.


Fixed pydantic error in RagTool.from_embedchain

Returning the new RagTool instance caused a pydantic error and IMHO it's okay to modify the instance variables directly

Damn, just noticed that I should have created a feature branch


Used exports are excluded with the "unused export" comment when using export * as ...

Bug report

What is the current behavior?
Some used exports are excluded with the "unused export" comment when using export * as ...
If the current behavior is a bug, please provide the steps to reproduce.
(not really minimal reproduction, but it shows the problem very well)

Clone https://github.com/microsoft/monaco-editor
cd samples/browser-esm-webpack-monaco-plugin
Add file file.js with one line export * as monaco from 'monaco-editor/esm/vs/editor/editor.api';
Replace import * as monaco from 'monaco-editor/esm/vs/editor/editor.api'; in index.js with import { monaco } from './file';
NODE_ENV=production npm run build

What is the expected behavior?
dist/main.bundle.js should contain
_common_network_js__WEBPACK_IMPORTED_MODULE_8__/* .RemoteAuthorities.setPreferredWebSchema */ .WX.setPreferredWebSchema(/^https:/.test(window.location.href) ? 'https' : 'http');
but instead it has
/* unused export .RemoteAuthorities.setPreferredWebSchema */ undefined(/^https:/.test(window.location.href) ? 'https' : 'http');
which cause a runtime error (you can open dist/index.html in browser to see)
You can switch between the old line in index.js and the new one to see the difference.
Works similar in development mode.

Other relevant information:
webpack version:  5.68.0
Node.js version: 16.13.0
Operating System: macos
Additional tools:

@mifopen Could you check without monaco plugin in webpack.config ?

Just checked and yes, works ok without a plugin (there is another sample called browser-esm-webpack).
But what kind of bug could cause such strange behaviour? Сould you please tell me where to start?

it looks like plugin produces cjs exports https://github.com/microsoft/monaco-editor/blob/main/webpack-plugin/src/loaders/include.ts#L34 ..


Greedy Algorithm/Car Refueling Problem [C++]

This issue is about Greedy Algorithm/Car Refueling

[x] Issue Title is in form Algo/DS Name [Language].
[x] I searched or browsed the repo’s other issues to ensure this is not a duplicate.
[x] This Algo/DS is worth adding to this repository.
[x] Assign this issue to me (I want to work on this).

[ ] I ensure that I am not already assigned to 2 or more issues.

[x] I will send my PR only after I'm assigned to this issue by any maintainer.
[x] I understand that I will be unassigned from this issue if I fail to send a PR for this within a week.

@iiitv/project-maintainers I want to work on this issue.

wrong algo link, please fix

@anshumanv Sir, I have fixed the link.

I ensure that I am not already assigned to 2 or more issues.

Can't assign till this is checked, complete your open PRs meanwhile :)

@anshumanv Sir, now I have created the PRs for the earlier issues assigned to me.

@anshumanv Sir, I have completed my open PRs now, also I have checked the above statement. If possible, I would like to work on this issue.


VM: Switch to using QMP for configuring CPUs and Memory

WIP

Closing this until it gets prioritised.


netsuite password fix

https://github.com/fylein/fyle-integrations-app/assets/55541808/6c6b8e39-4659-45a4-9c3c-0bcf6a98b80d
https://github.com/fylein/fyle-integrations-app/assets/55541808/f1ce9ded-82f3-46e2-8e94-10558451891d
https://github.com/fylein/fyle-integrations-app/assets/55541808/47503804-8077-47ca-89d2-3e3566d6df80


Rename .github/ISSUE_TEMPLATE/pull_request_template.md to .github/pul…

…l_request_template.md
Git does not recognize the template under wrong folder

https://stackoverflow.com/questions/52139192/github-pull-requests-template-not-showing/52139815


Add test cases ids

Signed-off-by: ipinto<EMAIL_ADDRESS>
What this PR does / why we need it:
Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #
Special notes for your reviewer:
Release note:

NONE

/lgtm
Apart from removing the auto addition in go.mod by maybe vscode

@kbidarkar
go.mod doesn't no exists in the last commit

/approve
needs rebase though

Close this PR new PR created with test ids
https://github.com/kubevirt/kubevirt/pull/3380


[linux] Linux DeviceLayer needs WiFi ConnectivityMgr

Problem
The Linux device layer needs a ConnectivityMgr that handles a single, primary WiFi interface.
Proposed Solution
Implement the ConnectivityMgr with WiFi support.
Propose to interface directly to wpasupplicant.
Not in scope: conman, shill, or ubuntu / debian network manager.
Device layer assumes it is master of connectivity.  Target is embedded Linux type devices such as RasPi or smart speaker.
Include instructions for testing on a laptop or workstation, including instructions on how to disable local network mgmt and give control to CHIP app.

Status - 2020-06-10

Draft PR posted -- DBUS client API to wpasupplicant implemented
No ProvisioningService integration yet in CHIP -- only tested in Weave environment thus far
DBUS using gmain_loop -- may cause conflicts with ThreadStack -- team to coord offline
Suggest unit tests based with credentials loaded from file
Script to launch required services such as wpasupplicant
Difficult to define AP in simulation w/o hardware
Evaluate using mac80211 hwsim module

@kghost : Once #1084, only need a few days to land since PR already uses gio.
@yunhanw-google : How do you test WiFi provisioning?
@kghost : No tests right now -- simulating WiFi not trivial

Status 2020-07-22:

Was blocking dbus
Have not had bandwidth to revisit since dbus PR merged
reassigning to @yufengwangca

Status 2020-08-05:

2 PRs on Wifi merged
1 more in review
next to add shell commands to control and test wifi manager

All PRs to support WiFi over GDBus are merged. Next to add shell commands to control and test wifi manager which is tracked by https://github.com/project-chip/connectedhomeip/issues/1696


Implement User Seeding Functionality

Creating the first user is complicated and requires multiple steps.  This is confusing for first time users.
I suggest creating a seed-user.js file which will automatically create a user.  For example:
node seed-user.js
This file will simply need a configuration object that will automatically be passed into the function.  This object will include the user's credentials needed to login.  Furthermore, this configuration will be thoroughly documented within the seed-user.js file for easy setup.
This task will also update the README with updated instructions.

Closed with e59a4d5bd84cbb074a7cb345e347b35fc96e82b9


Broken links and unclear docs

It seems docs/content/realtime-ingestion.md was removed.
All the links to it are thus broken.
Example, scroll to the bottom here:
https://github.com/druid-io/druid/blob/8ac77a964480b66cf904400b400fe58e8b7dd1d1/docs/content/ingestion/index.md
The link to 'realtime-ingestion' will 404.
Also the page says 'Ingestion Spec' at the top and then again say 'Ingestion Spec' at the bottom while referring to different pages for realtime and batch spec.
It is thus unclear what this 'ingestion spec' actually refers to.

this relates to #2408


Migrate Conan recipes to use CMakeDeps generator

Follow this guide: https://blog.conan.io/2021/10/19/New-CMake-Transparent-Integration-Guide.html
Links:

https://github.com/conan-io/conan/issues/9108
https://docs.conan.io/en/latest/reference/conanfile/tools/cmake/cmakedeps.html
https://docs.conan.io/en/latest/reference/conanfile/tools/cmake/cmaketoolchain.html

This is accomplished with #137.


How should I do Audio Decoding from RF and Morse Code using Pytorch

❓ Questions and Help
Please note that this issue tracker is not a help form and this issue will be closed.
We have a set of listed resources available on the website. Our primary means of support is our discussion forum:

Discussion Forum

I am looking for advise on how to process audio signals from a HAM radio, especially morse code patterns.  I want to try applying an AI decoding solution.   I am new to Python, PyTorch, and especially TorchAudio and have been using the Deep Learning and PyTorch textbook for learning and meanwhile writing some code to create spectrograms and make synthetic morse code audio patterns.  A discussion of my first attempts and idea was posted on https://github.com/deep-learning-with-pytorch/dlwpt-code/issues/25 with some images of actual frequency domain signals.  My coding attempts are posted in waszee/SignalStudies_RF.
I am now trying to decide how best to make images of a moving window of code patterns.  I know I can easily convert to a tensor from the numpy array but how big and how much data to take at a time is something new to me and I am looking for experts that might give me some hints.  The discussions here indicate some of you might have ideas.  The attached image will give you a picture of the problem I hope.  Please make suggestions as to where to get help and how to feed the information to the learning engine. I am thinking I should slide along a single tone but that might be limiting.

Thanks for opening an issue on torchaudio! However, the best way to get help for these topics is to open a question on the pytorch forum linked above. Have you tried this?
This looks like a "speech recognition problem" were the "speech" are the "beeps" and "language" is the "dash-dot". This models well as a sequence-to-sequence problem. In fact, there's a tutorial here that does just that with RNN. Have you seen it? Hope this help!
In the meanwhile, I will close this issue, and encourage you to open a question on the pytorch forum linked above for the best support. But please feel free to re-open the issue if you want :)

Thank you  for the advice.  I am too new at this to have seen anything and will follow up the suggested items. I did not even know where I should post the need some help question.  I saw the title Questions/Help in torchaudio and thought this might be a place to start.
Waszee
From: Vincent QB<EMAIL_ADDRESS>Sent: Friday, October 23, 2020 8:35 PM
To: pytorch/audio<EMAIL_ADDRESS>Cc: waszee<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>Subject: Re: [pytorch/audio] How should I do Audio Decoding from RF and Morse Code using Pytorch (#983)
Thanks for opening an issue on torchaudio! However, the best way to get help for these topics is to open a question on the pytorch forum linked abovehttps://discuss.pytorch.org/. Have you tried this?
This looks like a "speech recognition problem" were the "speech" are the "beeps" and "language" is the "dash-dot". This models well as a sequence-to-sequence problem. In fact, there's a tutorial herehttps://towardsdatascience.com/cracking-morse-code-with-rnns-e5883355a6f3 that does just that with RNN. Have you seen it? Hope this help!
In the meanwhile, I will close this issue, and encourage you to open a question on the pytorch forum linked abovehttps://discuss.pytorch.org/ for the best support. But please feel free to re-open the issue if you want :)
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/pytorch/audio/issues/983#issuecomment-715643748, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ARAKYG3CDKGARICYEHQAHGDSMIOJ5ANCNFSM4S5EF6AQ.

fyi - posted a query on the discuss.pytorch topic as you suggested.  The cracking morse code article is interesting but assumes you have the dit dah pattern and that is easily handled in python using a dictionary as many others have shown.  The real problem is getting the pattern isolated from the radio audio signal or from a RF band pass depending on the type of receiver you have as well as how to work with a continuous and live signal.  Thanks for the input and the suggestions will be interesting to see how goes from here.  Did play with the torchaudio spectrogram and it worked well except I am not sure why the FFT n/2+1 size as my numpy built arrays did not.  That is given 1024 window I expected 512 frequency channels not 513 frequency channels, assuming index starting at 0 and ending at 511.


Make it possible to show Reference Gallery for custom selection of plotting backend.

One thing that would help push the backends forward is if it becomes more visible what works and what does not work.
With quarto its possible to parameterize your the documentation. I.e. the documentation could be created for each of the 3 plotting backends. The user could select the backend in a dropdown in the docs.

This functionality probably makes the most sense for the Reference Gallery.

The request is to create reference gallery for all backends

Ok thanks that's more clear!


Add GCP cherry pick bot

What type of PR is this?
/kind enhancement
What does this PR do / why we need it:
This PR adds configuration for GCP cherry pick bot to streamline the process of back-porting PRs to release branches. This bot is already used in upstream argocd for cherry picking changes.
Bot Usage:
Any PR which needs to be back-ported can be automated by commenting on the PR with
/cherry-pick target-branch-name

More details regarding bot can be found here
Which issue(s) this PR fixes:
GITOPS-3286
How to test changes / Special notes to the reviewer:
Please install the GCP cherry pick bot  on redhat-developer/gitops-operator before merging the PR.

/retest

/retest

/retest


Proposal: Unbundling Keras modules into separate repositories

We should consider moving the modules preprocessing and applications to separate repositories under keras-team: keras-team/preprocessing and keras-team/applications.
They would be listed as a dependency of keras, and would still be importable from e.g. keras.preprocessing.
Why?

They're not part of "the core Keras API" (which is about model building and training)
They're not really coupled to the rest of the codebase, so unbundling is safe
Faster CI runs, better test coverage
Unbundling makes it easier to develop in an open-source setting
Easier to sync across keras-team/keras and tf.keras (shared dependencies instead of code duplication)
Other projects might have need of Numpy preprocessing utilities

Any comments or concerns? Who would be interested in taking greater ownership of these repos, should they materialize?

And most examples in keras/examples will rely on all 3 repos

They would just depend on Keras. Preprocessing/etc should still be importable from Keras. This change would be a pure refactoring, where the code is moved to a different place but the APIs don't change.

should Data augmentation Layers (proposed https://github.com/keras-team/keras/pull/9056/files) be in the keras/preprocessing or since it's a layer, we should push it on Keras?

For now I would suggest Keras contrib. But conceptually a layer would definitely be in Keras, not keras-processing, regardless of what it does. The question is thus whether we want data augmentation layers as part of the core Keras API.

Okay, all fine then!

If anybody hasn't begun working on this, I would like to give it a go!

The two new repositories will need "owners". An owner is someone who has write rights and bears responsibility for reviewing PRs (and merging if they think it should be merged), and to a large extent, for answering issues. I will still give guidance on API choices, but that will be limited to user-facing APIs decisions.
@srjoglekar246 which repo are you interested in working on, keras-processing, or keras-applications?
@taehoonlee would you be interested in becoming owner of the keras-applications repo, since you have done a lot of work on applications so far?

@farizrahman4u @ozabluda @ahundt do you have any interest in getting involved too (see comment above)?

keras-processing sounds good then! I will write up a doc.

@fchollet, I'd love to! I wonder if you intend to separate tests for the cores and the applications (as described in "Faster CI runs, better test coverage"). My minor concern is that the application tests will not be triggered whenever the cores are updated if the applications are unbundled.

My minor concern is that the application tests will not be triggered whenever the cores are updated if the applications are unbundled.

Testing applications when changing layers would fall into "integration tests", not unit tests. Issues with the layers should be caught by the layers unit tests. We could add one application-related integration test to fix this.

OK, I understand.

@fchollet Does it make sense to start work on this with a new personal repo and then move it all to one owned by keras-team?

@fchollet

They would be listed as a dependency of keras, and would still be importable from e.g. keras.preprocessing.

For applications, this would be the other way around right? keras would be a dependency of applications.
There is some "preprocessing" code in applications as well, (imagenet_utils). Would that be moved to applications repo or preprocessing repo?
What about the examples? Separate repo?
Overall I think this is great. I use the numpy-only stuff in projects that do not actually use keras, refactoring it into a seperate repo would be really useful.

@farizrahman4u I see that you maintain one of Keras' spin-offs. I will probably look over the commits in your repo to get an idea of TODOs :-P

For applications, this would be the other way around right? keras would be a dependency of applications.

The split modules need to be importable from Keras, so they would be a dependency of Keras. We're not changing any API.
That means that keras-applications will need to dynamically load Keras at runtime. This is required anyway since we'll want the same applications module to work for tf.keras too!

There is some "preprocessing" code in applications as well, (imagenet_utils). Would that be moved to applications repo or preprocessing repo?

That's application-specific, so it would stay in the keras-applications repo.

What about the examples? Separate repo?

I think the examples are fine as they are, we don't have the same constraints there. They're already well-separated from the codebase itself (and the API), and they're not tested so they don't inflict additional load on CI. They're also not replicated in tf.keras so we don't need a new "single source of truth" mechanism for them -- they're already in one place.

It's time for...

That's application-specific, so it would stay in the keras-applications repo.
A lot of people still use this function for their preprocessing.

In terms of timeline, I'll do the split, and it will take place in late March or early April.

Every widely used project I've worked with has inevitably wanted to modularize their repositories at some point. This process can be managed fairly smoothly if you'd like to do it, but I'd like to provide some food for thought based on lessons I've taken from seeing this process unfold in several other widely used projects (boost, homebrew, and ROS are examples). Here are some questions I'd ask and answer:

What concrete benefits are there from a user focused perspective?
Are these benefits for which a significant proportion of users actually care about?
Do these benefits to users offset the cost and complexity of added install and versioning steps?
Is there is a risk of some unforeseen problems + extra user issues?
How upset will users be about having to "fix" stuff when they update?

At this time, the effects of items like "easier to sync with tf.keras" seem murky with respect to users and might benefit from better definition and/or alternative solutions. What if CI problems were solved with backend infrastructure changes that would be invisible to users? Could "owners" simply be assigned keras subdirectories? Everything might be totally worthwhile in the end, I'm just hoping to give perspective on alternative options.
The project might also consider taking the following steps, if appropriate. I believe they turned out to be enormously helpful on other projects:

Establish a clear versioning + dependency updating process in advance
Responsibilities for between repository breakage should be clearly defined

Updates to one repository will very frequently break others in tricky ways.

Avoid assuming "you should always track either master / the latest release" will be feasible for everyone.

The homebrew project split into a bunch of "modular" repositories and two years later merged back down to two repositories (the "brew" app and "recipe" installer scripts) to better manage the manifold dependency problems that emerged.

Make a clear policy on how breakage between versions will be handled (or not).
Think about how additional policy changes will be made if the process doesn't turn out as expected.
Provide clear messages and automatic detection / acquisition of the correct dependencies with a way to override it.

Some of the above can probably come from pip so perhaps it isn't as big deal in this case when compared to others I'm familiar with.
Example issues that might come up include:

Import errors mysterious to users who aren't very experienced with python.
Someone will inevitably want to use their preprocessing workflow on a past version with the current Keras version and vice versa, and then post about the API conflicts they encounter.

Hopefully this information is helpful with navigating the process for any approach you decide to take!

@ahundt thanks for the detailed thoughts. These are all very good points.

the effects of items like "easier to sync with tf.keras" seem murky with respect to users

The idea is that this is a refactoring change (refactoring to get less code duplication): it makes life easier for devs (which ultimately had indirect user benefits) while not affecting the experience at all for end users.
Specifically, we're trying to:

avoid code duplication across tf.keras and keras-team/keras (duplication ultimately could create issues for end users of either tf.keras or keras-team/keras, since the same versioning problems you mention still exist in this setup and are arguably worse)
make CI faster, which helps contributors

I agree that we need a clear plan to make sure that the user experience is not affected. I think automatic dependency management (pip) solves most issues.

Establish a clear versioning + dependency updating process in advance

We should push out new releases of preprocessing and applications every time we do a Keras release. Inversely, having the new modules do independent releases doesn't seem to be an issue, assuming that the new releases keep backwards compatibility (which they should).
Let's say current Keras is 2.1.8 and current preprocessing is 1.0.1. We want to update Keras. Then we:

release preprocessing 1.0.2 which lists keras>=2.1.8 as a dependency
(same for applications)
release keras 2.1.9 which lists preprocessing 1.0.2 as a dependency

This implies that the bleeding edge version of preprocessing and applications is always compatible both with the latest Keras release and with the bleeding edge of Keras.

Responsibilities for between repository breakage should be clearly defined

Thankfully we're in a situation where this will not happen. If this was a risk we would not do it. In our case preprocessing and applications are very well scoped and separate from core Keras. Keras doesn't depend on them at all, and they're just consumers of a stable subset of the Keras API (Sequence and layers/models respectively). Any change in Keras that would break the new modules would be in itself a bug, since Keras isn't supposed to make breaking changes to these APIs. In particular the new modules do not and should not consume any private/unstable Keras API (which is not an issue at all given how little coupling there is between the modules).
What we should do is CI-test the new modules both against the last release of Keras, and against the bleeding edge version of Keras. That way we ensure that changes in bleeding-edge modules don't break any existing users, and that they are ready to be used with bleeding-edge Keras. This guarantees an additional layer of stability.
(the reverse is not a problem since the Keras logic does not depend on the new modules, the pip dependency will only be to keep the existing namespace keras.preprocessing/etc).

Avoid assuming "you should always track either master / the latest release" will be feasible for everyone.

I think this is solved by having the new modules be compatible with both the bleeding edge and the latest release.

Think about how additional policy changes will be made if the process doesn't turn out as expected.

If we have intractable issues, we can always revert to the current state of affairs at any time, since:

What we're doing, and the reverse op, would be 100% API-compatible with the past
Doing so is just a matter of copy/pasting code, so can be done quickly and without issues

Provide clear messages and automatic detection / acquisition of the correct dependencies with a way to override it.

pip + clear, actionable error messages in case of ImportError should suffice here.

This sounds like a pretty reasonable plan to me. I'll just mention a couple additional potential pain points to consider based on your comments:

In particular the new modules do not and should not consume any private/unstable Keras API (which is not an issue at all given how little coupling there is between the modules).

This would definitely mitigate many potential issues. I think to make this happen the following steps might be worthwhile:

Perhaps some currently private APIs might be worth making public in this case to avoid code duplication
Have travis check for and prevent external calls with leading underscores.
Have something in the readme on how to request an internal API be made external, this may simply be one sentence that goes in with the existing API change process.

Responsibilities for between repository breakage should be clearly defined

Thankfully we're in a situation where this will not happen. If this was a risk we would not do it.

I'll mention a couple specific examples of conflicts that showed up later in travis. keras-contrib actually broke quite a few times over the first few months due to internal keras changes for which internal calls were unavoidable. Examples include:

imagenet_utils was one major source when the _obtain_input_shape() API changed
_preprocess_conv2d_input() was removed
travis changes to CI itself
keras changes to .travis.yml
dylib breakage during travis setup

Everything can be resolved if there are cycles available to fix it, I just wanted to give a couple of specific examples of pain point categories that will multiply as the number of repositories increases.

I think this is solved by having the new modules be compatible with both the bleeding edge and the latest release.

Someone, or many individuals might deploy code in production and not be able to update for one valid reason or another. One option is to have periodic LTS releases are the typical way to meet the needs of groups like that, but it isn't the only way to go.
I hope that helps, overall your proposed plan sounds reasonable! Thanks for taking my ideas into consideration.

I've been delaying doing this because I wanted to think about it for a while and make sure this was the right thing to do. Thanks everyone for the various comments and feedback! Now, let's make a decision.
@Dref360 @taehoonlee: should we do this split of preprocessing and applications into separate repositories, and would you take ownership of the split repos? (@Dref360 for preprocessing and @taehoonlee for applications?
We'll make a decision asap based on your answers, and do the split next week if the decision is to do it.

We should definitely do the split for keras.applications, which would be the "official" keras-zoo.
For keras.preprocessing, I think your motivations are valid. And the community will be much more easier to manage with it. I'm kinda worried about the

That means that keras-applications will need to dynamically load Keras at runtime.

We should be very cautious with that.
A user that tries to do import keras.preprocessing and gets Cannot find keras with a traceback that goes through keras-preprocessing repositery will be confused.
Also, we would need CI for multiple keras versions and deprecate the versions overtime.
So overall I would like the repo to move foward with the split and we should follow every @ahundt advices. I think all of them are valid concerns.
On the ownership,  I can take care of keras.preprocessing, I would need some help with keras.preprocessing.sequence since it's not my main area of research. I am open to get some help from other contributors as needed. :)

Cool, thanks @Dref360!
@taehoonlee, any thoughts?

Also, we have tons of PR waiting for those modules. When the split is done, we would have to go through it.

@fchollet, It would be nice if we could do the split. Especially, I strongly agree with:

the justification, "They're not part of the core Keras API" [1],
the advantage for contributors, "Faster CI runs" [1],
the advantage for @fchollet, "Easier to sync across keras-team/keras and tf.keras" [1].

There are no benefits as @ahundt pointed out. However, in my opinion, this is not a problem because the intention of the split is to facilitate management without harming the user experience.
Of course, the potential issues may actually occur. I think that the issues are not the reasons why we should not split, but the matters we need to care about in the future.

And, I'd like to take ownership of the keras-team/keras-applications.

keras-contrib currently has a specific, clear example travis link of the sort of breakage I mentioned might be a concern in unbundled modules. I believe master moved some files around and removed some legacy code, and now the keras-contrib build doesn't work.
https://github.com/keras-team/keras-contrib/pull/263
I haven't figured out a fix yet, but haven't spent much time on it either.

@ahundt, The first one arose due to a dependency to legacy codes and seems to be already resolved by you. The preprocessing and applications exploit public and stable APIs and do not depend to legacy codes.
The second one is orthogonal to the split. It even fails to import keras (AttributeError: module 'pandas' has no attribute 'core'). Why don't you try pip install tensorflow==1.7 or python: 3.6 or export LD_LIBRARY_PATH based on the master CI config?

@taehoonlee yes, I know, thanks. The purpose of my post was to provide a clear example of admittedly very solvable problems that will need to be resolved around n times for n repositories.

We'll do it then.
There's a technical problem we'll need to solve:

We need to be able to set what Keras module to use when importing preprocessing (or applications) from inside keras (or from inside tf.keras), so that things like keras.backend.set_image_data_format() get reflected in keras.preprocessing.
This is doable via a method like preprocessing.set_keras_module(sys[__name__])
This implies that we can, in this order: 1) import preprocessing without any reference to Keras 2) set the Keras module right after the import
However Iterator(Sequence) leverages a Keras symbol in the class definition (Sequence), which we would need at import time (so before we would have the Keras module available).

A possible solution is to ship Sequence with preprocessing. @Dref360, what do you think?

For the record, I'm thinking of something like:
Contents of keras.preprocessing.image:
try:
   import keras_preprocessing  # Note: what should we name these modules?
   _check_version(keras_preprocessing)
   keras_preprocessing.set_keras_module(sys[__name__])
except ImportError:
   raise ImportError('...')

Iterator = keras_preprocessing.image.Iterator
# etc...

Nevermind, we can structure the code in a way such that we can import the submodule and call set_keras_module before we enter the files where we use Sequence. However, as a result users won't be able to import standalone symbols from image, e.g. from keras_preprocessing.image import Iterator, rather you'd have to do from keras_preprocessing import image; image.Iterator or keras_preprocessing.image.Iterator.
I don't think it matters, because presumably the module would get used from Keras almost every time (from keras.preprocessing.image import Iterator would still work), and also because from keras_preprocessing import image is the best practice in terms of Python style.

Side note: after consideration I think it would have been a bad idea to package Sequence into preprocessing, because:

We have the preprocessing.sequence submodule already, big semantic collision that would cause confusion
Sequence is used in training.py so it is more tightly coupled with the rest of the codebase, and it could be dangerous to unbundle it

I've implemented this initial plan, but I think we're going to have to go with a different design to make it possible to use tensorflow (which will set the keras_preprocessing keras module to tf.keras) and keras in the same session without subtle problems.

I've tried a couple different designs, but things are turning out to be incredibly complicated in practice when you start converting Keras to use keras_preprocessing which would refer to keras (in a way that would work coherently with tf.keras...).
I'm currently leaning towards a more verbose / advanced design, that would be hopefully safe:

in keras.preprocessing.__init__, import keras submodules utils, backend, then import keras_preprocessing
call keras_preprocessing.set_submodules(utils=utils, backend=backend) (defined in keras_preprocessing.__init__
in the keras_preprocessing.__init__ namespace, set keras_utils = utils, keras_backend = backend
in keras.preprocessing.image, do from . import keras_backend and use that

Implemented that last design (more or less) and I believe it's safe.
One side-effect is that keras_preprocessing isn't usable in a standalone way (otherwise we'd have a circular import once Keras relies on it). There are 2 ways to use it:
import keras
from keras_preprocessing import image  # for instance

Or (preferred):
from keras import preprocessing  # this is literally keras_preprocessing

The first stage of the split is executed (creating independent modules for preprocessing and applications). The next stage is to get Keras and tf.keras to import them.
@taehoonlee in the process I've run into a NASNet bug -- error wrt incorrect number of layers when loading ImageNet weights. I've verified that the weights files are up to date.

@fchollet, The error has been resolved in keras-team/keras-applications#1.

Thanks for the PR, merged!
@Dref360 @taehoonlee as you have noticed the repos are created, and you have write rights on them! Here are a few TODOs I haven't done yet (true for both repos), that you can work on if you like:

Run CI against both the latest Keras release and Keras at HEAD. Currently we only run against run the release. This should just be a small change to the travis.yml file.
Remove the unit tests for these modules from Keras tests, and instead add a couple high-level integration tests that should run quickly (e.g. for applications we could try loading a single model).

And please check out and review any PRs that our contributors send out 👍

Maybe it is time to release 2.1.7, @fchollet?

@taehoonlee yes, I will release this week. It will be 2.2.0 due to a large amounts of changes in this release.

@fchollet Before release please consider checking on https://github.com/keras-team/keras/pull/7766


Add observer mapping

This feature allows you to have cleaner observers mapping in EventServiceProvider which basically copies syntax of policies, event listeners and other similar stuff. Observers are automatically registered during booting hook. In general, this allows to write this:
/**
 * Register any events for your application.
 *
 * @return void
 */
public function boot()
{
    User::observe([UserObserver::class, UserOrderObserver::class]);
    Blog::observe(BlogObserver::class);
    Transaction::observe(TransactionObserver::class);
    Order::observe(OrderObserver::class);
    Payout::observe(PayoutObserver::class);
}

like this:
/**
 * The observers mapping for the application models.
 *
 * @var array
 */
protected $observers = [
    User::class             => [ UserObserver::class, UserOrderObserver::class ],
    Blog::class             => BlogObserver::class,
    Transaction::class      => TransactionObserver::class,
    Order::class            => OrderObserver::class,
    Payout::class           => PayoutObserver::class,
];

Not sure I want to add this on a patch release as someone may already be defining this property and doing it manually, etc.


Minor improvements in the hosting TTS on AWS ECS documentation

Summary
We have identified minor improvements in a few sections in hosting the TTS on AWS ECS documentation.
Why do we need this ?
To assist users in hosting The Things Stack on AWS ECS with ease.
What is already there? What do you see now?
Hosting TTS on AWS ECS documentation.
What is missing? What do you want to see?
Documentation improvements specific to each section:

Who is it for?

Update with more information on the minimum number of devices, gateways, and deployment size of the users in numbers to consider for hosting the TTS on AWS ECS.
Add optional concepts related to Prometheus, Grafana, and OpsGenie as nice to have context.
Ref: Who is it for?

Containers:

Update about the UDP container requirements in the deployment according to the latest changes. In TTS v3.26.0 of The Things Stack Cloud, the Gateway Server with a Network Load Balancer is using for the UDP gateways by removing the UDP gateway server service.
Ref: Containers

Prerequisites:

Update the NOC OAuth Client ID and Secret requirements similar to the Console and Device Claiming Server in the prerequisites section.
Context on the certificate requirement of the below two points.

Configuration and Certificates for Backend Interfaces interop (optional).
Configuration and Certificates for Packet Broker peering (optional).

Add information on the alerting and monitoring requirements.
Ref: Prerequisites

Deployment:

Update the below information in the ECS cluster section with the latest UI changes in the AWS ECS console.Under Amazon ECS ARN and resource ID settings the new ARN and resource ID format needs to be enabled for container instances, services and tasks. If you plan to deploy The Things Stack on container instances, AWSVPC Trunking needs to be enabled.

Database Operations:

Add Identity Server initialization section with the command tti-lw-stack, is-db, migrate
Add DCS OAuth client creation command.
Ref: Database Operations

Monitoring:

Add example instructions for configuring the grafana dashboards with the Prometheus deployed by The Things Stack.
Ref:  Monitoring

Updating:

Add NOC database migration command in the migrating section.

Best Practices:

Add insights on the infrastructure optimization for the user deployments based on their deployment size, i.e., number of devices and gateways.
Ref: Best Practices

How do you propose to document this?

...
Can you do this yourself and submit a Pull Request?
No, @KrishnaIyer

@Siva-7482: As discussed earlier, if you have all the required information, can you please open a pull request yourself? cc: @ymgupta

@KrishnaIyer,
We do not have the required information to file the pull request here. And it requires insights from your end. Could you please provide us with your insights for the topics mentioned above? Accordingly, we will open a PR with the changes.

Ah I see. Ok

Let's pick this up during the conference month.


[UF-299] disable git/ssh daemons by default when running tests

disabling the git/ssh daemons by default makes the tests more stable.
Maven build is usually executed in parallel (e.g. -TC), which means
tests in multiple modules are executed in parallel (in different JVM of course).
That sometimes leads into random test failures caused by failure
to bind to the default git/ssh ports.

Tested with both uberfire and uberfire-extensions repos (and with kie-wb running to make sure the ports are already bound).

+1

+1

@psiroky thanks!


Dashboard does not show all tabs and data

I googled and could not find any solution to my problem so please help me out.
I am using strapi v4 inside the index.js bootstrap function I defined my sockets. They are working with no problems.
I wanted to launch admin.socket.io  to start implementing some more logic, but when connected to the admin dashboard I don't see much. only two tabs.
index.js file
"use strict";
module.exports = {
  /**
   * An asynchronous register function that runs before
   * your application is initialized.
   *
   * This gives you an opportunity to extend code.
   */
  register(/*{ strapi }*/) { },

  /**
   * An asynchronous bootstrap function that runs before
   * your application gets started.
   *
   * This gives you an opportunity to set up your data model,
   * run jobs, or perform some special logic.
   */
  bootstrap({ strapi }) {
    const io = require("socket.io")(strapi.server.httpServer, {
      cors: {
        origin: ["https://admin.socket.io"],
        credentials: true
      },
    });

    const { instrument } = require('@socket.io/admin-ui');

    io.on("connection", function (socket) {
      console.log(`Connected ${socket.id}`);
    });

    strapi.io = io;
    instrument(io, { auth: false });
  },
};

Hi! Which version of @socket.io/admin-ui are you using?
It seems the mode is set to production, which is why only the overview information is displayed.

Hi @darrachequesne, Can we show some more information when the mode is set to production. The Admin UI is not very useful for me when the mode is production. I would prefer to see the sockets, rooms & clients tabs. Don't need the events or the other options to modify (leave, join, disconnect) but would like to see the sockets and rooms and additional metadata set on the sockets. Just seeing the dashboard and server tabs doesn't serve much purpose for me.  You can also perhaps add some new attributes to show these tabs in production mode. Like showSockets = true, showRooms=true etc.

@afrancis-caregility doesn't the development mode suit your use case? Tracking all sockets is quite CPU/memory hungry, hence the production/development modes.

Yes, it does but I am afraid to put development mode in Production. I will do some load testing and see how the platform behaves in development mode. This admin UI has been invaluable in helping me figure out what is going on with socket.io server & clients and so I would also prefer to have this capability in Prod to debug issues. I am more interested in seeing the data the socket holds and which rooms they are part of. So, that's why I wanted to see those two tabs. Is it possible to get data at a point in time by clicking a "load client data" rather than real time status updates if that makes it easier in production mode. This way we can load whatever sockets and rooms are available when user connects to Admin UI and then we can do a manual refresh to fetch the latest data again. We definitely don't want any monitoring tool taking up lot of memory/CPU in Prod.

https://socket.io/docs/v4/admin-ui/
This page:  "Default value: development"

@afrancis-caregility doesn't the development mode suit your use case? Tracking all sockets is quite CPU/memory hungry, hence the production/development modes.

https://socket.io/docs/v4/admin-ui/ This page: "Default value: development"

@afrancis-caregility doesn't the development mode suit your use case? Tracking all sockets is quite CPU/memory hungry, hence the production/development modes.

Yes but the NODE_ENV overrides this setting and in Heroku (for instance) it defaults to "production"

@IlyaDiallo in that case, you need to explicitely specify mode: development, which should override the NODE_ENV variable.
See also: https://github.com/socketio/socket.io-admin-ui/blob/158864989dddeba67df9975a4cad48ef310f8c80/lib/index.ts#L544-L554

Can we show some more information when the mode is set to production. The Admin UI is not very useful for me when the mode is production. I would prefer to see the sockets, rooms & clients tabs.

Tracking all sockets/rooms requires a lot of memory, this is why it's only enabled in development mode.
I'm closing this, as I don't think there is much we can do. Please reopen if needed.


[Languages] wenyan-lang

This is written in wenyan-lang(文言文編程語言) and it will output the exact 前前前世.
Check out https://github.com/wenyan-lang/wenyan

I'll merge. Thank you!


DBeaver client not opening due to "Jaava heap space" error. Heap has already been manually upped to 4GB.

System information:

Windows 10 Home 64-bit (10.0, Build 19041)
DBeaver 21.0.5 x86

Connection specification:

This is an issue with the dbeaver client, not a specific database.

Describe the problem you're observing:
Running "dbeaver.exe" causes the launcher to appear (see image below), hang on "Starting DBeaver," then disappear. This creates a dbeaver background process, but does NOT open the client.

According to the attached log file (see bottom of post), this is due to a java.lang.OutOfMemory error "UTF16 String size is<PHONE_NUMBER>, should be less than<PHONE_NUMBER>."
Running "dbeaver-cli.exe" instead causes this:
Console.txt
The notable line is "Internal error in workbench lifecycle java.lang.OutOfMemoryError: Java heap space"
What's odd is that I've already modified the "dbeaver.ini" file to read "-Xmx4096m" and the Java runtime parameters to contain the same.
Steps to reproduce, if exist:
Run dbeaver.exe or dbeaver-cli.exe; that's it.
Include any warning/errors/backtraces from the logs
From running dbeaver.exe:
java.lang.OutOfMemoryError: UTF16 String size is<PHONE_NUMBER>, should be less than<PHONE_NUMBER>
From running dbeaver-cli.exe:
Internal error in workbench lifecycle java.lang.OutOfMemoryError: Java heap space
dbeaver-debug.log

Please try to reset workspace. It should help.

there is no update on the issue for a long time.
If it is still actual for you - feel free to reopen it with additional info

Hi
I am having the same issues
OS: Windows 10 Home
DBeaver version: 22.2.0
In the dbeaver.ini , I have the following setting:
-vmargs
-XX:+IgnoreUnrecognizedVMOptions
--add-modules=ALL-SYSTEM
-Dosgi.requiredJavaVersion=11
-Xms64m
-Xmx4096m
-Djavax.net.ssl.trustStoreType=WINDOWS-ROOT
-Ddbeaver.distribution.type=exe
I only changed the old value to -Xmx4096m.
I had Java heap space Error before I change the old value
However, after I changed the value to -Xmx4096m
I got a new error:
UTF16 String size is<PHONE_NUMBER>, should be less than<PHONE_NUMBER>
I tried to reset workspace and use dbeaver.exe or dbeaver-cli.exe to open DBeaver
Both way give me same error
What should I do?
The SQL file that I am trying to run is 16.4 GB
Is it the cause of the error?


Log successful uploads and skip on re-run of script. Fixes Loungebuddy/LBApp#171

Add a log file for successful uploads and logic to skip over files in the log file when the script is re-run
For Loungebuddy/LBApp#171

I am slightly concerned at the memory usage involved with loading so much json into memory, but I think for out purposes it should be fine. I suppose we could have it take in a directly of files and traverse them in series, but I don't really think that's worth it.

Accidental PR to this repo, please close, we will be finishing our internal code review / testing before send it back here.  Thanks!


Restart a vpc tier with cleanup should not cleanup the whole VPC

ISSUE TYPE

Bug Report

COMPONENT NAME

VR/network

CLOUDSTACK VERSION

<IP_ADDRESS>-rc1

CONFIGURATION

advanced networking
OS / ENVIRONMENT

kvm
SUMMARY

ideally restart a vpc network will remove the guest nic and replug it in VPC VR.
but actually VPC VR is destroyed and a new VPC VR is created.
STEPS TO REPRODUCE

1. create vpc
2. create vpc tier
3. create vm in vpc tier
4. restart vpc tier with cleanup

EXPECTED RESULTS

guest nic is removed from vpc vr and replugged.

ACTUAL RESULTS

VPC VR is destroyed and a new VPC VR is created.

Fixed in #5873


extracting FORCE_CONSTANTS from the FORCE_SETS.

Dear atztogo,
I first calculated the FORCE_SETS using the finite difference method, and then I wrote the FORCE_CONSTANTS from the FORCE_SETS. However, I found that the matrix size of FORCE_CONSTANTS calculated by the DFPT method is different from the format converted from the FORCE_SETS, although the two are equivalent in lattice dynamics. My primitive cell contains 5 atoms, and the calculated FORCE_CONSTANTS for supercell (3x3x1)  are as follows:

calculated from DFPT:
45   45
1 1
18.702488206629997    -0.000000000000000     0.000000000000000
-0.000000000000000    18.702489647949999    -0.000000000000000
0.000000000000000    -0.000000000000000    13.630046166449999
1 2
..........

wrote from FORCE_SETS:
5   45
1 1
18.702488206629997    -0.000000000000000     0.000000000000000
-0.000000000000000    18.702489647949999    -0.000000000000000
0.000000000000000    -0.000000000000000    13.6300461664499991
1 2
............

I want to calculate the FORCE_CONSTANTS for ShengBTE to get thermal conductivity, but the ShengBTE can only recognize the format calculated by the DFPT method. So, when  extracting FORCE_CONSTANTS form FORCE_SETS, what should I do to get the same matrix size as DFTP? Thanks very much.
Best regards,
Junwei

Please send your question to the phonopy mailing list. I close this issue, but please reopen whenever needed.


Applying config changes on upgrade

I'm spinning up a cluster:
helm upgrade --install redpanda redpanda/redpanda -n redpanda --create-namespace --wait --debug --set tls.enabled=false -f tmp.yaml

Where tmp.yaml is:
config:
  cluster:
    delete_retention_ms: 50000
  tunable:
    kafka_batch_max_bytes: 1000000

This works (the two above values are set as expected). Then I change these values to something else and run the above upgrade command again, but the values don't get updated.

Can you get the logs for the post-upgrade job? kubectl -n redpanda logs job/redpanda-post-upgrade
Doing the same command (with dry-run enabled) I see the rpk cluster config set delete_retention_ms being set.
I assume your test to see if they're not set is via rpk cluster config get?z

In my tests, the config changes are applied. Closing as cannot-duplicate.


[do not review] Gather tests that have inter-test dependencies

The need for a LoadScopeScheduling instance to handle test scheduling is making proper reporting + retries difficult, so check if we can just fix the tests so they don't actually need a special scheduler.

@tvm-bot rerun

@tvm-bot rerun

@tvm-bot rerun

@tvm-bot rerun


.env file corrupted

It is corrupted due to the fact it is giving me issues. FIX THIS DAMMIT

There should be a fix on this. I need to help and fix this.


07-callback viewport size fix

It doesn't compile because needs uint16_t cast, so discard this if you want. And you can fix it without a PR.


email give an alarm do not work well

In GitLab by @guyingyan on Jun 26, 2018, 18:25
If you are reporting a problem, please make sure the following information are provided:

Version of board (last git commit number) , docker engine and docker-compose.
Config files of board, you can get them by packaging "board.cfg" and files in the same directory, including subdirectory.
Log files, you can get them by package the /var/log/board/.
Other necessery information.

without port can go to next step when create server

In GitLab by @wknet123 on Nov 8, 2019, 07:53
Incorrect form in data structure.

In GitLab by @wknet123 on Nov 8, 2019, 07:53
Fixed by #1570

In GitLab by @guyingyan on Nov 8, 2019, 07:53
verified on 1.0-780-gb2f8e94

In GitLab by @sokril on Nov 8, 2019, 08:24
closed

In GitLab by @sokril on Nov 8, 2019, 08:24
assigned to @guyingyan


Windows Error:pull model manifest return wsarecv: An existing connection was forcibly closed by the remote host.

i can't down the model:
app.log
server.log
PS C:\Users\heimu\AppData\Local\Ollama> ollama pull llama2 pulling manifest Error: pull model manifest: Get "https://ollama.com/token?nonce=A-QmGZFS0za-Kv0GKrDy3Q&scope=repository%!A(MISSING)library%!F(MISSING)llama2%!A(MISSING)pull&service=ollama.com&ts=1710339130": read tcp <IP_ADDRESS>:56798-><IP_ADDRESS>:443: wsarecv: An existing connection was forcibly closed by the remote host. PS C:\Users\heimu\AppData\Local\Ollama>

Hi @heimu-liu this seems similar to #1149, are you using ollama behind a proxy? If so checkout the proxy configuration guide:
https://github.com/ollama/ollama/blob/main/docs/faq.md#how-do-i-use-ollama-behind-a-proxy
Hopefully that helps, let me know if that is the case.

Contributor

Thank you very much for your reply!
I've tested it with both wireless and LTE, as well as proxy and non-proxy.
still get the same error.

Similar to #3191 and #2448

How can I fix this?

Looking at the request both from the error in your description and the server access logs, it appears the request is getting mangled somehow.
This is being sent
https://ollama.com/token?nonce=A-QmGZFS0za-Kv0GKrDy3Q&scope=repository%!A(MISSING)library%!F(MISSING)llama2%!A(MISSING)pull&service=ollama.com&ts=1710339130

instead of this
https://ollama.com/token?nonce=A-QmGZFS0za-Kv0GKrDy3Q&scope=repository%3Alibrary%2Fllama2%3Apull&service=ollama.com&ts=1710339130

This should be caused by passing the real URL into fmt.Printf or similar functions: example.
Is it possible there's a man-in-the-middle proxy mangling the request?

My experiments in #2448 suggest that a correct URL is called and it is only mangled later, when logged.

I am facing the same problem when connected to a corporate network (Ubuntu 22.04, Ollama installed with Homebrew):
$ ollama run llava

pulling manifest
Error: pull model manifest: Get "https://ollama.com/token?nonce=SW96RgmctcQmHJ37NXJ8KQ&scope=repository%!A(MISSING)library%!F(MISSING)llava%!A(MISSING)pull&service=ollama.com&ts=1711106784": read tcp <IP_ADDRESS>:40600-><IP_ADDRESS>:443: read: connection reset by peer

The problem first appeared in v0.1.28.
When I downgrade to v0.1.27, I am able to pull the models again.

My experiments in #2448 suggest that a correct URL is called and it is only mangled later, when logged.
I am facing the same problem when connected to a corporate network (Ubuntu 22.04, Ollama installed with Homebrew):
$ ollama run llava

pulling manifest
Error: pull model manifest: Get "https://ollama.com/token?nonce=SW96RgmctcQmHJ37NXJ8KQ&scope=repository%!A(MISSING)library%!F(MISSING)llava%!A(MISSING)pull&service=ollama.com&ts=1711106784": read tcp <IP_ADDRESS>:40600-><IP_ADDRESS>:443: read: connection reset by peer

The problem first appeared in v0.1.28.
When I downgrade to v0.1.27, I am able to pull the models again.

Thank you Very Much!

Problem solved by downgrade to v0.1.27

@heimu-liu Downgrading is merely a workaround, but hopefully the problem can actually be solved in future version of Ollama.
@mxyng Do you prefer to reopen this issue or should I create a new one with all the information known so far?

I opened #3452.

I'm facing the same issue with the latest version of ollama. Does not work in corporate proxy anymore.
Downgrading to v0.1.27 makes it work again.


New package: Sole v0.1.0

Registering package: Sole
Repository: https://github.com/aclai-lab/Sole.jl
Created by: @giopaglia
Version: v0.1.0
Commit: 26ae52427f5a6d0465e81887786baf89724957fe
Reviewed by: @giopaglia
Reference: https://github.com/aclai-lab/Sole.jl/issues/3#issuecomment-1640288143
Description: Sole.jl – Long live transparent modeling!

This seems find to me. Can you ping again after the three day waiting period (Friday)?
[noblock]

This seems find to me. Can you ping again after the three day waiting period (Friday)?

Yes. Thank you!
[noblock]

Hi again, this is my ping. Thank you again!


Set-FieldDataClassificationFromExcelFile needs excel

Don't we all love GDPR?
Trying to run Set-FieldDataClassificationFromExcelFile fails. First the container was missing the module from the DVD. After copying the psm1 to the container and running Import-Module WindowsPowerShellScripts\DataClassification\DataClassification.psm1 in the container we clould run the command. Fails though, looks like some excel component is missing in the container. That is no surprise, but what can we do about it?

Those cmdlets are clearly not designed to run inside containers.
You should be able to run the cmdlets on your pc using the database in the container or having the database on a seperate database server.
Issue cannot be solved in Docker.


 I think this issue should instead be reported against https://github.com/docker-library/official-images or https://github.com/docker-library/golang which seem to be the source of these images.

What @tklauser said - this seems to be the wrong issue tracker to report the issue to.


nomnom  pulls in underscore 1.6 which has security issue of high severity

dear all,
CodeMirror uses nomnom , that depends on underscore version 1.6, but that version has a security issue of Severity: high
Arbitrary Code Execution in underscore  see  https://github.com/advisories/GHSA-cf4h-3jhx-xvhq
Is there
TIA
a.

These are dev dependencies, and as such shouldn't concern you. But a PR that upgraded/replaces them would be welcome.

I have patched nomnom as follows
diff --git a/package.json b/package.json
index 6556f58..721ca0f 100644
--- a/package.json
+++ b/package.json
@@ -19,7 +19,7 @@
     "parser"
   ],
   "dependencies": {
-    "underscore": "~1.6.0",
+    "underscore": "~1.12.1",
     "chalk": "~0.4.0"
   },
   "devDependencies": {

and the patched CodeMirror as follows
diff --git a/package.json b/package.json
index 1794a38..18c29f1 100644
--- a/package.json
+++ b/package.json
@@ -43,5 +43,8 @@
     "dependencies": {},
     "devDependencies": {}
   },
-  "dependencies": {}
+  "dependencies": {
+    "nomnom": "^1.8.1",
+    "underscore": "^1.13.2"
+  }
 }

and I have reinstalled both, and it seems to be working fine
a.


EF team feedback on various APIs

The following improvements were identified while porting the EF Core provider to the Cosmos SDK version <IP_ADDRESS>-preview.
cc @AndriySvyryd @ajcvickers @kirillg

[ ] Add a constructor to PartitionKeyDefinition that accepts an IEnumerable of paths
[ ] Remove the parameterless constructor from CosmosContainerSettings since the id and partition keys are required
[ ] Add a way to get the query and parameters from CosmosSqlQueryDefinition, this would be useful for logging
[ ] There doesn’t seem to be an advantage to have CosmosResponse.Resource  be strongly typed, as the generic argument cannot always be inferred and frequently needs to be specified explicitly. Returning it as object makes using the methods like DeleteItemAsync<>() simpler as they won’t need the generic argument.
[ ] It might be better to rename the CreateItemQuery<>() overloads that don’t take a partition key as they could be invoked unintentionally if the partition key is an int. Or move maxConcurrency to requestOptions
[ ] Improve documentation for the partitionKey parameter in methods like CreateItemAsync<>(). PartitionKey doesn’t have any documentation. And a PartitionKey instance doesn’t seem to be accepted, consider changing the type to string or separating into overloads
[ ] Allow to provide a custom IDocumentClientRetryPolicy or provide a built in way of using exponential backoff.
[ ] (Not really API feedback, but a possible way to improve the codebase) Align directory structure of the code with the namespaces and apply other established C# coding conventions - https://docs.microsoft.com/dotnet/csharp/programming-guide/inside-a-program/coding-conventions

@AndriySvyryd @divega  Re 7:  You can apply your custom retry policy as a Handler, in our samples, there is one that covers using Polly: https://github.com/Azure/azure-cosmos-dotnet-v3/blob/master/Microsoft.Azure.Cosmos.Samples/CodeSamples/Handlers/ThrottlingHandler.cs

@ealsur Thanks, would that replace the built-in retry policy or would it run on top of it?

It would run on top of it. If the custom retry is covering a scenario the built-in retry handler already covers, then the default one will be applied first. V3 uses Retry handlers such as https://github.com/Azure/azure-cosmos-dotnet-v3/blob/master/Microsoft.Azure.Cosmos/src/Handler/RetryHandler.cs to apply the default retry policy. If no default retry handler exists to cover that case, then your custom handler would be the first responder.

@ealsur I see, then we would still need a way of running the custom handler before the default one or removing it.

Or expose the default one retry options, that would work too, just in the V2 SDK.

Please create specific issues for gaps if-any.


feature: EIP support

customize the snat of egress source ip

The development part has been completed, and we will release a preview version in v0.2.0-rc2.


Easel JS not working on codepen

TODO

[ ]BUG
[ ]NO i havent
[ ] version  1.0.0

Issue Details

Version used (Ex; 1.0.0):
easel js not working on codepen = https://codepen.io/coderteam1/pen/PoWbxrq?editors=1011

OS & Browser version = Windows 10, Chrome latest 2021

Do you know of any workarounds? = no

https://codepen.io/coderteam1/pen/PoWbxrq?editors=1011
Can you tell me if a error is there i copy your code it shows i write by myself it doesnt

Started working there was code error. Sorry for troubling


Cell::update

I was wondering why we don't have a method like the following for Cells:
impl<T: Copy> Cell<T> {
    fn update<F: FnMut(T) -> T>(&self, f: F) {
        self.set(f(self.get()));
    }
}

This makes some particularly annoying use cases like updating counters easier (see before and after for comparison):
data.dep_kind.set(cmp::max(data.dep_kind.get(), dep_kind));
data.dep_kind.update(|k| cmp::max(k, dep_kind));

self.has_errors.set(self.has_errors.get() | old_has_errors);
self.has_errors.update(|e| e | old_has_errors);

self.count_imm.set(self.count_imm.get() + 1);
self.count_imm.update(|c| c + 1);

Why require T: Copy instead of T: Default? Because otherwise you might accidentally run into bugs like:
foo.update(|f| {
    let x = bar.set(foo.get()); // Ooops, `foo.get()` returns `0` instead of `f`!
    f + x
});

And why not T: Clone instead? Because it might introduce hidden costs.
For example, to append "..." to a Cell<String>, you might do:
let mut s = foo.take();
s.push_str("...");
foo.set(s);

But if update cloned values, that'd introduce a hidden allocation:
let mut s = foo.update(|mut s| {
    s.push_str("...");
    s
});

Here's a nicer version of the same thing. It still has an allocation, but with T: Default there wouldn't be one, so one might (incorrectly) expect this to be efficient:
foo.update(|mut s| s + "foo");

All in all, T: Default and T: Clone have subtle pitfalls, but Cell::update with T: Copy seems like a safe choice and would be a nice addition to the standard library. After all, T: Copy is by far the most common use of Cell anyway, which can be confirmed by a quick ripgrep through the rustc codebase.

And why not  T: Clone instead? Because it might introduce hidden costs.

There's a more fundamental reason not to do it: cloning Cell contents is unsound.

T: Copy is by far the most common use of Cell anyway

That’s because it was the only use allowed until recently.
But yeah, Cell::update sounds nice to have, and requiring T: Copy allows the least surprising behavior as you explained.

I think this is small and straightforward enough to open a PR without going through the whole RFC process, with a FCP for libs team review in the PR.

I'd also like to ask what you think about two more ideas:

Extending Cell::get by changing T: Copy to T: Clone + Default.
A different Cell::update that allows not only T: Copy but any T: Default.

These additions would allow the following code to work:
let count = Cell::new(5);
count.update(|c| *c += 1);
println!("new count: {}", count.get2());

let v = Cell::new(Vec::new());
v.update(|v| v.push(5));
v.update(|v| v.push(6));
println!("a clone of the vector: {:?}", v.get2());

Playground link with a working example.

Extending Cell::get by changing T: Copy to T: Clone + Default.

That’s a breaking change, not an extension. (&T for example implements Copy but not Default.) Cell::get is stable.

A different Cell::update that allows not only T: Copy but any T: Default.

Neither Copy nor Default imply the other, so you have to pick one. Or have two almost-but-not-quite identical methods with different names, which doesn’t seem like good design.

That’s a breaking change, not an extension.

Would there be a way to somehow implement Cell::get for both T: Copy and T: Clone + Default, perhaps using specialization? I tried, but the two implementations keep conflicting. Perhaps specialization is not powerful enough yet?

Or have two almost-but-not-quite identical methods with different names, which doesn’t seem like good design.

Yeah... It's a pity that, in order to push something into a Cell<Vec<T>>, you have to do:
let mut v = cell.take();
v.push(foo);
cell.set(v);

Seems like there's still a lot of room to improve Cell, but hard to come up with the right solution.

Regarding get, I don’t think any implicit .clone() call is desirable (since those can be expensive).
I agree that the ergonomics of Cell<T> for T: !Copy are not great at the moment. Maybe two methods are warranted, with names that reflect their different behaviors:
impl<T> Cell<T> {
    /// This uses `.get()` then `.set()`
    fn get_set<F>(&self, f: F) where T: Copy, F: FnMut(T) -> T {
        self.set(f(self.get()));
    }

    /// This uses `.take()` to temporarily move out the value
    /// (leaving `Default::default()` behind), mutate it, then move it back.
    /// The default can be observed if the cell is accessed during the callback.
    fn temporary_take<F, R>(&self, f: F) -> R where T: Default, F: FnMut(&mut T) -> R {
        let mut value = self.take();
        let result = f(&mut value);
        self.set(value);
        result
    }
}

I don’t really like these particular names, but you get the idea.
Taking something and then giving it back… It’s tempting to call that "borrowing", but that term obviously already has a strong meaning in Rust that doesn’t apply here.
Usage examples:
cell.temporary_take(|vec| vec.push(foo));

let vec = cell.temporary_take(|vec| vec.clone());

update and mutate could work as short name to describe these respective methods in isolation, but if both exist I feel they’re not self-explanatory enough in how they differ from each other. Remembering which one is which becomes a memory exercise.

Adding to the bikeshed: modify.

The addition of temporary_take dilutes the original intention. I would identify two objectives here, which are orthogonal in my point of view. On the one hand we have distinct intents:

encapsulating a (possibly copyfree) change of internal state
encapsulating a (possibly copyfree) derivation of output information

While an example for the former was already given by the original author, a motivation for the latter could be retrieving the length of a stored vector. A combination of these is also possible, resulting in the proposed interface of temporary_take.
On the other hand, we have to consider zero-cost philosopy and implementation:

enable non-copy types
offer convenience for T: Default

Essentially, we have three possibilities of retrieving the input value for the function (get, replace, take) and three different ways of applying the function: as a value function (T -> T), as mutable reference (&mut T -> R) and as normal reference (&T -> R) which is redundant but more expressive. I would propose something along the lines of this:
/// Holds a temporary value that replaces the cells content when
/// it is dropped. A panic in its `map` function will forget about the
/// internal state and not perform the final replacement.
pub struct LendGuard<'a, T: 'a> {
  cell: &'a Cell<T>,
  value: Option<T>, // Panic safety
}

impl<T> Cell<T> {
  /// Give a copy of the content to the LendGuard
  fn lend_copy<T>(&self) -> LendGuard<T> where T: Copy {
    LendGuard{cell: self, value: Some(self.get())}
  }

  /// Replaces the cells value with T::default() until the LendGuard
  /// is dropped.
  fn lend<T>(&self) -> LendGuard<T> where T: Default {
    LendGuard{cell: self, value: Some(self.take())}
  }

  /// Replaces the cells value with the supplied value until the
  /// LendGuard is dropped.
  fn pawn<T>(&self, t: T) -> LendGuard<T> {
    LendGuard{cell: self, value: Some(self.replace(t))}
  }
}

impl<'a, T: 'a> LendGuard<'a, T> {
  /// Replaces the extracted value with the result of applying
  /// the function to it.
  fn map<F>(mut self, f: F) where F: FnOnce(T) -> T {
    self.value = self.value.take().and_then(|v| Some(f(v)));
  }

  /// Inspects the extracted value through an immutable reference
  /// before moving it back. The result of the inspection is returned
  /// to the caller.
  fn inspect<F, R>(self, f: F) -> R where F: FnOnce(&T) -> R {
    f(self.value.as_ref().unwrap())
  }

  /// Inspects the extracted value through a mutable reference,
  /// possibly modifying it before moving it back. The result of
  /// the inspection is returned to the caller.
  fn mutate<F, R>(mut self, f: F) -> R where F: FnOnce(&mut T) -> R {
    f(self.value.as_mut().unwrap())
  }
}

impl<'a, T> Drop for LendGuard<'a, T> {
  fn drop(&mut self) {
    self.value.take().map(|v| self.cell.set(v));
  }
}

This interface could then be used as follows:
let cell = Cell::new(5);
self.lend_copy().map(|v| v + cell.get()); // cell = Cell(10)
let inspect = cell.lend().inspect(|&v| v + cell.get()); // inspect = 10
cell.pawn(42).mutate(|v: &mut _| *v += cell.get()); // cell = Cell(52)
println!("{}", cell.into_inner() + inspect); //print 62

In case someone wants to try this out by simple copy&paste, here is an (even slightly more generic) implementation of this proposal: github gist

Cell::update has been added, gated under feature cell_update.
Tracking issue: https://github.com/rust-lang/rust/issues/50186


[blockchain] fix BlockPool

I think blockchain/infra/mem/block_pool.go file need fix.
Help wanted + Suggestion:

remove type Block = ygg.Block
blockMap map[uint64]Block => blockMap map[uint64]blockchain.DefaultBlock
declare Blockpool interface to block.go so any other domain or api can reference this interface
write test code!!

@zeroFruit


add TLS support

This adds the ability to connect to TLS-protected queue brokers, via the 'protocol' option. Update affected unit tests, add functional test (verified against activemq v5.15.0). Also document how to run the functional tests

...and those travis failures are totally not my fault. It looks like it's having networking issues?

...and those travis failures are totally not my fault. It looks like it's having networking issues?

I'll kick the build a few times until it passes :+1:

@clue :ping_pong:

@jsor I can think of a way. I'll make some time to try to enable TLS on one of the stomp providers configured in the travis tests. If I can do that, I should be able to add a new provider (or extend the tests) to test that secure connections work.


Fix style deprecation warnings, Atom no longer uses Shadow DOM.

This fixes the warnings outlined in #271. Note that the style class in
Less had to be escaped because the class now contains - which is a
reserved operator in Less.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


swift-api-checker.py: introduce a sub directory under framework name for ABI baselines

In case we need to also include framework API baselines in the future, we should have
an additional layer of directory for ABI baseline files.

@swift-ci please smoke test