RSS Reader Tutorial

Please provide link for referenced codes.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 9a621c9b-3a97-3d49-4de9-b309a491a75b
Version Independent ID: 492ab689-9940-02ee-2f38-bafcd1dc9ad2
Content: RSS reader tutorial (Rust for Windows with VS Code)
Content Source: hub/dev-environment/rust/rss-reader-rust-for-windows.md
Product: dev-environment
Technology: windows-rust
GitHub Login: @stevewhims
Microsoft Alias: stwhi

Hi, @whoiskevinrich. I'm not sure what you mean by "referenced codes". Could you quote (or somehow describe) the part of the topic that you're thinking of that's wrong? And what you'd expect to see there instead?
Are you referring to the source code examples there, by any chance?
Many thanks,
Steve

Yes, there are some inline code examples, but it would be helpful to see the context of the entire project.

Thanks, @whoiskevinrich. The topic is designed so that you follow the steps and build the project. Once you've followed along with the topic, followed the steps I mean, then you'll have the context of the entire project.
Many thanks!
Steve


Backend: Add livestream to application object for disabling livestream

Based on user groups

Why did we set this to 2 story points?
Am I missing something?

Was missing the push to firestore


Allow the use of raw images for Hyperkit

In reference to both the pull request: #206 and the revert #208 this issue is to track the actual needed changes to allow the imageformat to be changed.
Related:

[ ] https://github.com/code-ready/machine-driver-hyperkit/issues/11

Also see https://github.com/code-ready/snc/issues/188

Seems the issue #188 was never refered to in the commit and discussion on the PR


Adding typedef to Characteristics enum values.

Re-spin of PR #839 that got accidently closed.
This PR allows defining a type to the Characteristics enum values that were added recently so they can be re-used as return type, such as returning the security alarm state itself rather than its integer value:
  Characteristic::SecuritySystemCurrentState::SecuritySystemCurrentStateEnum_t getState() {
    return partitionCurrentState->getVal<Characteristic::SecuritySystemCurrentState::SecuritySystemCurrentStateEnum_t>();
  }

Also allows better Intellisense help when hovering over enum values which now shows:
enum Characteristic::SecuritySystemCurrentState::SecuritySystemCurrentStateEnum_t::DISARMED = 3

or
enum Characteristic::ContactSensorState::ContactSensorStateEnum_t::DETECTED = 0

instead of
enum Characteristic::SecuritySystemCurrentState::<unnamed>::DISARMED = 3

and
enum Characteristic::ContactSensorState::<unnamed>::DETECTED = 0

Any chance to add this to 1.10 (though I have not checked it it can be merged without conflict).

I'll have to review this one first.

Should I create a new branch and hope it gets added to the 2.x releases?

Wait until 2.0.0 comes out of pre-release and then you can start PR's into that branch for use in 2.1 (which will be a large release with lots of changes, ethernet support, new wifi stack, etc).   I'm waiting to see if folks are finding problems with the 2.0.0 pre-release (i.e. anything new because of the switch to the new IDF) before I re-issue as regular release and then can start on new functionality.


Feature account create alert

@micchickenburger : may need some help at the appropriate way to write the timeout function to close on line 19 of thriver-core/lib/client/alerts.js

@micchickenburger , need help resolving these travis-ci fails. Not sure what I'm doing wrong here.

This branch is dated and alerts will be built as toasts in a new branch.


Can't pickle local object if lambdas are used

Describe the bug
Starting the runner on a pipeline that contains lambdas leads to the following logs:
2024-04-06 19:58:54.544 [info] Launching Pipeline (ab133d54-709c-4605-ae98-e1babcb10642): file:///c%3A/Users/Lars/OneDrive/Desktop/test/test.sdstest - myPipeline
2024-04-06 19:58:54.544 [info] Sending message to python server: {"type":"program","id":"ab133d54-709c-4605-ae98-e1babcb10642","data":{"code":{"test":{"gen_test":"# Imports ----------------------------------------------------------------------\r\n\r\nimport safeds_runner\r\nfrom safeds.data.tabular.containers import Row, Table\r\n\r\n# Pipelines --------------------------------------------------------------------\r\n\r\ndef myPipeline():\r\n    titanic = safeds_runner.memoized_call(\"safeds.data.tabular.containers.Table.from_csv_file\", Table.from_csv_file, ['C:/Users/Lars/OneDrive/Desktop/test/titanic.csv'], [safeds_runner.file_mtime('C:/Users/Lars/OneDrive/Desktop/test/titanic.csv')])\r\n    safeds_runner.save_placeholder('titanic', titanic)\r\n    filtered = safeds_runner.memoized_call(\"safeds.data.tabular.containers.Table.filter_rows\", Table.filter_rows, [titanic, lambda param1: (safeds_runner.memoized_call(\"safeds.data.tabular.containers.Row.get_value\", Row.get_value, [param1, 'age'], [])) >= (18)], [])\r\n    safeds_runner.save_placeholder('filtered', filtered)\r\n","gen_test_myPipeline":"from .gen_test import myPipeline\r\n\r\nif __name__ == '__main__':\r\n    myPipeline()\r\n"}},"main":{"modulepath":"test","module":"test","pipeline":"myPipeline"}}}
2024-04-06 19:58:54.545 [info] [Runner-Err] DEBUG:root:Received Message: {"type":"program","id":"ab133d54-709c-4605-ae98-e1babcb10642","data":{"code":{"test":{"gen_test":"# Imports ----------------------------------------------------------------------\r\n\r\nimport safeds_runner\r\nfrom safeds.data.tabular.containers import Row, Table\r\n\r\n# Pipelines --------------------------------------------------------------------\r\n\r\ndef myPipeline():\r\n    titanic = safeds_runner.memoized_call(\"safeds.data.tabular.containers.Table.from_csv_file\", Table.from_csv_file, ['C:/Users/Lars/OneDrive/Desktop/test/titanic.csv'], [safeds_runner.file_mtime('C:/Users/Lars/OneDrive/Desktop/test/titanic.csv')])\r\n    safeds_runner.save_placeholder('titanic', titanic)\r\n    filtered = safeds_runner.memoized_call(\"safeds.data.tabular.containers.Table.filter_rows\", Table.filter_rows, [titanic, lambda param1: (safeds_runner.memoized_call(\"safeds.data.tabular.containers.Row.get_value\", Row.get_value, [param1, 'age'], [])) >= (18)], [])\r\n    safeds_runner.save_placeholder('filtered', filtered)\r\n","gen_test_myPipeline":"from .gen_test import myPipeline\r\n\r\nif __name__ == '__main__':\r\n    myPipeline()\r\n"}},"main":{"modulepath":"test","module":"test","pipeline":"myPipeline"}}}
2024-04-06 19:58:57.204 [info] [Runner] Message received: '{"type": "placeholder_type", "id": "ab133d54-709c-4605-ae98-e1babcb10642", "data": {"name": "titanic", "type": "Table"}}'
2024-04-06 19:58:57.205 [info] Placeholder was calculated (ab133d54-709c-4605-ae98-e1babcb10642): titanic of type Table
2024-04-06 19:58:57.205 [info] Placeholder was calculated (ab133d54-709c-4605-ae98-e1babcb10642): titanic of type Table
2024-04-06 19:58:57.205 [info] Placeholder was calculated (ab133d54-709c-4605-ae98-e1babcb10642): titanic of type Table
2024-04-06 19:58:57.205 [info] Placeholder was calculated (ab133d54-709c-4605-ae98-e1babcb10642): titanic of type Table
2024-04-06 19:58:57.245 [info] [Runner] Message received: '{"type": "runtime_error", "id": "ab133d54-709c-4605-ae98-e1babcb10642", "data": {"message": "Can't pickle local object 'myPipeli<truncated>'

To Reproduce
Run the following pipeline:
package test

pipeline myPipeline {
    val titanic = Table.fromCsvFile("C:/Users/Lars/OneDrive/Desktop/test/titanic.csv");
    val filtered = titanic.filterRows((param1) -> (param1.getValue("age") as Int) >= 18);
}

Replace the path to the CSV file with some other one.
The same issue occurs with block lambdas:
package test

pipeline myPipeline {
    val titanic = Table.fromCsvFile("C:/Users/Lars/OneDrive/Desktop/test/titanic.csv");
    val filtered = titanic.filterRows((param1) {
        yield param2 = (param1.getValue("age") as Int) >= 18;
    });
}

Expected behavior
The pipeline should run without error.
Screenshots (optional)
No response
Additional Context (optional)
It would also be nice if errors would be marked with [error] instead of [info] in the logs.

@lars-reimann The lambda is already solved by the memoization PR I will submit later

OK, you can override my change then (PR coming soon). I need it now for some final testing.

:tada: This issue has been resolved in version 0.8.1 :tada:
The release is available on:

v0.8.1
GitHub release

Your semantic-release bot :package::rocket:

:tada: This issue has been resolved in version 0.9.0 :tada:
The release is available on:

v0.9.0
GitHub release

Your semantic-release bot :package::rocket:


fix support nested type for schema change reorder

Proposed changes
backport: https://github.com/apache/doris/pull/39210
Issue Number: close #xxx

run buildall


Prefix stacking

Hey im having trouble understanding the prefix stacking. Ive read the wiki and forums and tried multiple things but I'm still having no luck.
I'm trying to have my chat show the following: {X} [Mystic] Vese883:
I've put all my member ranks on a track named Member
I've put all my staff ranks on a track named Staff
I've put all my Donor ranks on a track named Donor
Here is my config https://pastebin.com/NMsmnAj4

Please use Discord to ask questions: https://discord.gg/luckperms
This tracker is only for bugs/suggestions.


Faker issue: Byebug reporting error at require "faker"

Still very new to Rails and am trying to use Faker to -- of course -- seed my userdata db with realistic-looking data. When I execute the code below via rails db:seed, I don't get any response from the CLI, rather just a returned line with no output, messaging, etc.
Seeds.rb:
require "faker"

10.times do
    data = Userdatum.new(
        adds1: Faker::Address.street_address,
        adds2: '-',
        addcity: Faker::Address.city,
        addstate: Faker::Address.state,
        addzip: Faker::Address.zip,
        phone: Faker::PhoneNumber.cell_phone,
        taxpref: Faker::Boolean.boolean,
        mtgepref: Faker::Boolean.boolean,
        inspref: Faker::Boolean.boolean,
        created_at: '123456',
        updated_at: '123456',
        user_id: 10000,
        fName: Faker::Name.first_name,
        lName: Faker::Name.last_name,
        email: Faker::Internet.email
    )
    data.save
end

Byebug error message:
[4, 13] in /Users/robertmaxwell/RUBY/GULF/GULF/db/seeds.rb
    4: # Examples:
    5: #
    6: #   movies = Movie.create([{ name: 'Star Wars' }, { name: 'Lord of the Rings' }])
    7: #   Character.create(name: 'Luke', movie: movies.first)
    8:
=>  9: require "faker"
   10:
   11: 10.times do
   12:     data = Userdatum.new(
   13:         adds1: Faker::Address.street_address,

Schema.db
ActiveRecord::Schema.define(version: 2021_07_26_050246) do
create_table "userdata", force: :cascade do |t|
   t.string "adds1"
   t.string "adds2"
   t.string "addcity"
   t.string "addstate"
   t.string "addzip"
   t.string "phone"
   t.boolean "taxpref"
   t.boolean "mtgepref"
   t.boolean "inspref"
   t.datetime "created_at", precision: 6, null: false
   t.datetime "updated_at", precision: 6, null: false
   t.integer "user_id"
   t.string "fName"
   t.string "lName"
   t.string "email"
   t.index ["user_id"], name: "index_userdata_on_user_id"
 end

Please, please, please and thank you in advance for any assistance here!

Hey there @robertemaxwell , from the scenario shared above: rails loads the faker library(gem) i.e. require 'faker'. faker-bot is a command line application (CLI) that adds a $ faker command executibale on a terminal interface. See example here in the tests. Hence the calls to Faker:: namespace with a rails app are directed to the faker library/not faker bot. 🙂
Hope this was helpful and I wish you the best in your learning journey! Will go ahead and close this for now but feel free to respond if unclear. Cheers


Error: failed to parse [item]

Hi, I got the following error when uploading my stashes:

I went into the character's stash and deleted the item ("Soul Thirst" rare assassin claw) and now the stashes load problem-free.
I've attached the character's stash file so you can see for yourself what the issue is.
Garthly.zip
(I love this tool, by the way. Thanks!)

Fixed, thanks for the report. :)


Application crashes on start on iOS 9.1 device

CastVideos[2396:1326240] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '*** -[__NSPlaceholderArray initWithObjects:count:]: attempt to insert nil object from objects[0]'

Found a workaround - Go into CastVideos > Images and select CastFrameworkAssets.xcassets. In the right sidebar, change the Location from .../GoogleCastSDK-2.7.1-Release/... to .../GoogleCastSDK-2.10.0-Release/...
I will investigate the underlying issue.

Thanks. That worked immediately.

Fixed in commit 5f166c4b. The app contains a (somewhat hacky) link to the xcassets file containing the Cast icons to get around CocoaPods not fully supporting it. I missed updating the link target when the SDK version number changed. Thanks for catching this!


Increase simulator compatibility and add FuseSoC support

First two patches fixes support for other simulators than icarus. Tested successfully with ISim, ModelSim and Xsim
Third patch adds FuseSoC support. To try the FuseSoC support do the following
Create a workspace directory and run fusesoc library add tinyfpga_bootloader https://github.com/tinyfpga/TinyFPGA-Bootloader to register the repo as a FuseSoC library. If it's already on disk, it's enough to just add
[library.tinyfpga_bootloader]
location = /path/to/repo

to your fusesoc.conf
Now you can build or run any of the targets.
Run fusesoc core-info TinyFPGA-Bootloader to see available targets
Build for a FPGA target with fusesoc build --target=<target> TinyFPGA-Bootloader or run a simulation with fusesoc run --target=<target> --tool=<tool> TinyFPGA-Bootloader, where <tool> is any of the FuseSoC-supported RTL simulators (tested with icarus, isim, modelsim and xsim). If --tool is left out from the simulation, FuseSoC will use icarus as the default tool.

oh, and fusesoc run --target=lint TinyFPGA-Bootloader to run the linter
Don't have time right now to do a Diamond backend, and I also don't have access to a subscription version that supports ECP5 so TinyFPGA EX support will have to wait


Updated README.md, etc

Relevant Issue (if applicable)
n/a
Details
Updated README.md and ChangeLog

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


selinux: drop superflous relabel

The same relabel is already done in writeStringToRundir so we don't
need to do it twice.  The version in writeStringToRundir takes into
account the correct file path when using user namespaces.
Signed-off-by: Giuseppe Scrivano<EMAIL_ADDRESS>

/approve

LGTM

LGTM assuming happy tests


codeshift-cli package doesn't exist on npm

hey, the stuff you added in https://github.com/CodeshiftCommunity/CodeshiftCommunity/pull/58 is very exciting!
i think only problem is that the codeshift-cli package is not available on npm yet?
i would actually consider naming it codemods-cli - since end consumers of the library don't necessarily know what "codeshift" is, but do know what a "codemod" is - i think it'd be better understandable.

Hey @kiprasmel 👋
The cli should be available on npm after installing it via the module name npm install -g @codeshift/cli.
Unfortunately, there's no way to make the cli command match the module name due to an npm bug  😢 so the cli command is currently codeshift-cli. However I would be open to adding the alias you suggested to the bin property of the package.json. Feel free to contribute that if you like!

oh yes, after installing it, it's fine. but i was thinking - most people who will be consuming codemods,

they don't need to use the cli often, so using through npx would be better, and
they don't know much about the codeshift stuff, but they know about codemods, thus i think naming the binary codemods-cli, not codeshift-cli, would be better, and
combining both, it'd be best to have an npm package called codemods-cli, so that both 1. and 2. are utilised, and the end consumer can do npx codemods-cli - both short and intuitive.

so you'd need to somehow publish to another package, codemods-cli, and make the npx codemods-cli work out of the box.


'compileJava' task (current target is 11) and 'compileKotlin' task (current target is 1.8) jvm target compatibility should be set to the same Java version

Task :data:base:compileKotlin
'compileJava' task (current target is 11) and 'compileKotlin' task (current target is 1.8) jvm target compatibility should be set to the same Java version.
By default will become an error since Gradle 8.0+! Read more: https://kotl.in/gradle/jvm/target-validation
Consider using JVM toolchain: https://kotl.in/gradle/jvm/toolchain

https://github.com/lydavid/MusicSearch/commit/5240d64878e7a03efdb8264a67f5460d5dbe3954 should fix this


[Request for External] Page 21 "Customer Card" - procedure ContactOnAfterValidate

Hello, we need the following currently local procedure to be set to global. Can you please check this? Thank you!
[Request-For-External] Page 21 "Customer Card" - procedure ContactOnAfterValidate
procedure ContactOnAfterValidate()
begin
    ActivateFields();
end;

Should we made protected local procedure ActivateFields() instead?

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.
Build ID: 51059.


When restart a vault client , the token will be unavailable , I have to renew a token and restart all the client services

Care to explain the scenario? What is happening and what would you expect to happen so we can reproduce the issue?

When I use an invalid token to access Vault, it will cause the valid token to invalid.
When I restart the Vault client (a service), the same issue will show up. but not so often.

Can you post the metadata of the token?

What's the metadata of the token, the configuration?

The output of Vault CLI's vault token lookup. Example:
$ vault token lookup bc80d807-2952-33e3-c8ae-661596c9623e
Key                 Value
---                 -----
accessor            16674493-1f8f-d3c7-4290-bd844259e394
creation_time<PHONE_NUMBER>
creation_ttl        0
display_name        token
entity_id           n/a
expire_time         <nil>
explicit_max_ttl    0
id                  bc80d807-2952-33e3-c8ae-661596c9623e
issue_time          2018-06-11T10:04:19.052848876+02:00
meta                <nil>
num_uses            0
orphan              false
path                auth/token/create
policies            [root]
renewable           false
ttl                 0

[root@infbjvm109 vault_0.10.0_linux_amd64]# vault login ef100e61-b60a-8372-2ef1-e8ab733795a3
WARNING! The VAULT_TOKEN environment variable is set! This takes precedence
over the value set by this command. To use the value set by this command,
unset the VAULT_TOKEN environment variable or set it to the token displayed
below.
Success! You are now authenticated. The token information displayed below
is already stored in the token helper. You do NOT need to run "vault login"
again. Future Vault requests will automatically use this token.
Key                Value

token              ef100e61-b60a-8372-2ef1-e8ab733795a3
token_accessor     5a4f8b02-6ff3-3dbf-8aef-1a4b55296522
token_duration     ∞
token_renewable    false
token_policies     [root]
but when execute
[root@infbjvm109 vault_0.10.0_linux_amd64]# vault token lookup ef100e61-b60a-8372-2ef1-e8ab733795a3
Error looking up token: Error making API request.
URL: POST http://<IP_ADDRESS>:8200/v1/auth/token/lookup
Code: 403. Errors:

permission denied

Sorry, we can't assist you on Vault issues. Please come back once you've solved basic Vault issues.

Closing due to lack of requested feedback. If you would like us to look at this issue, please provide the requested information and we will re-open the issue.


BIGTOP-3727: Add HBase support for Bigtop 3.1.0 Mpack

Description of PR
How was this patch tested?
For code changes:

[ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'BIGTOP-3638. Your PR title ...')?
[ ] Make sure that newly added files do not have any licensing issues. When in doubt refer to https://www.apache.org/licenses/

Phoenix query server failed to start:
Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/stacks/BGTP/1.0/services/HBASE/package/scripts/phoenix_queryserver.py", line 91, in <module>
    PhoenixQueryServer().execute()
  File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 352, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/stacks/BGTP/1.0/services/HBASE/package/scripts/phoenix_queryserver.py", line 48, in start
    phoenix_service('start')
  File "/var/lib/ambari-agent/cache/stacks/BGTP/1.0/services/HBASE/package/scripts/phoenix_service.py", line 42, in phoenix_service
    environment=env)
  File "/usr/lib/ambari-agent/lib/resource_management/core/base.py", line 166, in __init__
    self.env.run()
  File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 160, in run
    self.run_action(resource, action)
  File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 124, in run_action
    provider_action()
  File "/usr/lib/ambari-agent/lib/resource_management/core/providers/system.py", line 263, in action_run
    returns=self.resource.returns)
  File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 72, in inner
    result = function(command, **kwargs)
  File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 102, in checked_call
    tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy, returns=returns)
  File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 150, in _call_wrapper
    result = _call(command, **kwargs_copy)
  File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 314, in _call
    raise ExecutionFailed(err_msg, code, out, err)
resource_management.core.exceptions.ExecutionFailed: Execution of '/usr/bgtp/current/phoenix-server/bin/queryserver.py start' returned 127. -bash: /usr/bgtp/current/phoenix-server/bin/queryserver.py: No such file or directory

Phoenix query server failed to start:
Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/stacks/BGTP/1.0/services/HBASE/package/scripts/phoenix_queryserver.py", line 91, in <module>
    PhoenixQueryServer().execute()
  File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 352, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/stacks/BGTP/1.0/services/HBASE/package/scripts/phoenix_queryserver.py", line 48, in start
    phoenix_service('start')
  File "/var/lib/ambari-agent/cache/stacks/BGTP/1.0/services/HBASE/package/scripts/phoenix_service.py", line 42, in phoenix_service
    environment=env)
  File "/usr/lib/ambari-agent/lib/resource_management/core/base.py", line 166, in __init__
    self.env.run()
  File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 160, in run
    self.run_action(resource, action)
  File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 124, in run_action
    provider_action()
  File "/usr/lib/ambari-agent/lib/resource_management/core/providers/system.py", line 263, in action_run
    returns=self.resource.returns)
  File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 72, in inner
    result = function(command, **kwargs)
  File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 102, in checked_call
    tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy, returns=returns)
  File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 150, in _call_wrapper
    result = _call(command, **kwargs_copy)
  File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 314, in _call
    raise ExecutionFailed(err_msg, code, out, err)
resource_management.core.exceptions.ExecutionFailed: Execution of '/usr/bgtp/current/phoenix-server/bin/queryserver.py start' returned 127. -bash: /usr/bgtp/current/phoenix-server/bin/queryserver.py: No such file or directory

Fixed

May I ask why did you remove phoenix here?

May I ask why did you remove phoenix here?

Phoenix is another service, we shouldn't include phoenix here, it'll be added in a separate PR

HBase has been bumped to 2.4.13. Let me rebuild and deploy 2.4.13 with this PR before merging it, thanks.

Merged, thanks @yzydhf


Global done

Hi Sebastian,
For each iframe upload done, triggers done event.
For each xhr upload done, triggers done event.
what about a global done, when all each done?
are you considering?
best way to do?
thx

Please have a look at the stop callback:
https://github.com/blueimp/jQuery-File-Upload/wiki/Options
https://github.com/blueimp/jQuery-File-Upload/wiki/API

Thx!


Support for icon-only buttons/fixed listener (un)installation in TextFieldUI

These two commits fix one issue and improve the library in another regard.

The listeners in the DarculaTextFieldUI classes weren't properly added and removed when the L&F was changed. Instead they were only added. This led to NPEs when changing the L&F and then opening the search field popup, as getComponent() returned null.
I couldn't find a way to create a button that only displays the icon. It turned out that the contentAreaFilled property was not honored. I added a flag to the code that make sure the rounded rectangle is not drawn, when contentAreaFilled==false .

I'd appreciate it, if you merged the two changes.
Thanks.

Thanks!


Fbe.overwrite must not modify the same value is there

@rultor release, tag is 0.0.45

@rultor release, tag is 0.0.45

@yegor256 OK, I will release it now. Please check the progress here

@rultor release, tag is 0.0.45

@yegor256 Done! FYI, the full log is here (took me 8min)


Fix load/save and usage of Installation Scope (fix #2615)

Your fix would store package scope as a localized string, breaking bundles when importing them on a machine with a different language. I am currently working on a cross-language fix
Thank you anyway!


Add flag to skip prompt for ignored folders

Hey,
I was trying to automate this command, and I needed a way to skip the prompt
Cheers,

I also want to automate some wml steps and would like to have this PR merged!
But @mlabrum, maybe you could update the README as well?


Bump jest-playwright-preset from v3.0.1 to v4.0.0

fixes #386
This PR proposes to bump jest-playwright-preset package from v3.0.1 to v4.0.0 in order to fix the moderate vulnerability stemming Axios reported by Dependabot.
<EMAIL_ADDRESS>Release note

Version
Published prerelease version: v0.15.3-next.1

Changelog
🐛 Bug Fix

Bump jest-playwright-preset from v3.0.1 to v4.0.0 #400 (@kemuridama)
Improve type safety and code quality #383 (@bryanjtc @yannbf)
Refactor: Improve internal code #378 (@bryanjtc @yannbf)

Authors: 3

Bryan Thomas (@bryanjtc)
Ryo Ochiai (@kemuridama)
Yann Braga (@yannbf)

Updated dependencies detected. Learn more about Socket for GitHub ↗
Packages	Version	New capabilities	Transitives	Size	Publisher
jest-playwright-preset 3.0.1...4.0.0	None	+1/-4	89.5 kB	mxschmitt


URL for linux and MacOS not working

Page: /setup/spin/
Feedback:

@GMafra , are you referring to the curl commands?  I just tested and they "work on my machine." :)
Can you provide more detail about what's happening. What environment are you running it in (like, Terminal on a Mac), and what error message are you gettting?

Sorry for the lack of Info!
I'm running on my MacOS running iTerm2 the command to download the binaries:
curl -LO https://storage.googleapis.com/spinnaker-artifacts/spin/$(curl -s https://storage.googleapis.com/spinnaker-artifacts/spin/latest)/darwin/amd64/spin
this is the result:
<?xml version='1.0' encoding='UTF-8'?><Error>NoSuchKey<Message>The specified key does not exist.</Message>No such object: spinnaker-artifacts/spin/latest)/darwin/amd64/spin</Error>%

Also running iTerm2. I copied the command from @GMafra's comment, and it worked for me.

It looks like you probably aren't using bash? The $() command substitution didn't seem to work...

Hi @GMafra, were you able to get this working by using bash instead?

Ping one more time @GMafra

Hi Brian, sorry for the late response, Yes, it worked with bash, thanks!

zsh being the default shell on Mac now, we should probably still fix the page by removing the slashes.

Yea i've just ran into this same problem using zsh, would be good to get the docs updated.


As a teacher I would like to create groups for my classes to be better prepared in the lessons

Preconditions: The teacher is logged in and his account is verified.

The teacher clicks in the "user" menu the corresponding item
-> The system shows an overview over the groups belonging tho the teacher's account
The teacher clicks on a "plus" button to create a new group
-> A popup with a form appears
The teacher fills in the form, name of the group, number of members, ... and confirms it
-> The overview (dashboard) shows now an additional group
The teacher clicks on the row of the group to get more information
-> The view of the groups provides the teacher more insides of this specific new group, especially the newly created names of the accounts for the students and the corresponding one-time passwords
The teacher can now export or copy & paste the information to allocate it to the students

Post-conditions: The new group and the corresponding user accounts are created and stored in the database
This procedure can be used as a test case.

@philippmaurer , currently there is an issue when I delete the user (teacher account) while one of the students is still logged in

The teacher can delete his account although he holds accounts for groups with active members
The student (while logged in) gets

instead of a dedicated message about his account which does not exist anymore.

Sorry for the german text ... its C&P from a discussion:
ich finde es SEHR irritierend, dass mein Gruppenaccountname z.B. g:03 ist (siehe Statusinformation) ist, ich aber 03 beim login eingeben muss (und nicht g:03), mein Passwort aber g:03 ist. Der Lehrer sieht ebenfalls in seiner Liste g:03 und wird dem Schüler bestimmt einen falschen Tipp geben, wenn der seine Accountnummer vergessen hat. Mein Vorschlag: im Gruppenlogin 'Name der Gruppe' WEGLASSEN und als Account g:03 eingeben (z.B. Klasse9:Reinhard).


Add Missing Favicon and Update Mobile Hamburger Menu to 'X' When Clicked

Favicon Missing: The website currently have default vercel favicon. Add a favicon that appears on the browser tab.

Mobile Hamburger Menu: On mobile devices, when the hamburger menu is clicked, it should change to an 'X' to indicate that the menu is open.

@Aryainguz Please assign this task to me(myselfshivams) labels gssoc, hackoberfest.

go ahead on this @myselfshivams


Responsive Navigation documentation incorrect/incomplete

Description
The documentation for Responsive Navigation is very incomplete.  https://foundation.zurb.com/sites/docs/responsive-navigation.html#js-module
It does not include all of the JS files that must be loaded in order to avoid error such as:
foundation css "Cannot read property 'DropdownMenu' of undefined"

It also does not mention at all which Sass partials need to be loaded.
Possible Solution
In my implementation of an Accordion Dropdown menu using this example code:
https://foundation.zurb.com/sites/docs/responsive-navigation.html#accordion-dropdown-menu
I found that I had to load all of these JS files in this order before the menu could function without errors:
    node_modules/foundation-sites/dist/js/plugins/foundation.core.min.js: {}
    node_modules/foundation-sites/dist/js/plugins/foundation.util.keyboard.min.js: {}
    node_modules/foundation-sites/dist/js/plugins/foundation.util.box.min.js: {}
    node_modules/foundation-sites/dist/js/plugins/foundation.util.nest.min.js: {}
    node_modules/foundation-sites/dist/js/plugins/foundation.dropdownMenu.min.js: {}
    node_modules/foundation-sites/dist/js/plugins/foundation.drilldown.min.js: {}
    node_modules/foundation-sites/dist/js/plugins/foundation.accordionMenu.min.js: {}
    node_modules/foundation-sites/dist/js/plugins/foundation.util.mediaQuery.min.js: {}
    node_modules/foundation-sites/dist/js/plugins/foundation.responsiveMenu.min.js: {}
    node_modules/foundation-sites/dist/js/plugins/foundation.util.triggers.min.js: {}
    node_modules/foundation-sites/dist/js/plugins/foundation.responsiveToggle.min.js: {}
    node_modules/foundation-sites/dist/js/plugins/foundation.util.mediaQuery.min.js: {}
    node_modules/foundation-sites/dist/js/plugins/foundation.util.motion.min.js: {}

Additionally, wouldn't it make much more sense for each of the different menu combinations (e.g. https://foundation.zurb.com/sites/docs/responsive-navigation.html#accordion-dropdown-menu) to have their own subsection for JS, in which the necessary JS files are listed? They all require different JS so there is no one size fits all solution as the current documentation implies.
Besides the JS documentation issues, there is no documentation at all for what Sass partials need to be loaded. There seems to be an assumption throughout the Foundation docs that we have loaded all of the Foundation CSS/Sass instead of loading only the necessary partials. In case this is helpful for anyone in the future, I think these are the necessary ones in order to implement the combination of an Accordion and Dropdown menu Responsive Navigation:
@include foundation-menu;
@include foundation-accordion-menu;
@include foundation-dropdown-menu;

Hi @maskedjellybean,
Thanks for reporting this. Do you want to provide a PR to improve the docs here?

In addition to incomplete it appears to be confused itself.  In the Javaascript section of the doc there are several references to a "Tab Bar". Tab bars are not referred to anywhere else, and a search of your doc for "tab bar" returns no results.  Did you mean Top bar?  Title bar?
I could do a PR if I knew what was intended.
In the mean time I will continue my quest to create a responsive accordion/dropdown menu that hides in off-canvas at small sizes.

PRs which improve the docs are very welcome @anitagraham.

Thanks Daniel - I will haveago.


Add Capture Engine support to SharpDX.MediaFoundation

Let me know if there's anything you'd like me to fix in this!

Thanks for this contribution!


Add support for Godot 4 annotations

This enables the highlighting of annotations in the form of @export, @export_multiline, @onready, etc.
I'm new to writing Vim syntax highlighting files so it's possible I did this in a non-idiomatic way. If there's a more concise way of matching all lowercase and underscores following an @, please let me know.
Perhaps ^@[a-z_]\+?

Also open to suggestions on whether it should be gdscriptStatement or something else.


Problems testing crouton downstream

Hi! I'm working on adding a new target to crouton with my own fork. However, if I download my forked version and run it, it still fetches info from the main crouton repo, meaning it won't recognize the targets I'm testing.
Here's output of the script, run with sudo bash -x ~/Downloads/crouton:
https://gist.github.com/anonymous/4374a63b9b32bfcfbb9eb70783eb61bc
As noted by line 2, the version fetched is 1-20161028111244~master:948ee6dc. My fork is 2 commits above that, at eeed0ef.
Is this a bug? Or is there some flag I should specify when running crouton for the first time?

If I'm testing a target locally, I usually invoke installer/main.sh directly in my crouton checkout. That takes the usual arguments and lets you test without interference from the crouton wrapper script.

Thank you! (for those curious, download a zip of your forked repo, unzip it in your chroot into ~/Downloads, and run installer/main.sh).
I'll report the results of my new target if I can get it working.


In business name tag, clear icon doesn't work

Entered the following occupant:
KSDavison & Associates, UNIT 207 -- 225 Francis Way, New Westminster, BC
then clicked on clear icon. Text box didn't clear.

Text box clear function works in prod.

Fixed in delivery now. Also I changed the example text for the businesses by tag  tab to the shortest example of a multi-word tag I could find, "first nations".


fix(angular): make route observables available on constructor

Short description of what this resolves:
The fix introduced in https://github.com/ionic-team/ionic/pull/17888 also broke certain expectations as observables on ActivatedRoute were no longer available at the constructor but only after ngOnInit, this could cause many breaking changes for users that expect to use the observables as they would in a vanilla angular app.
Changes proposed in this pull request:

Create observable proxies upon component construction

Ionic Version: 4.x
Fixes:

You sir, hats off to you my friend!


draft: Feat/examples knowledge graph

Adding knowledge graph PR draft
Also adding zod-validation-error, which improves zod error message readability. Fixing some role bugs & adding return statement when request is succesful

mind merging from main?

@njk112 are you able to add a documentation example for this? in /docs/examples/


[qt5-webengine] remove obsolete patches from qt5-webengine

Fixes #32218

[x] Changes comply with the maintainer guide
[x] SHA512s are updated for each updated download (no updated downloads)
[x] The "supports" clause reflects platforms that may be fixed by this new version (no change)
[x] Any fixed CI baseline entries are removed from that file. (no changes to that file)
[x] Any patches that are no longer applied are deleted from the port's directory.
[x] The version database is fixed by rerunning ./vcpkg x-add-version --all and committing the result.
[x] Only one version is added to each modified port's versions file.

This fix removes two patches that no longer apply due to changes in Qt 5.15.10.

Note that the command vcpkg x-add-version qt5-webengine moved the entry for baresip-libre in versions/baseline.json. This was not done manually.

Compile test pass with following triplets:
x86-windows
x64-windows


Bosh AWS CPI Should Tag Primary Volume of Bosh Director

We'd like to be able to programmatically determine which volume is the Bosh Directors' primary volume.  Volumes can have tags attached.  When bosh-init creates the volume for the director it should tag it as well.

Out of curiosity why do you need to know bosh's volume via tags instead of just looking at what bosh-init recorded in the *-state.json file? It seems that only AWS has tagging support for volumes.


Redesign data distribution to support elasticity

The ability to add and remove workers dynamically on a running cluster (required, e.g., by @jortiz16's PerfEnforce work) will require some fundamental changes to Myria architecture. Briefly, we need to add a level of indirection among workers and the data they host. This new level of data storage granularity will still be called "partitions", but these partitions will be much smaller than what can fit on a single worker, and will be immutable, though mobile (i.e., they will never be split or merged as workers are added and removed). The new hash space for our hash partitioning will be the range of these partition IDs (which will be fixed for the life of the cluster), rather than the range of worker IDs (which can grow and shrink over the life of the cluster). Instead of rehashing data when the range of worker IDs changes, existing data partitions will be moved around among workers, in a manner similar to but distinct from consistent hashing.
How Relations are represented in the Myria catalog
A Myria Relation will continue to represent a set of partitions, but those partitions will now be 1) fixed in number for the life of the cluster, 2) much more fine-grained (and hence more numerous), and 3) will no longer map 1:1 to workers.
Fixed partitions
The number of partitions can be configured only at the time of cluster creation. If it is ever changed, it will require all data in all relations to be repartitioned (by hashing or round-robin distribution). Conceptually, all relations have NUM_PARTITIONS partitions, but some of them may be empty if a minimum size threshold (in bytes or tuples) is enforced for each partition. The set of nonempty partitions is always a prefix of the range of all partitions.
Fine-grained partitions
To minimize the overhead of repartitioning, when workers are added to or removed from the cluster, we move immutable partitions around among workers rather than repartitioning the data. Instead of directly hashing tuples to workers, tuples are hashed to partitions, where the partition space is fixed for the life of the cluster. These partitions are fine-grained enough to be evenly redistributed across a large cluster when just one worker is added or removed. A reasonable value for the number of partitions might be 2^6 - 2^8.
Explicit mapping from partitions to workers
To achieve similar performance to consistent hashing (only a small amount of data needs to be moved when nodes are added or removed), without the overhead of rescanning all data to repartition it (as we would need to do when cluster membership changes), we exploit the fact that we have a centralized system with a stable leader and an existing metadata store. Rather than using a randomized approach to map partitions to workers, we maintain an explicit mapping at the coordinator from partitions to workers. (The mapping function can be essentially arbitrary, as long as it distributes partitions sufficiently uniformly across workers.)  The result is that like consistent hashing, we only need to move a small amount of data when cluster membership changes, but unlike consistent hashing, we only read the data we are going to move. Moreover, like consistent hashing, when a new node is added, only that node receives writes, so nodes currently servicing queries incur no additional write workload.
How Relations are stored and queried in Postgres
Each Relation in the Myria master catalog is distributed over NUM_PARTITIONS partitions physically implemented as Postgres tables on Myria workers. The Postgres implementation of a relation is a view which simply unions the tables representing all partitions of the relation. To simplify implementation, the view definition will be identical on all workers, and each partition will be represented by a Postgres table on all workers, including workers which do not host that partition. (Tables representing partitions not hosted on a worker will simply be empty, and as noted above, some partitions may be empty on all workers.)  Hence, moving a partition from a worker means issuing a TRUNCATE TABLE statement for the partition's local table, and moving a partition to a worker could be implemented with a Postgres COPY statement targeting the partition's local table. The overhead of scanning empty tables should be minimal relative to the ease of implementation this uniform approach affords (especially if Postgres maintains accurate statistics, which it should, given that the tables are immutable between partition rebalancings).
Changes to Myria data operations
Ingesting a new Relation
When a new Relation is ingested (e.g., via MyriaL IMPORT and STORE), we will create NUM_PARTITIONS new Postgres tables in the Postgres database of each worker in the cluster, each table named for its corresponding partition ID. We will additionally create a new Postgres view in each worker's database consisting of the UNION of all partition tables for the relation. We will then create an initial mapping of partitions to workers in the catalog, based on the number of bytes/tuples in the relation. Finally, we will physically partition the data on the workers assigned nonempty partitions, using a given PartitionFunction.
Deleting an existing Relation
When an existing Relation is deleted (e.g., through the DELETE REST API), we will first tombstone its metadata in the catalog in a single SQLite transaction, then delete the corresponding Postgres views and tables on each worker (in a local transaction, with idempotent retry globally), then delete its metadata from the catalog in a single SQLite transaction.
New Myria cluster membership operations
AddWorkers
When a new node is added to the cluster, we will create a fixed number of new workers (e.g., one for each CPU core on the node). Then we will query the catalog and create new tables and views on each new worker for each existing relation, as described above. Initially we could force the user to manually rebalance relations, e.g., by issuing SCAN and STORE commands from MyriaL. Later we could automate the rebalancing, carefully considering (and testing!) its impact on running queries (but as noted above, all writes will be confined to the new node, which is not yet online, and all reads will be evenly distributed across the cluster).
RemoveWorkers
When an existing node is removed from the cluster, we must first move all its existing partitions to workers on other nodes (this of course inverts the I/O workload for adding a new node: reads are confined to the node being removed, but all nodes, in general, must accept writes). In principle, this cannot be made fault-tolerant without replication, but hosting Postgres tablespaces on persistent EBS volumes would probably give us acceptable durability, and backing up data to S3 would yield essentially 100% durability.

Just some clarification for the ingest part. If I understand this correctly, each worker will have some number of partitions (basically num_partitions/num_workers, right? -- it sounds like num_partitions is the number of fine grained chunks for the whole cluster, not per worker). So as the coordinator is reading tuples to ingest, it will shuffle/hash the tuple to the appropriate worker, right? Also, I'm assuming we're going to use a WholeTupleHashPartitionFunction for the tuple?
I was looking at the difference between DELETE and TRUNCATE and saw this discussion: http://stackoverflow.com/questions/11419536/postgresql-truncation-speed/11423886#11423886.
In summary - even though the docs say that truncate is faster, it may not always be the case. Their claim is  "DELETE FROM table; is very cheap for small tables with no f/k refs".
Also, will happen to new tables that are created by users as they're building queries? Or are all STORE operations automatically fine-grained (not just at ingest?)?

@jortiz16 yes, the design is for each worker to store approximately num_partitions/num_workers partitions of each relation, where num_partitions is a constant fixed for the life of the cluster (as I mentioned, all data would have to be rehashed if we wanted to change it). I'm planning to use WholeTupleHashPartitionFunction as the new default for STORE (and make RACO insert a shuffle by default for FileScan). WholeTupleHashPartitionFunction is vulnerable to a theoretical worst case that RoundRobinPartitionFunction is not, but I doubt that will ever be a problem in practice (for a hash function as robust as MurmurHash3, the only potential cause of skew is duplicate tuples).
I skimmed the thread you linked on TRUNCATE vs DELETE in Postgres, and my impression is that we want the extra housekeeping performed by TRUNCATE (so we can hopefully avoid ever having to run VACUUM by hand).
All STORE operations will be partitioned as I described; there's nothing special about ingest. I should clarify that in the doc.

I see. Yes, describing how we can replace the STORE with a shuffle will make it more clear.
Although now I'm wondering whether this could cause a shuffle bottleneck for small relations. I guess this would also depend on the size of the cluster.

As I mentioned, we'll enforce a minimum size threshold for each partition, so a very small relation might be partitioned on only one or a few workers, which should mitigate against shuffle being a bottleneck.
I haven't thought about values for the threshold and would welcome any suggestions (the threshold would certainly be configurable at cluster creation).

This is a great summary of what we discussed. A few thoughts: To move a partition, I would be in favor of issuing a proper MyriaL query to use existing machinery as much as possible. This is instead of using a Postgres COPY statement.
Also, we should make sure that the function which assigns partitions to workers is the same for all relations such that tables hash-partitioned on their join attributes can be joined without any data shuffling.
How will we implement the following: "Finally, we will physically partition the data on the workers assigned nonempty partitions, using a given PartitionFunction". One approach is for shuffle operators to know about the mapping of partitions to workers. This is something that I've been advocating for a while: Shuffle Operators should have one function that maps tuples to buckets and another that maps buckets to output buffers. For parallel ingest, each worker can be told which partitions to ingest.

@mbalazin Your point 2) about ensuring that assignment of partitions to workers is consistent across relations is an important one and one that I overlooked. This goal would clearly be compromised by a purely arbitrary explicit mapping of partitions to workers, and also (partially) by the optimization I mentioned where small relations could be collapsed into one or a few partitions, to preserve the minimum size threshold for partitions. The latter would break our in-place optimizations for joins etc.
My first thought was to maintain the same mapping for all relations from the hash space to partition IDs, and from partition IDs to workers, but for relations that don't meet the minimum size threshold when partitioned across all workers, to broadcast them instead. That would increase ingest overhead for small relations, and possibly cost significant space in the aggregate, but would avoid any shuffle overhead for subsequent queries. (Another problem with this solution would be that for large clusters, the partition size for a relation partitioned across all workers might be too small to meet the minimum partition size threshold, but the relation itself might be large enough to exceed our maximum partition size threshold when broadcast as a single partition.)
Thinking about it more, though, I don't think this can work without something like consistent hashing. If we maintain an ordinary hash-based mapping from partition IDs to worker IDs, then even with the extra level of indirection, we end up moving almost all data when a worker is added or removed, just like we would without the level of indirection. The only difference is we don't have to rescan and rehash the data itself, we just rehash the partition IDs to worker IDs and move the partitions as a unit. So if we want to confine writes to only the new node when a node is added, and still maintain determinism in the partition->worker mapping, I don't think the approach of an explicit mapping will work, and consistent hashing is the only deterministic approach I can think of that will work (deterministic after workers have been assigned their vnodes on the hash ring, of course).
Re: point 1), I'm all in favor of reusing existing facilities, but I'm not sure how we'd use MyriaL to move a single partition from one worker to another, when the partition mapping is (intentionally) opaque to MyriaL. Did you have some new MyriaL syntax in mind?
Re: point 3), is it OK for shuffle operators to just query the catalog to determine the mapping of partitions to workers, or did you want them to be passed an explicit mapping? This will depend on our solution to 2), of course...

So I think I'm converging on a two-level hashing design: the first level is ordinary hashing mod NUM_PARTITIONS from tuples to partitions, and the second level is consistent hashing from worker-owned vnodes to the circle, with NUM_PARTITIONS points representing partitions uniformly spaced on the circle. In the usual fashion, the first vnode after the point representing a partition, moving clockwise on the circle, owns that partition.
The question remains how we should determine the number of vnodes for each worker, given NUM_PARTITIONS and the expected number of workers over the life of the cluster. I would like some mathematical justification for this choice, so here is my first naive attempt.  (In this case, somewhat counterintuitively, the number of balls is fixed and the number of bins is variable.) Using the usual approximation for the expected number of empty bins, the probability that any vnode has no partitions hashed to it is about exp(-m/n), for m balls and n bins, so P(empty vnode) ~ exp(-partitions/vnodes). From the coupon collector's problem, the expected minimum number of partitions needed to ensure that no vnode is empty is about n*ln(n), where n = number of vnodes. We can invert this expression to get the minimum number of vnodes we should choose for a given number of partitions. This is not easy, but we can just use a table and round/interpolate for our purposes. This also tells us how many partitions we should initially allocate. For example, if we have an 80-node cluster with 8 workers each, we have 640 workers total, so if we only assign 1 vnode/worker, we still need at least 640*ln(640) = 4135 partitions to ensure that no workers are left empty. On the other hand, if we have an 8-node cluster with just one worker each, we need just 8*ln(8) = 16 partitions to ensure that no workers are left empty. For large clusters, these partitions are much more fine-grained than we might have chosen otherwise, which is a significant disadvantage to the randomized approach of consistent hashing vs. my original approach of explicit assignment of partitions to workers. I don't know of a similarly deterministic alternative to consistent hashing that would meet our goals, though.

Whether we broadcast a relation or partition it is a physical tuning
decision. We can leave physical tuning to the user. The user will choose to
broadcast or partition. Then we wouldn't worry about minimal partition
sizes. We can deterministically hash into partitions and assign partitions
to workers.
We don't have to move most of the data when workers are added/removed. This
might be easier to discuss in front of a whiteboard, though.
One approach to using MyriaL to shuffle data might be to name the tables
differently at each worker. Then users will run queries on top of the views
but the system can issue queries that name individual tables by name.
For shuffle operators, might be better for them to receive the mapping in
parameter. Otherwise, we create a new communication path from operators to
the catalog.
Magdalena Balazinska
Jean Loup Baer Professor of Computer Science and Engineering
Associate Professor, Dept. of Computer Science & Engineering
Director of the IGERT PhD Program in Big Data / Data Science
Senior Data Science Fellow of the eScience Institute
University of Washington
On Thu, Sep 1, 2016 at 10:07 AM, Tobin Baker<EMAIL_ADDRESS>wrote:

@mbalazin https://github.com/mbalazin Your point 2) about ensuring that
assignment of partitions to workers is consistent across relations is an
important one and one that I overlooked. This goal would clearly be
compromised by a purely arbitrary explicit mapping of partitions to
workers, and also (partially) by the optimization I mentioned where small
relations could be collapsed into one or a few partitions, to preserve the
minimum size threshold for partitions. The latter would break our in-place
optimizations for joins etc.
My first thought was to maintain the same mapping for all relations from
the hash space to partition IDs, and from partition IDs to workers, but for
relations that don't meet the minimum size threshold when partitioned
across all workers, to broadcast them instead. That would increase ingest
overhead for small relations, and possibly cost significant space in the
aggregate, but would avoid any shuffle overhead for subsequent queries.
(Another problem with this solution would be that for large clusters, the
partition size for a relation partitioned across all workers might be too
small to meet the minimum partition size threshold, but the relation itself
might be large enough to exceed our maximum partition size threshold when
broadcast as a single partition.)
Thinking about it more, though, I don't think this can work without
something like consistent hashing. If we maintain an ordinary hash-based
mapping from partition IDs to worker IDs, then even with the extra level of
indirection, we end up moving almost all data when a worker is added or
removed, just like we would without the level of indirection. The only
difference is we don't have to rescan and rehash the data itself, we just
rehash the partition IDs to worker IDs and move the partitions as a unit.
So if we want to confine writes to only the new node when a node is added,
and still maintain determinism in the partition->worker mapping, I don't
think the approach of an explicit mapping will work, and consistent hashing
is the only deterministic approach I can think of that will work
(deterministic after workers have been assigned their vnodes on the hash
ring, of course).
Re: point 1), I'm all in favor of reusing existing facilities, but I'm not
sure how we'd use MyriaL to move a single partition from one worker to
another, when the partition mapping is (intentionally) opaque to MyriaL.
Did you have some new MyriaL syntax in mind?
Re: point 3), is it OK for shuffle operators to just query the catalog to
determine the mapping of partitions to workers, or did you want them to be
passed an explicit mapping? This will depend on our solution to 2), of
course...
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/uwescience/myria/issues/851#issuecomment-244145708,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACLOtS2nRVCuY_Vi4W5t2sila2yi9ARqks5qlwZbgaJpZM4JtnnU
.

Let's discuss this in person with a whiteboard at the next Myria meeting.
It will be easier I think.
Magdalena Balazinska
Jean Loup Baer Professor of Computer Science and Engineering
Associate Professor, Dept. of Computer Science & Engineering
Director of the IGERT PhD Program in Big Data / Data Science
Senior Data Science Fellow of the eScience Institute
University of Washington
On Thu, Sep 1, 2016 at 12:52 PM, Tobin Baker<EMAIL_ADDRESS>wrote:

So I think I'm converging on a two-level hashing design: the first level
is ordinary hashing mod NUM_PARTITIONS from tuples to partitions, and the
second level is consistent hashing from worker-owned vnodes to the circle,
with NUM_PARTITIONS points representing partitions uniformly spaced on
the circle. In the usual fashion, the first vnode after the point
representing a partition, moving clockwise on the circle, owns that
partition.
The question remains how we should determine the number of vnodes for each
worker, given NUM_PARTITIONS and the expected number of workers over the
life of the cluster. I would like some mathematical justification for this
choice, so here is my first naive attempt.  (In this case, somewhat counterintuitively,
the number of balls is fixed and the number of bins is variable.) Using the
usual approximation for the expected number of empty bins, the probability
that any vnode has no partitions hashed to it is about exp(-m/n), for m
balls and n bins, so P(empty vnode) ~ exp(-partitions/vnodes). From the
coupon collector's problem, the expected minimum number of partitions
needed to ensure that no vnode is empty is about nln(n), where n =
number of vnodes. We can invert this expression to get the minimum number
of vnodes we should choose for a given number of partitions. This is not
easy https://cs-people.bu.edu/lapets/resource/nlogn.pdf, but we can
just use a table and round/interpolate for our purposes. This also tells us
how many partitions we should initially allocate. For example, if we have
an 80-node cluster with 8 workers each, we have 640 workers total, so if we
only assign 1 vnode/worker, we still need at least 640ln(640) = 4135
partitions to ensure that no workers are left empty. On the other hand, if
we have an 8-node cluster with just one worker each, we need just 8*ln(8)
= 16 partitions to ensure that no workers are left empty. For large
clusters, these partitions are much more fine-grained than we might have
chosen otherwise, which is a significant disadvantage to the randomized
approach of consistent hashing vs. my original approach of explicit
assignment of partitions to workers. I don't know of a similarly
deterministic alternative to consistent hashing that would meet our goals,
though.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/uwescience/myria/issues/851#issuecomment-244192368,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACLOtX3FRjKRXkEoCENGm6c0iFxu6pGVks5qly0IgaJpZM4JtnnU
.

I think that rendezvous hashing would be a lot simpler to implement than consistent hashing and shouldn't suffer from the problems I noted above. Basically, we would just concatenate each partition ID with each worker ID, input the result to a hash function, and assign each partition ID to the worker ID which yields the largest hash value. This has similar properties to consistent hashing: when a worker goes away, you just assign each of its partitions to the worker which yields the next largest hash value, and when a worker is added, only partitions which now maximize their hash value for this worker need to be moved.

I guess the balls-in-bins argument above will apply to any function that assigns partitions to workers uniformly at random, so the number of partitions will always be O(n log n) in the number of workers.

Interestingly, Oracle's distributed in-memory SQL database uses rendezvous hashing to assign in-memory partitions to nodes: http://www.vldb.org/pvldb/vol8/p1630-mukherjee.pdf

Random idea: could we use quasirandom sequences to place workers/vnodes on the circle rather than pseudorandom hash functions? This should give us better uniformity and hopefully avoid superlinear growth in partitions WRT workers. Example: https://en.wikipedia.org/wiki/Van_der_Corput_sequence

So I think a very simple improvement on "discrete consistent hashing" (i.e., hashing to a set of n bins rather than random points on the unit circle) would be to use a bit-reversal permutation on the space of bins. (E.g., for 16 bins, the sequence of workers mapped to bins would look like 0 8 4 12 2 10 6 14 1 9 5 13 3 11 7 15. Imagine these numbers as points arranged on a circle, and it is clear that this is just the sort of uniform spacing one wants for consistent hashing.) Since this sequence is a permutation, no collisions are possible, so the collision problem of pseudorandom functions is completely avoided. Instead of requiring n*ln(n) partitions for n workers, we simply require that workers <= partitions. (For consistent hashing, but not rendezvous hashing, we still need multiple vnodes/worker to guarantee even redistribution of data when workers are added or removed, so the conditions are really partitions >= vnodes*ln(vnodes) and  vnodes <= partitions respectively.)
A similar approach should work for rendezvous hashing. One might concatenate the bit representations of worker IDs and partition IDs and bit-reverse them, or pick a different (pseudo)random permutation of worker IDs for each partition. Again, for a fixed partition, there can be no collisions between workers since this is a permutation. (One simple way to compactly specify a pseudorandom permutation on partition IDs, randomized over worker IDs, would be to hash all partition IDs using a hash function keyed on the worker ID and sort the partition IDs by their hashes.)

OK, I think I've devised a quasirandom version of rendezvous hashing. Let PARTITIONS be the fixed number of partitions for the life of the cluster. Imagine a (conceptual) square matrix of dimensions PARTITIONS x PARTITIONS, where each column corresponds to a partition ID and each row corresponds to a worker ID. (We can have any number of workers 1 <= WORKERS <= PARTITIONS.) Now (conceptually again), map the cells of the matrix to an unrolled vector (say in row-major order), and generate the full bit-reversal permutation of that vector. Each cell of the matrix now has an integer entry corresponding to its order in that permutation. To find the worker assigned to a partition, go to that partition ID's column and look at the entries for all rows which correspond to a valid worker ID. The row index with the largest entry of all such valid rows  is the worker ID assigned to that partition.
Note that because the bit-reversal permutation is its own inverse, we do not actually need to generate the sequence. We can reconstruct the sequence number for any cell in the matrix by mapping it to its position in the unrolled vector (i.e., row * PARTITIONS + column) and bit-reversing that position (keeping in mind that all positions are padded to 2 * log(PARTITIONS) bits).
I think that this should give us excellent uniformity (avoiding the pathological collision behavior we get from pseudorandom functions), while being fully deterministic and cheap to calculate. Like the original rendezvous hashing, it nicely avoids the virtual nodes hack of consistent hashing. I need to do some simulations to verify that partition allocation across workers is as uniform as I expect.

So there's a fly in the ointment: even if we have perfectly uniform hashing with no collisions, we still need a number of partitions quadratic in the number of workers to guarantee uniform redistribution of data when a worker joins or leaves. Since we want to relocate 1/(n+1) of the data on each existing worker when a worker joins a cluster with n existing workers, and since data can only be moved as entire partitions, we need at least n+1 partitions on each worker, or at least n*(n+1) partitions in total. We can reduce this number by a constant (1/WORKERS_PER_NODE)^2, since partitions physically reside on nodes, and we care more about even distribution across nodes than across workers, but it is still quadratic. How coarse we allow our partitions to be depends on how much variance we are willing to tolerate in our data distribution. For a large value of WORKERS_PER_NODE (say >=8), setting PARTITIONS = MAX_WORKERS might be acceptable.

Um, I guess I should mention that all this fancy hashing nonsense is strictly speaking unnecessary. I could just modify my original idea of maintaining an explicit mapping between partitions and workers, to now map partition IDs instead of partitions of relations to workers. This would be in some ways simpler than a deterministic function, but it would require its own logic to redistribute partitions uniformly across workers on cluster membership changes, and it would mean we would have to query the master catalog whenever we needed to know the worker hosting all partitions with a given ID. The deterministic approaches would allow workers to determine partition assignment knowing only cluster membership, and they would be adaptable to decentralized, masterless contexts as well.

Note on my "quasirandom" version of rendezvous hashing: it doesn't work. I simulated the algorithm for row-major, column-major, and Z-order traversals of the matrix, and the uniformity of the permutation always causes a single worker to be assigned all partitions. Using hash functions instead of a permutation to rank the workers in each column yields exactly the expected results: 1/e of the workers not assigned any partitions, etc. So I guess it's consistent hashing or nothing, if we want to stick with a deterministic function. I'll be simulating the quasirandom version of consistent hashing next.

So I did some simulations of "quasirandom consistent hashing" and found some interesting behavior.

There is no benefit from using virtual nodes; it doesn't improve distribution at all. This isn't surprising in retrospect, since the bit-reversal permutation distributes all virtual nodes belonging to a worker perfectly uniformly around the circle; there is no randomness to average out imbalances across virtual nodes.
All worker loads (i.e., number of assigned partitions) are skewed by a worst-case factor of 2. If the number of workers is a power of 2, then the distribution of partitions is perfectly uniform; otherwise it is bimodal between some x and 2*x, with few or no intermediate values. Again, virtual nodes don't seem to help smooth out this distribution because they're distributed too uniformly around the circle (it would be easiest to demonstrate this at the whiteboard). So the good news is that skew is tightly (and deterministically) bounded; the bad news is that it is always a factor of 2.

Given these drawbacks, we should consider whether we really need a deterministic, stateless mapping function from partitions to workers, or whether we're willing to live with an explicit mapping which can guarantee the most uniform possible distribution of partitions across workers (the logic could be as simple as redistributing one partition to/from each worker in worker ID order on each pass, with as many passes as necessary). If we want "coarse-grained partitions" (i.e., the number of partitions is of order the number of workers), then I don't think we can use a pseudorandom approach like classical consistent hashing (for reasons I described earlier), and "quasirandom consistent hashing" is the best stateless approach I can think of so far that is compatible with coarse-grained partitions.

I found a couple more variations on consistent hashing but haven't had time to read them yet, so I don't know if they are relevant to our "coarse-grained, sequentially numbered partition" scenario.
http://arxiv.org/pdf/1406.2294v1.pdf
https://arxiv.org/pdf/1503.04988v2.pdf

I simulated "jump consistent hashing" (http://arxiv.org/pdf/1406.2294v1.pdf) and the distribution was pretty terrible--not surprising since it's a pseudorandom algorithm and hence works well only for very fine-grained entities, i.e keys, not partitions.
./jump_consistent_hashing.py partitions=1024 workers=640
OWNED_PARTITIONS NUM_WORKERS
---------------- -----------
0                126
1                223
2                149
3                89
4                34
5                14
6                5

We might be able to mitigate partition skew somewhat for multitenant clusters by specializing the hash function per namespace: generate a random bitmask by hashing user + program and XOR that mask with the partition ID before applying the permutation. To invert the transformation, apply the permutation to the hashed partition ID and XOR the result with the mask. This way, each user + program namespace gets its own partition->worker hash function (assuming of course that users never need to join relations across namespaces).

Given the shortcomings of all the stateless approaches to partition assignment we've looked at, we've decided to go with the original approach: an explicit mapping of partition IDs to worker IDs.

Rereading the Dynamo paper, I noticed that there's extensive discussion of the partitioning optimization for consistent hashing in section 6.2. We should cite this as prior work. (The difference from our approach is that they assume a number of partitions much larger than the number of nodes, which is necessary for their randomized approach of assigning partitions to nodes to work.)


Copilot setup steps break AWS CLI Auth

The microservices set up commands for copilot have you set up your .aws/config file like:
mkdir -p ~/.aws
cat << EOF > ~/.aws/config
[default]
region = ${AWS_DEFAULT_REGION}
output = json
role_arn = $(aws iam get-role --role-name ecsworkshop-admin | jq -r .Role.Arn)
credential_source = Ec2InstanceMetadata
EOF

After I ran this, I was unable to authenticate with the AWS CLI, e.g. aws-sts get-caller-identity would return:
An error occurred (AccessDenied) when calling the AssumeRole operation: User: arn:aws:sts::12345:assumed-role/ecsworkshop-admin/i-098765 is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::12345:role/ecsworkshop-admin

Seems like the role is trying to assume itself. However, aws sts get-caller-identity command worked fine on earlier parts of the workshop, and runs fine without the .aws/config file.
My guess is this file doesn't need to exist because the Role is already assigned to the whole EC2 instance. I'd make a PR here, but I'm now using the CDK path instead of the Copilot path, and the equivalent CDK snippet does not have these lines. So I'm not sure if the Copilot path would work with these lines removed, or if it would have other issues.

PR #122 to fix this


Use maintained gziphandler

https://github.com/NYTimes/gziphandler doesn't seems to be maintained anymore and the code already includes
https://github.com/klauspost/compress which provides a maintained and faster gzip handler fork.
Enables Jitter to prevent BREACH attacks, as this seems to be possible in the context of the server.

(cherry picked from commit 45a16f8c3c6f7d5e4aab8fdde6a621cf36e4801c)

The library's License looks like some complicated.

The library's License looks like some complicated.

It looks like the gzhttp package simply uses its own apache2 license vs the base module also including the Go authors.

The removed license should be equal to the added one because the new code is a fork of the replaced code.


Change of IRIs without referencing old ones

Some license IRIs were changed without referencing the old ones.
Please add at least owl:sameAs or rdfs:seeAlso.

owl:sameAs relations are added for the changed license IRIs.

owl:sameAs relations are added for the changed license IRIs.


Add Video viewing quality Button to LBRY.tv video player

Add Video viewing Quality Button to LBRY.tv video player
similar to the 144p 240p 360p 480p 720p 1080p and 4k video viewing quality button on youtube.
Bandwidth is terrible for users with slow internet with lbry.tv defaulting to 1080p on 1080p videos without no way to change it.

Thanks for filing this issue! At the moment, LBRY does not post process any videos so we can't provide this feature anytime soon. We also hope to fix some of the stuttering issues by deploying more services across the globe.
It's definitely something we are thinking about, so check back later next year to see if we have any updates.

@tzarebczan I think this is worth leaving open, though it's more of a lbrytv issue than a desktop one.
Since all web data requests are proxied, we can offer dynamic and adjustable bit rates.


What do you think of adding logEvery(message, interval, timeunits, ..) method?

I find in many occasions I need to re-implement some logging throttling, wouldn't it be nice to have it already existing in our logging framework?

I'm not familiar with the term logging throttling. From the method signature, it seems you want to use one statement to log something periodically?
That would mean quite a change in philosophy. Now it's just a really thin layer on top of slf4j, it doesn't have any logging functionality on its own. I don't see how this would fit in there.
Also, I'm not sure that use case is really that common.


Mainfest reproduce options don't seem to do what they say on the box

The Payu docs say that with the manifest:reproduce:exe flag set in the config.yaml Payu "will refuse to run if hashes do not match". This does not seem to happen. Instead Payu silently runs using the exectuables specified in the manifest.
Probsibly the same is true for manifest:reproduce:input and manifest:reproduce:restart - I haven't tested.
To reproduce
Clone one of the access-om2-configs release branches (which all have manifest:reproduce:exe = True), modify an exectuable path in config.yaml, and run payu setup or payu run. Payu will behave as if no modifications were made to the config.yaml.

I agree this is bad. I think it's my fault, and weirdly it was a conscious decision, but it does not match the documentation.
I was sure there was an existing issue for it, but I looked and couldn't find anything, so thanks for making one.


Bug: Fanfares break Custom Music

Title is self-explanatory. Fanfares in both games kill Custom Music due to the audio engine getting overloaded, I think. Owlisnotacat has a PR for OOTR that fixes this issue here: https://github.com/OoTRandomizer/OoT-Randomizer/pull/2050
Although, unsure if it fixes it for vanilla fanfares or not.
I also don't know what MMR does to prevent fanfares killing music in MM but for 1.16 Zoey added an option to just disable fanfares completely and use some kind of minor-pick-up sound effect fanfare instead. Perhaps something we will want to add as an option instead? Or just add the option to disable Fanfares for both games anyway? And apply the same sound effect seen in MMR 1.16 (when it's out of beta).

Should be fixed (not 100%), reopen if issue persists


Unable to build after uploading the addon with Collaborate to second computer

Since the files are in the assets folder, they automatically sync with Unity Collaborate. It seemed to work fine, but when I wanted to build the game it couldn't and I got a "Error building Player because scripts had compiler errors" error but not in any script, just in general. Deleting the addon fixed this issue. I would really like it if this addon wasn't in the assets folder so that it wouldn't sync.

I would like to update this post by saying that I now can't even build on the first computer. I get the same error so I have had to uninstall the addon completely. I really like the customization in this so I would really appreciate if this issue was fixed so that you can actually use it. The UI also gets pretty slow while using this, but I presume that you already know that or that you can't fix that.

Sorry for the late response, Zufixx.  Build issues should be fixed if you are using the latest version, but if not, check for this solution https://github.com/zios/unity-themes/issues/8.
The UI shouldn't be too slow (in general) if you have hover response set to none.  It is a little clunky on load and while transitioning between play/edit mode, of course, due to the system reloading.
I'd love to be able to decouple the required assets for the system, but as of right now they need to be loaded into the project's asset database in the editor for me to use them properly.  The best solution I could do is some kind of disclusion flag/setting to prevent them from being in builds/collab or centralizing them cross-project in a neutral directory via a symlink.  The latter introduces some per-platform complications and still exposes the files in search per-project, however.

Does the latest release resolve your issue?


Rework a bit rhol_crc

There were multiple issues in the role:

no version check, meaning it would consume whatever version it finds
on the disk instead of ensuring it's consuming the target version
lack of molecule tests for some specific and sensitive parts

This change also moves the "crc" binary to a more conventional location
(~/bin) to allow an easier discovery, and avoiding potential conflicts.
It also removes some wrong/useless parameters that could lead to weird
situations regarding rights, users and so on.
This PR also bumps the CRC version in both role and molecule (this
creates a conflict with another PR, #2102).
As a pull request owner and reviewers, I checked that:

[x] Appropriate testing is done and actually running
[x] Appropriate documentation exists and/or is up-to-date:

[x] README in the role

Depends-On: https://github.com/openstack-k8s-operators/ci-framework/pull/2134

recheck

Fixes for pre-commit are in #2134

Comment addressed.

/hold
Some more testing via DS TP is requested.

/lgtm


Implement modin.pandas.DataFrame.as_blocks

Describe the problem
Currently as_blocks is not implemented.
Source code / logs
In [1]: import modin.pandas as pd
   ...: import numpy as np
   ...:
   ...: frame_data = np.random.randint(0, 100, size=(2**12, 2**8))
   ...: df = pd.DataFrame(frame_data)
   ...:
   ...:
Process STDOUT and STDERR is being redirected to /tmp/raylogs/.
Waiting for redis server at <IP_ADDRESS>:59321 to respond...
Waiting for redis server at <IP_ADDRESS>:36048 to respond...
Starting the Plasma object store with 27.00 GB memory.
Starting local scheduler with the following resources: {'GPU': 0, 'CPU': 8}.

======================================================================
View the web UI at http://localhost:8888/notebooks/ray_ui66732.ipynb?token=f0ef48a507f20daa610b2bd8da93e7e2f62955bec34e6bcb
======================================================================

In [2]: df.as_blocks()
---------------------------------------------------------------------------
NotImplementedError                       Traceback (most recent call last)
<ipython-input-2-0a8fd5d307e9> in <module>()
----> 1 df.as_blocks()

~/anaconda/lib/python3.5/site-packages/modin/pandas/dataframe.py in as_blocks(self, copy)
   1242     def as_blocks(self, copy=True):
   1243         raise NotImplementedError(
-> 1244             "To contribute to Pandas on Ray, please visit "
   1245             "github.com/modin-project/modin.")
   1246

NotImplementedError: To contribute to Pandas on Ray, please visit github.com/modin-project/modin.

This was deprecated, we will not support deprecated functionality moving forward.


Makefile task wrapping

I want to merge this change because I think the following 3 control changes can help in development process:

It was added a make file for handling and warpping common development tasks
It was added a environment variables file (.env) helping the application for better achievement of 12-factor pattern
It was moved initial data migrations to fixtures, avoiding some common development issues when working with migrations

Examples of make tasks:
make dependencies  # Install dependencies
make bakend-start  # Start database
make migrations  # Apply migrations
make fixtures  # Apply fixtures (it was needed to change from data migration to fixtures)
make tests  # Run tests
make deploy  # From 0 to up and running + data

Example of environment file (.env.sample)
Screenshot/Video
Asciinema
Pull Request Checklist

[] Privileged views and APIs are guarded by proper permission checks.
[] All visible strings are translated with proper context.
[] All data-formatting is locale-aware (dates, numbers, and so on).
[] Database queries are optimized and the number of queries is constant.
[] Database migration files are up to date.
[] The changes are tested.
[] GraphQL schema and type definitions are up to date.
[] Changes are mentioned in the changelog.

Hey @marquicus, thanks for opening the pull request! I'm not sure if we want to remove the migrations now. We're actively using them for development; also, projects based on Saleor that gets regular updates need to perform these migrations whenever we make any changes to models.
We will need to tackle the issue of migrations at some point since we have a lot of them and the number is growing, but it's something that we don't want to do yet. Anyway, thank you for your contribution!