workerpool: add ability to stop new work from queuing

TL;DR
Developers should be able to stop execution of future queued work.
Detailed design
We could possibly return a sentinel error or just stop the worker.

Alternatives considered
No response
Additional information
No response

@dcreey I see a few design options here:
context-based
This is somewhat how contexts are intended to be used:
ctx, cancel := context.WithCancel(context.Background())
defer cancel()

pool.Do(ctx, func() (string, error) {
  result, err := something()
  if err != nil {
    defer cancel() // stop future invocations
    return err
  }
})

With the current API, this will work, but each future call to Do will return ContextCancelled.
stop-based
This also works today:
pool.Do(ctx, func() (string, error) {
  result, err := something()
  if err != nil {
    defer pool.Done(context.Background()) // stop future invocations
    return err
  }
})

With the current API, this will work, but each future call to Do will return ErrStopped.
Wrapped concrete error
This one feels the least intuitive and too magical for me:
pool.Do(ctx, func() (string, error) {
  result, err := something()
  if err != nil {
    return fmt.Errorf("%w: failed to thing: %w", ErrHalt, err)
  }
})

where ErrHalt is a magical sentinel error.
Other
I don't want a generic configurable like "StopOnError" because it's too coarse. You might want to stop on certain errors but continue on others. Then you start down the slippery slope of allowlist/denylist errors, and it feels easier just to require a specific call in the function body to terminate.

@drevell for his thoughts too

My non-binding thoughts:
Thought number 1

These two use cases might be different enough to have separate APIs:

Case 1: "Run indefinitely with error tolerance," like a server.
Case 2: "I have a finite list of things to do, like HTTP gets, and want it to be fast so I'm doing it concurrently." Batch stuff.

For use case 1, context cancellation (or pool.Done()) makes good sense to me.
For use case 2, context cancellation (or pool.Done()) would work but it seems to me like it doesn't hit the API design sweet spot of "there's a single obvious way to do it." You have to know about the special errors returned from pool.Do(). You have to gather your own errors using a channel or something, without any common channel mistakes like accidentally blocking something important. You have to have read the docs about context cancellation.
For case 2, maybe we can provide a separate pool type that has a DoUntilError method that takes an ErrWorker func(T) error? This could be built on top of the other pool, I think.
Thought number 2
Maybe we're swimming against the tide here by not using sourcegraph/conc, even though it's not ideal.
Thought number 3

You might want to stop on certain errors but continue on others

There could be a config option with a callback like IsHaltableError(error) bool. Still a bit clunky though.

Piggy backing off of what @drevell said.
Thought number 1
I agree with Dave here. Its two explicit use cases. We can support both with a single api but it means greater complexity for the author, reviewer, and future reviewer.
Thought number 2
I prefer this over the context cancel since context cancel might not cancel all work if you have multiple contexts
Thought number 3
Too much complexity IMO.
Thought number 4 (other)
Honestly I think I like this one the most. The caller already has the option of whether or not to return an error.  And in fact, they could wrap their func with another func that handles retry/ignore. Seems fairly straightforward and leaves the user thinking about their application logic instead of the pool logic.
Additionally, I think this solves for the majority of the batch use case. The use case where someone is doing batch and wants to ignore some errors and do nothing (return the error but not retry and not cancel) seems more like an edge case. And in this scenario they can always fallback to using approach 1 or 2.
I'm a big fan of less LOC and less for the reader to think about. Forcing the user to cancel on error means that the user needs to consider the intent of the cancel - is this a special type of error we are cancelling on or are we just trying to cancel generally? In the latter case did the author remember to cancel correctly in all funcs?

This was fixed in https://github.com/abcxyz/pkg/pull/156


Add documentation

Ideally, compile with #![deny(missing_docs)]

Ok, thanks, will look into this!

Added a first, rough part of documentation in https://github.com/soc/directories-rs/commit/d744387ed3b0d4f05bc8440aab7272848d87db97?diff=unified

Status: some documentation has been added, but more is required.

Everything public should be implemented now in https://github.com/soc/directories-rs/pull/15.
Please reopen if something is missing!


Hide Accrescent from the Installed and Apps Tab

This information should be displayed in the settings instead

How come?

One thing I think should be done is to remove "open" button for Accrescent which is obviously already opened.

I agree, although that would be separate from this issue.

Correct me if I'm wrong, but F-Droid still appears in the F-Droid app itself as a listing, as does Play Store on Play Store.

Right, I checked again later and it seems to be common practice in most app stores (the Google Playstore has its update function separately in the settings).
I just thought that it would make sense if Accrescent is not in a list of apps that are downloadable via Accrescent, to keep it separate when there are hundreds to thousands of apps later.
Translated with DeepL https://www.deepl.com/app/?utm_source=android&utm_medium=app&utm_campaign=share-translation

I'm going to close this for now since I don't see a concrete reason Accrescent shouldn't be listed and there are some benefits to listing it as @matchboxbananasynergy explained. I will disable the "open" button for Accrescent as @PatrykMis suggested though. If a specific reason for hiding Accrescent comes up, feel free to continue this thread.


[topic-gpio] drivers: gpio_stm32: update to use new GPIO API

Update driver code to use new configuration flags such as GPIO_ACTIVE_LOW.
STM32F1 series not taken into account.
Only one board converted.
No backward compatibility: gpio_configure don't configure interrupts anymore
Based on https://github.com/zephyrproject-rtos/zephyr/pull/19256 as new API assumes possibility to disabling and re-enabling interrupts
TODO:

[ ]  So far STM32F1 series are not taken into account. Extend to change to this series
[ ] Get all STM32 based boards converted to the new API

@mniestroj, FYI

Tested ok on gpio_basic_api but FATAL error when test is run w/o wire.
So this requires some more analysis.

Tested ok on gpio_basic_api but FATAL error when test is run w/o wire.
So this requires some more analysis.

Seems to be intended behavior.

Tested ok on gpio_basic_api but FATAL error when test is run w/o wire.
So this requires some more analysis.

Seems to be intended behavior.

Yeah, seen the same thing myself

[ ]  Get all STM32 based boards converted to the new API

While maybe not the most efficient, here's an example to help convert the dts files over:
git grep -l \<st boards/arm | grep dts | xargs sed -i 's/GPIO_INT_ACTIVE_HIGH/GPIO_ACTIVE_LOW/g'

Marking dnm since there are some todo items.

rechek

recheck

@mnkp Thanks for review!

It makes it difficult to do the review since both PRs modify driver code.

This should not be the case PR #19067  only adds commit 717de74d7e2a91c3222f8d03979cfd93936bfebc which impact is limited to boards

Also by not running checks on all the boards we may miss some issues. In case of Shippable timeout it's better to add a temporary patch from @galak that fixes the issue.

Agree it's better to actually get the results on all boards. @galak how can we make this happen?

@erwango As suggested to me by @carlescufi:
From e888dbf355ca8a63ec17ed725368ebb2ec79a200 Mon Sep 17 00:00:00 2001
From: Peter Bigot<EMAIL_ADDRESS>Date: Sun, 6 Oct 2019 15:05:28 -0500
Subject: [PATCH] DNM shippable.yml: increase builds to avoid timeout failures

Signed-off-by: Peter Bigot<EMAIL_ADDRESS>---
 .shippable.yml | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/.shippable.yml b/.shippable.yml
index 605fd41be8..b50342a5ca 100644
--- a/.shippable.yml
+++ b/.shippable.yml
@@ -6,13 +6,20 @@ env:
     global:
         - ZEPHYR_SDK_INSTALL_DIR=/opt/sdk/zephyr-sdk-0.10.3
         - ZEPHYR_TOOLCHAIN_VARIANT=zephyr
-        - MATRIX_BUILDS="5"
+        - MATRIX_BUILDS="12"
     matrix:
         - MATRIX_BUILD="1"
         - MATRIX_BUILD="2"
         - MATRIX_BUILD="3"
         - MATRIX_BUILD="4"
         - MATRIX_BUILD="5"
+        - MATRIX_BUILD="6"
+        - MATRIX_BUILD="7"
+        - MATRIX_BUILD="8"
+        - MATRIX_BUILD="9"
+        - MATRIX_BUILD="10"
+        - MATRIX_BUILD="11"
+        - MATRIX_BUILD="12"

 build:
     cache: false
--
2.20.1

Also https://github.com/zephyrproject-rtos/zephyr/pull/19562#issuecomment-538075156

@pabigot

@erwango As suggested to me by @carlescufi:

Thanks, will use that.

@mnkp, @pabigot, @ydamigos, @avisconti,
Thanks for your reviews.
Pushed a new version of the work. I think I've addressed most of the remarks related to the drivers.
Also I squashed 2 PR (driver and gpio) to get everything at one place and add a patch to help building all w/o timeout in shippable (so I will close https://github.com/zephyrproject-rtos/zephyr/pull/19607)
What I didn't do is to test, so I plan to do that tomorrow.

Pushed a new version of the work. I think I've addressed most of the remarks related to the drivers.

The new version doesn't seem to implement any changes to gpio_stm32.c driver requested during the review. Maybe it was not the latest version that got pushed?

Indeed, odd, I'll have a look.

* `gpio_api_1pin`: Failed all tested boards so far. Investigating this is my next task

There is one issue I've noticed just recently that may cause 1 pin testcase to fail. The testcases testing the new port functions attempt to verify that the driver doesn't change other pins than the one that was requested. But the test doesn't take into account that the LED0 pin it's using for testing may be on the same port as UART used for the console. When reading port values the pin serving as UART RX will introduce inconsistency to the port value the testcase is expecting. If you see failures in any of test_gpio_port_* tests please ignore it for now. I'll provide a fix.

* `button`: Some boards button definitions are wrong.

The existing definitions in .dts files were unfortunately not always correct. It's best to fill in board DTS based on the board user manual / schematic.

* `gpio_api_1pin`: Failed all tested boards so far. Investigating this is my next task

There is one issue I've noticed just recently that may cause 1 pin testcase to fail. The testcases testing the new port functions attempt to verify that the driver doesn't change other pins than the one that was requested. But the test doesn't take into account that the LED0 pin it's using for testing may be on the same port as UART used for the console. When reading port values the pin serving as UART RX will introduce inconsistency to the port value the testcase is expecting. If you see failures in any of test_gpio_port_* tests please ignore it for now. I'll provide a fix.

Ok, thanks.

* `button`: Some boards button definitions are wrong.

The existing definitions in .dts files were unfortunately not always correct. It's best to fill in board DTS based on the board user manual / schematic.

Indeed, this wasn't clear, but this is in my todo list as well.

gpio_api_1pin: Failed all tested boards so far. Investigating this is my next task

Failed tests (when no conflict with other pins of the same port) are the following:

test_gpio_pin_configure_single_ended
test_gpio_int_edge_falling
test_gpio_int_edge_to_active
test_gpio_int_edge_to_inactive

Pushed new version with fixes on tests failing in previous version .
Tested on following series: F0/F1/F3/F4/F7/G0/L0/L1/L4/WB
Remain to be tested:

F2: Need to find a board. Should be doable tomorrow.
G4: Supported in master but not available on topic-gpio yet
L1: Only board available is 96b_wistrio, which I don't have. @Mani-Sadhasivam, can you perform tests on this board?

Test status:

gpio_basic_api: Passed on all available boards except stm32h747_disco_m7, cf below
gpio_api_1pin: Passed on available boards except stm32h747_disco_m7, cf below
button: Passed on all available boards

Test failures on stm32h747_disco_m7:

gpio_basic_api:

starting test - test_gpio_port
Validate device GPIOJ
Check GPIOJ output 9 connected to input 8
OUT 9 to IN 8 linkage works
- bits_physical
- pin_physical
- check_raw_output_levels
- check_input_levels
- bits_logical
- check_pulls

    Assertion failed at ../../src/test_gpio_port.c:389: check_pulls: (raw_in() not equal to false)
physical pull-down does not read low
FAIL - test_gpio_port

gpio_api_1pin:

starting test - test_gpio_pin_configure_push_pull
Running test on port=GPIOI, pin=12

    Assertion failed at ../../src/test_config.c:32: pin_get_raw_and_verify: (val_expected not equal to val_)
Test point 10: invalid pin get value
FAIL - test_gpio_pin_configure_push_pull

Pushed new version taking into account last review from @nmkp and testing on F2 and MP1.
Tested on following series: F0/F1/F3/F4/F7/G0/H7/L0/L4/MP1/WB
Remain to be tested:

G4: Supported in master but not available on topic-gpio yet
L1: Only board available is 96b_wistrio, which I don't have. @Mani-Sadhasivam, can you perform tests on this board?

Test status:

gpio_basic_api: Passed on all available boards except stm32h747_disco_m7, cf below
gpio_api_1pin: Passed on available boards except stm32h747_disco_m7, and stm32mp157c_dk2 cf below
button: Passed on all available boards

Test failures on stm32h747_disco_m7:

gpio_basic_api:

starting test - test_gpio_port
Validate device GPIOJ
Check GPIOJ output 9 connected to input 8
OUT 9 to IN 8 linkage works
- bits_physical
- pin_physical
- check_raw_output_levels
- check_input_levels
- bits_logical
- check_pulls

    Assertion failed at ../../src/test_gpio_port.c:389: check_pulls: (raw_in() not equal to false)
physical pull-down does not read low
FAIL - test_gpio_port

Playing with other boards, I've seen several times that this specific failure can happen when pin selected is in conflict with other usage. I've not been able to identify the conflict yet, but I heavily suspsect this issue is purely a configuration issue.

gpio_api_1pin:

starting test - test_gpio_pin_configure_push_pull
Running test on port=GPIOI, pin=12

    Assertion failed at ../../src/test_config.c:32: pin_get_raw_and_verify: (val_expected not equal to val_)
Test point 10: invalid pin get value
FAIL - test_gpio_pin_configure_push_pull

Can be PASSED if changing pin.
Test failures on stm32mp157c_dk2:

gpio_api_1pin: Failing point relates to port testing. Due to use of Linux on Cortex-A, lot of pins are configured for various purposes (ethernet, display, ...), there is no empty port available for testing matching pins available on arduino connector. I'll have a new try once #19692 is fixed.

@erwango Oops. I see couple of failures by executing the gpio_basic_api test. Earlier I just tested the button sample manually.
PASS - test_gpio_pin_read_write
===================================================================
starting test - test_gpio_callback_edge_high
callback triggered: 1
check enabled callback
PASS - test_gpio_callback_edge_high
===================================================================
starting test - test_gpio_callback_edge_low
FAIL - test_callback@68. config PIN_IN fail
    Assertion failed at ZEPHYR_BASE/tests/drivers/gpio/gpio_basic_api/src/test_callback_te

FAIL - test_gpio_callback_edge_low
===================================================================
starting test - test_gpio_callback_level_high
FAIL - test_callback@68. config PIN_IN fail
    Assertion failed at ZEPHYR_BASE/tests/drivers/gpio/gpio_basic_api/src/test_callback_te

FAIL - test_gpio_callback_level_high
===================================================================
starting test - test_gpio_callback_add_remove
callback_2 triggered: 1
callback_1 triggered: 1
callback_2 triggered: 1
PASS - test_gpio_callback_add_remove
===================================================================
starting test - test_gpio_callback_self_remove
callback_remove_self triggered: 1
callback_1 triggered: 1
callback_1 triggered: 1
PASS - test_gpio_callback_self_remove
===================================================================
starting test - test_gpio_callback_enable_disable
callback_2 triggered: 1
callback_1 triggered: 1
callback_2 triggered: 1
callback_1 triggered: 1
PASS - test_gpio_callback_enable_disable
===================================================================
starting test - test_gpio_callback_level_low
FAIL - test_callback@68. config PIN_IN fail
    Assertion failed at ZEPHYR_BASE/tests/drivers/gpio/gpio_basic_api/src/test_callback_te

FAIL - test_gpio_callback_level_low
===================================================================
Test suite gpio_basic_test failed.
===================================================================
PROJECT EXECUTION FAILED

@Mani-Sadhasivam, Thanks for this status. Can you check the port of the pins used in when testing gpio_baisc_api do not concflict port of pins used for other activity (like UART)?
Also, you can run new test gpio_api_1pin, which is also under tests/drivers/gpio

@erwango

Can you check the port of the pins used in when testing gpio_baisc_api do not concflict port of pins used for other activity (like UART)?

Yes, these are dedicated GPIO pins. So there is no conflict.

Also, you can run new test gpio_api_1pin, which is also under tests/drivers/gpio

For this test, only one testcase failed:
starting test - test_gpio_port_set_masked_get_active_low
Running test on port=GPIOA, pin=12
Step 1: Set logical, get logical and physical port value

    Assertion failed at ZEPHYR_BASE/tests/drivers/gpio/gpio_api_1pin/src/test_port.c:41: )
Test point 0: invalid logical port get value
FAIL - test_gpio_port_set_masked_get_active_low

@erwango

Can you check the port of the pins used in when testing gpio_baisc_api do not concflict port of pins used for other activity (like UART)?

Yes, these are dedicated GPIO pins. So there is no conflict.

Out of curiosity, which pins did you used?
Then, would you be able to analyze a bit the failure reason ?

Also, you can run new test gpio_api_1pin, which is also under tests/drivers/gpio

For this test, only one testcase failed:
starting test - test_gpio_port_set_masked_get_active_low
Running test on port=GPIOA, pin=12
Step 1: Set logical, get logical and physical port value

    Assertion failed at ZEPHYR_BASE/tests/drivers/gpio/gpio_api_1pin/src/test_port.c:41: )
Test point 0: invalid logical port get value
FAIL - test_gpio_port_set_masked_get_active_low

Tested pin is on GPIOA, which is same port as used by UART.
Any way you can test on other port than GPIOA (even if there is no led attached), to see if status changed.

@pabigot, @mnkp, shippable still times out despite the patch (even further increased up to 15 jobs). I think this is due to the fact I'm including commits that impact a lot of boards. But I also pushed these commits in master branch and they are now merged.
Anyway we can rebase topic-gpio branch on recent master, so I can get rid of these commits in current PR and get more chances to get shippable completing the testing?

@erwango I think something was messed up in my setup. Now, the gpio_basic_test succeeds but gpio_api_1pin_test still fails.

Any way you can test on other port than GPIOA (even if there is no led attached), to see if status changed.

Tested PORTB pin 12, but same failure.
starting test - test_gpio_port_set_masked_get_active_low
Running test on port=GPIOB, pin=12
Step 1: Set logical, get logical and physical port value

    Assertion failed at ZEPHYR_BASE/tests/drivers/gpio/gpio_api_1pin/src/test_port.c:41: )
Test point 0: invalid logical port get value
FAIL - test_gpio_port_set_masked_get_active_low

@erwango Looks like the failures are due to the pins owned by other peripherals as you said. So all good on the L1x side.

@erwango Looks like the failures are due to the pins owned by other peripherals as you said. So all good on the L1x side.

Is that true for gpio_basic_api as well?  gpio_api_1pin is subject to false negatives due to other pins, but gpio_basic_api is not known to have any such problem: it only uses the two pins specified in the board overlay (which must be correctly pinmux-configured as GPIO).

@pabigot, @mnkp, shippable still times out despite the patch (even further increased up to 15 jobs). I think this is due to the fact I'm including commits that impact a lot of boards. But I also pushed these commits in master branch and they are now merged.
Anyway we can rebase topic-gpio branch on recent master, so I can get rid of these commits in current PR and get more chances to get shippable completing the testing?

I don't have permission to push to this branch.  I have done a rebase (which required edits related to timeout parameters in the 2-pin test), available at https://github.com/pabigot/zephyr/commits/topic-gpio, if @carlescufi or somebody with maintainer permissions wishes to update.
tirzah[756]$ git push --force-with-lease=topic-gpio:r.upstream/topic-gpio
Enumerating objects: 526, done.
Counting objects: 100% (526/526), done.
Delta compression using up to 32 threads
Compressing objects: 100% (303/303), done.
Writing objects: 100% (398/398), 74.00 KiB | 7.40 MiB/s, done.
Total 398 (delta 329), reused 114 (delta 90)
remote: Resolving deltas: 100% (329/329), completed with 115 local objects.
remote: error: GH006: Protected branch update failed for refs/heads/topic-gpio.
remote: error: Cannot force-push to a protected branch
To github.com:/zephyrproject-rtos/zephyr
 ! [remote rejected]       topic-gpio -> topic-gpio (protected branch hook declined)
error: failed to push some refs to 'git@github.com:/zephyrproject-rtos/zephyr'

@erwango
The topic branch has been rebased. Could you please rebase this PR against it?

@pabigot, @mnkp: shippable passed w/o boards modifications (cf https://github.com/zephyrproject-rtos/zephyr/pull/19764).

Is the DNM label still applicable?

Is the DNM label still applicable?

I still need to test on G4, but I should be able to do it in the afternoon, otherwise I think we can remove it.

Status following last push:
Tested on all STM32 series.
Test status:

gpio_basic_api: Passed on all available boards except stm32h747_disco_m7, cf below
gpio_api_1pin: Passed on available boards except stm32h747_disco_m7, and stm32mp157c_dk2 cf below
button: Passed on all available boards

Test failures on stm32h747_disco_m7:

gpio_basic_api:
Playing with other boards, I've seen several times that this specific failure can happen when pin selected is in conflict with other usage. I've not been able to identify the conflict yet, but I heavily suspsect this issue is purely a configuration issue.
gpio_api_1pin:
Can be PASSED if changing pin.

STM32H7 port is still very basic for now (only GPIO, UART) and globally a wip.
Given this status and board complexity that might explain itself tests failures, I think it is safe to enable new GPIO API on this series.
Test failures on stm32mp157c_dk2:

gpio_api_1pin: Failing point relates to port testing. Due to use of Linux on Cortex-A, lot of pins are configured for various purposes (ethernet, display, ...), there is no empty port available for testing matching pins available on arduino connector. I'll have a new try once #19692 is fixed.

Given board complexity board complexity that might explain tests failures, I think it is safe to enable new GPIO API on this series.
Summary
Unless reviewers comment, I consider the work done on this PR.

@erwango could you please rebase?

@erwango could you please rebase?

@carlescufi, done.


Example pictures have all rights reserved?

Examples
One example from README.md: link
Using google's reverse image search I've found which looks like original picture from flickr (as stated in the README.md - originals are from flickr): flickr_link
There are more:
From README.md: link
From flickr.com: flickr_link
I have not searched more.
Problem
Both examples have All rights reserved for license.
In photography (or any other creative activity) just as in software programming, there are licenses which must be respected even if you can break them easily (how can let flickr download images with all rights reserved this easily - is another topic and mystery to me).
As an amateur photographer I have seen this a few times. People from different industries forget that other creative work is under some license as well and it's not always allowed to modify or share work without attribution or informing the original author.
Confusion
@fogleman, maybe you've asked for permission from authors to use these photos? Maybe there are originals somewhere, which are under public domain or creative commons? Or maybe google found some stolen content? Then I am sorry for my confusion.
Solutions
@fogleman - you have a very nice collection of open source software projects, where some of them I am following and/or using, though if the problem is true, probably the best solution would be to find other pictures on flickr which are allowed to be modified[0] (their search allows to filter by license) and use them as examples. Also, not forget to mention the original author and the original picture. It would be just very good practice to do so even if license does not enforces this. Example section would be much nicer as well, as we could compare original images with transformed ones.
macOS app
I have not done reverse search, but are all images in https://primitive.lol/ have licenses for sharing modified work?
[0] This is important - not all CC images are allowed to be modified.

I got permission for photos used in their original form:

I think an argument could be made regarding the other images. If I look at a picture and draw it, is that okay? That's essentially what the algorithm is doing. Sometimes it becomes recognizable, but I'm not sure I would call it a "modification".
The twitter bot does link to the originals, btw.

If you got the permission then it's a non-issue. :)

I think an argument could be made regarding the other images. If I look at a picture and draw it, is that okay? That's essentially what the algorithm is doing. Sometimes it becomes recognizable, but I'm not sure I would call it a "modification".

Probably depends on how you look at this. In one way software creates new image using the original as an example. On the other hand, if you look at the result picture it looks like a modification of the original, like some kind of 'geometric shape' or 'thick artistic brush' filter.


allow SDP ptime variable to accept float values

Why did you close this PR?

I meant to open a PR on a forked version but somehow managed to open one here by accident

Cool. Will you contribute here later?

If I find issues or potential additions I'll be sure to bring it up here. I can reopen this PR if you'd like, not sure if you'd want this change though :)

Well, I don't know if I'm interested in this feature if you don't provide a PR here :)
Honestly, I expect to receive something for a project that I provide for free, specially improvements.


Fixed a tiny typo in cli-intro.md

Studying this fantastic Rocket Pool documentation and came across a tiny typo. ⭐

Ah, good catch! Thank you!


terraform 0.6.11

Thank you for the contribution. It was merged directly as be7d76804312b0649ab72e983bbb4a072c674810 to keep commit history cleaner. Your contribution is still credited to you.


Mouse Key Remap

Description of the new feature / enhancement
Same functionality of remapping keyboard keys but on a mouse (ie. Middle Click, Button 4) would be an awesome addition to mouse utilities section.
Scenario when this would be used?
For adding shortcuts to extra mouse buttons or when a key is malfunctioning this enables mapping another key temporarily until the problem is fixed.
Supporting information
X-Mouse Button Control does this job to some extent but it would be nice if this feature was integrated into PowerToys.

Duplicate of #11578

Thanks for your suggestion! This issue is a duplicate.
I'm linking this issue to #11578 to centralize the discussion thread, please go to that thread for discussion.
Thanks for your help to make PowerToys a better piece of software!


Update hub-planner-tutorial.md

Hub Planner docs updated

Identifier was wrong. Should be https://app.hubplanner.com/sso/metadata

Hub Planner settings updated to reflect current UI.

Close/reopen for labels and automation

@fikatown : Thanks for your contribution! The author(s) have been notified to review your proposed change.

@v-nagta and @chetansriv can you please park this app for OneClickSSO?

#sign-off

@jeevansd @v-nagta - The update in this pull request includes a URL reference to an image:

One of the quality criteria for the repository is that images that are referenced in MD files must reside in the repo. Image (and article) file names can include only numbers, lowercase letters, and hyphens (no underscores), including the file extension.
Could you work with @fikatown to convert the URL image reference to a screenshot image file that can be added to the PR and referenced in the MD file? You might need to create a /media/ folder and convey that location information to the contributor (I don't see an existing /media/ folder for this article).
Thanks.

@fikatown @v-nagta @jeevansd The PR review team will move this commit to the private repo and create a screenshot file based on the link that's provided in the PR. When the new PR is open, we'll post the link here and then close this PR.

Moved to pivate repo https://github.com/MicrosoftDocs/azure-docs-pr/pull/120656. We moved the commit to the private repo so we can ensure that the update meets all PR review quality criteria. We'll close this PR now.

@fikatown This update has been merged. I would expect to see it on the live site around midday tomorrow.


Missing Biome Tint impacts leaves color with mod - SnowyLeavesPlus

When utilizing the SnowyLeavesPlus mod, it's supposed to change the leaves color from green to white however it is done utilizing biome tint. Since this pack has the biome tint removed in order to maintain the color, it results in this mod not being able to work. May result in other visual issues with other similar mods.

Reinstated biome tint in the leaves.json file and now works with the SnowyLeavesMod. Jungle and Cherry leaves will not work with this mod when having those trees in a snowy biome.


iOS 7 support

Is there a way to use the library in iOS7?

+1

@eikebartels You can use git submodule, then import Snap classes into project.

Yea sure you can do this but does it not make more sense to set the dependency version for cocoa pods to 7 instead of iOS 8?

Cocoapods cannot build Swift libraries for iOS 7 (app store releases) so unfortunately it's not just as simple as making it available for iOS 7 in the Podspec.

Ah I see... didn't know that. Thanks for the info

@robertjpayne
so it means we need to drag the sources files into the project?

@Heisenbean pretty much yea, I internally have a build script that can create a static lib from a pure Swift library and while it holds up well in my production apps it's not something I have in a state to release publicly.
It's a safe bet just to include the source files.

Cocoapods builds on iOS 7.
I do it right now with Cocoapods 0.37.0
There is a few warning but it goes smoothly.

@thedamfr could you elaborate? AFAIK Cocoapods does not support iOS 7 for projects that include Swift code unless that has changed recently?

I think it's time for us to pay close attention to the problem. We need stronger cocoapods, come on

@JunqiangTan the problem is that Xcode does not support building static libraries for Swift code and that Dynamic Frameworks are not supported via the App Store on iOS 7.0.

@robertjpayne If you wrap the SWIFT, the OC, in the final compile time to mix, how？ I'm just a beginner and cocoapods lovers, I thought may be a bit silly, but this is my best answer, hoping to help.

@JunqiangTan You indeed can mush the Object Code files together after Compile to generate a static lib, it's very messy though and very much not supported by Cocoapods or Apple. It's really just not worth the effort.
If you still need to target iOS 7.0 and want to use Swift I strongly recommend you just use submodules and include the source, it's just too unreliable to go another route.

@robertjpayne
This is my current Podfile
source 'https://github.com/CocoaPods/Specs.git'

platform :ios, '7.0'

xcodeproj 'Tripndrive'

use_frameworks!

pod "CNPGridMenu"
pod "NGAParallaxMotion"
pod "Alamofire", '~> 1.2'
pod "CryptoSwift"
pod 'SwiftyJSON'
pod 'MagicalRecord'
pod 'GoogleAnalytics'
pod 'KeychainAccess'
pod 'Facebook-iOS-SDK', '~> 3.23.0'
pod 'JGActionSheet'
pod 'JGProgressHUD'
pod 'SnapKit', '~> 0.12.0'
pod 'SHSPhoneComponent'
pod 'SDWebImage', '~>3.7'
pod 'Fabric'
pod 'Crashlytics'

Output for pod install with 0.37.~
pod install
Analyzing dependencies

CocoaPods 0.38.2 is available.
To update use: `sudo gem install cocoapods`

For more information see http://blog.cocoapods.org
and the CHANGELOG for this version http://git.io/BaH8pQ.

Downloading dependencies
Using Alamofire (1.2.3)
Using Bolts (1.2.0)
Using CNPGridMenu (0.2.1)
Installing Crashlytics 3.1.0 (was 3.1.0)
Using CryptoSwift (0.0.10)
Using Fabric (1.2.8)
Using Facebook-iOS-SDK (3.23.2)
Using GoogleAnalytics (3.12.0)
Using JGActionSheet (1.0.4)
Using JGProgressHUD (1.2.7)
Using KeychainAccess (1.2.1)
Using MagicalRecord (2.3.0)
Using NGAParallaxMotion (1.1.0)
Using SDWebImage (3.7.3)
Using SHSPhoneComponent (2.15)
Using SnapKit (0.12.0)
Using SwiftyJSON (2.2.0)
Generating Pods project
Integrating client project

[!] The platform of the target `Pods` (iOS 7.0) may not be compatible with `Alamofire (1.2.3)` which has a minimum requirement of iOS 8.0 - OS X 10.9.

[!] The platform of the target `Pods` (iOS 7.0) may not be compatible with `CryptoSwift (0.0.10)` which has a minimum requirement of iOS 8.0 - OS X 10.9.

[!] The platform of the target `Pods` (iOS 7.0) may not be compatible with `KeychainAccess (1.2.1)` which has a minimum requirement of iOS 8.0 - OS X 10.9.

[!] The platform of the target `Pods` (iOS 7.0) may not be compatible with `SnapKit (0.12.0)` which has a minimum requirement of iOS 8.0 - OS X 10.9.

[!] The platform of the target `Pods` (iOS 7.0) may not be compatible with `SwiftyJSON (2.2.0)` which has a minimum requirement of OS X 10.9 - iOS 8.0.

This is indeed going to build with SDK 8.4
But you are going to hit validation errors when submission to AppStore (not for Crashlytics).
So when I release for AppStore, I bind directly swift files in my project. This makes the build time comes to 40min which is totally not acceptable for Dev, but is okay for Release.
Cocoapods is okay for dev.
Indeed, if I upgrade Cocoapods to 0.38.2, I have an error because platform :ios, '7.0' break your dependency minimumSDK which is explicitly set at 8.0.
But all in all, if your Lib support 7.0, there is no reason that you don't set minSDK to 7.0
Your lib is not responsible of Cocoapods way of building. Your lib support 7.0. The fact that Cocoapods does not support 7.0 for frameworks is not your problem. You ship code, they build.
So my recommandation would be to set minSDK to 7.0 in next PodSpec.

@thedamfr unfortunately the amount of support requests we were getting from users submitting to app store when the deployment step was step to 7.0 just doesn't make it worthwhile.
Likewise with the rest of the libraries you've listed there they are all in the same boat. I'm also not mindful of iOS 7 vs iOS 8 API's anymore so it's best you maintain a sync'd fork of the Podspec and likely the repo as well.

:(

@thedamfr I know it's unfortunate but like all the other repo's the tradeoffs are simply not worth it.

Yeah ! But like all the other repos, I'm sending gifs of sad cat hoping they fix it :-(


Feature Request: support websockets

It would be great if websockets would be supported ( ws:// ).

Currently, we don't have any plans to implement WebSocket support in the near future, but welcome any external PRs. I'd guess that implementation using the Paho MQTT library should be pretty straightforward.


Move Bugsnag out of core

I'm working on moving all 3rd party hooks and formatters out of core to keep core as small as possible.
@ConradIrwin @burke @BlakeMesdag do one of you mind claiming this under a personal / company repository (in the case of Bugsnag who it'd make sense maintained this) and submitting a PR to logrus to remove it and reference it in the README?

I'll own this.

I've moved Bugsnag out: https://github.com/Shopify/logrus-bugsnag.

Thanks @csfrancis, I hadn't had a chance to do anything with this yet.


bst track refuses to track only one element

See original issue on GitLab
In GitLab by [Gitlab user @abderrahimk] on Apr 18, 2018, 15:05
Today, trying bst track core-deps/WebKitGTK+.bst, I see that it tries to track webkit and all its dependencies as well. I thought the default has changed, but bst track --deps none core-deps/WebKitGTK+.bst gives the same result.

In GitLab by [Gitlab user @abderrahimk] on Apr 18, 2018, 15:20
I tried bissecting, and it seems the problem was introduced in 39c4cd17b02f627c2a6c9c3d53fafedac8091e45.

In GitLab by [Gitlab user @tristanvb] on Apr 19, 2018, 06:50
mentioned in commit 6936b0d78291a8eca44a7bd32cc7f55292a966a3

In GitLab by [Gitlab user @tristanvb] on Apr 19, 2018, 06:50
mentioned in commit cbfc6055a970299e65a3bd308e39088567bb4d78

In GitLab by [Gitlab user @tristanvb] on Apr 19, 2018, 07:12
mentioned in commit 670487bed0019048aa48576442fffd4d7368d4e3

In GitLab by [Gitlab user @tristanvb] on Apr 19, 2018, 07:12
mentioned in commit d73d75c4e4091174cea307b2ff22687e8f513ef0

In GitLab by [Gitlab user @tristanvb] on Apr 19, 2018, 07:23
closed via merge request !422


Build algorithm out of a git remote repository

As a user I'd like to define a new algorithm similar to the way I would "Create Algorithm from code" today. Only instead of uploading a tar file with the code, I would point to a specific branch in a git remote repository.
Acceptance criteria:

[ ] Content of a tar currently used for algorithm automatic build  should now work the same out of a git branch using rest.
[ ] Support https git remote repository.
At this stage only working against repository that do not need authentication.

TID-590


:arrow_up: Update lock file

Includes many new patch releases.
Some noteworthy ones appear to be:

aiohttp (3.10.5 -> 3.10.10)
pydantic (2.8.2 -> 2.9.2)
yarl (1.9.4 -> 1.15.2)

Looks like the dependency upgrades has unearthed a buggy unit test:
FAILED acapy_agent/protocols/issue_credential/v2_0/tests/test_routes.py::TestV20CredRoutes::test_credential_exchange_send_bound_offer_linked_data_error - TypeError: argument of type 'LinkedDataProofException' is not iterable

Taking a look

aiohttp probably had a minor change to their exception handling.
We were passing a LinkedDataProofException object as the "reason" for the HTTPBadRequest. Which now results in a TypeError. This is fixed by converting the exception to a string


Update consensus interface

Now favor() and get_value() are totally synchronous.
This will make the implementation much easier.
I know that this update should have been early (because it's about a huge interface change)
but the 'lower layer' has just became more concrete and now it turns out to be feasible to fit in this interface.
Please consider this interface and leave comments or ask questions.
If it's ok, then apply to your work.
This has two significant effects.

Since the interface become much simpler, the implementation will be simple as well. Synchronous favor() and get_value() will directly fit into the pseudo code.
Possible transition to ordinary Tendermint will be super easy. (Non-interactive interface)

This is a DRAFT PR. This is not for merging

I have some confusing things. Does "faver()" become synchronous affect the consensus? Doesn't it matter because the consensus layer doesn't judge "favor()"?

From now, Everything is clear to me.

Before the comment, I want to check whether I understood right or not.
What I understood is that nodes vote to agenda in Governance layer, and dicided agenda by governance is handled by consensus layer(votemint). And it results in favor(), get_value(). Do I understand correctly in our situation?
If I understood correctly, BlockProposal can be received by two ways; from governance layer or other nodes.
then, we might need additional ConsensusEvent for representing those two cases.

@yeomjh00 The matter of 'governance' is irrelevant here. We're implementing Votemint, and this interface change doesn't go off the track.

I got it. and I will apply that interface in my next PR.


DeppySource Adapter Client

User Story
As a user I would like to access the entities I've made available through a deployment of the DeppySource Adapter Server
Summary / Background
DeppySource Adapter will be sub-framework of the Deppy framework that will facilitate the transformation of online resource into entities and the querying of those entities
Acceptance Criteria

Given a host:port, I can communicate with a deployment of the DeppySource Server to get all entities

Scenarios

I have an OLM CatalogSource and would like to provide the contained bundles as entities for resolution
This will probably end up in its own repository eventually, therefore it should be placed in its own package s.t. it can just be yanked from the code-base without side-effects (aside from recalculating project dependencies for go.mod)

Assumptions
Links (docs, drawings, designs, etc)
Deppy-as-a-Framework
Dependencies
Possibly https://github.com/operator-framework/deppy/issues/44, but if we can agree on the interface before hand, we can do this in tandem
Definition of Done

unit tests
extensive godoc (this is a framework after all)
documentation of design/architecture as needed
reviewed and merged

Obsolete, closing


Don't show the dashboards in the Folder with name Hidden

There is a need to prevent users of opening some dashboards directly
They should be open from other dashboards via Navigate or/and New window controls.
So let's introduce the folder with special name Hidden and this folder will not be shown in DSW and DSW mobile

Right now we use direct links to dsw dashboard. Is that contradict this issue?


Lock updates support deleting & replacing reqs.

Previously you could only add new top-level requirements to a lock or
refine constraints against an existing locked item. Now you can -d /
--delete a top-level requirement from the lock as well as replace a
top-level requirement with -p =<req>, where the leading = prefix to
the requirement indicates replacement.
Fixes #2332
Fixes #2334

Ok, another pretty big one. Thanks again for your review efforts.
The real changes are in:

pex/cli/commands/lock.py: This is the CLI for pex3 lock (update) ... and performs post-processing of the update to display what would-be / is updated.
pex/resolve/lockfile/updater.py: This is the core lock update logic.
tests/integration/cli/commands/test_lock_update_issues_2332_2334.py: The test of the new features.

One note to draw attention to is here: https://github.com/pantsbuild/pex/issues/2334#issuecomment-1897604016
Basically, this change introduces everything needed to implement minimal lock auto-update, even for Pants. IOW ~like cargo where if you change the input requirements, the lock just gets auto-minimally updated implicitly. That said, it would be clunky to poke around in the existing lock to come up with the full set of -p =<new-req-1> ... -p =<new-req-N> -d =<removed-proj-1> ... -d =<removed-proj-M> to affect that style of update. This change was large; so I'd like to work through this PR and likely release, then circle back and add the lock update with bare requirements / -r requirements feature that will enable doing this automatically.

Users will love this (post https://github.com/pantsbuild/pex/issues/2334#issuecomment-1897604016 anyway). And once we have that, Pants should be able to auto-update lockfiles when requirements change, a la Cargo.

As that comment alluded, nothing blocks that functionality as soon as this is merged. Its clunky for a human, but Pants is not that. IIUC it is easy for pants to create a lock if none present, or else update the present lock by gathering requirements, compare to prior requirements in lock, separate in 2 groups of changed + new and deleted, call pex3 lock update with the corresponding set of -R and -d. That said, I can see if you want to wait instead of writing a bunch of logic you then turn around and rip out shortly after.


refactor!: remove global config and flatten [APE-1455]

What I did
There's now a global config by default for every plugin that gets combined automatically, so it doesn't make sense to have the old config
Also flattening the old config file is a bit more friendly
How I did it
How to verify it
Checklist

[ ] Passes all linting checks (pre-commit and CI jobs)
[ ] New test cases have been added and are passing
[ ] Documentation has been updated
[ ] PR title follows Conventional Commit standard (will be automatically included in the changelog)

Note: breaking change for v0.7
also consider bundling as 1st class


support POST /v1/teams/:team_name/posts/:post_number/star #19

Close #19

Codecov Report

Merging #85 (6abfeef) into main (c547c69) will decrease coverage by 0.61%.
The diff coverage is 77.77%.

@@            Coverage Diff             @@
##             main      #85      +/-   ##
==========================================
- Coverage   97.49%   96.88%   -0.62%
==========================================
  Files          23       23
  Lines         839      866      +27
==========================================
+ Hits          818      839      +21
- Misses         14       18       +4
- Partials        7        9       +2

Impacted Files
Coverage Δ

esaapi/posts/types/posts_parameter.go
96.12% <ø> (ø)

esaapi/stars/types/stars_parameter.go
88.67% <73.91%> (-11.33%)
:arrow_down:

esaapi/stars/types/stars_response.go
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c547c69...6abfeef. Read the comment docs.


Domain (Verse and Quote) - index number

Add index number to Verse and Quote to be able to specify order.

not actual for scraper due to index of quote and verse are stored in their position in array


feat: port CategoryTheory.Bicategory.Free

[x] depends on: #2301

This PR/issue depends on:

leanprover-community/mathlib4#2301
By Dependent Issues (🤖). Happy coding!

This PR/issue depends on:

leanprover-community/mathlib4#2301
By Dependent Issues (🤖). Happy coding!

Amazing, thanks @mattrobball, you'll have to explain to me sometime what was going on there!
bors merge


⚠ 366.ru has degraded performance

In 9d08310, 366.ru (https://366.ru) experienced degraded performance:

HTTP code: 200
Response time: 1839 ms

Resolved: 366.ru performance has improved in 29935f4.


Uable to use vue-smooth-reflow due to ES7 Spread Operator.

Due to the ES7 Spread Operator on line 106 im unable to import the vue-smooth-reflow library.

ERROR in ./node_modules/vue-smooth-reflow/src/index.js
Module parse failed: Unexpected token (106:12)
You may need an appropriate loader to handle this file type.
|             hideOverflow: true,
|             debug: false,
|             ...userOptions
|         }
|         let properties = this.parsePropertyOption(options.property)

Is there a way to solve this?
Cheers, Michael

Yes, this should be solved on my end. Are you using a module bundle, and could you post its configuration?

It seems to be fixed in this commit: https://github.com/guanzo/vue-smooth-reflow/commit/c123673ea7b3f2b7fc6b88449a46c22f9911627b
Thanks.

Great!


OCPVE-370: chore: add codespell check

Adds automated spellchecks

/hold until podman can be used within verify

closing in favor of openshift-release job


[SUPPORT] upgrade from 0.11.1 to 0.14.0

Problem
We were running Spark 3.2.1 along with HUDI 0.11.1. The jar link is: https://repo1.maven.org/maven2/org/apache/hudi/hudi-spark3.2-bundle_2.12/0.11.1/hudi-spark3.2-bundle_2.12-0.11.1.jar.
We want to upgrade Spark to Spark 3.4.1 and HUDI to HUDI 0.14.0. The jar link is: https://repo1.maven.org/maven2/org/apache/hudi/hudi-spark3.4-bundle_2.12/0.14.0/hudi-spark3.4-bundle_2.12-0.14.0.jar
The HUDI configuration that we are using for both the above cases for a sample table is:
hudi_config = {

               'className': 'org.apache.hudi',
               'hoodie.datasource.hive_sync.use_jdbc': 'false',
               'hoodie.datasource.write.precombine.field': 'transact_id',
               'hoodie.datasource.write.recordkey.field': 'id,op',
               'hoodie.table.name': 'users_masteruser_spark_test_v9',
               'hoodie.consistency.check.enabled': 'false',
               'hoodie.datasource.hive_sync.table': 'users_masteruser_spark_test_v9',
               'hoodie.datasource.hive_sync.database': 'lake_luna_for_payments',
               'hoodie.datasource.hive_sync.enable': 'true',
               'hoodie.datasource.hive_sync.mode': 'hms',
               'hoodie.datasource.hive_sync.support_timestamp': 'true',
               'hoodie.datasource.write.reconcile.schema': 'true',
               'path': 's3_target_path',
               'hoodie.datasource.write.keygenerator.class': 'org.apache.hudi.keygen.ComplexKeyGenerator',
               'hoodie.datasource.write.partitionpath.field': '',
               'hoodie.datasource.hive_sync.partition_extractor_class': 'org.apache.hudi.hive.MultiPartKeysValueExtractor',
               'hoodie.datasource.hive_sync.partition_fields': '',
               'hoodie.datasource.write.hive_style_partitioning': 'true',
               'hoodie.upsert.shuffle.parallelism': 40,
               'hoodie.datasource.write.operation': 'upsert',
               'hoodie.cleaner.policy': 'KEEP_LATEST_COMMITS',
               'hoodie.cleaner.commits.retained': 1

              }

Step 1: Loading the table using Spark 3.2.1 along with HUDI 0.11.1. This is how all of our pipelines work.
Step 2: We want to append data in the same table using Spark 3.4.1 along with HUDI 0.14.0. Here is where we are getting an
error:
ArrayIndexOutOfBoundsException            Traceback (most recent call last)
Cell In[26], line 1
----> 1 df.write.format("org.apache.hudi").mode('append').options(**hudi_config).save('s3a://trusted-luna-sbox/luna_for_payments/public/users_masteruser_spark_test_v9')

File /opt/spark/python/lib/pyspark.zip/pyspark/sql/readwriter.py:1398, in DataFrameWriter.save(self, path, format, mode, partitionBy, **options)
   1396     self._jwrite.save()
   1397 else:
-> 1398     self._jwrite.save(path)

File /opt/spark/python/lib/py4j-<IP_ADDRESS>-src.zip/py4j/java_gateway.py:1322, in JavaMember.__call__(self, *args)
   1316 command = proto.CALL_COMMAND_NAME +\
   1317     self.command_header +\
   1318     args_command +\
   1319     proto.END_COMMAND_PART
   1321 answer = self.gateway_client.send_command(command)
-> 1322 return_value = get_return_value(
   1323     answer, self.gateway_client, self.target_id, self.name)
   1325 for temp_arg in temp_args:
   1326     if hasattr(temp_arg, "_detach"):

File /opt/spark/python/lib/pyspark.zip/pyspark/errors/exceptions/captured.py:175, in capture_sql_exception.<locals>.deco(*a, **kw)
    171 converted = convert_exception(e.java_exception)
    172 if not isinstance(converted, UnknownException):
    173     # Hide where the exception came from that shows a non-Pythonic
    174     # JVM exception message.
--> 175     raise converted from None
    176 else:
    177     raise

ArrayIndexOutOfBoundsException: Index 0 out of bounds for length 0

Other Info:
For Spark3.4.1
We are using this as base spark image: public.ecr.aws/ocean-spark/spark:platform-3.4.1-hadoop-3.3.4-java-11-scala-2.12-python-3.10-gen21
And these jars:
RUN wget https://repo1.maven.org/maven2/org/postgresql/postgresql/42.6.0/postgresql-42.6.0.jar && \
    wget https://repo1.maven.org/maven2/org/apache/hudi/hudi-spark3.4-bundle_2.12/0.14.0/hudi-spark3.4-bundle_2.12-0.14.0.jar && \
    wget https://repo1.maven.org/maven2/com/mysql/mysql-connector-j/8.3.0/mysql-connector-j-8.3.0.jar && \
    mv postgresql-42.6.0.jar /opt/spark/jars && \
    mv hudi-spark3.4-bundle_2.12-0.14.0.jar /opt/spark/jars && \
    mv mysql-connector-j-8.3.0.jar /opt/spark/jars

For Spark 3.2.1
Base Image: linux/amd64 gcr.io/datamechanics/spark:platform-3.2.1-hadoop-3.3.1-java-11-scala-2.12-python-3.8-dm18
Jars:
RUN wget https://repo1.maven.org/maven2/org/postgresql/postgresql/42.3.7/postgresql-42.3.7.jar && \
    wget https://repo1.maven.org/maven2/org/apache/hudi/hudi-spark3.2-bundle_2.12/0.11.1/hudi-spark3.2-bundle_2.12-0.11.1.jar && \
    wget https://repo1.maven.org/maven2/org/apache/hive/hcatalog/hive-hcatalog-core/3.1.3/hive-hcatalog-core-3.1.3.jar && \
    wget https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.30/mysql-connector-java-8.0.30.jar && \
         mv postgresql-42.3.7.jar /opt/spark/jars && \
         mv hudi-spark3.2-bundle_2.12-0.11.1.jar /opt/spark/jars && \
         mv hive-hcatalog-core-3.1.3.jar /opt/spark/jars && \
         mv mysql-connector-java-8.0.30.jar /opt/spark/jars

@ghrahul If this is a test setup, then can you try removing 'hoodie.datasource.write.reconcile.schema' config to be sure if that is causing.  Did you tried to run upgrade tool ? Can you try to upgrade to 0.13.1 first.

@ghrahul Can you try to upgrade to 0.12.3 or 0.13.1 first.

did you try hudi cli ?

@ghrahul Any updates here, let us know if you are still stuck here.

CLi should make it easy let us know if you need any help

https://hudi.apache.org/docs/cli/#upgrade-and-downgrade-table

Hi @ad1happy2go ,
I created an Athena table using spark3.2 and HUDI 0.11.1 . In that same table when I am appending data using spark3.2 and HUDI 0.13.1 I am facing this error:
---------------------------------------------------------------------------
Py4JJavaError                             Traceback (most recent call last)
/opt/spark/work-dir/launch_ipykernel.py in <module>
----> 1 df.write.format("org.apache.hudi").mode('append').options(**hudi_config).save({s3_path})

/opt/spark/python/lib/pyspark.zip/pyspark/sql/readwriter.py in save(self, path, format, mode, partitionBy, **options)
    738             self._jwrite.save()
    739         else:
--> 740             self._jwrite.save(path)
    741
    742     @since(1.4)

/opt/spark/python/lib/py4j-<IP_ADDRESS>-src.zip/py4j/java_gateway.py in __call__(self, *args)
   1319
   1320         answer = self.gateway_client.send_command(command)
-> 1321         return_value = get_return_value(
   1322             answer, self.gateway_client, self.target_id, self.name)
   1323

/opt/spark/python/lib/pyspark.zip/pyspark/sql/utils.py in deco(*a, **kw)
    109     def deco(*a, **kw):
    110         try:
--> 111             return f(*a, **kw)
    112         except py4j.protocol.Py4JJavaError as e:
    113             converted = convert_exception(e.java_exception)

/opt/spark/python/lib/py4j-<IP_ADDRESS>-src.zip/py4j/protocol.py in get_return_value(answer, gateway_client, target_id, name)
    324             value = OUTPUT_CONVERTER[type](answer[2:], gateway_client)
    325             if answer[1] == REFERENCE_TYPE:
--> 326                 raise Py4JJavaError(
    327                     "An error occurred while calling {0}{1}{2}.\n".
    328                     format(target_id, ".", name), value)

Py4JJavaError: An error occurred while calling o186.save.
: org.apache.hudi.exception.HoodieRollbackException: Failed to rollback {s3 path}
	at org.apache.hudi.client.BaseHoodieTableServiceClient.rollback(BaseHoodieTableServiceClient.java:826)
	at org.apache.hudi.client.BaseHoodieTableServiceClient.rollbackFailedWrites(BaseHoodieTableServiceClient.java:730)
	at org.apache.hudi.client.BaseHoodieTableServiceClient.rollbackFailedWrites(BaseHoodieTableServiceClient.java:714)
	at org.apache.hudi.client.BaseHoodieTableServiceClient.rollbackFailedWrites(BaseHoodieTableServiceClient.java:709)
	at org.apache.hudi.client.BaseHoodieWriteClient.lambda$startCommitWithTime$97cdbdca$1(BaseHoodieWriteClient.java:837)
	at org.apache.hudi.common.util.CleanerUtils.rollbackFailedWrites(CleanerUtils.java:156)
	at org.apache.hudi.client.BaseHoodieWriteClient.startCommitWithTime(BaseHoodieWriteClient.java:836)
	at org.apache.hudi.client.BaseHoodieWriteClient.startCommitWithTime(BaseHoodieWriteClient.java:829)
	at org.apache.hudi.HoodieSparkSqlWriter$.write(HoodieSparkSqlWriter.scala:371)
	at org.apache.hudi.DefaultSource.createRelation(DefaultSource.scala:150)
	at org.apache.spark.sql.execution.datasources.SaveIntoDataSourceCommand.run(SaveIntoDataSourceCommand.scala:45)
	at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:75)
	at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:73)
	at org.apache.spark.sql.execution.command.ExecutedCommandExec.executeCollect(commands.scala:84)
	at org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.$anonfun$applyOrElse$1(QueryExecution.scala:110)
	at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$5(SQLExecution.scala:103)
	at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:163)
	at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$1(SQLExecution.scala:90)
	at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:775)
	at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:64)
	at org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.applyOrElse(QueryExecution.scala:110)
	at org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.applyOrElse(QueryExecution.scala:106)
	at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDownWithPruning$1(TreeNode.scala:481)
	at org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(TreeNode.scala:82)
	at org.apache.spark.sql.catalyst.trees.TreeNode.transformDownWithPruning(TreeNode.scala:481)
	at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.org$apache$spark$sql$catalyst$plans$logical$AnalysisHelper$$super$transformDownWithPruning(LogicalPlan.scala:30)
	at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.transformDownWithPruning(AnalysisHelper.scala:267)
	at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.transformDownWithPruning$(AnalysisHelper.scala:263)
	at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDownWithPruning(LogicalPlan.scala:30)
	at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDownWithPruning(LogicalPlan.scala:30)
	at org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:457)
	at org.apache.spark.sql.execution.QueryExecution.eagerlyExecuteCommands(QueryExecution.scala:106)
	at org.apache.spark.sql.execution.QueryExecution.commandExecuted$lzycompute(QueryExecution.scala:93)
	at org.apache.spark.sql.execution.QueryExecution.commandExecuted(QueryExecution.scala:91)
	at org.apache.spark.sql.execution.QueryExecution.assertCommandExecuted(QueryExecution.scala:128)
	at org.apache.spark.sql.DataFrameWriter.runCommand(DataFrameWriter.scala:848)
	at org.apache.spark.sql.DataFrameWriter.saveToV1Source(DataFrameWriter.scala:382)
	at org.apache.spark.sql.DataFrameWriter.saveInternal(DataFrameWriter.scala:355)
	at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:239)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
	at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
	at py4j.Gateway.invoke(Gateway.java:282)
	at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
	at py4j.commands.CallCommand.execute(CallCommand.java:79)
	at py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182)
	at py4j.ClientServerConnection.run(ClientServerConnection.java:106)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: org.apache.hudi.exception.HoodieException: Unknown versionCode:6
	at org.apache.hudi.common.table.HoodieTableVersion.lambda$versionFromCode$1(HoodieTableVersion.java:60)
	at java.base/java.util.Optional.orElseThrow(Optional.java:408)
	at org.apache.hudi.common.table.HoodieTableVersion.versionFromCode(HoodieTableVersion.java:60)
	at org.apache.hudi.common.table.HoodieTableConfig.getTableVersion(HoodieTableConfig.java:501)
	at org.apache.hudi.table.upgrade.UpgradeDowngrade.needsUpgradeOrDowngrade(UpgradeDowngrade.java:66)
	at org.apache.hudi.client.BaseHoodieWriteClient.tryUpgrade(BaseHoodieWriteClient.java:1257)
	at org.apache.hudi.client.BaseHoodieWriteClient.doInitTable(BaseHoodieWriteClient.java:1124)
	at org.apache.hudi.client.BaseHoodieWriteClient.initTable(BaseHoodieWriteClient.java:1162)
	at org.apache.hudi.client.BaseHoodieWriteClient.initTable(BaseHoodieWriteClient.java:1191)
	at org.apache.hudi.client.SparkRDDWriteClient.upsertPreppedRecords(SparkRDDWriteClient.java:151)
	at org.apache.hudi.metadata.SparkHoodieBackedTableMetadataWriter.commit(SparkHoodieBackedTableMetadataWriter.java:185)
	at org.apache.hudi.metadata.HoodieBackedTableMetadataWriter.update(HoodieBackedTableMetadataWriter.java:962)
	at org.apache.hudi.table.action.BaseActionExecutor.lambda$writeTableMetadata$2(BaseActionExecutor.java:76)
	at org.apache.hudi.common.util.Option.ifPresent(Option.java:97)
	at org.apache.hudi.table.action.BaseActionExecutor.writeTableMetadata(BaseActionExecutor.java:76)
	at org.apache.hudi.table.action.rollback.BaseRollbackActionExecutor.finishRollback(BaseRollbackActionExecutor.java:246)
	at org.apache.hudi.table.action.rollback.BaseRollbackActionExecutor.runRollback(BaseRollbackActionExecutor.java:114)
	at org.apache.hudi.table.action.rollback.BaseRollbackActionExecutor.execute(BaseRollbackActionExecutor.java:135)
	at org.apache.hudi.table.HoodieSparkCopyOnWriteTable.rollback(HoodieSparkCopyOnWriteTable.java:270)
	at org.apache.hudi.client.BaseHoodieTableServiceClient.rollback(BaseHoodieTableServiceClient.java:809)
	... 50 more

@ghrahul Can you post hoodie.properties.
Looks like the table got written using 0.14. That's why it has table version as 6. So for 0.13.1  which uses table version as 5  table version is unknown.
Try to write using 0.14.X version

Hi @ad1happy2go ,
Thank you for sharing the above details. We were able to follow the process and tested it with multiple tables, it worked.
We rolled out the upgrade to Spark 3.2.1 and HUDI HUDI 0.13.1 for around 500+ table in a stepwise migration. Then rolled out the upgrade again for Spark 3.4.1 and HUDI 0.14.0. This process worked for 95% of the tables.
But we are seeing issues for some tables where  daily bulk_insert is happening. These pipelines are taking 10-20x time and 10-20x cost after the upgrade. There is a specific spark job which is taking a lot of time to complete after the upgrade which we have observed:
save at DatasetBulkInsertCommitActionExecutor.java:81
Could you please tell me if we are missing something in the HUDI configuration.
Spark HUDI Bundle Used
https://repo1.maven.org/maven2/org/apache/hudi/hudi-spark3.4-bundle_2.12/0.14.0/hudi-spark3.4-bundle_2.12-0.14.0.jar
https://repo1.maven.org/maven2/org/apache/hudi/hudi-spark3.2-bundle_2.12/0.13.1/hudi-spark3.2-bundle_2.12-0.13.1.jar
HUDI CONFIG for a Table with this issue
{'className': 'org.apache.hudi',
 'hoodie.datasource.hive_sync.use_jdbc': 'false',
 'hoodie.datasource.write.precombine.field': 'created_at',
 'hoodie.datasource.write.recordkey.field': 'master_user_id,created_at',
 'hoodie.table.name': 'table_name',
 'hoodie.consistency.check.enabled': 'false',
 'hoodie.datasource.hive_sync.table': 'table_name',
 'hoodie.datasource.hive_sync.database': 'lake_useractivity',
 'hoodie.datasource.hive_sync.enable': 'true',
 'hoodie.datasource.hive_sync.mode': 'hms',
 'path': 's3a://bucket_name/table_name/public/table_name/',
 'hoodie.index.type': 'BLOOM',
 'hoodie.bloom.index.update.partition.path': 'true',
 'hoodie.bloom.index.filter.type': 'DYNAMIC_V0',
 'hoodie.datasource.hive_sync.support_timestamp': 'true',
 'hoodie.datasource.write.reconcile.schema': 'true',
 'hoodie.write.markers.type': 'direct',
 'hoodie.datasource.write.keygenerator.class': 'org.apache.hudi.keygen.ComplexKeyGenerator',
 'hoodie.datasource.write.partitionpath.field': 'master_user_id_partition',
 'hoodie.datasource.hive_sync.partition_extractor_class': 'org.apache.hudi.hive.MultiPartKeysValueExtractor',
 'hoodie.datasource.hive_sync.partition_fields': 'master_user_id_partition',
 'hoodie.datasource.write.hive_style_partitioning': 'true',
 'hoodie.upsert.shuffle.parallelism': 40,
 'hoodie.datasource.write.operation': 'bulk_insert',
 'hoodie.cleaner.policy': 'KEEP_LATEST_COMMITS',
 'hoodie.cleaner.commits.retained': 1}

We use the above config for a same job with HUDI 0.11.1 it took 5 minutes and 0.36 dollars to complete and for HUDI 0.13.1 it took 1 hour 44 mins and 36 dollars to complete and with HUDI 0.14.0 as well we saw same kind of cost hike and time hike.
We compared the jobs which are getting created in Spark UI and we saw this job is getting created which is taking a lot of  after the upgrade. And this is failing as well for some tables.

Hi @ad1happy2go , can you please share your inputs regarding this.

i am looking for further solution on this if we have done.. i have to do similar activities from Hudi 0.10 to Hudi 0.14 for our all existing Hudi table which are running in production.
have to upgrade all the existing hudi table version from 0.10 to 0.14 and looking for suggestion what are changes required in config level or any hudi properties.
appreciate for your help.

Hi @SudhirSaxena ,
As discussed in this thread, here we have to first upgrade from HUDI 0.10 to HUDI 0.13.1 by running an incremental job and then we have to again run 1 incremental job using HUDI 0.14.
I did not have to change any config in HUDI upsert configuration.
And these are the bundles that I have used:
Spark HUDI Bundle Used:
https://repo1.maven.org/maven2/org/apache/hudi/hudi-spark3.4-bundle_2.12/0.14.0/hudi-spark3.4-bundle_2.12-0.14.0.jar
https://repo1.maven.org/maven2/org/apache/hudi/hudi-spark3.2-bundle_2.12/0.13.1/hudi-spark3.2-bundle_2.12-0.13.1.jar

thanks @ghrahul for the replying, have some follow up question on this from your answer.
" here we have to first upgrade from HUDI 0.10 to HUDI 0.13.1 by running an incremental job".  -  i have to run incremental job with new hudi (version 0.13) and compare the data against existing hudi(old v 0.10) table  as target and load the data into existing Hudi(with new version 0.13 ) as target now. do i need to do any changes on hudi properties or hudi config which i should know ? Can you please elaborate somewhat more if my understanding is not correct here.
" and then we have to again run 1 incremental job using HUDI 0.14." -
i have to run incremental job with new hudi (version 0.14) and compare the data against existing hudi(updated version 0.13) table  as target and load the data into existing Hudi(with new version 0.14 ) as target now.
thanks for your quick response.

Hi @SudhirSaxena ,
If you are running data ingestion pipeline in an incremental way. You have to just change HUDI Spark bundle and run 1 incremental job in the same table without changing any HUDI properties. HUDI upsert will take care of updating the table properties.
For EMR compatibility you have to check with AWS support. I am using emr-6.15.0 along with hudi-spark3-bundle_2.12-0.14.0-amzn-0.jar for some use cases.

Hi @ad1happy2go ,
can you please help me with this?
https://github.com/apache/hudi/issues/11126#issuecomment-2264735034

@ghrahul It shouldn't be taking so much time ideally. to traiage you may try to disable metadata table to see if that is impacting the performance.
Also can you share us the spark event logs and hoodie timeline to look into this further?

Okay thanks @ad1happy2go .

Hi @ad1happy2go , here is the spark driver log file.
spark_34_hudi_14_logs.zip


Dynamic batch size feature.

Sorry I broke it.
https://github.com/ONNC/onnc/pull/75#discussion_r221021658
Open an issue for the record.
I will fix it this week.

Sorry, I have no time to fix it last week, I will try to fix it this month. Orz......


feat: port Combinatorics.SetFamily.Compression.UV

The Prop to Bool replacements for filters have made the proofs a bit more annoying, but luckily a consistent scheme has worked: Rather than rewriting the lhs of a theorem that is now full of fun a => decide (...) expressions, rewrite the rhs with conv_rhs; then, congr and ext generally reduce the problem to getting rid of these decides, which simp will now do.
Last problem is that card_compress cannot find an instance of < on Finsets being a decidable relation. Decidability of this relation is shown in Mathlib/Data/Multiset/Basic.lean for Multisets, so the last thing to explore is why this is not inherited by Finsets.

Okay, I added a Finset.decidableLE to this file, which lean was not able to infer by itself here. Potential reviewers might want to be mindful of this (does this need a backport?).

Actually, #1652 is merged now, so could you revert your proof changes?

Actually, #1652 is merged now, so could you revert your proof changes?

With that, the original proofs only needed insertions of simp_rw [Function.comp] to work again. Also addressed Yaël's other remarks. Thank you!

bors merge

bors merge

Was worth a shot :D

bors d+

bors r+


fix(CommonJS idiom) Fixes directory references in CommonJS idiom

Added fix to handle directory references in imports (CommonJS idiom) by assuming the file should be directory + index.d.ts
Added test.

Resolves Issue #38 @see https://github.com/TypeStrong/dts-bundle/issues/38

@Bartvds || @tolemac can someone take a look and hopefully approve this PR? Currently facing the issue this solves.

I would like to have only changes related to the commit message in each commit.
I think you would have to make a separate PR for fix the format (white spaces, new lines, ...)
Could you leave only changes related to the PR title?

I'm a bit busy at the moment.  I'll try to amend the commit in the next day or so.

@kcjmowright Its good for now.
I'm going to merge it and publish npm package.
Thanks a lot.


recipes-bsp/u-boot: Ensure default environment is loaded, and its' si…

…ze is sufficient
If for some reason an env located on the eMMC is loaded, the board may fail to boot. Let's ensure the default env in the u-boot binary is used by forcing u-boot to first attempt to load the eMMC env from
an inxesistent eMMC device.
Changelog-entry: recipes-bsp/u-boot: Ensure default environment is loaded, and it's size is sufficient

@balena-ci I self-certify!


Update release script

Various fixes on release script due to binaries change and the new IT test in go

LGTM


Issue with building on PR

I've followed the instructions here religiously.
https://github.com/janinko/ghprb/blob/master/README.md
I've gone through it all 2-3 times to be sure I didn't miss something.
The only thing I see that is weird is I can use the token for the configuring the Github Pull Request Builder, but it doesn't show up in the job configuration. I had to add the user/password to the job config, but it connects and seems to work.
I can enable parameterized builds by replacing sha1 with a branch name. That seems to work perfectly. However, when I finished the job configuration, I run it once so it can fetch the current state, then I create  a PR and run it again. It gives the following output:
Started by user anonymous
No JDK named ‘null’ found
No JDK named ‘null’ found
Building in workspace /var/lib/jenkins/jobs/****/workspace
No JDK named ‘null’ found
No JDK named ‘null’ found
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/****/**** # timeout=10
Fetching upstream changes from https://github.com/****/****
 > git --version # timeout=10
using .gitcredentials to set credentials
 > git config --local credential.username **** # timeout=10
 > git config --local credential.helper store --file=/tmp/git7466803986828360574.credentials # timeout=10
 > git -c core.askpass=true fetch --tags --progress https://github.com/****/**** +refs/pull/*:refs/remotes/origin/pr/*
 > git config --local --remove-section credential # timeout=10
 > git rev-parse origin/${sha1}^{commit} # timeout=10
 > git rev-parse ${sha1}^{commit} # timeout=10
ERROR: Couldn't find any revision to build. Verify the repository and branch configuration for this job.
No JDK named ‘null’ found
Finished: FAILURE

note: **** = redacted text
If I run the fetch command on the command line, it seems to work, listing new pull request.
If there is any way I can help diagnose the issue, I'm happy to help in any way that I can.

Can you post your config?  You need to use the Pull Request Trigger, and not run the build manually.

For some reason the cronjob wasn't executing. I had even changed it from 5 minutes to 1 minute. When I rebooted the VM today, it immediately caught up on all the PRs.
Why did it not run the cron until reboot? Did I miss a step?

Do you have the webhook checked?

No, it is configured to poll because it isn't reachable from the internet.


RBoundedBlockingQueue.offer used an unexpected serializer

Expected behavior

RBoundedBlockingQueue<ProtobufObj> queue = redissonClient.getBoundedBlockingQueue("xx",myCodec);

queue.offer(proto);

List<ProtobufObj> pl = queue.poll(20);

Actual behavior
The problem is  :
RBoundedBlockingQueue.offer(V e) -> RBoundedBlockingQueue.offerAsync(V e)

at last use tryAcquireAsync(int permits) send data

it used default codec, not myCodec.
Conclusion:
RBoundedBlockingQueue<ProtobufObj> queue = redissonClient.getBoundedBlockingQueue("xx",**myCodec**);

queue.offer(proto);

Although the codec is specified, but it used default codec
List<ProtobufObj> pl = queue.poll(20);

When i poll data from RBoundedBlockingQueue which causes an exception to occur during deserialization.Because it used my specified codec, but "offer" not
Steps to reproduce or test case
Redis version
6.2.6
Redisson version
3.17.7
Redisson configuration

👀

Fixed! Thanks for report


Error in editing and trainning entity

I am installing rasa-ui in window 10 and open in chrome browser. However, it is not able to click the expand entity button in the intent detail page. Moreover, it can not train the entity in the model ! How to overcome this ?

@cypoon  click the expand entity button in the intent detail page : Did you select a word for entitiy and click the link icon first? Then the expand button will work.
it can not train the entity in the model Can you specify what error you get either in rasa-ui console of rasa_nlu?

I have input the entity in the Entities tab inside Agent. However, those entities cannot be shown in the training json !

I have input the entity in the Entities tab inside Agent. However, those entities cannot be shown in the training json !
You should see the list of entities for specific utterences only. Can you check that in the generated json file on the training page?


fixed function overloading breaking transitions

Hello,
I noticed that jira start, stop and review didn't work anymore. Debugged a bit and found the culprit in a overloaded function that got the wrong parameters.
Also fixed jira.open for jiras that are not on the root of a domain

Hi @dvcrn! Thanks for the PR. I will try to check it out this week. :+1:

ping @germanrcuriel

Sorry for the delay. I just published version 0.5.1. Thank you!


XSS Vulnerabilities (Cross Site Scripting)

There are XSS vulnerabilities in name, options/label and options/value, example:
VirtualSelect.init({
	ele: '#sample-select',
	multiple: true,
	name: 'aaa">
BIG HEADER
',
	options: [
		{ label: 'All will be bold', value: '1' },
		{ label: 'Value contains quote char', value: 'before"aaa' },
	],
});

This will result in the text BIG HEADER being rendered as an actual
 and all the options will be bold and selecting the last option would make the value of the hidden input "before" except that the input's className is removed by the name XSS.
These are technically 3 separate issues but i hope you don't mind me putting them in one ticket here.

@Jaaap thanks for letting me know it. I will fix those issues in the future release.

@Jaaap this issue got fixed in the release v1.0.19
To avoid this security issue, use enableSecureText: true property.


Fixed parameter in example code.

"–ModuleName" in the example code was using the wrong dash "–" instead of "-" which could cause problems if example is copied and pasted.

@gerane : Thanks for your contribution! The author, @mgoedtel, has been notified to review your proposed change.

@gerane - Thanks for your contribution to this article.  I've reviewed your proposed change and approve its implementation.  #sign-off


@observable: Support observable getters like polymer @published

This issue was originally filed by<EMAIL_ADDRESS>
I'm not sure why this ended up in polymer instead of observe, but it's just as useful for non polymer apps.  The method ObservableMixin._observed is almost identical to polymer/src/declaration _getProperties.
See https://code.google.com/p/dart/issues/detail?id=13937.

This issue has been moved to dart-lang/observe#47.


AWS key exposed

lol
https://github.com/noisebridge/library-org/commit/574a055694457a038a306b19ade36fb5af7ad3c5

I remember now these were howards and he deactivated them before sending me the code.


How to have a "save draft" use a different validate function?

I'm curious if anyone has any examples of using redux-form to have validation for the form, but allowing a "save draft" type button that would allow submitting with a different set of validation rules.
I.E. Some fields can be empty in draft state, but should be forced to be filled out in order to use the "Start" submit button.

Seems like that could be done via submit validation, no?

Sure! I was thinking this might actually make a good documentation example, which is one of the reasons I was opening an issue also.
I ended up doing something with submit validation, but now I've lost any sync validation capabilities.
I still think some way to allow a button to be bound to attempt to submit even though errors are present ala handleSubmit(saveDraft, { allowInvalid: true }) might be a useful future feature.
Or even tying it in with shouldValidate() somehow, passing a flag saying which submit handler is being used?
I'd really love to be able to have the sync validation errors showing, but still allow people to "save draft"

Meanwhile, StackOverflow is better at keeping popular questions visible, while separating feature-related questions from help-related questions.

I totally get that :smile: -- You might want to update the website to not link to stack overflow or issue trackers next to "Ask for help" as I interpreted this as you would welcome questions in either forum.

You can always ping us here or at Twitter with the question you need help with if you're not getting answers :)
Also, I'm afraid that the alert GitHub shows pointing to CONTRIBUTING.md when you're about to open an issue makes no difference at all 😢

Anyway -- issue at hand.
Would there be a way to tell redux-form to use a sync/async validate method for displaying errors, but to flag that my submit functions will do their own error check on submit?
Even a simple flag like reduxForm({ allowSubmitWhileInvalid: true }) (name bike-shedding seems appropriate ;) )

@gustavohenke & @erikras, I agree with @gnarf, this is an important use case of form validation that would be cool if it were properly covered by reduxForm.
Let's say that you are building a CMS interface. In it you'd have the ability to create posts. When you are creating a post you have 2 actions: Save as a draft or Publish.
When saving as a drafts, you'd be interested to validate that the title is filled. On the other hand, when publishing, you'd be interested to validate that both the title and body are field.
I agree that this is totally achievable with the submit validation example, however, what if you are working on an accounting CMS (real world example) and your post all of the sudden has 20 fields that require validation as a draft and 80 in total when publishing.
Therefore, in my opinion, a flag passed at submission level would be a nice future, because afterwards you could tailor your field validators after that. Take this as an example:

  validatePost = (validators) => {
    // get submissionType from somewhere
    if (submissionType === 'post') {
      return validators;
    }

    return undefined;
  };

  validateDraft = (validators) => {
    // get submissionType from somewhere
    if (submissionType === 'draft') {
      return validators;
    }

    return undefined;
  };

  handlePost = () => {
    // set submissionType to post,
    // validate,
    // if valid continue submission
  };

  handleSaveDraft = () => {
    // set submissionType to draft,
    // validate,
    // if valid continue submission
  };

  render() {

    return (
      <form>
        {/* validate on draft & post */}
        <Field name="title" type="text" component="input" validate={[ this.validateDraft([required]), this.validatePost([required]) ]} />

        {/* validate on draft & post */}
        <Field name="shortDescription" type="text" component="input" validate={[ this.validateDraft([required]), this.validatePost([required]) ]} />

        {/* validate on post */}
        <Field name="longDescription" type="text" component="input" validate={[ this.validatePost([required]) ]} />

        <button onClick={this.handlePost}>Post</button>
        <button onClick={this.handleSaveDraft}>Save Draft</button>
      </form>
    );
  }
}

export default reduxForm({
  form: 'postForm'
})(Example);```

To sum up, I think this should be opened and labeled as an `enhancement`.

Has anyone found a solution to doing this?


"Failed to monitor paths" behaviour changed

Hi @jlesage ,
I don't know if this is related to the newest release or if its because of crashplan itself.
But since November release which i pulled yesterday i get many windows with failed to monitor paths. Previously i got only one and could dismiss that. Then move on to login. Now its impossible to get to the login screen. Is that something you can affect?

Of course it would be best to fix the root problem of the crashplan user being able to access the failing paths. But that problem i don't fully understand.
I get this from the service log:
[10.31.22 05:08:55.573 WARN  Thread-628   ode42.jna.inotify.InotifyManager] Unable to add watch for path /volume1/NetBackup/DiskStation_0011320383DF/lorem og ipsum/3 lorem og ipsum/Foxtrot V/@eaDir, errno: , 13
but when i ssh as the crashplan user I have no problem cd'ing to that path.
Container has this user set:

Which corresponds to this user
uid=1043(crashplan) gid=100(users) groups=100(users),101(administrators)
Does anyone recognize this problem?

I confirm the multiple windows issue.  This will be fixed.
For the other issue, being able to cd to the path doesn't means that CrashPlan can read the file.  Execute ls -l <file> on one of the failed file to verify the owner/permission of the file.  It is probably incompatible with the configured user/group.

Thanks for confirming the multiple window issue :)
I tried to do the ls- l command on one of the files and got this:
$ ls -l \@eaDir/16386857_10154934418928536_5597665846367469425_n.jpg\@SynoEAStream
-rwxrwxrwx 1 rsync users 174 Feb  1  2017 @eaDir/16386857_10154934418928536_5597665846367469425_n.jpg@SynoEAStream

The owner is not the same, but group is, so i thought that would be sufficient. But maybe i'm wrong.
I could even cat the file with my crashplan user, so wouldn't that mean there is something else at play?

For this file, the log message had the errno 13 ?

I confirm the multiple windows issue. This will be fixed.
For the other issue, being able to cd to the path doesn't means that CrashPlan can read the file. Execute ls -l <file> on one of the failed file to verify the owner/permission of the file. It is probably incompatible with the configured user/group.

Confirmed that the multiple window issue is fixed. And i have tried to make folder exclusions of the @eaDir to avoid further failed paths notification... but i still got one (only one window), but no new entries in the service log. Strange!?


Chapter one poems

Ok, the major poems of chapter one of the handbook have been added, and they display in the requested meter on the site.
Time to try a pull request to see if the continuous integration setup works.

Passed unit test in CI and passed UAT on localhost, so I think we're good to go.

Ok, that worked beautifully, including the automatic deployment to the site.


rangeofficer style clean up/refactor

Näköjään jotkut värit ei nyt vastaa niitä mitä testit odottavat. Voit tsekata failaavat testit komennolla npm run test src/tabletview/rangeofficer.test.js, jos auttaisi korjaamisessa.

Codecov Report

Merging #129 (2159184) into develop (5a58d77) will decrease coverage by 0.25%.
The diff coverage is 80.00%.

@@             Coverage Diff             @@
##           develop     #129      +/-   ##
===========================================
- Coverage    51.51%   51.26%   -0.26%
===========================================
  Files           86       99      +13
  Lines         3368     3730     +362
===========================================
+ Hits          1735     1912     +177
- Misses        1633     1818     +185

Flag
Coverage Δ

back-end
46.81% <ø> (+0.53%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

front/src/tabletview/rangeofficer.js
66.12% <80.00%> (ø)

back/models/user.js
19.44% <0.00%> (-0.56%)
:arrow_down:

back/routes.js
100.00% <0.00%> (ø)

back/models/index.js
100.00% <0.00%> (ø)

back/services/user.js
82.35% <0.00%> (ø)

back/services/index.js
100.00% <0.00%> (ø)

back/validators/index.js
100.00% <0.00%> (ø)

back/controllers/index.js
100.00% <0.00%> (ø)

back/middlewares/index.js
100.00% <0.00%> (ø)

back/controllers/raffle.js
66.66% <0.00%> (ø)

... and 17 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5a58d77...2159184. Read the comment docs.

Some small fixes, but there are still issues
-redLight, greenLight, cream5 return as transparent
-rangeofficer status doesn't change, even though this seems to work the same way as in develop branch (eg. returns orange, when color should be redLight)


Decoding the query string before parsing it is problematic

Let's say I wanted to encode some JSON, like {"foo":1,"bar":2} in a query param. Usually I would encode this value and pass it in the URL like this, /my/api?skidoo=%7B%22foo%22%3A1%2C%22bar%22%3A1%7D. The way things currently work, the whole query string will be decoded and then parsed by splitting at commas. This results in a skidoo param with two values '{"foo":1' and '"bar":1}'.
If the query string were parsed and then each key and value decoded the result would be a single skidoo param with the value ``'{"foo":1, "bar":1}'`.

That sounds reasonable.  Perhaps the existing behavior can be kept in a method should a user desire this behavior, but default behavior should decode the query string without assuming the values are comma-delimited.

Yeah, so the issue isn't so much that the entire query string is being decoded in a single pass, but that each value is being split by comma. We could use RequestOptions to toggle splitting values on commas, and turn it off by default. That would be a breaking change, but it would be easy for anyone encoding lists into values to turn it back on and be back in business.

I don't think you can decode in one pass, and I've never seen it done that way.  The reason is that percent encoding is meant to take special characters with reserved meaning and turn that reserved meaning off.  It's a form of escaping, much like "\n" prevents the carriage return from going to the next line.  Therefore if I have an encoded "&" for example, decoding in one pass will mixup the special (non-escaped) & and escaped &.  You will split on all of the escaped &, which would be an error.
http://en.wikipedia.org/wiki/Percent-encoding
There is a nice side to all of this which is that if you slightly extended the special characters to include ",", then an escaped "," would not split (internal split to array, not param split).  This would allow things to mostly work as-is, with the small additional documentation that "," should not be percent encoded if their special split behavior is desired.

You may, of course, still have to have a flag on the off case that someone is percent encoding and their "," turned off would break.  However, you're at .3 so I'd think this type of thing can be squeezed through without the annoying flag.

I've seen lots of encoding/decoding errors in complex params (on both sides:  developers implementing, and frameworks).  With the niche falcon occupies, it would be great if parameter decoding/encoding just worked flawlessly.  I realize this is true spec-grunginess at it's finest, but...
Here's a pretty thoughtful article by someone who tried to get all of this absolutely correct.  I only fact checked some of it (the parts on escaping special characters, and some other parts) and it seems right on.
http://blog.lunatech.com/2009/02/03/what-every-web-developer-must-know-about-url-encoding

There's a PR (#461) to address this. Love to get some more eyes on it.

OK, commented in the ticket.  I'll look over the code when I get a chance.
In a nutshell, I suggest decoding very last (it's the only possible right thing to do).
This means you must split twice, once using literal "&" (split to name=value pairs),
and then subsplit the value on literal "," to get induced array values.
Once you've done that, you can iterate through this mess and decode each fragment
(i.e. each value, or the param name).  Not decoding the individual array values (induced
by ",") means you are treating these values differently than scalars - why would you want
that? (answer: you don't).
Thanks for jumping on this so quickly, you guys rock!

On Apr 1, 2015, at 12:33 PM, Kurt Griffiths<EMAIL_ADDRESS>wrote:
There's a PR (#461 https://github.com/falconry/falcon/pull/461) to address this. Love to get some more eyes on it.
—
Reply to this email directly or view it on GitHub https://github.com/falconry/falcon/issues/369#issuecomment-88604953.


units of * particle sometimes missing sometimes not

Looking at these two tests it appears we have the units of particles in the units for some cases and not for others
    def test_cell_tally_flux_with_no_std_dev(self):

        result = otuc.process_tally(
            tally=self.my_tally_2,
            required_units="centimeter / source_particle",
        )

        assert len(result) == 1
        assert result[0].units == "centimeter / source_particle"

    def test_cell_tally_flux_base_units(self):

        result = otuc.process_tally(tally=self.my_tally)

        assert len(result) == 2
        assert result[0].units == "centimeter * particle / source_particle"
        assert result[1].units == "centimeter * particle / source_particle"

Also occurs for neutron


[vectorized] (olap) optimize BlockReader's performance

Proposed changes
make _collect_iter as normal object instead of pointer object
replace _init_agg_state to avoid unnecessary condition branch
prefer to use reference instead of object copy
prefer to use emplace_back rather than push_back
pick the evaluation out from loop
ssb test indicate that this change have 5% improvement nearly(41 seconds reduce to 40.5 seconds)
test SQL: select sum(lo_revenue) as revenue from lineorder join dates on lo_orderdate = d_datekey where d_year = 1993 and lo_discount between 1 and 3 and lo_quantity < 25;
Close related #issue (replace it with issue number if it exists).
Describe the overview of changes, and introduce why we need it.
Types of changes
What types of changes does your code introduce to Doris?
Put an x in the boxes that apply

[ ] Bugfix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Documentation Update (if none of the other choices apply)
[ x] Code refactor (Modify the code structure, format the code, etc...)
[ x] Optimization. Including functional usability improvements and performance improvements.
[ ] Dependency. Such as changes related to third-party components.
[ ] Other.

Checklist
Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code.

[ ] I have created an issue on (Fix #ISSUE) and described the bug/feature there in detail
[ ] Compiling and unit tests pass locally with my changes
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] If these changes need document changes, I have updated the document
[ ] Any dependent changes have been merged

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

LGTM