Update Preferences page with push-to-page design

Problem:
The current Preferences page style is outdated. There are many toggles and a few dropdowns on that page which used to be fine, but we're working on standardizing our pages with form inputs (including toggles, dropdowns, etc) and we haven't yet determined how we want this Preferences page to look in the new push-to-page system
Solution
Create a design, get some feedback, then implement a new Preferences page!
Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~018aeb1cf896e75863
Upwork Job ID:<PHONE_NUMBER>637057024
Last Price Increase: 2023-01-20

Started a convo here

Thanks for starting the convo in slack @shawnborton  👍  And thanks for taking over @cristipaval ! 👍

No problem! Cristi - holler if you have any questions as you work on this.

Hey @shawnborton ! So we want to add some brand imagery at the top of the Preferences page, as I see here.

You said on Slack that is image is just a placeholder. What is the final image that we want?

Hmm, I think we still need to sort that out. Maybe we can spin up a follow up issue and tackle that separate from these changes? Thoughts?

Yeah I think the branding can come after the functionality

Agreed. I'd just tackle the push-to-page stuff first. @shawnborton wouldn't this just be part of this existing issue?

Hmm I suppose it could be, it does feel closely related.

I was about to create a follow up issue. I'll just mention about this page in the issue linked by you, @JmillsExpensify.

Reviewing label has been removed, please complete the "BugZero Checklist".

The solution for this issue has been :rocket: deployed to production :rocket: in version 1.2.66-0 and is now subject to a 7-day regression period :calendar:. Here is the list of pull requests that resolve this issue:

https://github.com/Expensify/App/pull/14591

If no regressions arise, payment will be issued on 2023-02-14. :confetti_ball:
After the hold period is over and BZ checklist items are completed, please complete any of the applicable payments for this issue, and check them off once done.

[ ] External issue reporter
[ ] Contributor that fixed the issue
[ ] Contributor+ that helped on the issue and/or PR

As a reminder, here are the bonuses/penalties that should be applied for any External issue:

Merged PR within 3 business days of assignment - 50% bonus
Merged PR more than 9 business days after assignment - 50% penalty

Based on the PR review, Rushat should also be assigned to to this issue as well.

Current assignee @JmillsExpensify is eligible for the External assigner, not assigning anyone new.

Current assignee @rushatgabhane is eligible for the External assigner, not assigning anyone new.

Triggered auto assignment to @francoisl (External), see https://stackoverflow.com/c/expensify/questions/7972 for more details.

Well that stinks. I was hoping for the Upwork job. Trying again.

Current assignee @JmillsExpensify is eligible for the External assigner, not assigning anyone new.

Current assignee @rushatgabhane is eligible for the External assigner, not assigning anyone new.

Triggered auto assignment to @Julesssss (External), see https://stackoverflow.com/c/expensify/questions/7972 for more details.

@rushatgabhane Mind applying to the Upwork job here? https://www.upwork.com/jobs/~012042025690f626f7

Great offer sent.

All paid out!


fix: recoveryservices model validation errors

Remove empty body for RESPONSE_SCHEMA_NOT_IN_SPEC errors

Automation for azure-sdk-for-python

Automation for azure-sdk-for-go

Can one of the admins verify this patch?


style/travis: Allow travis to build on container based system

Added required apt-packages
Added ability to run clippy as feature in nightly
set sudo false in travis
Several remaining compile fixes remain, but will be resolved with upcoming
fix with 0.2 code.

There were the following issues with your Pull Request

Commit: ef5808f074cade0987dca3566edb4b07ac6df2f9
Commits must be in the following format: %{type}/%{scope}: %{description}

Guidelines are available at https://github.com/autumnai/leaf/blob/develop/CONTRIBUTING.md#git-commit-guidelines

This message was auto-generated by https://gitcop.com

Is it possible to have some examples of the commit message format? I see you use "style/XXX" where XX is changed file or module. I am not sure if this is what is missing, sorry for any miscommunication here. I Will also dig more into gitcop to see what I can find.

I see you have figured it out, but you are right, having an example in the guidelines would certainly be nice (issue #48).
Since I saw that #46 passed with the other changes I will go ahead and merge it even though it's failing.
Thanks you! :)

Oh :open_mouth:, only the PR title had the correct format, not the commit. Oh well...

Ah yes I spotted that just after the merge. Oh well not to worry. Sorry chap, I ended up rebasing (hard) locally and could not get back to the commit. At least I now know. Next time though it will all be good.

Yup, better luck next time :)


Stats fail to import

Hi so I started converting another book and the stats are failing, likely due to how the stats are formatted, this is the statblock and I was just wondering if it was possible to have the importer read this properly and import it.
Kongamato
Huge dragon, neutral
Armor Class: 29 (natural armor +18, Dex +3, size -2)
Hit Points: 261 (18d12+144)
Speed: 40 ft., fly 160 ft. (average)
STR: 30 (+12) DEX: 17 (+3) CON: 26 (+8) INT: 9 (-1) WIS: 14 (+2) CHA: 17 (+3)
Skills: Athletics +20, Intimidation +9, Perception +13, Stealth +7, Survival +13
Damage Resistances: acid, cold, electricity, fire
Damage Immunities: magic sleep, magic paralysis
Condition Immunities: -
Senses: darkvision 60 ft., passive Perception 23
Languages: Draconic
Challenge: 20 (51,200 XP)
Piercing Beak. When the kongamato attempts to damage an object with its beak attack, the attack does double damage and is treated as if it were made with an adamantine weapon for the purposes of overcoming the object's hardness.
Wailing Dive (1/turn). When the kongamato makes a charge while flying or uses its Flyby Attack feat, its body becomes infused with energy, causing its bite attack to deal an extra 2d6 sonic damage on that attack.
Actions
Multiattack. The kongamato makes five attacks: one with its bite, two with its claws, one with its tail slap, and one with its wings.
Bite. Melee Weapon Attack: +20 to hit, reach 15 ft., one target. Hit: 24 (2d10+12) piercing damage plus 10 (2d6) sonic damage.
Claw. Melee Weapon Attack: +20 to hit, reach 10 ft., one target. Hit: 18 (2d8+9) slashing damage.
Tail Slap. Melee Weapon Attack: +20 to hit, reach 15 ft., one target. Hit: 17 (2d6+9) bludgeoning damage.
Wings. Melee Weapon Attack: +20 to hit, reach 10 ft., one target. Hit: 15 (1d8+9) slashing damage.
Breath Weapon (1/turn). The kongamato exhales a cone of sonic energy that is 50 feet long. Each creature in that area must make a DC 20 Dexterity saving throw, taking 49 (14d6) sonic damage on a failed save, or half as much damage on a successful one.

The problem is all the colons they put in after armor class, hit points, all the abilities, skills, damage resistances, etc. Adding support for them would make the code more complicated than it really needs to be. That makes it harder to maintain in the long run. If you delete them it imports fine though. That's still going to be faster than creating it from scratch. I did it with this one, like this...
Kongamato
Huge dragon, neutral
Armor Class 29 (natural armor +18, Dex +3, size -2)
Hit Points 261 (18d12+144)
Speed 40 ft., fly 160 ft. (average)
STR 34 (+12) DEX 17 (+3) CON 26 (+8) INT 9 (-1) WIS 14 (+2) CHA 17 (+3)
Skills Athletics +20, Intimidation +9, Perception +13, Stealth +7, Survival +13
Damage Resistances acid, cold, electricity, fire
Damage Immunities magic sleep, magic paralysis
Condition Immunities -
Senses darkvision 60 ft., passive Perception 23
Languages Draconic
Challenge 20 (51,200 XP)
Piercing Beak. When the kongamato attempts to damage an object with its beak attack, the attack does double damage and is treated as if it were made with an adamantine weapon for the purposes of overcoming the object's hardness.
Wailing Dive (1/turn). When the kongamato makes a charge while flying or uses its Flyby Attack feat, its body becomes infused with energy, causing its bite attack to deal an extra 2d6 sonic damage on that attack.
Actions
Multiattack. The kongamato makes five attacks: one with its bite, two with its claws, one with its tail slap, and one with its wings.
Bite. Melee Weapon Attack: +20 to hit, reach 15 ft., one target. Hit: 24 (2d10+12) piercing damage plus 10 (2d6) sonic damage.
Claw. Melee Weapon Attack: +20 to hit, reach 10 ft., one target. Hit: 18 (2d8+9) slashing damage.
Tail Slap. Melee Weapon Attack: +20 to hit, reach 15 ft., one target. Hit: 17 (2d6+9) bludgeoning damage.
Wings. Melee Weapon Attack: +20 to hit, reach 10 ft., one target. Hit: 15 (1d8+9) slashing damage.
Breath Weapon (1/turn). The kongamato exhales a cone of sonic energy that is 50 feet long. Each creature in that area must make a DC 20 Dexterity saving throw, taking 49 (14d6) sonic damage on a failed save, or half as much damage on a successful one.


SlackObjectFormationError: options attribute must have between 2 and 5 items whereas API allows a single option

Reproducible in:
The Slack SDK version
slack-sdk==3.16.1
Python runtime version
Python 3.10.3
OS info
ProductName:    macOS
ProductVersion: 12.3.1
BuildVersion:   21E258
Darwin Kernel Version 21.4.0: Fri Mar 18 00:46:32 PDT 2022; root:xnu-8020.101.4~15/RELEASE_ARM64_T6000
Steps to reproduce:
        Message(
            text="placeholder",
            blocks=[
                SectionBlock(
                    text="placeholder",
                    accessory=OverflowMenuElement(
                        options=[Option(value="placeholder", text="placeholder")]
                    ),
                )
            ],
        ).to_dict()

^ This will fail with slack_sdk.errors.SlackObjectFormationError: options attribute must have between 2 and 5 item
It used to be that the API required a minimum of two option entries in the overflow menu, but for a number of years now it has permitted just a single one.
Expected result:
No error is thrown and the object is converted to a dict.
Block kit builder screenshot to confirm that the API allows a single option:

{
	"blocks": [
		{
			"type": "section",
			"text": {
				"type": "mrkdwn",
				"text": "This is a section block with an overflow menu with a single option."
			},
			"accessory": {
				"type": "overflow",
				"options": [
					{
						"text": {
							"type": "plain_text",
							"text": "A single option",
							"emoji": true
						}
					}
				]
			}
		}
	]
}

Actual result:
An error is thrown: slack_sdk.errors.SlackObjectFormationError: options attribute must have between 2 and 5 item

Hi @wilhelmklopp 👋🏼  Thanks for raising this issue and for providing such thorough details. I've confirmed what you're seeing and verified that a single entry works as expected, despite documentation and the SDK indicating otherwise.
We'll get a fix out for this shortly!

Thank you! Appreciate the speedy turnout :)


github filter patterns are not quite regex

https://docs.github.com/en/actions/writing-workflows/workflow-syntax-for-github-actions\#filter-pattern-cheat-sheet
What this PR does / why we need it:

Which issue(s) this PR fixes:

Does this PR require a test?

Does this PR require a release note?

Does this PR require documentation?

Merging as this does not affect tests.


Leverage browser caching - Page Speed Insight

Hi,
I want to modify this but the rule is a htaccess conditions. How i can do it on caddy with php or caddyfile?
Example htaccess :
EXPIRES CACHING

ExpiresActive On
ExpiresByType image/jpg "access 1 year"
ExpiresByType image/jpeg "access 1 year"
ExpiresByType image/gif "access 1 year"
ExpiresByType image/png "access 1 year"
ExpiresByType text/css "access 1 month"
ExpiresByType text/html "access 1 month"
ExpiresByType application/pdf "access 1 month"
ExpiresByType text/x-javascript "access 1 month"
ExpiresByType application/x-shockwave-flash "access 1 month"
ExpiresByType image/x-icon "access 1 year"
ExpiresDefault "access 1 month"

## EXPIRES CACHING ##
Best Regards
Quentin Laffont

Another Problem Reduce server response time !
In our test, your server responded in 0.78 seconds. There are many factors that can slow down your server response time. Please read our recommendations to learn how you can monitor and measure where your server is spending the most time.

Caching features aren't yet implemented; see #390, #191, #10.
As for the response time, there are a number of things that could be slowing it down. I encourage you to look at their recommendations (whatever they are).

@mholt i check linked topic so we don't have any solution at this time to cache website ?

Not yet, but it's up for grabs if anyone wants to do it. (Will require more discussion though.)


Benchmarks

Hi and thank you for your incredible work!
Did you perform any performance benchmarks or do you plan to do that?
I'm wondering if wasm from this package can perform better than JS and what kind of impact we can see when calling back and forth js and wasm.

Wow thanks a lot!
Not yet. But from what I've heard, it is slow. 😅


decoder.embed_tokens.weight.pt not found

Hi, when I using distserve_api_server.py to start a server, it always raise an error during lauching it:
ray.exceptions.RayTaskError(RuntimeError): ray::ParaWorker.init_model() (pid=129376, ip=<IP_ADDRESS>, actor_id=27f5ee672314d5dfcee7384701000000, repr=<distserve.worker.ParaWorker object at 0x7efea016a110>)
  File "/share/share/DistServe/distserve/worker.py", line 100, in init_model
    self.model.load_weight(path)
RuntimeError

When I shut it down, it will output the following error:
(ParaWorker pid=129375) Gpt<T>::load() - /llama-2-13b-hf/decoder.embed_tokens.weight.pt not found [repeated 5x across cluster]

The model is llama-2-13b-hf, I can run it in vLLM.

May I have your startup command? I suspect that your model argument is incorrect.

I have resolved this issue. I discovered that the model weights need to be converted via the converter under the downloader. However, the function download_and_convert_weights returns immediately when encountering model path, without converting the weights. This led to the error where the weights could not be run.

but how can I run converter.py with a local model dir? it seems the converter get a parameter '--input' to get 'model.bin', but I have several *.bin files

but how can I run converter.py with a local model dir? it seems the converter get a parameter '--input' to get 'model.bin', but I have several *.bin files

You can use globs like /data/weights/*.bin
NOTE. In some shells you may need to wrap it in quotation marks to avoid being expanded by the shell.

but how can I run converter.py with a local model dir? it seems the converter get a parameter '--input' to get 'model.bin', but I have several *.bin files

You can use globs like /data/weights/*.bin
NOTE. In some shells you may need to wrap it in quotation marks to avoid being expanded by the shell.

I convert the model like this:
python distserve/downloader/converter.py --input /data/Llama-2-7b/pytorch_model.bin --output /data/Llama-2-7b --model llama
then run the model_server like this:
python distserve/api_server/distserve_api_server.py --model /data/llama-2-7B/
but it's still missing something:
(ParaWorker pid=715665) INFO 13:35:56 runtime peak memory: 12.922 GB
(ParaWorker pid=715665) INFO 13:35:56 total GPU memory: 39.392 GB
(ParaWorker pid=715665) INFO 13:35:56 kv cache size for one token: 0.50000 MB
(ParaWorker pid=715665) INFO 13:35:56 num_gpu_blocks: 2883
(ParaWorker pid=715665) INFO 13:35:56 num_cpu_blocks: 128
(ParaWorker pid=715665) Gpt::load() - /data/fx/cql/llama-2-7B/decoder.output_projection.weight.pt not found

but how can I run converter.py with a local model dir? it seems the converter get a parameter '--input' to get 'model.bin', but I have several *.bin files

You can use globs like /data/weights/*.bin
NOTE. In some shells you may need to wrap it in quotation marks to avoid being expanded by the shell.

I convert the model like this:
python distserve/downloader/converter.py --input /data/Llama-2-7b/pytorch_model.bin --output /data/Llama-2-7b --model llama
then run the model_server like this:
python distserve/api_server/distserve_api_server.py --model /data/llama-2-7B/
but it's still missing something:
(ParaWorker pid=715665) INFO 13:35:56 runtime peak memory: 12.922 GB (ParaWorker pid=715665) INFO 13:35:56 total GPU memory: 39.392 GB (ParaWorker pid=715665) INFO 13:35:56 kv cache size for one token: 0.50000 MB (ParaWorker pid=715665) INFO 13:35:56 num_gpu_blocks: 2883 (ParaWorker pid=715665) INFO 13:35:56 num_cpu_blocks: 128 (ParaWorker pid=715665) Gpt::load() - /data/fx/cql/llama-2-7B/decoder.output_projection.weight.pt not found

Paths and file names are case-sensitive under Linux. Please correct your path when running the API server.

thx, I correct my spell, but it seems missing decoder.output_projection.weight.pt
Gpt<T>::load() - /data/fx/cql/Llama-2-7b-hf/decoder.output_projection.weight.pt not found
I carefully check the conversion and its output files, they are three types:
decoder.embed_tokens.weight.pt
decoder.layer_norm.weight.pt
decoder.layers.*
but no file named decoder.output_projection

I have found the cause that led to this issue. I discovered that the model weights need to be converted via the converter under the downloader. However, the function download_and_convert_weights returns immediately when encountering model path, without converting the weights. This led to the error where the weights could not be run.

I am still missing decoder.output_projection.weight.pt not found,even I've converted weights

Sorry but I am unable to reproduce your issue. I have tried the following steps:

Cloning meta-llama/Llama-2-7b-hf to some directory
Running python3 distserve/downloader/converter.py --input "PATH/TO/DOWNLOADED_WEIGHTS/*.bin" --output PATH/TO/OUTPUT --dtype float16 --model llama.
Listing all files under the output directory, and finds out that it do generate decoder.output_projection.weight.pt.

May you check all your commands again?

I have changed the convert_weights function like this:
def convert_weights(
    input: str,
    output: str,
    dtype: torch.dtype,
    model: str
) -> None :
    """Function used by `downloader.py` to convert weights"""
    os.makedirs(output, exist_ok=True)
    print(f"Converting {input} into torch.jit.script format")

    # Load the state dict (tensor_dict)
    # If the whole model is saved in a single file, then load the state dict directly
    # otherwise, load them separately and merge them into a single state dict

    bin_files = glob(input + '*.bin')
    # input_files = glob(input)
    # print("input files: ", input_files)

    if len(bin_files) == 0:
        ValueError(f"Input {input} does not match any files")
        print(f"Input {input} does not match any files")
        exit(1)

    # Load file(s)
    state_dict = {}
    for file in bin_files:
        print(f"Loading {file}")
        state_dict.update(torch.load(file, torch.device("cpu")))

    # Change dtype
    for key in state_dict:
        state_dict[key] = state_dict[key].to(dtype)

    # Preprocess
    print("Preprocessing")
    preprocessor = PREPROCESSOR[model]
    tensor_dict, num_q_heads, head_dim = preprocessor(state_dict)

    # The final step: divide the weights and save them to files
    print("Resharding and saving weights")
    name_translator = NAME_TRANSLATOR[model]
    divideWeightAndSave(output, tensor_dict, name_translator, num_q_heads, head_dim)

and manually convert before running server

Hi, I find *.bin not exist in weights of Llama-2-7b, I only find consolidated.00.pth, and unfortunately it can not be converted by converter.py
$ python distserve/downloader/converter.py --input /debug/Llama-2-7b/consolidated.00.pth --output /debug/Llama-2-7b/ --model llama
Converting /debug/Llama-2-7b/consolidated.00.pth into torch.jit.script format
Loading /debug/Llama-2-7b/consolidated.00.pth
Preprocessing
Traceback (most recent call last):
File "/ktd/DistServe/distserve/downloader/converter.py", line 463, in
convert_weights(args.input, args.output, dtype, args.model)
File "/ktd/DistServe/distserve/downloader/converter.py", line 439, in convert_weights
tensor_dict, num_q_heads, head_dim = preprocessor(state_dict)
File "/ktd/DistServe/distserve/downloader/converter.py", line 117, in preprocess_llama2
num_layers = max(int(regex.findall(x)[0]) for x in filter(regex.match, tensor_dict)) + 1
ValueError: max() arg is an empty sequence
how to fix this problem...

Hi, I find *.bin not exist in weights of Llama-2-7b, I only find consolidated.00.pth, and unfortunately it can not be converted by converter.py
$ python distserve/downloader/converter.py --input /debug/Llama-2-7b/consolidated.00.pth --output /debug/Llama-2-7b/ --model llama
Converting /debug/Llama-2-7b/consolidated.00.pth into torch.jit.script format Loading /debug/Llama-2-7b/consolidated.00.pth Preprocessing Traceback (most recent call last): File "/ktd/DistServe/distserve/downloader/converter.py", line 463, in convert_weights(args.input, args.output, dtype, args.model) File "/ktd/DistServe/distserve/downloader/converter.py", line 439, in convert_weights tensor_dict, num_q_heads, head_dim = preprocessor(state_dict) File "/ktd/DistServe/distserve/downloader/converter.py", line 117, in preprocess_llama2 num_layers = max(int(regex.findall(x)[0]) for x in filter(regex.match, tensor_dict)) + 1 ValueError: max() arg is an empty sequence
how to fix this problem...

are you sure exist consolidated.00.pth file？ https://huggingface.co/meta-llama/Llama-2-7b-hf/tree/main

Hi, I find *.bin not exist in weights of Llama-2-7b, I only find consolidated.00.pth, and unfortunately it can not be converted by converter.py
$ python distserve/downloader/converter.py --input /debug/Llama-2-7b/consolidated.00.pth --output /debug/Llama-2-7b/ --model llama
Converting /debug/Llama-2-7b/consolidated.00.pth into torch.jit.script format Loading /debug/Llama-2-7b/consolidated.00.pth Preprocessing Traceback (most recent call last): File "/ktd/DistServe/distserve/downloader/converter.py", line 463, in convert_weights(args.input, args.output, dtype, args.model) File "/ktd/DistServe/distserve/downloader/converter.py", line 439, in convert_weights tensor_dict, num_q_heads, head_dim = preprocessor(state_dict) File "/ktd/DistServe/distserve/downloader/converter.py", line 117, in preprocess_llama2 num_layers = max(int(regex.findall(x)[0]) for x in filter(regex.match, tensor_dict)) + 1 ValueError: max() arg is an empty sequence
how to fix this problem...

maybe use  meta-llama/Llama-2-7b-hf？ https://huggingface.co/meta-llama/Llama-2-7b-hf/tree/main


Prepare networking as early as possible

In order to change how we provision networking, especially DHCP, we have
to get the virtual networks as early as possible in place.
Doing so, we'll be able to inject some specific VM, such as the nat64,
and properly manage networks, even for devscripts layout.
As a pull request owner and reviewers, I checked that:

[X] Appropriate testing is done and actually running

@cjeanner testing notes:

[X] CRC layout is passing fine, networks are present and working as expected
[X] 3-OCP layout is passing fine as well, networks are present and working as expected

If I understand correctly the tasks that we're copied from roles/devscripts/tasks/133_host_network.yml into roles/reproducer/tasks/prepare_networking.yml were protected by a when clause:
    - cifmw_devscripts_config.manage_br_bridge is defined
    - cifmw_devscripts_config.manage_br_bridge == 'n'

Is this no longer needed or being applied somewhere else?

It was never really tested in molecule, and I happened to trigger that path by passing that specific override in the molecule test, leading to.... well.. "some" issues with that untested track, mostly due to lacking data that are usually fed by a job definition.... So I changed the override to keep it simple in the other PR.

/lgtm


Rewrite autotune tutorials

https://github.com/scverse/scvi-tools/pull/2561

@ori-kron-wis Close this?


Card docs > Add single stat links

In GitLab by @tauriedavis on Apr 4, 2020, 24:27
In https://gitlab.com/gitlab-org/gitlab-services/design.gitlab.com/-/merge_requests/1609, we reference a single state component. This component was not yet added to Pajamas. Not that it is, link to it and add it is a related component.
Example from GitLab UI:

In GitLab by @tauriedavis on Apr 4, 2020, 24:28
marked this issue as related to gitlab-org/gitlab-design#817

In GitLab by @gtsiolis on Nov 9, 2020, 22:48
mentioned in merge request !2157

In GitLab by @jeldergl on Jul 6, 2021, 22:05
changed the description

In GitLab by @jeldergl on Jul 6, 2021, 22:09
@tauriedavis the design for the single state is in the Data Visualization file, so I think this can be closed. What's in GitLab UI doesn't match the design though, and the design doesn't have hover states like GitLab UI, but I don't remember those states being part of the design since these aren't interactive other than maybe tooltips/popovers.
/cc @npost

In GitLab by @tauriedavis on Jul 7, 2021, 01:25
I think this issue was to link to single stat from the card docs (such as here https://design.gitlab.com/components/card#single-stat). After we add the links, then we can close this. Seems like this is ready for development

In GitLab by @tauriedavis on Jul 7, 2021, 01:25
changed the description

In GitLab by @tauriedavis on Jul 7, 2021, 03:45
Hard for me to say currently but it does look like we do still have related components linked, so this would at least fall under that (unless we dont think they are really that related)

In GitLab by @npost on Jul 7, 2021, 12:22

What's in GitLab UI doesn't match the design though, and the design doesn't have hover states like GitLab UI, but I don't remember those states being part of the design since these aren't interactive other than maybe tooltips/popovers.

They are interactive when they have popovers.

This is to fix the hover state when no popover: https://gitlab.com/gitlab-org/gitlab-ui/-/issues/1488

ard for me to say currently but it does look like we do still have related components linked, so this would at least fall under that (unless we dont think they are really that related)

I wouldn't say they are related really.

In GitLab by @tauriedavis on Dec 17, 2021, 24:19
mentioned in merge request !2673

In GitLab by @tauriedavis on Dec 17, 2021, 24:36
mentioned in merge request !2674

In GitLab by @jareko on Dec 17, 2021, 21:26
mentioned in commit 727c9e42bddbacc345c4bea0274dfa2a2618fd63


What is the difference between input image and reference image?

Thansk for your great work! I have a question about the framework in the paper:

What is the meaning of input image? What is the difference between input image and reference image?
Hope for your reply. Thank you!

Input image is only used in training, which will be added the noise. This is a normal practice of the denoising diffusion model.

Is this the DDIM inversion operation?

Is this similar to the DDIM inversion?

For the better initialization, is it?


Allow CSV reader to read max_depth from CmdStan CSV file

Submission Checklist

[x] Run unit tests: ./runTests.py src/test/unit
[x] Run cpplint: make cpplint
[x] Declare copyright holder and open-source license: see below

Summary
This is a prerequisite needed to fix the bug described in https://github.com/stan-dev/cmdstan/issues/646 and implements the modifications to the CSV reader described in https://discourse.mc-stan.org/t/modifying-stan-csv-reader-to-fix-a-problem-in-cmdstan/5307.
Copyright and Licensing
Please list the copyright holder for the work you are submitting (this will be you or your assignee, such as a university or company): Myself, i.e., J. J. Ramsey
By submitting this pull request, the copyright holder is agreeing to license the submitted work under the following licenses:

Code: BSD 3-clause (https://opensource.org/licenses/BSD-3-Clause)
Documentation: CC-BY 4.0 (https://creativecommons.org/licenses/by/4.0/)

@jjramsey The checks did not pass because of some cpplint issues.  Would you mind running make cpplint locally, fixing the issues, and pushing back up?  Thanks for working on this, I'm looking forward to it.

Thanks @jjramsey for fixing the tab issues.  I started looking through this and realized that there are no tests associated with this code.  Would you please add some tests in  stan/src/test/unit/io/stan_csv_reader_test.cpp?
To ensure your code reads appropriately, would you mind changing the 10 of max_depth = 10 (Default) to another max_depth number in either  stan/src/test/unit/io/test_csv_files/metadata1.csv or  stan/src/test/unit/io/test_csv_files/metadata3.csv?
@bob-carpenter In the name of consistency and clarity, what do you think about changing the field name from max_depth to max_treedepth to match what RStan and PyStan use?  This would need to be changed in CmdStan too.

As long as it doesn't affect any downstream tools, that's fine.   It seems like this would make downstream tooling easier rather than more difficult.

@bob-carpenter @roualdes Could we not change CmdStan argument names without process?

It's not good process for Stan/CmdStan to be chasing naming conventions that rstan/pystan decided on without checking the core code.  This is an ongoing issue and there's no reason to encourage it.

I personally have code for various projects (simulation studies, some in progress) that generates CmdStan argument names.  While I do know enough 'sed' to make changes like this only be a small time sink I don't see the point of doing it now.  I'm sure others are in a similar situation since CmdStan is a great choice on clusters and that's the easiest place to do simulation studies.

@sakrejda You're right, let's hold off.  This is too close to a backward incompatible change, as it may break some user interaction with CmdStan.  Although the field name here would not affect CmdStan's argument name, it may be better to change both at the same time later on, maybe Stan3, if everyone feels it's right to do so.  Thanks for the good thought.

Good point.  This isn't in the printed output category, but something people may need for downstream processing.
So I agree we shouldn't change it.

Looks like the build on Travis CI almost succeeded, except for job 2780.12, which erred out with "The job exceeded the maximum time limit for jobs, and has been terminated."

Sorry---Travis has been performing very poorly lately and timing out a lot of our jobs.  The only recourse I know is to restart them by following the Details link and then rooting around for a job restart (there are probably instructions somewhere online, but it's a moving target).  I'm not sure if external users without merge permission on stan-dev have permission to restart.

I only needed to follow the Travis link, then scroll down to the relevant job and restart.  For some reason it doesn't show it as restarted here.

Ah, now it shows the restart.

If it were just a private people I'd purge it from the git history and force push but not sure what the policies are here on branches.. (@seantalts ?)

Go for it. It’s not in develop yet.
On Tue, Sep 25, 2018 at 9:39 AM Krzysztof Sakrejda<EMAIL_ADDRESS>wrote:

If it were just a private people I'd purge it from the git history and
force push but not sure what the policies are here on branches.. (
@seantalts https://github.com/seantalts ?)
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/stan-dev/stan/pull/2625#issuecomment-424346622, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAZ_F8IzE_XZ9wERFKiZgeinXnJAgcbXks5uejHtgaJpZM4WggB1
.

@jjramsey  if you're not sure which piece of git you're looking for it's
git-reset : https://git-scm.com/book/en/v2/Git-Tools-Reset-Demystified followed by a forced push:
https://stackoverflow.com/questions/10510462/force-git-push-to-overwrite-remote-files
.. you should be able to force push to your own branch.

@jjramsey if you're not sure which piece of git you're looking for it's
git-reset : https://git-scm.com/book/en/v2/Git-Tools-Reset-Demystified followed by a forced push:
https://stackoverflow.com/questions/10510462/force-git-push-to-overwrite-remote-files
.. you should be able to force push to your own branch.

I did the following steps. First I did this, just to ensure that I had a clean slate:
git clone https://github.com/jjramsey/stan.git -b feature/issue-646-csv-reader-max-depth fresh-dir/

Here, fresh-dir is a newly created empty directory. Once inside fresh-dir, I did the following.
git reset --hard
git push -u origin feature/issue-646-csv-reader-max-depth --force

This did not seem to do anything. All I got was the message "Everything up-to-date".
Note that lib/stan_math in the newly checked-out branch is empty.
Really not sure what to do here.

@jjramsey I'll give it a shot.

@jjramsey I think the only issue was that the math submodule was pointing at the wrong thing.  I changed it to point at develop on a branch on stan-dev and it says there are no conflicts:
https://github.com/stan-dev/stan/compare/feature/issue-646-csv-reader-max-depth
Can you look at that and see if it has everything you want?  I didn't want to push to your branch w/o explicit permission.

Can you look at that and see if it has everything you want? I didn't want to push to your branch w/o explicit permission.

Looks good to me.

I still don't know how to get the submodule problem fixed at my end. How do I get the submodule to point to the correct lib/stan_math instead of what appears to be the outdated one?

Can you use/clone the branch I created on stan-dev/stan? It should point to the right math lib.

Sure I can clone the branch, but that's not enough. The problem is how to transfer the changes from that branch to mine. If I can't do that, the pull request can't update to reflect the changes.
I thought about cloning the branch so that its downloaded files would overwrite the files in my own local copy of the branch, but
I'd think that would also overwrite the .git* files and directories as well, and thus just make an even more messed up git configuration.

When I made that branch started from your branch and only updated the math submodule to point to the right place.  It should have everything your branch has, unless you added it while I was working.  In that case take my version of the branch and copy any new files you have over the ones there, add/commit and you should be good.    There's no PR-specific history you need to keep so make another PR, make sure it tests out, close this one and let the new one get reviewed.

Sorry, just catching up on github notifications. A branch is just a pointer to a commit - you can change the commit that a branch points to with git branch -f or git reset (SO link).
I people with merge access (like me and @sakrejda) can actually even push updates to branches involved in PRs from your fork. I can take a look at it if you're still having any trouble @jjramsey.

I think people with merge access (like me and @sakrejda) can actually even push updates to branches involved in PRs from your fork. I can take a look at it if you're still having any trouble @jjramsey.

Please do. I tried creating a new PR from the branch sakrejda created, but it ended up having a conflict with the math submodule, so I closed that too so that I could start over from scratch.
Before I saw your comment, what I was planning to do was create a new branch starting from the latest version of stan-dev/develop, make the changes to src/stan/io/stan_csv_reader.hpp, src/test/unit/io/stan_csv_reader_test.cpp, and src/test/unit/io/test_csv_files/metadata3.csv, and then make a new pull request based on that branch. However, I'd rather hold off on that sort of extreme, ugly solution if you can do something better.

It says that the repository that submitted this pull request has been deleted on github...?

Also I played around with the branch and it seems like the easiest thing was to just git pull origin develop - that correctly changed lib/stan_math to point to the right thing for me.

@seantalts that's odd, when I did it I couldn't update by pulling from develop (the wrong math lib pointer stayed) but it wasn't showing conflicts after I manually moved the math lib to point at the right thing so I don't know what's going on..

It says that the repository that submitted this pull request has been deleted on github...?

 I nuked my old fork and then reforked the stan/stan-dev repo. I'm really screwing things up, aren't I?

There's a make target in stan:
> make math-update

that brings in the currently registered version of the submodule.  You shouldn't ever be committing a new version of the math library.  That's all done automatically in continuous integration.

I'll take care of this one, Bob! I have another idea, but need to push a
branch first...
jjramsey, don't worry, we'll get it sorted.
On Mon, Oct 1, 2018 at 14:09 Bob Carpenter<EMAIL_ADDRESS>wrote:

There's a make target in stan:

make math-update

that brings in the currently registered version of the submodule. You
shouldn't ever be committing a new version of the math library. That's all
done automatically in continuous integration.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/stan-dev/stan/pull/2625#issuecomment-426006979, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAxJ7E-582F_e1ASTHaMrtCoY4EBk7lSks5uglpsgaJpZM4WggB1
.

--
📲

@jjramsey On your fork, can you try the following:
git remote add upstream https://github.com/stan-dev/stan.git
git fetch upstream
git checkout -b feature/issue-646-csv-reader-max-depth upstream/feature/issue-646-csv-reader-max-depth
git push origin feature/issue-646-csv-reader-max-depth

And then open a PR from that branch?

@jjramsey:  If you don't have the synchronized math lib for Stan, git commit -a will commit the wrong version.  I'd suggest running git status before you commit to make sure that you aren't going to accidentally commit the wrong version of stan/math.


Reduce NtileTest timing

Changed data sizes and removed few redundant ntile function invocations(with constant cases) to improve timing. On my laptop the timing reduced from 1m 2 sec to 39s 420ms.

@aditi-pandit Here is the error:
[2023-04-10T07:02:02.764-07:00] Stderr: fbcode/velox/functions/prestosql/window/tests/NtileTest.cpp:48:21: error: loop variable 'function' creates a copy from type 'const std::basic_string<char>' [-Werror,-Wrange-loop-construct]
[CONTEXT] [2023-04-10T07:02:02.764-07:00]     for (const auto function : kNtileInvocations) {
[CONTEXT] [2023-04-10T07:02:02.764-07:00]                     ^
[2023-04-10T07:02:02.767-07:00] Stderr: fbcode/velox/functions/prestosql/window/tests/NtileTest.cpp:48:21: error: loop variable 'function' creates a copy from type 'const std::basic_string<char>' [-Werror,-Wrange-loop-construct]
[CONTEXT] [2023-04-10T07:02:02.767-07:00]     for (const auto function : kNtileInvocations) {
[CONTEXT] [2023-04-10T07:02:02.767-07:00]

@aditi-pandit Here is the error:
[2023-04-10T07:02:02.764-07:00] Stderr: fbcode/velox/functions/prestosql/window/tests/NtileTest.cpp:48:21: error: loop variable 'function' creates a copy from type 'const std::basic_string<char>' [-Werror,-Wrange-loop-construct]
[CONTEXT] [2023-04-10T07:02:02.764-07:00]     for (const auto function : kNtileInvocations) {
[CONTEXT] [2023-04-10T07:02:02.764-07:00]                     ^
[2023-04-10T07:02:02.767-07:00] Stderr: fbcode/velox/functions/prestosql/window/tests/NtileTest.cpp:48:21: error: loop variable 'function' creates a copy from type 'const std::basic_string<char>' [-Werror,-Wrange-loop-construct]
[CONTEXT] [2023-04-10T07:02:02.767-07:00]     for (const auto function : kNtileInvocations) {
[CONTEXT] [2023-04-10T07:02:02.767-07:00]

Thanks @mbasmanova. Have updated the code.


feat(storage): use timeout for filter_extractor acquire

I hereby agree to the terms of the Singularity Data, Inc. Contributor License Agreement.
What's changed and what's your intention?
as title
Please explain IN DETAIL what the changes are in this PR and why they are needed:

use timeout for notify to replace the unbound wait for debug

Checklist

~[ ] I have written necessary rustdoc comments~
~[ ] I have added necessary unit tests and integration tests~
[x] All checks passed in ./risedev check (or alias, ./risedev c)

Refer to a related PR or issue link (optional)

https://github.com/risingwavelabs/risingwave/issues/5437

when timeout
thread 'risingwave-main' panicked at 'called `Result::unwrap()` on an `Err` value: Elapsed(())', /Users/li0k/Documents/github/risingwave/src/storage/hummock_sdk/src/filter_key_extractor.rs:304:66

Can you provide some context? 👀

Can you provide some context? 👀

When creating mv, meta will push the catalog to compute and compactor. When the system has acuqire timeout, it will be a bug (which will cause unbounded wait in main branch), we can find the problem in time by panic over timeout. I simulate the scenario above by breaking the notify code.

@BugenZhao PTAL


As an instructor, I want to be able to create a class and sections within a class so that I can organize questions and students

Story/task details
Instructor might have used the product previously and have old classes lying around. They might be teaching multiple classes or sections of a class at the same time.
Instructor can create separate classes with separate questions. Separate sections have shared questions but track when questions are asked separately.
There should also be a UI for switching courses and sections.
Acceptance scenarios
Given
Instructor has an account and has logged in.
When
There are no classes, or the instructor accesses UI associated with creating new classes.
Then
Instructor can access a UI for creating a new class. This should allow the instructor to specify how many sections a particular class has, what time they are at, etc.
Definition of done
UI + backend for all the above is implemented in the product.

We probably won't do this since we don't have time.


[9.2.1] Cardboard Box crashes on mouseover with certain blocks inside

If you right-click the top side of a door (just an example) with the cardboard box, break it and afterwards mouse over the cardboard box, you will immediately crash to desktop.
Using Mekanism 9.2.1 for Minecraft 1.10

As a side note, you can't unpack the top half of the door either, so you need to destroy the cardboard box without hovering over it in your inventory to get rid of it.

duplicate issue. please check closed issues list.

My bad.


Fatal error: Call to a member function prepare() on null

Im using xammp. and I uploaded the sql. when I try to run the index. this error shows up. can you help me sir.
thank you.

As the error states, no database found


The messages I sent are displayed incorrectly

Hello! When I send a message, it does not stand out that I sent it. My and the other person's messages are displayed on the left, although mine should be displayed on the right and highlighted in blue. Everything works in the demo version, but installing this package Laravel 8 and Jetstream works incorrectly.

My demo is setup to use UUIDs, and my config in messenger.provider_uuids is true. When you installed into a fresh application, are your users using INT / auto-incrementing primary keys now? If they are, you need to be sure to set provider_uuids to false/true to match your user model key types. This used to be a bug, but was fixed some time ago (in my UI since I support UUID or auto-incrementing). Can you check your config in messenger and see if that fixes it?

I have identified the issue and will be implementing a fix shortly. When providers do not use morph maps, the FQN of the model is not parsed correctly into my Messenger.js manager.

@GitesClone This issue has been resolved with my latest version v1.1.0. Be sure to composer update, then re-publish my JS assets: php artisan vendor:publish --tag=messenger-ui.assets --force


Upload images as part of editorial workflow

This pull request is still a bit of a work-in-progress, but I wanted to open a PR here to start a discussion on what needs to be done before we can get this merged in.
This Addresses issue #1344 by allowing users to upload images as part of an editorial workflow, which adds the images to the same branch as collection content, allowing for a couple of things:

The default branch of the repo can be protected (e.g. only certain people approved to merge, require status checks before merge).
Images can be made part of the review process in addition to the entry content.

This has currently only been tested with GitHub, it probably doesn't work on other backends at the moment, this will need to be tested first. It's based on the work of @ro-savage in https://github.com/Encrypt-S/netlify-cms/commit/a86b5e87655dfcb60b3ae743219a614dca034157, but has been rebased ontop of the latest code, and cleaned up a little bit.
Later on down the line, if the editorial workflow is updated to be compatible with fork-and-pr workflows on GitHub, these changes should automatically work there too.

Hey @samlanning thanks for your hard work on this one, looks certainly like something that we want to use (uploading images as part of the PR as we don't allow commits direct to master)
I've straight away run into a problem wherein it looks like I've accidentally changed my PR into a mediaOnlyPr and now I can't see it in the editorial workflow or edit it :( I think I did this because I did a image uploader upload.

@samlanning any thoughts on what @wakeless ran into?

Sorry for not replying to this sooner.
Yeah that makes sense, I think the best way to solve for this would probably be to only allow media to be uploaded once a draft post has been "saved", and a PR opened, then using that existing PR.

Closing this as I'm working on the same issue #1344.
My initial commit is here: https://github.com/netlify/netlify-cms/commit/94934c29bba552543e8d9504bd4fd79043031ccf
I'm taking a slightly different approach of linking the media file to the draft entry and persisting everything together on save. It is still WIP though


Use Logging Services common pages #144

fixes #144

Point "Download" to the LS Download page
Remove the "Security Reports"
Point the "Security" in the left menu to the LS Security page
Replace "Mailing Lists", "Issue Tracking" links with a "Support" link pointing to the LS Support page

@vy can you please check on https://logging.staged.apache.org/log4net/ ?

@FreeAndNil, thanks so much! :heart_eyes:


Empty Email Body for Collective Updates

Describe the bug
From time to time we get email on fresh-desk with an empty email body. They usually stem from the template order.reminder.pendingFinancialContribution.hbs. An example is, https://opencollective.freshdesk.com/a/tickets/320927
I've checked the Mailgun logs and the email body for these in Mailgun is also empty. Therefore I am tempted to believe that we have some sort of bug in our code-base that sends these emails with the empty body.

Thanks for documenting this issue @SudharakaP!
I wasn't able to pinpoint the root cause. The email failed with one of these contributions, but they look pretty normal:

https://opencollective.com/handsoffuhuru/contributions/733277
https://opencollective.com/handsoffuhuru/contributions/733113

https://github.com/opencollective/opencollective-api/pull/9771 will help to investigate similar cases in the future.

@Betree : Yes this is difficult. I've done similar investigations before and brought up this issue and couldn't find what the root cause is. I hope the headers issue that you've mentioned helps solve it. 😺

Another instance where this happened; https://opencollective.freshdesk.com/a/tickets/347003


Change tostring() and fromstring() to tobytes() and frombytes()

tostring() is replaced by tobytes()
fromstring() is replaced by frombytes()

@lienching I'm closing this pull request. Please open a new one making sure you don't include egg file.


Add Azure content mirror

Add a Microsoft Azure blob storage content mirror hosted in Wyoming, USA (West Central US region).
I used the following command to mirror this content:
java -jar ./unreal-archive-exec.jar mirror --content-path=/mnt/ssdsingle/scratch/unreal/unreal-archive-data/ --store=az --az-acc=unrealarchiveusa --az-sas="<secret>" --az-container=files --since=1990-01-01 --concurrency=20
This commit is directly from the unreal-archive-data folder after success. There were no errors at the end of execution.
I have opened a PR into the main repo to add the AZ store class used here: https://github.com/unreal-archive/unreal-archive/pull/47

Awesome, thanks!


OP/Voice options in user context menu

Request the ability to choose +/- Operator and +/- Voice statuses in user context menu for current channel assuming user has sufficient privileges.
Issue is closely related to Issue #347

Closing as duplicate of #347


research: Get a grasp on the LAPS training parameters, to shorten training times.

Even if we feed LAPS a tiny (10 task) training dataset, the training still takes a very long time.
This is very impractical, especially in testing basic functionality.
As LAPS is not some corporate production grade code, the parameters and their description are often incomplete and expect deep knowledge about ML, topic, and the paper at hand.
If we would have at least one team member become an expert on controlling the LAPS execution, the development would become much easier, if not by training other team members, at least by having someone to help with execution and testing.

One thing to look into is lists of primitives, as using all of them might be an overkill when our task dataset is cut to 10 examples.


kms_id and kms_key_id cannot be dynamically provided

Currently the kms_id and kms_key_id are used to determine if kms encryption should be enabled. The terraform fails in the plan stage if either of these values is dynamically provided (i.e. provisioned or looked up by other modules)

Fixed in cloud-native-toolkit/terraform-ibm-ocp-vpc#v1.3.3


MSMPI binary is not redistributable.

cc @conda-forge/core
While MSMPI source code is under MIT license, the binary this package is distributing now is not redistributable.

b) Distribution Requirements. For any files you distribute, you must:
• add significant primary functionality to it in your applications;
• require distributors and external end users to agree to terms that protect it and Microsoft at least as much as this agreement; and
• indemnify, defend, and hold harmless Microsoft from any claims, including attorneys’ fees, related to the distribution or use of your applications, except to the extent that any claim is based solely on the unmodified distributable code.

Please remove this binary packaging and let's go back to building from source code.

It looks like some packages were marked broken in PR ( https://github.com/conda-forge/admin-requests/pull/260 ). Is that what is being referred to? Is this still an issue with the most recent package/recipe?

@leofang I am taking point on this thread for a bit. In order of preference, I'd like to do the following.

move back to building from source, avoiding the need for permission from the msmpi folks
have conda-forge itself get permission for redistribution from the msmpi folks - as it stands now they have given permission to you, but not to any other maintainer nor our bots
remove the repackaged binaries if neither 1 or 2 work out

@leofang Why do we need to repackage their binary at all? I couldn't find an answer to this question in the thread above or any linked issues but I may have missed it.


Is this library still alive?

I've not seen recent updates and there are plenty of interesting pull requests to merge.

Hi @manuelmazzuola
Sorry for the lack of feedback. Unfortunately, I don't have much time for this project so it's not actively maintained.
Also I don't plan to add a lot more functionality because I want to keep it a microlib and there are already other more powerful alternatives out there.
That said I'll try to have a look at the PR this week. But feel free to fork it if you want.

Ok, thank you for your time @typicode  :+1:


To many Eager Loadings

Summary of problem or feature request
If I use to many eager loadings, get error below.

Code snippet of problem
User::query()->with(['metadata', 'roles', 'assets', 'departments', 'managers.param'])->select('users.*')

System details
Windows 10 Home,
PHP 7.0.10
Laravel 5.4
LDT: 8.0

Update:
The problem is not too many relationships, but with the function
convertToArray($row) in file Helper.php line 156
It have problem with convert object from json:
{"doors": {"1": "1", "9": "9", "20": "20", "25": "25", "26": "26", "27": "27", "28": "28", "31": "31", "33": "33", "36": "36", "40": "40", "41": "41", "45": "45"}}

@rudashi thanks for digging this further. If possible, can you please submit a PR for the patch? Thanks!

Currently I have no idea how to fix it properly.

More research:
First dump is for $data in line 159
Second for $key in the same line.
If we try too check $data[$key] everything will crash.
When I try to check isset( $data[$key]) I get false.
Screenshot:


Revamp C# native loader

P/Invoke native method loading in .NET is somewhat dumb and it doesn't allow configuring where to look for the native library (see http://www.mono-project.com/docs/advanced/pinvoke/). That requires the native library to be installed (e.g. in /usr/local/lib or pointed to by LD_LIBRARY_PATH). Also, it doesn't allow selecting the right binary based on current environment bitness.
This pull request adds an intelligent loader that can autodetect where the application is currently running (windows/linux/mac and 64bit/32bit) and based on that, it loads the right version of the native library.
This is a step towards being able to ship a nuget package that has precompiled grpc_csharp_ext for all platforms and is standalone (meaning it doesn't require you to preinstall anything when on Linux/Mac).

I'll need to update copyrights.

Copyrights should be up-to-date now (tools/distrib/check_copyright.py --fix)


[YUNIKORN-1944] calculate headroom for wildcard cases

What is this PR for?
Can't get the correct headroom for the user wildcard limit.
What type of PR is it?

[X] - Bug Fix

What is the Jira issue?
https://issues.apache.org/jira/browse/YUNIKORN-1944
How should this be tested?
Covered by unit tests.

For example,
root.p1.c1 -> p1: wild user “” config is 10 cores, c1 -> user “user1" config is 11 cores. Is this allowed? Since we are comparing c1:user1:cores with p1::cores for headroom calculation, we should this validation in config parsing.
Please file a separate jira to handle this.

I think the parent level wildcard limit lower than child level user limit is not a problem, because we check whether there is childHeadrrom first. If there is childHeadroom, we don't check wildcard limit in parent level.
However, the reverse case is a problem. We need to check whether parent level user limit is lower than child level wildcard limit. Thanks for catching this. 👍 I will file a jira after this PR is merged.

Since this pr is bit old, Can you rebase and run the tests again?

Since this pr is bit old, Can you rebase and run the tests again?

Updated the comment and rebase it. Thanks

For example,
root.p1.c1 -> p1: wild user “” config is 10 cores, c1 -> user “user1" config is 11 cores. Is this allowed? Since we are comparing c1:user1:cores with p1::cores for headroom calculation, we should this validation in config parsing.
Please file a separate jira to handle this.

I think the parent level wildcard limit lower than child level user limit is not a problem, because we check whether there is childHeadrrom first. If there is childHeadroom, we don't check wildcard limit in parent level.
However, the reverse case is a problem. We need to check whether parent level user limit is lower than child level wildcard limit. Thanks for catching this. 👍 I will file a jira after this PR is merged.

Create YUNIKORN-2119 to follow up.


workaround: for ps2sdk, fallback to single job if multi job failed

can be that windows job fail due to j4 in gskit?


Add Movement CLI installation and usage instructions to setup.md

This covers the basics of getting setup with Movement CLI (installation and usage).
Please feel free to edit as needed for style consistency with the rest of the docs.
Important: Do not merge until public testnet is released.

@PrasoonPratham @Rahat-ch added a note that this should not be merged until public testnet is live, because movement init default testnet is Suzuka testnet which we don't want spammed too hard yet.


Further improvements

More details for README.md and documentation,
Support polytope for CYHypersurface,
First non-trivial test conducted,
Expected higher code coverage.

Codecov Report

Merging #11 (45c3639) into master (3b90e73) will increase coverage by 2.72%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master      #11      +/-   ##
==========================================
+ Coverage   70.00%   72.72%   +2.72%
==========================================
  Files           2        3       +1
  Lines          10       11       +1
==========================================
+ Hits            7        8       +1
  Misses          3        3

Impacted Files
Coverage Δ

src/CYTools.jl
100.00% <ø> (ø)

src/CYHypersurfaces/constructors.jl
50.00% <100.00%> (ø)

src/CYHypersurfaces/properties.jl
100.00% <100.00%> (ø)

:mega: Codecov can now indicate which changes are the most critical in Pull Requests. Learn more


Create a jenkins job to run tests

Description:
Setup a jenkins job to run all the unit tests.

Setup Pipeline 'dev-ballerina-integrator' in https://jenkins-support-ei.wso2.com/jenkins/

We will need to create a public Jenkins build once the Gradle build is complete #137

Related PRs:
https://github.com/wso2/ballerina-integrator/pull/327
https://github.com/wso2/ballerina-integrator/pull/323
https://github.com/wso2/ballerina-integrator/pull/320
https://github.com/wso2/ballerina-integrator/pull/319
https://github.com/wso2/ballerina-integrator/pull/306

Fixed, hence resolving the issue.


After login must refresh screen to show items

For some reason, after you login the first time, you must hit refresh to see the items. Need to fix.

Seems to be fixed now..


[BUG] AttributeError: 'DeferredItemListIterator' object has no attribute 'sorted_by'

To Reproduce
    with TikTokAPI() as api:
        utc = pytz.UTC
        challenge = api.challenge(tag_name)
        videos = takewhile(lambda video: video.create_time > utc.localize(SINCE_DATE),
            dropwhile(lambda video: video.create_time > utc.localize(UNTIL_DATE),
                challenge.videos.sorted_by(key=lambda video: video.create_time, reverse=True)))

Expected behavior
a sorted iterator of TikTok videos
Version Information
pydantic==2.0.3
pydantic_core==2.3.0
playwright==1.36.0
tiktokapipy==0.2.0.post2
System Information
OS: Windows 11 on desktop computer
Region Information
Thailand

This feature was removed in version 0.2.0. If I forgot to remove it from the documentation, then I'll have to remove it. TikTok supplies videos under a certain tag according to its algorithm, so there's no direct way to get the video list sorted by create time. The removed solution just sorted a fixed number of grabbed videos. To get the same effect you can use:
sorted(challenge.videos, key=...)

to collect all of the videos tagged with a challenge and sort them (this is a bad idea).
Or you can use:
sorted(challenge.videos.limit(20), key=...)

to collect the first 20 videos and sort them. This number can obviously be changed just by changing the limit parameter.


Separate concerns into targets/modules

❕ Problem Statement
Currently the main target/module AppStoreConnectCLI imports all other dependencies. As we move past simple commands into more complex synchronisation it'll be more important to separate our concerns.
💬 Task Description
Separate different concepts into their own modules, namely models and their encodings moving to Model and network calls and transformations moving to Network. This leaves the main target to deal with command line input and translating intent to the correct services to call down to. This also alleviates name collisions
👩🔧 Technical Design Notes
Model
Houses our concept of stored and rendered data for the CLI tool

Includes models structs
Includes information about how to serialise the models in their respective formats

e.g. JSON, CSV, YAML, Tables

As such model will import Yams, SwiftyTextTable and CodableCSV
Network
Takes in input options and translates these to API calls. Translates API models returned from the AppStoreConnectSDK into CLI Models to return to the caller.

Houses extension on Model to initialise them with API data
Should expose functions to return requested CLI Models
Keeps it's reactive implementation internal so that the caller can call it's functions in a blocking manner

Network would import Model, Combine and AppStoreConnect-Swift-SDK
As this is where a lot of the logic is housed there is potential for it to be broken up further. But that could stay internal to the Network module... It's definitely the spot that would need the highest test coverage
Filesystem
Exposes functions to retrieve CLI models or write CLI models from disk

Knows the concept of a sync filesystem setup

e.g. default locations and folder heirarcy

Pulls data from disk turning it into CLI models or writes CLI models to disk in the right (or provided) locations

Filesystem would depend on Model and Files
AppStoreConnectSDK
Takes command line input, validates it and calls the appropriate functions on Network or Filesystem

Owns the concepts that come with the command line

Validates command line input
Has a renderer to output serialised CLI models to stdout and errors to stderr

The entrypoint to the CLI tool would import ArgumentParser, Model, Network and Filesystem
🤝 Relationships
Supersedes the domain layer concept described in #41

This is just a draft, feedback on the design is welcomed


[Coder] Coder Should Preserve Tab Selection When Changing Class

Coder does not preserve tab selection when selecting another class from the tree. This makes it hard to browse. Use case: I'm looking through all the GtPhlowProtoView subclasses to find the one I need. I have to click the Comment tab over and over to see what the class does.

FYI still an issue in v1.0.405


Use pygal plot with Remi

Hello @dddomodossola,
it's me again. Remi is running great so far. Now I want to enhance my app with a chart. I wanted to create the chart with the library pygal which is a very easy plot library with output format SVG. I thought because there are already some svg widgets it should be possible to insert such a plot in my gui.
Unfortunately I can't figure out how I can show the plot and also update it during runtime. Could you please give me a hint if this should be possible and if yes, how I could do it?
Best regards!

Hello @shirtomatik ,
pygal seems a really powerful library. You can add your graph as a standard Image widget, or as a background for a basic Widget. Have you already tried it?

@shirtomatik look at this:
import remi.gui as gui
from remi import start, App
import remi.server
import os
import time
import pygal

class PyGal(gui.Widget):
    def set_content(self, chart):
        self.data = chart.render() #_data_uri()
        i = int(time.time() * 1e6)
        self.style['background-image'] = "url('/%s/download?update_index=%d')" % (id(self), i)
        self.style['background-repeat'] = "no-repeat"
        #self.style['background-image'] = "url('data:image/svg+xml;base64,%s')"%self.data

    def download(self, update_index=0):
        print("download")
        content = self.data
        headers = {'Content-type': 'image/svg+xml'}
        return [content, headers]

class MyApp(App):
    def main(self):
        #creating a container VBox type, vertical (you can use also HBox or Widget)
        self.main_container = gui.VBox(width=300, height=300)

        self.pygal_container = PyGal(width=300, height=300)

        hist = pygal.Histogram()
        hist.add('Wide bars', [(5, 0, 10), (4, 5, 13), (2, 0, 15)])
        hist.add('Narrow bars',  [(10, 1, 2), (12, 4, 4.5), (8, 11, 13)])
        self.pygal_container.set_content(hist)

        self.main_container.append([self.pygal_container])

        # returning the root widget
        return self.main_container

if __name__ == "__main__":
    # starts the webserver
    start(MyApp, address='<IP_ADDRESS>', port=8081, start_browser=True)

even better:
import remi.gui as gui
from remi import start, App
import remi.server
import os
import time
import pygal

class PyGal(gui.Svg):
    def set_content(self, chart):
        self.data = chart.render()
        self.add_child("chart", self.data)

class MyApp(App):
    def main(self):
        #creating a container VBox type, vertical (you can use also HBox or Widget)
        self.main_container = gui.VBox(width=300, height=300)

        self.pygal_container = PyGal(width=300, height=300)

        hist = pygal.Histogram()
        hist.add('Wide bars', [(5, 0, 10), (4, 5, 13), (2, 0, 15)])
        hist.add('Narrow bars',  [(10, 1, 2), (12, 4, 4.5), (8, 11, 13)])
        self.pygal_container.set_content(hist)

        self.bt_test = gui.Button("change chart")
        self.bt_test.onclick.connect(self.ontest)
        self.main_container.append([self.bt_test,self.pygal_container])

        # returning the root widget
        return self.main_container

    def ontest(self, emitter):
        hist = pygal.Histogram()
        hist.add('Wide bars', [(5, 0, 10), (4, 5, 13), (2, 0, 7)])
        hist.add('Narrow bars',  [(1, 1, 1), (1, 4, 4.5), (1, 1, 13)])
        self.pygal_container.set_content(hist)

if __name__ == "__main__":
    # starts the webserver
    start(MyApp, address='<IP_ADDRESS>', port=8081, start_browser=True)

Hello @dddomodossola ,
yes I already tried some hours to get it to work. I tried to use static image and load image from svg. So in pygal I did chart.render_to_file('/tmp/chart.svg')
I also tried to use svg container, but didn't know that I just can append the svg content as child and update the content respectively. Thank you very much for this great example. I will manage it with this to integrate a chart in my app. If I've done this, I will set this thread to solved / closed.
btw: pygal is really also a very nice libary...
Thank you very much - you are the best :+1:


Arlec Grid Connect pc44ha Outdoor Plug [BK7231T / WB2S / BL0937]

Hey
I have a outdoor Arlec Grid Connect plug (pc44ha). I have successfully flashed the kickstart esp home on it (it was using 1.3.5 firmware as the arlec pc191ha). This is the plug - https://www.bunnings.com.au/arlec-grid-connect-smart-ip44-heavy-duty-timer_p0273373
Using the work from #84 and some other templates etc I have found around the place I have created the following esphome yaml, but it's not really correct. Can anyone help figure out what I am doing wrong?
There is talk of the device here including tear-down and pin-out info - https://www.elektroda.com/rtvforum/topic3980140.html. Based on the talk here I think the pin selection is correct, but obviously I am missing something RE power monitoring.
EDIT: I should mention that I am mainly using this plug to monitor power use of my pool pump + filter. I very rarely use the switch. It does turn the device on and off from the web-portal and the button on the device, but the power monitoring is completely wrong as logs here show. This was compared to a kilawatt type plug on the same load (~15-18 watts). I am in Aus so voltage is ~230-250V
13:46:19 | [D] | [hlw8012:082] | Got power=2.7W, voltage=0.7V
13:46:19 | [D] | [sensor:093] | 'Voltage': Sending state 0.65699 V with 1 decimals of accuracy
13:46:19 | [D] | [sensor:093] | 'Power': Sending state 2.65077 W with 1 decimals of accuracy
13:46:19 | [D] | [sensor:093] | 'Daily Consumption': Sending state 0.00109 kWh with 3 decimals of accuracy

esphome yaml
substitutions:
  device_name: pool_pump_socket
  device_friendly_name: "Pool Pump Socket"
  device_ip: <IP_ADDRESS>
  outlet_icon: mdi:power-socket-au
  switch_restore: ALWAYS_ON

  # Higher value gives lower watt readout
  current_res: "0.0009 ohm"
  # Lower value gives lower voltage readout
  voltage_div: "769"

esphome:
  name: ${device_name}
  friendly_name: ${device_friendly_name}
  #name_add_mac_suffix: true

libretiny:
  board: wb2s
  framework:
    version: dev

logger:

web_server:
  port: 80

api:
  reboot_timeout: 15min
  encryption:
    key: !secret api_key_pool_pump

ota:
  password: !secret ota_password

captive_portal:

mdns:

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password
  manual_ip:
    static_ip: ${device_ip}
    gateway: <IP_ADDRESS>
    subnet: <IP_ADDRESS>
    dns1: <IP_ADDRESS>
  ap:
    password: !secret ap_password

button:
  - platform: restart
    name: Restart

debug:
  update_interval: 30s

sensor:
  - platform: uptime
    name: Uptime
  - platform: hlw8012
    model: BL0937
    cf_pin:
      number: P7
      inverted: true
    cf1_pin:
      number: P8
      inverted: true
    sel_pin:
      number: P24
      inverted: true
    current:
      name: Current
      #unit_of_measurement: A
    voltage:
      name: Voltage
      #unit_of_measurement: V
    power:
      id: ${device_name}_power
      name: Power
      #unit_of_measurement: W
    current_resistor: ${current_res}
    voltage_divider: ${voltage_div}
    change_mode_every: 8
    update_interval: 15s
  - platform: total_daily_energy
    name: "Daily Consumption"
    power_id: ${device_name}_power
    filters:
      - multiply: 0.001
    unit_of_measurement: kWh

  - platform: wifi_signal
    name: RSSI
    update_interval: 5min

  - platform: uptime
    id: uptime_sec
    name: Uptime
    update_interval: 5min

text_sensor:
  - platform: debug
    reset_reason:
      name: "Reset Reason"
  - platform: libretiny
    version:
      name: "Libretiny Version"
  - platform: template
    name: "Up Format"
    lambda: |-
      uint32_t dur = id(uptime_sec).state;
      int dys = 0;
      int hrs = 0;
      int mnts = 0;
      if (dur > 86399) {
        dys = trunc(dur / 86400);
        dur = dur - (dys * 86400);
      }
      if (dur > 3599) {
        hrs = trunc(dur / 3600);
        dur = dur - (hrs * 3600);
      }
      if (dur > 59) {
        mnts = trunc(dur / 60);
        dur = dur - (mnts * 60);
      }
      char buffer[17];
      sprintf(buffer, "%ud %02uh %02um %02us", dys, hrs, mnts, dur);
      return {buffer};
    icon: mdi:clock-start
    update_interval: 5min
  - platform: wifi_info
    ip_address:
      name: "IP Address"
      disabled_by_default: false

output:
  - platform: ledc
    id: output_led_1
    pin: P11

light:
  - platform: monochromatic
    id: light_switch_1
    output: output_led_1

binary_sensor:
  - platform: gpio
    id: binary_switch_1
    pin:
      number: P10
      inverted: true
      mode: INPUT_PULLUP
    on_press:
      then:
        - switch.toggle: outlet

switch:
  - platform: gpio
    id: outlet
    name: None
    pin: P26
    icon: ${outlet_icon}
    on_turn_on:
      - light.turn_on: light_switch_1
    on_turn_off:
      - light.turn_off: light_switch_1
    restore_mode: ${switch_restore}
    device_class: outlet

status_led:
  pin: P6

time:
  - platform: homeassistant
    id: homeassistant_time

I have created the following esphome yaml, but it's not really correct. Can anyone help figure out what I am doing wrong?

You're going to have to be more specific.  What isn't correct?  Is it simply that your voltage and wattage readings are incorrect?  You need to adjust the current resistor and voltage divider seen here in your code:
  # Higher value gives lower watt readout
  current_res: "0.0009 ohm"
  # Lower value gives lower voltage readout
  voltage_div: "769"

According to https://upk.libretiny.eu/ for the device you specified, the current resistor should be 0.001 ohm and usually that doesn't change, in which case you need to change the voltage divider substantially until you get closer to real values.  Again, the device example shows voltage divider should be closer to 1600.
This isn't really a LibreTiny issue, but general ESPHome component configuration.  See https://esphome.io/components/sensor/hlw8012.html?highlight=voltage_divider for notes about setting the voltage divider.

Thanks for the reply - yes my power metering is incorrect. Even when I change the voltage_div: 1600, I still get wildly inaccurate voltage. To get Voltage anywhere near
  # Higher value gives lower watt readout
  current_res: "0.001 ohm"
  # Lower value gives lower voltage readout
  voltage_div: "1600"

This is with a 9.9-10W constant draw on the circuit from a dc vornado fan
Voltage is so far from where it needs to be, I will work on current/power first
13:11:27 | [D] | [hlw8012:076] | Got power=2.9W, current=8.9A
13:11:27 | [D] | [sensor:093] | 'Current': Sending state 8.86578 A with 2 decimals of accuracy
13:11:27 | [D] | [sensor:093] | 'Power': Sending state 2.94147 W with 1 decimals of accuracy
13:11:40 | [D] | [sensor:093] | 'Uptime': Sending state 837.06000 s with 0 decimals of accuracy
13:11:42 | [D] | [sensor:093] | 'Power': Sending state 3.03339 W with 1 decimals of accuracy
13:11:57 | [D] | [hlw8012:082] | Got power=2.9W, voltage=1.5V
13:11:57 | [D] | [sensor:093] | 'Voltage': Sending state 1.51040 V with 1 decimals of accuracy
13:11:57 | [D] | [sensor:093] | 'Power': Sending state 2.94147 W with 1 decimals of accuracy
13:12:12 | [D] | [hlw8012:082] | Got power=2.9W, voltage=1.5V
13:12:12 | [D] | [sensor:093] | 'Voltage': Sending state 1.51884 V with 1 decimals of accuracy
13:12:12 | [D] | [sensor:093] | 'Power': Sending state 2.94147 W with 1 decimals of accuracy

So to get 2.94147W to be 10W I would need to multiple by 3.399661. I can't really see in the esp-home docs how to figure out what to do with the current res value to get to the value I want though...

Some of these chips are just really inaccurate and you can't correct them with voltage dividers or current resistors.  The generally recommendation is to measure a known load and use a calibrate_linear filter on the fields that are off.
I have several plugs by the same brand and one of them is massively off from all the others, calibrate_linear is the only way I can get usable results with it.

Ok cool, calibrating these things is a job for another day then! I'll put a pin in this one for now and read up on that process when I have the time.
Thanks!

I'll close this since it's not an issue. If you need any more help with this, feel free to reopen it whenever you like.

I know this is an old issue but I'm having the same problem. I'm pulling my hair out trying to get this right. @tbgoose did you ever  solve this?

Hi, I didn't end up fixing it. I ended up scrapping all my arlec/tuya plugs for zigbee versions.

Thanks for replying. I had a feeling you would say that. Did you find a suitable outdoor ZigBee pm plug? I only bought this because it's weather proof. Anything you recommend?


Add tagDocs parameter to docs pipeline

Changes
Ability to manually tag the docs w/o creating a new release.
Testing
Ran the docs pipeline to verify.

@Mergifyio backport master


interpolate.splder() failure on Fedora

From @opoplawski: With Fedora Rawhide (but not Fedora 19) I'm seeing:
ERROR: test_fitpack.TestSplder.test_kink
----------------------------------------------------------------------
Traceback (most recent call last):
   File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
     self.test(*self.arg)
   File
"/builddir/build/BUILDROOT/scipy-0.13.0-0.1.b1.fc21.x86_64/usr/lib64/python2.7/site-packages/scipy/interpolate/tests/test_fitpack.py",
line 326, in test_kink
     splder(spl2, 2)  # Should work
   File
"/builddir/build/BUILDROOT/scipy-0.13.0-0.1.b1.fc21.x86_64/usr/lib64/python2.7/site-packages/scipy/interpolate/fitpack.py",
line 1186, in splder
     "and is not differentiable %d times") % n)
ValueError: The spline has internal repeated knots and is not differentiable 2
times

and the same with python3.

@rgommers
hello,I am have the problem that ValueError: The spline has internal repeated knots and is not differentiable 2 times.Do you solve the problems?


URGENT: Unusual system clock change detected, soft-evicting connections from pool.

Brett,
I saw a ton of connections from a particular host last night...
dgcs=# select distinct usename, application_name, client_addr, count(*) from pg_stat_activity where client_addr = '<IP_ADDRESS>' group by  usename, application_name, client_addr order by client_addr, usename;
     usename      | application_name |  client_addr  | count
------------------+------------------+---------------+-------
 analytics_rw     |                  | <IP_ADDRESS> |   289
 catalog_rw       |                  | <IP_ADDRESS> |   192
 dgwatch_rw       |                  | <IP_ADDRESS> |    40
 member_rw        |                  | <IP_ADDRESS> |   508
 wps_processor_rw |                  | <IP_ADDRESS> |   200
(5 rows)

This morning, come to find the following in the log files every hour or so...
2016-01-25 21:09:15,498 WARN  [a3808e51-ff45-4d72-b5de-7efdd973ec27] [Hikari housekeeper (pool 1-MemberConnectionPool)] [pool.HikariPool] - 1-MemberConnectionPool - Unusual system clock change detected, soft-evicting connections from pool.
2016-01-25 21:09:17,948 WARN  [a3808e51-ff45-4d72-b5de-7efdd973ec27] [Hikari housekeeper (pool 2-AnalyticsConnectionPool)] [pool.HikariPool] - 2-AnalyticsConnectionPool - Unusual system clock change detected, soft-evicting connections from pool.
2016-01-25 22:09:44,837 WARN  [a3808e51-ff45-4d72-b5de-7efdd973ec27] [Hikari housekeeper (pool 1-MemberConnectionPool)] [pool.HikariPool] - 1-MemberConnectionPool - Unusual system clock change detected, soft-evicting connections from pool.
2016-01-25 22:09:47,267 WARN  [a3808e51-ff45-4d72-b5de-7efdd973ec27] [Hikari housekeeper (pool 2-AnalyticsConnectionPool)] [pool.HikariPool] - 2-AnalyticsConnectionPool - Unusual system clock change detected, soft-evicting connections from pool.
2016-01-25 23:10:45,845 WARN  [a3808e51-ff45-4d72-b5de-7efdd973ec27] [Hikari housekeeper (pool 1-MemberConnectionPool)] [pool.HikariPool] - 1-MemberConnectionPool - Unusual system clock change detected, soft-evicting connections from pool.
2016-01-25 23:10:48,285 WARN  [a3808e51-ff45-4d72-b5de-7efdd973ec27] [Hikari housekeeper (pool 2-AnalyticsConnectionPool)] [pool.HikariPool] - 2-AnalyticsConnectionPool - Unusual system clock change detected, soft-evicting connections from pool.

These are my configs:
<maxLifetime>1800000</maxLifetime>
<idleTimeout>600000</idleTimeout>
<mininumPoolSize>5</mininumPoolSize>
<maximumPoolSize>100</maximumPoolSize>
<connectionTimeout>30000</connectionTimeout>
<validationTimeout>5000</validationTimeout>

What is causing this?  And, what exactly is soft-evicting?  Does that fail to close the original connections?
Thanks!

The warning states your system clock has changed by an unusual amount of time (either backwards or into the future). This can happen when you update the system clock manually or using NTP and the clock drifted too much away from the real time.
Also, there was a bug in an older version of HikariCP which prevented HikariCP from recovering from that state and thus every time HikariCP's HouseKeeper Thread had run, connections had been soft-evicted again. So if you are not using the newest version of HikariCP I would recommend to update the library and then you should check whats up with your system clock.
soft-eviction itself means that HikariCP either closes the connection immediately if it is not handed out to the application or only marks it for eviction so HikariCP can evict it later when the connection has been returned to the pool again. If a connection is only marked, eviction happens on pool.getConnection() calls so it should happen naturally over time based on your pool usage.
After a rather quick look at the code it seems like the pool should not go rogue and suddenly create more connections than usual when the soft-eviction log appears because of system clock issues. But since your maximumPoolSize is set to 100 you might want to check how your application code behaves when the system clock suddenly changes by a larger amount (maybe even backwards). Maybe your application then simply starts to do more stuff which causes HikariCP to build up more physical connections to your DB to handle the getConnection() pressure.

I'm on the latest version 2.4.3.

Also, the machine was just sitting idle all night.  Nothing going on on it.

@victropolis Just to be complete: the log output can also occur if the JVM hangs for more than a minute (more precise: 2 * house keeper thread period in case you have configured your own period). But if the machine was idle thats probably not the case.
The code is pretty simple, you can see it here: https://github.com/brettwooldridge/HikariCP/blob/HikariCP-2.4.3/src/main/java/com/zaxxer/hikari/pool/HikariPool.java#L601
If it continues to happen you may want to turn on DEBUG logging. Don't be afraid, HikariCP does not log a lot more when going down to DEBUG level.

Is it possible that the following is not closing the connections, but only marking them as evicted?  It's being called with owner = false.
   private void softEvictConnection(final PoolEntry poolEntry, final String reason, final boolean owner)
   {
      if (owner || connectionBag.reserve(poolEntry)) {
         closeConnection(poolEntry, reason);
      }
      else {
         poolEntry.markEvicted();
      }
   }

@victropolis As a result of this issue, we're going to improve the logging of this message to reflect how much and in which direction the system clock shifted.
Regarding "soft eviction", you are correct that some connections are only being marked for eviction --  those that are in-use.  Connections that are not in-use (i.e. idle) are closed immediately.  Closing an in-use connection would disrupt a transaction in-progress, which is not our intent.
The issue is, when the system clock jumps backward or too far forward, we can no longer rely on our calculations for idle time, lifetime, or various timeouts.  So, we want to clear all connections created before the time-rift.  But as I said above, we do not want to disrupt in-use Connections.  Connections that are in-use will eventually be returned to the pool -- if they are not that is a different problem, an application-level connection leak -- and when they do return they will be evicted (closed).
We cannot know in this case whether the clock drifted backward or jumped forward.  Improved logging will help us to understand the magnitude of changes.
Currently, if the clock drifts backward at all we evict connections.  If the clock jumps forward by more than one minute, we also evict connections.  As @jnehlmeier mentioned, NTP might adjust time in the negative direction.  It is possible HikariCP needs to tolerate minor backward time movement if NTP is causing micro-second or nano-second backward adjustments -- and only evict connections if the millisecond threshold is crossed.

Brett, thanks for the comment, but I don't necessarily agree with this issue being closed just yet.  Thankfully, this happened on a development laptop.  If this happens in production, this could have harmful effects if postgres runs out of connection slots.  How is it possible to go from 15 member_rw connections (3 wars each with a min 5 connection datasource) to 508 member_rw connections on a machine that's sitting idle?

I will reopen pending further investigation.  If you can reproduce it with DEBUG logging enabled, it would be extremely useful.

Thanks @brettwooldridge .  I'm slammed right now for a deadline, but I'll try to get with him to enable to the extra logging.  In the meantime, looks like it happened again.

@victropolis If you're running on a laptop there are a lot of things that can happen.  Some operating systems rely on the CPU TSC (Time Stamp Counter) for high-resolution time measurements, but the TSC can be affected by power-saving modes providing unreliable time stamps as the computer transitions in and out of power-saving levels.
Power saving modes on laptops also affect the network stack, disabling the NIC without actually disconnecting sessions, and then coming back alive at some later time.  This can wreak havoc on applications.
Finally, power saving levels throttle CPU and disable cores, at more aggressive levels as the processor enters deeper levels of sleep.
| How is it possible to go from 15 member_rw connections (3 wars each with a min 5 connection
| datasource) to 508 member_rw connections on a machine that's sitting idle?
I could imagine several scenarios.  The DEBUG log will shed more light.  Here is one possible scenario:
Your pools are configured for 100 max. connections.  This is extremely high by the way.  Between three pools it is clear that 300 connections are theoretically possible, right?
Your pools also have an idle timeout of 5 min. and a max. lifetime of 30 min.  Let's say that each pool has 5 connections, and the pool is truly idle.  HikariCP will not evict the 5 connections due to idleness, because that would bring the pool below minimumIdle, but it will evict them when they reach maxLifetime, and will evict them if the system clock goes backwards by more than 30 seconds or jumps forward by more than 1 minute.
Clearly we recorded unusual time motion, so evictions occurred.  My current theory is that the process/thread was starved either due to sleep or some other event (virus scan, etc.)  This would appear as a time leap forward to HikariCP.
Once evictions have occurred, HikariCP will try to refill the pool to maintain minimumIdle.  If there are 0 connections, then 5 "add connection" tasks will be enqueued.  This happens on a separate thread from the housekeeper.  If, for whatever reason, connections are not added by the time the housekeeper runs again (in 30 seconds), refill is called again.  5 more "add connection" events are added to the queue.
If this continues, meaning either 1) the starvation of threads, or 2) something preventing Connection creation, the queue of "add connection" tasks will reach maximumPoolSize before it starts dropping requests to queue tasks.  Eventually, if the (system wakes up|starvation ends|connectivity is restored) it will blast through the queue and create maximumPoolSize of "pent-up" add connection requests.
That is one theoretical scenario.  Might I suggest running some other monitoring software on the laptop/server to measure CPU, disk, network.  Troubleshooting 101 is to correlate measurements.  Looking at one datapoint, such as connections on the DB tells us nothing about what is happening on the machine actually responsible for creating connections.

hibernate/sleeping/standby modes can be tricky to handle see: https://bugs.openjdk.java.net/browse/JDK-8074074

@brettwooldridge check this out.  Both of the following set of connections are actually from the SAME box.  the 10.100's are from the ethernet interface and the 10.101's are from the wifi interface.  Have you ever seen it where HikariCP establishes one set of connections per network interface?

@brettwooldridge just saw your post above.  thanks for the info.  Will see what I can glean tomorrow.  It IS a laptop (a MacBook Pro).  We never saw this issue with Proxool, the ancient and unsupported CP I'm replacing with HikariCP.  Many, if not most of our devs use laptops.  It cannot be that HikariCP doesn't play nice with laptops, can it?  I'm not sure how to proceed.  The fact that our dev postgres instance is regularly running out of connection slots is concerning some folks around here.  I will say that the containers that are running on our servers have not exhibited this behavior, so that is good.

@victropolis HikariCP is completely unaware of the physical interfaces.  Which interface a request is routed out of is an OS-level decision based on the default routes.  How MacOS X is determining route priority I don't know.
In running through my theoretical scenario it is clear that the fillPool() method invoked by the housekeeper need not enqueue any more "add connection" tasks if there are already more than minimumIdle tasks awaiting execution (blocked for whatever reason).  This would prevent a spike that would occur after the (starvation | connectivity) condition ends.
Either way, even with the spike, I would expect a return to minimumIdle within 5 minutes if the pool is truly idle.

@brettwooldridge regarding the max connections of 100, I agree that it's high.  We had our min and max set to 5 and 100 respectively previously (with proxool).  I've maintained those configs for now.  I plan on running some load and performance testing with some lower numbers before going to production in 10 weeks.

@brettwooldridge the dba's blocked connections from those IPs tonight so that our builds and tests would stop failing due to insufficient connection slots.  Tomorrow I'll ask them to unblock them again and ask the dev to leave his container up all day and to run that command when I see the situation happening again.  Thanks!

@victropolis pls do not forget to change log level to DEBUG

@brettwooldridge some findings.  I had him disable his wifi connection today and we still saw the ramp up in connections (no debug logging yet, that's next).  However, after he shutdown his container, postgres still thinks there are connections.
netstat -a | grep <IP_ADDRESS> | wc -l
695

Netstat returns 695 established connections on the db host.  But netstat returns 0 connections on the client.
I'll report what we find with the debug turned on.

@brettwooldridge struggling with getting the logging working since we use log4j.  Added dependency on slf4j-log4j12, but still not seeing it.  Still trying.

You'll need to set com.zaxxer to DEBUG in the log4j config.

That grep and wc will also list CLOSE_WAIT connections.  You only care about ESTABLISHED.

the were all ESTABLISHED.

@brettwooldridge OK, he left tomcat running on his laptop overnight again with DEBUG logging.
Currently, the database host shows 56 established connections from his ethernet interface:
netstat -a | grep <IP_ADDRESS> | grep ESTABLISHED | wc -l
56

And, 1150 established connections from his wifi interface:
netstat -a | grep <IP_ADDRESS> | grep ESTABLISHED | wc -l
1150

In the morning we'll check his logs and netstat and I'll report on what I find.

@victropolis Any way to boot his connections from the server before leaving for the night?  Would be good to have a baseline.
Maybe stupid question, but I do have to ask, is there any other application running on the laptop that connects to the DB?  A SQL ui client, or console, etc.?

@brettwooldridge There were zero connections before he started tomcat and left for the night.  We should be in the office in a couple of hours.  I doubt that he has any other clients connected, but I'll ask him when I get in.  It would be quite a coincidence if it exhibited this pooling behavior, however.
BTW, right now he has 919 connections from his ethernet interface:
netstat -a | grep <IP_ADDRESS> | grep ESTABLISHED | wc -l
919

And, 257 from his wifi interface:
netstat -a | grep <IP_ADDRESS> | grep ESTABLISHED | wc -l
257

I've published a snapshot, with out latest changes.  Though I have to say, if the pool is calling close() on connections as expected, and the OS stack (client-side) is showing no established connections, I'm at a bit of a loss.
<dependency>
  <groupId>com.zaxxer</groupId>
  <artifactId>HikariCP</artifactId>
  <version>2.4.4-SNAPSHOT</version>
</dependency>

Or direct download here.

@victropolis PostgreSQL Undead queries.  Similar symptoms:

Recently we noticed that a few of our queries were running for hours or even days without terminating.
...
We started to investigate and discovered that:

recipients of the data were non-existent;
...
the underlying TCP connection was in the TCP ESTABLISHED state, but the client was already gone,
so no data was being transmitted over it.

@brettwooldridge attached please find the logs I collected.
member_manager_hikari.txt
member_service_hikari.txt

@victropolis Wow, that is some crazy sh*t.  This...
2016-01-29 00:06:54,804 WARN  [e04...] [Hikari housekeeper (pool 1-MemberConnectionPool)] [pool.HikariPool] - 1-MemberConnectionPool - Unusual system clock change detected, soft-evicting connections from pool.
...
2016-01-29 00:06:54,809 DEBUG [e04...] [Hikari housekeeper (pool 1-MemberConnectionPool)] [pool.HikariPool] - Before cleanup	pool 1-MemberConnectionPool stats (total=0, active=0, idle=0, waiting=0)
2016-01-29 00:06:54,809 DEBUG [e04...] [Hikari housekeeper (pool 1-MemberConnectionPool)] [pool.HikariPool] - After cleanup	pool 1-MemberConnectionPool stats (total=0, active=0, idle=0, waiting=0)
2016-01-29 00:06:54,809 DEBUG [e04...] [Hikari housekeeper (pool 1-MemberConnectionPool)] [pool.HikariPool] - Before cleanup	pool 1-MemberConnectionPool stats (total=0, active=0, idle=0, waiting=0)
2016-01-29 00:06:54,809 DEBUG [e04...] [Hikari housekeeper (pool 1-MemberConnectionPool)] [pool.HikariPool] - After cleanup	pool 1-MemberConnectionPool stats (total=0, active=0, idle=0, waiting=0)
2016-01-29 00:06:54,809 DEBUG [e04...] [Hikari housekeeper (pool 1-MemberConnectionPool)] [pool.HikariPool] - Before cleanup	pool 1-MemberConnectionPool stats (total=0, active=0, idle=0, waiting=0)
2016-01-29 00:06:54,809 DEBUG [e04...] [Hikari housekeeper (pool 1-MemberConnectionPool)] [pool.HikariPool] - After cleanup	pool 1-MemberConnectionPool stats (total=0, active=0, idle=0, waiting=0)
2016-01-29 00:06:54,809 DEBUG [e04...] [Hikari housekeeper (pool 1-MemberConnectionPool)] [pool.HikariPool] - Before cleanup	pool 1-MemberConnectionPool stats (total=0, active=0, idle=0, waiting=0)

The housekeeper runs at a "fixed interval" of 30 seconds.  For each run, a "Before" and "After" message are logged.  Yet, the Java ScheduledExecutor is running the task 100+ times "in the same millisecond".
This is right after unusual clock motion is detected.  It appears that time is "frozen" or some other abherent condition that is causing the executor to think it is time to fire again...

@brettwooldridge must be related to what @nitincchauhan posted above: https://bugs.openjdk.java.net/browse/JDK-8074074

@brettwooldridge check this out: http://stackoverflow.com/a/5568375

@victropolis That is the case for Timer.  HikariCP does use the ScheduledExecutorService as recommended by that post ... which shouldn't exhibit "catch-up" behavior.

Could it be related to this?
https://github.com/brettwooldridge/HikariCP/blob/bd7e3ba6db56edba7ca29ed374d91f1996588ece/src/main/java/com/zaxxer/hikari/pool/HikariPool.java#L133
scheduleAtFixedRate - If any execution of this task takes longer than its
period, then subsequent executions may start late, but will not
concurrently execute.
scheduleWithFixedDelay - Creates and executes a periodic action that
becomes enabled first after the given initial delay, and subsequently with
the given delay between the termination of one execution and the
commencement of the next.
Sounds like scheduleAtFixedRate ensures the tasks will not execute
concurrently but they could still queue up. scheduleWithFixedDelay will not
queue the next task until after the completion + delay of the previous
execution.
On Fri, Jan 29, 2016 at 8:18 PM, Brett Wooldridge<EMAIL_ADDRESS>wrote:

@victropolis https://github.com/victropolis That is the case for Timer.
HikariCP does use the ScheduledExecutorService as recommended by that
post ... which shouldn't exhibit "catch-up" behavior.
—
Reply to this email directly or view it on GitHub
https://github.com/brettwooldridge/HikariCP/issues/559#issuecomment-177039379
.

@brettwooldridge IMO this behavior is 'also' because of fixed size of closing queue. this keeps housekeeper busy closing many connections in 'housekeeper' thread instead of 'connection closer' that it prompts an other 'clock changed'.
I already committed this fixed to proportionate size in PR https://github.com/brettwooldridge/HikariCP/pull/539

its confusing but it looks like the condition should be:
if (now < clockSource.plusMillis(previous, HOUSEKEEPING_PERIOD_MS)  || now > clockSource.plusMillis(previous, (2 * HOUSEKEEPING_PERIOD_MS))) {

@victropolis Published an updated snapshot.

com.zaxxer
HikariCP
2.4.4-SNAPSHOT

Direct download here.

@brettwooldridge thanks Brett.  Will test on Monday when we're back in the office.  Enjoy the rest of your weekend.

@brettwooldridge when will 2.4.4 be released?

@victropolis I'd like to put out a release candidate on Monday (Tokyo time) and release one week later if all goes well.

@brettwooldridge FYI, we're finally testing today and overnight tonight.  been getting pounded with snow.  looks good so far.

@brettwooldridge everything is still looking great through the night.  database connections remain stable at or near the minimumIdle setting.  I'll post the logs in the morning, if there's anything interesting in there.  Thanks again for re-opening this issue @brettwooldridge and thanks for resolving it.  I'll be waiting for 2.4.4 to be released.

BTW, I no longer see connections for wifi and ethernet adapter.  only the ethernet adapter.

@victropolis HikariCP 2.4.4 RC1 is available for testing.


the best items on the surface.

the most powerful items of your mod sometimes you can easly find them in silly structures. its not a issue or a bug is just when you have all those items make the game a wayyy more easy.

You can change this in config.  Under 'world generation' you can set ChestLoot=false to disable all Cyclic items from coming into chests, or you can change ChestLootChance to a lower or higher number to change the rarity.
Thanks for the feedback, its definitely something to keep an eye on!!!!!.   Last time i started a new world, in 1.11.2 modded, i didn't find any dungeons but in the nether fortress i found some good stuff.


Service connection delay of 60 seconds starting from version 0.10.0

Version:
v1.0.0
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--no-deploy local-storage,metrics-server,traefik" INSTALL_K3S_VERSION="v1.0.0" sh -
Describe the bug
In a multi-node cluster, my setup is currentlly 3 nodes, communication from one node to the IP service (10.43.x.y) on another node have a delay of around 60 seconds.
Additional context
My operating system is Centos7 on Scaleway cloud.
I've rebuild a new cluster with version 0.9.1 and all is working fine.
This problem starts to occur with version 0.10.0. Maybe the new flannel backend feature?

I've upgraded all my cluster to Ubuntu 18.04 and all is working fine with version 1.0.0 of k3s. It confirms that the problem is related to CentOS.