Windows/Linux line endings break a test during first execution only

This is an interesting one. The first time I ran the tests in a fresh/clean dev environment, one of the tests broke (codegen/tests/test_codegen_z.py::test_that_code_is_up_to_date). However, when I tried to re-run the test it passed nicely and since then subsequent re-runs also pass. I have put the full log of the run that broke below for future reference.
From what I can gather, the test in question modifies a file using codegen (not 100% sure what this is yet) and then checks if the content has changed as expected. The problem is that the file was originally created with windows-style line endings; however, after codegen has processed it, this switches to linux-style line endings which breaks the test.
On top of this, it looks like this original file gets permanently modified and changed to Linux-style line endings. As a result, the test passes for subsequent runs after the first.
So there seem to be two issues here, one regarding the linux/windows line ending and one regarding the test permanently changing a file and thus "bleeding" into subsequent runs (which isn't ideal from a testing standpoint).
Overall, this only seems to have a small impact, so maybe we can afford not worry about this for now; however, I thought I should at least bring it up, so that it is tracked.

Full Test Log
(.venv) PS C:\Users\Sebastian\Documents\Coding-Projects\wgpu-py> pytest
===================================================================================== test session starts ======================================================================================
platform win32 -- Python 3.8.8, pytest-7.2.0, pluggy-1.0.0
rootdir: C:\Users\Sebastian\Documents\Coding-Projects\wgpu-py
collected 98 items / 1 skipped

codegen\tests\test_codegen_apipatcher.py ..                                                                                [  2%]
codegen\tests\test_codegen_rspatcher.py ..                                                                                [  4%]
codegen\tests\test_codegen_utils.py .......                                                                                [ 11%]
codegen\tests\test_codegen_z.py F.                                                                                [ 13%]
examples\tests\test_examples.py .....ss                                                                                [ 20%]
tests\test_api.py ........                                                                                [ 28%]
tests\test_compute.py ........                                                                                [ 36%]
tests\test_gui_auto_offscreen.py ..                                                                                [ 38%]
tests\test_gui_base.py ....                                                                                [ 42%]
tests\test_rs_basics.py ..................                                                                                [ 61%]
tests\test_rs_compute_tex.py ............                                                                                [ 73%]
tests\test_rs_render.py ...........                                                                                [ 84%]
tests\test_rs_render_tex.py .............                                                                                [ 97%]
wgpu\__pyinstaller\test_wgpu.py ..                                                                                [100%]

=========================================================================================== FAILURES ===========================================================================================
_________________________________________________________________________________ test_that_code_is_up_to_date _________________________________________________________________________________

    def test_that_code_is_up_to_date():
        """Test that running the codegen updates what we expect, but does not introduce changes."""

        # Obtain mtime of all files
        mtimes = {}
        for root, dirs, files in os.walk(lib_dir):
            if "__pycache__" in root:
                continue
            for fname in files:
                filename = os.path.join(lib_dir, root, fname)
                mtimes[filename] = os.path.getmtime(filename)

        # Give it some time
        time.sleep(0.2)
        ref_time = time.time()
        time.sleep(0.2)

        assert all(t < ref_time for t in mtimes.values())

        # Create cache
        file_cache = {}
        for filename in changing_files:
            assert os.path.isfile(filename)
            with open(filename, "rb") as f:
                file_cache[filename] = f.read()

        # Double-check that mtimes have not changed
        all(t == os.path.getmtime(filename) for filename, t in mtimes.items())

        # Apply codegen
        main()

        # Get files that have changed
        updated = {
            filename for filename, t in mtimes.items() if t != os.path.getmtime(filename)
        }

        # Check that exactly the files updated that we expected
        assert updated == changing_files

        # But they should not have changed
        for filename in changing_files:
            content1 = file_cache[filename]
            with open(filename, "rb") as f:
                content2 = f.read()
>           assert content1 == content2
E           assert b'"""\r\nThe ...lobals())\r\n' == b'"""\nThe cl...(globals())\n'
E             At index 3 diff: b'\r' != b'\n'
E             Use -v to get more diff

codegen\tests\test_codegen_z.py:68: AssertionError
------------------------------------------------------------------------------------- Captured stdout call -------------------------------------------------------------------------------------
# Code generatation report
## Preparing
* The webgpu.idl defines 35 classes with 75 functions
* The webgpu.idl defines 5 flags, 33 enums, 58 structs
* The wgpu.h defines 131 functions
* The wgpu.h defines 5 flags, 48 enums, 74 structs
## Updating API
* Wrote 5 flags to flags.py
* Wrote 33 enums to enums.py
* Wrote 58 structs to structs.py
### Patching API for base.py
* Diffs for GPU: change request_adapter, change request_adapter_async
* Diffs for GPUCanvasContext: add present
* Diffs for GPUAdapter: add properties
* Diffs for GPUDevice: add adapter, add create_buffer_with_data, hide import_external_texture, hide pop_error_scope, hide push_error_scope
* Diffs for GPUBuffer: add map_read, add map_write, add size, add usage, hide get_mapped_range, hide map_async, hide unmap
* Diffs for GPUTexture: add dimension, add format, add mip_level_count, add sample_count, add size, add usage
* Diffs for GPUTextureView: add size, add texture
* Diffs for GPUQueue: add read_buffer, add read_texture, hide copy_external_image_to_texture
* Validated 35 classes, 102 methods, 35 properties
### Patching API for backends/rs.py
* Diffs for GPUAdapter: add request_device_tracing
* Validated 35 classes, 92 methods, 0 properties
## Validating rs.py
* Enum field FeatureName.shader-f16 missing in wgpu.h
* Enum CanvasCompositingAlphaMode missing in wgpu.h
* Wrote 231 enum mappings and 49 struct-field mappings to rs_mappings.py
* Validated 80 C function calls
* Not using 56 C functions
* Validated 70 C structs
======================================================================================= warnings summary =======================================================================================
.venv\lib\site-packages\PyInstaller\lib\modulegraph\modulegraph.py:20
  c:\users\sebastian\documents\coding-projects\wgpu-py\.venv\lib\site-packages\PyInstaller\lib\modulegraph\modulegraph.py:20: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
    import imp

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=================================================================================== short test summary info ====================================================================================
FAILED codegen/tests/test_codegen_z.py::test_that_code_is_up_to_date - assert b'"""\r\nThe ...lobals())\r\n' == b'"""\nThe cl...(globals())\n'
================================================================ 1 failed, 95 passed, 3 skipped, 1 warning in 61.90s (0:01:01) =================================================================

Arg, line endings ...

not 100% sure what this is yet

Wgpu-py is a mix of manual and autogenerated code. The code generation also performs check against the spec and the header file. The test that now fails basically checks that the code has not changed the autogenerated bits since the last time that the codegen was run.
This issue is harmless because it's only about line endings. But its annoying nonetheless.
What does git status say after running that test?

I agree, this seems harmless. That said, the files do show up as modified in git:
(.venv) PS C:\Users\Sebastian\Documents\Coding-Projects\wgpu-py> git status
On branch package-release-only
Your branch is up to date with 'origin/package-release-only'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        modified:   wgpu/backends/rs.py
        modified:   wgpu/backends/rs_mappings.py
        modified:   wgpu/base.py
        modified:   wgpu/enums.py
        modified:   wgpu/flags.py
        modified:   wgpu/resources/codegen_report.md
        modified:   wgpu/structs.py

no changes added to commit (use "git add" and/or "git commit -a")

Also, for wgpu/backends/rs.py only it actually modifies the file visually and eats up some of the blank lines.

Would it be easy to modify codegen to use platform-specific line endings?
Based on my brief search of the source it looks like at least some of the places have hard-coded line endings, so it might be possible. Not sure how much work it would be though:
https://github.com/pygfx/wgpu-py/blob/212e84e673bf9a281abd62dc70e9bdeb6ffcb3e3/codegen/utils.py#L198

Would it be easy to modify codegen to use platform-specific line endings?

Yes, I think the places where a change is needed are limited.
The line ending to use should be detected from a source file (I guess one that is not autogenerated), as it depends on your git settings.

I cannot test this myself easily, as I've set git up to keep line endings as they are.

Heh, I have it set to change line endings so I could look into this. Do you think it would be better to set them based on how they are in the file or based on the platform you are on (os.linesep)?

That would be great. It would have to be based on how they are in the file, since os.linesep is always \r\n on Windows.

I honestly don't care what we change to handle this. In my mind, the test shouldn't fail on new line differences.
I think that is more important than getting our tools such as git to play nice together.

I'm still not 100% sure what the purpose of this test is, but could a solution be to make a copy of the file it needs to test (in memory or in a tmp_path), change all the line endings to linux and to then run the test on that copy? This way we would avoid modifying existing files, normalize against differences in tooling, and still get to run the check against the current version of the file.

Well, maybe this is too big of an ask, but really the generation code should not write to a file directly. It should just write to a string or text stream/buffer/whatever IO abstraction is most convenient, and return the strings. Separate logic from side effects. That will make testing much easier.

I would argue to just fix this issue now that we're at it. I'm pretty sure that if we copy the line ending usage from another file, the result will be consistent.
At the moment I am the only one running the codegen script when I update for upstream changes. This issue will become annoying when someone else (with a different git setup) also performs this task, as we'd flip back and forth between line endings.

but really the generation code should not write to a file directly

Let me have a quick look at how much work this'd be. Though I think this becomes a non-issue when the line-ending is made consistent, which should be relatively easy to do and would be good to do anyway.


Formatting crosstalk

Describe the bug
When one function has poorly formatted annotations, swag fmt may incorrectly modify the formatting of other functions.
To Reproduce
main.go:
package main

// @Produce json
// @Success 200 {object} string
// @Failure 400 {object} string
func A() {}

// @Description Description of B.
// @Produce json
// @Success 200 {array} string
// @Failure 400 {object} string
func B() {}

Then run swag fmt.
The resulting main.go:
package main

// @Produce     json
// @Success 200 {object} string
// @Failure     400 {object} string
func A() {}

// @Description Description of B.
// @Produce json
// @Success     200 {array}  string
// @Failure 400 {object} string
func B() {}

Expected behavior

I do not expect the comments on function A to change.
I expect the Swag attributes on function B to have been correctly aligned.

package main

// @Produce json
// @Success 200 {object} string
// @Failure 400 {object} string
func A() {}

// @Description Description of B.
// @Produce     json
// @Success     200 {array}  string
// @Failure     400 {object} string
func B() {}

Your swag version
$ swag --version
swag version v1.8.5

...but I can reproduce on v1.8.4 and the latest development version.
Your go version
$ go version
go version go1.18.5 darwin/arm64

...but I can reproduce on Go 1.19, and on Linux AMD64.
Desktop (please complete the following information):

OS: macOS 12.5.1
Browser: n/a
Version: n/a

Additional context
This is the most messed-up example of this that I could come up with. It seems to be extremely sensitive to how well-formatted the input is to begin with, and the success/failure types.

Half-indenting attributes or changing their types produces further wild variations in output. For example, change B's success type from {array} string to {object} string, and all three of A's attributes will be indented equally, but none of B's:
package main

// @Produce     json
// @Success     200 {object} string
// @Failure     400 {object} string
func A() {}

// @Description Description of B.
// @Produce json
// @Success 200 {object} string
// @Failure 400 {object} string
func B() {}

Or, change A's success and failure types from {object} string to just string, and A's produce attribute and B's success and failure attributes will be indented:
package main

// @Produce     json
// @Success 200 string
// @Failure 400 string
func A() {}

// @Description Description of B.
// @Produce json
// @Success     200 {array}  string
// @Failure     400 {object} string
func B() {}

If there is another function between A and B, it will not be affected (but A and B both still will be).

This does not happen if the comments on C are already correctly formatted.

I have only been able to trigger this behaviour with a @Description attribute on the second function, which I suspect has something to do with that attribute being longer than any on the first.

@MrDOS This is an Open Souce project, and you can contribute via PR to improve it.

@ubogdan, sure thing: see https://github.com/swaggo/swag/pull/1317.


chore: update Edge link in README.md

/gcbrun

/gcbrun

Thanks @Shabirmean! - FY: I don't have access to the failing check, so not clear what is required.

@apeabody - yeah it is restricted to the repo owners. Ill take a look and have it merged 🙌🏼

/gcbrun

/gcbrun


unicode_loose_md5: Fix infinite loop in Collator.

@sougou
Fixes #1871.
I also added a sync.Pool for reusing Collators since this function is called at least once for every query that uses this vindex. If you think it's not worth it, I can remove that commit, but the microbenchmark looks good:
vitess/go/vt/vtgate/vindexes$ go test -bench . -benchmem
# Old thread-safe way
BenchmarkNormalizeSafe-12         200000              8331 ns/op            9784 B/op         16 allocs/op
# Fast but unsafe
BenchmarkNormalizeShared-12      3000000               455 ns/op             104 B/op          1 allocs/op
# sync.Pool
BenchmarkNormalizePooled-12      2000000               537 ns/op             134 B/op          2 allocs/op

This change is

LGTM
This is awesome! I'm thinking we should file a bug against collate. No library should go OOM on invalid input.

Filed https://github.com/golang/go/issues/16391


use more commonly understood terminology like polymorphism and explain what Zig pattern is used

See https://en.wikipedia.org/wiki/Polymorphism_(computer_science) and in the table "Examples" there could be brief words what the things does in general terminology, ie comptime/dynamic/static polymorphism etc.

To be honest I think this would confuse more than help people.

"Polymorphism" is so broad that I could include 500 completely different Zig patterns that perform polymorphism.
I really don't want to encourage people to think of this in an OOP way; these are not patterns that are meant to be copy-pasted into every other file; I myself have used these incredibly rarely, maybe once or twice in the course of three years.

I'd rather people see the examples and make their own connections / choices rather than connect these to prior connotations that may not be honest, correct, or useful.


Autocomplete within Path Box

Please add the ability to press tab and fill in the selected folder into the path.

enter will already do this.

This feature request is now a candidate for our backlog. The community has 60 days to upvote the issue. If it receives 20 upvotes we will move it to our backlog. If not, we will close it. To learn more about how we handle feature requests, please see our documentation.
Happy Coding!

enter will already do this.

Enter should confirm the selection.

This feature request has not yet received the 20 community upvotes it takes to make to our backlog. 10 days to go. To learn more about how we handle feature requests, please see our documentation.
Happy Coding!


Remove unused symlinks

to: @mistercrunch @bkyryliuk
Content:

Remove symlinks that point at non-existent directories. These can cause problems in some build systems, and, in any case, it doesn't look like they're being used anywhere.

LGTM


open_under_cursor not work

When using function illustrate.open_under_cursor(), report errors:
E5108: Error executing lua: .../share/nvim/lazy/illustrate.nvim/lua/illustrate/init.lua:71: attempt to concatenate local 'illustration_dir' (a nil value)
stack traceback:
	.../share/nvim/lazy/illustrate.nvim/lua/illustrate/init.lua:71: in function 'open_under_cursor'
	/home/fireond/.config/nvim/lua/plugins/write/latex.lua:110: in function </home/fireond/.config/nvim/lua/plugins/write/latex.lua:109>

It seems it cannot detect the 'illustration_dir', but illustrate.create_and_open_ai() can successfully create files in figures/name.svg, and can successfully add the svg file. Is anything I can try to fix this bug? Thanks!
Environments:

NVIM v0.10.3
LazyVim 14.6.0
6.12.6-arch1-1

Hello,
Thank you for using the plugin. Can you please provide your configuration of the plugin as well? I will have a look later and let you know.

I just use the config provided in README:
return {
'rpapallas/illustrate.nvim',
dependencies = {
    'nvim-lua/plenary.nvim',
    'nvim-telescope/telescope.nvim',
},
keys = function()
    local illustrate = require('illustrate')
    local illustrate_finder = require('illustrate.finder')

    return {
        {
            "<leader>is",
            function() illustrate.create_and_open_svg() end,
            desc = "Create and open a new SVG file with provided name."
        },
        {
            "<leader>ia",
            function() illustrate.create_and_open_ai() end,
            desc = "Create and open a new Adobe Illustrator file with provided name."
        },
        {
            "<leader>io",
            function() illustrate.open_under_cursor() end,
            desc = "Open file under cursor (or file within environment under cursor)."
        },
        {
            "<leader>if",
            function() illustrate_finder.search_and_open() end,
            desc = "Use telescope to search and open illustrations in default app."
        },
        {
            "<leader>ic",
            function() illustrate_finder.search_create_copy_and_open() end,
            desc = "Use telescope to search existing file, copy it with new name, and open it in default app."
        },
    }
end,
opts = {
    -- optionally define options.
},
}

Nothing changes. Also search_and_open() and search_create_copy_and_open() do not work too, I think they have same reason.

I found where the bug is: in illustrate.utils:37, for detecting the directory, you add twice directory_name. But after I modified this, when calling the function, it now has no errors but also no responses, and the file does not open.

@Fireond it looks like you are trying to open a figure that can't be found in the right place. It looks like the "illustration dir" doesn't exist where it's expected. Does the figure exist in the right place?
I suggest that you try first: <leader>is to create a new figure. This will also create the illustration directory in the right place. Then you try to open it using the under cursor command: <leader>io.

@Fireond you can also see examples of how to use the plugin here.
Please let me know if this is now fixed/working for you.

Yes, it now works well!


Implement workspaces' visibility

I added logic to display the visibility type of workspaces in the workspace chooser and to show the corresponding icon based on the visibility type.

I am getting an error related to this PR in the PR preview app app

I am getting an error related to this PR in the PR preview app app

@jurgenwerk It seems the error occurred because the preview app was pointing to the realm server in staging. I just deployed this PR to staging, and it worked. Please check it again.


fix/updated-foundry-guide

Proposed changes
updates foundry guide
Types of changes
Please put an x in the boxes related to your change.

[x] Minor Issues and Typos
[ ] Major Content Contribution
[ ] Others

Checklist
Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to reach out. We're here to help! This is simply a reminder of what we are going to look for before merging your code.

[ ] I have read the CONTRIBUTING GUIDELINES
[ ] I have signed the CLA
[ ] I have added necessary documentation (if appropriate)
[ ] Any dependent changes have been merged and published in downstream modules

Related issues

Please leave the issue numbers or links related to this PR here.

Further comments
If this is a relatively large content contribution, kick off the discussion by explaining why you would suggest the content contribution, etc...

@praveen-klaytn will make amends in a seperate PR


test: fix long output buffering

The tests runners sometimes forward the test output. But when the test
output is too long (over 1024 bytes), the runner output gets corrupted.
This is because unfinished lines are written without "%.*s" causing
bad behavior as the buffer is not always nul terminated (it's not the
case on windows).
The issue can be seen with the platform_output test on windows.
This commit changes the print_lines function so it keeps track of new
lines and never write more data than it should.

This is a duplicate of #1325, so closing now this PR.


Add ToString() override for UriEndpoint

Addresses #12656

This work is not approved for 3.1 and isn't likely to be (the scope for 3.1 is very narrow and only critical fixes/changes are likely to be approved). Can you re-target this to master?
Otherwise I think this is good, we'd also want some tests to validate the output of ToString (just create some endpoints, ToString them and validate the output). If you can retarget this to master I think we can review this and get it merged in!

This work is not approved for 3.1 and isn't likely to be (the scope for 3.1 is very narrow and only critical fixes/changes are likely to be approved). Can you re-target this to master?
Otherwise I think this is good, we'd also want some tests to validate the output of ToString (just create some endpoints, ToString them and validate the output). If you can retarget this to master I think we can review this and get it merged in!

Thanks for your feedback. Sure, I will retarget that to master and add some tests.
Any recommendations regarding where should I put them? I see some tests here but this looks more like servers comparison tests, so not sure that tests for Connections.Abstractions should live there as well.

There's already an open PR for this by @ste1nstone targeting master1: https://github.com/aspnet/AspNetCore/pull/12668. I approved it a while back, but at the time, I don't think master was open to 5.0 changes.
Sorry for the confusion!

Ok, got it. Closing that.


Add Support for modifying Container SecurityContext for Certgen job in Helm

Description:

Describe the desired behavior, what scenario it enables and how it
would be used.

[optional Relevant Links:]

Any extra documentation required to understand the issue.

Close this one as the default SecurityContext has been set to minimum permissions.
Customization on Cetgen job SecurityContext should not be required anymore.


Create setup.py, add entry point for esp8266

This allows installation as a command line tool.
When installed with python setup.py, it installs an executable that can start the visualization as follows:
esp8266_music --ip <IP_ADDRESS> -n 60 -d

Looking further into this I think it could be very useful to merge and once I get home and test this I will merge if there are no issues.


Added two new cli commands

Added two little features I found quite handy:

Get actor's current state with switch-state
Toggle actor's state with switch-toggle

Fixed parsing in Actor.get_state()
First time using Python, please be gentle ;-)

Thank your for your contribution.
Please update your code according to my comments (overriding methods is not a good idea), then I'll merge it. :-)

Thanks for noticing. Must have screwed up the method names while merging changes from the virtual env to the GitHub repo.


IDVA6-1321 Resources not loading in phoenix1

Link to Jira:
https://companieshouse.atlassian.net/browse/IDVA6-1321
Changes:

Environment variable changed to a correct one

CI: No security warnings found


Add Prince v11

Prince was previously in homebrew-binary.
Moving this here as homebrew-binary is deprecated and outdated.

If there’s a checkbox you can’t complete for any reason, that's okay, just explain in detail why you weren’t able to do so.
After making all changes to the cask:

[x] brew cask audit --download {{cask_file}} is error-free.
[x] brew cask style --fix {{cask_file}} reports no offenses.
[x] The commit message includes the cask’s name and version.

Additionally, if adding a new cask:

[x] Named the cask according to the token reference.
[x] brew cask install {{cask_file}} worked successfully.
[x] brew cask uninstall {{cask_file}} worked successfully.
[x] Checked there are no open pull requests for the same cask.
[x] Checked that the cask was not already refused in closed issues.

We will also need to add the entry "prince": "caskroom/cask" to tap_migrations in homebrew-binary.

@miccal Sure, I will create a PR to homebrew-binary.

There you go: https://github.com/Homebrew/homebrew-binary/pull/357

Thank you @lukasbestle.

Thanks! 👍


Multi select Picklist to django model: inspectdb

Hi,
I'm encountering a multiple picklist selector field.
Is there any integration for these fields in django-salesforce?
E.g. Is there an integration to fetch the possibilities of the picklist?
A quick search showed  me that there are some REST API endpoints where I could get the possibilities from.
e.g. https://developer.salesforce.com/forums/?id=906F000000092QuIAI

The title "Picklist" is too broad. It is not clear enough if you mean a Multi-Select Picklist by your term "multiple picklist selector field" and not a simple Picklist.
A) simple Picklist
A picklist has an equivalent Django field type: a CharField with choices option. This is supported by introspection by inspectdb command. You automatically get in the model the defined choices values. Supported operators for picklist in SOQL are =, !=, IN, NOT IN and their negation. Therefore also their equivalents in Django querysets are supported. Operators <, <=, >, >= are  only partly supported by SOQL (after my intervention) but negation of these operators is unsupported like not (field > 'value').
(I will continue about Multi-Select Picklist)

Oke, thanks, and sorry for the unspecific topic.
The problem is I'm not in control of the SalesForce setup, and I'm not really used to using this in practice.
However I do need to work with it, we choose to use django salesforce, because of it's ease of use.
the inspectdb command works really well, but I'm just not getting the choices in the multiselect thingy.

oke thanks a lot for the feedback and tips and tricks.


[FR]: Update README.md with new client info

What would this feature improve or what problem would it solve?
Documentation reflecting the new architecture
What is the feature you are proposing?
Update the main README.md to show how the turbinia-client works for common operations (create processing request and get results, etc) and show turbiniactl as the way to start/stop the components only.
What alternatives have you considered?
No response

Fixed in #1377


⚠ Spryr has degraded performance

In edbc610, Spryr (https://spryr.com) experienced degraded performance:

HTTP code: 200
Response time: 4494 ms

Resolved: Spryr performance has improved in e13e237 after 11 minutes.


Update dependencies and related code

Awaiting the merge of https://github.com/input-output-hk/ouroboros-network/pull/2094 to remedy the block forging problem.

That PR is now merged!

bors merge


Re-implementation paper works

Thank you for sharing your works and efforts!!
I'd try to re-implement your results based on uploaded code.
The output generated by the model trained using kitti360 work fine, which means that it well represents the 3d geometry in the kitti360 dataset.
Unfortunately, however, the result of models that are trained newly using mvl is quite different to that of the pre-trained model that you uploaded.
Indeed, the loss value decreased when I train the model according your introduction. However, it seems like fail to learn nerf information so that fail to generate vehicle-like point cloud swarm.
On the other hand, both loss value and variation of loss value estimated during validation process using uploaded pre-trained model is quite large (maximum value of loss is about 60, and vary 0.01 to 60).
Despite high loss value, it successfully generates the vehicle-shape point cloud swarm. However, strange thing is that the output in results folder well represent the object shape but one in validations are fail to generate point cloud data that showing object shape.
I guess that It seems like there some issue on loss configuration but not for sure.
Could you give me some advice for re-implementing your works using mvl dataset?
(Currently, I'm working on V100, with conda virtual environment)

Hello, thanks for your feedback. I will double-check the results on mvl, but this may take several days since I have been busy with some work recently. I provide the checking results here as soon as possible.


Question: Failure percentage

Hi,
It is not clear from the documentation whether the failure threshold is for buckets (i.e. percentage of buckets that are deemed failed) or total requests - and if it is for buckets, then does Hysterix trip open on the first bad bucket or applies extra logic.
If I get the answer, I can send a PR for an update in documentation clarifying this point.
Thanks for great work

Thanks for the kind words, @aliostad .
The failure percentage is determined by looking at all current buckets, and then summing successes/failures across all buckets.
Happy to review a PR for the docs - much appreciated!

Thank you so much!
Now I am afraid there is another question here, why even bother about having buckets? Is this an implementation detail mainly to help with performance and rolling active stats?

Yes, exactly.  Rather than recalculate the aggregate error percentage every time a metrics gets added, we recalculate only when the timer elapsed, and one bucket leaves the set and a new one gets added.  Purely a performance optimization.

Awesome. I will close the issue and send a PR to docs.
Please do not feel obliged to accept it. Feel free to accept, reject or accept and modify. Once again thank you for all the work.

Ah... I do not seem to be able to send a PR for wiki :/
I forked but wiki is empty.

Are you able to clone via: https://github.com/Netflix/Hystrix.wiki.git, and submit a PR on that?  I got that link from https://github.com/Netflix/Hystrix/wiki


enhance dotgraph functionality

for our usage and on my way implement tendermint state sync (https://github.com/tendermint/tendermint/issues/828), I found dotgraph is helpful while it somehow broken now:

node key and values are encoded []byte array, the encoding logic is depends on application, so sometimes they cannot be represented as string, we need show their hex representation on graph.
edges cannot be established because the lazy loading of children nodes, we need a set of traversal methods set child along the way we traverse. Currently I just do below on my way of debugging:

func (node *Node) getLeftNode(t *ImmutableTree) *Node {
	if node.leftNode != nil {
		return node.leftNode
	}
	node.leftNode = t.ndb.GetNode(node.leftHash)
	return node.leftNode
}

func (node *Node) getRightNode(t *ImmutableTree) *Node {
	if node.rightNode != nil {
		return node.rightNode
	}
	node.rightNode = t.ndb.GetNode(node.rightHash)
	return node.rightNode
}

I have already made these changes 80% done on my local machine, will push them soon.

this is something that came up recently in terms of checkpoint syncing.


[Suggestion] Creative Energy Storage

As the name suggest, it would be nice to have a creative energy storage that makes a network essentially have infinite energy buffer and removes and a network can transfer (sometimes when you have so much energy being drained from the network it will stop transferring all together)

I can add infinite energy feature. Currently, you can shift click a Flux Storage with Admin Configuration to fill it up.
I'm not quite sure under what circumstances the transfer stop you mentioned will occur, but flux networks have no transfer limit.


Support preferring TCP over UDP for upstream queries

Note (mainly to self): check sq->outnet->udp_buff use. No other comments on the rest of the code.


mrpt2: 2.1.0-3 in 'foxy/distribution.yaml' [bloom]

Increasing version of package(s) in repository mrpt2 to 2.1.0-3:

upstream repository: https://github.com/MRPT/mrpt.git
release repository: https://github.com/mrpt-ros2-pkg-release/mrpt2-release.git
distro file: foxy/distribution.yaml
bloom version: 0.9.7
previous version for package: 2.1.0-1

PS: new release after manual fix to debian/rules (see: https://github.com/ros-infrastructure/bloom/pull/600 )

@jlblancoc IIUC you want to wait for https://github.com/ros-infrastructure/bloom/pull/600/files to be reviewed and eventually merged before merging this release. Is that correct?

Oh sorry for the awful explanation: nope. This one can be merged.
The new release fixes a problem that could have been avoided if the linked PR was already merged.


Missing PinConfigs for ALC233 - ALC268 in v1.0.6 Release!!!

Just found that in v1.0.6 release files PinConfigs for ALC233 - ALC268 are missing but they defined in release notes.

Great… I must have missed something while compiling the binaries. Sorry.
Not a big deal, they will be present in 1.0.7. I will leave the issue open until 1.0.7 is released.
By the way, what would you recommend to include in 1.0.7?

Something messy appeared on im, are we interested in fixing/merging this?
Some pinconfig change arised for ASUS. I wonder if it is of any use as well.
ALC892 stuff as well, we tend to be using Mirone at least… and this codec feels tested.
Maybe the tools (e. g. Info.plist generator), although I do not think it is a good idea.

Many mistakes in this workaround (wrong codec-id, used reserved layout/platform numbers, deleted Mirone resources). You could merge this as another custom resource, but not instead working ones...
Need testing by m-dudarev
Why not? Just like ALC1150 with Toleda/Mirone resources
You right. It is not a good idea.

Might do if I have time
Kk
Will give it a look later

Well, I am done with the two :) Let me know if my changes cause any terrifying issues.

PinConfigs for ALC668 layout 27 and ALC892 layout 5, 7, 9?

Looks like they failed to commit and then merge properly. Hopefully fixed.

Now it's OK except Mirone's layout 9. It's for codecaddress 2 and similar to layout 7. And we found that patches for codecaddress 1 (layout 5, 7) also works fine with codecaddress 2 .

Kicked it out then

Fixed


:bug: fix date formatting in some cases

#14  👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @szamanr and the rest of your teammates on  Graphite

Merge activity

Apr 18, 3:56 PM GMT+2: @szamanr merged this pull request with Graphite.


Use existing consts for Cilium CRD names

Use the existing *{Plural,Singular}Name consts already defined in pkg/k8s/apis/cilium.io/v2 instead of duplicating strings for the Cilium CRD names.

test-me-please


ELB IP changes can bring the cluster down

I ran into https://github.com/kubernetes/kubernetes/issues/41916 twice in the last 3 days in my production cluster, with almost 50% of worker nodes transitioning to NotReady state almost simultaneously in both days, causing a brief downtime in critical services due to Kubernetes default (and agressive) eviction policy for failing nodes.
I just contacted AWS support to validate the hypothesis of the ELB changing IPs at the time of both incidents, and the answer was yes.
My configuration (multi-node control plane with ELB) matches exactly the one in that issue, and probably most kube-aws users are subject to this.
Have anyone else ran into this at some point?

@danielfm Thanks for the report.
I think I have never encountered the issue myself for my production cluster but anyway, I believe this is a BIG issue.
IMHO, ideally, kubelet should detect any dead long-polling connection and then re-discover ELB instances via DNS and reconnect to one of them. However I'm happy to work-around the issue in kube-aws.
Possible work-arounds
Perhaps:

periodically monitoring the list of ips an route53 alias record for an ELB and then
restarting kubelet whenever the monitor detects the list to be changed
so that kubelet can reconnect to one of active ELB ips before it marks the k8s node NotReady?

If we go that way, the monitor should be executed periodically with an interval of (<node-monitor-grace-period> - <amount of time required for kubelet to start> - <amount of time required for each monitor run>) / <a heuristic value: N> - <amount of time required to report node status after kubelet started> at least, so that we can provide kubelet at most N chances to successfully report its status to possibly available ELB.
Or we can implement a DNS round robin with a health-checking mechanism for serving k8s API endpoints like suggested and described in #281 #373

This hash changes only when backend IPs of an ELB has been changed.
$ dig my.k8s.cluster.example.com +noall +answer +short | sort | sha256sum | awk '{print $1}'
96ff696e9a0845dd56b1764dd3cfaa1426cdfbd7510bab64983e97268f8e0bc4

$ dig my.k8s.cluster.example.com +noall +answer +short | sort
<IP_ADDRESS>
<IP_ADDRESS>
<elb id>.ap-northeast-1.elb.amazonaws.com.

@danielfm I guess setting recordSetTTL in cluster.yaml(which sets the TTL of a CNAME record associated to a controller ELB) to be a value considerably lower than that of ELB's DNS A records TTL(=60sec) and kubelet's --node-monitor-grace-period and running the following service would alleviate the issue.
/etc/systemd/system/elb-ip-change-watcher.service:
[Service]
Type=notify
Environment=API_ENDPOINT_DNS_NAME={{.APIEndpoint.DNSName}}
ExecStart=/opt/bin/elb-ip-change-watcher

/opt/bin/elb-ip-change-watcher:
#!/usr/bin/env bash

set -vxe

current_elb_backends_version() {
  dig ${API_ENDPOINT_DNS_NAME:?Missing required env var API_ENDPOINT_DNS_NAME} +noall +answer +short | \
    # take into account only ips even if dig returned a CNAME answer(when API_ENDPOINT_DNS_NAME is a CNAME rather than an A(or Route 53's "Alias") record
    grep -o '[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}' | \
    # sort IPs so that DNS round-robin doesn't unexpectedly trigger a kubelet restart
    sort | \
    sha256sum | \
    # sha256sum returns outputs like "<sha256 hash value> -". We only need the hash value excluding the trailing hyphen
    awk '{print $1}'
}

run_once() {
  local file=$ELB_BACKENDS_VERSION_FILE
  prev_ver=$(cat $file || echo)
  current_ver=$(current_elb_backends_version)
  echo comparing the previous version "$prev_ver" and the current version "$current_ver"
  if [ "$prev_ver" == "" -o "$prev_ver" == "$current_ver" ]; then
    echo the version has not changed. no need to restart kubelet.
    if [ "$KUBELET_RESTART_STRATEGY" == "watchdog" ]; then
      echo "notifying kubelet's watchdog not to trigger a restart of kubelet..."
      local kubelet_pid
      kubelet_pid=$(systemctl show $KUBELET_SYSTEMD_UNIT_NAME -p MainPID | cut -d '=' -f 2)
      systemd-notify --pid=$kubelet_pid WATCHDOG=1
    fi
  else
    echo the version has been changed. need to restart kubelet.
    if [ "$KUBELET_RESTART_STRATEGY" == "systemctl" ]; then
      systemctl restart $KUBELET_SYSTEMD_UNIT_NAME
    fi
  fi
  echo writing $current_ver to $file
  echo "$current_ver" > $file
}

ELB_BACKENDS_VERSION_FILE=${ELB_BACKENDS_VERSION_FILE:-/var/run/coreos/elb-backends-version}
KUBELET_SYSTEMD_UNIT_NAME=${KUBELET_SYSTEMD_UNIT_NAME:-kubelet.service}
KUBELET_RESTART_STRATEGY=${KUBELET_RESTART_STRATEGY:-systemctl}
WATCH_INTERVAL_SEC=${WATCH_INTERVAL_SEC:-3}

systemd-notify --ready
while true; do
  systemd-notify --status "determining if there're changes in elb ips"
  run_once
  systemd-notify --status "sleeping for $WATCH_INTERVAL_SEC seconds"
  sleep $WATCH_INTERVAL_SEC
done

Also - I have never realized that but we're creating CNAME records - which has its own TTL(=300 seconds by default in kube-aws) other than ELB's A records' TTL - for controller ELBs.
We'd better make them Route53 Alias records so that there will be only one TTL.

Updated my assumption on the issue in my first comment https://github.com/kubernetes-incubator/kube-aws/issues/598#issuecomment-297586503

Hmm, a TTL for a CNAME record associated to ELB's DNS name seems to be capped at 60s, even though kube-aws' default recordSetTTL is set to 300s.
Then, we won't need to care about the balance of two TTLs(kube-aws CNAME and ELB's A) that much.

Anyway, to forcibly restart kubelet to reconnect apiserver(s) when necessary would still be important.
The script and the systemd unit I've suggested in https://github.com/kubernetes-incubator/kube-aws/issues/598#issuecomment-297615293 would be useful to achieve it..

@mumoshu does it mean that going ELB-less mode with just CNAME record in Route53 pointing to controller nodes is considered dangerous now?

@redbaron Do you mean a single CNAME record for your k8s API endpoint which points to just one controller node's DNS name?
I've been assuming that you would have a DNS name associated to one or more A records(rather than CNAME) each is associated to one of controller nodes public/private IP if you'd go without an ELB.

Doesn't change the fact that final set of A records returned from DNS request change when you trigger controllers ASG  update, right? From what I read here it matches the case when ELBs change their IPs

@redbaron Ah, so you have a CNAME record which points to a DNS record containing multiple A records each is associated to one of controller nodes, right?
If so, no, as long as you have low TTLs for your DNS records.
AFAIK, someone in the upstream issue said that it becomes an issue only when ELB is involved. Perhaps ELB doesn't immediately shutdown an unnecessary instance and doesn't send FIN? If that's the case, kubelet would be unable to detect broken connections immediately.
When your controllers ASG is updated, old, unnecessary nodes would be terminated before it becomes non-functional like ELB's instances.
So your ELB-less mode with CNAME+A records would be safe as long as you have health checks to update the route 53 record set to eventually return A records "only for healthy controller nodes", and you have a TTL lower than --node-monitor-grace-period

ELB-less mode I refer to is a recent feature in kube-aws, I didn't check which records it creates exactly,  just wanted to  verify that is still a safe option to do considering this bug report.

@redbaron If you're referring to the DNS round-robin for API endpoints, it isn't implemented yet.

I wonder if using ALB can help here. L7 load balancer precisely knows all incoming/outgoing requests and can forcibly yet safely close HTTP connection  when ELB scales/changes IP addresses.

@redbaron Thanks, I never realized that ALBs may help. I wish they can help, too.
Although I'd like to hear from contributors in the upstream issue to kindly clarify whether ALBs help or not,
shall we take this opportunity to add support for ALBs into kube-aws anyway? 😃

Today happened to me for the first time with a 128 days old cluster.

Seems like Amazon is doing some serious maintenance work in the ELB infrastructure these days...

@danielfm @camilb @redbaron For me it seems like there are still no obvious fixes other than:

Adding a dirty, periodic elb monitoring and kubelet restarting script of mine
Moving to Route 53 DNS round-robin based api endpoints (suggested by @tarvip in https://github.com/kubernetes/kubernetes/issues/41916#issuecomment-293243276 and #281)

A major challenge to include this into kube-aws is how to keep Route 53 records sets to be up-to-date with the running controller nodes at the time. Rolling-updates of controller nodes, sudden termination of controller nodes, unresponsive apiserver containers, etc., should ideally be considered while syncing record sets.

Would you like to proceed with any of them or any other idea(s)?

Rolling-updates of controller nodes, sudden termination of controller nodes, unresponsive apiserver containers, etc

I think sudden termination, unresponsive apiserver and etc shouldn't cause problems as long as these events are not happening at the same time, but in this case even ELB setup fails.
But keeping route53 record up-to-date is a bit more complicated without using Lambda and SNS.
When adding/restarting or replacing(terminating and recreating) controller nodes we could have unit in cloud-config that is executed when host comes up, this script gets all controller nodes that are part of ASG and modifies route53 record accordingly. But I don't know how to handle node removal (without adding new node back, although I think this is not happening very often).
Also I think it shouldn't be cause problems if route53 record is not fully up-to-date when restarting or replacing controller nodes, as long they are not replaced at the same time there should be at least one working controller.

Thanks @tarvip!

think sudden termination, unresponsive apiserver and etc shouldn't cause problems as long as these events are not happening at the same time, but in this case even ELB setup fails.

Sorry if I wasn't clear enough but the above concerns are not specific to this problem but more general. I just didn't want to introduce a new issue due to missing health checks / constant updates to route 53 record sets.

Also I think it shouldn't be cause problems if route53 record is not fully up-to-date when restarting or replacing controller nodes, as long they are not replaced at the same time there should be at least one working controller.

Probably that's true - then, my question is if everyone is ok with e.g. 50% k8s api error rate persists for several minutes when one of two controller nodes are being replaced?

But I don't know how to handle node removal (without adding new node back, although I think this is not happening very often).

If we go without cloudwatch events + lambda, we probably need a systemd timer which periodically triggers a script to update a route53 record set so that it periodically start reflecting controller nodes terminated either expectedly or unexpectedly, right?

@mumoshu , is ALB known not to help here?

I just didn't want to introduce a new issue due to missing health checks / constant updates to route 53 record sets.

I think missing health check is ok, that health check is just TCP check anyway, kubelet and kube-proxy can also detect connection failure and recreate connection to another host.
Regarding constant updates, I guess we can perform update only if current state is different compared to route53.

Thanks @redbaron - No but it isn't known to help either.

If we go without cloudwatch events + lambda, we probably need a systemd timer which periodically triggers a script to update a route53 record set so that it periodically start reflecting controller nodes terminated either expectedly or unexpectedly, right?

Yes, that is one way to solve it.

@tarvip Thanks for the reply!

I think missing health check is ok, that health check is just TCP check anyway, kubelet and kube-proxy can also detect connection failure and recreate connection to another host.

Ah, makes sense to me! At least it seems to worth trying for me now - thanks for the clarification.

@tarvip

Regarding constant updates, I guess we can perform update only if current state is different compared to route53.

Agreed

As a bandaid, can't we query kubelet itself to cause it to go to apiserver for a response? If such query exist, then running it on a timer would let kubelet to detect problems sooner

As a bandaid, can't we query kubelet itself to cause it to go to apiserver for a response? If such query exist, then running it on a timer would let kubelet to detect problems sooner

Maybe my example is not very good but like etcdctl endpoint health for etcd?
If so, that sounds good to me. Unfortunately I'm not aware of any concrete function of kubelet available for that.

50% k8s api error rate persists for several minutes when one of two controller nodes are being replaced?

I think this affects only other clients e.g kubectl, not kube-proxy and kubelet. Kubelet and kube-proxy detect that one api server is down and recreate connection to other controller and use this connection (and controller) until something happens with connection, I tested this some time ago and it worked well, AFAIK they are not even refreshing their connections.

If we go without cloudwatch events + lambda, we probably need a systemd timer which periodically triggers a script to update a route53 record set so that it periodically start reflecting controller nodes terminated either expectedly or unexpectedly, right?

I think using Lambda is not a solution since it's not available on every AWS region.

@mumoshu I'm using your periodic elb monitoring and kubelet restarting script for the time being.

@tarvip

I think this affects only other clients like kubectl, not kube-proxy and kubelet. Kubelet and kube-proxy detect that one api server is down and recreate connection to other controller and use this connection (and controller) until something happens again with that connection

Thanks again for the clarification! It did make sense to me. Then, assuming we went ahead with the systemd timer method, there won't be huge problems as long as we set its interval short enough(could be several seconds at minimum)

Thanks @danielfm,

I think using Lambda is not a solution since it's not available on every AWS region.

Good point 👍

there won't be huge problems as long as we set its interval short enough

I think in normal situation it can be even longer, lets say once in a minute, but when updating cloudformation thet it can replace controller nodes quite fast.
Then there is question where to run this timer, should all controller nodes execute this timer? If using 2 or 3 nodes and also short interval for timer then hopefully it wont create additional problems if they all try to update route53 record at the same time.

@tarvip

Then there is question where to run this timer, should all controller nodes execute this timer?

Yes, in the simplest case, I believe so.
IMHO we can even introduce some kind of leader election so that only one controller node is responsible to update record sets. However it temps me to make the record sets updater script to be a dockerized golang app which implements leader election using k8s client like cluster-autoscaler and other k8s products do 😃

Hi @tarvip @danielfm,
Regarding the missing health-check thing, just a quick thought but can we deploy a nginx-ingress controller by default to load-balance against healthy apiserver pods only?
nginx-ingress(or even the vanilla open-source edition of Nginx) doesn't support active monitoring but does seem to support passive-monitoring:

https://www.nginx.com/resources/admin-guide/tcp-load-balancing/#health_passive
https://github.com/kubernetes/ingress/blob/a83f17c/controllers/nginx/rootfs/etc/nginx/template/nginx.tmpl#L227https://github.com/kubernetes/ingress/blob/a83f17c/controllers/nginx/rootfs/etc/nginx/template/nginx.tmpl#L227

to remove a failed backend from the list of backends for a while.
It isn't a complete solution but better than nothing?

If it works then why not, but then there is to chicken and egg situation when bootstrapping cluster, kubelet and kube-proxy cannot connect to api as there is not ingress controller yet, but you cannot deploy ingress controller because there are no worker nodes available.

@tarvip Thanks!
I was blindly thinking about deploying an ingress-controller as a static pod per controllre node but obviously it doesn't work as we have no way to provid an ingress resource for it 😢

I ran into this issue a few weeks ago myself. Changed the CNAME TTL at the time but it appears that isn't really a fix. If I'm not mistaken, it seems like the check/restart script is probably the most reliable solution for the time being. Based on the upstream issue, kubelet is keeping the connection open for up to 15 mins because it didn't receive any data telling it to close. When the ELB instances are terminated and the old IPs are no longer available, it seems that kubelet never receives any data like it normally would if your controller is shut down, the controller manager is process was killed, etc..
DNS round robin would probably fix most cases since it gets rid of ELB which wasn't sending any data when the connection is closed... But I imagine there is still some possibility that other unexpected connection issues could result in kubelet just sitting there thinking nothing happened.  Seems like a lot of work to implement in kube-aws, and isn't necessarily going to fix the fact that kubelet needs improvement :).

FYI ALB doesn't seem to fit as a k8s API load balancer. Please see #608 and the corresponding PR.
So, we would need to settle on some of solutions other than ALB anyway.

Hi @roybotnik, thanks for sharing the hard won experience 👍
Good to know the CNAME TTL fix is missing the point.

Seems like a lot of work to implement in kube-aws

Could be 😄

@danielfm @redbaron @camilb @tarvip @roybotnik
In the meantime, would you agree that we should do the followings in order of priority:

Enable the check/restart script for the K8S API ELB

As the script is considered the most effective (though dirty) solution could be introduced without changing things too much

Implement the DNS round-robin based K8S API endpoint to see how it works in the wild - including the longer-term reliability when integrated with kubelets
Revert 1 once it turned out the issue is completely fixable outside of kube-aws - maybe in kubelet

I agree, although I'm not using this script at the moment.
At the moment we have increased only --node-monitor-grace-period=90s in kube-controller-manager, I don't like it, as it is not fast enough for detecting if node is really down, but I thought if it happens again I will set up DNS round-robin, but we haven't seen this issue since February.
Also maybe that script should restart also kube-proxy.

I was thinking about this yesterday, and a simple solution that works around the kubelet connection timeout issue (without caring about ELB specifically) might be to query the API server itself for the node's status.
Since kube-controller-manager won't evict pods until a node has been offline for a while by default (5 minutes I believe), we can query every 60s or so to see if the node status is not ready and then restart kubelet. I haven't tested this fully and the script was kinda thrown together so it could probably be simplified (not great w/ bash script):
cloud-config-worker
    - name: node-status-check.service
      enable: true
      command: start
      runtime: true
      content: |
        [Unit]
        Description=Restart kubelet if node is not ready

        [Service]
        Environment=API_ENDPOINT_DNS_NAME={{.APIEndpoint.DNSName}}
        ExecStart=/opt/bin/node-status-check
        Restart=always
        RestartSec=60

/opt/bin/node-status-check
#!/usr/bin/env bash

# Get node info from API server
NODE_INFO=$(curl https://$API_ENDPOINT_DNS_NAME/api/v1/nodes/$HOSTNAME --key /etc/kubernetes/ssl/worker-key.pem --cert /etc/kubernetes/ssl/worker.pem --cacert /etc/kubernetes/ssl/ca.pem --cert-type PEM -s)
if [ $? -ne 0 ]; then
  echo "Unable to query API endpoint. Verify API endpoint is online and accessible to nodes."
  exit 1;
fi

# Select node conditions from json response
NODE_CONDITIONS=$(jq '.status.conditions[] | {(.type): .status}' -M -c <<< "$NODE_INFO")
if [ $? -ne 0 ]; then
  echo "API returned an invalid JSON response, unable to check status."
  exit 1;
fi

echo "Current node conditions:"
echo "$NODE_CONDITIONS"

if grep -q "\\\"Ready\\\":\\\"True\\\"" <<< $NODE_CONDITIONS
then
  echo "API server reports node is ready, no actions taken"
else
  echo "API server reports node status not ready, restarting kubelet"
  systemctl restart kubelet.service
fi

Hi @danielfm @redbaron @camilb @tarvip @roybotnik, have you encountered this issue since then?
I still had no luck(??) to encounter this issue for my clusters and therefore am unable to decide this as resolved or not.

Didn't happen to us yet, but upstream issue has a clean way to reproduce problem without ELB or any cloud provide whatsoever.

We haven't seen this since initial bug report, but we also increased node-monitor-grace-period, maybe it helps.

Hi @mumoshu didn't happen to me since then and only happened once.

I just experienced another issue with ELBs that the script proposed by @mumoshu apparently isn't able to circumvent, which is the failure of one or more ELB nodes. When this happened, the DNS records for the affected ELB did not change (as reported by the systemd unit logs), which makes the script unable to recover the affected kubelets.
Moving to NLBs (#937) might help solve these issues altogether.

I guess nothing can help if part of underneath infrastructure fails, same thing can probably happen to NLB

Probably, but then AWS won't be able to point their fingers back at us when something like this happens.

AWS's NLB replacement for ELB's using one IP per zone/subnet and those IP be your EIP's. So using this new product you can get a LB with a set of fixed IPs that won't change.
http://docs.aws.amazon.com/elasticloadbalancing/latest/network/introduction.html
AWS new ALB (for HTTPS) and NLB (for TCP) seem to AWS's next-gen replacement for the old ELB, which AWS now calls 'Classic Load Balancers'. k8s and kube-aws should probably look to transition to the new products, which also appear have some advantages, such as fixed IP's.

@whereisaaron Thanks for the suggestion! I agree with your point. Anyway, please let me also add that ALB was experimented in #608 and decided as not appropriate for an K8S API load balancer.

Unfortunately NLBs don't support VPC Peering on AWS, so some users (including me) will need to use Classic ELBs in conjunction with NLBs to support kubectl commands.

Seen this today. Our set up use Consul DNS for kubelet to discover the apiserver, which means the apiserver DNS name are multiple A-record pointing to the exact IP addresses of the apiservers, which changes every time an apiserver node is replaced.
In our case the workers come back eventually but it took a long while. My feeling is kubelet is not really respecting DNS TTLs as all Consul DNS names have TTL set to 0. Can anyone confirm?

Thanks everyone.
At this point, would the only possible, universal work-around be the one shared by @roybotnik?
// At least mine won't work with @javefang's case of course.

I was in the impression that since some k8s version kubelet has implemented the clide-side timeout to mitigate this issue, but can't remember the exact github issue right now.

I noticed that after the master DNS record changed the underlying IP, all kubelet instances fail for exactly 15min. (Our master DNS TTL is 0). When it fails we get the following error.
Nov 15 13:08:08 dev-kubeworker-gen-0 kubelet[11954]: E1115 13:08:08.638348   11954 kubelet_node_status.go:390] Error updating node status, will retry: error getting node "dev-kubeworker-gen-0": Get https://apiserver-gen.service.dev.winton.consul/api/v1/nodes/dev-kubeworker-gen-0: net/http: request canceled (Client.Timeout exceeded while awaiting headers)

It recovered by its own without restarting after 15min (sharp). It feels more like kubelet (or the apiserver client used) is caching the DNS. I'm trying to pin-point the exact line of code which caused this behaviour. But anyone know the code-base better might be able to confirm this?

Seeing the following messages right before the worker came back. Last failure was at 13:15:18, then it reported some watch error (<IP_ADDRESS> was the previous master which got destroyed) and re-resolved the DNS name before the cluster report the worker as "Ready" again!
Nov 15 13:15:16 dev-kubeworker-gen-0 kubelet[11954]: I1115 13:15:16.994725   11954 qos_container_manager_linux.go:320] [ContainerManager]: Updated QoS cgroup configuration
Nov 15 13:15:18 dev-kubeworker-gen-0 kubelet[11954]: E1115 13:15:18.670478   11954 kubelet_node_status.go:390] Error updating node status, will retry: error getting node "dev-kubeworker-gen-0": Get https://apiserver-gen.service.dev.winton.consul/api/v1/nodes/dev-kubeworker-gen-0: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
Nov 15 13:15:22 dev-kubeworker-gen-0 kubelet[11954]: E1115 13:15:22.640287   11954 streamwatcher.go:109] Unable to decode an event from the watch stream: read tcp <IP_ADDRESS>:49178-><IP_ADDRESS>:443: read: no route to host
Nov 15 13:15:22 dev-kubeworker-gen-0 kubelet[11954]: E1115 13:15:22.640410   11954 kubelet_node_status.go:390] Error updating node status, will retry: error getting node "dev-kubeworker-gen-0": Get https://apiserver-gen.service.dev.winton.consul/api/v1/nodes/dev-kubeworker-gen-0: read tcp <IP_ADDRESS>:49178-><IP_ADDRESS>:443: read: no route to host
Nov 15 13:15:22 dev-kubeworker-gen-0 kubelet[11954]: E1115 13:15:22.640943   11954 streamwatcher.go:109] Unable to decode an event from the watch stream: read tcp <IP_ADDRESS>:49178-><IP_ADDRESS>:443: read: no route to host
Nov 15 13:15:22 dev-kubeworker-gen-0 kubelet[11954]: E1115 13:15:22.641445   11954 streamwatcher.go:109] Unable to decode an event from the watch stream: read tcp <IP_ADDRESS>:49178-><IP_ADDRESS>:443: read: no route to host
Nov 15 13:15:22 dev-kubeworker-gen-0 kubelet[11954]: W1115 13:15:22.663747   11954 reflector.go:334] k8s.io/kubernetes/pkg/kubelet/kubelet.go:413: watch of *v1.Service ended with: too old resource version: 1883 (16328)
Nov 15 13:15:22 dev-kubeworker-gen-0 kubelet[11954]: W1115 13:15:22.665010   11954 reflector.go:334] k8s.io/kubernetes/pkg/kubelet/kubelet.go:422: watch of *v1.Node ended with: too old resource version: 16145 (16328)
Nov 15 13:15:22 dev-kubeworker-gen-0 kubelet[11954]: W1115 13:15:22.665806   11954 reflector.go:334] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: watch of *v1.Pod ended with: too old resource version: 5602 (16345)

It seems like there is a problem. If the controller DNS entry has a 30 second TTL, the kubelet should be able to recover from an IP change within 30s + update period, so about 40s. @javefang you think the kubelet is using this long, up to 15 minute back-up when the old IP goes stale? And so not a DNS caching problem, but rather it just stops trying to update the controller for several minutes?
For AWS at least, an NLB using fixed EIP addresses would mostly obviate the IP address every changing I think? Even if you recreate or move the LB, you can reapply the EIP so nothing changes. However an extra wrinkle is we would want worker nodes in multi-AZ clusters to use the EIP for the NLM endpoint in the same AZ. NLB's have one EIP per AZ as I understand it?

@whereisaaron yep this is indeed taking 15min for kubelet to recover. I have reproduced it with the following setup:

OS: Centos 7.4 (SELinux on)
Docker: 1.12.6
K8S: 1.8.3
Apiserver: 3 instances running on separate VMs
Apiserver DNS names: all 3 registered as Consul Services, which does DNS round-robin for them (dig apiserver.service.consul will show 3 IPs, pointing to the VMs running the apiserver)

To reproduce:

Destroy the VM running apiserver 1
Create a new VM to replace (this will get a different IP)
30% of the worker nodes goes into "NotReady" state, kubelet prints error message kubelet_node_status.go:390] Error updating node status, will retry: error getting node "dev-kubeworker-gen-0": Get https://apiserver-gen.service.dev.winton.consul/api/v1/nodes/dev-kubeworker-gen-0: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
Repeat 1-3 for the other 2 apiservers
Now all workers should be in "NotReady" state
Wait 15min, kubelet on workers print the unable to decode an event from the watch stream and read: no route to host message before coming back to "Ready" state

I'm just curious about the mechanism in kubelet that can cause kubelet to be broken for 15min after any apiserver IP changes. We are deploying this on-premise. Tomorrow I'll try to put the 3 apiservers behind a load balancer with fixed IP to see if that fixes the issue.

UPDATE: putting all apiservers behind a load balancer (round-robin) with a static IP fixed it. Now all workers work fine even if I replace one of the master node.

I believe the 15 minute window break many of us are experiencing is described in https://github.com/kubernetes/kubernetes/issues/41916#issuecomment-312428731. Reading through issues and pull requests, I don't see where a TCP Timeout was implemented on the underlying connection. The timeout on the HTTP request definitely was implemented.

All there work around the real problem, that the connections are keep forever.
If we limit the livetime or an connection by time the problem would be not happened.
Also we would get a better load distribution, because new connections allow loadblancer todo new distribution.

the connections are keep forever. If we limit the livetime or an connection by time the problem would be not happened.

They don't live forever, they live for the operating system TCP timeout limit (typically 15 minutes by default)

Yes and this 15 min are to long for many cases, like here.
The dead connection from elb/alb when there get terminated after there are 6 days depricated, mean not visible in dns any more.
If we would reconnect every hour (or 10 min) we would not have the problem. And would get as site effect better load distribution. But still would have all benefits from keepalive.

The fix merged into the last several releases of kubernetes was to drop/reestablish the apiserver connections from the kubelet if the heartbeat times out twice in a row. Reconnecting every 10 minutes or every hour would still let nodes go unavailable.

What seen in other go projects, if you use pooling and frequently sent request that keepalive idle timeout get not reached you run into this issue. If you disable pooling and make only one request per connection, you have not that issue. But higher latency and overhead, this why keepalive make sense.
By the way, the old Apache httpd had not only keepalive ideal timeout but also keepalive max request count.  Which helped a lot in many of this problems.


Is it possible to match a record to a map? Should it be?

I am doing some tests and it seems that is currently not possible to match a Record against a Map
The following code produces No and true:
import 'dart:convert';

typedef FromMap = ({String foo, int bar});

void main() {
  final decoded = json.decode('{"foo": "yat", "bar": 123}');

  if (decoded case {'foo': String foo, 'bar': int bar}) { // matches
    final fromMap = (foo: foo, bar: bar);

    print(fromMap is FromMap); // Unnecessary type check, result always true
  }

  if (decoded case FromMap) { // nope, doesn't match
    print('Yes');
  } else {
    print('No'); // enters here
  }
}

Will it be possible to retrieve a Record from a Map?
I can see packages exposing builders that do that and I was wondering if this should be possible from the core.
If you think it through  Map matching is very close to it.

No.
Records are not maps, nor vice-versa. Records are more like objects with a special implicit class per shape, and that type is not a map.
You cannot create a record without writing the shape into the source, because if you don't, the compiler won't know the "class" of that record.
Since a map's content is not part of its type, it's not possible to create a record from a map in a sound statically typed way.
You can treat records as objects, and use object patterns, like:
if (decoded case FromMap(:foo)) {
  print("Just the string $foo");
}

but they're not map objects.


Update hooks-effect.md

What I understand, these are the effects which run after every re-render, on the contrary, cleanup run before every re-render.

This is a valid addition, thank you, @segmentationfaulter, for contributing! 👍
Looking at the "Synchronizing with effects" page I can see, however, that this is described in https://react.dev/learn/synchronizing-with-effects#step-3-add-cleanup-if-needed

React will call your cleanup function each time before the Effect runs again, and one final time when the component unmounts (gets removed).

I will close this PR as no longer required as the current wording describes it well, but I invite you to create new contributions by addressing other open issues.
Thank you!


Corrected oil well preset tag

@gregcrago pointed out that the oil well preset added for #2618 actually adds man_made=mast instead of man_made=petroleum_well, due to my copy-pasta. :disappointed:

Thanks for the fix, Minh..

@bhousel, do the strings need to be pushed to Transifex manually somehow? osm.org picked up the new release before translators got any opportunity to update their localizations.

Actually, strings should be synced automatically (daily if I remember correctly). But it looks like the last update of the source file at transifex was on March 25. Maybe there was some hiccup with the sync process and it therefore it stopped working and just needs a kick-start? (At least that's what happened for my overpass turbo's project on transifex a few weeks ago.) //cc @jfirebaugh

It should check for the latest core.yaml and presets.yaml files from our Github repository. And it looks like this is working.  On the dashboard I see core.yaml updated May 20th and presets.yaml updated May 25th.
Before each release, we run make translations to pull in the latest translations from Transifex.  I remember doing this release pretty quickly after the preset got added, so that's why the translations aren't there.  Going forward I'll try to wait a bit longer between changing the presets and cutting a release.

so, why aren't the changes from #2618 (from Apr 30) not on Transifex yet? Also, to me on transifex' dashboard the last update of the presets resource appears to have happened on March 19th:

That is weird.. I look on the "Resources" tab and I see this:

Oh maybe this is the issue.. Transifex thinks these URLs are no good:

I don't understand why it would say that.. Github seems to serve these from a varnish cache with 200 OK, so it's not even like there is a redirect.

Yeah, that's exactly what happened to me with overpass turbo on transifex, too. I don't know what the problem was, but as far as I remember the error got away after entering the same URL once again in the settings.

I updated the URLs to point to openstreetmap/iD so we'll see if that helps.   They don't say "URL not found" anymore, but we may have to wait a day or so for the next time Transifex tries to sync.

Good news, it looks like updating those URLs fixed the issue!   Transifex picked up a current core.yaml and presets.yaml last night, and these do include the oil well preset, as well as the other new presets I added yesterday.

:+1:


Android: RNDeviceModule.java uses or overrides a deprecated API.

Summary

Version
0.21.5

Affected OS
Android

OS Version
any

Current behavior
Running gradlew assembleRelease  for my project, I got this, it's only an info, but I prefer to report to you
:react-native-device-info:compileReleaseJavaWithJavac
Note: [cut] \node_modules\react-native-device-info\android\src\main\java\com\learnium\RNDeviceInfo\RNDeviceModule.java
        uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.

Expected behavior
No warning about use of deprecated API
PS
Thank your for developing and mantaining this project.

+1

Thank you so much for opening this issue @realtebo, this problem is already logged in #325, so I'm closing this bug report.
Cheers!


Avoid "KeyError: 'request'" when other templates error

When there is another syntax or parsing error in the django template, the original stack trace is hidden because django hijack then crashes when rendering the template tag with "KeyError: 'request'" on that line.
It’s particularly annoying when running unit tests, so making this a bit more exception safe.

Coverage remained the same at 92.975% when pulling 8554df82b1b170211bee0f8ba3c14675326609a1 on browniebroke:keyerror-request into e0cea3e55118c735732c7786a760c3b6973254f4 on arteria:master.

Thanks @browniebroke

Sorry @walterrenner did I miss anything? Tests? Some side effects maybe?
Let me know if you'd like me to include anything else, I'm happy to improve it. I'm a lot unfamiliar with the codebase, so I assume there are plenty of reason why you wouldn't want to merge it, but a small constructive feedback would be appreciated 😄
Thanks

Sorry, i closed this by mistake. I will take a look at your PR when I'm back from vacation.
I don't think any Test are required.

Coverage remained the same at 92.975% when pulling 8554df82b1b170211bee0f8ba3c14675326609a1 on browniebroke:keyerror-request into e0cea3e55118c735732c7786a760c3b6973254f4 on arteria:master.

Thanks @browniebroke for contributing to django-hijack. Your changes will be available with in the new version.


pgmodeler: 1.1.2 -> 1.1.3

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://github.com/pgmodeler/pgmodeler/releases.
meta.description for pgmodeler is: A database modeling tool for PostgreSQL
meta.homepage for pgmodeler is: https://pgmodeler.io/
Updates performed

Version update

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact
Checks done

built on NixOS
The tests defined in passthru.tests, if any, passed
found 1.1.3 with grep in /nix/store/makb31gzb997fpbyqmwnmgwhag1jv8mn-pgmodeler-1.1.3
found 1.1.3 in filename of file in /nix/store/makb31gzb997fpbyqmwnmgwhag1jv8mn-pgmodeler-1.1.3

Rebuild report (if merged into master) (click to expand)

2 total rebuild path(s)

2 package rebuild(s)

First fifty rebuilds by attrpath

pgmodeler

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/makb31gzb997fpbyqmwnmgwhag1jv8mn-pgmodeler-1.1.3 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A pgmodeler https://github.com/r-ryantm/nixpkgs/archive/9c21151cf0631ef92dd42b05271058c3aa49a265.tar.gz

Or:
nix build github:r-ryantm/nixpkgs/9c21151cf0631ef92dd42b05271058c3aa49a265#pgmodeler

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/makb31gzb997fpbyqmwnmgwhag1jv8mn-pgmodeler-1.1.3
ls -la /nix/store/makb31gzb997fpbyqmwnmgwhag1jv8mn-pgmodeler-1.1.3/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review --extra-nixpkgs-config '{ allowInsecurePredicate = x: true; }' run on x86_64-linux 1

1 package built:

pgmodeler

Maintainer pings
cc @esclear for testing.

Result of nixpkgs-review pr 311530 run on x86_64-linux 1

1 package built:

pgmodeler


NotImplementedError: unknown node: mermaid

Hi,
is there any plan to add support for sphinxcontrib.mermaid?
Currently, if I add restbuilder and mermaid in conf.py:
extensions = ['sphinxcontrib.restbuilder',
              'sphinxcontrib.mermaid']

And then add a diagram in a rst file:
This is a graph:

.. mermaid::

   graph LR
      id1[Item1] --> id2[Item 2];
      id2 --> id3[Item 3];

I get an NotImplementedError: Unknown node: mermaid.
Thanks in advance!
My environment:
Sphinx                        4.2.0
sphinxcontrib-applehelp       1.0.2
sphinxcontrib-devhelp         1.0.2
sphinxcontrib-htmlhelp        2.0.0
sphinxcontrib-jsmath          1.0.1
sphinxcontrib-matlabdomain    0.12.0
sphinxcontrib-mermaid         0.7.1
sphinxcontrib-qthelp          1.0.3
sphinxcontrib-restbuilder     0.3

Thanks for the feature request. The bad news is that I have no current plan to actively add support for non-standard extensions. The good news is that I would gladly accept pull requests for it!

Thanks for the info. I'd love to contribute but I am afraid I really would not know where to start!

Just passing by but I think this is relatively straightforward to resolve, probably with just a little hack in your own project's conf.py:
Problem

The root cause here is that the .. mermaid directive isn't recognized by docutils unless it's registered as a node class
The sphinxcontrib.mermaid extension adds a node using sphinx's add_node API in the sphinxcontrib.mermaid's setup() function, which is abbreviated below:

[...]
class mermaid(nodes.General, nodes.Inline, nodes.Element):
    pass

def html_visit_mermaid(self, node):
    render_mm_html(self, node, node['code'], node['options'])

[...]
def setup(app):
    app.add_node(mermaid,
                 html=(html_visit_mermaid, None),
                 latex=(latex_visit_mermaid, None),
                 texinfo=(texinfo_visit_mermaid, None),
                 text=(text_visit_mermaid, None),
                 man=(man_visit_mermaid, None))

In the above, nodes are added for each builder name in the **kwargs (ie html is for the html builder)
If you are rolling out your own builder in an extension, you would need to have an add_note(mermaid, yourbuilder=(html_visit_mermaid, None)) call added to the app

Example Solve

The solve is relatively straightforward: just register a node for the builder name
It's possible to add this into a local project's extension's setup(app) such as:

from sphinx.util import logging

def setup(app):

  if 'sphinxcontrib.mermaid' in app.config.extensions:
    try:
      from sphinxcontrib.mermaid import html_visit_mermaid, mermaid
      app.add_node(
        mermaid,
        somebuilder=(html_visit_mermaid, None),
      )
    except ImportError:
      logging.getLogger(__name__).error(
        "sphinxcontrib.mermaid was detected in conf.py extensions=[...] "
        "but we were unable to import html_visit_mermaid from to run app.add_node(mermaid)."
      )


Roadmap for mixed chain of multithread and multiprocessing pipelines?

🚀 The feature
pypeln has a nice feature to chain pipelines which may run on different kind of workers including process, thread or asyncio.
data = (
    range(10)
    | pl.process.map(slow_add1, workers=3, maxsize=4)
    | pl.thread.filter(slow_gt3, workers=2)
    | pl.sync.map(lambda x: print x)
    | list
)

I remembered that in the first proposal of pytorch/data, it claims to support something alike. I'd like to ask if it's still planed and the concrete roadmap.
Motivation, pitch
Initial proposed
Alternatives
No response
Additional context
No response

@ejguan

Sorry for the late response. TBH, this has been in our long-term roadmap when we createdTorchData project. But, unfortunately, me and @NivekT are not working on TorchData anymore. Stay tuned on the update later.


Using GopherLua instead of lua53.dll

lua53.dll has some problem.

In MinGW 6, lua53.dll built with MinGW gcc 6 requires not only not only  but also LIBGCC_S_DW2-1.DLL.

lua53.dll download from luaBinaries are built with MinGW gcc 4 or Visual C++

appveyor often fails downloading lua53.dll from luaBinaries.

and nyagos rarely requires third party's library using lua53.dll.
So should we use GopherLua instead of lua53.dll ? The project has started.

See also.

GopherLua
Lua Binaries

.nyagos loading => Done
alias => Done
bindkey => Yet
command-line-filter => Yet
args-filter => Yet
complete-hook => Yet
redirect to io.output / io.input => Yet

If you want to test the version using GopherLua (gopherSh.exe), try as below :
$ git checkout develop
$ make
$ cd gopherSh
$ go build

bindkey => Done on 99a0ad74581cfbcbd4038c92682531e1073f1767

Almost Done !（だいたいできた！）

By default, make builds the version using GopherLua.

Done with 75374abca91ec14e779edd593a72c22147fd924b


[5.7] Change morphs columns

This PR fixes issue when morphing to a model that its primary key is a string instead of integer, and morphing to the previous model like the following example:
Schema::create('settings', function (Blueprint $table) {
      $table->string('key')->primary();
      $table->text('value')->nullable();
});

When inserting a table for laravel-medialibrary :
Schema::create('media', function (Blueprint $table) {
     $table->increments('id');
     $table->morphs('model');
     $table->string('collection_name');
     $table->string('name');
     $table->string('file_name');
     $table->string('mime_type')->nullable();
     $table->string('disk');
     $table->unsignedInteger('size');
     $table->json('manipulations');
     $table->json('custom_properties');
     $table->json('responsive_images');
     $table->unsignedInteger('order_column')->nullable();
     $table->nullableTimestamps();
 });

The following error will occur:
Illuminate \ Database \ QueryException (22007)
SQLSTATE[22007]: Invalid datetime format: 1366 Incorrect integer value: 'logo' for column 'model_id' at row 1 (SQL: insert into `media` (`name`, `file_name`, `disk`, `collection_name`, `mime_type`, `size`, `custom_properties`, `responsive_images`, `manipulations`, `model_type`, `model_id`, `order_column`, `updated_at`, `created_at`) values (logo, logo.png, public, default, image/png, 12628, {"custom_headers":[]}, [], [], App\Models\Setting, logo, 91, 2018-04-23 14:07:01, 2018-04-23 14:07:01))

This is a breaking change for anyone that has used this functionality in migrations already. If you start renaming the columns your migration makes, how would a user make sure their migration generates the correct structure in the local environment as it is in the production environment?

Why did you renamed _id to _key instead just changing integer to string?


md-ink-ripple on the row

Hi, I would like to add the angular material ripple effect on the row. But it doesn't work. any suggest?

I've tried adding md-ink-ripple attribute on the tr tag but it will ripple on the whole table instead


Tagged versions not yet published to PSGallery

Hi Guys,
Is there a reason the latest tagged releases for this repo don't appear in PSGallery? Can we get them published?
My company only want to use published modules and the latest available is 3.4 which doesn't have the OpticalDriveLetter Resource. Unless I'm missing something obvious of course...

Ah. it was me missing something obvious after all. Looks like the project has been renamed to StorageDSC. Disregard


Every field should know what defines its RH limit (for parsing from file)

Important to remember that the file parser is working with consistently-formatted code, not user's typing so, for example, there will be no redundant spaces.
For example:

Most expression fields are limited by a NewLine
Variable identifier by a space (this is the formatted version, not what the user types in)
Function/procedure identifier by an open bracket
ArgList or ParamList by an unmatched close bracket

By this means, when the file-parsing is looking for a field, it can first extract the text as far as the delimeter for

Pattern is now implemented in the form of readToDelimiter method, but not yet implemented for all fields.


This is so cool

Hi! I have found a few issues; the discord login doesn't work, on the site the premium tab url doesn't exist, also on the site and the app the discord login doesn't work and lastly, the store in the app doesn't load. This is a really cool project and i hope it gets even better!

Hi there, sorry for the long response time.
This has been a known issue for several months but unfortunately i never had time to properly fix it
The API server is offline currently but should be active again very soon. Please keep up to date with the Releases tab for an update to fix this releasing in around the next 24 hours. Thanks!

yay

@KontrollFreek if i Sponsored this project, how much would make you happy?

Considering I do this all for free anyways I'd be extremely thankful for any sort of donation


Do not accept Strings as keys for nodes and edges

My data is not int-based. Because some of real data have leading zero. The graphview crush on next data:
    var json = {
      "nodes": [
        {"id": "01", "label": 'circle'},
        {"id": "02", "label": 'ellipse'},
        {"id": "03", "label": 'database'},
        {"id": "04", "label": 'box'},
        {"id": "05", "label": 'diamond'},
        {"id": "06", "label": 'dot'},
        {"id": "07", "label": 'square'},
        {"id": "08", "label": 'triangle'},
      ],
      "edges": [
        {"from": "01", "to": "02"},
        {"from": "02", "to": "03"},
        {"from": "02", "to": "04"},
        {"from": "02", "to": "05"},
        {"from": "05", "to": "06"},
        {"from": "05", "to": "07"},
        {"from": "06", "to": "08"}
      ]
    };

error:
    The following TypeErrorImpl was thrown building GraphView$(dirty, state: _GraphViewState#a4f6c):
Expected a value of type 'int', but got one of type 'String'

I tried change to:
var a = node.key!.value.toString();

But got error:
Node{position: Offset(0.0, 0.0), key: [<1>], _size: Size(77.0, 48.0)} is not connected to primary ancestor
Node{position: Offset(0.0, 0.0), key: [<2>], _size: Size(77.0, 48.0)} is not connected to primary ancestor

Hm, I might have to give ability to create keys using strings.

This is now released in 1.0.0
. Have a look

Is there anything required to do that? I get a type 'String' is not a subtype of type 'int?' in type cast error.
The function looks like this:
@override
void initState() {
  final node0 = Node.Id("0");
  builder
    ..siblingSeparation = (10)
    ..levelSeparation = (50)
    ..subtreeSeparation = (100)
    ..orientation = (BuchheimWalkerConfiguration.ORIENTATION_LEFT_RIGHT);
}


QA Web playbook

This PR defines a new playbook for deploying nodes used for the Web applications (registry, QA). Depending on their usage (staging/production), they can be used with/without ci-provision.yml to hook them to Jenkins. /cc @kennethgillen

Ooops...the nginx role wasn't tested properly when it was merged :) It relies on yum install epel-release having been previously run, so it should work fine on our LSC VMs which might have epel enabled, but not on the docker container I used for testing. Apart from that it works fine.
What do you think- install epel-release? Or deal with separately along with the other clean-ups/improvements?

I think the main target of this new playbook are LSC VMS right @kennethgillen? If so, I would defer the installation/enabling of epel-release to a cleanup PR.

ok, want me to merge then?

I assume this will be tested on the new QA Web VMs that will be spinned up so I defer to Kenny no whether we should merge now or later.

Merging. Provisioning issues to be opened.


Running the configure script twice doesn't work

I ran configure.sh once, forgot to set the admin password.  The script will default to "", which is a value that'll make the deployment invalid.  The VMR crashes (And hard to understand why) as a direct result of that.
I ran it a second time, but it had to effect.
Recommendations:
The script should fail if no password is specified to avoid users getting into the same issue.
The script should replace templates/secret.yaml with an original, otherwise the SOLOS_ADMIN_PASSWORD token won't be there, and sed will do nothing.
Probably should do the same with any other files which has sed TOKENs in it.

This is now resolved by PR#53, ready to be closed.


[BUG] Icon not visible in dark mode

Describe the bug
Buy me coffee Icon not visible in dark mode and I like to work it under GSSoC'24
To Reproduce
Steps to reproduce the behavior:
switch to Dark mode
Expected behavior
The icon should be visible even in dark mode
Screenshots

Desktop (please complete the following information):

OS:windows 11
Browser: Microsoft edge

please assign this issue to me, i can work on this, i have made the local setup as well, i can make the changes, and resolve this issue

hey @karthickkumarsm , please assign this issue to me.  I can fix this issue quickly and smoothly. I'd very much like to work on this issue. Please assign it to me.

@karthickkumarsm Assigned you the issue, Best of Luck :)

i would like to work on it #236@singodiyashubham87

@karthickkumarsm Are you working on the issue?

@singodiyashubham87  please assign me this issue I can fix this quickly

@karthickkumarsm Are you working on the issue?

yes bro

done bro


feat: added content_columns option to CSVLoader

Description:
Adding a new option to the CSVLoader that allows us to implicitly specify the columns that are used for generating the Document content.
Currently these are implicitly set as "all fields not part of the metadata_columns".
In some cases however it is useful to have a field both as a metadata and as part of the document content.

[ ] Add tests and docs: If you're adding a new integration, please include

a test for the integration, preferably unit tests that do not rely on network access,
an example notebook showing its use. It lives in docs/docs/integrations directory.

[ ] Lint and test: Run make format, make lint and make test from the root of the package(s) you've modified. See contribution guidelines for more: https://python.langchain.com/docs/contributing/

Additional guidelines:

Make sure optional dependencies are imported within a function.
Please do not add dependencies to pyproject.toml files (even optional ones) unless they are required for unit tests.
Most PRs should not touch more than one package.
Changes should be backwards compatible.
If you are adding something to community, do not re-import it in langchain.

If no one reviews your PR within a few days, please @-mention one of baskaryan, efriis, eyurtsev, ccurme, vbarda, hwchase17.

Hello I would like to propose a small improvement to the CSVLoader (currently draft)


Installed in virtual environment but permission denied with rvc commands (on Mac)

Hi there,
I've installed this in a virtual environment but in trying to run rvc init I get:
zsh: permission denied: rvc
I'm kind of stuck at this point. Do you have any insight into how I could get this working? Thank you in advance!

I was able to get it to open by right clicking and opening in terminal rvc in the bin and then copying and pasting that instead so like: "/Users/omni/Documents/rvc/projects/testing-rvc/venv/bin/python3.11" "/Users/omni/Documents/rvc/projects/testing-rvc/venv/bin/rvc"
This is ok but rvc-api doesn't work this way and it's generally not ideal. Any insight you have into getting it to work with the rvc and rvc-api commands on a mac would be very helpful!

Now that I have it running I've run into an error. Any insight would be greatly appreciated.
INFO:rvc.configs.config:No supported Nvidia GPU found
INFO:rvc.configs.config:overwrite configs.json
INFO:rvc.configs.config:Use mps instead
INFO:rvc.configs.config:is_half:False, device:mps
INFO:rvc.modules.vc.modules:Get sid: /Users/omni/Documents/rvc/weights/test.pth
INFO:rvc.modules.vc.modules:Loading: /Users/omni/Documents/rvc/weights/test.pth
/Users/omni/Documents/rvc/projects/rvc Desktop App/testing-rvc/venv/lib/python3.11/site-packages/torch/nn/utils/weight_norm.py:30: UserWarning: torch.nn.utils.weight_norm is deprecated in favor of torch.nn.utils.parametrizations.weight_norm.
warnings.warn("torch.nn.utils.weight_norm is deprecated in favor of torch.nn.utils.parametrizations.weight_norm.")
DEBUG:rvc.lib.infer_pack.models:gin_channels: 256, self.spk_embed_dim: 109
INFO:rvc.modules.vc.modules:Select index:
WARNING:rvc.modules.vc.modules:Traceback (most recent call last):
File "/Users/omni/Documents/rvc/projects/rvc Desktop App/testing-rvc/rvc/modules/vc/modules.py", line 114, in vc_single
self.hubert_model = load_hubert(self.config, hubert_path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/omni/Documents/rvc/projects/rvc Desktop App/testing-rvc/rvc/modules/vc/utils.py", line 23, in load_hubert
models, _, _ = checkpoint_utils.load_model_ensemble_and_task(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/omni/Documents/rvc/projects/rvc Desktop App/testing-rvc/venv/lib/python3.11/site-packages/fairseq/checkpoint_utils.py", line 448, in load_model_ensemble_and_task
raise IOError("Model file not found: {}".format(filename))
OSError: Model file not found:
Traceback (most recent call last):
File "/Users/omni/Documents/rvc/projects/rvc Desktop App/testing-rvc/venv/bin/rvc", line 6, in
sys.exit(main())
^^^^^^
File "/Users/omni/Documents/rvc/projects/rvc Desktop App/testing-rvc/rvc/utils/cli/cli.py", line 29, in main
cli()
File "/Users/omni/Documents/rvc/projects/rvc Desktop App/testing-rvc/venv/lib/python3.11/site-packages/click/core.py", line 1157, in call
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/omni/Documents/rvc/projects/rvc Desktop App/testing-rvc/venv/lib/python3.11/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "/Users/omni/Documents/rvc/projects/rvc Desktop App/testing-rvc/venv/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/omni/Documents/rvc/projects/rvc Desktop App/testing-rvc/venv/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/omni/Documents/rvc/projects/rvc Desktop App/testing-rvc/venv/lib/python3.11/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/omni/Documents/rvc/projects/rvc Desktop App/testing-rvc/rvc/utils/cli/handler/infer.py", line 130, in infer
wavfile.write(outputpath, tgt_sr, audio_opt)
File "/Users/omni/Documents/rvc/projects/rvc Desktop App/testing-rvc/venv/lib/python3.11/site-packages/scipy/io/wavfile.py", line 771, in write
dkind = data.dtype.kind
^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'dtype'

I believe this is because my hubert_path is set to nothing in .env but I'm not sure what I'm supposed to set it to

I think maybe the init command isn't actually going through even though it's not giving an error. I see that there is a get hubert script that never ran so I manually added hubert and it looks like other files are missing. Now trying to resolve rmvpe.pt.
This can probably be traced back to my original issue of not being able to run the rvc command in a normal way. Let me know if you have any recommendations!

Ok so I was able to get it working by copying rmvpe and hubert_base from the RVC Beta 0717 folder on hugging face and then explicitly adding their file paths to .env
If rvc init had the right permissions and was working properly, would all of this been automatic?
For anyone who needs it, here was my process:

Install RVC in a Python 3.11 virtual environment (3.12 wouldn't work with torch)
rvc commands wouldn't work for me, so I right-clicked rvc in the venv bin in vscode and selected run in terminal. I then copied that command (which is a python path and an rvc path) and used that in place of "rvc" whenever I want to run it
rvc init didn't seem to do anything so I did rvc env create - set manual paths for rmvpe and hubert and grabbed them from the RVC Beta 0717 on hugging face
run the rvc infer command using whatever settings you choose
works!

having the same issue @briangrider , how did you get it working on a fresh clone?
these are the steps i am doing:
git clone https://github.com/RVC-Project/Retrieval-based-Voice-Conversion.git
python -m venv .venv && source .venv/bin/activate
poetry install
rvc init

the init shows no output but seems to do nothing...

@beatsgithub I just ignored it and used env create
I also had to grab the hubert_path and rmvpe.pt from RVC 0717 beta from hugging face, put them in relative folders and put them in the env file