NetCDF chunking using extra memory with Iris 2.4

On behalf of Scitools user(s):
I have an Iris/Dask/NetCDF question about chunking data and collapsing dimensions, in this case to compute ensemble statistics (ensemble mean/median/standard deviation fields).
The NetCDF files that i'm using have chunking is optimized for this operation. Rather than using the NetCDF file's chunking, Iris/Dask is putting each ensemble member into its own chunk. I think this is resulting in my code trying to read all of the data into memory at once. Is there a simple way to make make Iris use the NetCDF file's chunking when it loads NetCDF data into a Dask array?
Also, the memory requirement for this and other operations seem to have gone up noticeably when moving from Iris 2.2 to Iris 2.4. I've been busy doubling my memory allocations to allow for it. Running on Iris 2.2 this task took a bit over an hour to compute with 16GB RAM allocated. With Iris 2.4 it no longer fits into that memory or within a 2 hour job.
To recreate issue:
First, I created two Conda installations, one with Iris v2.2 and one with Iris v2.4:
conda create --name iris-v2.2 -c conda-forge python=3.7 iris=2.2 memory_profiler
conda create --name iris-v2.4 -c conda-forge python=3.7 iris=2.4 memory_profiler
Then, after activating each environment, I ran (python code below):
python -m mprof run dask_issue.py
mprof plot
For Iris v2.2:
Loading the data:
Has lazy data: True
Chunksize: (1, 1, 36, 72)
Extracting a single year:
Has lazy data: True
Chunksize: (1, 1, 36, 72)
Realising the data of the single year:
Has lazy data: False
Chunksize: (9, 12, 36, 72)
For Iris v2.4:
Loading the data:
Has lazy data: True
Chunksize: (1, 2028, 36, 72)
Extracting a single year:
Has lazy data: True
Chunksize: (1, 12, 36, 72)
Realising the data of the single year:
Has lazy data: False
Chunksize: (9, 12, 36, 72)
The plots produced by memory_profiler shows the memory usage when realising extracted data using Iris v2.4 is over 5 times the memory usage when realising extracted data using Iris v2.2. The time taken to realise the data also increases. The cube prior to extraction contains data from 1850 to 2018. Is it possible that all the data (rather than just the year that had been extracted) are being realised when using Iris v2.4?
dask_issue.py code referenced above:
#!/usr/bin/env python
import iris
def main():
print('Loading the data:')
cube = load_cube()
print_cube_info(cube)
    print('Extracting a single year:')
    cube = extract_year(cube)
    print_cube_info(cube)

    print('Realising the data of the single year:')
    realise_data(cube)
    print_cube_info(cube)

@profile
def load_cube():
    filename = ('[path]/HadSST4/analysis/HadCRUT.<IP_ADDRESS>.SST.analysis.anomalies.?.nc')
    cube = iris.load_cube(filename)
    return cube

@profile
def extract_year(cube):
    year = 1999
    time_constraint = iris.Constraint(
        time=lambda cell: cell.point.year == year)
    cube = cube.extract(time_constraint)
    return cube

@profile
def realise_data(cube):
    cube.data

def print_cube_info(cube):
    tab = ' ' * 4
    print(f'{tab}Has lazy data: {cube.has_lazy_data()}')
    print(f'{tab}Chunksize: {cube.lazy_data().chunksize}')

if __name__ == '__main__':
main()

Looks like NetCDF chunk handling was changed in Iris between v2.2 and v2.4, here: https://github.com/SciTools/iris/pull/3361. That might be a good place to start looking; it's possible that that change had an uncaught consequence, and this is what's reported here.
For the sake of the original reporter...

Is there a simple way to make make Iris use the NetCDF file's chunking when it loads NetCDF data into a Dask array?

Iris already does - this was introduced in #3131.
@alastair-gemmell have you checked what the behaviour is in Iris v3?
Something else that would be interesting to know is what NetCDF reports the dataset chunking as for these problem files. For example:
import netCDF4
ds = netCDF4.Dataset("/path/to/my-file.nc")
print(ds.variables["my-data-var-name"].chunking())

Looks like NetCDF chunk handling was changed in Iris between v2.2 and v2.4, here: https://github.com/SciTools/iris/pull/3361. That might be a good place to start looking; it's possible that that change had an uncaught consequence, and this is what's reported here.

I did some cursory analysis on this problem and I can confirm it stems from the changes made in #3361 - if I substitute the v2.2.0 version of _lazy_data.py into the latest Iris the behaviour replicates the v2.2 behaviour described.

I suspect we (IMPROVER) are also effected by this issue...

Confirmed this issue is what is effecting us - substituting as_lazy_data as per https://github.com/SciTools/iris/issues/4107#issuecomment-826720339 results in considerable performance improvements.

Possibly some progress : #4572 aims to address #3333

@cpelley Is there no possibility (development path in dask) where dask arrays can be made to genuinely re-chunk the original dask array? (they are currently immutable)

I really don't think this is likely.
Chunking as a static ahead-of-time time decision is deeply wired in Dask.  Because, I think, its concept of a graph requires that chunks behave like 'delayed' objects -- i.e. they are graph elements representing a black-box task with a single result.  So, the chunks make the graph, and different chunks will mean a new graph.

@ehogan & @cpmorice you were the ones that originally brought this to our attention, thanks very much for your vigilance!
We have an open internal support ticket for it, but this has since moved well beyond support - we diagnosed the precise cause, and as you can see here it's difficult to find a resolution that gives everyone acceptable performance.
I'm therefore closing the internal support ticket. I wanted to tag you both here so that you can monitor our latest thoughts on the issue, should you wish.

Gonna keep this open until the release of 3.8 (#5363), since #5588 should allow users to avoid this problem. Would be good to know if this works in action.


The REST API doesn't return correct values of JSON fields

Using the automatically generated REST API to get entities contaning object fields, you'll get
[object Object] instead of the value string serialization, for each of such fields.

Thanks for reporting this! Can you please provide an example on how to reproduce this? SQL Dump and a sample API call would be enough

GET http://<IP_ADDRESS>:3042/games
Response:
[{"id":"2","name":"run","config":"[object Object]"}]
sql.zip

We should map json fields to JSON schema object with arbitrary values (in OpenAPI/REST) and use https://github.com/taion/graphql-type-json for GraphQL instead.


Detect .env file ?

Current Behavior
it does not detect .env file internally

Desired Behavior
it should detect .env file , as of now it detect the host env example react/next

Suggested Solution

Who does this impact? Who is this for?

me and my team and other tsdx users

Describe alternatives you've considered
-none

Additional context

Same need!

I need that too

I figured out by my self create .env.local and start variable with REACT_APP
for example:
REACT_APP_KEY=test


Time is not disabled when not set in a Talk page

Describe the bug
A previous closed issue supposedly introduced logic to suppress times be added to talk pages when time is not declared. This does not work for me and times are defaulting to 12AM.
To Reproduce
in content/event/example/index.md
# Talk start and end times.
#   End time can optionally be hidden by prefixing the line with `#`.
date: "2020-07-30"

# date_end: "2030-06-01"
all_day: false

# Schedule page publish date (NOT talk date).
publishDate: "2021-08-24"

Prints the following:

Jul 30, 2020 12:00 AM

Expected behavior
ignore time when it's not provided

Jul 30, 2020

Technical details:

Link to your GitHub project: https://github.com/drwilkins/mattwilkinsbio
Wowchemy Version (from your go.mod): v0.0.0-20210303111450-5529404d1942
Hugo Version (run hugo version): 0.81.0
Browser/OS: Mac OS Big Sur
Wowchemy Template (e.g. Academic): Academic

If applicable, add screenshots to help explain the issue.

Well, I guess the problem is fixed by setting all_day: true. The logic of this is different from ignoring time that is not supplied, so I seem to have just missed the obvious. It would be helpful to change the name of the variable to ignore_time or something, since nobody gives a talk all day, or add a helpful comment explaining that this will toggle Time on the event page. It's incredibly obvious in retrospect, but a simple comment would've saved a lot of time.


feat: init command

This Pull Request replaces the previous one (#200), which was closed due to incorrect branch.
Link to the previous PR: #200

Here are a few minor updates you should make:
Add support for cron jobs to the Go platform.
Ensure it prompts for the port when setting up liara.json for Python apps, as it currently does not.

Do we need disk and container health configurations for all platforms? The static platform doesn't have these configurations.

I referred to the Liara documentation for this command. As stated in the documentation, all platforms support disk and health check configurations. However, in some platforms, users cannot set these configurations when deploying via the console. Nevertheless, they are fully adjustable when deploying using the CLI.
To test this, you can deploy a static application on the static platform using the CLI with the following configurations in the liara.json file:
{
   "platform": "static",
    "disks": [
        {
            "name": "testDisk",
            "mountTo": "./assets"
        }
    ],

        "healthCheck": {
          "command": "CMD notExistingCommand  || exit 1",
          "interval": 3000,
          "timeout": 1000,
          "retries": 2,
          "startPeriod": 5000
        }

}


Rewrite and fixes for lot of grammar issues

Hi @sheetalkamat, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!

It looks like you're a Microsoft contributor (Sheetal Nandi). If you're full-time, we DON'T require a Contribution License Agreement. If you are a vendor, please DO sign the electronic Contribution License Agreement. It will take 2 minutes and there's no faxing! https://cla.microsoft.com.

TTYL, MSBOT;

merging this as suggested by @mhegazy

Nice work! Looking forward to testing it out.

When it will be available?

Should it also effect on this?
https://github.com/Microsoft/vscode/issues/11398
And when will it be available?

And when will it be available?

we should have this in a vscode extension for users to try it out next week. if every thing looks good, should be available in vscode's subsequent releases.

@mhegazy
Thanks a lot for the hard work! Can you give an overview of the most important grammar issues that get fixed in this PR?

there are some 120 issues that are fixed by this change (linked above). it is almost a rewrite at this point. give it a try and let us know if there are other issues we are not aware of.

give it a try and let us know if there are other issues we are not aware of.

So it is not available yet right? What will be need to it out, just use vscode insiders?

@whitecolor the issue you mentioned seems to be fixed.

@sheetalkamat is this going to be merged soon? Can i test it already?
Thanks

@farfromrefug you can find these changes in the VSCode extension: https://marketplace.visualstudio.com/items?itemName=ms-vscode.typescript-javascript-grammar

Thanks @mhegazy will test it right away!


V1.1.0 Release

Added new features and actually ran prettier

Need to remove vs-code config before release . My apologies


Suppressed Pylint Warnings not working when none of new warnings enabled

Describe the bug
I installed pylint-pytest plugin only to suppress pylint warnings related to pytest package usage. All introduced new warnings are disabled because other linters are checking them.
For some reason, the W0621 warning was constantly displayed on my test file, when the pytest fixture was defined and later used in the tests. After adding any newly introduced warnings to the config file, the W0621 warning stopped displaying.
To Reproduce
Package versions

pylint==3.1.0
pylint-pytest==1.1.8
pytest==8.1.1
pytest-cov==5.0.0
pytest-mock==3.14.0

Folder structure
Doesn't matter, just create a file with content below.
File content
import pytest

@pytest.fixture()
def my_fixture():
    return

def test_fixture(my_fixture):
    pass

pylint output with the plugin
$ python -m pylint tests/example.py
************* Module tests.example
tests\example.py:9:17: W0621: Redefining name 'my_fixture' from outer scope (line 5) (redefined-outer-name)

-----------------------------------
Your code has been rated at 8.00/10

Expected behavior
W0621 warning is suppressed by pylint-pytest plugin when fixture and tests using it are defined in the same file.

It seems that we do have a test case exactly like the one you are describing:
https://github.com/pylint-dev/pylint-pytest/blob/057510452686eedbbced44154c526ffd6e09d2ad/tests/input/redefined-outer-name/caller_not_a_test_func.py#L1-L11
(idk why it writes there "invalid test case" - probably it means "this is not a valid test case, since it does not start with test (https://docs.pytest.org/en/7.1.x/explanation/goodpractices.html#conventions-for-python-test-discovery))
Extending our test case
$ bat -pp tests/test_caller_not_a_test_func.py
import pytest

@pytest.fixture
def this_is_a_fixture():
    return True

def not_a_test_function(this_is_a_fixture):
    # invalid test case...
    assert True

def test_function_valid(this_is_a_fixture):
    assert True

and running pylint against it gives:
$ pylint tests --disable=C
************* Module test_caller_not_a_test_func
tests/test_caller_not_a_test_func.py:9:24: W0621: Redefining name 'this_is_a_fixture' from outer scope (line 5) (redefined-outer-name)
tests/test_caller_not_a_test_func.py:9:24: W0613: Unused argument 'this_is_a_fixture' (unused-argument)
tests/test_caller_not_a_test_func.py:14:24: W0621: Redefining name 'this_is_a_fixture' from outer scope (line 5) (redefined-outer-name)
tests/test_caller_not_a_test_func.py:14:24: W0613: Unused argument 'this_is_a_fixture' (unused-argument)
************* Module test_example
tests/test_example.py:7:4: W0101: Unreachable code (unreachable)
tests/test_example.py:10:17: W0621: Redefining name 'my_fixture' from outer scope (line 5) (redefined-outer-name)

-------------------------------------------------------------------
Your code has been rated at 5.38/10 (previous run: 10.00/10, -4.62)

It does certainly intrigue me why:

Our test works correctly
Our test fails to work, when moved to the (bare-bones)

... scratch all of the above: is pylint-pytest enabled?
$ pylint --generate-rcfile | grep pylint_pytest
load-plugins=pylint_pytest

Yeah I had that plugin enabled I even debugged the pylint run to make sure it was loaded and it was.

Yes, it is.
$ cat tests/test_fixture_issue.py
import pytest

@pytest.fixture()
def my_fixture():
    return True

def test_fixture(my_fixture):
    assert my_fixture

$ pylint tests --disable=C
************* Module test_fixture.tests.test_fixture_issue
tests\test_fixture_issue.py:9:17: W0621: Redefining name 'my_fixture' from outer scope (line 5) (redefined-outer-name)

-----------------------------------
Your code has been rated at 8.00/10

PS. Sorry for late response, I totally missed your question.

No worries. But "try to keep the loop", if it is possible 🙃
I have tried again reproducing your issue, but I fail to do so:
ua@h /tmp/pylint_pytest_testbench$ pylint tests --disable=C

-------------------------------------------------------------------
Your code has been rated at 10.00/10 (previous run: 7.69/10, +2.31)

ua@h /tmp/pylint_pytest_testbench$ pylint tests
************* Module test_example
tests/test_example.py:10:0: C0304: Final newline missing (missing-final-newline)
tests/test_example.py:1:0: C0114: Missing module docstring (missing-module-docstring)
tests/test_example.py:5:0: C0116: Missing function or method docstring (missing-function-docstring)
tests/test_example.py:9:0: C0116: Missing function or method docstring (missing-function-docstring)

-------------------------------------------------------------------
Your code has been rated at 2.00/10 (previous run: 10.00/10, -8.00)

ua@h /tmp/pylint_pytest_testbench$ ls -lah
total 12K
drwxrwxr-x   5 abcdefg abcdefg  160 Ιουν 27 00:11 .
drwxrwxrwt 118 root    root    3,4K Ιουν 27 00:11 ..
-rw-rw-r--   1 abcdefg abcdefg   37 Ιουν 27 00:07 .pylintrc
drwxrwxr-x   3 abcdefg abcdefg  120 Ιουν 27 00:11 .pytest_cache
-rw-rw-r--   1 abcdefg abcdefg   79 Ιουν 27 00:08 README.md
-rw-rw-r--   1 abcdefg abcdefg   87 Ιουν 27 00:07 requirements.txt
drwxrwxr-x   3 abcdefg abcdefg   80 Ιουν 27 00:11 tests
drwxrwxr-x   5 abcdefg abcdefg  140 Ιουν 27 00:07 .venv
ua@h /tmp/pylint_pytest_testbench$ tree
.
├── README.md
├── requirements.txt
└── tests
    ├── __pycache__
    │   └── test_example.cpython-38-pytest-8.1.1.pyc
    └── test_example.py

2 directories, 4 files

pylint_pytest_testbench.tar.gz
Am I missing something? 😕

Alright, I found the issue. Shorter version of my pyproject.toml config:
[tool.pylint.main]
disable = "all"
enable=["C0114", "W0621"]
jobs = 1
py-version = "3.11"
load-plugins = [
  "pylint.extensions.bad_builtin",
  "pylint.extensions.broad_try_clause",
  "pylint.extensions.code_style",
  "pylint.extensions.confusing_elif",
  "pylint.extensions.consider_refactoring_into_while_condition",
  "pylint.extensions.consider_ternary_expression",
  "pylint.extensions.dict_init_mutate",
  "pylint.extensions.docparams",
  "pylint.extensions.dunder",
  "pylint.extensions.empty_comment",
  "pylint.extensions.eq_without_hash",
  "pylint.extensions.for_any_all",
  "pylint.extensions.mccabe",
  "pylint.extensions.no_self_use",
  "pylint.extensions.overlapping_exceptions",
  "pylint.extensions.private_import",
  "pylint.extensions.redefined_variable_type",
  "pylint.extensions.set_membership",
  "pylint.extensions.typing",
  "pylint.extensions.while_used",
  "pylint_pytest",
]

That disable="all" part causes the issue. But I don't know why. I only needed checks in my "enable" part, so I am disabling all before, as explained in pylint's help.

Okay - now I replicated it
$ pylint tests --disable=all --enable=C0114 --enable=W0621
************* Module test_example
tests/test_example.py:1:0: C0114: Missing module docstring (missing-module-docstring)
tests/test_example.py:9:17: W0621: Redefining name 'my_fixture' from outer scope (line 5) (redefined-outer-name)

------------------------------------------------------------------
Your code has been rated at 6.00/10 (previous run: 6.00/10, +0.00)

I'll see what I can do

I know at least from where the issue is coming from:
"Somehow", when using pylint tests --disable=all --enable=C0114 --enable=W0621, the check https://github.com/pylint-dev/pylint-pytest/blob/8310a88916e4b0b7462fc525c56d60b9af4dc126/pylint_pytest/checkers/variables.py#L103 returns false.
Which means that, the seeding function https://github.com/pylint-dev/pylint-pytest/blob/46cf4500a51d3e25cabaa9c651ec1d8c2b8754ac/pylint_pytest/checkers/fixture.py#L131 is not executed either.
We depend on pytest telling us which function names are fixtures.
I'll have to ask pylint for help on that one.

... there might be a simpler explanation for that issue (and totally/trivially our fault), but I'd like to verify it first

I have debugged the issue on the https://github.com/pylint-dev/pylint-pytest/releases/tag/v2.0.0a0, and it seems to be easier to figure out there:
Given the way that pylint is invoked i.e., catch-all --disable, it seems that our checker FixtureChecker is deemed as useless, and its hooks not visited (pertinent to your issue, specifically visit_module).

There is a way to make it work, specifically for your case - there be dragons, though
It has a very simple workaround in v2.0.0a0:
diff --git a/pylint_pytest/checkers/variables.py b/pylint_pytest/checkers/variables.py
index 7dde882..df2bdb5 100644
--- a/pylint_pytest/checkers/variables.py
+++ b/pylint_pytest/checkers/variables.py
@@ -4,7 +4,6 @@ from astroid import Arguments, Module
 from astroid.nodes.node_ng import NodeNG
 from pylint.checkers.variables import VariablesChecker
 from pylint.interfaces import Confidence

 from pylint_pytest.utils import _can_use_fixture, _is_same_module

 from .fixture import FixtureChecker
@@ -13,6 +12,15 @@ from .fixture import FixtureChecker
 class CustomVariablesChecker(VariablesChecker):
     """Overrides the default VariablesChecker of pylint to discard unwanted warning messages"""

+    def visit_module(self, node: NodeNG) -> None:
+        FixtureChecker.visit_module(self, node)
+
+    # def visit_decorators(self, node: NodeNG) -> None:
+    #     FixtureChecker.visit_decorators(self, node)
+
+    # def visit_functiondef(self, node: NodeNG) -> None:
+    #     FixtureChecker.visit_functiondef(self, node)
+
     # pylint: disable=protected-access
     # this class needs to access the fixture checker registries

However it will fail when activating visit_functiondef - or fail to fact-check
tests/caller_not_a_test_func.py:9:1: W6403: Using a deprecated positional arguments for fixture (deprecated-positional-argument-for-pytest-fixture)

in
diff --git a/tests/input/redefined-outer-name/caller_not_a_test_func.py b/tests/input/redefined-outer-name/caller_not_a_test_func.py
index 985a519..67d59ea 100644
--- a/tests/input/redefined-outer-name/caller_not_a_test_func.py
+++ b/tests/input/redefined-outer-name/caller_not_a_test_func.py
@@ -6,6 +6,7 @@ def this_is_a_fixture():
     return True

+@pytest.fixture("function")
 def not_a_test_function(this_is_a_fixture):
     # invalid test case...
     assert True

.. and it stands to reason that we should actually do that 😕

For the "soon coming"™ https://github.com/pylint-dev/pylint-pytest/releases/tag/v2.0.0, I guess I should try to figure out a PassiveChecker of sorts that could be delegated to, to visit and build the fixture information that we need - which then both CustomVariablesChecker and FixtureChecker could ask for that information.
I am not sure what should I do for the v1.x series. v2.x is not "backwards compatible" code-wise per se, but I want to carve it out for semantic reasons (also big refactorings, like https://github.com/pylint-dev/pylint-pytest/pull/29)

For me, I can wait till v2.x if that helps you. Apparently, I'm the only one who was irritated enough to raise an issue. 😄 For now, we are just disabling that message, but would like to enable it in the future.

Shameless advertizing:
What would really help me, is if you know a Python Powerhouse Dev that would be interested in figuring out this Heisenbug: https://github.com/pylint-dev/pylint-pytest/issues/68
I haven't spent "countless hours" on it, but I have spent enough to drive me a little crazy 😅

I mean ... idk. Maybe I should say to pylint to actually evaluate our rule, as it affects the W0621 message?
https://github.com/pylint-dev/pylint-pytest/blob/b493935542e199148670245617be5c08ddeefee0/pylint_pytest/checkers/fixture.py#L42-L74
We don't issue it of course, but we do "omit" it 😅
More things to ask from pylint, as I am in the process of learning the pylint ropes

I can try looking at it, maybe with some of my friends. Would be fun, I just dunno about my time availability. Will let you know.


switch from raw esbuild to tsup

Description
this pr switch from esbuild to tsup built upon the prior work in #119 by @beam41
note: tsup uses esbuild under the hood
why tsup?

nice CLI output that shows the output size (see screenshot below)
no need to maintain a custom build script
it just works when building node application without having to manually configure esbuild-plugin-external-package
when putting #!/usr/bin/env node at the top of the entry file, the compiled output will automatically becomes executable (chmod +x)
has a built-in watch mode: pnpm build --watch

Type of change

[x] Refactor or other

Screenshot

Checklist:

[x] I have run pnpm format and my code don't have any linting issues

[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to the documentation

@Leomotors done, renamed to index although i'm more inclined to name it main
i think about a developer wanting to go to the entry point. they hit Cmd+P in VS Code and type index. they see many index.ts file.
can i change to main.ts instead?


fix(cn): modify errors in translation

Finish translate hooks-state.md. Please review it.

a an 等词不建议翻译


Array dispatch

Fixes #197.
This is based on #252 to avoid conflicts, therefors you can ignore the first commit when reviewing this PR. Splitting the array header is required because the implementation of array::visit includes layout/dispatch.hpp, which includes all the layouts, and some of them depends on array.

(Force pushed to drop first commit and resolve merging conflicts)


Unable to create local link (XCP-NG + CS 4.15 + PreSetup SR)

ISSUE TYPE

Bug Report

COMPONENT NAME

Hypervisor

CLOUDSTACK VERSION

4.15 (fresh install)

CONFIGURATION

Management Server : CentOS 7.9 (also tested on CentOS 8.2, Oracle Linux 8.3)
Hypervisor : 2 x XCP-ng 7.6 (also tested on 8.1, 8.2, in the same pool)
Zone : Advanced without SG PreSetup SR
SUMMARY

On a freshly installed CS 4.15 I can't deploy zone with presetup iscsi storage. Unable to create the local link network. With NFS deploy was complited. On CS 4.14 my configuration works well.
STEPS TO REPRODUCE

Add an XCP-ng 7.6 (8.1, 8.2) host under an Advanced Zone with PreSetup iSCSI storage.

EXPECTED RESULTS

Local link creation should be successful.

ACTUAL RESULTS

2021-02-02 19:56:04,663 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-7:ctx-090ea653) (logid:e138cefc) Seq 2-9037880027202453511: Executing request
2021-02-02 19:56:04,671 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-7:ctx-090ea653) (logid:8543a8cb) Can't find a vif on dom0 for link local, creating a new one
2021-02-02 19:56:04,673 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-7:ctx-090ea653) (logid:8543a8cb) Lowest available Vif device number: 0 for VM: Control domain on host: xcp-ng76-u4037-02
2021-02-02 19:56:04,677 WARN  [c.c.h.x.r.CitrixResourceBase] (DirectAgent-7:ctx-090ea653) (logid:8543a8cb) Unable to create local link network
The server failed to handle your request, due to an internal error.  The given message may give details useful for debugging the problem.
        at com.xensource.xenapi.Types.checkResponse(Types.java:1609)
        at com.xensource.xenapi.Connection.dispatch(Connection.java:395)
        at com.cloud.hypervisor.xenserver.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:457)
        at com.xensource.xenapi.VIF.create(VIF.java:319)
        at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.setupLinkLocalNetwork(CitrixResourceBase.java:4822)
        at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixReadyCommandWrapper.execute(CitrixReadyCommandWrapper.java:52)
        at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixReadyCommandWrapper.execute(CitrixReadyCommandWrapper.java:38)
        at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.java:122)
        at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:1759)
        at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:315)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)
2021-02-02 19:56:04,678 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-7:ctx-090ea653) (logid:8543a8cb) Seq 2-9037880027202453511: Response Received:
2021-02-02 19:56:04,678 DEBUG [c.c.a.t.Request] (DirectAgent-7:ctx-090ea653) (logid:8543a8cb) Seq 2-9037880027202453511: Processing:  { Ans: , MgmtId:<PHONE_NUMBER>22411, via: 2(xcp-ng76-u4037-02), Ver: v1, Flags: 110, [{"com.cloud.agent.api.UnsupportedAnswer":{"result":"false","details":"Unsupported command issued: com.cloud.agent.api.ReadyCommand.  Are you sure you got the right type of server?","wait":"0"}}] }
2021-02-02 19:56:04,678 DEBUG [c.c.a.m.AgentAttache] (DirectAgent-7:ctx-090ea653) (logid:8543a8cb) Seq 2-9037880027202453511: No more commands found
2021-02-02 19:56:04,678 DEBUG [c.c.a.t.Request] (qtp1026871825-24:ctx-64980d9f ctx-d8cf8904) (logid:8543a8cb) Seq 2-9037880027202453511: Received:  { Ans: , MgmtId:<PHONE_NUMBER>22411, via: 2(xcp-ng76-u4037-02), Ver: v1, Flags: 110, { UnsupportedAnswer } }
2021-02-02 19:56:04,678 WARN  [c.c.a.m.AgentManagerImpl] (qtp1026871825-24:ctx-64980d9f ctx-d8cf8904) (logid:8543a8cb) Unsupported Command: Unsupported command issued: com.cloud.agent.api.ReadyCommand.  Are you sure you got the right type of server?
2021-02-02 19:56:04,678 DEBUG [c.c.a.m.AgentManagerImpl] (qtp1026871825-24:ctx-64980d9f ctx-d8cf8904) (logid:8543a8cb) Details from executing class com.cloud.agent.api.ReadyCommand: Unsupported command issued: com.cloud.agent.api.ReadyCommand.  Are you sure you got the right type of server?
2021-02-02 19:56:04,679 INFO  [c.c.a.m.AgentManagerImpl] (qtp1026871825-24:ctx-64980d9f ctx-d8cf8904) (logid:8543a8cb) Host 2 is disconnecting with event AgentDisconnected
2021-02-02 19:56:04,680 DEBUG [c.c.a.m.AgentManagerImpl] (qtp1026871825-24:ctx-64980d9f ctx-d8cf8904) (logid:8543a8cb) The next status of agent 2is Alert, current status is Connecting
2021-02-02 19:56:04,680 DEBUG [c.c.a.m.AgentManagerImpl] (qtp1026871825-24:ctx-64980d9f ctx-d8cf8904) (logid:8543a8cb) Deregistering link for 2 with state Alert
2021-02-02 19:56:04,681 DEBUG [c.c.a.m.AgentManagerImpl] (qtp1026871825-24:ctx-64980d9f ctx-d8cf8904) (logid:8543a8cb) Remove Agent : 2
2021-02-02 19:56:04,681 DEBUG [c.c.a.m.DirectAgentAttache] (qtp1026871825-24:ctx-64980d9f ctx-d8cf8904) (logid:8543a8cb) Processing disconnect 2(xcp-ng76-u4037-02)
2021-02-02 19:56:04,681 DEBUG [c.c.a.m.AgentManagerImpl] (qtp1026871825-24:ctx-64980d9f ctx-d8cf8904) (logid:8543a8cb) Sending Disconnect to listener: com.cloud.hypervisor.xenserver.discoverer.XcpServerDiscoverer
2021-02-02 19:56:04,681 DEBUG [c.c.a.m.AgentManagerImpl] (qtp1026871825-24:ctx-64980d9f ctx-d8cf8904) (logid:8543a8cb) Sending Disconnect to listener: com.cloud.hypervisor.hyperv.discoverer.HypervServerDiscoverer
2021-02-02 19:56:04,681 DEBUG [c.c.a.m.AgentManagerImpl] (qtp1026871825-24:ctx-64980d9f ctx-d8cf8904) (logid:8543a8cb) Sending Disconnect to listener: com.cloud.vm.ClusteredVirtualMachineManagerImpl
2021-02-02 19:56:04,681 DEBUG [c.c.a.m.AgentManagerImpl] (qtp1026871825-24:ctx-64980d9f ctx-d8cf8904) (logid:8543a8cb) Sending Disconnect to listener: com.cloud.hypervisor.vmware.manager.VmwareManagerImpl
2021-02-02 19:56:04,681 DEBUG [c.c.a.m.AgentManagerImpl] (qtp1026871825-24:ctx-64980d9f ctx-d8cf8904) (logid:8543a8cb) Sending Disconnect to listener: com.cloud.deploy.DeploymentPlanningManagerImpl
2021-02-02 19:56:04,681 DEBUG [c.c.a.m.AgentManagerImpl] (qtp1026871825-24:ctx-64980d9f ctx-d8cf8904) (logid:8543a8cb) Sending Disconnect to listener: com.cloud.storage.secondary.SecondaryStorageListener
2021-02-02 19:56:04,681 DEBUG [c.c.a.m.AgentManagerImpl] (qtp1026871825-24:ctx-64980d9f ctx-d8cf8904) (logid:8543a8cb) Sending Disconnect to listener: com.cloud.network.security.SecurityGroupListener
2021-02-02 19:56:04,681 DEBUG [c.c.a.m.AgentManagerImpl] (qtp1026871825-24:ctx-64980d9f ctx-d8cf8904) (logid:8543a8cb) Sending Disconnect to listener: com.cloud.storage.listener.StoragePoolMonitor

I am able to experiment in my test environment as needed (including CS builds/reinstall or XCP reinstalls).

@edikevich did you face this issue only with an iscsi presetup SR and not with a NFS presetup SR? I tried with ACS 4.15, xcp-ng8.1 and a NFS presetup SR as primary store and zone worked fine.
@rhtyd @andrijapanicsb anything specific you guys can think of which can cause this?

Hi. Yes, I have issue only with iSCSI presetup SR

@edikevich  I wasn't able to reproduce the issue. I have the following SR on the host

Following were the details added for primary store during zone deployment

And I was able to successfully deploy the zone:

However, I did notice that the zone deployment view in the new UI has an issue wrt adding the Primary store with presetup protocol which has been addressed with https://github.com/apache/cloudstack/pull/4845
If you still notice an issue, can you please let us know at which phase of zone deployment do you see a failure. Thanks!

Hi. I will try again on next week. I will let you know If I have a problem.

@edikevich looking forward to you test results. I've removed the critical severity as we're unable to reproduce the fix, we however found and fixed a UI issue as @Pearl1594 advised above.

Hi. My test environment is now busy. If you want you can close this issue and if I have problem I will reopen it.


Iterate through list to make a graph per term

It is possible to iterate through a list and save the plots individually?
e.g
list = ["term1", "term2", "term3"]
for term in list:
gc.pl.filtration_curves.compare_conditions(
adata=adata,
node_labels=term,  # Set node_labels to the current term
metric_key=metric_key,
palette="Set2",
save=f"figures/{term}_filtration_curves.pdf"
)
Thanks!

Yes, that is possible. The only change you need to make is to put "term" between square brackets:
gc.pl.filtration_curves.compare_conditions(
        adata=adata,
        node_labels=[term], # Set node_labels to the current term
        metric_key=metric_key,
        palette="Set2",
        save=f"figures/{term}_filtration_curves.pdf"
)

Ill try that thanks!

@spatts14, can you please confirm that this solved your issue? If so, we'll close this issue.

Hi sorry!
I got this error:
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[13], [line 2](vscode-notebook-cell:?execution_count=13&line=2)
      [1](vscode-notebook-cell:?execution_count=13&line=1) for term in cell_type_names:
----> [2](vscode-notebook-cell:?execution_count=13&line=2)     gc.pl.filtration_curves.compare_conditions(
      [3](vscode-notebook-cell:?execution_count=13&line=3)         adata=adata,
      [4](vscode-notebook-cell:?execution_count=13&line=4)         node_labels=[term], # Set node_labels to the current term
      [5](vscode-notebook-cell:?execution_count=13&line=5)         metric_key=metric_key,
      [6](vscode-notebook-cell:?execution_count=13&line=6)         palette="Set2",
      [7](vscode-notebook-cell:?execution_count=13&line=7)         save=f"figures/{term}_filtration_curves.pdf"
      [8](vscode-notebook-cell:?execution_count=13&line=8) )

File [/opt/anaconda3/envs/graphcompass/lib/python3.11/site-packages/graphcompass/pl/_filtration_curves.py:91](https://file+.vscode-resource.vscode-cdn.net/opt/anaconda3/envs/graphcompass/lib/python3.11/site-packages/graphcompass/pl/_filtration_curves.py:91), in compare_conditions(adata, node_labels, metric_key, return_ax, figsize, dpi, palette, right, save, **kwargs)
     [89](https://file+.vscode-resource.vscode-cdn.net/opt/anaconda3/envs/graphcompass/lib/python3.11/site-packages/graphcompass/pl/_filtration_curves.py:89) for i, key in enumerate(node_labels):
     [90](https://file+.vscode-resource.vscode-cdn.net/opt/anaconda3/envs/graphcompass/lib/python3.11/site-packages/graphcompass/pl/_filtration_curves.py:90)     try:
---> [91](https://file+.vscode-resource.vscode-cdn.net/opt/anaconda3/envs/graphcompass/lib/python3.11/site-packages/graphcompass/pl/_filtration_curves.py:91)         axes[i].step(
     [92](https://file+.vscode-resource.vscode-cdn.net/opt/anaconda3/envs/graphcompass/lib/python3.11/site-packages/graphcompass/pl/_filtration_curves.py:92)             df.weight,
     [93](https://file+.vscode-resource.vscode-cdn.net/opt/anaconda3/envs/graphcompass/lib/python3.11/site-packages/graphcompass/pl/_filtration_curves.py:93)             df[key],
     [94](https://file+.vscode-resource.vscode-cdn.net/opt/anaconda3/envs/graphcompass/lib/python3.11/site-packages/graphcompass/pl/_filtration_curves.py:94)             where='pre',
     [95](https://file+.vscode-resource.vscode-cdn.net/opt/anaconda3/envs/graphcompass/lib/python3.11/site-packages/graphcompass/pl/_filtration_curves.py:95)             label=label[0],
     [96](https://file+.vscode-resource.vscode-cdn.net/opt/anaconda3/envs/graphcompass/lib/python3.11/site-packages/graphcompass/pl/_filtration_curves.py:96)             color=category_color_map[label[0]],
     [97](https://file+.vscode-resource.vscode-cdn.net/opt/anaconda3/envs/graphcompass/lib/python3.11/site-packages/graphcompass/pl/_filtration_curves.py:97)             alpha=0.15
     [98](https://file+.vscode-resource.vscode-cdn.net/opt/anaconda3/envs/graphcompass/lib/python3.11/site-packages/graphcompass/pl/_filtration_curves.py:98)         )
     [99](https://file+.vscode-resource.vscode-cdn.net/opt/anaconda3/envs/graphcompass/lib/python3.11/site-packages/graphcompass/pl/_filtration_curves.py:99)     except KeyError:
    [100](https://file+.vscode-resource.vscode-cdn.net/opt/anaconda3/envs/graphcompass/lib/python3.11/site-packages/graphcompass/pl/_filtration_curves.py:100)         continue

TypeError: 'Axes' object is not subscriptable

Hi again!
I've pushed changes to resolve the bug you're seeing. We will include the updated code in the next release. If you want to make the plots immediately, you can install the updated code from the development branch:
git clone https://github.com/theislab/graphcompass.git
cd graphcompass
pip install -e .['development']

Thank you!


feat: Add gcp_pubsub_topic_name parameter for gcp notification integr…

Add support for GCP_PUBSUB_TOPIC_NAME for notification integration
Test Plan

[x] acceptance tests

References

#1480

/ok-to-test sha=2e0ae34

its failing go formatting, but its fine i can just do that real quick. Looks good. Thanks @krasnoludkolo !


Pre-allocate memory for slices to prevent repeated reallocation and copy during capacity expansion

https://github.com/openGemini/opengemini-client-go/blob/5898fcc3966deb7f364cf7eedbefcd6108c541fb/opengemini/retention_policy.go#L81-L129
variable: retentionPolicy

Although query retention policy isn't performance sensitive, but I think it can be optimized. #102 also reveals another problems of this class method. cc @cyruslo @Chenxulin97

There are many other places with the same problem. I've only listed this one.

There are many other places with the same problem. I've only listed this one.

It's not a problem , we will find the whole places and fix them . thanx for ur contribution.

ok

@dream-kzx Let's keep this issue open util it's resolved :)

closed by #117


chore(deps): upgrade secp256k1+enr

new secp256k1 and enr releases
blocked by https://github.com/gakonst/ethers-rs/issues/2158

working on it here https://github.com/gakonst/ethers-rs/pull/2260

this includes code changes that we need to do once this is upgraded.
reopening to draft

Codecov Report

Merging #1715 (06b43e0) into main (91f26f4) will decrease coverage by 28.70%.
The diff coverage is 100.00%.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
@@             Coverage Diff             @@
##             main    #1715       +/-   ##
===========================================
- Coverage   73.66%   44.97%   -28.70%
===========================================
  Files         400      400
  Lines       49542    49549        +7
===========================================
- Hits        36496    22283    -14213
- Misses      13046    27266    +14220

Flag
Coverage Δ

integration-tests
19.92% <88.88%> (+0.01%)
:arrow_up:

unit-tests
35.72% <88.88%> (-32.30%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

crates/net/dns/src/lib.rs
34.29% <100.00%> (-44.88%)
:arrow_down:

crates/net/dns/src/tree.rs
66.97% <100.00%> (-18.61%)
:arrow_down:

... and 190 files with indirect coverage changes
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


CameraGeometry position_to_pix_index returns -1 for nan input

https://github.com/cta-observatory/ctapipe/blob/39762693534f7dc9b884684c3f82e153bd9da560/ctapipe/instrument/camera/geometry.py#L859
For example, of one uses it on image centroid coordinates to make a "pixelized" 2d histogram, all non-reconstructed images end up in the highest-index pixel (=-1). It would be better to return np.nan.

I'd call that a bug since the index -1 has a very explicit meaning in python, and does not mean "not found".   However, this function returns an int, not a float, so NaN will not work.  Maybe an exception like InvalidPosition,  since in any case the caller needs to handle the failure case?

Another option would be to return a masked array

Returning the ids and mask sounds like the best option then, as it forces the user to remember that there are undefined values, and since they probably won't read the documentation! :-)

Could also change the invalid index to something like -interger_max (-9223372036854775808) to be really sure they aren't used as indices...

That sounds like the simplest solution to me, it will force the user to check the index range.

x = np.array([1.,2.,3.,4.,5.])
ind = np.ma.masked_array([1,2,2,-1,2], mask=[False, False, False, True, False])
print(x[ind])
[2. 3. 3. 5. 3.]

That seems worthy of a report to numpy I think....

Seems to be a long-known issue, I wonder whether there is a reason for that
https://stackoverflow.com/questions/3854665/indexing-with-masked-arrays-in-numpy

Another issue I just noticed is that position_to_pix_index yields an error in case it gets an empty array. This is a problem e.g. when passing an array with a filter that removes all events. It would be better to return an empty array, right?


McuBootConnectionError() after NK3 firmware update

I got a similar error as mentioned here but as pointed out by robin-nitrokey it seems to be unrelated to the original issue so I'm opening this one.
I ran nitropy nk3 update --version v1.4.0 to update the firmware. The update reached 100% and then the following error occured:
Command line tool to interact with Nitrokey devices 0.4.37
Do you want to download the firmware version v1.4.0? [Y/n]:
Download v1.4.0: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 894k/894k [00:00<00:00, 1.90MB/s]
Current firmware version:  [unknown]
Updated firmware version:  v1.4.0

Please do not remove the Nitrokey 3 or insert any other Nitrokey 3 devices during the update. Doing so may damage the Nitrokey 3.
Do you want to perform the firmware update now? [y/N]: y
Perform firmware update: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████| 451k/451k [4:00:09<00:00, 31.3B/s]
Critical error:
Failed to perform firmware update
	Exception encountered: McuBootConnectionError()

This is the error log:
403        INFO pynitrokey.cli Timestamp: 2023-05-27 00:52:03.461393
403        INFO pynitrokey.cli OS: uname_result(system='Linux', node='ARCH-MEDION', release='6.1.30-1-lts', version='#1 SMP PREEMPT_DYNAMIC Wed, 24 May 2023 17:18:57 +0000', machine='x86_64')
404        INFO pynitrokey.cli Python version: 3.11.3
406        INFO pynitrokey.cli pynitrokey version: 0.4.37
407        INFO pynitrokey.cli cryptography version: 36.0.2
408        INFO pynitrokey.cli ecdsa version: 0.18.0
408        INFO pynitrokey.cli fido2 version: 1.1.1
409        INFO pynitrokey.cli pyusb version: 1.2.1
410        INFO pynitrokey.cli spsdk version: 1.7.1
900        INFO  libusbsio Loading SIO library: /home/laborratte/test/venv-nitropy/lib/python3.11/site-packages/libusbsio/bin/linux_x86_64/libusbsio.so
903        INFO  libusbsio HID enumeration[93961937202960]: initialized
903       DEBUG  libusbsio HID enumeration[93961937202960]: device #0: MCU HID GENERIC DEVICE
904       DEBUG  libusbsio HID enumeration[93961937202960]: device #1: 2.4G Wireless Mouse
904        INFO  libusbsio HID enumeration[93961937202960]: finished, total 2 devices
970       DEBUG fido2.hid.linux Failed opening device /dev/hidraw0
Traceback (most recent call last):
  File "/home/laborratte/test/venv-nitropy/lib/python3.11/site-packages/fido2/hid/linux.py", line 98, in list_descriptors
    devices.append(get_descriptor(hidraw))
                   ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/laborratte/test/venv-nitropy/lib/python3.11/site-packages/fido2/hid/linux.py", line 55, in get_descriptor
    with open(path, "rb") as f:
         ^^^^^^^^^^^^^^^^
PermissionError: [Errno 13] Permission denied: '/dev/hidraw0'
971       DEBUG       root print: Critical error:
971       DEBUG       root print: No Nitrokey 3 device found
971       DEBUG       root listing all connected devices:
1030      DEBUG       root :: 'Nitrokey FIDO2' keys
1030      DEBUG       root :: 'Nitrokey Start' keys:
1051      DEBUG       root :: 'Nitrokey 3' keys
1053       INFO  libusbsio HID enumeration[93961937843472]: initialized
1054      DEBUG  libusbsio HID enumeration[93961937843472]: device #0: MCU HID GENERIC DEVICE
1054      DEBUG  libusbsio HID enumeration[93961937843472]: device #1: 2.4G Wireless Mouse
1054       INFO  libusbsio HID enumeration[93961937843472]: finished, total 2 devices
1120       INFO spsdk.mboot.mcuboot Connect: MCU HID GENERIC DEVICE (0x20A0, 0x42DD)
1120      DEBUG spsdk.mboot.interfaces.usb Open Interface
1120       INFO libusbsio.hidapi.dev Opening HID device at path: 'b'/dev/hidraw1''
1177       INFO libusbsio.hidapi.dev HID device<PHONE_NUMBER>8784 is now open
1177       INFO spsdk.mboot.mcuboot CMD: GetProperty('UniqueDeviceIdent', index=0)
1178      DEBUG spsdk.mboot.mcuboot TX-PACKET: Tag=GetProperty, Flags=0x00, P[0]=0x00000012, P[1]=0x00000000
1178      DEBUG spsdk.mboot.interfaces.usb OUT[16]: 01, 00, 0C, 00, 07, 00, 00, 02, 12, 00, 00, 00, 00, 00, 00, 00
1178      DEBUG libusbsio.hidapi.dev HID device<PHONE_NUMBER>8784 writing[16]: 0x01000c00070000021200000000000000
1179      DEBUG libusbsio.hidapi.dev HID device<PHONE_NUMBER>8784 wrote 59 bytes
1179      DEBUG libusbsio.hidapi.dev HID device<PHONE_NUMBER>8784 read[60]: 0x03001800a70000050000000037d69462a9e4cf518b0f959c541f856d0000000000000000000000000000000000000000000000000000000000000000
1179      DEBUG spsdk.mboot.interfaces.usb IN [60]: 03, 00, 18, 00, A7, 00, 00, 05, 00, 00, 00, 00, 37, D6, 94, 62, A9, E4, CF, 51, 8B, 0F, 95, 9C, 54, 1F, 85, 6D, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00
1180      DEBUG spsdk.mboot.mcuboot RX-PACKET: Tag=GetPropertyResponse, Status=Success, v0=0x6294D637, v1=0x51CFE4A9, v2=0x9C950F8B, v3=0x6D851F54
1180      DEBUG       root /dev/hidraw1: Nitrokey 3 Bootloader (LPC55) 37D69462A9E4CF518B0F959C541F856D
1180       INFO spsdk.mboot.mcuboot Closing: MCU HID GENERIC DEVICE (0x20A0, 0x42DD)
1180      DEBUG spsdk.mboot.interfaces.usb Close Interface
1182       INFO libusbsio.hidapi.dev HID device<PHONE_NUMBER>8784 closed
1183      DEBUG       root print: --------------------------------------------------------------------------------
1183      DEBUG       root print: Critical error occurred, exiting now
1183      DEBUG       root print: Unexpected? Is this a bug? Would you like to get support/help?
1183      DEBUG       root print: - You can report issues at: https://support.nitrokey.com/
1184      DEBUG       root print: - Writing an e-mail to<EMAIL_ADDRESS>is also possible
1184      DEBUG       root print: - Please attach the log: '/tmp/nitropy.log.f062w8_k' with any support/help request!
1184      DEBUG       root print: - Please check if you have udev rules installed: https://docs.nitrokey.com/nitrokey3/linux/firmware-update.html#troubleshooting

Unfortunately I think this killed my device, as nitropy nk3 test now also returns some MBoot error:
Command line tool to interact with Nitrokey devices 0.4.37
Found 1 Nitrokey 3 device(s):
- Nitrokey 3 Bootloader (LPC55) at /dev/hidraw1

Running tests for Nitrokey 3 Bootloader (LPC55) at /dev/hidraw1

[1/4]	uuid     	UUID query              	FAILURE  	MBoot: Connection issue -> Device not opened
[2/4]	version  	Firmware version query  	SKIPPED
[3/4]	status   	Device status           	SKIPPED
[4/4]	fido2    	FIDO2                   	SKIPPED

4 tests, 0 successful, 3 skipped, 1 failed

Summary: 1 device(s) tested, 0 successful, 1 failed

Critical error:
Test failed for 1 device(s)

--------------------------------------------------------------------------------
Critical error occurred, exiting now
Unexpected? Is this a bug? Would you like to get support/help?
- You can report issues at: https://support.nitrokey.com/
- Writing an e-mail to<EMAIL_ADDRESS>is also possible
- Please attach the log: '/tmp/nitropy.log.gb50qf1o' with any support/help request!
- Please check if you have udev rules installed: https://docs.nitrokey.com/nitrokey3/linux/firmware-update.html#troubleshooting

I do have the udev rules installed.

What is the output of nitropy nk3 list?

nitropy nk3 list returns the following:
Command line tool to interact with Nitrokey devices 0.4.37
:: 'Nitrokey 3' keys
/dev/hidraw0: Nitrokey 3 Bootloader (LPC55) 37D69462A9E4CF518B0F959C541F856D

It seems that restarting the update does something. (I was certain that I tried that but seems like I didn't)
Maybe it works this time.
I will post the results as soon as the update is finished

After the update finished the same error appeared.
However this time the update took over 8h which is twice as much is it took the first time.
Also the log contains debug messages for every few bytes written. (Which the first one didn't)
nitropy.log.r609dj3a.log

It looks like this is an issue with the bootloader provided by NXP.  Can you please try to reproduce the problem using the official NXP tools?  If it also happens with these tools, we’ll have to forward this issue to NXP.

Install the NXP tooling:$ pipx install spsdk

Make sure that the device is detected:$  spsdk blhost --usb 0x20a0:0x42dd get-property 1
Response status = 0 (0x0) Success.
Response word 1 =<PHONE_NUMBER> (0x4b030000)
Current Version = K3.0.0

Extract the raw firmware image:$ cd /tmp
$ nitropy nk3 fetch-update --version v1.4.0 .
$ sha256sum firmware-nk3-v1.4.0.zip
d77286a8f1544214ceed29beec0c9bc6a27da12a099f50c8a8c0af943e210c35  firmware-nk3-v1.4.0.zip
$ unzip firmware-nk3-v1.4.0.zip
$ sha256sum --check sha256sums

Perform the update:$ spsdk blhost --usb 0x20a0:0x42dd receive-sb-file firmware-nk3xn-lpc55-v1.4.0.sb2

Reboot the device:$ spsdk blhost --usb 0x20a0:0x42dd reset

Using the NXP Tools seems to have worked.
The update completed without any error.
nitropy nk3 version now returns v1.4.0
However nitropy nk3 test fails at Device status but I think this is a different issue,
So let me know if I should open a new Issue for this
Command line tool to interact with Nitrokey devices 0.4.36
Found 1 Nitrokey 3 device(s):
- Nitrokey 3 at /dev/hidraw1

Running tests for Nitrokey 3 at /dev/hidraw1

[1/4]	uuid     	UUID query              	SUCCESS  	37D69462A9E4CF518B0F959C541F856D
[2/4]	version  	Firmware version query  	SUCCESS  	v1.4.0
[3/4]	status   	Device status           	FAILURE  	init error: EXTERNAL_FLASH_ERROR (0x4)
Please press the touch button on the device ...
Please press the touch button on the device ...
[4/4]	fido2    	FIDO2                   	SUCCESS

4 tests, 3 successful, 0 skipped, 1 failed

Summary: 1 device(s) tested, 0 successful, 1 failed

Critical error:
Test failed for 1 device(s)

--------------------------------------------------------------------------------
Critical error occurred, exiting now
Unexpected? Is this a bug? Would you like to get support/help?
- You can report issues at: https://support.nitrokey.com/
- Writing an e-mail to<EMAIL_ADDRESS>is also possible
- Please attach the log: '/tmp/nitropy.log.jgqwe598' with any support/help request!
- Please check if you have udev rules installed: https://docs.nitrokey.com/nitrokey3/linux/firmware-update.html#troubleshooting

Hmm, interesting.  Can you try to repeat the update with nitropy to check if still fails?
Depending on the firmware version you were running before the update, the External Flash error could be caused by a migration.  In that case, it should disappear after a power cycle.  If you run nk3 test again, does it still show the error?

If you run nk3 test again, does it still show the error?

A power cycle did solve the issue.

Can you try to repeat the update with nitropy to check if still fails?

The update still fails using nitropy. After that even the UUID query of nitropy nk3 test fails

I also tried your instructions again and this time it also failed with:
ERROR:spsdk.mboot.interfaces.usb:Cannot read from HID device, error=0 (39405177ms since start, usb.py:243)
ERROR:spsdk.mboot.mcuboot:RX: No Response, Timeout Error ! (39405178ms since start, mcuboot.py:227)

Moreover I think there is a correlation between the time to finish the update an the success. This time it also took roughly
10h to finish and fail.
After some more testing I think the issue has something to do either with my (computer)hardware (not the nitrokey) or my
operating system, because on a different Computer with different Operating System 3/3 times the update
succeeded in a matter of seconds.
I'll try to investigate this further if it is a problem with my specific OS installation or if it is a hardware problem, for the sake of completeness and in case it may help others.

After some more testing I think the issue has something to do either with my (computer)hardware (not the nitrokey) or my
operating system, because on a different Computer with a different Operating System (MX Linux in this case) 3/3 times the update
succeeded in a matter of seconds.
If I discover something new that may help others I will add it to this issue.
For now I'll close this issue as it seems like the problem has nothing to do with pynitrokey.
Nontheless thank you very much for your help


[stable/jenkins] Error: UPGRADE FAILED: Deployment.apps "jenkins" is invalid

Request for help:
Version of Helm and Kubernetes:
Kubernetes: 1.12.7-gke.10
Helm:
Client: &version.Version{SemVer:"v2.12.0", GitCommit:"d325d2a9c179b33af1a024cdb5a4472b6288016a", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.12.0", GitCommit:"d325d2a9c179b33af1a024cdb5a4472b6288016a", GitTreeState:"clean"}
Which chart:
stable/jenkins 1.0.0
What happened:
I am trying to update from chart version 0.0.33 to 1.0.0. I have updated my values.yaml to reflect the new updates:
master:
  installPlugins:
    - kubernetes:1.14.5
    - workflow-job:2.31
    - workflow-aggregator:2.6
    - credentials-binding:1.17
    - git:3.9.3
    - github-branch-source:2.4.2
    - basic-branch-build-strategies:1.1.1
    - google-oauth-plugin:0.7
    - google-source-plugin:0.3
    - google-login:1.4
    - matrix-auth:2.3
    - slack:2.18
    - github:1.29.4
    - cobertura:1.13
    - timestamper:1.9
    - thinBackup:1.9
    - role-strategy:2.10
  javaOpts: "-Xms3500m -Xmx3500m"
  serviceType: ClusterIP
  podLabels:
    hostName: "redacted.com"
  ingress:
    enabled: true
    annotations:
      kubernetes.io/ingress.class: "nginx"
      kubernetes.io/ingress.allow-http: "false"
      kubernetes.io/tls-acme: "true"
      certmanager.k8s.io/cluster-issuer: letsencrypt
      #certmanager.k8s.io/cluster-issuer: letsencrypt-staging
      hostName: "redacted.com"
      #kubernetes.io/ingress.global-static-ip-name: static-ip-gcp can be used to assign static ip
    tls:
      - secretName: redacted
      #- secretName: redacted
        hosts:
            - "redacted.com"

persistence:
  size: 100Gi
networkPolicy:
  ApiVersion: networking.k8s.io/v1
#rbac:
#  create: true
#serviceAccount:
#        create: true
#        name: "jenkins"

When issuing the update command:
./helm upgrade jenkins stable/jenkins --version 1.0.0 --values values.yaml
I receive the following error:
Error: UPGRADE FAILED: Deployment.apps "jenkins" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app.kubernetes.io/instance":"jenkins", "app.kubernetes.io/component":"jenkins-master"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable
List of current deployments:
$ ./helm list -a
NAME            REVISION        UPDATED                         STATUS          CHART                   APP VERSION     NAMESPACE
cert-manager    2               Thu Apr 18 10:57:57 2019        DEPLOYED        cert-manager-v0.5.1     v0.5.1          kube-system
jenkins         50              Thu Apr 25 12:12:21 2019        DEPLOYED        jenkins-0.33.0          lts             default
nginx-ingress   1               Mon Feb 25 11:07:19 2019        DEPLOYED        nginx-ingress-1.3.1     0.22.0          default

What you expected to happen:
Successful upgrade
How to reproduce it (as minimally and precisely as possible):
./helm upgrade jenkins stable/jenkins --version 1.0.0 --values values.yaml
*using listed values.yaml
Anything else we need to know:
I ran into a strange issue when issuing an upgrade where Helm reported:
Error: UPGRADE FAILED: no x with the name "x" found
This was resolved by issuing:
  kubectl delete ServiceAccount jenkins
  kubectl delete RoleBinding jenkins-schedule-agents
  kubectl delete Role jenkins-schedule-agents
  kubectl delete Role jenkins-casc-reload
  kubectl delete RoleBinding jenkins-watch-configmaps

Does this help? https://github.com/helm/charts/pull/13273

Thanks - I'll give it a shot. I see that it requires the deletion of the deployments.
kubectl delete deploy jenkins

Did that help? Do you have suggestions for the upgrade instructions?

Thanks - Deleting the deployment and then running the upgrade resolved the issue.
 kubectl delete deploy jenkins
./helm upgrade jenkins stable/jenkins --version 1.1.5 --values values.yaml

I ran into a problem with host not being set properly within my jenkins ingress configuration. The following gets deployed:
What gets deployed:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    certmanager.k8s.io/cluster-issuer: letsencrypt
    hostName: redacted
    kubernetes.io/ingress.allow-http: "false"
    kubernetes.io/ingress.class: nginx
    kubernetes.io/tls-acme: "true"
  creationTimestamp: 2019-04-18T16:13:54Z
  generation: 8
  labels:
    app.kubernetes.io/component: jenkins-master
    app.kubernetes.io/instance: jenkins
    app.kubernetes.io/managed-by: Tiller
    app.kubernetes.io/name: jenkins
    helm.sh/chart: jenkins-1.1.5
  name: jenkins
  namespace: default
  resourceVersion: "15338028"
  selfLink: /apis/extensions/v1beta1/namespaces/default/ingresses/jenkins
  uid: f637f8cf-61f4-11e9-bf9e-42010a80000c
spec:
  **rules:
  - http:**
      paths:
      - backend:
          serviceName: jenkins
          servicePort: 8080
  tls:
  - hosts:
    - redacted.com
    secretName: redacted
status:
  loadBalancer:
    ingress:
    - ip: redacted

The fix:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    certmanager.k8s.io/cluster-issuer: letsencrypt
    hostName: redacted.com
    kubernetes.io/ingress.allow-http: "false"
    kubernetes.io/ingress.class: nginx
    kubernetes.io/tls-acme: "true"
  creationTimestamp: 2019-04-18T16:13:54Z
  generation: 8
  labels:
    app.kubernetes.io/component: jenkins-master
    app.kubernetes.io/instance: jenkins
    app.kubernetes.io/managed-by: Tiller
    app.kubernetes.io/name: jenkins
    helm.sh/chart: jenkins-1.1.5
  name: jenkins
  namespace: default
  resourceVersion: "15338028"
  selfLink: /apis/extensions/v1beta1/namespaces/default/ingresses/jenkins
  uid: f637f8cf-61f4-11e9-bf9e-42010a80000c
spec:
  rules:
  **- host: redacted.com
    http:**
      paths:
      - backend:
          serviceName: jenkins
          servicePort: 8080
  tls:
  - hosts:
    - redacted.com
    secretName: redacted
status:
  loadBalancer:
    ingress:
    - ip: redacted

Manually modifying the ingress with -host: redacted.com under spec > rules allows the cert to properly load
The previous version of the chart 0.0.33 creates the following ingress:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    certmanager.k8s.io/cluster-issuer: letsencrypt
    kubernetes.io/ingress.allow-http: "false"
    kubernetes.io/ingress.class: nginx
    kubernetes.io/tls-acme: "true"
  creationTimestamp: 2019-02-25T15:07:48Z
  generation: 1
  name: jenkins
  namespace: default
  resourceVersion: "1007"
  selfLink: /apis/extensions/v1beta1/namespaces/default/ingresses/jenkins
  uid: 1d228f24-390f-11e9-bf2a-42010a800074
spec:
  rules:
  **- host: redacted.com
    http:**
      paths:
      - backend:
          serviceName: jenkins
          servicePort: 8080
  tls:
  - hosts:
    - redacted.com
    secretName: redacted
status:
  loadBalancer:
    ingress:
    - ip: redacted

The values.yaml I am using is posted above.

Thanks for the update. Do you want to create a PR to fix the chart?

Looks like there are two parameters where the hostname can be configured:
master.hostName
master.ingress.hostName => has been introduces as part of #12946
If I look at the template only master.ingress.hostName is used now. NOTES.txt still references the wrong value. I just created #13318 to correct the notes.

Thank you for the clarification. I'll correct my values and this should fix it.

I have tried setting my hostname redacted.com on both master.hostName or master.ingress.hostName. Neither populates the rules field within the ingress rules section. My workaround is to manually modify the ingress so our systems are operational. Just bringing this back up to the community in case anyone else is having a problem.

@marcingluc-clearobject I tried to reproduce your issue using the following steps:
git clone https://github.com/helm/charts.git
cd charts/stable/jenkins
helm template . -x templates/jenkins-master-ingress.yaml --set master.ingress.enabled=true --set master.ingress.hostName=redacted.com
---
# Source: jenkins/templates/jenkins-master-ingress.yaml

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  labels:
    "app.kubernetes.io/name": 'jenkins'
    "helm.sh/chart": "jenkins-1.1.10"
    "app.kubernetes.io/managed-by": "Tiller"
    "app.kubernetes.io/instance": "release-name"
    "app.kubernetes.io/component": "jenkins-master"
  name: release-name-jenkins
spec:
  rules:
  - http:
      paths:
      - backend:
          serviceName: release-name-jenkins
          servicePort: 8080
    host: "redacted.com"

It contains host: "redacted.com", so what's wrong with the generated ingress?

Looks like it was an indentation error in my values.yaml. Closing issue.


[feature idea] Double click for selecting word, triple click for selecting line

It may be useful to invoke specific functions by double or triple-clicking. The inspiration for this idea came from the following issue.
https://github.com/vhakulinen/gnvim/issues/174

It is possible to do this in the neovim settings


Missing semicolon warning on closures with asi: true

Even with asi: true the following will still complain about a missing semicolon, however the second case will not, with asi: true it shouldn't complain about this missing semicolon as per #1906
'use strict'

(function() {
})()

'use strict'

function() {
}

'use strict'

(function() {
})()

is not valid javascript, which is why we emit an error here.

you there? I don't think this is actionable, because we shouldn't let this obvious error sneak through unscathed by the ASI option :p I hope this makes sense

@caitp I'm a bit confused, how is this not valid javascript vs the second case?

@MikaAK well, it is "valid javascript", but it will always produce a runtime error as strings do not have a [[Call]] internal method (but the IsCallable() algorithm always returns false for primitives anyways).
Anyways, automatic semicolon insertion does not apply here, because the string itself does not terminate the ExpressionStatement.
So,
"use strict" // no problem, semi-colon is inserted :D
"some other string"

"use strict" // Oops, we're trying to "call" the string, because the ExpressionStatement
             // looks like a function call!
(...);


mvirga01:patch-1 Palo Alto Firewall

Purpose of this pull request
proposed update for ingesting Palo Alto v9 or v10 via installed collector and UDP/TCP log forwarding
Select the type of change:

[ ] Minor Changes - Typos, formatting, slight revisions
[x] Update Content - Revisions and updating sections
[ ] New Content - New features, sections, pages, tutorials
[ ] Site and Tools - Updates, maintenance, and new packages for the site, Gatsby, React, etc

@mvirga01 applied change in new PR, https://github.com/SumoLogic/sumologic-documentation/pull/671


MagmaCoreRemoteSparqlDatabase Transaction Error

The MagmaCoreRemoteSparqlDatabase uses an RDFConnection which doesn't support READ_PROMOTE transactions that are created by default, so the begin method needs to start a WRITE transaction instead.
An alternative is to use beginReadOnly and beginWritemethods in place ofbeginbut this will mean changing theMagmaCoreDatabaseinterface, all of its implementations, and all the client code that callsbegin`.

I have a solution implemented locally and will create a PR when the prerequisite PRs are merged.


Added backers and sponsors from Open Collective to README.md

Hi, I'm making updates for Open Collective. Either you or a supporter signed this repo up for Open Collective. This pull request adds backers and sponsors from your Open Collective https://opencollective.com/tenancy ❤
It adds two badges at the top to show the latest number of backers and sponsors. It also adds placeholders so that the avatar/logo of new backers/sponsors can automatically be shown without having to update your README.md. [more info].
You can also add a "Donate" button to your website and automatically show your backers and sponsors there with our widgets. Have a look here: https://opencollective.com/widgets
P.S: As with any pull request, feel free to comment or suggest changes. The only thing "required" are the placeholders on the README because we believe it's important to acknowledge the people in your community that are contributing (financially or with code!).
Thank you for your great contribution to the open source community. You are awesome! 🙌
And welcome to the open collective community! 😊
Come chat with us in the #opensource channel on https://slack.opencollective.com - great place to ask questions and share best practices with other open source sustainers!

It should have been pointing to https://github.com/tenancy/tenancy; it's the new successor. Also it's pretty odd seeing myself as back, is that intentional @monkeywithacupcake ?

Some of the Open Collectives show the owner as a backer and others do not. I do not look at the back-end when connecting repos. I imagine that you can remove yourself as a backer and you will no longer show up. However, I think it also helps future backers when they see they are not the first. Your choice.


append slice result output is wrong

when i make([][]int,0) and append slice,the output is not right.
examples

examples

import "fmt"

func threeSum(nums []int) [][]int {
	result := make([][]int,0)
	for i := 0; i < len(nums); i++ {
		sum := nums[i]
		tmp := make([]int, 0)
		tmp = append(tmp, nums[i])
		for j := i + 1; j < len(nums); j++ {
			sum += nums[j]
			tmp = append(tmp, nums[j])
			if sum == 0 {
				result = append(result,tmp)

				fmt.Println("data:", tmp)
				fmt.Println("result:", result)

				sum = nums[i]
				tmp = tmp[:1]
			}
		}
	}
	return result
}
func main() {
	fmt.Println("output result:",threeSum([]int{-1, 0, 1, 2, -1, -4}))
}

output,the result and output result is not same

data: [-1 0 1]
result: [[-1 0 1]]
data: [-1 2 -1]
result: [[-1 2 -1] [-1 2 -1]]
output result: [[-1 -4 -1] [-1 -4 -1]]

You are appending tmp to result, but then you are changing tmp.  Slices in Go refer to an underlying array, and you are changing that array.  See https://blog.golang.org/slices.


Helm deployment: Clickhouse cluster

Hello,
The helm support for clickhouse operator is great. I would like to know how to deploy CHI with helm as well.
As we are using ArgoCD, it would be more convenient to use helm chart to deploy clickhouse cluster along the operator and i can't find any valuable documentation regarding this topic.
Many thanks

look to additionalResources in values.yaml
https://github.com/Altinity/clickhouse-operator/blob/master/deploy/helm/clickhouse-operator/values.yaml#L635-L661

@Slach
Indeed additionalResources can do the job, but it's not very convenient.
I think it would be much better to deploy a CH cluster from a separated release than the operator itself, to be allowed to create a cluster without touching the operator values.
So having a dedicated chart for the cluster deployment would be super helpful.
As an example, minio operator has two distinct charts, one for the operator, and one for the tenants.

@clemcvlcs we are welcome to PR


[Feature request] Allow download

Add button to download original, allow downloading whole album as zip.

Yeah, I think this would be good as a configurable option for sure. I know my use-case I don't want to make it THAT easy but I can see the desire for some cases.


Improve debugging support on Android by integrating stetho

The stetho library allows you to peek inside several aspects of your app from the chrome developers.
http://facebook.github.io/stetho/
Currently Android only.

This has been completed. We will begin to address specific features / bugs with new issues.


Add support for more types of automatic breadcrumbs

Over on #162 @jacobmarshall mentioned some cases where our old lastEvent tracking would provide information we don't currently collect automatically with breadcrumbs. I wanted to open an issue to collect feedback on other types of breadcrumbs that might be worth adding, to make sure the new breadcrumb feature covers all the cases that last event did. Please suggest more. Here are some I pulled from Jacob's comments:

[ ] message event from an iframe
[ ] XHR/fetch tracking

onreadystatechange
load/error

[ ] Keyboard events

we'll want to do something to prevent logging user input, e.g. roll these up with a count and only show the last keypress value

message events from Web Workers may also be pretty cool to catch with breadcrumbs. Simply because for us, as an encrypted product, we use Web Workers everywhere, and knowing whether a message came through correctly would be a huge bonus. However I see definitely understand if this was pushed away from being part of the core library, as it's probably not as critical for most apps.
Regardless, I see a huge benefit in the originally listed events alone. I am happy to put together a PR, if you'd like? Or if you'd just like to collect feedback for now, I'll sit on this for the time being.
As a side note, I'm not sure how much pressure there is to keep the library tiny anymore. I know when Conrad was working with you guys there was definitely an effort to keep the library as TINY as possible, which is great, but the breadcrumb code has bulked it up a bit. I wonder whether or not it's worth making some alternatives for people who would like to make the most of the amazing  breadcrumb functionality, and for people who aren't interested? Such as providing bundles which provide bugsnag-js core & detailed breadcrumbs, and maybe even commonjs modules for people who wish to decide which type of breadcrumbs they're only interested in. But maybe this all ends up making it harder for people to jump straight into having the full Bugsnag experience? (Thinking aloud again)
Anyway, holler if you'd like to see a PR 💃

We'd welcome a PR for web workers. We currently have the process type set aside for these. We are already using it in the mobile/native libraries, but haven't started anything for JS.  You'd know better what kind of things you'd like to track.
The size goal is to keep the library under 4.5 KB minified

Sweet, cheers, web workers it is then 🏹

Need to decide whether we want to add this to the current version vs. the next major.


Add Hexagon Force V1.55 by i forgot their name

Song Name
Hexagon Force V1.55
Artist Name
i forgot their name
Source
Youtube
Youtube Link
6w68cuZn9g8
Direct File Link
No response
Song ID
15
Start Offset [ms]
0
Extra
Created from Auto Nong.
Created for level: 1629780
Downloaded: Yes

Pls accept


Running make (against a Makefile) inside the container now fails [shopware5]

Is there an existing issue for this?

[X] I have searched the existing issues

Output of ddev debug test
Expand `ddev debug test` diagnostic information
Running bash [-c /var/folders/46/j17c6b092cg91c41ms80v8hw0000gn/T/test_ddev.sh]
======= Existing project config =========
These config files were loaded for project shopware5-sellxed-5.7.18: [/Users/alberto/sellxed/shopware/wsm-shopware5-sellxed-5.7.18/.ddev/config.yaml]
name: shopware5-sellxed-5.7.18
type: php
docroot: .
php_version: 7.4
webserver_type: nginx-fpm
webimage: ddev/ddev-webserver:v1.22.0
additional_hostnames: []
additional_fqdns: []
database: {mariadb 10.4}
performance_mode: mutagen
mailhog_port: 8025
mailhog_https_port: 8026
webimage_extra_packages: [build-essential]
project_tld: ddev.site
use_dns_when_possible: true
composer_version: 2
nodejs_version: 18
default_container_timeout: 120
======= Creating dummy project named  tryddevproject-18722 in ../tryddevproject-18722 =========
OS Information: Darwin gviu.local 21.6.0 Darwin Kernel Version 21.6.0: Mon Dec 19 20:43:09 PST 2022; root:xnu-8020.240.18~2/RELEASE_ARM64_T6000 arm64
ProductName:	macOS
ProductVersion:	12.6.3
BuildVersion:	21G419
User information: uid=501(alberto) gid=20(staff) groups=20(staff),12(everyone),61(localaccounts),79(_appserverusr),80(admin),81(_appserveradm),98(_lpadmin),204(_developer),701(com.apple.sharepoint.group.1),33(_appstore),100(_lpoperator),250(_analyticsusers),395(com.apple.access_ftp),398(com.apple.access_screensharing),399(com.apple.access_ssh),400(com.apple.access_remote_ae)
DDEV version:  ITEM             VALUE
 DDEV version     v1.22.0
 architecture     arm64
 db               ddev/ddev-dbserver-mariadb-10.4:v1.22.0
 ddev-ssh-agent   ddev/ddev-ssh-agent:v1.22.0
 docker           24.0.2
 docker-compose   v2.20.0
 docker-platform  docker-desktop
 mutagen          0.17.2
 os               darwin
 router           traefik:v2.10
 web              ddev/ddev-webserver:v1.22.0
PROXY settings: HTTP_PROXY='' HTTPS_PROXY='' http_proxy='' NO_PROXY=''
======= DDEV global info =========
Global configuration:
instrumentation-opt-in=true
omit-containers=[]
performance-mode=mutagen
router-bind-all-interfaces=false
internet-detection-timeout-ms=3000
disable-http2=false
use-letsencrypt=false
letsencrypt-email=
table-style=default
simple-formatting=false
auto-restart-containers=false
use-hardened-images=false
fail-on-hook-fail=false
required-docker-compose-version=v2.20.0
use-docker-compose-from-path=false
project-tld=
xdebug-ide-location=
no-bind-mounts=false
router=traefik
wsl2-no-windows-hosts-mgt=false
router-http-port=80
router-https-port=443

======= DOCKER info =========
docker location: lrwxr-xr-x  1 root  wheel  54 Jul 12  2022 /usr/local/bin/docker -> /Applications/Docker.app/Contents/Resources/bin/docker
Docker Desktop Version: Docker Desktop for Mac 4.21.1 build 114176
docker version:
Client:
 Cloud integration: v1.0.35
 Version:           24.0.2
 API version:       1.43
 Go version:        go1.20.4
 Git commit:        cb74dfc
 Built:             Thu May 25 21:51:16 2023
 OS/Arch:           darwin/arm64
 Context:           default

Server: Docker Desktop 4.21.1 (114176)
 Engine:
  Version:          24.0.2
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.4
  Git commit:       659604f
  Built:            Thu May 25 21:50:59 2023
  OS/Arch:          linux/arm64
  Experimental:     true
 containerd:
  Version:          1.6.21
  GitCommit:        3dce8eb055cbb6872793272b4f20ed16117344f8
 runc:
  Version:          1.1.7
  GitCommit:        v1.1.7-0-g860f061
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0
DOCKER_DEFAULT_PLATFORM=notset
======= Mutagen Info =========
Mutagen is installed in ddev, version=0.17.2
--------------------------------------------------------------------------------
Name: prestashop-portal
Identifier: sync_Gc8Y92lBzQPMto4vhbE2M6O7jVXMFsq5usm6W0Ha0WD
Labels:
	com.ddev.config-hash: 0311359b0f6eedb428c2679de60b5d021c2d717b
	com.ddev.volume-signature: Users-alberto-docker-run-1690967542
Configuration:
	Synchronization mode: Two Way Resolved
	Hashing algorithm: Default (SHA-1)
	Maximum allowed entry count: Default (2⁶⁴−1)
	Maximum staging file size: Default (18 EB)
	Symbolic link mode: POSIX Raw
	Ignore VCS mode: Default (Propagate)
	Ignores:
		/.git
		/.tarballs
		/.ddev/db_snapshots
		/.ddev/.importdb*
		.DS_Store
		.idea
	Permissions mode: Default (Portable)
Alpha:
	URL: /Users/alberto/integrations/prestashop/webshop
	Configuration:
		Watch mode: Default (Portable)
		Watch polling interval: Default (10 seconds)
		Probe mode: Default (Probe)
		Scan mode: Default (Accelerated)
		Stage mode: Neighboring
		File mode: Default (0600)
		Directory mode: Default (0700)
		Default file/directory owner: Default
		Default file/directory group: Default
	Connected: Yes
	Synchronizable contents:
		27143 directories
		208710 files (1.8 GB)
		315 symbolic links
Beta:
	URL: docker://ddev-prestashop-portal-web/var/www/html
		DOCKER_HOST=unix:///Users/alberto/.docker/run/docker.sock
	Configuration:
		Watch mode: Default (Portable)
		Watch polling interval: Default (10 seconds)
		Probe mode: Default (Probe)
		Scan mode: Default (Accelerated)
		Stage mode: Neighboring
		File mode: Default (0600)
		Directory mode: Default (0700)
		Default file/directory owner: Default
		Default file/directory group: Default
		Compression: Default (DEFLATE)
	Connected: Yes
	Synchronizable contents:
		27143 directories
		208710 files (1.8 GB)
		315 symbolic links
Status: Watching for changes
--------------------------------------------------------------------------------
Name: shopware5-sellxed-5718
Identifier: sync_2INl9dVqgZ8tkg7dnMQnNfEW7RbT3f7X4Y0rtnhAQDH
Labels:
	com.ddev.config-hash: 0311359b0f6eedb428c2679de60b5d021c2d717b
	com.ddev.volume-signature: Users-alberto-docker-run-1691069534
Configuration:
	Synchronization mode: Two Way Resolved
	Hashing algorithm: Default (SHA-1)
	Maximum allowed entry count: Default (2⁶⁴−1)
	Maximum staging file size: Default (18 EB)
	Symbolic link mode: POSIX Raw
	Ignore VCS mode: Default (Propagate)
	Ignores:
		/.git
		/.tarballs
		/.ddev/db_snapshots
		/.ddev/.importdb*
		.DS_Store
		.idea
	Permissions mode: Default (Portable)
Alpha:
	URL: /Users/alberto/sellxed/shopware/wsm-shopware5-sellxed-5.7.18
	Configuration:
		Watch mode: Default (Portable)
		Watch polling interval: Default (10 seconds)
		Probe mode: Default (Probe)
		Scan mode: Default (Accelerated)
		Stage mode: Neighboring
		File mode: Default (0600)
		Directory mode: Default (0700)
		Default file/directory owner: Default
		Default file/directory group: Default
	Connected: Yes
	Synchronizable contents:
		2000 directories
		9670 files (97 MB)
		0 symbolic links
Beta:
	URL: docker://ddev-shopware5-sellxed-5.7.18-web/var/www/html
		DOCKER_HOST=unix:///Users/alberto/.docker/run/docker.sock
	Configuration:
		Watch mode: Default (Portable)
		Watch polling interval: Default (10 seconds)
		Probe mode: Default (Probe)
		Scan mode: Default (Accelerated)
		Stage mode: Neighboring
		File mode: Default (0600)
		Directory mode: Default (0700)
		Default file/directory owner: Default
		Default file/directory group: Default
		Compression: Default (DEFLATE)
	Connected: Yes
	Synchronizable contents:
		2000 directories
		9670 files (97 MB)
		0 symbolic links
Status: Watching for changes
--------------------------------------------------------------------------------
======= Docker Info =========
Docker platform: docker-desktop
Using docker context: default (unix:///Users/alberto/.docker/run/docker.sock)
docker-compose: v2.20.0
Using DOCKER_HOST=unix:///Users/alberto/.docker/run/docker.sock
Docker version: 24.0.2
Able to run simple container that mounts a volume.
Able to use internet inside container.
Docker disk space:
Filesystem                Size      Used Available Use% Mounted on
overlay                 157.2G     69.0G     80.1G  46% /

PhpMyAdmin (`dba`) is no longer part of DDEV core, please edit your `omit_containers` configuration to remove it
PhpMyAdmin (`dba`) is no longer part of DDEV core, please edit your `omit_containers` configuration to remove it
PhpMyAdmin (`dba`) is no longer part of DDEV core, please edit your `omit_containers` configuration to remove it
PhpMyAdmin (`dba`) is no longer part of DDEV core, please edit your `omit_containers` configuration to remove it
PhpMyAdmin (`dba`) is no longer part of DDEV core, please edit your `omit_containers` configuration to remove it
PhpMyAdmin (`dba`) is no longer part of DDEV core, please edit your `omit_containers` configuration to remove it
PhpMyAdmin (`dba`) is no longer part of DDEV core, please edit your `omit_containers` configuration to remove it
PhpMyAdmin (`dba`) is no longer part of DDEV core, please edit your `omit_containers` configuration to remove it
PhpMyAdmin (`dba`) is no longer part of DDEV core, please edit your `omit_containers` configuration to remove it
PhpMyAdmin (`dba`) is no longer part of DDEV core, please edit your `omit_containers` configuration to remove it
PhpMyAdmin (`dba`) is no longer part of DDEV core, please edit your `omit_containers` configuration to remove it
PhpMyAdmin (`dba`) is no longer part of DDEV core, please edit your `omit_containers` configuration to remove it
PhpMyAdmin (`dba`) is no longer part of DDEV core, please edit your `omit_containers` configuration to remove it
PhpMyAdmin (`dba`) is no longer part of DDEV core, please edit your `omit_containers` configuration to remove it
PhpMyAdmin (`dba`) is no longer part of DDEV core, please edit your `omit_containers` configuration to remove it
PhpMyAdmin (`dba`) is no longer part of DDEV core, please edit your `omit_containers` configuration to remove it
PhpMyAdmin (`dba`) is no longer part of DDEV core, please edit your `omit_containers` configuration to remove it
PhpMyAdmin (`dba`) is no longer part of DDEV core, please edit your `omit_containers` configuration to remove it
PhpMyAdmin (`dba`) is no longer part of DDEV core, please edit your `omit_containers` configuration to remove it
PhpMyAdmin (`dba`) is no longer part of DDEV core, please edit your `omit_containers` configuration to remove it
PhpMyAdmin (`dba`) is no longer part of DDEV core, please edit your `omit_containers` configuration to remove it
PhpMyAdmin (`dba`) is no longer part of DDEV core, please edit your `omit_containers` configuration to remove it
 Container ddev-prestashop-portal-db  Stopped
 Container ddev-prestashop-portal-web  Stopped
 Container ddev-prestashop-portal-web  Stopped
 Container ddev-prestashop-portal-db  Stopped
 Container ddev-prestashop-portal-db  Removed
 Container ddev-prestashop-portal-web  Removed
 Network ddev-prestashop-portal_default  Removed
Project prestashop-portal has been stopped.
 Container ddev-shopware5-sellxed-5.7.18-db  Stopped
 Container ddev-shopware5-sellxed-5.7.18-web  Stopped
 Container ddev-shopware5-sellxed-5.7.18-web  Stopped
 Container ddev-shopware5-sellxed-5.7.18-db  Stopped
 Container ddev-shopware5-sellxed-5.7.18-db  Removed
 Container ddev-shopware5-sellxed-5.7.18-web  Removed
 Network ddev-shopware5-sellxed-5718_default  Removed
Project shopware5-sellxed-5.7.18 has been stopped.
Stopped mutagen daemon
The ddev-ssh-agent container has been removed. When you start it again you will have to use 'ddev auth ssh' to provide key authentication again.
Existing docker containers:
CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES
Network ddev_default created
Starting tryddevproject-18722...
 Container ddev-ssh-agent  Created
 Container ddev-ssh-agent  Started
ssh-agent container is running: If you want to add authentication to the ssh-agent container, run 'ddev auth ssh' to enable your keys.
 Network ddev-tryddevproject-18722_default  Created
 Container ddev-tryddevproject-18722-web  Created
 Container ddev-tryddevproject-18722-db  Created
 Container ddev-tryddevproject-18722-db  Started
 Container ddev-tryddevproject-18722-web  Started
You have Mutagen enabled and your 'php' project type doesn't have `upload_dirs` set.
For faster startup and less disk usage, set upload_dirs to where your user-generated files are stored.
If this is intended you can disable this warning with `ddev config --disable-upload-dirs-warning`.
Starting mutagen sync process... This can take some time.
Mutagen sync flush completed in 5s.
For details on sync status 'ddev mutagen st tryddevproject-18722 -l'
 Container ddev-router  Created
 Container ddev-router  Started
Successfully started tryddevproject-18722
Project can be reached at https://tryddevproject-18722.ddev.site https://<IP_ADDRESS>:61287
======== Curl of site from inside container:
HTTP/1.1 200 OK
Server: nginx
Date: Thu, 03 Aug 2023 13:53:11 GMT
Content-Type: text/html; charset=UTF-8
Connection: keep-alive
Vary: Accept-Encoding

======== curl -I of http://tryddevproject-18722.ddev.site from outside:
HTTP/1.1 200 OK
Content-Type: text/html; charset=UTF-8
Date: Thu, 03 Aug 2023 13:53:11 GMT
Server: nginx
Vary: Accept-Encoding

======== full curl of http://tryddevproject-18722.ddev.site from outside:
Success accessing database... db via TCP/IP
ddev is working. You will want to delete this project with 'ddev delete -Oy tryddevproject-18722'
======== Project ownership on host:
drwxr-xr-x  4 alberto  staff  128 Aug  3 15:51 ../tryddevproject-18722
======== Project ownership in container:
drwxr-xr-x 4 alberto dialout 4096 Aug  3 13:52 /var/www/html
======== In-container filesystem:
Filesystem     Type 1K-blocks     Used Available Use% Mounted on
/dev/vda1      ext4 164815332 73697032  82713372  48% /var/www
======== curl again of tryddevproject-18722 from host:
Success accessing database... db via TCP/IP
ddev is working. You will want to delete this project with 'ddev delete -Oy tryddevproject-18722'
Thanks for running the diagnostic. It was successful.
Please provide the output of this script in a new gist at gist.github.com
Running ddev launch in 5 seconds
If you're brave and you have jq you can delete all tryddevproject instances with this one-liner:
    ddev delete -Oy $(ddev list -j |jq -r .raw[].name | grep tryddevproject)
In the future ddev debug test will also provide this option.

Please delete this project after debugging with 'ddev delete -Oy tryddevproject-18722'

Expected Behavior
When installing Shopware5, following their own instructions, I expect that the shop installation finish normally.
This process has been working perfectly until DDEV 1.21 included, but broken in DDEV 1.22
Actual Behavior
Shopware5 installs in the shop by running a Makefile. In particular, the command
make init
needs to run inside the web container.
The first target that runs make init is the clean-make-config which tries to delete previous make attempts. As shown in their Makefile, it just tries to delete files:
clean-make-config:
	rm -f .make.config.*
	rm -f .make.install.*

When running make init, this is their output:
rm -f .make.config.*
rm -f .make.install.*
make: *** [Makefile:112: clean-make-config] Error 1

As far as I have checked, this is related to a permissions issue. If you run sudo make init it works fine, but it then creates files with the root user which is not desirable.
Steps To Reproduce
This issue is happening in shopware5. It happens with previous versions as well, not only the latest 5.7.18.
Clone the repository locally with:
git clone git@github.com:shopware/shopware.git webshop
cd webshop
Create the DDEV project with this command:
ddev config --mutagen-enabled --project-type=php --php-version=7.4 --docroot=. --project-name=whopware5 --webimage-extra-packages=build-essential
ddev start
Create a .env file with this content:
DB_USER=db
DB_PASSWORD=db
DB_HOST=db
DB_NAME=db
DB_PORT=3306
SW_HOST=shopware5.ddev.site
SW_BASE_PATH=/
SELENIUM_HOST=selenium
ELASTICSEARCH_HOST=elasticsearch
Then run the make init:
ddev exec make init
Anything else?
I have the installation for shopware5 in a script and it has been working nicely until the upgrade to DDEV 1.22.
After investigating a bit, I noticed that the version of GNU Make in DDEV 1.21 is 3.81, and the version of GNU Make in DDEV 1.22 is 4.3
I have not noticed any other important difference between both versions.

Hi @agviu -
ddev-webserver has not included make in any version.
However, I see you're using webimage-extra-packages=build-essential, so that would include make.
What you'll want to do is to fiddle with the Makefile to figure out what's going on. rm -f should never return an error on any bash/linux version.
You definitely don't want to be running sudo make init for the reasons you say.
I think you'll want to take a look at what directory you're executing in, then manually try the commands that your Makefile is trying to execute. See what happens. Check the output. You'll figure out what's going on.
Make hasn't really changed in decades, so I doubt that's the issue. BTW, you don't have to add build-essential if you only need make, just webimage_extra_packages: [make] will do. And make is 4.3  after adding make in both drud/ddev-webserver:v1.21.5 and also in ddev/ddev-webserver:v1.22.0. That's what I'd expect,  because they're both based on Debian 11 Bullseye.

Hi @rfay ,
thanks for your fast reply.
I will try those commands manually as suggested. The thing that made me think that this was a problem with DDEV was/is that this error happens from a specific version of DDEV (1.22), and it doesn't with previous versions.
For me, as a workaround, I can run these commands with DDEV 1.21 and install the software stack. I wonder if there's something else that has changed, maybe not related to make but to the permissions in the volume or something with mutagen.

Let me know what you discover, thanks! Manually understanding what's going on and what's failing is the path to the answer.


AW2E Landfall Marine Edits

Changes / Comments
-Added hunger checkboxes below food clock
-Added available missions
Roll20 Requests
Comments are very helpful for reviewing the code changes. Please answer the relevant questions below in your comment.

[Y] Does the pull request title have the sheet name(s)? Include each sheet name.
[N] Is this a bug fix?
[Y] Does this add functional enhancements (new features or extending existing features) ?
[N] Does this add or change functional aesthetics (such as layout or color scheme) ?
[N] If changing or removing attributes, what steps have you taken, if any, to preserve player data ?
[N] If this is a new sheet, did you follow Building Character Sheets standards ?

If you do not know English. Please leave a comment in your native language.

@TheSirLancelot It looks like this pull request has merge conflicts. You'll need to resolve those before I can push the update live.

Greetings,
Its been 21 days since the PR was entered and the merge conflicts remain unresolved. I'm going to close the PR. Please submit it again when you're ready to resolve the conflicts.
Thanks,
Cassie


ZoneWidget right-click menu context and widget will be covered by terminal

VS Code Version:  1.75.1
OS Version:  Windows_NT x64

zone-widget z-index is 10
zone-widget and xterm are in different stacking context

This is to avoid creating a z-index 30+canvas context at the beginning by creating an integrated terminal with a context of 0 in advance, so that the terminal will not cover the rest of the code components, unless it is - 1, and the internal operation of terminal is still normal.
Used for several days.


Remove parallel tests from travis

The parallel tests are problematic in travis because of the limited resources in the travis VM. For example, starting 5 VMs in parallel is extremely slow and breaks the build often. We should leave the serial tests which verify only the functionality and leave out the parallel ones for testing on-premises with better hardware.

travis is gone


Revise OOB usage

According to https://github.com/ForNeVeR/kaiwa/issues/133#issuecomment-227101725 Kaiwa sometimes includes ordinary URLs as OOB data. At the same time, according to XEP-0066 (as a possible interpretation), we should use that mechanism only for some file-based URLs (such as links to images or maybe videos?).
I am opening this ticket for discussion of this question. Our current implementation is known for creating minor problems for some XMPP clients (e.g. TKabber). Maybe we would like to strip this functionality completely? What's the use of it?
This statement from XEP-0066 introduction may be of an additional interest:

[...] Although these mechanisms were originally used for out-of-band (OOB) data transfer, they are also used more generally to exchange or communicate URIs.

ping :)


AMBARI-25450. [ubuntu16] HDP install failed for upgrade

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/Ambari-Github-PullRequest-Builder/5625/
Test FAILed.
Test FAILured.


Update train.py

Made code more readable. Could be considered to use @click instead of parser - neater and can be used in front of functions.
Inputs is unused in io_test, as it wasnt used before either, this must be a bug.

Thank you very much for contributing @NIKKE1! It reads much clearer now:)

Inputs is unused in io_test, as it wasnt used before either, this must be a bug.

Indeed this line inputs = [X[k] for k in data_config.input_names] can be removed I think.

Could be considered to use @click instead of parser - neater and can be used in front of functions.

That could be a good improvement -- I have never used click before but it looks pretty nice from a quick glance. Would you be interested to make a PR?


Update py-evm version to use for Building an app that uses Py-EVM in py-evm docs

py-evm Version: 0.3.0a20
OS: Ubuntu 20.04
Python Version (python --version): Python 3.8.10

What is wrong?
When going through this guide https://py-evm.readthedocs.io/en/latest/guides/building_an_app_that_uses_pyevm.html running py main.py produces the error message:
Traceback (most recent call last):
File "app/main.py", line 36, in <module>
chain = MainnetChain.from_genesis(AtomicDB(), GENESIS_PARAMS, GENESIS_STATE)
File "/home/sabnock/.local/lib/python3.8/site-packages/eth/chains/base.py", line 251, in from_genesis
genesis_header = BlockHeader(**genesis_params)
TypeError: __init__() missing 2 required positional arguments: 'block_number' and 'gas_limit'

How can it be fixed
I was able to independently rectify this by changing the evm version in install_requires {} to a more recent version. I get the correct output using py-evm 0.5.0a0. The documentation should be updated to suggest a more recent version of py-evm.

@Sabnock01 Thanks for this! We will get an update up asap.


Update free-courses-en.md - add Design pattern resource English

What does this PR do?
Add resource(s) | This PR adds a resource for Design Pattern: Java in English.
For resources
Description
This resource covers Design Patterns with Creational, Structural & Behavioural Patterns in detail.
Why is this valuable (or not)?
This is completely free and very good resource for design patterns.
How do we know it's really free?
For book lists, is it a book? For course lists, is it a course? etc.
Book list
Checklist:

[x] Read our contributing guidelines.
[x] Search for duplicates.
[x] Include author(s) and platform where appropriate.
[x] Put lists in alphabetical order, correct spacing.
[x] Add needed indications (PDF, access notes, under construction).
[x] Used an informative name for this pull request.

Follow-up
Please add Hacktober fest label.

Check the status of GitHub Actions and resolve any reported warnings!

@eshellman

@eshellman Pls review & approve! Thanks in anticipation

the book on this website is not free


HIVE-27043: Upgrade ant to 1.10.12 in branch-3

Backported HIVE-26081, HIVE-26172
What changes were proposed in this pull request?
Upgrade ant version to 1.10.12
Why are the changes needed?
To fix CVE-2021-36373 and be in-sync with master branch
Does this PR introduce any user-facing change?
No
How was this patch tested?
Existing Tests

@vihangk1 can you review this?