fold() works with arrays too, not mentioned in the documentation

http://rethinkdb.com/api/javascript/fold/
most of the functions that can be used with arrays mentions that in the "command syntax" section of the documentation. Arrays are sequences, but some people, new users mostly, might benefit from it being specified in the api docs. Also would make fold consistent with the other functions.

I'm not sure what the current convention is, but historically "sequence" meant "stream or array", so the signature on that page should indicate that the operation works on arrays.

That's still the convention, yes. I looked around and I didn't find any other API signatures (the "Command syntax") section that list both "sequence" and "array" as input.

@chipotle huh? I started looking at random position and stopped after these:
filter has selection, stream, array http://rethinkdb.com/api/javascript/#filter
innerJoin & outerJoin has sequence and array http://rethinkdb.com/api/javascript/#inner_join
map has sequence1, array1, r.map http://rethinkdb.com/api/javascript/#map
skip & limit has sequence and array http://rethinkdb.com/api/javascript/#skip
Tho I don't really mind. I was coming from the perspective of new users who might be confused about sequences in the first place, seeing how we usually talk about them and mean the stuff that comes from the table and the fact that some functions specify them and then this one doesn't. I kind of would read that as "doesn't seem to work with arrays".

Some of them are also slightly wrong. For example for innerJoin, the signature says
sequence.innerJoin(otherSequence, predicate_function) → stream
array.innerJoin(otherSequence, predicate_function) → array

The first one takes a sequence as an input, and the documentation says that it results in a stream. However I assume that this is only the case if the input is actually a stream as well. sequence would mean stream or array, and in the array case innerJoin does definitely not return a stream as is specified in the second line.
Even though it generates some bloat in the signatures, I'm thinking that it might be best to never use the term sequence in the type signatures, and to instead split it up into stream and array everywhere.
It would a) be more consistent and b) would probably be easier to comprehend for users who are not familiar with what exactly constitutes a sequence.

I'm closing this in favor of #1174 as that seems to be what we've moved toward to in the comments. That one will, er, fold in fold as well.


Add Pytorch memory stats along with PyNVML

This PR adds two memory stats to the inference_results.csv file: max_memory_allocated and max_memory_reserved for both forward passand generate pass.

this PR introduce the deprecation of forward.peak_memory(MB) and generate.peak_memory(MB) in favor  of forward.max_memory_used(MB) and generate.peak_memory(MB)


Subject Area FVT does not run with local graph repo.

https://github.com/odpi/egeria/issues/3446 will knit this into the build

The first bug is that when a createTerm is issued with associate governance classifications - the entityDetail for creation has the associated classifications. The graph repo logic is stricter than the in memory repo logic and checks that the classification that is supplied has a type. The first fix is to supply the appropriate type.

The second bug is around updateTerm when there are changes to the classifications.

On the first issue - do we need to also tighten up the in memory repo?

On the first issue - do we need to also tighten up the in memory repo?
I have raised issue https://github.com/odpi/egeria/issues/3455 to have the CTS be enhanced to test any cases where the in memory repo and the graph repo act differently, that the current Subject Area FVT exposes. We should ensure both repos work before committing the CTS enhancements.

The 3rd issue is we have an entity with a classification that has an enum property set. In this case it is Criticality and the level is set to Catastrophic. The FVTI then issues an classifyEntity, the in memory repo makes the change the returns the level in the classification as null, the graph repo does not make the change. What actually happens is that the attribute is not sent to the server in the rest call, so the server gets a classification with no properties. The in memory repo does not check for null and just copies null over, the graph repo checks for null and makes no changes to the existing value. I think the graph repo is correct in this case and am amending the in memory repo logic and the FVT test.

fyi I have a fix for the 2nd issue -which is to move the remove out of the for loop to avoid the concurrentModificationException

3rd issue update" I amended the in memory repo code in so that if there was an existing classification of the same name as the new one, and the new classification had null instance properties - then use the existing classification instanceProperties, unfortunately this failed junit DatabaseContextHandlerTest with a null pointer in the build. I am not sure how to run this junit out side of the build @planetf1 any ideas?

2 approaches
a) Hack the groovy script to add a long sleep at the end. Everything needed should be setup. Then when the maven build gets to that point run the test in intellij
b) Start a server chassis manually (think the tests are coded to use 10443 so use -Dserver.port=10443) and just configure the inmem/graph repo + subject area OMAS as usual
I'm not sure the best way to support debugging in future. Perhaps easiest for server-side is to allow an option to set the agent string so that the server VM will pause until debugger attached? (which is something else you could do manually via the groovy script). Mostly though if the test fails the dev of the area is likely to run the server manually anyway.

That test is in information view. It's a regular unit test so though it's run from maven/groovy it should be runnable in intellij without doing anything special. Since information view is incomplete it could well be a put in that component's test code too.
Where do you get the NPE? If it's server-side I would launch a chassis, attach with debugger, then run the JUnit from intellij? And if it's client it's even easier - just debug the test case directly?

For the 3rd issue I am commenting out the check in the TermFVT and have raised issue https://github.com/odpi/egeria/issues/3457 for it to be looked at.

For the 3rd issue another option is to mark test as disabled (as it then shows up in test reports as skipped) - but whichever way works

That test is in information view. It's a regular unit test so though it's run from maven/groovy it should be runnable in intellij without doing anything special. Since information view is incomplete it could well be a put in that component's test code too.
Where do you get the NPE? If it's server-side I would launch a chassis, attach with debugger, then run the JUnit from intellij? And if it's client it's even easier - just debug the test case directly?

Thanks Nigel - I managed to run it using Maven install; junit was failing to find a DTD. I have commented out the failing test and reverted the change I made that caused this issue. I have raised this as a separate issue, so the FVT should now work after the pr goes in, with one check commented out.

If it's a failing test it's probably better to annotate with @Disabled or @Test(enabled=false) depending on test framework/similar - that way we get a report of any skipped tests in the overall results. Not critical and good we have an issue for it, just a minor thought.

If it's a failing test it's probably better to annotate with @disabled or @test(enabled=false) depending on test framework/similar - that way we get a report of any skipped tests in the overall results. Not critical and good we have an issue for it, just a minor thought.

Unfortunately the test is a sequences of API calls and checks that they complete successfully, as such disabling the error is not being one of the checks (i.e. commenting out an if). commenting out the test would remove the many good tests that are around the failing one.

fair enough :-)


Regression: Image custom object is not properly handled in RichText demo

Launch the RichTextDemo
Insert an image
Place cursor to start of paragraph
Press Enter
=> Image is shown twice, once in the upper (new) paragraph and once in the lower paragraph.
When using an older variant of the LinkedImageOps, it works well.

Modifying realSubSequence in NodeSegmentOpsBase so that it creates an empty segment when the sub sequence length is 0 fixes it:
    @Override
    public SEG realSubSequence(SEG seg, int start, int end) {
        if (start == end) {
            return createEmpty();
        }
        return seg;
    }

Needs some additional testing before submitting a PR, but I think this should be a safe assumption in any case ...

This issue no longer arises in #590

Closed by #590


python310Packages.rapidfuzz: 2.13.7 -> 2.14.0

~This breaks psycopg: https://github.com/psycopg/psycopg/issues/537~
Description of changes
Diff: https://github.com/maxbachmann/RapidFuzz/compare/refs/tags/v2.13.7...v2.14.0
Changelog: https://github.com/maxbachmann/RapidFuzz/blob/refs/tags/v2.14.0/CHANGELOG.rst

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

@ofborg eval

Result of nixpkgs-review pr 224109 run on aarch64-linux 1

2 packages marked as broken and skipped:

truvari
truvari.dist

18 packages failed to build:

napari (python310Packages.napari)
napari.dist (python310Packages.napari.dist)
paperwork
paperwork.dist
python310Packages.autofaiss
python310Packages.autofaiss.dist
python310Packages.paperwork-backend
python310Packages.paperwork-backend.dist
python310Packages.paperwork-shell
python310Packages.paperwork-shell.dist
python311Packages.autofaiss
python311Packages.autofaiss.dist
python311Packages.paperwork-backend
python311Packages.paperwork-backend.dist
python311Packages.paperwork-shell
python311Packages.paperwork-shell.dist
tts
tts.dist

107 packages built:

adl
anime-downloader
anime-downloader.dist
apostrophe
atomic-operator
atomic-operator.dist
ibus-engines.uniemoji
larynx-train (python310Packages.larynx-train)
larynx-train.dist (python310Packages.larynx-train.dist)
ledger-autosync
ledger-autosync.dist
paperless-ngx
pgcli (python310Packages.pgcli)
pgcli.dist (python310Packages.pgcli.dist)
pianotrans
pianotrans.dist
pitivi
poetry
poetry.dist
poetryPlugins.poetry-audit-plugin
poetryPlugins.poetry-audit-plugin.dist
poetryPlugins.poetry-plugin-up
poetryPlugins.poetry-plugin-up.dist
puddletag
puddletag.dist
pynitrokey
pynitrokey.dist
python310Packages.app-model
python310Packages.app-model.dist
python310Packages.cleo
python310Packages.cleo.dist
python310Packages.cython_3
python310Packages.cython_3.dist
python310Packages.fire
python310Packages.fire.dist
python310Packages.fuzzywuzzy
python310Packages.fuzzywuzzy.dist
python310Packages.in-n-out
python310Packages.in-n-out.dist
python310Packages.levenshtein
python310Packages.levenshtein.dist
python310Packages.librosa
python310Packages.librosa.dist
python310Packages.nkdfu
python310Packages.nkdfu.dist
python310Packages.pgspecial
python310Packages.pgspecial.dist
python310Packages.piano-transcription-inference
python310Packages.piano-transcription-inference.dist
python310Packages.psycopg
python310Packages.psycopg.dist
python310Packages.psycopg.doc
python310Packages.rapidfuzz
python310Packages.rapidfuzz.dist
python310Packages.soxr
python310Packages.soxr.dist
python310Packages.syncedlyrics
python310Packages.syncedlyrics.dist
python310Packages.thefuzz
python310Packages.thefuzz.dist
python310Packages.torchlibrosa
python310Packages.torchlibrosa.dist
trytond (python310Packages.trytond)
trytond.dist (python310Packages.trytond.dist)
python310Packages.videocr
python310Packages.videocr.dist
python311Packages.app-model
python311Packages.app-model.dist
python311Packages.cleo
python311Packages.cleo.dist
python311Packages.cython_3
python311Packages.cython_3.dist
python311Packages.fire
python311Packages.fire.dist
python311Packages.fuzzywuzzy
python311Packages.fuzzywuzzy.dist
python311Packages.in-n-out
python311Packages.in-n-out.dist
python311Packages.levenshtein
python311Packages.levenshtein.dist
python311Packages.nkdfu
python311Packages.nkdfu.dist
python311Packages.pgcli
python311Packages.pgcli.dist
python311Packages.pgspecial
python311Packages.pgspecial.dist
python311Packages.psycopg
python311Packages.psycopg.dist
python311Packages.psycopg.doc
python311Packages.rapidfuzz
python311Packages.rapidfuzz.dist
python311Packages.soxr
python311Packages.soxr.dist
python311Packages.syncedlyrics
python311Packages.syncedlyrics.dist
python311Packages.thefuzz
python311Packages.thefuzz.dist
python311Packages.trytond
python311Packages.trytond.dist
python311Packages.videocr
python311Packages.videocr.dist
spotdl
spotdl.dist
sublime-music
sublime-music.dist
ulauncher
ulauncher.dist

Result of nixpkgs-review pr 224109 run on x86_64-linux 1

39 packages failed to build:

napari (python310Packages.napari)
napari.dist (python310Packages.napari.dist)
paperless-ngx
paperwork
paperwork.dist
python310Packages.autofaiss
python310Packages.autofaiss.dist
python310Packages.paperwork-backend
python310Packages.paperwork-backend.dist
python310Packages.paperwork-shell
python310Packages.paperwork-shell.dist
python311Packages.autofaiss
python311Packages.autofaiss.dist
python311Packages.cleo
python311Packages.cleo.dist
python311Packages.fire
python311Packages.fire.dist
python311Packages.fuzzywuzzy
python311Packages.fuzzywuzzy.dist
python311Packages.levenshtein
python311Packages.levenshtein.dist
python311Packages.nkdfu
python311Packages.nkdfu.dist
python311Packages.paperwork-backend
python311Packages.paperwork-backend.dist
python311Packages.paperwork-shell
python311Packages.paperwork-shell.dist
python311Packages.rapidfuzz
python311Packages.rapidfuzz.dist
python311Packages.syncedlyrics
python311Packages.syncedlyrics.dist
python311Packages.thefuzz
python311Packages.thefuzz.dist
python311Packages.trytond
python311Packages.trytond.dist
python311Packages.videocr
python311Packages.videocr.dist
truvari
truvari.dist

88 packages built:

adl
anime-downloader
anime-downloader.dist
apostrophe
atomic-operator
atomic-operator.dist
ibus-engines.uniemoji
larynx-train (python310Packages.larynx-train)
larynx-train.dist (python310Packages.larynx-train.dist)
ledger-autosync
ledger-autosync.dist
pgcli (python310Packages.pgcli)
pgcli.dist (python310Packages.pgcli.dist)
pianotrans
pianotrans.dist
pitivi
poetry
poetry.dist
poetryPlugins.poetry-audit-plugin
poetryPlugins.poetry-audit-plugin.dist
poetryPlugins.poetry-plugin-up
poetryPlugins.poetry-plugin-up.dist
puddletag
puddletag.dist
pynitrokey
pynitrokey.dist
python310Packages.app-model
python310Packages.app-model.dist
python310Packages.cleo
python310Packages.cleo.dist
python310Packages.cython_3
python310Packages.cython_3.dist
python310Packages.fire
python310Packages.fire.dist
python310Packages.fuzzywuzzy
python310Packages.fuzzywuzzy.dist
python310Packages.in-n-out
python310Packages.in-n-out.dist
python310Packages.levenshtein
python310Packages.levenshtein.dist
python310Packages.librosa
python310Packages.librosa.dist
python310Packages.nkdfu
python310Packages.nkdfu.dist
python310Packages.pgspecial
python310Packages.pgspecial.dist
python310Packages.piano-transcription-inference
python310Packages.piano-transcription-inference.dist
python310Packages.psycopg
python310Packages.psycopg.dist
python310Packages.psycopg.doc
python310Packages.rapidfuzz
python310Packages.rapidfuzz.dist
python310Packages.soxr
python310Packages.soxr.dist
python310Packages.syncedlyrics
python310Packages.syncedlyrics.dist
python310Packages.thefuzz
python310Packages.thefuzz.dist
python310Packages.torchlibrosa
python310Packages.torchlibrosa.dist
trytond (python310Packages.trytond)
trytond.dist (python310Packages.trytond.dist)
python310Packages.videocr
python310Packages.videocr.dist
python311Packages.app-model
python311Packages.app-model.dist
python311Packages.cython_3
python311Packages.cython_3.dist
python311Packages.in-n-out
python311Packages.in-n-out.dist
python311Packages.pgcli
python311Packages.pgcli.dist
python311Packages.pgspecial
python311Packages.pgspecial.dist
python311Packages.psycopg
python311Packages.psycopg.dist
python311Packages.psycopg.doc
python311Packages.soxr
python311Packages.soxr.dist
spotdl
spotdl.dist
sublime-music
sublime-music.dist
tts
tts.dist
ulauncher
ulauncher.dist


Warn the user on links that dont match their text

Adds an interstitial for links in posts that dont match their text. If the text is not a URL, it just gives the user a normal "Leaving this site". If the text is a URL that doesnt match the target, it gives a "Possibly misleading" warning. If the link is the URL, it opens as usual.

Dark mode broken

because of the startsWith use, wouldn't it be worth to check what comes after the host? if it doesn't end with /, ?, # or undefined, then it should be rejected
can imagine a scenario where someone would link to website.co but in actually it links to website.com which is a different site entirely.

Something you might want to consider is an additional emphasis on the apex domain. Web users often do not understand the counterintuitive order of trust anchoring in domains. For example:
https://paypal.com.billing-department-enquiries.example.com/blah-blah-blah

@DavidBuchanan314

Something you might want to consider is an additional emphasis on the apex domain, in the interstitial

Might as well ask it here, what's the right way you consider to check whether something is a domain or an IP address? is checking the end of the hostname to see if they're alphabetical or numerical the right choice here?

 Fixed now.

@ansh Can't believe I missed dark mode, thanks for hitting that
Should be good to go


error() is broken after running exec()

Something seems broken with the error() function after running exec. I would expect error() to always return something if exec().code !== 0 but that is not the case. Some sample output from the console:
> x = sh.exec('curl -s -S capoie')
curl: (6) Could not resolve host: capoie
{ code: 6,
  output: 'curl: (6) Could not resolve host: capoie\n' }
> sh.error()
null

It works better with a command like cd but the output from error() is still incomplete:
> x = sh.exec('cd sfs')
/bin/sh: line 0: cd: sfs: No such file or directory
{ code: 1,
  output: '/bin/sh: line 0: cd: sfs: No such file or directory\n' }
> sh.error()
'exec: \n'
> x.output
'/bin/sh: line 0: cd: sfs: No such file or directory\n'

@amjibaly: That's because error() just returns stderr. This will be fixed by #269.

I'm going to close this, as it will be fixed.


extend_load_ft is not working for some filetype

I try to extend NeogitCommitMessage, which is a ft defined by Neogit, from gitcommit.
Setup is here. In the commit:

Clear the snippets of NeogitCommitMessage
Extend NeogitCommitMessage from gitcommit
No luasnip triggered from gitcommit in NeogitCommitMessage

When I type luasnip, it should have triggered this. The process is recorded luasnip: NeogitCommitMessage.
luasnip is working  in gitcommit. The process is recorded too luasnip: gitcommit.

Ah, load_ft_func is exclusively used by the loaders, for your purposes ls.ft_extend("NeogitCommitMessage", {"gitcommit"}) should do it.
load_ft_func is useful when files need to expand snippets from filetypes different from their own (eg. whatever is reported by ft_func and the ft_extend-system). One example for this is treesitter-aware expansion in markdown-files (fenced code blocks could be lua or json, for example)

I think it's ls.filetype_extend. I've seen that usage in the Examples, but I forgot it :(
Now it works!

treesitter-aware expansion in markdown-files

Could you give an example code for this? I tried the following, and no snippet defined in rust or lua snippet files shows up.
local load_ft_func = extend_load_ft {
  markdown = { 'rust', 'lua' },
}
ls.setup { load_ft_func = load_ft_func }

Ah, that's what ft_func is responsible for that:
ls.setup({
	ft_func = require("luasnip.extras.filetype_functions").from_pos_or_filetype,
	load_ft_func = require("luasnip.extras.filetype_functions").extend_load_ft({
		markdown = {"lua", "json"}
	})
})

Well, it doesn't work for me, either :(

Oh, it should only expand lua-snippet when inside a fenced code-block for lua, does that work?

it should only expand lua-snippet when inside a fenced code-block for lua

It didn't.
There is the setup and the screencast.

Ouh, looks like the markdown-parser is a tad unreliable when it comes to reporting the correct filetype :|
get_snippet_filetypes is used internally to get the active filetypes at the cursor-position, so anything reported there is valid for the next expand.
https://user-images.githubusercontent.com/41961280/187065228-412278cb-1a96-4c89-ba3d-6638a19bc625.mp4
(Whenever I've typed fn and nothing happens for a while, I'm furiosly mashing expand :D)

https://user-images.githubusercontent.com/25300418/187066928-d80076fa-30e4-493d-9fb5-f63956700d59.mp4
I tried get_snippet_filetypes too, and the filetype in codefence is always { "markdown", "all" } for me, though I see you get lua filetype somewhere.

Weird, which parser do you use? I'm using just markdown, not markdown_inline

markdown_inline and markdown.
This
is the same with markdown only:

Huh.
Which nvim-version are you on?
I'm on master

NVIM v0.8.0-dev-922-g42e9fe7d9
Build type: RelWithDebInfo
LuaJIT 2.1.0-beta3
Compilation: /usr/bin/gcc-10 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1 -DNVIM_TS_HAS_SET_MATCH_LIMIT -DNVIM_TS_HAS_SET_ALLOCATOR -O2 -g -Og -g -Wall -Wextra -pedantic -Wno-unused-parameter -Wstrict-prototypes -std=gnu99 -Wshadow -Wconversion -Wdouble-promotion -Wmissing-noreturn -Wmissing-format-attribute -Wmissing-prototypes -Wimplicit-fallthrough -Wvla -fstack-protector-strong -fno-common -fdiagnostics-color=always -DINCLUDE_GENERATED_DECLARATIONS -D_GNU_SOURCE -DNVIM_MSGPACK_HAS_FLOAT32 -DNVIM_UNIBI_HAS_VAR_FROM -DMIN_LOG_LEVEL=3 -I/home/runner/work/neovim/neovim/build/cmake.config -I/home/runner/work/neovim/neovim/src -I/home/runner/work/neovim/neovim/.deps/usr/include -I/usr/include -I/home/runner/work/neovim/neovim/build/src/nvim/auto -I/home/runner/work/neovim/neovim/build/include
Compiled by runner@fv-az130-680

Mhm, okay no difference there.
That was my last guess, no idea what's different then :/

Could you try if injected filetypes are recognized in other circumstances? For example, vim.cmd in lua-files (depends on the lua-parser though, so you might have to add it)

Still { "lua", "all" } everywhere in  vim.cmd... No idea...

Mmmmmm that's very weird, it looks like the highlight is even correct.
from_pos_or_filetype is defined in this file, you could insert some prints to find what's going wrong.
Maybe one of the nvim-treesitter-requires is not resolved?


Add modifier = Modifier parameter to all composable function in Search

Idea Description
Please make it configurable.
Reference images and links
https://github.com/DroidKaigi/conference-app-2022/blob/main/feature-sessions/src/main/java/io/github/droidkaigi/confsched2022/feature/sessions/Search.kt
https://chris.banes.dev/always-provide-a-modifier/

✋🏻
Can I challenge this work?

Yes you can!! Thanks!! assigned!!

I have some questions.
1. Does the Function annotated @Preview need to have a modifier parameter?
In this case, the function is SearchScreenPreview
2. I think that the modifier's position is the first. Is it OK?
ex.
private fun SearchTextField(
    modifier: Modifier = Modifier, // <- here
    searchWord: String,
    onSearchWordChange: (String) -> Unit
)

3. How many tasks are there?
I think there are 2 tasks for this issue.

Add modifier parameter to composable function signatures
Pass modifier to a suitable child composable function

Should I do both tasks or the first only?

👀

Sorry, I forgot the Preview functions. 😅 The function doesn't need to have a modifier parameter.
First optional parameter and before the content parameter is the best.

private fun Hoge(
    searchWord: String,
    onSearchWordChange: (String) -> Unit
    modifier: Modifier = Modifier, // I think here is the best. But I think it is not such an important rule.
    optionalParameter: String? = null
    content: @Composable () -> Unit
)

https://android.googlesource.com/platform/frameworks/support/+/androidx-main/compose/docs/compose-api-guidelines.md#elements-accept-and-respect-a-modifier-parameter

Can you do both of them? Please pass it to the root composable function in it. 🙏

Thank you for your reply.
I'll do them.


knn metric cosine ValueError: Unrecognized metric 'cosine'

clf = KNN(metric="cosine")
clf.fit(train_data)

results in
sklearn/neighbors/_binary_tree.pxi in sklearn.neighbors._ball_tree.BinaryTree.__init__()

sklearn/neighbors/_dist_metrics.pyx in sklearn.neighbors._dist_metrics.DistanceMetric.get_metric()

ValueError: Unrecognized metric 'cosine'

Also brute and kd_tree don't support cosine, meaning that cosine is not supported by any algorithm.

Thanks for the note. PR is welcomed :)

I was trying to contribute on this and did a little bit research, answer as below:
Thus I am wondering is this PR still necessary?

The cosine similarity is generally defined as xT y / (||x|| * ||y||), and outputs 1 if they are the same and goes to -1 if they are completely different. This definition is not technically a metric, and so you can't use accelerating structures like ball and kd trees with it. If you force scikit learn to use the brute force approach, you should be able to use it as a distance if you pass it your own custom distance metric object. There are methods of transforming the cosine similarity into a valid distance metric if you would like to use ball trees (you can find one in the JSAT library)
Notice though, that xT y / (||x|| * ||y||) = (x/||x||)T (y/||y||). The euclidean distance can be equivalently written as sqrt(xTx + yTy − 2 xTy). If we normalize every datapoint before giving it to the KNeighborsClassifier, then x^T x = 1 for all x. So the euclidean distance will degrade to sqrt(2 − 2x^T y). For completely the same inputs, we would get sqrt(2-21) = 0 and for complete opposites sqrt(2-2-1)= 2. And it is clearly a simple shape, so you can get the same ordering as the cosine distance by normalizing your data and then using the euclidean distance. So long as you use the uniform weights option, the results will be identical to having used a correct Cosine Distance.

If this PR is not necessary, change the documentation and delete the "cosine" support. It contradicts with the class documentation.


Drone CI/CD Pipeline Stuck in Pending State with 'http: no content returned: re-connect and re-try' in Runner Logs

I have been encountering repeated "http: no content returned: re-connect and re-try" errors in the logs of my Drone Runner. My build status in Drone CI/CD is always "pending" and does not proceed. I have been unable to identify the cause of this issue.
Here are the relevant configurations and logs:
My drone.yml configuration:
kind: pipeline
type: ssh
name: drone

clone:
  disable: true

steps:
- name: build-and-deploy-ipa
  environment:
    SCP_PASSWORD:
      from_secret: scp_password
    SSH_PASSWORD:
      from_secret: ssh_password
  commands:
  - sshpass -p $SSH_PASSWORD ssh -p 1022 xxx@xxx 'cd /Projects/project-gitlab/project_app && git fetch origin && git checkout feature/sit-clone && git pull origin feature/sit-clone'
  - sshpass -p $SSH_PASSWORD ssh -p 1022 xxx@xxx 'cd /Projects/project-gitlab/project_app && rm -f build/ios/ipa/app.ipa && flutter pub get'
  - sshpass -p $SSH_PASSWORD ssh -p 1022 xxx@xxx 'cd /Projects/project-gitlab/project_app && flutter build ipa --export-options-plist=ios/exportOptions.plist --dart-define=BYPASS_VERSION_CHECK=true'
  - sshpass -p $SSH_PASSWORD ssh -p 1022 xxx@xxx 'scp -p $SCP_PASSWORD build/ios/ipa/app.ipa root@xxx:/srv/nginx/content/download/ios/sunod'
  - sshpass -p $SSH_PASSWORD ssh -p 1022 xxx@xxx 'echo "Deployed ipa @LAB"

trigger:
  branch:
    - feature/test

The intention of this pipeline is to have the runner SSH into a Mac and execute several commands, particularly Flutter commands,
My docker-compose.yml configuration:
version: '3'

services:
  drone-server:
    image: drone/drone:latest
    container_name: drone-server
    ports:
      - 80:80
    volumes:
      - /var/lib/drone:/data
    restart: always
    environment:
      - DRONE_LOGS_DEBUG=true
      - DRONE_GITLAB_SERVER=xxx
      - DRONE_GITLAB_CLIENT_ID=xxx
      - DRONE_GITLAB_CLIENT_SECRET=xxx
      - DRONE_SERVER_HOST=xxx
      - DRONE_SERVER_PROTO=http
      - DRONE_RPC_SECRET=xxx
      - DRONE_USER_CREATE=username:xxx,machine:false,admin:true,token:xxx
      - DRONE_LOGS_TRACE=true

  drone-runner:
    image: drone/drone-runner-docker:latest
    container_name: drone-runner
    ports:
      - 3000:3000
    restart: always
    depends_on:
      - drone-server
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      - DRONE_RPC_HOST=drone-server
      - DRONE_RPC_PROTO=http
      - DRONE_RPC_SECRET=xxx
      - DRONE_RUNNER_CAPACITY=2
      - DRONE_RUNNER_NAME=drone-runner
      - DRONE_LOGS_TRACE=true

log messages from my Drone Runner:
time="2023-08-05T19:30:29Z" level=trace msg="http: no content returned: re-connect and re-try" time="2023-08-05T19:30:59Z" level=trace msg="http: no content returned: re-connect and re-try" time="2023-08-05T19:31:09Z" level=trace msg="http: no content returned: re-connect and re-try" ...

@ray2211 you ever figure this out? Runnin into the same thing after starting the runner.

I have been encountering repeated "http: no content returned: re-connect and re-try" errors

note that these are not errors, these are trace messages.  the runners perform long-polling and are constantly polling for work, which means they connect, disconnect, and retry until they pull work from the queue.
we have a comprehensive faq to triage why no builds are running here:
https://discuss.harness.io/t/nothing-happens-when-i-push-code-webhooks-not-working-no-builds-or-builds-stuck-in-pending/11522
and a comprehensive guide for builds stuck in pending here:
https://discuss.harness.io/t/builds-are-stuck-in-pending-status/11275
every single known root cause is covered in the above faqs

@bradrydzewski thank you for confirming it's intended and for providing additional information. Much appreciated!


Project maintenance

Landed in https://github.com/metarhia/metarhia.com/commit/265ea1be184c2347045aa064a70168830aeebb56, https://github.com/metarhia/metarhia.com/commit/5984c99010ecba62aacba11a6c8a89a9615dfea4, and https://github.com/metarhia/metarhia.com/commit/0d92111686c241db2268199ec751c94df5ba6c1b


RuntimeError: Sizes of tensors must match except in dimension 0. Got 1 and 2 (The offending index is 1)

Hi,
When running a test on a short sequence of less than 200 frames, there is no issue at all. But for some reason, as soon as the 201st frame is processed, I get the error described futher down this post.
In tracking/test_mixformer.sh is written the following :
python tracking/test.py mixformer2_vit_online 288_depth8_score \  --dataset got10k_test --threads 1 --num_gpus 1 \  --params__model models/mixformerv2_base.pth.tar \  --params__search_area_scale 5.0 \  --debug 1
And launching this command ends up in getting the following error when the  201st frame is processed :
`"""
Traceback (most recent call last):
File "/media/phebe_storage/users/de-faria_isl.fr/test_trackers/MixFormerV2/.conda/lib/python3.6/multiprocessing/pool.py", line 119, in worker
result = (True, func(*args, **kwds))
File "/media/phebe_storage/users/de-faria_isl.fr/test_trackers/MixFormerV2/.conda/lib/python3.6/multiprocessing/pool.py", line 47, in starmapstar
return list(itertools.starmap(args[0], args[1]))
File "tracking/../lib/test/evaluation/running.py", line 133, in run_sequence
output = tracker.run_sequence(seq, debug=debug)
File "tracking/../lib/test/evaluation/tracker.py", line 90, in run_sequence
output = self._track_sequence(tracker, seq, init_info)
File "tracking/../lib/test/evaluation/tracker.py", line 145, in _track_sequence
out = tracker.track(image, info)
File "tracking/../lib/test/tracker/mixformer2_vit_online.py", line 98, in track
out_dict = self.network(self.template, self.online_template, search, softmax=True, run_score_head=True)
File "/media/phebe_storage/users/de-faria_isl.fr/test_trackers/MixFormerV2/.conda/lib/python3.6/site-packages/torch/nn/modules/module.py", line 729, in _call_impl
result = self.forward(*input, **kwargs)
File "tracking/../lib/models/mixformer2_vit/mixformer2_vit_online.py", line 333, in forward
template, online_template, search, reg_tokens, distill_feat_list = self.backbone(template, online_template, search)
File "/media/phebe_storage/users/de-faria_isl.fr/test_trackers/MixFormerV2/.conda/lib/python3.6/site-packages/torch/nn/modules/module.py", line 729, in _call_impl
result = self.forward(*input, **kwargs)
File "tracking/../lib/models/mixformer2_vit/mixformer2_vit_online.py", line 238, in forward
x = torch.cat([x_t, x_ot, x_s, reg_tokens], dim=1)  # (b, hw+hw+HW+4, embed_dim)
RuntimeError: Sizes of tensors must match except in dimension 0. Got 1 and 2 (The offending index is 1)
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "tracking/test.py", line 72, in
main()
File "tracking/test.py", line 68, in main
args.threads, num_gpus=args.num_gpus, tracker_params=tracker_params)
File "tracking/test.py", line 34, in run_tracker
run_dataset(dataset, trackers, debug, threads, num_gpus=num_gpus)
File "tracking/../lib/test/evaluation/running.py", line 182, in run_dataset
pool.starmap(run_sequence, param_list)
File "/media/phebe_storage/users/de-faria_isl.fr/test_trackers/MixFormerV2/.conda/lib/python3.6/multiprocessing/pool.py", line 274, in starmap
return self._map_async(func, iterable, starmapstar, chunksize).get()
File "/media/phebe_storage/users/de-faria_isl.fr/test_trackers/MixFormerV2/.conda/lib/python3.6/multiprocessing/pool.py", line 644, in get
raise self._value
RuntimeError: Sizes of tensors must match except in dimension 0. Got 1 and 2 (The offending index is 1)`
Any idea of how to fix this issue ?
Thanks in advance !

Hi, thanks for your attention!
I checked the code and found the problem. Now this version of code can only support test with online size 1, otherwise this issue will occur.
You can set it in the config file experiments\mixformer2_vit_online\288_depth8_score.yaml:
TEST:
  ONLINE_SIZES:
    ...
    GOT10K_TEST: [1]

I will correct the config to avoid mistake and a version of code that supports multiple templates inference is in preparation and will be released when ready.

Thanks a lot for your response!

It worked

is there an update?

Hi, thanks for your attention! I checked the code and found the problem. Now this version of code can only support test with online size 1, otherwise this issue will occur. You can set it in the config file experiments\mixformer2_vit_online\288_depth8_score.yaml:
TEST:
  ONLINE_SIZES:
    ...
    GOT10K_TEST: [1]

I will correct the config to avoid mistake and a version of code that supports multiple templates inference is in preparation and will be released when ready.


UI: RFC: splash screen warning

Right now, on each start, a "scary" warning is shown. While I believe a warning in general isn't a bad thing, I'm wondering about:

Should this warning be shown on mainnet, testnet and regtest or only mainnet?
Should this warning be refined, e.g. with softer language?
Should this warning be shown on every start or only once, or once, mainnet and testnet?

Ping @oktype as UIX expert. :)
On Ubuntu 14.04.01 with QT4:

On Windows 8.1:

WARNING - THIS IS ONLY MY TAKE ON THE ISSUE AND MY ADVICE SHOULD NOT BE CONSIDERED AS A LEGAL ADVICE
This is mostly a legal issue, so I guess that the amount of focus and the language this notification contain should be checked first from that perspective (especially If the purpose of this notification is also to actually try and remove responsibility from the foundation/developers)
Set aside the obligations, we still want to inform our users in a responsible way. So with that in mind, I would suggest: Yes, the language should be refined.
Right now those paragraphs can get 2 results: either the user will stay away from the platform and never come back, or he will try to categorize this new information only as an (somewhat over-excessive) effort to remove responsibility from the foundation/developers, hence he won't treat it with the respect/attention it should get.
Not too sure what are our possibilities for that notification box design-wise
also, If possible, I think that making sure the user read this notification at least once is enough (even use a 5 sec timer if possible). while adding some "under development" prominent link from within the UI leading to that notification for later reads (same as on Omni Wallet)
So:
1.I'm not sure I'm fully up to date with the differences testnet/regtest/mainnet and the capabilities to spend real value on each so can't advise on that issue
2.in my opinion: yes.
3.make sure the user reads it once, then give him access to read it again when he want to ponder in the meaning of 'under development'

From my humble experience with legal stuff, it is best to not try and make it look legal.
Make the shortest and easiest text to read and understand. (not a legal advice)

The project is licsensed as MIT open source project and the warning is sort of similar, see: http://opensource.org/licenses/MIT.
I think the main goal here is to raise awareness with the underlying message that a) it's no bullet proofed software release (even though we're very restrictive and it should not result in loss of coins), and because the protocol is updated from time to time, a user should at least know, that an update may be required at some point.

Not too sure what are our possibilities for that notification box design-wise

Currently it's only a plain dialog box, but styling or changing this window shouldn't be very difficult, and it's basically drag and drop (see here).

I'm not sure I'm fully up to date with the differences testnet/regtest/mainnet and the capabilities to spend real value on each so can't advise on that issue

Regtest is a "local blockchain mode" for developers with completely isolated environment, where bitcoins can be generated on by the user. This is used to test applications. In my opinion: no warning at all required.
Testnet is a "playground". "Test" bitcoins have no value and it's basically as "public testing environment". Imho no warning required as well.
Mainnet is "real", with "real" bitcoins, and potential loss of monetary value, if something goes very, very wrong.

make sure the user reads it once, then give him access to read it again when he want to ponder in the meaning of 'under development'

I like the idea of providing a choice to open this warning once again. Maybe from within the "About Omni Core" dialog, which can be accessed via the menu? It looks a bit broken at the moment and also needs a refinement, but this seems like the way to go. Maybe we could simply copy the message and show it there as well:

Is there any landing page available for users? It states the software is under active developmenet, but what's really missing is any information where the user could get additional information about the project or updated. What should he or she be referred to?

-The decision of where to put the notification link how much we want to emphasize the warning.
Omni wallet also contains this warning as a prominent link from the footer (visible from every page).
-Depends on the length of the text, it could be squeezed into the about dialog, or as a separate box, also available from a link and a short description from the about box.
I like the 'under development' definition much more then the 'experimental' one since it holds more potential and imply on the development efforts , but still imply it is not the final product.
As for the regtest/testnet/mainnet. if the risk warning is only relevant to the mainnet environment, warn only the users that uses this environment. 'test' imply the experimental feature, and since there is no risk, there is no reason for a warning.
We should try to find a better way to communicate 'Mainnet' for the users as this is a developers term.
Mainnet should be considered as the actual product, So when you are not working on the test environments (which should be hinted) you are using the product. (I might be wrong with this since the Omni Core target audience is tec savvy)
I agree with the fact that a documentation is needed and that the project should make information as approachable as possible. I've suggested to make some kind of help pages 'info center' as part of Omni. there is also a already implemented pending more robust website ready to be published instead of the current omnilayer.org landing page, as soon as it will be filled with the suitable content.
If currently there is no other place to get such information, maybe we should rush such space.
@achamely @CraigSellars

We should try to find a better way to communicate 'Mainnet' ...

It could be replaced by something that referrs to "production environments".
I'm not suitable to rephrase the text, but I'd be happy to replace it.
In general: I'd try to avoid causing "fear" whenever possible and chose a friendly language. Instead of starting with ALL-CAPS-WARNINGS, the user should be welcomed and informed. Keep it positive, it's the very first impression right after starting the client that is created! :)

I agree with the fact that a documentation is needed
I've suggested to make some kind of help pages 'info center' as part of Omni.
If currently there is no other place to get such information, maybe we should rush such space.

I would not recommend to build a help center within the application itself, if this is what you were referring to, but instead providing some kind of documentation or landing webpage.
At minimum there should be landing page, which states something along the lines:
The current Omni Core version is vX.Y.Z.

Users with previous versions must update their client until *some-date*, or otherwise ...

However simple this landing page might be, it's very, very, very crucial in my opinion. And I'd really like to provide a link to such a page from within Omni Core.
Quick reasoning why this is so important @oktype: whenever there is a protocol update (and major client update), say for example when a new transaction type is enabled, like the distributed token exchange, users with outdated clients basically can end up with a very different "view of world", and it's very likely that those users would see completely different and wrong balances.
Big slap @CraigSellars by the way (but sorry, if you're not the one to blame) for not publishing any post on the blog, the forums and mailing lists, mentioning the "send to owners" enabling at all. I showed once that I was able to drain masterxchange, because I exploited exactly this, and this is something that all users of Omni Core (and others interacting with the Omni Layer) must be informed of...

To summarize the discussion so far:

[ ] No warning should be shown for testnet and regtest mode, but mainnet only, because this is where real monetary value is at stake.

[ ] The warning should be shown once, but not every time the client is started.

[ ] From within the application there should be an option/menu item to show the warning again.

[ ] The warning should be rephrased.

[ ] There should be a landing page with further information, or at minimum some information about the current release and whether an update for the client is available or required.

I can take care of 1, 2, 3, but need someone else for 4 + 5.
Quick question: is there any contact email available for support, bugs reports, ...? Right now there are a few places where<EMAIL_ADDRESS>is shown, but I'm pretty sure we should replace this.

Hey guys,
Just wanted to add I stand by a very scary warning.  If anything I stand by making it even more scary.
For me this ties into the default wallet discussion.  Omni Core is unlike any other 'alt'coin/wallet.  By using our software you are not just talking about your Omni tokens, but also putting all of your Bitcoins in Omni's hands by default (a situation I am still against).
Example scenario - I have 200 Bitcoins in my local wallet.  I download and run OmniCore.  I thus put all of those Bitcoins at the mercy of a bug in OmniCore (and for Joe Q User may not have any clue that OmniCore even has access to their Bitcoins without such a warning).  Thus, yep I do kind of want to really drive the point home that this is extremely new software and somewhat experimental.  We're effectively asking users to put their Bitcoins in the hands of code that has never seen end user usage before & trust that everything is going to be OK.
Now for those of us with half a brain, 200 BTC should be in cold storage not in a local QT wallet but I use it as the example to drive the point home - while there is no segregation of Bitcoin and Omni wallet files and no tangible history of end user usage to give us N level of confidence in its safety, is not the responsible thing to err on the side of caution, and make sure the user knows all this?
It's been on the back burner, but FYI I'd intended to push for an expansion of the warning (since it doesn't actually make it clear enough at all in my book that your Bitcoins are accessible to and will by used by OmniCore).  We just talk about 'wallets', and for the regular end user they may well assume just like pretty much any other QT wallet out there, that OmniCore uses its own wallet file which is not the case.
TL:DR; separate the wallets and I wouldn't care if we slimmed the message to a single line, but as long as we insist on using the users existing Bitcoin wallet for the sake of 'convenience', I'll always be on the side of making sure users know what's up - it's the responsible thing to do IMO.
Open to debate of course :) but being open, transparent, truthful and making sure our users have all the information to hand is of critical importance to me.
Thanks
Z

@zathras-crypto: I think the general consensus is that a warning is very reasonable for the reasons you stated. "Scary" was maybe not the right word. The intention is to make sure the user knows what he or she does, namely risking coins, and what I was referring to was the tone of delivering this message. My point is basically about finding a balance between really making sure the user understands and creating a positive user experience.
Here is a similar example to which context I'm referring to: https://github.com/mastercoin-MSC/omniwallet/issues/895
I'd be very curious about hearing your opinion about a) whether the warning should be shown for testnet and regtest, and b) whether it should be shown once or every time the client is started.

I'd be very curious about hearing your opinion about a) whether the warning should be shown for testnet and regtest, and b) whether it should be shown once or every time the client is started.

Not fussed about testnet/regtest there is no risk to end user funds there, and FYI I was instructed a while ago to code it so it shows once at first run and then is never shown again (I just haven't prioritized that to date).
I appreciate what you're saying, I guess the two main points I'm trying to get across are:

Nowhere at all in the message does it state we will use the users existing Bitcoin wallet.  A new user would not be informed that he/she is trusting their Bitcoins to our code.  This to me is unacceptable.
We should not provide any false senses of stability/security.  We should be transparent that this is brand new stuff and could go wrong.

I'm absolutely open to changed wording, but the core 'message' if you will needs to be communicated.
Hope that clarifies a bit :)
Thanks
Z

P.S. for anyone in this discussion that isn't aware, for the 'background' that warning is a direct copy and paste from an older project called the Masterchest Wallet and was written by yours truly intentionally to be scary (because I had quite a stressful time during/just after the maidsafe crowdsale watching people pump millions of dollars through a piece of software I'd written seemingly not caring about the risks).  Just FYI :)

Apologies, making a liar out of myself - seems has been reworded quite a bit since then...

Alright then, I'm going to address the mainnet-only and "show once" part. This is something I have tested earlier.
Another idea: there could be a third option besides "Exit" and "Continue": (in the sense of) "Continue, but show the warning on next start again". Not sure how to style this though, as a dialog with three buttons and long text would probably look horrible.

I guess the two main points I'm trying to get across are ...

Sounds fine to me!
In addition, but this is probably not that easy: there could be an option to create a wallet.dat backup before the start. The (huge) downside is that wallet.dat backups were not only once misunderstood, because the file actually only holds 100 pre-generated keys per default, but once they are used, the backup wouldn't have the keys for the destinations where the coins might be held.
For the warning text itself, a part which highlights that Omni Core probably uses the same datadir and wallet.dat of an earlier installation, and that there could be direct exposure of those files, would be great. As general warning, even though I'm not really a fan of all-caps, the MIT license provides a blueprint:
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.

The (huge) downside...

We can prompt the user for their password and up the keypool to whatever (eg 10,000) we like before backup but that's not necessarily where I see the risk (there is nowhere we interact directly with the wallet ourselves, that's all left to underlying bitcoin code) so I'm not so concerned with corruption as I am accidentally giving the users BTC to a miner or exodus/reference/multisig output etc - if that happens a wallet backup isn't going to help sadly.

Another idea: there could be a third option besides "Exit" and "Continue": (in the sense of) "Continue, but show the warning on next start again". Not sure how to style this though, as a dialog with three buttons and long text would probably look horrible.

From a UX perspective, the vanilla way to handle this is to have the "Exit" and "Continue" buttons and then chuck a checkbox on left align saying "Do not show this message again".

For the warning text itself...

It's still my honest belief that when we warn (which we absolutely must do) users that their existing Bitcoin are exposed to and used by our software, we will put some users off using it.  I still believe the best way to handle that is with a segregated wallet file (which can be overridden to use the existing bitcoin wallet.dat with one simple cmdline param for advanced users) but I also don't want to get us stuck going round in circles on an issue that has been discussed before (and IIRC didn't really get a definitive outcome).
Thanks
Z

This was now addressed by #38, but awaiting a reworded warning, so let's keep it open.

I'd really like to include a link to a landing page with version and update information etc., but this seems to be out of this scope.


HomeKit iOS 11 - Open Protocol Specifications

Hi everyone,
I'm new to Github and coding, but I have been following this project for awhile.
Stumbled upon this today: https://developer.apple.com/homekit/specification/
How will this effect the future of Homebridge? It seems like it could make things a lot easier!

This should be discussed on, https://github.com/KhaosT/HAP-NodeJS/issues/457 . Already discussed on this repo under issue #1328


sshnoports: Add unit tests

Is your feature request related to a problem? Please describe.
No unit tests currently
Describe the solution you'd like
Need unit tests which, while they do not test the whole thing end to end, can check the logic and behaviour of the atProtocol interactions - the records that are created and updated, the notifications that are sent and received, etc
Describe alternatives you've considered
No response
Additional context
No response
### Tasks
- [ ] impl
- [ ] impl/dart pure
- [ ] impl/openssh local
- [ ] impl/unsigned (legacy)
- [x] models
- [x] models/config file repo (mostly done - last piece requires stubbing file io)
- [x] models/config key repo
- [x] models/sshnp arg
- [x] models/sshnp device list
- [x] models/sshnp params
- [x] models/sshnp result (done, but subject to change)
- [ ] util
- [x] util/initial tunnel
- [ ] util/initial tunnel/dart
- [ ] util/initial tunnel/openssh
- [x] util/ssh key handler
- [x] util/ssh key handler/dart
- [x] util/ssh key handler/local
- [x] util/sshnpd channel
- [x] util/sshnpd channel/default
- [x] util/sshnpd channel/unsigned
- [x] util/sshrvd channel
- [x] util/sshrvd channel/dart
- [x] util/sshrvd channel/exec
- [x] sshnp core
- [x] sshnp (public interface)

Unassigning myself and moving to Triage again ... hoping someone else has capacity to pick this up in this sprint

@cconstab  Steps I followed

sshnpd in aws
ubuntu@ip-172-26-14-158:~/sshnp$ ./sshnpd -a @95consistent -m @aquamarine659 -d device1 -u
SHOUT|2023-04-06 11:24:48.698741| sshnpd |Starting @95consistent sync

SHOUT|2023-04-06 11:24:55.364158| sshnpd |@95consistent sync complete
SHOUT|2023-04-06 11:27:16.519554| sshnpd |ssh session started from: @aquamarine659 session: 45891ce5-1f0e-4248-a409-a35885cb01b6

sshnp

dart sshnp.dart -k<EMAIL_ADDRESS>-f @aquamarine659 -t @95consistent -h 27d00de9-3719-56b3-82d2-5b97212a9959.swarm0002.atsign.zone -p 2215 -d device1
sshnp_logs.txt
From the logs, I could see a failure as
Remote sshnpd error: Remote SSH Client failure : SSHAuthAbortError(Connection closed before authentication)
But at the end, it says notification as delivered
FINER|2023-04-06 16:57:17.067081|AtLookup|SENDING: notify:status:91752f94-991c-47e4-aa0b-232c0d7958eb
FINER|2023-04-06 16:57:17.332635|OutboundMessageListener|RECEIVED data:delivered

So the -h  and -p should point to the place you are running sshnp.. Currently you have the host and the port of the secondary server which is not where you are running ssh from..
My Host -------------------------> Hosty I want to log into
sshnp ----------------------------> sshnpd
sshd (port 22) <-------------------sshnpd
so in this case the -h "my host" and -p "22"
hope that helps

The sshnp repo has lots of clues https://github.com/atsign-foundation/sshnoports

@cconstab
We tried to run sshnpd on my machine and sshnp on @purnimavenkatasubbu machine
Remote machine:

dart bin/sshnpd.dart -a @91elephant -m @ninelivesfew11 -d sshnptest -u sshnptest -s -v
murali@murali-Latitude-3460:/etc/ssh$ ssh -R 0:localhost:2222 tunnel.us.ngrok.com tcp
Allocated port 15869 for remote forward to localhost:2222

ngrok (via SSH) (Ctrl+C to quit)
Account<EMAIL_ADDRESS>(Plan: Free)
Region      us
Forwarding  tcp://4.tcp.ngrok.io:15869

added public key from client to .ssh/authorized_keys
murali@murali-Latitude-3460:~/.ssh$ cat authorized_keys
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+thV+w+fkRlRLh5QkbYGYXYIjHzwtqK1yhzFzS2FXq9ww5AVmDlTSge1g8mZGEQ4h+HX9O7kHDmsUKzF017F/rQv5JVE93THM3CXxKgnohq9wYyDDEHs5BXmLisWNyc9I8vf+dN2ngJa8tSRmiPoiBc+qYsizYTpIXxI8FfJj3vJbQ/HhxTmi3V1Ilo7LV+WgfVn8QSZk+mQ5aS/uu835WTQGZ0YtY7ml38DX28xGclVWQ/2aSUxsQHQpPwJopL8rzucmvX6PQNZDeoMBB74UoFab4mDnAgd4XWxOtEsNzlg2+XcHSZhKhuvKt7TrGrrJuSP9CTXrMQEj84upPkmR shaikirfan@SHLT013

client machine:
shaikirfan@SHLT013:~/Desktop/sshnp/sshnoports$ dart bin/sshnp.dart -f @ninelivesfew11 -t @91elephant -h 4.tcp.ngrok.io -p 15869 -l 3456 -d sshnptest -s id_rsa.pub
remote machine:
murali@murali-Latitude-3460:/etc/ssh$ ssh -R 0:localhost:2222 tunnel.us.ngrok.com tcp
Allocated port 15869 for remote forward to localhost:2222
ngrok (via SSH) (Ctrl+C to quit)
Account<EMAIL_ADDRESS>(Plan: Free)
Region      us
Forwarding  tcp://4.tcp.ngrok.io:15869
TCP connection opened
TCP connection closed
TCP connection opened
TCP connection closed
TCP connection opened
TCP connection closed
TCP connection opened
TCP connection closed
murali@murali-Latitude-3460:~/git/at_protocol/sshnoports$ journalctl -fu ssh
-- Logs begin at Tue 2022-11-08 16:31:00 IST. --
Apr 11 15:15:50 murali-Latitude-3460 sshd[12548]: Invalid user shaikirfan from <IP_ADDRESS> port 60094
Apr 11 15:15:50 murali-Latitude-3460 sshd[12548]: Connection closed by invalid user shaikirfan <IP_ADDRESS> port 60094 [preauth]
Apr 11 15:17:23 murali-Latitude-3460 sshd[17638]: Invalid user shaikirfan from <IP_ADDRESS> port 53258
Apr 11 15:17:24 murali-Latitude-3460 sshd[17638]: Connection closed by invalid user shaikirfan <IP_ADDRESS> port 53258 [preauth]
Apr 11 15:18:19 murali-Latitude-3460 sshd[20664]: Invalid user shaikirfan from <IP_ADDRESS> port 37644
Apr 11 15:18:20 murali-Latitude-3460 sshd[20664]: Connection closed by invalid user shaikirfan <IP_ADDRESS> port 37644 [preauth]
Apr 11 15:18:49 murali-Latitude-3460 sshd[22016]: Invalid user shaikirfan from <IP_ADDRESS> port 33968
Apr 11 15:18:50 murali-Latitude-3460 sshd[22016]: Connection closed by invalid user shaikirfan <IP_ADDRESS> port 33968 [preauth]

I would make sure that ssh logins are allowed on the remote machine the logs would suggest they are not..
To test on the remote machine try
ssh 0
You should be able to login
The other thing I notice is the sshnpd has -u sshnptest but the -u is just a flag and will pick up the processes username that is running sshnpd.

ssh 0

In the remote machine on which sshnpd runs, I ran the ssh service on port 2222. ssh 0 didn't work.
ssh 0 -p 2222 works.
Should I run the ssh service on the remote machine on port 22 or 2222?

sshnp refactoring is complete; a small number of unit tests were added during PR64

no progress during PR65 but work should proceed during PR66 now that refactoring is complete for sshnpd and sshnp

Refactoring is mostly done, but there are still some bugs that need to be addressed first. Once v3.4.0 is officially released I will update this card.

Since we now have comprehensive end-to-end test automation, this is not as important; reducing priority to P2

Reducing to 5SP

Lowered priority to P1 now that we've released 4.0.0

Will resume in PR 78

Will resume in PR80

I think this is in a good place for now, e2e tests cover the rest.


BUG: fix name scope collision in tf.layers

Fix  #13429.
Since  #14390 has been merged into master branch, we can easily solve the problem with auxiliary_name_scope=False.
How to test

[x] add test case.
[x] pass all tests.

Can one of the admins verify this patch?

This change looks very promising, but let's test it carefully of course.

@tensorflow-jenkins test this please

Can one of the admins verify this patch?

Thank you, @lukaszkaiser @yifeif .

Thanks for your contribution @facaiy!


⚠ GitHub has degraded performance

In bd0e151, GitHub (https://www.githubstatus.com/api/v2/status.json) experienced degraded performance:

HTTP code: 200
Response time: 194 ms

Resolved: GitHub performance has improved in db4ae35.


Incorrect FHIR Query Pattern extensions

The FHIR Query Pattern extension generation logic does not correctly account for additional codeFilter and dateFilter elements.

Included in v1.5.5


Investigate slow tile generation

We've added a bunch of stuff to the tiles, and now they're slower to generate. We need some profiling numbers to target performance improvements, and should instrument some code (against dev) to collect the time spent in the query, filter and post-processing for each layer. Armed with actual data, we can put follow-up stories into the backlog for each problem query / process.

Instrument processing (targeting large DB, e.g. whole of North America)
Evaluate processes, look for edge cases, etc
Create specific new task issues based on that eval

We completed a large chunk of perf work in v0.8 milestone. Taking this out of scope for v0.9

Closing for inactivity / process changed.


No login token for Log In button

I completed the project until the step "Updated loginButtonTapped function in MailViewController.swift". When I hit "Log In", I don't get a token back. Instead I see a message "AADSTS900561: The endpoint only accepts POST, OPTIONS requests. Received a GET request."

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: d523ffb8-7e11-49a7-59d7-9a23e95b6cb4
Version Independent ID: d6c2a28c-539c-71be-ac5c-7ff6c42540e3
Content: How to use Outlook REST APIs in an iOS app - Outlook Developer
Content Source: docs/rest/ios-tutorial.md
Product: outlook
Technology: ms-graph
GitHub Login: @jasonjoh
Microsoft Alias: jasonjoh

Fixed my mistake. Delete the issue.

@tuf65651 what was your mistake?


Custom resource type not found GKE

I'm following the gke example but trying to use it to set secrets instead, however despite creating the types successfully - verified in the console - it always returns with (slightly opaque) error below:
- code: RESOURCE_NOT_FOUND
  message: The type [first-cluster-stitched-cluster-type:/api/v1/namespaces/default/secrets]
    was not found.

This is my manifest:
imports:
- path: cluster.jinja
  name: cluster.jinja
- path: cloudsql.jinja
  name: cloudsql.jinja
- path: support/password.py
  name: password.py
- path: kube-secret.jinja
  name: kube-secret.jinja

resources:
- name: my-cluster
  type: cluster.jinja
  properties:
    zone: europe-west2-a
    nodeCount: 3
  metadata:
    dependsOn:
      - my-db
- name: stitched-db
  type: cloudsql.jinja
  metadata:
    dependsOn:
      - db-password
      - db-user
  properties:
    cloudsql:
      dataDiskSizeGb: 20
      zone: europe-west2-a
      region: europe-west2
      databaseVersion: POSTGRES_11
      tier: db-custom-2-7680
    failover: false
    database:
      name: production
    dbUser:
      name: $(ref.db-user.password)
      password: $(ref.db-password.password)
- name: db-password
  type: password.py
  properties:
    length: 16
    includeSymbols: false
- name: db-user
  type: password.py
  properties:
    length: 8
    includeSymbols: false
- name: db-secret
  type: kube-secret.jinja
  properties:
    clusterName: $(ref.my-cluster.clusterName)
    name: db-secret
    secretValues:
      dbPassword: $(ref.db-password.password)
      dbUsername: $(ref.db-user.password)

This is where the type is created:
{% set CLUSTER_NAME = env['deployment'] + '-' + env['name'] %}
{% set TYPE_NAME = CLUSTER_NAME + '-type' %}
{% set K8S_ENDPOINTS = {'': 'api/v1', '-apps': 'apis/apps/v1beta1', '-v1beta1-extensions': 'apis/extensions/v1beta1'} %}

resources:
- name: {{ CLUSTER_NAME }}
  type: container.v1.cluster
  properties:
    zone: {{ properties["zone"] }}
    cluster:
      initialClusterVersion: "1.13"
      nodePools:
      - initialNodeCount: {{ properties['nodeCount'] }}
        name: pool1
        config:
          oauthScopes:
          - https://www.googleapis.com/auth/compute
          - https://www.googleapis.com/auth/devstorage.read_only
          - https://www.googleapis.com/auth/logging.write
          - https://www.googleapis.com/auth/monitoring

{% for typeSuffix, endpoint in K8S_ENDPOINTS.iteritems() %}
- name: {{ TYPE_NAME }}{{ typeSuffix }}
  type: deploymentmanager.v2beta.typeProvider
  properties:
    options:
      validationOptions:
        # Kubernetes API accepts ints, in fields they annotate with string.
        # This validation will show as warning rather than failure for
        # Deployment Manager.
        # https://github.com/kubernetes/kubernetes/issues/2971
        schemaValidation: IGNORE_WITH_WARNINGS
      # According to kubernetes spec, the path parameter 'name'
      # should be the value inside the metadata field
      # https://github.com/kubernetes/community/blob/master/contributors/devel/api-conventions.md
      # This mapping specifies that
      inputMappings:
      - fieldName: name
        location: PATH
        methodMatch: ^(GET|DELETE|PUT)$
        value: $.ifNull($.resource.properties.metadata.name, $.resource.name)
      - fieldName: metadata.name
        location: BODY
        methodMatch: ^(PUT|POST)$
        value: $.ifNull($.resource.properties.metadata.name, $.resource.name)
      - fieldName: Authorization
        location: HEADER
        value: >
          $.concat("Bearer ", $.googleOauth2AccessToken())
    descriptorUrl: https://$(ref.{{ CLUSTER_NAME }}.endpoint)/swaggerapi/{{ endpoint }}
{% endfor %}

outputs:
{% for typeSuffix, endpoint in K8S_ENDPOINTS.iteritems() %}
- name: clusterType{{ typeSuffix }}
  value: {{ TYPE_NAME }}{{ typeSuffix }}
{% endfor %}
- name: clusterName
  value: $(ref.{{ CLUSTER_NAME }}.name)

and this is the secrets.jinja file:
{% set CLUSTER_TYPE = properties['clusterName'] + '-type' %}

{% set SECRETS_COLLECTION_PREFIX = '/api/v1/namespaces/default/' %}
{% set SECRETS_COLLECTION = SECRETS_COLLECTION_PREFIX + 'secrets' %}

resources:
- name: {{ properties['clusterName'] }}-secrets
  type: {{ CLUSTER_TYPE }}:{{ SECRETS_COLLECTION }}
  properties:
    apiVersion: v1
    kind: Secret
    type: Opaque
    metadata:
      name: {{ properties['name']}}
    stringData:
      {% for key, value in properties['secretValues'].iteritems() %}
      {{ key }}: {{ value }}
      {% endfor %}

Any help is much appreciated, the API is reachable from the gcloud cloud shell console.

I can't delete this, so closing as have just given up and switched to Terraform


Added Quest mods created by MinionSquisher

Notice that Nexus site has barely got any update for LOOT so decided to add some to the master lists:
added 2 quests from the Creator MinionSquisher

Regrets
Lazarus Awakened

Thanks, but we usually don't add metadata to the masterlists that consists solely out of location data in the form of a url.
If we want to add one or more nodes from the following list
group
after
req
inc
tag
msg
dirty
clean

for a plugin then we have meaningful metadata to add for a plugin & then we also require a url.
Is there anything else you would add for the plugins in this PR?

theres nothing else also how would the proper way to PR nexus mods?

want to help out PRing Verified creations as well

We can add new plugin entries to the masterlists, if we want to add meaningful information regarding them. Looking at the list above, that means that one or more of such elements need to be given, so:

a group assignment
information that the plugin in question needs to load after some other plugin
that the plugin in question requires another plugin
that the plugin in question in incompatible to another plugin
Bash Tag assignments
some kind of message
information that a plugin is dirty
.. or clean

If one or more of these information is given, then a new plugin entry can be made. Though as said, we then also require a link to the mod the plugin is originating from.
All of this is how we manage the masterlists generally speaking and it applies to all kind of plugins, independent of the place where they have been uploaded. So Nexus Mods, AFKMods, Creations and other places are also okay, we don't operate exclusively towards one site.
If you haven't already, I would suggest that you take a look at our documentation:

Masterlist Editing
Regex

Those pages are a good start to understand our process of creating and editing the masterlists.

I will

Alright, then I'll close this PR without merging, as it's only intention was to add links to 2 plugins.


Automatic multi-threading of partial_pairwise_align

Hi there,
I've been running paste2 on my server and noticed it automatically uses all the cores on the server when running the partial_pairwise_align function. I looked at the function and can't seem to find a parameter which allows me to change that.
The problem is, i'm running it on quite a busy server and it's creating problems with other users. I was wondering if you plan to make the number of threads/cores parameterised soon?
Best wishes,
Ross

Hi Ross,
This is strange since I didn't write the code to be multi-threading and wasn't aware of any function call that uses multiple cores. I was wondering if it could be from the glmpca subroutine. If you do dissimilarity='pca' is it still using all the cores?


bump crest 1.1.0

crest 1.1.0 added support for custom params encoders. So you can use Crest::NestedParamsEncoder now.
https://github.com/mamantoha/crest/releases/tag/v1.1.0

@mamantoha Thanks!


MORE hot potato discrimination

It says in the Milky Way galaxy.
What if I live in the Cummy Way galaxy?

This is the worst GitHub issue I've ever seen in my life and I've seen the uwu issue on Linux repos

completely true


Add Ahsoka TV Series

https://www.themoviedb.org/tv/114461-ahsoka

Oh thanks! I just added the "Star Wars" tag, I'll make sure that the auto update tonight catches that show.

Hmm I'm not sure why it didn't update, I'm wondering if there is something wrong with the Auto Update workflow.
I went ahead and added it manually though


Fix bug in Scope end bound checking

Everything about this is named wrong.  Closing and trying again…


Switching network

Describe the bug
It always shows switch to network while in internal wallet
Steps to reproduce
Maybe adding an additional connection is better
Spam policy

[X] I verify that this issue is NOT SPAM and understand SPAM issues will be closed and reported to GitHub, resulting in ACCOUNT TERMINATION.

No details provided.


Refactor padding functions

The padLeft and padRight functions were nearly identical and could benefit from being refactored to share common code.
A new function called pad was added that applies padding based on the sign of the amount of padding needed. A positive value applies padding to the right side while a negative value applies padding to the left side.
Issue: #252

@meowgorithm Can I get approval to run the workflows? Thanks.

Done. Nice work on this one.

Can I get someone from @charmbracelet to review this. Thanks.

Good call on consolidating the two functions. If I'm reading this correctly negative integers applied to pad pad the left side, while positive integers pad the right, correct?

If I'm reading this correctly negative integers applied to pad pad the left side, while positive integers pad the right, correct?

Yes, you reading this correctly.
I am doing the conflict resolution now and will have this updated shortly.

Added function comment to explain the relationship between positive and negative values.

@meowgorithm Any improvements or changes you'd like to make?

Looking good, @mikelorant—let's get it done. Thanks for the attention to detail!


Translations update from Weblate

Translations update from Weblate for mattermost-languages-shipped/mattermost-mobile.
Current translation status:

I believe this also needs cherry-picking to the release.

I believe this also needs cherry-picking to the release.


[Question]: The output of AzureFileCopy@5 task was changed, cx would like to know which version did this change

Task name
AzureFileCopy
Task version
5
Environment type (Please select at least one enviroment where you face this issue)

[ ] Self-Hosted
[X] Microsoft Hosted
[ ] VMSS Pool
[ ] Container

Azure DevOps Server type
dev.azure.com (formerly visualstudio.com)
Azure DevOps Server Version (if applicable)
No response
Operation system
Windows
Question
we have a yaml file use AzureFileCopy@5 task. then use the task's output StorageContainerUri and StorageContainerSasToken to next ARM template deploy task. In the ARM template, the output StorageContainerUri and StorageContainerSasToken will be used to construct url to access blobs (StorageContainerUri+StorageContainerSasToken).

In the latest running, the ARM template task failed to access to blob due to shortage of '?' either in StorageContainerUri or StorageContainerSasToken. The constructed url is like below.

https://xx.blob.core.windows.net/xx/xx.jsonsv=2023-08-03&se=2024-01-08T07%3A13%3A55Z&sr=c&sp=rwdl&sig=***Sanitized Azure Storage Account Shared Access Signature

The correct one should be like below.
https://xx.blob.core.windows.net/xx/xx.json?sv=2023-08-03&se=2024-01-08T07%3A13%3A55Z&sr=c&sp=rwdl&sig=***Sanitized Azure Storage Account Shared Access Signature

The last successful run happened on 15th Nov, 2023 and it used below task version.

Task         : Azure file copy
Description  : Copy files to Azure Blob Storage or virtual machines
Version      : 5.230.0
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/deploy/azure-file-copy

The failed run used below task version
Task         : Azure file copy
Description  : Copy files to Azure Blob Storage or virtual machines
Version      : 5.231.2
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/deploy/azure-file-copy

It looks like the '?' mark was removed somewhere between this 2 version.

Please help to check which version did this change and whether the output StorageContainerSasToken without '?' mark is a permanent change.

I found that the task is using New-AzureStorageContainerSASToken to generate token. then https://www.powershellgallery.com/packages/Az.Storage/6.0.0, the '?' was removed from 6.0.0.

please kindly help to check if that is the cause and confirm which Az.Storage version that current task is using.

Just adding that this problem has just started happening (on Friday 22 March 2024) with the AzureFileCopy@4 task. The ? prefix is missing from the StorageContainerSasToken output parameter and this is breaking all ARM templates.
Can anybody confirm that this is a bug that will be fixed?
I can understand behaviour changing between versions, but something like this shouldn't just change for an existing version. I imagine this issue will have a very wide impact, as this task is usually used as the step before ARM Template execution to obtain the blob Sas Token - all of our ARM deployments are now completely broken.

Just adding that this problem has just started happening (on Friday 22 March 2024) with the AzureFileCopy@4 task. The ? prefix is missing from the StorageContainerSasToken output parameter and this is breaking all ARM templates.
Can anybody confirm that this is a bug that will be fixed?
I can understand behaviour changing between versions, but something like this shouldn't just change for an existing version. I imagine this issue will have a very wide impact, as this task is usually used as the step before ARM Template execution to obtain the blob Sas Token - all of our ARM deployments are now completely broken.

Yep, just been hit with this. All our pipelines are now broken.

I have also raised this as an issue in the Visual Studio Developer Community support page as recommended by the documentation.

Adding
${{ if startsWith(variables['AzureFileCopy.StorageContainerSasToken'], '?') }}:
  SasToken: $(AzureFileCopy.StorageContainerSasToken)
${{ else }}:
  SasToken: "?$(AzureFileCopy.StorageContainerSasToken)"

seems like a bit of a hack, but we can't trust the values coming from the AzureFileCopy task anymore.


Application Check Page

Check out branch: new-check-page
Acceptance Criteria

[ ] should look like this (when letter successfully uploaded):

[ ] should look like this (when letter not yet uploaded but needed):

@tom95 pls add branch where you stored your changes

@hendraet why is there an option to upload a letter if it's not needed (example 2)

Well...

@hpi-swt2/product-owner-2016 there is now a version in the code, that displays "Datei hochgeladen" and not "Einverständniserklärung hochgeladen am ..." which one is wanted by you?


it's not saving the screenshot image

it's not saving the screenshot image in Version 66.0.3359.181 (Official Build) (64-bit)

mine doesn't either, when you click on visible screenshot it just shows some black bars and doesn't do anything


pichunter ripper

Ripme version: 1.6.11
Java version: 1.8.0_151
Operating system: Win10

https://github.com/RipMeApp/ripme/issues/38#issuecomment-344999723
Sorry, forgot to clarify what under tags/models/sites just list of photosets.
For example: http://www.pichunter.com/tags/Sex the first photoset will be http://www.pichunter.com/gallery/3270642/Its_not_only_those_who#11
We need to download all of the photosets under tag/Sex instead of only one photo.
And adding the download option from pichunter.com/gallery will be great.

I second the ability to rip from /gallery urls

I made a PR that adds support for galleries, it should be in the 1.6.12 release.

@cyian-1756, something went wrong.
Ripme 1.6.12
ripme.log

@handsomebananas It's looks like I messed up the regex a bit. You can rip some urls by removing everything after the #

This is fixed in 1.6.13


Fix issue 914

Fixes #914
Possible esthetical improvement:  put a half transparent layer over the table being loaded in the background to make the end user aware of the loading process.
@mietcls I have added a sample overlay just within that
, called

. See assets/css/overlay.scss. Be sure to run npx mix --production, or run with reflex.

Two things:

The class "htmx-request" is not consistent with other behaviour. Can add the class "show" instead of "htmx-request" to be consistent with the modal component and be more descriptive?
To make sure we can reuse this component in other places: spinner and its backdrop are applied on card level now – not on table level. So it can be moved up in the "publications.gohtml" if needed.

@mietcls looks nice and clean ;-)

Important note: because we use the same underlying code (publication_index.go) to fetch facet values, the list of years in the select boxes cannot be filtered, because the option min_doc_count in the facet definition of year is set to 0.


Empty cache files causes "KillPodSandbox" errors when deleting pods

Description
I originally reported this issue at https://github.com/aws/amazon-vpc-cni-k8s/issues/2283, but I was asked to post the issue here instead.
I'm running into a problem where a node will occasionally have a bunch of pods stuck in the Terminating state because the pod's sandbox can't be deleted:
Pod event:
Warning  FailedKillPod   43s (x229 over 49m)  kubelet  error killing pod: failed to "KillPodSandbox" for "a7ea2280-7361-4329-8566-89975485b20e" with KillPodSandboxError: "rpc error: code = Unknown desc = failed to destroy network for sandbox \"4285ab7ef1b33097068f6e2dfbf2a71c96129f45d0d6655d47d3dc80db5f0399\": failed to get network \"aws-cni\" cached result: decoding version from network config: unexpected end of JSON input"

On the containerd side, I see similar logs.  First, the reason given is cni plugin not initialized:
time="2023-03-01T20:08:10.167194705Z" level=info msg="StopPodSandbox for \"4285ab7ef1b33097068f6e2dfbf2a71c96129f45d0d6655d47d3dc80db5f0399\""
time="2023-03-01T20:08:10.167346791Z" level=error msg="StopPodSandbox for \"4285ab7ef1b33097068f6e2dfbf2a71c96129f45d0d6655d47d3dc80db5f0399\" failed" error="failed to destroy network for sandbox \"4285ab7ef1b33097068f6e2dfbf2a71c96129f45d0d6655d47d3dc80db5f0399\": cni plugin not initialized"

This repeats a few times, then the error message changes to unexpected end of JSON input
time="2023-03-01T20:08:34.067124024Z" level=error msg="StopPodSandbox for \"4285ab7ef1b33097068f6e2dfbf2a71c96129f45d0d6655d47d3dc80db5f0399\" failed" error="failed to destroy network for sandbox \"4285ab7ef1b33097068f6e2dfbf2a71c96129f45d0d6655d47d3dc80db5f0399\": failed to get network \"aws-cni\" cached result: decoding version from network config: unexpected end of JSON input"

On the host itself, I noticed that the /var/lib/cni/results directory contains a cache file for the container with the ID 4285ab7ef1b33097068f6e2dfbf2a71c96129f45d0d6655d47d3dc80db5f0399 that's completely empty (zero bytes):
bash-5.1# ls -alh /var/lib/cni/results | grep 4285ab7ef1b33097068f6e2dfbf2a71c96129f45d0d6655d47d3dc80db5f0399
-rw-------. 1 root root    0 Mar 1 06:31 aws-cni-4285ab7ef1b33097068f6e2dfbf2a71c96129f45d0d6655d47d3dc80db5f0399-eth0

Meanwhile, the other aws-cni-*-eth0 files are not empty, and contain valid JSON.
So I believe the unexpected end of JSON input error message is caused by containerd attempting to json.Unmarshal an empty file, which I believe is happening here:
https://github.com/containerd/containerd/blob/a217b5ac8f66cfdbffd730f16b63f13e313013b8/vendor/github.com/containernetworking/cni/pkg/types/create/create.go#L31-L34
I am able to temporarily resolve the issue by deleting the empty cache file.  The stuck pods are instantly able to be deleted as soon as I do that.
Another note: it looks like most of the caching logic is actually handled in https://github.com/containernetworking/cni, so perhaps this particular issue needs to be raised upstream.  I'm not too sure.

My theory is that some other issue is causing the AWS VPC CNI plugin to crash or experience some issues, and containerd is mistakenly writing an empty cache file to /var/lib/cni/results while this is happening.  Then, when the AWS VPC CNI plugin starts working again, this empty cache file is read, causing the error I'm seeing.
Steps to reproduce the issue
I wish I was able to reliably reproduce the underlying issue.  The best I've been able to do is to delete the contents of one of the cache files within /var/lib/cni/results, which will reproduce the unexpected end of JSON input error message.  However, I have been unable to reproduce the issue that causes the empty cache file to exist in the first place.
Describe the results you received and expected
I would expect for an empty cache file to never be written to /var/lib/cni/results.  Or, if this does happen, perhaps containerd could catch this and remove the bad cache file before attempting to read from it.
What version of containerd are you using?
ctr github.com/containerd/containerd 1.6.15+bottlerocket

Any other relevant information
bash-5.1# runc --version
runc version 1.1.4+bottlerocket
commit: 5fd4c4d144137e991c4acebb2146ab1483a97925
spec: 1.0.2-dev
go: go1.19.4
libseccomp: 2.5.4

bash-5.1# uname -a
Linux ip-10-0-143-2.us-west-2.compute.internal 5.15.79 #1 SMP Wed Jan 25 03:35:23 UTC 2023 aarch64 GNU/Linux

bash-5.1# cat /etc/os-release
NAME=Bottlerocket
ID=bottlerocket
VERSION="1.12.0 (aws-k8s-1.24)"
PRETTY_NAME="Bottlerocket OS 1.12.0 (aws-k8s-1.24)"
VARIANT_ID=aws-k8s-1.24
VERSION_ID=1.12.0
BUILD_ID=6ef1139f
HOME_URL="https://github.com/bottlerocket-os/bottlerocket"
SUPPORT_URL="https://github.com/bottlerocket-os/bottlerocket/discussions"
BUG_REPORT_URL="https://github.com/bottlerocket-os/bottlerocket/issues"

CNI Plugin is https://github.com/aws/amazon-vpc-cni-k8s v1.12.0
Kubernetes version:
Client Version: version.Info{Major:"1", Minor:"25", GitVersion:"v1.25.4", GitCommit:"872a965c6c6526caa949f0c6ac028ef7aff3fb78", GitTreeState:"clean", BuildDate:"2022-11-09T13:36:36Z", GoVersion:"go1.19.3", Compiler:"gc", Platform:"darwin/arm64"}
Kustomize Version: v4.5.7
Server Version: version.Info{Major:"1", Minor:"24+", GitVersion:"v1.24.10-eks-48e63af", GitCommit:"9176fb99b52f8d5ff73d67fea27f3a638f679f8a", GitTreeState:"clean", BuildDate:"2023-01-24T19:17:48Z", GoVersion:"go1.19.5", Compiler:"gc", Platform:"linux/amd64"}

Show configuration if it is related to CRI plugin.
version = 2
root = "/var/lib/containerd"
state = "/run/containerd"
disabled_plugins = [
    "io.containerd.internal.v1.opt",
    "io.containerd.snapshotter.v1.aufs",
    "io.containerd.snapshotter.v1.devmapper",
    "io.containerd.snapshotter.v1.native",
    "io.containerd.snapshotter.v1.zfs",
]

[grpc]
address = "/run/containerd/containerd.sock"

[plugins."io.containerd.grpc.v1.cri"]
enable_selinux = true
# Pause container image is specified here, shares the same image as kubelet's pod-infra-container-image
sandbox_image = "602401143452.dkr.ecr.us-west-2.amazonaws.com/eks/pause:3.1-eksbuild.1"

[plugins."io.containerd.grpc.v1.cri".containerd]
default_runtime_name = "shimpei"

[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.shimpei]
runtime_type = "io.containerd.runc.v2"
base_runtime_spec = "/etc/containerd/cri-base.json"

[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.shimpei.options]
SystemdCgroup = true
BinaryName = "shimpei"

[plugins."io.containerd.grpc.v1.cri".cni]
bin_dir = "/opt/cni/bin"
conf_dir = "/etc/cni/net.d"

[plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
endpoint = ["https://docker-registry.redacted.com"]

hi @mrparkers , I think I have the exact same thing as you. I have all the same versions and everything and am even using a registry mirror.
I noticed that the empty aws-cni file seems to have a creation time that seems coincident with a host restart ? (16:36:15 below) . I was wondering if you see the same thing?
bash-5.1# ls -la --time-style=full-iso /var/lib/cni/results/
total 16
drwx------. 2 root root 4096 2023-03-07 05:07:37.644843952 +0000 .
drwx------. 3 root root 4096 2023-03-05 16:12:59.115640029 +0000 ..
-rw-------. 1 root root 2289 2023-03-05 16:37:32.858972330 +0000 aws-cni-aae559259afb19d5086194c3e0fd58fd71a67619a285bf10c33a2d1ceccfb809-eth0
-rw-------. 1 root root    0 2023-03-05 16:36:15.930198021 +0000 aws-cni-b10821e595aef417667637f481b45b5d0183fbc2f6341f53a0e5059d506eaaf5-eth0

bash-5.1# journalctl --list-boots
 -1 3a275f535dc245a6ab9f1a0758dc90d8 Sun 2023-03-05 16:12:30 UTC—Sun 2023-03-05 16:35:46 UTC
  0 d940a617b7c74bcd9676d2ffd02679c0 Sun 2023-03-05 16:37:15 UTC—Tue 2023-03-07 06:11:56 UTC

The previous boot logs run out at 16:35:46 , and then the new boot log starts at 16:37:15.
Somehow the cni results file is created in between that. It feels like a new issue in 1.24 to me, I don't think this happened in 1.23. Interested if you see the same thing.

Hi @gbucknel, thanks for sharing your experience here.  The information you provided is really useful, I didn't think to check the boot logs when I was debugging this last week.  However, this was one of the events on the node that I was troubleshooting:
Warning  Rebooted                 23m                   kubelet     Node ip-10-0-159-84.us-west-2.compute.internal has been rebooted, boot id: c492843a-e20f-4f07-a811-cba4401eed96

So it's possible that this was caused by a reboot of the node too, although I don't have access to the boot logs of this node to confirm (the node is long gone).
Unfortunately, I have been unable to reproduce this manually, but the next time this happens I'll check those boot logs and respond here.
Are you also running Bottlerocket by chance?

hi @mrparkers , yeah we're on Bottlerocket. It almost looks like we're on the same cluster! ;)
Just for completeness :
bash-5.1# runc --version
runc version 1.1.4+bottlerocket
commit: 5fd4c4d144137e991c4acebb2146ab1483a97925
spec: 1.0.2-dev
go: go1.19.4
libseccomp: 2.5.4

bash-5.1# uname -a
Linux <IP_ADDRESS> 5.15.79 #1 SMP Wed Jan 25 03:35:24 UTC 2023 x86_64 GNU/Linux
bash-5.1#

bash-5.1# cat /etc/os-release
NAME=Bottlerocket
ID=bottlerocket
VERSION="1.12.0 (aws-k8s-1.24)"
PRETTY_NAME="Bottlerocket OS 1.12.0 (aws-k8s-1.24)"
VARIANT_ID=aws-k8s-1.24
VERSION_ID=1.12.0
BUILD_ID=6ef1139f
HOME_URL="https://github.com/bottlerocket-os/bottlerocket"
SUPPORT_URL="https://github.com/bottlerocket-os/bottlerocket/discussions"
BUG_REPORT_URL="https://github.com/bottlerocket-os/bottlerocket/issues"

The thing I changed today was the CNI image, I was on 1.12.0 like you but updated one of my clusters to 1.12.5 today. It seems like some sort of race condition thing to me given it seems to happen during boot?
These two PRs in 1.12.1 look like they are related to initialisation (if you squint hard enough! ) , so thought it was worth a try.
https://github.com/aws/amazon-vpc-cni-k8s/pull/2138
https://github.com/aws/amazon-vpc-cni-k8s/pull/2145
I'll write back once I've tested it some more.

hi @jdn5126 cool, yes, I still see if with cni 1.12.5 . Is there any more info I can gather around this to help ?

xref: https://github.com/containernetworking/cni/issues/1055

xref: https://github.com/k3s-io/k3s/issues/6185

Is this issue solved by https://github.com/containernetworking/cni/pull/1072?  If so, can we rev up libcni for testing on the next release?  @dims for feedback.

@ryan-beisner the fix from the CNI lib was pulled into main by https://github.com/containerd/containerd/commit/025d1161e8f8986c7f1265d957477e1bc8cf9078


Update Metis calls to 5.x in load balancers

Original issue: https://charm.cs.illinois.edu/redmine/issues/423

Metis's stable version is 5.x, while our load balancers use 4.x. Also, if possible, remove extern based use of Metis functions in some LBs. Extern prevents compiler checking, and hence an API mismatch is not detected.

Original date: 2014-02-22 17:44:48

Marking it 6.6 but not sure.

Original date: 2014-04-10 22:45:58

My understanding of this issue's present status is that a few LB strategies have been updated to the METIS 5.x API, but several others have not. Is that correct?

Original date: 2014-04-10 22:48:31

Thats right. Key ones which are used have been converted to Metis 5.x whereas others will work with older version on Metis.


Windows 10 Command Prompt support?

It doesn't work in Windows 10 Command Prompt without first adding the following:
import os
os.system('')

Is that the expected behavior?  If so, can something like the above be added to the library?

Unfortunately, I have no experience with Windows. Basically all that is needed is a terminal with standard ANSI escape code support. I had also noticed this hint in the original chalk documentation. It is quite weird though that running an empty os.system (what is that launching actually?) can have such a side effect. I'm afraid I can be of much help in this case, because I don't have access to Windows, and this sounds like an idiosyncrasy of that terminal.

Yeah, this is a really wicked thing, that can't be explained without diving into the source code of that (Windows) python library. You can look at some related technical discussion about UTF-8 with and without BOM, and how it affects the Powershell's [native vs Core] differently and does weird stuff to the terminal, depending...
See:

https://github.com/PowerShell/PowerShell/issues/7233
https://github.com/python/cpython/blob/99fcf1505218464c489d419d4500f126b6d6dc28/Modules/clinic/posixmodule.c.h#L1733


Static analysis output file extension is  filename.nativecodeanalysis.xml

There is a typo on the page, it says static analysis output file extension is filename.nativeanalysis.xml but actually it is  filename.nativecodeanalysis.xml

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 09296a0d-dea6-bcf6-7370-5ec2cf061835
Version Independent ID: b6de3061-2d20-4508-6a76-663a2d61d098
Content: /analyze (Code analysis)
Content Source: docs/build/reference/analyze-code-analysis.md
Product: visual-cpp
Technology: cpp-tools
GitHub Login: @corob-msft
Microsoft Alias: corob

A fix should appear tomorrow.


refactor: image release

1. Does this PR affect any open issues?(Y/N) and add issue references (e.g. "fix #123", "re #123".):

[ ] N
[x] Y

re https://github.com/kcl-lang/kcl/issues/1669

Pull Request Test Coverage Report for Build<PHONE_NUMBER>6
Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 14.63%

Totals

Change from base Build<PHONE_NUMBER>5:
0.0%

Covered Lines:
91

Relevant Lines:
622

💛  - Coveralls


ADBCJ: Connection Pool: Reuse same connection for different users

From discussion:
@fooling:

Lately I got an idea that maybe different users can share one connection.
For example ,
We have a connection pool.
Then two users A and B want to use that pool.
A : sql1  sql2
B: sql3  sql4
They just need get a "connection" and execute sqls with "execute()" then they can get a Future which contains the result.
So , it doesn't matter whether the sqls are executed on the same connection or not !
The Future matters.
Then, we can design this pool, that the connections can be reused.
Because with the number of application servers growing, the number of connection lead to a great pressure to the database.And in this way, it could be possible to reduce the connection number.
I figured a way in which maybe a queue could be used to store the running tasks on every connection.When result received, it would be added to the oldest task , and return to the specified Future.
When transactions happens, that connection would be taken over by the user unitl the transaction ends,Other user could continue using their "connection" because the "connection" is not bind with fd.
What's your opinion?

@gamlerhart :

Just let me repeat your suggesting, to check if I understand it right:
You have many different db-users in your application, right. So, instead of creating a ''ConnectionManager' per user, you want to create one connection provider for a database. And then create a Connection per user.
And, now, when the connection pool is active, you actually can share a connection between users. So, before a statement is sent to the database, the right user is selected.
A great idea. =)
Only one concern: As soon as you start a transaction, we'll need to bind that connection to the pooled one.
So, the basic idea would be something like this:

Allow ADBCJ API to create a ConnectionManager just for a database, and then create connections per user.
Improve the ADBCJ connection pool to 'reuse' a non-transaction connection for different users.
When a 'transaction' is started, bind the 'native' connection to the 'pooled' connection.

I'll take a look. You started to implement this? Or should I take a look at it, since your busy with DBCJ?

Closed, stagle