Updating Connection to Local Services for Android

I am submitting an update for a non-trivial example of how SSL/TLS override should be handled for an Android application within Xamarin.Forms, rather than the developer performing an HTTP Cleartext override. Please feel free to submit suggestions for changes before final approval. Thanks!

Microsoft support for Xamarin will end on May 1, 2024 for all Xamarin SDKs including Xamarin.Forms. In preparation for this, all issues and PRs in this repository are being closed.
.NET MAUI is the replacement for Xamarin.Forms, and its documentation can be found here.


Update development guide to recommend ko v0.5.0

I recently tried to deploy pipelines at head with ko v0.4.0, and it failed to deploy. Instead, it preserved the "github.com/tektoncd/..." import paths, which when deployed failed to start.
Upgrading ko to the latest version (in my case v0.5.1) fixed the problem. Earlier versions of ko may work, but users will likely install at least this version on update.
/kind documentation

Changes

Submitter Checklist
These are the criteria that every PR should meet, please check them off as you
review them:

[ ] Includes tests (if functionality changed/added)
[x] Includes docs (if user facing)
[ ] Commit messages follow commit message best practices

See the contribution guide for more details.
Double check this list of stuff that's easy to miss:

If you are adding a new binary/image to the cmd dir, please update
the release Task to build and release this image.

Reviewer Notes
If API changes are included, additive changes must be approved by at least two OWNERS and backwards incompatible changes must be approved by more than 50% of the OWNERS, and they must first be added in a backwards compatible way.
Release Notes

/lgtm


Blog Page

Blog Feature

have title
have body
have description
have topic
have category
have owner
have comment
have clap
have pin starts ( only for admins )
can be reported
can be deleted ( only for admins )

Blog Topics

Blog
Project
Resources

We need something similar like this design

making this rq

the blog could have a markdown and we can use built in api to parse as well pkgs to do all the heavy lifting intead of relaying on the servers api


Handle restoring last closed tab

When a floating tab is closed, and the user presses Ctrl+Shift+T (Ctrl+Shift+N on Firefox) to restore the last closed tab, the floating tab reappears (as a popup), but it's not always on top and TabFloater doesn't know it is restored.
Catch the tab restore event (if possible), and either:

Enable the floating state
or convert the floating tab straight into a normal tab

For this, we'd need to somehow identify if the tab that's being restored is the same floating tab that was closed last time. This is not straightforward, as the tab IDs are different. I suppose it would be possible to compare window types and URLs, but it quickly becomes too complex. It also needs a lot of extra events, adding performance overhead for every tab open operation.
Closing as won't fix.


Workaround due to lack of IN filtering in PgVector?

Hi 👋
The chain I want to create is:
I perform a query on an LLM -> I extract some values from the output of the LLM (the output is an array of strings) -> These values will help me filter a VectorDB (specifically PgVector), but I need to perform a search in the VectorDB with the same input query that was passed to the LLM for each value extracted from the output of the LLM (remember that it is an array of strings) as filter for let's say the metadata field "project" -> I have a method that joins together in a specific way the N calls to the VectorDB with M documents and returns a string that will be used as a context for an LLM -> The LLM receives the context and the initial query and outputs an output.
I have thought about using a RunnableSequence. The problem is that I don't know if it is possible to assign the strings within the array extracted from the output of the first call to the LLM as a VectorDB/retriever filter.
Once I have obtained the N responses with M VectorDB documents, I would like to reorder them using the score and keep a maximum of X documents.
I have to do this workaround because the current implementation of PgVector does not allow to filter by metadata by using IN on a list of values.
Thanks in advance 😄

@dosu-bot
VectorDBQAChain is deprecated. What other approach can I use?

@dosu-bot
Is there any way to change retriever (vectordb.asRetriever()) parameters in runtime? For example, change the filter or any of the values within the filter.

@dosu-bot
Can I access a distance/score/similarity value in the returned documents by the retriever if I do retriever.map()?

There's a PR to implement this feature: #3745 . @bracesproul has this assigned to himself, maybe this can be completed, so you won't need a workaround.
Possible Workaround: The VercelPostgres vector store integration does have IN-filtering. As far as I can tell there's nothing stopping us from providing a standard pool instance from node-postgres. By installing the @vercel/postgres package and doing some "convincing", typescript should allow you to pass a pg Pool as the pool parameter in the config. I don't see anything in the implementation of the VercelPostgres vector store that suggests that it wouldn't work with a normal pool connecting to a PGVector database. I tried locally and successfully saved some embeddings to a pgvector db running in docker using the VercelPostgres vector store. This way you could use IN-filtering, although it is of course not ideal long-term, but might help until the implementation for PGVector is merged.
Cheers

Merged in #3745


Skip download of dataset if raw data exists

🚀 Feature

Skip the download of raw dataset if exists and just extract and convert.
Motivation
Reduce the download of datasets as possible.

Pitch
This will not be a new feature nor a big change.
Currently, some torchvision datasets download the raw datasets (e.g. MNIST) before extract and convert them to appropriate formats, e.g. (train/test).pt for MNIST dataset if download argument is True.
I misunderstood this option that if there exists a raw dataset, torchvision skips the download of raw data and only extracts and converts the raw dataset. But, actually, the criterion of skipping the download process is whether or not there exist the PyTorch files as follows.
https://github.com/pytorch/vision/blob/240792d446cfa65141a76b4bac0b0ecbf15aacc0/torchvision/datasets/mnist.py#L134-L138
And the below diff is the sketch of my proposal (not thoroughly validated).
diff --git a/torchvision/datasets/mnist.py b/torchvision/datasets/mnist.py
index e87cd46e..bfd59914 100644
--- a/torchvision/datasets/mnist.py
+++ b/torchvision/datasets/mnist.py
@@ -131,6 +131,11 @@ class MNIST(VisionDataset):
     def class_to_idx(self) -> Dict[str, int]:
         return {_class: i for i, _class in enumerate(self.classes)}

+    def _check_raw_data_exists(self) -> bool:
+        return all([
+            os.path.exists(os.path.join(self.raw_folder, filename)) for (filename, _) in self.resources
+        ])
+
     def _check_exists(self) -> bool:
         return (os.path.exists(os.path.join(self.processed_folder,
                                             self.training_file)) and
@@ -140,34 +145,36 @@ class MNIST(VisionDataset):
     def download(self) -> None:
         """Download the MNIST data if it doesn't exist in processed_folder already."""

+        if not self._check_raw_data_exists():
+            print('Dowanloading raw MNIST...')
+            os.makedirs(self.raw_folder, exist_ok=True)
+            os.makedirs(self.processed_folder, exist_ok=True)
+
+            # download files
+            for filename, md5 in self.resources:
+                for mirror in self.mirrors:
+                    url = "{}{}".format(mirror, filename)
+                    try:
+                        print("Downloading {}".format(url))
+                        download_and_extract_archive(
+                            url, download_root=self.raw_folder,
+                            filename=filename,
+                            md5=md5
+                        )
+                    except URLError as error:
+                        print(
+                            "Failed to download (trying next):\n{}".format(error)
+                        )
+                        continue
+                    finally:
+                        print()
+                    break
+                else:
+                    raise RuntimeError("Error downloading {}".format(filename))
+
         if self._check_exists():
             return

-        os.makedirs(self.raw_folder, exist_ok=True)
-        os.makedirs(self.processed_folder, exist_ok=True)
-
-        # download files
-        for filename, md5 in self.resources:
-            for mirror in self.mirrors:
-                url = "{}{}".format(mirror, filename)
-                try:
-                    print("Downloading {}".format(url))
-                    download_and_extract_archive(
-                        url, download_root=self.raw_folder,
-                        filename=filename,
-                        md5=md5
-                    )
-                except URLError as error:
-                    print(
-                        "Failed to download (trying next):\n{}".format(error)
-                    )
-                    continue
-                finally:
-                    print()
-                break
-            else:
-                raise RuntimeError("Error downloading {}".format(filename))
-
         # process and save as torch files
         print('Processing...')

Alternatives

Additional context

cc @pmeier

Hey @crcrpar, we are currently working on removing the caching in our custom binaries all together. See #3420 for details.

Oh, that is what exactly I want and +alpha. Thank you for sharing the information!


upgrade is_record_type to use TypeGuard

Would it be possible to utilise TypeGuard on this function in order to leverage the return value at the callsite for static type checking?
https://github.com/MarshalX/atproto/blob/d5e2293848a3506a1b13144062c24bae1cc10f2c/atproto/xrpc_client/models/utils.py#L202
I'd do it myself but I don't feel I understand enough about the details of the library at this point to get it right

sorry for the long response. I'll take a look after releasing #127

Making a note so nobody else falls over the same problem, hopefully:
I spent a while banging my head against the wall on this one only to find that there was a bug in pyright which only recently got fixed.
Specifically:

Bug Fix: Fixed several issues with logic that performs protocol matching against a module, including a false positive error when matching against a generic protocol.

Here's my current (incomplete) approach, if anyone's interested:
from typing import TypeVar, Protocol, Any, Type
from typing_extensions import TypeGuard
from atproto.xrpc_client.models.base import ModelBase
from atproto.xrpc_client.models.utils import is_record_type as original_is_record_type
from atproto.xrpc_client import models

T = TypeVar('T', bound=ModelBase)

class HasAMainModel(Protocol[T]):
    Main: Type[T]

def is_record_type(record: Any, model: HasAMainModel[T]) -> TypeGuard[T]:
    return original_is_record_type(record, model)  # type: ignore

def test(x: Any):
    if is_record_type(x, models.AppBskyEmbedImages):
        reveal_type(x) # correctly identified as atproto.xrpc_client.models.app.bsky.embed.images.Main

Also need to consider that this actually isn't quite the same as an is_instance check so we should be careful not to assert too much about the type via the TypeGuard.


ENH: IRTAM coefficient download

Description
Addresses #18 by adding a function to download the IRTAM coefficients.  Also cleans up some of the main library functions by:

Making use of subdirectories optional
Creating a function for naming files, to reduce duplicate code
Removing unused inputs for functions

Updates the documentation to include downloading coefficients in the example.
Type of change
Please delete options that are not relevant.

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality
to not work as expected)
This change requires a documentation update

How Has This Been Tested?
Added unit tests for the new functions.  @victoriyaforsythe please run the example in the documents and ensure it still works for you.
Test Configuration:

Operating system: OS X Big Sur
Version number: Python 3.9
Any details about your local setup that are relevant

Checklist:

[x] Make sure you are merging into the develop (not main) branch
[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[x] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes
[x] Any dependent changes have been merged and published in downstream modules
[x] Add a note to CHANGELOG.md, summarizing the changes
[x] Update zenodo.json file for new code contributors

Recorded documentation failure in #20


Change vault-template to work with vault API v2

Features

Modify vault-template to support vault API v2.
Add an option to override the default go template delimiters, in case you wish to render a file which already uses another templating language. (e.g ansible inventory file).

@nikita698 Thank you for your contribution.
We decided to close the PR as we are currently not using the vault kv engine in V2 and your changes would break our CD.
If we at some point migrate to the newer version we would reopen your PR.


runtime: TestLibraryCtrlHandler fails occasionally with register ABI enabled

Found a failure on the windows-amd64-regabi bot:
--- FAIL: TestLibraryCtrlHandler (6.32s)
    signal_windows_test.go:205: Program exited with error: exit status 1
        FAILURE: No signal received
FAIL
FAIL	runtime	52.447s

Seems like a flake, but we should look into it.

Looking at the build dashboard logs, it doesn't seem like this is regabi related. There's one other failure:
2021-04-16T03:16:11-cf2396c/windows-amd64-2012:
--- FAIL: TestLibraryCtrlHandler (6.17s)
    signal_windows_test.go:205: Program exited with error: exit status 1
        FAILURE: No signal received
FAIL
FAIL    runtime

The fact that there are no other failures suggests that the CL @aclements mentioned is probably the culprit (https://golang.org/cl/309632).
By the way, it's harder to reproduce than I thought.
Background on the test:

It's a C program that links up to a Go DLL, but it doesn't actually call anything in the DLL.
It waits to get a control signal and fails if it doesn't get one in time.
It expects the control signal to be produced externally, so the actual Go test (in signal_windows_test.go) sends the signal.

Here's my working theory: LoadLibrary (in the C code) returns before the runtime is fully initialized, signaling the Go test to send its signal. Note that rt0_amd64_lib (which is used to initialize a c-shared library) spawns a thread to finish initialization asynchronously. Basically, the signalling in the test was always racy with initialization and setup of the control handler.As of https://golang.org/cl/309632, the control handler is now set up slightly later in the initialization process and it takes longer to get there too, because we now call compileCallback which is non-trivial.
As a result, the test now flakes a little bit. I will try to confirm this by adding further delay in initialization to see if the test failure reproduces more easily.

I don't think my working theory is far off from the problem, but I don't fully understand anymore why the race pertains to this test. This test just makes sure that the C process receives the signal on its handler. It doesn't care about the Go handler at all.

Print debugging isn't working because the initialization code doesn't seem to have access to stdout (or the equivalent) on Windows.
I just tried switching the timeout in the test's C code from 5 seconds to INFINITE to see if it hangs and maybe I can get it into a debuggable state.
Unfortunately when I did that I came back to my terminal filled with binary garbage followed by "fail" reported by the parent process. I suspect this problem goes a little deeper than I realize...

For a taste, my terminal looks something like this:
        $)euz���
        JZ<���������Xn��r�����������Xn��w|�����
        _r���������Xnz������������XnAM�����������Xn�����������Xn��������������Xn^ez�1o7'�"NW|�8C���@C�]���������-Xn"(H���������/Xnjno����������/Xnuyz����������/Xn
*3I^hy�����������@1Xn�*3IQhy�������� 5Xn *16T\]�������� 5Xn *16���������5Xn7Fadgw���������������5Xndgw������������6Xn8��������@7Xn\e�����������9Xn$)FMk�����������`:Xn(*��������`:Xn=Hknq����������������`:Xn������������`:Xn�����������`:Xn!��Unq�����/9J��������`:Xn����
        9J��������`:Xnnq����������`:X]Tf��������`:Xn	59��������`:Xnn�JT��������`:Xnos����������`:Xn������������� >Xn8PS����������� >Xn{���������������?Xn*0;=��������?Xn@ELQe�������������?XnLQe�������������?Xnhq{���������������?Xn04Qp���#&O��������?Xn04�����������#&+/2EO��������?Xn��#&��������?Xn������������`CXnr����IL����r���������`CXn"R]r��������`CXn9MOU��������`CXn�������������`CXn,CFILSm�������������������@LXnfk���!L��������@LXn,�����������@LXn{�����������@LXn����!L��������@LXn%,���m���������PXn��>A��m���������PXn�����������������PXn7HJS���������PXn^oqz���������PXnkm�>As���������PXn���������������PXn���������������PXn9<@WZ]`h��������������������WXn^boty����������WXnboty���������WXnkoty��������@ZXn-=�����������^Xn�������������_Xn����/F���������_Xn%���������gXn4=Klny�����������gXnlny����������� iXnLOWa���u���������mXndlpx}��NVw���KT����'S�������������mXnpx��%*IV^cw
--- FAIL: TestLibraryCtrlHandler (15.03s)
FAIL
FAIL	runtime	306.632s
FAIL
Error running run: exit status 1

except the garbage extends far beyond what my scrollback buffer can carry.

Something is very broken here. This kind of reads like something is trying to print (maybe the printf("ready\n")?) and it's memory corruption leading to a buffer overflow. There aren't very many prints at all in all these code paths. In fact, that may be the only one.

I tried to reproduce this issue on my computer, but I could not.
Building programs (Go and especially C) is slow on Windows, so I modified this test to build everything once and then run executable in a loop.
diff --git a/src/runtime/signal_windows_test.go b/src/runtime/signal_windows_test.go
index d0f61fb6c2..ecacafd9d2 100644
--- a/src/runtime/signal_windows_test.go
+++ b/src/runtime/signal_windows_test.go
@@ -170,8 +170,14 @@ func TestLibraryCtrlHandler(t *testing.T) {
                t.Fatalf("failed to build c exe: %s\n%s", err, out)
        }

+       for i := 0; i < 10000; i++ {
+               testLibraryCtrlHandler(t, exe)
+       }
+}
+
+func testLibraryCtrlHandler(t *testing.T, exe string) {
        // run test program
-       cmd = exec.Command(exe)
+       cmd := exec.Command(exe)
        var stderr bytes.Buffer
        cmd.Stderr = &stderr
        outPipe, err := cmd.StdoutPipe()

And my test succeeds
c:\Users\Alex\dev\go\src>go test -count=1 -run=CtrlH runtime
ok      runtime 222.704s

c:\Users\Alex\dev\go\src>

Alex

@alexbrainman FWIW my -count to the test binary was at about 1000 (and even then it only failed roughly every other time, so maybe once in 2000, 2500 runs?), so it's not easy to reproduce but it is possible. Out of curiosity, what happens if you change the WaitForSingleObject from 5 seconds to INFINITE on your machine?

Ah, nevermind, I did not look closely enough at your code. My bad!
Huh, what version of Windows are you running? Maybe it's a bug specific to the windows-amd64-2016 builders?

3 consecutive runs with @alexbrainman's patch from https://github.com/golang/go/issues/45638#issuecomment-826046034 came up clean. That's... interesting.

Wow, I spoke too soon. It literally JUST failed. Once out 30000 runs, roughly.

Interestingly, this is how it failed:
=== RUN   TestLibraryCtrlHandler
    signal_windows_test.go:210: Program exited with error: exit status 1
        runtime: signal received on thread not created by Go.
        �o�<fN9꙾�&����vl��3U0�{\K���X�س���64oMˡ1��!
                                                    R��n��)�Џ�jG'��T��-�0�i�
                                                                            �!Ρ6�)��,p??�G�j�V���k6˩���G
                                                                                �
                                                                                �6�runtime: signal received on thread not created by Go.
        �o�<fN9꙾�&����vl��3U0�{\K���X�س���64oMˡ1��!
                                                    R��n��)�Џ�jG'��T��-�0�i�
                                                                            �!Ρ6�)��,p??�G�j�V���k6˩���G
                                                                                �
                                                                                �6�h�h�runtime: signal received on thread not created by Go.
        �o�<fN9꙾�&����vl��3U0�{\K���X�س���64oMˡ1��!
                                                    R��n��)�Џ�jG'��T��-�0�i�
                                                                            �!Ρ6�)��,p??�G�j�V���k6˩���G
                                                                                �
                                                                                �6�h�h�d@�?@DDD �"""C:\Windows\system32\runtime: signal received on thread not created by Go.
        �o�<fN9꙾�&����vl��3U0�{\K���X�س���64oMˡ1��!
                                                    R��n��)�Џ�jG'��T��-�0�i�
                                                                            �!Ρ6�)��,p??�G�j�V���k6˩���G
                                                                                �
                                                                                �6�h�h�d@�?@DDD �"""C:\Windows\system32\FAILURE: No signal received
--- FAIL: TestLibraryCtrlHandler (68.32s)
FAIL
FAIL	runtime	68.350s
FAIL
Error running run: exit status 1

Again with the terminal garbage, but it's very interesting to see C:\Windows\system32\ appear amongst that garbage.

I have a suspicion that callbackasm is not a valid mechanism for implementing ctrlHandler. Control signals are delivered as actual signals on Windows. Something tells me it's not safe to go through the whole cgo callback path in these contexts...
The runtime: signal received on thread not created by Go. message has me a little worried in this direction, anyway.

Er, actually I don't see why this would be a problem. The Windows docs claim (https://docs.microsoft.com/en-us/windows/console/handlerroutine) that a new thread is created to handle these signals, so actually callbackasm seems like exactly the right mechanism. Now I'm getting the impression that something else is going wrong, raises an exception, and the exception lands on a non-Go thread?

If that theory is true, then it raises two questions:

What causes the signal to trigger in the first place?
Is that cause related to why badsignallen is written over?

Good news: in the current failure mode I can actually print out things when the DLL is getting initialized.

I just run my test from https://github.com/golang/go/issues/45638#issuecomment-826046034 on current tip. It failed once:
c:\Users\Alex\dev\go\src>go test -count=1 -run=CtrlH runtime
--- FAIL: TestLibraryCtrlHandler (179.74s)
    signal_windows_test.go:210: Program exited with error: exit status 1
        FAILURE: No signal received
FAIL
FAIL    runtime 185.257s
FAIL

and succeeded 3 times.

Out of curiosity, what happens if you change the WaitForSingleObject from 5 seconds to INFINITE on your machine?

The test still succeeds. But that gives me an idea. If you modify the test to use INFINITE in WaitForSingleObject, your C program will block on that WaitForSingleObject Windows API call indefinetly. And you should be able to attach to the process with some debugger and try to analyze program state.
Unfortunately I am not familiar with debuggers, so I cannot help you, but surly we can find help with debugging. You can use Delve debugger on Windows - I am not sure how good it is with Cgo program. Or you can use WindDbg - it is used by real Windows developers (not like me) - you should be able to do anything with it, except it does not understand Go program symbols. Or you can use gdb.

Huh, what version of Windows are you running?

C:\Users\Alex>ver

Microsoft Windows [Version 10.0.17763.1697]

C:\Users\Alex>

Maybe it's a bug specific to the windows-amd64-2016 builders?

It certainly can be. Do not forget that we are testing C program with Go DLL (built with Cgo) here. And you need gcc to build these.
Windows does not come with gcc installed. so whatever gcc you have installed on the builder is what you use. So you would have to account for the fact that this bug could be in gcc tools or runtime. And this builder, I suspect, have very old version of gcc. For example, see #35006 and #43195.
I use
c:\Users\Alex\dev\go\src>gcc --version
gcc (x86_64-win32-seh-rev0, Built by MinGW-W64 project) 8.1.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

c:\Users\Alex\dev\go\src>

here. I suspect this gcc is different from what is installed on builder. And the fact that test failed once here (see above), probably means it is not gcc bug.

Again with the terminal garbage, but it's very interesting to see C:\Windows\system32\ appear amongst that garbage.

This was, probably, generated by gcc linker, not Go linker. But otherwise, I have not comment.

Here's a theory about how the above messages are getting produced: something causes us to get into an exception that lands on a thread that has GS set (where we put TLS on Windows). That exception tries to print the badsignalmsg but as it turns out badsignallen got corrupted or something. It's otherwise wrong. So then we get this buffer overrun, garbage data is printed out after the message until we reach a point where we hit invalid/unmapped/uncommitted memory, triggering another exception on the same thread. And so on a few more times.

Unfortunately am not familiar enough with Go runtime and gcc runtime to speculate. But that sounds possible. There a 2 control handlers installed by SetConsoleCtrlHandler - one by gcc runtime and another by Go runtime. And there are 2 exception handlers installed - one by gcc runtime and another by Go runtime. So it can get messy.
Alex

The test failed again just as I submitted previous message. Unfortunately this time the output is huge, so I am providing the end of it.
                                                                                �#    runtime     ��]n        .        Similarly, you should not run it with -counter parameter, because test build Go executable first. So just modify the test to build executable once and then run it many times in a loop.
If Go control handler code is broken, then this test might break too.
Alex

/cc @bufflig

Apologies for how much progress has slowed down on this. I'm fairly certain there's some very particular timing required for this failure to trigger. As I add print statements in various places, it gets harder and harder to reproduce.
I'll soon take @alexbrainman's advice and try setting an infinite timeout so I can hook in a debugger. For that I need to set up my Windows environment properly, which will take some time.
Another interesting avenue could be to check Go 1.16. It would be nice to somehow rule out that this can't happen in previous releases. If it can, then I think a workaround to whatever the issue is would be more acceptable and prevent blocking the release. But, I will continue trying to get to the bottom of this.

Does anything change if you add a call to the Dummy function in dummy.dll before printing ready?  And what happens if you don't load the DLL at all?  I agree with you that the current code has a race, but I wonder which way the race goes.

Coming back to this... I think my theory is subtly wrong.
According to the MSDN, control handlers are always called on a new thread (https://docs.microsoft.com/en-us/windows/console/console-control-handlers). That thread should be distinct from the runtime initialization thread, and so shouldn't impede progress in the runtime, presumably. That thread will just spin in needm until initialization gets far enough.
The only thing preventing progress though, is that check of cgoHasExtraM at the beginning of needm. If that somehow cascades into a bunch of failures, that would explain why the test fails in such a strange manner (though, admittedly, I still don't understand how it would do so).

Well, hey I learned one new thing. There should be a runtime.abort call after runtime.badsignal2 on Windows. That probably explains the cascading failures (there's no way to proceed, and yet we do!).
I'm going to try to see where that signal is coming from. I have access to the instruction pointer in that context, it's just a question of how to actually surface it.

The PC that triggers the failure isn't one that's known to Go (and it's consistently the same one). If I were to guess, it's probably a Windows DLL (win32?).

The saga continues. I fixed the garbage-getting-printed issue, but there's still that pesky signal firing (FWIW it's an access violation signal, so the equivalent of a segfault, and to catch you up, the PC that triggered it is NOT in the Go binary). The good news is, I've narrowed the source a little further. Namely, if I eliminate the call to SetConsoleCtrlHandler, but keep the compileCallback invocation (printing the PC so the call stays around... also I made sure it still reproduces with everything the same, just the print added in). Doing this does not reproduce.
Now what I did was I kept the print and the compileCallback in, but rather than use the result of compileCallback as the handler, I made my own little handler in assembly that basically just returns FALSE (so the next handler runs). And with that I can still reproduce. So, there's something wrong about where we're calling SetConsoleCtrlHandler specifically. It doesn't seem to involve the callback, or needm, or the cgocallback path at all.
I'm thinking now that there's, very simply (back to the very very beginning of this thread), a race between SetConsoleCtrlHandler and when the control signal lands. I do not understand what the problem could be. MSDN does not discuss any concurrency guarantees (e.g. is it safe to call SetConsoleCtrlHandler concurrently? (Note that in this case it is not being called concurrently, I just mean that as an example)).
At this point I have no idea how to proceed. My CL above is a reasonable workaround. I can also make it so that the test binary calls Dummy in the Go DLL before indicating readiness. That seems to fix it, too.
@ianlancetaylor @alexbrainman WDYT?

The saga continues. I fixed the garbage-getting-printed issue, but there's still that pesky signal firing (FWIW it's an access violation signal, so the equivalent of a segfault, and to catch you up, the PC that triggered it is NOT in the Go binary).

I tried running test on top of 1108cbe60b87196c and test failed once and I still see garbage printed:
c:\Users\Alex\dev\go\test>go test -count=10 -run=CtrlH runtime
--- FAIL: TestLibraryCtrlHandler (19.45s)
    signal_windows_test.go:211: Program exited with error: exit status 1
        runtime: signal received on thread not created by Go.
                                                                                ,�ݪY-�A��ZR��"on�5�M��.R�F�x58D�%�q�ч]�6I#�9�2�g�.i保C��ߞc�T;:���8�Av���8�*�Z��oZ��۞�X���Ո��~�ӛ��k�O��                                                                                runtime: signal received on thread not created by Go.
                                                                                ,�ݪY-�A��ZR��"on�5�M��.R�F�x58D�%�q�ч]�6I#�9�2�g�.i保C��ߞc�T;:���8�Av���8�*�Z��oZ��۞�X���Ո��~�ӛ��k�O��        ���     ���                                                                                runtime: signal received on thread not created by Go.
                                                                                ,�ݪY-�A��ZR��"on�5�M��.R�F�x58D�%�q�ч]�6I#�9�2�g�.i保C��ߞc�T;:���8�Av���8�*�Z��oZ��۞�X���Ո��~�ӛ��k�O��        ���     ���
... more garbage ...

Is this garbage expected?
I also noticed that this message would only be printed, if runtime.sigtramp called and it could not find g. The fact that runtime.sigtramp is called means we received an exception. It would be nice to see stack trace printed. Unfortunately runtime.sigtramp end-up  calling runtime.badsignal2 instead.
Perhaps we can disable runtime.badsignal2 check and hope stack trace is printed.
Another option would be run test program inside a debugger. Debugger will stop when exception. Maybe we can see the program state then. It would be nice to see exactly where exception occurs.
Perhaps your https://go-review.googlesource.com/c/go/+/315830/ fixes this error, and is good enough.

The good news is, I've narrowed the source a little further. Namely, if I eliminate the call to SetConsoleCtrlHandler, but keep the compileCallback invocation (printing the PC so the call stays around... also I made sure it still reproduces with everything the same, just the print added in). Doing this does not reproduce.

That means that your CL 315830 could not be a solution here. Correct?

I'm thinking now that there's, very simply (back to the very very beginning of this thread), a race between SetConsoleCtrlHandler and when the control signal lands. I do not understand what the problem could be. MSDN does not discuss any concurrency guarantees (e.g. is it safe to call SetConsoleCtrlHandler concurrently? (Note that in this case it is not being called concurrently, I just mean that as an example)).

What particular race are you thinking about? You are allowed to call SetConsoleCtrlHandler API many times - that is how the API is designed. In fact I am pretty sure our test calls SetConsoleCtrlHandler twice - Go calls it, and C runtime also calls it.
Perhaps the problem is that Go DLL does not remove SetConsoleCtrlHandler handler before it exits?
Similarly SetConsoleCtrlHandler is supposed to handle CTRL_CLOSE_EVENT. Perhaps we get that message (I don't see how that happens) after Go DLL runtime is off, but Windows SetConsoleCtrlHandler handler still points to Go code.
I plan to play with this more on the weekend. Let's delay the decision.
Also, perhaps @zx2c4 can help debug this, if you have time. Thank you.
Alex

The saga continues. I fixed the garbage-getting-printed issue, but there's still that pesky signal firing (FWIW it's an access violation signal, so the equivalent of a segfault, and to catch you up, the PC that triggered it is NOT in the Go binary).

I tried running test on top of 1108cbe and test failed once and I still see garbage printed:
c:\Users\Alex\dev\go\test>go test -count=10 -run=CtrlH runtime
--- FAIL: TestLibraryCtrlHandler (19.45s)
    signal_windows_test.go:211: Program exited with error: exit status 1
        runtime: signal received on thread not created by Go.
                                                                                ,�ݪY-�A��Z�R��"on�5�M����.R�F�x58D�%��q�ч]�6I#�9�2�g�.i保�C��ߞc�T;:����8���Av���8�*�Z���oZ���۞�X����Ո��~��ӛ��k��O��                                                                                runtime: signal received on thread not created by Go.
                                                                                ,�ݪY-�A��Z�R��"on�5�M����.R�F�x58D�%��q�ч]�6I#�9�2�g�.i保�C��ߞc�T;:����8���Av���8�*�Z���oZ���۞�X����Ո��~��ӛ��k��O��        ���     ���                                                                                � ��������������                                                                                runtime: signal received on thread not created by Go.
                                                                                ,�ݪY-�A��Z�R��"on�5�M����.R�F�x58D�%��q�ч]�6I#�9�2�g�.i保�C��ߞc�T;:����8���Av���8�*�Z���oZ���۞�X����Ո��~��ӛ��k��O��        ���     ���
... more garbage ...

Is this garbage expected?

It's not, that's very surprising. It's also surprising that you landed in badsignal2 more than once -- all the failures I got after that CL were clean. I'll try to reproduce this again.

I also noticed that this message would only be printed, if runtime.sigtramp called and it could not find g. The fact that runtime.sigtramp is called means we received an exception. It would be nice to see stack trace printed. Unfortunately runtime.sigtramp end-up calling runtime.badsignal2 instead.
Perhaps we can disable runtime.badsignal2 check and hope stack trace is printed.

Unfortunately I don't think that'll work. There's a lot of stuff that depends on having some kind of G (including a g0); I'm not even positive we can throw in that circumstance.

Another option would be run test program inside a debugger. Debugger will stop when exception. Maybe we can see the program state then. It would be nice to see exactly where exception occurs.

I've been putting this aside because I don't have an environment where I can easily attach a debugger and follow subprocesses. The Windows world feels alien to me. I don't have high confidence in Delve handling this very specific situation very well, but I can try.

Perhaps your https://go-review.googlesource.com/c/go/+/315830/ fixes this error, and is good enough.

The good news is, I've narrowed the source a little further. Namely, if I eliminate the call to SetConsoleCtrlHandler, but keep the compileCallback invocation (printing the PC so the call stays around... also I made sure it still reproduces with everything the same, just the print added in). Doing this does not reproduce.

That means that your CL 315830 could not be a solution here. Correct?

 However, it's clear that calling SetConsoleControlHandler later in initialization resolves the problem. Hence my theory about some kind of race condition.

I'm thinking now that there's, very simply (back to the very very beginning of this thread), a race between SetConsoleCtrlHandler and when the control signal lands. I do not understand what the problem could be. MSDN does not discuss any concurrency guarantees (e.g. is it safe to call SetConsoleCtrlHandler concurrently? (Note that in this case it is not being called concurrently, I just mean that as an example)).

What particular race are you thinking about? You are allowed to call SetConsoleCtrlHandler API many times - that is how the API is designed. In fact I am pretty sure our test calls SetConsoleCtrlHandler twice - Go calls it, and C runtime also calls it.

I don't have a specific one in mind, because I don't have any documentation on SetConsoleCtrlHandler's requirements. But I think it's clear to me that there's some kind of race going on. Runtime initialization is happening on a separate thread in parallel with the rest of the C code in the test. In particular, it's happening in parallel with the CTRL_BREAK landing in the process. I think there's some kind of invariant in the Windows APIs that we're violating. The sigtramp is coming from some PC that I can't identify, so my default assumption is that it's some shared library in the address space (could be wrong though).

Perhaps the problem is that Go DLL does not remove SetConsoleCtrlHandler handler before it exits?
Similarly SetConsoleCtrlHandler is supposed to handle CTRL_CLOSE_EVENT. Perhaps we get that message (I don't see how that happens) after Go DLL runtime is off, but Windows SetConsoleCtrlHandler handler still points to Go code.

Interesting, I hadn't thought about a situation where runtime initialization is complete. What do you mean by when the Go runtime is already "off"? Do you mean the rest of the process is already exiting?

I plan to play with this more on the weekend. Let's delay the decision.
Also, perhaps @zx2c4 can help debug this, if you have time. Thank you.
Alex

Thank you so much for taking the time to look at this!

I found a computer (with Windows 7) where this issue reproduces much more easily than on my Windows 10 computer.
Unfortunately I can reproduce this issue even on top of
https://go-review.googlesource.com/c/go/+/315830/
I will try to understand where the issue occurs, as promised. No success guraneteed.
Alex

@mknyszek Checking in from a release meeting, are there updates here? Thanks.

@dmitshur I have been working on this issue.
I think the problem here is with the runtime.TestLibraryCtrlHandler. The test is racy and broken.
For example, the test was introduced together with the fix in CL 211139. But the test still passes, if I revert to commit before CL 211139.
c:\Users\Alex\dev\go\src\runtime>git status
HEAD detached at 5756808ce8
nothing to commit, working tree clean

c:\Users\Alex\dev\go\src\runtime>go test -v -count=1 -run=yCtrlH
=== RUN   TestLibraryCtrlHandler
--- PASS: TestLibraryCtrlHandler (2.98s)
PASS
ok      runtime 3.569s

c:\Users\Alex\dev\go\src\runtime>git checkout HEAD~ os_windows.go
Updated 1 path from 2e2e0c0cf8

c:\Users\Alex\dev\go\src\runtime>git diff --staged
diff --git a/src/runtime/os_windows.go b/src/runtime/os_windows.go
index<PHONE_NUMBER>..bddc25729a 100644
--- a/src/runtime/os_windows.go
+++ b/src/runtime/os_windows.go
@@ -1031,11 +1031,7 @@ func ctrlhandler1(_type uint32) uint32 {
        if sigsend(s) {
                return 1
        }
-       if !islibrary && !isarchive {
-               // Only exit the program if we don't have a DLL.
-               // See https://golang.org/issues/35965.
-               exit(2) // SIGINT, SIGTERM, etc
-       }
+       exit(2) // SIGINT, SIGTERM, etc
        return 0
 }

c:\Users\Alex\dev\go\src\runtime>go test -v -count=1 -run=yCtrlH
=== RUN   TestLibraryCtrlHandler
--- PASS: TestLibraryCtrlHandler (3.86s)
PASS
ok      runtime 4.270s

c:\Users\Alex\dev\go\src\runtime>

The runtime.TestLibraryCtrlHandler assumes that by the time LoadLibrary returns that Go runtime has finished initialising (including calling SetConsoleCtrlHandler in runtime)
https://github.com/golang/go/blob/831573cd21e65c37b4e1bf5d44dc23b125084b7a/src/runtime/testdata/testwinlibsignal/main.c#L35-L42
But there is nothing in the test to verify that SetConsoleCtrlHandler was called or not. In fact, if I comment out LoadLibrary call in the test, the test still passes.
So we need to replace the test first. And then we should be able to tell, if we have a problem or not.
It is pretty hard to build proper test here. But I think I am making progress. I will report more when I am ready with the test.
I hope it helps.
Alex

But there is nothing in the test to verify that SetConsoleCtrlHandler was called or not. In fact, if I comment out LoadLibrary call in the test, the test still passes.

This should be very easy to fix. It's true that Go initialization is done asynchronously on another thread, so this test races. (Side note: even if it races, it shouldn't spew garbage like above.) But every cgo entry point calls _cgo_wait_runtime_init_done, which properly waits for initialization to complete. So an easy fix to that is to just call the dummy function from dummy.go before printing "ready". I'll submit a patch for that.

@alexbrainman https://go-review.googlesource.com/c/go/+/321769 passes tests and should now fix the races with the tests you mentioned. I haven't read all of this thread in detail, so I'm not sure if this is all that's required to close the issue. But at the very least, you can +2 that and we'll at least have a fixed test.
However, the garbage text you're getting is a bit disturbing, and maybe we should look more deeply into that after you review my test fix.  Do we have better options?

I added the call to Dummy in an earlier iteration of investigating this, but @alexbrainman brings up a good point that the test isn't properly checking the condition it's supposed to check (i.e. it's racy).
Maybe the path forward here is to fix the test, and then file a separate issue for the garbage being spewed due to the race between the signal landing and initialization completing. Then again, the garbage spewing might just be fixed by calling ExitProcess directly instead of runtime.abort in badsignal2. I'll send a CL for that.

I added the call to Dummy in an earlier iteration of investigating this, but @alexbrainman brings up a good point that the test isn't properly checking the condition it's supposed to check (i.e. it's racy).

Adding a call to Dummy DOES make the test properly check for the condition that it's supposed to check and FIXES the race. Why does it do this? Because the call to Dummy implies a call to _cgo_wait_runtime_init_done, which waits for initialization to complete, and hence waits for the signal handler to be installed.

then file a separate issue for the garbage being spewed due to the race between the signal landing and initialization completing. Then again, the garbage spewing might just be fixed by calling ExitProcess directly instead of runtime.abort in badsignal2. I'll send a CL for that.

Or you could just re-title this issue if you wanted? AFAICT, https://go-review.googlesource.com/c/go/+/321769 is the correct fix for the actual test failing, but indeed, I agree with you, that the garbage spewing is the lingering issue that definitely deserves attention.

I think we can close this issue because
https://go-review.googlesource.com/c/go/+/321769
fixes the runtime.TestLibraryCtrlHandler test properly.
The test fails if I revert
https://go-review.googlesource.com/c/go/+/211139/
But on current tip I could not make test fail in any way on few different systems.
Alex

@alexbrainman Sounds good. I'll file a new issue for the garbage we saw getting generated.


There is no series type enum

The README states that to configure the series type when creating/modifying a lobby, you need to use the series type enum. There is no such enum neither in the project nor in the proto files. The current version of the protobufs just states that it's a 32 bit integer. Has anyone already nethooked the different  values for best-of-1/2/3/5 ?

Nope. Let me know if you want the compiled nethook from a few months ago.


Linux support

I've started porting the install to support Linux. Is there any interest in adding my stuff to the official repo?

@jeblackburn a different team at pivotal uses Ubuntu exclusively and they have a similar repo to this at https://github.com/cloudfoundry/cli-workstation
@xenophex @tjvman @willmurphyscode would you ever be willing to fork it into pivotal/?

@cjcjameson That repo is (as the name implies) very CLI-specific about what it installs, so I don't think it seems like a good fit for folks looking for a general-purpose Linux workstation bootstrapper. There may be some stuff in there that could be adapted to a more portable script, though.
I'm curious as to why forking into the pivotal org would be needed - naively, I'd expect anyone who did want to base their own setup off of cli-workstation to just fork the repo as it currently stands. Am I missing something (such as some teams not being able to fork repos in the cloudfoundry org)?

@tjvman good points, all good to stay as is.


Training failing

Hi there,
I was quite successful using DeepRTplus last year, but I am now having trouble working with a new dataset.
I have attached my testing and training files. The training runs without error, but the resulting correlation with the test set is very low, 0.2-0.25.
I tried running the software again on the previous dataset I used with similar results.
I tried re-cloning the software and starting with unmodified files, with no luck. I then tried with your datasets using
python data_split.py data/mod.txt 9 1 2
python capsule_network_emb.py

and the following error was thrown:

Traceback (most recent call last):
File "capsule_network_emb_cpu.py", line 743, in
engine.train(processor, get_rt_iterator(True), maxepoch=NUM_EPOCHS, optimizer=optimizer)
File "/home/tsour.s/.local/lib/python3.7/site-packages/torchnet/engine/engine.py", line 63, in train
state['optimizer'].step(closure)
File "/home/tsour.s/.local/lib/python3.7/site-packages/torch/autograd/grad_mode.py", line 15, in decorate_context
return func(*args, **kwargs)
File "/home/tsour.s/.local/lib/python3.7/site-packages/torch/optim/adam.py", line 62, in step
loss = closure()
File "/home/tsour.s/.local/lib/python3.7/site-packages/torchnet/engine/engine.py", line 56, in closure
self.hook('on_forward', state)
File "/home/tsour.s/.local/lib/python3.7/site-packages/torchnet/engine/engine.py", line 31, in hook
self.hooksname
File "capsule_network_emb_cpu.py", line 575, in on_forward
meter_loss.add(state['loss'].data[0])
IndexError: invalid index of a 0-dim tensor. Use tensor.item() in Python or tensor.item<T>() in C++ to convert a 0-dim tensor to a number

Any idea what the issue might be?
Thanks!

This is caused by the out-of-date pytorch (<=0.4) this repo uses, see here for reference.
For a more advanced pytorch (>=0.5), please try to modify as:
For CPU version: #575
meter_loss.add(state['loss'].data)

For GPU version: #575
meter_loss.add(state['loss'].cpu().data)

Thanks, that solved the error I was seeing. However, now I am getting this error when testing with your data:
RuntimeError: cannot reshape tensor of 0 elements into shape [0, -1, 1] because the unspecified dimension size -1 can be any value and is ambiguous
I tried changing the batch size in capsule_network_emb_cpu.py, but this did not help.

By running which specific code there is this error? I tried capsule_network_emb_cpu.py and prediction_emb_cpu.py and didn't see this error. Maybe you can show the complete error message?

I was getting that error when running capsule_network_emb_cpu.py. I tried again and it worked on your dataset, but not mine.
The losses do seem to be lower than before though.
Here is the complete error message I am getting. I also attached my training and testing data.
[> 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 25000/25000 [40:22<00:00, 10.32it/s]

[Epoch 1] Training Loss: 0.0463 (MSE: 7.0000)
[Epoch 1] Testing Loss: 0.0414 (MSE: 7.0000)
Traceback (most recent call last):
File "capsule_network_emb_cpu.py", line 743, in
engine.train(processor, get_rt_iterator(True), maxepoch=NUM_EPOCHS, optimizer=optimizer)
File "/home/tsour.s/.local/lib/python3.7/site-packages/torchnet/engine/engine.py", line 67, in train
self.hook('on_end_epoch', state)
File "/home/tsour.s/.local/lib/python3.7/site-packages/torchnet/engine/engine.py", line 31, in hook
self.hooksname
File "capsule_network_emb_cpu.py", line 671, in on_end_epoch
pred_batch = model(test_batch)
File "/home/tsour.s/.local/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in call
result = self.forward(*input, **kwargs)
File "capsule_network_emb_cpu.py", line 275, in forward
x = self.primary_capsules(x)
File "/home/tsour.s/.local/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in call
result = self.forward(*input, **kwargs)
File "capsule_network_emb_cpu.py", line 149, in forward
outputs = [capsule(x).view(x.size(0), -1, 1) for capsule in self.capsules]
File "capsule_network_emb_cpu.py", line 149, in
outputs = [capsule(x).view(x.size(0), -1, 1) for capsule in self.capsules]
RuntimeError: cannot reshape tensor of 0 elements into shape [0, -1, 1] because the unspecified dimension size -1 can be any value and is ambiguous
]

ST_DeepRTplus_test.txt
ST_DeepRTplus_train.txt

Thanks for sharing the data, now I am able to reproduce the error. It turns out the cause is the same as before, the number of testing samples (100000) is divisible by the batch size (16), so please change line 632 to e.g.
PRED_BATCH = 166

It seems now the correlation is higher (~0.974 after the 1st epoch).

Ahhh, now I feel silly. I tested another batch size, but I guess it also by chance divided evenly. My mistake, I should have been more thorough checking that. Just tried it again, and it worked like a charm 👍
Thanks so much for your help, I really appreciate your responsiveness!


[Fix-#3958][api] files should not be created successfully in the directory of the authorized file

What is the purpose of the pull request
[Fix-#3958][api] files should not be created successfully in the directory of the authorized file
Brief change log
add the judge of whether has permission

+1


Allow tags to be saved on edit

A consequence of PR #241 was that tags were no longer saved at all, because getBlogTags() received tags as a string even when there were several of them, causing them to be returned as an empty array.

+1

+1

@FGM Thanks, should be in next release.

1


[3.1] Fixes for SR-3920

Explanation: There were problems handling the covariant result of a self.init delegation to a constructor inherited from a base class that would manifest themselves as internal inconsistencies, leading to assertion failures in debug builds and sporadic crashes in release builds of the compiler.
Scope: Makes code that worked only by accident in previous Xcode release builds of the compiler work reliably.
SR Issue: SR-3920
Risk: Low
Testing: Swift CI, test cases from Jira

@swift-ci Please test

@slavapestov Does this look OK for 3.1?

LGTM.


Not starting any torrent

Since a few weeks, Elementum doesn't start any torrent. I can search, see the list of trackers, but when I choose one Kodi is loading for a second and then returns to the main menu. When I add a torrent by magnet URL the same thing happens: it does not start buffering.
I checked the logs and found this:
ERROR: GetDirectory - Error getting plugin://plugin.video.elementum/playtorrent
I tried downgrading to version 0.1.44 and 0.1.41 (copying manually to /home/osmc/.kodi/addons/plugin.video.elementum) but it doesn't seem to make any difference. I know support has ended but maybe somebody else experiences the same problem and knows how to solve it?
Thanks in advance.

@Cyroq  no one can help you without log files.

I have the same issue. Here it is the logfile:
paste.kodi.tv/siquzoleku

@filippeof what you have in the log is that you select a magnet and it cannot be resolved for some reason. Maybe DHT is not working for you.
2020-04-09 12:39:19.578 T:1521992576  NOTICE: [plugin.video.elementum] INFO  bittorrent   ▶ AddTorrent       Waiting for information fetched for torrent: ee5240d6a3376b2733430b30f7889b11f6eeb033

@Cyroq you can try with 0.1.51, maybe it will solve your problem.

Thank you @elgatito. I tried 0.1.51 but that did not fix the problem. This is the log: https://paste.kodi.tv/yovinuzuza.kodi

@Cyroq your Elementum starts twice. Try to uninstall and re-install the addon.

I uninstalled the addon, rebooted and installed in again. Unfortunately, the problem persists. The log from re-install to error: https://paste.kodi.tv/qiwupokife.kodi

@Cyroq there is no information what is going on with the addon.

When I insert a magnet url, a loading icon shows up for a few seconds and Elementum goes back to the main menu. I have no other information about the addon. How can I retrieve more information that is useful to you?

@Cyroq you should enable debugging mode. http://elementum.surge.sh/contribute/log/

Thank you. This is the log with debugging and event logging: https://www.dropbox.com/s/x71c3trkabyzb17/kodi.log?dl=0 (it's too big for paste.kodi.tv)

@Cyroq I see you have a dialog to select file or enter torrent url and that is the last message there. What you do there?

I insert the magnet url and press OK, after which Elementum closes the dialog, shows the loading icon for a few seconds and then stops.
On previous attempts the 'Trending' and 'Popular' lists etc were empty, I am now able to start a torrent from there. Playing a torrent by url still doesn't seem to work.

On previous attempts the 'Trending' and 'Popular' lists etc were empty, I am now able to start a torrent from there. Playing a torrent by url still doesn't seem to work.

It is not normal, I don't know what happens with your installation.
Maybe someone ever will help you.

@Cyroq Please try with 0.1.52 and post a new log file after testing.

I was facing a similar issue and updating fixed it for me. Thank you for the update! You rule!

Unfortunately the update did not make any difference for me. Log: https://paste.kodi.tv/ovuqitaduv.kodi

ERROR: [plugin.video.elementum] Failed to wait till window has finished loading

You have a "busy" dialog, probably a running wheel, so playback does not start.
Not having a solution for that.

Strange, I did not see any dialog or running wheel. I pasted my magnet link, hit OK and the dialog window closed and nothing happened.
Maybe I'll try a complete reinstall of Kodi some day. Nevertheless, thank you for your help.

try changing your skin . i had aura and the same issue. changing theme fixed this for me.

@Cyroq Please, try with Elementum 0.1.56.

@Cyroq Should I close this?

Yes, you can close it. I ended up using a different approach by hitting the share button on magnet links and share it with Yatse.


Support build on windows #193

I will try to apply this patch to my branch, rather than both to master, because I think that will cause less rebase conflicts.

This code has be migrated into a different branch


implemented new frontend bundle structure in media bundle

implemented new frontend bundle structure in media bundle.
implemented services for data- and overlay-management.
implemented new datagrid decorator for media-list.
tasks:

[x] set up bower in media-bundle
[x] implemented new bundle structure
[x] implemented javascript services
[x] implemented masonry decorator
[x] refactored media overlays
[x] refactored media components
[x] added translations
[ ] test coverage
[ ] gather feedback for my changes

depending prs:
https://github.com/massiveart/husky/pull/536
informations:

q
a

Fixed tickets
None

BC breaks
None

Documentation PR
None

@nnatter 'mediaSelection.js' to 'media-selection.js'

@nnatter getLastVisitedCollection is not used in user-settingsmanager

@nnatter use MediaRouter instead of emiting an event in the bundle-extension-file


Add new exported function GetCellPixelsWithCoordinates

PR Details
Added GetCellPixelsWithCoordinates Function to Determine Cell Dimensions in Pixels
Related Issue
https://github.com/qax-os/excelize/issues/1813
Motivation and Context
To adjust the aspect ratio displayed in individual Excel cells, it is necessary to tweak ScaleX or ScaleY, but there isn't an appropriate public function available for this calculation. Although public functions like GetRowHeight and GetColWidth exist, they do not return values in pixels, and there's a discrepancy with the pixel calculations used within drawingResize in AddPicture. DrawingResize (using getRowHeight and getColWidth) seems to accurately calculate sizes for the most part.
-----GetRowHeight, GetColWidth-----
cellWidth:  306
cellHeight:  333
-----drawingResize(getRowHeight, getColWidth)-----
cellWidth:  360
cellHeight:  264

Therefore, the pixel function used in drawingResize has been extracted as a public function.
How Has This Been Tested
Types of changes

[ ] Docs change / refactoring / dependency upgrade
[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist

[ ] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[x] I have read the CONTRIBUTING document.
[ ] I have added tests to cover my changes.
[ ] All new and existing tests passed.

Hi @xuri
Could you please let me know if there's anything else needed from my end to move this forward for merge? Your feedback would be greatly appreciated!


When I build Haxe/Neko project I got errors from compiler marked as information

When I build Haxe/Neko project I got errors from compiler marked as information
Steps to reproduce:
Simply try to compile something not valid, and see, that output says there is 1 error and it is shown as information(instead should be marked as error)
TIR: STB-7741

Should be fixed by PR #180.

Fixed.  Will be in the next release after 0.9.3.  Please re-open if it recurs.


Add RBI for Hash#merge!

Motivation
To fix #99.
I used the same signature than Hash#merge.
Test plan
Updated tests for hidden-method-finder.

This produced 33 errors on the stripe repo: http://go/A4vCqs4k
I'll spend some time tomorrow getting your two RBI PRs merged in, and fix the errors on the stripe codebase.


README directions on running the examples?

It may be because it's been a while since I've used Rust, but I can't figure out how to run the examples in /examples - some build steps would be great.
I presume I'm doing something wrong because running just cargo build gives me a fatal error. As I work through this issues would love to help out building this project!

Hey thanks for the issue!
I think this is a great idea. I've just added PR #47 which adds a Running Examples section to the README. Is something like this what you were after?

running just cargo build gives me a fatal error

Hmm sounds strange! Would you mind pasting the entire output of the error (including the command(s) used to produce it) here so I can take a look? If you could also include the output of rustc --version and cargo --version and your operating system (and version) that would be great.

Yep, the release command is the piece that's helpful, a command I could get just copy/paste to get the examples running :)

Re: the fatal error
cargo run --release --example simple_window and cargo build both give me the following output:
   Compiling nannou v0.1.0 (file:///Users/andrescuervo/code/play/rust/nannou)
error[E0277]: the trait bound `for<'r> std::ops::FnMut(audio::cpal::cpal_impl::VoiceId, audio::cpal::UnknownTypeBuffer<'r>) + 'static: std::marker::Send` is not satisfied
   --> src/app.rs:261:18
    |
261 |                 .spawn(move || event_loop.run(move |v_id, out| loop_context.process(v_id, out)))
    |                  ^^^^^ `for<'r> std::ops::FnMut(audio::cpal::cpal_impl::VoiceId, audio::cpal::UnknownTypeBuffer<'r>) + 'static` cannot be sent between threads safely
    |
    = help: the trait `std::marker::Send` is not implemented for `for<'r> std::ops::FnMut(audio::cpal::cpal_impl::VoiceId, audio::cpal::UnknownTypeBuffer<'r>) + 'static`
    = note: required because of the requirements on the impl of `std::marker::Send` for `&'static mut for<'r> std::ops::FnMut(audio::cpal::cpal_impl::VoiceId, audio::cpal::UnknownTypeBuffer<'r>) + 'static`
    = note: required because of the requirements on the impl of `std::marker::Send` for `std::ptr::Unique<&'static mut for<'r> std::ops::FnMut(audio::cpal::cpal_impl::VoiceId, audio::cpal::UnknownTypeBuffer<'r>) + 'static>`
    = note: required because it appears within the type `alloc::raw_vec::RawVec<&'static mut for<'r> std::ops::FnMut(audio::cpal::cpal_impl::VoiceId, audio::cpal::UnknownTypeBuffer<'r>) + 'static>`
    = note: required because it appears within the type `std::vec::Vec<&'static mut for<'r> std::ops::FnMut(audio::cpal::cpal_impl::VoiceId, audio::cpal::UnknownTypeBuffer<'r>) + 'static>`
    = note: required because of the requirements on the impl of `std::marker::Send` for `std::sync::Mutex<std::vec::Vec<&'static mut for<'r> std::ops::FnMut(audio::cpal::cpal_impl::VoiceId, audio::cpal::UnknownTypeBuffer<'r>) + 'static>>`
    = note: required because it appears within the type `audio::cpal::cpal_impl::ActiveCallbacks`
    = note: required because of the requirements on the impl of `std::marker::Send` for `std::sync::Arc<audio::cpal::cpal_impl::ActiveCallbacks>`
    = note: required because it appears within the type `audio::cpal::cpal_impl::EventLoop`
    = note: required because it appears within the type `audio::cpal::EventLoop`
    = note: required because of the requirements on the impl of `std::marker::Send` for `std::sync::Arc<audio::cpal::EventLoop>`
    = note: required because it appears within the type `[closure@src/app.rs:261:24: 261:96 event_loop:std::sync::Arc<audio::cpal::EventLoop>, loop_context:audio::stream::output::LoopContext]`

error: aborting due to previous error

error: Could not compile `nannou`.

I'm on:

cargo 0.23.0 (61fa02415 2017-11-22)
rustc 1.22.1 (05e2e1c41 2017-11-22)
MacOS 10.12.6 (Sierra)

While there are other generated errors, changing line 261 as noted above to:
 .spawn(|| { println!("");

Ahh @AndresCuervo I think your problem relates to some CPAL macos backend issues that I believe were recently solved in tomaka/cpal#186 and tomaka/cpal#189. I've added a new PR that publishes a new version at tomaka/cpal#190 - once that is merged we can update nannou and hopefully get these fixes.
Going to close this issue now as the macos  cpal backend issues are a separate issue.

Ah, thanks for the info, will try again when that gets merged in then! :) Thanks @mitchmindtree


Fix handling of categorical column

I had a logic error in the handling of categorical columns.

I can confirm this fixes the problem. Thanks for the quick fix!

Thanks guys, merged.


[QUESTION] Why unpublish() is not declared as async in index.ts ?

Question
Hello Marc,
Why unpublish() is not declared as async in index.ts while in https://github.com/mrousavy/react-native-google-nearby-messages/blob/master/android/src/main/java/com/mrousavy/nearby/GoogleNearbyMessagesModule.kt#L191 it is declared as async?
I mean if I call publish() right after unpublish(), the unpublish might still be in progress, right?
Thanks
Environment
paste the output of `react-native info` here

"react": PASTE_VERSION_HERE,
"react-native": PASTE_VERSION_HERE,
"react-native-google-nearby-messages": PASTE_VERSION_HERE

You're right! Unpublish should be async too. Can you open a PR to fix this on the JS side?

nvm, fixed it

Thanks very much for your quick fix.
Now I have noticed the ios side func unpublish() does not seem to be async so the interface change will not break the ios side unpublish() ?


Numba deprecation warning

Thank you for the great software!
When using skan 0.8 with python 3.8.3 we get a warning message:
NumbaDeprecationWarning: The 'numba.jitclass' decorator has moved to 'numba.experimental.jitclass' to better reflect the
experimental nature of the functionality. Please update your imports to accommodate this change and see
http://numba.pydata.org/numba-doc/latest/reference/deprecation.html#change-of-jitclass-location for the time frame.
@numba.jitclass(csr_spec)

caused by this part of the csr.py module:
@numba.jitclass(csr_spec)
class NBGraph:

Indeed, on the Numba Website we can read:

Between versions 0.48 and 0.49 Numba underwent a large amount of refactoring. One of the decisions made by the core developers as part of this refactoring was to move numba.jitclass to a new location numba.experimental.jitclass. This is to help reinforce expectations over the behaviour and support for certain features by deliberately placing them in an experimental submodule.

Replacing numba.jitclass by numba.experimental.jitclass in csr.py would remove this warning.
@numba.experimental.jitclass(csr_spec)
class NBGraph:

Do you want me to address this?

Thanks for reporting @theodumont ! Indeed it would be best to fix this. If you are comfortable with the GitHub pull request workflow I would definitely appreciate a PR, otherwise I will try to get this done in the next few days.

Great!
I'll make a PR right now, don't hesitate to tell me if something is wrong with it.

Fixed in #87


Dependency version regressions in v0.2.1

I added versioning to swim crate local dependencies in f480af2efc1fb76cc8f9d28f3e91124a6b67fb8e which introduced a regression in published crates.io version.
The published version is unable to locate newly re-exported types:
error[E0433]: failed to resolve: could not find `routerify` in `swim_core`
   --> /home/syed/.cargo/registry/src/github.com-1ecc6299db9ec823/swim-0.2.1/src/lib.rs:155:24
    |
155 |     pub use swim_core::routerify::ext::RequestExt;
    |                        ^^^^^^^^^ could not find `routerify` in `swim_core`

error[E0432]: unresolved import `swim_core::async_trait`
   --> /home/syed/.cargo/registry/src/github.com-1ecc6299db9ec823/swim-0.2.1/src/lib.rs:154:13
    |
154 |     pub use swim_core::async_trait;
    |             ^^^^^^^^^^^^^^^^^^^^^^ no `async_trait` in the root

error[E0432]: unresolved import `swim_core::async_trait`
   --> /home/syed/.cargo/registry/src/github.com-1ecc6299db9ec823/swim-0.2.1/src/lib.rs:166:9
    |
166 | pub use swim_core::async_trait;
    |         ^^^^^^^^^^^^^^^^^^^^^^ no `async_trait` in the root
    |
help: consider importing this unresolved item instead
    |
166 | pub use crate::prelude::async_trait;
    |         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~

error[E0432]: unresolved import `swim_core::hyper`
   --> /home/syed/.cargo/registry/src/github.com-1ecc6299db9ec823/swim-0.2.1/src/lib.rs:167:9
    |
167 | pub use swim_core::hyper;
    |         ^^^^^^^^^^^^^^^^ no `hyper` in the root

error[E0432]: unresolved import `swim_core::routerify`
   --> /home/syed/.cargo/registry/src/github.com-1ecc6299db9ec823/swim-0.2.1/src/lib.rs:168:9
    |
168 | pub use swim_core::routerify;
    |         ^^^^^^^^^^^^^^^^^^^^ no `routerify` in the root

Some errors have detailed explanations: E0432, E0433.
For more information about an error, try `rustc --explain E0432`.
error: could not compile `swim` due to 5 previous errors

The solution seems to be directly depending on specific version strings. Including the patch component.

Fixed by pinning swim_core to 0.2.1.
v0.2.2 released in a64a320dd3832b9759b4198230e1e1e1e3e90317


Document question answering returns empty list without an error

Describe the bug
The document_question_answering method silently fails by returning an empty list. Same for visual_question_answering when set with a wrong model, like
  client = InferenceClient(
      model="impira/layoutlm-document-qa",
  )

Reproduction
from huggingface_hub import InferenceClient
client = InferenceClient()
image = "data/gaia/validation/cca530fc-4052-43b2-b130-b30968d8aa44.png"
question = "On what number of row and column is the white king?"
client.document_question_answering(image=image, question=question)

output: []

Logs
No response
System info
- huggingface_hub version: 0.22.2
- Platform: macOS-14.1-arm64-arm-64bit
- Python version: 3.11.6
- Running in iPython ?: No
- Running in notebook ?: No
- Running in Google Colab ?: No
- Token path ?: /Users/aymeric/.cache/huggingface/token
- Has saved token ?: True
- Who am I ?: m-ric
- Configured git credential helpers: osxkeychain
- FastAI: N/A
- Tensorflow: N/A
- Torch: 2.2.2
- Jinja2: 3.1.2
- Graphviz: N/A
- keras: N/A
- Pydot: N/A
- Pillow: 10.2.0
- hf_transfer: N/A
- gradio: 4.22.0
- tensorboard: N/A
- numpy: 1.26.3
- pydantic: 2.5.0
- aiohttp: 3.9.3
- ENDPOINT: https://huggingface.co
- HF_HUB_CACHE: /Users/aymeric/.cache/huggingface/hub
- HF_ASSETS_CACHE: /Users/aymeric/.cache/huggingface/assets
- HF_TOKEN_PATH: /Users/aymeric/.cache/huggingface/token
- HF_HUB_OFFLINE: False
- HF_HUB_DISABLE_TELEMETRY: False
- HF_HUB_DISABLE_PROGRESS_BARS: None
- HF_HUB_DISABLE_SYMLINKS_WARNING: False
- HF_HUB_DISABLE_EXPERIMENTAL_WARNING: False
- HF_HUB_DISABLE_IMPLICIT_TOKEN: False
- HF_HUB_ENABLE_HF_TRANSFER: False
- HF_HUB_ETAG_TIMEOUT: 10
- HF_HUB_DOWNLOAD_TIMEOUT: 10

Will look into it, thanks for reporting :+1:

Would it be possible to have more details on:

what's your input? (a real image example)
what would you expect from the client?

Thanks in advance!

Hello @Wauplin!
The input is a real image.
I'd prefer the output to be either a completion, or an error.
For instance if I use task visual_question_answering, as in:
from huggingface_hub import InferenceClient
client = InferenceClient()
image = "https://raw.githubusercontent.com/huggingface/text-generation-inference/main/integration-tests/images/chicken_on_money.png"
question = "What is on the image?"
client.visual_question_answering(image=image, question=question)

I get this completion, which is the desired behaviour for me:
[VisualQuestionAnsweringOutputElement(label=None, score=0.844512939453125, answer='rooster'),
 VisualQuestionAnsweringOutputElement(label=None, score=0.6880057454109192, answer='chicken'),
 VisualQuestionAnsweringOutputElement(label=None, score=0.043415617197752, answer='turkey'),
 VisualQuestionAnsweringOutputElement(label=None, score=0.026919541880488396, answer='yes'),
 VisualQuestionAnsweringOutputElement(label=None, score=0.0256495401263237, answer='bird')]

But with document_question_answering I get an empty list.
If document_question_answering does not support images it would be nice to get an informative error instead of an empty list.

Ok I understand better now.
So you provide an image without any text and expect document_question_answering to raise an error instead of returning an empty list? I don't find that returning an empty list is philosophically wrong TBH. If you ask a question, that the model is supposed to return a list of answers and that it didn't find any, then empty list makes sense.
In any case, this is not something that will be changed in InferenceClient and not even in Inference API. Those are simply forwarding inputs and outputs to / from transformers's pipeline. So I'll close this issue but feel free to open a new one in transformers if you feel strongly about it and think the document/visual question answering pipelines should be updated.

Thank you for your detailed answer! 🤗


change default horizontal padding of buttons to 20px

fixes #192
change-type: patch

Changed standard buttons' padding to 20px
Changed emphasized buttons' padding to 50px
Changed standard square buttons' min-width to 40px
Changed emphasized square buttons' min-width to 50px
Added story for emphasized square buttons

@konmouz This has broken the fixed width buttons -->


Allow specific nightly builds to be used

This allows a specific nightly build version to be used instead of forcing the action to use the latest nightly build.
Previously the normalizeVersionName function would transform any nightly version (like nightly-6c4c68a7031581bb8b8a10bb44db8dff4e04277f) to nightly, meaning we'd always use the nightly release. By removing this function and tweaking the url the github action now handles a specific version being passed in (like nightly-6c4c68a7031581bb8b8a10bb44db8dff4e04277f).  The default  versionnightly still works as before.
This would let us use a specific version of foundry in our CI and help keep our CI more consistent.

Closing this as I think I was wrong, I think the code functions correctly.


LPC55S69: TFM with MCUboot to run Zephyr NS

Features:

removes disabled BL2 macro, runs with MCUboot
automated building, signing and binary merging
single binary flash
partitioning to allow MCUboot upgrades
control build type as Minsizerel to make tfm_n smaller from Zephyr's cmake
disable building tfm_ns image as Zephyr is the non-secure image (currently tfm_ns fails linking as it's too large)

Depends on tfm module (also draft): https://github.com/zephyrproject-rtos/trusted-firmware-m/pull/13

Managed to configure Minsizerel when selecting LPC board.
By default don't build tfm_ns image.
Removing draft status.

Replaced by https://github.com/zephyrproject-rtos/zephyr/pull/30013 since it's incompatible with latest TFM repo.


Add prefix option to image column

Adds an optional prefix option to the image column, so it can work wit the prefix option defined in the image field.
Fixes #1054.

Totally agree - thank you @bul-ikana for the PR and everybody else for approving, it really helps!
Cheers!

Merged, on master, ready to use after a composer update.


unable to ssh to bastion vm created while CFCR deployment

CFCR deployment is created all resources as required in aws, but remote-exec provisioner in  kubo-infrastructure.tf is failing to ssh to the bastion VM created by terraform, which leading to timeout error
Any help would be appreciated thanks.
Error screen shot:
aws_instance.bastion: Still creating... (5m40s elapsed)

Error: Error applying plan:

1 error(s) occurred:

* aws_instance.bastion: 1 error(s) occurred:

* timeout

Terraform does not automatically rollback in the face of errors.
Instead, your Terraform state file has been partially updated with
any resources that successfully completed. Please address the error
above and apply again to incrementally change your infrastructure.

It's an permissions issue from our end, closing this issue


Highest rated Visual Studio extensions are based on VS 2019!

Maybe this category should be promoting VS 2022 extensions instead?

Can you please share more details , on the visual studio marketplace homepage we do have category pointing to VS 2022 extensions.

See More Link points to https://marketplace.visualstudio.com/search?sortBy=Rating&category=All categories&target=VS&vsVersion=vs2022 , with target as VS 2022 extensions.

Sure: On the frontpage the extensions are filtered to VS 2019, I think they should be filtered to VS 2022 these days.
You can confirm that the two lists are not identical:

Were you able to reproduce?

@ErikEJ Thanks for sharing more information. Yes we are able to reproduce the issue. Our team will be looking into it.

Thanks - for full disclosure, you can see that it is of quite some interest to me :smile:

@amit-kulkarni-in Any news on this?

@ErikEJ We will be taking this up in this sprint , will keep you posted on the progress.

@amit-kulkarni-in Any news? is "this sprint" not over now?

@joelverhagen Why was this not prioritized?

@joelverhagen Thanks a lot for the open feedback. Highly appreciated, and thanks for supporting VS marketplace as a free publishing platform

Cc @madskristensen FYI

I very much support this idea and it is related to one (https://github.com/microsoft/vsmarketplace/issues/465) I opened a while back which has been fixed.

@madskristensen wauw. So Trending was fixed but not Highest rated. Too bad.

@joelverhagen Just checking in if there are any plans to fix this - you mentioned the word "short term" in May this year :-D

Hey @ErikEJ, yes, I think I was a bit optimistic with saying "short term". I apologize for the delay, but our team remains constrained on our ability to fix all of the reported bugs and features improvements. Unfortunately, this bug hasn't made the cut.
However, I took some time this morning to identify the root cause. I found the problem and have left notes for whoever fixes the bug in the future. The root cause was we have a feature flag enabling 2019 extension filtering for the front-page extension list. This code path was never updated for the VS 2022 release. We should probably add a 2022 feature flag with a respective VS 2022 filter. Or this might be as simple as updating a "VS version = 16.0" to "VS version = 17.0" in our code reusing the same feature flag but that would be hacky. I still can't guarantee when this will be delivered, unfortunately.

@joelverhagen @madskristensen @amit-kulkarni-in I did some investigation recently, caused by another extender not very pleased with the UX, and it turns out that also the "Most popular" list is based on VS 2019 😢
Could we please, please get a fix for this, it is very demotivating to author an extension to then be faced with this bad UX that does NOT help promote new extensions.
//cc @giorgi

@madskristensen Also, how is "Trending" actually fixed, looks like some very odd extensions in thatr list (maybe old 2019 extensions? 😆 )

@joelverhagen Your kind reply is now getting around one year old - can you share any plans or roadmap for fixing this?


Font size

Hi,
i wanted to point out that w.r.t. the default "Github" theme, the size of the font is smaller; would it be possible to increase it as the one of github? Yes, your answer would be to set a different font size within the settings, and that would be perfectly fine if not for the fact that it increases the size of all the characters, and i believe that in this theme the dimension of code chunks' is perfect, and unfortunately it will be altered.
So, my question is, would it be possible to increase just the size of the plain text?

Hey! The default font is set to 14px, since I always use a custom font size I didn't realize that other themes used a large default. The Github theme uses 16px, so I will set the default font to 16px to match that. I'll add some more fixes and make a new release.
Thank you for your comment and hope that helps!


wasm: Question about ExportedType.IsForwarder

I'm trying out ilc for wasm on some larger programs, and I'm hitting
https://github.com/dotnet/corert/blob/5056fd45b076277579c84833060447415b7b9403/src/Common/src/TypeSystem/Ecma/EcmaModule.cs#L305
for a type that has IsForwarder == true.  (Type is System.Threading.PreAllocatedOverlapped and it looks in the implementation for System.Private.CoreLib).  Am I right in thinking this is for types that are forwarded to another assemby using System.Runtime.CompilerServices.TypeForwardedTo?  If so I suppose what could be happening is that the module which now contains the type, is not getting this type added through the dependency analysis as the actual reference is to the type when it was in the "older" module.  Does that make sense?

This problem is not specific to WASM - it seems like CoreLib doesn't have a definition for the PreAllocatedOverlapped class outside Windows (essentially, CoreLib is "corrupted"). We need a Unix version for WASM.
I'll cc a couple people who might know more about if there's any plans for it: @filipnavara @marek-safar

I'll look into it.

Do you have complete stack trace for the usage of PreAllocatedOverlapped?
It seems that both ThreadPoolBoundHandle and PreAllocatedOverlapped are missing on Unix implementation of CoreLib. Overlapped is due for moving to the shared CoreLib part and the CoreCLR implementation could probably be reused for most of the other classes with few modifications.
The thing that worries me a little is the fact that I wouldn't expect the code to be hit at all since the framework itself should not use it on non-Windows systems. ThreadPoolBoundHandle.BindHandle is not supported on Unix and neither is the overlapped I/O.

@filipnavara both types are part of netstandard so they will be required everywhere but could be PNSE

@marek-safar Yeah, that's totally understandable. I'm asking for the stack trace to identify whether there's a secondary problem where the Windows code paths are hit on WASM.

they will be required everywhere but could be PNSE

Right, Overlapped structure and everything related to it is very Windows specific. It should be fine to throw PNSE on non-Windows. I do not know the history why CoreCLR has some of it implemented on Unix.

@jkotas Out of curiosity, do you know whether the special GC behavior for OverlappedData in CoreCLR (https://github.com/dotnet/coreclr/blob/fd3afcaf0dc0ace3f84d0412c128f4e64c419105/src/vm/gcenv.ee.cpp#L1397-L1464) is still needed for some reason? CoreRT implements the same behavior using regular GCHandles and I don't see a reason not to do the same on CoreCLR. It looks like there could have been some historical reason related to appdomain unloading w/ pending overlapped I/O, but that doesn't seem to matter anymore. The user could also modify the array pointed by OverlappedData->m_userObject in which case it would behave differently, but that's likely not allowed and was never tested either AFAICT.

The CoreRT implementation has different performance characteristics. We would need to check that it does not regress anything important. Otherwise, I do not see a problem.

Ill post the stack tomorrow . From what I remember  it was something to do with HTTP (S?), and did have something Windows looking in one of the frames.

 That definitely points out to a secondary problem. The overlapped I/O APIs are referenced from SocketAsyncEventArgs implementation for Windows, so it looks like the CoreFX part incorrectly uses the Windows version of the code.

My question is answered, so happy to close this.  Do you want me to raise another issue in (corefx?) about why SocketAsyncEventArgs.Windows.cs is being included ?

@yowl I think it should be tracked somewhere.
Please don't close this issue just yet, I will submit patches to fix CoreLib and then it can be closed.

ok, thanks a lot.

why SocketAsyncEventArgs.Windows.cs is being included ?

We do not have the proper build of corefx for WebAssembly.
However, doing something about Overlapped definition in CoreLib would be a good idea.

We do not have the proper build of corefx for WebAssembly.

I thought there is one that is based on the Unix profile. I've seen it referenced couple of times in PRs and the source code.
https://github.com/dotnet/corefx/search?q=webassembly&unscoped_q=webassembly

I thought there is one that is based on the Unix profile

There was some work done on it, but it is not wired through the system.

Is it these kind of additions that would be required?
https://github.com/dotnet/corefx/compare/master...morganbr:WebAssemblyFlavor?expand=1
I guess they're not right/complete or @morganbr  would have PR'ed them.

@yowl, I think that got merged in https://github.com/dotnet/corefx/pull/28936/files. There's no automated build for it (and I would assume something's broken by now).

@morganbr Thanks!  While I've got you can you say if this branch is relevant to this discussion: https://github.com/morganbr/corefx/tree/WasmAot

Yes, that branch would contribute to being able to build and package the corefx tree for webassembly.

https://github.com/dotnet/corert/pull/7239 does indeed fix this so will close.


Add dl-libxcb feature

Enabling this feature will prevent from libxcb being linked to the resulting executable. Instead libxcb will be dynamically loaded at runtime.
This allows to compile an application with different graphic backends that may run with another backend if libxcb is not installed on the system.
Fixes https://github.com/psychon/x11rb/issues/522.

Just for future me when I want to see why libloading does not work on 1.37:
error[E0658]: non exhaustive is an experimental feature
  --> /home/travis/.cargo/registry/src/github.com-1ecc6299db9ec823/libloading-0.6.3/src/error.rs:23:1
   |
23 | #[non_exhaustive]
   | ^^^^^^^^^^^^^^^^^
   |
   = note: for more information, see https://github.com/rust-lang/rust/issues/44109


Purge option for Telegram.Bot

Coverage remained the same at 100.0% when pulling 744f702a54ed978aa67a04c76eb61f945a409070 on start-clean into 11df1d5c0bd39c41092a2de7a8ef482cc0c951ca on master.


⚠ Processors has degraded performance

In c7df231, Processors ($STATUS_URL) experienced degraded performance:

HTTP code: 200
Response time: 120 ms

Resolved: Processors performance has improved in 50ea1ff.


Logging should be clearer when service catalog fails to connect to etcd

I encountered a scenario where the SC failed to connect to etcd using TLS because the etcd certificate name did not match the connection hostname.
There was no obvious logging at loglevel 10 and the SC /healthz and /healthz/etcd endpoints did not indicate an issue.  The SC itself just seemed to be hanged and API requests returned errors like Error from server (Forbidden): clusterservicebrokers.servicecatalog.k8s.io "foo" is forbidden: not yet ready to handle request.
It would be good to add loggging -

to show etcd connection failures and detail
on /healthz
to be clearer about which admission controller is failing, when one is failing

@pmorie @jboyd01

I didn't, no.  I wonder if the etcd logging is just all disabled.  Perhaps enabling it when the -v loglevel is above a certain level, or something like that, might help?

I may be hitting this issue myself, still debugging.  However I do see that the test that is under CheckEtcdServers() is pretty lame - it just does a dial on the address:port and if the dial is accepted the connection is closed and thumbs up.  Certainly could be stronger.


Swith contribution from Org to user  (TypeORM Collective)

Tyler made a contribution to https://opencollective.com/typeorm but he is showing as an organization instead of a user. He would like to be shown as a user.

done


PoC-7 support

Look here, when it's finalized: https://github.com/ethereum/pyethereum/compare/poc7

Want to back this issue? Place a bounty on it! We accept bounties via Bountysource.

Make contracts input/output-aware?!

https://github.com/ethereum/wiki/wiki/Serpent tells us how to make our own macros, so we can do something like:
macro send_asset($assetid, $to, $amount):
    x = alloc(97)
    mstore(ref(x)+1, $assetid)
    mstore(ref(x)+33, $to)
    mstore(ref(x)+65, $amount)
    ~call(tx.gas/2, 4, 0, x, 97, 0, 0)

This converts "send_asset(x, y, z)" into a contract call to address 4 using x,y,z as arguments. We would then create a file, "counterparty.macro", that contains this macro and any other macros we want, and instruct users to do 'inset("counterparty.macro")'.
Then, the next step would be to modify https://github.com/ethereum/pyethereum/blob/develop/pyethereum/specials.py and all a similar special for address 4, where inside a proc_counterparty function we would actually go and modify the Counterparty database state.


Search finds apps with no installation candidate

Multiple times I have searched and found appimages and then when I sent to install I got no installation candidate warnings.
Example:
zap search nextcloud
zap install nextcloud

That's unfortunately because https://github.com/AppImage/appimage.github.io/commits/master/apps/Nextcloud.md hasn't been updated in a long time - and - https://appimage.github.io/Nextcloud/  (and here https://github.com/AppImage/appimage.github.io/blob/master/data/Nextcloud), does not have a download button.
And hence, zap uses this index to download and provide metadata.
You will have to, pass the URL manually
zap install --from <url_to_nextcloud_appimage> nextcloud


Refactor boundary conditions container

This PR refactors the boundary conditions container to reflect the fact that there are three boundary conditions: a tuple of FieldBoundaryConditions for the solution, a tuple of FieldBoundaryConditions for the tendencies, and a single FieldBoundaryConditions for the pressures (which is also used to set boundary conditions on diffusivities.
To reflect these facts, this model creates a new tuple called SolutionBoundaryConditions to hold the boundary conditions for solution. ModelBoundaryConditions then becomes a named tuple with fields solution, tendency, and pressure.
The model constructor may be passed either a SolutionBoundaryConditions (preserving existing behavior), or a ModelBoundaryConditions (convenient for checkpointing). Ditto for PoissonBCs --- though technically we don't need PoissonBCs anymore; we simply need to dispatch on the type of model.boundary_conditions.pressure (work for the future).
Previously, a new instance of boundary conditions for pressure was created every time-step. This PR avoids that unnecessary cost / allocation. It also moves boundary conditions out of timestepper, slightly changing the time stepper struct.
@ali-ramadhan, the tests pass so I think the checkpointer works, but it'd be worth thinking about whether its doing the right thing now.

Sounds like the plan is to switch to SolutionBoundaryConditions but that might involve more refactoring so maybe best left for another PR?

This is already done, because the model constructor calls the function ModelBoundaryConditions here:
https://github.com/climate-machine/Oceananigans.jl/blob/e41711f6b5db492d3aa5344d2e3bc9c2fe65dd70/src/models.jl#L64
which then dispatches based on whether the argument boundary_conditions is a SolutionBoundaryConditions or a ModelBoundaryConditions. Thus in either case the model is constructed with a ModelBoundaryConditions.

Seems like there are now two options for what model.boundary_conditions could be

There are not two options --- for time_step! to work correctly, model.boundary_conditions must be ModelBoundaryConditions.

On checkpointing, looks like the current Checkpointer will work as it just serializes the entire model.boundary_conditions struct which isn't a problem if it contains no functions. Although this does suggest that we need at least two extra checkpointer tests: 1. one where checkpointed boundary conditions are used, and 2. one where a boundary condition contains a function that we have manually restore. This can be the focus of another PR.

Yes, because the keyword argument can be either a SolutionBoundaryConditions or a ModelBoundaryConditions, checkpointing works properly.
We do need to extend the checkpointer to allow the user to pass in forcing functions and boundary conditions to restore_from_checkpoint in the case that these objects contain functions?

We should note the difference in the documentation to the Model constructor.

+1

We do need to extend the checkpointer to allow the user to pass in forcing functions and boundary conditions to restore_from_checkpoint in the case that these objects contain functions?

Don't think so as you can already pass in whatever kwargs you want to restore_from_checkpoint. But would be good to have a unit test for this functionality.

Don't think so as you can already pass in whatever kwargs you want to restore_from_checkpoint. But would be good to have a unit test for this functionality

Ah, great.

@ali-ramadhan check the new docstring.

Awesome! Although it does not belong in this PR.
Looking like it won't take much work to get the public API documented.

Awesome! Although it does not belong in this PR

Bleh, it is slightly relevant because of the changed behavior of boundary_conditions, sheesh!

@ali-ramadhan I will merge once tests pass.