[63] Querying search terms (keywords)

Closes #63
Implements simple keyword and key phrase search.
Usage examples are shown in examples/06_query_terms.pyand in the README.
NB: To run the notebook, you will need to first fetch updated, preprocessed data. Run the following from the terminal:
aws s3 sync s3://innovation-sweet-spots-lake/outputs/finals/pilot_outputs outputs/finals/pilot_outputs

Checklist:

[x] I have refactored my code out from notebooks/
[x] I have checked the code runs
[x] I have tested the code
[x] I have run pre-commit and addressed any issues not automatically fixed
[x] I have merged any new changes from dev
[x] I have documented the code

[x] Major functions have docstrings
[x] Appropriate information has been added to READMEs

[x] I have explained the feature in this PR or (better) in output/reports/
[x] I have requested a code review

Hi @Jack-0-0 , hope it's OK for you to review this PR. Apologies, it's a bit heftier than usual.
The main additions are:

module for keyword search query_terms.py
getter for preprocessed data getters/preprocessed.py

Great, thanks so much @Jack-0-0 !


deps: Bump build_bazel_rules_apple -> 0.32.0

Signed-off-by: Ryan Northey<EMAIL_ADDRESS>Commit Message:
Additional Description:
Risk Level:
Testing:
Docs Changes:
Release Notes:
Platform Specific Features:
[Optional Runtime guard:]
[Optional Fixes #Issue] Fix #18839
[Optional Fixes commit #PR or SHA]
[Optional Deprecated:]
[Optional API Considerations:]

/lgtm deps


Filtering Empty Objects in SDK to Avoid Invalid Resource Generation/Apply

What happened?
When using for example function-kcl to create resources, the following pattern can lead to the generation of invalid resources that result in client-side crossplane beta render or server-side errors during the application:
 _items = [{
    apiVersion: "s3.aws.upbound.io/v1beta1"
    kind: "Bucket"
    metadata: _metadata("bucket")
    spec: _defaults
}]

_items += [{
    apiVersion: "s3.aws.upbound.io/v1beta1"
    kind: "BucketVersioning"
    metadata: _metadata("bucketVersioning")
    spec: _defaults | {
        forProvider: {
            bucketSelector: {
                matchControllerRef: True
            },
            versioningConfiguration: [
                {
                    status: "Enabled"
                },
            ],
        },
    }
} if get(oxr, "spec.parameters.versioning", "False") else {}]

items = _items

This approach can produce an invalid resource object like the one shown below if the conditional logic produces an empty object ({}):
---
metadata:
  annotations:
    crossplane.io/composition-resource-name: ""
  generateName: configuration-
  labels:
    crossplane.io/composite: configuration
  ownerReferences:
  - apiVersion: example.org/v1alpha1
    blockOwnerDeletion: true
    controller: true
    kind: Template
    name: configuration
    uid: ""

The issue arises because the condition for else produces {}, which is added to the _items list. This can lead to errors during the resource application process.
To mitigate this, every individual composition currently requires an additional filtering step:
items = [
    i for r in [_items]
    for i in r if i
]

Can the SDK be enhanced to automatically filter out these invalid empty object resources? This would simplify the composition logic and prevent potential errors during resource application.
This issue has been discussed in the KCL community. You can find the discussion here: https://cloud-native.slack.com/archives/C05TC96NWN8/p1723020486395609

How can we reproduce it?

What environment did it happen in?
Crossplane version:

(I chatted with @haarchri on Slack, but capturing my thoughts here for posterity.)
My understanding of the problem is that KCL doesn't support "append X to this array if the condition is met, else append nothing". Instead it only supports "append X to this array if the condition is met, else append Y". So in the context of a composition function this means "append this resource to desired resources if the condition is met, otherwise append the empty object {}".
The SDK then sends the empty object to Crossplane, which fills in some required fields (generate name, owner refs, etc) resulting in the invalid object that can't be applied.
Fixing this at the function SDK level feels like fixing a KCL limitation at the function SDK level. As a KCL outsider it seems ideal to me to fix it at the language level. In the meantime it's possible to address this in KCL configuration using the for i in r if i workaround. I actually think it's better to do that explicitly in the KCL code than have it happen automatically at the function SDK level. Generally explicit is better than implicit/magical. Yes you need to write a little boilerplate, but that makes it really obvious what's happening to anyone who reads the config file. They can immediately see that empty resources are being filtered out.
The other concern with making a change like this at the SDK level is that there's more than on function SDK. This one's the most popular, but there's also https://github.com/crossplane/function-sdk-python/, https://github.com/crossplane/function-sdk-java and in future others. We'd need to encode this behavior into all the SDKs for consistency, or it'd be even more surprising to users.

@haarchri This seems to work fine:
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: function-template-go
spec:
  compositeTypeRef:
    apiVersion: example.crossplane.io/v1
    kind: XR
  mode: Pipeline
  pipeline:
  - step: normal
    functionRef:
      name: kcl-function
    input:
      apiVersion: krm.kcl.dev/v1alpha1
      kind: KCLRun
      metadata:
        name: basic
      spec:
        source: |
          import base64

          oxr = option("params").oxr
          count = oxr.spec.count or 1
          ocds = option("params").ocds

          dxr = {
              **oxr
              status.dummy = "cool-status"
          }

          _items = [{
              apiVersion: "s3.aws.upbound.io/v1beta1"
              kind: "Bucket"
              metadata: {
                annotations: {
                  "krm.kcl.dev/composition-resource-name": "a"
                }
              }
          }]

          if oxr.spec.parameters.versioning == "True":
            _items += [{
                apiVersion: "s3.aws.upbound.io/v1beta1"
                kind: "BucketVersioning"
                metadata: {
                  annotations: {
                    "krm.kcl.dev/composition-resource-name": "b"
                  }
                }
            }]

          items = _items

The BucketVersioning is appended to desired resources if spec.parameters.versioning == "True". Otherwise nothing is appended.


Adding APRSIS_PASSCODE

Allows to specify APRS passcode to connect to servers not allowing unverified connections.

Will do a new PR, missed one thing!


Nilaway reports impossible panic on http.Client

package main

import (
	"fmt"
	"net/http"
)

func main() {
	c := http.DefaultClient

	resp, err := c.Get("http://localhost:8080")
	if err != nil {
		panic(err)
	}

	fmt.Println(resp.StatusCode)
}

outputs:
❯ nilaway .
/Users/joe/workspace/nilawaytest/main.go:16:14: error: Potential nil panic detected. Observed nil flow from source to dereference point:
        -> http/client.go:644:12: unassigned variable `resp` returned from `do()` in position 0
        -> http/client.go:590:9: result 0 of `do()` returned from `Do()` in position 0
        -> http/client.go:488:9: result 0 of `Do()` returned from `Get()` in position 0
        -> nilawaytest/main.go:16:14: result 0 of `Get()` accessed field `StatusCode`

In client.go 's Do() function, there's a loop for processing redirects:
	var resp *Response
	for {
		// For all but the first request, create the next
		// request hop and replace req.
		if len(reqs) > 0 {
			loc := resp.Header.Get("Location")
			if loc == "" {
				// While most 3xx responses include a Location, it is not
				// required and 3xx responses without a Location have been
				// observed in the wild. See issues #17773 and #49281.
/*Line 644 -->*/		return resp, nil
			}
			u, err := req.URL.Parse(loc)
			if err != nil {
				resp.closeBody()
				return nil, uerr(fmt.Errorf("failed to parse Location header %q: %v", loc, err))
			}
			host := ""
			if req.Host != "" && req.Host != req.URL.Host {
				// If the caller specified a custom Host header and the
				// redirect location is relative, preserve the Host header
				// through the redirect. See issue #22233.
				if u, _ := url.Parse(loc); u != nil && !u.IsAbs() {
					host = req.Host
				}
			}
			ireq := reqs[0]
			req = &Request{
				Method:   redirectMethod,
				Response: resp,
				URL:      u,
				Header:   make(Header),
				Host:     host,
				Cancel:   ireq.Cancel,
				ctx:      ireq.ctx,
			}
			if includeBody && ireq.GetBody != nil {
				req.Body, err = ireq.GetBody()
				if err != nil {
					resp.closeBody()
					return nil, uerr(err)
				}
				req.ContentLength = ireq.ContentLength
			}

			// Copy original headers before setting the Referer,
			// in case the user set Referer on their first request.
			// If they really want to override, they can do it in
			// their CheckRedirect func.
			copyHeaders(req)

			// Add the Referer header from the most recent
			// request URL to the new one, if it's not https->http:
			if ref := refererForURL(reqs[len(reqs)-1].URL, req.URL, req.Header.Get("Referer")); ref != "" {
				req.Header.Set("Referer", ref)
			}
			err = c.checkRedirect(req, reqs)

			// Sentinel error to let users select the
			// previous response, without closing its
			// body. See Issue 10069.
			if err == ErrUseLastResponse {
				return resp, nil
			}

			// Close the previous response's body. But
			// read at least some of the body so if it's
			// small the underlying TCP connection will be
			// re-used. No need to check for errors: if it
			// fails, the Transport won't reuse it anyway.
			const maxBodySlurpSize = 2 << 10
			if resp.ContentLength == -1 || resp.ContentLength <= maxBodySlurpSize {
				io.CopyN(io.Discard, resp.Body, maxBodySlurpSize)
			}
			resp.Body.Close()

			if err != nil {
				// Special case for Go 1 compatibility: return both the response
				// and an error if the CheckRedirect function failed.
				// See https://golang.org/issue/3795
				// The resp.Body has already been closed.
				ue := uerr(err)
				ue.(*url.Error).URL = loc
				return resp, ue
			}
		}

		reqs = append(reqs, req)
		var err error
		var didTimeout func() bool
		if resp, didTimeout, err = c.send(req, deadline); err != nil {
			// c.send() always closes req.Body
			reqBodyClosed = true
			if !deadline.IsZero() && didTimeout() {
				err = &httpError{
					err:     err.Error() + " (Client.Timeout exceeded while awaiting headers)",
					timeout: true,
				}
			}
			return nil, uerr(err)
		}

		var shouldRedirect bool
		redirectMethod, shouldRedirect, includeBody = redirectBehavior(req.Method, resp, reqs[0])
		if !shouldRedirect {
			return resp, nil
		}

		req.closeBody()
	}

It's impossible for resp to be nil on line 644, because the Header access on line 639 (resp.Header.Get("Location")) would result in a nil panic.

Funny, I was just digging through the same false positive. Which happened right after going through https://github.com/uber-go/nilaway/issues/97. And I was like: http.Client and context.Context are so ubiquitous, this surely can't be a false positive.

Duplicate of #126.

Closed as a duplicate of https://github.com/uber-go/nilaway/issues/126. Let's continue the discussion there! Thanks for reporting such errors!


When scanlator groups are filtered out, they are still downloaded when "Download Unread" chapters is selected

Device information

Neko version: <IP_ADDRESS>
Android phone & version: Android 12 Samsung S21+

If you filter out scanlator groups and go to download the unread chapters through the "Next Unread" or "All Unread", the groups filtered out are still downloaded. This behaviour is also reflected in the reader, when the chapter is finished and you go to the next chapter, the next chapter would be of the group filtered instead of skipping that chapter and going to the next available not filtered chapter.

Added in 2.9.0


Dev kmr

lgtm - this gives a list of all supported benchmarks for a given runner. Is it possible to give a benchmark input, say "oslat", and check if its supported ? I think its doable now with this PR. This would a be another way of adding jobs, instead of a matrix, you specify one benchmark that you want to run on a runner. A 3rd way would be: run this benchmark xyz on a supported runner, no matter how.

The primary thing that this adds is it couples the benchmark and the endpoints together and allows for tags to be factored in as well.  Currently we have separate actions for benchmarks and endpoints and no support for tags at all. This leads to problems:

When the benchmarks and endpoints are separate it's impossible to eliminate the generation of certain combinations (such as "remotehost" and "oslat" for the github runners) without explicitly writing an exclusion rule in the workflow.
Without support for factoring tags into the decision making it is impossible support different types of self-hosted runners for different purposes.

I'm unsure right now on some of the benchmark based input ideas you suggest.  I think we may be able to get there eventually, but it may have to be iterative as we work through some of the complications.  For example, you can't just run "benchmark xyz on a supported runner, no matter how" without having explicitly added a job for that runner to a workflow and have it reference the appropriate inputs.  Essentially the workflows need some defined jobs for different runner environments and then within those jobs we can scale the different scenarios automatically through action output and matrix strategies.
For example, I want to add support for running cyclictest in crucible-ci and the proposed changes here would make that possible with the following changes only:

add cyclictest support to the integration-tests action
add cyclictest scenario generation to the appropriate places in the get-scenarios action (k8s endpoint on github runners, remotehost endpoint on self hosted runners)

With the crucible-ci workflow written as the PR suggests then the above changes would automatically add cyclictest running on k8s endpoint on github hosted runners and cyclictest running on remotehost endpoint on self hosted runners without any change to the workflow itself.  Since the workflow is the piece that has to be replicated into every single repo that we use crucible-ci for the more logic we can move to actions instead of the workflow itself the better for making changes across the entire scope of crucible-ci usage.


proposal: text/template: make and/or work like &&/||

Note that the documentation is correct regarding this, so I'm not claiming this is a bug/regression etc. But I suspect the original restriction was motivated by "simpler to implement" -- and since this has become increasingly painful for me lately, I'm testing the water with this issue.
There are at least 3 good arguments in favor of my case:

To avoid nil-pointer situations.
To avoid expensive data initialization when not needed.
The behavior would match the most common boolean AND and OR operators in Go. The current situation is very surprising to most people.

package main

import (
	"io/ioutil"
	"log"
	"text/template"
)

type I interface {
	B() bool
}

type T struct {
}

func (t *T) B() bool {
	return true
}

func main() {
	var t I
	templ, _ := template.New("").Parse("{{ if and .Foo .Foo.B }}TRUE{{ end }}")
	if err := templ.Execute(ioutil.Discard, map[string]interface{}{"Foo": t}); err != nil {
		log.Fatal(err)
	}

}

https://play.golang.org/p/gGdMuJ3-3er
Prints:
2019/03/28 08:45:28 template: :1:19: executing "" at <.Foo.B>: nil pointer evaluating interface {}.B

The execution engine really isn't set up to do this. It's not the answer you want, but use a nested if to avoid unwanted evaluation.

The execution engine really isn't set up to do this. It's not the answer you want, but use a nested if to avoid unwanted evaluation.

I have read and understood the code in text/template, and even if it's "not set up to do this", it should not be too hard to "set it up" to do exactly that. And note that I didn't create this issue only to make the "template life" easier for myself, but I'm on the receiving end of issues/questions from tens of thousands of Go template users, which I think you need to consider before brushing this off as a non-important issue. Because it is.

Please, I am not "brushing this off as a non-important issue". I am saying that the language evaluates all its arguments before calling a function and, even if it's easy to change (I don't think it is), it is unwise to make such a significant change to the language. The documentation does not offer the feature of early out for binary boolean operators, so it's not a bug. And as I wrote, although it's more inconvenient it's not difficult to solve the underlying issue by rewriting the template.
I believe the regularity of the current model is worth preserving.
For the actual example you posted, it's possible there's an unrelated bug. People have been fiddling with nil evaluation in the template engine and it may be that evaluation of methods with nil receivers has been broken, or never worked.

This would be a backwards incompatible change.
Rather, why not introduce some syntactic sugar where  "{{ if .Foo && if .Foo.B }}TRUE{{ end }}" gets expanded to  {{ if .Foo }}{{ if .Foo.B }}TRUE{{ end }}{{ end }} ?  That would make nil easier to handle and require no changes to the execution model.

@mvdan also since improvements have been made to error messages relating to this: https://github.com/golang/go/issues/29137

To avoid nil-pointer situations.

Do users misunderstand the nil pointer error and thus file issues on Hugo? I'm asking because perhaps making the errors a bit clearer could avoid confusion amongst users, without the need to make backwards incompatible changes.

To avoid expensive data initialization when not needed.

A template always evaluates all arguments at once, so I don't think this should surprise any user with some experience with the template package. Perhaps it's surprising at first, but we can't change this without breaking backwards compatibility or adding new syntax (see below).

The behavior would match the most common boolean AND and OR operators in Go.

I think you answered your own question here; and and or are functions, not operators with their own syntax that change how the arguments are treated. We have a few scenarios:

Changing all calls to evaluate arguments lazily. I think this would break far too many templates.
Changing the and/or functions to evaluate arguments lazily. This would make these calls inconsistent with others, which would add confusion, I think.
Start treating and/or as template language "keywords", as suggested by @bep above. Similar to option 2, but it would still be a breaking change.
Introduce new keywords or syntax, like @beoran's idea above. This is always an option, but it seems overkill given that we can already nest ifs.

I don't think any of these four scenarios is good even in the long run, so I lean towards smaller changes like improving the quality of errors.

For the actual example you posted, it's possible there's an unrelated bug. People have been fiddling with nil evaluation in the template engine and it may be that evaluation of methods with nil receivers has been broken, or never worked.

By the way, I tried the original code on Go 1.11, and it still failed. The nil pointer error did indeed change a bit, but it's largely still the same. If anyone finds a regression in a template that now errors when it didn't on older Go versions, please provide details.

@mvdan I wouldn't call it overkill when there are more than two conditions. Just compare {{ if .Foo && if .Foo.B && if .Foo.B.C }}TRUE{{ end }} with  {{ if .Foo }}{{ if .Foo.B }}{{ if .Foo.C }}TRUE{{ end }}{{ end }}{{ end }}. I'd say the former is just as clear as the latter, but more concise.

I would like to understand better (1) how much work it would be to change them, and (2) whether this would be a breaking change for real templates. Re (2) @beoran asserted that it would, but I can't see why.

Allow me to give evidence for my 'assertion'.
Now, it is possible to call functions and methods in templates, also as  as arguments to the and() function. If these functions or methods have side effects, then as it stands, the side effects of both functions will be performed. If we were to make and() shortcut evaluation, then in case the first function or method returns true, the second function's side effect will not be perfomed anymore. This breaks backwards compatibility as nothing in the documentation of text/template states that methods or functions used with it may not have side effects.
See this for an example:
https://play.golang.org/p/Zwkc1hIhBj5
package main

import (
	"fmt"
	"log"
	"text/template"
	"os"
)

type Data struct {
}

// Method with side effects
func (foo Data) Foo() bool {
	fmt.Println("Foo!")
	return false
}

// Another method with side effects
func (foo Data) Bar() bool {
	fmt.Println("Bar!")
	return false
}

// Simulate a shortcut and function
func Sand(object interface{}) bool {
  	data := object.(Data)
	return data.Foo() && data.Bar()
}

const TEMPLATE = `{{if and (.Foo) (.Bar)}}{{end}}{{if sand .}}{{end}}`

func main() {
	var data Data

	funcMap := template.FuncMap{
		// shortcut and
		"sand": Sand,
	}

	tmpl, err := template.New("andTest").Funcs(funcMap).Parse(TEMPLATE)
	if err != nil {
		log.Fatalf("parsing: %s", err)
	}

	err = tmpl.Execute(os.Stdout, data)
	if err != nil {
		log.Fatalf("execution: %s", err)
	}
}

Note the difference between the side effects of the template with and and the one with sand to see why they are not the same.

So, @beoran is right, this very unusual template construct might be considered to be broken with this change:
{{ if (and .InitA .InitB} }}
{{ end }}

But I think of this as a proposal with a net positive effect in this area: You may introduce 3 breaking changes, but you will fix 997 already broken template constructs.

Well, I don't know about the prevalence, I tried to search the web, but searching for {{if (or and {{if (and didn't yield much usable results. I am simply looking at it from a strict go1 compatibility promise point of view. Currently, the behavior as is specified. If the specs need to change, in such a backwards incompatible way, then I consider that this becomes a Go2 issue.

If the specs need to change, in such a backwards incompatible way, then I consider that this becomes a Go2 issue.

Or make this new (sensible) behaviour optional, which would make everyone happy.

Or make this new (sensible) behaviour optional, which would make everyone happy.

What would you envision the API looking like for this to be opt-in behavior?

What would you envision the API looking like for this to be opt-in behavior?

There is already an options API in place.
tmpl, _ := New("t1").Parse("{{ if (or .InitA .InitB) }}{{ end }}")
tmpl.Option("logicaloperators=legacy/sensible") // or something

Perhaps I'm missing something, but how would that work? You're passing the option after the parsing has completed. I don't think we want to add any form of lazy or delayed parsing to the package.

One way to provide options to be used by the parser is to put directives in the template itself. Not sure that we want that, just adding it as a possibility.

I spoke to Rob about this, and we are inclined to try changing the default to short-circuit or/and. It should fix far more than it breaks. Let's try this at the start of the Go 1.14 cycle, and if we need to add an option to get the old non-short-circuit operators back, we can do that.


if teams event remove < and > in notes before create htmldata

Hi,
this is a fast and dirty hack to restore urls in Microsoft Teams events.
This request could close issues #136 even if I think that it is necessary an ad hoc
parser for Microsoft Teams events

I appreciate this effort. I've attempted to address this issue in a more general way in the last release.


Add connection option for recursive triggers

Similar to foreign keys, the recursive triggers PRAGMA affects the
interpretation of all statements on a connection.

Coverage decreased (-0.09%) to 64.433% when pulling 848386d7a20cdd7b4314334b97e958c933a361ba on zombiezen:recursivetriggers into 9cbb0970444f7ee9ccc65376394a504150b25de1 on mattn:master.

Thank you


Recipe improvements

Checklist

[x] Used a fork of the feedstock to propose changes
[x] Bumped the build number (if the version is unchanged)
[ ] Reset the build number to 0 (if the version changed)
[x] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[x] Ensured the license file is being packaged.

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

@conda-forge-admin, please rerender

@conda-forge-admin, please rerender

Hi! This is the friendly automated conda-forge-webservice.
I tried to re-render for you, but it looks like there was nothing to do.

@isuruf any idea why only on OSX we get those warnings about transitive dependencies not being explicitly added in the run requirements?
Take for instance tbb, this is a private dependency of pagmo (in the sense that the public pagmo API does not make any reference to anything tbb-related, i.e., all the tbb usages in pagmo are within the .cpp files of the pagmo library). Thus, the pygmo module does not need to directly link to tbb, it will link to libpagmo.so which in turn links to libtbb.so.
The build process on OSX however seems to require that tbb is explicitly added as a run dependency, but there's no such warning on the Linux/Windows builds. Is this some difference in how the OSX linker works? Should we care about these warnings?

@mingwandroid, is this a bug in conda-build?

Transitive deps are never reported. TBB will be linked to whatever conda-build says it's linked to. If you don't want it linked to that then you probably need to take steps to figure out why it is being linked.

Saying something's private doesn't mean that the build system will make it so.

Saying something's private doesn't mean that the build system will make it so.

@mingwandroid would you mind elaborating a bit? Are you referring to the use of PRIVATE, PUBLIC, etc. attributes when using the target_link_libraries() command in CMake?
I think we do some overlinking in our project (pagmo) at the build level. For instance, regarding tbb, in the current conda package tbb is linked with the PUBLIC option when creating libpagmo, even though tbb is not part of the public API (all its uses are limited to the .cpp files of the library). I have a fix for this in https://github.com/esa/pagmo2/pull/373 , which makes the tbb dependency PRIVATE.
In case of nlopt and ipopt, however, we do include some of their headers in the public API, although what we need from those headers are only some type definitions. Thus, in the pagmo build system, the nlopt and ipopt targets (which include the compiled libraries for nlopt and ipopt) are linked to libpagmo as PUBLIC dependencies.
Would you recommend us to create separate targets in CMake for the headers and the compiled components of ipopt and nlopt, so that we have a public dependency on the headers but only a private one on the compile components?
And, last question, why is it that the conda build process in Linux/Windows is apparently able to determine and resolve the overlinking while that does not happen in OSX?

It is simply a requirement that any dso auto-loaded by any other dso or exe is declared as a direct dependency of the package correctly. This prevents conda from either not installing it or uninstalling it even it is needed. Ignore that you may consider it to be provided transitively. That could change when transitive deps change.
I'm not going to bring cmake into it because it's a completely general point and it applies to all build systems.
The use our non use of headers is irrelevant to this too because this refers to low level machine code and elf linking. Including a header does not immediately confer a need to also link a library. It could be header only or no functions could get called from that header.

It is simply a requirement that any dso auto-loaded by any other dso or exe is declared as a direct dependency of the package correctly.

Just to be clear, are you saying that if:

library B depends on library A, and
library C depends on library B,

then the recipe for C always needs to list A as a dependency? Or have I misunderstood your point?
Also, the fact that a dso is auto-loaded or not also depends on the current OS and compilation flags used (some of which are forced by the conda build process, as I understand). Does this mean that I need to make my dependencies in the recipe conditional on the OS?

No, I am saying the direct opposite of that and it is really simple. Declare what you use directly, do not declare anything you do not used directlyt.. DSO autoloading works consistently across all OSes. I'm not sure why you think otherwise.

You never declare transitive dependencies. Why would you? We know about them already by walking the tree of dependencies.
However here you have a DSO that is being used transitively, but actually, it's also being used directly.
You have 2 choices here:

Fix your build system implementation so that the thing that's being linked directly is not linked directly.
Add the package containing the "should be transitive only but isn't" as a direct dependency. This is not ideal as you don't actually call anything from it directly.

@mingwandroid Thanks a lot for the explanation. We are working towards removing the overlinking upstream, I'll merge this PR despite the current warnings on OSX in preparation for the next version.


Feature/sentry fixes

:tada: This PR is included in version 0.141.0-development.1 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:

:tada: This PR is included in version 0.141.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


'openURL:options:completionHandler:' is unavailable: not available on iOS (App Extension)

I'm getting this error whenever i try to build the app, it was working fine on 0.57.3, but after upgrading to the latest version (0.60.4), I started getting this error.

This is my podfile:
# Uncomment the next line to define a global platform for your project
# platform :ios, '9.0'
pod 'Firebase/Core'
pod 'Firebase/Database'
pod 'Firebase/Auth'
pod 'Firebase/Analytics'
pod 'Firebase/DynamicLinks'
pod 'Firebase/RemoteConfig'
pod 'Firebase/Storage'
pod 'Firebase/Performance'
pod 'Firebase/Messaging'
pod 'Fabric', '~> 1.7.5'
pod 'Crashlytics', '~> 3.10.1'
pod 'TwitterKit'
pod 'TwitterCore', '~> 3.0.2'
# pod 'FBSDKCoreKit'
pod 'GoogleSignIn'

target 'myapp' do
  # Uncomment the next line if you're using Swift or would like to use dynamic frameworks
  # use_frameworks!

  # Pods for mayapp
  pod 'React', :path => '../node_modules/react-native/'
  pod 'React-Core', :path => '../node_modules/react-native/React'
  pod 'React-DevSupport', :path => '../node_modules/react-native/React'
  pod 'React-fishhook', :path => '../node_modules/react-native/Libraries/fishhook'
  pod 'React-RCTActionSheet', :path => '../node_modules/react-native/Libraries/ActionSheetIOS'
  pod 'React-RCTAnimation', :path => '../node_modules/react-native/Libraries/NativeAnimation'
  pod 'React-RCTBlob', :path => '../node_modules/react-native/Libraries/Blob'
  pod 'React-RCTImage', :path => '../node_modules/react-native/Libraries/Image'
  pod 'React-RCTLinking', :path => '../node_modules/react-native/Libraries/LinkingIOS'
  pod 'React-RCTNetwork', :path => '../node_modules/react-native/Libraries/Network'
  pod 'React-RCTSettings', :path => '../node_modules/react-native/Libraries/Settings'
  pod 'React-RCTText', :path => '../node_modules/react-native/Libraries/Text'
  pod 'React-RCTVibration', :path => '../node_modules/react-native/Libraries/Vibration'
  pod 'React-RCTWebSocket', :path => '../node_modules/react-native/Libraries/WebSocket'

  pod 'React-cxxreact', :path => '../node_modules/react-native/ReactCommon/cxxreact'
  pod 'React-jsi', :path => '../node_modules/react-native/ReactCommon/jsi'
  pod 'React-jsiexecutor', :path => '../node_modules/react-native/ReactCommon/jsiexecutor'
  pod 'React-jsinspector', :path => '../node_modules/react-native/ReactCommon/jsinspector'

  # Explicitly include Yoga if you are using RN >= 0.42.0
  pod 'yoga', :path => '../node_modules/react-native/ReactCommon/yoga'

  # Third party deps podspec link
  pod 'DoubleConversion', :podspec => '../node_modules/react-native/third-party-podspecs/DoubleConversion.podspec'
  pod 'glog', :podspec => '../node_modules/react-native/third-party-podspecs/glog.podspec'
  pod 'Folly', :podspec => '../node_modules/react-native/third-party-podspecs/Folly.podspec'

  pod 'RNSVG', :path => '../node_modules/react-native-svg'

  pod 'RNFS', :path => '../node_modules/react-native-fs'

  platform :ios, '9.0'
  # pod 'react-native-fast-image', :path => '../node_modules/react-native-fast-image'

  pod 'TextToSpeech', :path => '../node_modules/react-native-tts'

  # pod 'react-native-splash-screen', :path => '../node_modules/react-native-splash-screen'

  pod 'Flurry-iOS-SDK/FlurrySDK'
  pod 'react-native-mixpanel', :path => '../node_modules/react-native-mixpanel'
  pod 'Mixpanel'
  pod 'CodePush', :path => '../node_modules/react-native-code-push'

  pod 'SentryReactNative', :path => '../node_modules/react-native-sentry'

  pod 'react-native-intercom', :path => '../node_modules/react-native-intercom'

end

target 'MyShareExt' do
  platform :ios, '9.0'
  # Explicitly include Yoga if you are using RN >= 0.42.0
  pod 'yoga', :path => '../node_modules/react-native/ReactCommon/yoga'

  # Third party deps podspec link
  pod 'DoubleConversion', :podspec => '../node_modules/react-native/third-party-podspecs/DoubleConversion.podspec'
  pod 'glog', :podspec => '../node_modules/react-native/third-party-podspecs/glog.podspec'
  pod 'Folly', :podspec => '../node_modules/react-native/third-party-podspecs/Folly.podspec'

  pod 'react-native-mixpanel', :path => '../node_modules/react-native-mixpanel'
  pod 'Mixpanel'
  inherit! :search_paths
end

post_install do |installer|
  installer.pods_project.targets.each do |target|
    if target.name == "yoga"
      target.remove_from_project
    end
  end

  installer.pods_project.build_configuration_list.build_configurations.each do |configuration|
    configuration.build_settings['CLANG_ALLOW_NON_MODULAR_INCLUDES_IN_FRAMEWORK_MODULES'] = 'YES'
  end
end

React Native version:
System:
    OS: macOS 10.14
    CPU: (4) x64 Intel(R) Core(TM) i5-6267U CPU @ 2.90GHz
    Memory: 44.29 MB / 8.00 GB
    Shell: 5.3 - /bin/zsh
  Binaries:
    Node: 8.15.0 - /usr/local/bin/node
    Yarn: 1.15.0-20190126.1635 - ~/.yarn/bin/yarn
    npm: 6.4.1 - /usr/local/bin/npm
    Watchman: 4.9.0 - /usr/local/bin/watchman
  SDKs:
    iOS SDK:
      Platforms: iOS 12.1, macOS 10.14, tvOS 12.1, watchOS 5.1
    Android SDK:
      API Levels: 22, 23, 24, 25, 26, 27, 28
      Build Tools: 23.0.3, 28.0.2
  IDEs:
    Android Studio: 3.1 AI-173.4907809
    Xcode: 10.1/10B61 - /usr/bin/xcodebuild
  npmPackages:
    react: 16.8.6 => 16.8.6
    react-native: 0.60.4 => 0.60.4
  npmGlobalPackages:
    react-native-cli: 2.0.1
    react-native-git-upgrade: 0.2.7

Steps To Reproduce

Add a share extension to the app.
build.

Is there any update here? I want to upgrade to the latest version but not able to do so due to this issue, can anyone help with that please? any workaround?

I am facing the same issue... any fix yet?

RCTSharedApplication is disabled when in App Extension:
https://github.com/facebook/react-native/blob/master/React/Base/RCTUtils.m#L486
I've tried to remove Linking from Pods, but I had no luck.
Here's my current Podfile.
platform :ios, '10.0'
require_relative '../node_modules/@react-native-community/cli-platform-ios/native_modules'
require_relative '../node_modules/react-native-unimodules/cocoapods.rb'

target 'RocketChatRN' do
  pod 'React', :path => '../node_modules/react-native/'
  pod 'React-Core', :path => '../node_modules/react-native/React'
  pod 'React-DevSupport', :path => '../node_modules/react-native/React'
  pod 'React-fishhook', :path => '../node_modules/react-native/Libraries/fishhook'
  pod 'React-RCTActionSheet', :path => '../node_modules/react-native/Libraries/ActionSheetIOS'
  pod 'React-RCTAnimation', :path => '../node_modules/react-native/Libraries/NativeAnimation'
  pod 'React-RCTBlob', :path => '../node_modules/react-native/Libraries/Blob'
  pod 'React-RCTImage', :path => '../node_modules/react-native/Libraries/Image'
  pod 'React-RCTLinking', :path => '../node_modules/react-native/Libraries/LinkingIOS'
  pod 'React-RCTNetwork', :path => '../node_modules/react-native/Libraries/Network'
  pod 'React-RCTSettings', :path => '../node_modules/react-native/Libraries/Settings'
  pod 'React-RCTText', :path => '../node_modules/react-native/Libraries/Text'
  pod 'React-RCTVibration', :path => '../node_modules/react-native/Libraries/Vibration'
  pod 'React-RCTWebSocket', :path => '../node_modules/react-native/Libraries/WebSocket'

  pod 'React-cxxreact', :path => '../node_modules/react-native/ReactCommon/cxxreact'
  pod 'React-jsi', :path => '../node_modules/react-native/ReactCommon/jsi'
  pod 'React-jsiexecutor', :path => '../node_modules/react-native/ReactCommon/jsiexecutor'
  pod 'React-jsinspector', :path => '../node_modules/react-native/ReactCommon/jsinspector'
  pod 'yoga', :path => '../node_modules/react-native/ReactCommon/yoga'

  pod 'DoubleConversion', :podspec => '../node_modules/react-native/third-party-podspecs/DoubleConversion.podspec'
  pod 'glog', :podspec => '../node_modules/react-native/third-party-podspecs/glog.podspec'
  pod 'Folly', :podspec => '../node_modules/react-native/third-party-podspecs/Folly.podspec'

  use_native_modules!
  use_unimodules!
end

target 'ShareRocketChatRN' do
  pod 'React', :path => '../node_modules/react-native/'
  pod 'React-Core', :path => '../node_modules/react-native/React'
  pod 'React-DevSupport', :path => '../node_modules/react-native/React'
  pod 'React-fishhook', :path => '../node_modules/react-native/Libraries/fishhook'
  pod 'React-RCTActionSheet', :path => '../node_modules/react-native/Libraries/ActionSheetIOS'
  pod 'React-RCTAnimation', :path => '../node_modules/react-native/Libraries/NativeAnimation'
  pod 'React-RCTBlob', :path => '../node_modules/react-native/Libraries/Blob'
  pod 'React-RCTImage', :path => '../node_modules/react-native/Libraries/Image'
#  pod 'React-RCTLinking', :path => '../node_modules/react-native/Libraries/LinkingIOS'
  pod 'React-RCTNetwork', :path => '../node_modules/react-native/Libraries/Network'
  pod 'React-RCTSettings', :path => '../node_modules/react-native/Libraries/Settings'
  pod 'React-RCTText', :path => '../node_modules/react-native/Libraries/Text'
  pod 'React-RCTVibration', :path => '../node_modules/react-native/Libraries/Vibration'
  pod 'React-RCTWebSocket', :path => '../node_modules/react-native/Libraries/WebSocket'

  pod 'React-cxxreact', :path => '../node_modules/react-native/ReactCommon/cxxreact'
  pod 'React-jsi', :path => '../node_modules/react-native/ReactCommon/jsi'
  pod 'React-jsiexecutor', :path => '../node_modules/react-native/ReactCommon/jsiexecutor'
  pod 'React-jsinspector', :path => '../node_modules/react-native/ReactCommon/jsinspector'
  pod 'yoga', :path => '../node_modules/react-native/ReactCommon/yoga'

  pod 'DoubleConversion', :podspec => '../node_modules/react-native/third-party-podspecs/DoubleConversion.podspec'
  pod 'glog', :podspec => '../node_modules/react-native/third-party-podspecs/glog.podspec'
  pod 'Folly', :podspec => '../node_modules/react-native/third-party-podspecs/Folly.podspec'
end

@diegolmello are you getting any warning after running pod install? I was getting following warning:
[!] The  [Release]target overrides theALWAYS_EMBED_SWIFT_STANDARD_LIBRARIESbuild setting defined in<.xcconfig location>'. This can lead to problems with the CocoaPods installation
- Use the $(inherited) flag, or
- Remove the build settings from the target.`
Solved it by adding:
post_install do |installer_representation|     installer_representation.pods_project.targets.each do |target|         target.build_configurations.each do |config|             config.build_settings['ALWAYS_EMBED_SWIFT_STANDARD_LIBRARIES'] = '$(inherited)'         end     end   end
to my podfile just before
use_native_modules!

Hi!
You are need to add use_native_modules! to both targets (App & Share Extension) and add it to your podfile:
post_install do |installer|
  installer.pods_project.targets.each do |target|
    target.build_configurations.each do |config|
      config.build_settings['APPLICATION_EXTENSION_API_ONLY'] = 'NO'
    end
  end
end

RCTSharedApplication is disabled when in App Extension:
https://github.com/facebook/react-native/blob/master/React/Base/RCTUtils.m#L486
I've tried to remove Linking from Pods, but I had no luck.
Here's my current Podfile.
platform :ios, '10.0'
require_relative '../node_modules/@react-native-community/cli-platform-ios/native_modules'
require_relative '../node_modules/react-native-unimodules/cocoapods.rb'

target 'RocketChatRN' do
  pod 'React', :path => '../node_modules/react-native/'
  pod 'React-Core', :path => '../node_modules/react-native/React'
  pod 'React-DevSupport', :path => '../node_modules/react-native/React'
  pod 'React-fishhook', :path => '../node_modules/react-native/Libraries/fishhook'
  pod 'React-RCTActionSheet', :path => '../node_modules/react-native/Libraries/ActionSheetIOS'
  pod 'React-RCTAnimation', :path => '../node_modules/react-native/Libraries/NativeAnimation'
  pod 'React-RCTBlob', :path => '../node_modules/react-native/Libraries/Blob'
  pod 'React-RCTImage', :path => '../node_modules/react-native/Libraries/Image'
  pod 'React-RCTLinking', :path => '../node_modules/react-native/Libraries/LinkingIOS'
  pod 'React-RCTNetwork', :path => '../node_modules/react-native/Libraries/Network'
  pod 'React-RCTSettings', :path => '../node_modules/react-native/Libraries/Settings'
  pod 'React-RCTText', :path => '../node_modules/react-native/Libraries/Text'
  pod 'React-RCTVibration', :path => '../node_modules/react-native/Libraries/Vibration'
  pod 'React-RCTWebSocket', :path => '../node_modules/react-native/Libraries/WebSocket'

  pod 'React-cxxreact', :path => '../node_modules/react-native/ReactCommon/cxxreact'
  pod 'React-jsi', :path => '../node_modules/react-native/ReactCommon/jsi'
  pod 'React-jsiexecutor', :path => '../node_modules/react-native/ReactCommon/jsiexecutor'
  pod 'React-jsinspector', :path => '../node_modules/react-native/ReactCommon/jsinspector'
  pod 'yoga', :path => '../node_modules/react-native/ReactCommon/yoga'

  pod 'DoubleConversion', :podspec => '../node_modules/react-native/third-party-podspecs/DoubleConversion.podspec'
  pod 'glog', :podspec => '../node_modules/react-native/third-party-podspecs/glog.podspec'
  pod 'Folly', :podspec => '../node_modules/react-native/third-party-podspecs/Folly.podspec'

  use_native_modules!
  use_unimodules!
end

target 'ShareRocketChatRN' do
  pod 'React', :path => '../node_modules/react-native/'
  pod 'React-Core', :path => '../node_modules/react-native/React'
  pod 'React-DevSupport', :path => '../node_modules/react-native/React'
  pod 'React-fishhook', :path => '../node_modules/react-native/Libraries/fishhook'
  pod 'React-RCTActionSheet', :path => '../node_modules/react-native/Libraries/ActionSheetIOS'
  pod 'React-RCTAnimation', :path => '../node_modules/react-native/Libraries/NativeAnimation'
  pod 'React-RCTBlob', :path => '../node_modules/react-native/Libraries/Blob'
  pod 'React-RCTImage', :path => '../node_modules/react-native/Libraries/Image'
#  pod 'React-RCTLinking', :path => '../node_modules/react-native/Libraries/LinkingIOS'
  pod 'React-RCTNetwork', :path => '../node_modules/react-native/Libraries/Network'
  pod 'React-RCTSettings', :path => '../node_modules/react-native/Libraries/Settings'
  pod 'React-RCTText', :path => '../node_modules/react-native/Libraries/Text'
  pod 'React-RCTVibration', :path => '../node_modules/react-native/Libraries/Vibration'
  pod 'React-RCTWebSocket', :path => '../node_modules/react-native/Libraries/WebSocket'

  pod 'React-cxxreact', :path => '../node_modules/react-native/ReactCommon/cxxreact'
  pod 'React-jsi', :path => '../node_modules/react-native/ReactCommon/jsi'
  pod 'React-jsiexecutor', :path => '../node_modules/react-native/ReactCommon/jsiexecutor'
  pod 'React-jsinspector', :path => '../node_modules/react-native/ReactCommon/jsinspector'
  pod 'yoga', :path => '../node_modules/react-native/ReactCommon/yoga'

  pod 'DoubleConversion', :podspec => '../node_modules/react-native/third-party-podspecs/DoubleConversion.podspec'
  pod 'glog', :podspec => '../node_modules/react-native/third-party-podspecs/glog.podspec'
  pod 'Folly', :podspec => '../node_modules/react-native/third-party-podspecs/Folly.podspec'
end

Below should also work with share extensions. So you don't have to copy/paste  the react pods twice:
target 'RocketChatRN' do
  // react pods

  use_native_modules!

  target 'ShareRocketChatRN' do
    use_native_modules!
    inherit! :complete
  end
end

Hi @mahmoudfelfel and @sebastian-sanchez-variv !
You need to add use_native_modules! to both targets (App & Share Extension) and add it to your podfile:
post_install do |installer|
  installer.pods_project.targets.each do |target|
    target.build_configurations.each do |config|
      config.build_settings['APPLICATION_EXTENSION_API_ONLY'] = 'NO'
    end
  end
end

Can you explain this? Plsss

One workaround is to set this flag to false in your extension's build settings.

@Jnlbr this code set Require Only App-Extension-Safe API to false on all Pod dependencies.

@djorkaeffalexandre  It does not work for me. I am trying to use OneSignalNotificationServiceExtension set up according to manual https://documentation.onesignal.com/docs/ios-sdk-setup.
When I use your solution I get another pack of errors (200+) about Mach-O Linker

@batai278 I got same errors when I used react-native-share-extension. Did you solved it?
I'm using react-native 0.61.2.


introduce calculation of missing value patterns

This PR introduces a function to calculate missing patterns as described in https://github.com/ResidentMario/missingno/issues/101.

NB: I have left some comments / suggestions to this PR in this comment on issue #101

This function may indeed be useful, but unfortunately I don't think missingno is an appropriate home for it. I'm already mulling over removing the existing filter utility functions.


Add API to obtain Receiver Poll state at run-time.

It is not currently possible to obtain the Polling state of the receiver from the radio layer via OTLink API's. This API is being added to add support for this use case.

@googlebot I signed it!

@adabreuti thanks for submitting.
I am just wondering what is a use-case for this API?

@adabreuti thanks for submitting.
I am just wondering what is a use-case for this API?

@abtink This API is used in conjunction with the existing MAC state based OTLink API's to make run-time decisions at the radio layer for RF command prioritization.

@adabreuti thanks for submitting.
I am just wondering what is a use-case for this API?

@abtink This API is used in conjunction with the existing MAC state based OTLink API's to make run-time decisions at the radio layer for RF command prioritization.

If the goal is to use this at radio platform layer can we consider adding such mechansim/API as part of platform layer APIs e.g., in otPlatRadio APIs? This way we avoid cross-layer interactions.

We have otPlat APIs which defines the interactions between platform layer (e.g., radio driver) and OT stack
And then we have public OT APIs (like otLink{} or otThread{}) which are intended for user of OT stack.
I think it'd be good to avoid platform layer using any public OT APIs (limit the platform interaction with OT to the platform APIs)

Can you please share what are some run-time decisions at radio layer that we want to enable with this API?

Radio platform should already be aware that is being told to enter "rx" mode (through the call to otPlatRadioReceive())
Does it need to know the reason why it is being asked to "rx"? (e.g., is it CSL sample or rx after data poll with ack from parent indicating pending frame)? Is this the idea?

Thanks.

@adabreuti thanks for submitting.
I am just wondering what is a use-case for this API?

@abtink This API is used in conjunction with the existing MAC state based OTLink API's to make run-time decisions at the radio layer for RF command prioritization.

If the goal is to use this at radio platform layer, can we consider adding such mechansim/API as part of platform layer APIs e.g., in otPlatRadio APIs? This way we avoid cross-layer interactions.

We have otPlat APIs which define the interactions between platform layer (e.g., radio driver) and OT stack
And then we have public OT APIs (like otLink{} or otThread{}) which are intended for user of OT stack.
I think it'd be good to avoid platform layer using any public OT APIs (limit the platform interaction with OT to the platform APIs)

Can you please share what are some run-time decisions at radio layer that we want to enable with this API?

Radio platform should already be aware that is being told to enter "rx" mode (through the call to otPlatRadioReceive())
Does it need to know the reason why it is being asked to "rx"? (e.g., is it CSL sample or rx after data poll with ack from parent indicating pending frame)? Is this the idea?

Thanks.

Thanks for the feedback, some more information regarding the use case:
We're tracking all of the Thread radio states, transmit polling, receive polling, direct vs indirect transmission etc. and using this information to prioritize radio commands in our multi-stack applications.
As such, we require the use of more specific data besides just being in the receive state as in the case of an FTD sitting in the Receive idle state would be less of a priority than actively receiving an ack following a transmission.
In terms of adding an otPlatRadio API, is your suggestion to update radio_platform.cpp/radio.h with a new otPlatRadioIsRxStatePolling() call which would call otLinkIsRxStatePolling()?

In terms of adding an otPlatRadio API, is your suggestion to update radio_platform.cpp/radio.h with a new otPlatRadioIsRxStatePolling() call which would call otLinkIsRxStatePolling()?

In terms of adding an otPlatRadio API, is your suggestion to update radio_platform.cpp/radio.h with a new otPlatRadioIsRxStatePolling() call which would call otLinkIsRxStatePolling()?

Yes. Something similar. I would say we don't even add otLinkIsRxStatePolling() at all. We add a new platform API for this purpose in radio.h which gets the info from Mac class. This makes it clear that this function is intended for use by platform layer.
In radio.h we have APIs (function declerations) where we expect the platform layer to implement and we use from OT stack. There are also functions (marked as extern in radio.h) which OT stack provides for use by platform layer (they are typically callbacks but they can be functions like otPlatRadioIsRxStatePolling() allowing radio driver to ask for additional info).
Thoughts?

Understood, the PR has been updated to reflect your feedback. Let me know if you see any issues with the changes. Thanks!

We're tracking all of the Thread radio states, transmit polling, receive polling, direct vs indirect transmission etc. and using this information to prioritize radio commands in our multi-stack applications.
As such, we require the use of more specific data besides just being in the receive state as in the case of an FTD sitting in the Receive idle state would be less of a priority than actively receiving an ack following a transmission.

@adabreuti , I believe the radio driver can determine whether OpenThread is waiting for data by observing that it just sent an IEEE 802.15.4 Data Request frame and received an ACK with the Frame Pending bit set. If so, can we avoid introducing a new API?

Thanks for the feedback, I'm closing PR as the implementation for determining Poll state is now in the radio layer. Thanks!


chore: upgrade Staging Kubernetes to 1.22

Summary
Upgrade Staging Kubernetes cluster to v1.22 and
bump the node group AMI release version.
Related

cds-snc/notification-planning#555

Blocked by cds-snc/notification-planning#643

I guess changing the k8s versions causes things to change for those certs and iam resources? 🤷

Yup, that's it - I've seen the exact same terraform plan for the other cluster version upgrades.


Log4j vulnerability

Is Omnisci affected by the recent log4j CVE ?
Thanks.
Regards.

Hi
we shouldn't but we are going in deeper about that, so we can't state anything official

No. We had been using log4j 1.2.17, which is not vulnerable to that CVE. However, for good measure, we've pushed https://github.com/omnisci/omniscidb/commit/2bfd0267ae4b45940aca705934069c82476e57ba which moves to log4j2 2.16.0.


[Feature]: Support for Seq classification/Reward models

🚀 The feature, motivation and pitch
Verifier/reward models are going to be very important moving forward for building:

High quality synthetic data pipelines
Verifying model reasoning
Multi agent systems

Could we add support for sequence classification models like Skywork/Skywork-Reward-Llama-3.1-8B
Alternatives
No response
Additional context
No response
Before submitting a new issue...

[X] Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

contribution is welcome!
vllm already supports embedding model. I think it is quite similar to reward model. I don't know what would be the obstacle to use vllm code to run reward models. We can pretend they are embedding models.

Seems like a lot of the reward models use a different architecture than embedding models.
ValueError: Model architectures ['Gemma2ForSequenceClassification'] are not supported for now.
ValueError: Model architectures ['LlamaForSequenceClassification'] are not supported for now.

 Supported architectures: ['AquilaModel', 'AquilaForCausalLM', 'BaiChuanForCausalLM', 'BaichuanForCausalLM', 'BloomForCausalLM', 'ChatGLMModel', 'ChatGLMForConditionalGeneration', 'CohereForCausalLM', 'DbrxForCausalLM', 'DeciLMForCausalLM', 'DeepseekForCausalLM', 'DeepseekV2ForCausalLM', 'ExaoneForCausalLM', 'FalconForCausalLM', 'GemmaForCausalLM', 'Gemma2ForCausalLM', 'GPT2LMHeadModel', 'GPTBigCodeForCausalLM', 'GPTJForCausalLM', 'GPTNeoXForCausalLM', 'InternLMForCausalLM', 'InternLM2ForCausalLM', 'JAISLMHeadModel', 'LlamaForCausalLM', 'LLaMAForCausalLM', 'MistralForCausalLM', 'MixtralForCausalLM', 'QuantMixtralForCausalLM', 'MptForCausalLM', 'MPTForCausalLM', 'MiniCPMForCausalLM', 'NemotronForCausalLM', 'OlmoForCausalLM', 'OPTForCausalLM', 'OrionForCausalLM', 'PersimmonForCausalLM', 'PhiForCausalLM', 'Phi3ForCausalLM', 'PhiMoEForCausalLM', 'Qwen2ForCausalLM', 'Qwen2MoeForCausalLM', 'Qwen2VLForConditionalGeneration', 'RWForCausalLM', 'StableLMEpochForCausalLM', 'StableLmForCausalLM', 'Starcoder2ForCausalLM', 'ArcticForCausalLM', 'XverseForCausalLM', 'Phi3SmallForCausalLM', 'MedusaModel', 'EAGLEModel', 'MLPSpeculatorPreTrainedModel', 'JambaForCausalLM', 'GraniteForCausalLM', 'MistralModel', 'Blip2ForConditionalGeneration', 'ChameleonForConditionalGeneration', 'FuyuForCausalLM', 'InternVLChatModel', 'LlavaForConditionalGeneration', 'LlavaNextForConditionalGeneration', 'LlavaNextVideoForConditionalGeneration', 'MiniCPMV', 'PaliGemmaForConditionalGeneration', 'Phi3VForCausalLM', 'PixtralForConditionalGeneration', 'QWenLMHeadModel', 'UltravoxModel', 'BartModel', 'BartForConditionalGeneration']

Here are two examples:
Skywork/Skywork-Reward-Gemma-2-27B
Ray2333/GRM-Llama3-8B-rewardmodel-ft

@youkaichao Added #8739 tried to convert it to the vllm formar, but running into some shape issues. Let me know if the conversion generally looks right.

looks like https://github.com/vllm-project/vllm/pull/8896 already implements it.

Hey - I've been working with reward models substantially in the open ecosystem building rewardbench, in reality most of the open models have subtly different architectures.
The easiest is *ForSequenceClassification, but it is much messier from here. Happy to take questions -- I'm going to look at the reward model RFC now too. Curious to follow these implementations.


added functions.sh -> checkIfInstalled function to check wether softw…

…are is installed before running a script and renamed/updated getCPUTemp.sh
Merging because of the functions.sh file, we should update all scripts to include this check :)

Also tested it on fedora, everything works as expected :)


Roll Flutter Engine from 703d45539059 to 2cbc56268529 (2 revisions)

https://github.com/flutter/engine/compare/703d45539059...2cbc56268529
2023-08-01<EMAIL_ADDRESS>Roll Skia from bddb07184f88 to 45290179e017 (1 revision) (flutter/engine#44215)
2023-08-01<EMAIL_ADDRESS>Manual roll Dart SDK from aca45d2e133c to 45405dd4ea87 (2 revisions) (flutter/engine#44204)
If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-engine-flutter-autoroll
Please CC jacksongardner@google.com,rmistry<EMAIL_ADDRESS>on the revert to ensure that a human
is aware of the problem.
To file a bug in Flutter: https://github.com/flutter/flutter/issues/new/choose
To report a problem with the AutoRoller itself, please file a bug:
https://bugs.chromium.org/p/skia/issues/entry?template=Autoroller+Bug
Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md

Commit queue failed; closing this roll.


model-viewer in VueJS

amazing product!
There is any option for using this model-viewer in VuwJS? thanx:)

I believe people have; please search the discussions for help. I'm not an expert on that framework.


Local captainhook.json support

Feature Request
Hey there,
We are many developers and thus, some of us have a docker configuration, others may have vagrant and so on.
I would prefer having a way to create example local configurations, where our developers can re-configure the projects configuration to their environment.
Example
Project configuration
captainhook.json
{
	"config": {
		"verbosity": "normal"
	}
}

Developers own configuration which is excluded from VCS
captainhook.local.json
{
	"config": {
		"verbosity": "verbose",
		"run-mode": "docker",
		"run-exec": "docker exec whatever"
	}
}

The deal could be, that the local file is always next to the captainhook.json.
This might make the implementation quite easy.
What do you think about it?

+1 for the issue
and add this problem with a team that shares docker and pc/mac with software installed on host.
If file captainhook.json is committed in git repository it's a problem to edit.
Ignore the file on git is a problem if someone makes some update
{
    "config": {
         "run-mode": "docker",
    }
}

The best option is to create the configuration file like dotenv component of symfony
https://symfony.com/doc/current/components/dotenv.html

The Cap'n supports config file inclusion, does that help?
You could commit a captainhook.inc.json with all your hooks and add files like captainhook.json.sample.docker captainhook.json.sample.vagrant  with something like this.
{
  "config": {
    "includes": [
      "captainhook.inc.json"
    ]
  }
}

Developers would have to choose and copy the right config file to captainhook.json and then everything would work.
The other option I see is something like the local configuration file you proposed. But I think I would prefer something like captainhook.config.json that is only allowed to overwrite the Cap'n config settings and is not allowed to add any custom hooks.
What do you think about using environment variables to overwrite config settings?

What would happen if the file is not available, which would be the default as it is a per user config and thus excluded from VCS.
And yes, the local config should be only allowed to override config. It should not be able to provide additional hooks.

Fixed in version 5.3.0
For details have a look at the documentation


Updated test cases for Edit Task

Codecov Report

Merging #282 (acc192e) into master (d98b0fc) will increase coverage by 3.32%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master     #282      +/-   ##
============================================
+ Coverage     58.03%   61.36%   +3.32%
- Complexity      683      706      +23
============================================
  Files           107      107
  Lines          2583     2583
  Branches        307      307
============================================
+ Hits           1499     1585      +86
+ Misses         1016      925      -91
- Partials         68       73       +5

Impacted Files
Coverage Δ

...rc/main/java/manageezpz/commons/core/Messages.java
0.00% <ø> (ø)

...ava/manageezpz/logic/commands/EditTaskCommand.java
74.07% <ø> (+74.07%)
:arrow_up:

src/main/java/manageezpz/model/task/Todo.java
88.23% <0.00%> (+11.76%)
:arrow_up:

.../main/java/manageezpz/logic/parser/ParserUtil.java
68.42% <0.00%> (+18.42%)
:arrow_up:

src/main/java/manageezpz/model/task/Event.java
100.00% <0.00%> (+24.32%)
:arrow_up:

src/main/java/manageezpz/model/task/Deadline.java
100.00% <0.00%> (+26.66%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d98b0fc...acc192e. Read the comment docs.


Time Cartesian Axis min max configuration documentation Typo

Documentation Is:

[ ] Missing or needed?
[X] Confusing
[ ] Not sure?

Please Explain in Detail...
It appears there's a probable typo in the Min Max configuration documentation.  The docs say the x axis will start at 7 October 2021, yet the code shows 2021-11-07 as the min property.
Your Proposal for Changes
Seems to me that the date should be 7 November 2021
Example
No response

Already fixed: https://github.com/chartjs/Chart.js/pull/10826
https://www.chartjs.org/docs/master/axes/cartesian/time.html#min-max-configuration


fix(maven): bump maven-compiler-plugin from 3.8.1 to 3.10.1 #dependabot

@fpibot kindly approve

@fpibot approval please


Added comments to test classes

Added coments to IntegrationTest.java and StaticContentUnitTest.java

Remove from your repository the following local files:

.classpath
.project
.settings
Then sync and do the pull request again.


Example .ttl with 10 datasets from Statbel Opendata

Thanks for the contribution, one tiny remark: would it be possible to use lowercase for the namespace prefixes ?
E..g dcat instead of DCAT ?
Technically it does not make a difference, but traditionally the prefixes in RDF are written in lowercase ...

Ok I will make the change this week!


Set res to empty object as default

Coverage remained the same at 97.768% when pulling 0e81013c5a942edc4f14a708ffccfc7eb914bdb0 on feature/handle-empty-response into 902a232939b76b278dbaba17883bfac23ba0aec0 on master.

Coverage remained the same at 97.768% when pulling 0e81013c5a942edc4f14a708ffccfc7eb914bdb0 on feature/handle-empty-response into 902a232939b76b278dbaba17883bfac23ba0aec0 on master.

Coverage remained the same at 97.768% when pulling 0e81013c5a942edc4f14a708ffccfc7eb914bdb0 on feature/handle-empty-response into 902a232939b76b278dbaba17883bfac23ba0aec0 on master.

Coverage remained the same at 97.768% when pulling 5ee24cfacf8a88be6dc091bbec341d16f43d43c0 on feature/handle-empty-response into 902a232939b76b278dbaba17883bfac23ba0aec0 on master.

Coverage remained the same at 97.768% when pulling 5ee24cfacf8a88be6dc091bbec341d16f43d43c0 on feature/handle-empty-response into 902a232939b76b278dbaba17883bfac23ba0aec0 on master.

Coverage remained the same at 97.768% when pulling 5ee24cfacf8a88be6dc091bbec341d16f43d43c0 on feature/handle-empty-response into 902a232939b76b278dbaba17883bfac23ba0aec0 on master.

published as 0.20.0


'dict' object has no attribute 'head'

it says "'dict' object has no attribute 'head'"
could you please help me?

seems like this is fixed..? I just tried, it seems to return DF

I had this same issue and was able to fix just now.
Originally I had installed this with pip install messari — then I got this error (and others)
However, the above code works as intended, returning a dataframe, when I install via pip install git+https://github.com/messari/messari-python-api.git.
So my guess is that it's an older version being installed with pip install messari?

I had the same experience; when I used pip3 install messari, it downloaded the 0.0.1 version of the package; I used pip3 install messari --upgrade and it downloaded the newest 1.0 version.


Use newer AWS SDK

Pull Request Test Coverage Report for Build 30

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 69.78%

Totals

Change from base Build 29:
0.0%

Covered Lines:
127

Relevant Lines:
182

💛  - Coveralls


[Bug]: Select property colour picker

Contact Details
<EMAIL_ADDRESS>What happened?
When in the colour picker if I move the mouse out of the window it closes the box.
Can we add a confirm button?
Maybe a way to save previous colours would be nice as well.
What platform were you using?
Desktop
Version of the plugin
2.7.1
Relevant log output
No response
Are you using the latest version of the plugin in your Obsidian vault?

[X] I have verified that I am on the latest version

Are you check if there is a similar issue?

[X] I have verified that there are not similar issues

the save triggers with onblur event. Could be improved with some confirm button, but I can not save previous colors.
will be added as visual

With 1.0 Obsidian API offers its own color picker selector. I will try it for sure

Migrated react-color component to native Obsidian color picker, resolving this issue.
https://user-images.githubusercontent.com/11924043/196004406-2774865a-d89f-49a4-bf45-d1aba8357329.mov
The component offers keyboard shortcuts:

esc to undo change / close selector
enter to confirm the color


theme: 'bars-square' not seem to work right

I have tried using the plugin with the bars-square theme which as far as I know it should be the squares with the numbers inside
            <div class="mui-textfield mui-col-xs-12 mui-col-md-12">
                <label>Rating: </label>
                <select id="squarefield" name="score" form="survey">
                    <option value="0">0</option>
                    <option value="1">1</option>
                    <option value="2">2</option>
                    <option value="3">3</option>
                    <option value="4">4</option>
                    <option value="5">5</option>
                    <option value="6">6</option>
                    <option value="7">7</option>
                    <option value="8">8</option>
                    <option value="9">9</option>
                    <option value="10">10</option>
                </select>
            </div>

But I am getting the attached image instead which looks more like the 1/10 rating

sorry found it
showValues: true,
showSelectedRating: false


feat(deployment): add app and version label to pods

What this PR does / why we need it:
This PR adds app and version labels to the pod. I "think" this is one of the standard practices. If not, it's a requirement for istio istio doc.
Checklist
[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

[x] PR is based off the current tip of the next branch and targets next, not main
[x] New or modified sections of values.yaml are documented in the README.md
[x] Commits follow the Kong commit message guidelines

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


ipodtouch0218: Blacklist +917439367021

ipodtouch0218 requests the blacklist of the number +917439367021. See the MS search here and the Stack Exchange search in text, in URLs, and in code.

Approved by Cow in Charcoal HQ


Removing std namespace before shuffle()

to be similar to the other  functions documentation.

@yves-dolce : Thanks for your contribution! The author, @, has been notified to review your proposed change.


Release Google.Cloud.BareMetalSolution.V2 version 1.5.0

Changes in this release:
New features

Change netstandard2.1 target to netstandard2.0 (commit 82bea85)

Created release for Google.Cloud.BareMetalSolution.V2-1.5.0


There is no redis on Pythonanywhere, but cookiecutter-django wants to read its configuration

What happened?
I created a new cookiecutter-django project directly on the Pythonanywhere ssh shell. Pythonanywhere does not support Redis currently, even though they say they are planning it. cookiecutter-django did not ask me whether I want to use Redis. But thereis a Redis related config in the settings/production.py that breaks the "python manage.py" command.
I am following the related pythonanywhere deployment documentation. When I issue
python manage.py migrate
I get the error
  ...
  File "/home/examaddin/cookiecutterdjango/mysite/config/settings/production.py", line 62, in <module>
    "LOCATION": env("REDIS_URL"),
  File "/home/examaddin/.virtualenvs/cookiecutterdjango/lib/python3.8/site-packages/environ/environ.py", line 123, in __call__
    return self.get_value(var, cast=cast, default=default, parse_default=parse_default)
  File "/home/examaddin/.virtualenvs/cookiecutterdjango/lib/python3.8/site-packages/environ/environ.py", line 277, in get_value
    raise ImproperlyConfigured(error_msg)
django.core.exceptions.ImproperlyConfigured: Set the REDIS_URL environment variable

This error indicates that I need to set REDIS_URL here in the settings file:
CACHES = {
    "default": {
        "BACKEND": "django_redis.cache.RedisCache",
        "LOCATION": env("REDIS_URL"),
        "OPTIONS": {
            "CLIENT_CLASS": "django_redis.client.DefaultClient",
            # Mimicing memcache behavior.
            # https://github.com/jazzband/django-redis#memcached-exceptions-behavior
            "IGNORE_EXCEPTIONS": True,
        },
    }
}

Pythonanywhere does not have Redis support, so the settings file is buggy in this case.
I am just starting my site so I do not think I need caching. So I just commented out this caching part for the time being. I hope that it works. Later if I have a busy enough site I can work on replacing this.
What should've happened instead?
Some solutions that come to mind:

cookiecutter-django can make using Redis a selection.
This behaviour is not mentioned in "deploying to pythonanywhere" documentation section. This and its solution can be mentioned there.

Additional details

Host system configuration:

Version of cookiecutter CLI (get it with cookiecutter --version): Cookiecutter 1.7.2
OS: Ubuntu 16.04.6 LTS
Python version: Python 3.8.0

Options selected and/or replay file:

Logs:

$ cookiecutter https://github.com/pydanny/cookiecutter-django
project_name [Project Name]: ...

It seems be a long requested feature indeed... Would be nice to add a paragraph about Redis in the PythonAnywhere page of our documentation. Might need to update the environment veriables which should be set as well.
I see several options here:

Disable the cache entirely
Use a different cache than Redis
If wanting to stick with Redis, signup to a separate service offering hosted Redis (Redislab, AWS, Google, ...)

I've never used Pythonanywhere so I don't know of the limitations of the 2 last options, in terms of filesystem/memory usage quota or colocation to hosted Redis server.

I read in stackoverflow that Pythonanywhere person Giles recommend your 3. option if you really need redis.

Hi there @browniebroke, is there anyone who is able to update the documentation here  ? It is good for the most  - thanks to your comment above I was able to get past the same error that I was getting as @razaltcom
I updated in my WSGI.py the line: os.environ['DJANGO_AWS_STORAGE_BUCKET_NAME'] = '' as explained in the tutorial, but now Botocore is complaining about me setting the environment var to a blank string.
File "/home/projectname/.virtualenvs/venv/lib/python3.9/site-packages/botocore/handlers.py", line 240, in validate_bucket_name 2021-09-18 13:55:01,717:     raise ParamValidationError(report=error_msg) 2021-09-18 13:55:02,672: Error running WSGI application 2021-09-18 13:55:02,679: botocore.exceptions.ParamValidationError: Parameter validation failed: 2021-09-18 13:55:02,679: Invalid bucket name "": Bucket name must match the regex "^[a-zA-Z0-9.\-_]{1,255}$" or be an ARN matching the regex "^arn:(aws).*:(s3|s3-object-lambda):[a-z\-0-9]*:[0-9]{12}:accesspoint[/:][a-zA-Z0-9\-.]{1,63}$|^arn:(aws).*:s3-outposts:[a-z\-0-9]+:[0-9]{12}:outpost[/:][a-zA-Z0-9\-]{1,63}[/:]accesspoint[/:][a-zA-Z0-9\-]{1,63}$"
Thanks for your help


MulticastSocket tests failing on AIX

Three MulticastSocket tests fail on Aix with same exception The socket name is not available on this system. (Error setting socket option)
Standard Error

java.net.SocketException: The socket name is not available on this system. (Error setting socket option)
	at java.net.PlainDatagramSocketImpl.socketSetOption0(Native Method)
	at java.net.PlainDatagramSocketImpl.socketSetOption(PlainDatagramSocketImpl.java:74)
	at java.net.AbstractPlainDatagramSocketImpl.setOption(AbstractPlainDatagramSocketImpl.java:309)
	at java.net.MulticastSocket.setNetworkInterface(MulticastSocket.java:554)
	at SetGetNetworkInterfaceTest.main(SetGetNetworkInterfaceTest.java:54)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.sun.javatest.regtest.agent.MainActionHelper$AgentVMRunnable.run(MainActionHelper.java:298)
	at java.lang.Thread.run(Thread.java:748)
java.lang.RuntimeException: Test Fail
	at SetGetNetworkInterfaceTest.main(SetGetNetworkInterfaceTest.java:71)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.sun.javatest.regtest.agent.MainActionHelper$AgentVMRunnable.run(MainActionHelper.java:298)
	at java.lang.Thread.run(Thread.java:748)

JavaTest Message: Test threw exception: java.lang.RuntimeException
JavaTest Message: shutting down test

https://ci.adoptopenjdk.net/view/Test_openjdk/job/openjdk8_hs_openjdktest_ppc64_aix/42/console
java/net/MulticastSocket/SetGetNetworkInterfaceTest.java
java/net/MulticastSocket/SetLoopbackMode.java
java/net/MulticastSocket/Test.java

Similar errors were seen on JDK11 and were reported here: https://bugs.openjdk.java.net/browse/JDK-8207404. There were some fixes made in JDK12 but these were not back-ported to JDK8

@groeges Thanks! Will the fix be back-ported to JDK8?

@sophia-guo I am not sure. The code in JDK8 is quite different from that in JDK12. They might have deemed it too much work to try and do it. I will take a look and see if we can Gert them to back port it or to see if we can do it ourselves.

Closed as fixed do not back port to JDK8


It is working fine now

[Enter feedback here]
I have tried again and it worked. It looks like the issue was with fork sync.
Thank you for acknowledging and now I see it is working perfectly fine.
Regards,
Amit

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 0717d289-c339-ed12-4d24-3d968121c75d
Version Independent ID: c20939c5-96b4-e4c5-6737-e3407d52f3de
Content: Deploy to Azure Kubernetes Service with Azure Pipelines - Azure Kubernetes Service
Content Source: articles/aks/devops-pipeline.md
Service: container-service
GitHub Login: @juliakm
Microsoft Alias: jukullam

@amkarmak Thanks for confirming that this is working.


Icon incosistency between states and logbook

Description of problem:
The chosen icon for a group is not being used in the logbook.

Expected:
The chosen icon being used in the logbook as well.
Problem-relevant configuration.yaml entries and steps to reproduce:
obergeschoss_kueche:
  name: Küche
  icon: mdi:silverware-spoon
  entities:
    - switch.untertischspeicher

Has this been resolved?

Nope, and this applies to most (if not all) entities. I just tested this with a switch (input_boolean) in my UI running Home Assistant 0.40.0:
The switch I toggled in my UI:

What it looks like in my logbook:

This is still an issue (on 0.43.2 anyway).

Stil an issue on 0.51.x

And on 0.52.0 as well.

This issue may be better suited for the home-assistant-polymer repository: https://github.com/home-assistant/home-assistant-polymer/issues

Still an issue on 0.64.0

Still an issue on 0.77.3

/move to home-assistant-polymer


Missing url for abuse report translate function

@robhudson whoops - translate was waffled off in my local dev :disappointed:

r+


Difficulties installing sacCer3 (yeast) RNA-Seq resources

Hi there -
I'm trying to analyze some yeast RNA-Seq data. I tried updgrading my bcbio install like so:
bcbio_nextgen.py upgrade --datatarget rnaseq --genomes sacCer

During the install process, I get lots of error messages like:
[localhost] local: wget --continue --no-check-certificate -O sacCer3-seq.tar.xz 'https://s3.amazonaws.com/biodata/genomes/sacCer3-seq.tar.xz'
--2017-01-27 17:07:40--  https://s3.amazonaws.com/biodata/genomes/sacCer3-seq.tar.xz
Resolving s3.amazonaws.com... <IP_ADDRESS>
Connecting to s3.amazonaws.com|<IP_ADDRESS>|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2017-01-27 17:07:40 ERROR 404: Not Found.

The genome and transcriptome indexes all eventually build correctly (I think), but the RNA-Seq references described in sacCer3=resources.yaml don't seem to exist in my install directory:
$ bcbio/genomes/Scerevisiae/sacCer3/> ls -1
bowtie2
bwa
rtg
seq
seq_prep
snap
snpeff
star
vep

Any thoughts on what might be happening? Alternately, is there another spot I can grab the transcript reference files from? Thanks!

Note - I've worked around this issue for now by downloading the yeast genes in GTF format and setting up a custom genome:
> bcbio_setup_genome.py --cores 8 --fasta genomes/Scerevisiae/sacCer3/seq/sacCer3.fa \
 --gtf sacCer3_sgd_genes.gtf --name Scerevisiae \
 --build sacCer3_override --indexes star

Hi @rdocking,
Thanks-- you nailed it, we don't have the transcriptome set up for yeast out of the box and your workaround is what we'd do to make it.

Hi @roryk - thanks for your response. Please feel free to close this issue if you'd like - it looks like my yeast RNA-Seq runs are going well.


Feature Title: Enhanced Display Name Tracking for Moderation

Feature Request:
Server Moderators often struggle to keep track of users who routinely alter their display names in game.
This means misconduct or rule violations can be difficult to attribute to the correct individual when their screen name is constantly changing. (when  video evidence is outdated, or contains their old name)
Proposed Solution:
Implement a slash command for moderators that allows lookup of a user's display name history within the Nakama, linked to their unique Discord ID. (Simliar to the discord "ModView")
Command Syntax:
/displayhistory [User Mention or User ID]

/lookup will have past display names in it. fix is in today.


Unable to find tests with FileNotFoundException

Description

IMPORTANT: if the defect is reproduced only in a workflow from within the Visual Studio IDE then do not report the issue here - instead, please report it using Visual Studio's "Send Feedback" option that can be accessed from the Help menu OR using this link https://developercommunity.visualstudio.com.
For a defect reproducable from the vstest command line, describe the issue you've observed.

I have a test which run fine locally but always meet issue in cloudtest complaint about FileNotFoundException and unable to find tests.
Steps to reproduce

What steps can reproduce the defect?
Please share the setup, commandline for vstest.console, sample project, target
framework etc.

dotnet.exe vstest HttpServiceE2ETests.dll  /Platform:X64 /Logger:TRX /Diag:diag.log
Expected behavior

Share the expected output

Expect test to discover 4 tests and run fine.
Actual behavior

What is the behavior observed?

System.IO.FileNotFoundException: Unable to find tests for D:\data\CHT\6311c6f0\E2ETests\HttpServiceE2ETests\HttpServiceE2ETests.dll. Make sure test project has a nuget reference of package "Microsoft.NET.Test.Sdk" and framework version settings are appropriate. Rerun with /diag option to diagnose further.
at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Hosting.DotnetTestHostManager.GetTestHostProcessStartInfo(IEnumerable1 sources, IDictionary2 environmentVariables, TestRunnerConnectionInfo connectionInfo)
at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyOperationManager.SetupChannel(IEnumerable`1 sources, CancellationToken cancellationToken)
at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyExecutionManager.StartTestRun(TestRunCriteria testRunCriteria, ITestRunEventsHandler eventHandler)
Test Run Aborted.
Diagnostic logs

Please share test platform diagnostics logs. Instructions to collect logs are here.
The logs may contain test assembly paths, kindly review and mask those before sharing.

TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.434,<PHONE_NUMBER>7, vstest.console.dll, Using platform:X64
TpTrace Verbose: 0 : 5108, 1, 2018/05/11, 23:17:40.451,<PHONE_NUMBER>5, vstest.console.dll, TestPluginCache: Discovering the extensions using extension path.
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.453,<PHONE_NUMBER>8, vstest.console.dll, AssemblyResolver: Microsoft.TestPlatform.Extensions.BlameDataCollector: Resolving assembly.
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.454,<PHONE_NUMBER>4, vstest.console.dll, AssemblyResolver: Microsoft.TestPlatform.Extensions.BlameDataCollector: Resolved assembly.
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.455,<PHONE_NUMBER>4, vstest.console.dll, AssemblyResolver: Microsoft.TestPlatform.Extensions.EventLogCollector: Resolving assembly.
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.458,<PHONE_NUMBER>1, vstest.console.dll, AssemblyResolver: Microsoft.TestPlatform.Extensions.EventLogCollector: Resolved assembly.
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.459,<PHONE_NUMBER>9, vstest.console.dll, AssemblyResolver: Microsoft.TestPlatform.TestHostRuntimeProvider: Resolving assembly.
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.460,<PHONE_NUMBER>7, vstest.console.dll, AssemblyResolver: Microsoft.TestPlatform.TestHostRuntimeProvider: Resolved assembly.
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.472,<PHONE_NUMBER>8, vstest.console.dll, AssemblyResolver: Microsoft.VisualStudio.TestPlatform.Extensions.Trx.TestLogger: Resolving assembly.
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.473,<PHONE_NUMBER>0, vstest.console.dll, AssemblyResolver: Microsoft.VisualStudio.TestPlatform.Extensions.Trx.TestLogger: Resolved assembly.
TpTrace Verbose: 0 : 5108, 1, 2018/05/11, 23:17:40.475,<PHONE_NUMBER>3, vstest.console.dll, TestPluginCache: Discovered the extensions using extension path ''.
TpTrace Verbose: 0 : 5108, 1, 2018/05/11, 23:17:40.475,<PHONE_NUMBER>3, vstest.console.dll, TestPluginCache: Discoverers are ''.
TpTrace Verbose: 0 : 5108, 1, 2018/05/11, 23:17:40.476,<PHONE_NUMBER>9, vstest.console.dll, TestPluginCache: Executors are ''.
TpTrace Verbose: 0 : 5108, 1, 2018/05/11, 23:17:40.476,<PHONE_NUMBER>4, vstest.console.dll, TestPluginCache: Setting providers are ''.
TpTrace Verbose: 0 : 5108, 1, 2018/05/11, 23:17:40.476,<PHONE_NUMBER>1, vstest.console.dll, TestPluginCache: Loggers are ''.
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.481,<PHONE_NUMBER>8, vstest.console.dll, RunTestsArgumentProcessor:Execute: Test run is starting.
TpTrace Verbose: 0 : 5108, 1, 2018/05/11, 23:17:40.481,<PHONE_NUMBER>0, vstest.console.dll, RunTestsArgumentProcessor:Execute: Queuing Test run.
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.481,<PHONE_NUMBER>7, vstest.console.dll, TestRequestManager.RunTests: run tests started.
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.496,<PHONE_NUMBER>5, vstest.console.dll, AssemblyMetadataProvider.GetFrameWork: Determined framework:'.NETCoreApp,Version=v2.0' for source: 'D:\data\CHT\6311c6f0\E2ETests\HttpServiceE2ETests\HttpServiceE2ETests.dll'
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.497,<PHONE_NUMBER>8, vstest.console.dll, Determined framework for all sources: .NETCoreApp,Version=v2.0
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.500,<PHONE_NUMBER>4, vstest.console.dll, AssemblyMetadataProvider.GetArchitecture: Determined architecture:X64 info for assembly: D:\data\CHT\6311c6f0\E2ETests\HttpServiceE2ETests\HttpServiceE2ETests.dll
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.503,<PHONE_NUMBER>6, vstest.console.dll, Determined platform for all sources: X64
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.506,<PHONE_NUMBER>8, vstest.console.dll, Compatible sources list :
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.507,<PHONE_NUMBER>7, vstest.console.dll, D:\data\CHT\6311c6f0\E2ETests\HttpServiceE2ETests\HttpServiceE2ETests.dll
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.510,<PHONE_NUMBER>1, vstest.console.dll, InferRunSettingsHelper.IsTestSettingsEnabled: Unable to navigate to RunSettings/MSTest. Current node: RunSettings
TpTrace Verbose: 0 : 5108, 1, 2018/05/11, 23:17:40.519,<PHONE_NUMBER>7, vstest.console.dll, TestPluginCache: Discovering the extensions using extension path.
TpTrace Verbose: 0 : 5108, 1, 2018/05/11, 23:17:40.520,<PHONE_NUMBER>7, vstest.console.dll, TestPluginCache: Discovered the extensions using extension path ''.
TpTrace Verbose: 0 : 5108, 1, 2018/05/11, 23:17:40.520,<PHONE_NUMBER>9, vstest.console.dll, TestPluginCache: Discoverers are ''.
TpTrace Verbose: 0 : 5108, 1, 2018/05/11, 23:17:40.520,<PHONE_NUMBER>5, vstest.console.dll, TestPluginCache: Executors are ''.
TpTrace Verbose: 0 : 5108, 1, 2018/05/11, 23:17:40.520,<PHONE_NUMBER>2, vstest.console.dll, TestPluginCache: Setting providers are ''.
TpTrace Verbose: 0 : 5108, 1, 2018/05/11, 23:17:40.521,<PHONE_NUMBER>0, vstest.console.dll, TestPluginCache: Loggers are 'logger://Microsoft/TestPlatform/Extensions/Blame/v1,logger://Microsoft/TestPlatform/TrxLogger/v1'.
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.522,<PHONE_NUMBER>7, vstest.console.dll, TestPluginManager.CreateTestExtension: Attempting to load test extension: Microsoft.VisualStudio.TestPlatform.Extensions.TrxLogger.TrxLogger
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.527,<PHONE_NUMBER>7, vstest.console.dll, TestPluginManager.CreateTestExtension: Attempting to load test extension: Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Hosting.DefaultTestHostManager
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.528,<PHONE_NUMBER>9, vstest.console.dll, TestPluginManager.CreateTestExtension: Attempting to load test extension: Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Hosting.DotnetTestHostManager
TpTrace Verbose: 0 : 5108, 1, 2018/05/11, 23:17:40.528,<PHONE_NUMBER>8, vstest.console.dll, TestEngine: Initializing Parallel Execution as MaxCpuCount is set to: 1
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.529,<PHONE_NUMBER>1, vstest.console.dll, InferRunSettingsHelper.IsTestSettingsEnabled: Unable to navigate to RunSettings/MSTest. Current node: RunSettings
TpTrace Verbose: 0 : 5108, 1, 2018/05/11, 23:17:40.539,<PHONE_NUMBER>9, vstest.console.dll, TestRequestSender is acting as server
TpTrace Verbose: 0 : 5108, 1, 2018/05/11, 23:17:40.541,<PHONE_NUMBER>7, vstest.console.dll, TestRunRequest.ExecuteAsync: Creating test run request.
TpTrace Verbose: 0 : 5108, 1, 2018/05/11, 23:17:40.542,<PHONE_NUMBER>7, vstest.console.dll, TestRunRequest.ExecuteAsync: Starting.
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.542,<PHONE_NUMBER>1, vstest.console.dll, TestRunRequest.ExecuteAsync: Starting run with settings:TestRunCriteria:
KeepAlive=False,FrequencyOfRunStatsChangeEvent=10,RunStatsChangeEventTimeout=00:00:01.5000000,TestCaseFilter=,TestExecutorLauncher=
Settingsxml=

D:\data\CHT\6311c6f0\TestResults
X64
.NETCoreApp,Version=v2.0
False
False

TpTrace Verbose: 0 : 5108, 1, 2018/05/11, 23:17:40.542,<PHONE_NUMBER>8, vstest.console.dll, TestRunRequest.ExecuteAsync: Wait for the first run request is over.
TpTrace Verbose: 0 : 5108, 1, 2018/05/11, 23:17:40.543,<PHONE_NUMBER>8, vstest.console.dll, ParallelProxyExecutionManager: Start execution. Total sources: 1
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.544,<PHONE_NUMBER>8, vstest.console.dll, ProxyParallelExecutionManager: Triggering test run for next source: D:\data\CHT\6311c6f0\E2ETests\HttpServiceE2ETests\HttpServiceE2ETests.dll
TpTrace Verbose: 0 : 5108, 1, 2018/05/11, 23:17:40.544,<PHONE_NUMBER>3, vstest.console.dll, ProxyParallelExecutionManager: No sources available for execution.
TpTrace Verbose: 0 : 5108, 5, 2018/05/11, 23:17:40.544,<PHONE_NUMBER>3, vstest.console.dll, ParallelProxyExecutionManager: Execution started. Started clients: 1
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:40.545,<PHONE_NUMBER>4, vstest.console.dll, TestRunRequest.ExecuteAsync: Started.
TpTrace Verbose: 0 : 5108, 5, 2018/05/11, 23:17:40.545,<PHONE_NUMBER>9, vstest.console.dll, ProxyExecutionManager: Test host is always Lazy initialize.
TpTrace Verbose: 0 : 5108, 1, 2018/05/11, 23:17:40.545,<PHONE_NUMBER>9, vstest.console.dll, TestRunRequest.WaitForCompletion: Waiting with timeout -1.
TpTrace Verbose: 0 : 5108, 5, 2018/05/11, 23:17:40.546,<PHONE_NUMBER>9, vstest.console.dll, TestRequestSender.InitializeCommunication: initialize communication.
TpTrace Information: 0 : 5108, 5, 2018/05/11, 23:17:40.559,<PHONE_NUMBER>1, vstest.console.dll, SocketServer: Listening on end point : <IP_ADDRESS>:1745
TpTrace Verbose: 0 : 5108, 5, 2018/05/11, 23:17:40.564,<PHONE_NUMBER>3, vstest.console.dll, DotnetTestHostmanager: Full path of dotnet.exe is C:\Program Files\dotnet\dotnet.exe
TpTrace Verbose: 0 : 5108, 5, 2018/05/11, 23:17:40.564,<PHONE_NUMBER>7, vstest.console.dll, DotnetTestHostmanager: Adding  --runtimeconfig "D:\data\CHT\6311c6f0\E2ETests\HttpServiceE2ETests\HttpServiceE2ETests.runtimeconfig.json" in args
TpTrace Verbose: 0 : 5108, 5, 2018/05/11, 23:17:40.564,<PHONE_NUMBER>0, vstest.console.dll, DotnetTestHostmanager: Adding  --depsfile "D:\data\CHT\6311c6f0\E2ETests\HttpServiceE2ETests\HttpServiceE2ETests.deps.json" in args
TpTrace Information: 0 : 5108, 5, 2018/05/11, 23:17:40.565,<PHONE_NUMBER>0, vstest.console.dll, AssemblyResolver: Microsoft.Extensions.DependencyModel: Resolving assembly.
TpTrace Information: 0 : 5108, 5, 2018/05/11, 23:17:40.565,<PHONE_NUMBER>6, vstest.console.dll, AssemblyResolver: Microsoft.Extensions.DependencyModel: Resolved assembly.
TpTrace Verbose: 0 : 5108, 5, 2018/05/11, 23:17:40.566,<PHONE_NUMBER>6, vstest.console.dll, DotnetTestHostmanager: Reading file D:\data\CHT\6311c6f0\E2ETests\HttpServiceE2ETests\HttpServiceE2ETests.deps.json to get path of testhost.dll
TpTrace Verbose: 0 : 5108, 5, 2018/05/11, 23:17:40.612,<PHONE_NUMBER>1, vstest.console.dll, DotnetTestHostmanager: Relative path of testhost.dll with respect to package folder is microsoft.testplatform.testhost/15.3.0\lib/netstandard1.5/testhost.dll
TpTrace Verbose: 0 : 5108, 5, 2018/05/11, 23:17:40.618,<PHONE_NUMBER>2, vstest.console.dll, DotnetTestHostmanager: Looking for path microsoft.testplatform.testhost/15.3.0\lib/netstandard1.5/testhost.dll in folder C:\Users_qcloud2.dotnet\store|arch||tfm|
TpTrace Verbose: 0 : 5108, 5, 2018/05/11, 23:17:40.618,<PHONE_NUMBER>5, vstest.console.dll, DotnetTestHostmanager: Looking for path microsoft.testplatform.testhost/15.3.0\lib/netstandard1.5/testhost.dll in folder d:\dbs\Cx\iri.N
TpTrace Verbose: 0 : 5108, 5, 2018/05/11, 23:17:40.619,<PHONE_NUMBER>0, vstest.console.dll, DotnetTestHostManager: Assume published test project, with test host path = D:\data\CHT\6311c6f0\E2ETests\HttpServiceE2ETests\testhost.dll.
TpTrace Information: 0 : 5108, 5, 2018/05/11, 23:17:40.619,<PHONE_NUMBER>9, vstest.console.dll, AssemblyResolver: Microsoft.TestPlatform.TestHostRuntimeProvider.resources: Resolving assembly.
TpTrace Information: 0 : 5108, 5, 2018/05/11, 23:17:40.619,<PHONE_NUMBER>8, vstest.console.dll, AssemblyResolver: Microsoft.TestPlatform.TestHostRuntimeProvider.resources: Failed to load assembly.
TpTrace Information: 0 : 5108, 5, 2018/05/11, 23:17:40.620,<PHONE_NUMBER>4, vstest.console.dll, AssemblyResolver: Microsoft.TestPlatform.TestHostRuntimeProvider.resources: Resolving assembly.
TpTrace Information: 0 : 5108, 5, 2018/05/11, 23:17:40.621,<PHONE_NUMBER>8, vstest.console.dll, AssemblyResolver: Microsoft.TestPlatform.TestHostRuntimeProvider.resources: Resolved from cache.
TpTrace Verbose: 0 : 5108, 5, 2018/05/11, 23:17:40.621,<PHONE_NUMBER>8, vstest.console.dll, DotnetTestHostmanager: Unable to find tests for D:\data\CHT\6311c6f0\E2ETests\HttpServiceE2ETests\HttpServiceE2ETests.dll. Make sure test project has a nuget reference of package "Microsoft.NET.Test.Sdk" and framework version settings are appropriate. Rerun with /diag option to diagnose further.
TpTrace Error: 0 : 5108, 5, 2018/05/11, 23:17:40.694,<PHONE_NUMBER>9, vstest.console.dll, ProxyExecutionManager.StartTestRun: Failed to start test run: System.IO.FileNotFoundException: Unable to find tests for D:\data\CHT\6311c6f0\E2ETests\HttpServiceE2ETests\HttpServiceE2ETests.dll. Make sure test project has a nuget reference of package "Microsoft.NET.Test.Sdk" and framework version settings are appropriate. Rerun with /diag option to diagnose further.
at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Hosting.DotnetTestHostManager.GetTestHostProcessStartInfo(IEnumerable1 sources, IDictionary2 environmentVariables, TestRunnerConnectionInfo connectionInfo)
at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyOperationManager.SetupChannel(IEnumerable`1 sources, CancellationToken cancellationToken)
at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyExecutionManager.StartTestRun(TestRunCriteria testRunCriteria, ITestRunEventsHandler eventHandler)
TpTrace Verbose: 0 : 5108, 5, 2018/05/11, 23:17:40.770,<PHONE_NUMBER>7, vstest.console.dll, TestRunRequest:SendTestRunMessage: Starting.
TpTrace Information: 0 : 5108, 5, 2018/05/11, 23:17:40.770,<PHONE_NUMBER>9, vstest.console.dll, TestRunRequest:SendTestRunMessage: Completed.
TpTrace Verbose: 0 : 5108, 5, 2018/05/11, 23:17:40.771,<PHONE_NUMBER>4, vstest.console.dll, ParallelProxyExecutionManager: HandlePartialRunComplete: Total completed clients = 1, Run complete = True, Run canceled: False.
TpTrace Verbose: 0 : 5108, 5, 2018/05/11, 23:17:41.842,<PHONE_NUMBER>9, vstest.console.dll, TestRunRequest:TestRunComplete: Starting. IsAborted:True IsCanceled:False.
TpTrace Verbose: 0 : 5108, 5, 2018/05/11, 23:17:41.843,<PHONE_NUMBER>0, vstest.console.dll, TestLoggerManager.HandleTestRunComplete: Ignoring as the object is disposed.
TpTrace Information: 0 : 5108, 5, 2018/05/11, 23:17:41.843,<PHONE_NUMBER>2, vstest.console.dll, TestRunRequest:TestRunComplete: Completed.
TpTrace Verbose: 0 : 5108, 1, 2018/05/11, 23:17:41.843,<PHONE_NUMBER>1, vstest.console.dll, TestRunRequest.Dispose: Starting.
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:41.844,<PHONE_NUMBER>3, vstest.console.dll, TestRunRequest.Dispose: Completed.
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:41.844,<PHONE_NUMBER>9, vstest.console.dll, TestRequestManager.RunTests: run tests completed.
TpTrace Information: 0 : 5108, 1, 2018/05/11, 23:17:41.844,<PHONE_NUMBER>5, vstest.console.dll, RunTestsArgumentProcessor:Execute: Test run is completed.
TpTrace Verbose: 0 : 5108, 1, 2018/05/11, 23:17:41.844,<PHONE_NUMBER>7, vstest.console.dll, Executor.Execute: Exiting with exit code of 1
Environment

Please share additional details about the test environment.
Operating system, Build version of vstest.console
.NET Core SDK (reflecting any global.json):
Version:   2.1.300-preview2-008530
Commit:    822ae6d43a

Runtime Environment:
OS Name:     Windows
OS Version:  10.0.14393
OS Platform: Windows
RID:         win10-x64
Base Path:   C:\Program Files\dotnet\sdk\2.1.300-preview2-008530\
Host (useful for support):
Version: 2.1.0-preview2-26406-04
Commit:  6833f3026b

Microsoft (R) Test Execution Command Line Tool Version 15.7.0-preview-20180221-13
here is httpservicee2etests.deps (3).json that indicate Microsoft.NET.Test.Sdk.
"Microsoft.NET.Test.Sdk/15.3.0": {
"dependencies": {
"Microsoft.TestPlatform.TestHost": "15.3.0"
}
},

Hi,
I'm facing the same issue when trying to execute some Selenium test as part of the release pipeline...
I have enabled diagnostics and a file seems to have been created, but not sure how I can access it.
Logging Vstest Diagnostics in file: D:\a\r1\a\diag.log
But how do I access that?
Thanks!