TypeError: string indices must be integers, not 'str' after MFA response

Expected Behavior
An AWS session should be stored in ~/.aws/credentials

Current Behavior
I receive the following python error message
gimme-aws-creds --profile my-profile
Using inherited config: DEFAULT
Okta Classic login flow enabled
Using password from keyring for<EMAIL_ADDRESS>Multi-factor Authentication required.
Okta Verify App: SmartPhone_IPhone: iPhone selected
Okta Verify push sent...
Authentication Success! Calling Gimme-Creds Server...
Traceback (most recent call last):
  File "/opt/homebrew/bin/gimme-aws-creds", line 17, in <module>
    GimmeAWSCreds().run()
  File "/opt/homebrew/Cellar/gimme-aws-creds/2.8.2_2/libexec/lib/python3.12/site-packages/gimme_aws_creds/main.py", line 453, in run
    self._run()
  File "/opt/homebrew/Cellar/gimme-aws-creds/2.8.2_2/libexec/lib/python3.12/site-packages/gimme_aws_creds/main.py", line 876, in _run
    for data in self.iter_selected_aws_credentials():
  File "/opt/homebrew/Cellar/gimme-aws-creds/2.8.2_2/libexec/lib/python3.12/site-packages/gimme_aws_creds/main.py", line 845, in iter_selected_aws_credentials
    aws_results = executor.map(generate_credentials_prepare_data, self.aws_selected_roles)
                                                                  ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/gimme-aws-creds/2.8.2_2/libexec/lib/python3.12/site-packages/gimme_aws_creds/main.py", line 727, in aws_selected_roles
    selected_roles = self._get_selected_roles(self.requested_roles, self.aws_roles)
                                                                    ^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/gimme-aws-creds/2.8.2_2/libexec/lib/python3.12/site-packages/gimme_aws_creds/main.py", line 718, in aws_roles
    self.saml_data['SAMLResponse'],
    ^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/gimme-aws-creds/2.8.2_2/libexec/lib/python3.12/site-packages/gimme_aws_creds/main.py", line 709, in saml_data
    self._cache['saml_data'] = saml_data = self.okta.get_saml_response(self.aws_app['links']['appLink'], self.auth_session)
                                                                       ^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/gimme-aws-creds/2.8.2_2/libexec/lib/python3.12/site-packages/gimme_aws_creds/main.py", line 702, in aws_app
    self._cache['aws_app'] = aws_app = self._get_selected_app(self.conf_dict.get('aws_appname'), self.aws_results)
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/gimme-aws-creds/2.8.2_2/libexec/lib/python3.12/site-packages/gimme_aws_creds/main.py", line 331, in _get_selected_app
    if app["name"] == aws_appname:
       ~~~^^^^^^^^

Possible Solution

Steps to Reproduce (for bugs)

run gimme-aws-creds --profile your-profile
respond to the 2FA push notification

Context

My .okta_aws_login_config was set up ~1.5 years ago and has always worked since.
Your Environment

App Version used: 2.8.2
Environment name and version<EMAIL_ADDRESS>(also tried<EMAIL_ADDRESS>Operating System and version: macOS Sonoma 14.5

I'm grateful if you can point me in any direction that leads to a solution! :)

I just realised there was an issue with my gimme_creds_server, it's updated and I'm able to login now.


[Feature Request] MAUI Support

What is the problem you are attempting to solve with this feature?
We were waiting for Xamarin support for quite some time. But with MAUI around the corner, it would have been a double effort. Can we expect complete support for MAUI directly - iOS, Android, UWP, and MacOS?
A description of your proposed solution to the problem
It is fine to have native support for iOS, Android, React. But having cross-platform support using MAUI will make adoption much faster. Generating UI is one of the crucial parts of any app design and if AdaptiveCards becomes a defacto rendering engine for most of the display and input, more community contribution will increase adoption.
What alternatives or workarounds have you considered
We tried building binding projects for native implementations, If something works, something doesn't. MacOS is still elusive.
Please start with MAUI asap

Xamarin/MAUI support is being tracked on product board here: https://adaptivecards.productboard.com/portal/1-adaptive-cards-features/tabs/c88173a2-a3a1-4bc5-ad15-11051b3c7942/features/1961753/portal/expanded
Please upvote the feature there!


Zipcode is a number but should be a string

Thanks for creating an issue! Please fill out this form so we can be sure to have all the information we need, and to minimize back and forth.

What are we trying to do?

What feature or behavior is this required for?

This is workable, but not totally accurate, and since we have some funds potentially on deck we should rep their area codes properly.

How could we solve this issue? (Not knowing is okay!)

The current DB field is a number. I think I would suggest making this a string and writing a validator for it.

Anything else?
nope!

@elimbaum you interested in maybe taking a whack at this?

This is also probably a generally good idea, i.e. zip+4. I guess we're probably not worried about international addresses.


About code and model checkpoint

Hi, could I know when you will release the code and model weights? Many thanks.

@rikeilong  Hi. Thanks for your great work! Do you have plans to release the model weights?


Support for VoIP pushes

VoIP pushes is a new type of push in iOS 8 for VoIP apps. See WWDC 2014 video
https://developer.apple.com/videos/wwdc/2014/
"Writing Energy Efficient Code, Part 2"
starting about 36:20
It may be necessary to support both VoIP and regular pushes for an app.

Added in v1.7.3


bug when preview and publish image

i used draw.io to make a diagram, when i've preview my diagram in small zoom, i found that the arrow is wrong but it's just fine when i preview it in previous zoom.
here i include screenshot so you can more understand:
first zoom, look smaller, the arrows become wrong

result from first image

second zoom, look bigger, everything become fine.

result from secound image

hello, how about this bug?

get this exact same bug all the time. Arrows are looking fine and twisted depending on zoom levels. also preview can look fine in most zoom levels but image export is broken.

Please vote the issue up, "me too" just creates noise that helps nobody.

I'm spotting this too. Has anyone found a workaround to this?

From further trial-and-error investigation, it seems to occur if the boxes that the line is joining are almost- but-not-quite in alignment.

This should be fixed in 11.2.4, if you update.


Template and aliases tweaks

Update FindCommand and LocateCommand templates.
Update LocateCommand alias.
Remove ListCommand alias.
Add home ResultDisplay message.
Update README.adoc.

Pull Request Test Coverage Report for Build 340

1 of 2 (50.0%) changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 74.575%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/main/java/seedu/address/logic/commands/LocateCommand.java
0
1
0.0%

Totals

Change from base Build 338:
0.0%

Covered Lines:
1930

Relevant Lines:
2588

💛  - Coveralls


[#31991][prism] Allow Empty Composites

Allows composites that only do SDK Side PCollection graph manipulation.
These composites have no payload, or subtransforms, so there's nothing for a runner to do with
them for execution. Simply ignore them.
This allows 3 Java validates runner tests in PipelineTest to pass: testIdentityTransform, testTupleProjectionTransform, and testTupleInjectionTransform.
Fixes #31991

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

R: @damondouglas
We don't allow expressing this in the Go SDK so no coverage is possible here. Covered by the Java tests.


ingressClassName missing from ingress

Hi,
The ingressClassName is missing from the helm chart and templates.
https://kubernetes.io/docs/concepts/services-networking/ingress/#the-ingress-resource
It is available in the Rasa-X chart: https://github.com/RasaHQ/rasa-x-helm/blob/87a311f92481a2f476a517fb9c80766b16696022/charts/rasa-x/values.yaml#L830
Thanks.

I definitly need it, I use a kubectl patch command as a workaround but please add it.
IngressClass resource and spec.ingressClassName is in stable state since Kubernetes 1.19, it's been a while.

I just do a proposal here :
https://github.com/RasaHQ/helm-charts/pull/111

Needed too please!

+1

IngressClassName resource is the way to specify IngressClass since kubernetes 1.19 (released in  2020). PR have been done almost 6 month ago ....
Please could you check the assiocated PR ?

Any news ? You closed my PR 6 months ago but nothing has changed since then.


testing: data race is attributed to wrong test in -json formatting

What version of Go are you using (go version)?
Reproducible with these versions:

$ go version
go version go1.19.6 darwin/amd64

$ go1.20.1 version
go version go1.20.1 darwin/amd64

$ go1.13 version
go version go1.13 darwin/amd64

$ gotip version
go version devel go1.21-99bc53f5e8 Mon Feb 20 02:28:50 2023 +0000 darwin/amd64

Does this issue reproduce with the latest release?
Yes.
What operating system and processor architecture are you using (go env)?
go env Output
$ go env
GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/vearutop/Library/Caches/go-build"
GOENV="/Users/vearutop/Library/Application Support/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/vearutop/go/pkg/mod"
GONOPROXY="github.com/adjust,github.com/vearutop,github.com/Unbotify"
GONOSUMDB="github.com/adjust,github.com/vearutop,github.com/Unbotify"
GOOS="darwin"
GOPATH="/Users/vearutop/go"
GOPRIVATE="github.com/adjust,github.com/vearutop,github.com/Unbotify"
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/opt/go/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/opt/go/libexec/pkg/tool/darwin_amd64"
GOVCS=""
GOVERSION="go1.19.6"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/vearutop/dev/teststat/go.mod"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/n7/jv2b0zv57jzc8y6zhmfzbn7c0000gn/T/go-build1651747709=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?
I'm running racy tests with -race -json.
The reproducing tests are available here https://github.com/vearutop/teststat/tree/faulty-tests.
go test -race -json -count 5 ./... > run.jsonl

imperfect_test.go
package main_test

import (
	"math/rand"
	"testing"
	"time"
)

func TestThatIsRacy(t *testing.T) {
	t.Parallel()

	a := 1

	for i := 0; i < 1000; i++ {
		go func() { a++ }()
	}

	time.Sleep(10 * time.Millisecond)
	a++
}

func TestThatFlakes(t *testing.T) {
	t.Parallel()

	if rand.Int()%3 == 0 { //nolint
		return
	}

	t.Fatal("oh, I'm so flaky")
}

func TestThatFlakesToo(t *testing.T) {
	t.Parallel()

	if rand.Int()%5 == 0 { //nolint
		return
	}

	t.Fatal("oh, I'm even more flaky")
}

func TestThatIsSometimesSlow(t *testing.T) {
	t.Parallel()

	if rand.Int()%3 == 0 { //nolint
		time.Sleep(1 * time.Second)
	}
}

func TestThatIsAlwaysSlow(t *testing.T) {
	t.Parallel()
	time.Sleep(1 * time.Second)
}

What did you expect to see?
Data race has reference to TestThatIsRacy.
...
{"Time":"2023-02-22T11:17:42.83469+01:00","Action":"output","Package":"github.com/vearutop/teststat","Test":"TestThatIsRacy","Output":"WARNING: DATA RACE\n"}
...

What did you see instead?
Data race has reference to another test (TestThatFlakesToo).
Part of run.jsonl:
...
{"Time":"2023-02-22T11:17:42.833996+01:00","Action":"cont","Package":"github.com/vearutop/teststat","Test":"TestThatFlakesToo"}
{"Time":"2023-02-22T11:17:42.834018+01:00","Action":"output","Package":"github.com/vearutop/teststat","Test":"TestThatFlakesToo","Output":"=== CONT  TestThatFlakesToo\n"}
{"Time":"2023-02-22T11:17:42.834246+01:00","Action":"cont","Package":"github.com/vearutop/teststat","Test":"TestThatFlakes"}
{"Time":"2023-02-22T11:17:42.834271+01:00","Action":"output","Package":"github.com/vearutop/teststat","Test":"TestThatFlakes","Output":"=== CONT  TestThatFlakes\n"}
{"Time":"2023-02-22T11:17:42.83467+01:00","Action":"output","Package":"github.com/vearutop/teststat","Test":"TestThatFlakes","Output":"==================\n"}
{"Time":"2023-02-22T11:17:42.83469+01:00","Action":"output","Package":"github.com/vearutop/teststat","Test":"TestThatFlakes","Output":"WARNING: DATA RACE\n"}
{"Time":"2023-02-22T11:17:42.834716+01:00","Action":"output","Package":"github.com/vearutop/teststat","Test":"TestThatFlakes","Output":"Read at 0x00c00011ad18 by goroutine 15:\n"}
{"Time":"2023-02-22T11:17:42.834739+01:00","Action":"output","Package":"github.com/vearutop/teststat","Test":"TestThatFlakes","Output":"  github.com/vearutop/teststat_test.TestThatIsRacy.func1()\n"}
{"Time":"2023-02-22T11:17:42.834748+01:00","Action":"output","Package":"github.com/vearutop/teststat","Test":"TestThatFlakes","Output":"      /Users/vearutop/dev/teststat/imperfect_test.go:15 +0x30\n"}
{"Time":"2023-02-22T11:17:42.83478+01:00","Action":"output","Package":"github.com/vearutop/teststat","Test":"TestThatFlakes","Output":"\n"}
{"Time":"2023-02-22T11:17:42.834797+01:00","Action":"output","Package":"github.com/vearutop/teststat","Test":"TestThatFlakes","Output":"Previous write at 0x00c00011ad18 by goroutine 17:\n"}
{"Time":"2023-02-22T11:17:42.834811+01:00","Action":"output","Package":"github.com/vearutop/teststat","Test":"TestThatFlakes","Output":"  github.com/vearutop/teststat_test.TestThatIsRacy.func1()\n"}
{"Time":"2023-02-22T11:17:42.834819+01:00","Action":"output","Package":"github.com/vearutop/teststat","Test":"TestThatFlakes","Output":"      /Users/vearutop/dev/teststat/imperfect_test.go:15 +0x44\n"}
{"Time":"2023-02-22T11:17:42.834825+01:00","Action":"output","Package":"github.com/vearutop/teststat","Test":"TestThatFlakes","Output":"\n"}
{"Time":"2023-02-22T11:17:42.834831+01:00","Action":"output","Package":"github.com/vearutop/teststat","Test":"TestThatFlakes","Output":"Goroutine 15 (running) created at:\n"}
{"Time":"2023-02-22T11:17:42.834837+01:00","Action":"output","Package":"github.com/vearutop/teststat","Test":"TestThatFlakes","Output":"  github.com/vearutop/teststat_test.TestThatIsRacy()\n"}
{"Time":"2023-02-22T11:17:42.834861+01:00","Action":"output","Package":"github.com/vearutop/teststat","Test":"TestThatFlakes","Output":"      /Users/vearutop/dev/teststat/imperfect_test.go:15 +0x5b\n"}
{"Time":"2023-02-22T11:17:42.834868+01:00","Action":"output","Package":"github.com/vearutop/teststat","Test":"TestThatFlakes","Output":"  testing.tRunner()\n"}
{"Time":"2023-02-22T11:17:42.834874+01:00","Action":"output","Package":"github.com/vearutop/teststat","Test":"TestThatFlakes","Output":"      /Users/vearutop/sdk/go1.20.1/src/testing/testing.go:1576 +0x216\n"}
{"Time":"2023-02-22T11:17:42.83488+01:00","Action":"output","Package":"github.com/vearutop/teststat","Test":"TestThatFlakes","Output":"  testing.(*T).Run.func1()\n"}
{"Time":"2023-02-22T11:17:42.834886+01:00","Action":"output","Package":"github.com/vearutop/teststat","Test":"TestThatFlakes","Output":"      /Users/vearutop/sdk/go1.20.1/src/testing/testing.go:1629 +0x47\n"}
{"Time":"2023-02-22T11:17:42.834912+01:00","Action":"output","Package":"github.com/vearutop/teststat","Test":"TestThatFlakes","Output":"\n"}
{"Time":"2023-02-22T11:17:42.834922+01:00","Action":"output","Package":"github.com/vearutop/teststat","Test":"TestThatFlakes","Output":"Goroutine 17 (finished) created at:\n"}
{"Time":"2023-02-22T11:17:42.83493+01:00","Action":"output","Package":"github.com/vearutop/teststat","Test":"TestThatFlakes","Output":"  github.com/vearutop/teststat_test.TestThatIsRacy()\n"}
{"Time":"2023-02-22T11:17:42.834939+01:00","Action":"output","Package":"github.com/vearutop/teststat","Test":"TestThatFlakes","Output":"      /Users/vearutop/dev/teststat/imperfect_test.go:15 +0x5b\n"}
{"Time":"2023-02-22T11:17:42.834948+01:00","Action":"output","Package":"github.com/vearutop/teststat","Test":"TestThatFlakes","Output":"  testing.tRunner()\n"}
{"Time":"2023-02-22T11:17:42.834955+01:00","Action":"output","Package":"github.com/vearutop/teststat","Test":"TestThatFlakes","Output":"      /Users/vearutop/sdk/go1.20.1/src/testing/testing.go:1576 +0x216\n"}
{"Time":"2023-02-22T11:17:42.834962+01:00","Action":"output","Package":"github.com/vearutop/teststat","Test":"TestThatFlakes","Output":"  testing.(*T).Run.func1()\n"}
{"Time":"2023-02-22T11:17:42.834969+01:00","Action":"output","Package":"github.com/vearutop/teststat","Test":"TestThatFlakes","Output":"      /Users/vearutop/sdk/go1.20.1/src/testing/testing.go:1629 +0x47\n"}
{"Time":"2023-02-22T11:17:42.834975+01:00","Action":"output","Package":"github.com/vearutop/teststat","Test":"TestThatFlakes","Output":"==================\n"}
{"Time":"2023-02-22T11:17:42.834984+01:00","Action":"output","Package":"github.com/vearutop/teststat","Test":"TestThatFlakes","Output":"    imperfect_test.go:29: oh, I'm so flaky\n"}
{"Time":"2023-02-22T11:17:42.834992+01:00","Action":"output","Package":"github.com/vearutop/teststat","Test":"TestThatFlakesToo","Output":"    imperfect_test.go:39: oh, I'm even more flaky\n"}
{"Time":"2023-02-22T11:17:42.835022+01:00","Action":"output","Package":"github.com/vearutop/teststat","Test":"TestThatFlakes","Output":"--- FAIL: TestThatFlakes (0.00s)\n"}
{"Time":"2023-02-22T11:17:42.835056+01:00","Action":"fail","Package":"github.com/vearutop/teststat","Test":"TestThatFlakes","Elapsed":0}
{"Time":"2023-02-22T11:17:42.835066+01:00","Action":"output","Package":"github.com/vearutop/teststat","Test":"TestThatFlakesToo","Output":"--- FAIL: TestThatFlakesToo (0.00s)\n"}
{"Time":"2023-02-22T11:17:42.848652+01:00","Action":"fail","Package":"github.com/vearutop/teststat","Test":"TestThatFlakesToo","Elapsed":0}
{"Time":"2023-02-22T11:17:42.84873+01:00","Action":"output","Package":"github.com/vearutop/teststat","Test":"TestThatIsRacy","Output":"    testing.go:1446: race detected during execution of test\n"}
{"Time":"2023-02-22T11:17:42.848836+01:00","Action":"output","Package":"github.com/vearutop/teststat","Test":"TestThatIsRacy","Output":"--- FAIL: TestThatIsRacy (0.02s)\n"}
{"Time":"2023-02-22T11:17:43.621831+01:00","Action":"fail","Package":"github.com/vearutop/teststat/foo","Test":"TestThatIsRacy","Elapsed":0.02}
{"Time":"2023-02-22T11:17:43.621899+01:00","Action":"output","Package":"github.com/vearutop/teststat/foo","Test":"TestThatIsAlwaysSlow","Output":"    testing.go:1446: race detected during execution of test\n"}
...

@bcmills per owners.
Possibly relaed to #57305?

Perhaps this is an "original" behavior from long ago, as it is reproducible with go1.13 (the least version I've tried) too.


update workshop with new titan embeddings

Issue #, if available:
Description of changes:

Update BedrockEmbeddings to use latest embeddings model v2.
Tidy up from typos and text, mainly in chatbot notebooks (reference to GPT-J, Chroma, etc).

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

We merged a chatbot update recently, can we resolve the conflict first?

approved

have the conflicts been resolved ?

This branch has conflicts that must be resolved
Use the web editor or the to resolve conflicts.
Conflicting files
04_Chatbot/00_Chatbot_Titan.ipynb

It is the kernel. I am using Data Science 3 compared to conda_python_3. It is the same for the other notebooks and wasn't flagged.

Resolved conflict.


When supporting default values for binary questions, add hidden images?

This would be to support loading and especially to cache them offline

options:

just replace jr://images/img.png to resolvable http(s) URL as with other data. Do the rest in the drawing widget, and form cache.

Problem is that the form cache depends on finding src attributes before widgets are loaded.

test form: http://localhost:8085/transform?xform=http://localhost:8081/image-default.xml

[ ] refactor


Could not extract the distinct types from Ballerina object at Java level

Description:
We are implementing HTTP interceptors and we have a requirement to differentiate some service objects based on the distinct type references at Java level.
Describe your problem(s)
We have service classes which have different distinct type references. For example,
service class requestInterceptor1 {
    *http:RequestInterceptor;

    resource function post greeting (http:RequestContext ctx, http:Caller caller, http:Request req)
                  returns http:InterceptorReturnValues {
       return ctx.next();
    }
}

where RequestInterceptor is defined inside HTTP module as
# The HTTP request interceptor service object
public type RequestInterceptor distinct service object {

};

We also have some other similar distinct service objects such as ResponseInterceptor, RequestErrorInterceptor and ResponseErrorInterceptor. And different service classes can be created using one of this type reference as shown above.
Then we pass them as an array though listener configuration :
listener http:Listener serverEP = check new(9090, config = {
    interceptors : [new requestInterceptor1(), new requestInterceptor2()]
    });

And we receive these objects at Java level as BObjects with reference to these distinct types in typeIdSet of the object's type. But there is no direct functions at java level to extract these distinct types. Also accessing these typeIdSet is not allowed outside the language and kept as internal.
Describe your solution(s)
Creating a function to extract these distinct types from BObject
Related Issues (optional):
Introduce HTTP Filters #692

Fixed via Api to access Informations of Type Inclusions at the Runtime #33684


Feature/public pypi

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers.
Motivation
Please describe the motivation of this PR and the goal you want to achieve through this PR.
Modification
Please briefly describe what modification is made in this PR.
Checklist

Pre-commit or other linting tools are used to fix the potential lint issues.
The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness.
If the modification has potential influence on downstream projects, this PR should be tested with downstream projects, like MMDet or MMCls.
The documentation has been modified accordingly, like docstring or example tutorials.

Codecov Report

:exclamation: No coverage uploaded for pull request base (main@098cb73). Click here to learn what that means.
Patch has no changes to coverable lines.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #131   +/-   ##
=======================================
  Coverage        ?   61.71%
=======================================
  Files           ?       63
  Lines           ?     1959
  Branches        ?      312
=======================================
  Hits            ?     1209
  Misses          ?      646
  Partials        ?      104

Flag
Coverage Δ

unittests
61.71% <0.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


mapcidr: 0.0.8 -> 1.0.1

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://github.com/projectdiscovery/mapcidr/releases.
meta.description for mapcidr is: "Small utility program to perform multiple operations for a given subnet/CIDR ranges"
meta.homepage for mapcidr is: "https://github.com/projectdiscovery/mapcidr"
meta.changelog for mapcidr is: ""
Updates performed

Golang update

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact

Checks done (click to expand)

built on NixOS
The tests defined in passthru.tests, if any, passed
found 1.0.1 with grep in /nix/store/rw621df6692d1g7rbsbalhcyzp3jb6gi-mapcidr-1.0.1
found 1.0.1 in filename of file in /nix/store/rw621df6692d1g7rbsbalhcyzp3jb6gi-mapcidr-1.0.1

Rebuild report (if merged into master) (click to expand)

1 total rebuild path(s)

1 package rebuild(s)

First fifty rebuilds by attrpath
mapcidr

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/rw621df6692d1g7rbsbalhcyzp3jb6gi-mapcidr-1.0.1 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A mapcidr https://github.com/r-ryantm/nixpkgs/archive/bfecef1ab22fe311e7665f25bb0de5e12d9edbd5.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/rw621df6692d1g7rbsbalhcyzp3jb6gi-mapcidr-1.0.1
ls -la /nix/store/rw621df6692d1g7rbsbalhcyzp3jb6gi-mapcidr-1.0.1/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

mapcidr

Maintainer pings
cc @HanEmile for testing.

Result of nixpkgs-review pr 183526 run on x86_64-linux 1

1 package built:

mapcidr


x/net/websocket: websocket.Conn.Close leaks file descriptors if remote has disconnected

In the current Conn.Close behaviour can leak file descriptors if the server has gone away before the client calls .Close
// Close implements the io.Closer interface.
func (ws *Conn) Close() error {
	err := ws.frameHandler.WriteClose(ws.defaultCloseStatus)
	if err != nil {
		return err
	}
	return ws.rwc.Close()
}

In the case that the remote side has closed the connection, WriteClose will return an error which will be returned to the caller without calling ws.rwc.Close. As it is customary to ignore the error from an io.Closer.Close, and even when not, the error is treated as opaque there is no indication that Close didn't actually release the file descriptor associated with ws.rwc. Even if the error was handled from the caller of Close, it would typically be io.EOF or some operating system specific version of "connection closed unexpectedly".
/cc @bradfitz

Closed by https://github.com/golang/net/commit/bb64f4dc73d4ab97978d5e1cb34515dcc570361b.


[macOS] SpineInterface can't access the shared memory

Bug description
SpineInterface cannot access the shared memory file, and my agent fails:
INFO: Analyzed target //agents/pid_balancer:pid_balancer (4 packages loaded, 13 targets configured).
INFO: Found 1 target...
Target //agents/pid_balancer:pid_balancer up-to-date:
  bazel-bin/agents/pid_balancer/pid_balancer
INFO: Elapsed time: 7.828s, Critical Path: 0.05s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
INFO: Running command line: bazel-bin/agents/pid_balancer/pid_balancer -c bullet
[2024-04-30 00:05:39,678] [info] Loading configuration 'config/common.gin' (main.py:78)
[2024-04-30 00:05:39,681] [info] Loading configuration 'config/bullet.gin' (main.py:78)
Waiting for spine /vulp to start...
FileNotFoundError: /vulp
Traceback (most recent call last):
  File "/private/var/tmp/_bazel_bora/ee02e7f8ec3e022d620c305986e67335/execroot/upkie/bazel-out/darwin-opt/bin/agents/pid_balancer/pid_balancer.runfiles/upkie/agents/pid_balancer/main.py", line 105, in <module>
    spine = SpineInterface()
            ^^^^^^^^^^^^^^^^
  File "/private/var/tmp/_bazel_bora/ee02e7f8ec3e022d620c305986e67335/execroot/upkie/bazel-out/darwin-opt/bin/agents/pid_balancer/pid_balancer.runfiles/vulp/vulp/spine/spine_interface.py", line 79, in __init__
    shared_memory = wait_for_shared_memory(shm_name, retries)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/private/var/tmp/_bazel_bora/ee02e7f8ec3e022d620c305986e67335/execroot/upkie/bazel-out/darwin-opt/bin/agents/pid_balancer/pid_balancer.runfiles/vulp/vulp/spine/spine_interface.py", line 53, in wait_for_shared_memory
    raise SpineError(
vulp.spine.exceptions.SpineError: spine /vulp did not respond after 1 attempts

Expected behavior
When launching try_pid_balancer.sh, the agent should be able to access the shared-memory file and run normally.
Reproduction steps
Steps to reproduce the behavior:

Run try_pid_balancer.sh on macOS (It works on Linux 20.04).

Additional context
The problem disappears once I checkout an earlier version of the spine_interface.py file, all other things remaining the same (e.g. git checkout b179d224da0468a93b6a9a051f6137b70f86498c -- vulp/spine/spine_interface.py ).
I am creating this issue here as it seems to be caused by vulp.
I suspect the issue is related to the standard library (or the way we use it), as posix_ipc works just fine.
System

OS: macOS 13.3
Vulp version: 2.2.1

Also, unit tests :wink: Would it be hard to make one for this particular case?
An integration test sounds good too :+1: We could add an option to try_pid_balancer.sh (forwarded to the Bullet spine) to stop after a fixed number of iterations, so that the test can run in an asynchronous and deterministic way. Can you propose a PR for this?

Hmm, I would be wary of using timeout in a CI workflow. Things timing-related tend to break in CI runners (e.g. in this instance it may time out out before the script does anything useful, therefore the test passes).

Maybe a Python test which creates a mock spine and tries to attach to it through SpineInterface could work, what do you think?

That's what the test in spine_interface_test.py does :see_no_evil: The reason it passed is that it already had the correct syntax for macOS:
https://github.com/upkie/vulp/blob/870bb20cd8fecc3a890968c54f6159eedc5c9f8c/vulp/spine/tests/spine_interface_test.py#L30


Add ability to disable the refreshing of import previews

Fixes #4009
Changes proposed in this pull request:

Added a checkbox to disable automatic preview updates
Implemented the feature for all parser types, google data, and databases
Added a test to preview_project.spec.js to test the feature on a csv file. The test passes (confirmed on cypress)

This pull request addresses issue 4009. This provides a way to hold off all preview updates when desired so the user can change all parameters without having to wait for the update between each change.
Note: I was able to test the feature on all file types except for google data. I was not able to successfully import data from a google doc (on my branch or on master)

@wetneb I thought the same thing about going through all of the different importers, but I wanted to stick with the design of the code. Maybe a refactorization down the road to make things like this easier would be a good idea. Going to resolve comments now...

Resolved comments from review.


Pango_lineage, Pangolin_version columns do not drop correctly

Check Documentation
I have checked the following places for your error:

[ ] nf-core website: troubleshooting
[ ] nf-core/scovtree pipeline documentation

Description of the bug

Steps to reproduce
Steps to reproduce the behaviour:

Command line:
See error:

Expected behaviour

Log files
Have you provided the following extra information/files:

[ ] The command used to run the pipeline
[ ] The .nextflow.log file

System

Hardware:
Executor:
OS:
Version

Nextflow Installation

Version:

Container engine

Engine:
version:
Image tag:

Additional context

Addressed by https://github.com/nhhaidee/scovtree/commit/d9295d05a0f64d08af35d292dac7e50e2ad4ef92


Speed Memory Tradeoff

Note: I'm putting this up here because it's a bit interesting, but not suggesting changes to the repo
I spent a couple of hours trying to see if I could get the benchmarks to run in the main process without having my memory explode, and after that failed, spent some more time trying to figure out why memory usage explodes in the first place, and it seems to be because of some unexpected (to me, at least) behavior on the part of the MLX memory allocator.
PyTorch's mps and cuda modules have cache clearing functions (e.g. torch.mps.empty_cache()), because unlike what usually happens in Python, the mps and cuda backends will greedily hold onto memory that has been allocated, even if there are no remaining references to a previously allocated tensor. I'm guessing this is because PyTorch often deals with huge amounts of memory, and because quickly giving back unused memory would mean lots of wasted time deallocating and then reallocating it when it's needed again. Eventually they will release the memory, but if you need to immediately clear out the memory that is not being used for currently referenced tensors, you can call torch.BACKEND.empty_cache(), and get the memory back.
This means that if the benchmark flow is changed from
for op in op_list:
    for _ in iterations:
        for framework in frameworks:
            do benchmark

to
for framework in frameworks:
    for op in op_list:
        for _ in iterations:
            do benchmark

then all the mps benchmarks can be done first, calling torch.mps.empty_cache() after each one, so once the benchmarks are going to be run on mlx, all or most of the torch memory has been deallocated, and mlx won't be competing with torch for memory. The effect, at least on my machine, is that I don't see spikes to >25 gb of memory usage, even though I'm not running each benchmark in a separate process!
However, once the mlx benchmarks begin, there doesn't seem to be anything that can be done to stop the mlx memory allocator from holing up ~15 gigs of memory. I tried for quite a while, however, and found several seemingly general rules:

removing all references to an mlx array that has been evaluated will not on its own cause its memory to be returned to the system
as far as I can tell, that memory will not, in general, be returned to the system until the program exits
there is no function equivalent to torch.mps.empty_cache(), so mlx can't be forced to give unused memory back
mlx will not allocate additional memory if a new array is evaluated that has the shape of a previously evaluated array that no longer has a reference
mlx will allocated additional memory if a new array is evaluated that has a different shape from no longer referenced evaluated arrays, even if it has plenty of memory already in terms of size! (I think this is because memory allocations are done on a per-array basis, so in most cases a different size or data type won't fit)
mlx will deallocate memory if it grows too big, but it's unclear to me whether this is something handled by mlx or handled by the operating system (it seems to take quite a while before mlx starts to deallocate memory when there's very high memory pressure, which makes me wonder if it's the os stepping in, and not mlx, but again, speculation)

Example
a = mx.random.normal(shape) # say this array takes 1 gig of memory
# 0 gigs allocated, because a hasn't been evaluated

mx.eval(a)
# 1 gig allocated, because a was evaluated

b = mx.random.normal(shape)
# 1 gig allocated

mx.eval(b)
# 2 gigs allocated

a = None
b = None
# 2 gigs allocated--deleting references doesn't free memory

c = mx.random.normal(shape)
mx.eval(c)
# 2 gigs allocated--because c is the same shape (and let's assume dtype) as a and b,
# mlx doesn't allocate new memory

d = mx.random.normal(shape)
mx.eval(d)
# 2 gigs allocated

e = mx.random.normal(shape)
mx.eval(e)
# 3 gigs allocated--c and d took up the space that was originally allocated for a and b;
# now more is needed for e

c = None
d = None
e = None
# 3 gigs allocated

f = mx.random.normal(different_shape)
mx.eval(f)
# > 3 gigs allocated--because f is a different shape than any of the previously initialized
# arrays, new memory is allocated, even if f is smaller than the previous arrays!

So, it seems like this is why memory usage gets pretty high when running the benchmark on mlx ops. Many different shapes of arrays are created by the benchmark class instances, so even if you delete the references to the input data created by each of those instances, mlx will not free the memory, and will often allocate new memory when an array with a new shape or dtype is created.
One final note, however, is that although the memory used by mlx gets very high, I haven't seen it crash while running the benchmark in the way described above. When pytorch mps and mlx ops were running interleaved with each other in the main process, sometimes the benchmark would exit because there wasn't enough memory (for me, this was usually pytorch throwing a memory related exception). If they aren't interleaved however, by first doing all the mps tests, clearing the mps cache, and then running all of the mlx tests, mlx is able to run all of the tests fine, albeit at the cost of putting some of your other programs into swap.
If anyone also finds this interesting and has a macbook with 8 or 16gb of memory, I'd be very curious to see what happens when you run this version of the benchmark on your machine! Wondering in particular what memory usage you get, how much your system swaps, etc.

This is very interesting @igm503, thanks for taking the time to make this analysis. I also tried to use a single process on which running all benchmarks in the first place but it resulted with the same issues. As you mentioned, I noticed that memory consistently grows when the mlx benchmark begins.
Based on your code (i.e. without multiprocessing), I tried to run each of mlx_gpu and mps first in the benchmark to study the allocated memory of the python process before and after each benchmark ran.
Here are the results:
mlx_gpu

Before running: 4GB swap
After running: 11GB Python process memory , 9GB swap

mps

Before running: 5.8GB swap
After running: 370MB Python process memory , 5.6GB swap

This clearly shows that many artefacts remain in the mlx memory, where mps successfully releases its allocated tensors.
When running your code as is, I get no memory error though, and I still have a memory peak of ~11GB, which I suppose to be the blocked memory allocated by mlx.

@awni this may be of interest for you. Is this expected behavior?

This is very interesting, thank you.

Is this expected behavior?

Expected yes, optimal no.

This is the related issue in MLX https://github.com/ml-explore/mlx/issues/742
I don't know yet the path forward yet, but it's a priority to either improve allocation strategy in a general way and/or expose options to make it easier to configure for the given situation.

Unless you use a lot of memory, then the cache will be garbage collected

So if I understand well: in theory, we shouldn't have any memory overflow if we run all MLX operations in a single process, as the memory will always be freed when a lot of memory is already allocated in the process. However, we should not run MLX/PyTorch (or other framework) operations together in a single process, as the memory allocated by MLX will not be freed to let the other frameworks allocate their memory within the process.
If this is the case, I guess we can reduce the number of process spawns in the benchmark @igm503.

So if I understand well: in theory, we shouldn't have any memory overflow if we run all MLX operations in a single process (unless the allocation is larger than the available memory), as the memory from past allocations will always be freed for future large allocations within the process.

Correct!

However, we should not run MLX/PyTorch (or other framework) operations together in a single process, as the memory allocated by MLX will not be freed to let the other frameworks allocate their memory within the process.

Also correct! MLX will hold buffers in the cache and PyTorch has no way to get them from us. It's an issue we are aware of and something to improve.

I would really like to know if your benchmarks change when you reduce # of processes. I don't think they should..but you never know.

If this is the case, I guess we can reduce the number of process spawns in the benchmark @igm503. We could even run all framework operations in the main process except MLX operations in a separate process.

I'll try it out!

I've put the mlx_gpu, mlx_gpu_compiled, and mlx_cpu benchmarks back into their own processes. To reduce the time spent spawning and cleaning up processes, it's running 10 ops per process, rather than 1 op per process. The max memory usage I'm seeing is about 8 gigs. If you think that's a little high, it can be reduced by decreasing the number of ops per process (running 5 ops per process gives a max of around 4-5 gigs, IIRC). I put a comment in the code where that parameter is defined.

There doesn't appear to be a difference between running in a separate process and not, but it looks like 2d convolutions are faster in 0.5.0!

Awesome, thanks for checking!!! And yea our convolutions got (sometimes a lot) faster in 0.5!
Also you may be interested in this PR https://github.com/ml-explore/mlx/pull/761. Adds some memory use apis to help understand and control memory / cache behavior in MLX.

Amazing work @igm503, thanks! It seems the benchmark is now running faster without these many processes. 🚢


Multiple apiKey security parameters

I have the following valid swagger.yaml:
paths:
  /assets:
    get:
      tags:
        - Assets
      summary: Returns base paths for asset files
      security:
        - ClientVersion: []
        - ClientApiKey: []
        - ApiVersion: []
      responses:
        '200':
          description: Assets Paths
          schema:
            $ref: '#/definitions/AssetsResponse'

Almost every request to my API uses the same 3 headers, so I found it reduced boilerplate a lot to specify them in securityDefinitions.
It may not be the best way, as I am new to swagger, so I am open to advice if you know of a better way to achieve this, but at the same time, given that this is syntactically valid in swagger, it is a problem that only the first item in the security list is sent to the template as data.headerApiKey (line 115 of index.js), and the rest are ignored.

well, I don't think it is a very nice way to send headers (they are not all security headers), but you already said that yourself :-)
That being said: I can imagine more that one security header needed, so looping over the security array seems ok. But we will need to think about naming, and rendering them in the templates.
That would make only feasible after we do #117


add sast tools

@ioribrn
Thanks for the Pull Request! I plan to organize the tools separately as tables later. I'll merge it first and clean it up later!
like this..?
https://github.com/hahwul/WebHackersWeapons#weapons


coprocessor: support auto scale read-pool

Signed-off-by: glorv<EMAIL_ADDRESS>
What is changed and how it works?

Issue Number: ref #13313
What's Changed:

- Support automatically scale read pool thread count based on the CPU usage.
- Adjust coprocessor batch-size based on the cost time of each round.

Related changes

PR to update pingcap/docs/pingcap/docs-cn:
Need to cherry-pick to the release branch

Check List
Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No code

Side effects

Performance regression

Consumes more CPU
Consumes more MEM

Breaking backward compatibility

Release note
Support automatically scale read pool thread count based on the CPU usage.

please fix the compilation issue.

/test

@sticnarf PTAL again, thanks~

@sticnarf PTAL again, thanks~

@Connor1996 @BusyJay PTAL, thanks~

@BusyJay PTAL

/cc @5kbpers

/merge


Searching players has no results

Hi,
I know there was a issue raised (https://github.com/trydis/FIFA-Ultimate-Team-Toolkit/issues/282) but it was closed without any explanation so I want to ask if there is as issue related to searching players? I can get results only when I pass ResourceId to search parameters.
When I search for gold cards I get "Status = WaitingForActivation" in PerformRequestAsync method of SearchRequest class.
So is WebApp see my account as not activated? Even I can do everything by WebApp or Companion App or is there problem with searching players?

One reason is certainly because i wasn't paying enough attention when merging a pull request a while ago.
It used to only use the ResourceId if it was set, but as you can see here it will use it no matter what at the moment.
I guess there have been few complaints because most people search for specific players.
Until i fix it or someone sends a PR, you'll have to pull down the source code and change it.

Ah! Thanks for tip, saved me a lot of time :) 👍

Only change to:
if (ResourceId != 0)             {                 if (ResourceId <= 16777216)                     uriString += "&maskedDefId=" + ResourceId.CalculateBaseId();                 else                     uriString += "&definitionId=" + ResourceId;             }


malformed  tracing ConfigMap causes pod CrashLoopBackOff

/area monitoring
/kind bug
/kind good-first-issue
Expected Behavior
If user uses a malformed ConfigMap config-tracing, e.g. setting enable to true without zipkin-endpoint configured, or sample-rate is not a valid float, activator should disable tracing and continue or use a default value. Tracing is an observability feature and its misconfiguration shouldn't cause any pods to stop working.
Refer to https://github.com/knative/serving/issues/4610 and https://github.com/knative/eventing/issues/1417.
Actual Behavior
When a bad ConfigMap config-tracing is configured, following pods will become CrashLoopBackOff and keep on restarting:

Serving activator
Eventing broker filter
Eventing broker ingress
Eventing in-memory channel dispatcher
Eventing gcppubsub channel dispatcher
Eventing kafka channel dispatcher
Eventing natss channel dispatcher

We have validation in WH now, so this should not really be an issue.

/close


feat: search on the interactive book page

Signed-off-by: Aman<EMAIL_ADDRESS>Fixes #224
Describe the changes you have made in this PR -
Screenshots of the changes (If any) -
Note: Please check Allow edits from maintainers. if you would like us to assist in the PR.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

13 of 134 (9.7%)  changed or added relevant lines in 6 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-1.3%) to 67.85%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

lib/ui/views/ib/builders/ib_headings_builder.dart
1
2
50.0%

lib/ui/views/ib/ib_page_view.dart
6
7
85.71%

lib/ui/views/ib/syntaxes/ib_highlight_syntax.dart
0
4
0.0%

lib/ui/views/ib/builders/ib_highlight_builder.dart
1
11
9.09%

lib/ui/views/ib/ib_landing_view.dart
5
48
10.42%

lib/viewmodels/ib/ib_landing_viewmodel.dart
0
62
0.0%

Totals

Change from base Build<PHONE_NUMBER>:
-1.3%

Covered Lines:
3993

Relevant Lines:
5885

💛  - Coveralls


[NEMO-421] Release v0.2

JIRA: NEMO-421: Release v0.2
Major changes:

Update the versions of components according to the release v0.2.

Minor changes to note:

N/A

Tests for the changes:

N/A

Other comments:

I'll open a release vote current version.  If it passed, I'll update the versions to 0.3-SNAPSHOT and merge this PR. Please do not merge any other PR before the vote. Thanks :)

Closes #421

@wonook Let's merge this PR after the Travis test to prevent more commit tree divergency.


about training

@sadeepj Hi, thx at first. I wonder is it possible to train a CRF-RNN model using the compiled "high_dim_filter.so" in Tensorflow/Keras environment? I mean is backprop part also included in "high_dim_filter.so"?Thx.

@XhqGlorry11 Yes, it is included. More specifically, it has been implemented here: https://github.com/sadeepj/crfasrnn_keras/blob/master/high_dim_filter_grad.py

@sadeepj Got that and thx again！


v-for strips private fields from type, causes type errors when trying to pass into a property

Consider this example:
App.vue:
<script setup lang="ts">
  import { ref } from 'vue';
  import MyComponent from './component.vue'

  export class TestClass {
    private privateField = "";
    public publicField = "";
  }

  const objects = ref<TestClass[]>([]);
</script>

<template>
  <MyComponent v-for="obj in objects" :key="obj.publicField" :test-prop="obj" />
  <!--                                                        ^^^ This will error -->
</template>

component.vue:
<script setup lang="ts">
  import { TestClass } from './App.vue';
  defineProps<{
    testProp: TestClass
  }>();
</script>
<template>
</template>

In App.vue, the inferred type of obj in the v-for clause will be {publicField: string}, which is not compatible with the definition for TestClass as this requires the additional private field. Hence, when you try and use obj as a parameter to test-prop, it fails and complains about the missing field privateField, even though it should actually be there.

This problem also occurs in vue-tsc:

A reproduction repo is available here:
https://github.com/protango/volar-v-for-issue-demo

Thanks for the report! Private fields strips by ref(), please forward this problem to vue-next repo.


Spawning A Player and a Non-Player Object

Hi,
I try to implement the following simple game. Two clients (the host and the remote-client) move a Cube object that is on the Server.
I have introduced the NetworkedObject, the NetworkedTransform and the TrackedObject components. onto the Cube's prefab. I have selected the "Server Only" box in its NetworkedObject component. In my Network Manager object I have set the size to 1 in order to have two spawnable Prefabs, the Cube and the Player. When I start the standalone application and I select Host, only the Player appears, not the Cube. Then, I run the application from unity's editor as a Client and the second player appears. The two scenes seem to be well synced but the Cube is missing.
I have two questions:

How do I spawn an object? I want the players to start from different initial position because some weird stuff happen when they collide in the very beginning.

How am I supposed to instantiate the Cube properly and make it appear in the game? In the Multiplayer Networking Unity-tutorial (where the HLAPI was used instead), the non-player objects were instantiated within the overridden function OnStartServer(). What is the equivalent function in MLAPI? If there is not one, where should I instantiate my Cube? Furthermore they were using NetworkServer.spawn(non-player-object). What's the equivalent of this in MLAPI?

Thanks!

NetworkedObjects serverOnly works the same as the UNET HLAPI NetworkIdentity serverOnly. If its ticked. It will NOT be automatically replicated. If you are on master / later than 0.1.6, scene objects are automatically replicated. If you want to manually spawn objects. You simply call the Spawn method on your NetworkedObject and it will be replicated automatically. Please see the wiki for more details

Thanks for your response. I will go for the newest version. However, if I do want to spawn an object manually, could you be help me on how to issue the spawn command you mentioned?

Call the Spawn method on the NetworkedObject. It's a public.
As for deciding the spawn pos & rot for a player. It's added here https://github.com/TwoTenPvP/MLAPI/commit/21b5942106a585270841f7e6ff10f0042e455876

Hi,
I built my project from scratch using your latest MLAPI-commit. In my NetworkManager object I have added the NetworkingManager component and a NetManageHud script based on your example. However, I'm facing the following problem: when I try to drag my Player prefab into the Networked Prefabs --> Player Prefab Name it doesn't allow me to do so and therefore I get the warning "MLAPI: There is no PlayerPrefabName set". And when I try to type manually the name of my prefab I get the warning "MLAPI: There is no NetworkedPrefab with the name specified in the PlayerPrefabName". That's really strange cause I have added the NetworkedObject into my Player's prefab. How do I resolve this?
Furthermore I have one question regarding your NetworkConfig cause I'm using some methods in the NetManagerHud script. This is what my code looks like for the Host button:
   if (GUI.Button(new Rect(20, 120, 100, 20), "Start host"))
    {
        NetworkConfig config = new NetworkConfig()
        {
            ConnectAddress = "<IP_ADDRESS>",
            ConnectPort = 7777,
            PlayerPrefabName = "Player",
        };
        config.RegisteredScenes = new System.Collections.Generic.List<string>();
        config.RegisteredScenes.Add("MenuScene");
        config.EnableSceneSwitching = true;
        config.EnableEncryption = false;
        NetworkingManager.singleton.StartHost(null, null);
    }

What I don't get is how is the config being passed to the StartHost method? On your previous NetworkingManager version I was doing it like this NetworkingManager.singleton.StartHost(config);
Thanks!

Well, you are no longer supposed to really touch the settings of the NetworkConfig other than in the inspector. Exceptions are the callbacks etc.
Thus, set everything in the editor. Make sure you add your player prefab in the inspector under spawnable prefabs. Then take the NetworkedPrefabName of set the PlayerPrefabName to the same.

Inspector only? That seems...unusual. What's the reason for this? I have to be able to set certain values at runtime, such as the listener address and port, and ideally would be able to configure pooling at runtime also.

Under the NetworkManager there is no such thing as Spawnable prefabs. There is only something called Handle Object Spawning, which is ticked already. I have taken MLAPI_Editor from https://github.com/TwoTenPvP/MLAPI/releases. Could it be that there is some mismatch with this editor and your most recent commit? Do you have a new version of MLAPI_Editor?

Let me clarify. The NetworkConfig was previously an object created by the user and given to the NetManager. Now there is a public Serialized version. You can still edit everything in code, and you pretty much have to. Like address etc.
This means that previously it was code only. Now it's a combination.
Thus, now you have to get the NetManager singleton and access the NetConfig. Not create a new instance anymore. I want to make it readonly for the API sake, but I cannot due to Unity serialization.
And I think the SpawnablePrefabs are now called NetworkedPrefabs

Ok, I got this. So, this is how my NetManagerHud.cs (component of my Network Manager object) looks like:
=================================================================
using MLAPI;
using MLAPI.Data;
using MLAPI.MonoBehaviours.Core;
using System.Net;
using UnityEngine;
public class NetManagerHud : MonoBehaviour
{
private void OnGUI()
{
if (GUI.Button(new Rect(20, 20, 100, 20), "Start client"))
{
NetworkingManager.singleton.NetworkConfig.ConnectAddress = "<IP_ADDRESS>";
NetworkingManager.singleton.NetworkConfig.ConnectPort = 7777;
NetworkingManager.singleton.NetworkConfig.RegisteredScenes.Add("MenuScene");
NetworkingManager.singleton.NetworkConfig.EnableSceneSwitching = true;
NetworkingManager.singleton.NetworkConfig.EnableEncryption = false;
NetworkingManager.singleton.NetworkConfig.PlayerPrefabName = "Player";
NetworkingManager.singleton.StartClient();
}
    if (GUI.Button(new Rect(20, 70, 100, 20), "Start server"))
    {
        NetworkingManager.singleton.NetworkConfig.ConnectAddress = "<IP_ADDRESS>";
        NetworkingManager.singleton.NetworkConfig.ConnectPort = 7777;
        NetworkingManager.singleton.NetworkConfig.RegisteredScenes.Add("MenuScene");
        NetworkingManager.singleton.NetworkConfig.EnableSceneSwitching = true;
        NetworkingManager.singleton.NetworkConfig.EnableEncryption = false;
        NetworkingManager.singleton.StartServer();
    }

    if (GUI.Button(new Rect(20, 120, 100, 20), "Start host"))
    {
        NetworkingManager.singleton.NetworkConfig.ConnectAddress = "<IP_ADDRESS>";
        NetworkingManager.singleton.NetworkConfig.ConnectPort = 7777;
        NetworkingManager.singleton.NetworkConfig.RegisteredScenes.Add("MenuScene");
        NetworkingManager.singleton.NetworkConfig.EnableSceneSwitching = true;
        NetworkingManager.singleton.NetworkConfig.EnableEncryption = false;
        NetworkingManager.singleton.NetworkConfig.PlayerPrefabName = "Player";
        NetworkingManager.singleton.StartHost();
    }
}

private void OnUPnPComplete(bool success, IPAddress ipAddress)
{
    //Did UPNP succeed
    Debug.Log(success);
    //If it succeded. This is the public ip address
    Debug.Log(ipAddress);
}

}
=================================================================
However, when I click in either of the three buttons (Server, Client, Host), nothing really happens. And I still cannot set the PlayerPrefabName from the inspector. As I said, when I drag the prefub under the Networked Prefabs --> Player Prefab Name it doesn't allow me, and when I either type it myself or include in the above script like this:
NetworkingManager.singleton.NetworkConfig.PlayerPrefabName = "Player";
it's not working. I also get the following errors:
SocketException: An attempt was made to access a socket in a way forbidden by its access permissions.
NullReferenceException: Object reference not set to an instance of an object
MLAPI.NetworkingManagerComponents.Core.SpawnManager.OnDestroyObject (System.UInt32 networkId, System.Boolean destroyGameObject) (at <77cc1175d53a4c389c6742103e62a9fa>:0)
MLAPI.MonoBehaviours.Core.NetworkedObject.OnDestroy () (at <77cc1175d53a4c389c6742103e62a9fa>:0)
What am I doing wrong here?
Thanks!

By the way, in your NetworkedObjectEditor.cs file, I have commented out the last line, meaning:
EditorGUILayout.LabelField("ServerOnly: ", networkedObject.ServerOnly.ToString(), EditorStyles.label);

Do you think this has any impact?

One more question. In the Multiplayer Networking Unity-tutorial this is how they spawn the non-player objects called enemies.
public override void OnStartServer()
    {
        for (int i=0; i < numberOfEnemies; i++)
        {
            var spawnPosition = new Vector3(
                Random.Range(-8.0f, 8.0f),
                0.0f,
                Random.Range(-8.0f, 8.0f));

            var spawnRotation = Quaternion.Euler(
                0.0f,
                Random.Range(0,180),
                0.0f);

            var enemy = (GameObject)Instantiate(enemyPrefab, spawnPosition, spawnRotation);
            NetworkServer.Spawn(enemy);
        }

In my case I only have a cube name Jenga as the non-player object and my code looks like this:
using MLAPI;
using MLAPI.Data;
using MLAPI.MonoBehaviours.Core;
using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class JengaSpawner : NetworkedBehaviour {

    public GameObject jengaPrefab;

    void Start () {
            var spawnPosition = new Vector3(
                Random.Range(-1.0f, 1.0f),
                0.0f,
                Random.Range(-1.0f, 1.0f));

            var spawnRotation = Quaternion.Euler(
                0.0f,
                Random.Range(0, 180),
                0.0f);
            var jenga = (GameObject)Instantiate(jengaPrefab, spawnPosition, spawnRotation);
            networkedObject.Spawn();
    }

                // Update is called once per frame
                void Update () {

                }
}

What is the equivalent of onStartServer() for the MLAPI? Which function should I override here?
Thanks!

The issue you are facing appears to be a duplicate of something. Such as you trying to add two NetworkedPrefabs with the same name, Multiple scenes with the same name, Multiple channels with the same name etc. I will add warning messages when this occurs in the future.
As for your second question, the NetworkingManager has a "OnServerStarted" callback you can use. It's invoked when the server is ready.

Hi,
Thanks for your response. And what about the problem I described above?
When I create an empty Network Manager object and I add the NetworkingManager component, the Player Prefab Name under the Networked Prefabs doesn't seem to be responsive.
When I try to drag my Player prefab into the  "Player Prefab Name" field it doesn't allow me to do so and therefore I get the warning "MLAPI: There is no PlayerPrefabName set". And when I try to type manually the name of my prefab I get the warning "MLAPI: There is no NetworkedPrefab with the name specified in the PlayerPrefabName". Even if I set the size of te Networked Prefabs from 0 to 1, and drag my Player prefab there, I still get the warning "There is no PlayerPrefabName set." What is the problem here?
By the way, when I import your MLAPI-Editor I always have to comment out this line:
EditorGUILayout.LabelField("ServerOnly: ", networkedObject.ServerOnly.ToString(), EditorStyles.label);

otherwise it complains that "NetworkObject does not contain a definition of 'ServerOnly' and no extension method 'ServerOnly' accepting a first argument of type 'NetworkedObject could be found (are you missing a directive or an assembly reference?)' " Is there a more proper way to deal with this than commenting it out?
Thanks!

The editor line has no effect. That's fine to comment out.
What you need to do is this:
Create a prefab with a NetworkedObject. Set the prefab name or use the default. Then add it as a NetworkedPrefab and set the name to the same thing. Then set the PlayerPrefabName to the same aswell.
The prefab name has to be set in all 3 places. I will work on a nicer solution hopefully before 1.0

Here is a example that works fine. Note that the two perfabs PlayerPrefab & MyObject has a NetworkedObject component with the correct names set.

Awesome. You were crystal clear. I haven't realized that I needed to set my Player's prefab name in all the 3 places. I appreciate your answer. Things seem to be working now. Clients appear in both screens now. I get this weird behavior though:
When I start my standalone app I press HOST and the Server together with the local-client start well. When I start the app from the Unity Editor and I press the Client button, the clients appear in both screens, but one of them gets push far away, as if they are exploding with each other.  It's weird cause I move my local-client far away from its initial position so it will not collide with Client joining later. Why is this happening?
Thanks again for your support.

If you are using connection approval you can now specify a spawn location. That should work. What probably happends is that they spawn and collide before you move them.

Alright let me get this clear. I have not really understood how and where I should use in my script your Spawn method. I would appreciate it if you could give me an example. Below I attach my PlayerController.cs:
using MLAPI;
using MLAPI.Data;
using MLAPI.MonoBehaviours.Core;
using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class PlayerController : NetworkedBehaviour
{

    // Use this for initialization
    void Start()
    {
        if (isLocalPlayer)
        {
            GetComponent<MeshRenderer>().material.color = Color.blue;
        }
    }

    // Update is called once per frame
    void Update()
    {
        if (!isLocalPlayer)
        {
            return;
        }
        var x = Input.GetAxis("Horizontal") * Time.deltaTime * 150.0f;
        var z = Input.GetAxis("Vertical") * Time.deltaTime * 3.0f;

        transform.Rotate(0, x, 0);
        transform.Translate(0, 0, z);
    }
}

Should the spawning take place here and if so, how? If not, where should I instantiate and spawn my players?
Thanks!

Player objects are spawned automatically.
As for objects instantiated at runtime. Call spawn straight after Instantiation.
As for scene objects. They get replicated automatically.
Essentially. Call spawn when you want the object to be replicated. Unless it's a player object OR a scene object. They get replicated for you.

What if I want to Spawn the Player objects manually? Cause if I don't then they will collide on the scene in the very beginning and then vanish. Is this the spawn method you are referring to btw SpawnPlayerObject(uint clientId, uint networkId, Vector3 position, Quaternion rotation) ?
Thanks

hmm, right now player objects are spawned for you. There is no way to actually do it yourself without small workarounds.
The spawn method I was talking about lets you tell the MLAPI where to spawn the player, before it's even spawned. If you need to spawn it manually. Open a feature request issue and explain why you need it etc.

That's exactly what I'm looking for now. How to tell Mlapi where to spawn the players so they will not collide in the very beginning. How should I do it? Should it be in the abovementioned playerController script? If so, could you please give me an example how to force MLAPI spawn the players in the very beginning where I want them to be spawned?
Thanks

https://github.com/TwoTenPvP/MLAPI/wiki/Connection-Approval That wiki page should give you everything you need.

Oh thanks. Indeed it seems I have missed this one. I will work on this tomorrow and let you know if I get stuck. Have no words to express my appreciation. Cheers

Hi,
where am I supposed to insert this code?
private void Setup()
{
    NetworkingManager.singleton.NetworkConfig.ConnectionApprovalCallback = ApprovalCheck;
    NetworkingManager.singleton.StartHost();
}

private void ApprovalCheck(byte[] connectionData, int connectionId, Action<int, bool, Vector3, Quaternion> callback)
{
    //Your logic here
    bool approve = false;
    //If approve is true, the connection gets added. If it's false. The client gets disconnected
    callback(connectionId, approve, positionToSpawnAt, rotationToSpawnWith);
}

Should I modify your NetworkConfig file under the Data folder and recompile the MLAPI library?
Btw, the Players in your new example behave normally although they collide with themselves and the sphere you have beneath them. Why is this not the case for me. I really want to solve this problem cause my objects vanish after the initial collision. Not all the time, but I want a stable thing. Any help is more than welcome.
Thanks.

I don't know. But you register that callback where you call StartHost. Then simply invoke the callback with aproove = true and set your position. Why your players are behaving a certain way when colliding, I don't know and it's not really down to the networking lib.

Hi,
This is what my NetManagerHad.cs looks like according to your instructions:
using MLAPI;
using MLAPI.Data;
using MLAPI.MonoBehaviours.Core;
using System.Net;
using System;
using UnityEngine;

public class NetManagerHud : MonoBehaviour
{
    private void OnGUI()
    {
        if (GUI.Button(new Rect(20, 20, 100, 20), "Start client"))
        {
            NetworkingManager.singleton.NetworkConfig.ConnectionData = System.Text.Encoding.ASCII.GetBytes("1234");
            NetworkingManager.singleton.StartClient();
        }

        if (GUI.Button(new Rect(20, 70, 100, 20), "Start server"))
        {
            NetworkingManager.singleton.StartServer();
        }

        if (GUI.Button(new Rect(20, 120, 100, 20), "Start host"))
        {
            NetworkingManager.singleton.NetworkConfig.ConnectionApprovalCallback = ApprovalCheck;
            NetworkingManager.singleton.StartHost();
        }

        if (GUI.Button(new Rect(20, 170, 100, 20), "Stop"))
        {
            if (NetworkingManager.singleton.isServer || NetworkingManager.singleton.isHost)
            {
                NetworkingManager.singleton.StopHost();
                NetworkingManager.singleton.StopServer();
            }
            else
            {
                NetworkingManager.singleton.StopClient();
            }
        }
    }

    private void ApprovalCheck(byte[] connectionData, int connectionId, Action<int, bool, Vector3, Quaternion> callback)
    {
        bool approve = true;

        if (connectionData.Length != NetworkingManager.singleton.NetworkConfig.ConnectionData.Length)
             approve = false;

        for (int i = 0; i < connectionData.Length; i++)
            if (connectionData[i] != NetworkingManager.singleton.NetworkConfig.ConnectionData[i])
                approve = true;

        // Set spawning position and rotation for players
        Vector3 positionToSpawnAt = new Vector3(0, 0, 0);
        Quaternion rotationToSpawnWith = new Quaternion();
        rotationToSpawnWith.Set(0, 0, 0, 1);

        //If approve is true, the connection gets added. If it's false. The client gets disconnected
        callback(connectionId, approve, positionToSpawnAt, rotationToSpawnWith);
    }
}

I haven't set the spawnPosition and spawnRotation to what I want yet. For now I just want to check how I should call this. I know that it should be before the StartHost() command, but I need some help here.
Thanks.

Btw, there is no such thing as NetworkingManager.singleton.NetworkConfig.ConnectionApprovalCallback in your NetworkConfig.cs. But you do mention this method on your wiki.

Just do
NetworkingManager.singleton.ConnectionApprovalCallback = ApprovalCheck;
NetworkingManager.singleton.StartHost();

When I try what you suggest I get the following
void NetManagerHud.ApprovalCheck(byte[] connectionData, int connectionId, Action<int, bool, Vector3, Quaternion> callback)
No overload for 'ApprovalCheck' matches delegate 'Action<byte[], uint, Action<uint, bool, Vector3, Quaternion>>'
What's wrong here?

Hi,
I have down what you suggested. I have also ticked the box Connection Approval. However, I can't get the Client connected now. I also get the following warning on my console:
MLAPI: The messageType MLAPI_OnReceiveTransformFromClient is not registered.
What is wrong here? I try to debug my code and I realized that the ApprovalCheck method is never called. My NetManagerHud.cs looks exactly like the one I have attached above. Could you help me?

This example works fine for me:
    private void OnGUI()
    {
        if (GUI.Button(new Rect(20, 20, 100, 20), "Start client"))
        {
            NetworkingManager.singleton.StartClient();
        }

        if (GUI.Button(new Rect(20, 70, 100, 20), "Start server"))
        {
            NetworkingManager.singleton.StartServer();
        }

        if (GUI.Button(new Rect(20, 120, 100, 20), "Start host"))
        {
            NetworkingManager.singleton.ConnectionApprovalCallback = HandleApproval;
            NetworkingManager.singleton.StartHost();
        }
    }

    private void HandleApproval(byte[] data, uint clientId, Action<uint, bool, Vector3, Quaternion> callback)
    {
        callback(clientId, true, Vector3.zero, Quaternion.identity);
    }

Essentially. When you call StartClient, the client will send the ConnectionData to the server. The server then calls HandleApproval and gives you the data, in your case, the password. Then you call the callback to decide wheter or not to approve. if you give it false, the server will decline it. If true, it will accept the connection.
Just make sure ConnectionApproval is turned on.

The example was very helpful. However only the Client obeys to the spawnPosition I define in the HandleApproval. The Host starts from (0,0,0) no matter what, although my player's prefab position has been set to (3,0,3). How can I force the Host player to be spawned onto the position of my preference?

The host's client start position can be set straight in the StartHost method. Like this:
NetworkingManager.singleton.StartHost(myPos, myRot);

Awesome. That solved my problem big time. One thing to pinpoint here: In my Scene I have a gameobject called Cube which wish to move with my Players. In order to make my Cube being synchronized between two different unity projects, I had to add not only the NetworkedObject component but the TrackedObject as well (of course rigidbody is a necessary component too). In case someone needs that in the future.
Again congrats for the great job and thanks for the support.

The TrackedObject component is ONLY needed if you are doing Lag compensation.

Btw where in your code do you actually call the ConnectionApprovalCallback method? I mean properly,with input arguments. It has to be somewhere but I can't find it. It's not in the NetworkingManager. You only declare it there, but since it's public you could basically use it from everywhere.

The NetworkingManager was recently split for clarity. Here is where it's called
https://github.com/TwoTenPvP/MLAPI/blob/baa6518c0ecf3c0822816d40ce629ca67d5c3c07/MLAPI/NetworkingManagerComponents/Core/InternalMessageHandler.Receive.cs#L33

Thanks buddy

Hi again,
I have been struggling to introduce more than 2 players in my game. When the 3rd player tries to join in the game I get the following error in the Unity console associated with the third player:
ArgumentException: An item with the same key has already been added.
So, the game doesn't really start for the 3rd player, but I see him being stuck in the middle of the Scene in the other two players' screen. Any idea why is this happening? How can I fix this?
Thanks!

I will look into it. You can have 2^16 * 32 clients.

As this appears to be a bug. Please open a new issue and post the full stacktrace.

Actually, I've managed to reproduce it. It's been fixed in this commit
https://github.com/TwoTenPvP/MLAPI/commit/566328530993356fb637eaf38f6b34a8f0c7d270

I want to instantiate an object at runtime. How am I supposed to Spawn it? I know that I should spawn it after its instantiation, but I can't find your Spawn method. So, in my script I have a something like that.
public Transform block;

private void Start()
{
    Instantiate(block, new Vector3(4, 0.5f, 4.0f), Quaternion.identity);
}

I instead want to have something like that:
public Transform block;

public override void OnStartServer()
{
    Instantiate(block, new Vector3(4, 0.5f, 4.0f), Quaternion.identity);
    Spawn(block);
}

How am I supposed to spawn the object properly?
Thanks!

Right now the Spawn method is on the NetworkedObject.
GameObject go = Instantiate(block, new Vector3(4, 0.5f, 4.0f), Quaternion.identity);
go.GetComponent<NetworkedObject>().Spawn();

Also worth noting. Instead of OnStartServer. You should override NetworkStart

I do that but now the block does not appear any more, neither in the Host nor on the Client. So, I have done this
    public override void NetworkStart()
    {
        Debug.Log("The block should have been created by now.");
        createBlock();
    }

    void createBlock()
    {
        GameObject go = Instantiate(block, new Vector3(2.0f, 0.5f, 6.0f), Quaternion.identity);
        go.GetComponent<NetworkedObject>().Spawn();
    }

I don't see my comment in the console, so it doesn't really enter inside the NetworkStart().
Any idea?

NetworkStart does not get called on SceneObjects. I will adjust that for a future release. But what you could do if you need a starting point is to use the normal Start method. Only do that if the server is running when it gets invoked. otherwise there is a equivalent of OnStartServer. It's an action called OnServerStarted on the NetworkingManager.
NetworkingManager.singleton.OnServerStarted = () =>
{
    //My spawn code here
};
//OR
NetworkingManager.singleton.OnServerStart = MyMethod;

Cool. Will do that. Thanks

https://github.com/TwoTenPvP/MLAPI/commit/3abb311f82ad345c7190751867ef3ba22a065ba1
NetworkSpawn will get invoked on Scene objects starting in that commit.

Just so you know I have the following buggy behavior:
Let's say that I have two players, the Host and the Client. I spawn them properly so, the Host starts from position(2, 0.5f, 2) and the Client from position(-2, 0.5f, 2). When the Client joins the game, he always sees the Host in position(0, 0, 0). It is only when my Host moves that the Client manages to see his correct position. After that everything is synchronized. But this is a problematic behavior cause if I have another object in the position(0,0,0) then it instantaneously collides with the ghost-Host and they explode. So, I'm sure it has to do with the initial information the Client receives from the Server.
What do you think causes that?

I will look into it.
Btw, github issues are not very good at long convos. Takes long to lead etc. Try to open new issues OR comment on the forum thread. https://forum.unity.com/threads/mlapi-hlapi-replacement.511277/

The issue has been fixed in this commit
https://github.com/TwoTenPvP/MLAPI/commit/e0ee0862c6e2b76b389cbd969e4a2be245bca012

Actually, might be something else. I will keep looking into it as it doesnt seem to be fixed.

Alright. Keep me updated. In the meanwhile if I end up with another problem I will open a new issue for convenience . I hope I have some minimum contribution in helping you improve MLAPI. Keep up the good work!

Well, you help a lot by refining the small things that make the biggest difference. Anyways, that commit fixed the issue. However, the issue persists if you use the NetworkedTransform. So i'll look into that.

https://github.com/TwoTenPvP/MLAPI/commit/20b315b7fc3de2f25c9ebc81f4b27624bf7010fb Fixed the NetworkedTransform issue. Now it works fine.


URL is not shown right

Tell us about your environment:

Puppeteer version:
Platform / OS version:
URLs (if applicable):
Node.js version:

What steps will reproduce the problem?
Please include code that reproduces the issue.
browser = await puppeteer.launch({headless: false, defaultViewport: {width: 1280, height: 720}, args: ['--proxy-server=socks5://<IP_ADDRESS>:9050']});
page = await browser.newPage();
await page.goto("https://web.telegram.org/#/login", {waitUntil: "load"});
console.log(page.url());

What is the expected result?
https://web.telegram.org/#/login
What happens instead?
https://web.telegram.org/

My fault! The # is an ID selection. I thought it was an endpoint. That part can be got with:
                const hash = await this.m_page.evaluate(() =>
                {
                    return window.location.hash;
                });


Decimal PrimitiveArray change datatype after try_unary

Describe the bug

To Reproduce
        let a = Decimal128Array::from(vec![1, 3, -44, 2, 4])
            .with_precision_and_scale(9, 6)
            .unwrap();

        let f: PrimitiveArray<Decimal128Type> = a.unary(|i| i.add(1));

datatype from (9, 6) -> (38, 10)
Expected behavior

Additional context

That is default datatype. In unary, it doesn't keep parameter of datatype e.g. precision/scale and timezone, I think.


[CS2103T-W16-3] TutorAid

TutorAid helps tech-savvy private tutors, who have busy schedules / many students, to keep track of each students' details. TutorAid can also help these tutors with the tedious task of tracking every student's study progress and any upcoming milestones.

Codecov Report

Merging #36 (9a4651c) into master (ba53b8c) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master      #36   +/-   ##
=========================================
  Coverage     72.15%   72.15%
  Complexity      399      399
=========================================
  Files            70       70
  Lines          1232     1232
  Branches        125      125
=========================================
  Hits            889      889
  Misses          311      311
  Partials         32       32

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ba53b8c...9a4651c. Read the comment docs.


🎨 Black-ened codebase & such

This pull request implements the black linter for Python inside the codebase. All Python files have been linted with the pull request, along with a GitHub Actions workflow to automate the linting process.  Slight changes have also been done to the README.md in order to show the linting status.

Thanks @hitblast 👍🏻


Stop command?

Hey,
As you can see, I'm changing the titles as I resolve issues I'm having.
First of all, great plug-in! thanks for making it.
Is there a way to tell it "Full opening takes x amount of seconds" and then have it work percentages according to that time you have provided?
It seems that to stop the blind, if I set it at 90%, it uses the opposite command to try to stop the blind.
So if it is open, and I set it at 90% it goes down to 90%, then it goes back up. Is there a way to code a stop command?

Hello @Cloudore, thanks for your issue!
I designed the plugin for my own use case which is a plain HTTP server which answers with a 200 directly after it sets the blinds in motion. So I only have "up" and "down" as commands, a variable which holds the current state and no percentages at all.
I also just saw #8 - I don't know why I missed that. Sorry @StryKaizer for the delay! Possibly it solves also your problem?

I'll try his pull request!
It's funny, I'm was just trying the noolite-blinds plug-in which seems to be what he is basing his on along with this one. but for some reason, percentages work as long as my phone is not connected to my own wifi network. If I'm on 3G and I ask it to go down to 50% it might, but if I'm connected to my wifi it just goes directly from "Closed" to "Open 50%" status. Opening and closing fully however works perfectly in both cases, need to do a little bit of tinkering this weekend to see if I can find the reason for that, but I would appreciate it if you could give be a hint since you're working on a very similar code @StryKaizer.
Thanks again for the plug-in!


src/grammar for the canonical grammar of the Rust language

Rendered

:+1: :heart_eyes: :100:

The semi-canonical step may be unnecessary. I don't expect there to be very many discrepancies between what we think the grammar is and what the grammar actually is, so we could just switch to the fully canonical stage directly. Even if a discrepancy is discovered after that point, all it would take would be an RFC to determine whether it should result in a change to the grammar or a change to the parser (obviously taking into account how much breakage would result from a change to the parser).

I'm hugely interested in this, many thanks!
One question that bothers me is how the consistency between libsyntax and src/grammar is to be enforced? I am afraid that if automatic comparison of parsers is not a part of the test suite, then the grammar will never become canonical (or the rust parser will never become conforming, if you prefer this half of the glass).
And if we speak of test suite, it would be ideal if it is implementation independent and included in something like ctrs.

We have vestigial grammar tests still in the makefiles, I think.

I share @matklad's concerns. It'd be awesome to have a canonical grammar, but maintaining consistency seems like a major challenge.
I'm not sure how we could actually go about testing it in a rigorous manner. Since the compiler's parser doesn't have to parse in the same way a generated parser (from the grammar) would, figuring out whether or not they are consistent with each other seems difficult.
That said, less-rigorous testing could be done with a combination of fuzzing and regression tests. The grammar could be used to generate syntactically-correct (if nonsensical) code which is run through the compiler's parser. If the sample fails to parse, we can look at it and decide if it's a "real" failure or not. If it is, we can fix the bug and use the sample (or a reduced version) as the regression test. The downside here is that it doesn't enforce the same parse. The compiler could parse a code fragment without error, but not actually be correct. I can't see this being a problem though.
With all that said, some discussion around testing the grammar and parser would be nice to see in the RFC proper.

For reference https://github.com/rust-lang/rust/blob/master/src/grammar/check.sh and https://github.com/rust-lang/rust/blob/master/src/grammar/verify.rs and other stuff in src/grammar is how we used to do the testing.

@steveklabnik I thought those are only lexer tests

They at least used to be referred to as "grammar tests", though maybe I cleared some of them out and they're not there anymore. We were like, using antlr and such...

@nagisa I am very positive on this idea. For some time now I have been wanting to have a grammar changes written up more rigorously. I think your plan is pretty coherent -- basically there's going to be a messy period where we have to make ad-hoc judgement calls on what is the 'canonical' grammar for a bit of time, but eventually we'll get to a clean state.
I do think it's important that we add automatic checking of the Rust grammar against the in-tree sources (at least). Ideally we'd maybe even do crater runs or something, but you gotta run before you can walk.
One question that I am wondering is how the in-tree grammar compares to e.g. the rust-grammar project in terms of completeness? (Also, is one project based on the other?) I know that @rprichard has been filing various issues about discrepancies between what rustc does and what rust-grammar does, so they must have some automated means of testing it.

cc @rprichard @bleibig @dgrunwald as people who seem likely to have an opinion on this RFC and related matters.

One question that I am wondering is how the in-tree grammar compares to e.g. the rust-grammar project in terms of completeness?

As far as I can see our in-tree LARL grammar is pretty much a copy of this project (sans the changes that have been made to the grammar). I suspect, without much digging, that it was imported from the rust-grammar project itself.

I’m aware it is usually possible to generate valid inputs from a given grammar (or its state machine), but there’s no existing attempts to do this as far as I know.
Answers to similar question are also pretty hand-wavy.

@nagisa

I’m aware it is usually possible to generate valid inputs from a given grammar (or its state machine), but there’s no existing attempts to do this as far as I know. While that sounds like a nice way to test the parser conformance, I’m not even sure how complex the implementation of such tester would be.

ISTM that the rust compiler, libs, and crates.io provide plenty of sample inputs :)

ISTM that the rust compiler, libs, and crates.io provide plenty of sample inputs :)

Different grammar productions can interact in surprising ways.
For example, the operator as allows invoking a lambda expression without parenthesizing the lambda expression:
fn main() {
    || println!("Hello World") as ()()
}

It's not obvious that this is even possible (and it probably shouldn't be), so I'd be surprised if you can find such code in the wild.

@nikomatsakis  sample inputs will allow to verify that the formal grammar accepts what rustc accepts. But even the trivial grammar can do this, namely, the grammar of all token sequences.
The hard problem is to make sure that the formal grammar does not accept invalid Rust. As I understand @Aatch suggestion was about this direction of verification.
And the second hard problem is to make sure that both the grammar and the rustc produce similar parse tree: you can easily mix up operator precedence, for a start.

@dgrunwald @matklad yes, ok, right, forgot about @Aatch's question. Nonetheless, I think that comparing the two against all the Rust code we can get our hands on (and yes, comparing the parse trees, which seems eminently doable) provides a fairly high degree of confidence that we are parsing what we want to, despite the theoretical disparities. Certainly generating inputs would let us go further (though we'd presumably be filtering these inputs against the rustc impl, which still means that there may be cases where both do the wrong thing in the same way).

I'd be interested in taking this a step further and experimenting with a grammar-generated frontend. I honestly assumed that was part of Niko's plan with LALRPOP.

@graydon

I'd be interested in taking this a step further and experimenting with a
grammar-generated frontend. I honestly assumed that was part of Niko's plan
with LALRPOP.

It is, of course. =)
On Thu, Nov 12, 2015 at 9:39 AM, Graydon Hoare<EMAIL_ADDRESS>wrote:

I'd be interested in taking this a step further and experimenting with a
grammar-generated frontend. I honestly assumed that was part of Niko's plan
with LALRPOP.
—
Reply to this email directly or view it on GitHub
https://github.com/rust-lang/rfcs/pull/1331#issuecomment-156119118.

I'd be interested in taking this a step further and experimenting with a grammar-generated frontend. I honestly assumed that was part of Niko's plan with LALRPOP.

To me it sounds like generating parser is a natural step to take after we have a proper (canonical and verified to be correct) grammar. We need some way to ensure a grammar matches current implementation (and/or becomes the canonical representation of the Rust’s grammar) before we migrate rustc to a parser generator of any sort, don’t we?
However, that sounds like something that would need to be discussed once we have a generator itself ready, right?

Why would we migrate rustc to a generated parser?
Good messages for syntax errors are so much easier in a hand-written recursive descend parser.
Also, LR parsers are especially problematic for use in IDEs: if you need a (partial) syntax tree for code completion, you'll have to puzzle it together from the partially matched productions remaining on the parser's stack. Or (equivalently) introduce a whole bunch of error productions, and you'll always forget some and have cases where nodes aren't integrated into the syntax tree and code completion suffers. This stuff will be much much easier with the existing hand-written parser (since it's LL-style, there's always an obvious choice for connecting the current node to the root of the tree when you encounter an error).

@dgrunwald Note that we did not necessarily say an LR parser. I was hoping
for LL(k) or maybe at worst GLL. But I agree that the benefits of an
automatic parser do not necessarily outweigh the downsides.
On Thu, Nov 12, 2015 at 12:30 PM, Daniel Grunwald<EMAIL_ADDRESS>wrote:

Why would we migrate rustc to a generated parser?
Good messages for syntax errors are so much easier in a hand-written
recursive descend parser.
Also, LR parsers are especially problematic for use in IDEs: if you need a
(partial) syntax tree for code completion, you'll have to puzzle it
together from the partially matched productions remaining on the parser's
stack. Or (equivalently) introduce a whole bunch of error productions, and
you'll always forget some and have cases where nodes aren't integrated into
the syntax tree and code completion suffers. This stuff will be much much
easier with the existing hand-written parser (since it's LL-style, there's
always an obvious choice for connecting the current node to the root of the
tree when you encounter an error).
—
Reply to this email directly or view it on GitHub
https://github.com/rust-lang/rfcs/pull/1331#issuecomment-156175868.

comparing the parse trees, which seems eminently doable

Careful there; you seem to be assuming that it will be possible to specify an unambiguous formal grammar that matches rustc's behavior (modulo rustc bugs that we can fix without breaking the world).
I'm not sure if this is possible. Some things are easy to specify in English (and in a hand-written parser), but hard in a formal grammar.
For example, take this small grammar, which looks like a tiny subset of rust's expression grammar (allowing integer literals, the "as" operator, and less than operator):
type ::= identifier | identifier "<" type ">"
expr ::= int_literal | expr "as" type | expr "<" expr

But this grammar already accepts some expressions that rustc does not: e.g. 1 as i32 < 2.
It's easy to say: "A '<' token after a type name is always considered to be part of the type." But doing this in the formal grammar is not so easy:
type ::= identifier | identifier "<" type ">"
type_LT_follow ::= identifier "<" type ">"
expr ::= int_literal
    | expr "as" type
    | expr_LT_follow "<" expr
expr_LT_follow ::= int_literal
    | expr "as" type_LT_follow
    | expr_LT_follow "<" expr_LT_follow

Note that we just duplicated the number of productions! A few more such disambiguation rules, and our grammar increases exponentially in size. It becomes unreadable and unmaintainable.
Keeping the disambiguation rule in plain English is much better. This is also the option taken in the C# language specification: the formal grammar in the appendix is just a rough approximation, and sections like "<IP_ADDRESS> Grammar ambiguities" explain how the parser should handle specific tricky cases.
But if the formal grammar is just an approximation, that makes automated consistency checks rather tricky...

@dgrunwald

Note that we just duplicated the number of productions! A few more such
disambiguation rules, and our grammar increases exponentially in size. It
becomes unreadable and unmaintainable.

We'll have to see how it goes. I'll note though that LALRPOP's macros
include an explicit feature for just this case, but I've not tried using it
"at scale" yet. This is the kind of thing I was thinking of when designing
it, though. (In particular, they let you specify flags that can filter
which productions are used, etc.) I seem to recall that JavaScript uses a
similar mechanism for cases like this in their spec, but I don't recall for
certain.
On Thu, Nov 12, 2015 at 7:37 PM, Daniel Grunwald<EMAIL_ADDRESS>wrote:

comparing the parse trees, which seems eminently doable
Careful there; you seem to be assuming that it will be possible to specify
an unambiguous formal grammar that matches rustc's behavior (modulo rustc
bugs that we can fix without breaking the world).
I'm not sure if this is possible. Some things are easy to specify in
English (and in a hand-written parser), but hard in a formal grammar.
For example, take this small grammar, which looks like a tiny subset of
rust's expression grammar (allowing integer literals, the "as" operator,
and less than operator):
type ::= identifier | identifier "<" type ">"
expr ::= int_literal | expr "as" type | expr "<" expr
But this grammar already accepts some expressions that rustc does not:
e.g. 1 as i32 < 2.
It's easy to say: "A '<' token after a type name is always considered to
be part of the type." But doing this in the formal grammar is not so easy:
type ::= identifier | identifier "<" type ">"
type_LT_follow ::= identifier "<" type ">"
expr ::= int_literal
| expr "as" type
| expr_LT_follow "<" expr
expr_LT_follow ::= int_literal
| expr "as" type_LT_follow
| expr_LT_follow "<" expr_LT_follow
Note that we just duplicated the number of productions! A few more such
disambiguation rules, and our grammar increases exponentially in size. It
becomes unreadable and unmaintainable.
Keeping the disambiguation rule in plain English is much better. This is
also the option taken in the C# language specification: the formal grammar
in the appendix is just a rough approximation, and sections like "<IP_ADDRESS>
Grammar ambiguities" explain how the parser should handle specific tricky
cases.
But if the formal grammar is just an approximation, that makes automated
consistency checks rather tricky...
—
Reply to this email directly or view it on GitHub
https://github.com/rust-lang/rfcs/pull/1331#issuecomment-156281190.

@dgrunwald

It's easy to say: "A '<' token after a type name is always considered to be part of the type." But doing this in the formal grammar is not so easy:

FWIW, the parser-lalr grammar does seem to handle this ambiguity correctly using precedence directives rather than duplicating the expression non-terminal.  I'm guessing a precedence directive is also the reason this parser-lalr bug isn't caught at compile-time.

This seems relevant.

(Nominating for final comment period discussion, since it seems like we've reached a fixed point here in this comment thread.)

Hear ye, hear ye! This RFC is now entering final comment period.

This is potentially very far off in future times, but would having a canonical grammar like this make it easier for people to develop alternate implementations?

@jarcane

This is potentially very far off in future times, but would having a canonical grammar like this make it easier for people to develop alternate implementations?

It would certainly help, though probably not by THAT much.

Huzzah! This RFC has been accepted by the language design subteam.