Add OpenAPI generation

This patch updates the openapi generator:
This patch fixes seg fault in openapi-gen/main.go and adds
a make rule to create the ./bin/openapi-gen binary as part
of the build process.
Generated go file gets placed under pkg/openapi/openapi_generated.go
@pwittrock @pmorie

If this is another call on a binary from upstream, I'd like to compile and call it directly.

I wouldn't recommend placing the openapi files in pkg/client/listers.
I'd recommend looking at https://github.com/openshift/origin/blob/master/hack/update-generated-openapi.sh for how OpenShift generates its openapi spec to include both Kubernetes and OpenShift api types.
This is OpenShift's main.go for it: https://github.com/openshift/origin/blob/master/tools/genopenapi/openapi.go

@ncdc thanks for the link to the openshift script. I have updated this PR to re-use the vendored openapi-gen with it, however when testing it, I realized that our vendored client-go is not structured as vendor/k8s.io/client-go/pkg/api... but rather as vendor/k8s.io/client-go/1.5/pkg/api... thus causing runtime errors for the openapi-gen script when trying to look for client-go deps. Do we do anything different in openshift when vendoring client-go?

@juanvallejo as client-go has advanced, its directory structure has changed. I don't know specifically what issue you're hitting, but you'll need to adjust things appropriately if you're getting import issues.

I was on a roll, so I whipped this out, https://github.com/MHBauer/service-catalog/commit/392a1ac812a20709edf8786271c8b555c22eefe1
It doesn't succeed. I get a panic:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x2b8 pc=0x4c6165]

goroutine 1 [running]:
panic(0x873e40, 0xc420014070)
	/usr/local/go/src/runtime/panic.go:500 +0x1a1
github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/gengo/parser.(*Builder).importBuildPackage(0xc42032c9b0, 0x7fff8a67ae00, 0x42, 0x20, 0x6, 0x1)
	/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/gengo/parser/parse.go:137 +0x205
github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/gengo/parser.(*Builder).addDir(0xc42032c9b0, 0x7fff8a67ae00, 0x42, 0x1, 0xb6ff60, 0xc425f58700)
	/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/gengo/parser/parse.go:245 +0x70
github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/gengo/parser.(*Builder).AddDirTo(0xc42032c9b0, 0x7fff8a67ae00, 0x42, 0xc4256ca7e8, 0x50, 0xc425f58780)
	/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/gengo/parser/parse.go:231 +0x11f
github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/gengo/generator.(*Context).AddDir(0xc4256ca7e0, 0x7fff8a67ae00, 0x42, 0x280, 0x2a6)
	/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/gengo/generator/generator.go:210 +0x4c
github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/kubernetes/cmd/libs/go2idl/openapi-gen/generators.Packages(0xc4256ca7e0, 0xc42049d300, 0x8f04b5, 0x3, 0xc4256ca7e0)
	/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/kubernetes/cmd/libs/go2idl/openapi-gen/generators/openapi.go:95 +0x26b
github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/gengo/args.(*GeneratorArgs).Execute(0xc42049d300, 0xc4202f9620, 0x8f04b5, 0x3, 0x9308f8, 0x40e479, 0xc420010118)
	/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/gengo/args/args.go:165 +0x170
main.main()
	/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/kubernetes/cmd/libs/go2idl/openapi-gen/main.go:43 +0x234

I figured it out, the package you are generating into has to 'exist' by having a go file in it already.
compare https://github.com/kubernetes-incubator/service-catalog/compare/master...MHBauer:vendored-openapi-gen?expand=1
I don't know if that looks correct, but I do see our service catalog types.
@pwittrock @juanvallejo @pmorie @duglin

@MHBauer

I figured it out, the package you are generating into has to 'exist' by having a go file in it already.

Yeah, I was running into this same error when trying this out earlier.
Maybe either have placeholder go files in the root of all packages that the script selects, or filter the list of packages to only those that have go files at the top level?
List of selected packages:
$ grep --color=never -rl '+k8s:openapi-gen=' vendor/k8s.io/kubernetes | \
    xargs -n1 dirname | \
    sed "s,^vendor/,," | \
    sort -u | xsel --input --clipboard

k8s.io/kubernetes
k8s.io/kubernetes/cmd/libs/go2idl/client-gen/test_apis/testgroup/v1
k8s.io/kubernetes/cmd/libs/go2idl/openapi-gen
k8s.io/kubernetes/federation/apis/federation/v1beta1
k8s.io/kubernetes/pkg/api/resource
k8s.io/kubernetes/pkg/apis/abac
k8s.io/kubernetes/pkg/apis/abac/v0
k8s.io/kubernetes/pkg/apis/abac/v1beta1
k8s.io/kubernetes/pkg/apis/apps
k8s.io/kubernetes/pkg/apis/apps/v1beta1
k8s.io/kubernetes/pkg/apis/authentication
k8s.io/kubernetes/pkg/apis/authentication/v1beta1
k8s.io/kubernetes/pkg/apis/authorization
k8s.io/kubernetes/pkg/apis/authorization/v1beta1
k8s.io/kubernetes/pkg/apis/autoscaling
k8s.io/kubernetes/pkg/apis/autoscaling/v1
k8s.io/kubernetes/pkg/apis/batch
k8s.io/kubernetes/pkg/apis/batch/v1
k8s.io/kubernetes/pkg/apis/batch/v2alpha1
k8s.io/kubernetes/pkg/apis/certificates
k8s.io/kubernetes/pkg/apis/certificates/v1alpha1
k8s.io/kubernetes/pkg/apis/componentconfig
k8s.io/kubernetes/pkg/apis/componentconfig/v1alpha1
k8s.io/kubernetes/pkg/apis/extensions
k8s.io/kubernetes/pkg/apis/extensions/v1beta1
k8s.io/kubernetes/pkg/apis/imagepolicy
k8s.io/kubernetes/pkg/apis/imagepolicy/v1alpha1
k8s.io/kubernetes/pkg/apis/meta/v1
k8s.io/kubernetes/pkg/apis/policy
k8s.io/kubernetes/pkg/apis/policy/v1beta1
k8s.io/kubernetes/pkg/apis/rbac
k8s.io/kubernetes/pkg/apis/rbac/v1alpha1
k8s.io/kubernetes/pkg/apis/storage/v1beta1
k8s.io/kubernetes/pkg/api/v1
k8s.io/kubernetes/pkg/genericapiserver/openapi
k8s.io/kubernetes/pkg/runtime
k8s.io/kubernetes/pkg/util/intstr
k8s.io/kubernetes/pkg/version
k8s.io/kubernetes/pkg/watch/versioned
k8s.io/kubernetes/staging/src/k8s.io/client-go/pkg/api/resource
k8s.io/kubernetes/staging/src/k8s.io/client-go/pkg/apis/apps
k8s.io/kubernetes/staging/src/k8s.io/client-go/pkg/apis/apps/v1beta1
k8s.io/kubernetes/staging/src/k8s.io/client-go/pkg/apis/authentication
k8s.io/kubernetes/staging/src/k8s.io/client-go/pkg/apis/authentication/v1beta1
k8s.io/kubernetes/staging/src/k8s.io/client-go/pkg/apis/authorization
k8s.io/kubernetes/staging/src/k8s.io/client-go/pkg/apis/authorization/v1beta1
k8s.io/kubernetes/staging/src/k8s.io/client-go/pkg/apis/autoscaling
k8s.io/kubernetes/staging/src/k8s.io/client-go/pkg/apis/autoscaling/v1
k8s.io/kubernetes/staging/src/k8s.io/client-go/pkg/apis/batch
k8s.io/kubernetes/staging/src/k8s.io/client-go/pkg/apis/batch/v1
k8s.io/kubernetes/staging/src/k8s.io/client-go/pkg/apis/batch/v2alpha1
k8s.io/kubernetes/staging/src/k8s.io/client-go/pkg/apis/certificates
k8s.io/kubernetes/staging/src/k8s.io/client-go/pkg/apis/certificates/v1alpha1
k8s.io/kubernetes/staging/src/k8s.io/client-go/pkg/apis/componentconfig
k8s.io/kubernetes/staging/src/k8s.io/client-go/pkg/apis/componentconfig/v1alpha1
k8s.io/kubernetes/staging/src/k8s.io/client-go/pkg/apis/extensions
k8s.io/kubernetes/staging/src/k8s.io/client-go/pkg/apis/extensions/v1beta1
k8s.io/kubernetes/staging/src/k8s.io/client-go/pkg/apis/imagepolicy
k8s.io/kubernetes/staging/src/k8s.io/client-go/pkg/apis/imagepolicy/v1alpha1
k8s.io/kubernetes/staging/src/k8s.io/client-go/pkg/apis/meta/v1
k8s.io/kubernetes/staging/src/k8s.io/client-go/pkg/apis/policy
k8s.io/kubernetes/staging/src/k8s.io/client-go/pkg/apis/policy/v1beta1
k8s.io/kubernetes/staging/src/k8s.io/client-go/pkg/apis/rbac
k8s.io/kubernetes/staging/src/k8s.io/client-go/pkg/apis/rbac/v1alpha1
k8s.io/kubernetes/staging/src/k8s.io/client-go/pkg/apis/storage/v1beta1
k8s.io/kubernetes/staging/src/k8s.io/client-go/pkg/api/v1
k8s.io/kubernetes/staging/src/k8s.io/client-go/pkg/federation/apis/federation/v1beta1
k8s.io/kubernetes/staging/src/k8s.io/client-go/pkg/runtime
k8s.io/kubernetes/staging/src/k8s.io/client-go/pkg/util/intstr
k8s.io/kubernetes/staging/src/k8s.io/client-go/pkg/version
k8s.io/kubernetes/staging/src/k8s.io/client-go/pkg/watch/versioned

Hm, after rebasing and re-running ./contrib/hack/update-generated-openapi.sh, I'm now getting the error:
F0209 01:14:17.389032    6159 main.go:44] Error: Failed making a parser: unable to add directory "k8s.io/kubernetes": unable to *find* "k8s.io/kubernetes": cannot find package "k8s.io/kubernetes" in any of:
        /home/jvallejo/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/kubernetes (vendor tree)
        /usr/local/go/src/k8s.io/kubernetes (from $GOROOT)
        /home/jvallejo/go/src/k8s.io/kubernetes (from $GOPATH)

@juanvallejo if the servicecatalog api types depend on kube api types (I assume they will), you will need to include the kube api types in --input-dirs

Probably need to run that script in our build container. @duglin @krancour thoughts?

@ncdc Why do we need any of the API information other than our own? We're not serving the core k8s API.

@ncdc thanks for the feedback, that was the part I was having some trouble with: some of the paths generated here: https://github.com/kubernetes-incubator/service-catalog/pull/349#issuecomment-278556609 did not have go files at the top-level, which caused https://github.com/kubernetes-incubator/service-catalog/pull/349#issuecomment-278534791 to happen

@MHBauer because the spec you generate is going to need to reference ObjectMeta, ListMeta, and other things imported from kube.

That should be exactly only "k8s.io/kubernetes/pkg/api/v1" & "k8s.io/kubernetes/pkg/apis/meta/v1" in our case?

Yeah, whatever you're actually importing and using

@ncdc updated, https://github.com/kubernetes-incubator/service-catalog/compare/master...MHBauer:vendored-openapi-gen
I think it pulled in a lot of other junk, but it has typemeta and objectmeta now.
Do we need the openshift script still?

Not if it's working for you as-is, no

@ncdc @MHBauer Thanks for the feedback and help. I have updated the Makefile to include both k8s paths in https://github.com/kubernetes-incubator/service-catalog/pull/349#issuecomment-278681835
pkg/openapi/openapi_generated.go now matches the one in https://github.com/kubernetes-incubator/service-catalog/compare/master...MHBauer:vendored-openapi-gen
Are we tracking the generated file as well?

unfortunately, yes. Usually done as two commits, one for the code that does the generation, one for the generated code.

Is there anything we can do to know it's working?
As far as you are aware the output looks correct?

@MHBauer

Is there anything we can do to know it's working?

maybe a test discovery client that uses https://localhost:6443/apis/servicecatalog.k8s.io/v1alpha1/... and is used to determine which types are supposed to have generated documentation by openapi-gen?

LGTM assuming the CI failures are fixed; thanks for doing this

LGTM, merging


Authentication failed when using Chinese Azure with modelbuilder

suspect 'setup workspace' refers to azure global endpoints but not Chinese endpoints.

@haoxin Sorry you ran into this problem. Are you have the same problem as this issue - https://github.com/dotnet/machinelearning-modelbuilder/issues/2118?

@haoxin Sorry you ran into this problem. Are you have the same problem as this issue - #2118?

Yes.

@haoxin What region are you in? According to docs looks like CH-East-2 is supported and CH-North-3 is not supported. Are you using CH-East-2?

Yes, we used CH-east-2

@haoxin What Azure region are you using? According to docs looks like CH-East-2 is supported and CH-North-3 is not supported. Are you using CH-East-2?

Yes, We used CH-east-2

@v-Hailishi Can you try to repro this with a CH-east-2 region?

@beccamc The "ML.NET Model Builder Demo" subscription cannot support the CH-east-2 region.
Could you help to provide a special subscription which can support the CH-east-2 region?

Hello Haili,
May I contact with you offline and provide you an Azure China ML workspace to do a test? thanks.

@beccamc  I have reproduced this issue after using the account provided by haoxin.
The following are the repro steps:

Add the China Azure cloud: https://login.chinacloudapi.cn

On the "Set up workspace" dialog, select "Azure in China".

After you enter the account, you will see the 401 error.

@v-Hailishi can you please test / validate that this has been fixed in the latest build. Thanks

@luisquintanilla The bug is still repro on the latest main branch: <IP_ADDRESS>1103

@haoxin We just released a new version that upgrades some calls to use the Azure SDK. Can you re-try on your side and see if you still see the same error message?

@v-Hailishi can you please validate using the latest version. Thanks.

@luisquintanilla On the latest main branch <IP_ADDRESS>6201 still can see 401 error:

Hmm. Thanks for checking.
cc @beccamc

@zewditu to confirm whether 401 comes back from SDK call. If so, file an issue with SDK.

The workspace creation part  is note done by the new Azure SDK we are still stick with the old SDK

What is the plan to update to the new Azure SDK for this?

The reason we still use the old SDK is the new SDK does not let us create dependent resources of a workspace like a key vault, application insight and storage account, and container registry automatically.

@v-Hailishi  can you validate this on this  private build
https://privategallery.blob.core.windows.net/gallery/refs/heads/u/zewd/Fixenv/atom.xml

@beccamc This bug is also repro on the private build
https://privategallery.blob.core.windows.net/gallery/refs/heads/u/zewd/Fixenv/atom.xml

@v-Hailishi  thanks for checking

@v-Hailishi, I made some changes, could you try this build ?https://privategallery.blob.core.windows.net/gallery/refs/heads/u/zewd/Fixenv/atom.xml

@zewditu This bug has been fixed on the private build <IP_ADDRESS>7202
https://privategallery.blob.core.windows.net/gallery/refs/heads/u/zewd/Fixenv/atom.xml

@v-Hailishi thanks for testing !

This will ship with the Fall release.

This fix has shipped with release 16.14.0


Fix typo - Dockerfile now uses COPY rather than ADD

I was reviewing the Spring Boot guide and noticed that the description of the Dockerfile references the previously used ADD command. Here's a suggested rewording so it uses the COPY command.

@barecode Please sign the Contributor License Agreement!
Click here to manually synchronize the status of this Pull Request.
See the FAQ for frequently asked questions.

@barecode Thank you for signing the Contributor License Agreement!


Update Yggtorrent domain

fixes https://github.com/elgatito/script.elementum.burst/issues/400

closed in favor of #411


BlWrapAroundChildrenLayout incorrectly computes its boundary when it has a BLLineElement as a child

BlWrapAroundChildrenLayout is supposed to adapt its boundary to its children. This works correctly when its children are nodes, but not when they are line elements. This demo script illustrates the problem:
eg := BlWrapAroundChildrenLayoutExamples new.

aContainer := eg container.
aContainer padding: (BlInsets all: 0).

aContainer addChild: (aChildA := eg node relocate: 150@50).
aContainer addChild: (aChildB := eg node relocate: 50@150).
aContainer addChild: (aChildC := eg node relocate: 250@250).

lineElement := BlLineElement new.
lineElement
	zIndex: -1;
	border: (BlBorder paint: Color blue width: 1);
	fromAnchor: (BlElementCenterAnchor element: aChildA);
	toAnchor: (BlElementCenterAnchor element: aChildB).
aContainer addChild: lineElement.

aContainer forceLayout.

The boundary should match the red square, not the grey background.

resolved :)

resolved :)


fix(es2016): update e2e test to babel6 settings

@ahmedshuhel please review

@EisenbergEffect while at it, have a look please


Auth::logout() when called for 2nd time does not exit gracefully with error code

Subject of the issue
When you call the logout api ( post request for logout with bearer token which does an Auth::logout() ) for the 2nd time, the API should ideally respond with a 400 or 404 instead of the below exception with status code 405 Method not allowed.
In production:
Oops! An Error Occurred
The server returned a "405 Method Not Allowed".
Something is broken. Please let us know what you were doing when this error occurred. We will fix it as soon as possible. Sorry for any inconvenience caused.

OR when debug is enabled:
Symfony\Component\HttpKernel\Exception\MethodNotAllowedHttpException: The GET method is not supported for this route. Supported methods: POST. in file /home/user/projects/octane-starter/octane-api/vendor/laravel/framework/src/Illuminate/Routing/AbstractRouteCollection.php on line 118

#0 /home/user/projects/octane-starter/octane-api/vendor/laravel/framework/src/Illuminate/Routing/AbstractRouteCollection.php(104): Illuminate\Routing\AbstractRouteCollection->methodNotAllowed()
#1 /home/user/projects/octane-starter/octane-api/vendor/laravel/framework/src/Illuminate/Routing/AbstractRouteCollection.php(41): Illuminate\Routing\AbstractRouteCollection->getRouteForMethods()
#2 /home/user/projects/octane-starter/octane-api/vendor/laravel/framework/src/Illuminate/Routing/RouteCollection.php(162): Illuminate\Routing\AbstractRouteCollection->handleMatchedRoute()
#3 /home/user/projects/octane-starter/octane-api/vendor/laravel/framework/src/Illuminate/Routing/Router.php(753): Illuminate\Routing\RouteCollection->match()
#4 /home/user/projects/octane-starter/octane-api/vendor/laravel/framework/src/Illuminate/Routing/Router.php(740): Illuminate\Routing\Router->findRoute()
#5 /home/user/projects/octane-starter/octane-api/vendor/laravel/framework/src/Illuminate/Routing/Router.php(729): Illuminate\Routing\Router->dispatchToRoute()
#6 /home/user/projects/octane-starter/octane-api/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(190): Illuminate\Routing\Router->dispatch()
#7 /home/user/projects/octane-starter/octane-api/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(141): Illuminate\Foundation\Http\Kernel->Illuminate\Foundation\Http\{closure}()
#8 /home/user/projects/octane-starter/octane-api/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php(21): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()
#9 /home/user/projects/octane-starter/octane-api/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/ConvertEmptyStringsToNull.php(31): Illuminate\Foundation\Http\Middleware\TransformsRequest->handle()
#10 /home/user/projects/octane-starter/octane-api/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(180): Illuminate\Foundation\Http\Middleware\ConvertEmptyStringsToNull->handle()
#11 /home/user/projects/octane-starter/octane-api/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php(21): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()
#12 /home/user/projects/octane-starter/octane-api/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TrimStrings.php(40): Illuminate\Foundation\Http\Middleware\TransformsRequest->handle()
#13 /home/user/projects/octane-starter/octane-api/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(180): Illuminate\Foundation\Http\Middleware\TrimStrings->handle()
#14 /home/user/projects/octane-starter/octane-api/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/ValidatePostSize.php(27): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()
#15 /home/user/projects/octane-starter/octane-api/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(180): Illuminate\Foundation\Http\Middleware\ValidatePostSize->handle()
#16 /home/user/projects/octane-starter/octane-api/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/PreventRequestsDuringMaintenance.php(86): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()
#17 /home/user/projects/octane-starter/octane-api/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(180): Illuminate\Foundation\Http\Middleware\PreventRequestsDuringMaintenance->handle()
#18 /home/user/projects/octane-starter/octane-api/vendor/laravel/framework/src/Illuminate/Http/Middleware/HandleCors.php(62): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()
#19 /home/user/projects/octane-starter/octane-api/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(180): Illuminate\Http\Middleware\HandleCors->handle()
#20 /home/user/projects/octane-starter/octane-api/vendor/laravel/framework/src/Illuminate/Http/Middleware/TrustProxies.php(39): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()
#21 /home/user/projects/octane-starter/octane-api/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(180): Illuminate\Http\Middleware\TrustProxies->handle()
#22 /home/user/projects/octane-starter/octane-api/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(116): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()
#23 /home/user/projects/octane-starter/octane-api/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(165): Illuminate\Pipeline\Pipeline->then()
#24 /home/user/projects/octane-starter/octane-api/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(134): Illuminate\Foundation\Http\Kernel->sendRequestThroughRouter()
#25 /home/user/projects/octane-starter/octane-api/vendor/laravel/octane/src/ApplicationGateway.php(37): Illuminate\Foundation\Http\Kernel->handle()
#26 /home/user/projects/octane-starter/octane-api/vendor/laravel/octane/src/Worker.php(92): Laravel\Octane\ApplicationGateway->handle()
#27 /home/user/projects/octane-starter/octane-api/vendor/laravel/octane/bin/swoole-server(118): Laravel\Octane\Worker->handle()
#28 [internal function]: {closure}()
#29 /home/user/projects/octane-starter/octane-api/vendor/laravel/octane/bin/swoole-server(164): Swoole\Server->start()
#30 {main}

Your environment:

Q
A

Bug?
yes

New Feature?
no

Framework
Laravel (using laravel octane)

Framework version
9.31

Package version
2.0.0

PHP version
8.1.2-1ubuntu2.9

Steps to reproduce
Generate a bearer token for the user / login.  Trigger a POST request for logout with token. The logout is successful.  Now, trigger the post request for logout with token again. You get the above error.
Expected behaviour
A more graceful exit and an error message instead of an error from somewhere within the framework / symfony.
Actual behaviour
As described above

Using your custom routes?
As per I see the error is not related with library and it's about the way you handling your custom routes.

It works for me. When I click the logout route a second time, I get the status code 401.
405 means that you are using the wrong http method. Your debug log mentions that you tried to call it via get instead of post.
You probably made a mistake there.

It works for me. When I click the logout route a second time, I get the status code 401.
405 means that you are using the wrong http method. Your debug log mentions that you tried to call it via get instead of post. You probably made a mistake there.

Can you please share your logout handler?
This is what I have - very basic, as per the documentation:
In the api.php file,
Route::controller(AuthController::class)->group(function () {
    Route::post('login', 'login')->name('login');
    Route::post('register', 'register')->name('register');
    Route::post('logout', 'logout')->name('logout');
    Route::post('refresh', 'refresh')->name('refresh');

In the Auth Controller,
    public function __construct()
    {
        $this->middleware('auth:api', ['except' => ['login','register']]);
    }

    public function logout()
    {
        Auth::logout();
        return $this->sendResponse(null, 'User logout successful');
    }

Using your custom routes?
As per I see the error is not related with library and it's about the way you handling your custom routes.

I am making a POST request to the logout route and the handler is pretty much from the example provided. I have added the auth routes & the controller code in the other comment,

Check your middleware? Maybe the middleware trying to redirect you to a route.
And also, can someone tell me how Auth::logout() work in JWT since the JWT is save in client side?

If you're not using custom routes the logout route by default uses the HTTP POST verbose method.
If you want to use GET try something like this:
Route::get("/logout", fn() => Auth::logout())->middleware(YourMiddlewareClass/DefaultMiddlewareClass::class);


Add test_masks back into CI

To not forget about these regression tests
https://github.com/nipreps/niworkflows/commit/f2442f22bc43abf671819e8df295c5b67a4ced46

... I did forget about these regression tests.


Problems loading GTF file

Hello,
I am using the following code to load a gtf file but nothing gets loaded. Could you please look into it?
track_list = [
            {
                "name": "HA",
                "url": "HA.gtf",
                "format": "gff",
                "type": "annotation",
                "indexed": False
            }
        ]

b = igv.Browser(
    {"reference": {
        "id": "ref",
        "fastaURL": "genome.fasta",
        "indexed": False,
        "tracks": track_list
    }}
 )

b.show()

But once I load it manually, I can see the features

Thank you.

I figured the error. The gtf file was empty.

Also, if it's a gtf file the format should be "gtf",  not "gff".
On Sun, Jan 16, 2022 at 11:46 AM Sagnik Banerjee @.***>
wrote:

Closed #48 https://github.com/igvteam/igv-jupyter/issues/48.
—
Reply to this email directly, view it on GitHub
https://github.com/igvteam/igv-jupyter/issues/48#event-5900725308, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAHD2HDO7OKAAYSXROKI2DDUWMOAVANCNFSM5MDDYZZQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>


[BUG][NOWEB] - Is there any way to turn off link preview?

Describe the bug
I am using the sendText API endpoint to send a message that contains a URL, and I want to disable the link preview for the URL.
Is there any way to turn off link preview?
Version
Get the WAHA version by calling GET /api/version
{
  "version": "2024.10.7",
  "engine": "NOWEB",
  "tier": "PLUS"
}

Is there any way to turn off link preview?

Not now, but we'll add it (no ETA for that tho, will check with https://github.com/devlikeapro/waha/issues/596)

@devlikepro I request you to please add feature to disable link preview, all Amazon links are posting with captcha image which looks very bad to our WhatsApp channel.

will work on that :+1:

Added linkPreview in 2024.11.3
{
  "session": "default",
  "chatId"<EMAIL_ADDRESS>  "text": "https://amzn.to/3YvPkOh",
  "linkPreview": false
}

https://waha.devlike.pro/docs/how-to/send-messages/#send-text


Error from long-running event handler

(Disclaimer I'm not sure if this error is coming from event store client, from event store itself, or something completely different that has not yet been identified.)
We have an event handler that throws the following error when it runs for a sufficiently long time:
Error: write after end
    at writeAfterEnd (_stream_writable.js:166:12)
    at Socket.Writable.write (_stream_writable.js:217:5)
    at Socket.write (net.js:650:40)
    at Connection.sendMessage (/home/app/tableservice/node_modules/event-store-client/lib/connection.js:533:21)
    at receiveMessage (/home/app/tableservice/node_modules/event-store-client/lib/connection.js:160:22)
    at Socket.onData (/home/app/tableservice/node_modules/event-store-client/lib/connection.js:81:21)
    at emitOne (events.js:96:13)
    at Socket.emit (events.js:188:7)
    at readableAddChunk (_stream_readable.js:172:18)
    at Socket.Readable.push (_stream_readable.js:130:10)
    at TCP.onread (net.js:542:20)
/home/app/tableservice/tableEventStore/index.js:404
            throw err;
            ^

Error: write after end
    at writeAfterEnd (_stream_writable.js:166:12)
    at Socket.Writable.write (_stream_writable.js:217:5)
    at Socket.write (net.js:650:40)
    at Connection.sendMessage (/home/app/tableservice/node_modules/event-store-client/lib/connection.js:533:21)
    at receiveMessage (/home/app/tableservice/node_modules/event-store-client/lib/connection.js:160:22)
    at Socket.onData (/home/app/tableservice/node_modules/event-store-client/lib/connection.js:81:21)
    at emitOne (events.js:96:13)
    at Socket.emit (events.js:188:7)
    at readableAddChunk (_stream_readable.js:172:18)
    at Socket.Readable.push (_stream_readable.js:130:10)
    at TCP.onread (net.js:542:20)

If I restart the service running this event handler, it picks up where it died just fine and continues processing events.
Something that might be relevant - the event handler that appears to be the source of this error is subscribed to one stream. When it picks up a certain class of event, it opens up a different stream to read detailed events relating to the subscribed stream.

My suspicion is that Event Store is closing the connection due to a timeout. If you need to maintain a long-running connection to Event Store, it's necessary to send data occasionally for the server to consider the connection to be active. The easiest way to do this is to call the ping method on a regular basis. See #24 for another issue describing this scenario.

Is there an idiomatic way of checking the current state of the connection, or would I need to track that myself in the onConnect/onError/onClose callbacks?

You would need to track that yourself from those events.

As near as we've been able to determine, this problem stems from the automated heartbeat messages that Event Store sends (adding the sendPing helped but the connection still eventually died). In event-store-client/lib/connection.js, method receiveMessage:
        // Handle the message
        if (command == Commands.HeartbeatRequest) {
            // Automatically respond to heartbeat requests
            instance.sendMessage(correlationId, Commands.HeartbeatResponse);

        }

For whatever reason, it seems that sometimes the connection can be closed between receiving a heartbeat request and sending the response. Is this something that we should be accounting for in our application?

And for any future visitors, this is roughly the ping logic we're using:
    var connected = false;

    connection = new client.Connection({
        // settings
        onConnect: function () {
            connected = true;
            setInterval(() => {
                if(connection && connected) {
                    connection.sendPing(
                        () => {} // empty callback to avoid "unknown corrlation ID"
                    );
                }
            }, 10 * 1000);
        // connection logic

        },
        onError: function (err) {
            connected = false;
            // whatever error-handling logic
            throw err;
            });
        },
        onClose: (err) => {
            connected = false;
        }
    });

Ok, if you're suspecting the heartbeat responses are a problem then we'll need some more information to diagnose it.
I would recommend looking at the Event Store logs to see what it has to say about the connection (that will tell you whether it has closed the connection, or it thinks everything is okay). The other thing that would help would be a packet capture from Wireshark, so that we can see if it's a particular sequence of events that is causing the problem, or if there's something wrong with the heartbeat response packets themselves.

Here's the relevant bit from /var/log/eventstore/:
[PID:09313:025 2018.03.27 15:28:05.991 INFO  TcpConnectionManager] Connection 'external-normal' [<IP_ADDRESS>:54834, {e4fc4784-93a0-4111-b8bb-d037ce0d8708}] closed: Success.
[PID:09313:031 2018.03.27 15:28:05.992 DEBUG PersistentSubscripti] Lost connection from <IP_ADDRESS>:54834
[PID:09313:032 2018.03.27 15:28:05.998 INFO  TcpConnection       ] ES TcpConnection closed [15:28:05.998: N<IP_ADDRESS>:54836, L<IP_ADDRESS>:1113, {a1e065d1-662f-429a-8a12-80ef35b81915}]:Received bytes: 106, Sent bytes: 16141
[PID:09313:032 2018.03.27 15:28:05.999 INFO  TcpConnection       ] ES TcpConnection closed [15:28:05.999: N<IP_ADDRESS>:54836, L<IP_ADDRESS>:1113, {a1e065d1-662f-429a-8a12-80ef35b81915}]:Send calls: 1, callbacks: 1
[PID:09313:032 2018.03.27 15:28:06.000 INFO  TcpConnection       ] ES TcpConnection closed [15:28:06.000: N<IP_ADDRESS>:54836, L<IP_ADDRESS>:1113, {a1e065d1-662f-429a-8a12-80ef35b81915}]:Receive calls: 2, callbacks: 2
[PID:09313:032 2018.03.27 15:28:06.000 INFO  TcpConnection       ] ES TcpConnection closed [15:28:06.000: N<IP_ADDRESS>:54836, L<IP_ADDRESS>:1113, {a1e065d1-662f-429a-8a12-80ef35b81915}]:Close reason: [Success] Socket closed
[PID:09313:031 2018.03.27 15:28:06.001 INFO  TcpService          ] External TCP connection accepted: [Normal, <IP_ADDRESS>:54838, L<IP_ADDRESS>:1113, {d2c0aaee-e55a-49b5-883d-e517f673618a}].
[PID:09313:032 2018.03.27 15:28:06.003 INFO  TcpConnectionManager] Connection 'external-normal' [<IP_ADDRESS>:54836, {a1e065d1-662f-429a-8a12-80ef35b81915}] closed: Success.
[PID:09313:021 2018.03.27 15:28:06.004 DEBUG PersistentSubscripti] Lost connection from <IP_ADDRESS>:54836
[PID:09313:019 2018.03.27 15:28:09.891 TRACE TcpConnectionManager] Closing connection 'external-normal' [<IP_ADDRESS>:54504, L<IP_ADDRESS>:1113, {0b40a3c5-8a8e-448f-b9aa-932d2b7ac315}] cleanly. Reason: HEARTBEAT TIMEOUT at msgNum 7
[PID:09313:019 2018.03.27 15:28:09.892 INFO  TcpConnection       ] ES TcpConnection closed [15:28:09.892: N<IP_ADDRESS>:54504, L<IP_ADDRESS>:1113, {0b40a3c5-8a8e-448f-b9aa-932d2b7ac315}]:Received bytes: 382, Sent bytes: 759
[PID:09313:019 2018.03.27 15:28:09.893 INFO  TcpConnection       ] ES TcpConnection closed [15:28:09.893: N<IP_ADDRESS>:54504, L<IP_ADDRESS>:1113, {0b40a3c5-8a8e-448f-b9aa-932d2b7ac315}]:Send calls: 8, callbacks: 8
[PID:09313:019 2018.03.27 15:28:09.893 INFO  TcpConnection       ] ES TcpConnection closed [15:28:09.893: N<IP_ADDRESS>:54504, L<IP_ADDRESS>:1113, {0b40a3c5-8a8e-448f-b9aa-932d2b7ac315}]:Receive calls: 8, callbacks: 7
[PID:09313:019 2018.03.27 15:28:09.894 INFO  TcpConnection       ] ES TcpConnection closed [15:28:09.894: N<IP_ADDRESS>:54504, L<IP_ADDRESS>:1113, {0b40a3c5-8a8e-448f-b9aa-932d2b7ac315}]:Close reason: [Success] HEARTBEAT TIMEOUT at msgNum 7
[PID:09313:019 2018.03.27 15:28:09.895 INFO  TcpConnectionManager] Connection 'external-normal' [<IP_ADDRESS>:54504, {0b40a3c5-8a8e-448f-b9aa-932d2b7ac315}] closed: Success.
[PID:09313:017 2018.03.27 15:28:09.896 DEBUG PersistentSubscripti] Lost connection from <IP_ADDRESS>:54504
[PID:09313:021 2018.03.27 15:28:10.242 INFO  TcpConnection       ] ES TcpConnection closed [15:28:10.242: N<IP_ADDRESS>:54838, L<IP_ADDRESS>:1113, {d2c0aaee-e55a-49b5-883d-e517f673618a}]:Received bytes: 106, Sent bytes: 16230
[PID:09313:021 2018.03.27 15:28:10.243 INFO  TcpConnection       ] ES TcpConnection closed [15:28:10.243: N<IP_ADDRESS>:54838, L<IP_ADDRESS>:1113, {d2c0aaee-e55a-49b5-883d-e517f673618a}]:Send calls: 2, callbacks: 2
[PID:09313:021 2018.03.27 15:28:10.244 INFO  TcpConnection       ] ES TcpConnection closed [15:28:10.244: N<IP_ADDRESS>:54838, L<IP_ADDRESS>:1113, {d2c0aaee-e55a-49b5-883d-e517f673618a}]:Receive calls: 2, callbacks: 2
[PID:09313:021 2018.03.27 15:28:10.244 INFO  TcpConnection       ] ES TcpConnection closed [15:28:10.244: N<IP_ADDRESS>:54838, L<IP_ADDRESS>:1113, {d2c0aaee-e55a-49b5-883d-e517f673618a}]:Close reason: [Success] Socket closed
[PID:09313:021 2018.03.27 15:28:10.245 INFO  TcpConnectionManager] Connection 'external-normal' [<IP_ADDRESS>:54838, {d2c0aaee-e55a-49b5-883d-e517f673618a}] closed: Success.
[PID:09313:026 2018.03.27 15:28:10.246 DEBUG PersistentSubscripti] Lost connection from <IP_ADDRESS>:54838
[PID:09313:027 2018.03.27 15:36:46.456 TRACE InMemoryBus         ] SLOW BUS MSG [manager input bus]: GetStatistics - 54ms. Handler: ProjectionManager.
[PID:09313:027 2018.03.27 15:36:46.457 TRACE QueuedHandlerAutoRes] SLOW QUEUE MSG [Projections Master]: GetStatistics - 54ms. Q: 0/0.
[PID:09313:027 2018.03.27 15:48:10.475 TRACE InMemoryBus         ] SLOW BUS MSG [manager input bus]: GetStatistics - 75ms. Handler: ProjectionManager.
[PID:09313:027 2018.03.27 15:48:10.476 TRACE QueuedHandlerAutoRes] SLOW QUEUE MSG [Projections Master]: GetStatistics - 76ms. Q: 0/0.

The exact timestamp of the error from the container was 2018-03-27T15:28:10.225626957Z. <IP_ADDRESS> is the IP of the consuming service. Still working on getting a packet capture.

 :/ Is there an easy way to extract the heartbeat packets?

The heartbeat packets by themselves probably aren't that useful, it's more useful to see the full stream of packets, in case there's something wrong with the order in which they're being sent.
Three possible approaches from here:

Share the capture with me privately via email (my first name dot my last name at gmail dot com) so that I can investigate.
Create a copy of your application and simplify it down to the code that deals with Event Store, and make sure it still reproduces the issue. Get it to send dummy data rather than proprietary data. Share the code to that application, so that I can reproduce the issue.
Dig through the packet capture yourself in Wireshark to see if you can see any odd behaviour.

Okay, I managed to randomize/obfuscate the sensitive data with relative ease. Let me know if this is what you need.
capture.zip

Ok, there's definitely something wrong with the heartbeat responses. Here's a list of packets from your capture showing the server requesting a heartbeat, but the client carries on regardless without sending a heartbeat response:

So the interesting thing to note there is that it does respond to the heartbeat request in packet 81343, but then fails to respond later to the second request in 81567. That would suggest that the receive logic is not handling incoming messages after the ReadStreamEventsForwardCompleted is received.
You could try enabling the debug flag on the connection to get log output from the client, to see if it recognizes the second HeartbeatRequest.


fixed build for cygwin

I tried multiple methods to fix the build for cygwin
_WIN32 can be defined when using cygwin, this however causes other errors, because some headers are not present on cygwin and other problems.
I find this to be the cleanest solution, maybe some comments should be added for explanation?

@Julianiolo I'm merging this patch but I sincerely think this is not the best solution to support Cygwin compilation, it mixes a specific compiler define with standard platform definitions.


Support adapter layer for desktop react-electron migrations

Dear team,
We love your work, please help consider our request of supporting adapter layer for desktop react-electron stack migrations;
Tons of thanks

@theankurkedia Will we have this in v3?


option to only record failed checks in CRD

Stanzas like the following are generally not adding value.  They add noise that is hard to filter when reviewing manually. I assume they are also consuming storage in the cluster etcd.  It would be nice to have an option to only include failed checks.  A report that only includes failed checks would be very concise and easy to review/action.
  - category: Kubernetes Security Check
    checkID: KSV105
    description: Containers should be forbidden from running with a root UID.
    messages:
    - ""
    severity: LOW
    success: true
    title: Containers must not set runAsUser to 0

Related #149

@chen-keinan as the reporter of #149 can you clarify what you are asking for there?  Is that issue asking for the option to disable the creation of a VulnerabilityReport if it would not contain any vulnerabilities?
I think vulnerability and config audit reports currently work differently, and i'm basically asking for an option to make configauditreports work like vulnerabilityreports currently do work - always creating the report (though it would be fine if that were an option) but not including checks: objects for which success=true.  (only found vulnerabilities are currently included, not all possible ones that were checked).

A report that only includes failed checks would be very concise and easy to review/action.

I understand what you mean.
Let's extend a little bit on that,  assuming all checks passed , would you expect to see an empty report (when the flag is on)?

I would expect to see a report that matches the spec, with all zeros in the summary section, and an empty checks section.  It would still also show up in the wide listing with zeros in each column.  Basically the way vulnerabilityreports currently work - only providing additional details for failed checks.


Use a context argument in System.register

Thanks so much for the previous work in removing the System global dependency. I'm in the process of finalising the new Traceur plugin for SystemJS and will report back further if there are any further hitches.
I just wanted to go back to this format adjustment in System.register before it is too late though.
While discussing with @sebmck in the related Babel PR, it seemed a context object might be more appropriate as it can the accommodate and expand to fill whatever role the ES Module contextual syntax uses in future.
I'd be interested to hear both your thoughts on this format adjustment.
The PR here updates Traceur to use this idea of a context object argument with just one id property for now, which may well be a much more adaptable approach.
This is in line with the updated Babel PR at https://github.com/babel/babel/pull/3166 and I will also see if I can update the TypeScript PR at https://github.com/Microsoft/TypeScript/pull/6098 as well.

My understanding is that your proposal here is "Replace the __moduleName module-environment string value with an object named $__context, initially containing one property keyed 'id' and having the _moduleName value".
My only concern is the name of the object. Using 'context' gives us no hints and searching online or through source code would lead down many blind alleys. Would you consider say "module" so the change would be __moduleName" -> "$__module.id?
($__ is also overused, to avoid collisions I guess say 16_module would work better.).

My understanding is that your proposal here is "Replace the __moduleName module-environment string value with an object named $__context, initially containing one property keyed 'id' and having the _moduleName value".
My only concern is the name of the object. Using 'context' gives us no hints and searching online or through source code would lead down many blind alleys. Would you consider say "module" so the change would be __moduleName" -> "$__module.id?
($__ is also overused, to avoid collisions I guess say 16_module would work better.).

How about $__moduleContext then?

And yes you're summary is exactly correct!

The Context in moduleContext seems redundant, but if you think its clear then its fine by me.

__moduleContext seems fine to me. The name is a placeholder and making it more explicit even though it is redundant makes it even more clear that that is the case.

LGTM with nits.
@johnjbarton Can you merge if it lgty?

Thanks, corrections made.

I'm confused by the diff, @guybedford  please check that github has the PR the way you want it.

@johnjbarton I've updated to include the comment as you suggested. Otherwise the diff looks good to me.

I was expecting to see $__moduleContext, or perhaps a reason you wanted to stick with $__context.

Ah right! Sorry I completely forgot, I've added that in now.

:+1: thanks for the quick review.

Hi all!
Traceur is parsing __moduleName to $__moduleContext.id right?
In my transpiled code, __moduleName has been parsed to $__moduleContext.id. I read about the change.
However, using __moduleName is just a matter of time before it disappears, correct?
How can correctly handles this now? I am using ES6 modules.
Thank you!
tbragaf


Fix pinout issue with FE310 schematic symbol

Pins 18 and 22 were swapped due to incorrect information in the preliminary manual.
Manual was corrected as of version 1.0.3.
Signed-off-by: Michael Welling<EMAIL_ADDRESS>

Thanks!


How to have rounded border of video player

hello,
Everything works wonderful for me only i am unable to figure out rounded border of video player?

Put the player in a cardView and give cornerRadius as required


Athenz installation problem - zms-go-client fails

Hi! I'm installing Athenz (with mvn clean install) but I keep getting this error and I'm not sure what to do next. Zms-go-client fails during build.

I installed the latest go version and have GOPATH=/home/claudiarmamede/go + GOROOT=/usr/local/go
I'm also using Ubuntu (VERSION="20.04.3 LTS (Focal Fossa)")

rdl/rdl-gen-athenz-go-model is the second module that is built in the project. Can you show me the output from that module? Did it built successfully?
More importantly, the build process expects that your $GOPATH/bin is in your $PATH. See the Go section in our guide: https://github.com/AthenZ/athenz/blob/master/docs/dev_environment.md.

Thank you, my PATH var was not properly updated!


[weibo] extractor fails

Please follow the guide below

You will be asked some questions and requested to provide some information, please read them carefully and answer honestly
Put an x into all the boxes [ ] relevant to your issue (like that [x])
Use Preview tab to see how your issue will actually look like

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2016.05.10. If it's not read this FAQ entry and update. Issues with outdated version will be rejected.

[x] I've verified and I assure that I'm running youtube-dl 2016.05.10

Before submitting an issue make sure you have:

[x] At least skimmed through README and most notably FAQ and BUGS sections
[x] Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

[x] Bug report (encountered problems with youtube-dl)
[ ] Site support request (request for adding support for a new site)
[ ] Feature request (request for a new functionality)
[ ] Question
[ ] Other

The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your issue

If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:
Add -v flag to your command line you run youtube-dl with, copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):
$ youtube-dl -v http://video.weibo.com/v/weishipin/t_zjUw2kZ.htm
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'-v', u'http://video.weibo.com/v/weishipin/t_zjUw2kZ.htm']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2016.05.10
[debug] Python version 2.7.11 - Darwin-15.3.0-x86_64-i386-64bit
[debug] exe versions: avconv 11.4, avprobe 11.4, ffmpeg 3.0.2, ffprobe 3.0.2, rtmpdump 2.4
[debug] Proxy map: {}
[Weibo] zjUw2kZ: Downloading JSON metadata
ERROR: zjUw2kZ: Failed to parse JSON  (caused by ValueError('No JSON object could be decoded',)); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 543, in _parse_json
    return json.loads(json_string)
  File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 339, in loads
    return _default_decoder.decode(s)
  File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 364, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 382, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 676, in extract_info
    ie_result = ie.extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 341, in extract
    return self._real_extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/weibo.py", line 37, in _real_extract
    info = self._download_json(info_url, video_id)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 537, in _download_json
    json_string, video_id, transform_source=transform_source, fatal=fatal)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 547, in _parse_json
    raise ExtractorError(errmsg, cause=ve)
ExtractorError: zjUw2kZ: Failed to parse JSON  (caused by ValueError('No JSON object could be decoded',)); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

If the purpose of this issue is a site support request please provide all kinds of example URLs support for which should be included (replace following example URLs by yours):

Single video: https://www.youtube.com/watch?v=BaW_jenozKc
Single video: https://youtu.be/BaW_jenozKc
Playlist: https://www.youtube.com/playlist?list=PL4lCao7KL_QFVb7Iudeipvc2BCavECqzc

Description of your issue, suggested solution and other information
Explanation of your issue in arbitrary form goes here. Please make sure the description is worded well enough to be understood. Provide as much context and examples as possible.
If work on your issue required an account credentials please provide them or explain how one can obtain them.

Duplicate of #9145.
Use search before submitting.


Failed to decompress the file containing garbled code in MAC

This file is a zip file compressed under Windows 10 (Simplified Chinese). I copied it to Mac and decompressed it. The following error occurred：
reading file in zip archive: /Users/xx/test_demo/d9915c63-812c-6c9e-92fb-66dd7645dc0b/��������: making directory: mkdir /Users/xx/test_demo/d9915c63-812c-6c9e-92fb-66dd7645dc0b/��������: illegal byte sequence
My test sample code:
func TestUnArchiver(t *testing.T) {

	convey.Convey("测试乱码文件解压", t, func() {
		basePath := "/Users/tt/test_demo"
		fileName := "test.zip"

		sourPath := path.Join(basePath, fileName)
		randomPath, err := uuid.GenerateUUID()

		convey.So(err, convey.ShouldBeEmpty)

		resPath := path.Join(basePath, randomPath)

		_, err = CheckAndMkDirPath(resPath)
		convey.So(err, convey.ShouldBeEmpty)

		files, err := Decompression(sourPath, resPath)
		if err != nil {
			fmt.Println(err.Error())
		}
		convey.So(len(files), convey.ShouldBeGreaterThan, 0)
		convey.So(err, convey.ShouldBeEmpty)
	})

}

Decompression function:
func Decompression(source, destination string) (files []string, err error) {
	err = archiver.Unarchive(source, destination)
	if err != nil {
		return files, err
	}
	return GetAllFiles(destination)
}

I debugged this code，in<EMAIL_ADDRESS>file 188 line
Archiver will splice the path and the file name in the compressed package, and then create a folder. However, the actual stitched path is a piece of garbled code and cannot be created normally

My test file:
test.zip
Looking forward to your reply
What version of the package or command are you using?

go version go1.14.6 darwin/amd64
github.com/mholt/archiver/v3 v3.3.0
What are you trying to do?

This file is a zip file compressed under Windows 10 (Simplified Chinese). I copied it to Mac and decompressed it. The following error occurred：
reading file in zip archive: /Users/xx/test_demo/d9915c63-812c-6c9e-92fb-66dd7645dc0b/��������: making directory: mkdir /Users/xx/test_demo/d9915c63-812c-6c9e-92fb-66dd7645dc0b/��������: illegal byte sequence
What steps did you take?

f the file is a compressed package compressed under Mac, it will be OK
What did you expect to happen, and what actually happened instead?

He should have extracted the files, but there was a mistake，
error:
reading file in zip archive: /Users/xx/test_demo/d9915c63-812c-6c9e-92fb-66dd7645dc0b/��������: making directory: mkdir /Users/xx/test_demo/d9915c63-812c-6c9e-92fb-66dd7645dc0b/��������: illegal byte sequence
How do you think this should be fixed?

We can talk about renaming files that cannot be decompressed
Please link to any related issues, pull requests, and/or discussion

Bonus: What do you use archiver for, and do you find it useful?

Very Good！！

最后怎么解决的呢？


chore: update to countervalues V3 api

📝 Description
This PR migrates the countervalues client usage of live-common/src/countervalues from V2 to V3 backend API. This do not include the Market page section of the app but everything else.

the modules logic has been reduced to remove unnecessary logic around the ethbtc rate inversion. (the API correctly allows to ask any from->to really)
Tests has been reinforced to cover more cases.
the pairs sorting has been moved from the api code directly into the logic part of the "inferTrackingPairForAccounts" logic.
the "inferTrackingPairForAccounts" logic will also filter out the accounts that don't have any history yet.

changes for user land of the countervalues module

in TrackingPair type, startDate becomes mandatory because the backend have required it to be mandatory since v3. Hopefully this did not impact a lot of parts (see Impact at the end of this PR description)
remove unused useCalculateMany

What this did not solved, food for future improvements

https://github.com/LedgerHQ/ledger-live/pull/5919
the only remain module is "WETH" which may help to reduce one request to server, but this seems too overkill and specific to have this client side? Sure we know by contract proof that ETH=WETH, but there may be more than WETH to cover if we start thinking this way, it's questionable weither this logic should stay on client side, can be rediscussed in future

❓ Context

JIRA or GitHub link:

https://ledgerhq.atlassian.net/browse/LIVE-10623
https://ledgerhq.atlassian.net/browse/LIVE-10624
https://ledgerhq.atlassian.net/browse/LIVE-3513
https://ledgerhq.atlassian.net/browse/LIVE-10925

✅ Checklist
Pull Requests must pass the CI and be code reviewed. Set as Draft if the PR is not ready.

[x] npx changeset was attached.
[x] Covered by automatic tests.
[x] Impact of the changes:

countervalues – everywhere we see token and currency prices in fiat / crypto values – except the Market page
LLD and LLM display of countervalues.
Hedera fees estimation (it directly uses the api to calculate it)
"Troubleshooting" panel under Settings should still give a green check

🧐 Checklist for the PR Reviewers

[ ] The code aligns with the requirements described in the linked JIRA or GitHub issue.
[ ] The PR description clearly documents the changes made and explains any technical trade-offs or design decisions.
[ ] There are no undocumented trade-offs, technical debt, or maintainability issues.
[ ] The PR has been tested thoroughly, and any potential edge cases have been considered and handled.
[ ] Any new dependencies have been justified and documented.
[ ] Performance considerations have been taken into account. (changes have been profiled or benchmarked if necessary)

=> https://github.com/LedgerHQ/ledger-live/pull/5919


Switch to NP from Jax to  improve attention manager performance

Current Jax implementation is slow. The operation array is tiny, leverage NP compute in CPU can boost the performance.
The performance improved >30x. Below are two benchmark comparison before vs after this PR:
Before
Total: 16 ms,  fill_new_page: 3ms, get_page_token_indices: 13ms
After
Total: 0.5 ms, to_nparray: 0.1 ms, fill_new_page: 0.01ms,  get_page_token_indices: 0.05 ms to_jax_array:0.3 ms

@qihqi @wang2yn84 Please take a look and review this PR.


Unable to use TFOptimizer from keras in tensorflow

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): No
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): NA
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: NA
TensorFlow installed from (source or binary): Running in colab, chrome
TensorFlow version (use command below): 1.10.1
Python version: 2
Bazel version (if compiling from source): NA
GCC/Compiler version (if compiling from source): NA
CUDA/cuDNN version: NA
GPU model and memory: NA
Exact command to reproduce: NA

Describe the problem
Using TFOptimizer from keras inside tensorflow gives error -
---> 22 opt = keras.optimizers.TFOptimizer(tf_opt)
     23 opt.lr = lr
     24

AttributeError: 'module' object has no attribute 'TFOptimizer'

I manually imported it to make the code work-
from tensorflow.python.keras.optimizers import TFOptimizer

Could TFOptimizer class be exported from tensorflow in the following file-
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/keras/optimizers.py#L691
Source code / logs

Added the PR #22467 for the fix.

@yongtang Thanks for the PR. Looks like this issue will be fixed separately with API changes at the level of TF optimizers which is in progress.

Nagging Assignee @wt-huang: It has been 14 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Closing as this will be fixed in API changes.

Hi, I am facing this error now, could anyone please help me out on how to solve it? Thanks.
`Epoch 00001: loss improved from inf to 2.88872, saving model to weights\weights-improvement-01-2.8887.hdf5
Traceback (most recent call last):
File "", line 16, in
history = model.fit(X, y, epochs=10, batch_size=128, callbacks=callbacks_list)
File "C:\Users\nrgra\anaconda3\lib\site-packages\keras\engine\training.py", line 1239, in fit
validation_freq=validation_freq)
File "C:\Users\nrgra\anaconda3\lib\site-packages\keras\engine\training_arrays.py", line 216, in fit_loop
callbacks.on_epoch_end(epoch, epoch_logs)
File "C:\Users\nrgra\anaconda3\lib\site-packages\keras\callbacks\callbacks.py", line 152, in on_epoch_end
callback.on_epoch_end(epoch, logs)
File "C:\Users\nrgra\anaconda3\lib\site-packages\keras\callbacks\callbacks.py", line 719, in on_epoch_end
self.model.save(filepath, overwrite=True)
File "C:\Users\nrgra\anaconda3\lib\site-packages\keras\engine\network.py", line 1152, in save
save_model(self, filepath, overwrite, include_optimizer)
File "C:\Users\nrgra\anaconda3\lib\site-packages\keras\engine\saving.py", line 449, in save_wrapper
save_function(obj, filepath, overwrite, *args, **kwargs)
File "C:\Users\nrgra\anaconda3\lib\site-packages\keras\engine\saving.py", line 541, in save_model
_serialize_model(model, h5dict, include_optimizer)
File "C:\Users\nrgra\anaconda3\lib\site-packages\keras\engine\saving.py", line 163, in _serialize_model
if isinstance(model.optimizer, optimizers.TFOptimizer):
AttributeError: module 'tensorflow.keras.optimizers' has no attribute 'TFOptimizer'`

I am getting the same error too with tf 2.2.0 and kears 2.4.3.
Any help?

@nrgopalrao @Mahmood-Hoseini  Hi ! I have the same problem than you. Did you find a solution please ?

@nrgopalrao I did not. It seems like a incompatibility of tf and keras versions. I moved to pytorch.

Try to replace all the keras to tensorflow.keras

Try to replace all the keras to tensorflow.keras


fix: skip kv pairs that have already been uploaded

This adds an optional param to the function which creates the kv pairs for upload. The logic was pulled from the filter_files function, which itself has been removed (along with its test).
The primary motivation here is to skip any kv pair that has already been uploaded (checked by listing keys from the namespace), so we don't grow the vec so needlessly large.

would be great if someone could verify that the asset_manifest should contain the full list of file metadata even when we can skip the pre-existing kv pair.


warning: no files found matching 'createdb.sql'

This file can probably be deleted.

https://github.com/yzernik/squeaknode/pull/532

https://github.com/yzernik/squeaknode/pull/532


Reslove issue #14

https://github.com/PranabKumarSahoo/sql-dev-tools/assets/65588552/b6fe80ee-c801-4d3a-a29e-3c7225cf6f8c

Great job, @sagarbee!
Your recent contribution with the cool feature is fantastic, and I've just merged the pull request. Thank you for your hard work and dedication to improving our project. Your efforts truly make a difference!
I'm looking forward to seeing more of your contributions in the future.
Feel free to create/work on more issues.
Again, Thanks for the contribution! ⭐

If you have any questions, ideas, or if there's anything else you'd like to work on, feel free to let me know.
#Good luck! ✌


How Internet Works?

Description
Via this issue I will try to explain how internet Works through documentation.
Domain
Frontend Dev HTML/CSS/JS
Type of Contribution
Documentation
Code of Conduct

[X] I follow Contributing Guidelines & Code of conduct of this project.

/assign

/assign


File checker - reduce size of text comparison

The file-checker is failing files, but the target content is actually present.
The tester reports:
Couldn't find source text from xxx.html in target document yyy.dita
Source text:
2.       A mid-life upgrade of these installations was announced in 2011.  ANCHORS was upgraded with a new heat
pump and ACME waste cleanser.  BRAVO is to be upgraded with NOBLE IOT data system and DRAGO halon drench,
as well as a new Drago Mills EMS system

This paragraph of text is present in the published dita. The text looks identical, including   chars, and erroneous double spaces between words.
To avoid the above false-error, I think we should trim the block of text.  I guess if the target contains, say, 30 matching chars successive chars from the source then it is valid.
I'll come back tomorrow and see if I can spot a pattern.
Aah, there is something. In the source html for another file there is a ° marker, but in the dita it is a ASCII degree symbol.  I guess we should strip these out of both strings before comparing.

For the 30 chars, I thought we could simplify the test - by taking a random 30 char long block of text from the string, and if they are also present in the target then it's valid.    Matching a longer string seems to invite more chances for false negatives (through different whitespace or the presence of special characters).
I've just pulled the value of 30 out of the air, since it's longer than Characteristics or some long word that could occur multiple times.
I won't get chance to look for a deeper understanding of why I'm getting false negatives for a few days - but if you're ok with reducing the length of the text being compared, then that itself may reduce the false negatives.


Rasa-ui new functionality 2 : Conversation history management

@paschmann
Following #145
The second huge change we made is the management of the conversation history.
We added a way to review conversation history and set intents and entities from the user history.
See the attached files for more information.
Small bug in add entity, the pop up say "add intention" instead of "add entity" but I'll correct it before the PR.
When an intention or an entity is set, the sentences is added to the training.

I imported agent. Dont show conversation history. What can be reason.

Importing the agent is not enough to have the conversation history. You have to configure something to fill the message table in your DB.
We are currently using logstash + rabbit MQ to fill those table from our core. Once all the tables are correctly filled, you'll see the conversations


Improve the error message for unescaped xml content

Discussed in #13414 as following:

@Mrcubix what error message would you think is better? Any proposal?

If it is possible, and it should be, with some changes that is, apparently :
The character '<' cannot be present within an attribute's value, The character may need to be escaped.
See [Link to a documentation page containing comprehensive list of characters] for more information.

I do not think that the link that robloo provided is appropriate for proper visualisation of those characters.
After doing some research (which may not be sufficient), i was not able to find a page that meet such criteria.
Most only provide character ranges, though one may convert those missing ranges into a table of illegal, and 'may be escaped' characters respectively.
Originally posted by @Mrcubix in https://github.com/AvaloniaUI/Avalonia/discussions/13414#discussioncomment-7407384

I would not have created an issue for this which will likely go unresolved for years.
There is a learning curve for XAML and developers have to take an active roll in improving their skill set.
We can link some place in the docs for XML syntax requirements. However, if the error message is good enough for Microsoft and all of .NET, it's definitely good enough for Avalonia. In fact it's one of the highest quality, easiest to understand error messages there is.
Also note that I provided no link as mentioned in the above text. That was another who helped in the discussion.

GuiLab XML parser is more relaxing and would allow detecting these problems, unlikely old XmlParser from .NET. It's still not really ready to be used in production as for right now.

Ok, if we can use a different parser I backtrack on some of the statements. I do think there was a bit of "blame everyone but myself" in that discussion as it seemed like simple miss (which of course we are all guilty of).

Yeah, it's definitely not as simple as it sounds there.


NaN losses with masked_lm

🐛 Bug
Training with masked_lm throws up NaN losses/perplexities every epoch -- training on the same data with transformer_lm, lstm_lm and fconv_lm works fine (single GPU). The log does seem to throw up "NaN in input tensor" errors but I can't figure out what the exact issue is.
To Reproduce
fairseq-train --task masked_lm \
    --criterion masked_lm ./data-bin \
    --tensorboard-logdir ./stats \
    --dropout 0.33 \
    --save-dir ./model --save-interval 100000 \
    --arch masked_lm --update-freq 16 --max-update 50000 --max-sentences 16

Relevant log:
2020-07-01 15:16:40 | INFO | fairseq_cli.train | training on 1 devices (GPUs/TPUs)
2020-07-01 15:16:40 | INFO | fairseq_cli.train | max tokens per GPU = None and max sentences per GPU = 16
2020-07-01 15:16:40 | INFO | fairseq.trainer | no existing checkpoint found
2020-07-01 15:16:40 | INFO | fairseq.trainer | loading train data for epoch 1
2020-07-01 15:16:41 | INFO | fairseq.data.data_utils | loaded 894888 examples from: <train_file>
2020-07-01 15:16:41 | INFO | fairseq.tasks.masked_lm | loaded 19001 blocks from: <train_file>
2020-07-01 15:16:42 | WARNING | fairseq.data.data_utils | 12 samples have invalid sizes and will be skipped, max_positions=512, first few sample ids=[13323, 5611, 15814, 1666, 11668, 6348, 17408, 3530, 15425, 9692]
2020-07-01 15:21:48 | INFO | fairseq.data.iterators | Data loading buffer is empty or nearly empty. This may indicate a data loading bottleneck, and increasing the number of workers (--num-workers) may help.
2020-07-01 15:29:47 | WARNING | root | NaN or Inf found in input tensor.
2020-07-01 15:29:47 | WARNING | root | NaN or Inf found in input tensor.
2020-07-01 15:29:47 | INFO | valid | epoch 001 | valid on 'valid' subset | loss nan | ppl nan | wps 33655.9 | wpb 8039.5 | bsz 16 | num_updates 75

An example of another LM that does work:
fairseq-train --task language_modeling ./data-bin \
  --save-dir ./model --save-interval 100000 \
  --tensorboard-logdir ./stats \
  --arch transformer_lm \
  --tokens-per-sample 512 --sample-break-mode none \
  --max-tokens 1024 --update-freq 256

Expected behavior
Non-NaN losses.
Environment

fairseq version: master
PyTorch version: 1.4
OS (e.g., Linux): Linux
How you installed fairseq: source
Build command you used: pip install --editable ./
Python version: 3.7
GPU models and configuration: Tesla P-100

You'll need to adjust the hyper-parameters.  You might try following the roberta example as a starting point.  I'm guessing the LR is way too high.  The default is 0.25 (which you are using since you don't specify), whereas the roberta example uses 0.0005.


Added support to bootstrap with --no-lwp option.

I had a problem on centos7 with bootstraping without the --no-lwp options so I added support to use that option when bootstraping.  Please take a look and if you like please merge.

Thank you!


"dotnet nuget push" can be used for local feeds, but not mentioned

Currently, it is suggested to use "nuget add -Source c:\mydir" to publish items to local feed for testing.
As it appears, "dotnet nuget push -s c:\mydir" can do this as well. This should be documented here. If this is the currently recommended way to publish to local feeds, it should be highlighted.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: b363ab07-ea19-574f-e0db-25fd5702255d
Version Independent ID: 6fa9479a-85db-5498-6b7e-2418fc7bac32
Content: dotnet nuget push command - .NET Core CLI
Content Source: docs/core/tools/dotnet-nuget-push.md
Product: dotnet-core
Technology: dotnet-cli
GitHub Login: @karann-msft
Microsoft Alias: tdykstra

https://docs.microsoft.com/en-us/nuget/hosting-packages/local-feeds
But .. are these two commands having same folder layout ?

@yangrongwei not currently, but I filed a bug on NuGet on this https://github.com/NuGet/Home/issues/9281


UI Style not rendering as expected.

Downloaded from github and imported in eclipse, clean install maven, tried to run app-layout-example-plain and app-layout-example-plain-annotation in tomcat 8 but not getting ui as shown in online demo, any one please help me out.

I think it's a duplicate of #166 and #154

@watho yes indeed. Closing in favor of #154.


universalLinks is undefined in IOS

universalLinks.subscribe is not registering in IOS, i am working on Visual Studio Cordova. Plugin is working fine in android but in IOS it is opening app but not registering events. Please let me know what am i doing wrong ?

Since you've closed the issue - my guess you've solved it. What was the problem, if it's not a secret?)

@nikDemyankov i don't know i should share this or not this would be my problem but i am unable to registered this universalLinks.subscribe in IOS. I am working on windows so i have read all your phonegap build discussion. But i have MAC OS also and i have installed this plugin in MAC too, still unable to register my deep paths because i saw after debugging that universalLinks is undefined on my side. So i guess plugin is not installed properly i don't know why because i am unable to see any error. It will helpful for me if you can do anything into it. Thank you

@nikDemyankov same issue is opened here #92

If universalLinks is undefined on your side (JS, I guess) - then there is definitely some installation issue. So I would suggest you to do it from scratch on en empty project via console.
Try to create a new project via console, add this plugin, set it up and see if it will work. Just to eliminate two factors:

some other plugin is in conflict with UL.
VS is not installing plugin as it should.


[Fix #10859] Fix Lint/Debugger to be able to handle method chains correctly

Lint/Debugger was not handling method chains properly, so this fixes it. It also removes the implicit Kernel prefix and makes it explicit in the configuration (it wasn't working as expected anyways).
Fixes #10859.

Before submitting the PR make sure the following are checked:

[x] The PR relates to only one subject with a clear title and description in grammatically correct, complete sentences.
[x] Wrote good commit messages.
[x] Commit message starts with [Fix #issue-number] (if the related issue exists).
[x] Feature branch is up-to-date with master (if not - rebase it).
[x] Squashed related commits together.
[x] Added tests.
[x] Ran bundle exec rake default. It executes all tests and runs RuboCop on its own code.
[x] Added an entry (file) to the changelog folder named {change_type}_{change_description}.md if the new code introduces user-observable changes. See changelog entry format for details.

tried all is working properly ✅

Thanks!


"hasGreenkeeper" now returns "false" for projects without repo-url

... instead of throwing an error
Plus explicit tests for the helper
using "nock" to moch http-responses to badges.greenkeeper.io
to reduce load on that site and make tests faster

Coverage decreased (-0.9%) to 99.052% when pulling 3791f234c017eb50bc478e850da276faa9b126ef on greenkeeper-tests into 6288eac02526d6abc018b8519356309382e50703 on master.

Coverage remained the same at 100.0% when pulling 823bdd60015a53e0b4e014f1dccb7d0f0af44e07 on greenkeeper-tests into 6288eac02526d6abc018b8519356309382e50703 on master.

Coverage remained the same at 100.0% when pulling 88ddff8e21e9e1b6f942115f68751d5102453818 on greenkeeper-tests into 6288eac02526d6abc018b8519356309382e50703 on master.

Coverage remained the same at 100.0% when pulling c5998c14f6ed1eea11af3b6375b91d0098a6fb9f on greenkeeper-tests into 6288eac02526d6abc018b8519356309382e50703 on master.

Coverage remained the same at 100.0% when pulling c5998c14f6ed1eea11af3b6375b91d0098a6fb9f on greenkeeper-tests into 6288eac02526d6abc018b8519356309382e50703 on master.


use latest version

Pull Request Test Coverage Report for Build #98

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 100.0%

Totals

Change from base Build #97:
0.0%

Covered Lines:
295

Relevant Lines:
295

💛  - Coveralls

Pull Request Test Coverage Report for Build #98

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 100.0%

Totals

Change from base Build #97:
0.0%

Covered Lines:
295

Relevant Lines:
295

💛  - Coveralls


Blocks on OVH host

Hi ;)
I've got domain host on OVH.com and I want to initialize a website by uploading it directly to the server. But it randomly blocks in the first uploaded files without errors. It just hangs. Is there a way to toggle debugging?

I've got exactly the same problem on OVH host. Hangs at first or second file.

I used wireshark to observe the dialog between machines.
01 user *login*
02 220-Bienvenue,
03 220-
04 220-    On Vous H..berge ?
05 220-
06 220-Vous ..tes connect.. sur *servname*.
07 220 This is a private system - No anonymous login
08 331 User *login* OK. Password required
09 pass *passwd*
10 230-User *login* has group access to:  users
11 230 OK. Current restricted directory is /
12 type I
13 200 TYPE is now 8-bit binary
14 mkd www
15 550 Can't create directory: File exists
16 mkd www/dir1
17 550 Can't create directory: File exists
18 mkd www/dir1/dir2
19 550 Can't create directory: File exists
20 mkd www/dir1/dir2/dir3
21 550 Can't create directory: File exists
22 pasv
23 227 Entering Passive Mode (213,186,33,208,82,3)
24 stor /www/dir1/dir2/dir3/dir4/s1.css
25 150 Accepted data connection
26 226-File successfully transferred
27 226 1.570 seconds (measured here), 131.02 Kbytes per second
28 pasv
29 227 Entering Passive Mode (213,186,33,208,82,88)
30 stor /www/dir1/dir2/dir3/dir4/s1.min.css
31 150 Accepted data connection
32 226-File successfully transferred
33 226 1.134 seconds (measured here), 130.05 Kbytes per second
34 pasv
35 227 Entering Passive Mode (213,186,33,208,82,174)
36 stor /www/dir1/dir2/dir3/dir4/s2.css
37 150 Accepted data connection
38 pasv
39 226-File successfully transferred
40 226 1.573 seconds (measured here), 130.76 Kbytes per second
[hangs here]

It seams that each time, it hangs when the client sends the pasv command before the server has sent back the '226-File successfully transferred' message for the previous file.
For example here a typical sequence would be lines [30-35], whereas on the [36-40] seqence, l.38 the 'pasv' command comes earlier.

Hmm that's very interesting.  I will have to look through some posts about OVH and see what I can dig up.  It seems like I have read a lot about others having issues with OVH but not with similar errors and have came across the occasional 'Bad passive command from server'  with some other plugins. (https://github.com/zonak/grunt-ftp-deploy/issues/57) although no resolution on this one.  Last time I did some digging I did not come across any obvious solutions.  I can try to open up a ticket with the creator of jsftp as that's the underlying ftp technology being used by this plugin and see if he is aware of any similar issues.

Thanks for the wireshark printout, that will definitely help with getting this resolved.

Created ticket with JSFTP owner to see if this is something he may be aware of.  https://github.com/sergi/jsftp/issues/113

Good to hear! I hope you'll find something with @sergi ;)

Thanks @mpiffault, once I hear something from the guys with the jsftp module I'll make sure to update it here.

Hi there,
It definitely seems that pasv is getting in the way before the 226 response comes back. The latest code in jsftp has a new debug mode, it would be interesting to see what's the output with that enabled. Using npm link should allow you to test it. The debug mode is activated by setting the DEBUG environment variable to 1, as in DEBUG=1 node server.js.

Hi @sergi,
Thanks for responding here.  I personally don't have an account to OVH to test so let me see if @pyrsmk  or @mpiffault would be willing to run a test for me.  Environment variables in grunt require an additional plugin which I would like to avoid so I am going to update this plugin to use jsftp 1.3.9 and give them an option to debug.  I noticed setDebugMode is going to be deprecated so for the moment I will use debugMode: true in the jsftp constructor.  If this is going away soon I can add the grunt-env plugin and set things up that way.
@pyrsmk @mpiffault
I have just updated now and things look good, I just need to update the readme so it should be published in the next 20 minutes.  If either of you have a chance could you try running this again but with debug option set to true and posting the results here or linking them in a text file. Make sure when you run this your on version 0.3.2 as that will be the version that includes this.  It should look like this in you Gruntfile to run in debug mode:

ftp_push: {
  default: {
    options: {
      authKey: "serverA",
      host: "sample.server.com",
      dest: "/html/test/",
      debug: true
    },
    files: [
      {expand: true,cwd: 'test',src: ['**/*']},
      {expand: true,cwd: 'tasks',src: ['**/*'], dest: 'tasks/' }
    ]
  }
}

NOTE: When you run in debug mode, before you post anything make sure your username and password are removed as they may show up in the console when using debug mode, so please be careful not to copy them and past them here.

Hi @Robert-W,
The version should actually be 1.4.0, it has just been published. I think the best way to test that would actually be to make a test with jsftp only first, not using grunt-ftp-push and just doing a put using it siilarly to how I do it in jsftp tests. Just a suggestion.

Thanks @sergi,
I'm in the process of rewriting this plugin with unit tests so Ill make a separate file to test specifically the put commands kind of how your doing with jsftp.  I'll see if I can get access to an OVH account or get someone with access to one to run these tests for me and put the results of running in debug mode up here.  Thanks for the help, much appreciated.

Hello,
I'm experiencing the problem on OVH FTP.
I can help you by testing what you want. Tell me how to run the tests and what is helping you to debug.
Regards,

Awesome thanks @lmeyer ,
Im gonna write some more specific tests this weekend to specifically test the jsftp portion but in the meantime if you want to try setting debug to true in the options like so:

ftp_push: {
  default: {
    options: {
      authKey: "serverA",
      host: "sample.server.com",
      dest: "/html/test/",
      debug: true
    },
    files: [
      { ... },
      { ... }
    ]
  }
}

It will run jsftp in debug mode.  This will output lots of text in the console and if you can copy the last bits of it around where it fails that could be very helpful, I don't think I will need the whole output just, the stuff around the last few transfers.
I probably don't need to say this but just make sure to exclude credentials in what you post because when in debug mode, the first part of the output may contain your credentials.

Hello,
I pasted the first commands and the 3 last transfers
The last one is just freezing at last line of the debug output
Hope it helps,
DEBUG:  event:connect
{}
DEBUG:  response
{
  "code": 220,
  "text": "220-Bienvenue,\n220-\n220-    On Vous Héberge ?\n220-\n220-Vous êtes connecté sur webm411.\n220 This is a private system - No anonymous login",
  "isMark": false,
  "isError": false
}
DEBUG:  response
{
  "code": 331,
  "text": "331 User XXXXXXX OK. Password required",
  "isMark": false,
  "isError": false
}
DEBUG:  user_command
"pass XXXXXXX"
DEBUG:  response
{
  "code": 230,
  "text": "230-User XXXXXXX has group access to:  users     \n230 OK. Current restricted directory is /",
  "isMark": false,
  "isError": false
}
DEBUG:  user_command
"type I"
DEBUG:  response
{
  "code": 200,
  "text": "200 TYPE is now 8-bit binary",
  "isMark": false,
  "isError": false
}
>> systemev successfully authenticated!
DEBUG:  user_command
"mkd www"
DEBUG:  response
{
  "code": 550,
  "text": "550 Can't create directory: File exists",
  "isMark": false,
  "isError": true
}

DEBUG:  user_command
"pasv"
DEBUG:  response
{
  "code": 226,
  "text": "226-File successfully transferred\n226 0.036 seconds (measured here), 18.53 Kbytes per second",
  "isMark": false,
  "isError": false
}
DEBUG:  response
{
  "code": 227,
  "text": "227 Entering Passive Mode (213,186,33,201,199,253)",
  "isMark": false,
  "isError": false
}
DEBUG:  user_command
"stor /www/wp-content/themes/test/_footer_newsletter.php"
DEBUG:  response
{
  "code": 150,
  "text": "150 Accepted data connection",
  "isMark": true,
  "isError": false
}
>> /www/wp-content/themes/test/_footer_newsletter.php transferred successfully.
DEBUG:  user_command
"pasv"
DEBUG:  response
{
  "code": 226,
  "text": "226-File successfully transferred\n226 0.340 seconds (measured here), 2.62 Kbytes per second",
  "isMark": false,
  "isError": false
}
DEBUG:  response
{
  "code": 227,
  "text": "227 Entering Passive Mode (213,186,33,201,200,11)",
  "isMark": false,
  "isError": false
}
DEBUG:  user_command
"stor /www/wp-content/themes/test/_footer_team.php"
DEBUG:  response
{
  "code": 150,
  "text": "150 Accepted data connection",
  "isMark": true,
  "isError": false
}
>> /www/wp-content/themes/test/_footer_team.php transferred successfully.
DEBUG:  user_command
"pasv"
DEBUG:  response
{
  "code": 226,
  "text": "226-File successfully transferred\n226 0.043 seconds (measured here), 35.57 Kbytes per second",
  "isMark": false,
  "isError": false
}

I'm available to do more tests

Great Thanks @lmeyer , Im going to write some more specific unit tests this weekend and I'll let you know how to run them. It will basically be a jsftp only test to try to put multiple files on to OVH server and do so in debug mode to give us some more specific details.
Hey @sergi, @lmeyer was kind enough to run my plugin in debug mode and paste the results here, is anything in there jumping out to you?  I will get the jsftp only test up this weekend and see if we can get some more specific debug info in here as well.

Hey @lmeyer ,
If your still up for it I committed a new test we can try.  The steps for running the test are as follows:

Clone the repo
npm install to install dependencies
Edit test/jsftp/jsftpSpec.js

there is a defaults object at the top, change the user, pass, and host
there is also an iterations variable below the defaults to control how many times to put and delete the test file, since OVH does not fail on the first file I added this so you can set how many times you run the test, it's set to 5 but if it does not throw that error after 5 try incrementing to 10.

var defaults = {
	    username: 'user',
	    password: 'pass',
	    host: 'sample.server.com'
};

npm run jsftp-test

After running the test, just do the same as last time, copy the last few transfers right before it fails and paste them here.  Thanks a bunch for the help on this.

Hello,
I did some tests :

With the test file you provided

Seems to be ok with, 5, 10, 20, 30, up to 80 iterations
Between 80 and 100 iterations i have random results (sometimes works, sometimes not)
With 100 iterations, i have results like

Failures:

  1) Grunt Plugin - FTP Push - JSFTP test for troublesome hosts Should push a file and remove the file multiple times successfully.
   Message:
     timeout: timed out after 5000 msec waiting for spec to complete
   Stacktrace:
     undefined

Finished in 9.005 seconds
1 test, 169 assertions, 1 failure, 0 skipped

Sometimes i have this one : i retrieve the Test.txt file on the FTP and the test is freezing
DEBUG:  response
{
  "code": 211,
  "text": "211-Extensions supported:\n EPRT\n IDLE\n MDTM\n SIZE\n REST STREAM\n MLST type*;size*;sizd*;modify*;UNIX.mode*;UNIX.uid*;UNIX.gid*;unique*;\n MLSD\n AUTH TLS\n PBSZ\n PROT\n UTF8\n TVFS\n ESTA\n PASV\n EPSV\n SPSV\n ESTP\n211 End.",
  "isMark": false,
  "isError": false
}
DEBUG:  response
{
  "code": 215,
  "text": "215 UNIX Type: L8",
  "isMark": false,
  "isError": false
}
DEBUG:  response
{
  "code": 215,
  "text": "215 UNIX Type: L8",
  "isMark": false,
  "isError": false
}
DEBUG:  response
{
  "code": 331,
  "text": "331 User XXXXXX OK. Password required",
  "isMark": false,
  "isError": false
}
DEBUG:  response
{
  "code": 331,
  "text": "331 User XXXXXX OK. Password required",
  "isMark": false,
  "isError": false
}
DEBUG:  response
{
  "code": 530,
  "text": "530 You aren't logged in",
  "isMark": false,
  "isError": true
}
DEBUG:  response
{
  "code": 530,
  "text": "530 You aren't logged in",
  "isMark": false,
  "isError": true
}

With a bigger file 5Ko the errors seems to be the same
With a bigger file 220Ko-480Ko the errors (both) happen very frequently, even with 5 iterations

Very strange that the tests need so much iterations for little files because when i tested in real conditions never more than 5 files where uploaded.
Tell me if it helps

I tried the same tests on an "older"/different OVH FTP.
Here are the resuls :

With you test file

Errors occurs from 5 iterations. I got a new message

DEBUG:  response
{
  "code": 530,
  "text": "530 You aren't logged in",
  "isMark": false,
  "isError": true
}
DEBUG:  response
{
  "code": 530,
  "text": "530 You aren't logged in",
  "isMark": false,
  "isError": true
}
DEBUG:  response
{
  "code": 421,
  "text": "421 Timeout - try typing a little faster next time",
  "isMark": false,
  "isError": true
}
DEBUG:  response
{
  "code": 421,
  "text": "421 Timeout - try typing a little faster next time",
  "isMark": false,
  "isError": true
}

I have almost always the timeout error

Failures:

  1) Grunt Plugin - FTP Push - JSFTP test for troublesome hosts Should push a file and remove the file multiple times successfully.
   Message:
     timeout: timed out after 5000 msec waiting for spec to complete
   Stacktrace:
     undefined

Didn't need to try with bigger files

Thanks a lot man, I think this info will definitely be helpful.
I was hoping it would fail with 5 iterations and did not think it would need to go so high.  Strange it's working now, I wonder if I need to modify the tests to push one of several files of different sizes and types and just keep iterating over the different files.
The timeout error is because jasmine-node defaults to timeouts of 5 seconds, I did not realize this initially as I thought async tests would stay running until done() was called.  A quick fix for the timeout is right around line 25, beneath the describe, add jasmine.getEnv().defaultTimeoutInterval = 10000; or whatever value is needed or just do a pull, I just pushed that in now.

@sergi We have some test results, Can you check these out when you have a minute and let me know if anything sticks out.

I pulled and tried different size, iterations, and FTPs
I get same errors as before :
DEBUG:  response
{
  "code": 331,
  "text": "331 User XXXXXXX OK. Password required",
  "isMark": false,
  "isError": false
}
DEBUG:  response
{
  "code": 530,
  "text": "530 You aren't logged in",
  "isMark": false,
  "isError": true
}
DEBUG:  response
{
  "code": 530,
  "text": "530 You aren't logged in",
  "isMark": false,
  "isError": true
}

and
Failures:

  1) Grunt Plugin - FTP Push - JSFTP test for troublesome hosts Should push a file and remove the file multiple times successfully.
   Message:
     timeout: timed out after 10000 msec waiting for spec to complete
   Stacktrace:
     undefined

Regards,

Ok thanks, the timeout will have to be adjusted to be a time long enough time to complete the tests otherwise it will throw that final error but that error is not really effecting anything anyway.  I wish with this version of jasmine-node they did not enforce the default timeout and let me decide when to end the test by calling done() but don't think that's possible on this version.

Closing due to inactivity


Fix the dependency to the version based

Change the dependency of Swift-Kuery to the version based dependency.
This will make the integration of Swifty-Kuery-PostgreSQL from multiple packages easier and avoid the error message type like this one:
error: because package swift-kuery-postgresql is required using a version-based requirement and it depends on unversion package swift-kuery and no versions of Swift-Kuery-PostgreSQL match the requirement 2.1.201..<3.0.0, Swift-Kuery-PostgreSQL >=2.1.200 is forbidden.

@mbarnach Thanks for making this change.  It looks like the Linux builds are failing due to docker rate limits.  If you'd like to incorporate some of the changes to the .travis.yml file you can model after the recent changes I made to Kitura-NIO to get it to pass.  Or I make the changes if you'd like.

I will have a look at Travis and let you know if I can't make it work properly. Thanks for the pointer.

@dannys42 I've updated the Travis builds. I get ride of 4.0.3, 4.1.3 and 4.2.4 as they are not (yet) bundled in docker.kitura.net. Do you plan to add them or shall we remove the support all together?
Concerning the development snapshot, I've used the same as in Kitura-NIO, but I have an access error. Do you have any plan to address this? Let me know if you want me to run/tests some other changes.

@dannys42 I've updated the Travis builds. I get ride of 4.0.3, 4.1.3 and 4.2.4 as they are not (yet) bundled in docker.kitura.net. Do you plan to add them or shall we remove the support all together?
Concerning the development snapshot, I've used the same as in Kitura-NIO, but I have an access error. Do you have any plan to address this? Let me know if you want me to run/tests some other changes.

@mbarnach Ah, thanks for mentioning that.  I updated both to swift-5.4-DEVELOPMENT-SNAPSHOT-2021-02-26-a (the Kitura project uses that).  I wish there was a better way to manage this as it's a bit of a pain to switch this for every project.  I think I might add some functionality to the PackageBuilder repo for this eventually so that we have only one place to update the development snapshot.
Regarding Swift 4.x... Unfortunately swift.org does not provide dockers for Swift 4.x, so I can't easily create docker images for it.  So I think it's reasonable to just remove the CI builds for Swift 4.x.  We should probably make a deprecation announcement along with what that means exactly.

Shall we make a new release to ease the integration?

@mbarnach Done!  I also added a link to the docs.  Don't suppose you know if there's a way to have GitHub populate that automatically?

Thanks!
Do you want to do the release creation automatically or the documentation generation?
I see that this framework needs some love, the README is still pointing to IBM. Do you still have your magic scripts or shall I try to do it manually?

@mbarnach I just want the docs to be linkable on the project page in GitHub.  If you click the "About" settings, it seems to have text that is what I want but I can't simply say "use it".  I have to retype the whole string.  Very strange.
I don't have any magic scripts.  Just vi and search&replace. :-)


spec: add FWFT extension support to SBI implementation

This commit introduces the necessary constants for Extension ID (EID) and Function ID (FID) associated with the Firmware Features Table (FWFT) extension, as per the RISC-V SBI specification. Documentation for these constants has also been updated to facilitate better understanding and usage.
Signed-off-by: Longbing Zheng<EMAIL_ADDRESS>

LGTM! Please squash the two commits, as the intermediate commit (the first one) does not pass CI test.

Thanks!


Reconsider the internal representation of Script

Script currently contains a boxed slice.
Pros/cons vs Vec<u8>:

Faster to move (but moves are fast anyway, so the difference is crazy small)
Consumes less memory: 1 usize less for capacity and guaranteed no extra memory
Construction may force costly reallocation even for ridiculous values (e.g. to save 1 byte of memory)
Having a shitload of scripts outside Transaction is unusual, the speed of moving Transaction is already independent of Script size (type; not to be confused with length)

Are the benefits worth extra reallocation on construction? My guess is no, but we should probably benchmark it.
Possible approaches:

It can be demonstrated that the current design is really the best and we just close this issue
It can be demonstrated Vec<u8> is the best and we switch
Depends on application we expose feature flag to globally switch the strategy - easier but less flexible
We make it generic over storage with some reasonable default

Possibly related issue: https://github.com/rust-bitcoin/rust-bitcoin/issues/522

I'm doubtful that Box<iu8]> is worth the hassle vs Vec<u8>. This was a premature optimization that I did when I discovered Box<[u8]> when the project was a toy and Script never changed, and I never benchmarked it.

FTR #1155 is surprisingly an improved version of the last option. The combination of relatively simple conversions between ScriptBuf and Box<Script>  and deref coercions make the implementation reasonably simple. Previously I expected this to be much more boilerplate-y with little value.


Call ::NewRelic::Agent::PrependSupportability.record_metrics_for in ActiveSupport.on_load

I think this gem(<IP_ADDRESS>7 and <IP_ADDRESS>8) breaks ActiveRecord configuration in Rails 5.0.x. Because it does not hook ActiveRecord correctly.
Rails 5.1.x does not have this problem. Because load_defaults method is called in config/application.rb before this gem is loaded.
I think Rails 4 looks fine because it hooks correctly.
https://github.com/newrelic/rpm/blob/35cb1eea0d9301d30bc5cd7aad63f8a332cdad1a/lib/new_relic/agent/instrumentation/active_record_4.rb#L30
Related to: https://github.com/rails/rails/issues/23589

Example
rails _5.0.6_ new ArHookBug
cd ArHookBug
rails g scaffold user name:string
rails g scaffold profile user:references address:string
rails db:migrate
rails s

# Open http://localhost:3000/profiles/new
# Click `Create Profile`
# You can't create a user because `User must exist`. This is correct.

echo "gem 'newrelic_rpm'" >> Gemfile
bundle
rails s

# Open http://localhost:3000/profiles/new
# Click `Create Profile`
# You can create a user, but it shouldn't.

Sorry, this is duplicated at #277.