Webhook and multiple files

We've now finally implemented a whole workflow for uploading files with the webhook as discussed at #475
It works perfect but only ever when one uploads a file sequential, i.e., if I upload 2 or more files simultaneously the event only ever shows the first file of the batch. I've tried it with Cyberduck and WinSCP.
I've tried it with custom events and without, but it made no difference. Is there something I need to change or configure?

Update:
I used Haproxy with two VersityGW servers. As soon as I stop one server it works with multiple files. Would stickiness help in this regard or is a load balancer configuration not supported?

@thenitai Just to make sure I understand the setup, the load balancer is distributing requests between two versitygw endpoints. And each of these is set to send events to the same webhook?
I wouldn't expect the load balancer to have any effect on how the events are sent to the webhook.  Could you verify that both versitygw instances can send to the webhook if sending requests directly to them? Could be that one of the gateways is unable to connect to the webhook for some reason?
I think we might have a bug where if the webhook is not accessible when the gateway first starts, then it will never send any events to that webhook. We may need to change this so that it starts sending events whenever the connection becomes available.

Both versityGW servers work when I only have one running behind the load balancer.
haproxy -> versityGW-ONE - WORKS
haproxy -> versityGW-TWO - WORKS
haproxy -> versityGW-ONE & versityGW-TWO - Only ONE file webhook request is sent
Does that make sense?

Both versityGW servers work when I only have one running behind the load balancer.
haproxy -> versityGW-ONE - WORKS haproxy -> versityGW-TWO - WORKS haproxy -> versityGW-ONE & versityGW-TWO - Only ONE file webhook request is sent
Does that make sense?

Yes, thank you. I'll see if I can reproduce this over here.

Ok, I ran a test that looks like its working ok. Let me know if this test is not testing the case you are looking for.  I have a simple webhook endpoint that just prints the client address and the payload:
package main

import (
	"fmt"
	"io"
	"log"
	"net/http"
	"strings"
)

func main() {
	http.HandleFunc("/s3events", func(w http.ResponseWriter, r *http.Request) {
		defer r.Body.Close()
		buf := new(strings.Builder)
		io.Copy(buf, r.Body)
		fmt.Println(r.RemoteAddr, buf.String())
	})

	log.Fatal(http.ListenAndServe(":8080", nil))
}

I started two gateways on different hosts, both pointing at the same webhook endpoint:
versitygw -a test -s test --event-webhook-url http://<IP_ADDRESS>:8080/s3events posix /mnt/gwtest

I see the test messages pop up in the webhook app:
<IP_ADDRESS>:54314 {"Bucket":"Test-Bucket","Event":"s3:TestEvent","Service":"S3","Time":"2024-06-04T19:23:34Z"}
<IP_ADDRESS>:38200 {"Bucket":"Test-Bucket","Event":"s3:TestEvent","Service":"S3","Time":"2024-06-04T19:23:35Z"}

I configured haproxy on one of the hosts to load balance across both of them:
# cat /etc/haproxy/haproxy.cfg
global
    log         <IP_ADDRESS> local2

    chroot      /var/lib/haproxy
    pidfile     /var/run/haproxy.pid
    maxconn     4000
    user        haproxy
    group       haproxy
    daemon

    # turn on stats unix socket
    stats socket /var/lib/haproxy/stats

defaults
    mode                    http
    log                     global
    option                  httplog
    option                  dontlognull
    option http-server-close
    option forwardfor       except <IP_ADDRESS>/8
    option                  redispatch
    retries                 3
    timeout http-request    10s
    timeout queue           1m
    timeout connect         10s
    timeout client          1m
    timeout server          1m
    timeout http-keep-alive 10s
    timeout check           10s
    maxconn                 3000

frontend  main *:5000
    default_backend             gw

backend gw
    balance     roundrobin
    server  gw1 <IP_ADDRESS>:7070
    server  gw2 <IP_ADDRESS>:7070

And I setup my client to point to the haproxy listening port:
export AWS_SECRET_ACCESS_KEY=test
export AWS_ACCESS_KEY_ID=test
export AWS_ENDPOINT_URL=http://<IP_ADDRESS>:5000
export AWS_REGION=us-east-1

I then ran the following to upload 10 objects at the same time:
% for i in $(seq 1 10); do aws s3 cp ./file1mb s3://testbucket1/testfile1mb$i & done; wait
[2] 65635
[3] 65636
[4] 65637
[5] 65638
[6] 65639
[7] 65640
[8] 65641
[9] 65642
[10] 65643
[11] 65644
upload: ./file1mb to s3://testbucket1/testfile1mb10
[11]  + done       aws s3 cp ./file1mb s3://testbucket1/testfile1mb$i
upload: ./file1mb to s3://testbucket1/testfile1mb9
[10]  + done       aws s3 cp ./file1mb s3://testbucket1/testfile1mb$i
upload: ./file1mb to s3://testbucket1/testfile1mb1
[2]    done       aws s3 cp ./file1mb s3://testbucket1/testfile1mb$i
upload: ./file1mb to s3://testbucket1/testfile1mb5
upload: ./file1mb to s3://testbucket1/testfile1mb3
[6]    done       aws s3 cp ./file1mb s3://testbucket1/testfile1mb$i
[4]    done       aws s3 cp ./file1mb s3://testbucket1/testfile1mb$i
upload: ./file1mb to s3://testbucket1/testfile1mb7
upload: ./file1mb to s3://testbucket1/testfile1mb8
[8]  - done       aws s3 cp ./file1mb s3://testbucket1/testfile1mb$i
[9]  + done       aws s3 cp ./file1mb s3://testbucket1/testfile1mb$i
upload: ./file1mb to s3://testbucket1/testfile1mb2
[3]    done       aws s3 cp ./file1mb s3://testbucket1/testfile1mb$i
upload: ./file1mb to s3://testbucket1/testfile1mb4
[5]  - done       aws s3 cp ./file1mb s3://testbucket1/testfile1mb$i
upload: ./file1mb to s3://testbucket1/testfile1mb6
[7]  + done       aws s3 cp ./file1mb s3://testbucket1/testfile1mb$i

And I see all events in the webhook app:
<IP_ADDRESS>:38326 {"Records":[{"eventVersion":"2.2","eventSource":"aws:s3","awsRegion":"us-east-1","eventTime":"2024-06-04T21:23:17Z","eventName":"s3:ObjectCreated:Put","userIdentity":{"PrincipalId":"test"},"requestParameters":{"sourceIPAddress":"<IP_ADDRESS>"},"responseElements":{"x-amz-request-id":"","x-amz-id-2":""},"s3":{"s3SchemaVersion":"1.0","configurationId":"webhook-global","bucket":{"name":"testbucket1","ownerIdentity":{"PrincipalId":"test"},"arn":"arn:aws:s3:::/testbucket1/testfile1mb10"},"object":{"key":"testfile1mb10","size":1048576,"eTag":"b6d81b360a5672d80c27430f39153e2c","versionId":null,"sequencer":"1"}},"glacierEventData":{"restoreEventData":{"lifecycleRestorationExpiryTime":"","lifecycleRestoreStorageClass":""}}}]}
<IP_ADDRESS>:54404 {"Records":[{"eventVersion":"2.2","eventSource":"aws:s3","awsRegion":"us-east-1","eventTime":"2024-06-04T21:23:17Z","eventName":"s3:ObjectCreated:Put","userIdentity":{"PrincipalId":"test"},"requestParameters":{"sourceIPAddress":"<IP_ADDRESS>"},"responseElements":{"x-amz-request-id":"","x-amz-id-2":""},"s3":{"s3SchemaVersion":"1.0","configurationId":"webhook-global","bucket":{"name":"testbucket1","ownerIdentity":{"PrincipalId":"test"},"arn":"arn:aws:s3:::/testbucket1/testfile1mb9"},"object":{"key":"testfile1mb9","size":1048576,"eTag":"b6d81b360a5672d80c27430f39153e2c","versionId":null,"sequencer":"1"}},"glacierEventData":{"restoreEventData":{"lifecycleRestorationExpiryTime":"","lifecycleRestoreStorageClass":""}}}]}
<IP_ADDRESS>:38326 {"Records":[{"eventVersion":"2.2","eventSource":"aws:s3","awsRegion":"us-east-1","eventTime":"2024-06-04T21:23:17Z","eventName":"s3:ObjectCreated:Put","userIdentity":{"PrincipalId":"test"},"requestParameters":{"sourceIPAddress":"<IP_ADDRESS>"},"responseElements":{"x-amz-request-id":"","x-amz-id-2":""},"s3":{"s3SchemaVersion":"1.0","configurationId":"webhook-global","bucket":{"name":"testbucket1","ownerIdentity":{"PrincipalId":"test"},"arn":"arn:aws:s3:::/testbucket1/testfile1mb1"},"object":{"key":"testfile1mb1","size":1048576,"eTag":"b6d81b360a5672d80c27430f39153e2c","versionId":null,"sequencer":"2"}},"glacierEventData":{"restoreEventData":{"lifecycleRestorationExpiryTime":"","lifecycleRestoreStorageClass":""}}}]}
<IP_ADDRESS>:38326 {"Records":[{"eventVersion":"2.2","eventSource":"aws:s3","awsRegion":"us-east-1","eventTime":"2024-06-04T21:23:18Z","eventName":"s3:ObjectCreated:Put","userIdentity":{"PrincipalId":"test"},"requestParameters":{"sourceIPAddress":"<IP_ADDRESS>"},"responseElements":{"x-amz-request-id":"","x-amz-id-2":""},"s3":{"s3SchemaVersion":"1.0","configurationId":"webhook-global","bucket":{"name":"testbucket1","ownerIdentity":{"PrincipalId":"test"},"arn":"arn:aws:s3:::/testbucket1/testfile1mb3"},"object":{"key":"testfile1mb3","size":1048576,"eTag":"b6d81b360a5672d80c27430f39153e2c","versionId":null,"sequencer":"3"}},"glacierEventData":{"restoreEventData":{"lifecycleRestorationExpiryTime":"","lifecycleRestoreStorageClass":""}}}]}
<IP_ADDRESS>:54404 {"Records":[{"eventVersion":"2.2","eventSource":"aws:s3","awsRegion":"us-east-1","eventTime":"2024-06-04T21:23:18Z","eventName":"s3:ObjectCreated:Put","userIdentity":{"PrincipalId":"test"},"requestParameters":{"sourceIPAddress":"<IP_ADDRESS>"},"responseElements":{"x-amz-request-id":"","x-amz-id-2":""},"s3":{"s3SchemaVersion":"1.0","configurationId":"webhook-global","bucket":{"name":"testbucket1","ownerIdentity":{"PrincipalId":"test"},"arn":"arn:aws:s3:::/testbucket1/testfile1mb5"},"object":{"key":"testfile1mb5","size":1048576,"eTag":"b6d81b360a5672d80c27430f39153e2c","versionId":null,"sequencer":"2"}},"glacierEventData":{"restoreEventData":{"lifecycleRestorationExpiryTime":"","lifecycleRestoreStorageClass":""}}}]}
<IP_ADDRESS>:38326 {"Records":[{"eventVersion":"2.2","eventSource":"aws:s3","awsRegion":"us-east-1","eventTime":"2024-06-04T21:23:18Z","eventName":"s3:ObjectCreated:Put","userIdentity":{"PrincipalId":"test"},"requestParameters":{"sourceIPAddress":"<IP_ADDRESS>"},"responseElements":{"x-amz-request-id":"","x-amz-id-2":""},"s3":{"s3SchemaVersion":"1.0","configurationId":"webhook-global","bucket":{"name":"testbucket1","ownerIdentity":{"PrincipalId":"test"},"arn":"arn:aws:s3:::/testbucket1/testfile1mb8"},"object":{"key":"testfile1mb8","size":1048576,"eTag":"b6d81b360a5672d80c27430f39153e2c","versionId":null,"sequencer":"4"}},"glacierEventData":{"restoreEventData":{"lifecycleRestorationExpiryTime":"","lifecycleRestoreStorageClass":""}}}]}
<IP_ADDRESS>:38326 {"Records":[{"eventVersion":"2.2","eventSource":"aws:s3","awsRegion":"us-east-1","eventTime":"2024-06-04T21:23:18Z","eventName":"s3:ObjectCreated:Put","userIdentity":{"PrincipalId":"test"},"requestParameters":{"sourceIPAddress":"<IP_ADDRESS>"},"responseElements":{"x-amz-request-id":"","x-amz-id-2":""},"s3":{"s3SchemaVersion":"1.0","configurationId":"webhook-global","bucket":{"name":"testbucket1","ownerIdentity":{"PrincipalId":"test"},"arn":"arn:aws:s3:::/testbucket1/testfile1mb7"},"object":{"key":"testfile1mb7","size":1048576,"eTag":"b6d81b360a5672d80c27430f39153e2c","versionId":null,"sequencer":"5"}},"glacierEventData":{"restoreEventData":{"lifecycleRestorationExpiryTime":"","lifecycleRestoreStorageClass":""}}}]}
<IP_ADDRESS>:54404 {"Records":[{"eventVersion":"2.2","eventSource":"aws:s3","awsRegion":"us-east-1","eventTime":"2024-06-04T21:23:18Z","eventName":"s3:ObjectCreated:Put","userIdentity":{"PrincipalId":"test"},"requestParameters":{"sourceIPAddress":"<IP_ADDRESS>"},"responseElements":{"x-amz-request-id":"","x-amz-id-2":""},"s3":{"s3SchemaVersion":"1.0","configurationId":"webhook-global","bucket":{"name":"testbucket1","ownerIdentity":{"PrincipalId":"test"},"arn":"arn:aws:s3:::/testbucket1/testfile1mb2"},"object":{"key":"testfile1mb2","size":1048576,"eTag":"b6d81b360a5672d80c27430f39153e2c","versionId":null,"sequencer":"3"}},"glacierEventData":{"restoreEventData":{"lifecycleRestorationExpiryTime":"","lifecycleRestoreStorageClass":""}}}]}
<IP_ADDRESS>:54404 {"Records":[{"eventVersion":"2.2","eventSource":"aws:s3","awsRegion":"us-east-1","eventTime":"2024-06-04T21:23:18Z","eventName":"s3:ObjectCreated:Put","userIdentity":{"PrincipalId":"test"},"requestParameters":{"sourceIPAddress":"<IP_ADDRESS>"},"responseElements":{"x-amz-request-id":"","x-amz-id-2":""},"s3":{"s3SchemaVersion":"1.0","configurationId":"webhook-global","bucket":{"name":"testbucket1","ownerIdentity":{"PrincipalId":"test"},"arn":"arn:aws:s3:::/testbucket1/testfile1mb4"},"object":{"key":"testfile1mb4","size":1048576,"eTag":"b6d81b360a5672d80c27430f39153e2c","versionId":null,"sequencer":"4"}},"glacierEventData":{"restoreEventData":{"lifecycleRestorationExpiryTime":"","lifecycleRestoreStorageClass":""}}}]}
<IP_ADDRESS>:54404 {"Records":[{"eventVersion":"2.2","eventSource":"aws:s3","awsRegion":"us-east-1","eventTime":"2024-06-04T21:23:18Z","eventName":"s3:ObjectCreated:Put","userIdentity":{"PrincipalId":"test"},"requestParameters":{"sourceIPAddress":"<IP_ADDRESS>"},"responseElements":{"x-amz-request-id":"","x-amz-id-2":""},"s3":{"s3SchemaVersion":"1.0","configurationId":"webhook-global","bucket":{"name":"testbucket1","ownerIdentity":{"PrincipalId":"test"},"arn":"arn:aws:s3:::/testbucket1/testfile1mb6"},"object":{"key":"testfile1mb6","size":1048576,"eTag":"b6d81b360a5672d80c27430f39153e2c","versionId":null,"sequencer":"5"}},"glacierEventData":{"restoreEventData":{"lifecycleRestorationExpiryTime":"","lifecycleRestoreStorageClass":""}}}]}

Is there something different in your setup that I might not have correct here?

I have two backends for testing. One listens over TCP, and the other over HTTP. I adjusted my backend route to match yours.
One file works without any issues. If I upload two or more at once, I always only get one event to my webhook.
It does work as soon as I only use one gw server...
Now sure how else to debug it. You can ping me directly too so not to litter up the thread here.

The route just needs to match whatever the webhook server is expecting. In my test case, my webhook server is only handling /s3events.  Would you be able to try the example webhook server that I pasted in above? It doesnt take any options. Just run go build and run the resulting executable.  It listens on port 8080 for the /s3events route.
When running both gateways, do you see both test events at the webhook server? For example, mine shows this message for each gateway that I run:
<IP_ADDRESS>:54314 {"Bucket":"Test-Bucket","Event":"s3:TestEvent","Service":"S3","Time":"2024-06-04T19:23:34Z"}

If there are any errors from versitygw sending to the webhook endpoint, they would be written to stderr. They would be something like one of these:
	fmt.Fprintf(os.Stderr, "failed to parse event data: %v\n", err.Error())
	fmt.Fprintf(os.Stderr, "failed to create webhook event request: %v\n", err.Error())
	fmt.Fprintf(os.Stderr, "failed to send webhook event: %v\n", err.Error())

I'm going to close this issue as not reproducible since the test webhook seems to be working as expected. If there is a webhook that fails, feel free to reopen this with more details of a failing case that we can try to reproduce. Since the gateway webhook events dont have any knowledge of a load balancer setup, I'm assuming the issue might be at the webhook side not able to accept events from multiple gateways at the same time.

I have over twenty app servers that listen for webhooks. Webhooks come in without an issue. It's just the Haproxy to versityGW setup that is causing an issue. I keep on experimenting, but so far have no success with multiple servers.

I'm going to close this issue as not reproducible since the test webhook seems to be working as expected. If there is a webhook that fails, feel free to reopen this with more details of a failing case that we can try to reproduce. Since the gateway webhook events dont have any knowledge of a load balancer setup, I'm assuming the issue might be at the webhook side not able to accept events from multiple gateways at the same time.

Found the issue. One server had an older version of versityGW, one that didn't include the webhook feature. Duh... Sorry about this.


Implement Youtube-DL "download in full" mode

I would appreciate having the ability to make this extension just download tracks in full using Youtube-DL, and then play them directly off the local disk. Reasons:

Youtube-DL is generally very fast to update when something is changed on Youtube's end, and I would prefer to depend on one parser, not two.
I've found seeking on long (multi-hour) videos to be unreliable with the existing implementation. I just had to restart mopidy twice for it to recover from seeking half an hour forward in a 4 hour video.
I suspect going forward we'll be playing a lot of these long ambience music style videos, and it would be good to not have to re-download them in full on a regular basis.

I wonder if this is better done outside of mopidy-youtube?

use youtube-DL to download the track you want,
add it to the mopidy library,
play it using mopidy-file or mopidy-local?

There's at least a couple of reasons why a roundabout solution may not be suitable:

Some users may not be technically savvy enough to perform all of those steps
As the person administering my household's multi-room audio system, I may not want to provide users with all of the access they'd need to perform those steps

I can guarantee you that if I told my wife she had to SSH into a server and run some commands just to play a music video from Youtube, she'd just do it some other way.

Okay - that is a persuasive argument - my partner would also just do it a different way. (And would be none too happy that I had tried it on with a "why don't you just SSH into the server...').  So, in the interests of happy partners, and a slightly less clogged up internet, try this: feature/cache
Config option required:
allow_cache = true

Feedback appreciated.

@djmattyg007 are you at all interested in having a look at this - no worries if not, but let me know and if you aren't, I'll close this issue.
Cheers.

I am, I just haven’t had a chance yet. I’m stuck in lockdown for at least another two weeks, so hopefully I’ll have time this weekend 👍

Thanks @djmattyg007 - yeah, lockdown (now with curfew) sucks. Will be interested to hear if it works for you (and any ideas you might get have).
Cheers

Oh, are you in Melbourne too?

Yep, sure am. WFH, home schooling, curfew, the whole box and dice.


Update error message to avoid ambiguity

EditForm requires a Model parameter, or an EditContext parameter, but not both.
When this error happened to me, I read it as "you can have one or the other, but not both". This made me think I had declared one of each but what I had actually done was declared neither.
I propose a new message:
EditForm requires a Model parameter or an EditContext parameter (but not both).
For me, adding the brackets around but not both makes it clearer that I have added neither Model nor EditContext and to fix the error I should add one of either a Model or EditContext, although I should not add both.

Thanks for your suggestion. We think the error message looks fine as it is.


keybase deprovision

There's still a failing test here, but the deprovision command itself is working.
Note to self: still need to test standlaone mode.

Can you add some more to the test?  Try logging in after deprovision?  Check that the device keys are revoked?  Also have two users (A and B), deprovision A, make sure B still functional.

:+1:

Thanks guys. @patrickxb yes, the test needs to expand a bunch.

:clap: :clap:

@patrickxb I extended the existing test, but I'd still like to add another test that does the thing you mentioned about having multiple users log in and switching between them. Could you remind me what code we have that already tests that sort of thing?


Map of all Geographical Places

Context
Many Geovistory projects have added a set of Geographical Places. Most of these Places have point coordinates and labels. A map, showing the points may give an impression of the geographical region of a project.
The following SPARQL query returns all places with label and coordinates (EPSG:4326).
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX ontome: <https://ontome.net/ontology/>

SELECT ?subject ?geoPlaceLabel ?long ?lat
    WHERE {?subject ^ontome:p147 ?presence.
  	?subject rdfs:label    ?geoPlaceLabel.
  	?presence ontome:p148 ?place.
   bind(replace(str(?place), '<http://www.opengis.net/def/crs/EPSG/0/4326>', "", "i") as ?rep)
   bind( replace( str(?rep), "^[^0-9\\.-]*([-]?[0-9\\.]+) .*$", "$1" ) as ?long )
   bind( replace( str(?rep), "^.* ([-]?[0-9\\.]+)[^0-9\\.]*$", "$1" ) as ?lat )
}

(To test, you can copy&paste the query here)
Requirement Spec
A web component displaying points on a map.
Interactivity:

click on point opens the URI of the place (above ?subject) in a new browser tab

Open Question

(how to) display the labels of geographical places?
how to limit data size? Show all? Add a bounding box to the query? Do a limit? (See geospatial query examples)

Technical Spec
Props:

sparql-endoint [string]: the URL of the sparql endpoint to query

@flicksolutions please review spec

to the questions:

I'd display POI-Icons and on click open a Title in a little bubble that is linked.
I'd use a bounding-box, yes. Also: The reply shouldn't be too heavy even when very many items are returned. For displaying the items, I would try clustering.

That's it for my review. spec seems solid.

@joschne sorry, something isn't quite clear:
What is the input of this component? Will it be a array of arrays? e.g. [["Label", lat, long], ...]?
And where will it be displayed - e.g. what's the environment it's in?

About limiting the queries:
The problem is, that we don't really have a starting point, like a center of the map, per project. Therefore we cannot fetch the closest points to that starting point.
This is why I think we should add an optional prop: center: [lat,long] - if it is not given, we take the center of Switzerland as the startingpoint.

@joschne sorry, something isn't quite clear: ~What is the input of this component? Will it be a array of arrays? e.g. [["Label", lat, long], ...]?~ (OK, no I got it, the input is just the endpoint and I will do the same query on different endpoints, right?) And where will it be displayed - e.g. what's the environment it's in?

Correct, the idea is that this query / component can be used with different endpoints having the same data structure, like e.g.:

https://sparql.geovistory.org/api_v1_project_84760
https://sparql.geovistory.org/api_v1_project_1483135
https://sparql.geovistory.org/api_v1_community_data

So the input is just the endpoint.
The component will be placed on pages like these...

https://www.geovistory.org/project/84760
https://www.geovistory.org/project/1483135

... but the general idea is to create standalone (embeddable) components that may be used on websites we don't (yet) know.

About limiting the queries: The problem is, that we don't really have a starting point, like a center of the map, per project. Therefore we cannot fetch the closest points to that starting point. This is why I think we should add an optional prop: center: [lat,long]|string - if it is not given, we take the center of Switzerland as the startingpoint.

The best strategy of limiting the number of places may depend on the use case. E.g. a project could have only a few places, spread across the whole globe. In this case any kind of limit is disturbing. Another project could have millions of places across the world and a bounding box would be helpful.
The one who uses the component should therefore be able configure the limit strategy.
Proposal:

There is an optional property limit (using SPARQL LIMIT clause). The default value of the limit is so that the map is still performant on a mobile device. This value has to be retrieved through empirical tests. The limit can't be undefined.
There is the optional prop for a center point (lat, long) and the zoomlevel that are used to render the initial map.
There is a boolean prop queryBoundingBox. If true, the query is constrained to the visible bounding box. If a user zooms and pans the map, the query is re-exectued.
If the limit is reached, the component displays a message 'maximum number of places reached'. If queryBoundingBox us true, it also says 'please zoom in'.


Adds JSON body parsing to Request handling

Adds JSON support to request body (particularly useful for processing POST requests).
Directly tested via flask handler.

This definitely solves a pain point, but I'm pretty keen on having this kind of stuff reviewed at this phase in the project.
There's a lot here that I need to think about in the longer term -- one of them is, how should Pale Contexts handle class methods..  The original idea behind the context is that it's a storage container, not a parser (and to be honest, my own build_args_from_request kind of violates this).
Also, there's no check here to validate that Content-Type is set to application/json before trying to decode it... What happens if someone sends application/x-www-form-urlencoded data in a request body?


Add support for await

We should be able to render something like {{ await myAsyncWork() }}. Currently, it throws an error saying await should be inside async. Even if we add async the value it will return will be a promise only but to print the content in HTML file we need that value to be the actual value instead of promise.

Hi.
Could you give an example of where in the code you're trying to implement this, and how exactly you're doing it when it fails? Thanks

Hi there, Yes I should've added steps to reproduce already.
Here are the steps to reproduce:

Create an example.abell file in any folder.
Add this content to that file.

<html>
  <body>
  {{ await (await fetch('https://reqres.in/api/users?page=2')).json() }}
   </body>
</html>

Currently, this will throw an error because in render when we execute this JS, we execute it on top level. This is the line of code that executes the code https://github.com/abelljs/abell-renderer/blob/master/src/execute.js#L60. As you can see we're assigning it to a variable called aBellSpecificVariable and return what is there inside that variable.
Instead the above snippet should return the json and not throw error.

Hi there, Yes I should've added steps to reproduce already.
Here are the steps to reproduce:

Create an example.abell file in any folder.
Add this content to that file.

<html>
  <body>
  {{ await (await fetch('https://reqres.in/api/users?page=2')).json() }}
   </body>
</html>

Run npx abell-renderer build --input example.abell

Currently, this will throw an error because in render when we execute this JS, we execute it on top level. This is the line of code that executes the code https://github.com/abelljs/abell-renderer/blob/master/src/execute.js#L60. As you can see we're assigning it to a variable called aBellSpecificVariable and return what is there inside that variable.
Instead the above snippet should return the json and not throw error.

Thanks. I'll take a proper look later today. I think I have an idea of what's going on, but I'd rather check first and be sure.

Hi there, Yes I should've added steps to reproduce already.
Here are the steps to reproduce:

Create an example.abell file in any folder.
Add this content to that file.

<html>
  <body>
  {{ await (await fetch('https://reqres.in/api/users?page=2')).json() }}
   </body>
</html>

Run npx abell-renderer build --input example.abell

Currently, this will throw an error because in render when we execute this JS, we execute it on top level. This is the line of code that executes the code https://github.com/abelljs/abell-renderer/blob/master/src/execute.js#L60. As you can see we're assigning it to a variable called aBellSpecificVariable and return what is there inside that variable.
Instead the above snippet should return the json and not throw error.

So I couldn't get your example to work.
First I got the error: missing ) after argument list
Then I realised I needed to both install node-fetch and then add the following line to the .abell file:
{{ const fetch = require("node-fetch") }}
and then run the command with --allow-require
and now I get the following error:
Cannot find module 'node-fetch'
This is my entire file
{{ const fetch = require("node-fetch") }}

<html>
    <body>
        {{ await (await fetch('https://reqres.in/api/users?page=2')).json() }}
    </body>
</html>

Any ideas why I can't get the example to exit with the correct error?

Oh yes you will have to npm install --save-dev node-fetch. With this, you will be able to require it in files.

I made this sandbox, https://codesandbox.io/s/abell-renderer-await-example-w6t9v?file=/src/index.abell
You can run npm run build in custom terminal there

I made this sandbox, https://codesandbox.io/s/abell-renderer-await-example-w6t9v?file=/src/index.abell
You can run npm run build in custom terminal there

Okay, so I spent a while having a real hack around with the code here.
I know exactly what's wrong, but implementing a solution is going to take a lot longer than I'd hoped.
So what's happening is that the vm is running the code just fine, but obviously it's just running everything synchronously. So at the point where you await the fetch request it's value is a promise (in a pending state).
To get the vm to wait until the request is completed before it returns a result requires some customisation as async/await isn't included out of the box for the vm.
Now, I will point out that this solution doesn't work - as in, it does not render the html as you'd like. The reason for that is that a proper refactor of the code is required to make it support async/await correctly.
But this 'hack' will let you see in the console that the vm implementation I've used does work, and hopefully can provide you with a basis on which to make the necessary changes.
Firstly in execute.js, I created a new async execute function. Within there we create a promise and once the VM has done its work, we consider the promised resolved. We can await that promise and only then return the result once fulfilled:
async function executeAsync(jsToExecute, sandbox= {}) {
  const numOfKeysBeforeExecution = Object.keys(sandbox).length;
  const codeToExecute =
    'aBellSpecificVariable = ' +
    jsToExecute.replace(/(?:const |let |var )/g, '');

  sandbox.value = "0";
  const result = await new Promise(resolve => {
    sandbox.resolve = resolve;
    const code = '(async() => await '+jsToExecute+'.then(async (result) => { value = await result.json(); resolve();}))()';
    const script = new vm.Script(code);
    const context = new vm.createContext(sandbox);
    script.runInContext(context);
  }).then(() => {
    // console.log(sandbox.value);
    return {
      type: 'value',
      value: sandbox.value,
      sandbox
    }
  });
  return result;
}

Then in index.js I implemented the following at line 76 to intercept when a user uses 'await' in their code, and make it redirect to my executeAsync method:
// Executes the expression value in the sandbox environment
      if (match[1].includes('await')) {
        (async () => { await executeAsync(match[1], sandbox).then((result) => {
          console.log(result.value);
        }); })();
      } else {
        const executionInfo = execute(match[1], sandbox);
        if (executionInfo.type === 'assignment') { ...

The console output will give you the actual result from the fetch.
What needs to be done from this stage is something akin to the following:

It would be easier to make all of execute() async and have it return the data in the same way so that the render method doesn't need to call two different Execute methods.
render() needs to be made async so that the html can be generated only once the response comes back.
generateHTMLFromAbell() probably also needs to be made async so that it can await the rendering of the html template before it writes it.
You will probably need to rethink execution time as well, as it will need to take into account the time taken for the await requests to come back.

Anyway, I'm sorry I couldn't help implement a full solution, but it did end up taking a lot longer than I anticipated. Hopefully though this puts you on track for a proper solution.

 Thank you so much 🎉 I will try out your snippet probably by tomorrow or the day after.
Again thank you so much!

 Thank you so much 🎉 I will try out your snippet probably by tomorrow or the day after.
Again thank you so much!

No problem. I'm glad I could help.

Node is also releasing support for top level await. That might be relevant here

https://twitter.com/codebytere/status/1293229801570881537

This is the next thing that I will pick up. Let me know if you have another idea other than the above discussion


Persist Configuration

Right now, the configuration is lost if you restart the Docker container. This needs to be addressed.

Will be addressed with backup/restore mechanism. You will map a local datastore using Docker's -v argument, or it will backup to the config folder if running locally. Backups will auto run every 24 hours, or by running the db_backup command:
python manage.py runscript db_backup
Job with id db_backup already in jobstore. I'll refresh it
Created backup: 20200521-152750.json

To restore, you will use the db_restore --script-args  command:
python manage.py runscript db_restore --script-args 20200521-152750.json
Installed 1903 object(s) from 1 fixture(s)


The Data Center param does not work correctly

I have a jdbc url like:
"jdbc:cassandra://<IP_ADDRESS>/test?loadbalancing=DCAwareRoundRobinPolicy(%22Test%20Cluster%22)";
'<IP_ADDRESS>' is a 5-node cluster with the name 'Test Cluster'.
I got such log info, which didn't connect the data center 'Test Cluster' correctly (using some default name).
16/09/08 15:03:07 WARN DCAwareRoundRobinPolicy: Some contact points don't match local data center. Local DC = Test Cluster. Non-conforming contact points:  <IP_ADDRESS>:9042 (datacenter1), <IP_ADDRESS>:9042 (datacenter1),ulpd326.madc.att.com/<IP_ADDRESS>:9042 (datacenter1), <IP_ADDRESS>:9042 (datacenter1)
Is this an issue or some thing I can work around?

Hi,
actually "Test Cluster" is the name of your cluster, while the DCAwareRoundRobinPolicy is expecting a datacenter name. Replace  DCAwareRoundRobinPolicy(%22Test%20Cluster%22) by DCAwareRoundRobinPolicy('datacenter1')" and you'll be ready to go.

Wow, response so fast. Thanks a lot.
I tried with the cluster with some other way (Spark 2.0 's SQLContext) and it works.  But I tried the jdbc way in Spark 2.0,  it has been unsuccessful:
The log showed the following, which seems to be incorrect somewhere.
16/09/08 15:17:46 INFO CassandraConnection: Connected to cluster: %s
16/09/08 15:17:46 INFO CassandraConnection: Datacenter: %s; Host: %s; Rack: %s
16/09/08 15:17:46 INFO CassandraConnection: Datacenter: %s; Host: %s; Rack: %s
16/09/08 15:17:46 INFO CassandraConnection: Datacenter: %s; Host: %s; Rack: %s
16/09/08 15:17:46 INFO CassandraConnection: Datacenter: %s; Host: %s; Rack: %s
16/09/08 15:17:46 INFO CassandraConnection: Datacenter: %s; Host: %s; Rack: %s
Exception in thread "main" java.sql.SQLTransientException: com.datastax.driver.core.exceptions.SyntaxError: line 1:29 no viable alternative at input '1' (SELECT * FROM sql_demo WHERE [1]...)
The program is just (Spark 2.0):
final Properties connectionProperties = new Properties();
//connectionProperties.put("user", MYSQL_USERNAME);
//put("password", MYSQL_PWD);
    final String dbTable
            = "sql_demo";

    //Load MySQL query result as Dataset
    Dataset<Row> jdbcDF
            = sparkSession.read()
            .jdbc(MYSQL_CONNECTION_URL, dbTable, "key", 0,10, 10, connectionProperties);

    List<Row> rows = jdbcDF.collectAsList();

    System.out.println(rows.size());

Hello, could you please help the issue? why the code like:
final String dbTable= "sql_demo";

        Dataset<Row> jdbcDF
                = sparkSession.read()
                .jdbc(SQL_CONNECTION_URL, dbTable, connectionProperties);

        List<Row> rows = jdbcDF.collectAsList();

generated the sql like the following log:
_16/09/20 13:16:35 INFO CassandraConnection  138: Datacenter: %s; Host: %s; Rack: %s
16/09/20 13:16:35 INFO CassandraConnection  138: Datacenter: %s; Host: %s; Rack: %s
16/09/20 13:16:35 TRACE CassandraPreparedStatement  98: CQL: SELECT * FROM sql_demo WHERE 1=0
16/09/20 13:16:35 TRACE RequestHandler  71: [19400322] com.datastax.driver.core.Statement$1@41ccb3b9_
How can I fix the problem? Your help is much appreciated.

More thinking: I think the Spark may want to get the schema by the sql generated, but Cassandra does not support the syntax 'where 1=0'.  Not sure how to get around of this.

Work around: Add 2 lines After Line 87 in  CassandraPreparedStatement.java:

       this.cql = cql.replace("WHERE 1=0", "limit 1");
       cql = this.cql;

Hi,
actually I would not recommend using the JDBC driver with Spark. The Cassandra-Spark connector maintained by Datastax will be much more efficient at doing this job, as the queries can be executed on colocated Cassandra/Spark nodes.
It now has a version that works with Spark 2.0 : https://github.com/datastax/spark-cassandra-connector/tree/v2.0.0-M3

Just let you know the issue is due to a Spark bug.  I filed it at here: https://issues.apache.org/jira/browse/SPARK-17614?filter=-2 and it will be fixed in  Spark 2.1


added brave browser

"added brave browser":

Added brave browser support
"fixed broken title in set-profile":
The results of the command chrome-profile, edge-profile, etc. where badly named e.g.: Profile 1, Default
Now they have the same name like in chrome/edge/brave
"nondefault profile bookmark-search fix":
Activating a profile had no affect on the bookmark-result. The reason was a piece of code in the bookmark-request which overwrote the saved ProfileName with "Default". - I deleted it. Therefore you can activate now a different profile and search through that other profile's bookmarks.

I hope the GUIDs are now to your liking.

@mdreizin can you review this please? :)

Is anyone looking into reviewing the PR?


Backports 0.20 pr8

Issue being fixed or feature implemented

Bitcoin Backports
What was done?

Misc v20 Backports, check commit log to see changes.
How Has This Been Tested?

Compilation and tested unit tests locally
Breaking Changes

Checklist:

[x] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have added or updated relevant unit/integration/functional/e2e tests
[ ] I have made corresponding changes to the documentation

For repository code-owners and collaborators only

[ ] I have assigned this pull request to a milestone

@PastaPastaPasta title needs to be updated to conventional commit format. backport:  prefix missing


Keyboard hiding the content until I type in something

I have the following content for my login page:
  <ion-content class="login">

      <button class="menu-button" ion-button menuToggle>
       <ion-icon name="menu"></ion-icon>
      </button>

      <ion-grid>
        <ion-row>
          <ion-col col-12 text-center>

                Incredible Weather

          </ion-col>
        </ion-row>
        <ion-row>
          <ion-col col-12 text-center>

                SignIn to access

          </ion-col>
        </ion-row>
      </ion-grid>

      <form #f="ngForm" (ngSubmit)="onSignIn(f)">
         <ion-grid>
          <ion-row>
            <ion-col col-10 offset-1>
              <ion-list>
                <ion-item>
                  <ion-label floating>Username</ion-label>
                  <ion-input type="text" ngModel name="Mail" required></ion-input>
                </ion-item>

                <ion-item>
                  <ion-label floating>Password</ion-label>
                  <ion-input type="password" ngModel name="Password" required></ion-input>
                </ion-item>
              </ion-list>
             </ion-col>
          </ion-row>

          <ion-row>
            <ion-col col-10 offset-1>
              <button ion-button block type="submit" [disabled]="!f.valid">Sign In</button>
             </ion-col>
          </ion-row>

        </ion-grid>
      </form>

Please sign up before loggin in.

  </ion-content>

Styling code:
page-signin {

    .scroll-content{
        background-image: linear-gradient(
            rgba(0, 0, 0, 0.45)
            ),url("../assets/imgs/weather.jpeg");
        background-repeat: no-repeat;
        background-attachment: fixed;
        background-size:cover;
        width:100%;
        padding-bottom: 0 !important;

    }

    .input-cover{
        position: static;
    }

    .button-full{
        position: fixed;
        bottom:0;
        left:0;
    }

    .menu-button{
        position: absolute;
        top:20px;
        left:25px;
        box-shadow: none;
        margin:0;
        padding: 0;
        background: transparent;

        ion-icon{
            font-size:2.9em;
            color:greenyellow
        }
    }

    .item-md,.item-ios{
        background: none !important;
        color:#fff;
        padding-left: 0;
    }

    .item-input .label-md{
        color:#fff;//text on label
    }

    .item-input.item-input-has-focus .label-md[floating], .input-has-focus .label-md[floating]{
        color:#fff;
    }

    .item-md.item-input.item-input-has-focus.item-inner,.item-md.item-input.item-input-has-focus .item-inner{
        border-bottom-color: #fff;
        box-shadow: inset 0 -1px 0 0 #fff;
        color:#fff;
    }

    .list-md .item-input.item-input-has-focus:last-child,.list-md .item-input.input-has-focus:last-child{
         border-bottom-color: #fff;
        box-shadow: inset 0 -1px 0 0 #fff;
    }

    .space{
        padding-top:100px;
    }

    h1{
        margin-top: 2rem;
    font-size: 4rem;
    margin-bottom: 3rem;
    color: greenyellow;
    }

    h2{
        margin-top:0;
        font-size: 3rem;
        color:#fff;
    }

}

Now when I tap on any field keyboard shows up but doesn’t align the input well with the keyboard; it instead hides it and then when I start to type it pushes the content a towards the top.
Attaching some images for clarity:
Initial Login screen:

When user taps on a field like username(hides the input):

When user starts typing(inputs moves a lot up):

It might be that I am not following proper ionic styling as this is one of my starter apps in ionic. It would be great if someone can point me in the right direction so that I can learn more and proceed.

Hello! Thank you for opening an issue with us!
Due to issues which are beyond our control (a lot of this has to do with the way the underlying web view chooses to do things), our general design advice is that screens be designed such that all inputs (or at least the inputs most likely to be entered first) be placed in the top 50% of the screen.
Thank you for using Ionic

I did think that as a workaround. But what if user has more than say 3 or 4 inputs on a page. What would you suggest in that case?

So I wrote something up to test this out and try things, but I don't actually get the same issue you are getting: https://github.com/kensodemann/test-input
Give that a try and let me know if the scrolling is working fine for you.
My Android testing device is a Note 4 running Android 6.0.1, though I also tried it on an emulator with a newer version of Android. In both cases, the input scrolled into view nicely.

Thanks a ton @kensodemann .
With your example I was able to come to a conclusion that ion-grid should not be used within the form as it caused the behavior of hiding the input behind the keyboard.
I removed the ion-grid and it works as in it works in your example.

Excellent!! I am going to close this then.


Update CEDS

Description
Checklist
Please confirm that this pull request has done the following:

[x] Documentation added (where applicable)
[x] Changelog item added to changelog/

Good to merge @durack1

Up to you whether you want to wait for any potential feedback on the docs from the CEDS team or not (we can always update in future)


suthesan/Edited title on homepage banner

Description
The title text on the homepage banner was changed
Changes:

The title text was changed from "Synthetic indices and stock indices" to "Synthetic and stock indices"

Type of change

[x] Bug fix
[ ] New feature
[ ] Refactor code
[ ] Update translation text
[ ] Breaking change
[ ] Dependency update
[ ] This change requires a documentation update
[ ] Release

I think it's better to make a PR based on the crowdin branch since we have some text changing :pray:


Format sync data for backend usage

Sync command on the backend is now running in a directed acyclic graph to produce the output of what needs to be created, updated, deleted. Therefore we need to modify the format of the data sent from the cli to the backend on sync / deploy.
See: https://app.shortcut.com/checkly/story/5525/migrate-be-to-the-new-graph-logic

@ndom91 do we have a sample of the data format?

@ianaya89 you knwo what, I think this issue is actually not necessary anymore. Umut mentioned he wanted to make it backward compatible which I didn't know at the time. But in general, yes it would send something like this:
request body
{
    "checks": {
        "example-api.yml": {
            "activated": true,
            "alertSettings": {
                "escalationType": "RUN_BASED",
                "reminders": {
                    "amount": 0,
                    "interval": 5
                },
                "runBasedEscalation": {
                    "failedRunThreshold": 1
                },
                "sslCertificates": {
                    "alertThreshold": 30,
                    "enabled": false
                },
                "timeBasedEscalation": {
                    "minutesFailingThreshold": 5
                }
            },
            "checkType": "API",
            "degradedResponseTime": 10000,
            "doubleCheck": true,
            "frequency": 10,
            "groupId": null,
            "groupOrder": null,
            "localSetupScript": null,
            "localTearDownScript": null,
            "locations": [
                "us-east-1",
                "eu-west-1"
            ],
            "logicalId": "example-api.yml",
            "maxResponseTime": 20000,
            "muted": false,
            "name": "API Check",
            "request": {
                "assertions": [],
                "body": "",
                "bodyType": "NONE",
                "headers": [],
                "method": "GET",
                "queryParameters": [],
                "url": "https://google.com"
            },
            "runtimeId": null,
            "settings": {},
            "setupSnippetId": null,
            "tearDownSnippetId": null,
            "useGlobalAlertSettings": true
        }
    },
    "groups": {},
    "projectId": 44
}

response body
[
    {
        "type": "groups",
        "typeResult": [
            {
                "action": "create",
                "results": []
            },
            {
                "action": "update",
                "results": []
            },
            {
                "action": "delete",
                "results": []
            }
        ]
    },
    {
        "type": "checks",
        "typeResult": [
            {
                "action": "create",
                "results": []
            },
            {
                "action": "update",
                "results": [
                    {
                        "logicalId": "example-api.yml",
                        "physicalId": "9a5e5dc0-c7a5-41df-b5de-df59277c1cc3",
                        "type": "checks"
                    }
                ]
            },
            {
                "action": "delete",
                "results": []
            }
        ]
    }
]

Nice, will close this for now.


Add new ini  to the install folder

Hi, could you maybe add an option to the installer that he would check if there is a new ini file in the installer folder. This is very useful if you deploy rdpwrap via group policy and it has no access to the internet.
Ok you could add some other group policy to update the ini file in the program files rdp wrapper folder but it isn't very elegant.

i would love to know how to deploy this using the GPO.

computer configuration
policies
windows settings
scripts
startup
\domainserver\xxxxx\rdpwinst.exe
don't forget the parameter -i

To update the ini file i used another gpo.
computer configuration,
preferences,
windows settings,
files,
Here i added the file, action = Replace, source file = \domainserver\xxxxx\rdpwrap.ini, destination file = %programfiles%\RDP Wrapper\rdpwrap.ini


[BUG] Recompilation error: "invalid value for type 'layout'. Expected a reference"

Information

Apktool Version (apktool -version) - 2.5.1
Operating System (Mac, Linux, Windows) - Ubuntu 20
APK From? (Playstore, ROM, Other) - apkpure.com

Stacktrace/Logcat
bro@local:\~/ig-apk$ **apktool d ig-196-apkpure.apk**
...
...
...
bro@local:~/ig-apk$ **apktool b --use-aapt2 -o out.apk ig-196-apkpure**
I: Using Apktool 2.5.1-eaab72-SNAPSHOT
I: Checking whether sources has changed...
I: Checking whether sources has changed...
I: Checking whether sources has changed...
I: Checking whether sources has changed...
I: Checking whether sources has changed...
I: Checking whether sources has changed...
I: Checking whether resources has changed...
I: Building resources...
W: /home/bro/ig-apk/ig-196-apkpure/res/values-v22/layouts.xml:12: error: invalid value for type 'layout'. Expected a reference.
W: /home/bro/ig-apk/ig-196-apkpure/res/values-v22/layouts.xml:13: error: invalid value for type 'layout'. Expected a reference.
W: /home/bro/ig-apk/ig-196-apkpure/res/values-v22/layouts.xml:14: error: invalid value for type 'layout'. Expected a reference.
W: /home/bro/ig-apk/ig-196-apkpure/res/values-v22/layouts.xml:15: error: invalid value for type 'layout'. Expected a reference.
W: /home/bro/ig-apk/ig-196-apkpure/res/values-v22/layouts.xml:16: error: invalid value for type 'layout'. Expected a reference.
W: /home/bro/ig-apk/ig-196-apkpure/res/values-v22/layouts.xml:17: error: invalid value for type 'layout'. Expected a reference.
W: /home/bro/ig-apk/ig-196-apkpure/res/values-v22/layouts.xml:18: error: invalid value for type 'layout'. Expected a reference.
W: /home/bro/ig-apk/ig-196-apkpure/res/values-v22/layouts.xml:19: error: invalid value for type 'layout'. Expected a reference.
W: /home/bro/ig-apk/ig-196-apkpure/res/values-v22/layouts.xml:20: error: invalid value for type 'layout'. Expected a reference.
W: /home/bro/ig-apk/ig-196-apkpure/res/values-v22/layouts.xml:21: error: invalid value for type 'layout'. Expected a reference.
W: /home/bro/ig-apk/ig-196-apkpure/res/values-v22/layouts.xml:22: error: invalid value for type 'layout'. Expected a reference.
W: /home/bro/ig-apk/ig-196-apkpure/res/values-v22/layouts.xml:23: error: invalid value for type 'layout'. Expected a reference.
W: /home/bro/ig-apk/ig-196-apkpure/res/values-v22/layouts.xml:24: error: invalid value for type 'layout'. Expected a reference.
W: /home/bro/ig-apk/ig-196-apkpure/res/values-v22/layouts.xml:25: error: invalid value for type 'layout'. Expected a reference.
W: /home/bro/ig-apk/ig-196-apkpure/res/values-v22/layouts.xml:26: error: invalid value for type 'layout'. Expected a reference.
W: /home/bro/ig-apk/ig-196-apkpure/res/values-v22/layouts.xml:27: error: invalid value for type 'layout'. Expected a reference.
W: /home/bro/ig-apk/ig-196-apkpure/res/values-v22/layouts.xml:28: error: invalid value for type 'layout'. Expected a reference.
W: /home/bro/ig-apk/ig-196-apkpure/res/values-v22/layouts.xml:29: error: invalid value for type 'layout'. Expected a reference.
W: /home/bro/ig-apk/ig-196-apkpure/res/values-v22/layouts.xml:30: error: invalid value for type 'layout'. Expected a reference.
W: /home/bro/ig-apk/ig-196-apkpure/res/values-v22/layouts.xml:31: error: invalid value for type 'layout'. Expected a reference.
brut.androlib.AndrolibException: brut.common.BrutException: could not exec (exit code = 1): [/tmp/brut_util_Jar_77580703989672702302554212405921978723.tmp, compile, --dir, /home/bro/ig-apk/ig-196-apkpure/res, --legacy, -o, /home/bro/ig-apk/ig-196-apkpure/build/resources.zip]

Sample of layouts.xml:
...
<item type="layout" name="guide_grid_row">L|46188|5C4|BCF2</item>
<item type="layout" name="guide_ufi_bar">L|4674C|F90|E5CC</item>
<item type="layout" name="gumstick">L|476DC|A50|AC5A</item>
...
Steps to Reproduce

apktool d ig-196-apkpure.apk
apktool b --use-aapt2 -o out.apk ig-196-apkpure

Frameworks
If this APK is from an OEM ROM (Samsung, HTC, LG). Please attach framework files
(.apks that live in /system/framework or /system/priv-app)
APK
armeabi-v7a / Android 5.0+
Questions to ask before submission

Have you tried apktool d, apktool b without changing anything? yes
Are you using the latest apktool version? yes

I replicate. Logging some info to help for debugging this later.
➜  2618 aapt d resources Instagram_v<IP_ADDRESS>.126_apkpure.com.apk --values | grep 'guide_grid_row'
      spec resource 0x7f0c04bc com.instagram.android:layout/guide_grid_row: flags=0x00000400
        resource 0x7f0c04bc com.instagram.android:layout/guide_grid_row: t=0x03 d=0x00002781 (s=0x0008 r=0x00)
        resource 0x7f0c04bc com.instagram.android:layout/guide_grid_row: t=0x03 d=0x00002174 (s=0x0008 r=0x00)

➜  Instagram_v<IP_ADDRESS>.126_apkpure.com grep -r -i 'guide_grid_row' *
Binary file build/resources.zip matches
res/values-v22/layouts.xml:    <item type="layout" name="guide_grid_row">L|46188|5C4|BCF2</item>
res/values/layouts.xml:    <item type="layout" name="guide_grid_row">L|2AE8F8|590|E594</item>
res/values/public.xml:    <public type="layout" name="guide_grid_row" id="0x7f0c04bc" />

Can confirm having the same issue on M1 Mac and Windows. Recompiling the Instagram app.

Any update on this issue?
Is there any workaround?

Hey there, just had the same error and steps how I fixed it in my case:

Run 'apktool empty-framework-dir' command before re-building an app
Run apktool b -o out.apk ig-196-apkpure (it seems that issue was with --use-aapt2 flag in my case)

it seems that issue was with --use-aapt2 flag

I do confirm, it fixed the latest whatsapp beta compilation errors.

not work for me

from what ive found
the file is decompiled correctly and those values are used to retrieve the layout resource from instagram's proprietary resource format (assets/layouts.bin.xz)
source: method X.C1Mj.getLayout(int) and X.C24601Mp.A00()
from instagram apk version 3<IP_ADDRESS>.6 alpha (371500513)
the resource format seems to contain axml blocks too
source: method X.C1NA.C1NA(Resources, File)

I thought about this a bit. Clearly Instagram is doing some trickey here with storing references to their odd layouts.bin.xz file. aapt2 screams at this and crashes compilation.
https://github.com/search?q=repo%3AiBotPeaches%2Fplatform_frameworks_base "invalid value for type"&type=code
We could easily patch out this check and see how the application reacts. It just seems dangerous for Apktool as this is a good build-time check that is only becoming a problem for apps like Instagram.

@iBotPeaches  Could this be done on purpose to prevent people from reverse engineering their way out of their recent mandatory dialog box to accept being tracked for ads?
The unskippable dialog box has been driving me crazy for weeks so I spent these last few days learning about apktool and Smali, and of course now it blocks compilation at the end.
You know I don't care if instagram steals and sells all my data but I'm not giving consent.
I think the resources to run this dialogBox get downloaded after logging in, and triggered in the second "onCreate" lifecycle of Instagram. Meaning the first time I log in on a clean Instagram install I'll be fine until I close the app.
Is there perhaps some other way to bypass this behaviour?
Any suggestions are highly appreciated.

I'm able to compile using --use-aapt1 but for some reason the momment I login the app crashes

First I tried to sign the original one (just to see if instagram plays any trickery) and worked just fine. Then I did the same for just the recompiled apk and (as mentioned above) althought I was able to open the app, the momment I logged-in it crashed... So I'm really skeptical as to if this is a problem with apktool itselft or instagram does some trickery there too.

Any updates on this bug? Has anyone found a workaround ?

i also got same error + 2 more (it's start on newest updated apk's only, on old apk version still works)
I: Building resources...
W: C:\apktool\mik\res\values-land\layouts.xml:3: error: invalid value for type 'layout'. Expected a reference.
W: C:\apktool\mik\res\values-land\layouts.xml:4: error: invalid value for type 'layout'. Expected a reference.
W: C:\apktool\mik\res\values-land\layouts.xml:5: error: invalid value for type 'layout'. Expected a reference.
W: C:\apktool\mik\res\values-land\layouts.xml:6: error: invalid value for type 'layout'. Expected a reference.
W: C:\apktool\mik\res\values-land\layouts.xml: error: file failed to compile.
W: C:\apktool\mik\res\values-ldrtl\layouts.xml:3: error: invalid value for type 'layout'. Expected a reference.
W: C:\apktool\mik\res\values-ldrtl\layouts.xml: error: file failed to compile.
W: C:\apktool\mik\res\values-night\mipmaps.xml:3: error: invalid value for type 'mipmap'. Expected a reference.
W: C:\apktool\mik\res\values-night\mipmaps.xml: error: file failed to compile.
W: C:\apktool\mik\res\values-sw600dp\layouts.xml:3: error: invalid value for type 'layout'. Expected a reference.
W: C:\apktool\mik\res\values-sw600dp\layouts.xml:4: error: invalid value for type 'layout'. Expected a reference.
W: C:\apktool\mik\res\values-sw600dp\layouts.xml: error: file failed to compile.
W: C:\apktool\mik\res\values-v26\layouts.xml:3: error: invalid value for type 'layout'. Expected a reference.
W: C:\apktool\mik\res\values-v26\layouts.xml:4: error: invalid value for type 'layout'. Expected a reference.
W: C:\apktool\mik\res\values-v26\layouts.xml: error: file failed to compile.
W: C:\apktool\mik\res\values-watch\layouts.xml:3: error: invalid value for type 'layout'. Expected a reference.
W: C:\apktool\mik\res\values-watch\layouts.xml:4: error: invalid value for type 'layout'. Expected a reference.
W: C:\apktool\mik\res\values-watch\layouts.xml: error: file failed to compile.
W: C:\apktool\mik\res\values\animators.xml:3: error: invalid value for type 'animator'. Expected a reference.
W: C:\apktool\mik\res\values\animators.xml:4: error: invalid value for type 'animator'. Expected a reference.
brut.androlib.exceptions.AndrolibException: brut.common.BrutException: could not exec (exit code = 1): [C:\Users\Username\AppData\Local\Temp\brut_util_Jar_160069859907321700514368744111369315017.tmp, compile, --dir, C:\apktool\mik\res, --legacy, -o, C:\apktool\mik\build\resources.zip]

I want to ask if this problem has been solved?

I would like to ask if this problem has been solved. I have also encountered

I would like to ask if this problem has been solved. I have also encountered

Same question, because I'm also having the same issue while recompiling Instagram apk

I also encounter this issue. Andy idea how to solve it?

any one found solution to this problem ?
i have tried "Delete framework\1.apk" and disable aapt2 , but did not work

✅ It worked, thank you
1 - I deleted the 1.apk From "C:\Users{username}\AppData\Local\apktool\framework\1.apk" .
2 - I installed version 2.7.0 :
https://bitbucket.org/iBotPeaches/apktool/downloads/
3 - Now run the command: apktool b <Your-File>

any one found solution to this problem ?
i have tried "Delete framework\1.apk" and disable aapt2 , but did not work
#EDIT 1:

also i tried using apktool V 2.10 but also did not Work

#EDIT 2: [Worked for me]

Download and use apktool V2.7.0.
Make sure to delete the file : "C:\Users{username}\AppData\Local\apktool\framework\1.apk" before compile (once).
Don't use aapt2.

✅ It worked, thank you @mikeanjelo
1 - I deleted the 1.apk From "C:\Users{username}\AppData\Local\apktool\framework\1.apk" .
2 - I installed version 2.7.0 :
https://bitbucket.org/iBotPeaches/apktool/downloads/
3 - Now run the command: apktool b <Your-File>

@Alawsi What is the solution for m1 mac?

definitely delete old framework file and use aapt1 works for build but app will crashes before login. Probably instagram added some new things...

#EDIT 2: [Worked for me]

Download and use apktool V2.7.0.
Make sure to delete the file : "C:\Users{username}\AppData\Local\apktool\framework\1.apk" before compile (once).
Don't use aapt2.

it's definitely work for latest alpha bases on instafel too


Version Packages

This PR was opened by the Changesets release GitHub action. When you're ready to do a release, you can merge this and the packages will be published to npm automatically. If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated.
Releases
<EMAIL_ADDRESS>Patch Changes

Updated dependencies [9b79d9b64]:

<EMAIL_ADDRESS>

<EMAIL_ADDRESS>Patch Changes

Set the __session cookie with samesite:none for secure iframes only (#1403) by @nikosdouvlis

Updated dependencies [137322862, 741012544, 0b55cdf27]:

<EMAIL_ADDRESS>

<EMAIL_ADDRESS>Patch Changes

Updated dependencies [9b79d9b64]:

<EMAIL_ADDRESS>

<EMAIL_ADDRESS>Patch Changes

Add missing fr-FR translations for reset password page (#1398) by @kohort-aymeric

Make resend link/code message clearer. (#1390) by @desiprisg

Add missing pt-BR translations (#1388) by @Gustavo-Pauli

<EMAIL_ADDRESS>Patch Changes

Add isApiRoute to AfterAuthHandler['auth'] type (#1397) by @dimkl

This PR has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.


Create Walkthrough video

A walkthrough video that demonstrates the functionality of the social media API must be submitted, and a link to the video should be included in your README file.

[x] The walkthrough video must show all of the technical acceptance criteria being met.

Satisfies all of the preceding acceptance criteria plus the following:
[ ] Uses the Mongoose packageLinks to an external site to connect to a MongoDB database.
[ ] Includes User and Thought models outlined in the Challenge instructions.
[ ] Includes schema settings for User and Thought models as outlined in the Challenge instructions.
[ ] Includes Reactions as the reaction field's subdocument schema in the Thought model.
[ ] Uses functionality to format queried timestamps properly.

[ ] The walkthrough video must demonstrate how to start the application’s server.
[ ] The walkthrough video must demonstrate GET routes for all users and all thoughts being tested in Insomnia.
[ ] The walkthrough video must demonstrate GET routes for a single user and a single thought being tested in Insomnia.
[ ] The walkthrough video must demonstrate POST, PUT, and DELETE routes for users and thoughts being tested in Insomnia.
[ ] Walkthrough video must demonstrate POST and DELETE routes for a user’s friend list being tested in Insomnia.
[ ] Walkthrough video must demonstrate POST and DELETE routes for reactions to thoughts being tested in Insomnia.

All items have been completed


Got 404 error from get_block_info

Hello,
I got 404 error when transferring token to others.
The error is "Invalid HTTP response (404) for https://eos.greymass.com/v1/chain/get_block_info."
The same code works under Jungle 3 Test Net (https://jungle3.cryptolions.io:443).
May I know how to fix it?
Cheers,

Do you mean using get_block to get refBlockNum and refBlockPrefix first, and then set those two values to the transaction before signing?

Yes, the code will determine that they are already set and will not call getBlockInfo (which calls get_block_info on the chain).  The SDK exposes get_block as public func getBlock(requestParameters: EosioRpcBlockRequest, completion: @escaping (EosioResult<EosioRpcBlockResponse, EosioError>) -> Void) so you can use that or the PromiseKit wrapped version to get the information you need.

I noticed that I need to pass a blockNumOrId in when calling this getBlock. May I know how to get this id please?

You'll have to get the last irreversible block from a getInfoBase() call.  The code here can give you an idea:
https://github.com/EOSIO/eosio-swift/blob/43bd15ce5abcdf2f8028f6484f677e98caf4c88d/Sources/EosioSwift/EosioTransaction/EosioTransaction.swift#L390

Thanks. I'll give it a try.

@jke661s did you get this working? Can you share any details on what worked for you?

Having the same problem

Version 1.0.0 of the SDKs is meant to align with release 2.1 of EOS.  Earlier releases of the SDK can always be used for earlier node versions.  When I get a chance I will try to post a code snippet that shows how to work around get_block_info not being present and post it here.

I have the same issue, luckily, the old version is still there and that one does not use the unsupported get_block_info. So this setup still works fine.
pod "EosioSwift", "~> 0.4" # pod for this library
pod "EosioSwiftAbieosSerializationProvider", "~> 0.4" # serialization provider
pod "EosioSwiftVaultSignatureProvider", "~> 0.4" # pod for this library
Nevertheless, I would expect that the API of this library reflects the stable version of the EOS API documentation. As of now there, there is no get_block_info method stated there.
https://developers.eos.io/manuals/eos/latest/nodeos/plugins/chain_api_plugin/api-reference/index#operation/get_account
I will wait until the chains are updated or the newer version supports the stable chain API's.
Thanks!

Hey @opi-smccoole, I fixed the issue and implemented simple versioning on my local branch, unfortunately, I cannot push it and open PR as I am getting:
remote: Permission to EOSIO/eosio-swift.git denied to CyrilCermak.
fatal: unable to access 'https://github.com/EOSIO/eosio-swift.git/': The requested URL returned error: 403

How can I open a PR? From your contribution readme, I understood that it should be by pushing a new branch.
Thanks!

Hi @opi-smccoole, I had a look at it and here is a PR that would give the option to fall back to an older version that does not support the new functionality.
https://github.com/EOSIO/eosio-swift/pull/296


Redact password from inspection of the config object. Resolves #28

This is easier than trying to filter it whenever it is logged, although may not be the cleanest solution.

Looks clean to me, that's a nice solution! thanks! :+1: :+1:
The new AMQP gem requires Ruby 2.0+, before merging I will have to do some checks to see how we can interop, or should we downgrade aqmp.

I am merging this in. We'll major bump Sneakers version and support Ruby from 2.0 in the future.


TLS-PSK

Title: Add TLS-PSK to Envoy
Description:
Envoy can be used as a TLS proxy for unsecure services, however currently supports TLS cert-based auth. TLS-PSK is a faster alternative that can be used for services running on low latency, low powered resources

@PiotrSikora if you have any comments on supporting TLS-PSK

hi @ggreenway
could we add in one of the suggested labels to this FR to revivie it?
for ref, stunnel supports  it and here's a trivial standalone example in c
thanks

@salrashid123 only if someone is planning to work on it. This hasn't seen any activity in years. Are you planning to submit a PR?


Add steel to projects using vulkano.

https://github.com/vulkano-rs/vulkano/issues/1555
Adds steel to Projects using Vulkano.

Can you make the list in alphabetical order?

Can you make the list in alphabetical order?

I just sorted the whole table alphabetically!

Thanks!


did:cheqd - Error while trying to receive credential

Hey everyone,
Does anyone know why when receiving a credential the error “Error occured while trying to receive credential: An operation is not implemented: Not yet implemented” occurs when I select a did:cheqd ? The error occurs here: https://wallet-dev.walt.id and on my self-hosted Walt.id instance.
Steps to reproduce:

Create an account at https://wallet-dev.walt.id/login
Create a did:cheqd (testnet)
Create a credential under: https://portal.walt.id/
Store the credential in the wallet under the did:cheqd

(After having seen that there will be no support on Discord after 1.11. I've created this issue here again)

Hi @c0ball !
Could you, please, test again? It seems to be working now.

Happy holidays!
Ale

Hi @alegomes,
which version of Walt.id are you using? The hosted one, or do you self-host?
Because with the hosted one available under https://wallet.walt.id I still receive this error.
Thanks and happy holidays :)

I've seen that a new version has been published yesterday, but also on my self-hosted instance using version 0.10.0 I do receive the same error.

One thing I notice in the screenshot is that in your credential, the issuer uses a did:cheqd. In my case, however, the recipient, i.e. the wallet owner, should use a did:cheqd. Can you confirm that this scenario works for you?

Many thanks in advance

@alegomes If you can confirm that you are also experiencing the error, then I would ask you to open the issue again

Oh, the recipient. You're right, @c0ball . I do have the same issue. It's already been included in our internal backlog. Thanks for letting us know.


test: fix git test temporary directory for macOS as symlink

@N-o-Z check if it is also the case where you use the local command on non temp link directory.


MousePassThrough option for windows version

-mousepassthrough parameter doesn't work in windows

Unfortunately, I don't have Windows machines to test the problem.

This is what I get tho, Might that be helpful

hi @crgimenes I have a Windows machine set up for testing, I'd be happy to help fix this. can you assign to me?

hi @crgimenes I have a Windows machine set up for testing, I'd be happy to help fix this. can you assign to me?

Thanks for the help! I believe the problem is with Ebitengine, but I'm not sure, I didn't find any issues about it.

yes correct, there no issue in my windows machine with latest version go.

Yay :D, so where to update or get the binary?


Always reposition window

This PR will allow the window to always reposition itself whenever it is toggled. First reported by @brpaz (#119) with usage sample in dual monitors.

Coverage remained the same at 83.129% when pulling c833e4c35b1aacc2127cebdb9759dd88bb8b8044 on repos-window into cb2ca5afe6e4daae6879b036b668dbeb8c7a2359 on develop.


feat: slider widgets cypress test cases

Description

Add cypress test cases for the slider widgets.

Fixes #17466
Type of change

New feature (non-breaking change which adds functionality)

How Has This Been Tested?

Ran all the cypress test cases locally.

Checklist:

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[x] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes

Working fine, moving to done.

/ok-to-test sha=7f266e0

/ok-to-test sha=0c282b0

/ok-to-test sha=3f2a8a0

/perf-test

/ok-to-test sha=83975a2

/ok-to-test sha=2234974


upgrade(installer): Make installer extract tmp dir owned by dd-agent

What does this PR do?
The installer tmp extract dir (/opt/datadog-installer/tmp) is owned by root which prevents the installer upgrade ran by the daemon. This PR makes that dir owned by dd-agent at installer bootstrap, so both the bootstrapper & the daemon can write in it.
Motivation
Working installer upgrades
Additional Notes

Possible Drawbacks / Trade-offs

Describe how to test/QA your changes

/merge


How to use MotionDirector to train a LoRA based on other base models?

Thanks for such amazing work！
I want to use MotionDirector to train LoRa based on Stable Video Diffusion or ModelScope_v1.5.
Is this easy to implement? Do I need to rewrite the code in utils to insert lora?
ps. The ModelScope_v1.5 I mentioned is https://modelscope.cn/models/iic/dreamvideo-t2v/summary.
Thanks for your reply!

Yes, you need to rewrite to adapt to their code base.


Allowable IPs

Add manual allowable IP list and IP autodetection by external request (for example if aserver behind nat with port forward only).

Closed by 3c3bd3eb7515130808739fe500587eb7ff638312


Game Creation: user input should be validated

The system should validate entered by an organizer when creating a new game:
It is possible to enter text instead of a number when setting the maximum number of rounds and the game still shows up as active.
It is possible to enter "30:00" as termination time. The system shows "Game sucessfully created", but the game doesn't show up.

It is also possible to create games in MatrixSelect mode where the number of features to be selected is greater than the number of shown features.

has been addressed in #122


V3 Microsoft Dependency Injection

Hi there,
Great work to getting to v3.
I'm trying to upgrade a console app from v2 to v3 but I'm having difficulty when using dependency injection.
I'm getting the following error:
Exception Message:   No service for type 'Foo.Commands.InfoCommand' has been registered. Exception Source:    Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService - System.Object GetRequiredService(System.IServiceProvider, System.Type) Runtime terminating: True
It seems I need to register the Commands now is that right?

I solved it, just realised I need to add
.AddTransient<InfoCommand>()
Should I be using scope or transient here?

It seems I need to register the Commands now is that right?

Yes, you should register the commands now.  You can use the appRunner.GetCommandClassTypes extension method to make that easier.

Should I be using scope or transient here?

Disclaimer: I'm not very familiar with the MS container, just enough to implement Resolve and TryResolve :)
From how I know transient to work in other containers, it should be fine since we will only resolve the command class once.  There wouldn't be multiple instances of it.

Let me know if there are any other upgrade issues.  We're also active on Discord for any questions or comments.

@drewburlingame Cheers, thanks for confirming. Just need to work out how MS container can loop through and register these. I don't think it natively does this might need to use a nuget package

@dotnetshadow would this do the trick?
var commandClassTypes = new AppRunner<App>().GetCommandClassTypes();
foreach (var type in commandClassTypes)
{
	serviceCollection.AddTransient(type);
}

@dotnetshadow can we close this issue?


[BUG]: (Problem installing Solaris on google Colab)

I am trying to install Solaris on Google Colab, but some problems appears when I install the dependences.
I install Conda first as the documentation says and also rtree and gdal, but at the end, when I do the pip install the result says that "InvalidVersionSpec: Invalid version '4.19.112+': empty version component"
%%bash
MINICONDA_INSTALLER_SCRIPT=Miniconda3-4.5.4-Linux-x86_64.sh
MINICONDA_PREFIX=/usr/local
wget https://repo.continuum.io/miniconda/$MINICONDA_INSTALLER_SCRIPT
chmod +x $MINICONDA_INSTALLER_SCRIPT
./$MINICONDA_INSTALLER_SCRIPT -b -f -p $MINICONDA_PREFIX
%%bash
conda install --channel defaults conda python=3.6 --yes
conda update --channel defaults --all --yes
!conda install -c conda-forge rtree gdal=2.4.1
!pip install solaris
I tried also installing the gdal>=3.0.2 as in the requirements before install Solaris but the problem remains.
!conda install -c conda-forge gdal=3.0.2
I haven`t been able to install Solaris yet because all of this dependences problems. Hope anyone can help! Thanks.

having same problem of installing in colab here. But my case is with gdal and installation instructions seem not to work

Anyone manage to get solaris running on google colab?

@Chris-Has I failed installing in colab runtime, but I just manually included solaris library in a folder and it seems to work pretty well. I don't remember where I got this pre-built version, but here it is. https://github.com/canoneod/Extracting-Building-Footprints-from-Satellite-imagery/tree/main/libs


Support disable_presym for mrbgems/mruby-errno

The mrbgems/mruby-errno/src/known_errors_def.cstub file has been extended, so the mrbgems/mruby-errno/src/known_errors_e2c.cstub file is no longer needed.

Thank you for the patch. I am still wondering if I should remove disable_presym though.

I do not object to disable_presym being removed.
I was wondering while working on this if it was really necessary to make it disable_presym compatible.
However, since disable_presym exists and is included in stdlib.gembox, I just decided that this action is necessary.

Reasonable. Let me think for a while to determine if we should remove disable_presym.


[1.12] Settings UI Nits

discovered in the 1.12 bug bash on 9/28
A collection of SUI nits that we should do for 1.12.

[x]  This setting should probably be something like "Startup behavior", rather than an incomplete sentence

Veto'd, because

[x] [Actions page] The 'S' in the "Open Settings" action is unnecessarily capitalized
[x] "Clear Buffer" Should be a lowercase b on buffer
[x] "Background Opacity" above the slider should have a lowercase "o"
[x]  Unspecified error when adding "opacity": 0.7 to JSON
[ ] This padding's gotta go:
[ ] [Appearance tab of a profile] "Reset to inherited value" button doesn't show up for the background opacity slider after moving the slider (unless you hit 'Save' after making the change)
[ ] [Appearance tab of a profile] "Reset to inherited value" button for the background opacity slider shows up despite hitting 'Discard changes' after moving the slider

Both the sliders are a little weird. Unsure why that is right now. Presumably there's a similar root cause?
I'm hoping that they go away when we figure out #11355


Standardize formatting of manifests/c/ChefSoftware/Workstation/21.7.524.1

Microsoft Reviewers: Open in CodeFlow

This package installs and launches normally in a Windows 10 VM.


Possible bug with referring to other components

Environment
System:

OS: Linux 5.4 Ubuntu 18.04.5 LTS (Bionic Beaver)
CPU: (4) x64 Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz
Memory: 449.05 MB / 15.52 GB
Container: Yes
Shell: 4.4.20 - /bin/bash

Binaries:

Node: 14.4.0 - ~/.nvm/versions/node/v14.4.0/bin/node
npm: 6.14.5 - ~/.nvm/versions/node/v14.4.0/bin/npm

npmPackages:

styled-components: ^5.1.1 => 5.1.1

Reproduction
Hover of child does not apply to parent:
I have a div which has an svg as first child. On hover of the svg I want the parent div to have a border top.
I have attached a screenshot of my own IDE how I tried:

Steps to reproduce
https://codesandbox.io/s/styled-components-dsz24
Expected Behavior
Change the style of the Nav div on hover of the Scissor svg.
Actual Behavior
Nothing happens.

If I'm reading this screenshot right then you have a selector that says ${Scissor}:hover & on Nav but your DOM structure has Nav wrapping Scissor.
This means that your hover rule is reversed and won't apply, since Scissor is a child of Nav but the selector targets the opposite relationship.

Yep. You're right. After reading through many stackoverflow issues I now realise that this is not possible through simple CSS (as the name states: it's Cascading Style Sheets), so it will not go up in selectors.
As it's not an issue with styled components I'm closing this.
Thank you very much for your answer!


change FunctionalComponentOptions to a generic interface

so as to specify props and slots()

Thanks! My concern is typing update is a breaking change for TS users. (I've been bitten by this before).
Should vue's semver also take this into consideration? Ping @ktsn @yyx990803

hm, as the typings is shipped with core, it should also follow the semver?

To avoid breaking change, how about rename the generic interface and create a specific interface that compatible current type? like:
export interface RenderContextG<P, S> {
  props: P;
  children: VNode[];
  slots: S;
  data: VNodeData;
  parent: Vue;
}

export interface RenderContext extends RenderContext<any, any> {}

It was a huge breaking change the changes of ComponentOptions that I did. However, compared to ComponentOptions, it will be a smaller impact to change FunctionalComponentOptions and RenderContext, I think...

I think ComponentOptions update was reasonable breaking change because it was not announced officially about typings and needed revisions for blush up as the stage of pre-release.
However, it is now already released and announced, and it seems many users depends on it.
It may be different problem between code and typings but I think we should be careful about this topic.

Then, how about changing only slot: any -> slot(): any?
I'm gonna make pull request again when preparing 2.1...

It's ok to me.


Add Scoop installation for Windows

dua is available in Scoop for Windows.

Oh, that's cool, thanks a lot for contributing!
(I wonder if there is a way to run it properly, in my attempt the interactive mode was quite botched in an msysgit terminal.)

It works fine for me in the Windows Terminal and the Conhost. However, I cannot interact with anything in the Mintty (MSYS2 and Git Bash).


The correct record is not yet found by the local resolver.

The issue originally reported in #1385 seems to have come back again, I'm using 2.1.18 - is there an option to bypass the local DNS validation and use the public DNS only or is this bug back? I tested with 2.1.7 which has a similar message, and gave an option to try the ACME validation anyway. It seems like when I use 2.1.5 the validation goes through without any delay.

Thanks for reporting your issue. There's a couple of things to unpack here.

When preliminary validation fails for the manual DNS plugin, you are shown a menu with options to retry the check, ignore the warning and continue, or abort. Did you not get this menu?
The program implements a recursive algorithm to find the authoritative servers. You can configure which DNS servers to use as a starting point in the settings file, but it will always follow through to other servers to get the most accurate (uncached) records.
You can disable this check alltogether by setting Validation.PrevalidateDns to false in the settings file.
I'd like to investigate if there is some kind of regression but you'd need to provide full --verbose logs from both the 2.1.5 and 2.1.18 runs so that I can compare what's happening in each of them.

I tested this out with v.2.1.20 and had the same issues, I did create a verbose result but would prefer to not post all the info here publicly.

You can also share it privately with me at<EMAIL_ADDRESS>

So I've analysed the logs and interestingly v2.1.20 discovers three nameservers for the domain (ns1, ns2, and ns3), whereas v2.1.5 only discovers two of them (ns1 and ns2). Pre-validation fails on the ns3, which is working when queried directly, but looks like it's somehow not updating (fast enough).
If I use nslookup to query Google (<IP_ADDRESS>), Cloudflare (<IP_ADDRESS>) or my own local provider about it, only the first two name servers are found. But various whois sources agree that there are three of them, and querying the root servers also agrees that there are three.
So the question is, how do Google and Cloudflare and others (including Let's Encrypt, supposedly) figure out that ns3 is something that lags behind and shouldn't be take all that seriously? I don't know yet, input from anyone would be appreciated.

So a temporarily work around of this issue would be to remove ns3, I can test that later this week when doing another round of manual certificates

Give it a try making sure that there were only 2 NS listed but still getting the same errors, I'll email the log report for review, thanks.


how to calculate how many users currently viewing my streams how to calculate?[Q]

@lelegard
tsp -v -m --synchronous-log -I ip -r $INPUT_URL
-P svrename -i 10
-P scrambler --ignore-scrambled --cw<PHONE_NUMBER>10016 --partial-scrambling 7 10
-P filter --pid 0x0014 --pid 0x0010 --pid 0x0001 --negate
-P pmt --increment-version --add-ca-descriptor "0x4b19/0x03e8"
-P analyze -i 30 -o report.txt
-O ip --ttl 2 -l $OUTPUT_INTERFACE $OUTPUT_URL \
i am streaming udp to udp live streams,
i want to calculate how many users currently viewing my streams(output udp) how to calculate please give me?

Hi @udhayNuk

i want to calculate how many users currently viewing my streams(output udp)

This is impossible by design of the UDP protocol.
If you use a unicast IP address, the answer is 0 or 1. If you use a multicast address, the answer is something between zero and the size of the Internet.

@lelegard thanking you


Modifier keys sometimes get stuck

Sometimes I'm not able to rename treeviews with the Enter or F2 key. If I'm not mistaken the up and down arrow keys are also broken when that happens.

I think this is a classic case of keyboard events not firing when the control key is released. Tons of browsers and operating systems suffer from issues like these. https://crbug.com/1224123 is one example.
I just ran into this issue when trying to drag an object with G. I wasn't able to use the shortcut until I pressed Ctrl once.


Score-CAM when the output function is sigmoid

Hello, I am trying to visualize the last convolutional layer of my model, which have a sigmoid function as the last layer. In this case, when the masked array is inputted, should we use softmax function to the output of sigmoid function and take dot with activation map (as this calculation results in all one vector)? Otherwise, should we just use the output of sigmoid function, to multiply with activation map?
Sincerely,

Hi, @tropidophis, is your model a binary classification model? I think it should be fine to use the output score from the sigmoid function directly, as the output has been normalized into a fixed range.


0.22.0 prefixes all over the place

I just downloaded the x64 windows zip version and replaced all files in C:\Program Files\PicoTorrent that had the previous version. When I opened up PT everything in main menu had "amp_" prefix and all statuses in the status column had "state_" prefix. When I manually selected english language in the preferences and restarted PT, all was good. Looks like there is default case missing in language switch select somewhere in the code :)

Hi! Did you also place the coredb.sqlite file in the same folder? That file contains all translations :)

yes

Odd. Is there any reason why you use the zip file and not the installer? The zip-file works best in a folder that you have write permissions on since it needs to create some files, such as the settings database and such.
Can you try to run the application from a folder with write permissions?

Odd. Is there any reason why you use the zip file and not the installer?

this was the first time, i used it because it is faster than installer

the zip-file works best in a folder that you have write permissions

i know, i gave permissions when overwriting old files
maybe it was just a fluke, who knows. no biggie. but there still should be some default case if language cannot be found.

Same here when I select pt language, no problems in English.
Also, folder contains write permissions. The necessary files are created without a problem.
I'm using the .zip version too (aka portable), much more practical for my needs (btw thanks for providing it! :)).

@nero-0 Can you check the log file? It should contain some information about language matching.

Here's the lastest log if you want to take a look
PicoTorrent.20201123060146.log

An update regarding this:
I noticed that it's not exactly a bug, but a bad behavior. I mean, if the selected language is not fully translated, instead of the untranslated space being taken up by the default language (English, I suppose), a key is put in place, e.g. amp_console.
I have no more problems here using pt-BR. Maybe OP uses pt-PT, which is not fully translated atm.
I also want to say that there is no difference indicating which is pt-BR or pt-PT, both are listed only as ''Portuguese''.
I think it would be a good idea to make it explicit. :p

Thanks for the info @nero-0! There is an issue in how we select the English locale which is mentioned in #1020. I'd say fixing this would resolve both of these.


Check lock in TC when use @GlobalLock

Why you need it?
For now if you want to use @GlobalLock, you must use it with @Transactional, and check the global lock when transaction commit.
But it will cause two problems:

Can not check global lock when use @GlobalLock Independently.
Check global lock can not retry.

If you want to fix it, please comment "Please assign it to me" is this issue.
How it could be?
Other related information

回复第二点，不只有GlobalLock+SelectForUpdate才会没有重试；在GlobalTransaction下，如果分支事务加了@Transactional注解，也重试不了。
Reply to the second point, not only GlobalLock+SelectForUpdate will not retry; under GlobalTransaction, if the branch transaction is annotated with @Transactional, it will not be retried.

Please assign it to me


Open source development - Bitbucket

Added resource for open source development
IMPORTANT: When creating your Pull Request, please use the following format for the title: Category - Resource Name & Section
PR Title samples:

Web development - CSS Portal & Tools
Web development - Leetcode & Challenges

Fixes Issue

closes #262
Description:

Added resource for open source development
Resource URL:

https://www.bitbucket.org
Checklist:
Before submitting your pull request, ensure that you have completed the following tasks:

[x ] I have carefully reviewed and adhered to the contributing guidelines before creating this pull request.
[x ] I followed the prescribed PR title template. (Check this if you're adding a resource)

@jfmartinz Carried out the changes.

@jfmartinz Carried out the changes.

I said the Open Source Technologies not in the Tutorial or Course. Please check, thanks


updating and moving to archived area

log-search-transition.md

@rboucher : Thanks for your contribution! The author(s) have been notified to review your proposed change.


fix: remove blank space between words when use pdf-loader

Extends #3218

I had a PDF that seemed to need this - see the test case.
There may be some better way to parse the loader output?

Ww could make it a config option?

i see, i always use pdf-loader to load chinese pdfs, so this question is very typical
i add a skipBlank option in pdf-loader for needed

Thanks for being persistent, will merge #3306 instead to allow potentially for other types of separators.


⚠ Bio Links has degraded performance

In e197044, Bio Links (https://bio.rheinklang.events/links) experienced degraded performance:

HTTP code: 200
Response time: 5271 ms

Resolved: Bio Links performance has improved in 9311909 after 1 hour, 45 minutes.


Ldap Auth support for managerDn, managerPassword, userSearchFilter

Support for additional ldap auth properties

ldapUserSearchFilter : The LDAP filter used to search for users (optional). For example "(&(objectClass=user)(sAMAccountName={0}))". The substituted parameter is the user's login name. This property can be substituted for ldapUserDnPattern
ldapManagerDn : Username (DN) of the "manager" user identity (i.e. "uid=admin,ou=system") which will be used to authenticate to a (non-embedded) LDAP server. If omitted, anonymous access will be used.
ldapManagerPassword : The password for the manager DN. This is required if the ldapManagerDn is specified.

fyi @rvema @Sbrenthughes

@sidbha007 Thanks a lot for your PR, We just migrated the api layer to its own repo. Can you please submit the PR against the new repo please? https://github.com/Hygieia/api

sure @rvema created PR https://github.com/Hygieia/api/pull/7
will close this


Scripts vs ERB files

I prefer to keep ERB files separate from scripts so that shellcheck can do its magic.
What are your thoughts for making tmux available to the shell profile like this?
It's subtle, but referencing job packages instead of global packages is a good pattern.

Thanks for the suggestions, @gerhard! I think you're proposing three separate changes:

Refer exclusively to job-specific packages.
Reorganize and rename the template files to keep the BOSH lifecycle scripts separate from the other files, as well as to
Make the profile.d script a pointer to a job template file that performs the environmental setup, instead of doing it inline.

Point 1 makes sense for sure, and I'll push out that change shortly. For point 2, I can see the benefits of that organizational separation, as well as separating ERB content from any complicated scripting logic. Technically, there is no non-trivial ERB content in the release so far, and BOSH always renders all the templates regardless of whether they have an .erb extension, so I think I'll remove the extension from all of the files in the templates directory.
For point 3, I'm not sure I see the value of the indirection in this case, since this release is currently so simple. If it gets to the point where several different scripts would need the same environmental setup, as seems to the case in the RabbitMQ release, I think then it would justify the indirection. Could you explain why you think it would be beneficial in this case?
Thanks,
Eric

Actually, I was thinking that you might want to write the file directly. pre-start would just become:
#!/usr/bin/env bash

cat > /etc/profile.d/add_tmux_to_path.sh <<EOF
if [[ "\$PATH" != *"/var/vcap/jobs/tmux/packages/tmux/bin"* ]]; then
  export PATH="/var/vcap/jobs/tmux/packages/tmux/bin:\$PATH"
fi
EOF

As for /var/vcap/jobs/*/env, I wish it became a BOSH standard since it makes it easier to reason about job-specific environment variables. I see this as stepping stone towards just-in-timing variables, via CredHub or similar. I'm sure @cppforlife has other/more ideas on the matter.

I'm happy to conclude this with 1. & 2.
Point 3. has more nuance and feels more of a preference than good practice.
Thanks!

Thanks again, @gerhard ! I published v0.1.0 earlier today with those changes.

👍


Remove deprecated option and sort volumes alphabetically

allowedUnsafeSysctls seems to be deprecated for some time now and is automaticly removed when adding the SCC to an Openshift cluster.
The volumes list may cause issues with a default GitOps (ArgoCD) installation as the list was not alphabetically sorted.

What type of PR is this?

Uncomment one (or more) /kind <> lines:

/kind bug

/kind cleanup

/kind design

/kind documentation

/kind failing-test

/kind feature

If this PR will release a new chart version please make sure to also uncomment the following line:

/kind chart-release

Any specific area of the project related to this PR?

Uncomment one (or more) /area <> lines:

/area falco-chart

/area falco-exporter-chart

/area falcosidekick-chart

/area event-generator-chart

What this PR does / why we need it:
Which issue(s) this PR fixes:

Fixes #
Special notes for your reviewer:
Checklist

[x] Chart Version bumped

Adding label do-not-merge/contains-merge-commits because PR contains merge commits, which are not allowed in this repository.
Use git rebase to reapply your commits on top of the target branch. Detailed instructions for doing so can be found here.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Hi @Hakulaku, could you please rebase on main?

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/falcosecurity/community.
/lifecycle stale

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/falcosecurity/community.
/lifecycle rotten

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.
Provide feedback via https://github.com/falcosecurity/community.
/close

@poiana: Closed this PR.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.
Provide feedback via https://github.com/falcosecurity/community.
/close

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.


(BOLT-1227) Enumerate OpenSSH configuration options available with bolt

This commit adds a list of configuration options that the net-ssh gem will respect (and wont be overridden by bolt).

Continuation of https://github.com/puppetlabs/bolt/pull/973#discussion_r279819964
I think this is the complete list. Basically i just dug through https://github.com/net-ssh/net-ssh/blob/master/lib/net/ssh/config.rb and found the options that were not overriden or ignored in how we net-ssh with bolt.


Update barrier and reduce_scatter_base to conform to PyTorch signatures

[x] Fixes #2543
[x] Adds missing op arg to reduce_scatter_base
[x] Adds support for PyTorch's monitored barrier (https://github.com/pytorch/pytorch/blob/07dd2fe6c32948e5ca0a2871e5eb31602a9684cf/torch/distributed/distributed_c10d.py#L3231)

@jeffra and @awan-10 -- This should be ready to merge now as long as all tests pass.

@jeffra -- I think the auto-merge won't go in due to an unrelated error in the nv-inference unit tests?


Support Qwen2-MoE Architecture

Qwen1.5-MoE-A2.7B-Chat is an open-sourced 14B MoE model, based on Qwen2-MoE architecture. It is possible to run on mobile devices.
Note that we need to support multi-device TP for the arch for future large models. However, I fail to do that because of limited devices.
cc @vinx13 @tqchen

@DiegoCao can you help to followup and add TP support?

depends on https://github.com/apache/tvm/pull/16848, waiting for the next sync

Just rebased mlc-ai/relax. Let's trigger the CI tomorrow.

Got it, working on the TP support

Waiting for dependencies: https://github.com/apache/tvm/pull/16887 and https://github.com/apache/tvm/pull/16886
BTW, there is a known numerical issue on Vulkan. Will fix it in a follow-up PR.

@Hzfengsy seems alll deps are landed, let us followup