Whitelisting not working with AWS (deis 2.5.0)

App specific whitelisting doesn't work for me with deis 2.5.0 unless I specify <IP_ADDRESS>/0
Using my ip or CIDR including /32, /24, /16 doesn't work.
I went back to another cluster running 2.4.2, and it looks like the problem is also there deis-router level whitelist with 2.4.2.
router level whitelisting was working in 2.1, so maybe this is a regression?
Chatted about it with @kmala in Slack. Channel is a bit busy right now, so thought to open an issue to track.

I followed the instructions here: https://github.com/deis/workflow/blob/master/src/managing-workflow/security-considerations.md#ip-whitelist
Adding CIDRs with Kubectl or deis results in 403 Forbidden when trying to access the app from an IP inside the CIDR (except <IP_ADDRESS>/0)

did you try setting the IP using X-Forwarded-For header.

No, how would I do that?

how are you testing whether it works or not......if you are using curl to test it out just add  -H "X-Forwarded-For: <ip>"

That still gets a 403 forbidden

@dmcnaught if you wouldn't mind describing how you provisioned your platform, the example application used that is failing (if using one, if not would you mind deploying one to test against?) how you set up using deis whitelist:list and the curl command you are using to test against your app, that would be most helpful for us to reproduce your issue and identify a fix.

My assumption is:

AWS
https://github.com/deis/workflow/blob/master/src/managing-workflow/security-considerations.md#ip-whitelist
example-go
curl -H "X-Forwarded-For: my.external.ip.address" go.mydomain.com

@bacongobbler Sure - we are currently using kube-up with some extra subnets for the minion ASG.
I've been testing with an internal app, but I'll deploy example-go and test with that now.
I did run the curl command as you describe. I'm a little confused though - since won't the AWS ELB be sending on the request with an X_Forwarded-For already?

ok: reproduced with example-go. Adding my ip to the whitelist results in 403 Forbidden.

@dmcnaught did you enable proxy protocol on the aws elb and deis router(https://github.com/deis/router#annotations).

Thanks for the tip - looks like I should do that.
When I do it though - I lost all connectivity to deis and the apps that are running:
kubectl --namespace=deis annotate deployments/deis-router router.deis.io/nginx.useProxyProtocol=true --overwrite
Only by reversing to false could I reach deis and the apps again:
--- kubernetes/deis ‹master› » kubectl --namespace=deis annotate deployments/deis-router router.deis.io/nginx.useProxyProtocol=true --overwrite
deployment "deis-router" annotated
--- kubernetes/deis ‹master› » deis apps:list
=== Apps
example-go
wellbot
welltok-arch
--- kubernetes/deis ‹master› » deis apps:list
Error: Unknown Error (504):
--- kubernetes/deis ‹master› » deis apps:list                                                                               1 ↵
Error: Unknown Error (504):
--- kubernetes/deis ‹master› » kubectl --namespace=deis annotate deployments/deis-router router.deis.io/nginx.useProxyProtocol=false --overwrite
deployment "deis-router" annotated
--- kubernetes/deis ‹master› » deis apps:list
Error: Unknown Error (504):
--- kubernetes/deis ‹master› » deis apps:list                                                                               1 ↵
Error: Unknown Error (504):
--- kubernetes/deis ‹master› » deis apps:list                                                                               1 ↵
Error: Unknown Error (504):
--- kubernetes/deis ‹master› » deis apps:list                                                                               1 ↵
Error: Unknown Error (504):
--- kubernetes/deis ‹master› » deis apps:list                                                                               1 ↵
Error: Unknown Error (504):
--- kubernetes/deis ‹master› » deis apps:list                                                                               1 ↵
=== Apps
example-go
wellbot
welltok-arch

did you enable the proxy protocol on the aws elb https://deis.com/docs/workflow/managing-workflow/configuring-load-balancers/#configuring-proxy-protocol

No. Trying that:
I ran ``kubectl --namespace=deis annotate deployments/deis-router router.deis.io/nginx.useProxyProtocol=true --overwriteandkubectl --namespace=deis annotate service/deis-router service.beta.kubernetes.io/aws-load-balancer-proxy-protocol=''and got the same issue: I loose connectivity to deis and deis apps. I left the config:kubectl --namespace=deis annotate service/deis-router service.beta.kubernetes.io/aws-load-balancer-proxy-protocol=''and rankubectl --namespace=deis annotate deployments/deis-router router.deis.io/nginx.useProxyProtocol=false --overwrite` to fix.

And to confirm you are on a kubernetes cluster that is at least v1.3.4, correct?

which version of kubernetes are you on?

1.3.5:
Client Version: version.Info{Major:"1", Minor:"3", GitVersion:"v1.3.5", GitCommit:"b0deb2eb8f4037421077f77cb163dbb4c0a2a9f5", GitTreeState:"clean", BuildDate:"2016-08-11T20:29:08Z", GoVersion:"go1.6.2", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"3", GitVersion:"v1.3.5", GitCommit:"b0deb2eb8f4037421077f77cb163dbb4c0a2a9f5", GitTreeState:"clean", BuildDate:"2016-08-11T20:21:58Z", GoVersion:"go1.6.2", Compiler:"gc", Platform:"linux/amd64"}```

the only other thing i did to make it work is add proxyrealipcidrs kubectl --namespace=deis annotate deployments/deis-router router.deis.io/nginx.proxyRealIpCidrs="<IP_ADDRESS>/8,<IP_ADDRESS>/16" --overwrite where <IP_ADDRESS>/8 is the CIDR for the k8s internal network and <IP_ADDRESS>/16 is the default aws vpc cidr which kube-up uses.

@kmala That command you sent an hour ago has got things working for me, and I didn't even do the kubectl --namespace=deis annotate deployments/deis-router router.deis.io/nginx.useProxyProtocol=true --overwrite ...

In fact kubectl --namespace=deis annotate deployments/deis-router router.deis.io/nginx.useProxyProtocol=true --overwrite still breaks my deis router and apps...
Thanks for the fix. I guess we should add to the docs.

In fact kubectl --namespace=deis annotate deployments/deis-router router.deis.io/nginx.useProxyProtocol=true --overwrite still breaks my deis router and apps...

@dmcnaught Proxy protocol support requires deis-router v2.6.2 or later to work properly. Did you give your ELB some time to apply the new configuration?
If you run aws elb describe-load-balancers --load-balancer-name <your-lb-name> and look at the output should list the k8s-proxyprotocol-enabled policy on the instance ports:
"BackendServerDescriptions": [
    {
        "InstancePort": 30523,
        "PolicyNames": [
            "k8s-proxyprotocol-enabled"
        ]
    },
    {
        "InstancePort": 31166,
        "PolicyNames": [
            "k8s-proxyprotocol-enabled"
        ]
    },
    {
        "InstancePort": 31252,
        "PolicyNames": [
            "k8s-proxyprotocol-enabled"
        ]
    },
    {
        "InstancePort": 32157,
        "PolicyNames": [
            "k8s-proxyprotocol-enabled"
        ]
    }
],

Unless both your deis-router and ELB are speaking PROXY protocol your connections will be broken.

Thanks @felixbuenemann - I do see that on the elb currently:
"BackendServerDescriptions": [
                {
                    "InstancePort": 30767,
                    "PolicyNames": [
                        "k8s-proxyprotocol-enabled"
                    ]
                },
                {
                    "InstancePort": 32133,
                    "PolicyNames": [
                        "k8s-proxyprotocol-enabled"
                    ]
                },
                {
                    "InstancePort": 32301,
                    "PolicyNames": [
                        "k8s-proxyprotocol-enabled"
                    ]
                },
                {
                    "InstancePort": 32446,
                    "PolicyNames": [
                        "k8s-proxyprotocol-enabled"
                    ]
                }
            ],

When I do a ku edit deployments deis-router
I see
router.deis.io/nginx.useProxyProtocol: "false"

I'll look into this a bit more when I upgrade another of our clusters to 2.5 and apply these configs.

@dmcnaught So you need to enable proxy protocol on your deis router using kubectl --namespace=deis annotate deployments/deis-router router.deis.io/nginx.useProxyProtocol=true --overwrite or it won't work. What happens now is that the ELB is adding a PROXY protocol header, but because it is not enabled on the router, it will ignore the header and will not understand the content of the packets, so your connections will fail.
Btw. you don't need to upgrade your whole cluster to deis workflow 2.6.0, switching the deis-router to the 2.6.2 image will work fine on a 2.5.0 cluster (and probably older).
You can edit the existing deployment with kubectl edit --namespace deis deployment/deis-router and change the line saying image: quay.io/deis/router:v2.5.0 (or whatever version) to the 2.6.2 image: image: quay.io/deis/router:v2.6.2.
If you run into any problem you can always revert to the old image by reverting the change.

I'll look into this a bit more when I upgrade another of our clusters to 2.5 and apply these configs.
Deis Workflow 2.5 is too old, you need 2.6.

Depending on how you VPC is configured you might also need to follow @kmala's advise and customise you router.deis.io/nginx.proxyRealIpCidrs to include your VPC private network CIDR.

Aaah - I kind of skipped over the 2.6.2 comment a little. You mean the reason it's not working for me, is that is doesn't work with earlier versions of the deis-router than 2.6.2...
Doing these two steps with 2.5.0 seemed to get things working...:
kubectl.sh --namespace=deis annotate service/deis-router service.beta.kubernetes.io/aws-load-balancer-proxy-protocol='*' --overwrite
kubectl.sh --namespace=deis annotate deployments/deis-router router.deis.io/nginx.proxyRealIpCidrs="${NON_MASQUERADE_CIDR},${KUBE_VPC_CIDR_BASE}.0.0/16" --overwrite

HTTP/HTTPS will work fine on deis-router < 2.6.2, but pushing new releases to the builder will fail, unless you are running 2.6.2, because of deis/router#263.

I just upgraded my second cluster from 2.4.2 to 2.5.0.
After upgrading I ran
kubectl.sh --namespace=deis annotate service/deis-router service.beta.kubernetes.io/aws-load-balancer-proxy-protocol='*' --overwrite
kubectl.sh --namespace=deis annotate deployments/deis-router router.deis.io/nginx.proxyRealIpCidrs="${NON_MASQUERADE_CIDR},${KUBE_VPC_CIDR_BASE}.0.0/16" --overwrite

and re-associated my CNAME, added my cert to the ELB.
My test site was working, but not whitelisting when I switched to a non whitelisted IP (as my first cluster is).
I ran
kubectl --namespace=deis annotate deployments/deis-router router.deis.io/nginx.useProxyProtocol=true --overwrite

and my cluster became unreachable. I left it for 30 minutes, and after that deis login and apps were still unreachable. I ran
kubectl --namespace=deis annotate deployments/deis-router router.deis.io/nginx.useProxyProtocol=false --overwrite

and within a minute or so my cluster and apps were reachable again.
... but no whitelisting.
I then ran
kubectl --namespace=deis annotate deployments/deis-router router.deis.io/nginx.enforceWhitelists=true

to see if that would help, but no difference.
This make things a little more confusing.
My first upgraded cluster is returning a nice 403 if I try to access my app from a non whitelisted IP...

@dmcnaught when you say your cluster is becoming unreachabe when you enable useProxyProtocol=true, did you check the logs of the router?

@dmcnaught You wrote you added your cert to the ELB. Are you running your ELB in HTTP/HTTPS mode instead of TCP? If you are you should be using X-Forwarded-For instead of proxy protocol. The default setup uses TCP mode on the ELB and terminates http / https and ssh on the deis router, so your cert would be on the router not on the ELB.

This is what I get when I try to login:
[2016-10-03T21:48:38+00:00] - deis/deis-controller - <myip> - - - 401 - "GET /v2/ HTTP/1.1" - 435 - "-" - "Deis Client v2.5.1" - "~^deis\x5C.(?<domain>.+)$" - <IP_ADDRESS>:80 - deis.deisCName - 0.002 - 0.002
2016/10/03 21:48:38 [error] 148#0: *3564 broken header: "POST /v2/auth/login/ HTTP/1.1
host: <deis.deisCName>
Accept-Encoding: gzip
Content-Type: application/json
User-Agent: Deis Client v2.5.1
X-Forwarded-For: <myip>
X-Forwarded-Port: 443
X-Forwarded-Proto: https
Content-Length: 60
Connection: keep-alive

" while reading PROXY protocol, client: <IP_ADDRESS>, server: <IP_ADDRESS>:8080

and this whenI try to access the app:
2016/10/03 21:48:17 [error] 149#0: *3543 broken header: "GET / HTTP/1.1
host: welltok-arch.deisCName
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Encoding: gzip, deflate, sdch, br
Accept-Language: en-US,en;q=0.8
Cache-Control: max-age=0
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.116 Safari/537.36
X-Forwarded-For: <myip>
X-Forwarded-Port: 443
X-Forwarded-Proto: https
Connection: keep-alive

" while reading PROXY protocol, client: <IP_ADDRESS>, server: <IP_ADDRESS>:8080

I'm also getting this message a lot (before and after useProxyProtocol=true
2016-10-03 21:48:22.635500 I | WARNING: Field "router.deis.io/whitelist" value "" does not satisfy constraint /^((([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/([0-9]|[1-2][0-9]|3[0-2]))?(\s*,\s*)?)+$/ -- skipping this field and using default value "[]".

@felixbuenemann Yes - I'm terminating ssl on the ELB

@felixbuenemann How should I set X-Forwarded-For instead of proxy protocol?

Well, terminating SSL on the ELB is a bad idea, because it breaks ALPN and as such HTTP/2 and your apps cannot distinguish between HTTPS and HTTP connection.
You can theoretically configure port 80 on the ELB to HTTP, 443 to HTTPS and 2222 to TCP. In this case the ELB will add X-Forwarded-For, X-Forwarded-Proto headers, but you are pretty much on your own with the configuration and doing this through Kubernetes annotations likely requires K8s 1.4 (see kubernetes/kubernetes#26268). You also loose HTTP/2 in this case.
If you want to save youself a lot of trouble, go with the default TCP mode ELB config and shell out some money for a non-acm wildcard ssl cert that you can configure on the deis router.
Once you've moved SSL termination to the Deis Router and upgraded to 2.6 proxy protocol and whitelisting should both work fine.
Btw. The warning you posted above is fixed in workflow 2.6.0.

So - no plans to support whitelisting with SSL terminating on the ELB? 😞

@dmcnaught Technically you should be able to get it working, but it will require quite some fiddling. If you configure port 443 on the ELB to SSL mode and forward it to the HTTP port on the Deis router and forward 80 and 2222 in TCP mode you can use proxy portocol on all those ports which should get whitelisting working.
This is what you could try:

Upgrade your cluster to K8s 1.4
Annotate you deis-router service with:

service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:…
service.beta.kubernetes.io/aws-load-balancer-ssl-ports: https
service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: '*'

Annotate deis-router deployment with:

router.deis.io/nginx.useProxyProtocol: "true"

Edit the ports section of the deis-router deployment so 443 on the ELB is mapped to 8080 instead of 6443 on the deis router pod:

spec:
  ports:
  - containerPort: 8080
    hostPort: 80
    protocol: TCP
  - containerPort: 8080
    hostPort: 443
    protocol: TCP
  - containerPort: 2222
    hostPort: 2222
    protocol: TCP
  - containerPort: 9090
    hostPort: 9090
    protocol: TCP

In theory the above should work (I can't try 'cause I'm still on K8s 1.3), but you will completely bypass deis' own certificate handling and the ability for your apps to differentiate between http and https.
I don't think this will ever be an officially supported configuration, because this limits you to a single certificate on the ELB and all app specific domains have to be added as SAN entries on the same cert, while deis supports different certificates for each app in addition to the default wildcard platform cert.

ok - we're still on 1.3 too, so I'll have to wait to try this until we go to 1.4
I'm still confused about why the whitelisting is working on one of my 1.3.5 clusters 😄  (and not the other)...

With regard to your most recent comment - if we put a wildcard cert on the ELB, there's no need to any other domains to be added (as SAN entries) - all apps can be served as SSL through the ELB by the deis-router in the wildcarded domain. It really is a great solution IMHO if you don't need ALPN/HTTP/2 and no desire to pay for a wildcard cert.
I do have all the settings you specify above in 1.3 - except router.deis.io/nginx.useProxyProtocol: "true" as this causes my whole deis cluster to become unreachable (I set the cert and ELB port on the ELB directly rather than with annotations). One cluster has whitelisting working fine, and on the other whitelisting doesn't work... I've done a diff of the settings across the two clusters and I haven't found any differences...

If you manually configure the ELB you risk loosing your modifications when kubernetes updates the ELB. I think this could be the reason why enabling proxy protocol on the router isn't working for you. Because if it was properly enabled on the ELB but disabled on the router, all connections should fail, but you are reporting the opposite behavior.

I don't quite follow @felixbuenemann - I re-run the aws script that adds the cert to the ELB if a new ELB is created, or it loses the cert for another reason.

ping, has this been resolved?

We have a 1.4 cluster up here now, so I can test - I'll put it on my todo list. Thanks

With Kubernetes 1.5.2 I can add an application specific whitelist with deis whitelist:add <IP> -a test, no other commands were needed. deis-router is being fronted by an AWS ELB with a wildcard cert on it.

Whitelist can be removed with deis whitelist:remove <IP> -a test - if there is only one IP in the whitelist, otherwise it just updates the list...

Oh - one other change - When I add the cert to the ELB, I delete the listeners on port 443 and port 80, and create a new listener on 443 pointing to the instanceport that 80 was connecting to..

I think we can consider this one closed then. Let's open separate tickets if there are other action items we need to fix/implement. Thanks!


Unable to gen model

Always keep getting "selected element is not instance of UMLPackage". (even when it's explicitly a UMLPackage)

Even I'm facing the same problem

+1

@yilikun @Pradyumna97 @mondrus Could you additional details to help me reproduce the behavior?


Use new resnet test infra for GS

Problem description
GS currently use the old test infra, which has fold on host
What's changed
change to new test infra allows use device fold.
Checklist

[x] Post commit CI https://github.com/tenstorrent/tt-metal/actions/runs/10621748751
[ ] Model perf https://github.com/tenstorrent/tt-metal/actions/runs/10621002817/job/29442506873
[ ] device perf https://github.com/tenstorrent/tt-metal/actions/runs/10621170840/job/29443026758

Could you also update the readme with the new test names? Probably also needs to be updated to mention it runs on gs/wh.

you mean the README in ttnn_resnet right?

Could you also update the readme with the new test names? Probably also needs to be updated to mention it runs on gs/wh.

you mean the README in ttnn_resnet right?

Yes


Downloaded file path

I am saving file in .pdf format, i need help that how to get downloaded file path.

Im sorry but your issue isn't really clear

The problem is that you will never know where the Browser saved it.

@arasabbasi can you please help me where the browser saved the file?

It is simply not possible to determine by javascript where you saved it.

@Balduin94 , pdf file has been generated , i wanna send generated pdf file to server. Api only accept file with .pdf extension.
After generating that pdf file, i am downloading file on button click. I wanna do download file and  send to server without button click.
The problem is , pdf downloaded in a browser, i need your help to get downloaded file.
Hope you will get my point.

It is simply not possible to determine by javascript where you saved it.

can you please guide me what i do to send pdf file to server?

What do you mean?

What do you mean?

pdf file has been generated , i wanna send generated pdf file to server. Api only accept file with .pdf extension.
After generating that pdf file, i am downloading file on button click. I wanna do download file and send to server without button click.
The problem is , pdf downloaded in a browser, i need your help to get downloaded file.
Hope you will get my point.

Well, based on your information this should be not possible at all. If it would be possible, every malicious hacker would like it. Imagine a browser would easily have access to your password storage and coule upload it to any server.
The only maybe possible way to solve this Would by generating a bloburl of the pdf, by doing doc.output('bloburl') and trying to supply it to the file upload field. You would have to do it all in the same window scope and possible used iframes.
Highly possible that you will get CORS restrictions.
You should more likely generate the pdf as dataurl by doing doc.output('dataurl') and sending the string per Ajax to your server.
This is more a thing for Stack Overflow, because here is the bugtracker for jsPDF.
Best Regards
Aras

Well, based on your information this should be not possible at all. If it would be possible, every malicious hacker would like it. Imagine a browser would easily have access to your password storage and coule upload it to any server.
The only maybe possible way to solve this Would by generating a bloburl of the pdf, by doing doc.output('bloburl') and trying to supply it to the file upload field. You would have to do it all in the same window scope and possible used iframes.
Highly possible that you will get CORS restrictions.
You should more likely generate the pdf as dataurl by doing doc.output('dataurl') and sending the string per Ajax to your server.
This is more a thing for Stack Overflow, because here is the bugtracker for jsPDF.
Best Regards
Aras

@arasabbasi Thanks, you save my time :D

I am sending generated pdfs to dropbox. I retrieve the pdf blob via  output('blob') and then upload with filesUpload`which is a method of the dropbox js api.


Handle nested dicts properly in .update_from_dict()

When updating from a nested dict:
dct = {
    "section": {
        "key": "entry"
    }
}

The whole section is replaced, instead of just updating the given entries

This actually works for dict objects. But it does not work if dct is a UserDict - including a Configuration

Fixed by https://github.com/gahjelle/pyconfs/commit/fcc7ab3cfbfbd3ced0d2e11d455daec910df65ae


Fix typo in comment

TIL I can merge PRs =P @rickyvetter, I haven't really contributed anything here, unfortunately. Feel free to remove me as a collaborator if you want =)

Oh, and thanks for the fix, @piperchester :)

Thanks @piperchester! @quicksnap happy to have you as a collab :)


Errors

Hello! Thanks everyone who has been contributing to this library!
I think that for the the errors we can throw ("Could not connect" and so on) should have an explanation
So in either the README or the wiki or even a new markdown file we should let the developers know what each error means and how to fix them.

Good idea!

Maybe you mean to fix the more detailed answer, for example
throw new ImapClientException('Error connecting to '.$mailbox);
fix on
throw new ImapClientException('Error connecting to '.$mailbox. '. Check your connection settings and Internet connection.');

Sorry. What i mean is not in the code. Another file in the repo for developers to use and read. Thats just what i think but that way works too. We can just see what everyone likes the most

I think need create docs/guide folders. Inside README.md with content like 'Install, Usage, Connect settings, Error' which will be links to the appropriate files. And accordingly create files Install.md, Usage.md and other. And in the future you can then create the appropriate folders with translations in various languages, like  docs/guide-fr,  docs/guide-de and other.

^^ sounds good to me

https://github.com/composer/composer/blob/master/doc/01-basic-usage.md

Package Names
The package name consists of a vendor name and the project's name. Often these will be identical - the vendor name just exists to prevent naming clashes. It allows two different people to create a library named json, which would then just be named igorw/json and seldaek/json.
Here we are requiring monolog/monolog, so the vendor name is the same as the project's name. For projects with a unique name this is recommended. It also allows adding more related projects under the same namespace later on. If you are maintaining a library, this would make it really easy to split it up into smaller decoupled parts.

Your pull request file composer.json

And may be add docs/guide folders. And Inside ... redme and etc. Because next step -> docs/guide-fr, docs/guide-de and other.

Fixed as of pr #68 Closing


Non-renewable resources and MMRCPSP

This PR adds non-renewable resources and an example for MMRCPSP.

I dislike how machines are now defined but the implementation should work.


kClassExtensions only check declared functions and properties

Library Version
0.2.9
Describe the bug
Extension functions getValidProperties and getValidFunctions in kClassExtension.kt only check declared members on KClass thus ommiting inherited properties and functions. This leads to validation error "object type 'SomeType' does not implement interface 'SomeInterface' because field 'SomeField' is missing". when using inheritance with abstract classes.
To Reproduce
My model:
interface SomeInterface {
  var someField: String?
}

abstract class SomeAbstractClass : SomeInterface {
   override var someField: String? = null
   var justAnotherField: String?
}

class SomeClass : SomeAbstractClass {
   override var justAnotherField: String? = null
}

Expected behavior
Interface generation should work with inherited fields, shouldn't it? kClassExtensions could use this.memberProperties/this.memberFunctions instead of this.declaredMemberProperties and this.declaredMemberFunctions.

I tried and changed the code in kClassExtensions and it basically works. Would it make sense to rewrite the generator to generate GraphQL interfaces from abstract class inheritance? Or am I overlooking something?

Thank you for reporting! I have pushed a PR as linked above solving the problem

Released in https://github.com/ExpediaDotCom/graphql-kotlin/releases/tag/0.2.11


[MINOR] [BUILD] master branch should reference 2.1.0-SNAPSHOT now that branch-2.0 exists

What changes were proposed in this pull request?
master branch should reference 2.1.0-SNAPSHOT now that branch-2.0 exists.
How was this patch tested?
Jenkins tests

Test build #61539 has started for PR 13995 at commit 1733e6f.

Test build #61539 has finished for PR 13995 at commit 1733e6f.

This patch fails MiMa tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61539/
Test FAILed.

Test build #61547 has started for PR 13995 at commit 1cb6daf.

Test build #61547 has finished for PR 13995 at commit 1cb6daf.

This patch fails MiMa tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61547/
Test FAILed.

Jenkins retest this please

Test build #61548 has started for PR 13995 at commit 1cb6daf.

Test build #61548 has finished for PR 13995 at commit 1cb6daf.

This patch fails MiMa tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61548/
Test FAILed.

Test build #61565 has started for PR 13995 at commit 0633d88.

Test build #61565 has finished for PR 13995 at commit 0633d88.

This patch fails MiMa tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61565/
Test FAILed.


WMCore-05-2017-issues

Seangchan, following Eric's suggestion, I created a trello board for making the management of these issues easier (well, let's see if that's true). We can chat over it today, including a "I don't like it and let's keep using github" :-)
The link is here:
https://trello.com/b/3SjExXhr/wmcore-issues

@ticoann as we discussed, to avoid updating things in two different places, we can use the WMCore milestone (for issues only, and considering each PR has an issue attached) for the issues tracking and to know the priority/assignee/etc:
https://github.com/dmwm/WMCore/milestone/136
From the next month on, we don't need to create this issue, simply move unsolved issues from HG1705 to HG1706 and add new stuff.

Milestone/issues were set for HG1706


module not found error

Describe the Bug
Hi all,
After downloaded all source code, I have mapped example folder and install all required npm packages.
While trying to build or start command I'm getting a module not found error. Any suggestion?
By the way ,even try to create new react template and use sample overview components, it has run but I see black pages with no error.

Your Example Website or App
No response
Steps to Reproduce the Bug or Issue

example folder
npm install
npm run
module not found error

Expected behavior
run localhost page
Screenshots or Videos

Platform

OS: [e.g. macOS, Windows, Linux]
Browser: [e.g. Chrome, Safari, Firefox]
Version: [e.g. 91.1]
windows 10
Chrome
Latest version

Additional context
No response

You need to do npm install and npm run build or npm start (for developing) in the root folder as well.


include: unified stream read and write interface

Summary
Provide simple macros for unified stream read and write interfaces
Impact
Testing

please rebase to the last master


Misleading sentence in this document

This sentence made me believe that these notifications will tell me when users are experiencing slow performance for the reports they are using: "Examples of these notifications include when users experience an extended delay in operations like opening reports, dataset refresh, or query executions."  Can you re-word this so that readers are not mislead on this functionality's purpose?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: b2c1d174-599a-d84e-d309-0987f24ef41f
Version Independent ID: 9e4735be-c98c-5887-3414-8413f5cb2881
Content: Service interruption notifications - Power BI
Content Source: powerbi-docs/service-interruption-notifications.md
Service: powerbi
Sub-service: powerbi-service
GitHub Login: @mgblythe
Microsoft Alias: mblythe

Thanks for your comment, @langjtl. I’ll add this request to our docs backlog.


Bug: Inappropriate delimiter used in fillRouteParams

Describe the bug
Trying to use a simple regex in the routes config file such as (a|b|c) results in an ErrorException being thrown at SYSTEMPATH/Router/RouteCollection.php at line 1331 with the error of preg_match(): Unknown modifier
Upon investigation i see that the fillRouteParams function is trying to preg_match using pipe (|) as a delimiter, which is not appropriate as the pipe is often used in regular expressions.
CodeIgniter 4 version
rc.3
Affected module(s)
Router
Reverse routing
Expected behavior, and steps to reproduce if appropriate
I expected $route->get('(a|b|c)', "Controller::index/$1", ['as' => 'testname']);
To result in calling Controller:index with either a, b or c passed as first parameter.
The excption is thrown as soon as  the route config files is being read.
P.S. I also tried escaping the pipe using \| but that resulted in CodeIgniter\Router\Exceptions\RouterException thrown in SYSTEMPATH/Router/Exceptions/RouterException.php at line 10

I'm having problems recreating this one.

Edited app/Config/Routes.php so the only route is: $routes->get('(a|b|c)', 'Home::index/$1', ['as' => 'testname']);
Edited app/Controllers/Home.php so index looks like:

public function index($val = null)
{
	dd($val);
	return view('welcome_message');
}

Fired up $ php spark serve and navigated to localhost:8080/a. As expected, the output was a dump of a.  Also navigated to /b,/c, /d with the first two showing the correct page and the 4th giving a 404 as expected.

What am I missing?

@korgoth can you provide any more insight?

Hello, Lonnie. Sorry for the delay but i am heavily on road last month.
After your first reply i made some more tests on my side and it seems that you are indeed correct.  Having only this one route setup - things are working as expected. It turns out however, that my setup is a bit more complex and thats whats raising the exceptions. Here are some more details:
I wanted to have a (:any) route, setup as my last route, that captures everything thats not defined and sends it to a controller. Thats why i had to define a / controller to act as a home.
Here is my complete routes setup:
$routes->get('/', 'Home::index', ['as' => 'home']); $routes->get('i/(:any)', 'Img::index'); $routes->get('g/(:any)', function($old_uri){ return redirect()->to('/'.url_title(urldecode($old_uri))); }); //REDIRECTS FROM OLD SITE $routes->get('(a|b|c)', 'Galleries::index/$1', ['as' => 'galleries.list']); $routes->get('chrono', 'Galleries::index', ['as' => 'galleries.list']); $routes->get('seek', 'Galleries::search', ['as' => 'galleries.search']); $routes->get('(:any)', 'Galleries::view/$1', ['as' => 'galleries.view']);
having it setup like this indeed produces correct results for (a|b|c) but throws an exception at /
ErrorException
Undefined offset: 0
SYSTEMPATH/Router/RouteCollection.php at line 1331

1324         // the appropriate places.
1325         foreach ($matches[0] as $index => $pattern)
1326         {
1327             // Ensure that the param we're inserting matches
1328             // the expected param type.
1329             $pos = strpos($from, $pattern);
1330
1331             if (preg_match("|{$pattern}|", $params[$index]))
1332             {
1333                 $from = substr_replace($from, $params[$index], $pos, strlen($pattern));
1334             }
1335             else
1336             {
1337                 throw RouterException::forInvalidParameterType();
1338             }

Now it might be that ive misunderstood how routing works, if so - please tell me what am i getting wrong.
Other than that i hope this helps debug the situation as i think such a setup is frequently used

Sorry it took so long. Still can't recreate, though. Used these routes:
$routes->get('/', 'Home::index');
$routes->get('(a|b|c)', 'Home::printer/$1', ['as' => 'testname']);
$routes->get('(:any)', 'Home::catchall/$1');

And this controller:
public function index()
	{
		return view('welcome_message');
	}

	public function printer($val)
	{
		dd('printer', $val);
	}

	public function catchall($val)
	{
		dd('catchall', $val);
	}

Going to /, a, b, c, or d all take me to the expected method.  I don't see any reason why it should when I look at your code. Sorry.


backup should be optional

plumb an option through into _install_from_cache

https://github.com/vivlabs/instaclone/pull/14


Loosen permissions on fetching/uploading/generating SSL keys for a delivery service

Currently, to fetch, upload or generate the SSL keys (Private Key, Certificate Signing Request and Certificate) for a delivery service where protocol > 0 (HTTPS, HTTP AND HTTPS, or HTTP TO HTTPS), you need to have the ADMIN role.
This doesn't lend itself very well to self-service. IMO delivery service owners should be able to manage the SSL keys of their own DS's.
My suggestion is to fetch, upload or generate the SSL keys for a delivery service you must:
A. Have the proper tenancy. I.e. you can't update SSL keys for delivery services outside of your tenant scope
B. You must have higher than the read-only role

<EMAIL_ADDRESS>

The problem here is that updating SSL keys requires a "queue updates", therefore allowing delivery service owners (non ops/admin) to update their SSL keys doesn't do any good at this point until "queue updates" is deprecated or automated.


pwsh as a dotnet global tool exhibits very unusual behavior with Ctrl-C

Steps to reproduce
Install powershell as a dotnet global tool. Run a long-running thing (like Start-Sleep), and then press Ctrl+C
Expected behavior
I am returned to a powershell prompt.
C:\Users\duhowett>.\.dotnet\tools\pwsh
PowerShell 6.2.3

(dhowett-dev2) ~ % start-sleep 10
^C
(dhowett-dev2) ~ % # i am returned to a powershell prompt

Actual behavior
The dotnet tool shim hosting powershell terminates and returns control to cmd, but powershell remains connected to the console and reads input simultaneously. PowerShell and CMD share the input buffer, and neither of them can agree on what goes into it.
If you manage to exit cmd, powershell re-asserts control.
The dir in the snippet below came from cmd
C:\Users\duhowett>.\.dotnet\tools\pwsh
PowerShell 6.2.3

(dhowett-dev2) ~ % start-sleep 10
^C
C:\Users\duhowett>
(dhowett-dev2) ~ % dir
 Volume in drive C is Cesium
 Volume Serial Number is E499-450A

 Directory of C:\Users\duhowett

The blue d came from powershell, and the ir came from cmd.
Environment data

Name                           Value
----                           -----
PSVersion                      6.2.3
PSEdition                      Core
GitCommitId                    6.2.3
OS                             Microsoft Windows 10.0.19041
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           <IP_ADDRESS>
WSManStackVersion              3.0

This isn't limited to pwsh as a dotnet global tool.  pwsh.exe exits whenever Ctrl + C is pressed in a long-running child process if pwsh.exe has been invoked from another terminal, e.g. powershell.exe or cmd.exe.  This makes it impossible to kill long-running native commands without also killing the whole terminal.
For example, the following works as expected:

Press Win + R
Type pwsh.exe
Press Enter
Run a long-running native command in pwsh (an EXE, not a cmdlet)
Press Ctrl + C while the command is running
The command will exit; pwsh will not

However, the following occurs, but isn't expected behavior:

Press Win + R
Type cmd.exe
Press Enter
Run a long-running native command in pwsh (an EXE, not a cmdlet)
Press Ctrl + C while the command is running
Both the command and pwsh exit; you're brought back to cmd.exe.

The following unexpected behavior also occurs, as described in #3869:

Open a PowerShell Core session in Windows Terminal
Run a long-running native command in pwsh (an EXE, not a cmdlet)
Press Ctrl + C while the command is running
Both the command and pwsh exit; the Windows Terminal tab will close.

This means pwsh is very difficult to use for anything serious in Windows Terminal, since cancelling a command kills the whole shell.

Update: I was on 6.2.3 when I observed that behavior.  I've since installed 7.0.0-rc1 and am no longer able to reproduce it.  There's a possibility I had installed it as a dotnet global tool before, but I thought I downloaded it directly from GitHub.  This time I installed it via the MSI on GitHub.

FWIW, I'm seeing this behavior in Windows Terminal with PowerShell installed via https://scoop.sh. Both 6.2.3 and 7.0.0-rc1 are acting similarly for me. Here's the output from 7.0.0-rc1:
C:\Users\brian>where pwsh
C:\Users\brian\scoop\shims\pwsh.exe

C:\Users\brian>pwsh --version
PowerShell 7.0.0-rc.1

C:\Users\brian>pwsh
PowerShell 7.0.0-rc.1
Copyright (c) Microsoft Corporation. All rights reserved.

https://aka.ms/powershell
Type 'help' to get help.

PS C:\Users\brian> start-sleep 10
^C
C:\Users\brian>
PS C:\Users\brian> gci
'gci' is not recognized as an internal or external command,
operable program or batch file.

Starting in cmd.exe, we can see that PowerShell version 7.0.0-rc.1 is the default pwsh and is located at C:\Users\brian\scoop\shims\pwsh.exe.
Running start-sleep 10, then pressing ctrl+c to stop the sleep results in a return to cmd.exe, with the PS prompt being output once for some reason. Attempting to run a PowerShell cmdlet results in a failure showing that we are, in fact, in cmd.exe.

Unless Scoop is using the dotnet global tool wrapper shipped with PowerShell, this is likely an issue affecting all Scoop-shimmed console-subsystem binaries. It’s almost certainly worth filing a duplicate of this issue on them so they can fix it for everyone.

Powershell Core installed as dotnet global tool.
same behaviour (only under windows terminal) on Compress-Archvie func, while asking for Path[0], press CTRL+C, no scoop installed.
Name                           Value
----                           -----
PSVersion                      6.2.4
PSEdition                      Core
GitCommitId                    6.2.4
OS                             Microsoft Windows 10.0.19555
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           <IP_ADDRESS>
WSManStackVersion              3.0

cmdlet Compress-Archive at command pipeline position 1
Supply values for the following parameters:
Path[0]:
[process exited with code<PHONE_NUMBER>]

Yes- this happens because Windows Terminal tracks the process that it has spawned, which in this case is the dotnet global tool wrapper. The wrapper is intercepting ^C and terminating when it shouldn’t.

@SteveL-MSFT This is interesting and we should figure this out. I means that anyone using PowerShell like this loses the whole process when pressing Ctrl-C

This happens so frequently, like whenever I want to stop the output of a long "dir" with Ctrl-C that I had to stop using Windows Terminal.

@ssg I am very sympathetic and I'll work to get it fixed, but a better solution would be to NOT use Pwsh as a .NET global Tool. Just install PowerShell Core and you won't have this issue and you can use Windows Terminal.

Please increase priority of the issue, I just moved from conemu, but the bug makes terminal+core near useless for work.

As noted, current mitigation is to install via the MSI or zip instead.

@SteveL-MSFT
It is clear about msi, thank you.
Could you please clarify about zip, because I supposed that it is the same that scoop does.

@inv2004 we also publish a zip pkg for Windows.  You can just unzip and run the contained pwsh executable.  I don't use scoop myself, so can't comment on that.  The team is currently busy with the rc.3 release, so perhaps someone can look at it to understand the complexity of a fix after that.  Can't commit to be addressed by 7.0 GA, but we can service it with a 7.0.1 release once a fix is understood.

@shanselman This workaround is great and it works, thanks! But the first hit on Google for downloading PowerShell Core still suggests dotnet global tool as the first option in the Tip box, and doesn't warn the user about implications, could it be revised at least?: https://docs.microsoft.com/en-us/powershell/scripting/install/installing-powershell-core-on-windows?view=powershell-7

Personally, I agree @ssg that while PowerShell Core installed via a .NET Global Tool would be the ideal way to get PWSH on a machine, we shouldn't recommend it OR we should include a warning that it won't work with Windows Terminal for now. @sdwheeler @KathleenDollard

Unless Scoop is using the dotnet global tool wrapper shipped with PowerShell, this is likely an issue affecting all Scoop-shimmed console-subsystem binaries. It’s almost certainly worth filing a duplicate of this issue on them so they can fix it for everyone.

Pretty simple fix in the end, just wasn't immediately obvious to me so here is what I did.
Just edit Windows Terminal settings such that your using pwsh.exe directly and not using the scoop shim.
Originally I had:
{
    "guid": "{b91d6aa8-b68c-416b-89b4-f29a2abebdcd}",
    "name": "Windows PowerShell Core",
    "commandline": "pwsh.exe", // which resolves to C:\Users\brad.jones\scoop\shims\pwsh.exe
    "hidden": false,
    "fontSize": 10,
    "padding": "1",
    "icon": "C:\\Users\\brad.jones\\Pictures\\PowerShell_Core_6.0_icon.png",
    "startingDirectory": "%USERPROFILE%"
}

Updated to:
{
    "guid": "{b91d6aa8-b68c-416b-89b4-f29a2abebdcd}",
    "name": "Windows PowerShell Core",
    "commandline": "C:\\Users\\brad.jones\\scoop\\apps\\powershell-core\\current\\pwsh.exe",
    "hidden": false,
    "fontSize": 10,
    "padding": "1",
    "icon": "C:\\Users\\brad.jones\\Pictures\\PowerShell_Core_6.0_icon.png",
    "startingDirectory": "%USERPROFILE%"
}

@brad-jones
Awesome. So simple and I can confirm that it works perfect.
            "commandline": "%USERPROFILE%\\scoop\\apps\\pwsh\\current\\pwsh.exe",

I agree with @ssg, the docs should mention that you need to use the msi package and not the dotnet tool installation. Searching for the cause of the problem finally brought me this issue here. I hope this is going to get fixed soon. Besides that: Nice job! The new terminal might replace cmdr for me. 👍

I can confirm this is still a problem with PowerShell 7.0.0 on Windows 10 build 1909 with Windows Terminal 0.11.1251.0

@SteveL-MSFT please re-open the ticket

Are you using scoop? If you're using scoop then you need to make sure you're not using the global shim for pwsh.exe and use scoop/apps/pwsh/current/pwsh.exe or replace your shim with the C-based alternative that doesn't come with showstopping problems like the one above, see https://github.com/lukesampson/scoop/issues/3634

I installed PowerShell 7 with Scoop. Here is the default configuration in Windows Terminal. It's using source instead of commandline.
            {
                "guid": "{574e775e-4f2a-5b96-ac1e-a2962a402336}",
                "hidden": false,
                "name": "PowerShell",
                "source": "Windows.Terminal.PowershellCore",
		"colorScheme": "Galaxy"
            }

Then replace your shims and it'll start working. it's a scoop issue, not a powershell issue.

Unless Scoop is using the dotnet global tool wrapper shipped with PowerShell, this is likely an issue affecting all Scoop-shimmed console-subsystem binaries. It’s almost certainly worth filing a duplicate of this issue on them so they can fix it for everyone.

Pretty simple fix in the end, just wasn't immediately obvious to me so here is what I did.
Just edit Windows Terminal settings such that your using pwsh.exe directly and not using the scoop shim.
Originally I had:
{
    "guid": "{b91d6aa8-b68c-416b-89b4-f29a2abebdcd}",
    "name": "Windows PowerShell Core",
    "commandline": "pwsh.exe", // which resolves to C:\Users\brad.jones\scoop\shims\pwsh.exe
    "hidden": false,
    "fontSize": 10,
    "padding": "1",
    "icon": "C:\\Users\\brad.jones\\Pictures\\PowerShell_Core_6.0_icon.png",
    "startingDirectory": "%USERPROFILE%"
}

Updated to:
{
    "guid": "{b91d6aa8-b68c-416b-89b4-f29a2abebdcd}",
    "name": "Windows PowerShell Core",
    "commandline": "C:\\Users\\brad.jones\\scoop\\apps\\powershell-core\\current\\pwsh.exe",
    "hidden": false,
    "fontSize": 10,
    "padding": "1",
    "icon": "C:\\Users\\brad.jones\\Pictures\\PowerShell_Core_6.0_icon.png",
    "startingDirectory": "%USERPROFILE%"
}

Awesome workaround!
For me, may this is more concise~
"commandline": "%SCOOP%/apps/pwsh/current/pwsh.exe"

For what it's worth, and for those who might not know the difference:
When using chocolatey, installing the powershell.portable will give you the "dotnet global" behavior and will close when Ctrl-C is pressed on a long running task.
Install the powershell-core chocolatey package instead.


Enable placement group

For now, there are two ways of using placement group.

Specify a placement group strategy. It will create a placement group according to the strategy and the configured resources for executors. The placement group will be removed when spark session stops.
Specify the placement group to use, and a list of bundle indexes. If no bundle indexes is specified, all bundles will be used.

I've tested placement group on the cluster. The current implementation choose the bundle to schedule executors in a round-robin manner, which is not good enough to handle some cases like executors failure. This can be improved in the future release.


Is library thread safe? Particularly the HasNextAsync/HasPreviousAsync methods?

I have been writing some unit tests for a project of mine which call an API endpoint which uses keyset paging and I have noticed that when I run the tests until failure, they will intermittently fail. They always seem to fail on checking whether HasMore is = true/false.
This is the line of my test that always fails:
paging!.HasMore.Should().BeFalse(); OR paging!.HasMore.Should().BeTrue();
This is my keyset pagination query:
var items = await dataContext.Items
    .KeysetPaginate(b => b.Ascending(m => m.Id), KeysetPaginationDirection.Forward, reference)
    .Query
    .Select(u => new
    {
        Id = u.Id
    })
    .Take(take)
    .ToListAsync();

This is how I determine whether hasMore (cut down for brevity):
if (keysetPaginationContext.Direction == KeysetPaginationDirection.Forward)
{
    hasMore = await keysetPaginationContext.HasNextAsync(items);
}
else
{
    hasMore = await keysetPaginationContext.HasPreviousAsync(items);
}

This is my Paging<T> model which contains hasMore:
public class Paging<T>
{
    public bool HasMore { get; init; }
    public List<T> Items { get; init; }

    public Paging(List<T> items, bool hasMore)
    {
        Items = items;
        HasMore = hasMore;
    }
}

I am wondering if because the 8 tests run in parallel, perhaps something is getting wires crossed between tests? Some state is being used somewhere in the HasNextAsync(items) or HasPreviousAsync(items) ?
I am also creating the EF Core 7 data context fresh for every test so there should be no issues regarding that.
Any advice would be much appreciated.

No internal state is kept for any of the Has* calls. Regarding thread safety, EFCore's DbContext itself isn't thread safe, and so obviously this library in extension isn't. The returned context that you chain Has* calls on isn't as well. But similarly to thread safety guarantees in .NET, any static state is guaranteed to be thread safe, you shouldn't have to worry about that.
In the Has* calls, this library only calls into EFCore's AnyAsync (you can check the code here). But you mentioned that you recreate the DbContext in each test, so that removes one possibility.
Where are you getting the keysetPaginationContext from in your second snippet? It doesn't look like you're storing it anywhere in the first snippet. I can't tell what the problem could be from just the snippets you have above, especially that they don't seem to be connected. If you can have a full small repro repo of this on GitHub I might be able to have a closer look.

Also, your second snippet looks weird to me. Why are you testing on the direction and then calling either HasNextAsync or HasPreviousAsync depending on that? You might be misunderstanding HasNextAsync/HasPreviousAsync.

Hi @mrahhal , thanks for the response.

You might be misunderstanding HasNextAsync/HasPreviousAsync.

Hmm, I had a look through the source code again and I think you might be right.
I was under the impression that if the pagination context had been configured for a Forward direction, then I needed to call HasNextAsync to determine if there is "more items to get" and then vice versa for a Backward direction? But I think what you're saying is, for my use case, I should just always call HasNextAsync (regardless of what direction the pagination context is set as) ?
If so, then I will make that change in my code but I'm not sure it would resolve the intermittent test failures I am getting. And if it still isnt working, then yes I will get a GitHub repo together for you to have a look at.
Thanks

Yes, HasNextAsync will always tell you if there's next relevant to your defined order, the direction doesn't matter. But a caveat is that you need to make sure EnsureCorrectOrder is called on your data list before calling these methods (this is explained in the README).

Hmm, so I went and made the change to only call HasNextAsync regardless of direction but then my code didnt work anymore.
For more context, I have two parameters, one called starting_after and another called ending_before. starting_after is for navigating forwards (i.e. getting the next page) and ending_before is for navigating backwards (i.e. getting the previous page).
Given a simplified paging example, with set of Ids like so:
1
2
3
4       <---- ending_before
5
6
If ending_before is set to 4 and page size of 10, my expected returned items are 1,2 and 3 and HasMore = false. If I use HasNextAsync, it actually returns HasMore = true because the DB query checks for Id values > 4 and not for Id values < 1 (as HasPreviousAsync does).
Using HasNextAsync works when used in conjunction with starting_after because I am navigating forwards through the result set.

It's very likely you're doing something incorrectly at some point (I feel in particular related to how and with what you're calling the Has* methods). The use case you explained above is extremely basic. You can check MR.AspNetCore.Pagination to compare and contrast with what you're doing, since I do something very similar here in a generalized way. Otherwise, I might need a small repro.

I'm closing this for lack of additional info. Please reopen if you still have issues.


default make example results in DRC errors

would be smooth if the default passed precheck

Ya, that would be nice wouldn't it :) Could you see if it passes with the text removed from the input png?


Builds put the theme files in a sub directory of the theme

Hi there. When I run the build command the theme files are created in a subdirectory of my theme (ex: my-theme/my-theme/style.css). I created a new theme to be sure the issue wasn't something related to my theme and saw the same behavior. Any ideas?

If I understand you correctly, that is by design. Check out the deploying instructions here:
Deploying Your Theme
I have struggled with this a bit... how best to communicate to the user about differences between dev and prod. I poking around with more ideas... some of the other issues I've filed myself in here are in that regards.
Let me know if I misunderstood you. Also, any ideas around developing vs deploying are welcome.
Thanks!

I see. Yes, that's what the documentation says. For our use-case it would make more sense if the files were in the root of the theme so when we deploy via git it works as is. This way I need to create a wrapper around yarn build that moves the generated files.

Yah, I just ran into so many people that didn't understand the folder layout for this project, and they would deploy their theme to prod, but ended up with the react-src folder on their prod server. Which... even worse, contains the node_modules folder. Ugh!
Like I mentioned, I am going to revisit the whole build process. Lots more hand-holding. But I think I can also add a setting or two to the user.prod.json file. Maybe an onSuccessMoveToFolder type of thing.
Someone else a while back had a similar work flow to yours. I recommended shelljs. Here's the issue link: #14 if you're interested.
I'm going to leave this ticket open for now, and look at it when I do the "build revisit".
Thanks!

This is done in the latest release: 3.2.0-wp.2.
Inside the user.prod.json file you can add a new property named "finalBuildPath" and when you run npm run build inside the react-src folder, you're compiled files should end up where you want them. I tested with a value of ".." so that they would end up in the theme's root folder, like you wanted.
Here's the contents of the user.prod.json file I tested with:
{
"finalBuildPath": "..",
"homepage": "/wordpress/wp-content/themes/foobar"
}
Let me know if it doesn't work for you.
Thanks!


Refactor the 'join' command

Refactor the 'join' command

@bhargodevarya Choose one of the available commands mentioned in #4 and create a new issue. This one is already taken by @chrisom79. If he does not finish it, I will open this issue for others again.


[Dubbo-5690]fix wrong telnet command package

What is the purpose of the change
fixed #5690
Brief changelog
fix wrong telnet command package in META-INF/dubbo/internal/org.apache.dubbo.remoting.telnet.TelnetHandler
Verifying this change
XXXXX
Follow this checklist to help us incorporate your contribution quickly and easily:

[x] Make sure there is a GITHUB_issue field for the change (usually before you start working on it). Trivial changes like typos do not require a GITHUB issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
[ ] Format the pull request title like [Dubbo-XXX] Fix UnknownException when host config not exist #XXX. Each commit in the pull request should have a meaningful subject line and body.
[ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
[ ] Write necessary unit-test to verify your logic correction, more mock a little better when cross module dependency exist. If the new feature or significant change is committed, please remember to add sample in dubbo samples project.
[ ] Run mvn clean install -DskipTests=false & mvn clean test-compile failsafe:integration-test to make sure unit-test and integration-test pass.
[ ] If this contribution is large, please follow the Software Donation Guide.

Codecov Report

Merging #5815 into master will decrease coverage by <.01%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master    #5815      +/-   ##
============================================
- Coverage     61.18%   61.17%   -0.01%
+ Complexity      497      495       -2
============================================
  Files           981      981
  Lines         39143    39095      -48
  Branches       5644     5611      -33
============================================
- Hits          23948    23915      -33
+ Misses        12567    12555      -12
+ Partials       2628     2625       -3

Impacted Files
Coverage Δ
Complexity Δ

...ache/dubbo/remoting/transport/AbstractChannel.java
37.5% <0%> (-50%)
0% <0%> (ø)

...rg/apache/dubbo/remoting/utils/PayloadDropper.java
15.38% <0%> (-30.77%)
0% <0%> (ø)

...va/org/apache/dubbo/remoting/TimeoutException.java
22.22% <0%> (-11.12%)
0% <0%> (ø)

...ubbo/rpc/protocol/dubbo/ChannelWrappedInvoker.java
39.13% <0%> (-8.7%)
0% <0%> (ø)

...mmon/threadpool/support/fixed/FixedThreadPool.java
80% <0%> (-7.5%)
0% <0%> (ø)

...moting/transport/netty4/NettyEventLoopFactory.java
30% <0%> (-6.37%)
0% <0%> (ø)

.../threadpool/support/limited/LimitedThreadPool.java
83.33% <0%> (-5.56%)
0% <0%> (ø)

...on/threadpool/support/cached/CachedThreadPool.java
85.71% <0%> (-4.29%)
0% <0%> (ø)

...e/dubbo/remoting/transport/netty/NettyChannel.java
52.27% <0%> (-3.41%)
19% <0%> (-1%)

...dubbo/remoting/exchange/support/DefaultFuture.java
80.35% <0%> (-3.27%)
0% <0%> (ø)

... and 34 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ca28815...76af026. Read the comment docs.


Fixed CI timeout for emulated SVE targets

Sadness 😔. Would be nice to fix the targets. Rvv comes - that's gonna be extra awful

We need to cut algo and compress tests in more pieces.

Not splitting, foguring out which test cases to drop

Yeah, yeah, you can merge this, I was just complaining


fix: docs typo - Update 20-load.md

an SPA -> a SPA

Please don't delete this checklist! Before submitting the PR, please make sure you do the following:

[ ] It's really useful if your PR references an issue where it is discussed ahead of time. In many cases, features are absent for a reason. For large changes, please create an RFC: https://github.com/sveltejs/rfcs
[ ] This message body should clearly illustrate what problems it solves.
[ ] Ideally, include a test that fails without this PR but passes with it.

Tests

[ ] Run the tests with pnpm test and lint the project with pnpm lint and pnpm check

Changesets

[ ] If your PR makes a change that should be noted in one or more packages' changelogs, generate a changeset by running pnpm changeset and following the prompts. Changesets that add features should be minor and those that fix bugs should be patch. Please prefix changeset messages with feat:, fix:, or chore:.

Edits

[ ] Please ensure that 'Allow edits from maintainers' is checked. PRs without this option may be closed.

"SPA" is pronounced as three separate letters and so begins with a vowel sound, and we want to use "an".


chore: a simple e2e test with knuu

This PR proposes to have a simple e2e test for celestia-node with knuu.
The main purpose of this PR is to show how to use knuu for e2e tests.

currently the following error is thrown and the progress is blocked due to that:
$ make test-e2e
--> Running end to end tests
go run ./test/e2e
2024/10/02 18:16:03 proto: duplicate proto type registered: celestia.core.v1.da.DataAvailabilityHeader
2024/10/02 18:16:04 proto: duplicate proto type registered: celestia.core.v1.proof.ShareProof
2024/10/02 18:16:04 proto: duplicate proto type registered: celestia.core.v1.proof.RowProof
2024/10/02 18:16:04 proto: duplicate proto type registered: celestia.core.v1.proof.NMTProof
2024/10/02 18:16:04 proto: duplicate proto type registered: celestia.core.v1.proof.Proof
2024/10/02 18:16:04 proto: duplicate proto type registered: celestia.core.v1.tx.TxStatusRequest
2024/10/02 18:16:04 proto: duplicate proto type registered: celestia.core.v1.tx.TxStatusResponse
panic: error with code 1 is already registered: "invalid signal version because signal version can not be less than the current version"

goroutine 1 [running]:
cosmossdk.io/errors.RegisterWithGRPCCode({0x314d082, 0x6}, 0x1, 0x2, {0x325ebed, 0x56})
	/home/moji/go/pkg/mod/cosmossdk.io/errors@v1.0.1/errors.go:43 +0x265
cosmossdk.io/errors.Register(...)
	/home/moji/go/pkg/mod/cosmossdk.io/errors@v1.0.1/errors.go:35
github.com/celestiaorg/celestia-app/v3/x/signal/types.init()
	/home/moji/go/pkg/mod/github.com/celestiaorg/celestia-app/v3@v3.0.0-20241002134031-0f446000fdd5/x/signal/types/errors.go:8 +0x45
exit status 2
make: *** [Makefile:158: test-e2e] Error 1

Codecov Report
Attention: Patch coverage is 0% with 532 lines in your changes missing coverage. Please review.

Project coverage is 45.13%. Comparing base (2469e7a) to head (53d8fc5).
Report is 321 commits behind head on main.

Files with missing lines
Patch %
Lines

test/e2e/testnet/node.go
0.00%
124 Missing :warning:

test/e2e/testnet/instance_options.go
0.00%
107 Missing :warning:

test/e2e/testnet/status.go
0.00%
86 Missing :warning:

test/e2e/simple.go
0.00%
64 Missing :warning:

test/e2e/testnet/testnet.go
0.00%
64 Missing :warning:

test/e2e/testnet/helper.go
0.00%
42 Missing :warning:

test/e2e/main.go
0.00%
28 Missing :warning:

test/e2e/testnet/executor.go
0.00%
17 Missing :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3795      +/-   ##
==========================================
+ Coverage   44.83%   45.13%   +0.30%
==========================================
  Files         265      322      +57
  Lines       14620    18625    +4005
==========================================
+ Hits         6555     8407    +1852
- Misses       7313     9201    +1888
- Partials      752     1017     +265

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

Update: by pointing to the feature_branch_app_v3 the issue of app version conflict is resolved.

Update: since we need to activate Minio and probably some other options for the node tests, we need to wait for this PR in the celestia-app repo to be reviewed.


Subscription RP v2021-10-01 endpoint fix, s360 correctness

MSFT employees can try out our new experience at OpenAPI Hub  - one location for using our validation tools and finding your workflow.

Changelog
Add a changelog entry for this PR by answering the following questions:

What's the purpose of the update?

[x] new service onboarding
[x] new API version
[x] update existing version for new feature
[x] update existing version to fix swagger quality issue in s360
[x] Other, please clarify

When are you targeting to deploy the new service/feature to public regions? Please provide the date or, if the date is not yet available, the month.
When do you expect to publish the swagger? Please provide date or, the the date is not yet available, the month.
If updating an existing version, please select the specific langauge SDKs and CLIs that must be refreshed after the swagger is published.

[x] SDK of .NET (need service team to ensure code readiness)
[x] SDK of Python
[ ] SDK of Java
[ ] SDK of Js
[ ] SDK of Go
[x] PowerShell
[x] CLI
[ ] Terraform
[ ] No refresh required for updates in this PR

Contribution checklist:

[x] I commit to follow the Breaking Change Policy of "no breaking changes"
[x] I have reviewed the documentation for the workflow.
[x] Validation tools were run on swagger spec(s) and errors have all been fixed in this PR. How to fix?

If any further question about AME onboarding or validation tools, please view the FAQ.
ARM API Review Checklist

Applicability: :warning:
If your changes encompass only the following scenarios, you should SKIP this section, as these scenarios do not require ARM review.

Change to data plane APIs
Adding new properties
All removals

Otherwise your PR may be subject to ARM review requirements. Complete the following:

[x] Check this box if any of the following apply to the PR so that label "WaitForARMFeedback" will be added automatically to begin ARM API Review. Failure to comply may result in delays to the manifest.

Adding a new service
Adding new API(s)
Adding a new API version
-[x] To review changes efficiently, ensure you copy the existing version into the new directory structure for first commit and then push new changes, including version updates, in separate commits.

[x] Ensure you've reviewed following guidelines including ARM resource provider contract and REST guidelines. Estimated time (4 hours). This is required before you can request review from ARM API Review board.

[ ] If you are blocked on ARM review and want to get the PR merged with urgency, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.

Breaking Change Review Checklist
If any of the following scenarios apply to the PR, request approval from the Breaking Change Review Board as defined in the Breaking Change Policy.

[ ] Removing API(s) in a stable version
[x] Removing properties in a stable version
[ ] Removing API version(s) in a stable version
[x] Updating API in a stable or public preview version with Breaking Change Validation errors
[ ] Updating API(s) in public preview over 1 year (refer to Retirement of Previews)

Action: to initiate an evaluation of the breaking change, create a new intake using the template for breaking changes. Addition details on the process and office hours are on the Breaking change Wiki.
Please follow the link to find more details on PR review process.

Approving from ARM API but you still need to get breaking change approval (see process in PR description) unless they are still letting people auto bypass that if its an s360 consistency fix.  You'd have to reach out to them to get the ack on that

Hi @qiaozha , @ArcturusZhang , @msyyc , Would you please take a look at the breaking in SDK, Thanks

The lint issues are not related to the change in this PR. Add the tag to suppress it.

/azp run


Is it possible to detect lanes from a another point of view?

I would like to detect lanes in an image but not from the viewpoint of the moving car. For example, the lanes are horizontal in the image. Is this possible? Are there any restrictions implemented like the sorting of the lanes that prevent this task?

Yes, there are restrictions. To detect lanes from another point of view would not be trivial.

Thanks for the answer.
But do you think your approach is conceivable for such an application? With some modifications. Or are the results achieved precisely because of the restrictions?

Yes, I think would work.

Thanks.

@stah-ba Have you managed to detect the horizontal lanes with this network ? Thanks in advance


clarify term : contact site

ID	GO:0044289
Term name contact site
Definition Sites of close apposition of the inner and outer mitochondrial membrane.
There are lots of contact sites
please rename
mitochondrial inner/outer membrane contact site (or similar)

It might work  from a biological perspective to broaden the definition of this term and put it under here:
GO:0044232    organelle membrane contact site
(even though this is "within organelle")
Anyway, there are lots of "contact sites"

Maybe the relation should be 'adjacent to' organelle rather than 'part of'?

also from duplicate (now closed) #14863
Is "bridge contact site"
Site of contact between the inner and outer mitochondrial membrane found in neuronal mitochondria; may play a role in maintaining the structural integrity of the inner and outer boundary membranes.
https://www.ebi.ac.uk/QuickGO/term/GO:0044285
the same thing too? It doesn't really sound as though it is specific to neuronal mitochondria...that would be odd?

mitochondrial inner/outer membrane contact site (or similar)

is OK within the mitochondrion though, do you agree ?

Actually this looks like it should be merged with   GO:0044284 mitochondrial crista junction
Def "A tubular structure of relatively uniform size that connects a mitochondrial crista to the mitochondrial inner boundary membrane"
See PMID: 22009199
Thanks, Pascale

I broaden the definition of organelle contact site to
"A zone of apposition between the membranes of an organelle with another membrane, either another membrane of the same organelle, that of another organelle, or the plasma membrane. Membrane contact sites (MCSs) are structured by bridging complexes. They are specialized for communication, including the efficient traffic of small molecules such as Ca2+ ions and lipids, as well as enzyme-substrate interactions."

Also moved 'mitochondrial crista junction' under  organelle contact site

I will merge GO:0044285 bridge contact site (0 annotations) into GO:0044289 mitochondrial name contact site- I cannot find evidence for 'bridge contact site' .

Yes i think that would be a useful grouping.


Missing nginx default file

The nginx.conf file is missing leading to built docker containers to fail to run.
Command: docker run -it -p 80:8000 -v ~/projects/iguana/files:/files <Iguana image ID>
Output:
Creating default files on the volume.
Traceback (most recent call last):
  File "/usr/local/bin/docker_entrypoint.py", line 57, in <module>
    copy_file(src_path, path.join(FILES_DIR, df))
  File "/usr/local/lib/python3.8/distutils/file_util.py", line 104, in copy_file
    raise DistutilsFileError(
distutils.errors.DistutilsFileError: can't copy '/iguana/files_default/nginx.conf': doesn't exist or not a regular file

Hello,
did you build the Docker image by yourself?
-> Then please provide the build command that you used.
Or did you use an existing image?
-> Please provide the used image tag.

Ok, I know where the problem is:
The current implementation of the Dockerfile does not intend that a development image contains a nginx server. Please have a look at the following snippet:
https://github.com/iguana-project/iguana/blob/f313ca4557456898d59c52dd60b9f09a86a1c742/Dockerfile#L94-L107
Here you can see, that the nginx.conf file is only created if the VARIANT variable is not development.
Unfortunately this behavior is not documented anywhere. Maybe I find some time this weekend to implement a quick check in the Dockerfile to prevent using --build-arg USE_NGINX=true and --build-arg VARIANT=development together.
But please keep also in mind that this project didn't get much attention in the last years. As an example it still uses Django version 2.2 which is EOL since April 2022!
Please don't expect any major changes in the future to the code of this project. In my opinion to bring this project back to life not only the dependencies have to be updated, but also a lot of code refactoring has to be done.

With commit 5e21064 I changed the Dockerfile to validate the build arguments. Now an error is thrown if USE_NGINX=true and VARIANT=development are both set.

I see, I'd love to help out as this project looks like exactly what I'm looking for. Unfortunately, I'm not confident with my programming skills.
I appreciate your help and insight for the project!

With commit 5e21064 I changed the Dockerfile to validate the build arguments. Now an error is thrown if USE_NGINX=true and VARIANT=development are both set.

This commit fixed the issue set out in the issue.

I see, I'd love to help out as this project looks like exactly what I'm looking for.

Thank you for liking our project. I appreciate that :)
Just an additional note: The project itself was part of a course during a semester at my university back in 2016/2017. At the end mostly all desired features were implemented and the application is still working today.
But as I stated out before, such a project needs continuous maintenance. And for an open source project this is not easy to ensure.
Therefore, if you use this application, I would suggest that - in it current stage with a lot of outdated dependencies - it should be not reachable from the internet. Just to be safe ;)