az acr login with private endpoint goes via public route

Describe the bug
Command Name
az acr login
Errors:
Error response from daemon: Get https://myregistryname.azurecr.io/v2/: denied: client with IP '<my public IP>' is not allowed access. Refer https://aka.ms/acr/firewall to grant access
Login failed.

To Reproduce:
Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.

Connect to Azure VPN
az acr login -n myregistryname --subscription mysubscription

Expected Behavior
Login to the ACR's private endpoint via the connected VPN
Environment Summary
Linux-4.4.0-19041-Microsoft-x86_64-with-debian-bullseye-sid
Python 3.6.10
Installer: DEB

azure-cli 2.19.1

Using WSL1

Additional Context
I am logged into a VPN (Azure Virtual Network Gateway), I have made the necessary adjustments to the /etc/resolv.conf so that the ACR resolves to the Private Link IP:
Server:         10.x.x.x.
Address:        10x.x.x#53

Non-authoritative answer:
myregistryname.azurecr.io canonical name = myregistryname.privatelink.azurecr.io.
Name:   myregistryname.privatelink.azurecr.io
Address: 10.x.x.x

The ACR is configured not to allow public access, but has a private endpoint configured which is known to work
I can curl the URL of the ACR and get a 200 response (as it goes via the private address), it seems that the issue is with azure-cli itself.
ip route 10.x.x.x also shows that the route goes via the VPN interface

route to appropriate team

@r3-jerrysteele, sorry for missing this. The best way to diagnose this is to verify the DNS setting. If not resolving to the private ip, then the error is expected. Doc is here:
https://docs.microsoft.com/en-us/azure/container-registry/container-registry-private-link#validate-private-link-connection.
Please let us know what you found out

We are experiencing a similar issue. We have created an Azure Container Registry with a Private Endpoint. The public access is denied.
Login to ACR with az acr login -n tftest4acr fails with the following error:
Unable to get AAD authorization tokens with message: An error occurred: CONNECTIVITY_REFRESH_TOKEN_ERROR Access to registry 'tftest4acr.azurecr.io' was denied. Response code: 403. Please try running 'az login' again to refresh permissions. Error response from daemon: Get https://tftest4acr.azurecr.io/v2/: denied: client with IP '<IP_ADDRESS>' is not allowed access. Refer https://aka.ms/acr/firewall to grant access.
'az login' was executed successfully several times. The error message above is always displayed.
The ACR FQDN tftest4acr.azurecr.io is correctly resolved to the private IP.

Hi. I am having this exact same error and have been pulling my hair out trying to solve it. It might be the underlying cause is different. I am not sure. I’ve been through the link above which gets you to validate your DNS setup, but it doesn't seem (unless I am misunderstanding, which is definitely possible) to give guidance on what to do if things still aren't working. I think it’s meant to say the DNS resolves to an external IP of the ACR if being accessed from outside. Or in internal IP if being accessed from a local vnet.
My objective is to connect from my home machine to a private container registry using Private Link (and so all public access on the ACR is turned off) which seems to be the recommended way.
To reproduce the error:
To try to login, for every scenario, I am doing:
az login
followed by
az acr login —name <my acr name>
It then says;
Unable to get AAD authorization tokens with message: An error occurred: CONNECTIVITY_REFRESH_TOKEN_ERROR
Access to registry ‘<my acr name>.azurecr.io' was denied. Response code: 403. Please try running 'az login' again to refresh permissions.
Unable to get admin user credentials with message: Admin user is disabled.

And it prompts me for a user name and password.
Using my Owner credentials, I always get;
Error response from daemon: Get “<my acr name>://my .azurecr.io/v2/": denied: client with IP ‘<my ‘external IP> is not allowed access. Refer https://aka.ms/acr/firewall to grant access.
Login failed.

And where I’ve tried other (Service Principal or user setup under ACR/Networking/Access keys, with admin enabled) credentials, I’ve done below:
az acr login <my acr name> -u <my acr name> -p <password provided on that page>
But I always get the error
The login server endpoint suffix .azurecr.io is automatically omitted.
Error response from daemon: Get "https://<my acr name>.azurecr.io/v2/": denied: client with IP ‘<my ‘external IP> is not allowed access. Refer https://aka.ms/acr/firewall to grant access.
Login failed.

As I understand it, the az login credentials expire in about an hour, so I am making sure I always do the az login for a test, just to be sure.
I’ve also tried connecting from VS code but get the same issue, ie it won’t connect and gives a 403 forbidden error.
For all these scenarios, if I look in the ACR logs, I always see the forbidden error but I am not sure if / how I can dig down deeper this way. One thing I did wonder is it mentions the User Agent as
My ACR setup so far:
I have an Azure Container Registry. Under Networking, Public Access is disabled and under Private Access a private end point is setup. It setup an end point for the login server of .azurecr.io and a data end point of .uksouth.data.azurecr.io
The private end point has a private DNS zone (setup by Azure)
Under customer visible FQDN I have:
.nic. with two 10. IP addresses. One of them maps to the data end point and the other to the login end point.
There is one Configuration which is called privatelink-azurecr.io for private dns zone privatelink.azurecr.io which contains two FQDNs which map to the same two 10. IP addresses mention above.
I’ve setup a VPN from my end to a Virtual Network Gateway on the Azure side:
I’ve been following the VPN Point to Site instructions at https://docs.microsoft.com/en-us/azure/vpn-gateway/vpn-gateway-howto-point-to-site-resource-manager-portal and at  https://docs.microsoft.com/en-us/azure/vpn-gateway/point-to-site-about .
I’ve tried to do what it says at https://docs.microsoft.com/en-us/azure/container-registry/container-registry-private-link for setup of ACR.
I am thinking that maybe the whole P2S isn’t the best way for dev / continuous integration but everything I am reading (and hopefully understanding but maybe not) is saying it could work, so I am just trying to see if it does.
I’ve created an IKEv2 VPN (from my Mac) with certificates and this connects fine, allocating me a 10. IP from the vnet I’ve setup. I followed the virtual network gateway instructions to do this.
I’ve tried all of the below with the VPN connected and not connected but it doesn’t make a difference.

I think is is because it’s more likely a connectivity issue.
As usual I am guessing it is probably something basic I am not seeing yet. I am pretty new to Azure so it’s definitely possible.
DNS settings:
Right now., I suspect the problem is DNS related but I can’t quite manage a solution yet.
If I do a dig for .azurecr.io from my machine, I get the external IP of the private link end point. And that’s with or without VPN.
If I dig the ACR from a Linux VM in the vnet, I get the 10. address of the ACR. And I can login with the method I mention below. I.e. az login followed by az acr login.
This may all be where my understanding of DNS is failing but my current theory (!) is that when I am connected to the virtual network gate way, my IP should appear to Azure as a 10. but it’s seeing the NAT address given out by my ISP. Does this make sense?
I looked at the DNS setup recommendations under https://docs.microsoft.com/en-us/azure/private-link/private-endpoint-dns#virtual-network-and-on-premises-workloads-using-a-dns-forwarder and tried setting up an Azure Firewall / Policy / DNS settings to: Enable DNS settings to apply to the policy, use the Azure provided DNS server and DNS proxy enabled. Under that setting, it says;
To ensure DNS traffic is directed to the Azure Firewalls associated with this policy, you must configure your virtual network DNS server settings and set the Azure Firewall’s private IP address as a Custom DNS server.
I’ve done what it says which is, under vnet / DNS servers, I’ve got a DNS server address of the firewall public IP.
Should I be looking at other methods of DNS setup, e.g. setting up a VM and doing forwarding? This is suggested as another way of working but not sure it makes sense. But I was just experimenting.
I have also tried setting up a DNS server of  in the Mac VPN config of <IP_ADDRESS>
Other things tried:
I’ve tried turning off local firewalls, though didn’t think that would make a difference. And it didn’t.
Just as a side experiment, but wasn’t really thinking it would work, I’ve tried the Azure CLI add network-rule to allow my IP but that doesn’t help.
I’ve tried setting up an NSG for the vnet and allowing traffic from that IP but as I understand it, NSGs are over-riden for private end points, so this was a bit of a desperate trial.
Any ideas (or solutions) greatly appreciated. Thanks, Paul

Hi. I have the exact same issue. Not sure if it's a CLI issue or user error, so I posted it on the Microsoft Q&A forum. [https://docs.microsoft.com/en-us/answers/questions/632408/trying-to-access-an-azure-container-registry-with.html]

Good to know not just me then. I gave up. I had a back and forth with MS
support but just could not solve it. If anyone has a new ideas I would
be happy try :-)
On May 16, 2022, fieldp @.***> wrote:

I am facing the same issue.
 The ACR DNS resolves to private IP address in the vnet -

however, az acr login throws this error -

—
Reply to this email directly, view it on GitHub
<https://github.com/Azure/azure-cli/issues/17137#issuecomment-
1127842531>, or unsubscribe
<https://github.com/notifications/unsubscribe-
auth/AVIA2WUDOTQPC4MBITOADXTVKJVGHANCNFSM4YIZXCPQ>.
You are receiving this because you commented.Message ID: <Azure/azure-
@.***>

I have this same issue, I've created a Azure Resourse Manager Service Connection and While using az acr build --image reponame:imagename --registry acrname --file Dockerfile . with self hosted agent, I'm getting the same error.

I found that using docker login / docker build correctly uses the private endpoint.
I created a token on the ACR with a PW
Using this to login as example
echo ${{ secrets.ACR_PW }}| docker login -u ${{ secrets.ACR_USER }} --password-stdin registryUrl
This to build as example
docker build -t registryUrl/imagename --file DockerFile .

I'm running private hosted devops agents. Login works but building the image using az acr build get still somehow routed over the public route:
az acr login --name myreg.azurecr.io
az acr build --registry myreg.azurecr.io --image xxx:yyy .

========================== Starting Command Output ===========================
/usr/bin/bash /agent/_work/_temp/771f368c-362a-4c11-8df9-b49ece019f58.sh
WARNING: The login server endpoint suffix '.azurecr.io' is automatically omitted.
Login Succeeded
WARNING: The login server endpoint suffix '.azurecr.io' is automatically omitted.
WARNING: Packing source code into tar to upload...
WARNING: Excluding '.git' based on default ignore rules
WARNING: Excluding '.gitignore' based on default ignore rules
WARNING: Uploading archived source code from '/tmp/build_archive_c4dc880b67914c5d9e0fdabd3ca87293.tar.gz'...
WARNING: Sending context (854.722 KiB) to registry: myreg...
WARNING: Queued a build with ID: cb4
WARNING: Waiting for an agent...
2022/07/06 22:12:39 Downloading source code...
2022/07/06 22:12:40 Finished downloading source code
2022/07/06 22:12:41 Using acb_vol_055ad1ce-aff6-4195-bf76-f619d26a7abf as the home volume
2022/07/06 22:12:41 Setting up Docker configuration...
2022/07/06 22:12:41 Successfully set up Docker configuration
2022/07/06 22:12:41 Logging in to registry: myreg.azurecr.io

failed to login, ran out of retries: failed to set docker credentials: Error response from daemon: Get "[https://acrlhggixiservicesp.azurecr.io/v2/"](https://myreg.azurecr.io/v2/%22): denied: client with IP '<IP_ADDRESS>' is not allowed access. Refer https://aka.ms/acr/firewall to grant access.
: exit status 1

Just want to add that I am also experiencing the same issue.  Doing a nslookup or dig on the acr endpoint resolves to the private IP address but az acr build routes via the public endpoint.
And because I am running the agent in an azure container instance I have had problems getting docker to run inside the container.  So the workaround noted above is not as easy for me to deploy.

I had found with a Microsoft Case that you need to whitelist those 2,3 public IPs on ACR which will appear with that error. Those IPs are somehow useful for build.

@thedheerendra  Unfortunately, when you turn off public access entirely you no longer are able to whitelist any public IPs.  The az acr build generates ACR tasks which use dedicated compute nodes to perform the container build that exist outside the virtual network the build agent is running in.  That seems to be the crux of the issue.

I have the exact same issue. Are there any updates?

confirmed this issue on macos and ubuntu operating systems today

We have the same issue. az acr build gets routed via public route :(

@Wompipomp see https://github.com/Azure/acr/issues/603

Any updates on this ?

Changing the assignment to @northtyphoon and @terencet-dev for next steps.

have same issue while running az cli on build agent and acr over private link

In my case it is az acr run --cmd "acr purge.." that is failing to work via private endpoint, probably the same root cause as for build.

have same issue while running az cli on build agent and acr over private link
Work around az acr login --name registryname cd $(Build.SourcesDirectory) docker buildx build -t registryname.azurecr.io/reponame:$(Build.BuildId) --platform=linux/arm64 . docker push registryname.azurecr.io/reponame:$(Build.BuildId) if ($LastExitCode -ne 0) { Write-Host “##vso[task.complete result=Failed;]DONE” exit 1 }

This problem seems to still be present. This workaround did the trick for us.

Did this ever get resolved?  I am seeing the same exact problem as the original post.  nslookup resolves the correct vnet address of the ACR, az acr login tries to work via the public address (and recommends adding my IP to the acr firewall)

I have the same problem. The xxx.azurecr.io correctly resolves to xxx.privatelink.azurecr.io and goes through the internal endpoint to the ACR. However, upon running az acr login it seems to go over public internet.
Ridiculous that this hasn't been fixed for this long.

still an issue at the moment. and no workaround for login command ?


Fail to syntax highlight sometimes.

Atom fail to syntax highlight ramdomly at : https://github.com/atom/atom/commit/0b34de005cfbff82c972830aa3bbe3938c7bec4f
Tested with javascript and python.
Had to switch other language highlight and back to fix that.

Can you please provide 1) a snippet of javascript in which syntax highlighting is not applied correctly, 2) a screenshot of what you see, and 3) clarify which part in the screenshot isn't highlighted correctly?

i reloaded them , i will test around and send, haven't tried using --safe.

Let us know when you can provide more detailed information, and please follow the submission guidelines in the future.


As a user I want to be able to apply a filter, to ensure the app is fun to use

AC:

On the "filter page" the image of the user now applies a filter in realtime

This only works in Chrome at the moment


GH Req

Precondition
First test precondition
Steps to Reproduce:
Step 1
First step
Expected Result
It goes
Step 2
Second
Expected Result
Alright

Open Requirement in TestQuality

Test First test.

Open Defect in TestQuality

Test First test in Run Run 001 (linked to Cycle All tests).


Switching to correct network requires refresh on send page (iOS)

Describe the bug
On iOS, switching to correct network results in an infinite loading status requiring a refresh/navigation to complete transaction
To Reproduce
Steps to reproduce the behavior:

select wrong chain in MM mobile,
connect wallet
see warning of incorrect chain
attempt to send tx
see error message (correct behavior) and dismiss
open MM and connect to correct network
return to app and attempt to send again
see infinite loading requiring a refresh/navigation to reload send page to send successful tx

Expected behavior
on switch to correct network, user should be able to complete transaction without refreshing or navigating away from send page
Smartphone (please complete the following information):

Device: iPhone

Additional context
Behavior only affects iOS

Resolved, on click 'send' user is again prompted to switch to correct network in order to proceed if initially dismissed before attempting transaction


XXX is typing...

It comes up as the orginial Discord colour

fixed in this commit https://github.com/WhatDaPuck/NotAnotherAnimeTheme/commit/8f364ffea3a9fa0e18f4a97121f35d154af9c6cf


add layering

scale and tilt inner layers of petals
off-set petal positions from layer to layer (in between rotation_increment)

empirically flowers tend to have an odd number of petals following the Fibonacci sequence, so I'll use this as a guide for petal_count (3, 5, 8, 13, 21, 34, 55)


[CLI] SIFT install fails

sift install

<EMAIL_ADDRESS>Warning: You are running as root.
Error: You cannot install as root without specifying the --user option.

The install user specified with --user must not be the root user
sift --user=ubuntu install

<EMAIL_ADDRESS>Warning: You are running as root.
sift-version: notinstalled

You cannot run as root. Running as root and specifying a user isn't allowed either.
You must use sudo.


Eval tip is failed in this situation.

Expected Behavior
my source is typescript:
constantStrings.ts
export class ConstantStrings {     static readonly teststr: string = "test string"; }
app.ts
import {ConstantStrings} from "./ConstantStrings"
function main() {
console.log(ConstantStrings.teststr);
}
main();
When i break in main console.log ... , move mouse to ConstantStrings.teststr, has not eval tip.
I guess, ts variable name[ConstantStrings] mapping to js is error in this situation.

NTVS Version: 1.2.40726.00
Visual Studio Version:  14.0.25425.01 Update 3
Node.js Version:  4.4.7
TypeScript Version: 2.0.6

Thank you. I confirmed this does currently seem broken. I'm not sure if the root cause is NTVS or TypeScript, but will investigate further.


Add method for refreshing an access token

Add a new refreshAccessToken method to the ApiClient, addressing #281.
Works exactly as the generateAccessToken with the exception that it expects a refresh token instead of an authorization code.
The test are not included because the existing tests for generateAccessToken are commented out as well (suggesting to me that these are not tested for some reason).

I think this is very necessary. Do you have any plans for an update?


Identify the potential impact of the possible Receiver Settings changes/improvements

Problem statement
The onboarding engineers do not use the existing Receiver Settings area of the Reportstream admin tools (on the website) due to lack of usability and function. However, a better Receiver Settings experience would help existing engineers with their COVID to UP Receiver Migration work and lay the foundation for "self-service" receiver settings by external users in the future.
After consolidating the existing research, we want to look at the potential impact of the different features.
What you need to know

This work picks up where #12459 left off

Acceptance criteria

[x] Identify the potential impact (why it would help) of each group of changes/features
[x] Group the changes/features/requirements of the Receiver settings proposal by "near term" vs "future" impact
[ ] Work with Chris/Jillian to devise next steps

First sketch of prioritization

@chris-kuryak @jillian-hammer
Link to mural with potential impact and groupings.

@chris-kuryak @jillian-hammer I've completed the prioritization but am still waiting to hear back from engineers. Should we find time to sync on next steps? Perhaps figure out a strategy for more directly engaging with engineers.

@jakefishbein-navapbc Sound good. Can you find us a time in the next few days?

@chris-kuryak @jillian-hammer I sent an invite for Monday afternoon

Next steps (for a new ticket):

Meet with engineers to gather feedback on how to accomplish the "happy path". Identify hidden complexity and dependencies.
Clarify the deployment infrastructure and align with settings functions.
Get a review from Victor on overall technical approach.


Slower memory fix

This ensures that datatypes are maintained throughout the weather model calculations.
Description

enforces datatype maintenance
removes one variable from the weather model object memory load (svp)
temp change to the NoDataValue that gets used; the previous one was too large.

Motivation and Context

This is mainly to show how a couple minimal changes eliminate the float64's in the weather model variable (post-load_weather). For ERA-5 I keep float32's all the way through.

How Has This Been Tested?

Screenshots (if appropriate):
Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

[ ] I have added an explanation of what your changes do and why you'd like us to include them.
[ ] I have written new tests for your core changes, as applicable.
[ ] I have successfully ran tests with your changes locally.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.

@dbekaert just checked this ends up writing float32 for both ERA-5 and HRRR.

@leiyangleon @dbekaert I'm getting an FTP error on this PR now that I've incorporated Yang's new PR. Obviously it didn't fail on the other so I don't know why it's failing here...didn't see anything obvious from the diff that would point to a change in the FTP call. @leiyangleon if you have a few minutes today could you take a quick look at the error message I'm getting in CircleCI and see if you have seen the problem before? Thanks!

@jlmaurer I have seen this error before. Basically, it tells you that the ftp link is wrong. When the server is down or the file does not exist, the 500 OOPS: cannot change directory:/usr1home error will appear. You can do a quick check by running one of the helper example commands locally and replacing ERA5 with NCMR. For the date/time, please use 2018-07-01 and 00:00:00 as used in test_scenario_1.py. That file should exist. Please remember to create a NCMR login file in your $HOME directory as detailed in my last PR #297.

@leiyangleon yes I assumed that was the problem on my local machine, but circleCI is also breaking with this problem as well. Is this an issue with the server being temporarily down, so if I re-start the test it should pass?

@leiyangleon thanks! @dbekaert probably we will want to think about the more, as it's a bit unstable to have our unit test suite depend on an external FTP server that evidently is down quite often. I think this PR is probably ready to be merged, but can't verify that until the server comes back on.

@leiyangleon thank you! I actually found a way to incorporate pytest natively for the catch, so I think we should be good to go.

@jlmaurer Cool! LGTM.


Eventspace notifications- Not recieving notification for all nodes

I get eventspace notifications for a randome node after subscribing using the following command:
pubsub.psubscribe('key*:*')
Is there any way to subscribe to all nodes?

No. Pubsub is not recommended to use in a cluster and until/unless redis-server itself solves the n**2 full mesh problem i will not make an effort to sort out pubsub subscription in this lib as i do not want to provide code features for something that is inherit broken on the server side when you run a cluster.
My recommendation is to use a sentinel setup for pubsub and run it separate from your regular cluster.
There is also lots of documentation i written about it if you look inside docs/ folder.

@Grokzen  My bad I was just trying to ask if there is any way to achieve this- "Every node of a Redis cluster generates events about its own subset of the keyspace as described above. However, unlike regular Pub/Sub communication in a cluster, events' notifications are not broadcasted to all nodes. Put differently, keyspace events are node-specific. This means that to receive all keyspace events of a cluster, clients need to subscribe to each of the nodes."
To be precise subscribe for events with respect to each node in cluster?

@hkd1996 When did this change happen as the links that is in the docs talking about it i think is still relevant and i know that antirez did not change anything inside redis-server to my knowledge at least to solve this problem. If there has been changes into this as from the old posts i read up on this there was lots of low level debugging and investigating that needed to be done to verify that fanout dont happen any longer in certain conditions. I have not tested this for some time now and it might be time to test this again with 6.0 and 6.2 as the old issue was from like 3.x and 4.x time and that was some time ago.
But to look at the code pov in your question, there is no way to send a subscribe event to multiple nodes at the same time as it would mean threads and more complexity in our lib compared to redis-py that only have to solve it for one node and not multiple nodes.
Is there some official documentation about this keyspace events in a clustered environment i can read up on where you found out about this information?

https://redis.io/topics/notifications Refer to "Events in a cluster" that is where i came to know about this! @Grokzen

Thank you, will dig into this a bit so i am updated to any changes that has happen the last few years, will ask redis-labs if they know anything and see what you can do.
If you want to solve your problem a hacky way because it is not possible to solve this with the default code as the default code is written to not be threaded in this case when you subscribe and it will only take your key and hash it with the crc16 algorithm to determine what node to talk to.
So what you can do is that you can spin up a RedisCluster instance and let it fetch the cluster state, then you can take what data that NodeManager/ClusterConnectionPool contains and build your own solution with threads to parallel the multiple subscribe that you are looking for by creating multiple Redis class instances and just send psubscribe to each node manually so you can talk to each node as you want.

I'm new to python development can you help me in achieving this? - "So what you can do is that you can spin up a RedisCluster instance and let it fetch the cluster state, then you can take what data that NodeManager/ClusterConnectionPool contains and build your own solution with threads to parallel the multiple subscribe that you are looking for by creating multiple Redis class instances and just send psubscribe to each node manually so you can talk to each node as you want."

@hkd1996 If you are new to python and not familiar with wither socket programming or threads programming then i think doing this yourself is a big task and might be out of your reach of what you are capable of.
But what i can give you advice on is that if you create a RedisCluster object and point it to a running cluster and you can run simple commands like get/set against the cluster successfully, then you can go through the connectionpool object into the nodes tracking object here https://github.com/Grokzen/redis-py-cluster/blob/master/rediscluster/connection.py#L149 and then futher on use this method https://github.com/Grokzen/redis-py-cluster/blob/master/rediscluster/nodemanager.py#L122 to hopefully get access to all nodes in some way where you can then build your own threaded solution to run parralell psubscribe to all nodes at the same time and listen to all nodes at the same time inside your threads.
I can't give you more then that help as i have no plans to support this keyspace feature in the near future due to the complexity of having to listen to all nodes at the same time in some kind of threaded environment. The solution would need to make to many assumptions on what each user wants and a generic use-case probably do not fit everyone and then i get into maintainers hell by needing to support each use-case and variant. I rather just say that how you can get access to each node object/connection and implement whatever you wish there instead.


[FIX] Make pipeline-catalog submodule available to built Docker image

Fixes #372

Changes proposed in this PR:

Fixes our GH actions workflows for building + pushing images to DockerHub to check out submodules

Checklist
This section is for the PR reviewer

[ ] PR has an interpretable title with a prefix ([ENH], [FIX], [REF], [TST], [CI], [MNT], [INF], [MODEL], [DOC]) (see our Contributing Guidelines for more info)
[ ] PR has a label for the release changelog or skip-release (to be applied by maintainers only)
[ ] PR links to GitHub issue with mention Closes #XXXX
[ ] Tests pass
[ ] Checks pass

For new features:

[ ] Tests have been added

For bug fixes:

[ ] There is at least one test that would fail under the original bug conditions.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>7
Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 98.372%

Totals

Change from base Build<PHONE_NUMBER>3:
0.0%

Covered Lines:
967

Relevant Lines:
983

💛  - Coveralls

Hey Seb, re-requesting your review here for an extra pair of eyes on the Dockerfile changes.
Agree with your suggestion to only copy in required files. As part of maybe https://github.com/neurobagel/bagel-cli/issues/350, I think we can also move the test suite out of the bagel directory for the same reason, since we don't need to copy the tests into the image (and it's best practice to keep them separate from the main program code, like how we're doing in the API repos).


An error occurred

Click to pay and keep waiting. Finally, an error message appears.
“An error occurred, please try again later.”
What is the problem?

Hi, the telegram bot from the example in the documentation have you already run and implemented the events: “pre_checkout_query” and “successfull_payment”?
Or is this demo template from the library, I don't quite understand?

If you need realtime help, you can join my forum, where questions about my libraries are discussed and problems are solved when developing telegram mini-apps!
https://t.me/unigram_tools

OK, I join,
I would like to ask where can I see the stars for successful payment?

You need to go to the bot's profile and the balance will be displayed after the first successful purchase.
On the mobile version of Telegram, you will have to click on the “3 dots” to open the balance.

Okay, I think I've finished it. Thank you

Okay, any more questions, get in touch!


Flushbar Not Testable With duration

Minimal reproducable example:
Widget with flushbar:
import 'package:another_flushbar/flushbar.dart';
import 'package:flutter/material.dart';

class FlushbarExample extends StatelessWidget {
  const FlushbarExample({Key? key}) : super(key: key);

  @override
  Widget build(BuildContext context) {
    return ElevatedButton(
      child: Text("Button"),
      onPressed: () {
        Flushbar(
          flushbarPosition: FlushbarPosition.BOTTOM,
          message: "Help",
          duration: const Duration(seconds: 3),
        ).show(context);
      },
    );
  }
}

Testclass:
import 'package:flutter/material.dart';
import 'package:flutter_test/flutter_test.dart';

import 'flush_example.dart';

main() {
  testWidgets(
    "test flushbar",
    (WidgetTester tester) async {
      await tester.pumpWidget(
        const MaterialApp(
          home: Scaffold(
            body: FlushbarExample(),
          ),
        ),
      );
      var button = find.text("Button");
      expect(button, findsOneWidget);
      await tester.tap(button);
      await tester.pump(const Duration(seconds: 2));
      expect(find.text("Help"), findsOneWidget);
      await tester.pump(const Duration(seconds: 2));
    },
  );
}

Error
══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
The following assertion was thrown running a test:
'package:flutter/src/widgets/navigator.dart': Failed assertion: line 5021 pos 12:
'entry.currentState == _RouteLifecycle.popping ||
          (entry.hasPage && entry.currentState.index < _RouteLifecycle.pop.index)': is not true.

Either the assertion indicates an error in the framework itself, or we should provide substantially
more information in this error message to help you determine and fix the underlying cause.
In either case, please report this assertion by filing a bug on GitHub:
  [https://github.com/flutter/flutter/issues/new?template=2_bug.md]()

When the exception was thrown, this was the stack:
#2      NavigatorState.finalizeRoute (package:flutter/src/widgets/navigator.dart:5021:12)
#3      OverlayRoute.didPop (package:flutter/src/widgets/routes.dart:74:18)
#4      FlushbarRoute.didPop (package:another_flushbar/flushbar_route.dart:393:18)
#5      _RouteEntry.handlePop (package:flutter/src/widgets/navigator.dart:2896:16)
#6      NavigatorState._flushHistoryUpdates (package:flutter/src/widgets/navigator.dart:3860:22)
#7      NavigatorState.pop (package:flutter/src/widgets/navigator.dart:4902:7)
#8      FlushbarRoute._configureTimer.<anonymous closure> (package:another_flushbar/flushbar_route.dart:403:22)
#15     FakeTimer._fire (package:fake_async/fake_async.dart:316:16)
#16     FakeAsync._fireTimersWhile (package:fake_async/fake_async.dart:240:13)
#17     FakeAsync.elapse (package:fake_async/fake_async.dart:137:5)
#18     AutomatedTestWidgetsFlutterBinding.pump.<anonymous closure> (package:flutter_test/src/binding.dart:986:28)
#21     TestAsyncUtils.guard (package:flutter_test/src/test_async_utils.dart:71:41)
#22     AutomatedTestWidgetsFlutterBinding.pump (package:flutter_test/src/binding.dart:982:27)
#23     WidgetTester.pump.<anonymous closure> (package:flutter_test/src/widget_tester.dart:608:53)
#26     TestAsyncUtils.guard (package:flutter_test/src/test_async_utils.dart:71:41)
#27     WidgetTester.pump (package:flutter_test/src/widget_tester.dart:608:27)
#28     main.<anonymous closure> ([file:///C:/projects/flutter/trainingsmagnet/test/example/flush_example_test.dart:22:20]())
<asynchronous suspension>
<asynchronous suspension>
(elided 13 frames from class _AssertionError, dart:async, and package:stack_trace)

The test description was:
  test flushbar
════════════════════════════════════════════════════════════════════════════════════════════════════

Remove the Duration property in the widget and it works fine.

We are encountering the same issue with flutter 2.10.3 and another_flushbar: 1.10.28

Any updates in this?


I am making the changes in my bot but getting the below error

EACCES: permission denied, open '/Users/rishabh.agarwal/github/prodeng-support-bot/language-generation/en-us/prodeng_support_bot.en-us.lg'

@rishabh-agarwal10
Assuming that you are still experiencing this issue; please get some additional logging when reproducing the issue.

Go to Composer's install location (mine is at C:\Users{username}\AppData\Local\Programs\Bot Framework Composer)
Open a terminal here
run set DEBUG=composer*
run the executable from the terminal Ex. "Bot Framework Composer.exe"

Then send the output of the terminal.


Improved symlink dereferencing on OSX by calling a perl script.

This is a hotfix for the problem described in #182.
Instead of removing readlink and realpath, we try a third option (a perl script) to resolve the symlink.

@truthbk can you check this pease?

Btw. This solution is from stackoverflow. It is not perfect, I had some situations with symlinked folders that were not dereferenced in linux environments (but there realpath and readlink -f would work).. But IMHO it should be sufficient in most cases.

Worked in Linux, as well, actually.

Merged into next.


Bugfix for exception in a race state. Simplification of the library by removing the ioc stuff.

Hello upta,
I prepared this pull request, after some problems with the library. There was a state, in which the handler list was changed, while a enumeration was running. That resulted in an exception.
Also after reading #25, i retought the ioc stuff i submitted earlier and removed it completly. Now there is only one interface for the hub, it is still ioc friendly, but much simpler.
Please have a look and let me know, what you think.
Greetings Bebu1985

I came across the same problem. Changing the handlers list while iterating generates an exception:
    System.InvalidOperationException: Collection was modified; enumeration operation may not execute.
      at at System.ThrowHelper.ThrowInvalidOperationException(ExceptionResource resource)
      at at System.Collections.Generic.List`1.Enumerator.MoveNextRare()
      at at System.Collections.Generic.List`1.Enumerator.MoveNext()
      at at System.Linq.Enumerable.WhereListIterator`1.MoveNext()
      at at PubSub.Hub.Publish[T](T data)

It can be reproduced when publishing an event that is handled by an object that itself creates a handler. The simple fix is returning list which is not modified by creation of new handlers, as you have implemented in this PR.
I wrote a unit test to verify it is fixed. Maybe this could be added to CoreHubTests.
        [TestMethod]
        public void Publish_NoExceptionRaisedWhenHandlerCreatesNewSubscriber()
        {
            // arrange
            _hub.Subscribe(new Action<Event>(a => new Stuff(_hub)));

            // act
            try
            {
                _hub.Publish(new Event());
            }

            // assert
            catch (InvalidOperationException e)
            {
                Assert.Fail($"Expected no exception, but got: {e}");
            }
        }

        internal class Stuff
        {
            public Stuff(Hub hub)
            {
                hub.Subscribe(new Action<Event>(a => { }));
            }
        }

Regards,
Guy

I gladly integrate your unittest in my pull request. But it needs some time, i am busy until next Monday.

No worries, I'm not in a hurry to integrate a new version of pubsub. The older version works well enough. I was interested in testing if the new Async methods can improve UI responsiveness before I ran into this bug.

The unittest is integrated.

Hi @Bebu1985. This PR contains both a bug fix and alot of refactoring. It might be worth splitting the PR into two - one for the bug fix and the other for refactoring. Then if there is any problem accepting refactoring changes the bug fix can still be approved separately. It would be good to have the bug fix merged because issue #30 looks alot like the same problem. @upta does that make sense to you?

@gb0o @Bebu1985 I'd agree with splitting it out a bit, especially so we can easily get that bug fixed.
From what I can see, it seems like this PR:

Potentially fixes the bug from #30
Does a bunch of refactoring
Adds an Azure Pipelines build definition (which I'm not strictly opposed to, but there is already an AppVeyor one so not really pressing)
Updates the major version

Quite a few separate things (none of them bad at all) for a single PR :)

I submitted a new pull request with only the bugfix and the unittest.

Closing this since the current state has conflicts and the key bits were done in #31


Corrected path to coco_labels

coco_labels.txt is always stored in ./data, not COCO_ROOT

I couldn't find coco_labels.txt ,when I search all the locations


Intrinsics & Extrinsics

Hello, can I get the Intrinsics & Extrinsics of the camera?

Sorry, we don't have such exact parameters because we've captured object profile images through mobilephone camera. We aim to immitate that we observe an object by eyes in real world. I suggest you consider using colmap to calculate these parameters from multiple profile images. If you are interested in Object Instance Detection, please check https://insdet.github.io.

Please check our InsDet Challenge Workshop @ ACCV 2024. The GoogleDrive link to test data is available now.

Hello, I checked the Google Drive link and it's exciting. Is it a close set task or is it an open set task? And what is the input of the model when I generate the testing result? For example, my model will use the scene image with several objects as input, or can I use the scene image with several objects and the query object image as input?


Parallelize Parquet Serialization

Which issue does this PR close?
Related to https://github.com/apache/arrow-rs/issues/1718
Rationale for this change
#7452 sped up writing of CSV and JSON files (even when writing only a single large file) by serializing RecordBatches in parallel on multiple threads. This PR attempts to accomplish the same but for Parquet files. This is more challenging vs. CSV/JSON since Parquet serialization is not embarrassingly parallelizable.
#7483 enabled writing different  parquet files in parallel, but this PR attempts to allow writing a single parquet file in parallel.
What changes are included in this PR?
This PR implements the following strategy to parallelize writing a single parquet file:

For each incoming RecordBatch stream, serialize a parquet file to an in memory buffer in parallel.
As files from step 1 finish, consume them in order on another task and stitch them together into one large parquet file.
As 2 progresses, flush bytes from a shared buffer to an ObjectStore in multiple parts.

Step 2 to 3 are streaming/incremental, but step 1 is not. In general, all N mini-parquet files are likely to be buffered in memory before they are consumed in step 2/3.
Given the tradeoff between execution time and memory usage in this implementation, this PR also provides a session config which allows turning off single file parallelization. This option is useful in memory constrained environments.
Further steps that could improve this PR:

Stream bytes from step 1. to step 2. to limit the increase in memory consumption with increase in parallelism
Enable parallelizing serialization of multiple large parquet files. E.g. a system with 32 cores wishes to write out 4 1GB parquet files. Each parquet file could be written with parallelization of 8 (i.e. stitch together 8 smaller parquet files for each of the 4 large parquet files).

Benchmarking
The following script is used to write parquet file(s) and capture execution time and peak memory consumption.
use datafusion::{dataframe::DataFrameWriteOptions, prelude::*};
use datafusion_common::DataFusionError;
use object_store::local::LocalFileSystem;
use peak_alloc::PeakAlloc;
use std::{sync::Arc, time::Instant};
use url::Url;

#[global_allocator]
static PEAK_ALLOC: PeakAlloc = PeakAlloc;

const FILENAME: &str =
    "/home/dev/arrow-datafusion/benchmarks/data/tpch_sf10/lineitem/part-0.parquet";

#[tokio::main]
async fn main() -> Result<(), DataFusionError> {
    let mut runtimes = Vec::new();
    let mut max_memory = Vec::new();
    for parallelism in [64, 32, 16, 8, 4, 1] {
        PEAK_ALLOC.reset_peak_usage();
        let _ctx = SessionContext::new();
        let local = Arc::new(LocalFileSystem::new());
        let local_url = Url::parse("file://local").unwrap();
        _ctx.runtime_env().register_object_store(&local_url, local);

        let _read_options = ParquetReadOptions::default();

        let _df = _ctx
            .read_parquet(FILENAME, _read_options.clone())
            .await
            .unwrap()
            // Optionally select a subset of 3 columns
            //.select_columns(&["l_orderkey", "l_partkey", "l_receiptdate"])?
            .repartition(Partitioning::Hash(vec![col("l_orderkey")], parallelism))? ;

        println!("Number of columns: {}", _df.schema().field_names().len());

        let out_path = format!(
            "file://local/home/dev/arrow-datafusion/test_out/bench{parallelism}.parquet"
        );

        let start3 = Instant::now();
        _df.clone()
            .write_parquet(
                out_path.as_str(),
                DataFrameWriteOptions::new().with_single_file_output(true),
                None,
            )
            .await?;
        let elapsed3 = Instant::now() - start3;
        println!("write as parquet with parallelism {parallelism} to disk took -> {elapsed3:?}");
        runtimes.push(elapsed3);

        let peak_mem = PEAK_ALLOC.peak_usage_as_mb();
        println!(
            "Peak memory usage with parallelism {parallelism} is: {}MB",
            peak_mem
        );
        max_memory.push(peak_mem);
    }

    println!("Runtimes: {:?}", runtimes);
    println!("Peak memory: {:?}", max_memory);

    Ok(())
}

Notes for test results:

All tests are run on a system with 16c/32t.
parallelism=1 always falls back to the same AsyncArrowWriter implementation as used on current main branch
"Output multiple files" is the same implementation developed in #7483 to write independent files in parallel
"Output single file (flush each subfile)" is the implementation in this PR
"Output single file (flush each 10MB)" is an alternative implementation in #7563

Test 1, All 16 Columns, ~3.6GB Parquet File (release build)
Execution Time(s)

Parallelism
Output multiple files**
Output Single File (flush each subfile)
Output Single File (flush each 10MB)

1*
22.48
22.53
22.53

4
12.24
16.39
14.40

8
10.79
13.83
12.37

16
10.52
14.31
12.67

32
10.91
14.10
12.07

64
10.21
14.93
12.97

Peak Memory Consumption(MB)

Parallelism
Output multiple files**
Output Single File (flush each subfile)
Output Single File (flush each 10MB)

1*
1753.3
1758.0
1759.1

4
2445.4
7104.0
7094.3

8
3387.0
7623.1
7611.9

16
5047.6
8262.6
8258.0

32
7683.6
7672.6
7658.2

64
10961.1
10370.2
10323.4

Test 2, Subset of 3 Columns, ~895MB Parquet File (release build)
Execution Time(s)

Parallelism
Output multiple files**
Output Single File (flush each subfile)
Output Single File (flush each 10MB)

1*
3.57
3.31
3.15

4
1.78
2.97
2.37

8
1.45
2.73
2.07

16
1.54
2.98
2.09

32
1.70
2.48
2.10

64
1.89
2.80
2.72

Peak Memory Consumption(MB)

Parallelism
Output multiple files**
Output Single File (flush each subfile)
Output Single File (flush each 10MB)

1*
450.6
447.9
451.6

4
584.5
1788.1
1659.1

8
759.4
1869.1
1939.7

16
1045.8
2058.7
2051.2

32
1564.6
1899.7
1899.7

64
2318.8
1718.3
1726.1

Test 1, All 16 Columns, ~3.6GB Parquet File (dev build)
Execution Time(s)

Parallelism
Output multiple files**
Output Single File (flush each subfile)
Output Single File (flush each 10MB)

1*
277.77
267.60
266.29

4
83.69
88.25
133.41

8
47.36
51.87
97.09

16
35.79
39.92
84.65

32
31.71
35.47
81.64

64
32.23
36.21
82.24

Peak Memory Consumption(MB)

Parallelism
Output multiple files**
Output Single File (flush each subfile)
Output Single File (flush each 10MB)

1*
1753.9
1758.9
1759.7

4
2465.2
7082.4
7092.9

8
3402.5
7611.2
7604.8

16
5145.3
7041.2
7884.3

32
7795.9
7720.1
7720.7

64
10957.3
10480.9
10290.9

Test 2, Subset of 3 Columns, ~895MB Parquet File (dev build)
Execution Time(s)

Parallelism
Output multiple files**
Output Single File (flush each subfile)
Output Single File (flush each 10MB)

1*
42.82
40.52
40.36

4
14.24
14.98
29.73

8
7.77
8.84
20.44

16
5.33
6.59
18.93

32
4.80
5.34
17.40

64
4.98
5.54
17.78

Peak Memory Consumption(MB)

Parallelism
Output multiple files**
Output Single File (flush each subfile)
Output Single File (flush each 10MB)

1*
445.9
449.7
447.7

4
577.1
1789.3
1693.9

8
756.9
1923.4
1865.5

16
1038.2
1940.8
1933.0

32
1554.2
1899.7
1907.0

64
2337.5
1710.4
1726.8

Are these changes tested?
Yes by existing tests
Are there any user-facing changes?
Faster single parquet writes and a new config option to enable/disable (on by default).

This is really cool to see, and thank for your empirical data gathering, I will try to review this over the next few days

I realized that the benchmarking numbers I ran last night I neglected to run on a release build. I added additional results in the description for a release build. Single threaded serialization is 10x faster in the release build and the relative gains from this parallelization strategy are significantly lower. In release build, #7563 is a decent bit faster than this PR as well.

This looks amazing -- thank you @devinjdangelo  -- I ran out of time to review this PR today but I plan to do so over the weekend

Thanks @alamb for the review. I agree with the plan of action you lay out!
I also merged in the changes from the alternative implementation PR given that it is about 10% faster and slightly lower memory usage in the release build benchmarks. I set the config default to false and expanded the doc comment to flag the missing features if you enable it.

Ok, I think all follow on tasks are tracked and this PR is ready to go. Thank you again @devinjdangelo !

PR to add ticket references: https://github.com/apache/arrow-datafusion/pull/7592


[FEATURE] Simplify broadcasting in apps

Feature you want to implement
Currently the apps emits signals and connects slots to broadcast or to receive broadcast messages.
This style is not so intuitive nor convenient.
Therefore, we want to provide an easy way to broadcast and receive messages.
How the feature is implemented
Basically BaseApp class will be modified.
1. Broadcasting
Implement a simple 'wrapper' method as follows:
def broadcast(self, channel: str, content: Any):
  content = json.dumps(content)
  self.broadcastRequested.emit(channel, content)

2. Receiving
Implement a pre-connected slot as follows:
def receivedSlot(self, channel: str, content: Any):
  """This will be overridden by child classes."""

@pyqtSlot(str, str)
def _receivedMessage(self, channel: str, msg: str):
  """This is connected to self.received signal."""
  self.receivedSlot(channel, json.loads(msg))

A child class which inherits BaseApp will then override receivedSlot() to handle the received messages.
Note that the class can still implement its own slots for received signal.
We just provide a new convenient way in addition to the previous method.

I changed the label refactor -> enhancement because this provides a new feature which is similar to syntax sugars.
When we say "refactoring", it usually does not change the code behavior (API).

I just implemented the above three methods in BaseApp.
Before applying it into all apps, please confirm these methods.
It can be checked in BECATRUE/133/simplify-handle-signals branch.
@kangz12345

Oh, I found that we shouldn't check if the content is instance of str in broadcast(), because it is JSON-loaded when received anyway. I updated the example code above.

In addition my reviews:

In line 63, it should be pyqtSlot rather than pyqtSignal.
In __init__(), you should connect _recevedMessage() to received signal.

Oh, I found that we shouldn't check if the content is instance of str in broadcast(), because it is JSON-loaded when received anyway. I updated the example code above.

I didn't know at all, too😂


[Console]::WriteLine raise an Exception: 'Multiple ambiguous overloads found for "WriteLine"...'

Prerequisites

[X] Write a descriptive title.
[X] Make sure you are able to repro it on the latest released version
[X] Search the existing issues.
[X] Refer to the FAQ.
[X] Refer to Differences between Windows PowerShell 5.1 and PowerShell.

Steps to reproduce
I tried to run the next script using WriteLine method in v7.4:
Function Test {
       $i = 3
       $j = 0
    [Console]::WriteLine("j={0} i={1}", $j,$i)
    }
Test

Expected behavior
j=0 i=3

Actual behavior
Error Exception is raised:

Error details
MethodException:
Line |
   4 |      [Console]::WriteLine("j={0} i={2}", $j,$i)
     |      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | Multiple ambiguous overloads found for "WriteLine" and the argument count: "3".

The workaround solution to fix this issue is using splat parameters as :
 $args= @($j,$i)
 [Console]::WriteLine("j={0} i={1}",$args)  # working fine

args in c# is params type.

### Environment data

```powershell
PS C:\Program Files\PowerShell\7> $PSVersionTable

Name                           Value
----                           -----
PSVersion                      7.1.4
PSEdition                      Core
GitCommitId                    7.1.4
OS                             Microsoft Windows 10.0.19042
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           <IP_ADDRESS>
WSManStackVersion              3.0

Visuals
No response

Try to use string type for formatting
Function Test {
       $i = 3
       $j = 0
    [Console]::WriteLine("j={0} i={1}", "$j","$i")
    }
Test

or
Function Test {
       $i = "3"
       $j = "0"
    [Console]::WriteLine("j={0} i={1}", $j,$i)
    }
Test

/cc @daxian-dbw for information. I think we could better pick up a method by signature.

@237dmitry, I think that the provided solution is another workaround ones  by converting data type to string.
It's supposed that  Powershell  use .Net methods as used in c#.
It's strange for powershell to fire this exception.
have a look to these cases:
1- [Console]::WriteLine("{0} {1}",10, [long]20)  #working, just cast one parameter to long
2- [Console]::WriteLine("{0} {1}",10, 20.0)  #working
3- [Console]::WriteLine("{0} {1}",10, [int]20)  #NOT WORKING !!! see working case 1
4- [Console]::WriteLine("{0} {1}",10, 20)  #NOT WORKING !!! see working case 2

My understanding as why [Console]::WriteLine("{0} {1}",10, 20) doesn't work is because it cannot differentiate between these 2 choices:
static void WriteLine(string format, System.Object arg0, System.Object arg1)
static void WriteLine(char[] buffer, int index, int count)

When you cast any of the 2nd values to something that isn't an int it's able to determine the first option is correct but if you have a string (automatically casts to char[]) and 2 integers both overloads can technically work with PowerShell.
I'm not saying which behaviour is correct as I feel there's no real correct answer personally; people will have different opinions.

@jborean93, I agree partially with you :), there is other one should be taken into account with params object[]
The signaure of  Console.WriteLine in .Net(core) framework is:
//ideal selection method for all types including int and undefined numbers of parameters
public static void WriteLine(String format, params Object[] arg)    //why ignored by powershell
public static void WriteLine(String format, Object arg0, Object arg1)
  //other overload methods

The next is working fine by casting int to object:
[Console]::WriteLine("{0} {1}",[object]10,  [object]20)   //working fine

So, signature with params object[]  may  be  the ideal for such a method.

From the specified arguments, there is no reason for PowerShell to think (String format, params Object[] arg) would be the best
match. With [Console]::WriteLine("{0} {1} {2} {3}",10, 20, 30, 40), powershell will choose (String format, params Object[] arg) as the best match.
The main ambiguity in the case of [Console]::WriteLine("{0} {1}",10, 20) is that the first argument is a string, which exactly matches (String format, ... but could be converted to char[], while the rest 2 arguments are of [int] , which exactly match int index, int count but could be converted to object. There is no strong argument to say which overload is the best fit in this case -- both have partial matching parameter types -- and thus the resolution failed. I believe this is by design.

@daxian-dbw
Although this code is running without issues in c# (compiler/or scripter), It's supposed that PowerShell  run .Net(Core) methods in the same way as c# (Except there is a document shows the  limitation or consideration to run .Net(Core) methods  in PowerShell)
What is the official way to run such a code without issue ?

The supported way for clarifying overloads is by casting.  Even if no cast actually happens, the compiler and method binder take it into account as intent when selecting the best candidate.
So for this I'd say probably:
[Console]::WriteLine([string]'{0} {1}', 10, 30)

Is the easiest way to clarify intent.

It's supposed that PowerShell run .Net(Core) methods in the same way as c#

PowerShell gives you the ability to interop with CLR methods but I'm not aware of any documentation that claims that it will be the same experience as C#.  There are many ways where it differs significantly (dynamic binding, type conversions are done for you, etc).

@SeeminglyScience
It seems that the direct cast of string as string is the only way to avoid the issue as you showed.
This code, although I show the intent that $format is string, is firing an Exception:
[string] $format='{0} {1}'
[Console]::WriteLine($format,10,20); #Fire Exception

The next is IL generated from LINQPad, shows that Console.WriteLine(String, Object, Object) is selected
IL_0000	nop
IL_0001	ldstr	"{0} {1}"
IL_0006	ldc.i4.s	0A  // 10
IL_0008	box	Int32
IL_000D	ldc.i4.s	14  // 20
IL_000F	box	Int32
IL_0014	call	Console.WriteLine (String, Object, Object)
IL_0019	nop
IL_001A	ret

@SeeminglyScience It seems that the direct cast of string as string is the only way to avoid the issue as you showed. This code, although I show the intent that $format is string, is firing an Exception:
[string] $format='{0} {1}'
[Console]::WriteLine($format,10,20); #Fire Exception

Correct, it's the explicit cast at the call site that signals intent to the method binder.

The next is IL generated from LINQPad, shows that Console.WriteLine(String, Object, Object) is selected

Yeah, C# is statically bound (overload is selected at compile time rather than runtime like PowerShell) and has much more strict type conversion rules.  PowerShell lets you implicitly convert from string and char[] where C# requires you to call ToCharArray().
From the perspective of PowerShell, string, object, object is just as likely as char[], int, int due to conversion rules.  In C#, the former is the only possibility.

Thanks @SeeminglyScience  for clarification.
As you said, explicit cast at the call site  is the solution for such issues even for string arguments.


PCF8574

Dear friends,  thanks a lot for porting to python 3. I tried to run the following code:
from Adafruit_GPIO. PCF8574 import PCF8574,OUT
from time import  sleep

expander=PCF8574(address=0x38)
expander.setup(0,OUT)

while True:
    expander.output(0,True)
    sleep(0.5)
    expander.output(0,False)

and caught an exception
Traceback (most recent call last):
  File "/share/rover/trash/PCF8574_test.py", line 6, in <module>
    expander.setup(0,OUT)
  File "/root/rover3/lib/python3.4/site-packages/Adafruit_GPIO-1.0.0-py3.4.egg/Adafruit_GPIO/PCF8574.py", line 68, in setup
  File "/root/rover3/lib/python3.4/site-packages/Adafruit_GPIO-1.0.0-py3.4.egg/Adafruit_GPIO/PCF8574.py", line 71, in setup_pins
AttributeError: 'dict' object has no attribute 'iteritems'

I think this is issue.
pep-0469

Doesn't this just need the change of .iteritems() to .items() in code?  I think this is a python2 to python3 migration issue?


limit possible values for parameters + required vs optional parameters

Hi,
I'm going to be honest, I'm a newbie at Go development and I mostly used Python + its argparse module for my commandline tools.
I've been going through the documentation, but I'm struggling with a few concepts:

How can I limit a certain commandline parameter to certain values? Eg: I have a commandline parameter "--type" which can only accept the following values: picture,sound or video. so "--type=video" would be valid, but "--type=program" will not. Is there a way to specify these constraints? Python's argparse module can solve this with the "choices" option.

How can I dictate a certain commandline should always be specified? Eg: "--type" should always be specified, with a value.

Hi @jeroenjacobs1205!

Restricting flag options is not directly supported by this library currently, but is achievable via the exposed GenericFlag interface (declaring a struct implementing the GenericFlag interface that validates the value during the .Set() operation).
Required flags have been the topic of much discussion, but are also not currently directly supported by the library (see #85 for more) . You can achieve similar results by validating that the flags are set after parsing using Context.IsSet..

Thank you for your interest! Just let me know if any of this is unclear.
I'll mark this as an enhancement with regard to the flag value restriction.

Given that this is from last year, I think I'm comfortable closing it 🙂 feel free to re-open / open a new issue / comment in support if there's still interest here!

/reopen /open /re-open

@abdennour I re-opened and pinned #1154 (so it won't go stale), let me know if there's anything from this issue that warrants re-opening in addition.

Thank you @rliebz  ! should this answer  https://github.com/urfave/cli/issues/602#issuecomment-416000118 cover this issue ?

I think so, seems pretty straightforward.


Range-ify std.path.relativePath by adding toRelativePath()

http://dlang.org/phobos/std_path.html#.relativePath
This means that relativePath is no longer CTFE'able due to a bug in std.range.choose() which is not CTFE'able. I don't think that actually matters, because it is usually used with getcwd() which is not CTFE'able, either.

https://issues.dlang.org/show_bug.cgi?id=14660

Auto-merge toggled on

Why was only one of the arguments rangified?


Signal server is spammy

Signal server is very chatty. Under normal circumstances it seems like more messages are received than expected, and under certain circumstances (under load?) bursts of messages are sent.
Symptoms:

In devtools see events for current peer from signal server
Frequency is high (e.g., every second)
Ability to correlated device key to members

The current hypothesis is that this is not the cause of real-world issues, but is not desirable behavior and should be investigated at some point.

Downprioritized for after Strange Loop

couple of fixes in the realm of spamming:

[x] make sure dxrpc knows how to stop streams returned by signal Join method https://github.com/dxos/kube/pull/923
[x] ignore multiple announces from different signals https://github.com/dxos/kube/pull/928

Signal messages are often ratelimited, but suspect this is a result of other issues
See https://app.datadoghq.com/dashboard/pb9-h7u-5g2/signal-server-messages?fromUser=true&refresh_mode=sliding&from_ts=1712000139397&to_ts=1712003739397&live=true


script DT does not honor the program visibility setting

Consider the following scenario:

A script installer (i.e. an .exe installer) does not need user interaction when run but does show progress or non-interactive dialog boxes
An Application is created with a Deployment Type of Script Installer with the script installer from Step 1
For the Deployment tab of the Application in Step 2, the Installation program command line under the Programs tab of does not contain any switches that suppresses UI or causes the program to run hidden or quiet
For the Deployment tab of the Application in Step 2, under the User Experience tab of the Deployment Type the Installation program visibility is set to Hidden
Application is deployed to a PC

there is no single way to suppress UI installation screens, and so the script DT does not honor the program visibility setting. The setting only works in the context of an MSI deployment type, because MSI packages allow us the ability to suppress their UI.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: db758241-5a9e-47dc-0104-8d8c1072a2f2
Version Independent ID: d54013ba-afcb-6795-9fb5-3668a8afec32
Content: Create applications - Configuration Manager
Content Source: sccm/apps/deploy-use/create-applications.md
Product: configuration-manager
Technology: configmgr-app
GitHub Login: @aczechowski
Microsoft Alias: aaroncz

Note that if you select "Run installation and uninstall program as a 32-bit process on 64-bit clients" then the "Allow users to view and interact with the program installation" setting IS honored (by launching the installer in session 0). It is only if you select to run the installer as 64-bit that the setting is not honored (the installation runs in the user's logon session).

Thanks again for the feedback. We recently moved this content to a new GitHub repository, so this issue is no longer tracked with the original article. As part of that process, we reviewed our backlog of feedback, and made some tough decisions on items that we don't think we'll realistically ever address. If you're passionate about this specific feedback, please refile it on the current article, now under docs.microsoft.com/mem/configmgr.


Compare Versions

Compare files with previous versions.
If hash hasn't changed, pull the file forward (create a symlink?)
Skip downloading file

Optional: Display verbose list of what files have changed.

Probably should address this in hash_check branch.


fix 1.7 build compatibility

lgtm

Current coverage is 88.12% (diff: 100%)

Merging #8 into master will not change coverage

@@             master         #8   diff @@
==========================================
  Files             8          8
  Lines           278        278
  Methods           0          0
  Messages          0          0
  Branches         41         41
==========================================
  Hits            245        245
  Misses           24         24
  Partials          9          9

Powered by Codecov. Last update 353942c...0b38add


libopae: merge bmc fixs from release branch

changes:
merge bmc fixes from dcp 1.1.1 release branch

@anandaravuri could you run valgrind to verify these changes don't cause any regressions?


perf(internal_format): optimize gq for long lines

I was editing a relatively big text file with a single line (4 Mb). I tried to format the line with gq and neovim stuck. Profiling with callgrind, I found out that the culprits were the n^2 nature of the algorithm (going backwards from the end of the line each time) and the expensive utf check for whether the line is longer than textwidth.
I implemented kind of hacky workarounds for both problems, involving a magic 8 * textwidth constant, and the time is now relatively ok (the new bottleneck is open_line, but I won't go there).
Benchmark 1: long lines
python -c "print((('a' * 79 + ' ') * 750)[:-1])" > line60000.txt
time vim -u NONE -c 'normal gqgq' -c 'q!' line60000.txt
Original:
real	0m0.420s
user	0m0.398s
sys	0m0.020s

Optimized:
real	0m0.031s
user	0m0.013s
sys	0m0.017s

The difference only grows with the line getting longer.
Benchmark 2: lots of small lines:
yes $(python -c "print('a' * 79 + ' a')") | head -n 10000 > lines10000.txt
time vim -u NONE -c 'normal ggVGgq' -c 'q!' lines10000.txt
Original:
real	0m0.095s
user	0m0.084s
sys	0m0.010s

Optimized:
real	0m0.094s
user	0m0.083s
sys	0m0.010s

I manually tested to see whether 8 * textwidth limit made the difference on the actual formatting done and found none.
This was originally a PR to neovim, but I got recommended to contribute here first.
Interestingly, in Neovim my changes slow down the many lines case, but in Vim there is no difference.

Nice, thank you.

Original:
real	0m0.095s
user	0m0.084s
sys	0m0.010s

Surely that one is not the original one. Copy/paste error?

ah of course. I read too fast.
Thanks for that!