presigned urls support

hello, is there any plan to generate presigned urls for arbitrary operations (get, put, etc)? Also, the readme mentions that the put operation returns a presigned url, but I get an empty vec instead, is it a common issue?

@Geal thank you for submitting this, its quite embarrassing as the presigned URLs are the cause I built this lib, I'll look int this right away, will keep you posted!

Hello. Any updates on this? I’m curious about capabilities around pre-signed URLs too. Specifically, I'd like to get pre-signed URLs for pre-existing objects in the bucket too!

@sidshank this turned out to be more broken than expected, looking at my schedule I should be able to push a fix through by the end of the month, a bit later than I would have liked... Thank you for the patience

Any idea when this might be resolved @durch ? Pre signed urls would really make this library be more useful as it is a far better way of uploading/downloading files for the user.

@elertan thanks for the nudge, my schedule is a bit messy, I'll try and squeeze some work towards this in the next week or two, will update the issue as soon as I a have a better estimate, sorry for the inconvenience :/

@Geal, @elertan version 0.22.8 pushed to crates.io today brings back support for presigned URLs, with two dedicated Bucket methods presign_put and presign_get


Add Faker::Types.boolean

This allows you to randomly generate a boolean by calling Faker::Types.boolean
I used Integer#even? instead of Integer#zero? because Integer#even? has existed for a far longer time in the ruby library.
Thanks for the library!

I believe there's a generator for booleans already, in another module, maybe you could implement an "alias"?

You are correct, I missed Faker::Boolean 😔  I'll close this PR out and use that.
Thanks for the extra 👀 !

You could add an alias 😜it still makes sense to have it in types, I believe


Improve validation and deduplication in backend lambda.

We want to add some checks to prevent people spamming and overpopulating the databases with unnecessary data:
Levels

When querying "all levels" we want to paginate the requests (maybe a max of 10 at a time?)
We might also want some sorting options / categories of search

Recently added
Highest/lowest rating
Most scores/played

Scores

When querying scores we want to return the 5 highest (smallest time) scores
We also only want to return one score per user (best score only) to prevent a user taking up the whole leaderboard with their 5 best scores.

Ratings

There should only be one rating per level, per user. If someone leaves an additional rating to a level that was already rated we want to update the previous rating rather than create another.

Okay, we could do all of these or a select few of them. I'm breaking these down so we can prioritize. My questions to you are:

Which ones do we definitely need? (Versus "nice-to-have")
Which should I prioritize first?

Units for Work:
1. When querying "all levels" we want to paginate the requests (maybe a max of 10 at a time?)
I think we already have this actually: https://github.com/LPGameDevs/EditarrrPublic/blob/develop/backend/lambda/function.mjs#L126
2. Level Recently added sorting options
To be clear: we already sort by "last updated". So this is to sort by "last created"? And we can easily support ascend vs descend.
3. Level Recently added filter option
Again, just clarifying: this is to limit the search to "levels created before/after X"? And/or "levels updated before/after X?" All of the above should be easily doable.
4. Level Rating sorting option
This would mean that we would need to average all the ratings for each level, so we could then sort by it, right? I could see this being a bit tricky....off the top of my head, we could:

Calculate on WRITE: Calculate the rating average for a level and update it on the Level data itself whenever a rating is submitted.
Calculate on READ: Calculate the averages for all the levels at read time, then sort by it. This seems very expensive and not really any simpler than "On Write". The advantage is a miscalculation of the average won't persist and would be an easy bug fix (fixing it for "on write" would involve overwriting the miscalculated value).
Calculate async: Have some process in the background that calculates the average.

Approach (1) Calculate on WRITE seems like the simplest to me.
5. Level Rating filter option
Similar implementation questions/thoughts as 4.
6. Level # of scores sorting option
Similar implementation questions/thoughts as 4.
7. Level # of scores filter option
Similar implementation questions/thoughts as 4.
8. When querying scores we want to return the 5 highest (smallest time) scores

When querying scores

Querying scores for a single level, correct? Or, for a player?
Assuming it's for a level, this seems pretty simple with the scoreLevelName-score-index GSI.
9. Return one score per user (best score only) to prevent a user taking up the whole leaderboard with their 5 best scores.
This is maybe a bit tricky. Two (maybe 3) approaches I can think of off the top of my head:

DynamoDB might have some query options that could make a query fulfills the reqs (scores with unique user IDs sorted by highest), but I'm skeptical that this exists
Assuming we want a page size limit of 10, we query for the top 10 scores, de-dupe, then query for more if de-duping resulted in <10 scores. Alternatively, query an excess # (100 or something), then de-dupe and limit the results, querying for more if necessary. A bit of a brute-force approach and maybe a bit slow if a single user has a lot of the high scores.
We could add a sk: USER#<userId> to the level (or the scores) table and accumulate score (and even rating) data about the user per level in that object whenever there's a score/rating update. This has discrepancy/maintenance risks of de-normalized data.

I think I would start with approach (2) (assuming (1) doesn't exist). If we find that isn't performant enough (I'm assuming it'll be fine given our small scale of data), we could look into (3).
Thought:
Between this and #10, this makes me consider whether we might actually want to restructure the DDB tables so that instead of two pk: LEVEL#<levelId> sk:RATING#<ratingId>/sk:SCORE#scoreId> tables, we have a single table with pk: LEVEL#<levelId> and sk: USER#<userId>, then store ratings and scores on the "user" objects per user per level (this makes de-duping of ratings/scores per level per user easier to accumulate).
However, I don't feel confident enough that this will be beneficial over time to put in the effort to make this change at the moment. But I could pursue this if you think we should.
10. There should only be one rating per level, per user. If someone leaves an additional rating to a level that was already rated we want to update the previous rating rather than create another.
This should be pretty easy to do by doing an "upsert" operation by checking, and creating if there isn't already a rating for the user for the level, and doing an update otherwise.

Notes from our discussion phone call:
Decisions
Have the query/filter/sort logic in the server (the alternative is have the game fetch all the levels upfront and do the query logic). Reason being:
(a) I can help if its in the backend (the logic has to live somehwere)
(b) it scales better (not expecting this to be a priority but a small benefit)
For (9), keep track of the best score on writes
Clarification
(1) we have a limit, but no pagination logic
Priorities
We aren't certain exactly which filter/sort options would be ideal, but the ideal is to create the best player experience and facilitate level discovery. We don't want the player to be overhwlemed with all the levels, but we want to help them find the "right ones" oto play/try. We'll get more player feedback soon (early Nov release), but we're getting ahead of the curve now, so I'll use my best judgment about priorities.
Along these lines, we should prioritize changes that affect the data storage (those will be harder to introduce later on when the game is live). This means work such as calculating a cached average score for a level should be prioritized
Scores for leaderboards probably the highest priority (most impactful upfront)
Future
For now, the strategy is: AWS is the data store (maybe Steam someday)
We're going to want to give awards on various measurements, but not sure what those are yet. Ideas include highest rated level, most played, player who played the most, most "competitive" level, etc. To that end, we should keep any data we think is interesting or relevant, so we can give awards on these various vectors
We'll optimize in the future (costs, performance, etc)
We are NOT storing user data right now, because (a) the overall ethos of making the game accessible & not requiring accounts/login (b) we aren't certain what we want to do with "users" in the future (c) we can always create a virtual user by querying all the data pertaining to a any userId/username and aggregating the data together
Follow-Up Thought
One thought we only touched on briefly, but I'm thinking is something worth adding to the list is the ability to "search" for a level - by name or creator or something like that. I could see there being a use case for "hey, I know yan made a level and I want to play it, give me all the levels created by player 'yan'". I'm going to add this as an item to the Units of Work list

Added another item: Level's by User Filter Option

After going through all this, I only have a few additional suggestions for filtering options:
-Remove all levels a user has previously downloaded
-Remove all levels currently downloaded (present on local machine)
-Remove all levels a user has beaten (could be tied to score submission)
Keep up the great work, Backenders! 🥇👍

@CalmPewter Thank you for the feedback!

Remove all levels a user has previously downloaded
Remove all levels currently downloaded (present on local machine)

Implementation-wise, I think we would most likely want to do this filtering client-side (in the game itself), because the client knows what is downloaded (I don't think we should have the backend DB keeping track of that). So the Editarrr game fetches all the levels, then filters out ones already downloaded (and can make a subsequent request(s) for more levels if necessary).
I'm not going to add this to the list I have here since I'm keeping this issue backend focused, but maybe we can/should create a separate issue for adding this filtering in the game?

Remove all levels a user has beaten (could be tied to score submission)

Good idea, I'll add this to the list!

I'm closing this Issue as "Done" - split out some of the remaining tasks into separate Issues:

https://github.com/LPGameDevs/EditarrrPublic/issues/237
https://github.com/LPGameDevs/EditarrrPublic/issues/236


Parsing status occasionally wrong

I'm noticing under some conditions, dps (docker-pretty-ps) reports containers as running when they are exited. This appears to be an issue with revised_line_split[4] on line 227.
Here's my output when I add this line to _clean_status(val)
    print('STATUS: {} [{}]'.format(val, val.lower().strip()))

STATUS:  Exited (0) About an hour ago [exited (0) about an hour ago]
STATUS:  Exited (1) About an hour ago [exited (1) about an hour ago]
STATUS: Created [created]
STATUS: Exited (127) 4 days ago [exited (127) 4 days ago]
STATUS:  4 days ago [4 days ago]
STATUS:  4 days ago [4 days ago]
STATUS:  4 days ago [4 days ago]
STATUS:  4 days ago [4 days ago]
STATUS:  4 days ago [4 days ago]
STATUS:  4 days ago [4 days ago]
STATUS: Exited (1) 4 days ago [exited (1) 4 days ago]
STATUS: Exited (1) 4 days ago [exited (1) 4 days ago]
STATUS: Exited (1) 4 days ago [exited (1) 4 days ago]

...(truncated)...
Total containers:	33
Total running:		6

Those 6 running appear to be coming from the failure to split the output properly. Here's some sample output that causes the splitting to fail.
CONTAINER ID        IMAGE                                                                                COMMAND                  CREATED             STATUS                         PORTS               NAMES

c17fa80d9493        79332bd117f0                                                                             "/bin/sh"                4 days ago          Created                                            pycharm_helpers_PY-183.5912.18
fdbf9b4b6df4        a9b6bbef0884                                                                             "/bin/sh -c 'curl ht…"   4 days ago          Exited (127) 4 days ago                            practical_keller
99f0f77fe65a        7dc5fcb2d5fd                                                                             "/bin/sh -c 'ln -s  …"   4 days ago          Exited (1) 4 days ago                              objective_wright
820a1a39c1b1        7dc5fcb2d5fd                                                                             "/bin/sh -c 'ln -s  …"   4 days ago          Exited (1) 4 days ago                              relaxed_chaum
b1edb9b13abb        7dc5fcb2d5fd                                                                             "/bin/sh -c 'ln -s  …"   4 days ago          Exited (1) 4 days ago                              serene_gagarin
b33e9c1a90ba        7dc5fcb2d5fd                                                                             "/bin/sh -c 'ln -s  …"   4 days ago          Exited (1) 4 days ago                              nostalgic_brown
caba00f3f0be        7dc5fcb2d5fd                                                                             "/bin/sh -c 'ln -s  …"   4 days ago          Exited (1) 4 days ago                              competent_ganguly
f875a1053e5c        74d93d2436c7                                                                             "/bin/sh -c 'ln -s  …"   4 days ago          Exited (1) 4 days ago                              awesome_mayer
50c2796bd7e1        1da54eb8ec63                                                                             "/bin/bash -c 'sourc…"   4 days ago          Exited (1) 4 days ago                              practical_sinoussi
f9120b6d9d74        5205d6f43877                                                                             "/bin/bash -c 'sourc…"   4 days ago          Exited (1) 4 days ago                              quizzical_golick
8bf6c7662fd8        b12e22934b40                                                                             "/bin/bash -c 'sourc…"   4 days ago          Exited (1) 4 days ago                              angry_einstein
7f230bb18ddf        e3ea4e8d8e34                                                                             "/bin/sh -c 'ln -s $…"   5 days ago          Exited (1) 5 days ago                              friendly_feistel

Oof. Was planning to rewrite this using go templating. I'll try and use this as a use case for that. If you've got a PR in the meantime I'd be happy to bake it in.


WithViewStore or @ObservedObject var viewStore:.  doesn't update view [iOS 13.1]

Describe the bug
State updates, but view doesn't
To Reproduce
Test.zip
Expected behavior
I expect view to be updated on state update inside WithViewStore
Environment

Xcode 11.6 (11E708)
Swift 5.2.4
iOS 13.1

** UPD:

Doesn't work with TCA 0.6.0, but works fine with 0.5.0 version (If I set Exact version)

Hey @romiroma, thanks for the report! I was able to reproduce this locally, and it's definitely a problem with iOS 13.1, and 13.2, and then due to some Combine/SwiftUI fixes in 13.3 it works correctly.
The bug you are seeing is due to this PR https://github.com/pointfreeco/swift-composable-architecture/pull/199 where we changed some internals of ViewStore in order to close up an escape hatch that was created in some of the new iOS 14 APIs. However, Apple has since fixed that escape hatch, and so maybe we should revert this now, especially since it breaks older iOS.
Stephen and I will discuss today and come up with a plan. Thanks again!

As an aside, I came across some other weird behavior in just iOS 13.1. Our TestScheduler crashes because it seems the mere act of comparing two scheduler times results in a crash in Combine 😬:
DispatchQueue.SchedulerTimeType(.init(uptimeNanoseconds: 1)) <= DispatchQueue.SchedulerTimeType(.init(uptimeNanoseconds: 2)) // 🛑 crash

So even once we fix the issue you reported you may wanna upgrade to iOS 13.2 so that you can use the test scheduler.

Thank you, guys, for fast response! 💪👍

@stephencelis For me, the problem persists with the current release (0.8.0) on iOS 13.2.2. This can be verified by using the test project created by @romiroma (Test.zip), updating the TCA package to the current release, and running it on an iOS 13.2.2 simulator.

@manmal @mbrandonw @stephencelis I'm also running into this with iOS 13.2. 13.1 and 13.3 work for me but I can replicate this bug in the simulator and on device.

I confirm this is still happening on iOS 14. and release 0.8.0.
As a workaround I did this
extension Publisher {
  // there seems to be an iOS 14 specific issue where sometimes
  // updating UI right after receiving events would fail.
  // adding an insignificant delay (postponing to next run loop) seems to fix it
  func jf_runLoopdelay(scheduler: RunLoop = RunLoop.main) -> AnyPublisher<Self.Output, Self.Failure> {
    if #available(iOS 13.2, *) {
      return delay(for: .milliseconds(1), scheduler: scheduler).eraseToAnyPublisher()
    } else {
      return eraseToAnyPublisher()
    }
  }
}

I confirm this is still happening on iOS 14. and release 0.8.0.

@apouche Do you mean it's still happening on iOS 13.2? Your code seems to suggest that you target that specific OS for the workaround.

I put 13.2 because other users mentionned this still occuring to them and this would solve their problem as well. However to me this is really an iOS 14 specific issue.

@apouche And the code you shared fixes it? I'm not sure I understand, since wouldn't it be the same as calling eraseToAnyPublisher() on iOS 14?
If possible could you provide an example project that exhibits the behavior along with instructions to reproduce?

I'm seeing this behavior in the 13.1 simulator when running the Case Studies (SwiftUI) after changing the deployment target:
IPHONEOS_DEPLOYMENT_TARGET = 13.1.
None of the case studies work though the reducers are firing. This is off of the main branch at c1f88bd8608e8a26fd61ab46c9f11a63b0f4023d with only the following diff:
diff --git a/Examples/CaseStudies/CaseStudies.xcodeproj/project.pbxproj b/Examples/CaseStudies/CaseStudies.xcodeproj/project.pbxproj
index d15751f..c3ef826 100644
--- a/Examples/CaseStudies/CaseStudies.xcodeproj/project.pbxproj
+++ b/Examples/CaseStudies/CaseStudies.xcodeproj/project.pbxproj
@@ -1099,6 +1099,7 @@
 				CODE_SIGN_STYLE = Automatic;
 				ENABLE_PREVIEWS = YES;
 				INFOPLIST_FILE = SwiftUICaseStudies/Info.plist;
+				IPHONEOS_DEPLOYMENT_TARGET = 13.1;
 				LD_RUNPATH_SEARCH_PATHS = (
 					"$(inherited)",
 					"@executable_path/Frameworks",
@@ -1117,6 +1118,7 @@
 				CODE_SIGN_STYLE = Automatic;
 				ENABLE_PREVIEWS = YES;
 				INFOPLIST_FILE = SwiftUICaseStudies/Info.plist;
+				IPHONEOS_DEPLOYMENT_TARGET = 13.1;
 				LD_RUNPATH_SEARCH_PATHS = (
 					"$(inherited)",
 					"@executable_path/Frameworks",

This issue should probably be re-opened

Also happens on Simulator 13.2 in the Case Studies (SwiftUI).
cc: @mbrandonw @stephencelis

@mbrandonw @stephencelis  I'm happy to get this reproducible in a tiny app or cut a branch with a failing test-case. Also happy to open a new bug. Let me know what'd be the most useful to y'all

FWIW, the problem didn't occur for me on iOS 13.3 and higher. On 13.2.2, it's easily reproducible with my sample project above.

Hi has there been a fix for this issue? I'm seeing it running iPhone 12 Pro - 14.2 Sim

I figured out my problem. Be sure your state structs implement the Equatable protocol correctly. I had hard coded a return value of true and I suspect when you change state there is a comparison done to determine if a view update is warranted.

@eqwalker The Equatable conformance for the test project posted at the beginning of this thread is certainly not to blame:
struct State: Equatable {
    var count: Int = 0
}

So I suspect that your finding only fixes the problem for the 14.2 sim.
Would you mind checking whether the problem occurs on iOS <13.3 (e.g. 13.2.2) for you?

@manmal looks like you're right. It started happening for me again iOS 14.2 iPhone 12 Pro Sim. :-(. Doesn't seem to be related to Equatable....

@mbrandonw @stephencelis can we re-open this, or would you like a new issue?

I've opened a new issue: #334
cc: @eqwalker @manmal

Thanks so much @nsillik, this fixes the issue for us 🎉


[BUG] Navbar links are not working [GSSoC'23]

Is there any existing issue for this?

[X] I have searched the existing issues

Description
Navbar links are not working
Can you assign this issue to me, I want to work on this
Screenshots
No response
Any additional information?
No response
What browser are you seeing the problem on?
Microsoft Edge

Please assign this to me!

I have fixed the About us link but the pages directory does not have any page for events and team

hey @Sukhendu2002 we are working on the teams page
You can refer to this issue for further discussion #61
If you have any design in your mind you can share it in that issue so it will not create any chaos
Rest you can add a scroll feature for the home,about and events page

Thank you, @himanshu1221, for assigning this issue to me. I will begin working on it promptly.


TOR access denied

when trying to install a file, everything goes fine until it has to use tor, says it's not in my PATH but it's installed

which tor
/usr/bin/tor

Does this problem still persist? I added some hints how to install Tor into README:
*Linux: apt get install tor

Windows: install TorBrowser and add directory with tor.exe to system %PATH%

Old issue, but info for others: See updated README, the part about setting Windows %PATH% was updated.


data file override support with precedence

Fix #891

Provides a custom path accessor for project data files to allow user's to override project provided datasets using $XDG_DATA_DIR/garak/data to mirror garak/data from the package.
Separates non-code data or dataset files from source code
Downloaded datasets are considered cached files and do not change location or behavior in the revision
Removes argument parser no longer supported by gcg

Consumers of the garak.data.path object can traverse the data directory as a standard Path object and will be returned existing files based on the order precedence from the supported locations on disk.
A possible drawback to the current implementation exists regarding directory interaction, if a folder exists at more than one path location the higher precedence path will be returned, when performing glob or iteration actions with such an instance the only files in the selected path on disk will be evaluated.  This is currently considered an acceptable limitation.
Verification
List the steps needed to make sure this thing works

[ ] Run the tests and ensure they pass python -m pytest tests/
[ ] garak -m nim -p donotanswer.MisinformationHarms -g 1 -n meta/llama3-70b-instruct --parallel_attempts 5
[ ] create a limited data set for donotanswer.MisinformationHarms

mkdir -p ~/.local/share/garak/data/donotanswer
head -n 10 garak/data/donotanswer/misinformation_harms.txt > ~/.local/share/garak/data/donotanswer/misinformation_harms.txt

[ ] garak -m nim -p donotanswer.MisinformationHarms -g 1 -n meta/llama3-70b-instruct --parallel_attempts 5
[ ] Verify run results:

[ ] the first run reported 155 attempts
[ ] the second run report 10 attempts (count depends -n count passed to head)

Note: #870 provides new data files that will need to migrate to this pattern depending on PR landing order.

Looks like there is some work to do with support for this on python 3.10 apparently!

A possible drawback to the current implementation exists regarding directory interaction, if a folder exists at more than one path location the higher precedence path will be returned, when performing glob or iteration actions with such an instance the only files in the selected path on disk will be evaluated. This is currently considered an acceptable limitation.

Flagging a tension here with payload enumeration. Currently payload enum globs but uses its own directory traversal code, rather than garak.data. The result is that this PR doesn't break payload enum, but does leave us with two bits of code using similar-but-not-identical logic.

Flagging a tension here with payload enumeration. Currently payload enum globs but uses its own directory traversal code, rather than garak.data. The result is that this PR doesn't break payload enum, but does leave us with two bits of code using similar-but-not-identical logic.

Depending on land order I am thinking this can be a use case to improve glob support on the custom Path objects to return a consolidated list of files based on searching all ORDERED_SEARCH_PATHS. This would then remove the similar-but-not-identical logic by supporting the need quickly as a next iteration.

NICE


Ensure basic auth when getting manifest

Include basic auth header when getting manifest, avoiding login failure with Nexus OSS repo.
Nexus OSS responds Sonatype Nexus Repository Manager as realm in the WWW-Authenticate header and does not play well with current implementation. However, getting a token should not be necessary with basic auth enabled. Go oras client works well already.
Current behavior
Targeting same nexus repository, that does not allow anonymous manifest/pull and requires basic auth.
oras CLI
oras pull my-nexus.example.com/my/image.db:latest
Downloading 394458a69d7a image.db
Downloaded  394458a69d7a image.db
Pulled my-nexus.example.com/my/image.db:latest
Digest: sha256:56d0ba1e3806f3af6b75d88eab8fe82f83d66b899e06efcda8d4eefa07b66e38

Test client
import logging
import oras.client

HOST='my-nexus.example.com'
IMG='my/image.db:latest'

logging.basicConfig(level=logging.DEBUG)

client = oras.client.OrasClient(hostname=HOST, insecure=False)
client.pull(target=IMG)

oras-py (with test client)
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): my-nexus.example.com:443
DEBUG:urllib3.connectionpool:https://my-nexus.example.com:443 "GET /v2/my/image.db/manifests/latest HTTP/1.1" 401 113
DEBUG:oras.logger:No Authorization, requesting anonymous token
DEBUG:oras.logger:Final params are {}
INFO:oras.logger:Retrying in 3 seconds - error: Invalid URL 'Sonatype Nexus Repository Manager': No schema supplied. Perhaps you meant http://Sonatype Nexus Repository Manager?
DEBUG:urllib3.connectionpool:https://my-nexus.example.com:443 "GET /v2/my/image.db/manifests/latest HTTP/1.1" 401 113
DEBUG:oras.logger:No Authorization, requesting anonymous token
DEBUG:oras.logger:Final params are {}
INFO:oras.logger:Retrying in 5 seconds - error: Invalid URL 'Sonatype Nexus Repository Manager': No schema supplied. Perhaps you meant http://Sonatype Nexus Repository Manager?
DEBUG:urllib3.connectionpool:https://my-nexus.example.com:443 "GET /v2/my/image.db/manifests/latest HTTP/1.1" 401 113
DEBUG:oras.logger:No Authorization, requesting anonymous token
DEBUG:oras.logger:Final params are {}
INFO:oras.logger:Retrying in 11 seconds - error: Invalid URL 'Sonatype Nexus Repository Manager': No schema supplied. Perhaps you meant http://Sonatype Nexus Repository Manager?
DEBUG:urllib3.connectionpool:https://my-nexus.example.com:443 "GET /v2/my/image.db/manifests/latest HTTP/1.1" 401 113
DEBUG:oras.logger:No Authorization, requesting anonymous token
DEBUG:oras.logger:Final params are {}
INFO:oras.logger:Retrying in 29 seconds - error: Invalid URL 'Sonatype Nexus Repository Manager': No schema supplied. Perhaps you meant http://Sonatype Nexus Repository Manager?
DEBUG:urllib3.connectionpool:https://my-nexus.example.com:443 "GET /v2/my/image.db/manifests/latest HTTP/1.1" 401 113
DEBUG:oras.logger:No Authorization, requesting anonymous token
DEBUG:oras.logger:Final params are {}
INFO:oras.logger:Retrying in 83 seconds - error: Invalid URL 'Sonatype Nexus Repository Manager': No schema supplied. Perhaps you meant http://Sonatype Nexus Repository Manager?
DEBUG:urllib3.connectionpool:Resetting dropped connection: my-nexus.example.com
DEBUG:urllib3.connectionpool:https://my-nexus.example.com:443 "GET /v2/my/image.db/manifests/latest HTTP/1.1" 401 113
DEBUG:oras.logger:No Authorization, requesting anonymous token
DEBUG:oras.logger:Final params are {}
Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/tst/.vscode-server/extensions/ms-python.python-2023.4.1/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/__main__.py", line 39, in <module>
    cli.main()
  File "/home/tst/.vscode-server/extensions/ms-python.python-2023.4.1/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 430, in main
    run()
  File "/home/tst/.vscode-server/extensions/ms-python.python-2023.4.1/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 284, in run_file
    runpy.run_path(target, run_name="__main__")
  File "/home/tst/.vscode-server/extensions/ms-python.python-2023.4.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 321, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "/home/tst/.vscode-server/extensions/ms-python.python-2023.4.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 135, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/home/tst/.vscode-server/extensions/ms-python.python-2023.4.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 124, in _run_code
    exec(code, run_globals)
  File "/home/tst/oras-fork/test.py", line 10, in <module>
    client.pull(target=IMG)
  File "/home/tst/oras-fork/oras/client.py", line 137, in pull
    return self.remote.pull(*args, **kwargs)
  File "/home/tst/oras-fork/oras/provider.py", line 741, in pull
    manifest = self.get_manifest(container, allowed_media_type)
  File "/home/tst/oras-fork/oras/decorator.py", line 35, in __call__
    return self.func(cls, *args, **kwargs)
  File "/home/tst/oras-fork/oras/provider.py", line 796, in get_manifest
    response = self.do_request(get_manifest, "GET", headers=headers)
  File "/home/tst/oras-fork/oras/decorator.py", line 60, in __call__
    return self.func(cls, *args, **kwargs)
  File "/home/tst/oras-fork/oras/provider.py", line 845, in do_request
    if self.authenticate_request(response):
  File "/home/tst/oras-fork/oras/provider.py", line 901, in authenticate_request
    if self.request_anonymous_token(h):
  File "/home/tst/oras-fork/oras/provider.py", line 965, in request_anonymous_token
    response = self.session.request("GET", h.realm, params=params)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 528, in request
    prep = self.prepare_request(req)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 456, in prepare_request
    p.prepare(
  File "/usr/lib/python3/dist-packages/requests/models.py", line 316, in prepare
    self.prepare_url(url, params)
  File "/usr/lib/python3/dist-packages/requests/models.py", line 390, in prepare_url
    raise MissingSchema(error)
requests.exceptions.MissingSchema: Invalid URL 'Sonatype Nexus Repository Manager': No schema supplied. Perhaps you meant http://Sonatype Nexus Repository Manager?

oras-py (with test client) in this patch
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): my-nexus.example.com:443
DEBUG:urllib3.connectionpool:https://my-nexus.example.com:443 "GET /v2/my/image.db/manifests/latest HTTP/1.1" 200 513
DEBUG:urllib3.connectionpool:https://my-nexus.example.com:443 "GET /v2/my/image.db/blobs/sha256:394458a69d7a4beb146fbeed181b6646b2dfb583df1b8fc5f71ada8034eb7726 HTTP/1.1" 200 809119
INFO:oras.logger:Successfully pulled /tmp/oras-tmp.4tafaoj1/image.db.

This looks good - could you please bump the version in oras/version.py and add a corresponding line to CHANGELOG.md?

Thanks, added both.


Ordinal NFT Blog Post

Dumping rough notes here:
ordinals are just a global namespace that maps numbers to addresses. addresses can be public keys, and public keys have private keys that can sign challenges to prove ownership, so it maps numbers to owners.
ordinal NFTs
What does an NFT or token even mean? I think of it as being about imbuing something with meaning.
What can be imbued?
Provenance! Provenance is where something came from, or who made it. Art and collectables revolve around provenance. Public key cryptography is a great way to establish provenance. If an artist has a well-known public key, they can sign messages with their private key that nobody can forge.
In a similar vein, sometimes provenance is used to make something redeemable for another. Stable coins, bearer shares, membership etc. All this can be done by signing something.
Ordinals already have the provenance of their number, and thus their origin in a particular bitcoin block. But we want to allow assigning additional provenance.
. But how?

informal agreement
signature with known pubkey
declaration by ordinal owner
timestamping
fully on-chain

There are a few different ways I can think of:

Verbally agree between interested parties to imbue some ordinals with a particular provenance. This happens completely off-chain and without any cryptography. It can be done in high-trust environments, or environments where the provenance matters only to a closed set of people. It is low overhead, and due to its informal nature, requires no special software beyond an ordinal aware wallet.
Create a message imbuing some ordinals with a particular provenance and sign it with a public key known to interested parties.
Create a message imbuing some ordinals with a particular provenance and sign it with a public key and publish it.
Create a message imbuing some ordinals with a particular provenance and commit to it in a transaction that moves those ordinals. Note that the message isn't signed. This commitment isn't meaningful because of a well-known public key. Instead it is meaningful because it was made by a particular ordinal owner.
Like the above, but also sign that message. Now it's a commitment that ties together an ordinal owner and an identity.
Like the above, but just commit to it in any old transaction. Proves age.
Like the above, but also reveal the message on-chain, for example by committing to it in a taproot script, and then revealing it on spend. This is expensive, but makes the data available on-chain. This is expensive, which increases rarity, so good for collectors and art. Publishing data on-chain also prevents equivocation, because the chain can be scanned for publication.

discoverability
limited supply
private vs non-private
can't equivocate
can't deny issuance
can't forge identity
can't forge owner
can prove age

Link types:

On-chain transaction using an ordinal, which reveals data. Link: Ordinal -> data
Off-chain signature using pubkey. Link: Pubkey -> Ordinal + data

This is really a personal blog post, so not a project issue.


How to check if a switch or checkbox input is active/checked?

Hello. I have a form I'm making and I want to only show certain inputs if a switch is active. So I have a variable and I've used bind:this to bind this switch input to the variable, but I'm not entirely sure how to check if the switch is active. Is there even a way to do this?

Hi @AshtonSnapp yes the svelte way of doing this is to bind to the prop you need to watch.  In this case it's the checked prop:


How can I use containers for test and production cases?

In other projects with strongly typed languages (e.g. C#) I've commonly set up interfaces for stuff I wanted to mock during testing. Then I'd Inherit a class for production implementation and another for the mock. However, I've not been able to figure out how to do that with Aurelia/typescript. I've created an interface:
export interface BTAuthService { 	signIn(userName:string, password:string): Promise<any>; }
Which is implemented by MockAuthService class.
I would expect to be able to do this:
`import {BTAuthService} from './serviceInterfaces';
import {MockAuthService} from './account/mockAuthService';
export function configure(aurelia:Aurelia) {
aurelia.use
.standardConfiguration()
.developmentLogging();
let container = aurelia.container;

container.registerSingleton(BTAuthService, MockAuthService);
aurelia.start().then(() => aurelia.setRoot());

}`
but I get the error "cannot find name BTAuthService."
What's the correct way to register a singleton by interface type?
In general, I'd love to see some documentation on putting together a real-world test setup.

This is because TypeScript erases the interfaces during compilation. There is no such thing as an interface in JavaScript, so there is no interface at runtime either.
Bear in mind, you don't need the interface, you can just use the original class but provide a Mock instance. JavaScript is dynamically typed, so it will work perfectly.
container.registerSingleton(TheConcreateType, MyMockType);

Thanks, Rob. I just realized the same thing in a "duh" moment. Your suggestion is exactly what I went with.

No problem. Sometimes with TS it's easy to forget the underlying platform's nature.


refactor: improve linting

This PR enables lit-analyzer with a limited ruleset.

We're currently processing your upload.  This comment will be updated when the results are available.


Windows store version of spotify possibly not working.

Reported on reddit, looking right now to see if that version uses the spotify local api or if they removed it from that version.

Seems to be just a repackaged version of the desktop app, spotify local api works fine.


rewarded_address in StateTransition is hard coded

As for today, reward address is hard coded to be address of sequencer.
There are couple problems with it:

Impossibility to have rewards to be sent to different address rather than sequencer.
It is unclear if rewards are DA layer currency or Rollup currency. If it is rollup currency, then it should be rollup address.

Original PR: https://github.com/Sovereign-Labs/sovereign-sdk/pull/598

I don't have strong opinions about what the reward address should be. I think it should indeed be dynamic instead of hardcoded (specially for the based sequencing feature), and when it comes to the currency I think it doesn't really matter (but need to double check with @preston-evans98 )

Yep, the rewarded address should definitely be dynamic. And it doesn't matter to me which format it is in - either a DA address or the corresponding rollup address is fine. Whichever is most natural for the module that needs to consume it.


Dialog with Select focus issue

How to reproduce

Go to Dashboard example
Click on the top left TeamSwitcher Alicia Koch
Click on Create Team
In the dialog, try to use TAB key to move from the Team name field to Subscription plan select element.

Actual:
Tab does not work properly.
Expected:
Tab should move to Select, another tab should move to Continue -> Cancel -> X -> Input -> Select

same problem here..

This issue has been automatically closed because it received no activity for a while. If you think it was closed by accident, please leave a comment. Thank you.


Classic 8 Ball Pool Game added

PR Description 📜
Please include summary related to the issue you have fixed and describe your PR in brief over here by specifying the issue number on which you were working below
I have added a new game "Classic 8 Ball Pool Game
Fixes # <352>

Mark the task you have completed ✅

[x] I follow CONTRIBUTING GUIDELINE & CODE OF CONDUCT of this project.
[x] I have performed a self-review of my own code or work.
[x] My changes generates no new warnings.
[x] I have followed proper naming convention showed in CONTRIBUTING GUIDELINE
[x] I have added screenshot for website preview in assets/images
[x] I have added README.md in my folder
[x ] I have specified the respective issue number for which I have requested the new game.

Add your screenshots(Optional) 📸

Thank you soo much for contributing to our repository 💗

Thank you @tanvigarg264 ,for creating the PR and contributing to our GameZone 💗
Review team will review the PR and will reach out to you soon! 😇
Make sure that you have marked all the tasks that you are done with ✅.
Thank you for your patience! 😀

I have made the changes. plz review

it is still conflicting with main index create PR again by following proper guidelines

Thank you  @tanvigarg264 , for your valuable time and contribution in our GameZone 💗.
It’s our GameZone, so Let’s build this GameZone altogether !!🤝
Hoping to see you soon with another PR again 😇
Wishing you all the best for your journey into Open Source🚀

Hey...should I make changes in index file or the game is added


azcopy vs PowerShell AzStorage executables

What is the difference between and in what situations is it better to use azcopy.exe or the PowerShell Core -AzStorage commands?
Searching the net has not yet resulted in anything clear.
https://docs.microsoft.com/en-us/azure/storage/common/storage-use-azcopy-v10?toc=/azure/storage/blobs/toc.json
https://docs.microsoft.com/en-us/powershell/module/az.storage/start-azstoragefilecopy?view=azps-7.1.0

I'm unaware of the particular downsides of the AzStorage commands compared to AzCopy, but I can at least describe AzCopy. AzCopy is a dedicated tool that can be executed from outside of the PowerShell environment. Additionally, it makes use of service-to-service copy APIs for transfers within Azure Storage services and multi-threading to enable high throughput transfers.

Thank you for your reply, @adreed-msft. There are 142 *-AxStorage* commands and only one (1) azcopy.exe with mand command parameters. Do they both have equivalent functionality?
Are these both based on a .Net API that uses host services?

Hi @Liturgist while we can't speak to the specifics of AzStorage commands, azcopy and AzStorage do not have equivalent functionality. As @adreed-msft mentioned, AzCopy is a dedicated tool for efficient Azure storage transfers. It is written in GoLang, and depends on various azure-sdk-for-go SDKs.

There appears to be no comparison of the capabilities of azcopy and PowerShell Core -AzStorage. Closing as dead end.


Display commit-hash in CLI about

Added commit hash to about section of the CLI when building in the CI
Running quickwit will output
./quickwit
Quickwit 0.1.0
Quickwit, Inc<EMAIL_ADDRESS>Indexing your large dataset on object storage & making it searchable from the command line.
Commit hash: 74aa655ef6ab27851bb002d8fd1e99ca237a080c
Telemetry Enabled
...

Checklist

[x] tested locally
[ ] added unit tests
[ ] included documentation

Awseome!


Dask ohne distributed

siehe https://github.com/neuroevolution-ai/NeuroEvolution-CTRNN_new/issues/22#issuecomment-725477304

closed because inactivity.
feel free to reopen, if we want to address this topic again in the future (even if it's just for discussion)

closed because inactivity.
feel free to reopen, if we want to address this topic again in the future (even if it's just for discussion)


Improve unit tests for ITestOutputHelper

@srogovtsev the file with tests has been properly renamed, and I've added a check that the injected ITestOutputHelper instance is the same both in the complex service and in the test class.


Fix warnings

Minor changes to reduce warnings seen when using gfxreconstruct, no (intentional) functional changes.
Using struct as in the definition for forward declaration of the ReplayDeviceInfo tag (clang -Wmismatched-tags)
Explicitly define assignment operators for clang
-Wdeprecated-copy-with-user-provided-copy warning.
clang output:

definition of implicit copy assignment operator for 'ShaderDescriptorInfo'
is deprecated because it has a user-provided copy constructor

Author dfriederich not on autobuild list. Waiting for curator authorization before starting CI build.

minor change, mostly to avoid getting warnings when using gfxreconstruct in a project as dependency with enabled warnings.

Author dfriederich not on autobuild list. Waiting for curator authorization before starting CI build.

@bradgrantham-lunarg  thanks for the format fix :-)

CI gfxreconstruct build queued with queue ID 255991.

CI gfxreconstruct build # 4827 running.

CI gfxreconstruct build # 4827 passed.


Readability improvements.

Readability improvements for ./doc/
Validation Steps Performed

[x] CLA signed.

Keep in mind, unless it's a spec or Roadmap most of the documentation has been moved to https://github.com/microsoftdocs/terminal

Thanks for the answer, I replaced it with the old one.

No doubt this is subjective, but I personally find the introduction of all those
 breaks less readable. If you're looking at the raw markdown (which is often the case for me), then they're just distracting. And in the rendered view they introduce what I'd consider a lot of unnecessary whitespace.
It seems to me that the amount of space to include before a heading should really be a stylistic choice for the renderer. Anyone that prefers more spacing could always use a different viewer, or adjust the css themselves. So I don't think it's appropriate to force a particular style on everyone by hacking in
 elements everywhere when they aren't strictly necessary.

@mertcandav Thanks for the contribution!
I'm inclined to agree with James on this one -- the spacing around the headings and the sections is something that should be handled on the client side... because there's a great number of different markdown renderers, with different rendering characteristics, I'm averse to introducing hard line breaks to serve just one of them.
I personally find them noisy as well, as I tend to read the plaintext versions of the documents as that's how they pass around in our review cycles and live in our working trees.
We can also ignore the entire user-docs/ folder. It's not long for this world.

Thanks for the comments!


Add disable_validation option to SymbolicPulse

Summary
A disable_validation option was added to the SymbolicPulse class, and to all individual symbolic pulses in the pulse library. This option improves the JAX compatibility of Qiskit Pulse, which is important for Qiskit Dynamics users.
Background
Qiskit Dynamics relies on JAX to massively improve run times. However, this improvement comes at a cost - the code has to be JAX compatible, and mundane operations become problematic. The parameter validation of SymbolicPulse objects falls under this problematic category and limits the ability to use Qiskit Pulse with Qiskit Dynamics. For example, this Qiskit Dynamics tutorial defines from scratch a Gaussian pulse, simply to avoid the parameter validation.
Details and comments
A disable_validation option was added in a similar fashion to the existing limit_amplitude. The option can be triggered either via the object creation, or via a class attribute. For example, the following pulse will trigger the validation automatically and fail it:
import qiskit.pulse as pulse

pulse.Constant(duration=100, amp=1.4)

however, these will not:
import qiskit.pulse as pulse

pulse.Constant(duration=100, amp=1.4, disable_validation=True)

pulse.SymbolicPulse.disable_validation = True
pulse.Constant(duration=100, amp=1.4)

The triggering of the validate_parameters method was moved from individual library pulses, directly into the instantiation of SymbolicPulse object. As all library pulses triggered the validation, this will not have any effect on them. However, user defined SymbolicPulsess which did not trigger the validation, will now trigger it, potentially causing a disruption. It will be an easy fix though.
With the Qiskit Pulse compiler coming along (#10759 ) it was likely that the validation would be delegated to the compiler anyway. In that sense, disabling the validation is not far off from what we are going for anyway. However, the addition of new disable_validation parameters across all pulses and classes, will create a deprecation need when we decide to move to the compiler.
I've taken the liberty to remove some limit_amplitude tests, which were redundant.
Release notes will be added once the design is accepted.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

3 of 3 (100.0%)  changed or added relevant lines in 1 file are covered.
14 unchanged lines in 2 files lost coverage.
Overall coverage decreased (-0.01%) to 87.065%

Files with Coverage Reduction
New Missed Lines
%

crates/qasm2/src/lex.rs
2
91.67%

crates/qasm2/src/parse.rs
12
97.13%

Totals

Change from base Build<PHONE_NUMBER>:
-0.01%

Covered Lines:
74012

Relevant Lines:
85008

💛  - Coveralls


Fixes typed paramaters for Psr\SimpleCache\CacheInterface

Hello!

Type: bug fix
Link to issue: https://github.com/phalcon/proxy-psr16/issues/5

In raising this pull request, I confirm the following:

[x] I have read and understood the Contributing Guidelines
[x] I have checked that another pull request for this purpose does not exist
[ ] I wrote some tests for this PR
[ ] I have updated the relevant CHANGELOG
[ ] I have created a PR for the documentation about this change

Small description of change:
Fixes typing issues around Psr\SimpleCache\CacheInterface
Thanks

Thank you @ALameLlama


[Pipfile] --ignore-installed to the install command

Fixes #725
While I'm confident this is appreciated overall by users of the PipfileBuildPack, as it can make it become usable as compared to unusable, but I'm not confident it comes without drawbacks.
It would be good to have someone evaluate the consequences of adding the --ignore-installed flag to pip that is used by pipenv for the PipfileBuildPackage?

The test are failing due to a Zenodo test using another buildpack, I think the PythonBuildPack with requirements.txt.
ERROR: Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-bqOCde/matplotlib/
Removing intermediate container f69e60ad26b4
The command '/bin/sh -c ${KERNEL_PYTHON_PREFIX}/bin/pip install --no-cache-dir -r "requirements.txt"' returned a non-zero code: 1

We definitely don't want --ignore-installed here. That can create a very big mess by installing duplicate packages, etc.
The issue we should be concerned with is that the conda-forge certifi package doesn't have correct metadata installed, which causes pip to balk.
It's possible that we should change how we are creating the initial environment for Pipenv. I'm not sure.

@minrk I think this issue is detected within the PipfileBuildPack, but is common across at least PythonBuildPack and CondaBuildPack as all of these install packages within the image system environment that has certifi installed. It was detected by the PipfileBuildPack as a Pipfile.lock may very well end up containing certifi of a specific version since it probably is a dependency of something, but a typical requirements.txt or environment.yml won't have it pinned.
I think the only really safe option is to systematically let KERNEL_PYTHON_PREFIX be a fresh environment unrelated to everything else, for all the Python based build packs.


How to modify the default value for autoCloseBrackets?

I'm unable to modify the defaults for the autoCloseBrackets option. In particular, I want to include the parenthesis (()) on the explode list, and include the backticks (` `) in the pairs list.
Here is a JSFiddle showing my failing attempt: https://jsfiddle.net/inad9300/1pn46vnn/

Mode-specific configuration (in this case, the one from the JavaScript mode) override the (assumedly language-neutral) settings given in the option. Attached patch adds support for an override option that you can enable to prevent this.

Mode-specific configuration (in this case, the one from the JavaScript mode) override the (assumedly language-neutral) settings given in the option. Attached patch adds support for an override option that you can enable to prevent this.


Warn users or fix issue regarding hanging process

Related to #3661,
Before finding the related issue above, my binary hung when using the RUN PROCESS. It has consumed a lot of my time investigating whether the robot test has found a bug or not. I found out that it is a limitation or weird behavior on RF based on the related issue. It could really help other developers to note this limitation, to avoid misunderstanding the test, and to avoid unnecessary investigation. Thank you in advance.
RobotFramework Version: 6.1.1
Actions to resolve the issue:

[ ] Add this limitation or behavior in the Process Library Documentation
[ ] Fix the bug ( if possible )

Large outputs hanging processes is already documented. Do you have concrete ideas how to enhance the documentation?
The general problem that output buffers get full cannot be fixed. We are not planning to look at that simply because redirecting outputs to files is such an easy workaround. If you have ideas how to fix the underlying issue and are willing to provide a PR doing that we certainly are interested, though.
The above said, there was an issue related to standard input configuration (#4065) that we have now fixed. We first introduced new stdin argument in RF 4.1.1 (#4102) to control the behavior and RF 7.0 changes the default value of stdin so that it shouldn't anymore cause issues (#4103). If your issue was related to stdin, then it's already fixed (although RF 7.0 is still under development).

Yeah, this could be mentioned in the Run Process and Start Process docs.


Design Check boxes

goal: check boxes style.
Current

Pain points

Disproportional label text in relation with the Check box element size
Low contrast of the label

Tasks

[x] Define use cases
[ ] Visual design
[ ] Document results on wiki
[ ] Update files on Box

Use Cases
The main use case for a Check Box is to mark a setting, option or flag as active.
For instance:

[ ] Send promotional emails to my email address

[ ] Use this object as reference

UX Elements

Element
Description

Input
The element that can reflect the status of the checkbox. Checked or Unchecked

Label
The label describes what will happen when the checkbox is activated

Status

Status
Description

Normal
The Check Box doesn't have the focus.

Mandatory
Setting a Check Box as mandatory means: the user have to activate the option to validate the screen or continue with the flow. The label is displayed in bold and a * at the end, indicating that field is mandatory.  This is the only case where the Check Box can be put to Invalid after performing the validation.

Focused
Like the rest of the Form Inputs the Input Element will highlight indicating to the user that the element have the focus. Only the input element will be highlighted, not the Label

Disabled
The Input element and the label are dimmed/grayed out so signal they cannot be used

Invalid
The Input Element with highlight in a error color and a message will be displayed right bellow the mandatory checkbox

@LeoT7508
These are the uses cases and status for the Check Box.
Please take in count the pain points from https://github.com/SAP/techne/issues/801#issue-242646618

@joseegm

Looking good, thanks @LeoT7508.
Things to improve

The checked check box is blue. When putting the check boxes along with other inputs. That color will draw too much attention to it. see attached screenshot:

The checked check box with the square filled looks very heavy in comparison with the other inputs -see above screenshot. **We can try removing the fill and using only the check icon inside the square.

There is no apparent difference between the focused and the checked.

Let's try this
Put the Normal checked checkbox color to black, leaving only the blue for the focused status. That will solve 1, 2 and 3

@LeoT7508
There is also a unbalance with the error messages. We are going to try and find a better solution that works for the inputs as well as the check boxes. The ticket is opened at #841

@joseegm
These comments are addressed the same way as radio buttons.

@joseegm
Updated Check Boxes

Results documented on:
https://github.com/SAP/techne/wiki/Check-Boxes

@xak
This is ready for implementation.


Threading

Hi,
First, let me thank you for your library. It works really well. I am using it inside celery tasks to send log message in an AMQP queue that gets further process by logstash (so i need thread safety). I was just wondering why does the library spawns a new thread to handle the amqp connection, can't it be done in the main process ? I would greatly appreciate if you explain to me the workflow of the library in a threading point of view, I don't really understand how rabbitmq and threading should be linked..
Cheers,

Hey
The thread is used to handle all incoming data. The reason behind this is primarily to prevent heavy workloads from preventing the server from receiving and responding to mission-critical messages; such as Heartbeats.
e.g.
channel.basic.publish(.....)
time.sleep(180) # The internal thread will keep receiving messages while the main process is sleeping
channel.basic.publish(.....)
As an example the AMQP library, pika that is only running one thread is running into a lot of these issues.
https://github.com/pika/pika/issues/418

When talking about the threading and amqp-storm you need to keep in mind that the Connection and Channels are all under the main process, and the thread will only handle incoming data and assign it to the appropriate Channel. When you send a message it is done under the main process.
If you want to dig into the code it all starts here.
https://github.com/eandersson/amqp-storm/blob/master/amqpstorm/connection.py#L329
Any incoming message is handled by the thread, and then depending on the Channel will pass it on to the appropriate on_frame handler. If it is Channel0 it is an Internal message (e.g. a heartbeat), if it is a Channel higher than 0 it is one of the users channels.
https://github.com/eandersson/amqp-storm/blob/master/amqpstorm/channel0.py = Channel0
https://github.com/eandersson/amqp-storm/blob/master/amqpstorm/channel.py = User Channel
I wanted the library to be as simple as possible (even though it uses multiple threads) so any incoming message frames are simply appended to a list that is then read by the main process.
https://github.com/eandersson/amqp-storm/blob/master/amqpstorm/channel.py#L106
Any message that needs to be sent in order (e.g. Open/Close Channel) is sent using a simple per-channel RPC mechanism using a threading lock.
https://github.com/eandersson/amqp-storm/blob/master/amqpstorm/base.py#L86
https://github.com/eandersson/amqp-storm/blob/master/amqpstorm/channel.py#L193
Honestly the append list part is something I would like to improve, but has worked really well so far. The only danger is that if QoS is not set, memory may sky rocket, as AMQP will push 32k messages to the client. ;)
Let me know if that was unclear, or if you have more questions! :)

Hi,
Thanks for the quick reply, you answered all my interrogations !

No worries! I am glad someone is using it! :smile:


in build tests, sort by extension then path

this fixes the wobbling tests (duckdb, sql, npm) that change when a commit modifies the content hash of inputs.js vs inputs.css

I think the call on L94 is safe because it's only reading the Map of aliases that were computed earlier, not setting any alias.


Paypal review page

Describe the solution you'd like
It would be beneficial to render the shipping methods, allowing users to select or change them during the review step.

Hello @AVATOR,
Thank you for you suggestion.
When this would become technically possible on Adyen side, we will consider this for the future feature enhancements.


Export-CSV should include milliseconds for DateTime data

Summary of the new feature / enhancement
In the current implementation, Export-CSV does not include milliseconds in the output:

It would be desirable that milliseconds be included.
For now, I'm relying on https://gist.github.com/sba923/ddc6e300e2bbe3f01d1ee20276d0cfe4

Proposed technical implementation details (optional)
No response

What's stopping you from using the format you need? For example, I don't need such an output to a file. I believe that users themselves should decide how they want to see the resulting file.
Get-date -f "yyyy-MM-dd HH:mm:ss.fff"

I just went and looked at the code, inside export-csv  it simply calls the .ToString() method of every property, so on my machine set for most-of-the-world's "Least significant first" dates  I get "19/04/2023 13:59:09"  in my CSV.
If I have US-only "Least-significant-in-the-middle"  I would get "04/19/2023 13:59:09"
The work-round for writing a field formatted differently to what .ToString() gives (e.g. Date without time, time with ms; floating point number to specific places of decimals) is to do
$o | select-object Name,Type,@{n="Date";e={$_.date.tostring("yyyy-MM-dd HH:mm:ss.fff")}} | convertTo-Csv

Changing the default so it was not the regional default would create at least many problems as it solves.

As far as I know, there is no such thing as a user-configurable format for date values that includes milliseconds (e.g. in HKEY_CURRENT_USER\Control Panel\International on Windows).
What could be done is to add to the string resulting of applying the user-configured (or culture-inferred) format:

the decimal delimiter (HKEY_CURRENT_USER\Control Panel\International\sDecimal on Windows)
the 3-digit value of the number of milliseconds

If precision and time-zone information are important in addition to culture-invariance, use .ToString('o'), i.e. the round-trip ("O", "o") format specifier, which produces an ISO 8601-compliant date-time string:
PS> (Get-Date).ToString('o')
2023-04-19T12:50:10.9160480-04:00

The resulting string can be cast back to [datetime] without loss of precision, in any culture (PowerShell's casts use the invariant culture):
# -> $true
$d = Get-Date; $d -eq [datetime] $d.ToString('o')

What's stopping you from using the format you need? For example, I don't need such an output to a file. I believe that users themselves should decide how they want to see the resulting file.
Get-date -f "yyyy-MM-dd HH:mm:ss.fff"

Our use case is massive time measurement gathering that later gets processed either machine-wise or via Excel. And we need milliseconds accuracy for all timestamps output by all PowerShell scripts driving those experiments.

If precision and time-zone information are important in addition to culture-invariance, use .ToString('o'), i.e. the round-trip ("O", "o") format specifier, which produces an ISO 8601-compliant date-time string:

Thanks for the hint.
I'll try using that in my Convert-DateTimePropertiesToStringWithMilliseconds.ps1 to avoid a hardcoded format. I just need to check whether Excel can process that format in a CSV.

I'll try using that in my Convert-DateTimePropertiesToStringWithMilliseconds.ps1 to avoid a hardcoded format. I just need to check whether Excel can process that format in a CSV.

Excel's CSV parser does not know how to handle ISO 8601's YYYYMMDDTHHMMSS.FFFFFF+DDDD.
What I probably need to do so that Convert-DateTimePropertiesToStringWithMilliseconds.ps1 is culture/user-settings independent is to create derive a millisecond-aware format from the configured format ((Get-Culture).DateTimeFormat.ShortDatePattern and (Get-Culture).DateTimeFormat.ShortDatePattern), instead of hardcoding yyyy-MM-dd HH:mm:ss.fff.

You can temporary change full datetime format pattern:
[threading.thread]::Currentthread.CurrentCulture.DateTimeFormat.FullDateTimePattern = 'yyyy-MM-dd HH:mm:ss.fff'
(Get-Date).ToString('F')

# or

Get-date -f 'F'

You can temporary change full datetime format pattern:
[threading.thread]::Currentthread.CurrentCulture.DateTimeFormat.FullDateTimePattern = 'yyyy-MM-dd HH:mm:ss.fff'
(Get-Date).ToString('F')

# or

Get-date -f 'F'

No I can't (in PowerShell 7.3.4, at least).

No I can't (in PowerShell 7.3.4, at least).

It did not... I tried on Linux laptop with 7.4.0-preview.2

I'm running 7.3.4 on Windows 11

I'm running 7.3.4 on Windows 11

Yes, I wrote above. I doing the same now. Not working on Windows 11, 7.3.4

Good idea to modify the culture properties, @237dmitry, but that is only worth doing if you modify those patterns that are used by the "G" format specifier, which is what the parameter-less .ToString() method uses,  so that the default stringification then does what you want.
In .NET (Core), [cultureinfo]::CurrentCulture / [threading.thread]::Currentthread.CurrentCulture are read-only, but you can easily create a copy of the current culture, modify it, and assign back to [cultureinfo]::CurrentCulture, for instance, as the following snippet demonstrates:
# Sample date.
$date = Get-Date
$originalDateString = $date.ToString()

# Save the original culture.
$original = [cultureinfo]::CurrentCulture

# Temporarily replace it with a read/write copy.
[cultureinfo]::CurrentCulture = [cultureinfo] $original.Name

# Modify the ShortDatePattern format
# Note: The following are actually placeholders for the culture-specific
#       separators "/" (date), ":" (time components down to minutes).
#       To use these characters literally, "\"-escape them.
#       By contrast, "." is treated literally.
[cultureinfo]::CurrentCulture.DateTimeFormat.ShortDatePattern = 'yyyy-MM-dd'
# Modify the LongTimePattern format to include milliseconds.
[cultureinfo]::CurrentCulture.DateTimeFormat.LongTimePattern = 'HH\:mm\:ss.fff'

# Now .ToString(), which uses ShortDatePattern and LongTimePattern,
# separated by a space, uses the modified formats.
$customizedDateString = $date.ToString()

# Restore
[cultureinfo]::CurrentCulture = $original

# Print results:
[pscustomobject] @{
  Original = $originalDateString
  Customized = $customizedDateString
} | Format-List

Note that the above hard-codes the separators so that the resulting format is the same, irrespective of the current culture.
I don't think you get control over to how to separate the date and the time part - a space seems to be hard-coded.
Output on an en-US machine:
Original   : 2023-04-19 13:51:20
Customized : 2023-04-19 13:51:20.608

In .NET (Core), [cultureinfo]::CurrentCulture / [threading.thread]::Currentthread.CurrentCulture are read-only, but you can easily create a copy of the current culture, modify it, and assign back to [cultureinfo]::CurrentCulture, for instance, as the following snippet demonstrates:

Thanks for the code.
I will think about turning this into an Export-CsvWithMilliseconds.ps1...

In .NET (Core), [cultureinfo]::CurrentCulture / [threading.thread]::Currentthread.CurrentCulture are read-only, but you can easily create a copy of the current culture

On Linux works. I remember you once said that the culture info could be cloned, but I did not find method Clone(). This was then we discussed about decimal delimeters.

Mysterious, @237dmitry, just tried in WSL (Ubuntu 20.4), and - as I would expect - [cultureinfo]::CurrentCulture / [threading.thread]::Currentthread.CurrentCulture are read-only there.
[cultureinfo] implements the IConeable interface, so you can call .Clone() (I've just updated my code snippet to use it.)
In this context, cloning is preferable to my original approach, which constructed a new instance based on the current culture's name ([cultureinfo]::CurrentCulture = [cultureinfo] $original.Name), which wouldn't pick up any potential prior in-session modifications.

[threading.thread]::Currentthread.CurrentCulture are read-only there.

7.4.0-preview.2 Arch Linux:

@237dmitry, the only explanation I can think of is that something in a $PROFILE file of yours assigns a read-write [cultureinfo] instance to [threading.thread]::Currentthread.CurrentCulture on session startup.
(Either that, or there is something specific to Arch Linux, but that seems unlikely).
Start a new session with pwsh -NoProfile and see if you still get the same behavior.
Note that you can query [threading.thread]::Currentthread.CurrentCulture.IsReadOnly / [cultureinfo]::CurrentCulture.IsReadOnly

the only explanation I can think of is that something in a $PROFILE file

Yes. Indeed, cultural info is set in $profile (I did not localize the OS, only pwsh). But why are there no errors when running pwsh? The error occurs with -NoProfile. I didn't even think about $profile, I last edited it a year ago, maybe even earlier.

If you assign a culture info that was obtained via a [cultureinfo] constructor, it will be read-write. If you use [cultureinfo]::GetCultureInfo('...'), it will be read-only. Sounds like your $PROFILE assigns a constructed instance.

Sounds like your $PROFILE assigns a constructed instance.

Where is set [Threading.Thread]::CurrentThread.CurrentCulture

I don't know what's in your $PROFILE; it may be using the culture via [cultureinfo]::CurrentCulture instead, and perhaps there are even other ways.
Please note that we're getting pretty far afield from the original issue here.
I've hidden my previous comments in this side discussion as "off-topic", and I encourage you to do the same.

I'll try using that in my Convert-DateTimePropertiesToStringWithMilliseconds.ps1 to avoid a hardcoded format. I just need to check whether Excel can process that format in a CSV.

Excel's CSV parser does not know how to handle ISO 8601's YYYYMMDDTHHMMSS.FFFFFF+DDDD.

You might want to look at at the .TOOADate()  that converts to the numbers excel uses for date time to what looks like "tick" accuracy.
Then you can apply a format in excel to show those numbers as dates including fractions of a second.

Sorry mis-clicked and closed it Reopened

That's good to know.
Given that DateTime.ToOADate() returns a [double] and that .ToString() stringification is culture-sensitive:
Does Excel expect the decimal mark to use the current culture's?
For instance, on a fr-FR machine, does Excel a value such as "45035,77023724537" (note the ,) in CSV input?
This is what Export-Csv / ConvertTo-Csv would produce.

@mklement0
I only know about dates from ones in invariant culture being misprocessed.  I believe that Excel treats CSV input as if the strings had been typed in the cells, so for french it would want the CSV file to use "," as a decimal separator, but I would need to test it to be sure.
If the destination is Excel, I'd also suggest having a look at the IMPORTExcel module.   Export-Excel will write the date to full accuracy, and with a couple of extra lines  you can set the excel format 'dd/mm/yyyy hh:mm.000'  on a column. Note that the fraction is zeros and not .fff as it is dotnet formatting.

Thanks, @jhoneill.
I looked into this, and you're right:
Excel expects culture-specific field-value representations as well as as the culture-specific list separator (e.g, ; for fr-FR), which can be applied via Export-Csv/ ConvertTo-Csv -UseCulture switch.
Since it wasn't obvious to me, I'll mention it: such values are displayed as numbers by default, but if you format the column as a date/time (possibly programmatically, as you suggest), the values are recognized properly.

Thanks to everyone who contributed to this topic.
This has turned into an interesting and quite useful discussion, I must say, on how I could tackle several aspects of my workflow without the proposed improvement. I do agree that there are many ways to support those scenarios (and some others), especially if one would be willing to invest into additional tasks such as writing and deploying code on the Excel side of the equation, which is fat from trivial (macro security settings and personal macro files just come to mind, for instance). Also, I forgot to mention that those CSVs won't only be consumed by humans with Excel, but also by other scripts (could even be non-PowerShell).
I hate to say I somehow have the feeling we're losing focus on the initial improvement request, which would IMVHO make everyone's life easier "in a turnkey way"...
Excel just happens to accept milliseconds-in-CSV out of the box (true, changing the format to yyyy-mm-dd hh:mm:ss.000 is needed for proper display, but nothing more).
I'll probably think harder about an Export-CsvWithMilliseconds.ps1 😜

Glad to hear it, @sba923.
As for the original proposal:
I think that changing the formatting now would be a backward-compatibility concern.
Generally, CSV is a purely text-based format, so I I think it is reasonable to consistently use .ToString() stringification by default, and leave it to the user to create specialized string representations if needed.
Currently, you can use calculated properties, for instance, as shown by @jhoneill above.
To me, the only conceivable improvement (possibly to be fleshed out in a new issue)  to ConvertTo-Csv / ExportCsv to make this easier - and faster - is to provide something like a -Format parameter that accepts a hashtable whose keys correspond to property names, and whose values contain .NET format strings for formatting those properties, along the lines of:
# Potential enhancement
[pscustomobject] @{
  Name = 'foo'
  Ratio = 1 / 9
  Type = 'known'
  Date = Get-Date
} | ConvertTo-Csv -Format @{ Date = 'yyyy-MM-dd HH:mm:ss.fff'; Ratio = 'F2' }

Output (Ratio value was formatted with [string]::Format('{0:F2}', $_); Date value with [string]::Format('{0:yyyy-MM-dd HH:mm:ss.fff}', $_):
"Name","Ratio","Type","Date"
"foo","0.11","known","2023-04-21 16:38:51.575"

to add some option that would add the milliseconds only when present.

The Millisecond property is part of the datetime object and is inherently always present.
(Get-Date).Millisecond

to add some option that would add the milliseconds only when present.

The Millisecond property is part of the datetime object and is inherently always present.
(Get-Date).Millisecond

Please read "... that would add the milliseconds to Export-Csv's / ConvertTo-Csv's output only when specified."

add the milliseconds to Export-Csv's / ConvertTo-Csv's output only when specified.

These cmdlets output objects to a file or to stdout in CSV format. What you feed them, so they will give. There are a lot of objects with different properties, but there are only two cmdlets, do you understand what an unexpected bad luck?
Build an object that meets your requirements and send it to *-csv. This is the obvious way.
$o = [pscustomobject] @{
    Name = 'foo'
    Type = 'known'
    Date = Get-Date -format 'yyyy-MM-dd HH:mm:ss.fff'
}

$o | Export-CSV file.csv

@237dmitry:
Recent comments have made it clear that we're talking about a potential enhancement to Export-Csv and ConvertTo-Csv, i.e. something that on an opt-in basis that makes something easier and/or faster that could undoubtedly done by hand too.
If you oppose such an enhancement, please argue against it, specifically - no need to restate what has already been stated.

If you oppose such an enhancement, please argue against it,

Ok. I want that *-CSV cmdlets have switch option for any parameter of any input object.  Why are the properties of other objects worse?
But what I want really is Format-Table with switch parameter to output floating as is, without rounding:

[pscustomobject] @{ a = 0.1234567 } | Format-Table -FloatingAsIs

Why was no one asked before secretly "improving" the tabular output? Someone (exms) just wanted...

Although you have said you want "ragged" formatting in tables, many, times, and never miss an opportunity to say so, you've never shown a real world use case.
2..10 | % { [pscustomobject] @{ a = 1/$_ } }

Outputting each row with different numbers of decimals is plainly absurd.
Not printing .000 when there are no milliseconds or only printing .1 or .11 when the last digits are zero seems to match what you want for numbers, but you seem to be arguing against that.
I asked for a format-all-floats-as parameter which could default empty or to local format. There was an issue opened (nothing done "secretly")  and @SteveL-MSFT  came up with a better way. You can take it up with him if you wish.
I'll remind you about the code of conduct again.

Outputting each row with different numbers of decimals is plainly absurd.

Let users round themself. This is not absurd:
2..10 | % { [pscustomobject] @{ a = [math]::Round((1 / $_), 4) }}

Who are you to decide for me what should be the output to the console, how many places am I allowed to leave after the decimal point? Compare this code in Windows Powershell and in PowerShell Core.

@sba923, yes, I think it makes sense to close this issue in favor of a new one that is focused on a concrete feature suggestion.
I suggest making the proposed feature as general as possible, not just focused on [datetime] and milliseconds.
My proposal above isn't good enough for what you're looking for - a data-type-based feature - but it can be generalized by (also) allowing [type]-keyed entries in the -Format hashtable:
# Potential enhancement, based on *types*, same output as above.
[pscustomobject] @{
  Name = 'foo'
  Ratio = 1 / 9
  Type = 'known'
  Date = Get-Date
} | ConvertTo-Csv -Format @{ [datetime] = 'yyyy-MM-dd HH:mm:ss.fff'; [double] = 'F2' }

Look I'm really sorry that you think this is an acceptable format.

Yes, I think so. I am tried to explain my position, I see that you are a kind person and I just feel uncomfortable arguing with you. Therefore, I promise not to return to this topic, let it remain in the past.

@sba923, yes, I think it makes sense to close this issue in favor of a new one that is focused on a concrete feature suggestion.
I suggest making the proposed feature as general as possible, not just focused on [datetime] and milliseconds.

@mklement0 I will think of a meaningful and unambiguous title for the new issue and write as good a description as I can.
But before I do that, I have a somewhat related question: how come doubles are formatted with 2 digits after the decimal points in some circumstances (including Out-GridView)? How does one change that format, say, to include all digits and no rounding to N decimal places?

@sba923  Have a read of the exchange dmitry and I had. In 7.3 format-table was updated to line up decimal points and thousand separators by using the default number formatting for the current culture (usually 2 decimal places, but it does vary).  I originally requested a parameter to specify it, and that would allow it to be over-ridden. If you see a need for that please provide something (in another issue) which can be put in front of the cmdlet working group and we'll see if we get this refined a bit.

Hope https://github.com/PowerShell/PowerShell/issues/19579 will do.

I will think about turning this into an Export-CsvWithMilliseconds.ps1...

@mklement0, what about this?
# proxy for Export-Csv that adds milliseconds to DateTime properties

# original proxy code produced by:
# $MetaData = New-Object System.Management.Automation.CommandMetaData (Get-Command  Export-Csv)
# [System.Management.Automation.ProxyCommand]::Create($MetaData)

[CmdletBinding(DefaultParameterSetName = 'Delimiter', SupportsShouldProcess = $true, ConfirmImpact = 'Medium', HelpUri = 'https://go.microsoft.com/fwlink/?LinkID=2096608')]
param(
    [Parameter(Mandatory = $true, ValueFromPipeline = $true, ValueFromPipelineByPropertyName = $true)]
    [psobject]
    ${InputObject},

    [Parameter(Position = 0)]
    [ValidateNotNullOrEmpty()]
    [string]
    ${Path},

    [Alias('PSPath', 'LP')]
    [ValidateNotNullOrEmpty()]
    [string]
    ${LiteralPath},

    [switch]
    ${Force},

    [Alias('NoOverwrite')]
    [switch]
    ${NoClobber},

    [ValidateNotNullOrEmpty()]
    [System.Text.Encoding]
    ${Encoding},

    [switch]
    ${Append},

    [Parameter(ParameterSetName = 'Delimiter', Position = 1)]
    [ValidateNotNull()]
    [char]
    ${Delimiter},

    [Parameter(ParameterSetName = 'UseCulture')]
    [switch]
    ${UseCulture},

    [Alias('ITI')]
    [switch]
    ${IncludeTypeInformation},

    [Alias('NTI')]
    [switch]
    ${NoTypeInformation},

    [Alias('QF')]
    [string[]]
    ${QuoteFields},

    [Alias('UQ')]
    [Microsoft.PowerShell.Commands.BaseCsvWritingCommand+QuoteKind]
    ${UseQuotes})

begin
{
    # derived from https://github.com/PowerShell/PowerShell/issues/19536#issuecomment-1515140128
    # Save the original culture.
    $original = [cultureinfo]::CurrentCulture

    # Temporarily replace it with a read/write copy.
    [cultureinfo]::CurrentCulture = [cultureinfo] $original.Name

    # Modify the LongTimePattern format to include milliseconds.
    [cultureinfo]::CurrentCulture.DateTimeFormat.LongTimePattern += ([cultureinfo]::CurrentCulture.NumberFormat.CurrencyDecimalSeparator + 'fff')

    # Now .ToString(), which uses ShortDatePattern and LongTimePattern,
    # separated by a space, uses the modified formats.

    try
    {
        $outBuffer = $null
        if ($PSBoundParameters.TryGetValue('OutBuffer', [ref]$outBuffer))
        {
            $PSBoundParameters['OutBuffer'] = 1
        }

        $wrappedCmd = $ExecutionContext.InvokeCommand.GetCommand('Microsoft.PowerShell.Utility\Export-Csv', [System.Management.Automation.CommandTypes]::Cmdlet)
        $scriptCmd = { & $wrappedCmd @PSBoundParameters }

        $steppablePipeline = $scriptCmd.GetSteppablePipeline($myInvocation.CommandOrigin)
        $steppablePipeline.Begin($PSCmdlet)
    }
    catch
    {
        throw
    }
}

process
{
    try
    {
        $steppablePipeline.Process($_)
    }
    catch
    {
        throw
    }
}

end
{
    # Restore
    [cultureinfo]::CurrentCulture = $original
    try
    {
        $steppablePipeline.End()
    }
    catch
    {
        throw
    }
}

clean
{
    if ($null -ne $steppablePipeline)
    {
        $steppablePipeline.Clean()
    }
}

<#

.ForwardHelpTargetName Microsoft.PowerShell.Utility\Export-Csv
.ForwardHelpCategory Cmdlet

#>

@sba923 Have a read of the exchange dmitry and I had. In 7.3 format-table was updated to line up decimal points and thousand separators by using the default number formatting for the current culture (usually 2 decimal places, but it does vary). I originally requested a parameter to specify it, and that would allow it to be over-ridden. If you see a need for that please provide something (in another issue) which can be put in front of the cmdlet working group and we'll see if we get this refined a bit.

Wasn't this ever discussed in another issue?
There are several aspects in that discussion:

rounding
alignment of decimal points
alignment of thousand separators

Depending on the scenario, user will want data preservation and/or pretty (more or less the same as "readable", but that's of course a matter of taste) output. As far as I'm concerned, my preference for the default behavior in Out-Gridview (and probably Format-Table -AutoSize) would be: preserve the data (no rounding) and align decimal points.

Thanks for creating the new issue, @sba923.
It occurred to me that we can make the same proposal for Format-Table.
If implemented, @237dmitry could then get the (unaligned) Windows PowerShell formatting back session-globally as follows:
$PSDefaultParameterValues['Format-Table:Format'] = @{ [single] = 'G', [double] = 'G'; [decimal] = 'G' }

Similarly, the following would make all integral types use thousands separators (helpful in Get-ChildItem output for the Size column, for instance):
$PSDefaultParameterValues['Format-Table:Format'] = @{ [int] = 'N0', [long] = 'N0' <#; ... #>  }

As for the alignment problem / new table-formatting behavior: Your column values include [int] instances (where the division happens to yield a whole number), whereas the new aligned, fixed-decimal-places formatting applies only to non-integral types ([single], [double], and [decimal])
Windows PowerShell didn't even have alignment for instances of non-integral types, so from that perspective the new behavior is an improvement - as is the predictable use of space (try [pscustomobject] @{ num = 1/9 }, [pscustomobject] @{ num = 1/2 } in Windows PowerShell).
The problematic aspect is the unconditional use of decimal places, with trailing 0s, notably also for the "dual-use" [decimal] type, which can also be used for large integers.
A G-based number format would avoid this (as is used with Format-List and direct number output), but .NET's string formatting currently doesn't support creating aligned representations for it, so there are two options:

Make PowerShell implement it, which may have performance implications
Suggest a .NET improvement and wait for its availability.

Specifically, what is needed is a number format for an optional number of fixed decimal places that are right-padded with spaces (instead of 0), including in lieu of the decimal mark so that the following numbers align as follows, for a format with 2 optional decimal places:
25      # integral value, irrespective of specific numeric type
 0.5    # [single], [double], [decimal] obtained with 1 / 2
 1.11   # [single], [double], [decimal] obtained with 1 / 9

Wasn't this ever discussed in another issue?

Oh, yes...

https://github.com/PowerShell/PowerShell/issues/18492
[best discussion] https://github.com/PowerShell/PowerShell/issues/19107
https://github.com/PowerShell/PowerShell/issues/19139
https://github.com/PowerShell/PowerShell/issues/19347

@sba923, as  for the proxy function:
Looks good, except I would move [cultureinfo]::CurrentCulture = $original to the clean block.
Also, based on https://github.com/PowerShell/PowerShell/issues/10863#issuecomment-545533276, I suggest removing the OutBuffer modification and omitting the try / catch handlers (or at least replacing throw with $PSCmdlet.ThrowTerminatingError(), but I any terminating error should be propagated anyway)

Looks good, except I would move [cultureinfo]::CurrentCulture = $original to the clean block.

Good point. I must confess I almost did that 😜. I'm not too familiar with clean blocks.

@sba923, as for the proxy function:
Looks good, except I would move [cultureinfo]::CurrentCulture = $original to the clean block.
Also, based on #10863 (comment), I suggest removing the OutBuffer modification and replacing throw with $PSCmdlet.ThrowTerminatingError().

New code:
# proxy for Export-Csv that adds milliseconds to DateTime properties

# original proxy code produced by:
# $MetaData = New-Object System.Management.Automation.CommandMetaData (Get-Command  Export-Csv)
# [System.Management.Automation.ProxyCommand]::Create($MetaData)

[CmdletBinding(DefaultParameterSetName = 'Delimiter', SupportsShouldProcess = $true, ConfirmImpact = 'Medium', HelpUri = 'https://go.microsoft.com/fwlink/?LinkID=2096608')]
param(
    [Parameter(Mandatory = $true, ValueFromPipeline = $true, ValueFromPipelineByPropertyName = $true)]
    [psobject]
    ${InputObject},

    [Parameter(Position = 0)]
    [ValidateNotNullOrEmpty()]
    [string]
    ${Path},

    [Alias('PSPath', 'LP')]
    [ValidateNotNullOrEmpty()]
    [string]
    ${LiteralPath},

    [switch]
    ${Force},

    [Alias('NoOverwrite')]
    [switch]
    ${NoClobber},

    [ValidateNotNullOrEmpty()]
    [System.Text.Encoding]
    ${Encoding},

    [switch]
    ${Append},

    [Parameter(ParameterSetName = 'Delimiter', Position = 1)]
    [ValidateNotNull()]
    [char]
    ${Delimiter},

    [Parameter(ParameterSetName = 'UseCulture')]
    [switch]
    ${UseCulture},

    [Alias('ITI')]
    [switch]
    ${IncludeTypeInformation},

    [Alias('NTI')]
    [switch]
    ${NoTypeInformation},

    [Alias('QF')]
    [string[]]
    ${QuoteFields},

    [Alias('UQ')]
    [Microsoft.PowerShell.Commands.BaseCsvWritingCommand+QuoteKind]
    ${UseQuotes})

begin
{
    # derived from https://github.com/PowerShell/PowerShell/issues/19536#issuecomment-1515140128
    # Save the original culture.
    $original = [cultureinfo]::CurrentCulture

    # Temporarily replace it with a read/write copy.
    [cultureinfo]::CurrentCulture = [cultureinfo] $original.Name

    # Modify the LongTimePattern format to include milliseconds.
    [cultureinfo]::CurrentCulture.DateTimeFormat.LongTimePattern += ([cultureinfo]::CurrentCulture.NumberFormat.CurrencyDecimalSeparator + 'fff')

    # Now .ToString(), which uses ShortDatePattern and LongTimePattern,
    # separated by a space, uses the modified formats.

    try
    {
        # removed according to https://github.com/PowerShell/PowerShell/issues/10863
        # $outBuffer = $null
        # if ($PSBoundParameters.TryGetValue('OutBuffer', [ref]$outBuffer))
        # {
        #     $PSBoundParameters['OutBuffer'] = 1
        # }

        $wrappedCmd = $ExecutionContext.InvokeCommand.GetCommand('Microsoft.PowerShell.Utility\Export-Csv', [System.Management.Automation.CommandTypes]::Cmdlet)
        $scriptCmd = { & $wrappedCmd @PSBoundParameters }

        $steppablePipeline = $scriptCmd.GetSteppablePipeline($myInvocation.CommandOrigin)
        $steppablePipeline.Begin($PSCmdlet)
    }
    catch
    {
        # changed according to https://github.com/PowerShell/PowerShell/issues/10863#issuecomment-545533276
        # throw
        $PSCmdlet.ThrowTerminatingError($_)
    }
}

process
{
    try
    {
        $steppablePipeline.Process($_)
    }
    catch
    {
        # changed according to https://github.com/PowerShell/PowerShell/issues/10863#issuecomment-545533276
        # throw
        $PSCmdlet.ThrowTerminatingError($_)
    }
}

end
{

    try
    {
        $steppablePipeline.End()
    }
    catch
    {
        # changed according to https://github.com/PowerShell/PowerShell/issues/10863#issuecomment-545533276
        # throw
        $PSCmdlet.ThrowTerminatingError($_)
    }
}

clean
{
    # Restore culture
    [cultureinfo]::CurrentCulture = $original

    if ($null -ne $steppablePipeline)
    {
        $steppablePipeline.Clean()
    }
}

<#

.ForwardHelpTargetName Microsoft.PowerShell.Utility\Export-Csv
.ForwardHelpCategory Cmdlet

#>

Time to close this issue?

I think so, yes.
However, I think more new issues are called for:

To address the behavior of Out-GridView and OutConsoleGridView (the install-on-demand cross-platform implementation, maintained in a different repo), which do not share code with respect to number formatting

Note that I'm not sure if Out-GridView is still actively maintained.

To suggest the -Format enhancement for Format-Table too, and possibly allow a $PSDefaultParameterValues['Format-Table:Format'] entry to customize Out-[Console]GridView formatting too.

Additionally, -AutoSize should arguably result in .ToString() formatting, as it did in Windows PowerShell, but with alignment at the decimal mark.

Re Out-GridView and OutConsoleGridView, the current behavior is:

Out-GridView

Both in PS Core and in WinPS it uses G2 formatting - i.e., 2 fixed decimal places and thousands separators.

Out-ConsoleGridView

Seemingly uses .ToString() stringification, just like Format-Table does in Windows PowerShell -> varying decimal places, non-aligned, no thousands separators.

Given that at least Out-GridView has resizable columns, you could argue that there was never a good reason to limit the display to 2 decimal places.

Aligning at the decimal mark without a fixed number of decimals is problematic,
Excel allows a custom format of a #.???????  (with as many ? characters as you are likely to get)  which inserts a space where a digit isn't needed.  Dotnet doesn't ,  formatting with .0000  pads with zeros, and .##### doesn't put the spaces in.
There is also the question of when the decision is made if something generates a few rows then a few more format-table will decide to output with one column width based on the early rows and later a row wants something wider.
Sticking to valid .net values for .toString()  should cover everything.

While using a fixed number of decimal places by default arguably makes sense (even though it'll make some unhappy), I think the unconditional 0-padding is problematic.

Therefore, as noted before (without referencing Excel), we need something like Excel does with #.??????? - preferably at the .NET level, but in a pinch with a custom implementation.

That said, if there's no .NET solution and we offer a -Format @{ [double] = '...' } customization option, such customizations won't be able to use such formatting. (unless PowerShell somehow makes its implementation public).

Either way, this formatting option should also align situationally intermingled integers, so that [pscustomobject] @{ n = 9 / 9 }, [pscustomobject] @{ n = 1 / 9 } results in aligned display - whether such intermingling is present would by default have to be determined within the 300 msecs. delay during which input is collected for determining suitable column widths.

The column-width issue isn't specific to the issue at hand - this needs to be dealt with either -AutoSize (which invariably collects all output first), or a calculated column that specifies a width explicitly.

Therefore, as noted before (without referencing Excel), we need something like Excel does with #.??????? - preferably at the .NET level, but in a pinch with a custom implementation.

.Net copied almost everything from Excel but omitted this one (and IIRC excel uses _ for a space and I think that is MIA in .NET.)
The ideal would be parity, but it might be a bit of struggle to make that happen.

if there's no .NET solution and we offer a -Format @{ [double] = '...' } customization option, such customizations won't be able to use such formatting. (unless PowerShell somehow makes its implementation public).

I've suggested it could allow a string or a script block.
.ToString("0.00000") -replace "[.0](?=[0\s]*$)" removes surplus zeros
There are a lot of possible pit falls trying to determine a formatting string in the first 300ms worth of data. (e.g. items arrive 500ms apart and the first 2 are integers, and the last ones have many decimal places).

I've suggested it could allow [...] a script block.

That's a good idea, and the regex is handy, but removing trailing 0s isn't enough - replacing them with spaces is necessary:
(0.5).ToString('0.00000') -replace '[.0](?=[0\s]*$)', ' ' |
  % { "[$_]" } # to illustrate the space-padding

For cross-culture use the decimal mark must be localized:
(0.5).ToString('0.00000') -replace "[$([cultureinfo]::CurrentCulture.NumberFormat.NumberDecimalSeparato)0](?=[0\s]*$)", ' ' |
  % { "[$_]" }

This assumes that PowerShell will implicitly also use right-alignment for numeric types that use custom formats, so that numbers align at the decimal mark.

There are a lot of possible pit falls trying to determine a formatting string in the first 300ms worth of data

Of course. The 300ms window is a best effort compromise and can yield flawed results:

Perhaps needless to say, best effort in this case means trying to accommodate mixed integer and non-integer numbers if found within that time window.
Note that even the current behavior can lead to problems, even with numbers that are all of the same type; e.g.:

$a, $b = @(
  [PSCustomObject]@{
    N = 0.01
  },
  [PSCustomObject]@{ # Pick wider values here.
    N = 99999999.0
  }
)

(, $a * 1e5) + $b |
  Format-Table -AutoSize |
  oss | Select-Object -First 3 -Last 3

Output  on Unix - note the truncated value in the last row:
    N
    -
0.010
….000

I somehow didn't copy the , " " with the regex. 🤦♂

Published my latest code at https://gist.github.com/sba923/bc52cf74197ab3cddce825e8b502e28b#file-export-csvwithmilliseconds-ps1, FWIW.
I still think this would be a valuable improvement to the built-in Export-Csv.

We could fold this into #19579.

@mklement0  OK if we let this be auto-closed, or do you think we'd rather resurrect this conversation?

It seems that any activity now automatically reopens an auto-closed issue; however, assuming that you agree that #19579 is the way forward, I suggest closing this issue manually as "not planned".


Remember previous scrolls

Closes #2463

Don't merge until #2486 is merged @displague @na3d

Coverage remained the same at 71.205% when pulling b450bc340ffe7bb2f4ed20ed58f71136ec33af2d on eatonphil:remember-previous-scrolls into 73af5e929a8aaa3739a3f7a05b84249a9befbe12 on linode:master.

Changes Unknown when pulling 4ee86e6f1c95c25349591815d8e4b2f59fcf749d on eatonphil:remember-previous-scrolls into ** on linode:master**.

Coverage remained the same at 71.205% when pulling 918a3bfec098851e7546cee3e7cde3188b3429f7 on eatonphil:remember-previous-scrolls into a9e3f76e97bb44159ddc3799a72f59773042404c on linode:master.

Coverage remained the same at 71.205% when pulling 918a3bfec098851e7546cee3e7cde3188b3429f7 on eatonphil:remember-previous-scrolls into a9e3f76e97bb44159ddc3799a72f59773042404c on linode:master.


CNDB-11137 v2: Allow SortedLocalRanges to be empty

Fixes #11137 Undoes v1 and instead allows SortedLocalRanges to be empty, allowing scrub to work and passing the test that v1 broke: https://jenkins-stargazer.aws.dsinternal.org/job/ds-cassandra-build-fast-ci-5.0/22/#showFailuresLink

Butler shows a regression in SortedLocalRangesTest.testNoRanges that might be related?
http://butler-stargazer.aws.dsinternal.org/#/ci/upstream/compare/ds-cassandra-pr-gate/CNDB-11137v2

Hmm, yep, looks like I need to rework that test now that the ranges can be empty.


support for building on Apple

Minimal changes necessary to support Rust FFI bindings on macOS (see https://github.com/xJonathanLEI/starkware-crypto-rs/issues/1)

I tested this branch on my Linux box, my M1 MacBook, and a GitHub Actions macOS runner. All working great.


Enable jsx with pragma

fixes #33
In a .mjs file (or other ESModules compatible file) if there is a @jsx pragma at the top, enable jsx syntax. This is similar to what happens for flow with @flow
The tests are failing due to the .mjs file extension. Thinking about the best way to solve that, currently I added .mjs as a valid fixture extension but it seems there is some code that's "forcing" it to be read as a script instead of a module for the test purposes

It seems like a .js file with
/** @jsx mdx */
const {mdx} = require('@mdx-js/preact');

module.exports = function MDXContent() {
  return
hi
;
};

Parses JSX syntax just fine, so moving the test file to be .js instead of .mjs wouldn't work to make the test pass, because that wasn't failing before.

Looks like you just need to update the snapshot and this is good to merge! (Just delete output.txt and rerun the tests)

@sebmck Ah! I didn't realize that those were snapshots. I had taken output I got from a successful parse and put that into the output.txt, which is why I was confused.

hmm, tests are passing locally now, but in CI it parses to JSXReferenceIdentifier instead of JSXIdentifier, probably due to https://github.com/facebookexperimental/rome/commit/0ab4f7380372a46158c2d7612190d2828ac91b0a. Rebasing the PR

This is great, thank you so much!

Thanks for you help :)