[concurrency] quick-and-dirty implementation of MainActor in the runtime system

To help unblock work that depends on the @MainActor, here's a hacky implementation of MainActor. This implementation should be replaced once we have better support for custom executors in the runtime system.
TODOs:

[x] Have code marked with @MainActor actually run on the main thread.
[x] Fix deadlock in test/Concurrency/Runtime/mainactor.swift. That test blocks the main thread while waiting on a task that needs the main thread.

Here's a simplified version of mainactor.swift:
@MainActor
func helloMainActor() {}

runAsyncAndBlock {
  let handle = Task.runDetached(operation: { await helloMainActor() })
  do {
    try await handle.get()
  } catch {}
}

A correct implementation of the MainActor attribute would deadlock in that code, and it does currently:
* thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGSTOP
  * frame #0: 0x00007fff204cd8e2 libsystem_kernel.dylib`__psynch_cvwait + 10
    frame #1: 0x00007fff204ffe6f libsystem_pthread.dylib`_pthread_cond_wait + 1254
    frame #2: 0x0000000100141bd9 libswift_Concurrency.dylib`swift::ConditionPlatformHelper::wait(condition=<unavailable>, mutex=<unavailable>) at MutexPThread.cpp:88:3 [opt]
    frame #3: 0x000000010013f6bb libswift_Concurrency.dylib`swift_task_runAndBlockThread [inlined] swift::ConditionVariable::Mutex::wait(this=<unavailable>, condition=<unavailable>) at Mutex.h:198:7 [opt]
    frame #4: 0x000000010013f6a5 libswift_Concurrency.dylib`swift_task_runAndBlockThread [inlined] void swift::ConditionVariable::Mutex::withLockOrWait<(anonymous namespace)::RunAndBlockSemaphore::wait()::'lambda'()>(this=<unavailable>)::RunAndBlockSemaphore::wait()::'lambda'())::'lambda'()::operator()() const at Mutex.h:248 [opt]
    frame #5: 0x000000010013f698 libswift_Concurrency.dylib`swift_task_runAndBlockThread [inlined] decltype(this=<unavailable>, criticalSection=(anonymous class) @ 0x00007fce9e5b1350)) swift::ConditionVariable::Mutex::withLock<void swift::ConditionVariable::Mutex::withLockOrWait<(anonymous namespace)::RunAndBlockSemaphore::wait()::'lambda'()>(swift::ConditionVariable&, (anonymous namespace)::RunAndBlockSemaphore::wait()::'lambda'())::'lambda'()>((anonymous namespace)::RunAndBlockSemaphore::wait()::'lambda'()) at Mutex.h:217 [opt]
    frame #6: 0x000000010013f690 libswift_Concurrency.dylib`swift_task_runAndBlockThread [inlined] void swift::ConditionVariable::Mutex::withLockOrWait<(anonymous namespace)::RunAndBlockSemaphore::wait()::'lambda'()>(this=<unavailable>, condition=<unavailable>, criticalSection=<unavailable>)::RunAndBlockSemaphore::wait()::'lambda'()) at Mutex.h:246 [opt]
    frame #7: 0x000000010013f690 libswift_Concurrency.dylib`swift_task_runAndBlockThread [inlined] (anonymous namespace)::RunAndBlockSemaphore::wait(this=<unavailable>) at Task.cpp:370 [opt]
    frame #8: 0x000000010013f690 libswift_Concurrency.dylib`swift_task_runAndBlockThread(function=<unavailable>, functionContext=0x0000000000000000) at Task.cpp:463 [opt]
    frame #9: 0x0000000100002495 a.out`main + 37
    frame #10: 0x00007fff2051a621 libdyld.dylib`start + 1
(lldb)

So that's good 😆
I need a working yield operation so that runAndBlockThread in that test doesn't prevent the detached task from progressing.

Here's a simplified version of mainactor.swift:
@MainActor
func helloMainActor() {}

runAsyncAndBlock {
  let handle = Task.runDetached(operation: { await helloMainActor() })
  do {
    try await handle.get()
  } catch {}
}

A correct implementation of the MainActor attribute would deadlock in that code, and it does currently:
* thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGSTOP
  * frame #0: 0x00007fff204cd8e2 libsystem_kernel.dylib`__psynch_cvwait + 10
    frame #1: 0x00007fff204ffe6f libsystem_pthread.dylib`_pthread_cond_wait + 1254
    frame #2: 0x0000000100141bd9 libswift_Concurrency.dylib`swift::ConditionPlatformHelper::wait(condition=<unavailable>, mutex=<unavailable>) at MutexPThread.cpp:88:3 [opt]
    frame #3: 0x000000010013f6bb libswift_Concurrency.dylib`swift_task_runAndBlockThread [inlined] swift::ConditionVariable::Mutex::wait(this=<unavailable>, condition=<unavailable>) at Mutex.h:198:7 [opt]
    frame #4: 0x000000010013f6a5 libswift_Concurrency.dylib`swift_task_runAndBlockThread [inlined] void swift::ConditionVariable::Mutex::withLockOrWait<(anonymous namespace)::RunAndBlockSemaphore::wait()::'lambda'()>(this=<unavailable>)::RunAndBlockSemaphore::wait()::'lambda'())::'lambda'()::operator()() const at Mutex.h:248 [opt]
    frame #5: 0x000000010013f698 libswift_Concurrency.dylib`swift_task_runAndBlockThread [inlined] decltype(this=<unavailable>, criticalSection=(anonymous class) @ 0x00007fce9e5b1350)) swift::ConditionVariable::Mutex::withLock<void swift::ConditionVariable::Mutex::withLockOrWait<(anonymous namespace)::RunAndBlockSemaphore::wait()::'lambda'()>(swift::ConditionVariable&, (anonymous namespace)::RunAndBlockSemaphore::wait()::'lambda'())::'lambda'()>((anonymous namespace)::RunAndBlockSemaphore::wait()::'lambda'()) at Mutex.h:217 [opt]
    frame #6: 0x000000010013f690 libswift_Concurrency.dylib`swift_task_runAndBlockThread [inlined] void swift::ConditionVariable::Mutex::withLockOrWait<(anonymous namespace)::RunAndBlockSemaphore::wait()::'lambda'()>(this=<unavailable>, condition=<unavailable>, criticalSection=<unavailable>)::RunAndBlockSemaphore::wait()::'lambda'()) at Mutex.h:246 [opt]
    frame #7: 0x000000010013f690 libswift_Concurrency.dylib`swift_task_runAndBlockThread [inlined] (anonymous namespace)::RunAndBlockSemaphore::wait(this=<unavailable>) at Task.cpp:370 [opt]
    frame #8: 0x000000010013f690 libswift_Concurrency.dylib`swift_task_runAndBlockThread(function=<unavailable>, functionContext=0x0000000000000000) at Task.cpp:463 [opt]
    frame #9: 0x0000000100002495 a.out`main + 37
    frame #10: 0x00007fff2051a621 libdyld.dylib`start + 1
(lldb)

So that's good 😆
I need a working yield operation so that runAndBlockThread in that test doesn't prevent the detached task from progressing.

I think the deadlock can be addressed by using https://developer.apple.com/documentation/dispatch/1452860-dispatchmain instead of runAsyncAndBlock, then exit from the main task.

I think the deadlock can be addressed by using https://developer.apple.com/documentation/dispatch/1452860-dispatchmain instead of runAsyncAndBlock, then exit from the main task.

Ok so I've fixed the deadlock with dispatchMain, but Thread.isMainThread returns false, even though the debugger appears to say it's running on the main queue:
Process 8627 stopped
* thread #2, queue = 'com.apple.main-thread', stop reason = instruction step into
    frame #0: 0x0000000100002b54 a.out`main.helloMainActor() -> Swift.Never + 20
a.out`main.helloMainActor() -> Swift.Never:
    0x100002b4d <+13>: movq   0x5b5c(%rip), %rdi        ; (void *)0x00007fff88476170: NSThread
->  0x100002b54 <+20>: callq  0x100006f54               ; symbol stub for: swift_getInitializedObjCClass
    0x100002b59 <+25>: movq   0x5a38(%rip), %rsi        ; "isMainThread"
    0x100002b60 <+32>: movq   %rax, %rdi
    0x100002b63 <+35>: callq  0x100006ebe               ; symbol stub for: objc_msgSend
Target 0: (a.out) stopped.

Perhaps the test is not correct here.

Ok so I've fixed the deadlock with dispatchMain, but Thread.isMainThread returns false, even though the debugger appears to say it's running on the main queue:
Process 8627 stopped
* thread #2, queue = 'com.apple.main-thread', stop reason = instruction step into
    frame #0: 0x0000000100002b54 a.out`main.helloMainActor() -> Swift.Never + 20
a.out`main.helloMainActor() -> Swift.Never:
    0x100002b4d <+13>: movq   0x5b5c(%rip), %rdi        ; (void *)0x00007fff88476170: NSThread
->  0x100002b54 <+20>: callq  0x100006f54               ; symbol stub for: swift_getInitializedObjCClass
    0x100002b59 <+25>: movq   0x5a38(%rip), %rsi        ; "isMainThread"
    0x100002b60 <+32>: movq   %rax, %rdi
    0x100002b63 <+35>: callq  0x100006ebe               ; symbol stub for: objc_msgSend
Target 0: (a.out) stopped.

Perhaps the test is not correct here.

Turns out the test was not correct, but I've now fixed it.

Turns out the test was not correct, but I've now fixed it.

@swift-ci please test

@swift-ci please test

Build failed
Swift Test OS X Platform
Git Sha - 2c26707d5d12f95345fac27122b44bc179be7bf4

Build failed
Swift Test OS X Platform
Git Sha - 2c26707d5d12f95345fac27122b44bc179be7bf4

@swift-ci please clean test macOS platform

@swift-ci please clean test macOS platform

Build failed
Swift Test Linux Platform
Git Sha - 2c26707d5d12f95345fac27122b44bc179be7bf4

Build failed
Swift Test Linux Platform
Git Sha - 2c26707d5d12f95345fac27122b44bc179be7bf4

Build failed
Swift Test OS X Platform
Git Sha - 2c26707d5d12f95345fac27122b44bc179be7bf4

Build failed
Swift Test OS X Platform
Git Sha - 2c26707d5d12f95345fac27122b44bc179be7bf4

@swift-ci please test macOS platform

@swift-ci please test macOS platform

@swift-ci please smoke test Linux platform

@swift-ci please smoke test Linux platform

@swift-ci please test Windows platform

@swift-ci please test Windows platform

I revised the test a bit so that it does not need an explicit call to CFRunLoopRun, since #35215 will enable async main. So, this PR is currently awaiting on that PR to land.

I revised the test a bit so that it does not need an explicit call to CFRunLoopRun, since #35215 will enable async main. So, this PR is currently awaiting on that PR to land.

@swift-ci please test

@swift-ci please test

@swift-ci please smoke test and merge

@swift-ci please smoke test and merge

@swift-ci please smoke test and merge

@swift-ci please smoke test and merge


[Feature] Option -t to display token information

Description
Add a -t or --token-usage option. When the program is run with this option, display additional information about token usage.
Would love to work on this issue myself. Feel free to assign me.
Feature Checklist

Support -t option
From the API response, read token information under the usage object
Output to stderr
Update help information to include this option
Update README

Sample Output
# Existing Output... #
----------------------------------
Completion Tokens: 17
Prompt Tokens: 57
Total Tokens: 74

Make sure to follow the expected output


About transaction location

When I run the app in swift, all the functions are working. But the location including (address, city, zip code, etc.) show the value is null. I wonder if there is no location data or I did wrong.

Hi, sorry for bothering you again. Last question, could you add logout feature in this project? Because I think people may have different bank account. Therefore, it will be more convenience if add log out feature.

This is more of a side project that's mostly for experience and fun, so it's not meant to be feature-complete (at least right now), which is why it doesn't have a log out button. In my testing, I would just add the appropriate logout function from PlaidManager to the AppDelegate's didFinishLaunching method in order to trigger it. It's not convenient, but it works for testing.
You're free to add one if you think it would be useful and something you're interested in doing!

Thank you so much.

Thank you so much.


react-native start and run-android fail, while yarn start and a succeeds

[v] I have reviewed the documentation
[v] I have searched existing issues
[v] I am using the latest React Native version

Environment

Environment:
OS: Windows 10
Node: 8.11.1
Yarn: 1.6.0
npm: 6.0.0
Watchman: Not Found
Xcode: N/A
Android Studio: Version  <IP_ADDRESS> AI-173.4670197
Packages: (wanted => installed)
react: 16.3.1 => 16.3.1
react-native: 0.55.3 => 0.55.3
(Problem first detected with react 16.3.1 and rn 0.54.0 and yarn 1.5.1)
I'm using expo and the project was created with init (not Create...).
Gradle and java settings:
settings.gradle:
rootProject.name = 'noam'
include ':app'
gradle.properties:  android.useDeprecatedNdk=true
gradle/gradle-wrapper.properties:
distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
zipStoreBase=GRADLE_USER_HOME
zipStorePath=wrapper/dists
distributionUrl=https://services.gradle.org/distributions/gradle-4.3-all.zip
Java_Home:   C:\Program Files\Java\jre1.8.0_121
Steps to Reproduce

Yarn start and then a, runs ok in my Genymotion virtual device. this is expected behavior, but...

react-native start in an admin-cmd window  I get ok: Loading dependency graph, done  this too is expected, although no need for this anymore, the new "run-android" does it automatically...

react-native run-android I get an error. See below...

Expected Behavior

Should run app in my Genymotion android emulated device.
Actual Behavior

error in red:
FAILURE: Build failed with an exception.

* What went wrong:
A problem occurred configuring project ':app'.
> Failed to notify project evaluation listener.
> com.android.build.gradle.tasks.factory.AndroidJavaCompile.setDependencyCacheDir(Ljava/io/File;)V

See #19070

that was a mistake. Fixed the environment info. Please remove "old version" tag

a gradlew clean in the android folder under the project folder, and then npm install in the project folder fixed this. As an extreme measure, preceding the npm install you could delete the node_modules folder under the project folder, and all the needed modules will be installed.


Add Oracle Coherence storage driver

This pull request aims to add Coherence implementation into storage.
Thanks for the review, Please let me know of any issues to resolve.

@tmiddlet2666 Thanks for the PR, lets us know when its ready for review

@tmiddlet2666 Thanks for the PR, lets us know when its ready for review

Thanks, not sure why the test is vailing on downloading our coherence-ce image https://github.com/gofiber/storage/actions/runs/5804095386/job/15734559701?pr=920
Getting Error: Process completed with exit code 56.
The image is around 257MB, so reasonably large. Are there any limits on disk or downloads?

@tmiddlet2666 Thanks for the PR, lets us know when its ready for review

Thanks, not sure why the test is vailing on downloading our coherence-ce image https://github.com/gofiber/storage/actions/runs/5804095386/job/15734559701?pr=920
Getting Error: Process completed with exit code 56.
The image is around 257MB, so reasonably large. Are there any limits on disk or downloads?

250MB is not large by any means. The problem is something on that health/start script. Have you tried running that shell code locally?

Adding TLS issue for refenrece here: https://github.com/oracle/coherence-go-client/issues/47
@tmiddlet2666 Overall it looks good so far, do you want to add go1.21 to the CI now that it was released yesterday?

Thanks @gaby
I have added the Config options and currently they set env variables, but when the above issue in coherence-go-client is fixed and released, i can update this package.
Will add go 1.21.

Thanks @gaby I have added the Config options and currently they set env variables, but when the above issue in coherence-go-client is fixed and released, i can update this package. Will add go 1.21.

For reference for the Redis driver, we allow the user to provide a tls.Config object. That way they can customize it to their needs.
https://github.com/gofiber/storage/blob/main/redis/redis.go#L50C4-L50C4

@gaby - have moved from pending to proper PR

@gaby - have moved from pending to proper PR

Thanks for the contribution, will start reviewing this week. So far it looks good.

@tmiddlet2666 @gaby

@tmiddlet2666 @gaby  what is the best thing to do in this case?

Let me work on adding the *tls.Config support.
I'll need to release a new go client.
will take me a couple of days and i'll ping you.
Thanks

@tmiddlet2666 @gaby  what is the best thing to do in this case?

Let me work on adding the *tls.Config support. I'll need to release a new go client. will take me a couple of days and i'll ping you. Thanks

Sounds like a plan, thank you!


Add new starter kit

I'm maintaining this react-scripts-ts https://github.com/jpavon/react-scripts-ts, is faster than the original fork. Let me know if you want me to do a pull request to add it @sw-yx @Attrash-Islam

oh man this is so tricky! but i like what you're doing and want to support
it. yea i think it would be nice to tell people it exists. PR welcome.
On Wed, Jun 20, 2018 at 6:01 AM Julio Pavón<EMAIL_ADDRESS>wrote:

I'm maintaining this react-scripts-ts
https://github.com/jpavon/react-scripts-ts, is faster than the original
fork. Let me know if you want me to do a pull request to add it @sw-yx
https://github.com/sw-yx @Attrash-Islam
https://github.com/Attrash-Islam
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/sw-yx/react-typescript-cheatsheet/issues/28, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AiT1gtGIzLc-ulsr0VFhS_M1EE05Pslrks5t-h14gaJpZM4Uu-M6
.


Crash

While on iPad simulator I received this crash:

*** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '*** -[__NSOperationInternal _start:]: something is trying to start the receiver simultaneously from more than one thread'
*** First throw call stack:
(
0   CoreFoundation                      0x0000000113438b0b __exceptionPreprocess + 171
1   libobjc.A.dylib                     0x0000000112e9d141 objc_exception_throw + 48
2   CoreFoundation                      0x00000001134a1625 +[NSException raise:format:] + 197
3   Foundation                          0x00000001129a5599 -[_NSOperationInternal start:] + 277
4   Toaster                             0x00000001109aefc4 TFC7Toaster5Toast5startfT_T + 132
5   Toaster                             0x00000001109af2ac TFFC7Toaster5Toast5startFT_T_U_FT_T + 108
6   Toaster                             0x00000001109af2f7 TTRXFo___XFdCb + 39
7   libdispatch.dylib                   0x00000001144574a6 _dispatch_call_block_and_release + 12
8   libdispatch.dylib                   0x000000011448005c _dispatch_client_callout + 8
9   libdispatch.dylib                   0x000000011446140b _dispatch_main_queue_callback_4CF + 411
10  CoreFoundation                      0x00000001133fd909 CFRUNLOOP_IS_SERVICING_THE_MAIN_DISPATCH_QUEUE + 9
11  CoreFoundation                      0x00000001133c3ae4 __CFRunLoopRun + 2164
12  CoreFoundation                      0x00000001133c3016 CFRunLoopRunSpecific + 406
13  GraphicsServices                    0x0000000115dc8a24 GSEventRunModal + 62
14  UIKit                               0x0000000110a0b0d4 UIApplicationMain + 159
15  carpolo                             0x000000010b4485b7 main + 55
16  libdyld.dylib                       0x00000001144cc65d start + 1
17  ???                                 0x0000000000000008 0x0 + 8
)
libc++abi.dylib: terminating with uncaught exception of type NSException

@mn288, can you reproduce the bug?

not really, its completely random, but at least I got this capture.
It has been driving me crazy.
This is what I can help you with:
I have callbacks from the server, I have a tabor controller with 3 views, if I switch between them all, sometimes it crashes and sometimes it doesn't.

@mn288, hmm. Was that on main thread?

the Toast? I don't call it from main thread or anything, I call it from the view after creating an extension calling it like

self.view.displayToast("Some Message")

where displayToast is an extension of UIVIew as follows
extension UIView{

    func displayToast(_ message: String){

        ToastCenter.default.cancelAll()
        ToastView.appearance().cornerRadius = 5
        Toast(text: message).show()
     }
}

@mn288, I think #110 could resolve this issue but I'm not 100% sure because I couldn't reproduce it. 🤔

I just released 2.0.4. Please reopen this issue if this issue still occurs :)

Trying it now. Thanks :) ill reopen this issue if the problem remains


update for consistency across the Azure docs

for task 1207158: Updates for consistency across the Azure docs. List the landing page as the top node of the TOC, and expand the Quickstart node by default. Since node expansion requires YAML, this PR converts TOCs to YAML if they were not already in YAML. More details about the guidance can be found here: https://review.docs.microsoft.com/en-us/help/contribute/contribute-toc-mvc-how-to?branch=master#structure

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs-ref-conceptual/spring-framework/toc.yml
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

#sign-off from apexprod

@rloutlaw and @openpublishbuild we requested #sign-off almost one week ago. @barbkess needs this change to go live ASAP. Please merge this PR at your soonest convenience. Thank you.


Correct Check for Original Owner

#1657
Problem was that I was comparing email with User.username, which
will not be the Google email, if the user first
signed in with GitHub. Now also check profile for an email in there.
Also:

Add the check when adding a permission; originally only checked
when removing a permission. Basically we shouldn't let you muck
with an original owner's permissions at all.

Please make sure that you've checked the following before submitting your pull request. Thanks!

[] Check that you pass the basic style checks and unit tests by running mvn clean install

Coverage increased (+10.4%) to 54.466% when pulling c7b8ba20bb4f4e3fc0448978cb0a548491f2be69 on feature/1657/googleshare into a352298e0aea68e66a980c20393c4298ac463f7f on develop.


Sandbox based on Docker Compose and official nginx, mysql and php

3 preconfigured containers based on official images. Start in 3 steps.
php composer.phar install
docker-compose up
open http://`boot2docker ip`/config.php

If you prefer access to your site via localhost do the following:
boot2docker ssh -vnNTL 8080:localhost:80 -L 8443:localhost:443
open https://localhost:8443/app_dev.php

@alexanderilyin The code looks good but I think this should be placed in a separate repository. You could create a cookbook in the symfony/symfony-docs repo instead.

does this work with boot2docker?

@cordoval I think it’s specifically created to work with boot2docker.

I agree with @hacfi here. The vagrant setup has already been rejected from the symfony-standard repo in the past

@alexanderilyin let this don't discourage you but work on a repo that has the docker files and share it in this thread, would rock to play with it.

@hacfi possible you're right. I should write small article why it's useful with some cool screenshots like this:

@cordoval It was tested only in boot2docker on OS X but should work in any docker environment. Basically only docker-compose.yml should be included in project. But to make it work there are some configurations for nginx and some customization because of problems with boot2docker and permissions in shared folders.
Some details about used versions:
$ docker --version
Docker version 1.6.0, build 4749651

$ docker-compose --version
docker-compose 1.2.0

i wonder if it would be useful at least a link to https://docs.docker.com/compose/ (btw i was using fig for my projects cc @alexkovalevych) would be useful and why it is more sanctioned than fig if it is?

@cordoval answer is on http://www.fig.sh/index.html
Fig has been replaced by Docker Compose, and is now deprecated.
The new documentation is on the Docker website.

P. S.
I'll add details about prerequirements.

oh my bad true, had forgotten indeed, yes that is perfect

@alexanderilyin Thanks for your contribution. Unfortunately, we do not want to go down this road in this repository. The SE is only about the code you need to bootstrap a new Symfony project... and for any kind of Symfony projects. And your configurations makes some assumptions like a MySQL database or the usage of nginx. It would much better to create a standalone repo or contribute a cookbook for the docs. Thanks.

@alexanderilyin I am interested in this kind of thing for the docs however - some unofficial (meaning community) repositories with Vagrant or Docker setups that we can inform people of in the official docs. See symfony/symfony-docs#4676 for an example. I think we're not doing a great service yet since this information is not included, and I'd be eager to discuss fixing that :).
Thanks!

@alexanderilyin is this project death or alive? are you following up?


How safe & trustworthy are hosting sites such as sourceforge, github or bitbucket for closed-source projects?

I am considering using sourceforge, bitbucket or github for managing source control for my business. I have open projects and I participate in open projects such as gcc. But I also have a business where I develop closed-source software for my living.
How trustworthy are sourceforge, github or bitbucket in terms of keeping software secure from prying eyes? How stable is the hosting in terms of data loss prevention? Has anyone out there based their business logic with such an outfit? Has anyone out there surveyed several of the hosting solutions?

Im not seeing any relevance to Fastlane.


Requirement clarification regarding MDE

In the section Requirements for managing tamper protection in Endpoint Manager is states that Your devices must be onboarded to Microsoft Defender for Endpoint.  This implies that MDAV (or maybe even MDB) customers can't use TP over MEM. Is this the case?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: f31e7b39-b7c3-188a-0b2a-02ff10a5be6d
Version Independent ID: 44940a13-0a2d-e281-560b-cb4d8a5d1ccf
Content: Protect security settings with tamper protection
Content Source: microsoft-365/security/defender-endpoint/prevent-changes-to-security-settings-with-tamper-protection.md
Product: m365-security
Technology: mde
GitHub Login: @denisebmsft
Microsoft Alias: deniseb

@rucam Thank you for your feedback. We have made changes to the document via PR#https://github.com/MicrosoftDocs/microsoft-365-docs/pull/9133
Even with Microsoft Endpoint Manager Tamper Protection enabled, your admins need to have the ability to continually review your security posture. Otherwise, you won’t fully benefit if you cannot see the tamper attempts or report them. Since this is not a must to onboard it to MDE, I have updated the document accordingly.
Thanks
Sri

@msbemba Thank you very much for the contribution and updating the documentation with PR. @rucam Hope this update is helpful for you. Thanks for taking out some time to open the issue. Appreciate and encourage you to do the same in future also.


event emitter library

use this https://github.com/component/emitter/blob/master/index.js

the one you used has a non standard api.


Don't display signed query popups unnecessarily

Also use <IP_ADDRESS> for sapphire-localnet (had weird issue with docker)

I've refactored the various providers, there's now 1 provider (not attached to metamask) and one signer (attached to metamask, auto-wrapped with sapphire when on sapphire)
Notable changes:

Handles network switching properly
Create a New Poll only shows if you've attached web3 wallet to dapp
Creating a poll replaces the buttons with a 'Switch chain' when on wrong chain, and shows 'connect to chain' when not connected
Voting on a poll shows 'Connect' if metamask not connected and 'switch' if on wrong chain


DeskBookingLocations and AreaManagement: switch from polling to subscribe/notify

Currently DeskBookingLocations and AreaManagement logic only poll the neighboring modules to get new status updates. The default poll rate is every 60s.
So when a frontend app user creates am adhoc desk booking on the map. They would expect that desk to turn red immediately. But it remains green for up to two minutes (depending on how closely the areamgmt poll follows the deskboookinglocations poll).
Many users even think the app is broken when the desk doesn't go red. So they try to create the booking again.
We should all of these logics to a subscribe/notify strategy instead of polling, to ensure that status is updated immediately after a notification from staffAPI or neighboring module.

Desk bookings listens for booking changed events and then notifies AreaManagement of a pending update
area_manager.update_available(matching_zones)
AreaManagement syncs these out of cycle updates 3 seconds so there is at least some batching going on
Desk bookings sync can also be increased to poll almost as fast as it places a consistent load on staff-api which is not an issue
But if the staff-api redis connection is functioning properly then there should not really be a need to have it polling very quickly - hence the 60 second poll default. although possibly a 15 second poll would be more appropriate


[ruff] implement redundant ternary rule (RUF034)

Summary
Adds a new rule, RUF102, to check for redundant ternary operators. Message shows when both the true and false branches return the same value, as discussed in #12516

Test Plan

cargo test

Hi Lucas,
I am new to Ruff an am just reading PRs to get a better understanding of how everything works. I was wondering, how did you decide to name it RUF102. Is there any logic behind the RUF0XX, RUF1XX sets?

The ecosystem check shows an interesting false positive:
https://github.com/apache/superset/blob/548d543efe81ecd6f0a6657550230b765ab4d955/superset/utils/decorators.py#L158

The ecosystem check shows an interesting false positive:
<EMAIL_ADDRESS>
Is that a false positive? The snippet in question is:
sorted_args = tuple(
    x if hasattr(x, "__repr__") else x for x in [*args, *sorted(kwargs.items())]
)

It seems like each element in the iterable there will be collected whether or not it has a __repr__ method, so I think it can just be simplified to
sorted_args = tuple(x for x in [*args, *sorted(kwargs.items())])

or even just
sorted_args = (*args, *sorted(kwargs.items()))

What they probably wanted to do was
sorted_args = tuple(
    repr(x) if hasattr(x, "__repr__") else x for x in [*args, *sorted(kwargs.items())]
)

But even that doesn't make much sense, because all objects in Python have a __repr__ attribute:
>>> hasattr(object, "__repr__")
True

There's some overlap with SIM108 that simplifies if expressions

I support this being a separate rule to SIM108. SIM108 is quite opinionated IMO: I think it's questionable whether rewriting an if/else block as a ternary expression is always easier to read in Python, or always more idiomatic. This rule, on the other hand, seems like it's pointing out something that you'd almost always want to avoid. A useless ternary expression is almost certainly a straight-up mistake that you didn't mean to write 😄

The scope of the rule is very narrow. Should it also apply to if-statements?

Also applying it to if/else statements sounds reasonable to me.

It would be great if the rule also provided a fix that removes the if expression with the true or false body.

I don't think a fix would fit this rule because users probably end up in this situation when they mistype one of the ternary ends. Instead of just removing the ternary, they probably need to correct one of the sides.

The scope of the rule is very narrow. Should it also apply to if-statements?

I wonder if rule SIM114 address that

SIM114 only seems to flag if-elif but not if-else. https://play.ruff.rs/ae79ec83-f045-4262-bcfb-dc8a08a19460


Redirect buttons to different URLs and pages

Conflict we're dealing with: We have the necessary pages all worked out, but we need to use the react-router-dom library to connect these pages together.
Expectations:

Clicking the logo successfully re-directs you to the home page (canvas)
Clicking the log-in button with the correct credentials directs you to the home page (canvas)
The sidebar should hold links to the canvas page, checklist, "home"

As of right now, we're accessing the pages through URL re-direction


Elyra pipeline describe enhancements

Enhancing "describe" to display Notebook/Scripts and Mount Volumes when applicable
Extended pipeline_app.py to display additional details
Extended pipeline_constants.py to add extra constant to represent notebook extension
Extended test_pipeline_app.py to add additional test functions
Added test pipeline file to surface scenario and data for displaying newer details
Test Details:
make test-server works perfectly well expect ~/kfp/test_bootstrapper.py which is expected in local setup
pytest -k "describe" runs successfully including existing and new test functions
FIXES #2685

I ran in a couple tests and the describe feature appeared to work as expected! Good start given the limited background information the original issue provided! There are a few things though that we should improve to help us get the output of the command to where it needs to be to properly support automation.
Starting with some background information: the describe command was introduced to enable users to identify dependencies that a pipeline might have. There are two output formats that this command supports: human readable (the default) and machine readable (if --json is specified). The machine readable output can be used to automate certain tasks. For example, a user might write a script that extracts the name of all PVCs and runs the appropriate kubectl get pvc .. command to verify that these PVCs are present in the Kubernetes cluster where the pipeline will be executed.
There are a few obstacles that would prevent the user from doing this:

[This behavior was not introduced by this PR.]The machine readable output includes a header, which shouldn't be printed because it renders the JSON output invalid:──────────...
Elyra Pipeline details
───────────...

The header should not be printed if machine-readable output is produced.
The machine-readable output for mounted volumes might look as follows:"mounted_volumes": [
   "/mnt/vol2=my-pvc2"
]

If a user wanted to use this output (see earlier example) to validate the existence of the specified PVC/s (here my-pvc2), it would be necessary to extract the name from /mnt/vol2=my-pvc2. We should avoid the need to do so by only returning the PVC name. @kiersten-stokes we had a similar  discussion in https://github.com/elyra-ai/elyra/pull/2715. Should 2722 piggy back on the changes you are making for that one?

For reference a few examples how a user could extract information from the machine-readable output using jq:
$ elyra-pipeline describe my-pipeline.pipeline | jq .scripts

[
  "/Users/.../workspace/a-script.py",
  "/Users/.../workspace/a-script.r"
]

$ elyra-pipeline describe my-pipeline.pipeline | jq .mounted_volumes[]

"my-pvc2"

@kiersten-stokes we had a similar  discussion in Add support for Kubernetes secrets environment variables #2715. Should 2722 piggy back on the changes you are making for that one?

Good question. I think we're safe to move forward with this PR whenever its ready and then perhaps any required changes as a result of #2715 can be addressed in a follow-up. Especially because 2715 may take a bit longer since I'll be out a few days coming up.

@binayakdutta even though this hasn't been part of the original requirements in https://github.com/elyra-ai/elyra/issues/2685, would you be open to also adding support to include  runtime images for Jupyter notebooks/scripts? Code wise this should be very similar to what you've already added for files (there's a 1-to-1 relationship between generic nodes and runtime images, which are used to execute the notebook/script in).
An example use case scenario that would be enabled by this:

User created several pipelines that utilize a particular container image as runtime, say docker.io/my-org/my-image:v1.0.0.
... time passes ...
User creates a new version of this container image, docker.io/my-org/my-image:v2.0.0, which includes newer versions of the installed packages.
User wants to update the pipelines that use the old image to use the newer version and needs to find out what those pipelines are.
User creates a script that locates pipeline files in the filesystem, runs the elyra-pipeline describe --json ... command and extracts the runtime image information

@binayakdutta even though this hasn't been part of the original requirements in #2685, would you be open to also adding support to include runtime images for Jupyter notebooks/scripts? Code wise this should be very similar to what you've already added for files (there's a 1-to-1 relationship between generic nodes and runtime images, which are used to execute the notebook/script in).
An example use case scenario that would be enabled by this:
* User created several pipelines that utilize a particular container image as runtime, say `docker.io/my-org/my-image:v1.0.0`.

* ... time passes ...

* User creates a new version of this container image, `docker.io/my-org/my-image:v2.0.0`, which includes newer versions of the installed packages.

* User wants to update the pipelines that use the old image to use the newer version and needs to find out what those pipelines are.

* User creates a script that locates pipeline files in the filesystem, runs the `elyra-pipeline describe --json ...` command and extracts the runtime image information

yes definitely. Will be happy to incorporate.

To close the remaining gaps in test coverage, can you please update the tests to

also verify that --json produces the expected results
also use a pipeline that that only includes custom components (or at least contains one custom component) - for these components your describe command enhancements should not report any dependencies. https://github.com/elyra-ai/elyra/blob/master/elyra/tests/pipeline/resources/validation_pipelines/kf_inputpath_parameter.pipeline would be a suitable candidate


CI Bot checks in environment files even if deployment fails

This MAY be an issue, but need to investigate the scope of the implications of this:
The deployment to a dev environment failed and then it goes and checks in the dev environment changes even though it failed:
Task 1695996 | 23:17:49 | Preparing deployment: Preparing deployment (00:00:00)
                     L Error: Instance group 'mysql' does not specify jobs key
Task 1695996 | 23:17:49 | Error: Instance group 'mysql' does not specify jobs key

Task 1695996 Started  Thu Oct 26 23:17:49 UTC 2017
Task 1695996 Finished Thu Oct 26 23:17:49 UTC 2017
Task 1695996 Duration 00:00:00
Task 1695996 error

Updating deployment:
  Expected task '1695996' to succeed but state is 'error'

Exit code 1
HEAD is now at <old commit>
Switched to branch 'master'
Your branch is up-to-date with 'origin/master'.
remote: Counting objects: 21, done.
remote: Compressing objects: 100% (21/21), done.
remote: Total 21 (delta 13), reused 0 (delta 0)
Unpacking objects: 100% (21/21), done.
From github.com/myorg/myrepo
 * branch            master     -> FETCH_HEAD
   3a04757..dbc400e  master     -> origin/master
Updating 3a04757..dbc400e
Fast-forward
 .genesis/cached/alpha/myorg.yml          |    2 +-
 .genesis/kits/cf-0.2.2.tar.gz            |  Bin 0 -> 34621 bytes
 .genesis/manifests/dev.yml               | 2597 ++++++++++++++++++++----------
 myorg.yml                                |    2 +-
 4 files changed, 1716 insertions(+), 885 deletions(-)
 create mode 100644 .genesis/kits/cf-0.2.2.tar.gz

This may be intentional, but if so, it would be good that the commit message indicated that the deployment failed.  Right now, it just states:
CI commit: deployed to dev

I feel like failure to deploy should not commit to that environment, since that makes it impossible to generate out-of-band changes to that environment from that point on (until it gets fixed, of course)

Need to investigate whether this still happens. Dennis is pretty sure it doesn't.


'ciphertext' schema is inconsistent between m.olm and m.megolm encrypted messages

In a m.olm.. encrypted message, ciphertext is a JSON object containing the following:
{
  <sender_key>: {
    "body": <body>,
    "type": <type>
  }
}

In a m.megolm.. encrypted message ciphertext contains a single string:
<body>

It might make sense for both to be specified by a singular format independent of algorithm?

Well:

there is no need for a 'type' field for megolm messages, so that would be redundant
olm messages are intended for a particular device; hence the sender_key. Megolm messages are for all devices, so a sender_key is not appropriate.

I'm not sure what a single format would look like. I am not convinced that it would be any clearer.


Update building-simple-cmdset-with-dialog-api.md

As far as I know the IListViewCommandSetExecuteEventParameters interface doesn't have a commandId property. It should be the itemId property:
https://docs.microsoft.com/en-us/javascript/api/sp-listview-extensibility/ilistviewcommandsetexecuteeventparameters

Q
A

content fix?
yes

New article?
no

Related issues?
fixes

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/spfx/extensions/get-started/building-simple-cmdset-with-dialog-api.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

thx Dennis, that's leftovers from pre-GA status. Good to get fixed.

No problem. I was trying the command set SPFx example on docs.microsoft and spotted this one. The rest of the documentation was pretty clear :)


Merge -lv output into stdout

To fix https://github.com/bpftrace/bpftrace/issues/3379
Checklist

[*] Language changes are updated in man/adoc/bpftrace.adoc
[*] User-visible and non-trivial changes updated in CHANGELOG.md
[*] The new behaviour is covered by tests

also while you're at it, do you mind updating commit message with the justification inline? it makes it easier to git-blame down the line without opening up github

Do you mean that I change the commit message to
Merge `-lv` output into `stdout` to fix issue 3379

@viktormalik @danobi Hi, I have fixed the commit message.


Update to mvn 3.9.6

resolves #922
Changes:

Update mvn from 3.9.5 to 3.9.6
Update resolver from 1.9.16 to 1.9.18
Update surefire from 3.1.2 to 3.2.2

Thanks @nathansit!


Broken link?

Is this link broken? Where should it lead? Trying to get started with Spring...
https://spring.io/guides/gs/spring-boot/

I think it was fixed. Looks different now and the link works for me.


Error when browsing extension node of a package unit

PBPackageUIItem is asked for isAboutPackage: but does not implement it.
It is called from ClyExtensionLastSortedClasses.
There should be the list of classes there but instead we have a set with the PBPackageUIItem node representing the package


ETA on Vulkan support

Hey. Is there an ETA on Vulkan support for unreal engine?

Heya, we don't have an ETA, but we released experimental RHI support in 0.2.2. We also released 0.2.3 yesterday, so I recommend grabbing that version.


Outdated links in the APIM Analytics 2.1.0 Management Console

Description:
There are four outdated links in the APIM Analytics Management Console (https://:9444/carbon) sign in page, as mentioned below.

User Guide - Change this link to https://docs.wso2.com/display/AM210/Analytics
Issue Tracker - Link this to https://github.com/wso2/analytics-apim/issues/
Docs - Change this link to https://docs.wso2.com/display/AM210/Analytics
About - Remove this link. (I already had a discussion with NuwanD in this regard and he confirmed my suggestion).

Suggested Assignees:
Chanaka Jayasena
Affected Product Version:
APIM Analytics 2.1.0
OS, DB, other environment details and versions:
Steps to reproduce:
Start API-M 2.1.0
Access https://:9444/carbon
Related Issues:
N/A

Not relevant for current versions.


drop deprecated result object

What does this PR do?
remove usage of returning ResultObject from the train loop
Needed pre-steps (particular PRs):

[x] drop unused API with results in tests
[ ] drop usage as return output in tests
[x] drop TrainResult
[ ] drop EvalResult

Before submitting

[x] Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
[x] Did you read the contributor guideline, Pull Request section?
[ ] Did you make sure your PR does only one thing, instead of bundling different changes together? Otherwise, we ask you to create a separate PR for every change.
[ ] Did you make sure to update the documentation with your changes?
[ ] Did you write any new necessary tests?
[ ] Did you verify new and existing tests pass locally with your changes?
[ ] If you made a notable change (that affects users), did you update the CHANGELOG?

PR review
Anyone in the community is free to review the PR once the tests have passed.
Before you start reviewing make sure you have read Review guidelines. In short, see the following bullet-list:

[ ] Is this pull request ready for review? (if not, please submit in draft mode)
[ ] Check that all items from Before submitting are resolved
[x] Make sure the title is self-explanatory and the description concisely explains the PR
[x] Add labels and milestones (and optionally projects) to the PR so it can be classified; Bugfixes should be including in bug-fix release milestones (m.f.X) and features should be included in (m.X.b) releases.

Did you have fun?
Make sure you had fun coding 🙃

the point with failing tests is that we use Train/EvalResult in about 60 places in tests...

Hey @Borda, mind resolving the failing tests ?

Hey @Borda, mind resolving the failing tests ?

@carmocca @tchaton mind check why the tests are failing, what else is missing?

@Borda What tests are you referring to in particular?


Support passing a UserTransformer through the processor to the reader

Can't even find the discussion right now, but somebody was missing this :)

Cool, was about to fix that too, we now have a use cases that needs this.
Looks like I also missed to wire up the OrganizationTransformer 🙈

Wait, did I also miss something or not? :)


fix: loosen up issue number regex

This is a follow up for #55.
Asserting the start of the line gets in the way of matching issue ids in
the issue buffer, which uses a border right next to the issue number.
For example, the following comment in GHOpenIssue 55:
│#53

Worth noting that we wouldn't have this problem if we were to add a padding of one blank space to the content of the comments, which could improve readability.

Thanks!


Preallocate a planet object

In Ruby, we introduced a shape transition based on the capacity of the object.  Objects will track their max size on the class, and subsequent allocations will pre-allocate room.  This means that two instances of the same class could be born with different shape ids.
This change ensures that the planet objects in the list all have the same shape ids so that the ivar inline caches can be monomorphic.
More discussion is on this PR: https://github.com/ruby/ruby/pull/6699

As an alternative here, if there's some specific shape behaviour you'd like to make sure is benchmarked over time, you could submit a new benchmark.


Bug: unable to install R package on CentOS 7

Installing from CRAN fails with and without NOT_CRAN=true. One error message below. I'm unable to install on a fresh CentOS 7 installation, so very reproducible. Any way I can fix it?
(I posted this on JIRA 1 week ago, but it's currently filled with spam bots)
installing to /home/tching/R/x86_64-pc-linux-gnu-library/4.1/00LOCK-arrow/00new/arrow/libs
 *
 ** R
 ** inst
 ** byte-compile and prepare package for lazy loading
 ** help
 *** installing help indices
 ** building package indices
 ** installing vignettes
 ** testing if installed package can be loaded from temporary location
Error: package or namespace load failed for ‘arrow’ in dyn.load(file, DLLpat/home86_64-pc-linux-gnu-library/4.1/00LOCK-arrow/00new/arrow/libs/arrow.so':
/home/tching/R/x86_64-pc-linux-gnu-library/4.1/00LOCK-arrow/00new/arrow/libs/arrow.so: undefined symbol: _ZNK5arrow8DataType18ComputeFingerprintB5cxx11Ev
Error: loading failed
Execution halted
ERROR: loading failed
 * removing ‘/home/tching/R/x86_64-pc-linux-gnu-library/4.1/arrow’
 * restoring previous ‘/home/tching/R/x86_64-pc-linux-gnu-library/4.1/arrow’

The downloaded source packages are in
‘/tmp/RtmpeEDbrP/downloaded_packages’
Warning message:
In install.packages("arrow") :
installation of package ‘arrow’ had non-zero exit status

Setting ARROW_R_DEV=TRUE let me figure out what the issue was. One difference from your test setup is I am using gcc 10 and compiling R from source, which is common on centos 7 which uses gcc 4.8.5 by default.
The shared system libraries did not play nicely with the arrow package (lz4 and snappy in particular), so uninstalling them let the package installation go through properly.
A feature request would be to add a configuration parameter to turn off searching for system libraries.
Anyway, feel free to close. Thanks!

@traversc just came across this issue again, due to the switch to C++17 the next release (10.0.0 in ~ 2-3 weeks) we (will) now build the centos 7 binaries with devtoolset 8 (PR here: https://github.com/apache/arrow/pull/14243 input/testing appreciated!)

A feature request would be to add a configuration parameter to turn off searching for system libraries.

If you build from source (via LIBARROW_BINARY=FALSE) you can use EXTRA_CMAKE_FLAGS=-DARROW_DEPENDENCY_SOURCE=BUNDLED to do that.

@assignUser I worry about things like ABI compatibility. From what I understand, there was an ABI break during GCC 5, so a binary compiled with devtoolset 8 would not be by default compatible with Centos 7 GCC 4.8.5 (please correct me if I'm wrong).
One solution could be to build two versions for Centos 7, one for the default GCC 4.8.5 and another for more modern compilers.
There are some GCC compatibility flags that might be useful.

one for the default GCC 4.8.5

This is not possible due to the switch to C++17 which is first supported in gcc >= 5 but with devtoolset the missing libc++ features are statically linked into the binary and will work on machines without dts (:magic_wand: ).
You can test this yourself by downloading the r-pkg__centos7 artifact from this workflow run: https://github.com/ursacomputing/crossbow/actions/runs/3159866027
It contains a binary package compiled in this way, would be great to have some external validation of the process :)

Awesome, sounds like I'm wrong! What is "dts"?
Would be happy to test it out for you, but need more instructions. How do I get the R package to point to the binaries in the zip file?

Ah, sorry thought I had written it out before: dts = devtoolset
You can unpack the zip anywhere you like, that contains a file called arrow_<IP_ADDRESS>20930_R_x86_64-pc-linux-gnu.tar.gz and you can install this with install.packages("/path/to/arrow_<IP_ADDRESS>20930_R_x86_64-pc-linux-gnu.tar.gz"), given that you have installed the needed dependencies (e.g. by having installed arrow before), otherwise you need to install any you are missing manually and try again after.
Let me know if you need any other info :)

Looks like it works on Centos 7 with GCC 10!
There was a warning that the R package was built with R 4.2, but I assume that's a minor thing.


Installation process in poetry

@engeir Could you check whether the installation process works for you too?

Installed as expected for me on the poetry_install branch.


Docs homepage isn't customized to whittle yet

The docs homepage isn't set up yet

[x] replace the content docs/index.md with something relevant to whittle

closed in #121


New tool JSON>Table/CSV/Excel

Pull request type
Please check the type of change your PR introduces:

[ ] Bugfix
[x] New feature or enhancement
[ ] UI change (please include screenshot!)
[ ] Code style update (formatting, renaming)
[ ] Refactoring (no functional changes, no api changes)
[ ] Build related changes
[ ] Documentation content changes
[ ] Internationalization and localization
[ ] Other (please describe):

What is the current behavior?
Requests for JSON<>CSV conversion in
#200
#671
#708
What is the new behavior?

Converts a JSON array of objects to a grid view that can be copied to clipboard as CSV/TSV/FSV.
Flattens nested JSON objects and ignores any nested arrays eg {"a":{"b":1,"c":{"d":2},"e":[3,4]}} becomes {"a_b":1,"a_c_d":2}
Smart detection triggered by any JSON. Should this be smarter and only detect on JSON arrays?
Any conversion errors (eg not valid JSON, not a JSON array) show as an error message in a single-cell datagrid, and cannot copy to clipboard (button is disabled)
Includes GUI and CLI tools.
Would resolve #708, and half of #200

Other information

Possible future ideas:

Convert TSV/CSV to JSON:

Add a 'paste' button to the datagrid to paste TSV/CSV/CSV-French (auto detect or use the setting?) then convert to json
Leave datagrid on right hand side but remove 'input'/'output' labels since the datagrid can be input via the paste mechanism
This removes complexity of error handling with a toggle between grid view and text view of the TSV/CSV.
Would resolve #671 and #200

Respond to compact mode, showing as vertical layout.

Quality check
Before creating this PR:

[x] Did you follow the code style guideline as described in CONTRIBUTING.md
[x] Did you build the app and test your changes?
[x] Did you check for accessibility? On Windows, you can use Accessibility Insights for this.
[ ] Did you verify that the change work in Release build configuration
[x] Did you verify that all unit tests pass
[ ] If necessary and if possible, did you verify your changes on:

[x] Windows
[ ] macOS (DevToys 2.0)
[ ] Linux (DevToys 2.0)

I couldn't build Release, but also couldn't build Release for the base dev/2.0 branch. Error message is:
Could not write to output file 'C:\Programming\DevToys.v2\obj\AnyCPU\DevToys.Business\Release\net7.0\generated\CommunityToolkit.Mvvm.SourceGenerators\CommunityToolkit.Mvvm.SourceGenerators.RelayCommandGenerator\DevToys.Business.ViewModels.ToolPageViewModel.ToggleSelectedMenuItemFavorite.g.cs' -- 'Could not find a part of the path 'C:\Programming\DevToys.v2\obj\AnyCPU\DevToys.Business\Release\net7.0\generated\CommunityToolkit.Mvvm.SourceGenerators\CommunityToolkit.Mvvm.SourceGenerators.RelayCommandGenerator\DevToys.Business.ViewModels.ToolPageViewModel.ToggleSelectedMenuItemFavorite.g.cs'.'	DevToys.Business (net7.0)	C:\Programming\DevToys.v2\src\app\dev\DevToys.Business\CSC

A thought on the UI:
This is currently NOT achievable with the API we have, but how about: instead of having a Setting where we chose between CSV, TSV, FSV, we have a drop down button in the command bar to copy as CSV, TSV, FSV.
Something similar to what there is in the text box when the UI gets too small:

Except that user could choose between CSV, TSV, FSV.
What do you think @sakana280 ?
Let's no block your PR on this idea, it's just something for me to think about for a future PR.

Cheers heaps for the review. I've addressed all the comments (I think?) except with a query about retaining "Excel" in the search terms only.

Hi @sakana280 ,
Thanks for addressing my feedback so quickly. It remains 1 tiny fix to do and we should be all good to go 😍
Cheers !

Many thanks @veler for helping me navigate the codebase and your speedy review of this PR :)


The aim of this PR is to add support to Group Chats Topics that introduced in Bot API 6.3

Bot API 6.3 (released November 5, 2022) added support for Topics in Groups that is not supported currently (please find issue #1).
So, the aim of this PR is to add support for Topics in Group Chats with more than 100 members. It required an update of python-telegram-bot  to version 13.15 where added full support for Bot API 6.3
Changes included in this PR:

python-telegram-bot updated to 13.15;
version of poetry locked on 1.2.2 (the latest version that still uses old lock format, please find changelog to keep maximum compatibility with your main branch) in Dockerfile and GitHub actions Update: locked on 1.1.15;
optional configuration variable THREAD_ID added to set message_thread_id with support of message_thread_id added into the code;
Annotations added in Dockerfile accordingly to OCI Image Format Specification
Updated README.md

Proposed changes that are not done (will add as separate commits when you ok with that):

add annotations org.opencontainers.image.authors="Max<EMAIL_ADDRESS>

Hi @F1ashhimself, I see that test failed here

@yurnov do not worry, I will take a look

As I see it, test become fail starting from the commit 17205414aa326bffdfbb977b429c9f14f0c33897, probably we should revert it

No need, I will fix tests tomorrow morning.


ts-importer steals the VSCode status bar space

Is it possible to add an settings flag to disable the statusbar icon? It's quite useless and just needs a lot of space...

Looks like that should be tsimporter.showNotifications - Show status notifications, default is false but it's not working.

New setting noStatusBar will be added. Notifications are popup Messages.


get latest horizon version

Should keep current on this, also /fixes #754

I think v22 should be late enough to fix #754 but there are some nice fixes in v24

No issues upgrading, going to merge this


Fixed the inability to serve static files greater than 140 KB

Added offset_ to BufferHolder so that retries would remember where they left off,
Fixed range of copying in struct Buffer so that large dynamic responses still work.

This fixes my specific issue mentioned on #35, let me know if there's any thing small that needs to be done before this gets merged in, because this seems like a pretty serious bug.

Hello,
First of all, thanks for taking time to fix this issue. If I remember correctly, someone also already raised the issue and came to the same conclusion (basically, we do not keep the offset of the last byte written in the buffer).
Your fix looks perfectly fine to me. I'll take some additional time to review it properly this week-end, and merge it in.

Thanks! It was a team effort over here, to give proper credit.

Say, just wondering if you had time to merge this pull request, and perhaps some others, into master.

Say, hate to nudge, but it's been a little bit, is this fix going to get merged? It seems like a potentially big issue to me. Or is there something I can do to help?

Hello,
Sorry for that, I did not find time yet, but will try to find some this week-end.

That's alright, thanks! and let me know if you find anything worth improving.

I'm going to have to say good catch :) I was a bit fuzzy on when toWrite had the fd in it, so I had written it to set status to FirstTry on any successful write, so it would always erase, even if not there, and then add the entry with the offset saved to the buffer. I'm now always inserting the entry on a EAGAIN and only erasing it if status == Retry.
I've made a new small commit that has this fix implemented that's ready to review.

Thank you @jeffvandyke, I've merged your changes into my local fork and it is now working like a charm.

Great, thanks @lucas-bremond for the feedback.
I've made some changes recently that are now in conflict with your changes. Do you want to rebase your changes on top of the new master @jeffvandyke ?
Hopefully I'll have some free time this week-end to close this PR. Kudos again

Sure, I'll try to get around to it before this weekend. Thanks!

Alright, sorry it's been a while, but I've finally got my changes rebased. I tested it with a 300 KB file and it still works.

Alright, great job, merging-it in !


full table names for manifest and catalog should be lowercase

Hey there!
I hope it is fine if I open this PR, please feel free to close if you feel like it breaks other functionalities.
While checking out this package, I stumbled on the following error:
Processing model model.dbt_project.model_name
/usr/local/lib/python3.9/site-packages/dbt_column_lineage_extractor/extractor.py:149: UserWarning: Table database.schema.model_name not found in node mapping
  warnings.warn(f"Table {table_name} not found in node mapping")

After some debugging, I realized the issue was in the following function, when trying to match the dict key with the selected model full name.
This happens because the manifest.json, for example, has the database and schema fields in uppercase
"database": "DB_NAME",
"schema": "SCHEMA_NAME"

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Thanks @i-dalbo for the inputs! LGTM!

thanks @wenwu35! I think someone with write access needs to merge in case :)


crates.io ?

Is there any move to put this library on crates.io or would there be any resistance to another project forking and doing this? Of course if this project were on crates.io it would make it much simpler to use and in fact is a requirement I think for any crate on crates.io and the like to be able to use it.
Thanks for your time.

I think libsodium-sys and sodiumoxide should be put on crates.io

I agree. I tried to upload it to crates.io a while ago, but crates.io didn't seem to handle subprojects (libsodium-sys is a subproject of sodiumoxide). If anyone knows how to work the Cargo.toml magic in order to make it work I would be grateful.

Maybe sodiumoxide and libsodium-sys have to be uploaded as independent projects.

It will be much easier if libsodium-sys were a separate crate and then sodiumoxide used it as a crate for sure.

They are separate crates but live in the same repo.

@actimehl I think they might have to be, but I haven't found out how to have Cargo depend on the in-tree libsodium-sys when building locally and depending on a crates.io version when installing with cargo
There is probably a smart way to solve this, but I haven't found it yet.

Look at https://github.com/alexcrichton/git2-rs for example. There are https://crates.io/crates/git2 and https://crates.io/crates/libgit2-sys. I think the *-sys packages has to be uploaded first.

SodiumOxide is now on crates.io

Fantastic, Thx very much


Docs: .NET Plugins + .NET Plugin Event Reference

Renaming the Community Plugins page to Node.js Plugins page.
Adding all the new Node.js plugins into that page.
Creating a new .NET Plugins page.
Creating a new and in-depth .NET Plugin Event Reference page.
Adding references to both event reference pages to the important pages in this plugin section.

@dericksozo - It looks good to me! Thanks!
I left two comments.
I would like also @mulygottlieb  to review and approve before merging.

@dericksozo - A comment from Yuval-
Let's update the .NET events to be like the Node.js events, i.e. a folder with files while each file represents an event- like the Node.js -

Hey @PazYanoverr and @mulygottlieb, I've implemented all your feedback.


Notes on physics engine

Hey @secretrobotron - I was messing around with how to run the physics simulation more often than once per frame, like we currently do. As discussed, if we only fire once per frame, we run a higher chance of things glitching out when they are moving fast. Currently we have...
oneFrame() {
  updatePhysics(durationOfFrame);  // where durationOfFrame is around 16.66 ms
  updatePaddle();
}

If we want to increase the physics 'resolution' we can do this...
oneFrame() {
  updatePhysics(durationOfFrame/2);
  updatePhysics(durationOfFrame/2);
  updatePaddle();
}

Now it runs two finer steps in the physics simulation, but now we run into some problems, like for example...

When we spin a paddle we apply an angular velocity of .2

Suddenly, with the code above, the paddle spins twice as fast. The reason being is that the velocity it has (.2) is now applied over half of the time span, making it way faster. So, we have to adjust the velocity we apply. We would have to apply 1/2 of the velocity instead, or .1 when we spin the paddle.
But wait there is more...
Let's say the paddle is spinning at our new velocity of .1, if we sample it's velocity after every update of the engine we'll see something like...
oneFrame() {
  // Coming into this, paddle speed is .1
  updatePhysics(durationOfFrame/2);
  // After one step, speed is let's say .09 because of air friction
  updatePhysics(durationOfFrame/2);
  // After the second step, speed is let's say .08 because of air friction
  updatePaddle();
  // Now it's set to .1 again...
}

So there's a problem here, even when we want to continuously spin, the extra steps in our simulation are slowing it down. I think this is bad because the paddle isn't behaving in a way that we want.
Another thing is, I wonder if we have to scale our air friction according to the time spent in each 'simulation' step. I'm not sure - I'll test it out.
So, what we probably want is...
oneFrame() {
  updatePhysics(durationOfFrame/2);
  updatePaddle();
  updatePhysics(durationOfFrame/2);
  updatePaddle();
}

For every time we update the physics simulation, we want to make sure we are applying the forces we want to the paddle.
I've been able to get the game to behave closer to how it's supposed to while testing while having more physics steps and these are the kinds of conclusions I'm reaching. We should figure out what to do in order to increase the physics steps to something we can change with a variable without changing the feel and behaviour of the game.
Lastly we should make these things dependent on the time it took to render the frame. We don't want to just say something like: "Fire the physics simulation 3 times per request animation frame" but rather, fire it every 5ms. We'll have to keep track of how long each frame takes and adjust things accordingly. If we have a 20 second frame, we should fire the simulation 4 times.
Anyway, wanted to write this up. Let me know your thoughts & reactions.

So in looking at this a bit more, if we double the physics framerate we need to...

Make the rotational velocity 1/2 of the current value (it should only move half as fast per step)

Makes sense right? It should spin half the previous distance (angle delta) per step

Double the force applied to the paddle for move commands

This part doesn't make sense to me.
I guess since the force is being applied over half of the timespan, it needs to be doubled to have the same effect?

You'd think this would just equal out or something.

Add a new physicsStep (or equivalent) function to user-controlled elements that sample the user inputs every time the physics simulation runs and apply the appropriate rotations / forces every frame.

cc @secretrobotron

So, let's say we decide to run physics steps every 8ms (twice the current rate) - what we need to do is basically keep track of the extra time in each frame.
On average our frames will take 16.666, so that lets us run two physics steps, but we have .666 👿 left over each frame. We need to store that up in an accumulated time bucket, and when it reaches 8ms, we fire another frame. I don't think this is super important, but it's recommended by people and some articles and makes sense to do. The simulation will stay truer to 'real time' this way.

Ok, so, just for some clarity, the loop should be like this (correct where wrong):

Hai!
Check inputs
Make updates to physics objects depends on inputs or other automated processes
Update the physics engine
Update the renderer
Bai!

So, I think your physicsStep idea makes a lot of sense, just to try to isolate it from the rest of what's going on. We could then run it as often as we'd like.
The force stuff about the paddle doesn't make a lot of sense to me yet either. But, I think the problem might be in updatePhysics(durationOfFrame/2); because durationOfFrame/2 seems like it should be more precisely measured if we're doing finer resolution or running the physics resolver more than once per frame. I think that's what you're talking about, but maybe write some more pseudocode here like you did above?
The other thing I'm thinking about is that we could just try to let the game run as fast as possible, and try to nail it at 30fps. If our physics engine and rendered updates happen faster than that, we can just let the CPU rest, or choose a higher framerate. That might be a better/different way to resolve this, because everything would still be "in sync", but just running on a clock we can control.

No, not quite... here's what needs to happen

Hai! Starting requestAnimationFrame cycle
Start a loop, do this however many times you want...

Check inputs
Make updates to physics objects depends on inputs or other automated processes
Update the physics engine

Update the renderer
Bai!

One other way we could explore

Hai! Starting requestAnimationFrame cycle
Check inputs
Make updates to physics objects depends on inputs or other automated processes based on delta for frame
Update the physics engine with same delta
Update the renderer
Sleep for remaining time to level out at 30 or 60 fps
Bai!

@secretrobotron no that won't work ^ because the idea is to sample the physics simulation MORE OFTEN than the framerate we have access to.

Ah yeah, sorry. I wonder then if we could measure how many physics frames we think we could do for each graphics frame. Like, attempt to do 5 physics updates for each rAF, and check to see if we've run out of time after each. It would basically do "as many physics updates as possible" before giving time back to the renderer.

Notes from @modeswitch:

Use Performance.now() to get time instead of Date.now()
Try to lock framerate of physics engine. e.g. if d=10 and fps=2, run physics engine 5 times.

But, it's probably not the case that we need to adjust our force measurements based on delta (weird)

Test on Firefox, because frames seem to be varying a lot

Looks like the FPS is pretty variable, like 29-49 FPS. So, need to tune the physics engine and do some profiling to nail it.

I looked through the matter.js source and it's not sub-sampling your physics simulation automatically. So you should be dividing your dt into the preferred resolution for your physics simulation (maybe 2ms?) and running it enough times to avoid issues with fast-moving objects. Once you figure out your timing, you can also set a maximum velocity so you can be sure objects won't pass through other objects.
Also, there was (is?) a bug here https://github.com/liabru/matter-js/issues/148 that might explain the behavior you were seeing. However, I still recommend locking your physics frame rate to help improve simulation stability.

@modeswitch thanks for digging into it! That's me chiming in at the bottom of the issue you were reading. In another branch I've got the game running at 8ms samples, which removes many of the glitches we were seeing. To do that, we've got to make sure that all of our direct velocity setting in our game know how bit the dt is, which isn't too hard. The annoying thing seems to be also updating the frictionAir properties, which is the part that feels buggy on matter.js's part.

To do that, we've got to make sure that all of our direct velocity setting in our game know how bit the dt is

Why is that the case? If you set the velocity to 10 m/s, the simulation should handle that in the update code. It has access to dt when you pass it in to the update handler. No?

I think because right now when we set the velocity, we don't provide the time unit. We just say 5 and I think the engine just fills in the rest, so whatever time delta you are using it fills that in. When running the simulation at 60FPS, the delta is roughly 16ms, so our velocity becomes 5 pixels / 16 ms, however when we increase the sampling time to 120 FPS, then the velocity becomes 5 pixel  / 8 ms and things go twice as fast.
I think it's because the engine was designed to just run at 60 FPS. Anyway, that's what I've come to believe with my experiences with it so far. It seems to handle applications of force better, we can keep the same values without changing anything, which is nice.

OK, that's pretty weird. You could resort to applying forces everywhere (Body.force and Body.torque) I suppose?

Yeah, we apply forces for the paddle movement, and I guess we could do the same with the paddle rotation except then we'd have to put in a limiter for 'max rotation velocity' so that the paddle can't spin like crazy so I guess we'd run into the same issue.

OK, what we have now seems to work pretty well. We are able to set the number of physics steps we want to simulate per frame and things seems to be working well when we change that number.


Investigate merging integration-tools

We can probably merge the integration-tools codebase into this repo and archive the tools altogether. This will allow for an easier platform development experience.
Note: We might have to update all docs in content repos to match the change.

We'll take a first step by mostly duplicating the code of the integration tools as part of our utilities suite. At a later time, we'll update docs and archive the current tooling suite.

This will be resolved in #326 with the introduction of the createSnippet method into the Content utility. As the rest of the tooling (i.e. repo creation) is not part of day-to-day operations for most contributors, we'll skip it for now and come back to it later.
As far as documentation is concerned, we'll introduce changes to all repos reflecting the gradual deprecation of the integration-tools, namely how contributors should duplicate the template to get started. The tooling repository will be archived and made private shortly after.


We just had this in #4221 and it looked like it's fixed in master (@Kriechi). But I can confirm that HEAD is broken. Interestingly it does work for the URL in #4221.
So it looks like some of the HTTP/2 code changed and made it compatible with one server (https://g.alicdn.com/kg/kmd-adapter/0.0.6/index.js) while breaking compatibility with the other (https://ww1.sinaimg.cn/mw600/5f789f8fgy1gjnnwi718sj20s50funk3.jpg).
I can confirm that 828ba0c works for both URLs.
Then c0f62cc broke both.
And then HEAD (currently 0f96324) only works for one of them.

I have tested this many times and found that the 413 error does not occur 100% of the time.
It's possible that it will be normal after a while, but these cases have one thing in common, all the servers that return exceptions are using Tengine (https://tengine.taobao.org/), which is a Nginx-based HTTP server.
www.taobao.com,www.bilibili.com uses Tengine, and you can see a lot of 413 code errors when you visit both sites.
Hopefully, this finding will help.
(I've also found a lot of 501 code errors that appear and disappear for no apparent reason, not sure if it's due to a change in the h2 module version.)

@Prinzhorn thanks for bisecting! 🎉
I can't spot an obvious bug - but based on past experience I'm inclined to blame a faulty or non-RFC-compliant implementation in tengine or an outdated library on the server... We've seen lots of incompatibilities in the early days of HTTP/2. Some vendors even claimed "due to performance benefit we chose to break RFC compliance"...
I'll try to find some time and do another deep-dive, but unless anyone reports a similar problem with other webserver (most importantly the "real" nginx), I think we need to find a workaround for tengine specifically...

@Prinzhorn thanks for bisecting! tada

@Kriechi that was mostly @bigfoxtail's work

I can't spot an obvious bug - but based on past experience I'm inclined to blame a faulty or non-RFC-compliant implementation in tengine or an outdated library on the server... We've seen lots of incompatibilities in the early days of HTTP/2. Some vendors even claimed "due to performance benefit we chose to break RFC compliance"...
I'll try to find some time and do another deep-dive, but unless anyone reports a similar problem with other webserver (most importantly the "real" nginx), I think we need to find a workaround for tengine specifically...

Even if the server is not RFC compliant, the URLs work in Firefox, Chromium and Burp Suite. Obviously they have a lot more resources to make their stack compatible to almost anything. I just wanted to point that out. But like you said, if this doesn't become a more widespread issue we might just wait and see what happens.
On the other hand it did work at some point.

I found another module httpx that uses http2 that works fine.
In the httpcore module used by httpx I found this part of the code.
https://github.com/encode/httpcore/blob/master/httpcore/_sync/http2.py
Line 286:
        # Send the request.
        seen_headers = set(key for key, value in headers)
        has_body = (
            b"content-length" in seen_headers or b"transfer-encoding" in seen_headers
        )

        self.send_headers(method, url, headers, has_body, timeout)
        if has_body:
            self.send_body(stream, timeout)

In c0f62cc559ae4f0778d6079a2b31d2ce0309dda6 you have modified some codes
-                end_stream=self.no_request_body,

and
-        if not self.no_request_body:
-            self.connections[self.server_conn].safe_send_body(
-                self.raise_zombie,
-                self.server_stream_id,
-                chunks
-            )

Data was sent without determining the existence of the request body.
That seems to be the problem.

I've pushed a PR with a potential fix: #4239
Please let me know if this looks good and works for you. I also included a new test to better handle this case.

Can confirm that both URLs now work on master


.osx: Show the /Volumes folder

e.g. to add it to the Sidebar

Thanks for pointing it out. I wrongly assumed that asking for the administrator password in the very beginning is enough

:+1:


Forwarding environment variables

The kernel unsets some environment variables, e.g. LD_LIBRARY_PATH and LD_PRELOAD, in setuid binaries for obvious security reasons.
This PR implements a feature that looks for environment variables that have the SQFSMNT_FWD_ prefix, and strip the prefix from the variable for the executed binary.
This can be used to forward variables like LD_LIBRARY_PATH safely. For example:
SQFSMNT_FWD_LD_LIBRARY_PATH=$LD_LIBRARY_PATH squashfs-mount $image:$mount -- bash

As far as I can see this isn't working unfortunately:
https://github.com/eth-cscs/squashfs-mount/tree/feature/forward-env-add-bats-test
(same result on my laptop)
When the setuid bit is set, execvpe isn't setting LD_LIBRARY_PATH from __envp anymore.


Replace *Id by a single data-type Id

Currently we have a lot of *Id modules in defs, like VarId, ChanId, etc, all of which are isomorphic and lead to a lot of code duplication and boilerplate. See for instance TxsShow.hs. We need to analyze the possibility of replacing this by a data type:
data Id v = Id { name :: Name, id :: Int }

and defining then:
data ChanIdT
type ChanId = Id ChanIdT

@pjljvandelaar any thoughts?

It is a small step in the right direction: I would like to have Identifiers (or maybe call them reference to make the distinction clearer) with only a single integer.
When comparing Identifiers/references, only one field needs to be compared!
(instead of currently at least two!)

And maybe it should be an Int to be really faster!

Could you also update the issue to include some examples  on the code? I think we should start doing this so that it becomes easier for anybody to pick an issue. What do you think?

@pjljvandelaar I'm raising the priority of this issue. I'm doing a small change that requires me to change all the *Id.hs files. This madness has to stop :P


Added .editorconfig

Added .editorconfig from STRV frontend docs.
Closes #2

@jkbktl please also add it to lib/dotfiles/.editorconfig and call copyDotfile in lib/configs/default.js

Although I am still kinda skeptical about this and personally won't ever use it, I understand it could be useful on projects where potentially many people collaborate.


[QUERY] When will bulk executor library be available for  SDKv4

Query/Question
I am upgrading my application from documentDb to cosmosDBV4.
I see bulk executor is the recommended solution for Java bulk support.
However, it is not currently supported for  SDK 4.x.
Is there any plan for bulk executor support for SDK 4.x. in near  future ?
This would help us plan accordingly.
Reference used: https://docs.microsoft.com/en-us/azure/cosmos-db/bulk-executor-java

Thanks for the question @zahiritpro, experts from the Cosmos team will follow up with you shortly.

TransactionBatch and Bulk API's will be native part of V4 Java SDK.
Tentative timeline is end-of-year.


update QOTD DB

add more questions carry over important repeats (functionality)

also, re-work the add-QOTD command so that they are put into a separate table as a queue instead of dumped directly into the unused questions bank

to do this, mod perms can be removed for the ADD qotd command. additionally, commands should be obscured and removed if they aren't useful, such as redoQOTD

comment #2 was implemented in ea213ea5c2bf3621111b6f09c123a8f4262009e9, awaiting test and merge


Datepicker doesn't work in chrome.

Hi. I am creating a form that has a datepicker, I do but doesn't work in chrome, I just try firefox and here if it works, when I select the field in chrome does not emerge any datepicker.

What happens if you go to the docs for Datepicker in Chrome?  It works fine for me, both there and in my apps.

I haven't yet seen an issue of datepicker not working in chrome.


Sharepoint Entry Strings Issues

Describe the bug
I am trying to specify an entrypoint into Sharepoint but I am unable to figure out how to format the strings. Let's say I have this path in Sharepoint (excuse my highlighting and anonymization skills)
For this directory:

How should my Sharepoint Entry look in the options? I've tried a few things including some of what I used below (Note, this is my options but I am programmatically setting it)

This is what I am seeing on my end and it works inconsistently with other paths:

For my application's user experience, the idea is for someone to set this path in the UI and I add it to the options for the initial entry point.
Thank you in advance!

I have added a sample for you showing how to address a folder path. Note you might need to review the permissions the app is consented to include additional scopes depending on the site(s) you are targeting. Delegated Sites.Read.All or Files.Read.All may be required.

Thank you @patrick-rodgers !!

Closing this issue as answered. If you have additional questions or we did not answer your question, please open a new issue, ref this issue, and provide any additional details available. Thank you!


⚠ Digital Ocean - https://status.digitalocean.com/ has degraded performance

In 1e61f4b, Digital Ocean - https://status.digitalocean.com/ (https://status.digitalocean.com/api/v2/status.json) experienced degraded performance:

HTTP code: 200
Response time: 176 ms

Resolved: Digital Ocean - https://status.digitalocean.com/ performance has improved in 857d44b.


AttributeError: module 'pyarrow' has no attribute 'compat' upong importing pyarrow

Hi folks,
I just installed pyarrow with pip. Here is the output:
Collecting pyarrow
  Using cached https://files.pythonhosted.org/packages/c9/b8/076ebd968f3c6d4e9b6e5de696f0b19964a3d4b7dc020fd32db4e868ad74/pyarrow-0.10.0-cp36-cp36m-manylinux1_x86_64.whl
Requirement already satisfied: six>=1.0.0 in /opt/conda/lib/python3.6/site-packages (from pyarrow) (1.11.0)
Requirement already satisfied: numpy>=1.10 in /opt/conda/lib/python3.6/site-packages (from pyarrow) (1.13.3)
Installing collected packages: pyarrow
Successfully installed pyarrow-0.10.0

However when I try to import the package I get an errror:
AttributeError                            Traceback (most recent call last)
<ipython-input-12-f1048abcb32d> in <module>()
----> 1 import pyarrow

/opt/conda/lib/python3.6/site-packages/pyarrow/__init__.py in <module>()
     48
     49
---> 50 import pyarrow.compat as compat
     51
     52

AttributeError: module 'pyarrow' has no attribute 'compat'

I tried uninstalling and reinstalling but it did not help. Any ideas?

Hmm. Installing with conda install -c conda-forge pyarrow fixes this issue. Let me know if you folks need any diagnostics to help debug.

With numpy=1.13.3:

You might have had two different versions of pyarrow installed (one with pip, another older one with conda)

Thanks @kszucs . I was able to replicate the issue.
If I install pyarrow with numpy version 1.13.1, I get the error.
If I uninstall numpy 1.13.1, install 1.15.1, and then install pyarrow (with pip) I can import pyarrow with error.
FYI @wesm ...

Is there an ABI conflict? AFAIK we are building wheels against NumPy 1.14

We're actually building against 1.10 to support old NumPy version. I'll have a look at the docker output to see what's going wrong there.

This is a problem introduced by installing tensorflow: https://travis-ci.org/apache/arrow/jobs/421710986#L9970

Actual cause and JIRA: https://issues.apache.org/jira/browse/ARROW-3141

I am getting this same error with numpy 1.16.4 and pyarrow 0.11.0

pyarrow 0.11.0 is not supported, recommend that you upgrade

for future readers of this thread: the issue can also be caused by pytorch, in addition to tensorflow; presumably other DL libraries may also trigger it.

I've been working on this problem for a week now and am still getting this error. Tried rearranging the import order, as well as uninstalling numpy and pyarrow.
It worked on a previous machine and now fails on a new ec2 instance.
numpy 1.21.6
pyarrow 9.0.0

myPacker_df = dd.read_parquet('s3://XXX/YYY/summary_i/', engine = "pyarrow")
---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-2-9934e78b02e8> in <module>
----> 1 myPacker_df = dd.read_parquet('s3://yse-bioecon/meat_packers/summary_i/', engine = "pyarrow")

~/anaconda3/lib/python3.7/site-packages/dask/dataframe/io/parquet/core.py in read_parquet(path, columns, filters, categories, index, storage_options, engine, gather_statistics, split_row_groups, chunksize, **kwargs)
    208
    209     if isinstance(engine, str):
--> 210         engine = get_engine(engine)
    211
    212     if hasattr(path, "name"):

~/anaconda3/lib/python3.7/site-packages/dask/dataframe/io/parquet/core.py in get_engine(engine)
    495     elif engine == "pyarrow" or engine == "arrow":
    496         pa = import_required("pyarrow", "`pyarrow` not installed")
--> 497         from .arrow import ArrowEngine
    498
    499         if LooseVersion(pa.__version__) < "0.13.1":

~/anaconda3/lib/python3.7/site-packages/dask/dataframe/io/parquet/arrow.py in <module>
      7 import pyarrow as pa
      8 import pyarrow.parquet as pq
----> 9 from pyarrow.compat import guid
     10 from ....utils import natural_sort_key, getargspec
     11 from ..utils import _get_pyarrow_dtypes, _meta_from_dtypes

ModuleNotFoundError: No module named 'pyarrow.compat'

For future readers: pyarrow compat was depreciated at some point, uninstalled pyarrow 9.0.0 and installed pyarrow 4.0.0 which fixed the problem.

@a5creel A better fix is probably to upgrade your dask version to be compatible with newer pyarrow, not downgrading pyarrow.


Usage without Trainer

For now there's only one example script, https://github.com/chainer/chainerui/blob/master/examples/train_mnist.py, which depends on Trainer.
Is it possible to use ChainerUI without Trainer? If no, is there any plan to support it?

ChainerUI depends on the output of chainer.training.extensions.LogReport.
If you make the log in the same format as LogReport, you can use ChainerUI without Trainer.
Currently, there is no plans to provide LogReport without Trainer by ChainerUI. I expect to provide it by Chainer.

I wrote train_mnist_without_trainer.py as an example.
This code has a log-output like LogReport and outputs the following log file.
[
     {
         "epoch": 1,
         "train_loss": 0.8433072566986084,
         "val_accuracy": 0.8198180198669434,
         "val_loss": 0.8316373229026794
     },
          :
          :
]

ChainerUI can visualize this log.

Restriction
Currently X-axis is hard-coded, so the key of the log must be named elapsed_time, epoch or iteration.

Thank you! I think it should be added as another example.

I wrote a util class to output log file, for without Trainer https://github.com/disktnk/chainerui/blob/80b8beaed90a41fb0be4c364a531550091c0989b/chainerui/utils/log_report.py
Usage
from chainerui.utils import LogReport
ui_report = LogReport(out_path, args)
# setup train_iter
while train_iter.epoch < max_epoch:  # training loop
    # train & validate
    stats = {'epoch': train_iter.epoch, 'loss': loss}  # set log values
    ui_report(stats)

I found out some issues in using util function.

Currently, ChainerUI requires X-axis values, "epoch", "iteration" and "elapsed_time", which are set by Chainer's LogReport  automatically. The above ui_report does not have the constraint and it is possible to cause an error.
The util class should have value type check, not allow to set like stats = {'epoch': -1, 'foo': 'bar'}. It may cause performance problem, however (it may be tiny?).
"elapsed_time" can be set automatically in the util class, or not.


Pass parameter to Bootgrid dynamically

As title mentioned that how can we pass parameter to bootgrid dynamically.
Here is the detail of my question on stackoverflow.
http://stackoverflow.com/questions/30438713/pass-parameter-to-bootgrid-dynamically

When you want pass parameters dynamically you can use this way
   $("[data-toggle=\"bootgrid\"]").bootgrid({
        rowCount : [10,25,50,-1],
        post: function() {
            return {
                dataOne: 'This is One',
                dataTwo: $('.fieldTwo').val()
            } ;
        }
    });


Final Paper

Good morning @AaronGullickson,
I am set to turn in my final paper. When I rendered the document I noticed two things. First, it didn't recognize my separate paragraphs in the methods section. So, it's one long paragraph for some reason. This doesn't happen in other sections. Two, when I refer to my graphs with the @-fig command it doesn't seem to work like I thought it would. It doesn't say figure 1 or 2, it prints the command. I also have some funny looks to the boarders of my table of models. Do these issues matter overall for this assignment, should I just turn it in as is? I committed and pushed my final work to git.
Thank you,
Kevin Gable

Regarding the massive paragraphs - you must leave an empty line between paragraphs for markdown to recognize paragraph breaks.
Regarding the @-fig command - I think most of them work, but the first one on line 57 you have Label rather than label. It works when I fix that.
On the table of models, remember to put #| results: asis in the code chunk options at the top.

Thank you @AaronGullickson
I think it is ready. When I tried to reference the table of models in the text (@fig-Rendering_Models) it doesn't work with that command. All the graphs are working great. Is this because it is a table and not a graph? I am also sure my model explanations need work, but that seems to be my struggle on each assignment. I would be thrilled fixing the table rendering reference. Either way, I will have it in by 5.
Thank you,
Kevin Gable

texreg is not fully quarto aware yet. Should be soon! Just refer to the table the old fashioned way in the text.


BL-796 feature/ Landing Page

Description
This ticket was to create the Asylum Office Grant Rate Tracker landing page.
With all of the visualizations planned in this epic, we needed a singular landing page for users to get started.
For this ticket we:

Reviewed different types of users, use cases, and visualizations.
Established information architecture and navigation.
Created a desktop and mobile mockup using Figma to present to the stakeholders what will be shown to users when they visit the web app.
Implemented the page as presented in the mockup.
Per our product and stakeholder meetings discussions, we removed previous sidebar navigation components.

There are 2 working routes that link the user to external sites, however, the routes to the data page are waiting for a PR to be merged.
Video Link
https://www.loom.com/share/2acc0648eb18492284bffc2942f0a28d
Jira Link
https://bloomtechlabs.atlassian.net/browse/BL-796
Type of change

[X] New feature (non-breaking change which adds functionality)

Checklist:

[X] My code follows the style guidelines of this project
[X] I have performed a self-review of my own code
[X] I have removed unnecessary comments/console logs from my code
[X] My changes generate no new warnings
[X] I have checked my code and corrected any misspellings
[X] No duplicate code left within changed files
[X] Size of pull request kept to a minimum
[X] Pull request description clearly describes changes made & motivations for said changes

Awesome job Austin and Simone!
The landing page looks clean and is easy to navigate. Nice visuals for the future data pages.
Great job implementing breakpoints using min and max in your media queries.


Add a "Delete cell" option in the notebook cells' right-button menu

There is no "Delete cell" option in either the code's or markdown's right-button menu's.

Thank you for filing this issue, given the fact that we haven't had sufficient upvotes on this issue I'm closing this as not planned.


395 end2end mini cluster example

What is is PR for?

[x] Code update
[ ] Documentation update
[ ] Infrastructure update

What does this PR do?
It provides a bundle example for creating a mini-cluster connected to the Mosquitto public test broker.
It also fixes a WiFi connection issue in CDP.
Is this related to an open issue?
#397 #395
Testing methodology
Use the 2.Mini-Cluster folder and build each duck.
The Ducks will send a counter message to the Papa Duck and the message is relayed to the MQTT broker.
Additional context
Add any other technical detail or considerations here.
Checklist
Before you submit this pull request, please make sure you have done the following:
General

[x] Contribution Guidelines: Have you read the contribution guidelines?
[x] Code of Conduct: Have you read the code of conduct?
[ ] Documentation: If applicable, have you added or updated all relevant documentation?

For Code Updates

[x] Hardware Validation: If relevant, have you validated your changes on actual supported Duck hardware?
[ ] Unit Tests: Have you run the unit tests on the device?
[x] Network Testing: If applicable, have you tested your changes on a Duck network?
[ ] Licensing: Have you added a copyright and license header to each new file?

Tested Targets (Please check all that apply)

[ ] All
[x] Heltec LoRa v3
[ ] Heltec LoRa v2
[ ] Heltec CubeCell Series
[ ] TTGO T-Beam (SX1262)
[ ] Others

I am going to merge this as we need the WiFi bug fix in the pre-release branch


Incremental constant pool resolution

Since the parser can be run several times, with different handlers and because only transitive closure of types reachable from the handlers is used to resolve the constant pool it would make sense to allow building up the constant pool progressively.
Technically it is already possible, just need to make sure we are not cleaning the data and type handlers between chunks and runs.

This will be addressed as a part of #2 - mostly because without this feature the parallel chunk processing is not possible.


How to customize file name?

I have two requirements, and I didn't find how to implement them when I read the documentation.

How to customize the file name
How to get the Id of the article

My usage in Obsidian looks like this:
I need to set the title of the article to (take the SimpRead application as an example):
  Extract-SR{{Id}}-{{Title}}

This indicates:

This is an Extract (to distinguish it from my normal notes, to avoid being polluted by the title when using `[[]]``)
This Extract comes from SR (SimpRead)
The serial number (Id) of this Extract

I don't know if my idea is reasonable, but it is effective in my practice.

Unfortunately there is no customisation of filename at this point in time. I am still looking into frontmatter that should be able to make this happen

id generated by Hypothesis only comes with the highlights, not the article. Hope that make sense.

Noting for posterity: I looked into doing this today, but encountered an architectural problem. If the user changes the file name template after a sync has already been performed, then the plugin won't be able to "line up" the old file names with new highlights added to those articles; it'll treat updates to previously-synced articles as it would a newly-highlighted article, creating a new note for those new highlights.
This is probably what @weichenw meant by the mention of frontmatter (#13): it seems like the "right" way to do this is probably to locate the article's file for updating by URL or some other unique indicator.

Noting for posterity: I looked into doing this today, but encountered an architectural problem. If the user changes the file name template after a sync has already been performed, then the plugin won't be able to "line up" the old file names with new highlights added to those articles; it'll treat updates to previously-synced articles as it would a newly-highlighted article, creating a new note for those new highlights.
This is probably what @weichenw meant by the mention of frontmatter (#13): it seems like the "right" way to do this is probably to locate the article's file for updating by URL or some other unique indicator.

If the file name template is changed during use, there is really no good way to match the new file name with the old file name. Matching through frontmatter is one way.
But I think the better way is not to match. Just use the new rules. This is my practice in SimpRead. The probability of changing the file name template midway is not too high, and the user can be guided to perform a full synchronization.
The synchronization between Obsidian and hypothesis is, after all, one-way synchronization, so many problems encountered in two-way synchronization will not be encountered here. I think full synchronization is an affordable cost.