iOS Keyboard Lags

Our users are reporting (on various iphones and OS versions) that there is extreme lag in the iOS keyboard. This does not seem to occur on Android.
flutter doctor
[✓] Flutter (Channel master, v1.4.11-pre.1, on Mac OS X 10.14.3 18D109, locale en-CA)

[✓] Android toolchain - develop for Android devices (Android SDK version 28.0.3)
[✓] iOS toolchain - develop for iOS devices (Xcode 10.1)
[✓] Android Studio (version 3.3)

We too are also facing this issue.
It is especially noticeable when using Japanese input with a T9 flick style keyboard.
It is also noticeable with emoji.
We dont see this issue in debug builds only in release builds and only on iOS.
The impact on performance is serve.
[✓] Flutter (Channel dev, v1.4.19, on Mac OS X 10.14 18A391, locale en-JP)

[✓] Android toolchain - develop for Android devices (Android SDK version 28.0.3)
[✓] iOS toolchain - develop for iOS devices (Xcode 10.1)
[✓] Android Studio (version 3.3)
[!] Connected device
    ! No devices available

This also happens with me with just plain numeric keyboard at random time (iPhone 5s + Flutter 1.4.19).

Also experiencing sporadic but severe iOS keyboard lag being reported by our iPhone 5, 6 and X users on the alphanumeric keyboard, doesn't occur on Android.

We found this issue does not occur on v1.3.13 but does occur on v1.3.14.

I'm not able to reproduce this on master. Are there any steps I can take to reproduce it more quickly? I've tried the emoji keyboard and alphanumeric.
The only thing that looks suspicious to me between 1.3.13 and 1.3.14 is that a new broadcast notification added for accessibility purposes, but that only fires if semantics are enabled and I'm not able to reproduce it with that on.

We still cannot find the root cause ourselves.
One thing we have found is setting maxLength on a TextField or TextFormField will be one way to trigger it.
We are try to make a minimal project to demonstrate the bug.
We cant do it at the moment in a new empty project so we are checking if maybe one of our dependencies is part of the problem

We got a minimum working example.
This should be run in --release mode, so your iOS project will need provisioning.
In Xcode set a custom target property key
io.flutter.embedded_views_preview of Type Boolean with a value YES

Setup main.dart like below.
import 'package:flutter/material.dart';

void main() => runApp(MyApp());

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        primarySwatch: Colors.blue,
      ),
      home: MyHomePage(title: 'Flutter Demo Home Page'),
    );
  }
}

class MyHomePage extends StatelessWidget {
  MyHomePage({Key key, this.title}) : super(key: key);
  final String title;

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: Text(title),
      ),
      body: Center(
          child: TextField(
        style: const TextStyle(
          fontSize: 48.0,
        ),
      )),
    );
  }
}

That's super helpful, thanks.  @iskakaushik this looks like a platform views issue

Can confirm we also have  io.flutter.embedded_views_preview key set to 'YES', which seems to be an remnant of our testing with the flutter web view.
We are currently testing @kitoko552's suggestion, by releasing an update of our app on flutter 1.3.13

In case it helps, some other clues, the problem we've seen in our app (Cronometer):

we can repro reliably on older iOS devices
On faster, newer devices we can repro only intermittently. When the keyboard comes up, it's either complete laggy, or completely functional for that typing session.
disabling predictive text / autocorrect made it a lot better, but didn't fully resolve the issue

@dnfield Any update on this issue?  We are coming through the same issue on Flutter 1.4.10 setting io.flutter.embedded_views_preview to YES

Thanks @kitoko552, as we compared the diff between v1.3.13 and v1.3.14, we found change in FlutterEngine.mm by this merge https://github.com/flutter/engine/pull/8045 causing this bug. @amirh

I'm trying to get someone more familiar with the Platform Views code to take a look.  I'm sure @amirh will be on this before long, he was on leave for a bit but is back now.

@dnfield hi,Is there any progress? estimate when we can solve this problem?

@amirh may know better now.

My users has reported very same problem. Some of them have problem to use the app. Delays up to 5-6 seconds. Today I have conducted several hours of stress testing. One page has several multiline TextFields. Entering random text for a while and the keyboards hangs for 4-10 seconds (iPad Pro). When hang is released all typed character is flushed into the TextField. Sometimes it is long between the hang situation. My first thought was the decoration or the TextCapitalization.sentences. Removing both these did not change the lags.
Another strange bug that I have found, very easy to reproduce:  Try to insert text between rows in a multiple TextField. Type space and it will not inserted until a none character is typed.
I hope very much that the lag problem will be solved soon. Otherwise the Flutter works like a charm.
Flutter (Channel master, v1.5.9-pre.27, on Mac OS X 10.14.3 . I have also tested with latest Stable Flutter version. Same result.

@amirh Could you please raise the priority to solve this issue. 'Cause text input is a base scenario of an app, all apps with Flutter v1.3.13+ should being facing this issue.

I have the same issues. Writing in an input text fields makes the app really laggy. This occurs only on iPhone XS for me. Android has no issues, or older iPhones like iPhone 7.
The issue is not reproducing on flutter channel stable, only on flutter channel beta.
@Hixie or someone else from the Flutter team, do you think you can prioritise this issue ? This makes our app unusable on iOS.
My flutter doctor -v:
Doctor summary (to see all details, run flutter doctor -v):
[✓] Flutter (Channel beta, v1.5.4-hotfix.2, on Mac OS X 10.14.4 18E226, locale en-GB)
[✓] Android toolchain - develop for Android devices (Android SDK version 28.0.3)
[✓] iOS toolchain - develop for iOS devices (Xcode 10.2.1)
[✓] Android Studio (version 3.4)
[✓] Connected device (1 available)

• No issues found!

Yesterday I sad: "I have also tested with latest Stable Flutter version. Same result."
The test was conducted on other mac that I was sure was on a stable channel. Checked it and it was on a master channel. Sorry about the desinformation.

I can't reproduce the issue.
I tried the sample app by @eoinfogarty on an iPhone 6s, but keyboard seems to perform as usual.
The regression is reported to start between v1.3.13 and v1.3.14, in this range we change the thread configuration for platform views from a single thread to merging just the gpu and platform threads.
I would suspect the thread configuration change as a potential culprit(though I do not see immediately why would that cause the reported regression).
Can anyone who is getting a consistent repo try to run the engine with a single thread configuration and report if there are still keyboard performance issues? Here's an engine patch to set a single thread configuration: https://github.com/amirh/engine/commit/486a88dce5b676749fad02385c19070511d81e31.patch (alternatively you can build the engine from this branch)

I am not sure about dates for v1.3.13 and v1.3.14. My first test user installed my app 2/4 (April) and reported lags the day after. iPhone XS and iPad (unknown version) had same the type of lag he reported.
One interesting observation:
When I run the app via AndroidStudio and tested the TestField input I was not able to get a hang during 30+ minutes. Typing between many fields. Back and forth. Works perfect.
I installed the app from App Store and got several hangs within 5 minuts. It was very ease. Typing text into on TextField and then tap another TextField. Hang directly. Did 4-5 times in row.
This pattern makes sense. Me and my users only found this hanging in a release version.

@eoinfogarty on which device do you get the minimal repro app you posted to show the issue? is it happening consistently? what are the repro steps on the device? should just using the English keyboard with some emojis show the issue?

I can confirm after following @kitoko552 suggestion to revert to v1.3.13, user reports of keyboard lag have evaporated. I've never been able to reproduce myself either with a device or the simulator.

@amirh Can you try on iPhone XS ? For me it works on older iPhones, but on iPhone XS I can reproduce the lag 100% of the time.

I just tried (a release build of the sample app by @eoinfogarty with io.flutter.embedded_views_preview set to YES) on an iPhone XS(and also on an iPhone X), and I can't see the issue. Are you seeing it reproducing with the same sample app?
Do you have any non default keyboard settings on the device?
Can you try with master and with the patch I posted?

If someone that can reproduce the issue with a profile build can record and share a timeline trace it might help us to understand what's happening. This article by Chinmay explains how to collect a trace: https://medium.com/flutter-io/profiling-flutter-applications-using-the-timeline-a1a434964af3

After several hours testing it seems that the release version, in my case, is that causing me most problem. I have had hang in debug mode but far from so often compared with the release mode.
I have made a recording:  https://youtu.be/m54CZEy_yHI . if you turn up the sound you can her how I tap without any text is showing. Problem seems to be when I first enter text and moving to next TextField. It is so ease to get a complete hang. 30 second of tapping and the page is hanging and after a while the page terminates.  This is worse that I first thought. It just not a minor hang.

@StephanCassel can you capture a trace when this is happening (try a profile build should be closer to release build but lets you take traces) and share it?

Thanks @kitoko552, as we compared the diff between v1.3.13 and v1.3.14, we found changes in FlutterEngine.mm by this merge flutter/engine#8045 causing this issue. @amirh
We reverted the changes in FlutterEngine.mm, the lags disappeared.

@amirh As I replied, it is the thread configuration change cause this issue.

@amirh I did a quick test under profiling mode and hang was as serious as in release mode.
I am not so familiar with the profiling tool. How can I capture a trace and share?
I understand that the lag problem is solved (@zhileichen) and that is very good news and I guess the next stable channel will include this fix, or exclude the bug. Any idea when next stable is planned?
(I have problem to roll back to current stable from master)

@StephanCassel could you follow the instructions here to capture the trace: https://medium.com/flutter-io/profiling-flutter-applications-using-the-timeline-a1a434964af3, thanks!

Thanks for information @iskakaushik
I guess it is not needed now when the hang is located but I will do this later today (after work). It looks interesting and very good to learn. Thanks again.

@StephanCassel unfortunately the issue is not yet understood, it seems that with a single threaded configuration it is less likely to happen but we still need to understand what's causing it.
We still need a trace, if anyone that can reproduce the issue can provide a trace it will be helpful.

@amirh   I did my very best.

Started the app in profile mode
Started the Dart DevTool  -> Timeline
Started recording
Typed a few character in two texfields. Had two lags.
Hope it will be of any help.

timeline_2019_5_7-1557250118986000.json.zip

@amirh

It is especially noticeable when using Japanese input with a T9 flick style keyboard.

It will occur every time for us using this keyboard.
To setup the Japanese input we can add it in settings.
Settings>General>Keyboard>Keyboards>Add New Keyboard... Japanese> Kana

This will give us this type of keyboard.
This type of input uses swipes when inputting instead of tapping.
Swipe a lot of characters rapidly up to 40-50 and the lag occurs every time for us.
We have seen it occur on
iPhone 8 (11.3)
iPodTouch 6G (12.2)
iPhone 6 Plus (12.2)

Can anyone who is getting a consistent repo try to run the engine with a single thread configuration and report if there are still keyboard performance issues?

I will try to set this up and also capture a trace

@amirh any process?

J@amirh
just updated my app to stable v1.5.4-hotfix.2 and was about to release it. App on iPad pro still has lag.
I did a profile test on iPhone 6 and sometimes the lag is severe. 10+ seconds delay.  I have to postpone the release or mention the problem in the release info.  This weekend I will make a stripped version for others to insert and test. It is very easy to reproduce. The risk is very high that users will uninstall the app under this condition. I have very hard to understand if this problem will not be solved soon and distributed in a stable v1.5.4-hotfix.3. This is showstopper. It is sad considering the fact that otherwise I found Flutter very good. Crossing my fingers.

@amirh
I could not wait with the test. I copy the "problem" page from my app to a new template app. No lags with same layout (number of fields) I start to suspect that it could be the size of my app. It has 30-35 pages. I am not sure about the flutter architecture. Will all widget be kept i memory? Could large widget trees be a problem? It seems that we have different lag problem in this thread.

@amirh
I have identified the problem in my case:
Before I get to my page of TextFields, the app shows a list of items from Firestore via a stream-snapshot. if I enter a Navigator.pop(content) before the form, i.e kills the stream everything works perfect. No lags. Android handles streams perfect.
@amirh: can you pass this problem to the "Stream-Group"? All iOS apps that has TextFields after stream ListView probably have this problem.
Thanks

@StephanCassel and @all, I've also reproduced this problem with just a single textfield within a page (using io.flutter.embedded_views_preview and release mode), trust me, I've tried so conditions and it lags everytime.  Please help to fix this as soon as possible, we are having 2 apps in sotre, it's quite annoying.  By the way, Flutter rocks!

Just wanted to update that I currently have no estimate for how long it's going to take until we figure out this issue(I'm personally won't be available to look at this for at least a few more work days, and even then as we don't understand the problem at this point I don't know how long it's going to take to fix).
For those of you who are launching apps with platform views (which is still in developers preview BTW), if you find a single threaded configuration performant enough and you can't wait, you could temporarily patch a local copy of the engine to use a single threaded configuration(or use Flutter v1.3.13).

@amirh and @ALL
Yesterday I was 100% sure it was the streaming that caused the problem due long test without lag.
To be sure I did more tests, with streams, this early morning to be sure I am on the right track.
Lags is back. Severe lags. Hmmm. Very sad indeed.

···dart

@amirh and @ALL
Breaking news
I have done some intense testing in order to find why there are severe hangs in TextFields.
Could not promote my app to larger audiens so I had to find the problem.
After stripping the code (edit form) down step by step the problem suddenly disappears.
It was the textCapitalization that caused my TextFields to lag and sometimes completely hang.
I put them back again and the lags came back. After removing them I did a 30+ minute typing test.
No hang. Now and then a small 1-2 sec delay when I change field, ie when a new TextField gets input focus but compared to earlier situation it is a minor problem.
Even textCapitalization removal did the trick for me I hope it will be fixed. The textCapitalization makes the TextField much more user friendly.

is there any progress?

I repreduce it also although very random on v1.5.4-hotfix.2 inside WebView which displays text fields inside the webpage. Also in textelements in flutter widgets. Different versions of iOS, only on phisical devices. Will try the patch of @amirh with single thread to see if works.

@StephanCassel thanks for the trace! Unfortunately it only shows the UI thread.
Can you make sure to check "All" for "Recorded streams:" before collecting the trace and post again?

flutter and textinput's, proof why shouldn't reinvent the wheel.

@amirh Absolutely. I am currently on a business trip. I do that later this week.

@amirh and all
Severe lags are back. I thought it was textCapitalization due the fact it was very stable for long period of time. Users report that they cannot signup due hang, probably long lags. I immediately tested the app on a iPhone 6 and I also got lags. Lag comes and goes.  Thread  #32584 hinted that Google Map can be the cause. This issues has become a showstopper for me and for 100% certainty other Flutter developers also.
I will of course do anything to help. Will upload a trace later.

I’ve never got rid of it, never had the luxury of it going away. 😔

Mind you, we are in the same boat. My lags is back 😔

@StephanCassel : no, it's not related to Google Maps because I get hangs constantly without any actual native widgets used in whole app, just with they are enabled (io.flutter.embedded_views_preview true).

@daniilguit true, anything uses io.flutter.embedded_views_preview hangs.  @StephanCassel FYI

@amirh Dev tool - Timeline gives me two channel UI and GPU. Both seems to be enabled. I am desperately looking for the checkbox "All". Are you talking about the Dev Tool web page?

@daniilguit  If "anything uses io.flutter.embedded_views_preview hangs" is the case, is anyone looking into the problem?

I have the same problem. Modifying this setup really solves the problem. （
io.flutter.embedded_views_preview   ->   false  ）No impact on the program was found.
I use these plug-ins
dio: ^2.1.3
intl: ^0.15.8
flutter_webview_plugin : ^0.3.5
url_launcher: ^5.0.2
event_bus: ^1.1.0
shared_preferences: ^0.5.2
cached_network_image: ^0.8.0
flutter_datetime_picker: ^1.2.0
image_picker: ^0.6.0+3
barcode_scan: ^1.0.0
flutter_picker: ^1.0.11
device_info: ^0.4.0+1
flutter_keychain: ^1.0.0
package_info: ^0.4.0+3
image: ^2.1.0
sy_flutter_qiniu_storage: ^0.1.4
flutter_swiper: ^1.1.6
flutter_umpush: ^0.0.3

[✓] Flutter (Channel stable, v1.5.4-hotfix.2, on Mac OS X 10.14.5 18F132, locale zh-Hans-CN)
    • Flutter version 1.5.4-hotfix.2 at /usr/local/flutter
    • Framework revision 7a4c33425d (2 weeks ago), 2019-04-29 11:05:24 -0700
    • Engine revision 52c7a1e849
    • Dart version 2.3.0 (build 2.3.0-dev.0.5 a1668566e5)

[✓] Android toolchain - develop for Android devices (Android SDK version 28.0.3)
    • Android SDK at /Users/baijikang/Library/Android/sdk
    • Android NDK location not configured (optional; useful for native profiling support)
    • Platform android-28, build-tools 28.0.3
    • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1343-b01)
    • All Android licenses accepted.

[✓] iOS toolchain - develop for iOS devices (Xcode 10.2.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 10.2.1, Build version 10E1001
    • ios-deploy 1.9.4
    • CocoaPods version 1.5.3

[✓] Android Studio (version 3.4)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin version 35.3.1
    • Dart plugin version 183.6270
    • Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1343-b01)

[✓] VS Code (version 1.33.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.0.2

[!] Connected device
    ! No devices available

! Doctor found issues in 1 category.

Your dependencies no need to use the option 'io.flutter.embedded_views_preview = true'. So there's no harm for your fixed.
flutter_webview_plugin use shared GL context, not platform view, but webview_flutter do use platform view.

I have the same problem. Modifying this setup really solves the problem. （
io.flutter.embedded_views_preview -> false ）No impact on the program was found.
I use these plugins
dio: ^2.1.3
intl: ^0.15.8
flutter_webview_plugin : ^0.3.5
url_launcher: ^5.0.2
event_bus: ^1.1.0
shared_preferences: ^0.5.2
cached_network_image: ^0.8.0
flutter_datetime_picker: ^1.2.0
image_picker: ^0.6.0+3
barcode_scan: ^1.0.0
flutter_picker: ^1.0.11
device_info: ^0.4.0+1
flutter_keychain: ^1.0.0
package_info: ^0.4.0+3
image: ^2.1.0
sy_flutter_qiniu_storage: ^0.1.4
flutter_swiper: ^1.1.6
flutter_umpush: ^0.0.3

[✓] Flutter (Channel stable, v1.5.4-hotfix.2, on Mac OS X 10.14.5 18F132, locale zh-Hans-CN)
    • Flutter version 1.5.4-hotfix.2 at /usr/local/flutter
    • Framework revision 7a4c33425d (2 weeks ago), 2019-04-29 11:05:24 -0700
    • Engine revision 52c7a1e849
    • Dart version 2.3.0 (build 2.3.0-dev.0.5 a1668566e5)

[✓] Android toolchain - develop for Android devices (Android SDK version 28.0.3)
    • Android SDK at /Users/baijikang/Library/Android/sdk
    • Android NDK location not configured (optional; useful for native profiling support)
    • Platform android-28, build-tools 28.0.3
    • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1343-b01)
    • All Android licenses accepted.

[✓] iOS toolchain - develop for iOS devices (Xcode 10.2.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 10.2.1, Build version 10E1001
    • ios-deploy 1.9.4
    • CocoaPods version 1.5.3

[✓] Android Studio (version 3.4)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin version 35.3.1
    • Dart plugin version 183.6270
    • Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1343-b01)

[✓] VS Code (version 1.33.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.0.2

[!] Connected device
    ! No devices available

! Doctor found issues in 1 category.

This happens not only in flutter text inputs but also if you open a webview and have a basic form with html input element. I have both cases in my app. However it is a random event only on iOS. Very annoying for customers. Looking forward for a hotfix to stable channel.

@dnfield Yes, it is very annoying for the customers. And it is extremely important for Flutter as the mobile development tool for the feature. I must say that I will be very surprised if no hotfix for the stable channel is out very soon.

I just took a quick stab at reproducing this and wasn't able to - tried using the Japanese Kana keyboard, and also tried with some values for textCapitalization.
@amirh and @iskakaushik know more about this feature than I do, and I don't think either of them have been able to reproduce it either.
It would be really great to get a valid timeline trace for when this bug occurs, or a consistent reproduction.  In the mean time, you have a few options as mentioned already by Amir:

Go back to an earlier stable version
Patch your engine with the patch he linked above to go back to the previous thread configuration
Avoid using the experimental feature altogether

We definitely want to fix this bug, but in the absence of a good reproduction or at least a trace file it's pretty tough right now.

@StephanCassel: If you question me, I'm not a Flutter developer, unfortunately :)

No, I just aggreed with your comment. Hopping for a hotfix soon 👍
Den fre 17 maj 2019 08:58Daniil Gitelson<EMAIL_ADDRESS>skrev:

@StephanCassel https://github.com/StephanCassel: If you question me,
I'm not a Flutter developer, unfortunately :)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/flutter/flutter/issues/31138?email_source=notifications&email_token=ABI7WWX2FF3EGXHGDFTLYN3PVZJP3A5CNFSM4HGN6Y6KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODVT5GEY#issuecomment-493343507,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABI7WWS7G7PWVVKPPP7YLR3PVZJP3ANCNFSM4HGN6Y6A
.

Ah, I just realized I was trying to repro in debug mode.  Got it to reproduce in profile mode.  Trace attached.
/cc @iskakaushik @amirh @chinmaygarde
dart-timeline-2019-4-17.json.zip

Another with Skia tracing enabled:
dart-timeline-2019-4-17_with_skia.json.zip
Little hard to tell if this is actually a Skia issue, since it could be that the thread is getting overloaded...  But it looks like Skia is taking too long to flush the canvas.

@amirh showed me the trace today. We still need more information. I'd collect another trace using an engine with instrumented GL calls. Once we have verified that the stall is in the OpenGL driver as suspected, lets also get some traces with the OpenGL frame debugger on iOS. If that turns up nothing, we could try tweaking the frame pipeline depth to see how that affects the wait.

Not sure about the cause, but the effect I see on my iPhone is a severe processor usage during the time the bug manifests. The keys hardly show when pressed. I stress that I noticed the same behavior inside HTML INPUT fields inside a flutter WebView. For me it seems some code uses the processor like in infinite loops. Loking at the fluter inspector I saw that each key pressed inside my HTML page in the webview generated redraw events inside the widgets of flutter, which I didn't expect to happen since I was typing inside webview. Hope you find a solution soon.
Thank you for all the hard work!

Hi
Any progress? I understand the Flutter team working hard on this issue.
I am about to release new functionality but would like to postpone the new release couple of days/weeks if there is any chance of a stable hotfix.3.
Crossing all my fingers.

I downgraded the flutter version to 1.3.13 and the issue was resolved but I one issue resurfaced https://github.com/flutter/flutter/issues/29289 which was fixed when I upgraded to 1.5.4 version.

@aabanaag I see the #29289 bug also in 1.5.4, but happens not so often and goes away after restart of the app. Looks like it is linked to lack of resources. This one however with keyboard lags is really annoying as it happens often in input fields which makes customers angry as they cannot type. @dnfield please push the team and give us a fix soon, some apps are already in production and high revenue clients experience on their shiny iPhonesX this crazy behavior. Thank you!

@dnfield My app is in production and I get comment about lags frequently. I will repeat my self and say that Flutter is one of the best app development tool yet. Rapid development. Clean code, thanks to Dart (I am a Java, Swift, C# type of guy. Not JavaScript/CSS/HTML). No doubt that Flutter will be a winner if all cards is played correct. The lag problem is very serious for end users and for Flutter. I am 100% sure this will be fixed. In fact there is no option. I am still a huge fan of Flutter and will keep on fluttering. Thank you.

Setting io.flutter.embedded_views_preview to NO on info.plist resolved the lag problem. But then google maps are not going to works.
<key>io.flutter.embedded_views_preview</key>
<string>NO</string>

Also we can't go back to Flutter v1.3.13 because we are using Dart 2.3 features.
To simulate this keyboard lag it MUST be executed in PROFILE or RELEASE mode.
We really need this to be fixed, our iOS app is useless.

Me too need fix the issue.

I am facing the same issue, wish it would be fixed. Thank you very much

I raise the bar offering 1 week of free electric scooter rides in Bucharest to whoever fixes this issue and patches the 1.5 stable flutter build. We deployed in production to 6k+ users, half of them seeing this bug on iPhone. I need google map and webview so cannot workaround it. @amirh @dnfield please help us!

Here's a trace with extra GL tracing available.  It looks like the Clear command is taking a while in some instances and ends up blocking the platform thread.  Unfortunately this trace was not as severe as some of the ones I've seen - this bug is hard for me to consistently reproduce, and there's no set of input or combination of changes I can find to consistently reproduce it.  I basically just keep trying to type for a few minutes until it happens.
Towards the end of the trace, there are a few longer SkCanvas::Flush calls.
dart-timeline-2019-4-28_gl.json.zip

I eventually realized that trace was done in debug mode, where the problem is not as severe.
Here's what I have at this point:

If we say NO to presentsWithTransaction, the lag goes away.
If we go back to single threaded configuration, the lag goes away.
The lag is very reproducible when run in profile mode - but does require steady typing for up to 30 seconds before manifesting.
It may get better with metal.  I did a local build with metal, but that's missing all the plumbing for embedded platform views.  I'll open a new bug to track that.
There may be some way to resolve this by more cleverly managing the CATransaction, but I haven't been able to figure anything futher out there.

Can you temporarily fix it now?

In my situation, this issue just happen occasionally. I have to try several times to reproduce it from focusing and inputting one textfield to another one.
flutter doctor -v
[✓] Flutter (Channel dev, v1.5.8, on Mac OS X 10.14.4 18E226, locale en-CN)
• Flutter version 1.5.8 at /Users/xxx/Development/flutter
• Framework revision 0ba67226ee (5 weeks ago), 2019-04-24 17:18:28 -0700
• Engine revision c63d1cf9c9
• Dart version 2.3.0 (build 2.3.0-dev.0.1 1f1592edce)
[✓] Android toolchain - develop for Android devices (Android SDK version 28.0.3)
• Android SDK at /Users/xxx/Library/Android/sdk
• Android NDK location not configured (optional; useful for native profiling support)
• Platform android-28, build-tools 28.0.3
• Java binary at: /Applications/Android
Studio.app/Contents/jre/jdk/Contents/Home/bin/java
• Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1343-b01)
• All Android licenses accepted.
[✓] iOS toolchain - develop for iOS devices (Xcode 10.2)
• Xcode at /Applications/Xcode.app/Contents/Developer
• Xcode 10.2, Build version 10E125
• ios-deploy 1.9.4
• CocoaPods version 1.5.3
[✓] Android Studio (version 3.4)
• Android Studio at /Applications/Android Studio.app/Contents
• Flutter plugin version 35.2.1
• Dart plugin version 183.6270
• Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1343-b01)
[✓] VS Code (version 1.34.0)
• VS Code at /Applications/Visual Studio Code.app/Contents
• Flutter extension version 3.0.2
[✓] Connected device (1 available)
• xxx's iPhone • xxxxxxx • ios • iOS 12.2

https://openradar.appspot.com/49442041 seems related - seeing similar blocks for 1 second in a somewhat similar case.  Will have to see if this reproduces in iOS <12.2 though.

https://openradar.appspot.com/49442041 seems related - seeing similar blocks for 1 second in a somewhat similar case. Will have to see if this reproduces in iOS <12.2 though.

Yup, same behaviour on an iPhone XS Max with iOS 12.1.1

Need hot fix now!!!!!!!!!!!!!!

@tvqphong001 Hold your horses. They are working 100% to find a solution as we all can see on previous posting. We all need a solid solution and that will come.

@Stephan-Enoro I believe They can find best solution.

@dnfield and @all  This bug is very sever for many users. Users are sometimes not able to signup, depending of iOS unit. My app uses 17 plugins. Each with a score > 98. I take it for granted that serious plugin developer test against the stable branch, or close to.  In production you should not use anything but stable branch. I have tested my app thoroughly and it works perfect except from the lags.  I cannot introduce new bugs in this situation. I don't know if all 17 plugin is tested to this new beta version.
I think that must many developers, with apps in production, that agree.
Talked to old colleague a week ago. She is a React Native developer. She had heard about this lag problem. Not from Git forum but from another colleague. I told here, of course, that it will soon be fixed.
It should be prioritised and released in a stable hot-fix. If this bug is not a candidate for a hot-fix I will surprised.

@StephanCassel we all made a bet when choosing to work with Flutter and I believe it was a good one. Let's let them work on it and I am sure we will get a good workaround until the hotfix will be available. It is indeed a strange and difficult to reproduce bug, mainly because of native controls which are not entirely supported from what I understand.

@bogdannedelcu Copy that and I agree. It will be fixed. Keep on fluttering.

We have a temporary improvement for this that just landed in the engine.  It should be available on master sometime in the next day-ish, and in dev sometime shortly after, and as far as I can tell leaves us in as good a place as we were before the threads were unmerged in v1.3.14.
The "real" fix for this is going to take some more time.  It's still possible to run into problems when using PlatformViews on iOS, but there are several additional efforts being undertaken to improve those (some of which are described in flutter/engine#9132).
Nevertheless, please keep in mind that platform views are still considered experimental - there are still some rough edges on them, and they still require special flags to be set to enable their use at all on iOS. If you're looking for this fix sooner than later, you could apply the linked patch to the engine code from stable, build the engine, and use that local release build for your apps.  Otherwise, you should probably wait until this lands in beta (or perhaps dev) before moving forward to upgrade.

Can you help me compile version 1.5.4-hotfix.2 of engine?

Now I'm waiting for the release, and I can't download engine.

@dnfield
Thanks for the effort, we appreciate it a lot!

@dnfield
Thank you for the fix for this issue.

If you're looking for this fix sooner than later, you could apply the linked patch to the engine code from stable, build the engine, and use that local release build for your apps.

I am attempting to do this. I have built a version of the engine with your change off stable and have this working:
flutter -v run --local-engine=ios_release --release
However, this returns an error:
flutter -v build ios --local-engine=ios_release --release

ld: warning: Could not find auto-linked framework 'Flutter'
Undefined symbols for architecture armv7:
"FlutterMethodNotImplemented", referenced from:
app_review.SwiftAppReviewPlugin.handle(: __C.FlutterMethodCall, result: (Any?) -> ()) -> () in SwiftAppReviewPlugin.o
function signature specialization <Arg[1] = Dead, Arg[2] = Owned To Guaranteed> of function signature specialization <Arg[1] = [Closure Propagated : reabstraction thunk helper from @escaping @callee_unowned
@convention(block) (@unowned Swift.Optional<Swift.AnyObject>) -> () to @escaping @callee_guaranteed (@in_guaranteed Swift.Optional) -> (), Argument Types : [@escaping @callee_unowned @convention(block) (@unowned
Swift.AnyObject?) -> ()]> of app_review.SwiftAppReviewPlugin.handle(_: __C.FlutterMethodCall, result: (Any?) -> ()) -> () in SwiftAppReviewPlugin.o
"OBJC_CLASS$_FlutterMethodChannel", referenced from:
objc-class-ref in SwiftAppReviewPlugin.o
ld: symbol(s) not found for architecture armv7

All my plugins get this error. Any idea why flutter run works, but flutter build does not? Is there something else I need to do to build/submit a version of the app with the local engine?
Thank you for your help!

I'm not sure off the top of my head, but I believe at this point the change should have landed in dev.  Can you try on the dev branch?  It may also be in beta, but dev for sure.

@dnfield
Thank you for the reply.
Your change appears to be in the latest dev release (not beta yet) but we don't feel comfortable releasing our app from dev. Releasing from stable with just this one fix seems better (unless I'm not understanding something?).
Any ideas about the ld: symbol(s) not found for architecture armv7 error?

may i know which version of dev fix this issue?

When can it be solved?

When can it be solved?

I need a solution, too :(

Hope this can be solved very soon. I get so many comment from my users about difficulty to enter text in the app. In addition to the keyboard lag: if you are using multiple line TextField? Type two lines of text. Move to cursor above these two lines and try to add space. Space shows app first after you type another non space character.
I am afraid to say that Flutter is difficult to use if you need to develop an app with text input.
That sad. Flutter is otherwise a fantastic app tool.
My app has now popup dialogs that describe the problem to prevent users to think that they have problem with the device.
I would be very happy if any Flutter framework developer could explain to me why there is no solution in a stable-hotfix branch.  I have great respect for hard solving problem but I think it important so be open about the progress. Any answer is good for me.
Thanks in advanced

The temporary improvement for this should be available in v1.6.7, which I think is either beta or the tag right after beta.
Are you still seeing this issue at that version?

@dnfield I guess this problem is solved in the next stable branch.
Apart from the TextField problem I have no other problem and I would keep using teh stable-hotfix2 in production.  My app imports about 20 plugin  and I have used other branches with mixed success. Some plugins do not go very well with beta/dev/master branches. It seem that the approach is to do the final test with stable branch. I support that approach. In the present situation I would not like to introduce new problems. I found it better to wait for new stable version

Hope to be settled in the stable version soon.

out of interest, now that is has been “fixed” does that mean it’s no longer being worked for a permanent solution? Temporary doesn’t make me feel very comfortable.

The patch I introduced had an upside of making the lag unreproducible in my tests. However, it also reduces concurrency in the rasterizer processing, which limits some of the benefits of splitting out the threads to begin with. I also cannot explain with 100% certainty why it's helping things - it just seems like it prevents the GPU and CPU from getting too far out of sync with each other, perhaps due to bugs or limitations in Apple's Open GL implementation or perhaps due to some other unknown issue in Flutter.
There are people working on a better solution for this, but none of them are simple to implement - so we agreed that the patch I made can land for now to at least make things better while the broader solution is worked on.  There are more details at https://github.com/flutter/engine/pull/9132

@calvinmuller @iskakaushik and others are working on a longer-term solution for this, but as @dnfield notes, those will involve multiple PRs (e.g. https://github.com/flutter/engine/pull/9436).

I appreciate all the efforts the flutter devs put into fixing this issue. We're about to release our app, unfortunately I am afraid users might instantly close the app when facing 5-seconds-lags on signup-page right after installing the app. May I ask what the status is? Any idea when a hotfix or something might be released? Dont get me wrong, flutter is awesome, but this issue is a big one for everyone using Google Maps or Webviews. Thank you :)

same expectation as well, it is a BIG SHOWSTOPPER for us.

This should not be showing up on the latest stable (as of today) - can you try on stable now?

I'll try it later.

Thank you, dnfield, for the update. So far so good, I cant reproduce the issue on my iphone 5c any more :)
Put my updated app on testflight, so hopefully there will be more confirmations flying in during the day.
I'll come back later with more results...

Three more confirmations: No more lags so far. Looking good! Thank you, dnfield and team :)

@dnfield we are struggling to upgrade to lastest stable version cause some uncompile issue, so far we revert back to 1.5.4-hotfix.2 for productive release. any suggestion? is it possible to downport the hotfix for lower version? meantime we are going to figure out a way to solve the upgrading issues.

If you can pick up some commit after https://github.com/flutter/flutter/commit/381fec583c62b8ba29821c4c2773352b04d23c04 it should work.  Otherwise you could patch the engine locally and build it from source.

now that it's solved in stable 1.7.8-hotfix.4, should this issue be closed even if the current one is the temporary fix only? We can open a new enhancement issue for more optimized fix.

I think that's a good idea.  I've opened https://github.com/flutter/flutter/issues/37665 to track the enhancement.

@dnfield

This should not be showing up on the latest stable (as of today)

Is there anything like a changelog I can watch to find out what's in the hotfix releases?

flutter-announce is the best place to check - see https://groups.google.com/forum/#!topic/flutter-announce/63ZfjAKMfB0
This bug was fixed well before that though...

This bug was fixed well before that though...

Ah!  I see 381fec58 has been in since v1.7.8+hotfix.1.  Thank you.
@kf6gpe Is filing an issue in this repo the best way to request more regular and detailed release notes?


Raise a warning upon import/export of a workflow with a custom task

Ref:

https://github.com/fractal-analytics-platform/fractal-server/issues/729

Note a task is "custom" if task.owner is not None

Re: export

Call the workflow-export endpoint
Check the workflow/task_list/element/task/owner attributes, and raise a warning if appropriate
Continue with the endpoint

Re: import

Call the workflow-import endpoint
Look at its response body, check the workflow/task_list/element/task/owner attributes, and raise a warning if appropriate

Example of the warning message upon workflow export:
    # Emit a warning when exporting a workflow with custom tasks
    for wftask in workflow.task_list:
        if wftask.task.owner is not None:
            logging.warning(
                f"Custom tasks (like the one with id={wftask.task.id} and "
                f'source="{wftask.task.source}") are not meant to be '
                "portable; re-importing this workflow may not work as "
                "expected."
            )

AttributeError: 'TaskExport' object has no attribute 'owner'
``

Let's do it in a different way, that is, by checking whether `task.source` starts with `pip_local:` or `pip_remote:`.

This is not the most robust choice ever (in case we change the source structure in the future, or in case a user has username `pip_local`), but it's acceptable - since it is only about raising a warning.


[SPARK-35259][SHUFFLE] Rename ExternalBlockHandler Timer variables to remove incorrect millis suffix

What changes were proposed in this pull request?
ExternalBlockHandler exposes 4 metrics which are Dropwizard Timer metrics, and are named with a millis suffix:
    private final Timer openBlockRequestLatencyMillis = new Timer();
    private final Timer registerExecutorRequestLatencyMillis = new Timer();
    private final Timer fetchMergedBlocksMetaLatencyMillis = new Timer();
    private final Timer finalizeShuffleMergeLatencyMillis = new Timer();

However these Dropwizard Timers by default use nanoseconds (documentation). It's certainly possible to extract milliseconds from them, but it seems misleading to have millis in the name here.
This causes YarnShuffleServiceMetrics to expose confusingly-named metrics like openBlockRequestLatencyMillis_count and openBlockRequestLatencyMillis_nanos (the actual values are in nanos). It should be up to the metrics exporter, like YarnShuffleServiceMetrics, to decide the unit and adjust the name accordingly, so the unit shouldn't be included in the name of the metric itself.
This PR originally proposed to remove the 'millis' suffix from these metrics; however, this has issues with backwards compatibility. For that reason, only the internal variable names are updated, hopefully helping to reduce confusion and prevent the same mistake in the future. The documentation has also been updated to address this discrepancy.
Does this PR introduce any user-facing change?
Minor changes in the documentation only.
How was this patch tested?
Unit tests have been updated. Changes are a trivial find-and-replace rename.

cc @jaceklaskowski @HyukjinKwon @Ngone51 @mridulm @otterc
My only concern here is whether this will be considered backwards-incompatible due to the metrics name changes. I'm not sure what the contract is for metric names, so any input here would be appreciated.

Got it, thanks for chiming in @dongjoon-hyun ! I'm fine with just renaming the variables for now and neutral on adding correct names alongside the old ones. I will wait for other folks to chime in before updating.

Test build #140361 has started for PR 33116 at commit faa0925.

Test build #140361 has finished for PR 33116 at commit faa0925.

This patch fails to build.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140361/

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44889/

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44889/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44889/

@dongjoon-hyun updated per your comments, PTAL. I also updated the documentation to address the discrepancy.

Test build #140544 has started for PR 33116 at commit 9681c36.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45057/

Thank you for updates, @xkrogen .

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45057/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45057/

Test build #140544 has finished for PR 33116 at commit 9681c36.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140544/

Just put up a new diff which implements a custom Timer subclass, TimerWithMillisecondSnapshots, which acts the same as a normal Timer and stores nanoseconds internally, but exposes all values as milliseconds. It works pretty cleanly.
For YarnShuffleServiceMetrics, I removed the nanos suffix that the reporter was adding, so it is now unit-agnostic. Note that this suffix only appeared in 3.2.0 (via SPARK-25358 / #32388), so it hasn't been released yet and we can safely remove it if we get this PR into branch-3.2.
My only concern with this approach is if some other metrics reporter (besides YarnShuffleService) may try to use these custom timers as if they still had nanosecond units. I'm not familiar enough with how the metrics are used in the non-YARN context to know if this will be an issue, so input is welcomed here.

Test build #140933 has started for PR 33116 at commit 8e8f6b3.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45445/

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45445/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45445/

Test build #140933 has finished for PR 33116 at commit 8e8f6b3.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
 static class TimerWithMillisecondSnapshots extends Timer

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140933/

My only concern with this approach is if some other metrics reporter (besides YarnShuffleService) may try to use these custom timers as if they still had nanosecond units. I'm not familiar enough with how the metrics are used in the non-YARN context to know if this will be an issue, so input is welcomed here.

Ideally, one should learn the usage before using the timer. That being said, I think we can force the TimerWithMillisecondSnapshots to receive a time-unit parameter to indicate which time-unit it exposes.
WDYT?

I think we can force the TimerWithMillisecondSnapshots to receive a time-unit parameter to indicate which time-unit it exposes.

I don't think we can force callers to supply a time unit, since (a) we should adhere to the Timer interface, which already exposes the unitless methods, (b) the whole issue here is that these metrics themselves are trying to define their unit, and if the caller supplies a unit, then the caller has to somehow know that this metric wants to be in milliseconds, which seems to defeat the purpose.
If we want to go down a route like this, it's basically what I was proposing initially where YarnShuffleServiceMetrics does the conversion. The Timer interface doesn't provide any way to extract unit information, but we could either check if the metric name ends in "millis" (and if so, treat the unit as millis instead of nanos) or create a custom extension which has a method defining the units, like:
  static class TimerWithUnits extends Timer {
    private final TimeUnit timeUnit;
    TimerWithUnits(TimeUnit timeUnit) {
      this.timeUnit = timeUnit;
    }
    public TimeUnit getTimeUnit() {
      return timeUnit;
    }
  }

Then within YarnShuffleServiceMetrics#collectMetric:
      TimeUnit timeUnit;
      if (metric instanceof TimerWithUnits) {
        timeUnit = ((TimerWithUnits) metric).getTimeUnit();
      } else {
        timeUnit = TimeUnit.NANOSECONDS;
      }
      // do some conversion on the snapshot based on the unit above

(a) we should adhere to the Timer interface, which already exposes the unitless methods,

Although the Timer exposes the unitless methods, it doesn't mean the methods don't have the time unit. Actually, I think this the key point that why we misused the Timer in the first place. The timer has hidden the time unit.  So, wouldn't it be clearer if we force a unit requirement while creating the (our custom) Timer?

(b) the whole issue here is that these metrics themselves are trying to define their unit, and if the caller supplies a unit, then the caller has to somehow know that this metric wants to be in milliseconds, which seems to defeat the purpose.

TBH, I don't understand this well...The use case I image is like this, e.g.,
allMetrics.put("metricA", new OurTimer(timeUnit = TimeUnit.SECONDS))
allMetrics.put("metricBMs", new OurTimer(timeUnit = TimeUnit.MILLISECONDS))

It's fine for the metric to omit the unit suffix. But, if the metric has the unit suffix, then, it's the developer's responsibility to ensure the time unit consistent between the metric and Timer.

Then within YarnShuffleServiceMetrics#collectMetric:
TimeUnit timeUnit;
 if (metric instanceof TimerWithUnits) {
  timeUnit = ((TimerWithUnits) metric).getTimeUnit();
 } else {
  timeUnit = TimeUnit.NANOSECONDS;
}
 // do some conversion on the snapshot based on the unit above

And, with my understanding, the time unit should be used at all related places inside the timer (Context, Histogram,  etc). So, all APIs exposed by the Timer would return the values based on the specified time unit directly. Thus, we don't have to do the manual conversion after getting the return value.

Ah, I see, I misunderstood what you meant. I thought you meant that the place where the values are extracted from the Timer (e.g. YarnShuffleServiceMetrics) would supply a unit, but you are saying that it would be where the Timer is created. I agree with you.

Pushed up a new commit changing TimerWithMillisecondSnapshots to TimerWithCustomTimeUnit, which will accept a configurable time unit as a constructor parameter. I also moved it into the org.apache.spark.network.util package within network-common since it is becoming more general. @Ngone51 let me know what you think.

Test build #141082 has started for PR 33116 at commit ca1c396.

Kubernetes integration test unable to build dist.
exiting with code: 141
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45596/

@dongjoon-hyun @Ngone51 gentle ping, any thoughts on the latest diff?

Sure, I'll take a look later.

Pushed up a new commit fixing a few old references to milliseconds and adding testing for the Timer.Context API. Thanks for the review @Ngone51 !

Test build #141445 has started for PR 33116 at commit d62dd57.

Kubernetes integration test unable to build dist.
exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45964/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45964/

Test build #141445 has finished for PR 33116 at commit d62dd57.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141445/

LGTM if tests pass.

@dongjoon-hyun it'd be great if you could take a look too.

@xkrogen Could you update the PR description?

Pushed up new commit fixing the indentation.
Looks like the GitHub Actions failures are due to the issues being worked on in #33475.
The Jenkins build also failed with one test:
org.apache.spark.sql.execution.DataSourceV2ScanExecRedactionSuite.SPARK-30362: test input metrics for DSV2

It looks unrelated and passes locally.

Also updated description, thanks for the reminder @Ngone51 ! It had gotten pretty outdated :)

Thanks, merged to master/3.2.

@xkrogen could you also backport this to 3.1/3.0?

+1, late LGTM. Sorry for missing pingings here. Thank you, @xkrogen and @Ngone51 .

Awesome, many thanks @Ngone51 and @dongjoon-hyun !
Backport PR for 3.1: #33523


feat: add support for filtering using excluding windows

What does this PR do?
This PR adds support for excluding certain windows while capturing a display.
Why do we need this change?
We want the ability to ignore certain windows ( like the camera, cropper etc ) when capturing screen using Helmer
How are we doing this change?

First, we needed to move the screen capturing with display to use ScreenCaptureKit instead of CoreGraphics. Here we also add support for excluding certain windows using the excludingWindows param using init. To achieve this we also make the corresponding changes in objc files.
Next we update CapturableContent to also return the list of windows to be excluded.
We also update the with_display func to take a list of windows to be excluded. This list is then propogated forward and used while creating a stream to capture.
Finally, we also expose a id() func on the window object to make it easier to get the ids of windows to be excluded.

What improvements can be done?

Add support for windows
Improve the excluding_windows fetching logic to only loop through windows once.

How has this been tested

Manual testing in Helmer by connecting this version with omni, which in turn is connected to Helmer

Merging it for now.
Will address review comments if any in a separate PR

LGTM. The only change I'd make is renaming the field to exclude_windows instead of excluded_windows but just a nitpick.


Fixing paused video doesn't react on changing device orientation

Reacting on #194 I found that problem is in condiiton:
frame == prev_frame_
in apple_video_renderer.cc. It returns nullptr in this case and renderLoop inside ShakaPlayerView.mm doesn't pass:
if (CGImageRef image = _player.videoRenderer->Render(nullptr, &aspect_ratio)) {..
condition
So solution would be remove frame == prev_frame_, but this could brings unnecessary overhead to rendering loop.
So I limit this condition only for case,  that video is paused. To make it possible, I have to make player_ property protected, to be inheritable from parent class. Also I had to move mutex_ up to player_, due to some compilations errors.
This works and bug is fixed.
I could imagine there should be better solution, directly in file ShakaPlayerView.mm - watching screen orientation change and only forcing recalculation in that case. Additionaly storing some informations from last render and use them for paused state. But I don't want to make huge updates in original source code.
Fixes #194

This will have some major performance impact.  When we're paused, users would expect the app to be doing (almost) nothing.  This would require us to regenerate the same image 30+ times a second.  This would be better done in ShakaPlayerView.  The orientation and bounds have no effect on this generated image, so you should only have to change the contentsRect and frame fields of the image based on the new bounds.  You basically just need to use this code in an event listener:
https://github.com/google/shaka-player-embedded/blob/9fc13728fa004d5928fbe72955ca3ebbae75ed8c/shaka/src/public/ShakaPlayerView.mm#L172-L188

I know it's overhead, but it was the simplest solution.
I tried to think your advice but:
shaka::ShakaRect<uint32_t> image_bounds = {0, 0, CGImageGetWidth(image),
                                            CGImageGetHeight(image)};

returns wrong image size (image size from previous orientation) after resize and I have to call
CGImageRef image = _player.videoRenderer->Render(nullptr, &aspect_ratio)

to get correct new ones.
So how can I do it, without calling Render() method. I understand I can't call render method because frame == prev_frame_ returns nill as result.

The base image is always the same, it is based on the image data without any resizing.  We use contentsRect and bounds to resize the image to fit based on the orientation.  We want the original image size for image_bounds.

Sorry, you're right. Image dimensions are really same, I was wrong.
I create new commit, where I reverse all previous patches and has working solution as you advice.
Two notes:

I had to make image or image_bounds as class property to access it in orientationChanged method.. I deciced to image property. I didn't find out way how to access original image from _imageLayer.contents. Maybe (I hope) there is better way?

I has to change self.bounds to self.superview.bounds because self.bounds returned wrong values.
I made same change also in renderLoop. Then I could remove my workaround in swift project:
self.playerView.frame = self.view.bounds; which was used to fix problem previously (calling on orientation change), but worked only when player was playing.

I updated all points, except superview workaround. I will try to figure out why view.bounds are wrong.

Ok, I found better solution:
In my own ViewController, in which I inserting ShakaPlayerView, I moved
self.playerView.frame = self.view.bounds;

from viewWillAppear
inside antoher override function:
override func viewDidLayoutSubviews() {
  if (self.playerView != nil)
  self.playerView.frame = self.view.bounds
}


larq-compute-engine installing issue

Hi, I used larq to create a BNN from a full precision network to make some comparisons.
I would like to install your lce library on a raspberry pi 4 model B.
However, by running the command "pip install larq-compute-engine, as suggested in the end to end guide, I get this error:
ERROR: Could not find a version that satisfies the requirement larq-compute-engine (from version: none)
Do you, please, have any idea about how i can fix this?
Regards.

Hi, nice to see you're interested in BNNs and the Larq ecosystem! Which python version are you trying to install LCE for? We support python versions 3.6 and above. It seems your python package manager can't find any LCE versions, which indicates that you're probably on an unsupported python version, or your pip is a bit outdated. Could you tell me the output of the pip --version command?
If pip is indeed the problem, you should be able to fix it by running python -m pip install --upgrade pip (assuming that python refers to python version >= 3.6 on your system). Let me know if that works for you!

Hello, thanks for the quick response.
I'm using python 3.7.3 and pip 20.2.4

Hello, thanks for the quick response.
I'm using python 3.7.3 and pip 20.2.4

Okay, in theory that should be absolutely fine, but now that I've re-read your original message I see you're trying to install LCE on the Raspberry Pi itself. The LCE converter currently only supports x86 architectures (you can see the list of wheels here), but the Raspberry Pi uses ARMv8.
We therefore recommend that you install the converter on another device instead (such as your laptop), and convert the model from there. The actual inference engine can of course still be run on (and is optimized for) ARMv8. Hope that helps!

Ok so i should create and train my model, convert it with LCE on my laptop. And then load the already converted model on the raspberry and run the inference with normal larq library?

Ok so i should create and train my model, convert it with LCE on my laptop. And then load the already converted model on the raspberry and run the inference with normal larq library?

Yes, correct! But you wouldn't use Larq for inference, as it's only a training library. That's where the Larq Compute Engine comes in: to benchmark your converted model on your Raspberry Pi, you can follow the benchmarking part of the end-to-end guide, which simply downloads a pre-built benchmarking binary that you can pass your converted model to.
If you want to actually use your model for inference in an application of some kind (i.e. if you care about the inputs and the predictions), check out our C++ or Android inference guides.

Exactly, my purpose is to test the performance of my BNN on the raspberry pi 4 device. And this involves running the inference directly on the raspberry. I will try to follow the guide for C++ to use my network on the raspberry.

Okay, good luck! If you experience any problems there, feel free to open another issue :+1:

Thank you very much for your help

Hi, all.
Is there any way to use python API like interpreter.predict(x) in raspberry PI?
basically tensorflow lite supports python API in here so i want to know whether i can use python function for larq inferencing

Hi, all
is there any way to use python API like interpreter.predict(x) ?
I want to know whether i can use python function for inferencing in larq like tensorflow lite here

is there any way to use python API like interpreter.predict(x) ?
I want to know whether i can use python function for inferencing in larq like tensorflow lite here

There is a Python interpreter, but it is not optimized for speed so it can not be used for benchmarking latency: https://docs.larq.dev/compute-engine/api/python/#interpreter

Hello everyone.... I am trying to do the inference using larq compute engine and i want to use the c++ script as shown in lce_minimal.cc but i am finding my self in trouble due to nan error....can someone provide a script where we have code for input and output as well? I would highly appreciate if you can provide me with the example for imagenet dataset. @Tombana @jneeven


JS_ToBigInt64 handles overflow incorrectly

JS_ToBigInt64 handles number larger than 2^64 incorrectly.
Divive into the function, it calls JS_ToBigInt64Free then JS_ToBigInt64Free:
static int JS_ToBigInt64Free(JSContext *ctx, int64_t *pres, JSValue val)
{
    bf_t a_s, *a;

    a = JS_ToBigIntFree(ctx, &a_s, val);
    if (!a) {
        *pres = 0;
        return -1;
    }
    bf_get_int64(pres, a, BF_GET_INT_MOD);
    JS_FreeBigInt(ctx, a, &a_s);
    return 0;
}

For numbers larger than 2^64, a has a valid value so it goes to bf_get_int64 where the problem occurs.
/* The rounding mode is always BF_RNDZ. Return BF_ST_INVALID_OP if there
   is an overflow and 0 otherwise. */
int bf_get_int64(int64_t *pres, const bf_t *a, int flags)
{
    uint64_t v;
    int ret;
    if (a->expn >= BF_EXP_INF) {
        ret = BF_ST_INVALID_OP;
        if (flags & BF_GET_INT_MOD) {
            v = 0;
        } else if (a->expn == BF_EXP_INF) {
            v = (uint64_t)INT64_MAX + a->sign;
        } else {
            v = INT64_MAX;
        }
    } else if (a->expn <= 0) {
        v = 0;
        ret = 0;
    } else if (a->expn <= 63) {
#if LIMB_BITS == 32
        if (a->expn <= 32)
            v = a->tab[a->len - 1] >> (LIMB_BITS - a->expn);
        else
            v = (((uint64_t)a->tab[a->len - 1] << 32) |
                 get_limbz(a, a->len - 2)) >> (64 - a->expn);
#else
        v = a->tab[a->len - 1] >> (LIMB_BITS - a->expn);
#endif
        if (a->sign)
            v = -v;
        ret = 0;
    } else if (!(flags & BF_GET_INT_MOD)) {
        ret = BF_ST_INVALID_OP;
        if (a->sign) {
            uint64_t v1;
            v = (uint64_t)INT64_MAX + 1;
            if (a->expn == 64) {
                v1 = a->tab[a->len - 1];
#if LIMB_BITS == 32
                v1 = (v1 << 32) | get_limbz(a, a->len - 2);
#endif
                if (v1 == v)
                    ret = 0;
            }
        } else {
            v = INT64_MAX;
        }
    } else {
        slimb_t bit_pos = a->len * LIMB_BITS - a->expn;
        v = get_bits(a->tab, a->len, bit_pos);
#if LIMB_BITS == 32
        v |= (uint64_t)get_bits(a->tab, a->len, bit_pos + 32) << 32;
#endif
        if (a->sign)
            v = -v;
        ret = 0;
    }
    *pres = v;
    return ret;
}

For example, the number 2^64 + 2, its a->expn is 64. While using BF_GET_INT_MOD flag, it goes to the final else block.
As BF_GET_INT_MOD implies that modulo 2^n instead of saturation. NaN and infinity return 0, finally we got the v as 2 which is from (2^64 + 2) % 2^64, and ret=0 which means returns a right result.
Then everything mess up.
That is to say that we cannot judge the return value from JS_ToBigInt64Free (as well as JS_ToBigInt64) whether it is the actual result or the actual result mods 2^64
It seems to be a problem that has been around for a long time, and a solution has already been suggested: https://github.com/theduke/quickjs-rs/blob/master/libquickjs-sys/embed/patches/js-tobigint64-overflow.patch
I've tried to apply the patch and the problem goes fixed.
So I think maybe it should be adopted?

Interesting! I guess there is no test that checks for this.
A good first step would be to add one that fails.

Here is a Rust example: https://github.com/Icemic/quickjspp-rs/blob/221fff9e177fa5b490bf349aa74f3550eb7f0397/tests/runtime.rs#L641

create a quickjs context
just evals<PHONE_NUMBER>854775807n + 11n, and get the result ret of type JSValue
try to get value from ret via JS_ToBigInt64 (will get 10 now)

Hello @Icemic, nice to see QuickJS use from the rust world...
I don't understand your point: JS_ToBigInt64Free is defined as computing BigInt value modulo 64-bit modulo.
bf_get_int64 never fails with BF_GET_INT_MOD
It follows the ECMA spec: 7.1.15 ToBigInt64 ( argument ).
You seem to expect some other behavior, what is your use case ?

Thanks for the reply! I hadn't noticed the ECMA spec aspect before. Given that this function adheres to an established specification, it should indeed be fine.
I'm maintaining a QuickJS to Rust binding, which is forked from quickjs-rs. There's a piece of code designed to extract the actual value from a JSValue tagged as bigint, and then convert it into a Rust type: https://github.com/Icemic/quickjspp-rs/blob/221fff9e177fa5b490bf349aa74f3550eb7f0397/src/value/value.rs#L276-L306
Its approach begins with an attempt to convert using JS_ToBigInt64. If that fails, it then resorts to a string conversion. The intention behind using JS_ToBigInt64 was presumably to serve as a performance optimization. However, it seems both the original author and I have misunderstood the true definition of this function.
So, my question now shifts to: In similar binding libraries or FFI scenarios, what is the recommended way to retrieve and convert values from a TAG_BIGINIT type JSValue to a native type?

So, my question now shifts to: In similar binding libraries or FFI scenarios, what is the recommended way to retrieve and convert values from a JS_TAG_BIGINT type JSValue to a native type?

I would recommend adding these APIs:

int JS_IsInt64(JSContext *ctx, JSValue obj): return 1 iff the value is an integer and in the range INT64_MIN to INT64_MAX.
int JS_IsUint64(JSContext *ctx, JSValue obj): return 1 iff the value fits is an integer and in the range 0 to UINT64_MAX.

With these your can determine if JS_ToBigInt64 will give you the expected result.  If the number is not in the proper range, you can use the string conversion, but I recommend using a base 16 conversion that is much faster than a base 10 conversion.
For efficient conversions to and from another language native representation, I would add these:

int JS_GetValueSign(JSContext *ctx, JSValue obj): returns 0 for positive or zero, 1 for negative, -1 for NaN and values that are not numerical.
JSValue JS_NegateFree(JSContext *ctx, JSValue obj): return the opposite of a numerical value (value is freed).
int JS_GetBigIntBitLength(JSContext *ctx, JSValue obj): returns the number of significant bits, eg: 0 for 0n, 1 for 1n and -1n, 64 for UINT64_MAX, etc.
int JS_GetBigIntBits(JSContext *ctx, JSValue obj, void *dest, size_t size, size_t count, size_t offset): get the bits by groups of 1, 2, 4 or 8 bytes (a word) into an array dest of such words (uint8_t, uint16_t, uint32_t, uint64_t respectively), from a given offset in the same unit and from least significant to most significant word.  Negative values would be retrieved as 2's complement.
JSValue JS_NewBigIntFromBits(JSContext *ctx, const void *src, size_t size, size_t count, int sign): create a JS_TAG_BIGINT value from the array word provided with sign sign. Negative values should be represented in 2's complement.

The current implementation uses a general floating point representation that makes these conversion APIs non trivial but still much faster than a string conversion. We are likely to change the internal representation in the near future to make this API straightforward.


Fixed png link redirection

I've fixed this issue
But I guess this isn't enough to properly parse text like this:
[Import Nov 27, 2021]((%E1%84%80%E1%85%AE)%20%E1%84%87%E1%85%B3%E1%86%AF%E1%84%85%E1%85%A9%E1%84%80%E1%85%B3%208393554aa3284ed0b41d8b9abbad2d7a/Import%20Nov%2027,%202021%20261953cd6063460fa4b415f57273c981.md)

It's underlying defection of regex parser which is already implemented.


put stream fails with larger files >64kB

this is hopefully not an issue,  just a "understanding" problem on my side.
BTW:  I first asked at https://stackoverflow.com/questions/48776919/node-js-request-module-put-stream-fails-with-larger-files-64kb  but probably its better here.
I use a small node.js express server to test/simulate a file upload download server with HTTP put/get:
app.use(express.static(SERVER_DIR))
app.put('/put/:filename', (req, res) => {
  req.pipe(fs.createWriteStream( SERVER_DIR + req.params.filename));
  res.send('OK')
})

the above server works fine if tested with (shell) curl put/get:
curl "http://localhost:8765/put/" -T 100MB.file
curl "http://localhost:8765/100MB.file" -o 100MB.get

(tested up to 1GB - check if uploaded /downloaded files are equal with cmp)
My problem ist to upload (put) the files with a node.js client using the request module:
fs.createReadStream(filename).pipe(request.put(SERVER_URL+'put/'+filename))

this upload works fine with small files > 64kB but it fails for larger. uploading file with 65400Byte is working / file with 65450Byte is not working - it stops after 65410Byte / EOF error.
How can I change this limit ?
BTW: download (get) with the node.js request client app:
request(SERVER_URL+filename).pipe(fs.createWriteStream(filename))
works with large 1GB files.

Same with:
request('http://google.com/doodle.png').pipe(fs.createWriteStream('doodle.png'))
If file size is >64.39KB I get only part of the file up to 64.39KB only.


custom:swipe-card compatibility

Describe the bug
It seems there is a problem with custom:swipe-card compatibility. All the items defined with such card, are still visible after bubble card popups.
Swipe card GitHub (https://github.com/bramkragten/swipe-card?tab=readme-ov-file)
To Reproduce
Steps to reproduce the behavior:

Create a custom:swipe-card card on a dashboard
Create a custom:bubble-card popup on the same dashboard
Click on the bubble card to make it show the popup overlay (which should be so far clear, without any cards)
Swipe card is visible on the overlay of the bubble card

Expected behavior
I would expect anything that is "under" the overlay of the bubble card to NOT be visible on the bubble card popup
Screenshots

Default dashboard. Some of the cards are defined with custom:swipe-card, some are not.

After clicking on Heating bubble card, I should not see any cards from the dashboard. They should be blurred, but they are not.

Desktop (please complete the following information):

OS: Win11
Chrome

I realise this is probably not a bug of the Bubble-card, but it seems to prevent a usage of these two great cards together. Maybe it could still be fixed?
Thank you! 🍻

This issue is now fixed in the last beta.

Great job! Just tried it out and confirming this is working.


Ckeck characters words typos cypress test

checks for Karaktere ,Fjale , Shenjimet
It also presses SHIFT to make sure it doesnt count as a character as commented on the pull request #158
I would like to not close this PR just yet and add the test for issue #159

Utilize the data-test attribute instead of id for #characterCountSpan.

Utilize the data-test attribute instead of id for #characterCountSpan.

Can you check it please I cant seem to make it work while testing it it doesn't grab the ID


Implement both circuit and assignments separation into small parts

For some tasks, including parallelization, we need to add many components to circuit and generate assignments for them independently or even at the same time. Since it's not possible with only one circuit and assignments state structs instance (because of memory management), we need to make it possible to construct many small circuits and then assemble these parts together.

Alternatively, we can make the circuit/assignment structures thread-safe. This seems to me to require less effort than a redesign involving subcomponents.
Moving to boost::containers::stable_vector would prevent resizes from invalidating assignment references in a different thread; and we can put mutexes only on operations which would cause size increase.
Other than that, we don't really need to worry about write safety -- writing two different elements to a single cell is going to cause an error regardless of the write being thread-safe or not.


yarn run build 生成的chunk-vendors.js 很大 首页加载速度很慢

Question （问题描述）
打包后的 chunk-vendors.js 大概有2.3MB 左右 首页首次进入非常慢
Describe the solution you'd like （你期待的是什么？）
不知有没有办法将把 此文件 进行分割
Additional context（附加信息）
Add any other context or screenshots about the feature request here.

gzip 之后只有 700~800KB，10M 贷款服务器 按理说 1秒之内就加载完了。如有必要拆分，请自行百度谷歌 或部分库采用 CDN 引入。

Same problem,
I have a bundle within size 2.3 MB, how to can handle this issue? could someone explain or refer to solution of this problem, it took me hours , please

Same problem,
I have a bundle within size 2.3 MB, how to can handle this issue? could someone explain or refer to solution of this problem, it took me hours , please


In dynamic routing scenario, one route failling due to wrong parameters or something else will make all routes invalid

Hi,
Just want to make a suggestion.
In dynamic routing scenario, we need to change the route configuration dynamically, and it is hard to validate all parameters of  multiple routes. But the routes of Spring Cloud Gateway will be crashed when even just one route failed due to wrong parameters or something like that. The reason is that the Flux<Route> is lack of error handling, and the stream will be terminated with error. It may be suitable for static routing in propertiy files or coding format, but it is intolerable for dynamic routing.
In class RouteDefinitionRouteLocator, method Flux<Route> getRoutes(), there is a TODO  case, indicate that error handling is needed at here. I think .onErrorContinue() is suitable for dynamic routing, and showing some error info then contuning other than making all routes invalid is the better choice in this case. In order to satisfy both static and dynamic routing, it can be designed configurable.

I had the same problem

PRs welcome

This will be opt-in, see https://github.com/spring-cloud/spring-cloud-gateway/pull/1496#issuecomment-592310381


SDK release

Finally they did it, great news! link

The SDK is for writing actions to be used in their control software, so isn't really relevant to this library. Especially as they do not support linux.


UITableViewCell does not update if the object property changed is contained in the sort descriptor

If the updated property of a managed object is contained in the sort descriptor, the change type is considered "moved" and not "updated". Because of this, the table view does not reload the updated rows.
According to Apple Docs:
A move is reported when the changed attribute on the object is one of the sort descriptors used in the fetch request. An update of the object is assumed in this case, but no separate update message is sent to the delegate.
NSFetchedResultsControllerDelegate Protocol Reference
Example
If I have a table view with names that's sorted by the name attribute, if I update the name, the table view does not reload the row, it just moves the row to the appropriate position.

Good catch! Want to submit a pull request so you get some credit?

Sure... Thanks!

Fixed in MDMCoreData 1.5.1.

Updated my project and confirmed! Thanks!


Shop by category dissapears on hover

Bug report
Issue Description
When you hover on the section the categories are being shown, but a lot of times when i want to go to a category the menu goes away
When i hover of category section a couple times the category menu stays and i can click on the categories
https://youtu.be/57nzPjeGUu4
Preconditions
Please provide as detailed information about your environment as possible.
1. master branch

Still happens in latest release 1.3.2


.net core 3.0 upload big file faild

Issue Title
.net core 3.0 upload big file faild
General
1 IDE & windows

2 .net core version (my project version)

3 controller & api

4 startup config

5 upload file

5 error stack (error file size is >16MB)

6.success condition (file size is 1KB )

small file is success accepted

is this config not effect or i config wrong?
when i use .net core 2.2 ,that file[5] was success uploaded...
here is my code on github
https://github.com/1369521908/Obsidian.git

This issue was moved to aspnet/AspNetCore#15036


Image and icon elements contain accessible alt attributes

#User Story
AS A marketing agency
I WANT a codebase that follows accessibility standards
SO THAT our own site is optimized for search engines
GIVEN A DEPLOYED WEBSITE
WHEN I navigate the website
Then image and icon elements contain useful alt attributes

All images in the html contain useful alt tags
The background image in the style file contains a useful title


Set up CI (Travis) for this repo

FIXED yesterday by @subhankarb - see https://travis-ci.org/frictionlessdata/dpr-api


Feature request: Add vnc web console

One suggestion is to use noVNC (https://novnc.com)

Hey,
I currently look in to this.
Currently I have found a quick and dirty way. Give me a few days :D
Maxi

Hello guys!
What do you think about serial console instead of vnc? Like virsh console command but in browser. I think it will be more convenient, it even allows copy/paste

Hello guys!
What do you think about serial console instead of vnc? Like virsh console command but in browser. I think it will be more convenient, it even allows copy/paste

Good idea!

Works like a charm, I hope :) https://github.com/subuk/vmango/commit/e08d470dedb9b9cb96ae117eda833db4a496b925
https://streamja.com/OJdo

With https://xtermjs.org it even better

I think this is the final version https://streamja.com/embed/LLEA


handle scenario when the file is created but doesn't have content

What this PR does / why we need it:
cherry pick https://github.com/kubernetes-sigs/aws-iam-authenticator/pull/778
Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #

/lgtm


Set upper bound for Sphinx.

Setting the upper bound for Sphinx for now to unblock PRs since Sphinx 3.1.0 which was published 5 hours ago causes lint-python failure.

I'd merge this now. Fixing the failures is tracked at #1570.
cc @HyukjinKwon


Virtual filesystem view on Vineyard Objects

Describe your problem
To match with the semantic of other storage systems, a virtual view that provides the "filesystem" semantic should be possible for vineyard.
e.g., for dataframe, we could provide a view of orc, parquet, or something like (e.g., arrow) on top of vineyard's Tensor, DataFrame, Table, etc., to provides better compatibility with other compute engines.
Note that the virtual fs view is implemented in the client side, and the read_parquet/read_orc/read_csv operation on vineyard objects is ZERO-COPY. And the most important vision is, the end users doens't need to aware that the data is placed on Vineyard.
Based on that, we could leverage libfuse to provides a local fs view for compute engines then vineyard could seamlessly integrated with existing programs, tasks, and workflows.
Additional context
And vineyard could work seamlessly with engines like fluid, kubeflow, argo, etc.
Such feature requires heavy investment on the encoding of those format, but it would help for big data engine integration ( #372 ).

The might be still a copy, but it does make sense, as the data may generated by another compute systems, and downstream engine (e.g., spark) can consume the data with only a memory copy (no network transfer, etc.), and, the downstream engine could be more efficient if they could modify the source code a bit.
BUT the modification is not required.


Error ITMS-90426 when uploading to Apple TestFlight with Godot 4.2/4.2.1

Tested versions
Reproducible: 4.2 and 4.2.1
Not reproducible in: 3.5.2
System information
macOS Ventura 13.5.11
Issue description
Any project made with Godot 4.2/4.2.1 that uses C# can't be uploaded to iOS AppStore/TestFlight, after the upload you'll get an email saying:
"ITMS-90426: Invalid Swift Support - The SwiftSupport folder is missing. Rebuild your app using the current public (GM) version of Xcode and resubmit it."
Btw I have the latest XCode version (15.1)
I have noticed that on XCode, under Build Phases->Embed Framework, there's a [project-name]_aot.xcframework, if removed, the error is gone but the app crashes at launch. This could be useful to track down the problem. It seems to be related to how the _aot.xcframework is being built.
Steps to reproduce

Open the attached Godot project
Export to iOS
Archive with XCode and upload it to TestFlight/AppStore
After some minutes you'll receive an ITMS-90426 error on your email.

Keep in mind that building locally will result in a fully functional app, this error only shows up when Apple process the app bundle.
Minimal reproduction project (MRP)
ITMS-90426.zip

I have found this info in the Apple documentation:
"Avoid using dynamic library files (.dylib files) for dynamic linking. [...]Dynamic linking on iOS, watchOS, and tvOS requires the XCFramework to contain .framework bundles."
https://developer.apple.com/documentation/xcode/creating-a-multi-platform-binary-framework-bundle
Godot indeed creates an .xcframweork that includes a .dylib:

I'm also experiencing this exact issue when uploading my exported app. I'm happy to help validate the changes made in the PR linked above if there is a godot build I can use.


Upgrade the code to Swift 5.

Applied Xcode recommended project settings to remove all compiler warnings.

Thanks!


fix boilerplate url

fix boilerplate url, add missing '-' in the url

@LuoLun  Thank you for proposing this fix. However, we will not merge partial fixes as per stated in the guideline. To find out what are the upstream changes to be synced, do this on the master branch:
./scripts/lsync.sh content/zh/docs/tasks/run-application/horizontal-pod-autoscale.md

Also, please sign the CLA in order for the PR to be reviewed.

/check-cla

Pls feel free to reopen and continue this work after your CLA is signed, @LuoLun
/close


Consider moving cmr to NASA Github Org

Hi Folks,
@jasonduley has been very helpful in aiding me in transitioning more of my code over to the NASA Github Org, namely with Podaacpy and Sitepod projects. I am going to send a couple of pull requests to tidy up some of the code in pyCMR. It got me thinking that I would encourage you to take the same route and get in touch with @jasonduley such that we can begin to build more community around the pyCMR client code.
PR's coming shortly on different tickets.
Lewis

@lewismc: Thanks, @jasonduley is familiar with pyCMR.  We went through NASA Open Source Software release process via MSFC for pyCMR.  Having said that, always open to ways to build community around pyCMR.

IMHO, I think we could build more community from other DAAC's if pyCMR were moved to the NASA Github Org. There are also some links which are GHRC DAAC-specific... I think that is another ticket. I'll log it.

@manilmaskey @amarouane any desire to transition this library to NASA Github? I've started using it over at PO.DAAC and I would like to begin building more community. I would also like to shepherd releases of the toolkit through Pypi and also condo through condo-forge.

@lewismc go ahead.  @amarouane can provide help as needed.  Currently, it is searchable through code.nasa.gov.

@lewismc Yes I don't have any objections at all.

OK folks, I've reached out to @jasonduley hopefully we can get the ball rolling with this one shortly.

https://github.com/nasa/cmr boom


Identify form, route of administration and strength of presentations

In order to be able to look at volume in terms of ADQs and DDDs (as described in #3), it is necessary to obtain information about the form, route of administration, and strength of presentations.
To the best of my knowledge, this information is not currently available publicly except expressed in the given BNF Name for the presentation.
My approach so far to extracting this information has been to build a parsing expression grammar (PEG) and use Arpeggio to parse the names with the PEG and apply semantic analysis. This is not very well put together at the moment, but it can already extract relevant information for many items. It can deal with concentrations. It currently copes poorly with combination products and products specifying a rate of release over time; the ADQs are defined for these in terms of tablets and patches anyway, so this may not be so important.
The examples below show names and what information I can currently extract from them:

BNFName
Product
Route of Administration
Mass (of Chemical)/g
Vol/l
Ds

Warfarin Sod_Tab 5mg
Warfarin Sod
Oral
0.005
N/A
1

Quetiapine_Tab 200mg
Quetiapine
Oral
0.2
N/A
1

Sitagliptin_Tab 50mg
Sitagliptin
Oral
0.05
N/A
1

Exenatide_Inj 250mcg/ml 2.4ml Pfs
Exenatide
Parenteral
0.0006
0.0024
1

Levothyrox Sod_Oral Soln 25mcg/5ml S/F
Levothyrox Sod
Oral
0.000005
0.001
1

Lofepramine HCl_Tab 70mg
Lofepramine HCl
Oral
0.07
N/A
1

Co-Codamol_Tab 8mg/500mg
Co-Codamol
Oral
N/A
N/A
1

Salbutamol_Pdr For Inh 200mcg (60 D)
Salbutamol
Inhaled
0.012
N/A
60

Buprenorphine_Patch 10mcg/hr
Buprenorphine
Transdermal
N/A
N/A
1

Clobazam_Oral Susp 5mg/5ml S/F
Clobazam
Oral
0.001
0.001
1

Metformin HCl/Pioglitazone_Tab850mg/15mg
Metformin HCl/Pioglitazone
Oral
N/A
N/A
1

Hydroxocobalamin_Inj 1mg/ml 1ml Amp
Hydroxocobalamin
Parenteral
0.001
0.001
1

Tryptophan_Cap 500mg
Tryptophan
Oral
0.5
N/A
1

Zolmitriptan_Orodisper Tab 2.5mg S/F
Zolmitriptan
Oral
0.0025
N/A
1

Co-Danthramer_Susp 25mg/200mg/5ml S/f
Co-Danthramer
Oral
N/A
0.001
1

Bisacodyl_Rectal Soln 2.74mg/ml
Bisacodyl
Rectal
0.00274
0.001
1

The Mass, Vol and Ds figures are intended to be the amounts for a single 'quantity' of the presentation. As the HSCIC's FAQ on the prescribing data says:

Where the formulation is tablet, capsule, ampoule, vial etc the quantity will be the number of tablets, capsules , ampoules, vials
Where the formulation is a liquid the quantity will be the number of MLS [millilitres]
Where the formulation is a solid form (eg. Cream, gel, ointment) the quantity will be the number of grammes

(The Prescription Cost Analyses published by the NHSBSA explicitly defines the unit of 'quantity' for different presentations.)
'Ds' here refers to the number of discrete administrations where relevant - single inhalers, for example, often provide several hundred puffs (and in fact some ADQs are defined in puffs).

This is great!
It could also be useful to indicate whether the presentation is generic or branded, which I believe can be extracted from the BNF code.

One of the interesting things you could do is use this to relate the prescriptions in the data to the objects found in the DM+D. This would give you a lot more information about the actual product/prescription. See this online browser version but the underlying files are available from trud.

I wonder if the DM+D supplementary files (from TRUD) would enable direct lookup from the BNF Code.

This is where it gets messy. You can use the BNF code to get to the substance, but the DM+D doesn't have the individual BNF Codes per preparation.

Just checked. The DM+D only has mapping to the 8 digit BNF code. Honestly it looks like the key for much of this would be to have more information about the 15 digit variant from the nhsbsa, and if they have a reference.

Small followup: Found another reference, login as guest here > Data > Drug Data > BNF Code Information
May bot be enough information to link it to the DM+D but should make your original effort easier.

Hello @stupidpupil! Sorry this has taken 2.6 years to take a new look at this - unfortunately I started on OpenPrescribing after you wrote this note.  By the time I'd noticed this issue, I'd already duplicated much of your work, as a result of investigating #905 :cry:
You'll see we've successfully mapped BNF codes to dm+d data since you wrote this, but the latter still doesn't include enough metadata to define "quantity" accurately (e.g. the 60D phenomenon, and mass-per-volume for suspensions), so we end up having to parse the BNF name in any case.
So I got that far, and then I found this issue! Do you still have the code for the PEG?
(We've also found it's possible to back-calculate ADQs from the PPI dataset from NHSBSA, which may be far preferable)

https://gist.github.com/stupidpupil/dd687f168db39eccc5f6040a6da25714
It's hideous!


Main page Find your space button not working

https://github.com/The-MindSpace/MindSpace-Web/assets/133562522/95d24db4-e19a-45fa-acb8-716de6060217
The find your space button is not properly visible
I am confident that I can make it look like the image above and improve the css styling of this page by making it more interesting with hover effects and transitions. @TeeWrath kindly assign me this issue under Swoc'24

@TeeWrath Please assign this issue to me under Swoc'24


Benchmark

See https://github.com/aspnet/benchmarks

I did add AspNetCore.Lambda to https://github.com/TheAngryByrd/dotnet-web-benchmarks/ .  AspNetCore.Lambda was right behind Kestrel Plain!
Disclaimer: These are super simple benchmarks with simple routing and plain text responses.
CPU - Intel(R) Core(TM) i7-4870HQ CPU @ 2.50GHz
Operating System - Unix <IP_ADDRESS> (OSX)
Processor Count - 8
Dotnet Version - 1.0.0-preview2-1-003177

Kestrel Plain at 2,260,301  requests over 30 seconds
AspNetCore.Lambda on Kestrel  2,241,695  requests over 30 seconds
MVC on Kestrel 2,027,391 requests over 30 seconds

Nice!

Nice, thanks for running those!

I had a look at the repo and looks like you have a nice collection of different frameworks there. Did you document the results somewhere publicly so I can reference to it?

Sure ill make a listing soon.

https://github.com/TheAngryByrd/dotnet-web-benchmarks/blob/master/README.md#results

These results look really great. I am glad that Giraffe (formerly known as ASP.NET Core Lambda :)) ranks so high. I would have been surprised if it wouldn't to be honest though, because the additional layer on top of plain kestrel is very thin and especially the app you tested (plain text on a GET request) is doing very little more than plain kestrel...
I wasn't sure how the conversion between C# Tasks and F# Async workflows will be, but from those tests looks like the impact is close to nothing which is great.
Many thanks for running those, I will edit the README in a bit to link to your project and the results.

I have linked it here. Let me know if you would like to change or add something to it.


Enforcing a specific version of Rust

Hi, I'm not familiar with Rust, but I'm using a Python module that has Rust dependencies (Huggingface's tokenizers). I'm trying to compile the code in a PowerPC cluster without access to the Internet and without admin permissions, but with a built-in Rust install, which doesn't work in my case since PyO3 requires Rust Nightly.
So, I have transferred to the cluster the right Rust version (nightly, PowerPC), but when I execute the corresponding Cargo, I still get the same error:
error: failed to run custom build command for "pyo3 v0.8.4"
Error: pyo3 requires a nightly or dev version of Rust.
My best guess is that since I haven't installed this new version of Rust (because I do not have permissions to do so, so I'm just calling the corresponding binaries), at some moment Cargo just calls "rustc", which in this case is the built-in Rust compiler that doesn't work in my case.
Is there any way to enforce a specific rustc path (instead of calling rustc, call /path/to/my/rustc)? So far, I haven't found any workaround.
For reference, this was the original issue in the tokenizers repository: https://github.com/huggingface/tokenizers/issues/176#issuecomment-594091846
Many thanks in advance.

The typical way of managing toolchain versions is with rustup.  This allows you to set overrides (globally, per-directory, etc.).  In this case I would run rustup override set nightly in the directory where you want to use nightly.
If you don't have rustup or can't use it, setting the RUSTC environment variable is the typical way to specify which compiler to use.
In the future, questions like these are better asked on one of the user forums.

@ehuss Many thanks for your answer and please excuse any inconvenience.


throwing error while using nodejs packages

TypeScript Version:
1.8.10
import {params} from "scrypt";

Expected  behavior:
since scrypt is already in node_modules compile with no error
Actual behavior:
throws error saying cannot find module seems like i cannot use typescript for nodejs development

By default, TypeScript does not magically understand random NodeJS packages.  It needs information to describe it so you can consume it.  scrypt does not provide that information in its package.  Either you have to provide it yourself, or you have to obtain it from somewhere (potentially using a tool like typings/typings or on DefinitelyTyped).

@kitsonk i tried to export an interface from a file called scrypt.d.ts and referenced it but still the same problem its behaving like the interface is not referenced
the scrypt.d.ts file content is as follows
//scrypt.d.ts
	export interface paramsObject extends Object{
		n: number;
		r: number;
		p: number;
	}
export interface scrypt{
	params(maxtime: number, maxmem?: number, maxmemfrac?: number):Promise<paramsObject>;
	paramsSync(maxtime: number, maxmem?: number, maxmemfrac?: number):paramsObject;
	kdf(key:string,paramsObject):Promise<Buffer>;
	kdf(key:Buffer,paramsObject):Promise<Buffer>;
	kdfSync(key:Buffer,paramsObject):Buffer;
	kdfSync(key:Buffer,paramsObject):Buffer;
	verifyKdf(encryptedPass:Buffer,password:string):Promise<Boolean>;
	verifyKdfSync(encryptedPass:Buffer,password:string):Boolean;
}

and in my actual file that i use scrypt i reference it by using tripple slash
/// <reference path="./scrypt/scrypt.d.ts" />
but no use it still says module is not found

This would work if you imported it using relative path, e.g. import d from "./scrypt", which is not the case.  you want to put the module name, "scrypt" in the global. so:
//scrypt.d.ts
declare module "scrypt" {
	export interface paramsObject extends Object {
		n: number;
		r: number;
		p: number;
	}
	export function params(maxtime: number, maxmem?: number, maxmemfrac?: number): Promise<paramsObject>;
	export function paramsSync(maxtime: number, maxmem?: number, maxmemfrac?: number): paramsObject;
	export function kdf(key: string, paramsObject): Promise<Buffer>;
	export function kdf(key: Buffer, paramsObject): Promise<Buffer>;
	export function kdfSync(key: Buffer, paramsObject): Buffer;
	export function kdfSync(key: Buffer, paramsObject): Buffer;
	export function verifyKdf(encryptedPass: Buffer, password: string): Promise<Boolean>;
	export function verifyKdfSync(encryptedPass: Buffer, password: string): Boolean;
}

also consider submitting a PR to definitely typed for this library: https://github.com/DefinitelyTyped/DefinitelyTyped/

@mhegazy thank you for help it clarifies everything :)


clipit: hicolor_icon_theme dependency

Motivation for this change
clipit conflicts with other packages that use hicolor_icon_theme
Things done

[x] Tested using sandboxing
(nix.useSandbox on NixOS,
or option build-use-sandbox in nix.conf
on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] OS X
[ ] Linux

[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[x] Fits CONTRIBUTING.md.

I have tried this change in the hope it would solve the 'clipit-trayicon' issue (I can only an icon with a red cross in the tray) but no luck.
@kamilchm do you have any advice, tip about this ?

The problem is how clipit loads its icon with gtk_status_icon_new_from_icon_name.
You could try to make it works somehow (dunno how) or patch it to use https://github.com/CristianHenzel/ClipIt/blob/a8de309ee006003ab3545d9abbe8c017a7400e62/data/clipit-trayicon.svg from nix store.