[Impeller] Stack + Positioned + Transform with FilterQuality clips the widget.

Is there an existing issue for this?

[X] I have searched the existing issues
[X] I have read the guide to filing a bug

Steps to reproduce
The following hierarchy clips the child when rotated using the Impeller renderer:
Stack --> Positioned --> Transform.rotate with FilterQuality --> Solid Color Container
Stack(
  fit: StackFit.expand,
  children: [
    Positioned.fromRect(
      rect: Rect.fromLTWH(0.0, 0.0, 500.0, 500.0),
      child: Transform.rotate(
        angle: 0.4,
        filterQuality: FilterQuality.low,
        child: Container(color: Colors.green),
      ),
    ),

Expected results
The solid color container should not be clipped.
Actual results
The solid color container is clipped.
This affects only the Impeller renderer and not the Skia one.
Removing the FilterQuality solves the issue.
Code sample
Code sample
import 'package:flutter/material.dart';
import 'dart:math' as math;

void main() {
  runApp(const MyApp());
}

class MyApp extends StatelessWidget {
  const MyApp({super.key});

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        colorScheme: ColorScheme.fromSeed(seedColor: Colors.deepPurple),
        useMaterial3: true,
      ),
      home: const MyHomePage(title: 'Flutter Demo Home Page'),
    );
  }
}

class MyHomePage extends StatefulWidget {
  const MyHomePage({super.key, required this.title});

  final String title;

  @override
  State<MyHomePage> createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {
  double _rotation = 0.0;

  static const _width = 500.0;
  static const _height = 500.0;

  static const _sliderBottom = 48.0;

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        backgroundColor: Theme.of(context).colorScheme.inversePrimary,
        title: Text(widget.title),
      ),
      body: LayoutBuilder(
        builder: (context, constraints) {
          final x = (constraints.maxWidth - _width) / 2.0;
          final y = (constraints.maxHeight - _sliderBottom - _height) / 2.0;
          return Stack(
            fit: StackFit.expand,
            children: [
              Positioned.fromRect(
                rect: Rect.fromLTWH(x, y, _width, _height),
                child: Transform.rotate(
                  angle: _rotation,
                  filterQuality: FilterQuality.low,
                  child: Container(color: Colors.green),
                ),
              ),
              Positioned(
                left: 0.0,
                right: 0.0,
                bottom: _sliderBottom,
                child: Slider(
                  value: _rotation,
                  min: 0.0,
                  max: math.pi * 2.0,
                  onChanged: (rotation) => setState(
                    () => _rotation = rotation,
                  ),
                ),
              )
            ],
          );
        },
      ),
    );
  }
}

Screenshots or Video

Screenshots / Video demonstration
Impeller

Skia

Flutter Doctor output
Doctor output
[!] Flutter (Channel unknown, 3.10.6, on macOS 12.6 21G115 darwin-arm64, locale en-US)
    ! Flutter version 3.10.6
    • Framework revision 3d8478a247 (2 hours ago), 2023-07-24 10:12:39 +0200
    • Engine revision 6f8ff0e653
    • Dart version 3.0.6
    • DevTools version 2.23.1
    • Flutter download mirror https://storage.googleapis.com/clay-flutter-storage
    • If those were intentional, you can disregard the above warnings; however it is recommended to use "git" directly to perform update checks and upgrades.

[✓] Android toolchain - develop for Android devices (Android SDK version 32.1.0-rc1)
    • Android SDK at /Users/aleto/Library/Android/sdk
    • Platform android-33, build-tools 32.1.0-rc1
    • Java binary at: /Applications/Android Studio.app/Contents/jre/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.12+0-b1504.28-7817840)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 14.2)
    • Xcode at /Applications/Xcode142.app/Contents/Developer
    • Build 14C18
    • CocoaPods version 1.11.3

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2021.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.12+0-b1504.28-7817840)

[✓] IntelliJ IDEA Community Edition (version 2022.3.1)
    • IntelliJ at /Applications/IntelliJ IDEA CE.app
    • Flutter plugin version 72.1.4
    • Dart plugin version 223.8214.16

[✓] VS Code (version 1.80.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension can be installed from:
      🔨 https://marketplace.visualstudio.com/items?itemName=Dart-Code.flutter

[✓] Network resources
    • All expected network resources are available.

Hi @AlessandroToschi. Thanks for filing this issue. I am able to reproduce this on the latest master and stable channels.

Impeller
Skia

flutter doctor -v

[✓] Flutter (Channel stable, 3.10.6, on macOS 13.4 22F66 darwin-arm64, locale
    en-NG)
    • Flutter version 3.10.6 on channel stable at /Users/dammya/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision f468f3366c (12 days ago), 2023-07-12 15:19:05 -0700
    • Engine revision cdbeda788a
    • Dart version 3.0.6
    • DevTools version 2.23.1

[✓] Android toolchain - develop for Android devices (Android SDK version
    32.1.0-rc1)
    • Android SDK at /Users/dammya/Library/Android/sdk
    • Platform android-33, build-tools 32.1.0-rc1
    • ANDROID_HOME = /Users/dammya/Library/Android/sdk
    • Java binary at: /Applications/Android
      Studio.app/Contents/jre/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build
      11.0.12+0-b1504.28-7817840)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 14.3.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 14E300c
    • CocoaPods version 1.12.1

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2021.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build
      11.0.12+0-b1504.28-7817840)

[✓] IntelliJ IDEA Community Edition (version 2022.3.2)
    • IntelliJ at /Applications/IntelliJ IDEA CE.app
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart

[✓] VS Code (version 1.80.0)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.50.0

[✓] Connected device (4 available)
    • sdk gphone arm64 (mobile) • emulator-5554                        •
      android-arm64  • Android 11 (API 30) (emulator)
    • iPhone 14 Pro (mobile)    • DCF2031A-7EA9-42FF-8CD4-D4584F3E7A6D • ios
      • com.apple.CoreSimulator.SimRuntime.iOS-16-4 (simulator)
    • macOS (desktop)           • macos                                •
      darwin-arm64   • macOS 13.4 22F66 darwin-arm64
    • Chrome (web)              • chrome                               •
      web-javascript • Google Chrome 114.0.5735.248

[✓] Network resources
    • All expected network resources are available.

• No issues found!

[✓] Flutter (Channel master, 3.13.0-8.0.pre.65, on macOS 13.4 22F66 darwin-arm64, locale en-NG)
    • Flutter version 3.13.0-8.0.pre.65 on channel master at /Users/dammya/fvm/versions/master
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 9c10151508 (2 hours ago), 2023-07-24 11:26:05 +0200
    • Engine revision 4734a709cb
    • Dart version 3.1.0 (build 3.1.0-333.0.dev)
    • DevTools version 2.25.0

[✓] Android toolchain - develop for Android devices (Android SDK version 32.1.0-rc1)
    • Android SDK at /Users/dammya/Library/Android/sdk
    • Platform android-33, build-tools 32.1.0-rc1
    • ANDROID_HOME = /Users/dammya/Library/Android/sdk
    • Java binary at: /Applications/Android Studio.app/Contents/jre/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.12+0-b1504.28-7817840)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 14.3.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 14E300c
    • CocoaPods version 1.12.1

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2021.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.12+0-b1504.28-7817840)

[✓] IntelliJ IDEA Community Edition (version 2022.3.2)
    • IntelliJ at /Applications/IntelliJ IDEA CE.app
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart

[✓] VS Code (version 1.80.0)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.50.0

[✓] Connected device (4 available)
    • sdk gphone arm64 (mobile) • emulator-5554                        • android-arm64  • Android 11 (API 30) (emulator)
    • iPhone 14 Pro (mobile)    • DCF2031A-7EA9-42FF-8CD4-D4584F3E7A6D • ios            • com.apple.CoreSimulator.SimRuntime.iOS-16-4 (simulator)
    • macOS (desktop)           • macos                                • darwin-arm64   • macOS 13.4 22F66 darwin-arm64
    • Chrome (web)              • chrome                               • web-javascript • Google Chrome 114.0.5735.248

[✓] Network resources
    • All expected network resources are available.

• No issues found!

FilterQuality.low introduces a new image filter and we have seen instance of this causing an unrelated issue with Impeller. We are tracking in https://github.com/flutter/flutter/issues/130355 and it has a pending PR out for the fix. We should consider cherry picking that. But closing this as a duplicate.

https://github.com/flutter/engine/pull/43943 actually causes a regression in this app.
When applying the rotation, the child widget is rendered at a different location with that PR.
@bdero

With PR #43943
Without PR #43943

The original clipping issue appears to be unrelated to https://github.com/flutter/flutter/issues/130355.
I can reproduce it on an iOS device in landscape mode.  If I drag the slider and release the pointer, then the background of the slider is sometimes painted as opaque, causing it to partially cover the green widget.
Tried bisecting and noticed that this started with the introduction of partial repaint (https://github.com/flutter/engine/commit/20be523fb62651955d70985c1c94369987263b16)
@jonahwilliams

If partial repaint is causing this issue to surface, then generally that means its a bounds computation somewhere. To debug this, we could check what regions of the screen are being repainted.

@jason-simmons I suspect that Impeller is culling out the transformed layer because the EntityPass culling logic doesn't take matrix filters into account. In other words, whenever the dirty region doesn't overlap with the untransformed layer (i.e. when the slider widget visually updates but the rotation of the layer doesn't change), it gets culled.

Comparing Impeller/Skia on ToT and I still see that the green rectangle is rendered differently then with the Skia backend (FYI @bdero )
Skia

Impeller

Now looking at partial repaint, the dirty rect  is computed as the bounds of the slider, however this ignores the fact that the rectangle is beneath it (or over it, I tried both approaches). I'm not sure if these bounds are being computed in DL, but by Impeller only ever gets the dirty rect so its not necessarily making any different decisions here.

FYI @knopp , is partial repaint supposed to be able to alpha composite with the backdrop? I had thought it was an all or nothing sort of approach.

Partial repaint is definitely not supposed to alpha composite.

Thanks, I suspect this is probably still a bounds computation issue then. Still investigating.

Adding some debug logging and looking at the values while one corner of the rectangle is pointed straight down. They look about the same.
Impeller:
[VERBOSE-2:compositor_context.cc(192)] Impeller Clip Rect72,2256,1026,219
[VERBOSE-2:compositor_context.cc(192)] Impeller Clip Rect72,2256,1026,218

Skia:
[VERBOSE-2:compositor_context.cc(169)] Skia Clip Rect72,2249,1026,218
[VERBOSE-2:compositor_context.cc(169)] Skia Clip Rect72,2260,1026,192

So perhaps we're computing the bounds wrong?

Same rotation (PI/4). Both angle and dimensions seem wrong.

Skia
Impeller

Yeah, that seems like an issue.
I think I've narrowed this down to a potential culling issue in the Impeller backend. Partial repaint damage rects are computed identically for both backends. We should see that the image filtered rect intersects with this and paint it as normal, but Impeller is culling it out entirely

@jonahwilliams, this is what causes the problem:
https://github.com/flutter/engine/blob/5b70ff0cfc6b36f964126bb8c269b25ea520a8d5/impeller/entity/entity_pass.cc#L109-L111
The coverage gets clipped to render target size, but that doesn't take into account that it will be later rendered rotated.

Well, part of the problem at least - rasterization. There seems to be  similar things somewhere that breaks the culling.

This might be tricky - because in this case only the image filter knows the rotation matrix.

Yeah we need to add the ability to instantiate the filter graph with deferred inputs in order to handle this correctly: https://github.com/flutter/flutter/issues/130355#issuecomment-1632917871
The short term hack to sidecar FilterContents mentioned in that comment wouldn't work very well in practice, since it's possible for users to compute multiple filters together in the API.
I pulled that comment into a separate issue and added ideas for querying for transformed bounds: https://github.com/flutter/flutter/issues/131632

Yeah, I was wondering if there is a way to instantiate filter before having the contents in order to calculate the coverage limit correctly. I guess that answers my question.

I think I might be misunderstanding the issue a bit, I'm not sure why we don't have enough information to do this today. At the point where we're querying the coverage to cull, we've already constructed the filter and given it an entity. We should know the exact bounds, no? Or is the problem more in the computation of the coverage limit?
Given that the coverage limit is itself an optimiation for culling (?) could we instead disable this if we know a matrix filter is present?
Another piece of data, the coverage limit doesn't just cause the issue with partial repaint, it also causes the rendering of the matrix filter to be "smaller" than with Skia.
Impeller

Skia

Impeller w/ Coverage Limit disabled

I think the filter is constructed in CreateContentsForSubpassTarget, which is after the coverage limit has been applied.

Also I don't think the matrix is smaller. It's the subpass texture size that is smaller (because the width clipped to parent pass render target width), so when it is later painted rotated by image filter it just looks smaller.
If you take the rectangle after rotatation and make it square with same center it should be exact the same size as skia.


Fig containing coneplot, surf, and fill3 won't print

I have a figure containing coneplot, surf, and fill3 plots.  I get the following error when I try to print to a jpeg file:
Error: /rangecheck in --makepattern--
Operand stack:
--dict:7/7(L)--   --nostringval--   false   --dict:7/8(L)--   --nostringval--
Execution stack:
%interp_exit   .runexec2   --nostringval--   --nostringval--   --nostringval--   2   %stopped_push   --nostringval--   --nostringval--   --nostringval--   false   1   %stopped_push   1951   1   3   %oparray_pop   1950   1   3   %oparray_pop   --nostringval--   1934   1   3   %oparray_pop   1820   1   3   %oparray_pop   --nostringval--   %errorexec_pop   .runexec2   --nostringval--   --nostringval--   --nostringval--   2   %stopped_push   --nostringval--   1894   2   3   %oparray_pop   --nostringval--
Dictionary stack:
--dict:1195/1684(ro)(G)--   --dict:0/20(G)--   --dict:129/200(L)--
Current allocation mode is local
Last OS error: Invalid argument
Current file position is 3091626
GPL Ghostscript 9.16: Unrecoverable error, exit code 1
export_fig error. Please ensure:
that you are using the latest version of export_fig
and that you have Ghostscript installed
and that you do not have multiple versions of export_fig installed by mistake
and that you did not made a mistake in the expected input arguments
If the problem persists, then please report a new issue.
Error using imread (line 349)
File "/private/tmp/tp7d6ce963_bc64_4e89_803f_4c2607ce3ba3.tif" does not exist.
Error in print2array (line 131)
A = imread(tmp_nam);
Error in export_fig (line 485)
[A, tcol] = print2array(fig, magnify, renderer);
I think it might have to do with the fact that I don't have any of the directories specified in the .ignore/gs_font_path.txt file.  Mine are stored in /usr/X11/share/fonts.  I tried adding this to the string in gs_font_path.txt, but to no avail.
Kyle

CORRECTION:
The error only shows up when I include the command "lighting gouraud" after creating a lighting object with "camlight right".  If I do not include the "lighting gouraud" command, the plot exports just fine.  This workaround satisfies my needs at the moment, but I suppose it is still an issue with the function.
Kyle

Please provide a simple reproducible Matlab script that produces this error

For some reason, I can't get the error to happen with any vector field
other than the one that I am using in my script.  So here is a script with
the variable attached as well.  I hope this works.
Kyle
On Tue, Aug 11, 2015 at 1:53 PM, Yair Altman<EMAIL_ADDRESS>wrote:

Please provide a simple reproducible Matlab script that produces this error
—
Reply to this email directly or view it on GitHub
https://github.com/altmany/export_fig/issues/95#issuecomment-130044176.

Looks ok on latest version and it's a pretty old issue, so closing it. Please open a new issue is you still see this problem.

issue #95.zip


Paste Translations Error

Hi again! I downloaded the latest version. I am getting an error while pasting the translations.
Traceback:
Traceback (most recent call last):
File "C:\Users\anan_\comic-translate\app\thread_worker.py", line 30, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\anan_\comic-translate\modules\rendering\render.py", line 87, in draw_text
font = ImageFont.truetype(font_pth, size=init_font_size)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\anan_\anaconda3\Lib\site-packages\PIL\ImageFont.py", line 819, in truetype
return freetype(font)
^^^^^^^^^^^^^^
File "C:\Users\anan_\anaconda3\Lib\site-packages\PIL\ImageFont.py", line 816, in freetype
return FreeTypeFont(font, size, index, encoding, layout_engine)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\anan_\anaconda3\Lib\site-packages\PIL\ImageFont.py", line 245, in init
self.font = core.getfont(
^^^^^^^^^^^^^
OSError: cannot open resource

I'm not a dev, but OSError seems to indicate an issue thats originating by the Operating system, namely can not open some resource (a font file, or a file withing the comic-translate project folder).
If this is your first time trying to get comic-translate to run and it has not worked in the past, note that I installed it for the first time yesterday and everything works on a Windows 11 installation.
Here is how to get to my "point". :)

win+r then enter %LOCALAPPDATA%\pip and delete the cache folder from there.
run the python installer you downloaded again, and chose the uninstall option.
download python-3.9.13-amd64.exe (if you are on Windows 11 - but essentially download python-3.9.13 and not any version thats newer than that. I tried it with the most current 3.10.* version first and got installation errors for the prerequisits it pulls)
install python-3.9.13
follow the installation instructions once more

Then only use the fonts supplied by comic-translate by default. And see if that solves your issue.
Or if you can pin down the problem further (as in does the error only occur, when you use a custom font that you provided...). If yes, copying the font file to the comic-translate folder also might solve your issue. (OSError cant open file, might be a permission issue of the font file being situated somewhere, where comic translate doesnt have the rights to read it from. It has permission to read its on folder though.. :) )

If that doesnt get you anywhere, the underscore in your username (anan_) might also cause an issue like this, if some calls in comic-translate were programmed badly. :) (But thats just guessing with nothing that that points that way.)


GCP - handle "Error 409: There were concurrent policy changes." errors

While destroying a GCP PKS deployment with three service account (opsman, pks-worker-node, pks-master-node), we found we had to run leftovers v0.48.0 three times to delete all of the service accounts.  Only one service account was deleted per run.
The errors we received were:
IAM Service Account<EMAIL_ADDRESS>Deleting...
[IAM Service Account<EMAIL_ADDRESS>Deleting...
[IAM Service Account<EMAIL_ADDRESS>Deleting...
[IAM Service Account<EMAIL_ADDRESS>Remove IAM Policy Bindings: Set Project IAM Policy: googleapi: Error 409: There were concurrent policy changes. Please retry the whole read-modify-write with exponential backoff., aborted
[IAM Service Account<EMAIL_ADDRESS>Remove IAM Policy Bindings: Set Project IAM Policy: googleapi: Error 409: There were concurrent policy changes. Please retry the whole read-modify-write with exponential backoff., aborted
[IAM Service Account<EMAIL_ADDRESS>Deleted!
[DNS Managed Zone: grape-staging-zone] Deleting...
[DNS Managed Zone: grape-staging-zone] Deleted!

2 errors occurred:
    * [IAM Service Account<EMAIL_ADDRESS>Remove IAM Policy Bindings: Set Project IAM Policy: googleapi: Error 409: There were concurrent policy changes. Please retry the whole read-modify-write with exponential backoff., aborted
    * [IAM Service Account<EMAIL_ADDRESS>Remove IAM Policy Bindings: Set Project IAM Policy: googleapi: Error 409: There were concurrent policy changes. Please retry the whole read-modify-write with exponential backoff., aborted

Would it be possible for leftovers to add retry logic in the case of a 409?

@markstokan Do you want to open this as an issue here https://github.com/genevieve/leftovers/issues?

No idea why I opened this in the "terraforming-gcp" repo.  Closing and re-opening in leftovers.


Kustomize install libs

Discussed on the addon-operators call, the group would like to see a standalone binary from this repo capable of using some ComponentConfig to prepare several kustomize directories for application to the cluster.
This binary is a POC that can be used to implement installer/management functionality into tools like kops/kubeadm/eksctl as well as many other tools that went unmentioned/unrepresented.
The decision on kustomize is heavily inspired by the ability to overlay arbitrary patches without requiring deps external to the k8s org.
It is also already built into kubectl which helps simplify the POC.
We expect to shell out to kubectl for simplicity of maintenance as its sophisticated apply behavior cannot easily be replicated at this point in time. (see serviside-apply)
The config should allow users to select a kustomize base for installing their operator.
CoreDNS operator is a good starting point.
Users should be able to specify versioned/packaged patch sets.
This is accomplished with a package ref that can start by loading the kustomize base/overlay from a git ref or local file-system location.
Further work may allow loading from other sources.
/kind feature

/lifecycle active


kernel: get rid of newlibc usage in kernel mode

newlibc use a lot of memory and does not provide too much advantages, should we get rid of it inside the kernel and only use it for user process?

Archive member included to satisfy reference by file (symbol)
arch/arm/arm-v7m/isr.o (memcpy)
arch/arm/arm-v7m/isr.o (memset)
arch/arm/arm-v7m/irq-handler.o (_impure_ptr)
arch/arm/common/semihosting.o (strlen)
drivers/core.o (strcmp)
drivers/serial/tty.o (asprintf)
drivers/serial/tty.o (free)
kernel/dev.o (strerror)
kernel/elf-loader.o (fprintf)
kernel/syscall.o (qsort)
kernel/syscall.o (bsearch)

Update list:
arch/arm/arm-v7m/isr.o (memcpy)
arch/arm/arm-v7m/isr.o (memset)
drivers/serial/tty.o (asprintf)
kernel/main.o (strerror)
kernel/syscall.o (qsort)
kernel/syscall.o (bsearch)


missing script: start

I use npm start to run the project but it get the error "missing script: start"
how can i run the project and see what is codelyzer and test it?

Hey @farshidmk take a look at this section in the readme.


Check for previous/existing GitHub issues

[X] I have checked for previous/existing GitHub issues

Issue Type?
Feature Request
Module Name
avm/res/network/network-interface
(Optional) Module Name if not listed above
No response
(Optional) Module Version
No response
Description
Test
(Optional) Correlation Id
No response

avm/res/test

@${issueAuthor}, thanks for submitting this issue for the avm/res/network/network-interface module!
A member of the @azure/avm-res-network-networkinterface-module-owners-bicep or @azure/avm-res-network-networkinterface-module-contributors-bicep team will review it soon!


[Go Live][April 14]

Pushing to live

Docs Build status updates of commit eea2fb6:
:warning: Validation status: warnings

File
Status
Preview URL
Details

msteams-platform/bots/how-to/conversations/subscribe-to-conversation-events.md
:warning:Warning
View
Details

msteams-platform/build-your-first-app/build-first-app-overview.md
:white_check_mark:Succeeded
View

msteams-platform/tabs/design/tabs-mobile.md
:white_check_mark:Succeeded
View

msteams-platform/bots/how-to/conversations/subscribe-to-conversation-events.md

Line 1294, Column 1: [Warning-invalid-tab-group] Tab group with different tab id set.

For more details, please refer to the build report.
If you see build warnings/errors with permission issues, it might be due to single sign-on (SSO) enabled on Microsoft's GitHub organizations. Please follow instructions here to re-authorize your GitHub account to Docs Build.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
Note: Your PR may contain errors or warnings unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


Analysis 2019-2.0

No suitcases this time. Per our discussion with @danielballan in Slack, the old suitcase is already explicitly installed via ansible's extra_packages. The new suitcase-{tiff,csv,jsonl,json-metadata} packages will be put manually into the extra-packages if beamlines don't use the old suitcase for HDF5 exports.
Also, I put the minimum versions for bluesky software stack that we just tagged for this deployment to avoid surprises.

Thanks for the review. Self-merging.


UnicodeDecodeError

`./web.sh

Running on http://<IP_ADDRESS>:8000/ (Press CTRL+C to quit)
Restarting with stat
Debugger is active!
Debugger pin code: 208-815-793
/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/default_comparator.py:153: SAWarning: The IN-predicate on "sightings.pokemon_id" was invoked with an empty sequence. This results in a contradiction, which nonetheless can be expensive to evaluate.  Consider alternative strategies for improved performance.
'strategies for improved performance.' % expr)
<IP_ADDRESS> - - [08/Aug/2016 21:32:10] "GET /report HTTP/1.1" 500 -
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 2000, in call
return self.wsgi_app(environ, start_response)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1991, in wsgi_app
response = self.make_response(self.handle_exception(e))
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1567, in handle_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1988, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1641, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1544, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1639, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1625, in dispatch_request
return self.view_functionsrule.endpoint
File "/hdd/Pokemon/pokeminer-ph3/web.py", line 198, in report_main
google_maps_key=config.GOOGLE_MAPS_KEY,
File "/usr/local/lib/python2.7/dist-packages/flask/templating.py", line 134, in render_template
context, ctx.app)
File "/usr/local/lib/python2.7/dist-packages/flask/templating.py", line 116, in _render
rv = template.render(context)
File "/usr/local/lib/python2.7/dist-packages/jinja2/environment.py", line 989, in render
return self.environment.handle_exception(exc_info, True)
File "/usr/local/lib/python2.7/dist-packages/jinja2/environment.py", line 754, in handle_exception
reraise(exc_type, exc_value, tb)
File "/hdd/Pokemon/pokeminer-ph3/templates/report.html", line 128, in top-level template code

UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 7: ordinal not in range(128)`

same issue here, all is running good just web.py isnt working, i had it working on my windows PC but when i migrated over to a linux box i get this error also.

I found the error to be in this line:

This controls the icons appearing for the top 30 line in the report. Not sure why this is happening other than there might be a utf-8 character in here and python threw a fit.

thanks,
grenadesonfire temp fix works perfectly.

duplicate of #161

It should be fixed as of c3fd59dc64f94 (in master).


Verifies arch in Dockerfile and polishes build

As we get more requests for architecture, we should be more careful and
at least lightly check what we intended to install.
Same change as https://github.com/openzipkin/docker-alpine/pull/3

verified remote docker build
docker build https://github.com/openzipkin/docker-java.git#verify-arch
[+] Building 151.9s (4/9)
[+] Building 152.1s (4/9)
[+] Building 253.6s (10/10) FINISHED
 => [internal] load git source https://github.com/openzipkin/docker-java.git#verify-arch                                                                                2.6s
 => [internal] load metadata for ghcr.io/openzipkin/alpine:3.12.1                                                                                0.0s
 => CACHED [base 1/2] FROM ghcr.io/openzipkin/alpine:3.12.1                                                                                0.0s
 => [base 2/2] WORKDIR /java                                                                                0.0s
 => [jdk 1/1] RUN PACKAGE=openjdk15 && apk --no-cache add ${PACKAGE}-jmods=~15.0.1_p9 ${PACKAGE}-jdk=~15.0.1_p9 binutils tar && java -version && jar --version && jlink --version  242.0s
 => [install 1/2] WORKDIR /install                                                                                0.0s
 => [install 2/2] RUN if [[ "15" = "15" && "$(uname -m)" = "aarch64" ]]; then STRIP=""; else STRIP="--strip-debug"; fi && jlink --vm=server --no-header-files --no-man-pages --comp  7.5s
 => [jre 1/2] COPY --from=install /install/jre/ /usr/lib/jvm/java-15-openjdk/                                                                                0.5s
 => [jre 2/2] RUN java -version                                                                                0.3s
 => exporting to image                                                                                0.4s
 => => exporting layers                                                                                0.4s
 => => writing image sha256:7712d2d119111a9e4729305d627faa10f9d26b86734bbb2206fb8cafa5331a5a


Adjust resource requests

With just 20Mi request/30Mi limits, my instance of the wave controller running in a Kubernetes cluster with only ~50 pods never really started. It was already stuck at the log line Starting metrics server.
Running the application locally in the debugger worked fine. The issue turned out to be starvation in garbage collection. Slightly increasing the memory requests resolved the issue entirely.
Experiments allowed stable (but slow) operations starting with 40Mi requests/limits; with 75Mi wave starts and performs required operations within few seconds. Proposing to set request=limit to get into guaranteed QoS class.

Hi @JensErat. Thanks for your PR.
I'm waiting for a pusher member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.
Once the patch is verified, the new status will be reflected by the ok-to-test label.
I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

/ok-to-test

@JensErat: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

pull-wave-test-1.13
550e74062a630f60a5b52122835f8292d0747ade
link
/test 1.13

Full PR test history

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@JensErat: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

pull-wave-test-1.13
550e74062a630f60a5b52122835f8292d0747ade
link
/test 1.13

pull-wave-test-1.12
550e74062a630f60a5b52122835f8292d0747ade
link
/test 1.12

Full PR test history

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.


fix store_artifacts_locally docs to reflect reality

discovered the docs for store_artifacts_locally are incorrect, so this corrects that.

I have read the CLA Document and I hereby sign the CLA
recheck

recheck

recheck

@tjstansell Not sure why the CLA bot fails. You clearly have signed the CLA 😄 I'll bypass the requirement and merge this.


Create Delete Columns to Make Sorted (Ahmed zedan).cpp

add the number of the day first in the name of the file


chore add branch-3.0 required checks

add required checks to accelerate the process of pipeline-related problem fixing

run buildall


Use field.to_python to do django type conversions on the field before ch...

...ecking if dirty.
This solves issues where you might have a decimal field that you write a string to, eg:

m = MyModel.objects.get(id=1)
m.my_decimal_field
Decimal('1.00')
m.my_decimal_field = u'1.00' # from a form or something
m.is_dirty() # currently evaluates to True, should evaluate to False
False

This pull request could probably use some unit testing, but it should be safe as the base class for django fields defines to_python as:
def to_python(self, value):
return value
So, any field type that does not have an explicit to_python method will behave as before this change.

I'm ok with that solution (call to_python) because we have to follow what Django is implicitly casting to be coherent.
@smn, your feedback about this ?

I'm :+1: on this assuming the tests we do have pass? Also, the recent stuff needs to be merged into this branch for us to be able to land it.

@smn, as we have some problems with different type of fields (#10, #11), I will propose a more global solution. Maybe, instead of managing a simple comparison == between old and new value, we can create something that is related to the field type.

I will work on a general solution in few time. There is no reason to wait for merging this, so I have merged it to develop !
Thanks @jimrollenhagen !


Update fastlane_core version

Changes since release 0.41.3:

Fixed scan conflicted parameters.
Fix link to issue that was lost during monorepo change
Make ConfigItem short_option optional (#4136)
Remove duplicated specs
Add support for connected devices

💎 ✨


Reduce amount of compilations

Fix: #45

I thought it was automatically disabled for production builds anyway? (From https://react.dev/reference/react/StrictMode: "All of these checks are development-only and do not impact the production build.") Have you seen a difference with this change?

Yea you right. Maybe we can improve loading time in another way.


feat: optimize page view after session start

Description

do not track page view after browser reload
add page view event when session start from reopen a hide page

General Checklist

[x] Added new tests to cover change, if needed
[ ] Security oriented best practices and standards are followed (e.g. using input sanitization, principle of least privilege, etc)
[ ] Documentation update for the change if required
[x] PR title conforms to conventional commit style
[ ] If breaking change, documentation/changelog update with migration instructions

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 100.00%. Comparing base (86cd73b) to head (0d39041).

Additional details and impacted files
@@            Coverage Diff            @@
##              main       #44   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files           25        25
  Lines         1082      1128   +46
  Branches       209       224   +15
=========================================
+ Hits          1082      1128   +46

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Add support for plugin ordering

What this PR does / why we need it:
Adds plugin ordering support. Uses unreleased KTF, deck, and go-kong for now. Adds integration tests against nightly Kong images. Requesting review in advance of dependencies since I want to avoid multiple upstream releases in case change requests affect stuff in libraries.
Depends on https://github.com/Kong/go-kong/pull/187, https://github.com/Kong/deck/pull/710, and https://github.com/Kong/kubernetes-testing-framework/pull/314
Fix https://github.com/Kong/kubernetes-ingress-controller/issues/2656
Special notes for your reviewer:
PR Readiness Checklist:
Complete these before marking the PR as ready to review:

[x] the CHANGELOG.md release notes have been updated to reflect any significant (and particularly user-facing) changes introduced by this PR

Debugging why the TLSRoute example test fails on nightly here. It doesn't locally when run in isolation. I expect that's part of adding nightly tests in general rather than because of plugin order, so basically ignoring it for the purpose of review.

?


Vite inlines icon resources that should not be inlined

Describe the bug
When Vite asset-ifies <link> resources it applies the default assetsInlineLimit for JS without considering whether it makes sense to inline the resource in HTML.
One issue this causes is in site icons. If a site icon is small enough, instead of just putting it in /assets with a cache buster (which can be desirable for site icons), Vite will inline it into the document.
This is a problem since now site icons are downloaded inline with every page request which is problematic for a number of reasons:

Site icons are a low priority but now must be downloaded before critical page content
The data now comes that comes with every page load on a likely uncacheable request instead of allowing the browser to cache the icon
All site icons are downloaded, even if its an icon the browser would not have downloaded (unsupported file type, media query, unused size)

Manifests are also inlined, but that bug was reported in #5962.
Reproduction
https://github.com/dantman/vite-excessive-inline-resources-bug
System Info
System:
    OS: Linux 5.10 Ubuntu 21.10 21.10 (Impish Indri)
    CPU: (24) x64 AMD Ryzen 9 3900X 12-Core Processor
    Memory: 15.18 GB / 24.99 GB
    Container: Yes
    Shell: 5.1.8 - /bin/bash
  Binaries:
    Node: 17.4.0 - ~/.nvm/versions/node/v17.4.0/bin/node
    Yarn: 1.22.11 - ~/.nvm/versions/node/v17.4.0/bin/yarn
    npm: 8.3.1 - ~/.nvm/versions/node/v17.4.0/bin/npm
  npmPackages:
    vite: ^2.7.13 => 2.7.13

Used Package Manager
npm
Logs
vite:config no config file found. +0ms
  vite:config using resolved config: {
  vite:config   root: '/mnt/c/Users/danie/vite-test/vite-project',
  vite:config   base: '/',
  vite:config   mode: 'production',
  vite:config   configFile: undefined,
  vite:config   logLevel: undefined,
  vite:config   clearScreen: undefined,
  vite:config   build: {
  vite:config     target: [ 'es2019', 'edge88', 'firefox78', 'chrome87', 'safari13.1' ],
  vite:config     polyfillModulePreload: true,
  vite:config     outDir: '/mnt/c/Users/danie/vite-test/vite-project/dist',
  vite:config     assetsDir: 'assets',
  vite:config     assetsInlineLimit: 4096,
  vite:config     cssCodeSplit: true,
  vite:config     cssTarget: [ 'es2019', 'edge88', 'firefox78', 'chrome87', 'safari13.1' ],
  vite:config     sourcemap: false,
  vite:config     rollupOptions: { input: '/mnt/c/Users/danie/vite-test/vite-project/index.html' },
  vite:config     minify: 'esbuild',
  vite:config     terserOptions: {},
  vite:config     write: true,
  vite:config     emptyOutDir: null,
  vite:config     manifest: false,
  vite:config     lib: false,
  vite:config     ssr: false,
  vite:config     ssrManifest: false,
  vite:config     reportCompressedSize: true,
  vite:config     chunkSizeWarningLimit: 500,
  vite:config     watch: null,
  vite:config     commonjsOptions: { include: [Array], extensions: [Array] },
  vite:config     dynamicImportVarsOptions: { warnOnError: true, exclude: [Array] }
  vite:config   },
  vite:config   configFileDependencies: [],
  vite:config   inlineConfig: {
  vite:config     root: undefined,
  vite:config     base: undefined,
  vite:config     mode: undefined,
  vite:config     configFile: undefined,
  vite:config     logLevel: undefined,
  vite:config     clearScreen: undefined,
  vite:config     build: {}
  vite:config   },
  vite:config   resolve: { dedupe: undefined, alias: [ [Object], [Object] ] },
  vite:config   publicDir: '/mnt/c/Users/danie/vite-test/vite-project/public',
  vite:config   cacheDir: '/mnt/c/Users/danie/vite-test/vite-project/node_modules/.vite',
  vite:config   command: 'build',
  vite:config   isProduction: true,
  vite:config   plugins: [
  vite:config     'alias',
  vite:config     'vite:modulepreload-polyfill',
  vite:config     'vite:resolve',
  vite:config     'vite:html-inline-script-proxy',
  vite:config     'vite:css',
  vite:config     'vite:esbuild',
  vite:config     'vite:json',
  vite:config     'vite:wasm',
  vite:config     'vite:worker',
  vite:config     'vite:asset',
  vite:config     'vite:define',
  vite:config     'vite:css-post',
  vite:config     'vite:watch-package-data',
  vite:config     'vite:build-html',
  vite:config     'commonjs',
  vite:config     'vite:data-uri',
  vite:config     'rollup-plugin-dynamic-import-variables',
  vite:config     'vite:asset-import-meta-url',
  vite:config     'vite:build-import-analysis',
  vite:config     'vite:esbuild-transpile',
  vite:config     'vite:reporter',
  vite:config     'vite:load-fallback'
  vite:config   ],
  vite:config   server: {
  vite:config     preTransformRequests: true,
  vite:config     fs: { strict: true, allow: [Array], deny: [Array] }
  vite:config   },
  vite:config   preview: {
  vite:config     port: undefined,
  vite:config     strictPort: undefined,
  vite:config     host: undefined,
  vite:config     https: undefined,
  vite:config     open: undefined,
  vite:config     proxy: undefined,
  vite:config     cors: undefined
  vite:config   },
  vite:config   env: { BASE_URL: '/', MODE: 'production', DEV: false, PROD: true },
  vite:config   assetsInclude: [Function: assetsInclude],
  vite:config   logger: {
  vite:config     hasWarned: false,
  vite:config     info: [Function: info],
  vite:config     warn: [Function: warn],
  vite:config     warnOnce: [Function: warnOnce],
  vite:config     error: [Function: error],
  vite:config     clearScreen: [Function: clearScreen],
  vite:config     hasErrorLogged: [Function: hasErrorLogged]
  vite:config   },
  vite:config   packageCache: Map(0) { set: [Function (anonymous)] },
  vite:config   createResolver: [Function: createResolver],
  vite:config   optimizeDeps: {
  vite:config     esbuildOptions: { keepNames: undefined, preserveSymlinks: undefined }
  vite:config   }
  vite:config } +13ms
vite v2.7.13 building for production...
✓ 4 modules transformed.
dist/assets/apple-touch-icon.2d7ab2e6.png   13.82 KiB
dist/index.html                             4.60 KiB
dist/assets/index.f72635f1.js               0.85 KiB / gzip: 0.49 KiB
dist/assets/index.06d14ce2.css              0.17 KiB / gzip: 0.14 KiB

Validations

[X] Follow our Code of Conduct
[X] Read the Contributing Guidelines.
[X] Read the docs.
[X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
[X] Make sure this is a Vite issue and not a framework-specific issue. For example, if it's a Vue SFC related bug, it should likely be reported to https://github.com/vuejs/core instead.
[X] Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
[X] The provided reproduction is a minimal reproducible example of the bug.

We have a similar issues when bundling vite apps to electron/nwjs. there are some 3rd party integrations that expect urls/file paths and do not support inlined base64 assets.

I think #2173 (and #10578 and #8717) are related

I think it makes sense to not inline rel="icon" by default, and the same for manifest files too

It also makes sense to inline link rel="apple-touch-icon" by default since safari throws an error when the url is base64
eg:


make test easier to run

Running the test from a fresh clone raises the following:
$ make test
nvim --headless --noplugin -u tests/minimal.vim -c "PlenaryBustedDirectory tests/ {minimal_init = 'tests/minimal.vim'}"
Error detected while processing /home/dfavato/workspace/my/openingh.nvim/tests/minimal.vim:
line   21:
E5108: Error executing lua [string ":lua"]:1: module 'plenary/busted' not found:
        no field package.preload['plenary/busted']
        no file './plenary/busted.lua'
        no file '/home/runner/work/neovim/neovim/.deps/usr/share/luajit-2.1/plenary/busted.lua'
        no file '/usr/local/share/lua/5.1/plenary/busted.lua'
        no file '/usr/local/share/lua/5.1/plenary/busted/init.lua'
        no file '/home/runner/work/neovim/neovim/.deps/usr/share/lua/5.1/plenary/busted.lua'
        no file '/home/runner/work/neovim/neovim/.deps/usr/share/lua/5.1/plenary/busted/init.lua'
        no file './plenary/busted.so'
        no file '/usr/local/lib/lua/5.1/plenary/busted.so'
        no file '/home/runner/work/neovim/neovim/.deps/usr/lib/lua/5.1/plenary/busted.so'
        no file '/usr/local/lib/lua/5.1/loadall.so'
stack traceback:
        [C]: in function 'require'
        [string ":lua"]:1: in main chunk
Error detected while processing command line:
E492: Not an editor command: PlenaryBustedDirectory tests/ {minimal_init = 'tests/minimal.vim'}^Cmake: *** [Makefile:10: test]

These are the changes I did to make it run.

Thank you for your changes, I'm really sorry I didn't even see your PRs


Update the Unity Package build step to accurately named.

I think that this step used to build nuget packages as well, until we did the later NuGet work (of packing it up at the end of CI). However, this text appears to have stuck around.
Note that the command it runs actually says "NoNuget"
I noticed this as I was doing some nuget/packaging work, figured I should clean it up while I'm in there.

The build definitions have changed since your commit. I think the easiest way to resolve this would be to just reapply the edits in the new files as this looks like a very small update. Sorry about that.


Another debug issue?

My colleague reported this one to me:
module WindowsApp1;

import std.stdio;

void textLength(const(char)[] text)
{
	writeln("length: ", text.length); // breakpoint here, can't inspect .length
}

int main()
{
	const(char)[] str = "Hello D World!\n";
	textLength(str);
    return 0;
}

This is just a tweak of the wizard app. In this code, break on that line, you can't inspect length.
I feel like I used to be able to inspect the length of arrays?

The problem here is that the text argument is actually passed by reference ABI-wise, and that's typed as a pointer in the debug info. The expression evaluator fails to crawl this additional indirection. If you assign the text to a local variable txt, then txt.length is evaluated correctly.

Is there any way to correct this? Can it be typed at a reference instead of a pointer?
I suspect this particular situation is causing a great many cases I've been seeing to fail in a similar way.

It should be possible to fix it in the compiler (LDC seems to work better), but in this case, the evaluator should also support the indirection when using '.' with a pointer as it is valid syntax. BTW: (*text).length works, too.

You can find a preliminary version of the fixed debugger plugin in the artifacts of the CI builds here: https://ci.appveyor.com/project/rainers/mago/build/artifacts
You just need to replace this file: "c:\Program Files\Microsoft Visual Studio\2022\Preview\Common7\Packages\Debugger\MagoNatCC.dll" where "Preview" is probably something else on your system.

I tried that DLL and this particular problem was resolved. I'll run for a while and see if any of my other issues are also resolved...
Cheers!

If it still stalls/takes forever (no changes with respect to that), you can also use the pdb-file to get an idea where it is wasting all the time when attaching a debugger to devenv.exe.

I'll definitely do that next time I notice it locking up.

fix released in https://github.com/dlang/visuald/releases/tag/v1.4.0-rc3


[REBASE FIXUPS] Self-serve Bulk Tagging

A general purpose interface for bulk tagging of GOV.UK content via
mapping spreadsheets stored on Google Drive. "General purpose" in the
sense that any piece of content can be tagged with any link type, as
long as the required fields are found in the source spreadsheet.
Includes:

A BulkTagging model storing the public URLs of the spreadsheets.
A TagMapping model capturing all of the fields required to tag a piece
of content.
Rudimentary error handling when fetching tags from the spreadsheet,
storing them locally, and sending them to the publishing API.
A series of screens for creating a new BulkTagging, previewing its
TagMappings, refreshing these TagMappings following an update to the
spreadsheet, and sending these tags to the publishing API.

trello https://trello.com/c/yndptBXH/22-allow-self-service-bulk-tagging-with-google-docs

Looking good! As disussed offline, this needs Sidekiq to work because the publishing-api calls will take too long.

👍


Volt: Fix resume bug on high resistance (steep hills etc)

@vntarasov @rbiasini
Fixed :)
apply_gas is the best determination I could find to figure out when OP wants to "go." We need to release the want_full_stop "request" when accelerating from a full stop. Previous code was working because it was winding up the PID loop to heavy (max) acceleration.
Side bonus is that the resume acceleration is much smoother/gentler (too gentle?)

@Jamezz looks good. Would like to have confirmation from @vntarasov or @TheMutley

Will take this PR for a test drive tonight on 0.5.3.

I'm going to merge this PR into #326 as it's related.


Confirm whether we should update the blob_sas_token as empty to match Azure Open Data Storage.

Hi guys,
When we run the Spark SQL job with following blob sas token.
blob_sas_token = r"?st=2019-02-26T02%3A34%3A32Z&se=2119-02-27T02%3A34%3A00Z&sp=rl&sv=2018-03-28&sr=c&sig=XlJVWA7fMXCSxCKqJm8psMOh0W4h7cSYO28coRqF2fs%3D"
There is one error message in log console:

After we replace with blob_sas_token = r"" , the job can show the first 10 rows recored.
Whether it is an azure open data storage currently, we should remove the previous SAS token in code?

Best Regards,
Edward Chen

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 355fba0e-958e-f572-18c2-892c91d20b13
Version Independent ID: 7fe76247-c969-2068-2c3c-bf327fbbe39e
Content: Quickstart - Run a Spark job on Azure Databricks Workspace using Azure portal
Content Source: databricks/scenarios/quickstart-create-databricks-workspace-portal.md
Service: azure-databricks
GitHub Login: @mssaperla
Microsoft Alias: saperla

Thanks for the feedback! We are currently investigating and will update you shortly.

Thanks for providing feedback that helps improve our documentation. This issue has been resolved and we have removed this outdated content. We are closing the issue.
#please-close

#please-close


Janitor getters for metrics should be public

NetflixOSS » SimianArmy » SimianArmy-pull-requests #30 SUCCESS
This pull request looks good

SimianArmy-pull-requests #202 SUCCESS
This pull request looks good

NetflixOSS » SimianArmy » SimianArmy-pull-requests #31 SUCCESS
This pull request looks good

SimianArmy-pull-requests #203 SUCCESS
This pull request looks good


per-column search: datatable.api is not a function

Hi,
I ran you example code (commit: 9f82b0051b00e547de509ae7f0282c849d8ce776) testing your per-column search. It doesn't work in my hands. The javascript console says:
(index):67 Uncaught TypeError: datatable.api is not a function
   at HTMLDocument.<anonymous> ((index):67)
    at l (jquery-2.0.3.min.js:4)
    at Object.fireWith [as resolveWith] (jquery-2.0.3.min.js:4)
    at Function.ready (jquery-2.0.3.min.js:4)
    at HTMLDocument.S (jquery-2.0.3.min.js:4)
The problem seems to be the var table = datatable.api();. Is this routine the same as datatable.DataTable.Api()?

Hi,
Datatables docs says you can get the API three ways:
$( selector ).DataTable();
$( selector ).dataTable().api();
new $.fn.dataTable.Api( selector );

The first method works if I invoke DataTables() manually.
If I use datatableview.initialize(), which of course I want to do, I don't seem to have access to the API from the object I get back from initialize.
However the third method seems to work:
var api = new $.fn.dataTable.Api( selector );


[Catalog API] Catalog leaf doesn't contain required fields

@jamesqo commented on Thu Apr 19 2018
I just encountered a catalog leaf like this:
{
    "@id": "https://api.nuget.org/v3/catalog0/data/2<IP_ADDRESS>.24.21/fluenttcx.<IP_ADDRESS>.json",
    "@type": ["PackageDelete", "catalog:Permalink"],
    "catalog:commitId": "47db396f-53df-47fb-8183-1c607d3c019d",
    "catalog:commitTimeStamp": "2015-10-28T13:24:21.9232363Z",
    "id": "FluentTcX",
    "originalId": "FluentTcXc",
    "published": "2015-10-28T13:24:20.2527322Z",
    "version": "<IP_ADDRESS>",
    "@context": {
        "@vocab": "http://schema.nuget.org/schema#",
        "catalog": "http://schema.nuget.org/catalog#",
        "xsd": "http://www.w3.org/2001/XMLSchema#",
        "details": "catalog:details",
        "catalog:commitTimeStamp": {
            "@type": "xsd:dateTime"
        },
        "published": {
            "@type": "xsd:dateTime"
        },
        "categories": {
            "@container": "@set"
        },
        "entries": {
            "@container": "@set"
        },
        "links": {
            "@container": "@set"
        },
        "tags": {
            "@container": "@set"
        },
        "packageContent": {
            "@type": "@id"
        }
    }
}

According to the docs the created and isPrerelease fields are non-optional, but they're not present here. This is also the first leaf (after scanning thru ~1162 catalog pages) that fails to contain: authors and description.

In short, we should:

[ ] Mark these fields as optional in the docs.
[ ] Suggest fallback behavior for clients interested in these fields.


http3: correctly use the quic.Transport

Fixes #3868. Depends on #3867

The integration test won't pass until #3867 is merged.

I merged #3867. Could you rebase this PR?

Question: Does the quic.Transport.Close() close the underlying UDPConn? Otherwise we should close it manually in the http3.RoundTripper.Close().

Question: Does the quic.Transport.Close() close the underlying UDPConn? Otherwise we should close it manually in the http3.RoundTripper.Close().

It doesn't. Want to send another PR before I cut v0.35.1?


Publish wasm binary

Based on https://github.com/corazawaf/coraza-proxy-wasm/pull/185/files
cc @M4tteoP

Closed by #11 & #12


Import destroys build

I don't know if this fits here, if not I'm sorry.
When I import a custom file at the top of my sketch.ts I get the error "error TS2686: 'p5' refers to a UMD global, but the current file is a module. Consider adding an import instead."
Why does this happen just because I add an import?
This is my sketch.ts:
import Carcasonne from './game'

const carcasonne = new Carcasonne()

var sketch = (p: p5) => {
 p.setup = () => {
   p.createCanvas(p.windowWidth, p.windowHeight);
 }

 p.draw = () => {
   p.background(0);
   p.fill(255);
   carcasonne.draw()
 }
}

new p5(sketch);

function windowResized() {
 createCanvas(windowWidth, windowHeight);
}

The rest is the same.

Since your using p5 in instanced mode.  You can delete the global.d.ts file and then include the following this to the top of your sketch.ts file
import * as p5 from "p5";

This previous issue #4 may help.
Looks like they ended up using parcel to ommit the tsconfig file and package the application for the browser.

Thank you, that works for me :)
Now it compiles just fine, but it still doesn't work somehow. I changed my sketch.ts to this:
 import Carcassonne from './game'

let game = new carcasonne()

function setup() {
  createCanvas(windowWidth, windowHeight);
}

function draw() {
  background(0);
  fill(255);
  rect(10, 10, 50, 50);
}

I abandoned the instance-mode because I thought it would fix my error before, but even in instance mode, the sketch doesn't work when I import my module. In both modes I just get a white screen in the browser :[

Are there any errors in the console? If no errors I think you can run.
new p5();

In the console.  That will tell you if p5 opened in global mode (I believe if it sees draw() or setup() it runs in global mode)

There are no errors, and it also doesn't work with new p5() :/
It's pretty weird when I import my module setup() and draw() gray out and I get the hint " 'setup' is declared but its value is never read." Do you have any idea why that might happen ?

Working example here:
https://github.com/Gaweph/p5-typescript-starter/tree/import_example
You are probably going to need some sort of bundler if you are going to start importing modules.  I used rollup in this small example, which appears to bundle everything into a single self loaded js file.
Heres an article if you want to go more in depth into this stuff:
https://medium.com/@ajmeyghani/javascript-bundlers-a-comparison-e63f01f2a364
Basically, I'm targetting ES2015 module type in tsconfig.  I removed the global.d.ts file and also this needs to run in instanced mode (I kept getting white screen you mentioned otherwise) and then told index.html to point to bundle.js
Good luck sir.

It's a little annoying to put p5 in front of everything, especially when setting it as a class-variable so it turns to 'this.p5', but hey, it works now :) Thanks a lot for helping ! When I find a solution that works in global mode too I'll make a pull-request :)

I ended up passing a variable called 'p' of type p5 around in instanced mode.  Which seems to work nicely.
Maybe this starter should be changed to use a bundler nad instanced mode out of hte box.  I wanted it to reflect the documentation of the p5 main site.  Which is in global mode with  sketch file.
Good luck with your project.


Releasing: Changelog and main branch

Thanks for your work!
Last releases were not correctly packaged.
For every release (stable tags), could you everytime:

move main branch
update changelog (currently is out to date 0.5.10 -> 0.5.15)

Many Thanks!

Sure thing. Now main tracks 0.5.15.
I'll update the changelog


Feature/protobuf bdo

Need to work on the conflicts before merge.  But wanted to get this pull request open so @geoffjentry and @tetron can comment (and anyone else interested of course!)

Issues to discuss:

Camel caps, underscore with mixed case, underscore with lowercase (I prefer the latter)... need to be consistent between TES and WES and within
Design choice of punting on file inputs/outputs... is this going to work?
Do we need some way to advertise supported file protocols e.g. s3://?  What about way to pass credentials?
I added "metadata" hash to workflow submission since I thought it would handy.  Does TES want to do this as well?  I see metadata hash in their workflow status response.
I'm using several items directly from the TES proto3... for example ga4gh_task_execJobLog.  Should these be renamed to remove "task"?
WES currently has an outputs section, which to me is strange because I think we should either just put everything in the JSON param for the workflow or we should be explicit about inputs and outputs in WES.  Question is, how are they related to each other?  If I put URLs in the input, then something will have to stage them.  Then when I write outputs something will have to find them locally, then push them to a final destination.  We need to discuss file provisioning and see how much of this should and should not be exposed with WES.
Should we have /wes/ in the beginning of the URL?  What about /tes/ for that service too?

@tetron and @geoffjentry I merged master into this branch

@briandoconnor Thanks! A few thoughts on your bullet points:

re formatting, I don't care what the decision is but agree it should be uniform
re file inputs. If you're saying what I think you're saying I like the idea as a first pass to be super unopinionated. "Hey WES server, here's a (CWL|WDL) and some related stuff. I'm assuming you know what to do with the related stuff now that you know if it is CWL or WDL".
re Protocols: Despite what I just said you raise a good point although I think that might start opening a can of worms leading us down the non-KISS path
re Credentials: None of these APIs are usable by most people w/o this, but we haven't let that stop us before :)
re Metadata: I made my comments in the code changes prior to seeing this. I'm against adding random bits of arbitrary info in the "here, hold my stuff" capacity. If there are use cases where an implementing engine would be able to act on these fields I'll retract my stance
re TES proto3: agree
re wes in URL: Which URL?

As written right now, the WorkflowStatus record only has a single ga4gh_task_exec.JobLog entry, so it only stores a single job. This should be an array.

@kellrott I think (I hope) that was unintentinoal. this should be returning (IMO) workflow status, not individual job statuses

Having the workflow_type as an enumeration seems a little limited. It might be better to have that as a string, so a server could extend the spec with additional workflow types.
It might also make sense to have a 'GetServiceInfo' method similar to what TES has. The return on the service info could list the different workflow/versions that the service provides.

@geoffjentry I would hope that the API provides a mechanism to introspect the individual jobs that were made as a part of the workflow.

@kellrott sure but not as part of the simple status endpoint, although perhpas we're not talking about the same thing

@briandoconnor I think this is the only WES implementation right now, do you want to update it to work with the proposed API?
https://github.com/common-workflow-language/cwltool-service/tree/ga4gh-wes

WES currently has an outputs section, which to me is strange because I think we should either just put everything in the JSON param for the workflow or we should be explicit about inputs and outputs in WES. Question is, how are they related to each other? If I put URLs in the input, then something will have to stage them. Then when I write outputs something will have to find them locally, then push them to a final destination. We need to discuss file provisioning and see how much of this should and should not be exposed with WES.

In my version this was intended to be the output object you get from a CWL workflow.  It is null until the workflow completes, and then it holds the result.

@kellrott  re:  "workflow_type as an enumeration seems a little limited." I agree, it also seems odd having a unknown type as an actual type. It seems oddly self-referential or something.
@briandoconnor  re:  "Should we have /wes/ in the beginning of the URL? What about /tes/ for that service too?" tool registry schema uses " /api/ga4gh/v1" as a prefix if we want to be consistent. https://github.com/ga4gh/tool-registry-schemas/blob/develop/src/main/resources/swagger/ga4gh-tool-discovery.yaml#L9

@denis-yuen Definitely agree about the "UNKNOWN" type. Initially I was very pro-enumeration but thinking about it over the last day or so I'm definitely coming around to Kyle's suggestion. The gotcha there is enforcing consistency -> i.e. "cwl" vs "CWL" vs "commonwl", etc.
Re the api prefixes that's a good point. In retrospect I'm not a huge fan of just saying ga4gh but it's also what is already there.

Hmmm... the ability to comment inline as a reply seems limited to comments on lines of code.  Am I missing something?  Really strange to not be able to reply to comments...
So I'll just respond to the above here and make edits to the schema as I go:

@kellrott let's discuss the status output tomorrow on the call. I can see this working in a few different ways and I think we need to decide how we want to handle this as a group.  I'm thinking WES is really focused on reporting the status of the workflow.  But the moment you report a failure the first question is going to be "what job in the workflow failed and what's it's log?"

@geoffjentry @denis-yuen @tetron @kellrott enumeration, I agree with moving this to a string although it opens up the "CWL" vs "cwl" etc issue.  I think the spec should define official spellings with a GetServiceInfo endpoint like Kyle recommended.  And if a given system supports something not covered by the spec, then that's fine.

@kellrott is it sufficient to simply report back the original CWL/WDL to get information about the jobs in a workflow?  Is that the minimum viable here?  Or are you specifically referring to status of each job rather than just information about what jobs a workflow contains?

For consistent endpoints, I vote for:
/ga4gh/trs/v1 -> the tool registry service
/ga4gh/tes/v1 -> the task execution service
/ga4gh/wes/v1 -> the workflow execution service
/ga4gh/dos/v1 -> the data object service

Can we try to lock this down on the next call?  Does this gel with the other read/variant/etc data endpoints?

@tetron yes, would love to see https://github.com/common-workflow-language/cwltool-service/tree/ga4gh-wes updated to this API.  Maybe after the call tomorrow when we incorporate feedback and hopefully get closer to consensus on a 1.0.0?

@tetron  I think we need to talk about how to handle outputs.  My gut feeling is the output is that JSON that comes out of a cwltool run or the equivalent.  But I think TES has something to say about this.

@briandoconnor  re: 4) we made up that convention so unless there's a convenient coincidence, it probably doesn't match anything outside our task team.
Taking a quick look, Beacon uses URLs with a prefix of api like https://beacon-network.org/api/chromosomes https://beacon-network.org/#/developers/api/beacon-network#bn-chromosomes
The GA4GH (read server) API looks like it just hangs off of the root  curl --data '{}' --header 'Content-Type: application/json' \ http://localhost:8000/datasets/search | jq . https://ga4gh-reference-implementation.readthedocs.io/en/latest/demo.html#demo

post on DWG from Matthew:
Dear all,
Thank you for all the useful feedback on the Use Cases document last month. As the next step in the refactoring and standardization for the API we've produced a high level design principles document.
https://github.com/Ensembl/schemas/blob/master/doc/source/specifications.rst
We'd very much welcome feedback on this document as well. The next step after this is refined will be creating some demonstration endpoints for Ensembl data using these principles.
Thanks again for all the comments

Comments to discuss on the 20170508 group call:

general

renamed the base URL, do folks like this?  Should we use it across our APIs?
switching to snake_case for better consistency with TES
moved the state enum into here and a few others, eliminated dependency on TES to avoid funky naming conventions... really should move common elements to GA4GH schema repo and both TES and WES should use it.
avoiding how authentication/authorization works, how credentials for object stores are handed around... at least for now deferring  to the implementation

service-info

service info endpoint renamed to be consistent with TES
added system state count (e.g. how many workflows are in each state)
added listing of file protocols (s3, sftp, etc) supported, we're going to punt on handing credentials around.

GET /workflows

no filtering, may need to revisit later since I think this will be a burden on calling systems
made a note about not guarantees about mutability of listings during paging

POST /workflows

made a note about the lack of mechanism to hand off object store credentials for now.

GET /workflows/{workflow_id}

add log for the whole workflow and for individual steps (expecting the individual steps to be optional)
simplified logs and outputs to remove some of the details of TES (maybe debatable?)
output may include parameter and directory types in addition to output file URLs

From the call on 20170508:

defer to implementors on authentication/authorization and on passing credentials for object storage access (can use key-values to pass extra info).  Later we'll re-evaluate after we learn more
add a workflow_descriptor_url which can be used in place of workflow_descriptor in the POST

Seems like, based on the call, this is in a reasonable state and it should be merged and released as a 0.1


Perfomance of evaluating uhat

This performance problem becomes serious when evaluating the basis functions on a dense frequency mesh.
Julia:
using SparseIR
beta = 1.0
wmax = 1000.0
basis = FiniteTempBasis(fermion, beta, wmax, 1e-7)
nvec = 2 .* collect(1:10000) .+ 1
basis.uhat(nvec)
@time basis.uhat(nvec)

22.467327 seconds (72.36 M allocations: 22.350 GiB, 6.70% gc time)

Python:
from sparse_ir import FiniteTempBasis
import numpy as np
import time
beta = 1.0
wmax = 1000.0
basis = FiniteTempBasis("F", beta, wmax, 1e-7)
nvec = 2 * np.arange(10000) + 1
t1 = time.time()
basis.uhat(nvec)
time.time() - t1

0.49092721939086914

Vectorizing the function w.r.t frequencies may alleviate the problem.

I think so.  The whole thing is dominated by the Bessel function evaluations ... which are however independent of l

I think so. The whole thing is dominated by the Bessel function evaluations ... which are however independent of l

I did some profiling and found the same thing. I'm not clear on how to proceed though: SpecialFunctions.jl doesn't provide a sphericalbesselj(n, x::Vector) as far as I can tell, and this call is where most of the time is spent.

@Samuel3008  That's no problem ... let's discuss tomorrow!

Perfect, see you tomorrow!

https://github.com/SpM-lab/SparseIR.jl/commit/dfc8499535b9355275c98a92ebfc2899eb2ac06e
@shinaoka your example takes 1.2 seconds on my laptop now. Could you please try again on your machine and let me know?

Great work! It took only 0.8 seconds on my Macbook Pro M1.


Current sortie log

I know the current sortie node names are not very useful as has been stated in other issues, but I still would like to have a little list to show what nodes I've been through already. Example: A -> C -> F or something like that.
Here's an example of use. It could be useful if I'm doing some mindless task but need to know where I currently am before continuing. Right now I'm farming for subs while grinding my little ones in 1-5, and I want to stop before I get to the boss node. As boring as that is, I just go do something else, and only get back in between battles to advance to the next one. Most likely I'll have fogotten which node I was at, and since there's little indication about which node I was at on the results screen, I currently need to count on my fingers what was the last node I visited so I know when to turn back.

You can visit the strategy room in those cases that you forgot where you are and see how many nodes are there in the latest sortie?

I did not know you could do this, I just assumed that the strategy room sorties would only update after I was done with it

Here's some update I have on this, based on how I feel about it:

Accessing the strategy room to check this info is just not practical enough, and I ended up never using it.
The way I've been keeping track of where I am is by leaving the node prediction pop-up open for the whole duration of the sortie

So I still think this kind of info would be useful, but there are other solutions that could be used in place of it that might be more interesting. Maybe a button to re-open the last pop-up? Or a list to view the last handful of events (such as node predictions).

@dragonjet I think this issue has been resolved?

ah yes since we already have nodes on panel, and supply gauges too


Install strategy to cover perccli rpm resource file.

Currently the InstallStrategy for PercCLI require user to transform the rpm file to deb file manually. However this make the chekcsum logic not possible to implementation because the hash will be different every time.
We can change the process to allow user to upload rpm file, check sum, and  do the transform inside the charm.

This link is used in hw-health.
https://dl.dell.com/FOLDER07576010M/1/PERCCLI_7.1623.00_A11_Linux.tar.gz
It contains deb package that ubuntu can install it directly.
I think we use replace the link in hardware-observer to this one Or even this newer version
https://www.dell.com/support/home/zh-tw/drivers/driversdetails?driverid=36g6n
PERCCLI_7.1910.00_A12_Linux.tar.gz

The newest version can be found in here: https://www.dell.com/support/search/zh-tw#q="LINUX PERCCLI Utility"&sort=date descending&f:iuxType=[Drivers %26 Downloads]
Instead of point to specific version, maybe we can point to this search page and limit the version by https://github.com/canonical/prometheus-hardware-exporter/issues/31

Close by #77


cupy garbage collection is 100-time slower than GPU computing

I found an interesting phenomenon about cupy.

This may be a known issue, but it is just enough to surprise me that [a: cp.ndarray * 3.0 for i in range(2)] is 100-time slower than [a: cp.ndarray * 3.0 for i in range(1)].
(It should be noted that in %timeit loop, garbage collection is forced to off, so the loop made by %timeit does not slow down the computation.)
$ pip freeze | grep cupy
cupy==5.0.0b1

Python 3.5.2 (default, Nov 23 2017, 16:37:01)
Type 'copyright', 'credits' or 'license' for more information
IPython 6.4.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import cupy as cp

In [2]: a = cp.arange(8*16*100*100*100, dtype=cp.float32).reshape((8, 16, 100, 100, 100))

In [3]: %timeit -n100 -r10 c = [a * 2.0 for i in range(1)]
The slowest run took 5.26 times longer than the fastest. This could mean that an intermediate result is being cached.
110 us +- 89.9 us per loop (mean +- std. dev. of 10 runs, 100 loops each)

In [4]: %timeit -n100 -r10 c = [a * 2.0 for i in range(2)]
The slowest run took 40.09 times longer than the fastest. This could mean that an intermediate result is being cached.
2.66 ms +- 2.53 ms per loop (mean +- std. dev. of 10 runs, 100 loops each)

I think the above script does not correctly measure the performance. I tried the above one, and the first run of the statement with range(1) returns immediately, but successive ones with even range(1) takes much longer time. I guess these measurements are coming from asynchronous nature of the kernel execution, and the kernel launch queue size of CUDA (which is 1,024 as I remember). While the queue is not full, the statement a * 2.0 returns immediately. After the queue got full, it blocks until the least recent kernel finishes.
I don't think GC is related to this result. All the objects in these statements are released by reference counting, and CuPy uses a memory pool with which the above script only consumes twice of the size of a.

@beam2d
Thanks for your kind reply. I also confirmed the behavior you mentioned (rapid repeat of range(1) takes long time for the execution).
However, I don't understand well the meaning of "the kernel launch queue size of CUDA". Is this mean the queue of kernel (i.e. computational operation)?
I tried smaller size of a, in which case the range(1) and range(2) are both fast. Does it relate to the queue-size issue?

GPGPU compute is not like a CPU.  The CPU digests a sequence of machine code to perform actions and doesn't have a "queue" -- that is managed at the OS level.  On a CUDA GPU, you send the GPU a kernel (or information about a kernel) that you want it to execute, and what resource (shader data, GPGPU array, etc) you want to compute over.
CPUs have high performance execution units (cores) but not lots of them.  GPUs have low performance execution units (~= cuda core or AMD equivalent) but many many of them (on the order of a thousand).  The GPU has the task of scheduling all of them, which takes time.  This time means that while GPUs have breathtaking throughput, they have high latency.
To hide the latency, most GPU compute defaults to an asynchronous mode of operation, where when a kernel is dispatched, it is put on the GPU's queue, which it consumes as quickly as possible.  Scheduling a kernel takes 10-20 microseconds.  Executing it takes a somewhat indeterminate amount of time, depending on the current load of the GPU.
When you dispatch thousands of kernels in short succession, the queue will fill and I believe then your async code blocks until the queue can accept another kernel.  In most normal usages, you will never experience these queue overflows, since the queue is quite large (1024 kernels it seems, from above) and most GPU workloads involve crunching 1s to 10s of much heavier kernels.
If you need to repeatedly operate on the same array, you should try cp.fuse to make the GPU only have to launch one kernel.

Cupy has the performance characteristic where your operations, e.g. b = 2 * a return "immediately" after it is confirmed that the kernel is successfully scheduled.  This is usually around 20 microseconds.  Your code will only resync (i.e. block) when you try to pull data back off of the GPU.  Because you access a over and over and over again the GPU likely takes quite a long time to schedule the kernel in some instances, hence the 5x spread in time in the first timeit.
In the second, there is a 40x(!) spread in performance, with the fastest run taking something close to 10-20 microseconds (notice 2.66, the average, less 2.53, the stdev, leaves around 10us).  This indicates something "went wrong" on some of the attempts.
The first timeit, with 100 * 10 * 1 = 1000 kernels, likely has scheduling difficulty around use of the same array but otherwise does not fill the queue, so it remains speedy.  In the second timeit, you have 100 * 10 * 2 = 2000 kernels, which will overflow the queue, and block until it is partially emptied.  Since this is a nasty workload for a GPU, as you only access one array which prevents it from parallelizing across its SMs (~= core on a CPU, GPUs tend to have about 16 of them), you stack this slowdown with exceeding the number of pending tasks it can track.
For your (1):  Note that a CUDA kernel, at least from the perspective of the warp scheduler, is more or less anything that operates on an array.  It is true that you only have two unique kernels, but you still deploy one per computation.
For your (2): see above comment re: (1), you have deployed an immense number of kernels.  You can confirm that you are overflowing the queue by using, say, %timeit -n10 -r10 ... range(10)]  This queues the same amount of work for the GPU as the timeit with a 110us result.  Then just increase n10 to n20 to repeat the 2000 kernel case, and you should also observe similar timing.

I was able to install VC++ build tools through my VS install.  Note that I am on cupy 5.0.0b1 w/ CUDA 9.2, which has some changes with regard to kernel launch and scheduling latency.    @kmaehashi I do not mean to take your time on this issue, but cupy 5b compiles fine with MSVC++ 14 build tools and CUDA 9.2.  A note about this may be added to cupy, or cuda 9.2 added to the build matrix, etc.
For prep, I used your line a = cp.arange(8*16*100*100*100, dtype=cp.float32).reshape((8, 16, 100, 100, 100)) in a jupyter notebook.
Then I ran
>>> %timeit -n100 -r10 c = a * 2

The slowest run took 89.71 times longer than the fastest. This could mean that an intermediate result is being cached.
365 µs ± 916 µs per loop (mean ± std. dev. of 10 runs, 100 loops each)

Note that in this case, there are 1000 computations done.
I then increased n by a factor of 10:
>>> %timeit -n1000 -r10 c = a * 2

The slowest run took 289.74 times longer than the fastest. This could mean that an intermediate result is being cached.
10.6 ms ± 3.68 ms per loop (mean ± std. dev. of 10 runs, 1000 loops each)

What is perhaps more interesting is the system monitor during execution:

Windows' monitor spans 60 seconds.  The initial load on the GPU is the first case with with n100.  The second spike in GPU usage and CPU usage is the run with n1000.  You can see the CPU runs out of work long before the GPU, and in fact GPU computation runs for another minute or so.
If I poke around the total kernels << 1000 space,
>>> %timeit -n1 -r10 c = a * 2
The slowest run took 11.73 times longer than the fastest. This could mean that an intermediate result is being cached.
108 µs ± 116 µs per loop (mean ± std. dev. of 10 runs, 1 loop each)

>>> %timeit -n10 -r10 c = a * 2
The slowest run took 4.01 times longer than the fastest. This could mean that an intermediate result is being cached.
68.2 µs ± 29.2 µs per loop (mean ± std. dev. of 10 runs, 10 loops each)

>>> %timeit -n50 -r10 c = a * 2
49.2 µs ± 8.74 µs per loop (mean ± std. dev. of 10 runs, 50 loops each)

>>> %timeit -n90 -r10 c = a * 2
55.7 µs ± 23.5 µs per loop (mean ± std. dev. of 10 runs, 90 loops each)

Things are constant time as expected.  I will not bore you with more outputs, but even at -n100 -r10, things are fine with ~50us time.
At -n103 -r10, 1030 kernels, just above the magic number 1024, I get this:
>>> The slowest run took 118.50 times longer than the fastest. This could mean that an intermediate result is being cached.
3.42 ms ± 4.86 ms per loop (mean ± std. dev. of 10 runs, 103 loops each)

If you take the prior time, ~50us, multiply by ~120, then you would have 6ms which is inline with the 3.42+/- 4.86.  This does not happen with -n102 -r10.
Is there a reason you want to dispatch so many kernels in so short a time on a single GPU?  My CPU takes 122ms to do the same c = a * 2, even with MKL and all the other goodies helping it out as much as possible, so I understand the desire to use a GPU (1000x increase in performance, nice!) but I think in your real workload you probably can use cp.fuse or hand rewrite some of your steps as fused kernels and get rid of this problem.
There may also be a way with nVidia's tools to increase the kernel queue size, but you likely cannot achieve e.g. factor of ten increase there, as the warp scheduler is a very low power subsystem of the GPU.

@kmaehashi I do not mean to take your time on this issue, but cupy 5b compiles fine with MSVC++ 14 build tools and CUDA 9.2. A note about this may be added to cupy, or cuda 9.2 added to the build matrix, etc.

Thanks for the heads-up! We're going to release Windows wheels for CUDA 9.2 in the next release.

@fiarabbit Can I close this issue?

@brandondube
I totally understood why the queue was stacked by the 1024~ kernels.
Thank you very much for your kind reply and thoughtful explanation.
@kmaehashi
As I understood why my code is so slow, so this issue can be closed.
(I'll close this issue.)


A question about the final vocabulary

Dear,
Thanks for your great jobs.
I have a question about the final vocabulary.
Should I maintain a vocabulary for the corpus, or use the exact vocabulary privided by the BertTokenizer.from_pretrained('bert-base-uncased').vocab ?
In other words, should I use the exact vocab in BerTokensizer when finetuning Bert?
I found that there are only 27615 English words in the vocab of BertTokenizer.
Best regards.

Hi,
No external vocabulary is used.
You should use BERT vocabulary, and it is a subtoken vocabulary, so 27615 is enough.
Please refer to original BERT paper for details.

Dear,
Thanks for your reply.
 But I haven't figured it out yet.
How can I generate the original word in the summary?
Since BERT vocabulary is a subtoken vocabulary, is there another function for mapping the subtoken to the original vocabulary?
Sorry about the question. Maybe I am getting to a dead end on this question.
Any help would be appreciated.
Best regards.

The process to sub-tokenize a text is invertible
For extractive summarization, we only predict the indexes of selected sentences, then we select raw sentence with these indexes.


Assumptions about queue message for a server are broken in pods

Currently when running ManageIQ in pods from https://github.com/ManageIQ/manageiq-pods workers associated with the same "server" don't actually share a process space or filesystem as they are running in different containers.
This upends the assumption that process or filesystem level shared resources will be accessible to all workers on the same server. We use that assumption when we queue messages targeted for a specific server.
An example of this is the AnsibleRunnerWorkflow class. This job queues each of its states to avoid tying up a worker as the ansible playbook runs, but those queue messages are pinned to a specific server, not to a particular worker. This means that when running workers as separate containers we could clone the playbook repo on one container then try to run a playbook from that repo in a separate one. In the course of a single playbook run at least 3 messages are sent over the queue between cloning the repo and cleaning it up. It's very unlikely that all three messages get picked up by the same worker that queued the first one, so it's fair to consider this feature completely broken.
I'm sure there are other examples of this kind of behavior, but I don't know where they might be.
It seems like this might be an issue for smartstate if we are mounting something or trying to copy data to/from the local disk. Maybe V2V could also run into issues?
The best idea I have currently for this is to add a worker guid to queue messages if we need the message to be processed by a particular worker instance and use deliver_on to ensure the worker waits before consuming the message (think thread context switching). I don't like the idea of adding to the queue though so this may not be the best way forward.
From a pods-first viewpoint, I would like to spin up Job instances for this kind of work, then poll for the status of the job. It would be that separate pod's responsibility to take care of most of what we're doing in the workflow class. That way we don't have to worry about separate workers trying to monitor a process. I'm not sure what we would implement on an appliance that would fulfill the same use-case though.
I made a quick fix to the AnsibleRunnerWorkflow case in #19956, but this bug report is really about the larger issue so we can get input about other features that might be affected.
cc @Fryguy @chessbyte @NickLaMuro

From a pods-first viewpoint, I would like to spin up Job instances for this kind of work

So I think I like this approach in general, I am just wondering if it makes sense to proactively look at more ways to increase the boot speed for Job pods, specifically since around a 10-15 second penalty (roughly) to have this booted into a new process every time just to spin up the work required seems like it wouldn't be scalable long term.
Though, speaking specifically about using this approach specifically for EmbeddedAnsible, only a very small subset of our app is really required to kick off a ansible-runner job (though, I guess a bit more if we are talking about having to run the GitWorktree code and load a few other database records.
Anyway, just the first thing that popped into my head after reading that.

I think the assumption is that ansible-runner itself would be the thing launched as a Kube Job.
In my mind I think we need a MonitoredProcess abstraction which then can have different backends for pods (Kube Jobs) and appliances (Processes) and then callers can just go through that abstraction.

I think the assumption is that ansible-runner itself would be the thing launched as a Kube Job.

I feel like that is still missing a step, since you need the repo checked out on the same container/volume/whatever, and that isn't part of the CLI call to ansible-runner.
That said, I am unsure the capabilities of "Kube Jobs", and if they allow passing a file system of sorts along to the job to be executed, so my point might be moot if that is the case.

Yeah, at the very start of development of ansible_runner we asked for a container image that would handle checking out the repo and all of that, but I don't think that ever happened. I would imagine that's what we would build for this.


Store all market tags together in IPFS

Since the tags all need to be loaded as part of the initial data load, they can't be stored separately -- requires too many lookups, and loads too much unneeded information!

gonna close, on chain now


Y24-239 - As developers we would like to add detailed documentation for pooling / multiplexing to inform other projects and users

Describe the Housekeeping
As developers we would like to add detailed documentation for  pooling / multiplexing to inform other projects and users:

Create documentation in the traction repository (not confluence)
Add any process diagrams as images
Provide detailed documentation about how it works from a user perspective
Provide detailed documentation about how it works technically
Maybe create Architectural Decision Records (ADR)??

Blocking issues
Describe any other issues or tickets that may be blocking this change.
Additional context
Add any other context about the problem here.

Link to Lucid Chart https://lucid.app/lucidchart/343de76c-d3e5-4a26-92c4-7bc10d919fb8/edit?viewport_loc=-1507%2C-3314%2C6654%2C3511%2C0_0&invitationId=inv_13a8dd3b-ef1b-4f4e-b160-0dbf45a6031f


Cleanup the pyo3 setup.

I recommend maturin here ! Very nice tool for the building process.
The  GH action enable building for many platforms too

Yeah at this stage I care more about something that is super easy to understand and audit than having something more involved but we should certainly set it up soon ish.


make minikube tldr section easier to copy/paste

had a request from community to remove $ before  each command to make it easier for copy/pasting (and thus a bit more user-friendly). this seemed like a better approach after some thought.

This change is

trivial merge.

@alanmeadows small changes included. i'm going to create an issue about OVS in minikube and mark as a higher priority. if we can figure this out, then I would like to cherry pick the changes into the 0.1.0 tagged release.


Typo

Typo located at Build transformations logic in the data flow.
Eleven step, second paragraph, at the end of fourth line.
I assume the '200-' meant to be 2000 as it mentioned several lines above.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: f7c75ee8-d021-62fe-c7e3-53a6e729e1fc
Version Independent ID: 7ae45fa9-cb4f-dbc5-dcb6-4a4f4c040d28
Content: Transform data using a mapping data flow - Azure Data Factory
Content Source: articles/data-factory/tutorial-data-flow.md
Service: data-factory
GitHub Login: @kromerm
Microsoft Alias: makromer

@ReemCohen
Thanks for your feedback! We will investigate and update as appropriate.

@ReemCohen thank you! I made a pull request to fix this. https://github.com/MicrosoftDocs/azure-docs-pr/pull/157268


Implementing Web support

This pull request is linked to issue #44.
Here you can find, an implementation of your plugin for web using google libphonenumber js library using  https://github.com/natintosh/plugin_libphonenumber/tree/develop/libphonenumber_web as a starting point.
There is a deprecated call on MethodChannel constructor with Registrar.messenger, but I failed to get the plugin to work without using it.

Hi there! Sorry for the late reply..
I've tried the code in this PR and unfortunately, I found an issue right from the start by trying to parse an international number, without providing a region code. it works with the native sdks, but the web sdk just fails. The libphonenumber's JS implementation might not be ready yet.
Do you have the same results ?

Hello,
I don't remember if I had this results. I had made some tests and everything seemed to work fine.
I will try to reproduce your issue and probably fix it.

I find the problem and fix it (I was forcing regionCode to not be null when parsing international phone numbers).
Tell me if you still encountered this issue or if you find another.
If you can provide me any scenarios to test in order to correct the rest of my code, it will be nice.
I find two other issues :

carrierRegionCode() : I can't get the carrier RegionCode with libphonenumber but I can try to get current browser region if you want
getAllSupportedRegions() : I can't get  the display names from RegionCode in dart or with libphonenumber. I can try to use some js code (using Intl.DisplayNames) or a flutter package (locale_names)

Tell me which solution you prefer.

Hey all, is this planned on being merge soon?

Take a look on this @gabriel8belts https://github.com/flutter-form-builder-ecosystem/phone_number/issues/86


Some File Missing

Hi, some file apparently missing namely RuleTileEnhanced where can i get this file?. Thanks for this awesome libs.

It was an old file for the Tilemap. Now tilemap has a lot of updates and it might be outdated. You can safely remove the file. I'll update the repository due to tilemap changes


Inconsistent behavior when returning Redirect object from canActivate() on navigate vs deep link page load

@landofjoe commented on Wed Aug 31 2016
I'm submitting a bug report

Library Version:
Aurelia-cli 0.18.0

Please tell us about your environment:

Operating System:
Windows 10
Node Version:
5.9.0
NPM Version:
3.10.6
JSPM OR Webpack AND Version
n/a
Browser:
all
Language:
all

Current behavior:
When implementing canActivate() in a child-routed view model, returning a new Redirect object results in the route being interpreted differently based on whether the route was loaded fresh (i.e. user loads the browser page directly to the route) or whether the route is navigated to (i.e. using router.navigate()).
   //this works from fresh page load, but not if you navigate to here
   canActivate(){
     return new Redirect("children/child1");
   }

   //this works if you navigate here using `router.navigate()`, but not for fresh page loads.
   canActivate(){
     return new Redirect("child1");
   }

Run the Gist  (https://gist.host/run/1472695581644/index.html#/children/child1)
View the Gist  (https://gist.run/?id=0185b61c44d99f682ffbfb5479d3995d)
Expected/desired behavior:
There should be a consistent, reliable way to handle redirects from a child's canActivate() callback using the relative child route.  (i.e. without having to hard code the parent route)

What is the motivation / use case for changing the behavior?

Currently, it is difficult to cleanly handle redirects when users deep-link into part of an app.  Because it is also possible to deep-link from within the app after it is already loaded, and that seems to have a different set of requirements.

Is there any progress with that? I'm facing the same problem in my application, so I cannot use redirect but just reject / throwing errors in order to cancel the route activation.

+1


hugo: Fine tune code-tabs height

By default the code-tabs grow with the content (no (max) height)
If you set limitHeight to true the code-tabs will have a set width of 500px. This can be used for code-tabs with long content.
Improve css usage of grid so the rows will try to fit the height of the parent instead.

To test: /examples/basic/code-block/
For https://linear.app/usmedia/issue/CUE-321

Imported as https://review.gerrithub.io/c/cue-lang/cuelang.org/+/1168108

Closed via b5d3d7362a557c559702aba53314d1230d3cc34f


CTCP-4545: IE043/44 - Cross Check Page - Consignment item.Packaging - Please select a packaging type for the consignment item

Normal Mode:

Check Mode:

rebuild