Bug: Tests using rootBundle.loadString(...) hang

Affected Flutter versions:
2.5.3 as well as 2.8.1 (probably more, I only tested with those two versions)
Steps to reproduce:

flutter create myapp
cd myapp
create file test/demo_test.dart with the following Dart code:

import 'package:flutter/services.dart';
import 'package:flutter_test/flutter_test.dart';

void main() {
  testWidgets('load asset twice', (tester) async {
    final s1 = await rootBundle.loadString('AssetManifest.json');
    final s2 = await rootBundle.loadString('AssetManifest.json');
    expect(s2, same(s1));
  });

  testWidgets('load asset again in another test', (tester) async {
    await rootBundle.loadString('AssetManifest.json');
  });
}

flutter test test/demo_test.dart

Expected behaviour: The test run should succeed.
Actual behaviour: The test run does not finish.
Here is my short analysis of the issue:
rootBundle is a PlatformAssetBundle which extends CachingAssetBundle.
Therefore the first call to loadString will put a Future into rootBundle._stringCache.
When we try to await that Future in the same test, everything works fine (as the first test demonstrates).
But when we try to await that Future in another test, the test hangs.
I suppose this is because every test uses a different Zone.
A possible solution might be to clear the caches in rootBundle between tests,
but unfortunately I don't see a way to do this with the current public API.

@MichaelTamm
Can you try adding TestWidgetsFlutterBinding.ensureInitialized(); after testWidgets('load asset twice', (tester) async { and see if it helps ?
Moreover, also try using runAsync method like below and see if it works.
await tester.runAsync(() async {

Added TestWidgetsFlutterBinding.ensureInitialized(); does not help.

That's interesting: Using tester.runAsync in both tests works:
import 'package:flutter/services.dart';
import 'package:flutter_test/flutter_test.dart';

void main() {
  testWidgets('load asset twice', (tester) async {
    await tester.runAsync(() async {
      final s1 = await rootBundle.loadString('AssetManifest.json');
      final s2 = await rootBundle.loadString('AssetManifest.json');
      expect(s2, same(s1));
    });
  });

  testWidgets('load asset again in another test', (tester) async {
    await tester.runAsync(() async {
      await rootBundle.loadString('AssetManifest.json');
    });
  });
}

@MichaelTamm
Glad to know. Is it safe to close the issue as resolved then ?

@MichaelTamm Glad to know. Is it safe to close the issue as resolved then ?

I don't think so, unless your advice is, that I should use tester.runAsync() inside all my testWidgets() tests -- I just extracted the root cause of my problem for this GitHub issue. The actual issue is, that my tests hang because I'm using flutter_svg and if you don't specify a bundle when using SvgPicture the rootBundle is used. I still think this is a serious bug.

Thanks for your input. With your original code sample, I do observe the test getting hanged while trying to load it second time:

Using runAsync is an option which makes it work, but yes, it would need to be added in every testWidgets() which would probably not be ideal.

 stable, master flutter doctor -v
[[✓] Flutter (Channel stable, 2.8.1, on Mac OS X 10.15.4 19E2269 darwin-x64,
    locale en-GB)
    • Flutter version 2.8.1 at /Users/dhs/documents/fluttersdk/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 77d935af4d (13 hours ago), 2021-12-16 08:37:33 -0800
    • Engine revision 890a5fca2e
    • Dart version 2.15.1

[✓] Android toolchain - develop for Android devices (Android SDK version 30)
    • Android SDK at /Users/dhs/Library/Android/sdk
    • Platform android-30, build-tools 30.0.3
    • ANDROID_HOME = /Users/dhs/Library/Android/sdk
    • Java binary at: /Users/dhs/Library/Application Support/JetBrains/Toolbox/apps/AndroidStudio/ch-0/202.7486908/Android
      Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6915495)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.3, Build version 12C33
    • CocoaPods version 1.10.1

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 4.1)
    • Android Studio at /Users/dhs/Library/Application Support/JetBrains/Toolbox/apps/AndroidStudio/ch-0/202.7486908/Android
      Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build
      1.8.0_242-release-1644-b3-6915495)

[✓] VS Code (version 1.57.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.21.0

[✓] Connected device (4 available)
    • Darshan's iphone (mobile)  • 21150b119064aecc249dfcfe05e259197461ce23 •
      ios            • iOS 14.4.1 18D61
    • iPhone 12 Pro Max (mobile) • A5473606-0213-4FD8-BA16-553433949729     •
      ios            • com.apple.CoreSimulator.SimRuntime.iOS-14-3 (simulator)
    • macOS (desktop)            • macos                                    •
      darwin-x64     • Mac OS X 10.15.4 19E2269 darwin-x64
    • Chrome (web)               • chrome                                   •
      web-javascript • Google Chrome 93.0.4577.82

• No issues found!

[✓] Flutter (Channel master, 2.9.0-1.0.pre.266, on Mac OS X 10.15.4 19E2269
    darwin-x64, locale en-GB)
    • Flutter version 2.9.0-1.0.pre.266 at
      /Users/dhs/documents/fluttersdk/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision dd4db8738a (2 hours ago), 2022-01-06 18:36:24 -0800
    • Engine revision 849877954b
    • Dart version 2.16.0 (build 2.16.0-144.0.dev)
    • DevTools version 2.9.2
    • PATH =
      /usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/binpplicat
      ions/Software_Setup/flutter/bin:/Users/dhs/triagemagic/bin/:/Users/dhs/doc
      uments/fluttersdk/flutter/bin:/emulator:/tools:/tools/bin:/platform-tools:
      /Users/dhs/.pub-cache/bin

[✓] Android toolchain - develop for Android devices (Android SDK version 30)
    • Android SDK at /Users/dhs/Library/Android/sdk
    • Platform android-30, build-tools 30.0.3
    • ANDROID_HOME = /Users/dhs/Library/Android/sdk
    • Java binary at: /Users/dhs/Library/Application Support/JetBrains/Toolbox/apps/AndroidStudio/ch-0/202.7486908/Android
      Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6915495)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.5.1, Build version 12E507
    • CocoaPods version 1.10.1

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 4.1)
    • Android Studio at /Users/dhs/Library/Application Support/JetBrains/Toolbox/apps/AndroidStudio/ch-0/202.7486908/Android
      Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build
      1.8.0_242-release-1644-b3-6915495)

[✓] VS Code (version 1.57.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.21.0

[[✓] Connected device (4 available)
    • Darshan's iphone (mobile)  • 21150b119064aecc249dfcfe05e259197461ce23 •
      ios            • iOS 14.4.1 18D61
    • iPhone 12 Pro Max (mobile) • A5473606-0213-4FD8-BA16-553433949729     •
      ios            • com.apple.CoreSimulator.SimRuntime.iOS-14-3 (simulator)
    • macOS (desktop)            • macos                                    •
      darwin-x64     • Mac OS X 10.15.4 19E2269 darwin-x64
    • Chrome (web)               • chrome                                   •
      web-javascript • Google Chrome 93.0.4577.82

• No issues found!

@MichaelTamm
Actually, I'll back track my earlier labeling for a bit and can provide below updated snippet using test method directly instead of TestWidgets. See if it helps:
void main() {
  test("load", () async {
  //  testWidgets('load asset twice', (tester) async {
      TestWidgetsFlutterBinding.ensureInitialized();
      final s1 = await rootBundle.loadString('AssetManifest.json');
      final s2 = await rootBundle.loadString('AssetManifest.json');
      expect(s2, same(s1));
  //  });
  });
  test('load again', () async {
  //  testWidgets('load asset again in another test', (tester) async {
      TestWidgetsFlutterBinding.ensureInitialized();
      await rootBundle.loadString('AssetManifest.json');
 //   });
  });

}

I'm sorry, but using test instead of testWidgets does not help. I want to use testWidgets. And because I don't know beforehand which test are going to render a SvgPicture, it is easy to run into the situation, that a testWidgets test hangs.
I really think the problem must be solved in Flutter itself, so that the test for the issue description runs without problems. Currently testWidgets breaks test isolation!

Thanks for the update. I was trying to see if different solutions using runAsync and test would fit your need or not.
The original code sample behavior persists, ie, hangs as below:

 stable, master flutter doctor -v
[[✓] Flutter (Channel stable, 2.8.1, on Mac OS X 10.15.4 19E2269 darwin-x64,
    locale en-GB)
    • Flutter version 2.8.1 at /Users/dhs/documents/fluttersdk/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 77d935af4d (13 hours ago), 2021-12-16 08:37:33 -0800
    • Engine revision 890a5fca2e
    • Dart version 2.15.1

[✓] Android toolchain - develop for Android devices (Android SDK version 30)
    • Android SDK at /Users/dhs/Library/Android/sdk
    • Platform android-30, build-tools 30.0.3
    • ANDROID_HOME = /Users/dhs/Library/Android/sdk
    • Java binary at: /Users/dhs/Library/Application Support/JetBrains/Toolbox/apps/AndroidStudio/ch-0/202.7486908/Android
      Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6915495)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.3, Build version 12C33
    • CocoaPods version 1.10.1

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 4.1)
    • Android Studio at /Users/dhs/Library/Application Support/JetBrains/Toolbox/apps/AndroidStudio/ch-0/202.7486908/Android
      Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build
      1.8.0_242-release-1644-b3-6915495)

[✓] VS Code (version 1.57.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.21.0

[✓] Connected device (4 available)
    • Darshan's iphone (mobile)  • 21150b119064aecc249dfcfe05e259197461ce23 •
      ios            • iOS 14.4.1 18D61
    • iPhone 12 Pro Max (mobile) • A5473606-0213-4FD8-BA16-553433949729     •
      ios            • com.apple.CoreSimulator.SimRuntime.iOS-14-3 (simulator)
    • macOS (desktop)            • macos                                    •
      darwin-x64     • Mac OS X 10.15.4 19E2269 darwin-x64
    • Chrome (web)               • chrome                                   •
      web-javascript • Google Chrome 93.0.4577.82

• No issues found!

[✓] Flutter (Channel master, 2.9.0-1.0.pre.295, on Mac OS X 10.15.4 19E2269
    darwin-x64, locale en-GB)
    • Flutter version 2.9.0-1.0.pre.295 at
      /Users/dhs/documents/fluttersdk/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 83bb08fb37 (14 hours ago), 2022-01-09 10:05:18 -0500
    • Engine revision 94786ae707
    • Dart version 2.16.0 (build 2.16.0-144.0.dev)
    • DevTools version 2.9.2
    • PATH =
      /usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/binpplicat
      ions/Software_Setup/flutter/bin:/Users/dhs/triagemagic/bin/:/Users/dhs/doc
      uments/fluttersdk/flutter/bin:/emulator:/tools:/tools/bin:/platform-tools:
      /Users/dhs/.pub-cache/bin

[✓] Android toolchain - develop for Android devices (Android SDK version 30)
    • Android SDK at /Users/dhs/Library/Android/sdk
    • Platform android-30, build-tools 30.0.3
    • ANDROID_HOME = /Users/dhs/Library/Android/sdk
    • Java binary at: /Users/dhs/Library/Application Support/JetBrains/Toolbox/apps/AndroidStudio/ch-0/202.7486908/Android
      Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6915495)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.5.1, Build version 12E507
    • CocoaPods version 1.10.1

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 4.1)
    • Android Studio at /Users/dhs/Library/Application Support/JetBrains/Toolbox/apps/AndroidStudio/ch-0/202.7486908/Android
      Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build
      1.8.0_242-release-1644-b3-6915495)

[✓] VS Code (version 1.57.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.21.0

[[✓] Connected device (4 available)
    • Darshan's iphone (mobile)  • 21150b119064aecc249dfcfe05e259197461ce23 •
      ios            • iOS 14.4.1 18D61
    • iPhone 12 Pro Max (mobile) • A5473606-0213-4FD8-BA16-553433949729     •
      ios            • com.apple.CoreSimulator.SimRuntime.iOS-14-3 (simulator)
    • macOS (desktop)            • macos                                    •
      darwin-x64     • Mac OS X 10.15.4 19E2269 darwin-x64
    • Chrome (web)               • chrome                                   •
      web-javascript • Google Chrome 93.0.4577.82

• No issues found!

/cc @chunhtai

I was able to work around the issue by clearing the bundle's cache between tests.
tearDown(rootBundle.clear);

You can also do the following:
rootBundle.loadString('path', cache: false);


don't have a problem I just have a question

Does this install DaVinci resolve? because if I install it from their website then it fails to launch, so I have wondering if packaging it as a flatpack installs it.

When you install the flatpak package after building it, DaVinci Resolve is installed.

Ok


⚠ ClamAV has degraded performance

In 892623d, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 285 ms

Resolved: ClamAV performance has improved in 9e68579 after 8 minutes.


feat: toEllipsis / truncate / fit

Please merge new toEllipsis feature.

It's just a comment to remove wontfix label

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. May the force be with you!

:eyes:

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. May the force be with you!

:eyes:

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. May the force be with you!

:eyes:

@vorillaz, Could you merge this feature please?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. May the force be with you!

:eyes:

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. May the force be with you!

👀

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. May the force be with you!

:eyes:

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. May the force be with you!

:eyes:

👀

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. May the force be with you!

:eyes:

Oops, sorry for that merging this one!


Scale smoothing effect

Hello, can I improve the zoom experience? Make the scaling smooth. Can refer to Google Earth's gradient zoom effect!
I tried“ viewer.scene.screenSpaceCameraController . inertiazoom = 1; ", but it doesn't work

For questions like this, please post it on our community forum. We have team members and others who regularly check the forum so you will receive a quicker response! Please include screenshots/videos/gifs if possible to demonstrate what you are looking for.


APEXCORE-222 Purging the stale data present in the bufferserver, dire…

…ctly from the streaming container.

All the review comments has been addressed.

@sandeshh Please rebase against latest master and I'll merge it.


Allow running tests from any directory without being part of the std lib

The previous solution (69f89d0) requires that runner.nu be already imported previously.

Nice one. Thank you.


Python TFlite not using multiple threads for float inference with xnnpack

Please make sure that this is an issue related to performance of TensorFlow.
As per our
GitHub Policy,
we only address code/doc bugs, performance issues, feature requests and
build/installation issues on GitHub. tag:performance_template
System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Not really.
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Raspberry Pi 64 bit.
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: Pi Zero 2 W
TensorFlow installed from (source or binary): Source.
TensorFlow version (use command below): 365a3b68471f158defc3aea79a25fdaa56be4ac8 (i.e. few days old)
Python version: 3.7
Bazel version (if compiling from source): cmake 3.16.3
GCC/Compiler version (if compiling from source): gcc (Debian 8.3.0-6) 8.3.0
CUDA/cuDNN version: NA
GPU model and memory: NA

Describe the current behavior
I've trained MobileNet v3 Small using Tensorflow Object Detection API - models here. When I infer with ./linux_aarch64_benchmark_model --graph=models/mobilenetv3_224x224_FLOAT.tflite --num_threads=4 --use_xnnpack=true I'm happy - around 25ms inference. Running with 1 thread is 57ms, 2 is 35ms, and 3 is 27ms, which is all expected. However, when I run the same tests with the simple script below, I'm getting 56-57ms seconds for any number of threads. Given this is similar to the single threaded performances from the benchmark tool, it'd imply that only a single thread is being used in python. Note that I am seeing INFO: Created TensorFlow Lite XNNPACK delegate for CPU. which seems to imply XNNPACK is being used (though I've also seen that log in the benchmark before it later on complaining and not using XNNPACK - I don't see that here, but I can't seem to get more logging.)
Lastly, it's worth noting that if I use a quantized model in Python, it behaves as expected (i.e. faster with more threads, and matches the benchmark tool). Also, while I compiled the pip package with the provided scripts, I tested the wheel from here, and had the same issue which is at least another data point.
Describe the expected behavior
I would expect to see similar behaviour with fp32 inference as with quantized i.e. increased performance with more threads. And I would expect it to match the benchmark tool.
Standalone code to reproduce the issue
import time

import numpy as np
from tflite_runtime.interpreter import Interpreter

interpreter = Interpreter("models/mobilenetv3_224x224_FLOAT.tflite", num_threads=4)
interpreter.allocate_tensors()
for _ in range(10):
    t0 = time.time()
    interpreter.invoke()
    print(int((time.time() - t0) * 1000))

Other info / logs NA

Thx for reporting this issue! We will look into this and get this fixed asap.

@kodonnell, I noticed that you performed the experiments on Raspberry Pi 64 bit. Could you re-compile your pip with an additional bazel flag "--define=tflite_with_xnnpack=true" which explicitly enables the xnnpack-by-default feature, and then re-run your test program?
Based on the latest bazel BUILD rule, I'm afraid xnnpack-by-default isn't enabled on Raspberry Pi 64 bit. There was an earlier attempt to enable this feature on more platforms, but it's rolled back due to some issues that were found later internally.

Thanks @multiverse-tf - as above, I compiled with cmake, which enables xnnpack by default (and, as above, there’s strong evidence to support xnnpack being used). Note I just built aarch64 I.e. independent of raspberry pi.

@kodonnell, thanks again for clarification! I failed to reproduce the issue on my side with the Python label image example on a Linux machine (where I explicitly turned on XNNPACK-by-default feature).
To help debug the issue further (as you did compilation on your own :-)), could you change the tensorflow  lite source code as follows to add more logs:

Change TFLITE_LOG here in tensorflow/lite/core/subgraph.cc to TFLITE_LOG_PROD.
Log the context_->recommended_num_threads here and the resulted delegate_ptr value (i.e. simply logging delegate_ptr.get() value itself) by using TFLITE_LOG_PROD?

Afterwards, pls recompile tflite, re-run your test program and share the log? Many thanks!

still can't reproduce the error on the latest tip, running XNNPACK on 1 - 4 threads results in: 39.441ms, 19.834ms, 14.066ms and 11.099ms respectively, matching expected behaviour


Callback is not executed when request fails due to lower level network issue

Hi,
I have a pretty simplistic use of this library:
logRequestInfo(method, url);
request(method, url, {json: json, headers: headers}).then(function (response) {
    logResponseInfo(response);
    done();
});

When a request fails, but the API returns a response with an error code e.g. 404, then the response gets printed.
PUT http://localhost:9000/api/endpoint
>> 404

This is expected.
The problem is when the underlying API is down. The aforementioned code breaks completely and no response gets ever printed.
PUT http://localhost:9000/api/endpoint

Processing is stopped.
How can I handle errors in this case?
Thanks,
Marcin

The call to request returns a promise.  At this point we consider any response from the server to be "success" regardless of status code.  The promise gets rejected if something more core goes wrong though, so you can handle your errors like:
logRequestInfo(method, url);
// N.B. calling `.done` instead of `.then` ensures that errors are never silenced
request(method, url, {json: json, headers: headers}).done(function (response) {
    logResponseInfo(response);
    done();
}, function (err) {
    logError(err);
    done();
});

You can also transform non-200 status codes into errors by calling .getBody:
logRequestInfo(method, url);
request(method, url, {json: json, headers: headers}).getBody('utf8').done(function (body) {
    logBody(body);
    done();
}, function (err) {
    logError(err);
    done();
});

.then is used for transforming the value of a promise, so it's a great place to parse the body (e.g. as JSON).  This lets you handle all errors in the same place, rather than having to keep adding try/catch blocks:
logRequestInfo(method, url);
request(method, url, {json: json, headers: headers}).getBody('utf8').then(function (rawBody) {
  return JSON.parse(rawBody);
}).done(function (body) {
    logBody(body);
    done();
}, function (err) {
    // error here could be invalid status code, failed to parse as JSON or some underlying network error
    logError(err);
    done();
});

Thank you for your reply. Somehow I missed the fact I can provide an error callback. Thanks again.


[Feature/multi_tenancy] Implement client.bulk in SDK Client

Description
Adds an SDKClient equivalent to client.bulk().
Summary of changes (helpful for reviewers):

Added a new DataObjectRequest parent class. I originally started with a tagging interface just to help method signatures, but realized that four classes duplicated the same code for the index, id, and tenantId fields, so refactored those (and the associated builder code) into the superclass. Existing unit tests cover this change. https://github.com/opensearch-project/ml-commons/pull/3192/commits/caeea219cef12a70d344f2639f17f13ac9a62956
Added a new DataObjectResponse parent class similar to above, refactoring the id and parser fields from the subclass. Existing unit tests cover this change. https://github.com/opensearch-project/ml-commons/pull/3192/commits/445936bcfa23410778cc024a485b69b307a44c62 (part 1)
Added new BulkDataObjectRequest and BulkDataObjectResponse classes and unit tests.  Added a failed field to the response superclass to track failures/errors. https://github.com/opensearch-project/ml-commons/pull/3192/commits/445936bcfa23410778cc024a485b69b307a44c62 (part 2)
Added the interface and stubbed implementations for the SdkClient .bulk() method, and unit tests https://github.com/opensearch-project/ml-commons/pull/3192/commits/22dccf4af0c41f07625ffe6857fd17df80b0bf79
Implemented the new interface method in all 3 clients.  The local and remote implementations use the OpenSearch bulk request natively.  The DynamoDB implementation simply iterates the bulk request and calls existing code for the individual requests; this is not the most efficient but necessary to preserve ordering.  If all the bulk requests are put and delete, it's possible to optimize this with DDB BulkWrite, which is a good future optimization; however since our primary use case uses update I'm deferring that complexity. https://github.com/opensearch-project/ml-commons/pull/3192/commits/ee30d210818a84b89027860e5c3a0ac7762a950b

In addition there are several whitespace / import reordering changes due to spotless not having been run on the common module.
Will be handled integ tests in #2818
Check List

[x] New functionality includes testing.
[x] Commits are signed per the DCO using --signoff.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

apply spotless

Will fix spotless in next commit.  Apparently my IDE import ordering is different than spotless, needed on common, not needed on plugin. See #3119 😁

@dhrubo-os @arjunkumargiri This is ready for (re-) review.


Fix not defined env APP_URL on new setups

1. Why is this change necessary?
To prevent the error Environment variable not found: "APP_URL". on new setups.
2. What does this change do, exactly?
Sets a default value of http://localhost for APP_URL
3. Describe each step to reproduce the issue or behaviour.
Run bin/console system:setup
4. Please link to the relevant issues (if any).
shopware/production#101
5. Checklist

[x] I have written tests and verified that they fail without my change
[x] I have squashed any insignificant commits
[x] I have created a changelog file with all necessary information about my changes
[x] I have written or adjusted the documentation according to my changes
[x] This change has comments for package types, values, functions, and non-obvious lines of code
[x] I have read the contribution requirements and fulfil them.

Could you change it here?

I created a new PR #1859 regarding just the change of the default value to http://localhost.


DocsAdd Job and Job tasks docs

Proposed changes
Issue Number: close #xxx

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall

run buildall


Reported traits and studies should be assigned to studies in the same input file

In the current version of depocuration, curators need to assign reported traits and EFO traits to studies in separate files. Curators should be able to assign both in the same file to make the process more efficient. For maximum flexibility the possibility to enter only reported traits or only EFO traits should be retained.

@eks-ebi tagging you for reference

closing as duplicate of #713


create tool to check performance of a given tool as a function of file length

can be generated using yunitator but the hope is that it can be used with any other tool

Here are measurements from running several tools, as a function of file length. This isn't a general tool for producing such measurements, but at least serves as example output from real measurements:
https://docs.google.com/spreadsheets/d/1SYsJoZTW4EnOP86xpIp9WhCn4e1RdzICFBAN45G1zDg/edit#gid=939480220

closing because it's not a priority


nocomma option generates spurious warning

Given the code:
/* jshint undef: false, nocomma: true */
x(1, 2);

JSHint generates the following warning:
tmp.js: line 2, col 4, Unexpected use of a comma operator.

1 error

Confirmed against JSHint version 2.6.0 and latest on master (2dc713a2eac387feb62debf5ff4f0c1fa467244a)

This bug is addressed by gh-2050


highlight on bulk edge where filtering expression

close #824

sorry, the rebase make things a mess, I will fix it via the command line.

already added this note to the English doc. @izhuxiaoqing

@abby-cyber we should cherry pick this to master and other newer branches :-P

Already did :)


Prevent hovering on existing reactions from shifting buttons down

Prior to this PR, in some cases, when highlighting an existing reaction,
the add reaction button would shift down to the right.
This would happen in circumstances where not all 8 reactions
were present on a comment.  This was due to buttons with no reaction
count being hidden with display: none.
Then Primer's button highlight CSS would add a border on the right
of the highlighted element and remove the left border on the following element.
This ensured that the row of buttons would remain a constant width.
However when some buttons were hidden, a border was added to the highlighted
element without the removal of a border from another element (since it was hidden).
This resulted in the row of buttons gaining a pixel, shifting everything down.
Resolved by removing Primer's border expanding behavior.
Before

After

This is a little tricky because primer button groups show/hide left and right borders based on focus/active/hover:
https://github.com/primer/css/blob/master/src/buttons/button-group.scss#L35-L45
Forcing border visibility via !important will cause some problems with keyboard navigation:

maybe switch to outline instead?


DRC violations in GF018green_ipio_5p0c_75_3lm

DRC violations in GF018green_ipio_5p0c_75_3lm vs. documented rule in the design manual
DF.1a

DF.14

@atorkmabrains please see comment from @RTimothyEdwards

@RTimothyEdwards Great. If that's the case that's fine, but keep in mind that we are testing each cell on it's own. And yes, as you said some of them might be used in that way.

@RTimothyEdwards and @mkkassem No longer needed. Will close the issue.


ValidationUtil can now validate response_type

Some response types are only valid for some client types. This
encapsulates the validation.

Coverage remained the same at 100.0% when pulling 1f937389ef51f056e35ed3d1238979a07eedaf5f on feature/validate_response_type into b5a9d5103f66ca54ab9340a82c2ad667e5efe924 on develop.


Unparsable Replays (fail on unknown data type in TrackerEventStructure)

These replays:
https://drive.google.com/drive/folders/19xOgMhJtKwBq9vRLTziRNCvGkwi-3Onj?usp=sharing
Fail on this line:
https://github.com/barrett777/Heroes.ReplayParser/blob/83638ae78fae439a636d28fb982649065dc0b5c1/Heroes.ReplayParser/MPQFiles/ReplayTrackerEvents.cs#L180

Hmm okay I'll look into it
Blizzard hasn't made any changes to their replay parser, so I don't think it's new content.  I'll see if maybe it's a bug in how I'm parsing

uploader_d0f3ccc6-f453-43e7-9bd2-534b72dbde37_109 - corrupted/desync
web_11d7ea9c-48f0-417e-ab9f-688d4d2dce98_9 - ok, has computer players
web_11d7ea9c-48f0-417e-ab9f-688d4d2dce98_11 - ok, has computer players
web_11d7ea9c-48f0-417e-ab9f-688d4d2dce98_13 - ok, has computer players
web_11d7ea9c-48f0-417e-ab9f-688d4d2dce98_10 - ok, has computer players
uploader_d0f3ccc6-f453-43e7-9bd2-534b72dbde37_112 - corrupted?, but replay plays fine in-game.

Thanks for looking into it - so it's either corrupt files or ai players? Out of curiosity why should AI players crash the parsing?

AI games are not supported by the parser.

I think there is something else wrong here, the replay web_11d7ea9c-48f0-417e-ab9f-688d4d2dce98_10 crashes at https://github.com/barrett777/Heroes.ReplayParser/blob/83638ae78fae439a636d28fb982649065dc0b5c1/Heroes.ReplayParser/MPQFiles/ReplayServerBattlelobby.cs#L232
Usually when an AI player is found the parser returns a result of ReplayParseResult.ComputerPlayerFound

Yeah, something could be different than normal in that replay, however I haven't spent much effort on AI replays.  If this is important to you, you could dig into it and see if you can get the parser to not blow up here.
I'm guessing the line above that probably has a bot name that would be easy to identify: https://github.com/barrett777/Heroes.ReplayParser/blob/83638ae78fae439a636d28fb982649065dc0b5c1/Heroes.ReplayParser/MPQFiles/ReplayServerBattlelobby.cs#L229
You could check if that's a computer name, and if so, don't throw the "Couldn't find BattleTag" exception and just keep parsing.  Maybe it will make it further along and be able to return ReplayParseResult.ComputerPlayerFound like normal

Actually, this line is key:
https://github.com/barrett777/Heroes.ReplayParser/blob/83638ae78fae439a636d28fb982649065dc0b5c1/Heroes.ReplayParser/MPQFiles/ReplayServerBattlelobby.cs#L227
It actually indicates if a battle tag exists, but that indication is ignored and a battle tag is read anyway. I'll look into it

For AI games, the extended battle lobby parsing doesn't work yet due to extra bytes. Maybe I actually can get it to work.

You want to reopen this issue until this is resolved (if ever)?

I uploaded 53 more replays to that folder and categorized them by the exception they throw - for your reference.

Thanks for providing more info.  I'll reopen so this stays visible in case others come across this.

All the replays with exceptions in the ReplayTrackerEvents are corrupted. The MpqArchive exceptions are even worse as the mpq archive is corrupted.

Thanks for fixing this @koliva8245 in version 1.2.12


[ISSUE]: Icons acting weirdly

Describe the bug
Hello 👋, just added this plugin and want to try it out.

But the icons are acting weirdly?
why the icon has a bg?
Maybe it's a treesitter issue?

Desktop Info
OS: Windows
Terminal: Windows terminal
Neovim Version: 0.6.1
(Sorry for bad english, im not too good at it)

Just found out that plenary is the one who's causing the issue...

when removing the requires = "nvim-lua/plenary.nvim" still does the same thing.
How can i fix this?

same issue here.
Desktop Info
OS: Linux
Terminal: Kitty
Neovim Version: 0.7.0


az from PyPI connectedk8s connect error kube-config file

Describe the bug
I am working on the creation of a Kubernetes cluster on Azure ARC using Raspberry Pi with Linux Raspbian operating system. As reported in the issue 20476, the mode of installation of azure-cli on this architecture is through PyPI and virtual environment. azure-cli is successfully installed but when I try to connect to the cluster I get the error:
Problem loading the kubeconfig file.Invalid kube-config file. No configuration found.
Related command
azure-cli-env/bin/az connectedk8s connect --name {} --resource-group {}
Expected behavior
Should be able to connect to K8s cluster.
Additional context
Probably the error derives from the different mode of installation and the virtual environment azure-cli-env.
Any suggestions to fix pointing to configuration kube-config file?

route to CXP team

Adding Service team to take a look into this issue.
@akashkeshari Could you please look into this ask and provide an update ?


cli docs say that 'method' param is optional in routes.js, but it doesn't seem to be

The cli getting started guide says that 'method' is optional, but for a very basic test, it didn't seem to be.  Does it default to POST?  This seems like an odd default.

Making that options was just merged to master yesterday, and may not have been published to npm yet. are you using master or a published version?

This went out with 0.2.0 yesterday.  Thanks for reporting @kcjonson!


[stable/fluentd-elasticsearch] Add PodSecurityPolicy support

Signed-off-by: Ludovic Cavajani<EMAIL_ADDRESS>
What this PR does / why we need it:
Allow one to enable pod security policy when deploying fluentd.
The PSP allows the privilege escalation and specify the hostPath allowed.
This is a redo of #8504
Which issue this PR fixes
(optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged)

fixes #

Special notes for your reviewer:
Regarding extraVolumeMounts, I assume this is only for secret or configMap, am I right ?
Checklist
[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

[x] DCO signed
[x] Chart Version bumped
[x] Variables are documented in the README.md

/assign @seanknox

/assign @lachie83 @sameersbn

/assign @cpanato @monotek
Can you have a look ?

/approve

/lgtm

/lgtm

@lcavajani
Please resolve merge conflict.

/lgtm


Fixing NumPy bug

Changed the NumPy version to 1.18.5 which is compatible with TensorFlow 2.3.0

give @thesauravkarmakar 5 points


i18n: zhCN Translations

Translation Status for zhCN.lua
Translation progress: 98.5%
Missing translations: 2

Missing Keys and their enUS values

Missing Key
enUS Value

Disable Row Highlight
Disable Row Highlight

DisableRowHighlightDesc
If checked, don't highlight a row when you loot the same item again and the quanity is updated.

You can even make changes for this file and open a PR directly in your browser

Please provide one or more of these values in a Pull Request or a Comment on this issue
L["Disable Row Highlight"] = ""
L["DisableRowHighlightDesc"] = ""

L["Disable Row Highlight"] = "禁用行高亮"
L["DisableRowHighlightDesc"] = "如果选中，当你再次拾取相同的物品并且数量更新时，不再高亮显示该行。"


Make sure RealApolloCall doesn't hold reference to user callback after complete or cancel

We need to nullify reference to user provided callback in com.apollographql.apollo.internal.RealApolloCall#enqueue that if call has been finished or canceled we don't hold the strong reference to user callback any longer.

I have to move around some of the cancel/dispose logic in https://github.com/apollographql/apollo-android/pull/556 because of the added Callback#onComplete for multi-response fetchers. I can try to fix in that PR.

@BenSchwab Yeah I was waiting on your PR to go first anyway and then fix this

So I will reassign this to you then


No backup after upgrade to v8.0

After upgrading to version 8.0 the backup doesnt work any more, it only gives you a backup with a sql file in it with the following details.
-- MySQL dump 10.13  Distrib 8.0.32, for Linux (x86_64)
-- Host: <IP_ADDRESS>    Database: pvault

-- Server version	5.5.5-10.5.21-MariaDB-1:10.5.21+maria~ubu2004
/*!40101 SET @OLD_CHARACTER_SET_CLIENT=@@CHARACTER_SET_CLIENT /;
/!40101 SET @OLD_CHARACTER_SET_RESULTS=@@CHARACTER_SET_RESULTS /;
/!40101 SET @OLD_COLLATION_CONNECTION=@@COLLATION_CONNECTION /;
/!50503 SET NAMES utf8mb4 /;
/!40103 SET @OLD_TIME_ZONE=@@TIME_ZONE /;
/!40103 SET TIME_ZONE='+00:00' /;
/!40014 SET @OLD_UNIQUE_CHECKS=@@UNIQUE_CHECKS, UNIQUE_CHECKS=0 /;
/!40014 SET @OLD_FOREIGN_KEY_CHECKS=@@FOREIGN_KEY_CHECKS, FOREIGN_KEY_CHECKS=0 /;
/!40101 SET @OLD_SQL_MODE=@@SQL_MODE, SQL_MODE='NO_AUTO_VALUE_ON_ZERO' /;
/!40111 SET @OLD_SQL_NOTES=@@SQL_NOTES, SQL_NOTES=0 */;
--
-- Table structure for table IFRACategories
DROP TABLE IF EXISTS IFRACategories;
/*!40101 SET @saved_cs_client     = @@character_set_client /;
/!50503 SET character_set_client = utf8mb4 /;
CREATE TABLE IFRACategories (
id int(11) NOT NULL AUTO_INCREMENT,
name varchar(255) NOT NULL,
description varchar(255) DEFAULT NULL,
type int(11) NOT NULL COMMENT '1=Standard, 2=Custom',
PRIMARY KEY (id)
) ENGINE=InnoDB AUTO_INCREMENT=19 DEFAULT CHARSET=utf8 COLLATE=utf8_general_ci;
/!40101 SET character_set_client = @saved_cs_client */;
--
-- Dumping data for table IFRACategories
LOCK TABLES IFRACategories WRITE;
/*!40000 ALTER TABLE IFRACategories DISABLE KEYS /;
INSERT INTO IFRACategories VALUES (1,'1','Toys, Lip products of all types (solid and liquid lipsticks, balms, clear or colored, etc).',1),(2,'2','Deodorant and antiperspirant products of all types (sprays, roll-on, stick, under-arm and body, etc).',1),(3,'3','Hydroalcoholic products (EdT and fine fragrance range) applied to recently shaved skin (3A and 3B), Eye products of all types including eye cream, mens facial creams and balms (3C), and tampons (3D).',1),(4,'4','Hydroalcoholic products (including scent strips) (EdT and fine fragrance range) applied to unshaved skin, ingredients of perfume kits, hair styling aids and hair sprays, hair deodorants, body creams, foot care products.',1),(5,'5A','Body lotion products applied to the body using the hands (palms), primarily leave-on',1),(6,'5B','Face moisturizer products applied to the face using the hands (palms), primarily leave-on',1),(7,'5C','Hand cream products applied to the hands using the hands (palms), primarily leave-on',1),(8,'5D','Baby Creams, baby Oils and baby talc',1),(9,'6','Products with oral and lip exposure',1),(10,'7A','Rinse-off products applied to the hair with some hand contact',1),(11,'7B','Leave-on products applied to the hair with some hand contact',1),(12,'8','Products with significant anogenital exposure',1),(13,'9','Products with body and hand exposure, primarily rinse off',1),(14,'10A','Household care excluding aerosol products (excluding aerosol/spray products)',1),(15,'10B','Household aerosol/spray products',1),(16,'11A','Products with intended skin contact but minimal transfer of fragrance to skin from inert substrate without UV exposure',1),(17,'11B','Products with intended skin contact but minimal transfer of fragrance to skin from inert substrate with potential UV exposure',1),(18,'12','Products not intended for direct skin contact, minimal or insignificant transfer to skin',1);
/!40000 ALTER TABLE IFRACategories ENABLE KEYS */;
UNLOCK TABLES;

Only way is to make a backup of the db via phpmyadmin or equivalent

Not a bug - please refer to docker documentation in KB
https://www.jbparfum.com/knowledge-base


Not getting any result for a given stock/date request

I'm not getting any result from this request on dates March 31, 2015 onwards
GET http://phisix-api.appspot.com/stocks/{symbol}.{yyyy-MM-dd}.{json|xml}
With result
http://1.phisix-api.appspot.com/stocks/MEG.2015-03-30.json
Without result
http://1.phisix-api.appspot.com/stocks/MEG.2015-03-31.json
http://1.phisix-api.appspot.com/stocks/MEG.2015-04-15.json
http://1.phisix-api.appspot.com/stocks/MEG.2015-05-15.json
http://1.phisix-api.appspot.com/stocks/MEG.2015-10-26.json

Me too. Can't see any content for 2016.
http://1.phisix-api.appspot.com/stocks/BDO.2016-01-19.json

FIXED: cron was disabled. you'll start seeing history starting from 2016-03-23
sample:
http://phisix-api.appspot.com/stocks/BDO.2016-03-23.json
i'll put in a job to back track lost history from 2016, and if possible 2015.

this issue persists again


⚠ AsianCinema has degraded performance

In fc5ba12, AsianCinema ($ACM) experienced degraded performance:

HTTP code: 525
Response time: 20301 ms

Resolved: AsianCinema performance has improved in d52846a after 5 minutes.


[DSR] Titan Gaols-esque extended text popup for DSR Skyward Leaps and Dragon's Rage stack (Strength of the Ward)

Reference
https://github.com/quisquous/cactbot/blob/main/ui/raidboss/data/04-sb/ultimate/ultima_weapon_ultimate.ts#L296
Summary
Have Cactbot print out sorted names for Skyward Leap and Dragon's Rage stack simalar to Titan Gaols.
Ex. Player 4, Player 1, Player 8. It's a pseudo Titan Gaol-esque text.
If all 8 players have cactbot they can agree on a priority of where Skyward Leaps go West/South/East relative to Thordan and which persons in the stack go to which tower Left/Middle/Right.
If all players don't have cactbot, proceed as normal but the extra information of who got which mechanic is helpful.

I think printing this out for everybody is a little bit much, but probably the skyward leap and the stack group could have a list of who was in their group with them, e.g. "Stack (Player1, Player2, Player3)" or "Skyward Leap (Player1, Player2, Player3)" (or possibly put those names in a silent info text underneath?).  But, printing out 6 names would be a lot of names.

Yes thats how I had it in my mind but didn't express it in words, sorry.

I think printing this out for everybody is a little bit much, but probably the skyward leap and the stack group could have a list of who was in their group with them, e.g. "Stack (Player1, Player2, Player3)" or "Skyward Leap (Player1, Player2, Player3)" (or possibly put those names in a silent info text underneath?). But, printing out 6 names would be a lot of names.

No need for apology, I was just trying to think through how to best convey this information.  It's a good suggestion, thanks!

Sorry for the belated response here.  I am not likely to be back in DSR any time soon and I don't think I am going to get around to this.  I'm going to close this (but would take a PR if anybody else wanted to implement this).


there is no multi_isa build support with armv8-a as the base micro architecture

Output of 'strings libarm_compute.so | grep arm_compute_version':
arm_compute_version=v22.11 Build options: {'Werror': '1', 'debug': '0', 'neon': '1', 'opencl': '0', 'os': 'linux', 'openmp': '1', 'cppthreads': '0', 'arch': 'armv8.2-a', 'multi_isa': '1', 'build': 'native'} Git hash=b'1b3192e8a23513031163dc14d248f47671986121'
Platform: range of armv8 paltforms : Cortex-A72, Neoverse N1, Neoverse V1 etc...
Operating System: Ubuntu 20.04

Problem description:
multi_isa support mandated armv8.2-a as the base micro architecture, so today it's not possible to build a single ACL binary that covers all the armv8 architectures.
Workaround:
Here is the patch to lower the base architecture requirement to armv8-a for multi_isa builds. This covers all armv8-a platforms, however, excludes the FP16 kernels, which may be an acceptable tradeoff to get the broader armv8 support with single binary.
scons-mutli-isa-build-with-armv8-a-baseline.patch
Request:
The request is to provide multi_isa build armv8-a as base architecture, and support fp16 feature similar to bf16 and sve features supported today. Otherwise at least make armv8-a multi_isa without fp16 as the default build option and provide armv8.2-a based multi_isa as an option to get fp16 kernels.

Hi @snadampal
Please submit the patch here for review : link to repository
We can't take the patch file here.
Thank you

Hi @SuhailMunshi , here is the review request I've raised on ComputeLibrary gerrit, please review and provide feedback.
https://review.mlplatform.org/c/ml/ComputeLibrary/+/9474


Right click for navigation options (back / forward / reload)

Feature request
In Firefox and Chrome, right mouse click gives a context menu which includes the navigation options of back, forward and reload:
Chrome:

Firefox:

Currently the only right click option in SAFE Browser is "Inspect element" - can navigation options (which exist as buttons / menu items) be added to the right click menu?

Recommending that this task not be completed until we complete migration from webview's to BrowserView's as view architecture will change drastically.

As I'm not sure about Browser View's prospects re: zindexing etc, I'm gonna fire into this as it's largely just adding some menu options.

Changes made and these have been added over here: https://github.com/joshuef/safe_browser/tree/FeatContextMenu
But waiting on an updated electron-context-menu (https://github.com/sindresorhus/electron-context-menu/pull/67)

I can now see the menu options in the right-click context menu. (Back, Forward, Reload, Open New Tab, etc.)


☂ [Roadmap]: Improve Testing

This is an umbrella issue to related to one of our priorities for code-server: testing.
There are three main areas we're focusing on in regards to code-server's testing situation.
1. Code Coverage
We need to be able to measure our testing efforts. The best way to do that is through code coverage.

✅ Add support for code coverage

#2564
#2621

✅ Track code coverage for Lines in README #2621
🚧 Increase Lines coverage to 90-100% (currently: 50.26%)

#2671
#2701
#2702
#2715
#2719
#2744

⏳ Increase threshold to 90% and fail CI if not met
⏳ Track code coverage with each PR using something like @codecov

2. End-to-End Tests
code-server is a complex app and deserves to be thoroughly tested. With end-to-end tests, we can fix bugs, prevent regressions and ship more confidently.

✅ Add support for end-to-end tests #2643
✅ Add end-to-end test for "Go Home" feature #2648  (removed in #2862)
⏳ Add end-to-end test for installing extension from VSIX file
⏳ Add end-to-end test for using the Integrated Terminal
⏳ Add end-to-end test for using the File Search
TODO - will add more here

3. Test Guidelines
To make it easier for both contributors and maintainers to write tests for code-server, we should have guidelines.

⏳ Add /docs/testing.md with guidelines for writing unit, integration and end-to-end tests

Related issues:

#2646

LEGEND

⏳ = to do
🚧 = in progress
✅ = done

Note: this is part of an experiment to be more transparent about our priorities and our roadmap. See everything related to this effort in the project board

We are moving away from the umbrella issue approach and instead only using the Project Board. Things that are not yet issues are cards. Once they are to be worked on, they will be converted to issues.
See items under "To do" in Roadmap: https://github.com/cdr/code-server/projects/4


Need to delay adding certain elements to the tree

I want to use HTML5 for parsing pages, as part of Zombie.js (http://zombie.labnotes.org), but ran into an issue with the way elements are added to the tree.
When an element gets added to the tree, JSDOM fires a DOMNodeInserted event. It also listens to this event, and when it's fired on a SCRIPT element, loads the script (external) or evaluates it (internal). That's consistent with the way browsers evaluate scripts immediately after they're added to the document.
However, when the DOMNodeInserted event is fired, the SCRIPT element has no contents. I'm guessing the element is added to the tree first, then any child nodes are added to it. I couldn't figure out where this takes place, or how to change it (If I could, I would and send a pull request).
Can you help with that?

And I think that scripts are working now, even parsed in part.


Number of Crawlers

The amount of crawlers are not being utilized. For example, when I set the number of crawlers to 20, only 2-4 are being shown crawling on the console. Why is that?

Hi,
running edu.uci.ics.crawler4j.examples.basic.BasicCrawlController with 20 thread will run and show all thread running. Would you provide more information about your configuration and use case ?

It depends on many factors, like how broad (many different hosts, you don't want to turn crawl4j into a DoS tool) the crawl is. If you have lesser starting URLs, sure your crawl is slow at the beginning. If you stay within that URLs, it remains slow as crawl4j has some politeness implemented, not to send to much requests per second to a single host (no DoS tool intended).
Next is, how many external (and if your code logic allows them) sites are linked, the more you have, the more can be crawled simultanously at the same time.


FROM python:3.10-slim

Pull Request
Description
Update to docker python 3.10
How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce.
Please also list any relevant details for your test configuration

[ ] Yes

If your changes affect data processing, have you plotted any changes? i.e. have you done a quick sanity check?

[ ] Yes

Checklist:

[ ] My code follows OCF's coding style guidelines
[ ] I have performed a self-review of my own code
[ ] I have made corresponding changes to the documentation
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] I have checked my code and corrected any misspellings

Codecov Report

Merging #54 (75b6976) into main (898b385) will increase coverage by 0.34%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main      #54      +/-   ##
==========================================
+ Coverage   90.30%   90.64%   +0.34%
==========================================
  Files          15       15
  Lines         722      727       +5
==========================================
+ Hits          652      659       +7
+ Misses         70       68       -2

Impacted Files
Coverage Δ

nowcasting_forecast/__init__.py
100.00% <100.00%> (ø)

nowcasting_forecast/app.py
88.88% <100.00%> (+8.24%)
:arrow_up:

nowcasting_forecast/models/cnn/dataloader.py
89.47% <100.00%> (+2.80%)
:arrow_up:

nowcasting_forecast/models/utils.py
91.86% <100.00%> (ø)

nowcasting_forecast/models/nwp_solar_simple.py
96.66% <0.00%> (-3.34%)
:arrow_down:

nowcasting_forecast/batch.py
96.96% <0.00%> (-3.04%)
:arrow_down:

nowcasting_forecast/models/sun.py
98.21% <0.00%> (-1.79%)
:arrow_down:

:mega: Codecov can now indicate which changes are the most critical in Pull Requests. Learn more


regress-vlg_def_nettype_none_iv

Surelog issue created: https://github.com/chipsalliance/Surelog/issues/2336

Issue fixed.


using with paginated api requests

So I'm basically using node-huntergatherer (https://github.com/erictj/node-huntergatherer) for one of my projects for my api pagination needs but as the library is not updated any more, i'm considering to switch superagent.
But what I couldn't find out yet is how to support paginated api calls with it;
To make it more clear, node-huntergatherer supports them natively. Here is the explanation from the project;

It's ideally suited to the types of calls you see via APIs. Here's one example from the Etsy API (http://developer.etsy.com/docs) that you need to make if you want to gather all of a users' active listings:
http://openapi.etsy.com/v2/listings/active?limit=50&offset=50
http://openapi.etsy.com/v2/listings/active?limit=50&offset=100
http://openapi.etsy.com/v2/listings/active?limit=50&offset=150
http://openapi.etsy.com/v2/listings/active?limit=50&offset=200
http://openapi.etsy.com/v2/listings/active?limit=50&offset=250

Traditionally, you'd loop through each call, process the results of that call, and then call the next offset, process, ad nauseum.
 Especially when this pattern occurs in many APIs, and all that changes between them for the most part are the URL and how you process the results. So let's abstract out everything else!
And lastly, with HunterGatherer each of these calls happen simultaneously, up to the maximum number of simultaneous connections allowed based on the remote API terms of usage. Non-blocking FTW!

Is this possible with superagent and I'll be really glad if i can see a working sample.

@bonesoul Hey I'm doing a similar thing, what did you go with in the end? ty

Superagent by itself doesn't have such specific functionality. Our recommendation is to wrap creation of Request into something higher-level, e.g.
function _requestForOffset(path, offset) {
     return superagent.get(`https://api.example.com/${path}`).query({offset});
}

async function getAllPages() {
    let offset = 0, total;
    do {
        const response = await _requestForOffset("listings/active", offset);
        total = response.body.total;
        offset += response.body.count;
    } while(offset < total);
}


Documentation for regex_pattern_subscribe

we may need documentation for regex_pattern_subscribe, once we have the code of this feature merged.

the PRs has been merged. we could start this issue

related PRs #1103 #1279 #1298


Pypi signing key changed?

Hi,
I just downloaded the latest (2.10) version and tried to verify the signature; however I get an "no public key" error.
$ gpg --verify asdf-2.1.0.tar.gz.asc
gpg: assuming signed data in 'asdf-2.1.0.tar.gz'
gpg: Signature made Tue Sep 25 19:51:34 2018 CEST
gpg:                using RSA key 6013BEA25FCA76FA2F2342BE9E23DF56E5AED017
gpg: Can't check signature: No public key

Can you publish the public key used here? What is your policy for the signing?
Cheers
Ole

Ah yes, I created a new key recently and put it on github for code signing, but I forgot to publish it on a key server. I just sent it, so if you try again in a little bit it should work. If not, let me know.

For simplicity: could you just put it here?

Thanks! Works. :-)

Yes, sorry about that. It depends on which computer I do the release from, so both are currently valid.

OK, thanks.


fix(rules): include possible body offset in footer leading blank

Description
Fixes #292
Motivation and Context

See comment #426696523
Usage examples

echo 'docs(plugins): fix old docs\n\nConfig -> Plugins\n\ncloses #21' | npx commitlint -x '@commitlint/config-conventional'

echo 'feat: mail is now created by the supplied template function\n\nInstead of supplying all mail fields in the queue the information from the queue is given to a template function. This helps to prevent creating open mail servers that can be abused for spam. Also refactored the tests to a more traditional style.\n\nBREAKING CHANGE: The `template` option is required.' | npx commitlint -x '@commitlint/config-conventional'

How Has This Been Tested?

Types of changes

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[x] I have added tests to cover my changes.
[x] All new and existing tests passed.

Thanks!


Master Question

Are these all the controls that have to be on the page? I want a completely blank masterpage with only the ribbon controls on the page. I need to completely brand the portal so it doesn't look like SharePoint at all and also want to use bootstrap and all custom spfx controls and no out of the box controls. Is this possible? Do i just need to wrap all the controls in a div with display="none" or can I just delete them from the page?

Think of the master page as the shell of a SP site and all the placeholders are required or SP will break. You can hide the ones you don’t want to show.
That being said if you don’t want to show ANY of the default controls you might be better suited making your own UI and using client code to bring in SP content? That would be outside of the scope of the starter master pages.
What would probably be ideal for you is the concept of “Full Page SPFx Apps” which is planned by Microsoft but not a reality yet: https://www.voitanos.io/blog/what-s-next-for-the-sharepoint-framework
From: John Fabian<EMAIL_ADDRESS>Sent: Thursday, March 8, 2018 4:02 PM
To: rdrisgill/sharepoint-starter-masterpages<EMAIL_ADDRESS>Cc: Subscribed<EMAIL_ADDRESS>Subject: [rdrisgill/sharepoint-starter-masterpages] Master Question (#2)
Are these all the controls that have to be on the page? I want a completely blank masterpage with only the ribbon controls on the page. I need to completely brand the portal so it doesn't look like SharePoint at all and also want to use bootstrap and all custom spfx controls and no out of the box controls. Is this possible? Do i just need to wrap all the controls in a div with display="none" or can I just delete them from the page?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly,  https://github.com/rdrisgill/sharepoint-starter-masterpages/issues/2 view it on GitHub, or  https://github.com/notifications/unsubscribe-auth/ALVq3iAb-c7Iaxn5pUVqtVQ_bRln-y8rks5tcZxHgaJpZM4SjVwg mute the thread.  https://github.com/notifications/beacon/ALVq3v5fSdp4gCCPZqY_ehOkO6XDg9-cks5tcZxHgaJpZM4SjVwg.gif

Don’t forget you can set visible=false on placeholders and they won’t render their markup in HTML. As for whether you are good without the delegate, it depends on the delegate and also the page context. Sometimes they behave differently on different types of pages or sites. Mostly some kind of functionality won’t happen once your remove them.

On Mar 9, 2018, at 7:22 PM, John Fabian<EMAIL_ADDRESS>wrote:
@rdrisgill I already do have a full page custom ui using angular and the rest api hosted in a document library for certain apps, I have custom web parts build with spfx but I wanted to be able to use some of the content place holders that are wrapped with bootstrap since my spfx parts is bootstrap but was wondering which controls are required.
I do understand the content place holders need to be on the page but is it ok to remove the delegate controls? I do have a working example that has the ribbon but mostly set things to display: none but just wanted to clean up the markup to the bare min that won't break but not sure what problems I'll have removing delegate controls?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

@rdrisgill ok, thanks so much for the tips.


Fix prepass in Render Target

Re-enable prepass for SSAO
Test and ensured all cases are covered

Fix by #9591

Fix by #9591


Schema error on object type from imported CRD

Pretty new to timoni and cue so still learning.  I've a validation issue that occurs from running timoni mod vet related to the following setup.  Whilst it is a schema error, I am uncertain if the CRD itself is at fault or the underlying cue generation.
The Problem
I see pretty much exactly what is described in issue #278 .  Generated properties for type "object" result in a {} rather than {...} being assigned.  I only conclude this as I'm using this repo as a reference and its generated object properties are using the latter syntax with no schema errors.  Example here
My Setup
timoni version:
api: timoni.sh/v1alpha1
client: 0.19.0
cue: 0.7.0

Fetch Crossplane master
Import CRD yaml: timoni mod vendor crds -f <repo>\cluster\crds\apiextensions.crossplane.io_composition.yaml
Define a minimal schema to use a Composition schema from api "apiextensions.crossplane.io/composition/v1".  Relevant CRD and schema code below.  The error is related to a property defined as input?: {} in the generated schema.

CRD defines "input" as:
input:
   description: |-
      Input is an optional, arbitrary Kubernetes resource (i.e. a resource
      with an apiVersion and kind) that will be passed to the Composition
      Function as the 'input' of its RunFunctionRequest.
   type: object
   x-kubernetes-embedded-resource: true
   x-kubernetes-preserve-unknown-fields: true

Minimal schema to reproduce error:
package templates

import (
	crossplane "apiextensions.crossplane.io/composition/v1"
)

#SampleComposition: crossplane.#Composition & {
    spec: {
		pipeline: [
			{
				input: {
					apiVersion: "pt.fn.crossplane.io/v1beta1"
				}
			}
		]
    }
}

Not sure if this is related but I also don't see an actual schema defined when a Crossplane CompositeResourceDefinition CRD is imported either.  The CRD defines openAPIV3Schema as shown here and yet the cue type produces openApiV3Schema?: {} which again causes mod validation errors.


GE_GiftEntryController: Check security on Data Import and Form Template

Critical Changes
Changes
Issues Closed
Community Ideas Delivered
Features Intended for Future Release
Features for Elevate Customers
New Metadata
Deleted Metadata

W-13641420


Flann 1.9.1 refactor

Motivation for this change
Flann is an important package in the robotics community, as it is a dependency of PCL, the pointcloud library. Unfortunately, it hasn't received a release since 2016 and the repo has accumulated a large number of open issues and PRs:
https://github.com/flann-lib/flann/tags
Most notably, the 1.9.1 release has an issue where it bundles a copy of the lz4 headers. This change piggy-backs on the already-reviewed patches used in the Debian version of the package, prepared by @jspricke.
Things done
I'm a first-time contributor, so I don't know exactly what is best here, but in my local clone of nixpkgs (on NixOS) I ran:
nix build .#flann

And confirmed a successful build, with enablePython in both possible states.

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
21.11 Release Notes (or backporting 21.05 Relase notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Am running nixpkgs-review pr 129491, and so far I have the following failure:
nix log /nix/store/0gx2hycrlqdw8lmbh36yqiykmh5m9kj3-hugin-2019.0.0.drv
...
[ 70%] Linking CXX executable cpfind
/nix/store/scs241fp7dlrvm45d2fjbpjvygirn0ml-binutils-2.35.1/bin/ld: CMakeFiles/cpfind.dir/PanoDetectorLogic.cpp.o: undefined reference to symbol 'LZ4_compress_HC_continue'
/nix/store/scs241fp7dlrvm45d2fjbpjvygirn0ml-binutils-2.35.1/bin/ld: /nix/store/3nf85zf44vyvwr32hx53v7swljxysyjm-lz4-1.9.3/lib/liblz4.so.1: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status

FYI @lopsided98

Result of nixpkgs-review pr 129491 at 4da9729a run on aarch64-linux 1

1 package failed to build:

hugin

2 packages skipped due to time constraints:

digikam
pcl

2 packages built successfully:

cloudcompare
flann

3 suggestions:

warning: missing-patch-comment
Consider adding a comment explaining the purpose of this patch on the line preceeding.
Near pkgs/development/libraries/flann/default.nix:26:5:
   |
26 |     (fetchpatch {
   |     ^

warning: missing-patch-comment
Consider adding a comment explaining the purpose of this patch on the line preceeding.
Near pkgs/development/libraries/flann/default.nix:34:5:
   |
34 |     (fetchpatch {
   |     ^

warning: missing-patch-comment
Consider adding a comment explaining the purpose of this patch on the line preceeding.
Near pkgs/development/libraries/flann/default.nix:30:5:
   |
30 |     (fetchpatch {
   |     ^

Note that build failures may predate this PR, and could be nondeterministic or hardware dependent.
Please exercise your independent judgement. Does something look off? Please file an issue or reach out on IRC.

So the issue with Hugin is that it uses an included FindFLANN.cmake script that just looks for the sofiles without considering the correctly-generated pkgconfig information:
# This file was generated by CMake for flann
prefix=/nix/store/xrnvfm5f6f9k5lkca0ik3s9z52d84iw3-flann-1.9.1
exec_prefix=${prefix}
libdir=${prefix}/lib
includedir=${prefix}/include

Name: flann
Description: Fast Library for Approximate Nearest Neighbors
Version: 1.9.1
Requires:
Libs: -L${libdir} -L/nix/store/3nf85zf44vyvwr32hx53v7swljxysyjm-lz4-1.9.3/lib;-llz4 -lflann -lflann_cpp
Cflags: -I${includedir}

There is a newer release of Hugin (2020.0), but it doesn't seem to address this issue. I don't see a patch for this in Debian, so I'm not sure how it's working there either. If it needs to be patched, my inclination would be just to grab the FindFLANN.cmake from PCL, since that one does the right thing (including being future-proof to a glorious day when FLANN ships its own find module):
https://github.com/PointCloudLibrary/pcl/blob/master/cmake/Modules/FindFLANN.cmake

@mikepurvis the hugin in Debian is version 2020.0.0 and uses pkg-config and gets -llz4 from there:
-- checking for module 'flann'
--   found flann, version 1.9.1
-- flann library found
...
../../celeste/libceleste.so.0.0 -llz4 -lflann -lflann_cpp ../../hugin_base/libhuginbase.so.0.0 /usr/lib/x86_64-linux-gnu/libX11.so

https://buildd.debian.org/status/fetch.php?pkg=hugin&arch=amd64&ver=2020.0.0%2Bdfsg-2&stamp=1609245416&raw=0

@jspricke Thanks for jumping in here! On closer inspection, I see that the FindFLANN module does in fact use pkgconfig:
if(UNIX)
  find_package(PkgConfig QUIET)
  pkg_check_modules(FLANN flann)

And it's unchanged between the 2019 and 2020 versions. Both fail in my local {{nix develop}} shell in the same way— it definitely doesn't have the proper link flag for lz4:
cd /home/ciadmin/hugin-src/hugin-2019.0.0/build/src/hugin_cpfind/cpfind && /nix/store/186fa5snplvdlvimx0jgw88ls2v62zxv-cmake-3.19.7/bin/cmake -E cmake_link_script CMakeFiles/cpfind.dir/link.txt --verbose=1
/nix/store/fdy0bnh3x01rysv3cig42wq3sfcqf0zz-gcc-wrapper-10.3.0/bin/g++ -O2 -g -DNDEBUG  -fopenmp -rdynamic CMakeFiles/cpfind.dir/PanoDetector.cpp.o CMakeFiles/cpfind.dir/PanoDetectorLogic.cpp.o CMakeFiles/cpfind.dir/TestCode.cpp.o CMakeFiles/cpfind.dir/Utils.cpp.o CMakeFiles/cpfind.dir/main.cpp.o -o cpfind  -Wl,-rpath,/home/ciadmin/hugin-src/hugin-2019.0.0/build/src/hugin_cpfind/localfeatures:/home/ciadmin/hugin-src/hugin-2019.0.0/build/src/celeste:/home/ciadmin/hugin-src/hugin-2019.0.0/build/src/hugin_base: ../localfeatures/liblocalfeatures.so.0.0 -lvigraimpex -lImath -lIlmImf -lIex -lHalf -lIlmThread -lz -ljpeg -ltiff -lpng -lz -lz -lexiv2 -lpano13 ../../foreign/levmar/libhuginlevmar.a -lGLEW -lboost_filesystem -lboost_system -lfftw3 -lvigraimpex -lImath -lIlmImf -lIex -lHalf -lIlmThread -lz -ljpeg -ltiff -lpng -lz -lz -lexiv2 -llcms2 ../../celeste/libceleste.so.0.0 -lflann -lflann_cpp ../../hugin_base/libhuginbase.so.0.0 -lX11 -lGL -lGLU -lsqlite3 -lpano13 ../../foreign/levmar/libhuginlevmar.a -lGLEW -lboost_filesystem -lboost_system -lfftw3 -lvigraimpex -lImath -lIlmImf -lIex -lHalf -lIlmThread -lz -ljpeg -lpng -lz -ljpeg -lpng -ltiff -lexiv2 -llcms2 -pthread
/nix/store/scs241fp7dlrvm45d2fjbpjvygirn0ml-binutils-2.35.1/bin/ld: CMakeFiles/cpfind.dir/PanoDetectorLogic.cpp.o: undefined reference to symbol 'LZ4_compress_HC_continue'
/nix/store/scs241fp7dlrvm45d2fjbpjvygirn0ml-binutils-2.35.1/bin/ld: /nix/store/3nf85zf44vyvwr32hx53v7swljxysyjm-lz4-1.9.3/lib/liblz4.so.1: error adding symbols: DSO missing from command line

I shall continue to investigate.

@mikepurvis
are you sure your flann.pc is correct?
What is the output of pkg-config --libs flann?
Your log contains -lflann -lflann_cpp which should be from pkg-config if I read the FindFLANN.cmake correct.

are you sure your flann.pc is correct?

Seems reasonable, I think:
$ cat result/lib/pkgconfig/flann.pc
# This file was generated by CMake for flann
prefix=/nix/store/xrnvfm5f6f9k5lkca0ik3s9z52d84iw3-flann-1.9.1
exec_prefix=${prefix}
libdir=${prefix}/lib
includedir=${prefix}/include

Name: flann
Description: Fast Library for Approximate Nearest Neighbors
Version: 1.9.1
Requires:
Libs: -L${libdir} -L/nix/store/3nf85zf44vyvwr32hx53v7swljxysyjm-lz4-1.9.3/lib;-llz4 -lflann -lflann_cpp
Cflags: -I${includedir}

And this config also works fine with the PCL 1.11 build.

Your log contains -lflann -lflann_cpp which should be from pkg-config if I read the FindFLANN.cmake correct.

Yeah, I would have thought so too, and yet:
$ pkg-config --libs flann
-L/nix/store/xrnvfm5f6f9k5lkca0ik3s9z52d84iw3-flann-1.9.1/lib -L/nix/store/3nf85zf44vyvwr32hx53v7swljxysyjm-lz4-1.9.3/lib\;-llz4 -lflann -lflann_cpp

I had a look, the difference is:
Debian:
$ pkg-config --libs-only-l flann
-llz4 -lflann -lflann_cpp

NixOS
pkg-config --libs-only-l flann
-lflann -lflann_cpp

the NixOS flann.pc
$ cat .nix-profile/lib/pkgconfig/flann.pc
# This file was generated by CMake for flann
prefix=/nix/store/jrpxk67fnda2laqlsij8mwc8g62kp0yk-flann-1.9.1
exec_prefix=${prefix}
libdir=${prefix}/lib
includedir=${prefix}/include

Name: flann
Description: Fast Library for Approximate Nearest Neighbors
Version: 1.9.1
Requires:
Libs: -L${libdir} -L/nix/store/3nf85zf44vyvwr32hx53v7swljxysyjm-lz4-1.9.3/lib;-llz4 -lflann -lflann_cpp
Cflags: -I${includedir}

The problem is the semicolon, which is coming from @LZ4_STATIC_LDFLAGS@ which should be converted to string, I guess. Can you work on a patch?

Will do!

Have pushed a new version which I believe addresses all the feedback, and am locally re-running nixpkgs-review pr 129491, will post the results of that when available, but I believe Hugin is now fixed based on a source build.
Regarding the Python story, it turns out it's pretty goofy; it isn't bindings at all, but rather just a handful of uninstalled files that make up a ctypes wrapper:
$ find result/ | grep py
result/share/flann/python
result/share/flann/python/setup.py
result/share/flann/python/pyflann
result/share/flann/python/pyflann/index.py
result/share/flann/python/pyflann/__init__.py
result/share/flann/python/pyflann/exceptions.py
result/share/flann/python/pyflann/flann_ctypes.py

So I've left this is in for now, but it won't be much use to anyone unless the packaging is further updated to actually {{python3 setup.py install}} these.

I'm not sure what the review-shell thing is all about here, but everything passed dependency rebuild, so I'm going to tick that off the list.
$ nixpkgs-review pr 129491
$ git -c fetch.prune=false fetch --force https://github.com/NixOS/nixpkgs master:refs/nixpkgs-review/0 pull/129491/head:refs/nixpkgs-review/1
$ git worktree add /home/ciadmin/.cache/nixpkgs-review/pr-129491-2/nixpkgs df0d321258a9b35c36cd9d8688985502d464105f
Preparing worktree (detached HEAD df0d321258a)
Updating files: 100% (26957/26957), done.
HEAD is now at df0d321258a Merge pull request #123765 from hyperfekt/update-bcachefs
$ nix-env --option system x86_64-linux -f /home/ciadmin/.cache/nixpkgs-review/pr-129491-2/nixpkgs -qaP --xml --out-path --show-trace
$ git merge --no-commit --no-ff 3fc8ddab5dd5653e9b6cebe78ed84ca87daceea5
Automatic merge went well; stopped before committing as requested
$ nix-env --option system x86_64-linux -f /home/ciadmin/.cache/nixpkgs-review/pr-129491-2/nixpkgs -qaP --xml --out-path --show-trace --meta
6 packages updated:
cloudcompare digikam digikam flann hugin pcl

$ nix --experimental-features nix-command build --no-link --keep-going --option build-use-sandbox relaxed -f /home/ciadmin/.cache/nixpkgs-review/pr-129491-2/build.nix
error: builder for '/nix/store/lpnx7yw3d21dk37alsl7xqpgm28dhj63-review-shell.drv' failed with exit code 1;
       last 2 log lines:
       > qtPreHook
       > Error: wrapQtAppsHook is not used, and dontWrapQtApps is not set.
       For full logs, run 'nix log /nix/store/lpnx7yw3d21dk37alsl7xqpgm28dhj63-review-shell.drv'.

Link to currently reviewing PR:
https://github.com/NixOS/nixpkgs/pull/129491

5 packages built:
cloudcompare digikam flann hugin pcl

Awesome, thanks for the merge!


Rework HTML slate renderer.

Changelog Description
Addresses the TODOs for HTML slate renderer:

Turn generator and renderer as regular python classes and not dataclasses anymore
Add unit tests

Changed behavior:

Renderer will now raise a ValueError by default if the provided data does not fill up all of the expected template field
Slate can ignore missing data and still goes through with the newly added SlateFillMode.HIDE_FIELD_WHEN_MISSING mode
Provide default data for resolution, date and input frame range

Additional info

Contributes to #26
Depends on #30
Testing notes:

New automated tests have been added under tests/test_renderers_slate.py
Either use poetry or we can directly run helper .\start.ps1 test (Windows only)

Unit tests are broken on Github but that does not seem to be related to the changes from this PR.
I'm investigating that aside here: https://github.com/ynput/LabLib/pull/29
But the tests should be working properly locally.

I've been following this a bit and just wanted to chime in on something that was marked to change:
"Renderer will now raise a ValueError if the provided data does not fill up all of the expected template field (instead of silently hidding them)"
The way we thought about it is that whenever there is a property that is not resolved we sub that prop with a specific string. That string becomes an identifier that is able to then hide the parent container if the info is not present. This is due to the fact that sometimes clients ask for very informative slates, but not always do we have data to show.
So what happens is that we just hide the row instead of

not rendering the info but keeping the caption.
failing.

If raising ValueError could be default behaviour but could be also opted out in favor of a workflow like that it would be most helpful. We do rely on a mechanism like that to hide and make the slates not crash most of the time in house.
As its is you are making the assumption that whoever fills the data

always have it.
always fills it up.

which is never the case in our experience, and Raising generates a lot of requests for pipe dept to fix things asap :)

I've been following this a bit and just wanted to chime in on something that was marked to change:
"Renderer will now raise a ValueError if the provided data does not fill up all of the expected template field (instead of silently hidding them)"
The way we thought about it is that whenever there is a property that is not resolved we sub that prop with a specific string. That string becomes an identifier that is able to then hide the parent container if the info is not present. This is due to the fact that sometimes clients ask for very informative slates, but not always do we have data to show. So what happens is that we just hide the row instead of

not rendering the info but keeping the caption.
failing.

If raising ValueError could be default behaviour but could be also opted out in favor of a workflow like that it would be most helpful. We do rely on a mechanism like that to hide and make the slates not crash most of the time in house.
As its is you are making the assumption that whoever fills the data

always have it.
always fills it up.

which is never the case in our experience, and Raising generates a lot of requests for pipe dept to fix things asap :)

Thanks a lot @maxpareschi for the user/experience feedback, definitely most valuable.
I get your point and I considered adding these "mode" as a future iteration.
All things considered, maybe I should actually go ahead and implement it right now.

Not sure where is this coming from but my results are produced following results.
BLD_010_0010_plateMain_v000.1000.exr.zip

Thanks for the material, this is very puzzling...
I cannot reproduce this issue on my machine (Linux nor Windows) nor on the GH actions host..
From the screenshot, you can see:

in green the burnins computed from the provided resolution (which confirms 1920x1080 is properly requested to selenium)
in blue the output resolution which comes out of selenium screen save

I had a look at the selenium reported issues on the repo but I could not find anything meaningful.
My guess is that it could comes from weird selenium window.outerWidth, window.innerWidth, window.outerHeight, window.innerHeight which would result in a wrong output screen resolution there.
Out of curiosity, would it be please possible to run the test on your side and confirm which values you obtain for those ?
In the meantime, I have also added an additional -resize in the final oiio command line to ensure final resolution.
I'm not super found of it but this should fixes the tests on your side. Can you please confirm ?
Honestly, this looks more a patch than a proper required thing here.. but I couldn't think of anything better.

my tests say let's go.

i noticed that the output directories are named after the temp dirs they're coming from. don't know if that's intended but it makes it a bit difficult to know which test run produced them. there's also 2 output.exrs that i can't really map to something.

here's a screen of a slate on my potato:

scrubbing through some slated sequences i noticed that the slates seem to be always smaller than the plates.
checking with resolve and djv for all slate related test result folders for resolutions i get:

for the 2 output.exrs:

resolve: 1920x1080
djv:

for all others

resolve: 4382x2310
djv: pretty different, some 1920x1080 some 4096x2048

i'd suggest to get some nice informative directory names so we can track this down better
but slates do be rendered 🥳

Awesome, well I do like your suggestion and ensure that directory names match test name.
This way it's now easier to troubleshoots result.

The result and resolution you got in Resolve match what I expected so I think we are good.
(I'm quite sure djv matches the higher resolution of the sequence).
That brings another question on my side, for now when no explicitly provided the slate resolution is HD 1080p.
That's what the existing code was doing on purpose, but maybe it makes more sense to adjust that to be the same resolution as the first frame of the input sequence ? What do you guys think ?

That brings another question on my side, for now , unless explicitly provided, the slate resolution is HD-1080p. That's what the existing code was doing on purpose, but maybe it makes more sense to adjust that to be the same resolution as the first frame of the input sequence ? What do you guys think ?

defaulting to 1920x1080 feels fine to me.
in my monkey brain these are 2 different use cases. i'll just refer to them as slate_only and prepend_to_sequence.
in slate_only u'd want to specify an explicit resolution, in prepend_to_sequence i'd assume the slate to be generated following the sequence's resolution.

Here are the resolution results I printed from line 256 in LabLib/lablib/generators/slate_html.py. I checked my virtual environment, and Selenium is version 4.16.0 on Windows, as defined in pyproject.toml. What versions do you have, @tweak-wtf  and @robin-ynput ?

test_Slaterenderer_missing_keys_hide: size [2459, 1859]
test_Slaterenderer_default: size [2459, 1859]
test_Slaterenderer_4K: size [4635, 2827]
test_Slaterenderer_explicit_output: size [2459, 1859]

The resulting resolutions seem to be off. In the example image, the output from test_Slaterenderer_default should be 1920x1080, but it's coming out as 2459x1859.

There's also an issue with the broken template. This might be caused by the miscalculation, which then affects other calculations. Here is the original template look and the resulting look.

Just to tell you that chrome in the —headless=new mode has two “bugs” (which apparently they are intended behavior but still)

the viewport will always contain the interface elements, so you need to make the slate bigger and then crop it
it you are using a hidpi monitor it’s going to screw up calculations. I’ve tried compensating with scale values but to no avail. You can try render slates again by setting the display scale size in windows or o 100%, this gave consistent results in our tests.
If you are stuck in that you could always use —headless=old and revert to the normal behavior, still I did not test that with hidpi.

Thanks a lot @maxpareschi, thanks to you I think I found something.
I played a bit with the Hight DPI / scale options in Windows and could reproduce a resolution mismatch.

I've managed to fix this introducing a --force-device-scale-factor=1 to Chrome, so I've removed my unecessary oiio resize at the end.
To me, even with 125% or 150% set, the resulting HD and 4K slates matches the expected resolution and visual (selenium-0.4.16.

@jakubjezek001 does it fix the issue on your end as well ?

my environment is:

selenium 4.16.0
windows display scale: 125%
windows display resolution: 2256x1504

i threw all slated sequences into resolve and made a screencast.
kinda misused the shot and scene here to refer to the test case name and the files resolution

from this i noticed:

slate templates seem to be applied correctly now however, all slates seem to deviate from the 4k one in terms of aspect ratio.

it's noticeable better side by side

seems there's a bug in the timecode offset. it should follow the sequence's start tc except the output.exr oc

Yeah the timecode issue is its own PR scope @tweak-wtf what do you thing. But could be added here if you found it important to include it.

cococool! i do think it's important that the TC is consistent but yeah probably a different PR. lemme go ahead on this


ROX-16531: Add orphaned resources tests to DeclarativeConfigTest.groovy

Description
This is the second PR in a series of PRs to introduce e2e test for behaviour implemented in issues.redhat.com/browse/ROX-14700.
This particular PR focuses on how declarative configuration logic handles deletion of dependent resources.
Checklist

[ ] Investigated and inspected CI test results
[ ] Unit test and regression tests added
[ ] Evaluated and added CHANGELOG entry if required
[ ] Determined and documented upgrade steps
[ ] Documented user facing changes (create PR based on openshift/openshift-docs and merge into rhacs-docs)

If any of these don't apply, please comment below.
Testing Performed

New tests added
CI is sufficient

/test gke-qa-e2e-tests


Chore: Update @civic/profile to 0.5.2

Upgrade the @civic/profile lib to version 0.5.2. This version adds the lib version to profile API calls as a header to improve analytical data.

Very Good


V2.0.8

Coverage decreased (-1.3%) to 85.356% when pulling ae46aad405e332f93eb0fee26ea587e6d9ec5dcf on v2.0.8 into 00360d39fa99d8c72d13d6f6d9068b8c4cc6c86e on master.

Coverage decreased (-1.3%) to 85.356% when pulling ae46aad405e332f93eb0fee26ea587e6d9ec5dcf on v2.0.8 into 00360d39fa99d8c72d13d6f6d9068b8c4cc6c86e on master.


feat: added tutorials for RAG evaluation with RAGAS+Gemini and DeepEval+Gemini

Description
added tutorials for RAG evaluation with RAGAS+Gemini and DeepEval+Gemini

Hi @AbiramiSukumaran ,

The linting works on local setup but break on the repo CI. How do I ensure it works?
Spellcheck fails on CI . "deepeval" is package name . Any idea how to resolve this?

Hi @AbiramiSukumaran ,

The linting works on local setup but breaks on the repo CI. How do I ensure it works?
Spellcheck fails on CI . "deepeval" is package name . Any idea how to resolve this?

You can add false positive spelling errors to the https://github.com/GoogleCloudPlatform/generative-ai/blob/main/.github/actions/spelling/allow.txt file

Several of the words found by the spell checker are actual misspellings. Please fix the actual spelling errors, then add the false-positives to allow.txt

Several of the words found by the spell checker are actual misspellings. Please fix the actual spelling errors, then add the false-positives to allow.txt

Done

See comments, and fix spelling errors

Done


Istio-Egressgateway doesn't respect Outboundpolicy = REGISTRY_ONLY

Bug Description
I've setup my Istio Cluster with an Registry_Only Outboundpolicy. So far, everything works as expected. Namespaces with no Serviceentry gives 502 as expected.
kubectl -n playground exec -ti ubuntu-deployment-c6d87b77c-jkfwv -- curl -v google.com
Defaulting container name to ubuntu-diagnose.
Use 'kubectl describe pod/ubuntu-deployment-c6d87b77c-jkfwv -n playground' to see all of the containers in this pod.
*   Trying <IP_ADDRESS>:80...
* TCP_NODELAY set
* Connected to google.com (<IP_ADDRESS>) port 80 (#0)
> GET / HTTP/1.1
> Host: google.com
> User-Agent: curl/7.68.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 502 Bad Gateway
< date: Tue, 31 Jan 2023 14:08:39 GMT
< server: envoy
< content-length: 0
<
* Connection #0 to host google.com left intact

Unfortunately, the "istio-system" namespace doesn't affect the outboundpolicy at all.
kubectl -n istio-system exec -ti istio-egressgateway-64f7b48476-fxhhr -- curl -v google.com
*   Trying <IP_ADDRESS>:80...
* Connected to google.com (<IP_ADDRESS>) port 80 (#0)
> GET / HTTP/1.1
> Host: google.com
> User-Agent: curl/7.81.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 301 Moved Permanently
< Location: http://www.google.com/
< Content-Type: text/html; charset=UTF-8
< Date: Tue, 31 Jan 2023 14:06:13 GMT
< Expires: Thu, 02 Mar 2023 14:06:13 GMT
< Cache-Control: public, max-age=2592000
< Server: gws
< Content-Length: 219
< X-XSS-Protection: 0
< X-Frame-Options: SAMEORIGIN
<
<meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>301 Moved</TITLE>

301 Moved

The document has moved
here.

* Connection #0 to host google.com left intact

Version
client version: 1.15.2
control plane version: 1.15.2
data plane version: 1.15.2 (87 proxies)

Additional Information
No response

This is expected. Gateways do not have traffic interception. Requests from a gateway pod directly will be unmodified.


Alert when PWD doesn't work

The idea is to write a simple script that:

Creates a new session
Creates an instance
Closes the session

We will run this script every few minutes on every PWD server. If it works, nothing happens. If it doesn't work, we will send a message to a slack channel.

I'd love to see something in the /ping endpoint so we don't have to depend on a  "script" to be alerted.

We've added some basic cloudwatch alarms but they only fix part of the problem. We still need to find a reliable way to make a healthcheck that asserts that an instance can be successfully created

Should be fixed by #137


Kubernetes Helm chart application installation fails when Portainer is running on a custom domain with SSL certificate configured.

Bug description
Kubernetes Helm chart installation fails when Portainer is running on a custom domain with SSL certificate configured.
Portainer is running in my system in Docker on an Ubuntu VM.  A separate machine is running Ubuntu with K3S and Portainer Edge Agent installed.
Expected behavior
When attempting to install a Helm application from the Chart repository, I expect installation to happen but it appears that because it is communicating with the Kubernetes API over "localhost", a certificate error occurs.
Portainer Logs
level=info msg="2022/03/02 18:03:16 http: TLS handshake error from <IP_ADDRESS>:56500: remote error: tls: bad certificate"
level=info msg="2022/03/02 18:03:16 http error: Unable to install a chart (err=failed to run helm install on specified args: Error: Kubernetes cluster unreachable: Get \"https://localhost:9443/api/endpoints/5/kubernetes/version?timeout=32s\": x509: certificate is valid for *.lab.(redacted).org, lab.(redacted).org, not localhost"
level=info msg=": exit status 1) (code=500)"

Steps to reproduce the issue:

From Home, go to Kubernetes Edge Agent environment
Click on 'Helm'
Select a Chart, add a name
Deploy Chart
See error

Technical details:

Portainer version: 2.11.1
Docker version (managed by Portainer): portainer/portainer-ce@sha256:6ae5068d1719755fb0fedd5644e41db5c6cbebaf3589aa712b90aae8e91cd157
Edge Agent version: portainer/agent:2.11.1
Platform (windows/linux): Linux (Ubuntu)
Command used to start Portainer (docker run -p 9443:9443 portainer/portainer --ssl --sslcert /certs/portainer.crt --sslkey /certs/portainer.key):
Browser: Chrome
Use Case: Using Portainer at Home
Have you reviewed our technical documentation and knowledge base? Yes

Additional context
My Portainer instance is running on and accessed through a custom domain, e.g. "nas-ubuntu.lab.(redacted).org:9443".
Note that a similar error also occurs when attempting to browse details of any application deployed via helm externally as well, but it does not occur when viewing details in the portainer UI of applications deployed with standard K8S templates.

fixed in 2.14.2


Update/Fix APK Release CI WorkFlow

IMPORTANT: Where possible all PRs must be linked to a Github issue
Fixes #2214
Engineer Checklist

[ ] I have written Unit tests for any new feature(s) and edge cases for bug fixes
[ ] I have added any strings visible on UI components to the strings.xml file
[ ] I have updated the  CHANGELOG.md file for any notable changes to the codebase
[ ] I have run ./gradlew spotlessApply and ./gradlew spotlessCheck to check my code follows the project's style guide
[ ] I have built and run the FHIRCore app to verify my change fixes the issue and/or does not break the app

Code Reviewer Checklist

[ ] I have verified Unit tests have been written for any new feature(s) and edge cases
[ ] I have verified any strings visible on UI components are in the strings.xml file
[ ] I have verifed the CHANGELOG.md file has any notable changes to the codebase
[ ] I have verified the solution has been implemented in a configurable and generic way for reuseable components
[ ] I have built and run the FHIRCore app to verify the change fixes the issue and/or does not break the app

This PR just disables generation of an AAB file - we don’t need that right now, its only required for a Google Playstore release. It cuts down the build time so its worth disabling


Explicitly stated minimum Django version for 0.12.0

As requested in https://github.com/alex/django-filter/issues/353#issuecomment-175499338

👍


Update TbbpmStatelessProcessRuntime.java

CodeScene Delta Analysis Results
View detailed results in CodeScene

Recommended Review Level
Lightweight sanity check


the main window of the GUI is disappeared.

From<EMAIL_ADDRESS>on 2013-12-08T17:15:41Z
Please note that reporting bugs from Spyder's help menu ("?" > "Report
issue...") will fill version details for Spyder, Python and Qt in this description for you.
What steps will reproduce the problem?

running spyder.bat
the green "Spyder 2" appears
the main windows is invisible. What is the expected output? What do you see instead? After the green "Spyder 2" screen disappears the main screen is invisible. Actually the icon is active and the process is there, but the window is kind of invisible. What version of the product are you using? On what operating system? Spyder 2.2.5 - win32 - Python 2.7 - Windows 7 - 64bits

Please provide any additional information below
. It was working recently very well, but suddenly the main window doesn't appear at all.
Original issue: http://code.google.com/p/spyderlib/issues/detail?id=1672

From ccordoba12 on 2013-12-11T17:27:57Z
What happens if you open a terminal (i.e. cmd.exe), then run in it
spyder --reset
then start Spyder again?
You may also need to install pywin32 (if you don't have it installed).

From contrebasse on 2013-12-12T11:59:49Z
It seems that the config file is easily corrupted as many issues are fixed by resetting the config. Maybe the file saving should be refactored somehow sometimes in the future ? If a very resilient interface is wanted, sqlite can be used as the db is "guaranteed" to never be corrupted even if the application crashes in the middle of the file writing.

From<EMAIL_ADDRESS>on 2013-12-12T11:03:43Z
So it may be considered fixed by the usual "spyder --reset" workaround.
At school :

this needs to be used about one time every 15 sessions on slow and very-poor in memory PC,
nearly never on recent PCs.

From<EMAIL_ADDRESS>on 2013-12-12T02:44:16Z
the spyder --reset apparently solved the problem. the output is:
C:\Users\reza\Desktop>spyder --reset
*** Reset Spyder settings to defaults ***
removing: C:\Users\reza.spyder2.inspector
removing: C:\Users\reza.spyder2.onlinehelp
removing: C:\Users\reza.spyder2.spyder.ini
removing: C:\Users\reza.spyder2.temp.py
removing: C:\Users\reza.spyder2\template.py
removing: C:\Users\reza.spyder2.history.py
removing: C:\Users\reza.spyder2.history_internal.py
removing: C:\Users\reza.spyder2.ropeproject
removing: C:\Users\reza.spyder2.workingdir
thanks

Closing this old bug.


[WIP] Verification

Most of the structure is in place already, but is taking longer than expected because of necessary changes in both incentive-layer and dispute-resolution-layer.
Another thing before completion would be a simple test.
I'm thinking we can get away with starting up tg,s,v processes. Submit a task like normal, wait for solution committed event, then challenge it from the verifier's account, and let the processes duke it out.

Merging for now so I can do a reorg


cache: add plex_insecure option to skip certificate validation

As requested in #2215, this adds a plex_insecure config setting to the cache backend, which will turn off all certificate verification's when connecting to the Plex server.
Fixes #2215

That looks very useful - thank you :-)