No way to test periodic Streams in testWidgets

Because of the way testWidgets modifies the passage of time, it seems to be impossible to work with periodic Streams in Flutter apps.
Periodic Streams are crucial. For example, ReactiveX's bufferTime and debounce let you buffer / ignore events based on time — and they use Stream.periodic internally (exhibit A).
Steps to Reproduce

Run the following code separately to ensure it doesn't hang (waiting for a stream to finish):import 'package:rxdart/rxdart.dart';

main() async {
  final controller = new PublishSubject<int>();

  final subscription = controller
      .bufferTime(const Duration(milliseconds: 500))
      .listen((list) => print(list));

  await subscription.cancel();
  await controller.close();
}

This finishes immediately.
Now try the same thing in testWidgets: testWidgets('close_stream', (WidgetTester tester) async {
   final controller = new PublishSubject<int>();

   final subscription = controller
       .bufferTime(const Duration(milliseconds: 500))
       .listen((list) => print(list));

   await subscription.cancel();
   await controller.close();
 });

This will hang, and the test will time out.00:09 +0 -1: - close_stream [E]
  TimeoutException after 0:00:05.000000: Test timed out after 5 seconds.
  dart:async  _startMicrotaskLoop

00:09 +0 -1: Some tests failed.

Try removing the awaittestWidgets('close_stream', (WidgetTester tester) async {
  final controller = new PublishSubject<int>();

  final subscription = controller
      .bufferTime(const Duration(milliseconds: 500))
      .listen((list) => print(list));

  subscription.cancel();
  controller.close();
});

and you get the "Timer is still running" error:00:04 +0: - close_stream
══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
The following assertion was thrown running a test:
A periodic Timer is still running even after the widget tree was disposed.
'package:flutter_test/src/binding.dart': Failed assertion: line 668 pos 7:
'_fakeAsync.periodicTimerCount == 0'

Either the assertion indicates an error in the framework itself, or we should provide substantially
more information in this error message to help you determine and fix the underlying cause.
In either case, please report this assertion by filing a bug on GitHub:
  https://github.com/flutter/flutter/issues/new

When the exception was thrown, this was the stack:
#2      AutomatedTestWidgetsFlutterBinding._verifyInvariants (package:flutter_test/src/binding.dart)
#3      TestWidgetsFlutterBinding._runTestBody (package:flutter_test/src/binding.dart:487:7)
<asynchronous suspension>
#6      TestWidgetsFlutterBinding._runTest (package:flutter_test/src/binding.dart:464:14)
#7      AutomatedTestWidgetsFlutterBinding.runTest.<anonymous closure> (package:flutter_test/src/binding.dart:646:24)
#13     AutomatedTestWidgetsFlutterBinding.runTest (package:flutter_test/src/binding.dart:644:16)
#14     testWidgets.<anonymous closure>.<anonymous closure> (package:flutter_test/src/widget_tester.dart:62:24)
#15     Declarer.test.<anonymous closure>.<anonymous closure>.<anonymous closure> (package:test/src/backend/declarer.dart:161:27)
<asynchronous suspension>
#16     Invoker.waitForOutstandingCallbacks.<anonymous closure> (package:test/src/backend/invoker.dart:249:15)
<asynchronous suspension>
#20     Invoker.waitForOutstandingCallbacks (package:test/src/backend/invoker.dart:246:5)
#21     Declarer.test.<anonymous closure>.<anonymous closure> (package:test/src/backend/declarer.dart:159:33)
#25     Declarer.test.<anonymous closure> (package:test/src/backend/declarer.dart:158:13)
<asynchronous suspension>
#26     Invoker._onRun.<anonymous closure>.<anonymous closure>.<anonymous closure>.<anonymous closure> (package:test/src/backend/invoker.dart:403:25)
<asynchronous suspension>
#40     _Timer._runTimers (dart:isolate/runtime/libtimer_impl.dart:382:19)
#41     _Timer._handleMessage (dart:isolate/runtime/libtimer_impl.dart:416:5)
#42     _RawReceivePortImpl._handleMessage (dart:isolate/runtime/libisolate_patch.dart:165:12)
(elided 28 frames from class _AssertionError, class _FakeAsync, package dart:async, and package
stack_trace)

The test description was:
close_stream
════════════════════════════════════════════════════════════════════════════════════════════════════
00:04 +0 -1: - close_stream [E]
  Test failed. See exception logs above.
  The test description was: close_stream

00:04 +0 -1: Some tests failed.

I tried adding await tester.pumpAndSettle(const Duration(seconds: 1)); to force the test to finish its timers but it doesn't help.

Logs

$ flutter test --plain-name=close_stream --verbose
[  +26 ms] [/Users/filiph/dev/flutter/] git rev-parse --abbrev-ref --symbolic @{u}
[  +28 ms] Exit code 0 from: git rev-parse --abbrev-ref --symbolic @{u}
[        ] origin/beta
[        ] [/Users/filiph/dev/flutter/] git rev-parse --abbrev-ref HEAD
[   +5 ms] Exit code 0 from: git rev-parse --abbrev-ref HEAD
[        ] beta
[        ] [/Users/filiph/dev/flutter/] git ls-remote --get-url origin
[   +4 ms] Exit code 0 from: git ls-remote --get-url origin
[        ] https://github.com/flutter/flutter.git
[        ] [/Users/filiph/dev/flutter/] git log -n 1 --pretty=format:%H
[   +6 ms] Exit code 0 from: git log -n 1 --pretty=format:%H
[        ] 44b7e7d3f42f050a79712daab253af06e9daf530
[        ] [/Users/filiph/dev/flutter/] git log -n 1 --pretty=format:%ar
[   +5 ms] Exit code 0 from: git log -n 1 --pretty=format:%ar
[        ] 4 weeks ago
[        ] [/Users/filiph/dev/flutter/] git describe --match v*.*.* --first-parent --long --tags
[   +7 ms] Exit code 0 from: git describe --match v*.*.* --first-parent --long --tags
[        ] v0.3.2-0-g44b7e7d3f
[ +234 ms] switching to directory LocalDirectory: 'test' to run tests
[   +1 ms] running test package with arguments: [--plain-name, close_stream, --, /Users/filiph/dev/reactive_exploration/shared/test/smoke_test.dart]
00:00 +0: ... /Users/filiph/dev/reactive_exploration/shared/test/smoke_test.dart                [ +153 ms] test 0: starting test /Users/filiph/dev/reactive_exploration/shared/test/smoke_test.dart
[   +9 ms] test 0: starting shell process in preview-dart-2 mode
[   +4 ms] Compiling /var/folders/p_/8567k9tn6cg7_f1bhzhhvh8c0028jd/T/dart_test_listener8vAOfG/listener.dart
00:03 +0: ... /Users/filiph/dev/reactive_exploration/shared/test/smoke_test.dart                [+3532 ms] /Users/filiph/dev/flutter/bin/cache/artifacts/engine/darwin-x64/flutter_tester --disable-observatory --flutter-assets-dir=/var/folders/p_/8567k9tn6cg7_f1bhzhhvh8c0028jd/T/flutter_bundle_directoryFRkpDQ --enable-checked-mode --enable-dart-profiling --non-interactive --use-test-fonts --packages=/Users/filiph/dev/reactive_exploration/shared/.packages /var/folders/p_/8567k9tn6cg7_f1bhzhhvh8c0028jd/T/dart_test_listener8vAOfG/listener.dart.dill
[   +6 ms] test 0: awaiting initial result for pid 25674
00:04 +0: ... /Users/filiph/dev/reactive_exploration/shared/test/smoke_test.dart                [ +507 ms] test 0: process with pid 25674 connected to test harness
[        ] test 0: awaiting test result for pid 25674
00:04 +0: - close_stream
══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
The following assertion was thrown running a test:
A periodic Timer is still running even after the widget tree was disposed.
'package:flutter_test/src/binding.dart': Failed assertion: line 668 pos 7:
'_fakeAsync.periodicTimerCount == 0'

Either the assertion indicates an error in the framework itself, or we should provide substantially
more information in this error message to help you determine and fix the underlying cause.
In either case, please report this assertion by filing a bug on GitHub:
  https://github.com/flutter/flutter/issues/new

When the exception was thrown, this was the stack:
#2      AutomatedTestWidgetsFlutterBinding._verifyInvariants (package:flutter_test/src/binding.dart)
#3      TestWidgetsFlutterBinding._runTestBody (package:flutter_test/src/binding.dart:487:7)
<asynchronous suspension>
#6      TestWidgetsFlutterBinding._runTest (package:flutter_test/src/binding.dart:464:14)
#7      AutomatedTestWidgetsFlutterBinding.runTest.<anonymous closure> (package:flutter_test/src/binding.dart:646:24)
#13     AutomatedTestWidgetsFlutterBinding.runTest (package:flutter_test/src/binding.dart:644:16)
#14     testWidgets.<anonymous closure>.<anonymous closure> (package:flutter_test/src/widget_tester.dart:62:24)
#15     Declarer.test.<anonymous closure>.<anonymous closure>.<anonymous closure> (package:test/src/backend/declarer.dart:161:27)
<asynchronous suspension>
#16     Invoker.waitForOutstandingCallbacks.<anonymous closure> (package:test/src/backend/invoker.dart:249:15)
<asynchronous suspension>
#20     Invoker.waitForOutstandingCallbacks (package:test/src/backend/invoker.dart:246:5)
#21     Declarer.test.<anonymous closure>.<anonymous closure> (package:test/src/backend/declarer.dart:159:33)
#25     Declarer.test.<anonymous closure> (package:test/src/backend/declarer.dart:158:13)
<asynchronous suspension>
#26     Invoker._onRun.<anonymous closure>.<anonymous closure>.<anonymous closure>.<anonymous closure> (package:test/src/backend/invoker.dart:403:25)
<asynchronous suspension>
#40     _Timer._runTimers (dart:isolate/runtime/libtimer_impl.dart:382:19)
#41     _Timer._handleMessage (dart:isolate/runtime/libtimer_impl.dart:416:5)
#42     _RawReceivePortImpl._handleMessage (dart:isolate/runtime/libisolate_patch.dart:165:12)
(elided 28 frames from class _AssertionError, class _FakeAsync, package dart:async, and package
stack_trace)

The test description was:
close_stream
════════════════════════════════════════════════════════════════════════════════════════════════════
00:04 +0 -1: - close_stream [E]
  Test failed. See exception logs above.
  The test description was: close_stream

[ +338 ms] test 0: process with pid 25674 no longer needed by test harness
[        ] test 0: cleaning up...
[        ] test 0: ensuring end-of-process for shell
[  +10 ms] test 0: deleting temporary bundle directory
[   +1 ms] test 0: deleting temporary directory
[   +2 ms] test 0: shutting down test harness socket server
[   +1 ms] test 0: finished
00:04 +0 -1: Some tests failed.
[  +40 ms] test package returned with exit code 1
[  +13 ms] "flutter test" took 4,812ms.

#0      throwToolExit (package:flutter_tools/src/base/common.dart:28)
#1      TestCommand.runCommand (package:flutter_tools/src/commands/test.dart:231)
<asynchronous suspension>
#2      FlutterCommand.verifyThenRunCommand (package:flutter_tools/src/runner/flutter_command.dart:333)
<asynchronous suspension>
#3      FlutterCommand.run.<anonymous closure> (package:flutter_tools/src/runner/flutter_command.dart:270)
<asynchronous suspension>
#4      AppContext.run.<anonymous closure> (package:flutter_tools/src/base/context.dart:142)
<asynchronous suspension>
#5      _rootRun (dart:async/zone.dart:1126)
#6      _CustomZone.run (dart:async/zone.dart:1023)
#7      runZoned (dart:async/zone.dart:1501)
#8      AppContext.run (package:flutter_tools/src/base/context.dart:141)
<asynchronous suspension>
#9      FlutterCommand.run (package:flutter_tools/src/runner/flutter_command.dart:261)
#10     CommandRunner.runCommand (package:args/command_runner.dart:194)
<asynchronous suspension>
#11     FlutterCommandRunner.runCommand.<anonymous closure> (package:flutter_tools/src/runner/flutter_command_runner.dart:309)
<asynchronous suspension>
#12     AppContext.run.<anonymous closure> (package:flutter_tools/src/base/context.dart:142)
<asynchronous suspension>
#13     _rootRun (dart:async/zone.dart:1126)
#14     _CustomZone.run (dart:async/zone.dart:1023)
#15     runZoned (dart:async/zone.dart:1501)
#16     AppContext.run (package:flutter_tools/src/base/context.dart:141)
<asynchronous suspension>
#17     FlutterCommandRunner.runCommand (package:flutter_tools/src/runner/flutter_command_runner.dart:265)
<asynchronous suspension>
#18     CommandRunner.run.<anonymous closure> (package:args/command_runner.dart:109)
#19     new Future.sync (dart:async/future.dart:224)
#20     CommandRunner.run (package:args/command_runner.dart:109)
#21     FlutterCommandRunner.run (package:flutter_tools/src/runner/flutter_command_runner.dart:174)
#22     run.<anonymous closure> (package:flutter_tools/runner.dart:54)
<asynchronous suspension>
#23     AppContext.run.<anonymous closure> (package:flutter_tools/src/base/context.dart:142)
<asynchronous suspension>
#24     _rootRun (dart:async/zone.dart:1126)
#25     _CustomZone.run (dart:async/zone.dart:1023)
#26     runZoned (dart:async/zone.dart:1501)
#27     AppContext.run (package:flutter_tools/src/base/context.dart:141)
<asynchronous suspension>
#28     runInContext (package:flutter_tools/src/context_runner.dart:41)
<asynchronous suspension>
#29     run (package:flutter_tools/runner.dart:49)
#30     main (package:flutter_tools/executable.dart:48)
<asynchronous suspension>
#31     main (file:///Users/filiph/dev/flutter/packages/flutter_tools/bin/flutter_tools.dart:8)
#32     _startIsolate.<anonymous closure> (dart:isolate-patch/dart:isolate/isolate_patch.dart:277)
#33     _RawReceivePortImpl._handleMessage (dart:isolate-patch/dart:isolate/isolate_patch.dart:165)

$ flutter analyze ./test/
Analyzing /Users/filiph/dev/reactive_exploration/shared/test...
No issues found!
Ran in 4.8s

$ flutter doctor -v
[✓] Flutter (Channel beta, v0.3.2, on Mac OS X 10.13.4 17E202, locale en-US)
    • Flutter version 0.3.2 at /Users/filiph/dev/flutter
    • Framework revision 44b7e7d3f4 (4 weeks ago), 2018-04-20 01:02:44 -0700
    • Engine revision 09d05a3891
    • Dart version 2.0.0-dev.48.0.flutter-fe606f890b

[✓] Android toolchain - develop for Android devices (Android SDK 27.0.3)
    • Android SDK at /Users/filiph/Library/Android/sdk
    • Android NDK location not configured (optional; useful for native profiling support)
    • Platform android-27, build-tools 27.0.3
    • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1024-b01)
    • All Android licenses accepted.

[✓] iOS toolchain - develop for iOS devices (Xcode 9.3.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 9.3.1, Build version 9E501
    • ios-deploy 1.9.2
    • CocoaPods version 1.4.0

[✓] Android Studio (version 3.1)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin version 23.2.2
    • Dart plugin version 173.4700
    • Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1024-b01)

[!] IntelliJ IDEA Community Edition (version 2018.1)
    • IntelliJ at /Applications/IntelliJ IDEA CE.app
    ✗ Flutter plugin not installed; this adds Flutter specific functionality.
    ✗ Dart plugin not installed; this adds Dart specific functionality.
    • For information about installing plugins, see
      https://flutter.io/intellij-setup/#installing-the-plugins

[✓] VS Code (version 1.22.2)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Dart Code extension version 2.9.2

[✓] Connected devices (1 available)
    • iPhone 8 • 3E8BFA02-AFFE-480A-8780-72E15DBA78C2 • ios • iOS 11.3 (simulator)

! Doctor found issues in 1 category.

// cc @brianegan because the easiest way to reproduce this is via using rxdart (but I'm pretty sure it's not a bug of that package)

In an attempt to figure out where the problem is, I wrote a few test cases to see where we might be running into trouble.
BufferTime: Normal Test vs FakeAsync vs TestWidget
In this example, I've provided the same test 3 times, using bufferTime from RxDart. Only the final test (using testWidgets fails).
  test('normal test + bufferTime passes', () async {
    new Observable.just(1)
        .bufferTime(const Duration(milliseconds: 500))
        .listen(print)
        .cancel();
  }, timeout: timeout);

  test('fakeAsync test + bufferTime passes', () {
    FakeAsync().run((fake) async {
      Observable
          .just(1)
          .bufferTime(Duration(milliseconds: 500))
          .listen(print)
          .cancel();
    });
  });

  testWidgets('testWidget + bufferTime fails', (WidgetTester tester) async {
    Observable
        .just(1)
        .bufferTime(Duration(milliseconds: 500))
        .listen(print)
        .cancel();
  });

Raw Stream.periodic: Normal Test vs FakeAsync vs TestWidget
In order to get at the heart of the problem, I've stripped away RxDart in these tests, using only the raw Stream.periodic constructor.
These tests show the same mode of failure: normal tests & FakeAsync tests pass, testWidget fails with a timeout.
  group('Stream.periodic', () {
    test('passes with normal test', () {
      expect(
        Stream.periodic(Duration(milliseconds: 100)).listen(print).cancel(),
        completes,
      );
    }, timeout: timeout);

    test('passes with FakeAsync test', () {
      FakeAsync().run((fake) async {
        expect(
          Stream.periodic(Duration(milliseconds: 100)).listen(print).cancel(),
          completes,
        );
      });
    }, timeout: timeout);

    testWidgets('testWidgets fails', (WidgetTester tester) {
      expect(
        Stream.periodic(Duration(milliseconds: 100)).listen(print).cancel(),
        completes,
      );
    }, timeout: timeout);
  });

Overall, I'm open to the idea that Rx could be at fault here, but even if we remove Rx from the equation, it looks like there's something going on in the testWidgets environment that I don't quite understand and is causing these tests to fail.

Thank you Brian!
I noticed that Flutter is using FakeAsync from pkg:quiver (and not pkg:fake_async) so I tested the second test with quiver (importing 'package:quiver/testing/async.dart') just to be sure, and that test passes as well.
So this is really something with testWidgets.

Soo... if I run the "Normal Test vs FakeAsync vs TestWidget" code against v0.3.2, it fails as you describe.  But if I run it again tip-of-tree, all three tests pass!  So I bisected, and it looks like this was fixed by https://github.com/flutter/flutter/pull/17029
@filiph are you still seeing failures against master?

I just verified that the fix in master fixes the Stream.periodic(Duration(milliseconds: 100)) failure.
I still see the original failure, but maybe that's just because I'm holding it wrong.
I'm closing Streams in my dispose() functions — but since the signature is void dispose(), I can't await the .close(). Could that be the problem? I've added a pumpAndSettle call at the end of my test in case that works, but that didn't help.

The issue is that the Flutter test framework asserts that there are no timers still pending in the fake async zone as the test body finishes.  The solution is to run your periodic streams in a real async zone:
  testWidgets('close_stream', (WidgetTester tester) async {
    await tester.runAsync(() async {
      final controller = new PublishSubject<int>();

      final subscription = controller
          .bufferTime(const Duration(milliseconds: 500))
          .listen((list) => print(list));

      await subscription.cancel();
      await controller.close();
    });
  });

  testWidgets('testWidget + bufferTime fails', (WidgetTester tester) async {
    await tester.runAsync(() async {
      Observable
          .just(1)
          .bufferTime(Duration(milliseconds: 500))
          .listen(print)
          .cancel();
    });
  });

Closing this, as a solution is available.  Please feel free to reopen if you think there's more to do here.

Thank you! This solution works.

This solution does not work for me.
I use periodic Streams in my page, which leads to fail  in the assertion _currentFakeAsync.nonPeriodicTimerCount == 0  however when I try to use real async the test fail with Unable to load asset: AssetManifest.json exception. If it's not possible or not easy to isolate the periodic streams to run in real async zone there is not way to test my code.
══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
The following assertion was thrown running a test:
A Timer is still pending even after the widget tree was disposed.
'package:flutter_test/src/binding.dart': Failed assertion: line 933 pos 7:
'_currentFakeAsync.nonPeriodicTimerCount == 0'

Either the assertion indicates an error in the framework itself, or we should provide substantially
more information in this error message to help you determine and fix the underlying cause.
In either case, please report this assertion by filing a bug on GitHub:
  https://github.com/flutter/flutter/issues/new?template=BUG.md

When the exception was thrown, this was the stack:
#2      AutomatedTestWidgetsFlutterBinding._verifyInvariants (package:flutter_test/src/binding.dart:933:7)
#3      TestWidgetsFlutterBinding._runTestBody (package:flutter_test/src/binding.dart:578:7)
<asynchronous suspension>
#6      TestWidgetsFlutterBinding._runTest (package:flutter_test/src/binding.dart:550:14)
#7      AutomatedTestWidgetsFlutterBinding.runTest.<anonymous closure> (package:flutter_test/src/binding.dart:893:24)
#13     AutomatedTestWidgetsFlutterBinding.runTest (package:flutter_test/src/binding.dart:890:15)
#14     testWidgets.<anonymous closure> (package:flutter_test/src/widget_tester.dart:71:22)
#15     Declarer.test.<anonymous closure>.<anonymous closure>.<anonymous closure> (package:test_api/src/backend/declarer.dart:168:27)
<asynchronous suspension>
#16     Invoker.waitForOutstandingCallbacks.<anonymous closure> (package:test_api/src/backend/invoker.dart:249:15)
<asynchronous suspension>
#21     Invoker.waitForOutstandingCallbacks (package:test_api/src/backend/invoker.dart:246:5)
#22     Declarer.test.<anonymous closure>.<anonymous closure> (package:test_api/src/backend/declarer.dart:166:33)
#27     Declarer.test.<anonymous closure> (package:test_api/src/backend/declarer.dart:165:13)
<asynchronous suspension>
#28     Invoker._onRun.<anonymous closure>.<anonymous closure>.<anonymous closure>.<anonymous closure> (package:test_api/src/backend/invoker.dart:399:25)
<asynchronous suspension>
#42     _Timer._runTimers (dart:isolate/runtime/libtimer_impl.dart:382:19)
#43     _Timer._handleMessage (dart:isolate/runtime/libtimer_impl.dart:416:5)
#44     _RawReceivePortImpl._handleMessage (dart:isolate/runtime/libisolate_patch.dart:171:12)
(elided 30 frames from class _AssertionError, class _FakeAsync, package dart:async, and package
stack_trace)

The test description was:
Test an invalid email
════════════════════════════════════════════════════════════════════════════════════════════════════
ERROR: Test failed. See exception logs above.

 See exception logs above.
The test description was: Test an invalid email

@guitcastro

Unable to load asset: AssetManifest.json

is probably another unrelated issue

it doesn't seem to be working for me
    testWidgets('StreamGuard should render the on data state',
        (WidgetTester tester) async {
      await tester.runAsync(() async {
        final stream =
            Stream.fromFuture(Future.delayed(Duration(milliseconds: 20)))
                .asBroadcastStream();
        final streamGuardApp = MaterialApp(
          home: StreamGuard(
            stream: stream,
            onData: (data) => homePage,
            onLoad: () => loadingPage,
            onError: (e) => errorPage,
          ),
        );
        await tester.pumpWidget(streamGuardApp);
        await tester.pump(Duration(milliseconds: 400));
        final homeFinder = find.text(home);
        final loadingFinder = find.text(loading);
        expect(homeFinder, findsOneWidget);
        expect(loadingFinder, findsNothing);
      });
    });


Update enable-controlled-folders.md

Made changes to Step 3
Fixes https://github.com/MicrosoftDocs/microsoft-365-docs/issues/11836

@JohanFreelancer9 Please copy review

@Dansimp Please merge this PR. Thanks


Feature request: Provide "skipNonTests" attribute as known from ANT 1.9 JUnit task

ANT 1.9 JUnit task provides a nice attribute called "skipNonTests". This attribute configures the JUnit task to "Do not pass any classes that do not contain JUnit tests to the test runner."
See https://ant.apache.org/manual/Tasks/junit.html
When using randomizedtesting you need to filter out classes w/o tests before providing them to randomizedtesting. Filtering is usually done using ANT pattern sets which rely on a certain naming scheme of your test classes.
Naming schemes are hard to enforce in a team with lots of people contributing to your code base.
It would be very helpful to have the same capability in randomizedtesting what ANT provides you with "skipNonTests".

This is intentional, not an oversight. When you're passing class names that are NOT tests, you need some kind of heuristic to verify whether something is a test or not. Quoting this attribute from JUnit:
"Tests are identified by looking for the @Test annotation on any methods in concrete classes that don't extend junit.framework.TestCase, or for public/protected methods with names starting with 'test' in concrete classes that extend junit.framework.TestCase. Classes marked with the JUnit4 org.junit.runner.RunWith or org.junit.runner.Suite.SuiteClasses annotations are also passed to JUnit for execution, as is any class with a public/protected no-argument suite method."
Note that these tests will require parsing the class as a bytecode or loading it from memory. To me neither is attractive -- parsing will repeatedly increase execution time and loading into memory may have other side effects (static class initializers being executed, for example).
So while technically doable, I don't think I buy the argument that it is actually a good practice to follow...

I agree on the fact about the technical impact of that flag.
However, from a business and QA perspective, a test case that does not get executed in your test automation is a loss of test coverage and eventually a loss of money if a bug does not get discovered.
If you have thousands of tests being executed per test run it happens quite quickly that you overlook that a test does not get executed.
While, in theory, it is good practice to follow a naming scheme for your test classes, in reality it is harder to enforce this policy.
I would vote for a "skipNonTests" attribute with default = “off” and a warning in the ANT doc  about its impact.
Best
Frank
From: Dawid Weiss<EMAIL_ADDRESS>Sent: Mittwoch, 27. April 2016 14:47
To: randomizedtesting/randomizedtesting
Cc: Frank Mueller; Author
Subject: Re: [randomizedtesting/randomizedtesting] Feature request: Provide "skipNonTests" attribute as known from ANT 1.9 JUnit task (#230)
This is intentional, not an oversight. When you're passing class names that are NOT tests, you need some kind of heuristic to verify whether something is a test or not. Quoting this attribute from JUnit:
"Tests are identified by looking for the @Testhttps://github.com/Test annotation on any methods in concrete classes that don't extend junit.framework.TestCase, or for public/protected methods with names starting with 'test' in concrete classes that extend junit.framework.TestCase. Classes marked with the JUnit4 org.junit.runner.RunWith or org.junit.runner.Suite.SuiteClasses annotations are also passed to JUnit for execution, as is any class with a public/protected no-argument suite method."
Note that these tests will require parsing the class as a bytecode or loading it from memory. To me neither is attractive -- parsing will repeatedly increase execution time and loading into memory may have other side effects (static class initializers being executed, for example).
So while technically doable, I don't think I buy the argument that it is actually a good practice to follow...
—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHubhttps://github.com/randomizedtesting/randomizedtesting/issues/230#issuecomment-215070765
This e-mail message and all attachments transmitted with it may contain privileged and/or confidential information intended solely for the use of the addressee(s). If the reader of this message is not the intended recipient, you are hereby notified that any reading, dissemination, distribution, copying, forwarding or other use of this message or its attachments is strictly prohibited. If you have received this message in error, please notify the sender immediately and delete this message, all attachments and all copies and backups thereof.

Feel free to provide a patch if this is your desire. I will only accept it if it uses bytecode parsing though, no loading of arbitrary classes just to inspect them via reflection.
Also, don't get me wrong, but if your organization is so hard to control then I'd really implement such checks as a separate integration-time plugin in your build system (verify that all test classes are named appropriately). This would be a better solution because it'd also clean up your code rather then just yielding to the bad habit of naming tests whatever you like...


Update repo links of Adrestia components

Update repo links of Adrestia components:

cardano-explorer
cardano-wallet
cardano-rosetta
cardano-graphql

@rphair I wrote the PR message even before I did the PR. These components moved to Cardano Foundation, so did the repos. Looks like cardano-explorer is not mentioned in the developer portal then. (maybe it should?)
Technically https://github.com/input-output-hk/cardano-rosetta is now a fork of https://github.com/cardano-foundation/cardano-rosetta

@katomm @rdlrt @os11k also combed the CIPs for these in PR referenced above, so one other fix will be appearing downstream soon.


MSI Install as User Context

Is your feature request related to a problem? Please describe.
As a Windows user without Administrative privileges it would be beneficial to have the option to install the MSI under the per-user installation context instead of per-machine
Describe the solution you'd like
Current install instructions:
Invoke-WebRequest -Uri https://aka.ms/installazurecliwindows -OutFile .\AzureCLI.msi; Start-Process msiexec.exe -Wait -ArgumentList '/I AzureCLI.msi /quiet'

Desired installation flags as per msiexec.exe property reference
Invoke-WebRequest -Uri https://aka.ms/installazurecliwindows -OutFile .\AzureCLI.msi; Start-Process msiexec.exe -Wait -ArgumentList "/I AzureCLI.msi /quiet ALLUSERS=2 MSIINSTALLPERUSER=1 EXECUTEMODE=None TARGETDIR=$env:APPDATA\azure-cli"

Describe alternatives you've considered
Installation via Python virtual environments using the below command results in an unmanageable installation with az authentication issues
python -m venv $env:APPDATA\azure-cli
cd $env:APPDATA\azure-cli\Scripts
activate
pip install --user azure-cli

Additional context
Developer machines are unfortunately very restrictive at some places of work and this will be an excellent way to provide an acceptable and manageable approach

@mozehgir can you route this request appropriately?

Have found a work around with /qb TARGETDIR=$env:APPDATA\azure-cli !!:
Invoke-WebRequest -Uri https://aka.ms/installazurecliwindows -OutFile $env:USERPROFILE\Downloads\AzureCLI.msi; Start-Process msiexec.exe -Wait -ArgumentList "/a AzureCLI.msi /qb TARGETDIR=$env:APPDATA\azure-cli /quiet"

Thanks for the help


DASH: Missing Variant Tracks for Audio Representations in a Single AdaptationSet

Have you read the FAQ and checked for duplicate open issues?
Yes
If the problem is related to FairPlay, have you read the tutorial?

Unrelated
What version of Shaka Player are you using?

4.5.0 (debug)
Can you reproduce the issue with our latest release version?
Yes
Can you reproduce the issue with the latest code from main?
Yes
Are you using the demo app or your own custom app?
Demo
If custom app, can you reproduce the issue using our demo app?
Yes
What browser and OS are you using?
Brave on Fedora
For embedded devices (smart TVs, etc.), what model and firmware version are you using?
Unrelated
What are the manifest and license server URIs?

Server URL: https://cwip-shaka-proxy.appspot.com/no_auth
What configuration are you using?  What is the output of player.getConfiguration()?

{
  manifest: {
    disableVideo: true,
    disableText: true,
    disableThumbnails: true
  }
}

What did you do?
Playing a MPD (https://storage.googleapis.com/shaka-demo-assets/angel-one-widevine/dash.mpd), with multiple representations in a single adaptation set (of id 5 and 10 in this case)
https://shaka-player-demo.appspot.com/demo/#audiolang=en-US;textlang=en-US;disableVideo=true;disableText=true;uilang=en-US;asset=https://storage.googleapis.com/shaka-demo-assets/angel-one-widevine/dash.mpd;panel=CUSTOM CONTENT;build=debug_compiled
What did you expect to happen?
player.getVariantTracks().map(i => i.originalAudioId)

To return variants with both 5 and 10 audio ids
What actually happened?

Here is an extract from the mpd
<AdaptationSet id="5" contentType="audio" lang="en" subsegmentAlignment="true">
  <ContentProtection value="cenc" schemeIdUri="urn:mpeg:dash:mp4protection:2011" cenc:default_KID="...."/>
  <ContentProtection schemeIdUri="urn:uuid:....">
    <cenc:pssh>.....</cenc:pssh>
  </ContentProtection>
  <Role schemeIdUri="urn:mpeg:dash:role:2011" value="main"/>
  <Representation id="5" bandwidth="135088" codecs="mp4a.40.2" mimeType="audio/mp4" audioSamplingRate="48000">
     <AudioChannelConfiguration schemeIdUri="urn:mpeg:dash:23003:3:audio_channel_configuration:2011" value="2"/>
     <BaseURL>a-eng-0128k-aac-2c.mp4</BaseURL>
     <SegmentBase indexRange="1042-1253" timescale="48000">
       <Initialization range="0-1041"/>
     </SegmentBase>
   </Representation>
   <Representation id="10" bandwidth="392386" codecs="mp4a.40.2" mimeType="audio/mp4" audioSamplingRate="48000">
     <AudioChannelConfiguration schemeIdUri="urn:mpeg:dash:23003:3:audio_channel_configuration:2011" value="6"/>
     <BaseURL>a-eng-0384k-aac-6c.mp4</BaseURL>
     <SegmentBase indexRange="1042-1253" timescale="48000">
       <Initialization range="0-1041"/>
     </SegmentBase>
   </Representation>
</AdaptationSet>

Only the lowest audio quality (of id 5) is available in getVariantTracks(), the higher audio quality (of id 10) is not utilized

Can confirm variants are still missing on the nightly demo page (v4.5.0-72-g2eedb12a1 (uncompiled)), just like in 4.5.0, when they have the same channel count. Worked fine in 4.4.0, so this is definitely a regression introduced since then, if I had to guess it's probably something to do with the addition of the audio groups feature.
Here is an example manifest to reproduce the issue (it's a modified version of: https://storage.googleapis.com/shaka-demo-assets/angel-one/dash.mpd):
<?xml version="1.0" encoding="UTF-8"?>
<!-- modified version of: https://storage.googleapis.com/shaka-demo-assets/angel-one/dash.mpd -->
<MPD xmlns="urn:mpeg:dash:schema:mpd:2011" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="urn:mpeg:dash:schema:mpd:2011 DASH-MPD.xsd" profiles="urn:mpeg:dash:profile:isoff-on-demand:2011" minBufferTime="PT2S" type="static" mediaPresentationDuration="PT60S">
  <BaseURL>https://storage.googleapis.com/shaka-demo-assets/angel-one/</BaseURL>
  <Period id="0">
    <AdaptationSet id="8" contentType="audio" lang="en" subsegmentAlignment="true">
      <Representation id="13" bandwidth="130803" codecs="mp4a.40.2" mimeType="audio/mp4" audioSamplingRate="48000">
        <AudioChannelConfiguration schemeIdUri="urn:mpeg:dash:23003:3:audio_channel_configuration:2011" value="2"/>
        <BaseURL>audio_en_2c_128k_aac.mp4</BaseURL>
        <SegmentBase indexRange="786-1009" timescale="48000">
          <Initialization range="0-785"/>
        </SegmentBase>
      </Representation>
      <Representation id="5" bandwidth="130432" codecs="mp4a.40.2" mimeType="audio/mp4" audioSamplingRate="48000">
        <!-- originally lang=de -->
        <AudioChannelConfiguration schemeIdUri="urn:mpeg:dash:23003:3:audio_channel_configuration:2011" value="2"/>
        <BaseURL>audio_de_2c_128k_aac.mp4</BaseURL>
        <SegmentBase indexRange="786-997" timescale="48000">
          <Initialization range="0-785"/>
        </SegmentBase>
      </Representation>
    </AdaptationSet>
    <AdaptationSet id="12" contentType="audio" lang="fr" subsegmentAlignment="true">
      <Representation id="19" bandwidth="130446" codecs="mp4a.40.2" mimeType="audio/mp4" audioSamplingRate="48000">
        <AudioChannelConfiguration schemeIdUri="urn:mpeg:dash:23003:3:audio_channel_configuration:2011" value="2"/>
        <BaseURL>audio_fr_2c_128k_aac.mp4</BaseURL>
        <SegmentBase indexRange="786-1009" timescale="48000">
          <Initialization range="0-785"/>
        </SegmentBase>
      </Representation>
      <Representation id="23" bandwidth="130582" codecs="mp4a.40.2" mimeType="audio/mp4" audioSamplingRate="48000">
        <!-- originally lang=it -->
        <AudioChannelConfiguration schemeIdUri="urn:mpeg:dash:23003:3:audio_channel_configuration:2011" value="2"/>
        <BaseURL>audio_it_2c_128k_aac.mp4</BaseURL>
        <SegmentBase indexRange="786-1009" timescale="48000">
          <Initialization range="0-785"/>
        </SegmentBase>
      </Representation>
    </AdaptationSet>
  </Period>
</MPD>

here as a data URI:
data:application/dash+xml;charset=utf-8,%3C%3Fxml%20version%3D%221.0%22%20encoding%3D%22UTF-8%22%3F%3E%3CMPD%20xmlns%3D%22urn%3Ampeg%3Adash%3Aschema%3Ampd%3A2011%22%20xmlns%3Axsi%3D%22http%3A%2F%2Fwww.w3.org%2F2001%2FXMLSchema-instance%22%20xsi%3AschemaLocation%3D%22urn%3Ampeg%3Adash%3Aschema%3Ampd%3A2011%20DASH-MPD.xsd%22%20profiles%3D%22urn%3Ampeg%3Adash%3Aprofile%3Aisoff-on-demand%3A2011%22%20minBufferTime%3D%22PT2S%22%20type%3D%22static%22%20mediaPresentationDuration%3D%22PT60S%22%3E%3CBaseURL%3Ehttps%3A%2F%2Fstorage.googleapis.com%2Fshaka-demo-assets%2Fangel-one%2F%3C%2FBaseURL%3E%3CPeriod%20id%3D%220%22%3E%3CAdaptationSet%20id%3D%228%22%20contentType%3D%22audio%22%20lang%3D%22en%22%20subsegmentAlignment%3D%22true%22%3E%3CRepresentation%20id%3D%2213%22%20bandwidth%3D%22130803%22%20codecs%3D%22mp4a.40.2%22%20mimeType%3D%22audio%2Fmp4%22%20audioSamplingRate%3D%2248000%22%3E%3CAudioChannelConfiguration%20schemeIdUri%3D%22urn%3Ampeg%3Adash%3A23003%3A3%3Aaudio_channel_configuration%3A2011%22%20value%3D%222%22%2F%3E%3CBaseURL%3Eaudio_en_2c_128k_aac.mp4%3C%2FBaseURL%3E%3CSegmentBase%20indexRange%3D%22786-1009%22%20timescale%3D%2248000%22%3E%3CInitialization%20range%3D%220-785%22%2F%3E%3C%2FSegmentBase%3E%3C%2FRepresentation%3E%3CRepresentation%20id%3D%225%22%20bandwidth%3D%22130432%22%20codecs%3D%22mp4a.40.2%22%20mimeType%3D%22audio%2Fmp4%22%20audioSamplingRate%3D%2248000%22%3E%3CAudioChannelConfiguration%20schemeIdUri%3D%22urn%3Ampeg%3Adash%3A23003%3A3%3Aaudio_channel_configuration%3A2011%22%20value%3D%222%22%2F%3E%3CBaseURL%3Eaudio_de_2c_128k_aac.mp4%3C%2FBaseURL%3E%3CSegmentBase%20indexRange%3D%22786-997%22%20timescale%3D%2248000%22%3E%3CInitialization%20range%3D%220-785%22%2F%3E%3C%2FSegmentBase%3E%3C%2FRepresentation%3E%3C%2FAdaptationSet%3E%3CAdaptationSet%20id%3D%2212%22%20contentType%3D%22audio%22%20lang%3D%22fr%22%20subsegmentAlignment%3D%22true%22%3E%3CRepresentation%20id%3D%2219%22%20bandwidth%3D%22130446%22%20codecs%3D%22mp4a.40.2%22%20mimeType%3D%22audio%2Fmp4%22%20audioSamplingRate%3D%2248000%22%3E%3CAudioChannelConfiguration%20schemeIdUri%3D%22urn%3Ampeg%3Adash%3A23003%3A3%3Aaudio_channel_configuration%3A2011%22%20value%3D%222%22%2F%3E%3CBaseURL%3Eaudio_fr_2c_128k_aac.mp4%3C%2FBaseURL%3E%3CSegmentBase%20indexRange%3D%22786-1009%22%20timescale%3D%2248000%22%3E%3CInitialization%20range%3D%220-785%22%2F%3E%3C%2FSegmentBase%3E%3C%2FRepresentation%3E%3CRepresentation%20id%3D%2223%22%20bandwidth%3D%22130582%22%20codecs%3D%22mp4a.40.2%22%20mimeType%3D%22audio%2Fmp4%22%20audioSamplingRate%3D%2248000%22%3E%3CAudioChannelConfiguration%20schemeIdUri%3D%22urn%3Ampeg%3Adash%3A23003%3A3%3Aaudio_channel_configuration%3A2011%22%20value%3D%222%22%2F%3E%3CBaseURL%3Eaudio_it_2c_128k_aac.mp4%3C%2FBaseURL%3E%3CSegmentBase%20indexRange%3D%22786-1009%22%20timescale%3D%2248000%22%3E%3CInitialization%20range%3D%220-785%22%2F%3E%3C%2FSegmentBase%3E%3C%2FRepresentation%3E%3C%2FAdaptationSet%3E%3C%2FPeriod%3E%3C%2FMPD%3E

Please test it tomorrow, the nightly is not updated yet

Can confirm that the issue persists on v4.5.0-76-gc2c832025 (uncompiled) with the manifest I provided. It seems like the audio channels change was just a workaround that fixes the specific manifest in the original issue. The manifest I provided works fine on 4.4.0, but is missing variants with 4.5.0 and the nightly demo.


Extension of flexibility in federated learning scenarios

Background Information:
My team and I (from Hasso Plattner Institute) will make a new MOOC lecture on the https://ki-campus.org/ platform. I want to use Sedna to design two practical tasks in my lecture: federated learning and cloud-edge collaborative inference.  Given our current practice environment, we hope to get extended support from Sedna.
Why is this needed:
Based on our current scenario settings, it is impossible to provide a suitable environment completely on the cloud for thousands of students. Therefore, it is hoped that Sedna can support the independent operation of the lib library and support the dynamic access of federated learning workers.
Value for customer:
Germany and the EU have extremely strict protection of data privacy, and Federal Learning can be used for developing machine learning applications under the premise of protecting privacy and can improve the overall accuracy for all parties.
Value for Sedba:
MOOC online lecture usually has a large number of participants. Including not only students but also a large number of practitioners from the industry. Therefore, Sedna can be well promoted through the course. Enhance its industry influence.
In addition, the dynamic joining of edge workers is a very promising feature for other scenarios as well, which can be designed as a general capability for Sedna federation learning.

GM/LC shield lots of things that algorithm developers don't need to pay much attention to. It is feasible to run the sedna lib library independently. However, problems such as network engineering (for example, you need to open ports) and log monitoring (for locating exceptions) may be introduced, and I'm not sure it's up to the community to block.
What my suggestion is, Sedna can provide an image with development status (like a notebook) that integrates relevant components and environments. Students only need to pull and run the image to connect to the sedna service (like aggregate worker) that teachers have deployed on the cloud in advance.

Thank you for the comments!
I want to explain further. Due to the scale of the course (up to 10k participants) and the control of the learning quality (maybe most people do not have enough computing power for DL model training at home), we will ask students to use Kaggle notebooks to complete practical assignments. Kaggle notebook can provide all participants with a free GPU quota to complete tasks. However, the Kaggle notebook cannot be used with the current Sedna. We hope to use Sedna to do the federal learning task with up to 10k clients in this course, and we also hope to promote Sedna in Germany.

Another limitation of the current Sedna is as mentioned in my first post. We cannot ask up to 10k clients to start the federated learning task at roughly the same time, due to its large number and the heterogeneous distribution regarding time and geography. So dynamic and asynchronous federated learning interactions are also needed.


Runtime property is missing for 'dotnet pack', 'dotnet run' and 'dotnet clean'

The dotnet pack, run, clean settings are missing the RunTime Property which is needed for RID build.
e.g.
https://github.com/cake-build/cake/blob/develop/src/Cake.Common/Tools/DotNetCore/Clean/DotNetCoreCleanSettings.cs
dotnet clean -h
Usage: dotnet clean [options] <PROJECT>

Arguments:
  <PROJECT>   The project file to operate on. If a file is not specified, the command will search the current directory for one.

Options:
  -h, --help                            Show command line help.
  -o, --output <OUTPUT_DIR>             The directory containing the build artifacts to clean.
  -f, --framework <FRAMEWORK>           The target framework to clean for. The target framework must also be specified in the project file.
  -r, --runtime <RUNTIME_IDENTIFIER>    The target runtime to clean for.
  -c, --configuration <CONFIGURATION>   The configuration to clean for. The default for most projects is 'Debug'.
  -v, --verbosity <LEVEL>               Set the MSBuild verbosity level. Allowed values are q[uiet], m[inimal], n[ormal], d[etailed], and diag[nostic].

#2556


Atom Not Responding

I use ZSH and I have my paths set to the following
export GOPATH=$HOME/Projects/golang
export GOROOT=/usr/local/opt/go/libexec
export PATH=$PATH:$GOPATH/bin
export PATH=$PATH:$GOROOT/bin

However, Atom just hangs when it tries to install go-plus. I think I isolated it to go-config as the culprit. I deleted the package and started Atom. It loaded fine, but as soon as it started to install go-config it hangs again. I can't even run the debugger to find the root cause.
Installing

Running $ atom --profile-startup .

Any ideas?

Lets see if it is actually caused by go-config itself or another package (installed with go-plus) which uses go-config.
Uninstall/remove all packages installed by go-plus (see https://github.com/joefitzgerald/go-plus/blob/v4.3.0/package.json#L43-L55) and then install only go-config. If go-config has been installed without an error or freezing atom -> install go-plus.
If go-plus fails try to disable all packages installed by go-plus except go-config (probably using apm) and enable them one by one to see which causes atom to hang.

So interesting turn of events. I did apm install PACKAGE in order from those lines you indicated and quit and started atom each time. Both gofmt and tester-go caused atom to freeze on boot.
I had go-plus installed prior and started deleting them one by one and restarting atom and as soon as I deleted go-config it worked so I'm not sure who the culprit is.
I deleted all packages and just installed gofmt and atom booted fine and started to install go-config and go-get, but it then hangs on from within atom. I can install them all fine using apm, but atom still hangs on boot.

I can install them all fine using apm, but atom still hangs on boot.

Because apm only downloads them into the .atom/packages folder. They freeze atom because they get executed in atom. This freeze is most likely caused by go-config as deleting unfreezes atom.

Both gofmt and tester-go caused atom to freeze on boot

Great! So we now have an idea where to start investigating!


entityClass constructor doesn't take parent type properties into account when using schema.org

Unfortunately I don't have a good repro for this, but the entityClass method doesn't seem to take properties of parent types into account.
My rawData looks something like this:
{description: "Shop at the Google Store for Android phones and ta…hromecast, Nest, plus a selection of accessories.", offers: Array(13), name: "Pixel"}
The description element is inherited by http://schema.org/Product from http://schema.org/Thing. However, in Schema.entityClass() (~schema.js:67), properties is:
["color", "gtin12", "gtin8", "model", "gtin14", "gtin13", "sku", "award", "mpn", "productID", "material", "category", "awards", "logo"]
These are only direct properties of schema.org/Product.

The codepath that I reached this on was using JSON.stringify(o), where 'o' is an instance of the Product entity.

I don't see the issue when using schema defined in our manifest files.

This was an interesting one. There seems to be a bug in the way schema.org generates structured representations - we were loading Thing correctly but it didn't contain anything useful.
I've worked around the issue for now by reconstructing Thing from the representation of Product.


Uart init

Fixes #1195 - doesn't include P4 since we don't have UART for P4, yet. Should be part of the driver implementation for that chip

Clippy fails with because of #1210

I will also do a couple of tests with the flasher stub to make sure we didn't break anything there.

Oh, that's good - somehow forgot about the flasher stub


Add peek and pop features

Peek and pop features are implemented in MoviesVC
and SearchMoviesVC to preview the detail view. It also
adds some quick actions to move a movie to a specific
list and delete a movie from a list.

#21


Add PlotSVCountsPerSample subworkflow to the end of ClusterBatch and FilterBatchSites

Addressing issue 544 here.
PlotSVCountsPerSample.wdl was imported into ClusterBatch.wdl and FilterBatchSites.wdl. Then the N_IQR_cutoff input to the workflow with a default value of 6 was added to both wdls. PlotSVCountsPerSample is called as a subworkflow at the end of each workflow, passing the final VCF's as the input and the batch as the prefix. The outputs of PlotSVCountsPerSample were added to each workflow's outputs with unique names. Tthe JSON templates for ClusterBatch and FilterBatchSites in test and terra were updated to include the N_IQR_cuttoff input with a value of 6. Then the ClusterBatch and FilterBatchSites wofkflows were validated with womtool and the Terra validation script, and ran on the ref_panel_1kg test data. There was successful completion and decent outputs.

@kirtanav98 Thank you for your valuable work! Integrating PlotSVCountsPerSample into ClusterBatch and FilterBatchSites is a great time-saving enhancement. Users will benefit from the convenience of having fewer workflows to run. Additionally, the standalone version of PlotSVCountsPerSample in the Terra workspace allows users to run it separately if needed.
The performed tests and the PR descriptions both look good to me.

Thanks for all your hard work on this, it's looking really good! A couple of small requests and then you should be good to go.
Did you test ClusterBatch with and without the N_IQR_cutoff_plotting input to check that it enables and disables plotting as expected?

Yes, using cromshell both situations seem to work as expected. The location of the input was also moved to a more appropriate location in the wdl as requested.

Addressing issue 544 here.
PlotSVCountsPerSample.wdl was imported into ClusterBatch.wdl and FilterBatchSites.wdl. Then the N_IQR_cutoff input to the workflow with a default value of 6 was added to both wdls. PlotSVCountsPerSample is called as a subworkflow at the end of each workflow, passing the final VCF's as the input and the batch as the prefix. The outputs of PlotSVCountsPerSample were added to each workflow's outputs with unique names. Tthe JSON templates for ClusterBatch and FilterBatchSites in test and terra were updated to include the N_IQR_cuttoff input with a value of 6. Then the ClusterBatch and FilterBatchSites wofkflows were validated with womtool and the Terra validation script, and ran on the ref_panel_1kg test data. There was successful completion and outputs were as expected.


Electron support with auto-spawning

Should come after #35 support so that the daemon can be auto-spawned by electron apps

Closing this unless it comes up again


update version string from 0.5 to 1.0 after release

WIP
Needs some clarification about a few things

assets used by the static site generator: ignore until the site generator is removed?
modelbench/src/modelbench/templates/content_mlc/general_purpose_ai_chat_benchmark.toml and a couple other files have scopes named general_purpose_ai_chat_benchmark-0_5. Should those be replaced altogether?

https://github.com/mlcommons/modelbench/issues/733

Closing until we have a group discussion about how this should be handled.


fix(logging): weird settings import

:tada: This PR is included in version 1.13.7 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


MaybeUninit::uninit().assume_init() is undefined behavior

https://github.com/PyO3/pyo3/blob/83d0ac4e70f1c622925470ba234b23cfafaa7d35/src/gil.rs#L303-L306
This is explicitly noted in the MaybeUninit docs as undefined behavior (std::mem:;unintialized was deprecated for a reason!).  The correct solution is to store [MaybeUninit<T>; BLOCK_SIZE] in the type, and to only call assume_init() on individual items that are known to be initialized.
(also unrelated:  the current implementation of ArrayList<T> is not safe for T that have destructors due to its use of indexed assignment instead of ptr::write.  It should have a T: Copy bound to reflect this)

To make matters even worse, you're using it for NonZero<*mut T>, a type which has invalid bit patterns!
(not that that makes a difference; mem::MaybeUninit::uninit().assume_init() is equally UB for all types!  I just found this... interesting)

Currently it's used just to avoid warnings for mem::uninitialized so yeah, maybe we should fix this.
But hmm :thinking: , now I have no good idea to make this module sound.

Good catch, thanks.

Closed via #537 for now


ci: update DAGGER_CLOUD_TOKEN

Updates our special telemetry-only token to use the new format, so we don't get constantly prompted to rotate our creds.
Thanks @marcosnils for the token :tada:

excellent


Re-create of container with latest image does not update healthcheck

Bug description
Asumme the following:

You created a container via "Containers->add container". The image used has a healthcheck in its dockerfile.
The image maintainer releases a new image with a new healthcheck
Navigate to your container. Click recreate.
Click re-pull image to get the latest image with the tag (simply assume the default :latest tag)
The result is a container that still has the old healtcheck.

Same behaviour if you use "dublicate/edit" instead of re-create and switch the image.
A specific example is a minidlna image from a maintainer called vladgh. You may be able to reproduce the issue with this image:

Create a container with Image "vladgh/minidlna:1.3.0". No need to change any options, ports, volumes, etc.
You may check docker inspect

docker inspect <container-id> | grep curl
                    "curl --silent --fail localhost:8200 || exit 1"

After it is running, go to the "Duplicate/Edit" option.
Only change Image to "vladgh/minidlna:1.3.3" (or latest)
Deploy the container with the same name (or even a new name, the healthcheck is not updated in both scenarios)
The health check is still the old one. You can see it as follow:
You will see the container will be shown as unheathy because the new image requires a new heathcheck.
You may inspect image vs container with docker inspect as well.
You will see a message "upnphttp.c:938: error: DNS rebinding attack suspected (Host: localhost:8200)" in the container log. This is a result of the old healthcheck.

Expected behavior
In general, new dockerfile / image options should be updated with the pull of a new image.
This should be independent of supported features by portainer (e.g. to display or modify healthcheck).
This may not be linited to the healthcheck value alone, but it is reproducible with it.
This is why I assume it is a bug and it is not a dublicate of the open issues/feature requests to support healthcheck views and modification in portainer.
Steps to reproduce the issue:
see "Bug description"
Technical details:

Portainer version: 2.17.1
Docker version (managed by Portainer): Docker version 23.0.1, build a5ee5b1
Kubernetes version (managed by Portainer): n/a
Platform (windows/linux): Debian 11
Command used to start Portainer (docker run -p 9443:9443 portainer/portainer): Done by OpenMediaVault/NAS software
Browser: independent
Use Case (delete as appropriate): Using Portainer at Home.
Have you reviewed our technical documentation and knowledge base? Yes, I assume, but may not find the correct notice...

Maybe related:
https://github.com/portainer/portainer/issues/8513
The issue sounds similar, instead of the HEALTHCHECK option of the image, the ENTRYPOINT option of the old image is kept

@n00ris
Thank you for the additional information.  Can you attach your Dockerfile?
Thanks!

Hello @tamarahenson , I'm unsure what you mean. I did not build the image myself.
Are you looking for the dockerfile the maintainer used to create the image?
It is here:
https://github.com/vladgh/docker_base_images/blob/main/minidlna/Dockerfile

We are also seeing this issue with self-built containers on the business edition.


Hold DNS entries sorted case-insensitively until just before sending

Instead of sorting when creating the outbound GossipData object, we sort in Encode().
Fixes #1603 and #1610
Since the extended unit tests revealed that it is expensive to re-compute ToLower() every time we look at a hostname, this PR also changes this to compute the lower-case version on entry to the data structure and hold on to it thereafter.
(replacement for #1611 which was against the wrong branch)

Yay! Unit tests passed!


User Home Page Layout initial commit

Description
Please include a summary of the change and which issue is fixed. Include relevant motivation and context. List any dependencies that are required for this change.
Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[X] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

How Has This Been Tested?
Please describe the tests that you ran to verify your changes.
Please also list any relevant details for your test configuration.

[ ] Unit Test
[ ] Postman Test
[ ] Integration Test
[ ] End-to-end Test (Frontend + Backend)

Checklist:
Before you submit your Pull Request, please check you have completed the following:

[ ] I have added tests that prove that my feature works or my fix is effective.
[X] I have commented my code, particularly in hard-to-understand areas.
[ ] I have made corresponding changes to the documentation.
[ ] My changes generate no new warnings.
[ ] Any dependent changes have been merged and published in downstream modules.

Additional Notes
Add any other details about the PR here.

Any tests done on this PR?

Any tests done on this PR?

Not yet, will do later


建议增加type:line时sampling配置

[ ] I am using English in this issue. 在这个 Issue 中我使用了英文（强烈建议）。

General Questions

[x] Required: I have read the document and examples and tried to solve it by myself. （必填）我读过了文档和教程，并且曾试图自己解决问题。
[x ] Required: I have searched for similar issues and that didn't help. （必填）我搜索过 issue 但是没有帮助。
[x ] Required: I have tried with the latest version of ECharts and still have this problem. （必填）我试过最新版本的 ECharts，还是存在这个问题。

In this issue, I have provided information with: 在这个 issue 中我提供了以下信息：

[x ] Required: issue type；（必填）issue 类型
[x ] Required: one sentence description in issue details；（必填）一句话的问题描述
[ ] Required: demo；（必填）能反映问题的例子（如果你想提问或报 bug）
[ ] Required: ECharts version；ECharts 版本

Issue Type

[x ] I have a question to ask about how to use ECharts to ...；我想提问如何使用 ECharts 实现某功能
[ ] I have a bug to report；我想要报 bug
[ ] I have a feature to request, e.g.: I'd like a new feature that ...；我需要一个新功能
[ ] I have a feature to enhance, e.g.: The current feature should be improved in the way that ...；我希望改进某个功能
[ ] There's something wrong with the documents；文档有些问题
[ ] Others, or I'm not sure which issue types to choose here；其他，或我不知道应该选什么类型

Issue Details

当前type=line时有个sampling配置 可取min max average ，只能选择其中一个。但是我想同时取最小和最大怎么办？ 最值点对于我的绘图是非常重要的特征点 如果我取min 会导致max丢失,导致绘图不准。
Expected Behavior

期待有个配置能够保留最大和最小点，或者支持一个回调函数，让用户自定义采样规则

Current Behavior

Online Example

Topics

[ ] Legend
[ ] Tooltip
[ ] Event
[ ] Performance
[ ] SVG
[ ] Map
[ ] ECharts GL
[ ] Third-party libraries, e.g.: Vue.js, React

Anything Else We Need to Know

Environment

ECharts version；ECharts 版本: {DESCRIBE_HERE}

[ ] It happens only on certain browsers or operating systems. 对于特定浏览器或操作系统才会出现的问题，请提供相应环境信息：{BROWSER_VERSION_OR_OS_INFORMATION_HERE}

sampling  can be a callback function. For example
sampling: function (windowData) {
  return windowData[0];
},

sampling can be a callback function. For example
sampling: function (windowData) {
  return windowData[0];
},

i want to return the maximum and minimum values at the same time，can i use this way?


Feat : Add download update feature in user app. Issue #755

/claim #755
What kind of change does this PR introduce?
Feature addition
Summary
Currently we don't support app update functionality in our app. With this addition, user can get the latest version of app from the app-tray itself.

App shows latest version to download if available
Else it shows no update found.

Video
https://github.com/OpenAdaptAI/OpenAdapt/assets/48802836/d3058697-5e03-4f9a-ad4a-8cb33930fe87
Checklist

[ ] My code follows the style guidelines of OpenAdapt
[ ] I have performed a self-review of my code
[ ] If applicable, I have added tests to prove my fix is functional/effective
[ ] I have linted my code locally prior to submission
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation (e.g. README.md, requirements.txt)
[ ] New and existing unit tests pass locally with my changes

How can your code be run and tested?

Other information

@shashank40 this a great start!
In the comment you linked, the third step is:

On clicking that, the newest version should be downloaded, and unzipped in the same directory as where the current app lives. There are no additional installation instructions for OpenAdapt, so we could just display a message box saying that the new app was downloaded, and maybe delete the current version as well (@abrichr thoughts on this as well)

If you implement this we can consider this. Keep going!

@abrichr
Just a thought!
According to me unzipping in the download directory  better than current directory as people generally look for downloads in the downloads folder rather than any random directory)? (Your thoughts @abrichr?)
I think that is the only part left in the comment.
But i will implement the unzipping part for sure.

Also @abrichr
i dont think we should implement deleting the old version.
Reason : what if the user is in a middle of the recording during download? And we delete this old build, this is not ideal
I think extracting the zip is the best thing we should do and not the deleting part.
Thoughts? @abrichr

@shashank40 typically how applications handle this is to say something like:

A new version is available: vxx.xx.xx (you are on vyy.yy.yy). Would you like to update? Yes/No/Skip this version

We can show a message on startup indicating that a new version is available, and let the user decide how to proceed.

@abrichr
Yess i agree, but isn't this similar to what is implemented but better?. Rather than popup we have it in a tray with the version to update visible.
User can choose to implement and not to implement.
Why i say this is, as OpenAdapt gets latest updates quite frequently(sometimes even 2-3 a day) and not weekly/monthly. So let's say i update it once and then few hours later there would be another popup hindering me.
This would be a bad user experience every time i open an app.
The implementation that you suggested might work better for apps/softwares that have scheduled updates that come weekly/monthly. But with these frequent updates, our user experience will go down.

It's an interesting point @shashank40. But I still think we want to make the user's life easier by automatically extracting the application for them if they do choose to update.

@abrichr You mean extracting the zip file?

@abrichr Done.
Added the functionality to unzip as well

@shashank40 I'm unable to test this on my machine because of the following:
136272 INFO: Converting EXE to target arch (arm64)                                                                                09:37:57 [256/80932]
136496 INFO: Removing signature(s) from EXE
136569 INFO: Appending PKG archive to EXE
136682 INFO: Fixing EXE headers for code signing
136684 INFO: Re-signing the EXE
137030 INFO: Building EXE from EXE-00.toc completed successfully.
137265 INFO: checking COLLECT
137265 INFO: Building COLLECT because COLLECT-00.toc is non existent
137265 INFO: Building COLLECT COLLECT-00.toc
Traceback (most recent call last):
  File "/Users/abrichr/Library/Caches/pypoetry/virtualenvs/openadapt-VBXg4jpm-py3.10/bin/pyinstaller", line 8, in <module>
    sys.exit(_console_script_run())
  File "/Users/abrichr/Library/Caches/pypoetry/virtualenvs/openadapt-VBXg4jpm-py3.10/lib/python3.10/site-packages/PyInstaller/__main__.py", line 228, in _console_script_run
    run()
  File "/Users/abrichr/Library/Caches/pypoetry/virtualenvs/openadapt-VBXg4jpm-py3.10/lib/python3.10/site-packages/PyInstaller/__main__.py", line 212, in run
    run_build(pyi_config, spec_file, **vars(args))
  File "/Users/abrichr/Library/Caches/pypoetry/virtualenvs/openadapt-VBXg4jpm-py3.10/lib/python3.10/site-packages/PyInstaller/__main__.py", line 69, in run_build
    PyInstaller.building.build_main.main(pyi_config, spec_file, **kwargs)
  File "/Users/abrichr/Library/Caches/pypoetry/virtualenvs/openadapt-VBXg4jpm-py3.10/lib/python3.10/site-packages/PyInstaller/building/build_main.py", line 1139, in main
    build(specfile, distpath, workpath, clean_build)
  File "/Users/abrichr/Library/Caches/pypoetry/virtualenvs/openadapt-VBXg4jpm-py3.10/lib/python3.10/site-packages/PyInstaller/building/build_main.py", line 1079, in build
    exec(code, spec_namespace)
  File "OpenAdapt.spec", line 40, in <module>
    coll = COLLECT(
  File "/Users/abrichr/Library/Caches/pypoetry/virtualenvs/openadapt-VBXg4jpm-py3.10/lib/python3.10/site-packages/PyInstaller/building/api.py", line 1092, in __init__
    self.__postinit__()
  File "/Users/abrichr/Library/Caches/pypoetry/virtualenvs/openadapt-VBXg4jpm-py3.10/lib/python3.10/site-packages/PyInstaller/building/datastruct.py", line 184, in __postinit__
    self.assemble()
  File "/Users/abrichr/Library/Caches/pypoetry/virtualenvs/openadapt-VBXg4jpm-py3.10/lib/python3.10/site-packages/PyInstaller/building/api.py", line 1138, in assemble
    src_name = process_collected_binary(
  File "/Users/abrichr/Library/Caches/pypoetry/virtualenvs/openadapt-VBXg4jpm-py3.10/lib/python3.10/site-packages/PyInstaller/building/utils.py", line 309, in process_collected_binary
    osxutils.binary_to_target_arch(cached_name, target_arch, display_name=src_name)
  File "/Users/abrichr/Library/Caches/pypoetry/virtualenvs/openadapt-VBXg4jpm-py3.10/lib/python3.10/site-packages/PyInstaller/utils/osx.py", line 351, in binary_to_target_arch
    raise IncompatibleBinaryArchError(
PyInstaller.utils.osx.IncompatibleBinaryArchError: /Users/abrichr/Library/Caches/pypoetry/virtualenvs/openadapt-VBXg4jpm-py3.10/lib/python3.10/site-packages/sqlalchemy/cresultproxy.cpython-310-darwin.so is incompatible with target arch arm64 (has arch: x86_64)!
Traceback (most recent call last):
  File<EMAIL_ADDRESS>line 816, in move
    os.rename(src, real_dst)
FileNotFoundError: [Errno 2] No such file or directory: '/Users/abrichr/oa/OpenAdapt/dist/OpenAdapt.app/Contents/MacOS/OpenAdapt' -> '/Users/abrichr/oa/OpenAdapt/dist/OpenAdapt.app/Contents/MacOS/OpenAdapt.app'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/abrichr/oa/OpenAdapt/openadapt/build.py", line 180, in <module>
    main()
  File "/Users/abrichr/oa/OpenAdapt/openadapt/build.py", line 164, in main
    shutil.move(
  File<EMAIL_ADDRESS>line 836, in move
    copy_function(src, real_dst)
  File<EMAIL_ADDRESS>line 434, in copy2
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File<EMAIL_ADDRESS>line 254, in copyfile
    with open(src, 'rb') as fsrc:
FileNotFoundError: [Errno 2] No such file or directory: '/Users/abrichr/oa/OpenAdapt/dist/OpenAdapt.app/Contents/MacOS/OpenAdapt'

Can you please record a video showing the updated functionality?

@shashank40 if you can upload a video showing this working, and fix the formatting issues causing the build to fail, then I think we can get this merged!

Surely @abrichr , will update this in sometime. Currently AFK.

Hi @abrichr.
I have replied back to the comments. I am also adding a new video here.
In the video you can see the latest version now shows as 0.34.0 and in yesterday's video it was 0.33.2 and so this makes the app quite reactive to new updates. That will be the beauty of this new update. Dynamically telling you the latest version available.
The video is mostly download, to show you the full download(you can move that part ahead). At the end it shows how new file is unzipped.
https://github.com/OpenAdaptAI/OpenAdapt/assets/48802836/c9a4cd16-44a1-4ad2-9aa9-c5a5ac8421b0

Hey @abrichr  can we go ahead and merge this?

@abrichr any updates on this?

@abrichr any update on this?

Hi @shashank40 , this looks good. However it's unfortunate that there is no visual indicator for the user about the progress, other than what is in the terminal (users who run the app not from the terminal won't see any progress).
I think we need some sort of status indicator, e.g. via show_toast. What do you think?

Yess @abrichr , I can add a toast after the zipping is complete.
Will do it in right now.

@abrichr Showing download complete at the end would fix the issue i suppose?

@abrichr As the video would get too long to upload, i am adding 2 parts of the video.
Hope this is fine?

Shows how the toast would popup with approximate time to download.

https://github.com/OpenAdaptAI/OpenAdapt/assets/48802836/939da167-3ec2-4d8f-a6ff-4aa293072655

Directly shows the end part , where after download a toast pops up with download complete

https://github.com/OpenAdaptAI/OpenAdapt/assets/48802836/a47d0708-4458-4563-a9f5-e185acf2187e

@shashank40 this is not bad but I think we can do better.
Can you please add a callback that clears the current toast, and updates it with a new one with an updated ETA?

@abrichr I initially did that, but as our main thread will be busy doing much more important things like recording/visualize, making main thread do regular UI upgrades(every few milliseconds) is degrading performance as main thread chokes.
I am okay with updating the time every few seconds, but i think giving an approximate at the ends works okay and does not hinder with more important tasks

No need to update every few milliseconds, but I believe every 1-10s would be much more usable at minimal performance cost. 🙏

@abrichr Okay, so i will do a new toast pop-up every 2 secs with updated time estimate? Will that be fine?

At the same time be sure to clear the previous toast so that we don't ever have more than one toast showing regarding the update progress. Ideally it looks like the same toast, so remove any animations etc.

Yess @abrichr i made sure of that. Here is the video.
Hopefully now we can get it merged. It's been too long now 😅
https://github.com/OpenAdaptAI/OpenAdapt/assets/48802836/68819acd-b467-43a9-9f1d-3982082d3209

Hi @shashank40 this is looking good! I haven't merged it yet because I saw you pushed a few commits after your last comment. Please fix the merge conflicts and let me know when this is ready for re-review, would love to get this merged! 🙏

@abrichr I did all the required changes. Removed the conflict.
Hopefully this now looks all good.

@abrichr Any update?

Hi @shashank40 , thank you for your patience,
I tried running this on my machine:
open_adapt-v0.36.0.app.zip:  31%|██████████████████████████████▍                                                                   | 9.00/29.0 [00:00<00:00, 75.8kB/s]
Exception in thread Thread-5 (download_latest_version):
Traceback (most recent call last):
  File<EMAIL_ADDRESS>line 1016, in _bootstrap_inner
    self.run()
  File<EMAIL_ADDRESS>line 953, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/abrichr/oa/OpenAdapt/openadapt/app/tray.py", line 297, in download_latest_version
    unzip_file(local_filename)
  File "/Users/abrichr/oa/OpenAdapt/openadapt/update_utils.py", line 19, in unzip_file
    shutil.unpack_archive(file_path, os.path.dirname(file_path))
  File<EMAIL_ADDRESS>line 1314, in unpack_archive
    func(filename, extract_dir, **kwargs)
  File<EMAIL_ADDRESS>line 1201, in _unpack_zipfile
    raise ReadError("%s is not a zip file" % filename)
shutil.ReadError: /Users/abrichr/Downloads/open_adapt-v0.36.0.app.zip is not a zip file

Any suggestions?

@abrichr Any more changes that might expect ?
I would complete it today so that we can get this pending PR in.

Hi @shashank40 , there should not be any more changes. I just noticed I introduced the latest bug with my changes 😅 will fix.

Yes @abrichr i saw that. It was actually working fine for me till that commit.

Even now works fine for me

@abrichr I have added started unzipping toast.
I have also added the check if i am on the latest version, menu action will be disabled.

@abrichr Please have a look at the latest code.

Hi @abrichr
Made all the specified changes. Is this better?

@abrichr any update?

@abrichr Can you please do a final review?

Thank you @shashank40. There appear to be a few loose ends:

The "extracting" toast disappears, and there is no indication to the user that anything is happening. Ideally we show progress on this as well, e.g. using the zip_file module:

def extract_with_progress_zip(zip_path: str, extract_to: str):
    with zipfile.ZipFile(zip_path, 'r') as zip_ref:
        total_files = len(zip_ref.infolist())
        for i, file in enumerate(zip_ref.infolist()):
            zip_ref.extract(file, extract_to)
            logger.info(f"Extracted {i + 1} of {total_files} files ({(i + 1) / total_files * 100:.2f}%)")
            # TODO: update toast, like with downloading

Once unzipping is complete, the user will have more than one copy of OpenAdapt on their hard drive. Ideally we:
a. Move the newly extracted version to the same directory as the old version.
b. Ask the user whether they would like to remove the previous version, and if so then remove it*.

Finally, I would expect that:
a. the currently running of OpenAdapt is shut down
b. the newly extracted version's executable is executed

The removing will probably need to be done on the first launch of the new version.

I would like to merge this now, but as it currently stands, it's not very user friendly without these additions, i.e. once the extraction is complete, the user likely expects to be running the new version, even though currently they won't be.
Once these additions have been implemented, the update feature will be complete and we can merge this.
Thank you @shashank40 , almost there!

@abrichr I think we should not do part 1), reason is extraction takes 5-10secs. Now we already show toast when it starts and when extraction completes. Now the overhead and resource consumption to just show extraction time(which in turn is shown by 2 different toast) seems excessive.
@abrichr i would request you to please review the PR all at once if you can. We have been going too and fro for too long and every time a new commit is pushed, there is a new change requested on the old code itself which could've been done earlier. It would make things easier for both of us.
Also to add, the new features requested is a bit out of scope of the initial aim of the project. I mean i am more than happy to do it but 3 weeks of too and fro and a new feature addition for just $100 bounty is not worth the effort.
If we can get the current PR in and then add these things as a new feature, we can do that.

Thank you @shashank40 , I appreciate you presenting an alternative perspective.
Regarding extracting the zip file: on my machine, it can take a while. The notification disappears before the extraction is complete, then it's not clear whether the process is complete or not. I think it's important we implement similar logic as for downloading to indicate the update is still occurring.
Regarding not removing the old application, the intention here is to emulate the behavior of other applications that update automatically. For example, when I update Chrome, I don't have to remove the old directory.

@abrichr When comparing it to chrome, browsers are installed applications which have their own directory on the hard-drive and change required files when updated. So they never touch any other directory/downloaded file. Even if we download a multiple zips for chrome, they are never deleted by chrome.
So installed apps work very differently in how updates work and how files are changed in their installed directory locations.
OpenAdapt is a desktop app and not and installed app. It doesn't have a fixed directory like installed apps and so deleting files from any other directory needs higher level permissions and makes it unsafe. If OpenAdapt can delete files from any directory, it makes it unsafe.

@abrichr updates?

Hi @shashank40 , thank you for your patience.

OpenAdapt is a desktop app and not and installed app.

We've implemented installers in https://github.com/OpenAdaptAI/OpenAdapt/pull/858. We would like the update functionality to work with the installed app.


Update the Back Office API Catalog

Description:
Update the Back Office API Catalog based on the changes done in [1]
[1] https://github.com/wso2/apk/pull/819

This has been done in https://github.com/wso2/docs-apk/pull/103
Live URL https://apk.docs.wso2.com/en/latest/catalogs/api-reference-back-office/


not compiled with CUDA error...solved?

I've encountered the same error that's outlined on in your readme, namely, the not compiled with cuda error.  Here's an example: Several users have reported it and all have been using Macs.  Here's one example:
IMAGE OF CONVO~~

According to this user, he solved the issue by commenting out a few lines of code...
IMAGE OF CONVO~~

This snippet he modified pertains to a user selecting "cuda" from a pulldown menu as a compute device...at which time my script populates another pulldown with the available quantizations from the ctranslate2 library per this class here
I've since revised the code to remove this feature, but still have ctranslate2 check for supported quantizations at initialization (BOTH for cpu and cuda...not merely cuda if it's available), and it solved the issue for yet another user...and haven't had any complaints since.
Not sure if it helps, but it might have something to do with the program trying to either use a cuda device when it should use mps or cpu...or if not explicitly use the device (e.g. in inference) simply run a command that is specific to cuda...Not sure if Ctranslate2's class that prompted this error is using PyTorch under the hood, but if it is, that might pinpoint the issue...Hope this helps!

You should be passing a device mps on a Mac. Mac doesn't support CUDA.
Ref: https://github.com/Vaibhavs10/insanely-fast-whisper#cli-options

I am aware that to use gpu-accleration on mac you set the device to "mps".    What I'm saying is...on the github readme for this repository it states:
{BEGIN QUOTE]
How to solve an AssertionError: Torch not compiled with CUDA enabled error on Windows?
The root cause of this problem is still unkown, however, you can resolve this by manually installing torch in the virtualenv like python -m pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121. Thanks to @pto2k for all tdebugging this.
[END QUOTE]
My post was an attempt to help troubleshoot this a little since the readme states "The root cause of this problem is still unknown."...Therefore, I gave my experience, a user's solution on my github, to perhaps it help find an answer.

Sorry I'm still a bit confused. The person in the screenshot appears to be using a CTranslate2 backend, whilst we use pure torch.
I also have zero experience with Windows so no clue how all of this would work tbh!

It's the same error, perhaps the person who helped you troubleshoot it who you gave credit for, can take a look and you guys might find the ultimate answer.  It's probably just a piece of the puzzle though.  I wasn't able to definitively determine what the issue was, but we received the same errors so wanted to let you guys know.  Might help piece things together down the road.
BTW, just made a new issue that's kind of important to me at least.  Hope you get a chance to consider it.  Thanks!

Responded! on the other thread!
(Closing this issue for now until this issue stops being resolved by the proposed fix)
(feel free to re-open if you feel otherwise 🤗 )


json.decoder.JSONDecodeError: Unterminated string starting at: line 1 column 164 (char 163)

Most of the time i got error like this "json.decoder.JSONDecodeError: Unterminated string starting at: line 1 column 164 (char 163)" but in same  some of time it runs well ,I don't know what to do.......can anyone fix this bug.
I'm currently using
Pytube3
Python 3.7.3
pip 20.1.1

i have same issue

@mehmetbarispolat, If you get any solution......then plz post....

Sometimes it will work and sometimes it throws that error. Seems to fail about 4/5 of the time.

Same here. Will occasionally work properly but a lot of the time I will get that error.

I am taking a look at this one, but the issue appears to be in a ~60k character blob of json...

From a little digging YouTube pages where this problem occurs seem to have different HTML to ones that work.
As a result the get_ytplayer_config(html) function in extract.py can't get the correct information from the html resulting in a botched json config file.
I don't know if YouTube have recently started playing around with their HTML structure - seems strange only certain pages are like this.

Could be my unreliable bandwidth but I think that they are gradually making this change to all pages, maybe some are cached. I can't find any working examples now.

Same issue here.

Same here. Following this issue.

Found this pull request from another issue (https://github.com/nficano/pytube/pull/726). Just edited extract.py and the issue seems to be fixed for me. It's no longer giving me errors nearly every time.

Has anyone found any workarounds to this problem?

I'm going to close this issue;
The original post is using pytube3, not pytube, and the most recent post suggests that the issue has been fixed by a PR that recently got merged.


Git-ignore rules inside a project should not match parts of the absolute path outside the project

Knip: 5.19.0
Discord discussion reference (please don't be confused about the symptoms. In the end the problem was found.)

Knip (5.19.0) may behave differently on different environments, depending on

gitignore rules
the absolute path, where the project is checked out

During the block https://github.com/webpro-nl/knip/blob/7c9b6455d302193961816229c9d30ce0525a794f/packages/knip/src/util/globby.ts#L136 git ignore rules are translated to glob ignore rules. Rules are prefixed by ** which leads to the problem, that git ignore rules now may match outside the project.
Example:
# .gitignore
# ignore all files in builds folders in the project
builds/

// bug in knip

// doesn't match any file, because absolute path matches a gitignore rule
fastGlob(['src/index.ts'], {
  cwd: '/Users/xxx/builds/project',
  ignore: ['**/builds/**', '**/builds'] // generated from git-ignore
})

// maybe-fix in knip

fastGlob(['src/index.ts'], {
  cwd: '/Users/xxx/builds/project',
  ignore: ['/Users/xxx/builds/project/**/builds/**', '/Users/xxx/builds/project/**/builds'] // fix
})

Thanks for the bug report.
I wonder how it's possible the **/build pattern matching files outside the pattern influences the results? Any chance a small explanation or even reproduction could be set up?
That might be a reason to either improve Knip's docs and/or implementation: perhaps by extending or doing something similar to the source mapping (dist/index.js → src/index.ts) that was recently introduced.

@webpro here it is: https://github.com/DaAitch/knip-issue-678
I simulated an absolute path, by checking in some folders.
The git-ignore rule builds/ in <PROJECT_ROOT>/Users/knipuser/cicd/builds/project42/.gitignore is translated to a glob-ignore for **/builds/**.

Thanks, that helped a lot.
I think I found the issue, it seems to originate here in fast-glob: https://github.com/mrmlnc/fast-glob/issues/441
Need to figure out whether it can/will be fixed downstream, or work around it in Knip.

@webpro Okay interesting. Here is a red knip unit test https://github.com/DaAitch/knip/commit/5989d4a12c8d4c0a61b5b849bf6a474745d60e5c to reproduce

Thanks, I'd prefer fixes and test coverage in the downstream lib, let's await that first: https://github.com/mrmlnc/fast-glob/pull/445


Creating a SparkSession always fails with not enough memory... unless Docker has enough memory

TL;DR
Docker on the Mac defaults to running with 2GB. For Windows, it might be 1GB. With 2GB, the JVM for BeakerX notebooks appears to be launched with ~460MB (< 25%) of the available memory, which is just below what Spark considers the minimum value, ~470MB. The workaround appears to be increasing the memory for Docker.
The ~470MB number is actually hard coded in Spark! (see below), so this isn't really a BeakerX bug, per se, but adding a warning to the Spark notebooks or other doc update might be helpful. If there's a more elegant solution possible in BeakerX, then even better. Note that I haven't observed this issue with the Jupyter's project's Docker image jupyter/all-spark-notebook. I could be they are setting spark.testing testing properties, which circumvent the 470MB threshold...
Details
Mac laptop, with macOS High Sierra and using the simple Spark notebook provided with BeakerX. When evaluating this cell, where Mac Docker environment is set to use up to 2GB:
import org.apache.spark.sql.SparkSession
val spark = SparkSession.builder()
                        .appName("Simple Application")
                        .master("local[4]")
                        .config("spark.ui.enabled", "false")
                        .getOrCreate()

... it throws the following:
java.lang.IllegalArgumentException: System memory 466092032 must be at least 471859200. Please increase heap size using the --driver-memory option or spark.driver.memory in Spark configuration.
  at org.apache.spark.memory.UnifiedMemoryManager$.getMaxMemory(UnifiedMemoryManager.scala:217)
  at org.apache.spark.memory.UnifiedMemoryManager$.apply(UnifiedMemoryManager.scala:199)
  at org.apache.spark.SparkEnv$.create(SparkEnv.scala:332)
  at org.apache.spark.SparkEnv$.createDriverEnv(SparkEnv.scala:175)
  at org.apache.spark.SparkContext.createSparkEnv(SparkContext.scala:257)
  at org.apache.spark.SparkContext.<init>(SparkContext.scala:432)
  at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2516)
  at org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:918)
  at org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:910)
  at scala.Option.getOrElse(Option.scala:121)
  at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:910)
  ... 46 elided

Looking at the logic in UnifiedMemoryManager, it's looking for 1.5 * RESERVED_SYSTEM_MEMORY_BYTES, which is 1.5*300*1024*1024 = 1.5*314572800 = 471859200.
Logging into the running docker image docker exec -it <image_id> bash and running top shows that the "system" has 2GB of memory and java has about ~25% of it (but varies in size).
If I set Mac Docker to use 2.5GB or greater, then restart the BeakerX image, all works as expected.

Hi Dean thanks for the report, I have long ago raised the limit so I didn't notice this.
Seems like it should be in the doc somewhere.


fix: exchange rate now is a tuple/list

Fiat conversion rates are sent as tuple

why?

use endpoint from core instead. and remove the dead code
@api_router.get("/api/v1/rate/{currency}")
async def api_check_fiat_rate(currency: str) -> dict[str, float]:
    rate, price = await get_fiat_rate_satoshis(currency)
    return {"rate": rate, "price": price}


OCM-9288 | test: automate id:60688 reuse operator-roles and oidc config

OCM-9288 | test: automate id:60688 reuse operator-roles and oidc config
Log:https://privatebin.corp.redhat.com/?7e86348b7c31c68e#GEvGyKHSSY3tk9F97CZbya3bfCnjMnrcPzhCUeyCHWv1

@yasun1 Fix the line of expecting err as your comment. Could you help review again?

/lgtm


Internal error detected VS2017

Traceback (most recent call last):
File "C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\IDE\Extensions\Microsoft\Python\Core\ptvsd_launcher.py", line 34, in
import ptvsd.debugger as vspd
File "C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\IDE\Extensions\Microsoft\Python\Core\ptvsd\debugger.py", line 34, in
import socket
File "C:\Users\acer\source\repos\socket\socket\socket.py", line 3, in
sock = socket.socket()
TypeError: 'module' object is not callable
I was trying to establish socket client for WIFI communication usin ESP32. But it keeps showing me this message and since I am new to Python, I do not know what the issue is. I was using the code listed below:
`import socket
sock = socket.socket()
host = "<IP_ADDRESS>" #ESP32 IP in local network
port = 80             #ESP32 Server Port
sock.connect((host, port))
message = "Hello World"
sock.send(message)
data = ""
while len(data) < len(message):
data += sock.recv(1)
print(data)
sock.close()
`
Thank you very much in advance

This is due to your project file named socket.py which conflicts with the name of the standard library module. The debugger does import socket and is getting your module definition. Renaming your project file should fix the issue.

Thank you very much huguesv. That was the only issue. You saved me a lot of time :)


Code Generate throw NullReferenceException

Hi,
I have  have a question
Here's some code
My Entitas version is 1.5.2. When I click the Generate Butoon in the Preferences Windows, I got an Exception below:
nityEngine.GUIContent label, UnityEngine.GUIContent label2, UnityEngine.GUIStyle style, UnityEngine.GUILayoutOption[] options) (at C:/buildslave/unity/build/Editor/Mono/EditorGUI.cs:6187) UnityEditor.EditorGUILayout.LabelField (System.String label, UnityEngine.GUIStyle style, UnityEngine.GUILayoutOption[] options) (at C:/buildslave/unity/build/Editor/Mono/EditorGUI.cs:6143) DesperateDevs.Unity.Editor.PreferencesWindow.drawException (System.Exception exception) DesperateDevs.Unity.Editor.PreferencesWindow.drawContent () DesperateDevs.Unity.Editor.PreferencesWindow.OnGUI () System.Reflection.MonoMethod.Invoke (System.Object obj, BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) (at /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Reflection/MonoMethod.cs:222) Rethrow as TargetInvocationException: Exception has been thrown by the target of an invocation. System.Reflection.MonoMethod.Invoke (System.Object obj, BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) (at /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Reflection/MonoMethod.cs:232) System.Reflection.MethodBase.Invoke (System.Object obj, System.Object[] parameters) (at /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Reflection/MethodBase.cs:115) UnityEditor.HostView.Invoke (System.String methodName, System.Object obj) (at C:/buildslave/unity/build/Editor/Mono/HostView.cs:295) UnityEditor.HostView.Invoke (System.String methodName) (at C:/buildslave/unity/build/Editor/Mono/HostView.cs:288) UnityEditor.HostView.OldOnGUI () (at C:/buildslave/unity/build/Editor/Mono/HostView.cs:107) UnityEngine.Experimental.UIElements.IMGUIContainer.DoOnGUI (UnityEngine.Event evt) (at C:/buildslave/unity/build/Runtime/UIElements/Managed/IMGUIContainer.cs:179)
but every component I writed is genertted successfully, I just canot tolerate the error. how can i fix it?

Hi,
see Unity GUI bugs on Windows #583
This error doesn't affect the functionality of Entitas and can be ignored. Afaik, Unity has to fix this. They are currently using UnityEngine.Experimental.UIElements


Add demo disclaimer

Describe the enhancement
@GBKS has requested we show a disclaimer stating that the current Saving Satoshi experience is a demo. This issue serves as a reminder to actually implement the design for that once it's ready.

I think this will be easiest to implement after #167 is merged as we can probably just use a similar modal pop-up

I'd prefer not to have a modal. Instead, we can just have a short text-based indicator on the home page, with a longer explainer on the about page (see #175), as well as a call-out on the chapter 1 completion page (see #186). I'll mock something up.

Assigning this to Christoph for the design.

Hey @GBKS, I don't mean to press but what's the current status for a demo disclaimer mockup?

+1 on "start the demo" feel.
I'm on the fence about the disclaimer at the top. I already kinda feel like there is a ton of space at the top. Without the disclaimer, I have to scroll down to get to the button "Start chapter 1".


EthBerlinZwei: NFT eth-bridge support

[x] use go mod instead of dep
[ ] Updates to the latest SDK (branch billy/nft)
[ ] Add NFT module support
[ ] Implement ERC1155 Multi token standard

@fedekunze what you think of merging this as merely the go.mod and update sdk version?

Sure, we can upgrade this to support NFTs afterwards

@fedekunze how should we handle the rest, seems the modules need some updating now. should we bring this up for a starting point for swish, or should we try to handle it?

I'll find some time early next week to update it

partially included in Update to SDK 0.37 #65, closing


fix field name in nameCards.json

[ ] fix field name to be a camel case.
[ ] add more mock data for nameCards.

This is done


Pick hosts randomized in RetryingJaxRsStrategy

This explains the skewed traffic to servers of clients using RetryingJaxRsStrategy.

I don't think we can merge this until we have changed the deploy API such that activate on an unprepared application will also prepare it, as we currently need affinity between those two calls when happening in close succession. If you would like to do that feel free :-)
Otherwise I hope to get to it soon...ish.

If the same RetryingJaxRsStrategy instance is used for both calls, we're still guaranteed the same order with this change. Is that good enough? If this is ApplicationController it's stability should be unchanged.

Actually, I think ApplicationController uses something else than RetryingJaxRsStrategy.

Yes, ApplicationController uses ConfigServerClientImp, but that one is only using  RetryingJaxRsStrategy for some of the calls, and its own retrying implementation for other calls. Entropy ... Anyway prepare and activate is in the second category, so this change is fine.


Not showing Suggestions on JavaScript

Type: Bug
I have linked my index.html with index.js file but I am not able to type "box[2].style.backgroundColor = "red";"
What is the reason I have searched in on google but couldn't find please help me with this issue
VS Code version: Code 1.85.2 (8b3775030ed1a69b13e4f4c628c612102e30a681, 2024-01-18T06:40:10.514Z)
OS version: Windows_NT x64 10.0.22631
Modes:

System Info

Item
Value

CPUs
Intel(R) Core(TM) i5-8300H CPU @ 2.30GHz (8 x 2304)

GPU Status
2d_canvas: enabledcanvas_oop_rasterization: enabled_ondirect_rendering_display_compositor: disabled_off_okgpu_compositing: enabledmultiple_raster_threads: enabled_onopengl: enabled_onrasterization: enabledraw_draw: disabled_off_okvideo_decode: enabledvideo_encode: enabledvulkan: disabled_offwebgl: enabledwebgl2: enabledwebgpu: enabled

Load (avg)
undefined

Memory (System)
7.85GB (1.25GB free)

Process Argv
--crash-reporter-id 5761ed93-b8e0-42bb-ac22-4ddacd84ec68

Screen Reader
no

VM
0%

Extensions (5)

Extension
Author (truncated)
Version

code-runner
for
0.12.1

cpptools
ms-
1.18.5

live-server
ms-
0.4.13

LiveServer
rit
5.7.9

vscode-icons
vsc
12.7.0

A/B Experiments
vsliv368cf:30146710
vspor879:30202332
vspor708:30202333
vspor363:30204092
vstes627:30244334
vscoreces:30445986
vscod805cf:30301675
binariesv615:30325510
vsaa593cf:30376535
py29gd2263:30899288
vscaat:30438848
c4g48928:30535728
azure-dev_surveyone:30548225
2i9eh265:30646982
962ge761:30951796
pythongtdpath:30769146
welcomedialogc:30910334
pythonidxpt:30866567
pythonnoceb:30805159
asynctok:30898717
pythontestfixt:30902429
pythonregdiag2:30936856
pyreplss1:30897532
pythonmypyd1:30879173
pythoncet0:30885854
pythontbext0:30879054
accentitlementsc:30887149
dsvsc016:30899300
dsvsc017:30899301
dsvsc018:30899302
dsvsc019acf:30951919
3ef8e399:30949928

/question

We closed this issue because it is a question about using VS Code rather than an issue or feature request. Please search for help on StackOverflow, where the community has already answered thousands of similar questions. You may find their guide on asking a new question helpful if your question has not already been asked. See also our issue reporting guidelines.
Happy Coding!


Mipmap texture issue

[Worker-ResourceReload-6/WARN]: Texture additional_rpg_jewelry:item/griffin_school_medallion with size 12x12 limits mip level from 3 to 2
Pls change this texture to 16x16 , 32x32 or so on

Thanks for reporting the issue @xR4YM0ND !
Fixed with versions:
2.0.3 - 1.21.1 & 1.21
1.0.9 - 1.20.1
Uploaded the new versions on Curseforge & Modrinth, should be available soon.


How to visualize the data? Can you provide more details?

Hi @Ha0Tang,
I think you can use Meshlab or Open3d.


fix: improved aqi resiliance

:tada: This PR is included in version 2.1.8 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Replace pyppeteer with playwright

Playwright seems better maintained :)

The transition is smooth. Usage of rsserpent.utils.Browser remains unchanged.


[uki/canary-*]: Canary packages added to workflow

Summary of the changes
This PR adds the canary packages as two new packages within the monorepo. The changes include:

Adding canary-web-components and canary-react
Added canary github action to build canary packages only
Updates to existing github actions to include canary builds
Setting lerna.json file to deal with versioning on an independent level (packages will no longer have synced version numbers)
Exporting helper functions from web-components to be used in canary-web-components
Updates to package.json. Moving @ukic/fonts to peerDependencies to reduce bundle size
Regenerated pacakge-lock files using npm i --package-lock-only
Updated getting started copy

The canary components within this PR are the same as the ones which are currently released so there shouldn't be any changes in that respect.
Related issue
N/A
Checklist
N/A

If possible, would be good to get @ad9242 and @GCHQ-Developer-112 eyes on this as they have been working off this branch.