Silencing exception on registered object without onEvent method

I have a BaseActivity class which acts as base for all Android activities in my project.
EventBus.register(this) is done in the BaseActivity without any onEvent methods in it. Some of its subclasses have onEvent and some don't. I got this exception when starting activities (subclass of BaseActivity) without onEvent methods:
Subscriber class com.example.LoginActivity has no public methods called onEvent

That's obviously because LoginActivity has been registered to EventBus in the super class but has no onEvent method. However I could get around this problem by putting empty onEvent in the BaseActivity (a little bit unclean solution)
I suggest that no exception should be thrown in that case (no onEvent methods). Let me know what you think.
Thanks

I've got the same issue while working with fragments hosted on a ViewPager of my main Activity. I think the best solution should be to not register unless you really listen to an event as @duongphuhiep said ;)

I've got same issue while working with activity.

has no public methods called onEvent


Remove unnecessary f-string.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 81.52%. Comparing base (6124683) to head (381d227).
 Consider uploading reports for the commit 4cb1202 to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4418      +/-   ##
==========================================
- Coverage   81.52%   81.52%   -0.01%
==========================================
  Files         109      109
  Lines       53163    53155       -8
==========================================
- Hits        43343    43336       -7
+ Misses       9820     9819       -1


🐛 BUG: CircleCI deploy pages

What version of Wrangler are you using?
2.0.25
What operating system are you using?
Ubuntu 20.04
Describe the Bug
When I'm running deploy locally:
CLOUDFLARE_ACCOUNT_ID=********** npx wrangler pages publish dist/apps/breath --project-name=breathe

It works fine.
However in circle ci when I set it through like the docs:
  deploy:
    docker:
      - image: cimg/node:18.7.0
    environment:
      CLOUDFLARE_ACCOUNT_ID: $CLOUDFLARE_ACCOUNT_ID
      CLOUDFLARE_API_TOKEN: $CLOUDFLARE_API_TOKEN
    steps:
      - checkout
      - bt-yarn-install
      - run: yarn build
      - setup_remote_docker:
          version: 20.10.14
      - run: yarn deploy

I'm getting this error:

I've set the environment variables on Circle CI as per the docs

Hiya! We just reproduced this and figured out what's wrong. You need to remove the environment block in your config and ensure the variables are declared in your project settings in CircleCI.
deploy:
    docker:
      - image: cimg/node:18.7.0
    steps:
      - checkout
      - bt-yarn-install
      - run: yarn build
      - setup_remote_docker:
          version: 20.10.14
      - run: yarn deploy

The above should work! Cheers. Closing this for now since it isn't a wrangler bug per se. Also updating our documentation in https://github.com/cloudflare/cloudflare-docs/pull/5607
Thanks!


landmarks

hi,
thanks for code, can you please guide me how to get 68 landmarks instead of 58 landmarks.
i am waiting for your kind reply.
Thanks

@elliotresearcher Hi, in fact, the COFW dataset has 29 landmarks. The size of a landmark vector is (29*2)X1. When you use 68 landmarks, the size will be 136X1. What you need to do is just replacing the  58XN shape matrix by the 136XN matrix for model training. Hope this is helpful.


Support EventHandler attributes

This issue has been moved to https://github.com/w3c/web-platform-tests/issues/7148; please continue all discussion there.


Update tagbot

This looks good to me. We seem to be failing for other reasons. Do you want me to go ahead and merge or wait on fixes to the testing?


Store labels for profiles in the database

Summary
Extends the profiles table with an additional column that stores profile
labels. This was one way I was considering, the other was to normalize the tables
and have a separate profile_labels table and then a "join table" linking profiles with
labels. This table also gets a GIN index so that we can index the composite field.
Reading postgres documentation, it appears like postgres should be able to use
the B-Tree index for names and IDs in conjunction with the GIN index itself.
I decided to go with a simple additional column for now because while the indexing
would have been easier with a separate table, the application logic for creating
and updating profiles would be much involved as we'd have to store the labels and
the link in a transaction. We might want to do that later, but I'm not sure if adding
database complexity when not needed is wise. At the same time, migration from the single
column to the normalized tables should be possible as well.
The second patch only allows setting labels on profiles that have a
subscription and disallows changing labels. We can change the validation logic, but IIRC
for now we're going to only have a single label and we already check the presence
of a subscription earlier, so this should be enough for now and we can change
the validation later when we actually have a better use-case.
Fixes #2764
Change Type
Mark the type of change your PR introduces:

[ ] Bug fix (resolves an issue without affecting existing features)
[x] Feature (adds new functionality without breaking changes)
[ ] Breaking change (may impact existing functionalities or require documentation updates)
[ ] Documentation (updates or additions to documentation)
[ ] Refactoring or test improvements (no bug fixes or new functionality)

Testing
Outline how the changes were tested, including steps to reproduce and any relevant configurations.
Attach screenshots if helpful.
Review Checklist:

[x] Reviewed my own code for quality and clarity.
[x] Added comments to complex or tricky code sections.
[ ] Updated any affected documentation.
[ ] Included tests that validate the fix or feature.
[ ] Checked that related changes are merged.

coverage: 47.553% (-0.01%) from 47.567%
when pulling 16942c68c9b896048400c6f1d3a4e79d66753101 on jhrozek:labels_in_db_for_profiles
into c5718357a175594ee400b5bf046098b5f2d9dfc2 on stacklok:main.

I did some investigation, including reading https://pganalyze.com/blog/gin-index , and I think we don't want to use a GIN index.  From a discussion elsewhere, and EXPLAIN testing:

The particular scenario I'm worried about is if we have e.g. 100k projects, and each of them has 5 profiles, one of which is labelled stacklok:health (for example).  The index for that particular label will contain 100k entries; each time we create a new project, we'll need to update that index entry, and at the end of the day, we're scanning the 100k-item index to avoid reading 4 items found by the B-Tree that don't have that label.

I did some investigation, including reading https://pganalyze.com/blog/gin-index , and I think we don't want to use a GIN index. From a discussion elsewhere, and EXPLAIN testing:

The particular scenario I'm worried about is if we have e.g. 100k projects, and each of them has 5 profiles, one of which is labelled stacklok:health (for example).  The index for that particular label will contain 100k entries; each time we create a new project, we'll need to update that index entry, and at the end of the day, we're scanning the 100k-item index to avoid reading 4 items found by the B-Tree that don't have that label.

I pushed a new patch that drops the GIN and instead indexes the profile_id on entity_profiles as discussed on Slack.

@dmjb PTAL


update command list

While this issue has been mostly resolved, there is still a lack of documentation concerning the ranked-user commands; they haven't been updated to reflect the changes in commands for mod+ and super-user+ users.

due to time, this has become an issue again, it may be a good idea to auto-generate some documentation, perhaps with xml comments?

I hadn't thought of using XML comments. I went ahead and made an example of what it might look like. We also should probably write some C# code that would also output all the access levels. The escaping of the < and > for arguments in the example is a bit annoying but not too bad. The param tags aren't necessary but maybe we could figure out a way to generate the example using those. They could also be put under an expandable section for more information, like this:

Arguments
targetPlayer - The player who should get the points.
bonus - The number of points for the user to get.

/// <summary>
/// Adds bonus points to a target player.
/// </summary>
/// <param name="targetPlayer">The player who should get the points.</param>
/// <param name="bonus">The number of points for the user to get.</param>
/// <example>!bonusscore <targetPlayer> <bonus></example>
[Command(@"bonus(?:score|points) (\S+) (-?[0-9]+)", AccessLevel.SuperUser, AccessLevel.SuperUser)]
public static void BonusPoints([Group(1)] string targetPlayer, [Group(2)] int bonus, string user)
{
	IRCConnection.SendMessageFormat(TwitchPlaySettings.data.GiveBonusPoints, targetPlayer, bonus, user);
	Leaderboard.Instance.AddScore(targetPlayer, new Color(.31f, .31f, .31f), bonus);
}

I'm going to close this as the new command reference is out: https://samfun123.github.io/KtaneTwitchPlays/


Update metrics-template.md

Signed-off-by: Matt Germonprez<EMAIL_ADDRESS>@klumb adding the disclaimer to the template

I suggest linking the issue number to the issue to make it easier for visitors to find the right issue and not have to guess which repository the issue is in.

Yes, thank you georg

this looks good I am going to merge this as we have other metrics template PRs coming

fixed merge conflict in browser -- breaks DCO sign-off -- manually overwriting and using squash merge.


delete session files

these are just skeletons, per brett and daniel, and they need to go.

@danlec I don't have write access to this repo. Any chance you can 🚀 ?


how to use a per document monitor - Alerting plugin

Signed-off-by: alicejw<EMAIL_ADDRESS>Description
Write how to create and use the new per document monitor as part of the OpenSearch Dashboards Alerting plugin.
Page URLs:
Alerting
Monitors
Issues Resolved
475
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@alicejw-aws Is there an API component to this? Can users use the API to create per query monitors?

@alicejw-aws Is there an API component to this? Can users use the API to create per query monitors?

yes will add API section and submit for review again.


Investigate dropping BouncyCastle dependency

Currently this library has dependency on two different BouncyCastle builds. It would be perfect to drop the BouncyCastle dependency, but it requires new APIs in CoreFX/NetFX.
[DEPENDS ON] dotnet/corefx#8158
[DEPENDS ON] dotnet/corefx#21833
[DEPENDS ON] tpeczek/Lib.Net.Http.EncryptedContentEncoding#4

Currently this library depends on Portable.BouncyCastle. In future it might be still worth to consider dropping the BouncyCastle dependency in favour of native CoreFX/NetFX APIs.

As I mentioned in an issue in web-push-libs/web-push-csharp , the required aes-gcm cipher will be available in .net core3 (it is already included in the preview 1).

@tuwrraphael Yes. It's not enough to drop BouncyCastle entirely (ASN.1 reader/writer missing) but it should be enough to close tpeczek/Lib.Net.Http.EncryptedContentEncoding#4.


Enhancement request: public static final variables cause full recompilation of project

Currently, as soon as you have a project that has a public static final variable, Gradle will automatically recompile the entire project, even if that constant hasn't changed and the project only contains a single unrelated class change.
Expected Behavior
It would be nice if Gradle could tell whether the constant was really changed or not and avoid full recompilation.
Or if this is not possible, it would be nice if there was a toggle/mode with incremental recompilation with Gradle where we could choose to trade this full recompilation in exchange for only changed file recompilation. This would be really useful to achieve fast IDE iteration times.
Current Behavior
Full recompilation as soon as you change a file in a sourceSet that has a public static final variable.
Context
We are trying to move from the old idea IntelliJ integration (where IntelliJ does the compilation) to the native Gradle+IntelliJ integration, however a significant blocker is the re-compilation time that this introduces since Gradle recompiles much more than only the changed files due to this quirk.
Our mono repo build currently has ~30k instances of this use case (+another 10k in generated code). As much as it might be considered "bad practice", they appear to be here to stay and are in almost every project.
Steps to Reproduce
Self-contained project attached. Note that maven behaves differently as well, pom included in the repro project.
Your Environment
macOS / Java 11 / Gradle 6
gradle_incremental_compilation.zip

Gradle used to analyze the constants in use but the analysis was wrong. See #1474 for context. Basically, the compilers inline constants, so we can't track where they come from. Imagine your code contains something like this:
int x() {
   return A.CONST_X + 2;
}

and A.CONST_X = 2.
Then the compiler will inline the result of the computation, and the body of the method will, bytecode-wise, be equivalent to:
int x() { return 4; }

We don't have any information to let us know that it came from A. So we had to disable this "smart" analysis. Some compilers actually keep a reference of inlined constants, but not all of them.
This is why we said that whenever a class which has a constant changes, we need to rebuild everything.
The point in this ticket is "but my constants didn't change and you still recompiled everything". That's because while we keep track of constants, we don't keep track of what symbol is used. So, say A contains:
class A {
    public static int C1 = 10;
    public static int C2 = 11;
}

and that it's changed to:
class A {
    public static int C1 = 11;
    public static int C2 = 10;
}

in both cases, the constant set is {10, 11}, but we need to rebuild all consumers of C1 and C2. It would be very expensive to keep track of the symbol/constant value pairs, so we don't do it.
Does it mean we can't do anything?
No. First of all, we could detect that the compiler you are using is actually one of those keeping track of inlined constants. I think this should be the case for all JDK 9+ compilers, but we need to double check that.
Second, for every Java 8+ compiler that we call using the API, we use a compiler plugin to perform some incremental compilation analysis. It means we could actually register where the constants come from in this case. The problem is that we don't always use the compiler API.

That would be an awesome improvement for build times in our case. The problem is, that constants sometimes need to be inlined (switch statements, annotation parameters etc) so you cannot workaround it with static {} block or static method or some other way. You can of course be careful to have them in some rarely modified class. But that practice can easily break on a project with many of contributors and then you have to hunt down those constants that cause full recompilation.
Also I believe they are really common in Java projects which probably means that full recompilation due to constants is one of the top reasons why incremental compilation was not possible with Gradle.


Fix kappa generator using certain expressions

The kappa generator was incorrectly substituting parameter names in
expressions if one parameter was a substring of the other. This PR
uses the sympy printer instead of string replacement.
Fixes: #345

Coverage increased (+0.03%) to 75.404% when pulling 24bbe25e0f130817ea73ec37c0eb915b99d8c0dd on alubbock:kappafix into 6746b1c56b9da7e686b95d25a933f40ecae1c596 on pysb:master.


unable to finish polling, context has been cancelled from a bridge update

What happened?
Pre-testing bridge candidate we found that acceptance tests started failing in https://github.com/pulumi/pulumi-gcp/pull/1390
Tracking the investigation here.
Example
 TestAccLoadbalancer (1m5.84s)
      program.go:1903: sample: /home/runner/work/pulumi-gcp/pulumi-gcp/examples/loadbalancer
      program.go:1908: pulumi: /opt/hostedtoolcache/pulumi/3.95.0/x64/pulumi
      program.go:1068: acquired yarn install lock
      command.go:34: **** Invoke '/usr/local/bin/yarn install' in '/tmp/p-it-runner-loadbalanc-a0080cef-3274804897'
      command.go:103: Wrote output to /tmp/p-it-runner-loadbalanc-a0080cef-3274804897/command-output/yarn-install.20231204-212632.a8cde.log
      program.go:1094: released yarn install lock
      program.go:2053: projdir: /tmp/p-it-runner-loadbalanc-a0080cef-3274804897
      program.go:1318: Initializing project (dir /tmp/p-it-runner-loadbalanc-a0080cef-3274804897; stack p-it-runner-loadbalanc-a0080cef)
      command.go:34: **** Invoke '/opt/hostedtoolcache/pulumi/3.95.0/x64/pulumi login --cloud-url file:///tmp/TestAccLoadbalancer3298498485/001' in '/tmp/p-it-runner-loadbalanc-a0080cef-3274804897'
      command.go:103: Wrote output to /tmp/p-it-runner-loadbalanc-a0080cef-3274804897/command-output/pulumi-login.20231204-212633.68f6a.log
      command.go:34: **** Invoke '/opt/hostedtoolcache/pulumi/3.95.0/x64/pulumi stack init p-it-runner-loadbalanc-a0080cef' in '/tmp/p-it-runner-loadbalanc-a0080cef-3274804897'
      command.go:103: Wrote output to /tmp/p-it-runner-loadbalanc-a0080cef-3274804897/command-output/pulumi-stack-init.20231204-212633.b6d20.log
      command.go:34: **** Invoke '/opt/hostedtoolcache/pulumi/3.95.0/x64/pulumi config set-all --plaintext gcp:config:project=pulumi-ci-gcp-provider --plaintext gcp:config:zone=us-central1-a --plaintext gcp:config:region=us-central1' in '/tmp/p-it-runner-loadbalanc-a0080cef-3274804897'
      command.go:106: Command completed without output
      program.go:1485: Performing primary preview and update
      command.go:34: **** Invoke '/opt/hostedtoolcache/pulumi/3.95.0/x64/pulumi up --non-interactive --yes --skip-preview --event-log /tmp/p-it-runner-loadbalanc-a0080cef-events.json' in '/tmp/p-it-runner-loadbalanc-a0080cef-3274804897'
      command.go:83: Invoke '/opt/hostedtoolcache/pulumi/3.95.0/x64/pulumi up --non-interactive --yes --skip-preview --event-log /tmp/p-it-runner-loadbalanc-a0080cef-events.json' failed: exit status 255
  Updating (p-it-runner-loadbalanc-a0080cef):
      pulumi:pulumi:Stack loadbalancer-p-it-runner-loadbalanc-a0080cef  warning: using pulumi-resource-gcp from $PATH at /home/runner/work/pulumi-gcp/pulumi-gcp/bin/pulumi-resource-gcp

   +  pulumi:pulumi:Stack loadbalancer-p-it-runner-loadbalanc-a0080cef creating (0s) warning: using pulumi-resource-gcp from $PATH at /home/runner/work/pulumi-gcp/pulumi-gcp/bin/pulumi-resource-gcp
  @ updating......
   +  pulumi:pulumi:Stack loadbalancer-p-it-runner-loadbalanc-a0080cef creating (2s) warning: using pulumi-resource-gcp from $PATH at /home/runner/work/pulumi-gcp/pulumi-gcp/bin/pulumi-resource-gcp
   +  gcp:compute:HealthCheck health-check creating (0s)
   +  gcp:compute:GlobalAddress external-address creating (0s)
   +  gcp:compute:Network default-network creating (0s)
  @ updating....
   +  gcp:compute:GlobalAddress external-address creating (0s) error: 1 error occurred:
   +  gcp:compute:GlobalAddress external-address **creating failed** error: 1 error occurred:
   +  gcp:compute:HealthCheck health-check creating (0s) error: 1 error occurred:
   +  gcp:compute:HealthCheck health-check **creating failed** error: 1 error occurred:
  @ updating.....
   +  gcp:compute:Network default-network creating (2s) error: 1 error occurred:
   +  gcp:compute:Network default-network **creating failed** error: 1 error occurred:
   +  pulumi:pulumi:Stack loadbalancer-p-it-runner-loadbalanc-a0080cef creating (5s) error: update failed
   +  pulumi:pulumi:Stack loadbalancer-p-it-runner-loadbalanc-a0080cef **creating failed (3s)** 1 error; 2 warnings
  Diagnostics:
    gcp:compute:HealthCheck (health-check):
      error: 1 error occurred:
      	* Error waiting to create HealthCheck: Error waiting for Creating HealthCheck: error while retrieving operation: unable to finish polling, context has been cancelled

    pulumi:pulumi:Stack (loadbalancer-p-it-runner-loadbalanc-a0080cef):
      warning: using pulumi-resource-gcp from $PATH at /home/runner/work/pulumi-gcp/pulumi-gcp/bin/pulumi-resource-gcp
      warning: using pulumi-resource-gcp from $PATH at /home/runner/work/pulumi-gcp/pulumi-gcp/bin/pulumi-resource-gcp
      error: update failed

    gcp:compute:GlobalAddress (external-address):
      error: 1 error occurred:
      	* Error waiting to create GlobalAddress: Error waiting for Creating GlobalAddress: error while retrieving operation: unable to finish polling, context has been cancelled

    gcp:compute:Network (default-network):
      error: 1 error occurred:
      	* Error waiting to create Network: Error waiting for Creating Network: error while retrieving operation: unable to finish polling, context has been cancelled

  Resources:
      + 1 created

  Duration: 7s

Output of pulumi about
N/A
Additional context
N/A
Contributing
Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

The initial step was reverting https://github.com/pulumi/pulumi-terraform-bridge/commit/a0d01d3f8bcd0872c68b89a6ba73871b0cb6ddce change to Context propagation and advanced logging. With the change reverted the tests passed again and the provider was able to update to the latest bridge.

Investigating further, ProviderConfigure from upstream grabs the Context and saves it in client objects that talk to various cloud services.
https://github.com/hashicorp/terraform-provider-google-beta/blob/main/google-beta/provider/provider.go#L2258
In particular it uses the deprecated schema.StopContext call.
// StopContext returns a context safe for global use that will cancel
// when Terraform requests a stop. This function should only be called
// within a ConfigureContextFunc, passing in the request scoped context
// received in that method.
//
// Deprecated: The use of a global context is discouraged. Please use the new
// context aware CRUD methods.
func StopContext(ctx context.Context) (context.Context, bool) {
	stopContext, ok := ctx.Value(StopContextKey).(context.Context)
	return stopContext, ok
}

So clearly the upstream provider is assuming that Configure is called with a global Context, but with that unfortunate bridge change it started being called with a gRPC-initiated Context pertaining to servicing the Pulumi Configure call. That context is cancelled by the gRPC server once the Configure call finishes, but subsequent Create/Update calls still reference it via the cloud clients.

https://github.com/hashicorp/terraform-plugin-sdk/blob/main/helper/schema/grpc_provider.go#L557

	// TODO: remove global stop context hack
	// This attaches a global stop synchro'd context onto the provider.Configure
	// request scoped context. This provides a substitute for the removed provider.StopContext()
	// function. Ideally a provider should migrate to the context aware API that receives
	// request scoped contexts, however this is a large undertaking for very large providers.
	ctxHack := context.WithValue(ctx, StopContextKey, s.StopContext(context.Background()))

	logging.HelperSchemaTrace(ctx, "Calling downstream")
	diags := s.provider.Configure(ctxHack, config)
	logging.HelperSchemaTrace(ctx, "Called downstream")

This finding is interesting. Looks like SDKv2 based framework is calling Configure with a special "global stop context hack". If this is the case, it should probably be something that we match in the bridge.


[Web][HTML] Button rendering issues after scrolling multiple times

Steps to Reproduce

Execute flutter run --web-renderer html on the code sample
Scroll up and down more than a dozen times

Expected results:
Sometimes the text on the button is not visible.
Actual results:
The text of the button should remain visible after multiple scrolls.

Code sample
import 'package:flutter/material.dart';

void main() {
  runApp(const MyApp());
}

class MyApp extends StatelessWidget {
  const MyApp({Key? key}) : super(key: key);

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      home: MyHomePage(),
    );
  }
}

class MyHomePage extends StatelessWidget {
  final scrollController = ScrollController();

  MyHomePage({Key? key}) : super(key: key);

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      body: Center(
        child: ListView.builder(
            controller: scrollController,
            itemCount: 100,
            itemBuilder: (_, index) {
              return row(index);
            }),
      ),
    );
  }

  Widget row(int index) {
    return Container(
      padding: const EdgeInsets.all(5.0),
      child: Row(
        children: [
          Text('$index'),
          OutlinedButton(onPressed: () {}, child: const Text('Check1')),
          ElevatedButton(onPressed: () {}, child: const Text('Check2')),
        ],
      ),
    );
  }
}

Logs

Analyzing sample5...
No issues found! (ran in 8.5s)

[✓] Flutter (Channel stable, 2.10.4, on macOS 12.3.1 21E258 darwin-x64, locale ja-JP)
    • Flutter version 2.10.4 at /Users/***name***/development/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision c860cba910 (3 weeks ago), 2022-03-25 00:23:12 -0500
    • Engine revision 57d3bac3dd
    • Dart version 2.16.2
    • DevTools version 2.9.2

[✓] Android toolchain - develop for Android devices (Android SDK version 30.0.3)
    • Android SDK at /Users/***name***/Library/Android/sdk
    • Platform android-31, build-tools 30.0.3
    • Java binary at: /Applications/Android Studio.app/Contents/jre/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.11+0-b60-7590822)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 13.3.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • CocoaPods version 1.11.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio
    • Android Studio at /Applications/Android Studio Preview.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.8+10-b944.6842174)

[✓] Android Studio (version 2021.1)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.11+0-b60-7590822)

[✓] IntelliJ IDEA Ultimate Edition (version 2021.1)
    • IntelliJ at /Applications/IntelliJ IDEA.app
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart

[✓] Connected device (1 available)
    • Chrome (web) • chrome • web-javascript • Google Chrome 100.0.4896.88

[✓] HTTP Host Availability
    • All required HTTP hosts are available

• No issues found!

https://user-images.githubusercontent.com/15129144/163769523-99b05649-045a-4245-91ed-0d2a82a7ad19.mov

Issue is reproducible on stable and master with the html renderer using the code sample above.

videos

HTML
Canvaskit

flutter doctor -v
[✓] Flutter (Channel stable, 2.10.4, on macOS 12.3.1 21E258 darwin-arm, locale en-GB)
    • Flutter version 2.10.4 at /Users/nexus/dev/sdks/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision c860cba910 (3 weeks ago), 2022-03-25 00:23:12 -0500
    • Engine revision 57d3bac3dd
    • Dart version 2.16.2
    • DevTools version 2.9.2

[✓] Android toolchain - develop for Android devices (Android SDK version 31.0.0)
    • Android SDK at /Users/nexus/Library/Android/sdk
    • Platform android-32, build-tools 31.0.0
    • Java binary at: /Applications/Android Studio.app/Contents/jre/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.11+0-b60-7772763)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 13.2.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • CocoaPods version 1.11.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2021.1)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.11+0-b60-7772763)

[☠] IntelliJ IDEA Community Edition (the doctor check crashed)
    ✗ Due to an error, the doctor check did not complete. If the error message below is not helpful, please let us know about this issue at https://github.com/flutter/flutter/issues.
    ✗ FormatException: Unexpected extension byte (at offset 5)
    • #0      _Utf8Decoder.convertSingle (dart:convert-patch/convert_patch.dart:1789:7)
      #1      Utf8Decoder.convert (dart:convert/utf.dart:351:42)
      #2      InputStream.readString (package:archive/src/util/input_stream.dart:207:30)
      #3      new ZipDirectory.read (package:archive/src/zip/zip_directory.dart:40:30)
      #4      ZipDecoder.decodeBuffer (package:archive/src/zip_decoder.dart:19:30)
      #5      ZipDecoder.decodeBytes (package:archive/src/zip_decoder.dart:14:12)
      #6      IntelliJPlugins._findPluginXml (package:flutter_tools/src/intellij/intellij.dart:130:44)
      #7      IntelliJPlugins._readPackageVersion (package:flutter_tools/src/intellij/intellij.dart:141:40)
      #8      IntelliJPlugins.validatePackage (package:flutter_tools/src/intellij/intellij.dart:63:35)
      #9      IntelliJValidator.validate (package:flutter_tools/src/intellij/intellij_validator.dart:103:15)
      #10     asyncGuard.<anonymous closure> (package:flutter_tools/src/base/async_guard.dart:111:32)
      #11     asyncGuard.<anonymous closure> (package:flutter_tools/src/base/async_guard.dart:109:18)
      #12     _rootRun (dart:async/zone.dart:1426:13)
      #13     _CustomZone.run (dart:async/zone.dart:1328:19)
      #14     _runZoned (dart:async/zone.dart:1861:10)
      #15     runZonedGuarded (dart:async/zone.dart:1849:12)
      #16     runZoned (dart:async/zone.dart:1780:12)
      #17     asyncGuard (package:flutter_tools/src/base/async_guard.dart:109:3)
      #18     Doctor.startValidatorTasks (package:flutter_tools/src/doctor.dart:205:9)
      #19     Doctor.diagnose (package:flutter_tools/src/doctor.dart:309:47)
      #20     DoctorCommand.runCommand (package:flutter_tools/src/commands/doctor.dart:50:48)
      #21     FlutterCommand.verifyThenRunCommand (package:flutter_tools/src/runner/flutter_command.dart:1320:12)
      <asynchronous suspension>
      #22     FlutterCommand.run.<anonymous closure> (package:flutter_tools/src/runner/flutter_command.dart:1161:27)
      <asynchronous suspension>
      #23     AppContext.run.<anonymous closure> (package:flutter_tools/src/base/context.dart:150:19)
      <asynchronous suspension>
      #24     CommandRunner.runCommand (package:args/command_runner.dart:209:13)
      <asynchronous suspension>
      #25     FlutterCommandRunner.runCommand.<anonymous closure> (package:flutter_tools/src/runner/flutter_command_runner.dart:281:9)
      <asynchronous suspension>
      #26     AppContext.run.<anonymous closure> (package:flutter_tools/src/base/context.dart:150:19)
      <asynchronous suspension>
      #27     FlutterCommandRunner.runCommand (package:flutter_tools/src/runner/flutter_command_runner.dart:229:5)
      <asynchronous suspension>
      #28     run.<anonymous closure>.<anonymous closure> (package:flutter_tools/runner.dart:62:9)
      <asynchronous suspension>
      #29     AppContext.run.<anonymous closure> (package:flutter_tools/src/base/context.dart:150:19)
      <asynchronous suspension>
      #30     main (package:flutter_tools/executable.dart:94:3)
      <asynchronous suspension>

[✓] VS Code (version 1.66.2)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.38.1

[✓] Connected device (2 available)
    • macOS (desktop) • macos  • darwin-arm64   • macOS 12.3.1 21E258 darwin-arm
    • Chrome (web)    • chrome • web-javascript • Google Chrome 100.0.4896.127

[✓] HTTP Host Availability
    • All required HTTP hosts are available

! Doctor found issues in 1 category.

[✓] Flutter (Channel master, 2.13.0-0.0.pre.602, on macOS 12.3.1 21E258 darwin-arm, locale en-GB)
    • Flutter version 2.13.0-0.0.pre.602 at /Users/nexus/dev/sdks/flutters
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision f4875ae865 (16 hours ago), 2022-04-17 15:19:07 -0400
    • Engine revision 3c6d4da868
    • Dart version 2.18.0 (build 2.18.0-24.0.dev)
    • DevTools version 2.12.2

[✓] Android toolchain - develop for Android devices (Android SDK version 31.0.0)
    • Android SDK at /Users/nexus/Library/Android/sdk
    • Platform android-32, build-tools 31.0.0
    • Java binary at: /Applications/Android Studio.app/Contents/jre/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.11+0-b60-7772763)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 13.2.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • CocoaPods version 1.11.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2021.1)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.11+0-b60-7772763)

[✓] IntelliJ IDEA Community Edition (version 2021.3)
    • IntelliJ at /Applications/IntelliJ IDEA CE.app
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin version 213.5744.122

[✓] VS Code (version 1.66.2)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.38.1

[✓] Connected device (2 available)
    • macOS (desktop) • macos  • darwin-arm64   • macOS 12.3.1 21E258 darwin-arm
    • Chrome (web)    • chrome • web-javascript • Google Chrome 100.0.4896.127

[✓] HTTP Host Availability
    • All required HTTP hosts are available

• No issues found!

Hello, I would like to highlight that this bug seems to be happening not only with buttons.
In our application (https://fantasy.formula1.com/) we are suffering this same error in a ListView with other elements aside buttons. I am attaching some screenshots to illustrate the issue. This error can be triggered here https://fantasy.formula1.com/app/#/team/1 after clicks to show the player list.


Want to know about Tauri

I'm going to replace electron, so I want to know if taui supports win7? Is Win XP supported?

Win XP? I don't think we've ever even run a test on Win XP, so I can't say for certain. Your best bet would be to ask at the discord server:
https://discord.gg/jSeuTeS


Linux: Support TCP_FASTOPEN

TCP_FASTOPEN requires to replace read and write syscalls to sendto and recvfrom.
I propose to modify the TcpStream and TcpListener in sys::unix like this:
pub enum TcpStream {
    // TCP_FASTOPEN requires this SocketAddr to be passed into sendto and recvfrom
    FastOpen(net::TcpStream, SocketAddr),
    Default(net::TcpStream),
}

You can use fast open right now w/o mio by managing this state yourself. The goal of Mio is to stay minimal. Could you explain why you think Mio should apply this change?

Well, the Read and Write traits that implemented for TcpStream will call read and write. And other crates, like tokio, will call Read::read and Write::write in their code. So if I want to use fastopen in my project with tokio, TcpStream in Mio must support reading and writing with sendto and recvfrom.

What's your opinion? @carllerche

Maybe this should be experimented with first out of crate. I don't think anything in mio/ tokio prevents a tcp-fast-open crate from existing?

Related to #268


Multipart mixed responses generate warnings

If you read a multipart/mixed response, the connectionpool issues a couple of warnings about defects in the message. I'm not sure what the expected, correct behavior is, but these warnings seem spurious.
Stick this perl script somewhere:
#!/usr/bin/perl

print "Server: Some Server Name\r\n";
print "Content-Type: multipart/mixed; boundary=36eeb8c4e26d842a\r\n";
print "Content-Length: 178\r\n";
print "\r\n\r\n";
print "--36eeb8c4e26d842a\r\n";
print "Content-Type: text/plain\r\n";
print "\r\n";
print "7\r\n";
print "--36eeb8c4e26d842a\r\n";
print "Content-Type: text/plain\r\n";
print "\r\n";
print "9\r\n";
print "--36eeb8c4e26d842a\r\n";
print "Content-Type: text/plain\r\n";
print "\r\n";
print "11\r\n";
print "--36eeb8c4e26d842a--\r\n";

Read it with requests (naturally, you'll have to change the URI to wherever you put the script):
import requests, logging

logging.basicConfig(level=logging.WARNING)
logging.getLogger("requests").setLevel(logging.DEBUG)

headers = {'accept': "multipart/mixed"}
r = requests.get("http://localhost:8124/cgi-bin/mpm.pl", headers=headers)

print(r)

The following errors are displayed:
DEBUG:requests.packages.urllib3.connectionpool:"GET http://localhost:8124/cgi-bin/mpm.pl HTTP/1.1" 200 178
WARNING:requests.packages.urllib3.connectionpool:Failed to parse headers (url=http://localhost:8888/http://localhost:8124/cgi-bin/mpm.pl): [StartBoundaryNotFoundDefect(), MultipartInvariantViolationDefect()], unparsed data: ''
Traceback (most recent call last):
  File "/home/ndw/.virtualenvs/pyapi/lib/python3.4/site-packages/requests-2.8.0-py3.4.egg/requests/packages/urllib3/connectionpool.py", line 390, in _make_request
    assert_header_parsing(httplib_response.msg)
  File "/home/ndw/.virtualenvs/pyapi/lib/python3.4/site-packages/requests-2.8.0-py3.4.egg/requests/packages/urllib3/util/response.py", line 58, in assert_header_parsing
    raise HeaderParsingError(defects=defects, unparsed_data=unparsed_data)
requests.packages.urllib3.exceptions.HeaderParsingError: [StartBoundaryNotFoundDefect(), MultipartInvariantViolationDefect()], unparsed data: ''

It took me quite a while to work out that they were spurious (because in real life, the server side that is generating the multipart/mixed is more complicated!)

Ah, can you raise this issue on the urllib3 repo! We'd love to merge a fix at the project! It's here.


:bug: saving input data for upload

Describe the bug
The dataset which is uploaded is saved locally. However, the data does not seem to be forwarded correctly to the function which should save it.
To Reproduce
Try to submit a FragPipe run.
Expected behavior
get the url with the pr
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):

OS: Windows
Browser chrome

Additional context
INFO:Writing the json to: C:\Users\user\AppData\Local\Temp\tmpyfi6_kwk\results.json
Following Github server redirection from /repos/Proteobot/Results_Module2_quant_DDA to /repositories/594        032348
INFO:Following Github server redirection from /repos/Proteobot/Results_Module2_quant_DDA to /repositorie        s/594032348
Following Github server redirection from /repos/Proteobot/Results_quant_ion_DDA/branches/master to /repo        s/Proteobot/Results_quant_ion_DDA/branches/main
INFO:Following Github server redirection from /repos/Proteobot/Results_quant_ion_DDA/branches/master to
DA/branches/master to /repos/Proteobot/Results_quant_ion_DDA/branches/main
2024-04-25 11:04:41.995 Uncaught app exception
Traceback (most recent call last):
  File "C:\Users\user\Anaconda3\envs\proteobench\lib\site-packages\streamlit\runtime\scriptrunner\script_runner.py", line 535, in _run_script
    exec(code, module.__dict__)
  File "C:\Users\user\Documents\repos\ProteoBench\webinterface\pages\DDA_Quant_ion.py", line 667, in <module>
    StreamlitUI()
  File "C:\Users\user\Documents\repos\ProteoBench\webinterface\pages\DDA_Quant_ion.py", line 56, in __init__
    self._main_page()
  File "C:\Users\user\Documents\repos\ProteoBench\webinterface\pages\DDA_Quant_ion.py", line 157, in _main_page
    self._populate_results()
  File "C:\Users\user\Documents\repos\ProteoBench\webinterface\pages\DDA_Quant_ion.py", line 217, in _populate_results
    self.generate_results("", None, None, False, None)
  File "C:\Users\user\Documents\repos\ProteoBench\webinterface\pages\DDA_Quant_ion.py", line 598, in generate_results
    pr_url = self.make_submission_webinterface(
  File "C:\Users\user\Documents\repos\ProteoBench\webinterface\pages\DDA_Quant_ion.py", line 368, in make_submission_webinterface
    self.ionmodule.write_intermediate_raw(
  File "C:\Users\user\Documents\repos\ProteoBench\proteobench\modules\dda_quant_base\module.py", line 219, in
write_intermediate_raw
    input_df.to_csv(os.path.join(path_write, "input_df.csv"))
AttributeError: 'NoneType' object has no attribute 'to_csv'

The issue is that assigning the session states does not seem to work:
https://github.com/Proteobench/ProteoBench/blob/170d83e9c537324f81d12893d4ad66c418233653/webinterface/pages/DDA_Quant_ion.py#L578-L580
Inspecting the session state after running this line, I find
st.session_state[self.variables_dda_quant.input_df] # None
self.variables_dda_quant.input_df # pandas.DataFrame as expected

So it would be easiest to just forward the input_df and all_datapoints directly?
https://github.com/Proteobench/ProteoBench/blob/170d83e9c537324f81d12893d4ad66c418233653/webinterface/pages/DDA_Quant_ion.py#L597-L602

Okay. that's not it. input_df is just none. so some problem upstream of the function I fear.


Properly sort archetype version

Fix #974
In addition to this work, the https://helidon.io/cli-data/versions.xml has to be updated. The lower version should be qualified with snapshot.
From:
<rule archetype="[2.0.0,3.0.0)" cli="[2.0.0,5.0.0)"/>

to
<rule archetype="[2.0.0-snapshot,3.0.0)" cli="[2.0.0-snapshot,5.0.0)"/>

Otherwise most of the qualified version are out of the range.

Yes, sure. Having snapshot does not make sense.


[Proposal] The 2nd version of implementation for materialised view

The motivation of 'materialised view' is in issue #5218 .
The implementation of 'Materialised View' is diveded into two parts. One is the management of DerivedDatasource, and the other is optimizing query with DerivedDatasource.
Management of DerivedDatasource
Each derived-datasource is managed by a derived-datasource supervisor(like kafkaSupervisor).
CREATE
DerivedDatasource is created when user submits a derived-datasource supervisor. The json file of derived-datasource supervisor should include base datasource name, dimensions and metrics at least. DerivedDatasource name can be setted by users or generated by base datasource name, dimensions and metrics. Segment granularity and query granularity should keep the same with base datasource. The information is stored as metadata of supervisors.
MAINTAIN
A derived-datasource supervisor should make sure timeline and segment versions are the same with its base datasource (It can sovle the problem about segment version management @himanshug pointed out):

When timeline of derived-datasource is less than base-datasource, supervisor will submit a derived-datasource-index-task.  The derived-datasource-index-task is a hadoop-index-task. The only difference is that the version of segments generated by derived-datasource-index-task is not the time when task is submitted, but the related base datasource segments version.
If the timeline of derived-datasource is more than base-datasource, supervisor will set used=false and submit kill tasks for those segments.
Once supervisor find the segment version of derived-datasource is different from its related base-datasource in the same interval, supervisor will set used=false and submit a derived-datasource-index-task for that interval (This idea comes from @jihoonson 's suggestions).

DELETE
When the supervisor is shutdown or resubmited, the previous data of the derived-datasource will be deleted.
When the base datasource is disable, all its derived datasource supervisor will be shutdown.
Optimize query with DerivedDatasource
A MaterialisedViewQueryRunner is added in method applyPreMergeDecoration() of FluentQueryRunnerBuilder, such as
    public FluentQueryRunner applyPreMergeDecoration()
    {
      return from(
          new UnionQueryRunner<T>(
              new MaterialisedViewQueryRunner(
                  toolChest.preMergeQueryDecoration(
                      baseRunner
                  )
              )
          )
      );
    }

In MaterialisedViewQueryRunner, query is rewritten into union queries of derived-datasource and base datasource and merged results of all queries before returnning. The process of optimization is as follows.

Check if the datasource is a table datasource. If not, do not optimize.
Check if the datasource has derived-datasources, and find the derived-datasources which involve the dimensions and metrics the query need. If no derived datasource meet the condition, do not optimize.
Split query interval by segment granularity into sub-intervals, and for each sub-interval, find out the derived-datasource which has the minimum amount of data. Then, replace the query datasource by the derived-datasource in the sub-interval.

In this way, query can be partially covered by the derived datasource and partially by the original one, and the problem @nishantmonu51 pointed out is solved.

Hi zhangxinyu1,
thank you so much for pushing for this feature. I've been wanting this for a long time and tried to raise awareness for it. This feature is definitely needed. I don't even understand how one can do without it.
I read the comments from the previous proposal.
Let me add a few things to those.

It was being said that the idea of such a mechanism isn't new and it was mentioned that Vertica has a similar feature named "projections". In Vertica, projections have the same number of rows as the original table. So same same but different. The actual prior art would be OLAP. Many databased call themselves OLAP although they are missing every feature of it. These derived datasources in OLAP terms are called cuboids. And yes, it DOES speed up queries and no it is NOT an option to just add more hardware to the query layer. We have a Druid cluster which is dramatically slow because we need to have many dimensions. If a user makes a wide query and trillions of rows need to be scanned, even a large Druid cluster will need minutes to complete that query. The current caching mechanisms will not help with that. How would I scale up the cluster? If a given query takes one minute to execute I would need to make my cluster 60 times bigger to get the query to subsecond latency.
It was being said that people are manually reindexing secondary datasources. However, this becomes quite unfeasible very quickly. How to make sure that query results are consistent?
This heavy lifting should be done by a database system for exactly that purpose that it then isn't required anymore for everyone using the database to do the heavy lifting.

@zhangxinyu1 as I already said, I have invested a lot of thought on bringing this feature to Druid already. I so far always referred to this feature as "Datasource Delegation". Lets stick our heads together to figure this out.
The query path isn't that difficult. One can indeed mix and match which of the many secondary databases to include within a query. That's the beauty of it. An incoming query could be rewritten such that 95% of the timeline of a given query could be covered by the best secondary database, then maybe another 4% is served by segments coming from a bigger secondary database and most likely there will alsways be some segments that have not been re-indexed to some secondary database yet and which therefore would still come from the master datasource.
The most difficult part is to keep track on a per segment granularty basis whether a secondary datasource is in syn with the master. Those timeranges that are not in sync are not elligible to querying.
For the query path, it would also be great if historical queries could contain segment lists coming from different datasources. A historical receive queries that the broker has already rewritten such that there is a list of segments contained in a historical's query. Looking at this segment list, each segment is referenced by its complete ID with the exception of the datasource that the segment belongs to. If one could improve the historical queries such that a list of segments could be passed where every segment also contains the datasource it belongs to, then historicals could already merge data across datasources.
This would totally work: if the broker has determined that a secondary datasource B is up-to-date with master datasouce A for a given segment-granularity slot and therefore picks some segments form B while choosing segments from A for some other time-interval, then a historical could execute a query on both kinds of segments because everything is column-oriented.
Some months back I started implementing this feature but never completed it.
There are other OLAP features that one could nicely layer on top of your proposed solution at a later time. For instance "cell materialization". I once posted a proposal for a different feature which then turned into a discussion of your feature. Perhaps you'd like to read up on it here: https://groups.google.com/forum/#!searchin/druid-development/delegation|sort:date/druid-development/Ij3gJjuD4nY/ryC_aCw7GQAJ

@zhangxinyu1 first of all, I really apologize for late response. I has been quite busy and forgot about this issue. Sorry.
The updated proposal looks very good to me. Especially the query optimization part sounds good enough as a first step. Regarding the management of derivedDatasources, I have some comments.

Segment granularity and query granularity should keep the same with base datasource.

The segment granularity of the base dataSource can be changed. For example, compactionTask merges all segments in the given interval regardless of whatever their segment granularity was. Keeping query granularity sounds good.

The only difference is that the version of segments generated by derived-datasource-index-task is not the time when task is submitted, but the related base datasource segments version.

I think this might be hacky because it needs to add an exception to our segment versioning system for only derived dataSources. I would think it might be better to keep the version of the base segments which are used for generating the latest derived dataSource. If the version of the base segments is larger than the one in derivedDataSourceSupervisor, the derived dataSource should be disabled and a new task to make the fresh derived dataSource can be submitted.

If the timeline of derived-datasource is more than base-datasource, supervisor will set used=false and submit kill tasks for those segments.

Probably we need to keep a mapping information between original segments and derived segments.

When the base datasource is disable, all its derived datasource supervisor will be shutdown.

I wonder how this can be achieved. Do you have an idea about where the relationships between base dataSources and their derived dataSources are stored?
I think this is a really nice proposal and hope we can make it.

@sascha-coenen thanks for your approval.
I'm working on implementing this feature. I'm thinking about how to call it, materialized view selection and maintenance or cuboid selection and maintenance, do you have any suggestions?

For the query path, it would also be great if historical queries could contain segment lists coming from different datasources. A historical receive queries that the broker has already rewritten such that there is a list of segments contained in a historical's query. Looking at this segment list, each segment is referenced by its complete ID with the exception of the datasource that the segment belongs to. If one could improve the historical queries such that a list of segments could be passed where every segment also contains the datasource it belongs to, then historicals could already merge data across datasources.

This is a good idea, It can reduce the cost of data merge on broker. I will try to add this feature in the future.
I will come up with a pr about this feature in few days. Can you help me check it out?
thx.

@jihoonson thx for your comments, I will submit a pr in these days. I already implement this function as following:

About the segment granularity, I generate data of derived datasource based on hadoop-index-task, and the segment granularity is the same with base datasource. There are two reasons why I implement it like this:

To make sure the segment versions of base datasource and derived datasource are the same. I used the max(created_date) of segment of  base datasource as the segment version of derived datasource. Thus once there are new segment of base datasource created, the  max(created_date) will change and derived datasource supervisor will rebuild segments of derived datasource.
I think it's easier for derived datasource supervisor to keep the timeline's consistency of base datasource and derived datasource.
e.g. If base datasource A's segment granularity is hour and derived datasource B's is day,  then the change of data during any hour of one day will lead to rebuilding of segment of derived datasource. On the other hand, if A only keeps recently 12 hours' data, it's hard for supervisor to maintain the derived datasource. So I think it's better to make them the same.

If base datasource is disabled, derived datasource supervisor will not be shut down. But it doesn't matter since the duty of derived datasource supervisor is to keep the consistency of timeline between derived datasource and base datasource. So when base datasource is disabled,  derived datasource won't have any data.

@zhangxinyu1 it sounds great that you're already working on this! Your explanation sounds good to me overall, but I have one question.

To make sure the segment versions of base datasource and derived datasource are the same. I used the max(created_date) of segment of base datasource as the segment version of derived datasource.

Does this mean that the segment version of base dataSource and derived dataSource are literally same? Would you elaborate more on how you could do this? I'm asking because our segment versioning system is already quite complicated and I hope we don't need to add a sort of special handling for derived dataSource.

Since I can get the max created_date of base datasource segments of specified interval, and it will change as the base segments change, then I use it as the version of derived segment of the specified interval. And I add a derived datasource supervisor to control these. Once the base segment changes, supervisor will find the version of derived segment is different with the max created_date of base one, then the related derived segments will be unused and supervisor will submit a hadoop-index-task with the information of all needed related segments of base datasource to generate the new derived segments.

@zhangxinyu1 ah sorry! My question was a little bit vague.

Since I can get the max created_date of base datasource segments of specified interval, and it will change as the base segments change, then I use it as the version of derived segment of the specified interval.

My question is, how will you implement the segmentAllocator to use the same segment version of the base dataSource for the corresponding derived dataSource. Do you assume that any types of tasks can't overwrite/append the derived dataSources unless they are submitted by the derived dataSource supervisor? I think it might be a valid assumption, and if so, segmentAllocator can be implemented to allocate segments based on only supervisor's local information without using IndexerSQLMetadataStorageCoordinator.

@jihoonson
Any types of tasks can overwrite/append the derived dataSource.  Derived datasource supervisor submit hadoop-index-tasks to generate data. In the HadoopTuningConfig of hadoop-index-task, version is the max created_date and 'useExplicitVersion' is true.

@zhangxinyu1 oh I see. That would work.
But, if possible, I think it would be better to use our existing versioning system (e.g., ActionBasedSegmentAllocator) rather than separately maintaining segment versions in derived dataSource supervisors. This is because segment version management can be done in one class (TaskLockBox) rather than being spread over different classes depending on ingestion types. Another reason is that only Hadoop index task supports explicitly specifying segment versions in the task spec. As a result, users should have a Hadoop cluster to use this cool feature.
Do you think it's possible? I guess maybe it is if we keep the segment versions of the original dataSource corresponding to the current derived dataSources in the supervisor. Once the actual segment version of the original dataSource is higher than the one kept in the supervisor, then it is triggered to rebuild derived dataSources.

@jihoonson  Thanks for your suggestions
Derived dataSource supervisor would not maintain any segment versions. It's stateless. The things supervisor do is:

Reading the max created_date of base dataSource of one interval from db.
Reading the version of derived dataSource of the interval from db.
compare the max created_date and the version. if they are the same, do nothing. otherwise, submit a task with bases datasource DataSegment to rebuild derived dataSource for the interval.
In this way, derived dataSource supervisor needn't store any information about the segment version.

But, you'r right, only hadoop-index-task supported is not enough. I think batch ingestion is suitable for derived datasource rebuilding, and the current batch ingestion can run on peon and run on hadoop cluster. Maybe we should implement one materialized-view-task to support both kinds of ingestion and use the existing version system in the next version.

if we keep the segment versions of the original dataSource corresponding to the current derived dataSources in the supervisor. Once the actual segment version of the original dataSource is higher than the one kept in the supervisor, then it is triggered to rebuild derived dataSources.

If the base datasource segments are generated by kafka-index-service, using the original dataSource version as the derived dataSource version may have some problems. This is because kafka-index-task can generate new segments with the same version. These new segments with the same version would not trigger derived datasource rebuilding.

@zhangxinyu1 thanks for the detailed explanation.

Derived dataSource supervisor would not store any segment versions. It's stateless.

Yes, I see your point. But actually the algorithm is stateful because it works based the states (created_date and version) stored in metastore. However, this doesn't always work. Druid's segment versioning system has a rule that recent segments have higher versions than old segments. Using time as the segment version usually works, but we still consider using something else if there is an issue in the future. created_time doesn't have such a rule. As a result, recent segments might have older created_time than old segments depending on overlord's wall clock time.
If you make the derived dataSource supervisor stateless, I think it should be something different from the proposed one. One possible solution is triggering updates on derived dataSources whenever a base dataSource segment is published. I already commented here about this idea even though it was too rough. So, I'll explain more details. Publishing segments is completed only after an overlord updates the metadata of those segments in metastore. Tasks should use SegmentTransactionalInsertAction to do it. I think we can add a something new feature like triggerAction which is triggered upon a successful segment insertion in SegmentTransactionalInsertAction. The triggerAction should be a general abstraction of any action. To update derived dataSources, tasks can specify a triggerAction which notifies the published segment information to derived dataSource supervisors. Once a supervisor receives an event from the triggerAction, it can submit a task to update its derived dataSource. Since the segment management is done by coordinators, the triggerAction should be able to be called in coordinators too. In this way, the supervisors don't have to check any state like comparing the created_date of base dataSource segments and the version of derived dataSource segments.
But, I think it's quite complicated and it's always a good idea to start with a simple implementation. If you want to implement a good stateless system for derived dataSource, it would be great. However, if it's not easy, I would say it's better to start with a simpler one which can be easily improved in the future. IMO, using created_date as the segment version might be simple to implement, but is not easy to improve in the future because it adds an assumption that the ordering of created_date of segments is guaranteed. I don't think this assumption should be guaranteed.

Could you please explain why using TaskLockBox to management segment version is better than use existing tasks.

All types of tasks use LockTryAcquireAction or LockAcquireAction which internally get a lock using TaskLockBox. And the acquired lock has a valid segment version. All task types are supposed to use this version.
Specifying a user-customized version in tasks (like Hadoop task) is dangerous. The result segments might not be visible or, even worse, overwrite existing segments unexpectedly. As a result, this should be used very carefully.
Maybe this is not an issue for derived dataSources if they can be created/updated/deleted via only their supervisors (BTW, is this the assumption you have?). However, I'm still not sure about using the created_time as the segment version.

Maybe we should implement one materialized-view-task to support both kinds of ingestion and use the existing version system in the next version.

That makes sense if needed. Would you elaborate a little bit more on the new task type you think?

If the base datasource segments are generated by kafka-index-service, using the original dataSource version as the derived dataSource version may have some problems. This is because kafka-index-task can generate new segments with the same version. These new segments with the same version would not trigger derived datasource rebuilding.

Thanks. I missed Kafka indexing service.

@jihoonson thank you so much for explaining the plan of segment version.
Yes, starting with a simple implementation is better, and we can impove it later. Using created_time as segment version is not good, but i can't thing of a better way to implement it.
I'm still wondering why the specified segment version is not allowed in TaskLockbox. I think it is flexible to use the version generated by TaskLockBox automatically by default and also allow users to set specified versions when they need. It can be considered as one of advanced usages. if users choose to use specified versions, they should know the results.

@zhangxinyu1 thank you for the explanation. Maybe it's better to add a restriction that only supervisors can modify derived dataSources in the future.

Using created_time as segment version is not good, but i can't thing of a better way to implement it. do you have any ideas?

Well, I would say it's better to start with a stateful implementation for the supervisor even though it's not scalable. For example, the supervisor may keep all mapping information between a base dataSource and its derived dataSources in somewhere like memory, disk or metastore. Since the number of segments of a dataSource is usually several hundreds, this might not be a big overhead while the number of derived dataSources is kept small.

I'm still wondering why the specified segment version is not allowed in TaskLockbox. I think it is flexible to use the version generated by TaskLockBox automatically by default and also allow users to set specified versions when they need. It can be considered as one of advanced usages. if users choose to use specified versions, they should know the results.

Actually, the current implementation of TaskLockbox allows to use a specified segment version. That's how Hadoop tasks allow a user-customized version in their spec. But I don't know why it's allowed in TaskLockbox and Hadoop tasks from the first place. Maybe there were some issues like bugs in TaskLockbox in the past, but now, AFAIK, there is no such issue.
IMO, when we add a new feature, we first need to check that the feature is really necessary and useful for users. For custom versions, however, it's difficult for me to imagine which kinds of applications need to specify custom versions instead of using automatically assigned always valid versions. Would you give some examples if you have?

@jihoonson

Well, I would say it's better to start with a stateful implementation for the supervisor even though it's not scalable. For example, the supervisor may keep all mapping information between a base dataSource and its derived dataSources in somewhere like memory, disk or metastore. Since the number of segments of a dataSource is usually several hundreds, this might not be a big overhead while the number of derived dataSources is kept small.

I thought about this mapping before. For each derived datasource, this mapping can be interval -> List of base datasource and stored in database when supervisor commit datasource metadata. However, in my experience, some dataSources have 200-400 segments and about 200G each day. Druid don't work well on these kinds of dataSource. We add more than 10 derived dataSources for each kind of this big dataSource to make query faster. If we keep a such mapping for each derived dataSource,  I'm afraid that the overhead is too large. Typically, this kind of big dataSource is the best user of materialized view. However, after seeing your suggestions, I do not know which method is better now.

Actually, the current implementation of TaskLockbox allows to use a specified segment version.

I mean TaskLockbox doesn't have a method to set preferredVersion, instead, tasks can choose to use a specified version but not the version TaskLockbox returns. Anyway, this feature is only used when I want to control the segment version of derived dataSources in derived dataSource supervisor. Maybe it's a pseudo demand.

in my experience, some dataSources have 200-400 segments and about 100-200G each day. Druid don't work well on these kinds of dataSource. We add more than 10 derived dataSources for each big dataSource to make query faster. If we keep such a big mapping for each derived dataSource, I'm afraid that the overhead is too large. Typically, this kind of big dataSource is the best user of materialized view.

That's a good point. Then, I think we need to design a fully stateless supervisor. The algorithm I suggested above is one of them. The implementation might not be such complicated even though it requires some design reviews which might take longer review cycles.
If you have any other idea for the stateless supervisor, Please let me know.

@jihoonson
Could we use the current implementation before the fully stateless supervisor is well designed?  Because I believe many druid users want to use this feature as soon as fast.
Besides, the current implementation have two extensions without any modify of core code. After we designed a stateless supervisor, it's easy to upgrade from current implementation.
Could you please look at the pr about Materialized view implementation?

@zhangxinyu1 thanks for raising #5556. I'll take a look.

Could we use the current implementation before the fully stateless supervisor is well designed?

I think it makes sense because you already have a full implementation even though it needs to be reviewed. However, it should be noted that this feature is still experimental and what happens if created_date of a new base segment is accidentally smaller than the version of an existing segment of a derived dataSource.

@jihoonson Thank you!

However, it should be noted that this feature is still experimental and what happens if created_date of a new base segment is accidentally smaller than the version of an existing segment of a derived dataSource.

Should these notes be added to the final documentation about materialized view?

@zhangxinyu1 I think that should be documented from the first place when https://github.com/druid-io/druid/pull/5556 is merged.


move dataset parameter to main section

move dataset parameter to main section

[ ] Validate

@irangareddy please add this on top of change log:
## [Unreleased]

### Changed

- "Messages Dataset" Parameter moved from the advanced section to "normal" section (will change again in future)

Sure, will do that!


build: Add support to strip the binary

This provides a flag "STRIP=yes" to strip the golang binary
After this patch, the binary size is reduced a lot:
19356680 containerd-shim-kata-v2*
25980728 containerd-shim-kata-v2.nostip*
4021784 kata-netmon*
5093992 kata-netmon.nostrip*
26339392 kata-runtime*
33097344 kata-runtime.nostrip*
Fixes: #2455
Signed-off-by: Jia He<EMAIL_ADDRESS>

/test

lgtm. Just wanted to confirm this has no impact on stack traces or tracing that we use today.

lgtm. Just wanted to confirm this has no impact on stack traces or tracing that we use today.

Hi @amshinde, I tested 2 cases to address your comments:

for the stack trace, the answer is no, no impact
I wrote a simple helloworld app and stripped it, the stack info can be printed as no stripping

root@entos-thunderx2-02:~/t# cat helloworld.go
package main
import "fmt"
import "runtime/debug"

func main(){
        fmt.Println("Hello World")
debug.PrintStack()
}
root@entos-thunderx2-02:~/t# go build -ldflags "-w -s" -o helloworld.strip
root@entos-thunderx2-02:~/t# ./helloworld.strip
Hello World
goroutine 1 [running]:
runtime/debug.Stack(0x4000242780, 0x1, 0x1)
        /usr/local/go/src/runtime/debug/stack.go:24 +0x88
runtime/debug.PrintStack()
        /usr/local/go/src/runtime/debug/stack.go:16 +0x18
main.main()
        /root/t/helloworld.go:7 +0x4c
root@entos-thunderx2-02:~/t# go build  -o helloworld.nostrip
root@entos-thunderx2-02:~/t# ./helloworld.nostrip
Hello World
goroutine 1 [running]:
runtime/debug.Stack(0x4000242780, 0x1, 0x1)
        /usr/local/go/src/runtime/debug/stack.go:24 +0x88
runtime/debug.PrintStack()
        /usr/local/go/src/runtime/debug/stack.go:16 +0x18
main.main()
        /root/t/helloworld.go:7 +0x4c
root@entos-thunderx2-02:~/t# file helloworld.nostrip
helloworld.nostrip: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, not stripped
root@entos-thunderx2-02:~/t# file helloworld.strip
helloworld.strip: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, stripped

as for tracing of Kata.
I tested on X86 server, there is also no impact on tracing

root@entos-dell:~/hj/src/github.com/kata-containers/tests/tracing# ./tracing-test.sh
30d9283ef678e753e332f8ca848224b3974ebc4ced3b5baa67bede9bfeea6053
INFO: Enabling kata tracing on /usr/share/defaults/kata-containers/configuration-qemu.toml
INFO: Checking runtime spans
INFO: Checking Jaeger status (and logging traces to /var/log/kata-tests/logs/traces/kata-runtime-traces.json)
INFO: found 489 spans (across 17 traces)
Journal file /var/log/journal/bcc730f60dcd4864b989b8e59fd20c39/user-1002.journal is truncated, ignoring file.
INFO: All 47 spans reported
INFO: test passed
INFO: Checking shim spans
INFO: Checking Jaeger status (and logging traces to /var/log/kata-tests/logs/traces/kata-shim-traces.json)
INFO: found 16 spans (across 2 traces)
Journal file /var/log/journal/bcc730f60dcd4864b989b8e59fd20c39/user-1002.journal is truncated, ignoring file.
INFO: All 8 spans reported
INFO: test passed
INFO: Removing cached trace logs from /var/log/kata-tests/logs/traces
jaeger
jaeger
INFO: Disabling kata tracing on /usr/share/defaults/kata-containers/configuration-qemu.toml

Thanks @justin-he for confirming.


Add alicloud_ecs_snapshot_[dashboard, age, public access]

[x] alicloud_ecs_snapshot_dashboard
[x] alicloud_ecs_snapshot_age_report
[x] alicloud_ecs_snapshot_public_access_report

Closed as a part of https://github.com/turbot/steampipe-mod-alicloud-insights/pull/20


Need Assistance on Start stop v2 service

We have logic app configured for scheduled start/stop, so in that schedule scope if one VM gets fail to start/stop other VM also did not start/stop bcz schedule gets terminated and schedule run status showing failed.

Please reach out to Azure Support for further assistance.


No "exports" main defined in helia package

I am encountering the following error when trying to import the helia package in my TypeScript project:
Error: No "exports" main defined in C:\Users\saeed\type_script_projects\digital-asset-tracker\Backend\node_modules\helia\package.json
    at exportsNotFound (node:internal/modules/esm/resolve:296:10)
    at packageExportsResolve (node:internal/modules/esm/resolve:586:13)
    at resolveExports (node:internal/modules/cjs/loader:640:36)
    at Function._findPath (node:internal/modules/cjs/loader:748:31)
    ...
    at Module.require (node:internal/modules/cjs/loader:134:12) {
  code: 'ERR_PACKAGE_PATH_NOT_EXPORTED'
}

Steps to Reproduce

Install the helia package using npm:
npm install helia

Attempt to import and use helia in my project:
import { createHelia } from 'helia';
Run the project, which leads to the above error.

what node version are you running?

what node version are you running?

Node.js v22.11.0

could you try importing any other package?  Just to confirm that it is no general problem with your environment


Change icon and tooltip for "Archive stream"

The "Archive stream" button in the stream settings menu has a trash can icon. This is confusing, as users may assume the stream is being fully deleted (i.e. all messages will be gone). We should replace the icon with the filing cabinet icon commonly used for archiving (https://fontawesome.com/v4.7/icon/archive).
We should also change the tooltip from "Archive stream" to "Archive stream for everyone" to make it clear that this is an org-wide action.

I would to take this up.


Guidelines

Hello,
Please add some help instructions & how to use this plugin ..

It’s not a plugin but a template. I will update the docs though 😁


Fixing tests for initialRouteParams PR

Fixed the tests from this pr: https://github.com/react-community/react-navigation/pull/150.
Came across needing this for my project,
Thanks!

For visibility @grabbou @satya164 @juhasuni

Looks like we got another solution in around the same time as I posted this. So closing!


feat: support compute DER for multiple pairs of ref and hyp

Support compute DER for multiple pairs of ref and hyp, by inputting dict or list type of ref and hyp.
Please check README and the code.

Thanks for your contribution. I'll review this later, but please ping here if I don't get around to it by end of this month.

Hi @desh2608. I came across a problem with this commit.
The double free or corruption (out) error occurred at the below position, when I was computing multiple DERs during the training of a fairseq-based model.
https://github.com/desh2608/spyder/blob/9dda8f40f2f2d54eade2a129d1394d00b7d1d1c5/src/spyder/der.cc#L137
Everything seems OK when out of the model training.
Do you have any suggestions on this? Thanks so much for your help.

Can you look at the segments for which the error occurred and see if you can find something weird? This may be a corner case that needs to be fixed.

I'll check this later.

Hi @desh2608. I believe the reason is memory conflict when I use multi-thread training.
Everything looks good when inference. This PR does not cause the error.

LGTM. Merging.


Fix pooch

Fixes a variety of problems with the implementation of fetch_dataset().  Also updates the version number and hash of the MNE-misc dataset, which just got a new release today.

Also updates the version number and hash of the MNE-misc dataset, which just got a new release today.

This will conflict with #9800, but @alexrockhill is an expert merger at this point so I think it's okay :)

@adam2392 can you review?

@adam2392 can you review?

@drammock can you force CIs to do some review work for us, too, by creating some new test that fails on main but passes on this PR? Presumably downloading some .mat file like in https://github.com/alphacsc/alphacsc/pull/44#issuecomment-932315770 (I would look at pymatreader for some stable .mat testing file you could use) ? It looks like the only test adjustment is for a dataset version which looks reasonable, but wouldn't have caught the alphacsc error.
Feel free to ignore if this does not make sense or you were going to do this anyway...

can you force CIs to do some review work for us, too, by creating some new test that fails on main but passes on this PR?

done.

CI fails are unrelated:

test_notebook_alignment and test_notebook_button_counts: DeprecationWarning: Passing a schema to Validator.iter_errors is deprecated and will be removed in a future release. Call validator.evolve(schema=new_schema).iter_errors(...) instead.
test_3d_functions[mayavi]:

ResourceWarning: unclosed context
ResourceWarning: unclosed event loop

Thanks @drammock !


App using Flutter 3.10 is more janky on iOS than previous versions.

Details

After upgrading to Flutter 3.10 for my app Hacki, there's obviously more janks than previous versions without impeller. (on iOS, I haven't tried for Android)
You can see for yourself by comparing the performance between master branch and flutter-upgrade branch
The app itself doesn't need any extra configurations, you can just clone it and run it with flutter run.

Flutter 3.7.11
Flutter 3.10

Target Platform: iOS
Target OS version/browser: iOS 16.4
Devices: iPhone 14
Logs

Logs

[!] Flutter (Channel stable, 3.10.0, on macOS 13.2.1 22D68 darwin-arm64
    (Rosetta), locale en)
    ! Warning: `flutter` on your path resolves to
      /Users/jiaqifeng/Dev/flutter/bin/flutter, which is not inside your current
      Flutter SDK checkout at /Users/jiaqifeng/dev/flutter. Consider adding
      /Users/jiaqifeng/dev/flutter/bin to the front of your path.
    ! Warning: `dart` on your path resolves to
      /opt/homebrew/Cellar/dart/2.17.0/libexec/bin/dart, which is not inside
      your current Flutter SDK checkout at /Users/jiaqifeng/dev/flutter.
      Consider adding /Users/jiaqifeng/dev/flutter/bin to the front of your
      path.
[!] Android toolchain - develop for Android devices (Android SDK version 31.0.0)
    ✗ cmdline-tools component is missing
      Run `path/to/sdkmanager --install "cmdline-tools;latest"`
      See https://developer.android.com/studio/command-line for more details.
    ✗ Android license status unknown.
      Run `flutter doctor --android-licenses` to accept the SDK licenses.
      See https://flutter.dev/docs/get-started/install/macos#android-setup for
      more details.
[✓] Xcode - develop for iOS and macOS (Xcode 14.3)
[✓] Chrome - develop for the web
[✓] Android Studio (version 2021.3)
[✓] VS Code (version 1.78.1)
[✓] Connected device (3 available)
[✓] Network resources

Hi @jonahwilliams, I just attached two clips showcasing the difference. Hopefully it helps.

Ahh yeah, if I scroll it enough on 3.10 I can trigger some frame back ups. I belive this is fixed in the 3.11 beta, and at ToT I can no longer reproduce it. Please give it a shot if you have the time

@Livinglist
Please check on latest beta as stated above and confirm.

looks like the issue is fixed in 3.11 beta, thanks!


Chore: Figure out how to upload preview builds to pypi

Description
I can do this with duckdb:
pip install duckdb --pre --upgrade

And the install version is 0.9.1.dev19. Having the ability to use dev builds would be helpful when testing changes in various notebook environments.

please do, it is a killer feature !!!


Add Option Name as a Variant Metafield for In Stock Variants

See: https://mechanic.canny.io/task-requests/p/add-option-name-as-a-variant-metafield-for-in-stock-variants

I have read the CLA Document and I hereby sign the CLA

Hi Steven, thank you for the contribution, this is a great start!
I made some updates to your PR by adding the ability to configure multiple variant options, adding tags for task filtering, and normalizing the code format to be closer to what is generally found within the task library.
I'll make one more commit to update the task name to contain plurals in line with the updated focus.

@tekhaus Awesome! When I tried the multiple option names functionality I was getting tied in knots with indexing the keyval pairs in a way that would usable outside of a for loop with an index, the way you use metafield_{{ metafield_key | md5 }} for that is very smart.
I figured once you guys took a look and saw proof of concept there'd be syntax/logic improvements to be made, this is great.

@stevenhoney Ha, yep! It was interesting to (re)discover that metafield keys can contain spaces and throw lots of gnarly GraphQL errors when used as aliases. :)


Is Clerk middleware setup usable with NextJS nodejs runtime?

Preliminary Checks

[X] I have reviewed the documentation: https://clerk.com/docs

[X] I have searched for existing issues: https://github.com/clerk/javascript/issues

[X] I have not already reached out to Clerk support via email or Discord (if you have, no need to open an issue here)

[X] This issue is not a question, general help request, or anything other than a bug report directly related to Clerk. Please ask questions in our Discord community: https://clerk.com/discord.

Reproduction / Replay Link
asdf
Publishable key
asdf
Description
I'm getting the following error when trying to yarn build for production with NextJS and the nodejs runtime
./node_modules/scheduler/cjs/scheduler.production.min.js
A Node.js API is used (setImmediate at line: 51) which is not supported in the Edge Runtime.
Learn more: https://nextjs.org/docs/api-reference/edge-runtime

Import trace for requested module:
./node_modules/scheduler/cjs/scheduler.production.min.js
./node_modules/scheduler/index.js
./node_modules/react-dom/cjs/react-dom.production.min.js
./node_modules/react-dom/index.js
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
Environment
asdf

Or maybe it's more that there are functions used which are not available in the Edge runtime? (I'm trying to use node, but middleware does not seem to be available in node...?)
./node_modules/scheduler/cjs/scheduler.production.min.js
A Node.js API is used (setImmediate at line: 51) which is not supported in the Edge Runtime.
Learn more: https://nextjs.org/docs/api-reference/edge-runtime

Import trace for requested module:
./node_modules/scheduler/cjs/scheduler.production.min.js
./node_modules/scheduler/index.js
./node_modules/react-dom/cjs/react-dom.production.min.js
./node_modules/react-dom/index.js
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
./node_modules/scheduler/cjs/scheduler.production.min.js
A Node.js API is used (setImmediate at line: 51) which is not supported in the Edge Runtime.
Learn more: https://nextjs.org/docs/api-reference/edge-runtime

Import trace for requested module:
./node_modules/scheduler/cjs/scheduler.production.min.js
./node_modules/scheduler/index.js
./node_modules/react-dom/cjs/react-dom.production.min.js
./node_modules/react-dom/index.js
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
./node_modules/scheduler/cjs/scheduler.production.min.js
A Node.js API is used (MessageChannel at line: 120) which is not supported in the Edge Runtime.
Learn more: https://nextjs.org/docs/api-reference/edge-runtime

Import trace for requested module:
./node_modules/scheduler/cjs/scheduler.production.min.js
./node_modules/scheduler/index.js
./node_modules/react-dom/cjs/react-dom.production.min.js
./node_modules/react-dom/index.js
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
./node_modules/scheduler/cjs/scheduler.production.min.js
A Node.js API is used (MessageChannel at line: 121) which is not supported in the Edge Runtime.
Learn more: https://nextjs.org/docs/api-reference/edge-runtime

Import trace for requested module:
./node_modules/scheduler/cjs/scheduler.production.min.js
./node_modules/scheduler/index.js
./node_modules/react-dom/cjs/react-dom.production.min.js
./node_modules/react-dom/index.js
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
<EMAIL_ADDRESS>A Node.js API is used (MessageEvent at line: 27) which is not supported in the Edge Runtime.
Learn more: https://nextjs.org/docs/api-reference/edge-runtime

Import trace for requested module:
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>


Add alias to AI logger provider

In logging 2.0 we added ability to configure logging filters in centralized way. To configure filters per logging provider provider full name should be used or alternatively alias specified by ProviderAliasAttribute.
ProviderAliasAttribute is late bound and is detected by full type name so no reference to new aspnetcore packages is required.
@dnduffy @SergeyKanzhelev

I removed copyright from this file, an one other file it was accidentally added to in the past.

Can we ship build to the feed when this change goes in, so we can start E2E testing?

@pakrym Yes, we can make the builds auto-upload to myget, usually we only do the upload when we want to make something available (such as this change you requested) but we should automate it like some of the other builds do.


Re-add TheHive alerter without any libraries

Re add TheHive alerter that was removed in https://github.com/Yelp/elastalert/pull/2738 using no libraries.
Tested by capturing the requests from the old version and the new version using some example data and the documentation's example config:
Config:
rule = {'name': 'qlos rule', 'index': 'testindex'}
match = {'@timestamp': '2020-04-15T00:00:00Z',
         'field1': 'field1value',
         'field2': 'field2value',
         'ip_field': '<IP_ADDRESS>'}
config = yaml.load('''
config:
    hive_connection:
       hive_host: http://localhost
       hive_port: 12346
       hive_apikey: ABC123DEF
       hive_proxies:
         http: ''
         https: ''

    hive_alert_config:
        title: 'Title'  ## This will default to {rule[index]_rule[name]} if not provided
        type: 'external'
        source: 'elastalert'
        description: '{match[field1]} {rule[name]} Sample description'
        severity: 2
        tags: ['tag1', 'tag2 {rule[name]}']
        tlp: 3
        status: 'New'
        follow: True

    hive_observable_data_mapping:
        - domain: "{match[field1]}_{rule[name]}"
        - domain: "{match[field]}"
        - ip: "{match[ip_field]}"
''')['config']

Yielding:
POST /api/alert HTTP/1.1
Host: localhost:12346
User-Agent: python-requests/2.22.0
Accept-Encoding: gzip, deflate
Accept: */*
Connection: keep-alive
Content-Type: application/json
Authorization: Bearer ABC123DEF
Content-Length: 734

{
    "artifacts": [
        {
            "data": "field1value_qlos rule",
            "dataType": "domain",
            "message": null,
            "tags": [],
            "tlp": 2
        },
        {
            "data": "<IP_ADDRESS>",
            "dataType": "ip",
            "message": null,
            "tags": [],
            "tlp": 2
        }
    ],
    "caseTemplate": null,
    "customFields": {},
    "date":<PHONE_NUMBER>000,
    "description": "field1value qlos rule Sample description",
    "follow": true,
    "severity": 2,
    "source": "elastalert",
    "sourceRef": "f1f8a7",
    "status": "New",
    "tags": [
        "tag1",
        "tag2 qlos rule"
    ],
    "title": "Title",
    "tlp": 3,
    "type": "external"
}

Which compared to the old alerter's request:
 diff old_req new_req
8d7
< Content-Length: 693
9a9
> Content-Length: 734
30c30
<     "date":<PHONE_NUMBER>000,
---
>     "date":<PHONE_NUMBER>000,
31a32
>     "follow": true,
34c35,36
<     "sourceRef": "56e52a",
---
>     "sourceRef": "f1f8a7",
>     "status": "New",

"status" and "follow" are both valid API parameters (https://github.com/TheHive-Project/TheHiveDocs/blob/master/api/alert.md) but don't seem to be inlcuded in TheHive4Py for some reason.
"sourceref" and "date" of course are dynamic values.

Hi @Qmando  ,
please I have troubles after #2738 commit. I have new version of elastalert and I need to connect it with theHive api. After command elastalert --verbose --config /home/thehive/elastalert/config.yaml I got an error:
elastalert.util.EAException: Error loading file /home/thehive/elastalert/rules/failed_ssh_login.yaml: Error initiating alert ['hivealerter']: Could not import module hivealerter: need more than 1 value to unpack

I know that is connected with this issue, but I have already tried also your rule config example, but it didn't work for me as well. My elastalert config.yaml is:
rules_folder: /home/thehive/elastalert/rules
run_every:
  minutes: 1
buffer_time:
  minutes: 15
es_host: x.x.x.x
es_port: 9200
use_ssl: False
writeback_index: elastalert_status
alert_time_limit:
  days: 2

and test rule config is:
es_host: x.x.x.x
es_port: 9200
name: SSH Failed Login
type: frequency
index: wazuh-alerts-3.x-*
num_events: 2
timeframe:
    hours: 1
filter:
- term:
    rule.id: "5710"
alert:
    - hivealerter
hive_connection:
    hive_host: http://<thehive_ip>
    hive_port: <thehive_port>
    hive_apikey: <thehive_api_key>
    hive_proxies:
        http: ''
        https: ''

hive_alert_config:
    title: "HiveAlert"
    type: 'external'
    source: 'elastalert'
    description: '{rule[name]}'
    severity: 2
    tags: ['{rule[name]}', '{match[agent][ip]}', '{match[predecoder][program_name]}']
    tlp: 3
    status: 'New'
    follow: True

hive_observable_data_mapping:
    - ip: "{match[src_ip]}"

Thank you very much for your advice.

@mihino89  Can you double check what version you are running? that error looks like what you'd get if you tried a version that didn't contain the hive alerter.

Hi @Qmando  I am running elastalert of version 0.1.28.

How can this commit affect 0.1.28 which is from 2.5 years ago? Try the latest version.


restart existing pods in service mesh for changes in meshConfig.defaultConfig

Feature request:
This issue was raised when working on issue#32602,  but it's different from issue#32602, there is no Istio re-install after changes in meshConfig.defaultConfig.
After some investigation, I found that these 2 issues focus on different parts. Issue#32602 need to restart all pods already exist in service mesh if the Istio is re-installed, and this issue need to restart all pods in service mesh if the meshConfig.defaultConfig changes.
[ ] Docs
[ ] Installation
[x] Networking
[ ] Performance and Scalability
[ ] Extensions and Telemetry
[ ] Security
[ ] Test and Release
[x] User Experience
[ ] Developer Infrastructure
Additional context

I do not think we want to be in the business of restarting users pods, its a highly risky and dangerous task.

+1. It is not Istio problem.

Hi @howardjohn & @ramaraochavali,  yes, it's not a good idea to restart users pods, so shall we just analyze them via analyzer?


Need a way to provide an offset

I want to display the map on the background with a floating panel on top. The floating panel will contain some extra contact details and will be displayed to the right of the screen.
In short, I want my marker to be centered but with an offset on de x-axis.
Is there anything available to do this? I’ve been using fitBounds and fitBoundsPadding but this doesn't seem to accomplish what I want.
This post on StackOverflow by somebody else describes exactly what I want to do.

the answer to the SO post is the answer to your question. AGM doesn't provide additinal facilities, beyond what google maps does. So you can do this calculation, and set maps center to a function of the marker's coords


Is it possible to deploy new model without rebuilding dlib.

If I modify the network structure,mmod for example, I must rebuild the library.
Is it possible to do that without rebuilding, like caffe(.prototxt and .caffemodel) or darknet(.cfg ang .weights) or tensorflow(.pb only).
Thanks a lot!

No

On Nov 25, 2019, at 2:12 AM, newforrestgump001<EMAIL_ADDRESS>wrote:

If I modify the network structure,mmod for example, I must rebuild the library.
Is it possible to do that without rebuilding, like caffe(.prototxt and .caffemodel) or darknet(.cfg ang .weights) or tensorflow(.pb only).
Thanks a lot!
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

@davisking I got it, thanks for your information.


Hello

i want help me
how i can use some sound file in raw folder
show in list and play
thanks

Hi, @ahmid9!
Thank you for using our library.
Please specify your question if it is about PlayWidget library. If your interest is just about using resources in raw folder - there is a lot of tutorials in Network. For example - official tutorial you can find here: https://developer.android.com/guide/topics/resources/providing-resources.html
With best regards, Cleveroad team.

like this not help me
i'm now in begnner

Hi, @ahmid9!
We are sorry, but we can't help you with features not related to our library. Use official developer tutorial: https://developer.android.com/guide/index.html. It contains all general information about  android applications developing.
With best regards, Cleveroad team.


Ignoring a json array

Hello,
We are using this as middleware for a sinatra application. On some apis we pass a json object { "a": "value"} on other apis we pass [{"a":"value1"}, {"a":"value2"}]
In the json object syntax everything is fine, in the case of a json array being passed rack expects to be able to call merge.
  no implicit conversion of Array into Hash
     # /Users/sarge/.rvm/gems/ruby-2.0.0-p247@shift72/gems/rack-1.6.0/lib/rack/request.rb:230:in `merge'
     # /Users/sarge/.rvm/gems/ruby-2.0.0-p247@shift72/gems/rack-1.6.0/lib/rack/request.rb:230:in `params'
     # /Users/sarge/.rvm/gems/ruby-2.0.0-p247@shift72/gems/sinatra-1.4.5/lib/sinatra/base.rb:893:in `call!'
     # /Users/sarge/.rvm/gems/ruby-2.0.0-p247@shift72/gems/sinatra-1.4.5/lib/sinatra/base.rb:886:in `call'
     # /Users/sarge/.rvm/gems/ruby-2.0.0-p247@shift72/gems/rack-protection-1.5.3/lib/rack/protection/xss_header.rb:18:in `call'
     # /Users/sarge/.rvm/gems/ruby-2.0.0-p247@shift72/gems/rack-protection-1.5.3/lib/rack/protection/path_traversal.rb:16:in `call'
     # /Users/sarge/.rvm/gems/ruby-2.0.0-p247@shift72/gems/rack-protection-1.5.3/lib/rack/protection/json_csrf.rb:18:in `call'
     # /Users/sarge/.rvm/gems/ruby-2.0.0-p247@shift72/gems/rack-protection-1.5.3/lib/rack/protection/base.rb:49:in `call'
     # /Users/sarge/.rvm/gems/ruby-2.0.0-p247@shift72/gems/rack-protection-1.5.3/lib/rack/protection/base.rb:49:in `call'
     # /Users/sarge/.rvm/gems/ruby-2.0.0-p247@shift72/gems/rack-protection-1.5.3/lib/rack/protection/frame_options.rb:31:in `call'
     # /Users/sarge/.rvm/gems/ruby-2.0.0-p247@shift72/gems/rack-1.6.0/lib/rack/nulllogger.rb:9:in `call'
     # /Users/sarge/.rvm/gems/ruby-2.0.0-p247@shift72/gems/rack-1.6.0/lib/rack/head.rb:13:in `call'
     # /Users/sarge/.rvm/gems/ruby-2.0.0-p247@shift72/gems/rack-1.6.0/lib/rack/methodoverride.rb:22:in `call'
     # /Users/sarge/.rvm/gems/ruby-2.0.0-p247@shift72/gems/sinatra-1.4.5/lib/sinatra/base.rb:180:in `call'
     # /Users/sarge/.rvm/gems/ruby-2.0.0-p247@shift72/gems/sinatra-1.4.5/lib/sinatra/base.rb:2014:in `call'
     # /Users/sarge/.rvm/gems/ruby-2.0.0-p247@shift72/gems/sinatra-1.4.5/lib/sinatra/base.rb:1478:in `block in call'
     # /Users/sarge/.rvm/gems/ruby-2.0.0-p247@shift72/gems/sinatra-1.4.5/lib/sinatra/base.rb:1788:in `synchronize'
     # /Users/sarge/.rvm/gems/ruby-2.0.0-p247@shift72/gems/sinatra-1.4.5/lib/sinatra/base.rb:1478:in `call'
     # ./models/rack_parser.rb:35:in `call'
     # /Users/sarge/.rvm/gems/ruby-2.0.0-p247@shift72/gems/rack-1.6.0/lib/rack/builder.rb:153:in `call'
     # /Users/sarge/.rvm/gems/ruby-2.0.0-p247@shift72/gems/rack-test-0.6.3/lib/rack/mock_session.rb:31:in `request'
     # /Users/sarge/.rvm/gems/ruby-2.0.0-p247@shift72/gems/rack-test-0.6.3/lib/rack/test.rb:244:in `process_request'
     # /Users/sarge/.rvm/gems/ruby-2.0.0-p247@shift72/gems/rack-test-0.6.3/lib/rack/test.rb:76:in `put'

What are your thoughts on only setting the FORM_HASH and FORM_INPUT if the parsed value is a hash, the expectation is that in this case the params is not the pattern to use when dealing with an array of items being PUT, and would need to be dealt with by accessing the body directly.
def call(env)
      type   = Rack::Request.new(env).media_type
      parser = parsers.detect { |content_type, _| type.match(content_type) } if type
      return @app.call(env) unless parser
      body = env[POST_BODY].read ; env[POST_BODY].rewind
      return @app.call(env) unless body && !body.empty?
      begin
        parsed = parser.last.call body

        # suggested update
        env.update FORM_HASH => parsed, FORM_INPUT => env[POST_BODY] if parsed.is_a?(Hash)

      rescue StandardError => e
        warn! e, type
        handler   = handlers.detect { |content_type, _|  type.match(content_type) }
        handler ||= ['default', ERROR_HANDLER]
        return handler.last.call(e, type)
      end
      @app.call env
    end

:+1:
furthermore event if the JSON data are Array I would like to get the parsed result, e.g.
 RACK_RESULT = 'rack.parser.result'
 # ...
 if parsed.kind_of(Hash)
   env.update FORM_HASH => parsed, FORM_INPUT => env[POST_BODY], RACK_RESULT => parsed
 else
   env[RACK_RESULT] = parser
 end

Good suggestion, at the moment I am double handling this.

closing this since #14 is merged and handles this issue.