Plugin ordering

This commit adds support for overriding the plugin hook run order, by explicitly registering a hook and specifying a priority between -100 (highest priority) and 100 (lowest priority) with the default priority being 0 (zero).
Additionally it improves the bin/haraka -o output by showing the plugin priorities and now only shows hooks with plugins actually registered on them:
Hook: connect ------------------------------------------------------------------
Plugin                                Method                                Prio
--------------------------------------------------------------------------------
access                                rdns_access                              0
access                                any                                      0
dnsbl                                 connect_first                            0
max_unrecognized_commands             hook_connect                             0

If I then modify the dnsbl plugin to specify:
plugin.register_hook('connect',  'connect_first', -100);

The output then becomes:
Hook: connect ------------------------------------------------------------------
Plugin                                Method                                Prio
--------------------------------------------------------------------------------
dnsbl                                 connect_first                         -100
access                                rdns_access                              0
access                                any                                      0
max_unrecognized_commands             hook_connect                             0

Additionally - I found a small speed improvement in doing this, because previously plugins.run_hooks() went through the entire plugin list and checked each plugin in turn to see if it was to run on the current hook, this commit does this via an Object to do a keyed lookup instead and therefore saves quite a number of Array loops on a system with a lot of plugins (like mine!).

My first reaction is that this makes a complicated aspect of Haraka (plugin / hook ordering) more complicated. I lean towards ideas like adding "next(!CONT) not allowed here" hooks that [mostly] remove the ordering contention. I presume that's insufficient for your purposes.

My second reaction is that this makes me want to replace a bunch of hook_[hook_name] function names with what_it_does names, so that haraka -o output is more meaningful. IE, instead of this:

lookup_rdns
----------------------
process_title
connect.p0f
connect.fcrdns
data.uribl
karma
limit                            incr_concurrency

we'd have this:
lookup_rdns
----------------------
process_title                   increment
connect.p0f                     get_tcp_fingerprint
connect.fcrdns                  check_fcrdns_match
data.uribl                      check_rdns_hostname
karma                           get_reputation
limit                           incr_concurrency

Putting the plugin hooks into an object at register() to check at run time instead of iterating over an array listing every plugin seems an obvious win.

Why limit the range of priority numbers? Wouldn't a 'lower numbers have higher priority" rule would be sufficient? I'm expecting that most Haraka installs will never use priority values until they run into an issue, and then they're only going to change the priority of a plugin they want to run sooner/later, and leave everything else at zero. Removing the limit avoids a "sorry, you have to modify plugin Y and increase its priority because that plugin author decided his plugin needed to run sooner than plugin Z and you disagree."

My one other thought, and this might be worth no more than the electrons it's printed on, is that I wish that the plugin ordering was maintained somewhere besides in the plugin hooks. I'm thinking of the future when plugins can be npm packages, and the interface for enabling/disabling them is a web page, and the output of haraka -o too is a web page. It'd be nice if there was a way to influence the plugin / hook ordering in that interface with little up and down arrows or a drag-n-drop list.

My first reaction is that this makes a complicated aspect of Haraka (plugin / hook ordering) more complicated. I lean towards ideas like adding "next(!CONT) not allowed here" hooks that [mostly] remove the ordering contention. I presume that's insufficient for your purposes.

I considered this - I was going to create a runtime flag that basically marked a plugin as 'always run' e.g. any plugin marked with the flag would always be run regardless of the earlier plugins return value.  I decided against it as it was much more complicated to code and still wouldn't fix the run order issue where you want one hook in your plugin to run before everything else and another hook to run after everything else.
I'm not keen on adding a boatload of hook_pre and hook_post hooks either - it's going to make things a lot more complicated as is making these hooks only allow specific return codes.
Limiting return types for specific hooks also isn't any less complicated IMO than this.   We'll have people turning up on IRC asking why when they return next(DENY) on them that it doesn't do as they intend (and it's more complicated to code).

My second reaction is that this makes me want to replace a bunch of hook_[hook_name] function names with what_it_does names

And that would make Haraka better how?  helo.checks already does this, it's not really necessary elsewhere as most hooks do one thing and one thing well; adding method names to dnsbl would be pretty worthless, same with spamassassin or messagesniffer or data.uribl.   It's just needless lines added to the code for no gain.

Putting the plugin hooks into an object at register() to check at run time instead of iterating over an array listing every plugin seems an obvious win.

nod - it's a reasonable win.  Even if a hook doesn't have any hooks registered it would still be iterating over the whole list, so it's definitely better.

Why limit the range of priority numbers? Wouldn't a 'lower numbers have higher priority" rule would be sufficient? I'm expecting that most Haraka installs will never use priority values until they run into an issue, and then they're only going to change the priority of a plugin they want to run sooner/later, and leave everything else at zero. Removing the limit avoids a "sorry, you have to modify plugin Y and increase its priority because that plugin author decided his plugin needed to run sooner than plugin Z and you disagree."

Because there's always going to be an upper/lower limit e.g. it's an integer after all.   I decided it was easier to limit this to +-100 for simplicity.  I figured that we would allow the core plugins to go +-10 or +-50 or whatever, so it's still simple to make local plugins take precedence.   In reality I don't think this will be much of an issue given that this is going to be pretty rare to need to specify a priority.

is that I wish that the plugin ordering was maintained somewhere other than the plugin register() call. I'm thinking of the future when plugins can be npm packages, and the interface for enabling/disabling them is a web page, and the output of haraka -o too is a web page. It'd be nice if there was a way to influence the plugin / hook ordering in that interface with little up and down arrows or a drag-n-drop list.

Wouldn't you do this by modifying config/plugins accordingly?  I'm not sure that having a priority for each plugin is the way to go for something like that.
Maybe making config/plugins a directory e.g. config/plugins.d and then dropping the files in lexical order and reading the directory instead as a method of loading plugins instead might be an idea or allowing the priority to be expressed in config/plugins somehow might be another way (or making config/plugins able to be read as config/plugins.json etc. etc. etc.

this makes me want to replace a bunch of hook_[hook_name] function names with what_it_does names

that would make Haraka better how?

I showed you how in the output above. Quite a few Haraka users don't even know where their plugin files are installed, yet you continue to assume that everyone that uses haraka wants to go read the source to determine what a plugin does. For myself and many others, this output:
data_post
----------------------
data.uribl                      check_from_header
data.uribl                      check_reply_to_header
data.uribl                      check_message_id_header
data.uribl                      check_body_urls

is much more informative and useful than this:
data_post
----------------------
data.uribl

It'd be nice if there was a way to influence the plugin / hook ordering

Wouldn't you do this by modifying config/plugins accordingly?

For plugins ordering sure, but that only goes so far with hook ordering.

I'm not sure that having a priority for each plugin is the way to go for something like that.

I misspoke:
- I wish that the plugin ordering was maintained somewhere other than the plugin register() call.
+ I wish that the hook ordering was maintained somewhere other than the plugin register() call.

I was thinking of hook priority. With a change like this PR, one needs to edit a plugins source file to change a hooks priority. If priority was specified in config/hook_priority, with simple file format for overriding the default priority of 0, it would be clearly within the control of the administrator.

I showed you how in the output above. Quite a few Haraka users don't even know where their plugin files are installed, yet you continue to assume that everyone that uses haraka wants to go read the source to determine what a plugin does. For myself and many others, this output:
data_post
data.uribl                      check_from_header
data.uribl                      check_reply_to_header
data.uribl                      check_message_id_header
data.uribl                      check_body_urls
is much more informative and useful than this:

data_post
data.uribl

So you'd modify data.uribl to run 4 different methods just to make this output more expressive for people that can't be arsed to read the documentation to know that data.uribl "Parses URIs from a message an looks them up against blacklist sources".   You might as well add a long description to the plugin and show that instead...

It'd be nice if there was a way to influence the plugin / hook ordering

Wouldn't you do this by modifying config/plugins accordingly?
For plugins ordering sure, but that only goes so far with hook ordering.

I'm not sure that having a priority for each plugin is the way to go for something like that.
I misspoke:

I wish that the hook ordering was maintained somewhere other than the plugin register() call.
I was thinking of hook priority. With a change like this PR, one needs to edit a plugins source file to change a hooks priority. If priority was specified in config/hook_priority, with simple file format for overriding the default priority of 0, it would be clearly within the control of the administrator.

Huh?  you can't change the priority of a hook - it's run based on the input of SMTP commands.  You can only influence the ordering of a plugin inside a given hook.
If you want to be able to express hook -> plugin -> priority in config/priorities or equivalent - you're welcome to add it in a future PR once this is merged.

Just added a display of the plugin timeout to the -o option:
Hook: connect ------------------------------------------------------------------
Plugin                              Method                              Prio T/O
--------------------------------------------------------------------------------
access                              rdns_access                            0  30
access                              any                                    0  30
dnsbl                               connect_first                          0  30
max_unrecognized_commands           hook_connect                           0  30

@baudehlo: I'd appreciate a yay/nay or any comments when your visitors have gone in a couple of weeks.  This fixes all of the issues I've had with plugin ordering where I've had to split a single logical plugin into multiple pieces so that the hooks are run in the correct order.

Just added a display of the plugin timeout

Nice. :+1:

So you'd modify data.uribl to run 4 different methods just to make this output more expressive

The URIBL data_post hook is already running 4 different functions wrapped in a bigger one. This would be a reduction in function calls.

So you'd modify data.uribl to run 4 different methods just to make this output more expressive
The URIBL data_post hook is already running 4 different functions wrapped in a bigger one. This would be a reduction in function calls.

I don't really care about the function calls; I'm merely stating that data.uribl (and most other plugins) have a single purpose:  in data.uribl's case it is to look up any URIs in URI DNS blacklists, having it show each separate function with a pretty name, just for the sake of it isn't particularly useful, interesting or helpful.   A regular user of Haraka probably isn't ever going to run -o unless they have an issue and having data.uribl being shown 4 times instead of once in the output is just going to add confusion.   Additionally, you'd then have to potentially set 4 different priorities instead of one if you wanted to move things around - it simply doesn't make sense for data.uribl that has one task to do.


Rpak Pilot Model Importing

I uh, messed up the first one. Here is a new pr. Still needs much cleaning up.

@GeckoEidechse if you have the time. Can you re-review this? If you remember any of the issues/problems you had with it.
I remember

Set relative link instead of direct link
Add screenshots (again)
The random quote of a video

Just so I don't forget, the main guide was made by Harb.


Jumping to split panes & tabs through numeric mappings

[Y] I am on the latest Hyper.app version
[Y] I have searched the issues of this repo and believe that this is not a duplicate

Feature Request

I've traditionally used iTerm on Mac, and now I'm switching to Hyper. I am aware of the current shortcut keys to switch panes/tabs in Hyper, but there's a feature that I believe would make work much faster for Hyper users.
If you look at the screenshot, you will see designated shortcuts on every split pane as well as every tab with a number - so Cmd+2 takes you straight to tab 2, and Alt+3 takes you straight to pane 3.
This makes it much easier and faster to work than the current scheme, especially when you have a large number of tabs and split panes open. The numbers save you the effort of thinking how many times to press the cycle button. It enables the power of multiple split panes, especially if one is using a large screen.
So, I can reach any tab and any split pane in that tab with O(1) taps of my keyboard (max 4, if I want to switch to a different tab and different split pane, usually 2)
On the other hand, with the current shortcut I need to go cycling around and pressing Ctrl+Alt+Tab repeatedly until I reach my pane/tab. This means that if I have X tabs open and Y panes open, in the worst case I need to press a shortcut of 3 buttons (X-1)+(Y/2) times. And I often have 7-8 tabs with around 5 split panes on every tab. This is especially more cumbersome if I have split the panes in an unnatural order - they don't cycle in a clockwise or anticlockwise way.
I'm sure many users already love Hyper, and would love it even more if it gets something like what is there in the screenshot - life is too short to spend cycling through tabs and panes.

Really more efficient than using arrow like suggested here https://github.com/zeit/hyper/issues/828
(and complementary).
I'll will suggest to show number of tab/pane only when CMD or ALT key is down to save some space

Good point. Would suggest that the hiding is optional. In most use cases, the number that is shown is far easier to remember that the title of the tab.
The name of the process running is typically less informative, but whereas I can easily remember what I have running in each tab number.
I am unfamiliar with the structure of Hyper, but I think this is essential functionality that would make life faster and easier for every user of Hyper. If I was a product manager for Hyper, I would ensure this comes out-of-the-box with Hyper.
As a user, I am happy with whatever is decided. :) Looking forward to seeing this implemented, so that I can deploy Hyper across my organisation with no second-thoughts.

For tab, number assignation is obvious. The left to the right and reassign a tab is deleted (and move by plugins, but out of our scope).
But for pane, it could be tricky (but don't really matter). Ordered by creation time and reassign when a pane is deleted seems the most logical and easy to remember (not change with new pane). But iTerm has chosen to assign them by TermGroup position/split

(Notice: iTerm keep the number 9 for the last (down-right) term)

If I understand it correctly, it should work if we sort panes in lex order (on a vector) with the key for each pane being the vector of creation times of its ancestral chain, starting from the root pane upto itself.

Hahah @chabou My rework of  the structure in #945 might help!

Made with a plugin : https://github.com/chabou/hyper-pane
Feedback warmly welcome

I want this feature. It is easily see which panel is it.


NUTCH-2399: Support multivalue fields.

Currently, if there is a NutchField with multiple values, only the first value is indexed (because this is what doc.getFieldValue returns). This fix checks if the NutchField has multiple values, they are added as an array (multivalue) field.

Hi @pipldev thank you for the patch... can you please open a JIRA issue over at https://issues.apache.org/jira/projects/NUTCH
Additionally, this change would need to be implemented for all indexers not just the elasticsearch indexer.

indexer-solr already indexes all values. That's why it's mandatory to declare such fields (eg. anchors) in the schema.xml as multiValued="true". But yes, it's a good idea to check the remaining indexers as well.

Opened a JIRA issue: https://issues.apache.org/jira/browse/NUTCH-2399.
I don't think I can test the other indexers, as I am not familiar with them and don't have them installed, but seeing as indexer-solr already implements it, it seems like this patch can go forward independently.

Also submitted a patch for indexer-elastic-rest https://github.com/apache/nutch/pull/203

This PR contains also the commits for #219 (NUTCH 2415) going to extract the patch to indexer-elastic into a separated PR and close this one.

Oops, sorry for that. Not sure how it happened.
Only the a614dd4 commit should be here.

@pipldev Don't worry, I'll just open a new PR with your changes. Thanks for the contribution, this was a good catch!


Need better copy for transactional emails

We need better copy for the emails that go out which are :
1. Password reset notifications
Subject : Reset your password for Green Up Vermont

Hello,

Follow this link to reset your GreenUpVermont password for your<EMAIL_ADDRESS>account.

https://greenupvermont-de02b.firebaseapp.com/__/auth/action?mode=resetPassword&oobCode=HRu0b_g9N0bLnjeyquwj-rhaQwwZfEtEIjBXeVyCcMsAAAFhtwRoSQ&apiKey=AIzaSyAjwSCpOvLPgYcFr26V3gmfwJlGb-VtWAs

If you didn’t ask to reset your password, you can ignore this email.

Thanks,

Your Green Up Vermont team

2. Invitations to join a team
Subject :  Welcome to Green Up Vermont!

Hey John Need! You've been invited to participate in Green Up Day.

What needs to be added here?  These messages seem direct and clear to me.


Change homepage url from http to https

I did a quick search and it looks like there might be some other places to make this change.  Wouldn't hurt to do them all at once if you have the time.
https://github.com/expressjs/express/search?utf8=✓&q=http%3A%2F%2Fexpressjs.com%2F&type=

@wesleytodd changes done as per review comments.

Yep, thanks!  All that it needs is for @dougwilson to merge.

+1 to merge

The links are still to http since we don't have control over the Cloudflare account which is backing the SSL. In other words, the https version is not under our control.

That is unfortunate. What do we need to do to get control of that?
On Fri, Nov 17, 2017 at 8:24 AM Douglas Wilson<EMAIL_ADDRESS>wrote:

The links are still to http since we don't have control over the
Cloudflare account which is backing the SSL. In other words, the https
version is not under our control.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/expressjs/express/pull/3465#issuecomment-345290641,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AA-uwM-jCr4KFX3ifGiGFOi2M1UPu0Zpks5s3bM2gaJpZM4QN79Y
.


Replaced many print() calls with logger.debug or info. Plus avoid one bad exception.

In views_admin.py, we were not passing the request to
elections_import_from_master_server() which threw an exception since
logging now works.

👍


removeItem

I have found this useful to me.
If anyone want to know how to remove an item from the cart:
function removeItem(item_id) { 	var item_index = Cart.indexOfItem(item_id); 	Cart.items.splice(item_index); 	Cart.save(); 	// update template }

Hello @xeux , thank you for your message. It is actually a very good point. I've been too much of a minimalist on this one. The way I use it is that generally the quantity is 1 or a little bit more so I just do addItem with "-1" until it is removed.
I will definitely add this functionality to remove an item completely, regardless of the quantity. I'll post a message back here when this is done.
Thank you.
mig


Hide nav profile with CSS instead - fix #40

This is a fix for #40.
This changes the method for hiding the profile in the nav to using CSS.

Awesome! I could swear I tried that, must have been a PEBCAK. This is much better than the previous hack :)

Withiut the direct child selector the nav disappears, so maybe you tried this way and didn't want to play with all these obfuscated class names? :smile:


Khmer text become multiple lines and lose some character when put \n between sentence on web.

Is there an existing issue for this?

[X] I have searched the existing issues
[X] I have read the guide to filing a bug

Steps to reproduce

Create new project
Add Khmer sentence that include \n between sentence to Textfield.

Expected results
The sentence should show 2 lines.
Actual results
The sentence show many lines and lost some character.
Code sample
Code sample
import 'package:flutter/material.dart';

void main() => runApp(MyApp());

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      debugShowCheckedModeBanner: false,
      theme: ThemeData(
        primarySwatch: Colors.blue,
      ),
      home: const MyHomePage(title: 'Flutter Demo Home Page'),
    );
  }
}

class MyHomePage extends StatefulWidget {
  final String title;

  const MyHomePage({
    Key? key,
    required this.title,
  }) : super(key: key);

  @override
  State<MyHomePage> createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {
  int _counter = 0;

  void _incrementCounter() {
    setState(() {
      _counter++;
    });
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: Text(widget.title),
      ),
      body: Center(
        child: Column(
          mainAxisAlignment: MainAxisAlignment.center,
          children: [
            const Text(
              'ឃ្លាអក្សរខ្មែរដែលបានសរសេរនេះគឺដើម្បីសាកល្បងទៅលើការចុះបន្ទាត់ដែលធ្វើឲ្យអក្សរធ្លាក់ចុះក្រោមនឹងមានបាត់អក្សរទៀតផង។\nដូច្នេះសូមធ្វើការត្រួតពិនិត្យទាំងអស់គ្នា',
            ),
            Text(
              '$_counter',
              style: Theme.of(context).textTheme.headlineMedium,
            ),
          ],
        ),
      ),
      floatingActionButton: FloatingActionButton(
        onPressed: _incrementCounter,
        tooltip: 'Increment',
        child: const Icon(Icons.add),
      ),
    );
  }
}

Screenshots or Video

Screenshots

Logs
Logs
[Paste your logs here]

Flutter Doctor output
Doctor output
[✓] Flutter (Channel stable, 3.10.1, on macOS 13.3.1 22E772610a darwin-x64, locale en-KH)
    • Flutter version 3.10.1 on channel stable at /Users/oone/development/flutter3
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision d3d8effc68 (6 days ago), 2023-05-16 17:59:05 -0700
    • Engine revision b4fb11214d
    • Dart version 3.0.1
    • DevTools version 2.23.1

[✓] Android toolchain - develop for Android devices (Android SDK version 33.0.1)
    • Android SDK at /Users/oone/Library/Android/sdk
    • Platform android-33, build-tools 33.0.1
    • Java binary at: /Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.15+0-b2043.56-8887301)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 14.2)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 14C18
    • CocoaPods version 1.12.0

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2022.1)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.15+0-b2043.56-8887301)

[✓] VS Code (version 1.78.2)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.64.0

[✓] Connected device (2 available)
    • macOS (desktop) • macos  • darwin-x64     • macOS 13.3.1 22E772610a darwin-x64
    • Chrome (web)    • chrome • web-javascript • Google Chrome 113.0.5672.126

[✓] Network resources
    • All expected network resources are available.

• No issues found!

Thanks for filing this issue @HongMenghak. I am able to reproduce this issue on the latest master and stable channels. This issue is only specific to the web and is reproducible using both html and canvaskit renderers.

Mobile
Desktop
Web

✅
✅
❌

✅ - Not reproducible
❌ - Reproducible
  flutter doctor -v
[✓] Flutter (Channel master, 3.11.0-2.0.pre.55, on macOS 13.3 22E252 darwin-arm64, locale en-NG)
    • Flutter version 3.11.0-2.0.pre.55 on channel master at /Users/dammya/fvm/versions/master
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 835b892d7f (2 weeks ago), 2023-05-07 01:30:22 +0530
    • Engine revision 23f730efbf
    • Dart version 3.1.0 (build 3.1.0-83.0.dev)
    • DevTools version 2.23.1

[✓] Android toolchain - develop for Android devices (Android SDK version 32.1.0-rc1)
    • Android SDK at /Users/dammya/Library/Android/sdk
    • Platform android-33, build-tools 32.1.0-rc1
    • ANDROID_HOME = /Users/dammya/Library/Android/sdk
    • Java binary at: /Applications/Android Studio.app/Contents/jre/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.12+0-b1504.28-7817840)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 14.3)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 14E222b
    • CocoaPods version 1.11.3

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2021.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.12+0-b1504.28-7817840)

[✓] IntelliJ IDEA Community Edition (version 2022.3.2)
    • IntelliJ at /Applications/IntelliJ IDEA CE.app
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart

[✓] VS Code (version 1.78.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.50.0

[✓] VS Code (version 1.78.0)
    • VS Code at /Users/dammya/.Trash/Visual Studio Code 20.07.05.app/Contents
    • Flutter extension version 3.50.0

[✓] Connected device (5 available)
    • sdk gphone arm64 (mobile)  • emulator-5554                            • android-arm64  • Android 11 (API 30) (emulator)
    • Damilola’s iPhone (mobile) • 71c843fa5f3dec767999f36224653c0faa78660e • ios            • iOS 16.4 20E247
    • iPhone 14 Pro Max (mobile) • D7F3302D-D66F-4B89-B8CF-DBEE1CDB5449     • ios            • com.apple.CoreSimulator.SimRuntime.iOS-16-4 (simulator)
    • macOS (desktop)            • macos                                    • darwin-arm64   • macOS 13.3 22E252 darwin-arm64
    • Chrome (web)               • chrome                                   • web-javascript • Google Chrome 113.0.5672.126

[✓] Network resources
    • All expected network resources are available.

• No issues found!

[✓] Flutter (Channel stable, 3.10.1, on macOS 13.3 22E252 darwin-arm64, locale
    en-NG)
    • Flutter version 3.10.1 on channel stable at /Users/dammya/sdks/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision d3d8effc68 (5 days ago), 2023-05-16 17:59:05 -0700
    • Engine revision b4fb11214d
    • Dart version 3.0.1
    • DevTools version 2.23.1

[✓] Android toolchain - develop for Android devices (Android SDK version
    32.1.0-rc1)
    • Android SDK at /Users/dammya/Library/Android/sdk
    • Platform android-33, build-tools 32.1.0-rc1
    • ANDROID_HOME = /Users/dammya/Library/Android/sdk
    • Java binary at: /Applications/Android
      Studio.app/Contents/jre/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build
      11.0.12+0-b1504.28-7817840)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 14.3)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 14E222b
    • CocoaPods version 1.11.3

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2021.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build
      11.0.12+0-b1504.28-7817840)

[✓] IntelliJ IDEA Community Edition (version 2022.3.2)
    • IntelliJ at /Applications/IntelliJ IDEA CE.app
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart

[✓] VS Code (version 1.78.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.50.0

[✓] VS Code (version 1.78.0)
    • VS Code at /Users/dammya/.Trash/Visual Studio Code 20.07.05.app/Contents
    • Flutter extension version 3.50.0

[✓] Connected device (5 available)
    • sdk gphone arm64 (mobile)  • emulator-5554                            •
      android-arm64  • Android 11 (API 30) (emulator)
    • Damilola’s iPhone (mobile) • 71c843fa5f3dec767999f36224653c0faa78660e •
      ios            • iOS 16.4 20E247
    • iPhone 14 Pro Max (mobile) • D7F3302D-D66F-4B89-B8CF-DBEE1CDB5449     •
      ios            • com.apple.CoreSimulator.SimRuntime.iOS-16-4 (simulator)
    • macOS (desktop)            • macos                                    •
      darwin-arm64   • macOS 13.3 22E252 darwin-arm64
    • Chrome (web)               • chrome                                   •
      web-javascript • Google Chrome 113.0.5672.126

[✓] Network resources
    • All expected network resources are available.

• No issues found!

Here's a smaller repro:
import 'package:flutter/material.dart';

void main() => runApp(MyApp());

const text = 'លធ្វើឲ្យ\n';

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return Directionality(
      textDirection: TextDirection.ltr,
      child: Text(text),
    );
  }
}


bug in threads.py

Summary
threads.py doesnt import required class
Reproduction Steps
for x in guildMembers:
hasViewPermissions = ctx.channel.permissions_for(x).view_channel
Minimal Reproducible Code
No response
Expected Results
returns true/false as expected
Actual Results
File "C:\Users*****\AppData\Roaming\Python\Python39\site-packages\disnake\threads.py", line 483, in permissions_for
denied = Permissions.all_channel()
NameError: name 'Permissions' is not defined
Intents
intents = disnake.Intents.default() intents.message_content = True intents.members = True intents.guilds = True
System Information
- Python v3.9.0-final
- disnake v2.9.0-final
    - disnake importlib.metadata: v2.9.0
- aiohttp v3.8.6
- system info: Windows 10 10.0.19041 AMD64

Checklist

[x] I have searched the open issues for duplicates.
[X] I have shown the entire traceback, if possible.
[X] I have removed my token from display, if visible.

Additional Context
workaround add "from .permissions import Permissions" to line 16 on threads.py

Thanks for reporting this issue! This will be fixed in v2.9.1 over the next few days.


emphasized selected action-button focus ring is not visible.

Description
Focus ring for emphasized action button is barely visible and does not conform to spectrum
Steps to reproduce

Go to http://opensource.adobe.com/spectrum-css/
Click on Action Button
Tab to the emphasized selected action buttons

Expected behavior
Should meet spectrum
Screenshots
Spectrum screenshot

Observed Behaviour - the first edit button has focus

Environment

Component version<EMAIL_ADDRESS>Browser(s) and OS(s): Edge 90.0.818.42 on MacOs 11.2.3

Additional context

Duplicated by #1177 with more information, closing this one!


request parameter types - enum, list, range

@RequestParameter(name=foo, type=string, allowedValues=foo,bar,baz) - for enum (just limiting scope, not really enum type)
@RequestParameter(name=foo, type=string, acceptMultiple=true) - for list (would return array of values)
cc @f3l1x

Problem with list is that it is represented in different ways, depends on a in parameter.
In query - foo[]=foo&foo[]=bar&foo[]=baz
In path - ???
In cookie - json encoded?
In header - probably multiple headers with same name?
(Note to self - check current implementation for headers - only one for string, multiple for string[])

In path: coma separated? /hello/world/foo,bar,baz

I think there is no practical reason to support enum in path, so we don't need to worry about implementation.

sure :) that was my second option to tell you: do not support enum in path

Note to self - arrays:

https://swagger.io/docs/specification/describing-parameters/#query-parameters
https://swagger.io/docs/specification/serialization/

@mabar How is it with the support for the arrays? Not sure if it was planned to implement or not. Looking for string[], int[], don't need enum, but list of ids. List of objects will be nice too :) Cannot inject own validation object.


Images missing in api_docs

A few images are missing at https://www.tensorflow.org/api_docs/python/math_ops/segmentation, e.g. SegmentSum.png and SegmentProd.png are not found

Thanks. We'll look into it.

Images are back on that page and others where the same transcription error happened.


Create raisoni.net

Dear Sir, We are From G.H. Raisoni college Of Engineering Which is a Part Of Raisoni Group Of Institution we request you to please add This file in it. Thank you.

@vksmgr Could you please rename lib/domains/net/raisoni.net to lib/domains/net/raisoni.txt to reflect this repository rules? Thank you!

@vksmgr Pull request merged. Thank you!


Embedding Url

<audio src="example.mp3" id = 'ex'></audio>
<audio src="example1.mp3" id = 'ex1'></audio>

var f = document.getElementById('ex').src
var f1 = document.getElementById('ex1').src

Whenever I do:
audio
    .fetchAudio("f", "f1")

it says it cant seem to find the file, even though Im calling the URl.

You are not  using the variables f nor f1. You did put these names into quotes, so the function received just these names, not they values. You have to do this:
audio.fetchAudio(f, f1)

I think that was already resolved, if you have any more questions let us know


Coordinates of intersection

How do we recover the coordinates of the intersection point from the vector?

You can sum each of the arrays and divide by the last component.

Can you show a JS example of this? I don't quite understand what "homogenous 3 vector" means :(

It is a vector [x, y, w] in homogeneous coordinates.  To get the original point of intersection, you need to divide by w, giving [x/w, y/w].
Though I don't think this module is really the best solution for this problem anymore.  Today I'd recommend using big-rat to compute the intersection, and eventually I'd like to switch to something that doesn't require any memory allocation, probably using code generation.

It is a vector [x, y, w] in homogeneous coordinates.

Hmm, but the output stated in the readme is [ [0], [0], [1] ]. Why are there single-element arrays?

The output vectors are non-overlapping increasing sequences, similar to the form used by the robust-* packages.   This allows them to represent higher precision floating point values, (though they can still underflow or overflow in some cases).  To get the nearest float value, you can sum all the components in the array.


stacked_bar

I think it was only added a few days ago: https://github.com/jwilber/roughViz/issues/5#issuecomment-554679129
Example:
https://blockbuilder.org/jwilber/ee35865631cb805057142568fa5fd090

Thank you for bringing the same to my notice. I'll definitely add that soon. Plus I am planning to add more charts to the library (like histogram) very soon.


How to write newlines?

I'd like to use your action to prepend generated changelog text to my CHANGELOG.md file, something like this:
- name: Write to CHANGELOG.md
  uses: DamianReeves/write-file-action@master
  if: ${{ !env.ACT }}
  with:
    path: ./CHANGELOG.md
    contents: ${{ steps.changelog.outputs.changelog }}${{ steps.package.outputs.content }}
    write-mode: overwrite

Except that I want a blank line between the new content and the existing content. I am not familiar enough with the syntax of actions and yaml.. how can add those newlines?
I tried this, but that just literally adds echo and \n\n to my file.
contents: |
  echo ${{ steps.changelog.outputs.changelog }}\n\n${{ steps.package.outputs.content }}

Any update on this request?

Sorry for the delay in response I'll take a look at this today.

@kevinrenskers YAML is interesting. Take a look here https://www.w3schools.io/file/yaml-multiline-strings/.
Have you tried?
contents: |
  echo ${{ steps.changelog.outputs.changelog }}

  ${{ steps.package.outputs.content }}

Yeah, this helps. I got this working.

Awesome closing, this.


Context path or SCRIPT_NAME when deployed in docker, and proxied throgh front-end apache

hello @GrahamDumpleton ,
when we server through mod_wsgi, apache inside docker, and setup a proxy throgh frontend Apache to this docker, every thing works as expected if it is mounted at root on front-end apache
in your article http://blog.dscpl.com.au/2015/06/proxying-to-python-web-application.html , there are ways specified for modifying forwarded headers like scheme, and port.
ProxyPass / http://docker.example.com:8002/

RequestHeader set X-Forwarded-Port 443
RequestHeader set X-Forwarded-Scheme https

but if proxied like below, with custom mount path
ProxyPass /my_api http://docker.example.com:8002/

then url reconstruction inside docker image not working, as SCRIPT_NAME for wsgi environ doesn't preserve frontend's mount path. what is recommonded way to do this?

Use:
ProxyPass /my_api/ http://docker.example.com:8002/my_api/

and then in the image when running mod_wsgi-express use the --mount-point /my_api/ option.
I have use a trailing slash on everything as I can't remember if there is a limitation with leaving it off in this case. Try with and without and see if there are any gotchas that I can't remember.
If you must be able to accept without a trailing slash on the front end, add a RewriteRule which redirects URL without training slash to have trailing slash, which would then match the ProxyPass.


Create testable Application

The old solution of a TestComponent extending ApplicationComponent was
not working as intended. All modules from parent were inherited, even
the NormalApplicationModule which was not intended for tests.
Instead we break out test specific code into a separate testable Application
and ApplicationComponent. This gives us more control over which modules gets
included. The Application extends from a common ApplicationGraph to make sure
we satisfy all dependencies required to run the app.
Since Application can't live in both main and a build type, we had to create
a Application for each build type (debug, release, acceptance). To minimize the
duplication a ApplicationController was created to break out the common code
between the Applications.
Each Application is reponsible to wire up and expose its ApplicationComponent and
Android injection. This introduces some duplication that could possible be improved.

Codecov Report

Merging #46 into develop will increase coverage by 0.23%.
The diff coverage is 0%.

@@              Coverage Diff              @@
##             develop      #46      +/-   ##
=============================================
+ Coverage      49.14%   49.38%   +0.23%
  Complexity       343      343
=============================================
  Files             93       94       +1
  Lines           1465     1458       -7
  Branches          91       91
=============================================
  Hits             720      720
+ Misses           703      696       -7
  Partials          42       42

Impacted Files
Coverage Δ
Complexity Δ

...va/net/erikkarlsson/smashapp/SmashApplication.java
0% <0%> (ø)
0 <0> (?)

...kkarlsson/smashapp/SmashApplicationController.java
0% <0%> (ø)
0 <0> (?)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f1633ef...9ee58ef. Read the comment docs.


We should merge "My Resources" and "Discover"

These two pages perform extremely similar functions but behave very differently. We should merge them into a single "Resources" page with a consistent set of filters, search methods etc.
My Resources could become this single page if there was a check box for "All resources" instead of just "viewable by me" "shared with me" etc.
However the filtering options seem to be better on "Discover".
Can we merge??

I vote against this one.  I want a My Resources page and a Discover page.
My Resources = My stuff
Discover = Other people's stuff + a way for unauthenticated users to interact with the system
I want a place where I can manage all of my content separately from everyone else's.  I don't want to have to filter out all of the cruft all of the time to get to to my content.
In my opinion, this would be taking a big step backward to where we were before.

What do you think about the fact that the filtering/searching methods are quite different on both pages? Is there a way to harmonize them?

It may be worth a look.  But, they are targeted at different users.  The filters on the My Resources page (including the labels that show up when I create and apply them) are for me to organize and narrow my own content (and I should already have some idea of what I've got), whereas the facets on the discover page include more comprehensive facets to help users (who may not know already what they are looking for) browse through the entire content of the catalog. I'm not sure we need the same facets on both pages.

I think that this is an issue where it is "been there, done that".  The boat has sailed.  We have vacillated on this issue enough.  You may recall endless earlier discussions on this.  I would vote that we truncate discussion on this issue and just get the pages working.   We have hydro value added functionality to serve our use cases to work on rather than this redesign.


.stop() method?

It's a very good project, I found it much easier in use then FlipClock.js which I used before (and it also requires jQuery).
But I was surprised when I did not see any sign of a stop() method which would be useful in a big complex projects when you need to stop the counter earlier then the deadline occurs according to some app logic.

Thanks for the feedback. Would this have the effect of pausing the clock, or zeroing it?

Hi! I think the clock should be stopped, then possibly zeroed (but not necessarily) and then completely removed from the parent container.
Right now I'm doing this by calling clearInterval with clock's "countdown" field and then removing clock's elements from the DOM:
if(myClock !== false){
clearInterval(myClock.countdown);
myClock = false;
}
var element = document.getElementById('shabbat-clock-outer');
element.innerHTML = '';

These features is must needed, stop(), pause() and play() are basic ones that are needed in a lot of projects

Hi, thanks for good lib.
Yup. start(), stop(), play(), pause(), init() methods will be much useful
Thanks


High DPI Artwork for Default Theme

Art only, still trying to figure out how to implement it. In the mean time someone else is welcome to try.
I'd also be interested in any opinions on the best way to implement it.

Thanks for this contribution, and sorry that it couldn't be used in the end. The default theme has changed quite a bit since 2014, so sadly those icons are not usable as is.
@reduz is working on HiDPI support right now, so we'll need hires artwork for the current default theme though, if you're still interested ;)


bug on IOS only. Android looks perfect

Project successfully prepared (iOS)
Executing after-prepare hook from /Users/piotrz/Projekty/ITR/itr-medandlife-energetyka-mobile/Med and Life/hooks/after-prepare/nativescript-dev-sass.js
Successfully transferred login.component.html.
Refreshing application...
Successfully synced application org.nativescript.MedandLife on device 76D91E5B-D795-4669-9EB5-573BEDAD6E6A.
1   0x10624fbc8 -[TNSRuntime executeModule:referredBy:]
2   0x105cf1c71 main
3   0x10bbe5d81 start
undefined:7:68: JS ERROR TypeError: undefined is not an object (evaluating 'b.prototype')

any hints?

Missed this previously. Have you tried rebuilding the app? Also can you provide some screenshots of what is different?

Let me know if still any issue lingering here. Thanks 😄


Add AudioEffectCapture

AudioEffectCapture allows access to the microphone and other audio on an audio bus in real-time.
Implementation of the updated proposal at godotengine/godot-proposals#2013
Thanks to @fire and @Faless for their input and help.
Bugsquad edit:
Fixes: https://github.com/godotengine/godot-proposals/issues/2013, https://github.com/godotengine/godot-proposals/issues/399, https://github.com/godotengine/godot/pull/35402

Thanks!

Hi. Great news! Will this be available in 3.2?

Hi. Great news! Will this be available in 3.2?

3.2.4 is still in release candidate stage, so it's up to Remi if new functionality can still be added.
Anyway, I filed pull request #46304

Why doesn't this support signals, and instead require every user to poll?

@paulicka  I cited your feedback here.

@paulicka  I cited your feedback here.


compiler: resolve panic in substituteTypeArgs Function for *ast.IndexListExpr

Fixes: #141
This PR addresses a panic occurring in the substituteTypeArgs function within the types.go file. The panic is caused by the lack of handling for the *ast.IndexListExpr type, resulting in unexpected behavior during type substitution.

I don't think @achille-roussel that we need a test for this because there are other cases in substituteTypeArgs for which tests were not added, so I'll just follow the same approach of not adding tests. But if you want, I can write a test for all the cases defined in the substituteTypeArgs function.

Also to follow up on this:

there are other cases in substituteTypeArgs for which tests were not added

The git history may look like that was the case, but it's missing context about in-person development that happened at the time the project was being bootstrapped before it was made public. In general, we have strong test coverage for the features we support, and we maintain a rigorous testing culture to ensure we can continue to provide a high quality solution over time 👍

Fixed by https://github.com/dispatchrun/coroutine/pull/148


PHP code Syntax is not highlighted if is between a

VSCode Version: Code 1.13.1 (379d2efb5539b09112c793d3d9a413017d736f89, 2017-06-14T18:21:47.485Z)
OS Version: Windows_NT ia32 10.0.15063
Extensions:

Extension
Author
Version

language-stylus
sysoev
1.7.3

Steps to Reproduce:

PHP syntax won't work if is inside a

Duplicate of #28818
cc @roblourens


Not working

on my server the file is not working why is that and please my you fix this ASAP because i have a private SMP and my friends want the mod on the server. If you need more infomation contact me on discord at KiaTEC#1195

this is a client side only mod, you may be thinking of the base Galacticraft mod which can be found here.


incredible thing! amazing!

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.

This problem cannot be reproduced so I'll turn it into "close as not planned"

Other issues are welcome


OAuthCard is not working on Chrome or IE using latest BotChat.js

•	We are trying to change our BOT (Bot Framework V3) authentication mechanism from AuthBoT to Sign-in card based AAD authentication, as per recent recommendation from Microsoft. Authentication mechanism has been configured as per the documentation available.
•	But, we are facing issue with default web chat as well as custom web chat. Sign-in card is not rendering properly on these (for custom web chat we are using direct line key).
•	As per the issues mentioned in GitHub https://github.com/Microsoft/BotBuilder/issues/4632 & https://github.com/Microsoft/BotFramework-WebChat/issues/1001 we have tried with latest BotChat.js file. After that, sign-in card is rendering in Edge browser but not rendering in Chrome or IE. We have tried with BotChat-ES5.js file as well with no luck.
Please let us know if there is any other work around or if we are expecting any further stable build in near future.
Also, for our custom web chat, earlier we modified BotChat.js & css files. But, un-minified version of latest BotChat.js file is not available. Even “npm run build” on “WebChat-BotFramework” produces minified version of BotChat.js file.
Please let us know from where we can get the un-minified version.

We love to help. Can you tell us a little bit more about "not rendering properly"? Can you take a screenshot of what is actually showing up (and expected, if you can get it from Edge)?

Hello William,
Thanks for your reply. Please find attached the screenshots. One with OAuthcard is rendered on Edge and the blank one is from Chrome.

@ArnabHCL can you open F12 developer tools in Chrome to see if there is any errors over there?
I am trying to get a repro but couldn't. I am not able to repro with our MockBot which will connect to GitHub via OAuth, in Chrome.
Do you know which steps I am missing here?

closing due to inactivity


Nios2 qspi unaligned read

Modify nios2 flash interface to allow unaligned flash reads.
This has not been tested as I do not have access to such a system.

Where is the test case modification to show that this works, for those that do have access to a Max10 board?

Test added


Wanting to know workflow of OpenSFM

I am trying to understand, how the opensfm works when i use
Command Line Source according to the documentation
bin/opensfm command source
Example
bin/opensfm extract_metadata image/path
My understanding

Run opensfm(Linux)/opensfm.dat(Windows) with command
opensfm calls opensfm_main.py with command & data_path

There is no args inside opensfm_main.py where are the arguments captured?
This is what is called when opensfm_main python file is called
if __name__ == "__main__":
    commands.command_runner(
        commands.opensfm_commands, create_default_dataset_context, dataset_choices=["opensfm"]
    )

I am finding it hard to know how and where command meta_data is being captured and how is it proceeding further
Looking forward for guidance
Thank You

Hi @santhoshnumberone,
Commands arguments are parsed here : https://github.com/mapillary/OpenSfM/blob/main/opensfm/commands/command_runner.py#L30
Let me know if that helps,
Yann


failing tests

Hello,
Do you have any idea why the checks are failing?
https://paste.archlinux.de/B7l68f/

It would be error of PIL or Pillow.
In my test environment, using Pillow 4.0.0.

We have pillow version 4.2.1 here on arch linux

I tried with pillow 4.2.1, and got same error as above.
With 4.0.0, no error occured.
For piexif test, use pillow 4.0.0.

I'll check how to avoid this error with latest pillow.

@hMatoba thanks!

The problem has been solved in my environment.
Could you try the latest commit of piexif?


allow liccheck to be used as a pre-commit hook

Enabling liccheck as a pre-commit hook can be done by just adding simple yaml file in the root of the repository which declares the hook so users can use it.
See https://github.com/pre-commit/pre-commit/blob/master/.pre-commit-hooks.yaml
I may make a pull request to add if you want, it is very easy.

Sure, please go ahead and submit a PR.

@ochedru I will do this as soon #12 is fixed and included in a release (tag), as this was a requirement for implementing it. I already have the code in my fork but I am waiting for the release to make the PR, it has to be after as pre-commit is always pointing to a release tag.

Please go ahead: I just released 0.3.1 (0.3.0 was good also but had a problem with Travis deploying multiple times to PyPI - fixed with skip_existing).

Hey @ssbarnea, I'm considering using pre-commit and would like to integrate liccheck with it, are you still willing to submit your hook? Thanks.

This would be a great addition and will also increase the adoption of this library as pre-commit is getting really popular :+1:


HWP #260 - Removed fetch requests from frontend, added auth to fetch calls

Description
This PR includes the following proposed change(s):

HWP #260
Removed fetch calls from frontend components and moved them to redux actions
Added the profile duck to deal with profile pages
Including the jwt access token in api requests

Requires
Add 'Requires Attention' label if appropriate.

[ ] Requires an update to the .env file
[ ] Requires adding a file listed in the .gitignore
[ ] Requires an opinion or answer to a question, see comments.
[ ] Requires other attention, see below.

If there are any requirements for the developer to make after this PR is merged please list them here.
Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[x] Refactoring / Documentation
[ ] Version change

if your change is a breaking change, please add breaking change label to this PR
How Has This Been Tested?
Please describe the tests that you ran to verify your changes.
For pull requests that are not just a simple fix, please check that the branch works on your local machine.

[x] Tested by Zach.
[ ] Tested by Brandon.
[ ] Tested by Brady.

Does the change impact or break the Docker build?

[ ] Yes
[x] No

If Yes: Has Docker been updated accordingly?

[ ] Yes
[ ] No

Checklist:

[ ] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[ ] I have made corresponding changes to the documentation
[ ] New and existing unit tests pass locally with my changes
[x] If attention is required by the developer such as updating the .env file, these requirements have been listed.

Looks great, will pull the branch and test in the morning.


updating security policy

This PR fixes

removal of email security contact parameter
removal of phone number parameter

This is already merged.


Switch outstanding 'contains' queries to 'eq' queries

This moves our list queries to eq queries so that they'll be faster.

Much better.


[Actions] Separate android build common part

:memo: TAOS-CI Version: 1.5.20200925. Thank you for submitting PR #553. Please a submit 1commit/1PR (one commit per one PR) policy to get comments quickly from reviewers. Your PR must pass all verificiation processes of cibot before starting a review process from reviewers. If you are new member to join this project, please read manuals in documentation folder and wiki page. In order to monitor a progress status of your PR in more detail, visit http://ci.nnstreamer.ai/.


Link to GetChanges API points to unmaintained reference page on microsoft.com

The link to the GetChanges API (https://msdn.microsoft.com/library/office/dn531433.aspx#bk_ListGetChanges) is listed as no longer maintained.  Although it might be valid, from the content on that page, it is difficult to understand how to interpret the response structure from GetChanges.  In particular, the ChangeType field appears to be an integer, and I cannot find a reference for how to map that integer to a meaningful change type.
I cannot find newer online documentation for the GetChanges API, nor can I find any documentation for the structured returned by GetChanges that is listed under:   "https:///_api/SP.ChangeItem",

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: a6393d95-8c1d-2341-0cbd-282cd7016204
Version Independent ID: b7506441-6d16-1689-91aa-e9f45a2bb9e0
Content: Overview of SharePoint webhooks
Content Source: docs/apis/webhooks/overview-sharepoint-webhooks.md
Product: sharepoint
GitHub Login: @VesaJuvonen
Microsoft Alias: vesaj

Quick google search shows yields the values of hte enumeration:
https://docs.microsoft.com/en-us/previous-versions/office/sharepoint-visio/jj245903(v=office.15)


Update extension-dll-macros.md

The first two parameters of the structure AFX_EXTENSION_MODULE are BOOL and HMODULE. So initialisation with = { NULL, NULL } is inaccurate: first NULL is FALSE (or 0 due to BOOL is defined as int). In fact this expression cannot be transformed to modern C++ equivalent = { nullptr, nullptr }.
struct AFX_EXTENSION_MODULE
{
BOOL bInitialized;
HMODULE hModule;
HMODULE hResource;
CRuntimeClass* pFirstSharedClass;
COleObjectFactory* pFirstSharedFactory;
};

@yury-fedorov : Thanks for your contribution! The author(s) have been notified to review your proposed change.


Verify Traction plugin can run alongside other Aca-Py plugins

As an Aca-Py plugin, Traction should be able to run alongside other plugins (such as the issuer registration plugin for Aries VCR).
In order to assess next steps for standing-up an Aries VCR/OrgBook issuer agency (see this) we need to be reasonably sure other plugins will be able to work alongside Traction.

I've opened 2 draft PR's:
traction - includes the orgbook plugin in the traction build - https://github.com/bcgov/traction/pull/424
aries-vcr issuer - updated to use a traction tenant as orgbook issuer - https://github.com/bcgov/aries-vcr-issuer-controller/pull/74
Both are in draft due to some outstanding issues/questions, but demonstrate that traction can be used as an orgbook issuer

Some items we need to address in traction:

use of the ADMIN_API_KEY - we shouldn't give this to tenants
configuration of webhooks - I currently need to call the multitenant API directly
use of proxy - I didn't use this for the aries-vcr issuer so not sure of the requirements, configuration etc.
how to install plug-ins - pip vs poetry

Use of ADMIN_API_KEY. We deploy traction with a proxy of the admin API that hides the ADMIN_API_KEY, so tenants do not have to set it or know it. They simply need their token. The proxy is a work in progress, it makes some functions (such as multitenancy wallet calls - except token; startup/shutdown, etc) unreachable. All tenants (including the innkeeper - which is a tenant) go through the proxy, never the admin API.
webhooks are added and available for tenants to manage their own. See PR 439. The shapes of the calls and responses are identical to the multitenancy wallet GET / PUT that an admin used to call on behalf of a tenant.
not sure what this is about exactly. if the wallet/agent used by aries VCR is a traction agent, then it should be going through the proxy. So if it requires some functions that are unavailable, we should discuss. particularly if it was doing both admin calls (no Authorization Bearer token, just API_ADMIN_KEY) and "tenant" calls.
still up in the air, but I think having a plugin delivered with requirements.txt for pip is probably best, poetry can export a requirements.txt file, so it could sit beside pyproject.toml.

The FE for setting the webhooks is https://github.com/bcgov/traction/issues/448

@usingtechnology I think 1 and 3 are related. I will try and catch-up with @ianco about what we need exactly. The Aries VCR agents would not be making calls that would require the ADMIN_API_KEY so we should be good. I will start poking at things.
I'm leaning myself towards using pip and not adding extra dependencies to the toolchain unless extremely necessary: what changes are necessary to have pip as default package manager?

I don't think any... assuming building onto von-image. poetry was used simply because almost all of the other plugins were using that as the dependency manager. but as I said, I think devs can use poetry for their work, but just export the requirements and leave that alongside their code and it should work nicely. better to go that route than make devs use poetry and have a pyproject.toml/lock file(s).

I updated this plugin to remove the issuer registration plug-in, and install from github (aries-vcr repo).
I've added the poetry config file to the plug-in in the aries-vcr repo (https://github.com/bcgov/aries-vcr/pull/737) so the plug-in can be installed using pip or poetry.

I think we can consider this issue done. I will log an issue in the aries-vcr-issuer-controller repo to track the changes required to run in traction mode.
The pr in Aries VCR @ianco submitted makes it possible to install the issuer-registration plugin alongside the traction plugin: I do not think we need changes to the traction repo to include the plugin by default, it would be a capability that is added at build time, if required.


Remove duplicate dependency on Rails to prevent RubyGems error.

See https://github.com/rubygems/rubygems/blob/master/lib/rubygems/specification.rb#L2795

Great thanks.


remake gifs using ttyrec and ttygif

The gifs are fast, light, pretty, and faithful to the originals.
Size changes
interactive.gif: 2.0M -> 104K
simple.gif:      3.4M -> 108K

Fixes #3

Coverage remained the same at 48.111% when pulling c387ba5bf47d3234cf0035a7e48dc1dae800c40e on alxgnon:gifs into 625c80ec4987a9c995a8662a7e49e3d686fef586 on meeDamian:master.

That failing build is most likely not my fault

Yup, gifs most definitely didn't break the build :D
Thanks and merging.


Update IAM URL in notebook number 2

The Link to KG authorization endpoint points to nexus-iam.humanbrainproject.org, needs to be updated to the new one.

should be resolved now, that siibra-python migrated to device flow.


PalWorldSettings.ini Changes Not Reflected in Game

Describe the bug
None of the inputs shown in PalWorldSettings.ini are actually implemented in game.
To Reproduce

Pick up a rock +1 EXP with EXP rate set to 9
/AdminPassword  results in error "Admin password not set for this server"

Expected behavior
Changes in PalWorldSettings.ini to reflect in game.

Root of this issue is:
I am creating PalWorldSettings.ini in this format:
if [[ ${use_custom_palworldsettings} == "false" ]]; then
mkdir -p /palworld-server/Pal/Saved/Config/LinuxServer/
cat > /palworld-server/Pal/Saved/Config/LinuxServer/PalWorldSettings.ini <<EOF
[/Script/Pal.PalGameWorldSettings]
OptionSettings=(Difficulty=${difficulty},
...
...
ServerDescription="${server_description}",
%{ if admin_password != "" ~}
AdminPassword="${admin_password}",
%{ else ~}
AdminPassword="",
%{ endif ~}
%{ if server_password != "" ~}
ServerPassword="${server_password}",
%{ else ~}
ServerPassword="",
%{ endif ~}
PublicPort=${public_port},
%{ if public_ip != "" ~}
PublicIP="${public_ip}",
%{ else ~}
PublicIP="",
%{ endif ~}
RCONEnabled=${enable_rcon},
%{ if rcon_port != "" ~}
RCONPort=${rcon_port},
%{ else ~}
RCONPort="",
%{ endif ~}
%{ if region != "" ~}
Region="${region}",
%{ else ~}
Region="",
%{ endif ~}
bUseAuth=${use_auth},
BanListURL="${ban_list_url}"
)
EOF
fi

This is generating the PalWorldSettings.ini file on the host in the correct format with the correct inputs.
HOWEVER, once Palworld starts it is overwriting some of the values and stripping quotes from strings, a missing closing ), and commas after some values. I believe this has something to do with Palworld doing something in the background to format the file, but because my file is multiple lines its doing whacky stuff.
I have verified that changing the PalWorldSettings.ini file to 1 line and adding "" quotations around the strings such as AdminPassword and restarting Palworld fixes the issue.
Will update module with a fix to prevent this.


Track each run and allow stop, resume, rerun functions from a previous unsuccessful job

What this PR does:
Track each run and allow stop, resume, rerun functions from a previous unsuccessful job
Checklist:

[ ] Automated Tests added/updated
[ ] Documentation added/updated
[ ] CLA Signed:  DataStax CLA

@msmygit Addressed all your PR comments


Expectations when applying config : m.route to a link & binding onclick

Description
As discussed in the chatroom recently there's an ambiguity over expected behaviour when a link has both a config : m.route and an onclick handle. As seen here, the routeUnobtrusive callback never fires.
Steps to Reproduce:
m.route( document.body, '/', {
  '/' : {
    controller : function(){
      alert( 'Route loaded' )
    },

    view : () =>
      m( 'a', {
        textContent : 'Hello',
        href        : '/',
        config      : m.route,
        onclick     : e => {}
      } )
  }
} )

Expected:
The onclick handler should trigger first, followed by the routeUnobtrusive handler (unless the onclick handler stops immediate propagation). You can see this generic behaviour — without Mithril's m.route behaviour — in action here.
Actual:
routeUnobtrusive never fires.

So, basically, it should be using addEventListener("click", ...) + IE fallback instead.

False alert: this does work — attributes including inline event handlers are bound during build, unobtrusive route config listeners are attached after injection — the test case was broken. Here's a demo.
To prevent a route change from within a click handler, either:

Return false
Explicit invoke e.stopImmediatePropagation() to stop the config handler firing, and e.preventDefault() to prevent native link behaviour


Fail to install due to docker socket file not in right place.

Description
while setting up Velacore using velad install, the following error occurred.

The following three containers are running.

System info
MacOS ventura 13.1

Hi, what's your docker version? Are you using docker desktop?

Docker version is 20.10.20 and I'm using docker desktop.

Can you check whether /var/run/docker.sock is ok?

Thanks @chivalryq ,
The issue got resolved by running sudo ln -s $HOME/.docker/run/docker.sock /var/run/docker.sock
Now Velacore is installed perfectly.

But while running vela addon enable ./addon it is showing command not found: vela

vela CLI is not installed and its showing
fail to install vela CLI: Fail to create symlink: symlink /usr/local/bin/velad /usr/local/bin/vela: permission denied

You can create a symlink from /usr/local/bin/velad to /usr/local/bin/vela @NeerajGartia21

Thanks @chivalryq , It got successfully installed.

Cool! I'm gonna close this issue since it's addressed.

@NeerajGartia21 @chivalryq
If you run Docker Desktop, /var/run/docker.sock is no longer created. See: https://github.com/docker/for-mac/issues/6529
A work-around without needing to sudo ln -s is to explicitly set the DOCKER_HOST environment variable before running velad:
export DOCKER_HOST="unix://$HOME/.docker/run/docker.sock"
velad install

@justenwalker Great job!


fix spacing of different blocks in register panel

We must keep both the blocks in same position.
Basic information table must be moved to the same level of the first flavor.
Note - the spacing seem to be fine, if there are two flavors only. but misplaced when there are more than 2 .

let me do it

let me do it


POWER: AIX: Make use of power10 optimization

POWER10 optimizations are disabled when using default AIX assembler.
As we have fixed many issues recently, enabling optimization path for default assembler.

@kavanabhat FYI


question about evaluators.py

hi!
in the  open-reid/reid/evaluators.py
def pairwise_distance(features, query=None, gallery=None, metric=None):
if query is None and gallery is None:
n = len(features)
x = torch.cat(list(features.values()))
x = x.view(n, -1)
if metric is not None:
x = metric.transform(x)
dist = torch.pow(x, 2).sum(1) * 2
dist = dist.expand(n, n) - 2 * torch.mm(x, x.t())
return dist

the result seems not euclidean metric?
in this 'pairwise_distance' function,the second part is computing euclidean metric,but the first part looks not right?
why they are different?
thanks!

Say we have a matrix X with N samples. The first part actually computes the pairwise distances inside X, resulting in a NxN distance matrix.


Caffe model in opencv

I tried running the converted caffe models via opencv dnn caffe script and am unable to interpret the results.
Ideally the detection outputs from the forward pass during inference should be a set of bboxes, conf.. predictions.
But what I am receiving is a list of (1, 13242, 2) values. What do these represent exactly?
self.net = dnn.readNetFromCaffe(self.prototxt_file, self.model_pathinp_width =480, inp_height = 480)
self.net.setInput(dnn.blobFromImage(frame, 1.0, (self._f_width, self._f_height), (self._m1, self._m2, self._m3), False, False))
self.detections = self.net.forward()
print(np.shape(self.detections))

The converted caffe model does not contain the detectionout layer，the outputs has two arrays, one is  scores(1,4420,2) and the other is boxes(1,4420,4)，so you need to add the post processing.

Thanks!
I am getting outputs in the following form:
-0.06667982 -0.58126396 -1.1865556 -0.27930307
-0.086416796 -0.89094937 -1.3859854 -0.5046445
0.049911253 -0.33554888 -1.1623797 -0.08576904
0.06991154 -0.5340975 -1.2665751 -0.25954255
0.05625076 -0.8642361 -1.4737158 -0.5085962
0.016570471 -0.38922626 -1.5873095 -0.37555012
are these in the scaled form (xc, yc, w, h) ?

@pjspillai This is the coordinates of the center point format before post-processing. You can refer to my implementation of post-processing in  Python implemententation of this project  with MNN.

There are discrepancies in the model output caffe vs pytorch for a given image.
I tried modifying the Predictor class from vision.ssd.predictor.py and do the post processing of the boxes and scores.
I made sure that the image preprocessing steps, nms params, thresholds, img size, is al matching. But couldn't figure out why the detection outputs are so different!

I previously tested that the caffe model can be converted into the ncnn model normally and inference  normally. When I have time I will run this model in caffe .

@pjspillai .I just tested it, and the caffe inference results are normal.

@pjspillai . I uploaded the caffe inference code.

Thanks Linzaer!
I was referring to the **opencv DNN module (ONNX, Caffe) ** and loading the models there directly.
I made slight modifications to your code, and getting empty bounding boxes after the post processing
`def inference():
net = dnn.readNetFromCaffe(caffe_prototxt_path, caffe_model_path)

input_size = [int(v.strip()) for v in args.input_size.split(",")]
witdh = input_size[0]
height = input_size[1]
priors = define_img_size(input_size)

img_ori = cv2.imread(imgs_path)

print(np.shape(img_ori))
tmp_batch = np.zeros([1, 3, height, witdh], dtype=np.float32)
image = cv2.resize(img_ori, (witdh, height))
image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
# image = (rect - image_mean) / image_std
# tmp_batch[0, :, :, :] = image.transpose(2, 0, 1)
# image = image.astype(np.uint8)
# print((image.dtype))

net.setInput(dnn.blobFromImage(image, 1.0, (witdh, height), (127, 127, 127), False, False))

# net.blobs['input'].data[...] = tmp_batch
time_time = time.time()
# scores = net.forward()['scores'][0]
# boxes = net.forward()['boxes'][0]
boxes, scores = net.forward(["boxes", "scores"])

boxes = boxes[0]
scores = scores[0]

print(boxes)
print("inference time: {} s".format(round(time.time() - time_time, 4)))
boxes = np.expand_dims(np.reshape(boxes, (-1, 4)), axis=0)
scores = np.expand_dims(np.reshape(scores, (-1, 2)), axis=0)
boxes = convert_locations_to_boxes(boxes, priors, center_variance, size_variance)
boxes = center_form_to_corner_form(boxes)
boxes, labels, probs = predict(img_ori.shape[1], img_ori.shape[0], scores, boxes, threshold)
print(boxes.shape[0])

`

Change net.setInput(dnn.blobFromImage(image, 1.0, (witdh, height), (127, 127, 127), False, False)) tonet.setInput(dnn.blobFromImage(image,  **0.0078125** , (witdh, height), (127, 127, 127), False, False))

It works beautifully! Thanks so much!


[GOBBLIN-1693] Move kafka ingestion health model logic to separate class

Dear Gobblin maintainers,
Please accept this PR. I understand that it will not be reviewed until I have checked off all the steps below!
JIRA

[x] My PR addresses the following Gobblin JIRA issues and references them in the PR title. For example, "[GOBBLIN-1693] My Gobblin PR"

https://issues.apache.org/jira/browse/GOBBLIN-1693

Description

[X] Here are some details about my PR, including screenshots (if applicable):
I moved the kafka ingestion health check stats tracking logic to a model class. (Think model-view-controller kind of model).
The goal is to clean up the code s.t.:

KafkaIngestionHealthCheck: Emits GTE if health model fails. And is responsible as the "controller" for the model
KafkaIngestionHealthModel: Has API's for updating state while also keeping state about the current health over a sliding window.

Another change is to generify the ContainerHealthCheckFailureEvent to a MetadataBasedEvent. This means we can add other types of failures that will automatically be submitted as GTE.

Future change will add another MetadataBasedEvent specifically for helix dynamic workunit allocation

Tests

[X] My PR adds the following unit tests OR does not need testing for this extremely good reason:
Purely a refactor without any functional change

Commits

[X] My commits all reference JIRA issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":

Subject is separated from body by a blank line
Subject is limited to 50 characters
Subject does not end with a period
Subject uses the imperative mood ("add", not "adding")
Body wraps at 72 characters
Body explains "what" and "why", not "how"

Codecov Report

Merging #2 (f0f4de2) into helix-dynamic-workunit (593eacd) will decrease coverage by 2.97%.
The diff coverage is 0.00%.

@@                     Coverage Diff                      @@
##             helix-dynamic-workunit       #2      +/-   ##
============================================================
- Coverage                     46.74%   43.76%   -2.98%
+ Complexity                    10514     2054    -8460
============================================================
  Files                          2100      406    -1694
  Lines                         82023    17529   -64494
  Branches                       9134     2142    -6992
============================================================
- Hits                          38343     7672   -30671
+ Misses                        40138     9002   -31136
+ Partials                       3542      855    -2687

Impacted Files
Coverage Δ

.../org/apache/gobblin/cluster/GobblinTaskRunner.java
63.63% <0.00%> (-0.32%)
:arrow_down:

...n/util/event/ContainerHealthCheckFailureEvent.java
0.00% <0.00%> (ø)

...actor/extract/kafka/KafkaIngestionHealthCheck.java

.../compaction/listeners/SerialCompactorListener.java

.../org/apache/gobblin/test/SequentialTestSource.java

...ce/modules/orchestration/AzkabanAjaxAPIClient.java

...he/gobblin/writer/WatermarkAwareWriterWrapper.java

...g/apache/gobblin/converter/jdbc/JdbcEntryData.java

...n/google/webmaster/GoogleWebMasterSourceDaily.java

... and 1688 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


SPI dump fix & target flash disambiguation

Hi Bricky,
You want me to uninstall mono :laughing:
According to the UART command sent it seems the "read" command dump from SPI flash but it produce a 64k dump from various adresses.
This PR try to align the behavior with the .net tool from UP (same 4 MB dump produced) and I took the opportunity to disambiguate the term "flash" in the help section.

Hey reppad,
I had these changes internally as I was hoping to decipher the offsets before committing. I'm happy to accept the PR given what I wrote was from my understanding of the CLI tool, and given it didn't have the restore stuff in it I had made assumptions of how it worked.


calc doesn't unfold target when it is a relation

def Commute (a b : Nat) := a * b = b * a

example : Commute 1 2 :=
  calc
    1 * 2 = 2 * 1 := by simp
-- invalid 'calc' step, left-hand-side is
--   1 * 2 : ?m.178
-- expected
--   1 : Nat

As reported on Zulip.

I think this might be the same issue. This used to work:
example (h : y ≤ x) : 3 + x > 2 + y := calc
  _ < 3 + y := Nat.add_lt_add_right (by decide) y
  _ ≤ 3 + x := Nat.add_le_add_left h 3

The quick workaround is:
example (h : y ≤ x) : 3 + x > 2 + y :=
  show 2 + y < 3 + x from calc
  _ < 3 + y := Nat.add_lt_add_right (by decide) y
  _ ≤ 3 + x := Nat.add_le_add_left h 3

Yes, that's the same issue. See also https://github.com/leanprover-community/mathlib4/pull/1927/files which has a bunch of examples of the issue.

I've reverted the corresponding part of #2066.  It's a bit of a tradeoff, there's a few regressions with coercions but maybe that's better for mathlib overall.


Proxy different paths on the same domain to different ports

Hi Peter,
First of all thanks again for a great tool! As mentioned before it's become essential to my workflow 🙏
Now than, a feature request, or maybe question, of sorts.
I've ran into a situation where I'd like to have a main domain linked to a port, but then have a unique sub-directory assigned to a different port. In terms of localias config, something like:
# localias config file syntax

domain.local: 3000
domain.local/subdirectory: 3001

Right now, this gives the following error:
error: loading new config: loading http app module: provision http: server srv0: setting up route handlers: route 0: loading handler modules: position 0: loading module 'map': http.handlers.map: invalid configuration: mapping 1 has a duplicate input 'domain.local' previously used with mapping 0

which is understandable, but could this potentially work? Or do you see any blockers?
Thanks beforehand for taking the time to review this issue!

Hi Peter,
Awesome! Thanks for considering the feature request & the thorough response.
Indeed, one of the things I really like about the .localias.yaml configuration file is that it's essentially an abstraction layer that takes away a lot of the complexity of Caddyfile config. This allows colleagues who don't have a lot of experience in that field to still achieve complex stuff 👍
If you require any further information on how I / we would use this at an enterprise-type scenario, please let me know, I'd be happy to provide more details :-)


Suggestions for new indexer APIs

Based on my understanding, currently the Lumos indexer mainly has two APIs for querying the live cells and all the transactions. As reported in the issue, when there is a huge volume of transactions under a lock script, it will take a very long time for Lumos to process and response. In that case, it took more than 2 hours to return all the transactions under a lock script on my machine.
Because of the needs of flexibly querying the transactions and cells, either live or dead,  applications, such as Neuron, would still need to cache the data in a certain way while utilizing Lumos as the basic indexing engine, which helps maintain the full index with small disk space footprint and stream the data based on lock script to the client-side.
Therefore, to fulfill the requirements for certain applications, such as Neuron, some new APIs will be needed to work around the performance issues.
Below are some suggestions on the new APIs on Lumos that I think would be helpful in providing more flexibility for the applications to efficiently process necessary transactions and be able to customize the queries that are easier to divide and conquer. Please feel free to comment or suggest other solutions.

#listenByLockScript provides an event sourcing API for the applications to listen to new transactions under a lock script via WebSocket.

This would allow the applications to process the new transactions as soon as possible without the need to poll and check if there are new transactions under the monitored lock scripts recorded on-chain, simplifying the codebase in the application layer while avoiding unnecessary performance overheads introduced by the potential polls, which would need to check block numbers / compare the transaction counts between indexer and the ones in local cache.
In addition to new transactions event, it would be also helpful to emit the forked event when there is a fork occurred, so as to notify the application layer in order for them to do necessary rollbacks in their local caches.

#getTotalTransactionsCountByLockScript allows to fetch the total number of transactions under a lock script.

This will be useful when the applications want to compare with the local cache, and check if it is necessary to sync the new transactions in their cache after disconnected the network for a while.

#getTransactionsByLockScript(offset, fetchSize) basically supports the pagination when fetching transactions under a lock script. With the #getTotalTransactionsCountByLockScript suggested above, the applications can customize fetching patterns either in parallel fashion in a batch or just a page of a certain size of transactions.

#getTransactionsByLockScript(fromBlock, toBlock) allows querying the transactions under a lock script between a block number range.

I get that a new cell listener can be quite useful, but there are more concerns to this:

The implementation is actually more tricker to get right. Any subscription mechanism will have a problem: should it be reliable or not? If there is a downstream listening at the updates, but the downstream cuts off for a while, when it reconnects, should we send it all the missing updates it didn't receive, or should we continue here? This is actually a more complicated problem to get right if you think in the framework directly. So for now, I still recommend a polling solution.
Even if we have a subscription service, having WebSocket in a dapp framework doesn't really make sense. What we have is a single entity here, if the dapp wants to relay updates to client side, it can then decide to use WebSocket, the lumos framework, won't need this level of details.

Yes this is an API we can fetch

I think there are 2 problems:

The slowdown is an implementation quirk within lumos-indexer, we can fix this without affecting the current API.
With that considered, I'm not sure if we need a pagination solution. The question I would ask, is what could be achieved in a pagination solution, but couldn't achieved in an async interator? Again, we are talking about a framework that is directly called by the dapp, we don't have the usual gap of web, where you have to pass in parameters to work around pagination. In the lumos indexer case, I do believe an iterator provides more potentials.

Yes, this is something we can add as query parameters.

Thanks @xuejie for the quick response. Glad to know 2 and 4 will be supported.

Any subscription mechanism will have a problem: should it be reliable or not? If there is a downstream listening at the updates, but the downstream cuts off for a while, when it reconnects, should we send it all the missing updates it didn't receive, or should we continue here?

I think this is a good point. In terms of streaming "cut-off", I think the Lumos Indexer just needs to push whatever they newly received from the CKB-indexer to the subscribers. The downstream clients can use #getTotalTransactionsCountByLockScript to check if there is a gap to fill in the missing transactions. If there is mismatch between the count from Lumos Indexer and the one from local cache, then it can call #getTransactionsByLockScript(fromBlock, toBlock) to re-sync the transactions between a block range.

What we have is a single entity here, if the dapp wants to relay updates to client side, it can then decide to use WebSocket, the lumos framework, won't need this level of details.

But in the case, Neuron is the client of Lumos, isn't it? :)

The slowdown is an implementation quirk within lumos-indexer, we can fix this without affecting the current API.

Sounds good. Just FYI, on my machine, it took more than 2 hours to fetch all transactions for a certain lock script. Hopefully some optimizations can be done on Lumos side to dramatically improve the fetching efficiency so that the process can be done within an acceptable timeframe that is user friendly enough.

The question I would ask, is what could be achieved in a pagination solution, but couldn't achieved in an async interator?

I think I kind of getting what you mean. To do the pagination on the generator, it could call the #next without the await before calling the next #next, until it has done a certain number of #next, which is to simulate how the traditional pagination works. But there are two critical problems, if I understand correctly:

When calling #next, it is not just iterating into next generator, but also calls the actual fetch function. If the desired section of transactions is very deep in the pagination, then all the precedent transactions will need to be unnecessary loaded onto memory. Also, these precedent transaction fetch calls will be run in parallel against the CKB-indexer (or node actually), which is likely to get bombarded with the RPC calls(assume the transaction number is 100k+) and slow to respond.

Let's assume the first one is non-issue, based on my test with the same lock script mentioned in the other thread, getting all the transaction keys took at least 5 seconds, which I think is too long for the clients to wait for.

Maybe I missed something here, so it will be very helpful if you can provide an example of doing the pagination using TransactionsCollection#collect for a non-zero page without introducing the unnecessary RPC calls.

I think the Lumos Indexer just needs to push whatever they newly received from the CKB-indexer to the subscribers.

If neuron is the sole client of lumos, I would agree with you. But lumos is designed for many different scenarios, people might rely on it in unexpected ways, I'm not sure if it is a good idea unless we have a good enough solution for the problem.

But in the case, Neuron is the client of Lumos, isn't it? :)

We need to carefully define the word "client" here. Neuron will be expected to use lumos-indexer as a library, between 2 libraries in a single binary, I doubt if WebSocket is needed here. Even between the client side code and the underlying backend code of neuron the electron app, I doubt a WebSocket connection makes sense. It just complicates the logic for no obvious gains.

But there are two critical problems, if I understand correctly: (...)

I think we've talked about this and I mentioned this several times in offline chat. Please don't make decisions based on the current implementation of lumos. The only thing that matters, and what we should discuss, is if the API designed makes sense, not that current implementation of the API has some quirks that might slow things down. Both issues you are mentioned here, IMO, could perfectly be solved without changing any of the async iterator API.

Also one more word regarding pagination: I think pagination is merely an abstract term when you have 2 separate entities. Such as between the frontend running in the user's browser, and the app running on remote server; or between a backend server, and the SQL database. It's the fact that you have 2 separate entities that don't share anything, that you need an abstract term like pagination. But what we are talking about here, is lumos the framework, and the upper level dapp code, they coexist in the same process where they can share data with each other. In this sense, I do feel an async iterator, or in its core, a closure, is more than enough to achieve the use case with better performance potential, and less resource usage.

Ok, cool.
I think for now 2 and 4 are useful enough for Neuron to have some basic processes to digest the data from the Indexer, assuming they won't inherit the performance issue from the current implementation.
If possible, an estimation on when these two API would be available is highly appreciated.
Thanks.

We can aim at the end of next week to provide 2 and 4 here.


[Enhancement] (nereids)implement showWaringErrorCountCommand in nereids

showWaringErrorCount command is supported in legacy planner but not in nereids(new doris fe planner). you need implement the following showWaringErrorCount command in nereids:
unsupportedshowWaringErrorCountStatement
SHOW COUNT LEFT_PAREN ASTERISK RIGHT_PAREN (WARNINGS | ERRORS)

@LiBinfeng-01 will pick this up


consider using ECIES for message encryption

https://en.wikipedia.org/wiki/Integrated_Encryption_Scheme
https://github.com/ecies/go

That was implemented in easyecc, but it didn't work right away on IMAP mailbox. I suspect there is a bug in Golang ECIES library. Closing it for now, the existing encryption works fine.


Adds PyTorch DLRM Workload

This is a Preliminary PR for Pytorch DLRM workload.
Still trying to sort out the deadlock issue. But here is what I have so far:
Reference model implementation ported from JAX
Dataloader working fast enough
Basic setup of optimizer inplace - might need tweaking and equivalence checking to JAX version
Has DDP support
I have some scripts testing the forward prop and reverse prop time, workload takes about 30 ish mins to train with the full dataset.
Still doesn't converge to target AUC.
Once I sort the deadlock issue, I can get to the convergence issue.
Data loader loads the whole data in 600 ish seconds. Just iteration takes 200 - 600s (NVIDIA recommends to place this dataset on a fast storage).
**TODO: **

[ ] Add readme that shows preprocessing steps (this preprocessing takes several days - I have it still running from over 4 days now) maybe accelerate this preprocessing step.
[ ] Add test cases for equivalence
[ ] Improve data loading if needed by splitting the large dataset into shards
[ ] Remove AUC from metrics if not needed and ensure the target metric is finalized to convergence
[ ] There are certainly opportunities to improve the training time, will focus once the model is ready and at functional stage

I noticed I have some print statements I used for debugging here. Please ignore them. I'll remove them in the upcoming PRs.
Example:
https://github.com/rakshithvasudev/algorithmic-efficiency/blob/989ae3eb9bcd53fdf36dbee31977ba4dac279098/algorithmic_efficiency/workloads/criteo1tb/criteo1tb_pytorch/workload.py#L199

Thanks for reviewing the PR @runame. Appreciate the detailed review. Will do these changes. I'd put second priority to these cosmetic changes until I resolve the deadlock issue and convergence issue.
Great pointers on the convergence issue, will test them and see :)
Please ignore the inconsistencies with the numbers like batch size and eval time, etc. from the JAX workload not being replicated here. Those are some knobs I was tweaking to try to debug the deadlock.
Thanks again for the PR review! Will push another one soon.

Closing as this has been subsumed by https://github.com/mlcommons/algorithmic-efficiency/pull/146.


Consider adding pyupgrade to linting

Just another linting tool to add to your linting workflow.
I can send a PR.

Thank you, if you can send a PR that would be great


Add dockerized debug option for stride node [WIP]

Context and purpose of the change
Golang apps can be debugged using dlv. This allows inspection of variables and program execution flow of running instances of binaries.
This PR adds dlv to a Docker file and runs dlv inside the first stride container.
To use:
DOCKER_DEBUG=true make start-docker build=sgr

This will start dockernet as usual, with one exception: strided in stride1 starts a headless debugging session, which waits for a connection from the local debugger. Either start the debugger with dlv connect <IP_ADDRESS>:2345 or run it from an IDE.
For example, with VSCode you'd add lauch options. Go to its debugging page (CTRL+SHIFT+D) where you can edit launch.conf to add the option to attach to the headless server:
{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
		{
			"name": "Attach",
			"type": "go",
			"request": "attach",
			"mode": "remote",
			"remotePath": "/src/",
			"port": 2345,
			"host":"<IP_ADDRESS>",
			"showLog": true,
			"trace": "log",
			"logOutput": "rpc"
		}
    ]
}

Then add your breakpoints to stop the execution of stride1 for inspection of variables and execution stack/threads:

When the network is running, you can run the integration tests as follows:
DOCKER_DEBUG=true make test-integration-docker

Brief Changelog
This PR adds a separate Docker compose file (docker-compose-debug.yml) for starting dockernet with dlv running inside the first stride container.

Add the environment variable DOCKER_DEBUG as a flag to build and start the debug ready dockernet
Add Dockerfile.debug and docker-compose-debug.yml to allow building and starting the debug dockernet with minimal changes to the dockernet initalization routine

Author's Checklist
I have...

[x] Run and PASSED locally all GAIA integration tests
[x] Manually verified the change by inspecting various states of execution

Reviewers Checklist
All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.
I have...

[ ] reviewed state machine logic
[ ] reviewed API design and naming
[ ] manually tested (if applicable)
[ ] confirmed the author wrote unit tests for new logic
[ ] reviewed documentation exists and is accurate

Documentation and Release Note

[ ] Does this pull request introduce a new feature or user-facing behavior changes?
[ ] Is a relevant changelog entry added to the Unreleased section in CHANGELOG.md?
[ ] This pull request updates existing proto field values (and require a backend and frontend migration)?
[ ] Does this pull request change existing proto field names (and require a frontend migration)?
How is the feature or change documented?

[ ] not applicable
[ ] jira ticket XXX
[ ] specification (x/<module>/spec/)
[ ] README.md
[ ] not documented

Thanks for opening this PR. Could you add some instructions on how to start a simple debugging session to the description or maybe record a short 2 min Loom screenshare explaining how to use the debugger?
Running make build-docker-debug gives me
Building STRIDE-DEBUG Docker...
[+] Building 0.0s (1/2)
 => [internal] load build definition from Dockerfile.stride-debug                                                                        0.0s
 => => transferring dockerfile: 2B                                                                                0.0s
failed to solve with frontend dockerfile.v0: failed to read dockerfile: open /var/lib/docker/tmp/buildkit-mount685298487/Dockerfile.stride-debug: no such file or directory
Failed
make: *** [build-docker-debug] Error 1

Thanks for opening this PR. Could you add some instructions on how to start a simple debugging session to the description or maybe record a short 2 min Loom screenshare explaining how to run a simple example case?

Looks like I didn't clean up properly after rebasing the changes.. Could you try again?
Also, I added a brief explanation to the PR. Does that help?

Trying again, thanks for cleaning this up. I'm able to run dockernet now and attach to the debug headless server with VSCode's CRTL+SHIFT+D pane. Changing launch.json's "remotePath": "/src/", to "remotePath": "" allowed me to set breakpoints and pause execution. Will play around with it a bit more. Thanks for adding this!

Rebased on the updated main branch