Catchup loop with verifier start & stop

Preliminary Checks

[X] This issue is not a duplicate. Before opening a new issue, please search existing issues: https://github.com/MinaProtocol/mina/issues
[X] This issue is not a question, feature request, RFC, or anything other than a bug report. Please post those things in GitHub Discussions: https://github.com/MinaProtocol/mina/discussions

Description
I have one node (1.3.0 or 1.3.1) running docker that ends up in a catch up loop. It starts to sync normally, then it ends up in a loop of verifier start & stop.
The logs look all normal, until it ends up like this:
2022-05-24 08:22:30 UTC [Info] Received a block from $sender
	sender: {
  "Remote": {
    "host": "<IP_ADDRESS>",
    "peer_id": "12D3KooWBNSKpD5ErAKCRW36xq1wGG5vYUsBWweeHnEemrjvGXir",
    "libp2p_port": 8302
  }
}
2022-05-24 08:22:37 UTC [Info] Daemon started process of kind "Verifier" with pid 787
2022-05-24 08:22:37 UTC [Error] Verifier stderr: $stderr
	stderr: "Using password from environment variable MINA_PRIVKEY_PASS\nUsing password from environment variable MINA_PRIVKEY_PASS\nUsing password from environment variable MINA_PRIVKEY_PASS\n"
2022-05-24 08:22:43 UTC [Error] verifier terminated unexpectedly
2022-05-24 08:22:43 UTC [Info] Starting a new verifier process
2022-05-24 08:22:43 UTC [Info] verifier successfully stopped
2022-05-24 08:22:58 UTC [Info] Received a block from $sender
	sender: {
  "Remote": {
    "host": "<IP_ADDRESS>",
    "peer_id": "12D3KooWPCBShZwQVY2P5KHqCtur1QH1ngJoxRc9bu3KBcfcSjsj",
    "libp2p_port": 8302
  }
}
2022-05-24 08:23:11 UTC [Info] Daemon started process of kind "Verifier" with pid 801
2022-05-24 08:23:12 UTC [Error] Verifier stderr: $stderr
	stderr: "Using password from environment variable MINA_PRIVKEY_PASS\nUsing password from environment variable MINA_PRIVKEY_PASS\nUsing password from environment variable MINA_PRIVKEY_PASS\n"
2022-05-24 08:23:17 UTC [Error] verifier terminated unexpectedly
2022-05-24 08:23:17 UTC [Info] Starting a new verifier process
2022-05-24 08:23:17 UTC [Info] verifier successfully stopped
2022-05-24 08:23:45 UTC [Info] Daemon started process of kind "Verifier" with pid 815
2022-05-24 08:23:45 UTC [Error] Verifier stderr: $stderr
	stderr: "Using password from environment variable MINA_PRIVKEY_PASS\nUsing password from environment variable MINA_PRIVKEY_PASS\nUsing password from environment variable MINA_PRIVKEY_PASS\n"
2022-05-24 08:23:51 UTC [Error] verifier terminated unexpectedly
2022-05-24 08:23:51 UTC [Info] Starting a new verifier process
2022-05-24 08:23:51 UTC [Info] verifier successfully stopped
2022-05-24 08:24:19 UTC [Info] Daemon started process of kind "Verifier" with pid 840
2022-05-24 08:24:20 UTC [Error] Verifier stderr: $stderr
	stderr: "Using password from environment variable MINA_PRIVKEY_PASS\nUsing password from environment variable MINA_PRIVKEY_PASS\nUsing password from environment variable MINA_PRIVKEY_PASS\n"
2022-05-24 08:24:26 UTC [Error] Validation error: external transition with state hash $state_hash was rejected for reason "verifier error"
	state_hash: "3NKLGVNZhu5ZNZBs7p4xaRC3CuJb545H8pfFijmNVKgTEfqhqwEY"
2022-05-24 08:24:26 UTC [Error] Error in verifier verifying blockchain proof for $state_hash: $error
	state_hash: "3NKLGVNZhu5ZNZBs7p4xaRC3CuJb545H8pfFijmNVKgTEfqhqwEY"
	error: {
  "commit_id": "9b0369c27bb85c8ab2f8725c6e977eb27b53b826",
  "sexp": [
    [ "rpc_error", [ "Connection_closed", [ "EOF or connection closed" ] ] ],
    [
      "connection_description",
      [ "Client connected via TCP", [ "localhost", "41639" ] ]
    ],
    [ "rpc_tag", "rpc_parallel_plain_2" ],
    [ "rpc_version", "0" ]
  ]
}
2022-05-24 08:24:26 UTC [Error] verifier terminated unexpectedly
2022-05-24 08:24:26 UTC [Info] Starting a new verifier process
....

Steps to Reproduce
Normal docker usage
Expected Result
Expected to sync
Actual Result
Catch up loop
How frequently do you see this issue?
Frequently
What is the impact of this issue on your ability to run a node?
Blocker
Status
docker exec -it mina mina client status

Using password from environment variable MINA_PRIVKEY_PASS
Using password from environment variable MINA_PRIVKEY_PASS
Using password from environment variable MINA_PRIVKEY_PASS
Mina daemon status
-----------------------------------

Global number of accounts:                     107849
Block height:                                  140710
Max observed block height:                     140819
Max observed unvalidated block height:         140841
Local uptime:                                  1h56m5s
Ledger Merkle root:                            jx8DPSrjS3pdnuTganWudnnGg75ALuarivi8hR3LpuconTSFi8c
Protocol state hash:                           3NK7HU9LTGwxNFeynhpU9iaWFC9vWYXEH568tXPhyJ7FGqPVg98y
Chain id:                                      5f704cc0c82e0ed70e873f0893d7e06f148524e3f0bdae2afb02e7819a0c24d1
Git SHA-1:                                     9b0369c27bb85c8ab2f8725c6e977eb27b53b826
Peers:                                         32
User_commands sent:                            0
SNARK worker:                                  None
SNARK work fee:                                100000000
Sync status:                                   Catchup
Catchup status:
	To initial validate:           12
	Finished:                      2
	To download:                   0
	Waiting for parent to finish:  23
	To verify:                     74

### Additional information

_No response_

Hello @nerdvibe. Could you please share the content of mina-verifier.log from your mina config directory? That should have the information we need to determine why the verifier process is failing to start.

Closing this issue because of being inactive for long time.
Please feel free to open another ticket should you experience any further issues.


Then

Then - ✨ Super sweet syntactic sugar for Swift initializers. :large_orange_diamond:[e]

Utility?

Thanks, @victorgama


Fix #5046 compatible with MacOS

Signed-off-by: Marc Khouzam<EMAIL_ADDRESS>What this PR does / why we need it:
Fix issue #5046 for MacOS.
The sed tool on MacOS does not support the use of \w.  This PR uses a more portable syntax to convert bash completion to zsh completions.
Different regex could have been used, but I tried to mimic what the original \w\+ aimed to do, which was to match one or more characters.  Granted, . matches more things than \w, but I don't believe it is necessary to be that strict in the matching rule.
I've tested the syntax on my mac and on alpine linux in docker.
I've also built helm with the change on my mac and confirmed that it fixes the zsh completion.

@kalinon would you have the time available to test this? It appears that you also identified this issue in https://github.com/helm/helm/pull/5072#issuecomment-450047959.

@kalinon would you have the time available to test this? It appears that you also identified this issue in #5072 (comment).

Ill give it a whirl.

Confirmed this works!

Thanks @kalinon!

Thanks for the quick turnaround!

this fix will be added to 2.13.1 (https://github.com/helm/helm/milestone/62) ?

This could be interpreted as a patch, yes. I'll add it to the milestone for cherry-picking into 2.13.1.


dictionary changed size during iteration when calling Limb.joint_angles()

I'm calling joint_angles from a different thread than the thread in which I called rospy.init_node(), and I get about 20% of the time, the exception thrown below. Is there a way to make sure this method is thread safe?
Exception in thread Thread-38:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 763, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/home/brawner/rosjava/src/helpful_baxter/baxter_action_server/nodes/server.py", line 92, in pick_and_place
    MoveHelper.move_to_neutral("left", True)
  File "/home/brawner/workspace/src/baxter_h2r_packages/baxter_pick_and_place/src/baxter_pick_and_place/move_helper.py", line 32, in move_to_neutral
    return MoveHelper._move_to_neutral_follow_joint(limb)
  File "/home/brawner/workspace/src/baxter_h2r_packages/baxter_pick_and_place/src/baxter_pick_and_place/move_helper.py", line 58, in _move_to_neutral_follow_joint
    pick_up_shoulder = MoveHelper._create_joint_angles({"left_s1" : -0.55, "left_e1":left_e1, "left_w1": left_w1}, arm)
  File "/home/brawner/workspace/src/baxter_h2r_packages/baxter_pick_and_place/src/baxter_pick_and_place/move_helper.py", line 75, in _create_joint_angles
    base_angles = arm.joint_angles()
  File "/home/brawner/workspace/src/baxter_interface/src/baxter_interface/limb.py", line 198, in joint_angles
    return deepcopy(self._joint_angle)
  File "/usr/lib/python2.7/copy.py", line 163, in deepcopy
    y = copier(x, memo)
  File "/usr/lib/python2.7/copy.py", line 256, in _deepcopy_dict
    for key, value in x.iteritems():
RuntimeError: dictionary changed size during iteration

I've attempted to recreate this issue through this gist:
https://gist.github.com/rethink-imcmahon/04e57bd556667a2fa951
I cannot seem to replicate the issue described even when I spawn 10 threads all attempting to access the same limb's joint angles. If this issue surfaces again, please post here with a distilled test setup and I will attempt to remedy it.


Host documentation for ilc-* crates

crates.fyi will be a viable option after onur implements /latest/ redirection, which is apparently tbd soon.


Fix for adding the table-helper package in chapter 3 (Build system)

In both Zig 0.8 and master this didn't comply with the API.

This API usage still works in master

I might be misunderstanding something, but this did not for me.
$../zig version
0.9.0-dev.952+0c091feb5

$cat build.zig
const std = @import("std");

pub fn build(b: *std.build.Builder) void {
    // Standard target options allows the person running `zig build` to choose
    // what target to build for. Here we do not override the defaults, which
    // means any target is allowed, and the default is native. Other options
    // for restricting supported target set are available.
    const target = b.standardTargetOptions(.{});

    // Standard release options allow the person running `zig build` to select
    // between Debug, ReleaseSafe, ReleaseFast, and ReleaseSmall.
    const mode = b.standardReleaseOptions();

    const exe = b.addExecutable("p", "src/main.zig");
    exe.setTarget(target);
    exe.setBuildMode(mode);

    exe.addPackage(.{
        .name = "table-helper",
        .path = "libs/table-helper/table-helper.zig",
    });

    exe.install();

    const run_cmd = exe.run();
    run_cmd.step.dependOn(b.getInstallStep());
    if (b.args) |args| {
        run_cmd.addArgs(args);
    }

    const run_step = b.step("run", "Run the app");
    run_step.dependOn(&run_cmd.step);
}

$../zig build
./build.zig:20:17: error: expected type 'std.build.FileSource', found '*const [34:0]u8'
        .path = "libs/table-helper/table-helper.zig",
                ^
<omitted>:1355:24: note: std.build.FileSource declared here
pub const FileSource = union(enum) {
                       ^

What made this API method work for me is this (or the diff in this PR):
    exe.addPackage(.{
        .name = "table-helper",
        .path = std.build.FileSource{ .path = "libs/table-helper/table-helper.zig" },
    });

Sorry you were totally right, I misread the diff.

I see now how my PR might have just been seen as a pointless method switch.
Thank you for reconsidering my PR and accepting it.


Cannot save

So, I created a twitch profile with Inv tweaks and Schematica, right? So I created a ship in creative and want to port it over to my survival. The only problem is, for whatever reason when I hit the off button after pulling up the red/blue point menu, it doesn't change. It just stays there. I've tried restarting the client, but that didn't do anything.

Can you take a screenshot of the UI?

I asked around, and it seems that Schematica didn't work. Sorry for
bothering you :(
On Thu, Jul 19, 2018, 5:51 PM Jadran Kotnik<EMAIL_ADDRESS>wrote:

Can you take a screenshot of the UI?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/Lunatrius/Schematica/issues/360#issuecomment-406425430,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AnW7O05yrLlLhmwVbtRs3GL3ptA2HnVTks5uIP9TgaJpZM4VQV3c
.

If it's a server it may have disabled the mod.

Closing due to lack of feedback.


“can-types.iterator is deprecated” warning when building

I’m not sure where to file this because the issue happens during the steal-tools transpile but is a CanJS warning.
How often can you reproduce it?
Always
Description:
When I build my DoneJS app, I get the following warning multiple times while transpiling: WARN: can-types.iterator is deprecated, use canSymbol.iterator || canSymbol.for("iterator") instead.
Steps to reproduce:

Clone https://github.com/chasenlehara/iterator-deprecated (it’s a fresh DoneJS app, no modifications)
npm install
node build

Expected results: no warnings.
Actual results: this error multiple times while transpiling: WARN: can-types.iterator is deprecated, use canSymbol.iterator || canSymbol.for("iterator") instead.
Environment:

Software
Version

donejs -V
1.0.1

node -v
8.1.4

npm -v
5.3.0

This steal-tools release fixes the warnings: https://github.com/stealjs/steal-tools/releases/tag/v1.8.3
@pYr0x @phillipskevin


Add ecotone support to op_alloy_rpc_types::OptimismTransactionReceiptFileds

Motivation
Closes https://github.com/alloy-rs/op-alloy/issues/57
Solution
Port fields from depreciated alloy type https://github.com/alloy-rs/alloy/commit/ab462f12ac40fd16bd9882c9451d7d62f7c31d37
PR Checklist

[ ] Added Tests
[x] Added Documentation
[x] Breaking changes

lgtm, you're right this should be simplified once we integrate this with the OpTxReceipt, which has the deposit_ fields already.
but for reth we need this as is rn

correct, since we're still using alloy_rpc_types_eth::AnyTransactionReceipt in reth RPC


Can't reassign actions to more specific types

Description

The following code:
import { Action } from "xstate";

const actionImpl: Action<{}, { type: "FOO" | "BAR" }> = {} as any;
const action: Action<{}, { type: "FOO" }> = actionImpl;

Has the following error:
Type 'Action<{}, { type: "FOO" | "BAR"; }>' is not assignable to type 'Action<{}, { type: "FOO"; }>'.
  Type 'ActionObject<{}, { type: "FOO" | "BAR"; }>' is not assignable to type 'Action<{}, { type: "FOO"; }>'.
    Type 'ActionObject<{}, { type: "FOO" | "BAR"; }>' is not assignable to type 'ActionObject<{}, { type: "FOO"; }>'.
      Types of property 'exec' are incompatible.
        Type 'ActionFunction<{}, { type: "FOO" | "BAR"; }, ActionObject<{}, { type: "FOO" | "BAR"; }>> | undefined' is not assignable to type 'ActionFunction<{}, { type: "FOO"; }, ActionObject<{}, { type: "FOO"; }>> | undefined'.
          Type 'ActionFunction<{}, { type: "FOO" | "BAR"; }, ActionObject<{}, { type: "FOO" | "BAR"; }>>' is not assignable to type 'ActionFunction<{}, { type: "FOO"; }, ActionObject<{}, { type: "FOO"; }>>'.
            Types of parameters 'meta' and 'meta' are incompatible.
              Type 'ActionMeta<{}, { type: "FOO"; }, ActionObject<{}, { type: "FOO"; }>>' is not assignable to type 'ActionMeta<{}, { type: "FOO" | "BAR"; }, ActionObject<{}, { type: "FOO" | "BAR"; }>>'.
                The types of 'action.exec' are incompatible between these types.
                  Type 'ActionFunction<{}, { type: "FOO"; }, ActionObject<{}, { type: "FOO"; }>> | undefined' is not assignable to type 'ActionFunction<{}, { type: "FOO" | "BAR"; }, ActionObject<{}, { type: "FOO" | "BAR"; }>> | undefined'.
                    Type 'ActionFunction<{}, { type: "FOO"; }, ActionObject<{}, { type: "FOO"; }>>' is not assignable to type 'ActionFunction<{}, { type: "FOO" | "BAR"; }, ActionObject<{}, { type: "FOO" | "BAR"; }>>'.
                      Types of parameters 'event' and 'event' are incompatible.
                        Type '{ type: "FOO" | "BAR"; }' is not assignable to type '{ type: "FOO"; }'.

This is relevant when I define an action that I use in my machine
type Context = {};
type Event = { type: "FOO" } | { type: "BAR" };
type State = { value: "first"; context: Context };
const createMyMachine = (action: Action<Context, Event>) =>
  createMachine<Context, Event, State>({
    initial: "first",
    states: {
      first: {
        on: {
          FOO: {
            target: "first",
            actions: [action]
          }
        }
      }
    }
  });

which gives the error
Type '{ FOO: { target: string; actions: Action<Context, Event>[]; }; }' is not assignable to type 'TransitionsConfig<Context, Event> | undefined'.
  Types of property 'FOO' are incompatible.
    Type '{ target: string; actions: Action<Context, Event>[]; }' is not assignable to type 'TransitionConfigOrTarget<Context, { type: "FOO"; }>'.
      Types of property 'actions' are incompatible.
        Type 'Action<Context, Event>[]' is not assignable to type 'Actions<Context, { type: "FOO"; }> | undefined'.
          Type 'Action<Context, Event>[]' is not assignable to type 'Action<Context, { type: "FOO"; }>[]'.
            Type 'Action<Context, Event>' is not assignable to type 'Action<Context, { type: "FOO"; }>'.
              Type 'ActionObject<Context, Event>' is not assignable to type 'Action<Context, { type: "FOO"; }>'.
                Type 'ActionObject<Context, Event>' is not assignable to type 'ActionObject<Context, { type: "FOO"; }>'.
                  Types of property 'exec' are incompatible.
                    Type 'ActionFunction<Context, Event, ActionObject<Context, Event>> | undefined' is not assignable to type 'ActionFunction<Context, { type: "FOO"; }, ActionObject<Context, { type: "FOO"; }>> | undefined'.
                      Type 'ActionFunction<Context, Event, ActionObject<Context, Event>>' is not assignable to type 'ActionFunction<Context, { type: "FOO"; }, ActionObject<Context, { type: "FOO"; }>>'.
                        Types of parameters 'meta' and 'meta' are incompatible.
                          Type 'ActionMeta<Context, { type: "FOO"; }, ActionObject<Context, { type: "FOO"; }>>' is not assignable to type 'ActionMeta<Context, Event, ActionObject<Context, Event>>'.
                            The types of 'action.exec' are incompatible between these types.
                              Type 'ActionFunction<Context, { type: "FOO"; }, ActionObject<Context, { type: "FOO"; }>> | undefined' is not assignable to type 'ActionFunction<Context, Event, ActionObject<Context, Event>> | undefined'.
                                Type 'ActionFunction<Context, { type: "FOO"; }, ActionObject<Context, { type: "FOO"; }>>' is not assignable to type 'ActionFunction<Context, Event, ActionObject<Context, Event>>'.
                                  Types of parameters 'event' and 'event' are incompatible.
                                    Type 'Event' is not assignable to type '{ type: "FOO"; }'.
                                      Type '{ type: "BAR"; }' is not assignable to type '{ type: "FOO"; }'.

Expected Result
No type error
Actual Result
Type error
Reproduction
https://codesandbox.io/s/relaxed-bush-sb326?file=/src/index.ts

Additional context
I think this has something to do with how Typescript checks recursive types. I've created the following example in CodeSandbox, which gives a similar error.
https://codesandbox.io/s/silly-star-lt4c5?file=/src/index.ts

I wanted to chime in again and highlight the part that's weird:
Type 'ActionFunction<{}, { type: "FOO" | "BAR"; }, ActionObject<{}, { type: "FOO" | "BAR"; }>>'
  is not assignable to type
  'ActionFunction<{}, { type: "FOO"; }, ActionObject<{}, { type: "FOO"; }>>'.

because
Type 'ActionFunction<{}, { type: "FOO"; }, ActionObject<{}, { type: "FOO"; }>>'
  is not assignable to type
  'ActionFunction<{}, { type: "FOO" | "BAR"; }, ActionObject<{}, { type: "FOO" | "BAR"; }>>'.

Hey folks, wanted to check in here. This is still an issue and is growing for us.

Hey folks, wanted to check in here. This is still an issue and is growing for us.

Will take a look at this soon, thanks for the bump.

This is a tough one to fix at the moment. The "type/dependency layout" of our structures is incorrect and it manifests in issues like this. I will give this a deeper thought tomorrow but the essence of this issue is that TEvent here is used to configure an event specific to those actions but it spills into State that assumes TEvent is the list of all events accepted by the corresponding machine. In a way, to fix this issue we should add more generics to Action but OTOH I still believe that we should remove state (in its current form) from the meta parameter and that would fix this too. Obviously, the removal can't happen in v4 and we'd need to figure out a replacement for this.

Thanks @davidkpiano and @Andarist.
Internally, we've been discussing the possibility of just migrating everything to string actions, but have hesitated to do so because of the weak typing for string actions. If we can use a restricted set of strings instead of just a generic ActionType, I think we'd have a viable workaround.

Have you seen our typegen? It mostly addresses the weak typings issues

We will be thinking about how to fix this in the upcoming v5. It's a "classic" variance problem - our generics, accidentally, became invariant but conceptually they should be contravariant.


fix package version for release repo

For CentOS 6 platform_version return '6.6' but packages in official repo has release '1-el6' like "icinga2-2.2.3-1.el6.x86_64"
This patch fix this problem.

:+1:


Gpfdist sink updates

Option port to gpfdistPort
Added docs
Support for error table and reject limit
Support for NULL field definition
Generic polish for some of a classes
(more to come in future PR's)
More tests for option handling and integration
testing to verify sql clauses.
Fixes #219
Fixes #171
Fixes #242
Fixes #220

Merged docs, rebased and merged as 07c910860e6129b0bd808dcb1cf09095d2efbf4e


Reclassification

I'm having some problems to reclassify a movie. For example, I open a movie profile, classify it and, after that, remove the classification, when I try to set a new classification it's not happen. The RatingBar loses its click listener when I remove a classification to set a new one.

It was solved during an improvement done in this commit related of this issue #5


1.5.5 breaks BlockScan API

Fixed in the above commit for v1.6.3.  Renumbered releases to better indicate a change in API.


[Scheduled Maintenance] Server Hardware maintenance

Maintenance on system hardware will be taking place during this time.

Completed! All systems should have returned to normal operation.


Refactor upgrade role to not include "specification" at top level

Describe the bug
In upgrade role there are include_vars tasks used without the name parameter what loads variables at top level with higher precedence than task vars.
So having in upgrade role:
- name: Node Exporter | Include specification vars from node_exporter role
  include_vars:
    file: roles/node_exporter/vars/main.yml

causes that specification of node_exporter is global with high precedence.
Then the following task (postgresql upgrade is run after node_exporter) has unexpected content (from node_exporter) in specification var:
- name: repmgr for PG {{ pg_version }} | Replace repmgr config file
  template:
    src: repmgr.conf.j2
    dest: "{{ upgrade_defaults.repmgr.config_dir[ansible_os_family] }}/repmgr.conf"
    owner: postgres
    group: postgres
    mode: u=rw,g=,o=
  vars:
    specification:
      extensions:
        replication:
          replication_user_name: "{{ postgresql_manifest.specification.extensions.replication.replication_user_name }}"
          repmgr_database:       "{{ postgresql_manifest.specification.extensions.replication.repmgr_database }}"

Expected behavior
Task include_vars should be used with name parameter, for example:
- name: Node Exporter | Include specification vars from node_exporter role
  include_vars:
    file: roles/node_exporter/vars/main.yml
    name: node_exporter_vars

then specification would not override task vars but be available under node_exporter_vars.specification.
Additional context
Having globally loaded vars with high precedence may cause difficult to debug errors.
For local vars we can use underscore in names, e.g. _specification.

DoD checklist

[x] Changelog updated (if affected version was released)
[x] COMPONENTS.md updated / doesn't need to be updated
[ ] Automated tests passed (QA pipelines)

[ ] apply
[ ] upgrade

[ ] Case covered by automated test (if possible)
[ ] Idempotency tested
[x] Documentation updated / doesn't need to be updated
[ ] All conversations in PR resolved
[ ] Backport tasks created / doesn't need to be backported

:heavy_check_mark: Tested upgrade with affected components enabled (kafka, node exporter, logging, rabbitmq, repmgr for postgresql).


Shutdown handling/signal example

Hi tokio-cron-schedule team!
Thanks a lot for making this crate!
I am currently trying to handle a graceful shutdown of my application, but I'm having a hard time making it work properly.
The following application never stops, even after catching the Ctrl-C, unless I uncomment the std::process::exit() line. I don't understand why.
Thanks!
use tokio_cron_scheduler::{Job, JobScheduler};

#[tokio::main]
async fn main() {
    let mut sched = JobScheduler::new();

    let _ = sched.add(
        Job::new_async("1/7 * * * * *", |_uuid, _l| {
            Box::pin(async {
                println!("I run async every 7 seconds");
            })
        })
        .unwrap(),
    );

    sched.shutdown_on_ctrl_c();
    let _ = sched.set_shutdown_handler(Box::new(|| {
        Box::pin(async move {
            println!("Shut down done");
            // std::process.exit(0); // This is needed for the application to shutdown
        })
    }));

    let _ = sched.start().await;
}

Hi @fmorency
Thanks for the report. I'll check into it

yeah this is hitting me as well

I ran into this also. The shutdown mechanism seems to block (in my debugging, it seemed to be related to not properly starting all its tokio tasks). I fixed it on my fork by simplifying the shutdown method and making it async, but I imagine that doesn't meet the design you're going for:
https://github.com/tuck182/tokio-cron-scheduler/commit/10ac05fe7e6515a782ba74aac438c10de25f8e4a

Hey @tuck182 that'll work, please open a PR?

So, could this be fixed?

I've been meaning to create a PR; I'll do that this weekend


Day79: LeetCode-1347. Minimum Number of Steps to Make Two Strings Anagram - HashTable

Minimum Number of Steps to Make Two Strings Anagram
https://leetcode.com/problems/minimum-number-of-steps-to-make-two-strings-anagram/
You are given two strings of the same length s and t. In one step you can choose any character of t and replace it with another character.
Return the minimum number of steps to make t an anagram of s.
An Anagram of a string is a string that contains the same characters with a different (or the same) ordering.

Example 1:
Input: s = "bab", t = "aba"
Output: 1
Explanation: Replace the first 'a' in t with b, t = "bba" which is anagram of s.
Example 2:
Input: s = "leetcode", t = "practice"
Output: 5
Explanation: Replace 'p', 'r', 'a', 'i' and 'c' from t with proper characters to make t anagram of s.
Example 3:
Input: s = "anagram", t = "mangaar"
Output: 0
Explanation: "anagram" and "mangaar" are anagrams.


[Feature] Document aspectRatio contraints

Document the aspectRatio contraints wrt how and when it applies under the cldimage/config page
Per https://github.com/cloudinary-community/next-cloudinary/issues/352#issuecomment-2115636574

hey @\antoinerousseau as it stands, and i think i need to update the documentation to reflect this, aspectRatio doesn't get applied unless combined with a crop mode, for instance:
<CldImage
  src="examples/mario_hkgavf"
  width={2048}
  height={1600}
  aspectRatio="16:9"
  alt="mario"
  crop="fill"
/>

will give you:
https://res.cloudinary.com/colbycloud-examples/image/upload/c_fill,ar_16:9,w_3840,h_3000,g_auto/f_auto/q_auto/v1/examples/mario_hkgavf?_a=BAVFAaAO0
however that's not very useful because the width and the height prevent the aspect ratio from applying as a transformation
where aspectRatio really makes sense is when using the fill prop, such as:
<CldImage
  src="examples/mario_hkgavf"
  aspectRatio="16:9"
  fill
  alt="mario"
  crop="fill"
/>

which would give you this:
https://res.cloudinary.com/colbycloud-examples/image/upload/c_fill,ar_16:9,w_2048,g_auto/f_auto/q_auto/v1/examples/mario_hkgavf?_a=BAVFAaAO0
the issue is that width and height are required props for the component. this is a Next Image component requirement that's passed along to this component and it's required to prevent layout shift when applied to the <img tag in the DOM
but thats where the fill option comes in, which doesn't require a width and height, but then you control the size of the parent container which the image then "fills". this is a good use case for the aspectRatio prop in that regard
i'm open to feedback and ideas on this, as it's not outright intuitive with how that works. i'll create a ticket to update the docs to make this more clear to start

@colbyfayock  where we need to add the document for aspectRatio , your explanation was on point it explains it very well

seems like we may want to add aspectRatio to https://next.cloudinary.dev/cldimage/configuration#basic-transformations
create a new description for it similar to angle in that list
then add it there

when will hacktoberfest tags be added to issues

@RaghavMangla the tag is on the entire repository, so any Issue marked with "help wanted" is game

Hi @colbyfayock , I would like to work on this issue. Please assign it to me.

sure thing, all yours!

@colbyfayock
I have made the changes in the docs and raised a PR. Please review.
Thanks!


Dropdown height large scale text fix

This PR fixes the problem where large scale text gets cut off in the dropdown button. The current calculation of height of the button only uses fontSize, which does not reflect the actual size of the text when a text scale factor of more than 1.0 is specified.
To fix this problem, I scaled the fontSize according to text scale, before using this new text size to calculate the button height.
Fixes https://github.com/flutter/flutter/issues/106359

use height to fix the error here

Thanks for the comment, could you elaborate a little more about what you mean by "use height to fix the error here" ? My fix involves fixing the way the button height is calculated.

@foongsq This looks great. If you are able to accept the Contributor License Agreement (CLA), we can move forward with reviewing it.
Review the failure for more instructions on how to accept the CLA.

@gspencergoog I have already accepted the CLA, thanks

Manually merging as discussed in the Flutter rollers chat.


PEP 8

Hi, there.
I try to improve the code by applying PEP 8.
Thanks.

Thank you very much! I started this as a personal project to learn python, and just used github to host my code. I didn't think anyone would actually be looking at the code, but I guess that's not the case. I'll merge your suggestions now!

Haha :+1:

Hi, I'm interested to try your code but i do not know how. May i ask for instructions? I do not know python or what :) I'm interested in getting information from facebook's messages.htm to get old messages haha
Thank You

@ervinpython Hi :)
By the way, It did not contribute to make this code.
I just tried to improve this code.
You must go to @CopOnTheRun to obtain the help.

Hey @ervinpython I was actually thinking about updating the README.md to provide some basic instructions as to how to use the program. Also I may make a command line interface because I want to practice with the argparse module, but that may have to wait for a bit. I'll see if I can update the README.md with instructions this weekend or so. If you wish to continue conversation on this topic, please start a new issue as this thread has diverged from it's original purpose :) Thanks!


Fine-tuning on an uncolored labeled dataset

Hello,
Thanks for your amazing work!
I have a custom uncolored labeled dataset, with precise labels for the instances.
It seems that it does not really fit the SAM pipeline, where the SAM segmentation does not bring any value... Do you have any guideline for fine tuning ? I thought to totally change the pipeline and introduce directly the labels, modifying the loss for example.
Thanks for any tip!

I am not the author, but maybe you could use normal map for segmentation?

Hi, thanks for your interest in our work!
Perhaps you could directly use this data to train a 3D network by adopting SAM's training method and loss configuration?
Or conditioned on a 3D scale value?

For our design, we directly set the color value (after rgb augmentation) to (0,0,0) for uncolored data.


refactor(all): modify motor hardware to be templated rather than use virtual functions

Overview
Closes RET-1313 and RET-1316.
TODO

Still need to fully fix the simulators

Test plan

[ ]  Test that booting the system doesn't cause the 96 channel to drop anymore

[ ] Run a full protocol with all of the boards flashed to this branch

Closing in favor of simply creating an extended class for the head board 😿


RPM fails to install in Centos 7 / rpm ver. 4.11.1

I have been trying to install the produced RPMs in various systems and it appears that for rpm version <=4.8 all is good but in Centos 7 we have rpm version 4.11.1 which complains as follows:
rpm -ivh test.rpm
error: test.rpm: headerRead failed: hdr load: BAD
error: test.rpm cannot be installed
rpm --version
RPM version 4.11.1

Anything progress with this?

I investigated a little and compared a meta-rpm built with Redline with another one I found online (that works).
Seems like Redline adds a bunch of empty headers when there are no files and therefore the rpm is viewed as corrupt by rpm >4.8.
Two outputs using the Redline Scanner:
Working meta package:
rpmversion[tag=1064,type=6,count=1,size=6,offset=793]
4.8.1
payloadformat[tag=1124,type=6,count=1,size=5,offset=6324]
cpio
payloadcompressor[tag=1125,type=6,count=1,size=3,offset=6329]
xz
payloadflags[tag=1126,type=6,count=1,size=2,offset=6332]
2
Non-working meta package (Redline):
rpmversion[tag=1064,type=6,count=1,size=6,offset=293]
4.4.2
filesizes[tag=1028,type=4,count=0,size=0,offset=140]
filemodes[tag=1030,type=3,count=0,size=0,offset=140]
filerdevs[tag=1033,type=3,count=0,size=0,offset=140]
filemtimes[tag=1034,type=4,count=0,size=0,offset=140]
filemd5s[tag=1035,type=8,count=0,size=0,offset=140]
filelinktos[tag=1036,type=8,count=0,size=0,offset=140]
fileflags[tag=1037,type=4,count=0,size=0,offset=140]
fileusername[tag=1039,type=8,count=0,size=0,offset=140]
filegroupname[tag=1040,type=8,count=0,size=0,offset=140]
fileverifyflags[tag=1045,type=4,count=0,size=0,offset=140]
filedevices[tag=1095,type=4,count=0,size=0,offset=300]
fileinodes[tag=1096,type=4,count=0,size=0,offset=300]
filelangs[tag=1097,type=8,count=0,size=0,offset=300]
dirindexes[tag=1116,type=4,count=0,size=0,offset=324]
basenames[tag=1117,type=8,count=0,size=0,offset=324]
dirnames[tag=1118,type=8,count=0,size=0,offset=324]
filecontexts[tag=1147,type=8,count=0,size=0,offset=356]
payloadformat[tag=1124,type=6,count=1,size=5,offset=324]
cpio
payloadcompressor[tag=1125,type=6,count=1,size=5,offset=329]
gzip
payloadflags[tag=1126,type=6,count=1,size=2,offset=334]
9

Thanks for the new info.  The suggested quick fix (null at the end of empty string arrays) did not fix the issue by itself, so it seems we should be suppressing the file headers for a metapackage as well.


关于块同步验证效率加速的建议

观察下来，如果cli不联网（acc文件加速），是cli联网效率的很多倍，是否可以增加启动参数比如“--nonet”，让cli不检测新块，只快速处理已下载的块。联网情况下，经常遇到下载的块数据高度比验证的高度多10w+，而acc加速（断网）只有几十个块的差距。

Observe that if cli is not networked (acc file acceleration), it is a multiple of cli's networking efficiency. Whether it can increase the startup parameters such as "--nonet", so that cli does not detect new blocks, only the downloaded blocks are processed quickly. In the case of networking, it is often encountered that the downloaded block data height is 10W+ more than the verification height, while the acc acceleration (breaking network) has only a gap of several tens of blocks.
(from google translate)

I'm closing this due to lack of activity.
If you think this was a mistake, please create a new issue. Thanks


Add rubocop-packaging as a known extension in the docs

This PR adds another known extension, rubocop-packaging to the extension docs.
And lastly, thanks @bbatsov, for this wonderful tool! :heart:
Signed-off-by: Utkarsh Gupta<EMAIL_ADDRESS>
Before submitting the PR make sure the following are checked:

[x] Wrote good commit messages.
[x] Feature branch is up-to-date with master (if not - rebase it).
[x] Squashed related commits together.
[x] Added an entry to the Changelog if the new code introduces user-observable changes. See changelog entry format.
[x] The PR relates to only one subject with a clear title and description in grammatically correct, complete sentences.
[x] Run bundle exec rake default. It executes all tests and RuboCop for itself, and generates the documentation.

Cool package! I've always been fond of Debian! :-)

And lastly, thanks @bbatsov, for this wonderful tool! ❤

You're welcome! 🙇♂

Cool package! I've always been fond of Debian! :-)

Awesome, thanks! :smile:

Hi @bbatsov 👋🏻
Once again, thanks for all your work so far! ❤
I have got a small request to make..
Based on rspec-style-guide, minitest-style-guide, rails-style-guide, and so on, I've created packaging-style-guide, here:
https://github.com/utkarsh2102/packaging-style-guide
(it follows the same style like other guides)
Would you be kind enough to give a little space to host it at rubystyle.guide? So like https://{rspec, minitest, rails}.rubystyle.guide, there'll be https://packaging.rubystyle.guide, too?
Please let me know if there's something more required at my end to make this happen?
It'd really mean a lot to us (the Debian core devs and other d/s maintainers) ❤

I think you'll have to transfer this guide to the RuboCop HQ org (you'll still be it's owner), so I can deploy the HTML to the org's GitHub pages like the other guides.

Aha, okay. As long as I can keep maintaining this, this won't be a problem!
However, whilst trying to transfer this to the RuboCop HQ org, I get this:
 You don’t have the permission to create public repositories on rubocop-hq
How do I proceed from here?

I've sent you an invite to join the org. You'll be able to transfer the repo after you accept it.

Many thanks. The transfer is now done! 🎉

@utkarsh2102 See http://packaging.rubystyle.guide :-)

@bbatsov, you're the best, thank you! ❤


test norm enum pr for first ga package

API change check
APIView has identified API level changes in this PR and created following API reviews.
@azure/arm-computefleet


Single Command for Values and Certificate Configuration

As Is
Currently separate commands (merge config and merge certs) exists to merge/configure values and certificates.
To Be
Provide single command to merge/configure values and certificates in one step.

Single config command combines the lint, merge config and merge certs commands.


Add functionality to update individual transactions

This request adds a new definition in monarchmoney.py which enables updating of a single transaction.  I tested all parameters from the web calls to see which are writeable by the user.  These writable parameters are what are in the definition call.  Testing was done on both a manually created transaction and one source from a live account.
I was surprised to see that the merchant value was updated by using the "name" of the merchant (e.g. Amazon) as opposed to the id.  It is quite possible that someone with better gql skills could move this to use the merchantId for cleaner implementation.
Sample call:
mm.update_transaction(
            transaction_id="160820461792094418",
            category_id="160185840107743863",
            merchant_name="Amazon",
            goal_id="160826408575920275",
            amount=123.45,
            date="2023-11-09",
            hide_from_reports=False,
            needs_review=False,
            notes=f'Updated On: {datetime.now().strftime("%m/%d/%Y %H:%M:%S")}',
        )

Finally, I am new to making PRs, so please let me know if there are basic things that I have missed.

Updates made per your request @hammem.  Thanks for letting us participate.

@hammem Okay, I gave it my best shot.  I think what is complicating things is the lack of robust error protection (and I am not suggesting we do that at this stage).  For example, I could remove a lot of the None checks and let the API respond with any problems.  However, imo the current code has an acceptable amount of protection and avoids the most common pitfalls.
If you find everything acceptable, I believe you can squash the huge number of commits I have made.  If you need me to do anything on my side please let me know.

@hammem Let me know if you have any questions about this.

@KOEPKEJR Nice work thus far and thanks for contributing. Friendly ping on the status of this PR. Thanks
Background. I am looking to port my project from Mint over to Monarch Money. In addition to needing the ability to edit transactions, I necessarily need the ability to split/unsplit transactions. Once editing is in place, I'd be happy to look at splits (or would be happy to follow along if @KOEPKEJR is interested).
Cheers!

Careful! They are going to take away our access (joking…kind of). On Jan 2, 2024, at 09:48, Jeff Prouty @.***> wrote:
@jprouty commented on this pull request.
In monarchmoney/monarchmoney.py:

@@ -880,6 +880,168 @@ async def get_cashflow_summary(
operation="Web_GetCashFlowPage", variables=variables, graphql_query=query
)

async def update_transaction(

   self,

   transaction_id: str,

   category_id: Optional[str] = None,

   merchant_name: Optional[str] = None,

From within the web app, I attempted to test the length limits for merchant_name and note. I gave up after getting to 10k+ characters for both! I don't know if it's worth searching further (for the sake of documentation - or adding a guard) but just writing this as an FYI.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>

Haha, well the good news is my tests were all performed on the web app,
just as any ordinary user might do (and I haven't used the API yet). So
there's no indication of faul play, yet 😄
On Tue, Jan 2, 2024, 8:07 AM KOEPKEJR @.***> wrote:

Careful! They are going to take away our access (joking…kind of). On Jan
2, 2024, at 09:48, Jeff Prouty @.***> wrote:
@jprouty commented on this pull request.
In monarchmoney/monarchmoney.py:

@@ -880,6 +880,168 @@ async def get_cashflow_summary(
operation="Web_GetCashFlowPage", variables=variables, graphql_query=query
)

async def update_transaction(
self,
transaction_id: str,
category_id: Optional[str] = None,
merchant_name: Optional[str] = None,

From within the web app, I attempted to test the length limits for
merchant_name and note. I gave up after getting to 10k+ characters for
both! I don't know if it's worth searching further (for the sake of
documentation - or adding a guard) but just writing this as an FYI.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are
receiving this because you were mentioned.Message ID: @.***>
—
Reply to this email directly, view it on GitHub
https://github.com/hammem/monarchmoney/pull/31#issuecomment-1874223013,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAMS2VC3EHLOI43BWI5UWDYMQWENAVCNFSM6AAAAAA7G7WP66VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNZUGIZDGMBRGM
.
You are receiving this because you were mentioned.Message ID:
@.***>


Prepare Application for UAT

Need to:

Set up fake Strava account + populate with activities.
Set up fake Spotify account + populate with 50 songs.
Set up any other accounts for testing if more data sources are added eventually.
Publish app

UAT in progress


Add rescure_domain_blacklist.txt to DnsCrypt-proxy

@mimugmail I can create a PR for this blocklist if you like? https://rescure.fruxlabs.com/rescure_domain_blacklist.txt
Just want to run it by you first just to make sure it would work before I did the work of adding it, thanks

Sure, just add a PR. It's a bit small, but if it helps, why not 👍


Website tutorials as downloadable notebooks

Currently, it is not easy for users to run the tutorials themselves such as https://auto.gluon.ai/stable/tutorials/tabular_prediction/tabular-quickstart.html
We should provide an option for the user to download the python notebook itself.

Resolved at present with the website refactor + Open in Colab button


Invoke-RestMethod / Invoke-WebRequest do not honor installed CA certificates on Ubuntu 16.04 with Linux Subsystem

It does not appear that the PowerShell Core on Ubuntu 16.04, installed on the Linux Subsystem for Windows 10, honors installed CA certificates.
Steps to reproduce
Attempt to connect to internal resource, fails due to invalid certificate chain/validation:
PS /home/cbarton> Invoke-RestMethod -Method 'GET' -Uri 'https://icusuite.somedomain.net'
Invoke-RestMethod : The remote certificate is invalid according to the validation procedure.
At line:1 char:1
+ Invoke-RestMethod -Method 'GET' -Uri 'https://icusuite.somedomain.net'
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : InvalidOperation: (Method: GET, Re...rShell/6.1.1
}:HttpRequestMessage) [Invoke-RestMethod], HttpRequestException
+ FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand

Make same call with the Ubuntu subsystem's curl utility, successfully connects using the subsystem's CA certificates:
PS /home/cbarton> curl https://icusuite.somedomain.net
<!DOCTYPE html>

        <meta charset="UTF-8">
        <title>ICU Suite</title>

Expected Result
PowerShell should honor the underlying system's trusted SSL relationships, and/or provide a clear method for specifying which certificates should be used to evaluate trust.

@Christophoclese Thanks for your report!
PowerShell Core uses .Net Core HttpClient. I think the issue should be moved to CoreFX repository.

Please include the version of PowerShell Core that you used to repro the problem?  There were significant changes between 6.0 and 6.1

But, @iSazonov 	 is probably correct.

PS /home/cbarton> $PSVersionTable

Name                           Value
----                           -----
PSVersion                      6.1.1
PSEdition                      Core
GitCommitId                    6.1.1
OS                             Linux 4.4.0-43-Microsoft #1-Microsoft Wed Dec 31 14:42:53 PST 2014
Platform                       Unix
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
PSRemotingProtocolVersion      2.3
SerializationVersion           <IP_ADDRESS>
WSManStackVersion              3.0

@Christophoclese Version 6.1 doesn't use curl. You could make a test with PowerShell Core 6.0 and get confirmation that it works.
@markekraus Could you please look the issue and confirm that it is CoreFX issue?

Results with PowerShell 6.0.5 -- still does not work, but with a much more ambiguous error message:
cbarton@cbarton-181026:~$ pwsh
PowerShell v6.0.5
Copyright (c) Microsoft Corporation. All rights reserved.

https://aka.ms/pscore6-docs
Type 'help' to get help.

PS /home/cbarton> $PSVersionTable

Name                           Value
----                           -----
PSVersion                      6.0.5
PSEdition                      Core
GitCommitId                    v6.0.5
OS                             Linux 4.4.0-43-Microsoft #1-Microsoft Wed Dec 31 14:42:53 PST 2014
Platform                       Unix
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
PSRemotingProtocolVersion      2.3
SerializationVersion           <IP_ADDRESS>
WSManStackVersion              3.0

PS /home/cbarton> Invoke-RestMethod -Method 'GET' -Uri 'https://icusuite.somedomain.net'
Invoke-RestMethod : SSL connect error
At line:1 char:1
+ Invoke-RestMethod -Method 'GET' -Uri 'https://icusuite.somedomain.net'
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : InvalidOperation: (Method: GET, Re...rShell/6.0.5
}:HttpRequestMessage) [Invoke-RestMethod], HttpRequestException
+ FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand
PS /home/cbarton> curl https://icusuite.somedomain.net
<!DOCTYPE html>

        <meta charset="UTF-8">
        <title>ICU Suite</title>

Is this a certificate you added or built into ca-certificates?  Would it be possible to give an actual domain we can verify against.  There is a variable in DotNet 2.1 (which we use in PowerShell Core 6.1) to force it to use libcurl.


when adding a new setting on the backend and changing it through the API, it doesn't get inserted into the settings document

example:

added textWrap setting, default to true
through the api set textWrap to false
setting is still missing in the settings document on the user

might not be an issue anymore


Random Crash Fatal Exception: java.lang.UnsupportedOperationException: JsonObject

What is seen in app
During development itself app would start and immediately crash. This is not always the behavior, at times app would start and then later randomly crash again. On looking more closely with Android Studio's logcat we are able to find the same error as listed on Firebase Crashlytics. The most weird thing is, this crash is only noticed in the application from around 8.30am morning to 7.00pm in evening.
Our use case involves using Channels to pass events from a backend to mobile application. It'll be of great help if someone can point towards a solution or possible causes triggering such an issue.
Can we share code-base
Unfortunately not
Exception
Fatal Exception: java.lang.UnsupportedOperationException: JsonObject
       at com.google.gson.JsonElement.getAsString(JsonElement.java:179)
       at com.pusher.client.channel.PusherEvent.fromJson(PusherEvent.java:88)
       at com.pusher.client.connection.websocket.WebSocketConnection.lambda$onMessage$5$WebSocketConnection(WebSocketConnection.java:212)
       at com.pusher.client.connection.websocket.-$$Lambda$WebSocketConnection$LH620kMWP6e8WFdl-2ZQYqgldrg.run(:4)
       at com.pusher.client.util.Factory.lambda$queueOnEventThread$0(Factory.java:140)
       at com.pusher.client.util.-$$Lambda$Factory$zJWSbSBBJ9pXjI_LpAkJ-3_Kedw.run(:2)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
       at java.lang.Thread.run(Thread.java:920)

On looking into the issue a bit more yesterday, I found the error was more prominent on version of this lib that were using the latest pusher-java-client 2.4.1-2 (in build gradle you'll find pusher-java-client:2.+ or something like that).
For our use case we were only using public channels not private channels but this issue still surfaced. So we removed the React Native lib and moved to pusher-java-client 2.2.1 and worked with bridge to create connections and subscriptions to channels. It has been working since yesterday, I am not sure if strictly defining pusher-java-client 2.2.1 in this lib's gradle using patch-package can help fix this issue but worth a try.

This is an issue in the 2.4.1 version of the java lib, we're working on a fix

pusher-websocket-java 2.4.1 was release about a month ago, are any of you still getting the error?

I have not tried 2.4.1 again @benjamin-tang-pusher
We have shifted to this https://github.com/pusher/pusher-websocket-react-native/issues/33#issuecomment-1253188988

@barsumek How about you? I've tried repeating your steps but mine is able to connect to a private channel. The build.gradle in node_modules/pusher-websocket-react-native will get api com.pusher:pusher-java-client:2.+ which should resolve to the latest 2.4.2.

Due to lack of response, and the changes 2.4.2 version of pusher-websocket-java that likely fixed this, I'm going to close this thread.


Request to add a "virtual keyboard" example

Sorry, I'm new to rust and there are many things I don't know in depth, so this topic may be very abrupt.
I have encountered some problems with the "virtual keyboard".
For example, showing + top window + inactivity (not stealing input focus) is the most important feature of the special window that is the virtual keyboard.
For example, clicking on a button to send a keystroke also does not activate the window.
I didn't find a solution to any of these problems in the documentation or in the example, so if possible I would ask to add an example of a virtual keyboard (even if it has only one keyboard button). Thank you very much.
The "iced" examples are very good and I learned a lot. In the "bezier_tool" I have a small comment, could it be transformed into a brush tool? I think the "brush or scribble" feature could be more widely used, and more people could modify their own applications directly from a brush example.

this would be useful for me, albeit for different reasons - iced seems to work fully on iOS, but there's no keyboard input whenever selecting a text input or something.


Google Titan USB Keys not being recognized

Hi all,
Not sure if this is a known/solved issue, but my Google Titan USB keys are not being recognized when using the list_devices() options when trying to check ctap or hid. Am I missing something simple here, or is this simply not supported?
When I was trying to debug and go through each device, none of the page information matched 0xF1D0.
Edit: Additional info:
Both USB A and USB c keys are not being recognized on Windows 10 using both front-panel IO and back-panel IO. They are, however, recognized inside of device manager as fido compliant devices.
Thanks in advance

Additional info:

Both USB A and USB c keys are not being recognized on Windows 10 using both front-panel IO and back-panel IO. They are, however, recognized inside of device manager as fido compliant devices.

After checking the commit 7376b989a4008955cdd11926b3c22279c7b5481d, the latest update to README specifies that you must run as administrator and that there's an example under credential.py.


VSTS agent cannot access Azure CLI

Agent Version and Platform
2.188.2 windows
OS: windows (Self hosted)
What's not working?
From raw logs:
Using Azure CLI, trying to execute the powershell script, this is what I get:

On the agent's VM, I have installed the cli when logged in as my user account,

The vsts agent is running as a network service

The same thing when ran through the Microsoft hosted agent works as expected. What am I missing here?

Hello @AmitShewani , did you try to restart the service after you had installed Azure CLI?

@tatyana-kostromskaya , yes I did restart the service and restarted the server as well but it still didn't work.

@tatyana-kostromskaya , I reinstalled the agent, verified that the system variables exist and restarted the machine. Still getting the same error.


Don't let modules loose

Rule

every module is referenced in Erlang.mk or rebar.config or Emakefile

@elbrujohalcon I'm not sure what is the usefulness of this rule.

@jfacorro you're right


Migrate maintainer to transaction provider

Maintainer still uses ristretto

Add MaxLockTime in the configuration

Fixing the tests is part of this task


URL redirection via Namecheap doesn't work well

No https->https
Try to handle it in remix instead

Was closed by #152


[bug]:

Hi, I am running a node LND 15.4 on a Raspberry 4b 8GB HDD 2TB.
I believe everything works fine but while looking at the log file I noticed this error:
2022-11-19 19:47:09.501 [INF] CRTR: Processed channels=1 updates=115 nodes=4 in last 59.999796477s
2022-11-19 19:47:25.363 [INF] WTCL: (legacy) Client stats: tasks(received=0 accepted=0 ineligible=0) sessions(acquired=0 exhausted=0)
2022-11-19 19:47:25.438 [INF] WTCL: (anchor) Client stats: tasks(received=0 accepted=0 ineligible=0) sessions(acquired=0 exhausted=0)
2022-11-19 19:48:09.502 [INF] CRTR: Processed channels=0 updates=119 nodes=11 in last 1m0.000624001s
2022-11-19 19:48:25.363 [INF] WTCL: (legacy) Client stats: tasks(received=0 accepted=0 ineligible=0) sessions(acquired=0 exhausted=0)
2022-11-19 19:48:25.438 [INF] WTCL: (anchor) Client stats: tasks(received=0 accepted=0 ineligible=0) sessions(acquired=0 exhausted=0)
2022-11-19 19:48:25.678 [INF] DISC: Broadcasting 169 new announcements in 17 sub batches
2022-11-19 19:48:29.079 [ERR] DISC: unable to validate channel update announcement for short_chan_id=(lnwire.ShortChannelID) 759405:2134:1
: invalid max htlc for channel update (*lnwire.ChannelUpdate)(0x4006484a50)({
 Signature: (lnwire.Sig) (len=64 cap=64) {
  00000000  24 d6 96 f4 46 e7 02 03  b8 59 a2 f0 3f 9f 39 9f  |$...F....Y..?.9.|
  00000010  fa 57 e1 3f 14 9a ad 50  19 e6 bb 91 f8 1e 72 b3  |.W.?...P......r.|
  00000020  15 1b 60 59 6b 0a d5 36  66 03 f4 28 9b f0 61 f8  |..`Yk..6f..(..a.|
  00000030  b1 ad 93 29 61 bf df 8f  94 63 8b 47 ab 6b f4 c7  |...)a....c.G.k..|
 },
 ChainHash: (chainhash.Hash) (len=32 cap=32)<PHONE_NUMBER>19d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f,
 ShortChannelID: (lnwire.ShortChannelID) 759405:2134:1,
 Timestamp: (uint32)<PHONE_NUMBER>,
 MessageFlags: (lnwire.ChanUpdateMsgFlags) 00000001,
 ChannelFlags: (lnwire.ChanUpdateChanFlags) 00000000,
 TimeLockDelta: (uint16) 34,
 HtlcMinimumMsat: (lnwire.MilliSatoshi) 1 mSAT,
 BaseFee: (uint32) 0,
 FeeRate: (uint32) 833,
 HtlcMaximumMsat: (lnwire.MilliSatoshi) 0 mSAT,
 ExtraOpaqueData: (lnwire.ExtraOpaqueData) {
 }
})

2022-11-19 19:49:09.502 [INF] CRTR: Processed channels=0 updates=117 nodes=9 in last 59.99954211s
2022-11-19 19:49:25.363 [INF] WTCL: (legacy) Client stats: tasks(received=0 accepted=0 ineligible=0) sessions(acquired=0 exhausted=0)
2022-11-19 19:49:25.437 [INF] WTCL: (anchor) Client stats: tasks(received=0 accepted=0 ineligible=0) sessions(acquired=0 exhausted=0)
2022-11-19 19:49:55.679 [INF] DISC: Broadcasting 233 new announcements in 18 sub batches
2022-11-19 19:50:09.506 [INF] CRTR: Processed channels=1 updates=148 nodes=6 in last 1m0.003594617s

Could someone please explain to me?
Thank you.
Kind regards,

I don't think this is not a bug or unintended behaviour. Your processing a channel update announcement with an invalid configuration. This has no implications for your own node.
Moving this to a discussion.

Thank you my friend 👍


⚠ Catalog API Endpoint has degraded performance

In 74af0de, Catalog API Endpoint (https://catalog.roblox.com/v1/bundles/details?bundleIds=192) experienced degraded performance:

HTTP code: 200
Response time: 6425 ms

Resolved: Catalog API Endpoint performance has improved in c57424a.


Can't send numeric values through the Zabbix output

from https://logstash.jira.com/browse/LOGSTASH-2223
In Logstash 1.4.1 and Zabbix 2.2, exceptions are thrown when I try to send a number to Zabbix. For example, I set the send_field value to "some-metric.count".
Logstash throws an error:
{code}
{
:timestamp=>"2014-05-24T04:59:12.348000-0500",
:message=>"Error during receiving message for sending",
:event=>#<LogStash::Event:0x3f6600b8 ...>,
:exception=>#<NoMethodError: undefined method empty?' for 17:Fixnum>,  :backtrace=>[ 	"file:/opt/logstash/vendor/jar/jruby-complete-1.7.11.jar!/META-INF/jruby.home/lib/ruby/1.9/shellwords.rb:75:in shellescape'",
"/opt/logstash/lib/logstash/outputs/zabbix.rb:115:in receive'",  	"org/jruby/RubyArray.java:1613:in each'",
"org/jruby/RubyEnumerable.java:977:in each_with_index'",  	"/opt/logstash/lib/logstash/outputs/zabbix.rb:112:in receive'",
"/opt/logstash/lib/logstash/outputs/base.rb:86:in handle'",  	"(eval):456:in initialize'",
"org/jruby/RubyProc.java:271:in call'",  	"/opt/logstash/lib/logstash/pipeline.rb:266:in output'",
"/opt/logstash/lib/logstash/pipeline.rb:225:in outputworker'",  	"/opt/logstash/lib/logstash/pipeline.rb:152:in start_outputs'"
],
:level=>:warn
}
{code}
Am I doing something wrong, or is sending numbers to Zabbix just not supported?

Zabbix receives numeric strings and converts internally

This will be addressed by https://github.com/logstash-plugins/logstash-output-zabbix/pull/2

This issue should be resolved.  Re-open if necessary.


[question] Hosting on vercel

I'm very new to self-hosting in general, and my knowledge is limited to what I was able to do with littlelink
Can someone perhaps guide me in the right direction?

we need backend support, so you should have your own server to host memos


Add public traffic configuration for admin server

-- Add nsgNestedTemplate.json to append NSG rules for admin server
-- Update mainTemplate.json to adopt NSG rules configurations
-- Add UI for the configurations
-- Remove legacy sample code
-- Update .gitignore to ignore .classpath file
-- Fix code format

@edburns Hi Ed, could you please review this? I looks a bit messy because I corrected a lot of format problems as I went through the code.

@edburns Hi Ed, could you please review this? I looks a bit messy because I corrected a lot of format problems as I went through the code.


Document planning cadence

document planning cadence
update PR-FAQ guidelines, wrap old page under cleaned up page
document customer-facing roadmap sequence

Merging this.
Please open follow-up PRs with feedback.
@kalanchan is working on figuring out an actual calendar for Q2, using this as the baseline but making adjustments so as not to collide with reviews and Starship.


test-branch/execute-job/comment-from-wrong-user-on-pr/dc9c28aa-0dfe-4ea5-b537-6c9b8712d346

PR for testing 454f88f4aaaac8bf5f33f1602f9217734a6fff57

/canonical/self-hosted-runners/run-workflows 454f88f4aaaac8bf5f33f1602f9217734a6fff57


Bump build number; fix license type

Nothing has changed upstream for v2.0.1 since its release (Dec 11, 2020). Rebuilding (with bump of the build number) because we are no longer supporting noarch builds; rebuilding to be arch-specific.
Changelog: https://github.com/python-excel/xlrd/blob/master/CHANGELOG.rst#201-11-december-2020
License: https://github.com/python-excel/xlrd/blob/master/LICENSE
Requirements:
https://github.com/python-excel/xlrd/blob/master/setup.py#L39
Actions:

Remove noarch python
Updated the license type to be SPDX-compliant
Update host deps to include setuptools and wheel

Closing this because the noarch build of xlrd-2.0.1 (build num 0) should meet our needs for now.


added options to cask

addressing issue #68

Hi. I am an automated pull request bot named Curry. There are commits in this pull request whose authors are not yet authorized to contribute to Chef Software, Inc. projects or are using a non-GitHub verified email address. To become authorized to contribute, you will need to sign the Contributor License Agreement (CLA) as an individual or on behalf of your company. You can read more on Chef's blog.
GitHub Users Who Are Not Authorized To Contribute
The following GitHub users do not appear to have signed a CLA:

@bcg62

Please sign the CLA here.

Hi. Your friendly Curry bot here. Just letting you know that all commit authors have become authorized to contribute. I have added the "Signed CLA" label to this issue so it can easily be found in the future.


update documentation on website

Description
while I was working with Guzzle, I come through the documentation ,
http://docs.guzzlephp.org/en/stable/request-options.html#body
and when sending Body in the param array i got this exception message .

InvalidArgumentException Passing in the "body" request option as an array to send a POST request has been deprecated. Please use the "form_params" request option to send a application/x-www-form-urlencoded request, or the "multipart" request option to send a multipart/form-data request.

the problem fixed when i replace body with form_params
Example
the following code through the above  exception
$response = $client->request('GET', $url, ['headers' => [
            'content-type' => 'application/x-www-form-urlencoded'
        ],
        'body' => [
            'grant_type' => 'XXX',
            'username' => 'XXXX',
            'password' => 'XXXX'
        ]
    ]);

this code will not through the exception
$response = $client->request('GET', $url, ['headers' => [
            'content-type' => 'application/x-www-form-urlencoded'
        ],
        'form_params' => [
            'grant_type' => 'XXXX',
            'username' => 'XXXX',
            'password' => 'XXXX'
        ]
    ]);

advice
please update the docs.

Unless I'm missing something, the accepted types shown on the docs are:
string
fopen() resource
Psr\Http\Message\StreamInterface

Which is correct?
I can't see anything stating that array is supported.

@corexz this is the expected functionality and as far as I can tell, it's documented properly.
http://docs.guzzlephp.org/en/stable/request-options.html#body
http://docs.guzzlephp.org/en/stable/request-options.html#form-params


Concurrent requests with number values in the filter fail from time to time

Describe the bug
We noticed that from time to time, our tests that call Tigris fail when we run them as usual (in parallel), but pass if we set -parallel=1. After some investigation, I was able to reproduce this behavior with a smaller data set.
Here I have an example where I set "balance": 5e-40 and then search for it. Everything works as expected. I tried running it multiple times, and it always finds and returns the expected result.
Here I have an example where I also set "balance": 5e-40, but I do some parallel requests to the collections. In this case, Tigris can’t find the records for the filter with  "balance": 5e-40. However, if I set the filter as"balance": 5e-20., everything works as expected, and I can find it even if I send parallel requests.
To Reproduce
Steps to reproduce the behavior:
Run https://github.com/FerretDB/tigris-experiments/blob/parallel-requests/main.go, it will not find the record for the filter "balance": 5e-40.
Expected behavior
It is possible to find such records even if there is a number of parallel requests sent.
Desktop (please complete the following information):

OS: macOS 13.2
Server: tigrisdata/tigris-local:1.0.0-beta.36 (docker image), driver: github.com/tigrisdata/tigris-client-go v1.0.0-beta.22.

This is not related to concurrency but the floating point handling when the value can’t fit in 32-bit. The indexing of floating numbers was parsing the floating values in float(32bit) which means there were two problems:

The smallest floating point normal number needed to be less than ~ 1.1755e-38
The number has to be in the range -3.4e38/3.4e38

Outside of this range, an error was getting swallowed, resulting in empty documents returned for that query.
I have fixed it to ensure the full 64-bit range is covered and also made sure that errors are returned in similar cases.


Performance drop for neural-chat 7b with new repo of ipex-llm(2.5.0b20240425) vllm serving.

We have seen a significant difference in performance drop with the env created with the latest repo for vllm serving for the neural-chat model as compared to the old env built with the old repo. With the offline_inference.py script, for the default prompt  Old env gives inference time between 7-11 sec with GPU utilization of only 50% while the new env gives 18-24 sec, with GPU utilization of 100% on Flex 170. I tried the docker env also but it's also giving the inference time between 18-23 sec. Given below are the env details :
Old env
accelerate                  0.21.0
annotated-types             0.6.0
anyio                       4.3.0
bigdl-core-xe-21            2.5.0b20240402
bigdl-core-xe-esimd-21      2.5.0b20240402
certifi                     2024.2.2
charset-normalizer          3.3.2
click                       8.1.7
exceptiongroup              1.2.0
fastapi                     0.110.1
filelock                    3.13.3
fsspec                      2024.3.1
h11                         0.14.0
httptools                   0.6.1
huggingface-hub             0.17.3
idna                        3.6
intel-extension-for-pytorch 2.1.10+xpu
intel-openmp                2024.1.0
ipex-llm                    2.1.0b20240402
Jinja2                      3.1.3
MarkupSafe                  2.1.5
mpmath                      1.3.0
networkx                    3.2.1
numpy                       1.26.4
packaging                   24.0
pillow                      10.3.0
pip                         23.3.1
protobuf                    5.26.1
psutil                      5.9.8
py-cpuinfo                  9.0.0
pydantic                    1.10.15
pydantic_core               2.18.0
python-dotenv               1.0.1
PyYAML                      6.0.1
regex                       2023.12.25
requests                    2.31.0
safetensors                 0.4.2
sentencepiece               0.2.0
setuptools                  68.2.2
sniffio                     1.3.1
starlette                   0.37.2
sympy                       1.12.1rc1
tabulate                    0.9.0
tokenizers                  0.14.1
torch                       2.1.0a0+cxx11.abi
torchvision                 0.16.0a0+cxx11.abi
tqdm                        4.66.2
transformers                4.34.0
typing_extensions           4.11.0rc1
urllib3                     2.2.1
uvicorn                     0.29.0
uvloop                      0.19.0
watchfiles                  0.21.0
websockets                  12.0
wheel                       0.41.2
New env
accelerate                    0.21.0
aiosignal                     1.3.1
annotated-types               0.6.0
anyio                         4.3.0
attrs                         23.2.0
bigdl-core-xe-21              2.5.0b20240425
bigdl-core-xe-esimd-21        2.5.0b20240425
certifi                       2024.2.2
charset-normalizer            3.3.2
click                         8.1.7
cloudpickle                   3.0.0
cmake                         3.29.2
diskcache                     5.6.3
einops                        0.7.0
fastapi                       0.110.1
filelock                      3.13.4
frozenlist                    1.4.1
fsspec                        2024.3.1
h11                           0.14.0
httptools                     0.6.1
huggingface-hub               0.22.2
idna                          3.7
intel-extension-for-pytorch   2.1.10+xpu
intel-openmp                  2024.1.0
interegular                   0.3.3
ipex-llm                      2.1.0b20240425
Jinja2                        3.1.3
joblib                        1.4.0
jsonschema                    4.21.1
jsonschema-specifications     2023.12.1
lark                          1.1.9
llvmlite                      0.42.0
MarkupSafe                    2.1.5
mpmath                        1.3.0
msgpack                       1.0.8
nest-asyncio                  1.6.0
networkx                      3.3
ninja                         <IP_ADDRESS>
numba                         0.59.1
numpy                         1.26.4
oneccl-bind-pt                2.1.100+xpu
outlines                      0.0.34
packaging                     24.0
pandas                        2.2.2
pillow                        10.3.0
pip                           23.3.1
prometheus_client             0.20.0
protobuf                      5.26.1
psutil                        5.9.8
py-cpuinfo                    9.0.0
pyarrow                       16.0.0
pydantic                      2.7.1
pydantic_core                 2.18.2
pynvml                        11.5.0
python-dateutil               2.9.0.post0
python-dotenv                 1.0.1
pytz                          2024.1
PyYAML                        6.0.1
ray                           2.12.0
referencing                   0.35.0
regex                         2023.12.25
requests                      2.31.0
rpds-py                       0.18.0
safetensors                   0.4.3
scipy                         1.13.0
sentencepiece                 0.2.0
setuptools                    68.2.2
six                           1.16.0
sniffio                       1.3.1
starlette                     0.37.2
sympy                         1.12.1rc1
tabulate                      0.9.0
tiktoken                      0.6.0
tokenizers                    0.19.1
torch                         2.1.0a0+cxx11.abi
torchvision                   0.16.0a0+cxx11.abi
tqdm                          4.66.2
transformers                  4.40.1
transformers-stream-generator 0.0.5
triton                        2.1.0
typing_extensions             4.11.0
tzdata                        2024.1
urllib3                       2.2.1
uvicorn                       0.29.0
uvloop                        0.19.0
vllm                          0.3.3+xpu0.0.1     /root/vllm
watchfiles                    0.21.0
websockets                    12.0
wheel                         0.41.2
xformers                      0.0.25.post1
Also with the new env its giving the below error with bfloat16 datatype is it not supported now?

I don't know if it's related, but I also noticed a drop in performance running Llama 3 under Ollama recently (so using the IPEX llama.cpp implementation). I was originally getting ~50t/s on inference, ran a rebuild (still with the intelanalytics/ipex-llm-xpu:latest image as a base) and suddenly it dropped to ~30t/s. I couldn't find any combination of host drivers or OneAPI packages that would get the performance back.
At this point, it's difficult to justify sticking with the Intel platform - my old RX 6600 XT is 30% faster than the A770 is now!

Hi, I am working to reproduce this issue.

Can you post the result of the offline_inference.py within your old environment?
We fix a bug recently that may cause the generation ends early.  So if the generation ends early with wired output, the inference will be quicker.

Hi @digitalscream , based on our local test, Llama3 could get ~50 tokens/s on a single A770.

I was originally getting ~50t/s on inference, ran a rebuild (still with the intelanalytics/ipex-llm-xpu:latest image as a base) and suddenly it dropped to ~30t/s.

I wonder is there anything changed in this rebuild progress? This performance degradation looks more like a driver related issue.

Can you check if your old environment's vLLM have the following code:
https://github.com/analytics-zoo/vllm/blob/sycl_xpu/vllm/worker/model_runner.py#L216
Also, you can try benchmark_throughput to get a more accurate performance estimation:
Try follow the instructions here: https://github.com/intel-analytics/ipex-llm/tree/main/docker/llm/serving/xpu/docker#offline-benchmark-through-benchmark_throughputpy
The benchmark_throughput.py can be acquired at here

Hi @digitalscream , based on our local test, Llama3 could get ~50 tokens/s on a single A770.

I was originally getting ~50t/s on inference, ran a rebuild (still with the intelanalytics/ipex-llm-xpu:latest image as a base) and suddenly it dropped to ~30t/s.

I wonder is there anything changed in this rebuild progress? This performance degradation looks more like a driver related issue.

Sorry, should've mentioned - that's using a single A770, the only change in the Docker image was that it pulled the intelanalytics/ipex-llm-xpu:latest base image again. I do have some new information, though - seems like it's heavily CPU-bound now, where it wasn't before. After a bit of experimentation, I get 30t/s on my Ryzen 3600 machine, and 41t/s on my i5-13600 machine. Originally, I was getting ~50t/s on the Ryzen machine.

Can you post the result of the offline_inference.py within your old environment?
We fix a bug recently that may cause the generation ends early. So if the generation ends early with wired output, the inference will be quicker.

Can you check if your old environment's vLLM have the following code: https://github.com/analytics-zoo/vllm/blob/sycl_xpu/vllm/worker/model_runner.py#L216
Also, you can try benchmark_throughput to get a more accurate performance estimation: Try follow the instructions here: https://github.com/intel-analytics/ipex-llm/tree/main/docker/llm/serving/xpu/docker#offline-benchmark-through-benchmark_throughputpy
The benchmark_throughput.py can be acquired at here

Yes, my old environment's vllm has the code mentioned here. I Will try benchmark_throughput.py script. Thanks.

Sorry, should've mentioned - that's using a single A770, the only change in the Docker image was that it pulled the intelanalytics/ipex-llm-xpu:latest base image again. I do have some new information, though - seems like it's heavily CPU-bound now, where it wasn't before. After a bit of experimentation, I get 30t/s on my Ryzen 3600 machine, and 41t/s on my i5-13600 machine. Originally, I was getting ~50t/s on the Ryzen machine.

Thanks for more information provided! When you updated the Docker image, did you update the version of the ipex-llm [cpp] and updated the ollama binary file you used?
We have a conjecture about the performance degradation on Ryzen 3600, suspecting that this issue is related to one of our previous PR which affecting a certain function on the CPU. We have already reverted that PR. Perhaps you can try our latest release tomorrow (pip install --pre --upgrade ipex-llm[cpp] and don't forget to init-ollama again) to see if this issue can be resolved?

Sorry, should've mentioned - that's using a single A770, the only change in the Docker image was that it pulled the intelanalytics/ipex-llm-xpu:latest base image again. I do have some new information, though - seems like it's heavily CPU-bound now, where it wasn't before. After a bit of experimentation, I get 30t/s on my Ryzen 3600 machine, and 41t/s on my i5-13600 machine. Originally, I was getting ~50t/s on the Ryzen machine.

Thanks for more information provided! When you updated the Docker image, did you update the version of the ipex-llm [cpp] and updated the ollama binary file you used? We have a conjecture about the performance degradation on Ryzen 3600, suspecting that this issue is related to one of our previous PR which affecting a certain function on the CPU. We have already reverted that PR. Perhaps you can try our latest release tomorrow (pip install --pre --upgrade ipex-llm[cpp] and don't forget to init-ollama again) to see if this issue can be resolved?

Ah, OK - yes, I updated everything when I rebuilt it from the base image. Is there another issue regarding the Ryzen performance? Don't want to pollute this one if there's a more appropriate place to discuss it.

Can you check if your old environment's vLLM have the following code: https://github.com/analytics-zoo/vllm/blob/sycl_xpu/vllm/worker/model_runner.py#L216
Also, you can try benchmark_throughput to get a more accurate performance estimation: Try follow the instructions here: https://github.com/intel-analytics/ipex-llm/tree/main/docker/llm/serving/xpu/docker#offline-benchmark-through-benchmark_throughputpy
The benchmark_throughput.py can be acquired at here

Yes, my old environment's vllm has the code mentioned here. I Will try benchmark_throughput.py script. Thanks.

Hi @gc-fu I tried to run offline_inference.py with the latest code still getting 17sec of latency.  Also, I tested the benchmark_throughput.py script, could you suggest how to get the inference latency from end to end?

The offline_inference.py is not designed for performance benchmark.
If you wanna get latency from end to end or get request per second, you should start the service according to this readme. Then you can send requests to the service using benchmark tools like wrk or jmeter.
The result of benchmark_throughput.py should give you some insight about token per second, which should indicate the performance of different versions.

Thanks @gc-fu, with docker env the benchmark_throughput.py gives a throughput of 489.68 token/sec for 1000 prompt (default settings), however, this script is not available in the docker directory of the old repo.

Can you check if this official benchmark script https://github.com/vllm-project/vllm/blob/main/benchmarks/benchmark_throughput.py can be used or not?
If not, can you post the docker image name and tag so that I can see if I can find an proper script for you :smiley:  And it would be most helpful if you can post the entire offline_inference.py script in your old environment.  It has been a long time since we post the old docker image :cry:

I tried running the script in the old repo but facing import issues.
This is the docker image ipex-llm-serving-xpu:2.1.0-SNAPSHOT
This is the offline_inference.py script
from ipex_llm.vllm.entrypoints.llm import LLM
from ipex_llm.vllm.sampling_params import SamplingParams
import time
prompts = [
    "Hello, my name is",
    "The president of the United States is",
    "The capital of France is",
    "The future of AI is",
]
sampling_params = SamplingParams(temperature=0.8, top_p=0.95)
llm = LLM(model="/root/neural-chat-7b-v3/", load_in_low_bit="sym_int4", dtype="bfloat16", device="xpu")
st_time = time.time()
outputs = llm.generate(prompts, sampling_params)
en_time = time.time()
print(f'Inference time: {en_time-st_time} s')
for output in outputs:
    prompt = output.prompt
    generated_text = output.outputs[0].text
    print(f"Prompt: {prompt!r}, Generated text: {generated_text!r}")

In this case, can you try the following script?
"""Benchmark offline inference throughput."""
import argparse
import json
import random
import time
from typing import List, Optional, Tuple

import torch
from ipex_llm.transformers import AutoModelForCausalLM
# from transformers import AutoModelForCausalLM

from transformers import PreTrainedTokenizerBase
from tqdm import tqdm

#from vllm import LLM, SamplingParams
from ipex_llm.vllm.entrypoints.llm import LLM
from ipex_llm.vllm.sampling_params import SamplingParams
#from vllm.transformers_utils.tokenizer import get_tokenizer
from ipex_llm.vllm.transformers_utils.tokenizer import get_tokenizer

device = 'xpu'
if device == 'xpu':
    import intel_extension_for_pytorch as ipex

def sample_requests(
    dataset_path: str,
    num_requests: int,
    tokenizer: PreTrainedTokenizerBase,
    fixed_output_len: Optional[int],
) -> List[Tuple[str, int, int]]:
    if fixed_output_len is not None and fixed_output_len < 4:
        raise ValueError("output_len too small")

    # Load the dataset.
    with open(dataset_path) as f:
        dataset = json.load(f)
    # Filter out the conversations with less than 2 turns.
    dataset = [data for data in dataset if len(data["conversations"]) >= 2]
    # Only keep the first two turns of each conversation.
    dataset = [(data["conversations"][0]["value"],
                data["conversations"][1]["value"]) for data in dataset]

    # Tokenize the prompts and completions.
    prompts = [prompt for prompt, _ in dataset]
    prompt_token_ids = tokenizer(prompts).input_ids
    completions = [completion for _, completion in dataset]
    completion_token_ids = tokenizer(completions).input_ids
    tokenized_dataset = []
    for i in range(len(dataset)):
        output_len = len(completion_token_ids[i])
        if fixed_output_len is not None:
            output_len = fixed_output_len
        tokenized_dataset.append((prompts[i], prompt_token_ids[i], output_len))

    # Filter out too long sequences.
    filtered_dataset: List[Tuple[str, int, int]] = []
    for prompt, prompt_token_ids, output_len in tokenized_dataset:
        prompt_len = len(prompt_token_ids)
        if prompt_len < 4 or output_len < 4:
            # Prune too short sequences.
            continue
        if prompt_len > 1024 or prompt_len + output_len > 2048:
            # Prune too long sequences.
            continue
        filtered_dataset.append((prompt, prompt_len, output_len))

    # Sample the requests.
    sampled_requests = random.sample(filtered_dataset, num_requests)
    return sampled_requests

def run_vllm(
    requests: List[Tuple[str, int, int]],
    model: str,
    tokenizer: str,
    quantization: Optional[str],
    tensor_parallel_size: int,
    seed: int,
    n: int,
    use_beam_search: bool,
    trust_remote_code: bool,
    dtype: str,
    max_num_seqs: int,
) -> float:
    llm = LLM(
        model=model,
        tokenizer=tokenizer,
        quantization=quantization,
        #tensor_parallel_size=tensor_parallel_size,
        seed=42,
        trust_remote_code=trust_remote_code,
        dtype=dtype,
				# change here
		device=device,
        max_num_batched_tokens=204800,
        max_model_len=2048,
        max_num_seqs=max_num_seqs,
    )
    warm_prompt = "hi " * (1024 - 1)
    warm_requests = [(warm_prompt, 1024, 1024)
                    for _ in range(1)]
    for prompt, _, output_len in warm_requests:
        sampling_params = SamplingParams(
            n=n,
            temperature=0.0 if use_beam_search else 1.0,
            top_p=1.0,
            use_beam_search=use_beam_search,
            ignore_eos=True,
            max_tokens=output_len,
        )
        llm._add_request(
            prompt=prompt,
            prompt_token_ids=None,
            sampling_params=sampling_params,
        )
    llm._run_engine(use_tqdm=True)

    # Add the requests to the engine.
    for prompt, _, output_len in requests:
        sampling_params = SamplingParams(
            n=n,
            temperature=0.0 if use_beam_search else 1.0,
            top_p=1.0,
            use_beam_search=use_beam_search,
            ignore_eos=True,
            max_tokens=output_len,
        )
        # FIXME(woosuk): Do not use internal method.
        llm._add_request(
            prompt=prompt,
            prompt_token_ids=None,
            sampling_params=sampling_params,
        )

    start = time.perf_counter()
    # FIXME(woosuk): Do use internal method.
    llm._run_engine(use_tqdm=True)
    end = time.perf_counter()
    return end - start

def run_hf(
    requests: List[Tuple[str, int, int]],
    model: str,
    tokenizer: PreTrainedTokenizerBase,
    n: int,
    use_beam_search: bool,
    max_batch_size: int,
    trust_remote_code: bool,
) -> float:
    assert not use_beam_search
    llm = AutoModelForCausalLM.from_pretrained(
        model, load_in_4bit=True,  optimize_model=True,
                                                 trust_remote_code=True,
                                                 use_cache=True)
    # llm = AutoModelForCausalLM.from_pretrained(
    #     model, trust_remote_code=True, use_cache=True, torch_dtype=torch.bfloat16,
    # )

    tokenizer.pad_token = tokenizer.eos_token
    if device == 'xpu':
        llm = llm.to('xpu')

    # warmup
    warm_prompt = "hi " * (1000 - 1)
    input_ids = tokenizer(warm_prompt, return_tensors="pt",
                              padding=True).input_ids

    if device == 'xpu':
        input_ids = input_ids.to('xpu')
    _ = llm.generate(
            input_ids=input_ids,
            do_sample=False,
            num_return_sequences=n,
            num_beams=1,
            temperature=1.0,
            top_p=1.0,
            use_cache=True,
            max_new_tokens=1024,
            pad_token_id=tokenizer.pad_token_id,
        )

    pbar = tqdm(total=len(requests))
    start = time.perf_counter()
    batch: List[str] = []
    max_prompt_len = 0
    max_output_len = 0
    for i in range(len(requests)):
        prompt, prompt_len, output_len = requests[i]
        # Add the prompt to the batch.
        batch.append(prompt)
        max_prompt_len = max(max_prompt_len, prompt_len)
        max_output_len = max(max_output_len, output_len)
        if len(batch) < max_batch_size and i != len(requests) - 1:
            # Check if we can add more requests to the batch.
            _, next_prompt_len, next_output_len = requests[i + 1]
            if (max(max_prompt_len, next_prompt_len) +
                    max(max_output_len, next_output_len)) <= 2048:
                # We can add more requests to the batch.
                continue

        # Generate the sequences.
        # print(batch)
        input_ids = tokenizer(batch, return_tensors="pt",
                              padding=True).input_ids
        if device == 'xpu':
            input_ids = input_ids.to('xpu')
        llm_outputs = llm.generate(
            input_ids=input_ids,
            do_sample=False,
            num_return_sequences=n,
            num_beams=1,
            temperature=1.0,
            top_p=1.0,
            use_cache=True,
            max_new_tokens=max_output_len,
            pad_token_id=tokenizer.pad_token_id,
        )
        # Include the decoding time.
        tokenizer.batch_decode(llm_outputs, skip_special_tokens=True)
        pbar.update(len(batch))

        # Clear the batch.
        batch = []
        max_prompt_len = 0
        max_output_len = 0
    end = time.perf_counter()
    return end - start

def main(args: argparse.Namespace):
    print(args)
    random.seed(args.seed)

    # Sample the requests.
    tokenizer = get_tokenizer(args.tokenizer,
                              unk_token="<unk>",
                              trust_remote_code=args.trust_remote_code)
    if args.dataset is None:
        # Synthesize a prompt with the given input length.
        prompt = "hi " * (args.input_len - 1)
        requests = [(prompt, args.input_len, args.output_len)
                    for _ in range(args.num_prompts)]
    else:
        requests = sample_requests(args.dataset, args.num_prompts, tokenizer,
                                   args.output_len)

    if args.backend == "vllm":
        elapsed_time = run_vllm(requests, args.model, args.tokenizer,
                                args.quantization, args.tensor_parallel_size,
                                args.seed, args.n, args.use_beam_search,
                                args.trust_remote_code, args.dtype, args.max_num_seqs)
    elif args.backend == "hf":
        assert args.tensor_parallel_size == 1
        elapsed_time = run_hf(requests, args.model, tokenizer, args.n,
                              args.use_beam_search, args.hf_max_batch_size,
                              args.trust_remote_code)
    else:
        raise ValueError(f"Unknown backend: {args.backend}")
    total_num_tokens = sum(prompt_len + output_len
                           for _, prompt_len, output_len in requests)
    print(f"Throughput: {len(requests) / elapsed_time:.4f} requests/s, "
          f"{total_num_tokens / elapsed_time:.2f} tokens/s")

if __name__ == "__main__":
    parser = argparse.ArgumentParser(description="Benchmark the throughput.")
    parser.add_argument("--backend",
                        type=str,
                        choices=["vllm", "hf"],
                        default="vllm")
    parser.add_argument("--dataset",
                        type=str,
                        default=None,
                        help="Path to the dataset.")
    parser.add_argument("--input-len",
                        type=int,
                        default=None,
                        help="Input prompt length for each request")
    parser.add_argument("--output-len",
                        type=int,
                        default=None,
                        help="Output length for each request. Overrides the "
                        "output length from the dataset.")
    parser.add_argument("--model", type=str, default="facebook/opt-125m")
    parser.add_argument("--tokenizer", type=str, default=None)
    parser.add_argument('--quantization',
                        '-q',
                        choices=['awq', None],
                        default=None)
    parser.add_argument("--tensor-parallel-size", "-tp", type=int, default=1)
    parser.add_argument("--n",
                        type=int,
                        default=1,
                        help="Number of generated sequences per prompt.")
    parser.add_argument("--use-beam-search", action="store_true")
    parser.add_argument("--num-prompts",
                        type=int,
                        default=1000,
                        help="Number of prompts to process.")
    parser.add_argument("--seed", type=int, default=42)
    parser.add_argument("--hf-max-batch-size",
                        type=int,
                        default=None,
                        help="Maximum batch size for HF backend.")
    parser.add_argument("--max-num-seqs", type=int, default=8)
    parser.add_argument('--trust-remote-code',
                        action='store_true',
                        help='trust remote code from huggingface')
    parser.add_argument(
        '--dtype',
        type=str,
        default='auto',
        choices=['auto', 'half', 'float16', 'bfloat16', 'float', 'float32'],
        help='data type for model weights and activations. '
        'The "auto" option will use FP16 precision '
        'for FP32 and FP16 models, and BF16 precision '
        'for BF16 models.')
    args = parser.parse_args()

    if args.backend == "vllm":
        if args.hf_max_batch_size is not None:
            raise ValueError("HF max batch size is only for HF backend.")
    elif args.backend == "hf":
        if args.hf_max_batch_size is None:
            raise ValueError("HF max batch size is required for HF backend.")
        if args.quantization is not None:
            raise ValueError("Quantization is only for vLLM backend.")
    if args.tokenizer is None:
        args.tokenizer = args.model

    if args.dataset is None:
        assert args.input_len is not None
        assert args.output_len is not None
    else:
        assert args.input_len is None

    main(args)

Also, can you try to run the following command in your docker environment and post the result?
find / -name "bigdl_mistral.py"

If you successfully find the file in your environment, then you are using an quite old version of vLLM that not fully implement the PagedAttention algorithm.

I Faced this error while trying out the above code, could you suggest how to resolve it?

Hi, the vLLM you used is deprecated and will not be supported anymore :cry:
The old vLLM does not use PagedAttention and do not perform good enough in our tests.  Besides, the old vLLM suffered from Out of Memory issue in GPU environment.
Try using the latest vLLM instead, I am pretty sure the new vLLM is quicker than the old one.

Hi @gc-fu , I couldn't locate the benchmark_throughput.py file inside docker, could you share the path? This is the docker image built, earlier I was able to locate it, but rebuild the image and now I couldn't find it.

The image you should use is this one: intelanalytics/ipex-llm-serving-xpu
Try check the README here: https://github.com/intel-analytics/ipex-llm/tree/main/docker/llm/serving/xpu/docker

The image you should use is this one: intelanalytics/ipex-llm-serving-xpu
Try check the README here: https://github.com/intel-analytics/ipex-llm/tree/main/docker/llm/serving/xpu/docker

Can we remove the deprecated docker image?

The image you should use is this one: intelanalytics/ipex-llm-serving-xpu
Try check the README here: https://github.com/intel-analytics/ipex-llm/tree/main/docker/llm/serving/xpu/docker

Can we remove the deprecated docker image?

Yes, we can.  There are basically two images related to vLLM:
vLLM-CPU: ipex-llm-serving-cpu.  Since vLLM-v1 is removed from our codebase, this image no longer contains any code related to vLLM.
vLLM-XPU: ipex-llm-serving-xpu.  This is the only available image that users should use for now.
At the beginning of this issue, the user uses the ipex-llm-serving-xpu image that was build long ago which contains the vLLM-v1 code that is deprecated.  If the user pulls the image again, the old code will disappear.
I will remove the vLLM-CPU example page later.


Features/propagate goflags env

Description:
This PR will pick GOFLAGS from fyne-cross host environment and propagate them to the call in the container.
Where applicable:

[x] Public APIs match existing style.

Would be great to have this merged and released asap now that the tests are in!


Fix getChangeMap logic for year filter

Closes #22
Changes the order of filtering and change identify selection. The current code identifies change segment first and then filters it. This flow order can result in missed change when the year filter is used and change identified as greatest or least. The fix is to filter the segments first and then select by change identify.

I've done some testing to ensure that the changes WAI, but if you are aware of specific cases where the old implementation was incorrect, please test them. You can use the script mentioned here for testing - change parameters as needed (see docs for parameter defs).

Robert, if you are satisfied with the changes, please approve your review and merge.

🎉😃

Change synced to git-on-borg: https://earthengine.googlesource.com/users/emaprlab/public/+/42cceae74ff928d6909aee0a8b02f095a47da8a7/Modules/LandTrendr.js


Too many proposals

'x' and 'data' should not appear.

Seems like a global scope problem.
Why does it consider the formal parameters as a global variable?

Close by #30


Review script for paper "2024ApJ...964..139P." made with predictor "DataLinkerCondenseDetailsWithTemplate"

Created by Paper Data Linking Bot
Please review the download script and instrument details for the paper.
bibcode: 2024ApJ...964..139P.
predictor: DataLinkerCondenseDetailsWithTemplate
annotated pdf: not available
Checklist
Download Script

[ ] Correct time ranges specified

[ ] Correct VSO instruments specified

[ ] Query returns good results or contains warning with reasoning if not

[ ] Physical observables omitted or correct

[ ] Wavelengths omitted or correct

[ ] Cadences omitted or correct

Instrument Details

[ ] Consistent with download_script.py

Metadata

[ ] Contains the bibcode for the relevant paper
[ ] VSO sources consistent with download_script.py

For more details, refer to the Contributing Guide.

@abuonomo PDF not available