ConvertError Ed25519 public key

Bug description
Trying to build a Shimmer chronicle from scratch. After a few days Inx-chronicle crashes complaining about not being able to convert "compressed bytes"  to "Ed25519 public key". During that period shutting down and starting again was no problem. The shimmer node used for syncing is also healthy. The last milestone in the mongodb is 6430726:

Rust version

Rust version: from Dockerfile.debug

Chronicle version

Chronicle version (version number, commit, or branch): v1.0.0-rc.2

Hardware specification

Operating system: Arch Linux 6.6.4-arch1-1
RAM: 12 GB
Cores: 8
Device: Bare metal

Steps to reproduce the bug

Start syncing from milestone 0.
Shut down and restart a few times inx-chronicle and the node, that should not lead to problems.
Around milestone 6430726 inx-chronicle crashes and can't sync anymore.

Expected behaviour
Sync up to and including the LMI.
Actual behaviour
Syncing crashes and can't continue
Errors
docker compose logs inx-chronicle
inx-chronicle  | 2023-12-06T19:24:52.595733Z ERROR inx_chronicle::inx: error=Packable(Crypto(ConvertError { from: "compressed bytes", to: "Ed25519 public key" }))
inx-chronicle  | 2023-12-06T19:24:52.595763Z ERROR inx_chronicle::inx: error=Packable(Crypto(ConvertError { from: "compressed bytes", to: "Ed25519 public key" }))
inx-chronicle  | 2023-12-06T19:24:52.595849Z ERROR inx_chronicle: a worker failed with error: Packable(Crypto(ConvertError { from: "compressed bytes", to: "Ed25519 public key" }))
inx-chronicle  | Error: Packable(Crypto(ConvertError { from: "compressed bytes", to: "Ed25519 public key" }))
inx-chronicle  |
inx-chronicle  | Location:
inx-chronicle  |     src/bin/inx-chronicle/inx/mod.rs:347:25

Milestone 6430726 in the Mongo db:
{
  "_id": {
    "$binary": {
      "base64": "QoBV7IcY4eliFuV8a4qHYZanTRxy6E28fcnXDxxPvqI=",
      "subType": "00"
    }
  },
  "at": {
    "milestone_index": {
      "$numberLong": "6430726"
    },
    "milestone_timestamp": {
      "$numberLong": "1697593020"
    }
  },
  "payload": {
    "essence": {
      "index": {
        "$numberLong": "6430726"
      },
      "timestamp": {
        "$numberLong": "1697593020"
      },
      "protocol_version": 2,
      "previous_milestone_id": {
        "$binary": {
          "base64": "UZEQEb8x85JqE+kjfD5ezsCUW89IuCE85PqKZdG5SB4=",
          "subType": "00"
        }
      },
      "parents": [
        {
          "$binary": {
            "base64": "j/K3rDuBFDnIQynIYrctF0zQwC97Jyw3U9MZPHFSDv8=",
            "subType": "00"
          }
        },
        {
          "$binary": {
            "base64": "vHpJ4wuc5Lvkuhs7CMLfFbEyXlmHQKCb05iA05J0ZSQ=",
            "subType": "00"
          }
        }
      ],
      "inclusion_merkle_root": {
        "$binary": {
          "base64": "xBmCYtMQoKFFuAlVc9T4p6AELx0hz2zqn7euS2wwqfk=",
          "subType": "00"
        }
      },
      "applied_merkle_root": {
        "$binary": {
          "base64": "DldRwCblQ7Loqy6wYJnaodHl30d3j3eH+qtFzfEv46g=",
          "subType": "00"
        }
      },
      "metadata": {
        "$binary": {
          "base64": "jzfCAAAAAAC8eknjC5zku+S6GzsIwt8VsTJeWYdAoJvTmIDTknRlJA==",
          "subType": "00"
        }
      },
      "options": []
    },
    "signatures": [
      {
        "kind": "ed25519",
        "public_key": {
          "$binary": {
            "base64": "Fu4zVsIeQQoKqrQolgIbGoV+uNl6FKZv7ZsT1jTCExc=",
            "subType": "00"
          }
        },
        "signature": {
          "$binary": {
            "base64": "4qH7datetU74pPcQBaukZN8CJcaps80Ebay1OekSuuRtoDZ3TSVM4whpjfug8Uw8hppg/yoECykdAQ87tVQMAQ==",
            "subType": "00"
          }
        }
      },
      {
        "kind": "ed25519",
        "public_key": {
          "$binary": {
            "base64": "HfJheKeRQSb9jLk0x6dDcHN5TByM6ZMZFyQ2sdSXPro=",
            "subType": "00"
          }
        },
        "signature": {
          "$binary": {
            "base64": "kF2CfH2o59dJqhwUcKR9OGQYEX/BkG15oyBE0wWSuvrpxAmRkxSISxe5D5udHWoySSEVEyEJvX+LUjCLuR/qBw==",
            "subType": "00"
          }
        }
      },
      {
        "kind": "ed25519",
        "public_key": {
          "$binary": {
            "base64": "RUMtfHZ+FlhkAyYjMaclx+qgst156kQvNzyEWuNEOqk=",
            "subType": "00"
          }
        },
        "signature": {
          "$binary": {
            "base64": "r4SU15PaQ18sV8HUZZU6+XTmZ97n7/5o1Q4/S8zTEITlEKbxsnpCMuki59HwzM+pt5QUAV0J0WSfHU49D8cTBg==",
            "subType": "00"
          }
        }
      },
      {
        "kind": "ed25519",
        "public_key": {
          "$binary": {
            "base64": "SvZHkQukcAAQi4fGOr4FRWQ/myA+rO4rcTcpsEUJg/4=",
            "subType": "00"
          }
        },
        "signature": {
          "$binary": {
            "base64": "vb2tH+hgxuJ/2zUq56boBSasb1wRNuIJJbYsVUTkXH5jc4fpDR5AHph/eD1M30KiA4WV8rLQ+H2ehiHhtOBcBA==",
            "subType": "00"
          }
        }
      },
      {
        "kind": "ed25519",
        "public_key": {
          "$binary": {
            "base64": "caCXdESaCBRQpR4CRaHphQGQ+TUI/Y8hu5ucoWl2XzA=",
            "subType": "00"
          }
        },
        "signature": {
          "$binary": {
            "base64": "Fzrf8cYDj6BAh713A1IxCa0dwZGTH2zCfLUIvjsv7oKdQmdgdBbyNqY3T4FMt2unDf6bHqv3wCwh61Tx5deyCA==",
            "subType": "00"
          }
        }
      },
      {
        "kind": "ed25519",
        "public_key": {
          "$binary": {
            "base64": "mcfZdSwpXLVrVQGRAVq1pAIm+2MuiwLsFc/ldOoXz2c=",
            "subType": "00"
          }
        },
        "signature": {
          "$binary": {
            "base64": "lHCQpSpG00GBh3/cLP+0xIEdRA5sQ10eba/5gn2N4KlUb/Y/HfeesIes3Cj9BC8jWa0ognUTfK6mfvb31T6fBw==",
            "subType": "00"
          }
        }
      },
      {
        "kind": "ed25519",
        "public_key": {
          "$binary": {
            "base64": "nYe00lOLEHmbWC4lrORybZLXeY3ftpb/CORQ23kXya0=",
            "subType": "00"
          }
        },
        "signature": {
          "$binary": {
            "base64": "5218g2ePzep0wQ6cIY96OaTaKxHQZVi5tsqFzlr1yXVoKm/NNr08z/KGGEw3TwtyKpNLWuYwnpcpcrTPuXsiAA==",
            "subType": "00"
          }
        }
      },
      {
        "kind": "ed25519",
        "public_key": {
          "$binary": {
            "base64": "o3VRW/5a33/ttk70zr4eYh6FoFawzNXbcrwNR0Mlvzg=",
            "subType": "00"
          }
        },
        "signature": {
          "$binary": {
            "base64": "Qr0JgJbX9LaTbvIOLOJT5UqUtgv4pTge7vkjNmGVDoWI7tQFh5mQWwEFzS6WMQVo7r5OHyTlSE538/w+QgnEAg==",
            "subType": "00"
          }
        }
      },
      {
        "kind": "ed25519",
        "public_key": {
          "$binary": {
            "base64": "pQfSpZKl8EJO2FMGA8CKzr4IiuJiEekLeb/sCXCiOX8=",
            "subType": "00"
          }
        },
        "signature": {
          "$binary": {
            "base64": "mcz0IRkLDX8X6G5d068Wby6Y/J82onBv9CqMGbZZbmRrnR/CCo+HqZ579jVTSPZL7jutjULcWT9Kcq8Nh4CpDw==",
            "subType": "00"
          }
        }
      },
      {
        "kind": "ed25519",
        "public_key": {
          "$binary": {
            "base64": "qSGEFijWTD8IvTRBGLgQat4HLmjHdL7/MBNeA2GUSTo=",
            "subType": "00"
          }
        },
        "signature": {
          "$binary": {
            "base64": "s8J77WqFJvpnZQDfknKcTRHzrR/2JAT2ImZluGFwpL6fTTzOMC+fMeljXAnrDmTX1ikp/uFaUXCEYubGd6CHBg==",
            "subType": "00"
          }
        }
      }
    ]
  }
}

This appears to be an issue with the data itself, as this occurs while unpacking a Block that we receive as a raw byte representation from the node. Alternatively, there could be a bug in the unpack logic, though I suspect this would have caused issues long before this milestone. Are you able to export the blocks within the erroneous milestone from the node?

Exporting those blocks should be doable If you can assist me with buliding the curl command ;)

curl -L -X GET 'http://<IP_ADDRESS>:14265/api/core/v2/milestones/by-index/6430726' -H 'Accept: application/json'|jq

{
  "type": 7,
  "index": 6430726,
  "timestamp":<PHONE_NUMBER>,
  "protocolVersion": 2,
  "previousMilestoneId": "0x51911011bf31f3926a13e9237c3e5ecec0945bcf48b8213ce4fa8a65d1b9481e",
  "parents": [
    "0x8ff2b7ac3b811439c84329c862b72d174cd0c02f7b272c3753d3193c71520eff",
    "0xbc7a49e30b9ce4bbe4ba1b3b08c2df15b1325e598740a09bd39880d392746524"
  ],
  "inclusionMerkleRoot": "0xc4198262d310a0a145b8095573d4f8a7a0042f1d21cf6cea9fb7ae4b6c30a9f9",
  "appliedMerkleRoot": "0x0e5751c026e543b2e8ab2eb06099daa1d1e5df47778f7787faab45cdf12fe3a8",
  "metadata": "0x8f37c20000000000bc7a49e30b9ce4bbe4ba1b3b08c2df15b1325e598740a09bd39880d392746524",
  "signatures": [
    {
      "type": 0,
      "publicKey": "0x16ee3356c21e410a0aaab42896021b1a857eb8d97a14a66fed9b13d634c21317",
      "signature": "0xe2a1fb75ab5eb54ef8a4f71005aba464df0225c6a9b3cd046dacb539e912bae46da036774d254ce308698dfba0f14c3c869a60ff2a040b291d010f3bb5540c01"
    },
    {
      "type": 0,
      "publicKey": "0x1df26178a7914126fd8cb934c7a7437073794c1c8ce99319172436b1d4973eba",
      "signature": "0x905d827c7da8e7d749aa1c1470a47d386418117fc1906d79a32044d30592bafae9c409919314884b17b90f9b9d1d6a32492115132109bd7f8b52308bb91fea07"
    },
    {
      "type": 0,
      "publicKey": "0x45432d7c767e16586403262331a725c7eaa0b2dd79ea442f373c845ae3443aa9",
      "signature": "0xaf8494d793da435f2c57c1d465953af974e667dee7effe68d50e3f4bccd31084e510a6f1b27a4232e922e7d1f0cccfa9b79414015d09d1649f1d4e3d0fc71306"
    },
    {
      "type": 0,
      "publicKey": "0x4af647910ba47000108b87c63abe0545643f9b203eacee2b713729b0450983fe",
      "signature": "0xbdbdad1fe860c6e27fdb352ae7a6e80526ac6f5c1136e20925b62c5544e45c7e637387e90d1e401e987f783d4cdf42a2038595f2b2d0f87d9e8621e1b4e05c04"
    },
    {
      "type": 0,
      "publicKey": "0x71a09774449a081450a51e0245a1e9850190f93508fd8f21bb9b9ca169765f30",
      "signature": "0x173adff1c6038fa04087bd7703523109ad1dc191931f6cc27cb508be3b2fee829d4267607416f236a6374f814cb76ba70dfe9b1eabf7c02c21eb54f1e5d7b208"
    },
    {
      "type": 0,
      "publicKey": "0x99c7d9752c295cb56b550191015ab5a40226fb632e8b02ec15cfe574ea17cf67",
      "signature": "0x947090a52a46d34181877fdc2cffb4c4811d440e6c435d1e6daff9827d8de0a9546ff63f1df79eb087acdc28fd042f2359ad288275137caea67ef6f7d53e9f07"
    },
    {
      "type": 0,
      "publicKey": "0x9d87b4d2538b10799b582e25ace4726d92d7798ddfb696ff08e450db7917c9ad",
      "signature": "0xe76d7c83678fcdea74c10e9c218f7a39a4da2b11d06558b9b6ca85ce5af5c975682a6fcd36bd3ccff286184c374f0b722a934b5ae6309e972972b4cfb97b2200"
    },
    {
      "type": 0,
      "publicKey": "0xa375515bfe5adf7fedb64ef4cebe1e621e85a056b0ccd5db72bc0d474325bf38",
      "signature": "0x42bd098096d7f4b6936ef20e2ce253e54a94b60bf8a5381eeef9233661950e8588eed4058799905b0105cd2e96310568eebe4e1f24e5484e77f3fc3e4209c402"
    },
    {
      "type": 0,
      "publicKey": "0xa507d2a592a5f0424ed8530603c08acebe088ae26211e90b79bfec0970a2397f",
      "signature": "0x99ccf421190b0d7f17e86e5dd3af166f2e98fc9f36a2706ff42a8c19b6596e646b9d1fc20a8f87a99e7bf6355348f64bee3bad8d42dc593f4a72af0d8780a90f"
    },
    {
      "type": 0,
      "publicKey": "0xa921841628d64c3f08bd344118b8106ade072e68c774beff30135e036194493a",
      "signature": "0xb3c27bed6a8526fa676500df92729c4d11f3ad1ff62404f6226665b86170a4be9f4d3cce302f9f31e9635c09eb0e64d7d62929fee15a51708462e6c677a08706"
    }
  ]
}

@DaughterOfMars is this enough? Not sure which route I should use to export the blocks you want.

If I recall correctly, the URI would be http://<IP_ADDRESS>:14265/api/core/v2/milestones/by-index/6430726/blocks, which will return a list of block indexes. You would then need to get all of the blocks using http://<IP_ADDRESS>:14265/api/core/v2/blocks/{blockId}.
If you are able to run a small rust program, you could use iota-sdk to create a client, connect to your node, and call the API requests in a loop. I can write the program for you if this is something you want to do.

curl -L -X GET 'http://<IP_ADDRESS>:14265/api/core/v2/milestones/by-index/6430726/blocks' -H 'Accept: application/json'|jq

gives a 404. I can run the a rust program, but not write it ;)

No problem. @Thoralf-M found the offending block! We're investigating now, but I would guess there is a bug in the iota-crypto library.

Just leaving this here in case it's needed again in the future
use std::{collections::HashMap, pin::Pin};

use futures::FutureExt;
use iota_sdk::{
    client::{Client, Result},
    types::block::{Block, BlockId},
};

#[tokio::main]
async fn main() -> Result<()> {
    let milestone_index = 6430727;

    let client = Client::builder()
        .with_node("https://chronicle.shimmer.network")?
        .with_ignore_node_health()
        .finish()
        .await?;

    let milestone = client.get_milestone_by_index(milestone_index).await?;
    println!("{milestone:#?}");

    let mut milestone_blocks = HashMap::new();
    get_parents_in_milestone_index(
        &client,
        milestone.essence().parents().to_vec(),
        milestone_index,
        &mut milestone_blocks,
    )
    .await?;

    println!("milestone {milestone_index} blocks: {milestone_blocks:#?}");
    Ok(())
}

fn get_parents_in_milestone_index<'a>(
    client: &'a Client,
    parents: Vec<BlockId>,
    milestone_index: u32,
    milestone_blocks: &'a mut HashMap<BlockId, Block>,
) -> Pin<Box<dyn futures::Future<Output = Result<()>> + std::marker::Send + 'a>> {
    async move {
        for parent in parents {
            if milestone_blocks.contains_key(&parent) {
                continue;
            }
            let metadata = client.get_block_metadata(&parent).await?;
            if metadata.referenced_by_milestone_index.unwrap() == milestone_index {
                println!("Requesting block {parent}");
                let block = client.get_block(&parent).await?;
                milestone_blocks.insert(parent, block.clone());
                get_parents_in_milestone_index(client, block.parents().to_vec(), milestone_index, milestone_blocks)
                    .await?;
            }
        }
        Ok(())
    }
    .boxed()
}

with https://github.com/iotaledger/iota-sdk/commit/2b44b886538b3c5dbbaf55a8ace38e8f44273bfe

Thanks, I will try to run the code with my internal node !

I'm getting a compile error for the code from @Thoralf-M
PS D:\temp\iota_sdk_test\block-test> cargo build
   Compiling iota-sdk v1.1.3 (https://github.com/iotaledger/iota-sdk?branch=develop#2b44b886)
   Compiling block-test v0.1.0 (D:\temp\iota_sdk_test\block-test)
error[E0432]: unresolved import `futures`
 --> src\main.rs:3:5
  |
3 | use futures::FutureExt;
  |     ^^^^^^^ use of undeclared crate or module `futures`

error[E0433]: failed to resolve: use of undeclared crate or module `tokio`
 --> src\main.rs:9:3
  |
9 | #[tokio::main]
  |   ^^^^^ use of undeclared crate or module `tokio`

error[E0433]: failed to resolve: use of undeclared crate or module `futures`
  --> src\main.rs:40:18
   |
40 | ) -> Pin<Box<dyn futures::Future<Output = Result<()>> + std::marker::Send + 'a>> {
   |                  ^^^^^^^ use of undeclared crate or module `futures`

Some errors have detailed explanations: E0432, E0433.
For more information about an error, try `rustc --explain E0432`.
error: could not compile `block-test` (bin "block-test") due to 3 previous errors

rustup update stable
info: syncing channel updates for 'stable-x86_64-pc-windows-msvc'

  stable-x86_64-pc-windows-msvc unchanged - rustc 1.75.0 (82e1608df 2023-12-21)

info: checking for self-update

[package]
name = "block-test"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
iota-sdk = { git = "https://github.com/iotaledger/iota-sdk", branch = "develop" }

You need to add the dependency to the Cargo manifest. The simplest way is like this:
futures = "*"

.....
Requesting block 0x70fbccfa2b968ef39e7e222a718e6abd94d47fcc64873f09196c1ece94cb33a7
Requesting block 0x530ff700606bf496754e9176a206c7ec6c48529f4440ca5a09f2e4a66399552a
Requesting block 0x9b41be3155019465385148b2b46d13ab14b10eff0666d9ca522006982655159d
Requesting block 0x1227b05634468b57ee9431cc309d9d0bf98d90ae8cf69231d9215e079c0be42a
Requesting block 0xb7e1c5885e4f4b8bb09aa96727469bf5647d2aec48db39a10d8f91237e0462b9
Requesting block 0x4b21ca29d2c51fd8c05e985437989fb3e7c7f674da34a068668367167367459c
Error: Block(Crypto(ConvertError { from: "compressed bytes", to: "Ed25519 public key" }))

Indeed, last line shows the error!

Any progress on this bug? Would like to finish my own synced chronicle...

@TeeVeeEss I think it's fixed in v1.0.0-rc.3, but there is already a newer release https://github.com/iotaledger/inx-chronicle/releases/tag/v1.0.0-rc.4 with more other fixes, please try that

Just updated to rc.4 but there is now another error:
inx-chronicle  | 2024-02-03T15:13:56.118528Z ERROR chronicle::db::mongodb::collections::ledger_update: error=Kind: An invalid argument was provided: No documents provided to insert_many, labels: {}
inx-chronicle  | 2024-02-03T15:13:56.123680Z ERROR chronicle::db::mongodb::collections::ledger_update: error=Kind: An invalid argument was provided: No documents provided to insert_many, labels: {}
inx-chronicle  | 2024-02-03T15:13:56.128983Z ERROR chronicle::db::mongodb::collections::ledger_update: error=Kind: An invalid argument was provided: No documents provided to insert_many, labels: {}
inx-chronicle  | 2024-02-03T15:13:56.134137Z ERROR chronicle::db::mongodb::collections::ledger_update: error=Kind: An invalid argument was provided: No documents provided to insert_many, labels: {}
inx-chronicle  | 2024-02-03T15:13:56.139417Z ERROR chronicle::db::mongodb::collections::ledger_update: error=Kind: An invalid argument was provided: No documents provided to insert_many, labels: {}
inx-chronicle  | 2024-02-03T15:13:56.144480Z ERROR chronicle::db::mongodb::collections::ledger_update: error=Kind: An invalid argument was provided: No documents provided to insert_many, labels: {}
inx-chronicle  | 2024-02-03T15:13:56.149640Z ERROR chronicle::db::mongodb::collections::ledger_update: error=Kind: An invalid argument was provided: No documents provided to insert_many, labels: {}
inx-chronicle  | 2024-02-03T15:13:56.154840Z ERROR chronicle::db::mongodb::collections::ledger_update: error=Kind: An invalid argument was provided: No documents provided to insert_many, labels: {}
inx-chronicle  | 2024-02-03T15:13:56.159911Z ERROR chronicle::db::mongodb::collections::ledger_update: error=Kind: An invalid argument was provided: No documents provided to insert_many, labels: {}
inx-chronicle  | 2024-02-03T15:13:56.165419Z ERROR chronicle::db::mongodb::collections::ledger_update: error=Kind: An invalid argument was provided: No documents provided to insert_many, labels: {}
inx-chronicle  | Error: Kind: An invalid argument was provided: No documents provided to insert_many, labels: {}
inx-chronicle  |
inx-chronicle  | Location:
inx-chronicle  |     src/bin/inx-chronicle/migrations/migrate_2.rs:51:21

My chronicle.application_state in the Mongo db has only 1 record:
{
  "_id": {
    "$oid": "655ddffc9561feacddca56b1"
  },
  "last_migration": {
    "id": {
      "$numberLong": "1"
    },
    "app_version": "1.0.0-beta.37",
    "date": [
      2023,
      73
    ]
  },
  "starting_index": {
    "milestone_index": {
      "$numberLong": "7008862"
    },
    "milestone_timestamp": {
      "$numberLong": "1700650996"
    }
  }
}

In the end I decided to start from scratch again. The application state is now:
{
  "_id": {
    "$oid": "65bfa4e6677b1fced39f0215"
  },
  "last_migration": {
    "id": {
      "$numberLong": "2"
    },
    "app_version": "1.0.0-rc.3",
    "date": [
      2024,
      12
    ]
  },
  "starting_index": {
    "milestone_index": {
      "$numberLong": "8227219"
    },
    "milestone_timestamp": {
      "$numberLong": "1707058401"
    }
  }
}

Closing the issue, but not being able to migrate is maybe another bug.


Features alignment

Proposed Changes

Provider multichain #1879
Fix no pricing schema available in publish preview page #1901
optimize c2d order logic and fix compute output price tooltip #1896
fix wrong asset count for 0 results searches #383
get compute algo list for assets with multiple trusted algo publishers #389
service credentials id check #391
update dockerhub proxy #396
update ecosystem links #404
address whitelist indicator #403
update dependencies

We need to remove the Faucet from the Onboarding Guide


Fix auth bug

What

There was an error sometimes when logging in to the control panel:
did not find expected authorization request details in session, req.session["oidc:alpha-analytics-moj.eu.auth0.com"] is undefined
This seems to be caused by a race-condition in ExpressJS, which can send the redirection headers to the client before the session has been stored. Then the client can load the redirection URL, which fails to retrieve the session because it hasn't been stored yet. This PR explicitly saves the session and only redirects when the save function returns.

Fixed a minor issue where the render_debug template macro failed to display the default items (current user and session) because macros do not have access to the calling template context by default.

Added a Makefile, because I forget which commands to run when switching between projects.

I don't love ensure_session_saved's name but can't think of anything better


I've been working on some E2E tests for this, to make sure that we don't have regressions on it. For testing I was going to use the logs-provider idea that I have in the works... However, we realized with @voliva that it's better to wait until he has finished the work that he is currently doing improving the vitest tests, before I bring these E2E tests in.

Update:
It is exciting automatically but it is taking around 30 seconds to do so.
Is this behaviour expected?
Example test:

Test Suites: 1 passed, 1 total
Tests:       1 passed, 1 total
Snapshots:   0 total
Time:        32.689 s
Ran all test suites.
Jest did not exit one second after the test run has completed.
'This usually means that there are asynchronous operations that weren't stopped in your tests. Consider running Jest with --detectOpenHandles to troubleshoot this issue.
✨  Done in 62.02s.

Update:
It is exciting automatically but it is taking around 30 seconds to do so.
Is this behaviour expected?
Example test:

Test Suites: 1 passed, 1 total
Tests:       1 passed, 1 total
Snapshots:   0 total
Time:        32.689 s
Ran all test suites.
Jest did not exit one second after the test run has completed.
'This usually means that there are asynchronous operations that weren't stopped in your tests. Consider running Jest with --detectOpenHandles to troubleshoot this issue.
✨  Done in 62.02s.

It depends on which provider you are using. If you are using smoldot, then I think that this is normal, as smoldot is going to try to gracefully close all the open connections with its peers... A way around this (although it's a bit hacky) is to put smoldot into a worker, and then create a JSON-RPC provider from the worker, then we it exists they you can just destroy the worker.


An inactive user shouldn't be able to be added as a collaborator

Fixes #4534

This check must be also added in API

I am trying to add some tests but it keeps on failing... would fix tomorrow

LGTM

@adelowo  conflicted

Resolved confilcts

Thanks @lafriks

So I'd like to pitch in. Though a bit late as this seems to already be a done deal.
I would object to it being impossible to add an inactive user as a colaborator.
An inactive user added as a collaborator, I would interpret as a 'inactive collaborator'. Nothing strange in principle about that. To me the issue would more be, do you list inactive collaborators, or do you maybe only list them to team administrators &owners?
I think not allowing this is an arbitrary restriction in workflow. For example an admin may be in the process of setting up a new user's account, giving him the correct permissions etc., keeping the account under creation hidden/inactive.
Then as a final step after all account details check out correct, the admin may like to activate the account.
I would be interested to know how others view this.


Enhancement - Mongo DB - Mandatory fields should be marked with *

Summary
Release Environment - Mongo DB - Mandatory fields needs to be highlighted with *
Motivation
For the update statement user focus will be mainly on the 'Update Syntax' and they might forget to fill other fields.

This requires changes in the Label strings. To make this change, please head to mongoPlugin/src/main/resources/editor/. This folder contains the UI configuration for all the query types.


Get the Encoder output without the trainer object?

How do I get the EncoderOut  using the model object gotten from the model = task.build_model(args) for transformer most especially?
The .build_model(args) in transformer.py returns  cls(args, encoder, decoder). For Transformer, there is a class TransformerEncoder(FairseqEncoder) which returns the following:
return EncoderOut (             encoder_out=x,  # T x B x C             encoder_padding_mask=encoder_padding_mask,  # B x T             encoder_embedding=encoder_embedding,  # B x T x C             encoder_states=encoder_states,  # List[T x B x C]      )
I need to get the EncoderOut using model. I am afraid using just  model.encoder.forward() wont work in this case. Please I need to get the encoder_out and encoder_embedding. Thanks!
Please kindly confirm that I am right with the following, B = Batch size, T = number of samples, C = embedding_dim

It should be fine to call model.encoder directly. You can see how model.forward is implemented here: https://github.com/pytorch/fairseq/blob/7201ebc57ec9bff916ecd2e72b4f1e02ffa74f66/fairseq/models/fairseq_model.py#L200-L225

Thanks for your response. Sorry I'm asking too much questions.
In the case of embeddings, this model.encoder.forward_embedding() should work right?

Yes, I meant that the implementation for model.forward is very simple. It just calls self.encoder followed by self.decoder, so you can easily call model.encoder and it will work.
Similarly model.encoder.forward_embedding should be fine to call as well.

Ok! Thank you very much.
Finally, I discovered the lstm.py class did not implement the .forward_embedding method, but the self.forward() method in the lstm.py returns a dictionary of the form:
return {             'encoder_out': (x, final_hiddens, final_cells),             'encoder_padding_mask': encoder_padding_mask if encoder_padding_mask.any() else None         } where x is the embedding.
Is there a way to identify the generic architecture of the model(Transformer/LSTM) in use, such that with an IF statement I can get the embeddings and encoder output of the different model:
if (model.arch == 'Transformer'):          self.encoder.forward(....)        self.encoder.forward_embedding(....) elif (model.arch == 'LSTM'):  args.arch         self.encoder.forward(....) #returns embedding and encoder output in a dictionary tuple
Using args.arch could have worked but I discovered the architecture are too specific .e.g. transformer_wmt_en_de_big
Thanks! Thanks!! Thanks!!!

Is there a way to identify the generic architecture of the model(Transformer/LSTM)

Not really. Fairseq tries not to make many assumptions about the encoder/decoder format, so there's no specified interface for these things.

Using args.arch could have worked but I discovered the architecture are too specific .e.g. transformer_wmt_en_de_big

You can do something like this:
>>> from fairseq.models import ARCH_MODEL_REGISTRY
>>> print(ARCH_MODEL_REGISTRY['transformer_wmt_en_de_big'])
<class 'fairseq.models.transformer.TransformerModel'>

Hi @myleott , In a multi-GPU setting, model.encoder.forward_embedding(..) seems to return the output of the encoder n-number of times. Where n is the number of GPUs that is in use. Is my observation right? Is the implementation right?


feat: update deps

PR Checklist
Please check if your PR fulfills the following requirements:

[x] I am not introducing a breaking change (if you are, flag in conventional commit message with BREAKING CHANGE: describing the break)
[x] I am not introducing a new dependency (add notes below if you are)
[ ] I have added unit tests for the new feature or bug fix (if not, why?)
[x] I have fully tested (add details below) this the new feature or bug fix (if not, why?)
[ ] I have opened a PR for the related docs change (if not, why?)

Check failed…

i'll see if i can get it sorted... thx

looks like it just needed a tidy. thx


introduce reranker in our knowledgebase retriever

To enhance the relevance of context(retrieved from our knowledgebase),we can do reraking on searched documents ( more than top_k )and find the top_k documents.
There are two ways to do reraking:

Let a LLM decides
Use a reraking model

examples:

LlamaIndex

langchain with cohere reranker

langchain with pinecone

reference:

https://www.pinecone.io/learn/series/rag/rerankers/
https://github.com/FlagOpen/FlagEmbedding/tree/master/FlagEmbedding/reranker
https://github.com/netease-youdao/BCEmbedding

@nkwangleiGIT  reranker requires a different ranking models  comparing to embedding models. The api provided by fastchat can not be used for this reranker.
So we should write a apiserver  to handle rerank requests and use our model repository(minio) to store the reraking model files.
The task tracked at See https://github.com/kubeagi/core-library/issues/13
@Abirdcfly Please add more details on how you would like to use this reraking api.

@Abirdcfly  It's worthy to decide a reranking model now . Since we use bge for embedding, how about  https://huggingface.co/BAAI/bge-reranker-large#using-flagembedding ?

just like DocumentLoader, we can add a CRD rerank to config rerank model.
just like vectorstore CRD, user web interface now don't need config rerank CRD.
Because almost all rerank models can be invoked through transformers, app.py does not need adjustment. For the model download section, we can reuse some of the model CRD handling logic.


Enemies should surround the player instead of forming a conga line

Currently, because the pathfinding doesn't check to see if a tile is occupied, it doesn't consider other blockables such as enemies when calculating a route. This causes penguins to line up when following the player instead of properly swarming.

Penguin conga line sounds like a feature to me


Correct argument name in `defadvice!' docstring

Self-explanatory: make the docstring agree with the macro definition.

Thanks for the help!


[BUG] Missing class properties transform.

Describe the bug
project crash
How to Reproduce
Steps to reproduce the behavior:

npm i pdf-viewer-reactjs
import PDFViewer from 'pdf-viewer-reactjs';

Screenshots

Details (please complete the following information):

Device: Laptop
OS: Windows 11
Browser [any
Version latest

Any updates on this? I am getting the same error.


Fix array extraction in derived table instances_electronic_access

Update the query to use the corrected method of extracting JSON arrays with cross join and json_extract_path_text().
See:  https://github.com/library-data-platform/ldp/blob/master/doc/User_Guide.md#7-json-arrays
https://github.com/folio-org/folio-analytics/blob/master/sql/derived_tables/instances_electronic_access.sql

Fix submitted by Jennifer Eustis

See also: #92


Reuse a component input in the command.

I'd like to be able to ask someone to input a number once (e.g., 99) and use it in the 3 places the command requires the pakchunk name.  I couldn't figure out any syntax to reuse an input component.

Hi, you can use id to name components.
{
    "label": "IDs",
    "command": "echo %x% %x% %x%",
    "components": [
        {
            "type": "text",
            "label": "Value of x",
            "id": "x"
        }
    ]
}

Awesome, thank you.


Fix typo on README (downlaod -> download)

klerpi .. is the code working for you ? cause for me its not.
here is the error
File "/content/udemy-dl/udemy-dl.py", line 611, in <module>     main()   File "/content/udemy-dl/udemy-dl.py", line 595, in main     skip_hls_stream=args.skip_hls_stream,   File "/content/udemy-dl/udemy-dl.py", line 288, in course_download     skip_hls_stream=skip_hls_stream,   File "/content/udemy-dl/udemy/udemy.py", line 40, in course     return Udemy(url, username, password, cookies, basic, skip_hls_stream, callback)   File "/content/udemy-dl/udemy/internal.py", line 65, in __init__     super(InternUdemyCourse, self).__init__(*args, **kwargs)   File "/content/udemy-dl/udemy/shared.py", line 300, in __init__     self._fetch_course()   File "/content/udemy-dl/udemy/internal.py", line 78, in _fetch_course     self._info = self._real_extract(self._url, skip_hls_stream=self._skip_hls_stream)   File "/content/udemy-dl/udemy/extract.py", line 608, in _real_extract     course_json = self._extract_course_json(url, course_id, portal_name)   File "/content/udemy-dl/udemy/extract.py", line 347, in _extract_course_json     resp = self._extract_large_course_content(url=url)   File "/content/udemy-dl/udemy/extract.py", line 311, in _extract_large_course_content     data = self._session._get(url).json()   File "/usr/local/lib/python3.6/dist-packages/requests/models.py", line 898, in json     return complexjson.loads(self.text, **kwargs)   File "/usr/lib/python3.6/json/__init__.py", line 354, in loads     return _default_decoder.decode(s)   File "/usr/lib/python3.6/json/decoder.py", line 339, in decode     obj, end = self.raw_decode(s, idx=_w(s, 0).end())   File "/usr/lib/python3.6/json/decoder.py", line 357, in raw_decode     raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Look at the commit diff. I only changed the "download" word. Unless the README text is tied to the code somehow I can't see why the PR is the one breaking it.
If you're asking me generally if the program is working on my computer, the answer is yes. Make sure you've installed all the dependencies properly. If you're still facing problems after that open an issue. Best of luck!


Focused TextField in readOnly state emits continuous Frame Events

Hey Flutter Team -
I have identified what seems to be an issue related to a focused readOnly: true TextField.
What's happening is the TextField in this state emits a continuous stream of Frame Events reflecting similar behavior to that of a readOnly: false TextField where the cursor animates in and out.
Your help would be very much appreciated.
Thanks!
Details
Steps to reproduce

Clone the demo repository here: https://github.com/mack-at-pieces/flutter_text_field
Run the demo app in profile mode using the command flutter run -d macos --profile.
Open the Flutter DevTools debugger and profiler at the link provided by the command output
Navigate to the running demo app and tap to bring into focus.
At this point, the Frame Chart is idle and no events are occurring.
Next, tap the TextField to bring focus as if you are going to input a value.
Observe the Frame Chart and you will now see a continuous stream of events being emitted. This represents that of a readOnly: false TextField where the cursor animates in and out.

Demo
https://github.com/mack-at-pieces/flutter_text_field
Target Platform
macOS
I did not attempt to reproduce this on Windows, Linux, iOS, or Android, however I believe this problem may still be present on those platforms as well
Target OS version/browser
macOS Monterey Version 12.0.1
Devices
MacBook Pro (16-inch, 2021)
Chip: Apple M1 Max
Memory: 64 GB
Logs

Logs
mackmyers@Macks-MacBook-Pro flutter_text_field % flutter analyze
Analyzing flutter_text_field...
No issues found! (ran in 0.9s)

mackmyers@Macks-MacBook-Pro flutter_text_field % flutter doctor -v
[✓] Flutter (Channel stable, 3.0.5, on macOS 12.0.1 21A559 darwin-arm (Rosetta), locale en-US)
    • Flutter version 3.0.5 at /Users/mackmyers/Documents/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision f1875d570e (9 days ago), 2022-07-13 11:24:16 -0700
    • Engine revision e85ea0e79c
    • Dart version 2.17.6
    • DevTools version 2.12.2

[✓] Android toolchain - develop for Android devices (Android SDK version 31.0.0)
    • Android SDK at /Users/mackmyers/Library/Android/sdk
    • Platform android-31, build-tools 31.0.0
    • Java binary at: /Applications/Android Studio.app/Contents/jre/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.11+0-b60-7772763)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 13.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • CocoaPods version 1.11.3

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2021.1)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.11+0-b60-7772763)

[✓] IntelliJ IDEA Ultimate Edition (version 2021.3.2)
    • IntelliJ at /Applications/IntelliJ IDEA.app
    • Flutter plugin version 67.1.3
    • Dart plugin version 213.7371

[✓] VS Code (version 1.69.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension can be installed from:
      🔨 https://marketplace.visualstudio.com/items?itemName=Dart-Code.flutter

[✓] Connected device (2 available)
    • macOS (desktop) • macos  • darwin-arm64   • macOS 12.0.1 21A559 darwin-arm (Rosetta)
    • Chrome (web)    • chrome • web-javascript • Google Chrome 103.0.5060.134

[✓] HTTP Host Availability
    • All required HTTP hosts are available

• No issues found!

I am also encountering the same bug mentioned above
Demo (Still happening with flutter 3.0.4)

Reference this video for context: https://vimeo.com/manage/videos/733077757

Device info

Macbook Pro (16 inch 2021)
Chip M1 max
Memory 64 gb

Thanks for the detailed report and steps to replicate @mack-at-pieces
I tried your code sample on macOS desktop, web (chrome) and on mobile (ios simulator) and noticed same observation, ie, when we tap in the textfield, the Logging section in devtools shows continuously emits frame events as below:

macOS desktop:

https://user-images.githubusercontent.com/67046386/180724557-02405e49-e7a8-4b45-b844-5badfba03348.mov

web (chrome):

mobile (iOS), wherein, the emit is fast as compared to other platforms:

All these observations are similar to when we use readOnly: false. Is the expected result here, not to emit the frames when readOnly: true ?

Hey @darshankawar - Thanks for getting back to me. That is exactly correct.
The reason is because when the TextField is readOnly: true, there is no visible widget in the UI that would indicate continuous rebuilds & events (like an animation), and therefore as a developer, you don't know this is the case until opening devtools. Once you open devtools, you wonder why your app is emitting continuous events while the UI is completely idle and in my case, this was the cause.
There may also be unintended side effects of this that I am unaware of.

Thanks for the update.
Based on my findings earlier, adding labels as applicable.

 stable, master flutter doctor -v

[✓] Flutter (Channel stable, 3.0.5, on macOS 12.2.1 21D62 darwin-x64, locale
    en-GB)
    • Flutter version 3.0.5 at /Users/dhs/documents/fluttersdk/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision f1875d570e (2 days ago), 2022-07-13 11:24:16 -0700
    • Engine revision e85ea0e79c
    • Dart version 2.17.6
    • DevTools version 2.12.2

[!] Xcode - develop for iOS and macOS (Xcode 12.3)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    ! Flutter recommends a minimum Xcode version of 13.
      Download the latest version or update via the Mac App Store.
    • CocoaPods version 1.11.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] VS Code (version 1.62.0)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.21.0

[✓] Connected device (5 available)
    • SM G975F (mobile)       • RZ8M802WY0X • android-arm64   • Android 11 (API 30)
    • Darshan's iphone (mobile)  • 21150b119064aecc249dfcfe05e259197461ce23 •
      ios            • iOS 14.4.1 18D61
    • iPhone 12 Pro Max (mobile) • A5473606-0213-4FD8-BA16-553433949729     •
      ios            • com.apple.CoreSimulator.SimRuntime.iOS-14-3 (simulator)
    • macOS (desktop)            • macos                                    •
      darwin-x64     • Mac OS X 10.15.4 19E2269 darwin-x64
    • Chrome (web)               • chrome                                   •
      web-javascript • Google Chrome 98.0.4758.80

[✓] HTTP Host Availability
    • All required HTTP hosts are available

! Doctor found issues in 1 category.

[✓] Flutter (Channel master, 3.1.0-0.0.pre.1861, on macOS 12.2.1 21D62
    darwin-x64, locale en-GB)
    • Flutter version 3.1.0-0.0.pre.1861 on channel master at
      /Users/dhs/documents/fluttersdk/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision ca6cecf034 (7 hours ago), 2022-07-25 15:23:08 -0700
    • Engine revision 7f8925b1f6
    • Dart version 2.19.0 (build 2.19.0-33.0.dev)
    • DevTools version 2.15.0

[!] Xcode - develop for iOS and macOS (Xcode 12.3)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    ! Flutter recommends a minimum Xcode version of 13.
      Download the latest version or update via the Mac App Store.
    • CocoaPods version 1.11.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] VS Code (version 1.62.0)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.21.0

[✓] Connected device (5 available)
    • SM G975F (mobile)       • RZ8M802WY0X • android-arm64   • Android 11 (API 30)
    • Darshan's iphone (mobile)  • 21150b119064aecc249dfcfe05e259197461ce23 •
      ios            • iOS 14.4.1 18D61
    • iPhone 12 Pro Max (mobile) • A5473606-0213-4FD8-BA16-553433949729     •
      ios            • com.apple.CoreSimulator.SimRuntime.iOS-14-3 (simulator)
    • macOS (desktop)            • macos                                    •
      darwin-x64     • Mac OS X 10.15.4 19E2269 darwin-x64
    • Chrome (web)               • chrome                                   •
      web-javascript • Google Chrome 98.0.4758.80

[✓] HTTP Host Availability
    • All required HTTP hosts are available

! Doctor found issues in 1 category.

/cc @justinmc @LongCatIsLooong Do we still run some unnecessary AnimationControllers even when there is no blinking cursor in read-only mode?

It looks like we do (after https://github.com/flutter/flutter/pull/106893 on macOS the caret no longer uses animation controllers but still the timer is not disabled when it's readonly)

Hi @darshankawar & team - Is there a timeframe or future Flutter release we could expect this to be addressed?

@darshankawar @LongCatIsLooong @goderbauer @justinmc just want to say thanks 🤝 looking forward to this being live. going to make some Pieces users and team members very happy 😄


Doesn't work for many tidy options

After a bit of testing, I'm starting to realise that this plugin doesn't really work very effectively. The reason behind this seems to be that the flow is as follows:

posthtml parser parses HTML to tree
posthtml-tidy calls render() to retrieve raw markup from posthtml (I'm still unsure whether this is the original markup or whether posthtml is rendering new markup from its tree)
posthtml-tidy runs tidy on the raw markup
posthtml-tidy passes the tidied markup back to posthtml using the parser() function
posthtml's parser() converts the tided markup back to a posthtml tree
posthtml then renders the tree to its own renderers interpretation of HTML markup - this renderer does not respect the tidy options passed, so this step is lossy

Despite the obvious point that the above steps involve a lot of probably unnecessary churn, it looks like posthtml's API may not support the use-case of this plugin at all.
If I have time, I'll investigate further and possible prepare a PR if this is possible, but I suspect it isn't. Open to correction. If it isn't, I'll create an issue on the posthtml repo.

@lucideer Hi, You caught the right direction.
For such cases, a plug-in that extends the native API posthtml will be developed soon and one of the options is to work with the regular parser and the renderer provided by the postchart itself will only modify with the tree.

If I have time, I'll investigate further and possible prepare a PR if this is possible, but I suspect it isn't. Open to correction. If it isn't, I'll create an issue on the posthtml repo.

It will be great if you can offer an PR 👍
I will render you any assistance if you need something just ping me)

It might be a while before I have time to look at a PR, but will do my best as soon as I can.
For now - could this possibly be added as a warning at the top of the README? It took me too long to figure this out.

For now - could this possibly be added as a warning at the top of the README? It took me too long to figure this out.

Yep ) you a heve add this as PR ?


[FLINK-21802][table-planner-blink]LogicalTypeJsonDeserializer/Serializer support custom RowType/MapType/ArrayType/MultisetType

What is the purpose of the change
We should custom RowType/MapType/ArrayType/MultiSetType deserialize/serializer method to allow some special LogicalType such as TimestampType's kind field.
Brief change log

8a320ad LogicalTypeJsonDeserializer/Serializer add corresponding method to support RowType/MapType/ArrayType/MultisetType

Verifying this change
(Please pick either of the following options)
This change is already covered by existing tests, such as LogicalTypeSerdeTest.
Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (no)
The public API, i.e., is any changed class annotated with @Public(Evolving): (no)
The serializers: (no)
The runtime per-record code paths (performance sensitive): (no)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: (no)
The S3 file system connector: (no)

Documentation

Does this pull request introduce a new feature? (no)
If yes, how is the feature documented? (not documented)

@flinkbot run azure

@flinkbot run azure


azure-core prevents native image builds

Integrating Azure libs (mainly azure-core dependency) into Quarkus application prevents my native build to complete successfully.
Workaround suggested in #161 does not solve the issue, probably due to the use of quarkus-resteasy-reactive dependency in pom.xml
Native build error logs look like this:
Error: Classes that should be initialized at run time got initialized during image building:

io.netty.buffer.UnpooledDirectByteBuf the class was requested to be initialized at run time (subtype of io.netty.buffer.AbstractReferenceCountedByteBuf). To see why io.netty.buffer.UnpooledDirectByteBuf got initialized use --trace-class-initialization=io.netty.buffer.UnpooledDirectByteBuf
...
io.netty.buffer.ByteBufAllocator the class was requested to be initialized at run time (from 'META-INF/native-image/io.netty/netty-buffer/native-image.properties' in 'file:///project/lib/io.netty.netty-buffer-4.1.94.Final.jar' with 'io.netty.buffer.ByteBufAllocator' and from feature io.quarkus.runner.Feature.beforeAnalysis with 'ByteBufAllocator.class'). To see why io.netty.buffer.ByteBufAllocator got initialized use --trace-class-initialization=io.netty.buffer.ByteBufAllocator

@fsantagostinobietti Could you pls provide a sample project which is able to reproduce the issue you described in the issue?

@fsantagostinobietti Building a Quarkus native image while integrating Azure libs (e.gl, azure-core dependency) does need some extra configuration, this is exactly the goal this repo tries to achieve. Basically, you have a few options:

Utilize extensions of Azure libs this repo releases including extensions and internal, see samples from integration-tests and https://github.com/quarkiverse/quarkus-opentelemetry-exporter/pull/143 which utilized internal extension quarkus-azure-http-client-vertx.
Reference raw native-image config from azure-sdk-for-java and re-write it using Quarkus way, e.g., https://github.com/Azure/azure-sdk-for-java/tree/main/sdk/core/azure-core-http-netty/src/main/resources/META-INF/native-image/com.azure/azure-core-http-netty.
Open issue(s) in this repo for your specific requirements on what Azure libs you want to build in a Quarkus native image, we'll prioritize accordingly.

I'm going to close the issue as for now and feel free to re-open or open new issues if you have more information and comments.


Appium: Method closeApp() is allowed for iOS only, though it works with Android

What are you trying to achieve?
I want to use I.closeApp() in an Appium test against Android device.
Appium docs for closeApp() says that it's allowed on Android/UiAutomator
What do you get instead?
CodeceptJS allows it for iOS only.
node_modules/codeceptjs/docs/build/Appium.js:
  /**
   * Close the given application.
   *
   * ```js
   * I.closeApp();
   * ```
   *
   * Appium: support only iOS
   */
  async closeApp() {
    onlyForApps.call(this, 'iOS');
    return this.browser.closeApp();
  }

Note that I created the same method (without blocking of Android platform) in my custom helper and it works well with Android:
async closeApp() {
    const browser = this.helpers["Appium"].browser;
    await browser.closeApp();
}

Details

CodeceptJS version: 3.0.5
NodeJS Version: v12.21.0
Operating System: Ubuntu 20.04
Appium 1.15.1
Android Studio 4.1.3 emulator Nexus 9 (Android 9)

Also CodeceptJS docs says that only iOS is supported: https://codecept.io/helpers/Appium.html#closeapp


bug: gv does not respect which-key delay.

Did you check docs and existing issues?

[X] I have read all the which-key.nvim docs
[X] I have updated the plugin to the latest version before submitting this issue
[X] I have searched the existing issues of which-key.nvim
[X] I have searched the existing issues of plugins related to this issue

Neovim version (nvim -v)
0.10.0 (homebrew, stable::)
Operating system/version
macOs 15.5
Describe the bug
Hi,
it seems like that gv(Start Visual mode with the same area as the previous area and the same mode) does not respect the delay set in which-key options. When pressing gv, the which-key popup is shown immediately.
Steps To Reproduce

select some text via visual mode and yank text
press gv
which-key popup is shown immediately, without delay.

Expected Behavior
Use the set delay.
Health
No response
Log
No response
Repro
No response

Can't reproduce. Provide a repro with exact steps to reproduce if you want me to further look into this.

Hi Folke,
its ver strange, it seems happening for only one value of delay, namely 1000.
How to reproduce:
vim.env.LAZY_STDPATH = ".repro"
load(vim.fn.system("curl -s https://raw.githubusercontent.com/folke/lazy.nvim/main/bootstrap.lua"))()

require("lazy.minit").repro({
  spec = {
	  { "folke/which-key.nvim",
		  event = "VeryLazy",
		  opts = {
			  delay = 1000,
		  },

	 },
    -- add any other plugins here
  },
})

for this config the popup does not have any delay, changing the delay value to 1010 or 999 the delay is present as expected.

ok, sow when it's exactly the same as timeoutlen. Will check

yep

just checked, when changing timeoutlen to (exactly) the delay value (no matter what value) the delay is gone.

Thnx for the quick fix!


A restart of the tracker doesn't reset the packets seen / gateways seen

Node-red checks for a reset of the msg counter (due to a restart of the tracker). This should reset the tracking counters as send back to the tracker. This is not the case. Counters continue where they were.

fixed by commit 2943af369fcb69a4a3930493e40bdc51daf14711


Add basil to aws-beanstalk-publisher

Description
As part of JEP-233, I made the decision to leave this plugin behind, but since then a number of users have complained about this plugin being broken, and I have been unsuccessful in convincing any of them to adopt the plugin. Therefore I am planning on adopting this plugin to release the Guava fix in https://github.com/jenkinsci/aws-beanstalk-publisher-plugin/pull/58. I have no actual interest in maintaining this plugin, and I may keep maintaining the POM/BOM and merging/releasing minor bug fixes, but I can't promise major feature development, security fixes, or in-depth code reviews.
https://github.com/jenkinsci/aws-beanstalk-publisher-plugin
Submitter checklist for adding or changing permissions

Always

[x] Add link to plugin/component Git repository in description above

For a newly hosted plugin only

[ ] Add link to resolved HOSTING issue in description above

For a new permissions file only

[ ] Make sure the file is created in permissions/ directory
[ ] artifactId (pom.xml) is used for name (permissions YAML file).
[ ] groupId / artifactId (pom.xml) are correctly represented in path (permissions YAML file)
[ ] Check that the file is named plugin-${artifactId}.yml for plugins

When adding new uploaders (this includes newly created permissions files)

[ ] Make sure to @mention an existing maintainer to confirm the permissions request, if applicable
[ ] Use the Jenkins community (LDAP) account name in the YAML file, not the GitHub account name
[ ] Make sure to @mention the users being added so their GitHub account names are known if they require GitHub merge access (see below).
[ ] All newly added users have logged in to Artifactory at least once

Reviewer checklist (not for requesters!)

[ ] Check this if newly added person also needs to be given merge permission to the GitHub repo (please @ the people/person with their GitHub username in this issue as well). If needed, it can be done using an IRC Bot command
[ ] Check that the $pluginId Developers team has Admin permissions while granting the access.
[ ] In the case of plugin adoption, ensure that the Jenkins Jira default assignee is either removed or changed to the new maintainer.
[ ] If security contacts are changed (this includes add/remove), ping the security officer (currently @Wadeck) in this pull request. If an email contact is changed, wait for approval from the security officer.

There are IRC Bot commands for it

Make sure to @mention an existing maintainer to confirm the permissions request, if applicable

missing cc ^^

cc @DavidTanner can you approve please?

Sure. David Tanner has not responded to any of my pings since last November, but the rules are the rules, so let us follow them and start the two-week timer.


consider adding wget

There is a download package available which acts like wget. Install with package install download.
Then you can simply say:
download https://domain.tld/path/file.ext


Chef-client should support unattended upgrades regardless of init system

Currently, upgrading chef-client can be wrought with danger for users on operating systems using SysV init.  After installation of the new package, the chef-client daemon needs to be restarted.  Simply notifying a service restart (the strategy taken by omnibus_updater, one of the more popular recipes for managing Chef) can leave orphaned chef-client processes.
One possible solution would be to support a new signal that would force the chef-client to re-exec itself after any running convergences were complete.

This should be resolved as of Chef 12.  If you hit this same issue in Chef 12, please open a new issue.


update packages

:x: Build TheIdServer 7.4.7-fix-dependencies1-0167 failed (commit https://github.com/Aguafrommars/TheIdServer/commit/ad51e4a638 by @invalid-email-address)

:x: Build TheIdServer 7.4.7-fix-dependencies1-0170 failed (commit https://github.com/Aguafrommars/TheIdServer/commit/7928960858 by @invalid-email-address)


verify clockSync3.qnt fails with Operator OPER_APP cannot be applied to arguments of types

A bug affecting the Quint -> Apalache transpilation.
ℹ I need to further investigate if this is a shortcoming on the Quint or Apalache side.
🗒 For now, using this issue for note taking.
In clockSync3.qnt, we have
https://github.com/informalsystems/quint/blob/f2994abce791cb2be5d8a67e9b5f51d77f12d7ba/examples/classic/distributed/ClockSync/clockSync3.qnt#L87-L89
msgs has a type of Set[{src : PROC, ts: int}], and hc is typed PROC -> int.
We also have
https://github.com/informalsystems/quint/blob/f2994abce791cb2be5d8a67e9b5f51d77f12d7ba/examples/classic/distributed/ClockSync/clockSync3.qnt#L34-L35
which gets typed (a, b) => {src: a, ts: b}.
So from get/2 on (hc, p) we should infer p: PROC and the result of the application get(hc, p) : int.
The application result of msgFrom(p, hc.get(p)) should be inferred as {src: PROC, ts: int}.
However, in the transpiled ApalacheIR, the type that reaches the typed builder is {src: a, ts: b}, causing it to fail with
Operator OPER_APP cannot be applied to arguments of types (((Set(a), a) => Set(a)), Set({ src: { id: Int }, ts: Int }), { src: a, ts: b })

I thought I had an issue open for this but I can’t find it. Iiuc, this is a symptom of the need to do fresh renaming of type variables when we combine type signatures from different places? Without that, we can end up with type variable collisions.

Jure fixed this in #2620.
We still need to consider the collision I described in https://github.com/informalsystems/apalache/issues/2619#issuecomment-1611332942, but that's a separate issue.


[SPARK-16540][YARN][CORE] Avoid adding jars twice for Spark running on yarn

What changes were proposed in this pull request?
Currently when running spark on yarn, jars specified with --jars, --packages will be added twice, one is Spark's own file server, another is yarn's distributed cache, this can be seen from log:
for example:
./bin/spark-shell --master yarn-client --jars examples/target/scala-2.11/jars/scopt_2.11-3.3.0.jar

If specified the jar to be added is scopt jar, it will added twice:
...
16/07/14 15:06:48 INFO Server: Started @5603ms
16/07/14 15:06:48 INFO Utils: Successfully started service 'SparkUI' on port 4040.
16/07/14 15:06:48 INFO SparkUI: Bound SparkUI to <IP_ADDRESS>, and started at http://<IP_ADDRESS>:4040
16/07/14 15:06:48 INFO SparkContext: Added JAR file:/Users/sshao/projects/apache-spark/examples/target/scala-2.11/jars/scopt_2.11-3.3.0.jar at spark://<IP_ADDRESS>:63996/jars/scopt_2.11-3.3.0.jar with timestamp<PHONE_NUMBER>637
16/07/14 15:06:49 INFO RMProxy: Connecting to ResourceManager at /<IP_ADDRESS>:8032
16/07/14 15:06:49 INFO Client: Requesting a new application from cluster with 1 NodeManagers
16/07/14 15:06:49 INFO Client: Verifying our application has not requested more than the maximum memory capability of the cluster (8192 MB per container)
16/07/14 15:06:49 INFO Client: Will allocate AM container, with 896 MB memory including 384 MB overhead
16/07/14 15:06:49 INFO Client: Setting up container launch context for our AM
16/07/14 15:06:49 INFO Client: Setting up the launch environment for our AM container
16/07/14 15:06:49 INFO Client: Preparing resources for our AM container
16/07/14 15:06:49 WARN Client: Neither spark.yarn.jars nor spark.yarn.archive is set, falling back to uploading libraries under SPARK_HOME.
16/07/14 15:06:50 INFO Client: Uploading resource file:/private/var/folders/tb/8pw1511s2q78mj7plnq8p9g40000gn/T/spark-a446300b-84bf-43ff-bfb1-3adfb0571a42/__spark_libs__6486179704064718817.zip -> hdfs://localhost:8020/user/sshao/.sparkStaging/application_1468468348998_0009/__spark_libs__6486179704064718817.zip
16/07/14 15:06:51 INFO Client: Uploading resource file:/Users/sshao/projects/apache-spark/examples/target/scala-2.11/jars/scopt_2.11-3.3.0.jar -> hdfs://localhost:8020/user/sshao/.sparkStaging/application_1468468348998_0009/scopt_2.11-3.3.0.jar
16/07/14 15:06:51 INFO Client: Uploading resource file:/private/var/folders/tb/8pw1511s2q78mj7plnq8p9g40000gn/T/spark-a446300b-84bf-43ff-bfb1-3adfb0571a42/__spark_conf__326416236462420861.zip -> hdfs://localhost:8020/user/sshao/.sparkStaging/application_1468468348998_0009/__spark_conf__.zip
...

So here try to avoid adding jars to Spark's fileserver unnecessarily.
How was this patch tested?
Manually verified both in yarn client and cluster mode, also in standalone mode.

Test build #62302 has started for PR 14196 at commit 86205fc.

Test build #62302 has finished for PR 14196 at commit 86205fc.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62302/
Test PASSed.

I was gonna say the Utils method now doesn't need to be called from SparkContext, which would avoid the new argument, but it does the parsing from a String into a Set, so that avoids some duplication. LGTM, merging into master / 2.0.
(Mental note: eventually all these should be config constants so they can use the common parsing code in ConfigBuilder.scala...)


Update tab-sso-graph-api.md

Acrolinx Scorecards
We currently enforce a minimum score of 80.
Click the scorecard links for each article to review the Acrolinx feedback on grammar, spelling, punctuation, writing style, and terminology:

Article
Score
Issues
Scorecard
Processed

msteams-platform/tabs/how-to/authentication/tab-sso-graph-api.md
91
13
link
:white_check_mark:

More info about Acrolinx
Use the Acrolinx extension, or sidebar, in Visual Studio Code to check spelling, grammar, style, tone, clarity, and key terminology when you're creating or updating content. For more information, see Use the Visual Studio Code extension to run Acrolinx locally.

Learn Build status updates of commit bda7318:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

msteams-platform/tabs/how-to/authentication/tab-sso-graph-api.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the learn.microsoft.com contributor guidesPost your question in the Learn support channel

Acrolinx Scorecards
We currently enforce a minimum score of 80.
Click the scorecard links for each article to review the Acrolinx feedback on grammar, spelling, punctuation, writing style, and terminology:

Article
Score
Issues
Scorecard
Processed

msteams-platform/tabs/how-to/authentication/tab-sso-graph-api.md
91
13
link
:white_check_mark:

More info about Acrolinx
Use the Acrolinx extension, or sidebar, in Visual Studio Code to check spelling, grammar, style, tone, clarity, and key terminology when you're creating or updating content. For more information, see Use the Visual Studio Code extension to run Acrolinx locally.

Learn Build status updates of commit 7eb9da1:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

msteams-platform/tabs/how-to/authentication/tab-sso-graph-api.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the learn.microsoft.com contributor guidesPost your question in the Learn support channel

@ydogandjiev , @iambmelt - could you please review this change?

@ydogandjiev , @iambmelt - could you please review this change?

A gentle reminder

Acrolinx Scorecards
We currently enforce a minimum score of 80.
Click the scorecard links for each article to review the Acrolinx feedback on grammar, spelling, punctuation, writing style, and terminology:

Article
Score
Issues
Scorecard
Processed

msteams-platform/tabs/how-to/authentication/tab-sso-graph-api.md
91
13
link
:white_check_mark:

More info about Acrolinx
Use the Acrolinx extension, or sidebar, in Visual Studio Code to check spelling, grammar, style, tone, clarity, and key terminology when you're creating or updating content. For more information, see Use the Visual Studio Code extension to run Acrolinx locally.

Acrolinx Scorecards
We currently enforce a minimum score of 80.
Click the scorecard links for each article to review the Acrolinx feedback on grammar, spelling, punctuation, writing style, and terminology:

Article
Score
Issues
Scorecard
Processed

msteams-platform/tabs/how-to/authentication/tab-sso-graph-api.md
90
16
link
:white_check_mark:

More info about Acrolinx
Use the Acrolinx extension, or sidebar, in Visual Studio Code to check spelling, grammar, style, tone, clarity, and key terminology when you're creating or updating content. For more information, see Use the Visual Studio Code extension to run Acrolinx locally.


Support repo format where each file has its own separate chain of commits

git rebase takes a long time to rewrite history, which may start becoming unacceptable for tutorials as large as lua-tutorial. However, large projects like lua tend to be already split up into many files, and if a tutorial only makes changes to one file per step (as I think they should), there's no reason to spend time rewriting the entire git history since there can only be conflicts with other steps that change that one file.
So the git repo would consist of multiple roots/branches, each one referencing the chain of commits that modify one particular file. There would have to be some kind of manifest file somewhere that would specify the order of steps between all the different files, which leg would have to keep track of (I don't think the user would ever want to have to deal with that themselves).
There should still be the option of using the normal git format, and the ability to switch between the two, in case users want a repo they can go in and edit by hand.

Changing multiple files per step could actually be supported, by having the manifest file map multiple commits to a single step. I'm thining the manifest file would be a text file where each line would describe a step, in order. So each line would have a list of commit hashes that are part of that step.
Also, I think the "normal git format" should be a separate Representation, which can be gotten by running leg export, and imported back in with leg import.


feat(docs): Note for path-style-access with Minio

Common gotcha among people installing Spinnaker

@dorbin
I don't have a strong opinion on where it should be displayed in the page structure as long as the user understand it's an optional step depending on how Minio was configured by the user. Feel free to move it where you think it belongs.


Investigate gitlab repositories for javacard projects

No repositories found on gitlab as of 2022-01-19
Search string https://gitlab.com/search?scope=projects&search=javacard returns no hits


[CONTP-500] Including the internal prefix in tagger for filtering

What does this PR do?
Addresses a bug in the new 7.60 agent tagger implementation. Ensures that the internal prefix for the global entity (global tags) is properly fetched when filters are in use.
Motivation
Missing prefix in the EntityIDPrefix list. Noticed missed entity when executing tagger-list.
Describe how to test/QA your changes
Spin up the cluster agent locally and execute tagger-list
k exec -it datadog-agent-linux-cluster-agent-5d889975fc-wwcmk -- agent tagger-list
=== Entity internal://global-entity-id ===
== Source workloadmeta-static =
=Tags: [kube_cluster_name:gabedos-v13102-cluster]
===

Possible Drawbacks / Trade-offs
N/A
Additional Notes

We should be defining the Internal prefix in entity_id.go as an EntityIDPrefix to maintain consistency as to how EntityIDPrefix is used from entity_id.go to entity_id_builder.go.

/merge

/merge


Fix .poly file parsing and add support for multiple polygons

First problem was that latitude and longitude were reverted, so now it parses lon 1st
I also added support for parsing multiple polygons

Here is example:
https://download.geofabrik.de/europe/liechtenstein.poly
Download that file and try to use it on https://protomaps.com/extracts
Hopefully after this change it will work...

Merged! thank you, I also fixed up two test cases to handle lat/lon and multiple outer loops https://github.com/protomaps/OSMExpress/blob/master/test/test_region.cpp#L178

Download that file and try to use it on https://protomaps.com/extracts

That site only takes GeoJSON as an input right now and .poly files won't work; do you expect to use those more often? If so I can look into adding that as an option.

Thank you for adding tests, I didn't notice there are tests...
Usually .poly file are more readily available like and seems more standard input for extract...
But no need to do it for me, there are websites/tools where you can do conversions easily...
I do have additional question... There is no API for https://protomaps.com/extracts right?
I'm working on tool for semi-automatic importing, where tool creates .osc file to be loaded into JOSM, and I want to apply imported data on fresh OSM data, areas is smalls <100k nodes, and this is only tool that can do the job, working on setting up my own instance, dealing mostly on how to limit updates and stuff to Slovenia...

There is an API, depending on your frequency and nature of use (commercial, hobby project, etc) I can grant you a key to access it - please email me<EMAIL_ADDRESS>if you would like details.
The other options are to run your own planet.osmx with minutely or daily replication from planet.openstreetmap.org, or you can use just https://download.geofabrik.de/europe/slovenia.html which has daily osm.pbf or diffs, it would be much less resources to just use a country extract instead of the planet which requires >1TB of fast disk.

Problem with using https://download.geofabrik.de/europe/slovenia.html is, that I want minutely updates... and there are none for Slovenia, I was planning to use europe and than pass each .osc.gz to osmconvert to filter it before passing to osmx to do updating...
Sending email soon...


Add snip_momentum structured pruning example with 80% sparsity ratio

This PR is used to demonstrate the functionality of snip_momentum structured pruning algo implemented in here.
User can reproduce below result by running source ./bash_script/pruning_sparse_snip_momentum.sh with the PR mentioned at above.

pattern
sparsity ratio
pruning method
epochs
acc & mm-acc

1x1
80%
DeepSpeed L1
2
0.8113/0.822

1x1
80%
Snip_momentum
2
0.8176/0.822

4x1
80%
snip_momentum
10
0.8248/0.8305

@ftian1, thanks for your patience. I have tested the method and it looks good. I will do some minor changes for the checkpoint saving.

@xiaoxiawu-microsoft  appreciate for your effort


EventGrid integration

Please add some information about EventGrid integration and the prerequisites that must be met.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: b7ee5166-ff40-f4ed-ce07-5312f935e388
Version Independent ID: 68f01b43-e3b2-003b-24cb-446a08aa0c1b
Content: Create event-based triggers in Azure Data Factory - Azure Data Factory
Content Source: articles/data-factory/how-to-create-event-trigger.md
Service: data-factory
GitHub Login: @chez-charlie
Microsoft Alias: chez

@patsql we'll review your feedback and get back to you shortly. Thanks.

@patsql Thanks for the feedback! I have assigned the issue to the content author @chez-charlie  to evaluate and add information about EventGrid integration  in the document.

Pre-requisite: The integration described in this article depends on Azure Event Grid. Make sure that your subscription is registered with the Event Grid resource provider. For more info, see Resource providers and types. You must be able to do the Microsoft.EventGrid/eventSubscriptions/* action. This action is part of the EventGrid EventSubscription Contributor built-in role.

#close


[motion-controls] No thumbstick navigation on Quest 3 after Aframe update from v1.5 to v1.6

After updating from Aframe 1.5 to Aframe 1.6, the movement via the thumbsticks on the Quest 3 no longer works (WASD movement works still fine). I don't have this problem with Aframe 1.5, version 7.2 of the aframe-extras and version 7.5.2 work perfectly.
Is the problem known? Thanks for any hints/advice!
Quest 3 v72
Aframe 1.6
aframe-extras 7.5.2

Last time I checked 2 weeks ago while updating https://github.com/AdaRoseCannon/aframe-xr-boilerplate aframe 1.6.0 aframe-extras 7.5.2 and I think I'm currently on Quest 3 v71 it worked well.

Thank you Vincent, you gave me the right direction, yes everything works fine, something was wrong with my aframe1.6 implementation, using

solved my problem
best regards!


403 Forbidden

Tried using this extension with default settings, but was prompted to enter credentials fort a Wordpress site.  To try to resolve plus as my primary is OZNZB, I entered "https://www.oznzb.com/search/" for the base subpath.  When I try it now, it simply goes that link and gives me a 403 forbidden.
Tried leaving Omdb data field blank as well as trying few things but alwayw gtot 403 result (even when logged into site in different tab.)  What variables do I need to use and if that doesn't work, how do I fix?
BTW - the base path + string when searching OZNZB for "hbo doc" is "https://www.oznzb.com/search/hbo doc" if that helps.
Thanks

Your approach sounds correct. The only missing piece is that the base path needs a %s so that the plugin knows where to put the movie name/year.So, for your indexer, the setting would be:https://www.oznzb.com/search/%s
You can leave the rest of the settings as default. OMDB is used to look up the movie in the OMDB database, so that movies with common titles don't return irrelevant results.
  From: BzowK<EMAIL_ADDRESS>
To: applestoapples/AddNzbSearchLinkToIMDb<EMAIL_ADDRESS>Sent: Monday, July 6, 2015 11:56 AM
Subject: [AddNzbSearchLinkToIMDb] 403 Forbidden (#1)
Tried using this extension with default settings, but was prompted to enter credentials fort a Wordpress site. To try to resolve plus as my primary is OZNZB, I entered "https://www.oznzb.com/search/" for the base subpath. When I try it now, it simply goes that link and gives me a 403 forbidden. Tried leaving Omdb data field blank as well as trying few things but alwayw gtot 403 result (even when logged into site in different tab.) What variables do I need to use and if that doesn't work, how do I fix?BTW - the base path + string when searching OZNZB for "hbo doc" is "https://www.oznzb.com/search/hbo doc" if that helps.Thanks—
Reply to this email directly or view it on GitHub.

Your approach sounds correct. The only missing piece is that the base path needs a %s so that the plugin knows where to put the movie name/year.
So, for your indexer, the setting would be:
https://www.oznzb.com/search/%s
You can leave the rest of the settings as default. OMDB is used to look up movie metadata (e.g., year) in the OMDB database, so that movies with common titles don't return irrelevant results.


Insecure hashing algorithm MD5 used

Please Use only secure hashing algorithms. Examples of secure hashing are SHA-256, SHA-512⁄256,
SHA-384, SHA-512, SHA3-256, SHA3-384, SHA3-512.

Is there a workaround for this issue?


Add support for MSVC 2022 series

Visual Studio 2022, which wraps the 17.x compiler series and an unknown toolset versions (v142 is still listed as "latest", that's the one that came with VS 2019/16.x) is in preview now.  SCons will need some work to support it, since it has to know some things at least about version numbers.
Note for development during pre-release period - vswhere, which SCons relies on for recent compilers, will not report unreleased versions without the -prerelease switch.

The Visual Studio versions detected via vswhere and the latest installed VC toolset versions are shown in the table below as of 2021-09-11:

Visual Studio
VC Ver
VC Ext
VS Version
VC Toolset

2022 Community
14.3
14.3Com
17.0.31612.314
14.30.30423

2022 Build Tools
14.3
14.3BT
17.0.31612.314
14.30.30423

2019 Community
14.2
14.2Com
16.11.31624.102
14.29.30133

2019 Build Tools
14.2
14.2BT
16.11.31624.102
14.29.30133

2017 Community
14.1
14.1Com
15.9.28307.1622
14.16.27023

2017 Build Tools
14.1
14.1BT
15.9.28307.1622
14.16.27023

2017 Express
14.1Exp
14.1Exp
15.9.28307.1622
14.16.27023

The vc toolset version number for 2022 is 14.30.30423 so 14.3 (v143) is likely correct.
SCons vswhere query configuration with 2022 (untested via SCons):
_VCVER_TO_VSWHERE_VER = {
    '14.3': [
        ["-prerelease", "-version", "[17.0, 18.0)", ], # default: Enterprise, Professional, Community  (order unpredictable?)
        ["-prerelease", "-version", "[17.0, 18.0)", "-products", "Microsoft.VisualStudio.Product.BuildTools"], # BuildTools
        ],
    '14.2': [
        ["-version", "[16.0, 17.0)", ], # default: Enterprise, Professional, Community  (order unpredictable?)
        ["-version", "[16.0, 17.0)", "-products", "Microsoft.VisualStudio.Product.BuildTools"], # BuildTools
        ],
    '14.1':    [
        ["-version", "[15.0, 16.0)", ], # default: Enterprise, Professional, Community (order unpredictable?)
        ["-version", "[15.0, 16.0)", "-products", "Microsoft.VisualStudio.Product.BuildTools"], # BuildTools
        ],
    '14.1Exp': [
        ["-version", "[15.0, 16.0)", "-products", "Microsoft.VisualStudio.Product.WDExpress"], # Express
        ],
}

Local command-line vswhere command and truncated output:
"C:\program files (x86)\microsoft visual studio\installer\vswhere" -prerelease -version [17.0,18.0)

installationName: VisualStudioPreview/17.0.0-pre.3.1+31612.314
installationVersion: 17.0.31612.314
productId: Microsoft.VisualStudio.Product.Community

"C:\program files (x86)\microsoft visual studio\installer\vswhere" -prerelease -version [17.0,18.0) -products Microsoft.VisualStudio.Product.BuildTools

installationName: VisualStudioPreview/17.0.0-pre.3.1+31612.314
installationVersion: 17.0.31612.314
productId: Microsoft.VisualStudio.Product.BuildTools

This looks like good stuff... I'll check it out presently on my local setups, I have a VM with 2022-only, and a full system with several versions installed including 2022.  I can't see why the above wouldn't work on a test basis.
The question, I guess, is how to handle the period in between now and when 2022 is actually released, so the -prerelease switch is no longer needed? Do we do an scons version that has some sort of flag or envionment var to enable the pre-rel verision, or wait until it's released?  @bdbaddog?

I guess we can add a MSVC_CHECK_PRERELEASE variable to add that flag, and add the logic to support the new versions?

I'm a little fuzzy on the details, but I believe the default version of MSVC is the one with the latest/largest version number.  This would make a pre-release version of MSVC 14.3 the "default MSVC version".
In the current implementation, the "first vswhere query match wins".  The release version queries could be added before the pre-release version queries as follows:
_VCVER_TO_VSWHERE_VER = {
    '14.3': [
        ["-version", "[17.0, 18.0)", ], # default: Enterprise, Professional, Community  (order unpredictable?)
        ["-version", "[17.0, 18.0)", "-products", "Microsoft.VisualStudio.Product.BuildTools"], # BuildTools
        ["-prerelease", "-version", "[17.0, 18.0)", ], # default: Enterprise, Professional, Community  (order unpredictable?)
        ["-prerelease", "-version", "[17.0, 18.0)", "-products", "Microsoft.VisualStudio.Product.BuildTools"], # BuildTools
        ],
    '14.2': [
        ["-version", "[16.0, 17.0)", ], # default: Enterprise, Professional, Community  (order unpredictable?)
        ["-version", "[16.0, 17.0)", "-products", "Microsoft.VisualStudio.Product.BuildTools"], # BuildTools
        ],
    '14.1':    [
        ["-version", "[15.0, 16.0)", ], # default: Enterprise, Professional, Community (order unpredictable?)
        ["-version", "[15.0, 16.0)", "-products", "Microsoft.VisualStudio.Product.BuildTools"], # BuildTools
        ],
    '14.1Exp': [
        ["-version", "[15.0, 16.0)", "-products", "Microsoft.VisualStudio.Product.WDExpress"], # Express
        ],
}

There would be no maintenance once 14.3 is released.
However, there might be unintended consquences if multiple versions of the release editions and pre-release editions are installed at the same time.  For example, this would prefer a release version of the Build Tools over a pre-release version of the Community edition.
P.S.: I'm not saying this is a good idea.

A working SCons branch with support for VS 2022 is in my GitHub repository at:
https://github.com/jcbrill/scons/tree/jbrill-msvs-2022
As discussed above, the prerelease versions are enabled via an environment variable:
# Enable prerelease version(s) via vswhere query argument.
# When enabled, an installed prerelease version will likely be the default msvc version.
_MSVC_CHECK_PRERELEASE = os.environ.get('MSVC_CHECK_PRERELEASE') in ('1', 'true', 'True')

_VCVER = [
    "14.3",
    "14.2",
    ...
    ]

_VCVER_TO_VSWHERE_VER = {
    '14.3': [
        ["-version", "[17.0, 18.0)"], # default: Enterprise, Professional, Community  (order unpredictable?)
        ["-version", "[17.0, 18.0)", "-products", "Microsoft.VisualStudio.Product.BuildTools"], # BuildTools
        ] + [
        # TODO: remove after VS 2022 is released
        ["-prerelease", "-version", "[17.0, 18.0)"], # default: Enterprise, Professional, Community  (order unpredictable?)
        ["-prerelease", "-version", "[17.0, 18.0)", "-products", "Microsoft.VisualStudio.Product.BuildTools"], # BuildTools
        ] if _MSVC_CHECK_PRERELEASE else [],
    '14.2': [

Notes:

To enable prerelease versions from a console/command window:
set MSVC_CHECK_PRERELEASE=1

If the MSVC_CHECK_PRERELEASE environment variable is considered "internal use only", then there is no maintenance required once 2022 is released (i.e., there is no hurry to remove the prerelease vswhere queries as they should not be enabled).

A release version will be used before a prerelease version.

The version '14.3' was added to Tools\msvc.xml.  Should this make it into an SCons release before 2022 is released, the documentation would indicated that 14.3 is supported.  No changes would be necessary once 2022 is actually released.

The added test script test\MSVS\test-14.3-exec.py failed on the test computer and in a vmware vm with the following message: Visual Studio has encountered an unexpected error.
The target executable was successfully built.  There is a hidden empty folder tree following failure.  Curiously, when adding either /RunExit or /Upgrade after /Build Release in the test script command argument list, the build is successful.  I'm not sure what else to do at this point.
The generated SConstruct file, code, and visual studio files build successfully when moved to another folder and scons is invoked from the command line.

With the exception of test\MSVS\test-14.3-exec.py, the following tests are either skipped or pass:

SCons\Tool\MSCommon\vcTests.py
SCons\Tool\msvsTests.py
Test\MSVC\*.py
Test\MSVS\*.py

Unrelated to VS 2022, import os was added to the test-N.N-exec.py scripts for MSVS versions 6.0, 7.0, and 7.1 to fix a NameError exception raised in each script.

A PR can be issued if desired.

@jcbrill - Please PR. I may have a few minor changes with out MSVC_CHECK_PRERELEASE is implemented, but if you leave the PR such that I can update your branch should be able to get this merged fairly quickly.

I use Visual Studio 2019 16.11.5 Preview 1
There always exists newer Preview version than release ver. and enabling by MSVC_CHECK_PRERELEASE is not a good idea.
People report other projects that use scons that it doesn't build because it does not support Visual Studio Preview version.
Maybe use this for solution:
    '14.3': [
        ["-version", "[17.0, 18.0)"], # default: Enterprise, Professional, Community  (order unpredictable?)
        ["-version", "[17.0, 18.0)", "-products", "Microsoft.VisualStudio.Product.BuildTools"], # BuildTools
        ["-version", "[17.0, 18.0)", "-prerelease"], # for Preview version
        ],
    '14.2': [
        ["-version", "[16.0, 17.0)"], # default: Enterprise, Professional, Community  (order unpredictable?)
        ["-version", "[16.0, 17.0)", "-products", "Microsoft.VisualStudio.Product.BuildTools"], # BuildTools
        ["-version", "[16.0, 17.0)", "-prerelease"], # for Preview version
        ],

As of the latest VS2019 update to 16.11.5 on October 12, 2021, it appears that there will be no more updates to the VS2019 Preview version. At this point in time, there seems to be little utility in extending the pre-release detection to VS2019.  See the notes below for sources and details.
However, your point is taken with respect to VS2022 and VS2022 Preview.  I, for one, was unaware that the preview versions continued to be extended after the release version was introduced.
The solution in the previous post would work provided VS2022 and VS2020 Preview versions are not installed at the same time.  Should instances of VS2022 and VS2022 Preview both be installed at the same time, the preview version would never be chosen.  Nor would there be a mechanism to override the "first detected wins" in the current implementation.
The environment variable approach (or possibly a command-line argument) is a mechanism to alter the "first detected wins" behavior by ordering the vswhere queries in such a manor to achieve the desired outcome.  That said, the implementation as merged into the main branch earlier extends the query list by favoring release versions over pre-release versions when the environment variable is set in the presence of both release and pre-release/preview installations.  This made sense when assuming once a production version was released there was no utility in choosing the pre-release/preview version in the presence of both installations.  It appears that this assumption could be problematic.
One fairly straightforward approach would be to change the query list order to favor preview/pre-release versions over release versions when the environment variable is set.  This would allow a preview/pre-release version to be selected in the presence of an installed release version via the environment variable.  If the environment variable is not set, the release version would be selected. In this approach, the environment variable should be public, documented, and probably renamed (e.g., SCONS_MSVC_PREVIEW).  I would prefer that all SCons public environment variables begin with "SCONS_" but that is a problem for another day.
Preview/pre-release versions are not intended for production.  The requirement to set an environment variable does not seem that onerous when using a pre-production instance of VS and has the advantage of not requiring any changes to sconscript/sconstruct files when switching between release and preview/pre-release versions.
Another alternative would be to add a specific symbol for preview/pre-release versions of VS2022 (e.g., 14.3Pre).  This would require changing the value passed to the SCons environment to select a preview/pre-release version.  The implementation is likely straightforward given the current version detection implementation.
My 2 cents, take it for what its worth. As always, I could be wrong.
Joe
Visual Studio 2019 Notes:

Visual Studio 2019 v16.11 Release Notes:

16.11.5 was released on October 12, 2021.
Post dated October 26, 2021 (emphasis added):
https://docs.microsoft.com/en-us/visualstudio/releases/2019/release-notes#16.11.5
Visual Studio 2019 version 16.11 is the fifth and final supported servicing baseline for Visual Studio 2019. Enterprise and Professional customers needing to adopt a long term stable and secure development environment are encouraged to standardize on this version.

Visual Studio 2019 v16.11 Preview Release Notes:

Post dated October 26, 2021 (emphasis added):
https://docs.microsoft.com/en-us/visualstudio/releases/2019/release-notes-preview
With the release of version 16.11, we are no longer offering feature updates to Visual Studio 2019 Preview. To continue previewing new Visual Studio features, we recommend you install Visual Studio 2022 Preview. Use the buttons below to install. We also recommend that you install Visual Studio 2022 Preview before you uninstall Visual Studio 2019 Preview.

@jcbrill - you misunderstood me when I initially suggested that an Environment variable should enable selecting pre-release software. I didn't mean a shell environment variable, but rather SCons' Environment().
As such no need to prefix such variable with SCONS_
In general SCons should not be controlled by shell's environment.
There are a couple exceptions:

SCONS_LIB_DIR
SCONSFLAGS
SCONS_CACHE_MSVC_CONFIG

The last doesn't select the MSVC version to be used, just speeds up detecting the versions available if one can expect no new installations since SCons was last run without it.
So if we were to add a command line option to specify preprelease msvc, a user could add that to SCONSFLAGS, but really the preferred method is to have reasonable defaults in your SConstruct.
In general, if you want a specific version you should be explicit about requesting that in your SConstructs.
So the issue being currently discussed is allowing SCons to detect any installed pre-release MSVS's.
Which it currently can't. Additionally, it should be possible to explicitly prefer any version (including pre-release), which isn't possible for pre-release yet.
That said at this point none of this is needed for initial MSVC2022 release.
So it's probably worthwhile to open a new issue to discuss allowing SCons to select prerelease versions of MSVC explicitly, or as the preferred default (these are two different behavoirs)
I'll add an issue later today, please move discussion there.

@bdbaddog - The code merged in #4030 uses a shell environment variable to enable pre-release versions of VS2022 as described in https://github.com/SCons/scons/pull/4030#issuecomment-934429159

@bdbaddog - Just a reminder, if you are going to remove the environment variable and the prerelease query prior to release, two minor edits to CHANGES.TXT and RELEASE.TXT are probably necessary as well:

Remove the following in RELEASE.TXT:

Add support for pre-release versions of Visual Studio 2022 via an environment variable.

Remove the parenthetical content in the following CHANGES.TXT entry:

Add support for Visual Studio 2022 (release and prerelease).

If you would would like me to make the changes, let me know.
VS 2022 is being released in 4 days (Nov 8) so there is no harm in excising the prerelease code now.

@jcbrill
Thanks for the reminder. I'll make the edits.
My revamp plan is to just call vswhere once always with the prerelease flag, and then store the results (vswhere can output json).  Then the flag in Environment() would indicate if SCons should consider those items marked pre-release.
Hoping to get to this today.

that logic will all go away before release and it will be revamped to better handle prerelease.
Choosing a prerelease by default without explicit user request would be against SCons standards.
So if user had VS2019 for production and 2022 preview to work on C++20 features, if choosing prerelease by default happened, the wrong version of the compiler would be chosen.
So, firstly that shell env variable will be removed prior to release.
Secondly, support for VS2022 non-preview will be in next release (soon).
Then I'll revamp MSVC detection and have an Environment() variable to allow use of prerelease.
That should still work for your usage @yurit85, you'll just have to add one line to your SConstruct.

@bdbaddog
I've done the vswhere to json a couple of different times now both in the scons source tree and as an external stand-alone script.  If interested, I could probably find the code snippets.
For example, the following function was once added to vc.py after msvc_find_vswhere and before find_vc_pdir_vswhere:
def vswhere_query_json(vswhere_args, env=None):
    """ Find MSVC instances using the vswhere program.

    Args:
        vswhere_args: query arguments passed to vswhere
        env: optional to look up VSWHERE variable

    Returns:
        json output or None

    """

    if env is None or not env.get('VSWHERE'):
        vswhere_path = msvc_find_vswhere()
    else:
        vswhere_path = env.subst('$VSWHERE')

    if vswhere_path is None:
        debug("vswhere path not found")
        return None

    debug('VSWHERE = %s' % vswhere_path)

    vswhere_cmd = [vswhere_path] + vswhere_args + ['-format', 'json']

    debug("running: %s" % vswhere_cmd)

    #cp = subprocess.run(vswhere_cmd, capture_output=True)  # 3.7+ only
    cp = subprocess.run(vswhere_cmd, stdout=PIPE, stderr=PIPE)
    if not cp.stdout:
        debug("no vswhere information returned")
        return None

    vswhere_output = cp.stdout.decode("mbcs")
    if not vswhere_output:
        debug("no vswhere information output")
        return None

    try:
        vswhere_json = json.loads(vswhere_output)
    except json.decoder.JSONDecodeError:
        debug("json decode exception loading vswhere output")
        vswhere_json = None

    return vswhere_json

Similarly, to process the json output in my current WIP stand-alone script for processing detected versions (ignore everything that does not apply to internal scons):
    @classmethod
    def vs_process_json_output(cls, vs_rel, vswhere_output, vs_cfg):

        msvs_instances = []

        for instance in vswhere_output:

            #print(json.dumps(instance, indent=4, sort_keys=True))

            productId = instance.get('productId','')
            if not productId:
                debug('productId not found in vswhere output')
                continue

            installationPath = instance.get('installationPath','')
            if not installationPath:
                debug('installationPath not found in vswhere output')
                continue

            installationVersion = instance.get('installationVersion','')
            if not installationVersion:
                debug('installationVersion not found in vswhere output')
                continue

            major = installationVersion.split('.')[0]
            if major != vs_cfg.vs_major:
                raise InternalError('Internal error: msvs major version mis-match: %s, %s' % (major, vs_cfg.vs_major))

            component_id = productId.split('.')[-1]
            if component_id not in vs_cfg.vs_products:
                debug('ignore component_id:%s' % component_id)
                continue

            component_rank = _VSDetectConst.MSVC_COMPONENTID_SELECTION_RANKING.get(component_id,0)
            if component_rank == 0:
                # JCB|TODO: exception and/or stderr?
                debug('unknown component_id:%s' % component_id)
                continue

            isPrerelease = 1 if instance.get('isPrerelease', False) else 0
            isRelease = 0 if isPrerelease else 1

            vs_root = _VSDetectUtil.process_path(installationPath)

            if vs_root in vs_rel.msvs_roots:
                continue

            vs_rel.msvs_roots.add(vs_root)

            msvs_base = MSVSBase._factory(

                title = vs_cfg.vs_title,

                root_dir = vs_root,
                root_version = installationVersion,

                vc_release = isRelease,

                vc_version = vs_cfg.vc_version,
                vc_runtime = vs_cfg.vc_runtime,

                vc_component_id = component_id,
                vc_component_rank = component_rank,

            )

            msvs_instance = MSVSInstance._factory(

                msvs_base = msvs_base,

                vs_dir = vs_root,
                vs_version = installationVersion,

                vs_executable = _VSDetectUtil.vs_executable(vs_root, vs_cfg.vs_executable),
                vs_script = _VSDetectUtil.vs_batchfile(vs_root, [vs_cfg.vs_batchfile]),
                vs_script_errs = vs_cfg.vs_script_errs,

                vs_environ_vars = vs_cfg.vs_environ_vars,
                vs_preserve_vars = vs_cfg.vs_preserve_vars,

            )

            msvs_instances.append(msvs_instance)

        msvs_instances = sorted(msvs_instances, key=cmp_to_key(MSVSInstance._default_order))

        return msvs_instances

The above is from a WIP stand-alone script that does MSVS detection for all versions from 6.0 to 14.3 and builds a data structure for all toolsets and individual tools.
On my current box, it detects 17 installed instances of VS/VC, 30 toolsets, and 170 individual tools.
It might save you some time just by seeing how not to do it :)
Happy to share, let me know.

Adding this content here is not ideal. Lemme create another issue.

Please move ongoing discussion to Issue #4048

Should we close?

MSVC2022 support will be in next release.
Enhancements beyond that (more flexible selection of actual compiler,etc) will come later.


Consider adding FlatpickrBlazor

Hey there,
i´ve created a (yet simple) but working overlay for Flatpickr.js. In the coming days i will add more features and options.
FlatpickrBlazor Repo

With Nuget Repo at https://www.nuget.org/packages/FlatPickrBlazor

@AdrienTorris did you forget this?


Add analysis submodule

Add analysis submodule

Add get_source_rms function
Add dynamic_range function that calculates the dynamic range as a function of the maximum pixel value of an image and the RMS of the source

Add rms function to radiotools.utils

Codecov Report
Attention: Patch coverage is 92.85714% with 2 lines in your changes missing coverage. Please review.

Project coverage is 47.00%. Comparing base (180195c) to head (c327ce7).

Files with missing lines
Patch %
Lines

radiotools/utils/utils.py
66.66%
1 Missing and 1 partial :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##             main       #8      +/-   ##
==========================================
+ Coverage   44.51%   47.00%   +2.48%
==========================================
  Files          10       11       +1
  Lines         474      500      +26
  Branches       54       55       +1
==========================================
+ Hits          211      235      +24
- Misses        252      253       +1
- Partials       11       12       +1

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.