CANNOT_READ_ALL_DATA when calling end()

Hi! I'm having this kind of issue, already encountered by someone else:
ERROR  bad response: Code: 33. DB::Exception: Cannot read all data. Bytes read: 38. Bytes expected: 116.: (at row 1) : While executing BinaryRowInputFormat. (CANNOT_READ_ALL_DATA) (version <IP_ADDRESS>88 (official build))
I cannot figure out what is going wrong...
Here are my rust code:
use std::{
    net::{Ipv4Addr, Ipv6Addr},
    u64,
};
use serde_repr::{Deserialize_repr, Serialize_repr};
use time::OffsetDateTime;

#[derive(Debug, serde::Serialize, serde::Deserialize, clickhouse::Row)]
pub struct MyLogMessage {
    #[serde(with = "clickhouse::serde::time::datetime64::millis")]
    pub timestamp: OffsetDateTime,
    #[serde(skip_serializing_if = "Option::is_none")]
    pub api_key: Option<String>,
    #[serde(skip_serializing_if = "Option::is_none")]
    #[serde(with = "clickhouse::serde::ipv4::option")]
    pub ip4: Option<Ipv4Addr>,
    #[serde(skip_serializing_if = "Option::is_none")]
    pub ip6: Option<Ipv6Addr>,
    #[serde(skip_serializing_if = "Option::is_none")]
    pub auth: Option<bool>,
    #[serde(skip_serializing_if = "Option::is_none")]
    pub user_agent: Option<String>,
    pub req_size: u32,
    #[serde(skip_serializing_if = "Option::is_none")]
    pub req_content_type: Option<String>,
    pub content_length: u64,
    #[serde(skip_serializing_if = "Option::is_none")]
    pub content_type: Option<String>,
    pub method: HttpMethod,
    pub host: String,
    pub service: String,
    #[serde(skip_serializing_if = "Option::is_none")]
    pub path: Option<String>,
    #[serde(skip_serializing_if = "Option::is_none")]
    pub query: Option<String>,
    pub status_code: u16,
    pub execution_time_ms: u32,
    pub version: String,
}

#[derive(Debug, Serialize_repr, Deserialize_repr)]
#[repr(u8)]
pub enum HttpMethod {
    _UN = 0,
    GET = 1,
    HEAD = 2,
    POST = 3,
    PUT = 4,
    DELETE = 5,
    CONNECT = 6,
    OPTIONS = 7,
    TRACE = 8,
    PATCH = 9,
}

fn main() {
    let mut inserter = clickhouse
        .inserter::<MyLogMessage>("my_log")
        .unwrap();

    let mut messages = vec![MyLogMessage { ... }, ...];
    while let Some(msg) = &messages.pop() {
        inserter.write(&msg).await.unwrap();
    }

    match inserter.end().await {
        Ok(_) => {
            consumer.commit_consumer_state(CommitMode::Sync).unwrap();
            tracing::info!("ALL MSGs SAVED.");
        }
        Err(e) => {
            tracing::error!("{e}");  // Here is the CANNOT_READ_ALL_DATA DB::Exception
        }
    };
}

And the clickhouse DB:
CREATE TABLE my_log
(
    `timestamp` DateTime64(3),
    `api_key` String,
    `ip4` IPv4,
    `ip6` IPv6,
    `version` String,
    `auth` Bool,
    `user_agent` String,
    `req_size` UInt32,
    `req_content_type` String,
    `content_length` UInt64,
    `content_type` String,
    `method` Enum8('_' = 0, 'GET' = 1, 'HEAD' = 2, 'POST' = 3, 'PUT' = 4, 'DELETE' = 5, 'CONNECT' = 6, 'OPTIONS' = 7, 'TRACE' = 8, 'PATCH' = 9),
    `host` String,
    `service` String,
    `path` String,
    `query` String,
    `status_code` UInt16,
    `execution_time_ms` UInt32
)
ENGINE = MergeTree;

Maybe I missed something from docs?
Could be any of those Option<_> that are not annotated in any way (despite I guess skip_serializing_if should be enough)?
Or, do I need to put Nullable for each column to match rust impl? as I test, CH put default values anyway, so, as long as they are skipped, there should not be any exception, right?
Or, is it better to put serde defaults without Option<_> wrappers?
EDIT: I also tried the wa-37420 feature flag. Same results

I tested some of MyLogMessage elements built from json raw data:

with this example, it raises CANNOT_READ_ALL_DATA

{
    "timestamp":1718746919678
    "ip4":2130706433
    "user_agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/<IP_ADDRESS> Safari/537.36"
    "req_size":0
    "content_length":0
    "content_type":"image/x-icon"
    "method":1
    "host":"localhost"
    "service":"favicon.ico"
    "path":"/"
    "status_code":200
    "execution_time_ms":0
    "version":"HTTP/1.1"
}

with this example, it raises ATTEMPT_TO_READ_AFTER_EOF

{
    "timestamp":1718788113783
    "api_key":"<my-key>"
    "ip4":2130706433
    "auth":true
    "user_agent":"insomnia/9.2.0"
    "req_size":0
    "req_content_type":"application/json"
    "content_length":4242
    "content_type":"application/json"
    "method":1
    "host":"localhost"
    "service":"my-service"
    "path":"/"
    "query":"f1=true&f2=true"
    "status_code":200
    "execution_time_ms":77
    "version":"HTTP/1.1"
}

Or, is it better to put serde defaults without Option<_> wrappers?

Ok. I managed it to fix only in this way! the other ones don't seem to work

maybe, it is better to close the issue only after some of your checks

Hi,
I'm getting a very similar issue, get this error message:
BadResponse("Code: 33. DB::Exception: Cannot read all data. Bytes read: 0. Bytes expected: 1.: (at row 13)\n: While executing BinaryRowInputFormat. (CANNOT_READ_ALL_DATA) (version <IP_ADDRESS>70 (official build))")
FWIW -- trying this against Clickhouse Cloud.
I have changed my struct to make sure there are no options, and all values are populated, even thought the table allows most to be null.
Ideas on where to look would be greatly appreciated.

as an update, I was using chrono for DateTime and after seeing this issue and re-reading that time support, I switched over from chrono to time and still have same issue.

Hi @apsaltis ! If you have any example snippet that can reproduce the actual data structures, please send here. Just to compare with those of mine and with all the attempts I did 😁

Hi @riccardogabellone -- here is the data structure:
#[derive(Row,Debug, Deserialize, Serialize)] struct AiLiteracyRow {     #[serde(with = "clickhouse::serde::time::datetime")]     created_on_timestamp: OffsetDateTime,     #[serde(with = "clickhouse::serde::time::datetime")]     last_update_timestamp: OffsetDateTime,     organization_id: String,     user_id: String,     raw_results: String,     ethics_assessed_score: i32,     ethics_self_score_avg: i32,     ethics_self_score_strongly_disagree_count: i32,     ethics_self_score_disagree_count: i32,     ethics_self_score_somewhat_disagree_count: i32,     ethics_self_score_neither_count: i32,     ethics_self_score_somewhat_agree_count: i32,     ethics_self_score_agree_count: i32,     ethics_self_score_strongly_agree_count: i32,     eval_create_assessed_score: i32,     eval_create_self_score_avg: i32,     eval_create_self_score_strongly_disagree_count: i32,     eval_create_self_score_disagree_count: i32,     eval_create_self_score_somewhat_disagree_count: i32,     eval_create_self_score_neither_count: i32,     eval_create_self_score_somewhat_agree_count: i32,     eval_create_self_score_agree_count: i32,     eval_create_self_score_strongly_agree_count: i32,     know_understand_assessed_score: i32,     know_understand_self_score_avg: i32,     know_understand_self_score_strongly_disagree_count: i32,     know_understand_self_score_disagree_count: i32,     know_understand_self_score_somewhat_disagree_count: i32,     know_understand_self_score_neither_count: i32,     know_understand_self_score_somewhat_agree_count: i32,     know_understand_self_score_agree_count: i32,     know_understand_self_score_strongly_agree_count: i32,     use_and_apply_assessed_score: i32,     use_and_apply_self_score_avg: i32,     use_and_apply_self_score_strongly_disagree_count: i32,     use_and_apply_self_score_disagree_count: i32,     use_and_apply_self_score_somewhat_disagree_count: i32,     use_and_apply_self_score_neither_count: i32,     use_and_apply_self_score_somewhat_agree_count: i32,     use_and_apply_self_score_agree_count: i32,     use_and_apply_self_score_strongly_agree_count: i32,     ai_001: i16,     ai_002: i16,     ai_003: i16,     ai_004: i16,     ai_005: i16,     ai_006: i16,     ai_007: i16,     ai_008: i16,     ai_009: i16,     ai_010: i16,     ai_011: i16,     ai_012: i16,     ai_013: i16,     ai_014: i16,     ai_015: i16,     ai_016: i16,     ai_017: i16,     ai_018: i16,     ai_019: i16,     ai_020: i16,     ai_021: i16,     ai_023: i16,     ai_024: i16,     ai_025: i16,     ai_026: i16,     ai_027: i16,     ai_028: u8,     ai_029: u8,     ai_030: i16,     ai_031: i16,     ai_032: i16,     ai_033: i16,     ai_034: i16,     ai_035: i16,     ai_036: i16,     ai_037: i16,     ai_038: i16,     ai_039: i16,     ai_040: i16,     ai_041: i16,     ai_042: i16,     ai_043: i16,     ai_044: i16,     ai_045: i16,     ai_046: i16,     ai_047: i16,     ai_048: i16,     ai_049: i16,     ai_050: i16,     ai_051: i16,     ai_052: i16,     ai_053: i16,     ai_054: i16,     ai_055: i16,     ai_056: i16,     ai_057: i16,     ai_058: i16,     ai_059: i16,     ai_060: i16,     ai_061: i16,     ai_062: i16,     ai_063: i16,     ai_064: i16,     ai_065: i16,     ai_066: i16,     ai_067: i16,     ai_068: i16,     ai_069: i16,     ai_070: i16,     ai_071: i16,     ai_072: i16,     ai_073: i16,     ai_074: i16,     ai_075: i16,     ai_076: i16,     ai_077: i16,     ai_078: i16,     ai_079: i16,     ai_080: i16,     ai_081: i16,     ai_082: i16,     ai_083: i16,     ai_084: i16,     ai_085: i16,     ai_086: i16,     ai_087: i16,     ai_088: i16,     ai_089: i16,     ai_090: i16,     ai_091: i16,     ai_092: i16,     ai_093: i16,     ai_094: i16,     ai_095: i16,     ai_096: i16,     ai_097: i16,     ai_098: i16,     ai_099: i16,     ai_100: i16,     ai_101: i16,     ai_102: i16,     ai_103: i16,     ai_104: i16,     ai_105: i16,     ai_106: i16,     ai_108: i16,     ai_109: i16,     ai_110: i16,     ai_111: i16,     ai_112: i16,     ai_113: i16,     ai_114: i16,     ai_115: i16,     ai_116: i16,     ai_117: i16,     ai_118: i16,     ai_119: i16,     ai_120: i16,     ai_121: i16,     ai_122: i16,     ai_123: i16,     ai_124: i16,     ai_125: i16,     ai_126: i16,     ai_127: String,     ai_128: i16,     ai_129: i16,     ai_130: i16,     ai_131: i16,     ai_132: i16,     ai_133: i16,     ai_134: i16,     ai_135: i16,     ai_136: i16,     ai_137: String,     ai_138: String,     ai_140: String,     ai_141: String,     ai_142: String,     ai_143: u8,     ai_144: String,     ai_145: String, }
and here is the row populated and serilized as JSON
{
"created_on_timestamp":<PHONE_NUMBER>,
"last_update_timestamp":<PHONE_NUMBER>,
"organization_id": "123",
"user_id": "456",
"raw_results": "this ends up being a JSON string",
"ethics_assessed_score": 0,
"ethics_self_score_avg": 2,
"ethics_self_score_strongly_disagree_count": 14,
"ethics_self_score_disagree_count": 0,
"ethics_self_score_somewhat_disagree_count": 0,
"ethics_self_score_neither_count": 0,
"ethics_self_score_somewhat_agree_count": 0,
"ethics_self_score_agree_count": 0,
"ethics_self_score_strongly_agree_count": 0,
"eval_create_assessed_score": 0,
"eval_create_self_score_avg": 2,
"eval_create_self_score_strongly_disagree_count": 14,
"eval_create_self_score_disagree_count": 0,
"eval_create_self_score_somewhat_disagree_count": 0,
"eval_create_self_score_neither_count": 0,
"eval_create_self_score_somewhat_agree_count": 0,
"eval_create_self_score_agree_count": 0,
"eval_create_self_score_strongly_agree_count": 0,
"know_understand_assessed_score": 2,
"know_understand_self_score_avg": 7,
"know_understand_self_score_strongly_disagree_count": 50,
"know_understand_self_score_disagree_count": 0,
"know_understand_self_score_somewhat_disagree_count": 0,
"know_understand_self_score_neither_count": 0,
"know_understand_self_score_somewhat_agree_count": 0,
"know_understand_self_score_agree_count": 0,
"know_understand_self_score_strongly_agree_count": 0,
"use_and_apply_assessed_score": 0,
"use_and_apply_self_score_avg": 2,
"use_and_apply_self_score_strongly_disagree_count": 17,
"use_and_apply_self_score_disagree_count": 0,
"use_and_apply_self_score_somewhat_disagree_count": 0,
"use_and_apply_self_score_neither_count": 0,
"use_and_apply_self_score_somewhat_agree_count": 0,
"use_and_apply_self_score_agree_count": 0,
"use_and_apply_self_score_strongly_agree_count": 0,
"ai_001": 1,
"ai_002": 1,
"ai_003": 1,
"ai_004": 0,
"ai_005": 4,
"ai_006": 1,
"ai_007": 1,
"ai_008": 1,
"ai_009": 1,
"ai_010": 1,
"ai_011": 1,
"ai_012": 3,
"ai_013": 3,
"ai_014": 3,
"ai_015": 2,
"ai_016": 1,
"ai_017": 1,
"ai_018": 1,
"ai_019": 1,
"ai_020": 1,
"ai_021": 1,
"ai_022": 1,
"ai_023": 1,
"ai_024": 1,
"ai_025": 1,
"ai_026": 1,
"ai_027": 1,
"ai_028": true,
"ai_029": false,
"ai_030": 1,
"ai_031": 1,
"ai_032": 1,
"ai_033": 1,
"ai_034": 1,
"ai_035": 1,
"ai_036": 1,
"ai_037": 0,
"ai_038": 0,
"ai_039": 2,
"ai_040": 1,
"ai_041": 1,
"ai_042": 1,
"ai_043": 1,
"ai_044": 1,
"ai_045": 1,
"ai_046": 1,
"ai_047": 1,
"ai_048": 1,
"ai_049": 1,
"ai_050": 1,
"ai_051": 3,
"ai_052": 4,
"ai_053": 0,
"ai_054": 1,
"ai_055": 1,
"ai_056": 1,
"ai_057": 1,
"ai_058": 1,
"ai_059": 1,
"ai_060": 1,
"ai_061": 1,
"ai_062": 1,
"ai_063": 1,
"ai_064": 1,
"ai_065": 1,
"ai_066": 1,
"ai_067": 1,
"ai_068": 1,
"ai_069": 1,
"ai_070": 1,
"ai_071": 4,
"ai_072": 4,
"ai_073": 0,
"ai_074": 0,
"ai_075": 0,
"ai_076": 0,
"ai_077": 4,
"ai_078": 4,
"ai_079": 1,
"ai_080": 1,
"ai_081": 1,
"ai_082": 1,
"ai_083": 1,
"ai_084": 1,
"ai_085": 1,
"ai_086": 1,
"ai_087": 1,
"ai_088": 1,
"ai_089": 1,
"ai_090": 1,
"ai_091": 1,
"ai_092": 1,
"ai_093": 1,
"ai_094": 1,
"ai_095": 1,
"ai_096": 1,
"ai_097": 1,
"ai_098": 1,
"ai_099": 1,
"ai_100": 1,
"ai_101": 1,
"ai_102": 1,
"ai_103": 1,
"ai_104": 1,
"ai_105": 1,
"ai_106": 1,
"ai_108": 1,
"ai_109": 1,
"ai_110": 0,
"ai_111": 0,
"ai_112": 4,
"ai_113": 1,
"ai_114": 1,
"ai_115": 1,
"ai_116": 1,
"ai_117": 1,
"ai_118": 1,
"ai_119": 1,
"ai_120": 1,
"ai_121": 1,
"ai_122": 1,
"ai_123": 1,
"ai_124": 0,
"ai_125": 0,
"ai_126": 0,
"ai_127": [4, 2, 5, 3, 1],
"ai_128": 0,
"ai_129": 0,
"ai_130": 4,
"ai_131": 3,
"ai_132": 0,
"ai_133": 4,
"ai_134": 4,
"ai_135": 1,
"ai_136": 0,
"ai_137": ["ml", "nlp", "robotics"],
"ai_138": "no_knoledge",
"ai_140": ["dev_models", "manage_projects"],
"ai_141": "adsfasdasdf",
"ai_142": "less_than_1",
"ai_143": true,
"ai_144": "asdfadfadf",
"ai_145": "asdfasdf"
}
I also just tried with crate feature "wa-37420"

just changed the "raw_results" which was a JSON string to be just.a string literal  "some_data" and now got back a different error:
BadResponse("Code: 33. DB::Exception: Cannot read all data. Bytes read: 0. Bytes expected: 1.: (at row 2)\n: While executing BinaryRowInputFormat. (CANNOT_READ_ALL_DATA) (version <IP_ADDRESS>70 (official build))")
so now row 2 and not 13, not sure where that leads.

Hi @riccardogabellone -- here is the data structure:
#[derive(Row,Debug, Deserialize, Serialize)] struct AiLiteracyRow { #[serde(with = "clickhouse::serde::time::datetime")] created_on_timestamp: OffsetDateTime, #[serde(with = "clickhouse::serde::time::datetime")] last_update_timestamp: OffsetDateTime, organization_id: String, user_id: String, raw_results: String, ethics_assessed_score: i32, ethics_self_score_avg: i32, ethics_self_score_strongly_disagree_count: i32, ethics_self_score_disagree_count: i32, ethics_self_score_somewhat_disagree_count: i32, ethics_self_score_neither_count: i32, ethics_self_score_somewhat_agree_count: i32, ethics_self_score_agree_count: i32, ethics_self_score_strongly_agree_count: i32, eval_create_assessed_score: i32, eval_create_self_score_avg: i32, eval_create_self_score_strongly_disagree_count: i32, eval_create_self_score_disagree_count: i32, eval_create_self_score_somewhat_disagree_count: i32, eval_create_self_score_neither_count: i32, eval_create_self_score_somewhat_agree_count: i32, eval_create_self_score_agree_count: i32, eval_create_self_score_strongly_agree_count: i32, know_understand_assessed_score: i32, know_understand_self_score_avg: i32, know_understand_self_score_strongly_disagree_count: i32, know_understand_self_score_disagree_count: i32, know_understand_self_score_somewhat_disagree_count: i32, know_understand_self_score_neither_count: i32, know_understand_self_score_somewhat_agree_count: i32, know_understand_self_score_agree_count: i32, know_understand_self_score_strongly_agree_count: i32, use_and_apply_assessed_score: i32, use_and_apply_self_score_avg: i32, use_and_apply_self_score_strongly_disagree_count: i32, use_and_apply_self_score_disagree_count: i32, use_and_apply_self_score_somewhat_disagree_count: i32, use_and_apply_self_score_neither_count: i32, use_and_apply_self_score_somewhat_agree_count: i32, use_and_apply_self_score_agree_count: i32, use_and_apply_self_score_strongly_agree_count: i32, ai_001: i16, ai_002: i16, ai_003: i16, ai_004: i16, ai_005: i16, ai_006: i16, ai_007: i16, ai_008: i16, ai_009: i16, ai_010: i16, ai_011: i16, ai_012: i16, ai_013: i16, ai_014: i16, ai_015: i16, ai_016: i16, ai_017: i16, ai_018: i16, ai_019: i16, ai_020: i16, ai_021: i16, ai_023: i16, ai_024: i16, ai_025: i16, ai_026: i16, ai_027: i16, ai_028: u8, ai_029: u8, ai_030: i16, ai_031: i16, ai_032: i16, ai_033: i16, ai_034: i16, ai_035: i16, ai_036: i16, ai_037: i16, ai_038: i16, ai_039: i16, ai_040: i16, ai_041: i16, ai_042: i16, ai_043: i16, ai_044: i16, ai_045: i16, ai_046: i16, ai_047: i16, ai_048: i16, ai_049: i16, ai_050: i16, ai_051: i16, ai_052: i16, ai_053: i16, ai_054: i16, ai_055: i16, ai_056: i16, ai_057: i16, ai_058: i16, ai_059: i16, ai_060: i16, ai_061: i16, ai_062: i16, ai_063: i16, ai_064: i16, ai_065: i16, ai_066: i16, ai_067: i16, ai_068: i16, ai_069: i16, ai_070: i16, ai_071: i16, ai_072: i16, ai_073: i16, ai_074: i16, ai_075: i16, ai_076: i16, ai_077: i16, ai_078: i16, ai_079: i16, ai_080: i16, ai_081: i16, ai_082: i16, ai_083: i16, ai_084: i16, ai_085: i16, ai_086: i16, ai_087: i16, ai_088: i16, ai_089: i16, ai_090: i16, ai_091: i16, ai_092: i16, ai_093: i16, ai_094: i16, ai_095: i16, ai_096: i16, ai_097: i16, ai_098: i16, ai_099: i16, ai_100: i16, ai_101: i16, ai_102: i16, ai_103: i16, ai_104: i16, ai_105: i16, ai_106: i16, ai_108: i16, ai_109: i16, ai_110: i16, ai_111: i16, ai_112: i16, ai_113: i16, ai_114: i16, ai_115: i16, ai_116: i16, ai_117: i16, ai_118: i16, ai_119: i16, ai_120: i16, ai_121: i16, ai_122: i16, ai_123: i16, ai_124: i16, ai_125: i16, ai_126: i16, ai_127: String, ai_128: i16, ai_129: i16, ai_130: i16, ai_131: i16, ai_132: i16, ai_133: i16, ai_134: i16, ai_135: i16, ai_136: i16, ai_137: String, ai_138: String, ai_140: String, ai_141: String, ai_142: String, ai_143: u8, ai_144: String, ai_145: String, } and here is the row populated and serilized as JSON { "created_on_timestamp":<PHONE_NUMBER>, "last_update_timestamp":<PHONE_NUMBER>, "organization_id": "123", "user_id": "456", "raw_results": "this ends up being a JSON string", "ethics_assessed_score": 0, "ethics_self_score_avg": 2, "ethics_self_score_strongly_disagree_count": 14, "ethics_self_score_disagree_count": 0, "ethics_self_score_somewhat_disagree_count": 0, "ethics_self_score_neither_count": 0, "ethics_self_score_somewhat_agree_count": 0, "ethics_self_score_agree_count": 0, "ethics_self_score_strongly_agree_count": 0, "eval_create_assessed_score": 0, "eval_create_self_score_avg": 2, "eval_create_self_score_strongly_disagree_count": 14, "eval_create_self_score_disagree_count": 0, "eval_create_self_score_somewhat_disagree_count": 0, "eval_create_self_score_neither_count": 0, "eval_create_self_score_somewhat_agree_count": 0, "eval_create_self_score_agree_count": 0, "eval_create_self_score_strongly_agree_count": 0, "know_understand_assessed_score": 2, "know_understand_self_score_avg": 7, "know_understand_self_score_strongly_disagree_count": 50, "know_understand_self_score_disagree_count": 0, "know_understand_self_score_somewhat_disagree_count": 0, "know_understand_self_score_neither_count": 0, "know_understand_self_score_somewhat_agree_count": 0, "know_understand_self_score_agree_count": 0, "know_understand_self_score_strongly_agree_count": 0, "use_and_apply_assessed_score": 0, "use_and_apply_self_score_avg": 2, "use_and_apply_self_score_strongly_disagree_count": 17, "use_and_apply_self_score_disagree_count": 0, "use_and_apply_self_score_somewhat_disagree_count": 0, "use_and_apply_self_score_neither_count": 0, "use_and_apply_self_score_somewhat_agree_count": 0, "use_and_apply_self_score_agree_count": 0, "use_and_apply_self_score_strongly_agree_count": 0, "ai_001": 1, "ai_002": 1, "ai_003": 1, "ai_004": 0, "ai_005": 4, "ai_006": 1, "ai_007": 1, "ai_008": 1, "ai_009": 1, "ai_010": 1, "ai_011": 1, "ai_012": 3, "ai_013": 3, "ai_014": 3, "ai_015": 2, "ai_016": 1, "ai_017": 1, "ai_018": 1, "ai_019": 1, "ai_020": 1, "ai_021": 1, "ai_022": 1, "ai_023": 1, "ai_024": 1, "ai_025": 1, "ai_026": 1, "ai_027": 1, "ai_028": true, "ai_029": false, "ai_030": 1, "ai_031": 1, "ai_032": 1, "ai_033": 1, "ai_034": 1, "ai_035": 1, "ai_036": 1, "ai_037": 0, "ai_038": 0, "ai_039": 2, "ai_040": 1, "ai_041": 1, "ai_042": 1, "ai_043": 1, "ai_044": 1, "ai_045": 1, "ai_046": 1, "ai_047": 1, "ai_048": 1, "ai_049": 1, "ai_050": 1, "ai_051": 3, "ai_052": 4, "ai_053": 0, "ai_054": 1, "ai_055": 1, "ai_056": 1, "ai_057": 1, "ai_058": 1, "ai_059": 1, "ai_060": 1, "ai_061": 1, "ai_062": 1, "ai_063": 1, "ai_064": 1, "ai_065": 1, "ai_066": 1, "ai_067": 1, "ai_068": 1, "ai_069": 1, "ai_070": 1, "ai_071": 4, "ai_072": 4, "ai_073": 0, "ai_074": 0, "ai_075": 0, "ai_076": 0, "ai_077": 4, "ai_078": 4, "ai_079": 1, "ai_080": 1, "ai_081": 1, "ai_082": 1, "ai_083": 1, "ai_084": 1, "ai_085": 1, "ai_086": 1, "ai_087": 1, "ai_088": 1, "ai_089": 1, "ai_090": 1, "ai_091": 1, "ai_092": 1, "ai_093": 1, "ai_094": 1, "ai_095": 1, "ai_096": 1, "ai_097": 1, "ai_098": 1, "ai_099": 1, "ai_100": 1, "ai_101": 1, "ai_102": 1, "ai_103": 1, "ai_104": 1, "ai_105": 1, "ai_106": 1, "ai_108": 1, "ai_109": 1, "ai_110": 0, "ai_111": 0, "ai_112": 4, "ai_113": 1, "ai_114": 1, "ai_115": 1, "ai_116": 1, "ai_117": 1, "ai_118": 1, "ai_119": 1, "ai_120": 1, "ai_121": 1, "ai_122": 1, "ai_123": 1, "ai_124": 0, "ai_125": 0, "ai_126": 0, "ai_127": [4, 2, 5, 3, 1], "ai_128": 0, "ai_129": 0, "ai_130": 4, "ai_131": 3, "ai_132": 0, "ai_133": 4, "ai_134": 4, "ai_135": 1, "ai_136": 0, "ai_137": ["ml", "nlp", "robotics"], "ai_138": "no_knoledge", "ai_140": ["dev_models", "manage_projects"], "ai_141": "adsfasdasdf", "ai_142": "less_than_1", "ai_143": true, "ai_144": "asdfadfadf", "ai_145": "asdfasdf" }
I also just tried with crate feature "wa-37420"

at first glance, I note ai_137 and ai_140 are String while in JSON I see arrays ...anyway, I didn't try with clickhouse arrays yet

how is the corresponding clickhouse table declared?

Sorry, should have included that:
CREATE TABLE default .ai_literacy (     created_on_timestampDateTime64(3),    last_update_timestampDateTime64(3),    organization_idNullable(String),    user_idNullable(String),    raw_resultsNullable(String),    ethics_assessed_scoreNullable(Int32),    ethics_self_score_avgNullable(Int32),    ethics_self_score_strongly_disagree_countNullable(Int32),    ethics_self_score_disagree_countNullable(Int32),    ethics_self_score_somewhat_disagree_countNullable(Int32),    ethics_self_score_neither_countNullable(Int32),    ethics_self_score_somewhat_agree_countNullable(Int32),    ethics_self_score_agree_countNullable(Int32),    ethics_self_score_strongly_agree_countNullable(Int32),    eval_create_assessed_scoreNullable(Int32),    eval_create_self_score_avgNullable(Int32),    eval_create_self_score_strongly_disagree_countNullable(Int32),    eval_create_self_score_disagree_countNullable(Int32),    eval_create_self_score_somewhat_disagree_countNullable(Int32),    eval_create_self_score_neither_countNullable(Int32),    eval_create_self_score_somewhat_agree_countNullable(Int32),    eval_create_self_score_agree_countNullable(Int32),    eval_create_self_score_strongly_agree_countNullable(Int32),    know_understand_assessed_scoreNullable(Int32),    know_understand_self_score_avgNullable(Int32),    know_understand_self_score_strongly_disagree_countNullable(Int32),    know_understand_self_score_disagree_countNullable(Int32),    know_understand_self_score_somewhat_disagree_countNullable(Int32),    know_understand_self_score_neither_countNullable(Int32),    know_understand_self_score_somewhat_agree_countNullable(Int32),    know_understand_self_score_agree_countNullable(Int32),    know_understand_self_score_strongly_agree_countNullable(Int32),    use_and_apply_assessed_scoreNullable(Int32),    use_and_apply_self_score_avgNullable(Int32),    use_and_apply_self_score_strongly_disagree_countNullable(Int32),    use_and_apply_self_score_disagree_countNullable(Int32),    use_and_apply_self_score_somewhat_disagree_countNullable(Int32),    use_and_apply_self_score_neither_countNullable(Int32),    use_and_apply_self_score_somewhat_agree_countNullable(Int32),    use_and_apply_self_score_agree_countNullable(Int32),    use_and_apply_self_score_strongly_agree_countNullable(Int32),    ai_001Nullable(Int16),    ai_002Nullable(Int16),    ai_003Nullable(Int16),    ai_004Nullable(Int16),    ai_005Nullable(Int16),    ai_006Nullable(Int16),    ai_007Nullable(Int16),    ai_008Nullable(Int16),    ai_009Nullable(Int16),    ai_010Nullable(Int16),    ai_011Nullable(Int16),    ai_012Nullable(Int16),    ai_013Nullable(Int16),    ai_014Nullable(Int16),    ai_015Nullable(Int16),    ai_016Nullable(Int16),    ai_017Nullable(Int16),    ai_018Nullable(Int16),    ai_019Nullable(Int16),    ai_020Nullable(Int16),    ai_021Nullable(Int16),    ai_023Nullable(Int16),    ai_024Nullable(Int16),    ai_025Nullable(Int16),    ai_026Nullable(Int16),    ai_027Nullable(Int16),    ai_028Nullable(UInt8),    ai_029Nullable(UInt8),    ai_030Nullable(Int16),    ai_031Nullable(Int16),    ai_032Nullable(Int16),    ai_033Nullable(Int16),    ai_034Nullable(Int16),    ai_035Nullable(Int16),    ai_036Nullable(Int16),    ai_037Nullable(Int16),    ai_038Nullable(Int16),    ai_039Nullable(Int16),    ai_040Nullable(Int16),    ai_041Nullable(Int16),    ai_042Nullable(Int16),    ai_043Nullable(Int16),    ai_044Nullable(Int16),    ai_045Nullable(Int16),    ai_046Nullable(Int16),    ai_047Nullable(Int16),    ai_048Nullable(Int16),    ai_049Nullable(Int16),    ai_050Nullable(Int16),    ai_051Nullable(Int16),    ai_052Nullable(Int16),    ai_053Nullable(Int16),    ai_054Nullable(Int16),    ai_055Nullable(Int16),    ai_056Nullable(Int16),    ai_057Nullable(Int16),    ai_058Nullable(Int16),    ai_059Nullable(Int16),    ai_060Nullable(Int16),    ai_061Nullable(Int16),    ai_062Nullable(Int16),    ai_063Nullable(Int16),    ai_064Nullable(Int16),    ai_065Nullable(Int16),    ai_066Nullable(Int16),    ai_067Nullable(Int16),    ai_068Nullable(Int16),    ai_069Nullable(Int16),    ai_070Nullable(Int16),    ai_071Nullable(Int16),    ai_072Nullable(Int16),    ai_073Nullable(Int16),    ai_074Nullable(Int16),    ai_075Nullable(Int16),    ai_076Nullable(Int16),    ai_077Nullable(Int16),    ai_078Nullable(Int16),    ai_079Nullable(Int16),    ai_080Nullable(Int16),    ai_081Nullable(Int16),    ai_082Nullable(Int16),    ai_083Nullable(Int16),    ai_084Nullable(Int16),    ai_085Nullable(Int16),    ai_086Nullable(Int16),    ai_087Nullable(Int16),    ai_088Nullable(Int16),    ai_089Nullable(Int16),    ai_090Nullable(Int16),    ai_091Nullable(Int16),    ai_092Nullable(Int16),    ai_093Nullable(Int16),    ai_094Nullable(Int16),    ai_095Nullable(Int16),    ai_096Nullable(Int16),    ai_097Nullable(Int16),    ai_098Nullable(Int16),    ai_099Nullable(Int16),    ai_100Nullable(Int16),    ai_101Nullable(Int16),    ai_102Nullable(Int16),    ai_103Nullable(Int16),    ai_104Nullable(Int16),    ai_105Nullable(Int16),    ai_106Nullable(Int16),    ai_108Nullable(Int16),    ai_109Nullable(Int16),    ai_110Nullable(Int16),    ai_111Nullable(Int16),    ai_112Nullable(Int16),    ai_113Nullable(Int16),    ai_114Nullable(Int16),    ai_115Nullable(Int16),    ai_116Nullable(Int16),    ai_117Nullable(Int16),    ai_118Nullable(Int16),    ai_119Nullable(Int16),    ai_120Nullable(Int16),    ai_121Nullable(Int16),    ai_122Nullable(Int16),    ai_123Nullable(Int16),    ai_124Nullable(Int16),    ai_125Nullable(Int16),    ai_126Nullable(Int16),    ai_127Nullable(String),    ai_128Nullable(Int16),    ai_129Nullable(Int16),    ai_130Nullable(Int16),    ai_131Nullable(Int16),    ai_132Nullable(Int16),    ai_133Nullable(Int16),    ai_134Nullable(Int16),    ai_135Nullable(Int16),    ai_136Nullable(Int16),    ai_137Nullable(String),    ai_138Nullable(String),    ai_140Nullable(String),    ai_141Nullable(String),    ai_142Nullable(String),    ai_143Nullable(UInt8),    ai_144Nullable(String),    ai_145 Nullable(String) ) ENGINE = SharedMergeTree('/clickhouse/tables/{uuid}/{shard}', '{replica}') ORDER BY     (created_on_timestamp, last_update_timestamp) SETTINGS index_granularity = 8192

Some checks I'd do:

are those ai_* JSON array values actually strings?
try to remove all Nullable from clickhouse (a temp table) and fill them with some default value that may suitable to emulate null value
your timestamps are DateTime64(3) so I would try with this annotation instead: #[serde(with = "clickhouse::serde::time::datetime64::millis")]

Hi @riccardogabellone

RE: the ai_* JSON array values, are actually Vec in the incoming data and then they are converted like this: ai_137: response.answers.ai_137.join(", ")

RE: Nullable -- changed the table to have everything as NOT NUL, as that is reality of the system anyways.

RE: Timestamps -- changed to #[serde(with = "clickhouse::serde::time::datetime64::millis")] that certainly has an impact.

Everything is working right now like a charm. For certian if I have the wrong serde for the datetime it blows up as expected.
Thanks for all the help @riccardogabellone

Hi @riccardogabellone ,
I've run into this again, not a show stopper, but more connivence. I have a table that has an Array(UUID) when I then try and serialize a Vec then it fails. I cannot use the #[serde(with = "clickhouse::serde::uuid")] as that does not expect a Vec but just a Uuid.
For now, I changed my table to be an Array(LC(String)) and it works just fine with a Vec
Thoughts on the Vec?
Thanks,
Andrew

@apsaltis
Hm, I think it's easy to implement clickhouse::serde::uuid::vec in the same way as serde::uuid::option.
It looks clumsy; I'm sure there is a better approach here by changing
    pub fn serialize<S>(uuid: &Uuid, serializer: S) -> Result<S::Ok, S::Error>

into something like
    pub fn serialize<U: Wrapper<Uuid>, S>(uuid: &U, serializer: S) -> Result<S::Ok, S::Error>

Sounds good @loyd -- let me take a stab at implementing this and submit a pr.

Hello, I have a similar issue here as well, here is what I sent to clickhouse and they said it was more likely an issue in the rust client.  Do you want a separate issue @loyd ?
Company or project name
ACUTE Project, plenty of data inside the database (5.6B records so far).
Describe the unexpected behaviour
I'm trying to insert a few records into a table with the Rust client (https://docs.rs/clickhouse) and it generates the following error every time, usually on the 3rd or 4th records (quite small).
How to reproduce
(version <IP_ADDRESS> (official build))
Using the HTTP interface through the Rust client.

Non-default settings, if any

        let r = r##"
        CREATE TABLE ids (
    drone_id VARCHAR,
    callsign VARCHAR,
    journey INT,
    en_id VARCHAR,
) ENGINE = Memory
##;

The code later uses this to insert records:
        #[derive(Clone, Debug, Serialize, Deserialize, Row)]
        struct Tc {
            #[serde(skip_deserializing)]
            en_id: String,
            journey: u32,
            drone_id: String,
            callsign: String,
        }

        let total = dbh.query("SELECT count() FROM today_close").fetch_one::<usize>().await?;

        let r = format!(r##"
    SELECT
      journey,
      drone_id,
      callsign,
    FROM today_close
    WHERE
      dist_drone_plane < 1852
    GROUP BY ALL
        "##);

        trace!("Fetch close encounters from today_close.");
        let all = dbh.query(&r).fetch_all::<Tc>().await?;

        if all.len() == 0 {
            return Err(eyre!("No encounters found out of {total}").into());
        }

        trace!("Insert updated records.");
        // Insert the records
        //
        let mut batch = dbh.insert("acute.ids")?;

        trace!("Add en_id.");
        all.iter()
            .enumerate()
            .for_each(|(id, elem): (usize, &Tc)| {
                let journey = elem.journey;
                let elem = Tc {
                    en_id: format!("{}-{}-{}-{}", site, day_name, journey, id),
                    journey: elem.journey,
                    drone_id: elem.drone_id.clone(),
                    callsign: elem.callsign.clone(),
                };
                debug!("{elem:?}");                   // DISPLAYS THE DEBUG LINES BELOW
                block_on(async { batch.write(&elem).await.unwrap(); });
            });
        let _ = batch.end().await?;            // CRASH IS HERE -- LINE 358

         retrace(v): process_data::cmds::distances::planes::compute::insert_ids{day_name="20231128", site="BRU"}
          TRACE process_data::cmds::distances::planes::compute Fetch close encounters from today_close.
          TRACE process_data::cmds::distances::planes::compute Insert updated records.
          TRACE process_data::cmds::distances::planes::compute Add en_id.
           DB::Exception: Cannot read all data. Bytes read: 14. Bytes expected: 54.: (at row 3)
   : While executing BinaryRowInputFormat. (CANNOT_READ_ALL_DATA) (version <IP_ADDRESS> (official build))

   Location:
       process-data/src/cmds/distances/planes/compute.rs:358:17)]
Task failed: bad response: Code: 33. DB::Exception: Cannot read all data. Bytes read: 14. Bytes expected: 54.: (at row 3)
: While executing BinaryRowInputFormat. (CANNOT_READ_ALL_DATA) (version <IP_ADDRESS> (official build))

I tried with Client::insert and the inserter (Client::Inserter).

Queries to run that lead to unexpected result

Expected behavior
A clear and concise description of what you expected to happen.
Error message and/or stacktrace
/var/log/clickhouse-server/clickhouse-server.err.log:

2024.07.22 16:09:14.002843 [ 163639 ] {06fe1319-1575-4a85-810c-ef462cad190d} <Error> DynamicQueryHandler: Code: 33. DB::Exception: Cannot read all data. Bytes read: 14. Bytes expected
: 54.: (at row 3)
: While executing BinaryRowInputFormat. (CANNOT_READ_ALL_DATA), Stack trace (when copying this message, always include the lines below):

0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c5c527b
1. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x00000000077960ec
2. DB::Exception::Exception<unsigned long&, String>(int, FormatStringHelperImpl<std::type_identity<unsigned long&>::type, std::type_identity<String>::type>, unsigned long&, String&&)
@ 0x0000000008f51b0b
3. DB::ReadBuffer::readStrict(char*, unsigned long) @ 0x000000000c63be89
4. DB::SerializationString::deserializeBinary(DB::IColumn&, DB::ReadBuffer&, DB::FormatSettings const&) const @ 0x000000000fd520d6
5. DB::BinaryFormatReader<false>::readField(DB::IColumn&, std::shared_ptr<DB::IDataType const> const&, std::shared_ptr<DB::ISerialization const> const&, bool, String const&) @ 0x00000
00011f8f676
6. DB::RowInputFormatWithNamesAndTypes::readRow(std::vector<COW<DB::IColumn>::mutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::mutable_ptr<DB::IColumn>>>&, DB::RowReadExtens
ion&) @ 0x0000000011f941c0
7. DB::IRowInputFormat::read() @ 0x0000000011f751fc
8. DB::IInputFormat::generate() @ 0x0000000011f1b156
9. DB::ISource::tryGenerate() @ 0x0000000011ef7ff5
10. DB::ISource::work() @ 0x0000000011ef7a82
11. DB::ExecutionThreadContext::executeTask() @ 0x0000000011f11907
12. DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000011f061f0
13. DB::PipelineExecutor::execute(unsigned long, bool) @ 0x0000000011f05682
14. DB::CompletedPipelineExecutor::execute() @ 0x0000000011f03f92
15. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr<DB::Context>, std::function<void (DB::QueryResultDetails const&)>, DB::QueryFlags, std::optional<DB::Form
atSettings> const&, std::function<void (DB::IOutputFormat&, String const&, std::shared_ptr<DB::Context const> const&, std::optional<DB::FormatSettings> const&)>) @ 0x0000000010e321af
16. DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional<DB::CurrentThread::QueryScope>&, StrongTypede
f<unsigned long, ProfileEvents::EventTag> const&) @ 0x0000000011e23534
17. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef<unsigned long, ProfileEvents::EventTag> const&) @ 0x0000000011e287c5
18. DB::HTTPServerConnection::run() @ 0x0000000011ea5f63
19. Poco::Net::TCPServerConnection::start() @ 0x0000000014767fa7
20. Poco::Net::TCPServerDispatcher::run() @ 0x0000000014768439
21. Poco::PooledThread::run() @ 0x000000001485e8a1
22. Poco::ThreadImpl::runnableEntry(void*) @ 0x000000001485ce7d
23. ? @ 0x00007f1241bdaac3
24. ? @ 0x00007f1241c6c850
 (version <IP_ADDRESS> (official build))

Additional context
Having async_insert set to 1 does not change anything.
I don't think I can use a single INSERT INTO SQL statement with the data as parameter, can I?

I found what is bothering the client:  the serde directive skip_deserializing.  If I remove it, it works.

@keltia, would you like to open a separate issue? The client should generate a different subset of field names for serialize and deserialize processes. Now, it skips it in both cases if any skip_serializing and skip_deserializing are provided instead of separate handling. It's a bug and can be easily fixed.


Some artifacts appear after rendering

Attach (recommended) or Link to PDF file here:
camp.pdf
Configuration:

tested on Chrome 73, Safari 12.03
tested on OS X Mojave 10.14.3
PDF.js 2.2.93 (reproduced on demo page https://mozilla.github.io/pdf.js/web/viewer.html)

Steps to reproduce the problem:

Open attached PDF on demo page
Notice white artifacts on the yellow t-shirt background

This is how OSX Preview displays it:

This is how it is displayed in PDF.js viewer:

Probably difficulties with the type 7 shadings… PDFBox, Chrome and Edge have the same problem (some more, some less). See also the Kerstin Upmeyer frog:
http://kupmeyer.com/wp-content/uploads/2012/02/K_UPMEYER_SPRING10.pdf


Running multiple webpack dev servers in the same host

Currently I am running a webpack dev server on the localhost port 3000 for one UI project
webpack-dashboard -- webpack-dev-server --config webpack.dev.js --inline --hot --watch --host <IP_ADDRESS> --port 3000

Now when I want to run another webpack dev server on the localhost port 4000 for another UI project it throws EADDRINUSE error because webpack-dev-server is bound to first UI project in port 3000
How do I configure webpack-dev-server to run both UI projects simultaneously?

I am facing the same problem right now. Have you found a solution?

I am facing the same problem right now. Have you found a solution?

Did you?

@norbiu Unfortunately, no. I just start multiple dev servers on different ports

I know it has already been 4 years since the last comment, but it might help people in the future.
What I did was run both servers on different ports and configure a proxy on the host server that I want to interact with.
So I had my local server run on port 4200 and my some other project running on port 3000. I did add this config to the server running on port 4200.
{
  "/assets/other-application/": {
    "target": "http://localhost:3000"
  }
}


caldav icloud and CORS?

Hello,
I tried to get my calendar events from browser js. For that purpose, I created an app-specific password.
I import tsdav in the browser like this:
<script src="./tsdav.js" type="module" ></script>
<script type="module">
        import { Tests } from './tests.js';
        Tests.testTSDAV();
</script>

Relevant part of test.js:
static async testTSDAV() {

        const client = new DAVClient({
            serverUrl: 'https://caldav.icloud.com',
            credentials: {
                username: '$ICLOUD ACCOUNT LOGIN',
                password: '$APP_SPECIFIC PASSWORD',
            },
            authMethod: 'Basic',
            defaultAccountType: 'caldav',
        });
        await client.login(); // CORS error here
}
}

On Chrome I get the error message:
Access to XMLHttpRequest at 'https://caldav.icloud.com/.well-known/caldav' from origin 'http://<IP_ADDRESS>:5500' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource.
tsdav.js:581          PROPFIND https://caldav.icloud.com/.well-known/caldav net::ERR_FAILED

I think I understand what is happening: The icloud server has no "Access-Control-Allow-Origin:*" in its response.
I'm just wondering that I haven't seen anyone else asking about this - am I the first who tried to access iCloud?
Or hopefully, there is something I missed - because the tsdav readme says it works in a browser...?
Note that I was able to successfully connect to iCloud with curl, so it's not the authentication that fails...
Thanks in advance!

This library is not really intended to be used in browser with google/icloud etc. since they will not allow CORS requsts for obvious security reasons.
In the case where you have your own caldav server which is hosted in the same domain of your web app, you can use it without problem. Or you can even configured your caldav server to return Access-Control-Allow-Origin:* header , although it's not recommended to do it that way.

I successfully use TSDAV with my KaiOS app, now I would like to have the Google Oauth process verified. Google needs a video of the app as the Caldav API uses. To do this, I have to carry out the app in the desktop browser, but I always get a CORS message when calling the Google calendar, is there a way to use TSDAV in the browser?

Couldn't you use Charles Proxy or some other proxy to edit the server repsonses?


EntityFramework LINQ expression cannot be translated

Issue / Steps to reproduce the problem

Use Identityserver4.EntityFramework package
Enable token cleanup

Relevant parts of the log file
The LINQ expression 'where ([x].Expiration < Convert(DateTime.UtcNow, Nullable`1))' could not be translated and will be evaluated locally.

Line:
https://github.com/IdentityServer/IdentityServer4.EntityFramework/blob/1d7a862903b82c08f28d0787b1dafe1590e2f526/src/IdentityServer4.EntityFramework/TokenCleanup.cs#L113
Fix
.Where(x => x.Expiration.HasValue && x.Expiration.Value < DateTime.UtcNow)

Which DB are you using?

I am currently using Sqlite for the Operational Store.
I only realized the Warning on our Seq Server ;)
I attached a RAW Log file from Seq
event-826a571e6c7408d5e7a6b10200000000.txt

Ok... we'll have to determine is this is provider specific or not. Do you have other DBs you use?

Currently we only have the SqliteDB.
I can test that tomorrow on a SqlServer too.

Please do and let us know. Thanks.

SqlServer works fine
aspnet/EntityFrameworkCore#10887

Ok, so that tells me it's provider specific... so question is then -- what should be done (if anything)?

It will be fixed in the next EfCore release, so nothing should be necessary to do.
Thanks for the investigation :)


Added ability to add handled exceptions as an array

This is a follow up of #2900. I messed up trying to make changes to his branch and it's not recoverable.
Credit goes to @nickdnk

Coverage remained the same at 100.0% when pulling 6537a86c5b893d488adc08a1e0914912ab1917dc on l0gicgate:AddMultipleErrorHandlers into d420a9e81b27293215da797109e1460b06e4e774 on slimphp:4.x.


Predis Client::multiExec no longer exists, but is still used in RateLimit

As you can see in the Predis changelog entry for v0.8.5 (2014-01-16):

Predis\Client::multiExec(): superseded by Predis\Client::transaction() and to be removed in the next major release.

https://github.com/nrk/predis/blob/84060b9034d756b4d79641667d7f9efe1aeb8e04/CHANGELOG.md
However, the multiExec method is still used in the RateLimit class:
https://github.com/snc/SncRedisBundle/blob/6d513735ec0465013ec51b191e21894f057eb683/Extra/RateLimit.php#L35
https://github.com/snc/SncRedisBundle/search?utf8=✓&q=multiexec
I noticed this because I'm getting this error when trying to use RateLimit:

Command 'MULTIEXEC' is not a registered Redis command.

I assume this error is related to the non-existent function.
Potentially related to #222.

Fixed in master/2.x


Verify that "Bărăbanț" is listed in the dropdown menu when searched

Description:
"Bărăbanț" should appear in the dropdown menu with correct spelling and locality information.
Preconditions:
The website is up and running.

Step 1
Type "Bărăbanț" in the search bar.
Expected results:
The dropdown is listing "Bărăbanț" as the first location in the dropdown menu.

Step 2
Observe the dropdown menu.
Expected results:
"Bărăbanț" is displayed and selectable from the dropdown menu.


release: v0.1.1-alpha.17

Build issues should be fixed by https://github.com/spotify/backstage/pull/1813, backend packages weren't pulled in correctly in the e2e test.

 The e2e test was not properly testing the backend packages in the repo, but testing published versions instead. Because of that a bunch of issues had crept into master.


Support Unsafe signer algorithms

Since https://github.com/lcobucci/jwt/releases/tag/4.2.0 some signer algorithms requires specific key length, for example, Sha256 requires > 32 symbols.
But my company has shorter keys, which we can't change because we have dozens of services.
The PR adds possibility to use deprecated signers

I am not in favor of adding this as they are currently deprecated and are removed in version 5

Closing for the reason given by @maxhelias. PR welcome for supporting the v5 btw :)


[Windwos10] Battlefield 2 Hud not displaying

When I start the game on SP mode, and die I can't select the respawn point on the map view and I can't select any type of soldier/weapon. Frame does not appear but is still selectable because I can hear the sound on button selection, even view is not shown/rendered.
If you use Windows, please check the following page: https://github.com/doitsujin/dxvk/wiki/Windows
Software information
Name of the game, settings used etc.
System information

GPU: Raadeon 6900 XT
Driver: 23.7.2
Wine version: N/A
DXVK version: 2.2
OS: Windows 10

Apitrace file(s)

Put a link here

For instructions on how to use apitrace, see: https://github.com/doitsujin/dxvk/wiki/Using-Apitrace
Log files
BF2_d3d9.log

Please make an apitrace if possible.
When making one without dxvk on Windows it's expected that the issue will not reproduce when making it but we will replaying it together with dxvk to see if it then does.

Well i have a problem with Battlefield 2 is crashing when i press "join Match" using DXVK 2.2 on Wine 8.0.1
Playing on Android with a SD 8+ Gen 1 with Adreno 730

Android is not a supported platform.
I am not aware of any issues with Battlefield 2 currently on Linux related to dxvk. If you do find some do open an issue of course

Finally managed to reproduce the original issue. See https://github.com/doitsujin/dxvk/pull/3797#issue-2080902765


Ability to open server feature logs

For example, being able to open a Directory services log and seeing the description for the events.
Currently I can open such logs but the description is empty.

There seems to be a bug with the way we resolve events from the Microsoft-Windows-ActiveDirectory_DomainService provider specifically. Other providers that log to that log, such as NTDS ISAM, appear to work.
Investigating.

This is fixed in the release that will be up later today:

Note that if you created event databases that you are using to resolve these events, those databases will need to be updated with the latest EventDbTool. You may need to close EventLogExpert, and manually clear out the %localappdata%\Packages\eventlogexpert_8wekyb3d8bbwe\LocalState\Databases in order to replace the old provider databases with new ones.

Thanks, this now works as expected


recursive predictions

Can you make recursive predictions?

Hi,
I never tried and I'm not completely sure what you mean. But, let's say that at inference time the forecasting model progressively generates new predictions (with an associated confidence interval) by using as input the previous predictions.
Then yes, it is possible to adjust such confidence intervals with the offsets estimated with conformal prediction on the calibration set.
I hope this answers your question.

Thank you very much for your answers. I would like to consult you on another issue. If I want to directly use the TCN model for ensemble conformalized quantile regression, I would like to consult on the specific steps. Should I directly call the EnCQR method, or do I need to first use the TCN model for training and prediction before calling the EnCQR method?Because I found that the EnCQR method trains on each subset.


Update for Elixir 1.5 compatibility

GenEvent is deprecated since Elixir 1.5 and therefore moving the formatter to use GenServer instead.
Fixes https://github.com/victorolinasc/junit-formatter/issues/25

Coverage decreased (-75.0%) to 17.308% when pulling 5bc75d9be7ea746dc5bf124e9cfc2cbbc70c1fe2 on navinpeiris:master into 2a1bad963a05dcace4a9ad04314994e25ad27567 on victorolinasc:master.

Coverage decreased (-75.0%) to 17.308% when pulling 5bc75d9be7ea746dc5bf124e9cfc2cbbc70c1fe2 on navinpeiris:master into 2a1bad963a05dcace4a9ad04314994e25ad27567 on victorolinasc:master.

Coverage decreased (-75.0%) to 17.308% when pulling 5bc75d9be7ea746dc5bf124e9cfc2cbbc70c1fe2 on navinpeiris:master into 2a1bad963a05dcace4a9ad04314994e25ad27567 on victorolinasc:master.

Coverage decreased (-75.0%) to 17.308% when pulling 5bc75d9be7ea746dc5bf124e9cfc2cbbc70c1fe2 on navinpeiris:master into 2a1bad963a05dcace4a9ad04314994e25ad27567 on victorolinasc:master.

Besides the failing tests this seems all right. Will take a look at this more closely later.

Any update on this?

I am still looking at the test failures. Not sure why on master I can't reproduce. It seems to me like when switiching to a GenServer it leaves the current test suite in a race condition (it fails trying to find files that don't exist because other tests are trying to set envs).
I will probably have to rewrite the way we are testing this. Along the way I will probably remove elixir's 1.1 support too since coveralls demands a 1.2.
Sorry it is taking a while =/

Well, using asdf I'm reliably triggering the error caught by Travis.
Perhaps we can use a GenServer only for version 1.5. I am still investigating. Just wanted to update here.

So, I've opted to make a release that made it incompatible with Elixir version lower than 1.4. This way we can use only the GenServer implementation from now on. If people need to support Elixir before 1.4 than use the 1.3 version of the library.
Thanks @navinpeiris for your PR. I thanked you in the release notes as well.


Doc for manual project.json defines "macros" as a list when it should be a map

invalid type: sequence, expected a map at line 16 column 17
Where line 16 pointed to "macros" : [] . Changing to "macros" : {} and no complaints.

More precisely here: https://github.com/WhatsApp/eqwalizer/blob/main/README.md?plain=1#L73


fix($httpBackend): send null when post-data is undefined

IE11 (and maybe others) converts an undefined argument to xhr.send() to string ('undefined') for certain request methods (e.g. DELETE). This causes the request to appear having a body, when it shouldn't.
Fixes #12141
Fixes #12739

This should also fix https://github.com/angular/angular.js/issues/12141

Thx @pkozlowski-opensource - added that to the commit message.

LGTM

Merged as 6f39f10.


redux-form adds more than 100k to the production bundle

I'm trying to reduce the size of my bundle and realized that redux-form takes a lot and also imports a big part (all?) of lodash.
To be sure that it was not due to my particular setup, I did a quick test by adding redux-form to a fresh react app created with create-react-app.
It appears that adding redux-form makes the production bundle go from 176k to 281k.
I pushed the test project if you want to have a look: https://github.com/testerez/alalyse-redux-form-bundle-size
Just run npm run build to see webpack-bundle-analyzer UI.
Here are the screenshots from webpack-bundle-analyzer:
Without redux-form

With redux-form

Is there any way we can reduce it?

lodash-es seems to be a huge chunk of what's below redux-form 🤔
I'm not sure if it'll make a huge difference for you, but can you try http://redux-form.com/6.7.0/docs/faq/BundleSize.md/?

Thanks, @gustavohenke. I just tested on my trivial example with the babel plugin and it works pretty well, reducing the bundle size to 207k (was 176k/281k with/without redux-form).
I'll give it a try on my project but it will be a bit more complicated as I don't use babel but typescript.

30kb is still a lot to import.  this makes me wonder the usefulness of using this library.  hmmm

And tbh my trivial example is importing only the reducer. So the minimum footprint in a real project is probably more than 30k...

In theory the babel-plugin-lodash is restricting the parts of lodash in the bundle to only what is being used. Unfortunately, what is being used is deepEqual, which is a complicated algorithm no matter where it's implemented. redux-form v5 implemented it itself and it was a common source of bugs, because it's just such a complex task.
Like Redux itself, this library is a solution that is most useful when your project is large, has complex requirements (i.e. complex validation), and needs to scale. For simple projects, it might be overkill.

Because this issue saw almost no activity for a few months, I'm closing it.


The region_distribution should returns RegionDistribution instead of Result<RegionDistribution>

What type of enhancement is this?
Tech debt reduction
What does the enhancement do?
https://github.com/GreptimeTeam/greptimedb/blob/b9302e4f0d431be5c50a1e8c53b5aaf8bfe84db4/src/common/meta/src/rpc/router.rs#L33-L46
Here is no reason to return  Result<RegionDistribution>, Let's refactor it.
Implementation challenges
No response

I'd like to take a look.

I'd like to take a look.

Have fun!


Add an in memory credential providers

Is your feature request related to a problem? Please describe.
Currently we have two keystore for the CLI and Client. these are Keystore and FileCredentialProvider.
While working with the Cli these are two sufficient solutions. The SDK can use a better and more secure Provider, which is a simply in memory credential provider. which holds the api-key and the rest of credentials data in memory for as long as the app running
Describe the solution you would like
Create a simple CredentialProvider implementing CredentialsStoreInterface.
The Client should accept CredentialsStoreInterface instance as a parameter. this way the user can decide which CredentialProvider to use.
This means we need to expose our factory method outside as well. so the user could inject the right instance
Describe alternatives you have considered
Additional context
Add any other context information about the feature request here.

I like the idea! I feel though that I do not have enough information to properly evaluate. The issue/design would greatly benefit from pseudocode / flow diagrams so that the reader can fully capture your vision and form an opinion.
How should the client be updated? What are the consequences of this change? How do you plan to account for backwards compatibility (if necessary), why is memory safer than a cred store for example? Should we also provide this in option for the CLI? These are all points that should be touched on.

A direction of extracting a common interface for all types of CredentialProvider sounds a good direction to me.
The only question is if all the methods (APIs) of these Providers look then same
If they are - there is no problem to extract this interface

We Allready have this interface.. So basically you are favor for this idea.
@sigalsax gave some very important insights. I'll open a small PR that will address those insights.


fix: optic release action permissions

Codecov Report

Merging #92 (06fb5b0) into master (cf506dc) will not change coverage.
The diff coverage is n/a.

:exclamation: Current head 06fb5b0 differs from pull request most recent head d912d87. Consider uploading reports for the commit d912d87 to get more accurate results

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
@@            Coverage Diff            @@
##            master       #92   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files            1         1
  Lines           16        16
=========================================
  Hits            16        16

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


License Missing

Hi there, the project looks good. I would like to use it for https://github.com/ikks/tldrtranslate/ .  I also noticed that the db is created, but the keys are not stored in the database.
The objective is to cross compile from Debian to Windows and Macos.

I also noticed that the db is created, but the keys are not stored in the database.

Thanks, yeah, I'm aware of that. There are also some other issues with the code in test.zig. I've been experimenting with lmdb in zig here if you want to see a better usage example: https://github.com/Syndica/sig/pull/373
Once that code is stabilized and merged, I'll add some improved helper functions and example code in this repository.

Thank you for addressing the license and all the hard work. I confirm for the case use of reading an already created db,  I can cross compile and use with no issues.


fix(str-277)

Pull request type
Jira Link: STR-277

[x] Bugfix
[ ] Feature
[ ] Code style update (formatting, renaming)
[ ] Refactoring (no functional changes, no api changes)
[ ] Other (please describe):

How to test this PR
The best scenario to test this change is described on the ticket

What is the new behavior?

Introduces a new prop to SbMenu to prevent focusing the menu item when opening it

Other information

@fernandacanto, may we open a tech-debt ticket to add a new story in the future.
This is a small fiz of a specific bug that was happening on the Storyblok Editor Page.


Add support for building Racket-on-Chez images

This PR builds on top of #33 and should be merged after (or in place of) it.

@jackfirth I've rebased and addressed your comments.

Looks great to me! Merging


Add shortDescription to appsSchema

Flake apps could benefit from a description. As an example, here’s a simple flake:
{
  inputs.nixpkgs.url = "github:nixos/nixpkgs/nixpkgs-unstable";
  outputs = { self, nixpkgs, ... }:{
    apps.aarch64-darwin.hello = {
      program = "${nixpkgs.lib.getExe nixpkgs.legacyPackages.aarch64-darwin.hello}";
      type = "app";
      description = "I say hello!";
    };
  };
}

and the corresponding nix flake show --default-flake-schemas github:shivaraj-bh/flake-schemas/apps --json output, where nix is compiled from https://github.com/DeterminateSystems/nix-src/tree/flake-schemas:
{
  "apps": {
    "doc": "The `apps` output provides commands available via `nix run`.\n",
    "output": {
      "children": {
        "aarch64-darwin": {
          "children": {
            "hello": {
              "leaf": true,
              "shortDescription": "I say hello!",
              "what": "app"
            }
          }
        }
      }
    }
  },
  ...
}

Question
Should we stick to the meta attribute of derivations for apps as well?

description for flake apps should be standardised after https://github.com/NixOS/nix/pull/11297


VueJS

Hello! How to setup this icon in VueJS?

Hi @Heineken0617
You have some ways, for example, try to add the project in your package.json by running npm i linearicons --save and follow this example shows how you can insert an icon.


Blurry @x2 images with WebGl enabled. Everything good without WebGl.

Hello,
Chrome browser. We are using @x2 images for retina displays and the method itself works fine except the image is blurry if WebGL support is enabled in browser.
Once we turn off WebGl everything looks good. As expected.
We do not resize this image. Antialiase on|off doesn't do anything. LINEAR | NEAREST doesn't do anything. cacheAsBitmap = false.
Same issue in Mozilla except there we get blurry result with and without WebGl. Always (.
Here is the link to screenshot. Left is good version without WEBGL right is with WEBGL enabled.
https://snag.gy/WDhErK.jpg
Thank you.

Hi! Do you pass correct "Resolution" parameter in renderer through options in constructor or set it later?
Dynamic change of resolution of renderer requires knowledge of how WebGLRenderer works, like, you have to change rootRenderTarget resolution too and then call resize on renderer itself.
For Canvas its easier. it just works.

Ivan,
We do this only in constructor.
new PIXI.Application(width, height, {'transparent': true,  'resolution': window.devicePixelRatio, 'roundPixels': true})

Ivan,
Also, we checked to be sure:
app.renderer.rootRenderTarget.resolution
app.renderer.resolution
=2
=window.devicePixelRatio

Then it should work, congratulations!

I mean, get me more information ;)

That's the problem. We don't see anything unusual more (. That is why I'm asking here. Maybe we don't know any hidden parameters.

ok, you have multiple bugs there
first, its possible that you dont use canvas CSS, either pass "autoResize:true" in renderer options, either do it yourself: https://github.com/pixijs/pixi.js/blob/dev/src/core/renderers/SystemRenderer.js#L218
second, "baseTexture.resolution = window.devicePixelRatio" fixes it partially , now your texture has two times less size. Dont do that, texture resolution depends only on the source you loaded.
third, if you didnt specify "@2x" in the name, there's no way pixi recognizes that it has to load different texture.
fourth, generateTexture returns new texture, and you dont use whatever texture it returned => why did you even call it?

Ivan,
Sorry the developer is out of office already:

We do not set any additional css. The game canvas size for regular displays is 960x640. On the retina it becomes twice bigger:
canvas width="1920" height="1280"

Looking in to inspector the css is set automatically:
element.style {
width: 960px;
height: 640px;

Removed. No changes.

We use @x2 in names. More to say, we now doing all the experiments with @x2 image only, even for regular displays. To be sure.

Yes, sorry, this is the part from old code. You right, in case of our bitmap image we do not need to generateTexture again.

ok, then it should work :) give us more details and we'll be happy to help!

Ivan,
Fixed. It was our fault. There was a layer on top of the image which was hidden (we need it only some times). And this layer has the Color Matrix filter. PIXI filters also require resolution setting:
PIXI.settings.FILTER_RESOLUTION  = window.devicePixelRatio
Without this setting this layer made bluring for image layer behind.

Ugh, I knew i forgot something. Sorry, my telepathy failed, this issue appeared 3 times already and i had it too :) Yes, you have to modify FILTER_RESOLUTION.

Looks solved to me!


Add Quick Append and Attach

Can be tried out at https://camunda.github.io/improved-canvas/.
Changes

added quick append that uses append entries provided by context pad
added quick attach that provides attach entries for most common bondary events (according to @NPDeehan)
styled quick append and attach to match context pad
moved context pad up a bit
reduced the context pad padding to occupy less space
removed plus icon from comments context pad entry
removed text annotation entry from context pad

Looks great! My only outstanding concern, which is minor, is that when an element is in edit mode, the edit box overlaps the boundary event append button and append pad. I think the former is OK, but the latter is a bit odd -- if I've hovered over the boundary event append button to pull up the pad, then I would expect the pad to be on top.

@lmbateman We are aware of the issue (cf. https://github.com/camunda/web-modeler/issues/8477) and are working on fixing it (cc @marstamm). It's not part of this PR, so you can ignore it for now.

related to https://github.com/bpmn-io/diagram-js-direct-editing/issues/23

I gave it another round of tests and found it to work well :clap:


Fix: Report Action revert on flag approved decision

Details

Update the flagged message when moderation decision get's changed (approved)
Fixed Issues

$ https://github.com/Expensify/App/issues/22024
PROPOSAL: https://github.com/Expensify/App/issues/22024#issuecomment-1633299125
Tests

Report any User B's comment (Intimidation).
Observe that message get's hidden.
Need to Approve the flagged message, can be done by internal team.
Message should be back (not hidden)

[x] Verify that no errors appear in the JS console

Offline tests

Same as Above
QA Steps

Same as Above

[x] Verify that no errors appear in the JS console

PR Author Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[x] I added steps for the expected offline behavior in the Offline steps section
[x] I added steps for Staging and/or Production testing in the QA steps section
[x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[x] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[x] Android / native
[x] Android / Chrome
[x] iOS / native
[x] iOS / Safari
[x] MacOS / Chrome / Safari
[x] MacOS / Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method

[x] If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:

[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is approved by marketing by adding the Waiting for Copy label for a copy review on the original GH to get the correct copy.
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] I verified that if a function's arguments changed that all usages have also been updated correctly
[x] If a new component is created I verified that:

[x] A similar component doesn't exist in the codebase
[x] All props are defined accurately and each prop has a /** comment above it */
[x] The file is named correctly
[x] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[x] The only data being stored in the state is data necessary for rendering and nothing else
[x] If we are not using the full Onyx data that we loaded, I've added the proper selector in order to ensure the component only re-renders when the data it is using changes
[x] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[x] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[x] All JSX used for rendering exists in the render method
[x] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(themeColors.componentBG))

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
[x] I have checked off every checkbox in the PR author checklist, including those that don't apply to this PR.

Screenshots/Videos

Web

Mobile Web - Chrome

https://github.com/Expensify/App/assets/59088937/2c750df9-29a3-4673-9180-8a25a5b65db9

Mobile Web - Safari
https://github.com/Expensify/App/assets/59088937/2c750df9-29a3-4673-9180-8a25a5b65db9

Desktop
https://github.com/Expensify/App/assets/59088937/e3d647c8-7b55-49b7-acdc-f7e7d33daf28

iOS
https://github.com/Expensify/App/assets/59088937/b8587d2c-cf07-4f49-8471-45f308e09b8d

Android
https://github.com/Expensify/App/assets/59088937/76f80b84-abf6-4fff-b1c4-9c923d7e51e2

cc: @dangrous

@b4s36t4 Attachments are not visible can you fix thanks!

@Santhosh-Sellavel I can't able to re-produce this can you please tell re-production steps or share some video? Thanks!

@b4s36t4 I think he means your screencast videos in the PR body - not the code! The links are a little messed up

Ohh ok understood. Will update them soon.

@dangrous @Santhosh-Sellavel updated the PR body, please check.

@Santhosh-Sellavel is there anything I need to do?!

@b4s36t4 Testing steps are confusing
Where to Approve the flagged message.?
@dangrous Can this be tested by QA?

Approved can be done by internal team I think, we tested it in kind of different way. @dangrous can help here.

Approved can be done by internal team I think, we tested it in kind of different way. @dangrous can help here.

@b4s36t4 If that's the case test we should clearly mention that to avoid confusion that. If we can't perform the steps it shouldn't mention it.

3. Need to Approve the flagged message, can be done by internal team. made change like this, works? @Santhosh-Sellavel

I've added the internalqa label, which will let one of us QA it. I'd modify it to:

Have Concierge approve the flagged message, can be requested by internal team - reach out to @dangrous with any questions.

@Santhosh-Sellavel as for testing, let me know if you want to use my backend to test - I set it up so that it will automatically send an approval update for any flags, which is how we tested this.

Looks good, then. I'll let you know, thanks!

Waiting for @dangrous to test this one, dm'ed in slack!

Reviewer Checklist

[x] I have verified the author checklist is complete (all boxes are checked off).
[x] I verified the correct issue is linked in the ### Fixed Issues section above
[x] I verified testing steps are clear and they cover the changes made in this PR

[x] I verified the steps for local testing are in the Tests section
[x] I verified the steps for Staging and/or Production testing are in the QA steps section
[x] I verified the steps cover any possible failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)

[x] I checked that screenshots or videos are included for tests on all platforms
[x] I included screenshots or videos for tests on all platforms
[x] I verified tests pass on all platforms & I tested again on:

[x] Android / native
[x] Android / Chrome
[x] iOS / native
[x] iOS / Safari
[x] MacOS / Chrome / Safari
[x] MacOS / Desktop

[x] If there are any errors in the console that are unrelated to this PR, I either fixed them (preferred) or linked to where I reported them in Slack
[x] I verified proper code patterns were followed (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick).
[x] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method
[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is approved by marketing by adding the Waiting for Copy label for a copy review on the original GH to get the correct copy.
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I verified that this PR follows the guidelines as stated in the Review Guidelines
[x] I verified other components that can be impacted by these changes have been tested, and I retested again (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar have been tested & I retested again)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] If a new component is created I verified that:

[x] A similar component doesn't exist in the codebase
[x] All props are defined accurately and each prop has a /** comment above it */
[x] The file is named correctly
[x] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[x] The only data being stored in the state is data necessary for rendering and nothing else
[x] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[x] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[x] All JSX used for rendering exists in the render method
[x] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(themeColors.componentBG)

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
[x] I have checked off every checkbox in the PR reviewer checklist, including those that don't apply to this PR.

Screenshots/Videos

Web & Desktop
https://github.com/Expensify/App/assets/85645967/70fc94a0-158f-4550-932d-21f9829226ad

Thanks for your patience with testing this! Merging.

:hand: This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

🚀 Deployed to staging by https://github.com/dangrous in version: 1.3.47-0 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Deployed to production by https://github.com/Beamanator in version: 1.3.47-6 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅


[5.7] Add middleware disable feature with specified middleware group name o…

Current method withoutMiddleware defined in Illuminate\Foundation\Testing\Concerns\MakesHttpRequests cannot disable middleware by specified middleware group name when testing.
We could list up all middleware class name and pass it to this method, such as
$middlewares = $this->app->make(Kernel::class)->getMiddlewareGroups();
$this->withoutMiddleware($middlewares['group_name']);

However, I'll be happy if this prepared method is ready on TestCase. thanks!

To be honest I'm not a big fan of encouraging people to disable middleware during tests. I think it can lead to bad surprises later on when you're actually running the application with middleware.


ordering broke with no reason

so i was ordering for fun and out nowhere ordering broke trying to use a fresh install and resetting database doesnt have any effect and it stays broken, attached below is the log
<IP_ADDRESS> - - [08/Aug/2022 21:39:12] "POST /nwapi.php HTTP/1.1" 500 -
Traceback (most recent call last):
File "C:\Users\purpe\AppData\Local\Programs\Python\Python310\lib\site-packages\flask\app.py", line 2548, in call
return self.wsgi_app(environ, start_response)
File "C:\Users\purpe\AppData\Local\Programs\Python\Python310\lib\site-packages\flask\app.py", line 2528, in wsgi_app
response = self.handle_exception(e)
File "C:\Users\purpe\AppData\Local\Programs\Python\Python310\lib\site-packages\flask\app.py", line 2525, in wsgi_app
response = self.full_dispatch_request()
File "C:\Users\purpe\AppData\Local\Programs\Python\Python310\lib\site-packages\flask\app.py", line 1822, in full_dispatch_request
rv = self.handle_user_exception(e)
File "C:\Users\purpe\AppData\Local\Programs\Python\Python310\lib\site-packages\flask\app.py", line 1820, in full_dispatch_request
rv = self.dispatch_request()
File "C:\Users\purpe\AppData\Local\Programs\Python\Python310\lib\site-packages\flask\app.py", line 1796, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
File "C:\Users\purpe\Downloads\food-server-master\food.py", line 115, in error_api
return action_listaction
File "C:\Users\purpe\Downloads\food-server-master\helpers.py", line 78, in serialization_wrapper
returned_value = func(*args, **kwargs)
File "C:\Users\purpe\Downloads\food-server-master\responses.py", line 476, in basket_add
order: list = query.basket
AttributeError: 'NoneType' object has no attribute 'basket'

Now im not super qualified for this, but was ordering working before this broke, or were you ordering for the first time? @Sonic4Tails

Now im not super qualified for this, but was ordering working before this broke, or were you ordering for the first time? @Sonic4Tails

actually hold on ima hand this off to @spotlightishere because they are much more qualified than i am lmao

Yes ordering worked beforeVan: LambdaVerzonden: dinsdag 9 augustus 2022 23:26Aan: WiiLink24/food-serverCC: Sonic4Tails; MentionOnderwerp: Re: [WiiLink24/food-server] ordering broke with no reason (Issue #16) Now im not super qualified for this, but was ordering working before this broke, or were you ordering for the first time? @Sonic4Tailsactually ima hand this off to @spotlightishere because they are much more qualified than i am lmao—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID<EMAIL_ADDRESS>
You most likely had your save data removed. When the query returns None, this means that the zip_code that your save data contained was not stored in the database. On every new save data creation, we add the zip_code to our database.
Please try and delete your save data from Wii System Management and try again.

That did the trick thank you for your tip!Van: SketchVerzonden: woensdag 10 augustus 2022 13:59Aan: WiiLink24/food-serverCC: Sonic4Tails; MentionOnderwerp: Re: [WiiLink24/food-server] ordering broke with no reason (Issue #16) You most likely had your save data removed. When the query returns None, this means that the zip_code that your save data contained was not stored in the database. On every new save data creation, we add the zip_code to our database.Please try and delete your save data from Wii System Management and try again.—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID<EMAIL_ADDRESS>


Dedupe destinations before forwarded request is retried

There exists a bug in Ringpop's forwarding mechanism that will return a "divergence error" if any request containing multiple keys is ever retried. This happens to be the case because LookupKeys does not dedupe destinations when a lookup is attempted just prior to the retry.
@uber/ringpop

lgtm


Is the Eflow Edge Device a real iothub device?

Question Summary
The Iothub allows devices to use it's File Upload API handshake to upload files to Blob storage using SAS Tokens.   Can the EFLOW device leverage this api workflow too?  And if so Could I think have a module leverage it through EFLOW?
Just wondering

The file upload API is limited to device identities and cannot be used in IoT Edge since edge only uses module identities.
Perhaps the blob storage module can help? https://docs.microsoft.com/en-us/azure/how-to-store-data-blob?view=iotedge-2018-06

THanks!
That link didn't take me anywhere.  but googling found what I think you are referring too,
.. if you had a requirement to rotate SAS tokens how does that works?   Desired properties (as the article implies?  The advantage of the File Upload API on iothub is that my device doesn't have to hold onto a long term credential.  and I get to leverage the security channel of IOT Hub.  If I took the desired property approach outlined I'd have to orchestrate something from the cloud that generates and replaces SAS tokens for thousands (or hopefully more) devices.   That seems inefficient.
And I'm not sure but it doesn't seem like I can send messages to modules from the cloud?   Sending a message from a module for a new token and hoping it gets filled and either a property updated or Direct method invocation seems complex.
According to your understanding are these my options?
Tahnks for all your help!

Fixed link in my previous comment.
Agreed, it is not ideal but currently the only option with IoT Edge. For SAS token rotation, you could leverage layered deployments - create a layer that adds the blob storage module to your devices and then any updates to the layer will get sent to all devices.


deps(serde): use rc feature in plonky2 crate

building plonky2 currently fails:
$ cd plonky2 && cargo build
--> plonky2/src/plonk/circuit_data.rs:348:39
... the trait `Serialize` is not implemented for `Arc<Vec<(u16, u16)>>`

We have to enable rc feature in serde.
Note that this fix is also present in #1114

@dlubarov Please have a look.  Without this fix, it's very hard to use plonky2 as a dependency.

Hm this looks fine but just wondering, are you using a different version of Rust or something? It's been building fine for the rest of the contributors and also on GitHub Actions

@npwardberkeley To reproduce, you have to cd into plonky2 and cargo build from there - just tested with the latest nightly. The build works fine in the workspace level, though I'm not entirely sure why.
The culprit seems to be https://github.com/mir-protocol/plonky2/commit/c0fc349c580ab573872844cddc8df75779f33332, which introduced an Arc to the lookup tables, which I guess requires rc feature in serde:
for _ in 0..length {
    luts.push(Arc::new(self.read_lut()?));
}

Ohh I see — thanks!

Perhaps it'd be worth adding a CI job for building solely, say the plonky2 and starky crates? That would prevent this kind of issue in the future.

Ah yes that's a great idea, I'll set that up!


Poor performance with astype on an array

Reporting a bug

[x] I have tried using the latest released version of Numba (most recent is
visible in the change log (https://github.com/numba/numba/blob/main/CHANGE_LOG).
[x] I have included a self contained code sample to reproduce the problem.
i.e. it's possible to run as 'python bug.py'.

The expression np.sum(np.square(x.astype(np.int64))) when run on a 1D int8 array is more than twice as slow as evaluating it in plain numpy, and 80+ times slower than the function written serially with a loop. I expected performance to be similar to the explicit loop version (maybe faster because the compact version doesn't try to tell the compiler how to structure the implementation).
Sample code:
#!/usr/bin/env python3

import time

import numba
import numpy as np

@numba.njit
def sum_squared(x):
    total = np.int64(0)
    for i in range(len(x)):
        sample = np.int64(x[i])
        total += sample * sample
    return total

@numba.njit
def sum_squared_compact(x):
    return np.sum(np.square(x.astype(np.int64)))

def time_it(name, func, x):
    func(x)  # Warmup pass e.g. to JIT compile
    passes = 100
    start = time.monotonic()
    for _ in range(passes):
        func(x)
    stop = time.monotonic()
    print(f"{name:25} {(stop - start) / passes * 1e3:6.3f} ms")

def main():
    rng = np.random.default_rng()
    x = rng.integers(-100, 100, size=1000000, dtype=np.int8)
    time_it("sum_squared", sum_squared, x)
    time_it("sum_squared_compact", sum_squared_compact, x)
    time_it("pure numpy", sum_squared_compact.py_func, x)

if __name__ == "__main__":
    main()

Output:
sum_squared                0.090 ms
sum_squared_compact        7.379 ms
pure numpy                 3.355 ms

Environment:

Python 3.10.12
numba 0.58.1
numpy 1.26.2

Thanks for the report and nice reproducer - I can reproduce similar performance (with slightly different multiples):
$ python repro.py
sum_squared                0.074 ms
sum_squared_compact        4.553 ms
pure numpy                 3.112 ms

According to this comment, astype uses array.copy which in turn uses cgutils.loop_nest to perform the iteration in LLVM IR. Would that interfere with the loop fusion? I'm a little hazy but I got the impression that the loop fusion is a Numba IR transformation.


Use functional writing style for method calls

Source: https://github.com/SAP/code-pal-for-abap/pull/442/files
Code:

Functional writing style:

Syntax error message:

False-positive? It looks like exceptions (classic) can only be used with the CALL METHOD statement.
Btw, that's strange. The eclipse auto-complete (shift+enter) created the functional writing style, but it looks like it is no longer accepted.

Yah, you are right. Problem solved. Thanks, Lars!


.github/ISSUE_TEMPLATE/bug_report: update bug report template

Mention that OKD 3.11 issues should be filed in openshift-ansible/origin repos
Add a link to gather log bundle / must-gather

/cc @LorbusChris

Thank you @vrutkovs!


Remove query strings from URLs when searching for files

We have a project that has begun using hashes in the query string to break through image caches, but the warp-proxy doesn't know how to parse them.

@renanborgez Thanks for the review! I'm not able to merge because the Peril check is not running for some reason. Also let's have a quick chat when you're available about publishing the new version of the library

@renanborgez Thanks for the review! I'm not able to merge because the Peril check is not running for some reason. Also let's have a quick chat when you're available about publishing the new version of the library

@francisrstokes if I'm not mistaking, @indebanvdhamer has access to publish versions


Plotting updates

Ok - I fiddled around a little to change the aspect ratio in plots (main change). There may be a more elegant way to do this, but it was because the mplstyle only has one figure size option. I would like a way to have full width and half width plots or otherwise a little more overall control on the figure size generated.
Hence - two more variables at the top of plots.py for the widths, and a couple figures where I then used the width to generate within the figure plot) an appropriate height of the figure.
Also a couple label changes.
-- I'll need to fix the tests.

Ok - one change I made in here, which isn't nice, is a selection of the plot width (full_width or half_width) and then scaling the figure height based on the width (since what is plotted looks best with a specified aspect ratio).
Chiara suggested a better way to do this would be to have a single figsize in the mplstyle sheet, and then a boolean within the plot to decide whether to make it full width or half width.  But, I can't figure out how to nicely get values out of the style sheet.  It looks like yaml, so could read as a yaml, but this would mean needing to import another package.
Or - maybe just allow the figure to generate with the default size, and then pull out the figure width and rescale within the plot?

Love the conditional time axis formatter! I agree with Chiara that it probably belongs in utilities
With regards to setting user-defined figure sizes, don't stress too much about it. One of the reasons we return fig, ax is so that the user can e.g. resize the plot afterwards to match whatever style they want e.g. with figure.set_size_inches
This gives lots of flexibility, without needing to add a bunch of optional arguments to each funtion

This is something we should demonstrate in the notebook I think!


Add docker to Unraid community application

First of all, thanks for the development. This is great!!!
Would be great to see native support for Unraid by adding the docker container to Unraid's application.
Currently, it can installed via dockerhub, but think it would see wider use if added as community application.

@steve1977 thanks for using magzdb. Currently I'm not much familiar with Unraid platform, but I think we can support this.


Adding propertiesMetadata with isHtmlString nulls existing webpart property value

Category

[x] Bug

Describe the bug
I have an existing webpart with html content stored in a property that i want to be added to search.
Here is a basic example of some HTML in a webpart property (in the webpart manifest json):
"properties": {
      "text": "

Testing

"
 }

I then add the code below for the propertiesMetadata:
protected get propertiesMetadata(): IWebPartPropertiesMetadata {
    return {
      'text': { isHtmlString: true }
    };
}

On refresh, the value of my property is nulled out so i lose anything i originally had.
Steps to reproduce

Scaffold SPFx webpart
Add a property to the webpart eg "text" and with a value that is some basic HTML.
Render the HTML string as the webpart output
Build and deploy to SharePoint
Update the webpart and add the propertiesMetadata to be isHtmlString for your property
Build and deploy to SharePoint
Value is now null and nothing is being output from the webpart

Expected behavior
Existing value in the webpart is not nulled out and the value is crawled by search.
Environment details (development & target environment)

Your Developer Environment: Windows 10
Target Environment: SharePoint Online
Framework: Node.js v10
Browser(s): Chrome v81
Tooling:  SPFx v1.10.0

Hi team,  Any update on this issue ?  I am also getting the same error and using same tooling and environment as @clarktozer.
my configuration:
protected get propertiesMetadata(): IWebPartPropertiesMetadata {
    return {
      "content": { isHtmlString: true },
    };
  }

value received for content property:
{
content: undefined,
description: "sample webpart",
title: "Test One title",
uniqueId: "xxx-ccvddd"

}

Also, In my case, this issue is not happening for new web-parts added, only for existing ones.

Exact same as me @nithincvpoyyil. We have had to resort to creating powershell scripts for copying and storing the current value when upgrading our webparts, then adding it back again.

@clarktozer : As a workaround, I have changed those to  { isSearchablePlainText: true } , hope you knows the cons.

This is happening for me as well. We have a number of deployed web parts where we want to add the isHtmlString property in after they've been used.
We can see that the property is still around in the persisted data when we look at the maintenance mode before republishing the content. We preferably don't want to fix and republish the content for this to work.

The right way to do this is not to change the metadata on the existing field.  Changing that metadata changes how the data is stored and retrieved, and you have effectively said "Look for my content over here ==>" and it isn't there.
What you need to do is bump your dataversion, create a new field w/ the right metadata (where new properties will be store), and override the onAfterDeserialize method on your webpart.  You can then take the 'old' property and copy the value to the new property.  In your code, you then only use the new property.


You may need an appropriate loader to handle this file type

Hi, got this error in a Svelte project setup with Webpack
ERROR in<EMAIL_ADDRESS>1:1
Module parse failed: Unexpected character '#' (1:1)
You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders
> {#if componentNotChanged}
| 	<div
| 	in:transition={{ duration, easing, delay }}
<EMAIL_ADDRESS>3:0-67 3:0-67
 @ ./src/App.svelte
 @ ./src/main.js
 @ multi ./src/main.js

I wasn't able to reproduce the same error with the newest router version 3.0.0 and the official webpack template sveltejs/template-webpack.
But I hope this isn't a problem any more, as a lot has been fixed and changed since version 1.0.0.
If your problem still persist, feel free to reopen this issue! 😃