Migration 01 - CI/CD, Docs and Project Configs

[ ] - Add flake8 line length limit

LGTM
Not fussed about passing workflows until we integrate the migrations together.


⚠ Bofrost.fr has degraded performance

In c14bb15, Bofrost.fr (https://www.bofrost.fr) experienced degraded performance:

HTTP code: 200
Response time: 1116 ms

Resolved: Bofrost.fr performance has improved in 16ca1ef after 6 minutes.


Attempting to load javascript modules in HTML cells fails.

Describe the bug
I'm trying to load Plotly as a javascript module, using a <script> tag in an HTML cell, so that I can render the charts that I am creating in another cell using XPlot.Plotly. Plotly will return the inner HTML required to render a chart, but that HTML includes a script tag which references Plotly. Attempting to load Plotly through javascript using the instructions here throws an Mismatched anonymous module define() error.
Is it possible to load javascript modules in HTML cells or through javascript?
Please complete the following:
Which version of .NET Interactive are you using? (In a notebook, run the #!about magic command. ):
Version: 1.0.360602+9bf026dabac44a6256a65168fa93dcb7e2edcca4

OS

[ ] Windows 11
[ ] Windows 10
[x] macOS
[ ] Linux (Please specify distro)
[ ] iOS
[ ] Android

Browser

[x] Chrome
[ ] Edge
[ ] Firefox
[ ] Safari

Frontend

[ ] Jupyter Notebook
[ ] Jupyter Lab
[ ] nteract
[x] Visual Studio Code
[ ] Visual Studio Code Insiders
[ ] Visual Studio
[ ] Other (please specify)

Screenshots
If applicable, add screenshots to help explain your problem.

For what it's worth, Xplot.Plotly is deprecated. But the custom formatter there should emit the necessary script content so that you don't have to do it separately. The Plotly.NET notebook integration uses the same mechanism.


feat(csharp/src/Drivers/Apache): extend capability of GetInfo for Spark driver

Extend capability of GetInfo for Spark driver

Adds dynamic calls to get the following from the DBMS

vendor name
vendor version

vendor sql (true - hard-coded default)
driver version (using file info/product version)

Adds tests for supported and unsupported info.

@CurtHagenlocher - if you have a chance, can you review and let me know if there is anything that needs addressing? Thanks!


Run tests verbosely in CircleCI

We can afford some extra noise in the CI logs. It would be nice to see the list of tests that were run (rather than just the count per suite).

Can you please suggest some resources from where i can learn a bit more about what your project is all about.
As far as I have understood, it is a CI library which shows dynamic logs of the program while running and testing.

And also if there are some resources which can help me deep dive into the technologies which you have used to create your project.

Hi, @Xander23432. :wave:

As far as I have understood, it is a CI library which shows dynamic logs of the program while running and testing.

I'm very interested in what gave you that impression, because this project doesn't have much to do with that at all. I would like to improve my README and tags to make it clearer what the library is for.
As for documentation, your best bet is the project's page on Read the Docs — that describes the library from a user perspective. It's linked in the README, but I appreciate that it's only as a badge. I'll update that soon to make the documentation more prominent.
I don't have have much in the way of documentation about the technologies yet. I should probably add a CONTRIBUTING document to describe development, testing, and building. The project is using stock Python, and Circle CI for, well, CI.

I've created #20 to track improving the documentation, and I'll try to get to that soon.


Media skip buttons are greyed out.

Hi,
I've been a long time user of Xposed modules, one of which being PBMC to map physical keys to any action. I remember being able to change songs even with the screen turned off. But some day certain funny guy working for Google disabled the ability to do so by making their app immune to keyevents 87 and 88!!
I think that's why the media next/previous buttons are disabled on the notification panel from your great module.
This is not exactly a request since I'm aware that if it were an easy thing to do it would've been already done...but I'm curious about it, really, is there a way to make the YouTube app sensitive to the aforementioned keyevents ? Would it consist of modifying the app itself (difficult/impossible unless you're a "hacker") or...would it be an extra Xposed module for example ??
Thanks for reading.

The buttons are working only when using a playlist. I also don't know why Google didn't make them available outside of queues :P
We probably need to spam their product support forum to make them look at it...


When will the new version be released?

We use cocoapods and would like to use spacingPolicy. When is the new version planned to be released? Thanks!

I am currently experiencing some issues with the Pod environment, which is preventing me from updating to the latest version of the Pod.
As a temporary solution, I recommend using SPM (Swift Package Manager).
I will work on resolving the Pod issues as soon as possible. Thank you for your understanding.

The pod version 1.7.0 has been released. Thank you for your patience.


When prefix is left blank, for preload task, task won't run

Hi there,
I have similar grunt config to your sample-app, but use different directory names:
grunt.initConfig({

    dir: {
        src: 'src',
        dist: 'dist'
    },

    openui5_preload: {
        component: {
            options: {
                resources: {
                    cwd: '<%= dir.src %>',
                    prefix: ''
                },
                dest: '<%= dir.dist %>'
            },
            components: true
        }
    },

When run, I get:
Warning: No files found for pattern(s): "/**"! Use --force to continue.
I don't need/want a prefix. I've also noticed that if I start adding in prefixes to keep the warning quiet, the app doesn't use the Component-preload.js, it still reads in each of the Controllers/Views.
Snippet of my index.html:

Directory structure:
app
-- util
-- view
---- App.controller.js
---- App.view.xml
Declaration from App controller:
jQuery.sap.declare("view.App");

// Provides controller view.App
sap.ui.define(['jquery.sap.global', 'com/ffa/dash/util/Controller'],
function(jQuery, Controller) {
"use strict";

var App = Controller.extend("view.App", /** @lends view.App.prototype */ {

});

    return App;

}, /* bExport= */ true);

Could you please provide the complete folder structure (including Component.js).
It might be a problem that you have multiple resourceroots which do not match with the Component's namespace.

Certainly

Okay, so the issue is that all your modules (views, controller, ...) have to be in the same namespace as the Component is. This is the reason why it did not use the preload.
So the easiest solution would be to remove the view, util and thirdparty resourceroot and adopt the module names to com.ffa.dash.view.*, com.ffa.dash.util.* and com.ffa.dash.thirdparty.*
Furthermore you need to configure the prefix with com/ffa/dash as this seems to be the namespace of your Component, which is not reflected by the folder structure.

Thank you for the advice @matz3. I have done as you suggested, and the preload builds nicely. The application also runs in the browser. I have one further question however: the resources loaded by the preload file are located in a folder 'undefined', in the web inspector (Chrome). Is this to be expected, or is something still not quite right? See below...

This looks like an issue. I will look into it, thanks for reporting!

Hi @matz3 we're you able to look into the undefined/ namespace prefix at all? It doesn't affect operation of Component-preload files, but is odd. I will look into this next week also.

@hagen no, I did not have a deeper look into it. Please feel free to do so.

I'm closing this issue as there wasn't an update since quite a while. Please let me know in case this is still relevant. Please also checkout the UI5 Tooling, which is the successor of the grunt tasks: https://sap.github.io/ui5-tooling/


Use tsup to generate CJS and ESM outputs

Fixes #1

tsup makes it super simple to bundle into multiple formats
This PR generates CJS, ESM and .d.ts, and adds new fields to package.json to support ESM
Passes all the publint tests 🚀

Thank you so much for this! 🙏 It's been released as 0.2.0.


Make reset mappings in between tests configurable so the wiremock server port is retained

I'm unable to reuse the dynamically assigned port in between multiple tests runs.
Given I have 2 tests classes annotated with @AutoConfigureWireMock(port = 0)
And the spring application receives the dynamically assigned port through usage of the replaced${wiremock.server.port} property. In between the first and second test the port has changed and the 2nd test always fails being unable to reach the server because it's not running on the first assigned port but a new one.
Describe the solution you'd like
WireMockTestExecutionListener is responsible for this behavior and it could be made configurable to not reset mappings after test execution, hence not restarting the server.
wirenock.reset-mappings=false property could be introduced
Alternatively or additionally a property on the @AutoConfigureWireMock annotation could also be added.

Do you have a small sample that replicates the problem? Which version of SCC is this problem related to?

I tried to replicate it in a separate project but to my surprise was unable to.
I went back to my actual project and found out that when I put a breakpoint in WireMockConfiguration where it starts the server for the second time and wait a bit,  both tests pass. The same behavior can be noticed when I simply put a sleep right at the beginning of my test.
It might also be worth noting that the tests are written using groovy and the spock framework.

Closing due to lack of requested feedback. If you would like us to look at this issue, please provide the requested information and we will re-open the issue.


🐛 BUG - Duplicate image names

Cc: @william-liebenberg @bradystroud @JackDevAU

Hi Team
Describe the Bug

As discussed, there are some duplicate image.png files on the rules site for the Strongly Typed IDs and User Journey rules. The Journey mapping rule uses the image from strongly typed ids – which is incorrect.
https://www.ssw.com.au/rules/do-you-use-strongly-typed-ids/
https://www.ssw.com.au/rules/user-journey-mapping/
I did a bit of leg work to find all the duplicate PNG and JPG filenames and paths for you guys to have a look.
Most of the occurrences look like there are even duplicated / renamed rules! So might not be a noticeable problem.
I attached an XLSX file with all the data.

Please fix duplicates!

Expected Behavior

Images shouldn't have issues with duplicated names

Tasks

[ ] Investigate
[ ] Fix

More Info
SSW.Rules.Content.Duplicates.xlsx
Thanks!

CC @BrookJeynes
Hey @Aibono1225
I made some scripts for this when I was merging the rules to .mdoc for the Keystatic PoC.
Hopefully these might be able to help out:
https://github.com/JacksDevPlayground/rules-conversion-tool

Duplicates-090224.xlsx updated to 09/02/2024, with all duplicate image names in Rules.Content

To Myself,

[ ] Fix highlighted duplicate name

As per my conversation with @JackDevAU , some image files are not saved in the same folder as their associated rule.md file. This makes it hard for users to find the actual location of the image files when they need to delete or modify them.

[ ] Move these image files to the correct folder

Done - Duplicate image issue caused by random naming is fixed. Some image-only folders were deleted, and the image files were moved to their associated rule folders.


Implementation of Teams batch APIs

#minor
Description
This PR implements the new Teams batch APIs in TeamsOperations.

[x] Send message to a list of users
[x] Send message to all users    in a tenant
[x] Send message to all users in a team
[x] Send message to a list of channels
[x] Get Operation State
[x] Get failed entries paginated
[x] Cancel Operation

Specific Changes

Implemented the new methods in TeamsOperations, TeamsOperationsExtensions, and TeamsInfo.

Testing
Currently, the request is failing with a 400 Bad Request error.

/promoted 6655


Add hooks to start / stop background services alongside karma

In both projects I've used karma on, I've had to write a wrapper script which runs both karma and the project's development server, something like:
start_my_server()
run_karma(process.argv)
process.on('end', kill_my_server)

Plus additional fiddling to only do this when we're doing a karma start not a karma run.
Could we add hooks to the config file to allow projects to have their own hooks run:

when karma serve starts
after karma serve finishes

e.g something like:
karma.set({
    beforeServe: start_my_server,
    afterServe: kill_my_server,
});

Closing due to inactivity.


New Search Page - Page shows blank UI

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Version Number: 1.4.78-2
Reproducible in staging?: y
Reproducible in production?: n (test preference is not in production)
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Expensify/Expensify Issue URL:
Issue reported by: Applause internal team
Slack conversation:
Action Performed:

Access new.staging.expensify.com
Sign into a valid account
Go to Account Settings > Troubleshoot > New search page

Expected Result:
User expects the new search page to show and display all IOUS
Actual Result:
A blank UI is shown
Workaround:
unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[ ] Android: Native
[ ] Android: mWeb Chrome
[ ] iOS: Native
[ ] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[ ] MacOS: Desktop

Screenshots/Videos

View all open jobs on GitHub

@alexpensify FYI I haven't added the External label as I wasn't 100% sure about this issue. Please take a look and add the label if you agree it's a bug and can be handled by external contributors.

Moving this to Daily since help is on the way

@luacmartins It looks like the PR was rolled back, what's the plan for this one? Thanks for the update!

Correct, we reverted this PR along with many others due to the fire. However, we already merged the re-revert PR and it should go out with the next Auth deploy

This is now fixed. We're done here


Weekly Updates: Working on Preprocessing.

The main task for this week is still to implement some basic functionalities of Preprocessing.
cppLPS generates tokens based on JIT, which means that during the parsing process, tokens are dynamically generated.
In other words, when the parser is executed, it cannot obtain the complete list of expanded tokens for the entire Translation Unit (TU).
We have implemented an algorithm of using a virtual file to abstract the tokens generated by local macro expansion into a file called token_file.
Each token will record its next position, which is the offset in a certain file.
When the next file is a normal file (i.e., a real string file char_file), the offset corresponds to the offset of the character pointer relative to the base address.
When the next file is a token_file, the offset represents the index in the token list.

bors r+


Exposing a way to delete all match cache

Context:  #250
First of all, this is one of my first times I've ever contributed code to a project like this, I'm a WIP developer very much in a beginner level, so please excuse me if this is a trivial or wrong thing to work on, or if I didn't do it right.
It's just 3 lines of code that would help the project I'm working on right now.
Usage:
const router = useRouter();

if (!isLoggedIn) {
  router.removeAllCache();
}

Or:
const router = useRouter();

const loagoutHandler = (event: React.MouseEvent<HTMLButtonElement>) => {
  event.preventDefault();
  authCtx.logout();
  router.removeAllCache();
}

Or something similar.
Thanks.

Is this something the Author is not interested in? Any thoughts or comment? Is there any plan to implement any other way to destroy the preloaded data?
Thanks,

Next version will have this as a first class api.

TanStack Router now has router.clearMatchCache()


Trunk lib: Bundle & hash images/other resources referenced in Rust WASM source code

In ParcelJS, the JS import system is overloaded to allow users to "import" images, css, sass and the like into their JS, and then Parcel will intercept those non-JS imports and build/bundle/hash them. It would be awesome to do something similar for trunk.
option 0
Create a compile-time macro which will take a path to a resource, maybe a few config options as well. Something like trunk::include!("assets/my-image.png"), or trunk::include!("my-component.css", Opt::Hash, Opt::Compress).

first argument will be a path to the asset, verified at compile time.
any values passed after the path parameter will be treated as pipeline configuration options for the asset.
generate a manifest of all assets included during the cargo build of the Rust WASM app. Will use a std::sync::Once var to ensure the manifest is cleared at the beginning of each build (probably).
after the cargo build is finished, trunk will spawn asset pipelines for each entry in the manifest and add them to the bundle.
we could also do glob processing on these, something like trunk::include!("src/**/*.css"), to spawn pipelines for, and include, all css files under the src dir. A similar pattern is already planned as part of #3. Will probably support both.

A variant of the macro will be exported, say trunk::include_ref!(...), which will return a String value which will be the public url of the asset, hashed and all, once the trunk pipeline is complete. This will allow applications to include an image directly in their Rust source code and have a correct ref to the asset after hashing and all.
considerations

the macros will need to work the same way even when cargo build is executed outside of the context of trunk.
however, when applications ship, and are being served over the network, they will need to be able to reference assets via their public URL (typically something like /<asset> or /static/<asset> &c).
as such, the macros should default to / as the base path of the asset, but users will be able to set an env var to overwrite the default. When cargo is invoked by trunk during a trunk build (which is how all of this is intended to be used), trunk will be able to set the env var based on the value of --public-url to coordinate the values.
trunk should look for a manifest generated in the cargo output dir matching the debug/release mode.

option n
Definitely open to other design options here. Please comment and let me know if you've got ideas you would like to share.

I would love to see something like this supported by Trunk.
One way it could be achieved would be by having Trunk do a pre-build compile step, which only exists to create access to a dynamically linked function trunk_config(), defined by the library or application being compiled. This function would return a struct TrunkConfig that specifies all available assets for this library/application. The only trick would be making sure the version of Trunk matches or is at least compatible with the version defined in the library/application.
I started something similar to this on an old Yew branch that was abandoned. https://github.com/yewstack/yew/pull/1419/files#diff-f7645ea6b210138a842badb6858a7f96809d3f7ae896312a10f4f74e62aeea49R4

One way to implement this would be via build.rs in 2 stages.
Let's say we have an images folder and we add one gif
images/
            never-gonna-give-you-up.gif

Stage 1
build.rs renames the image with a hash.

One way to do this I think is via build.rs in 2 stages
Say we have an images folder and we add a gif.
images/
       never-gonna-give-you-up.gif

Stage 1
Trunk copies the file to the dist folder and adds the hash.
dist/
       never-gonna-give-you-up-a23f4576.gif

Stage 2
When the image is added or changes in the images folder, build.rs runs and generates a struct that allows us to access the image without looking up the hash.
pub struct Images {
    pub get_never_gonna_give_you_up_gif() -> String {
        String::from("never-gonna-give-you-up-a23f4576.gif")
    }
}

And in my code I do something like
html! {

}

The image will bust the cache on change. Also with this pattern if I remove an image or change the name  the compiler will let me know.

I just came up with a barebones implementation. For now, it has nothing to do with Trunk, but I would love to work on this (should I wait for #207?). Here's what it does:
First, the user is supposed to configure the build.rs:
use preprocess_hash::ConfigHashPreprocessorExt;

fn main() {
    preprocess::configure()
        .hash() // adds hash to file name
        .run();
}

Then, the build script processes files in the assets folder (src/assets by default) and writes the results to the dist folder (dist by default). The following code will be generated:
pub const NEVER_GONNA_GIVE_YOU_UP_GIF: &str = "/never_gonna_give_you_up.e55dbac5.gif";

But in order to actually use it, the user has to manually include it:
mod assets {
    include!(concat!(env!("OUT_DIR"), "/assets.rs"));
}

#[function_component(App)]
fn app() -> Html {
    html! {

    }
}

This is all good until it comes to using Trunk. The user runs trunk serve, which cleans up the dist folder, deleting all processed files. To work around this, they must add the following to the Trunk.toml (not ideal):
[[hooks]]
stage = "post_build"
command = "sh"
command_arguments = ["-c", "cp -r ./dist/* $TRUNK_STAGING_DIR"]

Hi the approach use by ructe https://github.com/kaj/ructe wouldn't require the hook step.
The don't transform the files they just use the files to calculate a hash.
So for example
images/
       never-gonna-give-you-up.gif

build.rs generates some helper.
So in your code you call
#[function_component(App)]
fn app() -> Html {
    html! {
         // <-- src="never_gonna_give_you_up-1234545.gif"
    }
}

never_gonna_give_you_up_gif.name generates the name + hash.
Then on the server you have another helper that converts hashed names back to the original file name.
In Axum it looks like the following

async fn static_path(Path(path): Path<String>) -> impl IntoResponse {
    let path = path.trim_start_matches('/');

    if let Some(data) = StaticFile::get(path) {
        Response::builder()
            .status(StatusCode::OK)
            .header(
                header::CONTENT_TYPE,
                HeaderValue::from_str(data.mime.as_ref()).unwrap(),
            )
            .body(body::boxed(Body::from(data.content)))
            .unwrap()
    } else {
        Response::builder()
            .status(StatusCode::NOT_FOUND)
            .body(body::boxed(Empty::new()))
            .unwrap()
    }
}

Notice the StaticFile::get(path).

The issue is not about just hashing. An average app will also need compression, minification, compilation (e.g. sass) etc.


redis ip count is different from csv approach

Issue
I am using csv approach which need manual human efforts to get results . tried to use redis to automate but the total count of redis approach is almost double then the count i was getting from csv approach . commands are given below can you guys please help me out in this issue .
CLI Arguments
Please paste your ZMap invocation below
Redis commands ..
sudo zmap <IP_ADDRESS>/0 -B 300M -p 80 -n 1% --shards=4 --shard=0 --seed=9820 -l zmapLogs/Zmap_B300M_P80_n1%.logs --quiet -f saddr --output-module=redis-csv --output-args=tcp://localhost:6379/queue:2_80
sudo zmap <IP_ADDRESS>/0 -B 300M -p 80 -n 1% --shards=4 --shard=1 --seed=9820 -l zmapLogs/Zmap_B300M_P80_n1%.logs --quiet -f saddr --output-module=redis-csv --output-args=tcp://localhost:6379/queue:2_80
sudo zmap <IP_ADDRESS>/0 -B 300M -p 80 -n 1% --shards=4 --shard=2 --seed=9820 -l zmapLogs/Zmap_B300M_P80_n1%.logs --quiet -f saddr --output-module=redis-csv --output-args=tcp://localhost:6379/queue:2_80
sudo zmap <IP_ADDRESS>/0 -B 300M -p 80 -n 1% --shards=4 --shard=3 --seed=9820 -l zmapLogs/Zmap_B300M_P80_n1%.logs --quiet -f saddr --output-module=redis-csv --output-args=tcp://localhost:6379/queue:2_80
=====================================CSV commands==================
sudo zmap <IP_ADDRESS>/0 -B 300M -p 80 -n 1% --shards=4 --shard=0 --seed=9820 -l zmapLogs/Zmap_B300M_P80_n1%.logs --quiet  -f saddr --output-file=100_percent_p_80.csv

sudo zmap <IP_ADDRESS>/0 -B 300M -p 80 -n 1% --shards=4 --shard=1 --seed=9820 -l zmapLogs/Zmap_B300M_P80_n1%.logs --quiet  -f saddr --output-file=100_percent_p_80.csv
sudo zmap <IP_ADDRESS>/0 -B 300M -p 80 -n 1% --shards=4 --shard=2 --seed=9820 -l zmapLogs/Zmap_B300M_P80_n1%.logs --quiet  -f saddr --output-file=100_percent_p_80.csv
sudo zmap <IP_ADDRESS>/0 -B 300M -p 80 -n 1% --shards=4 --shard=3 --seed=9820 -l zmapLogs/Zmap_B300M_P80_n1%.logs --quiet  -f saddr --output-file=100_percent_p_80.csv

Example Target IP
this was full scan of internet for port 80 .
Expected behavior
from CSV approach i was getting 62086619 ips total
and from Redis approach i got 128973584 ips which is almost double of csv.
it could be command errors or any thing else please help me out to get out this issue .

I suspect that either RSTs or duplicates are coming through. The default mode in ZMap filters these out, but when you choose a different output module it inherits an empty output module. If you used --output-module=csv, then I think that they'd be the same number. If you want to filter those out, then an --output-filter can help you get there.
This is really confusing. And to be honest, I'm not 100% sure what's the right way to make this clearer. We could have --ouput-filter be the same as the default, unless it's specified otherwise, but that would require the user to set --output-filter='' if they want all packets.

@zakird  the count is unique i have parsed somewhere in persistence. the count i had shared is unique ip count .
this will be great if you share complete command for redis approach, I will give a try and get back here

We decided to drop the Redis output module from ZMap. It's been very bug-prone, may have memory-safety issues, and is likely better handled by piping into some data processing scripts. Example:
zmap -p 80 --output-module=csv | ztee --raw | python add_to_redis.py

@dadrian  thanks for your response that will be great if you can share add_to_redis.py script as well so  I can get idea how you are capturing output in python script . again really appericate your support.

I don't have anything written, but roughly it'd look something like:
import redis
import csv
import sys

r = redis.Redis()

for row in csv.reader(sys.stdin):
    redis.set(row[0], ','.join(row))

To avoid escaping issues, you might be better off with the JSON output module.
import redis
import csv
import sys

r = redis.Redis()

for line in sys.stdin:
    obj = json.loads(line)
    redis.set(obj['saddr'], line)


A minor nit...

      - name: Upload coverage to Codecov
        run: bash <(curl -s https://codecov.io/bash)
        if: ${{ success() }}

Is this backwards?  If success then run... -- I know that it runs either way but I find it a bit confusing.

I don't think it matters but for readability's sake I will change the order.

This podcast episode is about the building of the gh CLI https://changelog.com/gotime/153


Update audio-conferencing-on-network.md

Requesting an update to this article as the capability is moving out of being in Open Preview.

@JohanFreelancer9 Copy editing is needed for this PR. Thanks.

@CarolynRowe PR has been copyedited and is ready for final review, could you please check and merge? Thanks!


fix DW_END_* constants

The endian constants were incorrect.

Coverage increased (+0.6%) to 89.676% when pulling 6ff5279c37c8450f3104156d1825ddc5c53971a5 on tromey:endian-constant-fix into 97a1789d161a0d2187a8a62ad8b6c5add0fb1542 on gimli-rs:master.

Thanks!


Add ability to add manual alert rules?

This can be done at moment via a custom alert template but maybe could make it easier


Question: mount host volume to etcd 'data-dir'

Environment
--
Linux 4.4.0-42-generic x86_64
--
NAME="Ubuntu"
VERSION="16.04.1 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.1 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
UBUNTU_CODENAME=xenial
--
systemd 229
+PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ -LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD -IDN

**What did you do?**
With rkt v1.17.0, I run the following commands
sudo rm -rf /var/lib/etcd
sudo mkdir -p /var/lib/etcd
sudo chown -R root:$(whoami) /var/lib/etcd
sudo chmod -R a+rw /var/lib/etcd

sudo /rkt --trust-keys-from-https --dir=/var/lib/rkt run \
    --stage1-name coreos.com/rkt/stage1-fly:1.17.0 \
    --net=host \
    --volume data-dir,kind=host,source=/var/lib/etcd \
    coreos.com/etcd:v3.1.0-rc.0 -- \
    --name my-etcd-1 --data-dir /var/lib/etcd \
    --listen-client-urls http://localhost:2379 --advertise-client-urls http://localhost:2379 \
    --listen-peer-urls http://localhost:2380 --initial-advertise-peer-urls http://localhost:2380 \
    --initial-cluster my-etcd-1=http://localhost:2380 \
    --initial-cluster-token my-etcd-token --initial-cluster-state new \
    --auto-compaction-retention 1

ls /var/lib/etcd

**What did you expect to see?**
$ ls /var/lib/etcd
my-etcd-1.data/

**What did you see instead?**
$ ls /var/lib/etcd
# rkt run does not write to host volume

I have been building etcd ACI image with old version of acbuild (v0.2.x), and does not understand why `rkt` doesn't host the local volume when the image was built with
acbuild --debug environment add ETCD_DATA_DIR /data-dir
acbuild --debug mount add data-dir /data-dir

https://github.com/coreos/etcd/blob/master/scripts/build-aci#L52-L58
Since it was built with mount, I cannot overwrite it.
So my question is if the image was built with mount add data-dir /data-dir, how do I overwrite this config and mount host machine's /var/lib/etcd directory to use as data-dir when running with rkt run.
Hope this is the right place to ask.
Thanks.

Thanks for the detailed report, @gyuho.
Looking at the snippets above, I think the problem may be in how you are running the image.
From the non-working case:
acbuild --debug environment add ETCD_DATA_DIR /data-dir
...
--name my-etcd-1 --data-dir /var/lib/etcd

It seems that you are setting an env flag, but overriding it later via CLI. When running, this results in:
I | pkg/flags: recognized environment variable ETCD_DATA_DIR, but unused: shadowed by corresponding flag

So I tried switching to --data-dir /data-dir, and this seems to just work :smiley:.

Unrelated question: do you know (or is it documented somewhere) why this is running in a fly pod instead of a normal stage1-coreos?

@lucab I was using fly as in https://github.com/coreos/coreos-overlay/pull/2205/files.
No strong preference :)
As you suggested, this works as expected.
sudo /rkt --trust-keys-from-https --dir=/var/lib/rkt run \
    --stage1-name coreos.com/rkt/stage1-fly:1.17.0 \
    --net=host \
    --volume data-dir,kind=host,source=/var/lib/etcd \
    coreos.com/etcd:v3.1.0-rc.0 -- \
    --name my-etcd-1 \
    --listen-client-urls http://localhost:2379 --advertise-client-urls http://localhost:2379 \
    --listen-peer-urls http://localhost:2380 --initial-advertise-peer-urls http://localhost:2380 \
    --initial-cluster my-etcd-1=http://localhost:2380 \
    --initial-cluster-token my-etcd-token --initial-cluster-state new \
    --auto-compaction-retention 1

pkg/flags: recognized and used environment variable ETCD_DATA_DIR=/data-dir

ls /var/lib/etcd
member/

Thanks!


1.0: [OpenStack] Provider is hardcoded to require an image called "CoreOS"

The OpenStack provider is hardcoded to require an image with the name "CoreOS":
https://github.com/supergiant/supergiant/blob/ae5fe4b6d9cece77493c3837073bfe6cd541c103/pkg/provider/openstack/create_kube.go#L110-L116
A configurable name should be used here.

For 1.0 this is expected behaviour. We will have more customizable settings for 2.0. In order to reduce complexity for provisioning, they use must make sure that the coreOS image exists under this name. For 2.0 the user will be able to specify any OS type they wish.


Observer entities should have names

What problem does this solve or what need does it fill?
Observers are spawned as entities. When using a tool like the egui inspector, observer entities are listed under the generic "Entity" name which makes it difficult to understand what they are at a glance
What solution would you like?
Observers should spawn with a Name component which indicates that they are observers.

This is partly a bug with bevy-inspector-egui, in that it should special-case Observer entities like it does for other classes of objects. But yeah, we should use the system name ideally.

This should be easy to fix in inspector egui. See my recent commit fixing lights associations. https://github.com/jakobhellermann/bevy-inspector-egui/commit/8b684b3608aa321c01b3eb730fefc8307e9a30c6
If someone doesn’t get to it first I’ll try and do a PR over there, separately to this ticket.

Does adding a Name component to every Observer create significant overhead for the sake of a a feature which could be implemented instead by the editor GUI? In other words, can the editor just see what the system name is and use that for Observers? (Special-cased)

Does adding a Name component to every Observer create significant overhead for the sake of a a feature which could be implemented instead by the editor GUI? In other words, can the editor just see what the system name is and use that for Observers? (Special-cased)

I sincerely doubt it! Observers are not spawned often and there should not be that many of them in a game. Personally I think it's good practice for every entity to have a name, and strive to enforce this in my projects, but this may not be standard in the community.

I think we should aim to use Name more aggressively. Long-term, I'd like to move more of our name-like identifiers (like system names) out into the proper Name component.

In that case, maybe bevy_cli could recommend that with a lint?

Yeah, I would say that's a good style level lint.

Putting names on everything means allocating strings for everything. They are nice to debug, but will have a cost

FWIW it's possible to include Name components behind a feature flag with one extra line:
commands.spawn((
    #[cfg(feature = "entity_names")]
    Name::new("Foo"),
    Foo,
));

Putting names on everything means allocating strings for everything. They are nice to debug, but will have a cost

This cost should be very minimal in the grand scheme of things. In my project I have a few dozen observers, and the total memory allocation for names would probably be somewhere south of 1kb. I struggle to see how this would be significant even for a much larger project.

FWIW Rust makes it easy to put Name components behind a feature flag:

It could be feature gated but I don't think this should be this big of a big deal, and Bevy should not offer more obscure flags than it has to. I could see it being gated by #[cfg(debug_assertions)].

Yeah, I'd love to see this unified under a dev_mode feature or something.

Should this use the same Name component we already have? This component is used for animations, and fast lookup (faster than string comparison). The Name component we already have could actually not hold a name and still work the same way.
Maybe a QueryableName and a DisplayName components?

Yeah, I would prefer to avoid using the component named Name for animations if the performance/UX requirements are distinct.

As far as I'm concerned this was fixed in https://github.com/jakobhellermann/bevy-inspector-egui/pull/223. Closing.


cmd/bosun: make graph/expr more keyboard-tab friendly

also make it so shift-enter executes on the graph page on text input fields (similar to expr page)

It looks like go generate is out of date. Or maybe there is a js file you didn't commit?

ya will fix
On Fri, Sep 2, 2016 at 11:49 AM, Craig Peterson<EMAIL_ADDRESS>wrote:

It looks like go generate is out of date. Or maybe there is a js file you
didn't commit?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/bosun-monitor/bosun/pull/1893#issuecomment-244413214,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABnT0IV_3OBMT5nmGgGCeQYTYUxCpayUks5qmEWPgaJpZM4Jzydu
.

Ok then. Go ahead when the build passes


fix(chat): fix filter text box hint (Issue #2371)

Description:
Hint text is not visible in filter's text box
Font size is 14px for hint for Contains and Equals -> bug (should be 12px)
Issues:

Issue #2371

UI changes

Checklist:

[x] the pull request name complies with Conventional Commits
[x] the pull request name starts with fix(<scope>):, feat(<scope>):, feature(<scope>):, chore(<scope>):, hotfix(<scope>): or e2e(<scope>):. If contains breaking changes then the pull request name must start with fix(<scope>)!:, feat(<scope>)!:, feature(<scope>)!:, chore(<scope>)!:, hotfix(<scope>)!: or e2e(<scope>)!: where <scope> is name of affected project: chat, chat-e2e, overlay, shared, sandbox-overlay, etc.
[x] the pull request name ends with (Issue #<TICKET_ID>) (comma-separated list of issues)
[x] I confirm that do not share any confidential information like API keys or any other secrets and private URLs

/deploy-review


【问题排查】at模式grpc出现问题

What happened:
跑at模式的grpc调用，会报错

What you expected to happen:
How to reproduce it (as minimally and precisely as possible):
Anything else we need to know?:

pls assign to me :)

ssign to me


Remove sbom-json-check Task

sbom-json-check Task has been deprecated and should be removed from the pipeline.

Multiarch controller doesn't seem to create the ssh secret so the build times out in Konflux.


[BUG] Incomplete search results against rollup index composed of data from multiple rollup jobs

Describe the bug
Incomplete results when querying rollup index with date histogram.
To Reproduce
Enter the following into the dev console and execute each query in sequence (waiting for completion of rollup jobs at that step)
# Insert some test data

POST sharej-data-2023-07-25/_doc
{
  "timestamp": "2023-07-25T14:10:43.1",
  "numberOfCalls": 1
}

POST sharej-data-2023-07-25/_doc
{
  "timestamp": "2023-07-25T14:12:43.1",
  "numberOfCalls": 1
}

POST sharej-data-2023-07-24/_doc
{
  "timestamp": "2023-07-24T13:14:43.1",
  "numberOfCalls": 4
}

POST sharej-data-2023-07-23/_doc
{
  "timestamp": "2023-07-23T15:11:43.1",
  "numberOfCalls": 2
}

POST sharej-data-2023-07-23/_doc
{
  "timestamp": "2023-07-23T17:18:43.1",
  "numberOfCalls": 4
}

# Create rollup job for above indexes, emulating what happens when you have an ISM policy applying the rollup to each index after X days
PUT _plugins/_rollup/jobs/sharej-rollup-2023-07-23
{
  "rollup": {
    "enabled": true,
    "source_index": "sharej-data-2023-07-23",
    "target_index": "rollup-sharej-data-2023",
    "schedule": {
      "interval": {
        "start_time": 1,
        "period": "1",
        "unit": "Minutes"
      }
    },
    "description": "Test rollup",
    "page_size": 1000,
    "delay": 0,
    "continuous": false,
    "dimensions": [
      {
        "date_histogram": {
          "source_field": "timestamp",
          "fixed_interval": "1h",
          "timezone": "UTC"
        }
      }
    ],
    "metrics": [
      {
        "source_field": "numberOfCalls",
        "metrics": [
          {
            "avg": {}
          },
          {
            "sum": {}
          },
          {
            "max": {}
          },
          {
            "min": {}
          },
          {
            "value_count": {}
          }
        ]
      }
    ]
  }
}

PUT _plugins/_rollup/jobs/sharej-rollup-2023-07-24
{
  "rollup": {
    "enabled": true,
    "source_index": "sharej-data-2023-07-24",
    "target_index": "rollup-sharej-data-2023",
    "schedule": {
      "interval": {
        "start_time": 1,
        "period": "1",
        "unit": "Minutes"
      }
    },
    "description": "Test rollup",
    "page_size": 1000,
    "delay": 0,
    "continuous": false,
    "dimensions": [
      {
        "date_histogram": {
          "source_field": "timestamp",
          "fixed_interval": "1h",
          "timezone": "UTC"
        }
      }
    ],
    "metrics": [
      {
        "source_field": "numberOfCalls",
        "metrics": [
          {
            "avg": {}
          },
          {
            "sum": {}
          },
          {
            "max": {}
          },
          {
            "min": {}
          },
          {
            "value_count": {}
          }
        ]
      }
    ]
  }
}

PUT _plugins/_rollup/jobs/sharej-rollup-2023-07-25
{
  "rollup": {
    "enabled": true,
    "source_index": "sharej-data-2023-07-25",
    "target_index": "rollup-sharej-data-2023",
    "schedule": {
      "interval": {
        "start_time": 1,
        "period": "1",
        "unit": "Minutes"
      }
    },
    "description": "Test rollup",
    "page_size": 1000,
    "delay": 0,
    "continuous": false,
    "dimensions": [
      {
        "date_histogram": {
          "source_field": "timestamp",
          "fixed_interval": "1h",
          "timezone": "UTC"
        }
      }
    ],
    "metrics": [
      {
        "source_field": "numberOfCalls",
        "metrics": [
          {
            "avg": {}
          },
          {
            "sum": {}
          },
          {
            "max": {}
          },
          {
            "min": {}
          },
          {
            "value_count": {}
          }
        ]
      }
    ]
  }
}

# Watch status of rollup jobs until complete

GET _plugins/_rollup/jobs/sharej-rollup-2023-07-23/_explain

GET _plugins/_rollup/jobs/sharej-rollup-2023-07-24/_explain

GET _plugins/_rollup/jobs/sharej-rollup-2023-07-25/_explain

# Execute query against source data. Three buckets returned
GET sharej-data-2023-*/_search
{
  "size": 0,
  "query": {
    "match_all": {}
  },
  "aggregations": {
    "by_day": {
      "date_histogram": {
        "field": "timestamp",
        "fixed_interval": "1d"
      },
      "aggregations": {
        "totalCalls": {
          "sum": {
            "field": "numberOfCalls"
          }
        }
      }
    }
  }
}

# Execute query against rollup data. Only 1 bucket returned!?!?!? Where's the rest of the data?
GET rollup-sharej-data-2023/_search
{
  "size": 0,
  "query": {
    "match_all": {}
  },
  "aggregations": {
    "by_day": {
      "date_histogram": {
        "field": "timestamp",
        "fixed_interval": "1d"
      },
      "aggregations": {
        "totalCalls": {
          "sum": {
            "field": "numberOfCalls"
          }
        }
      }
    }
  }
}

# Execute against rollup data without query. Expected result again (but this isn't the query we get when adding a visualisation)
GET rollup-sharej-data-2023/_search
{
  "size": 0,
  "aggregations": {
    "by_day": {
      "date_histogram": {
        "field": "timestamp",
        "fixed_interval": "1d"
      },
      "aggregations": {
        "totalCalls": {
          "sum": {
            "field": "numberOfCalls"
          }
        }
      }
    }
  }
}

Expected behavior
All three queries at the end should return the same results. What appears to be happening is that the results from only one of the rollup jobs are returned when the query parameter is provided to the search against the rollup index.
Host/Environment (please complete the following information):

OS: Linux (discovered in Aiven hosted version but behaviour reproduced locally with docker image)
Version 2.8.0

Additional context
We've got some metrics that we have posted to daily indexes. We have an ISM policy applied to the daily index pattern that after three days, performs a rollup to an annual index and deletes the source index. When trying to create visualisations based upon the rollup index we're getting strange results. When hand crafting a search against the rollup index I'm able to see that all the expected data is there, but when placing the equivalent query via a visualisation on a dashboard we're missing data. The difference between my hand-crafted search and the search from the dashboard is the presence of the query field that narrows down the time-frame and optionally drills down on other facets (not included in code example above). How do we get our visualisations to show all the data, or have I stubled upon a genuine bug here?

I ran into the same problem. It looks like it only returns the result from the last rollup index (Not just date_histogram but others as well)
Same as when I create a visualization, data only show for the last rollup index.

Is there any update on this? I'm having the same problem!

Hi @sharebear and @msfroh,
I've identified the cause of the reported behavior and believe this issue can now be closed.
The behavior is related to the code snippet found here:
https://github.com/opensearch-project/index-management/blob/d4ee795e22f4490b78662f171f62d566a81c1abc/src/main/kotlin/org/opensearch/indexmanagement/rollup/interceptor/RollupInterceptor.kt#L347
This code references the setting "plugins.rollup.search.search_all_jobs", documented here:
https://opensearch.org/docs/2.4/im-plugin/index-rollups/settings/
To modify the current behavior, you can update your cluster configuration using the following API call:
PUT https://localhost:9200/_cluster/settings
Content-Type: application/json

{
  "persistent": {
    "plugins.rollup.search.search_all_jobs": true
  },
  "transient": {
    "plugins.rollup.search.search_all_jobs": true
  }
}

This update will enable searching across all rollup jobs, both persistently and transiently.
Please don't hesitate to let me know if you have any questions or need any more help.

Thanks, I've confirmed that the setting does seem to resolve my issue in local testing, just need to work out how to get that set in my Aiven hosted instance (not your problem)


Volume API

Description
Hey guys, do you have any plans to add support for volume API to create and manage docker volumes? i.e. https://docs.docker.com/engine/reference/api/docker_remote_api_v1.23/#2-4-volumes

I have a PR for Volumes almost ready to go. I have them implemented compatible with the API when they were introduced, 1.21, but I want to add the 1.22 and 1.23 changes before I submit.

@johnflavin cool thanks!

Any progress?

Sorry, I forgot all about this! I'll rebase my branch back up to master and open a PR.
I'm not entirely sure what state it is in—i.e. whether it's fully compatible with all the latest API changes—but it's probably better to have something than nothing.


Error

firebase_analytics 9.1.8
[!] CocoaPods could not find compatible versions for pod "firebase_analytics":
  In Podfile:
    firebase_analytics (from `.symlinks/plugins/firebase_analytics/ios`)

Specs satisfying the `firebase_analytics (from `.symlinks/plugins/firebase_analytics/ios`)` dependency were found, but they required a higher minimum deployment target.

aleks@iMac-Aleksey ios % flutter doctor -v
[✓] Flutter (Channel stable, 3.0.1, on macOS 12.4 21F79 darwin-x64, locale ru-RU)
• Flutter version 3.0.1 at /Users/aleks/Documents/flutter
• Upstream repository https://github.com/flutter/flutter.git
• Framework revision fb57da5f94 (3 дня назад), 2022-05-19 15:50:29 -0700
• Engine revision caaafc5604
• Dart version 2.17.1
• DevTools version 2.12.2
[✓] Android toolchain - develop for Android devices (Android SDK version 32.1.0-rc1)
• Android SDK at /Users/aleks/Library/Android/sdk
• Platform android-32, build-tools 32.1.0-rc1
• Java binary at: /Applications/Android Studio.app/Contents/jre/Contents/Home/bin/java
• Java version OpenJDK Runtime Environment (build 11.0.12+0-b1504.28-7817840)
• All Android licenses accepted.
[✓] Xcode - develop for iOS and macOS (Xcode 13.4)
• Xcode at /Applications/Xcode.app/Contents/Developer
• CocoaPods version 1.11.3
[✓] Chrome - develop for the web
• Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome
[✓] Android Studio (version 2021.2)
• Android Studio at /Applications/Android Studio.app/Contents
• Flutter plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/9212-flutter
• Dart plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/6351-dart
• Java version OpenJDK Runtime Environment (build 11.0.12+0-b1504.28-7817840)
[✓] VS Code (version 1.67.2)
• VS Code at /Applications/Visual Studio Code.app/Contents
• Flutter extension version 3.40.0
[✓] Connected device (4 available)
• SM T970 (mobile)                    • R52N71EFXPN                          • android-arm64  • Android 12 (API 31)
• iPhone SE (3rd generation) (mobile) • 7D4BC744-811F-4C5E-9DAC-3A29B7EA69AD • ios            • com.apple.CoreSimulator.SimRuntime.iOS-15-5 (simulator)
• macOS (desktop)                     • macos                                • darwin-x64     • macOS 12.4 21F79 darwin-x64
• Chrome (web)                        • chrome                               • web-javascript • Google Chrome 101.0.4951.64
! Device J6AXB7654974VZ8 is not authorized.
You might need to check your device for an authorization dialog.
[✓] HTTP Host Availability
• All required HTTP hosts are available

@alex9153
As error message indicates Specs satisfying the firebase_analytics (from .symlinks/plugins/firebase_analytics/ios) dependency were found, but they required a higher minimum deployment target., you'll need to update your minimum deployment target. It must be in your ios/podfile` and if you have it commented at the top of the file, uncomment it and try again.
Also, this isn't related to flutter repo, but flutterfire, so if issue persists, open the issue in dedicated repo: https://github.com/firebase/flutterfire/issues


yandex[major]: Add @langchain/yandex package

This pull request adds yandex library with new Foundation Models API. Marks community modules as deprecated.

Looks good, a few nits but nothing blocking. Will merge tonight!


Multiple rightToggle views?

I have a project where I have a sort button and filter button on the right side of a tableView. When pressed, each button is suppose to slide a menu left to reveal options for sorting or filtering media.
I can get it to slide open when each button is pressed, but it only reveals one view and not a separate view for each.
Does anyone know how to do this?

Can you explain more, please? Images would be helpful...

(void)customSetup
{
SWRevealViewController *revealViewController = self.revealViewController;
[revealViewController setDelegate:self];

if (revealViewController)
{
[self.sort addTarget:revealViewController
action:@selector(rightRevealToggle:)
forControlEvents:UIControlEventTouchUpInside];
The issue I'm running into, is that I'm trying to add a second slide menu, with different content and from a separate button,  coming from the right. I tried adding separate views in the storyboard with different segues, and it still wants to pull the first one only.
Hope that is enough information. Thanks for any help

When the button is taped you have to set the rightViewController with the good one :
[self.revealViewController setRightViewController:theController animated:YES];

Amazing!! Lol! Thanks a lot for your help.


Feature/bump dependencies

Updated dependencies to latest ones for:

terraform-provider-heroku
pulumi-terraform-bridge
pulumi

Had some problems in the pipeline, now everything is green and ready to merge.


GPL-616 [Research] Warehouse 2 is not running, is it needed anymore?

Description
warehouse 2 has not been updated since the server was restarted, as the consumer didn't restart with it. Can we check if this is not needed anymore? If it is not needed we could decomission this machine, which is one of our current dependencies from old tin machines.
Who the primary contacts are for this work
Eduardo Martin
Knowledge or Stake holders
PSD helpdesk (Neil)
NPG (other warehouses access)
Additional context or information
Any other useful context or information that may be useful.

WH2 is currently our only source of historic information, making it very useful for debugging purposes. However we could consider just piping the messages straight into a mongo database or something like elastic-search. (Although that still leaves the question of existing  historic data.)

No. It's gone.


test_buffer_numpy sometimes hangs

I don't have more information for now but I noticed that sometimes the following tests just hangs
zmq/tests/test_message.py::TestFrame::test_buffer_numpy

This happens say 1 in 20 times that I run the testsuite. By pressing CTRL-C, I found out that it seems stuck here:
                while numpy.isnan(A).any():
                    # don't let nan sneak in
                    A = numpy.ndarray(shape, dtype=dt)

I can confirm this for python 3.6, whereas the test passes for lower python versions. Found this in connection with https://github.com/gentoo/gentoo/pull/9284.

With Python 3.6, it hangs for me every time.


update comment for instance generation check

The comment here does not match the current state of the world where updates are supported and async operations are dealt with before creation/updates.

thanks
LGTM
Just waiting for CI


add gpu support for glances

Describe the bug
installing any of these pkgs should probably fix the problem
python311Packages.py3nvml
python310Packages.py3nvml
python3Packages.py3nvml
but well it doesn't
btw how to know if gpu support exists supposedly press 6
Steps To Reproduce
Steps to reproduce the behavior:

press 6

Expected behavior
gpu support should work
Notify maintainers
@jonringer
@primeos
@k0ral
question
if installing those pkgs should help tell me the right way to do it
Add a :+1: reaction to issues you find important.

like I am using this overlay
{ pkgs, ... }:
let
myglances = pkgs.glances.overrideAttrs (old: {
      propagatedBuildInputs = (old.propagatedBuildInputs or []) ++ (with pkgs.python3Packages; [pygal py3nvml]);
    });
in
{
  environment = {
    systemPackages = with pkgs; [
      myglances
    ];
  };
}

and that actually makes glances --export graph work
so my problem here is making the gpu plugin work as well

This issue has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/how-to-add-optional-dependencies-the-generic-way/40597/1

It looks like py3nvml hasn't been maintained for years and hasn't been patched for nixos like pynvml is.
I'm not sure why glances uses that library, but it should be fairly easy to port the patch to py3nvml:
https://github.com/NixOS/nixpkgs/blob/068d4db604958d05d0b46c47f79b507d84dbc069/pkgs/development/python-modules/pynvml/default.nix#L20-L25


Add Validator_Monitoring_Service proposal

Project Abstract
Validator Monitoring Service is a monitoring platform designed to track the performance of validators in Polkadot and Kusama networks. Validator Monitoring Service has several key features that make it an essential tool for validator operators and nominators in Polkadot and Kusama networks.
We provide a live dashboard that users can access through an easy-to-use telegram bot (@p2pvalidator_monitoring_bot). This live dashboard allows users to track essential metrics such as session/era progress, rewards points, consensus participation, position in the active set, and more. We collect data on all active validators and store it for up to one month, providing users with historical data on validator performance. This allows users to analyse their validator's behaviour over time and identify any potential issues.
Grant level

[ ] Level 1:  Up to $10,000, 2 approvals
[x] Level 2:  Up to $30,000, 3 approvals
[ ] Level 3:  Unlimited, 5 approvals (for >$100k: Web3 Foundation Council approval)

Application Checklist

[x] The application template has been copied and aptly renamed (project_name.md).
[x] I have read the application guidelines.
[x] Payment details have been provided (bank details via email or BTC, Ethereum (USDC/DAI) or Polkadot/Kusama (USDT) address in the application).
[x] The software delivered for this grant will be released under an open-source license specified in the application.
[x] The initial PR contains only one commit (squash and force-push if needed).
[x] The grant will only be announced once the first milestone has been accepted (see the announcement guidelines).
[ ] I prefer the discussion of this application to take place in a private Element/Matrix channel. My username is: @_______:matrix.org (change the homeserver if you use a different one)

Thanks for the application @avtishin, I appreciate you comparing your implementation to the existing solutions, you certainly did a lot of research. A couple of initial questions:

I'm curious why there is a need to authenticate validators to make sure they own the address. Why not create a read-only functionality similar to the Parity staking dashboard that allows me to monitor any validator address?
The second milestone seems a bit expensive for optimizing the telegram bot. It also seems very specific to your use case. Would others be able to re-use the code to create telegram bots of their own, as an alternative to using yours?
If the data is only stored for one month, what happens after it is deleted? Does this mean users can only view the historical performance data for the last month?

Hey @keeganquigley , thank you for your questions. So,

It seems there might be a slight misinterpretation. We’re not authenticating the owners of the validators. The Telegram bot’s authentication is just for establishing your personal dashboard credentials. You’re free to submit any validator addresses you wish to monitor - we simply verify that these are indeed addresses of validators, not who owns them. In fact, in our 2nd Milestone, we will eliminate the need to provide validator addresses to the Telegram bot entirely. Instead, we are transitioning this feature to the Grafana dashboard. That means you can choose your validators directly from the Grafana dashboard. This step is aimed at making the user onboarding process simpler and more intuitive. In essence, the primary role of the Telegram bot will be limited to generating dashboard credentials.

Regarding the Telegram bot and code, our goal is to offer a user service rather than just a codebase. We believe it’s significant that users don’t need to host anything; they simply log into the bot and access the dashboard. We’ve also made available a repository containing all the files necessary to run your own monitoring system. In this case, you won’t need the Telegram bot, as you can specify the validators you wish to follow directly (in fact, all validators will be accessible on the dashboard when using our Docker setup, they just need to be filtered). Hence, we haven’t made the bot’s code public because there is no need for it. Additionally, from point 1, we plan to further simplify the bot’s use by transferring validator selection from the bot to the dashboard, so the bot’s role will be limited to generating dashboard credentials.

Concerning data storage, you’ll have access to the past 30 days’ worth of data once your dashboard is set up. Each subsequent day, the oldest day’s data is removed, meaning you consistently have a 30-day data window. Our primary objective is to offer a monitoring tool, which doesn’t necessitate extensive data storage. This feature is also useful for those setting up their own monitoring using our repository, as initially, they won’t have access to data until their local monitoring begins to collect it.

Does that answer your questions?

Thanks for your thorough answers @avtishin yes that definitely answers my questions! I'll reply here:

Ah thanks for clarifying, that makes a lot more sense.
Got it, thanks yep that's what I was getting at - the ability to host your own monitoring system if you wanted to. I also wanted to bring up the bot code specifically because, as specified in our guidelines, all code delivered "must be open-sourced, and it must also not rely on closed-source software for full functionality". So beware that we would likely ask you to make it public upon submitting the first milestone.
Makes sense! Thanks for the explanation.

That being said, I will go ahead and mark the application as ready for review and ping the committee internally for questions/comments.

@keeganquigley thank you again very much for your questions. Regarding 2. rules are the rules, we will prepare a code for deploying a tg bot. However, it's not possible to make deployment fully automized because the tg bot relies on personal tg token and github actions (we described it in the proposal, btw), which are needed to configure manually. So, we will prepare an instruction for all of it.

Hey @keeganquigley am I correct that we should receive more more approve?

That's correct. You need one more approval. I ping the rest of the team again.

Hi @avtishin how is milestone 1 coming along?

Hey @keeganquigley https://github.com/w3f/Grant-Milestone-Delivery/pull/971 -- we are working on comments by external evaluator

Whoops sorry @avtishin didn't realize you had submitted a delivery. Thanks!

Hi @avtishin how is milestone 2 coming along?

@keeganquigley hey, planing to publish till the end of the year

Hi @avtishin still planning to submit M2 soon?

Yes, we are on it

Hi @avtishin are you able to provide an update? If you don't think a delivery will be submitted in the next couple of weeks we would ask for an amendment PR to extend the timeline.

Let's say end of the next week otherwise we will do PR to extend

@keeganquigley https://github.com/w3f/Grant-Milestone-Delivery/pull/1155 here we are!


Build pyOCD into Single Executable File - file missing in pyyaml

I followed the guide https://github.com/mbedmicro/pyOCD/blob/master/docs/how_to_build.md to build an executable from pyOCD. Before diving into the problem, let's first list my system settings.

1. My system

I work on 64-bit Windows 10
I've got Python 3.8
I did not set up a virtual environment when starting the procedure, but work directly from my Python 3.8 installation.

2. The procedure
I issue the following commands in a Windows console:
# We need to use upstream version of pyinstaller due to
# http://comments.gmane.org/gmane.comp.python.pyinstaller/6457
pip install https://github.com/pyinstaller/pyinstaller/archive/develop.zip

# Create single-file executables
pyinstaller --onefile pyocd/tools/gdb_server.py
pyinstaller --onefile pyocd/tools/flash_tool.py
pyinstaller --onefile pyocd/tools/pyocd.py

Everything goes well until the last command:
FileNotFoundError: [Errno 2] No such file or directory:
'c:\\python38\\lib\\site-packages\\pyyaml-5.2b1-py3.8-win-amd64.egg\\EGG-INFO\\top_level.txt'

I check if I've got pyyaml properly installed. I run the command pip install pyyaml and everything seems okay:
C:\Seafile\EMBEETLE IDE\pyOCD>pip install pyyaml
    Requirement already satisfied:
    pyyaml in c:\python38\lib\site-packages\pyyaml-5.2b1-py3.8-win-amd64.egg (5.2b1)

3. Solution
I believe I found a solution. I created the file top_level.txt myself in the right place:

After googling the keywords yaml and top_level.txt, I discovered that this file should have only two lines:
_yaml
yaml

I issue again the failed command:
$ pyinstaller --onefile pyocd/tools/pyocd.py

Now it works! At least - the compilation works.

4. Problem with the executable
Unfortunately, the executable itself doesn't work. I get import errors:
C:\Seafile\EMBEETLE IDE\pyOCD\dist>pyocd.exe --version
    Traceback (most recent call last):
      File "pyocd\tools\pyocd.py", line 35, in <module>
    ImportError: attempted relative import with no known parent package
    [7508] Failed to execute script pyocd

Sorry, that how_to_build.md document is waaay out of date. It should be removed.
Take a look at my feature/binary_release branch. This has a .spec file that will correctly build pyocd using PyInstaller. The branch is a little out of date, but should still work if you rebase it.
I haven't merged the changes from this branch because I intend to integrate it into CI.

Thank you @flit for your very quick answer ^_^
I will give it a try :-)

Sure, that would have the same effect as rebasing.

Hi @flit ,
Maybe it was a naive attempt, but this is what I just tried:
1. I cloned the master branch of PyOCD to a folder on my computer.
2. I copy-pasted your pyocd.spec into that folder.
3. I install a couple of Python packages, because of your suggestions in dev-requirements.txt:
pip install pytest
pip install pytest-cov
pip install coverage
pip install elapsedtimer
pip install pylint
pip install tox

The pyinstaller package should already be okay on my computer, because I had already issued the following command before:
pip install https://github.com/pyinstaller/pyinstaller/archive/develop.zip

4. I navigate to the PyOCD folder and issue the command pyinstaller pyocd.spec:
C:\Seafile\EMBEETLE IDE\pyOCD>pyinstaller pyocd.spec

59 INFO: PyInstaller: 4.0.dev0+9dd34bdfba
59 INFO: Python: 3.8.0
60 INFO: Platform: Windows-10-10.0.18362-SP0
61 INFO: UPX is not available.
Traceback (most recent call last):
  File "<string>", line 2, in <module>
ModuleNotFoundError: No module named 'capstone'
Traceback (most recent call last):
  File "c:\python38\lib\site-packages\PyInstaller\utils\hooks\__init__.py", line 319, in get_module_file_attribute
    attr = loader.get_filename(package)
AttributeError: 'NoneType' object has no attribute 'get_filename'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\python38\lib\runpy.py", line 192, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "c:\python38\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Python38\Scripts\pyinstaller.exe\__main__.py", line 7, in <module>
  File "c:\python38\lib\site-packages\PyInstaller\__main__.py", line 112, in run
    run_build(pyi_config, spec_file, **vars(args))
  File "c:\python38\lib\site-packages\PyInstaller\__main__.py", line 63, in run_build
    PyInstaller.building.build_main.main(pyi_config, spec_file, **kwargs)
  File "c:\python38\lib\site-packages\PyInstaller\building\build_main.py", line 732, in main
    build(specfile, kw.get('distpath'), kw.get('workpath'), kw.get('clean_build'))
  File "c:\python38\lib\site-packages\PyInstaller\building\build_main.py", line 679, in build
    exec(code, spec_namespace)
  File "pyocd.spec", line 52, in <module>
    capstone_libs = collect_dynamic_libs("capstone")
  File "c:\python38\lib\site-packages\PyInstaller\utils\hooks\__init__.py", line 673, in collect_dynamic_libs
    pkg_base, pkg_dir = get_package_paths(package)
  File "c:\python38\lib\site-packages\PyInstaller\utils\hooks\__init__.py", line 535, in get_package_paths
    file_attr = get_module_file_attribute(package)
  File "c:\python38\lib\site-packages\PyInstaller\utils\hooks\__init__.py", line 337, in get_module_file_attribute
    raise ImportError
ImportError

Hi @flit ,
I just noticed my question is actually not complete. I've added a fourth paragraph explaining what happens when I run the resulting executable.
My apologies.

Why wouldn't just install directly from dev-requirements.txt?
Similar question for pyinstaller, why not just pip install pyinstaller?
For the error, just install capstone.

Hey @flit, @kristofmulier is kinda fresh in Python, he is just moving away from OpenOCD towards pyOCD.. for some reason still wants to build binaries, where VirtualEnv and interpreter would do the job, but that seems like a nice training anyway.. after some time he will get into that point :-)
Kris you can install all python dependencies bundle with pip install -r dev-requirements.txt this is why that file was created :-) If you want to install capstone just run pip install captone.
Read the docs in the first place :-)

https://pip.pypa.io/en/stable/
https://pyinstaller.readthedocs.io/en/stable/

Hi @flit and @cederom ,
Thanks a lot to both of you guys. It worked!
I installed capstone as suggested:
pip install capstone

Then I navigated to the cloned PyOCD folder and issued the command:
pyinstaller pyocd.spec

with 'pyocd.spec' being the spec-file from @flit .
I get a nice pyocd.exe in the dist folder. I run it to see its version, and that works:
C:\Seafile\EMBEETLE IDE\pyOCD\dist>pyocd.exe --version
    0.23.1.dev10

Note:
I just noticed the following in the error message from yesterday:
ModuleNotFoundError: No module named 'capstone'

I should have seen that before asking you guys for help. My sincere apologies.

No worries. I'm really glad it worked! Sorry if I sounded rude earlier, was just in a hurry.
Can this issue be closed?

Hi @flit , no worries :-)
Thanks a lot for your great help.
I think it would be good to push your pyocd.spec and dev-requirements.txt files from https://github.com/flit/pyOCD/tree/feature/binary_release to the master branch.
This way, anybody can compile PyOCD.
Again many thanks for your help :-)

The intent was to integrate CI support first, before merging the feature/binary_release branch. Though I guess it would make sense to go ahead and merge and deal with CI later.

I also need to look into PyOxidizer.

Hi @flit ,
PyOxidizer looks promising. So that would replace PyInstaller?

Yep.

In the meantime, if you are still planning to use PyInstaller I'd recommend to have a look at perhaps implementing the logic from the spec file as a hook in pyOCD, since there are plans to provide a way to expose them to PyInstaller: https://github.com/pyinstaller/pyinstaller/issues/4232
Btw, also wanted to thank you @flit for that spec file in your https://github.com/flit/pyOCD/tree/feature/binary_release branch, it has certainly helped me as well.

It looks like the binary_release branch is no longer in the repo.
Is there any way to see what was in that .spec file?

In case its useful, this is the spec file I'm currently using (you can ignore the stuff that excludes and replaces svd files, as I am doing that only to reduce the executable size):
https://github.com/carlosperate/ubittool/blob/master/package/pyinstaller-cli.spec

Thanks so much for taking the time @carlosperate, That solved the issue we were having.


HttpClient crashes when using lantern VPN on Ubuntu.

Dart SDK Version (dart --version)
Dart VM version: 2.5.0-dev.1.0 (Unknown timestamp) on "linux_x64"
OS: Ubuntu 18.04
Issue: flutter pub get error, reproduced by the following sample code.

Hi, Dart team,
I get an error when updating flutter sdk using flutter pub get. same issue as https://github.com/flutter/flutter/issues/12301. I think the root cause may related to the dart:io package.
Here's my analyze:

I set up a Dart Command line app and reproduced the error:

`
import "dart:io";
void main() {
var google = "https://www.google.com/";
var client2 = HttpClient();
// same crash
//  client2.findProxy = (uri) {
//    return "PROXY <IP_ADDRESS>:45653";
//  };
// same crash
client2.findProxy = (url) {
return HttpClient.findProxyFromEnvironment(
url, environment: {"http_proxy": "<IP_ADDRESS>:45653", "https_proxy": "<IP_ADDRESS>:45653"});
};
// no respond.
//  client2.findProxy = HttpClient.findProxyFromEnvironment;
// no respond.
//  client2.findProxy = null;
client2.getUrl(
Uri.parse(google))
.then((HttpClientRequest request) {
print(request.headers);
return request.close();
}).then((HttpClientResponse response) {
// Process the response.
print(response.headers);
});
}
`

Error message is:

Unhandled exception: NoSuchMethodError: The setter 'readEventsEnabled=' was called on null. Receiver: null Tried calling: readEventsEnabled=false #0      _rootHandleUncaughtError.<anonymous closure> (dart:async/zone.dart:1112:29) #1      _microtaskLoop (dart:async/schedule_microtask.dart:41:21) #2      _startMicrotaskLoop (dart:async/schedule_microtask.dart:50:5) #3      _runPendingImmediateCallback (dart:isolate-patch/isolate_patch.dart:116:13) #4      _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:173:5)

VPN settings:

I think this may related to my VPN environment settings. The following  parameters are set automatically by Lantern VPN software.  https://github.com/getlantern/lantern
`lixiang@lixiang-TM1701:~$ env | grep proxy
https_proxy=http://<IP_ADDRESS>:45653/
http_proxy=http://<IP_ADDRESS>:45653/
no_proxy=localhost,<IP_ADDRESS>/8,::1
`

curl is working.

curl https://www.google.com

JAVA is working with proxy
`
package com.tony.builder;

import org.apache.http.HttpEntity;
import org.apache.http.HttpHost;
import org.apache.http.ParseException;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.util.EntityUtils;
import java.io.IOException;
public class Main {
public static void main(String[] args) {
    String result = sendGet("http://www.google.com");
    System.out.println(result);
}

public static String sendGet(String url) {
    HttpHost proxy = new HttpHost("<IP_ADDRESS>", 45653, "http");

    RequestConfig defaultRequestConfig = RequestConfig.custom()
            .setProxy(proxy)
            .build();

    CloseableHttpClient httpclient = HttpClients.custom().setDefaultRequestConfig(defaultRequestConfig).build();

    HttpGet httpget = new HttpGet(url);
    CloseableHttpResponse response = null;
    try {
        response = httpclient.execute(httpget);
    } catch (IOException e1) {
        e1.printStackTrace();
    }
    String result = null;
    try {
        HttpEntity entity = response.getEntity();
        if (entity != null) {
            result = EntityUtils.toString(entity);
        }
    } catch (ParseException | IOException e) {
        e.printStackTrace();
    } finally {
        try {
            response.close();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
    return result;
}

}
`
Any ideas or suggestion will be helpful, thanks.
@sgjesse https://github.com/sgjesse Maybe you have more ideas? thanks.

/cc @ZichangG

Status update:
I build a local sdk, and find that the crash is because:
in secure_socket.dart, we got a null socket object.
  static Future<RawSecureSocket> secure(RawSocket socket,
      {StreamSubscription<RawSocketEvent> subscription,
      host,
      SecurityContext context,
      bool onBadCertificate(X509Certificate certificate),
      List<String> supportedProtocols}) {
     print('RawSecureSocket.secure socket = $socket');
    **//crashed at the following line. socket == null**
    socket.readEventsEnabled = false;
    print(' socket.readEventsEnabled = false;');
    socket.writeEventsEnabled = false;
    return _RawSecureSocket.connect(
        host != null ? host : socket.address.host, socket.port,
        is_server: false,
        socket: socket,
        subscription: subscription,
        context: context,
        onBadCertificate: onBadCertificate,
        supportedProtocols: supportedProtocols);
  }

I added some log to track why socket == null, it seems that after createProxyTunnel send CONNECT message,  the connection is closed and socket is set to null.
_HttpClient
_HttpClient._openUrl, method = get, uri = https://www.google.com/
_HttpClient._getConnection, uriHost = www.google.com, uriPort = 443
_ConnectionTarget.connect www.google.com 443 <IP_ADDRESS>  45653 Instance of '_HttpClient'
_ConnectionTarget.connect isSecure && proxy.isDirect = false
_ConnectionTarget.connect connectionTask, return from startConnect
_ConnectionTarget socketFuture 1 times trying, socket = Instance of '_Socket'
before connection.createProxyTunnel
_HttpClientConnection.createProxyTunnel host = www.google.com port = 443 proxy.host = <IP_ADDRESS> proxy.port = 45653 false
_HttpClientConnection.send //www.google.com:443 443 CONNECT Instance of '_Proxy'
send()
_HttpClientRequest.close
_HttpClientRequest.get done _response = null
_HttpClientRequest.get return _response = Instance of 'Future'
_HttpClientRequest.get done _response = Instance of 'Future'
_HttpClientRequest.get return _response = Instance of 'Future'
send() _streamFuture send socket = Instance of '_Socket'
send() got response
send() incoming.dataDone incoming.upgraded = false 200
send() incoming.dataDone closing = false, _dispose = true, incoming.headers.persistentConnection = true true
send() incoming.dataDone - > destroy()
destroy()
_Socket.destroy
_Socket._closeRawSocket
createProxyTunnel, get response from tunnel request. connectionInfo = null 200 OK
createProxyTunnel, SecureSocket.secure. Instance of '_Socket'
SecureSocket.secure Socket socket = Instance of '_Socket'
_Socket.detachRaw
_Socket.detachRaw._detachReady.future. null
SecureSocket.secure detachedRaw = null
RawSecureSocket.secure socket = null
Unhandled exception:
NoSuchMethodError: The setter 'readEventsEnabled=' was called on null.
Receiver: null
Tried calling: readEventsEnabled=false
#0      _rootHandleUncaughtError. (dart:async/zone.dart:1112:29)
#1      _microtaskLoop (dart:async/schedule_microtask.dart:41:21)
#2      _startMicrotaskLoop (dart:async/schedule_microtask.dart:50:5)
#3      _runPendingImmediateCallback (dart:isolate-patch/isolate_patch.dart:116:13)
#4      _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:173:5)
socket should be closed at here:
http_impl.dart ->
_HttpClientRequest send(Uri uri, int port, String method, _Proxy proxy) {
......
          if (!closing &&
              !_dispose &&
              incoming.headers.persistentConnection &&
              request.persistentConnection) {
            // Return connection, now we are done.
            _httpClient._returnConnection(this);
            _subscription.resume();
          } else {
            print('send() incoming.dataDone - > destroy()');
            ///////////////////////////
            destroy();
          }

Is this closing action corrent? should it keep a connection fot http proxy? please advise, thanks.
@ZichangG

Thanks for posting! Let me assign myself first and take a look later.

Hi @ZichangG,
I think I found the cause: Lantern's response has a Keep-Alive field but dart sdk did not parse it and closed the socket.
I set up a tinyproxy as a test proxy server, the HttpClient could use it as the http proxy tunnel.
I used tcpdump to compare the Http Response of tinyproxy vs. lantern:

tinyproxy:

HTTP/1.0 200 Connection established
Proxy-agent: tinyproxy/1.8.4

lantern:

HTTP/1.1 200 OK
Date: Wednesday, 14-Aug-19 16:13:22 CST
Keep-Alive: timeout=58
Content-Length: 0

I tracked the code that in http_parser.dart, _HttpParser._onData will process the response. And when received lanterns response, because of "Content-Length: 0", _HttpParser will close the socket, which will cause the above crash.
bool _headersEnd() {
......
if (_transferLength == 0 ||
(_messageType == _MessageType.RESPONSE && _noMessageBody)) {
_reset();
var tmp = _incoming;
// socket will closed here as "Content-Length: 0"
_closeIncoming();
_controller.add(tmp);
return false;
} else if (_chunked) {
_state = _State.CHUNK_SIZE;
_remainingContent = 0;
} else if (_transferLength > 0) {
_remainingContent = _transferLength;
_state = _State.BODY;
} else {
// tinyproxy will go to this branch. not closing socket
// Neither chunked nor content length. End of body
// indicated by close.
_state = _State.BODY;
}
currenttly, I add a _keepAlive flag to indicate that even content length == 0, it should not close the socket. and it works for me.
void _doParse() {
...
if (headerField == "keep-alive") {
_keepAlive = true;
}
...
if ((_transferLength == 0 && !_keepAlive)
...
Next, I am going to check the definition of "Keep-Alive: timeout=58", check if it need to translate into more details.
Thanks.

Thanks for the effort! But I'm not an expert on this area. The problem is parser does not handle this field correctly, considering both httpserver and httpclient both have a field called [idleTimeout]. Maybe RFC2616 can give more info on how to parse [keep-alive].

@mkustermann I saw you were the reviewer of original cl. Any thought?

@ZichangG The c6e632bcdba you refer to seems only related to be more lenient when parsing the headers. I don't think parsing is the problem here.
It seems that dart:io's proxy code is issuing a CONNECT and if it gets a response with content-length: 0 / keep-alive: ... it closes the connection where it shouldn't.
@sortie Could you take a look at this?

Status update:
Hi Dart team,
I re-checked the issue and found that when _HttpParser received a lantern's HttpRespond with empty body, it closed the HttpIncoming buffer, which I think it is reasonable behavior. But after that, the call back funtion located in _HttpClientConnection.send closed the socket, which is unconsiderable.
I think at this point, I should check if it is a "CONNECT"'s response and return directly if success.
So I modified the code, only change 1 line in http_impl.dart. And in this way, HttpClient will behave the same using tinyproxy or lantern.
Please check the patch atached.
Thanks.
0001-fix-crash-issue-when-HttpClient-using-lantern-proxy.patch.txt

Is there a solution?

@liamappelbe u are very good

Closing because as far as I understand, 236dd991e540e26db22473998cc5354051464bf1 fixes the issue.

@sortie It does not seem to be resolved, I still have this problem in version 2.8.1. The tag of 236dd991e540e26db22473998cc5354051464bf1 contains version 2.8.1

@zichangg What do you think?

I can confirm the problem does have. I'm able to have a repro locally. Work on the fix now.

Not only exists with Ubuntu, this is a bug not caused by any platform, but the code bug of Http Client

I've just started getting this issue in a Flutter project, due to an update to the Proxyman app on OSX.
My Dart/Flutter details:
 Flutter (Channel beta, 1.18.0-11.1.pre, on Mac OS X 10.15.4 19E287, locale en-AU)
    • Flutter version 1.18.0-11.1.pre at /Users/athor/dev/flutter
    • Framework revision 2738a1148b (5 weeks ago), 2020-05-13 15:24:36 -0700
    • Engine revision ef9215ceb2
    • Dart version 2.9.0 (build 2.9.0-8.2.beta)

Due to a bunch of issues on null safety, the review process is quite slow recently. It probably needs another 1 or 2 days to land. Also thanks to @lixiangthinker  for diagnosing the problem.

still have this problem in Flutter 1.20.0-1.0.pre
fluttert --version:
Flutter 1.20.0-1.0.pre • channel dev • https://github.com/flutter/flutter.git
Framework • revision f73f498da1 (4 days ago) • 2020-06-18 08:23:22 -0700
Engine • revision c0d52b98d1
Tools • Dart 2.9.0 (build 2.9.0-16.0.dev 2b917f5b6a)

@lyf571321556 "dart" you were using was still build 2.9.0-16.0.dev. The fix went in 2.9.0-19.0.dev.


Minor tweaks to README

Thanks @eliotsykes !

You're welcome @michelson - and thanks for taking the initiative to build an engine like this, I've been keeping my fingers crossed for something like this to appear (and too lazy to do it myself!)

haha, nice!, I will glad to help if you have questions on how to implement this on a production rails app
best!


how-tos: document image builds for VPN clients

Document the process followed for the OpenVPN image in
https://github.com/openshift/release/pull/32777

Example of this setup being used in: https://issues.redhat.com/browse/DPTP-2852.

https://deploy-preview-302--zealous-shirley-0f56b3.netlify.app/docs/how-tos/adding-a-cluster-profile#image


Documentation does not list all domains required to create redis cache

In the section Additional vnet network connectivity requirements, the documentation lists the domains required for redis cache to function properly. However, when I tried to create a redis cache with only the listed domains added to my FQDN filter, I was still encountering deployment failures.
After trial and error, I was able to successfully create a redis cache instance with the following FQDN whitelist:
#Storage
.table.core.windows.net,.blob.core.windows.net,.queue.core.windows.net,.file.core.windows.net
#PKI
.ocsp.digicert.com,.crl4.digicert.com,.ocsp.msocsp.com,.mscrl.microsoft.com,.crl3.digicert.com,.cacerts.digicert.com,.oneocsp.microsoft.com,.crl.microsoft.com,ocsp.digicert.com,crl4.digicert.com,ocsp.msocsp.com,mscrl.microsoft.com,crl3.digicert.com,cacerts.digicert.com,oneocsp.microsoft.com,crl.microsoft.com
#Metrics
.shoebox2-black.shoebox2.metrics.nsatc.net,.north-prod2.prod2.metrics.nsatc.net,.azglobal-black.azglobal.metrics.nsatc.net,.shoebox2-red.shoebox2.metrics.nsatc.net,.east-prod2.prod2.metrics.nsatc.net,.azglobal-red.azglobal.metrics.nsatc.net,shoebox2-black.shoebox2.metrics.nsatc.net,north-prod2.prod2.metrics.nsatc.net,azglobal-black.azglobal.metrics.nsatc.net,shoebox2-red.shoebox2.metrics.nsatc.net,east-prod2.prod2.metrics.nsatc.net,azglobal-red.azglobal.metrics.nsatc.net
#Newly discovered domains
.events.data.microsoft.com,.windowsupdate.com,*.vault.azure.net
Can someone confirm whether or not my list is sufficient? Can the missing domains be added to the documentation? Thanks!

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 3bced23e-5c67-3a63-06f7-c659ba56b5e7
Version Independent ID: 10d48e1f-e090-ab51-553c-64655a102c9d
Content: Configure a Virtual Network - Premium Azure Cache for Redis
Content Source: articles/azure-cache-for-redis/cache-how-to-premium-vnet.md
Service: cache
GitHub Login: @yegu-ms
Microsoft Alias: yegu

@yichungyu Thank you for the feedback. We are actively investigating and will get back to you soon.

@yichungyu This is being assigned to the content author to evaluate and update as appropriate.

Thanks for the updates @AnuragSharma-MSFT !

Hi @AnuragSharma-MSFT could you please also help me with another question? My customer is also concerned with the data that is being transferred over port 80, and wishes to ask us if we can confirm that no sensitive data is being transferred from the point of view of Redis. Do you by any chance have that info, or know someone whom I can check with? If you prefer to chat over Teams, you should be able to find me by searching for Joe Yu. I'm also MSFT. Thanks!

Thanks for your dedication to our documentation. Unfortunately, at this time we have been unable to review your issue in a timely manner and we sincerely apologize for the delayed response. Though the scope of our feedback channel here on GitHub covers specific documentation fixes, we can help redirect you to the right support channel to get an answer to your question: https://docs.microsoft.com/en-us/answers/topics/azure-cache-redis.html. Because this issue does not relate to documentation, we are closing this issue. #please-close


Increase duration for tests using containers

Sometimes a network glitch causes these tests take a little bit
more than 5 minutes to complete (fetch image and dnf metadata).

Encountered today in #885 and #886. Now all tests under the /plans/features/advanced should be covered.

Hmmm, or shall we rather raise the 5m default for all tests?

yeah, well, I have to say I often hit the default timeout :) so raising seems valid to me. TBH I would like to see default raised to even hour, which is like a standard for gitlab (and I believe also others)
https://gitlab.cee.redhat.com/help/ci/pipelines/settings#timeout

At least for the shell based tests, and tests directly in execute step ....

This part is already covered: https://tmt.readthedocs.io/en/latest/spec/plans.html#shell


test: add more /container/name/start tests

Signed-off-by: letty<EMAIL_ADDRESS>
1.Describe what this PR did
2.Does this pull request fix one issue?

3.Describe how you did it
4.Describe how to verify it
5.Special notes for reviews

LGTM


OCPBUGS-39108: avoid appending directory as file path in s3 driver Walk

when a directory is empty, the s3 api lists it with a trailing slash. this causes the path to be appended twice to the walkInfo slice, causing purge uploads path transformations to panic when the _uploads is emtpy.
this adds a check for file paths ending on slash, and do not append those as regular files to the walkInfo slice.

Putting this on hold until we get QE approval.
/hold

/jira refresh

/lgtm

/label qe-approved

/hold cancel


CA - Amascut's Remnant Doesn't Appear to be Completable Repost of #6003 - Wasnt Fixed

The issue was not fixed. Tested after the latest news post and it still does not award the Combat Achievement.

#6003 Shouldn't have been closed yet. thats my by bad. the fix was for OSB only. BSO will take longer to get the fix implemented. I've added the fix in this PR https://github.com/oldschoolgg/oldschoolbot/pull/5984 which should be on the chopping block to get merged.

Tried doing this on OSB couple days ago, didn't get the CA. Is it still broken?

Tried doing this on OSB couple days ago, didn't get the CA. Is it still broken?

This thread is for bso specifically and i don't believe CA are working properly on BSO. Post your send/return and none-completed CA in a new thread and i can investigate it more for OSB.


Cocoapod Issue

When I add the pod 'FaveButton', and install the pod file, I get the error message:
[!] Unable to find a specification for FaveButton
Can you help me solve this?

I have solved the issue, I run the code pod setup in terminal and then running pod install solved the issue. Successfully installed.


[Search Experience][Langchain]Add a notebook to demo approx search with NLP model

Add a notebook on how to perform a Similarity search with approx search and hugging face model minilm-l6-v2

Nice! Couple of minor changes

change the name of the notebook to be "langchain-using-own-model"
"Note: For demonstration we will explicity set strategy to ElasticsearchStore.ApproxRetrievalStrategy although ElasticsearchStore uses ApproxRetrievalStrategy strategy by default.

use vector_store = ElasticsearchStore.from_documents instead
-> use vector_store for the similarity query


Specify whether InfluxDBClient can be reused

Currently I'm creating a new InfluxDBClient for each query/write but it may be beneficial to re-use the same instance. HttpClient can reuse the connection for example. I'm not so sure reusing the instance will work as intended given the usage of HttpClient doesn't appear to follow the current guidelines. The easiest solution would be to implement IHttpClientFactory I reckon. It'll be nice to specify in the docs if reuse is supported, and perhaps look into making it reusable if that currently isn't the case.

Hi @ben-propflo,
Thank you for using our client.
Our intention is for the client to be used as a singleton - a long-lived client. You can find the client initialization here: https://github.com/bonitoo-io/influxdb3-csharp/blob/75cc2f7c88fc8c1ec039d83a1ef6be474469a55f/Client/InfluxDBClient.cs#L315.
Best

Great thanks @bednar. I was going to raise a PR to fix the bit about respecting DNS changes but it appears netstandard doesn't expose the property required. This sums it up nicely https://github.com/dotnet/aspnetcore/issues/28385#issuecomment-853766480


Different number of devices initially found and finally added!

Great Integration! I have a problem though. while I installed it and it finds 12 devices initially only nine appear in my configiration.
I am adding here a link of the initially found devices and finally.
https://photos.app.goo.gl/7z3ZrSysPuvWqP288
https://photos.app.goo.gl/UKDAm9NYTWJf8Z9y6
Thanks in advance.

lumi.sensor_switch.v2 and xiaomi.repeater.v2 have no any properties in miot-spec.


Set bounds on the parsed line number int in developer API

This ensures there is no overflow from the YAML line number when reported in the developer API
Fixes https://github.com/authzed/spicedb/security/code-scanning/1

This fix was included in #378.


Add manual job to freeze image tags and versions after code freeze

Description
Add manual job to freeze image tags and versions
Issues
n/a
Type of change
List the type of change like below. Please delete options that are not relevant.

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds new functionality)
[ ] Breaking change (fix or feature that would break existing design and interface)

Dependencies
n/a
Tests
n/a

we need a NEWTAG environment variable when call update_manifest.sh to set the opea image tag in GMC manifest

we need a NEWTAG environment variable when call update_manifest.sh to set the opea image tag in GMC manifest

Yes, I did by
        env:
          NEWTAG: ${{ inputs.newappversion }}


e2e: pr success testThu, 30 Nov 2017 06:22:51 GMT

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

E2E_DocsBranch_Internal_Dynamic/index.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


It would be nice to have a flag to check the version of the probe

It would be nice to have a flag to check the version of the probe. Currently I am unable to find the version of the probe that is running on my docker image. Will be really helpful.

@raniloptus valid request. You can probably compare the checksums, assuming it was downloaded from github releases section? We should look at adding this at some point.

now that the go embeds git version info directly, this should be easy to implement


Add FirstByteIntPartitioner

Add a partitioner that allows for manually specifying a partition to write to. This is done by composing the first four bytes of the object as a byte array into an integer value and then applying the module of the number of partitions.

Abandoning


Small engineering issue in the front-end

[Sorry if this is a duplicate; I swear I posted this a while back, but now I don't see it.]
As background: P4_16's Bison parser relies the "lexer hack." That is, the parser maintains a symbol table that records, for each identifier, whether it is the name of a type or just an ordinary name. This symbol table is consulted by the lexer to produce two distinct tokens IDENTIFIER and TYPE_IDENTIFIER. And these tokens are treated differently in the parser.
Currently, the P4_16 front-end receives the entire program -- i.e., after the C preprocessor has run. And the declarations are processed from start to end. So, per language in the spec,

The order of declarations is important; with the exception of parser states, all uses of a symbol must follow the symbol's declaration. (This is a departure from P414, which allows declarations in any order. This requirement significantly simplifies the implementation of compilers for P4, allowing compilers to use additional information about declared identifiers to resolve ambiguities.)

the front-end can determine which identifiers denote types and which ones do not.
If we are developing a system where smaller program pieces are processed, we may need to refactor the front-end. Currently, if you point the parser at a file that starts like this:
parser MyParser(packet_in...)

you'll get a syntax error, because packet_in, which is declared in core.p4 is not known to be a type.
The problem may persist even if we imagine a syntax like this:
import core;
parser MyParser(core.packet_in...)

unless a side-effect of parsing the tokens import core causes the symbol table in the parser to realize that core.packet_in is now a type.
This is not a show-stopping issue, but there are some details to work out. For instance, one approach could be to write a lighter-weight front-end to parse and analyze just the import statements, and topologically sort them into the right order so references are known when they are encountered. (But there are questions about when the C pre-processor runs.) Alternatively, we could interrupt parsing when we get to an import and go off and actually parse and load the referenced module. (But that makes parsing even more effectful, and we also need to be super careful about introducing loops!) And probably there are other solutions...
What we can't do, is to load files using the existing parser (or a small extension to it) because the "lexer hack" means we don't even get an AST for program pieces like the snippet above.

There is another way of doing this. Rewriting the parser to be a top-down parser which will give a better control over even error messages. I don't think there is a modern C/C++ front-end which even uses bison these days even.
I am actually working on my own front-end to parse P4-16 which could be reused for this. I think Marvell is willing to open source it and contribute it if wanted. The AST is different from the reference compiler and is still rough round the edges right now even.
The tokenizer just does identifiers and then does looks while parsing if it is a type or not. I have been thinking about how to add namespace/modules support (it should not be hard; there is one case where there is an issue dealing parsing of "(type." vs "(type<>)" which will needed to be extended for modules but should not be hard).


fix: run workflow when main completed

Thanks. This allowed for automatically triggering tagged-release workflow. However, there's still issue that it cant collect the version tag. Investigating.


Patch Event wrong path

in events_api.py>EventsApi>patch_event_with_http_info
the call is made to /events/{id} which returns HTTP response body: b'{"code":404,"message":"Resource Not Found","description":"The resource was not found."}'
        return self.api_client.call_api(
            '/events/{id}', 'PATCH',
            path_params,
            query_params,
            header_params,
            body=body_params,
            post_params=form_params,
            files=local_var_files,
            response_type='Event',  # noqa: E501
            auth_settings=auth_settings,
            async_req=params.get('async_req'),
            _return_http_data_only=params.get('_return_http_data_only'),
            _preload_content=params.get('_preload_content', True),
            _request_timeout=params.get('_request_timeout'),
            collection_formats=collection_formats)

I think this should be /event/{id}

Hello,
Thank you for reporting this issue. It is fixed in https://github.com/elabftw/elabapi-python/releases/tag/0.4.1


Necklace of Skulls and Lotus Statue - Now work with Vampires!, change description

This issue's commits have been done at (#665).