[Bug] Newtype enum variant deserialization failure when used with #[serde(flatten)]

TL;DR
If you create an enum that contains a newtype variant, wrap the enum in a struct, wrap the said struct in another struct and use #[serde(flatten)] on it, the struct can no longer successfully deserialize (while serialization is unaffected).
Deserialization fails with message: untagged and internally tagged enums do not support enum input.

That's not a very helpful description so here's a MRE:
use serde::{Deserialize, Serialize};

fn main() {
    // this works
    let gender = Gender::Other("F22".into());
    assert_eq!(
        &gender,
        &serde_yaml::from_str(&dbg!(serde_yaml::to_string(&gender).unwrap())).unwrap()
    );

    // this still works
    let info = Info { gender };
    assert_eq!(
        &info,
        &serde_yaml::from_str(&dbg!(serde_yaml::to_string(&info).unwrap())).unwrap()
    );

    // this errors
    let user = User { info };
    assert_eq!(
        &user,
        &serde_yaml::from_str(&dbg!(serde_yaml::to_string(&user).unwrap())).unwrap()
    );
}

#[derive(Clone, Debug, Eq, PartialEq, Deserialize, Serialize)]
enum Gender {
    Female,
    Male,
    Other(String),
}

#[derive(Clone, Debug, Eq, PartialEq, Deserialize, Serialize)]
struct Info {
    gender: Gender,
}

#[derive(Clone, Debug, Eq, PartialEq, Deserialize, Serialize)]
struct User {
    #[serde(flatten)]
    info: Info,
}

Misc
I tried the exact same example using serde_json, no errors. So I think this is a serde_yaml issue rather than a serde issue.
A quick search did find this issue though, not sure if it's related: https://github.com/ron-rs/ron/issues/217.
Versions

OS: Linux 6.0.8 x86_64
rust: 1.65.0 (stable-x86_64-unknown-linux-gnu)
serde: 1.0.147
serde_yaml: 0.9.14

I just experimented with different enum representations. The results may be interesting for diagnosis:
With #[serde(untagged)]
Deserialization of Gender::Other now works. But obviously this won't work because Gender::Male and Gender::Female both serialize to null.
With #[serde(tag = "foo")]
Serialization fails with message: cannot serialize tagged newtype variant Gender::Other containing a string.
This is expected because newtype by definition does not have a tag.
With #[serde(tag = "foo", content = "bar")]
This no longer errors, but of course it's very undesirable.

To add some maybe helpful information: This was introduced with the !Tag syntax of 0.9. Using serde_yaml with 0.8 works perfectly fine with #[serde(flatten)].

To add some maybe helpful information: This was introduced with the !Tag syntax of 0.9. Using serde_yaml with 0.8 works perfectly fine with #[serde(flatten)].

Also if you serialize with 0.8 (it does not use tags) you can deserialize with 0.9, but an item serialized with 0.9 cannot be deserialized with neither.

Same problem here. Had not expected that serde_yaml's Serializer would use a format that serde_yaml's Deserializer cannot parse, but this is exactly what is happening here.
Have downgraded our project to serde_yaml 0.8.26 for now.
CC @dtolnay

Your "F-22" example resembles a transphobic joke. Consider changing it.

Your "F-22" example resembles a transphobic joke. Consider changing it.

Seriously?? PC's gone mad these days.
First of all, stop the bike shedding.
Second, no I'm not going to change a word.  I would even go so far to say that the inability to take a joke is itself the ultimate phobia.

I am seeing this issue as well with the flatten and deserialization of enums.

I am seeing this issue as well with the flatten and deserialization of enums.

It seems that inside a field marked with #[serde(flatten)], the expected format for externally tagged enums changes to what serde_json uses:
use indoc::indoc;
use serde::Deserialize;
use std::{collections::HashMap, fmt::Debug};

#[derive(Deserialize, Debug)]
enum Inner {
    A,
}

#[derive(Deserialize, Debug)]
struct Outer {
    #[serde(flatten)]
    flattened_field: HashMap<String, Inner>,
}

fn main() {
    // doesn't work (as expected)
    println!("{:?}", serde_yaml::from_str::<Inner>(indoc! {"
        A: null
    "}));
    // works (as expected)
    println!("{:?}", serde_yaml::from_str::<Inner>(indoc! {"
        !A
    "}));
    // works (bug)
    println!("{:?}", serde_yaml::from_str::<Outer>(indoc! {"
        blah: { A: null }
    "}));
    // doesn't work (bug)
    println!("{:?}", serde_yaml::from_str::<Outer>(indoc! {"
        blah: !A
    "}));
}

This seems to be caused by  #[serde(flatten)] generating code that goes through serde::__private::de::content::Content for both keys and values before deserializing to the final types present in the hashmap. When serde_yaml sees the !A, it calls .visit_enum() on the corresponding serde::[...]::ContentVisitor, which results in the "untagged and internally tagged enums do not support enum input"-error that occurs in cyqsimon's original code.
I'm not sure how to fix this. Ideally the code generated by #[serde(flatten)] would directly deserialize to the desired types, or it could go through serde_yamls's Value instead of their Content, but we can't change that from here (and they probably have a good reason for that intermediate type). We could also handle !A [...] the same as A: [...] (allowing both to deserialize to an externally tagged enum), which would fix this and make serde_yaml more compatible with json generated by serde_json. But that would probably break existing custom visitors that expect !A to result in a call to .visit_enum instead of .visit_map.
I hope some of this could help. I'd like to fix this myself, but I don't know enough about serde/rust to come up with a good fix.


Adding an example

Adding an example to the Step # 1
Also, corrected the current use of the -SearchName parameter for the New-ComplianceSearchAction cmdlet which does not accepts the search name using ""
Reference Issue https://github.com/MicrosoftDocs/microsoft-365-docs/issues/1500

@get-itips Thanks for doing these. However, why you are  adding the additional step to create the variable, particular in the compliancesearchaction examples? The previous examples are technically correct and not sure the advantage of variable? AFAIK, there isn't a standard to do this.
e.g.
$SearchName="Remove Phishing Message"
New-ComplianceSearchAction -SearchName $SearchName -Purge -PurgeType SoftDelete

@get-itips Thanks for doing these. However, why you are adding the additional step to create the variable, particular in the compliancesearchaction examples? The previous examples are technically correct and not sure the advantage of variable? AFAIK, there isn't a standard to do this.
e.g.
$SearchName="Remove Phishing Message"
New-ComplianceSearchAction -SearchName $SearchName -Purge -PurgeType SoftDelete

Hello Mark, how are you? 👍
The cmdlet New-ComplianceSearchAction is not currently accepting string literals for the SearchName parameter, I tested this and the only way that it accepted the parameter value was supplying it as a string variable.
It just won't find the Compliance Search:

So I thought in providing a working example that people can actually use.

@get-itips I should have also said that creating the variable for the New- and Start-ComplianceSearch examples does make sense and is a good suggestion. Thanks for heads up about error. I will test as well.


can't disable multiple experiemtal fatures via helm config

Rancher Server Setup

Rancher version: 2.6.3
Installation option (Docker install/Helm Chart):

If Helm Chart, Kubernetes Cluster and version (RKE1, RKE2, k3s, EKS, etc): RKE1

Information about the Cluster

Kubernetes version: v1.21.7
Cluster Type (Local/Downstream): local

Describe the bug

After reading documentation and chart code I set  features to "istio-virtual-service-ui=false,harvester=false" in my helm config to disable these 2 experimental features. But with this only istio-virtual-service-ui is disabled. If I change value to "harvester=false,istio-virtual-service-ui=false" then it disables only harvester.
To Reproduce

Install rancher chart with features="istio-virtual-service-ui=false,harvester=false" in your config.
Result
Only first feature is disabled.
Expected Result

All features in the comma-separated list are disabled.

Yes, the issue is relevant to me. Also seeing "Feature flags allow Rancher to gate certain features behind flags. You should enable these features with caution, and they should be considered beta functionality, which could cause problems for your system." warning in rancher UI at /g/settings/feature-flags makes it more relevant. We should be able to disable them easily via config.

stil valid.

This was my bad, it works as expected. I was using the set option of helm_release resource in terraform provider wrong. I had to double check this, sorry.


Question: Advanced Template

Is there any implementation for the yii2 advanced template?

I have been try implement this in advanced template


MM-31717: Remove HTTP clustering

Gossip is now GA, so we remove the option
from admin console.
https://mattermost.atlassian.net/browse/MM-31717

Yeah, but there's no rush on this. I'll prefer to merge them together.

I'll assign you then just so no one else happens to merge the PR once it's done reviews.

/update-branch
Gentle ping on this @calebroseland


Formatting eats text

The formatter often eats my text, like so:
      el.parentElement.insertBefore(wrapper, el)

after:
    .parentElement.insertBefore(wrapper, el)

it happens only if I have: "formatOnSave": true in my tsconfig.json.

This has to be a duplicate of
https://github.com/TypeStrong/atom-typescript/issues/803
I had already made a guess that formatOnSave could be turned off to fix it, even before reading your note! So this gives the developers a lot to go on.
I'm betting it's a concurrency bug. For now, the developers of this plugin need to completely disable formatOnSave because it VERY RELIABLY corrupts files in a very bad way. Randomly deletes a single characters from all over the file. This is litterally the worst kind of bug a text editor can have. Random single character corruptions that can easily go unnoticed and get checked into source control. If this is not their top priority, then it would be a HUGE mistake. They need to fix on this one ASAP, before it destroys the reputation of this plugin.

Note to develpers: When you remove the 'formatOnSave' please also change the CTRL+ALT+L to something else, because for Ubuntu users this cmd logs you out of Ubuntu completely. lol. Can't have my cake and format it too.

Yes, I think it's a concurrency bug too, the way the file is corrupted hint to this direction.
Of course, until it's fixed be sure to have formatOnSave to false.


Add orx:max-pixels attribute for images

As described in opendatakit/xforms-spec#79.
The output should be of the structure
<bind ref="/path/to/node" orx:max-pixels="1024" type="binary"/>
I'm thinking something like
| survey |        |          |       |                     |
|        | type   |   name   | label | parameters          |
|        | image  | my_image |...    | max-pixels=1024  |

Does it mean that any key/value pair in parameters should be added as orx:[key]? What should determine if parameters are meant for body attributes or bind attributes or instance attributes? In range, the parameters column was applied as body attributes.
At the moment this can be achieved by a bind::attribute,
|        | type        | name           | label   | bind::orx:max-pixels    |

To hide this difficulty from the user, I think pyxform should magically put in the right location based on the known parameter name and the question type (and usually also change the user-friendly parameter name into an XForm-friendly attribute name - though not in this case, and add namespaces if required).

Should have made my thinking more explicit, thanks @MartijnR for filling in the gaps!
The way I'm thinking about parameters is that it's a convenience column that makes it possible to hide XForms details from users a bit more. Unlike appearances, values in parameters wouldn't be passed through but instead "translated" in some way, as @MartijnR describes. Does that sounds acceptable, @ukanga? Were you thinking about it in a different way?

So far, I want to use parameters for external functions e.g simprints.IDENTIFY(apiToken=XXX) which applies to a binding.
I suppose we could leave the choice to the widget to generate the appropriate XML and determine the location of the attribute.

@ukanga For the Simprints example, are you working with the Simprints folks directly? It looks like @LimaEchoCharlie has a widget implementation over at https://github.com/SimPrints/odk-collect and perhaps it's something that could be coordinated across the ecosystem and go into core. On the XLSForms side, seems it would be nice to use something like simprints-api-token=XXX in the parameters column and do a translation similar to what I'm suggesting for orx:max-pixels. Might be good to discuss in a separate issue and perhaps starting at the XForms level. It looks like the sample widget pulls the API key from a CSV but it would certainly make more sense to specify when defining a form.

Yes, we are discussing on the best way the sim print app can work with ODK Collect with minimal changes to ODK itself, I think once that's clear we can look into how to simplify the XLSForm for the user when defining external apps. We will create an issue as soon as that's clear for discussion.

That sounds good.
Are you ok with the proposal for orx:max-pixels  or would you prefer an alternate approach?

Yes, I like it.

In the JSON representation, it might be easier to describe it like:
{
  "type": "image",
  "name": "my_image",
  "label": "my label",
  "max-pixels": 1024
}

Because any number of things might feel appropriate to put into a "parameters" column.

@dorey that seems fine to me except perhaps the JSON should also be namespaced so we'd have orx:max-pixels. In effect, we'd be agreeing that as far as the XLSForm abstraction is concerned, there's only one thing called max-pixels and it is a parameter to an image. That would translate to the developer-facing JSON and ODK XForms representations that use namespace and a separate attribute/key. That makes it clear from an implementation standpoint that max-pixels is custom and places it meaningfully for each representation (on the bind in XML and as its own key in JSON) while hiding those details from users who really don't need to know them.

@ukanga, @MartijnR any last thoughts on this? I'd like to get it in soon!

Nope, no objections on this.

Good to go, I think!


support for unorchestrated rootfull and rootless Podman using OCI hook

Purpose of PR?:
This PR aims to handle enforcement for unorchestrated rootfull Podman containers using OCI hook
Fixes #1814
Does this PR introduce a breaking change?
If the changes in this PR are manually verified, list down the scenarios covered::
Additional information for reviewer? :
Mention if this PR is part of any design or a continuation of previous PRs
Checklist:

[ ] Bug fix. Fixes #
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update
[ ] PR Title follows the convention of  <type>(<scope>): <subject>
[ ] Commit has unit tests
[ ] Commit has integration tests

Hi @itsCheithanya! Do you need some assistance over this? I also researched about Podman and would be happy to contribute for this!

Hi @itsCheithanya! Do you need some assistance over this? I also researched about Podman and would be happy to contribute for this!

Thanks @Manik2708 but its ready and only need to be reviewed


feat(linter): add @typescript-eslint/consistent-type-imports rule

#3895  👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @mysteryven and the rest of your teammates on  Graphite


Write locations to file

Resolves #80
This is still not complete as I need to support iOS and I'm very new to swift.
Any help there would be greatly appreciated.
Also I'm not sure how to pass the file path there into the location update method so again, help would be appreciated.
I didn't want to add file plugin to the example, but I tested this code to work with my app basically.
Let me know how you want to proceed with this.

I've also explored the ability to use a rotating log file in order to make sure this doesn't explode in terms of disk space and it works nicely.
I still don't have a solution for iOS though...
Let me which direction seems better to you.

I would leave it to the application to manage the log file, which might include log rotation. It's too much complexity to add to the plugin.

Also I'm not sure how to pass the file path there into the location update method

I'm not sure what you mean by this. What is the problem?

I'm not sure how to pass the log file path to the relevant method in iOS. Seems weird to me how the flow of data is done in swift (mainly because I don't know swift I guess)...


Cannot read property 'registerPlugin' of undefined (using webpack)

Hello again,
So I'm trying to import pixi-tilemap in a project built using webpack. The following fails :
import 'pixi.js';
// `PIXI` now available in global scope
import 'pixi-tilemap';

When I inspect, the error seems to be thrown by this line. Any idea why this fails ? I'm already using this method for other pixi plugins (like pixi-particles).
Here are my libraries versions from yarn.lock :
<EMAIL_ADDRESS>  version "1.2.3"
  resolved "https://registry.yarnpkg.com/pixi-tilemap/-/pixi-tilemap-1.2.3.tgz#63306cb5c1c847279a249f3e1f064140f26e0470"
  dependencies:
    glslify "^5.0.2"
    typescript "^2.2.2"

<EMAIL_ADDRESS>  version "4.5.2"
  resolved "https://registry.yarnpkg.com/pixi.js/-/pixi.js-4.5.2.tgz#22b63b0c42185ac204e01086aff7ebeb60de0e0c"
  dependencies:
    bit-twiddle "^1.0.2"
    earcut "^2.0.7"
    eventemitter3 "^2.0.0"
    ismobilejs "^0.4.0"
    object-assign "^4.0.1"
    pixi-gl-core "^1.0.3"
    remove-array-items "^1.0.0"
    resource-loader "^2.0.6"

<EMAIL_ADDRESS>  version "2.6.1"
  resolved "https://registry.yarnpkg.com/webpack/-/webpack-2.6.1.tgz#2e0457f0abb1ac5df3ab106c69c672f236785f07"
  dependencies:
    acorn "^5.0.0"
    acorn-dynamic-import "^2.0.0"
    ajv "^4.7.0"
    ajv-keywords "^1.1.1"
    async "^2.1.2"
    enhanced-resolve "^3.0.0"
    interpret "^1.0.0"
    json-loader "^0.5.4"
    json5 "^0.5.1"
    loader-runner "^2.3.0"
    loader-utils "^0.2.16"
    memory-fs "~0.4.1"
    mkdirp "~0.5.0"
    node-libs-browser "^2.0.0"
    source-map "^0.5.3"
    supports-color "^3.1.0"
    tapable "~0.2.5"
    uglify-js "^2.8.27"
    watchpack "^1.3.1"
    webpack-sources "^0.2.3"
    yargs "^6.0.0"

I dont use webpack, but we have something in pixi-spine plugin: https://github.com/pixijs/pixi-spine/blob/master/sandbox/webpack.config.js
Looks like you need to specify pixi and other things in config.
webpack.config.js
module.exports = {
  entry: {
    'your_app': ['./lib/pixi-spine.js', ...]
  },
  externals: {
    'pixi.js': 'PIXI'
  },
}

I tried with externals and ProvidePlugin without success. It seems that PIXI is undefined when webpacks loads the bin/pixi-tilemap.js.

I really cant help more. I personally think that PIXI and all libs should not be processed through webpack.

I have to disagree on that ^^' Since PIXI uses es6 with module imports it allows tree shaking and other nice features with webpack ! A bit tedious to setup but once it works it rocks ! Also I really don't want to manage third-party libs as additional entries in my webpack config.
I'll dig more on webpack config, probably a simple tweak that I miss here can solve the issue. Otherwise I'll try to require the source files directly with a TypeScript loader dedicated to pixi-tilemap. If I manage to process it with webpack I'll report here ;)
Thanks for the advice anyway !

Yeah, you might have to edit resulting js file to make it compatible with webpack. It wasnt easy to make pixi-spine compatible with everything, and I still dont understand how it works.

@Oza94 did you manage to get this working?
I am currently having this same issue and don't see any alternative to pixi-tilemap

i  didnt make it compatible with webpack, sorry. you have to wait more or do it yourself.

@roryclaasen Add<EMAIL_ADDRESS>"^4.7.1" to dependencies in pixi-tilemap package.json. Build it agian, and it should be working.
The problem was: webpack compiler is creating new PIXI variable because pixi.js is missing in dependency list.

Try new version (1.2.4) from npm or from dist folder of this repo.

@ivanpopelyshev sorry for getting back late. This works a charm now thank you!


BAH-3163 | Add. node:20-alphine As Docker Base Image

In this PR, the following modifications have been implemented:
Base Image Update: The base image node has been upgraded from version 18.15.0-alpine to version 20-alpine.
Package Version Update: The package serve has been upgraded from version 13.0.4 to version 14.2.0.

Did u test if this image will work?


Dropdown inside Table Responsive in Safari of iOS 12.4

When using dropdowns inside scrollable elements, such as table-responsive, there is an option to set the dropdown's boundary to the viewport, this way the dropdown is not clipped by the scrollable parent overflow.
It works great, I use it and works fine in many browser. However it is not working on Safari for iPhone. Even with the boundary set to the viewport, the dropdown gets clipped inside the table-responsive element.
The iPhone I tested against is an IPhone 8 using iOS 12.4.
I also tested against an iPad Pro (10") using iPadOS 13 Beta. In this device everything works as expected.
I do not have access to a machine with macOS, therefore I cannot tell if this problem happens in the desktop version of Safari. I tried with Gnome/Web (Epiphany) which uses WebKit and it works as expected on desktop. In all other browsers I tested it worked as expected (Firefox, Chrome, both on Desktop)
The reduced test case is the following:
<!DOCTYPE html>

    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">

    <title>Demonstration</title>

    <link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.3.1/css/bootstrap.min.css">

                    Lorem ipsum dolor.
                    Laboriosam, optio, ratione.
                    Consequatur dolorum, quisquam.
                    Accusamus in, quaerat!
                    Ducimus excepturi, inventore.

                            <button
                                type="button"
                                class="btn btn-outline-secondary"
                                data-boundary="viewport"
                                data-toggle="dropdown">
                                Toggle
                            </button>

                                <button type="button" class="dropdown-item">
                                    Option 1
                                </button>
                                <button type="button" class="dropdown-item">
                                    Option 2
                                </button>
                                <button type="button" class="dropdown-item">
                                    Option 3
                                </button>
                                <button type="button" class="dropdown-item">
                                    Option 4
                                </button>

I also created a Code Pen with the same code, the error only happens on Safari for iPhone:
https://codepen.io/rodrigopedra/pen/gOYWQPZ
As it is difficult to me to debug this code on Safari for iOS (because there is no Mac around) I could not dive into the problem to make a suggested fix.
I searched for previous issues regarding this same problem and I found these closed issues:
#11037 / #15288 / #15374  / #24251
In any of them there were mentions to this behavior happening only on Safari for iPhone.
It was not very clear (to me at least, not a native English speaker) in the issue template if I should have added a comment on a closed issue or open a new one. As I think it is a different problem I opened a new one. Sorry if I misunderstood something.
EDIT Just tested on my son's iPad Air 2 which is also running iOS 12.4 and it also does not work as expected. I updated he issue's title to reflect that

It seems it's a positioning issue so maybe @FezVrasta can help us here

If you could please prepare an example using only Popper.js and not any of the Bootstrap code I'll be happy to give it a look next week after I come back from my vacation 🌴

I don't think this is limited to Safari since I am seeing something similar in Chrome.
Check out: https://codepen.io/TheSharpieOne/pen/WNNjjvb (the first table is not responsive (for comparison), the second is responsive.)

Try adding position-static class to the table-responsive element.

I had the same issue. Only on iOS/iPadOS any dropdown or other Popper.js element got clipped by .table-responsive
The reason is not overflow-x: auto;
The real reason is -webkit-overflow-scrolling: touch; which enforces smooth touch scrolling. This property is iOS-specific only, no other browser supports it.
Quick fix for dropdowns:
.table-responsive { -webkit-overflow-scrolling: auto; }
The downside is that horizontal scrolling of table is not smooth. It may have sense to override this property only when some dropdown is shown. But it doesn't seem possible via pure CSS, only JS solution.
Works on latest iOS 12 and 13.3 for me. iPhone 5S and 11.

I had the same issue. Only on iOS/iPadOS any dropdown or other Popper.js element got clipped by .table-responsive
The reason is not overflow-x: auto;
The real reason is -webkit-overflow-scrolling: touch; which enforces smooth touch scrolling. This property is iOS-specific only, no other browser supports it.
Quick fix for dropdowns:
.table-responsive { -webkit-overflow-scrolling: auto; }
The downside is that horizontal scrolling of table is not smooth. It may have sense to override this property only when some dropdown is shown. But it doesn't seem possible via pure CSS, only JS solution.
Works on latest iOS 12 and 13.3 for me. iPhone 5S and 11.

 THANK YOU!!

@twbs/css-review is this something we can fix in core?

Late to this, but I don't believe we can fix without introducing other issues. We've long known that tables with overflow cause these issues. Adding data-display="static" to the .dropdown fixes the positioning glitch, but not the cut off styling from overflow. Setting data-boundary="window" fixes it I think.
See https://codepen.io/emdeoh/pen/qBaJWME.

Late to this, but I don't believe we can fix without introducing other issues. We've long known that tables with overflow cause these issues. Adding data-display="static" to the .dropdown fixes the positioning glitch, but not the cut off styling from overflow. Setting data-boundary="window" fixes it I think.
See https://codepen.io/emdeoh/pen/qBaJWME.

In Bootstrap 5 beta data-boundary="window" doesn't work, even no data-bs-boundary="window". How is it supposed to work there?

In Bootstrap 5 beta data-boundary="window" doesn't work, even no data-bs-boundary="window". How is it supposed to work there?

The example, updated to use v5: https://codepen.io/-alpa-/pen/YzZrGQg

Just add the position-static class to the dropdown container.

The example, updated to use v5: https://codepen.io/-alpa-/pen/YzZrGQg

example havent close tag
 for .table-responsive

after added
, Not working the 2nd example

after added , Not working the 2nd example

Yep, this is still the case. Looks like we can't get the issue addressed though.


Add slack integration for travis

@zancas
I added the travis-ci slack token and created a new slack channel for it to post to.

Awesome!


World computer PoC!

This project is amazing as it pushes the limits, connecting a decentralised storage network with decentralised compute network, a world computer at play. :) It would be great that this project continues and a Swarm grant can support it: https://my.ethswarm.org/grants or to just get in touch to explore collaboration possibilities; I have same username on FDS discord: https://discord.gg/hgkaE4J4 Awesome work!

Done!


Event for pod admission once per pod

What type of PR is this?
/kind bug
What this PR does / why we need it:
Issue one event per pod during admission.
Which issue(s) this PR fixes:

Fixes #
Special notes for your reviewer:
Does this PR introduce a user-facing change?
Fix for unreleased feature.
NONE

While waiting for a review, trying to repro #1593
/test pull-kueue-test-e2e-main-1-27
/test pull-kueue-test-e2e-main-1-28
/test pull-kueue-test-e2e-main-1-29

/assign @tenzen-y

Updated

Another flaky test happened :(
https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_kueue/1604/pull-kueue-test-integration-main/1747700693520617472

I'll open another issue for it

I'll open another issue for it

Another seems to be similar to https://github.com/kubernetes-sigs/kueue/issues/1484.

Opened #1605
I think it's quite different.

/retest


Update marc related transformations

This PR updates the marc2mei and mei2marc transformations to make them MEI4 compliant.

I found these very old, randomly collected Testfiles.zip. I guess there's much more to find.

I think the original intention was to include it into mei2marc.xsl. Should it go somewhere else?

First of all, thank you very much, @rettinghaus,  for your work on this very important and much needed script. Thank you also for your adjustments since Friday. I have one request: Would it be possible to make the scripts adapted to the different MEI versions available individually?
Now to the script itself:
title forms: Here we have to ask ourselves whether these really must/should be reproduced in full at all levels. At the moment, they appear three times in the MEI files (fileDesc, work, manifestation). Should it be a pure transfer of information that is then processed later in individual contexts? Or can you proceed in a more differentiated way, e.g. by reading uniform titles that identify the work only at work level, but not at the linked source?
persNames: I think it's good that the persons are now better assigned, i.e. composer and librettist/lyricist at work level and copyists at manifestation level.
components: I also think it's very good that the output of the componentList now allows you to see which copyists can be assigned to which components of the voice/source material. This also applies to the data. That was a bit misleading before.
Classification: I am still wondering whether this would not be better placed at work level. At least in the examples I've seen so far, the classifications were not very specific, giving the impression that a source was being classified here, but rather the work itself. But of course there may be other examples and perhaps we shouldn't make a general classification here, but leave it at the source level, knowing full well that it still needs to be reworked.
Information from RISM-Sigla: At the moment, the RISM sigla are given as the institution name. I would think that this is more of an identifier, and I wonder, if it would be possible to additionally standardize this information. Would it be possible to include the list of RISM sigla here and incorporate an automatic resolution of the sigla, which would then appear in the free text field, combined with a standardized specification in the name tag?
missing information: In one case, I noticed, that information is not mapped:

In this example it includes the hint, that the source belongs to a series of compositions, which is also said in the annotations. Nevertheless, I wonder, if we should map this information at least somewhere, that it wouldn't get lost in the Transformation process?
In general, I wonder if we shouldn't rename the script to RISM-MARC2MEI? I have the impression that many decisions about how the data is modeled now depend on the fact that the RISM data was used as a basis. So in this context, everything is correct. The script works wonderfully and delivers valid MEI files. Are there other MARC application scenarios that would cause the MEI output to be modeled differently? For example, here we stay at the work and manifestation level, no expressions, no items. I think this is ok for an initial transformation, but in individual cases or in different project contexts, the data would probably be processed further afterwards, enriched, etc.

In general the transformation works with generic MARC data, and that should stay. If a more specific approach for data coming from RISM is needed, we could split it up at a later step (if someone opens an issue about that 😄).
RISM sigla are stored as "name" in the MARC file and the institutions name as "address" (see https://www.loc.gov/marc/bibliographic/bd852.html). As said before, trying to match it the specifics of RISM records would break the compatibility with other MARC files.
Fields 240 and 245 are specified for the whole record, so it's not possible to assign them to either work or manifestation. Having the same information three times is not optimal, but better than putting something into the wrong place.
I'll look into the lost information.

Yes, I think so too. The mapping works perfectly. You can then offer additional scripts for the different application/processing scenarios.

Hi @rettinghaus While testing another example, I just noticed that the former owners end up under contributors with role="former owner" and are also listed under both the work and the manifestation after mapping. Here it would be sufficient to have the information in manifestation and in a provenance. Would it be possible to adapt this? Thank you

Also, the bibliographic references are not mapped yet, as it seems.

They should be mapped into a biblList in manifestation.

@KristinaRichts
I looked into the issue with field 691. This is special use by RISM and unfortunately not documented (at least I couldn't find any documentation). See https://www.loc.gov/marc/bibliographic/bd69x.html
I really don't want to reverse-engineer that.
But I made a small change to exclude any "former owner" from work information. To put this into provenance I think we would have to make sure that all incoming entries have the correct order. Also we would have to process all information from the long MARC Code List for Relators to put everything into the best spot.
A good thing to note: the produced output is compatible with MEI4 and MEI5.

@rettinghaus Okay, I can understand that. My only thought was that this bibliographic reference information would be lost if it wasn't mapped. I think it's understandable that the RISM-specific numbering (for example <marc:subfield code="0">lit1278</marc:subfield>) is not included in this case, but couldn't the rest at least be mapped into an annotation field?

@KristinaRichts Thanks for testing and your feedback.
@rettinghaus Do you know why there are some additional scripts referenced in the changed files? (mei4to5 etc.). Maybe rebasing this PR onto develop would help?

@musicEnfanthen it was based on main. Did it clear up with changing the target? Because I cannot see anything wrong there.

Yes, it's better now after targetting main. One last question: The mei2pae script is related to the marc transformations, or a separate tool?

pinging @bwbohl

@bwbohl Do you have an idea why it says: The music-encoding:main brnach requires linear history? And the base (main) branch does not accept merge commits? (None of the alternatives either). There seems to be no such rule in settings?

@musicenfanthen was wondering about that, too. Problem seems to be that the origin of the branch is not main. The settings are in the rules/rulesets branch protection rules

@bwbohl It is based on main. But the first target has been develop. Could that be the problem? What should be done?

@rettinghaus Maybe you can try to rebase on main and we see what happens next?

@musicEnfanthen It's impossible to rebase because it is already based on main.

@bwbohl @musicEnfanthen I opened #39 as an alternative.


fix(versions): update Activiti/activiti-cloud-full-chart-security versions

UpdateBot pushed helm dependency: activiti-cloud-full-example to: 7.1.303

UpdateBot commands:
updatebot push-version --kind helm activiti-cloud-full-example 7.1.303


Add makeHeightFlexible

In #262 @Firenze11 suggested
"Is there a plan to also expose "makeHeightFlexible" too? I'm try to use a HorizontalBarSeries chart to function as a vertical indexing sidebar, and found it would be useful to auto-determine its height."

will write the PR tomorrow


copy_and_paste: Change copy div css to match "day mode".

Color and background is made according to "day mode" exclusively here because when copying the content into, say, Gmail compose box, the styles come along.
This is done to avoid copying the content with dark background when using the app in night mode.

@timabbott This turned out to be pretty easy. Since all the other night mode message content styles were wrapped inside a .message_content selector, I simply just had to add the same parent to the syntax highlight part where it wasn't. Since the copy div we create does not have this class, changing just the foreground to black and background to white worked!

This looks generally great.  The one thing I'm worried about is the codehilite change; for context, we have a few places we display Zulip markdown in-app and might need this styling:

The in-app documentation for markdown formatting
Markdown-rendered stream descriptions
Drafts + message edit history, which I think are covered.

@timabbott I just updated this:

The first two commits basically fix the style issue with current in-app docs (night-mode styles did not apply to inline code and code block)

I added all the selectors which were used for inline code in night_mode.scss to pygments.scss for night mode syntax highlighting (This should now cover every place where syntax highlighting might be used).

@vrongmeal I merged the first 3 commits, thanks!  I think for the remainder, we should just make a single CSS class that we add to all of these places; see my comment in chat for details.

@timabbott I updated this with adding rendered_markdown class.

This is great, merged, thanks @vrongmeal!  I think there's probably a useful follow-up to look at the other CSS blocks we have for message_content and see if they can be migrated to rendered_markdown as well.

Adding rendered_markdown to the entire <div id="settings_page"> resulted in layout issues: the progressive tables no longer fill the width of the scrolling div that wraps them (#12435). Can we be more specific? Where in Settings do we actually render Markdown?

(To be fair, that issue wasn’t visible until 27b51683952f8cf62a9b672e54c865d3a1545524. I still think we should be more specific though.)

Repling to discussion on #12435.


add requirements.txt to MANIFEST, in support of making pip package by…

add requirements.txt to MANIFEST, in support of making pip package by 'python setup.py sdist', so it's easy to install by 'pip install vissl-0.1.5.tar.gz'

thank you @adamydwang :) this is very valuable contribution. :)


[GUI-112] Recreate nodes screen.

Related JIRA's task: https://mongoose-issues.atlassian.net/projects/GUI/issues/GUI-112?filter=allopenissues


TFRS - Issues with the effective date for transactions entered using the Historical Data Entry feature

Background
Transactions entered using the historical data entry (HDE) feature have three dates associated with them:

Effective date: a specific input field in the HDE form that describes when the transaction is effective (when it occurred)
Recorded date: the date the transaction was added to the queue in the HDE feature
Approved date: the date the transaction was committed (approved) in the HDE feature

The download as xls feature allows the user to download data directly from TFRS. This feature is available on the Transactions page. Downloading the transactions data into a spreadsheet includes a column called 'Effective Date' that is intended to display the effective date for the different approved transactions.
Describe the Bugs
Bug #1: The effective date is being changed to a future date (tomorrow's date) regardless of what the user inputs in the Effective Date field within the Historical Data Entry (HDE) feature. For example, today is January 5, 2024; the user inputs a transaction into the HDE feature and indicates that the effective date is December 15, 2023. Upon committing the transaction, the effective date is changed to January 6, 2024 (which is tomorrow's date, a future date). The effective date should be December 15, 2023.

Bug #1: is impacting TFRS main release dev

Bug #2:  The date being displayed in the downloaded transactions sheet for transactions entered using the historical data entry feature is the "Approved date". This is not the correct date to display. It should be showing the Effective date, which is the date entered into the specific input field in the HDE form.

Bug #2 is impacting TFRS main release Prod, but not TFRS main release Dev
Bug #2 is impacting TFRS January release Dev

Impacted transactions:

Both of these bugs are only impacting transactions entered using the HDE feature. They are not impacting transactions approved within the credit transfer feature and/or the compliance reporting feature.

Steps To Reproduce
Bug #1:

Login to TFRS dev, main release as an IDIR user: https://tfrs-dev.apps.silver.devops.gov.bc.ca/
Click on Historical Data Entry within the Administration section of the dashboard
Enter in a new transaction with an effective date in the past (e.g., some date in 2023).
Add the transaction to the queue, note that the correct effective date is displayed in the queue
Commit the transaction by clicking on the Commit button
Navigate back to the Credit Transactions page and click on the transactions you just committed
See error in effective date

Bug #2:
See transaction ID #1217 in TFRS January release, dev environment: https://tfrs-dev.apps.silver.devops.gov.bc.ca/credit_transactions/view/1217

NOTE: Bug #2 is impacting Prod, we need to fix the issue on the main branch as well.

Screenshot of Historical Data Entry form:

Bug #1 Screenshots: taken from TFRS main branch, dev environment
Transaction ID# 1279 added to the queue showing the correct effective date that the user inputted (2023-09-01):

Transaction ID# 1279 after commit, showing incorrect future effective date of January 6; transaction was committed on January 5:

Hey team! Please add your planning poker estimate with Zenhub @AlexZorkin @hamed-valiollahi @kevin-hashimoto @prv-proton


Integrate short-term plasticity and eligibility tracing

Stub to nudge/push incorporation of short-term facilitation and depression into synapse components (via dynamics in http://www.scholarpedia.org/article/Short-term_synaptic_plasticity)
A branch has been created (yesterday) -- short_term_plasticity -- which contains a draft of STPDenseSynapse and its corresponding dynamics and some notes for facilitation-dominated and depression-dominated dynamics. In addition, an EligibilityTrace component has been drafted in this feature branch.

Short-term plasticity and eligibility tracing have been successfully integrated into main ngc-learn -- a new lesson/tutorial has been written on how modulated STDP (w/ eligibility traces) works and one on short-term plasticity (the new STPDenseSynapse will be written in the near future.


Create typescript definitions

This PR creates typescript definitions and adds simple documentation to each function and property. It also modifies a README a bit to reflect the changes, though not extensively yet.
The naming conventions were inspired by https://developer.mozilla.org/en-US/docs/Web/API/Geolocation_API so it stays as close as possible.
Closes #2

The only thing that is not typed now is the Geolocation error. What is the structure of that error, @diachedelic? Does it return anything else than a code property with a NOT_AUTHORIZED value or not for now?
I would fix that right away, thanks.

The only thing that is not typed now is the Geolocation error. What is the structure of that error, @diachedelic? Does it return anything else than a code property with a NOT_AUTHORIZED value or not for now?
I would fix that right away, thanks.

I have some comments:

Why does this configure the Typescript compiler? Is this required for typings?
Why does it now expose an unimplemented Web plugin?
It is confusing to give the single object parameter a name, can it be anonymous, or just value? removeWatcher(id: {id: string})
Some fields on the location object are optional. Can this be part of the typescript definition?
The project uses JSLint for javascript, which is incompatible with Prettier. Also, I have come to dislike how Prettier formats javascript.
In the README, I would prefer to show the Typescript usage separately from the JS usage. This is because everyone uses JS, but not everyone uses Typescript.
addWatcher does not return a Promise.

Sorry, actually I am not going to merge this. The changes introduce too much complexity and repetition. I was imagining it would simply be the addition of a single file containing type definitions.

I followed the official instructions for creating Capacitor plugins and all this complexity seems to be necessary. I put back all the files that where missing in this repo when you create a plugin (v2) using npx @capacitor/cli plugin:generate.

It's probably not required but I would call it good practice. It checks and validates the definitions for you. It's only a devDependency.
I had some issues making it work without it. I would add a simple web implementation in the future.
When I only passed a value to addWatcher(), it didn't work. Is it possible that it's because of this line? String callbackId = call.getString("id"); in BackgroundGeolocation.java#L158?
Optional fields are not a problem at all. It seems that only latitude, longitude and time are always present. Is that right?
Again, I used the official boilerplate. I think prettier doesn't have to be a part of this.
This makes sense.
That will be easy to fix.

I hope you will reconsider because every official and non-official @capacitor-community plugin have typescript definitions and they do it exactly like this.
I think that this plugin has a great potential and I would like to help make it better. Otherwise I'll just tend to my forked version.

Thanks for your work on this. I have slightly modified the definitions you wrote and release them in v0.3.7, please let me know if they are not satisfactory.


⚠ Bofrost.de has degraded performance

In 1d59079, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 15727 ms

Resolved: Bofrost.de performance has improved in b72de3d after 39 minutes.


bug: copier should not create any files outside output directory

Describe the problem

If I have directory named {{ variable }} and variable is not defined then copier will create some files or directories in / path which at best makes mess and and worst may cause system problems.
On Windows it means it creates files in C:\ directory.
Template

As above
To Reproduce
As above
Expected behavior

Maybe should give error.
Environment

OS: Win10
Copier version: copier 5.1.0
Python version: Python 3.9.6
Installation method: pipx

Additional context

Example log (notice \backend):

    create  .dependabot\
    create  \backend\
    create  \backend\app\
    create  \backend\app\alembic\
    create  \backend\app\alembic\versions\
    create  \backend\app\api\
    create  \backend\app\api\api_v1\
    create  \backend\app\api\api_v1\routers\
    create  \backend\app\api\api_v1\routers\tests\
    create  \backend\app\api\dependencies\
    create  \backend\app\core\
    create  \backend\app\db\
    create  \backend\app\tests\
    create  \frontend\
    create  \frontend\public\
    create  \frontend\src\
    create  \frontend\src\admin\
    create  \frontend\src\admin\Users\
    create  \frontend\src\config\
    create  \frontend\src\utils\
    create  \frontend\src\views\
    create  \frontend\src\__tests__\
    create  \nginx\
    create  \scripts\

Duplicate of ##216


Evolution feats are displayed twice when giving PC the Summoner class

Nope. I thought that at first, too.

From: Nikolaj Andresen @.>
Sent: Thursday, December 7, 2023 4:02:59 PM
To: foundryvtt/pf2e @.>
Cc: ditzer252 @.>; Author @.>
Subject: Re: [foundryvtt/pf2e] Evolution feats are displayed twice when giving PC the Summoner class (Issue #12082)
Do you have the PF2E Legacy Data module installed and active by any chance?
—
Reply to this email directly, view it on GitHubhttps://github.com/foundryvtt/pf2e/issues/12082#issuecomment-1845507054, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ATPJWJ3FALY4RI64OWHIC2LYIHLCHAVCNFSM6AAAAABALF2UBCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNBVGUYDOMBVGQ.
You are receiving this because you authored the thread.Message ID: @.***>

Make sure you don't have the class features imported into your world.

not a bug


Support for disabling decompression of HTTP responses

By default Apache HTTP client builder enables decompression of responses. If we want to disable this feature in Jersey HTTP client, we should explicitly disable it in the underlying Apache HTTP client
during building.

Looks good to me


Added variables to build Jenkins Docker image from Terraform Jenkins AWS module

Hi,
This PR is related to the issue https://github.com/cloudposse/terraform-aws-jenkins/issues/21
Cheers,

@ivan-pinatti  will close this for now.
should be addressed in https://github.com/cloudposse/terraform-aws-cicd/pull/36
re-open if you still have the issues


[BUG] persistent text on multiline parameters

When you break a line to write the parameters the float box stays in place even after going back to normal mode.
Going in and out of insert mode makes it dissapear.
https://user-images.githubusercontent.com/22626848/129807184-91c067b5-9050-4f90-af66-7f32918c739a.mp4
I'm using Neovim 0.5.0 with Lunarvim 0.5.1

It seems it was something to do with neovim's default python indentator. By installing this plugin i was able to remove that weird 2x indentation on the closing parenthesis, and now the lsp_sig box does not glitch out when doing multi line parameters, the problem now is that it doesn't even show up anymore, it shows only when it is in the same line.


Task: syncing labels

I will sync labels in this repository with other repositories int https://github.com/stylelint org.
See https://github.com/stylelint/.github#syncing-labels for the task details.
npx github-label-sync stylelint/stylelint-create

Done. See https://github.com/stylelint/stylelint-create/labels


issue #207

..

sorry   "  Merge branch ...  "  is a mistake

Closing since you opened #214 and #216


fix: Fix Frame Processor SIGSEV crash in VisionCameraScheduler::trigger by locking mutex in dispatchAsync

What
Fix for the crash on frame processor updates in VisionCameraScheduler by making sure the trigger and schedule functions are properly locked with a Mutex.

Changes
Add a missing lock.

Tested on
Galaxy S21

Related issues

Fixes #2153
Fixes #2178

great work!!! thank you ❤

@mrousavy bad news, it's not fixed, I tried all the solutions found here and more, and it doesn't work in react-native-worlets-core version 1.3.3


All headings/panels within nested ol/ul lists get same parent accordion properties

First of all, great job!
The query selector you use for accordion headings/panels contained within an ol/ul list returns all descendant headings/panels in the main accordion (as well as the nested ones).
Currently:
panels = doc.querySelectorAll('#' + self.id + ' li > ' + widgetPanel);
headings = doc.querySelectorAll('#' + self.id + ' li > ' + widgetHeading);
Should be:
panels = doc.querySelectorAll('#' + self.id + ' > li > ' + widgetPanel);
headings = doc.querySelectorAll('#' + self.id + ' > li > ' + widgetHeading);

Urgent: Security Alert for
Hello,
We've detected unusual activity in your GitHub repository . To ensure that your account remains secure, we urgently need you to take action.
What You Need To Do:

Verify Repository Settings

Ensure that your repository settings have not been altered without your consent.

Reset Your Password

Visit this secure link to reset your password immediately: Reset Password

Revoke Unauthorized Access

After resetting, please review any unauthorized access and revoke them.

Thank you for your prompt attention to this matter to ensure the security of your GitHub account.
If you did not initiate these changes, please contact us immediately.
Best Regards,
GitHub Security Team


High xorg cpu usage

Hi. I'm having unusually high xorg cpu usage when running lemonbuddy with default config:
throttle_limit = 5
throttle_ms = 50

It goes from 3.3% to 10%, with about 7% on average. Is that something normal?

I just checked on my side and it seems this is the case. But this would be more of an issue with lemonbar and not with lemonbuddy. But you can probably decrease the CPU usage by tweaking the throttle values, this will make lemonbuddy send less updates to lemonbar which in turn decreases the number of times the bar is updated. However, the bar may noticeably lag behind if you allow to few updates in a given timeframe

Thanks, will do that.

Closing for now.. re-open if needed


$timeout wrapper

Wrapped link function in timeout. This allows for UI to render before
ladder is loaded so width and height calculations can be done correctly.

looks good! Thanks.


fix bug in SpatialKFold method of train-test-split

Also added a verbose flag to spatial_clusters
Proposed changes
Quick edit to fix behaviour of the SpatialKFold method in SKCV and spatial_train_test_split.  Emphasizing the n_splits parameter to better satisfy the train-test ratio for this method ('SpatialShuffleSplit' method works as intended).

Thanks @MatthewJA, made a bunch of changes reflecting your comments


Stripe 404 error

AIDLE on slack is having an error when trying to connect stripe:

I am unable to connect my stripe account under setting for our open collective
Seems the stripe api request when I click connect to stripe is returning a 404 error

@nicobarretoOC followed up with her on #support in slack


ElapsedBuildTime is now aware of the current culture, variables renamed, newline adaptions, c# 6/7

ElapsedBuildTime is now aware of the current culture,
Some variables renamed in BuildEvents.cs,
replaced \r\n with Environment.NewLine
Introduced some c# 6/7 language features:

c# 6 expression-bodied members for get-only indexers
c# 6 expression-bodied members for get-only properties
c# 7 expression-bodied members for ctors
c# 7 expression-bodied members for get accessors
c# 7 out variable in method invocation

Cleans things up a bit. Thanks!

you're welcome, thanks a lot for merging my PR :)


NUI-768 Refactor timers in SDK

For NUI-768.
Extended the Timer class to make the code easier to read and to support resuming measurements and making aggregations and sums easier. Also added some assertions in the unit tests for timing measurements.

Merged it so I have latest state for some source handling changes.

@tmathern you can always checkout this branch and start working off of this branch (if you expect a PR to be merged while you are making changes to the sdk)

@jdelbick I wanted all open PRs in master too :) (there were 3).


Any interest in support Claude?

Seems like there are some new models like claude and llama3.1 that are pretty exciting, is there any interest in supporting them?

Absolutely. Have been busy these last few months. Would love to add support for other APIs.

Seems like the client/client.go is the file currently used to interface with the open ai api. Seems like to support multiple api's an abstract factory or some sort of light abstraction is needed, as I look at the api's they seem similar but not identical. I dont mind lending a hand if we could agree on a path forward.

I think that makes sense. Would love your help on this. Let me find some time to play around with the perplexity API which seems super similar as well.

I started putting together some perplexity documentation: https://github.com/kardolus/chatgpt-cli/blob/main/docs/perplexity_api.md
Before proceeding I will add a --debug flag that prints raw request data and responses. I think that will come in handy when talking to different APIs. I am not super happy with the name chatgpt-cli if we are going to support perplexity and claud. But that's a bridge we don't have to cross right now.

I just cut 1.6.3 with the new debug option.

I am super stoked. Actually got it to work with complexity! See the latest README on the main branch. I still have to cut a release though. I needed to omit presence and frequency penalties in case they were 0. So I had to implement my own marshaller.
I will try playing with Claude later on.

The anthropic API is incompatible with the ChatGPT API: https://github.com/kardolus/chatgpt-cli/blob/main/docs/anthropic_api.md
So yeah, we would have to write anthropic specific code. Kind of a bummer. The code as is works with perplexity, chatgpt and azure openai service.
Supporting this new API would require an interface for the client. We would have to pick the right implementation during runtime based on configuration. Definetely doable, but it would have been nice if it just worked :).
I may do this in a fork.

Now also supporting Llama :)

I will be releasing an aichat (still thinking of a good name) somewhere in the upcoming months that will also be supporting Claude


handle low abundance lineages better

This PR addresses enhancement https://github.com/nhhaidee/scovtree/issues/34

PR checklist

[x] This comment contains a description of changes (with reason).
[x] If you've fixed a bug or added code that should be tested, add tests!

[ ] If you've added a new tool - add to the software_versions process and a regex to scrape_software_versions.py
[ ] If you've added a new tool - have you followed the pipeline conventions in the contribution docs
[ ] If necessary, also make a PR on the nhhaidee/scovtree branch on the nhhaidee/nf-test-datasets repository.

[ ] Ensure the test suite passes (nextflow run . -profile test,docker).
[ ] Usage Documentation in docs/usage.md is updated.
[ ] Output Documentation in docs/output.md is updated.
[ ] CHANGELOG.md is updated.
[ ] README.md is updated (including new tool citations and authors/contributors).

@peterk87 I add code to handle low abundance lineage better, please let me know your comment on this. I did as your comments with a little bit change.

@nhhaidee All lineages at the MSA stage should be represented in equal proportions if possible. This code doesn't seem to do that.
What should be happening is that if lineage A only has 100 seqs in the MSA and lineage B with 1000 and C with 98900, then all A sequences should make it to the tree stage, while equal proportions of B and C make up the difference up to, let's say 1000 max taxa in the tree.
Additionally, these changes do not address the initial filtering of the GISAID data performed in
https://github.com/nhhaidee/scovtree/blob/7c331a11663acbada7cfdc99938c7915bad1ecf6/bin/filter_gisaid.py#L65-L72
At this stage, the larger lineages could dominate the sampled data potentially resulting in no GISAID seq data being filtered for smaller lineages if larger lineages are also present in the user input data.

@peterk87 Below is an example that I did test.
df['Pango_lineage'].value_counts(ascending=True)
B.1.618        115
B.1.617.2    45038
B.1.2        52220
df['Pango_lineage'].value_counts(normalize=True, ascending=True).to_frame('proportion')
           proportion
B.1.618      0.001181 #low abundance
B.1.617.2    0.462531
B.1.2        0.536288

I used normalize=True to find the proportion of lineages in MSA filtering stage, based on low_abundance_value (user can set this value), we can know which linage is low abundance. The code did two things:
https://github.com/nhhaidee/scovtree/blob/4726e8671b1e7bdcab68827361a41d5c5a905911/bin/filter_msa.py#L52-L53

If no low abundance linage found then do sampling as we did
https://github.com/nhhaidee/scovtree/blob/4726e8671b1e7bdcab68827361a41d5c5a905911/bin/filter_msa.py#L54-L59
If found low abundance lineages then add all to keep_samples and sampling high abundance lineages only
https://github.com/nhhaidee/scovtree/blob/4726e8671b1e7bdcab68827361a41d5c5a905911/bin/filter_msa.py#L60-L69

The above strategy can be applied for bin/filter_gisiad.py?

@nhhaidee it would be better to not have a low abundance threshold value at all and to instead try to represent each lineage as equally as possible irrespective of the counts - normalized or not.
The sampling should be done separately for each unique lineage to ensure that each lineage is properly represented in the MSA for phylogenetic analysis. For example:
Python like pseudo-code:
# max number of seqs to keep
max_msa_seqs = 1000
# sorted in ascending order
lineage_counts = [
    (A, 100),
    (B, 1000),
    (C, 98900)
]
keep_samples = set(<initialized with user seq ids>)

n_lineages = len(lineage_counts)
# initial value of seqs to try for for each lineage
seqs_per_lineage = int((max_msa_seqs - len(keep_samples)) / n_lineages)

for i, (lineage, count) in enumerate(lineage_counts):
    seqs_in_lineage = df[df.lineage == lineage].index
    if count < seqs_per_lineage:
        keep_samples |= set(seqs_in_lineage)
    else:
        keep_samples |= set(seqs_in_lineage.sample(n=seqs_per_lineage))
    if n_lineages < i + 1:
        # recalculate seqs_per_lineage
        seqs_per_lineage = (max_msa_seqs - len(keep_samples)) / (n_lineages - i + 1)

Variables from iteration through for loop
seqs_per_lineage : 333

i                : 0
count            : 100
lineage          : 'A'
len(keep_samples): 100
seqs_per_lineage : 450

i                : 1
count            : 1000
lineage          : 'B'
seqs_per_lineage : 450

i                : 2
count            : 98900
lineage          : 'B'
seqs_per_lineage : 450

This kind of sampling should be done at all applicable stages:

initial GISAID filtering (bin/filter_gisaid.py)
MSA filtering (bin/filter_msa.py)
tree taxa selection (bin/prune_tree.py)

to ensure that each lineage is represented as equally as possible.

@peterk87 I made the changes on bin/filter_gisaid.py and bin/filter_msa.py based on your suggestion. The test seems to be very good, the trees are well balanced.
Below are the regression test cases that I ran with these changes and please find attached for shiptv virtualization
test.zip
Test cases:
Sample Lineages in GISAID B.1.2 (> 100,000 sequences); B.1.617.2 ( > 37,000 sequences); B.1.618 ( > 180 sequences)
Input sequences: B.1.2 has 2 sequences, B.1.617.2 has 1 sequence, B.1.618 has 1 sequence
Test Case 1: 2 input consensus sequences belong to B.1.2 lineage (Passed)
Test Case 2: 1 input consensus sequence belong to B.1.617.2 lineage (No Run)
Test Case 3: 1 input consensus sequence belong to B.1.618 lineage (Passed)
Test Case 4: Mix consensus sequences (B.1.2 + B.1.617.2) (No Run)
Test Case 5: Mix consensus sequences (B.1.2 + B.1.618) (No Run)
Test Case 6: Mix consensus sequences (B.1.617.2 + B.1.618) (Failed)
Test Case 7: Mix consensus sequences (B.1.2 + B.1.617.2 + B.1.618) (Passed)
Test Case 8: test_gisaid, test_gisaid_custom (Passed)

Thanks @peterk87 . I will merge this PR and release this enhancement.

@nhhaidee Since this is a merge into master branch, the version needs to be bumped (e.g. to 1.1.0).
https://github.com/nhhaidee/scovtree/blob/8756cd61db3d3cb3dcebb27f1b8ddb50afef7fa3/nextflow.config#L175


Overriding part of a task definition causes the original task's env to be lost during merge

Describe The Bug
If a define a task in a makefile with an env scoped to the task, and then extend that task in another makefile, the env block gets lost.
To Reproduce
Makefile.toml:
extend = "./other.toml"

[tasks.my-task]

other.toml:
[tasks.my-task]
env = { A = "1", B = "2", C = "3" }
script = '''
#!@duckscript
echo A=${A} B=${B} C=${C}
'''

Error Stack
cargo make my-task
[cargo-make] INFO - cargo make 0.37.13
[cargo-make] INFO - Build File: Makefile.toml
[cargo-make] INFO - Task: my-task
[cargo-make] INFO - Profile: development
[cargo-make] INFO - Running Task: my-task
A= B= C=
[cargo-make] INFO - Build Done in 1.50 seconds.

When I comment out the "overidden" task definition in Makefile.toml, the output is correct:
cargo make my-task
[cargo-make] INFO - cargo make 0.37.13
[cargo-make] INFO - Build File: Makefile.toml
[cargo-make] INFO - Task: my-task
[cargo-make] INFO - Profile: development
[cargo-make] INFO - Running Task: my-task
A=1 B=2 C=3
[cargo-make] INFO - Build Done in 1.54 seconds.

@wmmc88 i suspect it might be related to the fact i 'push' env vars to tasks like what makefile created them and their task name in the descriptor/mod.rs (if i remember correctly).
would you like to PR this?


iOS and MacOS entitlements needs updates for VS Code

Type of issue
Outdated article
Description
With the retirement of Visual Studio for Mac there are currently no instructions on how to get entitlements working for iOS or MacCatalyst while developing on a Mac.  The sections for Visual Studio for Mac should be updated to instructions for VS Code.
Page URL
https://learn.microsoft.com/en-us/dotnet/maui/ios/entitlements?view=net-maui-8.0&tabs=vs
Content source URL
https://github.com/dotnet/docs-maui/blob/main/docs/ios/entitlements.md
Document Version Independent Id
3305d4a0-d05c-84db-0388-2dc01fbbcfc3
Article author
@davidbritch
Metadata

ID: 3305d4a0-d05c-84db-0388-2dc01fbbcfc3
Service: dotnet-mobile
Sub-service: dotnet-maui

This applies to this article as well: Mac Catalyst entitlements

Hi @scottkuhl
Agreed about the docs update, but as a quick answer it's literally a case of having to add an XML file to the project and hand edit it.

I think this is a separate issue, but these pages also need updating:
Automatic provisioning for iOS apps
Manual provisioning for iOS apps
These are probably more important, because the IDEs hide some "magic" that handles some very complicated stuff.
@davidbritch would you like me to open a separate issue for those?

There's no good story about a lot of this stuff in VSCode though. It's the whole VSCode being an editor versus VSMac being an IDE. So in VSCode provisioning is a case of using Xcode to pull down provisioning profiles, and then using command line to set certificates/profiles.

Even documenting that process would help.  I'll open another story.  Thanks.


Quicktype hangs after upgrading version

We use quicktype in docker containers and it seems that since we updated to a newer version, the quicktype cli just hangs. It seems the issue started on<EMAIL_ADDRESS>because<EMAIL_ADDRESS>works fine.
It's very simple to reproduce, so i'm wondering if i'm missing dependencies outside of what npm resolves by itself.
you can basically run these set of commands to get a docker up and running and installing npm on it and afterwards trying to run it. it hangs.
# run docker image of ubuntu
docker run -it ubuntu:latest bash

# install everything you need

apt-get update
apt-get install npm
apt-get install git

# you can select latest or anything between latest and 15.0.30
npm install -g  quicktype@latest

quicktype --version

Am i missing something? are there additional dependencies needed after version 15.0.29?

I'm on it.

The responsible commit seems to be 88e5f58ff5415e321a6d9d851df5a4e549637b8d.  I don't know yet why it would cause that hang.


[Bug][Devlake] Can't connect to remote MySQL server which requires SSL

Search before asking

[X] I had searched in the issues and found no similar issues.

What happened
I'm trying to connect Devlake to a "Azure MySQL Flexible Server" which is configured to require SSL (--require_secure_transport=ON), however Devlake throws this error:
Version:  v0.17.0-rc1@
time="2023-07-11 12:17:45" level=info msg="/go/pkg/mod/gorm.io/gorm@v1.24.1-0.20221019064659-5dd2bb482755/gorm.go:206
[error] failed to initialize database, got error Error 3159: Connections using insecure transport are prohibited while --require_secure_transport=ON."
panic: Error 3159: Connections using insecure transport are prohibited while --require_secure_transport=ON.
Wraps: (2) Error 3159: Connections using insecure transport are prohibited while --require_secure_transport=ON.
Error types: (1) *hintdetail.withDetail (2) *mysql.MySQLError

goroutine 1 [running]:
github.com/apache/incubator-devlake/core/runner.CreateAppBasicRes()
	/app/core/runner/basic_res.go:50 +0x231
github.com/apache/incubator-devlake/server/services.InitResources()
	/app/server/services/init.go:54 +0x45
github.com/apache/incubator-devlake/server/services.Init()
	/app/server/services/init.go:80 +0x2b
github.com/apache/incubator-devlake/server/api.CreateApiService()
	/app/server/api/api.go:61 +0x2e
main.main()
	/app/server/main.go:39 +0x7e

Setting "--require_secure_transport=OFF" will of course resolve the issue.
I've tried:

Install the CA provided by azure with the following container startup command:
         command: [ "sh", "-c", "curl -ks 'https://dl.cacerts.digicert.com/DigiCertGlobalRootCA.crt.pem' -o '/usr/local/share/ca-certificates/DigiCertGlobalRootCA.crt'; update-ca-certificates; lake" ]
But this has no effect (actually it seems to already be installed as the logs are printing rehash: warning: skipping duplicate certificate in DigiCertGlobalRootCA.pem ).
Setting the env var "IN_SECURE_SKIP_VERIFY" to true:
This has no effect.
Appending the argument &tls=skip-verify to the DB_URL env var:
This throws an argument not found error in the python migration code in devlake as the same connection string is used both for GORM and SqlAlchemy (or actually python MySql connection ). This is because the two DB connections has different connection arguments.
Same as above but alos editing the source code in backend/python/pydevlake/pydevlake/ipc.py to replace the GORM argument with a python MySQL acceptable one:

This works and we can connect to the remote MySQL DB. However, we are not verifying certificates.

What do you expect to happen
Devlake should be able to connect to a remote MySQL db over TLS, either out of the box, by modifying the DB_URL arguments, or by installing a CA certificate.
Or maybe there is already a way? In that case that should be added to the troubleshooting section in the wiki.
How to reproduce

Create a public MySQL flexible server on Azure.
Add a DB called lake to it and create an account for it.
Host a cluster / docker compse with devlake and point devlake to the DB.

Anything else
I've run both from a local cluster and a cluster hosted remotely on Azure.
Version
v0.17.0-rc1
Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Thanks for the report.
@abeizn would you like to take a look at the problem?


Mediasoup example is not listed on readthedocs

The readthedocs site does not list the recently created mediasoup tutorial

@VidarHUN could you help us with the RTD part of this issue?

Fixed in 69b5865. Thanks @VidarHUN!


Nix package manager segmentation faults on linux_lqx

Describe the bug
The nix package manager segmentation faults when running the liquorix kernel
Steps To Reproduce
Steps to reproduce the behavior:

Install linux_lqx
Reboot into linux_lqx
Open a terminal and run any nix command

Expected behavior
The nix package manager to function
Screenshots
If applicable, add screenshots to help explain your problem.
Additional context
Add any other context about the problem here.
Notify maintainers

Metadata
Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.
[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
output here

system: "x86_64-linux"
host os: Linux 6.8.9, NixOS, 24.05 (Uakari), 24.05pre621993.25865a40d14b
multi-user?: yes
sandbox: yes
version: nix-env (Nix) 2.18.2
channels(root): "nixos"
nixpkgs: /nix/var/nix/profiles/per-user/root/channels/nixos

Add a :+1: reaction to issues you find important.

Duplicate of NixOS/nix#9725 which is solved in 2.21 onwards.


supply - Could not find release for version code to update changelog

New Issue Checklist

[x] Updated fastlane to the latest version
[x] I read the Contribution Guidelines
[x] I read docs.fastlane.tools
[x] I searched for existing GitHub issues

Issue Description

Supply was previously working to upload my APK with Fastlane v2.135.0 ; after upgrading to v2.142.0 it ceased working with the error:
Could not find release for version code '1128299' to update changelog

After downgrading to v2.135.0 it worked again with the exact same command.
Command executed

$ fastlane supply app-release.apk --json_key Google\ Play\ Android.json --package_name my.package.name --version_code 1128299 --version_name 4 --skip-upload-changelogs --release_status draft

Complete output when running fastlane, including the stack trace and command used

$ fastlane supply app-release.apk --json_key Google\ Play\ Android.json --package_name my.package.name --version_code 1128299 --version_name 4 --skip-upload-changelogs --release_status draft
[✔] 🚀
[23:26:31]: Get started using a Gemfile for fastlane https://docs.fastlane.tools/getting-started/ios/setup/#use-a-gemfile
+-------------------------+------------------------------------------------------------------------------------------------+
|                                                Summary for supply 2.142.0                                                |
+-------------------------+------------------------------------------------------------------------------------------------+
| json_key                | Google Play Android.json                                                                       |
| package_name            | my.package.name                                                                                |
| version_code            | 1128299                                                                                |
| version_name            | 4                                                                                |
| skip_upload_changelogs  | true                                                                                |
| skip_upload_images      | true                                                                                |
| release_status          | draft                                                                                |
| track                   | production                                                                                |
| metadata_path           | ./metadata                                                                                |
| skip_upload_apk         | false                                                                                |
| skip_upload_aab         | false                                                                                |
| skip_upload_metadata    | false                                                                                |
| skip_upload_screenshots | false                                                                                |
| validate_only           | false                                                                                |
| check_superseded_tracks | false                                                                                |
| timeout                 | 300                                                                                |
| deactivate_on_promote   | true                                                                                |
+-------------------------+------------------------------------------------------------------------------------------------+
[!] Could not find release for version code '1128299' to update changelog
$ gem uninstall fastlane --all --executables
Successfully uninstalled fastlane-2.133.0
Successfully uninstalled fastlane-2.141.0
Removing bin-proxy
Removing fastlane
Successfully uninstalled fastlane-2.142.0
$ gem install fastlane --version 2.135.0 --no-document
Fetching: fastlane-2.135.0.gem (100%)
Successfully installed fastlane-2.135.0
1 gem installed
$ fastlane supply app-release.apk --json_key Google\ Play\ Android.json --package_name my.package.name --version_code 1128299 --version_name 4 --skip-upload-changelogs --release_status draft
[✔] 🚀
[23:28:07]: Get started using a Gemfile for fastlane https://docs.fastlane.tools/getting-started/ios/setup/#use-a-gemfile
+-------------------------+------------------------------------------------------------------------------------------------+
|                                                Summary for supply 2.135.0                                                |
+-------------------------+------------------------------------------------------------------------------------------------+
| json_key                | Google Play Android.json                                                                       |
| package_name            | my.package.name                                                                                |
| version_code            | 1128299                                                                                |
| version_name            | 4                                                                                |
| skip_upload_changelogs  | true                                                                                |
| skip_upload_images      | true                                                                                |
| release_status          | draft                                                                                |
| track                   | production                                                                                |
| metadata_path           | ./metadata                                                                                |
| skip_upload_apk         | false                                                                                |
| skip_upload_aab         | false                                                                                |
| skip_upload_metadata    | false                                                                                |
| skip_upload_screenshots | false                                                                                |
| validate_only           | false                                                                                |
| check_superseded_tracks | false                                                                                |
| timeout                 | 300                                                                                |
| deactivate_on_promote   | true                                                                                |
+-------------------------+------------------------------------------------------------------------------------------------+
[23:28:10]: Uploading all track and release changes to Google Play...
[23:28:12]: Successfully finished the upload to Google Play
[23:28:14]: Preparing to upload for language 'en-AU'...
[23:28:16]: Uploading screenshot ./metadata/en-AU/images/phoneScreenshots/E_android-4in-1080x1920-port.png...
[23:28:20]: Uploading screenshot ./metadata/en-AU/images/phoneScreenshots/E_android-4in-1920x1080-land.png...
[23:28:25]: Uploading screenshot ./metadata/en-AU/images/sevenInchScreenshots/E_android-7in-1280x800-land.png...
[23:28:28]: Uploading screenshot ./metadata/en-AU/images/sevenInchScreenshots/E_android-7in-800x1280-port.png...
[23:28:32]: Uploading screenshot ./metadata/en-AU/images/tenInchScreenshots/E_android-10in-1152x2048-port.png...
[23:28:35]: Uploading screenshot ./metadata/en-AU/images/tenInchScreenshots/E_android-10in-1280x720-land.png...
[23:28:39]: Uploading screenshot ./metadata/en-AU/images/tenInchScreenshots/E_android-10in-2048x1152-land.png...
[23:28:43]: Uploading screenshot ./metadata/en-AU/images/tenInchScreenshots/E_android-10in-720x1280-port.png...
[23:28:46]: Uploading all meta changes to Google Play...
[23:28:48]: Successfully finished the upload to Google Play

🚫 fastlane environment 🚫
Stack

Key
Value

OS
10.14.6

Ruby
2.5.3

Bundler?
false

Git
git version 2.24.1

Installation Source
~/.rbenv/versions/2.5.3/bin/fastlane

Host
Mac OS X 10.14.6 (18G3020)

Ruby Lib Dir
~/.rbenv/versions/2.5.3/lib

OpenSSL Version
OpenSSL 1.1.1d  10 Sep 2019

Is contained
false

Is homebrew
false

Is installed via Fabric.app
false

Xcode Path
/Applications/Xcode.app/Contents/Developer/

Xcode Version
11.3.1

System Locale

Variable
Value

LANG
en_US.UTF-8
✅

LC_ALL
en_US.UTF-8
✅

LANGUAGE

fastlane files:
No Fastfile found
No Appfile found
fastlane gems

Gem
Version
Update-Status

fastlane
2.135.0
🚫 Update available

Loaded fastlane plugins:
No plugins Loaded
Loaded gems

Gem
Version

did_you_mean
1.2.0

slack-notifier
2.3.2

atomos
0.1.3

CFPropertyList
3.0.2

claide
1.0.3

colored2
3.1.2

nanaimo
0.2.6

xcodeproj
1.14.0

rouge
2.0.7

xcpretty
0.3.0

terminal-notifier
2.0.0

unicode-display_width
1.6.1

terminal-table
1.8.0

plist
3.5.0

public_suffix
2.0.5

addressable
2.7.0

multipart-post
2.0.0

word_wrap
1.0.0

tty-screen
0.7.0

tty-cursor
0.7.1

tty-spinner
0.9.3

babosa
1.0.3

colored
1.2

highline
1.7.10

commander-fastlane
4.4.6

excon
0.72.0

unf_ext
<IP_ADDRESS>

unf
0.1.4

domain_name
0.5.20190701

http-cookie
1.0.3

faraday-cookie_jar
0.0.6

faraday
0.17.3

faraday_middleware
0.13.1

fastimage
2.1.7

gh_inspector
1.1.3

mini_magick
4.10.1

multi_xml
0.6.0

rubyzip
1.3.0

security
0.1.3

xcpretty-travis-formatter
1.0.0

dotenv
2.7.5

bundler
2.1.4

naturally
2.2.0

simctl
1.6.7

jwt
2.1.0

uber
0.1.0

declarative
0.0.10

declarative-option
0.1.0

representable
3.0.4

retriable
3.1.2

mime-types-data
3.2019.1009

mime-types
3.3.1

multi_json
1.14.1

signet
0.12.0

memoist
0.16.2

os
1.0.1

googleauth
0.6.7

httpclient
2.8.3

google-api-client
0.23.9

emoji_regex
1.0.1

json
2.3.0

digest-crc
0.4.1

google-cloud-errors
1.0.0

google-cloud-env
1.3.0

google-cloud-core
1.5.0

google-cloud-storage
1.16.0

openssl
2.1.2

date
1.0.0

fileutils
1.0.2

ipaddr
1.2.0

psych
3.0.2

generated on: 2020-02-23

Actually although 2.135.0 appears to work, the APK doesn't seem to have made it to Google Play.

fastlane supply init give me version name in changelog, like 1.0.5, but I need to provide the version code to update changelog, like 24. Is this the expected behavior for fastlane? I'm new on fastlane, and also confused on this issue.

Same question here. Not sure how to configure the action:
[18:07:19]: ----------------------------------
[18:07:19]: --- Step: upload_to_play_store ---
[18:07:19]: ----------------------------------

+-------------------------+-----------------------------+
|              Summary for supply 2.143.0               |
+-------------------------+-----------------------------+
| version_code            | 100008                      |
| version_name            | 2020.3.2                    |
| track                   | alpha                       |
| json_key_data           | ********                    |
| package_name            | tuerantuer.app.integreat    |
| release_status          | completed                   |
| metadata_path           | ./fastlane/metadata/android |
| skip_upload_apk         | false                       |
| skip_upload_aab         | false                       |
| skip_upload_metadata    | false                       |
| skip_upload_changelogs  | false                       |
| skip_upload_images      | false                       |
| skip_upload_screenshots | false                       |
| validate_only           | false                       |
| check_superseded_tracks | false                       |
| timeout                 | 300                         |
| deactivate_on_promote   | true                        |
+-------------------------+-----------------------------+

+------------------+--------------------------+
|                Lane Context                 |
+------------------+--------------------------+
| DEFAULT_PLATFORM | android                  |
| PLATFORM_NAME    | android                  |
| LANE_NAME        | android playstore_upload |
+------------------+--------------------------+
[18:07:20]: Could not find release for version code '100008' to update changelog

related? https://github.com/fastlane/fastlane/issues/15681


Evaluation memoization service

Is your feature request related to a problem? Please describe.
I am always frustrated whenever I build something that relies on listening to channel bumps and consuming evaluation results, I cannot find a single place where I could source them and would need to memoize them myself.
This is expensive on my weak web services.
Describe the solution you'd like
I would like an evaluation memoization service that could reuse Hydra evaluation results.
Something where I could:
(a) listen to new channel bumps
(b) ask for any evaluation result for any channel bump
If it does exist, I would like documentation to rely on this and best practices.
Describe alternatives you've considered

Re-evaluate everything I care about, etc. Note that I build it in https://github.com/Nix-Security-WG/nix-security-tracker/blob/main/src/website/shared/listeners/nix_evaluation.py.
Do nothing

Additional context

Meta comment: Ideally, we can get members of the community involved in the design and development of this service. And then have a good hand-over to the infra team for deployment. I don't know how well the infra team can scale if they also become responsible for R&D.

Meta comment: Ideally, we can get members of the community involved in the design and development of this service. And then have a good hand-over to the infra team for deployment. I don't know how well the infra team can scale if they also become responsible for R&D.

I agree, as I mention it, I developed similar stuff (linked in the body of the original issue) and I can develop a specialized tool for it, but it would make sense to reuse Hydra evaluation results for that and pass them on to the right tool. Because of that, it seems hard to me to put such an issue elsewhere than the infra issue tracker.

@tomberek you wanted to do something similar if I remember correctly our discussion at OceanSprint. I personally am also interested for research purposes.

This issue has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/when-should-meta-attributes-be-computed-vs-raw-data/42833/14


Seams that task sheduler handles timeouts incorrectly.

System components timeout issue
Call to system functions like tx_event_flags_get, tx_semaphore_get etc. with timeout != TX_WAIT_FOREVER  results in thread wake before timeout in some conditions.
Description
In my program I am using threadx with the netxduo on with STM32H723 processor.
From the high level the program creates server socket and receives commands from the network. There is a command to make a USART transaction and return the response back via the socket. The transaction is just USART send/receive with timeout.
There is "DeviceMaster" thread, it makes USART transactions. USART is configured to transmit/receive data using DMA and USART idle interrupt.
DMA TX ISR handle:
...
tx_event_flags_set(&deviceMaster.eventFlags, DM_WRITE_FLAG, TX_OR);
...

USART ISR:
    /* Check for IDLE line interrupt */
    if (LL_USART_IsEnabledIT_IDLE(UART_INSTANCE) && LL_USART_IsActiveFlag_IDLE(UART_INSTANCE)) {
        LL_USART_ClearFlag_IDLE(UART_INSTANCE);                     /* Clear IDLE line flag */
        ...
        if (tx_event_flags_set(&deviceMaster.eventFlags, DM_READ_FLAG, TX_OR) != TX_SUCCESS) {
            Error_Handler();
        }
    }

Transaction processing in the thread pseudo code, simplified:
// Checks if previous send completed
tx_event_flags_get(&deviceMaster.eventFlags, DM_WRITE_FLAG, TX_OR_CLEAR, &flags, 10);
sendDataToUsart();
// wait for remote device response with timeout=10
tx_event_flags_get(&deviceMaster.eventFlags, DM_READ_FLAG, TX_OR_CLEAR, &flags, 10);
getDataFromUsart()

Problem
Sometimes call tx_event_flags_get(&deviceMaster.eventFlags, DM_READ_FLAG, TX_OR_CLEAR, &flags, 10);  takes 1-2 ticks instead of 10 (ends before timeout). I have investigated the behavior a little bit using debug out and TraceX. What I have fonud is that ethernet ISR happened when the  "DeviceMaster" thread suspended wakes not only ethernet-related thread but the  "DeviceMaster" as well. Attaching TraceX file.
TraceX good/expected flow
Event/Description
1017 - Transaction start marker; start DMA transfer
1018 - check for DM_WRITE_FLAG
1019 - DM_WRITE_FLAG set in USART TX DMA
1020 - Get time to check later (tx_get_time)
1021 - Wait for DM_READ_FLAG flag
1022 -  "DeviceMaster" suspended
...
1024 - DM_READ_FLAG set in the USART RX ISR
1025 - system resumes "DeviceMaster"
1026 - Transaction end marker

TraceX problem flow
1036 - Transaction start marker; start DMA transfer
1037 - check for DM_WRITE_FLAG
1038 - DM_WRITE_FLAG set in USART TX DMA
1039 - Get time to check later (tx_get_time)
1040 - Wait for DM_READ_FLAG flag
1041 - "DeviceMaster" suspended
1042 - "NetX ip instance" has started
...
1047 - set event
1048 - scheduler resumes "NetX ip instance"
1049 - scheduler resumes "DeviceMaster", but it still should wait for the timeout.
1050 - "DeviceMaster" gets console mutex to print timeout error
1051 - "DeviceMaster" puts console mutex back
1052 - "DeviceMaster" gets time to check how many ticks the thread was suspended.
1053 - Custom TraceX event that indicates the error and the number of ticks the thread was suspended = 2 ticks, not 10

Assumptions about root cause
I have taken a quick look at the threadx sources and assume that the following logic in  tx_timer_expiration_process.c file probably is not correct. this and this
if (current_timer -> tx_timer_internal_remaining_ticks > TX_TIMER_ENTRIES)
{
...
} else {
 /* Timer did expire.  */
}

Not sure, but looks like in case if  tx_timer_internal_remaining_ticks < 32 the system considers the timer is expired. But in my case the timer value is just 10...
Expected behavior
The system scheduler wakes threads when the timeout happened not before.
Impact
Timeouts work incorrectly across the system
TraceX files
CustomEvents.zip
Ticks to microsecond is 544 in this case.

Hi @asamuta ,
The code you showed in threadx timer looks OK, current_timer is timer that expired:
https://github.com/eclipse-threadx/threadx/blob/485a02faec6edccef14812ddce6844af1d7d2eef/common/src/tx_timer_expiration_process.c#L203
And the check you are showing is to check if it expired due to real timeout or due to 32 ticks finished (in threaedx architecture, if some timer defined to more than TX_TIMER_ENTRIES, it will be in expired state each TX_TIMER_ENTRIES ticks, and then re-trigger till getting to 'real' expiration).
For me (as none threadx expert) it is hard to understand your problematic flow. If you can create minimal reproducible example it may be easier for you and for other users to debug it.

Hi @amgross,
The high level description of the problem:
result = tx_event_flags_get(&deviceMaster.eventFlags, DM_READ_FLAG, TX_OR_CLEAR, &flags, 10);

doesn't really wait 10 tick and wakes the thread after 2 ticks with result == 0x7 witch is
#define TX_NO_EVENTS                    ((UINT) 0x07)

And it hapens when another flag is set while my thread is waiting for its flag. (Ethernet ISR set a flag that is used by netxduo in my case).

Sorry, without full code with minimal reproducible example (probably main which create one thread that one catch the event flag and the other releases it) I can't help, maybe someone else can


accuracy alway be 1.0

I train a resnet50 with num_label = 2，but when trianing the accuracy alway be 1.0, use the same Dataset  training on caffe1.0 version  with googlenet v1, I can only get accuracy not more than 0.85.
what's the problem it may be ?
follow is some log
INFO:data_parallel_model:Parameter update function not defined --> only forward
INFO:resnet50_trainer:Starting epoch 0/100
INFO:resnet50_trainer:Finished iteration 1/1562 of epoch 0 (34.41 images/sec)
INFO:resnet50_trainer:Training loss: 0.693147480488, accuracy: 1.0
INFO:resnet50_trainer:Finished iteration 2/1562 of epoch 0 (477.39 images/sec)
INFO:resnet50_trainer:Training loss: 0.602652907372, accuracy: 1.0
INFO:resnet50_trainer:Finished iteration 3/1562 of epoch 0 (549.62 images/sec)
INFO:resnet50_trainer:Training loss: 0.495978534222, accuracy: 1.0
INFO:resnet50_trainer:Finished iteration 4/1562 of epoch 0 (574.50 images/sec)
INFO:resnet50_trainer:Training loss: 0.390794098377, accuracy: 1.0
INFO:resnet50_trainer:Finished iteration 5/1562 of epoch 0 (572.76 images/sec)
INFO:resnet50_trainer:Training loss: 0.298585027456, accuracy: 1.0
....

I'm having the same pb. However, when running it from the multi gpu training tutorial, accuracy seems to be correct. Still inquiring what's the difference between the 2 codes

DUP with #810 , we find that all data and label blobs fetch from workspace 0

Ok, so just change use_caffe_datum in "brew.image_input" to "False" if you're using input data in caffe2 format, and it will work !

thanks @neobarney , I thank you can send a pull request.


Could not validate credentials popup

Popup keeps appearing when jwt expired

If someone is logged out, please change the visible state in the user area back to the login button