Automatic finalAttrs pattern migration

I recently wrote a tool that automatically performs a migration to the finalAttrs pattern (#119942), and I believe it is now robust enough to be let loose on nixpkgs.
However this is the first large treewide change I'm performing, so I'm not 100% sure how to proceed. Some questions:

Is this desirable at all?
Should I perform the migration in steps or all at once?

Do we review all changes or only a sample?
@infinisil you are working on the by-name migration (#258650). Do you think this could interfere? Should I wait until your migration is complete?

cc @drupol who was interested in this
Example run of the tool on pkgs/applications and pkgs/by-name: https://github.com/fgaz/nixpkgs/commits/finalattrs-test-8/

Discussed this in the Architecture Team meeting:

@roberth: Not sure if it's worth doing the tree-wide change, there's likely going to be a better replacement

The finalAttrs pattern does fix problems, but not always very useful, might be better on a case-by-case
There is some push to use the pattern throughout.
Ideally it should also work for language-specific builders, see https://github.com/NixOS/nixpkgs/pull/234651

Easy to make a mistake, needs good testing

@infinisil: Tending towards doing the mass replacement
@roberth: Also fine with doing it if it can be pulled off without problems
@roberth: Potential problem with rev being set to finalAttrs.version: Overriding it reuses the previous source, no error

@fgaz: There was a discussion about this on Matrix

@infinisil: This seems worse than the problems finalAttrs fixed

@roberth: Original motivation was the passthru tests

@infinisil: How about improving the docs to better mention that src needs to be overridden

@roberth: Could have a more convenient function for overriding the source, something like
pkgs.hello.overrideAttrs (pkgs.changeSrc {
  # ...
  tag = "v1.2"; # potentially automatically strip v prefix
  hash = "...";
})

pkgs.haskell.lib has something like that

Or maybe the other way around: Deriving the package version from the fetcher.
Fetchers could have a standardised attribute to pass thru the rev/tag version, so the package can reuse.
Change stdenv.mkDerivation to use src.version as a default.

@fgaz: Could look into that

Should I perform the migration in steps or all at once?

To avoid merge conflicts, doing it in batches only helps when the batches are for packages that don't have open PRs
Probably doesn't cause too many merge conflict anyways
Suggestion: Do a smaller batch first (e.g. only ones in pkgs/by-name), and then the whole tree if there's no problems

One big commit or one commit per package?

@infinisil: One big commit
Trivial to revert manually

Do we review all changes or only a sample?

@infinisil: Review the tooling

@infinisil you are working on the by-name migration (RFC 140 migration plan #258650). Do you think this could interfere? Should I wait until your migration is complete?

@infinisil: With automated tooling, it can just be re-run to "resolve" the merge conflicts, so there's no problem doing multiple tree-wide changes

@roberth: Potential problem with rev being set to finalAttrs.version: Overriding it reuses the previous source, no error

This isn't caused nor exacerbated by finalAttrs at all. If you override version and not src, even without finalAttrs, you'll also silently use the existing sources.
The finalAttrs pattern in fact makes it easier to override src, as you may only need to set outputHash:
packageName.overrideAttrs (oldAttrs: {
  version = "0.1.1";

  src = oldAttrs.src.overrideAttrs {
    outputHash = "...";
  };
})

Without finalAttrs, you'd still need to set src, and more verbosely at that:
packageName.overrideAttrs (oldAttrs: {
  version = "0.1.1";

  src = fetchFromGitHub {
    repo = "...";
    owner = "...";
    rev = "...";
    hash = "...";
  };
})

This isn't caused nor exacerbated by finalAttrs at all.

The problem is that it looks more correct.
Something like "oh, I see the author is aware of the rec problem. Great, now I can override the version easily."
It's a flawed thought when you think about the hash, but if someone's in a hurry or they're distracted, and the build still works, they'll be confused:
"The build worked, so it can't be the hash."
Yes, it is scope creep, but it's quite a serious issue that is exacerbated by this, even if slightly.

Deriving the package version from the fetcher.

I think this is the most promising pattern; something like:
  version = finalAttrs.src.version;
  src = fetchFromGitHub {
    rev = "v2.1";
    # ...
  };

if someone's in a hurry or they're distracted, and the build still works, they'll be confused

I doubt most users are aware of the implementation details of everything they override. If they were, surely after the first couple of "FOD moments" they'd know better?
Either way this seems more like an opportunity to document/educate. FODs are already discussed in the nix manual, so the nixpkgs manual's section on overriding might direct them there with some ancillary notes.

Deriving the package version from the fetcher.

I think this is the most promising pattern; something like:

This forces users to not use their intuition to override version; this behaviour would be extremely surprising to encounter.
Moreover, extracting version from src.rev would be more tortured atm, as you can't use string interpolation; are you suggesting to change the fetcher API as well?

Yes, the idea was that each fetcher sets version.
It could be something like

A short rev if rev is a git sha
stripPrefix "v" rev if it's v[0-9].*
The rev itself if it's [0-9].*
No version attribute otherwise

I don't think fetchers currently set a version attribute, so this should be fine.

This forces users to not use their intuition to override version;

At least the package won't have any code that could make them believe that overriding just version could work.
I don't think this is an intuition, but rather an idea that occurs because they read the code wrong.

IMO: this should be done after https://github.com/NixOS/nixpkgs/pull/234651

seems like it has potential to silently break packages that don't have enough unit tests due to subtle differences in semantics.

How would he ensured that there won't be an unmanageable amount of silent breakages especially in people's overlays?
Also it would probably be a good idea to first write a tool to verify the correctness of FOD hashes to easily spot breakages in any urls which might change because of wrong usages of pname as part of URLs/paths/binary names in combination with overlays which append prefixes like -headless or -minimal.

i think packages should be updated to use finalAttrs as they're touched honestly, even if you made the commits automatically, you would still need to review them manually, and that would probably be about the same amount of work, but because github only allows you to approve a PR and not individual files of a PR, all that review work would probably have to be done by one person (actually several, a change this big should require multiple approvals)
there's lots of patterns that are inoccuous with rec that are a disaster waiting to happen with finalAttrs

there's lots of patterns that are inoccuous with rec that are a disaster waiting to happen with finalAttrs

I'd agree, if this tool didn't account for the main point of rec misuse: name/pname references. I was wary myself before testing the tool - and I think fgaz undersold the considerations put in to design the tool - but I currently think it is robust enough to at least initiate a PR.

I'd agree, if this tool didn't account for the main point of rec misuse

but we also need to look at the less often used misuses, otherwise we could run into a situation where a tiny fraction of the changes, still takes a significant amount of the time to fix things afterwards. I don't have good idea, how we could tell apart packages which are very safe to change and which will likely cause issues. Maybe we would need to analyze the AST and can then categorize the packages based on that and start with the very safe packages and do the complicated ones by hand.
Also the following comment from the readme makes me think that this is not rock solid and deeply tested:

Warning: Even though the library checks that derivations did not change, skimming the output for errors is a good idea. The program could have incorrectly changed something in meta, passthru, or an expression that is only evaulated under certain conditions.

Without finalAttrs, you'd still need to set src, and more verbosely at that:
packageName.overrideAttrs (oldAttrs: {
  version = "0.1.1";

  src = fetchFromGitHub {
    repo = "...";
    owner = "...";
    rev = "...";
    hash = "...";
  };
})

This can already be shortened with override https://github.com/NixOS/nixpkgs/blob/01048ff3150ac438b9ffac42fb1ab8054fcae9d8/pkgs/applications/finance/odoo/default.nix#L14-L20

Indeed I forgot to mention the measures I took to ensure a robust migration:

Only actual undefined references are substituted. The tool gets the exact reference position from eval errors.
The tool checks that packages evaluate after the change (including meta and passthru).
Derivation hashes are compared to ensure they do not change.
If pname misuse is detected, the package is skipped.

Also the following comment from the readme makes me think that this is not rock solid and deeply tested:

Warning: Even though the library checks that derivations did not change, skimming the output for errors is a good idea. The program could have incorrectly changed something in meta, passthru, or an expression that is only evaulated under certain conditions.

That text is outdated, I'll remove it. meta and passthru are now checked. Regarding conditionals, as far as I can see they don't really hide undefined references from the same file.

write good tests for it

I should definitely add some. I've been using nixpkgs itself to test the tool, and I can extract some golden tests from it.

but we also need to look at the less often used misuses

Maybe I can have the tool output a list of all the used recursive variable names?

I think this is the most promising pattern; something like:
  version = finalAttrs.src.version;
  src = fetchFromGitHub {
    rev = "v2.1";
    # ...
  };

I thought about this, and I agree with @SuperSandro2000 that it isn't the right solution.

Ontologically, src depends on version, not the opposite
In the real world, src very often isn't as simple as fetchgit { rev = "v${version}", ... }, and extracting version from src can be a lot more complex than the opposite.

Even a simple fetchurl { url = "https://example.org/foo-1.2.3-1.tar.gz", ... } requires a regex
What if there are multiple srcs?
What if the version depends on something else?
It might not even be possible: try running git grep 'replaceStrings \[ *"." *\] \[ *"" *\]' in nixpkgs

We can do this only on packages where rev is "v${version}", but I think this is too much of a POLA violation

The problem of the irrelevant FOD hashes can be solved by embedding version or rev into the name of the FOD. The store path will change every time the version changes, forcing a re-fetch.
We could make fetchers support pname and version, and provide something like revPrefix to enable auto-construction of rev if not otherwise specified. The down side is that the file extension would be lost, which might break some tryUnpack hooks.

The problem of the irrelevant FOD hashes can be solved by embedding version or rev into the name of the FOD. The store path will change every time the version changes, forcing a re-fetch.
We could make fetchers support pname and version, and provide something like tagPrefix to enable auto-construction of rev if not otherwise specified. The down side is that the file extension would be lost, which might break some tryUnpack hooks. To fix this, nameSuffix could optionally be specified.

I made a PR #294068 to demonstrate this solution.

The problem of the irrelevant FOD hashes can be solved by embedding version or rev into the name of the FOD. The store path will change every time the version changes, forcing a re-fetch.

just want to let you know this has been tried several times before.

The problem of the irrelevant FOD hashes can be solved by embedding version or rev into the name of the FOD. The store path will change every time the version changes, forcing a re-fetch.

just want to let you know this has been tried several times before.

Are there some PR about these attempts? What obstacles did they encounter?

here's a few attempts at solving the fixed-output derivation problem
NixOS/nix#7999
NixOS/nix#969

Thank you!

here's a few attempts at solving the fixed-output derivation problem
NixOS/nix#7999 NixOS/nix#969

Those proposal targets Nix itself, will mine works on the fetchers inside Nixpkgs and doesn't affect the behavior of Nix.

see also NixOS/rfcs#171: Default name of fetchFromGithub FOD to include revision

@ShamrockLee also note that nix flakes seem to have implemented some kind of workaround for this, as i haven't seen this problem occur there.

@ShamrockLee also note that nix flakes seem to have implemented some kind of workaround for this, as i haven't seen this problem occur there.

@lolbinarycat I haven't dug deep into the C++ implementation of flake inside Nix, but since flake.lock includes the input specifications (including revision specification) as nodes.<input>.original, it's quite easy for compare such specifications between flake.nix and flake.lock, and re-fetch upon mismatch.
At Nix level, they can see the original store derivation (/nix/store/*.drv). and have space to decide if they would like to re-realize the outPaths or just reuse them. The latter is the current implementation.
At Nixpkgs level, all we have is the current specification, nothing else. All we could use to invalidate previous store paths are new store paths. As the drvPath of a FOD is determined by both name and outputHash and that the outputHash is specified by the user, our best bet is to code the revision into the name attribute. That's what #294068 and NixOS/rfcs#171 is about. (The review process of #294068 is paused in favour of NixOS/rfcs#171.)

The finalAttrs pattern in fact makes it easier to override src, as you may only need to set outputHash:
packageName.overrideAttrs (oldAttrs: {
  version = "0.1.1";

  src = oldAttrs.src.overrideAttrs {
    outputHash = "...";
  };
})

:thinking: So here oldAttrs is actually finalAttrs which picks the new overridden version? It seems weird to me, if it's not oldAttrs, why does src = finalAttrs.src.overrideAttrs ... not causing an immediate infinite recursion? If it's actually not oldAttrs, we also need to update all relevant documentations.

@oxalica oldAttrs here is prevAttrs not finalAttrs

@oxalica no, oldAttrs in that example is prevAttrs, not finalAttrs. Here's a full example that should hopefully demonstrate, why this works:
# hello_2_10/package.nix
{ stdenv, fetchurl }:

stdenv.mkDerivation (finalAttrs: {
  pname = "hello";
  version = "2.10";

  src = fetchurl {
    url = "mirror://gnu/hello/hello-${finalAttrs.version}.tar.gz";
    hash = "sha256-MeBmE3qWJnbon2nRtlOC3pWn732RS4y5VvQepy4PUWs=";
  };
})

# some_overlay.nix
hello_2_12 = hello_2_10.overrideAttrs (
  finalAttrs: prevAttrs: {
    version = "2.12";
    src = prevAttrs.src.overrideAttrs {
      outputHash = "sha256-zwSvhtwIUmjF9EcPuuSbGK+8Iht4CWqrhC2TSna60Ks=";
    };
  }
);

The reason, why you don't have to manually propagate version to src is because the original derivation used finalAttrs.version when specifying the url in fetchurl (or rev = finalAttrs.version in the case of fetchFromGithub).

yeah the arguments are opposite in overrideAttrs and mkDerivation
mkDerivation final is first
overrideAttrs prev is first

yeah the arguments are opposite in overrideAttrs and mkDerivation
mkDerivation final is first
overrideAttrs prev is first

Not really? Here are the valid "signatures":
{
  # Old-style signatures
  # You can still use them, but they don't cooperate well with multiple overrideAttrs
  # rec { } was commonly used with these signatures, but now it's considered an antipattern
  _ = mkDerivation { };
  _ = overrideAttrs { };
  _ = overrideAttrs (prevAttrs: { });

  # New-style signatures
  # (added in PR 119942)
  _ = mkDerivation (finalAttrs: { });
  _ = overrideAttrs (finatlAttrs: prevAttrs: { });
}

mkDerivation takes zero or one argument (it never takes prevAttrs, because there is no such thing as "previous" attributes, when creating the derivation from scratch) and overrideAttrs takes zero, one or two arguments.


Add extension initialization hook

Description

Prep for implementing AWS SDK instrumentation https://linear.app/serverless/issue/SC-690/python-sdk-instrument-aws-sdk-requests
This provides a way for the lambda SDK to perform initialization logic specific to the lambda runtime

Testing done
Unit tested

@Mmarzex ready for review please 🙏

@medikoo ready for review please


WIP: Special name for deinit

No longer use the known identifier deinit for destructors but use the special name infrastructure created for subscripts.
This allows classes to have members called deinit. Previously, creating a function named deinit resulted in weird function redecoration errors and adding an instance variable deinit crashed the compiler (there are no SR numbers for these issues as I only discovered them while implementing this PR).
It also removes another identifier from known identifiers ;-)

@swift-ci Please test

@swift-ci Please test

Build failed
Jenkins build - Swift Test Linux Platform
Git Commit - 14f1d5fb1c4d211a3b59d769b0466c583a189e37
Test requested by - @ahoppen

Build failed
Jenkins build - Swift Test OS X Platform
Git Commit - 14f1d5fb1c4d211a3b59d769b0466c583a189e37
Test requested by - @ahoppen

The way the PR description is phrased this sounds like some kind of troll.  Just sayin' .

@dabrahams That's certainly not what I wanted. Thanks for the feedback. I've updated it and hope it sounds more serious now.

@jrose-apple Could you have another look through this? I think all the questions have been answered and your comments taken care of.
@swift-ci Please test

Build failed
Jenkins build - Swift Test Linux Platform
Git Commit - 8a839ed81cafbb8de0f163d1d202d876fcdb0e01
Test requested by - @ahoppen

Build failed
Jenkins build - Swift Test OS X Platform
Git Commit - 8a839ed81cafbb8de0f163d1d202d876fcdb0e01
Test requested by - @ahoppen

Ah, I see, I thought by all keywords you meant escaping all keywords that clashed with special decl names. I'll create another PR for it then.
@swift-ci Please test and merge

Test once again before merging since old test is already several days old.
Btw: Any idea why swift-ci wasn't able to merge the PR?
@swift-ci Please smoke test and merge

swift-ci didn't run. Let's try again
@swift-ci Please smoke test and merge


Deserialize json into a struct

Hello! I need to deserialize the following json into a struct:
{
  "blocks": {
    "field1": ["142A538F36833D1CC78B94E11C766F75818F8B940771335C6C1B8AB880C5BB1D"],
    "field2": ["4C1FEEF0BEA7F50BE35489A1233FE002B212DEA554B55B1B470D78BD8F210C74"]
  }
}

I tried this, but doesn't work:
#[derive(Deserialize, Debug)]
struct WalletPending {
    blocks: HashMap<String, Vec<Option<String>>>,
}

The arrays can have more than one element or be empty and there can be more "fields". How can I do this? Thanks!

Seems to work. https://play.rust-lang.org/?edition=2018&gist=d3986a2e9cf610278ccebacf784cc28a

I was using "from_slice" to deserialize from bytes. When I convert the bytes to a string, blank spaces appear as "/", and that's why it gives me the error "Error("invalid type: string "")". Anyway I can solve this?

Foud this issue: https://github.com/serde-rs/serde/issues/1425.
However, I can't seem to adapt it to my struct. Can someone help me, please?

@Fiono11 The current version of serde_with is unfortunately not very composable, so your case is not easily supported. With the development version it git however, this is possible. Note the value of field1 is different compared to your playground link.
:dep serde = {version = "*", features = ["derive"]}
:dep serde_json
:dep serde_with = { git = "https://github.com/jonasbb/serde_with" }

use serde::{Deserialize, Serialize};
use std::collections::HashMap;
use serde_with::{As, Same, NoneAsEmptyString};

#[derive(Serialize, Deserialize, Debug)]
struct WalletPending {
    #[serde(with = "As::<HashMap<Same, Vec<NoneAsEmptyString>>>")]
    blocks: HashMap<String, Vec<Option<String>>>,
}

fn main() {
    let j = r#"
        {
          "blocks": {
            "field1": [""],
            "field2": ["4C1FEEF0BEA7F50BE35489A1233FE002B212DEA554B55B1B470D78BD8F210C74"]
          }
        }
    "#;

    let value = serde_json::from_str::<WalletPending>(j).unwrap();
    println!("Value:\n{:#?}", value);
    println!("---");
    println!("JSON:\n{}", serde_json::to_string_pretty(&value).unwrap());
}
main()

Value:
WalletPending {
    blocks: {
        "field2": [
            Some(
                "4C1FEEF0BEA7F50BE35489A1233FE002B212DEA554B55B1B470D78BD8F210C74",
            ),
        ],
        "field1": [
            None,
        ],
    },
}
---
JSON:
{
  "blocks": {
    "field2": [
      "4C1FEEF0BEA7F50BE35489A1233FE002B212DEA554B55B1B470D78BD8F210C74"
    ],
    "field1": [
      ""
    ]
  }
}


BUG: build warning of download tensorflow runtime package

Description
Tried build jaxlib in WSL2 on Windows 11 and got below warnings. Seems the files cannot be downloaded with below URL, but the files are available in the original URL like "https://github.com/tensorflow/runtime/archive/e3da90c2cb1bd594a90ffb06b80f6d8df47c4e11.tar.gz". Are these build warnings critical for a successful build?
WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/tensorflow/runtime/archive/e3da90c2cb1bd594a90ffb06b80f6d8df47c4e11.tar.gz failed: class java.io.FileNotFoundException GET returned 404 Not Found
WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/llvm/llvm-project/archive/2c3ca3b684bb2b188d977d47548e79dc559fb8ad.tar.gz failed: class java.io.FileNotFoundException GET returned 404 Not Found

What jax/jaxlib version are you using?
Build from latest source
Which accelerator(s) are you using?
CPU
Additional System Info
Windows 11, WSL2.

These are benign warnings. They happen because we haven't added that particular file to a mirror repository.


[analyzer] Break-up sections of diagnostic documentation

Looking for feedback :)
Background
Currently in the analyzer's messages.yaml and the CFE's messages.yaml, the documentation used to generate https://dart.dev/diagnostics is grouped into one field, with the headers (description, example, and common fixes) manually specified. This makes it harder for consumers of this file to configure the generation, since the headers are already specified at specific level. Beyond that, some of the analyzer's test tooling splits up the sections as well.

Currently it looks roughly like this
DIAGNOSTIC_EXAMPLE:
  ...
  hasPublishedDocs: true
  documentation: |-
    #### Description

    Description goes here....

    #### Examples

    Example 1 goes here...

    Example 2 goes here...

    #### Common fixes

    Common fix 1 goes here...

    Common fix 2 goes here....

Initial thoughts/proposal
To make it easier for tools/generators to choose the output format and tests to look at individual sections:

Split the three documentation categories into their own key-value pairs under documentation
To solve the lack of "Example" vs "Examples" with this change, we could either have two separate pairs or instead accept a list of examples.
Move generation of the headers and combination of examples/fixes to the generator

I'm not 100% sure the list for examples and fixes makes sense or is worth it, so perhaps you have some thoughts.

Depending on what direction seems best, it could look something like this:
DIAGNOSTIC_EXAMPLE:
  ...
  hasPublishedDocs: true
  documentation:
    description: |-
      Description goes here....
    examples:
      - |-
        Example 1 goes here...
        Example 1 continues here...
      - |-
        Example 2 goes here...
        Example 2 continues here...
    fixes:
      - |-
        Fix 1 goes here...
      - |-
        Fix 2 goes here...

\cc @bwilkerson @MaryaBelanger @pq

The other aspect of this, IIRC, was a thought that we might want more than just the three existing sections. For example, we might want justification or seeAlso sections for the proposed pages that help users decide which lints they want to enable. There was even a mention (though not necessarily a plan) to allow the sections to be mixed and matched to some extent, such as by including the seeAlso text in both locations.
What I don't know is which pieces really make sense to mix and match. I'm absolutely in favor of avoiding duplication. But if being able to more easily change the heading levels in generated output were the only goal then I'd be skeptical that it was worth the effort.

Split the three documentation categories into their own key-value pairs under documentation (or top-level)

If we're going to split them, it's definitely better to have them under documentation; it removes the possibility of conflicts between doc needs and other needs.

If we're going to split them, it's definitely better to have them under documentation; it removes the possibility of conflicts between doc needs and other needs.

That sounds good to me 👍

The other aspect of this, IIRC, was a thought that we might want more than just the three existing sections. For example, we might want justification or seeAlso sections for the proposed pages that help users decide which lints they want to enable.

Sorry, you're right. I was focusing on the existing diagnostic structure here, since we haven't exactly determined the "justification" part of linter rules, but especially for the linter rules, there will definitely be other information we'll want to include within the documentation map.
You mention seeAlso, and I think that would be useful for lints like you mentioned, but potentially diagnostics as well, such as pointing to related docs or even videos describing how to address commonly hit diagnostics.

What I don't know is which pieces really make sense to mix and match.

I think at this point, it's probably fine not to worry about what exactly will be mixed and matched, and focus on being consistent with the current diagnostics while we are still migrating since that is the primary benefit of this work. Splitting up the categories now will allow us to use those entries for the justification docs if/when it makes sense relatively easily.

As a note, one thing I did not address in this issue since it's a bit unrelated, but may reside in the linter equivalent file or sibling file(s), is extra information such as version support information and quick fix status.

As for modifying the existing messages and the generator, I'd be happy to do that work once we determine a structure we're happy with. Do you know if there are any other consumers of the documentation part of these files? If not, it's pretty low cost to adjust as needed.

... since we haven't exactly determined the "justification" part of linter rules ...

It might be worth thinking about it a bit before we choose an initial split. For example, would we want to use the same examples in both places? The current lint docs include both bad and good examples, but the diagnostic messages page format only has the bad. So do we need something like
documentation:
  description:
  examples:
  commonFixes:
  justification:
  goodExamples:
  badExamples:

Or would badExamples be sharable between the two?
Maybe an incremental approach is fine and I'm just overthinking this, but that is kind of what I get paid to do. :-)

You mention seeAlso, and I think that would be useful for lints like you mentioned, but potentially diagnostics as well ...

(Minor nit: lints are diagnostics too. :-) But yes, that seems like it might be a sharable piece, unless the links are specific to the user journey that brings users to that page.

Do you know if there are any other consumers of the documentation part of these files?

As far as I know, there aren't any clients outside of the analyzer (though I believe the "parsing" code is shared with the front end).

If not, it's pretty low cost to adjust as needed.

It will become higher cost if we move the generator out of the SDK repo because it's always harder when there are multiple repos involved.


Remove 10s reconnect for nwc

Need this first: https://github.com/MutinyWallet/blastr/pull/23

This works great! I'm noticing sometimes I refresh and get sent the same events I had once already failed on but it doesn't hurt it. Probably just because cloudflare isn't always instant with KV updates. But all new events seem to come in within the 10s so nice work.


[BUGFIX] Handle AWS API Error while deleting S3 objects

This change prevents having duplicated data in a table ( incremental, insert_overwrite, partition ) that can be caused by missing permission.
Fixes #85

@smoens you can add the change I made, then you can see the more detailed log
OR, you can try a sample code that delete the partitions with your AWS Role.
import boto3

s3_resource = boto3.resource('s3')
s3_bucket = s3_resource.Bucket(bucket_name)
response = s3_bucket.objects.filter(Prefix=prefix).delete()
print(response)

The response result will tell you if there is any error

Typo in the first post: this PR addresses #84


Feature/scalar quantized off heap scoring

This adds off-heap scoring for our scalar quantization.
Opening as DRAFT as I still haven't fully tested out the performance characteristics. Opening early for discussion.

Half-byte is showing up as measurably slower with this change.
Candidate:
0.909	 0.54
0.911	 0.58
0.919	 0.88

baseline:
0.909	 0.30
0.911	 0.33
0.919	 0.47

Full-byte is slightly faster
candidate:
0.962	 0.41
0.966	 0.43
0.978	 0.66

baseline:
0.962	 0.47
0.966	 0.48
0.978	 0.73

are you reporting indexing times? query times?

are you reporting indexing times? query times?

Query times, single segment, 10k docs of 1024 dims.

Ok, I double checked, and indeed, half-byte is way slower when reading directly from memory segments instead of reading on heap.
memsegment_vs_baseline.zip
The flamegraphs are wildly different. So much more time is being spent reading from memory segment and then comparing the vectors
candidate (this PR):

baseline:

@ChrisHegarty have you seen a significant performance regression on MemorySegments & JDK22?
Doing some testing, I updated my performance testing for this PR to use JDK22 and now it is WAY slower, more than 2x slower, even for full-byte.
For int7, this branch is marginally faster (20%) with JDK21, but basically 2x slower on JDK22.
I wonder if our off-heap scoring for byte vectors also suffers on JDK22. The quantized scorer for int7 is just using those same methods.

To verify it wasn't some weird artifact in my code, I slightly changed it to where my execution path always reads the vectors on-heap and then wraps them in a memorysegment. Now JDK22 performs the same as JDK21 & the current baseline.
Its weird to me that reading from a memory segment onto ByteVector objects would be 2x slower on JDK22 than 21.
Regardless that its already much slower for the int4 case on both jdk 21 & 22.


v0.3.7 - Add reporting of allowed origins and which header failed in …

…headerNotAllowed. Tweak logs. Changed phpunit to remove checkForUnintentionallyCoveredCode

Coverage increased (+0.1%) to 91.2% when pulling 545496ebb1616db1dc09c3245df40d9eb66e0337 on develop into 2775ae70a0bc37506604d7f56857dd0eccfaa62a on master.


[LIVY-585][SERVER] Avoid duplicate stopping log when stop/interrupt a session

What changes were proposed in this pull request?
There's duplicate stopping log when stop/interrupt a session. This happens because server logs stopping info when we stopping/interrupting a session at the first time. And when the session expired, it would be removed from session manager and call stop() and logs stopping info for the second time(Though, it does not really stop the session again at this time).

The fix is quite simple by adding a _stopped flag in Session's stop() method.
How was this patch tested?
N.A.(?)

@vanzin  please have a look, thx.

@mgaido91 Thank you for your review. I've updated it.

Codecov Report

Merging #166 into master will decrease coverage by 0.06%.
The diff coverage is 37.5%.

@@             Coverage Diff              @@
##             master     #166      +/-   ##
============================================
- Coverage     68.82%   68.75%   -0.07%
+ Complexity      907      906       -1
============================================
  Files           100      100
  Lines          5662     5669       +7
  Branches        848      852       +4
============================================
+ Hits           3897     3898       +1
- Misses         1213     1216       +3
- Partials        552      555       +3

Impacted Files
Coverage Δ
Complexity Δ

.../main/scala/org/apache/livy/sessions/Session.scala
73.27% <37.5%> (-0.18%)
20 <1> (+1)

...cala/org/apache/livy/scalaapi/ScalaJobHandle.scala
52.94% <0%> (-2.95%)
7% <0%> (ø)

...ain/java/org/apache/livy/rsc/driver/RSCDriver.java
78.81% <0%> (-1.28%)
42% <0%> (ø)

...e/livy/server/interactive/InteractiveSession.scala
69.11% <0%> (+0.07%)
44% <0%> (ø)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d5b27dd...3f5d530. Read the comment docs.

If we are calling stop when it is not necessary, I think we should rather avoid calling it in those cases.

I was thinking about that way, but I was not sure whether it's the only place where calling stop unnecssary. So, I think it may be better to cover those cases(if any) in method stop (just like the thing we do in IteractiveSession.stopSession).

I think it may be better to cover those cases(if any) in method stop

I don't really agree with you. Let me cc @vanzin for his opinion too on this.


Create a testing channel in github

Closing as done, working prototype is https://github.com/rancher-sandbox/elemental-channels


Subset Sum Problem

Hi,
it will be great a swift version of the Subset Sum Problem. Do you agree?

I agree :) Feel free to open up a PR on it!


Make sure each TCP/UDP/WebSocket server uses an independent acceptor thread

By default, reactor-netty uses the first worker thread as the acceptor thread.
Note:

The worker threads register the channel in reactor.netty.transport.ServerTransport.Acceptor#channelRead
The acceptor thread is registered in reactor.netty.transport.TransportConnector#doInitAndRegister. channel.unsafe().register(elg.next(), monoChannelPromise);

BTW: The name of reactor.netty.transport.ServerTransport.Acceptor (The code is copied from io.netty.bootstrap.ServerBootstrap.ServerBootstrapAcceptor#channelRead) seems wrong and should be Worker instead of Acceptor

By default, reactor-netty uses the first worker thread as the acceptor thread.
Note:

The worker threads register the channel in reactor.netty.transport.ServerTransport.Acceptor#channelRead
The acceptor thread is registered in reactor.netty.transport.TransportConnector#doInitAndRegister. channel.unsafe().register(elg.next(), monoChannelPromise);

BTW: The name of reactor.netty.transport.ServerTransport.Acceptor (The code is copied from io.netty.bootstrap.ServerBootstrap.ServerBootstrapAcceptor#channelRead) seems wrong and should be Worker instead of Acceptor


Cannot write to cell.

Trying to write to a cell gives no result.
frstWorksheet.cell(forCellReference: "B2", shouldCreate: true).setStringValue("new value")
The above code does not work, or reflect the changes in the  #corresponding cells.
Also, as found in the previous issues, adding a border to the cell also does not help.
Please help. @renebigot

Who did you close it ? Did you manage to fix this ? I had the same problem.

@vishwas513 yes i got it fixed.


Ability to choose a MC Version

Describe the feature or problem you’d like to solve
Give users the ability to choose from 2 different MC versions when they initially rent the server

Describe the feature or problem you’d like to solve
Give users the ability to choose from 2 different MC versions when they initially rent the server

This is a really good idea!

This is a interesting issue so I have made some research my self and I have the following
How to get any version available
I have found this question which describes form which endpoint the current launcher gets all the availables versions
https://launchermeta.mojang.com/mc/game/version_manifest.json
As describe in the question from above you can get any version and most important its respective server.jar of each given version
{
  "latest": {
      "release": "1.18.1",
      "snapshot": "1.18.1"
  },
  "versions": [
      {
          "id": "1.18.1",
          "type": "release",
          "url": "https://launchermeta.mojang.com/v1/packages/b0bdc637e4c4cbf0501500cbaad5a757b04848ed/1.18.1.json",
          "time": "2021-12-10T08:26:34+00:00",
          "releaseTime": "2021-12-10T08:23:00+00:00"
      }
    ]
}

{
  "downloads": {
      "client": {
          "sha1": "7e46fb47609401970e2818989fa584fd467cd036",
          "size": 20042090,
          "url": "https://launcher.mojang.com/v1/objects/7e46fb47609401970e2818989fa584fd467cd036/client.jar"
      },
      "client_mappings": {
          "sha1": "99ade839eacf69b8bed88c91bd70ca660aee47bb",
          "size": 6611464,
          "url": "https://launcher.mojang.com/v1/objects/99ade839eacf69b8bed88c91bd70ca660aee47bb/client.txt"
      },
      "server": {
          "sha1": "125e5adf40c659fd3bce3e66e67a16bb49ecc1b9",
          "size": 46324407,
          "url": "https://launcher.mojang.com/v1/objects/125e5adf40c659fd3bce3e66e67a16bb49ecc1b9/server.jar"
      },
      "server_mappings": {
          "sha1": "9717df2acd926bd4a9a7b2ce5f981bb7e4f7f04a",
          "size": 5115621,
          "url": "https://launcher.mojang.com/v1/objects/9717df2acd926bd4a9a7b2ce5f981bb7e4f7f04a/server.txt"
      }
  }
}

Deploy any selected version
To get to this step I was thinking the following

Save the version of each server in the DB and maybe other properties
Implement a custom flow for building images given a specific version
Manage a central registry from where you can fetch a pre-build image of each version given a specific version tag (this is thinking for a deployment perspective)

@Arturo-Lopez interesting find with that manifest.json.  I'm currently working on a basic implementation, but I did have in mind using a custom registry with pre-built images to speed up deploy times and to reduce the need to run docker build on the agents.

@codyseibert That's a really cool project it saves you a lot of time and looks like the way to go

The initial work is setup for this, closing


JSON format

We are using Draft JS for our product in production. I wanted to confirm the following

Does Draft JS publish a JSON schema as a public spec or is it strictly a private format?
What are the chances that schema could change in coming versions and if that happens what will be the migration plan for the JSON already stored in DB.
What will be the frequency of change in schema?

Thanks
Arvind

Hi @arvsharm ,
I assume that by JSON schema you refer to the EditorState in 'raw' format? In which case the blocks and entityMap and the entity's mutability, data, type, block styling type and inline style, etc. have been around for a long time and that's essentially the schema.
I am a little confused, because you have mentioned that you are already using it in production? There isn't a private format, the EditorState is as per documentation and @types/draft-js. I haven't come across any talks of changing the above structure. A migration seems out of scope of the project, even if the same data has to be stored in a radically different structure, then migrating would be done by writing a conversion function and then executing against all pre-existing DB stored RawDraftContentState, obviously how is up programming language and kind of DB.

Yes EditorState in raw format is what we are storing in database. My question was regarding this only.
It looks like there is no plan to change this format as of now.
If a change happens we have to write our own conversion function and draftjs will not provide the same right?

Hi @arvsharm that's a brilliant question, and the docs partially answer it:

Because a text editor doesn't exist in a vacuum and it's important to save contents for storage or transmission, you will want to be able to convert a ContentState into raw JS, and vice versa. To that end, we provide a couple of utility functions that allow you to perform these conversions.

There was an effort underway, currently on hold (see #143 and #388) to represent the data structure as a tree instead of a flat list of content blocks. Keeping backwards compatibility with raw data created with the flat block map would be a design goal, but we'd surely provide conversion utilities if the underlying raw tree data structure weren't backwards compatible.


Dark Mode and CollectionView DataTemplate

Describe the bug
The generated CollectionView DataTemplate for ListDetail is unreadable when run on an OS set to Dark Mode. This has been reproduced under both Windows and iOS.
To Reproduce
Steps to reproduce the behavior:

Generate the default app template that includes the ListDetail page.
Run the app on a Windows or iOS machine set to use Dark Mode
The list will effectively render with black text on a black background

Expected behavior
Text should be reasonably colored (foreground and background) as appropriate to both Light and Dark modes on all platforms.
Screenshots

Versions (please complete the following information):

OS: Windows 11 and iOS 17
Visual Studio version 2022
Extension Version [e.g. 1.0 -- it's in the bottom-righthand corner of the wizard]: 1.3.3

Additional context
Just removing the TextColor setting in the DataTemplate appears to resolve the issue.

Hmmm.
I can't repro this.
Here's the app running on Win11 in both light and dark modes.

In both, the frame background is white so that you're not seeing this at all in your screenshots makes me wonder if any content has loaded at all.
If I remove the setting of the TextColor I get this:

Please can you share your generated project to allow me to investigate further.

Hmm... that's very odd. Alright, I'll gen up a new project with the latest release and see if it still repros here. I'll also screen shot the selections I make along the way in case that makes a difference. I'll let you know either way.

I ran into a situation where this showed up for me today.
It seems to be based on a combination of an expectation that the background of a Border will always be relative to the Theme, but on some devices/platforms/versions the color is affected by system-level theme settings. The simple solution is to explicitly set a theme relative color for the background, rather than relying on an assumption that we now know isn't always correct.
The next release (v1.7.1) will include a fix for this.

v1.7.1 is now available


Research JavaScript wrapper

I now wonder if this use case is not another point for dvc-render to have some support for javascript. If there is some pythonic wrapper for js maybe we could leverage vega to produce images in non-linear template use cases. But I guess that would require some research to determine this idea's feasibility.
Originally posted by @pared in https://github.com/iterative/dvc-render/pull/69#pullrequestreview-1033988010

maybe we could leverage vega to produce images in non-linear template use cases

@pared could you please clarify (expand) a bit on this? :)

@shcheklein sorry for not responding at the time, I thought I answered it at the time, but clearly I haven't. I guess answer now will be fuller after top-level plots are implemented for vscode and studio is on the way.
Why do we have dvc-render? - initially dvc plots was producing an HTML, later, when we started producing derivative projects plots started to have additional requirements, for example splitting data, and returning partially filled templates. When the requirements started to stack we moved dvc-render functionalities out of DVC - to not focus only on DVC, but let, for example, studio use it too. To some extent it worked, though not as ideally as I presumed originally. Later, we also created vscode extension, where dvc-render could not be used at all.
What is the main problem with my thinking? I assumed that "DVC" is a source of our plots, and that DVC should be responsible for parsing configs and data and passing the results to derivative projects. This assumption is wrong on a few levels.

This approach results in delays in development:
a.  vscode bases its plots on what it gets from DVC, so we cannot implement top-level plots there without doing that first in DVC, and then adjusting vscode
b. studio has already been parsing plots by itself, reusing parts of DVC parsing code, but still they were processing ingredients for plots differently (for example storing plots configs in database and data, separately, in S3) - in this case top-level plots could be implemented there in parallel, but more people would have to be involved.
We cannot create single data processing pipeline that would fit all projects - simply because of differences of UI in those projects. For example error handling: DVC logs it to stderr, the studio needs to collect them and display them in web UI, vscode does nothing, since it's not implemented yet: iterative/dvc#7692

So why do I think we should have a js dvc-render? Because both studio and vscode do some processing of data/plots configuration on frontend. For example, assigning different revision names, and overriding colors for particular data series.
What should we be doing:

source the data from plots.collect() method - it returns the actual source of truth - data, plot configs, and errors, if we cannot get the data/configs - of course for vscode we need to serialize the errors
Let all the projects handle the data however they see fit, especially handling the errors returned in the previous step, and for example do a caching (studio already does it, and it has already been said that data collection for vscode is too slow), and infer some additional requirements basing on config and data - for example, coloring, dashing of particular data series
use a common library (dvc-render-js) that could be used to display the plots - it would take care of ingesting loaded data, parsing config, extracting data series, and applying coloring if needed. Basically what  dvc.render.convert.vega.VegaConverter and dvc-render tries to do, but cannot for all projects as they cannot be used in frontend.

I should have seen this earlier but realized only after getting involved to our other projects.


Automated sample detector for LLM

Description of changes made
When the user asks to generate tests, the plugin should automatically detect some tests in code base that are using the CUT. then, it asks user if they want to provide any of these as samples.
Other notes
Closes ICTL-751 Automated sample detector for LLM

[x] I have checked that I am merging into correct branch

Thank @arksap2002 ,
In general, as the first implementation, it works pretty good. But, I have some comments to improve the implementation and UX:
1- currently, users can only select one method. Is it possible for us to add an option like append so users can add multiple test cases?
2-Current sample list is not user-friendly when we have lots of test methods from various test files (see image below). It only provides the test method name. We should find a way to categorize them according to the test file.

3- Please add a description for new keywords:


api-docs: Fix small typos and copy/paste errors in API changelog.

Current documentation
Screenshots and screen captures:

Feature levels 164 & 163
Feature level 164 - added missing colon
Feature level 163 - moved comma out of inline code tag for EMAIL_ADDRESS_VISIBILITY_EVERYONE.

Feature level 160
Removed square brackets since endpoints are not yet documented / nothing to link to

Feature level 150
Removed square brackets since PATCH /realm is not yet documented / nothing to link to

Feature level 148
Replace colon with comma

Feature level 43
Added link to endpoint documentation

Self-review checklist

[ ] Self-reviewed the changes for clarity and maintainability
(variable names, code reuse, readability, etc.).

Communicate decisions, questions, and potential concerns.

[ ] Explains differences from previous plans (e.g., issue description).
[ ] Highlights technical choices and bugs encountered.
[ ] Calls out remaining decisions and concerns.
[ ] Automated tests verify logic where appropriate.

Individual commits are ready for review (see commit discipline).

[ ] Each commit is a coherent idea.
[ ] Commit message(s) explain reasoning and motivation for changes.

Completed manual review and testing of the following:

[ ] Visual appearance of the changes.
[ ] Responsiveness and internationalization.
[ ] Strings and tooltips.
[ ] End-to-end functionality of buttons, interactions and flows.
[ ] Corner cases, error conditions, and easily imagined bugs.

Merged, thanks @laurynmm!


Conflict with HtmlWebpackPlugin - "Error: No chrome or browser runtime found"

Hello,
I'm trying to fix the dynamic imports in MV3 so I've added new WebExtension into the plugins array:
      new WebExtension({
        background: {
          entry: `background_gsd.worker`,
          // !! Add this to support manifest v3
          manifest: 3,
        },
      }),

And it seems to fix the background service worker, but at the same time my compilation now fails in the HtmlWebpackPlugin, which doesn't make sense to me:
ERROR in   Error: No chrome or browser runtime found

  - reload.html:81 Object.get runtime [as runtime]
    C:/Users/juraj/git/addons/GroupSpeedDial/src/reload/reload.html:81:92

  - reload.html:83
    C:/Users/juraj/git/addons/GroupSpeedDial/src/reload/reload.html:83:89

  - reload.html:84
    C:/Users/juraj/git/addons/GroupSpeedDial/src/reload/reload.html:84:13

  - reload.html:118
    C:/Users/juraj/git/addons/GroupSpeedDial/src/reload/reload.html:118:12

  - node:vm:139 Script.runInContext
    node:vm:139:12

  - index.js:142 HtmlWebpackPlugin.evaluateCompilationResult
    [GroupSpeedDial]/[html-webpack-plugin]/index.js:142:28

  - index.js:324
    [GroupSpeedDial]/[html-webpack-plugin]/index.js:324:26

  - async Promise.all

  - async Promise.all

Removing the new WebExtension(... from the plugins list fixes the problem so there must be some correlation I don't see.
Can you help? Or do I need to provide some example project?
Thanks!

Thank you! It works now.
I should have read the readme till the very end :)


Update modman

Generated automatically from generate-modman (see https://github.com/mhauri/generate-modman). This is necessary because modman uses symlinks and the lib/ entry which is currently in the modman file is too general, the resulting symlink wipes out files from other modules which use the same base paths like lib/. Same goes for the entries in the app/locale directory.

This is kind of an important issue, it makes installation with modman basically useless if using alongside other modules which also use the lib directory.


No page access with DNScrypt on Win 7 & Antergos

Hi I have installed DNScrypt on both OS. But neither of them works when I set the DNS server on IPv4 to <IP_ADDRESS>.
For Windows: I have followed all the steps and I got the confirmations that it was working properly. Connected to dnscrypt.eu-nl and for a while it did work.
Then I couldn't access pages anymore. I just got stuck after I type the domainname (firefox). And mentions unable to access the page.
So I reinstalled and again got all confirmations it was okay and for some reason it worked again after 2 or 3 times repeating this. But now it fails once again and I don't understand what's wronng. Is the server dnscrypt-eu.nl down or did I make an error somewhere?
I have the same with Antergos. Here I managed to install the package and have it running, get all the confirmations as well it's working. And it did work for a (short) while. Here the same as with Win 7, it doesn't work anymore as well. It seems to coincide.
The only way I can get my browser back working is by removing the <IP_ADDRESS> line and on both OS'es and set to automatic DNS. Now I can browse again. But I am not using DNScrypt anymore,. am I?
Can someone give me some advice on how to determine where the problem lies and// or how to solve this?

Hi,
You can run the proxy in foreground mode, with --loglevel=7 in order to get some logs displayed.
Also make sure that your system clock is correct, as certificates are short-lived and really need a clock that isn't way off to be verified.
Also try different servers. dnscrypt-eu.nl seems to work fine from my networks, but as dnscrypt is getting some traction, it may start getting blocked by ISPs (the China's Great Firewall has blocked dnscrypt.org for a while).


reset timer on item

Hey, I can do this!

I'm thinking the command usage would mirror the stop command - so it'll work either based on provided POSITION or the current timer.

Just to ensure I'm clear on the behavior:

When the timer is running: reset would reset it back to 0 and it will continue running from there.

When the timer is stopped: reset would clear the timer entirely.

So running these commands:

start 1
reset
stop
reset 1

with a 2 second delay between then results in this:

woah yes to all you wrote !! 😀  thats a nice behavior using these commands.
Please feel free to create that, thank you :)


Delete the torrent file question

Description
first i am a newbie at python, so i am sorry if its been asked, after the process i cant delete the torrent file(or move it), even if it is not an transmission-rpc problem, i thought to ask here.
Environment
transmission-rpc   3.4.0
How to reproduce
https://pastebin.com/QmrS38GS
Expectations
file deleted
Actual result
"Exception has occurred: PermissionError [WinError 32] The process cannot access the file because it is being used by another process"

if it is not an transmission-rpc problem, do not ask it here.


Legend in heatmap

I did a heatmap on the titanic dataset and it is a little hard to interpret. Some of the cells have numbers in them while others don't. I have no idea what the colors represent as there is no legend to interpret them.

Some of the cells have numbers in them while others don't.

Numbers are nullity correlation scores scaled in the range [-1, 1]. Cells without numbers in them score close to 0, e.g. have confidently completely random nullity correlations. It would be tedious to interpret a plot with a lot of 0 scores so the visualization leaves them out.
In general, for matters of interpretation, the corresponding README.md section should be sufficient.

I have no idea what the colors represent as there is no legend to interpret them.

A legend is not a bad feature to add. But I think that once you know what the scores mean, you do know what the colors mean. Stronger red means stronger negative correlation, and stronger blue (not present here) means stronger positive correlation. 🙂

Thanks for your response.
Consider my point of view (been doing ML for years) as someone fresh to your project and your's as the developer (you know the code inside and out) are probably not seeing the results as someone new to the project would. You understand the plot and it makes intuitive sense to you. To me it is not interpretable. Even after going through the README. If it weren't for the numbers, I would say that boat and body are highly (pos or neg) correlated, much more than boat and home.dest. I would venture to guess that most people new to your project would have a similar reaction to mine with this plot.
I don't want to have to read the README to interpret the heatmap. A good visualization should stand on its own. If I'm going to include it in a slideshow, I want it to be self-explanatory. A legend would greatly assist that. At the very least it should be an option.
I'm not trying to be negative. I think the project is awesome, and I love the other plots. I think a few small tweaks would be really beneficial to others who would like to take advantage of your project!

To me it is not interpretable. Even after going through the README.

Fair enough, I can add a legend feature to the plot.

In my local copy I added this by updating the signature to:
def heatmap(df, inline=False,
        filter=None, n=0, p=0, sort=None,
        figsize=(20, 12), fontsize=16, labels=True, cmap='RdBu',
        cbar=False
        ):

and in the body added the cbar, vmin, and vmax arg:
if labels:
    sns.heatmap(corr_mat, mask=mask, cmap=cmap, ax=ax0, cbar=cbar,
                annot=True, annot_kws={'size': fontsize - 2},
                vmin=-1, vmax=1)
else:
    sns.heatmap(corr_mat, mask=mask, cmap=cmap, ax=ax0, cbar=cbar,
                vmin=-1, vmax=1)

Would love to get this committed as I'm currently writing a book and would like to include this library in the book ;)

😅 I'll look into it this weekend.

This is fixed on master. Some behavioral changes to heatmap:

The colorscale is now pinned to -1, 1 by default. This aids interpretability, since the color scale will now always be smoothed out and pinned. However, it also decreases the effective color range in the relevant part of your data (e.g. your nullity is unlikely to scale fully between those values). The new vmin and vmax parameters can be used to control this normalization.
The colorbar is now displayed by default. You can omit it with cbar=False.

It may be a few more days before I release to PyPI.


fix a dtype mismatch when use mps

Test passed?

As i tried this fixed it came to another issue :
Mismatch is fixed though but still we cannot use mps with this
python3 -m tools.api_client \     --text "I don't really care what you call me." Request failed with status code 500 {'statusCode': 500, 'message': 'Output channels > 65536 not supported at the MPS device. As a temporary fix, you can set the environment variablePYTORCH_ENABLE_MPS_FALLBACK=1to use the CPU as a fallback for this op. WARNING: this will be slower than running natively on MPS.', 'error': 'Internal Server Error'}


Letter Template Inconsistencies

Describe the bug
Upon letter generation in Hawaii, there are lots of inconsistencies between the different templates

Spacing between header + letter content
Page footers show Page of
No page footers
Header logo on 2nd page of letter
No header logo on 2nd page of letter
A page just for one line/signature
Complainant info + Page on top left of letter

https://github.com/PublicDataWorks/complaint-manager/assets/134408135/9bbfdea5-5e7b-4345-98db-75d8b501d41f

Steps to reproduce
List any steps to reproduce behavior

Create a new case
On incident details page, click Generate Letter and generate all 17 letters

Expected behavior
All letter templates should be consistent

Agree it's pretty low since clients have not brought this up as bugs to fix but we will need to KIV, so moving to icebox for now


CDK-188: Add inputformat format

This is based on CDK-861 until it is merged to avoid unrelated test failures.

Overall it looks good, I had a few questions listed above.
My only major issue is that this further entangles the core module with MapReduce classes which would make it that much harder to create a core that doesn't include all of those dependencies.

About the mapreduce classes: Core is already fairly entangled and this would be moved along with the other FS code to a module outside of the API that can depend on MR. There isn't a great way to get around that and provide real access to data locality information (we could plan our own splits, but letting the delegate formats do it is better).
I think the larger issue for separating the API from MR is to fix the MR implementation to avoid delegating to an InputFormat, more like Crunch does.

That's fair. If we can get to a clean API module and eventually a free core/common module that'd be good.

We discussed this feature in https://issues.cloudera.org/browse/CDK-188, and there was some concern about treating MR input and output formats as Kite formats, since they aren't necessarily readable by other tools (like Hive). Maybe that objection is addressed by only supporting the read path, as you do here.

I agree it's awkward as a "format", but I view it as part of a more general import feature to read raw data. This is similar to the CSV support we added which also isn't readable by Hive.

I think this is a must-do for data integration. There needs to be some way of reading things like Sequence files and using Kite on that data. This in particular isn't ideal because we get some unknown object from the InputFormat, but as you noted, I think it is okay as a read-only format.

I think this is ready to merge after fixing the issue that Joey found. Thanks for the reviews!

Can we at least change "informat-import"? I would vote for "inputformat-import".

@tomwhite, I've updated the command name to inputformat-import. Anything else we should change before this goes in?

+1 from me.


Provide Circe codec instances of common Indigo types

Perhaps they could be generated and live in the JSON module to keep all the circe stuff together? Be nice if you could do:
import indigo.circe.given (you have to import given's explicitly)

Reference from a recent project:
import indigo.*

import io.circe.*
import io.circe.syntax.*
import io.circe.generic.semiauto.*

object CirceIndigoInstances:

  given Decoder[Size] = deriveDecoder
  given Encoder[Size] = deriveEncoder

  given Decoder[Point] = deriveDecoder
  given Encoder[Point] = deriveEncoder

  given Decoder[Rectangle] = deriveDecoder
  given Encoder[Rectangle] = deriveEncoder

  given [A](using Decoder[A]): Decoder[Batch[A]] =
    Decoder.instance(_.as[List[A]].map(Batch.fromList))

  given [A](using Encoder[A]): Encoder[Batch[A]] =
    Encoder.instance(_.toList.asJson)


Bugfix: docker oneshot doesn't appear to be working

Describe the bug
When using docker, oneshot seems broken.
Steps to reproduce
Open the container CLI, run archivebox oneshot "http://website", or run it using docker-compose run archivebox oneshot ... and it outputs:
    ! Failed to archive link: TypeError: __init__() missing 4 required positional arguments: 'timestamp', 'url', 'title', and 'tags'
Traceback (most recent call last):
  File "/usr/local/bin/archivebox", line 33, in <module>
    sys.exit(load_entry_point('archivebox', 'console_scripts', 'archivebox')())
  File "/app/archivebox/cli/__init__.py", line 122, in main
    run_subcommand(
  File "/app/archivebox/cli/__init__.py", line 62, in run_subcommand
    module.main(args=subcommand_args, stdin=stdin, pwd=pwd)    # type: ignore
  File "/app/archivebox/cli/archivebox_oneshot.py", line 55, in main
    oneshot(
  File "/app/archivebox/util.py", line 112, in typechecked_function
    return func(*args, **kwargs)
  File "/app/archivebox/main.py", line 506, in oneshot
    archive_link(oneshot_link[0], out_dir=out_dir, methods=methods, skip_index=True)
  File "/app/archivebox/util.py", line 112, in typechecked_function
    return func(*args, **kwargs)
  File "/app/archivebox/extractors/__init__.py", line 82, in archive_link
    link = load_link_details(link, out_dir=out_dir)
  File "/app/archivebox/util.py", line 112, in typechecked_function
    return func(*args, **kwargs)
  File "/app/archivebox/index/__init__.py", line 363, in load_link_details
    existing_link = parse_json_link_details(out_dir)
  File "/app/archivebox/util.py", line 112, in typechecked_function
    return func(*args, **kwargs)
  File "/app/archivebox/index/json.py", line 111, in parse_json_link_details
    return Link.from_json(link_json, guess)
  File "/app/archivebox/index/schema.py", line 234, in from_json
    return cls(**info)
TypeError: __init__() missing 4 required positional arguments: 'timestamp', 'url', 'title', and 'tags'

Software versions
ArchiveBox v0.4.21 using the latest docker-compose.yml

Yes. It seems like it is not working as expected. I will fix it asap.

@poblabs I created a PR that should help with the command. Things to note:

The oneshot command should be ran in an empty folder, or it will try to download the archive in the current folder
You can use the --out-dir option to better manage where to send the output of the command

Thanks! I downloaded your branch with the fix to test it, but docker isn't building. I'll keep trying it

Oh, master is not working at the moment. https://github.com/pirate/ArchiveBox/pull/482 after this PR is merged, I will rebase and it should start working again.

@poblabs please check again with master. The fix has been merged and should work properly now. Let me know if it still fails to reopen this.


Don't hardcode x86-64 architecture, to allow aarch64 builds …

…(for the new graviton T4g and M6g instance types in particular).

That would indeed be helpful. I'm not sure if this project is active.


添加项目配置

添加了格式化工具和git提交校验

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

1 of 1 (100.0%)  changed or added relevant line in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 61.765%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
21

Relevant Lines:
34

💛  - Coveralls


Bookmark files for quick access

During workflow i have to have toggle few files for certain things. But if sometimes i do close them i have to search through the tree to reopen them. Also sometimes you can't keep tabs open just to keep it clean. But if there was an option to bookmark files for quick access, it would've been a useful feature.

Please search for bookmark extensions in the marketplace.


Speed up val2mask

Hi,
I think one quick and easy way to speed up the val2mask.R util would be to build the index of values to search for as you do already in the loop as seen below here:
outImgVec               <- refImg[refImg<Inf] * 0 valIdx                  <- c() for (maskVal in maskVals) {    bounds               <- as.numeric(unlist(strsplit(maskVal,':')))    valIdx               <- refImg >= bounds[1] & refImg <= bounds[length(bounds)]    outImgVec[valIdx]    <- 1 }
But take the conversion outside of the loop, something like below:
`
outImgVec               <- refImg[refImg<Inf] * 0
valIdx                  <- NULL
boundsAll <- NULL
for (maskVal in maskVals) {
bounds               <- as.numeric(unlist(strsplit(maskVal,':')))
boundsAll <- append(boundsAll, bounds)
}
This is untested code
index <- which(refImg %in% boundsAll)
outImgVec[index] <- 1
`
Haven't tested this yet, but growing very frustrated with the speed of this script =/.
I'll assign this to my self but wanted to write this down somewhere.

hmm, this was formatted terribly...


Ensure that pt_formatted_text always returns a FormattedText inst…

…ance.
This is important, so that print_formatted_text recognises output of
to_formatted_text correctly.

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@f5cfb7c). Click here to learn what that means.
The diff coverage is 92.3%.

@@            Coverage Diff            @@
##             master     #760   +/-   ##
=========================================
  Coverage          ?   70.68%
=========================================
  Files             ?      140
  Lines             ?    13231
  Branches          ?        0
=========================================
  Hits              ?     9352
  Misses            ?     3879
  Partials          ?        0

Impacted Files
Coverage Δ

tests/test_print_formatted_text.py
84.21% <100%> (ø)

prompt_toolkit/formatted_text/html.py
96.72% <100%> (ø)

tests/test_formatted_text.py
100% <100%> (ø)

prompt_toolkit/formatted_text/ansi.py
71.42% <100%> (ø)

prompt_toolkit/shortcuts/utils.py
75.38% <100%> (ø)

prompt_toolkit/formatted_text/pygments.py
100% <100%> (ø)

prompt_toolkit/formatted_text/base.py
86.56% <75%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f5cfb7c...2aa85ff. Read the comment docs.


Added Turkish language support

This PR adds Turkish language support to the Riverpod documentation. The following changes were made:

Added a new tr folder in the website/i18n directory.
Updated the docusaurus.config.js file to include Turkish in the list of supported languages.

I see that this pr ve been staying here. Even though i didn't understand whats the problem i ve put lots of effort into translating the whole documentation. I would love to fix/add stuff if somebody would assist me with what im missing.

Sorry I'm in holidays at the moment. It's a bit big as a PR, so it'll wait until I'm back

No problem at all. Enjoy your holiday!


I wanted to take advantage of tflite in vitis-ai, but I ran into a bit of a problem with the official steps given.

I'm running Edge Deployment (aarch64 host) A2 Compile libvitisai_delegate.so .When I run to 'bash run.sh',I always fail.Have anyone tried this?

can you please give me more detailed information on this problem?

what is your board;
what is your version;
what is the "A2 compile" ?
what is the full path of your test program?

Closing since no activity for more than 3 months, please reopen if you still have any questions, thanks


Added arguments usage example

You could find something similar on the code but it was not clear. I had to use this and maybe it could help someone.

Hi there. It's something really useful for sure. Just need some fixes before merging.

fix'd.
create more issues, i'm willing to help :)

Lovely. Thank you.
Also sorry for the delay


Shortcuts for WET

Hi Lusito,
are there any shortcuts for WET? I'm especially interested in a shortcut that switches to the next string saving the previous string.
Thanks.
Michael

Normal keyboard navigation shortcuts exist like in any other browser.
In this case, pressing tab a couple of times would do the trick (and shift + tab for the reverse direction).
Other than that, I'm afraid there is nothing like it at the moment.
It shouldn't be too hard to create something if tab does not suffice, but I'm currently busy moving to a new job and city.

Ok. It's a pity. I know that I can use the tab key a couple of times but it is somewhat inconvenient. I thought of "Strg+Enter" as on Pontoon. Nevertheless, thank you for creating WET.

Just for reference, when I get to it:
Ctrl+Enter = next line, but same language?

Yes, it should go to the next string and the just translated string should be saved at the time. For now, I have to click in the next string and I must not forget to save the translated strings that I don't loose them by mischance.

Forgot to mention: These shortcuts exist in addition to normal browser keyboard shortcuts:
ctrl + z / y (undo/redo)
ctrl + s (save)
What do you mean by save? Save to files? => press ctrl + s

There is no way to just save one language or even just one string. At the moment when you press save, all languages are persisted in the json files.

OK. I hoped that there is a more comfortable way. It would make the translation process smoother. One shortcut for two actions at the same time.

I'll consider it for future versions.


Add examples from the tutorial

I'm currently going through the tutorial and coding up these as examples. I think this might benefit whoever uses this library, so I'll eventually open a pull request once I'm done with this.

@ChristopherRabotin  Look forward to a PR!


fix(render): add condition over shrink dir to prevent some crashes

this cover crashes in case at least one vertex of a face is on its center of gravity

Newb question but will this just panic at the unwrap if not returning the Ok(_) variant?

Yes, it used to panic when not Ok, now it is just skipped.

Newb question but will this just panic at the unwrap if not returning the Ok(_) variant?

It won't, this would be somewhat equivalent to:
// res: Result<T, E>
if res.is_ok() {
    let val = res.unwrap(); // val: T
    // ...
}


Single confirmation of exclusion for multiple hosts

I ran the following command: docker-machine rm mh-keystore swarm-master swarm-node-01 swarm-node-02
But it asked to confirm the exclusion for each machine:
Do you really want to remove "mh-keystore"? (y/n): y
(mh-keystore) Stopping VM...
Successfully removed mh-keystore
Do you really want to remove "swarm-master"? (y/n): y
(swarm-master) Stopping VM...
Successfully removed swarm-master
Do you really want to remove "swarm-node-01"? (y/n): y
(swarm-node-01) Stopping VM...
Successfully removed swarm-node-01
Do you really want to remove "swarm-node-02"? (y/n): y
(swarm-node-02) Stopping VM...
Successfully removed swarm-node-02

For standard purpose it should act as docker-compose: docker-compose rm
Going to remove swarm_wildfly_1, db, modcluster
Are you sure? [yN] y
Removing swarm_wildfly_1 ... done
Removing db ... done
Removing modcluster ... done

Hi @rafabene, yes, I think you are right. Would you like to contribute?

I would love it but I'm still learning Golang to be a more active contributor.

That fix is a good opportunity to practice you know!

Exactly! :+1:
So challenge accepted :D (and since it's not a priority bug fix)
I'll be fully busy until March 2016 but after then I would love to contribute.
Thanks @dgageot


Ordinary

This is a pedantic submission of some documentation and UI wording changes, based on the language guidance at https://docs.wagtail.io/en/latest/contributing/general_guidelines.html which prefers British English for user-facing text. Regular in British English doesn't have the US English meaning of usual or ordinary.
The page aliases UI uses the term "regular page" to refer to a page that is not an alias. This MR changes the terminology to refer to ordinary pages. Elsewhere  I have also replaced some uses of regular with normal, basic, ordinary, conventional, or standard, depending on the context.
One use of regularly has been changed with often, based more on context than on a dialect difference.
The guidance on language is slightly more ambiguous about dialect choice for text that is not user facing and not a built-in language keyword. I've been conservative, so this MR does not include changes to:

comments and docstrings in Python code
strings in test suites
Python variable names
text in release notes, as they might be considered immutable

Thanks @nimasmi!


Bq-ingest failure while processing 16 backfilled releases and 1 release from 2024-12-15

It succeeded on the first 16 ones. Failed partway through the 12-15 release.
Job began at 2:45AM EST Dec 17 2024.
Failed at 11:24AM EST Dec 17 2024
clinvar-bq-ingest-5tczr | Dec 17, 2024, 2:45:05 AM | 1/1 completed | Dec 17, 2024, 11:24:41 AM
Will investigate why the job succeeded with exit code 0 even though there was an exception.

Google Cloud Logs:
downloaded-logs-20241218-144327.json

Looks like just an intermittent network error, but the next time the bq-ingest job ran ~15 minutes later it should have re-run this one

Deleted the record for the partial bq-ingest run and deleted the 2024-12-15 v2_1_0 dataset in bigquery so it'd just run again
delete from `clingen-dev.clinvar_ingest.processing_history`
where file_type = 'bq'
 and xml_release_date = '2024-12-15'


Bug fixes and package migration

Packages moved to packagecloud.io

Fixes for bro and snort integration via the unified2 file reader

Bro version pinned to 2.4.1 until the beta solidifies

Fixes to make GeoIP work reliably

Updated the es_cleanup.sh so it closes indexes after 15 days and deletes them after 60. (As an example)

Tested in ESX, bare metal, and Vagrant.


GSD Request

--- GSD JSON ---
{
  "vendor_name": "Linux",
  "product_name": "Kernel",
  "product_version": "versions from  to before v5.15.25",
  "vulnerability_type": "unspecified",
  "affected_component": "unspecified",
  "attack_vector": "unspecified",
  "impact": "unspecified",
  "credit": "",
  "references": [
    "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=23b2a25382400168427ea278f3d8bf4ecfd333bf"
  ],
  "extended_references": [
    {
      "type": "commit",
      "value": "23b2a25382400168427ea278f3d8bf4ecfd333bf",
      "note": "fixed"
    }
  ],
  "reporter": "joshbressers",
  "reporter_id": 1692786,
  "notes": "",
  "description": "net: ieee802154: at86rf230: Stop leaking skb's\n\nThis is an automated ID intended to aid in discovery of potential security vulnerabilities. The actual impact and attack plausibility have not yet been proven.\nThis ID is fixed in Linux Kernel version v5.15.25 by commit 23b2a25382400168427ea278f3d8bf4ecfd333bf. For more details please see the references link."
}
--- GSD JSON ---

This issue has been assigned GSD-2022-1000450


Add documentation as vim help

I really missed the vim help so I added it.
Documentaion is taken from http://projects.mikewest.org/vimroom/

You're very welcome :)


Added tutorial in Grid Paint Activity.

It don't work. The screen become gray but the popup doesn't appears.

It don't work. The screen become gray but the popup doesn't appears.

This is the problem emerging with me ... I did all the steps correctly but I am not getting why the popups doesn’t appear... They work if we click right arrow from the keyboard but the css is not showing up... I don’t know why... Can you suggest me a way to solve this issue...

Not sure it's related but webL10n is not initialized in this activity.

Not sure it's related but webL10n is not initialized in this activity.

Its not related to wbL10n.....

It's related to webL10n, when I'm removing all call to l10n.get method in your file, the popup is displayed...

It's related to webL10n, when I'm removing all call to l10n.get method in your file, the popup is displayed...

Oh yes.. Thanks. I will do the necessary changes now..

Added Popups and fixed the Issue.
Thanks.

You must not hard code the text for the tutorial, you should make localization work!

I've also encountered the problem with localization not being initialized at all. Anyway fixing it is really simple. You need to go back to step 5 of the tutorial and add some code in activity.js and index.html
I hope I could help you a little :)

I've also encountered the problem with localization not being initialized at all. Anyway fixing it is really simple. You need to go back to step 5 of the tutorial and add some code in activity.js and index.html
I hope I could help you a little :)

Thanks man...

You must not hard code the text for the tutorial, you should make localization work!

Ok sir I will add localization

Added Localization but i have not translated for other languages..

It's better.
Two remarks:

The webL10n is not initialized so the tutorial language is always the language of the browser. You should initialized by using code mentioned here.
Will be nice to display a different tutorial in list mode and in detailed mode. You could differentiate it using the global variable mode. See here.

It's better.
Two remarks:

The webL10n is not initialized so the tutorial language is always the language of the browser. You should initialized by using code mentioned here.
Will be nice to display a different tutorial in list mode and in detailed mode. You could differentiate it using the global variable mode. See here.

I will do the changes right away..

Added different tutorial for different modes and fixed Localization.

Nice.
One more issue, a horizontal scrollbar appears when the tutorial is launched:

Changed Placement.. to center

@llaske Is this approach fine.


feat: pixi

we can draw some graphics with it.
https://pixijs.io/pixi-react/
https://pixijs.com/

/close


SnatypeClient logs not forwarded to sbt client

Describe the bug
When I trigger sonatypeBundleRelease from an sbt client shell (using sbtn) I cannot see the logs from SonatypeClient. All I see is:
sbt:root> sonatypeBundleRelease
[info] Preparing a new staging repository for [sbt-sonatype] root 3.0.4
[info] Updating sonatypePublishTo settings...
[info] Welcome to scala-debug-adapter 3.0.4
sbt:root>

How to reproduce the issue
Describe how to reproduce the issue and show the current configurations.

sbt version: 1.8.0
sbt-sonatype version: 3.9.13

$ sbtn
> sonatypeBundleRelease
...

Additional context
I see that you use wvlet.log here and that's probably why the logs are not forwarded to sbt clients. Is it possible to configure wvlet.log to forward the logs to the sbt logger in state.streams.log?

Thanks. I also noticed this issue. airframe-log sends log messages to the standard java.util.logging and stderr, but some sbt-plugins (e.g., sbt-release is one of the examples) may set an unexpected log output target.
It would be possible to add a log handler to redirect the default logger output to state.streams.log, but as long as there are other plugins that configure java.util.logging, the same problem can still happen, but the impact can be reduced.
Just in case, could you try using the last sbt-sonatype 3.9.17?

Confirmed the reproduction. Let me handle this issue in airframe-log https://github.com/wvlet/airframe/issues/2777

sbt-sonatype 3.9.18 has been released. This will fix the issue

Confirmed the fix is working well with sbtn

Thanks a lot!


[Event Request] Codeunit "Carry Out Action" - OnInsertProdOrderLineOnBeforeValidateUnitCost

Hi there,
I would like to ask you to add the following publisher to the procedure InsertProdOrderLine() to Codeunit 99000813 (Carry Out Action):
 ProdOrderLine."Inventory Posting Group" := Item."Inventory Posting Group";
 //NEW CODE - BEGIN
 OnInsertProdOrderLineOnBeforeValidateUnitCost(ReqLine, ProdOrde, ProdOrderLiner, Item);
 //NEW CODE - END
 ProdOrderLine.Validate("Unit Cost", ReqLine."Unit Cost");
 ProdOrderLine."Routing No." := ReqLine."Routing No.";

    [IntegrationEvent(false, false)]
    local procedure OnInsertProdOrderLineOnBeforeValidateUnitCost(var ReqLine: Record "Requisition Line"; ProdOrder: Record "Production Order"; var ProdOrderLine: Record "Prod. Order Line"; Item: Record Item)
    begin
    end;

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.
Build ID: 11863.


Update test deploy example with orb-tools/pack

Changes copied from #234

Your development orb(s) have been published. It will expire in 30 days.
You can preview what this will look like on the CircleCI Orb Registry at the following link(s):
https://circleci.com/developer/orbs/orb/circleci/orb-tools?version=dev:b259e20cc6ecf8a4e9a95bbc9412ff62b2b00834
https://circleci.com/developer/orbs/orb/circleci/orb-tools?version=dev:alpha


[thread-cert] add case 8.1.6 using pktverify

Add provisionURL set and check
Check all JOIN_ENT & JOIN_FIN

@FiveDimensions , please help resolve conflicts.

@FiveDimensions , the packet verification tests seem to be failing. Can you take a look?

@jwhui , sure, I am checking the packet files

@jwhui , due to choose the wrong thread_meshcop.tlv.steering_data type in conflict resolve, fixed now


Promote user "tari" to traige permission level

Background
The TSC Project Roles defines the main roles for the Zephyr Project, including
Maintainer, Collaborator, and Contributor.
By default anyone that contributes code or documentation is a Contributor, but
with the lowest GitHub Permission Level of Read. For example, Contributors
with Read permission do not have the permission to add reviewers to a pull
request.
Use this template to nominate a GitHub user for an elevated permission level in
the Contributor role.
Nomination
GitHub User
Provide the following information about the GitHub user:

Peter Marheine
tari
Google

Supporting Documents
Zephyr authored PRs
Zephyr reviewed PRS

@tari for visibility.

Added as a contributor with triage permission.

Thanks for the quick response.  However, I am still unable to add @tari as a reviewer on an existing PR.  Are there other permissions that must be granted?

Are there other permissions that must be granted?

It seems @tari has not accepted the invitation yet.

Are there other permissions that must be granted?

It seems @tari has not accepted the invitation yet.

Ok, thank you for the clarification.


Encoding parameter only for reading, not for writing

I think the encoding parameter should only be used to read in files. Anything produced should be in utf-8 regardless.
I have to set the parameter because my bank returns files in windows-1252 but I have zero interest in producing a file in that same format. I think that may be the case for more users.
Alternatively, there could be a new parameter to override the output encoding.

Good point! Not sure when I'll have the time to address this. PRs welcome of course :).


Jupyterlab support of spyder.

Hi,
Can spyder support Jupyterlab now?
Regards

@hongyi-zhao what do you mean by

Can spyder support Jupyterlab now?
?

If you are asking about notebook support you, please refer to https://github.com/spyder-ide/spyder-notebook/

I mean the supporting for https://github.com/jupyterlab/jupyterlab from within spyder.
Regards

I mean the supporting for https://github.com/jupyterlab/jupyterlab from within spyder.

No, we will support the Notebook component of jupyterlab, not the entire jupyterlab. Fot that you should use JupyterLab itself 🤷

Agreed. If you want JupyterLab, then please use JupyterLab. It doesn't make sense to me to embed JupyterLab inside Spyder just for the sake of embedding it.


Update Iptables from 1.6.2 to 1.8.x in flatcar OS version 2605.8.0

Current situation
Currently, we are running Kubernetes clusters on flatcar OS in production.
Kubernetes 1.16.x onwards updated their internal iptables version within the hyperkube images to 1.8.3(nft) . However the iptables version on flatcar OS remains 1.6.2.
Kube-proxy which is a component of kubernetes utilizes iptables internally for all pod routing and the version of iptables they are using 1.8.3( using --random-fully as a mandatory flag)  which is incompatible with the host OS(flatcar). This is causing the iptables rules to break on the host.
Impact
We are unable to upgrade kubernetes running on flatcar OS because of iptables incompatible version with Kubernetes hyperkube image 1.16.x onwards.
Ideal future situation
Update iptables to 1.8.3 on flatcar OS.

This shipped in stable 3033.2.0 (https://www.flatcar.org/releases/#release-3033.2.0), please try that version.

Thanks @jepio.  I will test this flatcar version in coming weeks and get back to you if I have any further questions/issues.


--dep in directory with spaces

If you include a directory with spaces in it, then the dependencies generated by --dep full are not quoted, and thus it generates a broken Makefile.
To reproduce:
mkdir "a b"
cd "a b"

cat > A.fst <<EOF
module A
EOF

cd ..

cat > B.fst <<EOF
module B
open A
EOF

fstar.exe --include "a b" --dep full B.fst

One of the parts causing the issue:
/tmp/asd/a b/A.fst.checked: /tmp/asd/a b/A.fst \
	/home/jay/everest/FStar/ulib/FStar.Pervasives.fst.checked\
	/home/jay/everest/FStar/ulib/prims.fst.checked

Expected output:
/tmp/asd/a\ b/A.fst.checked: /tmp/asd/a\ b/A.fst \
	/home/jay/everest/FStar/ulib/FStar.Pervasives.fst.checked\
	/home/jay/everest/FStar/ulib/prims.fst.checked

Basically, all spaces in filenames need to be replaced with \  instead. It does indeed seem to do this in one place (in particular, the first rule), but then stops doing so.
Not a very high priority issue, but I thought it best to have it documented.

Sorry, I should have reused this issue instead of making a new one. Here's the remaining problem:

Directories that contain a colon :, or a newline, or backslashes, will make --dep make and --dep full spit out garbage. Example with a colon:
$ ls 'colon:dir'
A.fst  B.fst
$ ./bin/fstar.exe --dep make colon\:dir/A.fst
....
colon:dir/A.fst: /home/guido/r/fstar/master/ulib/FStar.Pervasives.fsti /home/guido/r/fstar/master/ulib/prims.fst
...

This will make make complain with:
a.mk:7: *** target pattern contains no '%'.  Stop.

and the colon should be escaped for it to work.
Very low prio, as these are insane choices for directory names, and I hope no one actually names them like that. Related: #1926, #1587.


Poster method doesnt work

Poster method returns nil for every movie

The Poster method is still returning nil for every movie.

Fixed it.
See: https://github.com/ariejan/imdb/pull/89


Title: Addition of a maze solver

Initiative (Required)
GSSoC 2024 Extd 🚀
Is your proposal related to a problem? Please describe.
I propose adding a basic AI project, i.e. a maze solver.
It will take input in .txt file and produce the output in .png file. It uses both bfs as well as dfs algorithms for an optimized solution. It will be a great project for a beginner level developer.
Add any other context or screenshots about the proposal request here.
No response

I am the mentor of this project.
@Mahak-Agrawal-304 Hello
I appreciate your pull issue , can you describe little bit and give working video, screenshots after update the output!!

Hello @SakiraAli1115 , sure I'll share how inputs are taken and output is displayed.
The basic logic behind this is the implementation of queues and stacks. With these implementations, we traverse the maze until we find the exit or target value. As for the output, the image is generated after the solution is found and automatically saved on your device.
A demo input in the form of .txt file:

The solution in the form of form of .png file for the above demo maze:

Green node: Starting point
Red node: Exit/Target point
White node: All possible traversal points
Light red node: Traversed by the program but was not the correct path
(Cannot share working demo due to size issues)

@Mahak-Agrawal-304 Is this simple python program ?

Yes @Kushal997-das , it's a beginner project I made when I began learning Artificial Intelligence

Hi @Mahak-Agrawal-304,
Please ensure that this is developed as a desktop application rather than a standard program. It should have the appearance of a complete project. I’m assigning this to you for now and will check in when you open the PR.
Thank you!


Migration that sets unique nullable field to non-nullable fails

I've discovered this issue in a project where we are using djangorestframework-api-key.
This package has a migration that creates a CharField with unisque=True and null=True, the populates data for that field and afterwards it sets null=False. This is a pretty common scenario when you need to add non-nullable unique field to a pre-existing model.
Migrations of this kind fail with the following error:
42S11] [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]The operation failed because an index or statistics with name 'testapp_testalternullableinuniquefield_a_67b9d427_uniq' already exists on table 'testapp_testalternullableinuniquefield'. (1913) (SQLExecDirectW)
Oddly enough, this only happens the first time django tries to run the migration, but the second time it passes. It is not yet clear to me whether the indexes and constraints are in place after the migration passes.
I've been able consistently to reproduce the error by creating a migration with the following operations:
    operations = [
        migrations.CreateModel(
            name='TestAlterNullableInUniqueField',
            fields=[
                ('a', models.CharField(max_length=50, null=True, unique=True)),
            ],
        ),
        migrations.AlterField(
            model_name='testalternullableinuniquefield',
            name='a',
            field=models.CharField(max_length=50, unique=True),
        )
    ]

I would like to contribute a PR to fix this issue but at this point it is not clear to me how to generate a unit test for the project as I would like to test the migration itself. Any suggestions?
By the way, this error is preventing us from normally run the tests in our project since test database cannot be created due to this error.

You can take a look at the code in this commit for the example:
https://github.com/ivissani/django-mssql-backend/commit/349e58f8ccf43953f8b5cd7090a63d4f5a5bfbb0

I have created the following PR #77 that attempts to fix this error (successfully). It is still not clear to me whether the solution should be more general than what I implemented.

Same issue here?
https://github.com/ESSolutions/django-mssql-backend/issues/38

Same issue here?
#38

I don't think  they are the same. Here the problem was that some operations created the constraint right away while others deferred that until the end of the migration therefore resulting in an attempt to create the same constraint twice. This does not seem to be the case of the issue you linked as far as I can see.


[CAMEL-20405] renamed classes from *IntegrationTest.java to *IT.java

Description
Several test classes are named *IntegrationTest.java. These have been renamed to *IT.java so that they match the default naming pattern for the failsafe plugin.
Target

[ X] I checked that the commit is targeting the correct branch (note that Camel 3 uses camel-3.x, whereas Camel 4 uses the main branch)

Tracking

[ X] If this is a large change, bug fix, or code improvement, I checked there is a JIRA issue filed for the change (usually before you start working on it).

Apache Camel coding standards and style

[X ] I checked that each commit in the pull request has a meaningful subject line and body.

[X ] I have run mvn clean install -DskipTests locally and I have committed all auto-generated changes

failsafe

Otavio do you mean that some of the component have customized configuration for failsafe-plugin in their pom? It seems so as if you do a git grep failsafe-plugin then there are about 10 components.
But even so what changes are we looking for to do? Would it be if they do some naming matching where they expected names of xxxIntegrationTest style, and this PR renames the names back into xxxIT style.

I think you also need to review the failsafe plugin configuration on the component's pom file.

Otavio do you mean that some of the component have customized configuration for failsafe-plugin in their pom? It seems so as if you do a git grep failsafe-plugin then there are about 10 components.
But even so what changes are we looking for to do? Would it be if they do some naming matching where they expected names of xxxIntegrationTest style, and this PR renames the names back into xxxIT style.

No, I just mean that (I think) the failsafe plugin configuration for camel-safesforce also needs to be updated because we do seems to adjust it for the *IntegrationTest.java pattern.

Otavio do you mean that some of the component have customized configuration for failsafe-plugin in their pom? It seems so as if you do a git grep failsafe-plugin then there are about 10 components.

I don't mean those, but just as a matter of explanation, there are some components that have custom configuration to support a few cases:

Enable testing on "alternative" platforms (ppc64le, s390x, etc). For instance, camel-consul and camel-zookeeper.
Add backwards/forwards compatibility testing. For instance, camel-kafka has extra goals to test the code against Kafka 2.x
Other component-specific testing tweaks/configuration. For instance camel-infinispan and camel-salesforce.

That's why I said mentioned that the configuration for camel-salesforce needed to be adjusted.

@orpiske Good catch on the pom.xml changes! I have updated them.

You cannot run the salesforce integrations tests automatically. You need to setup a real account and more, see the readme file.

@davsclaus There are many integration tests that require external resources, not just the Salesforce tests. This will obviously cause test failures because most people won't have the environments needed to run any of the integration tests.
Should integration tests run as part of mvn verify? They are enabled by default but can be skipped with mvn verify -DskipITs or with a property definition in pom.xml or parent/pom.xml.

I looked at the POM for the Salesforce component and the incremental build log. I don't think the name change was responsible for the build failures. I think that any code change would have caused the integration tests to fail, as they are executed automatically as part of running 'mvn verify' regardless of how they are named, because there is an explicit ' element in the 'failsafe` configuration.

I renamed the Salesforce integration tests to *ManualIT.java to match the naming convention for other Camel manual integration tests. I also disabled execution by mvn verify unless the integration profile was being used.

LGTM


Allow searches for users containing "@" to work

Make it so that an "@" at the beginning of a search is ignored.
Make it so that an "@" in the middle of a search can properly match against an email address.
Fix case-insensitive comparisons for nicknames and emails.
Summary
Improves the filtering which happens in the frontend when searching for users, such that the "@" symbol is handled as expected.
Ticket Link
GH-9755
Checklist
[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

[x] Ran make check-style to check for style errors (required for all pull requests)
[x] Ran make test to ensure unit tests passed
[x] Ran make flow to ensure type checking passed
[x] Added or updated unit tests (required for all new features)

Test Information
This PR was tested on: Safari 12.0, macOS 10.14

Thanks @zeroimpl for the pull request!
Per the CONTRIBUTING.md file displayed when you created this pull request, we need to add you to the list of approved contributors for the Mattermost project.
Please help complete the Mattermost contribution license agreement?
This is a standard procedure for many open source projects. Your form should be processed within 24 hours and reviewers for your pull request will be able to proceed.
Please let us know if you have any questions.
We are very happy to have you join our growing community! If you're not yet a member, please consider joining our Contributors community channel to meet other contributors and discuss new opportunities with the core team.

Note there is an associated backend API change associated with this - https://github.com/mattermost/mattermost-server/pull/9780

@zeroimpl I left some comments on the related web-app PR.

UI Testing is being done on https://github.com/mattermost/mattermost-webapp/pull/2012.  We should be good to continue development review.