Improve Quarto websites projects inspection and generated configuration

Intent
The goal of this PR is to improve the experience of Quarto website projects. Focused in two main points:

pre- and post- scripts are now automatically picked up to be added to the configuration on project inspection.
Users can select _quarto.yml as entrypoint and configuration will be properly generated.

Fixes #2266
Type of Change

[x] Bug Fix

[ ] New Feature

[ ] Breaking Change

[ ] Documentation

[ ] Refactor

[ ] Tooling

Approach
In order to allow points mentioned above, it was necessary to reorganize a bit the code that triggers the quarto inspect command and that, sub sequentially, uses the output to generate the deployment configuration, having specific checks for _quarto.yml picked up as entrypoint and extending the inspect output struct to account for directory scanning scenarios.
User Impact
Users now will:

Get pre- and post- scripts included - if any - to the configuration on Quarto project inspection.
Users can select _quarto.yml as entrypoint and configuration will be properly generated.

Automated Tests
New tests were created to cover this new functionality.
Directions for Reviewers
Try publishing a Quarto website project, even better if it has pre-render and post-render scripts to confirm such files are included in the resulting configuration.
The jumpstart example of a Quarto website can be used as example.
Checklist

[ ] I have updated CHANGELOG.md to cover notable changes.

@marcosnav The code within the PR is looking good, but I started hitting some issues with the quarto examples within connect-content repo, so I would like to have you test each of the quarto content types within the repo, as I think you'll want to make some changes.
I started with connect-content/bundles/quarto-book-none. After needing to fix up the renv issues (which are not within scope of your changes), my deployment (after picking _quarto.yml as the entrypoint) would not render successfully do to missing files. Since you've got changes within this code which adds additional files into the included file list, I think you should update the code to grab every file which quarto inspect returns.
In this case, I saw quarto inspect return:
{
  "quarto": {
    "version": "1.5.57"
  },
  "dir": "/Users/billsager/dev/connect-content/bundles/quarto-book-none",
  "engines": [
    "markdown"
  ],
  "config": {
    "project": {
      "type": "book",
      "lib-dir": "site_libs",
      "output-dir": "_book",
      "render": [
        "index.qmd",
        "intro.qmd",
        "summary.qmd",
        "references.qmd"
      ]
    },
    "book": {
      "title": "quarto-book-none",
      "author": "Jane Doe",
      "date": "2/9/2022",
      "chapters": [
        "index.qmd",
        "intro.qmd",
        "summary.qmd",
        "references.qmd"
      ],
      "render": [
        {
          "type": "chapter",
          "file": "index.qmd",
          "depth": 0
        },
        {
          "type": "chapter",
          "file": "intro.qmd",
          "number": 1,
          "depth": 0
        },
        {
          "type": "chapter",
          "file": "summary.qmd",
          "number": 2,
          "depth": 0
        },
        {
          "type": "chapter",
          "file": "references.qmd",
          "depth": 0
        }
      ]
    },
    "bibliography": "references.bib",
    "format": {
      "html": {
        "theme": "cosmo"
      }
    },
    "language": {},
    "website": {
      "title": "quarto-book-none",
      "sidebar": {
        "title": "quarto-book-none",
        "contents": [
          "index.qmd",
          "intro.qmd",
          "summary.qmd",
          "references.qmd"
        ],
        "style": "floating",
        "tools": []
      },
      "page-navigation": true
    }
  },
  "files": {
    "input": [
      "/Users/billsager/dev/connect-content/bundles/quarto-book-none/index.qmd",
      "/Users/billsager/dev/connect-content/bundles/quarto-book-none/intro.qmd",
      "/Users/billsager/dev/connect-content/bundles/quarto-book-none/summary.qmd",
      "/Users/billsager/dev/connect-content/bundles/quarto-book-none/references.qmd"
    ],
    "resources": [],
    "config": [
      "/Users/billsager/dev/connect-content/bundles/quarto-book-none/_quarto.yml"
    ],
    "configResources": [
      "/Users/billsager/dev/connect-content/bundles/quarto-book-none/references.bib"
    ]
  },
  "fileInformation": {
    "/Users/billsager/dev/connect-content/bundles/quarto-book-none/index.qmd": {
      "includeMap": [],
      "codeCells": []
    },
    "/Users/billsager/dev/connect-content/bundles/quarto-book-none/intro.qmd": {
      "includeMap": [],
      "codeCells": []
    },
    "/Users/billsager/dev/connect-content/bundles/quarto-book-none/summary.qmd": {
      "includeMap": [],
      "codeCells": []
    },
    "/Users/billsager/dev/connect-content/bundles/quarto-book-none/references.qmd": {
      "includeMap": [],
      "codeCells": []
    }
  }
}

The initial error I saw within the render logs on Connect was that references.bib was not included.
If we are going to include all of the referenced files automatically for quarto projects, (at least the ones that quarto inspect does tell us about), then I think you'll want to make sure you parse out all of the locations in which the inspect results will return file references.
Let me know if you agree with me that we should address these additional file references within your PR. Thanks so much!

@sagerb Thanks for trying it out and noticing that. I tested out mainly the jumpstart examples and a couple other projects, I'll take a look to this other bundles to see other cases that we might be missing.
I do think that it makes sense to address some of these other file inclusions on this PR, I'll take another pass to the schema we have and see what we are missing.

Also, Quarto 1.6 was just released and it now supports a new _brand.yml file, I'll take a look and see if this new file is supported by default or if it has it's own new place within the inspect command output.

Do you think we need to first capture the version of quarto and then deserialize into a version specific structure? Or are you seeing that they are simply adding additional fields vs. removing/changing ones?

@sagerb So far I haven't seen the need for versioning the structure, changes are additive. Specifically for the new _brand.yml, it is a special file like _metadata.yml, nothing brakes if it is not present and we just add it to the deployment if it exists, nothing happens if it exists and it is not used.

@sagerb PR ready for another look, with some more adjustments to identify and add project related resources.

NOT YET COMPLETED...
Validation details from projects in repo: connect-content, on main. Using Quarto v 1.6.39 on Mac, but v 1.5.57 on Connect Server.

NOTE: should we call out the mismatch, like we do with Python?

bundles/quarto-doc-none

success (selected bundles/quarto-doc-none/quarto-doc-none.qmd)

bundles/quarto-multidoc-proj-none

????? Always rendered to be document 1, even though have tried three different entrypoints: bundles/quarto-multidoc-proj-none/_quarto.yml, bundles/quarto-multidoc-proj-none/document1.qmd and bundles/quarto-multidoc-proj-none/document2.qmd.
Would have expected w/ entrypoint selected as document2.qmd, that we would render and show document2?

bundles/quarto-proj-none

success Selected bundles/quarto-proj-none/_quarto.yml and updated existing deployment to bundles/quarto-proj-none/quarto-proj-none.qmd.

bundles/quarto-proj-none-ojs

success Selected bundles/quarto-proj-none-ojs/quarto-proj-none-ojs.qmd and updated existing deployment to bundles/quarto-proj-none-ojs/_quarto.yml

bundles/quarto-proj-py

success Selected bundles/quarto-proj-py/quarto-proj-py.qmd and updated to existing deployment to bundles/quarto-proj-py/_quarto.yml

bundles/quarto-proj-r

Repo has existing renv directory populated. Was unable to deploy, got error:

Could not scan R packages from lockfile: renv.lock, R6: package not found in current libPaths; consider running renv::restore() to populate the renv library
NOTE: This seems to be another example of how we do require a working R environment to deploy?
- success Selected bundles/quarto-proj-r/_quarto.yml

bundles/quarto-proj-r-py

success Selected bundles/quarto-proj-r-py/quarto-proj-r-py.qmd

Summary of validation findings:
Problem: Not detecting styles.css and adding it to the file list, if do not pick _quarto.yml as entrypoint file. See finding #9 above, to see how quarto inspect about.qmd did return a reference to the file.
Questions

Are findings within #2 expected?
Multiple examples of how we need to either re-run renv::restore() or renv::snapshot() and restore packages (or not) to allow proper package discovery. Doesn't this conflict with the objective of not needing to be able to execute code?
Did we want to update the entrypoint selection dialog to allow selection of directories (to allow Quarto directory/projects?)
Should have quarto inspection found the dependency of deps.R? (See 11, 13, 15, and 16)
Should we be warning when local quarto version doesn't match with what is on Connect Server?

@marcosnav See my findings above. I think you should address the problem item above, as technically, it seems within reach. The other items I placed into Questions, just need some group discussion or actions.

Multiple examples of how we need to either re-run renv::restore() or renv::snapshot() and restore packages (or not) to allow proper package discovery. Doesn't this conflict with the objective of not needing to be able to execute code?

This is exactly the place where we are stuck currently given the ecosystem of manifest.json + needing more information than the renv.lock file has to send up to connect. We would like to live in a Connect world where this isn't a requirement, but today it is. So today we do need to effectively be in a place (well the renv library does) where one could execute the code. But this isn't transitive: just because today we do effectively need to be in that state, that doesn't mean that executing the code is a requirement we should always assert/strive for in other situations or in the future if the manifest.json/renv.lock issue is solved.

NOTE: This seems to be another example of how we do require a working R environment to deploy? Should we detect this by running renv::status() and parsing the output, so we can show a quicker error?

I couldn't quite tell from the formatting if this was a quote of something somewhere or what. But, yes this sounds like a good idea. We almost certainly should not do it in this PR, but using renv::status to diagnose the renv setup (and then "just" passing whatever renv says to the user to resolve) would be an excellent way to get folks into an renv state that works for publishing.


.NET Core 2 does not work....

Can't find the issue... I've installed dotnet core 2 SDK and Runtime from here https://github.com/dotnet/core/blob/master/release-notes/download-archives/2.0.0-download.md
Restarted my PC (Windows 10 Pro x64 (Latest Version))
Ran my Visual Studio 2017 Enterprise 15.3.3.
Started new project (Console App (.NET Core)) and then this happened...

Could someone please explain me what I did wrong??

That all seems correct to me. Can you try opening a command window and typing dotnet run in the project folder? That would help determine if its something wrong with the SDK install or something wrong with Visual Studio.

I have not been able to reproduce this issue. I removed all core frameworks and put just 2 into my VM and nothing seems to be wrong.
I have seen VS behave like this before though with various other things and it usually has to do with a number of language services crashing or locking up. Have you tried a hard reset of your computer?

@Petermarcu I've just tried with CMD and here's what I got...

@itanex no I have not tried hard reset my computer and it would be nice to stay that way...

Can you create a diag log dotnet build /flp:v=diag? It will put an msbuild.log file next to your project. Can you zip that up with your project and share it? Something unexpected is going on and that should give us the info we need to understand it.

So i made diag log, zipped my entire project and added it to Google Drive. Here is the link:
https://drive.google.com/drive/folders/0Byw8_OcVom5XWFl4SXNJN1hOYXc?usp=sharing
Hope you'll find the problem, because I'm desperete...

@ericstj to also take a look to see if he can see what's wrong.

Adding @nguerrera and @dsplaisted as well.

Something is wrong with the assets file.  Perhaps your NuGetFallBack folder (C:\Program Files\dotnet\sdk\NuGetFallbackFolder) is corrupt?  If you are able to reproduce this perhaps @emgarten could find out why NuGet is returning partial results.

Now, if I look at the log file, I see RAR is finding System.Runtime.dll inside the package.
Unified Dependency "System.Runtime, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a". (TaskId:17)
...
Resolved file path is "C:\Program Files\dotnet\sdk\NuGetFallbackFolder\microsoft.netcore.app\2.0.0\ref\netcoreapp2.0\System.Runtime.dll". (TaskId:17)
Reference found at search path location "C:\Program Files\dotnet\sdk\NuGetFallbackFolder\microsoft.netcore.app\2.0.0\ref\netcoreapp2.0". (TaskId:17)
...
                     Output Item(s):
                         ReferenceDependencyPaths=
                             C:\Program Files\dotnet\sdk\NuGetFallbackFolder\microsoft.netcore.app\2.0.0\ref\netcoreapp2.0\System.Runtime.dll

This makes me think it is unlikely that the package folder is incomplete, since MSBuild is able to see the files in there, just not NuGet.  Perhaps it has to do with a difference between the nupkg file and the expanded file layout (nuget may be looking at the nupkg).  I imagine this could happen if the setup operation that creates the fallback folder was interrupted.  That's actually consistent with an error message I'm seeing in your screenshot (which @Petermarcu pointed out to me) : Permissions denied to modify the ...  Do you think you can rename C:\Program Files\dotnet\sdk\NuGetFallbackFolder then run dotnet new (or restore, build, etc) command from an elevated command prompt to see if that fixes the issue?  /cc @wli3
If so, then I think a fix here to make this a less broken error would be to make the NuGetFallbackFolder more transactional: extract to temp folder then rename on completion.

@NikasZalias try removing all obj folders also and restoring again, this will force restore to read the package directory again for the file list.

I had the same issue. A new class library build failed. The issue was fixed by renaming C:\Program Files\dotnet\sdk\NuGetFallbackFolder and deleting obj/bin folders for the project.

I had the same issue, a new console application build failed. I've tried the fix that @hakonrossebo did, but it didn't work for me. I've also tried to reinstall .NET Core SDK 2.0 and to repair my VS 2017, but without any success.

@bubert Can you provide the same information we asked Nikas for?  IE, what errors you're seeing, whether you can successfully build from the command line, and a zip of your project with a log?  You can create a binary log with the /bl argument, which is easier for us to investigate than an /flp:v=diag log.  You might want to create a separate issue for all this, since it seems like the original issue reported here has been fixed.

@dsplaisted I've got exactly the same errors in IDE and when called dotnet restore as a Nikas, dotnet --version also returned 2.0.0. However, completely removing the IDE and .NET Core 2.0 SDK and installing them from scratch fixed the error, but now I can't reproduce it and provide the logs, sorry.  Anyway, thank you for your response!

@ericstj thank you, saved my week. I got the same problem. Renaming the NuGetFallbackFolder, deleting the bin and obj folders from the project, running dotnet restore from command prompt with elevated privileges resolved the problem.

Looks like this issue can be closed as folks have identified a work around. @emgarten so you need an issue to track root causing this?

@Petermarcu I've opened https://github.com/NuGet/Home/issues/5995 to track this

Thanks. I'll close this one then.


My settings page mobile refactor

Refactored the my settings page to be mobile friendly using the include media library. Desktop site looks the same, mobile view shown below:

To-do:

Profile picture mobile modal fix
Change password mobile modal fix

Stacked the inputs in one row instead of 2:

Also changed breakpoints for the grid to line up with the includemedia ones. For a larger overhaul, might need to do a separate issue/PR and convert stuff over. Do you want to move away from the material-ui grid system?

Also please add issues for:

Profile picture mobile modal fix
Change password mobile modal fix

if they still need fixing (Change password looks ok to me)

They both look kinda ok to me locally - do we want to just fix up the X button overlap for the pfp upload one?


Errors preforming boolean operations with extruded svg

I am attempting to preform a difference operation on a CSG cube with an extruded svg image. In this example, I am simply trying to remove a small disk from the surface of a block, using an svg.
Here is the script I am attempting to execute:

const scad = require('@jscad/scad-api');
const stlSerializer = require('@jscad/stl-serializer').serialize;
const svgDeSerializer = require('@jscad/svg-deserializer').deserialize;

const { cube } = scad.primitives3d;
const { difference } = scad.booleanOps;
const { center } = scad.transformations;
const { linear_extrude } = scad.extrusions;

// Text Engraving
let csgImage = svgDeSerializer("<svg height=\100\" width=\"100\"><circle cx=\"50\" cy=\"50\" r=\"40\" stroke=\"black\" stroke-width=\"3\" fill=\"red\" /> </svg>", null, {output: 'csg'});

//The Actual Construct
let block = cube({ size: [20, 20, 5]}).center('x', 'y');
let engraving = linear_extrude({height: .5}, csgImage).center('x', 'y');

let output = difference(block, engraving);

//Output Construct as STL
let raw = stlSerializer(output);
let stl = Buffer.concat([Buffer.from(raw[0]), Buffer.from(raw[1]), Buffer.from(raw[2])]);

process.stdout.write(stl);

Here is the error output from Node.JS:
[calob@asgard the-squire]$ node engrave-test.js > engraving_text.stl
/home/calob/Projects/BattlePrints/product-models/node_modules/@jscad/csg/csg.js:3216
                    var splitresult = plane.splitPolygon(polygon);
                                            ^

TypeError: plane.splitPolygon is not a function
    at CSG.PolygonTreeNode._splitByPlane (/home/calob/Projects/BattlePrints/product-models/node_modules/@jscad/csg/csg.js:3216:45)
    at CSG.PolygonTreeNode.splitByPlane (/home/calob/Projects/BattlePrints/product-models/node_modules/@jscad/csg/csg.js:3198:22)
    at CSG.Node.addPolygonTreeNodes (/home/calob/Projects/BattlePrints/product-models/node_modules/@jscad/csg/csg.js:3444:41)
    at CSG.Tree.addPolygons (/home/calob/Projects/BattlePrints/product-models/node_modules/@jscad/csg/csg.js:3331:27)
    at new CSG.Tree (/home/calob/Projects/BattlePrints/product-models/node_modules/@jscad/csg/csg.js:3304:28)
    at CSG.subtractSub (/home/calob/Projects/BattlePrints/product-models/node_modules/@jscad/csg/csg.js:334:21)
    at CSG.subtract (/home/calob/Projects/BattlePrints/product-models/node_modules/@jscad/csg/csg.js:327:33)
    at difference (/home/calob/Projects/BattlePrints/product-models/node_modules/@jscad/scad-api/src/ops-booleans.js:42:13)
    at Object.<anonymous> (/home/calob/Projects/BattlePrints/product-models/the-squire/engrave-test.js:17:14)
    at Module._compile (module.js:612:30)

It appears that somewhere between calling the boolean operator and spliting the PolygonNodeTree, something besides the CSG.Plane object is passed as the first agrument to _splitByPlane.
I honestly don't have enough experience with these libraries, or the math behind them, to hunt the error on my own.
Any ideas?

As an aside: if you call the operator with the extruded svg object as the first argument, and the standard CSG.Cube second, it works just fine.  If attempting a union, this is fine, but as I am trying to engrave svg images into the CSG.Cube, there isn't a work around for this that I can see.

@CalobHumble thanks for using the libraries.
I don't see anything wrong with your script. You can call difference(CSG,CSG) or CSG.subtract(CSG).
The outcome should be the same.
I'll try the same script this evening. @kaosat-dev might have some suggestions as well.

Quick question... are you using the latest IO libraries for SVG?

@z3dev it seems I'm using @jscad/io version 0.3.5 from npm

Update:  I have deleted and reinsatlled the dependcies directly from npm. The original problem with the subtract operator seems to be resolved, however a new issue has occured:
[calob@asgard tests]$ node engrave-test.js > test.stl
/home/calob/Projects/BattlePrints/product-models/node_modules/@jscad/csg/src/CSG.js:1189
            delete sidestocheck[sidetag]
                                ^

ReferenceError: sidetag is not defined
    at CSG.fixTJunctions (/home/calob/Projects/BattlePrints/product-models/node_modules/@jscad/csg/src/CSG.js:1189:33)
    at transform (/home/calob/Projects/BattlePrints/product-models/node_modules/@jscad/io-utils/ensureManifoldness.js:11:46)
    at ensureManifoldness (/home/calob/Projects/BattlePrints/product-models/node_modules/@jscad/io-utils/ensureManifoldness.js:15:40)
    at serialize (/home/calob/Projects/BattlePrints/product-models/node_modules/@jscad/stl-serializer/index.js:13:10)
    at Object.<anonymous> (/home/calob/Projects/BattlePrints/product-models/tests/engrave-test.js:20:11)
    at Module._compile (module.js:612:30)
    at Object.Module._extensions..js (module.js:623:10)
    at Module.load (module.js:531:32)
    at tryModuleLoad (module.js:494:12)
    at Function.Module._load (module.js:486:3)

My dependicies list:
    "@jscad/csg": "^0.3.1",
    "@jscad/io": "^0.3.5",
    "@jscad/openjscad": "^1.0.3",
    "@jscad/scad-api": "^0.3.5",
    "@jscad/stl-serializer": "0.0.5",
    "@jscad/svg-deserializer": "^0.2.1",

I will investigate further, and see if I can resolve the issue myself, but any assistance is greatly appreciated.

I have temporarily solved the issue by rolling back the stl-serializer library to 0.0.4.

Hi @CalobHumble , thanks for the report !
I'll try to reproduce the error.
Is your script above the complete script ?

Actually I was able to reproduce the error straight away, I will work on a fix asap

interestingly enough if output is set to either just engraving or block there are no issues, so the output of difference is wrong somehow.
The error you are getting is very likely related to the very recent (yesterday) addition of a function that ensures manifoldness of the stl ouput via fixTJunctions

ok so after digging into the issue further , it turns out that fixTJunctions itself is broken , so I will have to patch that before impacting the rest.

@CalobHumble could you please update your project with the latest version of the @jscad modules ?
It should now be fixed: I tested your script locally and it now works fine for me.

@kaosat-dev sorry about the long wait.  But yes, with the new modules, everything works just as expected.


Fails to build with musl due to incorrect arguments to libc::ioctl

   Compiling below-ethtool v0.8.1 (below-0.8.1/below/ethtool)
error[E0308]: mismatched types
   --> below/ethtool/src/reader.rs:36:58
    |
36  |     let exit_code = unsafe { libc::ioctl(fd.as_raw_fd(), nix::libc::SIOCETHTOOL, &ifr) };
    |                              -----------                 ^^^^^^^^^^^^^^^^^^^^^^ expected `i32`, found `u64`
    |                              |
    |                              arguments to this function are incorrect
    |
note: function defined here
   --> .cargo/registry/src/index.crates.io-6f17d22bba15001f/libc-0.2.153/src/unix/linux_like/linux/musl/mod.rs:850:12
    |
850 |     pub fn ioctl(fd: ::c_int, request: ::c_int, ...) -> ::c_int;
    |            ^^^^^
help: you can convert a `u64` to an `i32` and panic if the converted value doesn't fit
    |
36  |     let exit_code = unsafe { libc::ioctl(fd.as_raw_fd(), nix::libc::SIOCETHTOOL.try_into().unwrap(), &ifr) };
    |                                                                                ++++++++++++++++++++

Alpine Linux Edge
musl libc 1.2.5
rust 1.77.0

Just confirming, are you building on a 32 bit system?

No, this is from x86_64.

termux/termux-packages#19609

This problem is there for libfs as well.
cargo install --target x86_64-unknown-linux-musl xcp fails too.
error[E0308]: mismatched types
   --> /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/libfs-0.7.2/src/linux.rs:153:45
    |
153 |     if unsafe { libc::ioctl(fd.as_raw_fd(), FS_IOC_FIEMAP as u64, req_ptr) } != 0 {
    |                 -----------                 ^^^^^^^^^^^^^^^^^^^^ expected `i32`, found `u64`
    |                 |
    |                 arguments to this function are incorrect
    |


Allow designspace XML editing

On input, the app should immediately attempt to compile the XML, report errors.

works now


remove unnecessary unit tests

What type of PR is this?
/kind cleanup
What does does this PR do / why we need it:
In this PR I have removed all the unit tests that try to test golang's core library feature like string.EqualFold and at times just string formating. Ever test removed has been removed with careful deliberation keeping in mind to not hamper with the quality of the tests.
Which issue(s) this PR fixes:
No issue
PR acceptance criteria:

[x] Unit test

[ ] Integration test

[ ] Documentation

[ ] I have read the test guidelines

How to test changes / Special notes to the reviewer:

platform issue
/test v4.5-integration-e2e

/approve


fix(styles): Fix styles for day mode and signed in avatar

Closes #286
Closes #288

Merging without CI due to usability issues. Style changes only.


Setting project up in dev mode

Hi folks!
I was setting up the project on my local system. I was installing the package in dev mode (pip install -e '.[dev]'). For some reason, it has been taking me quite a while to install it, perhaps due to the versions of the libraries not being specified in requirements.txt.  How much time should it ideally take?

it may take some time since there's a lot of dependencies - maybe your internet connection isn't that good?


Allow for multiple attachments

Would be great if the form accepted multiple file attachments, with the fields named attachment[0], attachment[1], etc.

Resolved with 7412f3247


Still getting mssing foreign key exception of pool metadata tables

While working on another aspect I was getting an error "aggregate functions are not allowed in WHERE" which was then followed (after a roll back to the start of the epoch) by:
DbInsertException "PoolOfflineFetchError"
  ( SqlError
    { sqlState = "23503"
    , sqlExecStatus = FatalError
    , sqlErrorMsg = "insert or update on table \"pool_offline_fetch_error\" violates foreign key \
                                       \constraint \"pool_offline_fetch_error_pmr_id_fkey\""
    , sqlErrorDetail = "Key (pmr_id)=(1312) is not present in table \"pool_metadata_ref\"."
    , sqlErrorHint = ""
    }
  )

I have checked that commit d658f389b25753d9be7e152dd8625b02ec99d617 is present in the branch I am working on.

:cry: Turns out the insertOfflineResults function had:
    insert :: (MonadBaseControl IO m, MonadIO m) => FetchResult -> ReaderT SqlBackend m ()
    insert fr =
      case fr of
        ResultMetadata md -> void $ DB.insertPoolOfflineData md
        ResultError fe -> void $ DB.insertPoolOfflineFetchError fe

Probably as the result of a bad git merge.
That should have been using DB.insertCheckPoolOfflineData and DB.insertCheckPoolOfflineFetchError.
To reiterate:

The test for this was correctly using the checked versions of these functions
The actual real code was using the old unckecked versions of these functions

The solution is to permanently remove the unchecked versions of those functions

This affects both master and the latest 11.0.3 release :cry:

This will be in release 11.0.4 .


DotNetNuke.UI.Modules.IModuleControlPipeline - Crash

Description of bug
After an indeterminate amount of time, site crashes with the following error on log:
[ERROR] DotNetNuke.Services.Exceptions.Exceptions - DotNetNuke.Services.Exceptions.PageLoadException: Unhandled error loading page. ---> System.InvalidOperationException: No service for type 'DotNetNuke.UI.Modules.IModuleControlPipeline' has been registered.
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
   at DotNetNuke.UI.Skins.Skin..ctor()

By recycling the IIS application pool, or by forcing a recompilation, the issue is fixed.
Is this a regression on https://github.com/dnnsoftware/Dnn.Platform/issues/2806 maybe?
Error information
[ERROR] DotNetNuke.Services.Exceptions.Exceptions - DotNetNuke.Services.Exceptions.PageLoadException: Unhandled error loading page. ---> System.InvalidOperationException: No service for type 'DotNetNuke.UI.Modules.IModuleControlPipeline' has been registered.
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
   at DotNetNuke.UI.Skins.Skin..ctor()
   at ASP.portals__default_skins_dnnbootsterv2_home_ascx..ctor()
   at __ASP.FastObjectFactory_app_web_50qugp1r.Create_ASP_portals__default_skins_dnnbootsterv2_home_ascx()
   at System.Web.UI.TemplateControl.LoadControl(IWebObjectFactory objectFactory, VirtualPath virtualPath, Type t, Object[] parameters)
   at System.Web.UI.TemplateControl.LoadControl(VirtualPath virtualPath)
   at DotNetNuke.UI.ControlUtilities.LoadControl[T](TemplateControl containerControl, String ControlSrc)
   at DotNetNuke.UI.Skins.Skin.LoadSkin(PageBase page, String skinPath)
   --- End of inner exception stack trace ---

Affected version

[ ] 10.00.00 alpha build
[X] 09.08.00 release candidate
[ ] 09.07.02 latest supported release

Affected browser

[X] Chrome
[X] Firefox
[ ] Safari
[X] Internet Explorer 11
[X] Microsoft Edge (Classic)
[ ] Microsoft Edge Chromium

You said "forcing a recompilation" fixes the issue, are you recompiling the platform code?  In general usage, you shouldn't need to adjust the platform code to use DNN.  Leave the platform code as-is, and use the many built-in extension points to adjust the site.  If that's not what you mean, let us know what steps you're taking to get here.

Forcing the recompilation is for example by making a change in web.config:
<compilation debug="false" strict="false" targetFramework="4.7.2">
to
<compilation debug="true" strict="false" targetFramework="4.7.2">
and vice-versa

Oh, that is not a recompilation, it just causes the site to reload

Are you able to make the issue happen also using the default theme, it looks like on you second log it is pointing at the theme.

Are you able to make the issue happen also using the default theme, it looks like on you second log it is pointing at the theme.

Yes, it happens with the default theme too. Also, the skin doesn't seem to be related, because even if I replace it with an "empty" skin, it still occurs.
This may have to do with an App_Code folder of the project that has carried over from previous versions, looking into it.

This may have to do with an App_Code folder of the project

Indeed this seems to be it:
If you have custom code in App_Code folder, it will work properly the first time, and then fail on reload with the cryptic "DotNetNuke.UI.Modules.IModuleControlPipeline" message.
If you compile your code into a class library (.dll), paste it into "bin" folder and remove the App_Code folder, it will work properly without crash.
The hint was from this thread: https://dnncommunity.org/forums/aft/1153
Maybe it's more of a documentation issue than a bug then: It should mention on the installation page not to use App_Code folders any more.

I was wrong, this still occurs even after compiling the code into a class library. It also still occurs on latest version 9.8.1


Add ability to update QuiltiX graph from in-memory MaterialX document

Issue
This allows for integrations which create an in-memory MaterialX doc which can be routed to QuiltiX to update it's graph.
Fixes #45
Changes
Split out the file read part of load_graph_from_mx_file into a load_graph_from_mx_doc (load from in memory document). The file handling part would stay in load_graph_from_mx_file which would call load_graph_from_mx_doc.

Looks good, but then the mx_file_loaded signal makes more sense to be emitted from the load_graph_from_mx_file function. Then the load_graph_from_mx_doc wouldn't require a mx_file_path input attribute.

Thanks for catching this. I've remove the mx_file_path argument from load_graph_from_mx_doc and move the signal to load_graph_from_mx_file.


Unintended keys with TTL = -1

We're seeing keys with -1 in our redis cache but after thoroughly examining the code base we cannot see any possible way for the -1 to get passed as a value. Is this possibly a bug in predis? Or is there a situation where assigning TTL -1 unexpectedly might be the intended behavior?

As stated in the Redis documentation for the TTL command:

Starting with Redis 2.8 the return value in case of error changed:
The command returns -2 if the key does not exist.
The command returns -1 if the key exists but has no associated expire.


Stabilize simulations to avoid differences between CI and local runs

There were always problems with failing tests (mostly the subcell blast waves simulations) with different setups (e.g. local, CI and also different julia versions 1.9 vs. 1.10 vs. 1.11).
Moreover, simulations on macOS were completely different to windows and Linux (see https://github.com/trixi-framework/Trixi2Vtk.jl/pull/67#issuecomment-2186068769).
I realized that most of the time it's the same elixirs with local limiting (see https://github.com/trixi-framework/Trixi2Vtk.jl/pull/67#issuecomment-2205358329).
Looking into those simulations showed me that in these simulations there are also deviations from the calculated subcell bounds.
I adapted parameters to hopefully stabilize the simulations and hope that this fixes also the different results with different setups.
Updates are following...

In c47e09a, I added CI tests on macOS, windows and with julia1.11.
Open problems:

[ ] elixir_euler_sedov_blast_wave_sc_subcell.jl, which was initialized with numbers from local run (and works on roci), fails everywhere (linux macos windows) with julia 1.10 (with errors of about 1e-5) but not with julia 1.11 on linux.
Ideas:

Pass the Linux CI numbers and look if the tests work for macOS and windows. But they will probably fail with julia 1.11

Adding positivity limiting of rho. Maybe it stabilizes the simulation further. Although, it should be stable enough with deviations of only 3.55e-15 for entropy

[ ] elixir_eulermulti_shock_bubble_shockcapturing_subcell_minmax.jl, structured/elixir_euler_sedov_blast_wave_sc_subcell.jl (local bounds) are failing on macOS with large differences partly (e.g. here)

This is probably acceptable. This instability(?)/differences was removed for elixir_euler_blast_wave_sc_subcell_nonperiodic.jl in this PR. Therefore the test in Trixi2Vtk should run through.

[ ] elixir_euler_sedov_blast_wave_sc_subcell.jl, which was initialized with numbers from local run (and works on roci), fails everywhere (linux macos windows) with julia 1.10 (with errors of about 1e-5) but not with julia 1.11 on linux.
Ideas:
Pass the Linux CI numbers and look if the tests work for macOS and windows. But they will probably fail with julia 1.11
Adding positivity limiting of rho. Maybe it stabilizes the simulation further. Although, it should be stable enough with deviations of only 3.55e-15 for entropy --> still failing (see here)

There are a few other options to test (decrease the newton tolerances or CFL number further). But I don't think that this makes sense to take it to the extreme. The differences are relatively small anyway. Alternatively, I will probably take over the number of the CI run and then adapt the numbers when moving to julia 1.11.

[ ] elixir_eulermulti_shock_bubble_shockcapturing_subcell_minmax.jl, structured/elixir_euler_sedov_blast_wave_sc_subcell.jl (local bounds) are failing on macOS with large differences partly (e.g. here)
This is probably acceptable. The instability(?)/differences for elixir_euler_blast_wave_sc_subcell_nonperiodic.jl was removed in this PR. Therefore, the test in Trixi2Vtk should run through.

I think, I will just accept those differences for now.

In c3e6469, added tests which use the pure FV version of the code. These tests are working throughout all machines. E.g. here.

Removing the use of @muladd in subcell_limiters_2d.jl does not change the result. See d5ec1f6.

I checked if the maximum number of iterations is reached for the newton method. I will add the results here (each tspan=(0.0, 1.0), initial_refinement_level=4):

sedov blast, tols=(1.0e-13, 1.0e-15), max_iterations=60: Not reached
sedov blast, tols=(1.0e-13, 1.0e-14), max_iterations=60: Not reached
sedov blast, tols=(1.0e-13, 1.0e-15), max_iterations=50: Reached (2 times)
sedov blast, tols=(1.0e-13, 1.0e-15), max_iterations=30: Reached (3 times)
sedov blast, tols=(1.0e-13, 1.0e-14), max_iterations=30: Not reached

I checked the results of
trixi_include("examples/structured_2d_dgsem/elixir_euler_sedov_blast_wave_sc_subcell.jl", tspan=(0.0, 0.5), max_iterations_newton=40, newton_tolerances=(1.0e-13, 1.0e-15), cfl=0.6, interval=5)

on a local macOS machine (arm64-apple-darwin22.4.0 with an 10 × Apple M2 Pro CPU). The results are identical (up to the 8th digit) to my local results (ubuntu) and the ubuntu CI results. And therefore different to the macOS CI results.
That's a good sign.

Because I received the "correct" results on a local macOS machine (see comment), I added a macOS test using aarch64 instead of x64 in ed77e0f.
The tree_part2 job just does not start (even after rerunning and reopening the PR).
On the other hand, the structured test case, which failed with x64 is working now.

So, as a conclusion

there is only 1 test case left, where the errors are different between the CI run and my local runs (differences of about 1e-5).
Of course, I can (and probably will) just adapt the numbers and use the ones from the CI. Although, it is a potential cause of failing tests in the future (e.g. when updating to julia 1.11.)

The differences between macOS and all other runs only occur on the one CI macOS infrastructure. Since we don't even test these things on macOS normally, I'd just accept it.

Many tests are still disabled and should be re-enabled before merging, but the changes to the elixirs seem to be reasonable.

Yes of course, I just wanted to keep it with all the different variants until it decided which one I will keep

However, does this also mean that the subcell limiting methods as implemented here is not unconditionally safe and that one has to choose "correct" parameters? Or was this already known and just surfaced here in an unsuspecting way?

Since we use an iterative method to find a stable amount of limiting for nonlinear variables, you can say that, yes.
But it's like with the cfl number. From some point, all parameters are "correct".
To find this point, we implemented the local bounds checking routines, where you can see whether a limiting is working as expected.
Or what do you say @amrueda?
When one only has to look at the bounds checking to see if the limiting works properly, why did we have these issues here? This is basically just because I didn't pay too much attention to this before. And of course, when changing setups within the tests, this has to be checked again.

However, does this also mean that the subcell limiting methods as implemented here is not unconditionally safe and that one has to choose "correct" parameters? Or was this already known and just surfaced here in an unsuspecting way?

Since we use an iterative method to find a stable amount of limiting for nonlinear variables, you can say that, yes.
But it's like with the cfl number. From some point, all parameters are "correct".
To find this point, we implemented the local bounds checking routines, where you can see whether a limiting is working as expected.
Or what do you say @amrueda?

I agree. It seems that the major issue was the choice of the Newton method parameters and the CFL. The subcell-limiting methods are safe when the CFL is below a threshold and when the bounds are selected properly for the equation being solved. For instance, positivity of density and pressure is "safe" for the Euler equations. In the case of non-linear constraints, such as pressure, the subcell-limited method converges to the right/safe scheme only if the non-linear solver manages to converge (related to the Newton parameters).

OK, thanks for the clarification! If not already in there, I think it would be good if you could add such an (extended) description to the docs for future reference.

So, a final conclusion before I remove all additional tests.
The simulations with local limiting are expectedly more susceptible to small differences. Especially, the elixirs elixir_eulermulti_shock_bubble_shockcapturing_subcell_minmax.jl (on TreeMesh), basically all elixir_euler_sedov_blast_wave_sc_subcell.jl and sometimes also the elixir_euler_blast_wave_sc_subcell_nonperiodic.jl eilixirs.


PWA v1

Although main purpose for this application is to listen online radios, it wouldn't hurt to make make it PWA and have some features work while disconnected (Playlist and Station maintenance).
That requires a few changes:

[x] A "Notification Center" to let user know when connection is down or went back, also notify whenever an update is available.
[ ] Allow edit a Station from Playlist view.
[x] App Cache that relies on Cache Buster info file to revoke cached files.

I was planning to cache handler taking into account also UI5 updates to refresh app, but on second thought I dropped that, since it would be too many self-updates (as app is pointing to latest framework version).


get_register_name(&self, location: VarNode) Should not consume VarNode

Hi! I believe that get_register_name should obtain the location by reference instead of consuming the varnode. Or probably I am missing something? https://github.com/toolCHAINZ/jingle/blob/main/jingle/src/translator.rs#L68

Ahh that is probably a leftover from back when I had Varnode impl Copy. I removed that for some reason, so yeah, it would be a little more ergonomic to avoid a clone here. I'll change this when I get a chance.

Fixed in 262d530


New version: UnstructuredGrids v0.2.0

Registering package: UnstructuredGrids
Repository: https://github.com/lssc-team/UnstructuredGrids.jl
Created by: @fverdugo
Git reference: register/UnstructuredGrids/v0.2.0
Version: v0.2.0
Commit: 1e5709379df40fa1ba86d99d5dcc1e37f2889338
Reviewed by: @fverdugo
Reference: https://github.com/lssc-team/UnstructuredGrids.jl/commit/1e5709379df40fa1ba86d99d5dcc1e37f2889338#commitcomment-33507039

Consider enabling TagBot to make sure the release is persistent in your repo, see https://discourse.julialang.org/t/ann-tagbot-creates-tags-and-releases-for-your-julia-packages-when-theyre-registered/23084

@fredrikekre Thanks for the info! For the moment, I am manually copying and pasting the git commands that @juliaRegistrator generates for me.


Compilations get messed up when importing with Koel

I realized that after 2 clean installs, that compilation album information gets messed up. When I import a compilation (of 30 songs), under the album page, there's about 26 entries for that compilation, with each artist getting it's own album for their songs in the compilation.
The same occurs with an album of 17 songs, 15 of which are under 1 album for the main artist, and the other 2 songs under separate albums.
The only feasible fix I see to this is modifying the metadata of each song before importing them, in which the artist name is the exact same for all songs for an album.
OS is Ubuntu Server 16.04.1 LTS. No errors, just an application issue. Resyncing doesn't help, either.

I think this whole compilation issue should be fixed by now, so I'm closing this. Feel free to open another if I'm wrong.


Audit log user is saved

ei vielä muuta paljoa koska toiminnallisuus vaatii muutoksia kontrollereihin

Codecov Report

Merging #208 (72d01b1) into main (a203e29) will increase coverage by 0.18%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main     #208      +/-   ##
==========================================
+ Coverage   86.84%   87.03%   +0.18%
==========================================
  Files          17       17
  Lines         342      347       +5
  Branches       33       38       +5
==========================================
+ Hits          297      302       +5
  Misses         41       41
  Partials        4        4

Impacted Files
Coverage Δ

backend/models/log.model.js
100.00% <ø> (ø)

backend/services/admissionService.js
93.93% <100.00%> (+1.08%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a203e29...72d01b1. Read the comment docs.


Fix Camera2D jitter

Fix #35606 by rounding Camera2D transform origin to the nearest whole pixel before setting canvas transform.
Bugsquad edit: fixes https://github.com/godotengine/godot/issues/2667

I have compiled your Camere2d code and tested myself I can tell the jittering problem is gone, but I ran into another problem related with round to the nearest position nad it´s the camera smoothing is too sharp, it doesn't seem right.
my project has a resolution of  324*120 perhaps with higher resolution the problem disappears, however, I haven't tested yet.

I have compiled your Camere2d code and tested myself I can tell the jittering problem is gone, but I ran into another problem related with round to the nearest position nad it´s the camera smoothing is too sharp, it doesn't seem right.
my project has a resolution of 324*120 perhaps with higher resolution the problem disappears, however, I haven't tested yet.

Yeah... I don't think smoothing will ever work at such low resolutions unless you render at a higher resolution, or use a (sub)pixel shader, where you scale the whole image up (5 to 6 times in your case) and then smoothly interpolate using the shader.
Like so: https://code-disaster.com/2016/02/subpixel-perfect-smooth-scrolling.html
But even in this case the camera should always snap to whole pixels on the grid to avoid rendering artifacts, then the offset will be applied with the shader. You could skip the subpixel step if you want.
Side by side comparison: https://code-disaster.com/gdx/gdx-mellow-demo-2/

t.

ok, I'm going to test it I tell you the results later.

Someone should probably test it in a high resolution game, with smoothing enabled. I see your point, however i think that this is the problem that actually exists in the camera now, that this issue is fixing.
Imagine sprite1 at position 10.4 and sprite2 at position 20.6. If camera is at position 15.0, these will be drawn at perfectly at position 10.0 and 21.0. However, If camera moves to position 15.3, sprite 1 will still draw at position 11.0 and sprite2 draw at position 21.0. This is the whole problem, and those two sprites will appear to jump 1 pixel back and forth.
If we always round the camera rect, this back and forth jumping will never happen.

If we always round the camera rect, this back and forth jumping will never happen.

Sure it will. It will just happen when the sprites move, not if the camera moves. And we solved that problem by using pixel_snap.

If we always round the camera rect, this back and forth jumping will never happen.

Sure it will. It will just happen when the sprites move, not if the camera moves. And we solved that problem by using pixel_snap.

You can still get other forms of visual jitter, for example between the camera and the sprite it's following (I made some changes to my gdscript workaround to fix this a few days ago), but with the canvas transform being rounded, you should never get any inconsistencies between different sprites as the camera moves, which is what this issue is about.

You can still get other forms of visual jitter, for example between the camera and the sprite it's following (I made some changes to my gdscript workaround to fix this a few days ago), but with the canvas transform being rounded, you should never get any inconsistencies between different sprites as the camera moves, which is what this issue is about.

I agree. The question is, whether this should be enabled with pixel_snap, or be always on. Pixel_snap removes jitter from sprite positions, but its an optional toggle. Similarly, this fixes jitter caused by Camera positions, why should it be treated differently?

The question is, whether this should be enabled with pixel_snap, or be always on.

Is there any downside at all to always doing it? It's not like it's possible to render subpixels, and even if the jitter is less noticeable with higher resolutions, it's still not IMO correct or desirable behavior.

Alright, I tested this a bunch and I now think it should be a separate option, not just tacked on to pixel snap.

I tested a "high res" 2D game with texture filtering on and it is a trade off there. It is indeed a bit smoother without snapping, but instead of jittering you get shimmering, which isn't as bad. Snapping the camera makes it slightly less smooth, but you don't have that shimmering effect. You also have a lot more pixels to work with in a HD 2D game, so it's not like it feels choppy or anything.

On a pixel graphics game this commit works very nicely and I didn't even need to turn pixel snap on, making pixel snap feel almost unnecessary to me.

If you have a rotating camera it might be better to leave it off, but it didn't feel too bad with it on either.

So, personally I'd want to use this, even in a high res 2D game, but without having to turn on pixel snap to do so.
I'd 100% use it in a pixel art game of course, but as I said, I'm not sure pixel snap is needed if you use this and unfiltered textures.

Moving sprites with pixel art results in some undesired visual artifacts (even more if smoothing is enabled).

Definately needs to be optional. Figured out that when using zoom on the 2D camera, this snapping is undesired.

@forbjok please continue on this! It would be very useful to have

Presumably if you want to do any snapping to the camera, it might be better to do it on creation of the view matrix, not by setting the origin.
The reason being, that even a pixel camera might be required to smoothly move by sub pixel amounts. Only the visual representation needs to be snapped. Otherwise you would need to maintain the float position outside of the xform.origin.

Presumably if you want to do any snapping to the camera, it might be better to do it on creation of the view matrix, not by setting the origin. The reason being, that even a pixel camera might be required to smoothly move by sub pixel amounts. Only the visual representation needs to be snapped

Absolutely this. Even if you're snapping to pixels, the camera smoothing can look jerky you don't let the lerp algorithm reach sub-pixels where you're moving less than 1 pixel per frame.

Is there any downside at all to always doing it? It's not like it's possible to render subpixels, and even if the jitter is less noticeable with higher resolutions, it's still not IMO correct or desirable behavior.

In terms of the HTML5 canvas, if you render at a half-coordinate like [0, 0.5] you will see 2 light gray pixels next to each other rather than a single black pixel like if you render at [0, 0]. There is definitely a sub-pixel smoothing effect that can be useful for low resolution games that aren't trying to be pixel-perfect.

Sorry I am kinda new so how do I use it??

@skullleeep This pull request hasn't been merged yet, so you'd have to compile the engine from source with the pull request applied. However, since this pull request is based on the master branch, you'd effectively be compiling an unstable pre-release 4.0 version, which you most likely don't want. (We recommend using Godot 3.2.x in a production environment.)

@skullleeep This pull request hasn't been merged yet, so you'd have to compile the engine from source with the pull request applied. However, since this pull request is based on the master branch, you'd effectively be compiling an unstable pre-release 4.0 version, which you most likely don't want. (We recommend using Godot 3.2.x in a production environment.)
All in all, it might not be trivial to use this until the pull request is merged and a new release is made. We don't know for certain if this pull request will be merged still (as always). #41535 may also fix this in a more robust way.

OK thanks

@lawnjelly Now that dust has settled around 2D smoothing, is this pull request still relevant?

@lawnjelly Now that dust has settled around 2D smoothing (for now), is this pull request still relevant?

I think it has been superseded by #43194 (which did camera snapping as the viewport / canvas). Although these approaches have worked in a few test cases, the whole area of transform snapping has proved problematic for users in practice (with trials through the 3.2.4 betas).


Test/e2e setup basic tests

add example test to https://github.com/surrealdb/surrealdb.js/pull/34

Resolves https://github.com/surrealdb/surrealdb.js/issues/20
Closes https://github.com/surrealdb/surrealdb.js/pull/34

@tobiemh ready for merge!
Adding Bun to test-setup will happen when I have more time (in a few weeks)


[SPARK-43952][CORE][CONNECT][SQL] Add SparkContext APIs for query cancellation by tag

What changes were proposed in this pull request?
Currently, the only way to cancel running Spark Jobs is by using SparkContext.cancelJobGroup, using a job group name that was previously set using SparkContext.setJobGroup. This is problematic if multiple different parts of the system want to do cancellation, and set their own ids.
For example, BroadcastExchangeExec sets it's own job group, which may override job group set by user. This way, if user cancels the job group they set in the "parent" execution, it will not cancel these broadcast jobs launches from within their jobs. It would also be useful in e.g. Spark Connect to be able to cancel jobs without overriding jobGroupId, which may be used and needed for other purposes.
As a solution, consider add API to set tags on jobs, and to cancel jobs using tags:

SparkContext.addJobTag(tag: String): Unit
SparkContext.removeJobTag(tag: String): Unit
SparkContext.getJobTags(): Set[String]
SparkContext.clearJobTags(): Unit
SparkContext.cancelJobsWithTag(tag: String): Unit
DAGScheduler.cancelJobsWithTag(tag: String): Unit

Also added SparkContext.setInterruptOnCancel(interruptOnCancel: Boolean): Unit, which previously could only be set in setJobGroup.
The tags are also added to JobData and AppStatusTracker. A new API is added to SparkStatusTracker:

SparkStatusTracker.getJobIdsForTag(jobTag: String): Array[Int]

Use the new API internally in BroadcastExchangeExec instead of cancellation using job group, to fix the issue with these not being cancelled by user-set jobgroupid. Now, the user set jobgroupid should propagate into broadcast execution.
Also, switch cancellation in Spark Connect to use tag instead of jobgroup.
Why are the changes needed?
Currently, there may be multiple places that want to cancel a set of jobs, with different scopes.
Does this PR introduce any user-facing change?
The APIs described above are added.
How was this patch tested?
Added test to JobCancellationSuite.

If we don't want to add public APIs like that, I'm also fine with having all that as private[spark]; my planned use of it is inside Spark in Spark Connect.

If we don't want to add public APIs like that, I'm also fine with having all that as private[spark]; my planned use of it is inside Spark in Spark Connect.

I am fine with adding them as a public API

cc @gengliangwang - I also added the job tags to AppStatusTracker.

@gengliangwang it does complain about the incompatible constructor to JobData in MIMA check:
[error] spark-core: Failed binary compatibility check against org.apache.spark:spark-core_2.12:3.4.0! Found 1 potential problems (filtered 3979)
[error]  * method this(Int,java.lang.String,scala.Option,scala.Option,scala.Option,scala.collection.Seq,scala.Option,org.apache.spark.JobExecutionStatus,Int,Int,Int,Int,Int,Int,Int,Int,Int,Int,Int,scala.collection.immutable.Map)Unit in class org.apache.spark.status.api.v1.JobData does not have a correspondent in current version
[error]    filter with: ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.status.api.v1.JobData.this")
[error] java.lang.RuntimeException: Failed binary compatibility check against org.apache.spark:spark-core_2.12:3.4.0! Found 1 potential problems (filtered 3979)
[error] 	at scala.sys.package$.error(package.scala:30)
[error] 	at com.typesafe.tools.mima.plugin.SbtMima$.reportModuleErrors(SbtMima.scala:89)
[error] 	at com.typesafe.tools.mima.plugin.MimaPlugin$.$anonfun$projectSettings$2(MimaPlugin.scala:36)
[error] 	at com.typesafe.tools.mima.plugin.MimaPlugin$.$anonfun$projectSettings$2$adapted(MimaPlugin.scala:26)
[error] 	at scala.collection.Iterator.foreach(Iterator.scala:943)
[error] 	at scala.collection.Iterator.foreach$(Iterator.scala:943)
[error] 	at scala.collection.AbstractIterator.foreach(Iterator.scala:1431)
[error] 	at com.typesafe.tools.mima.plugin.MimaPlugin$.$anonfun$projectSettings$1(MimaPlugin.scala:26)
[error] 	at com.typesafe.tools.mima.plugin.MimaPlugin$.$anonfun$projectSettings$1$adapted(MimaPlugin.scala:25)
[error] 	at scala.Function1.$anonfun$compose$1(Function1.scala:49)
[error] 	at sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFunctions.scala:62)
[error] 	at sbt.std.Transform$$anon$4.work(Transform.scala:68)
[error] 	at sbt.Execute.$anonfun$submit$2(Execute.scala:282)
[error] 	at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:23)
[error] 	at sbt.Execute.work(Execute.scala:291)
[error] 	at sbt.Execute.$anonfun$submit$1(Execute.scala:282)
[error] 	at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:265)
[error] 	at sbt.CompletionService$$anon$2.call(CompletionService.scala:64)
[error] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[error] 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[error] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[error] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[error] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[error] 	at java.lang.Thread.run(Thread.java:750)
[error] (core / mimaReportBinaryIssues) Failed binary compatibility check against org.apache.spark:spark-core_2.12:3.4.0! Found 1 potential problems (filtered 3979)

is this a part of API that needs a compatible constructor to be added, or is it something to filter and ignore?

Resolved problem after merge.
@gengliangwang @rednaxelafx could you let me know what additional testing of SHS than what described in https://github.com/apache/spark/pull/41440/#discussion_r1222849401 would be needed?

@gengliangwang @rednaxelafx I updated the SHS tests canons. I claim it's fine because it's and additive change that should be backwards compatible (older versions ignoring this field) and forwards compatible (new versions reading old data with this missing will get empty)

LGTM on the UI-related changes.

https://github.com/juliuszsompolski/apache-spark/actions/runs/5276958824/jobs/9544388805
2023-06-15T10:14:29.8570402Z [info] - single listener, check trigger events are generated correctly *** FAILED *** (493 milliseconds)
2023-06-15T10:14:29.8579106Z [info]   Assert on query failed: : Queue() was empty
2023-06-15T10:14:29.8586265Z [info]   org.scalatest.Assertions.newAssertionFailedException(Assertions.scala:472)
2023-06-15T10:14:29.8593211Z [info]   	org.scalatest.Assertions.newAssertionFailedException$(Assertions.scala:471)
2023-06-15T10:14:29.8601173Z [info]   	org.scalatest.Assertions$.newAssertionFailedException(Assertions.scala:1231)
2023-06-15T10:14:29.8608400Z [info]   	org.scalatest.Assertions$AssertionsHelper.macroAssert(Assertions.scala:1295)
2023-06-15T10:14:29.8621283Z [info]   	org.apache.spark.sql.streaming.StreamingQueryListenerSuite.$anonfun$new$7(StreamingQueryListenerSuite.scala:103)
2023-06-15T10:14:29.8629373Z [info]   	org.apache.spark.sql.streaming.StreamingQueryListenerSuite.$anonfun$new$7$adapted(StreamingQueryListenerSuite.scala:102)
...
2023-06-15T11:26:23.9466495Z [info] *** 1 TEST FAILED ***
2023-06-15T11:26:23.9615725Z [error] Failed: Total 11179, Failed 1, Errors 0, Passed 11178, Ignored 45
2023-06-15T11:26:24.0041695Z [error] Failed tests:
2023-06-15T11:26:24.0051627Z [error] 	org.apache.spark.sql.streaming.StreamingQueryListenerSuite

flaky again, retrigger once more

https://github.com/juliuszsompolski/apache-spark/actions/runs/5278491450/jobs/9547955928
======================================================================
FAIL [6.087s]: test_read_images (pyspark.ml.tests.test_image.ImageFileFormatTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/__w/apache-spark/apache-spark/python/pyspark/ml/tests/test_image.py", line 34, in test_read_images
    self.assertEqual(df.count(), 4)
AssertionError: 0 != 4

----------------------------------------------------------------------
Ran 1 test in 10.116s

FAILED (failures=1)

flaky again, retrigger again

https://github.com/juliuszsompolski/apache-spark/actions/runs/5280624891/jobs/9553060736
Flake: SparkR tests failed to initialize containers.
@HyukjinKwon @hvanhovell  I have restarted CI on this PR 4 times, and each time I got a failed run due to a different flake. Do I need to keep restarting until I get a clean run?

I guess https://ci.appveyor.com/project/ApacheSoftwareFoundation/spark/builds/47350391 has just issues at the moment.
Start-FileDownloadInternal : Error downloading remote file: One or more errors occurred.
Inner Exception: Remote server returned 404: Not Found

AppVeyor is fine.
Merged to master

Hi @juliuszsompolski, I got a local build failure related to the master branch (https://github.com/apache/spark/commit/5d840eb455350ef3f6235a031a1689bf4a51007d).  For example, when I try to run ClientDistributedCacheManagerSuite. The error indicates associated with the following two lines. I found nothing related to addJobTags and getJobTagsList. Do you have any suggestions? Thank you.
value addJobTags is not a member of org.apache.spark.status.protobuf.StoreTypes.JobData.Builder
    jobData.jobTags.foreach(jobDataBuilder.addJobTags)
value getJobTagsList is not a member of org.apache.spark.status.protobuf.StoreTypes.JobData
      jobTags = info.getJobTagsList.asScala,

@shuwang21 these functions come from code generated by https://github.com/apache/spark/pull/41440/files#diff-28490b011ea75c4442d8ac95a3d8599abaffcd050672ed3416d7d259a22057c7R50
If it does not get generated, maybe something is wrong with the protobuf compiler in your dev env?

I see. Thank you. @juliuszsompolski  Solved. Need to recompile the project.


Ardana Gerrit merger and dependency event handler

Implements a new Jenkins job - openstack-ardana-gerrit-events - which
monitors Gerrit events and takes the following actions (i.e.
implements the Gerrit CI strategy [1]):

when a Code-Review+2, Workflow+1 or Verified+2 label
is set on a change, it verifies and attempts to merge it, if
it is submittable and all dependencies have already been merged
when a change is merged, it also tries to merge other open
changes that depend on the first
when a new patchset is published for a change, it finds and
invalidates other open changes that directly or indirectly
depend on the first (i.e it resets the Verified label value)

The job may also be triggered by posting a reverify comment
on a Gerrit change, or triggered manually, by supplying a
change id. In both these cases, the job will attempt to
merge the target Gerrit change, if it meets all the criteria.
The python scripts used to implement the job can also be triggered
manually, from a local git clone, if necessary, although not all
actions (merge, voting) are available to all users.
This commit also moves everything Gerrit related in the python
scripts to a standalone python module, which is used in common
by all scripts, including build_test_package.py.
Other improvements:

simple caching for Gerrit REST API calls, to reduce
the number of redundant calls made to Gerrit
use the Gerrit API to get implicit dependencies instead
of relying on a local git clone
command line parameters for build_test_package.py
documentation
moved all settings to a gerrit-settings.json file
the OBS target repository and linked project can be
computed automatically based on the target Gerrit branch
and the gerrit-settings.json file
removed redundant test package builder parameters from all jobs
build packages targeting a specific patchset instead of always
targeting the last patchset

NOTE: this depends on and will include #2923 until that PR is merged
[1] https://github.com/SUSE/cloud-specs/blob/master/specs/release-independent/gerrit-ci-strategy.rst

@jhesketh thank you for the in-depth review, those were some great points you made. I hope I covered everything (I even spotted a bug or two on top of that).

I'll have to re-run some tests after addressing review feedback, thus keeping the do not merge yet label.

Awesome, this looks good to me. Or at least good enough to merge and iterate on later if we need. This PR is growing unwieldy

Thanks @jhesketh , I'm still running some tests and fixing minor issues here and there, but at this point I'm satisfied with the current implementation.

@jhesketh I squashed all commits together and updated the commit message and PR description to better reflect what's included.


Button at the side of shorten url to use as it directly or part of the url

I would like to work on this issue!

Button to the side of the shorten url, you can either copy the link through clicking the button or can select the part of the link if wanted. As of now, if we click on the link it copies, but then you can also select the part of it or the whole link by the button.

@rachitag112 You can open a PR about it.
Not a problem.

@rachitag112
There should be a "Shorten URL" button which will shorten the URL and directly copy it to clipboard. Is that what you mean?

@SandeshGC
That thing is already happening, you can refer to the screenshot to understand which kind of button I want to convey


The fee for 6 digit user issue asset is too low

100USD worth is more doable .

Maybe , we should reserve some key word as prefix for typical assent
1.S_   :  is stock assent for company
2.P_   :  is personal assent
3.E_   :  is assent for star


add session level traces

It's possible to add handlers on an aws session level.
In my code I'm using singleton session and multiple clients - so it's more convenient for me to use it this way.

looks like PR is not merged although marked as merged


Missing Large Product Images in result.media

Hello,
I am encountering an issue while using the crawl4ai library to scrape product details from Amazon.  For example when crawling the following,
Amazon Essentials Slim Fit Stretch Vintage Jeans
I noticed that the result.media object does not contain the src URLs for the large images related to the product. Instead, it only includes thumbnail images. I found that the URLs for these large images are present in the extracted HTML but not being identified
Steps to reproduce:
import asyncio
from crawl4ai import AsyncWebCrawler
import base64

async def main():
    async with AsyncWebCrawler(verbose=True) as crawler:
        result = await crawler.arun(url="https://www.amazon.in/Amazon-Essentials-Slim-Fit-Stretch-Vintage/dp/B07XC1Y7BL", magic=True, screenshot=True, headless=False)
        return result

if __name__ == "__main__":
    result = asyncio.run(main())
    for image in result.media['images']:
        print(image['src'])

If anyone else is facing the same issue or has suggestions on how to resolve it, I would greatly appreciate your help!

In this page, when you hover on small images on the left, then the real image will be added dynamically. So that's why the first image URL is in the HTML, but the rest need you to conduct an action in order to be able to bring it up. For example, one action that you can do is the following: a JavaScript code that you can see here.
let items = document.querySelectorAll('.a-spacing-small.item.imageThumbnail.a-declarative');
for (let item of items) { item.click(); }

After this code, all the main images will be injected into the page. And then after that, if you crawl, everything will work. So, I patch up a code that will do this for you. Please take a look at the code below.
async def main():
    async with AsyncWebCrawler(
            headless=False,  # Set to False to see what is happening
            verbose=True,
    ) as crawler:
        result = await crawler.arun(
            url="https://www.amazon.in/Amazon-Essentials-Slim-Fit-Stretch-Vintage/dp/B07XC1Y7BL",
            cache_mode=CacheMode.BYPASS,
            js_code = """let items = document.querySelectorAll('.a-spacing-small.item.imageThumbnail.a-declarative'); for (let item of items) { item.click(); }""",
            delay_before_return_html=0.2
        )

        for img in result.media['images']:
            if img['src'].startswith('https://m.media-amazon.com/images/I/') and img['src'].endswith('.jpg') and 'SX679_' in img['src']:
                print(img['src'])

if __name__ == "__main__":
    asyncio.run(main())

When you run the code above without passing any JavaScript, you will get only one image, the URL of which matches the pattern that I have defined here. This pattern is just based on the page that you shared. Of course, different pages may have different patterns. However, when I pass JavaScript this time, I will get all seven images. I hope this can help.
[INIT].... → Crawl4AI 0.3.74
[FETCH]... ↓ https://www.amazon.in/Amazon-Essentials-Slim-Fit-S... | Status: True | Time: 8.48s
[SCRAPE].. ◆ Processed https://www.amazon.in/Amazon-Essentials-Slim-Fit-S... | Time: 705ms
[COMPLETE] ● https://www.amazon.in/Amazon-Essentials-Slim-Fit-S... | Status: True | Total: 9.55s
https://m.media-amazon.com/images/I/81dgvYKNQHL._SX679_.jpg
https://m.media-amazon.com/images/I/61XaglRjfmL._SX679_.jpg
https://m.media-amazon.com/images/I/81HHojfDN1L._SX679_.jpg
https://m.media-amazon.com/images/I/A1XKstKg43L._SX679_.jpg
https://m.media-amazon.com/images/I/8113QBdmTkL._SX679_.jpg
https://m.media-amazon.com/images/I/81ngh3TqglL._SX679_.jpg
https://m.media-amazon.com/images/I/91r3R3+88kL._SX679_.jpg

Just pay attention that I'm using the new version 0.3.74, which will be released tonight or tomorrow.

async def main():
    async with AsyncWebCrawler(
            headless=False,  # Set to False to see what is happening
            verbose=True,
    ) as crawler:
        result = await crawler.arun(
            url="https://www.amazon.in/Amazon-Essentials-Slim-Fit-Stretch-Vintage/dp/B07XC1Y7BL",
            cache_mode=CacheMode.BYPASS,
            js_code = """
                const delay = ms => new Promise(resolve => setTimeout(resolve, ms));

                async function clickWithDelay() {
                    const items = document.querySelectorAll('.product-media-carousel_thumbnails___mBYl');

                    for (let item of items) {
                        item.click();
                        await delay(1000);
                    }
                }

                clickWithDelay();
            """,
            delay_before_return_html=5
        )

        for img in result.media['images']:
            if img['src'].startswith('https://m.media-amazon.com/images/I/') and img['src'].endswith('.jpg') and 'SX679_' in img['src']:
                print(img['src'])

if __name__ == "__main__":
    asyncio.run(main())

I'm running the above code but still getting only one src url, am I doing anything wrong ?

@AtulKulkarni01 Please folloe the code below:
async def main():
    async with AsyncWebCrawler(
            headless=False,  # Set to False to see what is happening
            verbose=True,
    ) as crawler:
        result = await crawler.arun(
            url="https://www.amazon.in/Amazon-Essentials-Slim-Fit-Stretch-Vintage/dp/B07XC1Y7BL",
            cache_mode=CacheMode.BYPASS,
            js_code = """
                const delay = ms => new Promise(resolve => setTimeout(resolve, ms));

                // scroll to the top of the page
                window.scrollTo(0, 0);

                async function clickWithDelay() {
                    // const items = document.querySelectorAll('.product-media-carousel_thumbnails___mBYl');
                    const items = document.querySelectorAll('#altImages .a-button.a-button-thumbnail');

                    for (let item of items) {
                        item.click();
                        await delay(1000);
                    }
                }

                clickWithDelay();
            """,
            # delay_before_return_html=5
        )

        for img in result.media['images']:
            if img['src'].startswith('https://m.media-amazon.com/images/I/') and img['src'].endswith('.jpg') and 'SX679_' in img['src']:
                print(img['src'])

if __name__ == "__main__":
    asyncio.run(main())

Most of such websites, the area of images that you want to extract must be visible in the viewport. So, in this code, I scroll to the top of the page, and that's all.


Skill-Counter

Reflect physical attacks at melee range.

see what vanilla devil do

I'm gonna have to close this because the changes to FE-CLib have made it unviable for me to test this branch any further.
I'll reopen again later.


cesium for unity‘’s Tileset Source support wmts form？

Now I have some source that are wmts form。I had tried times ，cant load the source

Hi @Nguwing,
If you're still having trouble with this, please make a post on the community forum. We use Github to track bugs and feature requests related to the contents of the repository.


Missing UST from ANC-UST LP Wirhdrawal

I withdrew from the ANC-UST LP but only received the ANC portion of the 50/50 LP tokens, no UST has hit my Terra Wallet. Here is the transaction/contract:
https://finder.terra.money/columbus-4/tx/3A6D3E7F85DE374965BE12F18B423CEC2084851D82D62DA1AFB84903FEF90F69
Assistance very much appreciated.

Cancel that, UST has just arrived some 15 mins after the ANC did. Phew.


Improve api doc tests by eliminating 218 Zephyr-related checks

I also do some general cleanup.

This is ready for review again.  I addressed the overly precise check for 110 endpoints and moved a better version of that commit to the end of the PR.
I also added a commit to remove some redundant checks.

Nice, marked to merge once CI passes, thanks @showell!


Fix misspellings (lenght)

The word 'Lenght' is changed to 'Length'.

Hi, looks good, could you sign on clahub ?
Regards

all ok now, merged


Invalid character error

Hi all,
when perfoming Pull on our entire Azure structure, we recieve this error and pull fails
WARNING: [10:39:20][ConvertTo-AzOpsState] The specified AzOpsState file contains invalid characters (remove any "[" or "]" characters)! Skipping
This is happening on a single resource containing "brackets" in its name ("start-stop-vm[la-sap-vm-scheduler].json")
I have seen this FAQ @https://github.com/azure/azops/wiki/frequently-asked-questions#subscriptions-or-resources-not-showing-up-in-repository ;
Anyway there are resources created by microsoft that contains square brackets by default.
So far the only solution I found is to skip all the objects having the same type of the one causing the problem, but its quite a weird solution. Is there any way at least to simply skip one reource based on his name/ID?
Thank you all in advance.
D

Thank you for raising this, this is currently by design. We are aware of the fact that some Microsoft-created resources use these characters.
We will discuss internally how to improve this and update as soon as possible.

Thank you @daltondhcp for your quick answer. I really hope to find this new feature soon available.


Failed tests when using RetryAnalyzer are marked as ignored instead of failed

TestNG Version
7.1.0

Note: only the latest version is supported

Expected behavior
Failed tests when using RetryAnalyzer are marked as failed.
Actual behavior
Failed tests when using RetryAnalyzer are marked as ignored.

Is the issue reproductible on runner?

[ ] Shell
[ ] Maven
[ ] Gradle
[ ] Ant
[ ] Eclipse
[X] IntelliJ
[ ] NetBeans

Test case sample

Please, share the test case (as small as possible) which shows the issue

The retry impl class
public class RetryAnalyzerImpl implements IRetryAnalyzer {
	private int maxRetryCount = 3;
	private int retryCount = 0;

	public boolean retry(ITestResult result) {
		if (retryCount < maxRetryCount) {
			retryCount++;
			return true;
		}
		return false;
	}
}

The test method
@Test(retryAnalyzer= RetryAnalyzerImpl.class)
public void VerifyRetrievePublicaAndActiveCommunity(){
	int i = ThreadLocalRandom.current().nextInt(0, 2);
	System.out.println(i);
	Assert.assertEquals(i,0);
}

@gavrilfb - Couple of things:
When there's a retry analyses and there are failure attempts, then the intermediate failures are always reported as skipped (because the test is being retried once again).
If even the last attempt fails, then its status alone gets reported as "failed" and if the last attempt passes, then its status alone gets reported as "passed".
So TestNG is working as designed to report the interim test execution attempts as SKIPPED
The TestNG team does not own the IntelliJ TestNG plugin codebase. Its being maintained by the JetBrains team. So if there are any bugs that you feel need to be addressed in IntelliJ, you should be logged in https://youtrack.jetbrains.com/issues/IDEA
Closing this issue with resolution as Question answered


[Backport 1.x] Switch to log4j logger (#1751)

In OpenSearch 1.3.0 we removed an explicit dependency on log4j
https://github.com/opensearch-project/security/pull/1563 this caused the
log4j-slf4j-impl-X.XX.X.jar file no longer to be included in the plugin.
When the plugin started up the default no-op logger was used instead.
This prevented the security plugin from logging anything, yikes.
When looking at the other opensearch plugins, none of them use slf4.
Rather than continue using a seperate logging process, moving to the
standard log4j Logger/LogManager.
Tested this change on 2.0.0-alpha distribution and logging works as
expected.
Signed-off-by: Peter Nied<EMAIL_ADDRESS>(cherry picked from commit 54a920b20bd520f25aef328c1bb4997b325f46ce)
Description

Category : Refactoring

Issues Resolved

Resolves #1788

Backport PR: #1751
Testing
Check List

[ ] New functionality includes testing
[ ] New functionality has been documented
[x] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@DarshitChanpura Looks like there is a build issue with the changes

@DarshitChanpura Looks like there are still some failures. Note; I don't think we need to merge this into 1.x as we don't have plans for 1.4 at this time. How would you feel about tabling this PR?

Seems like it is failing due to check style violations. See here
yes.. we can table this PR for now!


Fixed templates issue

Reference to Issue #352

This change is

@shrutir25,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot

@acs-bot test this please

@JackQuincy could you take a look

@acs-bot test this please

LGTM just going to wait on @acs-bot  to finish before signing off.

@acs-bot  test this please

was strange acs-bot appeared to have ran the tests and they all passed but acs bot didn't realize they had ran/finished. Will try again

@acs-bot test this please

I'm suprised this merged as cole made changes to templates.go. Shruti can you pull down latest and merge with coles changes? I'm thinking we will run into issues in the merge, should be trivial to regenerate, but we should do that before we merge.

@acs-bot test this please

@acs-bot test this please

@acs-bot test this please

I can't tell if the test failures are caused by this review or if it is flakiness. It appears to be flakiness but I'm not sure.


test: Update tests that use proto DebugString to use regex

Additional Description:
Upcoming updates to proto library use different whitespace between name and value pairs.This change preemptively modifies test to use regex that can match both new and old versions on proto library.
Risk Level: Low, Test Only
Testing: Unit Test
Docs Changes: N/A
Release Notes: N/A
Platform Specific Features: N/A
Signed-off-by: Yan Avlasov<EMAIL_ADDRESS>

/retest

/assign-from envoyproxy/first-pass-reviewers


Create a simple app to fetch data asynchronously on form submit

Tasks

[x] Use axios on another, to compare the two
[x] Fire one async data call with a submit event
[x] Use the data to create a visual, like a list or table (why not both?)

Related Issues
#9

Closing with commit 5c885fe


Cut over from Girder to Flask

This is a big PR that we need to handle carefully. It swaps out Girder for Flask. The following changes still need to be made:

[x] merge in all changes from master and ensure they work in the flask app
[x] fix all versions in Pipfile
[x] remove Girder and move the flask files into the multinet directory
[x] remove Girder dependencies from Pipfile
[x] remove Mongo from the docker-compose setup
[x] rename the repository to "multinet"
[x] update documentation

~Currently, if you check this branch out, re-run pipenv install, then run flask run in the top level, the flask app will run on port 5000, and the client app should run properly without any changes.~ The usual sequence of commands will now start the server and client as usual (a few steps that had to do with building the Girder client, and setting up a user, are no longer necessary).
@AlmightyYakob, I will probably want to do some pair programming to get all issues squared away before final merge.

I reverted your quotation mark commit and instead kept the multiline string, but used double quotes to delimit it.
flake8 was complaining about the use of triple single-quotes, rather than the multiline string itself. I prefer the multiline string because it's far more readable than the concatenated single-line strings.


Deezer issue

I was using Material skin till now.
Before v2.9.1 my Deezer playlists where also in the global playlists. So I was able to get them using the LMS CLI.
With v2.9.1, Deezer playlists are only reachable via the Deezer app and don't appear anymore in the my music -> playlists.
Tried to uninstall Material Skin and the deezer playlists, after a while, where again in my playlists.
Installed again Material Skin, and playlist disappeared after a while.
Is there something I am missing (configuration ?)
Thanks

Sorry, but nothing has changed in material 2.9.1 that would effect this.

I was able to get them using the LMS CLI

How? There is no way Material could affect access to LMS from other methods. So, you are stating that before 2.9.1 you could use some other method to access deezer playlists, but after installing MaterialSkin 2.9.1 this stopped? I find that very hard to believe. Just looked at the diff between 2.9.1 and 2.9.0 - nothing related to playlists.


Add processes endpoint

Relates to Issue #6
Adds /processes endpoint to query availabled openeo Processes from openeo-processes dask

Yeah, minor comments, otherwise lgtm!


WIP - bringing over posh-git package

WIP reminder
https://github.com/ferventcoder/chocolatey-packages

Close this thing once you get this figured out.

0.5.2 worked well enough that I forgot this was still open. Closing with a bump to 0.6.1.


Qmaps 2289 history in directions

Description

Forked @bbecquet 's branch QMAPS-1689
Discussed with the team that Direction's history should actually be shared with TopBar's history (at least for now, let's see if it's ok in plive)
Converted DirectionInput to a Functional React Component (pretty sure I didn't break anything)
Enabled history display inside Direction's suggests
TODO: save the queries typed/selected inside the directions inputs inside the global history (as we do in TopBar)
TODO: check/finalize style once Benjamin's branch is merged.
TODO (@bbecquet) : we should probably find a way to display both history and favorites items inside the suggests,. Currently, if an item is in favorites but not in the history (it can happen), it is not listed at all. It could be present before or after the history's items, no?

Screenshots

@xem this is dangerous to mix a non-trivial refacto (class to functional) with the feature development. This component has tricky focus side-effects, and we must be extra sure nothing breaks.
At the very least it should be done in two separate commits, but better still, it would make a great separate PR now that you want to tackle the subject :)


fix(proxy) do not clear upgrade header (case-insensitive), fix #4779

Summary
Ensures that Upgrade header is not cleared on websocket. This is the second patch after the initial patch PR #4719. It seems like some servers send back Upgrade: WebSocket instead of Upgrade: websocket which caused the proxy to clear the Upgrade header. This ensures that the value is compared case-insensitive.
The test is missing as it is a bit tricky to implement that as our mock websocket server always sends the Upgrade header back as lower case websocket which does not cause this issue. But the fix in itself is rather simple.
Issues resolved
Fix #4779

@bungle @thibaultcha @gszr Was the intent to release 1.2.2? Out websocket services are broken in Stage on 1.2.1 and we need this adjustment I reckon. I checked luarocks, docker-kong repo and bintray and 1.2.2 was never released.


[DDW-364] remove theme decorator from component stories

This PR removes the decorateWithSimpleTheme decorator from all component stories.

decorateWithSimpleTheme is kept but the directory in which it lives changes from stories/helpers to stories/decorators for use in the future.

Note: The components don't need to be wrapped in an instance of ThemeProvider because they will automatically fall back on their defaultContext. This is intended behavior and cuts back on the amount of code running in all the stories. I've also attached a short video to the associated YouTrack issue for clarification if needed! 😸

@MarcusHurney yes but the defaultContext theme is empty now … so that can't work?

ah sorry … just saw that this PR was closed ;)


[nodejs] fix ranking csv bug

fix bug where ranking table csv is in the wrong order
updated response for family earnings in california counties
https://paste.googleplex.com/6309252029743104

thanks for the review!


Add a VScript func to sanitize variant values (3.2)

Backport of #41861 to 3.2. Main difference is the CallError going from Callable:: to Variant::.

Thanks!


Two way SSL

Is there a way to use two way SSL with dockerode as described here?
Currently I'm able to make this work manually with the standard https library built into node:
var https = require('https'),
    fs = require('fs');

var options = {
    hostname: 'ec2-54-187-244-245.us-west-2.compute.amazonaws.com',
    port: 2376,
    path: '/info',
    method: 'GET',
    key:  fs.readFileSync('./certs/client-key.pem'),
    cert: fs.readFileSync('./certs/client-cert.pem'),
    ca:   fs.readFileSync('./certs/ca.pem')
};
options.agent = new https.Agent(options);

var req = https.request(options, function(res) {
    console.log("statusCode: ", res.statusCode);
    console.log("headers: ", res.headers);

    res.on('data', function(d) {
        process.stdout.write(d);
    });
});
req.end();

req.on('error', function(e) {
    console.error(e);
});

However, the code above is not ideal and I'd much rather use dockerode but I don't see any documentation indicating the ability to use required authentication PEM files.

@apocas @ReedD do you know how to generate ca.pem, cert.pem and key.pem into separate 3 files ?


[1.16.5] [forge] Add compat with Untamed Wilds spawn - WYML doesn't handle it well.

For a while now untamed wilds has had an issue with stabilizing their entity count. Recently the dev have done a bit better job with it but it still struggles on the server side and I would love if WYML could be more compatible with it!
The reason I'd love if WYML could take care of untamed wilds spawn is for various reasons but mainly because I love how well WYML does its job. The ticket on their github #4 was posted months ago and still has not had a solution for it since.
I've also taken the time to list all the mob ids so it could hopefully be a bit easier for the WYML team to address this issue and hopefully add better compat for it? Best regards, Cixon!
mob ids:
WATER_AMBIENT(s)
untamedwilds:aardvark
untamedwilds:arowana
untamedwilds:football_fish
untamedwilds:giant_clam
untamedwilds:giant_salamander
untamedwilds:shark
untamedwilds:softshell_turtle
untamedwilds:sunfish
untamedwilds:trevally
AMBIENT_CREATURE(s)
untamedwilds:bear_black
untamedwilds:bear_blind
untamedwilds:bear_brown
untamedwilds:bear_cave
untamedwilds:bear_panda
untamedwilds:bear_polar
untamedwilds:bear_spectacled
untamedwilds:bear_sun
untamedwilds:bigcat_cave_lion
untamedwilds:bigcat_jaguar
untamedwilds:bigcat_leopard
untamedwilds:bigcat_lion
untamedwilds:bigcat_marsupial_lion
untamedwilds:bigcat_puma
untamedwilds:bigcat_snow_leopard
untamedwilds:bigcat_tiger
untamedwilds:boar
untamedwilds:hippo
untamedwilds:hyena
untamedwilds:large_snake
untamedwilds:newt
untamedwilds:rhino
untamedwilds:snake
untamedwilds:tarantula
untamedwilds:tortoise

We can give it a try, we was planning to do some settings per mob category, so we will probably use the issue in this mod as a test case.

excellent! thank u!

Done something similar to what you asked, but more... Expansive.
On first world load, it'll generate configs in config/wyml-SpawnRules
Each config in there will be the name of the mod that added it.
Inside that is an object for each mob type (So say, WATER_AMBIENT
Then inside that will be an object for each mob, with a limit that is how many can spawn in a single chunk
This will, by default, do nothing.
In the main config, there is an open to tell it to respect these configs (ENABLE_PER_MOD_CONFIGS)
Once enabled, this will apply those limits to any natural spawns.
However, if for some reason whatever mod is not using natural spawns and is adding the entities directly (like mods like soul shards do) then you can enable "HARD_MOB_LIMITS", this will then intercept entity adds and apply the limits there too.
Let me know if you encounter any issues.


Simple Title Change (Distribution Manifest Page)

Change Title on Distribution Manifest Page (has nothing to do with diaper drives).
Change to: "Total Items Distributed".

Expected behavior
N/A
Actual behavior
N/A
Steps to reproduce the problem
N/A
Computer Information (Only fill this out if you are a diaper bank)
MacOS High Sierra
10.13.4
Browser: Chrome

Image is from master - issue resolved.

Image from master:


"You must enable preview features" prompt should have option to enable preview features

SQL Operations Studio Version: master

Steps to Reproduce:

Disable preview feature and run Restore

Expected: If told I need to enable preview features the message provides ability to enable these features or a link to documentation on how to do so.
Actual: Message doesn't provide these options

Note: We should reword the message to be something like "Preview features must be enabled to use this feature.  Would you like to enable preview features now?  Yes\No"

Closing out older issues.


Z2-Converter API Mainnet : GetBalance

Issue Summary:
Calling GetBalance on the ZQ1 and ZQ2 has the following diff.
Here are the URLs for ZQ1 and ZQ2.
ZQ2 : http://<IP_ADDRESS>:4201/
ZQ1 : http://<IP_ADDRESS>:4201/
Result:
Missing keys:

Different Keys:
Values that differ between ZQ2 and ZQ1:
  root['result']['balance']:
    ZQ2:<PHONE_NUMBER>836755835157
    ZQ1:<PHONE_NUMBER>927233584237

Observations:
When i query an account balance - zil1ascnfxmlzl4z3l6ecavxcae5hc37gdzr23enz0 , the API returns a match. However, I used the staking contract address -zil15lr86jwg937urdeayvtypvhy6pnp6d7p8n5z09  , and then there is a mismatch.
Conclusion:
If the above differences are expected and align with the migration to ZQ2, no further action is required. However, if these differences are unexpected, they may indicate issues that must be addressed.

The issue occurred due to the state deltas not being migrated on ZQ2. To address this, we need to ensure that we use the backup taken after the MoveUpdateToDisk operation at the 99th block, with transactions paused before. This approach will guarantee that no state deltas remain in ZQ1's persistence. Do you have any other suggestions, or could you point out any gaps in my understanding?
cc @bzawisto @JamesHinshelwood @DrZoltanFazekas

As discussed, we shall do what you described above @chetan-zilliqa

@chetan-zilliqa Is this fixed now?

Closing the issue. The issue is fixed.


[SPARK-49566][SQL] Add SQL pipe syntax for the EXTEND operator

What changes were proposed in this pull request?
This PR adds SQL pipe syntax support for the EXTEND operator.
This operator preserves the existing input table and adds one or more new computed columns  whose values are equal to evaluating the specified expressions. This is equivalent to SELECT *, <newExpressions> in the SQL compiler. It is provided as a convenience feature and some functionality overlap exists with lateral column aliases.
For example:
CREATE TABLE t(x INT, y STRING) USING CSV;
INSERT INTO t VALUES (0, 'abc'), (1, 'def');

TABLE t
|> EXTEND x + LENGTH(y) AS z;

+----+-----+-----+
|  x |   y |   z |
+----+-----+-----+
|  0 | abc |   3 |
|  1 | def |   4 |
+----+-----+-----+

Like the |> SELECT operator, aggregate functions are not allowed in these expressions. During the course of developing reasonable error messages for this, I found that the SQL pipe syntax research paper also specified that the |> AGGREGATE operator should require that each non-grouping expression contains at least one aggregate function; I added a check and reasonable error message for this case as well.
Why are the changes needed?
The SQL pipe operator syntax will let users compose queries in a more flexible fashion.
Does this PR introduce any user-facing change?
Yes, see above.
How was this patch tested?
This PR adds a few unit test cases, but mostly relies on golden file test coverage. I did this to make sure the answers are correct as this feature is implemented and also so we can look at the analyzer output plans to ensure they look right as well.
Was this patch authored or co-authored using generative AI tooling?
No

This PR now just covers the EXTEND operator.

cc @gengliangwang @cloud-fan this one is ready for review at your convenience.

thanks, merging to master!


Creating Awareness for Sales Funnel

User Story-
I want to create awareness for our sales funnel among potential customers to educate them on our products, nurture leads, and ultimately drive conversions.
Actions-
Clearly define the goals of creating awareness for the sales funnel, including increasing brand visibility, educating potential customers, and attracting qualified leads.
Results-
Achieve increased awareness among the target audience regarding the sales funnel, ensuring that potential customers understand its importance in their purchasing journey.

Awareness: This business address points including basketballs, production summary, and content of the basketballs we are selling along with promotions and likeability. Social Media Content that engages buyers why they should purchase this product compared to other brands. This is where the user will find our website when looking up basketballs for purchase.