Add a modern web UI based on React, MaterialUI and Vite

Changes Proposed

Add a new boolean argument modern_ui (--modern-ui, LOCUST_MODERN_UI). When true, will use the new frontend
Have UIExtraArgOptions returned as a dict. This is needed as otherwise the NamedTuple is returned as an Array when JSON stringified. This is tested and working in the old web ui
Modify the response from /stats/reset to be JSON. I propose this change to be consistent with all other responses from the API (without this change we may see console errors in the new UI as it expects responses to always be JSON)
Add React frontend using Typescript and Material UI. The UI supports all existing features from the old UI
Add an example of how it can be possible to extend upon the new modern UI
Add documentation for developing the dashboard

Fixes #2396
Demo
Light and dark mode:

Advanced options and custom arguments:

About Modal:

Running Test:

Exceptions:

Second Test Markers:

Class Picker:

Custom Shape:

Custom Shape with Class Picker:

Multiple Hosts:

Full Requests Statistics History:

With Workers:

With examples/extend_modern_web_ui.py

@heyman @KasimAhmic I know both of you suggested using Vite for this project, would either of you be able to assist in changing the Webpack code to Vite? As I mentioned, I am not at all familiar with the tool. I tried with the following config:
import reactSwcPlugin from '@vitejs/plugin-react-swc';
import { defineConfig, splitVendorChunkPlugin } from 'vite';
import checkerPlugin from 'vite-plugin-checker';
import tsconfigPaths from 'vite-tsconfig-paths';

// https://vitejs.dev/config/
export default defineConfig({
  root: 'src',
  publicDir: '../public',
  plugins: [
    tsconfigPaths(),
    reactSwcPlugin(),
    splitVendorChunkPlugin(),
    checkerPlugin({
      typescript: true,
      eslint: {
        lintCommand: 'eslint --ext .ts,.tsx',
      },
    }),
  ],
  build: {
    outDir: '../dist',
    emptyOutDir: true,
  },
  server: {
    port: 4000,
  },
});

This unfortunately would not load any images properly (I even tried directly importing, such as they suggest here).
I additionally couldn't seem to find any replacement in Vite for the Webpack watch command. Since the locust instance will read from the dist, we need to have the build emitted for development while running  a locust instance.
Speaking of development - I propose in Webpack to use two different html files, one for developing using the Webpack dev server (it uses test data for the template arguments so we do not need to have a Locust server running) and one for production / developing with a Locust instance. Could the same be easily achieved in Vite?

Checked out your code and got it working with the config below:
import reactSwcPlugin from '@vitejs/plugin-react-swc';
import { defineConfig, splitVendorChunkPlugin } from 'vite';
import checkerPlugin from 'vite-plugin-checker';
import tsconfigPaths from 'vite-tsconfig-paths';

// https://vitejs.dev/config/
export default defineConfig({
  plugins: [
    reactSwcPlugin(),
    tsconfigPaths(),
    splitVendorChunkPlugin(),
    checkerPlugin({
      typescript: true,
      eslint: {
        lintCommand: 'eslint --ext .ts,.tsx',
      },
    }),
  ],
  build: {
    outDir: '../dist',
    emptyOutDir: true,
  },
  server: {
    port: 4000,
    open: './dev.html'
  },
});

Main changes are:

Removed the root and publicDir options
Added the open option pointing to the dev.html file

In addition to the config, I also moved the index.html and dev.html files from the src directory to the dashboard directory as per Vite's requirements. Similarly, you will need to add the following line in each HTML file.

Because we specified the dev file in the server configuration, this should cover your use case of having two HTML files as I believe Vite will still use index.html on production builds.
Regarding watch mode, Vite does have it built in. You can use it like so: vite build --watch
And lastly, with regard to images, when loading images from the public directory, you would reference them as if the images are available at the root of the site like so:
-
+

@cyberw There is one other thing that I believe is missing here. I believe for the UI to be properly bundled when pip install locust is ran, we need to have some additional configuration in the pyproject.toml. The output of the project, dashboard/dist is being gitignored, meaning it will not exist when others install the project. Do you know of some way that we could have setup tools build the dist on install? Or do we need to store the dist in the project? It seems like when we were using SASS the bundled CSS was being uploaded to github right?  I am unfortunately not so familiar with this side of the Python workflow

Hey @KasimAhmic thanks so much for the assist! Looks like the --watch flag works great and I was able to configure the HTML file for dev / prod using the following:
import reactSwcPlugin from '@vitejs/plugin-react-swc';
import { defineConfig, splitVendorChunkPlugin } from 'vite';
import checkerPlugin from 'vite-plugin-checker';
import tsconfigPaths from 'vite-tsconfig-paths';

const PRODUCTION = process.env.PRODUCTION === 'true';

// https://vitejs.dev/config/
export default defineConfig({
  plugins: [
    reactSwcPlugin(),
    tsconfigPaths(),
    splitVendorChunkPlugin(),
    checkerPlugin({
      typescript: true,
      eslint: {
        lintCommand: 'eslint ./src/**/*.{ts,tsx}',
      },
    }),
  ],
  server: {
    port: 4000,
    open: PRODUCTION ? './index.html' : './dev.html'
  },
});

The images do work when running the dev server, however there seems to be an issue with how the assets are placed in the dist. Try running PRODUCTION=true yarn vite build --watch in one terminal and then run a locust instance in another: locust -f locust.py --modern-ui

@andrewbaldwin44 Couple things

I ran into the same issue with the production build actually. This feels like it might be a Flask configuration issue though, I saw something about Flask only checking a static directory for static assets. Is it possible to change this?

You can get rid of this line: const PRODUCTION = process.env.PRODUCTION === 'true'; and replace the value passed to defineConfig with a function that takes a configuration object as an argument. For example:

import reactSwcPlugin from '@vitejs/plugin-react-swc';
import { defineConfig, splitVendorChunkPlugin } from 'vite';
import checkerPlugin from 'vite-plugin-checker';
import tsconfigPaths from 'vite-tsconfig-paths';

// https://vitejs.dev/config/
export default defineConfig((config) => ({
  plugins: [
    reactSwcPlugin(),
    tsconfigPaths(),
    splitVendorChunkPlugin(),
    checkerPlugin({
      typescript: true,
      eslint: {
        lintCommand: 'eslint ./src/**/*.{ts,tsx}',
      },
    }),
  ],
  server: {
    port: 4000,
    open: config.mode === 'production' ? './index.html' : './dev.html'
  },
}));

The server configuration is only used in development mode so I think you should be able to delete the conditional open configuration and just leave it as open: 'dev.html'. Could be wrong, never did something like this before.

@cyberw There is one other thing that I believe is missing here. I believe for the UI to be properly bundled when pip install locust is ran, we need to have some additional configuration in the pyproject.toml. The output of the project, dashboard/dist is being gitignored, meaning it will not exist when others install the project. Do you know of some way that we could have setup tools build the dist on install? Or do we need to store the dist in the project? It seems like when we were using SASS the bundled CSS was being uploaded to github right? I am unfortunately not so familiar with this side of the Python workflow

I dont know the best way to do this off hand, and I'm kinda busy atm. Let me know if you're really stuck on it and I'll have a look.

@andrewbaldwin44 For adding the files, you should just need to include them in MANIFEST.in, like this:
recursive-include dashboard/dist *

The failures page is missing the "type" column? (but lets change the column name to "Message" or "Error message", while we are at it)

old school:

Thanks everyone! @cyberw updated now with the missing "Message" column and the missing line in the MANIFEST.in

Thanks @KasimAhmic you were correct, I have Flask configured to server assets from /assets and Webpack was copying the assets to dist/assets but Vite was not. The dashboard is now using Vite for compiling 👍

Cool stuff! Can you add a basic test case in test_web.py as well?

@cyberw Sure, I can add a test to ensure the correct behavior is happening with the --modern-ui flag. For testing the new web ui, we should do it using React Testing Library. That part will take time, do you prefer me to do it in this feature, or in a future ticket?

@cyberw Updated now to support run_time from CLI as default arg (spawn_rate was already supported in the new UI)

From a code perspective this looks really good, apart from the flake8 and black fails :) (I really should make them pre-commit checks)
I think we should mention the availability of the modern UI explicitly in the manual. I’m thinking maybe as a note at the end of the web ui section of quickstart? And make it clear that it is experimental.

Apologies for the failing pipeline, I committed the new tests a bit too quickly! The pipeline is passing now and I have added a line to the quick start guide mentioning the new web UI as well as a warning that this feature is experimental.

Couple things:

If you refresh the page, everything disappears. The view is not updated until a new request comes in (I think, or maybe it is some other delay)
The --users parameter is not prefilled correctly (it is "1" no matter what value I set on command line)
If I use --autostart the UI still shows the "new" button (when it should show stop/edit)

Also, the reports download page still looks like its the old version:

@cyberw I've resolved the issues mentioned above. The issue with the users not autofilling was because the value provided by the command line is num_users, however the value that needs to be passed to /swarm is user_count. I think we may need to review the naming of these variables at some point :)
As for the 'modern' html report, I have looked into it and I think there is a fair amount of additional changes or refactoring that would need to be made to the backend for the feature to work (for example the stats and exceptions would need to be json serializable, the logic for generating the percentile data would need to be moved to the backend, and the logic for adding the javascript into the html file would need to be a bit different since the name of the js files will change with every build). After all of this we will probably want to add some additional tests as well. What do you think, would it be better to have this as a separate ticket? Or do you prefer to have everything done together?

If it is a lot of work then we can leave it for later. I’ll do a few more tests tomorrow or so.

I'm still experiencing some issues. If I launch with --autostart (or open a new browser window against a running test), there are no updates at all (except if I reload)..

@cyberw The stats are now being properly fetched even while users are spawning and the issue with the text escaping in the failures table has now been resolved as well

Nice! Seems to have introduced a new issue though: After starting the test, the UI sometimes switches back to the "start load test" page for a second.

Once those two things are fixed I think we're good to go (as this is still opt-in I'm not too worried about bugs ets).
Maybe @mquinnfd @KasimAhmic or @heyman would like to chime in. Especially regarding anything structural, or any other things that are important to get right from the start.

I suspect locust/webui/dist/** needs to be purged and added to .gitignore, seems to be some built artifacts sneaking in to the refs there

I suspect locust/webui/dist/** needs to be purged and added to .gitignore, seems to be some built artifacts sneaking in to the refs there

@mquinnfd Yes I wasn't sure how this should be done in a Python workflow using pyproject.toml. For the dashboard to be available to the users, the static files need to built and bundled with locust. I noticed that even with Sass we were commiting the compiled CSS

I've left comments on some of the files and I have some more general notes.

I'm curious about the decision to use the connect() pattern for the connecting the components to the Redux store. Why did we go this route as opposed to creating selectors for the various pieces of the state?
I dislike the use of the sx prop. While it is what MUI recommends, I find it a bit limiting at times and it makes the code messier IMO. I would prefer if we switched to another CSS-in-JS solution like tss-react. Open to comments on this one though.

Overall, the PR looks good for a first pass save for a few minor things outlined in the comments. Once those are addressed, I'd have no issues merging this and then additional improvements like using createApi for requests and using Redux selectors as opposed to the connect approach can be made later.

@KasimAhmic

Yes the use of connect is perhaps a personal preference of keeping the concerns separate, rather than polluting the component with hooks. I prefer to keep the components quote 'dumb', in that they should be able to receive a list of props and not need to know where to get the props from. I have worked on projects using createStructuredSelector from reselect for example, however this seems to be a feature missing from RTK. Is there ever a time where we may particularly need the memorization from selectors, or do you not find the connect function to be explicit enough?
Regarding the use of MUI's sx, I was actually opting for this because it is limiting. I wanted to avoid using custom CSS as much as possible in this project (the more custom css is allowed, the more magic numbers pop up, and the less 'standard' things look). You'll notice I limit the sx properly to things such as positioning or flex direction / centering and any spacing used is using the MUI grid system (rather than random pixel values). I think there's not really a need for custom CSS, nor do we really want to encourage it. Not to mention this saves us the use of another library

I suspect locust/webui/dist/** needs to be purged and added to .gitignore, seems to be some built artifacts sneaking in to the refs there

@mquinnfd Yes I wasn't sure how this should be done in a Python workflow using pyproject.toml. For the dashboard to be available to the users, the static files need to built and bundled with locust. I noticed that even with Sass we were commiting the compiled CSS

@andrewbaldwin44 Ah, I get you. Are we just looking for a sort of "on-build" hook in poetry that we can use to run the yarn build then? I'll have a look at any sensible options for this

P.S I'm amazed you got this together so quickly @andrewbaldwin44, this is all looking really great, truly epic effort 👍

A side note: Having a tab for viewing the log/console output of locust in the UI would be an absolute killer feature (maybe even having it light up if there are warning/error loggings etc - it is all too common that people are only looking in the UI and dont see issues that are reported on terminal). But maybe for 2.0 of this UI :)

On the build process, I note that Locust currently uses plain build to generate the released .whl.
It looks as though this tool is designed to be as simple as possible, so I don't really see pre/post build hooks or ways to extend the behaviour (by design).
I suppose we have a few options, and it depends on the requirements. This is not exhaustive but a few ideas which @cyberw will probably have strong feelings about, having been around during the project's evolution.

Use Poetry: Given that we already maintain pyproject.toml - it might be an idea to consider moving to poetry (which is close to being the new build/packaging standard). This would allow for running yarn as part of a prebuild subprocess a bit like this. This would change the build process a little bit, we could either just use poetry to build the artifact, and publish it in the same way, or use poetry to publish it as well. There are obviously a fair few implications of changing the established build procedure for the product, however, and it would need to be thoroughly regression-tested - though it would make the local and remote build/package procedures more unified (as opposed to encapsulating the front end build/dist logic in Github actions).

Separate local and remote builds: If we care less about local and remote build processes being unified, then in the publishing workflow within Github actions, we could have a multi-step workflow, and pass the artifact between the steps like so, keeping a node-specific action runner for the yarn build and then trying to bundle this into the built Python .whl as part of a series of actions. Locally, we could then just adapt the Makefile to run both builds as part of the build process, and if necessary for testing, write some bash to do this multi-build stitching process to generate a representative .whl file.

Back to setup.py: Move back to a setup.py file, which is less in-vogue, may have incompatibilities with what we're currently doing tools-wise, but would most certainly allow for arbitrary build logic like this.

Store generated files: Commit the compiled files to the repository, which is a little grim for obvious reasons.

On the build process, I note that Locust currently uses plain build to generate the released .whl.
It looks as though this tool is designed to be as simple as possible, so I don't really see pre/post build hooks or ways to extend the behaviour (by design).

IMO, Poetry is only good for/worth the hassle for applications (where pinning exact dependencies makes sense) not frameworks (where users may add extra dependencies or want to use different versions of dependencies etc).
Kind of sad, but possible.
Yea, that could totally work. I really dont like some of the limitations in pyproject.toml.
🤢

So I guess what I'm saying is going back to setup.py makes the most sense to me.

I'm not the biggest poetry fan, but 1.2+ has been pretty good, we switched our production stuff over to it (frameworks, applications, everything). It's only worth the hassle if you're creating packages for publication, etc.
Practically, just reverting to the old setup.py would be convenient, but I do feel that in general, the Python community is moving away from setuptools etc. which would make this a bit of a step backwards, and potentially hamper future development.

pip's own docs about the subject
build's own docs about the subject
Older PEP-517 vs Newer PEP-518

I could find some time to make a PR branch for each system I suppose (poetry, setuptools), and see which feels better for this use-case?

Yes I am also not a fan of tracking the dist, however I think it may be out of scope to change the build process in this ticket. What do you suggest @cyberw? Do we create another PR to revert #2336 before merging this one?

Hmm... how much space would it require if we were to commit them (option 4), as a temporary workaround?

@cyberw Yeah that's what I had already done, right now we're tracking the dist. It was the only simple possible solution :/

I could find some time to make a PR branch for each system I suppose (poetry, setuptools), and see which feels better for this use-case?

Imo, lock files make little sense for frameworks, and poetry is very centered around its lock file.
I'd love a PR for both options, but I feel it is unlikely that poetry will fit my taste :) (rn I dont think it even works, see https://github.com/locustio/locust/issues/2356, but that would be nice to fix of course)

@cyberw Yeah that's what I had already done, right now we're tracking the dist. It was the only simple possible solution :/

Ah, I thought there was something more needed. In that case we leave it as is for now.

@andrewbaldwin44 I'd like to make a new release of locust (primarily adding support for Python 3.12) within a few days. Are there any final adjustments we can/should make before merging this, or do you feel it is more than a few days work left and I should just make the release first?

@cyberw Sorry I have been a little busy the last couple days, I have some time tonight to fix the bug that you were experiencing, but I may need a little more time for re-writing the API logic as @KasimAhmic and @KasimAhmic suggested

No worries.
It might even make sense to have this in a release of its own (so we dont confuse any bugs), and it makes sense to have it as "git/prerelease only" for a while too. I'll probably make a new locust release (with all the other stuff) this weekend or so.

Hey guys, I have just pushed an update that should resolve all the comments posted above. I will still need some more time for getting the API logic working with RTK, I will need to look into it more next week. I think using RTK may be a bit tricky for certain requests, namely /stats/requests, as we want the response from this request to be placed into several different reducer paths, as well as some other logic that needs to be handled. What do we think about using an API solution such as axios to handle this?

tests on python 3.8 are flaky for some reason. ignore that.

Sorry for the delay everyone, all the comments are now resolved and I have replaced the API logic with RTK's createApi. Thanks for the feedback! :)

Nice! Can you update the pr description? I think the following is no longer relevant and maybe there are other things to add.

Modify the response from /stats/reset to be JSON. I propose this change to be consistent with all other responses from the API (without this change we may see console errors in the new UI as it expects responses to always be JSON)

@cyberw Yes once this is merged I can open a PR for updating the HTML report and then a third PR to introduce a suite of tests using React testing library. From there I am happy to support with the community requested features, such as boolean custom arguments and having a tab for viewing the logs :)

How to get this new html report when using headless model?

@mquinnfd Does your offer to implement poetry to build the frontend still stand? If so, I'm more interested now :)
I've heard more good stuff about it (and the bad stuff I heard/inferred was maybe a little misguided).
The one thing I want to make sure is that we dont lose the ability to do an editable install (-e) that can be used even outside the directory/venv.


owners: maintainer updates

Remove Dmitry
Add Kuat

Also other CODEOWNERS cleanups.

Very sad to see you go @rojkov :cry:
Welcome @kyessenov and thank you! :tada:
cc @envoyproxy/envoy-maintainers


[QnA Maker, Question] Problem with new QnA bot

I created a new web app bot with question and answer from the azure portal and i created a services qna for this web app. i have a qnamaker created , but is the old version, the preview.
In the settings application it requires auth key what is in the new qnamaker.
How do I run the old version, at the moment I do not want to migrate the old qnamaker?

@bettina86 which SDK and version of the SDK are you using?

I'm using the platform azure and his editor online.

@bettina86 the QnAMaker Dialog will default to the preview (v3) QnA Maker API. There are no major differences in using the v3 BotBuilder SDKs with QnA Maker. Under the App Service Settings tab for your web app bot, you simply need to provide it with two values for these two keys:

QnAKnowledgebaseId
QnAAuthKey


[Bug]: Failing Go Zitadel Setup

Preflight Checklist

[X] I could not find a solution in the documentation, the existing issues or discussions
[ ] I have joined the ZITADEL chat

Environment
Self-hosted
Version
v2.39.3
Describe the problem caused by this bug
Failing ./main.go:43:59: undefined: http_mw.Token
I am following your docs demo
To reproduce
package main
import (
"flag"
"log"
"net/http"
"time"
http_mw "github.com/zitadel/zitadel-go/v2/pkg/api/middleware/http"
"github.com/zitadel/zitadel-go/v2/pkg/client/middleware"

)
var (
issuer = flag.String("issuer", "", "issuer of your ZITADEL instance (in the form: https://.zitadel.cloud or https://)")
)
func main() {
flag.Parse()
introspection, err := http_mw.NewIntrospectionInterceptor(*issuer, middleware.OSKeyPath())
if err != nil {
    log.Fatal(err)
}

router := http.NewServeMux()
router.HandleFunc("/public", writeOK)
router.HandleFunc("/protected", introspection.HandlerFunc(writeOK))

lis := "<IP_ADDRESS>:5001"
log.Fatal(http.ListenAndServe(lis, router))

}
func writeOK(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("OK " + time.Now().String()))
}
Screenshots
No response
Expected behavior
No response
Operating System
No response
Relevant Configuration
No response
Additional Context
No response

@Qodestackr Closed as of inactivity and problem should be solved through the SDKs, if not so please reopen the issue or a new issue.


Enhanced support for surface finish characteristics

More support is required for surface finish characteristics.
The element SurfaceTextureCharacteristicDefinition has the mandatory field RoughnessAverage (Ra) but there are many more surface finish characteristics that need representation. Depending on the parameter being evaluated there might be other parameters that need to be defined in addition to the surface finish characteristic. Examples are the type of filtering applied, use of the 'max' rule, short and long cut off values, the lay direction, the limit type (upper, lower or bilateral) a second target value for bilateral tolerances, bearing ratio etc.
Another detail that a user might wish to see is the manufacturing method, for example grinding, honing, polishing, buffing etc.

Good point. I believe that Capvidia will be putting together some ideas for improvement in this area as well.

There has been changes in surface texture standards.
ASME Y14.36-2018 & B46.1-2019 are currently the latest from ASME.

QIF member recruitment opportunity?
Mark Malburg at Digital Metrology Solutions is very active in surface texture standards and the surface texture industry.
Maybe others as well?


ERROR in this. symbolResolver. getSymbolByModule is not a function

While compiling i got this error
chunk    {0} polyfills.bundle.js, polyfills.bundle.js.map (polyfills) 158 kB {4} [initial] [rendered]
chunk    {1} main.bundle.js, main.bundle.js.map (main) 5.47 kB {3} [initial] [rendered]
chunk    {2} styles.bundle.js, styles.bundle.js.map (styles) 9.77 kB {4} [initial] [rendered]
chunk    {3} vendor.bundle.js, vendor.bundle.js.map (vendor) 2.36 MB [initial] [rendered]
chunk    {4} inline.bundle.js, inline.bundle.js.map (inline) 0 bytes [entry] [rendered]

ERROR in this.symbolResolver.getSymbolByModule is not a function

Angular version 4.0 and angular/angualr-cli version 1.0

Last night my package was Ok,
but today I installed "@angular/platform-server": "^4.0.2" and got this ERROR too.
ng serve runs fine, but build fails :(

npm update -D && npm update -S

did the trick, 4.02 out :D

@matheo  I'm having the exact same issue but running the commands which you provided doesn't help at all, the error remains the same.
Did you by any chance do anything extra ?

I also needed to make sure that the "@angular/compiler-cli" vrsion matched the @angular version. In my case the version was upgraded to "^4.0.2"

I also have this issue, but then I re-save project in the editor, and 'ng serve' automatically re-compile again and the error disappears, but it is still strange

My project has started doing this as well. At least it runs but I am concerned about automatic builds not working.

Seems like this was a dependency problem. Thanks for everyone that helped out!

I have the same problem

My package.json is this:
{
  "name": "hello world",
  "version": "0.0.0",
  "license": "MIT",
  "angular-cli": {},
  "scripts": {
    "start": "ng serve",
    "lint": "tslint \"src/**/*.ts\"",
    "test": "ng test",
    "pree2e": "webdriver-manager update",
    "e2e": "protractor"
  },
  "private": true,
  "dependencies": {
    "@angular/common": "2.4.8",
    "@angular/compiler": "2.4.8",
    "@angular/core": "2.4.8",
    "@angular/forms": "2.4.8",
    "@angular/http": "2.4.8",
    "@angular/platform-browser": "2.4.8",
    "@angular/platform-browser-dynamic": "2.4.8",
    "@angular/router": "3.4.8",
    "core-js": "^2.4.1",
    "rxjs": "5.1.1",
    "ts-helpers": "^1.1.1",
    "zone.js": "^0.7.7"
  },
  "devDependencies": {
    "@types/jasmine": "^2.2.30",
    "@angular/cli": "1.0.0-beta.32.3",
    "codelyzer": "~2.0.1",
    "jasmine-core": "2.5.2",
    "jasmine-spec-reporter": "3.2.0",
    "karma": "1.4.0",
    "karma-chrome-launcher": "^2.0.0",
    "karma-cli": "^1.0.1",
    "karma-jasmine": "^1.0.2",
    "karma-remap-istanbul": "^0.6.0",
    "protractor": "5.1.1",
    "ts-node": "2.1.0",
    "tslint": "4.4.2",
    "typescript": "2.1.6"
  }
}

my node and npm version is:
node -v : v7.9.0
npm -v : 4.2.0

install with save exact. remove '^' from all and then rm -Rf node_modules/
npm install. That might help


Mutation working only once

I am using Aws AppSync with Ionic framework. I tried checking the chat application which is provided by AWS Samples. Mutation call is called only once and there is no post call after that in the network calls. I dont know what the exact problem is, whether it is regarding the cache or the socket connection.
import { Injectable } from '@angular/core';
import AWSAppSyncClient from 'aws-appsync';
import appSyncConfig from './aws-export';
import { ApolloModule, Apollo } from 'apollo-angular';
import { Subject } from 'rxjs/Subject';
@Injectable()
export class GraphqlService {
constructor(private apollo: Apollo) {
}
hydrated() {
const appsyncClient = new AWSAppSyncClient({
url: appSyncConfig.graphqlEndpoint,
region: appSyncConfig.region,
auth: {
type: appSyncConfig.authenticationType,
apiKey: appSyncConfig.apiKey,
}
});
this.apollo.setClient(appsyncClient);
return appsyncClient.hydrated();
}
}
send(input: any) {
const chat = {
userId: input.userId,
createdAt: new Date().toISOString(),
message: input.message,
};
this.chats.next([...this.chats.getValue(), chat]);
this.apollo.mutate({mutation: post, variables: chat}).subscribe();
}

As this is not an issue specifically related to this repo or the code published here, suggest to create an issue in the AppSync SDK repository.


Update post-data-with-the-javascript-xmlhttprequest-method.md instructions

Uses common language to describe API data request. Improves clarity of instruction.
Checklist:

[X] I have read freeCodeCamp's contribution guidelines.
[X] My pull request has a descriptive title (not a vague title like Update index.md)
[X] My pull request targets the master branch of freeCodeCamp.
[X] All the files I changed are in the same world language, for example: only English changes, or only Chinese changes, etc.

Closes #XXXXX

LGTM @raisedadead  can you take a final look?

Congrats @jameshbush on your first pull request (PR)! 🎉
Thank you for your contribution to the page! 👍
We're happy to accept these changes, and look forward to future contributions. 📝

Welcome @RandellDawson 🎊🎉🕺


FEA：add split token and generate related resource

Update split token, generate word2vec, copy_mask, token2id, load pretrained model
Fix some bugs in redial, inspired and tgredial model

FIX: problem of data type


Instruction to instrument non-Spring applications using the SDKs.

@zhengyal There are currently no instruction on how to implement tracing for non spring java applications. There is simply one line in the instructions that specifies https://github.com/aws/aws-xray-sdk-java#applications-not-using-javaxservlet-may-include-custom-interceptors-to-begin-and-end-trace-segments that * Applications not using javax.servlet may include custom interceptors to begin and end trace segments
Directly call beginSegment and endSegment as necessary. Note: this method requires additional work to ensure that the X-Amzn-Trace-Id header is properly propogated and sufficient information about the request and response is captured with the segment.* .
This seems like a very convenient way to leave the most important intricacies of tracing to the developer. Are there any guidelines on how we can implement this in our Java Play applications ?

Closing. Please feel free to put additional comments here or open a new issue should any further help needed.


🐛 Bug Report: No matching host for Bitbucket Server

📜 Description
I have a clean installation of Backstage
Here's what we added for the configuration:
        bitbucketServer:
          - host: <redacted>
            apiBaseUrl: <redacted>
            token: ${BITBUCKET_TOKEN}

and
catalog:
  providers:
    bitbucketServer:
      bitbucket:
        host: '<redacted>'
        catalogPath: /catalog-info.yaml
        filters:
          skipArchivedRepos: true
        schedule:
          # supports cron, ISO duration, "human duration" as used in code
          frequency: { minutes: 1 }
          # supports ISO duration, "human duration" as used in code
          timeout: { minutes: 3 }
  import:
    entityFilename: catalog-info.yaml
    pullRequestBranchName: feature/backstage
  rules:
    - allow: [Component, System, API, Resource, Location]
  locations:
    - type: file
      target: ../../catalog-info.yaml
    - type: file
      target: ../../examples/org.yaml
      rules:
        - allow: [User, Group]

👍 Expected behavior
It should connect with our Bitbucket server
Please note: we have tried the bitbucket host with and without https:// and with and without a trailing slash character to no avail.
👎 Actual Behavior with Screenshots
During Backstage startup, we are seeing the following error:
2024-05-07T08:05:04.855Z search info Starting all scheduled search tasks.
/opt/app-root/src/node_modules/@backstage/backend-app-api/dist/index.cjs.js:1781
throw new errors.ForwardedError(
^
ForwardedError [InputError]: Module 'bitbucket-server-entity-provider' for plugin 'catalog' startup failed; caused by InputError: No BitbucketServer integration found that matches host <redacted>
at /opt/app-root/src/node_modules/@backstage/backend-app-api/dist/index.cjs.js:1781:21
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /opt/app-root/src/node_modules/@backstage/backend-app-api/dist/index.cjs.js:1780:13
at async processNode (/opt/app-root/src/node_modules/@backstage/backend-app-api/dist/index.cjs.js:1302:22)
... 7 lines matching cause stack trace ...
at async BackstageBackend.start (/opt/app-root/src/node_modules/@backstage/backend-app-api/dist/index.cjs.js:1881:5) {
cause: InputError: No BitbucketServer integration found that matches host <redacted>
at /opt/app-root/src/dynamic-plugins-root/backstage-plugin-catalog-backend-module-bitbucket-server-dynamic-0.1.26/dist/index.cjs.js:229:15
at Array.map (<anonymous>)
at BitbucketServerEntityProvider.fromConfig (/opt/app-root/src/dynamic-plugins-root/backstage-plugin-catalog-backend-module-bitbucket-server-dynamic-0.1.26/dist/index.cjs.js:223:40)
at Object.init [as func] (/opt/app-root/src/dynamic-plugins-root/backstage-plugin-catalog-backend-module-bitbucket-server-dynamic-0.1.26/dist/index.cjs.js:368:88)
at /opt/app-root/src/node_modules/@backstage/backend-app-api/dist/index.cjs.js:1780:35
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async processNode (/opt/app-root/src/node_modules/@backstage/backend-app-api/dist/index.cjs.js:1302:22)
at async Promise.all (index 2)
at async processMoreNodes (/opt/app-root/src/node_modules/@backstage/backend-app-api/dist/index.cjs.js:1297:7)
at async _DependencyGraph.parallelTopologicalTraversal (/opt/app-root/src/node_modules/@backstage/backend-app-api/dist/index.cjs.js:1308:5)
at async /opt/app-root/src/node_modules/@backstage/backend-app-api/dist/index.cjs.js:1777:9
at async Promise.all (index 1)
at async BackendInitializer.doStart_fn (/opt/app-root/src/node_modules/@backstage/backend-app-api/dist/index.cjs.js:1757:3)
at async BackendInitializer.start (/opt/app-root/src/node_modules/@backstage/backend-app-api/dist/index.cjs.js:1611:5)
at async BackstageBackend.start (/opt/app-root/src/node_modules/@backstage/backend-app-api/dist/index.cjs.js:1881:5) {
cause: undefined
}
}

👟 Reproduction steps
1 Have a clean install for backstage
2 Follow the documentation instructions to add Bitbucket Server as a catalog provider
📃 Provide the context for the Bug.
All our code repos are on Bitbucket. It is highly necessary for this connection to work properly.
🖥 Your Environment
Apple M3
We are also running Linux distros
👀 Have you spent some time to check if this bug has been raised before?

[X] I checked and didn't find similar issue

🏢 Have you read the Code of Conduct?

[X] I have read the Code of Conduct

Are you willing to submit PR?
Yes I am willing to submit a PR!

it looks like you have put bitbucketServer under catalog.providers instead of under integrations as mentioned in https://backstage.io/docs/integrations/bitbucketServer/locations
integrations:
  bitbucketServer:
    - host: bitbucket.mycompany.com
      apiBaseUrl: https://bitbucket.mycompany.com/rest/api/1.0
      token: ${BITBUCKET_SERVER_TOKEN}

it looks like you have put bitbucketServer under catalog.providers instead of under integrations as mentioned in https://backstage.io/docs/integrations/bitbucketServer/locations
integrations:
  bitbucketServer:
    - host: bitbucket.mycompany.com
      apiBaseUrl: https://bitbucket.mycompany.com/rest/api/1.0
      token: ${BITBUCKET_SERVER_TOKEN}

This was correct and solved this issue.
Random question: do you know how we can configure SSL certificates?

Closing since the issue is resolved 🎉
HTTPS can be configured in the backend config, although generally I'd recommend terminating HTTPS in a reverse proxy in front of the service. If you mean adding trusted certificates for outgoing connections you can use NODE_EXTRA_CA_CERTS.


Can't deploy to DevSandbox, if the image is not previously hosted on a registry

As a user, I should be informed that my image is currently not available on a registry (as it's a local one).
When doing a deploy to kubernetes, I should be invited to use the internal registry of DevSanbox

Image can be deployed to OpenShift cluster (Developer Dandbox, Dedicated or Local) using public image registry (docker.io, quai.io) or internal OpenShift registry.

Depends on https://github.com/containers/podman-desktop/issues/2623.

Available from github snapshot image.


How are ~ (tilde) in Path treated?

Currently, Path treats ~ tilde as a normal character. It hence passed on to the server (or local file system) which may treat it specially, or not. For the local file adaptor, as special replacement of ~ to the user home directory is done. Note that this does not work for the home dir of other users, i.e. ~notme. Webdav and s3 servers handle the tilde as an ordinary character. I have not tested this, but I think that ftp and scp servers resolve ~ to the homedir of the user. I am going to try this out now.

The vFTP server as in FTP docker file, does resolve ~ to the correct directory.

The SFTP server as in the ssh docker file, does not resolve ~ to the correct directory. Not that in unix filesystems, it is also possible to create a file or directory called ~

BTW the vFTP server resolving ~ to the user home dir depends on #524

Should we make Xenon oblivious to ~ ? In that case we'll just pass it to the adaptor.

Will implement + add test


About Glsl.Std.450 support

When porting rpcs3 to mvk, one of the major challenges was that some functions that are part of the spir-v extended functions for glsl are not supported e.g unpackHalf2x16. Is it a bug or are is it safe to assume none of the glsl extensions are implemented?

Closing, looks to be an issue with SPIR-V cross's MSL compiler being incomplete


Major Userscript changes and Server changes

I can't really name EVERYTHING I did because there were a lot of things I changed over a long period of time, I'll try to list some things I see through the Github code compare.
FEEDER.JS:

Fixed bot.log issue where wouldn't log sometimes
Moved the spawnCount to the leaderboard catch
Made some changes to verbosityLevel
Very minor formatting fixes
Fixed issue where maxBots wasn't working

AGARIO-FEEDER-BOT.USER.JS:

Changed version and added description
Made some changes to io.connect
Added a bot count overlay
Fixed missing  in the UUID box
Added another box for bot server IP
Made option for fast feed using S key.

Hope you guys enjoy this update as much as I did :D
(Yes David, I did test this on my server. Works fine, everything I tested worked fine. But it would be nice if someone else also tested this update on their server!)

Nice updates. I tested it out and it works great. I however disagree with having the spawn count increment on leaderboard update. I had bots right in front of me with the counter displaying "0". Feels more accurate where it was at on successful connect.

Hi MastaCoder, nice update.
But the number of bots does not appear to me, yet I am twenty bots.

Weird. It worked for me counting up and everything was fine, what server were you using for the server? The public one or your own?

Really love being able to change the bot server ingame.  Really helpful.

@MasonBurdette Not yet, well actually. I can add that too :+1:

@MastaCoder that would be really nice ;)

One quick update to move the spawnCount back to where it was.

and another quick update.

:)

@MastaCoder I use public server

does it work now ?
sounds great!

@davidmann4 not yet

@MastaCoder on the bot count it just says ...

@Petterholm Are you using the latest version of the server? Try using a private on instead of the free public one.

Last night it worked, but it appears to me this morning : 128/8192 /undefined

so yes or no ? :)

can you fix conflicts ?

undefined/undefined bots, any fix?

I probably won't be planning to update this as I am busy working on Ogar. Sorry bout that.

Only try to fix undefined/undefined, i want to see what amount of bots i got plz :(

But the issue is since @davidmann4 accepted the other pull before this one, I have to rewrite all my changes again since my repo is deleted now. :/

Ohh... It's because i don't know much java and i can't write...

I'll see if I have time o.o

@MastaCoder Once the coords are fixed will look into properly adding in the counter.

sorry, but I think the two pull requests where basically the same - you had a few lines of codes for the counter. Maybe someone reworks your pull request.


[Bug] Using Thumbar buttons on Acrylic windows causes them to not appear.

Pretty self explanatory but using electron-acrylic-window breaks the thumb-bar controls and causes them not to appear but they appear just fine with it disabled.
Broken (With EAC):

Working (No EAC):

Solved - was the result of the window not being shown in time


suggestion: improve footer tabs

Please enhance the footer tabs. There is this limitation listed in the  bottom tab bar:

⚠ DISCLAIMER: ONLY Use tabs at the root level (The first view that shows up on launch). Any other usage may introduce unexpected behavior.

I found this to be true. However, forcing all apps to display a footer with tabs from the launch view if they ever want to use tabs is rather restrictive and generally very awkward for app design and UX.
One work-around I found to work is to transition from a view without footer to a view with a footer via a "replace" transition, so that the user cannot go "back" into the non-tabbed view. I found this to work reliably, though I am not sure if it's something the project can condone. If this is "safe" to do, it may be a somewhat acceptable work-around in general and would be good to have documented for others, making the framework more versatile.
I typically design my apps to have a footer when the user is logged in. Replacing the "top" view into and out of tabs works for me.

+1 for using tabs in other than root page. Really weird behaviour.

@IvRRimum What weird behavior do you found? So far when presenting with it, it did not give problem for me.


[Hub Generated] Publish private branch 'amatukmolina-graphservicesprod-Microsoft.GraphServices-2023-04-13'

This is a PR generated at OpenAPI Hub. You can view your work branch via this link.
ARM API Information (Control Plane)
Changelog
Add a changelog entry for this PR by answering the following questions:

What's the purpose of the update?

[ ] new service onboarding
[x] new API version
[ ] update existing version for new feature
[ ] update existing version to fix swagger quality issue in s360
[ ] Other, please clarify

When are you targeting to deploy the new service/feature to public regions? Please provide the date or, if the date is not yet available, the month. May 2023
When do you expect to publish the swagger? Please provide date or, the the date is not yet available, the month. May 2023
By default, Azure SDKs of all languages (.NET/Python/Java/JavaScript for both management-plane SDK and data-plane SDK, Go for management-plane SDK only ) MUST be refreshed with/after swagger of new version is published. If you prefer NOT to refresh any specific SDK language upon swagger updates in the current PR, please leave details with justification here.

Contribution checklist (MS Employees Only):

[x] I commit to follow the Breaking Change Policy of "no breaking changes"
[x] I have reviewed the documentation for the workflow.
[x] Validation tools were run on swagger spec(s) and errors have all been fixed in this PR. How to fix?

If any further question about AME onboarding or validation tools, please view the FAQ.
ARM API Review Checklist

Applicability: :warning:
If your changes encompass only the following scenarios, you should SKIP this section, as these scenarios do not require ARM review.

Change to data plane APIs
Adding new properties
All removals

Otherwise your PR may be subject to ARM review requirements. Complete the following:

[x] Check this box if any of the following apply to the PR so that the label "ARMReview" and "WaitForARMFeedback" will be added by bot to kick off ARM API Review.  Missing to check this box in the following scenario may result in delays to the ARM manifest review and deployment.

Adding a new service
Adding new API(s)
Adding a new API version
-[ ] To review changes efficiently, ensure you copy the existing version into the new directory structure for first commit and then push new changes, including version updates, in separate commits. You can use OpenAPIHub to initialize the PR for adding a new version. For more details refer to the wiki. Note that this doesn't apply if you are trying to merge a PR that was previously in the private repository.

[ ] Ensure you've reviewed following guidelines including ARM resource provider contract and REST guidelines. Estimated time (4 hours). This is required before you can request review from ARM API Review board.

[ ] If you are blocked on ARM review and want to get the PR merged with urgency, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.

Breaking Change Review Checklist
If you have any breaking changes as defined in the Breaking Change Policy, request approval from the Breaking Change Review Board.
Action: to initiate an evaluation of the breaking change, create a new intake using the template for breaking changes. Additional details on the process and office hours are on the Breaking Change Wiki.
NOTE: To update API(s) in public preview for over 1 year (refer to Retirement of Previews)
Please follow the link to find more details on PR review process.

The PR is created based on the updates in the private branch. The updates in the PR has already been reviewed and approved with this PR Azure/azure-rest-api-specs-pr/12109

@zedy-wj Regarding the API Readiness error, right now the operations RT is updated in Canary for testing purposes and I assume that's why the pipeline says the new api-verison can't be found. Is that a blocker or can we wait for this swagger spec to be merged so we can rollout to every region?

Regarding the API Readiness error, right now the operations RT is updated in Canary for testing purposes and I assume that's why the pipeline says the new api-verison can't be found. Is that a blocker or can we wait for this swagger spec to be merged so we can rollout to every region?

@josefree Could you help to check if it is a blocker?

The PR is created based on the updates in the private branch. The updates in the PR has already been reviewed and approved with this PR Azure/azure-rest-api-specs-pr/12109
Is there any diff between this PR and the one checked into the private branch ?

The PR is created based on the updates in the private branch. The updates in the PR has already been reviewed and approved with this PR Azure/azure-rest-api-specs-pr/12109
Is there any diff between this PR and the one checked into the private branch ?

@raosuhas  There's no difference, we sent the PR as is from the OpenAPI hub tool

ArmSignedOff as has been reviewed and approved in private pr https://github.com/Azure/azure-rest-api-specs-pr/pull/12109.


Something wrong in yolo_head.py

it should be like this ：
yv, xv = meshgrid...

Could you also tell me the reason of doing this?

Could you also tell me the reason of doing this?
“def get_in_boxes_info” gets x_shifts and y_shifts from “def get_output_and_grid”. If the input has the same length and width, it's ok, or it gets wrong shifts.
More, this is the “def get_output_and_grid” in yolox of torch version .


Replace github.com/sloonz/go-qprintable with mime/quotedprintable

Go 1.5 introduces a new package into the standard library,
mime/quotedprintable. Use it instead of a third party library

Forgot to update .travis.yml, made a new pull request.


🗣 Discussion: Plugin notes

I've done some thinking about the plugin system (or lack thereof) recently, as I was exploring adding native support for snapshot tests.
(I'm not sure adding native support for snapshot tests is a great idea, because e.g. snap-shot-it already exists and works fine; this is not the right issue in which to discuss it)
Here's a breakdown of the way things currently work:

You can load an arbitrary script with --require before the main Mocha object is instantiated with any user-supplied options (and loads any test files)
If such a module sets a prop in require('mocha').reporters or require('mocha').interfaces, e.g., my-reporter or my-interface, the custom reporter or interface implementation will be able to be referenced by --reporter my-reporter or --ui my-interface, respectively (because "registration" comes before instantiation of Mocha)
At this point, Mocha will begin loading test files in whatever order is defined by the user.  Using --file will allow the user to be explicit about this; anything else will not make guarantees.  Thus, a plugin loaded with --file will have access to the root suite, describe, it, hooks, etc.  Not every file "run" by Mocha needs to be a "test" file.  If a global setup/teardown needs to happen, use --file <./path/to/harness.js>; if it's a 3rd-party package, --file <packagename>  works identically.

Some shortcomings of this (there are many, but here are a few), in order of (IMO) "least problematic" to "most problematic":

The way in which reporters and interfaces are "registered" is awkward.
--file <packagename> is rather unintuitive and cumbersome; it's not really designed for arbitrary plugins that need to operate at the root-suite level, but it works.
--require'd files which are not reporters nor interfaces cannot access any settings or options short of monkeypatching require('mocha').prototype.
Arbitrary plugins (packages) loaded with --file or otherwise via require() cannot access individual suites, hooks, nor test objects; all they can do is work with what Mocha injects into the global context, the same as test files (describe(), it(), etc.).  They cannot access the Mocha instance, nor the Runner instance, nor reporters, etc.

Some ways in which we can solve these issues:
Required Module Execution
Expand --require to look at the "return type" of a module.  If it has a return type matching "a Mocha plugin", then save a reference to it, instantiate Mocha, and pass this Mocha instance to each module we've saved a reference to.

A "Mocha plugin" cannot just be module.exports = function(mocha) {}, because existing modules/package export a function, like @babel/register. this would break stuff and cause bad weirdness

What this would allow plugins to do (keep in mind this would be before mocha begins even loading files):

Inject hooks into the root suite by modifying mocha.suite
Listen on events emitted by the root suite and/or any other suite once Mocha starts loading test files.  This would allow a plugin to do something when, e.g., a new suite is created, or a new test/hook/suite is created within a suite.
Modify options before run
Add files to the list of files Mocha attempts to load (other plugins, harnesses, tests??)
etc

Drawbacks of this approach:

Exposes events which may be previously considered internal
Because event listeners run synchronously, prevents async code from being run with any sort of guarantees.  A workaround may be to set the delay option programmatically and prepend a file into the list which executes Mocha's global.run().  Otherwise, there may be many breaking changes necessary to support "async listeners".  On the plus side, it'd probably give us async suites "for free!"

Probably best to define a more explicit plugin API surface where something loaded via --require would return some metadata and handler(s); maybe auto-loading of plugins could be a thing too.  This would ideally provide a more "official" interface for defining custom interfaces and reporters, as well.  That's... more work.
Expose Events via Singleton
Given:

test file a.spec.js which contains require('mocha-plugin')
package mocha-plugin which has a main module that contains const Mocha = require('mocha') (Mocha is the main class found in lib/mocha.js)

We could expose Mocha.events (or whatever), which is a singleton EE that listens on events emitted from Mocha's Runner instance, and re-emits them with two parameters: the current Mocha instance and whatever else the original event emits.  For example:
// mocha-plugin main
const {events, Runner} = require('mocha');
const {EVENT_HOOK_BEGIN} = Runner.constants;
events.on(EVENT_HOOK_BEGIN, (mocha, hook) => {
  // display the full hook title whenever a hook is about to run
  console.error(`running hook ${hook.fullTitle()`)
});

This suffers from the same problem as before, where these listeners must be synchronous.  Again, a "proper" API surface would be better, so we can avoid exposing more internals.  The mocha instance is passed along because while we can reasonably be sure that we only have one mocha instance, we cannot always guarantee it.
It would also be a good idea to allow packages loaded with --require to listen to these events.  We don't even have a Runner instance at this point, so we'd need to do something to work around that.

It'd be ideal to allow plugins to function at both "setup time" and "run time", and have a similar API for each.  The latter would necessarily have some restrictions on what it can do.
I don't think EE's are the right solution, as much of a time-saver they might be.
This will be very helpful to tooling authors trying to extend Mocha.  But whatever we do, we're stuck with it.
cc @mochajs/core

cc @papandreou and @sunesimonsen who I discussed this sort of thing with

Thanks for writing this down! First thing that popped up in my head: Would it work roughly the same way in the browser? I assume you(/plugins) could interact with the mocha global before kicking off the show with mocha.run();?

@papandreou You can interact with the mocha global, yes--I mean, you can do this now.  It's a monkeypatched instance of Mocha.
Now that I wrote that, I'm somewhat less hesitant to introduce a mocha global in Node.js, since it's already in the browser...

a mocha global, assuming it's a singleton, would solve the "expose events" question.  still... I don't really want to make private APIs public.

These are very good notes. Cross-linking them with #1457 and closing this issue to keep our queue small.


Docker connect denied

it was working yesterday but today when i tried docker logs aternos-discord-bot it gave me

pls help

i am using linux deepin version 20

This isn't an issue with this project. Learn the basics of docker; you need to add your user to the sudoers group or use the docker commands with sudo.


python312Packages.ignite: refactor

Description of changes
Add changelog, refactor meta, update homepage, add nativeImportsCheck, add disabled, etc.
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

i'm able to reproduce the darwin ofborg failure, which is introduce in this pr. Any ideas?

i'm able to reproduce the darwin ofborg failure, which is introduce in this pr. Any ideas?

I don't have a Darwin machine to investigate, but it seems likely this issue was present before and exposed by the added pythonImportsCheck. A few other packages (finetuning-scheduler, tensordict, torchsnapshot) are marked broken on Darwin for exactly the same reason, but I have no idea why..

if the error was pre-existing then i'm in favor of merging this and trigger the issue at build time. Lets leave it to those, if any, who rely on this on darwin to figure out the issue. :+1:


Bug: Unable to delete items from gem bag

Gems cannot be deleted from a character's gem bag.

Foundry version12.331
Shadowdark system version3.2.2
Enabled modulesNone

Steps to reproduce

Create a Shadowdark game world. Do not enable any modules.
Create a character
Add one or more items to the character's gem bag
Right-click on an item in the gem bag.
Select "delete item"

At this point I would expect the item to be removed from the gem bag. However nothing happens. In the developer console the following error message appears.
GemBagSD.mjs:134 Uncaught ReferenceError: tr is not defined
    at Object.callback (GemBagSD.mjs:134:21)
    at #onClickItem (foundry.js:68634:11)
    at HTMLElement.dispatch (jquery.min.js:2:40035)
    at v.handle (jquery.min.js:2:38006)

Work around
In order to remove an item from a character's gem bag you can sell it, then manually remove the appropriate number of coins from the character's inventory.
Video
https://github.com/user-attachments/assets/9cb30300-4542-4670-88fa-41cccd490ba6

Looking at the code, I think tr.data is meant to be element.data:
			{
				name: game.i18n.localize("SHADOWDARK.sheet.general.item_delete.title"),
				icon: '<i class="fas fa-trash"></i>',
				condition: element => canEdit(element),
				callback: element => {
					const itemId = tr.data("item-id");
					this._onItemDelete(itemId);
				},
			},

Many thanks for the thorough bug report.  This will be fixed in the next release.


Enable adding PointLight through editor

Add a factory for PointLight so it can be added/removed through the entity context menu. This was originally left out due to issues related to #469.

Done


feat(climate): add collapsible controls to climate card

Description

Related Issue

Motivation and Context

How Has This Been Tested

Types of changes

[ ] 🐛 Bug fix (non-breaking change which fixes an issue)
[x] 🚀 New feature (non-breaking change which adds functionality)
[ ] 🌎 Translation (addition or update a translation)
[ ] ⚠ Breaking change (fix or feature that would cause existing functionality to change)

Checklist

[x] My code follows the code style of this project.
[x] My change requires a change to the documentation.
[x] I have updated the documentation accordingly.
[x] I have tested the change locally.

Looking forward to this! Super quick turn around!

Got it now! Looks great!!!
Thanks again!


"convert to ES6 module" breaks code formatting

TypeScript Version:  2.90-insiders.20180503 (fixed in 3.0.0-dev.20180605)

Search Terms:
"convert to ES6 module" formatting fixes else newline "newline before else"
Code
-        Object.keys(properties).forEach(function(key) {
+        Object.keys(properties).forEach(function (key) {
             var value = properties[key];
             if (value && typeof value.valueOf() === 'object') {
                 Object.assign(element[key], value);
-            } else {
+            }
+            else {
                 element[key] = value;
             }
         });
     }

Expected behavior:
code style for } else { should be preserved.
Actual behavior:
a newline is added after } and there is no setting to prevent this behavior.
Related Issues:
https://github.com/Microsoft/vscode/issues/19335
https://github.com/Microsoft/vscode/issues/21723

Looks like it's fixed in 3.0.0-dev.20180605

fixed in 3.0.0-dev.20180605, just thought i'd leave documentation since I couldn't find anything


Fake death in City of Threads

When someone is being entagled by the patrols in City of Threads, it is counted (by KeyCount) as a death. It is no death though and you only get ported back to the start of the section and lose some health.

I don't think this is actually fixable easily since for some reason (apparently) the ingame combat log counts it as a death. Will try to look into it


Update ews-throttling-in-exchange.md

Modified streaming connection limit to be correct value based on product behavior and https://docs.microsoft.com/en-us/exchange/client-developer/exchange-web-services/how-to-maintain-affinity-between-group-of-subscriptions-and-mailbox-server#what-throttling-values-do-i-need-to-take-into-consideration
Plus I looked in the code ... it is 10.

Docs Build status updates of commit b77cf55:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/exchange-web-services/ews-throttling-in-exchange.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel


KServe Python SDK s3 endpoint url bug

/kind bug
What steps did you take and what happened:
I wanted to run Inference Server with code (Python SDK):
from kubernetes import client
from kserve import KServeClient
from kserve import constants
from kserve import utils
from kserve import V1beta1InferenceService
from kserve import V1beta1InferenceServiceSpec
from kserve import V1beta1PredictorSpec
from kserve import V1beta1SKLearnSpec

# Get namespace
namespace = utils.get_default_target_namespace()

# Define name of Inference Service (Model Server) and api version
name = "sklearn-iris1"
api_version = "serving.kserve.io/v1beta1"

# Define Inference Service
isvc = V1beta1InferenceService(api_version=api_version,
                               kind="InferenceService",
                               metadata=
                               client.V1ObjectMeta(
                               name=name, namespace=namespace, annotations={'sidecar.istio.io/inject':'false'}),
                               spec=V1beta1InferenceServiceSpec(
                               predictor=V1beta1PredictorSpec(
                               service_account_name="sa-s3-kserve",
                               sklearn=(V1beta1SKLearnSpec(
                                    storage_uri = "s3://ml-models-test/0/8827db8248f84ab6bc7d6cfcbbcf2f6a/artifacts/iris_rf_0_attempt/model.pkl"
                                   ,runtime_version = "v0.8.0"
                               ))))
)

KServe = KServeClient()
KServe.create(isvc)

s3-kserve-secret.yaml:
apiVersion: v1
kind: Secret
metadata:
  name: s3-kserve-secret
  namespace: XXX
  annotations:
     serving.kserve.io/s3-endpoint: "https://YYY:443"
     serving.kserve.io/s3-usehttps: "0"
     serving.kserve.io/s3-useanoncredential: "false"
type: Opaque
stringData:
  AWS_ACCESS_KEY_ID: "ZZZ"
  AWS_SECRET_ACCESS_KEY: "WWW"
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: sa-s3-kserve
  namespace: XXX
secrets:
- name: s3-kserve-secret

I got error in pod storage-initializer:
[I 230413 14:55:38 initializer-entrypoint:13] Initializing, args: src_uri [s3://ml-models-test/0/8827db8248f84ab6bc7d6cfcbbcf2f6a/artifacts/iris_rf_0_attempt/model.pkl] dest_path[ [/mnt/models]
[I 230413 14:55:38 storage:54] Copying contents of s3://ml-models-test/0/8827db8248f84ab6bc7d6cfcbbcf2f6a/artifacts/iris_rf_0_attempt/model.pkl to local
[I 230413 14:55:38 credentials:1111] Found credentials in environment variables.
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 175, in _new_conn
    (self._dns_host, self.port), self.timeout, **extra_kw
  File "/usr/local/lib/python3.7/site-packages/urllib3/util/connection.py", line 72, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
  File "/usr/local/lib/python3.7/socket.py", line 752, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/botocore/httpsession.py", line 412, in send
    chunked=self._chunked(request.headers),
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 786, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/usr/local/lib/python3.7/site-packages/urllib3/util/retry.py", line 525, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.7/site-packages/urllib3/packages/six.py", line 770, in reraise
    raise value
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 710, in urlopen
    chunked=chunked,
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 398, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 239, in request
    super(HTTPConnection, self).request(method, url, body=body, headers=headers)
  File "/usr/local/lib/python3.7/http/client.py", line 1281, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/local/lib/python3.7/site-packages/botocore/awsrequest.py", line 93, in _send_request
    method, url, body, headers, *args, **kwargs)
  File "/usr/local/lib/python3.7/http/client.py", line 1327, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.7/http/client.py", line 1276, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.7/site-packages/botocore/awsrequest.py", line 120, in _send_output
    self.send(msg)
  File "/usr/local/lib/python3.7/site-packages/botocore/awsrequest.py", line 204, in send
    return super(AWSConnection, self).send(str)
  File "/usr/local/lib/python3.7/http/client.py", line 976, in send
    self.connect()
  File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 205, in connect
    conn = self._new_conn()
  File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 187, in _new_conn
    self, "Failed to establish a new connection: %s" % e
urllib3.exceptions.NewConnectionError: <botocore.awsrequest.AWSHTTPConnection object at 0x7f780c25f450>: Failed to establish a new connection: [Errno -2] Name or service not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/storage-initializer/scripts/initializer-entrypoint", line 14, in <module>
    kserve.Storage.download(src_uri, dest_path)
  File "/usr/local/lib/python3.7/site-packages/kserve/storage.py", line 74, in download
    Storage._download_s3(uri, out_dir)
  File "/usr/local/lib/python3.7/site-packages/kserve/storage.py", line 119, in _download_s3
    for obj in bucket.objects.filter(Prefix=bucket_path):
  File "/usr/local/lib/python3.7/site-packages/boto3/resources/collection.py", line 83, in __iter__
    for page in self.pages():
  File "/usr/local/lib/python3.7/site-packages/boto3/resources/collection.py", line 166, in pages
    for page in pages:
  File "/usr/local/lib/python3.7/site-packages/botocore/paginate.py", line 255, in __iter__
    response = self._make_request(current_kwargs)
  File "/usr/local/lib/python3.7/site-packages/botocore/paginate.py", line 334, in _make_request
    return self._method(**current_kwargs)
  File "/usr/local/lib/python3.7/site-packages/botocore/client.py", line 391, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python3.7/site-packages/botocore/client.py", line 706, in _make_api_call
    operation_model, request_dict, request_context)
  File "/usr/local/lib/python3.7/site-packages/botocore/client.py", line 725, in _make_request
    return self._endpoint.make_request(operation_model, request_dict)
  File "/usr/local/lib/python3.7/site-packages/botocore/endpoint.py", line 106, in make_request
    return self._send_request(request_dict, operation_model)
  File "/usr/local/lib/python3.7/site-packages/botocore/endpoint.py", line 183, in _send_request
    success_response, exception):
  File "/usr/local/lib/python3.7/site-packages/botocore/endpoint.py", line 304, in _needs_retry
    caught_exception=caught_exception, request_dict=request_dict)
  File "/usr/local/lib/python3.7/site-packages/botocore/hooks.py", line 357, in emit
    return self._emitter.emit(aliased_event_name, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/botocore/hooks.py", line 228, in emit
    return self._emit(event_name, kwargs)
  File "/usr/local/lib/python3.7/site-packages/botocore/hooks.py", line 211, in _emit
    response = handler(**kwargs)
  File "/usr/local/lib/python3.7/site-packages/botocore/retryhandler.py", line 192, in __call__
    if self._checker(**checker_kwargs):
  File "/usr/local/lib/python3.7/site-packages/botocore/retryhandler.py", line 266, in __call__
    caught_exception)
  File "/usr/local/lib/python3.7/site-packages/botocore/retryhandler.py", line 292, in _should_retry
    return self._checker(attempt_number, response, caught_exception)
  File "/usr/local/lib/python3.7/site-packages/botocore/retryhandler.py", line 332, in __call__
    caught_exception)
  File "/usr/local/lib/python3.7/site-packages/botocore/retryhandler.py", line 232, in __call__
    attempt_number, caught_exception)
  File "/usr/local/lib/python3.7/site-packages/botocore/retryhandler.py", line 374, in _check_caught_exception
    raise caught_exception
  File "/usr/local/lib/python3.7/site-packages/botocore/endpoint.py", line 248, in _do_get_response
    http_response = self._send(request)
  File "/usr/local/lib/python3.7/site-packages/botocore/endpoint.py", line 317, in _send
    return self.http_session.send(request)
  File "/usr/local/lib/python3.7/site-packages/botocore/httpsession.py", line 432, in send
    raise EndpointConnectionError(endpoint_url=request.url, error=e)
botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "http://https://a.bst.s3.storage.psnc.pl:443/ml-models-test?prefix=0%2F8827db8248f84ab6bc7d6cfcbbcf2f6a%2Fartifacts%2Firis_rf_0_attempt%2Fmodel.pkl&encoding-type=url"

Is it error caused by weird endpoint URL (last row of error text): "http://https..." ?
botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "http://https://YYY:443/ml-models-test?prefix=0%2F8827db8248f84ab6bc7d6cfcbbcf2f6a%2Fartifacts%2Firis_rf_0_attempt%2Fmodel.pkl&encoding-type=url"

How can I fix it?
What did you expect to happen:
Successfully running Inference Service.
What's the InferenceService yaml:
[To help us debug please run kubectl get isvc $name -n $namespace -oyaml and paste the output]
Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]
Environment:
kserve==0.10.0
Openshift

Reason of weird URL was "https://" in serving.kserve.io/s3-endpoint in yaml


Check if need to run Databricks pre-merge [databricks]

Follow up of https://github.com/NVIDIA/spark-rapids/pull/8608/
Check if need to run Databricks pre-merge CI, otherwise Databricks pre-merge CI
will always be triggered even not changed any Databricks related files
or not tag PR's title with [databricks]

build

databricks

build


feat(subcommands): improvements and consistencies

Changes

Fixed this not being accessible in subcommand functions during runtime.
There were inconsistencies between the usage of SubCommand and Subcommand, it seemed to make most sense for everything to use Subcommand to match how Discord.js and Discord's API docs formatted this, as well as how people seem to more commonly write the word subcommand. This does include 2 extra breaking changes for SubCommandPluginCommand -> SubcommandPluginCommand and subCommands -> subcommands in options.
Converted the context sent to events to payloads to match framework.
Names of events, types, etc. were renamed to also match the formatting in framework of Message* and ChatInput*.
Some other fixes for incorrect event emits and typings

Thanks


Can't compile

evm.py: error: the following arguments are required: --saving_path/-s
[--lambda_cutoff: command not found
[--high_omega: command not found
[--mode: command not found
i can't seem to run it after running the usage, any help is greatly appreciated

what i did was put this into the terminal :
python src/evm.py [-h] --video_path VIDEO_PATH [--level LEVEL] [--alpha ALPHA]
          [--lambda_cutoff LAMBDA_CUTOFF] [--low_omega LOW_OMEGA]
          [--high_omega HIGH_OMEGA] --saving_path SAVING_PATH
          [--mode {gaussian,laplacian}] [--attenuation ATTENUATION]

python src/evm.py [-h] --video_path data/face.mp4 [--level 4] [--alpha 50]
          [--lambda_cutoff LAMBDA_CUTOFF] [--low_omega 0.8333]
          [--high_omega 1] --saving_path results
          [--mode {gaussian}] [--attenuation 1]

Hi, an exemple of execution: python src/evm.py -v data/baby.mp4 -l 2 -a 2 -lo 0.4 -ho 0.8 -s results/test -m gaussian -at 0.5

Hello, Thank you so much for your reply!
I have created a file named results/test, and ran the following command :
python src/evm.py -v data/face.mp4 -l 4 -a 50 -lo 0.833  -s results/test -m
gaussian -at 1
everything runs smoothly except for the saving part. This is the error:
@.*** global /io/opencv/modules/videoio/src/cap.cpp (595) open
VIDEOIO(CV_IMAGES): raised OpenCV exception:
OpenCV(4.6.0) /io/opencv/modules/videoio/src/cap_images.cpp:253: error:
(-5:Bad argument) CAP_IMAGES: can't find starting number (in the name of
file): results/test in function 'icvExtractPattern'
Saving Video:
100%|################################################################################################################################|
301/301 [00:00<00:00, 499.62it/s]
I am able to solve this problem by renaming the folder results/0test,
everything runs smoothly, but the folder is empty!
How do I solve this?
Best Regards
Stanley
On Tue, Nov 8, 2022 at 6:10 PM Hussem Ben Belgacem @.***>
wrote:

Hi, an exemple of execution: python src/evm.py -v data/baby.mp4 -l 2 -a 2
-lo 0.4 -ho 0.8 -s results/test -m gaussian -at 0.5
—
Reply to this email directly, view it on GitHub
https://github.com/hbenbel/Eulerian-Video-Magnification/issues/2#issuecomment-1306955614,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ASHBXHTXY2EQNBG6SGGDWLDWHIRJBANCNFSM6AAAAAARYZ3CAE
.
You are receiving this because you modified the open/close state.Message
ID: @.***>

try results/test.mp4, i think the extension is mandatory in order to have the output video generated

Hello,
Do you know the main reasons why the gaussian filtering gets killed? I
tried with the parameter that you put in your readme, but the process gets
killed .  How do you know which parameters to tweak, to not get it killed?
Thank you!
On Fri, Nov 11, 2022 at 3:09 AM Hussem Ben Belgacem <
@.***> wrote:

Closed #2
https://github.com/hbenbel/Eulerian-Video-Magnification/issues/2 as
completed.
—
Reply to this email directly, view it on GitHub
https://github.com/hbenbel/Eulerian-Video-Magnification/issues/2#event-7786282434,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ASHBXHUKWZCRNOSFO56TMSTWHVB7JANCNFSM6AAAAAARYZ3CAE
.
You are receiving this because you modified the open/close state.Message
ID: <hbenbel/Eulerian-Video-Magnification/issue/2/issue_event/7786282434@
github.com>


duffle bundle remove not working

Errors every time.

I thought it wasn't working but it turns out it was just not working how I expected. I expected to be able to be able to remove a bundle like this duffle bundle rm helloworld:0.1.0 but the correct syntax is duffle bundle rm helloworld -r 0.1.0 which works just fine.

Implemented in #852


add tags to the tasks in availability panel

We want to add a category tag to the tasks in the availability panel e.g react, ember, etc.

pic for reference⬇

A PR to add tags to all the tasks in the tasks screen is already raised https://github.com/Real-Dev-Squad/website-status/pull/322

I want to work on this issue, and have following doubts:

Category tag will be inside task box/view or outside?
How the category tag will look/styled? (possibility to be similar to tag in this PR)
As of now there is no prop for category on task type, will it be named as category? (Resolved by @vinayak-trivedi): Generic item tag-levelling model


mngids.py allows to create users/groups that are not in ids.tables

The mngids.py is faking useradd and groupadd to ensure we can control UID/GID when creating users/groups.
But if the user or group is not mentionned in the ids.tables file of the distro (i.e. RH7-ids.tables), the user/group will be created anyway.
It's a dangerous behavior because that could lead to upgrades errors later.
We should fail if during the build we try to create a user/group that are not controlled to be sure upgrade will run smoothly without permissons issues one day.

Also, we could add a test to validate by the end of the build that each user/group is member of ids.tables. It would be worth to test it with the goal to validate that the role is ready to be deployed and upgradable later.

Also, we could add a test to validate by the end of the build that each user/group is member of ids.tables. It would be worth to test it with the goal to validate that the role is ready to be deployed and upgradable later.


Minikube cluster not visible on the Resources page when created with Podman driver in Podman Desktop

When creating a Minikube cluster via Podman Desktop with the default driver set to Podman, the cluster information and control buttons do not appear on the Resources page, and there is no Minikube container listed on the Containers page. However, if the Minikube cluster is created using the Docker driver, the cluster is visible on the Resources page, and a Minikube container is also visible.
Extension version: v0.3.0
OS: Fedora 40
Steps to reproduce:

Install the Minikube extension.
Navigate to Settings → Resources page.
Go to the Minikube cluster creation page.
Ensure the Podman driver is selected.
Create the Minikube cluster.

The code is looking for the label name.minikube.sigs.k8s.io on the node container.

@afbjorklund Thank you for your response. I've tried to investigate it further, but it seems that when I deploy a Minikube cluster with the system Podman, it still doesn't appear.

Does it appear in sudo podman ps in the terminal?

Yes, there is log output:
sudo podman ps
CONTAINER ID  IMAGE                                COMMAND     CREATED        STATUS        PORTS                                                                                NAMES
d93ae3067ce0  gcr.io/k8s-minikube/kicbase:v0.0.45              2 minutes ago  Up 2 minutes  <IP_ADDRESS>:39645->22/tcp, <IP_ADDRESS>:46529->2376/tcp, <IP_ADDRESS>:37253->5000/tcp, <IP_ADDRESS>:39475->8443/tcp, <IP_ADDRESS>:38349->32443/tcp, 22/tcp, 8443/tcp  minikube

However, I don't see it on the resources page or as a container on the containers page.
On the Kubernetes page, it shows that I'm connected to the cluster, and I can see the running cluster node.

The issue was caused by my local Minikube configuration. Setting Minikube to rootless mode solved it: minikube config set rootless true


Get Smarter with Audio-Visual

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.

This seems to be reported by a bot, closing.


Rake fails on pi using latest albacore

Running rake mono fails to install on the raspberry pi after following the instructions at https://github.com/NancyFx/Nancy/wiki/Running-Nancy-on-your-Raspberry-Pi due to the albacore dependency.
Fails using the latest albacore 2.2.0
pi@raspy ~/Nancy $ sudo gem install albacore
Fetching: albacore-2.2.0.gem (100%)
Successfully installed albacore-2.2.0
1 gem installed
Installing ri documentation for albacore-2.2.0...
Installing RDoc documentation for albacore-2.2.0...
pi@raspy ~/Nancy $ rake mono
rake aborted!
NameError: uninitialized constant Albacore::Task
/home/pi/code/Nancy/customassemblyinfo.rb:5:in `<class:CustomAssemblyInfo>'
/home/pi/code/Nancy/customassemblyinfo.rb:4:in `<top (required)>'
/home/pi/code/Nancy/rakefile.rb:6:in `<top (required)>'
(See full trace by running task with --trace)

Fails forcing albacore 1.0.0
pi@raspy ~/Nancy $ sudo gem install albacore --version "~> 1.0"
Fetching: albacore-1.0.0.gem (100%)
Successfully installed albacore-1.0.0
1 gem installed
Installing ri documentation for albacore-1.0.0...
Installing RDoc documentation for albacore-1.0.0...
pi@raspy ~/Nancy $ rake mono
Version: 0.23.2
Main project does not update assembly info
D, [2014-11-10T22:48:31.066781 #2588] DEBUG -- : Executing xbuild: "" "src/Nancy.sln" /p:configuration="MonoRelease" /p:TagetFrameworkProfile="" /p:TargetFrameworkVersion="v4.0"
sh: 1: : Permission denied
F, [2014-11-10T22:48:31.109831 #2588] FATAL -- : XBuild failed, see the build log for more details.
rake aborted!
XBuild failed, see the build log for more details.
/var/lib/gems/1.9.1/gems/albacore-1.0.0/lib/albacore/support/failure.rb:12:in `fail_with_message'
/var/lib/gems/1.9.1/gems/albacore-1.0.0/lib/albacore/xbuild.rb:30:in `execute'
/var/lib/gems/1.9.1/gems/albacore-1.0.0/lib/albacore/support/createtask.rb:24:in `block in xbuild'
Tasks: TOP => mono => compilemono
(See full trace by running task with --trace)

After finding the version available when the wiki article was written found it works forcing albacore 1.0.0.rc2 (rc3 showed the same error as above).
sudo gem install albacore --version "1.0.0.rc2"

I'll update the wiki to reflect this, could it be fixed by forcing rc2 in the Gemfile?

Is there any solution to this issue? I am trying to compile Nancy in Mac OSX.

I think #1956 will address this?

BTW, Travis builds Nancy on UNIX on every commit on master, so it can't be that bad :wink:

The fix for me on the pi was to force albacore version 1.0.0.rc2.
Anything newer or older would fail.

Yeah, it's very picky; https://github.com/NancyFx/Nancy/blob/5dc9e2a179c5f3a50f6caf238d0341cdd2dd1474/.travis.yml#L12


Package for the book

@brendanhcullen I know you already started on the package for learnr tutorials. I think we need a standalone package for the book. It could be the one you started, or we could start a new one and port what you have in that over. The only reason I think the latter might be a better idea is just because then we could have the package name be sds or sdsr to match the book title.
The main reason I think we want a package for the book is because (a) we can create our own themes for different types of plots that we use throughout the book, and (b) we can use it as a way to share data, so they don't have to clone the repo or something to get the data. Incorporating the learnr tutorials there also makes sense.
Happy to start this or work from what you have, Brendan. Let me know what you each think.

@datalorax Totally happy to try this out! The reason I created a separate package for the learnr apps is that Desirée recommended it (more info here + screenshot below of her recommended workflow). I'm actually gonna chat with her tomorrow so I can ask her about this. But I think having a package for the book is a good idea either way. If you want to create the file structure we will need for a package, I can test out adding an example learnr app in a separate branch.

Okay, great! I'll try to get a basic package infrastructure up there today. Will come comment on this issue when it's up.

Closing this because it's now setup here


Add caching functionality (v2)

A while ago, caching was removed due to some issues (see https://github.com/lycheeverse/lychee/pull/349).
This is a new implementation with the following improvements:

Architecture: The new implementation is decoupled from the collector, which was a major issue in the last version. Now the cache has a single responsibility: collecting links. This also avoids race-conditions when running multiple collect_links instances, which probably was an issue before.
Performance: Uses DashMap under the hood, which was noticeably faster than Mutex<HashMap> in my tests.
Simplicity: The cache format is a CSV file with two columns: URI and status. I decided to create a new struct called CacheStatus for serialization, because trying to serialize the error kinds in Status turned out to be a bit of a nightmare and at this point I don't think it's worth the pain (and probably isn't idiomatic either).

TODO
Open points that evolved during our discussion:

[x] Disable writing cache to disk by default. Add a --cache flag to enable.
[ ] Support different cache formats. E.g. a compact format using bloomfilters/xor filters.
[x] Don't cache filesystem access.
[ ] Follow-up: Add caching example to Github action.

Future improvements

Requests for the same URI can get fired in parallel, resulting in the URI getting checked multiple times — rendering the cache worthless. I think this is an acceptable trade-off for the moment because it allows us to avoid additional locking/synchronizing. In the future we might add website-based sharding. This will also help to avoid overloading the server with requests and can be nicely combined with rate-limiting.
The request handler could be refactored to use a custom#[cache] attribute.
We can track the number of duplicate links and show the info when --format=detailed is activated.

Fixes https://github.com/lycheeverse/lychee/issues/348
@MichaIng @lebensterben @pawroman @untitaker fyi

Do I see it right, that the cache is stored as dedicated file .lycheecache automatically? IMO tools like lychee shouldn't automatically create files on the filesystem, if no explicit flag was set, especially since it's unknown whether write permissions are actually given 😉. It should be possible to keep it memory for a single lychee call only by default, and have a dedicated file (which is indeed nice when multiple calls are done for some reason) created only with an opt-in flag?

IMO tools like lychee shouldn't automatically create files on the filesystem, if no explicit flag was set, especially since it's unknown whether write permissions are actually given

Agreed.

Storing cache result in CSV seems a straightforward choice, since we've mature and performant serializer/deserializer and it's human readable.
But if we choose some other format such as apache arrow, it can be faster and the non-readability may even be desired by some users.

After a quick glance I don't understand how cache invalidation works. You're invalidating on a per-file level, but the cached requests can be really old, no?
I agree that the caching is a surprising default. If the caching is automatically set up to work in the gh action, it could be nice to have enabled by default though
in CI, filesize matters, as the cache is downloaded/uploaded. Perhaps you can store bloom filters instead.
I would never cache filesystem access (not sure if you're doing that)
otherwise lgtm

Yes indeed. We could add an individual timestamp for each cached request. It would be quite repetitive I guess. Any alternative ideas?

if stored as bloom filter, you need to store a bloom filter per (status, timestamp) tuple. if timestamp is rounded down to days or weeks, this might be feasible

Good idea. We should enable it and add an example on how to use caching with the cache action.

I think the entire performance tradeoff will hinge on CI, as having a cache in CI is really costly.
I don't think it's necessary to support storing the cache in multiple formats or data stores. what do you get from redis support? do you want to do partial cache updates? then use a cache dir. why support both bloom and xor? is there a tradeoff I'm missing?

what do you get from redis support?

The idea was that if you run a cluster of lychees you could have a shared cache. This can be helpful when lychee is running behind an API endpoint, which I'm planning to do.

do you want to do partial cache updates? then use a cache dir.

Yeah, partial updates. You might be right actually. We should try to get as far as possible with simple file-based caching.

why support both bloom and xor? is there a tradeoff I'm missing that prevents you from picking the universally best format+datastore?

No I meant either bloom or xor, not both. 😉 On top of that I would still like to keep CSV because removing cache entries can be done with a simple editor and it's great for debugging (and compresses well). Does that make sense?

The only issue with CSV is the redundant timestamp storage.
We could get around that by using a slightly more sophisticated format like TOML, which is still human-readable.
[success]
time = 1979-05-27T07:32:00-08:00
requests = [ ... ]

[success]
time = 1980-05-27T07:32:00-08:00
requests = [ ... ]

But then it's not easily "sed-able" anymore. Is there a different format that comes to mind?

The idea was that if you run a cluster of lychees you could have a shared cache. This can be helpful when lychee is running behind an API endpoint, which I'm planning to do.

that's fair. it's still possible to do it with filesystem (see how clustering with afl works), but redis seems much easier.

@untitaker, your comment got me thinking:

if stored as bloom filter, you need to store a bloom filter per (status, timestamp) tuple. if timestamp is rounded down to days or weeks, this might be feasible

For CSV, we could store the cache files like so
.lycheecache
├── 2022-01-05T07:32:00-08:00.csv
└── 2022-01-05T08:32:00-08:00.csv

The upside would be that we only have to store the timestamp once — in the filename. I think that was also @MichaIng's idea IIRC. The downside would be that we have to manage multiple cache files and think about cleaning them up to not waste disk space. Also we'd have to merge the files on startup, so maybe it's more trouble than it's worth.

Note that due to block sizes, usually multiple files are magnitude of orders larger than redundant timestamps. Not sure if this is a big issue to have some KiB more?

I'm okay with adding the timestamps to every entry. If we use Unix timestamps, they are quite compact anyway. If we use full RFC3339 dates, then they are a little longer, but also human-readable. Then again I don't think many people will edit the file manually, let alone based on the request time, so it shouldn't matter.

how about a single file that contains tuples/csv rows:
(status, timestamp * NUM_BUCKETS / cache_expiry_seconds, [xor filter of urls])

cache_expiry_second is a user-defined value to determine how old cache items can be

timestamp is unix timestamp in seconds

NUM_BUCKETS is a hardcoded value that will be important later

a cache row is expired when (timestamp * NUM_BUCKETS / cache_expiry_seconds) (i.e. the second column) is smaller than (current_timestamp - cache_expiry_seconds) * NUM_BUCKETS / cache_expiry_seconds
since cache_expiry_seconds controls how the second row is formatted, the setting has to be written to the cache file. if lychee finds that the setting in the cache file is different from its own config, it can either 1) throw away the cache file 2) do more math to recover the cache file. on the upside, the second column is now much smaller than a timestamp
this puts a theoretical upper bound on the number of rows: there can only be NUM_BUCKETS rows. we can put an upper bound on how large each row is. this puts an upper bound on the entire file size.
lastly you can choose NUM_BUCKETS freely to define the file size. you could make it dependent on file system block size, or make it a user setting, however then you also need to write it into the file to do proper cache invalidation

lastly you can add % NUM_BUCKETS to the second column, and additionally store a timestamp that tells you when you last wrote to the file. this makes cache invalidation really complicated (but not impossible), but now a very small integer type can be chosen for the second column

Since you mentioned timestamp and cluster, it's worth mentioning that time format such as

2022-01-05T07:32:00-08:00
isn't robust.

Epoch time in UTC is not location specific, but is not human friendly.
ISO 8601 time + time zone seems better but is not robust because sometimes a location changes its time zone which results in local time discontinuity.

Guess a UNIX timestamp would be the easiest as it's guaranteed to be UTC.

Question: if --cache is not set but the .lycheecache file exists, should we still load it? I guess not, but I wanted to hear people's thoughts.

I think the existence of --cache (an option to pass cache path) pretty much implies there's no magic filenames that lychee picks up automatically
…
On Wed, Jan 12, 2022, at 23:28, Matthias wrote: Question: if --cache is not set but the .lycheecache file exists, should we still load it? I guess not, but I wanted to hear people's thoughts. — Reply to this email directly, view it on GitHub <#443 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGMPRIKEPJGA2ZTWPUY67TUVX6A3ANCNFSM5LIQVLSQ. You are receiving this because you were mentioned.Message ID: @.***>

--cache is a flag, telling lychee to use .lycheecache in current working dir (?). And I agree that without this flag set, .lycheecache should be neither written nor read 👍.

Added the timestamp functionality we talked about. @untitaker, your approach is superior and quite clever, but I decided to keep it simple for now and just add a timestamp to every entry. It's wasteful, but quite easy to explain and most people wouldn't use a CSV file anyway if they had space constraints.
As for the bloomfilter / xor filter, I quite frankly didn't get far.
StoreExt would have to be modified to have a
fn load<T: AsRef<Path>>(path: T, max_age_secs: u64) -> Result<dyn Cache>;

however the trait object doesn't have a size. impl trait is not allowed in traits at the moment and I don't want to introduce a Box here to avoid the runtime overhead. I also don't think it's very elegant to use a Box here.
Played around with an associated type like in the Handler in this blog post about tower, but in the end it always wants a concrete type from me. ¯\_(ツ)_/¯
If someone has an idea I'd welcome PRs. 😆
Other than that I'd say we could merge this from my side. It's an optional feature anyway, so let's not sweat it. If anyone has the time to give it a spin, I'd be happy for any final comments.

looks good enough to merge. no need to finish all features in this PR.


Transform Closures (Or something like that)

Motivation Behind Feature

Now, convert the results to other structs/public responses is not posible, for example, a common case is the User model which have sensitive data like password, confirmation token, even more, but that fields can't be returned in the response, so we need to create a PublicUser struct to exclude that fields
Feature Description
Maybe a solution is to pass some kind of transform closure which can be used in the pagination process

I've been out of town for a while and haven't had much time to work on doing something like that. Originally I wanted to do something that conformed to a protocol and would allow easy transformation. My goal is still that, however, I haven't given it much thought to work on it since I've just been swamped. I'll be home in a couple of weeks and will be able to start thinking about some ideas again.

@anthonycastelli so is this possible rn? From #12, I can't really tell how to do it...


Data collection health monitoring workbook is not available in Sentinel

Hi, I can't find the Data collection health monitoring workbook on Sentinel. I searched for it in the template and it is not there.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 52fb32ef-0801-1b21-d279-5223f813f9de
Version Independent ID: f10bf47e-13e5-861e-021a-03d3aa0e36cb
Content: Monitor the health of your Microsoft Sentinel data connectors
Content Source: articles/sentinel/monitor-data-connector-health.md
Service: microsoft-sentinel
GitHub Login: @limwainstein
Microsoft Alias: lwainstein

@janmigs1
Thanks for your feedback! We will investigate and update as appropriate.

@janmigs1
Thanks for bringing this to our attention.
I'm going to assign this to the document author so they can take a look at it accordingly.

@cwatson please review it.

@SaibabaBalapur-MSFT you tagged the wrong person.

As per @limwainstein's update, I have reassigned the case to @cwatson. Now, I have assigned it to @yelevin, who will look into it.
Thanks

@yelevin
please review it.