CSS beautify overzealous with spaces preceding pseudo elements

.example ::-webkit-input-placeholder becomes .example::-webkit-input-placeholder

Please run Atom Beautify: Help Debug Editor from Atom command palette and put into a Markdown file (extension .md) in a Gist and paste the link back here. Thanks.

Thanks @Glavin001, here you go: https://gist.github.com/rediris/5477efe4cc6eda1e999e

I can't remember if Pretty Diff is used for regular CSS, but I can reproduce the error in Pretty Diff.

Fixed at http://prettydiff.com/ignore/testlocation/prettydiff.com.xhtml.  I will publish this fix soon.

I can't remember if Pretty Diff is used for regular CSS, but I can reproduce the error in Pretty Diff.

Currently JS Beautify is used for regular CSS. See https://github.com/Glavin001/atom-beautify/blob/master/src/languages/css.coffee#L28
However, if Pretty Diff can pass all of the current tests that JS Beautify does for CSS, then I see no reason why we cannot do a silent migration for the default beautifier in Atom Beautify that users will not even notice, except for the superior support of Pretty Diff :smiley: .

Fixed at http://prettydiff.com/ignore/testlocation/prettydiff.com.xhtml. I will publish this fix soon.

For users who want to use Pretty Diff as your beautifier, you can go into into the Atom Beautify package settings and find Language Config - <language> - Default Beautifier option and change it to Pretty Diff. For instance, Language Config - CSS - Default Beautifier is currently JS Beautify and you can change that to Pretty Diff and have this fixed!
Be sure to uninstall and reinstall Atom Beautify such that you get the latest update of Pretty Diff.

@Glavin001, I know this have been in the air for a while and this issue is still "open", but it doesn't seem like it is fixed in atom-beautify, even though prettydiff mentions that a fix was made.
For me, neither using JS Beautify, CSS Comb or Pretty Diff as Default beautifier works as expected.
It still removes a space before : (colon), which makes that CSS rule have a totally different behaviour.
If there is a reason for not doing anything to fix this or implement the mentioned fix, please update this issue ticket or comment.

@prettydiff is there a required option to make this fix in Pretty Diff work?
@Bubbi : Have you tried uninstalling and reinstalling Atom-Beautify? Often the issue is that your local copy of Pretty Diff is not the latest, although this issue is fairly old so that should not be the case. Maybe there is an option to get it to work.
To help @prettydiff and I debug this with you, please provide a link to the required debug.md Gist which includes debugging information that answers our most commonly asked questions: https://github.com/Glavin001/atom-beautify/blob/master/ISSUE_TEMPLATE.md#how-to-create-debugmd-gist
Thank you.

@Glavin001 This resolution does not require an option.  It was purely a defect that has since been resolved.
@Bubbi Could you please provide a code sample to ensure you are not talking a new defect that I am not aware of?

Closing as this issue is old and likely resolved.  If I am closing this issue prematurely please reopen it.


throw in onError fails silently and stops flow, while Promise.reject() works as expected and shows Uncaught Error in console

Describe the bug
Issue
Throwing from inside onError fails silently (no hint in the console about Uncaught Error).
However, using Promise.reject works as expected and the error can be seen in the console - "Uncaught error...".
This is the case for all three onError definitions:

mutationCache
useMutation
mutate

And the second thing - throwing from inside onError defined in mutationCache also prevents onError defined in useMutation from running at all.
However, onError defined on mutate is called always:

so no matter if we rejected or thrown from mutationCache
no matter if we rejected or thrown from useMutation
onError in mutate will be called anyway.

Here's a codesandbox demo.
Your minimal, reproducible example
https://codesandbox.io/p/sandbox/sweet-villani-clpw7d?file=%2Fsrc%2FApp.tsx%3A52%2C8
Steps to reproduce
Code to reproduce the issue:
queryClient:
export const queryClient = new QueryClient({
  mutationCache: new MutationCache({
    onError: (error) => {
      console.log("[mutationCache][onError] triggered with error: ", error);

      try {
        // do something that might throw an error
        throw new Error("(ERROR A: thrown from mutationCache onError)");
      } catch (e) {
        console.log("[mutationCache][onError] caught e: ", e);
        /**
         * Throwing here (or simply not wrapping previous call in try/catch) causes it
         * to stop here - so useMutation::onError in App.tsx is not even called
         * and there is no hint in console apart from the explicit console.log(s) above
         */
        throw e;
        /**
         * Instead of throwing, rejecting like this makes useMutation::onError in App.tsx run as expected,
         * and after that there is clear "Uncaught (in promise) Error: (ERROR A: thrown from mutationCache onError)"
         * shown in the console
         */
        // Promise.reject(e);
      }
    },
  }),
});

useMutation:
const { mutate } = useMutation({
    mutationFn: async () => {
      throw new Error("DUMMY POST REQUEST FAILED");
    },
    onError: (error) => {
      /**
       * If in index.tsx in MutationCache::onError we throw, we don't even get here,
       * and there is no hint about uncaught error, only the explicit console.log(s) that are there.
       *
       * But if we use Promise.reject there, we do get here, so let's say we've done that
       * and what comes next is the following:
       */

      console.log("[useMutation][onError] triggered with error: ", error);
      try {
        // do something that might throw an error
        throw new Error("(ERROR B: thrown from useMutation onError)");
      } catch (e) {
        console.log("[useMutation][onError] caught e: ", e);
        /**
         * Throwing here (or simply not wrapping previous call in try/catch) means
         * that the only hint about ERROR B being thrown is the explicit console.log above.
         * Then after those, the only thing we will see is:
         * "Uncaught (in promise) Error: (ERROR A: thrown from mutationCache onError)"
         * which comes from the mutationCache::onError in index.tsx
         */
        throw e;
        /**
         * However, instead of throwing, if we reject, in the console we will see:
         * "Uncaught (in promise) Error: (ERROR A: thrown from mutationCache onError)"
         * followed by:
         * "Uncaught (in promise) Error: (ERROR B: thrown from useMutation onError)"
         */
        // Promise.reject(e);
      }
    },
  });

mutate:
mutate(undefined, {
  onError: (error) => {
    /**
     * Now for this onError inside mutate call, this one is called no matter
     * if we throw or reject inside the mutationCache::onError.
     * So if we reject in mutationCache, the callbacks are called in the following order:
     *   1. mutationCache::onError
     *   2. useMutation::onError
     *   3. mutate::onError
     *
     * But if we throw in mutationCache, it's like this:
     *   1. mutationCache::onError
     *   2. mutate::onError
     * So useMutation::onError is not called, but this one still is
     */
    console.log("[mutate][onError] triggered with error: ", error);
    try {
      // do something that might throw an error
      throw new Error("(ERROR C: thrown from useMutation onError)");
    } catch (e) {
      console.log("[mutate][onError] caught e: ", e);

      /**
       * But here it is the same as for the other cases,
       * if we throw (or don't wrap previous call in try/catch), we won't see a hint about ERROR C apart from the console.log above
       */
      throw e;

      /**
       * If we reject, we can see in the console:
       * "Uncaught (in promise) Error: (ERROR C: thrown from useMutation onError)"
       */
      // Promise.reject(e);
    }
  },
  });

Expected behavior
I'd expect to see the Uncaught error in the console even if I use throw.
The case for me where I discovered this issue was that I was calling Object.keys(someObject.errors) inside onError in mutationCache and using a toast to show the error.
I also had some logic in onError defined in useMutation, setting an error for a form.
In one case the errors property was undefined, which caused Object.keys to throw before the toast could show anything - onError in useMutation was not called so no form error was set. So there was no hint at all about an error occurring.
So the only safe approach here seems to be wrapping the logic in onError in mutationCache in try/catch and Promise.reject the error from the catch clause - but this seems like an extra step that should not be necessary.
And having no hint about an error occurring seems like a bug.
How often does this bug happen?
Every time
Screenshots or Videos
No response
Platform

OS: macOS
Browser: Chrome

Tanstack Query adapter
react-query
TanStack Query version
v5.12.2
TypeScript version
No response
Additional context
No response

If I return the rejected Promise, I see the same behaviour as with throwing

Yeah, with the returned Promise it works the same as throwing.
So if I don't return it, it basically resolves and then later rejects, but it's already out of the error handling mechanisms and that's why it can be seen as uncaught? Or something else is happening here?

the mutationCache callback is called first, so we would need to wrap each call of onError into their own try/catch. Also, with which error should we continue to call the next onError then? The one from the actual mutation or the one from the previous onError callback? This all gets very confusing and very complicated pretty fast.

So how would you recommend structuring this use case:

I want mutation cache to have general error handling - basically it does some parsing and of the error and then it is shown in a toast. This happens for all mutations.
Let's say I have a form, e.g. login, and a useLoginMutation that has onError which on 401 sets a form error so that the user knows the credentials are invalid.

The thing is, if by any chance an error is thrown due to some parsing error in the mutation cache onError before it could show the toast, the user has no hint about something going wrong - no toast shown, and since onError in useLoginMutation is not called, no form error is set either.
I get it that the parsing error that happened is basically a bug that can be fixed. But if the error didn't fail silently, I could at least use an error boundary that would get triggered and explicitly show that there is an error in the app. It's much more obvious that there is an issue to be fixed than if it just failed silently, didn't login, didn't toast, didn't show an error.
So is there some idiomatic way of doing it?
Maybe moving setError for the form into onError for mutate would suffice since it's always called (at least if the login page is mounted)?
And also if I use Promise.reject(e) without the return as I mentioned, the error is uncaught and I could handle it in an error boundary I suppose - is there any issue with not returning the Promise?

Is there a way to do still do this check somehow?

Not really because a mutation could have multiple observers. We don't generally have a concept of chaining error handlers. Like, if it was handled globally, it won't be handled anymore in the component ...


How do I know the source of the picture use exif.js?

How do I know the source of the picture use exif.js? camera or album?

it's very hard to understand what you mean by this. the source uses exif.js - what does that mean? do you mean if the picture contains exif? you just read it using this lib, and if you'll get a non-empty object.

i want to know where the image from，album or camera？can i use exif.js to know the image from？
发自邮洽 (http://mailchat.cn/)
from: Carl-Erik Kopseng<EMAIL_ADDRESS>To: exif-js/exif-js<EMAIL_ADDRESS>Cc: wasonxie (mailto:wasonxie@gmail.com)、Author<EMAIL_ADDRESS>2018-02-08 00:21
Subject: Re: [exif-js/exif-js] How do I know the source of the picture use exif.js? (#139)
it's very hard to understand what you mean by this. the source uses exif.js - what does that mean? do you mean if the picture contains exif? you just read it using this lib, and if you'll get a non-empty object.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub (https://github.com/exif-js/exif-js/issues/139#issuecomment-363823264), or mute the thread (https://github.com/notifications/unsubscribe-auth/AIUVfvOVyoABBb6sY_NYtlE_Peuv4WTPks5tSc1sgaJpZM4Rx7_p).

EXIF-information is stored in the picture at creation time (but can be added through other editing programs as well, after the fact). There is no EXIF tag that will show you where an image was stored. That would make no sense: that would mean all handling software would need to update this information every time the picture was moved from one storage medium to the next.
So no, you cannot use exif.js to know whether the image was directly captured by the camera or was already existing. You can use other ways though, but they have nothing to do with EXIF:

Restrict the file input to image types and set the capture attribute to camera. That will force phones to open the native camera. See this link for more info.

Supply an alternative input method for normal files.

This is only supported by mobile os'es, btw.


Add websocket support

https://pooja-anandani.medium.com/web-sockets-with-django-graphene-48cd179392fd

https://blog.logrocket.com/django-graphql-project-graphene/

https://docs.graphene-python.org/projects/django/en/latest/subscriptions/


Error Building Web Chat

Hello,
Currently i am experiencing the following error when i try to build a custom web chat.
src/Attachment.tsx(310,14): error TS2678: Type '"image/svg+xml"' is not comparable to type '"image/png" | "image/jpg" | "image/jpeg" | "image/gif" | "audio/mpeg" | "audio/mp4" | "video/mp4"...'.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY<EMAIL_ADDRESS>(node_modules\chokidar\node_modules\fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for<EMAIL_ADDRESS>wanted {"os":"darwin","arch":"any"} (current: {"os":"win32","arch":"x64"})
npm ERR! Windows_NT 10.0.16299
npm ERR! argv "C:\Program Files\nodejs\node.exe" "C:\Program Files\nodejs\node_modules\npm\bin\npm-cli.js" "install"
npm ERR! node v6.11.2
npm ERR! npm  v3.10.10
npm ERR! code ELIFECYCLE
npm ERR<EMAIL_ADDRESS>prepublish: tsc && npm run build-css && webpack --progress --config webpack.production.config.js
npm ERR! Exit status 2
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>prepublish script 'tsc && npm run build-css && webpack --progress --config webpack.production.config.js'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the botframework-webchat package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     tsc && npm run build-css && webpack --progress --config webpack.production.config.js
npm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs botframework-webchat
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!     npm owner ls botframework-webchat
npm ERR! There is likely additional logging output above.
Thanks in advance for your help and support.

This error should be fixed if you get directlinejs version 0.9.13:

src/Attachment.tsx(310,14): error TS2678: Type '"image/svg+xml"' is not comparable to type '"image/png" | "image/jpg" | "image/jpeg" | "image/gif" | "audio/mpeg" | "audio/mp4" | "video/mp4"...'.

Can you delete the botframework-directlinejs folder from your node_modules folder and npm install again?

After deleting the botframework-directlinejs folder from my node-modules folder, ocurred the same error.

you may need to also delete your package-lock.json. Then repeat the above.
OR it may be easier to clone to a new place on your disk.

Thanks, it worked


Add prompt to GlobalOpts

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

all the prompts appear to work:
archer init:
Looks like you have some projects already.
? Would you like to use one of your existing projects? No
Ok, let's create a new project then.
? What would you like to call your project? davids-face
? Which type of infrastructure pattern best represents your application?  [Use arrows to move, type to filter, ? for more ? Which type of infrastructure pattern best represents your application? Load Balanced Web App
? What do you want to call this Load Balanced Web App? dolly
Error: prompt for app init: failed to select Dockerfile: list of provided options is empty

archer project init:
~/workspace/amazon-ecs-archer(refactor-globalopts)$ archer project init
Looks like you have some projects already.
? Would you like to use one of your existing projects? No
Ok, let's create a new project then.
X Sorry, your reply was invalid: project name the mess is invalid: value must start with a letter and contain only lower-case letters, numbers, and hyphens
? What would you like to call your project? the-mess
⠇ Creating the infrastructure to manage container repositories under project the-mess.

archer app init:

~/workspace/amazon-ecs-archer(refactor-globalopts)$ archer app init
Note: It's best to run this command in the root of your workspace.
? Which type of infrastructure pattern best represents your application?  [Use arrows to move, type to filter, ? for more ? Which type of infrastructure pattern best represents your application? Load Balanced Web App
? What do you want to call this Load Balanced Web App? the-stuff
Error: failed to select Dockerfile: list of provided options is empty

archer env init:
~/workspace/amazon-ecs-archer(refactor-globalopts)$ archer env init
? What is your environment's name? food
⠇ Proposing infrastructure changes for the food environment.

archer env ls has no prompt?
archer app package:
~/workspace/aws-reinvent-2018-trivia-game(master)$ archer app package
? Which application would you like to generate a CloudFormation template for?  [Use arrows to move, type to filte? Which application would you like to generate a CloudFormation template for? backend
? Which environment would you like to create this stack for? stage
# Copyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: Apache-2.0
AWSTemplateFormatVersion: 2010-09-09
...

archer app deploy:
~/workspace/aws-reinvent-2018-trivia-game(master)$ archer app deploy
? Select an application backend
Only found one environment, defaulting to: stage
unable to prepare context: context must be a directory: /Users/hsinghui/workspace/aws-reinvent-2018-trivia-game/trivia-backend/Dockerfile
Error: build Dockerfile at trivia-backend/Dockerfile with tag b1756e7, building image: exit status 1

archer pipeline init:
~/workspace/aws-reinvent-2018-trivia-game(master)$ archer pipeline init
? Would you like to add an environment to your pipeline? Yes
? Which environment would you like to add to your pipeline?  [Use arrows to move, type to filter, ? for more help? Which environment would you like to add to your pipeline? stage
? Would you like to add an environment to your pipeline? No
X Sorry, your reply was invalid: GitHub repository name foo is invalid. value must be a valid GitHub repository, e.g. https://github.com/myCompany/myRepo
? What is your application's GitHub repository? https://github.com/goose/face
? Please enter your GitHub Personal Access Token for your repository: https://github.com/goose/face [? for help]*✔ Success! Created secret: github-token-badgoose-face for GitHub repo: https://github.com/goose/face'

✔ Success! Wrote the pipeline for https://github.com/goose/face app at '/Users/hsinghui/workspace/aws-reinvent-2018-trivia-game/ecs-project/pipeline.yml'
Your pipeline manifest contains configurations for your CodePipeline resources, such as your pipeline stages and build steps.


Add a default password to Zoom meetings

When a Zoom organization is configured to always require authentication in meetings, not generating a meeting password implies the use of the waiting room feature, which is annoying for example when sending a DM to a colleague with a Zoom link.
This changes how we call the Zoom API to have Zoom generate a meeting password, using the settings in the user's Zoom account. The password will be included in the meeting link, users won't have to type it in manually. Example of the newly generated URLs:
https://DOMAIN.zoom.us/j/MEETING_NUMBER?pwd=ENCODED_PASSWORD

Fixes: https://chat.zulip.org/#narrow/stream/137-feedback/topic/default.20password.20for.20zoom
Screenshots and screen captures:

Self-review checklist

[x] Self-reviewed the changes for clarity and maintainability
(variable names, code reuse, readability, etc.).

Communicate decisions, questions, and potential concerns.

[ ] Explains differences from previous plans (e.g., issue description).
[ ] Highlights technical choices and bugs encountered.
[ ] Calls out remaining decisions and concerns.
[x] Automated tests verify logic where appropriate.

Individual commits are ready for review (see commit discipline).

[x] Each commit is a coherent idea.
[x] Commit message(s) explain reasoning and motivation for changes.

Completed manual review and testing of the following:

[x] Visual appearance of the changes.
[ ] Responsiveness and internationalization.
[ ] Strings and tooltips.
[x] End-to-end functionality of buttons, interactions and flows.
[ ] Corner cases, error conditions, and easily imagined bugs.

Awesome, thanks @pietroalbini! @andersk can you review this one as our zoom integration maintainer?

Looks great, merged, thanks for making this happen @pietroalbini! I expect this will be live in Zulip Cloud next week.


CAD-3092 bench:  move locli and cardano-topology

This moves the locli and cardano-topology binaries to bench/.

bors r+

bors r+

bors r+


Autostart: Unifi WAP Plugin doesn't work

I run Unifi AP Software and Homeassistant on the same device (Banana Pi). Both are autostarted with systemd.
However, Homeassistants device tracker sees every device as "not home", although they show up in Unifi Server.
I found out it has something to do with Autostart.
If i disable home assistant autostart
sudo systemctl disable home-assistant@pi
and start it with a sleep in crontab:
crontab -e
@reboot /bin/sleep 60 ; /usr/local/bin/hass --runner
everything works fine and as expected.
Editing the<EMAIL_ADDRESS>and adding
After=network.target unifi.service
does not help.

Only a guess but I think you can confirm this with the home-assistant logs.
Most likely even tough the unifi service is up and running there might be a delay before it responds. If the initialization of the component is already done by then the component fails to initialize.
Your workaround with crontab is a excellent way of working around this.

Does the unifi service have dependencies? Are those specified in the unifi unit file?

Has this been resolved?

At least on my system  /lib/systemd/system/unifi.service is
[Service]
Type=forking
Which means I don't think that After is going to be reliable in starting home-assistant after this is ready. Systemd will start home assistant after this is started.
Honestly, I think that the discovery portion of the unifi device tracker probably needs to be a bit more robust and include it's own retries if things didn't work.

Hey,
Bit of a late reply but I have a different solution I just came up with today that makes use of oneshot mechanics and simple curl to check if unifi service is available
Create the following /etc/systemd/system/hass-pre.service file (or similar)
[Unit]
Description=Home Assistant helper for UniFi
After=network-online.target unifi.service
Wants=unifi.service

[Service]
Type=oneshot
User=%i
# Enable the following line if you get network-related HA errors during boot
#ExecStartPre=/usr/bin/sleep 60
# Use `whereis hass` to determine the path of hass
ExecStart=/bin/bash -c 'while true; do curl -ksI https://0:8443/manage/account/login?redirect=%2Fmanage | grep "200 OK" > /dev/null; if [ $$? == 0 ]; then break; fi; sleep 5; done;'
RemainAfterExit=yes

Then add hass-pre.service to your After= and Wants= inside the main homeassistant systemd file.
Oneshot makes this call block dependent services (hass) until it's done with the script and RemainAfterExit makes it remain active after the script is done.
So the fake "service" will appear as activating as long as the script is in the loop and will be "activated" when it exits.
Adjust values accordingly. That one-liner simply queries a known URL that responds with "200 OK" header every 5s until it gets the correct response. I've tested and it works fine :slight_smile:
Let me know!
EDIT: If you're not running unifi.service in the same environment, you can remove it from Wants= and After= and just rely on curl result

Can't edit posts on mobile, for good measure also add -m 5 to curl parameters so it doesn't wait too long for each response


Image alt attribute with empty string should not be removed

PLEASE NOTE: make sure the bug exists in the latest patch level of the project. For instance, if you are running a 2.x version of Apostrophe, you should use the latest in that major version to confirm the bug.
To Reproduce
Step by step instructions to reproduce the behavior:
npm install<EMAIL_ADDRESS>
import sanitizeHtml from 'sanitize-html';

// EXPECT:
// ACTUAL:
console.log(
  sanitizeHtml('', {
    allowedAttributes: { img: ['alt', 'src'] },
    allowedTags: ['img']
  })
);

// EXPECT:
// ACUTAL:
console.log(
  sanitizeHtml('', {
    allowedAttributes: { img: ['alt', 'src'] },
    allowedTags: ['img'],
    nonBooleanAttributes: []
  })
);

Expected behavior
alt="" should be left as-is.
Describe the bug
Empty alt attribute means the alt text of the image is intentionally left blank. For example, border graphics may not need alt attribute if they are unimportant decoration.
When alt is an empty string, screen reader will skip the element and read nothing.
When alt is not set, screen reader will read "image".
So, alt="" is not same as alt or unset.
To workaround the issue, use .
Details
Version of Node.js: 18.16.0
PLEASE NOTE: Only stable LTS versions (10.x and 12.x) are fully supported but we will do our best with newer versions.
Server Operating System: Ubuntu 20.04 on WSL
The server (which might be your dev laptop) on which Apostrophe is running. Linux? MacOS X? Windows? Is Docker involved?
Additional context: Seems regression of #529.

Add any other context about the problem here. If the problem is specific to a browser, OS or mobile device, specify which.

Screenshots
If applicable, add screenshots to help explain your problem.
$ cat index.mjs
import sanitizeHtml from 'sanitize-html';

// EXPECT:
// ACTUAL:
console.log(
  sanitizeHtml('', {
    allowedAttributes: { img: ['alt', 'src'] },
    allowedTags: ['img']
  })
);

// EXPECT:
// ACUTAL:
console.log(
  sanitizeHtml('', {
    allowedAttributes: { img: ['alt', 'src'] },
    allowedTags: ['img'],
    nonBooleanAttributes: []
  })
);
$ node ./index.mjs

How to work on this issue

Fork this repo into your own account
Make changes to the code - since it is in your account, you could work directly on main, but wouldn't you rather get some Git practice by working on a branch, and then merging to your own main?
Write one or more tests to make sure your feature works
Make sure your code changes don't break any existing tests.
Update the CHANGELOG.md file - since other people might contribute during the same sprint cycle, add your change log message under 'UNRELEASED', not a specific version. We will add the version number when everything is approved and published on npm.
When you are done, return to this repository and create a PR to pull code from your fork. Read more about this here. Make sure to fill out the PR template as best as you can.
Navigate to our Discord server here if you have already joined, or here if you need an invitation and post a message in the open-source contribution channel that you need a PR review.
After (hopefully) a short amount of time, one of our team engineers will review your PR and potentially advise you about things they want to see changed.
If you need to make changes, go back to your local fork, make changes, and contribute those back to the main repo. This will update the PR.
When you are done with changes and want a re-review, ask in Discord once again.
After your PR is accepted, celebrate!!! 🎉

Code suggestions for this issue

Start by writing a test that should pass. This will give you back info on how it is failing.
Next, focus on the section(s) of code that are important for attributes. The code base is a little large, so don't get lost in it.

Should alt be a non-boolean attribute? Does it matter for this code change?
What line of code actually deletes the =""?

Rather than letting any attribute have a value of ="", create a new option (potentially with default values) that lets the end user define what tags can have this value - like the allowedAttributes option.
Make sure to write additional tests that show this option works!

Finally, please make use of the Discord to ask questions. Try to answer the questions using internet resources, but don't be afraid to ask questions on the Discord about anything. We are here to help!

Hi. I am new here and just had a look at this issue. The 2nd case above seems correct to me - alt is implicitly alt="".  Is that right? (unless we actually want to specify alt="")
// EXPECT:
// ACUTAL:
console.log(
  sanitizeHtml('', {
    allowedAttributes: { img: ['alt', 'src'] },
    allowedTags: ['img'],
    nonBooleanAttributes: []
  })
);

@zhna123 - There are cases where the image being added is purely decorative. In that case, we don't want explicit alt text. However, for your HTML to be in spec and pass validation, it needs an alt attribute. Leaving that attribute as a boolean without string will cause some screen readers to output "no alt text" and others will output the filename of the image. Both of these are disrupting to the UX of the page. So, the OP is correct, the alt tag should be allowed to be set to an empty string.

I agree it would make sense to fix this one specifically for the alt
attribute.
On Tue, Sep 26, 2023 at 6:27 AM Robert Means @.***>
wrote:

@zhna123 https://github.com/zhna123 - There are cases where the image
being added is purely decorative. In that case, we don't want explicit alt
text. However, for your HTML to be in spec and pass validation, it needs an
alt attribute. Leaving that attribute as a boolean without string will
cause some screen readers to output "no alt text" and others will output
the filename of the image. Both of these are disrupting to the UX of the
page. So, the OP is correct, the alt tag should be allowed to be set to
an empty string.
—
Reply to this email directly, view it on GitHub
https://github.com/apostrophecms/sanitize-html/issues/633#issuecomment-1735261269,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAH27OAQN37VYC6CLHLMETX4KUXZANCNFSM6AAAAAA4RB5HGI
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

--
THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER
APOSTROPHECMS | apostrophecms.com | he/him/his

Okay. I used VoiceOver to test alt without explicitly specify ="", it didn't announce anything, but removing it will be "unlabeled image". But it's true that I can't guarantee all the screen reader will behave the same way.. I created a pr to fix this.


Why Can't We Run Multiple Dependent Transactions?

What I want to do: Have a group of transactions be atomic. That is, if at least one in the group fails, all of the transactions fail (the transaction succeeds only if all of the transactions in the group succeed).
Why I need it: I want users to be able to buy cosmetics with in-game currency. As you can make out, there are two dependent transactions here:

Transaction to subtract the cost of the item from the user's available currency.
Transaction to update that the user now owns this item.

How I am doing it:
// (This is inside the method where the user selects to buy a certain cosmetic.)
// Pay for the cosmetic.
DataManager.ModifyCoinsBy(-cosmetic.Cost, task =>
{
    // Payment was not successful.
    if (task.Exception != null || task.IsCanceled || task.IsFaulted)
    {
    	// Notify user of reason.
    }

    // Payment was successful.
    else
    {
    	// Register the user as now owning this new cosmetic.
    	// NOTE: This is the problem. What if this transaction fails? We took the user's payment
    	// without successfully registering their newly owned cosmetic item.
        DataManager.AddNewOwnedCosmeticId(cosmetic.Id);
    }
});

// Modifies the user's coins by amount and runs continueWith when the modification has completed.
internal static void ModifyCoinsBy(int amount, Action<Task<DataSnapshot>> continueWith = null)
{
    coinsReference.RunTransaction(data =>
    {
        if (Convert.ToInt32(data.Value) + amount >= 0)
        {
            data.Value = Convert.ToInt32(data.Value) + amount;
            return TransactionResult.Success(data);
        }
        return TransactionResult.Abort();
    }).ContinueWith(continueWith);
}

// Adds a new owned cosmetic ID to the user's list of owned cosmetic IDs.
internal static void AddNewOwnedCosmeticId(int id, Action<Task<DataSnapshot>> continueWith = null)
{
    ownedCosmeticIdsReference.RunTransaction(data =>
    {
        List<object> currentlyOwnedCosmeticIds = data.Value as List<object>;
        if (currentlyOwnedCosmeticIds == null)
        {
            data.Value = new List<int> { id };
        }
        else
        {
            currentlyOwnedCosmeticIds.Add(id);
            data.Value = currentlyOwnedCosmeticIds;
        }
        return TransactionResult.Success(data);
    }).ContinueWith(continueWith);
}

Picture of my Firebase database:

Any help is appreciated! Is there any way to do this properly as I've described above? I've searched high and low.

I think I've solved my own problem. Why didn't I realize I could just do this!
internal static void PurchaseCosmeticItem(int cosmeticPrice, int cosmeticID, Action<Task<DataSnapshot>> continueWith = null)
{
    // Run the transaction to modify the user's coins.
    coinsReference.RunTransaction(currentCoinsData =>
    {
        // If the user can afford this cosmetic.
        if (Convert.ToInt32(currentCoinsData.Value) + cosmeticPrice >= 0)
        {
            // Modify their coins accordingly.
            currentCoinsData.Value = Convert.ToInt32(currentCoinsData.Value) + cosmeticPrice;

            // Now run the transaction to modify that they now own this cosmetic.
            Task<DataSnapshot> task = ownedCosmeticIdsReference.RunTransaction(currentOwnedCosmeticIdsData =>
            {
                // Get the user's currently owned cosmetic IDs.
                List<object> currentOwnedCosmeticIds = currentOwnedCosmeticIdsData.Value as List<object>;

                // This is the first cosmetic they've purchased.
                if (currentOwnedCosmeticIds == null)
                {
                    currentOwnedCosmeticIdsData.Value = new List<int> { cosmeticID };
                }
                else
                {
                    // Abort if the user already owns this cosmetic (this shouldn't ever happen!).
                    if (currentOwnedCosmeticIds.Contains(cosmeticID))
                    {
                        return TransactionResult.Abort();
                    }

                    // They now own this cosmetic.
                    currentOwnedCosmeticIds.Add(cosmeticID);
                    currentOwnedCosmeticIdsData.Value = currentOwnedCosmeticIds;
                }
                return TransactionResult.Success(currentOwnedCosmeticIdsData);
            });

            // If adding the new cosmetic ID failed, make the transaction to modify their coins fail as well.
            if (task.Exception != null || task.IsCanceled || task.IsFaulted)
            {
                return TransactionResult.Abort();
            }

            // The transaction to modify the user's coins and add their newly owned cosmetic item succeeded!
            return TransactionResult.Success(currentCoinsData);
        }

        // The user could not afford the cosmetic.
        return TransactionResult.Abort();

    }).ContinueWith(continueWith);
}

Let me know if this is good! 😁

Regarding to your solution, I don't think it will work because you have to wait until the nested transaction is done before you evaluate task.Exception, task.IsCanceled and task.IsFaulted.  However, you can chain the second transaction using the ContinueWith() from the first transaction.  This would not allow you to abort the first transaction but I will explain how you can do it later.
Also, if you are using Unity 2017+, and your app works well with .NET 4.6, I would recommend you to utilize await, which would make your chain of transaction looks cleaner.
https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/keywords/await
I do not know anyway to abort one transaction when another one fails.  However, you can use a third transaction to credit the balance back if the second one fails.  You can chain it with ContinueWith() or use await for cleaner code.
Another consideration is: what is the scenario that it fails to grant cosmetic?  You may not even need a transaction to grand the cosmetic.

You can use ownedCosmeticIdsReference.Push().SetValue() to add your cosmetic id to the list with auto-generated key, which is a combination of server time and a random key.
https://firebase.google.com/docs/reference/unity/class/firebase/database/database-reference#class_firebase_1_1_database_1_1_database_reference_1a0eb02bbcb15c6ac458a1a4240a25e1af
You can use the cosmetic id as the key and quantity as value (1 for example).  If you are expecting single client scenario, or no consumable , SetValue() should be sufficient. Otherwise, you may still need to use transaction, but in smaller scale.

In addition, you may consider to use Firebase Function for slightly complicated transaction like this.
https://firebase.google.com/docs/reference/unity/namespace/firebase/functions
You may still need to use RunTransactoin() in the Function, depending on your case, however, it has some benefits.

It is generally a better practice to have the server to handle currency and entitlement.  You can control the client to only able to write to certain part of the database through Firebase Function and only allow the client to read data.
Simpler client side code.  Only one request/response should handle multiple transactions.
Able to extend on the server side.  Ex. add more validation without pushing new client, add real-money transaction.

Shawn

@chkuang-g

Regarding to your solution, I don't think it will work because you have to wait until the nested transaction is done before you evaluate task.Exception, task.IsCanceled and task.IsFaulted.
That is precisely what I want to do, though. I want to wait for the nested transaction to finish before I evaluate those. Also, it works great! 😉

I do not know anyway to abort one transaction when another one fails. However, you can use a third transaction to credit the balance back if the second one fails. You can chain it with ContinueWith() or use await for cleaner code.
This method would only add to the current problem. What if this third transaction fails?...
Also, a way to "abort one transaction when another one fails" is in my solution.

In addition, you may consider to use Firebase Function for slightly complicated transaction like this.
I've been looking into Firebase Cloud Functions for other parts of my project. Thanks. 😄

Closed until I guess my "solution" is shown to not work.


Close developer tools window - window reopens

Steps to reproduce:

Launch browser.
Toggle open developer tools window, either View >> Toggle Developer Tools or CTRL + ALT +I.
Switch back to main SAFE browser window and attempt to close developer tools - again either by the menu option or the keyboard shortcut.

Expected:
Developer tools window remains closed.
Actual:
Developer tools window reopens by itself.
https://maidsafe.slack.com/files/UENBNQHSA/FHUMXQZHQ/2019-04-25_10-11-42.mp4

blocked until #593 , where we should more easily be able to pass around 'openDevTools' for a given window/active tab combo

this was fixed in #766

Fixed in #781


Is FTP supported?

Benthos only supports SFTP, but FTP does not. Is that right?

Have you considered supporting FTP?

Hey @togettoyou, it's not supported currently, we'd need to add a new input.


The integration is overwriting an asynchronous queue

@hankim813 The Kissmetrics integration is ignoring all information set before Segment is loaded by executing this line in initialize: window._kmq = []; here: https://github.com/segment-integrations/analytics.js-integration-kissmetrics/blob/master/lib/index.js#L50
This breaks the Optimizely/Kissmetrics integration. Is it possible to fix this?

Ahhh so we should be doing window._kmq = window._kmq || [];?

To be honest: I think that should work but I haven't read through the
entire code of the integration. I was just looking for scripts that were
overwriting the _kmq variable and found this line.
On Wed, Jun 15, 2016 at 8:27 PM, Han<EMAIL_ADDRESS>wrote:

Ahhh so we should be doing window._kmq = window._kmq || [];?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/segment-integrations/analytics.js-integration-kissmetrics/issues/17#issuecomment-226379336,
or mute the thread
https://github.com/notifications/unsubscribe/ADv9sWnYpVwJTkv7Tqt496VUxPSv0NNtks5qMMKagaJpZM4I2-pD
.

--
Lucas Swartsenburg
Partner Solution Architect | Optimizely, Inc. http://www.optimizely.com/
<EMAIL_ADDRESS>| (1) 415 996 97 96
We're hiring! http://www.optimizely.com/jobs Talk to me if you're
interested!

This was fixed in d1e7b30834656298b4b6ee1a0eabf55c5653a69a.

@lucasoptimizely yep will close this once the fix is deployed. Right now we're undergoing a rollout of new version of analytics.js so I'm hoping to be able to release the new KM fixes/changes sometime next week

Hi @lucasoptimizely, as part of the monorepo migration, this issue has been moved to new issue. Our engineers have been notified and will prioritize and work on it ASAP. Thank you!
For more information, see README.md.


Added pkg-config file

The pkg-config file helps external libraries to find and handle this library.

Unfortunately it is more complex than this. If you wan't to improve it, please have a look at https://github.com/goldhoorn/libpointmatcher/blob/master/libpointmatcherConfig.cmake.in. This file is basically doing the same job but via cmake. So the variables used to generate should be pretty much the same.
But before you invest more time I recommend waiting till we decided whether we'll want to support pkg-config at all. We need to bound the maintenance effort..

Out build system depends on pkg-config (there are other ways but this is the standart)
To help for decitions regardin pkg-config:
pkg-config is more portable than cmake, pkg-config files can be used within makefile, autotools and cmake, even some windows tools already support this.
The CMake-find scripts are nice too. But supporting pkg-config should not be that much overhead from my expirience.
Let me know when you have taken a decition regarding the pkg-config support i will update my PR then as soon as possible. Furthermore feel free to ask me furhter questions if needed.

@pomerlef, I think it is worth the little effort and it can be easily dropped as feature at any time. Agreed?

Furthermore it's a open source project, the file can cause only problem
for people that using the pkg-config system.
The are open to deliver patches if they are unhappy or discovered a bug
(like me ;) ).
On 24.08.2015 12:27, Hannes Sommer wrote:

@pomerlef https://github.com/pomerlef, I think it is worth the
little effort and it can be easily dropped as feature at any time. Agreed?
—
Reply to this email directly or view it on GitHub
https://github.com/ethz-asl/libpointmatcher/pull/109#issuecomment-134136132.

Okay let's do it :). If your follow my suggestion concerning variables (https://github.com/ethz-asl/libpointmatcher/pull/109#issuecomment-133371114), I'll merge it :).
Thanks for the patience.

Just had a look,
you generate complete paths, this is not how pkg-config (normally) works. library normally given as -L and -lpointmatcher in your variables you generate complete paths. I could use them but i would recommend to keep the file-layout as it is currently.

If you don't want to use hard coded names within the pkg-config i can propose a cleanup of the CMakeLists.txt that also for the lib-generation a variable is used, and then re-use this variable within your cmake-based find-file and mine pkg-config file.

Hm, I see the point. But unusual doesn't mean it won't work. If I understand it right it actually should work with absolute paths for libraries as well. So instead of -L${libdir} -lpointmatcher at https://github.com/ethz-asl/libpointmatcher/pull/109/files#diff-bf5130bdd1d9320751e7c57f2fed0d13R9 use @POINTMATCHER_LIB<EMAIL_ADDRESS>Could you try that out? If it works, I would prefer this because it would require minimal changes to the CMakeLists.txt and is simpler.
There is one complication more:
We actually generate two libpointmatcherConfig.cmake based on one libpointmatcherConfig.cmake.in (see https://github.com/goldhoorn/libpointmatcher/blob/master/CMakeLists.txt#L338 and https://github.com/goldhoorn/libpointmatcher/blob/master/CMakeLists.txt#L360). We install then the second version only (see https://github.com/goldhoorn/libpointmatcher/blob/master/CMakeLists.txt#L366) the first is sufficient (containing paths pointing to the build folder) for where it is build.  For the pkg-config file this would need to be done similarly. But I don't know whether the first version (actually the one that you generate with this PR) makes sense for pkg-config. To access the paths for installation it needs to generated below https://github.com/goldhoorn/libpointmatcher/blob/master/CMakeLists.txt#L353.

If it works, I would prefer this because it would require minimal changes to the CMakeLists.txt and is simpler.

I will try out soon

We install then the second version only

Why you do need two version?, are you using somewhere (externally?) the non-installed version of the lib?, i never saw such layout. Even if you do use the non-installed version, you need already to know where it is (to find the .cmake files). So why you need a local Find.cmake file then?
I can prodive a generated .pc file which sets the PREFIX to the actual build dir if this would fit your need, but i don't understand the need for shuch setup.

Apparently there are or have been people using this type of layout, e.g. by adding the build folder to the CMAKE_PREFIX_PATH (see http://www.cmake.org/cmake/help/git-master/manual/cmake-packages.7.html#creating-a-package-configuration-file-for-the-build-tree).
Maybe even ros_build or the early catkin versions were using this (pure speculation).
But I think it is better to start with the install tree pkg-config file. And if somebody is missing a build tree version it can be added.

Sorry for the delay, I was on holiday...
Thanks Matthias for the patch and Hannes for the support!
The local  libpointmatcherConfig.cmake was their to simplify the work of the developers and for a custom installation layout for partners in a EU project. It evolved like this more than was planed like this.
Following Hannes comment, I'm glad to add another building tool to the library but as we are not familiar with it, you will have to help for the support.
Where are we at with the modifications asked?

Sorry for the delay from my side, i just updated the PR regarding to your last comments.
This is working for my use case. Would be great to have this in the mainline.

...and rebased

Thanks for the new feature. If your lab is having nice results to show with the library, just let me know. I'll add them to Projects and Partners and Applications and Publications.

I will do, thanks for your support to adding this feature!
On 02.10.2015 15:40, François Pomerleau wrote:

Thanks for the new feature. If your lab is having nice results to show
with the library, just let me know. I'll add them to Projects and
Partners
https://github.com/ethz-asl/libpointmatcher#projects-and-partners
and Applications and Publications
https://github.com/ethz-asl/libpointmatcher/blob/master/doc/ApplicationsAndPub.md.
—
Reply to this email directly or view it on GitHub
https://github.com/ethz-asl/libpointmatcher/pull/109#issuecomment-145023396.


Improve gRPC IPC doc

Fixes https://github.com/grpc/grpc-dotnet/issues/1837
Internal preview: https://review.docs.microsoft.com/en-us/aspnet/core/grpc/interprocess?view=aspnetcore-7.0&branch=jamesnk%2Fipc-refactor

Hey, I tried and could not work it out. Do you have an example of IPC using GRPC in .NET core. Please assist.

What couldn't you work out? Please be specific so the doc can be improved.

There is an example of using UDS here: https://github.com/grpc/grpc-dotnet/tree/master/examples#transporter
What couldn't you work out? Please be specific so the doc can be improved in the future.

I want to have two WPF apps, one WPF client may be able to send a message to another WPF client using GRPC.
I have made an app that 1 WPF can communicate to the server and the response is sent back to WPF client 1. I want to send a message from WPF Client 1 to GRPC Server then to WPF Client 2. How can I achieve that? I am using Bidirectional stream at both ends (Client-Server)


Error messages take too much space / hard to read

I think error messages should take only a single line (instead of 4+ like above).
See https://github.com/sdispater/pastel/issues/1 for the readability issue.

As per https://github.com/sdispater/pastel/issues/1#issuecomment-383604180 this sounds to be a wont-fix unfortunately.
I've had some hopes regarding poetry fixing the mess with pipenv, but obfuscating error messages like this really reminds me of the CLI issues with pipenv itself.
btw: I've used click for an own project and really liked it, but always found that pipenv does not get it right.  poetry using cleo made me look briefly at it, but as for poetry's CLI it wasn't much for the better.

Well, for me they are unreadable (i.e. "hidden"), and I've meant obfuscated in the sense of decorating them (with 2 lines above and below).
For me it's easier to parse
ERROR: foo

than

  ERROR
  foo

(disregarding the colors)

That's a matter of preferences, I guess. I prefer how it's currently formatted so I won't change it.
But if that's what turns you off poetry, so be it. I just think it's a pity to disregard the features poetry brings just for a matter of color choice and error formatting.

I just think it's a pity to disregard the features poetry brings just for a matter of color choice and error formatting.

I agree.  That's why I am pointing them out explicitly - I think others might agree eventually - espcially when considering pastel as a lib (https://github.com/sdispater/pastel/issues/1).
And with regard to "just for a matter of color choice": it's just unreadable for me, and therefore not accessible - it's like a webpage that is not accessible (due to missing alt texts or just outputting audio by default, and not providing any text).
Anyway, I'll be watching this project and really like it for trying to fix the things with pipenv already.


librespot_playback could not start/write audio

hi,
I am using raspotify and librespot to make my raspberry pi to one spotify connect client.
When I was using an USB-Headset, this has worked. Now I am trying to make this work with bluetooth speaker. I can play sound on raspberry pi with pulseaudio an thus, I have build the librespot with pulseaudio as backend.
When I start raspotify, I get this error:
Aug 03 14:41:42 raspberrypi librespot[1186]: [2020-08-03T12:41:42Z ERROR librespot_playback::player] Could not start audio: Connection refused
Aug 03 14:41:42 raspberrypi librespot[1186]: [2020-08-03T12:41:42Z ERROR librespot_playback::player] Could not write audio: Not connected to pulseaudio
Could someone help me with this?
Thank you :)

This isn't a librespot problem.  If you can get your system to play audio from the device with other programs then librespot will also work.
It looks like you have configured librespot to use pulseaudio but you aren't running pulseaudio, so once you fix your audio config them librespot should start working.  Hopefully your Linux distro support can point you in the right direction there.

All the others playback programs are already working with pulseaudio. But I still have the same error with librespot..

In the same terminal as you are using to run librespot, what does the command "pacmd list-sinks" report?  This will ask PulseAudio to show which playback device it is using.


Publish 02/14/2022 3:30 PM PT

Docs Build status updates of commit 9ce3953:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

skype/skype-ps/skype/New-CsTeamsMeetingPolicy.md
:white_check_mark:Succeeded
View (skype-ps)

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


cant sync the right day，when i sync tody's todoist completed tasks,it contain yesterday's and some of them are today's

I found this bug too, I guess time zone error and I set template.
%% TCT_TEMPLATED_START <% tp.date.now("YYYY-MM-DD", 0) %> 08:00 %%

%% TCT_TEMPLATED_END <% tp.date.now("YYYY-MM-DD", 0) %> 07:59 %%

Same problem for me, see example here:
%% TCT_TEMPLATED_START 2024-12-29 00:00 %%
* Io
    * MP_s 2024-12-28 23:11
    * MP_m 2024-12-29 16:54
* Salute
    * Acqua 1 l 2024-12-28 23:11
    * Acqua 1,25 l 2024-12-28 23:11
%% TCT_TEMPLATED_END 2024-12-29 23:59 %%

Three tasks from 28 December appear as completed inside 29 December. This is happening since some days. It worked perfectly beforehand.


Cannot get schema name when using modifier

Package version
2.1.0
Describe the bug
When using a modifier on a schema, e.g., .optional(), you can no longer get the schema name.
Example:
const symbol = Symbol.for("schema_name");

const schema = vine.string();
const withOpt = vine.string().optional();

console.log(schema[symbol]); // "vine.string"
console.log(withOpt[symbol]); // undefined

Reproduction repo
No response

Hey, can you share the purpose for which you are reading the schema_name?

@thetutlage The purpose is to dynamically populate a form with fields depending on the type.
i.e., name is a string, so use a text input, age is a number, so use a number input.
The goal is that we can reuse the Vine schema for this information, instead of creating another object with the mappings.

The schema_name property is for internal usage via unionOfTypes schema type and using it for other purposes may not provide the desired results.
So, for your use-case, you can use the JSON representation of the schema.
const validator = vine.compile(
  vine.object({

  })
)

console.log(validator.toJSON())

Ah, I see.
The toJSON() representation doesn't appear to give the type as "string", "number", "boolean", etc. Only "literal".

Is there a way to get this type information?

Yeah. It will be nice to encode that info. Lemme do that

May I make a request that it be exposed in both the compiled toJSON() representation, and the schema.getProperties() object?
Thank you for looking at this, it's really only a minor thing in the grand scheme of things, but it does mean less technical maintenance for us, so it is greatly appreciated.


⚠ AsianCinema has degraded performance

In 2fb9b91, AsianCinema ($ACM) experienced degraded performance:

HTTP code: 200
Response time: 4083 ms

Resolved: AsianCinema performance has improved in f69e35f after 22 minutes.


docs about default http methods

https://github.com/channelcat/sanic/blob/master/docs/sanic/routing.md#http-request-types

By default, a route defined on a URL will be used for all requests to that URL. However, the @app.route decorator accepts an optional parameter, methods, which restricts the handler function to the HTTP methods in the given list.

The documentation about request methods still reads to me as if the default is that all methods are accepted. With #316 this default was changed, so that only GET is the default.

Closing this per https://github.com/channelcat/sanic/pull/377


testcontainers patch on graviton2 (ec2 aws arm64)

Hi Teams,
Currently i'm working on porting airbyte to ec2 graviton2 arm processor. One of dependencies is testcontainers as it's used a lot on the e2e test.
I've manage to fix some lombok issues. Basically just upgrading to the new version 1.18.22.
Here's the commits and repos.
https://github.com/weltam/testcontainers-java/commit/c95e0142acb9fa58193af284a74bdc019acc981c

Nowadays, we are using lombok 1.18.24. That should fix your issue @welly87 ! Feel free to reopen if still having issues with latest versions of testcontainers.


fix: allow webview to load non-web URLs

By default Chromium does not allow webview to load non-web URLs like file://, this patch disables the behavior.
This is required by https://github.com/electron/electron/pull/13869.

This seems to bypass external protocol and web safe scheme check for other non-web schemes, can we just add a check for file alongside and preserve the existing behavior ? If electron custom protocol schemes are failing to load, we have an api which consumers should use to register them as web safe scheme https://github.com/electron/electron/blob/master/atom/browser/api/atom_api_protocol.cc#L42 , thoughts ?
Another solution I would suggest from my experience with integrating PDF is, it had a similar problem because pdfs are embedded in a webui page using  which are essentially browser plugins and they aren't allowed to load local schemes by default. But the way its done is by intercepting the initial request where we tell chromium that the resource will be handled as a stream and we make chromium believe it by changing the origin to chrome://pdf-viewer https://github.com/electron/electron/blob/master/atom/browser/atom_resource_dispatcher_host_delegate.cc#L116-L143 . Then the final resource is loaded by us a stream https://github.com/electron/electron/blob/master/atom/browser/ui/webui/pdf_viewer_ui.cc#L235-L249, that way local schemes can be loaded selectively. Maybe similar thing can be attempted for <webview> ?

@deepak1556 Currently webview is supposed to be able to load arbitrary URL, requiring custom protocols to be web-safe would be a breaking change.
While the comment in the related Chromium code says "do not allow browser plugin guests to navigate to non-web URLs", it actually only affects OOPIF based guests. Our old browser plugin based webview did not hit this check, and this check only starts to affect us after migrating webview to OOPIF.
So this patch should not affect normal pages, and it only makes OOPIF based webview behave the same with browser plugin based webview.
I'm not sure if I can adapt the solution for PDF viewer for OOPIF webview. PDF is loaded as resource, while the non_web_url_in_guest check happens as part of page navigation, and it happens before any kind of loader.

requiring custom protocols to be web-safe would be a breaking change.

Hmm yeah you are right, can we consider this for https://github.com/electron/electron/blob/master/docs/api/breaking-changes.md in the future, seems to make this check a bit strict.

So this patch should not affect normal pages, and it only makes OOPIF based webview behave the same with browser plugin based webview.

👍

I'm not sure if I can adapt the solution for PDF viewer for OOPIF webview. PDF is loaded as resource, while the non_web_url_in_guest check happens as part of page navigation, and it happens before any kind of loader.

Yeah, makes sense. Maybe if we associate the <webview> with an internal mime type then we could get the loaders to kick in and intercept them. But yeah this can be done as an experiment separately, not in the scope of the current migration. Thanks for the clarification!

Hmm yeah you are right, can we consider this for https://github.com/electron/electron/blob/master/docs/api/breaking-changes.md in the future, seems to make this check a bit strict.

I'm 👍  matching Chromium's security rules on webview.


Lens option not working ?

First a huge thank you for this package, which is really easy to use and gives great cornerplot !
But I think there is a problem with the "lens" option. I can't reproduce your examples with Julia version 1.7. The errors depend on the type of plot I need.
If I try to "lens" an histogram (e.g. corner(table, lens=(:b, :a)) in your example), I have :
no method matching hist
Then, If I try to lens a scatter plot (e.g. corner(table, lens=:a) in your example), I have :
no method matching (::PairPlots.var"#fmt#31")(::String)
I really used your example in a copy-paste way, in a fairly new and simple Julia environment. I will take a look on the code If I have some time :) But I'm not a very advanced Julia user so...

Hi @Simon-Delcamp!
Thanks for reporting this issue.  I can confirm that functionality broke.
I'll take a look over the coming week.
Thanks!

No problem! The new branch lensoption is now created.

Thanks @Simon-Delcamp, merged and submitted!
https://github.com/JuliaRegistries/General/pull/61423


minor followup on #510

Codecov Report
Patch coverage has no change and project coverage change: -2.21 :warning:

Comparison is base (d6ec227) 88.82% compared to head (7c6a043) 86.61%.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #517      +/-   ##
==========================================
- Coverage   88.82%   86.61%   -2.21%
==========================================
  Files          10       10
  Lines        2970     2958      -12
==========================================
- Hits         2638     2562      -76
- Misses        332      396      +64

Impacted Files
Coverage Δ

src/JET.jl
78.80% <ø> (-1.41%)
:arrow_down:

src/analyzers/jetanalyzer.jl
79.70% <ø> (-8.55%)
:arrow_down:

src/analyzers/optanalyzer.jl
93.26% <ø> (ø)

... and 4 files with indirect coverage changes

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

Thanks for cleaning up after me 🙂 !


improve performance of count_data

What problem does this PR solve?
When we precisely count data for a large table, it will cost minutes or hours.
What is changed and how does it work?
Actually,we just need the count of data.So we just need transfer the count of data from server to client,  but not the detailed data.
In our test, it will 10x faster than before.
Tests

Unit test
Manual test (add detailed scripts or steps below)

create a table with millions of data.
use "count_data -c -f"

Related changes

Need to update the documentation
Need to be included in the release note

@neverchanje @levy5307
Can you help to review?Thanks

@Shuo-Jia @levy5307
Thanks your review.
In our scenario, count data precisely is a frequent operation after we bulkload sst files.And the table has a large amount of data, so it often takes a long time.
So the performance of count data precisely is very helpful for us.
Refactor of scan is better idea.Do I need to optimize the code to another RPC, or close this PR?

@Shuo-Jia @levy5307
Thanks your review.
In our scenario, count data precisely is a frequent operation after we bulkload sst files.And the table has a large amount of data, so it often takes a long time.
So the performance of count data precisely is very helpful for us.
Refactor of scan is better idea.Do I need to optimize the code to another RPC, or close this PR?

Yes, I think it's good to open a new pull request to add another new rpc. @ZhongChaoqiang

@Shuo-Jia @levy5307
Thanks your review.
In our scenario, count data precisely is a frequent operation after we bulkload sst files.And the table has a large amount of data, so it often takes a long time.
So the performance of count data precisely is very helpful for us.
Refactor of scan is better idea.Do I need to optimize the code to another RPC, or close this PR?

Yes, I think it's good to open a new pull request to add another new rpc. @ZhongChaoqiang

@levy5307 @Shuo-Jia
我们开发这个功能除了可以优化count_data的性能外，还有另外一个场景，就是快速查询某个scan条件(例如指定范围或前缀条件等的scan)的kv数量。
如果使用单独的RPC，感觉和scan重复的功能太多了。所以，可以再帮忙看看，是不是还是放在现在的scan功能中会更合适一些呢？这样代码会简洁很多。谢谢！

对于重复的代码，可以考虑抽出来以复用，这样是否可以？@ZhongChaoqiang

@Shuo-Jia @levy5307
Thanks your review.
In our scenario, count data precisely is a frequent operation after we bulkload sst files.And the table has a large amount of data, so it often takes a long time.
So the performance of count data precisely is very helpful for us.
Refactor of scan is better idea.Do I need to optimize the code to another RPC, or close this PR?

Yes, I think it's good to open a new pull request to add another new rpc. @ZhongChaoqiang

@levy5307 @Shuo-Jia
我们开发这个功能除了可以优化count_data的性能外，还有另外一个场景，就是快速查询某个scan条件(例如指定范围或前缀条件等的scan)的kv数量。
如果使用单独的RPC，感觉和scan重复的功能太多了。所以，可以再帮忙看看，是不是还是放在现在的scan功能中会更合适一些呢？这样代码会简洁很多。谢谢！

对于重复的代码，可以考虑抽出来以复用，这样是否可以？@ZhongChaoqiang

主要还不是代码的问题。由于我们多个scan的接口都有用到，例如get_scanner/async_get_scanner/get_unordered_scanners/async_get_unordered_scanners，如果count功能不走scan这条路线的话，就可能要新增多个count的接口，以对应原来的scan的功能了，因为count接口的查询条件要和scan接口的保持一致。感觉是不是更复杂了？@Shuo-Jia

@Shuo-Jia @levy5307
Thanks your review.
In our scenario, count data precisely is a frequent operation after we bulkload sst files.And the table has a large amount of data, so it often takes a long time.
So the performance of count data precisely is very helpful for us.
Refactor of scan is better idea.Do I need to optimize the code to another RPC, or close this PR?

Yes, I think it's good to open a new pull request to add another new rpc. @ZhongChaoqiang

@levy5307 @Shuo-Jia
我们开发这个功能除了可以优化count_data的性能外，还有另外一个场景，就是快速查询某个scan条件(例如指定范围或前缀条件等的scan)的kv数量。
如果使用单独的RPC，感觉和scan重复的功能太多了。所以，可以再帮忙看看，是不是还是放在现在的scan功能中会更合适一些呢？这样代码会简洁很多。谢谢！

对于重复的代码，可以考虑抽出来以复用，这样是否可以？@ZhongChaoqiang

主要还不是代码的问题。由于我们多个scan的接口都有用到，例如get_scanner/async_get_scanner/get_unordered_scanners/async_get_unordered_scanners，如果count功能不走scan这条路线的话，就可能要新增多个count的接口，以对应原来的scan的功能了，因为count接口的查询条件要和scan接口的保持一致。感觉是不是更复杂了？@Shuo-Jia

不太明白，scan一个rpc，count一个rpc，count和scan共用一套"迭代器"，scan的client接口完全可以保持不变；只需要添加count的api不就行了？

@ZhongChaoqiang 你好，这项工作可以继续吗？我们的使用场景中对这个功能也有比较强烈的需求

共用rpc我觉得倒没有问题。问题点应该在count数值应该放在哪里，序列化在 struct get_scanner_request 里面不是很合适，在 scan_request 里面加一个标记表示是否只取count，并在 scan_response 把count结果带出来或许更合适。
新增的字段都是 optional 的，当server端是老版本，即不支持这个新增字段的判断时，会使用老接口逻辑把kv原始数据带出去，新的client可以以此判断server端是否支持，如果不支持，业务逻辑层可以返回失败。
更干净的做法当然还是新增一个独立的rpc，但是get_scanner还是可以复用的。

Has been implemented by https://github.com/apache/incubator-pegasus/pull/1091, so I close it.
Thanks @ZhongChaoqiang


[TC-ETOCONC-2.1] SDK has to be implemented for Ethylene Oxide Concentration Measurement Cluster

All-clusters-app has to be enabled with below mentioned Attributes :

[ ]  MeasuredValue
[ ]  MinMeasuredValue
[ ]  MaxMeasuredValue
[ ]  PeakMeasuredValue
[ ]  PeakMeasuredValueWindow
[ ]  AverageMeasuredValue
[ ]  AverageMeasuredValueWindow
[ ]  MeasurementUnit
[ ]  MeasurementMedium
[ ]  LevelValue

Bug prevalence
Whenever I try to read all these attributes
GitHub hash of the SDK that was being used
7ae5919a046a18c7e60537aff814977ee424b9cf - Master
Platform
raspi
Platform Version(s)
RPI - 4, 8GB RAM

Will be fixed by https://github.com/project-chip/connectedhomeip/issues/27662


ADAT Improvements

Issues and bug fixes related to #388
Bug Fixes:

[x] Alternate scoring - make optional with hierarchy
[x] Alternate scoring - graphs missing
[x] Hierarchy info missing from Images folder

New Features:

[x] Load command line arguments from json/ini file with --file option
[x] Compare two files - new script to generate diff options
[x] GUI for saving/editing configs, invoking visualizations

Notes

New python module "csvdiff" is required. Can be installed using pip install csvdiff.

GUI uses pyqt4.

The diff is visible in "Image Result" table in generated HTML.

The code is more modular with main functions moved to "adat_classification.py" file.

"generate-visualization.py" is responsible for parsing command-line arguments and invoking html generate function from "adat_classification.py"

"adat_classification.py" can now be imported to external python scripts.

@bijayraj missing 'import argparse`
 python toolkit/amd_data_analysis_toolkit/classification/generate-visualization.py
2020-11-09 17:53:36,540 [MainThread  ] [DEBUG]  AMD Data Analysis Toolkit - Classification
Traceback (most recent call last):
  File "toolkit/amd_data_analysis_toolkit/classification/generate-visualization.py", line 70, in <module>
    main()
  File "toolkit/amd_data_analysis_toolkit/classification/generate-visualization.py", line 37, in main
    parser = argparse.ArgumentParser()

@bijayraj running `python toolkit/amd_data_analysis_toolkit/classification/generate-visualization.py' script should open the GUI if no arguments are passed. Check out this app - https://github.com/kiritigowda/mivisionx-inference-analyzer for reference.
python toolkit/amd_data_analysis_toolkit/classification/generate-visualization.py
2020-11-09 17:58:05,712 [MainThread  ] [DEBUG]  AMD Data Analysis Toolkit - Classification
Traceback (most recent call last):
  File "toolkit/amd_data_analysis_toolkit/classification/generate-visualization.py", line 71, in <module>
    main()
  File "toolkit/amd_data_analysis_toolkit/classification/generate-visualization.py", line 67, in main
    generateAnalysisOutput(argsDict)
  File "/Users/kiriti/work/adat/MIVisionX/toolkit/amd_data_analysis_toolkit/classification/adat_classification.py", line 1058, in generateAnalysisOutput
    validateConfig(argsDict)
  File "/Users/kiriti/work/adat/MIVisionX/toolkit/amd_data_analysis_toolkit/classification/adat_classification.py", line 1037, in validateConfig
    raise Exception(errorMessage)
Exception: Following required arguments were missing:
--inference_results
--label
--output_dir
--output_name

@bijayraj any updates on these?

@bijayraj any updates on these?

@kiritigowda recent commits were made to fix the issues.

@bijayraj - the following error when the new feature compare is used
python generate-visualization.py
Traceback (most recent call last):
  File "/home/kiriti/adat/MIVisionX/toolkit/amd_data_analysis_toolkit/classification/adat_gui.py", line 161, in runGenerate
    ac.generateAnalysisOutput(self.configDict)
  File "/home/kiriti/adat/MIVisionX/toolkit/amd_data_analysis_toolkit/classification/adat_classification.py", line 1086, in generateAnalysisOutput
    diffPatch = getDifferencePatch(inputCSVFile, compareFile, ['FileName'])
  File "/home/kiriti/adat/MIVisionX/toolkit/amd_data_analysis_toolkit/classification/adat_classification.py", line 1146, in getDifferencePatch
    patch = csvdiff.diff_files(csv1, csv2, id_field)
  File "/home/kiriti/.local/lib/python2.7/site-packages/csvdiff/__init__.py", line 44, in diff_files
    ignore_columns=ignored_columns)
  File "/home/kiriti/.local/lib/python2.7/site-packages/csvdiff/patch.py", line 205, in create
    to_indexed = records.index(to_records, index_columns)
  File "/home/kiriti/.local/lib/python2.7/site-packages/csvdiff/records.py", line 58, in index
    raise InvalidKeyError('invalid column name {k} as key'.format(k=k))
csvdiff.records.InvalidKeyError: invalid column name 'FileName' as key

@kiritigowda New commit has fixed the problem. The problem was with incompatible compare file chosen. Extra file format checking has been added to avoid the error.


Wagtail allows truncated images to be uploaded (which crash whenever displayed)

Steps to reproduce:

Find a valid jpeg file
Truncate it a bit
Upload it to wagtail (should work if the headers are still intact)
Find a view that renders it (will crash with TruncatedImageException)

I I had this problem also with the version 1.9.5

Images can be deleted via the Django Admin Interface. As workaround...

Closing in favour of #1354 which looks like the exact same issue (assuming we’d handle truncated images and other broken images in the same way)


minor UX changes

Codecov Report

Merging #6 into master will decrease coverage by 0.15%.
The diff coverage is n/a.

@@            Coverage Diff            @@
##           master      #6      +/-   ##
=========================================
- Coverage       9%   8.85%   -0.16%
=========================================
  Files           7       7
  Lines         522     531       +9
=========================================
  Hits           47      47
- Misses        472     481       +9
  Partials        3       3

Impacted Files
Coverage Δ

pkg/stub/psmdb.go
17.03% <0%> (-1.22%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b8d3efb...f13fd0f. Read the comment docs.


Physics Interpolation - fix stale interpolation data when unhiding

A previous optimization prevented instances being added to the interpolation lists when hidden to save processing. This caused a regression when unhiding nodes outside of the physics tick - the interpolated transforms would be stale until the next physics tick, causing a glitch.
This PR readds instances immediately to the interpolation lists when they are unhidden, preventing this glitch.
Fixes #62019
Notes

Discovered during testing of beat invaders, a regression between 3.5 beta 5 and RC1.
Caused by #60915

Thanks!


Add nice error message when attempting to import removed modules/functions in v2

If a user attempts to import removed functions, from removed modules, or both, we should give a better error message than a plain ImportError. Ideally the error message will include some information about the migration path, or possibly a request to create an issue if the changes break something in their codebase in a way that is difficult for them to resolve.
As an example, it would be preferable to get such an error message when attempting to run from pydantic.generics import replace_types.
Probably anything that could be construed as part of the "public" api of v1.X (i.e., doesn't use leading underscores in the name or import path) should be handled neatly.

Keen to work on this! Will require a fair but of discussion on how we wish to inform the user, how long this will be persisted for etc.
I have an initial idea (see PR https://github.com/pydantic/pydantic/pull/5113)
Using this PEP-562 standard, we could add a __getattr__ method to all files (and the old file names - noting this will be all that is in them)
Let me know your thoughts :)

We could also monkeypatch internals, this seems like a nice way to do it, but potentially risky...
Traceback (most recent call last):
  File "/projects/pydantic/caller.py", line 1, in <module>
    from pydantic import Required
  File "/projects/pydantic/import_helper.py", line 30, in _handle_fromlist_override
    raise exception
pydantic.import_helper.V2MigrationRemoved: pydantic.Required has been removed in favour of ...

@samuelcolvin @dmontagu thoughts?


warn: Login err: {} on Companion RaspberryPi

Describe the bug
Adding Client ID and Client secret (followed the tutorial from the module info)
To Reproduce
Steps to reproduce the behaviour:

Configure the app with the credentials, save, and check logs

Expected behaviour
Should authenticate
Screenshots

System (please complete the following information):

Companion Version 3.0.0 (3.0.0+6000-beta-52f8eec5)

Additional context
I even added the actual RPi IP to redirects (not sure is needed)

I don't know of any users running Discord on a Raspberry Pi so I haven't been able to test this situation, but just to check, is whatever OS you're running on your Raspberry Pi one that is officially supported by Discord?

Discord isn't the main use for the companion PI, mostly is useful for HomeAssistant stuff, to have it always available event with PC turned off, but I like to use other modules on this stream deck. I'm using the image from https://user.bitfocus.io/product/companion-pi so not sure what OS is used.

Discord can only be connected to over IPC, meaning it has to be running on the same machine as Companion. There's no capability for Companion to be running on one machine/device, and Discord running on a completely separate device.
It is planned for a future release to be able to download and run a proxy service on the same machine as Discord, and then Companion can connect over the network to that proxy, but that is not yet ready. So for your situation where you're running Companion on a Raspberry Pi, unless you also install and use Discord on that same Raspberry Pi, this module isn't currently supported.


Review 4: TestimonyList.jsx and NavBar.jsx

Review Branch
review-4
E10
Files to review

TestimonyList.jsx
NavBar.jsx

Checklists

Architecture
Design
JS
React
Testing

Due date
9/29/2022
For more information
The review process is documented at: http://courses.ics.hawaii.edu/ics414s21/morea/review/reading-idpm-review.html

Testimony.jsx

L9 Should include a component ID for testcafe to use

NavBar.jsx

Looks good

Testimony.jsx

Looks Good

NavBar.jsx

Looks Good

Testimony.jsx

looks good to me

NavBar.jsx

looks good to me

TestimonyList.jsx

Looks good

NavBar.jsx

Looks good

Both TestimonyList.jsx and NavBar.jsx looks good with the current implementation.


file name and line number for error logging

We're using some code similar to the AMPMClient.cpp to send error messages to AMPM, however it seems like the file name and line number don't show up in the logs. Is this by design, or could it be made optional?
For now I'm just prepending the location info the the message string that send across, but wondering if I'm not meant to do that.

ampm won't know anything about your source code, so yes, you'd need to send it things like line numbers and file names. If AMPMClient.cpp could be changed such that it had an error method which automatically collected that info and sent it on to ampm, that would be most welcome.
The JavaScript client does send the stack trace and line numbers on errors, maybe you could do similar for c++.
https://github.com/stimulant/ampm/blob/master/samples/web/libs/ampm.js#L134

The current AMPMClient.cpp sends through line and lineNum to the server, but these are ignored by the server?
I'm happy just prepending the location data to the message, but looking at AMPMClient.cpp made me think this wasn't expected.

I looked into this. The log object is making it through to _logMessage here, but only the .message attribute ends up in the log file.
https://github.com/stimulant/ampm/blob/master/model/logging.js#L215
AMPMClient.cpp probably shouldn't construct a message object like that, because you'd reasonably assume something is happening with the other attributes... but it's not. A great PR would be to change it to examine the level to see if it's error, and if so, construct a string with all the info you want in it, which is what the JavaScript client does.

Okay, that makes sense to me. The file and line number already have to come from the client app, so we might as well bake it into the message on the client side (which is what I ended up doing).


Pseudo-code for Crime Analysis

from arcgis.hub import *

#Search for DC's Hub --> returns object with 2 orgs (enterprise, community)
myhub = arcgis.hub.Hub.search("DC")

#Connect to enterprise org
dc_eOrg = myHub.orgs[0]
gis = GIS(dc_eOrg.url, username, password)

#Fetch crime indicator
dc_crime_url = dc_eOrg.indicators["crime"]

#Perform analysis, and save maps in your org
hotspot_crime = arcgis.analysis.hotspot(with parameters)
result_map = webmap.add(hotspt_crime)

#Add 'result_map' Item to 'City Safety' initiative
safety_eOrg = dc_eOrg.Initiatives.search("City Safety")
mymap = safety_eOrg.add(result_map, name='Crime Cluster Map', type='webmap', tags=['dc crime', 'safety'])

#Make it public to add to community org
mymap.share(everyone=True)
mymap_id = mymap.item

#Add it to community org
dc_cOrg = myHub.orgs[1]
gis = GIS(dc_cOrg.url, username, password)

#Search for mymap and add to 'City Safety' initiative
mymap_item = gis.content.search(mymap_id)
safety_cOrg = dc_cOrg.Initiatives.search("City Safety")
c_mymap = safety_cOrg.add(mymap_item, name='Crime Cluster Map', type='webmap', tags=['dc crime', 'safety'])
c_mymap.share(organization=True)

#For charts, do analysis, but don't visualize here
bar_chart_inputs = {"murder":540, "theft":600, ..}
pie_chart_inputs = {"weekday":[40%, 80], "weekend":[60%, 120]}

#Add to file
file = open("dc_crime_stats.txt", "a")
file.write(bar_chart_inputs)
file.write(pie_chart_inputs)

#Share file on a web server for access to the ArcGIS JS-API for creating visualizations

where did webmap come from for result_map?
what is the concept of a "webmap" as an indicator?
why do you have to add it to the COmmunity org?


Tooltips not working in command inspector

Hover over a command property in Unity 5.3 and the tooltip does not display (reported by Llaura)

This appears to be working again in Unity 5.3.4.


XGBoost4J-Spark fails on EMR

Hello everybody,
I am trying to implement XGBoost4J-Spark in a scala project. Everything works fine locally (on an intel MacBook), however when deploying to EMR, I receive the following error (running on EMR 6.12.0 and Spark 3.4.0 with Scala 2.12.17):
java.lang.NoClassDefFoundError: Could not initialize class ml.dmlc.xgboost4j.java.XGBoostJNI

For my build.sbt I added the following lines to libraryDependencies, as suggested by the tutorial (running with sbt 1.9.2):
"ml.dmlc" %% "xgboost4j" % "1.7.6",
"ml.dmlc" %% "xgboost4j-spark" % "1.7.6"

I packaged everything up into a single JAR via the sbt-assembly plugin. I believe that this would pack all the dependencies into the JAR that is needed to run the Spark Application on EMR, so I am really out of ideas about this error. Not sure if this is an error on my end or an actual bug. Help is appreciated!

Can you install XGBoost4J-Spark from Maven Central? Locally building JARs is more complex, as you might have issues with bundling the native library (libxgboost4j.so).

I assume you mean to build the packages from source?
I tried that on the main of the EMR node, but ran into errors. I ran the following steps:
Installing maven:
wget https://dlcdn.apache.org/maven/maven-3/3.9.4/binaries/apache-maven-3.9.4-bin.tar.gz
tar xzvf apache-maven-3.9.4-bin.tar.gz
PATH=$PATH:/home/hadoop/apache-maven-3.9.4/bin

Cloning Repo and switching to 1.7.6 and then packaging it up (following steps in tutorial):
git clone --recursive https://github.com/dmlc/xgboost

cd xgboost
git checkout 36eb41c
cd jvm-packages
mvn package

This ended up in the following error:
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for XGBoost JVM Package 1.7.6:
[INFO]
[INFO] XGBoost JVM Package ................................ SUCCESS [  2.298 s]
[INFO] xgboost4j_2.12 ..................................... FAILURE [  1.973 s]
[INFO] xgboost4j-spark_2.12 ............................... SKIPPED
[INFO] xgboost4j-flink_2.12 ............................... SKIPPED
[INFO] xgboost4j-example_2.12 ............................. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  4.651 s
[INFO] Finished at: 2023-08-22T23:09:24Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.6.0:exec (native) on project xgboost4j_2.12: Command execution failed.: Process exited with an error: 1 (Exit value: 1) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <args> -rf :xgboost4j_2.12

Similarly I tried to install it via running ./xgboost/jvm-packages/dev/build-linux.sh as suggested by the README in jvm-packages. This too fails somewhere down the line with:
docker: Error response from daemon: pull access denied for dmlc/xgboost4j-build, repository does not exist or may require 'docker login'

I feel like I am well of the beaten path here and probably miss something quite obvious...

Do you have a working Python 3 installation?
I didn't realize you have to build from the source when using EMR. Do you need an uber-JAR where all dependencies are included? I found it hard to build such a JAR.

Yes I do have a python3 installation, but it seems like this error is caused by a python2 invocation on the "create_jni.py" file. Invoking python from the command line opens python 3.7.16 shell. I am not sure how and why python2 is invoked.
I just want something that reliably works in production.

@nawidsayed, I guess you can hack the python path from here https://github.com/dmlc/xgboost/blob/master/jvm-packages/xgboost4j/pom.xml#L88

Thanks for your help so far everybody! I noticed that I am running on EMR Graviton 2 processors (r6gd instance) which are ARM based and I believe that might not be well supported by XGBoost4J. Switched to r5d instances and the library now seems to be present. However I am still encountering a very generic error (from the master node):
23/08/23 08:55:07 WARN XGBoostSpark: train_test_ratio is deprecated since XGBoost 0.82, we recommend to explicitly pass a training and multiple evaluation datasets by passing 'eval_sets' and 'eval_set_names'
23/08/23 08:55:07 ERROR RabitTracker$TrackerProcessLogger: Tracker Process ends with exit code 1
23/08/23 08:55:08 ERROR XGBoostSpark: the job was aborted due to
ml.dmlc.xgboost4j.java.XGBoostError: XGBoostModel training failed.
	at ml.dmlc.xgboost4j.scala.spark.XGBoost$.trainDistributed(XGBoost.scala:418) ~[test.jar:1.0.0]
	at ml.dmlc.xgboost4j.scala.spark.XGBoostClassifier.train(XGBoostClassifier.scala:202) ~[test.jar:1.0.0]
	at ml.dmlc.xgboost4j.scala.spark.XGBoostClassifier.train(XGBoostClassifier.scala:34) ~[test.jar:1.0.0]
	at org.apache.spark.ml.Predictor.fit(Predictor.scala:114) ~[spark-mllib_2.12-3.4.0-amzn-0.jar:3.4.0-amzn-0]
	at com.truthset.jobs.PrototypeSparkJob$.run(PrototypeSparkJob.scala:65) ~[test.jar:1.0.0]
	at com.truthset.jobs.PrototypeSparkJob$.run(PrototypeSparkJob.scala:18) ~[test.jar:1.0.0]
	at com.truthset.jobs.SparkJobWithJson.main(SparkJobWithJson.scala:34) ~[test.jar:1.0.0]
	at com.truthset.jobs.PrototypeSparkJob.main(PrototypeSparkJob.scala) ~[test.jar:1.0.0]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_382]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_382]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_382]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_382]
	at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52) ~[spark-core_2.12-3.4.0-amzn-0.jar:3.4.0-amzn-0]
	at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:1066) ~[spark-core_2.12-3.4.0-amzn-0.jar:3.4.0-amzn-0]
	at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:192) ~[spark-core_2.12-3.4.0-amzn-0.jar:3.4.0-amzn-0]
	at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:215) ~[spark-core_2.12-3.4.0-amzn-0.jar:3.4.0-amzn-0]
	at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:91) ~[spark-core_2.12-3.4.0-amzn-0.jar:3.4.0-amzn-0]
	at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1158) ~[spark-core_2.12-3.4.0-amzn-0.jar:3.4.0-amzn-0]
	at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1167) ~[spark-core_2.12-3.4.0-amzn-0.jar:3.4.0-amzn-0]
	at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) ~[spark-core_2.12-3.4.0-amzn-0.jar:3.4.0-amzn-0]
Exception in thread "main" ml.dmlc.xgboost4j.java.XGBoostError: XGBoostModel training failed.
	at ml.dmlc.xgboost4j.scala.spark.XGBoost$.trainDistributed(XGBoost.scala:418)
	at ml.dmlc.xgboost4j.scala.spark.XGBoostClassifier.train(XGBoostClassifier.scala:202)
	at ml.dmlc.xgboost4j.scala.spark.XGBoostClassifier.train(XGBoostClassifier.scala:34)
	at org.apache.spark.ml.Predictor.fit(Predictor.scala:114)
	at com.truthset.jobs.PrototypeSparkJob$.run(PrototypeSparkJob.scala:65)
	at com.truthset.jobs.PrototypeSparkJob$.run(PrototypeSparkJob.scala:18)
	at com.truthset.jobs.SparkJobWithJson.main(SparkJobWithJson.scala:34)
	at com.truthset.jobs.PrototypeSparkJob.main(PrototypeSparkJob.scala)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
	at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:1066)
	at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:192)
	at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:215)
	at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:91)
	at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1158)
	at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1167)
	at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Command exiting with ret '1'

I haven't found a remedy for this. Almost all references for this error are usually caused when using the GPU implementation. This however is not the case for me here. It's confusing because if I check stderr on the executor, it's clear that the training is actually happening and there is no indication of an error there.
[09:34:54] [0]	train-mlogloss:0.98398036223191476
[09:34:54] [0]	train-mlogloss:0.97309246502424540

[09:34:54] [1]	train-mlogloss:0.88586563941759944

[09:34:54] [1]	train-mlogloss:0.86604528834945282

[09:34:54] [2]	train-mlogloss:0.80109514334262943

[09:34:54] [2]	train-mlogloss:0.77383518846411459

[09:34:54] [3]	train-mlogloss:0.72730388396825540

[09:34:54] [4]	train-mlogloss:0.66267788104521919

[09:34:54] [3]	train-mlogloss:0.69377712826979787

[09:34:54] [5]	train-mlogloss:0.60579290756812465

[09:34:54] [4]	train-mlogloss:0.62382606613008595

[09:34:54] [6]	train-mlogloss:0.55597261434946310
....

23/08/23 09:34:56 INFO Executor: 1 block locks were not released by task 0.0 in stage 4.0 (TID 4)
[rdd_25_0]
23/08/23 09:34:56 INFO MemoryStore: Block taskresult_4 stored as bytes in memory (estimated size 6.1 MiB, free 6.1 GiB)
23/08/23 09:34:56 INFO Executor: Finished task 0.0 in stage 4.0 (TID 4). 6442557 bytes result sent via BlockManager)
23/08/23 09:34:56 INFO YarnCoarseGrainedExecutorBackend: Driver commanded a shutdown
23/08/23 09:34:56 INFO MemoryStore: MemoryStore cleared
23/08/23 09:34:56 INFO BlockManager: BlockManager stopped
23/08/23 09:34:56 INFO ShutdownHookManager: Shutdown hook called
23/08/23 09:34:56 INFO ShutdownHookManager: Deleting directory /mnt/yarn/usercache/hadoop/appcache/application_1692778153848_0006/spark-d1d9df8a-dde2-47ce-8ca7-fc09fde80055

So it seems like it's related to Spark & XGBoost versioning. Using Spark 3.4.0 on Scala 2.12 and XGBoost packages versions 1.7.6 I get the aforementioned error which is probably related to the Rabbit Tracker. StdOut prints Tracker started, with env={} just before erroring out.
However I don't have any issues when running with Spark 2.4.8 on Scala 2.11 and using XGBoost4j and XGBoost4j-spark with version 1.1.2 . In that case just before the training routine, stdOut read: Tracker started, with env={DMLC_NUM_SERVER=0, DMLC_TRACKER_URI=<IP_ADDRESS>, DMLC_TRACKER_PORT=9091, DMLC_NUM_WORKER=36}.

Is there any way to make it properly work for Spark 3.4 ?

XGBoost 1.7.6 supports Spark 3.0.1:
https://github.com/dmlc/xgboost/blob/36eb41c960483c8b52b44082663c99e6a0de440a/jvm-packages/pom.xml#L37
You can use XGBoost 2.0.0 to use Spark 3.4.0:
https://github.com/dmlc/xgboost/blob/4301558a5711e63bbf004d2b6fca003906fb743c/jvm-packages/pom.xml#L38

Thanks for pointing this out. Unfortunately adding the library according to instructions here, fails in the following way:
[error] (update) java.net.URISyntaxException: Illegal character in path at index 106: https://s3-us-west-2.amazonaws.com/xgboost-maven-repo/release/ml/dmlc/xgboost4j_2.12/2.0.0-RC1/xgboost4j_${scala.binary.version}-2.0.0-RC1.jar

Even when manually adding the 2.0.0-RC1 packages to the jar, we run into the Rabbit Tracker error:
Tracker started, with env={}
23/08/23 16:53:11 ERROR RabitTracker$TrackerProcessLogger: Tracker Process ends with exit code 1
23/08/23 16:53:12 ERROR XGBoostSpark: the job was aborted due to
ml.dmlc.xgboost4j.java.XGBoostError: XGBoostModel training failed

I think we should prioritize the refactoring of the tracker, otherwise JVM related issues are quite difficult to resolve

is it possible the tracker is also running with python 2?

I bumped into the exact same generic error reported by the OP, using a very similar setup (EMR 6.5.0, Spark 3.1.2). Even if I am using Scala Spark, there is  a python through RabitTracker, which requires python >= 3.8. But EMR 6.5.0 provides python3.7. Setting up a virtual environtment that allows the cluster to use a higher python version solved the problem for me.

Coming back again, after the solution I suggested in my post in Nov 28 didn't seem to work out on a second attempt. For me it was important to activate the virtual environment with the right python version before starting my spark-shell session in the master node.

That's caused by the Python dependency. We have removed the use of Python in the master branch.

Thanks @trivialfis. I am bound to use version 1.7.3, but it's great to hear the python dependency has been removed in recent versions. It was really a pain to deal with.


Enable .NET

Update to the latest TFBridge and generate the .NET SDK

Going to merge this and fix it up for the python generation!


Add Error Handling to 'Collectors.Feed.LoadAllPost'

Closed with https://github.com/jguadagno/jjgnet-broadcast/commit/f218b3e0a286f835b307a02b0a4b9509c6ed6ca2


How to generate our own datasets for training and inference stage?

Hi! That was a great work proposed by your team!
I am wondering how I am supposed to generate our own dataset from two individual images to a rectangle stitched image with M_s?

Hi,
for the image stiching step, you could try [UDIS2](https://github.com/nie-lang/UDIS2), I think you could as well obtain a corresponding M_s. BTW, if you have an already stiching image without M_s, you could consider using this function  define_mask_zero_borders to generate one

hi,
you could refer to Deep Rectangling for Image Stitching: A Learning Baseline for more detail about creating dataset.


Feature Request: Remote Indexing

Having all the Markdown files be available locally isn't always going to be feasible. Being able to index files on some remote, either via a URL, or ideally via an S3-compatible API would be highly beneficial.

@jacobranson great suggestion. This would be very easy to implement here. I think it could start as an example.
My suggestion is you'd start with a copy paste of processFile called e.g. processRemoteFile https://github.com/datopian/markdowndb/blob/main/src/lib/process.ts#L17
If you want an entire folder in e.g. s3 or a github repo then work on indexFolder called e.g. indexRemoteFolder` https://github.com/datopian/markdowndb/blob/main/src/lib/indexFolder.ts#L7


(M) As a developer I would like a unit test suite for the current securitas codebase

Proof of concept tests
Acceptance Criteria:
Creation of integration & unit tests
Coverage
DOD:
Tests to be completed identified and documented
Reviewed by team/sign off by team

As an AAA developer I want clear descriptions of my issues.

any thoughts on how to approach unit tests for the web front end here? since we kind of require an IPA server for it to work properly.

I see two options:

We mock the client that python-freeipa gives us, and in each test we use return_value and side_effect to make it return what we want to test. If we choose that we'll probably end up building a mocking client class
We use python-vcr to record and replay the HTTP requests. That means that we need to have a working freeipa server when the unit test is first written, but I think that's a good thing because with mocking there's always the risk that the return value you set is not something that can actually happen and then you end up testing against a false service and your test is useless.

Mocking the client is easier to start with but I think VCR can be easier in the long run.

Thanks @abompard!
python-vcr seems like it might be the way to go here. However, still kind of blocked on having a running free-ipa instance i can point to to start writing these tests.

@ryanlerch I managed to get freeipa running in a container. I'm still a little bit off having a full local dev environment but if all you need is a freeipa server running locally then the following should get you there. The steps are:
clone this repo: https://github.com/freeipa/freeipa-container
from within run podman build -t freeipa-server
run the following: setsebool -P container_manage_cgroup 1
run sudo mkdir /var/lib/ipa-data
to bring up the container run
podman run --name freeipa -ti -p 80:80 -p 443:443 \
   -h ipa.example.test \
   -v /sys/fs/cgroup:/sys/fs/cgroup:ro \
   --tmpfs /run --tmpfs /tmp \
   -v /var/lib/ipa-data:/data:Z freeipa-server

configure the above with the defaults it suggests except for the ca cert suggestion where you should type yes
add the following to your /etc/hosts file <IP_ADDRESS> ipa.example.test
sorry its a mess i typed it in a rush before finishing I hope it works for you. I'm hoping to have it in a compose file tomorrow so that should make things easier.