Installing lucent causes pytorch downgrade

I installed pytorch 1.7.0 on a fresh Python 3.8 conda environment. Then I installed lucent via pip, and it uninstalled pytorch 1.7 and installed 1.6. That's because kornia 0.4.0 specifically wants pytorch < 1.7. Can you upgrade to kornia 0.4.1 or 0.4.2? It's less picky about Pytorch versions.
(lucent) pmin@patrick-deep:/mnt/d/Documents/brain-scorer$ pip install torch-lucent
Collecting torch-lucent
  Using cached torch_lucent-0.1.4-py3-none-any.whl (45 kB)
Collecting pytest
  Using cached pytest-6.2.2-py3-none-any.whl (280 kB)
Collecting future
  Using cached future-0.18.2-py3-none-any.whl
Requirement already satisfied: ipython in /home/pmin/miniconda3/envs/lucent/lib/python3.8/site-packages (from torch-lucent) (7.20.0)
Requirement already satisfied: torchvision in /home/pmin/miniconda3/envs/lucent/lib/python3.8/site-packages (from torch-lucent) (0.8.2)
Collecting kornia==0.4.0
  Using cached kornia-0.4.0-py2.py3-none-any.whl (195 kB)
Requirement already satisfied: pillow in /home/pmin/miniconda3/envs/lucent/lib/python3.8/site-packages (from torch-lucent) (8.1.0)
Collecting pytest-mock
  Using cached pytest_mock-3.5.1-py3-none-any.whl (12 kB)
Requirement already satisfied: tqdm in /home/pmin/miniconda3/envs/lucent/lib/python3.8/site-packages (from torch-lucent) (4.57.0)
Collecting coverage
  Downloading coverage-5.4-cp38-cp38-manylinux2010_x86_64.whl (245 kB)
     |████████████████████████████████| 245 kB 672 kB/s
Requirement already satisfied: decorator in /home/pmin/miniconda3/envs/lucent/lib/python3.8/site-packages (from torch-lucent) (4.4.2)
Collecting coveralls
  Using cached coveralls-3.0.0-py2.py3-none-any.whl (13 kB)
Requirement already satisfied: scikit-learn in /home/pmin/miniconda3/envs/lucent/lib/python3.8/site-packages (from torch-lucent) (0.24.1)
Requirement already satisfied: numpy in /home/pmin/miniconda3/envs/lucent/lib/python3.8/site-packages (from torch-lucent) (1.20.1)
Requirement already satisfied: torch>=1.5.0 in /home/pmin/miniconda3/envs/lucent/lib/python3.8/site-packages (from torch-lucent) (1.7.1)
Collecting torch>=1.5.0
  Using cached torch-1.6.0-cp38-cp38-manylinux1_x86_64.whl (748.8 MB)
Collecting docopt>=0.6.1
  Using cached docopt-0.6.2-py2.py3-none-any.whl
Requirement already satisfied: requests>=1.0.0 in /home/pmin/miniconda3/envs/lucent/lib/python3.8/site-packages (from coveralls->torch-lucent) (2.25.1)
Requirement already satisfied: certifi>=2017.4.17 in /home/pmin/miniconda3/envs/lucent/lib/python3.8/site-packages (from requests>=1.0.0->coveralls->torch-lucent) (2020.12.5)
Requirement already satisfied: chardet<5,>=3.0.2 in /home/pmin/miniconda3/envs/lucent/lib/python3.8/site-packages (from requests>=1.0.0->coveralls->torch-lucent) (4.0.0)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/pmin/miniconda3/envs/lucent/lib/python3.8/site-packages (from requests>=1.0.0->coveralls->torch-lucent) (1.26.3)
Requirement already satisfied: idna<3,>=2.5 in /home/pmin/miniconda3/envs/lucent/lib/python3.8/site-packages (from requests>=1.0.0->coveralls->torch-lucent) (2.10)
Requirement already satisfied: traitlets>=4.2 in /home/pmin/miniconda3/envs/lucent/lib/python3.8/site-packages (from ipython->torch-lucent) (5.0.5)
Requirement already satisfied: pygments in /home/pmin/miniconda3/envs/lucent/lib/python3.8/site-packages (from ipython->torch-lucent) (2.8.0)
Requirement already satisfied: jedi>=0.16 in /home/pmin/miniconda3/envs/lucent/lib/python3.8/site-packages (from ipython->torch-lucent) (0.18.0)
Requirement already satisfied: pexpect>4.3 in /home/pmin/miniconda3/envs/lucent/lib/python3.8/site-packages (from ipython->torch-lucent) (4.8.0)
Requirement already satisfied: setuptools>=18.5 in /home/pmin/miniconda3/envs/lucent/lib/python3.8/site-packages (from ipython->torch-lucent) (52.0.0.post20210125)
Requirement already satisfied: backcall in /home/pmin/miniconda3/envs/lucent/lib/python3.8/site-packages (from ipython->torch-lucent) (0.2.0)
Requirement already satisfied: prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0 in /home/pmin/miniconda3/envs/lucent/lib/python3.8/site-packages (from ipython->torch-lucent) (3.0.16)
Requirement already satisfied: pickleshare in /home/pmin/miniconda3/envs/lucent/lib/python3.8/site-packages (from ipython->torch-lucent) (0.7.5)
Requirement already satisfied: parso<0.9.0,>=0.8.0 in /home/pmin/miniconda3/envs/lucent/lib/python3.8/site-packages (from jedi>=0.16->ipython->torch-lucent) (0.8.1)
Requirement already satisfied: ptyprocess>=0.5 in /home/pmin/miniconda3/envs/lucent/lib/python3.8/site-packages (from pexpect>4.3->ipython->torch-lucent) (0.7.0)
Requirement already satisfied: wcwidth in /home/pmin/miniconda3/envs/lucent/lib/python3.8/site-packages (from prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0->ipython->torch-lucent) (0.2.5)
Requirement already satisfied: ipython-genutils in /home/pmin/miniconda3/envs/lucent/lib/python3.8/site-packages (from traitlets>=4.2->ipython->torch-lucent) (0.2.0)
Collecting attrs>=19.2.0
  Using cached attrs-20.3.0-py2.py3-none-any.whl (49 kB)
Collecting py>=1.8.2
  Using cached py-1.10.0-py2.py3-none-any.whl (97 kB)
Collecting pluggy<1.0.0a1,>=0.12
  Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting iniconfig
  Using cached iniconfig-1.1.1-py2.py3-none-any.whl (5.0 kB)
Collecting toml
  Using cached toml-0.10.2-py2.py3-none-any.whl (16 kB)
Collecting packaging
  Downloading packaging-20.9-py2.py3-none-any.whl (40 kB)
     |████████████████████████████████| 40 kB 6.2 MB/s
Requirement already satisfied: pyparsing>=2.0.2 in /home/pmin/miniconda3/envs/lucent/lib/python3.8/site-packages (from packaging->pytest->torch-lucent) (2.4.7)
Requirement already satisfied: scipy>=0.19.1 in /home/pmin/miniconda3/envs/lucent/lib/python3.8/site-packages (from scikit-learn->torch-lucent) (1.6.1)
Requirement already satisfied: joblib>=0.11 in /home/pmin/miniconda3/envs/lucent/lib/python3.8/site-packages (from scikit-learn->torch-lucent) (1.0.1)
Requirement already satisfied: threadpoolctl>=2.0.0 in /home/pmin/miniconda3/envs/lucent/lib/python3.8/site-packages (from scikit-learn->torch-lucent) (2.1.0)
Installing collected packages: toml, py, pluggy, packaging, iniconfig, future, attrs, torch, pytest, docopt, coverage, pytest-mock, kornia, coveralls, torch-lucent
  Attempting uninstall: torch
    Found existing installation: torch 1.7.1
    Uninstalling torch-1.7.1:
      Successfully uninstalled torch-1.7.1
Successfully installed attrs-20.3.0 coverage-5.4 coveralls-3.0.0 docopt-0.6.2 future-0.18.2 iniconfig-1.1.1 kornia-0.4.0 packaging-20.9 pluggy-0.13.1 py-1.10.0 pytest-6.2.2 pytest-mock-3.5.1 toml-0.10.2 torch-1.6.0 torch-lucent-0.1.4

Thanks for raising this @patrickmineault! I've updated the dependency to kornia 0.4.1 instead.


Misspelt identifiers are never completed

If a completion list includes a word such as abcdefg, typing abdc will exclude that word despite the similarity.
We should fix the matcher so that transposed characters do not fully exclude the word.

We're closing old issues with no recent activity. If this is still important, please open a new issue.


BAU: Test-driven development

Content explaining what TDD is and why we advocate for it

I very much support adding TDD to The GDS Way! When I learned about TDD it was with a peer group who benefitted from a lot of time and support from our managers, which included regular, safe opportunities to practise our TDD skills with expert advice. I wonder if guidance is needed for those who are new to TDD and/or haven't had any training or coaching in it?

I very much support adding TDD to The GDS Way! When I learned about TDD it was with a peer group who benefitted from a lot of time and support from our managers, which included regular, safe opportunities to practise our TDD skills with expert advice. I wonder if guidance is needed for those who are new to TDD and/or haven't had any training or coaching in it?

I completely agree - I've added some sections about "Getting started" and "Getting better" but I had a similar experience - pairing/mobbing and TDDing every day, which turbocharged my learning.
I'd like us to do more training / support for people to get the same experience I did (which I'm also happy to do).

Content looks great,
We might consider adding some links to some more pratical / worked examples? I feel like folks can struggle translating the theory into that early practice of "simple test" -> "oh i didn't import from my lib file", "oh i didn't write a class", "oh i didn't initalise the class with the required parameter", "oh there's no method called sum" to get to that first green?
This page makes sense as a set of principles, but wonder if we can endorse a good worked example at the bottom that would help folks see the detail and build that muscle memory?
Can very much be added later. Also a few nitpicks on Markdown formatting.

Agree, I've linked two very good books (GOOS and By Example) but also Jason Gorman's page which has (a) his TDD book with many worked examples and (b) his 101 TDD Tips which is full of bitesize ... tips to improve your TDD.

I support this approach from a quality engineering and testing perspective. The content looks really good. To achieve good quality tests that drive development, it's crucial to create a testing mindset and culture. Without that TDD may fail to produce intended outcomes. Completely agree with principles and the value TDD brings, but it is important that teams don't misunderstand it as one-stop solution for all testing needs. Hence I think it’s worth also define what TDD is not, to clarify its boundaries or distinguish it from other types of testing in the software development lifecycle. A few suggestions:

TDD is not a replacement for all test levels: TDD focuses on unit testing level and component integration. It doesn’t replace other levels of testing, such as functional integration testing, system testing, or feature acceptance testing. But good quality at unit testing reduces quality risk and extensive testing in higher levels
TDD is not a replacement for exploratory testing: Exploratory testing is used to examine the application to find issues that automated tests may not capture. Exploratory testing uncovers unexpected behaviours outside the scope of TDD. It is a specialised task to prevent unknown - knowns (unaware of behaviour, but can exist in real usage) or unknown-unknowns (latent issues within our systems—problems we didn't even know could be problems or manifest when conditions line up just the right way) in the system.
TDD is not a Catch-All for bugs: TDD helps prevent bugs in isolated units of code but does not catch all types of issues, especially those arising from system interactions or complex edge cases.
TDD is more technically focused: Tests designed may miss view on business logic scenarios or user focused tests (else I think we need to experiment and evidence how module or class focused tests can have wider view on business and user needs).

Thanks @mboban2024! I will fold your suggestions into the content


add a DUMP_BUFFER command

Buffers can already be compared in a single run using the ASSERT_EQUAL command but it does not permit to save buffer results to compare them across different runs.
While the binary representation could already be useful, a human-readable version would be best for debugging. As the type of the underlying data is bound at buffer creation (and by the underlying interface block in the shader), it could use the format provided at creation to recognize uint, int, etc.. when dumping to the file.

Good idea to add this command.
I would prefer not to utilise the format that was specified at buffer creation time. It does seem tempting, bu @paulthomson and I feel quite strongly that buffers should be trated as untyped sequences of bytes, and that using types at initialization is just a means of conveniently specifying the initial sequence. I would prefer for DUMP_BUFFER to have syntax for explaining how the buffer should be laid out.
I suggest we have two commands:
// Dump "buffer" to "file", in binary form
DUMP_BUFFER_BINARY BUFFER buffer FILE file

and:
DUMP_BUFFER_TEXT BUFFER buffer FORMAT format1 format2 ... formatn

where each format is one of:

a quoted string
type count - i.e., a type followed by the number of things of that type, where type is one of float, int, uint or byte, as in CREATE_BUFFER

Quoted strings will be included in the text output directly. A type with a count will cause the buffer data to be interpreted as that number of things of the given type, which will be emitted separated by spaces.
So, for example, if you had a buffer that represented a struct:
struct S {
  float x;
  mat2x2 m;
  int A[8];
}

with no padding then you could do:
DUMP_BUFFER_TEXT BUFFER my_buf FORMAT float 5 " " int 8

to have the numbers associated with the struct fields printed in a space-separated way, or you could do:
DUMP_BUFFER_TEXT BUFFER my_buf FORMAT
  "x: " float 1
  "\nm: " float 4
  "\nA: " int 8

to have the struct printed with field names.
What do you think?
@asuonpaa I'd also value your opinion.

Sounds good to me. To support buffers with offset or padding something like skip BYTES might be useful too. For example if you had a vec3 padded to vec4 you could do "vec3: " float 3 skip 4.

Good idea.


Smaller update size of code & code-insiders

Presently, whenever the package needs to be updated, we need to fetch the whole new version , and install the new version over the existing version, and then the old package is deleted. Is it possible to fetch just the changes, saving internet bandwidth? Fetching almost daily updates for vscode-insiders expending ~50Mb of data is really inconvenient.

Unfortunately, while it's a good idea, it's not in our planning horizon.


Fix "browser-cookie-lite" broken export by replacing it with "cookie"

Instead of exporting function as default, browser-cookie-lite do
this.cookie = …

where this === module.exports
I've fixed it by replacing litejs/browser-cookie-lite with RFC 6266 capable and more popular implementation - jshttp/cookie
Also implemented .clear method and isolated local-storage-fallback cookies from regular cookies using prefix

Excellent, thank you for the PR!

Will publish 1.3 shortly


Enable FastFixtureTestCase on Django 1.8+

This changeset allows the FastFixtureTestCase to be used on Django 1.8+.
This is not a rewrite or a comprehensive set of tests for the test case class. It is however an incremental move forward testing a baseline, can FastFixtureTestCase be used in tests? The first commit adds the basic test - which will fail - and next three, specifically 8cb7e91daf3984e3c79f10544a5507930e01b942 apply the changeset to pass.
I'm sure the FastFixtureTestCase could use some additional work, certainly additional testing, getting it working seems like a good start.
Also solves the issue in #200 where no fixtures are specified and the test case fails (a necessary fix for the application I'm supporting).
Does not solve problems specifically for #162 or #208 as those issues reference now unsupported versions of Django. I'd reckon those issues could probably be closed since they're out of date now.

any issues with this PR?

Only that I avoid FastFixtureTestCase because it is buggy, and so haven't tested the patch. Have you tested for your use cases?

currently attempting to implement the changes. does this PR need test coverage? what will make you confident?

I'd like someone that actually uses FastFixtureTestCase in their project to confirm that it works with their projects' tests.  I don't think there's any change that would tempt me to use them in my own code.

I've been using this patch with a project running Django 1.8.12 since creating the pull request. Would it be worthwhile adding warnings for the TestCase's execution, i.e. that the FastFixtureTestCase is largely unsupported or is untested legacy code?
I totally understand why you're not eager to include this. In my case the project using this feature has a legacy test suite that would take a significant effort to rewrite, relies on large fixture files, and originally took over 90 minutes to run in full (now under 60 seconds after trebling the number of tests).

@bennylope, I'm glad it works for you. I need a second project to test it out to confirm that the fixes are at least applicable to one other codebase.

Hi @bennylope ,
I have a question with this PR. Does removing transaction completely solves the problem ??? I think that is for setting up database right??/


Refs not forwarding

According to the README refs should be forwarded.
Here's a codesandbox example. You'll need to open view the console to see the following error.
sandbox.c7332c5c.js:1 Warning: Stateless function components cannot be given refs. Attempts to access this ref will fail.

Check the render method of `Jss(Component)`.
    in Unknown (created by Jss(Component))
    in Jss(Component) (created by InnerReference)
    in InnerReference
    in Reference (created by App)
    in Manager (created by App)
    in div (created by App)
    in App (created by Jss(App))
    in Jss(App)

The example provided implies it forwards to functional components so I expected this to work. I plan to update it to class component to see if that fixes the issue.
Also to note I tried innerRef and ref.
Versions:
react: 16.3.2
react-jss: 8.6.1

Tried this with class and React.forwardRef. No dice.
https://codesandbox.io/s/o54lj508r5

We had to take out the React.forwardRef because it hat problems with react-redux connect function.
The innerRef can only work if the inner component is a class component.
https://codesandbox.io/s/v0xy2656wy

So you want to have a reference to the button component and not your actual 'component'. Then yeah, you need some other name.


Extending Rebass Button component with styled-components renders , not

When extending the Button using the styled-components method, a
 is rendered and not the expected <button>. It seems like my extended component is in fact extending the <Box> component and not the  component since the default <Button> props from Rebass is also not applied to the extended <Button>.
Example Button extension:
import { Button as RebassButton } from "rebass";

export const Button = styled(RebassButton)`
  cursor: pointer;
`;

Button.defaultProps = {
  px: "10px"
};

Rendering components using Storybook.js like this:
import React from "react";
import { storiesOf } from "@storybook/react";
import { Button } from "./Button";
import { Button as RebassButton } from "rebass";

storiesOf("Components/Buttons", module).add("Button", () => (

    <Button>Test Button</Button>
    <RebassButton>RebassButton</RebassButton>

));

Renders the following HTML:
Expectation is obviously that the HTML and styling should be same, except for the pointer: 10px  and cursor: pointer (which is successfully applied to the extended <Button> component).

Test Button

<button font-size="inherit" font-weight="bold" color="white" class="sc-bdVaJa sc-EHOje fCPMYz">RebassButton</button>

Example screenshot of result:

Details:
"styled-components": "^4.1.3",
"styled-system": "^4.0.1",
"babel-plugin-styled-components": "^1.10.0",

Using the same method when extending my own custom components with styled-components in a similar way as above does not result in the same issue.

Fix: I never actually created the component... Pasting this here for future reference.
Complete component should be:
const Button = styled(RebassButton)`
  cursor: pointer;
`;

export const Button = ({ children, ...props }) => (

);

instead of only this:
export const Button = styled(RebassButton)`
  cursor: pointer;
`;

I'm having the same issue that @iaanvn initially reported. I'm trying to extend the rebass <Button> component and add some defaultProps. As soon as I add defaultProps, the button gets converted to a
.
I tried the suggested fix, but that didn't work for me. I've created a [codesandbox] based on the example from this issue (https://codesandbox.io/embed/m5974q90rj).

I just ran into this and I think the solution is to include the default props from RebassButton e.g.,
export const Button = styled(RebassButton)`
  cursor: pointer;
`;
Button.defaultProps = { ...RebassButton.defaultProps, yourProp: 'here' }


[🐞] By placing a component inside the layout.tsx file after the slot, visible via a signal value a rerender of the entire slot takes place

Which component is affected?
Qwik Runtime
Describe the bug
By placing a component inside the layout.tsx file after the slot, visible via a signal value a rerender of the entire slot takes place
You can play it back by clicking on the switch at the top right of the header. Once the kitten appears click again to make it disappear and you will notice that the get resume button, in the hero section, will do the entry animation, this means that the component has been reloaded.
layout.tsx code below:
export default component$(() => {
  const show = useSignal(false);

  return (
    <>
      <Header show={show} />
      <main>
        {show.value && <Cat />}
        {/* If I move this after the Slot, on value change slot reload */}
        {/* {show.value && <CatWalk />} */}
        <Slot />
        {show.value && <CatWalk />}
      </main>
      <Footer />
    </>
  );
});

if you move {show.value && <CatWalk />} before the slot, it don't reload
The full repo: https://github.com/VarPDev/pako
Reproduction
https://pako-git-ssr-varpdev.vercel.app/?_vercel_share=DU0WyukXXApiQswlWQPBtLb2pXz3168A
Steps to reproduce
No response
System Info
System:
    OS: macOS 13.1
    CPU: (8) arm64 Apple M1
    Memory: 62.70 MB / 8.00 GB
    Shell: 5.8.1 - /bin/zsh
  Binaries:
    Node: 21.1.0 - ~/.nvm/versions/node/v21.1.0/bin/node
    Yarn: 1.22.19 - ~/.nvm/versions/node/v21.1.0/bin/yarn
    npm: 10.2.0 - ~/.nvm/versions/node/v21.1.0/bin/npm
  Browsers:
    Chrome: 119.0.6045.123
    Safari: 16.2
  npmPackages:
    @builder.io/qwik: ^1.2.15 => 1.2.15
    @builder.io/qwik-city: ^1.2.15 => 1.2.15
    undici: ^5.26.0 => 5.27.0
    vite: ^4.4.11 => 4.5.0

Additional Information
No response

@VarPDev can you make a minimal repro in https://qwik.builder.io/playground?

@wmertens
I can't use the playground well. You can clone the repository I linked though, run "npm i" and "npm run dev". You can find the code on the "layout.tsx" page.

Yes is a cute cat ahah
The console.log, was not triggered again, but the html was rerender.
If you can add tailwind in playgroud, add this button
<Link href="/Pasquale_De_Lucia-Resume.pdf" target="_blank">
  <button class="btn btn-primary text-black">Get resume</button>
</Link>

in Hello component and when the cat disappears, you can see the button have a entry animation again

@VarPDev on my phone but you can add TW by including the CDN css link in the root.tsx tab

Ok here's a simpler playground with the problem. The animation on Hello should only happen on initial render, but instead it happens every time you click Toggle to make the before and after disappear.
https://qwik.builder.io/playground/#v=1.2.17&f=7VZbTsMwEPzvKUwkaCLlCX2gpqk4ABI%2FHKBp65SINK0SKlSq3J1Z28VxlSIhvpD4c%2BSM157dnR2jaEZhV9W4jArZ1UNYLuX0OtFvnafLAgVqvkuVS9i856dlviGhJQrkGknwVvtKLCYs9MfDOr6wGdUxC4JqX5bCSUB4SU4xEF8gwTzLoJ8mkgSKzjw%2FkrRxor65l0EHzH0INSJ4p8KBx9kv8qW34B85Os%2B%2FHbn%2B8M71xwM38gf3ILINzvKi8DbbFSLQXHpPq9Vvw%2FO0xi9N7%2BGVH7IK6NpACzbDayRtu0uX%2BRu9OdbDfMLqZVpwGzy4RIYTN%2Fg%2FCk1E1IGIXBY5MWsQem7W82N6IF1PumWibbz6yhQgFUD0nZ7aR11Q2hJdVjaNSpB5uTuk89SG62g7yYyO8SHGGLMJu9Jfzex5u14X%2FMuwnk6hAabb5NjC39yQtC04OABsNyOeFIhaQHdTFyzNYLXaqGmgI%2BmWowcaVD7BEFSkDceywXOE3cZS%2Bm3zVykEPyBdAMC5Vm4cKh17YqkSsqS%2BqHgNEvTdJFDBWvZVtjfF6LqB0OTWBeypLB1Fi3yRIGcaqB2HLvE%2FS%2F7eLPkE
When you "inspect element" on Hello and toggle the button, you'll see the before and after components appear and disappear, but you'll also see that the Hello div flashes as being re-rendered. The component itself doesn't re-render but the browser does re-render the contents. (In your example site you can see that then entire page re-renders like this).
This doesn't happen the element before or after is a string (try it by selecting the entire <p>...</p> and quoting it), only if they both are elements.
Furthermore, when I do "store as global variable" on the Hello div and toggle and then store again, these two global variables are in fact the same.
Finally, the Render Stats say that 3 operations are being made and none of them touches the Hello component, so I don't understand why the browser thinks it's changing. When I manually call insertBefore() or remove(), it doesn't show the Hello element as changing.
Paging @mhevery, it looks like there is a bug with applying operations in this particular case?

@wmertens
The mutation of the child list can triggers the rerendering.
Changed the forward <p> element to be kept, then the rerendering is not happened.
https://qwik.dev/playground/#v=1.2.17&f=7VbtTsIwFP3PU9QlwpZsgyqIMkZ8AH%2F6ABTocHF8ZJMYJHt3z12LXZdhYvxl4r8t7b23Pb33nGM1DX9o6xqfUSP7RoTVp1KvM%2FxO87kcQKD1XbFczOadUGzTDREtQaC%2B8QjB6pBXHxM2CMejIrqwyIuI9fv5YbutnASIl%2BgUgvgCCpZJAv60I4mgKGczJXHjRP%2FLIAEP2OsgalQIzo0Dj3NYpMtgIT9STF54c%2BeHo1s%2FHA99Hg7vAWQ9OEmzLNjsVqhAuvQu8tVvy0tRYEvZeXyVxyRHdGFFV2gOrvFou71Ypm9058iI%2BYQVS5FJFzj4BIYXldjPB3YEb4ngPuNexEqUntv9%2FCSOxOtxO03UjVdPmwI8BSJ6Xkevoy%2Fo2WLTVi5JJcC8PB3KeRrDdXK9eEZpQpAxZDZmV%2BavnD3v1utMfhnWcxYSsNqY7GenWopulzkLCRCkUxKzmX00BGaeGGtEIY9IYLYoiPBVg2ZqmaGjK1pgqvn%2BAZZVAKA0hAxvrox47OjOcBRtKNuOot8TvC5Wc6VqaqlG2wkqqq0dwJ2qjtB3VTeqbjzt6xWPDvEvEX9PIj4B


cmake: scripts: generate zip files for app and mcuboot updates

When enabled, this feature will generate a zip file for each
upgradeable component. This is the file which is typically given
to the DFU backend. The ZIP file also contains a 'manifest.json'
file which describes the contents.
Signed-off-by: Håkon Øye Amundsen<EMAIL_ADDRESS>

No sauce tag and looks like it might be nrf specific?

Fixed!

ping @oyvindronningstad  @sigvartmh

How does this interact with https://github.com/NordicPlayground/fw-nrfconnect-nrf/pull/1508?
Is it possible to put this script in nrf and have it called from the applications which generate FOTA packages?

How does this interact with NordicPlayground/fw-nrfconnect-nrf#1508?

AFAICS this PR does not introduce any "extra" interactions between this #1508.

Is it possible to put this script in nrf and have it called from the applications which generate FOTA packages?

Do you mean the .py script? I guess that makes sense, as it is not specific for mcuboot.

Is it possible to put this script in nrf and have it called from the applications which generate FOTA packages?

Do you mean the .py script? I guess that makes sense, as it is not specific for mcuboot.

That would be ideal if possible then. We shouldn't be putting things into our downstream forks without good reason, as it creates a bigger maintenance / synchronization burden.

Moved the script file to nrf/scripts/bootloader/generate_zip.py.

Is it possible to put this script in nrf and have it called from the applications which generate FOTA packages?

Moved the script file to nrf/scripts/bootloader/generate_zip.py.

OK, thanks! So that's the first half ("put this script in nrf"), but what about "have it called from the applications which generate FOTA packages"? E.g. from nrf/applications/asset_tracker/CMakeLists.txt. Then no OOT patches are needed in mcuboot.
I'm just generally trying to understand why we need to keep adding downstream specific stuff into our mcuboot fork. Would doing what I'm asking about require waiting for @tejlmand 's new multi-image patch series? I was under the impression that that work would let you specify build outputs from one child image in your main application CMakeLists.txt, where they could be post-processed (e.g. to create the FOTA package).

OK, thanks! So that's the first half ("put this script in nrf"), but what about "have it called from the applications which generate FOTA packages"? E.g. from nrf/applications/asset_tracker/CMakeLists.txt. Then no OOT patches are needed in mcuboot.

AFAIK this entire file is a OOT patch? In other words its just a bunch of commits to drag along, no conflicts.

I'm just generally trying to understand why we need to keep adding downstream specific stuff into our mcuboot fork.

I guess it is because this is the place we put all cmake code which needs to be executed when mcuboot is included in the build. One alternative could be to make a wrapper script in the nrf repo, and have all of this code there. I would love that change :)
Since the information needed to generate the zip file in this cmake-context, we can not move only the generate-zip part of the logic. Either we move the entire file, or we keep the generate-zip logic here.

@mbolivar , I've made a proposal for moving this whole file here https://github.com/NordicPlayground/fw-nrfconnect-nrf/pull/1576

This repository's history has been rewritten for the v1.2 release. Please rebase this PR if still relevant, or close it if no longer needed. Thanks!


'If' should force curly braces

As stated in the Android Kotlin Style Guide under Formatting, even if an If statement have only one line, it should be surrounded by a curly brace.
In my project, my code:
if (foo)
    bar = "hello world"

was deemed acceptable by ktlint. Is there anything I can do short of writing my own rule? Thanks in advance.

Hi @jasondhl.
Unfortunately this is not enforced at the moment.
PR is welcome.

Thanks for the prompt reply @shyiko ! Not very sure how to go about adding this Rule, but I'll check it out.

@jasondhl take a look inside https://github.com/shyiko/ktlint/tree/master/ktlint-ruleset-standard and https://github.com/shyiko/ktlint#ast (that's more or less all there is to it).

@jkaan that would be great!

This rule was contributed by @yokotaso (thank you, Tomoya!). It will be included in the next release.

I'm reopening the issue as auto-correction isn't quite there yet.

I can try to add auto-correction

@AleksandrSl that would be awesome! (here is a little bit of context)

Can I rely on IndentationRule to be run after other rules, or this rule should generate properly indented code?


UP-4731: Update dependency versions (rel-4-3-patches backport)

Checklist

[x] the individual contributor license agreement is signed
[x] commit message follows commit guidelines

Description of change

PRs #837, #844, and #851 cherry-picked for rel-4-3-patches.
See description in ticket UP-4731.

Coverage increased (+0.004%) to 22.056% when pulling bf25017e1baf41360853c7a4dc7059909e5f04b6 on ChristianMurphy:UP-4731-rel-4-3 into 3736af9156894c523349fd62fb2afebf8d57509a on Jasig:rel-4-3-patches.

Coverage decreased (-0.002%) to 22.05% when pulling 84dd47d3d5bf8ce8f7367929102816640137d770 on ChristianMurphy:UP-4731-rel-4-3 into 3736af9156894c523349fd62fb2afebf8d57509a on Jasig:rel-4-3-patches.

LGTM :+1:
Did you run a $ ant clean initportal with these changes, start tomcat, and click around a bit?

@drewwills I've check that it compiles, I have not smoke tested this yet.

Testing locally...

@ChristianMurphy -- I un-did the version updates that were causing regressions:  https://github.com/Jasig/uPortal/pull/900/commits/98ee0982f539246414de17ea8798d0655ecef661
We should discuss tomorrow whether we are satisfied without those changes, or if we should really resume this work and do the difficult things it will take to update those libraries as well.

👍 Upgrading commons and groovy are worth while on their own.


fix: Update the Stripe integration with code examples

Description
Provided code example for the integration of Stripe with VSF and Vendure
Related Issue

Motivation and Context

How Has This Been Tested?

Screenshots (if appropriate):
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[ ] My code follows the code style of this project.
[x] My change requires a change to the documentation.
[x] I have updated the documentation accordingly.
[x] I have read the CONTRIBUTING document.
[ ] I have added tests to cover my changes.
[ ] All new and existing tests passed.

Hi @julienTeam
Thanks for creating that! I will review it in the upcoming days :)


Update path documentation

Because this check https://github.com/ramda/ramda/blob/cffff75259a5b251811bc5197ea6ec41b33731e2/src/path.js#L26
both null and undefined are checked. Or fix code and use !== instead.
R.path(['a', 'b'], {c: {b: 2}}); //=> undefined or null

Do you mean something like this is incorrect?
R.path('a', 'b', {a: 1, b: null}) => // undefined

R.path(['foo', 'bar'], {foo: null}) === null

It returns value of obj.foo. But as I understand it should return value of obj.foo.bar or undefined

Ah yes, I get what you mean now. Yeah, I'd expect undefined there too.

I see @svozza's fine fix in #1537.
My only question is whether we want to fix the code, or whether we're better adjusting the documentation.  Which is the better response for
R.path(['foo', 'bar'], {foo: null})

undefined?  or null?

I think documentation should be updated. We need to test both values.
undefined is absence of evidence, while null is evidence of absence.

I'm conflicted about this, on the one hand I wish JavaScript only had one null value (or Maybe even none :wink:) but we are where we are. I've used APIs where null had meaning as opposed to undefinedand a user doing so is not necessarily doing anything incorrect by respecting the distinction the language makes. End of the day, this is just wrong:
R.path(['a', 'b', 'c'], {a: null}); // => null


Add high score feature

It is the only way to properly demonstrate my Pac-Bar prowess, as illustrated here:

I've (finally) added this alongside Swift 4 conversion in commit 23a2fca.


Bugs noticed during book writing

Noticed these issues as I attempted to reproduce various log messages for the Graphite book. Consider this issue a work-in-progress; I'll add anything else that I encounter. I intend to submit PRs for these once I feel relatively confident I've sniffed them all out.

[ ] We will never actually trigger this because we set a default archive. Affects both 0.9.x and master.
[ ] We need to wrap this in a try block and raise exceptions. Affects both 0.9.x and master.
[ ] This needs to inform the user as to which rules file has the bad line. Affects both 0.9.x and master.

Decided against removing this sanity check. Even though it's currently impossible for us to hit it due to our default archive, there's always a chance that may change in the future.


See when a user was last logged

As an organization administrator, I want to be able to see when a user was last logged in in order to find stale user accounts that could potentially be removed.
gz#321

Please assign to me when there's API available

@kyle-ssg API was added in the PR here: https://github.com/Flagsmith/flagsmith/pull/1906
New last_login field has been added to GET /api/v1/organisations/:id/users/. This is available in the staging API.


Not working on ios

I just noticed demo does not work on ios:  https://foobar404.dev/wave.js/
Not just visualizer but audio does not play at all.
Is this something that can be addressed?

@aproni34f I dont have access to an ios device, so its tough for me to fix these kinds of bugs, usually they are simple to fix but hard to find.
If someone has access to an IOS device please feel free to submit a pull request.

I reopen this problem.
the library does not work in safari on ios, however on macOS and ipad it does.
has anyone been able to fix it?

i'm seeing the example on an iPhone 14, in Safari, and all seems good!


Normalize custom authorize validator API style

fixes #848

@brockallen,
Thanks for having already signed the Contribution License Agreement. Your agreement was validated by .NET Foundation. We will now review your pull request.
Thanks,
.NET Foundation Pull Request Bot


Support byte array result when comparing

@kabunchi please ask for rereview when you address comments

@sopel39 we get byte[] as a result of checksum(col)
Also, fixed the comment, code ready for review.

FYI, also tested this branch on a private run of a branch in benchmarks-gha having checksum in its queries.


got 0 ops for nn.MultiheadAttention

My thop version: 0.1.1
Minimal code to replicate:
import thop
import torch.nn as nn
import torch

f = nn.MultiheadAttention(100, 1, batch_first=True)

x = torch.ones((1, 1000, 100))

result = thop.profile(f, (x,x,x))
print(result)

I got (0.0, 0). Is this module not supported yet?

Does it only support modules listed here?
https://github.com/Lyken17/pytorch-OpCounter/blob/43c064afb71383501e41eaef9e8c8407265cf77f/thop/profile.py#L21

you are right. I guess this module doesn't supported MHSA layer yet.

facing the same problem

You may use https://github.com/HaoKang-Timmy/torchanalyse or torchprofile for NLP models

You may use https://github.com/HaoKang-Timmy/torchanalyse or torchprofile for NLP models

These two repo could profile transformers, which I have tried.

You may use https://github.com/HaoKang-Timmy/torchanalyse or torchprofile for NLP models

thank you for your reply. I'll try it!

It seems thop does not account for any parameters created with nn.Parameter. For any created with nn.Parameter, it simply yields 0.


Add RunwayProject

spec_version: v1.27
identifier: RunwayProject
name: RWP Modpack
abstract: A modpack for Scott Manley's Runway Project
version: '1.0'
$kref: '#/ckan/spacedock/2487'
tags:
  - parts
  - plugin
  - combat
ksp_version_min: 1.12.0
depends:
  - name: ModuleManager
  - name: PhysicsRangeExtender
  - name: ModeratelyPlaneRelated
  - name: ColdWarAerospace
  - name: Mk2Expansion
  - name: Mk3Expansion
  - name: B9PartSwitch
  - name: AirplanePlus
  - name: B9-PWings-Modified
  - name: BDArmoryExtended
x_via: Generated by Metadata Webtool

Added by request of the Runway Project team, who were looking for an easy way to do a modpack install without having to mess around with copying most of a gamadata.

Hmm, that's just the link to BDArmoryPlus, which is already indexed (same SpaceDock ID number in the $kref):

https://github.com/KSP-CKAN/NetKAN/blob/master/NetKAN/BDArmoryForRunwayProject.netkan
#8063

Indexing the same mod again would cause a lot of confusion (at best), and a dependency on BDArmoryExtended (which is the "other" BDArmory fork) would probably break it outright.
Maybe this should be a metapackage .ckan file?

https://github.com/KSP-CKAN/CKAN/wiki/Sharing-a-modlist-(metapackages)

Hmm. Agreed that that is the simpler way to go about it.


Void / broken links for Python documentation ... https://www.vapor.ucar.edu/pages/pythonDocumentation.html

Examples and Class Reference links are void:
Python Documentation
Links have been void for a while... Is there any plan to provide this?
Examples
Examples
Coming soon.
Class Reference
Class Reference
Coming soon.

@richlysakowski Thanks for reaching out. Yes, the links will be updated within the next two weeks with our next release.
Meanwhile, you can access some example jupyter notebooks and python scripts provided with the package. You can find instructions on how to access them here
Class references can be accessed through help
For instance for help on animation class:
import vapor
from vapor import animation
help(animation)
Please let us know if you have any further questions. We also welcome any other feedback on how we can improve documentation. Thank you!

Fixed.


Suggestion for the Current Speed module

Environment
Docker
Version
0.10.0
Describe the problem
I think, the information of this module (title, info...) should be re-worked, as it may cause confusion.
I mean, you can easily confuse it with the down/upload speed of the host where Homarr is installed, rather than the torrent client where the module collects data from.
An example: I have Homarr in a different server than where qBittorrent is installed.
Additional info
No response
Please tick the boxes

[X] You've read the docs
[X] You've checked for duplicate issues
[X] You've tried to debug yourself

Thank you for your suggestion. I understand that the title and info may cause confusion. What do you think should it be renamed to?

@SmartPhoneLover bump, do you agree with the new naming of this widget ?

@SmartPhoneLover bump, do you agree with the new name of this widget ?

Yes, I agree. Now it's better explained about its function.
Thank you!

Closing as completed


ansible playbooks for deploy

would it be possible to get some ansible roles for the deployment?

or a guide on how to deploy without using the scripts

@casualjim
Are you trying to use ansible to call docker to deploy?  This is easy as the docker-compose.yml is self-explanatory.
Or are you asking for using ansible to deploy without docker,  I believe this is doable, but we haven't tried that, so maybe difficult.

memo:
https://github.com/supervised-io/machines/blob/dockerRegistryAnsible/terraform/playbooks/harbor.yml

@casualjim
Did you work on it already? I'd like to write an ansible galaxy role. If someone already started it, I needn't work it from beginning.

yeah we have something in the repo linked by @reasonerjt

@casualjim
The link as marked as memo is 404 for me.

@reasonerjt
Could you open it to public for this link: https://github.com/supervised-io/machines/blob/dockerRegistryAnsible/terraform/playbooks/harbor.yml ?

It gives me 404 now also...

@reasonerjt & @casualjim
If this link is broken, could you please upload the code from your fork, if you did that before?

ansible role for harbor
tasks/main.yml
---
- name: ensure data folders
  file: path=/var/lib/harbor/{{ item }} mode=0755 state=directory
  with_items:
    - database
    - logs
    - job_logs
    - registry

- name: ensure config folder
  file: path=/etc/harbor/ mode=0755 state=directory

- name: copy config files
  copy: src=config/{{ item }} dest=/etc/harbor  mode=0644
  with_items:
    - db
    - jobservice
    - nginx
    - registry
    - ui

- name: add app.config
  template: src=app.conf dest=/etc/harbor/ui/app.conf mode=0644

- name: install docker registry
  docker_container:
    name: registry
    image: library/registry:2.5.0
    restart_policy: always
    volumes:
      - "/var/lib/harbor/registry:/storage"
      - "/etc/harbor/registry:/etc/registry/"
    env:
      GODEBUG: "netdns=cgo"
    ports:
      - "5001:5001"
    command: "serve /etc/registry/config.yml"

- name: install mysql
  docker_container:
    name: mysql
    image: jcali/mysql
    restart_policy: always
    env:
      MYSQL_ROOT_PASSWORD: root123
    volumes:
      - /var/lib/harbor/database:/var/lib/mysql

- name: install ui
  docker_container:
    name: ui
    image: jcali/ui
    restart_policy: always
    links:
      - "mysql:mysql"
    env:
      MYSQL_HOST: mysql
      MYSQL_PORT: 3306
      MYSQL_USR: root
      MYSQL_PWD: "{{harbor_db_pass}}"
      REGISTRY_URL: http://registry:5000
      UI_URL: http://ui
      CONFIG_PATH: /etc/ui/app.conf
      HARBOR_REG_URL: "{{inventory_hostname}}"
      HARBOR_ADMIN_PASSWORD: "{{harbor_admin_pass}}"
      HARBOR_URL: http://{{inventory_hostname}}
      AUTH_MODE: db_auth
      LDAP_URL: ldaps://ldap.mydomain.com
      LDAP_BASE_DN: "uid=%s,ou=people,dc=mydomain,dc=com"
      UI_SECRET: "{{harbor_ui_secret}}"
      SELF_REGISTRATION: "on"
      USE_COMPRESSED_JS: "on"
      LOG_LEVEL: debug
      GODEBUG: "netdns=cgo"
      EXT_ENDPOINT: http://{{inventory_hostname}}
      TOKEN_URL: http://ui
      VERIFY_REMOTE_CERT: "on"
      TOKEN_EXPIRATION: 30
    ports:
      - "8000:80"
    volumes:
      - /etc/harbor/ui/app.conf:/etc/ui/app.conf
      - /etc/harbor/ui/private_key.pem:/etc/ui/private_key.pem

- name: install jobservice
  docker_container:
    name: jobservice
    image: jcali/jobservice
    restart_policy: always
    links:
      - "mysql:mysql"
    env:
      MYSQL_HOST: mysql
      MYSQL_PORT: 3306
      MYSQL_USR: root
      MYSQL_PWD: "{{harbor_db_pass}}"
      UI_SECRET: "{{harbor_ui_secret}}"
      CONFIG_PATH: /etc/jobservice/app.conf
      REGISTRY_URL: http://registry:5000
      VERIFY_REMOTE_CERT: "on"
      MAX_JOB_WORKERS: 3
      LOG_LEVEL: debug
      LOG_DIR: /var/log/jobs
      GODEBUG: "netdns=cgo"
      EXT_ENDPOINT: http://{{inventory_hostname}}
      TOKEN_URL: http://ui
    volumes:
      - /var/lib/harbor/job_logs:/var/log/jobs
      - /etc/harbor/jobservice/app.conf:/etc/jobservice/app.conf

- name: Create certs directory
  file: path=/etc/nginx/cert mode=0755 state=directory

- name: add certificate
  copy: dest="/etc/nginx/cert/celloproject.io.crt" content="{{ celloproject_io_certificate }}"

- name: add key
  copy: dest="/etc/nginx/cert/celloproject.io.key" content="{{ celloproject_io_private_key }}"

- name: set up nginx
  docker_container:
    name: nginx
    image: library/nginx:1.9.0
    restart_policy: always
    ports:
      - "80:80"
      - "443:443"
    links:
      - "mysql:mysql"
      - "registry:registry"
      - "ui:ui"
    volumes:
      - "/etc/harbor/nginx:/etc/nginx/"
      - "/etc/nginx/cert/:/etc/nginx/cert/"

templates/app.conf
appname = {{ harbor_appname }}
runmode = {{ harbor_runmode }}

[lang]
types = en-US|zh-CN
names = en-US|zh-CN

[dev]
httpport = {{ harbor_dev_httpport }}

[mail]
host = {{ smtp_host }}
port = {{ smtp_port }}
username = {{ smtp_username }}
password = {{ smtp_password }}
from = {{ email_from }}
ssl = false

this is most of the work, can't share the repo publicly because it requires all kinds of corp hoops.

Ended up here after looking at the installation instructions and the intstall.sh, which don't look very idempotent (yet?).
Will try out the above to see how far that gets me.

In foreseeable future, here will be three deployment approaches supported:

docker-compose:  For quickly deployment on single host.
bosh-release/tile: For product integration.
helm chart: For deployment on top of k8s cluster.

So I'm closing this issue as won't fix, any interest to maintain other form of installation would be appreciated.

I have started this: https://github.com/nicholasamorim/ansible-role-harbor
It does some thins like install (obviously) but also creates users and projects. It also allows you to change some things like the default Redis settings, in case the user already has another docker instance of redis running and so on.
I'd love to get input on it to make it better as I'm a novice to Harbor.


Fix Daffodil svg logo bug where the petals don't show up on Edge and Firefox

PR Checklist
Please check if your PR fulfills the following requirements:

[x] The commit message follows our guidelines: https://github.com/graycoreio/daffodil/blob/master/CONTRIBUTING.md#commit
[ ] Tests for the changes have been added (for bug fixes / features)
[ ] Docs have been added / updated (for bug fixes / features)

PR Type
What kind of change does this PR introduce?

[x] Bugfix
[ ] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Build related changes
[ ] CI related changes
[ ] Documentation content changes
[ ] Other... Please describe:

What is the current behavior?

Issue Number: Fixes #117
What is the new behavior?
Does this PR introduce a breaking change?
[ ] Yes
[x] No

Other information

@damienwebdev can you check it out on Edge for me?


Question(storage): delete one record from MD

To delete one record from MD need execute two opers (remove all MD and sets interested MD items)? It is ok?

step 1 - create object with MD (two items) but without data!
step 2 - get object - got all MD
step 3 - upd MD - delete one item from MD
step 4 - get object - got all MD, but I want one item from MD (because step 3 deleted one item)

        c := mustCSClient()
	b, _ := c.Bucket(bucketName)

	// step 1
	fw := b.Object(fileName).NewWriter(context.Background())
	// NOTE: if sets data then metadata is not writed
	// fw.Write([]byte("foobar"))
	fw.Metadata = map[string]string{"a": "b", "c": "d"}
	err := fw.Close()
	assert.NoError(t, err)

	// step 2
	obj := b.Object(fileName)
	attrs, err := obj.Attrs(context.Background())
	assert.NoError(t, err)
	t.Logf("Step 2: %v", attrs.Metadata)

	// step 3
	delete(attrs.Metadata, "c")
	t.Logf("Step 3: %v", attrs.Metadata)
	_, err = obj.Update(context.TODO(), gcStorage.ObjectAttrsToUpdate{
		Metadata: attrs.Metadata,
	})
	assert.NoError(t, err)

	// step 4
	obj = b.Object(fileName)
	attrs, err = obj.Attrs(context.Background())
	assert.NoError(t, err)
	t.Logf("Step 4: %v", attrs.Metadata)

PS If create an object with data, then MD will not be written? It is ok?
UPD: cloud.google.com/go/storage v1.15.0

Hi @gebv,
Thanks for the issue. I don't think we currently support deleting a metadata key from an object per https://github.com/googleapis/google-cloud-go/issues/380#issuecomment-250321401.
I'll reach out to @tritone when he's back in the office to double check, but this looks like a gap in implementation.

@frankyn thx for answer. I don't think MD item with an empty value makes any sense. And I expected should overwritten MD - all items and not try to merged (server side).

Hi @gebv,
I synced with @tritone and this is a gap in our support and I'm looking into a way to potentially support this but not certain if it will be the right approach.
As a workaround you could do the following:
delete(attrs.Metadata, "c")
// Delete all object metadata
_, err = obj.Update(context.TODO(), gcStorage.ObjectAttrsToUpdate{
	Metadata: nil,
})

// re-Insert remaining metadata without key "c"
_, err = obj.Update(context.TODO(), gcStorage.ObjectAttrsToUpdate{
	Metadata: attrs.Metadata,
})

Thank you for your patience.

Yes, with two requests I solve the problem. Thx


Help to tell any sample for paint app on canvas/webgl?

sorry, is it possible to use raylib to make web paint app (mobile browser almost ) and does it support fallback to Canvas API if WebGL is not available ? Many thanks

Well, I think so. You can use raylib to create a web painting app... but I haven't tried it, depending on your requirements it could take a lot of work.
raylib on web only uses WebGL, no fallback to Canvas API.

The easiest starting point would be:

record points, for what the mouse is doing when a button is pressed
use DrawPolyEx to draw that information

Then the next step from that would be to:

record points
draw lines to an image (see bresenhams line drawing)
UpdateTexture
draw the texture to screen

Just closing this issue...


InteractionRequiredAuthError doesn't include the claims when implementing conditional access

Core Library
MSAL Node (@azure/msal-node)
Core Library Version
1.14.5
Wrapper Library
N/A
Wrapper Library Version
0
Public or Confidential Client?
Confidential
Description
When trying to Implement conditional access (Require MFA) in a ConfidentialClientApplication using the acquireTokenOnBehalfOf API, we are not receiving the claims in the error object to acquire a new access token with the required claims.
{
  "errorCode": "invalid_grant",
  "errorMessage": "AADSTS50076: Due to a configuration change made by your administrator, or because you moved to a new location, you must use multi-factor authentication to access ''<recourse-id>.\r\nTrace ID: <app-id>\r\nCorrelation ID: <Correlation-ID>\r\nTimestamp: 2022-12-19 22:37:41Z",
  "subError": "basic_action",
  "name": "InteractionRequiredAuthError"
}

When looking at the raw response on fiddler, we receive the claims from the token endpoint as shown below:
{
    "error": "invalid_grant",
    "error_description": "AADSTS50076: Due to a configuration change made by your administrator, or because you moved to a new location, you must use multi-factor authentication to access '<recourse-id>'.\r\nTrace ID: <Trace-id>\r\nCorrelation ID: <Correlation-id>\r\nTimestamp: 2022-12-19 22:36:37Z",
    "error_codes": [
        50076
    ],
    "timestamp": "2022-12-19 22:36:37Z",
    "trace_id": "trace_id",
    "correlation_id": "correlation_id",
    "error_uri": "https://login.microsoftonline.com/error?code=50076",
    "suberror": "basic_action",
    "claims": "{\"access_token\":{\"capolids\":{\"essential\":true,\"values\":[\"GUID\"]}}}"
}

The issue is when instantiating the InteractionRequiredAuthError class, the claims are not passed to the class. Is there a way to receive the complete error response from msal?
Error Message
{
  "errorCode": "invalid_grant",
  "errorMessage": "AADSTS50076: Due to a configuration change made by your administrator, or because you moved to a new location, you must use multi-factor authentication to access ''<recourse-id>.\r\nTrace ID: <app-id>\r\nCorrelation ID: <Correlation-ID>\r\nTimestamp: 2022-12-19 22:37:41Z",
  "subError": "basic_action",
  "name": "InteractionRequiredAuthError"
}

Msal Logs
[Tue, 20 Dec 2022 01:28:53 GMT]<EMAIL_ADDRESS>: Info - acquireTokenOnBehalfOf called
[Tue, 20 Dec 2022 01:28:53 GMT] : [ed732fe7-13fd-4e60-a6e4-719fd554968f]<EMAIL_ADDRESS>: Info - SilentFlowClient:acquireCachedToken - No access token found in cache for the given properties.
MSAL Configuration
const msalConfig = {
    auth: {
        clientId: config.credentials.clientID,
        authority: `https://${config.metadata.authority}/${config.credentials.tenantID}`,
        clientSecret: config.credentials.clientSecret,
        clientCapabilities: ["CP1"],
    },
    system: {
        proxyUrl: "http://localhost:8866",
        loggerOptions: {
            loggerCallback(loglevel, message, containsPii) {
                console.log(message);
            },
            piiLoggingEnabled: false,
            logLevel: msal.LogLevel.Info,
        },
    },
};

Relevant Code Snippets
const msal = require('@azure/msal-node');
const config = require('./authConfig');

const msalConfig = {
    auth: {
        clientId: config.credentials.clientID,
        authority: `https://${config.metadata.authority}/${config.credentials.tenantID}`,
        clientSecret: config.credentials.clientSecret,
        clientCapabilities: ["CP1"],
    },
    system: {
        proxyUrl: "http://localhost:8866",
        loggerOptions: {
            loggerCallback(loglevel, message, containsPii) {
                console.log(message);
            },
            piiLoggingEnabled: false,
            logLevel: msal.LogLevel.Info,
        },
    },
};

// Create msal application object
const cca = new msal.ConfidentialClientApplication(msalConfig);

const getOboToken = async (oboAssertion) => {
    const oboRequest = {
        oboAssertion: oboAssertion,
        scopes: config.resources.downstreamAPI.scopes,
    };

    try {
        const response = await cca.acquireTokenOnBehalfOf(oboRequest);
        return response.accessToken;
    } catch (error) {
        console.log(error)
        throw error;
    }
};

Reproduction Steps

Create a ConfidentialClientApplication.
Use acquireTokenOnBehalfOf API to get an access token for a recourse.
Create a CA policy to enable MFA for the recourse.
Check the InteractionRequiredAuthError.

Expected Behavior
Receiving the claims with the InteractionRequiredAuthError to acquire an access token.
Identity Provider
Azure AD / MSA
Browsers Affected (Select all that apply)
Chrome
Regression
No response
Source
Internal (Microsoft)

I think this is only relevant for OBO so I'll let @bgavrilMS address this

Thanks @tnorling . This is about exposing "Claims" field that comes with AAD errors. This is confidential client specific, it covers public client flows as well.
The flow is:

AAD signals that "a more powerful" token is required (when redeeming an RT or when doing OBO). They send back error invalid_grant + claims
MSAL should throw appropriate error (UserInteractionRequired) and expose the claims sent by AAD
app developer calls AcquireTokenInteractive / AcquireTokenByOBO and adds claims
MSAL bypasses the cache and fetches new token

There seems to be several issues related to handling claims and CAE in MSAL JS:

https://github.com/AzureAD/microsoft-authentication-library-for-js/issues/5494
https://github.com/AzureAD/microsoft-authentication-library-for-js/issues/5486

Thanks @tnorling . This is about exposing "Claims" field that comes with AAD errors. This is confidential client specific, it covers public client flows as well.

@bgavrilMS Fixing this will likely address it for all flows anyway but I am curious, what is the public client scenario where AAD would throw claims back to the caller rather than just handling it on their own? My thought was that the absence of claims on the error object is only an issue for OBO because the caller in that scenario is not capable of invoking interaction on its own. I'm not sure I understand why a claims challenge from AAD (rather than a resource) would be necessary in public clients?

Thanks @tnorling . This is about exposing "Claims" field that comes with AAD errors. This is confidential client specific, it covers public client flows as well.

@bgavrilMS Fixing this will likely address it for all flows anyway but I am curious, what is the public client scenario where AAD would throw claims back to the caller rather than just handling it on their own? My thought was that the absence of claims on the error object is only an issue for OBO because the caller in that scenario is not capable of invoking interaction on its own. I'm not sure I understand why a claims challenge from AAD (rather than a resource) would be necessary in public clients?

Excellent question. Apart from OBO there are 2 more scenarios documented here: https://learn.microsoft.com/en-us/azure/active-directory/develop/v2-conditional-access-dev-guide#scenario-app-accessing-multiple-services - one is around SPAs and one is "accessing multiple services" - the docs mention a web app calling web apis, but this can very well be a mobile app accessing multiple web apis. CC @jmprieur to keep me honest on this.

@bgavrilMS Thanks for sharing that. After reading through it, it's still not clear to me why the claims are important to the caller in a web app scenario (I also don't see the claims called out explicitly like they are in the OBO section).
Let's ignore silent calls for a moment, if I make an interactive token request in a web app and AAD determines that claims are needed I would expect it to present any prompts it needs to and then return the token with the needed claims. In what scenario would it throw the claims back to the caller and expect the caller to make another interactive request with the claims it just threw?
Now if we agree that AAD should "do the right thing" in an interactive request regardless of whether or not claims are included, then the claims aren't really relevant in a web app at all, even if they are returned in a silent request error, since the "interaction_required" error is enough information for the caller to know what to do to resolve it.

I am not sure about this point @tnorling , hoping that @jmprieur or @salman90  can clarify. It may be a limitation of CA?
The docs and diagrams do show that AAD will reply with an error + claims and that there is an expectation for the app developer to include the claims in a new token request.

Just to chime in, so far we didn't run into an issue with any of the CA scenarios we tried using a public client -since STS knows which claims are required, it knows how to get a token with those claims. So @tnorling is right on that account.
Coming back to the silent calls however, since msal.js caches/distinguishes tokens on the basis of claims in addition to scopes and claims are needed in silent requests (as of msal-common@v6), what happens if devs want to retrieve a token from the cache with claims as opposed to going to the network? (my memory is failing me here, as we tried most CA scenarios before this change -@salman90 would be good check this out)
On a more general note, I'm not sure if the SDKs should hide something that the platform itself choses to disclose. If the fact that the claims will be exposed to the app is shared knowledge, there might be features in future that rely on this (though perhaps this is a YAGNI case..)

@bgavrilMS @tnorling I tried both scenarios: SPA calling CA-protected web API using MSAL.js and a web app accessing multiple services (one service is CA protected). In both cases, an interaction_required is enough for the CA to work as expected; as @tnorling  said, Azure AD does the right thing with or without a claim.
@derisen  in the SPA case, we are not hiding anything because the claim is not sent with the interaction_required error from the auth server. I still need to confirm that the same behavior is happening in the web app scenario. I will debug the app further and confirm.

@bgavrilMS @tnorling  I finish debugging the web app scenario; we do receive the claims in the error object:
{"error":"invalid_grant","error_description":"AADSTS50076: Due to a configuration change made by your administrator, or because you moved to a new location, you must use multi-factor authentication to access 'GUID'.\r\nTrace ID: 64ab9ad5-447a-432c-ba42-83784ad85800\r\nCorrelation ID: GUID\r\nTimestamp: 2023-01-04 19:44:18Z","error_codes":[50076],"timestamp":"2023-01-04 19:44:18Z","trace_id":"GUID","correlation_id":"GUID","error_uri":"https://login.microsoftonline.com/error?code=50076","suberror":"basic_action","claims":"{\"access_token\":{\"capolids\":{\"essential\":true,\"values\":[\"GUID\"]}}}"}

The scenario works fine with just an interaction, but I think @derisen  is correct that we shouldn't hide something (claims) that the platform provides.

I agree, we need to expose the claims as part of the exception data. We have been doing this for years in ADAL and MSAL .NET

Agreed, if it's returned from the service we should return it as well. I started this discussion mostly for my own education so thanks for indulging me. To summarize my understanding of the above: we do not have a functional miss in PCAs today and addressing this bug would just serve to future-proof or add completeness to our interaction_required errors (and address a gap in OBO scenarios).

@Robbie-Microsoft - please pick this up next.


Set up continuous integration to automatically test the project

Catch bugs, enforce style, and increase confidence in the project before someone merge.

Which one do you intend to integrate:

CircleCI. I'm using this one in a private project and was a piece of cake to configure.
Travis
Semaphore


Test devinfoedit component

Description
ADD test suites for DevInfoEditz.js component
Type of change
Please delete options that are not relevant.

[x] New feature (non-breaking change which adds functionality)

How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

[x] Runnign all test suites.

Checklist:

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[ ] My code has been reviewed by at least one peer
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[x] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes

Hello Danny,
Issue #109 , consider the unique key warnings.
Are you available next Monday to meet? so you can show me the errors updating and creating entries.

Hey Jean. If you've got the time tomorrow I can show you. If not then any other day between 10-12pm PST.


Package installation

(Notes from video sync with @thanethomson.)
For internal (and eventually external users) we'll want some packaging that can handle the system dependencies. We might start with a bash script (as discussed) but we could also consider using using system package managers (e.g.g, homebrew, apt).
We could also consider making a nix package, which should provide flexible cross-platform package management. I'd be willing to explore packing it via nix if we agreed this would be worth exploring.
In any case, we'll probably want to help users configure their node packages to install locally to a user, so as not to need root access. See https://github.com/glenpike/npm-g_nosudo for a script that does this.

brew

brew


[Bug]: Disconnect from Station

Version
1.3.0
Node version
1.3.0
Operating System type
Other
Operating system version
Rpi4
Describe the bug
I have a HB2 with a Doorbell T8210.
For well over a week now my homeassistant integration (fuatakgun Eufy add-on) stopped working.  I stopped having Thumbnail for the Doorbell. I saw a lot of errorlogging (below) regarding the connection to the station.
I tried a few suggestions in some of the threads regarding similar issues.
removed add-on, reinstalled it, configured it with ‘Eufy’ as Trusted_Device_Name and pre-used accountsettings, and still no thumbnail wenn Doorbell is pressed and keep getting the same disconnect to station logging.
It used to work yust fine before Eufy changed their security policy...
2022-12-28 14:36:39.762  INFO  Eufy Security server listening on host <IP_ADDRESS>, port 3000
2022-12-28 14:36:44.418  INFO  Push notification connection closed
2022-12-28 14:36:50.080  INFO  Push notification connection successfully established
2022-12-28 14:36:50.116  INFO  Push notification connection successfully established
2022-12-28 14:36:50.169  INFO  Connected to station T8010P2320361053 on host <IP_ADDRESS> and port 17267
2022-12-28 14:37:53.637  INFO  Disconnected from station T8010P2320361053
2022-12-28 14:37:58.712  INFO  Connected to station T8010P2320361053 on host <IP_ADDRESS> and port 14954
2022-12-28 14:39:02.184  INFO  Disconnected from station T8010P2320361053
2022-12-28 14:39:07.261  INFO  Connected to station T8010P2320361053 on host <IP_ADDRESS> and port 17351
2022-12-28 14:40:10.718  INFO  Disconnected from station T8010P2320361053
2022-12-28 14:40:15.789  INFO  Connected to station T8010P2320361053 on host <IP_ADDRESS> and port 23958
2022-12-28 14:41:09.491  INFO  Client disconnected with ip: <IP_ADDRESS> port: 35612 code: 1000 reason: Normal Closure
2022-12-28 14:41:19.261  INFO  Disconnected from station T8010P2320361053
2022-12-28 14:41:24.320  INFO  Connected to station T8010P2320361053 on host <IP_ADDRESS> and port 23811
2022-12-28 14:42:27.785  INFO  Disconnected from station T8010P2320361053
2022-12-28 14:42:32.886  INFO  Connected to station T8010P2320361053 on host <IP_ADDRESS> and port 14958
2022-12-28 14:43:36.347  INFO  Disconnected from station T8010P2320361053
2022-12-28 14:43:41.407  INFO  Connected to station T8010P2320361053 on host <IP_ADDRESS> and port 13385
2022-12-28 14:44:44.870  INFO  Disconnected from station T8010P2320361053
2022-12-28 14:44:49.931  INFO  Connected to station T8010P2320361053 on host <IP_ADDRESS> and port 17975
Eufy Security add-on 1.3.0
Eufy security V5.1.4
To reproduce
removed add-on, reinstalled it, configured it with ‘Eufy’ as Trusted_Device_Name and pre-used accountsettings, and still no thumbnail wenn Doorbell is pressed and keep getting the same disconnect to station logging.
Screenshots & Logfiles
No response
Additional context
No response

Same issue here, bug #168 might be closed, things are still not working and see the same issue in the log file. Did delete the persistent.json and added 'Homebase' which is the name in my Eufy app as the TRUSTED_DEVICE_NAME as the description " (viewable with 2fa activated in Eufy App; default: eufyclient)" is not very clear on where to look and what to use....!?

I have the same problem, same Eufy hardware. I did a clean install of version 1.3.0 in docker on an Ubuntu server. First it didn't seem to work without the TRUSTED_DEVICE_NAME variable (even though I don't have MFA enabled for the dedicated account). Now the notifications work fine, video stream isn't very stable, and every minute the connection drops for 5 seconds.

Just an update, I switched from rpi3 to a x86 setup, with a fresh install. Hasn't fixed the bug.

This is currently happening to me, disconnects every 5 seconds.  Tried resetting persistent.json and changing trusted device names and nothing helps.
Everything works while it's connected - but it just drops over and over again.

Yeah this issue is still ongoing for me. Much of my setup still works, slightly unreliably, with missed alerts a couple times a day.
I'd imagine a lot of people have this issue and haven't seen it in their logs.

Fixed in release 1.3.1 .

Hi I have installed the addons  Bropats version 1.5.2
and fuatakgun verison 6.2.0
I have setup a secondary account which I have confirmed works
Doorbell is a T8222-Z
Chime is a T8020
on latest firmware
and I am getting
s6-rc: info: service s6rc-oneshot-runner: starting
s6-rc: info: service s6rc-oneshot-runner successfully started
s6-rc: info: service fix-attrs: starting
s6-rc: info: service fix-attrs successfully started
s6-rc: info: service legacy-cont-init: starting
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service legacy-services: starting
s6-rc: info: service legacy-services successfully started
2023-05-24 16:43:59.586  INFO  Eufy Security server listening on host <IP_ADDRESS>, port 3000
2023-05-24 16:44:04.282  INFO  Connected to station T8020P1123021XXX on host 192.168.41.XXX and port 18450
2023-05-24 16:44:11.034  INFO  Push notification connection successfully established
2023-05-24 16:44:12.672  INFO  Disconnected from station T8020P1123021XXX
2023-05-24 16:44:17.853  INFO  Connected to station T8020P1123021AA2 on host 192.168.41.XXX and port 24504
2023-05-24 16:44:25.750  INFO  Disconnected from station T8020P1123021XXX
This is causing many missed events per day
Is there something I have missed ??


[BUG]mars.new_ray_session  error

Describe the bug
Create a mars session on ray cluster has. error
To Reproduce
To help us reproducing this bug, please provide information below:

Your Python version

3.7.7

The version of Mars you use

0.9.0b1

Versions of crucial packages, such as numpy, scipy and pandas

numpy==1.21.5 pandas==1.3.5 pymars==0.9.0b1 xgboost "xgboost_ray" lightgbm

Full stack of the error.

(base) root@cd15e694a26a:~# python
Python 3.7.7 (default, May  7 2020, 21:25:33)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.

import ray
import mars
ray.init(address='ray://<IP_ADDRESS>:30772')
ClientContext(dashboard_url='<IP_ADDRESS>:8265', python_version='3.7.7', ray_version='1.9.2', ray_commit='ef593fe5d3c864836b80ae77be32635cef42b537', protocol_version='2021-09-22', _num_clients=1, _context_to_restore=<ray.util.client._ClientContext object at 0x7fb75b902190>)
import mars.tensor as mt
import mars.dataframe as md
session = mars.new_ray_session(worker_num=2, worker_mem=2 * 1024 ** 3)
(RayMainPool pid=671, ip=<IP_ADDRESS>) 2022-02-27 23:46:24,651	ERROR serialization.py:283 -- 'pickle'
(RayMainPool pid=671, ip=<IP_ADDRESS>) Traceback (most recent call last):
(RayMainPool pid=671, ip=<IP_ADDRESS>)   File "/home/ray/anaconda3/lib/python3.7/site-packages/ray/serialization.py", line 281, in deserialize_objects
(RayMainPool pid=671, ip=<IP_ADDRESS>)     obj = self._deserialize_object(data, metadata, object_ref)
(RayMainPool pid=671, ip=<IP_ADDRESS>)   File "/home/ray/anaconda3/lib/python3.7/site-packages/mars/oscar/backends/ray/communication.py", line 78, in _deserialize_object
(RayMainPool pid=671, ip=<IP_ADDRESS>)     message = deserialize(*value.message)
(RayMainPool pid=671, ip=<IP_ADDRESS>)   File "/home/ray/anaconda3/lib/python3.7/site-packages/mars/serialization/core.py", line 423, in deserialize
(RayMainPool pid=671, ip=<IP_ADDRESS>)     gen_deserialized = _deserialize(*gen_to_deserial)
(RayMainPool pid=671, ip=<IP_ADDRESS>)   File "/home/ray/anaconda3/lib/python3.7/site-packages/mars/serialization/core.py", line 397, in _deserialize
(RayMainPool pid=671, ip=<IP_ADDRESS>)     serializer = _deserializers[serializer_name]
(RayMainPool pid=671, ip=<IP_ADDRESS>) KeyError: 'pickle'
Traceback (most recent call last):
File "/home/ray/anaconda3/lib/python3.7/site-packages/mars/deploy/oscar/ray.py", line 320, in new_cluster
await cluster.start()
File "/home/ray/anaconda3/lib/python3.7/site-packages/mars/deploy/oscar/ray.py", line 481, in start
NodeRole.WORKER, self.supervisor_address, self.supervisor_address
File "/home/ray/anaconda3/lib/python3.7/site-packages/mars/deploy/oscar/ray.py", line 95, in create
address=lookup_address,
File "/home/ray/anaconda3/lib/python3.7/site-packages/mars/oscar/api.py", line 27, in create_actor
return await ctx.create_actor(actor_cls, *args, uid=uid, address=address, **kwargs)
File "/home/ray/anaconda3/lib/python3.7/site-packages/mars/oscar/backends/context.py", line 105, in create_actor
result = await self._wait(future, address, create_actor_message)
File "/home/ray/anaconda3/lib/python3.7/site-packages/mars/oscar/backends/context.py", line 83, in _wait
return await future
File "/home/ray/anaconda3/lib/python3.7/site-packages/mars/oscar/backends/context.py", line 74, in _wait
await asyncio.shield(future)
File "/home/ray/anaconda3/lib/python3.7/site-packages/mars/oscar/backends/core.py", line 49, in _listen
message: _MessageBase = await client.recv()
File "/home/ray/anaconda3/lib/python3.7/site-packages/mars/oscar/backends/communication/base.py", line 262, in recv
return await self.channel.recv()
File "/home/ray/anaconda3/lib/python3.7/site-packages/mars/oscar/backends/ray/communication.py", line 196, in recv
result = await object_ref
ray.exceptions.RayTaskError: ray::RayMainPool.on_ray_recv() (pid=671, ip=<IP_ADDRESS>, repr=<mars.oscar.backends.ray.pool.RayMainPool object at 0x7f1565adcdc0>)
At least one of the input arguments for this task could not be computed:
ray.exceptions.RaySystemError: System error: 'pickle'
traceback: Traceback (most recent call last):
File "/home/ray/anaconda3/lib/python3.7/site-packages/ray/serialization.py", line 281, in deserialize_objects
obj = self._deserialize_object(data, metadata, object_ref)
File "/home/ray/anaconda3/lib/python3.7/site-packages/mars/oscar/backends/ray/communication.py", line 78, in _deserialize_object
message = deserialize(*value.message)
File "/home/ray/anaconda3/lib/python3.7/site-packages/mars/serialization/core.py", line 423, in deserialize
gen_deserialized = _deserialize(*gen_to_deserial)
File "/home/ray/anaconda3/lib/python3.7/site-packages/mars/serialization/core.py", line 397, in _deserialize
serializer = _deserializers[serializer_name]
KeyError: 'pickle'

The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "", line 1, in
File "/home/ray/anaconda3/lib/python3.7/site-packages/mars/deploy/oscar/ray.py", line 363, in new_ray_session
client = new_cluster_in_ray(**new_cluster_kwargs)
File "/home/ray/anaconda3/lib/python3.7/site-packages/mars/deploy/oscar/ray.py", line 335, in new_cluster_in_ray
client = fut.result()
File "/home/ray/anaconda3/lib/python3.7/concurrent/futures/_base.py", line 435, in result
return self.__get_result()
File "/home/ray/anaconda3/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
raise self._exception
File "/home/ray/anaconda3/lib/python3.7/site-packages/mars/deploy/oscar/ray.py", line 327, in new_cluster
raise stop_ex from ex
File "/home/ray/anaconda3/lib/python3.7/site-packages/mars/deploy/oscar/ray.py", line 325, in new_cluster
await cluster.stop()
File "/home/ray/anaconda3/lib/python3.7/site-packages/mars/deploy/oscar/ray.py", line 519, in stop
await stop_supervisor(self.supervisor_address, self._config)
File "/home/ray/anaconda3/lib/python3.7/site-packages/mars/deploy/oscar/service.py", line 66, in stop_supervisor
await stop_services(NodeRole.SUPERVISOR, address=address, config=config)
File "/home/ray/anaconda3/lib/python3.7/site-packages/mars/services/core.py", line 185, in stop_services
await asyncio.gather(*[inst.stop() for inst in instances])
File "/home/ray/anaconda3/lib/python3.7/site-packages/mars/services/task/supervisor/service.py", line 52, in stop
uid=TaskConfigurationActor.default_uid(), address=self._address
File "/home/ray/anaconda3/lib/python3.7/site-packages/mars/oscar/api.py", line 37, in destroy_actor
return await ctx.destroy_actor(actor_ref)
File "/home/ray/anaconda3/lib/python3.7/site-packages/mars/oscar/backends/context.py", line 121, in destroy_actor
result = await self._wait(future, actor_ref.address, message)
File "/home/ray/anaconda3/lib/python3.7/site-packages/mars/oscar/backends/context.py", line 83, in _wait
return await future
File "/home/ray/anaconda3/lib/python3.7/site-packages/mars/oscar/backends/context.py", line 74, in _wait
await asyncio.shield(future)
File "/home/ray/anaconda3/lib/python3.7/site-packages/mars/oscar/backends/core.py", line 49, in _listen
message: _MessageBase = await client.recv()
File "/home/ray/anaconda3/lib/python3.7/site-packages/mars/oscar/backends/communication/base.py", line 262, in recv
return await self.channel.recv()
File "/home/ray/anaconda3/lib/python3.7/site-packages/mars/oscar/backends/ray/communication.py", line 196, in recv
result = await object_ref
ray.exceptions.RayTaskError: ray::RayMainPool.on_ray_recv() (pid=671, ip=<IP_ADDRESS>, repr=<mars.oscar.backends.ray.pool.RayMainPool object at 0x7f1565adcdc0>)
At least one of the input arguments for this task could not be computed:
ray.exceptions.RaySystemError: System error: 'pickle'
traceback: Traceback (most recent call last):
File "/home/ray/anaconda3/lib/python3.7/site-packages/ray/serialization.py", line 281, in deserialize_objects
obj = self._deserialize_object(data, metadata, object_ref)
File "/home/ray/anaconda3/lib/python3.7/site-packages/mars/oscar/backends/ray/communication.py", line 78, in _deserialize_object
message = deserialize(*value.message)
File "/home/ray/anaconda3/lib/python3.7/site-packages/mars/serialization/core.py", line 423, in deserialize
gen_deserialized = _deserialize(*gen_to_deserial)
File "/home/ray/anaconda3/lib/python3.7/site-packages/mars/serialization/core.py", line 397, in _deserialize
serializer = _deserializers[serializer_name]
KeyError: 'pickle'

(RayMainPool pid=671, ip=<IP_ADDRESS>) 2022-02-27 23:46:24,700	ERROR serialization.py:283 -- 'pickle'
(RayMainPool pid=671, ip=<IP_ADDRESS>) Traceback (most recent call last):
(RayMainPool pid=671, ip=<IP_ADDRESS>)   File "/home/ray/anaconda3/lib/python3.7/site-packages/ray/serialization.py", line 281, in deserialize_objects
(RayMainPool pid=671, ip=<IP_ADDRESS>)     obj = self._deserialize_object(data, metadata, object_ref)
(RayMainPool pid=671, ip=<IP_ADDRESS>)   File "/home/ray/anaconda3/lib/python3.7/site-packages/mars/oscar/backends/ray/communication.py", line 78, in _deserialize_object
(RayMainPool pid=671, ip=<IP_ADDRESS>)     message = deserialize(*value.message)
(RayMainPool pid=671, ip=<IP_ADDRESS>)   File "/home/ray/anaconda3/lib/python3.7/site-packages/mars/serialization/core.py", line 423, in deserialize
(RayMainPool pid=671, ip=<IP_ADDRESS>)     gen_deserialized = _deserialize(*gen_to_deserial)
(RayMainPool pid=671, ip=<IP_ADDRESS>)   File "/home/ray/anaconda3/lib/python3.7/site-packages/mars/serialization/core.py", line 397, in _deserialize
(RayMainPool pid=671, ip=<IP_ADDRESS>)     serializer = _deserializers[serializer_name]
(RayMainPool pid=671, ip=<IP_ADDRESS>) KeyError: 'pickle'
exit()

Minimized code to reproduce the error.

import ray
import mars
ray.init(address='ray://<IP_ADDRESS>:30772') # ray cluster wit h3 node  with ray version 1.9.2

import mars.tensor as mt
import mars.dataframe as md
session = mars.new_ray_session(worker_num=2, worker_mem=2 * 1024 ** 3)

Expected behavior
KeyError: 'pickle'
Additional context
Ray cluster docker is built with the following docker file:
ARG BASE_IMAGE=rayproject/ray:1.9.2
FROM ${BASE_IMAGE}
# Setup commands[Optional]
RUN sudo apt-get update \
    && sudo apt-get upgrade -y \
    && sudo apt-get install -y emacs vim\
    && sudo apt-get clean \
    && sudo rm -rf /var/lib/apt/lists/*

# ML packages[Optional]
RUN pip3 install  numpy==1.21.5 pandas==1.3.5 pymars==0.9.0b1 xgboost "xgboost_ray" lightgbm

@chaokunyang could you please take a look?

Hi @jyizheng，your code works on my mac machine without using docker and ray client. Have you tried the code without ray client? Seems it's related with ray client. Have you tried out without ray client?

@chaokunyang , thanks for the reply. We really want to run mars on a ray cluster.
I have tried the steps here: https://github.com/mars-project/mars#mars-on-ray
But I cannot run these lines of code successfully.  Can you ?


Add the hability to end session on the server but without redirecting to postLogoutRedirectUri

Currently, in our project, we have the config variables postLogoutRedirectUri and redirectUri, properly specified in our configuration file for the angular-oauth2-oidc, so when the user do a logout manually, he is redirected to our home page. However, in one scenario, we'd like to end the session in the OIDC server without having to redirect the user to home page. This should be possible by using the 'logOut' method with the customParameters parameter set as true ( oauthservice.logout(true) ). The issue is that when we do so, the library deletes all the auth info stored in the browser but does not end the session in the server because of that part of the code:
if (noRedirectToLogoutUrl) {
return;
}
I think the variable 'noRedirectToLogoutUrl' should be used when the method constructs the 'logoutUrl' in order to not add the 'post_logout_redirect_uri' parameter in the URL:
const postLogoutUrl = noRedirectToLogoutUrl ? '' : this.postLogoutRedirectUri ||
(this.redirectUriAsPostLogoutRedirectUriFallback && this.redirectUri) ||
'';
As an alternative I had to remove the postLogoutRedirectUri paraemeter from my config file, add the parameter redirectUriAsPostLogoutRedirectUriFallback as false and include always in my code the 'post_logout_redirect_uri' as a custom parameter for the logout method. I think this is not best way to handle this issue.
Thanks in advance.

I am late to the party but if I understand your post correctly you are not fully grasping the difference between postLogoutRedirectUri and logoutUri (as in noRedirectToLogoutUrl). The former tells the authorization server where to redirect after logout while the latter is the authorization server itself.
Logout is done by calling proper logout url from authorization server.
For your need of doing full logout while staying on page, you could maybe use an iframe for the logout or open your angular logout route in another tab. I don’t really understand the purpose though.


Introducing Chris Unleashed

This is the comment thread for Introducing Chris Unleashed.

Testing comments for the first time.

!


Build qt5.9 webkit

What is the plan for the webkit. I have some issues with webengine, so it would be nice to have an option to install webkit for qt5.9.

There are no plans, sorry.

There is an archived feedstock here: https://github.com/conda-forge/qtwebkit-feedstock if you want to get it going again

thanks @gillins . I will have a look.

If you want make that work and want to send a PR let me know so I can restore it from the archive. We cannot maintain it but it is available to new maintainers.

FreeCAD now runs with qtwebengine. So qtwebkit is not needed anymore.


DOC: Added enhancement entry for to_datetime

Supplementary Patch for pull request #11470
(Issues #11276, #11745)

this needs to go on the original pr


Getting error "Task <...> got Future <...> attached to a different loop"

Configuration

Python version : 3.9.6
FastAPI version : 0.68.0
FastAPI Users version : 6.1.1

Steps to reproduce
The following sample reproduces the above issue.
sample_v2.zip
Describe the bug
I've found an interesting issue, not really sure if it's related to fastapi_users but I think so.
For development purposes, I have two options to launch my app:

Running the command "uvicorn main:app --reload"
Putting the python code below in my main.py and calling "python main.py"

import uvicorn
if __name__ == "__main__":
    uvicorn.run(
        "main:app", host="<IP_ADDRESS>", port=8000, reload=settings.DEBUG_MODE
    )

Option 2 is good for me because it allows me to debug the backend inside vscode.
If I run the full example code available in the docs https://frankie567.github.io/fastapi-users/configuration/full-example.html everything is fine, using option 1 or 2.
Now, if I put fastapi_users part of the code in a different file and import this code in the main fastapi file like this app.include_router(auth.router), Option 1 runs fine, but Option 2 gives me the following error during login
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "c:\Users\scherzo\Downloads\vida-swipe-v2\backend\.venv\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 396, in run_asgi
    result = await app(self.scope, self.receive, self.send)
  File "c:\Users\scherzo\Downloads\vida-swipe-v2\backend\.venv\lib\site-packages\uvicorn\middleware\proxy_headers.py", line 45, in __call__
    return await self.app(scope, receive, send)
  File "c:\Users\scherzo\Downloads\vida-swipe-v2\backend\.venv\lib\site-packages\fastapi\applications.py", line 208, in __call__
    await super().__call__(scope, receive, send)
  File "c:\Users\scherzo\Downloads\vida-swipe-v2\backend\.venv\lib\site-packages\starlette\applications.py", line 112, in __call__
    await self.middleware_stack(scope, receive, send)
  File "c:\Users\scherzo\Downloads\vida-swipe-v2\backend\.venv\lib\site-packages\starlette\middleware\errors.py", line 181, in __call__
    raise exc from None
  File "c:\Users\scherzo\Downloads\vida-swipe-v2\backend\.venv\lib\site-packages\starlette\middleware\errors.py", line 159, in __call__
    await self.app(scope, receive, _send)
  File "c:\Users\scherzo\Downloads\vida-swipe-v2\backend\.venv\lib\site-packages\starlette\middleware\cors.py", line 78, in __call__
    await self.app(scope, receive, send)
  File "c:\Users\scherzo\Downloads\vida-swipe-v2\backend\.venv\lib\site-packages\starlette\exceptions.py", line 82, in __call__
    raise exc from None
  File "c:\Users\scherzo\Downloads\vida-swipe-v2\backend\.venv\lib\site-packages\starlette\exceptions.py", line 71, in __call__
    await self.app(scope, receive, sender)
  File "c:\Users\scherzo\Downloads\vida-swipe-v2\backend\.venv\lib\site-packages\starlette\routing.py", line 580, in __call__
    await route.handle(scope, receive, send)
  File "c:\Users\scherzo\Downloads\vida-swipe-v2\backend\.venv\lib\site-packages\starlette\routing.py", line 241, in handle
    await self.app(scope, receive, send)
  File "c:\Users\scherzo\Downloads\vida-swipe-v2\backend\.venv\lib\site-packages\starlette\routing.py", line 52, in app
    response = await func(request)
  File "c:\Users\scherzo\Downloads\vida-swipe-v2\backend\.venv\lib\site-packages\fastapi\routing.py", line 219, in app
    raw_response = await run_endpoint_function(
  File "c:\Users\scherzo\Downloads\vida-swipe-v2\backend\.venv\lib\site-packages\fastapi\routing.py", line 152, in run_endpoint_function
    return await dependant.call(**values)
  File "c:\Users\scherzo\Downloads\vida-swipe-v2\backend\.venv\lib\site-packages\fastapi_users\router\auth.py", line 26, in login
    user = await user_db.authenticate(credentials)
  File "c:\Users\scherzo\Downloads\vida-swipe-v2\backend\.venv\lib\site-packages\fastapi_users\db\base.py", line 54, in authenticate
    user = await self.get_by_email(credentials.username)
  File "c:\Users\scherzo\Downloads\vida-swipe-v2\backend\.venv\lib\site-packages\fastapi_users\db\mongodb.py", line 49, in get_by_email
    user = await self.collection.find_one(
RuntimeError: Task <Task pending name='Task-4' coro=<RequestResponseCycle.run_asgi() running at c:\Users\scherzo\Downloads\vida-swipe-v2\backend\.venv\lib\site-packages\uvicorn\protocols\http\h11_impl.py:396> cb=[set.discard()]> got Future <Future pending cb=[_chain_future.<locals>._call_check_cancel() at C:\Program Files\Python39\lib\asyncio\futures.py:384]> attached to a different loop

The following sample reproduces the above issue.
sample_v2.zip

Hi there!
Sorry for the delayed response!
It actually happens because Uvicorn tends to instantiate its own event loop. I gave explanations about this in the following discussion: https://github.com/frankie567/fastapi-users/discussions/663#discussioncomment-1025558
In a nutshell, put your FastAPI app instantiation and the Uvicorn invokation in two separate files.


Port SslHandler from ByteBuf to Buffer

Motivation:
We are replacing ByteBuf with Buffer in Netty.
Modification:
SslHandler has been changed to use Buffer instead of ByteBuf.
This in turn caused many other changes that were required to make this transition.
Result:
SslHandler now uses Buffer instead of ByteBuf.
This PR is a draft because there are many tests that are not passing yet, with this change.

Also, there are probably a good number of changes in here that we want to extract into separate PRs.

Mission accomplished? @chrisvest :D

Some of the changes here have been extracted into #12248, so I want to rebase this on that before merging.

FYI now playing with SslHandlerEchoBenchmark :)

Please @chrisvest can you share your JMH parameters so I can run the same benchmark as you?
(For both netty 4.1 and netty 5 versions)?
For netty 5 I'm using the IDEA JMH plugin and running it with
io.netty5.microbench.handler.ssl.SslHandlerEchoBenchmark.* -p bufferType=HEAP -p cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 -p messageSize=128 -p numWrites=10 -p sslProvider=JDK -prof "async:output=flamegraph;dir=/tmp;libPath=/home/forked_franz/IdeaProjects/async-profiler/build/libasyncProfiler.so" -prof perfasm --jvmArgs="-server -dsa -da -ea:io.netty5... -XX:+HeapDumpOnOutOfMemoryError -XX:+UnlockDiagnosticVMOptions -XX:+DebugNonSafepoints -Dio.netty5.leakDetection.level=disabled -Dio.netty5.buffer.leakDetectionEnabled=false -Dio.netty5.buffer.lifecycleTracingEnabled=false -Xms768m -Xmx768m -XX:MaxDirectMemorySize=768m -XX:BiasedLockingStartupDelay=0 -Djmh.executor=CUSTOM -Djmh.executor.class=io.netty5.microbench.util.AbstractMicrobenchmark$HarnessExecutor"

with # VM version: JDK <IP_ADDRESS>, OpenJDK 64-Bit Server VM, <IP_ADDRESS>+1
and I've noticed that it uses ByteBufferMemoryManager ...is it correct?

sent https://github.com/netty/netty/pull/12276 to fix https://github.com/netty/netty/pull/12221#issuecomment-1087513252

Ran SslHandlerEchoBenchmark with JDK 11.0.7 and #12275, #12276, #12277, and this is what I got:
Benchmark                           (bufferType)                               (cipher)  (messageSize)  (numWrites)  (sslProvider)   Mode  Cnt       Score      Error  Units
SslHandlerEchoBenchmark.wrapUnwrap          HEAP  TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256            128           10            JDK  thrpt   10  138318,159 ± 4456,938  ops/s

This is faster than my 4.1 measurement of 122991.137 ±   974.148.
With JDK 17 I get 149713,981 ± 8081,987, while on 4.1 I see 227256,425 ± 12608,357, so it looks like JDK 17 is helping 4.1 more than 5.

And @chrisvest , the benchmark is supposed to use nio buffers instead unsafe? Just to be sure we are running the same test

And @chrisvest , the benchmark is supposed to use nio buffers instead unsafe? Just to be sure we are running the same test

That looks like a regression in main branch.

and I've noticed that it uses ByteBufferMemoryManager ...is it correct?

That was due to the regression I mentioned. Should be fixed by #12278. That said, I was also using that memory manager, so our numbers are aligned in that sense.

Recycling entries on PoolThreadCache::allocate
    private UntetheredMemory allocate(MemoryRegionCache cache, int size) {
        if (cache == null) {
            // no cache found so just return false here
            return null;
        }
        UntetheredMemory allocated = cache.allocate(size, this);
        if (++allocations >= freeSweepAllocationThreshold) {
            allocations = 0;
            trim();
        }
        return allocated;
    }

it's using JCTools offering with multiple offered threads ie using a cas operation
that's pretty expensive for what's supposed to be a thread-local allocation cache: we can maybe specialize
ObjectPool (and the inner Recycler) to just use a newChunkedSpscQueue (instead of newChunkedMpscQueue), not yet exposed on PlatformDependent if the recycling cannot happen from multiple different threads (until a quiescient state is reached -> on the cache disposal its thread local reference onRemove).
This should alleviate the cas cost, because the spsc variant is not using any full memory barrier on x86 (and I believe on ARM as well).

More comments later

I got a different proposal for https://github.com/netty/netty/pull/12221#issuecomment-1088485274, let me send a PR

@chrisvest @normanmaurer  just sent https://github.com/netty/netty/pull/12284

And I suppose the 4.1 results on the same benchmark was using unsafe instead of NIO buffer(s), am I correct?

Just checked. Yes, it has access to Unsafe on both Java 11 and 17. On 17, 4.1 does not have access to the DirectByteBuffer constructor that doesn't take a Cleaner, but that's not stopping 4.1 from using Unsafe.

but that's not stopping 4.1 from using Unsafe.

That means that is not an Apple 2 Apple comparison here, but still let me know how things change when both run with Unsafe or both without using unsafe buffer(s).
I suppose the most of the cost of the new version will be because of the cleaner usage (masking phantom ref under the hood) now vs nothing before -> for pooled buffers

@franz1981 The two PRs you mentioned in https://github.com/netty/netty/pull/12221#pullrequestreview-930499050 have been brought in via rebase.
@normanmaurer I've addressed your comments, punting 1) removal of implicit buffer conversion, and 2) adding reachability fences to Buffer.forEach, to follow-up PRs.

This is currently generating failures like the following, which needs to be investigated. Possibly being surfaced by the fixes to the PoolThreadCache recently.
2022-04-06T06:09:02.3512630Z 06:09:02.350 [testsuite-epoll-worker-7-2] WARN  i.n.t.t.socket.SocketSslEchoTest - Unexpected exception from the client side:
2022-04-06T06:09:02.3513514Z io.netty5.handler.codec.DecoderException: java.lang.IllegalArgumentException: newPosition > limit: (777728 > 749384)
2022-04-06T06:09:02.3514448Z 	at io.netty5.handler.codec.ByteToMessageDecoderForBuffer.callDecode(ByteToMessageDecoderForBuffer.java:353)
2022-04-06T06:09:02.3516493Z 	at io.netty5.handler.codec.ByteToMessageDecoderForBuffer.channelInputClosed(ByteToMessageDecoderForBuffer.java:295)
2022-04-06T06:09:02.3517751Z 	at io.netty5.handler.codec.ByteToMessageDecoderForBuffer.channelInputClosed(ByteToMessageDecoderForBuffer.java:268)
2022-04-06T06:09:02.3518815Z 	at io.netty5.handler.codec.ByteToMessageDecoderForBuffer.channelInactive(ByteToMessageDecoderForBuffer.java:251)
2022-04-06T06:09:02.3520207Z 	at io.netty5.handler.ssl.SslHandler.channelInactive(SslHandler.java:919)
2022-04-06T06:09:02.3521046Z 	at io.netty5.channel.DefaultChannelHandlerContext.invokeChannelInactive(DefaultChannelHandlerContext.java:251)
2022-04-06T06:09:02.3522035Z 	at io.netty5.channel.DefaultChannelHandlerContext.findAndInvokeChannelInactive(DefaultChannelHandlerContext.java:246)
2022-04-06T06:09:02.3523147Z 	at io.netty5.channel.DefaultChannelHandlerContext.fireChannelInactive(DefaultChannelHandlerContext.java:233)
2022-04-06T06:09:02.3524000Z 	at io.netty5.channel.ChannelHandler.channelInactive(ChannelHandler.java:229)
2022-04-06T06:09:02.3524859Z 	at io.netty5.channel.DefaultChannelHandlerContext.invokeChannelInactive(DefaultChannelHandlerContext.java:251)
2022-04-06T06:09:02.3525791Z 	at io.netty5.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:806)
2022-04-06T06:09:02.3528542Z 	at io.netty5.channel.AbstractChannel$AbstractUnsafe.lambda$deregister$5(AbstractChannel.java:623)
2022-04-06T06:09:02.3529415Z 	at io.netty5.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:355)
2022-04-06T06:09:02.3530384Z 	at io.netty5.channel.SingleThreadEventLoop.run(SingleThreadEventLoop.java:195)
2022-04-06T06:09:02.3531364Z 	at io.netty5.util.concurrent.SingleThreadEventExecutor.lambda$doStartThread$4(SingleThreadEventExecutor.java:768)
2022-04-06T06:09:02.3532237Z 	at io.netty5.util.internal.ThreadExecutorMap.lambda$apply$1(ThreadExecutorMap.java:68)
2022-04-06T06:09:02.3533029Z 	at io.netty5.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
2022-04-06T06:09:02.3533710Z 	at java.base/java.lang.Thread.run(Thread.java:829)
2022-04-06T06:09:02.3534338Z Caused by: java.lang.IllegalArgumentException: newPosition > limit: (777728 > 749384)
2022-04-06T06:09:02.3535136Z 	at java.base/java.nio.Buffer.createPositionException(Buffer.java:318)
2022-04-06T06:09:02.3535792Z 	at java.base/java.nio.Buffer.position(Buffer.java:293)
2022-04-06T06:09:02.3536421Z 	at java.base/java.nio.ByteBuffer.position(ByteBuffer.java:1094)
2022-04-06T06:09:02.3537119Z 	at io.netty5.buffer.api.internal.Statics.bbsliceFallback(Statics.java:187)
2022-04-06T06:09:02.3537843Z 	at io.netty5.buffer.api.internal.Statics.bbslice(Statics.java:162)
2022-04-06T06:09:02.3538672Z 	at io.netty5.buffer.api.bytebuffer.ByteBufferMemoryManager.sliceMemory(ByteBufferMemoryManager.java:91)
2022-04-06T06:09:02.3539643Z 	at io.netty5.buffer.api.pool.PoolChunk$UntetheredChunkAllocation.memory(PoolChunk.java:586)
2022-04-06T06:09:02.3540440Z 	at io.netty5.buffer.api.pool.PooledBufferAllocator.allocate(PooledBufferAllocator.java:321)
2022-04-06T06:09:02.3541306Z 	at io.netty5.buffer.api.DefaultBufferAllocators$UncloseableBufferAllocator.allocate(DefaultBufferAllocators.java:129)
2022-04-06T06:09:02.3542082Z 	at io.netty5.handler.ssl.SslHandler.allocate(SslHandler.java:2081)
2022-04-06T06:09:02.3542742Z 	at io.netty5.handler.ssl.SslHandler.unwrap(SslHandler.java:1193)
2022-04-06T06:09:02.3543432Z 	at io.netty5.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1080)
2022-04-06T06:09:02.3544118Z 	at io.netty5.handler.ssl.SslHandler.decode(SslHandler.java:1140)
2022-04-06T06:09:02.3545011Z 	at io.netty5.handler.codec.ByteToMessageDecoderForBuffer.decodeRemovalReentryProtection(ByteToMessageDecoderForBuffer.java:379)
2022-04-06T06:09:02.3546035Z 	at io.netty5.handler.codec.ByteToMessageDecoderForBuffer.callDecode(ByteToMessageDecoderForBuffer.java:322)
2022-04-06T06:09:02.3546725Z 	... 17 common frames omitted

The allocator issue should be fixed by #12291

@chrisvest still concerned about numbers?
have you tried the latest change removing the volatile barrier on ArcDrop creation?

@franz1981 I'm personally satisfied with the performance right now. I didn't try the ArcDrop changes.

Two classes of test failures have been looked into here. One only occurs on Windows, and is also not caused by these changes AFAICT. The other might have been fixed by the duplicate() call of the last commit, but is also probabilistic, and exists in the code prior to these changes (I believe these changes just make it more likely to occur due to making more use of the new Buffer API), and if it occurs again, we'll have more logging to hopefully narrow it down a bit.
Hence I'm merging this.


qoutes/search/ failing

Some quotes are failing to search in prod such as 'Hmm.' but work in dev.

Made min input to 8 to help Supabase with request