[BUG]usb-sniffer software cannot write firmware unless sudo in linux

In windows10(22H2 19045.3086),after driver loaded,
./usb_sniffer --mcu-sram usb_sniffer.bin cannot write firmware
got: Error: libusb_open(): LIBUSB_ERROR_NOT_SUPPORTED
with elevate(run as admin), still got same error

in linux,got: Error: libusb_open(): LIBUSB_ERROR_ACCESS
but sudo will work

For Windows - can you show how it appears in the Device Manager? Did you actually install the INF file?
Does it work after you program the EEPROM fro Linux?
For Linux, this is also expected. For it to work under normal user you need to add the file /etc/udev/rules.d/90-fx2.rules  with the following contents:
ATTRS{idVendor}=="04b4", ATTRS{idProduct}=="8613", MODE="0666"
Re-plug the device and it should get proper permissions. You may need to add one more line for 6666:6620 with the same mode for normal operation.
I'll document this a bit later and provide full files.

I've added udev rules file https://github.com/ataradov/usb-sniffer/blob/main/bin/90-usb-sniffer.rules . Copy that into /etc/udev/rules.d/ and Linux permissions would be sorted out.
For the Windows side - this behavior means that INF file was not installed. It is necessary for the blank device only. If you program EERPOM first time under Linux, device should be  recognized by Windows automatically.

I've added udev rules file https://github.com/ataradov/usb-sniffer/blob/main/bin/90-usb-sniffer.rules . Copy that into /etc/udev/rules.d/ and Linux permissions would be sorted out.
For the Windows side - this behavior means that INF file was not installed. It is necessary for the blank device only. If you program EERPOM first time under Linux, device should be recognized by Windows automatically.

I found out that during the driver installation,there is a error

is your driver not signed?

This is not a "driver" really, it is a dummy INF file that installs a standard generic WinUSB driver from Windows. This INF file was created automatically by Zadig tool. I have no clue what is necessary for Windows to consider it "signed". I've included a catalog file (blank_fx2lp.cat), which  I think related to that.
This INF file installed fine on my test Windows 10 PC.
You can download Zadig tool (https://zadig.akeo.ie/) and make your own "drivers". They may work better. Select your device from the list and select a WinUSB driver.

This is not a "driver" really, it is a dummy INF file that installs a standard generic WinUSB driver from Windows. This INF file was created automatically by Zadig tool. I have no clue what is necessary for Windows to consider it "signed". I've included a catalog file (blank_fx2lp.cat), which  I think related to that.
This INF file installed fine on my test Windows 10 PC.
You can download Zadig tool (https://zadig.akeo.ie/) and make your own "drivers". They may work better. Select your device from the list and select a WinUSB driver.

if the driver is from zadig,then let user use zadig to install,it will work

This is not a "driver" really, it is a dummy INF file that installs a standard generic WinUSB driver from Windows. This INF file was created automatically by Zadig tool. I have no clue what is necessary for Windows to consider it "signed". I've included a catalog file (blank_fx2lp.cat), which I think related to that.
This INF file installed fine on my test Windows 10 PC.
You can download Zadig tool (https://zadig.akeo.ie/) and make your own "drivers". They may work better. Select your device from the list and select a WinUSB driver.

and yes,problem because driver is not signed

I'm not sure what the difference is, I tested that on my Win10 PC and it worked. I'm not sure is there is Pro vs Home or something like that. I don't use Windows.
I though catalog file contained the signature (as confirmed by this https://learn.microsoft.com/en-us/windows-hardware/drivers/install/catalog-files). Do you have it present in the same directory as INF file?

I'm not sure what the difference is, I tested that on my Win10 PC and it worked. I'm not sure is there is Pro vs Home or something like that. I don't use Windows.
I though catalog file contained the signature (as confirmed by this https://learn.microsoft.com/en-us/windows-hardware/drivers/install/catalog-files). Do you have it present in the same directory as INF file?

I just using usb-sniffer\bin folder as the driver folder
this is the fix and you might do it before,and mine might just been disable by windows again
https://thegeekpage.com/windows-encountered-a-problem-installing-the-driver-software/

It might have been enabled by default. I don't remember doing anything like this before. The Windows PC I use for testing was purchased just for testing of stuff like this because I did not have any Windows PCs. I have not used it for 5+ years and updated it to the latest version just to test this driver. I also used full bin directory and it just silently worked.
If Zadig does not supply sufficiently signed drivers, there is not much I can do.
This step is needed just for the initial programming. Once the real firmware runs, it tells the OS to install the same WinUSB driver though the MSOSv2 descriptor requested over USB on the first connection.
This also highlights the issue with this signing thing - any USB device that you plug into the PC can install that driver automatically without user involvement at all.

I just had the same issue on Windows 11, I could not install the initial driver with the supplied INF file. When using Zadig directly it works, it might be worth making this note in the instructions as I had to search the tickets to get to this solution.

On Tue, Sep 10, 2024 at 01:12:50PM -0700, Notedop wrote:

it might be worth making this note in the instructions
as I had to search the tickets to get to this solution.

"Patches welcome" and do know that a poor merge request
is much better as no merge request


Dex is not working unless a new release will be released

Hi everyone 👋 !
I am using the current operator release v0.0.14 configuring the authentication with an external dex.
I saw that with this change https://github.com/argoproj-labs/argocd-operator/blob/a5171dbe936341924dcacfba7f1aba6a109ccf7e/pkg/controller/argocd/util.go#L193-L196 everything could potentially work, but this requires a new release in order to can use the code accordingly with the latest changes.
Can anyone give some visibility on when will be available a new release?
Thanks in advance to everybody!

Yea, let's get the next release v0.0.15 out.   Hopefully, we can get it out in a coupe weeks.

This issue seems obsolete. we are currently at release v0.3.0. @lzecca78 Can I close this issue ?


Add support for NLog

Hello.
log4net is quite good. But NLog is ...quite good too !
A simple PR to add support for NLog in BrowserLog.
A demo projet has been created for NLog. Demo projet for log4net has been renamed.
If the PR is accepted. I'll add documentation. (Or I can add it directly to the PR ...As you want).
Warn : I didn't touch to AssemblyInfo files. I let you change or add description. Or just tell me what to do :-)
[assembly: AssemblyDescription("log4net appender based on HTML5 SSE")]

:sob:  Build seems broken because of a timeout of 2 sec during a unit test Should_parse_lines_till_cancelled. Any idea ?


Upcoming Java Lessons

We need help figuring out what the upcoming Java lessons are going to be on. What do you think we should cover next?
If you have a suggestion for a new topic comment it below, and we will update the poll.

Java Inheritance

Java Regular Expressions

Java Threads

Swing


Install-Package command doesn't have the parameter name -version

Install-Package command doesn't have the parameter name -version, guess should be -RequiredVersion.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 2e9e1a8a-6284-a230-061e-7cb27258571a
Version Independent ID: 43390e30-04d6-6d4e-018e-21419dbb11e8
Content: Get started with the WebJobs SDK - Azure App Service
Content Source: articles/app-service/webjobs-sdk-get-started.md
Service: app-service
GitHub Login: @ggailey777
Microsoft Alias: glenga

@v-tawe , Thanks for raising the issue. We will investigate and update you.

@v-tawe , There’s some confusion here between using the Package Manager console (which is technically PowerShell) in VS and the PowerShell commandlet Install-Package in plain-ol’ PowerShell.
The article does state using the Package Manager console, which is probably what most VS users do.
We could change the code block from powershell to ps, which is what the Package Manager docs use.
Please let me know if you need further assistance.

@v-tawe do you think seeing ps in the example command have made it more clear as a Package Manager command?

Yeah, that will make sense.

@v-tawe , Just wanted to understand if we can go ahead and close the issue for now?

@v-tawe , I have not heard back from you in a couple of days. i will go ahead and close the issue. Feel free to reopen the issue if you need further assistance.


Added caching

Port of symfony-cmf/RoutingAutoBundle#78

Why close this? Would we do this in a different way now?

This PR has been open for very long and I can no longer follow the logic in here. Maybe I'll write caching again in some other PR.


Design specs for text input question type

Description
Design specs for our text input type that include USWDS component/styles and spacing guidelines. These designs will depict how the text input question type renders on desktop, mobile and other responsive screen types. This is a part of the thymeleaf/htmx front end overhaul work.
Current implementation:
Text question- standard screen
Text question - medium screen
Text question - mobile screen
Text question - standard screen, error state
Text question - medium screen, error state
Text question - mobile screen, error state
Requirements:

[ ] Responsive screen sizes (mobile, medium, standard)
[ ] Padding and spacing guidelines
[ ] Max and min width
[ ] Error states for different screen types

Figma links:

Text input question type
error state

@greg-hyde I've added initial figma links. Please let me know if you have questions about any of the design specs.


PP-6627 Patch calling open Connector route with no accounts

Patch this behaviour and avoid an unnecessary call to the backend by
returning nothing if no accounts will have valid permissions.
The default behaviour in newer backend routes is to require a gateway
account id to be passed in to any resource, passing in nothing key=[]
or
in the case of comma separated values key= will either filter the data
for no account ids (resulting in nothing returned []) OR return 400
requiring non empty values.
In the case of the frontend accounts route being called in this utility,
passing no accounts will treat the route as if there is no filter and
return accounts.
This behaviour will be patched in the backend as part of this ticket.

Superseded by https://github.com/alphagov/pay-selfservice/pull/2039


All LoRA weights are 0 in InternLM-XComposer-VL?

In paper,  InternLM-XComposer-VL is trained by multi-task training in StageB.
It seems training LoRA, but the weights are 0 when loading the released model.

The parameters in LoRA are merged into the original linear layers, so that the remaining LoRA layers have 0 parameters.


Reuse resize observer

Closes https://github.com/parsable/react-truncate-markup/issues/26
Current behavior:

<TruncateMarkup /> is rendered
resize observer is attached on the DOM element, first event is ignored for such element, because it just rendered
whenever the DOM tree is updated, <TruncateMarkup /> is re-rendered by React , a new resize observer is attached although it could be the very same DOM element. Again, first event is ignored, but this time the element has already been rendered before
This accidentally improved the performance, because some events got skipped - instead of proper event throttling. Additionally, it caused the issue https://github.com/parsable/react-truncate-markup/issues/26

Bahaviour with this PR

<TruncateMarkup /> is rendered
resize observer is attached on the DOM element, first event is ignored for such element, because it just rendered
whenever the DOM tree is updated, <TruncateMarkup /> is re-rendered by React, the observer stays the same if the DOM elements stays the same. If a new DOM element is created, React cleans after itself and it first sets the ref for element to null which, in effect, cleans up the resize observer. For the new DOM element, new resize observer is created and again, first event is ignored because it just rendered
The way the demo page is written, it had some performance issues. The example mentioned in https://github.com/parsable/react-truncate-markup/issues/26   re-renderes the whole demo page with each resize event (well, with optimized React state updates batches). As a result, I also extracted that example to a separate component

FYI, I have approximate plan how to remove the UNSAFE_componentWillReceiveProps and this PR is also a little step towards that goal.

Good catch @stropho, thanks for the PR!


check request bundles exist in matrix service db

This PR accomplishes #221. We will now return a failure in v0 driver if the request bundles do not all exist within our database (specifically the analysis table)

Do you think we should do anything in the v1 api too? There's an equivalent situation with a query like {"op": "in", "field": "dss_bundle_fqid", "value": ["fake-bundle-1", "fake-bundle-2"]}

Looks good to me. +1 to Marcus' suggestion - I think it makes sense to deliver the same experience for anyone migrating from v0 to v1


chore: use accountId from websocket messages, delete unused selector and tests

Description

draft - requires https://github.com/shapeshift/lib/pull/991
Notice

[x] Have you followed the guidelines in our Contributing guide?
[x] Have you checked to ensure there aren't other open Pull Requests for the same update/change?

Pull Request Type

[ ] :bug: Bug fix (Non-breaking Change: Fixes an issue)
[x] :hammer_and_wrench: Chore (Non-breaking Change: Doc updates, pkg upgrades, typos, etc..)
[ ] :nail_care: New Feature (Breaking/Non-breaking Change)

Issue (if applicable)

n/a - cleanup discovered during multi account
Risk

we break incoming tx's over websockets

Testing

do a transaction on an evm chain, a utxo chain, and a cosmossdk based chain. ensure they all come
in over websockets

Screenshots (if applicable)

nothing user facing

closing per discussion with @asamere @kaladinlight about more elegant approach


merge from arlac77/template-arlac77-github,arlac77/template-esm-only

.travis.yml

chore(travis): remove true (jobs.include.skip_cleanup)

:tada: This PR is included in version 1.1.2 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Chromium BSU game

Motivation for this change
Closes #115993
And provides a classic Invaders game! PEW PEW PEW
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[x] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 116460 at aa450866 run on aarch64-linux 1

2 packages marked as broken and skipped:

k3d
yoda-with-root

16 packages skipped due to time constraints:

fastnlo
hepmc3
kodi (kodiPackages.kodi)
kodi-gbm
kodi-wayland
kodiPackages.inputstream-adaptive
kodiPackages.joystick
kodiPackages.pvr-hdhomerun
kodiPackages.pvr-hts
kodiPackages.pvr-iptvsimple
...

9 packages built successfully:

chromium-bsu
ftgl
gource
logstalgia
megaglest
pianobooster
python38Packages.pyftgl
python39Packages.pyftgl
zaz

This is a semi-automatic executed nixpkgs-review with nixpkgs-review-checks extension. It is checked by a human on a best effort basis and does not build all packages (e.g. lumo, tensorflow or pytorch).
If you have any questions or problems please reach out to SuperSandro2000 on IRC.
Result of nixpkgs-review pr 116460 run on x86_64-linux 1

1 package marked as broken and skipped:

yoda-with-root

1 package failed to build and already failed to build on hydra master:

sladeUnstable: log was empty

31 packages built:

fastnlo
ftgl
glpng
gource
haskellPackages.FTGL
hepmc3
k3d
kodi (kodiPackages.kodi)
kodi-gbm
kodi-wayland
kodiPackages.inputstream-adaptive
kodiPackages.joystick
kodiPackages.kodi-platform
kodiPackages.pvr-hdhomerun
kodiPackages.pvr-hts
kodiPackages.pvr-iptvsimple
kodiPackages.steam-controller
kodiPackages.vfs-libarchive
kodiPackages.vfs-sftp
logstalgia
megaglest
pianobooster
python38Packages.hepmc3
python38Packages.pyftgl
python39Packages.hepmc3
python39Packages.pyftgl
root
setbfree
slade
x42-plugins
zaz

The following issues got detected with the above build packages.
Please fix at least the ones listed with your changed packages:

fastnlo:

warning: unclear-gpl
gpl3 is a deprecated license, check if project uses gpl3Plus or gpl3Only and change meta.license accordingly.
Near pkgs/development/libraries/physics/fastnlo/default.nix:28:5:
   |
28 |     license      = lib.licenses.gpl3;
   |     ^

See: https://github.com/jtojnar/nixpkgs-hammering/blob/master/explanations/unclear-gpl.md

I looked up the source code of fastnlo, it is published under licenses.gpl3Plus.


Edit calendar of an event after its creation

Even though Google Calendar does not allow this, Sunrise and other gcal clients enabled the ability to switch an event to another calendar after it's creation in case you made a mistake and didn't select the calendar you wanted.
I suppose this was done by deleting and recreating the event, and possibly re-sending updates if people were invited to the previous event ?

yep, it's an issue that already surfaced in our beta testers group on fb. Like you said, there is no "native" way of doing it via Google APIs: we need to delete the event and recreate it on the other calendar.
For now, we have no ETA for this though, PRs welcomed.


NaCl Fork Overhead

Currently, the Fork system call is taking a huge chunk of time that overshadows execution time for small programs and large pipelines. 90+% of fork is from the NaClCopyExecutionContext function which copies context between processes, roughly two-thirds of that function is composed of memmove's.
This seemingly is because we're losing the CopyOnWrite semantics that a native fork would have. Can we re-implement this to utilize CopyOnWrite?

@jesings mentioned to me: I think what I was talking about for the MAP_SHARED and MAP_ANONYMOUS solution, the memfd_create syscall, is handled nearly the same way in ram as a shared memory object except it doesn't create a file entry (except in /proc) associated with it, so it should be superior to a shmget system call
I think this should work better than the shared memory solution I was pushing in the meeting.  Regardless, we definitely shouldn't need to create a RAM FS backed file or something else that actually lives on a FS somewhere.  Let's try these out and see how they work!

So I've been looking at potential solutions to COW for fork, and I have a couple of pieces of input:

In order to do the MAP_ANONYMOUS/MAP_SHARED solution, I would need to have parts of process memory in some object referred to by an fd (say a memory object made by memfd_create). I'm not sure how to do that without copying the memory over which would defeat the purpose of COW
I think it might also be worth looking into a different solution, although I doubt it will work, involving the clone3 system call (which is used to implement pthreads but which we probably shouldn't work with directly, as we want lots of the things that pthreads offers us). It seems like there is a way (by specifying CLONE_THREAD and not specifying CLONE_VM in the flags) that you can create a new thread in the same process that has its memory as COW. Perhaps there is some way to access this from pthreads? If so, I couldn't find it.

Ah I just looked a little deeper into the man page for clone3, and it says
Since Linux 2.5.35, the flags mask must also include CLONE_SIGHAND if CLONE_THREAD is specified (and note that, since Linux 2.6.0, CLONE_SIGHAND also requires CLONE_VM to be included).

So it looks like we can't use CLONE_THREAD without CLONE_VM

I wanted to get some real performance stats for Jonathan so I hacked bash to get time statistics for the fork() call with real workloads.
Natively we're seeing fork take 80us on average
while with Lind we're seeing 160 ms
So Lind's fork is ~2000x slower currently.

Okay, good to know.  Would be interested in seeing a finer grained breakdown too, if possible.  I know memory copying is the main thing, but what else is there if that is left out?

I've been doing a lot of work with splice, and I've implemented it and have been profiling it since yesterday. I don't think it's doing copy-on-write given some of the in-kernel calls that it's making, although why is not exactly clear given a separate problem we've been having which I will describe below. I may be able to mess with splice a little bit more in order to get true copy-on-write or at least some other better performance, but this is how it stands for now. The current flame graph for lscatwc is:

and it does run significantly faster than the memcpy solution (a little over .6 seconds when compared to a little over .7).
The problem I noticed is that a suspicious amount of time is being taken up in NaClPatchAddr (which in this image is shown as NaClPa..., and is towards the top-left of the sel_ldr section). After inspecting the actual code of NaClPatchAddr, the reason became much clearer:
static INLINE void NaClPatchAddr(uintptr_t child_bits, uintptr_t parent_bits, uintptr_t *start, size_t size) {
  size_t cnt = size / sizeof(uintptr_t);
  for (size_t i = 0; i < cnt; i++) {
    if ((start[i] & ~UNTRUSTED_ADDR_MASK) != parent_bits) {
      continue;
    }
    NaClLog(1, "patching %p\n", (void *)start[i]);
    /* clear upper bits */
    start[i] &= UNTRUSTED_ADDR_MASK;
    /* add child prefix */
    start[i] |= child_bits;
    NaClLog(1, "new addr %p\n", (void *)start[i]);
  }
}

It iterates over the whole memory that was copied and writes to any aligned pointers with cage bits in the upper half, which seems like it would incur a stupid amount of overhead, invalidate a lot of the good parts of copy-on-write, and also could potentially corrupt some legitimate data. This solution in general really doesn't sit well with me, but I can't really think of a better one. I can likely improve performance by manually vectorizing this pointer comparison operation, however I don't think that's really worth it. It would help me a lot to hear what other people have to say about any of this.

I think I have an idea of the culprit for why splice isn't doing mmu manipulation like it says it should
The wikipedia page for splice begins with the following

splice() is a Linux-specific system call that moves data between a file descriptor and a pipe without a round trip to user space. The related system call vmsplice() moves or copies data between a pipe and user space. Ideally, splice and vmsplice work by remapping pages and do not actually copy any data, which may improve I/O performance. As linear addresses do not necessarily correspond to contiguous physical addresses, this may not be possible in all cases and on all hardware combinations.

The end is what I think the issue is:

"As linear addresses do not necessarily correspond to contiguous physical addresses, this may not be possible in all cases and on all hardware combinations."

We're mapping a file that ends up at over 2 GB long, so I have a very hard time believing that the memory we splice would all be consecutive in physical memory. Additionally, it definitely is running a lot faster than an actual memcpy solution (despite having to context switch for every megabyte copied), so I do think that it is actually mmu manipulating for a lot of the pages but not others, and that would point to the issue being related to non-contiguity.
The solution I can think of for now is to make a bunch of smaller files as backing for memory rather than a single huge file. This might also have benefits in terms of actually freeing resources, but further complicates implementation.

@moyix if you have any input on my theory about why splice isn't working, we'd love to hear it.

I tested out my theory today, and pretty soundly debunked it. When we back the memory with 300 smaller files rather than 1 larger file, the program's behavior is unchanged, and it also runs slightly slower because more time is spent in pipe locks because more splices and vmsplices are being made, with each copying less data.

Good news and bad news:
First, the bad news:
I have been unable to make splice particularly performant, even outside of lind. I have been doing some tests on my own machine with vmsplice/splice and comparing it to memcpy. Vmsplice/splice is pretty much always slower for the amounts of memory we're working with, regardless of whether or not it's actually doing copy-on-write. This is mostly because on my host machine the vmsplice/splice solution seems to take tons of memory while splicing, forcing me to swap a fair bit (or in the case of my host machine zswap to compressed RAM). Although the actual program time is less, it's not that much less, being very similar to the reduced overhead we see in the NaCl solution which is significant but not orders of magnitude different. The reason we don't see so much overhead on the server is likely because it has more RAM, but if we really want to have multiple lind processes running at once, it wouldn't be great to increase memory pressure so much.
Here is the splice performance (just for the actual program runtime)

As far as I can see, splice and vmsplice do seem to be almost entirely doing Copy-on-Write but are spending a ton of time on getting pages from the page cache or freeing them manually--likely because of the greatly increased memory pressure.
Memcpy did not suffer from those same memory pressure issues, and though the actual program time was longer, it ran in a much shorter amount of time because it didn't have to swap anything.
Here is the performance for memcpy (only program runtime):

But here's the good news:
I was digging through the kernel to see how splice was actually implemented and to look for how to set it to actually behave in a copy-on-write way. I then came upon a syscall that I had seen last week but had dismissed because of a line in its man page. This syscall is process_vm_writev (or process_vm_readv which is essentially the same syscall in the other direction). It is used to take memory from one process' address space and put it into another's. When I saw it last week, I assumed that it would be able to use the process' own pid to set it to copy from one part of its address space to the other, and was intrigued. However, when I saw in the man page's notes section that:

These system calls were designed to permit fast message passing by allowing messages to be exchanged with a single copy operation (rather than the double copy that would be required when using, for example, shared memory or pipes).

I believed that this function could not behave in a zero-copy manner. I did see it in the wikipedia page for zero-copy but didn't put 2 and 2 together until I found some functions called by it in the kernel, in the same paths that splice would ideally go down if it were doing copy-on-write. I added this in to the test on my own machine and it ran blisteringly fast compared to the other implementations: (still only including program time)

Luckily for me, its interface as a vectored uio syscall is very similar to that for vmsplice, so I get to feel like my work over the past couple weeks wasn't entirely wasted. I should be able to create a working implementation tomorrow that can test if this works well in Lind.

This sounds really promising. Awesome work!

New developments:
I implemented a fork using process_vm_writev, and it ran slower than the splice fork but faster than the memcpy fork. This was not at all the behavior I was expecting, so I spent a while gathering data about it. Here is a flame graph for it:

I noticed one crucial thing about this flame graph: a significant amount of time in the process_vm_writev was being taken up by shared memory operations.

I had seen shared memory operations in the splice fork, but I had assumed that they were the write operations to the memfd (which uses the shared memory write and read inside the kernel). This assumption was somewhat reinforced when I took a test of splice using a temp file (in an ext4 file system) as the backing memory last week:

The shared memory operations in a splice to a memfd were replaced by ext4 operations, making me think that nothing out of the ordinary was happening.
However the fact that these shared memory operations (i.e. shmem_fault) are happening in process_vm_writev leads me to suspect that there's something more to this.
One of the things I knew was happening in my splice implementation is that every entry in the page table was being copied, even those explicitly marked as map_shared. I was going to leave this optimization for later, but now I have reason to believe it may well be the cause of some significant slowdowns in process_vm_writev and splice performance. A fair amount of memory in the memory maps of the Lind tests I was running was taken up by shared memory mappings of files like this one: /dev/shm/google-nacl-shm--12671.1. These are being fully copied each time, which is not only incorrect behavior but is something I believe may be tanking performance of splice and  process_vm_writev. The primary implementation issue is that there is no information about what file each entry in the NaCl page table corresponds to. The fact that it's unlinked shouldn't matter if there is still an open file handle to it. I will talk with @rennergade about the best way to go about tackling this issue, and if @moyix or @JustinCappos have any thoughts on any of this, it would be great to hear from them.

I talked to @rennergade and we looked for where these shared memory regions are actually getting mapped. We both agreed that eventually we will need to attach the file descriptor to the vmmap entry so that forked processes can actually share MAP_SHARED segments like is specified by POSIX. However, I figured out a different solution for the mean time that improves fork performance. The NaClApp stores a text_shm member, which is a pointer to a NaCl shared memory descriptor, which is equivalent to the desc field of the /dev/shm/google-nacl-shm--12671.1 entry in the NaCl vmmap, so I compared each entry's desc field to this value, and if it was not equal, I skipped the copying and patching. This page table entry is 1 GB long and mostly consists of pages that are actually mapped as PROT_NONE on the host machine although the whole vmmap entry is listed as being mapped as PROT_RX. This means we were copying and patching a huge amount of unnecessary data. This also seems to be the source of the shmem_fault program time--I believe it was in changing the shared memory file's mappings from PROT_NONE to PROT_RX.
Here are the performance statistics:

The original memcpy version runs as follows:

@(2:476) $ time lind -t /bin/bash lscatwc
executing: [sel_ldr -a -t -- "/lib/glibc/runnable-ld.so" --library-path "/lib/glibc" /bin/bash lscatwc]
Lind v1.0-rc2
Opening file system...
done.
[914,953295936:10:16:08.933762] BYPASSING ALL ACL CHECKS
      5       5      34
[TimeInfo] NaCl main program time spent = 0.633700
[TimeInfo] NaCl initialization time spent = 0.092144
[TimeInfo] NaCl user program time spent = 0.541541
Persisting metadata: ...
Done persisting metadata.
Terminated
real    0m0.682s
user    0m0.444s
sys     0m0.270s

The new splice version runs as follows:

@(2:476) $ time lind -t /bin/bash lscatwc
executing: [sel_ldr -a -t -- "/lib/glibc/runnable-ld.so" --library-path "/lib/glibc" /bin/bash lscatwc]
Lind v1.0-rc2
Opening file system...
done.
[6416,610423872:10:36:03.540575] BYPASSING ALL ACL CHECKS
      5       5      34
[TimeInfo] NaCl main program time spent = 0.222044
[TimeInfo] NaCl initialization time spent = 0.091623
[TimeInfo] NaCl user program time spent = 0.130329
Persisting metadata: ...
Done persisting metadata.
Terminated
real    0m0.239s
user    0m0.223s
sys     0m0.047s

The new process_vm_write version runs as follows:

executing: [sel_ldr -a -t -- "/lib/glibc/runnable-ld.so" --library-path "/lib/glibc" /bin/bash lscatwc]
Lind v1.0-rc2
Opening file system...
done.
[7503,3873137728:10:48:20.610146] BYPASSING ALL ACL CHECKS
      5       5      34
[TimeInfo] NaCl main program time spent = 0.221416
[TimeInfo] NaCl initialization time spent = 0.092199
[TimeInfo] NaCl user program time spent = 0.129047
Persisting metadata: ...
Done persisting metadata.
Terminated
real    0m0.241s
user    0m0.230s
sys     0m0.038s

The splice and process_vm_write versions are roughly equivalent in performance, but I think the process_vm_write version is preferred because of its simpler API and its lack of a backing file, leading to resources being more easily freed. The downside is that splice is an older and more established syscall, and may be more portable, although I think neither are portable at all, both being listed as linux specific on their admittedly horrendously written manpages.
The flame graphs for the performance of the program with splice and process_vm_write are the following:

I'm really glad to have this done in time for the meeting tomorrow.

I have fixed the fork depth issue that I found in the new version of fork which was caused by an eventual stack overflow due to caching information during fork. I gathered some final performance statistics about the new CoW version of fork. It appears to be something like a 10 times speedup from the previous implementation, which is still unfortunately much slower than a native fork. However, the sources of this slowdown are at least much more straightforward now:

We are copying more pages (because NaCl uses more memory for bookkeeping and other things) than native does so vm_writev is significantly slower than the native fork. vm_writev takes slightly over 1/3 of the total time of fork
We reload the executable on each fork in order to obey NaCl's memory guarantees and pointer bounding. The function NaClLoadFileAslr takes slightly less than 1/3 of the total fork time, primarily in the function NaClValidateCode which calls ValidateChunkAMD64 which I believe (at first glance, I haven't gone that deep into analyzing it) iterates over the whole executable and ensures that it obeys NaCl's memory guarantees. We may be able to eliminate this cost by telling whether the executable has already been validated, but we may not.
Around 1/4 of the time is in NaClCopyDynamicText, some for actually copying dynamic text, but the vast majority for NaClPatchAddr, which we've discussed earlier. This time, as far as we can tell, would not be easy to remove or eliminate while maintaining NaCl's guarantees about address space isolation.

We may be able to reduce some of this time by copying the stack only up to the stack pointer, but that's unlikely to be an order of magnitude improvement and might even be negligible. So this is probably around the best we're going to get for fork for now. Looking forward to getting this implementation merged in!
Data:
Non-CoW:

CoW:

Non-CoW:
(ins)(2:1387) $ time lind -t /bin/bash -c "ls & echo test & dir"

executing: [sel_ldr -a -t -- "/lib/glibc/runnable-ld.so" --library-path "/lib/glibc" /bin/bash -c ls & echo test & dir]
Lind v1.0-rc2
Opening file system...
done.
[1864,221680704:09:55:07.233237] BYPASSING ALL ACL CHECKS
[ERROR][dispatch] Failed to look up 29 with error 29
/bin/bash: //.bashrc: Bad file descriptor
timing forkwrite msec: 188 ms
bin
dev
lib
timing forktiming forkwrite msec: 138 ms
test
timing forktiming forktiming forkwrite msec: 117 ms
bin  dev  lib
timing forktiming forktiming fork[TimeInfo] NaCl main program time spent = 0.650609
[TimeInfo] NaCl initialization time spent = 0.092339
[TimeInfo] NaCl user program time spent = 0.558254
Persisting metadata: ...
Done persisting metadata.
Terminated

real    0m0.667s
user    0m0.430s
sys     0m0.300s

CoW:
(ins)(2:1387) $ time lind -t /bin/bash -c "ls & echo test & dir"

executing: [sel_ldr -a -t -- "/lib/glibc/runnable-ld.so" --library-path "/lib/glibc" /bin/bash -c ls & echo test & dir]
Lind v1.0-rc2
Opening file system...
done.
[876,3300222016:09:53:11.509390] BYPASSING ALL ACL CHECKS
/bin/bash: //.bashrc: Bad file descriptor
timing forkwrite msec: 11 ms
timing forktiming forkwrite msec: 20 ms
test
bin
dev
lib
timing forktiming forktiming forkwrite msec: 17 ms
bin  dev  lib
timing forktiming forktiming fork[TimeInfo] NaCl main program time spent = 0.242784
[TimeInfo] NaCl initialization time spent = 0.092222
[TimeInfo] NaCl user program time spent = 0.150526
Persisting metadata: ...
Done persisting metadata.
Terminated

real    0m0.233s
user    0m0.226s
sys     0m0.062s

Non-CoW:
@(2:2289) $ lind -t /bin/bash -c 'i=0; while [[ "$i" -lt 100 ]]; do i=$(($i+1)); (factor $i) & done;'

executing: [sel_ldr -a -t -- "/lib/glibc/runnable-ld.so" --library-path "/lib/glibc" /bin/bash -c i=0; while [[ "$i" -lt 100 ]]; do i=$(($i+1)); (factor $i) & done;]
Lind v1.0-rc2
Opening file system...
done.
[2670,1328026688:10:45:28.543699] BYPASSING ALL ACL CHECKS
[ERROR][dispatch] Failed to look up 29 with error 29
/bin/bash: //.bashrc: Bad file descriptor
timing forkwrite msec: 193 ms
1:
timing forktiming forkwrite msec: 122 ms
2: 2
timing forktiming forktiming forkwrite msec: 120 ms
3: 3
...
[TimeInfo] NaCl main program time spent = 14.264195
[TimeInfo] NaCl initialization time spent = 0.093447
[TimeInfo] NaCl user program time spent = 14.170701
Persisting metadata: ...
Done persisting metadata.
Terminated

CoW:
@(2:2290) $ lind -t /bin/bash -c 'i=0; while [[ "$i" -lt 100 ]]; do i=$(($i+1)); (factor $i) & done;'

executing: [sel_ldr -a -t -- "/lib/glibc/runnable-ld.so" --library-path "/lib/glibc" /bin/bash -c i=0; while [[ "$i" -lt 100 ]]; do i=$(($i+1)); (factor $i) & done;]
Lind v1.0-rc2
Opening file system...
done.
[4268,3328635968:10:47:36.799348] BYPASSING ALL ACL CHECKS
/bin/bash: //.bashrc: Bad file descriptor
timing forkwrite msec: 11 ms
1:
timing forktiming forkwrite msec: 10 ms
2: 2
timing forktiming forktiming forkwrite msec: 9 ms
3: 3
...
[TimeInfo] NaCl main program time spent = 5.361019
[TimeInfo] NaCl initialization time spent = 0.093153
[TimeInfo] NaCl user program time spent = 5.267819
Persisting metadata: ...
Done persisting metadata.
Terminated

This is great!
The validation definitely seems like a place where we can make improvement, it seems silly to do that each fork. Other than that we'll probably have to live with some of this, its probably not possible to get super close to native performance.

With that PR, the huge first steps of this are done. I'm going to close this issue as a milestone, but reference it in a new issue to further our efforts.
Great job @jesings !


Commit vendor/ with LF-style line endings

What this PR does / why we need it:

Commits vendor/ with LF-style line endings to ensure vendor/ doesn't change every time glide install is run on Windows
No functional change.
acs-engine devs using Windows must set git config --global core.autocrlf input so git will automatically convert Windows-style CRLF endings to UNIX LF.

Which issue this PR fixes vendor/ was changing every time I ran make  prereqs on macOS. This will commit + Windows devs setting core.autocrlf=input in their gitconfig should prevent noise.
Special notes for your reviewer:
Release note:

This change is

@squillace @anhowe @colemickens To test, I did the following:

ran glide install on my Mac which repulls dependencies with UNIX LF line endings
committed (this PR)
On a Windows 10 VM with acs-engine, ran glide install
Confirmed with git status that vendor/ was unchanged

If either of y'all are on Windows, you can test by cloning/pulling, git checkout fix-line-encodings, and running make prereqs (assuming you have make on your box. Ultimately what's important is that you download and run glide:
prereqs:
	go get github.com/Masterminds/glide
	go get github.com/jteeuwen/go-bindata/...
	glide install

Oh, and you'll need to configure Git to convert CRLF to LF on commit:
just for this repo:
git config core.autocrlf input
globally (all projects you use git with)
git config --global core.autocrlf input

@acs-bot test this please

@acs-bot test this please

How does this behave when you check it out in Linux?

Didn't try Linux, though *NIX OSes (Linux, macOS, etc.) use LF-style line endings.

Awesome. I confirmed Linux does the right thing too (just in case my neckbeard lore was flimsy):
$ docker run --rm -it ubuntu bash
root@29a1767814c2:~# go get github.com/Azure/acs-engine && cd $GOPATH/src/github.com/Azure/acs-engine
root@29a1767814c2:~/go/src/github.com/Azure/acs-engine# git checkout fix-line-encodings
root@29a1767814c2:~/go/src/github.com/Azure/acs-engine# make prereqs
go get github.com/Masterminds/glide
go get github.com/jteeuwen/go-bindata/...
glide install
[INFO]	Downloading dependencies. Please wait...
[INFO]	--> Found desired version locally github.com/Azure/azure-sdk-for-go 5841475edc7c8725d79885d635aa8956f97fdf0e!
[INFO]	--> Found desired version locally github.com/Azure/go-autorest 58f6f26e200fa5dfb40c9cd1c83f3e2c860d779d!
[INFO]	--> Found desired version locally github.com/dgrijalva/jwt-go 6c8dedd55f8a2e41f605de6d5d66e51ed1f299fc!
...
root@29a1767814c2:~/go/src/github.com/Azure/acs-engine# git status
On branch fix-line-encodings
Your branch is up-to-date with 'origin/fix-line-encodings'.
nothing to commit, working directory clean

CI timed out after nearly an hour. We'll fix this. Re-running again.
Error [deploy_template test-acs-kubernetes-westus-472-0]
SUCCESS [cleanup test-acs-kubernetes-westus-472-0]
Error: Test failed
Build step 'Execute shell' marked build as failure
Archiving artifacts
Setting status of 1ecacee64c597a9f760b3779c944f230de966977 to FAILURE with url https://jenkins.azure-containers.io/job/acs-engine-pr/472/ and message: 'Build finished. '
Finished: FAILURE

@acs-bot test this please


Add AppVeyor status badge (#9)

Coverage remained the same at 87.963% when pulling 59a3f097313ba4ba97f76774f1daf9ca527785cc on techtonik:patch-5 into d4ddd3ae90c9c9ee337374b0acc6ec572a48a2e1 on WoLpH:develop.


Updates for v0.9.2

Updates to library, sender, and receiver to match OpenTrackIO 0.9.2.

@makkbe the tests are failing on the 'leader' vs. 'master' change. You should just be able to find replace in test_model.py to get them passing...


Remove pyproject.toml

✅
Depends on #48
Solves:

vscode failed to index packages with pyproject.toml along with a setup.py

Will create a new PR after the merge of #64
Too many conflict so I rather avoiding minor mistakes while resolving them..


Feat/vite config

Documentation on how to configure Vite in the context of a Laravel Package.
Development is done using a "main" app and the production build works when publishing the assets.

Hi Thanks for the pr will look at this now :)

Would you mind doing the following changes:

Moving your changes to a new file called 16-assets
Moving the assets chapter from the routing to the same new file

I pushed the requested changes for a new assets file.


[go_router] How to remove other routes in the routing line?

Is there an existing issue for this?

[X] I have searched the existing issues
[X] I have read the guide to filing a bug

Use case
My current routing line is A -> B -> C -> D now, I am on page D , i want to remove B . What should I do? I searched in the issue and found no answer
Proposal
Provide a method to remove a route in the routing line

Can you please be more specific on the reason/use case that needs to remove a route?

I have encountered similar situations but did not find a better answer.
[How to close a specific Flutter AlertDialog?](https://stackoverflow.com/questions/63262042/how-to-close-a-specific-flutter-alertdialog)
[how to close a specific dialog in flutter?](https://stackoverflow.com/questions/74554870/how-to-close-a-specific-dialog-in-flutter?noredirect=1)

Thank you for more information. I see there's an open issue addressing the case you described https://github.com/flutter/flutter/issues/99112 (removeRoute is also mentioned there).
Closing this issue in favor of that issue, so please follow up on it for further updates.


[POC, not for review] Make changes to read artifacts from storage blob

Replaced by another PR: https://github.com/Azure/azure-sdk-for-java/pull/16634
Closed this one.


class is replaced with the name of the class in the AST

https://github.com/IronLanguages/ironpython3/pull/721 replaces __class__ with the name of the class in the AST.
import ast
x = """
class test(object):
    def test(self):
        return __class__
"""
a = compile(x, "", "exec", flags=0x400)
assert a.body[0].body[0].body[0].value.id == "__class__"

Related to https://github.com/IronLanguages/ironpython3/issues/722

Resolved by https://github.com/IronLanguages/ironpython3/pull/1087


[ADD] Support for get and create of sequences

[FIX] Set auto increment to False  …

Set auto increment to False on all primary_keys to allow inserts

[ADD] Support for get and create of sequences  …

Add get_next_sequence method to run stored proc on CP for next

Dependent:

[x] #9

@t3ddftw - build is failing due to bad coveragerc. Working on it, but the tests are included so we're skewed a bit.


Channel feature rest endpoint

In Swagger UI, there are no channel REST endpoints such as create, update, or delete features for channels.
Does anyone know if the channel feature exists in the REST endpoints?

Thanks,
Sajjad

Olá @SajjadAhmad904 , the cannels is newsletters.
Docs:
https://wppconnect.io/swagger/wppconnect-server/#operation/createNewsletter


Unable to get audio stream

System OS
Windows
Python Version
3.9 (CPython)
Install Source
pip / PyPi
Install version / commit hash
v3.7.2
Expected Behavior vs Actual Behavior
I was trying to download Spotify list.
14 songs out of 36 didnt get downloaded.
Steps to reproduce

cmd as administarator
spotdl <playlist_link> --ignore-ffmpeg-version

Traceback
Unable to get audio stream for "Toca" by "Carnage" from video "https://www.youtube.com/watch?v=C4DV53fFyCU"
Unable to get audio stream for "Tsunami - Original Mix" by "DVBBS" from video "https://www.youtube.com/watch?v=ncq4eiFG9C0"
Unable to get audio stream for "Muzik" by "Knoc-Turn'al" from video "https://www.youtube.com/watch?v=CbGGVFyXvhk"
Unable to get audio stream for "Open Your Eyes" by "Guano Apes" from video "https://www.youtube.com/watch?v=7KyfLXnKe20"
Unable to get audio stream for "Bang Bang" by "Hollywood Undead" from video "https://www.youtube.com/watch?v=PgIiNdYRYME"
Unable to get audio stream for "Satisfy" by "NERO" from video "https://www.youtube.com/watch?v=HBuMAaWK0iw"
Unable to get audio stream for "Lies" by "CHVRCHES" from video "https://www.youtube.com/watch?v=UmunvW1UlU8"
Unable to get audio stream for "We Are Rebels" by "Alice In Videoland" from video "https://www.youtube.com/watch?v=WqJ9_VFkGe0"
Unable to get audio stream for "Hold That Sucker Down - Radio Edit" by "OT Quartet" from video "https://www.youtube.com/watch?v=3ZxS7MTiw4c"
Unable to get audio stream for "96 Quite Bitter Beings" by "CKY" from video "https://www.youtube.com/watch?v=He8_4KWFMi8"
Unable to get audio stream for "Celebrity Skin" by "Hole" from video "https://www.youtube.com/watch?v=Bv9d27d1Byg"
Unable to get audio stream for "Everything Black (feat. Mike Taylor)" by "Unlike Pluto" from video "https://www.youtube.com/watch?v=tZKe908QmIg"
Unable to get audio stream for "Days Go By" by "Dirty Vegas" from video "https://www.youtube.com/watch?v=azVokn2PRKc"
Unable to get audio stream for "Blood Brothers" by "Papa Roach" from video "https://www.youtube.com/watch?v=b-Hm9q_WwWU"
Unable to get audio stream for "Immigrant Song - Remaster" by "Led Zeppelin" from video "https://www.youtube.com/watch?v=5eHkjPCGXKQ"
Unable to get audio stream for "Hole in Your Heart" by "Royal Blood" from video "https://www.youtube.com/watch?v=Ep5ELC1NM50"
Unable to get audio stream for "Thunderstruck" by "AC/DC" from video "https://www.youtube.com/watch?v=lhg9bYNLvOg"

Other details
Windows 10 Home 64
Youtube Music works in my country, Poland.
The second try ended with the same resault.
The listed links work fine when I open them in Chrome.

Same behavior. Ubuntu, python 3.8.10, Belarus

@paweleck don't run spotDL in administrator and try again

@Silverarmor still the same problem

pip install -U --force spotdl pytube ytmusicapi youtube-dl spotipy
have you reinstalled as per FAQ?

also try clearing youtube-dl cache youtube-dl --rm-cache-dir

@Silverarmor
the command didnt help. What FAQ?
@xnetcat
Didnt help.
It worked fine on another computer however.

So I ran into this problem on OSX myself. So my problem was originating from a faulty openssl installation through homebrew. I simply ran brew reinstall<EMAIL_ADDRESS>and this regenerated the CA certificate bundle for me again.
==> Regenerating CA certificate bundle from keychain, this may take a while...

Hopes this helps for people who encounter this on OSX

Same issue on OSX, no solution for now...
I tried all on this issue

@Felix83000 we had this issue on bigsur (archived thread on discord), pip install --no-cache-dir --force --upgrade youtube-dl resolved it for them - can you try

pip install --no-cache-dir --force --upgrade youtube-dl

Hello @Silverarmor, I autoremove spotdl and then tried what you proposed, it did not work...

I have the same Bug and tried all of the above. On OSX.

Could you try downloading the audio manually using youtube-dl
youtube-dl -x <YT_URL>

youtube-dl -x https://www.youtube.com/watch?v=lhg9bYNLvOg
[youtube] lhg9bYNLvOg: Downloading webpage
[download] Destination: Thunderstruck-lhg9bYNLvOg.m4a
[download] 100% of 4.52MiB in 01:38
[ffmpeg] Correcting container in "Thunderstruck-lhg9bYNLvOg.m4a"
ERROR: "debug"

Some kind of error

@paweleck What's your youtube-dl version? youtube-dl --version

@xnetcat 2021.06.06

How about ffmpeg? ffmpeg -version
also can you try running this youtube-dl -f bestaudio https://www.youtube.com/watch?v=Z1tF3DdhUNk

ffmpeg -version
ffmpeg version N-50911-g9efcfbe
built on Mar 13 2013 21:26:48 with gcc 4.7.2 (GCC)
configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-libass --enable-libbluray --enable-libcaca --enable-libfreetype --enable-libgsm --enable-libilbc --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxavs --enable-libxvid --enable-zlib
libavutil      52. 19.100 / 52. 19.100
libavcodec     55.  0.100 / 55.  0.100
libavformat    55.  0.100 / 55.  0.100
libavdevice    54.  4.100 / 54.  4.100
libavfilter     3. 45.103 /  3. 45.103
libswscale      2.  2.100 /  2.  2.100
libswresample   0. 17.102 /  0. 17.102
libpostproc    52.  2.100 / 52.  2.100

Donwloaded from this page: https://www.gyan.dev/ffmpeg/builds/

youtube-dl -f bestaudio https://www.youtube.com/watch?v=Z1tF3DdhUNk
[youtube] Z1tF3DdhUNk: Downloading webpage
[download] Destination: Hodak_2K - Omerta (feat. Avi)-Z1tF3DdhUNk.webm
[download] 100% of 3.93MiB in 00:00

Works fine.

this ffmpeg is from 2013 💀, remove it and follow this guide
https://windowsloop.com/install-ffmpeg-windows-10/

where ffmpeg
C:\Program Files\ImageMagick-6.9.2-Q16\ffmpeg.exe
C:\ffmpeg\bin\ffmpeg.exe

Ive deleted the ImageMagick thing and now everything works fine :D Thx


Add support to create a Dataset from spark dataframe

Feature request
Add a new API Dataset.from_spark to create a Dataset from Spark DataFrame.
Motivation
Spark is a distributed computing framework that can handle large datasets. By supporting loading Spark DataFrames directly into Hugging Face Datasets, we enable take the advantages of spark to processing the data in parallel.
By providing a seamless integration between these two frameworks, we make it easier for data scientists and developers to work with both Spark and Hugging Face in the same workflow.
Your contribution
We can discuss about the ideas and I can help preparing a PR for this feature.

if i read spark Dataframe error on  multi-node Spark cluster.
did the Api (Dataset.from_spark) support Spark cluster, read dataframe and save_to_disk

How to perform predictions on Dataset object in Spark with multi-node cluster parallelism?

Addressed in #5701

Hi ! for your information we are working on some more documentation on how to use Spark with HF Datasets repositories (without the need for the datasets library) here: https://github.com/huggingface/datasets/pull/5701
Cc @lu-wang-dl @maddiedawson let me know what you think !

sorry, wrong link: https://github.com/huggingface/hub-docs/pull/1392


Replace RPC call with near-api-js when checking account existence

Is this PR still relevant? @Pessina

Is this PR still relevant? @Pessina

I believe yes.
We will replace the RPC call and rely on the already implemented function in near-api-js simplifying the code and saving us some lines.
What do you think?


[bug]: Man command highlighting as error

Describe the bug
The argument passed to man is always highlighted as red, indicating something is wrong when the command is perfectly valid
Steps to reproduce
Install and source the plugin
Expected behavior
The argument passed should be colored red only if the man page does not exist
Screenshots and recordings

Operating System & Version
linux-gnu | pc | x86_64 | x86_64 | x86_64 unknown
Zsh version
zsh 5.8.1 (x86_64-pc-linux-gnu)
Terminal emulator
xterm-kitty
If using WSL on Windows, which version of WSL
No response
Additional context
No response

From what I tested, this only happens in Arch-based distributions.

WORSE I type man   and it delays for about 30 seconds!
macOS arch64

From what I tested, this only happens in Arch-based distributions.

Sorry for the late reply, I indeed am on an arch based distro

Similar to/duplicate of https://github.com/zdharma-continuum/fast-syntax-highlighting/issues/27.

I think I have finally found the reason why this bug occurs (at least in Arch Linux). This plugin seems to be using the whatis command to highlight the man argument. If this bug occurs on your system, whatis probably does not work properly either. You can check this by typing the command whatis ls for example.
The thing is that whatis depends in a index of all the available man pages in the system and Arch Linux does not generate it if you don't tell it to do so.
How to fix it?
If you are using man-db (most likely):
Arch Linux has a service called man-db.service as well as a timer called man-db.timer, which runs man-db.service daily.
This service is responsible for generating the index of the man pages, so the bug should be fixed executing the following command:
sudo systemctl enable man-db.timer

You must run sudo mandb if you want to create an index instantly.f
If you are using mandoc:
I don't know if there is a similar service as in man-db, but you can do sudo makewhatis /usr/share/man to generate the index.
I hope you find it helpful!

@Warbacon I came to a similar conclusion,see https://github.com/zdharma-continuum/fast-syntax-highlighting/issues/27 for details and another possible workaround.


sequencer/devnet: implement faucet module

To get funds to users for the devnet, we can add a simple faucet module to the sequencer. This is not a feature for mainnet or even testnet, but nice for our initial devnet.
Features:

an action type that will distribute funds to the specified recipient
tracks how much has been received by each recipient address in a certain time period
rate-limit the amount each account can receive per time period (day?)
ideally can be easily configured to be on/off

closing in favour of #358


Question regarding capability

Hey guys! I am really impressed by this library.
So I have a video fingerprinting technology (no, not histogram, perceptual identification) which is pretty freaking accurate to a high resolution of precision (10 seconds from the source video is enough to match). The only "weak point" I have is when color saturation is changed from the source video.
Something I've always really wanted was a way to "normalize" the color. In fact, since my system works way better with color than black-and-white video, colorizing a black and white video would be awesome! This is something I absolutely must play with.
The question I have that you may be able to answer is whether this would produce reasonably consistent results:

Take source image, copy it, make it black and white
Take source image, copy it, saturate the color a bit, make it black and white
Colorize the result of #1
Colorize the result of #2

When you compare those two, how similar are they? Assume reasonable saturation change so it does not harm the user perception of the original image too much.
If you know this off-hand, that'd be amazing! (I can't test right now)
--Collin

Hi Collin, It sounds like you may want a white balancing algorithm.  Thanks for your interest!
Richard

Thanks! Yes, that is close, but consistency is the key and AWB's tend to vary depending on the illuminate pixel samples... making it very picture-dependent. What fascinated me about your system is it could have the effect  need without balancing at all, or perhaps only balancing a grayscale variant of the picture.
If you do wind up implementing this, or even want someone to assist in testing, I would be interested. Do you have a prefered notification method for updating on progress of such things?

Hi, you can check my webpage or my group members webpages (find them on here: https://people.eecs.berkeley.edu/~efros/) for our most recent work!

Did you ever do anything with white balancing?

We didn't end up pursuing that. We may get back to it at some point. Thanks!


[FLINK-12894][doc][Chinese-translation]add Chinese documentation of how to configure and use catalogs in SQL CLI

What is the purpose of the change
Support Chinese version of sqlClient.md
Brief change log
(for example:)

The TaskInfo is stored in the blob store on job creation time as a persistent artifact
Deployments RPC transmits only the blob storage reference
TaskManagers retrieve the TaskInfo from the blob cache

Verifying this change
(Please pick either of the following options)
This change is a trivial rework / code cleanup without any test coverage.
(or)
This change is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(example:)

Added integration tests for end-to-end deployment with large payloads (100MB)
Extended integration test for recovery after master (JobManager) failure
Added test that validates that TaskInfo is transferred only once across recoveries
Manually verified the change by running a 4 node cluser with 2 JobManagers and 4 TaskManagers, a stateful streaming program, and killing one JobManager and two TaskManagers during the execution, verifying that recovery happens correctly.

Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (yes / no)
The public API, i.e., is any changed class annotated with @Public(Evolving): (yes / no)
The serializers: (yes / no / don't know)
The runtime per-record code paths (performance sensitive): (yes / no / don't know)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes / no / don't know)
The S3 file system connector: (yes / no / don't know)

Documentation

Does this pull request introduce a new feature? (yes / no)
If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

i have updated @bowenli86

@yiduwangkai , please rebase your commit on the latest master and do not use "merge".

yes i used rebase at first, But when I execute this command, when I push,
git asks for pull, maybe it is improper operation before, causing the
situation to be merged when the pull occurs, but please be assured that no
other files have been changed. thx
Jark Wu<EMAIL_ADDRESS>于2019年7月21日周日 下午10:19写道：

@yiduwangkai https://github.com/yiduwangkai , please rebase your commit
on the latest master and do not use "merge".
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/apache/flink/pull/9164?email_source=notifications&email_token=AD2ZDBFT26AVY4Q23H3A7K3QARV63A5CNFSM4IE2BQKKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2OEPGA#issuecomment-513558424,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AD2ZDBFZCY2L4D3U63X2CG3QARV63ANCNFSM4IE2BQKA
.

@yiduwangkai , please rebase your commit on the latest master and do not use "merge".

yes i used rebase at first, But when I execute this command, when I push, git asks for pull, maybe it is improper operation before, causing the situation to be merged when the pull occurs, but please be assured that no other files have been changed. thx

Why is there a new merge commit extra information, because I use the pull
command to pull the latest code of the upstream branch, this record will be
generated if the local branch lags behind the upstream branch
kai wang<EMAIL_ADDRESS>于2019年7月21日周日 下午11:28写道：

yes i used rebase at first, But when I execute this command, when I push,
git asks for pull, maybe it is improper operation before, causing the
situation to be merged when the pull occurs, but please be assured that no
other files have been changed. thx
Jark Wu<EMAIL_ADDRESS>于2019年7月21日周日 下午10:19写道：

@yiduwangkai https://github.com/yiduwangkai , please rebase your
commit on the latest master and do not use "merge".
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/apache/flink/pull/9164?email_source=notifications&email_token=AD2ZDBFT26AVY4Q23H3A7K3QARV63A5CNFSM4IE2BQKKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2OEPGA#issuecomment-513558424,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AD2ZDBFZCY2L4D3U63X2CG3QARV63ANCNFSM4IE2BQKA
.

Hi @yiduwangkai , can you rebase this PR?

Hi @yiduwangkai , can you rebase this PR?

Ok

i want to close this pr, merge [apache/flink] [FLINK-13086]add Chinese
documentation for catalogs (#9163)
and [apache/flink] [FLINK-12894]add Chinese documentation of how to
configure and use catalogs in SQL CLI (#9164) together and submit a new
one, do you agree
Bowen L<EMAIL_ADDRESS>于2019年8月13日周二 上午2:36写道：

Hi @yiduwangkai https://github.com/yiduwangkai , can you rebase this PR?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/apache/flink/pull/9164?email_source=notifications&email_token=AD2ZDBGRGYEJ7TQCPQ5UNVTQEGUT5A5CNFSM4IE2BQKKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4DN7HY#issuecomment-520544159,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AD2ZDBBQAV7UT2CA6I6T7VDQEGUT5ANCNFSM4IE2BQKA
.


ImportError: libpython3.7m.so.1.0: cannot open shared object file

Hi!
I was trying to inference with ARM CPU and follow the step of Building ARM CPU plugin with LINUX.
However, when I try to build a Docker image with docker image build -t arm-plugin -f dockerfiles/Dockerfile.RPi64_focal . , I faced the error
Error
        esac; \
         _PYTHON_PROJECT_BASE=/Python-3.7.9 _PYTHON_HOST_PLATFORM=linux-aarch64 PYTHONPATH=/Python-3.7.9/build/lib.linux-aarch64-3.7:./Lib _PYTHON_SYSCONFIGDATA_NAME=_sysconfigdata_m_linux_aarch64-linux-gnu python3.7 -m ensurepip \
                $ensurepip --root=/ ; \
fi
Traceback (most recent call last):
  File "/Python-3.7.9/Lib/runpy.py", line 183, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/Python-3.7.9/Lib/runpy.py", line 142, in _get_module_details
    return _get_module_details(pkg_main_name, error)
  File "/Python-3.7.9/Lib/runpy.py", line 109, in _get_module_details
    __import__(pkg_name)
  File "/Python-3.7.9/Lib/ensurepip/__init__.py", line 6, in <module>
    import tempfile
  File "/Python-3.7.9/Lib/tempfile.py", line 45, in <module>
    from random import Random as _Random
  File "/Python-3.7.9/Lib/random.py", line 42, in <module>
    from math import log as _log, exp as _exp, pi as _pi, e as _e, ceil as _ceil
ImportError: libpython3.7m.so.1.0: cannot open shared object file: No such file or directory
make: *** [Makefile:1151: altinstall] Error 1

My environment is
Raspberry PI 4G
Ubuntu20.04 (64bit)
Docker version 20.10.18, build b40c2f6
Cause
It seems that Python-3.7.9.tar.xz doesn't include libpython3.7m.so.1.0 from the begining.
(https://www.python.org/ftp/python/3.7.9/Python-3.7.9.tar.xz)
Questions
How can I install libpython3.7m.so.1.0 mannually?
Please help me if you know any solutions.
Thanks.

Hi @mio903061
I've updated build process and dockerfiles. Could you please try this PR https://github.com/openvinotoolkit/openvino_contrib/pull/447?
git remote add ilya-lavrenov https://github.com/ilya-lavrenov/openvino_contrib.git
git fetch ilya-lavrenov
git checkout docker-build-update

And all others as usual


DynamicCastRework, Part 2: Test Suite

This validation test exercises a large matrix of types and invariants for
dynamic casting.  It's formulated as a Python script that emits a number of
Swift test programs, compiles, and executes them.  The programs are compiled in
Swift 4 and 5 mode, with -O and -Onone.
The invariants used by these tests follow the specification presented
in PR #33010.  It should be easy to add more as desired.
I've tried to design this in such a way that CI logs can provide enough
information to narrowly identify the problem area:  Separate test cases
are generated for each invariant and each type (in particular, this helps
with compiler crashes that report the full body of the function in question).
The files and test suites are named to identify the optimization mode.
The goal of this test suite is to cover a broad cross-section of casting
capabilities and combinations, and to make it easy to expand the matrix of
combinations.  New invariants can easily be added and applied to many types;
as new types are added to this test, they can exploit the existing invariants
and be exercised across all optimization modes.

A few important notes:

This is another part of the work begun with PR #29658.
This is the broadest of the test work from that effort, but there are many other test changes and additions that will land with the various code changes.
I expect this test suite will cover everything from the specification in PR #33010 by the time it is merged.
This PR will only be merged after the associated code changes are merged.

@swift-ci Please test Linux

Please test with the following pull request:
https://github.com/apple/swift/pull/29658
@swift-ci Please test macOS platform

Build failed
Swift Test Linux Platform
Git Sha - 2ca7e86b20be7c1a01eaefb0aa68d2fafc4e7847

@swift-ci Please test Linux

Build failed
Swift Test Linux Platform
Git Sha - 2ca7e86b20be7c1a01eaefb0aa68d2fafc4e7847

Build failed
Swift Test OS X Platform
Git Sha - 2ca7e86b20be7c1a01eaefb0aa68d2fafc4e7847

@swift-ci Please test macOS

Build failed
Swift Test OS X Platform
Git Sha - 2ca7e86b20be7c1a01eaefb0aa68d2fafc4e7847

@swift-ci Please test macOS

Build failed
Swift Test OS X Platform
Git Sha - 60f3bfd9302ccca60c3668c23acb976f017c407e

It's very exciting to see a tested, hopefully complete, specification shared by the runtime and optimizer!

apple/swift/pull/33561
@swift-ci Please test

Build failed
Swift Test Linux Platform
Git Sha - 3cff56feb665638b49d99d2a50e4b2b2837730b8

@swift-ci Please test

Build failed
Swift Test OS X Platform
Git Sha - 994971113f931b4bc2e90e4cf34be27ee36fbc0b

Build failed
Swift Test Linux Platform
Git Sha - 994971113f931b4bc2e90e4cf34be27ee36fbc0b

Please update the base branch to main by Oct 5th otherwise the pull request will be closed automatically.

How to change the base branch: (Link)
More detail about the branch update: (Link)

@swift-ci Please test

Build failed
Swift Test Linux Platform
Git Sha - 5ddd0ff9ce2729e506a38b38b95c28885850da9a

Build failed
Swift Test OS X Platform
Git Sha - 5ddd0ff9ce2729e506a38b38b95c28885850da9a

Build failed
Swift Test Linux Platform
Git Sha - 5ddd0ff9ce2729e506a38b38b95c28885850da9a

Build failed
Swift Test OS X Platform
Git Sha - 5ddd0ff9ce2729e506a38b38b95c28885850da9a

An improved version of this work has been merged via other PRs.


[WIP] Enable CUDA 12 ARM builds

When integrating upstream changes into this feedstock, CUDA 12 ARM builds were kept off (they haven't yet been supported here) ( https://github.com/rapidsai/xgboost-feedstock/pull/28#discussion_r1304878573 ). This is due to the fact that this feedstock depends on building with RMM, which isn't built for CUDA 12 ARM yet.
In this PR, CUDA 12 ARM builds are enabled. This still depends on a CUDA 12 ARM build of RMM. So the jobs here will fail until RMM is built for CUDA 12 ARM. Still this provides a place to track this work and build XGBoost for CUDA 12 ARM once RMM is available. This should be fixed by PR ( https://github.com/rapidsai/rmm/pull/1330 )

We'd like to rerun CI on this job but it's too old to rerun CI (over 30 days old) and I lack permissions to push to jakirkham-feedstocks/xgboost-feedstock. I'm not sure how to proceed so we're going to put xgboost CUDA 12 + ARM support on hold for the moment.


[WIP] GLText selection

Summary
This PR implements selection for GLText command.
As for other commands (i.e. Cubes, Cones, etc), we need to set the getChildrenForHitmap property in  to a valid function. But in the case of GLText, that's not enough.
In contrast with other commands that uses fromGeometry() or withPose() utility functions (i.e. Cubes), GLText has a custom shader and properties which prevent the correct rendering of the selectable objects to the hitmap. One of those problems is that the color is not set correctly (when hitmap is on, Color is a vec4 instead of a Color object) and nothing gets rendered.
In addition, we don't want to render the text as is into the hitmap because that will lead to a bad user experience. If so, users will have to click exactly in those pixels where the glyphs are, preveting selection if a user clicks inside the letter "O", for example. For that reason, whenever we need to render the hitmap for GLText, we're rendering the polygons instead of the glyphs (see comment in fragment shader).
I also added integration tests, described below.
Test plan
I added several stories handling special cases for text selection:

Single GLText object with default properties
Single GLText with combinations of billboard, background and scale invariance
GLText and background object, with stacked objects on and off.
GLText just containing the letter "O", to prove that selection works when using polygons instead of glyphs (with and without background objects).

Versioning impact
Minor

Also the title still says [WIP], I'm assuming that is not the case anymore?


Remove timeout for the mesh runtime calls

I think this was set for tests. Why are we removing it?


modules: hal: nordic: Remove dependency on DT from NRF_* mappings

Update the module revision so that the following commit becomes
effective:

nrfx_config_nrf9160: Remove dependency on DT from NRF_* mappings

Requires https://github.com/zephyrproject-rtos/hal_nordic/pull/4 to go in first.

Pushed amended commit with no changes, to re-trigger CI and check the recent changes made in zephyrproject-rtos/hal_nordic#4.

Pushed amended commit with no changes, to re-trigger CI and check the recent changes made in zephyrproject-rtos/hal_nordic#4.

Could re-trigger by close/re-open, too.

Pushed amended commit with no changes, to re-trigger CI and check the recent changes made in zephyrproject-rtos/hal_nordic#4.

Could re-trigger by close/re-open, too.

I know, just thought this way it would be looking a bit nicer.

Updated west.yml with the proper revision of the hal_nordic module as https://github.com/zephyrproject-rtos/hal_nordic/pull/4 has been merged.

Sorry, I haven't included the change done to west.yml in the previous push.

Don't you want to include also #5 from the hal-nordic repo?

This one could be merged, if you run CI here, that includes that commit

...or you can do that in a separate PR.


Roll Flutter Engine from d73878d9b8ee to c8ba879d2ee3 (1 revision)

https://github.com/flutter/engine/compare/d73878d9b8ee...c8ba879d2ee3
2022-08-24<EMAIL_ADDRESS>Roll Dart SDK from 539f256f8b4b to 859e9bbe6f0a (1 revision) (flutter/engine#35668)
If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-engine-flutter-autoroll
Please CC<EMAIL_ADDRESS>on the revert to ensure that a human
is aware of the problem.
To file a bug in Flutter: https://github.com/flutter/flutter/issues/new/choose
To report a problem with the AutoRoller itself, please file a bug:
https://bugs.chromium.org/p/skia/issues/entry?template=Autoroller+Bug
Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md

Commit queue failed; closing this roll.


[TIR] Fold const or empty iter partition

Some minor updation to loop partition pass

Use tvm::min/tvm::max instead of Min/Max
Since currently VectorizeLoop pass is just after LoopPartition, there are no extra simplifications. The loop like T.vectorize(Min(1, 8 - 3))  will just fail on vectorizer check. We could just fold constants immediately when we create partitioned loops.

Remove unneccesary warning log, and do not create zero-extent loop if we could prove the extent is less or equal to zero.

CC: @tqchen


Missing licence File

I am assuming that the code and model provided in the public repository is available for reuse in other applications and personal projects but It would be great to have a licence such as MIT added to the repository to help in making the decision easier.

Fixed the problem


Fix nightly not being applied after rebase

metric
value

change

binary-size-nk3xn-test
547504
:white_check_mark:
-37052 (-0.06%)

Insignifcant changes

metric
value

change

binary-size-nk3am
1197644
:white_circle:
+0 (+0.00%)

binary-size-nk3am-test
2001735
:white_circle:
+0 (+0.00%)

binary-size-nk3xn
431364
:white_circle:
+0 (+0.00%)

binary-size-nkpk
736974
:white_circle:
+0 (+0.00%)


Pie charts and a few suggestions

A few suggestions:
V 5.0.16 was used.

In the Cookbook if you filter by "Pie" or "AddPie" it should show a quick sample. I couldn't find documentation for Pie charts (there was in v4).
In V4 I used to have plt.YAxis.TickLabelFormat("#,##0,,M", dateTimeFormat:=False), not sure how it's now handled in V5? Couldn't find details in the docs.
plt.Legend.Orientation = Orientation.Horizontal has changed. Couldn't find how to keep this in V5.
ScottPlot Demo 5.0.16.exe has no app icon. V4 had a nice one. V4 also shows a URL to the site which is pretty useful but got lost in v5.

This is a gem of a repo!

Hi @sterenas, thanks for these suggestions! I'll tackle them in separate issues:
1, It's there but your browser may be serving old content. You can check out https://scottplot.net/cookbook/5.0/Pie/ and I created #3278 to improve the cache issue

https://scottplot.net/cookbook/5.0/search/ can help get you started but I agree a DateTime-specific example would be helpful so I'll create one and add it to the cookbook #3279

Horizontal legend orientation is not yet supported. I'll create a ticket to track that feature being added.  #3280

Good catch! I'll fix that in the next version of the demo. #3281

Since items have their own tickets I'll close this issue and follow-up individually as I get to each. Thanks again for the suggestions!

Related to suggestion 2,
// plot sample DateTime data
DateTime[] dates = Generate.DateTime.Days(100);
double[] ys = Generate.RandomWalk(100);
myPlot.Add.Scatter(dates, ys);
myPlot.Axes.DateTimeTicksBottom();

// add logic into the RenderStarting event to customize tick labels
myPlot.RenderManager.RenderStarting += (s, e) =>
{
    Tick[] ticks = myPlot.Axes.Bottom.TickGenerator.Ticks;
    for (int i = 0; i < ticks.Length; i++)
    {
        DateTime dt = DateTime.FromOADate(ticks[i].Position);
        string label = $"{dt:MMM} '{dt:yy}";
        ticks[i] = new Tick(ticks[i].Position, label);
    }
};


Logging and Raw Link Layer Cleanup

Some clean up of the raw link layer to improve logging usefulness.

Codecov Report

Merging #1541 into master will increase coverage by 0.34%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #1541      +/-   ##
==========================================
+ Coverage   69.81%   70.16%   +0.34%
==========================================
  Files         151      151
  Lines       20119    19587     -532
  Branches     2456     2400      -56
==========================================
- Hits        14046    13743     -303
+ Misses       5149     4953     -196
+ Partials      924      891      -33

Impacted Files
Coverage Δ

src/ncp/ncp_base.cpp
24.85% <ø> (-10.71%)
:arrow_down:

src/core/thread/lowpan.cpp
93.01% <0%> (-0.72%)
:arrow_down:

src/core/thread/network_data_leader_ftd.cpp
67.99% <0%> (-0.47%)
:arrow_down:

src/core/thread/mle_router.cpp
80.1% <0%> (-0.11%)
:arrow_down:

src/core/thread/mle.cpp
81.89% <0%> (+0.07%)
:arrow_up:

src/core/thread/mesh_forwarder.cpp
74.48% <0%> (+0.1%)
:arrow_up:

src/core/net/dhcp6_client.cpp
78.71% <0%> (+0.33%)
:arrow_up:

src/core/thread/lowpan.hpp
97.36% <0%> (+10.52%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 959b648...dff0d3b. Read the comment docs.


Calculating event_length in eventalign

Hi, firstly thanks for developing such an awesome tool. I have been using nanopolish eventalign for most of my work with Nanopore data.
Just curious, how does nanopolish calculate the event length feature in the output for eventalign? Am I right in interpreting this as some estimate on the time it takes for the 5-mer or 6-mer to stay in the pore?
Thank you!

Prior to analysis the raw current signal (made up of current samples) is segmented into the "events" nanopolish uses. Each event has a duration (measured in seconds), which corresponds to the number of samples in the segment.
Jared

Hi Jared. thanks for the reply. Let me see if I get this right, so we are assuming a constant sampling rate for the pore and this number is multiplied by the number of data points assigned to each event segment detected by nanopolish to get the event length?
Thank you

Yes that's right, the sampling rate is constant.


Add policy_name and policy_group to node attributes

Prior to this change, policy_name and policy_group would get set to null on Node.save().

This feature works for saving node attributes with policy_name and policy_group but it breaks node.create(). The problem is, there is no default value set for policy_name and policy_group. I could set a default but I don't think that's the right thing to do. I would prefer to set them to null by default but the chef api does not like getting null values. The other thought I had was to remove the policy_name and policy_group attributes before calling the api but an exception gets thrown if the node object does not contain all the attributes defined for it in the class.
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/chef/utils/json.py", line 26, in <lambda>
    dumps = lambda obj, **kwargs: json.dumps(obj, cls=JSONEncoder, **kwargs)
  File "/usr/lib64/python2.7/json/__init__.py", line 250, in dumps
    sort_keys=sort_keys, **kw).encode(obj)
  File "/usr/lib64/python2.7/json/encoder.py", line 207, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib64/python2.7/json/encoder.py", line 270, in iterencode
    return _iterencode(o, 0)
  File "/usr/lib/python2.7/site-packages/chef/utils/json.py", line 18, in default
    return maybe_call(obj.to_dict)
  File "/usr/lib/python2.7/site-packages/chef/utils/json.py", line 10, in maybe_call
    return x()
  File "/usr/lib/python2.7/site-packages/chef/base.py", line 132, in to_dict
    d[attr] = getattr(self, attr)
AttributeError: 'Node' object has no attribute 'policy_group'


Improved information on running Dojo in production

uwsgi, gunicorn, nginx, etc

@devGregA currently Dojo seems to be tied to MySQL, would be good if there was less coupling.
Also current set up using Django test server, a basic guide would be great for setting up with gunicorn, nginx etc. I'm slowly making progress, will advise if I can add some notes.

@devGregA I'll include this with #249


Request: Additional parameters for search and objects endpoints

It would be nice to have ability to pull back more-curated lists from the api via additional parameters in the request, rather than receive broad data and make individual requests to qualify what has been returned. These could/should be related to the nodes available in an object.
Examples:
v1/search?q=&isHighlight=true
v1/objects?isPublicDomain=true&culture=french

Hi @jperegoy
Thank you for the request.  We have a filter feature in our backlog, and plan to add it in the coming weeks.
Thanks again,
Spencer


[Bug] 通过vscode官方扩展运行playwright，monocart报告的自定义字段会重复展示

Describe the bug
通过vscode官方扩展运行playwright，monocart报告的自定义字段会重复展示
如图，自定义的owner 和mantisBT 字段重复出现，规律是运行几次就重复几次，除非重启vscode。

目前这个问题只出现在vs code里面运行，命令行运行没有这种问题
To Reproduce

Version:
"monocart-reporter": "^2.4.8",
"playwright/test": "^1.44.1",
vscode playwright插件 1.1.6
macOS 14.5

Have you tried the latest version (including node), and does the issue still exist?
Yes

provide a GitHub repository with a minimal (but complete) example and simple/clear steps on how to reproduce the bug.
https://github.com/jiawen94/playwright-code-snippet/tree/monocart-issue-122

It should be fixed, try<EMAIL_ADDRESS>

谢谢，2.4.9 测试OK了

发现changelog的fixed repeated columns (#122)对应的版本写错了，应该是2.4.9而非2.4.8

感谢提醒，下次改一下就行


Setting to turn type hints off?

Is there a way to turn these type hints off?

Ah, found it:
"editor.codeLens": false


Some special characters cannot show up on the site title

Hello again, I have come across with another problem when building my site using this theme which is about the site title.
I edited the site title by following the instructions written on the config.toml in the exampleSite folder.
I went through the usual steps which include copying the contents of the config.toml of the exampleSite folder into the config.toml in the site root, and changing the title variable in the file. I have changed the site title to what as shown below:
  title            = "Tze-Heng Tan  |  Welcome"

However, as a result, the special characters which I have typed within the quotation marks did not show up accordingly at the web browser, instead it appeared like this:
Tze Heng Tan Welcome
(even the extra spacing is not shown)
I have tried out other special characters like curly hyphen (~), hyphen (-) and colon (:), but they do not show up at the browser either.
Strangely enough, when I entered symbols like the single and double quotation marks (' "), and also the backtick (`), they do show up at the browser, but those symbols are not what I want to include in my site title.
May I know the reason behind this and how to resolve this issue? Thank you.

Also, this problem only appears on the homepage; when the "sub-pages" are opened (e.g. the list page showing all of my blog posts), the special characters are shown normally instead, like this:
Blog - Tze-Heng Tan | Welcome

The title ist generated by the metadata.html file.
As you can see the title ist put together with a "Page Title" (title parameter from the markdown file) and a "Site Title" (title parameter from the config.toml file) variable.
The title generating code line also contains text formating intstructions like humanize. So only your special characters in the "Page Title" variable are formated away by the humanize function.
On subpage level they are different variables. But on homepage level they have the same value - title parameter from the config.toml file. This leads to your described issue.
Removing the humanize function should fix your issue.


Allow multiple folders and multiple files (nested or not)

Hello, I know that already exist a feature to create multiple nested folders. But there isn't a feature to create not nested multiple folders. It would be great to have it.
For example:
When click "New Folder"
Enter: "{folder1/{folder2,folder3,folder4}"....
It will create a tree of Folders folder1>folder2,folder3,folder4 (at same level).
Idk if how we could invest in this, but combine this with multiple files creation sounds good too. e.g. In that way:
Enter: "{folder1/{folder2[file1.js,file2.js],folder3[file1.js],folder4}"....
it will create folders: folder1>folder2,folder3,folder4, and inside folder2, two js files, and one file to folder3.
We talked about a releated subject in #30744, but isn't the same ideia.

This feature request is now a candidate for our backlog. The community has 60 days to upvote the issue. If it receives 20 upvotes we will move it to our backlog. If not, we will close it. To learn more about how we handle feature requests, please see our documentation.
Happy Coding!

This feature request has not yet received the 20 community upvotes it takes to make to our backlog. 10 days to go. To learn more about how we handle feature requests, please see our documentation.
Happy Coding!


Player Stats

Sorry for interupting but when i try to search a player it comes as Sorry Something went wrong. When i checked in heroku log i got this :
TypeError: Cannot read property '0' of undefined
at getPlayerStats (/app/src/dataFetch.js:58:35)
at processTicksAndRejections (internal/process/task_queues.js:97:5)

Maybe the player name was wrong. The API that I used sometimes give the closest name, else sends a status 400. So what I did was put that part in a try catch block.

Umm I tried to search KL Rahul as u did but no luck. Maybe I did something wrong?

Not able to reproduce :cry:

Is it because I changed prefix?

That shouldn't be an issue

Maybe there are some files missing to run that command?

okay, so you need to create an account in the https://www.cricapi.com/ and save the token in an env file, that token is used in calling the apis. I have made a sample.env file, you can try look at that.

okay, so you need to create an account in the https://www.cricapi.com/ and save the token in an env file, that token is used in calling the apis. I have made a sample.env file, you can try look at that.

Okay. Can I leave discord bot token empty in sample.env for henroku??

sample.env is just a sample file for .env . You can directly put the Tokens and the API key in the code, but it is not considered a good practice, but yeah it will work.

Thanks for the help again👍

ummm sorry but its the same.........

Any Breakthrough??


[Python] Expose QuotingStyle to Python

This exposes the QuotingStyle option to Python. I tested this with the v10 tag since I wasn't able to build the master branch for some reason.
I'm happy to use enums instead of strings. Both seem to be used in Python and I wasn't sure which one was preferred.
(no Jira ticket since I don't have an account and can't create one)

Thanks for the feedback @pitrou. I'll make the changes. I also created ticket https://github.com/apache/arrow/issues/14755.
Should I change the interface to an Enum instead of strings? I see both in the project and I'm not sure which one is preferable.

Should I change the interface to an Enum instead of strings? I see both in the project and I'm not sure which one is preferable.

I think a string is ok in this case.

@jorisvandenbossche Do you also want to take a look at this?


remove skirt strokes from calculation

Skirt strokes affect CoM calculation, very visible after adding z coordinate of CoM.

there isn't a universal or standardized rule that requires .gcode files to include a specific comment (like ;MESH: ...) to indicate a skirt is done printing. SOLUTION: disabling skirt when producing G-code.

there isn't a universal or standardized rule that requires .gcode files to include a specific comment (like ;MESH: ...) to indicate a skirt is done printing. SOLUTION: disabling skirt when producing G-code.


add python fibonacci

This Pull Request resolves Issue #560
Description:
add some code for using fibonacci algorithm in python

Are there any updates or should we close the PR?

PR closed due to inactivity!


Add "sys:node-uuid" metadata support

Description of Problem:
When a document is imported, alfresco checks if the "sys:node-uuid" metadata is present and if it's the case, it creates the new document's nodeRef using the uuid specified in the "sys:node-uuid" metadata.
Unfortunately, this cannot work using your bulk import because the file is first created with only the file name in the properties, so a new nodeRef is generated, then the document is updated with the other properties and the "sys:node-uuid" metadata is ignored.
It would be appreciated if support for the "sys:node-uuid" metadata was added.
For information, we are currently using Alfresco 5.0.3 and alfresco-bulk-import 2.1.0.
Thank you very much!

Hi Laurant - apologies for commenting on a year old commit - but we need to set the sys:node-uuid and the pull request is still open.  From the looks of it, the change was a minor one, but just wanted to check your 'quick and dirty' fix worked out ok for you.


handleToolEnd does not get triggered, how else to receive agent tool output?

I am trying to collect the results of tools called by a ChatConversationalAgent. I get the handleAgentAction callback with tool name and input, but I don't know where to get the output of the tool.
The handleToolEnd callback which should be the one for the job does not get triggered in https://github.com/hwchase17/langchainjs/blob/main/langchain/src/agents/executor.ts

I do see the result of the tool logged on console. Might it be that ConsoleCallbackHandler is still attached to the agent as a callback manager even after passing a new callback manager to AgentExecutor.fromAgentAndTools with AgentExecutor.fromAgentAndTools({...}) ?

handleToolEnd and handleToolError are called by the tool and not by the agent, so a CallbackMa ager needs to be passed explicitly to the tool.
CallbackManager is the second parameter to the base Tool at https://github.com/hwchase17/langchainjs/blob/main/langchain/src/tools/base.ts but most tools don't have it as a parameter and just call super().
I will just use something like toolInstance.callbackManager = ....

I didn't understand that a CallbackManager needs to be passed to each tool and it is not enough to pass it the agent.


Linear semantic segmentation with ViT-L models

Hi,
I was wondering if you had evaluated the ViT-L pretrained on ImageNet1k and ViT-L pretrained on ImageNet22k on the linear semantic segmentation benchmark on ADE20k, similar to column 3 of right table of Table 6 in the paper ?
If yes, can you share the results and the corresponding log files ?
Thanks!

Hi @Adrien987k,
I haven't tried those experiments.


testServerSuite depends on /tmp/test_pd_check_clusterID_0/.touch

Please answer these questions before submitting your issue. Thanks!

What did you do?
If possible, provide a recipe for reproducing the error.

Use PD with the fix PR #1945 (or the test will be blocked), and execute the following commands with a normal user.
sudo mkdir /tmp/test_pd_check_clusterID_0
cd ${PD_SRC_ROOT}/server
go test -check.f testServerSuite

What did you expect to see?

Test pass.

What did you see instead?

FAIL: server_test.go:152: testServerSuite.TestCheckClusterID

server_test.go:128:
    c.Assert(err, IsNil)
... value *errors.withStack = cannot access data directory: open /tmp/test_pd_check_clusterID_0/.touch: permission denied ("cannot access data directory: open /tmp/test_pd_check_clusterID_0/.touch: permission denied")

What version of PD are you using (pd-server -V)?

PR #1945

Is can not occur in real env. Closing.


Typed Arrays with a named enum type cannot be set using Object.set

Tested versions

reproducible in v4.4.dev2.mono.official [97ef3c837], v4.3.stable.mono.official [77dcf97d8] and v4.2.2.stable.mono.official [15073afe3]

System information
Godot v4.3.stable.mono - Windows 10.0.22631 - GLES3 (Compatibility) - NVIDIA GeForce RTX 4070 Ti SUPER (NVIDIA; <IP_ADDRESS>09) - AMD Ryzen 7 7800X3D 8-Core Processor (16 Threads)
Issue description
If you declare an array with a named enum type, the Object.set method will not set that array's value, but should.
Please see code in steps to reproduce or MRP for more details.
Steps to reproduce
Use the MRP, or create a single scene project with this script attached to a single Node2D and run it:
extends Node2D

enum THING {FOO, BAR, BAZ}

var my_typed_list: Array[THING] = []

func _ready() -> void:
	self.set("my_typed_list", [THING.FOO, THING.BAR])
	print("(Typed array) Should have 2 elements but doesn't: ", my_typed_list, " <--------- Incorrect")

	my_typed_list = [THING.FOO, THING.BAR]
	print("(Typed array) Should have 2 elements and does: ", my_typed_list, " <--------- Correct ")

Expected behavior: The array should print with 2 elements.
Actual behavior: The array prints with 0 elements.
Minimal reproduction project (MRP)
setenum.zip

This is likely part of a broader limitation of typed arrays as you can't assign a non-typed array to a typed one, see:

https://github.com/godotengine/godot/issues/83511
https://github.com/godotengine/godot-proposals/discussions/7364

Thanks. You're probably right that this is an expression of a broader issue. To add some more clarity here, the issue I am raising here is specific to Object.set as, for example, I am able to directly assign the untyped literal array (with =): eg, if you add these two lines to my reproduction code above:
	my_typed_list = [THING.FOO, THING.BAR]
	print("(Typed array) Should have 2 elements and does: ", my_typed_list, " <--------- Correct ")

It is unintuitive that Object.set would not work when direct assignment does. If this behavior is intended (even if temporarily) it should likely be documented, or raise a warning at runtime.

Array[^1] literals (the [] syntax) create untyped arrays in untyped contexts such as Object.set(). You can use type casting to create a typed array (note that the as operator only works on array literals, not arbitrary expressions).
self.set("my_typed_list", [THING.FOO, THING.BAR] as Array[THING])
#                                                ^^^^^^^^^^^^^^^

See also #71336.

[^1]: And dictionary since 4.4 dev 1.

It may also be helpful to add a note about such arrays (and in 4.4 dicts) to the Saving Games example which is where I first ran into this issue.

@dalexeev Appreciate the input, and your workaround has led me to a similar workaround. Given that, here's some more context for this issue.
When adapting the saving games guide I linked in my previous comment, I tried the workaround you suggested in your previous comment.
For the arrays in question, turns out this does not work:
	new_object.set(i, node_data[i] as Array[Effect.EFFECT])

but, this does work:
	var placeholder: Array[Effect.EFFECT]
	for item in node_data[i]:
		placeholder.append(item as Effect.EFFECT)
	new_object.set(i, placeholder)

I'm not yet sure why this is, because I can use the original example above to show that when casting the full array, if the elements inside are integers, they are also cast to enum.
self.set("my_typed_list", [0 as int, 1 as int] as Array[THING]) # <--- ends up correct

For the arrays in question, turns out this does not work:
	new_object.set(i, node_data[i] as Array[Effect.EFFECT])

@brevven This only works for literals, I wrote about this above:

note that the as operator only works on array literals, not arbitrary expressions

For arbitrary expressions, you can use the typed array constructor:
Array(other_array, TYPE_INT, &"", null) # Array[int]
Array(other_array, TYPE_OBJECT, &"Node", null) # Array[Node]
Array(other_array, TYPE_OBJECT, &"Node", MyNode) # Array[MyNode]

But the constructor is primarily intended for dynamically creating typed arrays (when you don't know the element type in advance).

Thanks, somehow skipped right over that clarification, my apologies.


[Bug] Thumbnail displayed in grayscale

Describe the Bug
Scans are displayed in RGB in the viewport, however thumbnail is displayed in grayscale
Steps to Reproduce

Upload any DICOM endoscopy file which have color image (RGB - 3 channel).
Images are displayed properly in the viewport
Thumbnails are not displayed in color

The current behavior
Scans are displayed in RGB in the viewport, however thumbnail is displayed in grayscale

The expected behavior
Thumbnail should be displayed in RGB
OS
Windows 10
Node version
18.15.0
Browser
Chrome 114.0.5735.199

Merged the changes done on 11May2023 to v3-stable branch - commit id-84cb709b44074c7553fabd28bbcd2da4c1bd60d6.
This has updated to cornerstone libraries.
After this the thumbnail is working fine.


TextInput inside of AlertiOS

Hi, is there any way we can use a TextInput inside a AlertiOS? I checked the docs for AlertiOS but couldn't find any mention of it. Is that a negative? Also, there are no references to it on SO as well. So any help here would be appreciated.

Are you looking for AlertIOS.prompt?
AlertIOS.prompt("Type a value", null, [
     { text: "Confirm", onPress: (value) => { console.log("You typed", value) } }
]);

@sankhadeeproy007
Lets go ahead and start tracking this here #3893.