How to distinguish between both volumes for flashing

Hello there,
I am willing to build an automation that will deposit the left and right firmware files whenever their respective volumes are mounted. The problem is, I cannot figure out a way to distinguish between the two volumes. Indeed, they both have the same name (ADV360PRO by default) and same volume UUID. I haven't tried renaming one of the volumes, but if possible I'd like to avoid doing that.
Here's what I've tried so far:

Looking up the volume UUID via system utilities (in my case, macOS):

diskutil info $(mount | awk '/ADV360PRO/ {print $1}') | awk '/Volume UUID/ {print $3}'

This prints the same UUID for both volumes.

Trying to map the USB product ID and vendor ID to a mounted volume:
It seems this isn't possible.

Attempting to parse info from the hex representation of the CURRENT.uf2 firmware file:
This approach is not practical and gave up even after trying the uf2tool from Microsoft.

Any help would be appreciated. Thank you in advance.

The boards can be differentiated outside of bootloader mode but once in bootloader mode it's not possible using the stock bootloader. A 360 pro user came up with a neat script to differentiate outside of bootloader mode
https://github.com/KinesisCorporation/Adv360-Pro-ZMK/discussions/119#discussioncomment-5383832

Unfortunately, the script you linked is gone. So I've tried to go the hex parsing route again, but turns out that both firmware files contain the same exact UF2 32 bytes header.
On macOS, I've discovered that a given USB peripheral has a volume mounted if it has a "Media" key in the command system_profiler SPUSBDataType.
With a bit of jq - for which I'm by no means an expert - here's what I've been able to script out:
system_profiler SPUSBDataType -json | jq '
  any(.SPUSBDataType | .. | objects | with_entries(select(.key == "_name")); ._name == "Adv360 Pro") as $left |
  any(.SPUSBDataType | .. | objects | with_entries(select(.key == "_name")); ._name == "Adv360 Pro rt") as $right |
  any(.SPUSBDataType | .. | objects | with_entries(select(.key == "_name" or .key == "Media")); ._name == "ADV360PRO" and (.Media[].bsd_name | length) > 0) as $bootloaded |

  if $right and $bootloaded then "left"
  elif $left and $bootloaded then "right"
  else "none"
  end
'

Given that:

When bootloader mode is not active, left side is listed as Adv360 Pro and right side as Adv360 Pro rt.
If bootloader mode becomes active for either left or right side, it becomes listed as ADV360PRO.

This command outputs:

left if right side is listed and the other side (= left) is in bootloader mode.
right if left is listed and the other side (= right) is in bootloader mode.
none otherwise.


move category field to the playlist model

https://github.com/thenewboston-developers/Website-API/blob/development/v1/videos/models/video.py#L22
Here, the category is added on the video model, but it should be on the playlist model instead.
Check design for reference: https://www.figma.com/file/tfm2m2vGTdaZT8u5iQZS3B/Youtube-Playlists?node-id=631%3A0

@thesanjeevsharma Since we will be getting content from youtube I went per the documentation in which it's the video that has a category. But we can have them in both places since the data will be added manually

@ThaDeveloper In the design, I can only see playlists listed, not sure if we'll have independent videos listed. Also, the category on video that you get from youtube is - Gaming, Education, Kids, etc. (not 100% sure, but I think that's that). The categories we've added to playlists are different. Those are tech domains mainly. So, you are right, it will be added manually. I think we can add that field on the playlist model too.
@angle943

We will be manually adding in our own categories, not using the one from YouTube
Yes I believe videos can exist without a playlist. So let's have categories in both places


error occurs in table_highermoments.sas

When input data set "prices" only contains variables of Date, IBM, GE, there will be error in the macro.

Done by #280


Top level docs and re-exports

Usually if you have a nested library you'll re-export the most important types at the top level to make life easier.
In this case the api module only contains a single thing so you may want "inline" the api module and just export the BittrexAPI directly.
i.e. Make your lib.rs look something like this:
...
pub mod error;
pub mod values;
mod api;

pub use api::BittrexAPI;

Also, if you were to describe the BittrexAPI, would you say it's an API or a Client? I was thinking you'd call it a BittrexClient seeing as it's a client which is interacting with the Bittrex API and the *Client is a fairly common naming scheme.

Thank you very much for the re-export hint! This was exactly what I was missing here. And you are totally right. The BittrexAPI should definitely be BittrexClient.


Revert "Refactor account deletion and clearing"

Reverts ethereum/execution-specs#640

closing this in favour of #668


[WIP] Improved epoch-based GC

This is a proposal for improvements on the epoch-based garbage collector.
Rendered

This RFC is still a work in progress.
The "Motivation" sections offers a comprehensive overview of different aspects of epoch-based garbage collection, and what problems come up with it. I wrote the text mostly by recollecting the previus discussions on GitHub, Rust forums, and IRC.
I'd like to get some eyeballs on all this and hear what you think. The text is already long and dense, but there might still be important topics I missed, so let me know!
The "Detailed design" section is empty, but I will write it up after we get a clearer picture on what kind of design we want.
Here are my thougts on all this:

The way epochs are tracked in Coco is pretty good. Pinning and garbage collection is very efficient, and we need something like that in Crossbeam.

Incremental collection is a must. I like the idea of keeping collected garbage in the staging area until the thread gets unpinned.

Regarding specialized GCs (local to data structures), I'd like to discuss the topic in detail, but probably leave hard decisions for a later RFC. I'm unsure whether we'll need them at all, for various reasons.

I'd like support for destructors in the global epoch GC, and the DeferredDropSafe seems like a good solution.

@joshtriplett If you have time and interest in participating in this RFC, your expertise might be of great help!

Some initial thoughts:
+1 for "style 4" tracking, it's understandable and there's an working implementation. Any possible extreme scaling issues could be tackled later.
+1 for thread garbage buffers for the reasons already stated in the rfc.
-1 for delayed garbage collection, we really shouldn't try to tackle problems that we don't have (yet). I also fell that people requiring that level of precision need much more specific things for us to discuss at this point (knobs being a big one).
Regarding drops, I fell that trying to protect against bad interactions is a lost battle. As long as they're ran there's no way to make it completely safe. That being said, we should provide just the bare minimum safety net (Send + 'static) and the collection/user using crossbeam know better and should figure/document the rest.

To collect some garbage, a thread should pick up some garbage from its thread-local cache and, if it has time for more, also steal some from the global pool.

has time requires explanation, or just remove any timing constraints for now.

Some more thoughts:
+1 on style 4 tracking and thread-local garbage buffers for reasons stated in the rfc.
I give a +1 on basic delayed GC since it's pretty simple to implement and is quite useful. You can just pin without doing any local/global GC (or at least no allocations, deallocations, or extra atomics).
+1 on incremental GC as well. I have an implementation here that can easily be updated for any sane thread-local buffer scheme. At first we should probably just consider 'time' to be some limit on allocations and later think about advanced knobs and tuning.
Addressing how to prevent garbage stuck locally could be done at first with a global queue structure like the one used in Coco. It's probably worth adding some checks so that threads don't pointlessly push garbage into the global queue.
On thread registration, I think there might a way to do this with a vector of pointers instead of a linked list.  That would be great for reducing latency of the epoch check although would complicate thread registration.

A little while ago I was talking with @stjepang over RFC and I proposed that each thread tracked the amount garbage they generated. By doing that each thread can estimate a minimum amount they need to collect and how urgent it is.
The tracking can be done in several ways: it can be as simple as the number of deallocations or maybe a bit better by considering the size of the deallocation; maybe by number of bags; maybe only considering stuff pushed to the global pool to amortize the cost, etc..
If thread A produced X units of garbage it it tries to collect something like min(Y * X, MIN) units, starting with the local and then the global garbage.
Y needs to be > 1, so threads can help each other, proportionally to the amount of garbage they generate themselves.
Auto collection can be triggered manually, every P pins or .

@schets

I give a +1 on basic delayed GC since it's pretty simple to implement and is quite useful. You can just pin without doing any local/global GC (or at least no allocations, deallocations, or extra atomics).

Cool, glad you like this idea! You expressed interest (over IRC) in using Crossbeam for low/predictable latency stuff. Would something like epoch::delay_gc be good enough? Do you have any use cases where we should provide even stricter control over GC?

At first we should probably just consider 'time' to be some limit on allocations and later think about advanced knobs and tuning.

I like your implementation - that's a good way of measuring garbage and time spent on collection. But I think we should have even more precision when dealing with garbage. Not all garbage is created equal, as explained in the "Garbage objects" section.
For example, if we're destroying a queue node or skiplist node, that's worth one unit. However, if we're destroying a B-tree node containing 100 keys, it's worth 100 units. Likewise, destroying a large array of length 10K backing a Chase-Lev deque would be worth 10K units.
These units are not perfect. For example, destroying a skiplist node containing an i32 is not the same as destroying one that contains a [u64; 100] or a String. Perhaps it'd be wise to measure garbage in terms of memory use or the cost of destructors, but that may be too difficult to estimate. I think measuring garbage in terms of "elements" (e.g. [u64; 100] has 100 elements) is generally a pretty good approximation.
The function that adds garbage might have the following signature:
// Array of `T`s stored at address `object`, containing `length` elements.

If the bag now has more than HUGE_BAG units, it is sealed and moved directly into the global queue and a new empty thread-local bag is created.
Otherwise, if it has more than FULL_BAG units, it is sealed and a new empty thread-local bag is created.
While the sum of costs of all sealed thread-local bags is more than MAX_SEALED, we push the oldest sealed bag into the global queue.

On thread registration, I think there might a way to do this with a vector of pointers instead of a linked list. That would be great for reducing latency of the epoch check although would complicate thread registration.

That may work, but it's tricky to do. The problem is that we must load the current vector, clone it, push a new entry, and then use CAS to install the new vector. But someone may destroy the current vector while we're cloning it!
With linked lists this is easier. We only need to get the address of the current head and use CAS to install a new node as the head. We don't read the contents of the nodes, so other threads can't wreak havoc by destroying them.
@arthurprs

-1 for delayed garbage collection, we really shouldn't try to tackle problems that we don't have (yet). I also fell that people requiring that level of precision need much more specific things for us to discuss at this point (knobs being a big one).

Fair enough. We certainly don't need fine control over GC right now, but I'd still like to at least think about these problems in order to see what kind of flexibility we need to build into the core GC design.

Regarding drops, I fell that trying to protect against bad interactions is a lost battle. As long as they're ran there's no way to make it completely safe.

What do you mean by "there's no way to make it completely safe"? If we restrict deferred drops to types T: Send + 'static, executing these drops will be completely safe. Perhaps you meant "there's no way to make it safe without overrestricting types"? If so, I agree.

If thread A produced X units of garbage it it tries to collect something like min(Y * X, MIN) units, starting with the local and then the global garbage.

Yep, that sounds great! If we are going to use units as described above (1 unit = 1 "element") for deciding when to seal bags and move them into the global queue, then we could simply count sealings to decide how much garbage to destroy. For example, if the thread has sealed 5 bags, then we can destroy Y * 5 bags of garbage in order to make up for it.

Auto collection can be triggered every P pins or after X exceeds a limit.

Fully agree - we want both triggers.

I like and agree with your ideas, but for a first cut it's too much on the complicated side.
Could we stick with something simpler for the first cut? Like basic auto-weighting based on the size of T? See how it works out and take it from there.

Thank you for the efforts for this PR.  It's huge :)  Here are my thoughts:

+1 for "style 4" and the staged area for garbages.  Though I didn't quite understand the diff between style 1 and 4.

"Slow thread pinning": you said that the pin operation in Crossbeam executes 6 atomic operations and a full fence, while that in Coco executes 3 atomic operations and a full fence, and that's the reason of the performance difference.  But all atomic operations seems to be Relaxed, so I am not really sure the diff is due to the atomic operations.  Rather, I guess that the special code for x86 in Coco shines in this place (https://github.com/stjepang/coco/blob/master/src/epoch/thread.rs#L102).  What do you think?

Side note: the x86 code for pin() seems working, but I think we need a method to establish the correctness beyond the testing.  We don't even know yet what is the semantics of a program mixing C/C++ and x86 concurrency primitives.  Maybe it is beyond our concern, but I just want to note the difficulty.

On the interface of delay_gc(): How about this alternative set of methods?

pin(): as is
pin_no_gc(): pin(), but does not perform GC
gc(): it may or may not be in a Crossbeam scope.  It waits for the other threads for performing GC.
try_gc() -> bool: similar to gc(), but doesn't wait for the other threads.  Returns if GC is performed.

Motivation: I feel it is cumbersome to nest pin() inside delay_gc().  Rather I would like to specify the location where GC is worth performed.

+1 for "Large objects must be collected more eagerly than small objects in order to reclaim wasted memory faster."  Maybe we can send the GC priority to unlink() as an argument.  What do you think?

On oversubscription: can we have per-processor garbages rather than per-thread ones?  Then maybe we can overcome the oversubscription problem.

May I ask why? "Crossbeam's GC must work seamlessly and be as unintrusive as possible. For example, using process signals to coordinate garbage collection is out of the question."  As far as the user can ignore the implementation details, I think we can utilize everything we can have.
I know this issue is subtle, and I also want to stick to the standard language definition.  But to my eyes, using x86 assembly (instead of C/C++ concurrency), sys_membarrier, or process signals all deviate from the standard, and I don't object to one more than the others.

For thread registration:

That may work, but it's tricky to do. The problem is that we must load the current vector, clone it, push a new entry, and then use CAS to install the new vector. But someone may destroy the current vector while we're cloning it!

I agree with @schets that vector-based DS will improve the latency of epoch check.  I guess we can avoid the problem @stjepang anticipates by employing the trick used in the fetch-add-based wait-free queue (http://chaoran.me/assets/pdf/wfq-ppopp16.pdf).  It may be better to be discussed in a different RFC.

I haven't read "Destructors"..  I will reread it and leave more comments.

On oversubscription: can we have per-processor garbages rather than per-thread ones? Then maybe we can overcome the oversubscription

I thought about that before but my impression was that it would be a performance hit in most cases. It's more expensive to fetch the corresponding struct (there are some tricks we can use to avoid calling sched_getcpu often) and it may require extra synchronization (the preemption unit is still the thread). Once you take into account that crossbeam pin() is measured in low ns it starts to look really hard.

@jeehoonkang

Though I didn't quite understand the diff between style 1 and 4.

Style 1 is current Crossbeam (three distinct vectors of garbage), and style 4 is Coco (just one long queue of garbage).

But all atomic operations seems to be Relaxed, so I am not really sure the diff is due to the atomic operations. Rather, I guess that the special code for x86 in Coco shines in this place

The CAS trick definitely helps a lot, but is not the only contributor to better performance. Without it pinning takes 20 ns, which is still quite a bit faster than pinning in Crossbeam.

We don't even know yet what is the semantics of a program mixing C/C++ and x86 concurrency primitives. Maybe it is beyond our concern, but I just want to note the difficulty.

Valid point. That CAS is definitely a bit sketchy. :) We'd probably be safe if we used inline assembly to directly call lock xchg, though.

Motivation: I feel it is cumbersome to nest pin() inside delay_gc(). Rather I would like to specify the location where GC is worth performed.

Hmm, I think we have two different kinds of actors:

Crossbeam users. They might want to avoid GC in some sections of the code. They usually don't care about pinning.
Crossbeam data structures. They don't care about avoiding GC. They call pin in order to protect nodes from being reclaimed.

Users of Crossbeam would simply call delay_gc and that's it. They wouldn't nest calls to pin - that's what data structure implementations do.
I suspect some confusion arose because the text is a bit weak - I need to explain all this better.

Maybe we can send the GC priority to unlink() as an argument. What do you think?

Yes, absolutely. We need some measure of priority/urgency/size. I'll try to write a concrete proposal on that.

On oversubscription: can we have per-processor garbages rather than per-thread ones? Then maybe we can overcome the oversubscription problem.

The problem mostly stems from the fact that a pinned thread might get preempted and prevent epoch advancement until it gets resumed. This may block garbage collection for a long time.
Having per-processor garbage wouln't help much. What would help, though, is if we could tell the OS: "If possible, please don't preempt the thread while executing this critical section."
AFAIK, RCU in the Linux kernel works that way - it is guaranteed that no critical section ever gets preempted.

As far as the user can ignore the implementation details, I think we can utilize everything we can have.

Certainly. I think my main gripe with signals is that they reserve a signal (e.g. USR1), so users of Crossbeam must be aware of that because now they can't use that signal for themselves. In that sense signals are intrusive - they may restrict code unrelated to Crossbeam in some way.

I agree with @schets that vector-based DS will improve the latency of epoch check.

Unless I misunderstood something, that vector would simply be a vector of pointers to thread entries, which is no better than a linked list of thread entries.
Perhaps you had in mind a long vector with adjacent entries in memory? Registration would then involve iterating over the vector and reserving a vacant entry. If so, yeah, that would probably make checking epochs faster.
This week I'll make another iteration over the whole text, try to clean it up, and make a more concrete proposal.

I've got a high-level comment on this - this seems like another mega-rfc. Many of the items seem orthogonal to each other (pinning method, priority tracking, object holding methods)
@stjepang a vector of pointers is significantly better than a linked list in terms of iteration time! With a linked list, each iteration of the loop carries a data dependency on the last while in a vector there is no data dependency. This means that for any out-of-order processor one can schedule multiple iterations of the loop in parallel and as a result have multiple cache misses in flight. Since this list isn't being read frequently and almost every is being written to by a different thread than the reader, the case where each element is in the shared cache or main memory is the normal use case.
I have another gripe with signals - they can come at inconvenient times! With crossbeam as-is, you only incur epoch+GC costs when you enter an epoch. A signal might hit at any time including latency-sensitive periods.
I'm not sure if forcing priority in default unlinked is a good idea. Users probably don't want to try and decide how to prioritize garbage amongst stuff from other unknown data structures. We could add a different unlinked_priority function, but I expect that in normal use that would almost always prioritize by size.
@stjepang @jeehoonkang is right about the atomics - relaxed atomics compile to standard loads/stores and the performance gain is from merging the addition and the fence on x86. There may be a minor performance gain from merging the load of the in_critical counter and the thread-local epoch counter but I don't think that's enough to show in benchmarks
@stjepang I think I misunderstood the difference between GC #1 and GC #4. They appear extremely similar with some small implementation details in how garbage is internally tracked and pinning announced to the global GC. They both collect garbage locally, announce the current epoch by recording the current global one, and pin by fencing. We should delegate small implementation details there to another rfc because I don't actually think it's clear which is better.

First, apologies for the late reply. I'll try to be more responsive in the future.
@schets is right - this is too much material to digest in a single RFC. I've simplified the text and would like us to start with a reasonably good and simple design.  What is proposed here is the first step towards better garbage collection. I'm open to future alterations of the design, API, and implementation details.
@schets Regarding the GC variant we should use - I see the GC variant #4 as basically a generalization of variant #1 - instead of having three vectors where two of them are blocked we have an infinite number of vectors. I'm leaning towards #4, but you're right that it's not a simple decision - let's just try both and see what works best. shrug. :) Implementation details are up in the air.
Regarding deferred drops - let's keep it simple for now. There will be a time in the future to explore this direction in more detail.
For urgent reclamation (e.g. deque must be able to quickly reclaim old buffers) let's just use flush-based approach (just like in Coco) for now. Ideally I'd prefer to have something like @schets' incremental branch in Crossbeam.

Very nice! Great, all-around understandable RFC with a clear direction forward. I agree with the layout for the new design. I'm not quite sure I got the section on "Destructors and local GCs" though.

Those that allow borrowing elements. Examples are sets and maps. Find operation simply returns a reference to the found element. Removing an element involves finding it and marking it as removed.

This is mentioned, but the following section only outlines solutions to the problem of elements containing non-static references, rather than the problem of returning references from the structure:

The epoch GC will destroy garbage at some time in the future, but it doesn't guarantee when exactly. Since T might hold non-static references, we mustn't destroy Ts after those references expire. There are three solutions to the problem:

To expand on returning references from a structure, since the reference is now in user code, it might live as long as the structure. For this reason, we cannot destroy any element before the structure destructor runs - this is very bad, since a lot of garbage will accumulate until then. Combined with the problem of non-static references in elements, we have to destroy everything that the structure ever touched in the structure's destructor. For this reason, I think we really shouldn't allow this. A crossbeam issue outlines two solutions - either return some kind of EpochGuard, also pretty bad, since it may pin for long if the user is careless, or provide a with(key: &Key, fn: F) where F: FnOnce(Option<&Val>). The closure can then operate on the reference, but hopefully (and we document this) returns relatively quickly to not pin for too long.
If we restrict ourselves to just allowing some non-static references inside elements, then we have to make sure everything is destroyed before the structure, but there isn't the problem of not being able to destroy anything until the structure itself is destroyed.
Another thing that we discussed on IRC sometime ago is that we should make it clear that even with non-static references in deferred drop, not all kinds of them are allowed - specifically, only references that outlive the structure (or references to the structure itself) are valid. If for example we were to reference a method's local stack, that would crash horribly after the method finishes and we are left with a structure that still references some local variable from that method. This is important, so I feel like it should be mentioned.

@Vtec234 Good suggestion - I've added a paragraph or two that expand on the topic.

Has anyone else had the time to review the recent changes to the text?
I'd really like us to push this forward, and soon start implementing the basic features that are missing in the current GC. Then we can improve it over time and try all the competing implementation details (like how to track epochs, how to store garbage in memory, how to represent a list of registered threads, etc.).

Great RFC.  My only comment is, what is the semantics for the case that a Ptr<T> is called defer* multiple times?  If it is not safe to do so, where should we put unsafe?

@jeehoonkang
If I got that right, your question is basically "where are the boundaries of unsafe code"? It's really difficult to say - I believe we still don't have a fully conclusive answer to that question.
We've always had this rule of thumb: "if you have an unsafe block anywhere in your code, then the whole surrounding module is already polluted". Gankro talks about that in his talk (around 10:00).
Niko also published a great deal of blog posts on this topic (e.g. this one).

@stjepang FYI, on the semantics of unsafe, the RustBelt people made their draft paper online (very) recently: https://people.mpi-sws.org/~dreyer/papers/rustbelt/paper.pdf  I am reading it now.

This looks great, and we can (and should) hash out more minor differences in more specialized rfcs and PRs


fixes #50 - Laravel 5.6.24 renamed function

This fixes #50. The function allowServerToRun in the scheduler was renamed so this fails en no crons for one server are called. The code in the command was refactored in laravel/framework@6563ba6#diff-0de50829a544cd21193aef291b5a9baf so the new created function could be used here too.

Awesome. Thanks @glamorous

@laurencei Will you create a new release with all the latest fixes?

@glamorous - as this is the "beta" branch - I think should always pull the latest commit if you run composer update (as it was installed from the beta branch)?

Indeed, my bad! 👍


How to get import statements detected?

When I install xdm as a dependency, running the following:
xdm.compileSync(`import foo from 'foo'`)

interprets the import statement as text:
/*@jsxRuntime automatic @jsxImportSource react*/
import {Fragment as _Fragment, jsx as _jsx} from "react/jsx-runtime";
function MDXContent(_props) {
  const _components = Object.assign({}, _props.components), {wrapper: MDXLayout} = _components;
  const _content = _jsx(_Fragment, {
    children: "import foo from 'foo'"
  });
  return MDXLayout ? _jsx(MDXLayout, Object.assign({}, _props, {
    children: _content
  })) : _content;
}
export default MDXContent;

But the same thing works fine inside the xdm repo! 😅
/*@jsxRuntime automatic @jsxImportSource react*/
import {Fragment as _Fragment, jsx as _jsx} from "react/jsx-runtime";
import foo from 'foo';
// ...

Any ideas why? 🙏
I'm using the latest xdm, 1.7.0.

The logic for parsing ESM imports and exports lives in https://github.com/micromark/micromark-extension-mdxjs-esm
From https://github.com/micromark/micromark-extension-mdxjs-esm/blob/c1b0734136b3d954f56c3384361b6bb4e17698b9/test.js#L315-L321 it appears this should be supported :thinking:

I'm guessing you have an older version of mdx-js/mdx in there too. It uses locked versions of dependencies, whereas xdm uses wide ranges (I think the problem is specifically mdast-util-to-hast through remark-rehype).
So, u installing that, reinstalling, should do the job!

Fix deployed!

It works, awesome! 🎉


Bump the version of ripgrep

As per https://github.com/Microsoft/vscode/commit/a8978c6a9ee219d3f467571216a6d5582c67f236

I just bumped this one since it has to be changed in yarn.lock too - thanks


rails 6

From what I understand the explicit dependency on arel (which is baked into activerecord in Rails 6) totally breaks activerecord 6.0. See https://github.com/rails/rails/issues/34983#issuecomment-523512469


cmd/internal/obj/arm64: minor mistake in pre-increment example

The following URL contains a minor mistake: https://golang.org/pkg/cmd/internal/obj/arm64/
Under point 2 ("Go uses .P and .W suffixes to indicate post-increment and pre-increment.") there is a mismatch between the Go assembly and the GNU syntax (for the pre-increment with .W suffix).

So either the example should be this
MOVB.W 16(R16), R10        <=>      ldrsb x10, [x16,#16]!

or
MOVD.W 16(R16), R10        <=>      ldr x10, [x16,#16]!

or even
MOVBU.W 16(R16), R10        <=>     ldrb x10, [x16,#16]!

/cc @dr2chase @cherrymui

cc @zhangfannie

@fwessels  Thank you for pointing out the mistake in https://golang.org/pkg/cmd/internal/obj/arm64/. The arm64 go assembler has the correct encoding for  instruction " MOVB.W 16(R16), R10 ", we just have a typo in this document. I will submit a patch to fix this error. Thanks.

Yes, correct, it is just a documentation error, luckily not in the assembler itself.


Fix #16

This fixes #16 and makes some minor improvements in other areas, tested with 2.2.14

Tested and everything was working, no duplicates anymore.
I am doing a new release now.


Redirect seed-enterprise-investment-scheme-procedures

Migration to redirect /guidance/seed-enterprise-investment-scheme-procedures
to /guidance/venture-capital-schemes-apply-to-use-the-seed-enterprise-investment-scheme
as per request in zendesk ticket https://govuk.zendesk.com/agent/tickets/2406584

@gpeng Thanks - I've updated the approach now. This should take care of the zendesk ticket.
The actual bug that is stopping the publisher from doing this will hopefully be taken care of by https://github.com/alphagov/whitehall/tree/return-empty-array-when-expanded-links-needs-id-are-nil once it all builds and reviewed.


helm upgrade --install not working

Command
helm upgrade ac-passive .helm/ac-passive --install --set image.tag=7f537bc346a70db2c367b48ddbf2033adfaf570a

Error
Error: UPGRADE FAILED: "ac-passive" has no deployed releases

Output of helm version:
version.BuildInfo{Version:"v3.1.1", GitCommit:"afe70585407b420d0097d07b21c47dc511525ac8", GitTreeState:"clean", GoVersion:"go1.13.8"}

Output of kubectl version:
Client Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.2", GitCommit:"c97fe5036ef3df2967d086711e6c0c405941e14b", GitTreeState:"clean", BuildDate:"2019-10-15T23:42:50Z", GoVersion:"go1.12.10", Compiler:"gc", Platform:"darwin/amd64"}

Server Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.9", GitCommit:"2e808b7cb054ee242b68e62455323aa783991f03", GitTreeState:"clean", BuildDate:"2020-01-18T23:24:23Z", GoVersion:"go1.12.12", Compiler:"gc", Platform:"linux/amd64"}

Cloud Provider/Platform (AKS, GKE, Minikube etc.): Digital Ocean k8s

➜  ac-passive git:(master) helm del ac-passive
release "ac-passive" uninstalled
➜  ac-passive git:(master) helm upgrade ac-passive .helm/ac-passive --install --set image.tag=7f537bc346a70db2c367b48ddbf2033adfaf570a
Release "ac-passive" does not exist. Installing it now.
NAME: ac-passive
LAST DEPLOYED: Sat Feb 22 12:59:25 2020
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None

I just tried running helm upgrade with the --install flag on the mariadb package in the stable repository and had no issues with it installing. I was using v3.1.1. If you use helm install do you get any errors? I ask because there could be a problem with the chart and we want to check that.
Is the chart ac-passive something you can share?
Some more information may help troubleshoot this issue.

@mattfarina I tried to update the chat using helm update --install, but got an error. After I tried to delete the chat and start it again, the error was fixed.

@polRk Do you have any logs or details that might shed light on this?
If not and your case is solved please feel free to close this issue.

You might have deleted the release ac-passive already with --keep-history, so it keep release history, but all release status is not deployed, so when you upgrade --install , --install is useless。
And when upgrading, it will firstly query deployed release, if not exsited,  xxx has no deployed releases is throwed.
// prepareUpgrade builds an upgraded release for an upgrade operation.
func (u *Upgrade) prepareUpgrade(name string, chart *chart.Chart, vals map[string]interface{}) (*release.Release, *release.Release, error) {
	if chart == nil {
		return nil, nil, errMissingChart
	}

	// finds the deployed release with the given name
	currentRelease, err := u.cfg.Releases.Deployed(name)
	if err != nil {
		return nil, nil, err
	}

func (s *Storage) Deployed(name string) (*rspb.Release, error) {
	ls, err := s.DeployedAll(name)
	if err != nil {
		return nil, err
	}

	if len(ls) == 0 {
		return nil, errors.Errorf("%q has no deployed releases", name)
	}

	// If executed concurrently, Helm's database gets corrupted
	// and multiple releases are DEPLOYED. Take the latest.
	relutil.Reverse(ls, relutil.SortByRevision)

	return ls[0], nil
}


Errors should move when editing the document

After editing a document, the errors will still show on the previous line number, instead of moving.
Would it be possible to detect line changes and shift the errors?
Example: before

Example: after
Expected: error should show on line 17

I would need steps to reproduce the issue (to fix). Decorations usually follow errors properly.

@erikschul I believe this is a rust-analyzer issue. This happens when rust-analyzer executes cargo check and reports the errors in the UI. Then, when editing the file, rust-analyzer only has the old cargo check results to work with.
https://github.com/rust-lang/rust-analyzer/issues/4056


Update hardware-requirements-for-the-teams-app.md

https://github.com/MicrosoftDocs/OfficeDocs-SkypeForBusiness/issues/7818
Added info

@JohanFreelancer9 Copy editing is needed for this PR. Thanks.

@SerdarSoysal PR has been copyedited and is ready for final review, could you please check and merge? Thanks!

Sent email to Mike RothKugel and Bill Weidenborner to review.

This update is incorrect, so I'm closing this PR.


[vkquake] set an arg to be at desktop resolution at launch

I suggest to set vid_desktopfullscreen to 1 to ensure to be at desktop resolution at launch. Otherwise it is set to 800x600 by default.
New Engine Package Submissions

[x] Have you followed the instructions in the build documentation?
[x] Have you run the build locally and ensured the package allowed you to launch and play the Steam game?
[x] Have you updated the packages.json file?
[x] Have you ensured that there is not already a pull request or active engine package for the one you are adding?

Common Code Submissions

[x] Have you verified that the changes are isolated to the features or fixes you are wanting to do?
[x] Have you tested at least one of the engines to ensure that your changes do not break existing workflow?
[x] Have you described what your changes are accomplishing?

Looks good to me

Looks good to me


[SPARK-39894][SQL] Combine the similar binary comparison in boolean expression.

What changes were proposed in this pull request?
If boolean expression have two similar binary comparisons have the same symbol(e.g., >) and one side is literal and connected with And.
For example, 'a > 1 and 'a > 2.
In fact, we can combine them as 'a > 2.
The idea is compare literal value and combine them.

'a > 1 and 'a > 2 combined to 'a > 2
1 < 'a and 2 < 'a combined to 'a > 2
'a >= 1 and 'a >= 2 combined to 'a >= 2
1 <= 'a and 2 <= 'a combined to 'a >= 2
'a < 1 and 'a < 2 combined to 'a < 1
1 > 'a and 2 > 'a combined to 'a < 1
'a <= 1 and 'a <= 2 combined to 'a <= 1
1 >= 'a and 2 >= 'a combined to 'a <= 1
'a === 1 and 'a === 1 combined to 'a === 1
1 === 'a and 1 === 'a combined to 'a === 1
'a === 1 and 'a === 2 combined to FalseLiteral
1 === 'a and 2 === 'a combined to FalseLiteral

This PR also considers swap binary comparison if the literal on the left of And.
For example, 1 < 'a and 2 < 'a will be combined as  'a > 2 too.
Why are the changes needed?
Simplify the binary comparisons in boolean expression.
Does this PR introduce any user-facing change?
'No'.
Just change the inner rule.
How was this patch tested?
New test cases.

ping @MaxGekk @gengliangwang @dongjoon-hyun cc @cloud-fan


Custom PGDATA or change mount path of /var/lib/postgresql/data?

Is it possible to customize the PGDATA value used by the bootstrap image?
By default, the directory /var/lib/postgresql/data/pgdata is used to initialize the database because /var/lib/postgresql/data is backed by a PVC, making it writable.
I’d like to change it to /var/lib/postgresql/data/db/pgdata. The reason is that I want to create a separate folder at /var/lib/postgresql/data/plugins where I can copy PostgreSQL extension files while the pod is running. Copying these files requires a writable path (PVC) but don't want to conflict with the actuall database files.
I cannot copy the extensions into the same directory as PGDATA, as it causes a conflict with the init container’s command, which reports that the "data folder isn’t empty."
Another option is could it be possible to change the mount path used by the pods? For example mouting /var/lib/postgresql/ path

You won't be able to do that from the chart, but you can modify the Docker image to use a different path in the PostgreSQL server configuration.

I think this is the data_directory parameter.


Issues fixed and code enhanced

[x] download_general_pdf styling issue fixed & download_paybook_pdf code enhanced
[x] group member will not be deleted if there is a savings account with balance
[x] deactivating the member based on loan, savings account balance code enhanced

Coverage decreased (-0.9%) to 85.121% when pulling 2874f645b3fa5692ffdfcb23d43272cfcd76468f on vidyasagar-r:enhancements_branch into 648cdd9f61bdc2fec863be0bce0e9bc880267d19 on MicroPyramid:master.

Coverage decreased (-0.5%) to 85.546% when pulling 2874f645b3fa5692ffdfcb23d43272cfcd76468f on vidyasagar-r:enhancements_branch into 648cdd9f61bdc2fec863be0bce0e9bc880267d19 on MicroPyramid:master.


how to guarantee every pixels of training images are being used?

As my understanding for training procedure, RGB input patches are randomly cropped from training images. Then, it seems that not every pixels in training images are being processed in one epoch. Is it for some reason?

This is also my question, is it possible we miss some pixels, or how we can make sure that overlapping has effects on  results or not?


Bump pretty dependency

Especially to include https://github.com/tidwall/pretty/commit/c41759c9c423289cac31ff6098a8b44150f66b61

Good catch. Thanks.


codeActionsOnSave shows file changes as unsaved

Issue Type: Bug
Since the update to VS Code 1.33 the editor.codeActionsOnSave setting does not work as expected. When saving a file the defined actions run but then the file is unsaved. Sometimes the file gets saved but not always. This worked fine in VS Code 1.32.
My VS Code settings are:
{
  "workbench.startupEditor": "newUntitledFile",
  "editor.formatOnSave": true,
  "[handlebars]": {
    "editor.formatOnSave": false
  },
  "files.exclude": {
    "node_modules/": true
  },
  "git.autofetch": true,
  "git.enableSmartCommit": true,
  "prettier.tslintIntegration": true,
  "typescript.updateImportsOnFileMove.enabled": "always",
  "editor.codeActionsOnSave": {
    "source.organizeImports": true,
    "source.fixAll": true
  },
  "[typescriptreact]": {
    "editor.defaultFormatter": "esbenp.prettier-vscode"
  }
}

If I change the source.organizeImports and source.fixAll settings to false the issue everything works fine.
VS Code version: Code 1.33.1 (51b0b28134d51361cf996d2f0a1c698247aeabd8, 2019-04-11T08:27:14.102Z)
OS version: Windows_NT x64 10.0.17134

System Info

Item
Value

CPUs
Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz (8 x 1992)

GPU Status
2d_canvas: enabledchecker_imaging: disabled_offflash_3d: enabledflash_stage3d: enabledflash_stage3d_baseline: enabledgpu_compositing: enabledmultiple_raster_threads: enabled_onnative_gpu_memory_buffers: disabled_softwarerasterization: enabledsurface_synchronization: enabled_onvideo_decode: enabledwebgl: enabledwebgl2: enabled

Memory (System)
15.73GB (5.74GB free)

Process Argv

Screen Reader
no

VM
0%

Extensions (8)

Extension
Author (truncated)
Version

cform
aws
0.0.15

gitlens
eam
9.6.1

EditorConfig
Edi
0.13.0

tslint
eg2
1.0.43

prettier-vscode
esb
1.8.1

dotenv
mik
1.0.1

debugger-for-chrome
msj
4.11.3

vscode-yaml
red
0.4.0

Can you try to reproduce with our nightly insider builds? You can give our preview releases a try from: https://code.visualstudio.com/insiders/

I just tried the nightly insider build without any extensions and it does the same thing. I also played a bit with the settings and it seems that the only setting causing this is the source.organizeImports, as setting it to false fixes the issue.

@manospasj can you give me an exact repro how you get there?

@manospasj ping

@bpasero I tried to create a repro but I can't reproduce it with a hello world app. It only seems to be happening with the project I'm working on. I'll do some more investigation and see if I can find something more helpful.

/duplicate https://github.com/Microsoft/vscode/issues/72343

@bpasero Sorry for my late response but I was not in the office for a couple of days. This is not a duplicate of #72343. I tried the latest insiders build as suggested in #72343 but this is not fixed.
I know I'm not giving you enough info to reproduce it,  but I'm not sure what's causing it and thus I can't create a repro to demonstrate it. I can only reproduce it in a work project which is not open source and I can't share. Maybe I can show you using TeamViewer?

@manospasj this is with a typescript project right?

Yes this is a react with typescript project. Here is my full package.json, if that helps. I have also noticed that it happens more often when I'm running the project.
{
  "name": "my-app",
  "version": "0.1.0",
  "private": true,
  "dependencies": {
    "@types/google-map-react": "^0.23.10",
    "@types/history": "^4.7.2",
    "@types/jest": "23.3.9",
    "@types/lodash": "^4.14.119",
    "@types/node": "10.12.10",
    "@types/react": "16.7.7",
    "@types/react-dom": "16.0.10",
    "@types/react-router-dom": "^4.3.1",
    "@types/styled-components": "^4.1.1",
    "cross-env": "^5.2.0",
    "google-map-react": "^1.1.2",
    "history": "^4.9.0",
    "i18next": "^15.0.6",
    "lodash": "^4.17.11",
    "node-sass": "^4.10.0",
    "react": "^16.8.4",
    "react-dom": "^16.8.4",
    "react-grid-layout": "^0.16.6",
    "react-i18next": "^10.4.2",
    "react-router-dom": "^4.3.1",
    "react-scripts": "^2.1.8",
    "react-testing-library": "^5.4.4",
    "rxjs": "^6.3.3",
    "styled-components": "^4.1.1",
    "tslint": "^5.11.0",
    "tslint-react": "^3.6.0",
    "typescript": "3.1.6",
    "yargs": "^12.0.5"
  },
  "scripts": {
    "start": "node scripts/environment --config development react-scripts start",
    "start:local": "node scripts/environment --config local react-scripts start",
    "build": "node scripts/environment --config development react-scripts build",
    "test": "react-scripts test",
    "eject": "react-scripts eject",
    "lint": "tslint 'src/**/*.tsx' 'src/**/*.ts'",
    "gen": "plop --plopfile ./src/utils/generators/index.js && prettier --loglevel error --write \"./src/**/*.tsx\""
  },
  "eslintConfig": {
    "extends": "react-app"
  },
  "browserslist": [
    ">0.2%",
    "not dead",
    "not ie <= 11",
    "not op_mini all"
  ],
  "devDependencies": {
    "@types/react-grid-layout": "^0.16.7",
    "inquirer-directory": "^2.1.0",
    "plop": "^2.1.0",
    "prettier": "^1.14.3"
  }
}

@manospasj When this happens, can you please run the Developer: Toggle developer tools command in VS Code to open the developer tools. In the developer tools console, do you see any messages about codeActionOnSave timeout or other errors?

@mjbvz yes there is a warning WARN Aborted codeActionsOnSave after 750ms at workbench.main.js:1401

Are other language features, such as intellisense, responsive in this workspace or are they slow/sluggish?
Can you also please make sure you have the latest version of @types/styled-components installed. See https://github.com/DefinitelyTyped/DefinitelyTyped/issues/34391 for details

Upgrading to the latest version of @types/styled-components did the trick. Thanks @mjbvz


Remove remnants of Python 2 support

Removed a no longer required statement
Removed future imports
Updated shebangs to explicitly require python3
Updated documentation samples to explicitly call python3
Used standard syntax for @classmethod parameter
Removed useless test in Python 3

This improves on the work of #408.

A, nice! I didn't even consider these.

I don't think you meant closing the PR but rather merge it.

I'm sure I merged it but apparently not. Good that you caught it.

No problem :)


Remove the API presenter and JSON format responses

This class appears to be created for a side project to provide a JSON formatted version of the question and results. This functionality doesn't appear to be used anywhere in the codebase, and logs indicate that JSON formatted responses haven't been used in 6 months.
We should remove the API Presenter class and logic from the controller to reduce complexity and remove unused code.

Removed by #4524


Fixing bug where writeBuffer was throwing an error for mode w+

Thanks, @fresheneesz!  On further review, we cannot accept this pull request. Concurrent append operations are unsafe as they can easily corrupt the files. Making this possible would require a re-design of the gridfs storage specification.

@imlucas
"Concurrent append operations are unsafe"
Of course, concurrent appends (or writes in general) are unsafe, but that's not what I'm doing. I'm simply trying to append to a file - nothing concurrent is happening. That line of code is causing things to fail. I'm confused why you're assuming that I'm doing multiple appends to the same file concurrently.

It does not matter if you are doing it in a single thread. it's an unsafe operation for the driver and a giant shotgun to shoot yourself in the foot and the reason we removed the feature. For every 1 person doing it right there was 10 people doing it wrong unfortunately.
There are some npm packages that pertain to work around this
https://www.npmjs.com/package/gridfs-locking-stream
However I'm not sure if it works and the state of the project.

@christkv So are you saying that the "w+" mode was removed entirely as a feature? Why isn't an error thrown if you pass in "w+" to the GridStore constructor then? And why do the docs still say that "w+" is an accepted mode?

w+ is only supported for meta-data changes not the file content itself as the meta-data can be changed in one atomic operation.

@christkv That makes 0 sense. The documentation says "'w+' (Append): Finds the last chunk, and keeps writing after it." It says nothing about meta data. You don't need to open up the file to write meta data anyway, as you can just modify the document in fs.files directly with a basic mongo update. So what are you talking about?

And please explain to me how a single append operation is in any way unsafe as long as other appends aren't happening concurrently. If that were true, your basic OS filesystem w+ writes would also be unsafe.

@christkv Can you please answer the following question that you previously ignored. I have to believe we're misunderstanding eachother:
"it's an unsafe operation"
What is the "it" you're talking about? What is an unsafe operation? Surely simply appending to a file isn't unsafe.

Append does not lock a file in gridfs. Multiple concurrent operations can try to append to the same file causing data to be overwritten. Unsafe means there is a very high likely hood of data corruption when using w+ in drivers that allowed it. It was removed due to users running into this exact issue in production systems. It's been out for several years already. Until the GridFS specification allows some sort of safe concurrent appending (either via locks or special support in mongodb) it will not be re-introduced as an option. GridStore is also a legacy API at this point as we implement the joint new API.
https://github.com/mongodb/specifications/blob/master/source/gridfs/gridfs-spec.rst

@christkv
"Append does not lock a file in gridfs."
This raises so many questions. Does overwrite (mode "w") lock a file in gridfs? It should, because otherwise you can get read corruption if a file is overridden in the middle of a set of reads. Isn't the whole point of opening a file to lock that file?
If mode "w" locks the file, why wouldn't the same be done for "w+"?
"GridStore is also a legacy API at this point as we implement the joint new API."
Is the new API implemented? What's it called? Gridfs? Are GridStore and GridFS different?
"it will not be re-introduced as an option"
Why doesn't new mongodb.GridStore(db, _id, "+w") throw an error if that option isn't valid?

w creates a new file with a unique objectId unless you override with your own custom id. No concurrency ever possible and thus no file locking through the API provided needed.
http://mongodb.github.io/node-mongodb-native/2.2/api/GridFSBucket.html and http://mongodb.github.io/node-mongodb-native/2.2/tutorials/gridfs/streaming/
w+ is valid but only for changing details about the document in the files collection not any of the actual binary chunks in the chunks collection.

There is no way to guarantee the integrity of a GridFS file append as the specification stands now. Any such guarantee would have to include options as mentioned above to make it feasible.
If you need append please have a look at the provided link above that addresses your specific concern.
https://www.npmjs.com/package/gridfs-locking-stream

@christkv
"w+ is valid but only for changing details about the document in the files collection not any of the actual binary chunks in the chunks collection."
The docs I were looking at give no mention of how to manipulate the metadata with a GridStore object. How would you do that? And why would you use "w+" to modify metadata if you could just use "w"? This still doesn't make any sense to me.
"No concurrency ever possible [with mode w]"
That's not true. If you create the file, and then open it again with mode w, it'll overwrite just fine. If no locking is happening, someone could be reading the file as you're overwriting it. There are obvious read-corruption problems there.

@imlucas Could you add some clarity here? I'm just not understanding what christkv is saying.

Mongodb has generally the problem with multi-document updates as there are not such thing as transactions. Fortunately, in a few weeks we'll get Mongodb 4.0 which has transactions. So GridFS implementations will be able to have an append method + it's way more performant than using custom locks using for example with gridfs-locks.

@fresheneesz Having an append mode means updating multiple collection at the same time (add fs.chunks row + update fs.files to set the correct length), which is simply not possible in Mongodb atomatically -> which means it will result in errors sooner or later. So it's a good decision to not even implement it.

@marcj I see what you're saying, but if you add chunk rows and never get to set the correct length, you'll take up some additional space but this file will still look just like nothing happened, right? (Since the length wasn't changed, it'd pretend those new chunk rows just don't exist).


fatal error: boost/config.hpp: No such file or directory

Hi,
I ran the make command in the SALSA folder in my conda env that has python=2.7, boost=1.70, boost-cpp=1.70 and networkx=1.11 but got the following error
correct_links.cpp:12:28: fatal error: boost/config.hpp: No such file or directory
It compiled break_contigs and break_contigs_start but not the correct_links.cpp file.
Can someone please let me know what the issue could be?

How do I get the include path correct so that
#include <boost/config.hpp> and all thereafter #include <boost/whatever.hpp> lines work in the correct_links.cpp file?
I have checked that the salsa_env in miniconda has the boost folder containing all the .hpp files needed and the path is
/gpfs01/home/sbzsm11/miniconda3/envs/salsa_env/include/boost
I have tried adding this folder to the SALSA folder but that doesn't work.

I found the solution in via Issue #112. I added -I filepath/to/boost_folder/in/salsa_env to my makefile in the g++ command for correct_links.


indexDocument - performance in IE11

Hello!
I'm using redux-search for a big chunk of data. In fact I'm operating the international classification of diseases (ICD 10) on client-side (ajax-loaded though). It contains about 13.000 terminal diagnosis. redux-search performs superb so far, yet IE11 stutters a bit while operating the search function on that classifiaction, but there is a (big...) performance issue with the initial indexing process.
redux-search is indexing the data field initially with a web worker (?) which takes 10 seconds in Firefox and 12-13 in Chrome, whereas IE 11 takes over. one. whole. Minute. I understand that a 13k entries list is far from a daily scenario, but I would like to know, why IE11 takes so much time for it. Running a profiler on IE boils it down on a forEach which the profiler cannot identify clearly, but it points to indexDocument one hierarchy above.
Do you have any ideas what could take IE so long on indexDocument? I'm combing through your code already but it would be nice to have some help on it. I'm using babel for transpilation.

Hey @wwarme,
Good to hear that redux-search is generally working well for you. I'm sorry that IE11's giving you performance problems though. For what it's worth, that's likely not due to redux-search at all- but to the underlying search lib, js-worker-search, since that's the code that actually manages the search index in the web worker.
I'm guessing this is indexDocument method you're referring to.
Maybe it would be more helpful if we moved this discussion/profiling over to js-worker-search? (I also maintain that library.) It might simplify the profiling a bit since it would remove all of the redux-middleware-specific code from the loop.

I'm going to close this issue because (unless I'm wrong) the IE bottleneck isn't occurring in redux-search. Please open an issue here along with any profiling info or steps reproducing and I'll try to help you with it. :)

Thanks for the fast reply! Yes I already figured that out, its probably js-worker-search. See you there. :)


Disable unknown directive warnings for sdl

When defining a schema in the schema definition language (sdl) in a .graphql file, you get errors for missing directives:

These errors don't come when having a .graphqls file. This is a convention, that is new to me.
Is this already adopted in the wider community?
We at Graphcool recommend people to write their schema in a .graphql file.
It would be great to either find a consensus on this or make this configurable :)
Thanks!

Related #111

I'm merging this and related issues into #121


Fix coverage service

Current coverage is 69.41% (diff: 100%)

No coverage report found for master at 522562a.

Powered by Codecov. Last update 522562a...209c97f


Tweak docs about QoS classes

Builds on PR #39118
Previews:

https://deploy-preview-39308--kubernetes-io-main-staging.netlify.app/docs/tasks/configure-pod-container/quality-service-pod/
https://deploy-preview-39308--kubernetes-io-main-staging.netlify.app/docs/concepts/workloads/pods/pod-qos/

Thank you for this followup to PR https://github.com/kubernetes/website/pull/39118
/approve


Agent should log a warning when hostname is set to hostname

Add logging to include a warning that the hostname for the agent host, is hostname. This stemming from requirements when using a kubernetes environment where nodes fail to appear when the hostname is generic and non unique.

With the release of Rancher 2.0, development on v1.6 is only limited to critical bug fixes and security patches.


improve Typescript compatibility

These small changes make the library compatible with much earlier versions of Typescript.

I'd be happy to merge this if this is required for your use case but I have a few concerns:

The main one is: how do we verify that the imports are organized in a proper way for future commits if we use only modern versions of TS for the development and CI? I doubt that we want to add an extra matrix run for that.
What are the blockers for the users to upgrade their TS version?
Is it possible to set up WebStorm's "optimize imports" feature to comply with the older import syntax?

There's no need to merge on my behalf. This just solved an immediate problem for me, and I thought it might open up the library to more users.
We (@causalhq) probably will upgrade Typescript soon anyway. Tomorrow I'll check if it's as simple as bumping the number :)

@KMahoney yep, please let me know.
If it helps immediately, I will just merge this one and release a new tag with your changeset.
So there will be at least one version that works with the older TS versions.
But it will not be guaranteed that the imports will not be "broken" for the old versions of TS in the future.

Closing this due to inactivity.


soccer.wbt.template robot names

I tried to set up a new simulation world using in scripts > README.md.
In the template world in scripts > templates > soccer.wbt.template, the robots are named "b1", "b2", etc.
In worlds > soccer.wbt, the robots are named "B1", "B2", etc.
This was a breaking change for me (self.name evaluated to "b1" but data uses key "B1"), and worked when I capitalized them.  I wasn't sure if this was somehow related to my setup, so I didn't submit a pull request for it.
If this is not just me, the change needed would be all in the file scripts > templates > soccer.wbt.template on lines: 511, 638, 762, 886, 1010, and 1134.  Each change is just "b1" to "B1", etc.

Thanks for the bug report @TBSDrJ, this is indeed a bug and I'll do my best to fix it shortly.

I'll fork it and post a pull request so you can just merge it.

@TBSDrJ thanks again for the fix, I believe we can close this.


Fix devmapper backend in docker info

Signed-off-by: Anusha Ragunathan<EMAIL_ADDRESS>

gofmt errors.

@anusha-ragunathan that's weird :) I suggest to fix it anyway.

LGTM

LGTM


tz2 Signing Error

Proof value has signature prefix "4sLJ" instead of expected "edsig", causing CI failure:
https://github.com/spruceid/ssi/runs/2668304500#step:5:2114
Edit: it's because the signature is 65 bytes instead of 64. The recovery byte should not be added in signing for spsig.

Fixed in #200


Upgrade to nsq-ruby version 2.0.3

This upgrade gains some performance fixes that we need to avoid Thread
leakiness but comes with a slightly newer verbage for tls/ssl options
passed in.
Changes

upgrade to nsq-ruby 2.0.3 from 1.6.0
change SSLContext class to be called TlsOptions to new gems wants
add support for passing in a verify_mode

I like it. Let's :shipit:


Documentation and Implementation of Topological Sorting in a DAG

Implement Topological Sorting in a DAG in Java/Cpp and explain the algorithm (proper documentation).
Note :
The code should be well commented so that anyone can understand that.
Proper documentation should be provided (either edit the readme or create a new md file and add a link in the readme file under relevant header.

Assign this issue to me @Vishal1003

Sure

Assign this issue to me


Docs: video tutorials

They don't need to be long, it could even be 30 secs.

Mint token
Transfer to polygon
Unlockables

Via offered to do VO for this.

Open to do VO if/when required. Redone audio input with voice recording in mind.


chromium: fix aarch64 build

The skia patch no longer applies, and is no longer necessary.
The current version of chromium doesn't build with gcc. This has been
fixed upstream[1] and could be backported, but switching aarch64 to
clang/llvm and making the build the same on all platforms is simpler.
[1] https://chromium.googlesource.com/chromium/src/+/9662ec844017690d5fd56bf0f05ef6a540dd29c1

Motivation for this change
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Notify maintainers
cc @

This requires a libjack2 to build, which was fixed in #73779 and is currently in staging.

Can you fix the merge conflict, and rebase on top of staging, so this should build?

fixed commit is here: https://github.com/bendlas/nixpkgs/commit/c829108321dadfc9afde310e3c479a765ff61db2
I held off merging it, because of the staging isse. should probably be merged, because build is already broken

ofborg timing out on 3600 seconds is to be expected, but the log up to that looks normal (we should do something about the warnings, that recently popped up, at some point)

Oh, I just realized, the PR was against staging. That wouldn't have been necessary, I think, since chromium doesn't have any downstream. Or is this an aarch64-specific thing?
Let's pick this into stable, on next staging merge.

That wouldn't have been necessary, I think, since chromium doesn't have any downstream. Or is this an aarch64-specific thing?

I went from the recommendation earlier in the thread. There's no real advantage in putting it onto master before staging merges since it depends on the libjack2 fixes in staging.

@thefloweringash if you use chromium on aarch64, could you please add yourself to the chromium maintainers, so that you are pinged and can test version bumps like #75516? I cannot build or test on aarch64.


No way to group agenda

It would be better to have Agenda by group so different page or sections can have different agenda lists based on the group.

This is feature is added and resolved


issue with tf.saved_model.save

https://github.com/tensorflow/tensorflow/issues/51587
I cannot understand why this serious issue is not paid any attention for such a  long time

Duplicates #51587
Please don't open duplicate issues to raise awareness to other other as this wastes time. Issues are triaged and will be resolved pending time / questions will be asked on the original issue as needed.


Performance : Eagerly Initialising memoized Suppliers (to calculate hashcodes) in data structures is expensive

Change to lazily initialising the memoized suppliers on first use.

Impacts Seq and Vector - slows down all functional operations

Released.


Bump T1 dependencies

Updated: nix/t1/_sources/generated.json
nix/t1/_sources/generated.nix

Closed in favor of https://github.com/chipsalliance/t1/pull/647.


auto hide by a pause

That will hide the icon if you pause your song.

I made the changes but it doesn’t work


Cleanup

Based on #65, because I'm too lazy to rebase directly on master and have all the documentation break and because some of the new exception stuff is used to make the final change (connect() function) actually useful.
Main feature of this branch is the removal of the commands.py layer of indirection. There were originally intended to be used directly, but, as noted in #5, this resulted in no documentation being attached to them. The (later added) workaround was to add another layer™, but even back then it was suggested to remove the now redundant "commands". I happen to agree, since "commands" were the most confusing thing to understand for me when I first got to know the codebase.
Other changes include the removal of some Python 2.6 compatibility code in the unit tests and and API for checking whether the daemon is actually available and compatible when connecting (as suggested by @joshisa in #53).

Sorry that it's taking me so long to get to this. I'm planning on doing code review for this tomorrow.

@whereswaldon @alexander255 Will this lead to a new tagged release?   Pypi is pulling some pretty stale bits from March. ;-)

@joshisa  I think that we've done enough that it's time for a another tagged release.
@alexander255 I've never published to PYPI before. Do we just need to tag the commit from Github, or is there some more complex process?

Since these commits do contain some breaking changes (I tried to keep it at a minimum), I think we should go ahead and get the renaming done once all the refactoring stuff is in. ipfsApi/ipfs-api would then continue to support IPFS 0.3.x on Python 2.x, while ipfsapi would support IPFS 0.4.x on Python 2.7+.
Publishing to PyPI is done by registering a project there (or have somebody give you access), then run ./setup.py upload. Details here:
https://packaging.python.org/distributing/#uploading-your-project-to-pypi
Since we talked about renaming the package as well, we should register a new project for it and give all py-ipfs-api developers access to it (there is no formal organization support on PyPI). What do you think @whereswaldon?

@alexander255 I agree. The renamed package should have different support (>=2.7) and start at version 1.0.0 (or 0.1.0, depending on whether you think a client for an alpha technology should pretend to be stable).

Now that I've seen it without commands.py, I can tell that this is vastly superior. Why would we ever want to maintain that entire file when it added practically nothing? Thanks for this refactor, I think the code is in a far better place now. Other than one typo, I have no gripes with these changes. :+1: for merge whenever that typo is resolved (or whenever you tell me that it isn't a typo, whichever)

Fixed the typo

Should I prepare another PR that does the renaming of the module?
Regarding version numbers: IMHO the first "new" version number should be 0.4.0 ("first release that targets IPFS v0.4"), after that we increase the revision number for each subsequent release. Breaking changes need to be postponed until the next IPFS daemon version (0.5.x in this case) is released.
What do you think?

@alexander255 I think that is the logical way to handler versioning. It will be less confusing for users if they don't have to keep up with a mapping of compatible version numbers. Another PR for renaming the module seems like the way to go.


Unclear whether the hub supports Availability Zones

From the response under the content for the question How are Availability Zones and resiliency handled in Virtual WAN?, it is unclear whether the Hub resource supports Availability Zones. The document says resources in VWAN such as ExpressRoute, Firewall etc support AZs.
Can you confirm whether

VWAN Hub supports Availability Zones?
Why Firewall can not be deployed in Availability Zones? My hub is in Australia East region that supports AZs.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 67c9a675-e1f5-0c17-30b7-69e33a5cf24b
Version Independent ID: e4f9ba49-1efb-d10c-7c8b-5163226de9bd
Content: Azure Virtual WAN FAQ
Content Source: articles/virtual-wan/virtual-wan-faq.md
Service: virtual-wan
GitHub Login: @cherylmc
Microsoft Alias: cherylmc

@akki-s , Thanks for reaching out. We will investigate and update this thread.

@akki-s , please find the answers to your queries below:

VWAN Hub supports Availability Zones?

Yes, Azure Virtual WAN supports Availability Zones.
You can see the Azure services that support availability zones doc as well.

Each of the services (VPN - S2S/P2S, ExpressRoute etc.) deployed inside a vWAN hub is automatically deployed across Availability Zones (except Azure Firewall), if the region supports Availability Zones. If you deployed a service into your hub when the region was not supporting availability zone but now it supports, then you can recreate the gateways, which will trigger an Availability Zone deployment.

Why Firewall can not be deployed in Availability Zones? My hub is in Australia East region that supports AZs.

I've provided the answer to this question in your other post : https://github.com/MicrosoftDocs/azure-docs/issues/89316
To deploy a secured virtual hub Firewall or normal Firewall with availability zones, you need to deploy it using Azure Firewall Manager Portal, PowerShell or CLI.
Please refer https://github.com/MicrosoftDocs/azure-docs/issues/89316 for steps and screenshots.
We are closing this issue for now. If there are further questions regarding this matter, please reply and we will gladly continue the discussion.


Incorporate write-fund and license plate funds to the funds routes

Current write-in fund page: https://giving.cu.edu/fund/write-fund
Current license plate fund: https://giving.cu.edu/fund/cu-scholarship-license-plate-fund
While the write-in fund doesn't have an allocation code, it is similar enough to a fund node that the allocation code can be faked and still fit into the current Fund model in the new backend.
The license plate fund does have an allocation code; however, it doesn't have the same giving form options.
I think both of these "fund types" can be merged into the default Fund model, but the giving form will have to switch based on what fund type is displayed. The React code can take this into account with an environmental variable for which funds are license plate and which ones are write-ins. If not in either list, you'd get all of the default forms.
It might be most appropriate to keep "fund type" and add license plate and write-in to that list so that they could be distinguished from other fund types.
However, the current implementation doesn't merge these two fund types into the default one so that shouldn't happen for the initial migration. It is easy enough to keep them separate on the frontend.
AC:

[ ] Add license plate and write-in to the /fund/ path prefix
[ ] Add "Giving Forms" all fund types and add them in fund page render

Old issue:
This causes 404's since the app attempts to pre-fetch the route since it is visible to the user.

Should start by redirecting to the current site since the fund search might be needed in a pinch before full migration finishes.

This is blocked due to not having fund.fund_type in the backend so that the giving form can be substituted based on fund type. I'm putting this in the backlog and replacing it with https://github.com/CUCentralAdvancement/giving-backend/issues/34

Actually, I'm going to keep the license plate and write-in funds separate on the frontend and ensure the URL paths aren't taken on the backend in the /fund/* route space.

First of all, you can't add named routes in a directory that already has a variable path. So adding /pages/fund/write-in.js does not work with /pages/fund/[slug].js. But you say, "why not use the splat operator and ..." Nope. I'm not doing that.
File-based routing works well until you get into these situations and using /pages/fund/[...who-knows].js doesn't make any sense to me. How can you look at that pattern and know what the actual routes are? You can't. At this point, you want an actual routing file, which, to me, not having one is a strong suit of easily working with Next.js.
I had already started to dislike node/JS/React/Next.js after getting past very basic rendering of static content, and I'm not spending any more time trying to do something with this on the frontend.
I will leave the fund routes in the frontend alone so they are standardized. Instead, I will add these funds to the backend and keep a list of special funds that map to specific render components for funds. Most will default to the regular forms but license plate and write-in funds will load different forms.


dtls: Don't crash when calling recvmsg to calculate packet size

sol_socket_dtls_recvmsg was crashing if buf or cliaddr were NULL and
this is used to calculate the packet size.
@glima and @vcgomes: This replaces my fix in commit #1576
Signed-off-by: Otavio Pontes<EMAIL_ADDRESS>

merged


Please update jQuery and jQuery UI To Latest Stable

Hello, I am working on an add-on for C<IP_ADDRESS> and I am hitting road blocks because of the plugin i want to use is not compatible with the version of jQuery UI that C5 currently ships with.
I also notice its  a few years old (2013) and probably could use upgrading. Would this be something possible? And if not, what would be the best way to override this in a package?
Current jQuery UI - v1.10.3 - 2013-06-01
Best: jQuery UI 1.11.3 (Stable, for jQuery1.6+)
See my original post:
https://www.concrete5.org/index.php?cID=707159&editmode=
Thank you.

Yes, this sounds like a good idea. We will update jQuery and jQuery UI to its latest stable version in 5.7.4.

I'm kind of curious, would it be possible to do our external assets like jquery / bootstrap in such a way that they are also brought in via composer instead of residing in the concrete5 repo? In an ideal world I would have thought that we only have stuff in the concrete5 repo that makes everything work together and maybe some grunt scripts to minify / mangle the appropriate external stuff together. Just a thought.


Some issues on Debian Buster (10.1)

I followed the Debian Guide (SWRAID-1 LVM LUKS) and encountered some issue.
Problem 1:
After restoring the vg config I got this message about different sector sizes:
Device /dev/mapper/cryptroot has size of<PHONE_NUMBER> sectors which is smaller than corresponding PV size of<PHONE_NUMBER> sectors. Was device resized?

I was able to solve this by simply deleting the volume group again and re-creating everything from scratch instead of restoring the backup:
vgremove vg0
lvcreate -n home -L100G vg0
lvcreate -n root -L300G vg0
lvcreate -n swap -L32G vg0

Problem 2:
update-grub hangs and shows no output at all. I debugged this by using strace -f update-grub
This did show a problem:
[pid 15012] openat(AT_FDCWD, "/run/udev/data/b1:0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

Solution was to also mount /run before chroot'ing:
mount --bind /run /mnt/run

Problem 3:
update-grub now run executes successfully, but the resulting config has an empty root= parameter. Thus after SSH'ing into dropbear and unlocking the cryptroot nothing happened, because grub wasn't able to continue boot. The offending menu entry in grub.cfg looks like
'gnulinux-5.0.21-2-pve-advanced-826c461f-7527-4f81-8899-2752ff825c7c' {
                load_video
                insmod gzio
                if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi
                insmod part_gpt
                insmod part_gpt
                insmod diskfilter
                insmod mdraid1x
                insmod ext2
                set root='mduuid/2d47ce5ebb51653216be521ba6329c11'
                if [ x$feature_platform_search_hint = xy ]; then
                  search --no-floppy --fs-uuid --set=root --hint='mduuid/2d47ce5ebb51653216be521ba6329c11'  2c07da8f-e0d8-4dca-b772-df4e7c2fc250
                else
                  search --no-floppy --fs-uuid --set=root 2c07da8f-e0d8-4dca-b772-df4e7c2fc250
                fi
                echo    'Loading Linux 5.0.21-2-pve ...'
                linux   /vmlinuz-5.0.21-2-pve root= ro  nomodeset consoleblank=0
                echo    'Loading initial ramdisk ...'
                initrd  /initrd.img-5.0.21-2-pve
        }

Notice that the root parameter on this line is empty: linux   /vmlinuz-5.0.21-2-pve root= ro  nomodeset consoleblank=0
This is where I'm stuck now. I was able to to temporary fix it by simpy manually editing /boot/grub/grub.cfg and adding /dev/vg0/root as root argument. But this will obviously get overwritten on the next grub-update. I really dislike grub and I don't know how to make it autodetect. I've spent the last few hours searching for a solution, but grub is such a complex behemoth....
Any ideas/help would be greatly appreciated.

Some comments on problem 1 (had the same issue on Ubuntu 18.04):
It relates to https://github.com/TheReal1604/disk-encryption-hetzner/issues/9
I also had to fix it by recreating the volume groups. Specifically in my case:
vgcreate vg0 /dev/mapper/cryptroot
lvcreate -n swap -L16G vg0
lvcreate -n root -l100%FREE vg0

(I note the comment above doesn't mention vgcreate?)
Intuitively, I'm kinda surprised it ever worked without altering the vg config - surely cryptsetup needs a slice of the disk to store its own machinery (e.g. the actual key), in which case the encrypted disk is a little smaller?

Installed successfully with Debian Buster.
I had an additional HDD in spare, so I gone with LVM-Management instead of rsync.
Requirment is using  lvm group.
My setup:

nvme ssd raid1 md0 for /boot
nvme ssd raid1 md1 with vgroup vg0 for swap and /, which I wanted to get encrypted
hhd sda in spare (for temporary tasks)

Move Volumes to other disk

add partition to spare disk via fdisk to sda
create a new physical device for lvm: pvcreate /dev/sda1
extend vgroup to sda1: vgextend vg0 /dev/sda1
move volumes to sda1:

pvmove -n /dev/vg0/swap /dev/md1 /dev/sda1
pvmove -n /dev/vg0/root /dev/md1 /dev/sda1

reduce vgroup: vgreduce vg0 /dev/md1

Encrypt your raid partition

create encrypted device: cryptsetup luksFormat -c aes-xts-plain64 -s 512 /dev/md1 cryptroot

Move Volumes to encrypted disk

create a new phyicial device for lvm: pvcreate /dev/mapper/cryptroot
extend vgroup to sda1: vgextend vg0 /dev/mapper/cryptroot
move volumes to /dev/mapper/cryptroot:

pvmove -n /dev/vg0/swap /dev/sda1 /dev/mapper/cryptroot
pvmove -n /dev/vg0/root /dev/sda1 /dev/mapper/cryptroot

reduce vgroup: vgreduce vg0 /dev/sda1

Then you go ahead with mounting all that to /mnt, chroot, editing /etc/crypttab etc..
This way looked to me very clean. You don't touch any data in volumes.
If you don't have a disk in spare, you should be able to remove a disk from raid array and use it as temporary device.

followed the guide on 10.3 just seconds ago. in the resulting chroot of the old setup in hetzners rescue system, i needed to apply changes to the lvm config as outlined in https://bbs.archlinux.org/viewtopic.php?pid=1513816#p1513816 - otherwise accessing the LVs wouldn't be possible.
thank you.

@razorness @BloodyNora would either of you be able to submit a PR with these updates?  It's been forever since I wrote the debian guide and to be honest I'm not really certain I could include your changes correctly because I don't really remember the process.
Having the guide has really helped a bunch of people, but if we just leave updates here in the issues it makes it really difficult for others to follow.

Merged the PR. Thanks guys!


Polyfill for CSS breaks prefix attribute selector

If I have a CSS selector like
:host [my-type^="f"]

the polyfill converts this to
my-element span[my-type="f"]

which obviously is wrong. Other attribute selector like the suffix attribute selector however work fine.
See this demo: http://plnkr.co/edit/9QdOAvJHRTJopyrYqiiG?p=preview

Closing this issue due to age and the release of version 1 of Polymer - please feel free to re-open if this is incorrect.


[#4157] fix(doc): Fix the doc format in how-to-build

What changes were proposed in this pull request?
[#4157] fix(doc): Fix the doc format in how-to-build
Why are the changes needed?
https://github.com/apache/gravitino/blob/main/docs/how-to-build.md#quick-start
The code block should only show the git clone git@github.com:apache/gravitino.git

Fix: #4157
Does this PR introduce any user-facing change?
No
How was this patch tested?
The doc format is fine in my dev branch.
https://github.com/1996fanrui/gravitino/blob/4157/fix-doc-format/docs/how-to-build.md#quick-start

thanks for the quick review.


Chess: Move C7 to C6

Please do not change the title. Just click "Submit new issue". You don't need to do anything else :D

@IamTemmy done! Successfully played move c7c6 for current game.
Thanks for playing!


Fix an error on SSL config reload (plus some cleanup).

Fix an error on SSL config reload (plus some cleanup).

This seems to have eliminated some ERROR diags we were seeing in Yahoo Prod when
doing config reloads.

The SSLSecret public functions no longer return pointers into the unorded_map of
secrets, they return a copy of the secret data. This seemed thread unsafe. A
periodic poll running in the background can update the secret data for an entry
for a secret name in the map.

To avoid exporting pointers, I had to change the prototype of TSSslSecretGet().
Hopefully there are no existing plugins that are already using this TS API function,
so breaking this rule will be moot.

This PR replaces https://github.com/apache/trafficserver/pull/8790 , which was laid waste by zwoop's merciless purge of personal branches in the main repo.
The TSSslSecretGet() function is now compatible with the rest of the TS API (and bell bottoms and stagflation).

Oh no, we released 9.2.0 without this fix. This means 9.2.0 has a problematic TS API and we can't fix it until 10.0.

@zwoop What do you think about releasing 9.2.1 soon with this technically an incompatible change?

Is this something we want for 9.2.1 ?

I would recommend backporting, I'll do if you want, in case there are any conflicts.

It will change the prototype of one TS API function.  But the current prototype for that function (which returns a pointer to data that's modified by multiple threads) is inherently crashy to use (hopefully no one's using it).

We talked about it on weekly bug scrub. The API was newly added on 9.2.0. It's now (before many people start using it) or 10.0. Fixing it on 9.2.2 doesn't make sense.


[Pypi initial] reset version to 0.0.4

Codecov Report

Merging #7 (6ec3f93) into main (6b41a9b) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main       #7   +/-   ##
=======================================
  Coverage   25.90%   25.90%
=======================================
  Files           3        3
  Lines         166      166
=======================================
  Hits           43       43
  Misses        123      123

Flag
Coverage Δ

unittests
25.90% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6b41a9b...6ec3f93. Read the comment docs.


[edgetpu_compiler] Internal compiler error. Aborting!

Description
I tried to convert a tensorflow model to tflite model using post-training quantization.
The conversion is completed without any error.
When I use edgetpu_compiler to compile the tflite model for TPU, it shows this error message:
Internal compiler error. Aborting!

I have no clue what cause this issue.
Need help, please.
Model
frozen pb
converted tflite
System Information

Ubuntu 18.04
TensorFlow 2.1.0
Edge TPU Compiler Version: 2.1.302470888

Code for reference
I used the following script to convert frozen pb model to quantized tflite
import os
os.environ["CUDA_DEVICE_ORDER"] = "PCI_BUS_ID"  # using CPU
os.environ["CUDA_VISIBLE_DEVICES"] = "-1"
import tensorflow.compat.v1 as tf
from tensorflow.python.platform import gfile
from argparse import ArgumentParser
import numpy as np

def build_argparser():
    parser = ArgumentParser()
    parser.add_argument("-m", "--model", help="Path to a .pb file with a trained model.", required=True, type=str)
    parser.add_argument("-o", "--output", help="Path to a .tflite file with a converted model.", default=None, type=str)
    return parser

def representative_dataset_gen():
    num_calibration_steps = 10
    input_size = 1024
    for _ in range(num_calibration_steps):
        # Get sample input data as a numpy array in a method of your choosing.
        input = np.random.random((1,input_size,input_size,3)).astype(np.float32)
        yield [input]

if __name__ == "__main__":
    args = build_argparser().parse_args()
    GRAPH_PB_PATH = args.model
    OUTPUT_TFLITE = args.output

    tf.enable_eager_execution()

    with tf.Session() as sess:
        print("LOAD GRAPH")
        with gfile.FastGFile(GRAPH_PB_PATH,'rb') as f:
            graph_def = tf.GraphDef()
        graph_def.ParseFromString(f.read())
        sess.graph.as_default()
        tf.import_graph_def(graph_def, name='')

        x = sess.graph.get_tensor_by_name("inputs:0")
        y1 = sess.graph.get_tensor_by_name("detector/yolo-v3-tiny/Conv_9/Conv2D:0")
        y2 = sess.graph.get_tensor_by_name("detector/yolo-v3-tiny/Conv_12/Conv2D:0")

        # for accelerator
        converter = tf.lite.TFLiteConverter.from_session(sess, [x], [y1, y2])
        converter.optimizations = [tf.lite.Optimize.DEFAULT]
        converter.representative_dataset = representative_dataset_gen
        converter.target_spec.supportrf_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
        converter.inference_input_type = tf.uint8
        converter.inference_output_type = tf.uint8

        tflite_buffer = converter.convert()
        tf.gfile.GFile(OUTPUT_TFLITE, "wb").write(tflite_buffer)

    print("CONVERSION COMPLETE")

@KodeWorker
Unfortunately, yolo models are not officially supported by us. You can check that from the outputs of the visualized tools, there are many ops in your model that aren't on the list of supported operations

@Namburger
Thanks for your reply. I used Netron to view my model operations. It is true that frozen *.pb model contains many unsupported ops, but the converted tflite model should be fine. Why edgetpu_compiler fails?

@KodeWorker Sorry, should have gave more details, the exact op that fails is:
detector/yolo-v3-tiny/Conv/LeakyRelu/alpha

LeakyRelu isn't a supported op yet, but I don't that why it fails, something also just caught my attention, the input dimension for that layer is [1x1024x1024x32] which is probably just too big to fit in the TPU's RAM. I suggest shrinking the input size and try again!

@Namburger Thank you for the advice. I'll try the following options and keep tracking this issue.

[ ] replace activation function with supported one
[ ] shrink the input size

@Namburger
Thanks again for your help.
It's no longer an issue for me, but you are welcome to re-open this issue again, if there would be any support for leaky_relu.

@Namburger Thank you for the advice. I'll try the following options and keep tracking this issue.

[x]  replace activation function with supported one
[x]  shrink the input size

Update

Change the activation function (leaky -> relu) and retrain the model solved this issue.
Shrink the input size to [1x384x384x3] for better runtime efficiency.
Encountered some problems similar to this post, and load_delegate works!

Wow this is very good to know, so yolo model can actually be converted to over and get working with our libraries, thanks for sharing!

@Namburger
Sorry for the late response. I have little experience in Yolov3-tiny. According to my colleagues, the use of relu activation function rather than regular leaky relu will lead to about 10% drop in mAP.

@KodeWorker thanks for the info, I think I shall try prelu in that case

Hey, FYI, I found this implementation: https://github.com/guichristmann/edge-tpu-tiny-yolo


io_uring for copies from disk

Some early io_uring experiments
disk_read_speed.py:

import io
iouring

amd
9.93 GB/s
14.73 GB/s

nv
9.54 GB/s
15.36 GB/s

llama 70b ~9.5gb/s for now

Do not if we want to use io_uring, but raw performance (disk->gpu) using it is about 18-20gb/s. Will test any alternatives, like aio, maybe it will be a simpler api or just syscalls with the same performance.

I want it to be async, but if normal aio async is the same speed that's fine.

aio is only ~11-12gb/s, so uring is the fastest. Will cleanup and merge uring.

@geohot any comments/suggestions?
I have removed calling liburing and just doing some mmap+syscalls now to not have this dependency. Maybe this adds some lines, but I think it's worth it.

Very readable!
copy_from_disk is clearly copy paste between NV and AMD, but maybe this is part of a bigger refactor to share?


[Issue 6000][pulsar-client] C++ client ACK grouping feature.

Fixes #6000
Motivation
In our production environment, we created a sinker: It combines messages read from Pulsar, and writes the combined data into ClickHouse, which is not good at processing highly concurrent inserting (max. 100 QPS).
In the sinker, we used failover subscription with cumulative ACK. The reason for not using shared subscription is that, individual ACK in C++ client siginificantly reduces throughput. Our analysis shows that huge amount of individual ACK requests results in highly concurrent accessing, which finally results in terrible throughput. BTW, the machines, where we deploy brokers and bookies, are not equipped with SSD. There could be some configuration for brokers or bookies helping improving the situation, but we don't find them.
Therefore, the idea is that combining individual ACK requests within one request. After reading the protocol (PulsarApi.proto:CommandAck), we find that this feature is supported. According to this issue #6000 , Pulsar C++ client SDK does not implement ACK grouping feature, but it's already implemented by Java client.
After discussing with @jiazhai and @sijie , we are authorized to implement this feature and contribute this feature back to community later. It should follow the Java client's behavior and provide similar interfaces that Java client provides.
Modifications
Interfaces
Similar to Java client, C++ client implements three trackers to cache ACK requests within a configured time window:

AckGroupingTracker: this is the base class of the other two trackers, it defines interfaces and provides empty implementation which does not send ACK requests to broker. This tracker is used for non-persistent topic, which actually does not require ACK.
AckGroupingTrackerDisabled: child class of AckGroupingTracker. It does not provide ACK grouping ability, and acts just like the previous individual ACK.
AckGroupingTrackerEnabled: child class of AckGroupingTracker. This is the real implementation of ACK grouping.

The trackers provides following public interfaces:

isDuplicate: checking if the given message ID exists in the TO-BE-ACKED group.
addAcknowledge and addAcknowledgeCumulative: unlike Java client, which combines these two interfaces into one (addAcknowledge), C++ clients provides them for individual and cumulative ACK requests. Such design can provide slightly better performance than if-else implementation.
close: closing the tracker.
flush and flushAndClean: flushing all pending ACK requests, the later one also resets internal status.

Consumer's Configuration
Two new configuration options are added:

ackGroupingTimeMs: time window in milliseconds for grouping ACK requests. If setting to positive value, ACK requests are grouped and sent to broker every ackGroupingTimeMs milliseconds (AckGroupingTrackerEnabled). For non-positive values, ACK requests are sent one by one to brokers as before (AckGroupingTrackerDisabled). Default is 100.
ackGroupingMaxSize: maximum number of grouped message IDs. Java client hard-coded it to 1000 for now. However, 1000 is too small in our scenario. Once the grouped message number reaches this limit, the ACK requests will be packed into one and sent to broker, even the ACK grouping deadline (ackGroupingTimeMs) is not reached. Non-positive values remove such limit.

In addition, these configurations are added into C API as well.
Commands for this feature.
A few new command factory interfaces are implemented, just like in Java, incl.

newMultiMessageAck: command object for multi-message ACK requests.
peerSupports*: interfaces for checking versions.

Topic Domain
Used to help defining and distinguish non-persistent and persistent topics.
Verifying this change

[x] Make sure that the change passes the CI checks.

This PR added 7 unit tests in BasicEndToEndTest, all of them start with testAckGroupingTracker. They cover the ACK grouping tracker's logic.
Does this pull request potentially affect one of the following parts:
NO

Dependencies (does it add or upgrade a dependency): (no)
The public API: (no)
The schema: (no)
The default values of configurations: (no)
The wire protocol: (no)
The rest endpoints: (no)
The admin cli options: (no)
Anything that affects deployment: (no)

Documentation

Does this pull request introduce a new feature? (no)
If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)
If a feature is not applicable for documentation, explain why?
If a feature is not documented yet in this PR, please create a followup issue for adding the documentation

@tongsucn Thanks for the great work. Would you please add a unit tests in BasicEndToEndTest to protect your code?

@tongsucn Thanks for the great work. Would you please add a unit tests in BasicEndToEndTest to protect your code?

No problem. BTW, the Backwards Compatibility CI process failed because of storage issue. Do I need to deal with it or just try again later?

/pulsarbot run-failure-checks

/pulsarbot run-failure-checks

/pulsarbot run-failure-checks

/pulsarbot run-failure-checks

/pulsarbot run-failure-checks

@jiazhai Hi, I added necessary unit tests and updated the description above. However, the cpp-tests continuously and randomly fail on different cases, which are passed in my container.  Can you rebase this pull request to the latest master?

/pulsarbot run-failure-checks

/pulsarbot run-failure-checks

@tongsucn yes. Can you rebase this pull request to the latest master?

@sijie DONE

@tongsucn Can you rebase this pull request to the latest master? Looks there are some conflicts with the master branch.

/pulsarbot run-failure-checks

/pulsarbot run-failure-checks

/pulsarbot run-failure-checks

@tongsucn Could you please take a look at the CI CPP tests? Looks related to the code format.

@tongsucn Could you please take a look at the CI CPP tests? Looks related to the code format.

@codelipenghui Hi, it passes make check-format in my container, and running make format does not affect any code. This is really weird....
root@THOSTNAME:/pulsar/pulsar-client-cpp# git rev-parse HEAD
77c84c7efef4732534a38d126a26ba624f7e17f9
root@THOSTNAME:/pulsar/pulsar-client-cpp# git status
On branch ackgroup
Your branch is up-to-date with 'origin/ackgroup'.
nothing to commit, working directory clean
root@THOSTNAME:/pulsar/pulsar-client-cpp# make check-format
Scanning dependencies of target check-format
Built target check-format

/pulsarbot run-failure-checks

last fix was on mac, I will make format under ubuntu env.

/pulsarbot run-failure-checks

/pulsarbot run-failure-checks

/pulsarbot run-failure-checks


chore: bump dependencies

blocked by: https://github.com/lucacasonato/esbuild_deno_loader/pull/22
closes #984
supersedes: https://github.com/denoland/fresh/pull/952

The blocking pr was merged

@sigmaSd thanks for the reminder, will update this PR when I get the chance.

@kt3k PTAL!

@kt3k PTAL!

@kt3k PTAL

The project is in hibernation

This caused the below error in deploy and islands didn't work unfortunately..

Will look into it now. Very weird.

Looks like it's an issue with specifically the wasm version. We should add tests for that. Has caused trouble at least twice before.

We should add tests for that. Has caused trouble at least twice before.

Yeah, we need a way to test the rendering with wasm in CI

After scratching my head at this for some time, I think I've narrowed it down to either being a breaking change to esbuild or a bug with esbuild_deno_loader. I will investigate more tomorrow and hopefully create a PR with the bug fixed.


[BUG]: Error Code 80041A06

Describe the bug
Trying to upgrade CoE core Components to 1.98 fails with ErrorCode 80041A06
Component (please tell us which flow or app you are experiencing issues with):

Solution: CenterofExcellenceCoreComponent
Version: 1.45

To Reproduce
Steps to reproduce the behavior:

Import Solution
Select File CenterofExcellenceCoreComponents_1_98_managed (1).zip
Fails
ErrorCode 80041A06, Exception message Attribute admin_rpamodifiedon is a String, but a DateTime type was specified. Patch: No

Expected behavior
Successful import of the solution
Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
Add any other context about the problem here.

Hello, you will need to upgrade to the interim solution (1.62) shown in #515
And then try again to upgrade to the latest version
Thanks for using CoE!
Closing as duplicate of #515


Use a test matrix

Right now only one Elixir / OTP combination is tested.
The best practice is to use matrixes. Like this: https://github.com/wemake-services/ecto_autoslug_field/blob/master/.github/workflows/test.yml#L18-L21

Added in https://github.com/shirokovnv/phx_json_rpc/pull/17