RFC: Store strings off-stack (in per-CPU scratch buffer)

I've iterated on https://github.com/iovisor/bpftrace/pull/1357 based on review feedback.
Resolves https://github.com/iovisor/bpftrace/issues/305.
Incorporates @mmarchini's RFC https://github.com/iovisor/bpftrace/pull/750.
The design is a lot simpler now. We allocate a per-CPU array map, big enough for your largest string. str() probe-reads strings into this map. Then, printf() probe-reads strings out of that map into its own map, which is submitted as a perf event.
Compared to the previous proposal: this design does not have the data race and coordination with userspace, nor does it introduce a "different" string structure, so should interoperate better with the existing string functions.
I found that I had to change printf()'s argvalue memcpy() into a probe_read(). We may find other interoperability problems like that between string functions, but they're easily solved.

Also please clang-format and get the builds to pass

Noticed my comment from the last version got lost:

Being able to read in loooooooong strings would be great but I don't think we should change the current behaviour of str(), as a construct like @[str(arg0)]=arg1 should continue to work. I'd rather see a new call for this async version of str.

Being able to read in loooooooong strings would be great but I don't think we should change the current behaviour of str(), as a construct like @[str(arg0)]=arg1 should continue to work. I'd rather see a new call for this async version of str.

@fbs thanks for carrying over this comment, and sorry to not address it directly. I felt that this was a very valid criticism of the previous RFC (https://github.com/iovisor/bpftrace/pull/1357); that version certainly could be called async (BPF emitted a map coordinate via perf, and userspace had to lookup that map entry later). and yes, in that version the output of str() was not compatible with other function calls.
But the design in this review iteration is far simpler; when str() returns, expr_ just points to string contents stored in a map value. this should give it interoperability with other functions which consume strings.
Do you have a program handy that uses a syntax like @[str(arg0)]=arg1? I'd like to try it out and see if the new str() is still compatible. I may need to tweak some MEMCPY() calls similarly to what I did with printf:
      if (expr_points_to_map_value) {
        b_.CreateProbeRead(ctx_, offset, arg.type.size, expr_, call.loc);
      } else {
        b_.CREATE_MEMCPY(offset, expr_, arg.type.size, 1);
      }

Moreover, @danobi suggested in https://github.com/iovisor/bpftrace/pull/1346#issuecomment-634312784 that it'd be okay to change all strings to be map-backed. so, perhaps str() won't be a special case.
Certainly, it seems to me at the moment that it should be possible to implement this such that the user notices no difference to the bpftrace language (i.e. no additional str() call required, and all string-using functions continue to interoperate).

Also please clang-format and get the builds to pass

@danobi I think I'm not quite ready to write tests for this, since I have to resolve the TODOs around "what happens if map_lookup_elem returns a nullptr" (i.e. during str() or during printf()). I'm planning to make it Return from the BPF program and emit a perf event to produce an error message (dropped event). Progress so far on that: https://github.com/Birch-san/bpftrace/compare/long-strings-4...Birch-san:long-strings-5

Also please clang-format and get the builds to pass

@danobi I've had a stab at that, but maybe I'm doing something wrong?
git --no-pager diff --name-only master | xargs clang-format -i

The output looks correct, but it changes 4000 lines:
https://github.com/Birch-san/bpftrace/compare/107371e...long-strings-4-clang-format
There's plenty of stuff absent from my own changes that got caught by using clang-format in this way. Is there a way to just target the new lines I've added? Do we normally run clang-format?

Update: I've now added handling for the case where str() or printf() fails to acquire a map element. I figured it made sense to re-use the existing helper_error infrastructure, except make it a bit harsher. Because this isn't a user-managed map where it could be normal to encounter missing entries -- it's one of bpftrace's maps, where we really expect the contents to exist. I don't expect a user to be interested in sniffing an error to decide how to continue, I think one of their invariants has been violated and it makes sense to return from the BPF program.
I'm even tempted to make that harsher still, and detach the probe if this ever happens. Since you could easily be in an invalid state. Imagine having logic after a str() or printf() — you'd expect this logic to get hit every time, but if str()/printf() fails to acquire a map entry then we'll have broken one of your guarantees.

Do you have a program handy that uses a syntax like @[str(arg0)]=arg1? I'd like to try it out and see if the new str() is still compatible. I may need to tweak some MEMCPY() calls similarly to what I did with printf:

Not really. #1067  is somewhat related but does not currently work

Moreover, @danobi suggested in #1346 (comment) that it'd be okay to change all strings to be map-backed. so, perhaps str() won't be a special case.

str() is currently the only thing that probe_read_str into a stack var, I'm not convinced that we  should remove that completely. There is some room for improvement as @[str()] should read directly into  the map,  not through a stack buffer.

Also please clang-format and get the builds to pass

@danobi I've had a stab at that, but maybe I'm doing something wrong?
git --no-pager diff --name-only master | xargs clang-format -i

The output looks correct, but it changes 4000 lines:

You  need git-clang-format to only format changed lines, your line clang-formats the whole file. Can use the ci  job as example

@Birch-san

Do you know a good probe I could invoke that would give me two different strings to try str() upon (so I can try and make one clobber the other)?

rename(2) seems like it could do the trick:
tracepoint:syscalls:sys_enter_rename
    int __syscall_nr;
    const char * oldname;
    const char * newname;

CI still seems dissatisfied. Not sure what I'm missing.

I usually just run git clang-format master on my topic branch and it "just" works. I also do it per-commit like this: git clang-format HEAD~1.

@fbs

str() is currently the only thing that probe_read_str into a stack var, I'm not convinced that we should remove that completely. There is some room for improvement as @[str()] should read directly into the map, not through a stack buffer.

Correct me if I'm wrong -- it seems like there's 2 concerns here:

Concern that the user may have a use case where they need a string on stack
The proposed implementation in this PR could be further optimized

Does that sound right?
I read the reference docs on str() again and it doesn't seem like there's promises the strings will be on stack. My feeling is if map-based strings can be pulled off without regressions it would be a win and the user experiences no visible side effects (other than max string size being increased)

I'm even tempted to make that harsher still, and detach the probe if this ever happens. Since you could easily be in an invalid state. Imagine having logic after a str() or printf() — you'd expect this logic to get hit every time, but if str()/printf() fails to acquire a map entry then we'll have broken one of your guarantees.

I'd tend to agree but even harsher still. I'm looking at the kernel implementation ( https://github.com/torvalds/linux/blob/master/kernel/bpf/arraymap.c#L240-L250 ) and I don't really see how a lookup on a percpu map can fail unless there's a bug in codegen. So I propose abort()ing bpftrace so it's clear it's a bug.

@fbs
Does that sound right?
I read the reference docs on str() again and it doesn't seem like there's promises the strings will be on stack. My feeling is if map-based strings can be pulled off without regressions it would be a win and the user experiences no visible side effects (other than max string size being increased)

I've tried to come up with a use case that would require keeping it but I couldn't.  If someone comes up with a good usecase we can always add it back in, so let's not block this on it :)

I'd tend to agree but even harsher still. I'm looking at the kernel implementation ( https://github.com/torvalds/linux/blob/master/kernel/bpf/arraymap.c#L240-L250 ) and I don't really see how a lookup on a percpu map can fail unless there's a bug in codegen. So I propose abort()ing bpftrace so it's clear it's a bug.

yes, I agree "stop everything" makes sense for a "well this should never happen anyway".
when you say abort(), do you mean like the abort() calls in codegen_llvm.cpp? (https://github.com/iovisor/bpftrace/blob/9515965/src/ast/codegen_llvm.cpp#L64) I think we can only use those to abort compilation of the LLVM IR. sadly I think we don't find out the result of map_lookup_elem until BPF runtime, so we would not be able to use an abort().
I think we'd instead have to handle it the same way exit() works: send a perf event, return from the current BPF function, (https://github.com/iovisor/bpftrace/blob/9515965/src/ast/codegen_llvm.cpp#L759), and then have userspace do a bpftrace->request_finalize() (https://github.com/iovisor/bpftrace/blob/f38b144/src/bpftrace.cpp#L466). easy enough; just needs AsyncAction::helper_error to have a "fatal error" boolean.

@Birch-san

Do you know a good probe I could invoke that would give me two different strings to try str() upon (so I can try and make one clobber the other)?

rename(2) seems like it could do the trick:
tracepoint:syscalls:sys_enter_rename
    int __syscall_nr;
    const char * oldname;
    const char * newname;

Brilliant; I'll give that a try.

CI still seems dissatisfied. Not sure what I'm missing.

I usually just run git clang-format master on my topic branch and it "just" works. I also do it per-commit like this: git clang-format HEAD~1.

okay, I'll give it another stab with one of those approaches and see if it goes any better.

There is some room for improvement as @[str()] should read directly into the map, not through a stack buffer.

@fbs I wonder whether an LLVM optimization pass could be introduced to elide this copy in the general case, rather than complicating the codegen.
there'll be a lot of cases like this. for example, with printf("%s", str()):
str() writes into a map as scratch space, then printf copies that content into another map, which perf_event_output exfiltrates. there's a wasted step (i.e. str() doesn't need its own scratch space -- it could output directly into printf's map). and I think when we used AllocaInst() for scratch space, LLVM knew how to optimize away these copies (the optimized LLVM IR looks like str() writes directly into an offset of the printf buffer).
I think if we annotated these particular maps as "we use this like stack-allocated memory", an optimization pass could be written… somehow. :)

I think we'd instead have to handle it the same way exit() works: send a perf event, return from the current BPF function, (https://github.com/iovisor/bpftrace/blob/9515965/src/ast/codegen_llvm.cpp#L759), and then have userspace do a bpftrace->request_finalize() (https://github.com/iovisor/bpftrace/blob/f38b144/src/bpftrace.cpp#L466). easy enough; just needs AsyncAction::helper_error to have a "fatal error" boolean.

Yeah that's what I meant

I'd tend to agree but even harsher still.

@danobi I've now made map_lookup_elem fatal, removed Types::fmtstr and run clang-format.
Now need to investigate how well this interoperates with other string functionality (e.g. storing in maps, assigning to variables).

I'm wondering now what happens if you do something like:
$x = str(one);
$y = str(two);
printf("%s\n", $x);

Does $y clobber $x?

@danobi I've now tried that with rename, and it works fine:
sudo bpftrace -e 'tracepoint:syscalls:sys_enter_renameat2 {
  $x=str(args->oldname);
  $y=str(args->newname);
  printf("<%s; %s>\n", $x, $y);
}'
Attaching 1 probe...
</var/tmp/rename_me; /var/tmp/rename_me2>

Triggered with:
mv /var/tmp/rename_me /var/tmp/rename_me2

a construct like @[str(arg0)]=arg1 should continue to work

@fbs I just tried this map assignment, and it looks like off-stack strings broke it. I got a segfault during semantic analysis, because this case came out as a nullptr:
https://github.com/iovisor/bpftrace/blob/9f975da07f1019d51e18d7362ecc93492dc00860/src/ast/codegen_llvm.cpp#L1982-L1984
The specific program I tried was this (press Ctrl+C :
bpftrace -e 'tracepoint:syscalls:sys_enter_renameat2 {
  @[str(args->oldname)]=1;
}
interval:s:1 {
  print(@);
  clear(@);
}'

Triggered with:
mv /var/tmp/rename_me /var/tmp/rename_me2

Until now: map keys would've always been pointers to AllocaInst memory. But strings are now CallInst memory provided by map_lookup_elem.
So, I needed to update all map functions to accept keys of either AllocaInst or CallInst. This actually went pretty well:
https://github.com/Birch-san/bpftrace/compare/long-strings-4...Birch-san:long-strings-7
And it works; I got the expected output when my probe fired:
@[/var/tmp/rename_me]: 1

But I'm wondering whether std::variant<AllocaInst *, CallInst *> is overkill; maybe I should just downcast everything; make getMapKey() return Value *, make all Map helpers accept Value *, and then have the helpers sniff what value they receive:
if (auto *AI = dyn_cast<AllocaInst>(Val))

Depends whether we want "specific types + compile-time safety" of std::variant, or "smaller diff, reliance on RTTI" of dynamic cast.
I guess I'd like to know "why do our map helpers ask for a key as specific as AllocaInst?" -- I notice that we call the key's getType(). AllocaInst overloads getType() (to return a PointerTy, rather than a vague llvm::Type), so I suppose this is the reason.
Anyway, any thoughts on whether to simplify this to use just Value * and a dyn_cast<AllocaInst>?

I tried changing the CreateMap helpers to accept Value *, and that went fine. I think they weren't relying on any specific features of AllocaInst *.
The assertion key->getType()->IsPointerTy() succeeds without needing to know the subtype. CallInst memory passes this assertion fine too.
I'm not sure whether there's any consequences here:
// does it matter that key is now a Value* instead of an AllocaInst*?
// key->getType() will no longer use AllocaInst*'s overload (which casts to more specific `PointerType`)
FunctionType *lookup_func_type = FunctionType::get(
      getInt8PtrTy(), { map_ptr->getType(), key->getType() }, false);

Seems to be working fine though.
So, regarding "make str() keys work again", here's the std::variant approach (from my previous comment): https://github.com/Birch-san/bpftrace/compare/9f975da...long-strings-7
and here's the simplification I've applied to do it without std::variant (i.e. downcast to Value* instead): https://github.com/Birch-san/bpftrace/compare/long-strings-7...long-strings-7-simpler
I think the simpler version is what I'll merge into this PR, since it's the smaller change.

I've fixed another regression with associative arrays: storing string values.
sudo ./src/bpftrace -e 'tracepoint:syscalls:sys_enter_renameat2 {
  @[str(args->oldname)]="test";
  exit();
}'
Attaching 1 probe...

@[/var/tmp/rename_me]: test

Triggered with:
mv /var/tmp/rename_me /var/tmp/rename_me2

The fix is d371a38cf7ffdd4c5180a5d10837c981f2ee46d0, but it'll need a better design.
The problem was that "all maps that people store strings into" became per-CPU array maps.
Whereas I only wanted my two scratch maps — "str" and "fmtstr" — to become per-CPU array maps. In all other cases, the usual hash map suffices.
=====
The next problem I've found is:
sudo ./src/bpftrace -e 'tracepoint:syscalls:sys_enter_renameat2 {
  @[str(args->oldname)]=str(args->newname);
  exit();
}'
Attaching 1 probe...

@[/var/tmp/rename_me]: /var/tmp/rename_me

Triggered with:
mv /var/tmp/rename_me /var/tmp/rename_me2

Unfortunately the key and the value both become /var/tmp/rename_me, which means that the key "clobbers" the value.
I think that makes sense; the map assignment consumes one str() without making a copy of its contents, then consumes another str() without copying too. They use the same scratch space, so both key and value will point to the same string.
Guess I'll need to introduce a copy.
=====
Another known problem is that constant strings are sized to STRING_SIZE rather than bpftrace_.strlen_. This limits how big printf()'s format string can be.
=====
Another problem: variables are stored on-stack, so you can't assign big strings to them. Not sure whether to solve that; would it be weird for variables to become map-backed?
=====
I also still need to check whether string functions like strncmp still work.

I noticed that when I originally tested the limits of BPFTRACE_STRLEN, I left a comment:
~1024= <A call to built-in function 'memset' is not supported.>

I wonder if that's why @mmarchini used probe read instead of memset to zero out the map-backed printf buffer.
If there's a precise threshold at which memset ceases to work, we could build an IRBuilder helper for zeroing buffers (e.g. memset for small buffers, probe read for big buffers).

ac648a0f2f1cf256c935e6e4af12bfcd75acad43 + 52fc17d8fd34dd6c7088067e85e8682c32bc578f We can now assign values to maps with composite keys comprised of strings. There's a lot of special branches here which exist purely for performance… it'd be simpler if getMapKey() just always used a map-backed buffer. And if we're still concerned with performance, we should consider the extent to which we can mitigate this by writing an LLVM optimizer pass.
e072e8f67f1c167a4a09fdefa61def654c52f4c0 Whilst looking at composite keys e.g. @[comm, pid]: I found that MEMCPY() works just fine to copy contents from one map to another. I revisited createFormatString and tried the same there (to move str() content into printf buffer), and that worked fine too. I can no longer reproduce the problem that made me resort to probe_read, so let's enjoy the simpler code.

Things that accept strings that now accept map-backed strings:

join()
buf()
ternary expressions
strncmp()
join()
printf()

hardcoded length limit now removed

time()

hardcoded length limit now removed

associative array keys

composite keys too

assignment of string to associative array
removed hardcoded length limit from probe names…

…but they're backed by string literals, so see the following:

I just noticed that string literals are still stack-allocated. seems easy enough to make them map-allocated, and this should enable longer probe names (important for C++ uprobes) and longer format strings for printf.
I think the only case left is variables, and there's an argument that those don't need to be in this PR. I'll check what the effort is… the solution will be different to the stuff I've done so far (i.e. passing a scratch map down the stack). I'm thinking I'll use one per-CPU hash keyed on variable name, with values sized to strlen_, and clear its contents every time our probe is entered.
I'm also seeing that some care is needed to safely receive contents of scratch maps via accept(). You don't need to lifetime end them (they're reusable). But the re-use introduces a nuance: you need to make sure you've copied the contents somewhere safe (or finished using them) before the scratch map gets re-used (i.e. if you accept() multiple args that will write over the same scratch space). I've dealt with all of these cases already, but I'm wondering if there's any helper function I can introduce so that you don't need to think about the difference between receiving an AllocaInst or a scratch CallInst via accept(). think I'll look at all the cases I've done so far and see whether they have anything in common.
another option is "rather than one map shared by all str() calls: use the map key to distinguish between instances of str(), and so have a map entry per str() call". this way there'd be no re-use of map entries… so you don't need to work out "do I need to make a copy of this content". it'd increase our memory use (by n times the number of times we call any given function) and add (n times) the number of distinct map lookups (and error handling conditions). maybe LLVM can optimize that away… they have a lot of instructions in common. but the upside would be simpler, safer codegen. and we'd be avoiding copies (probe read copies, which I suspect LLVM can't elide), so it might be better for perf as well as simplicty (albeit with marginally higher memory allocation… well, same amount of allocations as when we stack-allocated… but into maps now, and potentially multiplied by number of CPUs). hm, I'm tempted to try this option.

(Just passing by to say thanks for the work! This has been annoying me occasionally.)
FWIW I tested this, doesn't seem to break anything I'm using (nothing very sophisticated).
There's still a size limitation (trying to set BPFTRACE_STRLEN to > 31kish (so 32k onwards)) fails but it's a very nice step forward!
If there's anything a bystander can do to help (review on #1943?) feel free to say

@martinetd thanks for trying out the branch 🙂
yeah, I think this implementation supports everything, and doesn't break anything else. for the platform I was testing on, I think I got it down to either "all tests passed" or "all tests passed, until I rebased". probably there were other-platform tests failing, which could presumably be made to pass by regenerating fixtures.
hah, I never tried a number as large as 32k. that's interesting; I suspect that's a limitation of probe_read(), and wonder whether it could be bypassed by emitting multiple probe_read()s in a loop.

If there's anything a bystander can do to help (review on #1943?) feel free to say

Assuming "1943"=https://github.com/iovisor/bpftrace/pull/1493: I haven't done everything I could to get that over the line — i.e. I haven't implemented @fbs's suggestion of "reduce test churn by leaving the lines closer to their original order, or splitting that into a separate review".
I agree with it (reviewing 93 bytecode fixtures is hard enough as it is, and I'm happy there are people willing to do so), but at the same time it opened my eyes to how much effort it's going to take to make this reasonably reviewable.
There are a few reasons I'm not sure how to move forward with this:

probe_read() generates substantially messier bytecode than memcpy

i.e. because we have to verify whether it succeeded — which creates a conditional branch, whose negative case has to send an error message to userspace and end the BPF program

our strategy for exiting the BPF program is "return from the current function" — which relies on being the outermost function… frequently true, but I'd prefer exit if I knew a way to do so. [0]
we emit a "verify probe_read() outcome, and handle negative case" per copy, which can mount up to a lot… but I wonder whether we could define it as a function and re-use it? again, this would require some way to do exit (returning from a function doesn't end our program).

variables stored off-stack require allocation of maps — again, messier bytecode

and again, we have the same "verify whether it succeeded" messiness mentioned above
though, I wonder whether we could verify all BPF map allocations (i.e. all bpftrace arrays, sufficiently large variables, and all strings) in one condition at the start of the program? i.e. AND all of their pointers together, and require a non-nullptr result.
dunno whether the BPF verifier is clever enough to understand that one statement is capable of verifying multiple variables?

the test churn would actually be far more than the large amount already evident in this PR. I'm currently taking advantage of "for strings smaller than 128 bytes, we use memcpy()", so currently the tests don't exercise the new probe_read()-based Copy/ZeroInit strategy at all (though they do test the new "strings and sufficiently large variables are stored in BPF maps)

for full coverage, I think that requires adding an entire dimension to our test matrix (i.e. duplicate almost every existing test, but for "big string" mode)
tests involving probe_read() are harder, because you have to mock out the memory address to be deterministic

I wonder whether there's a performance penalty for using BPF maps instead of on-stack variables, or for using probe_read() instead of memset/memcpy.

in which case, should there be a threshold over which we switch to BPF map + probe_read()?

this could make codegen and testing complicated

would we even go as far to say "not all strings/buffers have to be the same length"?

this could make codegen complicated (becomes harder to assign a string to a variable — the variable may not be big enough to hold it)

though, I do think there's enough information available at the time of semantic analysis to prove that a variable only ever has small constants assigned to it. otherwise you could fall back to "make it big enough for largest known string".

LLVM's BPF backend is slow at emitting (or optimizing a program which contains) lots of 1-byte store instructions — which is how we use memset/memcpy (note: we switch to probe_read() strategy for strings larger than 128 bytes)

well, maybe it's faster in newer LLVM; I only tried LLVM 6.
I think LLVM knows this is slow, which is why they set a "do not allow more than 128 stores in a row" limit (you can't even get around it by calling memcpy/memset multiple times).

by the way, if LLVM's memset/memcpy could be made fast (and the limit removed): then I would strongly consider using that instead of the probe_read() helper (to avoid the difficulty of having to verify whether it succeeded). that said, there's still an argument that probe_read() makes more sense for large values — emitting one call to a helper (albeit with lots of messy verification/error-handling) rather than emitting many, many stores. dunno what the cost of "lots of stores" is — makes the BPF program longer, and more instructions for the verifier to scrutinize. but once verified: I think executing many stores is probably very performant, if they match the CPU's native word size.

8-byte store instructions (i.e. machine's native word size) would be far faster, and I expect result in a substantially more performant BPF program

to be honest, I actually tried this (align all CREATE_MEMSET() and CREATE_MEMCPY() to 8 bytes), and almost all tests passed. it's definitely something we would benefit from pursuing. the specific approach I'd recommend is "Copy and ZeroInit helpers should be given full information of the layout of structure that they're clearing, so that they can choose the appropriate byte alignment).

there are probably some memset()s that we don't even need to do. but I think we've lost the knowledge of "in what situations is memset necessary" (there was some issue on some old kernel, whose BPF verifier wasn't as clever as nowadays) https://github.com/iovisor/bpftrace/issues/1392

we haven't defined how variables are scoped in bpftrace (i.e. block-local versus program-local). it actually turns out that they're program-local, because we hoist all Allocas to the start of the program. this creates a guarantee (was this intentional??) that no matter which condition or block you're inside: the variable has already been declared.

thus, in this PR: when I changed variables to be off-stack: I hoisted them such that the BPF maps for them get allocated, verified and zero-inited at the start of the program. I think though that that's not quite how it worked before (i.e. the zero-init used to be block-local? something like that). anyway, this'll have resulted in yet more test churn.

So… overall there's a lot of fronts on which I'm not sure of how to proceed. some of it is "how do we implement this" and some of it is "how do we test this".
I did begin a (non-exhaustive) checklist of how to carve up the PR:

getMapKey() safety
remove join type
remove STRING_SIZE
printf() use scratch buffer

"fatal" support on probe_helper
align_to?
probe_read use Value* ?

remove conditions in join() codegen

moved into CreateGetJoinMap()'s fatal probe_helper

str() use scratch buffer
buf() use scratch buffer

32-bit size
align_to?
printers

variables
plenty more stuff, probably

But that checklist was "how to get it in in its current state", and doesn't address "should we have implemented/tested it like this".
[0] I think exit does exist in BPF (but not in the LLVM BPF backend). I saw that Alessandro Decina's succeeded in doing it via inline ASM. I had a stab at doing the same; here's what I tried, but LLVM told me it wasn't supported:
#include <llvm/IR/InlineAsm.h>
CallInst   *CreateExit();
/*
 * The LLVM BPF target does not support core::intrinsics::abort().
 * Thanks to Alessandro Decina for explaining how to emit an exit instruction
 * https://blog.redsift.com/labs/oxidised-ebpf-ii-taming-llvm/
 * https://github.com/redsift/redbpf/blob/320e3d1/cargo-bpf/src/llvm.rs#L83
 */
CallInst* IRBuilderBPF::CreateExit() {
  FunctionType *func_ty = FunctionType::get(getVoidTy(), false);
  InlineAsm *inlineAsm = InlineAsm::get(func_ty, "exit", {}, false);
  CallInst *call = CreateCall(inlineAsm);
  call->addAttribute(AttributeList::FunctionIndex, Attribute::NoUnwind);
  // call->addAttribute(AttributeList::FunctionIndex, Attribute::NoInline);
  // call->addAttribute(AttributeList::FunctionIndex, Attribute::ReadNone);
  call->addAttribute(AttributeList::FunctionIndex, Attribute::NoReturn);
  return call;
}

// and… maybe we have to add this to the CMake config? I tried that too though.
add_compile_options("-D__ASM_SYSREG_H")

wow. Thanks for the very detailled answer, I didn't expect quite that much!
I don't know if that helps, but the reason I tried >32k was for a uprobe actually -- in there it's frequent enough to pass large arrays in arguments and I wanted to dump SSL read/writes as described in https://confused.ai/posts/intercepting-zoom-tls-encryption-bpf-uprobes (for which they implemented https://github.com/alessandrod/snuffy based on https://github.com/redsift/redbpf -- I boasted that was overkill and I could do the same in a few minutes with bpftrace and well that wasn't quite so easy due to the size limitation!)
honestly at this point performance isn't so critical, but I assume the similar could be done for read/write kernel calls if required :P
I'm not too familiar with it but the magic seems to be in https://github.com/alessandrod/snuffy/blob/master/snuffy-probes/src/snuffy/main.rs#L254 where they iterate BUF_LEN (368 bytes) at a time through bpf_probe_read+corresponding event -- pretty close to what you were first suggesting; they don't care about checking everytime to break the loop ; maybe a helper like they have could help keep code sane-ish.
performance-wise it would obviously need testing but intuitively probe_read should be faster than byte per byte copy on large buffers, I honestly don't know... :/
Well, that was more of a play thing than an actual need, so let's put >32k aside for now if that helps.
Regarding the question, would printf("%s", str()) be good enough for me? Probably, I'm not sure.
For that particular example I gave, I'd probably want to print it a bit more nicely (e.g. hexdump or something, people send weird things over tls!), but that would probably be more practical to postprocess than print in bpftrace itself? So just format  and postprocess would work I guess.
However the most common usecase I'd actually have that often requires more than 200 is just dumping paths (think opensnoop for example); 200 is too short with a deep filesystem like I often have to deal with, I'd need at least 500 and if possible 4k. filenames can have linefeeds so just printing "is bad" but in the context of debugging it's ok for me.
I was thinking that wouldn't work because I usually want to print at sys_exit the arg I got at sys_enter, but for things like openat it's fine storing the address of the buffer at enter and not the string, so it works ok... There might be syscalls where I'd want to store the actual data on enter like write?
Well, it's safe to say it would cover most use cases for sure anyway.
All in all if you don't know I don't either :D Please do what you think is best within the time you have ;) and thanks again!

detailled answer

ah, that was something long overdue — I'd left it to languish, so wanted to at least document the blockers.

do the same in a few minutes with bpftrace

🙂 always the dream. seems like a perfect use-case to try it on.

intuitively probe_read should be faster than byte per byte copy on large buffers

Aye, though I suspect the lesson here is "it'd be really nice to copy word-by-word instead".

I'd probably want to print it a bit more nicely (e.g. hexdump or something)

Okay, so probably printf("Datagram bytes: %r\n", buf(args->buff, args->len)) is more your speed — that's fine; it works very similar to str(), so still achievable.

However the most common usecase I'd actually have that often requires more than 200 is just dumping paths (think opensnoop for example)

If you just want to print them, then this printf("%s", str()) proposal should be able to do that without storing anything on stack.
But if you need to store it in a variable on-stack, do some calculations and then printf() something afterwards… well, at least moving the printf() buffer off-stack gives you a bit more room to play with that string on-stack. 500~4k would need off-stack variables at that point.

Just a note that with bpf ring buffers now available that might be easier to use than per cpu scratch buffers, albeit a bit less performant perhaps? It doesn't remove the need to check for potential failures and bail out though so that might not actually help at all for the problems you listed :/
For my own self-centered needs, I was able to straight-forwardly update bcc's libbpf-tools/opensnoop program to handle paths > 254 that way, since ring buffers can be used as a mostly drop in replacement. It does require a recent kernel and is far from as flexible as bpftrace but I'll try to waddle through like that for now when the need occasionally arises.

interesting! presumably that eliminates the need to allocate maps? unfortunately that's not the hard part 😅
yeah, the hard part is the validation/bailout for acquiring a map. as you say: we'd still have that problem.
and I think we'd still have the problem of "now that you have this big storage: how do you do big copies/clears onto it?" presumably that'd still be "do a bpf_probe_read, with validation/bailout".
and yeah, as you say: it's new-kernel only. if we wanted to enable it conditionally, that adds a condition to the codegen and adds another dimension to the test matrix.
so, glad to know about it. but my initial reaction is that the hardest problems are still there. 😅

@mmarchini hey, good to see you back. 🙂 I'm not actively developing Big Strings. still interested.
what approach are you thinking of taking? there's a spectrum of how far it can be taken.

map-backed scratch buffer for formatting strings (e.g. printf)
map-backed string datatype (affects every expression which consumes strings)

here's most of them

map-backed buffer datatype
map-backed variables
implement copying via probe_read()

this PR demonstrates a working implementation of all of those.
hard parts:

all this map validation adds a lot of blocks to the IR

lots of churn in the test fixtures
hard to read, maintain or review

using probe_read() to do large memset/memcpy creates challenges for tests

the address changes on each run, so you need mocking or suchlike to overlook that
I cheated here; only utilised probe_read() for large strings (the tests use small strings)

some ideas:

there may be alternatives to probe_read() for large strings (>1014 bytes)

emit many store instructions (I haven't tried this, and don't know whether LLVM tries to fuse them into a memset)
there is actually an LLVM option -bpf-expand-memcpy-in-order to turn off the "max stores" limitation, but I don't know how to enable it (or whether it's exposed at all via programmatic API).

it may be disabled for a good reason (e.g. causing some optimization pass to go too slow)

we may need a solution for "how to keep our IR simple when we have many maps to validate"

proposal of how to lay out our blocks

sub-proposal (stop emitting allocas in reverse order)

is there a way to make re-usable "map validation failure" handling?

"send perf event explaining the error" is a lot of IR, and we do this per map.
I did try (in a branch somewhere) defining this sequence as a function, but LLVM inlined it to each call-site so I got duplicate IR regardless.
maybe another option is to && every map validation together, so we only need to handle one failure for n maps?

you'd get poorer error feedback (i.e. which map failed, and which AST node is to blame), but better IR
only possible if the BPF verifier is smart enough

once you have big strings, you get a new problem: tiny loads/stores.

with (our default) 1-byte-aligned accesses: a bpftrace program where "all strings are big" takes super long to compile (tens of seconds)
with 8-byte-aligned accesses: the same program can compile sub-second.

not sure whether this was because it's simpler to compile once word-aligned, or instead because there were fewer instructions to run the optimizer on
pretty much nothing broke when I set everything to 8-byte-aligned (maybe 1 runtime test? maybe an alignment-specific one?)
LLVM can compute a common alignment given a source and destination address. in this PR, I refactored a few places where we weren't yet using StructTy or were using vaguer types than we needed to. I think it's achievable to give our memset/memcpy calls enough information to pick an optimal alignment. this would be a good general improvement for bpftrace, and I think isn't too hard to implement — just not sure how to test.

another consideration is the proposal of a non-LLVM backend (which I assume is what you were getting at with "when installing llvm is unfeasible"). if there's any part of shipping this feature that's made harder because of LLVM (e.g. the fixture-based testing), then we do have the option of supporting it via another backend.

This work is now ongoing in https://github.com/iovisor/bpftrace/pull/2746 . There's been a lot of great activity in this PR, but due to age I'm closing it out.


Separate Organizations into Meetups, Conferences, Code Schools

We already have this data https://data.openupstate.org/organizations but the same would be very useful on Hackgreenville.com
There is an "organization_type" in the organizations API to filter by organization type
https://github.com/codeforgreenville/OpenData/issues/17

Organizations are now seperated by type and inactive.
Links to the homepage and events page are also show when an organization is clicked (collapse link idea via @SpaceCowboy326 )
https://hackgreenville.com/orgs

@magoun be sure to do a git pull before you do any future work.


Given examples

I run MacOS 10.11.5. Regarding the given examples I would like to know if they should work out of the box and what the expected audio output should be. Maybe I have interpreted the examples wrongly but the following code should produce no hearable output (or even silence the system if music is running) but actually results in "tr-tr-tr-tr-..." noises coming out of the speakers that get louder over time.
var coreAudio = require('node-core-audio')
var engine = coreAudio.createNewAudioEngine()

engine.addAudioCallback(processAudio)

function processAudio (buffer) {
  console.log('%d channels', buffer.length)
  console.log('Channel 0 has %d samples', buffer[0].length)

  for (var sample = 0; sample < buffer[0].length; sample++) {
    buffer[0][sample] = 0.0
  }

  return buffer
}

Check out the #26.

You can continue your experiments Mr.


Add editable version of docs images

This is a Feature Request
What would you like to be added
As a contributor to docs localization I would like to have an editable version of image so I can translate it without need to recreate the diagram from scratch.
Why is this needed
Not all SVG images have text editable and due to the requirement to reflect the translated docs text, images need to be translated as well. But since some SVG images do not have an editable text it is not possible to translate them without redrawing the content from scratch which is not convenient.
Comments
e.g. https://kubernetes.io/docs/concepts/overview/#going-back-in-time

There isn't a simple way to add back the original SVG source for existing SVG images. Probably we have to create new SVG files for these images.

@mboukhalfa what scope did you want for this issue:

all raster images on the website
just some of those images

If the scope is for all of the raster images, we'd probably treat this as an umbrella (overall) issue, and then file issues for making (or tracking down) vector versions of specific images.

The image in https://kubernetes.io/docs/concepts/overview/#going-back-in-time doesn't have editable text. You can however load it into a vector editor and type in new text, deleting what is already there.

I used Git to look at the history of this diagram. It looks as if it was added in https://github.com/kubernetes/website/pull/555 and was later edited to use the Calibri typeface.
If editing it, try to use Open Sans.

@mboukhalfa what scope did you want for this issue:
* all images on the website that don't have editable text

* all raster images on the website

* just some of those images

If the scope is for all of the images that don't have editable text, we'd probably treat this as an umbrella (overall) issue, and then file issues for making (or tracking down) vector versions of specific images.

Umbrella issue sound good

The image in https://kubernetes.io/docs/concepts/overview/#going-back-in-time doesn't have editable text. You can however load it into a vector editor and type in new text, deleting what is already there.

you mean just erase the old text and insert editable text which tools you recommend would be great if we have something online

I used Git to look at the history of this diagram. It looks as if it was added in #555 and was later edited to use the Calibri typeface.
If editing it, try to use Open Sans.

For me, I would use this diagram in new ar localization so I would use a different font

The tool I use to edit SVG images is Inkscape, which is open source donationware. Any other tool that talks SVG is also viable.

/area localization
I'm not overly familiar with the images/diagrams on the site. Some questions i have around this:

Do we have an idea of how many images will need to be updated? Is there an easy way to get a list of all the to update? Or is the "best" way to manually review the images that we should update
Are you able to localize mermaid diagrams? (more info on mermaid diagramming for context)

I'm able to edit text on a copy of an SVG image using a regular text editor. Is this an acceptable way of localizing SVG graphics from the website?

/reopen

I guess this one https://kubernetes.io/docs/concepts/overview/#going-back-in-time is not editable as you confirmed on slack discussion https://kubernetes.slack.com/archives/C1J0BPD2M/p1686152844473749?thread_ts=1686074817.552909&cid=C1J0BPD2M

Perhaps we could replace SVG images that don't have editable text nodes for ones that do. That would enable us to localize specific images in the website.
I'm not super experienced with Mermaid diagrams, but in my (limited) previous experience I was able to edit text inside of those as well.


Proxy Support

Can we have proxy server support added to the node library? This is blocking use in an environment I have.
Ref on a similar request in the java SDK: https://github.com/Azure/azure-documentdb-java/pull/22

Would be happy to add this to the backlog. If you want this sooner, we'd be happy to accept Pull Requests.

Any eta?  This is a blocker certainly for me and I would imagine many other people working in enterprise.

@ButchersBoy We haven't got to this yet but if you could send us a PR with the changes, we can get it reviewed and merged sooner.

This is supported.

This issue is close... so what`s the solution?


Addition of BackToTop Button | Issue #112

📥 Pull Request
Description
I have added a back to top button in the web app.
https://github.com/user-attachments/assets/8c0b1728-a01f-41a2-a1fd-69ab30cc32d3
Fixes #112
Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[X] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Documentation update

Checklist:

[X] My code follows the style guidelines of this project
[X] I have performed a self-review of my code
[X] I have commented my code, particularly in hard-to-understand areas
[X] I have added tests that prove my fix is effective or that my feature works
[X] New and existing unit tests pass locally with my changes
[X] Any dependent changes have been merged and published in downstream modules

👍

But not merge

This is a good start. I think you can improve the UI/UX by using https://github.com/CodeHarborHub/codeharborhub.github.io/tree/main/src/components/Buttons. You can also add a button at the bottom of the page.

This is a good start. I think you can improve the UI/UX by using https://github.com/CodeHarborHub/codeharborhub.github.io/tree/main/src/components/Buttons. You can also add a button at the bottom of the page.

@ajay-dhangar sir, so have to update ?

This is a good start. I think you can improve the UI/UX by using https://github.com/CodeHarborHub/codeharborhub.github.io/tree/main/src/components/Buttons. You can also add a button at the bottom of the page.

@ajay-dhangar sir, so have to update ?

You can try to work on the UI/UX by using the given link. You can also add a button at the bottom of the page.

@ajay-dhangar sir, I have added both the buttons just like in the above repo but with smooth scrolling 😁


Added ability to establish the local IP address

I had to use an unsafe block round about here.  I'd have avoided this if I could, but I can't find a cleaner way to get the hostname, which is required in order to get the local interfaces.  The solution is drawn from here.

Coverage decreased (-1.66%) to 85.15% when pulling 09ff2ffead4fe362a1e1e33e63c3d7eb71c6b9ce on Fraser999:get_local_address into f1269b6ad9f5359245a16c78cdffae848c17aed0 on maidsafe:master.


relative path locations wrong when using @use

Fixes https://github.com/microsoft/vscode/issues/142836
Fixes https://github.com/microsoft/vscode/issues/153237

Thanks a lot for this guys! Great to see you found a way to fix this!


Prevent Python interpreter from crashing when the blossom algorithm fails

Adds BlossomFailureException which is thrown when lemon::MaxWeightedPerfectMatching fails to solve the matching problem, preventing a segmentation fault. If a BlossomFailureException is thrown when local matching is used, it is caught by pymatching.matching._local_matching, which increases num_neighbours and retries decoding. If multiple attempts of this fail, the BlossomFailureException is re-raised.
Fixes #11

Codecov Report

Merging #14 (a142e3e) into master (ce5e913) will not change coverage.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##            master       #14   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files            3         3
  Lines          165       176   +11
=========================================
+ Hits           165       176   +11

Impacted Files
Coverage Δ

src/pymatching/matching.py
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ce5e913...a142e3e. Read the comment docs.


Don't serve requests with incorrect or empty Host header

Generally is not a good idea to serve all requests on the blog IP even with empty or unknown host header. The good practice is to serve requests with correct 'Host' header and return 400 otherwise.
See rfc2616 for details.

Hi rmrio,
thanks for your PR. Could you submit it again against the development branch? The master branch is just updated with releases. I will update the readme to reflect this.
A couple of things:

Do we really need a new dependency for this?
How much overhead does this create for every request?
Journey has a "pages" feature (see #18 ) that lets users host static pages. Will this change allow redirecting different domains to journey pages (e.g. www.mygreatproject.com to www.myjourneyblog.com/pages/greatproject/)?


added newline character check in url in response.redirect function

currently express framework do not check for newline characters
it is vulnerable to http splitting attack
example:
start the test server https://gist.github.com/1654555
browse http://localhost:3000/?foo=bar
Set-Cookie%3Aname%3Dvalue
you will be re redirected to another page and cookie will be set

if you're accepting user-input it should be filtered at that point

I completely agree that user-input should be checked this doesn't imply that user-input is not required.
I agree with you that this issue would apply to any method that may cause unintended header injection.
And I was also thinking where it should be handled within nodejs or express. I personally think checking it should be at least in express for now.
But looking at the redirect method, my opinion is that url parameter itself is the user-input (from the API point of view), and url parameter should be a valid url that can be redirected safely. This method passes url parameter to headers as it is and newline will definitively cause input to be injected to headers.
Example shown actually is a simple scenario which accepts the url parameter directly from request parameter. But considering applications that you have not accepted the url from "user" but may be coming from a database where it have passed user-input validations due to missing or insufficient controls in another entry-point, or being provided to your application as a return url.
I added the check right after the url where it is mapped. Here again the question was trustfulness of final url, since it may also be constructed via app.redirect mappings. It may be constructed using some data that comes from model which has such injection and not escaped.
In asp.net framework, redirect method of response class checks the url parameter if it contains any newline characters at first place.If it contains it throws exception and I followed the same approach.
I am not php/ruby guy but as I researched it seems that;

In php this been fixed in version 5.1.2 (http://php.net/releases/5_1_2.php).
In rails framework this has been reported and fixed removing CRLF in this commit (https://github.com/rails/rails/commit/ba80ff74a962). Their approach and case was also similar (as mentioned in the comment "CGI.rb and mongrel don't do any sanitization of the contents of HTTP headers, so care needs to be taken"), as nodejs behaves same.

I am not sure which behavior fits best, throwing error or sanitizing, but personally I think there should be a prevention in express framework.

I'm -1 for throwing because then nearly every res.redirect() call would need a try/catch. If we were to solve it lower in the stack it should be even lower than res.redirect since that still leaves everything else open.

if you're taking it to node core you can show them something similar to:
http.createServer(function(req, res){
  var body = 'oh noes';

  var userInput = 'foo\r\n'
    + 'Content-Length: ' + body.length + '\r\n'
    + '\r\n'
    + body;

  res.setHeader('Location', userInput);
  res.end();
}).listen(3000);

I cant remember if the nul byte thing got in or not but this is pretty similar

I had also fixed and pushed to my fork :) I was opening an issue with node and then received the notification :)
I did it it at the same location but I also checked for null bytes.
It is nice that you have also included test :)


key not appearing on coBarplot

Describe the issue
I am making a coBarplot, but not all the variants present are appearing in the key. e.g. The red does not appear in the key.  Is there any advice you have that I can enter into the R command to make the full key appear please.
Command
coBarplot(m1 = adenoma, m2 = normal, m1Name = 'adenomas', m2Name = 'normal tissue')

Hi,
Could you please post the output plot from the above command? By key do you mean the bottom legend?
You can scratch out the gene names if you don't want to display the results (set geneSize = 0.01).

Sorry... I'm new on here and learning. Yes I meant the legend size... and it worked now after I made the legendTxtSize smaller.
Thanks for getting back to me.

Sorry... I'm new on here and learning. Yes I meant the legend size... and it worked now after I made the legendTxtSize smaller.
Thanks for getting back to me.

No problem. I am closing the issue for now. Feel free to comment if you have more questions.


[BUG] Harvester UI Virtual Machine page shows "Restore" column even after none of the restore is in progress

Description:
Upon restoring a VM, the Virtual Machine page on Harvester UI shows the Restore column with the progress bar, but upon the completion of the restore, it does not go away.

Steps to reproduce the behaviour:

Setup backup target.
Create a backup for a virtual machine in harvester.
Restore backup to a new VM.
Check the Virtual Machine page the "Restore" column does not go away

Expected behaviour:

Restore column should be removed after completion of the restore.

Workaround:
You may need to delete the completed restore from the harvester k8s cluster. We have identified it as a bug and as a workaround for now, you can delete the completed restore.
kubectl get virtualmachinerestores.harvesterhci -A
NAMESPACE   NAME                               TARGET_KIND      TARGET_NAME   COMPLETE   AGE     ERROR
default     restore-devendra-s3-backup-z5h5j   VirtualMachine   restored-vm   true       5m26s

kubectl delete virtualmachinerestores.harvesterhci restore-devendra-s3-backup-z5h5 -n default

Environment:

Harvester ISO version:  v1.3.1

@torchiaf Explained the condition:
The restore column is displayed in the following case:
The VMs list contains at least 1 VM restored from a backup, even if that restore is completed
The column contains:
the progress bar +  percentage number if the restore is still in progress
completed if the restore action is completed
- (dash) if the VM is not created from a backup

https://docs.harvesterhci.io/v1.4/vm/backup-restore#vm-snapshot--restore has no detailed information about the VM page which with restore colume. It is worthy to update the document.

I can reproduce with the same steps https://github.com/harvester/harvester/issues/6195#issue-2416320314

@torchiaf Explained the condition:
The restore column is displayed in the following case:
The VMs list contains at least 1 VM restored from a backup, even if that restore is completed
The column contains:
the progress bar +  percentage number if the restore is still in progress
completed if the restore action is completed
- (dash) if the VM is not created from a backup

Hi @torchiaf @a110605
This does a little not make sense to me, what if the virtualmachinerestore is created before Harvester v1.3.0? It is completed but has no information about the progress, it seems like a similar situation as #5068, but it's for  virtualmachinerestore .
IMHO, the front end may not have to show this column if the related virtualmachinerestore is completed. WDYT?

Workaround:
You may need to delete the completed restore from the harvester k8s cluster. We have identified it as a bug and as a workaround for now, you can delete the completed restore.
kubectl get virtualmachinerestores.harvesterhci -A
NAMESPACE   NAME                               TARGET_KIND      TARGET_NAME   COMPLETE   AGE     ERROR
default     restore-devendra-s3-backup-z5h5j   VirtualMachine   restored-vm   true       5m26s

kubectl delete virtualmachinerestores.harvesterhci restore-devendra-s3-backup-z5h5 -n default

This workaround can't be applied.
The restoring VM's pvc has ownerReferences points to the matched CR virtualmahcinerestore, for example:
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  ....
  name: restore-vm-b1-434fd2c1-fd7a-4c0f-bda7-9ac31becef13-disk-0
  namespace: default
  ownerReferences:
  - apiVersion: harvesterhci.io/v1beta1
    blockOwnerDeletion: true
    controller: true
    kind: VirtualMachineRestore
    name: restore-vm-b1-5xcnx
    uid: 434fd2c1-fd7a-4c0f-bda7-9ac31becef13
  resourceVersion: "100298"
  uid: c7db72ed-d9c9-4fee-a424-14a3cefbb0bb
spec:
  accessModes:
  - ReadWriteMany
  ....
  storageClassName: longhorn-image-l8mlc
  volumeMode: Block
  volumeName: pvc-c7db72ed-d9c9-4fee-a424-14a3cefbb0bb

so once the virtualmachinerestore CR is deleted, the PVC will turn into Terminating status. If the restoring VM is stopped, the PVC will be completely deleted.

IMHO, the front end may not have to show this column if the related virtualmachinerestore is completed. WDYT?

Totally agree, I fixed the behavior so that the restore column will be removed once all VMs are restored.

Verified fixed on v1.3-5445e91-head. Close this issue.
Result
$\color{green}{\textsf{PASS}}$ After restore to new vm complete, the restore column will disappear $~~$
https://github.com/user-attachments/assets/b94521f4-be09-4ab0-a756-45221917e719

$\color{green}{\textsf{PASS}}$ After restore to replace vm (retain volume) complete, the restore column will disappear $~~$
https://github.com/user-attachments/assets/863d3a45-4304-415b-8f3d-1058248b0d52

$\color{green}{\textsf{PASS}}$ After restore to replace vm (delete volume) complete, the restore column will disappear $~~$
https://github.com/user-attachments/assets/81d539c9-768d-4cbf-b3b3-89785e052e3d

Test Information

Test Environment: Single nodes on local kvm machine
Harvester version: v1.3-5445e91-head

Verify Steps
 Restore backup to create new vm

Create a vm names opensuse
Setup nfs backup target
Create a backup named opensuse-backup
Restore the backup
Select restore to new vm
Enter the new vm name
Check the restore process complete
Check the restore column will disappear after restore process finish

 Restore backup to replace existing vm

Create a vm names opensuse
Setup nfs backup target
Create a backup named opensuse-backup
Restore the backup
Shutdown the vm
Select restore to existing vm
Select to retain (or delete) volume
Check the restore process complete
Check the restore column will disappear after restore process finish

Verified blocked on v1.4-4fe69de-head. Move back to Ready for testing.
Blocked by issue: #6312
Result
$\color{green}{\textsf{PASS}}$ After restore to new vm complete, the restore column will disappear $~~$
https://github.com/user-attachments/assets/4a744956-b4eb-40b3-80bf-bb7fbd53dce7

$\color{red}{\textsf{Failed}}$ Blocked since failed to restore backup to existing vm, stuck in Restoring and can't complete $~~$

https://github.com/user-attachments/assets/367b5227-2c44-44c7-ad7e-8218735e56ee

$\color{grey}{\textsf{Not started}}$ After restore to replace vm (delete volume) complete, the restore column will disappear $~~$

Test Information

Test Environment: Single nodes on local kvm machine
Harvester version: v1.4-4fe69de-head

Verify Steps
 Restore backup to create new vm

Create a vm names opensuse
Setup nfs backup target
Create a backup named opensuse-backup
Restore the backup
Select restore to new vm
Enter the new vm name
Check the restore process complete
Check the restore column will disappear after restore process finish

 Restore backup to replace existing vm

Create a vm names opensuse
Setup nfs backup target
Create a backup named opensuse-backup
Restore the backup
Shutdown the vm
Select restore to existing vm
Select to retain (or delete) volume
Check the restore process complete
Check the restore column will disappear after restore process finish

@WebberHuang1118 @torchiaf. Can you please confirm if I am missing something ?
I verified this on v1.4-00cd4ee-head (sep 19 build) and looks like restoring an existing vm is stuck.It is more than 40 mins its been restoring

Hi @rrajendran17
I can restore the existing VM with the master-head builds 9/18 (0e15f01377df856b7b00316fbeb57c4a25f13833) and the latest (606f485579d087def560e2a8a255a2b0f4bad019).
Can you check again with the latest build? and provide the SB if needed, thanks.

Hi team,
I just have a chance to retest this issue on recent Harvester v1.4-head.
According to following test result, it looks like can restore backup well to the new vm and existing vm with both retain and delete volume options.
After restore  complete, the Restore column would also disappear.

Test result
Verified Fixed on v1.4-97e60e1-head (24/09/29)
Result
$\color{green}{\textsf{PASS}}$ After restore to new vm complete, the restore column will disappear $~~$
https://github.com/user-attachments/assets/18852e6c-e3a3-4cb9-9669-5d59e38d4863

$\color{green}{\textsf{PASS}}$ After completely restore backup to existing vm with retain volume, the restore column will disappear $~~$
https://github.com/user-attachments/assets/c4915aea-c245-4e2f-aeca-a9e6fe594ca5

$\color{green}{\textsf{PASS}}$ After completely restore backup to existing vm with delete volume, the restore column will disappear $~~$
https://github.com/user-attachments/assets/bb4d6beb-8d8f-429e-b3ff-5b822a1cae1a

Test Information

Test Environment: Three nodes kvm machines
Harvester version: v1.4-97e60e1-head (24/09/29)

Verify Steps
 Restore backup to create new vm

Create a vm names opensuse
Setup nfs backup target
Create a backup named opensuse-backup
Restore the backup
Select restore to new vm
Enter the new vm name
Check the restore process complete
Check the restore column will disappear after restore process finish

 Restore backup to replace existing vm

Create a vm names opensuse
Setup nfs backup target
Create a backup named opensuse-backup
Restore the backup
Shutdown the vm
Select restore to existing vm
Select to retain (or delete) volume
Check the restore process complete
Check the restore column will disappear after restore process finish

@rrajendran17,
May I know the vm image used in your previous testing scenario.
And I am thinking whether the following factor would affect the restore result

Failed to attach volume for some reason
The image have been encrypted
Webhook not immediately reflect the changes (Maybe can try refresh the page)

@TachunLin I used v1.4-00cd4ee-head (sep 19 build) when testing the changes.

After checking with @rrajendran17,
In the stuck-restoring case, the NFS service is running in the VM, which is going to be replaced, so the volume can't be restored.
Since it's not the expected use scenario, and @TachunLin already double-check this issue, after discussion with @rrajendran17, we agree to close this ticket, thanks.


what is the difference between tile_dir and tile_extract in the config

I'm trying to get the opentraffic servers running again and run into troubles getting the services to run at all. In the end I discovered that the planet_YYYY_MM_DD-HH_MM_SS file & directory weren't created by the scheduled task which calls scripts/cut_tile.sh
Diving into the entire infrastructure I discovered that the valhalla config json contains:
   "tile_dir": "/data/valhalla/tiles",
   "tile_extract": "/data/valhalla/tiles.tar",

so it seems like it expects a tiles.tar file, but the script doesn't generate that. I've stepped through the script manually and once I got a /tiles directory I created the tar file, restarted the process and now at least the various servers boot again, but the routing results show gaps which shouldn't be there.
I'm wondering why you need both settings

You dont need both settings. But its convenient to have them. Basically what we do is we have a config generation tool which is self documenting. The way it achieves the self-documentation is that it provides default values for all the config parameters (even the optional ones). Hence both are present in the generated config. Now there is a further distinction.
We use the config both for running the route server but also for generating routing tiles. For the former we prefer the extract (ie tar) over the dir of tiles because its more performant. For the latter though, we need the tile_dir`.
I'd have to check to through the code to be 100% sure but I do think there are some cases where even though you are using the tile_extract the code will expect the tile_dir to be populated. This is just a result of the fact that we had the dir first and added the extract capability later.


fix direct link in docs/concepts/configuration/pod-priority-preemption

also a direct link issue

/assign
/approve
/lgtm


Initial setup for unit test using mocktail added

Initial setup for unit test using mocktail added

Closing this PR as it is only library reference. Please open a new one when you have some test case associated with the mock service.


unexpected keyword 'overwrite_appengine_config' error while deploying the app to app engine

I'm having issues following the steps to deploy
+ echo Deploying to App Engine...
Deploying to App Engine...
+ bazel run upvote/gae:monolith_binary.deploy -- rv3-upvote app.yaml santa_api.yaml
INFO: Invocation ID: bbd6d94c-828d-4fa4-97e6-80cad0f54c3c
Loading:
Loading: 0 packages loaded
ERROR: E:/server/upvote/upvote/gae/BUILD:59:1: Traceback (most recent call last):
        File "E:/server/upvote/upvote/gae/BUILD", line 59
                upvote_appengine_test(name = "appengine_config_test", si...", <2 more arguments>)
        File "E:/server/upvote/upvote/builddefs.bzl", line 8, in upvote_appengine_test
                py_appengine_test(name = name, srcs = srcs, deps = dep..., <3 more arguments>)
unexpected keyword 'overwrite_appengine_config' in call to py_appengine_test(name, srcs, deps = [], data = [], libraries = {}, size = None)
ERROR: error loading package 'upvote/gae': Package 'upvote/gae' contains errors
INFO: Elapsed time: 0.402s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)
ERROR: Build failed. Not running target
FAILED: Build did NOT complete successfully (0 packages loaded)

I have tried overwrite_appengine_config = false, on upvote\gae\BUILD  file but that didnt work.
And the build is still failing after the last update.
Any ideas?
Thanks!

Alexander, can you confirm again that this is fixed before I close it out?

ya personally I didn't have this issue, nor do I since the last merge to master.


Windows Unit Test Support

Adds projects to build the unit test code and run it in appveyor.
Reverts appveyor image to previous to get kernel mode building again. They seem to me mucking around with the images right now, causing things to break.

I filed a bug for the spurious build failure: #631. This is the second time I've seen the failure on that particular test.

Current coverage is 65.15% (diff: 100%)

Merging #629 into master will increase coverage by 0.01%

@@             master       #629   diff @@
==========================================
  Files            98         98
  Lines         14386      14416    +30
  Methods        1971       1972     +1
  Messages          0          0
  Branches       1683       1687     +4
==========================================
+ Hits           9371       9393    +22
- Misses         4480       4486     +6
- Partials        535        537     +2

Powered by Codecov. Last update 9a2beff...406f9ea


chore: switch to github app automation tokens

coverage: 100.0%. remained the same
when pulling 2133014df6cecf7bbb9d140228d77e2257a9b272 on automation
into 39953d3a5910c23e6a08a4a20adc20744571c4a1 on main.


Do the Object "Motion detected" works in the actual version?

Do it works with V1.0.0 and DSM 6.2.4-25556 and SS 8.2.9-6459?
I activate the history for this, but no motion are detected, but on DSCAM app I can easily jump to each recorded detected motion.
Regards

I have no idea what's in the log? (listCameras - Response: )


Cleanup Readme content of management plane libraries

The following libraries have the content of the Readme.md with the actual guidelines of the readme and these should be cleaned up with relevant introduction for the particular libraies -

This is the content that is pulled from the template readme and should be replaced with relevant content for the individual libraries -
README.md template
Use the guidelines in each section of this template to ensure consistency and readability of your README. The README resides in your package's GitHub repository at the root of its directory within the repo. It's also used as the package distribution page (NuGet, PyPi, npm, etc.) and as a Quickstart on docs.microsoft.com. See Azure.Template/README.md for an example following this template.
Title: The H1 of your README should be in the format: # [Product Name] client library for [Language]

All headings, including the H1, should use sentence-style capitalization. Refer to the [Microsoft Style Guide][style-guide-msft] and [Microsoft Cloud Style Guide][style-guide-cloud] for more information.
Example: # Azure Batch client library for Python

Azure Template client library for .NET
Introduction: The introduction appears directly under the title (H1) of your README.

DO NOT use an "Introduction" or "Overview" heading (H2) for this section.

First sentence: Describe the service briefly. You can usually use the first line of the service's docs landing page for this (Example: Cosmos DB docs landing page).

Next, add a bulleted list of the most common tasks supported by the package or library, prefaced with "Use the client library for [Product Name] to:". Then, provide code snippets for these tasks in the Examples section later in the document. Keep the task list short but include those tasks most developers need to perform with your package.

Include this single line of links targeting your product's content at the bottom of the introduction, making any adjustments as necessary (for example, NuGet instead of PyPi):
Source code | Package (PyPi) | API reference documentation | Product documentation

This work is in progress.
AB#3997

Given the age of this and linked internal tracking issue having been closed, closing this out as stale.  Please feel free to reopen if this is still needed.


Error: There is no event type keyPress

Describe the bug
onKeyPress is not working in react.
<Editor
onKeyPress={()=>console.log("pressed..")}
/>

To Reproduce
Use the editor in react and pass onKeyPress={()=>console.log("pressed..")}
The same is with onKeyDown
Expected behaviour
It should work
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):

OS: [e.g. iOS]
Browser [e.g. chrome, safari]
Version [e.g. 22]

Smartphone (please complete the following information):

Device: [e.g. iPhone6]
OS: [e.g. iOS8.1]
Browser [e.g. stock browser, safari]
Version [e.g. 22]

Additional context
Add any other context about the problem here.

@ats1999
We do not provide the keypress event.
I'll add this event if necessary.
When I checked, onKeydown works well.
ReactDOM.render(
  <>
    <Editor
      previewStyle="vertical"
      initialValue={content}
      height="400px"
      onKeydown={() => console.log("pressed..")}
    />
  </>,
  document.getElementById('editor')
);

Is there any other reason?

@js87zz thanks


Prevent migrations being published multiple times

This pull request changes the way migrations are published so they use fixed timestamps, to prevent issues where the same migrations could be published multiple times.
When you run the vendor:publish command and attempt to publish something which has already been published, Laravel will ignore publishing the file and give you a warning.
However, in the case of the Eloquent driver, because our "destination filename" was dynamic based on the current date & time, the filename Laravel would check against would never match up to an existing migration. This could lead to the same migration being published multiple times.
While Laravel 11 does include a new publishesMigrations method (introduced in laravel/framework#49246), which replaces the timestamps in migration filenames, it has the same pitfall we were running into before in that it doesn't check for existence for past migrations.
This PR adjusts how migrations are published so all the migrations have a fixed timestamp, rather than a dynamic one so Laravel can recognise any duplicates before publishing.
I adjusted the name of the migration files themselves (not the stubs in updates though) and also moved the migration code into a new publishMigrations method to tidy things up a little.

Seems good. Do you want it merged or is the plan to hold off for v5?

Seems good. Do you want it merged or is the plan to hold off for v5?

Yeah, let's get it merged, it'll allow for testing the Core PR. Just wanted to check it looked good for you before merging since I've haven't touched this package much 😄

This causes issues with existing installations as my system now tries to migrate all of those again as the name changed from 2024_02_07... to 2024_03_07... am i overseeing something i should do differently ?

Me too @marvinosswald.
The migrations seem to be loaded from the package now. So I have removed all targeted migrations from our repository and have renamed the migrations in the migrations table with this PostgreSQL query:
UPDATE migrations
SET migration='2024_03_07_100000' || SUBSTRING (migration, '^2024_01_09_[0-9]{6}(_create.*$)')
WHERE migration LIKE '2024_01_09_%_create_%_table';

Change the 2024_01_09 date with your installation date of course!
@duncanmcclean Can you confirm we need to do this?

When are the migrations being re-published? When you run composer update?

https://github.com/statamic/eloquent-driver/pull/253/files#diff-abf1fa77a19f720052b75bc802df0260157b9064d8c3c12a80794f01e9acc065L216
Doesn't need to be republished for this problem.
this is the problematic line, this makes laravel "discover" the migrations without them being published afaik.

Doesn't need to be republished for this problem.
this is the problematic line, this makes laravel "discover" the migrations without them being published afaik.

That line is only related to loading the migrations within the test suite, it won't affect "real" applications.
I'm asking since I'm unable to replicate the issue myself, on a site using the Eloquent Driver. Updating to the latest version doesn't result in migrations being published so trying to figure out what could be causing this 🤔

Ah, I've figured it out...
In addition to letting you publish the migrations yourself into your database/migrations directory, the Eloquent Driver is also directly loading the migration files from it's database/migrations directory, which leads to errors when running php artisan migrate. I'll tag a release to fix this shortly.


Add EBS optimized setting for master node

Enabling EBS optimization on the master node helps with NFS performance.

This may take a while but in the mean time, all volumes are SSD (gp2 type: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSVolumeTypes.html), so the larger the volume, the better the performance.

I see, thanks. The problem is that the IO bandwidth can be affected by "noisy neighbors" since storage traffic and general network traffic share the same path in non-EBS-optimized instances.

Is bug fixing on hold because of the rewrite (#49)?

When I said a while, I was thinking it may not happen today/tomorrow but
I'm looking into it.
On Tue, Mar 8, 2016 at 11:37 AM, Ilya Sytchev<EMAIL_ADDRESS>wrote:

Is bug fixing on hold because of the rewrite (#49
https://github.com/galaxyproject/cloudlaunch/issues/49)?
—
Reply to this email directly or view it on GitHub
https://github.com/galaxyproject/cloudlaunch/issues/53#issuecomment-193853725
.

OK, thank you! I didn't mean to rush you, just wanted to figure out the state of the project.

I managed to squeeze a basic implementation in today - it's live now on launch.usegalaxy.org (with cloudman-test bucket) so feel free to give it a shot and let me know if you'd like to see something different.

Thank you. While the volume IOPS setting is also useful, I actually had the --ebs-optimized instance attribute in mind.

Is that just a flag to add when launching an instance? And if the instance is already EBS-optimized, nothing changes but if it's not, Amazon charges the extra fee?

I guess I should have kept reading... yes: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSOptimized.html#ebs-optimization-support

Yes to both questions. Also, it's worth noting that "EBS-optimized instances enable EC2 instances to fully use the IOPS provisioned on an EBS volume"

Should be all good to go now; live on launch.usegalaxy.org.
I guess the IOPS provisioned volumes came in as a bonus from the original comment... Let me know how performance changes with all this enabled.

Thanks, the IO throughput is a lot more consistent now when writing to /mnt/galaxy on master via NFS. Btw, it would be great to enable the --ebs-optimized setting on worker nodes too (for example, it is not enabled by default on c4.large, contrary to what the docs say).

I wasn't sure if that would be beneficial since all the volumes are mounted from the master but I added it in now (available from cloudman-test bucket for the time being).

Great, thanks, working as expected. It is mostly beneficial for speeding up the boot process on worker nodes but also, gp2 is the new default EBS volume type.


Validator Damping Function

This pr replaces static functions to modify the validator count with use a dynamic damping function which will modify the target validator count based on the staking participation.

[x] tests

Damping function:
Let x be the current number of validators, k be the minimum number of validators, and K be the maximum number of validators.

If the bottom 1% of validators ( ceil( x / 100 ) ) have an average staked value greater than 40% of the global average, we want to increase the validator count by 1%. I.e. the final validator count should be: x = min(K, x + ceil( x / 100 ).

If the bottom 2% of valiators ( 2 * ceil( x / 100 ) ) have an average staked value below 20% of the global average, we want to decrease the validator count by 1%. I.e. the final validator count should be: x = max(k, x - ceil( x / 100)).

Thus the following guarantees should be satisfied:

The number of active validators in any era can only change by at most 1%.
The number of valiators is always between the minimum and maximum number of desired validators. (k ≤ X ≤ K)
The number of validators that we select is based on the current state of the staking system.

Changes

Introduced new configuration value maximum_validator_count which is used by dumpling function. When configuration conditions are met (maximum_validator_count > minimum_validator_count) validator damping function is taken in to action at the end of each era.

Follow-up
One of the requirements says that change at the end of era is at most 1%. The existing staking pallet test suit mock is built in a small scale:

number of validators is limited (num_validators <= 8)
pallet balances for the genesis configuration are around 20

So in order to have more comprehensive tests mock should be rewritten to work with a bigger set of validators and balances.
The damping functions makes sense at a bigger scale, starting from 100 validators. This will give ability to more correctly represent testing conditions:

change at most 1%
number of validators is always between the minimum and maximum number of desired validators

We likely already support parachains whose collator numbers adjust, so long as they make the adjustment using on-chain logic and multiple epochs in advance.
We could explore work like https://arxiv.org/ftp/arxiv/papers/1807/1807.11218.pdf to discover adjustment schemes we'd endorsed by inclusion in cumulus, but again probably only using on-chain logic and multiple epochs in advance.
Ad hoc adjustment schemes should not be endorsed obviously.  And kusama/polkadot do not adjust validator numbers dynamically.  In particular, anytime we shrink the validator set by 1% then we could easily remove validators who do not currently lie in the bottom 2%.  And shrinking after NPoS could remove validators who could remain with higher backing.

@tarassh I just wanted to briefly ping you that we've been a little too busy in the last few days, and therefore I couldn't provide any feedback yet. Hopefully more to come soon!

The overall code looks correct to me, except one major flaw: You are assuming here that every single chain that uses staking will want to very particular update mechanism. Is this a good way to go? Clearly no.

Slightly better is to make this configurable so that you can opt out of it. So you'd have type DynamicUpdate: Get<bool>. But that's still not great, what if someone wants their own logic which is slightly different?
Ideal would be to abstract all of this away in the top level runtime fil. Staking would only receive an opaque update funciton type DynamicUpdate<(Vec<Balance> /* all exposures*/, u32 /* current count */), u32 /* new count */> and does trigger this per new era.

You might have different opinions on what A and B should be in our Convert<A, B>, I'll leave it up to you to decide.
If a chain needs no update, then we can use Identity.

https://crates.parity.io/sp_runtime/traits/trait.Convert.html
https://crates.parity.io/sp_runtime/traits/struct.Identity.html

Here's a sketch of a scheme that runs with Jeff's idea of just running the validator selection algorithm for a larger set. Let me know what you think. I add my own comments at the end.
Suppose k and K are a lower and an upper bound on the number of active validators allowed, we currently have x active validators, and we want to increase this number to at most X > x in the next era, where for instance X = min{ K, ceil( 1.01 * x ) } because we don't want big increases between consecutive eras.
In particular, we want to increase the committee size to the largest possible number x' <= X, such that the election algorithm still finds an "unskewed" stake support distribution for the validators, meaning that the bottom 1% of validators ( ceil( x'/100) ) has an average stake support no smaller than 40% of the global average for all validators. Notice that the value of x' is not known a priori and will be set automatically by the algorithm, and it could even end up being smaller than the current value x (we should also set up lower bounds for x').
Now recall that the election algorithm that we use (called Phragmms) starts with an empty validator committee, and then in each iteration adds a validator and computes a "balanced" stake support distribution for the current committee, which tries to make it as unskewed as possible. So, we can run x' iterations, and for each iteration we register a boolean variable that says whether the current committee is "unskewed" or not. Then, we keep the largest unskewed committee, and then move on with the algorithm, i.e. execute the reduce operation on this committee (the operation that reduces the number of active nominations while keeping the supports unchanged).
My comment: this approach is nice because it covers both increases and decreases at once, and and general tries out several sizes of committees "for free", picking the largest possible size, as opposed to the current approach that only tries one size. However, the fact that we don't know the size of the elected committee in advance could be an issue. For instance if we run the validator election algorithm off-chain, different people could submit solutions of different sizes, and we'd need to decide how to rank these solutions.

There need not exist any ranking rules in full generality, but we're discussing the situation where we want more validators, so..
We'd rank them using some rule for comparing how skewed the distribution become, no?  We think nominators remove skew over time so the question boils down to how quickly nominators react.
As an aside, I think that nominators remove skew over time indicates that nominators do not do their job correctly, meaning their only attempting to get rewards and not assessing risks, but we kinda knew this..

@kianenigma I see that discussion split in to directions: what kind of selection algorithm can be applied and how to implement  algorithm selection interfaces so that any 3rd party can apply its own selection schema.
So I would focus more on second and made already provided algorithm as option which can be replaced with 3rd party.

@kianenigma I see that discussion split in to directions: what kind of selection algorithm can be applied and how to implement algorithm selection interfaces so that any 3rd party can apply its own selection schema.
So I would focus more on second and made already provided algorithm as option which can be replaced with 3rd party.

Indeed, I'll try and move the conversation with @burdges and @AlfonsoCev off-band to keep this PR cleaner for our feedback loop. Please only focus on my comments earlier, the rest is beyond the scope of this PR.
If you have opinions though, we're happy to hear them.


Fix: before pool token transfer margincall status

Pool token transfer is only possible if from and to addresses haven't invoked a margin call. In the current code, the check on margin call status was opposite. This PR fixes the issue.

Looks good


Performance: Allow evaluating the production and destruction terms in one function

Right now, all algorithms compute first the production terms in one function call and the destruction terms (given the same states) in another function call directly afterward. However, it would be more efficient for some problems to have an API where the production and destruction terms are computed in a single function call.
For backward compatibility, we could wrap the two functions in one struct and call them one after the other. However, that would mean that we have to touch the out-of-place version a bit more (since we use some if statements in them). The in-place versions are fine since conservative and non-conservative systems are already separated.
What do you think about this, @SKopecz?

I'd suppose that at such an early stage of this package, we don't need to care too much about backward compatibility. And users of the old version can adapt their code to the new version with little effort.

Well, the other argument in favor of the wrapping could be that is may look more clear for simple ODE problems. And if we make breaking changes, we can indicate it in the version number and we're fine.

Okay, let's try wrapping.

This is not as straightforward as hoped: We get problems with constructors like
PDSProblem(P, D, u0, tspan, p = NullParameters())
PDSProblem(prod_dest, u0, tspan, p = NullParameters())

since it is not clear how to differentiate the first form with the default argument for p and the second form with the given p - both have the same number of arguments and I don't see any obvious type restrictions we could use for dispatch (since we want to keep everything quite generic). We could consider imposing P::Function, D::Function, but that makes it impossible to use custom callable structs that are no Functions.
Thus, it looks like this has to be a breaking change. I see the following options:

require a single function in all cases
allow a single function or two functions - but they have to be Functions

Which version do you prefer?


undefined is not an object (evaluating '_$$_REQUIRE(_dependencyMap[2], "react-native-bluetooth-escpos-printer").BluetoothManager')

I've just installed the package:
npm i react-native-bluetooth-escpos-printer

Added this in the file PrintRecipt.js where I'm going to show devices:
import React, { useEffect } from 'react';
import { ScrollView } from 'react-native';

import { BluetoothManager, BluetoothEscposPrinter, BluetoothTscPrinter } from 'react-native-bluetooth-escpos-printer';

function PrintRecipt(props) {

    useEffect(() => {
        BluetoothManager.isBluetoothEnabled().then((enabled) => {
            alert(enabled) // enabled ==> true /false
        }, (err) => {
            alert(err)
        });
    }, [])

    return (
        <ScrollView>

        </ScrollView>
    );
}

export default PrintRecipt;

As soon as I navigate to this screen I get:

undefined is not an object (evaluating '_$$_REQUIRE(_dependencyMap[2], "react-native-bluetooth-escpos-printer").BluetoothManager')

I have the following permissions in android:
   <uses-permission android:name="android.permission.BLUETOOTH" />
    <uses-permission android:name="android.permission.BLUETOOTH_ADMIN" />

and my android SDK versions are:
        buildToolsVersion = "30.0.2"
        minSdkVersion = 21
        compileSdkVersion = 30
        targetSdkVersion = 30

And other information:
    "react": "17.0.2",
    "react-native": "0.67.3",
    "react-native-bluetooth-escpos-printer": "0.0.5",

Any idea why it's not working?

same on my side too

same

same =/


"Press t to filter by a test name regex pattern", not parsing nested describes (v19)

describe('foo', () => {
  it('should be a foo', () => {})

  describe('bar', () => {
    it('should get drunk', () => {})
  })
})

It will find should be a foo but not should get drunk.

descibe => desrcibe and it will work

@thymikee please open the ticket again, I had a typo and corrected it.

Please provide more information about when it happens – your OS, Node version, your Jest configuration. It's best to provide a repo with actual bug so we can get into it.

"jest": {
    "testEnvironment": "node",
    "setupTestFrameworkScriptFile": "./lib/__setup__/tests.js",
    "collectCoverageFrom": [
      "lib/*.js"
    ],
    "coverageThreshold": {
      "global": {
        "branches": 100,
        "functions": 50,
        "lines": 33,
        "statements": 25
      }
    }
  }

cj at cj-mac in ~/Sync/npmjs/vues on master!
± node --version
v6.9.5

When you "Press t to filter by a test name regex pattern" it does not pick up any tests inside the nested describe

I'm making a repo right now.

@thymikee it's working in the clean repo.  So I think something else is causing it, I have a hefty test suite, so it could be a number of things.  It's 3am here, so I'll dig into it more tomorrow.  Thank you for the quick responses!  Feel free to close this until I find the root cause.

Maybe the problem is with the cache. Jest only know test names after first successful watch run and caches them. Have you tried run watch on all/just these files and try to match test names again?
I'm trying this on 2 different projects (including Jest itself) and it doesn't repro.

I'll close this, but we can keep the discussion going and reopen if you actually find a bug. Get some rest and let me know how it goes tomorrow 🙂

Thank you!  sounds good 🙂


Extend function BoxFileName to handle more common image names

The function derives the file name for the .box file from an image name.
For training from existing line images, it is useful to directly support
the image names which are commonly used.
While generated images for Tesseract training typically use the name
pattern NAME.tif, other ground truth sets use NAME.bin.png for binarized
or NAME.nrm.png for grayscale images.
BoxFileName is also now a local function as it is only used locally.
Signed-off-by: Stefan Weil<EMAIL_ADDRESS>

I updated and rebased the pull request and think that it is ready for merging now.

thanks


VScode lacks the dropdown trees like Juno

I find the dropdown things of whatever is returned from code execution to be super useful. I don't think there is such a thing in VScode. It doesn't seem very complicated. Is anyone working on filling that gap? What do you think? It is useful enough to me that I still am using Juno for a lot of stuff.

Slightly miss-placed issue ;-)

And VSCode has them.

Sorry for the misplaced issue! I was thinking of a Makie solution to it.
How do I get them working in VScode?


pvalue=0 needs to be evaluated as True

Description
When pvalue=0, drift was returned as None; As 0 is a legit value, drift needs to be 0, and not None.
General Checklist

[ ] Tests added for this feature/bug
if it was a bug, test must cover it.
[ ] Conform by the style guides, by using formatter
[ ] Documentation updated
[ ] (optional) Please add a label to your PR

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

1 of 1 (100.0%)  changed or added relevant line in 1 file are covered.
1 unchanged line in 1 file lost coverage.
Overall coverage remained the same at 82.595%

Files with Coverage Reduction
New Missed Lines
%

src/whylogs/viz/utils/profile_viz_calculations.py
1
86.73%

Totals

Change from base Build<PHONE_NUMBER>:
0%

Covered Lines:
4938

Relevant Lines:
5814

💛  - Coveralls


Make shift-scroll do horizontal scrolling on all platforms

Closes https://github.com/emilk/egui/issues/1135

Please test it if you have time @jbuckmccready

Nice! Works on windows 10 binary and Firefox wasm for me.


Test Python 3.6

Codecov Report

Merging #292 into master will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff          @@
##           master   #292   +/-   ##
=====================================
  Coverage     100%   100%
=====================================
  Files          12     12
  Lines        1764   1764
=====================================
  Hits         1764   1764

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5e46c9d...2bf8c1d. Read the comment docs.


CSV Export of MultiSelect fields is broken

Since I added the locale stuff.
For a temporary fix, see the new Select field.
Should fix properly in that you can pass a locale to the export command.

Temp fix done - now  won't crash.
Should fix properly in that you can pass a locale to the export command.


Clearly specify which floating-point instructions will raise exceptions on invalid rounding modes and/or unsupported EEW of operands

https://github.com/riscv/riscv-v-spec/commit/7c63e970fca3f05575ca0588189e682f02182faf covers invalid rounding mode exceptions

Invalid widths covered by https://github.com/riscv/riscv-v-spec/commit/50fab57354c3d6d136de5b844f843e59acf2846e#diff-4fa320b4e415eee7270590bb06607389.
Closing this issue.


no drops of xp in certain chucks whatsoever

i have the issue since im using playerex that certain chunks drop no xp whatsoever even if the setting for it is set to 100 for vanilla but even then its not droping xp in certain chucks even if i use xp bottles

Hi there,
Try the command /playerex resetChunk in the chunks where this is happening. If you're not already using the latest version of PlayerEx, please do so as it contains some fixes regarding this mechanic.
Thanks.

il try thanks
also im on the newest version
its gonna be fun tho as a mayority of my spawn chucks are affected 2000 blocks on each side radius
might mbe usefull to make a command that massresets all chucks in case somone has a lot of chunks affected


Bug in example code in step 7?

Is there a bug in the example code in step 7?
It shows this line...
    SrsReportRunController formLetterController = SalesConfirmControllerExt::construct();

But shouldn't it be:
    **SalesConfirmControllerExt** formLetterController = SalesConfirmControllerExt::construct();

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: a4112e96-d7e5-08a0-8754-9752d03d072d
Version Independent ID: 1a1aec89-ba1d-4c16-4f25-9ddd84ac5b56
Content: Create custom designs for business documents - Finance & Operations | Dynamics 365
Content Source: articles/fin-ops-core/dev-itpro/analytics/custom-designs-business-docs.md
Service: dynamics-365-cross-app
Product: ****
Technology: ****
GitHub Login: @RichdiMSFT
Microsoft Alias: richdi

@danielzook, thanks for the feedback. @RichdiMSFT, please review

Bug 573551 logged to track this issue.