Error when trying to save an MKV file with extra elements

Hi there,
I'm trying to use SpawnDev.EBML to alter the contents of an .mka (Matroska audio) file.
My test code aims at adding an attached picture :
using FileStream outputStream = new<EMAIL_ADDRESS>FileMode.OpenOrCreate, FileAccess.ReadWrite, FileShare.Write, Settings.FileBufferSize, FileOptions.RandomAccess);
inputStream.Seek(0, SeekOrigin.Begin);
var doc = new WebMDocumentReader(inputStream);

var segment = doc.Segment;
if (segment != null)
{
   // Add a new embedded cover
    var attachments = new MasterElement(MatroskaId.Attachments);
    var file = new MasterElement(MatroskaId.AttachedFile);
    file.Add(new UTF8StringElement(MatroskaId.FileName, "cover.jpg"));

    attachments.Add(file);
    segment.Add(attachments);
}

doc.CopyTo(outputStream);

The app crashes while saving, displaying the following log :
The lazily-initialized type does not have a public, parameterless constructor.
   at System.LazyHelper.CreateViaDefaultConstructor[T]()
   at System.Lazy`1.ExecutionAndPublication(LazyHelper executionAndPublication, Boolean useDefaultConstructor)
   at System.Lazy`1.CreateValue()
   at System.Lazy`1.get_Value()
   at SpawnDev.EBML.BaseElement.get_Stream() in C:\Users\zeugm\source\repos\Zeugma440\atldotnet\SpawnDev.EBML\BaseElement.cs:line 64
   at SpawnDev.EBML.MasterElement.CopyTo(Stream stream, Nullable`1 bufferSize) in C:\Users\zeugm\source\repos\Zeugma440\atldotnet\SpawnDev.EBML\MasterElement.cs:line 126
   at SpawnDev.EBML.MasterElement.CopyTo(Stream stream, Nullable`1 bufferSize) in C:\Users\zeugm\source\repos\Zeugma440\atldotnet\SpawnDev.EBML\MasterElement.cs:line 138
   at SpawnDev.EBML.MasterElement.CopyTo(Stream stream, Nullable`1 bufferSize) in C:\Users\zeugm\source\repos\Zeugma440\atldotnet\SpawnDev.EBML\MasterElement.cs:line 138
   at ATL.AudioData.IO.MKA.Write(Stream s, TagData tag, ProgressToken`1 writeProgress) in C:\Users\zeugm\source\repos\Zeugma440\atldotnet\ATL\obj\Debug\net6.0\Zomp.SyncMethodGenerator\Zomp.SyncMethodGenerator.SyncMethodSourceGenerator\ATL.AudioData.IO.MKA.WriteAsync.g.cs:line 25

and the generated file looks like this

Any idea what's going on / what should I be doing?
Thanks in advance~

Version 1.0.4 Nuget is up and should work in your example.

Sorry. The update seems to fix the error, but the output is missing the segment block in my test. Looking into it...

Let me know if 1.0.4 is working for you. My test using your code is working now.

Looks fine to me now. Thanks for the quick fix, man :)
PS : I might submit other PRs and/or issues in the near future as I'm actively working on MKA write support.


Fix Log In to & Sign Up error

secretOrPrivateKey must have a value

[ ] .env file added (for some reason it was previously deleted) & entered a JWT token
[ ] made sure .env was listed via .gitignore so the file isn't listed via the repo


Test the verification of Sprout anchors in state

Solution
This PR tests whether Sprout anchors are checked properly.
It follows the implementation for Sapling.
Review
@dconnolly can review.
Reviewer Checklist

[ ] Code implements Specs and Designs
[ ] Tests for Expected Behaviour
[ ] Tests for Errors

Some quick thoughts on what we may have to change to make this a successful and useful test of the changes in the destination branch:
Sprout pool and note commitment tree exists since genesis, so have to check what anchors those JoinSplits are referencing - vs that Sapling anchors test, those first 2-3 blocks in the Sapling anchor test are the first 2-3 blocks in the history of the Sapling pool, they are constructing the first 2-3 possible tree roots possible.
We may be parsing out JoinSplit<Groth16Proof> to exercise our code which only checks anchors for JoinSplits with Groth16 proofs, but the note commitment trees are the same/same type/same history across BCTV14/Groth16 and are independent of them (sha256Compress hash outputs).
Maybe instead, we should construct a test that takes the sprout commitments and anchors from the first few blocks from genesis (I think you found the first one @Marek around ~2000 on mainnet) and construct empty JoinSplit<Groth16Proof> except for the note commitments and anchors, so that it exercises our new code that will be running after checkpoints stop, but with the minimal amount of (artificial) state, like we do for other tests


support both unary calls and stream calls

support unary calls

[x] Generate go files with proto
[x] integration with Service - unary calls
[x] stream calls

done


OETHb: Strategy Mint Whitelist

Code Change Checklist
To be completed before internal review begins:

[ ]  The contract code is complete
[ ]  Executable deployment file
[ ]  Fork tests that test after the deployment file runs
[ ]  Unit tests *if needed
[ ]  The owner has done a full checklist review of the code + tests

Internal review:

[ ] Two approvals by internal reviewers

Deploy checklist
Two reviewers complete the following checklist:
- [ ] All deployed contracts are listed in the deploy PR's description
- [ ] Deployed contract's verified code (and all dependencies) match the code in master
- [ ] The transactions that interacted with the newly deployed contract match the deploy script.
- [ ] Governance proposal matches the deploy script
- [ ] Smoke tests pass after fork test execution of the governance proposal

Looks good, just curious if we need those extra function that can change the target account when minting/burning tokens


what's the meaning of AverageTrainReturn_all_train_tasks, AverageReturn_all_train_tasks, AverageReturn_all_test_tasks?

Hi, there, thank you for your excellent work. In the output log I found three types of return:  AverageTrainReturn_all_train_tasks, AverageReturn_all_train_tasks, AverageReturn_all_test_tasks.
What's the difference among them and which one is the return in your paper?

Hello, thanks for your interest in our work!
AverageTrainReturn_all_train_tasks - average return achieved by an agent in a sampling of training tasks using context sampled from the replay buffer (implemented here: https://github.com/katerakelly/oyster/blob/44e20fddf181d8ca3852bdf9b6927d6b8c6f48fc/rlkit/core/rl_algorithm.py#L415-L436)
AverageReturn_all_train_tasks - average return achieved by an agent in a sampling of training tasks using context sampled by the current policy
AverageReturn_all_test_tasks - average return achieved by an agent in a sampling of testing tasks using context sampled by the current policy
(these last two implemented via this function: https://github.com/katerakelly/oyster/blob/44e20fddf181d8ca3852bdf9b6927d6b8c6f48fc/rlkit/core/rl_algorithm.py#L379-L395)
The final metric is the one reported in our paper.


perf: no-op on svg avatars, add a key to image and sigil avatars

Fixes the crashing we were seeing on scroll in the ChatList in iOS 18 (may have been happening on other iOS versions, but iOS 18 was the only one we noticed) by adding a key to the UrbitSigil in SigilAvatar.
Fixes a bug that would cause high CPU usage while apparently idle, which was caused by rendering SVG icons for groups/contacts (we now just no-op on SVG icons, at some later date we could dig into this and figure out which kinds of SVGs could be causing this issue and maybe figure out how to sanitize them).

Lgtm! Only thing I'd say is I think it'd make sense to filter these out higher up the stack so we can fall back appropriately. Maybe In toClientMeta/toClientContact?

we talked out of band, the way this is written it'll just fallback to text avatars or sigil avatars as-is.


Include 'complete' message in streaming buffer

Fixes #122

Will take a look at this later today, with any luck. I need to look at it carefully, as we need to consider how the TryComplete is going to work... it was previously just "Complete if you can, but it's okay if it's already completed" but now we may need to make it obvious to the caller whether or not it couldn't complete because there wasn't buffer space.

OK, thanks.
This PR doesn't address that issue at the moment. It simply returns null if the queue is full OR completion has already occurred.
(Also just noticed I haven't updated the xmldoc, I'll fix that now)

Instead of returning Task from the [Try]Write and [Try]Complete methods, we could return a BufferedWriteResult, which encodes the various results:

Queued successfully - provides the Task
Queue full
Stream completed

This would be a change of API, so needs doing soon if we'd like it.
What do you think?

I think it's probably simpler to expose a Completed property - possibly named in a slightly more specific way (WriteCompleteCalled?). That only ever goes from false to true, which helps in terms of not worrying about the value being out of date (or only being out of date in one direection). We may still want to have a way of finding the current buffer count, more for logging / diagnostics than anything else. All of that can come in another PR.

SGTM :)


Proposal for alternate interface for segment load/store and tuple types.

We've had some implementation issues supporting the tuple types in clang and LLVM IR. This is due to lack of support for structures of scalable vectors in LLVM IR. AArch64 SVE was able to work around this limitation, but the existence of LMUL for RISCV has made it impossible to adopt their approach.
We are able to represent the load/store intrinsics in IR without issue, but the problems show up when we need to store the tuple to a local variable to implement the assignment operator for tuple types. Or when we need to load the tuple from the local variable to use it.
To avoid these implemention issues we at SiFive would like to propose a new interface that avoids the tuples and removes  the vget, vset, vcreate, and vcopy intrinsics. We're looking for feedback on this.
For segment stores, we propose to pass between 2 and 8 vectors as separate arguments to the intrinsic rather than a tuple. This combines vcreate and store into a single intrinsic. For example
void vsseg2e8_v_i8mf8 (int8_t *base, vint8mf8_t v0, vint8mf8_t v1, size_t vl);
void vsseg8e8_v_i8mf8_m (vbool64_t mask, int8_t *base, vint8mf8_t v0, vint8mf8_t v1, vint8mf8_t v2, vint8mf8_t v3, vint8mf8_t v4, vint8mf8_t v5, vint8mf8_t v6, vint8mf8_t v7, size_t vl);
void vssseg2e8_v_i8mf8 (int8_t *base, ptrdiff_t bstride, vint8mf8_t v0, vint8mf8_t v1, size_t vl);
void vssseg8e8_v_i8mf8_m (vbool64_t mask, int8_t *base, ptrdiff_t bstride, vint8mf8_t v0, vint8mf8_t v1, vint8mf8_t v2, vint8mf8_t v3, vint8mf8_t v4, vint8mf8_t v5, vint8mf8_t v6, vint8mf8_t v7, size_t vl);

Loads are more complicated because we can't return multiple values in C and we can't make a struct of scalable vectors in C either. We propose to make the intrinsics return void and pass the address of local variables to store the results in 2-8 pointer arguments. These pointers cannot be null. Maskedoff values will be passed as 2-8 additonal arguments. For example:
void vlseg2e8_v_i8mf8 (vint8mf8_t *v0, vint8mf8_t *v1, const int8_t *base, size_t vl);
void vlseg8e8_v_i8mf4_m (vint8mf4_t *v0, vint8mf4_t *v1, vint8mf4_t *v2, vint8mf4_t *v3, vint8mf4_t *v4, vint8mf4_t *v5, vint8mf4_t *v6, vint8mf4_t *v7, vbool32_t mask, vint8mf4_t maskedoff0, vint8mf4_t maskedoff1, vint8mf4_t maskedoff2, vint8mf4_t maskedoff3, vint8mf4_t maskedoff4, vint8mf4_t maskedoff5, vint8mf4_t maskedoff6, vint8mf4_t maskedoff7, const int8_t *base, size_t vl);

An additional restriction is that tuples can no longer be passed as arguments or return values. I don't think that was likely to be a common scenario.
We would need to add intrinsics independent of tuple types to handle the request from #83.
We believe we can implement this is in clang quite quickly on top of the existing intrinsic support and have it available in LLVM/clang 13 this fall.
We look forward to your feedback.

I understand the difficulties with structs of sizeless vectors, think this proposal makes sense, and have no concerns about usability.

Although I really don't like the new interface, but I can understand how difficulty for the compiler infrastructure change, so + 0.5 from me.

First, I'm worried about usability. A load to multiple pointers look really weird and un-user-friendly. It's also relying on the compiler being able to properly allocate registers, and in register-constrained situation that's not a given...
I've used the EPI implementation (from BSC's @rofirrim, see for instance https://github.com/rdolbeau/EPI-test-codes-vector/blob/56a364f9222e83e8a0f2673110ef6f68478ddaf2/simplefft16-rvv/test_fft16_zgemv.c#L160), it's using a tuple interface with no visible issue (I did not investigate how it would mesh with LMUL<>1, I admit).
Another issue is that loosing tuples mean loosing some useful capabilities; for instance, an interesting use case is here: https://github.com/rdolbeau/fftw3/blob/d856efb28255f99212d9553cf15cb36d7eba6a84/simd-support/simd-r5v-split.h#L73-L75, where complex-valued vector are implemented as a tuple. This simplify the code greatly compared to the 'interleaved' implementation (real and imaginary part in alternating lanes of a single vector, of course, at the cost of register pressure...) and can leverage Zvlsseg. Without either structures, tuples or arrays, the format is unlikely to be implementable in FFTW3 whose interface (dated, I know) assume it can return vector of complex values.
For me, removing the tuples is not just a question of a handful of intrinsics changing interface; it's really about being competitive programming-wise by having the ability do do some minimalist 'structuring' with the vectors (at the time, I had asked BSC for structures or fixed-size arrays of vectors; the tuples were the compromise they offered and it was satisfactory).

I've used the EPI implementation (from BSC's @rofirrim, see for instance https://github.com/rdolbeau/EPI-test-codes-vector/blob/56a364f9222e83e8a0f2673110ef6f68478ddaf2/simplefft16-rvv/test_fft16_zgemv.c#L160), it's using a tuple interface with no visible issue (I did not investigate how it would mesh with LMUL<>1, I admit).

I'd like to give some context here. While we (BSC) were able to provide a convenient interface in the context of EPI, we (SiFive and BSC) deemed the (BSC) implementation too unsafe/unsound/risky for LLVM upstream.
@Hsiangkai  and @topperc (SiFive) have been working hard looking for alternatives, including making proposals LLVM upstream, in order to provide an interface like the vget/vset one. My understanding is that those alternatives haven't been meet with enthusiasm upstream.
I understand we should not compromise an interface because of compiler limitations but at least this can be implemented in a way that is not too controversial and in a reasonable amount of time. From it we can gather some experience and then determine evaluate again if a better interface is needed (even if this time it means pushing for fundamental changes in LLVM IR itself).

@rofirrim Well it worked so well for me I didn't realize it was "unsafe/unsound/risky" :-)
Anyway I would ask a question: which intrinsics are using tuples now? Because if it's only Zvlsseg, then my suggestion would be to remove all Zvlsseg-related intrinsics (and the tuples) from the current draft/standard to not delay inclusion of the 'primary' V extension, document that a clean solution is still needed for Zvlsseg (if I'm honest, this looks more like a quick'n'dirty hack to have Zvlsseg than an actual solution...), and document some basic inline assembly examples to facilitate interoperability between the intrinsics and the 'missing' instructions in Zvlsseg - those examples will be needed anyway for people who prefer ASM (... I assume proper constraint for consecutive registers [groups] will exist and allow such inline assembly?)
That way, V intrinsics aren't delayed, and the inclusion of tuples (or structures, or fixed-width array, or ...) can be pushed back to when a clean solution is found/agreed upon by all interested parties.

Only Zvlsseg are using tuple types today.
I'm not sure there's a clean inline assembly interface here. There's no C variable type you can pass to a single constraint to represent all the segments. Constraints are independent so you can't have multiple constraints and say that one must be the register after another.
@rofirrim did the tuple types in the BSC implementation work with inline assembly?

@rofirrim Well it worked so well for me I didn't realize it was "unsafe/unsound/risky" :-)

True but your case was reasonably well understood.

@rofirrim did the tuple types in the BSC implementation work with inline assembly?

No, it didn't.

@topperc Worst case scenario, you hard-wire your variables to a given register pair but that's not pretty...
__epi_1xf64 tmp0 asm("v4");
__epi_1xf64 tmp1 asm("v5");
asm("vlseg2e64.v %0, (%2)" : "=v" (tmp0), "=v" (tmp1) : "m" (ptr));

That's a workaround rather than a (not-good) solution, so there's still hope for a good solution eventually. Not-good solution tends to stick around a lot longer than anticipated :-(

Only Zvlsseg are using tuple types today.
I'm not sure there's a clean inline assembly interface here. There's no C variable type you can pass to a single constraint to represent all the segments. Constraints are independent so you can't have multiple constraints and say that one must be the register after another.

Well there could be one, right? I don't think LLVM would have a problem here in the backend.  The register classes for tuples do exist. The issue is more on the LLVM IR side of things where we can't represent the tuple because struct falls short.
(Maybe in the future we need a tuple type in the IR as a first-class type equivalent to struct only useable via extractvalue/insertvalue. No geps or other offset trickery. :smiley:. This is a joke don't take it seriously )

Only Zvlsseg are using tuple types today.
I'm not sure there's a clean inline assembly interface here. There's no C variable type you can pass to a single constraint to represent all the segments. Constraints are independent so you can't have multiple constraints and say that one must be the register after another.

Well there could be one, right? I don't think LLVM would have a problem here in the backend. The register classes for tuples do exist. The issue is more on the LLVM IR side of things where we can't represent the tuple because struct falls short.

Yes the IR is a problem. Also inline assembly goes through SelectionDAG and uses MVT types to map the constraints to register classes and MVT can't represent tuples either.

I'm not sure the decision about the interface should be made based on what a specific compiler can or cannot do at a specific point in time. For instance, could e.g. GCC implement the tuples? Could it be done in LLVM with some hypothetical changes in the IR that would be acceptable policy-wise?
A clean but as-yet-unimplemented solution will push the compilers to improve whereas a quick'n'dirty workaround, for all its immediate usefulness, likely won't.
Registers of non-architecturally-specified-width are already in SVE and V, they're likely here to stay. Compilers (and maybe even languages) will have to figure out how to enable them for the users, and that includes the ability to have them in tuples at first, and presumably in structures/array/classes/... later as well.

I believe gcc was able to implement the tuples. @kito-cheng would know for usre.
We posted patches to llvm to implement tuples earlier this year. They were not met very favorably. We were pointed to conversations AArch64 had a few years ago where longtime llvm developers indicated they were against the idea of scalable vectors in structs which is what is required to support tuples in llvm IR. Those developers aren't as active these days and scalable vectors have been around for a few years and maybe aren't as scary now, but so far we haven't received much support to go against those earlier objections.
AArch64 was able to implement tuples in llvm by using vector types that are 2x, 3x, or 4x as large as their regular vectors rather than structs. This doesn't work for LLVM due to LMUL creating 7 different sizes of vectors so it becomes ambiguos whether a 4x vector type is 4 lmul=1 values or 1 lmul=4 value. I thought I might still be able to make something work with maybe bad register allocation, but I couldn't figure out how to make it work with tuples of fractional lmul.
We acknowledge that the solution proposed is a quick and dirty hack, but we wanted to get something people could use in the near term rather than waiting another year until we maybe had a good solution. I don't believe telling people they need to use inline assembly is a good workaround. Inline assembly can be intimidating and confusing. Especially since you would need to do manual register allocation for these instructions.

@topperc

whether a 4x vector type is 4 lmul=1 values or 1 lmul=4 value

I see that it could be a problem - if the storage is different. I haven't had time to follow in details, but is it different these days? If it's just LMUL consecutive registers vs. N consecutive registers, it may not matter if you can't tell the difference when LMUL == N as they are effectively the same anyway. Didn't look at the problem in details so I could be way off the mark here.

waiting another year until we maybe had a good solution

Is is really that urgent? V isn't even ratified, I don't think there's commercially available silicon yet, and the handful of IP implementations I'm aware of don't have Zvlsseg anyway... (and usually target pre-v0.10 as it takes a while to implement a core with V support). I know some people already write code for V (I have, as mentioned earlier), but I don't think it's the kind of people that are going to be worried by an ASM wrapper for that one specific extension-to-the-extension if there's a clean solution in the work.

To GCC, I believe both interface are implement-able, and tuple type are implemented last year, that because AArch64 support struct with scalable vector on GCC IR.

For segment stores, we propose to pass between 2 and 8 vectors as separate arguments to the intrinsic rather than a tuple. This combines vcreate and store into a single intrinsic. For example
void vsseg2e8_v_i8mf8 (int8_t *base, vint8mf8_t v0, vint8mf8_t v1, size_t vl);
void vsseg8e8_v_i8mf8_m (vbool64_t mask, int8_t *base, vint8mf8_t v0, vint8mf8_t v1, vint8mf8_t v2, vint8mf8_t v3, vint8mf8_t v4, vint8mf8_t v5, vint8mf8_t v6, vint8mf8_t v7, size_t vl);
void vssseg2e8_v_i8mf8 (int8_t *base, ptrdiff_t bstride, vint8mf8_t v0, vint8mf8_t v1, size_t vl);
void vssseg8e8_v_i8mf8_m (vbool64_t mask, int8_t *base, ptrdiff_t bstride, vint8mf8_t v0, vint8mf8_t v1, vint8mf8_t v2, vint8mf8_t v3, vint8mf8_t v4, vint8mf8_t v5, vint8mf8_t v6, vint8mf8_t v7, size_t vl);

How ugly the interfaces are，hope these are not present in the toolchain, include LLVM

For segment stores, we propose to pass between 2 and 8 vectors as separate arguments to the intrinsic rather than a tuple. This combines vcreate and store into a single intrinsic. For example
void vsseg2e8_v_i8mf8 (int8_t *base, vint8mf8_t v0, vint8mf8_t v1, size_t vl);
void vsseg8e8_v_i8mf8_m (vbool64_t mask, int8_t *base, vint8mf8_t v0, vint8mf8_t v1, vint8mf8_t v2, vint8mf8_t v3, vint8mf8_t v4, vint8mf8_t v5, vint8mf8_t v6, vint8mf8_t v7, size_t vl);
void vssseg2e8_v_i8mf8 (int8_t *base, ptrdiff_t bstride, vint8mf8_t v0, vint8mf8_t v1, size_t vl);
void vssseg8e8_v_i8mf8_m (vbool64_t mask, int8_t *base, ptrdiff_t bstride, vint8mf8_t v0, vint8mf8_t v1, vint8mf8_t v2, vint8mf8_t v3, vint8mf8_t v4, vint8mf8_t v5, vint8mf8_t v6, vint8mf8_t v7, size_t vl);

How ugly the interfaces are，hope these are not present in the toolchain, include LLVM

I am not sure if it is more beautiful to use 8 arguments vcreate then use vssseg to store the tuple. The reality is LLVM IR could not deal with scalable struct in the current status. I ever proposed a RFC[1] to expand the semantic of LLVM IR in the llvm-dev mailing list. No feedback for the proposal. Craig and I joined the ARM SVE sync-up meeting to get feedback from ARM compiler engineers. They involved the development for scalable types in the LLVM earlier than us. The answer we got is they also ever proposed similar idea about 2 years ago and the idea is rejected by the LLVM IR experts at that time. I did not object to the idea of tuple types. Actually, we have a workable downstream version for tuple types. I created several proof of concept patches[2] to demonstrate the idea in the RFC. We are working on an open standard. It is unacceptable we could not upstream the implementation for the open standard in my point of view. We need a feasible solution in the current stage. To remove tuple types is a solution. Do you have any other better suggestions to resolve the issue?
[1] [llvm-dev] [RFC] Permit load/store/alloca for struct containing all scalable vectors. https://lists.llvm.org/pipermail/llvm-dev/2021-March/149296.html
[2] Proof of concept patches:
https://reviews.llvm.org/D97264
https://reviews.llvm.org/D98161
https://reviews.llvm.org/D98169
https://reviews.llvm.org/D99482
https://reviews.llvm.org/D99590
https://reviews.llvm.org/D99592
https://reviews.llvm.org/D99593

I agree with this approach.

Hi everyone, I am the NSIMD maintainer and we are currently adding support for RISC-V extension "V" into NSIMD.
We propose several APIs (when possible depending on how the headers are written):

C89 (usually GCC headers are more C89 than Clang ones)
C99
C11
C++98
C++11
C++14
C++20

All the C++ APIs are just templated/operator overloaded wrappers aroung the C APIs. And all our tuple API is based upon structs  from the C APIs up to the C++ APIs:
template <NSIMD_CONCEPT_VALUE_TYPE T, int N, NSIMD_CONCEPT_SIMD_EXT SimdExt>
NSIMD_STRUCT packx2 {
  typedef typename simd_traits<T, SimdExt>::simd_vector simd_vector;
  typedef T value_type;
  typedef SimdExt simd_ext;
  static const int unroll = N;
  static const int soa_num_packs = 2;

  pack<T, N, SimdExt> v0;
  pack<T, N, SimdExt> v1;
};

The NSIMD_STRUCT macro is struct except when compiling for Arm SVE in which case it is __sizeless_struct. So it would break NSIMD's public API to change this paradigm and I must admit that I am really not happy about that.
Another reason that I need to box SIMD/vector types into structs is for operator overloading. Indeed I cannot rely on compilers extensions to provide this kind of features because:

Some compilers do not provide this kind of overloading
Sometimes it is just impossible (Intel __m128i types can represents any SIMD vector of signed/unsigned integers with different sizes)
It is non standard C/C++ so I avoid it

Anyway I need to box SIMD/vector types into struct for operator overloading in C++ and function overloading in C++ and C11. Summing up my long post I just need a __sizeless_stuct keyword and I will adapt the internals of NSIMD to the API that you decide for tuples (structs or opaque types with getters and setters).
My opinion on this subject concerning upstream LLVM is the same as @rdolbeau. I know it is easy to say but I think you should stick to the design of the API and make as good as possible without taking into account what LLVM IR can or canont do. Arm is pushing really hard SVE so I think that at one point they will be forced to support sizeless types.

The interface is now implemented as this issue had proposed. Document is updated in #98. Closing the issue.


A crash found on tensorflow_jni.so when create interpreter using byteBufferMode

Please go to Stack Overflow for help and support:
https://stackoverflow.com/questions/tagged/tensorflow
If you open a GitHub issue, here is our policy:

It must be a bug or a feature request.
The form below must be filled out.
It shouldn't be a TensorBoard issue. Those go here.

Here's why we have that policy: TensorFlow developers respond to issues. We want to focus on work that benefits the whole community, e.g., fixing bugs and adding features. Support only helps individuals. GitHub also notifies thousands of people when issues are filed. We want them to see you communicating an interesting problem, rather than being redirected to Stack Overflow.

System information
Have I written custom code (as opposed to using a stock example script provided in TensorFlow):
•OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 14.04.4 LTS
•TensorFlow installed from (source or binary):use the pip install
•TensorFlow version (use command below):1.4.0
•Python version: Python 2.7.6
•Bazel version (if compiling from source):0.9.0
•GCC/Compiler version (if compiling from source):(Ubuntu 4.8.4-2ubuntu1~14.04.3) 4.8.4
•CUDA/cuDNN version:NA
•GPU model and memory:NA
•Exact command to reproduce:NAYou can collect some of this information using our environment capture script:
https://github.com/tensorflow/tensorflow/tree/master/tools/tf_env_collect.sh
You can obtain the TensorFlow version with
python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)"
Describe the problem
Describe the problem clearly here. Be sure to convey here why it's a bug in TensorFlow or a feature request.
I put my training model to the tflitecamerademo projects, but I encounter a crash issue.
The crash stack is
01-23 10:46:45.292 21514 21536 E AndroidRuntime: Process: android.example.com.tflitecamerademo, PID: 21514
01-23 10:46:45.292 21514 21536 E AndroidRuntime: java.lang.IllegalArgumentException: Invalid handle to Interpreter.
01-23 10:46:45.292 21514 21536 E AndroidRuntime: 	at org.tensorflow.lite.NativeInterpreterWrapper.getInputDims(Native Method)
01-23 10:46:45.292 21514 21536 E AndroidRuntime: 	at org.tensorflow.lite.NativeInterpreterWrapper.run(NativeInterpreterWrapper.java:82)
01-23 10:46:45.292 21514 21536 E AndroidRuntime: 	at org.tensorflow.lite.Interpreter.runForMultipleInputsOutputs(Interpreter.java:112)
01-23 10:46:45.292 21514 21536 E AndroidRuntime: 	at org.tensorflow.lite.Interpreter.run(Interpreter.java:93)
01-23 10:46:45.292 21514 21536 E AndroidRuntime: 	at com.example.android.tflitecamerademo.ImageClassifier.classifyFrame(ImageClassifier.java:117)
01-23 10:46:45.292 21514 21536 E AndroidRuntime: 	at com.example.android.tflitecamerademo.Camera2BasicFragment.classifyFrame(Camera2BasicFragment.java:663)
01-23 10:46:45.292 21514 21536 E AndroidRuntime: 	at com.example.android.tflitecamerademo.Camera2BasicFragment.access$900(Camera2BasicFragment.java:69)
01-23 10:46:45.292 21514 21536 E AndroidRuntime: 	at com.example.android.tflitecamerademo.Camera2BasicFragment$5.run(Camera2BasicFragment.java:558)
01-23 10:46:45.292 21514 21536 E AndroidRuntime: 	at android.os.Handler.handleCallback(Handler.java:789)
01-23 10:46:45.292 21514 21536 E AndroidRuntime: 	at android.os.Handler.dispatchMessage(Handler.java:98)
01-23 10:46:45.292 21514 21536 E AndroidRuntime: 	at android.os.Looper.loop(Looper.java:180)
01-23 10:46:45.292 21514 21536 E AndroidRuntime: 	at android.os.HandlerThread.run(HandlerThread.java:65)
I see the similar issue in https://groups.google.com/a/tensorflow.org/forum/?hl=es-VE#!topic/discuss/jJSH5RQO4Mo, but no one answer.
P.S. My training tflite file size is about 248293KB
Could you kindly to help?
Thanks
Source code / logs
Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached. Try to provide a reproducible test case that is the bare minimum necessary to generate the problem.

Nagging Awaiting Response: It has been 14 days with no activityand the awaiting response label was assigned. Is this still an issue?

I have the same problem. When I use tf.gather(), I encounter a crash issue:
E/AndroidRuntime: FATAL EXCEPTION: CameraBackground
Process: android.example.com.tflitecamerademo, PID: 11148
java.lang.IllegalArgumentException: Invalid handle to Interpreter.
at org.tensorflow.lite.NativeInterpreterWrapper.getInputDims(Native Method)
at org.tensorflow.lite.NativeInterpreterWrapper.run(NativeInterpreterWrapper.java:82)
at org.tensorflow.lite.Interpreter.runForMultipleInputsOutputs(Interpreter.java:112)
at org.tensorflow.lite.Interpreter.run(Interpreter.java:93)
at com.example.android.tflitecamerademo.ImageClassifierFloatInception.runInference(ImageClassifierFloatInception.java:104)
at com.example.android.tflitecamerademo.ImageClassifier.classifyFrame(ImageClassifier.java:110)
at com.example.android.tflitecamerademo.Camera2BasicFragment.classifyFrame(Camera2BasicFragment.java:664)
at com.example.android.tflitecamerademo.Camera2BasicFragment.-wrap0(Camera2BasicFragment.java)
at com.example.android.tflitecamerademo.Camera2BasicFragment$4.run(Camera2BasicFragment.java:560)
at android.os.Handler.handleCallback(Handler.java:743)
at android.os.Handler.dispatchMessage(Handler.java:95)
at android.os.Looper.loop(Looper.java:150)
at android.os.HandlerThread.run(HandlerThread.java:61)

Again, 'Invalid handle to Interpreter' means you failed to create a valid model interpreter with the given model file. Please run your model with the most recently updated tf lite in Github, then post the exceptions here. The updated tf lite should throw exceptions when it fails to create a valid model interpreter with more detailed messages.

I'm trying this with what's in master (trying to get a yolo model working and I only get the same  Invalid handle to Interpreter. when calling tflite.run().  Are you sure the new logging/exceptions are in place and output to logcat?

Yes, it is in the github head: please check line 382 at  https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/lite/java/src/main/native/nativeinterpreterwrapper_jni.cc.
An IllegalArgumentException will be thrown in Java with the messages if it fails to create a valid Interpreter.

It has been 14 days with no activity and the awaiting response label was assigned. Is this still an issue?

It has been 14 days with no activity and the awaiting response label was assigned. Is this still an issue?

It has been 29 days with no activity and the awaiting response label was assigned. Is this still an issue?

It has been 68 days with no activity and the awaiting response label was assigned. Is this still an issue?

We are closing this issue for now due to lack of activity. Please comment if this is still an issue for you. Thanks!


update YdkIds

What do I do now? Is there supposed to be a step before that?Or after that

As mentioned in the documentation that command updates YdkIds.txt. By running that command you updated YdkIds.txt. There aren't any other steps. Though you are missing the file for mapping alternative card art https://github.com/pixeltris/YgoMaster/blob/master/Docs/AltCardsYdk.json

I'm sorry I didn't understand where to put this file or what to do with it


Resolve or remove TODO 3.0 in crypto/rsa/rsa_ameth.c

There is the following TODO 3.0:
/*
 * For the moment, we trust the call path, where keys going through
 * rsa_pkey_export_to() match a KEYMGMT for the "RSA" keytype, while
 * keys going through rsa_pss_pkey_export_to() match a KEYMGMT for the
 * "RSA-PSS" keytype.
 * TODO(3.0) Investigate whether we should simply continue to trust the
 * call path, or if we should strengthen this function by checking that
 * |rsa_type| matches the RSA key subtype.  The latter requires ensuring
 * that the type flag for the RSA key is properly set by other functions
 * in this file.
 */
static int rsa_int_export_to(const EVP_PKEY *from, int rsa_type,
                             void *to_keydata, EVP_KEYMGMT *to_keymgmt,
                             OSSL_LIB_CTX *libctx, const char *propq)
{
    RSA *rsa = from->pkey.rsa;

It seems like the check should be added so we do not inadvertently transform RSA-PSS key into an RSA key.

I tried adding some code and a test for this usecase..
the ->export_to is only called from evp_pkey_export_to_provider() and that does the following before the export_to..
        /* Make sure that the keymgmt key type matches the legacy NID */
        if (!ossl_assert(EVP_KEYMGMT_is_a(tmp_keymgmt, OBJ_nid2sn(pk->type))))
            goto end;

So I dont think it makes any difference if we test the flags of not inside the export_to method.
Should it be doing this assert here? (I am not sure it should)..
All I did was put a little more code in keymgmt_internal_test in test_pass_rsa i.e
TEST_ptr(km3 = EVP_KEYMGMT_fetch(fixture->ctx1, "RSA-PSS", NULL))

...
provkey2 = evp_pkey_export_to_provider(pk, NULL, &km3, NULL)


ARC-36: fix failing test

Currently, when you follow CONTRIBUTING.md in Github for Jira and get to the npm test step, one of the tests fail.
Failing test: updates all Subscriptions for a given jira clientKey when a site is renamed in test/unit/models/index.test.js
Fails with:
Expected: "https://renamed-user.atlassian.net"
Received: "https://existing-instance.atlassian.net"

This was happening because we were calling Jest's beforeEach function which executes before every test in the scope. This meant that the renamed url of https://renamed-user.atlassian.net was being reset:

By replacing beforeEach with beforeAll, we only execute this logic once, before any test is run. Now all tests pass:

Above comment was by me. Didn't realize I was signed into my secondary account.

Just updated: pulled the truncate calls out into beforeEach and left the rest in beforeAll. All tests are passing still 🎉

Accidentally opened in wrong account. Closing and while open a new PR.

New PR https://github.com/integrations/jira/pull/431


Project Conversion Crash - Exported String Enum

Godot version
4.0 rc 6
System information
Ubuntu 22.10
Issue description
After converting a project to rc6 from 3.x I ended up with a crash on load. There seems to be a few things wrong with the project but through some testing I was able to determine that exporting a String enum causes a crash on conversion, see below code and project for reproduction.
If I can figure out the other reasons (there is at least one other) for crashing I will upload those as well.
Steps to reproduce
Add an exported string enum such as: export(String, "string one", "string two", "string three") var test_export to a script, then convert the project. It will crash on load.
Or download the attached MRP and convert it to 4.0 RC6 to get the crash.
Minimal reproduction project
EnumExportCrash.zip

Can't reproduce this with the MRP. I only noticed that the window closed without a success message, but this is not a bug. Please describe your steps in more detail.
The line is commented out automatically, I didn't edit anything:

(Note: The converter could be improved as the string enum export is supported - #72305.)

I think this isn't actually wasn't what was causing my crash then. I was just seeing the crash so much on my main project when it just closed after conversion I assumed it was a crash instead of trying to reload it. Thanks for checking though. I'll get some coffee and see if I can't figure out what is actually crashing the main project...

@Calinou Whatever is causing the crash in my project conversion is not the bug report above, and is something I have yet to figure out (I've been too busy to trace it further). The most I know so far, is that it has to do with tilemaps (or whatever scripts I have in that scene), because if I remove it from my main scene and then convert, the project no longer crashes, until I try to open that scene.
This can be closed, and when I figure out the specifics of the crash on the project conversion I will open a new report.
Thanks!


flame_riverpod forceBuild can call setState() during build

What happened?
I have a FlameGame using riverpod that is moved around the widget tree quite a bit, and reparented. Typically it's fine, but about 5% of the time an error is thrown: setState() or markNeedsBuild() called during build.
Full error log below, but the important part is #3, the forceBuild method, which can be called during an existing widget build, but has no protections from calling setState during a build.
setState() or markNeedsBuild() called during build.
This RiverpodAwareGameWidget<FooGame> widget cannot be marked as needing to build because the framework is already in the process of building widgets. A widget can be marked as needing to be built during the build phase only if one of its ancestors is currently building. This exception is allowed because the framework builds parent widgets before children, which means a dirty descendant will always be built. Otherwise, the framework might not visit this widget during this build phase.
The widget on which setState() or markNeedsBuild() was called was:
  RiverpodAwareGameWidget<FooGame>-[LabeledGlobalKey<RiverpodAwareGameWidgetState<FooGame>>#d5fcf]
The widget which was currently being built when the offending call was made was:
  LayoutBuilder
      #0      Element.markNeedsBuild.<anonymous closure> (package:flutter/src/widgets/framework.dart:5177:9)
      #1      Element.markNeedsBuild (package:flutter/src/widgets/framework.dart:5189:6)
      #2      State.setState (package:flutter/src/widgets/framework.dart:1223:15)
      #3      RiverpodAwareGameWidgetState.forceBuild (package:flame_riverpod/src/widget.dart:64:5)
      #4      RiverpodComponentMixin.rebuildGameWidget (package:flame_riverpod/src/consumer.dart:124:42)
      #5      RiverpodComponentMixin.onRemove (package:flame_riverpod/src/consumer.dart:112:7)
      #6      Component._remove.<anonymous closure> (package:flame/src/components/core/component.dart:984:13)
      #7      Iterable.every (dart:core/iterable.dart:427:16)
      #8      Component.propagateToChildren (package:flame/src/components/core/component.dart:387:10)
      #9      Component._remove (package:flame/src/components/core/component.dart:981:5)
      #10     Component.handleLifecycleEventRemove (package:flame/src/components/core/component.dart:840:7)
      #11     ComponentTreeRoot.processLifecycleEvents (package:flame/src/components/core/component_tree_root.dart:125:19)
      #12     FlameGame.updateTree (package:flame/src/game/flame_game.dart:158:5)
      #13     FlameGame.update (package:flame/src/game/flame_game.dart:152:7)
      #14     GameWidgetState.build.<anonymous closure>.<anonymous closure>.<anonymous closure> (package:flame/src/game/game_widget/game_widget.dart:387:37)
      #15     GameWidgetState._protectedBuild (package:flame/src/game/game_widget/game_widget.dart:228:21)
      #16     GameWidgetState.build.<anonymous closure>.<anonymous closure> (package:flame/src/game/game_widget/game_widget.dart:376:28)
      #17     _LayoutBuilderElement._rebuildWithConstraints.updateChildCallback (package:flutter/src/widgets/layout_builder.dart:191:77)
      #18     BuildOwner.buildScope (package:flutter/src/widgets/framework.dart:3038:19)
      #19     _LayoutBuilderElement._rebuildWithConstraints (package:flutter/src/widgets/layout_builder.dart:231:12)
      #20     RenderObject.invokeLayoutCallback.<anonymous closure> (package:flutter/src/rendering/object.dart:2719:59)
      #21     PipelineOwner._enableMutationsToDirtySubtrees (package:flutter/src/rendering/object.dart:1098:15)
      #22     RenderObject.invokeLayoutCallback (package:flutter/src/rendering/object.dart:2719:14)
      #23     RenderConstrainedLayoutBuilder.rebuildIfNecessary (package:flutter/src/widgets/layout_builder.dart:278:5)
      #24     _RenderLayoutBuilder.performLayout (package:flutter/src/widgets/layout_builder.dart:369:5)
      #25     RenderObject.layout (package:flutter/src/rendering/object.dart:2608:7)
      #26     RenderProxyBoxMixin.performLayout (package:flutter/src/rendering/proxy_box.dart:111:21)
      #27     RenderObject.layout (package:flutter/src/rendering/object.dart:2608:7)
      #28     RenderProxyBoxMixin.performLayout (package:flutter/src/rendering/proxy_box.dart:111:21)
      #29     RenderObject.layout (package:flutter/src/rendering/object.dart:2608:7)
      #30     RenderProxyBoxMixin.performLayout (package:flutter/src/rendering/proxy_box.dart:111:21)
      #31     RenderObject.layout (package:flutter/src/rendering/object.dart:2608:7)
      #32     RenderProxyBoxMixin.performLayout (package:flutter/src/rendering/proxy_box.dart:111:21)
      #33     RenderObject.layout (package:flutter/src/rendering/object.dart:2608:7)
      #34     RenderAspectRatio.performLayout (package:flutter/src/rendering/proxy_box.dart:580:12)
      #35     RenderObject.layout (package:flutter/src/rendering/object.dart:2608:7)
      #36     RenderPadding.performLayout (package:flutter/src/rendering/shifted_box.dart:234:12)
      #37     RenderObject.layout (package:flutter/src/rendering/object.dart:2608:7)
      #38     RenderProxyBoxMixin.performLayout (package:flutter/src/rendering/proxy_box.dart:111:21)
      #39     RenderObject.layout (package:flutter/src/rendering/object.dart:2608:7)
      #40     RenderPositionedBox.performLayout (package:flutter/src/rendering/shifted_box.dart:451:14)
      #41     RenderObject.layout (package:flutter/src/rendering/object.dart:2608:7)
      #42     RenderProxyBoxMixin.performLayout (package:flutter/src/rendering/proxy_box.dart:111:21)
      #43     RenderObject.layout (package:flutter/src/rendering/object.dart:2608:7)
      #44     RenderConstrainedBox.performLayout (package:flutter/src/rendering/proxy_box.dart:291:14)
      #45     RenderObject.layout (package:flutter/src/rendering/object.dart:2608:7)
      #46     MultiChildLayoutDelegate.layoutChild (package:flutter/src/rendering/custom_layout.dart:173:12)
      #47     _ScaffoldLayout.performLayout (package:flutter/src/material/scaffold.dart:1092:7)
      #48     MultiChildLayoutDelegate._callPerformLayout (package:flutter/src/rendering/custom_layout.dart:237:7)
      #49     RenderCustomMultiChildLayoutBox.performLayout (package:flutter/src/rendering/custom_layout.dart:404:14)
      #50     RenderObject._layoutWithoutResize (package:flutter/src/rendering/object.dart:2446:7)
      #51     PipelineOwner.flushLayout (package:flutter/src/rendering/object.dart:1052:18)
      #52     PipelineOwner.flushLayout (package:flutter/src/rendering/object.dart:1065:15)
      #53     RendererBinding.drawFrame (package:flutter/src/rendering/binding.dart:602:23)
      #54     WidgetsBinding.drawFrame (package:flutter/src/widgets/binding.dart:1164:13)
      #55     RendererBinding._handlePersistentFrameCallback (package:flutter/src/rendering/binding.dart:468:5)
      #56     SchedulerBinding._invokeFrameCallback (package:flutter/src/scheduler/binding.dart:1397:15)
      #57     SchedulerBinding.handleDrawFrame (package:flutter/src/scheduler/binding.dart:1318:9)
      #58     SchedulerBinding._handleDrawFrame (package:flutter/src/scheduler/binding.dart:1176:5)
      #59     _invoke (dart:ui/hooks.dart:312:13)
      #60     PlatformDispatcher._drawFrame (dart:ui/platform_dispatcher.dart:419:5)
      #61     _drawFrame (dart:ui/hooks.dart:283:31)

What do you expect?
setState should not be called during build, so the build phase will have to be checked so that setState is either called, or queued.
How can we reproduce this?
Hmm. I was doing it with navigation, but you could try placing a Game in a layoutBuilder that changes the widget tree if the window size is even or odd. That'll trigger dozens of calls to rebuild as you resize a window.
What steps should take to fix this?
No response
Do have an example of where the bug occurs?
No response
Relevant log output
No response
Execute in a terminal and put output into the code block below
No response
Affected platforms
All
Other information
No response
Are you interested in working on a PR for this?

[ ] I want to work on this

@markvideon any clue how this can be avoided?

addPostFrameCallback is a blunt instrument but it works.

@markvideon any clue how this can be avoided?

addPostFrameCallback is a blunt instrument but it works.

The implementation of forceBuild currently looks like this:
  void forceBuild() {
    if (_isForceBuilding) {
      _hasQueuedBuild = true;
      return;
    }
    _isForceBuilding = true;
    setState(() {});
  }

and it works in conjunction with this persistent frame callback defined in initState:
    WidgetsBinding.instance.addPersistentFrameCallback((_) {
      _isForceBuilding = false;

      if (_hasQueuedBuild) {
        _hasQueuedBuild = false;
        forceBuild();
      }
    });

I haven't attempted to replicate this but I have some ideas.

We probably should check the game widget's context is mounted for scenarios where it could be re-parented
We could potentially replace the use of persistent frame callback with individual post frame callbacks when force rebuilds are scheduled
We could add checks for SchedulerPhase to determine whether it is safe to call setState within forceBuild, perhaps returning early if we are not in the 'idle' phase. Based on the description of SchedulerPhase.persistentFramecallbacks, we might be in a danger zone using persistent frame callbacks as we are now!

@TonyDowney I made an attempt to replicate this but so far have been unable to do so. Would it be possible for you to provide a code sample?


Move yourself down the line

Sometimes you just solved a bug that you were working on and then: Yikes! You're next in line and you have nothing to talk about! Wouldn't it be nice to move yourself down the line a few spots so you have time to find a bug to talk about?

There's a pull request that implements this and will hopefully be merged soon!


New Trace log level in logrus

Hello, I'm just opening this issue to let you know, the next release of logrus (v1.2.0) will add a new trace level named Trace below Debug.
Here is the PR for reference sirupsen/logrus#844
You may want to take into account this new level.

Just checking out the code, it looks like no level mapping is done when sending events, but the Levels() function is explicit about handling a subset of properties. Since Seq will happily accept both Verbose and Panic, it seems like the best response to this is to make logruseq transparent from a level perspective and just send through to Seq whatever logrus gives it.


Add missing arg to regexp.replace example

I think this was a typo. Let me know (and feel free to close) if I'm misunderstanding something!

@reid47 Please sign the Contributor License Agreement!
Click here to manually synchronize the status of this Pull Request.
See the FAQ for frequently asked questions.

@reid47 Thank you for signing the Contributor License Agreement!

🌝

yup, we just merged regexp.replace. will be available in the next release. thanks for catching typo.


Fix remove -> remote

Fix issue reported from twitter - https://twitter.com/attila_ibs/status/1766376180662329773

Why are these changes needed?

Related issue number

Checks

[ ] I've included any doc changes needed for https://microsoft.github.io/autogen/. See https://microsoft.github.io/autogen/docs/Contribute#documentation to build and test documentation locally.
[ ] I've added tests (if relevant) corresponding to the changes introduced in this PR.
[ ] I've made sure all auto checks have passed.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 34.91%. Comparing base (0e1a4b9) to head (ce30bc5).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1930      +/-   ##
==========================================
- Coverage   37.09%   34.91%   -2.18%
==========================================
  Files          64       64
  Lines        6836     6836
  Branches     1502     1502
==========================================
- Hits         2536     2387     -149
- Misses       4083     4244     +161
+ Partials      217      205      -12

Flag
Coverage Δ

unittests
34.91% <ø> (-2.18%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


BUG: unresolved import std::hash::DefaultHasher after update to 0.5.0

Short description
Updated from oasysdb 0.4.4 to 0.5.0. Building my program now gives this error:
--> ~/.cargo/registry/src/index.crates.io-6f17d22bba15001f/oasysdb-0.5.0/src/db/mod.rs:9:17
  |
9 | use std::hash::{DefaultHasher, Hash, Hasher};
  |                 ^^^^^^^^^^^^^ no `DefaultHasher` in `hash`
  |
  = help: consider importing this struct instead:
          std::collections::hash_map::DefaultHasher

For more information about this error, try `rustc --explain E0432`.
error: could not compile `oasysdb` (lib) due to previous error

Compiling on a M1 MacBookPro.

Hey 👋
Thank you for using OasysDB and sorry for the inconvenience.
Could you share the Rust version you’re running for your project?
rustc --version

According to the official documentation, the std::hash::DefaultHasher should be available from version 1.7.0.

Sure, the version is:
rustc --version
rustc 1.75.0 (82e1608df 2023-12-21)

Thank you! That's pretty new and should work with OasysDB.
Well, I can implement the fix and make a release tomorrow.
Once again, sorry for the inconvenience.

No problem!
Thanks for the quick fix!

No problem!
Thanks for the quick fix!

Hey, I just make a release of 0.5.1 to introduce the fix for you.
Let me know if it works 😁

I will close this issue as this should be fixed with the v0.5.1 release.
If there is something wrong with this fix, please let me know 😁

Works now, thanks!


Definitions: nested modules, single file, refactors

implemented nested modules with the module foo { ... } syntax
exposed builtin function and operators definitions as separate functions
added some basic configuration for generation and made some functions internally reconfigurable; the user now can choose whether we should include module ... at the beginning of the definitions, but we always need those when writing to files.
with nested modules we can actually put everything under module static;, so I exposed this feature as well.
refactored so that write_to_dir actually uses iter_files, I seem to have forgotten that in the previous PR...

I did not really bother with indentation and formatting, I made sure that everything is readable, but beyond that we'll have to run the generated content through a formatter eventually.
Also I'd probably not include this code in releases or at least with a note that it's not semver-compatible. I imagine we might have a quite a few breaking changes as it evolves alongside the LSP.

Also I'd probably not include this code in releases or at least with a note that it's not semver-compatible. I imagine we might have a quite a few breaking changes as it evolves alongside the LSP.

We can probably put this under internals for the time being... anything internals is supposed to be volatile.

https://github.com/rhaiscript/rhai/pull/597  gates definitions under the internals feature.


[FLINK-29375][rpc] Move getSelfGateway() into RpcService

Give RpcServices control over how self gateways are implemented.
(Note that we don't really have a definition of what a self-gateway even is. Right now it's a sort of local gateway that skips serialization; but generally speaking that isn't required and more-or-less a performance optimization).
This is particularly interesting because how we implement it for Akka is rather strange, with the AkkaInvocationHandler impklementing both the RpcGateway and RpcServer interface, despite never actually acting as an rpc server, making it difficult to wrap your head around what this handler actually does.
I don't think we should replicate that for other rpc implementations.

@flinkbot run azure

I came up with another diagram (see gist) that visualizes the relationships between the different classes of the RPC system to get into that topic again :-D

I'm not too sold on that particular diagram and tried to create a revision, but dear god these online UML renderers are horrible.

I'm not too sold on that particular diagram and tried to create a revision, but dear god these online UML renderers are horrible.

:-D Yeah, the class diagram in general might not be the best solution. I use Intellij's PlantUML to edit diagram code.


Please invite me to the community

Please invite me to the community

Hello there, welcome to openAOD, I have sent an invite to join the org. Happy to have you here : )


fix test setup

What does this change?
Change CDN, and change aws-sdk version(for test).
No dependency version error has been fixed in version 3.39.0 .
References

https://github.com/raiden-rs/raiden-dynamo/issues/160
https://github.com/aws/aws-sdk-js-v3/issues/2810

Looks great


Documentation for base-<IP_ADDRESS> doesn't mention Show Double or Show Float instances

The current release candidate documentation for base-<IP_ADDRESS> on Hackage does not mention the Show instance for Float in Double in any of the places that I can think of. The code for the Show instances themselves still appear to be fine (in GHC.Float).

That and many other instances are now not showing up but it's not due to a change on Haddock end at the very least. Maybe @hvr can explain.

This issue is popping up more and more frequently. Some other victims of this issue include:

aeson (example)
MonadRandom (example)
text (example)
text-show (example)

The discussion in https://github.com/ku-fpg/hermit/issues/173 possibly reveals a workaround?
cc @andrewthad

Based on https://github.com/haskell/haddock/issues/297#issuecomment-239144362, #469 might be a dup. At least in the case of Text, in #469 and in ku-fpg/hermit#173, the trigger is that the datatype and the instances are not defined in the same module (even if in the same package), and the problem started with GHC 7.10 apparently.
Rather than the workaround in https://github.com/ku-fpg/hermit/issues/173#issuecomment-185562501 (newtype wrappers to avoid orphans), which I guess is what @RyanGlScott meant to link to, most packages could just move instance definitions together with datatype definitions — or at least, text could.

@Blaisorblade: By "workaround", I mean this comment in particular, which reveals how GHC solves this problem internally. I don't know if the same idea can be used in Haddock.
I agree that while most packages should just define instances alongside their datatype/class definitions, it's simply not feasible for every package. text-show in particular suffers from this, since I split up the multitude of TextShow instances across several orphan-instance containing modules to (1) avoid having to compile everything in one mega-module, which would likely eat up all your RAM, and (2) keep me sane as I try to make incremental changes when developing it.

@RyanGlScott Ah, now I get why you linked to that specific comment, thanks for elaborating.
I agree these orphans are fine.
OT on orphans (sorry but couldn't resist): text-show sounds like an excellent use case for modules and separate compilation! Why would anyone add restrictions on that? </snark> (The snark is not against you but against coherence).
Also, in the cases under discussion, datatypes and instances are probably exported together (modulo internal packages), so they're safe I assume. I only checked for text though.

This was fixed in https://github.com/haskell/haddock/pull/724.


Fix support for IE11 in es5 build

-Add core-js@3 as a dependency
-Add preset-env configurations for core-js in es5 build config
-Keep webpack from transpiling core-js and babel libraries (see https://github.com/zloirock/core-js/issues/514 )

(I don't love the loose: true, but I also don't love having to support IE11 and yet here I am. loose is the only way I could figure to produce IE11-usable JS with useBuiltIns: 'usage')

Yeah, unfortunately this was the smallest I could figure out and have it work, because unfortunately the existing es5 bundle only threw errors for me in IE11. If nothing else, hopefully other people who need IE11 support will see this :)

I'll take a look at it! There should at least be an official way to use it within things like ie11 :D
Happy Halloween! ☠ 👻

I'll merge it since if people really need to support IE11 they gonna need it (unfortunatly) :/
I've updated the README since the size-difference is quite big and adding polyfills later would be way better.
Thanks for the PR :)


Update with Azure AD Graph Commands and Link

Promotion needs to be updated to state:
Azure AD Graph not Microsoft Graph
Give the link to Azure AD Graph
https://graphexplorer.azurewebsites.net/
and commands are different
GET https://graph.windows.net//domains/child.maindomain.com?api-version=1.6
POST https://graph.windows.net//domains/child.maindomain.com/promote?api-version=1.6
Run again to confirm promotion:
GET https://graph.windows.net//domains/child.maindomain.com?api-version=1.6
Also noted: The federated child verified domain after promotion immediately was Managed, did not have to run the MSOL commands

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 02c07bd2-01c5-77f4-11e0-6d942c5c0885
Version Independent ID: f61490c2-0f18-4609-29f1-99ccb4215ad2
Content: Change subdomain authentication type using PowerShell and Graph - Azure Active Directory
Content Source: articles/active-directory/enterprise-users/domains-verify-custom-subdomain.md
Service: active-directory
Sub-service: enterprise-users
GitHub Login: @curtand
Microsoft Alias: curtand

@debaxtermsft
Thanks for your feedback! I've assigned this issue to the author who will investigate and update as appropriate.

#reassign:@barclayn

@debaxtermsft thank you for your feedback. I am making changes to the article based on your feedback and the feedback of others. #please-close


[Snyk] Security upgrade notebook from 6.4.10 to 6.4.12

This PR was automatically created by Snyk using the credentials of a real user.Snyk has created this PR to fix one or more vulnerable packages in the `pip` dependencies of this project.
Changes included in this PR

Changes to the following files to upgrade the vulnerable dependencies to a fixed version:

requirements.txt

⚠ Warning
notebook 5.7.13 requires terminado, which is not installed.
nmrglue 0.8 requires scipy, which is not installed.
nmrglue 0.8 requires numpy, which is not installed.
nbformat 4.4.0 requires jsonschema, which is not installed.
nbconvert 5.6.1 has requirement mistune<2,>=0.8.1, but you have mistune 2.0.1.
ipython 5.10.0 requires simplegeneric, which is not installed.
google-auth 2.3.3 requires rsa, which is not installed.

Vulnerabilities that will be fixed
By pinning:

Severity
Priority Score (*)
Issue
Upgrade
Breaking Change
Exploit Maturity

521/1000  Why? Recently disclosed, Has a fix available, CVSS 4.7
Access Restriction Bypass SNYK-PYTHON-NOTEBOOK-2928995
notebook:  6.4.10 -> 6.4.12
No
No Known Exploit

(*) Note that the real score may have changed since the PR was raised.
Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the effected dependencies could be upgraded.
Check the changes in this PR to ensure they won't cause issues with your project.

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.
For more information:
🧐 View latest project report
🛠 Adjust project settings
📚 Read more about Snyk's upgrade and patch logic

Learn how to fix vulnerabilities with free interactive lessons:
🦉 Learn about vulnerability in an interactive lesson of Snyk Learn.

Fixed in #67


Generalize private field duck typing

🔍 Search Terms
private, field, class, types
✅ Viability Checklist

[X] This wouldn't be a breaking change in existing TypeScript/JavaScript code
[X] This wouldn't change the runtime behavior of existing JavaScript code
[X] This could be implemented without emitting different JS based on the types of the expressions
[X] This isn't a runtime feature (e.g. library functionality, non-ECMAScript syntax with JavaScript output, new syntax sugar for JS, etc.)
[X] This feature would agree with the rest of our Design Goals: https://github.com/Microsoft/TypeScript/wiki/TypeScript-Design-Goals

⭐ Suggestion
Currently, private fields are considered a part of a class's interface resulting in errors when trying to duck-type a given class for another one which matches all properties except the private field.
Given that private field access is validated at access, we suggest to remove the above restriction.
📃 Motivating Example
class Foo {
  #test = 'foo';
  hi() {
    console.log(this.#test);
  }
}

class Bar {
  #test = 'bar';
  hi() {
    console.log(this.#test);
  }
}

function sayHi(a: Foo) {
    a.hi();
}

// Before: Property '#test' in type 'Bar' refers to a different member that cannot be accessed from within type 'Foo'.
// After: No error!
sayHi(new Bar());

💻 Use Cases

What do you want to use this for?
To avoid boilerplate and simplify API designs.
What workarounds are you using in the meantime?
Defining a separate interface.
What shortcomings exist with current approaches?
Defining a separate interface creates a lot of boilerplate. The suggestion is also valid behavior.

The suggestion is also valid behavior.

Class instances can access each others’ private properties, not just their own.  If that happens, Bar may not be substitutable for Foo.  Hence why they aren’t interchangeable.

Your example works not because of any comparison between the #test fields, but because sayHi only uses hi().  It would be better to extract that as an interface, and then use that interface as the parameter type to sayHi
interface Hiable { hi() }
class Foo implements Hiable { ... }
class Bar implements Hiable { ... }
function sayHi(a: Hiable) { ... }

The suggestion is also valid behavior.

Class instances can access each others’ private properties, not just their own. If that happens, Bar may not be substitutable for Foo. Hence why they aren’t interchangeable.

There are already checks%3B%0A%20%20%20%20%7D%0A%0A%20%20%20%20function-,checkPrivateIdentifierPropertyAccess,-(leftType%3A%20Type%2C%20right) for that, so if one attempted to access a private property in a different class, this is already caught.

Your example works not because of any comparison between the #test fields, but because sayHi only uses hi(). It would be better to extract that as an interface, and then use that interface as the parameter type to sayHi

https://www.typescriptlang.org/play?#code/FAYwNghgzlAEBiB7RsDexawMQBcCmUOsAvLAOQBmyZA3BrABYCWAFAJRr2YiIB2UiMHgB0YRAHMWOZlGG4CONnUwBfYGtCQYsAEIQATp0zzCJcgCMDtes3ZHMsHv0Eixk6U1knFy2Go0UAK68IDhMfLBQEACeABKsEABcCMgc6A6wEMK2SurAwFFxrLx4AO66Buy5QA

There are already checks

Okay but that’s a runtime error.  The type system’s main job is to prevent runtime errors caused by using the wrong type.

The current behavior is intentional.

so if one attempted to access a private property in a different class, this is already caught.

You're talking about a runtime crash - the whole idea of having a type system is to prevent those from occurring.

IIUC, TypeScript checks this in their type checker (which is compile time, not runtime). See checkPrivateIdentifierPropertyAccess in src/compiler/checker.ts

Right, and you're proposing that we not check it...

if one attempted to access a private property in a different class, this is already caught.

It is not… or rather it is, but only through the error you’re suggesting be removed:
class Foo {
  #foo = 777;
  stuff(x: Foo) { console.log(x.#foo); }
}

class Bar {
  #foo = 1206;
  stuff(x: Bar) { console.log(x.#foo); }
}

new Foo().stuff(new Bar());  // runtime error

Right, and you're proposing that we not check it...

No, I am not proposing that. We should check that. What I am suggesting is allowing the example outlined in the issue above which is governed by a separate check.

By what means should this example have an check-time error if private fields are treated the way you're proposing they're treated?

By what means should this example have a check-time error if private fields are treated the way you're proposing they're treated?

Yes, this should result in compile time error.

How? The proposal is that a Foo should be usable as a Bar. So there's no error in that program now.

Ah, you're right. Sorry, misread that example. Indeed, it seems to be problem when you have this swap of classes in such a way.
I guess this has no solution then :/ The main idea was to capture the fact that
class T {
  #prop: string;
}

can be compiled down to
const PROP = new WeakMap<T, string>();
class T {
}

But it seems the allowance of same-instance private field access puts a full stop on removing this type check. Such a shame...


Mandatory fields

Implements the changes discussed in #118 except for those that are to do with inline formsets, which will be tackled separately in the next PR.
In general, several fields have been made mandatory and a data migration adds the most common device types and configurations.

The problems in #118 relating to formsets have been moved to a separate issue with more detail in #249. @AdrianDAlessandro please feel free to add your thoughts to that issue as you did the original review of frontend forms and mandatory fields.

Thanks @AdrianDAlessandro.

One thing in https://github.com/ImperialCollegeLondon/Faraday-liionsden/issues/118 that isn't ticked and doesn't appear to be addressed is the DeviceParameter.value field. It's still not mandatory, but the issue claims it should be.

Yeh well spotted - I actually added that to the bottom of the list while writing the new formset issue. I'll add it to this PR before merging (like you say - it's quite a small change).


Add test vectors

Just slowly making my way through.

[x] NUT-01
[ ] NUT-02
[ ] NUT-03
[ ] NUT-04
[ ] NUT-05
[ ] NUT-06
[ ] NUT-07
[ ] NUT-08
[ ] NUT-09

Feel free to comment or request additional tests.

@thunderbiscuit thanks for taking the time to add tests 🎉  -- for NUT-3, this issue might be useful for some vectors: https://github.com/cashubtc/cashu/issues/228
Effectively implementations should support any amount (including over 21,000,000 * 100,000,000 sats, which is impossible), and let their upstream (lightning or whatever) fail rather than the mint request itself breaking, but it's probably Ok for impls ("must") to be able to support up to 51 bit uints (ceil log2 of 2.1 quadrillion sats), so probably 8-byte integers (uint64)
Also notable here the issue was with the backing database, rather than the mint itself because Python handles very large integers pretty well

@AngusP thanks for the idea for the extra tests! Related to this is #28.
This PR is getting a bit old and I don't want people to wait for me to finish it all before merging, just because I'm not sure how much time I'm going to have to dedicate to this over the fall.
@callebtc if you like the tests here, feel free to comment/request changes/merge and not wait for me to add tests for all the NUTs.


Implement authorized app prefix matching

TL;DR
For development purposes - it would be nice to allow the authorized app allow-list to match AppPackageName/BundleID by prefix
Design
Proposal
likely needs column in database to indicate if prefix match is allowed.
The cache is currently by exact match. This would involve adding the ability to to an exact match and then a prefix match.
/kind enhancement

Radix trees! (somehow every Go project I work on inevitably needs a radix tree)


support echo with no parameter for v6 change in syntax

For your review @siowyisheng, I tested on my mac with below and works for normal and live mode.
echo 1
echo
echo 2
echo
echo 3

tested and LGTM


Confusing text on the about page

"The TripleSec library encrypts data in four steps:
Key derivation. Given a user-provided password, and a random salt value, generate four separate secret keys, one for each cipher (see Step 3), and two final keys for signing the ciphertext (see Step 4). This "key stretching" is done via standard scrypt, with parameters N=215, r=8, p=1, and output length of 192 bytes. The output of this step is five separate keys, used below. "

Shouldn't that be "...three separate keys, one for each cipher..." ?

Fixed in #54.

@maxtaco to be closed, probably.

Yeah, the change has been committed to GitHub but not published to the site yet, I'm sure Max has a lot on his plate.  Either he or I can close this once it goes live to the site.

Confirmed this was still incorrect on the website

Should be fixed on keybase.io/triplesec. Note that we released a new version as well.


PARQUET-455: Fix OS X / Clang compiler warnings

There actually was a legitimate bug fixed here for malformed Parquet files, but we are not yet in a position to write a decent test for it until PARQUET-497. I will make a note on that JIRA.
I also set our Travis CI build to fail on future compiler warnings.
This also closes #15.

LGTM, +1

Cool, fire when ready. Thank you


Azure add host

What does this PR do?
This PR fixes the generation of add_host.sh from bastion.sh.  Without this update the add_host.sh contains variable expansion from bastion.sh instead of the supplied arguments given when calling add_host.sh.
How should this be manually tested?
This can be tested by deploying the bastion host and verifying the add_host.sh script doesn't contain variable expansion from bastion.sh.
Is there a relevant Issue open for this?
Who would you like to review this?
cc: @tomassedovic @bogdando PTAL

@e-minguez @glennswest PTAL

LGTM. @glennswest since you merged the script with the bastion.sh, do you mind to check it? Thx!

@glennswest do you mind to take a look? Thx


[release-0.7] Start centralizing APIExport indexers

Backport of #1762 from main to release-0.7

/approve


Naming problems

Thank You for Your work.
I found some errors inside code.
Class Cgsmith\Validate\ValidateRecaptcha should be named 'Recaptcha' or Zend will not load it.
Because You are using namespaces every Zend_Exception should be executed as \Zend_Exception.
That's alle I think.

Thanks @xorock
I updated the code and pushed it to 1.0.1


Citizen science page missing

Just spoke with Karen Soacha of SiB Colombia at the Citizen Science Association conference in the U.S., and had her ask where the citizen science page on the new site went. Now I'm asking, too ;-)
I suspect it's a Drupal-to-Contentful related menu issue, no?

@kcopas https://github.com/gbif/portal16/issues/86 indicates that was still a work in progress, is that true?
I had a similar question about the missing infrastructure section. https://github.com/gbif/portal16/issues/226 indicates a decision is still pending how to show that content in the new portal? Any ideas? Thanks

Pages authored using the "generic content" template in Drupal were not part of the Drupal->Contentful import. Because the number is low, and content still needs to be revised, I've agreed to manually re-recreate these in Contentful. This affected pages such as "Data papers", "Contact us", "About GBIF", etc. but I've started bringing them back. CitSci page is next! ;)

https://demo.gbif.org/citizen-science is back up now. I've added a picture to keep it in line with the look of the other pages. Let me know if anything needs changing. Note that this content now lives in Contentful and is not imported from Drupal.


🐛Bug: Backspace on Tags Input closes keyboard on mobile

https://twitter.com/pawelblaszczyk_/status/1674072874167009281

Mm. I know why ..
I designed it that only 1 element is in focus. If a tag is in focus, then the input goes out of focus.
When you hit backspace and are at position 0 on the input, the last tag is selected , gets focus and the input looses forcus
Zag operates differently so this isn’t an issue. The input is always in focus, even when you tab or arrow through tags. Only when you click/tab off the component, does input lose focus.
I could rework it to use this approach. It’s a big change but happy to do it if you prefer input always in focus …?

@geerew we could do that, or have backspace delete the last input instead of focusing on it. Need to see if there's any accessibility concerns with that though.

I wouldn’t be happy with that approach, as you could accidentally delete a tag.
The approach now is such that when you hit backspace the last tag is set as selected and if you were to hit backspace again, then it would be deleted.
To be honest, always having the input be in focus solves a lot of problems with the features that I want to add such as Blur and it kind of makes sense that you can always type a new tag, even if other types are selected. As it stands now, there isn’t even a point in selecting a tag because I haven’t implemented the tag edit feature yet.
If you are happy with it, I think I will go ahead and make it that input is always in focus ..?


Doesn't fully render characters

When using the font found here: https://www.freejapanesefont.com/aozora-mincho-download/
it doesn't fully render characters that characters like g and j. Specifically it cuts off the bottoms of said characters.

An update at some point appears to have broken Japanese characters in general. I just updated the version of FC that I use for one of my games, and Japanese characters no longer render.

@TG-Galeforce Do you know a version which did work?  I can bisect from there.

Let me poke around with it and get back to you on it.


Number Comparator

Implements Comparator for natural-numbers, signed integers and fractionals.

Fractional comparison tested only for decimals.

TODO: support scientific notation, bin(0b), oct(0o), hex(0x)

TODO: resolve "ts(2589) type instantiation is excessively deep" error for test case 42.04, 42.02.

Implements...

digit-list from-string method

exposes _$fromString2 method which doesn't trim leading zeros.

number absolute method

converts sign of any input number to positive.

I think there are some very valuable contributions here, and I'm intrigued about your approach.
I am concerned about performance - I notice that you are performing subtraction internally in some substeps. To me, comparison should be 'strictly simpler' than subtraction, even for real numbers.
Re ontology, I am drawn to potential future modules DecimalDigitList and Decimal, e.g. Decimal.Compare. To me, Fractional would be a hypothetical module that performs arithmetic by representing numbers as fractions, i.e. would be capable of exact representation of 1 / 3. That doesn't seem to be the case here, however.
By encoding core arithmetic logic in DecimalDigitList, I believe we could implement a more performant version, and avoid excessive conversion between numeric and string-list representation.
Finally, re your two failed test cases, I have a suspicion that this might be related to an interesting bit of nuance I found the other day re the current DigitList.Subtract implementation.

I definitely agree with having distinct Decimal and Fractional modules. How to handle exact values vs. approximate representations was the question I had going into this, and this PR is only intended to cover the first use case.
I've rewritten the _$decimalCompare method to avoid using subtract by reusing logic from digitlist-compare. I believe it should be more performant. The excessive depth error hasn't gone away, however, although one of the two test cases previously affected isn't anymore (error for 42.04, 42.02 but not 42.02, 42.04).

Also, observed something interesting. Haven't investigated it further yet.

Anomaly: Replacing A_DONE, B_DONE in line 42 with their assigned types from lines 39-40 results in the following error. Possible bug in Boolean._$and or Conditional._$equals? Also happens for Conditional._$extends.

error TS2344: Type '0 | 1 | -1' does not satisfy the constraint '1 & _'.
  Type '0' is not assignable to type '1 & _'.
    Type '0' is not assignable to type '1'.

I see you're still working on this (and you may have fixed the only remaining issue while I've been investigating), but I've noticed that there is no as such infinite loop: rather, it's just returning never:

Because it was the only way I figured out how to cause an error, if you pass never into Test.Expect it causes an infinite loop. This is because otherwise since never is a universal subtype, I could "never" cause it not to otherwise pass.

This strategy I use sometimes for debugging of complicated, multistage types may prove useful to you:

This strategy I use sometimes for debugging of complicated, multistage types may prove useful to you:

That is a fantastic idea. Can't believe I never thought to try that.

I suspect you will find this in short order. I have gotten all tests to pass on my side. Check out my feature/compare-fractions-testing branch on this repo if you want to see the solution :D

Wow that turned out to be a really simple fix. Thanks for the sanity-restoring debugging tools.😂


Improved performance of fft imports/updates (10s -> 0.5s)

This is just a small change that caches the attributes instead of querying them on every update. The mongo query itself is quite inefficient (it joins the fc, fg for every document when we only need that at the end of aggregation (1000 fetches vs 80) but since it only takes 0.1-0.3s we may stop here for now.
@laura-king

Should we remove the upload file size limiter in this PR? If performance is improved then there's no reason to cap the file size at this low of a size.

Perfect, looks good.


Unequal backtesting and forward walking periods

It would be great to use unequal backtesting and forward walking periods.
I.e., it would be nice to backtest over e.g. 6 months and walk forward those parameters 1 month, then rinse and repeat. As the tool stands right now, I don't see a way this can be achieved.

It is possible to specify an in sample period and a walk forward that skips an unequal window, so for instance you can have an in sample of 5 days that walks forward to skip the weekend.  You can specify this pattern with this kind of setting:
"hourSegments": [ 24, 24, 24, 24, 72 ]
This was only ever quite rudimentary as I've turned my attention to a different approach using an adaptive fitness measure. The AdaptiveSharpeRatioFitness will increase the size of in sample period directly proportional to an increase in Sharpe Ratio.
Neither of these things are exactly what you're asking for but may keep you going whilst I decide how to approach the overlapping walk forward you're interested in.

I have now merged a first working version of a walk forward optimizer into my new .net core port of the Lean Optimizer here:
https://github.com/jameschch/LeanParameterOptimization
This is not fully tested at this stage but I do recommend you switching over to the other project as I will probably not be updating this codebase with any new features.

Closing this as walk forward is now supported in other repo


Feature Request: Subscription Dependencies on Check Status

A new feature that allows a client to automatically unsubscribe from a subscription based on the status of a check, and resubscribe when the status of the check returns to OK -- Particularly focused around clients that are part of a roundrobin subscription.
Expected Behavior
It would be useful to provide a means of having a client dynamically drop in & out of a subscription (particularly thinking of roundrobin subscriptions) based on the clients current health, to prevent checks being ran against a client that is not healthy.
Current Behavior
Currently a client will remain part of a given subscription until configuration is changed & the client is reloaded. A client will never unsubscribe from a subscription in the face of degraded availability, except in the event of a client failure.
Possible Solution
Client attribute to define the checks that should be used to determine if a node is healthy & ready to accept roundrobin checks.
i.e.
"subscription_dependencies": [
    "general_node_health_check"
]

Context
Scenario 1
In cases where a roundrobin subscription is used to execute a check (specifically metric checks) against a clustered service, often specific nodes might need to be used to run this check (i.e. where an API is not available, and metrics must be gathered from one of the machines in the cluster.
In the event that the service in question on one or more of these nodes fails, it will still receive roundrobin requests to collect these metrics, however due to the failing service it will not be able to collect these metrics.
Example: A cluster that is split between two AZ's suffers a complete service failure in one AZ, however the nodes are still up and sending keepalives. All nodes will continue to receive the roundrobin requests, however only half will be executing successfully, resulting in a loss of metrics.
Scenario 2
In cases where dedicated Sensu servers are used to run remote checks (i.e. runners), a similar scenario could arise where checks are intermittently failing due to degraded performance on nodes

Hi @evesy,
The idea you have proposed falls outside of the scope of Sensu's intent. It would require sensu-server to have specific knowledge of your infrastructure and services being monitored. Even if the server had this level of information, sensu-client configuration is immutable while running and thus a client's subscriptions cannot be altered.
Thank you for your suggestion but we will not be implementing this at this time.


Switching network to add liquidity throws error

Describe the bug
See title
To Reproduce
Steps to reproduce the behavior:

Go to router page: https://amarok.connextscan.io/router/0xf26c772c0ff3a6036bddabdaba22cf65eca9f97c
Manage router.
Select a different network than your connected network.
Try to add liquidity.
See error.

Additional context
Refreshing the page makes it work but that's a bad UX.

Understand that it's duplicated to #2983. @rhlsthrm please confirm.

Understand that it has the exact root cause as #2983.
We are still not sure why it happens. We interact with the SDK only for creating and changing the signer address (as shown in the image -- filtering console by "sdk"). The UI wasn't informed whether the user had changed the network or not. @sanchaymittal, please suggest if you have any clue about this. Thank you!

closing as dupe #2983

This issue is still active? i can't reproduce this again. even though I exactly did as above. Already fixed?


fix(ephemerals): Quick fix for ephemeral spin-up

SUMMARY
Quick fix to add support for the "new" requirement to include a SECRET_KEY

@craig-rueda @rusackas Why not store an actual SECRET key in this Github Repo and let the Github runner pass it to ECS via the very same 'render-task-definition' action using the environment-variables parameter? https://github.com/aws-actions/amazon-ecs-render-task-definition/blob/b451ae8d4af191e94b479ca69435c85a0bd25140/action.yml#L16-L18

We might still do that... but it requires an Apache Infra task to set that up, and we wanted to unblock the broken ephemerals as expediently as possible. While it would be a nice thing to do, I'm not particularly worried about the security of these workspaces, as everyone on the internet knows the admin credentials anyway :)

I suspected that something like that would be the reason. :slightly_smiling_face:
I disagree with with the security stance though. It's not the fact that these workspace are public that I am lamenting, but the fact that the actual practice for publicizing secret keys is unchanged as a result of this PR. Totally understand the need for expediting of this fix in this case of course.
Another option would be to use an actual random string with high entropy by following the Flask documentation recommendations: https://flask.palletsprojects.com/en/2.3.x/quickstart/#sessions
https://flask.palletsprojects.com/en/2.3.x/config/#SECRET_KEY

We could definitely provide a randomized secret key on these ephemeral environments, but it seems a bit moot when we publish the admin credentials publicly on the thread as they're spun up. Maybe we can add an extra warning on the PR comment that lists those credentials with a "⚠ This instance is public: do not connect production databases! ⚠" warning, or something similar.


As the PRC, 1946, the game always crashes after the ceasefire event with Nationalist China.

Needs to be confirmed

Unable to reproduce. Please reopen with specific event that cause the CTD


Also show full file conflict solving buttons when there's no hunks

Fixes #241

It works very well. Thank you for everything, folks.
I can't wait for the flatpak to come out. ;)
In the meantime, happy with this build.


WindowShade Service; Disconnecting Client

Hi,
I'm using the WindowsShade service which works great with custom A4988 driver for a stepper motor as long has the steps are kept below and certain threshold if the time is longer the slider becomes unresponsive after the travel completes and I get the "Disconnecting Client"  message (see below) is there anyway I can keep the connection open longer?
PRINT OUT from Serial Monitor
Lowering Shade
current position :75 %
New target position %: 50 %
Stepper target No. steps: 250
Updating aid=2 iid=7 (okay)
In Put Characteristics #0 (<IP_ADDRESS>)...
Lowering Shade
current position :50 %
New target position %: 0 %
Stepper target No. steps: 500
Updating aid=2 iid=7 (okay)
In Put Characteristics #0 (<IP_ADDRESS>)...
Raising Shade
current position 0 %
New target position 100 %
Stepper target No. steps: -1000
Updating aid=2 iid=7 (okay)
** Disconnecting Client #0  (105 sec)

This message means the client (i.e. the Home App) dropped the connection.  I'm unaware of how to extend this time out, though there may be setting deeper within one of Espressif's WiFi APIs.  When operating your stepper motor, does it block the rest of the code from running?  If so, I think you are seeing a time-out from the Home App itself.

Yep it was the stepper motor blocking the rest of code, I moved the Stepper motor drive logic into a service loop, then passed +/-numberOfSteps into two separate dedicated helper functions one to drive the motor forward (lower Window shade) and the other to reverse (Raise Window shade).
I'm constructing a Sky Lantern shade for a 3m x 2m opening so was concerned about timing-out but recently tested the code with 10,000 step and didn't experience any time-out or dropped connections.
The WindowsShade is configured in STEPS of 25%, I have two homing functions which resets the currentPosition once the shade reaches the fully open or fully closed limits switches in case of any motor skip steps.
The issue I need to resolve is preventing the user from pressing multiple Slides (putting the motor and slider positioning out of sync) before the motor has arrived at target.
Q. do you know if they is anyway to freeze/unfreeze the Home APP; WindowShade Slider?
Thanks bro!

I don't know of any way to freeze the slider in the Home App to prevent a user from pressing while in motion.  However, for my own setup I keep track of the absolute position of the shade so that is the user changes direction, the sketch can make adjustments accordingly.  This does require a known starting point (such as fully open or fully closed), but if you know it's 10,000 steps to go from one to the other, you can move the shade only the absolute number of steps needed to properly set the position.

@MnemonicFish Hi, I am starting a similar project and if it's not too much asking I would love to get your code to get started... I have read through a few pages here and there and it seems like there are many issues
your project is pretty much exactly what I am looking for including the end limit switches to reset the position
Thanks in advance!


ci: added prerelease pipeline

This PR expands the support for Node.js pre-release versions, with focusing on Release Candidate (RC) versions of upcoming Node.js releases.
This will ensure that the Node.js tracer is ready for any breaking changes or new features introduced in the next major Node.js release.
Support for nightly builds is also considered, though the priority remains on the RC versions.
RC version registry- https://nodejs.org/download/rc
Nightly version registry- https://nodejs.org/download/nightly

[x] install node v23 rc/nightly version
[x] install v23 in every tasks
[x] verify npm install packages
[x] incorporate prerelease version check in supportedVersionCheck
[x] incorporate prerelease version check in nodeJsTooOld function

TODO

[ ] fix/skip test groups

refs INSTA-9632

echo-env shows
Node: v20.17.0 NPM: 10.8.2

Currently I have added the release versions toinstall-npm-dependencies.yaml task and other test group tasks because that's where we need the new versions to run tests.
I can add this to echo-env and other tasks as well but IMO we don't need this installation in every steps. Because after the echo-env step, we are re installing node in other steps, so the execution env changes right?, so what do you think?

Because after the echo-env step, we are re installing node in other steps, so the execution env changes right?, so what do you think?

The echo env is the visibility for the versions. I would love to see the used node --version and npm --version.


[2010-03-09] Use Zend_Locale to replace/simplify many i18n routines.

created by: lhudson
created at: 2010-03-09
original ticket: http://open.silverstripe.org/ticket/5163

We want to get rid of dependence on such things as i18n::$all_locales, and have a robust system.
Zend_Locale seems to do what we want and is already included in 2.4 beta.
See relevant notes in  http://open.silverstripe.org/ticket/5098
We need to find a way to extend Zend_Locale so as to provide such locales as M?ori/mi_NZ.
W.I.P. patch forthcoming...

https://github.com/silverstripe/silverstripe-framework/pull/6607


Possible mistake of adding two "route" statements in creat_rout_and_subscribe.py file

In the file creat_rout_and_subscribe.py, which is present in every "Smart Grid Model", following two lines of code are present.
    line = line + "route \"commit:Market_1.current_price_mean_24h -> HOUSE_"+str(index)+"/avgPrice; 0\";\n";
    line = line + "route \"commit:Market_1.current_price_mean_24h -> HOUSE_"+str(index)+"/avgPrice; 0\";\n";

As we can see both of the lines are exactly same.
This file is used to generate the content for another file called fncs_msg.txt. The  variable line used in above two statement is used to write the content of the required file line by line.
I am still understanding the meaning of the "route" and "subscribe" stataments(which are written in this file). So, I'm not sure why we need to have the same statements twice. Though, it looks to me that it's probably a mistake.
If we look at the body of the while loop from where above 2 statements are taken then we have the following code.
    line = line + "route \"commit:Market_1.current_market.clearing_price -> HOUSE_"+str(index)+"/clearPrice; 0\";\n";
    line = line + "route \"commit:Market_1.market_id -> HOUSE_"+str(index)+"/mktID; 0\";\n";
    line = line + "route \"commit:Market_1.current_price_mean_24h -> HOUSE_"+str(index)+"/avgPrice; 0\";\n";
    line = line + "route \"commit:Market_1.current_price_mean_24h -> HOUSE_"+str(index)+"/avgPrice; 0\";\n";
    line = line + "route \"commit:Market_1.current_price_stdev_24h -> HOUSE_"+str(index)+"/stdevPrice; 0\";\n";

    line = line + "subscribe \"function:auction/submit_bid_state <- ns3_1/fncs_msg/HOUSE_"+str(index)+"@Market_1/submit_bid_state\";\n";
    line = line + "subscribe \"precommit:HOUSE_"+str(index)+".proxy_clear_price <- ns3_1/fncs_msg/Market_1@HOUSE_"+str(index)+"/clearPrice\";\n";
    line = line + "subscribe \"precommit:HOUSE_"+str(index)+".proxy_market_id <- ns3_1/fncs_msg/Market_1@HOUSE_"+str(index)+"/mktID\";\n";
    line = line + "subscribe \"precommit:HOUSE_"+str(index)+".proxy_average <- ns3_1/fncs_msg/Market_1@HOUSE_"+str(index)+"/avgPrice\";\n";
    line = line + "subscribe \"precommit:HOUSE_"+str(index)+".proxy_standard_deviation <- ns3_1/fncs_msg/Market_1@HOUSE_"+str(index)+"/stdevPrice\";\n";

It seems like for every "route" statements there is a "subscribe" statement. For example, we have a "route" statement for "clearPrice" and one "subscribe" statement for "clearPrice" as well. The same goes for "avgPrice", "stdevPrice" and "mktID". But we don't have such a pair for "submit_bid_state"(only a "subscribe" statement but not "route" statement). And at the same time there are 2 "route" statement for "avgPrice".
So, may be(by mistake), the author have written two "route" statement for "avgPrice" and no "route" statement for "submit_bid_state".
As I said previously, I'm still understanding what these "route" and "subscribe" statement mean. I will comment about my understanding as soon as possible.

Also, I tried finding official docs of fncs(Framework for network co-simulation) for understanding GridAttackSim better but couldn't find one. I request authors to kindly provide any link for those official docs(if exists) or any other resource to understand fncs.
Thanks.

Hello,
I'm Tan from the project team, and I want to thank you for your inquiry.
Here are some resources related to the FNCS tutorial you mentioned:
FNCS tutorial: You can access it at this link: FNCS Tutorial
Official document: You may refer to this document for more information: Official Document
Official website: Visit the official website of FNCS here: FNCS Official Website
It's worth noting that the FNCS project concluded five years ago. Consequently, some libraries are outdated and may not function correctly. Moreover, FNCS is incompatible with Ubuntu 20. Considering this, I highly recommend transitioning to HELICS.
You can explore HELICS by visiting their website: HELICS
Please let me know if you have any further questions or need additional assistance.
Best regards,
Tan


DO NOT MERGE! Adding google analytics for prod/dev environments

Using the current/live GDI tracking code for production environment so not sure if we want to merge this yet? The development code (made with new dev account) is tried and tested on my local machine so we're good there. 'Least it's ready when we are! :+1:

Adding a "DO NOT MERGE" warning to this one, as we'll need to wait until the DNS switch to do this and be able to test properly. THANK YOU for knocking this our @kstack7! You :guitar:

Looks perfecto @kstack7! We'll wait for the DNS to officially propagate before merging in.