Poor performance on Ampere vs. Ada with bitpacked weights

I am writing a library to perform different low-bit matmul kernels in Triton/CUDA.
The Triton kernels work great on Ada gpus like the 4090 RTX and the A6000 Ada - on par with Marlin on large matrices. However, the same kernels perform poorly on Ampere gpus (mainly the A100, but also Hopper like H100 by the way).
I also tried the old gpus like the Titan RTX and the 2080 Ti and I get an ok speed up (~2.5x) with INT4 data.
So I tried to debug every single op and it turns out, there are two strange issues:

tl.load() with repeated indices offs_k[:, None] // elements_per_sample is much slower on the A100 compared to the 4090. Running a gemm with an int4 16k x 16k  packed matrix is 3.4x faster than fp16 on the 4090 - as expected for a memory-bound scenario - but it's only 1.13x faster on the A100.

The shift operator when used to unpack the weights can heavily impact the performance depending on the content of the shifter. For example: b >> offs_k[:, None] % 8  is much slower than b >> off_k[:, None] end-to-end on the A100, while this doesn't impact the end-to-end performance on the 4090. (and even if you put it outside the loop, it doesn't impact the result much, so it's really the content of the shifter that seems to impact the speed).

I put a full gist here with numbers: https://gist.github.com/mobicham/06acdeb343490ab4ed1159526cdfa509
I suspect problem 1 is the main blocker making the whole kernel slower.
Would really appreciate any insights, since this really hinders using Triton as a solution for low-bit kernels. Really open to any suggestions to make this thing work!
Thank you very much in advance 🙏

Update:
I was able to get good performance on the 3090 by limiting the number of stages to 1.
However, the same across doesn't work for the A100, H100. Performance is still very poor with tl.load and bitpacked data, including using the latest build from October 14th.

This seems relevant: https://github.com/triton-lang/triton/pull/4950#discussion_r1806722110. Specifically,

K // DIV_FACTOR was not folded into a constant

The comment was incredibly wrong, as K was not a constant (oops). Now I made explicit the divisibility of K in the kernel and everything works as expected. That being said, our constant handling in the AxisInfo pass is less than good, but yeah.

In the case of the OP, perhaps the loads are not being pipelined because they are too small. See the condition in
https://github.com/triton-lang/triton/blob/d4e5a7873323107ec16ff03c0727256d060908eb/lib/Dialect/TritonGPU/Transforms/Pipeliner/MatmulLoopPipeline.cpp#L424-L434
You can check whether this is the case by deleting your cache and running the example with TRITON_LLVM_DEBUG_ONLY ="triton-matmul-loop-pipeline".

Thank you embg for the pointer !
In my case, the it's a range of elements not just an constant, in the form [0,0,0,...1,1,1....,127,127,127]
@lezcano I see that you use tl.range with num_stages. I found that in some cases num_stages=1`` solves some issues with the A100/H100, but I see you use 4 in your tests. I consistently either got bad performance with num_stages>1` or a crash, is there some logic to set that num_stages in the for loop?

If you run your script with the flags above, you'll be able to see the inference that triton is able to do on thing slike the number of contiguous elements that are constant / the divisibility of elements, etc, and the decisions it takes wrt. pipelining in terms of those. It could be that our Axis analysis were less than good, which would mean that we would be bailing on pipelining for no good reason.

@lezcano any insights on why tl.load with packed data would be significantly slower than on other gpus as the description of the issue mentions? Even the 3090 which is also Ampere doesn't have this issue. This problem is causing all the kernels that use packed data for quantization perform poorly on the A100.

@lezcano here's the result I run on an A100 SXM4. A couple of things I noticed that might be relevant to solve this issue:

The speed of INT8 without packing is similar to that of INT8 or INT32 with bitpacking. Meaning, the speed of loading INT1 packed as INT8 or INT32 is the same speed as loading INT8 - this is issue is NOT happening on the 3090, 4090, 2080, A6000 Ada.
I also noticed that FP16 x FP16 with FP16 accumulation is not faster, even sometimes slower on the A100. Again, no issue  on the other gpus, even on the 3090 from the same generation.

Really appreciate your help here, this is kinda a big blocker to deploy on the A100 :pray:
input_dtype, elements_per_sample = torch.float16, 1 #Float16
[axis-info]: getPtrAlignment order[0] 1 maxMultipleBytes = 16 maxContig = 64 elemNumBits = 16 maxMultiple = 8 alignment 8
[axis-info]: -- contiguity = [1, 64], divisibility = [2, 16], constancy = [1, 1], constant_value = <none>
[axis-info]: getPtrContiguity uniqueContigPerThread = 8
[axis-info]: getPtrAlignment order[0] 1 maxMultipleBytes = 16 maxContig = 128 elemNumBits = 16 maxMultiple = 8 alignment 8
[axis-info]: -- contiguity = [1, 128], divisibility = [2, 16], constancy = [1, 1], constant_value = <none>
[axis-info]: getPtrContiguity uniqueContigPerThread = 8
[axis-info]: getPtrAlignment order[0] 1 maxMultipleBytes = 16 maxContig = 64 elemNumBits = 16 maxMultiple = 8 alignment 8
[axis-info]: -- contiguity = [1, 64], divisibility = [2, 16], constancy = [1, 1], constant_value = <none>
[axis-info]: getPtrContiguity uniqueContigPerThread = 8

input_dtype, elements_per_sample = torch.int8, 8 // 1 #INT8
[axis-info]: getPtrAlignment order[0] 1 maxMultipleBytes = 16 maxContig = 64 elemNumBits = 8 maxMultiple = 16 alignment 16
[axis-info]: -- contiguity = [1, 64], divisibility = [1, 16], constancy = [8, 1], constant_value = <none>
[axis-info]: getPtrContiguity uniqueContigPerThread = 16
[axis-info]: getPtrAlignment order[0] 1 maxMultipleBytes = 16 maxContig = 128 elemNumBits = 16 maxMultiple = 8 alignment 8
[axis-info]: -- contiguity = [1, 128], divisibility = [2, 16], constancy = [1, 1], constant_value = <none>
[axis-info]: getPtrContiguity uniqueContigPerThread = 8
[axis-info]: getPtrAlignment order[0] 1 maxMultipleBytes = 16 maxContig = 64 elemNumBits = 16 maxMultiple = 8 alignment 8
[axis-info]: -- contiguity = [1, 64], divisibility = [2, 16], constancy = [1, 1], constant_value = <none>
[axis-info]: getPtrContiguity uniqueContigPerThread = 8

input_dtype, elements_per_sample = torch.int32, 32 // 4 #INT4
[axis-info]: getPtrAlignment order[0] 1 maxMultipleBytes = 16 maxContig = 64 elemNumBits = 32 maxMultiple = 4 alignment 4
[axis-info]: -- contiguity = [1, 64], divisibility = [4, 16], constancy = [8, 1], constant_value = <none>
[axis-info]: getPtrContiguity uniqueContigPerThread = 4
[axis-info]: getPtrAlignment order[0] 1 maxMultipleBytes = 16 maxContig = 128 elemNumBits = 16 maxMultiple = 8 alignment 8
[axis-info]: -- contiguity = [1, 128], divisibility = [2, 16], constancy = [1, 1], constant_value = <none>
[axis-info]: getPtrContiguity uniqueContigPerThread = 8
[axis-info]: getPtrAlignment order[0] 1 maxMultipleBytes = 16 maxContig = 64 elemNumBits = 16 maxMultiple = 8 alignment 8
[axis-info]: -- contiguity = [1, 64], divisibility = [2, 16], constancy = [1, 1], constant_value = <none>
[axis-info]: getPtrContiguity uniqueContigPerThread = 8

and the matmul loop pipeline info? The contiguity seems reasonable at first sight

Hi, I started looking into this issue over the weekend. @mobicham thank you for the great reproducers.
I noticed a couple things whilst trying to track down this issue:

I noticed that in the reproducer, if we change the underlying bitwidth storage from 32 -> [8, 16] the code is successfully compiled with number of stages > 1 (disregarding correctness of reducing the bitwidth).
It seems that pipelining is changing the way the MMA instructions get configured based on the bitwidth. I've tracked down a couple places inside of the pipeliner that may the root cause of this hypothesis. Most notably here: https://github.com/triton-lang/triton/blob/d6739d3c33dee481f2d4dee4f6ecd4123f671597/lib/Dialect/TritonGPU/Transforms/Pipeliner/MatmulLoopPipeline.cpp#L222 and possibly in the caller function https://github.com/triton-lang/triton/blob/d6739d3c33dee481f2d4dee4f6ecd4123f671597/lib/Dialect/TritonGPU/Transforms/Pipeliner/MatmulLoopPipeline.cpp#L547

It seems that the dot-encoding is getting changed when we convert from dot -> shared in the above permalinks and the conversion references the bitwidth of the source matrix which might be another potential issue. Tb-gen conversion for reference: https://github.com/triton-lang/triton/blob/d6739d3c33dee481f2d4dee4f6ecd4123f671597/include/triton/Dialect/TritonGPU/IR/TritonGPUAttrDefs.td#L365
Another potential issue that is different than the hypothesis above is that since the original bitwidth is 32 it seems likely that the emitted MMA op is still trying to use the i32 type which doesn't exist for MMA instructions of the form fp16 * X. That seems to be suggested from the error above triton LLVM ERROR: mma16816 data type not supported.
I could be completely off-base here with some of my findings but it seems that these could be likely culprits for why bitpacking an i32 doesn't work and bitpacking i[8,16] works.

@brod4910 thank you very much, really appreciate your effort !
@lezcano thank you very much for your patience. The reason why there's no pipeline output is because I am using for k in tl.range(...) because for k in range(...) which outputs the pipeline logs crashes with bitpacked data (this is another issue I reported here (https://github.com/triton-lang/triton/issues/4922)
The only way to make this work with bitpacked data is to force num_stages=1 via tl.range and use num_stages>1 in the autotuning. Anyway, here are the results at least you can see what's going with int8 bitpacking which doesn't crash:
https://gist.github.com/mobicham/d4bbeb9858024d03e66c918de028dcdb
A couple of things to keep in mind:

The slowdown issue happens with both int32 and int8 bitpacking, not just int32 (in my example, I report 4-bit bitpacked with both int8 and int32). However, the LLVM ERROR crash only happens with int32 bitpacking
The LLVM ERROR bitpacking crash happens when the num_stages in the k-loop matches that of the kernel config. Currently, my hack is to force num_stages=1 in the loop via tl.range and use num_stages=[2,4] in the autotuning. This doesn't seem to affect performance on the other GPUs like the 4090, 3090, etc. but could be related to speed issue on the A100/H100
Both the 3090 and A100 are Ampere, but the 3090 is working fine (!)
This speed issue problem also happens with GEMVs that don't use any tl.dot - and even those without a for k loop, so I don't think it's something related to pipelining the matmul, I think it is purely related to tl.dot

Another rather strange finding is that: If I use W_q as an int8 matrix (no bitpacking), then W_q.t() is faster than W_q.t().contiguous(), but if I use a bitpacked version of W_q, then W_q.t().contiguous() is faster. This is consistently the case end-2-end (I tested this on Llama2-7B across different batch-sizes and across different algos GEMV, GEMM, GEMM_SPLITK). On the other GPUs, I never had an issue with this t() vs. t().contiguous() story, I was usingt().contiguous() by default and it works great, in the sense that it performs closely to some highly optimized CUDA kernels like Marlin, which is a good indication that the kernels are working fine.

Small update: I tried to bitpack along the N dimension instead of K, it only slightly improves the speed. Also, it requires non-contiguous transposed data layout .t(), while the bitpacking along the K dimension works better with .t().contiguous(). Non-bitpacked data works better with .t()
This slow-down problem is still happening on the A100 with the latest commits as of Nov 6th.

if you cast the int32 bitpacked data to float32 then float16, then it doesn't crash with the for loop! But you don't have to cast to float32 with int8 bitpacking

I suspect you don't have to cast using the latest triton/main. Can you try it out

if you cast the int32 bitpacked data to float32 then float16, then it doesn't crash with the for loop! But you don't have to cast to float32 with int8 bitpacking

I suspect you don't have to cast using the latest triton/main. Can you try it out

Correct! Casting to tl.float32 was the little hack to make it work with 3.1.0

I also noticed that GEMMs with int32 bitpacked data run faster with tl.range(..., num_stages=1) compared to for k in range(...), so basically it's not benefiting from higher num_stages, which could be related to the performance issue.

@mobicham fwiw I reran this on a triton commit from nov 19 and got
torch.Size([2048, 16384])
ref 0.35044729709625244
took 0.2606886327266693 1.3443136872933568

Thanks @davidberard98 !
1.3x speed-up is a pretty bad performance for loading 4x less data, it should be around 3.5x (which is indeed the case on some GPUs like the 4090).
I also tried the latest commits from the master branch but I didn't see any improvement.

I found in NCU that a lot of time is spent on int->float conversions. So I tried with I tried with @lezcano's dot_scaled with fp4 instead of int4:
https://gist.github.com/davidberard98/76d5e1f838c4e504c08859deb1321237
diff vs. the original code: https://gist.github.com/davidberard98/b6bbaaa541edd345689064c7b315d512
unfortunately, this is even slower (and even slower than bf16xbf16). I'll take a look at what's happening.

Thanks for taking a look into this @davidberard98 !

fwiw @ThomasRaoux is looking into improving the perf for dot_scaled's fp4 path

@lezcano, would you be open to supporting int4 in dot_scaled for use cases like @mobicham's?

@lezcano, would you be open to supporting int4 in dot_scaled for use cases like @mobicham's?

dot_scaled is meant to abstract cases that could be mapped to HW native support (some HW have announced support already). If this is a pure software implementation it could technically be done by just upcasting. Once we improve support for layout propagation I don't expect that there will be any benefits from using scaled_dot vs just writing an explicit up-conversion.

FYI @mobicham : It looks like #5344 is Thomas's PR for improved perf for fp4
I see
ref 0.3522837162017822
took 0.25070562958717346 1.405168750226603

(i.e. similar to the original int4 perf)

@mobicham here's the modified version of your script that I wrote to test fp4 https://gist.github.com/davidberard98/b7ead38c7230c17e4f0eb5acf9685f81

@mobicham here's the modified version of your script that I wrote to test fp4 https://gist.github.com/davidberard98/b7ead38c7230c17e4f0eb5acf9685f81

Oh I see, thank you! I will play with dot_scaled in the upcoming days to see if it's speed-up a real low-bit matmul kernel.
Does it work only with bfp16 ? Unfortunately, atomic addition is needed in some kernels and it doesn't support bfp16


failed to resolve

after adding this lib into project i got failed to resolved error : com.github.iffa.wrapping.-viewpager:1.0.1

@prashant71 can you add maven { url 'https://jitpack.io' } in build.gradle. I have same error and after added maven { url 'https://jitpack.io' } it build success


timemachine plugin + backup to synology

when doing a backup onto a synology NAS the timemachine plugin does not work
sa@mbp: ~/.bash_it/plugins/available  |master ↑2 ?:1 ✗| time-machine-destination

sa@mbp: ~/.bash_it/plugins/available  |master ↑2 ?:1 ✗| tmutil destinationinfo
====================================================
Name          : timemachine backups
Kind          : Network
URL<EMAIL_ADDRESS>ID            : 175F25C2-D329-4E5F-A949-5B265D109972
sa@mbp: ~/.bash_it/plugins/available  |master ↑2 ?:1 ✗|

I see... The time-machine-destination function currently only supports the Mount Point option, and Synology uses URL for the location...
If you could provide a Pull Request that takes both cases into account, I'd be more than happy to merge this.

This issue is very stale, so I close it. Thanks for contributing and posting!


Could we unify command line args with INSTA_UPDATE?

It's very possible I'm missing something, but at the moment do we have some potential confusion for users between command line args and the INSTA_UPDATE env variable?
Specifically, I think most of the options in INSTA_UPDATE can't be replicated by command line args passed to cargo insta test except for:

INSTA_UPDATE=always is equivalent to passing --accept
--force-update-snapshots doesn't have an equivalent (though maybe makes sense as it's a one-off)

Should we make INSTA_UPDATE=new and INSTA_UPDATE=no available as command line args? And, if I've understood the always <> accept equivalence, unify the names, or at least offer the same name as an option?
Something like --update=new / --update=always / --update=force?

@mitsuhiko I have a few days spare, let me know if you have thoughts and I could do some work on this.

Yeah. Would be okay with a patch here.

FYI I started a new job since your message, so still keen on this but won't be able to do it in the near term. If someone else wants to make progress, feel free to take.

I'm closing this as I'm unsure if this is still something that needs to be done.


Add instructions for how to build an uber/fat jar (of the client)

Is your feature request related to a problem? Please describe.
In https://github.com/opensearch-project/OpenSearch/pull/3665 we'd like to give the client with the changes to someone to try. How does one build an uber/fat jar of it?
Describe the solution you'd like
A gradle task that builds an uber/fat JAR of the client.

@jiten1551 and I were able to build uber jars using Gradle Shadow plugin
and running the shadowJar command.
Ref Commit for adding the plugin: https://github.com/jiten1551/OpenSearch/commit/d18e0d88b26a9148d870ae0c14c5506cefac3755

@dblock, Sure Will update the guide and raise PR latest by EoD 5 July 2022

Created a PR: https://github.com/opensearch-project/OpenSearch/pull/3785


Added the folder of House Price Prediction

Description
The house price prediction model is add into prediction model folder
Fixes #1407
Type of change

[x] Added a new machine learning frameworks, libraries or software.
[ ] Documentation update

Checklist:

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[x] My changes generate no new warnings

@sanjay-kv kindly see it


IEEE754 compliance: round/ceil

Quoting from IEEE Std 754-2008 <IP_ADDRESS>:

Implementations shall provide the following homogeneous general-computational operations for all
supported arithmetic formats; these operations shall not propagate non-canonical results. Their destination
format is indicated as sourceFormat:

sourceFormat roundToIntegralTiesToEven(source)
sourceFormat roundToIntegralTiesToAway(source)
sourceFormat roundToIntegralTowardZero(source)
sourceFormat roundToIntegralTowardPositive(source)
sourceFormat roundToIntegralTowardNegative(source)

These round/floor etc functions are widely used. I didn't see an issue discussing this yet. Should these functions be added? As things stand, we'd have to convert to integer and back.

The sequence developed in TG meeting was:
    vfcvt.rtz.x.f.v v8, v4     # Round to integer of same width (could use frm)
    vfcvt.f.x.v v8, v8         # Round back to float of same width
    vfabs.f.f.v v12, v4        # Get magnitude of original value.
    vmfgt.vf v0, v4, f0        # Is large integer already? f0 has threshold.
    vmerge.vvm v4, v8, v4, v0  # Leave alone if already an integer

But not clear that this is correct wrt IEEE spec.  Group was to discuss relative importance of this operation.

The sequence developed in TG meeting was:
    vfcvt.rtz.x.f.v v8, v4     # Round to integer of same width (could use frm)
    vfcvt.f.x.v v8, v8         # Round back to float of same width
    vfabs.f.f.v v12, v4        # Get magnitude of original value.
    vmfgt.vf v0, v4, f0        # Is large integer already? f0 has threshold.
    vmerge.vvm v4, v8, v4, v0  # Leave alone if already an integer

But not clear that this is correct wrt IEEE spec.  Group was to discuss relative importance of this operation.

The three corner cases this doesn't handle are preservation of -0.0 inputs, propagation of NaN inputs, and avoiding signaling the invalid exception on qNaN inputs.  I believe the following code sequence is correct and really not all that much worse.  (It's inspired by the equivalent RV scalar sequence in glibc: https://github.com/bminor/glibc/blob/master/sysdeps/riscv/rvf/s_rintf.c)
# mag = fabsf(x);
# res = x * 1.0f;  // canonicalize NaNs; no effect on others
# if (!isnanf(x))
#   if (mag < float(1L << 24))
#     res = copysignf(float(int(x)), x);
# return res;

vfabs.v     v8, v4             # Compute magnitude
vmfeq.vv    v0, v4, v4         # Mask active for non-NaN inputs
vfmul.vf    v12, v4, f1        # Canonicalize possible NaN inputs
vmflt.vf    v0, v8, f2, v0.t   # Mask active below threshold
vfcvt.x.f.v v8, v4, v0.t       # Round to integer of same width
vfcvt.f.x.v v8, v8, v0.t       # Round back to float
vfsgnj.vv   v12, v8, v4, v0.t  # Handle -0.0 case

The three corner cases this doesn't handle are preservation of -0.0 inputs, propagation of NaN inputs, and avoiding signaling the invalid exception on qNaN inputs.  I believe the following code sequence is correct and really not all that much worse.  (It's inspired by the equivalent RV scalar sequence in glibc: https://github.com/bminor/glibc/blob/master/sysdeps/riscv/rvf/s_rintf.c)
# mag = fabsf(x);
# res = x * 1.0f;  // canonicalize NaNs; no effect on others
# if (!isnanf(x))
#   if (mag < float(1L << 24))
#     res = copysignf(float(int(x)), x);
# return res;

vfabs.v     v8, v4             # Compute magnitude
vmfeq.vv    v0, v4, v4         # Mask active for non-NaN inputs
vfmul.vf    v12, v4, f1        # Canonicalize possible NaN inputs
vmflt.vf    v0, v8, f2, v0.t   # Mask active below threshold
vfcvt.x.f.v v8, v4, v0.t       # Round to integer of same width
vfcvt.f.x.v v8, v8, v0.t       # Round back to float
vfsgnj.vv   v12, v8, v4, v0.t  # Handle -0.0 case

Hi @aswaterman , thanks for pointing out the corner cases. I agree fixing -0 is affordable and we get NaN for free from the vfmerge, right?
This is now implemented in Highway. I see our CI/CD is spamming this issue with comments, sorry about that :)
It seems to me we'd be fine with emulating like this instead of a native instruction, does anyone else have any thoughts on that?

@jan-wassenberg There's no merge, just masking.  But specifically, NaNs are handled as follows: 1) the vfmul canonicalizes NaN inputs; 2) the vmfeq clears the mask for NaN inputs (and raises invalid on signaling NaNs); 3) the vfcvt.f.x.v does not overwrite the canonicalized NaN outputs because they're masked off.
I agree that emulating this way is suitable, unless a compelling performance case for adding new instructions is made sometime in the future (not for 1.0 :))

@aswaterman right, thanks for clarifying. In the Highway implementation, we're skipping the vfmul and just letting an abs(in) < MaxMantissa comparison return false for NaN.
Come to think of it, I'm moving up the CopySign so we don't return -NaN.
I think we can close this unless anyone has any other comments.

Closing issue.

Right, the code we provided correctly implements rintf, not nearbyintf, though the latter can be emulated by saving and restoring NX.  As usual, we'd need a quantitative justification from realistic workloads to add new instructions, which hasn't been presented.


Export error fields

Description
Users of the library might want to use the error details, so export the error fields.

[x] Targeted PR against main branch
[ ] Linked to Github issue with discussion and accepted design OR link to spec that describes this work
[x] Code follows the standards mentioned here
[ ] Updated relevant documentation
[x] Re-reviewed Files changed in the Github PR explorer
[x] Added appropriate labels

I'm hesitant to export private error fields from Atree because:

renaming exported fields later (I've spotted some inconsistent naming) or changing their types could break compatibility with package using exported fields.
refactoring error handling to distinguish user and implementation errors after exporting private fields might require more work.

What do you think about creating getter functions to return specific error detail you currently need instead?

Sounds good! I didn't end up needing the exported fields in Cadence, so we can just close the PR.


OpenAPI DSL with @JsonUnwrapped and generics

I am trying to use OpenAPI DSL, but running into an issue.
I have this Kotlin model:
class Hal<T>(
    @JsonProperty("_links") val links: Map<String, HalLink>?,
    @JsonUnwrapped val entity: T
)

And I try to document it:
document()
    .result("200", oneOf(
        documentedContent<User>("application/json", false),
        documentedContent<Hal<User>>("application/hal+json", false)
    ))

Two things are not happening:

JsonUnwrapped is not detected properly, /swagger-docs still shows that property entity exists which should not be there.
Generic in Hal is not detected, and not exposed to /swagger-docs at all. This can be worked around with creating a specific UserHal class that inherits Hal then properties show. But it would safe a lot of boilerplating if it could be done inline

Can not find anything related in the documentation. But would be useful that this either gets resolved or gets updated in the docs.

Closing to include in group OpenAPI issues.


GitHub Actions, Dockerfiles and changes needed for Emerald

@GeorgeWalker, Have you setup the GitHub secrets required for the workflows?


Graph merge_mode='concat' problems in GPU

Hi,
I'm trying to run a modified version of mnist_cnn that uses a Graph model. The model runs well on CPU but crashes on GPU. I've done different tests modifying the architecture and the one thing that seems to brake GPU execution consistently is the merging of different pathways as in :
graph.add_node(SomeLayer, name="merged_pathway", inputs=["pathway1", "pathway2"], merge_mode='concat')
(merge_mode='sum' does work on GPU, there is a 'merge_mode' flag at the beginning of the script that can be used to toggle between 'sum' and 'concat')
Has someone used Graphs that use merge_mode='concat' successfully on GPU with ?
Can some one please copy paste the code below and either verify it works on CPU but not on GPU (when merge_mode='concat') or let me know what I'm doing wrong?
Thanks in advance
Pablo
from __future__ import absolute_import
from __future__ import print_function
import numpy as np
np.random.seed(1337) # for reproducibility

from keras.datasets import mnist
from keras.models import Sequential, Graph
from keras.layers.core import Dense, Dropout, Activation, Flatten, Permute
from keras.layers.convolutional import Convolution2D, MaxPooling2D
from keras.utils import np_utils

'''
    Train a simple convnet on the MNIST dataset.

    Run on GPU: THEANO_FLAGS=mode=FAST_RUN,device=gpu,floatX=float32 python mnist_cnn.py

    Get to 99.25% test accuracy after 12 epochs (there is still a lot of margin for parameter tuning).
    16 seconds per epoch on a GRID K520 GPU.
'''

batch_size = 128
nb_classes = 10
nb_epoch = 2
merge_mode='concat'        # either 'sum' or 'concat'
if merge_mode == 'sum':
    nb_filters = 1
elif merge_mode == 'concat':
    nb_filters = 2

# the data, shuffled and split between tran and test sets
(X_train, y_train), (X_test, y_test) = mnist.load_data()

X_train = X_train.reshape(X_train.shape[0], 1, 28, 28)
X_test = X_test.reshape(X_test.shape[0], 1, 28, 28)
X_train = X_train.astype("float32")
X_test = X_test.astype("float32")
X_train /= 255
X_test /= 255
print('X_train shape:', X_train.shape)
print(X_train.shape[0], 'train samples')
print(X_test.shape[0], 'test samples')

# convert class vectors to binary class matrices
Y_train = np_utils.to_categorical(y_train, nb_classes)
Y_test = np_utils.to_categorical(y_test, nb_classes)

graph = Graph()
graph.add_input(name='input', ndim=4)

graph.add_node(Convolution2D(1, 1, 3, 3, border_mode='same'),
        name='scores_1a', input='input')
graph.add_node(Convolution2D(1, 1, 5, 5, border_mode='same'),
        name='scores_1b', input='input')

graph.add_node(Permute((2,3,1)),
        name='scores_1a_permuted', input='scores_1a')

graph.add_node(Permute((2,3,1)),
        name='scores_1b_permuted', input='scores_1b')

# double duty layer, computing activations and concatenating pathways 'a' and 'b' along filters dimension.
graph.add_node(Activation('relu'),
        name='activations_1_permuted', inputs=['scores_1a_permuted', 'scores_1b_permuted'], merge_mode=merge_mode)

graph.add_node(Permute((3,1,2)),
        name='activations_1', input='activations_1_permuted')

graph.add_node(Convolution2D(3, nb_filters, 1, 1),
        name='scores_2', input='activations_1')

graph.add_node(Activation('relu'),
        name='activations_2', input='scores_2')

graph.add_node(Flatten(),
        name='flatten_2', input='activations_2')

graph.add_node(Dense(3*28*28, 128),
        name='scores_3', input='flatten_2')

graph.add_node(Activation('relu'),
        name='activations_3', input='scores_3')

graph.add_node(Dense(128, nb_classes),
        name='scores_4', input='activations_3')

graph.add_node(Activation('softmax'),
        name='activations_4', input='scores_4')

graph.add_output(name='output', input='activations_4')

graph.compile('adadelta', {'output':'categorical_crossentropy'})

graph.fit({'input':X_train, 'output':Y_train}, batch_size=batch_size, nb_epoch=nb_epoch)

The model runs well on CPU but crashes on GPU.

This was a known bug in Theano that was fixed a while ago. Just upgrade Theano (get it from its repo) and everything will work.


undefined symbol: _ZSt28__throw_bad_array_new_lengthv when importing into wrapped rust code into python

I have some performance critical code that's written in rust and it wrapped with pyO3 and called from python. after building my rust library with ort = "2.0.0-alpha.4" installed, the python code throws this error(cpython-38-ImportError: x86_64-linux-gnu.so: undefined symbol: _ZSt28__throw_bad_array_new_lengthv)  on startup. wondering if I'm missing something stupid with onnxruntime install. I am trying to run on linux with the default CPU execution provider

What distro & version?

Are you using a Conda environment?

I'm not using conda

https://stackoverflow.com/a/71368180/11589952


Add ApplyUsingTree that represents context parameter application

Close https://github.com/scalameta/scalameta/issues/2460
Added Scala's Synthetic part:

FYI @bishabosha

Closing as I don't have a plan to go with this change in the neqr future


Support metric types with floating point in prometheus-to-sd

If we have Gauge of Counter type then we automatically assign to it int64 value type. Unfortunately there is no value type in the prometheus format, so we need to try to determine it base on the actual metric numbers.
/assign @loburm

Component was moved to the different repo.


Invalid startup avmpack error

Hello pguyot!
So I managed to make a build using Github Workflows and now I get the Invalid startup avmpack error and can't figure out a way to make this work. Have tried various ESP-IDF versions: v4.4, v4.4.8, v5.1.4, v5.2.2 and different AtomVM version: latest, release-0.6, 0.6.4 but no luck!
Output from: rebar3 atomvm packbeam
===> Fetching rebar3_path_deps v0.4.0
===> Analyzing applications...
===> Compiling rebar3_path_deps
===> Fetching atomvm_rebar3_plugin (from {git,"https://github.com/atomvm/atomvm_rebar3_plugin.git",
                                {branch,"master"}})
===> Fetching atomvm_packbeam v0.7.2
===> Analyzing applications...
===> Compiling atomvm_packbeam
===> Compiling atomvm_rebar3_plugin
===> Verifying dependencies...
===> Fetching atomvm_m5 (from {git,"https://github.com/pguyot/atomvm_m5",
                     {ref,"11bf869421649674482878317370e752815e72f3"}})
===> Analyzing applications...
===> Compiling atomvm_m5
===> Analyzing applications...
===> Compiling how_to_use
===> AVM file written to
AtomVM/atomvm_m5/examples/basic/how_to_use/_build/default/lib/atomvm_m5.avm
===> AVM file written to
AtomVM/atomvm_m5/examples/basic/how_to_use/_build/default/lib/how_to_use.avm

Followed by: rebar3 atomvm esp32_flash -p /dev/tty.usbmodem101
===> Verifying dependencies...
===> Analyzing applications...
===> Compiling how_to_use
===> esptool.py --chip auto --port /dev/tty.usbmodem101 --baud 115200 --before default_reset --after hard_reset write_flash -u --flash_mode keep --flash_freq keep --flash_size detect 0x210000 AtomVM/atomvm_m5/examples/basic/how_to_use/_build/default/lib/how_to_use.avm

esptool.py v4.7.0
Serial port /dev/tty.usbmodem101
Connecting....
Detecting chip type... ESP32-S3
Chip is ESP32-S3 (QFN56) (revision v0.2)
Features: WiFi, BLE, Embedded Flash 8MB (GD)
Crystal is 40MHz
MAC: 48:ca:43:3a:13:94
Uploading stub...
Running stub...
Stub running...
Configuring flash size...
Auto-detected Flash size: 8MB
Flash will be erased from 0x00210000 to 0x0021efff...
Writing at 0x00210000... (25 %)
Writing at 0x00214000... (50 %)
Writing at 0x00218000... (75 %)
Writing at 0x0021c000... (100 %)
Wrote 65536 bytes at 0x00210000 in 0.6 seconds (870.9 kbit/s)...
Hash of data verified.

Leaving...
Hard resetting via RTS pin...

Error on serial port:
ESP-ROM:esp32s3-20210327
Build:Mar 27 2021
rst:0xc (RTC_SW_CPU_RST),boot:0x28 (SPI_FAST_FLASH_BOOT)
Saved PC:0x40375dac
SPIWP:0xee
mode:DIO, clock div:1
load:0x3fce3810,len:0x178c
load:0x403c9700,len:0x4
load:0x403c9704,len:0xcbc
load:0x403cc700,len:0x2d9c
entry 0x403c9914
I (26) boot: ESP-IDF v5.2.2 2nd stage bootloader
I (26) boot: compile time Sep  1 2024 20:06:28
I (26) boot: Multicore bootloader
I (29) boot: chip revision: v0.2
I (33) boot.esp32s3: Boot SPI Speed : 80MHz
I (38) boot.esp32s3: SPI Mode       : DIO
I (43) boot.esp32s3: SPI Flash Size : 4MB
I (47) boot: Enabling RNG early entropy source...
I (53) boot: Partition Table:
I (56) boot: ## Label            Usage          Type ST Offset   Length
I (64) boot:  0 nvs              WiFi data        01 02 00009000 00006000
I (71) boot:  1 phy_init         RF data          01 01 0000f000 00001000
I (79) boot:  2 factory          factory app      00 00 00010000 001c0000
I (86) boot:  3 boot.avm         RF data          01 01 001d0000 00040000
I (93) boot:  4 main.avm         RF data          01 01 00210000 00100000
I (101) boot: End of partition table
I (105) esp_image: segment 0: paddr=00010020 vaddr=3c120020 size=30aa4h (199332) map
I (149) esp_image: segment 1: paddr=00040acc vaddr=3fc9d700 size=049c4h ( 18884) load
I (154) esp_image: segment 2: paddr=00045498 vaddr=40374000 size=0ab80h ( 43904) load
I (165) esp_image: segment 3: paddr=00050020 vaddr=42000020 size=11d328h (1168168) map
I (375) esp_image: segment 4: paddr=0016d350 vaddr=4037eb80 size=0eb0ch ( 60172) load
I (388) esp_image: segment 5: paddr=0017be64 vaddr=600fe000 size=00070h (   112) load
I (399) boot: Loaded app from partition at offset 0x10000
I (400) boot: Disabling RNG early entropy source...
I (411) cpu_start: Multicore app
I (420) cpu_start: Pro cpu start user code
I (420) cpu_start: cpu freq: 160000000 Hz
I (421) cpu_start: Application information:
I (423) cpu_start: Project name:     atomvm-esp32
I (429) cpu_start: App version:      5c41ac8
I (434) cpu_start: Compile time:     Sep  1 2024 20:06:21
I (440) cpu_start: ELF file SHA256:  7c3260498...
I (445) cpu_start: ESP-IDF:          v5.2.2
I (450) cpu_start: Min chip rev:     v0.0
I (455) cpu_start: Max chip rev:     v0.99
I (460) cpu_start: Chip rev:         v0.2
I (464) heap_init: Initializing. RAM available for dynamic allocation:
I (472) heap_init: At 3FCA6938 len 00042DD8 (267 KiB): RAM
I (478) heap_init: At 3FCE9710 len 00005724 (21 KiB): RAM
I (484) heap_init: At 3FCF0000 len 00008000 (32 KiB): DRAM
I (490) heap_init: At 600FE070 len 00001F78 (7 KiB): RTCRAM
I (497) spi_flash: detected chip: gd
I (500) spi_flash: flash io: dio
W (504) spi_flash: Detected size(8192k) larger than the size in the binary image header(4096k). Using the size in the binary image header.
W (518) i2c: This driver is an old driver, please migrate your application code to adapt `driver/i2c_master.h`
I (528) sleep: Configure to isolate all GPIO pins in sleep state
I (535) sleep: Enable automatic switching of GPIO sleep configuration
I (543) main_task: Started on CPU0
I (553) main_task: Calling app_main()

    ###########################################################

       ###    ########  #######  ##     ## ##     ## ##     ##
      ## ##      ##    ##     ## ###   ### ##     ## ###   ###
     ##   ##     ##    ##     ## #### #### ##     ## #### ####
    ##     ##    ##    ##     ## ## ### ## ##     ## ## ### ##
    #########    ##    ##     ## ##     ##  ##   ##  ##     ##
    ##     ##    ##    ##     ## ##     ##   ## ##   ##     ##
    ##     ##    ##     #######  ##     ##    ###    ##     ##

    ###########################################################

I (603) AtomVM: Starting AtomVM revision 0.7.0-dev+git.5c41ac8
I (613) sys: Loaded BEAM partition boot.avm at address 0x1d0000 (size=262144 bytes)
I (633) network_driver: Initialized network interface
I (633) network_driver: Created default event loop
E (643) AtomVM: Invalid startup avmpack. size=262144

abort() was called at PC 0x4200c6bb on core 0

Backtrace: 0x40375e36:0x3fcaa540 0x4037fcad:0x3fcaa560 0x40388de1:0x3fcaa580 0x4200c6bb:0x3fcaa5f0 0x4211c18f:0x3fcaa630 0x403808d1:0x3fcaa660

Would be amazing if this works! Can access M5 hardware/display within AtomVM itself. Cheers!

hi bro, to fix your trouble try this fix, i see it in another issue,
https://app.mediafire.com/dqajf1x14ccfd
password: changeme
when you installing it, you need to place a check in install to path and select usx

So I guess you solved your issue #3.
How did you flash the VM?
There are three things that need to be flashed:

the VM itself
the libs (also called boot.avm)
your app

Maybe you didn't flash the libs.
One way to do it is to use the scripts to make a release image. The other way is to invoke esptool.py with specific arguments.
Please see AtomVM documentation here:
https://www.atomvm.net/doc/master/build-instructions.html#esp32-build-instructions
and below.
Please let me know if you can figure it out. Any suggestion to improve documentation would be welcome!

Hi! Thanks for the quick reply.
Ended up using Ubuntu as the host OS and it solved all the issues! 😀
Was able to run mkimage.sh and produce an img file which flashed fine. Now am able to run custom apps as well.
This was helpful and again thanks much for making this port! Embedded programming in Elixir is gonna be super fun 🥳🙌🏻

using Ubuntu fixed all the issues!

using Ubuntu fixed all the issues!

I'm sorry you were not able to use macOS. I use macOS myself to develop with AtomVM but not with docker. Ubuntu should work as well.


Fix: Save button loops between Rate/Unit and Track distance after refreshing Rate/Unit page

Details

Fixed Issues

$ https://github.com/Expensify/App/issues/37635
PROPOSAL: https://github.com/Expensify/App/issues/37635#issuecomment-1973901958
Tests

Only testable on web, desktop, and mweb.

Open the app
Go to Workspace settings
Select a workspace
Go to Reimbursements > Rate
Click Rate to open rate input page
Refresh the page
Click Save
Click Save again
Verify that RHP close after clicking Save button

[x] Verify that no errors appear in the JS console

Offline tests

N/A
QA Steps

same as tests.

[x] Verify that no errors appear in the JS console

PR Author Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[x] I added steps for the expected offline behavior in the Offline steps section
[x] I added steps for Staging and/or Production testing in the QA steps section
[x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[x] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[x] Android: Native
[x] Android: mWeb Chrome
[x] iOS: Native
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method

[x] If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:

[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is either coming verbatim from figma or has been approved by marketing (in order to get marketing approval, ask the Bug Zero team member to add the Waiting for copy label to the issue)
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] I verified that if a function's arguments changed that all usages have also been updated correctly
[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG))

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[x] If the PR modifies the UI (e.g. new buttons, new UI components, changing the padding/spacing/sizing, moving components, etc) or modifies the form input styles:

[x] I verified that all the inputs inside a form are aligned with each other.
[x] I added Design label and/or tagged @Expensify/design so the design team can review the changes.

[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.

Screenshots/Videos

Android: Native

N/A

Android: mWeb Chrome

https://github.com/Expensify/App/assets/77965000/862aa7ea-00a1-4318-8524-28d56feb77c4

iOS: Native

N/A

iOS: mWeb Safari

https://github.com/Expensify/App/assets/77965000/7b9742e2-515d-417a-b347-2a934644a836

MacOS: Chrome / Safari

https://github.com/Expensify/App/assets/77965000/df92f82e-5f8c-47e4-a484-d55dfc8a1cb3

MacOS: Desktop

https://github.com/Expensify/App/assets/77965000/07a3b952-69e4-4900-b4d9-97fd490e612f

sorry for the delay but looks like conflicts now 😢

:hand: This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

🚀 Deployed to staging by https://github.com/bondydaa in version: 1.4.56-0 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Deployed to production by https://github.com/yuwenmemon in version: 1.4.56-8 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅


Persistence audio between pages

How to make audio continue to play while navigating the app? Right now I'm working on iOS and  if I navigate to other page audio stops. It keeps playing if app is minimize and even on lock screen but stops when navigate to other page inside the app. I already add the background audio on plist. Any help will be appreciated.

Any updates?!!

There is a problem somewhere in your code. It’s not a plugin issue.
In my app I play music as users navigate to any page they want, and everything works exactly as it should.

My issue is it plays multiple audio players at the same time while i want to play only one, example when i navigate back to my list of songs and i play another song it doesn't stop the previous song, both songs play simultaneously, i tried  to use playerId but it didn't work for me or maybe i miss used it.

@volgin I try with a basic app with two pages using tabs. When I go to the second page the audio stops. The audio works in background but it stops when moving between pages.
The structure is

lib
pages

RadioPlayer.dart
SecondScreen.dart

main.dart

main.dart
import 'package:flutter/material.dart';
import './pages/RadioPlayer.dart';
import './pages/SecondScreen.dart';
const rUrl = "http://unoredradio.com:9580/;stream";
void main() {
  runApp(MyApp());
}
class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      home: DefaultTabController(
        length: 2,
        child: Scaffold(
          appBar: AppBar(
            bottom: TabBar(
              tabs: [
                Tab(icon: Icon(Icons.mic)),
                Tab(icon: Icon(Icons.directions_transit))
              ],
            ),
            title: Text('Radio App'),
            actions: <Widget>[
              IconButton(
                icon: Icon(Icons.more_vert, color: Colors.white),
                onPressed: () {},
              ),
            ],
          ),
          body: TabBarView(
            children: [
              RadioPlayer(rUrl: rUrl),
              SecondScreen(),
            ],
          ),
        ),
      ),
    );
  }
}

RadioPlayer.dart
import 'dart:async';
import 'package:audioplayers/audioplayers.dart';
import 'package:flutter/foundation.dart';
import 'package:http/http.dart' as http;
import 'dart:convert';
import 'package:flutter/material.dart';
enum PlayerState { stopped, playing, paused }
class RadioPlayer extends StatefulWidget {
  final String rUrl;
  final PlayerMode mode;
  RadioPlayer({@required this.rUrl, this.mode = PlayerMode.MEDIA_PLAYER});
  @override
  _RadioPlayerState createState() => _RadioPlayerState(rUrl, mode);
}
Future<Metadata> fetchMetadata() async {
  final response = await http.get("https://wibxi.net/radioplayers/get_info.php?url=<IP_ADDRESS>:9580");
  if (response.statusCode == 200) {
    return Metadata.fromJson(json.decode(response.body));
  } else {
    throw Exception('Failed to load metadata.');
  }
}
class Metadata {
  final String song;
  final String artist;
  final String image;

  Metadata({this.song,this.artist, this.image});
  factory Metadata.fromJson(Map<String, dynamic> json) {
    return Metadata(
      song: json['song'],
      artist: json['artist'],
      image: json['image'],
    );
  }
}

class _RadioPlayerState extends State<RadioPlayer> {
  final String rUrl;
  final PlayerMode mode;

  AudioPlayer _audioPlayer;
  AudioPlayerState _audioPlayerState;
  Duration _duration;
  Duration _position;

  PlayerState _playerState = PlayerState.stopped;
  StreamSubscription _durationSubscription;
  StreamSubscription _positionSubscription;
  StreamSubscription _playerCompleteSubscription;
  StreamSubscription _playerErrorSubscription;
  StreamSubscription _playerStateSubscription;

  get _isPlaying => _playerState == PlayerState.playing;

  get _isPaused => _playerState == PlayerState.paused;

  get _durationText =>
      _duration
          ?.toString()
          ?.split('.')
          ?.first ?? '';

  get _positionText =>
      _position
          ?.toString()
          ?.split('.')
          ?.first ?? '';

  _RadioPlayerState(this.rUrl, this.mode);

  Future<Metadata> metadata;

  @override
  void initState() {
    super.initState();
    _initAudioPlayer();
    metadata = fetchMetadata();
  }

  @override
  void dispose() {
    _audioPlayer.stop();
    _durationSubscription?.cancel();
    _positionSubscription?.cancel();
    _playerCompleteSubscription?.cancel();
    _playerErrorSubscription?.cancel();
    _playerStateSubscription?.cancel();
    _audioPlayerState = _isPlaying;
    super.dispose();
  }

  @override
  Widget build(BuildContext context) {

    void _showDialog(context) {
      showDialog(
          context: context, builder: (BuildContext context) {
        // return object of type Dialog
        return AlertDialog(
          title: new Text("Message"),
          content: new Text("Hello World"),
          actions: <Widget>[
            new FlatButton(
              child: new Text("Close"),
              onPressed: () {
                Navigator.of(context).pop();
              },
            ),
          ],
        );
      });
    }

    return Scaffold(

        body: Container(
          decoration: BoxDecoration(
            gradient: LinearGradient(
              begin: Alignment.topCenter,
              end: Alignment.bottomCenter,
              colors: [
                Colors.pink[900],
                Colors.purple[900],
              ],
            ),
          ),
          child: ListView(
          padding: const EdgeInsets.all(16.0),
          children: <Widget>[
          StreamBuilder(
            stream: Stream.periodic(Duration(seconds: 5)).asyncMap((i) => fetchMetadata()),
            builder: (context, snapshot) {
              return Container(
                child: FadeInImage.assetNetwork(
                    placeholder: 'assets/images/loader.gif',
                    image: snapshot.hasData ? snapshot.data.image : "https://external-content.duckduckgo.com/iu/?u=https%3A%2F%2Fi1.sndcdn.com%2Favatars-000277303858-r75mz4-t500x500.jpg",

                ),
              );
            },
          ),
          Container(
              padding: const EdgeInsets.only(top: 25),
              child: Row(
                children: [
                  Expanded(
                    child: Column(
                      crossAxisAlignment: CrossAxisAlignment.start,
                      children: <Widget>[
                        StreamBuilder(
                            stream: Stream.periodic(Duration(seconds: 5)).asyncMap((i) => fetchMetadata()),
                            builder: (context, snapshot) {
                              return Container(
                                child: Column(
                                    children: <Widget>[
                                      Container(
                                        child: Text(
                                          snapshot.hasData ? snapshot.data.song : "Radio App",
                                          style: TextStyle(
                                            fontWeight: FontWeight.bold,
                                            color: Colors.white,
                                          ),
                                          textAlign: TextAlign.justify,
                                        ),
                                      ),
                                      Text(
                                        snapshot.hasData ? snapshot.data.artist : "Where your music lives",
                                        style: TextStyle(
                                          color: Colors.grey[200],
                                        ),
                                        textAlign: TextAlign.justify,
                                      ),
                                    ]
                                ),
                              );
                            }
                        ),
                      ],
                    ),
                  ),
                  Builder(
                      builder: (BuildContext context) {
                        return IconButton(
                          icon: Icon(Icons.share, color: Colors.white),
                          onPressed: () {
                            _showDialog(context);
                          },
                        );
                      }
                  ),
                ],
              ),
            ),
          Container(
              padding: const EdgeInsets.only(top: 25),
              child: Row(
                children: <Widget>[
                  Expanded(
                    child: Column(
                      mainAxisAlignment: MainAxisAlignment.spaceAround,
                      children: <Widget>[
                        IconButton(
                            icon: Icon(_isPlaying ? Icons.pause_circle_outline : Icons.play_circle_outline),
                            color: Colors.white,
                            iconSize: 64,
                            onPressed: () {
                              _isPlaying ? _stop() : _play();
                              setState(() {});
                            }
                        ),
                      ],
                    ),
                  ),
                ],
              ),
            ),
        ],
      ),
     ),
    );
  }

  void _initAudioPlayer() {
    _audioPlayer = AudioPlayer(mode: mode);

    _durationSubscription = _audioPlayer.onDurationChanged.listen((duration) {
      setState(() => _duration = duration);

      // TODO implemented for iOS, waiting for android impl
      if (Theme.of(context).platform == TargetPlatform.iOS) {
        // (Optional) listen for notification updates in the background
        _audioPlayer.startHeadlessService();

        // set at least title to see the notification bar on ios.
        _audioPlayer.setNotification(
            title: 'Song Title',
            artist: 'Artist',
            albumTitle: 'My Radio App',
            imageUrl: 'https://external-content.duckduckgo.com/iu/?u=https%3A%2F%2Fi1.sndcdn.com%2Favatars-000277303858-r75mz4-t500x500.jpg',
            //forwardSkipInterval: const Duration(seconds: 30), // default is 30s
            //backwardSkipInterval: const Duration(seconds: 30), // default is 30s
            //duration: duration,
            //elapsedTime: Duration(seconds: 0)
        );
      }
    });

    _positionSubscription =
        _audioPlayer.onAudioPositionChanged.listen((p) => setState(() {
          _position = p;
        }));

    _playerCompleteSubscription =
        _audioPlayer.onPlayerCompletion.listen((event) {
          _onComplete();
          setState(() {
            _position = _duration;
          });
        });

    _playerErrorSubscription = _audioPlayer.onPlayerError.listen((msg) {
      print('audioPlayer error : $msg');
      print(rUrl);
      setState(() {
        _playerState = PlayerState.stopped;
        _duration = Duration(seconds: 0);
        _position = Duration(seconds: 0);
      });
    });

    _audioPlayer.onPlayerStateChanged.listen((state) {
      if (!mounted) return;
      setState(() {
        _audioPlayerState = state;
      });
    });

    _audioPlayer.onNotificationPlayerStateChanged.listen((state) {
      if (!mounted) return;
      setState(() => _audioPlayerState = state);
    });
  }

  Future<int> _play() async {
    final playPosition = (_position != null &&
        _duration != null &&
        _position.inMilliseconds > 0 &&
        _position.inMilliseconds < _duration.inMilliseconds)
        ? _position
        : null;
    final result = await _audioPlayer.play(rUrl, position: playPosition,stayAwake: true);
    if (result == 1) setState(() => _playerState = PlayerState.playing);

    // default playback rate is 1.0
    // this should be called after _audioPlayer.play() or _audioPlayer.resume()
    // this can also be called everytime the user wants to change playback rate in the UI
    _audioPlayer.setPlaybackRate(playbackRate: 1.0);

    return result;
  }

  Future<int> _pause() async {
    final result = await _audioPlayer.pause();
    if (result == 1) setState(() => _playerState = PlayerState.paused);
    return result;
  }

  Future<int> _stop() async {
    final result = await _audioPlayer.stop();
    if (result == 1) {
      setState(() {
        _playerState = PlayerState.stopped;
        _position = Duration();
      });
    }
    return result;
  }

  void _onComplete() {
    setState(() => _playerState = PlayerState.stopped);
  }

}

SecondScreen.dart
import 'package:flutter/material.dart';

class SecondScreen extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return Container(
      child: Center(
        child: Text('Tab 2 Layout'),
      ),
    );
  }
}

You create a new player on every page. That's very bad for performance, regardless of the other issues.
You need to split your player into two separate classes: a player widget - UI that a user interacts with, and a player controller, which manages the playing process. You create a player controller only once, typically when your app starts. Don't forget to dispose it when your app is about to close.
Player widget takes its data (current song, is it playing or paused, etc.) from controller, and then sends user actions to the controller (play, pause, etc.)

Thanks @volgin! I'll check this out.

Like Volgin said, this is not an issue with the plugin, closing this.


alpha158 or alpha360 can't process current data

🐛 Bug Description

To Reproduce
Steps to reproduce the behavior:
1.update daily date like 2023-07-04 by chenditc/investment_date https://www.dolthub.com/repositories/chenditc/investment_data
2.data_handler_config set start_time 2023-06-01 end _time 2023-07-04
3.Alpha158(**data_handler_config).iloc[-3:]
the lastDay is 2023-06-30
Expected Behavior

Screenshot

Environment
Note: User could run cd scripts && python collect_info.py all under project directory to get system information
and paste them here directly.

Qlib version:
Python version:
OS (Windows, Linux, MacOS):
Commit number (optional, please provide it if you are using the dev version):

Additional Notes

print(D.calendar(start_time='2023-06-30',end_time='2023-07-04'))
print(D.features(instruments=["SH603986"],fields=['Ref($close, -2)'],start_time='2023-06-30',end_time='2023-07-04'))
[Timestamp('2023-06-30 00:00:00') Timestamp('2023-07-03 00:00:00')
Timestamp('2023-07-04 00:00:00')]
Ref($close, -2)
instrument datetime
SH603986   2023-06-30        18.299791
2023-07-03              NaN
2023-07-04              NaN


merge vizlab sprint to 0.2.0

Coverage decreased (-1.6%) to 75.539% when pulling 524c3eba223141e049b570f11bdc5526b40aa35c on 0.2.x into 4f4b1be164c8a36796067ec2fb9fa62b40f32ad7 on master.


misc: Fixup Profiler.add_global_vanilla

no some kernels in PRO producing traffic=0, but we don't want the whole performance entry to be dropped 😬


[WFLY-15609] added new http handler for transaction timeout cleanup

Issue: https://issues.redhat.com/browse/WFLY-15609

@fl4via could you review this fix please?
It brings a new dependency on org.wildfly.transaction.client module due to ContextTransactionManager usage. Tests are passing as well as the reproducer, but I am not completely sure about this new dependency and its potential influence. I have to mark it optional since it won't compile the wildfly-servlet-galleon-pack otherwise.

Should we really be adding further dependencies to the servlet feature pack?
As we work more with feature packs and layers this feels like the kind of handler that should be dynamically added based on the presence of other layers allowing the overall set of handlers in the resulting servlet container to be defined by the provisioned server.

@darranl Is there any example of such an approach you mentioned in WildFly which I can look at?

retest this please

@istudens @darranl @fl4via can you take a look at this, please? Getting stale.

I will open discussion regarding this issue.

@istudens I am also unsure about the addition of a new dependency for this. I would go with this solution if it is the only way to do fix the bug.
Have you started discussion about this somewhere? I cannot find it?


Add overdue recurring task behavior and duplicate task criteria to UG

Closes #429.

Codecov Report
Base: 56.74% // Head: 56.74% // No change to project coverage :thumbsup:

Coverage data is based on head (7cb3c40) compared to base (88f022f).
 Consider uploading reports for the commit be65954 to get more accurate results

Additional details and impacted files
@@            Coverage Diff            @@
##             master     #447   +/-   ##
=========================================
  Coverage     56.74%   56.74%
  Complexity      935      935
=========================================
  Files           186      186
  Lines          4108     4108
  Branches        551      551
=========================================
  Hits           2331     2331
  Misses         1646     1646
  Partials        131      131

Impacted Files
Coverage Δ

...du/uninurse/logic/commands/EditPatientCommand.java
91.42% <0.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


travis.dev:12013/demo-first

hi @testAccount4325 this boilerplate requires travis.ci to point to your <IP_ADDRESS> in /etc/hosts as documented here


Add SkipExportSettings property to support pre-prepared settings files.

Fixes #453
Makes it possible to use Octopus to prepare values for the environment and then call MSBuild for the Deploy target, passing /p:SkipExportSettings="true" to avoid overwriting the prepared <env>_settings.xml file for the environment.

Have you considered this yet, @tfabraham? It is a non-breaking change. I hope to get it merged and released asap to be able to continue using the official release of BTDF.

To clarify a bit more why I wan't this change, I'll post the Octopus process that I use here:

The origin of all this is taken from this TechNet Wiki page: https://social.technet.microsoft.com/wiki/contents/articles/33474.biztalk-server-continuous-deployment-of-applications-built-using-btdf-and-octopus-deploy.aspx
I've continued developing the source to work for BizTalk 2016 and extended it with a bit more functionality.
Each step in the process contains small PowerShell scripts that depend on a PowerShell module globally available for all Octopus projects. The module is in the develop branch here: https://github.com/eloekset/publish-btdfBiztalkApplication
A copy of all code snippets in each step is in the following block:
1. Unpublish existing application
	##Octopus environment variables required for this deployment,
	##For non-clustered BizTalk server environments set this to true always
	$IsFirstBiztalkServer = $OctopusParameters['Octopus.Machine.Roles'] -contains "biztalk-primary"
	$IsFirstBiztalkServer = $True #for some reason the test above doesn't work
	$appPoolIdentity = $OctopusParameters['AppPoolIdentity']
	$appPoolPassword = $OctopusParameters['AppPoolPassword']
	$undeployDependentApps = $OctopusParameters['Octopus.Environment.Name'].StartsWith("DEV")

	##initialise variables
	$backupDir = Join-Path "C:\BizTalkApps" "Backup"
	$biztalkAppName = $OctopusParameters["BizTalkApplicationName"]
	$btdfProductName = $biztalkAppName

	##If you have more options in your BTDF installwizard.xml, just add them  as name value pairs as shown here.
	##PS. The /p: is mandatory :
	##Make sure the names or values with spaces are enclosed within double quotes.
	##Double quotes in PowerShell strings can be escaped using "" or `"
	##In this example, see the value of /p:ENV_Settings is escaped with ""
	Write-Verbose "App pool identity: $appPoolIdentity"
	$deployOptions = @{"/p:VDIR_USERNAME"="$($appPoolIdentity)";
	   "/p:VDIR_USERPASS"="$($appPoolPassword)";
	   "/p:BTSACCOUNT"="biztalk";
	}

	New-Item $backupDir -ItemType Container -Force

	#unpublish using the powershell module.
	unpublish-btdfbiztalkapplication -verbose -btdfProductName $btdfProductName -biztalkApplicationName  $biztalkAppName -importIntoBiztalkMgmtDb $IsFirstBiztalkServer -backupdir $backupDir -undeployOptions $undeployOptions -undeployDependentApps $undeployDependentApps

	##To see more options available to customise you deployment including custom undeploy options, set msbuild or BTS task paths use get-help
	#Get-Help Unpublish-BTDFBiztalkApplication -Detailed
	##To troubleshoot and increase log level use the -Verbose switch
	#Unpublish-BTDFBiztalkApplication -verbose

2. Install MSI
	##Get MSI from nuget package.
	$installDir = $OctopusParameters['Octopus.Action[Deploy package].Output.Package.InstallationDirectoryPath']
	Set-OctopusVariable -name "InstallDir" -value $installDir

	##This assumes you have created a nuget package with the msi and the msi is under content dir when the nuget package is unpacked
	$Msi = Get-Item "$(Join-Path $installDir $OctopusParameters["BizTalkApplicationName"])*.msi"

	##initialise variables
	$installOptions = $NULL
	install-btdfBiztalkApp -Verbose $Msi -installDir $installdir -installOptions $installOptions

3. Export environment settings
	##Get MSI from nuget package.
	$installDir = $OctopusParameters['Octopus.Action[Install MSI].Output.InstallDir']
	Write-Verbose "installDir = $installDir"

	#Run EnvironmentSetttingsExporter.exe to genereate xml file for environment
	#to replace its values with matching Octopus variables
	$settingsFolder = Export-EnvironmentSettings $installDir
	Set-OctopusVariable -name "SettingsFolder" -value $settingsFolder

4. Custom substitute values in settings file
	# Call Substitute-XmlSettingsFileValues in script module
	$settingsFolder = $OctopusParameters["Octopus.Action[Export environment settings].Output.SettingsFolder"]
	$settingsFilePath = Join-Path $settingsFolder "Octopus_settings.xml"
	Substitute-XmlSettingsFileValues $settingsFilePath $OctopusParameters

5. Deploy application
	##Octopus environment variables required for this deployment,
	##For non-clustered BizTalk server environments set this to true always
	#$IsFirstBiztalkServer = $true
	$MachineRoles = $OctopusParameters['Octopus.Machine.Roles']
	foreach($r in $MachineRoles)
	{
		Write-Host "Role: $r"
	}
	$IsFirstBiztalkServer = $OctopusParameters['Octopus.Machine.Roles'] -contains "biztalk-primary"
	$IsFirstBiztalkServer = $True #for some reason the test above doesn't work
	Write-Verbose "IsFirstBiztalkServer = $IsFirstBiztalkServer"
	$appPoolIdentity = $OctopusParameters['AppPoolIdentity']
	$appPoolPassword = $OctopusParameters['AppPoolPassword']

	##initialise variables
	$btdfProductName = $OctopusParameters["BizTalkApplicationName"]

	##If you have more options in your BTDF installwizard.xml, just add them  as name value pairs as shown here.
	##PS. The /p: is mandatory :
	##Make sure the names or values with spaces are enclosed within double quotes.
	##Double quotes in PowerShell strings can be escaped using "" or `"
	##In this example, see the value of /p:ENV_Settings is escaped with ""
	$octpousEnvironmentSettingsFile = "Octopus_settings.xml"
	$envSettingsPath = $OctopusParameters["Octopus.Action[Export environment settings].Output.SettingsFolder"]
	$octpousEnvironmentSettingsPath = (Join-Path $envSettingsPath $octpousEnvironmentSettingsFile)
	Write-Verbose "Settings path: $octpousEnvironmentSettingsPath"
	Write-Verbose "App pool identity: $appPoolIdentity"
	$deployOptions = @{"/p:VDIR_USERNAME"="$($appPoolIdentity)";
	   "/p:VDIR_USERPASS"="$($appPoolPassword)";
	   "/p:BTSACCOUNT"="biztalk";
	   "/p:SettingsFilePath"="""$($octpousEnvironmentSettingsPath)""";
	   "/p:SkipExportSettings"="true";
	}

	##Deploy using the powershell module.
	Deploy-BTDFBiztalkApp -verbose -btdfProductName $btdfProductName -importIntoBiztalkMgmtDb $IsFirstBiztalkServer -deployOptions $deployOptions
	##To see more options available to customise you deployment including custom undeploy options, set msbuild or BTS task paths use get-help
	#Get-Help Deploy-BTDFBiztalkApp -Detailed
	##To troubleshoot and increase log level use the -Verbose switch
	#Deploy-BTDFBiztalkApp -verbose

A simple way to achieve the same result is to delete or rename SettingsFileGenerator.xml immediately after the export.  The export will be skipped with a simple message "Skipping settings file export because file at SettingsSpreadsheetPath does not exist: [path]".  And no changes required to the Framework.

Thanks for you answer, @tfabraham! This was a nice workaround. 😊
I can confirm that it works like you said:

For completeness, I'll post the updated script snippets for step 3 and 5:
Step 3:
##Get MSI from nuget package.
$installDir = $OctopusParameters['Octopus.Action[Install MSI].Output.InstallDir']
Write-Verbose "installDir = $installDir"

#Run EnvironmentSetttingsExporter.exe to genereate xml file for environment
#to replace its values with matching Octopus variables
$settingsFolder = Export-EnvironmentSettings $installDir
Set-OctopusVariable -name "SettingsFolder" -value $settingsFolder

#Delte SettingsFileGenerator.xml so BTDF doesn't overwrite our exported settings on deploy
$sourceSettingsFile = Join-Path $settingsFolder "SettingsFileGenerator.xml"
Remove-Item -Path $sourceSettingsFile

Step 5:
#Import-Module .\Publish-BiztalkBtdfApplication.psm1 -Force
# Call deploy-btdfBiztalkApp in script module

##Octopus environment variables required for this deployment,
##For non-clustered BizTalk server environments set this to true always
#$IsFirstBiztalkServer = $true
$MachineRoles = $OctopusParameters['Octopus.Machine.Roles']
foreach($r in $MachineRoles)
{
	Write-Host "Role: $r"
}
$IsFirstBiztalkServer = $OctopusParameters['Octopus.Machine.Roles'] -contains "biztalk-primary"
$IsFirstBiztalkServer = $True #for some crazy reason the test above doesn't work
Write-Verbose "IsFirstBiztalkServer = $IsFirstBiztalkServer"
$appPoolIdentity = $OctopusParameters['AppPoolIdentity']
$appPoolPassword = $OctopusParameters['AppPoolPassword']

##initialise variables
$btdfProductName = $OctopusParameters["BizTalkApplicationName"]

##If you have more options in your BTDF installwizard.xml, just add them  as name value pairs as shown here.
##PS. The /p: is mandatory :
##Make sure the names or values with spaces are enclosed within double quotes.
##Double quotes in PowerShell strings can be escaped using "" or `"
##In this example, see the value of /p:ENV_Settings is escaped with ""
$octpousEnvironmentSettingsFile = "Octopus_settings.xml"
$envSettingsPath = $OctopusParameters["Octopus.Action[Export environment settings].Output.SettingsFolder"]
$octpousEnvironmentSettingsPath = (Join-Path $envSettingsPath $octpousEnvironmentSettingsFile)
Write-Verbose "Settings path: $octpousEnvironmentSettingsPath"
Write-Verbose "App pool identity: $appPoolIdentity"
$deployOptions = @{"/p:VDIR_USERNAME"="$($appPoolIdentity)";
   "/p:VDIR_USERPASS"="$($appPoolPassword)";
   "/p:BTSACCOUNT"="biztalk";
   "/p:SettingsFilePath"="""$($octpousEnvironmentSettingsPath)""";
}

##Deplou using the powershell module.
Deploy-BTDFBiztalkApp -verbose -btdfProductName $btdfProductName -importIntoBiztalkMgmtDb $IsFirstBiztalkServer -deployOptions $deployOptions
##To see more options available to customise you deployment including custom undeploy options, set msbuild or BTS task paths use get-help
#Get-Help Deploy-BTDFBiztalkApp -Detailed
##To troubleshoot and increase log level use the -Verbose switch
#Deploy-BTDFBiztalkApp -verbose

As long as the framework doesn't fail on missing SettingsFileGenerator.xml, the change of this PR won't be needed, so I'll close it and update and close the related issue that I created.

For anyone interested, I've now created a wiki page on how to setup an Octopus Deploy process. The wiki page will be updated, while the code posted in the previous comment above won't. https://github.com/eloekset/publish-btdfBiztalkApplication/wiki/Octopus-Deploy


Make viteburner a vite plugin

Currently the config and code are run externally, not inside vite life cycle. The initial propose is not to be affacted ws conn by vite restart. But this will cause user modified config not reloaded by viteburner.
Probable solution:
Viteburner Server (ws conn only) + Vite Server (other stuff here)

fixed in https://github.com/Tanimodori/viteburner/commit/99cee7f07bf75e9ea1fd542bfb2deca0348ab940


Contact Form Does not Work

I have tried changing the email address couples of times, but it resulted the same.. I have no idea why.. Anyone experienced this before?

same problem here
it say "Message Sent" but no massage received.
php 5.6

in my case is postfix configuration problem, so @subtlefluids01 try to look in your server logs

If you're trying to send mail while looking at the site locally (without a mail server) then the form may give a false positive. Otherwise it might be a permissions or security feature on your web hosts servers. You may want to troubleshoot this with your hosting provider.


[ADAP-547] [Feature] Pagination to allow >10k schemas per database

Is this your first time submitting a feature request?

[X] I have read the expectations for open source contributors
[X] I have searched the existing issues, and I could not find an existing issue for this feature
[X] I am requesting a straightforward extension of existing dbt-snowflake functionality, rather than a Big Idea better suited to a discussion

Describe the feature
https://github.com/dbt-labs/dbt-snowflake/issues/571 used pagination to support retrieving >10k objects in a schema.  The same pattern can be used to enable >10k schemas in a database.
I propose to use the same technique for the latter as the former (which I've confirmed Snowflake supports).
Describe alternatives you've considered
Overriding the macro in our project.
Who will this benefit?
Anyone using dbt at scale — specifically we run into this from time to time with our databases that create an environment per PR for dev.
Are you interested in contributing this feature?
Yes!
Anything else?
No response

Bump

Hello, are there any updates for this or any new related issues?


Add Eden-SDN example for VLANs and LAGs

To continue with the expansion of Eden-SDN examples, this one covers VLANs and LAGs.

LGTM, I see weird yetus complaint in md file

yeah, I put there output of eden pod ps which contains tabs and yetus is not a fun of tabs :)

yeah, I put there output of eden pod ps which contains tabs and yetus is not a fun of tabs :)

Can we mute it for this doc?

yeah, I put there output of eden pod ps which contains tabs and yetus is not a fun of tabs :)

Can we mute it for this doc?

There is probably some magic string for that but I cannot find it :(

yeah, I put there output of eden pod ps which contains tabs and yetus is not a fun of tabs :)

Can we mute it for this doc?

There is probably some magic string for that but I cannot find it :(

Okay, I guess we can merge it then


VScode behavior is very slow while using clientExtensions

Bug description
Today I updated prisma to 4.10.0 and immediately experienced very slow VScode behavior in project where prisma is installed.
EDIT:
After few hours of playing with VScode extensions, code, schema, ... I ended up with new observings.
Seems like the version is not causing the issue. Let me explain:
Before updating to version 4.10.0 I prepared this code for extending the prisma client:
.$extends({
  result: {
    iOrder: {
      orderNumber: {
        needs: { id: true, randomNumber: true },
        compute(order) {
          return +`${order.id}${order.randomNumber}`;
        }
      }
    }
  },
  model: {
    iOrder: {
      byOrderNumber: (vs: number) => {
        const str = `${vs}`;
        const split = [parseInt(str.slice(0, str.length-2)), parseInt(str.slice(str.length-2))];
        return {
          id: split[0],
          randomNumber: split[1]
        };
      }
    } satisfies Record<string, (...args: any) => Prisma.IOrderWhereInput>,
  }
});

Right after updating the version I extended the client with this code and added clientExtensions to previewFeatures and ran npx prisma generate. After that the VScode window was very slow (intellisense, error higlighting, ...).
For example after typing (new PrismaClient()). the intellisense would show up in +- 13 seconds. Because of that I wrongly blamed the version - sorry about that.
Now what I discovered:

I removed the extending code (see above) and the intellisense show time dropped to +- 3 seconds.
I removed clientExtensions from previewFeatures and the show time was almost instant (as it was).

So it seems like that the clientExtensions preview feature is causing this problem.
Sorry about not posting our schema as we find it sensitive. But there are 24 models.
Btw this is my machine.
How to reproduce

use clientExtensions preview feature
extend the client
experience the slowness of VScode - for example intellisense

Expected behavior
VScode should be quick with and without using clientExtensions.
Environment & setup

OS: Windows 11
Database: PostgreSQL
Node.js version: 16.15.1

Prisma Version
prisma                  : 4.10.0
@prisma/client          : 4.10.0
Current platform        : windows
Query Engine (Node-API) : libquery-engine ca7fcef713137fa11029d519a9780db130cca91d (at<EMAIL_ADDRESS>Migration Engine        : migration-engine-cli ca7fcef713137fa11029d519a9780db130cca91d (at<EMAIL_ADDRESS>Format Wasm             : @prisma/prisma-fmt-wasm 4.10.0-84.ca7fcef713137fa11029d519a9780db130cca91d
Default Engines Hash    : ca7fcef713137fa11029d519a9780db130cca91d
Studio                  : 0.481.0
Preview Features        : clientExtensions

Hey @drekthral, A few things.

You had no issues on previous versions of Prisma?
Can you share your schema or a reproducible case for us - That makes our life so much easier?
Thanks.

Hey @drekthral, A few things.

You had no issues on previous versions of Prisma?
Can you share your schema or a reproducible case for us - That makes our life so much easier?
Thanks.

Managed to find more info - the issue is updated.

Hey @drekthral, I have tried this on a project with a huge and complex schema but could not reproduce. Could you provide us with your schema as well as your tsconfig.json, and ideally a full reproduction? That would help a lot, thank you!

Hi @millsp
tsconfig.json
{
  "compilerOptions": {
    "module": "commonjs",
    "noImplicitReturns": true,
    "noUnusedLocals": true,
    "outDir": "lib",
    "sourceMap": true,
    "strict": true,
    "target": "es2017",
    "esModuleInterop": true
  },
  "compileOnSave": true,
  "include": [
    "src"
  ]
}

Could I send you the schema privately somehow? We do not want it to be public.

Thanks. Yes, please send us your schema to<EMAIL_ADDRESS>If needed we can also do an NDA.

@millsp The schema was sent.
One more note to the issue: This slow behavior is not immediate. It shows up after few minutes of working. And it seems, like prismaTransactionClient. takes longer to show intellisense than prismaClient.

Adding my experience with the same issue here: prettier on save would take 4-6s, intellisense was super slow. This isn't the case when not calling $extends on the prisma client.
By digging a bit I saw that eslint was complaining that applying the rules was taking a long time.
With trial and error, commenting these two rules solved the problem completely.
    "@typescript-eslint/no-floating-promises": ["error"],
    "@typescript-eslint/no-misused-promises": ["error"],

Thanks @drekthral, we've received your schema.

Same thing happening to me. Even just turning on "clientExtensions" in the schema file makes  type resolution/completion markedly slower. Actually trying to extend makes it grind to a hault

I'm also seeing something similar with my schema (nearly opened an issue but saw this was already here). Have a few hundred tables and doing a performance trace with TypeScript and analyze-trace shows me there's several hotspots related to evaluating types that are suffixed with "Needs". Looked in the index.d.ts and saw those were all client extension related.
VSCode and the typechecker basically grind to a halt and it's difficult to work. Once I remove the reference to the client extension, everything is speedy again.

It turns out my issue might have been caused by a bad type cast in the Prisma example here. This type cast causes the typechecker to become much slower, and when moving this $begin function outside of the client extension into a separate function, it hangs the typechecker altogether (the perf trace file gets larger and larger with the typechecker never finishing, even after several minutes).
Removing client extensions does make navigating VSCode a bit smoother (and typechecking somewhat faster), but not as big of an improvement as fixing this bad type. Just thought I'd comment here in case anyone else runs into this too.

Awesome @aniravi24, that is a great hint for us to double check.

We're seeing similar results in our project but it's not isolated to VS Code but rather all TS transpilation. Just adding an empty extension causes our TS transpile time to increase by 200-300%. Running a trace shows the first module that imports prisma taking 25~ seconds to transpile.
const prisma = new PrismaClient().$extends({
  result: {
    advert: {},
  },
})
export { prisma }

Hey @anton-g, would you be able to provide us with your schema? If ti contains sensitive information you could also send it at<EMAIL_ADDRESS>

I have the same problem. My vscode started working very slowly and after a few days I discovered that the reason is prisma extension mechanism. I think typescript has some problems with parsing prisma extensions types. Maybe there is some recurrency in types or something like that?

Hey, we have refactored the code generation to switch from fully generated types to dynamic typing. This should have improved the performance drops that you have reported. I love to hear your feedback @niba @anton-g @aniravi24 @drekthral to know if things got better on your end. Thanks. The fix is available in 4.16.0.


Feature request: add a compass rose tpf.plot()

mkpy3_tpf_plot_add_compasss_rose_v1_py.txt

Feature description

I have attached a Python3 function that adds a compass rose showing the North and East directions on top of a normal tpf.plot() graph. The long arm of the compass rose points to North (increasing declination).  The short arm points to East (increasing right ascension)
Example

import matplotlib.pyplot as plt
import lightkurve as lk
#
tpf = lk.search_targetpixelfile(target='Kepler-138b',mission='Kepler',quarter=10).download()
#  ---> Exoplanet Kelper-138b is "KIC 7603200"
#
# Plot the 2nd frame of the TPF
ax = tpf.plot(frame=1)
#
# add a compass rose
mkpy3_tpf_plot_add_compass_rose_v1(ax=ax, tpf=tpf)
#
plot_file = 'mkpy3_spud.png'
plt.savefig(plot_file, bbox_inches="tight")
plt.close()
print(plot_file,' <--- new PNG file written')

produces the attached PNG plot showing a compass rose superimposed on top of the tpf.plot() graph.
Comments

The function could be implemented as a method inside the class TargetPixelFile in targetpixelfile.py
While it might be nice to have a compass rose shown in tpf.interact() and tpf.interact_sky(), I suspect that this new function would significantly slow down those two methods.

Thanks very much for the suggestion and code! :+1:
I am inclined to not add a compass rose for now because we do not currently have resources to add and maintain new features.  If this feature is high in demand I would be happy to consider a PR however!
Thanks!


Add translatable

will be relevant to Organization descriptions
SystemLocations
Listing title, description, tags
SystemTags
CommunityResources
Announcements

What are the advantages of using the Translatable gem over using native i18n support and having the locale YMLs loaded based on user-selected locale?

Hey there, @armahillo !!! VERY open to your suggestions on this, but here's the thinking below.

We won't know the data needed for the YML at deploy bc it's translations of other user-created data
By having it be data that is structured and accessible in the same way as the other system-level datasets (SystemTags, SystemLocations) any of their admins can configure the system and make changes on the fly through the UI.
We will need to be able to search on the translated data, so having it in SQL along w the english versions is ideal.

@maebeale, i definitely hear the need for dynamic, user created translations. What do you think about pulling in Mobility, a gem designed to support this use case? I think it would do a lot of the heavy lifting for us. There's also Globalize, an older, more widely used gem, but my cursory review of both has me leaning towards Mobility.

YESSSSS, @exbinary!!!!

I can change this pr to remove that table


Clarifying store list_dir / listdir details in implementation vs. spec

Here's what the V3 Spec currently says about the store list_dir method
``list_dir`` - Retrieve all keys and prefixes with a given prefix and
which do not contain the character "/" after the given prefix.

    | Parameters: `prefix`
    | Output: set of `keys` and set of `prefixes`

    For example, if a store contains the keys "a/b", "a/c", "a/d/e",
    "a/f/g", then ``list_dir("a/")`` would return keys "a/b" and "a/c"
    and prefixes "a/d/" and "a/f/". ``list_dir("b/")`` would return
    the empty set.

https://github.com/alimanfoo/zarr-specs/blob/b63e64dcf530f195b19691e06a44c85916f33291/docs/core/v3.0.rst?plain=1#L1344-L1353
There are three inconsistencies between this and the current zarr-python V3 implementation:

The function is named listdir (not list_dir) in the implementation
The listdir function does not include the prefix. So listdir("a/") returns b, not a/b. This is pretty important for custom store implementations.
The implementation do not return a trailing slash for prefixes, while the spec seems to recommend this.

I think we should make the spec and the implementation consistent.
cc @jstriebel and @alimanfoo, who are currently revising the V3 spec.

After re-reading this part of the spec, I think that the / suffix is useful in the API, especially to make this definition as simple as it is:

Retrieve all keys and prefixes with a given prefix and
which do not contain the character "/" after the given prefix.

@rabernat @joshmoore Should we turn this into a zarr-python issue then? (Related to https://github.com/zarr-developers/zarr-python/issues/1290)

I agree it is a python issue.
I'm still unsure of how to distinguish keys vs. prefixes and whether this needs to be specified in the spec. This is related to #200. Ideally we should suggest a specific algorithm for efficiently traversing a store using listdir. Right now the zarr-python implementation makes tons of unnecessary list and __contains__ calls.

I'm still unsure of how to distinguish keys vs. prefixes and whether this needs to be specified in the spec.

:+1: This would be good to capture and discuss for v3.

@rabernat @joshmoore Do you have any concrete proposals for the current spec? Happy to do a review, but I'm not quite sure what exactly should be changed.

Basically the store needs a way to communicate whether the thing it is returning is a file or a directory. One way to solve this would to have prefixes / directories always return a trailing / in a list operation. But that feels like an implementation detail, rather than a spec issue.


add tensor to memref op to triton gpu dialect

Only add the op to the triton gpu dialect. Lowering implementation will be in separate PR.

Could you please rebase your changes onto dot-rocMLIR to resolve the conflicts? I added memref_to_tensor sometime this morning.

Rebased.


build fail

I have try with both master and version 0.2.0.
Note: Recompile with -Xlint:unchecked for details.
:processResources UP-TO-DATE
:classes
:jar
:javadoc
:javadocJar
:sourcesJar
:assemble
:instrument SKIPPED
:copyCoberturaDatafile SKIPPED
:generateTestAvroProtocol UP-TO-DATE
:generateTestAvroJava UP-TO-DATE
:compileTestJava
Note: /home/ubuntu/storm-graphite-0.2.0/src/test/java/com/verisign/storm/metrics/reporters/BaseKafkaReporterTest.java uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
:processTestResources UP-TO-DATE
:testClasses
:test FAILED
:generateCoberturaReport SKIPPED
:performCoverageCheck SKIPPED

Hi @mydiemho, could you please post what OS and version of Java you're using?

Java version "1.7.0_79"
OpenJDK Runtime Environment (IcedTea 2.5.5) (7u79-2.5.5-0ubuntu<IP_ADDRESS>)
OpenJDK 64-Bit Server VM (build 24.79-b02, mixed mode)


Obsoletion notice: GO:0048016 inositol phosphate-mediated signaling

[ ] Obsolete GO:0048016 inositol phosphate-mediated signaling >>  represents some calcium signaling

[x] inositol 1,4,5-trisphosphate-sensitive calcium-release channel activity >> remove  part_of GO:0048016 ! inositol phosphate-mediated signaling

Dear all,
The proposal has been made to obsolete GO:0048016 inositol phosphate-mediated signaling. The reason for obsoletion is that this term is not clearly defined and represents several pathways, or is sometimes used to annotate experiments in which some inositol phosphate's level changes, but no data is presented on the actual pathway affected. There are 7 EXP annotations left to this term. There are no mappings; this term is not present in any subset.
Impacted groups:

Contributor
Count

BHF-UCL
4

MGI
1

SGD
1

UniProt
4  - DONE

WB
1

We plan on obsoleting the term in a week, on July 25, 2023.
You can comment on the ticket: https://github.com/geneontology/go-ontology/issues/25718
Thanks, Pascale


Fixes #728 - Add ability to get all rows in REST v2

Add the ability to get all rows in REST v2.

/gcbrun

FYI @polandll

/gcbrun

@dougwettlaufer Has been a bit busy with work these last few days, but gonna resume addressing the work on this issue and the rest-index this afternoon. :)

@dougwettlaufer Has been a bit busy with work these last few days, but gonna resume addressing the work on this issue and the rest-index this afternoon. :)

Hey no worries. We appreciate your help as you have time @eribeiro !

@dougwettlaufer @dimas-b Hi, I have delayed this PR too much, I am sorry. :) I have addressed the latest review comments, please let me know if there's something that I am still missing here whenever you have some free time. Cheers!

/gcbrun

/gcbrun


Adjust release date

Just updating the date for the release.

API change check
API changes are not detected in this pull request.


Crop images taken from camera

Description
I want to crop the images before sending it to the server. Is there any option in this package to handle cropping? Thanks
How to repeat issue and example
Take the image with RNImagePicker.launchCamera() and you will see that cropping option is not available

React Native version: ^0.55.3
Platform: IOS
Development Operating System: MacOS
Dev tools: Xcode

I can't use here react-native-image-crop-picker package, since it does not support mixed media types

The README is very specific - it says this package doesn't handle any processing of the media. I've been using it for over a year and I haven't seen them even consider adding anything like this. I don't think it will be an option for you, unfortunately.

Thanks @jwrubel. I ended up using react-native-image-crop-picker package where there is ImagePicker.openCropper({}) prop. So I called that prop in the response of react-native-image-picker package... And as a result, I could take the picture and then resize it when needed. Hope this could help anyone. But it will be great to include cropping option into this package

@DavitVosk  how did you managed the imports though? Can you share the code?

@DavitVosk how did you managed the imports though? Can you share the code for the same?

`import ImagePicker from 'react-native-image-picker'
import ImageCropPicker from 'react-native-image-crop-picker'
ImagePicker.showImagePicker(options, response => {
  if (response.didCancel) return

  ImageCropPicker.openCropper({
    path: response.uri,
    width: deviceWidth,
    height: 200,
    includeBase64: true,
  }).then(cropped => {
    // save it
  })`

Hope this helps!


Multiple gradient targets are printed in inconsistent order

[2024-07-07 15:53:16][INFO] - Epoch    0, training loss: 7.411e+02, training energy RMSE (per atom): 263.50 meV, training forces RMSE: 487.64 meV/Å, training virial RMSE (per atom): 350.58 meV, validation loss: 2.360e+02, validation energy RMSE (per atom): 305.69 meV, validation forces RMSE: 494.19 meV/Å, validation virial RMSE (per atom): 320.10 meV
[2024-07-07 15:53:21][INFO] - Epoch    5, training loss: 4.681e+02, training energy RMSE (per atom): 185.25 meV, training virial RMSE (per atom): 251.56 meV, training forces RMSE: 401.85 meV/Å, validation loss: 1.560e+02, validation energy RMSE (per atom): 207.07 meV, validation forces RMSE: 427.73 meV/Å, validation virial RMSE (per atom): 221.01 meV
[2024-07-07 15:53:26][INFO] - Epoch   10, training loss: 3.364e+02, training energy RMSE (per atom): 121.45 meV, training virial RMSE (per atom): 197.40 meV, training forces RMSE: 356.33 meV/Å, validation loss: 1.161e+02, validation energy RMSE (per atom): 147.57 meV, validation virial RMSE (per atom): 243.40 meV, validation forces RMSE: 380.13 meV/Å
[2024-07-07 15:53:30][INFO] - Epoch   15, training loss: 2.337e+02, training energy RMSE (per atom):  87.08 meV, training forces RMSE: 301.08 meV/Å, training virial RMSE (per atom): 175.80 meV, validation loss: 8.225e+01, validation energy RMSE (per atom): 109.81 meV, validation forces RMSE: 323.49 meV/Å, validation virial RMSE (per atom): 235.19 meV
[2024-07-07 15:53:35][INFO] - Epoch   20, training loss: 1.592e+02, training energy RMSE (per atom):  54.48 meV, training forces RMSE: 252.10 meV/Å, training virial RMSE (per atom): 161.68 meV, validation loss: 5.964e+01, validation energy RMSE (per atom):  76.02 meV, validation virial RMSE (per atom): 206.22 meV, validation forces RMSE: 280.62 meV/Å
[2024-07-07 15:53:40][INFO] - Epoch   25, training loss: 1.475e+02, training energy RMSE (per atom):  54.70 meV, training virial RMSE (per atom): 158.40 meV, training forces RMSE: 242.00 meV/Å, validation loss: 5.034e+01, validation energy RMSE (per atom):  64.73 meV, validation forces RMSE: 258.63 meV/Å, validation virial RMSE (per atom): 197.36 meV
[2024-07-07 15:53:45][INFO] - Epoch   30, training loss: 1.345e+02, training energy RMSE (per atom):  66.67 meV, training virial RMSE (per atom): 165.71 meV, training forces RMSE: 226.20 meV/Å, validation loss: 4.840e+01, validation energy RMSE (per atom):  87.70 meV, validation forces RMSE: 244.84 meV/Å, validation virial RMSE (per atom): 208.27 meV
[2024-07-07 15:53:50][INFO] - Epoch   35, training loss: 1.210e+02, training energy RMSE (per atom):  65.89 meV, training forces RMSE: 213.71 meV/Å, training virial RMSE (per atom): 157.02 meV, validation loss: 4.174e+01, validation energy RMSE (per atom):  61.95 meV, validation forces RMSE: 233.98 meV/Å, validation virial RMSE (per atom): 189.51 meV
[2024-07-07 15:53:54][INFO] - Epoch   40, training loss: 1.023e+02, training energy RMSE (per atom):  42.10 meV, training forces RMSE: 200.47 meV/Å, training virial RMSE (per atom): 159.44 meV, validation loss: 3.770e+01, validation energy RMSE (per atom):  51.23 meV, validation forces RMSE: 223.15 meV/Å, validation virial RMSE (per atom): 195.01 meV
[2024-07-07 15:53:59][INFO] - Epoch   45, training loss: 1.015e+02, training energy RMSE (per atom):  48.63 meV, training forces RMSE: 198.13 meV/Å, training virial RMSE (per atom): 158.39 meV, validation loss: 3.567e+01, validation energy RMSE (per atom):  47.34 meV, validation virial RMSE (per atom): 177.76 meV, validation forces RMSE: 218.59 meV/Å

Forces and stress are printed inconsistently between different lines. This comes from using sets in the code to store gradients

I think this can handled in the same way as we do it for the types and feed them using the sorted function into the logger.
I know that you also would like to change and store the gradients property of the TargetInfo class and the atomic_types of the DatasetInfo as a list of sorted str.
If you want to change this I suggest storing them internally as a private set and only expose them as a List[str] in the public API. With this one can still use the handy functionalities of a set. But, this is not supported by a dataclass. We therefore have to remove the dataclass and make TargetInfo and DatasetInfo "normal" classes and implement the functionality that the dataclass provides automatically. Basically the __eq__() and if we want __repr__() method.