Feature 418 sphinx use case reorg

Moved sphinx py files to docs/use_cases and updated conf.py to read files from there.

@georgemccabe am I supposed to resolve the conflicts or just review the changes?

I will resolve them. There weren't conflicts earlier but changes have been pushed to develop.

OK thanks. I don't think there's any options for me to choose until the conflicts are resolved, otherwise I can just comment here if I feel the changes "pass my review"?

OK, conflict resolved.

These are the only commits I need to review right:
https://github.com/NCAR/METplus/pull/440#commits-pushed-ff1154b

You can go to Files Changed (https://github.com/NCAR/METplus/pull/440/files) and review the differences. Most of the changes are renaming files, so there isn't much content there. You should check out this branch, build the docs, and make sure everything looks OK. If you'd prefer, you can look at the docs I built (http://icculus.rap.ucar.edu/metplus-test) but it would probably be good for you to build them so you can review the warnings/errors as well.

Which version of sphinx-gallery should I be using? I think I need to build from source outside the py3.6.3 YML file right?

Dan, there are conda environments set up on some of the NCAR machines that
can be used. There are also instructions to set up an environment to build
the docs. I will send you both in a separate email.
On Mon, Apr 27, 2020 at 11:27 AM Dan Adriaansen<EMAIL_ADDRESS>wrote:

Which version of sphinx-gallery should I be using? I think I need to build
from source outside the py3.6.3 YML file right?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/NCAR/METplus/pull/440#issuecomment-620124875, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AFSSZNZBAPRQAMOOW2YGDNTROW6BZANCNFSM4MNRP7TQ
.

--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
My working day may not be your working day. Please do not feel obliged to
reply to this email outside of your normal working hours.


Feature/2729 vectorize abs

Summary
Adds vectorized signatures for abs function for integer, real, and complex-valued functions.
Comments

After this is merged, we should be able to deprecate fabs.
@SteveBronder might want to suggest more general signatures to take expression templates without copying, but the ones I have in place are functional.

Tests
Full testing of primitives and types in test/unit/math/prim/fun/abs_test.cpp and of autodiff using the test framework in test/unit/math/mix/fun/abs_test.cpp.
Side Effects
No.
Release notes

added vectorized abs() for all argument types

Checklist

[x] Math issue #2729

[x] Copyright holder:  Simons Foundation
The copyright holder is typically you or your assignee, such as a university or company. By submitting this pull request, the copyright holder is agreeing to the license the submitted work under the following licenses:
- Code: BSD 3-clause (https://opensource.org/licenses/BSD-3-Clause)
- Documentation: CC-BY 4.0 (https://creativecommons.org/licenses/by/4.0/)

[x] the basic tests are passing

unit tests pass (to run, use: ./runTests.py test/unit)
header checks pass, (make test-headers)
dependencies checks pass, (make test-math-dependencies)
docs build, (make doxygen)
code passes the built in C++ standards checks (make cpplint)

[x] the code is written in idiomatic C++ and changes are documented in the doxygen

[x] the new changes are tested

Using this branch with https://github.com/stan-dev/stanc3/pull/1195 (which exposes the signatures to the compiler)
causes expression tests to fail to compile: https://jenkins.flatironinstitute.org/blue/organizations/jenkins/Stan%2FStanc3/detail/PR-1195/2/pipeline/304/
It looks like the issue is when the input is an eigen expression of a  complex matrix

Thanks, @WardBrian.    I need help with the following to fix.
@SteveBronder and @rok-cesnovar:  Is there doc for the template traits somewhere that'll let me figure out how to carve out the right argument constraints?  My problem isn't with C++, it's with our template traits lib and what it does and what its naming conventions are?
@SteveBronder and @rok-cesnovar:   Is there doc for testing requirements for the math lib so we don't run into downstream problems in integration like @WardBrian is reporting?  We don't want to have to wait until integration with stanc3 to find bugs in the math lib.  In particular, how can I test a math function does the right thing with generic matrix-like args?  Does it have to work for expression templates?  For blocking?
@SteveBronder and @rok-cesnovar:  Where is the doc for all of the var mat stuff?  There's a cryptic definition for abs in rev, but I don't understand it.  I also don't understand how this is being tested.  Is it now a requirement that every vectorized function works for var_mat?

We don't want to have to wait until integration with stanc3 to find bugs in the math lib. In particular, how can I test a math function does the right thing with generic matrix-like args? Does it have to work for expression templates? For blocking?

I would like to echo this is important and currently kind of a chicken-egg situation. The math CI does run the same expression tests as the compiler, but the expression tests require as input a list of signatures to test, and that list is generated by the compiler. This means the math library cannot test until the compiler is updated, so the best we can do at the moment is run experimental compiler branches against the experimental math branches, which is far from ideal

Is there doc for the template traits somewhere that'll let me figure out how to carve out the right argument constraints? My problem isn't with C++, it's with our template traits lib and what it does and what its naming conventions are.

I think https://mc-stan.org/math/d8/de1/group__type__trait.html and https://mc-stan.org/math/d1/db9/group__require__meta.html
are all the general docs we have on that topic, but Steve would know more here. Is this what you are looking for or should we expand this more?

We don't want to have to wait until integration with stanc3 to find bugs in the math lib. In particular, how can I test a math function does the right thing with generic matrix-like args? Does it have to work for expression templates? For blocking?

There are expression requirements for functions with arguments that are Eigen types. They have to accept expressions and they have to not evaluate expressions multiple times. I think this is the only requirement that was added in the last versions or last few years.
But this is only a requirement if a function is exposed in the Stan language. If it is exposed, it MUST be able to accept expressions, otherwise Stan users would see real long and bad C++ errors. This is what happened two years ago when we enabled functions returning expressions: https://github.com/stan-dev/math/issues/1643 This is why we added this huge chunk of CI testing that is testing expressions.
We could, in theory, have Stan Math C++ functions that are not exposed in the Stan language and don't accept expressions. This is how Eigen works sometimes, where some functions don't accept expressions and you just need to .eval() them in that case. We probably don't want to have those, but I don't see a way of enforcing this policy for functions that are not exposed in the language. It would require either the developer having to write tests for expressions and put the burden of checking that on the reviewer.
The things that are probably not documented well enough are:

the fact that we are enforcing this policy wrt expressions
that fact that you can test a currently non-existent function if it works properly with expressions even if you don't have a modified stanc3:

python3 runTests.py test/expressions/ --only-function "foo(vector) => vector"

Running this will do all the expression tests for this signature of a function that is not currently in Stan without having to touch stanc3 at all.
I don't think there is a way of testing this in CI though before we try to expose it -  I am definitely open to ideas we could try.

Is it now a requirement that every vectorized function works for varmat?

No. We turn "varmat" on in stanc3 manually for selected function. It is better if there are more of them, but it is not a requirement at all.

There's a cryptic definition for abs in the rev/fun directory, but I don't understand it.

The abs() implementation just uses fabs https://github.com/stan-dev/math/blob/43ec11b55f3f6d35d8962ec37e879038d19321dc/stan/math/rev/fun/fabs.hpp#L70
which looking at it quickly support varmat.

I also don't understand how this is being tested.

I believe this line tests varmat stuff for abs(): https://github.com/stan-dev/math/blob/develop/test/unit/math/mix/fun/abs_test.cpp#L72
Also see https://mc-stan.org/math/df/d82/reverse_mode_types.html

Thanks, @rok-cesnovar.  I'm afraid I need something more example-oriented than the API doc, because I don't understand the implications of the API doc (like what extending MatrixBase does or what var_value does).    Then my problems are compounded by traits for scalar having non-standard meaning (complex numbers are scalars in the mathematical sense, but not in the Stan traits sense).
So what we need is something to help someone who can do basic C++ bridge the gap from the matrix calculus to a multivariate function implementation.

@bob-carpenter so something like the following guide for distributions https://mc-stan.org/math/df/d67/new_distribution.html, but for regular math functions?
I think that is a good idea. We also have a ton of great resources in a bunch of places, like https://mc-stan.org/math/d1/d66/getting_started.html we just need to give it a bit more structure.

. something like the following guide for distributions

Yes, @rok-cesnovar.  But after spending an hour or two talking to @SteveBronder about this yesterday, I don't think I'm going to be able to help with the doc or with future C++ programming--the gap between the current doc and my understanding of C++11 and the current Stan math conventions is just too great.
I sent Steve feedback on the first page, but it was pretty much one question per line and it just gets worse in the actual doxygen API doc.  I also have no idea how to figure out analytic derivatives for what you are calling "var mat".  So I don't know if even the request I'm making is reasonable.  I'm happy to keep providing feedback on doc iterations until the doc approaches something I can understand or you get tired dumbing things down and fleshing things out to make me happy.  Until then, I'll leave the C++ coding to people like you who understand C++11 and Stan math better than me.

Given that #2737 was merged, I am going to close this one.


feat: switch completely to libbpf kepler image

With Kepler release 0.7, the default kepler image has switched to
using libbpf based ebpf attacher. This removes the need for supporting
the libbpf hack and switch completely using that.
This commit makes the following changes

deploys kepler 0.7.2 image
removes the libbpf annotation hack

Fixes #330
Signed-off-by: Sunil Thaha<EMAIL_ADDRESS>

Perfect, thank you!


Block_reduce fails to distribute correct answer to all lanes when hipBlockDim > 64

Describe the bug
Block_reduce seems to output incorrect values when hipBlockDim > 64 on my Vega64. There seems to be a bug in distributing the output to other warps. The attached example.cpp provides an easy demonstration of the bug.
The #define for NUM_THREADS provides an easy set of tests for the values 64, 128, 192, and 256. NUM_THREADS == 64 outputs all correct answers. However, 128 only produces the correct answer in two threads (Thread0 and Thread1). 192 produces the correct answer in one lanes (Only thread0), while 256 produces the correct answer in only four lanes.
The block_reduce code is clearly missing the "broadcast" step in some cases, where Thread0 broadcasts the value back to all 256 threads.
To Reproduce
example.cpp.txt
Compile the attached "example.cpp" file with hipcc example.cpp -I/opt/rocm/rocprim/include/. Play with the #define NUM_THREADS value as you see fit.
Expected behavior
In the case of 192-threads, I expect 18336 (0+1+2+3+4...+191) to be the output for all 192-lanes.
Log-files
The following output is from a NUM_THREADS of 192. The value of 18336 in thread0 is correct, but has failed to be distributed to other lanes.

18336
16320
10208
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
2016
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
6112
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208
10208

@dragontamer Thanks for including the sample code.  A developer is now reviewing this report.

@dragontamer
I believe your assumptions about how a block reduction algorithm works within our library (or even CUB) are wrong.
A block reduction algortihm holds the result only on the first thread of the block. This is in fact how it is implemented.
The reason that you think this is incorrect for block sizes of larger than 64 is simply because the warp size is 64 and with block sizes of less than 64 only one warp reduction is being done.
But when the block size is bigger than a warp. The block reduction algorithm only stores the result only in the first thread(lane_id = 0 warp_id = 0). Keep in mind that block reduction is different than warp reduction.
So if you require propagation of the results to all threads you have to do them manually. For example:
__global__ void helloworld(int64_t* buffer){

    // We need a shared variable to store the result of the reduction for propagation in all threads
    __shared__ int64_t final_result;

    int64_t reduction;
    typedef rocprim::block_reduce<int64_t, NUM_THREADS> MyReduce;
    MyReduce().reduce(hipThreadIdx_x, reduction);

    // The first thread will update the final_result shared variable so that all threads have access to it
    if(hipThreadIdx_x == 0)
    {
        final_result = reduction;
    }
    sync_threads();

    // All threads will now fill the buffer with the final_result
    buffer[hipThreadIdx_x] = final_result;
}

@ncrkblacksmith : If that is the intended use of block_reduction, then it should be documented somewhere.
None of the Doxygen comments indicate that ouput is only valid in lane0. In fact, the documentation here suggests that the answer is distributed to all 256 lanes: https://github.com/ROCmSoftwarePlatform/rocPRIM/blob/develop/rocprim/include/rocprim/block/block_reduce.hpp#L222

We will get the documentation updated in a future release.


refactor: rename native asset info

Changes in the PR:

Change native asset's info, from 'CKB' to 'pCKB'
Only changing V1's info

Remaining issues:

request wallet_addEthereumChain won't work for Users who have added godwoken's chain info to their MetaMask, so the changes to native asset info are transparent to them, is there anything we can do?
On MetaMask, symbols are all uppercase, so 'pCKB' on MetaMask is displayed as 'PCKB', does it bother?

Codecov Report

:exclamation: No coverage uploaded for pull request base (develop@f9153b6). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             develop     #167   +/-   ##
==========================================
  Coverage           ?   50.53%
==========================================
  Files              ?       29
  Lines              ?     1401
  Branches           ?      208
==========================================
  Hits               ?      708
  Misses             ?      501
  Partials           ?      192

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f9153b6...a491099. Read the comment docs.


[HELP] How to merge multiple indexes into one ?

Hi,
I want to create several collections in my Zep server, and then when a prompt/query arrives, I want to combine several collections into one index and be able to query the combined index.
Is this even possible?
What I tried to do, unsuccessfully, is something like:
def load_zep(collection_name):
        vector_store = ZepVectorStore(
            api_url=zep_api_url, collection_name=collection_name
        )

        return StorageContext.from_defaults(vector_store=vector_store,persist_dir=f'./storage/{collection_name}')

                from llama_index import load_index_from_storage, StorageContext

                #load different collections
                storage_context1 = load_zep('collection1')
                storage_context2 = load_zep('collection2')
                storage_context3 = load_zep('collection3')

                #some how combine the above 3 collections into one

                #tried various ways like combined = list(), but nothing worked

                #load index from above combined
                index = load_index_from_storage(combined)

The logic behind this is I have various user levels and based on the user I want to give them a different index. Higher level users gets more indexes, while low level user gets a single index to query. I don't want to create the same index over and over if I can combine them somehow.
Thanks in advance!

Search scores represent cosine distance normalized to (0,1). As such, you may be able to accomplish this by combining results from separate searches over the different Collections, ordering by score, and selecting your top n results.


web return from fullscreen throws error

web return from fullscreen throws error and results in a black video player

error fixed, but when going back from full screen the video_player is dark, guess its a video_player bug

Facing same error

Please resolve it

I am trying to do it, and I figured out a fix but it didn't work 😢
,So I will try again since you guys has the same problem

Hi @zezo357. Thanks for your response. Am waiting for your resolution.
Suggestion: if possible try to implement multiple audio track in this player.
Like English, Hindi etc.

Hi @zezo357. Thanks for your response. Am waiting for your resolution.
Suggestion: if possible try to implement multiple audio track in this player.
Like English, Hindi etc.

Sadly, video_player doesn't support that, meanwhile media_kit does
You can use my flutter_meedu_media_kit and use media_kit functions, if you don't need my controls you can use media_kit directly they have controls too.

4.2.16 fixes the web black screen.

Thanks buddy😇


Improvement: please move GND and 3V3 vias from FPGA pins

Hi Alex, first of all thank you for this great project.
I had a hard time doing hardware debug because of these vias near FPGA pins:

Because I'm soldering manually with hot iron, the solder move very easily to these vias. I removed many components until I found this issue. After cutting the wire from C8 to 3V3 in the other side of the board the issue disappeared.
Maybe you could move the vias from C5, C7 and C9 too, because they also could end up causing similar issue.

I won't be making hardware changes just for that. And I don't expect any other hardware changes. This is my standard clearance and I never had any issues soldering those devices by hand.
Provided Gerber files specify tented vias. You would have to try really hard to create shorts. I don't even know how I would do that if I wanted to.
You likely need to use a better soldering equipment or techniques.

Hi @ataradov,
actually I just soldered using the standard way I used to solder many other projects (that is basically as doogulass did in this video as well.
So no, I didn't "tried hard to create the shorts".
I only opened it issue to help making it easier for other people doing it, I didn't expect you receiving it as a criticism. The only purpose was to avoid other people facing same issues I faced. I manufactured the board at JLCPCB that has a good quality, so it was not the soldermask getting removed easily.
I suspect the solder flowed from bottom side (lower pad of C8) and made contact with some FPGA pin or the GND via at side of it. BTW, this is just a supposition because I don't have x-ray machine here to confirm my suspect.

Your description does not match the possible short you are describing.  If the short is under the IC, how would removing C8 help it? It is much more likely that C8 itself has a short.
The solder flow you are describing is extremely hard to achieve.

I didn't remove C8, I just did a sharp cut the wire trace from C8 via to 3V3 (that I cold re-wire again with a drop of solder) and the short disappeared.

Which is equivalent to removing the C8 from the circuit. If the short was on the other side of the board under the IC, then doing that cut would not have solved the issue.
If you still have C8 disconnected, then you can measure it for continuity. If there was a short, it would be in C8.


integer key max value

Hi,
Why exists a max limit to integer key value of 0xFFFFFFFF ?
I have greaters keys values that I can´t use. With string keys is allowed lenght until 255 characters
thanks

"The supported integer values include all integers between -2^53 and 2^53, and some integers with larger magnitude" - https://docs.hivedb.dev/#/more/limitations is all I can find on this. It probably has something to do with the limits of integer and double types across all programming languages.
A good explanation I found is in the Safe Integers section of this page: https://exploringjs.com/es5/ch11.html

Is it not between -0xFFFFFFFF - 0xFFFFFFFF

Yes you are correct, and it has a reason beyond the creators of Hive's control.


Req. additional bit counts

The following 2's complement bit counts are easy enough to compute even for sign-magnitude models. As a result, I will consider adding them to NBKBinaryInteger as protocol requirements.
var nonzeroBitCount:     Int { get }
var leadingZeroBitCount: Int { get }
var mostSignificantBit:  Int { get }

I say this, but the current sign-magnitude bitWidth is ~2x slower and nonzeroBitCount is ~5x slower. Hm.

I think nonzeroBitCount and leadingZeroBitCount are slam dunks, but I'm undecided on mostSignificantBit. The flexible-width magnitude I'm working on normalizes zero to a zero-bit-width, meaning that this bit does not exist. Should it default to false?  Should it be optional? Should I require binary integers to always have nonzero bit widths?


Improve the example code included in the package

IRDIS and IFS reduction
ipython notebook
Include a dataset?

Done in 474ac7c78abb9d0fdfefc0a037607e6597c6bc7f


Mave moudle has wrong compile order, sharding-sql-test should be compiled before sharding-core

Bug Report
Mave moudle has wrong compile order.
Before report a bug, make sure you have:

Searched open and closed GitHub issues.
Read documentation: ShardingSphere Doc.

Please pay attention on issues you submitted, because we maybe need more details.
If no response more than 7 days and we cannot reproduce it on current information, we will close it.
Please answer these questions before submitting your issue. Thanks!
Which version of ShardingSphere did you use?
4.0.0-RC1-SNAPSHOT
Which project did you use? Sharding-JDBC or Sharding-Proxy?
Expected behavior
    <modules>
        <module>sharding-sql-test</module>

        <module>sharding-core</module>

        <module>sharding-jdbc</module>
        <module>sharding-proxy</module>

        <module>sharding-transaction</module>
        <module>sharding-orchestration</module>
        <module>sharding-opentracing</module>

        <module>sharding-spring</module>
    </modules>

Actual behavior
sub modules of sharding-core rely on module sharding-sql-test, compile will fail if you are doing this for the first time.

    <modules>
        <module>sharding-core</module>

        <module>sharding-jdbc</module>
        <module>sharding-proxy</module>

        <module>sharding-transaction</module>
        <module>sharding-orchestration</module>
        <module>sharding-opentracing</module>

        <module>sharding-spring</module>
        <module>sharding-sql-test</module>
    </modules>

Reason analyze (If you can)
Steps to reproduce the behavior, such as: SQL to execute, sharding rule configuration, when exception occur etc.
Example codes for reproduce this issue (such as a github link).

Can you explain the reason for close this issue?

Can you explain the reason for close this issue?

In fact, yesterday night, i did ran into error when i try to compile the code. But this morning, after i post this issue , i can't reproduce this problem again (clean related jars in  local maven repo and compile).
Though sharding-sql-test is the last one  in the <modules> element of the parent pom.xml, it's still compiled before some modules of sharding-core.
[INFO] Reactor Build Order:
[INFO]
[INFO] sharding-sphere
[INFO] sharding-core
[INFO] sharding-core-api
[INFO] sharding-core-common
[INFO] sharding-core-parse
[INFO] sharding-core-parse-spi
[INFO] sharding-sql-test
[INFO] sharding-core-parse-common
[INFO] sharding-core-parse-mysql
[INFO] sharding-core-parse-postgresql
[INFO] sharding-core-parse-oracle
[INFO] sharding-core-parse-sqlserver

Before, i think module building order is just the same as what we define in  element of the parent pom.xml.
And after Googling, maven is smarter than I thought.
https://stackoverflow.com/questions/18629897/maven-build-order-multiple-modules
http://maven.apache.org/guides/mini/guide-multiple-modules.html

Reactor Sorting
Because modules within a multi-module build can depend on each other, it is important that the reactor sorts all the projects in a way that guarantees any project is built before it is required.
The following relationships are honoured when sorting projects:
a project dependency on another module in the build
a plugin declaration where the plugin is another module in the build
a plugin dependency on another module in the build
a build extension declaration on another module in the build
the order declared in the  element (if no other rule applies)


[do not merge]Content/guides/webpack and typescript

Read and sign the CLA. This needs to be done only once. PRs that haven't signed it won't be accepted.
Make sure your PR complies with the writer's guide.
Read through the PR diff carefully as sometimes this can reveal issues. The work will be reviewed, but this can save some effort.
Remove these instructions from your PR as they are for your eyes only.

@dear-lizhihua 这个PR不要merge，是为了对比中文diff创建的。我改动的地方以comment的形式标出来了( #231 )。


HHH-13280 - HINT_PASS_DISTINCT_THROUGH does not work if 'hibernate.use_sql_comments' property is enabled

https://hibernate.atlassian.net/browse/HHH-13280

I've added some failing tests.
If the comment contains "distinct", then it is removed from the comment only. The default comment for a query is the HQL query itself, so this fix does not work unless an explicit comment is added that does not have "distinct" in it.
Also, if there is a subquery with "distinct" and an explicit comment that does not have "distinct", then "distinct" is removed from the subquery.
IIUC, "distinct" should only be removed from the outer-most select.
IMO, "distinct" should not be removed from a comment. WDYT?

@gbadner Thanks for the tests. Your assumptions are correct. We need to rework it to make sure the "distinct" is never removed from a comment or hint and only for the top-level select clasue.

Hi all, is this issue please planned to be fixed in the near future? Currently I cannot have comments in my hibernate logs because of this. Thanks a lot! :)

@japro If you provide an update of this PR that fixes this in a proper way (address Guillaume's comments) rebased on top of main, I can look into this.


Allow vite 5

Allow vite 5 in package.json

This seems to break the test suite. We're working on this and hope to have a Vite 5 compatible version soon.


Segmentation fault (core dumped)

Has anyone meet this problem before?
****@*******:~/Documents/flownet2-tf$ python -m src.flownet2.test --input_a data/samples/0img0.ppm --input_b data/samples/0img1.ppm --out ./
Segmentation fault (core dumped)

I meet the same problem when run: python -m src.flownet2.train.
I think the problem may be in slim.learning.train() in net.py. But I don't know how to fix it.
.

Did you already consider issue #10 ?
If you haven't already, try compiling with gcc/g++ 4.8.

I have the same problem when I run src.flownet2.test with gcc/g++ 4.8 but it does not  fix it.  Have you solved this problem ？@zengzhi2015

Hi guy @zengzhi2015, @BonjourLemon, Having the same issue here. Did you solved the problem?


Make Whatsapp Bot

Make a bot which will automate some commands in whatsapp

I would like to try making this under jwoc22. Although I do have much prior experience in bot development I can easily work on Python.

@neelshah2409  I would like to work on this issue under JWoC 2022. Kindly assign it to me. I can make a user-friendly Whatsapp Bot with my prior knowledge of making bots in hackathons in python / js.

I would like to try making this under jwoc22. Although I do not have much prior experience in bot development I can easily work on Python.

If this issue is still open, I would like to work on this issue

@r3yc0n1c @Ankit62042 you both can work on the issue as there may be number of bots for whatsapp

Make a bot which will automate some commands in whatsapp

I would like to work on the issue


quick return when scan package is empty

What is the purpose of the change
quick return when scan package is empty
Brief changelog
XXXXX
Verifying this change
XXXXX
Follow this checklist to help us incorporate your contribution quickly and easily:

[x] Make sure there is a GITHUB_issue field for the change (usually before you start working on it). Trivial changes like typos do not require a GITHUB issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
[ ] Format the pull request title like [Dubbo-XXX] Fix UnknownException when host config not exist #XXX. Each commit in the pull request should have a meaningful subject line and body.
[ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
[ ] Write necessary unit-test to verify your logic correction, more mock a little better when cross module dependency exist. If the new feature or significant change is committed, please remember to add sample in dubbo samples project.
[ ] Run mvn clean install -DskipTests=false & mvn clean test-compile failsafe:integration-test to make sure unit-test and integration-test pass.
[ ] If this contribution is large, please follow the Software Donation Guide.

two question for this PR:

what's purpose for this change?
why this change need go to 3.x-dev branch instead of master branch

two question for this PR:

what's purpose for this change?
why this change need go to 3.x-dev branch instead of master branch

1、just for better code style?
2、i am not very cleaer whether it can be submitted to master branch


Does Quiver support multi-node training now?

I've noticed that Quiver support training in multi-gpu in single machine. I wonder if it supports training in multi-gpu in multiple machines as well?

I think it has not been released yet, however, they have worked on this code.

@prenc Do you mean that they've open-sourced the test code, however, the package itself may not operate with the test code?


<IP_ADDRESS> Support

I want to know if this mod will be updated to <IP_ADDRESS>.

Most likely not, Patch Manager has been released in the meantime and can do the same thing.


.gitignore and .vscode

Should the .gitignore file include an entry for .vscode?

IMO, the answer is yes.  All of my DX projects are doing this


initial value of markdown is not represented correctly in editor

When I set the initial value from markdown, it is not properly formatted. What am I doing wrong?
initial content:

result in DB (saved as markdown)

Getting data back for editing:
#using class as I can have multiple instances of editor on page

{{item.text}}

#initializing editor
$(".tui-editor").each(function () {
    //check if something is in the div, and if so set this to intial content
    var content = $(this).html();

    $(this).toastuiEditor({
        usageStatistics: false,
        height: '250px',
        initialEditType: 'wysiwyg',
        setValue: content
    });
});

Result:

Oh, never mind, it's a jQuery thing:
solution is to use .text() instead if .html()
var content = $(this).text();


Normalize users email field to lowercase

The users email field currently allows lowercase and uppercase mails. We should normalize it to be lowercase so that the function retrieve_user_by_email (used in tokenCreate mutation) can do .filter(email= inside it, instead of .filter(email__iexact= which is currently the case and it may cause a massive sequence scan.
If we cannot make it with normalization, we can create an index like so
CREATE INDEX CONCURRENTLY IF NOT EXISTS email_upper_idx ON account_user (upper(email));

and it will solve the issue until a proper solution (that would be normalization of emails).
To be discussed: how to handle a case when there is a huge database and it would have duplicates (like<EMAIL_ADDRESS>and ADMIN@example.com).

I added index as temporary fix in:
https://github.com/saleor/saleor/pull/14914


Tutorial notebook documentation issues

Notebooks in docker/luna_tutorial/vmount/notebooks have various issues with their Markdown documentation

NB1 Grammar issues & a block of text referencing missing code block to add patient IDs (fixed in #333)
NB2 (File version) is missing documentation on step by step Luna CLI tiling
NB2 (Waystation version) is missing a lot of general documentation (and also does not work and throws errors with the API)

More info about missing/incorrect documentation will be referenced in this issue as it is discovered, along with relevant commits/PRs addressing these issues. For now, pr #333 fixes NB1's current problems.

Currently, NB3 (model training) doesn't even run past the first step for me, so I will go over that once it's fixed.


v0.0.12-backport: Do not merge.

Do not merge.

released in https://github.com/komuw/kama/releases/tag/v0.0.12-bacckport


Reflection Missing Symbols

I am using some minimal reflection in my (closed-source) app to implement System.Resources.ResourceManager. The use of reflection is limited to the following calls:

Type.GetTypeInfo()
TypeInfo.Assembly getter
Assembly.GetName()
AssemblyName.Name getter

However, when I try to compile this code with CoreRT, ILC finishes successfully, but the linker gives undefined-symbol errors for the following methods:
System_Private_Reflection_Core_System_Reflection_Runtime_TypeInfos_NativeFormat_NativeFormatRuntimeNamedTypeInfo__get_Assembly
System_Private_Reflection_Core_System_Reflection_Runtime_TypeInfos_RuntimeBlockedTypeInfo__get_Assembly
System_Private_Reflection_Core_System_Reflection_Runtime_TypeInfos_RuntimeNoMetadataNamedTypeInfo__get_Assembly
System_Private_Reflection_Core_System_Reflection_Runtime_TypeInfos_RuntimeConstructedGenericTypeInfo__get_Assembly
System_Private_Reflection_Core_System_Reflection_Runtime_TypeInfos_RuntimeHasElementTypeInfo__get_Assembly
System_Private_Reflection_Execution_Internal_Reflection_Execution_ConstraintValidator_InstantiatedTypeInfo__get_Assembly
System_Private_Reflection_Core_System_Reflection_Runtime_TypeInfos_RuntimeGenericParameterTypeInfo__get_Assembly

This all worked several months ago, before the switch to csproj. Is there anything I can do to resolve this? Thanks!

This looks like a bug with LazilyBuiltVTableSliceNode. One part of the system believes that TypeInfo.Assembly method is necessary, but other part believes that it is not necessary.
You can try working around this by calling type(MyProgram).GetTypeInfo().Assembly at the start of your program. Also, if you can distill this to a repro that you can share with us, we will be happy to take a look?
@MichalStrehovsky Any other ideas?

@jkotas No good. I have added typeof(Program).GetTypeInfo().Assembly at the top of my Main method, and it still fails to link.

I have seen linking failures similar to this if there's a generic virtual method (or generic interface method) somewhere in the compiled app. We don't support them yet and the blocking we have in place is subpar. We need a repro, otherwise this is not very actionable.

@MichalStrehovsky I'll try to create a repro for you sometime tomorrow.

I think I've got a repro. Using a copy of CoreRT as of commit  2c70a97c46c7eaba1df56ce687d8a87682c61fa1 (I am currently using the tip of the PR branch for #2390, but that commit is the last change that affected ILC itself), attempt to compile the following program with ILC:
using System;
using System.Reflection;

class Program
{
    static void Main(string[] args)
    {
        // This one line is really all you should need.
        Assembly asm = typeof(Program).GetTypeInfo().Assembly;
    }
}

I have found that if TypeInfo.get_Assembly is referenced anywhere in the program, it will trigger this bug. Let me know if this helps!

Yup, this helps. This is because TypeInfo.get_Assembly was moved to System.Type as part of the NetStandard 2.0 work. The compiler expects a callvirt to the method on System.Type, but sees a callvirt to the one on System.Reflection.TypeInfo instead. This is legit IL, but the C# compiler would never have generated it if we were compiling against the NetStandard 2.0 reference assemblies.
We should keep this open because this is legit IL and we should support it, but it's probably lower priority (until we start looking at supporting Visual Basic, I guess).
You can work around this by adding a typeof(Program).GetTypeInfo().Module line somewhere. get_Module internally calls get_Assembly (but through System.Type) and makes all the dependencies light up.

it's probably lower priority (until we start looking at supporting Visual Basic, I guess)

It is a problem for all existing code (nuget libraries in particular) that is compiled against 1.X NetStandard.


Confidence intervals for skill scores and classification metrics

I think the best way to do this is to build a dataframe of statistics for each algorithm, then I can bootstrap the results!

The question is whether or not to do this against average climatology or a random sample?


Add GrafanaLabs and FarmersEdge to Adopters

@Dieterbe @tanner-bruce please review

Coverage remained the same at 100.0% when pulling ed70c22afa91c6406d1931b2fc543025b6f8d48c on more-adopters into d17b46832a3f4043a22f00aa0d11e2f2f39f93bb on master.

LGTM

:+1:


Remove k8s 1.22 "beta" prior to the 2.6 release

Need to remove the beta for kubernetes 1.22 prior to the 2.6 release.

@Jono-SUSE-Rancher Can we clarify a bit on what the scope is on 1.22? To make k8s 1.22 working we need a change in RKE and in KDM, if this is in, who is going to use it until we revert it? Would a custom build also be good enough for the goal?

Now that k8s 1.22 has officially been released, we can close this issue as we will introduce support for k8s 1.22 GA when it's ready.
Created this ticket to track k8s 1.22 into RKE1: https://github.com/rancher/rancher/issues/34054


Archive ladi-tutorial in Zenodo with a DOI #

Make ladi-tutorial citable using Zenodo  (https://guides.github.com/activities/citable-code/). Once archived, update the README with the DOI badge

Addressed by d555ff7


Add integration event for the Danish OIOUBL app

Describe the request
Add an a new event  "OnAfterExportXMLFile(SalesInvoiceHeader, TempBlob, FileName);" in the codeunits:
codeunit 13636 "OIOUBL-Export Sales Invoice"
codeunit 13637 "OIOUBL-Export Sales Cr. Memo
location is shown in the following for the invoice codunit, but must also be implemented similar for the cr.memo:
procedure ExportXML(SalesInvoiceHeader: Record "Sales Invoice Header");
var
SalesInvHeader2: Record "Sales Invoice Header";
ElectronicDocumentFormat: Record "Electronic Document Format";
OIOUBLManagement: Codeunit "OIOUBL-Management";
TempBlob: Codeunit "Temp Blob";
FileOutStream: OutStream;
FileName: Text[250];
begin
TempBlob.CreateOutStream(FileOutStream);
CreateXML(SalesInvoiceHeader, FileOutStream);
    SalesSetup.Get();

    FileName := ElectronicDocumentFormat.GetAttachmentFileName(SalesInvoiceHeader, SalesInvoiceHeader."No.", 'Invoice', 'xml');
    OIOUBLManagement.UpdateRecordExportBuffer(SalesInvoiceHeader.RecordId(), TempBlob, FileName);

    OIOUBLManagement.ExportXMLFile(SalesInvoiceHeader."No.", TempBlob, SalesSetup."OIOUBL-Invoice Path", FileName);

OnAfterExportXMLFile(SalesInvoiceHeader, TempBlob, FileName);
    SalesInvHeader2.Get(SalesInvoiceHeader."No.");
    SalesInvHeader2."OIOUBL-Electronic Invoice Created" := true;
    SalesInvHeader2.Modify();

    Codeunit.Run(Codeunit::"Sales Inv.-Printed", SalesInvHeader2);
end;

Additional context
Please add an event to allow postprocssing after document has been exported in the following CUs:
codeunit 13636 "OIOUBL-Export Sales Invoice"
codeunit 13637 "OIOUBL-Export Sales Cr. Memo
Internal work item: AB#536966

Availability update: We will publish a fix for this issue in the next update for release 24.
Build ID to track: 20287.

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.
Build ID: 20285.


how to use it in yolov5

You should be able to replace any pooling layers with the corresponding layers from the repo. You can have a look at the Usage section of the README to call either the functional or Module interfaces.


protocol service Stop() fix

Add @ryanolson as part of the fix is on nvRPC

Subsumed by https://github.com/NVIDIA/tensorrt-inference-server/pull/159


Add logical operators and expressions

This PR adds the possibility of having logical operators and expressions in a formula.
Possible operators are >, <, >=, <=, = and !=, the meaning of which is trivial.
Since the final goal is to use those operators in a mathematical formula, the result of a logical expression will always return a number: either 1 (when the expression is true) or 0 (when it is false), which can easily be used in a formula to achieve conditional calculations.
An example of a logical expression used in a formula is:
const f1 = new Formula( '[price] * ( 1 + 0.25 * ( [weight] > 10 )' );

const res = f1.evaluate( { price: 100, weight: 12.4 } );
console.log( res );

// res = 125

This would be essential to implement some logical functions – such as IF() – in the same form Microsoft Excel or Google Sheets do:
const f1 = new Formula( 'IF( [weight] > 10, 1.25, 1 ) * [price]' );

f1.IF = ( value, ifTrue, ifFalse ) => {
    return value ? ifTrue : ifFalse;
};

const res = f1.evaluate( { price: 100, weight: 12.4 } );
console.log( res );

// res = 125

This is a useful addition, as it opens the possibility to implement conditional logic, thanks.
Would you mind:

adding some Unit tests (see spec/specs/ for examples)
merging the develop branch to resolve merge conflicts

Then I will consider your addition to be merged.
Many thanks for your contribution.

Hi, @bylexus,
Thanks for reviewing my PR so swiftly. I followed both of your recommendations. While adding Unit tests, I spotted a few missing declarations for the new Logical Expression, which I added now, and rebuilt all the distribution files.
Let me know if the tests are enough and if there is anything else I can do.

OK, I have decided that your solution and your implementation of the operator precedence is correct. I will merge your contribution into my develop branch, add some documentation and examples, and release a new version.
many thanks for your useful contribution.

Thanks for reviewing the PR so thoroughly. I am glad that, in the end, it worked out as intended!


input data normalisation

Can you clarify what preprocessing is expected for the RNA assay (gene expression) data.
Is the expected input integer (raw) counts or normalised&log transformed counts or similar?
this isn't totally clear in the vignette.

I also have a similar question. I am not sure if the returned seurat object contaions the data used for genearlized liear regression, usch as vecotor from RNA assay, or ATAC assay.

I also have a similar question. I am not sure if the returned seurat object contaions the data used for genearlized liear regression, usch as vecotor from RNA assay, or ATAC assay.

Here is an example. I can access normalized expression level of PAX6 but I am not sure how to access value of "chr1-911275-911316".

I also have a similar question. I am not sure if the returned seurat object contaions the data used for genearlized liear regression, usch as vecotor from RNA assay, or ATAC assay.

 Here is an example. I can access normalized expression level of PAX6 but I am not sure how to access value of "chr1-911275-911316".

@bjstewart1 Using this as an exmaple, you question is what is the values for PAX6. Are they norlized values, scaled values, or raw counts.

I also have a similar question. I am not sure if the returned seurat object contaions the data used for genearlized liear regression, usch as vecotor from RNA assay, or ATAC assay.

 Here is an example. I can access normalized expression level of PAX6 but I am not sure how to access value of "chr1-911275-911316".

@bjstewart1 Using this as an exmaple, you question is what is the values for PAX6. Are they norlized values, scaled values, or raw counts.

no my question is what are the input data for the tool. Are the RNA integer counts meant to be processed to normalised/log transformed values?

Hi @bjstewart1, currently Pando would expect log-normalized data (for RNA) and tf-idf-normalized data (for ATAC) as input and would also generally use that by default if it's in the data slot of your assay. I've thought about implementing an option to run it on raw counts though - essentially that would require other noise models for the GLMs and accounting for library size covariates in the models.

Thanks @joschif really helpful .. -  can I suggest that you make it a bit clearer what these input requirements are in the readme/vignettes?

Thank you for developing Pando, it's a really nice and convenient tool for multiomic data analysis!
Following up on the above questions about input data: which assays do you recommend to use in Pando when dealing with an integrated object of multiple multiome libraries from different samples generated by Signac/Seurat pipelines (e.g. rlsi for ATAC, and sctransform -> anchor-based integration for RNA, etc.) ?
Can Pando take the SCT assay or the "integrated" assays for instance or do you recommend always starting from  log-normalized "RNA" and "Peaks" assays? Is Pando's process affected by sequencing depth differences between samples for instance? Is there any preprocess steps you recommend in that case?
Thank you for your help!

Hi @joschif ,
Thanks for your response above! I have a follow-up question. When pre-processing data, do you suggest standard QC and filtering for min_cells and min_genes? I believe this tutorial has not performed QC steps to filter out genes since I see ~31k genes for RNA data. Could you please clarify if we need to keep all the genes, then normalize and get log1p?
Thanks,
Elham


Windows Support

Any objections to a PR adding windows support?

That would certainly be much appreciated (I wouldn't be able to test it though).

@LaurentMazare I think it actually does work.
I removed -Wl,-rpath flags from build script and basic example works just fine.
When I tried to compile reinforcement-learning or cifar example, I got this error:
libtch-f6fb773124c0e3c3.rlib(tch-f6fb773124c0e3c3.2d5yw0723k7kqxuq.rcgu.o) : error LNK2019: unresolved external symbol ato_rms_prop referenced in function _ZN3tch8wrappers9optimizer10COptimizer8rms_prop17h85fbf712274e8a21E
C:\Users\Pah\tch-rs\target\debug\examples\cifar-7657a28e1908fa9c.exe : fatal error LNK1120: 1 unresolved externals

Just commenting out references to rms_prop, RmsProp, ato_rms_prop made it compile:
     Running `target\debug\examples\reinforcement-learning.exe pg`
action space: 2
observation space: [4]
epoch: 0   episodes: 199   avg reward per episode: 25.20
[...]
epoch: 41  episodes: 27    avg reward per episode: 189.48

On the other hand a2c:
     Running `target\debug\examples\reinforcement-learning.exe a2c`
usage: main pg|a2c|a2c-sample|ppo|ppo-sample
usage: main pg|a2c|a2c-sample|ppo|ppo-sample
usage: main pg|a2c|a2c-sample|ppo|ppo-sample
[...]
usage: main pg|a2c|a2c-sample|ppo|ppo-sample
usage: main pg|a2c|a2c-sample|ppo|ppo-sample
usage: main pg|a2c|a2c-sample|ppo|ppo-sample

which might be a problem with conda, in PowerShell terminal (not activated virtualenv) error is different:
     Running `target\debug\examples\reinforcement-learning.exe a2c`
Error: PyErr { ptype: <class 'ImportError'>, pvalue: Some(ImportError('DLL load failed: The specified module could not be found.')), ptraceback: Some(<traceback object at 0x000001BDDB4B3F08>) }

Setting Py_ENABLE_SHARED didn't help. The pg example works in both terminals.
Thanks for your awesome work, I appreciate it a lot.

The rms-prop bit is quite strange, I've never seen anything like this so I would just suggest checking whether the _ZN3tch8wrappers9optimizer10COptimizer8rms_prop17h85fbf712274e8a21E symbol is properly defined in one of the libtorch dll that you installed (I imagine that you're using v1.1.0 as recommended, e.g. for the cpu version https://download.pytorch.org/libtorch/cpu/libtorch-win-shared-with-deps-v1.1.0.zip ).
For a2c, you probably already installed the atari gym envs in your conda environment. Maybe you could check that they work well with python, e.g. for a2c you can try loading "SpaceInvadersNoFrameskip-v4".
Sorry for not being more helpful, I don't have a windows box - I'll try to get access to one. If you find ways to fix this, happy to get a PR or some change suggestions.

I just downloaded the latest libtorch before https://download.pytorch.org/libtorch/cu100/libtorch-win-shared-with-deps-latest.zip I hope it is really the latest release, 142c973f4179e768164cd578951489e89021b29c.
I have gym installed but turns out: No module named 'atari_py'.
Installed atari_py, can render SpaceInvadersNoFrameskip-v4 from REPL but the output from reinforcement-learning example didn't change.

Just to mention that I gave this a try using windows subsystem for linux today and everything worked out of the box. If that's an option for you this is probably easier to set up.
Below are the commands that I ran for an ubuntu version.
# install rust
curl https://sh.rustup.rs -sSf | sh
# install a bunch of dependencies
sudo apt-get install gcc libssl-dev pkg-config python python3 python3-pip libpython3.6-dev
# install the python gym environment
pip install gym pillow gym[atari] --user

git clone https://github.com/LaurentMazare/tch-rs.git
cd tch-rs
cargo run --example reinforcement-learning --features=python a2c

On Windows machine, it was a Python 3.7.
I'm happy to help if there is anything you would like me to try.

Not sure where we stand on this issue but I can confirm that the project can be compiled on Windows 10 by just:

Commenting out the lines adding the -Wl,-rpath flags in the torch-sys/build.rs file (as advised above)
Adding a LIBTORCH environment variable  pointing to the downloaded libtorch folder

The following warnings appeared during the torch-sys compilation:
warning: cl : Command line warning D9002 : ignoring unknown option '-std=c++11'
warning: cl : Command line warning D9002 : ignoring unknown option '-fPIC'

Then, in order to run the tests successfully, the last step is to:

Add the libtorch/lib folder to the user Path

All tests successfully ran afterwards.

It's nice that it's not too hard to get it work. I haven't done anything on this as I don't have a windows box but if anyone wants to craft a PR to add support for this, that would be a welcome addition.

I am not familiar with the default Windows compiler at all. A flag adjustment in torch-sys/build.rs based on the running OS would work but there should be new flags to optimise the code adequately in Windows I would assume.

I feel that having flags that work reasonably well but are not very optimized could already be interesting. If I get my hands on a windows box at some point I may give it a try but that's not very likely to happen in the near future.

@vegapit @crackcomm @paddyhoran My windows patch is merged hours ago. Could you see if it's reproducible on your machine? The instruction is left on README.
Note that the libtorch should be manually downloaded, and set env vars explicitly to make it work. It's because of the limitations that Windows does not support rpath or runpath. I think the procedure is not much harder than Linux.

@jerry73204 I cloned and cargo test passed. I tried basics example, fully functional.
Seems you did a great job, thanks for your effort!

@vegapit @crackcomm @paddyhoran My windows patch is merged hours ago. Could you see if it's reproducible on your machine? The instruction is left on README.
Note that the libtorch should be manually downloaded, and set env vars explicitly to make it work. It's because of the limitations that Windows does not support rpath or runpath. I think the procedure is not much harder than Linux.

All tests on the repo ran successfully on Windows 10 with Rust 1.37. Neat work.

@jerry73204 great work!  Thank you!

Also kudos to @jean-airoldie for adding some windows CI support!


RuntimeError: CUDA error: an illegal memory access was encountered

I am facing this issue while using zigzag_ring_attn with 128k context length. Has anyone run into the same problem?
[rank0]:   File "/app/c2j-long-context-model-training/EasyContext/easy_context/zigzag_ring_attn/monkey_patch.py", line 69, in new_decoder_forward
[rank0]:     hidden_states, self_attn_weights, present_key_value = self.self_attn(
[rank0]:   File "/opt/conda/envs/ai/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
[rank0]:     return self._call_impl(*args, **kwargs)
[rank0]:   File "/opt/conda/envs/ai/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1603, in _call_impl
[rank0]:     result = forward_call(*args, **kwargs)
[rank0]:   File "/opt/conda/envs/ai/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 469, in forward
[rank0]:     attn_output = self._flash_attention_forward(
[rank0]:   File "/app/c2j-long-context-model-training/EasyContext/easy_context/zigzag_ring_attn/monkey_patch.py", line 29, in new_flash_attn_forward
[rank0]:     attn_output = zigzag_ring_flash_attn_func(
[rank0]:   File "/opt/conda/envs/ai/lib/python3.10/site-packages/ring_flash_attn/zigzag_ring_flash_attn.py", line 312, in zigzag_ring_flash_attn_func
[rank0]:     return ZigZagRingFlashAttnFunc.apply(
[rank0]:   File "/opt/conda/envs/ai/lib/python3.10/site-packages/torch/autograd/function.py", line 574, in apply
[rank0]:     return super().apply(*args, **kwargs)  # type: ignore[misc]
[rank0]:   File "/opt/conda/envs/ai/lib/python3.10/site-packages/ring_flash_attn/zigzag_ring_flash_attn.py", line 202, in forward
[rank0]:     out, softmax_lse = zigzag_ring_flash_attn_forward(
[rank0]:   File "/opt/conda/envs/ai/lib/python3.10/site-packages/ring_flash_attn/zigzag_ring_flash_attn.py", line 59, in zigzag_ring_flash_attn_forward
[rank0]:     out, lse = update_out_and_lse(
[rank0]:   File "/opt/conda/envs/ai/lib/python3.10/site-packages/ring_flash_attn/utils.py", line 44, in update_out_and_lse
[rank0]:     slice_out, slice_lse = _update_out_and_lse(
[rank0]: RuntimeError: The following operation failed in the TorchScript interpreter.
[rank0]: Traceback of TorchScript (most recent call last):
[rank0]:   File "/opt/conda/envs/ai/lib/python3.10/site-packages/ring_flash_attn/utils.py", line 24, in _update_out_and_lse
[rank0]:     # For additional context and discussion, please refer to:
[rank0]:     # https://github.com/zhuzilin/ring-flash-attention/pull/34#issuecomment-2076126795
[rank0]:     out = out - F.sigmoid(block_lse - lse) * (out - block_out)
[rank0]:                 ~~~~~~~~~ <--- HERE
[rank0]:     lse = lse - F.logsigmoid(lse - block_lse)
[rank0]:   File "/opt/conda/envs/ai/lib/python3.10/site-packages/torch/nn/functional.py", line 2013, in sigmoid
[rank0]:     See :class:`~torch.nn.Sigmoid` for more details.
[rank0]:     """
[rank0]:     return input.sigmoid()
[rank0]:            ~~~~~~~~~~~~~ <--- HERE
[rank0]: RuntimeError: CUDA error: an illegal memory access was encountered

With smaller context length I got
W0904 15:27:23.932000<PHONE_NUMBER>7216 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 261407 closing signal SIGTERM
W0904 15:27:23.932000<PHONE_NUMBER>7216 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 261408 closing signal SIGTERM
W0904 15:27:23.932000<PHONE_NUMBER>7216 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 261410 closing signal SIGTERM
E0904 15:27:34.716000<PHONE_NUMBER>7216 torch/distributed/elastic/multiprocessing/api.py:826] failed (exitcode: -11) local_rank: 2 (pid: 261409) of binary: /dccstor/udit/env/easy_context/bin/python
Traceback (most recent call last):
  File "/udit/env/easy_context/bin/accelerate", line 8, in <module>
    sys.exit(main())
  File "/udit/env/easy_context/lib/python3.10/site-packages/accelerate/commands/accelerate_cli.py", line 48, in main
    args.func(args)
  File "/udit/env/easy_context/lib/python3.10/site-packages/accelerate/commands/launch.py", line 1091, in launch_command
    deepspeed_launcher(args)
  File "/udit/env/easy_context/lib/python3.10/site-packages/accelerate/commands/launch.py", line 787, in deepspeed_launcher
    distrib_run.run(args)
  File "/dccstor/udit/env/easy_context/lib/python3.10/site-packages/torch/distributed/run.py", line 870, in run
    elastic_launch(
  File "/udit/env/easy_context/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 132, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "/udit/env/easy_context/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 263, in launch_agent
    raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:
========================================================
EasyContext/train.py FAILED
--------------------------------------------------------
Failures:
  <NO_OTHER_FAILURES>
--------------------------------------------------------
Root Cause (first observed failure):
[0]:
  time      : 2024-09-04_15:27:23
  host      : cccxc614.pok.ibm.com
  rank      : 2 (local_rank: 2)
  exitcode  : -11 (pid: 261409)
  error_file: <N/A>
  traceback : Signal 11 (SIGSEGV) received by PID 261409
========================================================


[Bug] TypeError: 'str' object is not callable, when unnormalize with customize mean and std

torchshow.set_image_mean([0., 0., 0.])
torchshow.set_image_std([1., 1., 1.])
torchshow.show(img)

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)

File /somepath/python3.8/site-packages/torchshow/visualization.py:289, in vis_image(x, unnormalize, **kwargs)
    [287]     if user_std == None:
    [288]         user_std = [1.] * x.shape[-1] # Initialize std to 1 if not specified.
--> [289]     x = unnormalize(x, user_mean, user_std)
    [291] elif unnormalize=='auto':
    [292]     x = auto_unnormalize_image(x)

TypeError: 'str' object is not callable

It seems to be caused by the variable name and method name being the same 'unnormalize'.
And this case should be added to the test.

Thank you for spotting this bug! I really forgot this function since I barely use it myself.


What is the purpose of on_new_message and where it is defined. I need to check the source code

What is the use of On_new_message . Where is it defined.  I need to know the Source code.
Rasa-core 0.11.12
Python 3.5
OS windows 10

Thanks for raising this issue, @tmbo  will get back to you about it soon.

That is the function that gets passed to the input channels. The channels are supposed to call that function for every new incoming user message. The function we pass in as on_new_message is agent.handle_message (https://github.com/RasaHQ/rasa_core/blob/master/rasa_core/agent.py#L281).

Thanks Tom,
One more question how did you made it callable.
Did u define
on_new_message=agent.handle_message
Or some other way.

Yes that should be the way it is implemented


pulsar manager didn't start postgres. So it didn't start to work with

Describe the bug
If you have the problem that one will have a 404 Error in the WebConsole during Login on pulsar manager. I have installed it from
https://pulsar.apache.org/docs/3.1.x/helm-deploy/
During my debug, I found out that this is ok. Because the Backend didn't start.
After connecting to the shell of ird-pulsar-pulsar-manager I've seen in the tail -f pulsar_manager.log of the spring instance that it can't connect to postgres on <IP_ADDRESS>. There wasn't any available.
Installing apt-get update && apt-get -y install procps lsof vim and I was possible to figure out that there was nothing listening on port 7750 or 5432
So I've created my own postgres instance somewhere, create the schema with this link and startet the helm install like
helm install \
    --values pulsar-ird/values.yaml \
    --set initialize=true \
    --set proxy.ports.http=8080 \
    --set proxy.securityContext.runAsUser=0 \
    --set proxy.securityContext.runAsGroup=0 \
    --set proxy.service.loadBalancerIP=<IP_ADDRESS> \
    --set pulsar_manager.admin.user=postgres \
    --set pulsar_manager.admin.password=POSTGRES_PASSWORD \
    --set pulsar_manager.configData.URL=jdbc:postgresql://<IP_ADDRESS>:5432/pulsar_manager \
    --set pulsar_manager.service.loadBalancerIP=<IP_ADDRESS> \
    --namespace pulsar \
    ird-pulsar apache/pulsar

I've to switch the proxy to 8080 because bind to 80 wasn't possible. Also the possible "solutions" setting the runAs* parameter to root didn't help. So I take port 8080.
After the helm command is finished, I connect to the shell again and tail -f pulsar_manager.log again. Look out for Hikari Logs. That went well.
After the spring service started up and was listening on 7750 I was possible to create an admin user!
Yes! One need this. It's not directly documented by the how tos!! Just during my investigations at least here in the issue board.
CSRF_TOKEN=$(kubectl get secret --namespace pulsar ird-pulsar-token-admin -o jsonpath="{.data.TOKEN}" | base64 -d)
curl \
    -H "X-XSRF-TOKEN: $CSRF_TOKEN" \
    -H "Cookie: XSRF-TOKEN=$CSRF_TOKEN;" \
    -H 'Content-Type: application/json' \
    -X PUT http://THE_MANAGER_IP:9527/pulsar-manager/users/superuser \
    -d '{"name": "admin", "password": "BirnenBurner123_", "description": "test", "email"<EMAIL_ADDRESS>
Then you should be able to login with admin:BirnenBurner123_
And fun-fact : pulsar_manager.admin.user and pulsar_manager.admin.password are the db access credits (may be also some credits for other things..)
Bug Report 1 : In the end I am still struggling to set my desired internal IPs. They got ignored and always got new ip's on re-deploys.
Bug Report 2 : What is the definition of "production ready" ? And why uses the pulsar_manager by default on internal postgres instance (which is not starting up) in a non-stateful-set?
Bug Report 3: The helm chart is not compatible kubernetes api v1.27. It's using deprecated (beta) apis and webhooks from within prometheus

To Reproduce
As written up.
Expected behavior

That a database is spawned up,
that it's statefull and
that I can set ip-addresses
would be easier to find the spring logs from pulsar manager

Additional context
Deployed in a GKE 1.27 Cluster.

I'm having the same issue with DB start...local postgresql DB inside manager pod is not running.

Try to configure as describe an external Database. I've switched to installations on VM's. There is also the default an internal postgres (which really starts..) but on restart of the manager you loose your configured UI. But you can also configure an external database.

Yeah..I'll have to go with external DB. I've managed to get running internal postgres DB by using newer manager image (helm is using 0.3 which does not load DB). But I can see your point. If no Volume is provisioned for the manager pod it is useless for production so only choice is to use external DB

Fixed by #457


[pr] release version to 0.0.24

@joanhey @dvershinin
A new release 0.0.24 that 's some of the bug has been fixed.


md

Please use the following bug reporting template to help produce actionable and reproducible issues. Please try to ensure that the reproduction is minimal so that the team can go through more bugs!

A brief description

Expected results

Actual results (with terminal output if applicable)

Your Windows build number

Steps / All commands required to reproduce the error from a brand new installation

Strace of the failing command

Required packages and commands to install

See our contributing instructions for assistance.

Looks like this was posted accidentally.  Could you add a little more information and reopen please?


A UPS is unnecessary and adds to the complexity of a home staking setup

Summary
So, I get the value of a uninterrupted power supply (UPS). They are vital when running mission-critical infra where downtime is absolutely unacceptable and to be avoided at all costs.
I would argue that in the case of a home validator setup, they are an unnecessary addition due to the added complexity of another 2 components (UPS, smartplug) in the setup.
I acknowledge that these are not listed as "required" but "recommended", but this still unnecessarily complicates what could otherwise be a very simple setup (see illustration below).
Also consider the environmental impact of including battery infrastructure (Lead Acid) and try to think about reducing Ethereum's contributions to sustainability.
Let's not forget that the Ethereum network is designed from the bottom-up to be resilient due to its distributed nature, and thus building moar complexity into individual nodes is in some ways counterintuitive.
From a cost/benefit perspective, it is also highly questionable whether the cost of the additional components is at all necessary below a certain number of validators, given the limited punitive nature of missed attestations.
An illustration
Taking the setup without the UPS, there are 5 components required in addition to your existing home router (Motherboard, SSD, RAM, Power, Ethernet), all of which are readily available consumer products. It requires 4 connections (SSD to Motherboard, RAM to Motherboard, Motherboard to Power, Motherboard to Ethernet), all of which are well documented, and this requires no alteration to existing network infra (home router).
Adding the UPS increases the total number of components to 7, with the additional 2 items being specialist products (not to mention only compatible with Type A / Type B plugs optimised for North America). It increases the number of connections by 1 (UPS to Power) and also requires a change to a staker's existing network infrastructure (plug Router to UPS), which while being minimal, is non-zero.
This 40% increase in number of components increases the number of failure modes to consider, and the introduction of specialist region-specific hardware adds to the burden of international wannabe home-stakers.
Proposed solution
I propose to communicate the UPS as being "optional" instead of "recommended", and for the guide to split out the UPS and smart plug into a separate section of the hardware setup guide, and explain that these are for advanced users, or for people staking many Validators, and for whom missed epochs are very costly.
Alternative solution
To subtract the UPS altogether, and show the 4-component option in all its simplicity.
This can have the benefit of fully highlighting how simple it really can be to get started, without risking overwhelming novice hardware infrastructure operators with unnecessary complexity.

i like this. i've asked the illustrator to do a second 'minimal validator setup' illustration and maybe @hanniabu can add a toggle to show a minimal setup vs a comprehensive one?

@chrishobcroft the hardware page has been updated with the ability to toggle between a minimal and comprehensive setup

Nice


NonFree lib in F-Droid build

com.google.android.gms:play-services-oss-licenses is not FOSS. See https://maven.google.com/com/google/android/gms/play-services-oss-licenses/12.0.0/play-services-oss-licenses-12.0.0.pom. Could you please replace with an FOSS alternative (see https://gitlab.com/-/snippets/1896503)? Thanks!

Sorry, I'll fix it.

I changed com.google.android.gms:play-services-oss-licenses to AbountLibraries on Ver.1.6.0.

Thanks!


brew --prefix openssl returns a path to openssl@3 which doesn't exist

brew config output
HOMEBREW_VERSION: 3.3.15
ORIGIN: https://github.com/Homebrew/brew
HEAD: d603fd6426ddae27327bb51afb085653eb2d84a7
Last commit: 10 hours ago
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
Core tap HEAD: 5b940ea7dc51725612bcf86fca6e9dd5e49be546
Core tap last commit: 30 minutes ago
Core tap branch: master
HOMEBREW_PREFIX: /usr/local
HOMEBREW_CASK_OPTS: []
HOMEBREW_CORE_GIT_REMOTE: https://github.com/Homebrew/homebrew-core
HOMEBREW_MAKE_JOBS: 12
Homebrew Ruby: 2.6.8 => /usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.6.8/bin/ruby
CPU: dodeca-core 64-bit kabylake
Clang: 12.0.0 build 1200
Git: 2.35.1 => /usr/local/bin/git
Curl: 7.64.1 => /usr/bin/curl
macOS: 10.15.7-x86_64
CLT: <IP_ADDRESS>.29
Xcode: 10.3

brew doctor output
Please note that these warnings are just used to help the Homebrew maintainers
with debugging if you file an issue. If everything you use Homebrew for is
working fine: please don't worry or file an issue; just ignore this. Thanks!

Warning: The contents of the SDKs in your Command Line Tools (CLT) installation do not match the SDK folder names.
A clean reinstall of Command Line Tools (CLT) should fix this.

Remove the broken installation before reinstalling:
  sudo rm -rf /Library/Developer/CommandLineTools

Install the Command Line Tools:
  xcode-select --install

Warning: Your Xcode (10.3) is too outdated.
Please update to Xcode 12.4 (or delete it).
Xcode can be updated from the App Store.

Warning: Your Xcode (10.3) is outdated.
Please update to Xcode 12.4 (or delete it).
Xcode can be updated from the App Store.

Verification

[X] I ran brew update and am still able to reproduce my issue.
[ ] I have resolved all warnings from brew doctor and that did not fix my problem.

What were you trying to do (and why)?
I need the path to the OpenSSL libraries so I can set DYLD_LIBRARY_PATH.
What happened (include all command output)?
On all machines which are up to date, I see this:
macsup:~ admin$ brew list | grep openssl
<EMAIL_ADDRESS>macsup:~ admin$ brew --prefix openssl
/usr/local/opt/openssl@3
macsup:~ admin$ ls -ld /usr/local/opt/openssl*
lrwxr-xr-x 1 admin admin 28 Jan 28 09:23 /usr/local/opt/openssl -><EMAIL_ADDRESS>lrwxr-xr-x 1 admin admin 28 Jan 28 09:23<EMAIL_ADDRESS>-><EMAIL_ADDRESS>macsup:~ admin$

What did you expect to happen?
I expected brew --prefix openssl to return what it used to, the path to the installed OpenSSL files.  It wasn't long ago that this worked perfectly.
Step-by-step reproduction instructions (by running brew commands)
They are already above.

This is (mostly) intended behaviour. The unversioned openssl alias refers to openssl@3:
❯ brew info openssl
openssl@3: stable 3.0.1 (bottled) [keg-only]

openssl used to refer to<EMAIL_ADDRESS>but the convention in Homebrew (with two exceptions) is that the unversioned name refers to the latest version.

How can I find the path to the 1.1 version that is installed?

This is (mostly) intended behaviour. The unversioned openssl alias refers to openssl@3:
❯ brew info openssl
openssl@3: stable 3.0.1 (bottled) [keg-only]

openssl used to refer to<EMAIL_ADDRESS>but the convention in Homebrew (with two exceptions that we aim to rectify) is that the unversioned name refers to the latest version.

How can I find the path to the installed 1.1 version?

Try brew --prefix<EMAIL_ADDRESS>

Thanks, @carlocab


Allow server-side connection with kratos admin endpoints.

For security reasons, the admin endpoint should communicate only within the private network, and it will not be bothered by CORS problems.
Is there any plan to achieve this instead of current browser to server connection? Or is the current architecture more advantageous?
If you don't mind, I'll contribute to this re-architecture.
version: v1.2.0

Hi @knaoe ,
that's for that issue!
Yeah, you're right, the current client-to-server connection doesn't really work out. When I was starting it was just the simplest solution the put a cors proxy in front of the admin-ui.
What is your idea to fix that design flaw?

@knaoe i tried something, could you please test following branch?
git checkout ft/reverse-proxy
cd kratos-admin-ui
docker build -t test .
docker run -p 80:80 --network kratos_intranet -e KRATOS_ADMIN_URL="http://kratos:4434" -e KRATOS_PUBLIC_URL="http://kratos:4433" --rm test

Note that kratos_intranet is the docker compose network from the kratos quickstarter:
-> docker network ls
NETWORK ID     NAME              DRIVER    SCOPE
ab66a132942b   kratos_intranet   bridge    local

@dfoxg Thanks for the PR. I tried it out and it's really good! I hadn't considered a reverse proxy and was more focused on the SSR pattern. Your approach seems to be a smarter choice 👍
With this update, since it's a breaking change requiring users to tweak their environment variables, I'm kind of wondering if you should bump up the major version 🤔

Thanks for testing, I just released version 2.0.0🎉
My first thoughts were also in the direction of SSR, but I didn't want to add the big memory-foodprint of nodejs for this small admin ui. I think nginx just works fine for that use case :)


Info needed about windows-mgmt app VM lifecycle

I recently discovered I have a windows-mgmt app VM lying around and wonder if this serves any purpose after the initial installation of the Windows qube.
Can the windows-mgmt app VM be deleted? It would be nice to find a hint about this somewhere near the explanation of what it does and why it's there.
Thanks for listening :)

The purpose of the windows-mgmt AppVM is to securely isolate everything that goes on as part of the Windows installation to one machine. That way, the exploitation of any bugs that exist in, for example, in the Linux ISO filesystem parsing code is limited in the amount of harm it can do. This is the security principle upon which all of Qubes OS is built upon, "security by isolation" or "security by compartmentalization" they call it.
Feel free to delete windows-mgmt  if you are sure there are no more Windows VMs you would like to create. However, if it's just the disk space you're worried about then you can simply delete the ISOs at /home/user/Documents/qvm-create-windows-qube/windows-media/isos (in windows-mgmt) to save the vast majority of that space.
I will add this blurb to the FAQ section in the README as other people have also been curious about this in the past.

Yes it is indeed mainly the disk space I want to reclaim. Thanks for the hint in this direction and for the insights!

Documented in https://github.com/elliotkillick/qvm-create-windows-qube/commit/ffab3facaa35597f746a42a6f4a7ac457dea4a7c.


Georgian translation

I'm working on Georgian translation of this great app ^_^
Feel free to contact me.

Is there any progress? I'd love to help.


CNN and SVM models

This PR wants to merge the two models @rmoretti9 implemented into the development branch.
Please, review the code and suggest changes.
Keep also in mind that at least two reviewers must accept the changes and all the conversation must be resolved before we can merge.
Thanks

Thank you so much for your suggestions @marcorossi5, I modified the code accordingly. I will think about ways to improve speed and readability, and from now I will also use black.


🔖  Bump version to 2.0.4

Changes:
Still WIP CHANGELOG.
Bugfix: 🐛 The CLI no longer raise an unexpected exception when no encoding has been found (#70)
Improvement: ❇ Adjust the MD to lower the sensitivity, thus improving the global detection reliability (#69 #76)
Bugfix: 🐛 Fix accessing the 'alphabets' property when the payload contains surrogate characters (#68)
Internal: 🎨  Fix line endings from CRLF to LF for certain files (#67)
Bugfix: 🐛 ✏ The logger could mislead (explain=True) on detected languages and the impact of one MBCS match (in #72)
Bugfix: 🐛 Submatch factoring could be wrong in rare edge cases (in #72)

Codecov Report

Merging #72 (30602ce) into master (d283ef5) will not change coverage.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##           master      #72   +/-   ##
=======================================
  Coverage   85.70%   85.70%
=======================================
  Files          11       11
  Lines        1077     1077
=======================================
  Hits          923      923
  Misses        154      154

Impacted Files
Coverage Δ

charset_normalizer/version.py
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d283ef5...30602ce. Read the comment docs.

The dataset was updated to include more files.
See

https://github.com/Ousret/char-dataset/commit/a2e1adb16bd8acbd6cae20a12f1b3c2960a0e653
https://github.com/Ousret/char-dataset/commit/a21eb00e6cbec13ba702750051ad6604cd7817e2
https://github.com/Ousret/char-dataset/commit/d051a95f356cf41147104dfb7baf80602aac8402
https://github.com/Ousret/char-dataset/commit/2366c3f8113e703d692bbff22a280f319940933e
https://github.com/Ousret/char-dataset/commit/987b25c2721594953e3e3714b6b80f38d15c79ae

The README will be updated accordingly to new results.
------------------------------
--> Chardet Conclusions
   --> Avg: 0.1948392037470726s
   --> 99th: 0.87039s
   --> 95th: 0.34092s
   --> 50th: 0.0313s
------------------------------
--> Charset-Normalizer Conclusions
   --> Avg: 0.05083911007025761s
   --> 99th: 0.49343s
   --> 95th: 0.26959s
   --> 50th: 0.02116s

The diff coverage is acceptable (84.71% (-0.79%)).
Will be addressed in another PR.


Refresh of "Fantasy" for AMP 2.6.x; Refactored into Less...

Aww shiiit....

@IceOfWraith

You took my dumpster fire of a theme and turned it into a work of art. And in one night too! Thank you and wonderful work here!


RTL and bidi support

I've tested this code with sample Arabic and bi-directional text and verified it looks as it's supposed to.
I confirm I have signed the Contributor License Agreement.

Also, need to check the tests that are failing. Use 'npm run test' to check locally


[GUI-121-1] Add & configure records repository, records DAO, records provider.

Since the project is getting more complicated, it'd be better to separate logic related to working with Mongoose Run Records (constructing from Prometheus response, filtering, etc.).

Also, it'll probably reduce the huge amount of refactor the project has been receiving within the previous couple of weeks.


ListView doesn't respect color scheme and appears disabled.

I have 2 versions of the example application within this repository, one was compiled right away and the other was compiled after adding the Form_Load event. Other than that, nothing else changed. Oddly enough, the one without the form load event respects everything and the listview appears as expected while the other doesn't. Take a look yourself:

same issue +1


Too many implementations of state tomography

We have three implementations of state tomography:

for 1-qubit states,
for 2-qubit states,
for n-qubit states for arbitrary n.

The last one isn't well tested (yet). Once it is, we should compare performance and if there isn't a clear advantage to the fixed-size implementations we should remove them.

Should we keep this open? The linked issue fixes the bug in n-qubit tomography but the larger issue is still present

Yes, thank you. My comment on PR #2739 incorrectly said that it fixed #2737. It should have said that it fixed #2738.

There can never be too many implementations of state tomography :)
But yes we should unify these standard method of state tomography (there are many others).

Is this still an open concern ?

I'd say yes this is still a concern. It'd be nice to unify these tomography functions, but not a blocker for 1.0.


Schematic update recorder indices need to be different for BOM files

🐞 Bug report
Package
@angular-devkit/schematics
Is this a regression?
No
Description
Currently schematics which leverage the public UpdateRecorder API to make source-file transformations will get different results for source-files which contain a BOM.
This is because the CLI magically tries to account for the BOM in source-files by always adding 1 to passed indices. This is unexpected and causes confusing behavior as this implies that the UpdateRecorder only expects character indices for visible characters.
Schematics that currently determine the character indices by parsing the source-files (e.g. through TypeScript compiler API or fs.readFileSync) will get broken behavior for source-files that contain a BOM as the replacements are accidentally shifted by one.
We need to do one of the following:

Either provide a clear API description explaining the story with BOM and what indices the
update recorder expects. This forces schematic authors to know about BOM and to strip the BOM when reading source-files.

A long-term solution where the UpdateRecorder expects character indices which are truly based on the source-file. No magic differentiation between BOM / non-BOM files.

@alan-agius4 wrote

The main reason for this is that when a developer is writing a schematic they shouldn't need to compute the offset based if a file has a BOM or not and is handled out of the box.

I want to add to that: I see the point of doing that, but it is simply resulting in magic behavior as schematic indices are not determined by just looking at the visible rendered characters but rather by parsed source-files where the BOM is a valid invisible character that still adds to the amount characters.
e.g. fs.readFileSync('./with_bom.txt', 'utf8').length will respect the BOM characters and therefore it's common that indices passed to the update recorder will be truly absolute.
Ideally the magic differentation between BOM and non-BOM files would have not happened at all, but for now the easiest way to fix this in a backwards-compatible way (where older CLI versions are used) is to strip the BOM when parsing the TypeScript source-files.
We need to find a proper solution for this in the future though. It adds unnecessary bloat and confusion to schematic authors as they need to know about different behavior with BOM / and need to handle BOM characters in a special way for now. See the unnecessary bloat we need to add:

https://github.com/angular/angular-cli/pull/14556
https://github.com/angular/angular/pull/30719

 A clear and concise description of the problem...
🔬 Minimal Reproduction
See test-case in  https://github.com/angular/angular/pull/30719 or various other affected issues:

https://github.com/angular/angular-cli/issues/14551
https://github.com/angular/angular/issues/30713

🔥 Exception or Error
Replacements are shifted by one character if the source-file contains a BOM. Can result in
broken static-query migration where ViewChild will be converted to VViewChild or in a broken lazy route syntax migration where routes are incorrectly converted (#14551)

We should probably return the contents of the file already without a BOM.
//cc @clydin, feel free to add more details :)

@alan-agius4 would you think this is a feature? I definitely think he severity is confusing and might even go through as a bug as it is not clearly documented what the UpdateRecorder expects.
Magic behavior (how it is right now) is not always good, especially since I said above that in most cases the BOM is part of the characters in a file unless the BOM is explicitly stripped off.

The readText and readJson methods on Tree instances will automatically handle the presence of the BOM and remove as needed via the internal use of a TextDecoder.  If a schematic uses these methods than the indices will match and no offsetting will be required.  If a schematic continues to use the lower-level read method which provides the raw byte data, then the presence of a BOM will need to be handled manually by such a schematic.  Any existing schematics outside of the CLI (these have already been converted) should probably update to readText/readJson where appropriate.


ExportVectorDataService: add custom kmlWriter [#45104484]

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

9 of 9 (100.0%)  changed or added relevant lines in 1 file are covered.
1 unchanged line in 1 file lost coverage.
Overall coverage decreased (-0.02%) to 99.642%

Files with Coverage Reduction
New Missed Lines
%

src/modules/olMap/handler/draw/OlDrawHandler.js
1
98.25%

Totals

Change from base Build<PHONE_NUMBER>:
-0.02%

Covered Lines:
12492

Relevant Lines:
12494

💛  - Coveralls

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

15 of 15 (100.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.01%) to 99.653%

Totals

Change from base Build<PHONE_NUMBER>:
-0.01%

Covered Lines:
12499

Relevant Lines:
12500

💛  - Coveralls

replaced by #2426


aqara 零火线双键开关，一个不可用

Logger: homeassistant.components.switch
Source: helpers/entity_platform.py:566
Integration: 开关 (documentation, issues)
First occurred: 22:48:32 (5 occurrences)
Last logged: 22:51:26
Platform xiaomi_miot does not generate unique IDs. ID lumi.158d0003a4900a-d5a90887440cb9e81f608b3365a837a3-2 is already used by switch.lumi_ctrl_86plug_aq1_900a_switch - ignoring xiaomi_miot.lumi_ctrl_86plug_aq1_900a_switch
Platform xiaomi_miot does not generate unique IDs. ID lumi.158d0003f03c3a-d5a90887440cb9e81f608b3365a837a3-2 is already used by switch.lumi_ctrl_86plug_aq1_3c3a_switch - ignoring xiaomi_miot.lumi_ctrl_86plug_aq1_3c3a_switch
Platform xiaomi_miot does not generate unique IDs. ID lumi.158d0003fad713-d5a90887440cb9e81f608b3365a837a3-2 is already used by switch.lumi_ctrl_86plug_aq1_d713_switch - ignoring xiaomi_miot.lumi_ctrl_86plug_aq1_d713_switch
Platform xiaomi_miot does not generate unique IDs. ID lumi.158d000445495d-d5a90887440cb9e81f608b3365a837a3-2 is already used by switch.lumi_ctrl_ln1_aq1_495d_switch - ignoring xiaomi_miot.lumi_ctrl_ln1_aq1_495d_switch

存在同样的问题，aqara零火双开只有一个实体可用

I have the same issue too.

同样的问题，另外按实体按键trigger的event不会被nodered捕捉到

解决了吗？我也出现这情况。

没解决，现在在HA里是可用状态，然并卵，实际上用不了，只能操作个寂寞

https://github.com/ha0y/xiaomi_miot_raw/issues/373
lumi.ctrl_neutral2.v1 aqara双键单火线开关无法显示状态
是同一个问题吗

最近發現沒有問題可以正常使用了


Unexpected nodes in layer tree (KML)

[Originally from review of #136, but actually occurs without that change].
With https://raw.githubusercontent.com/mapbox/togeojson/master/test/data/multitrack.kml I see some UI artifacts that look like styles and Document properties being interpreted as features. I would expect to see the tour and the two placemarks, but not the other leaf elements in the tree.

Interestingly, doesn't occur in the debug app.
The test is a regular expression, which looks like
/^NetworkLink|Placemark|GroundOverlay|ScreenOverlay|Tour|$/
That last part is caused by a schema with no name.
<Schema id="schema">
<gx:SimpleArrayField name="power" type="int">
<displayName><![CDATA[Power (W)]]></displayName>
</gx:SimpleArrayField>
<gx:SimpleArrayField name="cadence" type="int">
<displayName><![CDATA[Cadence (rpm)]]></displayName>
</gx:SimpleArrayField>
<gx:SimpleArrayField name="heart_rate" type="int">
<displayName><![CDATA[Heart rate (bpm)]]></displayName>
</gx:SimpleArrayField>
</Schema>

Now it gets more difficult.
The google docs (https://developers.google.com/kml/documentation/kmlreference#schema) say that name is required - the schema field is ignored if not present.
However that does not appear in OGC KML 2.2 spec (07-147r2, section 9.8) or the OGC KML 2.3 spec.
Its not clear to me whether we should even be using the name attribute here. I'm only familiar with schema-defined elements being nested inside ExtendedData. Is there a data format we need to support here that does require this?

Urgh: https://github.com/google/libkml/blob/master/src/kml/engine/parse_old_schema.cc#L62

The name attribute shouldn't be required. That allows you to create elements like MyCustomPlacemark that are parsed as a Placemark, but use the provided schema for their ExtendedData. You can also use a normal Placemark with ExtendedData that references the schema in a <SchemaData schemaUrl=“#schema”> element, though our parser does not fully support that. OpenLayers sort of reads them, but interprets every value as a string instead of using the defined types.
We also don't support gx:SimpleArrayField or the 2.3 spec version, which is another enhancement we may want to look into in the future. That's used to provide per-coordinate metadata for tracks, which OpenSphere doesn't have a way to visualize right now.


Compiling throws an error due to extra paranthesis

Arduino error log:
C:\Users\bzhang\Documents\Arduino\libraries\xSN01-master\src\xSN01.cpp:132:25: error: expected ';' before ')' token
return (mySN01.NumSats));

@brandon-zhang should be fixed now.
Please state if it can be closed now ?


Redirect to Appflowy not working

Describe the bug
When using the 'magic link' to login in the Appflowy app (both stable and unstable versions). when clicking on the link provided in the mail, it doesn't redirect to appflowy making it unable to login.
Steps To Reproduce
Steps to reproduce the behavior:

Launch the appflowy under Hyprland
Add the email to login in 'Please enter your email address' field
Click continue
The link will be emailed to the provided mail, (I use firefox and open online proton mail)
Click on 'Login/Sign up'
The redirect will open a new tab and just stop there

Expected behavior
It should redirect back to appflowy and allow me to login
Screenshots

Additional context
Add any other context about the problem here.
Notify maintainers
@DarkOnion0 - Maintainer of appflowy app

Metadata

 - system: `"x86_64-linux"`
 - host os: `Linux 6.6.54, NixOS, 24.05 (Uakari), 24.05.20241006.ecbc1ca`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.18.8`
 - nixpkgs: `/nix/store/5cypwijbjdxmr4i7ckpq1in7f09547fc-source`

Add a :+1: reaction to issues you find important.

#338413

#338413

Didn't see that PR.

No problem, leave it open, It could be useful for other folks that didn't see the PR. I will take care of closing it when the PR will be merged (if it hasn't been linked to it already).


a11ySuite should allow for imperative setup steps

If an element has some lazily created DOM, there is currently no way to test this with a11ySuite.

It might be enough to give a11ySuite a callback that the user can control.

Should be fixed via #493 -> #513


fix(sdk): aentity_class was missing a positional argument

[ ] I have added tests that cover my changes.
[ ] If adding a new instrumentation or changing an existing one, I've added screenshots from some observability platform showing the change.
[ ] PR name follows conventional commits format: feat(instrumentation): ... or fix(instrumentation): ....
[ ] (If applicable) I have updated the documentation accordingly.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

@Carlososuna11 do you mind signing the CLA? I won't be able to merge your PR otherwise

friendly ping @Carlososuna11

Done! Sorry


Add a splash screen example

What problem does this solve or what need does it fill?
I would like to make a desktop app with a splash screen to show some basic information (a logo, current version, etc.) of my software.
What solution would you like?
Something like this:
pub struct WindowPlugin {
    pub primary_window: Option<Window>,
    pub splash_window: Option<Window>,
    pub exit_condition: ExitCondition,
    pub close_when_requested: bool,
}

What alternative(s) have you considered?
Tauri Splashscreen

just draw your splash screen inside your main window

Borrowing from https://github.com/TheBevyFlock/bevy_quickstart would make a great example.


about Semantic Nerf (or autonerfacto) pretrained weights

Hello!
Thank you so much for sharing your incredible work! I have read the README.md and followed each step carefully.
I am interested in testing mesh generation (or NeRF inference),
but to the best of my knowledge, I was unable to find the pretrained weights for Semantic NeRF (or autonerfacto).
Would it be possible for you to share those weights if available?
Thanks.

Hi!
Thank you for your interest in our work! This code allows you to collect data with a modular policy (pre-trained weights of the most used policy in the paper are released), and then train a NeRF model (Semantic NeRF or autonerfacto) from the collected data. Have you been able to try this? Once training is done, you will be able to test the final NeRF model (mesh generation and/or NeRF inference).
A different NeRF model is trained for each policy rollout, so the pre-trained weights depend on the specific 3D scene, but also policy trajectory within such scene. I would thus suggest you train a NeRF for the specific scenario you are interested in. Let me know if you have any issues with training a NeRF from the code, or please provide more information about what you would like to do, and I’ll see what I can do to help!

Thank you for kind answering!
Sure, I tried to train Semantic NeRF first following your instructions in README.md.
I also got the mesh generation, too. (It looks nice!)
I'm curious about how to do NeRF Inference using trained Semantic NeRF model.
In my best knowledge, I could not find how to inference NeRF (to check NeRF quality).
I imagined that hopefully looking around the room and looking at novel view supported by NeRF.
I have trained Semantic Nerfacto, too. and maybe I can try BEV map generation, planning, Pose refinement.
But, I'm not sure I can check NeRF quality.
Is there any way to check NeRF quality? (or just inference of NeRF).
I will check your code more, too.
I expected that mesh generation code might help to find NeRF inference part.
Again, Thank you for sharing your interesting work :)

I read "mesh generation" code and realized that I misunderstand.
So, we can define cube (256, 256, 256) and trained NeRF model can predict each point's color (rgb) and density, too.
Using density, we can create occupancy with a certain threshold.
Finally we can generate mesh.
Awesome!
If you think some parts I misunderstood, feel free to let me know! :)

Doing inference with a NeRF model is about predicting density, color (and semantics in our case) for a given point in the considered 3D space. This is what we indeed use to build a mesh representation.
However, it looks like you are mostly interested in performing novel view synthesis, i.e. rendering (NeRF inference + volumetric rendering), to evaluate the ability of the NeRF model to render frames from novel camera views, right? Such evaluation (on a held-out set of test camera poses) is already performed during training. You can find some inference samples on the nerfstudio tensorboard (“IMAGES” section — when training autonerfacto) for example, and mean rendering metrics for all test images (“SCALARS/Eval Images Metrics Dict (all images)”). Let me know if you need more help.


gateway pass header to each grapql service

I`m testing gateway example and I don´t know how pass header recived by gateway to each service.
There are some documentation about that?

The config you generate should look similar to this:
https://github.com/jensneuse/graphql-go-tools/blob/e69d4234c60ecb812ef0ac4d608a49621843aa23/pkg/engine/datasource/graphql_datasource/graphql_datasource_test.go#L217

Please reopen in case you have additional questions.

Hi, I have took a look on different issue posted here in Github #115 #128 #270 about passing request headers to new request made by the engine (Client - Gateway -> Gateway - Service) but I couldn't find out how, I have tried for example the following
	if err = g.engine.Execute(r.Context(), &gqlRequest, &resultWriter, graphql.WithAdditionalHttpHeaders(r.Header)); err != nil {
		g.log.Error("engine.Execute", log.Error(err))
		w.WriteHeader(http.StatusInternalServerError)
		return
	}

but each time I fail the other service doesn't have access to the request headers. Can you please send us an example of code shows how to pass those headers or blog or even adding above your example
thanks in advance

Stuck with the same issue. Can't understand how to pass headers using WithAdditionalHttpHeaders.
Any good example?
thanks

How do you this with subscriptions?


Fix raytracing origin

Adds the laser ray origin to RangefinderPoint to enable
correct raytracing of range data misses into a 2D probability
grid also after scan accumulation.
Previously, all rays were casted from their hit point to the
tracking frame. This is wrong for any setup where the tracking
frame is not the sensor frame and leads to artifacts in the map.
Fixes: #947
Below an example of a map generated from a multiple 2D laser scanners configuration, which are not centered in the tracking frame, before and after the proposed fix. It can be seen clearly that the previous version of the software was cutting and cleaning the corners of the map due to the imprecise origin used for raytracing. This caused the exclusion of large structures from the map.

@tulku thanks for your suggestion and sorry delay!
Now the origin is added to the operator == and I also some tests that were not initializing the origin are now fixed.
Regarding the TimedPointCloud I left it without the origin because for that purpose we have the TimedPointCloudOridinData.