IPv6 link local routes on WAN interface are being broadcast on LAN interface

Important notices
Before you add a new report, we ask you kindly to acknowledge the following:

[x] I have read the contributing guide lines at https://github.com/opnsense/core/blob/master/CONTRIBUTING.md
[x] I am convinced that my issue is new after having checked both open and closed issues at https://github.com/opnsense/core/issues?q=is%3Aissue

Describe the bug
My cable ISP repeatedly sends 8 RAs for a default route (::/0) to the connected CPE, opnSense's WAN interface in this case.  These RAs are all for link local addresses in the fe80::/64 block.  Every one of these is reachable from the opnsense machine and suitable for routing.
However these advertisements are also being passed along by opnSense to the LAN interface somehow, and picked up by clients.  Those clients cannot reach the addresses, since they are link-local addresses on opnSense's WAN interface.  This results in very spotty ipv6 network connectivity for SLAAC LAN clients.
Note: It's possible this is in fact normal and expected behavior, and that I've configured something wrong or misunderstood something, but it seems like a bug to me at present.
To Reproduce
Steps to reproduce the behavior:

Enable DHCPv6 on WAN interface (using an ISP which uses this type of routing)
Enable Prefix Delegation requesting a prefix.
Enable "Track Interface" of WAN for IPv6 on LAN interface, prefix ID 0x0.
Set LAN router advertisement service to "Unmanaged".
Connect SLAAC client on LAN interface segment.

Expected behavior
SLAAC clients should receive a /64 delegation from opnsense, and a default route of only opnsense's LAN link-local fe80::/64 address.  They should not be given routes to an upstream link-local address they cannot reach.
Describe alternatives you considered
Turning off "advertise default gateway" prevents the correct LAN link-local address from being advertised to clients, but does not stop opnsense from passing the unreachable link-local addresses it receives from the WAN upstream.
Relevant log files
WAN interface tcpdump output:
root@wanfirewall:~ # tcpdump -ni vmx0 icmp6 and 'ip6[40] = 134'
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vmx0, link-type EN10MB (Ethernet), capture size 262144 bytes
21:58:43.707390 IP6 fe80::dead:beef:4:1 > ff02::1: ICMP6, router advertisement, length 24
21:58:43.850804 IP6 fe80::dead:beef:3:1 > ff02::1: ICMP6, router advertisement, length 24
21:58:43.898575 IP6 fe80::dead:beef:1:1 > ff02::1: ICMP6, router advertisement, length 24
21:58:43.947052 IP6 fe80::dead:beef:2:1 > ff02::1: ICMP6, router advertisement, length 24
21:58:44.018638 IP6 fe80::dead:beef:5:1 > ff02::1: ICMP6, router advertisement, length 24
21:58:44.114489 IP6 fe80::dead:beef:80:1 > ff02::1: ICMP6, router advertisement, length 24
21:58:44.171078 IP6 fe80::dead:beef:6:1 > ff02::1: ICMP6, router advertisement, length 24
21:58:44.338511 IP6 fe80::dead:beef:70:1 > ff02::1: ICMP6, router advertisement, length 24
21:58:46.907154 IP6 fe80::dead:beef:3:1 > ff02::1: ICMP6, router advertisement, length 24
21:58:47.010280 IP6 fe80::dead:beef:4:1 > ff02::1: ICMP6, router advertisement, length 24
21:58:47.178516 IP6 fe80::dead:beef:2:1 > ff02::1: ICMP6, router advertisement, length 24
21:58:47.211428 IP6 fe80::dead:beef:1:1 > ff02::1: ICMP6, router advertisement, length 24
21:58:47.346970 IP6 fe80::dead:beef:6:1 > ff02::1: ICMP6, router advertisement, length 24
21:58:47.483377 IP6 fe80::dead:beef:5:1 > ff02::1: ICMP6, router advertisement, length 24
21:58:47.483476 IP6 fe80::dead:beef:80:1 > ff02::1: ICMP6, router advertisement, length 24
21:58:47.538631 IP6 fe80::dead:beef:70:1 > ff02::1: ICMP6, router advertisement, length 24

Windows 10 SLAAC client netstat -anr output:
IPv6 Route Table
===========================================================================
Active Routes:
 If Metric Network Destination      Gateway
  3    281 ::/0                     fe80::20c:29ff:fe21:25a6
  3    281 ::/0                     fe80::dead:beef:4:1
  3    281 ::/0                     fe80::dead:beef:1:1
  3    281 ::/0                     fe80::dead:beef:3:1
  3    281 ::/0                     fe80::dead:beef:70:1
  3    281 ::/0                     fe80::dead:beef:80:1
  3    281 ::/0                     fe80::dead:beef:2:1
  3    281 ::/0                     fe80::dead:beef:6:1
  3    281 ::/0                     fe80::dead:beef:5:1

The first route in this list is the correct route, the link local address of opnsense.  All the others are those being sent from the ISP and should not (I believe) be promulgated onto a different ethernet segment by opnsense unless it's going to respond on their behalf, which it does not.
Environment
Software version used and hardware type if relevant, e.g.:
OPNsense 22.7.3_2-amd64
Core i5-8259U CPU
VMware virtual NICs (hosted on ESXi 7.0)

I've also tried manually deleting the problematic routes on the client but they reappear almost instantly, though I don't see any RAs going over the LAN interface.  I did see a bunch of neighbor solicitations for those addresses, however.

Hi @alandsidel,
I've never seen that before, slightly considering ISP behaviour seems odd. It would indicate this is a radvd-internal at play here since that is the only thing propagating the routes received.
Mind sharing your radvd.conf -- although I think it doesn't have any visible additions to the standard one.
# cat /var/etc/radvd.conf

CC my favourite IPv6 expert @maurice-w
Cheers,
Franco

Thanks @fichtner
Haven't ever seen it myself.  Conf file looks normal to me.
root@wanfirewall:~ # cat /var/etc/radvd.conf
# Automatically generated, do not edit
# Generated for DHCPv6 server lan
interface vmx1 {
        AdvSendAdvert on;
        MinRtrAdvInterval 200;
        MaxRtrAdvInterval 600;
        AdvLinkMTU 1500;
        AdvDefaultPreference medium;
        prefix 2601:18c:8a80:b40::/64 {
                DeprecatePrefix on;
                AdvOnLink on;
                AdvAutonomous on;
        };
        RDNSS 2601:18c:8a80:b40:20c:29ff:fe21:25a6 {
        };
        DNSSL localdomain {
        };
};

Reddit poster thought that maybe a switch might be to blame allowing multicast from WAN to LAN port(s).
https://www.reddit.com/r/opnsense/comments/x7ps49/comment/infrdly/?context=3

That was exactly it, thanks.


Exclude numeric data type mismatches from casewhen type checking

Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR.

No Jira issue created. This PR was opened per a conversation with @sfc-gh-jrose.

Fill out the following pre-review checklist:

[ ] I am adding a new automated test(s) to verify correctness of my new code

[ ] If this test skips Local Testing mode, I'm requesting review from @snowflakedb/local-testing

[ ] I am adding new logging messages
[ ] I am adding a new telemetry message
[ ] I am adding new credentials
[ ] I am adding a new dependency
[ ] If this is a new feature/behavior, I'm adding the Local Testing parity changes.

Please describe how your code solves the related issue.

Resolves an issue with Local Testing mode where numeric data types used in a CaseWhen expression return an error if there is a precision mismatch.

I have read the CLA Document and I hereby sign the CLA

Thanks for the PR Chris.
For reviewers, here is a repro for the issue that this PR is addressing:
session = Session.builder.config("local_testing", True).create()

dataframe = session.create_dataframe(
    data=[
        ["value1", 1.23],
        ["value1", 1.23],
        ["value2", 4.56],
        ["value2", 7.89],
    ],
    schema=StructType(
        [StructField("col1", StringType()), StructField("col2", DecimalType(38, 2))]
    ),
)

dataframe2 = session.create_dataframe(
    data=[["value1", 5.0]],
    schema=StructType(
        [StructField("col3", StringType()), StructField("col4", DecimalType(38, 6))]
    ),
)

dataframe.join(
    right=dataframe2, on=dataframe.col1 == dataframe2.col3, how="left"
).select(
    dataframe.col1,
    dataframe.col2,
    dataframe2.col3,
    dataframe2.col4,
    when(dataframe2.col4.is_null(), dataframe.col2)
    .otherwise(dataframe2.col4)
    .alias("new_col"),
).show()

Hey Chris,
I'm going to go ahead and close this PR. We merged a PR yesterday that adds support for type coercion which has changed both code locations touched in this PR. I've run your example against the new code paths and it looks like it handles it correctly now. Your change was helpful in identifying edge cases in that new code so thank you for taking the time to open this PR.


Switch to calendar versioning

Use a model like PyCharm: YYYY.MINOR.PATCH
http://calver.org/
The first release will be 2016.5.0

Done in https://github.com/saghul/sjs/commit/bd4acdd34946659b38c366e036c28d30cdee1878
The first version will be 17.0.0.


Leverage type information in no-array-destructuring rule

For a tuple with fixed length, array destructuring should be safe.

Compare #3


Modify JupyterHub networkPolicy to match existing policy

Reference Issues or PRs

What does this implement/fix?
Fixes #1965
Having the CHP networkPolicy enabled for JupyterHub deployed to AWS, Azure and GCP was no trouble so I'm not sure why this only seems to affect Digital Ocean. We also have the networkPolicy disabled for hub and singleuser already.
We should definitely test this change on the other clouds before merging - I don't expect this to cause problems since removing the network policy makes internal networking more permissible.
Put a x in the boxes that apply

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds a feature)
[ ] Breaking change (fix or feature that would cause existing features not to work as expected)
[ ] Documentation Update
[ ] Code style update (formatting, renaming)
[ ] Refactoring (no functional changes, no API changes)
[ ] Build related changes
[ ] Other (please describe):

Testing

[x] Did you test the pull request locally?
[ ] Did you add new tests?

Any other comments?

@costrouc do you see any reason why we shouldn't disabled the chp.networkPolicy?

@pavithraes @kcpevey could you please review?


Add support for dedenting XString heredoc

Motivation
Sorbet currently doesn't supportdedenting XString heredocs. This adds support for that.
Fixes #1628
Test plan
See included automated tests.

@DarkDimius
Thanks for the quick turn-around time.
Parsing shouldn't error and I can't think of a reason this would, I am however no expert in XString heredocs.
As for indentation level, dedentString will be called with an indentLevel equal to the minimum amount of indents for the lines. So "wrong number of whitespaces" is not really a thing. However, I've expanded the test to include a case with varying indentation to show that.
There is one bug I fixed in #2955 that the XString heredoc is subject to as well. The fix in that PR will however also fix it for XString heredocs so landing that separately will fix it here too.

@DarkDimius It looks like the tests failed with this error:
common/test/common_test.cc:30
Value of: FileOps::ensureDir("common_test_dir")
  Actual: false
Expected: true

I didn't touch that and looks a bit like an issue in CI to me. I tried reproducing locally with both dynamic and static linking but couldn't reproduce.
If you think it's related to my changes, any pointers on being able to reproduce locally would be greatly appreciated.

Could you rebase off of master? There was a bug in the FileOps::ensureDir test that was causing it to fail.

@elliottt Sure thing, that's done now. Anything I can do to kick off another test run?

Kicked off tests :-)


Import Collection OpenAPI v3 Spec does not seem to work

Hey there, thanks for developing this tool!
I managed to import json openapi specs once on v1.12.2, but except that one occasion, every time I select the openapi.json file, a location and then nothing happens when I click import.
I also tried converting my openapi json to yaml, but I still can't successfully import it.
In the dev tools I see this warning printed every time I click Import:
Warning: An unhandled error was caught from submitForm() ReferenceError: enableTranslation is not defined
    at handleSubmit (708-91b98ba4d9a5d350.js:1:34731)
    at onSubmit (708-91b98ba4d9a5d350.js:1:32368)
    at _app-3dfff338a00a1088.js:11:43927
    at _app-3dfff338a00a1088.js:11:47917
    at _app-3dfff338a00a1088.js:11:43100

I am experiencing the same issue and see the same error in the dev console. I also recall being able to import a v3 openapi spec about 3 months ago on my old laptop, but I don't know which version of Bruno that was.
Given the error message, I suspect this is related to the Auto translate postman scripts option that is given in the "Import Collection" modal. I get the same error message whether or not this option is checked.

Closing this as stale/not reproducible. Please create a new issue if the problem persists and provide more details for debugging.


Ledger | App Error: | 28417 apdu sign verify error

Whenever I try to sign MASP transactions on Ledger app v0.0.28 I get the following error after pressing selecting/pressing "Approve" on the hardware wallet: Ledger | App Error: | 28417 apdu sign verify error. I assume the error emanates from the vicinity of crypto_check_masp. Some notes:

I'm using the ZIP 32 (over SLIP 10) path: m/32'/877'/0' (from the mnemonic equip will roof matter pink blind book anxiety banner elbow sun young).
The hardware wallet internally uses the extended spending key: 0x03bc4834ed00000080fe5f31e2d3bd521177e96903bafaeea1ff38afbfc87bc92ffd8bafa9f8f939c707bc9208dcd6e622a6aacab839c23c26404cd362a33a6fb5f9e9871b65652e09a1a8f2975e2d21edeac47b57f70ade8fff6ede55049d5be4a1c95dcbec47ed0c90790db1d8232c742fa3c4437e958276951958bb2392c9debd325f306582daa6ad0a2591b6c6ad459fa2d181005eeef982e378b807eb109275ae89f288f86697 .
The client only knows the extended full viewing key generated from this: 0x03bc4834ed00000080fe5f31e2d3bd521177e96903bafaeea1ff38afbfc87bc92ffd8bafa9f8f939c7031b9d3d838fb0343ac264aeb78ba909e5a731cd7a3b4d601af55bfde40e2b6df0c1485f8201ab1675c00d1201cb18f0eea8c9e92d26189fc9738544c2e4143d90790db1d8232c742fa3c4437e958276951958bb2392c9debd325f306582daa6ad0a2591b6c6ad459fa2d181005eeef982e378b807eb109275ae89f288f86697.
The client is also given the proof generation key by the hardware wallet (the nsk can be found inside the extended spending key and the ak inside the extended full viewing key).
The attached vectors contain dummy spend authorization signatures since the client generated them without knowledge of the true spend authorizing keys.
The nth SpendBuildParams is used in the construction of the nth spend description. Analogously, the same thing is true for the convert descriptions and output descriptions.
Sometimes more randomness parameters are generated than are actually needed/used. This is intentional because these parameters are generated by namadac before it knows the exact constitution of the Transaction.

See attached some logs of the calls I'm making. Am I doing something wrong?
apdu_sign_verify_error.txt

Hey @murisi, one question regarding what we talked before, are you using the first N random values on the builder ?

I am trying to debug this on zemu, using the info from your debug file

@murisi I have one question requerdinf the process you followed:

Generate randomness with spend/output/convert randomness methods;
Generate the spends signatures using the sign_masp_spends;
Extract the spends signatures;
Sign complete masp blob with sign method;
This error is appearing on step number 2 ?

Hey @murisi, one question regarding what we talked before, are you using the first N random values on the builder ?

Yes, I believe so. The nth spend description should be using the nth alpha and rcv values. See https://github.com/anoma/masp/blob/f2b0cae3e495e4f7d482e587432ec4e5f2793528/masp_primitives/src/transaction/components/sapling/builder.rs#L1000-L1010 . The nth convert description should be using the nth rcv value. See https://github.com/anoma/masp/blob/f2b0cae3e495e4f7d482e587432ec4e5f2793528/masp_primitives/src/transaction/components/sapling/builder.rs#L1053-L1058 . The nth output description should be using the nth rcm and rseed values. See: https://github.com/anoma/masp/blob/f2b0cae3e495e4f7d482e587432ec4e5f2793528/masp_primitives/src/transaction/components/sapling/builder.rs#L1088-L1094 . Though I'll recheck this code, especially its interaction with dummy notes.

@murisi I have one question requerdinf the process you followed:
1. Generate randomness with spend/output/convert randomness methods;

2. Generate the spends signatures using the sign_masp_spends;

3. Extract the spends signatures;

4. Sign complete masp blob with sign method;

Hi @chcmedeiros . I believe that I've followed the above process. See: https://github.com/anoma/namada/blob/5a3b3c7af8276df74a457bb48edfcc0afb1894f5/crates/apps_lib/src/client/tx.rs#L934-L1067 . Breaking it down:

Line 947 generates the spend randomness
Line 958 generates the convert randomness
Line 967 generates the output randomness
Line 981 builds a MASP Tx based on the randomness parameters
Line 1021 signs the MASP Transaction. This is where this issue's error is displayed after the user presses "approve"
Line 1030 is not reached, but this is where the spend authorization signature is extracted
Line 1067 is not reached, but this is where the sign method is called to sign the entire Tx with spend authorization signatures attached

   This error is appearing on step number 2 ?

Yes exactly, the error happens on step 2. The hardware wallet displays the entire MASP transaction along with "approve" and "reject" buttons. Pressing the "approve" button causes the CLI to display, "Ledger | App Error: | 28417 apdu sign verify error".

Hi @chcmedeiros . Also, I've integrated the clean_randomness_buffers as the step 0 here: https://github.com/anoma/namada/blob/aa8aaa022c3350b5b1df79bfee51b016eeb894b0/crates/apps_lib/src/client/tx.rs#L837 . This call seems to be working, but I just wanted to check whether I'm using it correctly since it takes two parameters: a BIP 44 path and a Tx blob. Since the Tx has not been built at this point, I supply it with a  dummy Tx and path. Is this okay/correct usage of the function? How does the function use these two parameters?

@murisi need your input here on the cv computation for the converts. We should use the generator and value rigth ?

@murisi need your input here on the cv computation for the converts. We should use the generator and value rigth ?

@chcmedeiros I think it should be computed as cv = asset_generator*value + VALUE_COMMITMENT_RANDOMNESS_GENERATOR*rcv (similar to how it's done for spend and output descriptions). See https://github.com/anoma/masp/blob/8d83b172698098fba393006016072bc201ed9ab7/masp_primitives/src/convert.rs#L65 and https://github.com/anoma/masp/blob/8d83b172698098fba393006016072bc201ed9ab7/masp_primitives/src/sapling.rs#L194 .

asset_generator is here: https://github.com/Zondax/ledger-namada/blob/c52229ad26aa0a60c644b44b130b241baffa4c28/app/src/parser_impl_masp.c#L316
value is here: https://github.com/Zondax/ledger-namada/blob/c52229ad26aa0a60c644b44b130b241baffa4c28/app/src/parser_impl_masp.c#L312
rcv is generated by the hardware wallet
VALUE_COMMITMENT_RANDOMNESS_GENERATOR is the same as the one used to compute the value commitment for spend and output descriptions
Thanks!

Hey @murisi are you able to test using the fixes branch ?

Hey @murisi are you able to test using the fixes branch ?

Hi @chcmedeiros , thanks for looking into this. I've ran the test using the fixes branch, but the 28417 apsu sign verify error is still occurring at the same place. The issue seems to be in checkSpends. I'm currently rechecking the Transactions produced by namadac, perhaps there's an error in the computations there...

We can try and remove the rk verification to check if indeed the issue is there. The cv comparison is similar to the converts and outputs, and at least in zemu it is working without problem.
I added the checks to check if the correct randomness values were indeed the ones being used.

@chcmedeiros Apologies, the remaining problem was on my side. In order to be able to test the Ledger app with namadac, I've been applying https://github.com/Zondax/ledger-namada/pull/61 on top of the fixes branch. (This is because the namada repo including the Transaction builder interface, CLI interfaces, and the software wallet are built around extended spending keys and viewing keys.) However there was a bug in that PR that I've now fixed with https://github.com/Zondax/ledger-namada/pull/61/commits/60ded6c2fdc6eb3c604dbbfeac35ad3f74af611d . So this error is now gone, MASP transaction signing now works, and the signatures produced are accepted by node validation functions. I'll continue testing though... Thanks.


In the seconds after reboot VerneMQ accepts new subscriptions but doesn't build the routing trie (in initialize trie window)

In a reboot situation, VerneMQ will go through existing subscriptions and create live routing tries (tables) while loading those subscriptions.
Initializing the tries, VerneMQ will accept SUBSCRIBES but not build up the actual routing tries.
This is due to the fact that the vmq_reg_trie event handler subscribes to Subscriber DB events, but misses those events created before the event handler hooks into the DB. It's a concurrency bug.
This is most likely to affect clients that are directly connected to a broker node, with pre-existing sessions that try re-connecting in a tight loop.
Related to https://github.com/vernemq/vernemq/issues/1363#issuecomment-650788413
Solution: Verne should fail the subscriptions and tell the Subscriber (sending failure code 128, for MQTT 3.1.1). At least the Subscriber will then not wrongly assume it has a working subscription. Unfortunately there's no way to send a more precise response code in SUBACK except 'failure'. (the client might think it's an authorization failure, i.e. not allowed to subscribe).
I confirmed/solved the bug by using a global state variable to check the Trie event handler is ready.

Always been thinking some boot level/run level system would be interesting... :)
Could possibly address stuff like this.


Wisper with PublicActivity

How can I use Wisper with PublcActivity ?
I want to send push notification after new activity been added to the table

Take the code from here and here, mash it up, see if it works.


socket file does not exist

My /tmp/.job_ pidnnnn has been deleted. In this case, how can I use the generate tool or restore it job_ pidNNNN.

What are you trying to do? How this is related to Java profiling?

I tried to execute this command
sh async-profiler-2.0-linux-x64/profiler.sh -d 10 -o flamegraph  -f /var/lib/hadoop-hdfs/test/2.html 27112
But an error is reported
Could not start attach mechanism: No such file or directory
This is .job_ pidnnnn is caused by deletion. Is there any method to solve it

I have no idea what .job_pid is. Perhaps you mean /tmp/.java_pidNNN socket?
The problem is described in Troubleshooting section:

Attach socket /tmp/.java_pidNNN has been deleted. It is a common practice to clean /tmp automatically with some scheduled script. Configure the cleanup software to exclude .java_pid* files from deletion.
How to check: run lsof -p PID | grep java_pid
If it lists a socket file, but the file does not exist, then this is exactly the described problem.

What is the result of the above lsof command?

Attach socket /tmp/.java_pidNNN has been deleted.Is there any solution?

Attach socket is automatically recreated since JDK 14.
This has been fixed in JDK-8225690 and backported to JDK 11.0.3 and JDK 8u332.
In earlier versions of JDK a deleted attach socket cannot be recreated until JVM restarts.

Get it,thanks.


a bug about tf.function

WARNING:tensorflow:AutoGraph could not transform <function converge_to_2 at 0x7f6e047dae50> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output.
Cause: invalid value for "node": expected "ast.AST", got "<class 'NoneType'>"; to visit lists of nodes, use "visit_block" instead
WARNING: AutoGraph could not transform <function converge_to_2 at 0x7f6e047dae50> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output.
Cause: invalid value for "node": expected "ast.AST", got "<class 'NoneType'>"; to visit lists of nodes, use "visit_block" instead
tf.Tensor(1.9999981, shape=(), dtype=float32)

@tf.function
def converge_to_2(n_iters):
total = tf.constant(0.)
increment = tf.constant(1.)
for _ in range(n_iters):
total +=increment
increment /= 2.0
return total
print(converge_to_2(20))

@paokuwansui, Check the gast version. Tensorflow 2.x supports gast ==0.2.2. Thanks!

Thank you, but my gast == 0.2.2
docutils                   0.16
entrypoints                0.3
et-xmlfile                 1.0.1
future                     0.18.2
gast                       0.2.2   #-----------------------
google-auth                1.10.0
google-auth-oauthlib       0.4.1
google-pasta               0.1.7
grpcio                     1.27.0
h5py                       2.10.0
html5lib                   1.0.1
idna                       2.8

@paokuwansui, Can you provide the complete code to replicate the reported issue. Thanks!

@paokuwansui, Is this still an issue?

import tensorflow as tf
@tf.function
def converge_to_2(n_iters):
total = tf.constant(0.)
increment = tf.constant(1.)
for _ in range(n_iters):
total +=increment
increment /= 2.0
return total
print(converge_to_2(20))

Is this still an issue?
Yes, this problem is still unresolved
My system is manjaro
tensorflow == 2.1.0

Source code：
import tensorflow as tf
@tf.function
def converge_to_2(n_iters):
total = tf.constant(0.)
increment = tf.constant(1.)
for _ in range(n_iters):
total +=increment
increment /= 2.0
return total
print(converge_to_2(20))

@paokuwansui, I tried on colab with Tf 2.1. Its working without any warnings.
Please take a look at gist here. Thanks!

@paokuwansui, Let us if still issue persists. Thanks!

Yes, I hope I can provide you with more information
But i don't know what information to give you
[lzh@lzh-pc ~]$ python
Python 3.8.1 (default, Jan 22 2020, 06:38:00)
[GCC 9.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.

import tensorflow as tf
@tf.function
... def a (n):
...     return n*2
...
a(10)
2020-03-13 19:09:06.982192: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE3 SSE4.1 SSE4.2 AVX AVX2 FMA
2020-03-13 19:09:07.030306: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency:<PHONE_NUMBER> Hz
2020-03-13 19:09:07.031321: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55f35ef5d720 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-03-13 19:09:07.031349: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
WARNING:tensorflow:AutoGraph could not transform <function a at 0x7fe0080c50d0> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output.
Cause: Unable to locate the source code of <function a at 0x7fe0080c50d0>. Note that functions defined in certain environments, like the interactive Python shell do not expose their source code. If that is the case, you should to define them in a .py source file. If you are certain the code is graph-compatible, wrap the call using @tf.autograph.do_not_convert. Original error: could not get source code
WARNING: AutoGraph could not transform <function a at 0x7fe0080c50d0> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output.
Cause: Unable to locate the source code of <function a at 0x7fe0080c50d0>. Note that functions defined in certain environments, like the interactive Python shell do not expose their source code. If that is the case, you should to define them in a .py source file. If you are certain the code is graph-compatible, wrap the call using @tf.autograph.do_not_convert. Original error: could not get source code
<tf.Tensor: shape=(), dtype=int32, numpy=20>

I use a laptop
The system is manjaro
CPU is i5 8550u
The graphics card is Max150

@paokuwansui You can safely ignore the warning log as its intended to debug logging in AutoGraph issues. Thanks !

Thank you very much------------------ 原始邮件 ------------------
发件人<EMAIL_ADDRESS>发送时间: 2020年3月31日(星期二) 凌晨5:48
收件人<EMAIL_ADDRESS>抄送: "paokuwansui"1059773473<EMAIL_ADDRESS>主题: Re: [tensorflow/tensorflow] a bug about tf.function (#36794)
@paokuwansui You can safely ignore the warning log as its intended to debug logging in AutoGraph issues. Thanks !
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.


[Feature] Logo design for Hirehub

Design Professional Logo for hirehub and replace with current logo.png in src folder
also make corrosponding favicon and put it in index.html
best design will be accepted

Hi i would like to design logo for Hirehub. Can i work on it?

HEY @Yashgabani845  please assign me this issue, i want to work on it.

@meghanakn473 start working on it
but i clarify Best logo will be accepted since you both want to work on same issue

I designed new logo, now it looks like this.

@slavomirStucka good work
@meghanakn473 any update from your side?

@slavomirStucka good work
@meghanakn473 any update from your side?

Yah im working on I'll share it within 8

hey @Yashgabani845 i have actually updated my logo and related code link in the fork file....but im not able run the file and take the screenshot for proof...what do i do?

@Yashgabani845  please check and review

i think @slavomirStucka has better one
@meghanakn473 this logo is good but it is not matching theme and size of requred space
@slavomirStucka
you can make the PR

@slavomirStucka in logo make image of the logo smaller same as text in logo or slighlty larger then make PR for it


Instant crash on start up

When i open the freepie bridge it immediately crash
just saying It stopped working any idea how to fix it...

Take a look for Freepie Bridge 16 version. Worked perfectly for me.


Downed Leaseholder node impact on query latency

Providing a description in our documentation covering the latency from the point where a Query is issued, a Leaseholder is detected as down, and the subsequent query response would be helpful. There are videos covering this in our Getting Started courses, however, these aren't easily referenced for questions asking this type of question. Covering the variety of scenarios with the latency impacts would be helpful in general for customers.

I think this is SQL observability.


[diffusers]: model management (configure and pre-load)

write config first and then download everything in config,
or download first and write config for downloads?

The diffusers API is happy to load model weights on demand from the huggingface.co model hub. But we'd still like the configure script to download model weights ahead of time because (1) the web UI has no "download in progress" indicator, and (2) we want invoke.py to be able work offline even on its first run.
In its current state, #1583 has two model download functions:

one that takes some keys defined in INITIAL_MODELS, does some downloads, and then adds the successfully downloaded ones to the local models.yaml.
another for diffusers, that looks for models in the local models.yaml and loads them to make sure they've been downloaded.

Having both those approaches is a sign of internal confusion and makes it hard to figure out how to specify which things to load.
Since the diffusers API can download models on-demand, I think we should assume it will be used this way at least part of the time, so whatever pre-loading method the configure script uses should be compatible with that.
The most straightforward thing would be to do what download_diffusers_in_config does now: https://github.com/invoke-ai/InvokeAI/blob/23eb80b40421b2bb8f4b6d3dd30490d11c447b36/scripts/configure_invokeai.py#L428-L430
so there's never any difference in the download-cache-load code between the ModelCache and the configure script, but I'm not sure whether that offers the same set of functionality as configure_invokeai's current download functions regarding progress indicators, behavior with interrupted downloads, etc.
There's still some uncertainty about what Invoke will do with ldm-flavored model files after our #1583 diffusers integration, but for new installations I think the configure process should get the diffusers flavored files.
There is a version of INITIAL_MODELS that's diffusers-friendly here: 6e4dad6 (#1583)
(I didn't find a hosted version of the Trinart Characters models, so I left those out.)

I'll also note, I think there is value in being able to boot the WebUI without a model. This would allow users to skip configure, download the support models, and then boot the WebUI to scan a folder and upload.
As far as ldm vs diffusers models, well... you know my opinion there.

done, configure script now gets diffusers-format models.


Add option to enable tracing/appinsights

This issue is for a: (mark with an x)
- [ ] bug report -> please search issues before submitting
- [X] feature request
- [ ] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Expected/desired behavior

Add option to enable tracing/logs/metrics to flow to App Insights to enable usage reports

Good idea! For Python apps, the OpenCensus package is the current way to hook into AppInsights, but OpenTelemetry is going to supercede that soon.
This should be the way to do it for OpenCensus, though I haven't tried it yet myself:
https://learn.microsoft.com/en-us/azure/azure-monitor/app/opencensus-python-request#track-flask-applications

@lenisha  #415 Here is my code enabling the azure tracing/appinsights

We've merged a PR (https://github.com/Azure-Samples/azure-search-openai-demo/pull/515) that adds opentelemetry and also updated the README with information about enabling it (it's opt in). Please file an issue if you see issues with it.


option to not get publications for an author

Sometimes it might be useful to get the statistics for an author but not his/her publication (the parsing takes quite long). This adds an option to the fill method to speed up fetching without publications.

Good point, it currently goes through every page for the user. I made some changes to the code to make sure tests were working, so I can't directly merge this PR. Could you please revisit? An alternative is to make the publications list a generator.

I took a stab at a generator. Note that this replaces the self.publications line and adds a generator function to the Author class. Opinions?
        # Generator for publications
        self.publications = self.__publication_generator(soup)

    def __publication_generator(self, soup):
        """ This is a generator for publications, which can be more efficient than populating on fill. """
        url_citations = _CITATIONAUTH.format(self.id)

        pubstart = 0
        while True:
            for row in soup.find_all('tr', class_='gsc_a_tr'):
                yield Publication(row, 'citations')
            if 'disabled' not in soup.find('button', id='gsc_bpf_more').attrs:
                print('Querying Scholar')
                pubstart += _PAGESIZE
                url = '{0}&cstart={1}&pagesize={2}'.format(url_citations, pubstart, _PAGESIZE)
                soup = _get_soup(_HOST+url)
            else:
                break

Addressed with https://github.com/OrganicIrradiation/scholarly/pull/78

I think the publication generator is a good idea. And I believe it could be tied in with #59 . However, items for a separate issue and pull request.


Use build-time versioneer, stop pinning versioneer as a dependency

Closes N/A
I noticed this when attempting to add fideslang as a dependency to another project. No need to have it as a runtime dependency.
Code Changes

[x] Move to using build instead of setup.py
[x] Upgrade versioneer to latest
[x] Configure versioneer in pyproject.toml
[x] Migrate to build-time dependency configuration

Steps to Confirm

[ ] Note that versioneer still works!

Pre-Merge Checklist

[ ] All CI Pipelines Succeeded
[ ] Documentation Updated
[ ] Issue Requirements are Met
[ ] Relevant Follow-Up Issues Created
[ ] Update CHANGELOG.md

Description Of Changes
This is mainly so that the built distribution doesn't rely on versioneer!

@ewdurbin great change! Didn't realize build-time was possible, I guess it's time to fully move over to pyproject.toml land 🙂


Possible NullReferenceException when using Nested Classes in Model

tl;dr
When the expression argument is for a property on a sub-model, such as Model.Address.SelectedCity, and the model is instantiated but the submodel is null, rendering will fail with a NullReferenceException.
Detail:
When evaluating the current property value during render so that it can be pre-selected in the drop-down, rendering can fail if the property is on an uninstantiated sub-class / sub-model.
If the control's expression argument is for a property that is directly on the model such as Model.SelectedCity, and the model is null, the control handles the null model instead of throwing. However, if the model is instantiated, but the control's expression argument is for a property on a sub-model such as Model.CurrentAddress.SelectedCity, and the submodel is null, evaluating the expression throws a NullReferenceException and rendering fails.
namespace Mvc.CascadeDropDown.Test.Models
{
    using System.ComponentModel.DataAnnotations;
    using System.Web.Mvc;

    public class CascadingDropdownsModel
    {
        public IList<SelectListItem> Countries { get; set; }

        public AddressModel CurrentAddress { get; set; }
    }

    public class AddressModel
    {
        [Required]
        public string SelectedCountry { get; set; }

        [Required]
        public string SelectedCity { get; set; }

        [Required]
        public int? SelectedStreet { get; set; }
    }
}

        public ActionResult Index()
        {
            var model = new CascadingDropdownsModel
            {
                Countries =
                    new List<SelectListItem>
                    {
                        new SelectListItem
                        {
                            Text = "US",
                            Value = "US"
                        },
                        new SelectListItem
                        {
                            Text = "UK",
                            Value = "UK"
                        },
                    },
                CurrentAddress = null
            };
            return View(model);
        }

        <div class="form-group">
            @Html.CascadingDropDownListFor(expression: m => m.CurrentAddress.SelectedCity,
                                triggeredByProperty: m => m.CurrentAddress.SelectedCountry,
                                url: Url.Action("GetCities", "Home"),
                                ajaxActionParamName: "country",
                                disabledWhenParentNotSelected: true,
                                htmlAttributes: new { @class = "form-control" },
                                options: new CascadeDropDownOptions
                                {
                                    BeforeSend = "beforeSend",
                                    HttpMethod = "post",
                                    OnCompleteGetData = "onComplete",
                                    OnFailureGetData = "onFailure",
                                    OnSuccessGetData = "onSuccess"
                                })
            @Html.ValidationMessageFor(m => m.CurrentAddress.SelectedCity)
        </div>

The exception is thrown within GetPropStringValue when the control is attempting to get the current value of the property.
https://github.com/alexanderar/Mvc.CascadeDropDown/blob/5641ced7fccd8975bc124c11e044978cf72d1e8a/Mvc.CascadeDropDown/DropDownListExtensions.cs#L539
Expected Behavior:
The control handles the NullReferenceException when evaluating the expression argument, returning an empty string similar to when the model is null, and allowing the page to fully render.

@jayharris
I fixed the issue and released a new 2.0.2 version.
I am waiting for your feedback on this issue to close ticket.


[Bug]: Failed deployment with index errors

What happened?
When running cargo shuttle deploy (or using the github action), the build fails with a index error.
Version
v0.32.0
Which operating system(s) are you seeing the problem on?
In deployment
Which CPU architectures are you seeing the problem on?
In deployment
Relevant log output
{error="Build error: `cargo metadata` exited with an error:     Updating `shuttle-crates-io-mirror` index\nerror: failed to select a version for the requirement `itertools = \"^0\"` (locked to 0.12.0)\ncandidate versions found which didn't match: 0.11.0, 0.10.5, 0.10.4, ...\nlocation searched: `shuttle-crates-io-mirror` index (which is replacing registry `crates-io`)\nrequired by package `project v0.1.0 (/opt/shuttle/shuttle-builds/project)`\nperhaps a crate was updated and forgotten to be re-vendored?\n"} Service build encountered an error

Duplicate declaration

[X] I have searched the issues and there are none like this.

Also, it says "failed to fetch crates" which could imply the Deployer is not correctly fetching crates.

This error happens due to our crates-io mirror not having the latest version of a crate. The crate was released 11 hours ago at the time of writing, so it is strange that our mirror does not have it yet.
To temporarily resolve this, you can either revert the lock file to an older one (if you have that option), or add a version lock  to an older version, for example itertools = "=0.11.0".
We will investigate why the mirror is falling behind.

We have corrected the issue and started syncing the mirror again. It should be possible to use new crates now.


Set camera parameters via file -> parameter not found

Hi,
when I set the parameters e.g. "intg_time" via the HaddockCDKCameraNormal.conf file the following error comes up while starting my program.
ERROR: DepthCamera:Could not get value for parameter HaddockCDKCamera(0::0451:9100::CK91001001235200122T).intg_time
ERROR: DepthCamera: Failed to update value for parameter 'unambiguous_range'
ERROR: DepthCamera: Failed to update value for parameter 'intg_time'
ERROR: DepthCamera:Could not get value for parameter HaddockCDKCamera(0::0451:9100::CK91001001235200122T).intg_time
ERROR: DepthCamera: Failed to update value for parameter 'unambiguous_range'
ERROR: DepthCamera: Failed to update value for parameter 'intg_time'
ERROR: DepthCamera:Could not get value for parameter HaddockCDKCamera(0::0451:9100::CK91001001235200122T).intg_time
ERROR: DepthCamera: Failed to update value for parameter 'unambiguous_range'
ERROR: DepthCamera: Failed to update value for parameter 'intg_time'
ERROR: CameraSystem: Could not refresh parameters for HaddockCDKCamera(0::0451:9100::CK91001001235200122T).
INFO: Successfully loaded depth camera for device 0::0451:9100::CK91001001235200122T
INFO: ToFCameraBase: Starting with 320x240@20fps
If I set the same parameter with the VoxelCLI via the command set intg_time = 30 the parameter could be set correctly.
Could anybody please tell me the reason?
Thanks in advance.

Could you share your modified conf file?
Also, please clarify whether streaming happens in the case where you set
parameter via conf file.
Thanks
On 28 February 2016 at 18:01, christophk86<EMAIL_ADDRESS>wrote:

Hi,
when I set the parameters e.g. "intg_time" via the
HaddockCDKCameraNormal.conf file the following error comes up while
starting my program.
ERROR: DepthCamera:Could not get value for parameter
HaddockCDKCamera(0::0451:9100::CK91001001235200122T).intg_time ERROR:
DepthCamera: Failed to update value for parameter 'unambiguous_range'
ERROR: DepthCamera: Failed to update value for parameter 'intg_time' ERROR:
DepthCamera:Could not get value for parameter
HaddockCDKCamera(0::0451:9100::CK91001001235200122T).intg_time ERROR:
DepthCamera: Failed to update value for parameter 'unambiguous_range'
ERROR: DepthCamera: Failed to update value for parameter 'intg_time' ERROR:
DepthCamera:Could not get value for parameter
HaddockCDKCamera(0::0451:9100::CK91001001235200122T).intg_time ERROR:
DepthCamera: Failed to update value for parameter 'unambiguous_range'
ERROR: DepthCamera: Failed to update value for parameter 'intg_time' ERROR:
CameraSystem: Could not refresh parameters for
HaddockCDKCamera(0::0451:9100::CK91001001235200122T). INFO: Successfully
loaded depth camera for device 0::0451:9100::CK91001001235200122T INFO:
ToFCameraBase: Starting with 320x240@20fps
If I set the same parameter with the VoxelCLI via the command set
intg_time = 30 the parameter could be set correctly.
Could anybody please tell me the reason?
Thanks in advance.
—
Reply to this email directly or view it on GitHub
https://github.com/3dtof/voxelsdk/issues/65.

--
~ Prasad Bhat

My two modified conf files:
HaddockCDKCameraNormal_Modified.zip
The streaming starts with the modified conf files, but the error messages above.

I think the problem is that you are setting intg_time to 100 which
integration duty cycle is 100%. That might be the cause of this issue. Can
you try setting for instance 30 in configuration and see whether this
problem recurs?
On 28 February 2016 at 22:04, christophk86<EMAIL_ADDRESS>wrote:

My two modified conf files:
HaddockCDKCameraNormal_Modified.zip
https://github.com/3dtof/voxelsdk/files/149993/HaddockCDKCameraNormal_Modified.zip
The streaming starts with the modified conf files, but the error messages
above.
—
Reply to this email directly or view it on GitHub
https://github.com/3dtof/voxelsdk/issues/65#issuecomment-189899713.

--
~ Prasad Bhat

I restart the eval kit.
I set the integration duty cycle to 30. -> No error
I set the integration duty cycle back to 100 -> No error.
I check the illumination power by "get illum_power_percentage" in the VoxelCLI.
In the conf file it set to 75, but the return value from "get illum_power_percentage" was 20%.
It seems that the values of the conf file are not set.

Could you please switch off/on the board (NOT RESET) and try again?
The current profile ID is stored in temporary register. If applied profile
ID is same as current profile ID then parameters are not freshly applied
causing this confusion.
However, the actual problem is still than after last switch on, profile
with intg_time = 100 was tried which failed, but current profile ID got
saved. Subsequent changes in profile defining_params did not get applied by
design.
Thanks.
On 28 February 2016 at 22:37, christophk86<EMAIL_ADDRESS>wrote:

I restart the eval kit.
I set the integration duty cycle to 30. -> No error
I set the integration duty cycle back to 100 -> No error.
I check the illumination power by "get illum_power_percentage" in the
VoxelCLI.
In the conf file it set to 75, but the return value from "get
illum_power_percentage" was 20%.
It seems that the values of the conf file are not set.
—
Reply to this email directly or view it on GitHub
https://github.com/3dtof/voxelsdk/issues/65#issuecomment-189907908.

--
~ Prasad Bhat

Ok, switch off/on solve the problem.
Thank you for the solution and explanation.


Account for multiple fs with the same target name

The MGS target has the same name for multiple filesystems. As an
example, a filesystem created with an MGT on MDS1 and a filesystem
created with an MGT on OSS1 will both have a target name of MGS. This
needs to be accounted for when building the target table.
Signed-off-by: johnsonw<EMAIL_ADDRESS>

This change is

Looks like this is introducing a number of clippy warnings.


Update colorize_labels tool

For the contributor:

[x] I have read the CONTRIBUTING.md document.
[x] License permits unrestricted use (educational + commercial).
[x] This PR adds or updates a tool or tool collection.

[ ] This PR adds a new tool or tool collection.
[x] This PR updates an existing tool or tool collection.
[x] Tools added/updated by this PR comply with the Naming and Annotation Conventions for Tools in the Image Community in Galaxy (or explain why they do not).

[ ] This PR does something else (explain below).

Update colorize_labels tool. The colors are now distributed more evenly, so that distinct objects are easier to perceive.

I will remove the old version and just install the new version.


feat[imdsv2]: support for imdsv2

Description
This PR updates the _get_instance_id() function to support AWS IMDSv2 (version 2 of the AWS metadata endpoint). Right now, prodsec and infrasec are trying to enable this feature to help defend against Server-Side Request Forgery (SSRF) attacks, but it requires that call calls to the metadata endpoint first make a PUT request to grab a short-lived token and then make a GET request to the metadata endpoint with that token in the header.
References

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/configuring-instance-metadata-service.html
https://czi.atlassian.net/wiki/spaces/INFRASEC/pages/2090696711/AWS+Metadata+V2+Migration
https://github.com/chanzuckerberg/SSRFs-Up
https://aws.amazon.com/blogs/security/defense-in-depth-open-firewalls-reverse-proxies-ssrf-vulnerabilities-ec2-instance-metadata-service/
https://github.com/chanzuckerberg/SSRFs-Up/wiki/FAQ#i-read-the-other-docs-but-i-still-dont-really-get-why-this-matters-can-you-provide-an-example-of-how-ssrf-affects-my-application

I'm closing this PR in favor of #599 since we don't need this functionality at all anymore. I'm continuing conversations with @jacobrheath to see if there's anything else we need to do to support IMDSv2

Thanks!


Raise InvalidError in more places

Previously, deploys of secrets that were invalid or conflicted with another secret would raise an GRPCError from the client, or an internal server error on the web.
This translates them to the correct client error type.
Before (web UI)

After

Resolves MOD-1612.

:100


Testcontainers via test-resources version catalog

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


[Question] Correctly detect app using WKWebView

README.md currently states:
An easy way to verify that WKWebView has been installed on iOS is to check if window.indexedDB exists.  For example:
if (window.indexedDB) {
   console.log("I'm in WKWebView!");
} else {
   console.log("I'm in UIWebView");
}

The thing is that window.indexedDB is true on iOS 10 whether an app is using this plugin or not.
Any better way to detect if the app itself is using WKWebView?

ping @jgw96

@zarko-tg Are you running this test in safari on ios 10 or in a cordova app?

@jgw96 cordova/ionic 1.3.x app.

@zarko-tg I am currently working on a method to accurately detect whether an app is running in wkwebview no matter the version of iOS you are using. Thanks!

Any update about this @jgw96?

I'd like to know about this as well. Thanks! :)


Adding an unblock method?

Is it possible to loosen on the rules a little bit and add an unblock method? E.g. by file streams we could open multiple stream after each other with the same flow...

I don't think this feature is needed. We can always replace the flow to a new one, and voila everything is solved. The only thing we might need is handling a blocked await more efficiently, probably with an event instead of a throw. That should be nice because the Pump should be async after all.


tls1.3~

https://github.com/bifurcation/mint
A Minimal TLS 1.3 Implementation in Go

https://github.com/cloudflare/tls-tris  get
覆盖  go/src/crypto/tls
ok~


EsLint Plugin Lit A11y

Is your feature request related to a problem? Please describe.
There is already some baseline a11y testing that we support with our unit tests. Can we be better?
Describe the solution you'd like
Review https://open-wc.org/docs/linting/eslint-plugin-lit-a11y/overview/
Will this add value to our current scenario?
Describe alternatives you've considered
Not use this new library and stick with what we have.

This is already in the Auro eslint config - see https://github.com/AlaskaAirlines/eslint-config-auro/issues/2.
https://github.com/AlaskaAirlines/eslint-config-auro/blob/8a9dfc6a42f4e56e6e7fdf59ea82e1ef9d02b250/.eslintrc.js#L10


🐛 BUG: Template literal attributes aren't highlighted properly

Describe the Bug
Astro v0.21 introduce template literal attributes with the following syntax:
<div atrr=`some ${value}`>

These currently don't have any syntax highlighting.
Steps to Reproduce

Paste the snippet above into an .astro file

Syntax highlighting was added in https://github.com/withastro/astro-language-tools/pull/115


[Bug] how to use svelte csf with storyshots?

Describe the bug
I'am using svelte to describe my stories, but storyshots doesn't work with this syntax, only works with .js and .ts files.
Steps to reproduce the behavior

using this template https://github.com/chromaui/intro-storybook-svelte-template
install storyshots following what is described here: https://storybook.js.org/tutorials/intro-to-storybook/svelte/en/simple-component/
add "^.+\\.stories\\.svelte$": "@storybook/addon-svelte-csf/dist/cjs/jest-transform" to the jest transform map
run npm test storybook.test.js

Here is an example Repo: https://github.com/01oseluiz/svelte-csf-storyshots

Expected behavior
It is expected that storyshots takes a snapshot of the 'Button.stories.svelte '. But only 'Button.stories.js' is working.
Screenshots and/or logs
 console.warn
    Unexpected error while loading ./stories/Button.stories.svelte: SyntaxError: Unexpected token '<'

      at Object.warn (node_modules/@storybook/client-logger/dist/cjs/index.js:67:65)
      at node_modules/@storybook/core-client/dist/cjs/preview/loadCsf.js:92:34
          at Array.forEach (<anonymous>)
      at node_modules/@storybook/core-client/dist/cjs/preview/loadCsf.js:85:20
          at Array.forEach (<anonymous>)
      at node_modules/@storybook/core-client/dist/cjs/preview/loadCsf.js:84:12
      at ConfigApi.configure (node_modules/@storybook/client-api/dist/cjs/config_api.js:26:7)

 PASS  ./storybook.test.js
  Storyshots
    Example/Button-js
      ✓ Large (16 ms)
      ✓ Primary (4 ms)
      ✓ Secondary (4 ms)
      ✓ Small (4 **ms)**

Environment
Environment Info:

  System:
    OS: Linux 5.10 Ubuntu 20.04.3 LTS (Focal Fossa)
    CPU: (4) x64 Intel(R) Core(TM) i5-7300HQ CPU @ 2.50GHz
  Binaries:
    Node: 12.16.1 - ~/.asdf/installs/nodejs/12.16.1/bin/node
    Yarn: 1.22.10 - /home/linuxbrew/.linuxbrew/bin/yarn
    npm: 6.13.4 - ~/.asdf/installs/nodejs/12.16.1/bin/npm
  npmPackages:
    @storybook/addon-actions: ^6.3.12 => 6.3.12
    @storybook/addon-essentials: ^6.3.12 => 6.3.12
    @storybook/addon-links: ^6.3.12 => 6.3.12
    @storybook/addon-svelte-csf: ^1.1.0 => 1.1.0
    @storybook/svelte: ^6.3.12 => 6.3.12

Additional context
Add any other context about the problem here.

This project has a demo storybook which uses jest and storyshot. You can look at the jest configuration.


Credentials: 'include' not working when using application/json Content-Type

When I'm making POST cors request with credentials using Content-Type: 'application/json' it won't send cookies
fetch(`${baseUrl}/Comments/AddComment/${taskId}`, {
      credentials: 'include',
      method: 'POST',
      body: JSON.stringify(values),
      headers: new Headers({
        'Accept': 'application/json',
        'Content-Type': 'application/json'
      }),
    })

However, if I change Content-Type to application/x-www-form-urlencoded (like in this comment) and change JSON.stringify to formurlencoded from form-urlencoded package, cookies are sended properly, so it's kinda workaround

When using PUT request, credentials: 'include' isn't woking either, but in that case even application/x-www-form-urlencoded isn't helping. As a second workaround I temporarily changed all my PUT and DELETE (which I didn't check by the way) to POST. Semantics is ruined, but thanks gods, it isn't a public api.

Sorry, it was poor ASP.NET WebAPI OPTIONS request handling


Allow creation of Mojaloop peers

Context
In order to be able to support Mojaloop as payment method, we need to be able to initialize the peering relationship similar to how we do with ILP.
Tasks

[ ] Make peers table more method agnostic
[ ] Create createMojaloopPeer Admin API

Closing in favour of larger Rafiki architecture overhaul


"Target a Specific Child of an Element Using jQuery" detail

Challenge http://www.freecodecamp.com/challenges/waypoint-target-a-specific-child-of-an-element-using-jquery has an issue.
The instructions say to input $(".well :nth-child(2)").addClass("animated bounce"); or $(".target:nth-child(2)").addClass("animated bounce"); —and in fact, both work well—, but I doubt most beginners will gather this nuance from the text.
I believe an explanation is due on how the one with space searches for the second child inside, whereas the one without searches for the second sibling.

Also, I need coffee haha

Hey thanks for posting!
See #1927 for my explanation there.  It is still open and will can remain as the main issue for this problem.  Feel free to close this as a duplicate.
Also, take a look at our Bug Posting Guidelines to see the best way to resolve a problem with the site.

Closing as duplicate


Document.createTouch() is deprecated

source: https://developer.mozilla.org/en-US/docs/Web/API/Document/createTouch
https://github.com/adobe-webplatform/Snap.svg/blob/b242f49e6798ac297a3dad0dfb03c0893e394464/src/mouse.js#L17
this should be replaced with some other way to check for touch capabilities

This has been resolved in this pull request - https://github.com/adobe-webplatform/Snap.svg/pull/616
Any chance of getting this merged?


Provide option to hide tags in Transactions view

The recent change to show tags on the Transactions view requires much more scrolling to see the same number of transactions. Screenshot below is typical of tags when using the Firefly importer through GoCardless.
Would it be possible to add a setting to allow all tags to be hidden in the Transactions view?
Sightly better might be to give the user the option to only show specific tags if they exist.

Works perfectly, thanks very much!


[Feature] Add Playlist Carousel

Description
Playlist Carousel
Screenshots

Checklist

[X] I have checked the existing issues.
[X] I follow Contributing Guidelines of this project.
[X] I am a IWOC'24 contributor.
[ ] I am a JWOC'24 contributor.
[ ] I am a SWOC'24 contributor.

Code of Conduct

[X] I agree to follow this project's "Code of Conduct".

Proceed @AbhineshJha


Fix links pointing to dev branch to point to master branch

Found some more links pointing to non-existing dev branches. Also found a link in docfx.json which was non-existent.

@Rutix  thanks again. cc/ @guardrex


Show 3d model results in the search result page

Problem
For the 3D model support milestone, we need to display 3D model results on the search results page in two ways, as part of the All content layout and as a unique view.
Considerations
Try reusing the All content layout for the standalone view showing only 3D model results.
Solution
Here are two screen recordings with the whole flow from the search results page to the single result view, and the mockups related to the search results.
Desktop
Screen recording
https://user-images.githubusercontent.com/895819/169101493-7185bbeb-9f77-48d5-81e5-61d8ea895bde.mp4
Mockups

All content
All content + Switcher open

Search results. 3D models

Mobile
Screen recording
https://user-images.githubusercontent.com/895819/169101916-9ffe5a59-152d-45a8-92aa-ceefd640835c.mp4
Mockups

All content
Content switcher
Search results. 3D models

Mockup
Figma: 3D model support
Description
On desktop, the 3D model component is the same box as the image one for both search results and 3d model results. Following the currently implemented behavior, the box also resizes as the filter sidebar appears.
On mobile, the element is the same image box as in all content results; but on 3d results-only, the element is wider like in the single result view and it has the license visible below. I am thinking of using the same single-column layout for image results after we discussed keeping the row layout on desktop. This should ease the arrangement of elements.
I am hesitant about showing the content links to see Images, Audio, or 3D models on both desktop and mobile. They fit perfectly on desktop, despite not working well with future content integrations, but also because the list pushes the content downwards significantly on mobile. What are your thoughts on this?
Looking forward to see your comments

Closing as 3d models are indefinitely on hold and no work should be done on them until we re-plan this project.
The information and plans from this thread will absolutely be taken into account if/when we revisit 3d models.


test: add integration testing

Fixes #341

TODO:

[ ] Use TypeScript in tests
[ ] Fix issue where tests always fail on first run
[ ] Signal when extension is ready instead of waiting
[ ] Fix Unable to resolve filesystem provider with relative file path 'flixcompiler:141183124/CHANGELOG'
[ ] Write FlixRelease to somewhere persistent to avoid downloading compiler every time (unless it should just be part of the test?)
[ ] GitHub action

Fix issue where tests always fail on first run

Seems to be entirely dependent on the existence of .vscode-test/user-data/Workspaces/<some number>/workspace.json.
If this does not exist the test will quit after encountering the first await.

Got this far with GitHub action
- run: |
        apt-get update -y
        apt-get install libnss3-dev libgdk-pixbuf2.0-dev libgtk-3-dev libxss-dev -y
        apt-get install libasound2 -y
        # Run tests through Xvfb to give Electron a screen to draw to
        apt-get install xvfb -y
        xvfb-run -a npm test

But currently blocked by
| Warning: vkCreateInstance: Found no drivers!
| Warning: vkCreateInstance failed with VK_ERROR_INCOMPATIBLE_DRIVER

Does the internet say anything about that? You would think it would be possible to run on GitHub CI since VSCode and GitHub are both Microsoft.

Maybe this is useful: https://github.com/microsoft/vscode-test/blob/master/sample/azure-pipelines.yml

https://code.visualstudio.com/api/working-with-extensions/continuous-integration#github-actions

https://code.visualstudio.com/api/working-with-extensions/continuous-integration#github-actions

This was the one I followed, but at least when running locally with act, it needs a lot of extra installations.
Perhaps it works when running it on the real thing. I can try pushing it.
The installations also take quite a lot of time each run, so either way, it might also be preferable to create a Docker image to run on.

Do you want to wrap this PR up and then we can try GitHub CI in a new PR? I think that would be best.

Ill give some feedback on the code.

Write FlixRelease to somewhere persistent to avoid downloading compiler every time (unless it should just be part of the test?)

Lets let it be part of the test!

Do you want to wrap this PR up and then we can try GitHub CI in a new PR? I think that would be best.

It should be working as is, so sounds good 👍

Roger. So if you cleanup my comments then we merge it in. Then in a subsequent PR we can fiddle with the GitHub CI part.

Can we also get a Readme about how to run the tests?

Yes please. @sockmaster27 Can you add it to README.md?

Merged! What are the next steps?


Fix/statement logs

Hi,
thank your for initiative to contribute to this project. Could you please provide some information regarding the change you made?

Pipeline is failing:

#30 ERROR: failed to push ghcr.io/exacaster/lighter:fix-statement-logs-spark3.3.3: unexpected status from POST request to https://ghcr.io/v2/exacaster/lighter/blobs/uploads/: 403 Forbidden

Something is wrong with the authorization

I've cherry-picked commits related to the fix. Regarding CI changes, we've chosen a different approach for selectively building images from Pull requests.
Thank you for your contribution!


After shrinking of image for raspberry pi there are mounting problems when booting raspi

After shrinking an raspberry pi image and flashing it to a smaller SD-card, the raspi boot sucessfully but with a different kind of mount structure:  no mounting of partition 1, partition 2 is mounted read-only
The shrinked image was put to SD-cards in two manners: balenaEtcher and dd, both shows the same result.
See output below of fdisk -l, lsblk and mount.
Original 32GB SD card:
Device         Boot  Start      End  Sectors  Size Id Type
/dev/mmcblk0p1        8192   532479   524288  256M  c W95 FAT32 (LBA)
/dev/mmcblk0p2<PHONE_NUMBER>3951 61801472 29,5G 83 Linux

mmcblk0     179:0    0 29,7G  0 disk
├─mmcblk0p1 179:1    0  256M  0 part /boot
└─mmcblk0p2 179:2    0 29,5G  0 part /

mount | grep "/dev/"
/dev/mmcblk0p2 on / type ext4 (rw,noatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
mqueue on /dev/mqueue type mqueue (rw,relatime)
/dev/mmcblk0p1 on /boot type vfat (rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,errors=remount-ro)

Shrinked version on 16GB card:
Device         Boot  Start      End  Sectors  Size Id Type
/dev/mmcblk0p1        8192   532479   524288  256M  c W95 FAT32 (LBA)
/dev/mmcblk0p2<PHONE_NUMBER>6287 30583808 14,6G 83 Linux

mmcblk0     179:0    0 14,9G  0 disk
├─mmcblk0p1 179:1    0  256M  0 part
└─mmcblk0p2 179:2    0 14,6G  0 part /

mount | grep "/dev/"
/dev/mmcblk0p2 on / type ext4 (ro,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
mqueue on /dev/mqueue type mqueue (rw,relatime)

The 1st partition is not mounted?, raspi boots nevertheless, and the 2nd partition is mounted read-only!
Thus the grafical system did not start.
sudo mount -o remount -o rw /dev/mmcblk0p2
will start the X11 Session.
Using an SD-Card-Adapter on Linux, the SD-Card looks like pretty:
sdb                      8:16   1  14,9G  0 disk
├─sdb1                   8:17   1   256M  0 part /media/andi/boot
└─sdb2                   8:18   1  14,6G  0 part /media/andi/rootfs
Here the log of the shrinking process:
sudo pishrink.sh Bilderrahmen.img Bilderrahmen-shrinked.img

pishrink.sh v0.1.2
pishrink.sh: Copying Bilderrahmen.img to Bilderrahmen-shrinked.img... ...
pishrink.sh: Gathering data ...
Creating new /etc/rc.local
pishrink.sh: Checking filesystem ...
rootfs: 113564/1876672 files (0.3% non-contiguous), 1474723/7725184 blocks
resize2fs 1.45.5 (07-Jan-2020)
pishrink.sh: Shrinking filesystem ...
resize2fs 1.45.5 (07-Jan-2020)
Resizing the filesystem on /dev/loop0 to 1430172 (4k) blocks.
Begin pass 2 (max = 103980)
Relocating blocks             XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Begin pass 3 (max = 236)
Scanning inode table          XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Begin pass 4 (max = 9411)
Updating inode references     XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
The filesystem on /dev/loop0 is now 1430172 (4k) blocks long.

pishrink.sh: Shrinking image ...
pishrink.sh: Shrunk Bilderrahmen-shrinked.img from 30G to 5.8G ...

What can I do?

The solution is to include 3 lines to the empty file /etc/fstab
proc            /proc           proc    defaults          0       0
PARTUUID=xxxxxxxxx-01  /boot           vfat    defaults          0       2
PARTUUID=xxxxxxxxx-02  /               ext4    defaults,noatime  0       1


Hb streamx 0324 1.12.2release

What problem does this PR solve?
Issue Number: close #xxx
Problem Summary:
What is changed and how it works?
Proposal: xxx
What's Changed:
How it Works:

I'm very glad to see your contribution. You can attach relevant design drawings, architecture diagrams and other relevant information. Next, we need to have a full discussion


Set experiment as Default on create run/schedule page

JIRA: RHOAIENG-9429
Description

When creating a run/schedule from the pipelines page, it will auto-fill the experiment field as Default if the Default experiment is in the selector (not archived on the create run page).

How Has This Been Tested?

Create a run from the pipeline version row or details page
See if the Default experiment is auto-filled
Archive the Default experiment
When creating a run, the experiment field should be empty
When creating a schedule, the experiment field should still be set to Default
Try to duplicate a run/schedule, make sure this change doesn't have an impact on the field
Try to create a run/schedule from the experiment runs page, make sure this change doesn't have an impact on the field
Try to switch between create run/schedule page, make sure this change doesn't have an impact on the field

Test Impact

Add a cypress test to verify the auto-filled experiment.
Request review criteria:

Self checklist (all need to be checked):

[x] The developer has manually tested the changes and verified that the changes work
[x] Commits have been squashed into descriptive, self-contained units of work (e.g. 'WIP' and 'Implements feedback' style messages have been removed)
[x] Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
[x] The developer has added tests or explained why testing cannot be added (unit or cypress tests for related changes)

If you have UI changes:

[ ] Included any necessary screenshots or gifs if it was a UI change.
[ ] Included tags to the UX team if it was a UI/UX change (find relevant UX in the SMEs section).

After the PR is posted & before it merges:

[ ] The developer has tested their solution on a cluster by using the image produced by the PR to main

@DaoDaoNoCode , If experiment is more than 10, experiment along with default gets hidden in view more, so initially the field will be empty and when I click on view more I get  default field
https://github.com/user-attachments/assets/881ca851-75a5-4969-a38d-bee6b1683c51

@jpuzz0 I updated the PR, and the issue you found above is not related to this PR, could you submit a ticket for it?

@jpuzz0 I updated the PR, and the issue you found above is not related to this PR, could you submit a ticket for it?

https://issues.redhat.com/browse/RHOAIENG-10208


run_openai_gpt.py issues with Adamw

Hi
Adamw in this script has parameters not existing anymore, ...
Thanks for updates in advance.

Yes, so this should be fixed by (your own :) PR #845 Thanks again!

thanks :)
Best regards,
Rabeeh
On Tue, Jul 23, 2019 at 3:30 PM Thomas Wolf<EMAIL_ADDRESS>wrote:

Yes, so this should be fixed by (your own :) PR #845
https://github.com/huggingface/pytorch-transformers/pull/845 Thanks
again!
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/huggingface/pytorch-transformers/issues/837?email_source=notifications&email_token=ABP4ZCAGTX6DNAYMBCHNORTQA4BYFA5CNFSM4IFGMGK2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2TDV6Y#issuecomment-514210555,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABP4ZCGDZR3TG2SS2LXIBTDQA4BYFANCNFSM4IFGMGKQ
.

Thanks @rabeehk! #845 with flat learning rate gives a good result of 87.2% on ROCStories. Here are the args (the defaults in the file work well).
python run_openai_gpt.py  \
  --model_name openai-gpt  \
  --do_train  \
  --do_eval  \
  --train_dataset "./ROCStories/cloze_test_val__spring2016 - cloze_test_ALL_val.csv"  \
  --eval_dataset "./ROCStories/cloze_test_test__spring2016 - cloze_test_ALL_test.csv"  \
  --train_batch_size 8  \
  --eval_batch_size 16  \
  --num_train_epochs  3

@prrao87  great :)


Can tests running in a Docker container run in a temporary directory instead?

See https://github.com/ClimeTrend/DMD-ERA5/pull/14/files/71795001e2e9f3e9a5099dbe7d4626ecf8adeb2c#r1861114346

@ots22 wrote:

As far as I can tell, Docker is used only to create a fresh repository and to then run some tests that mutate the git repository with some git/dvc commands. Would isolating these tests in a temporary directory be enough, rather than in a full-blown container? That is, could this be a shell script instead of a Dockerfile?

@dsj976 replied:

yes, could be done, and it would definitely be a cheaper setup for testing. However, given how the package works currently, I see one main disadvantages to doing that: you currently need to run the tool from the root of the repo, and cannot choose where the downloaded data will be saved. Additionally, I'm personally not a big fan of having tests creating/deleting files on my local file system. Will open an issue about this in any case, and can be looked at in the future.

I see one main disadvantages to doing that: you currently need to run the tool from the root of the repo, and cannot choose where the downloaded data will be saved

It sounds like this might be easier to solve after #17 then, although shouldn't it be possible just to run the same commands in a script rather than a Dockerfile (making a fresh clone of the repository if that's necessary for instance)?

I'm personally not a big fan of having tests creating/deleting files on my local file system.

I don't think it is unreasonable for a test to interact with the filesystem if it's related to what it's testing! It's possible to do this safely, e.g. with mktemp to give you a unique temporary directory.
The other benefit of running 'locally' is that it tests with the versions of the tools that you have installed, rather than some other version that happens to get installed in the container (even on a different OS potentially).
Containers provide a much stronger degree of isolation than is really needed at the cost of quite a heavy dependency.


Multi GPU support

Given there is already support for nccl, whats the overhead to add support for multi node gpu support for training/inference

There's 2 different ways to do multi-gpu: multi-device and multi-host. We'll need to do both to truly reach llm scale training, but we should start with multi-device.
The primary task to be solved is figuring out the optimal way to distribute nodes to different devices. In theory, a compiler can go through a complete graph and, with the knowledge of the available devices, make decisions on which ops should happen where.
A few decisions need to be made before any progress happens:

Does this partitioning happen before or after the bulk of the cuda compilers (if before, how can we deal with the changing compute from downstream compilers, if after, how do we get the charactaristics of each op (compute intensity, etc.) in a standardized way and/or handle the case where we don't know about an op).
How can we do parallelism. It's fine if we have a graph distributed among many gpus, but currently luminal is set up to run singular ops at once, so how can we run multiple ops at once. A seperate MultiGpuOp that contains multiple ops on each device and runs them at once?
Different parallelism has different compiler considerations here

Data parallel
Pipeline parallel
Tensor parallel

I'm absolutely interested in moving forward on this, but I need to dedicate some time to really thinking deeply on the best path.

There's 2 different ways to do multi-gpu: multi-device and multi-host. We'll need to do both to truly reach llm scale training, but we should start with multi-device.
The primary task to be solved is figuring out the optimal way to distribute nodes to different devices. In theory, a compiler can go through a complete graph and, with the knowledge of the available devices, make decisions on which ops should happen where.
A few decisions need to be made before any progress happens:

Does this partitioning happen before or after the bulk of the cuda compilers (if before, how can we deal with the changing compute from downstream compilers, if after, how do we get the charactaristics of each op (compute intensity, etc.) in a standardized way and/or handle the case where we don't know about an op).

How can we do parallelism. It's fine if we have a graph distributed among many gpus, but currently luminal is set up to run singular ops at once, so how can we run multiple ops at once. A seperate MultiGpuOp that contains multiple ops on each device and runs them at once?

Different parallelism has different compiler considerations here

Data parallel
Pipeline parallel
Tensor parallel

I'm absolutely interested in moving forward on this, but I need to dedicate some time to really thinking deeply on the best path.

I agree that multi-device first. Candle has this interesting multi-process example. You can even adapt that for multi-device using mpi or just nccl and build a communication layer p2p. In terms of a complete graph of ops, nccl handles inter/intra-node well.
Notes;

Heterogenous HW inference or training is challenging.
I looked into deepspeed and pytorch distributed modules and they support many model parallelism techniques with different backends (gloo, mpi, nccl) or a combination.
I found this recently async cuda could be interesting to introduce.
one point about sequential operations in luminal, to support parallelism the main thread (initiator of the inference or training in an all reduce or ring reduce) must not be blocked. After that it depends on which algorithm you will use (Data, pipeline, tensor parallel). Pipeline executes different model stages concurrently on separate nodes. But thats concurrently.

Goals

nodes must not wait for other nodes to finish
async cuda
in terms of DAG, i created an example recently for fune tuning across k8s nodes using argo DAG workflows. A similar approach can adopted for DAG-based jobs
start with a small example

This is a very relevant topic for applications. @jafioti to be honest, and I am not an expert on the approach taken by Luminal, it makes sense to me to have tensor parallelism at the graph level, so graph ops can be distributed across multiple GPUs.
On the other hand, having full graph operations being distributed across distinct devices could lead to some subtleties around interdependencies of states across various devices.
Another topic that could be relevant to be taken into consideration in a future implementation is determinism. Given the non-transitive nature of all reduction operations for floating points, the order in which multi-GPUs finish their execution might have an impact onto the final output. Having an implementation that realizes full determinism (at least on a single machine) will be relevant for certain applications (even at a possible cost of performance).

jorgeantonio21

Determinism is essential. Given the project I am working on, cross-platform consistency and verifiable communication (moot) across nodes are paramount.
Given this paper Agatha, here are some conclusions we have taken into consideration in writing our paper:
Problem
As you said, a computation will usually give two different results on two different machines, even when the same source of randomness is used, whether in inference or training. This is due to the accumulation of rounding errors in floating-point arithmetic.
Even when using the same randomness seed, two different machine learning computations on two separate machines may produce different results due to potential variations in hardware architecture, software implementations, or runtime conditions. Different hardware's inherent parallelism and optimization strategies can lead to subtle numerical differences in floating-point operations, impacting the model's intermediate states during training or inference. Variations in software libraries, compiler optimizations, or even the underlying operating system can also contribute to discrepancies. Moreover, differences in the order of parallelized operations or the timing of asynchronous tasks may introduce subtle divergences in the computations.
While randomness seeds aim to provide reproducibility, these inherent system-level variabilities can cause small numerical divergences, resulting in distinct outcomes across different machines despite identical seeds.
Solutions
One way to address this issue is to use Quantization, which addresses the variability in computations by reducing the precision of numerical representations. In traditional neural networks, computations are performed using high-precision floating-point numbers, susceptible to subtle variations across different hardware architectures. Quantization involves mapping these floating-point values to a lower precision, typically fixed-point or integer representations. This reduction in precision accelerates inference by requiring less memory and computation and enhances reproducibility.

Another critical topic is inter-node training/inference communication efficiencies. As claimed in this section ml-engineering, the gap regarding comm overheads is closing. But this assumes all nodes operate and speak the same language with the same set of HW. What if they don't? Can we make them good enough for multi-GPU multi-node setups?

I think for now let's just think about how we can make this work on a single machine. Multi-machine determinism seems quite difficult (though maybe not impossible).
For single-machine inference, I'm not too concerned with slight differences in floating points, as inference is generally more accepting of errors (see low bit quantized inference). Training is much more sensitive, but if other frameworks are able to cope with the errors, we should be able to as well.
What I am interested in with regards to determinism is the ability to precisely hide latency. If we can deterministically know how long data travel takes, we can do things like ring-attention and pipeline parallelism by computing stuff while data is in transit, and know how long it will take for that data to arrive. A well choreographed dance where hardware util is maximized. This might be a ways away for now though.
For the immediate future, I think this can be tackled in a few steps:

Get tensor-parallel matmuls working
This is simple in that you can have matmuls over a certian size get split among N devices to do the same matmul. We'll likely need some additional op for copying data between devices (d-to-d) which will reference the shapetracker to see which data to copy (only part of the tensor)
Get device-parallel ops working, where say you have:
b = a.expensive_op1(); c = a.expensive_op2(); d = b + c;
Then the expensive_op1 and expensive_op2 can happen on different devices.

Then we can start thinking about pipeline parallelism. This will be non-trivial because luminal is fundamentally single-threaded, so it executes a single stream of ops at once. The likely solution here is to have wrapper ops that contain all parts of the pipeline and execute all parts of the pipeline at once on different devices. So if you have op1 -> op2 -> op3 and 3 devices, you'll compile to parallel(d1:op1, d2:op2, d3:op3) -> parallel_transfer -> parallel(d1:op1, d2:op2, d3:op3)  -> parallel_transfer -> parallel(d1:op1, d2:op2, d3:op3)
This can all be derived by a compiler given the original graph.
I'll have to think about this more, but the first step, tensor parallel matmuls, seems to be the most straightforward. The pipeline parallel approach can also likely power data parallel as well, by using the parallel() ops.


Why JedisCluster provides select()?

Hope to clarify the usage of select() on JedisCluster. According to http://redis.io/topics/cluster-spec which mentioned:
Redis Cluster does not support multiple databases like the stand alone version of Redis, there is just database 0, and SELECT is not allowed.
But why JedisCluster provides select()? Please see https://github.com/xetorthio/jedis/blob/master/src/main/java/redis/clients/jedis/JedisCluster.java#L1182.

It will be removed, please refer to https://github.com/xetorthio/jedis/pull/671 for more details.

Thanks @HeartSaVioR
One more follow-up question which might be more related to Redis: since Redis/Jedis cluster doesn't attempt to support multiple databases. Does it mean a set of Redis nodes in a cluster can only serve to manage one feature? It sounds a kind of bit waste to me because all the CPU/memory resources only work for one application.
How to make the maximum usage of a cluster on multiple features/applications?

Yeap, I post the question on https://github.com/antirez/redis/issues/2280 as well.

@airpingu
Hey, Redis only uses single thread for processing commands by each instance, which means you can't use multi-core powers whether you use multi-DBs in one instance or not.
Other way you can use your machine efficiently is, you can construct crossed masters-slaves which means one node serves one master and multiple slaves (or just one slave) that slaves are replicated to other nodes.

@airpingu Could I close this issue? Seems like I've answered root issue and follow-up question. :D

Sure! Appreciate your help!


pokerogue offline crash

last day, I play pokerogue offline version on android (samsung galaxy s23). But when I press the button to play offline mode today, game crashed immediately. I tried some ways - reinstalled pokerogue app, delete offline version and re-download, download futaba's build and so on. But, these are not working.
If you have any suggestion to solve these problem, please comment about it. thx.

This is something I've had a few people report, but I really don't know what the cause would be; any extra help from others would be appreciated.

I have had this Problem in the Beginning on my Pixel 8 with the pre Patch APK. Idk weather the upgrade or the permission Settings fixed it.
The way I did it is as follows:

Install the pre patch APK (dont open)
Install the Patch APK (dont open)
Go to the apps settings and manually set the storage permissions so that it has maximum permissions. Then open, update files and import you own ones.

PS: Since the last Android 14 Update on my Pixel, the storage-permissions are not accessible for this App in any way. Maybe this has somthing to do with it as well.
I hope this helps.

As I tested with two one ui 6.1 devices, looked like Galaxy using one ui 6.1(android 14) uses 8000 port for sort of internal handling of wifi connection, causing the crash as offline tries to use 8000 port that its already in use. As I changed the code the of the offline mode to use 8081 port instead it worked.

Where did you find the code you needed to edit?

https://github.com/Admiral-Billy/PokeRogue-App-Android/blob/959903439fc8f810938a5dfde229035f616a353a/app/src/main/java/com/example/pokerogueoffline/MainActivity.kt#L554-L585
The app sets the port to 8000 from here.

Thank you!

Worked like a charm by the way, thanks for the help.

@AngeredBarbeque how to install the code im new

@AngeredBarbeque  can u send me the updated apk i dont know hoe to change code in the apk


Getting "Uncaught, unspecified 'error' event."

I'm getting a node error when the host is unavailable.  It appears that restler doesn't quite know what to do with an 'error' event?
node.js:134
throw e; // process.nextTick error, or 'error' event on first tick
^
Error: Uncaught, unspecified 'error' event.
at EventEmitter.emit (events.js:47:15)
at EventEmitter._respond (/home/dbrogdon/stadium/node_modules/restler/lib/restler.js:127:12)

What version are you on?

0.2.3

Do you have a simple test case for this I can look at?

this is fixed in 2.x @ayoung i think you may close


no-remote-status being ignored?

I am trying to ignore the remote status section (as checking my current branch against origin/master isn't helpful for me). I can't seem to get the flag to work:
export PS1='[$NC]\u@\h:[$Red]\w[$NC]$(git-radar --bash --fetch --no-remote-status) $ '
Still gives me this output:
mike@compy:~/Development/project git:(𝘮 7 ⇄ 4 develop)
Thanks!

Hi, sorry I was so late to address this. #65 allows you to remove any feature from executing without the need for execution flags like this so I wanted to get it in and kill to birds.
To do this all you need to do is export GIT_RADAR_PROMPT with the parts you do want in. e.g.:
GIT_RADAR_PROMPT="%{branch}%{ :local}%{ :changes}"

In that example I've removed the colour codes and the git:( part as well. You can add in whatever bits you want. It supports colour codes as well.


Fixed shader hotrealoading issues

The last_hotload_time was initialized to weird values causing the hotreload to never go through the first ifstatement.
I fixed this by explicitly initializing the last_hotload_time when creating the pipeline.

merged fix


Update sbt-mdoc to 2.2.20

Updates org.scalameta:sbt-mdoc from 2.2.19 to 2.2.20.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.scalameta", artifactId = "sbt-mdoc" } ]

labels: sbt-plugin-update, semver-patch

Codecov Report

Merging #202 (4dbfb1b) into main (f90feaf) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #202   +/-   ##
=======================================
  Coverage   96.07%   96.07%
=======================================
  Files           7        7
  Lines         969      969
  Branches       96       96
=======================================
  Hits          931      931
  Misses         38       38

Impacted Files
Coverage Δ

core/shared/src/main/scala/cats/parse/Parser.scala
95.75% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f90feaf...4dbfb1b. Read the comment docs.


Release 4.1.1

[ ] Test tickets
[ ] Prepare release notes
[ ] Create release

@nasirhjafri any updates on it?:)

@auvipy @daadu can someone help?


Benchmark comparison with grpc-rs

Background: I've been working on an implementation of the remote execution API using tonic for both clients and the server. I've had some trouble getting good performance out of it, so I started looking into grpc-rs to compare speed.
Unfortunately converting my whole project to grpc-rs for a full comparison would be quite a bit of work, and anyway the tonic API+docs seem much friendlier (not to mention the convenience of using modern futures with async/await). So instead I put together a small benchmark of a very simple grpc service:
https://github.com/nicholasbishop/grpc-bench-rs
The benchmark does show grpc-rs performing quite a bit better, as well as supporting a larger number of client connections. (Following the discussion in https://github.com/hyperium/tonic/issues/209 I added a simple retry loop so that failed requests from the tonic client would be retried, but the loop doesn't seem to terminate in a reasonable time once the number of attempted connections gets too high.)
I'd be interested in any feedback on the benchmark itself -- have I made a mistake somewhere that could be hurting tonic's performance numbers?
I used the Linux perf tool to do some basic profiling. The items above 5% are:
  Children      Self  Command          Shared Object       Symbol
+   45.04%    35.69%  tokio-runtime-w  tonic_server        [.] <bytes::bytes_mut::BytesMut as bytes::buf::buf_mut::BufMut>::put_slice
+   32.74%    23.53%  tokio-runtime-w  libc-2.30.so        [.] __memmove_avx_unaligned_erms
+   24.64%    24.04%  tokio-runtime-w  tonic_server        [.] <tonic::codec::prost::ProstEncoder<T> as tokio_util::codec::encoder::Encoder>::encode

Thanks for running these benches! I've been meaning to write some but it doesn't make to much sense right now because we are waiting on a new release of prost that should allow us to avoid an extra copy which I believe the put_slice is showing that. https://github.com/hyperium/tonic/blob/master/tonic/src/codec/prost.rs#L54
Pretty much tonic wise there is a decent amount of work left to optimize things and the same goes for h2. Mostly, this is just the first attempt at a pure rust http/2 gRPC implementation. So I hope that once the stack starts to stabilize a bit we can put some good work into optimizing things.

Thanks for the info. I'll be sure to give the test another run once prost is updated.

Great! I'd also be very happy to work on this with you, expanding benchmarks and improving stuff is very much welcome here and I do plan on doing a bunch just kinda waiting for the last few things to flush out.

Hi, I also did some benchmark test, to compare the performance of tonic and grpc-go, the service code is very easy, just return an empty struct
   async fn read_file(&self, req: Request<PathOpts>) -> Result<Response<ReadFileResult>, Status> {
        Ok(Response::new(ReadFileResult{
            content: vec![],
            error_message: "".to_string()
        }))
    }

go codes:
func (rs *RemoteServer) ReadFile(_ context.Context, opts *api.PathOpts) (*api.ReadFileResult, error) {
	return &api.ReadFileResult{
		Content:      []byte(""),
		ErrorMessage: "",
	}, nil
}

the client send requests in 500 goroutines
        var wg sync.WaitGroup
	concurrentCount := 500
	for i := 0; i < concurrentCount; i++ {
		wg.Add(1)
		go func() {
			for i := 0; i < 500; i++ {
				if _, err := exec.Command("ls", "-al").CombinedOutput(); err != nil {
					fmt.Printf("failed to start remote command %v\n", err)
					atomic.AddInt32(&failedCount, 1)
				} else {
					atomic.AddInt32(&succeedCount, 1)
				}
			}
			wg.Done()
		}()
	}
	wg.Wait()

When sending the tonic server, it costs about 8-9 seconds to finish, while sending to go server, only 4-5 seconds, I thought it is because of the setting of tokio runtime, so I adjust the parameter in the attribute #[tokio::main(core_threads = 16, max_threads = 32)], but the change of core_threads from 8 to 16 do not change the result.
By the way,  I also compared the performance in this  single connection scenario and the tonic has better performance, the C++ implements of grpc which grpc-rs utilize has a bug which makes all requests on one connection handled in a single thread, sequentially.
But I can not explain the performance degradation of tonic compared with grpc-go, maybe some codes in tonic has to be modified. but I have to do more investigation to find out where is the problem.

@abel-von can you please elaborate on the grpc-rs bug that makes all requests be handled in a single thread?

Is there a new benchmark comparison with grpc-rs? @LucioFranco

https://github.com/LesnyRumcajs/grpc_bench/wiki/2022-01-11-bench-results @LucioFranco

It's better to adding continuous performance benchmarking like grpc as following.
https://www.grpc.io/docs/guides/benchmarking/

I believe the benchmark test has some unexpected changes, because the chart is weird.


Why not an API for openIssueReporter?

Testing https://github.com/microsoft/vscode/issues/203094
What is the benefit of doing the Issue Reporter via a command, compared to introducing an API?
Using the current command looks complex, and does not offer intelli-sense to extension authors.
fyi @TylerLeonhardt @jrieken

I agree with this. It is difficult to make sense of what should be passed in.

We had a lot of discussion about the value of the API vs potentially just making a better version of the command. Wanted to get some eyes/implementations of this to see how the command would do vs. the API so we could get some more discussion on it for sure!
appreciate the feedback as always, and will keep iterating on this

I suggest integrating Github Issue Notebooks, without needing to install an extension.

closing, tracking the rest in https://github.com/microsoft/vscode/issues/196863


fix(git-node): update README parser to account the recent TSC changes

Refs: https://github.com/nodejs/node/pull/47126
Refs: https://github.com/nodejs/TSC/pull/1350
BREAKING CHANGE: Old README format is no longer supported

Refs: nodejs/node#46985
BREAKING CHANGE: Old README format is no longer supported

Is that going to break using node-core-utils to land PR's in LTS lines? cc @nodejs/releasers

Is that going to break using node-core-utils to land PR's in LTS lines?

I was about to comment regarding that. We need to either backport the README recent updates first, or backporters would need to use the main version of the README (which might be the default?).

Not sure if it will help here, but here's what I did to make find-inactive-tsc.mjs backwards compatible: https://github.com/nodejs/node/blob/a4808b6c3b111608e0b9945c7f62534b285b0c96/tools/find-inactive-tsc.mjs#L56-L86

I think we should land this, the README change have landed on all supported release lines. wdyt @nodejs/node-core-utils?


Entering year manually changes the day

Below I enter full date into the 'When did you play/watch xyz?` window.
When entering year, the day is moved to 3 days earlier:

It can (from my experience) be reproduced easily with any combination of dates.
I'm on the latest docker release.

Fixed in 0.1.0-beta.42

Can confirm, thanks!


Issue comparing a word against a the same word plus a blank and another letter

Hello, I found an issue comparing a word against a the same word plus a blank and another letter.
Eg:
"Iphone" compared with "Iphone X" gives me a match of 1, but the texts are not equal. It should be close to 1 but not 1.
I'm using version 2.0.0
findBestMatch('Iphone', ['Iphone 8', 'Iphone 10', 'Iphone X', 'Iphone XS'])

Hello, I found an issue comparing a word against a the same word plus a blank and another letter.
Eg:
"Iphone" compared with "Iphone X" gives me a match of 1, but the texts are not equal. It should be close to 1 but not 1.
I'm using version 2.0.0
findBestMatch('Iphone', ['Iphone 8', 'Iphone 10', 'Iphone X', 'Iphone XS'])

It's a major bug.

It's not matching to expected string.

Fixed in 3.0.0