SIGPIPE when OpenSSL is used with a socket

Please provide a way for user apps to specify socket send and receive flags. On UNIX systems when the remote side terminates the socket, the user app will crash with SIGPIPE. On Linux, apps using sockets can specify the MSG_NO_SIGNAL flag in the send function, to avoid receiving SIGPIPE, however when OpenSSL is used, there is no way to any flags when sending and receiving from sockets. On macOS and BSD, the app can specify SO_NOSIGPIPE in setsockopt to overcome this issue.
OpenSSL 1.1.1k
Ubuntu 20.04
https://github.com/openssl/openssl/issues/8075#issuecomment-902145012

We just caught this issue in our code. It's pretty serious issue for us as we cannot easily suppress SIGPIPE as we need to have visibility when it happens due to extensive use of popen() in our logic.
For plain TCP logic we use send() instead of write() and we set last flag argument of send() to MSG_NOSIGNAL and it works just fine.
Would be great to have some way to pass flags field to SSL_write().
Thank you!

 See the following man page for creating a custom BIO:
https://www.openssl.org/docs/man3.1/man3/BIO_meth_new.html
Using the custom BIO you set that on your SSL object using SSL_set0_wbio() (or similar function). Then, whenever OpenSSL wants to write to the socket, it will use your custom BIO code instead.

I had the same issue. My solution was to create a custom BIO method and tell OpenSSL to send/receive that way (BIO_meth_new, BIO_meth_set_write_ex, BIO_meth_set_read_ex, BIO_meth_set_ctrl (just return 0 for everything, except BIO_CTRL_FLUSH which must return 1), BIO_new, SSL_set_bio, and BIO_set_data/BIO_get_data to tell it which fd to use).
But that has the obvious side effects on kTLS support, and probably some other features.

@mattcaswell doesn't it make sense to allow overwriting readsocket/writesocket in standard BIOs?

Yes - that might make sense, with some ctrl enabling you to set what flags to use. But that really is a new feature - my suggestion was a workaround for what you can do right now. I've added the "help wanted" flag if someone wants to implement this.

@mattcaswell wouldn't it be possible to use BIO_new_socket() and SSL_set0_[rw]rbio() here?  Set the socket option before calling the former.

@mattcaswell wouldn't it be possible to use BIO_new_socket() and SSL_set0_[rw]rbio() here? Set the socket option before calling the former.

Unfortunately, MSG_NOSIGNAL isn't a socket option. It's a flag on the send call - so it has to be provided every time you try to write something.

Huh?  The original issue includes this:

On macOS and BSD, the app can specify SO_NOSIGPIPE in setsockopt to overcome this issue.

MSG_NOSIGNAL is not SO_NOSIGPIPE. There is no equivalent of SO_NOSIGPIPE on Linux.

I'm also fighting with the SIGPIPE problem for a library that must not alter the users process.
SO_NOSIGPIPE does not help on macOS and BSD to workaround the SIGPIPE problem with OpenSSL. It has no effect on system call write() that is used by OpenSSL. You can find a note at SIGPIPE and how to ignore it

This is a bit niche as it only exists on FreeBSD and OS X. Use setsockopt(2) to set this option on a socket and all subsequent send(2) calls will behave as if MSG_NOSIGNAL was set. [..] This seems to be of limited utility as calling write(2) on the socket will still generate SIGPIPE.

With man send you will find:

The only difference between send() and write(2) is the presence of flags. With a zero flags argument, send() is equivalent to write(2).

So my suggestion is to use send() instead of write() in OpenSSL and and make setting the flags available to the user. This also help on Linux platforms with setting MSG_NOSIGNAL that isn't possible with write().

@pavel-odintsov  I just see that you are already partially using send() https://github.com/openssl/openssl/issues/16399#issuecomment-1785649956. This could easily(?) be expanded?

For a workaround now I have made a solution with general suppressing the SIGPIPE signal temporary on the current thread. For those who are interested may have a look at C++ & OpenSSL: SIGPIPE when writing in closed pipe.


Merging datasets with different amount of features. Is it possible?

I have samples with different number of features to merge.
ds_HC1119D

DataStore has 15985 (15985) cells with 1 assays: RNA
   Cell metadata:
            'I', 'ids', 'names', 'RNA_nCounts', 'RNA_nFeatures',
            'RNA_percentMito', 'RNA_percentRibo'
   RNA assay has 20310 (36601) features and following metadata:
            'I', 'ids', 'names', 'dropOuts', 'nCells',

ds_SLE1D

DataStore has 19539 (19539) cells with 1 assays: RNA
   Cell metadata:
            'I', 'ids', 'names', 'RNA_nCounts', 'RNA_nFeatures',
            'RNA_percentMito', 'RNA_percentRibo'
   RNA assay has 18181 (33538) features and following metadata:
            'I', 'ids', 'names', 'dropOuts', 'nCells',

Merging two samples
scarf.ZarrMerge(
    # Path where merged Zarr files will be saved
    zarr_path='scarf_datasets/GSE179633_merged.zarr',

    # assays to be merged
    assays=[ds_HC1119D , ds_SLE1D],

    # these names will be preprended to the cell ids with '__' delimiter
    names=['HC1119D', 'SLE1D'],

    # Name of the merged assay. `overwrite` will remove an existing Zarr file.
    merge_assay_name='RNA',
    overwrite=True
).dump()

That command produces an error.
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[17], line 1
----> 1 scarf.ZarrMerge(
      2     # Path where merged Zarr files will be saved
      3     zarr_path='scarf_datasets[/GSE179633_merged.zarr](http://localhost:8888/GSE179633_merged.zarr)',
      4
      5     # assays to be merged
      6     assays=[ds_HC1119D , ds_SLE1D],
      7
      8     # these names will be preprended to the cell ids with '__' delimiter
      9     names=['HC1119D', 'SLE1D'],
     10
     11     # Name of the merged assay. `overwrite` will remove an existing Zarr file.
     12     merge_assay_name='RNA',
     13     overwrite=True
     14 ).dump()

File [~/Documents/Scurf/venv/lib/python3.11/site-packages/scarf/writers.py:1164](http://localhost:8888/lab/tree/venv/lib/python3.11/site-packages/scarf/writers.py#line=1163), in ZarrMerge.__init__(self, zarr_path, assays, names, merge_assay_name, in_workspaces, out_workspace, chunk_size, dtype, overwrite, prepend_text, reset_cell_filter)
   1160 self.mergedCells: pd.DataFrame = self._merge_cell_table(
   1161     reset_cell_filter, prepend_text
   1162 )
   1163 self.nCells: int = self.mergedCells.shape[0]
-> 1164 self.featCollection: List[Dict[str, str]] = self._get_feat_ids(assays)
   1165 self.mergedFeats = self._merge_order_feats()
   1166 self.nFeats = self.mergedFeats.shape[0]

File [~/Documents/Scurf/venv/lib/python3.11/site-packages/scarf/writers.py:1239](http://localhost:8888/lab/tree/venv/lib/python3.11/site-packages/scarf/writers.py#line=1238), in ZarrMerge._get_feat_ids(assays)
   1236 ret_val = []
   1237 for i in assays:
   1238     ret_val.append(
-> 1239         i.feats.to_pandas_dataframe(["names", "ids"])
   1240         .set_index("ids")["names"]
   1241         .to_dict()
   1242     )
   1243 return ret_val

AttributeError: 'DataStore' object has no attribute 'feats'

It is came out that assays=[ds_HC1119D , ds_SLE1D] should be specified correctly  as assays=[ds_HC1119D.RNA , ds_SLE1D.RNA]. My mistake.


Added loading state to make it look and feel nicer and also prevent m…

…ultiple simulation requests in a row.

There are some weird diffs in here... all I did in simulateCombat is change some variable names and wrap the call in a timeout.


How to set new line ?

setNotification({message: 'Line 1 <br> Line 2'})

Hi @gentritabazi01,
That was not possible. I did a small change and now it should work the way you want. Please have a look.
Thanks for pointing it out.

@dafcoe Thank you so much, it's work.


fix toast message on widget cut

Description

Common function to create a widget copy
Trigger separate toast/events for cut

Fixes # (issue)

fixes #983

Type of change

Bug fix

How Has This Been Tested?

both copy and cut shortcuts manually

Checklist:

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes

@hetunandu have updated at the root of the copy/cut sagas


Build Error

root@server:/certs/cert-chain-resolver# glide install
[WARN]  The name listed in the config file (github.com/zakjan/cert-chain-resolver) does not match the current location (.)
[INFO]  Downloading dependencies. Please wait...
[INFO]  --> Fetching github.com/urfave/cli
[INFO]  --> Fetching github.com/stretchr/testify
[INFO]  --> Fetching github.com/fullsailor/pkcs7
[INFO]  Setting references.
[INFO]  --> Setting version for github.com/fullsailor/pkcs7 to 2585af45975b11f1d7502bb6c01556c29efb54ce.
[INFO]  --> Setting version for github.com/stretchr/testify to f390dcf405f7b83c997eac1b06768bb9f44dec18.
[INFO]  --> Setting version for github.com/urfave/cli to 01a33823596e7cffafa9e037acdd2441dc0eb03f.
[INFO]  Exporting resolved dependencies...
[INFO]  --> Exporting github.com/fullsailor/pkcs7
[INFO]  --> Exporting github.com/stretchr/testify
[INFO]  --> Exporting github.com/urfave/cli
[INFO]  Replacing existing vendor dependencies
root@server:/certs/cert-chain-resolver# go build
main.go:5:2: cannot find package "github.com/urfave/cli" in any of:
/usr/local/go/src/github.com/urfave/cli (from $GOROOT)
/root/work/src/github.com/urfave/cli (from $GOPATH)
main.go:6:2: cannot find package "github.com/zakjan/cert-chain-resolver/certUtil" in any of:
/usr/local/go/src/github.com/zakjan/cert-chain-resolver/certUtil (from $GOROOT)
/root/work/src/github.com/zakjan/cert-chain-resolver/certUtil (from $GOPATH)
please help..

Fixed in #24


ghcDocumentUpdate

@ashwinikatur Please sign the Contributor License Agreement!
Click here to manually synchronize the status of this Pull Request.
See the FAQ for frequently asked questions.

@ashwinikatur Thank you for signing the Contributor License Agreement!

Fixes gh-341.

Ah @OlgaMaciaszek  found some issues! Apparently she's got better 👀 than I do 😬

Congratulations @ashwinikatur on your first contribution to Spring Cloud Contract :)


Sync fileio

This is only the sync fileio. I plan to implement async io at some point but this can get us started for the time being. I will note I've not really tested this code super extensively.
This PR is just to bring in sync io I'll keep the branch up though for async.

Also a note to be made. This implementation kinda only handles binary files. I can add specializations for text files but in all honesty I don't think text files are going to be AS important till much later.

Also added a bunch of useful stuff we will need later. Not really related to this PR or branch, but just gonna bunch em' together.

async-fileio is still a WIP branch


upgrade(package): Bump sudo-prompt 6.1.0 -> 8.0.0

This updates sudo-prompt from v6.1.0 to v8.0.0
Changes:

v8.0.0:

Breaking: Windows: Set code page of command batch script to UTF-8

v7.1.1:

readme: explicitly mention that no child process is returned

v7.1.0:

Detect when PowerShell fails to launch command
Escape ampersand and other characters special to PowerShell

v7.0.0:

Add status code to error on Windows and macOS

v6.2.0:

Rename OS X to macOS

Change-Type: patch

Maybe worth testing this in all OSes to ensure nothing is broken? Mainly on Windows, given that most of the changes described in sudo-prompt's CHANGELOG affect that platform.

Yeah, we also don't need this for the release, whereas it'd be good if we can ship with it


Update AboutUs and add portfolio

Pull Request Test Coverage Report for Build 54

0 of 0 (NaN%) changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 91.523%

Totals

Change from base Build 51:
0.0%

Covered Lines:
1544

Relevant Lines:
1687

💛  - Coveralls


Move a few components to es6 with mixin decorator

This PR moves app-bar, app-canvas, and avatar to es6 style classes with a stage-1 decorator to handle the mixin. Most of the code changes were done with react-codemod with some minor changes to account for things in es7 syntax the tool does not account for (like static properties on the class).
Each component took about 1 minute to migrate from es5 to es6/es7.
No behavioral changes were introduced, the demo still runs as expected, and all tests still pass.
This PR is in reference to #458.

I think that we should wait to release v0.14 before migrating to ES6 class.

That's fair; this was just a swing to see how much the codemod tool could do with or without the mixins, and try and move a few components over. Realistically, I believe the whole library could be moved in a couple of hours of fairly idle work, just from trying this.
This also kind of proves that nothing breaks; everything behaves exactly the same as before.

:scream: I think this is worth it :+1: :+1:

I'd be more than happy to move some more components over and keep adding to this PR, or I could deal with it however you think is best (I can totally understand not wanting one very bloated PR, so I could split it up or something).

 I completely agree, I don't think a syntax change on the entire library should be put into a release that's in an RC phase.

@BobertForever I have another suggestion if @oliviertassinari agrees. I think it's best to add an es6-classes branch, do all the migration, planning, etc there and when all is ready merge with master.

es6-classes branch

Sounds ok to me. @shaurya947?

Cool! I can re-direct this PR there when you make it.

@BobertForever Please redirect this to the es6-classes branch. thanks :+1:


Remove github.com/docker/distribution/uuid dependency

This PR replaces the uuid generator we are currently using with a custom ciao generator, allowing us to remove one dependency.  The old generator basically just generates random 16 byte sequences so we're only reinventing a very small wheel here.
This PR

Adds a new uuid package to ciao, github.com/01org/ciao/ssntp/uuid complete with its own unit tests
Updates the travis build to run those unit tests
Replaces the old uuid generator import statements with imports for the new generator
unvendors github.com/docker/distribution/uuid

The changes have been tested on the single machine VM setup.
Fixes #222

Coverage increased (+0.05%) to 62.731% when pulling fb75064e059759341dedcfbd524b5b5b5c52a931 on markdryan:master into 21de96b763d5f0fb8354e15d02765559147a916a on 01org:master.

The SSNTP changes look good to me.
The uuid package also looks fine to me as well. I was wondering why @markdryan decided to use the encoding package explictly for BE translation, and the reason is otherwise it would pretty much be a docker uuid package copy.

@markdryan could you add a negative test case for Parse()?
Otherwise lgtm!

@tpepper Sure.  Actually, I forgot to document the existing tests as well.  I'll do this and push a new patch.

Coverage increased (+0.1%) to 62.786% when pulling e62dc9be126133b914be89e07e9d60c3880a3562 on markdryan:master into 2aeade150fc8e3c69b5910a1bc52e773060a0811 on 01org:master.


RKE2 and K3S - KDM Update for K8s December patch release for Rancher 2.7

KDM Updates

New Kubernetes Version
Min Rancher Version
Max Rancher Version
Default Version

v1.22.17+rke2r1
v2.6.3-alpha1
v2.6.99
x

v1.23.15+rke2r1
v2.6.4-alpha1
v2.7.99

v1.24.9+rke2r1
v2.6.7-alpha1
v2.7.99

v1.22.17+k3s1
v2.6.3-alpha1
v2.6.99
x

v1.23.15+k3s1
v2.6.4-alpha1
v2.7.99

v1.24.9+k3s1
v2.6.7-alpha1
v2.7.99

Chart Changelog

Chart Name
New Version
Affected Kubernetes Version(s)

rke2-cilium
1.12.402
v1.23.15+rke2r1, v1.24.9+rke2r1

rke2-canal
v3.24.5-build2022120101
v1.23.15+rke2r1, v1.24.9+rke2r1

rke2-calico
v3.24.501
v1.23.15+rke2r1, v1.24.9+rke2r1

rke2-calico-crd
v3.24.501
v1.23.15+rke2r1, v1.24.9+rke2r1

rke2-ingress-nginx
4.1.007
v1.23.15+rke2r1, v1.24.9+rke2r1

rke2-metrics-server
2.11.100-build2022101107
v1.23.15+rke2r1, v1.24.9+rke2r1

rke2-multus
v3.9-build2022102805
v1.23.15+rke2r1, v1.24.9+rke2r1

Argument Changelog

Argument Name
Affected Kubernetes Version(s)

Additional Notes
N/A

Validated on rancher 2.7.0, versions 1.23 and 1.24 are available.

This has passed post-release validations now as well and KDM PRs have been merged.

Reopening for tracking replacement of original December releases with r2s coming out soon.

This has been re-validated with r2 versions


TB3 slowly drifting in Gazebo

Using ROS2 foxy and turtlebot_simulations v. 2.2.2.
I start the simulation without any controlling node:
ros2 launch turtlebot3_gazebo turtlebot3_house.launch.py

Robot slowly drifts as follows. The sudden movement at the beginning is due to simulation restart:

I would expect that the robot would stop when the motors are not driven. I also tried to stop the robot using teleop_keyboard after which the robot still drifts.
I tinkered around with wheel friction parameters which did not help, e.g., here:
https://github.com/ROBOTIS-GIT/turtlebot3_simulations/blob/7603541b97ee824ca41e45c8d2a026eaa36b17e7/turtlebot3_gazebo/models/turtlebot3_waffle_pi/model.sdf#L186-L220
Can someone help me with this?

Hi @goekce
Looks like the inertial values need to be revised.
I'll update these values soon.
Thanks for reporting the issue!

I recently noticed that the robot flies if I place TB3 Waffle Pi on the empty world in Gazebo. I noticed that the default parameters for the ode solver do not suffice to appropriately simulate TB3 physics. I have to change ode solver iters parameter to 150 to avoid flying as Robotis does in its worlds, e.g.,:
https://github.com/ROBOTIS-GIT/turtlebot3_simulations/blob/77151384a09c25271679d9baec5c5cf6587bd9bd/turtlebot3_gazebo/worlds/empty_worlds/waffle_pi.model#L30-L32
Here is a minimal world with minimal parameters:
<?xml version='1.0' ?>
<sdf version='1.7'>
    <world name="world">
        <physics type="ode">
            <ode>
                <solver>
                    <iters>150</iters>
                </solver>
            </ode>
        </physics>
        <include><name>sun</name><uri>model://sun</uri></include>
        <include><name>ground</name><uri>model://ground_plane</uri></include>
        <include><name>tb3</name><uri>model://turtlebot3_waffle_pi</uri><pose>0.5 0.5 0 0 0 1.5707963267948966</pose></include>
    </world>
</sdf>

If you remove iters (then it is set to 50) then the robot flies. Maybe SDFormat default parameters were changed recently. I use Gazebo 11.4.

Hi,
The iters tag governs the simulation frequency and higher value uses more resource to create more accurate simulation result.
The drifting is actually caused by incorrect inertia value of left and right wheels, and they can be fixed by modifying /turtlebot3_simulations/turtlebot3_gazebo/models/turtlebot3_waffle_pi/model.sdf file as below.
In line 168, where wheel_left_link is defined,
    <link name="wheel_left_link">

      <inertial>
        <pose>0.0 0.144 0.023 -1.57 0 0</pose>
        <inertia>
          <ixx>1.8158194e-03</ixx>
          <ixy>-9.3392e-12</ixy>
          <ixz>1.04909e-11</ixz>
          <iyy>3.2922126e-03</iyy>
          <iyz>5.75694e-11</iyz>
          <izz>1.8158194e-03</izz>
        </inertia>
        <mass>2.8498940e-02</mass>
      </inertial>

In line 226, where wheel_right_link is defined,
    <link name="wheel_right_link">

      <inertial>
        <pose>0.0 -0.144 0.023 -1.57 0 0</pose>
        <inertia>
          <ixx>1.8158194e-03</ixx>
          <ixy>-9.3392e-12</ixy>
          <ixz>1.04909e-11</ixz>
          <iyy>3.2922126e-03</iyy>
          <iyz>5.75694e-11</iyz>
          <izz>1.8158194e-03</izz>
        </inertia>
        <mass>2.8498940e-02</mass>
      </inertial>

How did you calculate these values Will?
I'll leave the issue here until it is fixed in the codebase.
:heart_decoration: for the quick fix!

@goekce
The inertial value is calculated by the CAD tool based on the 3D modeling file.
I ran a quick test on Gazebo after modifying these values, and the issue was resolved, but I'll check again.
This fix will be updated on foxy-devel branch and released in this week.
Thanks!

I installed the foxy-devel branches of turtlebot3 and turtlebot3_simulations branches from scratch, and now the drifting is gone :partying_face: Somehow it was not enough to only patch the TB3 Waffle Pi model.
I also do not need <iters>150</iters> anymore. The default setting of sdformat (50) seems to be sufficient. The flying robot was probably also caused by the wrong inertia setting.

@goekce The inertial value is calculated by the CAD tool based on the 3D modeling file. I ran a quick test on Gazebo after modifying these values, and the issue was resolved, but I'll check again. This fix will be updated on foxy-devel branch and released in this week. Thanks!

What CAD tool did you use? Did it allow you to set a mass or did it give it to you based on a density of 1? @ROBOTIS-Will


px4io depend on NuttX submodules

This prevents a race condition in the builds (mainly CI) where multiple components might be touching the submodules simultaneously.

Edit: check that compile failure though. Might be unrelated, isn't anything I haven't seen before.

Thanks, looks like another angle of the same problem. In CI we're force updating all submodules. In a build like px4_fmu-v2 which contains another NuttX build internally (px4_io-v2) it's force updating the submodule contents right after it's already been updated and breaking the hard links.
Should be fixed with https://github.com/PX4/Firmware/pull/11379/commits/a430dac8bf10108b14757e5882f8cdf98684c4ba


refactor memory

Type of Issue
Select the type of issue:

[ ] Bug report (to report a bug)
[ ] Feature request (to request an additional feature)
[x] Tracker (I am just using this as a tracker)
[x] Refactor request
[ ] Documentation Ask

Description
There are a large number of methods in the main memory that should be moved to be class methods of the memory nodes.  The behavior of the methods should not change, just their location in the codebase.  If the memory node files get too big, we can split these up.  When this is done, none of the memory-node specific methods (e.g. ~line 370 to line 940 in sql_memory.py) should reside in sql_memory; ditto for mc_memory.py etc.   In addition, many of these methods are overly specific, and should be removed altogether.  For these removed method, mentions in the code should use other more general methods, favoring basic_search as much as possible.  If there is a search necessary amongst the deleted methods that cannot be done using basic search, make an issue to add that functionality.
We should register the memory nodes classes in the main memory, and continue to allow using the methods by agent_memory.nodes[node_key].method(agent_memory, *args, **args)
Move to class methods of memory nodes:

[ ] https://github.com/facebookresearch/fairo/blob/8fcc711866af63ee5dd1839a0d356f2208ad54d5/droidlet/memory/sql_memory.py#L412
[ ] https://github.com/facebookresearch/fairo/blob/8fcc711866af63ee5dd1839a0d356f2208ad54d5/droidlet/memory/sql_memory.py#L430
[ ] https://github.com/facebookresearch/fairo/blob/8fcc711866af63ee5dd1839a0d356f2208ad54d5/droidlet/memory/sql_memory.py#L453
[ ] https://github.com/facebookresearch/fairo/blob/8fcc711866af63ee5dd1839a0d356f2208ad54d5/droidlet/memory/sql_memory.py#L472
[ ] https://github.com/facebookresearch/fairo/blob/8fcc711866af63ee5dd1839a0d356f2208ad54d5/droidlet/memory/sql_memory.py#L500
[ ] https://github.com/facebookresearch/fairo/blob/8fcc711866af63ee5dd1839a0d356f2208ad54d5/droidlet/memory/sql_memory.py#L590
[ ] https://github.com/facebookresearch/fairo/blob/8fcc711866af63ee5dd1839a0d356f2208ad54d5/droidlet/memory/sql_memory.py#L600
[ ] https://github.com/facebookresearch/fairo/blob/8fcc711866af63ee5dd1839a0d356f2208ad54d5/droidlet/memory/sql_memory.py#L609
[ ]  https://github.com/facebookresearch/fairo/blob/8fcc711866af63ee5dd1839a0d356f2208ad54d5/droidlet/memory/sql_memory.py#L656

In craftassist:

[ ] https://github.com/facebookresearch/fairo/blob/8fcc711866af63ee5dd1839a0d356f2208ad54d5/droidlet/memory/craftassist/mc_memory.py#L420
[ ] https://github.com/facebookresearch/fairo/blob/8fcc711866af63ee5dd1839a0d356f2208ad54d5/droidlet/memory/craftassist/mc_memory.py#L431
[ ] https://github.com/facebookresearch/fairo/blob/8fcc711866af63ee5dd1839a0d356f2208ad54d5/droidlet/memory/craftassist/mc_memory.py#L457
[ ]  https://github.com/facebookresearch/fairo/blob/8fcc711866af63ee5dd1839a0d356f2208ad54d5/droidlet/memory/craftassist/mc_memory.py#L478
[ ] https://github.com/facebookresearch/fairo/blob/8fcc711866af63ee5dd1839a0d356f2208ad54d5/droidlet/memory/craftassist/mc_memory.py#L643
[ ] https://github.com/facebookresearch/fairo/blob/8fcc711866af63ee5dd1839a0d356f2208ad54d5/droidlet/memory/craftassist/mc_memory.py#L665
[ ] https://github.com/facebookresearch/fairo/blob/8fcc711866af63ee5dd1839a0d356f2208ad54d5/droidlet/memory/craftassist/mc_memory.py#L769

to delete:

[ ] https://github.com/facebookresearch/fairo/blob/8fcc711866af63ee5dd1839a0d356f2208ad54d5/droidlet/memory/sql_memory.py#L370
[ ] https://github.com/facebookresearch/fairo/blob/8fcc711866af63ee5dd1839a0d356f2208ad54d5/droidlet/memory/sql_memory.py#L573
[ ] https://github.com/facebookresearch/fairo/blob/8fcc711866af63ee5dd1839a0d356f2208ad54d5/droidlet/memory/sql_memory.py#L582
[ ] https://github.com/facebookresearch/fairo/blob/8fcc711866af63ee5dd1839a0d356f2208ad54d5/droidlet/memory/sql_memory.py#L590
[ ] https://github.com/facebookresearch/fairo/blob/8fcc711866af63ee5dd1839a0d356f2208ad54d5/droidlet/memory/sql_memory.py#L635
[ ] https://github.com/facebookresearch/fairo/blob/8fcc711866af63ee5dd1839a0d356f2208ad54d5/droidlet/memory/sql_memory.py#L643
[ ] https://github.com/facebookresearch/fairo/blob/8fcc711866af63ee5dd1839a0d356f2208ad54d5/droidlet/memory/sql_memory.py#L669
[ ] https://github.com/facebookresearch/fairo/blob/8fcc711866af63ee5dd1839a0d356f2208ad54d5/droidlet/memory/sql_memory.py#L684
[ ] https://github.com/facebookresearch/fairo/blob/8fcc711866af63ee5dd1839a0d356f2208ad54d5/droidlet/memory/sql_memory.py#L694
[ ] https://github.com/facebookresearch/fairo/blob/8fcc711866af63ee5dd1839a0d356f2208ad54d5/droidlet/memory/sql_memory.py#L706
[ ] https://github.com/facebookresearch/fairo/blob/8fcc711866af63ee5dd1839a0d356f2208ad54d5/droidlet/memory/sql_memory.py#L714
[ ] https://github.com/facebookresearch/fairo/blob/8fcc711866af63ee5dd1839a0d356f2208ad54d5/droidlet/memory/sql_memory.py#L726
[ ] https://github.com/facebookresearch/fairo/blob/8fcc711866af63ee5dd1839a0d356f2208ad54d5/droidlet/memory/sql_memory.py#L734

In craftassist:

[ ] https://github.com/facebookresearch/fairo/blob/8fcc711866af63ee5dd1839a0d356f2208ad54d5/droidlet/memory/craftassist/mc_memory.py#L402
[ ] https://github.com/facebookresearch/fairo/blob/8fcc711866af63ee5dd1839a0d356f2208ad54d5/droidlet/memory/craftassist/mc_memory.py#L526
[ ] https://github.com/facebookresearch/fairo/blob/8fcc711866af63ee5dd1839a0d356f2208ad54d5/droidlet/memory/craftassist/mc_memory.py#L539
[ ] https://github.com/facebookresearch/fairo/blob/8fcc711866af63ee5dd1839a0d356f2208ad54d5/droidlet/memory/craftassist/mc_memory.py#L556
[ ] https://github.com/facebookresearch/fairo/blob/8fcc711866af63ee5dd1839a0d356f2208ad54d5/droidlet/memory/craftassist/mc_memory.py#L560
[ ] https://github.com/facebookresearch/fairo/blob/8fcc711866af63ee5dd1839a0d356f2208ad54d5/droidlet/memory/craftassist/mc_memory.py#L568
[ ] https://github.com/facebookresearch/fairo/blob/8fcc711866af63ee5dd1839a0d356f2208ad54d5/droidlet/memory/craftassist/mc_memory.py#L571
[ ] https://github.com/facebookresearch/fairo/blob/8fcc711866af63ee5dd1839a0d356f2208ad54d5/droidlet/memory/craftassist/mc_memory.py#L579
[ ] https://github.com/facebookresearch/fairo/blob/8fcc711866af63ee5dd1839a0d356f2208ad54d5/droidlet/memory/craftassist/mc_memory.py#L592
[ ] https://github.com/facebookresearch/fairo/blob/8fcc711866af63ee5dd1839a0d356f2208ad54d5/droidlet/memory/craftassist/mc_memory.py#L596
[ ] https://github.com/facebookresearch/fairo/blob/8fcc711866af63ee5dd1839a0d356f2208ad54d5/droidlet/memory/craftassist/mc_memory.py#L627
[ ] https://github.com/facebookresearch/fairo/blob/8fcc711866af63ee5dd1839a0d356f2208ad54d5/droidlet/memory/craftassist/mc_memory.py#L830

skipped things
For now have left out Tasks; will do those separately as refactor is less simple than just move/delete.   For each move/delete, one needs to make sure all the mentions of that method are properly replaced (and in the case of deletes, this may require a non-mechanical change).  Also left out item stacks in craftassist, these should be rewritten

Hi @kavyasrinet and @aszlam. Thanks for adding info around this issue, I have a few questions as I am trying to understand the context.

Is there an architectural overview on how memory is used? I wish to understand connections across components.
What do we mean by “We should register the memory nodes classes in the main memory”? Is there a central “factory-like” construct which requires explicit registration beyond including methods in class body? A pointer to such registration syntax would be helpful.
My understanding is that methods which are to be deleted should be replaced with a call to basic_search with whatever SQL query the body contained. Could you confirm?

What do we mean by “We should register the memory nodes classes in the main memory”? Is there a central “factory-like” construct which requires explicit registration beyond including methods in class body? A pointer to such registration syntax would be helpful.

"registering the node classes" is already done here https://github.com/facebookresearch/fairo/blob/main/droidlet/memory/sql_memory.py#L116,
but we don't tend to use methods from the nodes in the rest of the codebase

My understanding is that methods which are to be deleted should be replaced with a call to basic_search with whatever SQL query the body contained. Could you confirm?

yes; but the syntax of "basic_search" is not quite SQL.  if there is some query in these that cannot be done with basic_search, we should consider expanding the scope of basic_search

I made the required changes for get_player_by_eid and verified that droidlet/memory/tests/test_low_level_memory.py ran OK.
Could you please take a look at: 1f93cee4fdc5ce5f6f058e7997726b2d5102a68c, 74e53f2ff0b79f0841d475476de6686712124bfe
If this makes sense and aligns with the requirements I can go ahead and make the change for rest of the methods similarly. Thanks!

I made the required changes for get_player_by_eid and verified that droidlet/memory/tests/test_low_level_memory.py ran OK.
Could you please take a look at: 1f93cee, 74e53f2
If this makes sense and aligns with the requirements I can go ahead and make the change for rest of the methods similarly. Thanks!

the change is fine.  a few minor comments:
1: instead of self.memory.nodes[PlayerNode.NODE_TYPE] can you do self.memory.nodes["Player"] (so you don't have to import the node
2: go ahead and make a PR, even if its a draft- its just easier to review/see what is going on
3: test is failing :)   you can run the tests locally to check if you don't want to wait for CI.

Style note: Both using a string literal and referencing via PlayerNode.NODE_TYPE are OK with me in Python.  My personal preference is to use PlayerNode.NODE_TYPE because:

It tells the reader where to look for other node type strings.
*   For this, you might want a shared ‘NodeTypes’ list somewhere that references all the NODE_TYPE variables in individual Node classes.
If you ever needed to change the string the code using it doesn’t break.
*   This is more of an issue in compiled languages, where using the string literal would be highly discouraged, since it allows dependent objects to use the updated string without recompilation.

From: aszlam @.>
Date: Friday, February 18, 2022 at 11:12 PM
To: facebookresearch/fairo @.>
Cc: Subscribed @.***>
Subject: Re: [facebookresearch/fairo] refactor memory (Issue #982)
I made the required changes for get_player_by_eid and verified that droidlet/memory/tests/test_low_level_memory.py ran OK.
Could you please take a look at: 1f93ceehttps://github.com/facebookresearch/fairo/commit/1f93cee4fdc5ce5f6f058e7997726b2d5102a68c, 74e53f2https://github.com/facebookresearch/fairo/commit/74e53f2ff0b79f0841d475476de6686712124bfe
If this makes sense and aligns with the requirements I can go ahead and make the change for rest of the methods similarly. Thanks!
the change is fine. a few minor comments:
1: instead of self.memory.nodes[PlayerNode.NODE_TYPE] can you do self.memory.nodes["Player"] (so you don't have to import the node
2: go ahead and make a PR, even if its a draft- its just easier to review/see what is going on
3: test is failing :) you can run the tests locally to check if you don't want to wait for CI.
—
Reply to this email directly, view it on GitHubhttps://github.com/facebookresearch/fairo/issues/982#issuecomment-1045711724, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ANVW4NZRBWGEQLB4OOCF3UDU34KC3ANCNFSM5NUFFG3Q.
Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign=notification-email&utm_medium=email&utm_source=github.
You are receiving this because you are subscribed to this thread.Message ID: @.***>

The nodes being tested are imported at the top anyways in unit-tests; I also like being more explicit in tests to point the reader to where the string literals are defined.
I only made the changes in one test file so the entire test-suite failing is expected. I will make the changes in all test-files, verify tests are OK and publish a PR.

ok, I am fine with PlayerNode.NODE_TYPE if y'all have stronger opinion.  @stuart-fb we do explicitly list and register nodes in each agent's main memory file, and in the associated memory_nodes.py files.    The reason I wanted to use the string literals was to allow the node to have different implementations (and particular, to allow eventually replacing nodes by blobs of neural weights).  Seeing as we aren't there yet, my opinion isn't strong.


Fixed E3SM results for Part I

Hi @twjuliano, we fixed the model altitude issue in our E3SM SCM simulations. Please let me know if you have other questions. Thanks.

Approved. Thank you @mengz1993 !


show weather in the mode line

Nice package to show weather in Emacs!
I've made some efforts to show it in the mode line. If you were interested, I would create a pull request.
(only works in Emacs 26.1+ for thread support)

I think this would be nice to have especially showing or notifying the incoming event such as storm or rain in a specific area.


new structure

as discussed in https://github.com/yoshuawuyts/nanocomponent/pull/25#issuecomment-293267762

v3.0.0 :tada:

❤


(FLW-1515) Changing wallet name for wallet details screen

I had to use the event service instead of the output, because we need a global update point. The wallet name can be changed, for example, from a wallet tab, but we also need to know about that in profile module.


Change config path to absolute path

Config path is currently a relative path, which means the script will not work if run in a different directory. Change to an absolute path.

(I think it's the other way around - the config path is currently relative, but it should be absolute)

@helrond I am struggling a lot with this for some reason--do I want to do some variation of Path("local_settings.cfg").absolute()?

Take a look at https://stackoverflow.com/questions/3430372/how-do-i-get-the-full-path-of-the-current-files-directory


Run SFTPSensorAsync in integrations tests

Added the SFTPSensorAsync  example DAG into the master DAG, created the connection details via code and other details needed for the DAG to run as a self-sufficient
closes: #662

@rajaths010494 @phanikumv Addressed review comments


chore: Added CI jobs to check classification training

As per #429, this PR introduces the following modifications:

added option to change the font family in the character classification script
added also an option to change the number of samples during training & validation
added CI jobs to run the classification training for a small epoch
increased the sleep in CI job before checking if the demo is up

Any feedback is welcome!

I've checked on my end, there is some seriously obscure issue with TF backprop:
the script works perfectly on GPU, but if you run on CPU, it throws a model sizing error ...
Traceback (most recent call last):
  File "references/classification/train_tensorflow.py", line 261, in <module>
    main(args)
  File "references/classification/train_tensorflow.py", line 198, in main
    fit_one_epoch(model, train_loader, batch_transforms, optimizer, mb)
  File "references/classification/train_tensorflow.py", line 41, in fit_one_epoch
    grads = tape.gradient(train_loss, model.trainable_weights)
  File "/home/fg/miniconda3/lib/python3.8/site-packages/tensorflow/python/eager/backprop.py", line 1074, in gradient
    flat_grad = imperative_grad.imperative_grad(
  File "/home/fg/miniconda3/lib/python3.8/site-packages/tensorflow/python/eager/imperative_grad.py", line 71, in imperative_grad
    return pywrap_tfe.TFE_Py_TapeGradient(
  File "/home/fg/miniconda3/lib/python3.8/site-packages/tensorflow/python/eager/backprop.py", line 159, in _gradient_function
    return grad_fn(mock_op, *out_grads)
  File "/home/fg/miniconda3/lib/python3.8/site-packages/tensorflow/python/ops/nn_grad.py", line 581, in _Conv2DGrad
    gen_nn_ops.conv2d_backprop_input(
  File "/home/fg/miniconda3/lib/python3.8/site-packages/tensorflow/python/ops/gen_nn_ops.py", line 1247, in conv2d_backprop_input
    _ops.raise_from_not_ok_status(e, name)
  File "/home/fg/miniconda3/lib/python3.8/site-packages/tensorflow/python/framework/ops.py", line 6897, in raise_from_not_ok_status
    six.raise_from(core._status_to_exception(e.code, message), None)
  File "<string>", line 3, in raise_from
tensorflow.python.framework.errors_impl.InvalidArgumentError: Computed input depth 960 doesn't match filter input depth 1 [Op:Conv2DBackpropInput]

I can confirm this is at TF level and this is because of grouped convolutions. We cannot do much about this right now


req.user is empty

First I have to say I had a lot of problems running this passport plugin.

Not sure why the googleplus flow seems to me so different than all the others, and I have used quite a few in passport - linkedin, google openid, facebook, github,custom etc..
When I ran your example, I kept getting "INVALID_AUDIENCE". but I still cannot figure why..
When I inserted the snippets to my code it worked better, but for some reason, req.user is always undefined.
I see the req.body.code exists and with the correct value.
I am using post even though I am not sure why. Why not tell google+ to redirect to my backend directly? Why do I need the frontend to post me a message?

my main problem is that req.user returns empty. how can I resolve it?

never mind, I use authorize and I got the details.


v4: Add an actual data-touch="false" example in the carousel docs

Backport of https://github.com/twbs/bootstrap/pull/32638

xfcfv

once this is approved/merged, i'll revisit v5 (since the original https://github.com/twbs/bootstrap/pull/32638 has already been merged) and make matching changes there @mdo

once this is approved/merged, i'll revisit v5 (since the original https://github.com/twbs/bootstrap/pull/32638 has already been merged) and make matching changes there @mdo


Export model from path files

An exported object named MODEL will appear in the request arguments as model.
import type { HTTP_GET, HTTP_PUT } from "../path-types/hello.types.js";

class HelloModel {
   message: "hello world"
}

export const MODEL = new HelloModel();

export const GET:HTTP_GET = ({ model }) => {
    return model.message;
}

export const PUT:HTTP_PUT = ({ body, model, response }) => {
    model.message = body;
    return response[201];
}

The model is loaded the first time Counterfact encounters a file.
The model stays in memory and is not affected by hot-reloading.
The context object is deprecated. model is easier to use and a better name.
If ./paths/a/b/c/d.js does not export a MODEL, Counterfact will look for MODEL in ./paths/a/b/c.js, ./paths/a/b.js, ./paths/a.js, ./paths.js and ultimately fall back to {}.

In TypeScript, every path file MUST export MODEL so that /path-types/**/*.type.ts can find and use it.
export { MODEL } from '../parent.js';

export { MODEL } from '../parent.js';

This might cause a slight inconsistency with hot-reloading.

On load /paths/a.ts exists.
Make a change to the MODEL in /paths/a.ts
Make a change to the OpenAPI document, generate code, and now /paths/a/b.ts is created with a reference to MODEL from /paths/a.ts. It will be a reference to the new version of MODEL.

Now that I've written that out I think this is a weird edge case and not a big deal. But we could work around it. When b.js is loaded, check if the exported object is the same object that was imported (but not used) from the updated a.js. If so, use the original model from a.js instead of the new one.
There also may be a possibility to respond to a change in the model by adding any new properties without overwriting existing properties.
const liveModel = {...updatedModel, ...liveModel}

Instead of sticking the MODEL in a paths file, it might make more sense to have a file with a special name like $MODEL.js which has the model for all endpoints in that directory (and subdirectories, if they don't provide their own $MODEL.js).
That means ./paths/a.js and ./paths/b.js would have to share the same model. That's probably okay most of the time. A couple of ways to work around that.

To give b a different model, export const b from $MODEL.js. The default export will apply to any files that don't have a corresponding variable.
An optional file called b$MODEL.js

I think I like this better, with option 1. It also breaks a circular dependency with imports, which is nice.

ModelLoader will look for $Model.js files (on the first load) and add them to the Registry.  Inside Registry, a handler will have a model as well as a module. That model will get passed to the lambda function.
Registry will store the models in a new ModelFactory object. The counterfact() function will return the modelFactory so that we can manipulate data programmatically.

Almost there. The naming convention I finally settled on is #model.ts which has an export called model.
I still need to make sure everything works end-to-end incorporate #model.ts files in the code generator.

Ended up changing the name from model to context and the file name is $context.ts


KAFKA-9274: handle TimeoutException on task reset

part of KIP-572

congratulations on PR 10,000 :)

congratulations on PR 10,000 :)

Updated this PR.


Feature Request 「plz support InternLM2.5」

Hi,
I noticed that the repository currently lacks support for the InternLM2.5-7B (1.8B, 20B) model, which may cause compatibility issues or missing steps for users trying to implement it. It would be beneficial to update the repository with detailed instructions or tools for integrating and using the InternLM2.5-7B model, ensuring the content remains relevant.
I believe that adding this support would significantly enhance the usability of the project. While some manual adjustments are possible, official guidance or toolchain support would be much more efficient and advantageous, especially for new users.
If possible, including example scripts or demonstrating integration with InternLM2.5-7B would also be a valuable addition.
For further support, please add the InternLM Assistant (WeChat search: InternLM) or join the InternLM Discord(https://discord.com/invite/xa29JuW87d).

hi, thanks for the concern. I checked the documentation and found that I can seamlessly request directly using the openai api format.
endpoint:
https://internlm-chat.intern-ai.org.cn/puyu/api


Tech vias with multiple cuts only generate a single cut in pdngen

Describe the bug
I think this is a regression in https://github.com/The-OpenROAD-Project/OpenROAD/commit/ab7c0f0e5fca2f1a572b7b7337dc80988e1099c3, I believe this was working back in April.
When add_pdn_connect is forced to use a multiple-cut tech via using fixed_vias, only one cut per tech via instantiation is actually generated.
Expected Behavior
There's a 3-cut tech file in our (made-up, but representative of an actual case) tech lef called via1_0_x3h that we force pdngen to use. In this case, the tech via is repeated 4 times horizontally. 12 cuts should be generated, but in fact, only 4 are generated:

Environment
[WARNING] Your current OpenROAD version is outdated.
It is recommened to pull the latest changes.
If problem persists, file a github issue with the re-producible test case.
kernel: Linux 6.10.10-arch1-1
os: Arch Linux
cmake version 3.30.3
CMake Warning at CMakeLists.txt:106 (message):
  OpenROAD git describe failed, using sha1 instead

-- The CXX compiler identification is GNU 14.2.1

To Reproduce
As a shareable minimum example, I've created a version of nangate45 with a three-cut tech via called "via1_0_x3h". The entire example is uploaded here (run nangate45_multicut.tcl):
nangate45_multicut.zip
Relevant log output
No response
Screenshots
No response
Additional Context
No response

I just double checked, and indeed reverting https://github.com/The-OpenROAD-Project/OpenROAD/commit/ab7c0f0e5fca2f1a572b7b7337dc80988e1099c3 makes this work as expected:

@gadfort thoughts?

@maliberty I'll take a look this afternoon, I do recall there being some odd behavior with multicut vias.


Extract evaluator task from _rabit_allred_task and fix evaluator task

We need to release a new version of tf-yarn before


Fix non-transparent SVG icons and drawings

The SVG icons and drawings are showing a non-transparent black background. This needs to be fixed.

Fixed in 0.0.6


ubuntu22.4安装好GPU版本2.1.0后不能使用GPU

Environment
Hardware Environment(Ascend/GPU/CPU):

Uncomment only one  /device <> line, hit enter to put that in a new line, and remove leading whitespaces from that line:
/device gpu

Software Environment:

MindSpore version (source or binary):2.1.0
Python version (e.g., Python 3.7.5):3.7.5
OS platform and distribution (e.g., Linux Ubuntu 16.04):Linux Ubuntu 22.04
GCC/Compiler version (if compiled from source): 7/8

Describe the current behavior
(mindspore_py37) ***:~/mindyolo$ python -c "import mindspore;mindspore.set_context(device_target='GPU');mindspore.run_check()"
MindSpore version:  2.1.0
The result of multiplication calculation is correct, MindSpore has been installed on platform [GPU] successfully!
[ERROR] DEVICE(2382696,7fe93dd0b740,python):2023-11-28-21:28:17.917.989 [mindspore/ccsrc/runtime/pynative/async/async_queue.cc:232] WorkerJoin] WorkerJoin failed:

Cuda error:

The cudaStreamCreateWithFlags failed, ret[2], out of memory

C++ Call Stack: (For framework developers)

mindspore/ccsrc/plugin/device/gpu/hal/device/cuda_driver.cc:182 CreateStream
[ERROR] ME(2382696,7fe93dd0b740,python):2023-11-28-21:28:17.918.038 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:541] WaitTaskFinishOnDevice] SyncStream failed
Describe the expected behavior
不知道是什么原因
Related log / screenshot

Hello, welcome to MindSpore. Here is a message indicating that your memory is insufficient. It is recommended that you clean up the content and try again.


Conflict with auto-pairs plugins in Vim

This may be difficult problem to solve...

What auto-pairs does is something like: inoremap ( (), so there would be a ) after your cursor. And autocompletion only depends on what you enter before the cursor.

I have tested TabNine's output.
TabNine seems ignore ) for old_suffix.
I think TabNine should set old_suffix as ).

And autocompletion only depends on what you enter before the cursor.

Current TabNine checks the after of the cursor.
So it can be implemented by TabNine.


Automatic email to users when their pass is cancelled in DUP admin

Description
As a BC Parks admin (sysadmin) user, I want a cancellation email to be sent to a pass holder when I cancel their pass in the backend so that this process is faster and I do not need to manually send a bunch of emails.
This ticket is to automate the cancellation process. Currently, once a pass is cancelled by an admin user (in the instance of a park or facility closure) the list of pass holders is exported and an email sent to all of them indicating the reason for closure and that their pass is cancelled. This ticket is to automate this process.
Would likely need a cancel email template with the reason for closure from admin included.
Acceptance Criteria:
Notes
Linked issues
is blocked by

[ ] #314

relates to

[x] ~BRS-1156~

November 21, 2024 - DUP prioritization session. This ticket/issue was identified as medium priority.
https://app.mural.co/t/bcparks2575/m/bcparks2575/1729124366479/8adea59d9e76fb3eb1e33395f1c1857a8c07fb7b


Extend EquationSystem solving to eliminate the number of equations = number of variables limitation

e.g. AngouriMath.MathS.Equations("a+b-c", "a-b-c").Solve("a", "b", "c")

Still unclear what you want. What is a possible scenario using it and at the same time it is too hard for the user to solve it in a loop for all variables?

What behavior do you expect? EquationSystem is used to solve a system of equations with finite number of solutions, which is not true in your case

The solution is finite: a=c, b=0, c=a

We decided not to implement such functional as there is still no need to solve N-set of equation over K variables for K > N. If you want to solve N equations for K variables you will get C(N, K) * F solutions, where F - number of solutions for N variables and for N equations. There is no need at all, and if you need to solve an equation or system for one specific variable - solve for it. Even WA works similarly (you see, it took 2 variables from the system) to AM in this case

solve an equation or system for one specific variable

Solving an equation system for one variable is forbidden here:
https://github.com/asc-community/MathS/blob/58364d4ba64b9820e727840f72a5fc556be4df4c/AngouriMath/Functions/Algebra/Solver/EquationSolver.cs#L65-L68
So I do not understand how this is currently possible.

I meant you include your variable into the list of variables you are going to solve for. If you need to solve for b, you include b. You cannot solve for more variables than the number of equations and what you suggest is unnecessary and not desired

I meant you include your variable into the list of variables you are going to solve for. If you need to solve for b, you include b. You cannot solve for more variables than the number of equations

These are two contradictory statements. I need to solve for a, b and c, but this is more then the number of equations.

You don't need to solve for a, b, and c. Not a possible scenario.

On the other hand, only including b will still hit the exception.

Even WA works similarly (you see, it took 2 variables from the system) to AM in this case

Not really. Try a^2+b=c+d, a-c^2=b+2d: three solutions appear.

As for your system, not sure, but then it shows two solutions for two variables. I don't know what is meant by the first set of solution as it doesn't make any sense.

As a user, you can just go over the system in a loop and work out x for each of them

Can't this be built-in?

There are too many problems to do now instead of something that would be never used.

Fine


Explicitly use int64 value of StructuredData

Currently StructuredData.Number is limited to int, uint and double, which is perfectly fine for people who use 32 bit integers and 64 bit integers on 64 bit platforms.
But if you are on a 32 bit platform and you expect aStructuredData value to be e.g. an unsigned 64 bit integer, you will probably have some unexpected silent problems as Int and UInt are limited to 32 bits andStructuredData will return Int.max for integers bigger than 32 bits.
I am currently working on a vapor library for Telegram chatbots and they have stated in their documentation that some values are actually bigger than 32 bits which means I must ignore 32 bit platforms and add a warning to the README that this library should not be used on 32 bit platforms.
What do you guys think about that? Do we want to ignore 32 bit platforms as they already become extinct or do we want to add support for explicit int64 and uint64 values inside StructuredData.Number?
Reference (Number.swift file in this library):
Number.swift

If an application requires native integers to be 64 bits in width then I think it makes sense to not allow that application to run on 32 bit machines.

Int64 works just fine on 32 bit systems too. Consider C, the long long type has been available since C99, years before 64 bit systems became generally available 😛.
Indeed, there might be performance penalty involved, but some applications really require 64 bit integers.

@vzsg good point. :)
As a workaround for now, 64 bit integers could be stored as strings in the structured data and converted to 64 bit integers in an extension.
I think this issue will be made easier to tackle in upcoming updates to Vapor that will make the StructuredData type much less prevalent.
@Ybrin what type specifically are you using in this case? Are you using a Node or StructuredData directly or are you using one of the sub types like Row or JSON? In 3.0 we're hoping to transition the subtypes to being their own types not reliant on StructureData. This means that it would be much easier to add say Int64 support to just Row if it's needed there. Right now StructuredData is a bit of a god object. Changing it even slightly affects a ton of code. For example, JSON currently extends StructuredData and all JSON can just be Doubles. Adding Int64 support there (or having Int support in the first place) breaks spec and could potentially cause interoperability problems, unexpected behavior, or unnecessary overhead .


chore(node,core): add more test types

Trying to understand better how these tests and scenarios interact.

Current dependencies on/for this PR:

boneskull/policy-schema

PR #648

PR #651   👈

This comment was auto-generated by Graphite.

It looks like some of the types (like "file") aren't test-specific, though.  These are defined in test/scenarios/scenario.d.ts: ScenarioFile, NormalizedScenarioFile, NormalizedScenarioJSFile, etc.

@naugtur I'm not sure what you're seeing here that is overlapping.  Anyway, I retargeted this to #753 since it was struggling with the ESLint settings.


fix: update the implementation method of obtaining the git root direc…

The path returned by function get_git_root is always ".", resulting in the Env file not being found.

@all-contributors add @bytedaring for code,doc.


docs(parser): share oxc_parser module docs with oxc::parser

What This PR Does
Moves oxc_parser module documentation to a README.md file so that the
oxc crate can benefit from it as well.
The downside of this PR is that it breaks example inclusion with
#![doc include_str(..)], since macros in loaded files are not run. I'm
not 100% sure that the tradeoffs are worth it, but this is worth
discussing.

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

#5832  👈
#5831
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @DonIsaac and the rest of your teammates on  Graphite

since macros in loaded files are not run

Did you mean examples in loaded files are not run?

since macros in loaded files are not run

Did you mean examples in loaded files are not run?

I did not.
Given this README:
Here's a description for some stuff

## Example
'''rust
#![doc = include_str!("../examples/foo.rs")]
'''

note: using single quotes instead of backticks above as to not break formatting of this comment

When this gets loaded in as documentation in lib.rs
#![doc = include_str!("../README.md")]

then the macro invocation in README.md (include_str!("../examples/foo.rs")) wiil not be invoked, and will instead be displayed as-is.

Ah. Now I understand. Thanks for explaining.

This seems confusing, maybe just a docs.rs link to the actual crate should suffice?


fix: 修正一些对内的地址

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


API Issue?

This looks to be a promising alternative to salt enterprise. I am testing it out in docker.
Describe the bug
Getting 500 Error when refreshing key/minions. API socket returns OK. Not seeing any other data.
API Config:
 rest:
    ^url: http://docker:8001/api/token/verify/
    fred:
      - .*
      - '@jobs'
      - '@wheel'
      - '@runner'

Am I missing something in the API config?
Here is the docker env's.

Thanks for giving Alcali a try!
the docs should definitely be clearer.
There's currently two different ways to authenticate Alcali against the Salt-Api.

Using a custom auth module. In that case, Alcali just need to be able to reach the Salt-Api.

Using the Api Endpoint. In that case, the Salt-Master also need to be able to reach Alcali.

Is your docker container reachable from the Salt-Master? If not, you should consider using the first option.

I am going with the API endpoint, and the servers live on the same subnet and should have full communications enabled.

Ok, here is the master's log in debug mode during a rest call:
master_1  | 20:59:41 master.1 | [DEBUG   ] Requesting URL http://web:8000/api/token/verify/ using POST method
master_1  | 20:59:41 master.1 | [DEBUG   ] Using backend: tornado
master_1  | 20:59:41 master.1 | [DEBUG   ] Response Status Code: 200
master_1  | 20:59:41 master.1 | [DEBUG   ] eauth REST call returned 200: {'body': '{"admin":null}', 'dict': {'admin': None}, 'status': 200}

And alcali's log:
web_1     | [29/Oct/2019 15:59:41] "POST /api/token/verify/ HTTP/1.1" 200 14

Could you provide the same logs?
Here is the curl command to reproduce the Rest Call:
curl -sSk -H 'Accept: application/json' -d username=admin -d password=b811a13c582fe24f287a1c13ba308f634f04159c http://web:8000/api/token/verify/

Where password is the user's token

Response from the Salt Master when querying keys:
[DEBUG   ] Reading configuration from /etc/salt/minion
[DEBUG   ] Including configuration from '/etc/salt/minion.d/_schedule.conf'
[DEBUG   ] Reading configuration from /etc/salt/minion.d/_schedule.conf
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: salt-stack.amplex.net
[DEBUG   ] Requesting URL http://dock1.amplex.net:8001/api/token/verify/ using POST method
[DEBUG   ] Using backend: tornado
[DEBUG   ] Response Status Code: 200
[DEBUG   ] eauth REST call returned 200: {'body': '{"admin":null}', 'status': 200, 'dict': {'admin': None}}
[DEBUG   ] Could not LazyLoad rest.groups: 'rest.groups' is not available.
[DEBUG   ] LazyLoaded localfs.mk_token

Response from the curl command issued from the salt-master to the docker server running ALCALI.
curl -sSk -H 'Accept: application/json' -d username=admin -d password=xxxxxxx http://dock1:8001/api/token/verify/
{"admin":null}
So couple of oddities, I am running the docker instance with the GUI on port 8001 instead of 8000. Could this cause an issue?

Unfortunately, according to your logs, it looks like the authentication is successful.
That mean that the problem is after that, i''m guessing a database issue.
an easy way to find out is:

Stop Alcali

Add: DJANGO_DEBUG=true to your env var.

Restart Alcali

It should show you the traceback.
/!\ Be advise that anyone who can access the service while debug mode is on can probably retrieve all environment variable!
Possible database issue:
Is the db schema properly inserted ? (Salt would scream a lot if not but you never know)
Have you run the database migration?
What does alcali check return?

So after enabling debug logging.....
Watching for file changes with StatReloader,
Performing system checks...,
,
System check identified no issues (0 silenced).,
October 31, 2019 - 09:42:37,
Django version 2.2.6, using settings 'config.settings',
Starting development server at http://<IP_ADDRESS>:8001/,
Quit the server with CONTROL-C.,
[31/Oct/2019 09:42:52] "GET / HTTP/1.1" 200 1184,
[31/Oct/2019 09:42:52] "GET /static/css/chunk-vendors.47ff193e.css HTTP/1.1" 304 0,
[31/Oct/2019 09:42:52] "GET /static/css/app.321866b5.css HTTP/1.1" 304 0,
[31/Oct/2019 09:42:52] "GET /static/js/chunk-vendors.439a09fa.js HTTP/1.1" 304 0,
[31/Oct/2019 09:42:52] "GET /static/js/app.b0501687.js HTTP/1.1" 304 0,
[31/Oct/2019 09:42:53] "POST /api/token/verify/ HTTP/1.1" 200 14,
Unauthorized: /api/jobs/graph,
Unauthorized: /api/keys/keys_status/,
[31/Oct/2019 09:42:53] "GET /api/keys/keys_status/ HTTP/1.1" 401 58,
Unauthorized: /api/stats/,
[31/Oct/2019 09:42:53] "GET /api/jobs/graph?fun=all&period=7 HTTP/1.1" 401 58,
[31/Oct/2019 09:42:53] "GET /api/stats/ HTTP/1.1" 401 58,
Unauthorized: /api/jobs/filters/,
[31/Oct/2019 09:42:53] "GET /api/jobs/filters/ HTTP/1.1" 401 58,
Unauthorized: /api/jobs/,
[31/Oct/2019 09:42:53] "GET /api/jobs/?limit=10 HTTP/1.1" 401 58,
Unauthorized: /api/userssettings/1/,
[31/Oct/2019 09:42:53] "GET /api/userssettings/1/ HTTP/1.1" 401 58,
Bad Request: /api/token/refresh/,
[31/Oct/2019 09:42:53] "POST /api/token/refresh/ HTTP/1.1" 400 43,
Not Found: /favicon.ico,
[31/Oct/2019 09:42:53] "GET /favicon.ico HTTP/1.1" 404 4894,
Unauthorized: /api/token/,
[31/Oct/2019 09:42:59] "POST /api/token/ HTTP/1.1" 401 63,
Bad Request: /api/token/refresh/,
[31/Oct/2019 09:42:59] "POST /api/token/refresh/ HTTP/1.1" 400 43,
[31/Oct/2019 09:43:35] "POST /api/token/ HTTP/1.1" 200 492,
[31/Oct/2019 09:43:36] "POST /api/token/verify/ HTTP/1.1" 200 14,
[31/Oct/2019 09:43:36] "GET /api/keys/keys_status/ HTTP/1.1" 200 53,
[31/Oct/2019 09:43:36] "GET /api/jobs/graph?fun=all&period=7 HTTP/1.1" 200 146,
[31/Oct/2019 09:43:36] "GET /api/minions/conformity/ HTTP/1.1" 200 70,
[31/Oct/2019 09:43:36] "GET /api/stats/ HTTP/1.1" 200 35,
[31/Oct/2019 09:43:36] "GET /api/jobs/filters/ HTTP/1.1" 200 25,
[31/Oct/2019 09:43:36] "GET /static/fonts/materialdesignicons-webfont.0aee1bb1.woff2 HTTP/1.1" 304 0,
[31/Oct/2019 09:43:36] "GET /api/userssettings/1/ HTTP/1.1" 200 232,
[31/Oct/2019 09:43:36] "GET /api/jobs/?limit=10 HTTP/1.1" 200 2,
[31/Oct/2019 09:43:39] "GET /api/minions/ HTTP/1.1" 200 2,
[31/Oct/2019 09:43:41] "POST /api/minions/refresh_minions/ HTTP/1.1" 200 16,
[31/Oct/2019 09:43:42] "GET /api/minions/ HTTP/1.1" 200 2,
[31/Oct/2019 09:43:58] "GET /api/keys/ HTTP/1.1" 200 2,
[31/Oct/2019 09:44:00] "POST /api/token/verify/ HTTP/1.1" 200 14,
Internal Server Error: /api/keys/refresh/,
Traceback (most recent call last):,
  File "/opt/alcali/.local/lib/python3.7/site-packages/pepper/libpepper.py", line 227, in req,
    f = urlopen(req, context=con),
  File "/usr/local/lib/python3.7/urllib/request.py", line 222, in urlopen,
    return opener.open(url, data, timeout),
  File "/usr/local/lib/python3.7/urllib/request.py", line 531, in open,
    response = meth(req, response),
  File "/usr/local/lib/python3.7/urllib/request.py", line 641, in http_response,
    'http', request, response, code, msg, hdrs),
  File "/usr/local/lib/python3.7/urllib/request.py", line 569, in error,
    return self._call_chain(*args),
  File "/usr/local/lib/python3.7/urllib/request.py", line 503, in _call_chain,
    result = func(*args),
  File "/usr/local/lib/python3.7/urllib/request.py", line 649, in http_error_default,
    raise HTTPError(req.full_url, code, msg, hdrs, fp),
urllib.error.HTTPError: HTTP Error 401: Unauthorized,
,
During handling of the above exception, another exception occurred:,
,
Traceback (most recent call last):,
  File "/opt/alcali/.local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner,
    response = get_response(request),
  File "/opt/alcali/.local/lib/python3.7/site-packages/django/core/handlers/base.py", line 115, in _get_response,
    response = self.process_exception_by_middleware(e, request),
  File "/opt/alcali/.local/lib/python3.7/site-packages/django/core/handlers/base.py", line 113, in _get_response,
    response = wrapped_callback(request, *callback_args, **callback_kwargs),
  File "/opt/alcali/.local/lib/python3.7/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view,
    return view_func(*args, **kwargs),
  File "/opt/alcali/.local/lib/python3.7/site-packages/rest_framework/viewsets.py", line 114, in view,
    return self.dispatch(request, *args, **kwargs),
  File "/opt/alcali/.local/lib/python3.7/site-packages/rest_framework/views.py", line 505, in dispatch,
    response = self.handle_exception(exc),
  File "/opt/alcali/.local/lib/python3.7/site-packages/rest_framework/views.py", line 465, in handle_exception,
    self.raise_uncaught_exception(exc),
  File "/opt/alcali/.local/lib/python3.7/site-packages/rest_framework/views.py", line 476, in raise_uncaught_exception,
    raise exc,
  File "/opt/alcali/.local/lib/python3.7/site-packages/rest_framework/views.py", line 502, in dispatch,
    response = handler(request, *args, **kwargs),
  File "/opt/alcali/code/api/views/alcali.py", line 66, in refresh,
    get_keys(refresh=True),
  File "/opt/alcali/code/api/backend/netapi.py", line 45, in get_keys,
    api_ret = api.wheel("key.list_all")["return"][0]["data"]["return"],
  File "/opt/alcali/.local/lib/python3.7/site-packages/pepper/libpepper.py", line 436, in wheel,
    return self.low([low]),
  File "/opt/alcali/.local/lib/python3.7/site-packages/pepper/libpepper.py", line 298, in low,
    return self.req(path, lowstate),
  File "/opt/alcali/.local/lib/python3.7/site-packages/pepper/libpepper.py", line 239, in req,
    raise PepperException('Authentication denied'),
pepper.exceptions.PepperException: Authentication denied,
[31/Oct/2019 09:44:00] "POST /api/keys/refresh/ HTTP/1.1" 500 16018,

Alcali check returns...
  File "/opt/alcali/.local/bin/alcali", line 6, in <module>
    from pkg_resources import load_entry_point
  File "/opt/alcali/.local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3251, in <module>
    @_call_aside
  File "/opt/alcali/.local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3235, in _call_aside
    f(*args, **kwargs)
  File "/opt/alcali/.local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3264, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/opt/alcali/.local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 583, in _build_master
    ws.require(__requires__)
  File "/opt/alcali/.local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 900, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/opt/alcali/.local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 786, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'alcali' distribution was not found and is required by the application

Looks like my docker build is hosed. Perhaps I should start over.

When running ./manage.py check:
env:    ['MASTER_MINION_ID']```

when I am `./manage.py migrate`:
``` ./manage.py migrate
Operations to perform:
  Apply all migrations: admin, api, auth, contenttypes, sessions
Running migrations:
  No migrations to apply.```

Ok, i got it:
Do you have a user named  admin defined in external_auth with sufficient rights?
external_auth:
  rest:
    ^url: http://web:8000/api/token/verify/
    admin:
      - .*
      - '@runner'
      - '@wheel'

from the traceback, the auth to Alcali is ok, it's salt who's not happy.
And i can reproduce the same behaviour.

rest_cherrypy:
  port: 8080
  host: <IP_ADDRESS>
  log_access_file: '/var/log/salt/api_access.log'
  log_error_file: '/var/log/salt/api_error.log'
  ssl_crt:  **
  ssl_key: **

external_auth:
  pam:
    amplexadmin:
      - .*
      - '@wheel'   # to allow access to all wheel modules
      - '@runner'  # to allow access to all runner modules
      - '@jobs'    # to allow access to the jobs runner and/or wheel module
    silica_api:
      - .*
      - '@wheel'   # to allow access to all wheel modules
      - '@runner'  # to allow access to all runner modules
      - '@jobs'    # to allow access to the jobs runner and/or wheel module
  rest:
    ^url: http://dock1.amplex.net:8001/api/token/verify/
    admin:
      - .*
      - '@runner'
      - '@wheel'```

The PAM auth works correctly. I just ran some queries to the API. ALCALI is still not happy.

And it just started working after a couple more restarts. Very Odd. I need to add the job returner for postgres next. Thanks for taking the time to assist! I will plan on using this tool, it may save me from getting SSE.


RemixSite - allow esbuild configuration for external

I'm trying to use external dependency with RemixSite, in this case sharp image processing. I've put the sharp in a Lambda layer, but from what I understand it is impossible to use it. The same case applies for any other external package. The reason being that with RemixSite you can not configure external option with esbuild.
https://esbuild.github.io/api/#external

You can mark a file or a package as external to exclude it from your build. Instead of being bundled, the import will be preserved (using require for the iife and cjs formats and using import for the esm format) and will be evaluated at run time instead.

Sharp installation instructions (https://sharp.pixelplumbing.com/install#esbuild) says this package should be set as external.
buildSync({
  entryPoints: ['app.js'],
  bundle: true,
  platform: 'node',
  external: ['sharp'], // use sharp as external dependency
})

If I look at the RemixSite.ts source external property is hard coded.
https://github.com/serverless-stack/sst/blob/master/packages/sst/src/constructs/RemixSite.ts#L140
const result = esbuild.buildSync({
      entryPoints: [serverPath],
      bundle: true,
      target: "node16",
      platform: "node",
      external: ["aws-sdk"],
      outfile: path.join(outputPath, "server.js"),
      // We need to ensure that the polyfills are injected above other code that
      // will depend on them. Importing them within the top of the lambda code
      // doesn't appear to guarantee this, we therefore leverage ESBUild's
      // `inject` option to ensure that the polyfills are injected at the top of
      // the bundle.
      inject: [polyfillDest],
    });

Is there a workaround?

I don't believe there is an option for this at the moment but would affect this issue too as the ability to query a PostgreDB needs the pg package. This has a dependency on pg-native which we would need esbuild configuration in order to deploy: https://github.com/serverless-stack/sst/issues/2412

I'll ask Frank to take a look at this.

I just fixed this with patch-package, by adding the relevant packages to the array passed to the external argument of createServerLambdaBundle in RemixSite.js, though that isn't a very durable solution.
Seems like this might be resolved by changing the relevant lines to be something like:
const { runtime: runtimeRaw, timeout, memorySize, environment, cdk, external = [] } = this.props;
// ...
const bundlePath = this.createServerLambdaBundle("regional-server.js", runtime === "nodejs18.x" ? external : ["aws-sdk", ...external]);

If that seems like a good solution, I could put an MR in.

It seems it's always those few trouble makers, namely sharp and pg. I'm planning to automatically add them as external if they are detected. I can get this in this week.
Any other packages we should add to the list?

I've been experimenting with postgres, which also attempts to include pg-native.

@fwang I think this would be great and could solve the issue of postgres causing remix to not build correctly. Just for clarification I believe we would only need to add pg-native to the external list, not pg itself. Here is an esbuild issue that captures other running into this and the solution too: https://github.com/brianc/node-postgres/issues/2800
@kazdegroot I have looked at quite a few ORMs/libraries to connect to postgres and I think all of them use pg somewhere down the line (which is trying to include pg-native) so I believe this addition would fix the build issues from the package you linked and (hopefully) all others.
This would fix the issue I created here #2412 as well btw.

@SpencerDuball Just tried that, and seems like excluding just pg-native works. Apparently went a bit over-eager there.
Though postgres does not use pg; it also does not seem to use pg-native, but it was still causing bundling errors when I was only excluding pg, so that's interesting.

Just released 2.1.21.
SST will auto detect sharp and pg-native, mark them as externals, and bundle them in node_modules.
Side node.. now you can also pass in nodejs like this, but that shouldn't be necessary to get sharp and pg-native to work.
  const site = new RemixSite(stack, "regional", {
    path: "sites/remix",
    bind: [f, s],
    environment: {
      FUNCTION_NAME: f.functionName,
    },
    nodejs: {
      install: ["sharp"],
      format: "cjs",
      esbuild: { ... },
    },
  });

Lemme know if this works.
cc @SpencerDuball @kazdegroot @pehtoorismies

@fwang Just redeployed after removing my patch and updating, and everything seems to be working great. Thanks!

Just writing this as a FYI. I ended up using a lambda layer with RemixSite. The solution is a bit cumbersome since it is not possible to add any lambda layers with RemixSite constructor or stack.addDefaultFunctionLayers().
export const RemixStack = ({ stack, app }: StackContext) => {
  // create layer
  const sharpLayer = new lambda.LayerVersion(
    stack,
    `${app.stage}-SharpLayer'`,
    {
      code: lambda.Code.fromAsset('stacks/layers/sharp'),
      compatibleArchitectures: [lambda.Architecture.ARM_64],
    }
  )

  // create RemixSite
  const site = new RemixSite(stack, 'remix-site', {
    // needed for layer to work
    nodejs: {
      esbuild: {
        external: ['sharp'],
      },
    },
  })

  // a bit smelly solution to add layer to Remix-handler lambda
  const serverHandler = site.cdk?.function as lambda.Function
  serverHandler.addLayers(sharpLayer)

  stack.addOutputs({
    URL: site.url ?? 'localhost',
  })
}

I was trying to add Sharp to my remix app and had the same issue. I added the configuration to the sst.aws.Remix:
nodejs: {         install: ["sharp"],         // loader: {         //   ".node": "binary",         // },         esbuild: {           external: ["sharp"],         },       },
The sst console got the following error:
06:50.641 Unknown application error occurred Runtime.Unknown 12:06:52.414 Uncaught Exception	{   "errorType": "Error",   "errorMessage": "Could not load the \"sharp\" module using the linux-x64 runtime\nPossible solutions:\n- Ensure optional dependencies can be installed:\n    npm install --include=optional sharp\n    yarn add sharp --ignore-engines\n- Ensure your package manager supports multi-platform installation:\n    See https://sharp.pixelplumbing.com/install#cross-platform\n- Add platform-specific dependencies:\n    npm install --os=linux --cpu=x64 sharp\n- Consult the installation documentation:\n    See https://sharp.pixelplumbing.com/install",   "stack": [     "Error: Could not load the \"sharp\" module using the linux-x64 runtime",     "Possible solutions:",     "- Ensure optional dependencies can be installed:",     "    npm install --include=optional sharp",     "    yarn add sharp --ignore-engines",     "- Ensure your package manager supports multi-platform installation:",     "    See https://sharp.pixelplumbing.com/install#cross-platform",     "- Add platform-specific dependencies:",     "    npm install --os=linux --cpu=x64 sharp",     "- Consult the installation documentation:",     "    See https://sharp.pixelplumbing.com/install",     "    at Object.<anonymous> (/var/task/node_modules/sharp/lib/sharp.js:114:9)",     "    at Module._compile (node:internal/modules/cjs/loader:1369:14)",     "    at Module._extensions..js (node:internal/modules/cjs/loader:1427:10)",     "    at Module.load (node:internal/modules/cjs/loader:1206:32)",     "    at Module._load (node:internal/modules/cjs/loader:1022:12)",     "    at Module.require (node:internal/modules/cjs/loader:1231:19)",     "    at require (node:internal/modules/helpers:179:18)",     "    at Object.<anonymous> (/var/task/node_modules/sharp/lib/constructor.js:10:1)",     "    at Module._compile (node:internal/modules/cjs/loader:1369:14)",     "    at Module._extensions..js (node:internal/modules/cjs/loader:1427:10)"   ] }
I checked the source code and found the npm install command in the .sst/platform/src/runtime/node.ts is incorrect.
if (installPackages.includes("sharp")) {         cmd.push(           "--platform=linux",           input.architecture === "arm64" ? "--arch=arm64" : "--arch=x64",         );       }
According to the docs of sharp : https://sharp.pixelplumbing.com/install#cross-platform
I changed the code to:
if (installPackages.includes("sharp")) {         cmd.push(           "--platform=linux",           input.architecture === "arm64"             ? "--arch=arm64"             : "--cpu=x64 --os=linux",         );       }
Then the sharp installed correctly.


removed redundant code in populatePage function, tested by deploying …

…new script and there seem to be no issues

JorBeit


Can you develop a version for pytorch models?

Great job!
Would you please develop a pytorch version so that I can use it to visualize the attention of NN trained on Pytorch & fast.ai ?
Thanks a lot!

Hey @wshuyi, thanks for the suggestion. However, I am quite busy with other stuff. I remember seeing a post on using fastai for such purpose. I will see if I can find it.

Just started working it, will update here.


Proposal: Revoke types narrow-downs/guards on object properties for the rest of the block after function call

I couldn't find a better title, you are welcome to edit it for a better one :)
Given this code:
const tree: {
  branch: "banana" | undefined,
  pick(): void
} = {
    branch: "banana",
    pick() {
      this.branch = undefined;
    }
};

function climb() {
  if (tree.branch) {
    const { branch } = tree;

    tree.pick();
     // this will work
    branch.toString();

    // this will throw
    // Uncaught TypeError: Cannot read property 'toString' of undefined
    // But ts typecheck strict null check won't catch it
    // If you revoke the narrow downs the checker would can't it
    tree.branch.toString();
  }
}

climb();

The expression
 tree.branch.toString();

Typescript compiler with strictNullCheck, won't report any error, but it will throw runtime error, because the code in tree.pick()changed it to another possible type.
It doesn't have to be null/undefined, but any types narrow-downs.
We could make the type checker more robust and to catch more possible  runtime errors by revoking any type narrow-down on object props, forcing the developer the handle the possibility the function call changed the type

So basically:
const obj: { x?: number } = { x: 0 };
if (obj.x !== undefined) {
    evil();
    obj.x.toString(); // Should we allow this?
}

function evil() {
    obj.x = undefined;
}

Looks like a duplicate of #9998.

Seems so,
Thanks!


[Android] IntentAndroid.getInitialURL is undefined

I was trying to call IntentAndroid.getInitialURL using example from the React docs
componentDidMount() {
  var url = IntentAndroid.getInitialURL(url => {
    if (url) {
      console.log('Initial url is: ' + url);
    }
  });
}

I've configured intent-filter in manifest file but I've got an error that's saying getInitialURL is undefined. Don't know if I've done something wrong or it's just a bug or I need to make something more to get this to work. I also want to note that canOpenURL and openURL methods works fine.
React Native v0.17

getInitialURL isn't implemented in v0.17. It's in master but they haven't released a version including it yet. I've just run into the same problem and am currently trying to get it working using react-native: "facebook/react-native" in my package.json to use the master version directly from the git repo.

@grimor Here is a reference to the commit: https://github.com/facebook/react-native/commit/eb188c8d98fa08686c6b9335e3dcbefe3f524b7e
Feel free to close this if you get it working after testing on master.

@christopherdro Thanks! I thought it could be not released yet. But It should be be available in the docs then ? Or maybe it should have some information about it's upcoming feature
I've managed to solve my problems using WebView but I'll test it anyway later today and close issue if it will work.

Ok, I've managed to update react-native from github master branch. And getInitialState is working, but not able to listen for app launch from an external url registered to my app. Anyway when I tried to update to newest react-native I ran into that bug #5030, I had to update MainActivity.java and had to recompile react-native from source using NDK. Then everything worked fine

@christopherdro have the same issue. I installed react-native from github but it throw error IntentAndroidModule.getInitialURL is not a function
NOTE: I have method IntentAndroid.getInitialURL on IntentAndroid but IntentAndroidModule(seems it's native module) does not have getInitialURL(callback);
and IntentAdroid failed when I call function getInitialURL

You recompiled react-native using NDK ? I had to do taht in order to get
that method working
On Jan 14, 2016 5:22 PM, "Maksim Ryzhikov"<EMAIL_ADDRESS>wrote:

@christopherdro https://github.com/christopherdro have the same issue.
I installed react-native from github but it throw error IntentAndroidModule.getInitialURL
is not a function
NOTE: I have method IntentAndroid.getInitialURL on IntentAndroid but
IntentAndroidModule(seems it's native module) does not have
getInitialURL(callback);
and IntentAdroid failed when I call function getInitialURL
—
Reply to this email directly or view it on GitHub
https://github.com/facebook/react-native/issues/5023#issuecomment-171688468
.

@grimor thanks for response.

You recompiled react-native using NDK
I think no, how can I do it?

There is explanation how to do that
https://facebook.github.io/react-native/docs/android-building-from-source.html#content
On Jan 14, 2016 8:25 PM, "Maksim Ryzhikov"<EMAIL_ADDRESS>wrote:

@grimor https://github.com/grimor thanks for response.
You recompiled react-native using NDK
I think no, how can I do it?
—
Reply to this email directly or view it on GitHub
https://github.com/facebook/react-native/issues/5023#issuecomment-171750921
.

Note that you can also try the 0.18.0-rc.


Missing documentation

[ ] builds/relationships/property

[ ] companies/relationships/properties

[ ] data_elements/updated_with_extension

[ ] extensions/updated_with_extension_package

[ ] libraries/relationships/builds

[ ] properties/relationships/libraries

[ ] rule_components/relationships/updated_with_extension

[ ] rules/relationships/rule_components

[ ] update extensions

[ ] delete extension_packages

@spencerroan do we just need to make these pages and then get the appropriate liquid tags in them?  Or are there kessel update also needed?

builds.*property
properties.get_the_property_for_a_build
companies.*properties

not in kessel
data_elements.*updated_with_extension
not in kessel
extensions.*updated_with_extension
not in kessel
extensions.*updated_with_extension
not in kessel
libraries.*builds
not in kessel
rules/relationships/rule_components
the relationships endpoint no longer exists
extensions update
not in kessel
extension_packages.delete
not in kessel

@adobe export issue to Jira project PDCL


Starting portal doesn't start Mongodb

Please remove the doneSet when starting mongo in the cuisine module.

https://github.com/Jumpscale/jumpscale_core8/commit/55f03f0e5f5497939ce3a54d82c55635e0ebde08


If public urls are disabled, hasDeviceUrl(uuid) doesn't return a boolean value

Expected Behavior
It's expected that  balena.models.device.hasDeviceUrl(uuid) should always return a boolean value.
Actual Behavior
If public urls are disabled, balena.models.device.hasDeviceUrl(uuid) doesn't return a boolean value, but instead returns an error.
Steps to Reproduce the Problem

disable the public url for the device
use the node-sdk to run hasDeviceUrl(uuid)
error is thrown

I'm pretty sure that this was never the case since we have tests cases for this and I just tried it on my side and wan't able to reproduce.
See: https://github.com/balena-io/balena-sdk/blob/bac5f0e69c2049f84b131e72e6016ee6467a0eb6/tests/integration/models/device.spec.js#L879-L923
The only case that I could guess that this might throw an error (other than connectivity issues and internal server errors) would be if this was run against openBalena, which doesn't support public device urls.


Internationalisation

All text will be encoded in the configuration file.
The configuration file should be queried with the devices language as part of the route.
Fallback is: English.

We need to add a settings area to change the current language and other settings that may arise #20

[x] English
[x] Italian
[ ] German
[ ] Danish
[ ] Dutch
[ ] Spanish


INTLY-107 Allow single step tasks without a title

Motivation
Currently, when creating a Walkthrough if a writer would like to
have a task that contains a single step they must still define a
h3 block and give the block a title. Otherwise features like the
verification block cannot be used.
In some cases, when creating this type of single step task, the
title of the task is the title of the step also.
What
This change allows tasks to contain verification blocks, instead
of them only being allowed in steps. Meaning that a writer can
create a single step task without having to specify a h3 block.
Verification blocks will still work as expected in the web app.
How
The WalkthroughTask#fromAdoc function now also allows for verifications to be included at its top level, instead of needing to be nested inside steps.
Verification Steps

Add a task that looks something like the following to an existing walkthrough, and ensure it renders correctly and that the Next button only goes blue when all verifications are complete.

== Example task

An example task with no steps, but it includes verification blocks.

[type=verification]
This should be the first verification

[type=verificationFail]
This should be the first verification fail message

[type=verification]
This should be a second verification

[type=verificationFail]
This should be a second verification fail message

Ensure the rest of the walkthrough verifications show and work as expected

@david-martin Would you mind taking a look?
Here's a link to the Jira issue: https://issues.jboss.org/browse/INTLY-107


Port Projecttype to netstandard

See for ref. https://oren.codes/2016/06/23/portable-is-dead-long-live-netstandard/

Blocked by https://github.com/PrismLibrary/Prism/issues/683

Voting too.

I changed the Prism.Validation PCL to .Net Standard 1.4 by using "imports": "portable45-net45+win8+wp8+wpa81" in the project.json file.
I don't know if this workaround is acceptable, because it seems not very user friendly.
Currently a user which wants to use the nuget package Prism.Validation has to do the following steps:

Create a PCL which targets .Net Standard 1.4
Make sure your NETStandard.Libary is up to date
Open project.json and replace

  "frameworks": {
    "netstandard1.4": { }
  }

with
 "frameworks": {
   "netstandard1.4": {
     "imports": "portable45-net45+win8+wp8+wpa81"
   }
 }

Install Nuget Package Prism.Validation
Make sure your WPF Project uses at least .Net Framework 4.6.1
Install Nuget Package Prism.Validation
Reference the PCL Project in the WPF Project

I'm not sure if I should publish the current version as nuget package.


PGPSignature verification is not thread-safe

A user recently noticed, that some PGP operations (decryption of a message in this case) would sometimes fail randomly.
It turns out, their code was multi-threaded, and PGPainless would prior to message decryption check binding signatures on the encryption subkey.
This caused the following error in BCs PGPSignature class:
java.lang.ArrayIndexOutOfBoundsException: length=8; index=8
at org.bouncycastle.crypto.digests.LongDigest.update(Unknown Source:8)
at org.bouncycastle.crypto.digests.LongDigest.update(Unknown Source:8)
at org.bouncycastle.openpgp.operator.bc.BcImplProvider$EdDsaSigner.update(Unknown Source:2)
at org.bouncycastle.openpgp.operator.bc.SignerOutputStream.write(Unknown Source:2)
at org.bouncycastle.openpgp.PGPSignature.blockUpdate(Unknown Source:2)
at org.bouncycastle.openpgp.PGPSignature.update(Unknown Source:16)
at org.bouncycastle.openpgp.PGPSignature.update(Unknown Source:2)
at org.bouncycastle.openpgp.PGPSignature.updateWithPublicKey(Unknown Source:21)
at org.bouncycastle.openpgp.PGPSignature.verifyCertification(Unknown Source:4)

After some digging, we figured out, that this is probably caused by multiple threads working on the same PGPPublicKeyRing instance (and therefore on the same PGPSignature instance, hence the same SignerOutputStream using the same EdDsaSigner, using the same LongDigest).
Upon further inspection, it turns out that PGPSignature is not designed to be used in a multi-threaded environment. The SignerOutputStream is a member of the class, so parallel verification would probably write garbage to that stream, resulting in verification fails.
Would you consider a patch that refactors the PGPSignature class in such a way that the stateful verification code is moved into a separate class, even though such a patch would break the API backwards compatibility?
I could imagine something along these lines:
PGPSignature signature = ...;
SignatureVerifier verifier = signature.init(verifierBuilderProvider, pubKey); // stateful stuff is moved here
boolean valid = verifier.verifyCertification(masterKey, pubKey);

I think this would allow multiple threads to work on the same PGPSignature instance without interfering (since each thread would have its own copy of the SignatureVerifier class).
Now that I think about it, if instead of refactoring the init method, we add a separate method (betterInit?), we can even change the behavior without breaking backwards compat.
What do you think?
Related resources:
original bug report
reproducer for PGPainless
ported reproducer for BC

I prototyped an implementation of the discussed patch here.
As you can see, the patch would be backwards compatible, but would enable thread-safe verification of signatures.


Automatically  published Bookmarks will not return in a search from SharePoint.

According to the product group, bookmarks must be manually published ("editorial")  to show in SharePoint searches. Auto-published ("mined") bookmarks for SharePoint sites will show in Bing work search, but not SharePoint search. (IcM at https://portal.microsofticm.com/imp/v3/incidents/details/294631573/home.)
This should be added to the section "Publish or review recommended bookmarks" but I'm not sure how to word it.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 0c0da86d-a561-e5d5-15ff-c0863de2a58b
Version Independent ID: 66ecc43a-029f-70f7-62e9-3e7b77adc5d9
Content: Manage bookmarks
Content Source: MicrosoftSearch/manage-bookmarks.md
Service: mssearch
GitHub Login: @dawholl
Microsoft Alias: dawholl

@fselkirk added note about availability of both manually and autopublished recommended bookmarks to the bookmarks docs article, Manage bookmarks | Microsoft Docs. You may need to do a hard refresh to see the updated content. Also updated the ICM with the same info. Closing this issue with this comment.

Thank you Dawn! The note was clear, and I was able to send a link to the improved documentation to the customer when closing his case. This addition will prevent a lot of frustration in the future, as more customers begin using these features.
Frances Selkirk
Sr. Support Escalation Engineer
SharePoint and OneDrive Technical Support
+1<PHONE_NUMBER>
Working Hours: M-Th, 9:30-6:00 ET  (Friday hours limited)
Need help outside of my working hours?
Manager: Priyanka Mendiratta<EMAIL_ADDRESS>From: Dawn Hollingsworth @.>
Sent: Wednesday, May 25, 2022 2:15 PM
To: MicrosoftDocs/OfficeDocs-MicrosoftSearch @.>
Cc: Frances Selkirk @.>; Mention @.>
Subject: Re: [MicrosoftDocs/OfficeDocs-MicrosoftSearch] Automatically published Bookmarks will not return in a search from SharePoint. (Issue #260)
@fselkirkhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ffselkirk&data=05|01|fselkirk%40microsoft.com|0c0ccd343b25418fe6d808da3e7a6b24|72f988bf86f141af91ab2d7cd011db47|1|0|637890992751702554|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=lvCEMEBREOgK59GJgIVln2HXysbZ%2FyUk0xg7vFFgPTg%3D&reserved=0 added note about availability of both manually and autopublished recommended bookmarks to the bookmarks docs article, Manage bookmarks | Microsoft Docshttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fmicrosoftsearch%2Fmanage-bookmarks&data=05|01|fselkirk%40microsoft.com|0c0ccd343b25418fe6d808da3e7a6b24|72f988bf86f141af91ab2d7cd011db47|1|0|637890992751702554|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=%2F0FbOMYdYNnCEbvU7WkOCy%2BfAJ1Sai6xIJaN%2By6H2x4%3D&reserved=0. You may need to do a hard refresh to see the updated content. Also updated the ICM with the same info. Closing this issue with this comment.

Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoftDocs%2FOfficeDocs-MicrosoftSearch%2Fissues%2F260%23issuecomment-1137664646&data=05|01|fselkirk%40microsoft.com|0c0ccd343b25418fe6d808da3e7a6b24|72f988bf86f141af91ab2d7cd011db47|1|0|637890992751702554|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=NtralLl7CpBcg9awkOckEZ0mYerZdfFrBUszI5SBWKk%3D&reserved=0, or unsubscribehttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAJFSDWKPSMFOAVN7GRDMRA3VLZUYRANCNFSM5WWROQ6Q&data=05|01|fselkirk%40microsoft.com|0c0ccd343b25418fe6d808da3e7a6b24|72f988bf86f141af91ab2d7cd011db47|1|0|637890992751702554|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=98IFKByu4ll%2B%2Fw%2BfsaQ8SLNv89MDbK6mEScVIifD3YQ%3D&reserved=0.
You are receiving this because you were mentioned.Message ID<EMAIL_ADDRESS>


[ARCTIC-1570] Remove LOG.isDebugEnabled() and use {} instead +

Why are the changes needed?
Fix #1570
Brief change log

Remove LOG.isDebugEnabled()
Use {} instead +

How was this patch tested?

[ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

[ ] Add screenshots for manual tests if appropriate

[x] Run test locally before making a pull request

Documentation

Does this pull request introduces a new feature? (yes / no)
If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

@zhongqishang The code formatting did not pass, could you fix it?