Make it possible to support runtime metrics on a seperate path

@SuperQ
This PR introduces a new flag to make it possible to serve the go runtime metrics on a seperate endpoint. The way we are orchestrating the deployment of the exporter (multiple k8s deployments), this flag will effectively halve our stackdriver API calls.
Manual testing done:

With and without flag.
Launch process locally with one of our GCP projects.
Verify endpoint(s) contain what they should.

Interesting idea. What specifically about your deployments will reduce the number of calls?

Interesting idea. What specifically about your deployments will reduce the number of calls?

@SuperQ we intend to orchestrate our scrapers to have a seperate scrape job for the runtime metrics -- possibly with a tighter interval.

I wonder if we should have this flag do the opposite. Have web.telemetry-path be where the runtime metrics live. But move allow moving the stackdriver metrics to a new path via a separate registry. For example --web.stackdriver-telemetry-path=/stackdriver.
I'm suggesting this because we're likely to move the web.telemetry-path feature to the exporter-toolkit. It would be easier to move the exported metrics, rather than the standard telemetry metrics. Similar to how some exporters have /scrape endpoints for the multi-target exporter pattern.

@SuperQ yes changing the polarity of the flag would be the same for us. I'll update the PR.

We explored using query params to provide the filters and this might allow us to have a single deployment. The stackdriver API call metrics are specific to the registry created for the exposition and they reset on each call with the approach (IIRC). I think we would want those metrics to probably be cumulative/global (or with a  use-case label --e.g., pubsub) and available with the runtime metrics. I hope this will be possible in the future.


Screen capture fix

This should fix the Screen Capture bug.
Unfortunately I seem to have messed up the branch a bit so I cannot merge. (Need to learn Git properly)
The changes in the Screen Caputre Fix should fix it.
The problem was the memory was being de-allocated even though the ownership of the memory was transferred to the Buffer object. By adding the callback, which will be called when the object is Garbage collected, there wont be any memory leaks.

Wow, fantastic work! Thank you so much. I'll get to this tomorrow, I'm very excited about getting this in RobotJS.

Got it merged! Do we need to support Node.js 0.8? That build failed for Windows, but the rest are fine:
https://ci.appveyor.com/project/octalmage/robotjs/build/81

Here's an idea for the build matrix:
https://gist.github.com/octalmage/326fffd925f8e517a961

We don't need to support 0.8, just 0.12 which Node.js is maintaining until the end of 2016. So I think we're good here. I also want to support 0.10 since nw.js is maintaining a branch that uses 0.10.
I'll work on getting this released!


Token Action HUD compatibility broken

Token Action HUD recently had a 2.0 release. The HUD now won't render and I see the following error repeatedly:
foundry.js:655  Error: Error thrown in hooked function '' for hook 'renderTokenActionHud'. Cannot read properties of undefined (reading 'dataset')
[Detected 1 package: pf2e-dorako-ui(3.7.6)]
    at Hooks.onError (foundry.js:654:24)
    at 🎁Hooks.onError#0 (libWrapper-wrapper.js:188:54)
    at #call (foundry.js:636:36)
    at Hooks.callAll (foundry.js:589:17)
    at #doEvent (foundry-esm.js:44921:17)
    at #render (foundry-esm.js:44366:20)
    at async Semaphore._try (foundry-esm.js:6645:19)Caused by: TypeError: Cannot read properties of undefined (reading 'dataset')
[Detected 1 package: pf2e-dorako-ui(3.7.6)]
    at Object.fn (ui-theme.js:84:20)
    at #call (foundry.js:632:20)
    at Hooks.callAll (foundry.js:589:17)
    at #doEvent (foundry-esm.js:44921:17)
    at #render (foundry-esm.js:44366:20)
    at async Semaphore._try (foundry-esm.js:6645:19)

Disabling Dorako UI ( :( ) makes the HUD work again.
Not sure if the error stems from Dorako or TAHUD but I'm reporting on both since it's the stack trace seems to indicate TAHUD but the error literally specifies dorako :/

Re-enabled Dorako (my UI looks terrible without it :( ) and now the token action HUD is rendering, and working but I still have the same error.
Hadn't changed any settings at that point, maybe re-enabling triggers some process?

That error basically means that the hook triggered, but when it reached the point where it tried to add the data theme attribute to the application, there was no element to add it to.
This should never actually stop TAH from rendering, it just didn't get to add Dorako UI styling on top.
It's possible that TAH has significantly changed its guts such that I can no longer assume that when the hook fires there's an element to style. Has TAH changed to app v2?

Aye, I think it was V2.0.6 IIRC.

Had a session:
None of my players had the TAH visible. After the sesh I asked a player to help test this.

he had a different error to the one I showed you, something to do with deconstructing an object. Didn't think to ask for a copy paste, sorry.
He then updated the Interface Theme from No Theme to Foundry 2 and the error changed to the one I showed you, but TAH now renders.
Changing the interface theme to No Theme makes the error go away entirely.

That's all we could work out, hope it helps.


Doc issue about image pull error?

https://github.com/andyzhangx/demo/blob/master/issues/acr-issues.md#2-image-pull-error-from-acr-anonymous-repository said that image pull error for public ACR was fixed in v1.13.4. That looks not right -- https://github.com/kubernetes/kubernetes/blob/v1.13.4/pkg/credentialprovider/azure/azure_credentials.go doesn't include the fix. Instead, it is v1.13.5 including the fix: https://github.com/kubernetes/kubernetes/blob/v1.13.5/pkg/credentialprovider/azure/azure_credentials.go.

correct, I have changed the fixed version from 1.13.4 to 1.13.5


Runtime v0.2.0 and field rename

Part of https://github.com/aws-controllers-k8s/community/issues/790
Description of changes:

Regenerate controller with ack-generate v0.2.0
Updated go.mod to use runtime v0.2.0
Updated generator.yaml to rename RepositoryName field to Name
Executed go mod tidy

New generator.yaml:
resources:
  Repository:
    renames:
      operations:
        CreateRepository:
          input_fields:
            RepositoryName: Name
        DeleteRepository:
          input_fields:
            RepositoryName: Name
        DescribeRepositories:
          input_fields:
            RepositoryName: Name
    exceptions:
      errors:
        404:
          code: RepositoryNotFoundException
    list_operation:
      match_fields:
        - Name
    update_operation:
      custom_method_name: customUpdateRepository

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

/lgtm


App crashes

App crashes with "Memory Stream is not expandable" when i try to extract one particullar file.
problematic file.zip

For some reason expected decompression size of asset bundle doesn't match to actual one. Maybe it is a result of custom compression or something like that. If you don't know how to reproduce it in unity, I consider it as unsupported file format.


💚 fix query benchmark for native aot benchmark

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 68.99%. Comparing base (2b25d6d) to head (c8dbb1b).

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##             main      #82      +/-   ##
==========================================
- Coverage   69.11%   68.99%   -0.12%
==========================================
  Files         110      110
  Lines        4419     4419
  Branches      433      433
==========================================
- Hits         3054     3049       -5
- Misses       1147     1151       +4
- Partials      218      219       +1

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Upgrade eslint to 0.12.0

Upgrades eslint to 0.12.0, which introduces support for ES6 and JSX.

Closing in favor of https://github.com/makepanic/broccoli-eslint/pull/4


Newfeatures

hey qoosim,
that was for javascript and I don't have any javascript thats why it
failed. it is only css and html I cant change anything
On Fri, Aug 19, 2022 at 1:48 PM Qoosim AbdulGhaniyy <
@.***> wrote:

@.**** requested changes on this pull request.
Hi @david63011 https://github.com/david63011 ,
While you made a great effort in this project, unfortunately, I cannot
proceed to review your code.
Invalid Code Review Request
You have submitted a project, that

has linter error.

[image: linter-error]
https://user-images.githubusercontent.com/26597432/185677677-72dcbd31-9b1d-41a9-bb7a-572ee917e34e.png

Kindly fix the error and submit for another review

Your Code Review Request will be marked as invalid in your Dashboard, so
please submit a new one once you are ready 🙏
Cheers and Happy coding!👏👏👏
Feel free to leave any questions or comments in the PR thread if something
is not 100% clear.
Please, remember to tag me in your question so I can receive the
notification.
Please, do not open a new Pull Request for re-reviews. You should use the
same Pull Request submitted for the first review, either valid or invalid
unless it is requested otherwise.
Invalid Code Review Request does not count into the code reviews limit.
—
Reply to this email directly, view it on GitHub
https://github.com/david63011/Pro-website/pull/3#pullrequestreview-1079178485,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AYBOZP53E74DVO5HNELWONTVZ7CHNANCNFSM57BKIUIA
.
You are receiving this because you were mentioned.Message ID:
@.***>

there wasnt an error I can change
On Fri, Aug 19, 2022 at 2:09 PM David underwood @.***> wrote:

hey qoosim,
that was for javascript and I don't have any javascript thats why it
failed. it is only css and html I cant change anything
On Fri, Aug 19, 2022 at 1:48 PM Qoosim AbdulGhaniyy <
@.***> wrote:

@.**** requested changes on this pull request.
Hi @david63011 https://github.com/david63011 ,
While you made a great effort in this project, unfortunately, I cannot
proceed to review your code.
Invalid Code Review Request
You have submitted a project, that

has linter error.

[image: linter-error]
https://user-images.githubusercontent.com/26597432/185677677-72dcbd31-9b1d-41a9-bb7a-572ee917e34e.png

Kindly fix the error and submit for another review

Your Code Review Request will be marked as invalid in your Dashboard, so
please submit a new one once you are ready 🙏
Cheers and Happy coding!👏👏👏
Feel free to leave any questions or comments in the PR thread if
something is not 100% clear.
Please, remember to tag me in your question so I can receive the
notification.
Please, do not open a new Pull Request for re-reviews. You should use
the same Pull Request submitted for the first review, either valid or
invalid unless it is requested otherwise.
Invalid Code Review Request does not count into the code reviews limit.
—
Reply to this email directly, view it on GitHub
https://github.com/david63011/Pro-website/pull/3#pullrequestreview-1079178485,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AYBOZP53E74DVO5HNELWONTVZ7CHNANCNFSM57BKIUIA
.
You are receiving this because you were mentioned.Message ID:
@.***>


why I got "sync database struct error: Default addr for network 'localhost' unknown"

I set the config file
HOST = localhost
USER = root
PASSWD = XXXX
but,I got "sync database struct error: Default addr for network 'localhost' unknown",what is the problem?
Thanks!

for me
    db, err:= sql.Open("mysql", "user:pass@tcp(localhost:3306)/scf")

tcp() works (without I have the same error)

@exu I guess it depends on how localhost is defined in hosts file?

@Unknwon my is <IP_ADDRESS>   localhost  (arch linux) so I think it's standard definition

@exu sorry I get confused, :sob:   so do you have issue? Or just join the discussion?

@Unknwon only join discussion

@exu I see. Cool, so exceptions happen, maybe somehow his localhost is modified or MySQL listens on non-default address.


[MXNET-620]Fix flaky test batchnorm training

Description
Fix issue #8044
tracked at: https://issues.apache.org/jira/browse/MXNET-620
Solution:
Bump up atol value from 1e-4 to 1e-2
The following 3 tests related to batchnorm training can all pass  10000 runs:
tests/python/unittest/test_operator.py:test_batchnorm_training
tests/python/gpu/test_operator_gpu.py:test_batchnorm_training
tests/python/mkl/test_mkldnn.py:test_batchnorm

Finding 1:
Using fp32 with atol=1e-2 or fp64 with atol=1e-3 both works, chose the former one.
Tried different numeric_eps=1e-3, didn't work for all cases
Finding 2:
Reason of failing due to batchnorm result slightly different than result calculated by numeric gradient. Current implementation of batchnorm should be correct.
Finding 3:
Verified  the gradient of 3 different batchnorm implementation are close but they are all different than numeric gradient. (CPU, GPU without cuDNN, GPU with cuDNN)
Conclusion

Current batchnorm implementation is correct and result is consistent across all implementations (mkldnn, w/ cudnn, w/o cuddn)
The batchnorm gradient is slightly different than numeric gradient calculated using finite differential method. Difference is less than rtol=0.16. atol=1e-2, it's large comparing to other operators, but acceptable in this case due to large precision loss in batchnorm.
Enable this test with larger atol is better than just disable it. (it could caught #11448 if enabled), it serves more like a sanity check to prevent the gradient results are far too off from numeric calculations.
Batchnorm errors can also be caught in test_batchnorm_with_type where consistency between each implementation is compared.

See comments in #8044 for detailed results.
Checklist
Essentials
Please feel free to remove inapplicable items for your PR.

[x] The PR title starts with MXNET-620created (except PRs with tiny changes)
[x] Changes are complete (i.e. I finished coding on this PR)
[x] To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

Comments

If this change is a backward incompatible change, why must this change be made.
Interesting edge cases to note here

@haojin2 @eric-haibin-lin  could you help take a look? Thanks

@rahul003

"Enable this test with larger atol is better than just disable it." - Sure. The goal now is to do it properly, dive deep and track down the root causes and possible implications.

@marcoabreu if you take a closer look at "finding 3", we do deliver the same results as the CuDNN version, so the root cause is just that the numeric gradient having a larger error margin compared to the symbolic backward one.

Updated conclusion.
Removed 'row_sparse' from stypes for the following reasons:

It's failing at testing varying channel axis , this test already has rtol=0.2 and atol=0.01, we cant further increase rtol and atol.
row_sparse is never used and always fallback in this test, the logic is also tested tested at mkl/test_mkldnn.py

@zheng-da  Hi, I have modified your unit test for mkldnn batchnorm, increased atol to 1e-2 as its flaky at 1e-3 and 1e-4.  Reasons listed in PR description, all batchnorm implementation (mkldnn, cudnn) gradient is slightly off from numeric gradient.  Using 1e-2 is more stable and passed 10000 runs.

@roywei thanks for fixing the flaky test.

I don't really get the statement about row_sparse being removed because of the mkldnn test. Considering we run without mkldnn, how is that related?
I might be missing something here, but I got the feeling that row_sparse is being removed because it's causing trouble in some environment configurations. Could you please elaborate here?

@marcoabreu We don't have a batch norm implemented for sparse arrays, so 'row_sparse' was there to test if we can fallback correctly. The storage type inference logic for sparse ops share some code paths with the dispatch inference logic for MKLDNN operators, which is actually what we want to test, so that's why it's now moved there.

I see, that makes sense. Where does it fall back to if MKLDNN is not available?
My concern here is that this fallback actually seems to be failing in some cases. This shouldn't happen, right? To me, it seems like we're now removing this problematic test although the problem might lie in whatever algorithm implementation we use in that fallback - and the test actually serving its purpose to reveal that problem.

@marcoabreu You are right, and when row_sparse is not available, it should fallback to default mxnet implementation, it could be mkldnn or not. However, for mkldnn enabled operators, they changed the fallback logic and introduced a bug tracked at #11448.  Thus it make sense to moved the fallback check under mkl/test_mkldnn.py, and for each operator impacted, there should be a similar unit test.
@haojin2  is double checking if the fallback values and logic is fixed and correct.
We will see if we need additional tests for fallback logic and value checks .

I think we should leave the tests as they are and there can be additional specific tests for the mkldnn backend.
Our users should not have to care what backend is being selected and the behaviour should always be identical. This means that however I call a certain operator, row_sparse in this case, I would expect that I get a proper result in any case. It's great that the issue is already tracked, but I'm heavily opposed to removing that operator from our test suite. All operator tests have to be implementation independent and should not differentiate or exclude certain cases. From my point of view, this flaky test is only resolved when the mkldnn implementation has been fixed and when this test passes all the time. Removing it is not an option.

@marcoabreu I think we can merge this first since I've identified the root cause of the inconsistencies between sparse and dense matrices, and will be submitting a fix with corresponding tests soon.

Well the problem is not really solved but the disabling of a certain test has just been narrowed down - instead of disabling the entire suite, this PR now disables a single sub-test.
I don't really feel comfortable merging as-is because the actual root cause is still the row_sparse and that one has not been solved yet.

@marcoabreu I just submitted a fix for that, and I need the fix for the flakiness of the existing test in this PR in that PR.

I just edited by post, please check the last section

Excellent. Let's submit this PR with the test disabled and you'll take it from there in your PR. How does that sound?

Very good point about the sparse operator tests, I wasn't aware of them. Thank you!
I will make a last pass and merge if everything is as expected.

Thanks @roywei and @haojin2 for your work on this PR and the detailed explanations!

@marcoabreu Thanks for understanding! Please also take a look at my PR when you have time to do so, thanks! @roywei Thanks for your work on this complicated issue!


RGI04 (Arctic Canada, South) full update

Where is it? Philipp and Frank worked on it but I can't seem to find it yet

Trello card: https://trello.com/c/Pzpzapbb

Done.


Decouple service from global state

This PR effectively refactor the service to stop depending on global state, which allows us to start testing the service in it's component pieces. Quite a lot of the codebase depended on the global config, which means we'd have to mock it every time in order to run tests. This PR fully separates out the code from the config which will allow us to start testing,
Review Guide
I'm afraid this is fairly wide ranging, so I'll try to explain my changes:

The index.ts file now creates a class ConferenceBot, which retains the true global state of everything.
We now have a ConferenceMatrixClient that extends MatrixClient with a prepared Identity Client to allow early failure.
The config is no longer global, but passed between the major classes.
The command handlers are now constructed with their dependencies, rather than passed them via doAction for a cleaner API.
There is now the beginnings of a test harness that can create a homeserver on homerunner and run through some tests for us.
Conference bot does a best effort to clean up on closure, rather than exiting immediately.

I think this is probably ready for a first pass review.

The config is no longer global, but passed between the major classes.

thank you so much, the global config bothered me a lot but I didn't know if it was just conventional in node apps or so (I'm not a JS/TS developer by trade)


Update net.py

Hey man - just updating for Canada.  This is my first time being useful in life, please let me have it.

Thanks for the pull request (and sorry for the late response!)
Was there any particular reason you'd added this one to the list? I just tested it locally on my version, and the IANA server gave back the nic.cira.ca server correctly when querying for nic.ca.
The list of TLDs in net.py is only really intended as a fallback for TLDs that IANA doesn't know about - that way, python-whois won't break if common TLDs change their WHOIS server address.


Missing pushdowns for pyarrow datasets

Some predicates are not pushed down to pyarrow datasets correctly. I'm guessing it's just not implemented but I couldn't find any issue tracking what is and what isn't implemented.
This example is a bit contrived but the point is that in both the integer and timestamp case datafusion should push down the filter and thus match no fragments, read no files and return an empty result set. The missing file trick is just a way to verify that no files are being read.
from datetime import datetime, timezone
import shutil

import datafusion
import pyarrow
import pyarrow.compute
import pyarrow.dataset
import pyarrow.fs

shutil.rmtree('data', ignore_errors=True)  # proof that there's no data in the data directory

format = pyarrow.dataset.ParquetFileFormat()
filesystem = pyarrow.fs.SubTreeFileSystem('data', pyarrow.fs.LocalFileSystem())
fragments = [
    # note that the partition_expression is totally wrong
    format.make_fragment(
        '1.parquet',
        filesystem=filesystem,
        partition_expression=(pyarrow.dataset.field('a') <= pyarrow.compute.scalar(1)),
    )
]

dataset = pyarrow.dataset.FileSystemDataset(
    fragments, pyarrow.schema([pyarrow.field('a', pyarrow.int64())]), format, filesystem
)

fragments = list(dataset.get_fragments(pyarrow.dataset.field('a') == pyarrow.scalar(2)))
assert fragments == []

ctx = datafusion.SessionContext()
ctx.register_dataset('dataset', dataset)

df = ctx.sql('SELECT * FROM dataset WHERE a = 2')
assert df.collect() == [], fragments

format = pyarrow.dataset.ParquetFileFormat()
filesystem = pyarrow.fs.SubTreeFileSystem('data', pyarrow.fs.LocalFileSystem())
fragments = [
    # note that the partition_expression is totally wrong
    format.make_fragment(
        '1.parquet',
        filesystem=filesystem,
        partition_expression=(
            pyarrow.dataset.field('a') == pyarrow.scalar(datetime(2000, 1, 1, tzinfo=timezone.utc), pyarrow.timestamp('ns', '+00:00'))
        ),
    )
]

dataset = pyarrow.dataset.FileSystemDataset(
    fragments, pyarrow.schema([pyarrow.field('a', pyarrow.timestamp('ns', '+00:00'))]), format, filesystem
)

fragments = list(
    dataset.get_fragments(
        pyarrow.dataset.field('a')
        == pyarrow.scalar(datetime(2024, 1, 1, tzinfo=timezone.utc), pyarrow.timestamp('ns', '+00:00'))
    )
)
assert fragments == [], fragments

ctx = datafusion.SessionContext()
ctx.register_dataset('dataset', dataset)

df = ctx.sql("SELECT * FROM dataset WHERE a = '2024-01-01T00:00:00+00:00'")
# error because it tries to access the file that doesn’t exist
assert df.collect() == []

Hi @adriangb,
Thanks for the issue.
Could you highlight a little more clearly where the behavior in your script deviates from expectations and how you'd expect it to behave?

I would expect that for the timestamp case the result is empty and the file is never accessed, just like the integer case.

Sorry for the delay.
 Hopefully will get this in next release.


Match items on search isn't be update after delete a translation

Deleting a translation that matches the current search does not update the list of matching items

Fix problem : Performing a CRUD action on the list of translations stops the current search.


Please offer additional wholerow styling option

You can format the anchor, its icon and the actual line's text by inserting (a) CSS class(es) into the "icon" property which is transferred with the JSON data to the client.
But while working on a jsTree style which uses the "wholerow"-plugin, I discovered that there's no way to set a row's background-color depending on a given CSS class via the JSON data returned from the server.
It would be very nice to have such a feature. :)

Actually you can:

Supply a li_attr property for each node you want to modify, that property should contain an object, which has a key named class - set that to whatever you need:
{ "id" : 1, "text" : "1", "li_attr" : { "class" : "my-custom-class" } }
Specify CSS to modify the looks of the node:
#tree .my-custom-class > .jstree-wholerow { background:red; }

This should be enough to get you going - you can modify selected & hovered colors, etc.
Best regards,
Ivan

Ivan, you're awesome! Thanks a lot! :+1: :smile:


fix warning

solution by @515orestis https://github.com/minube/geo-timezone/issues/14

Hi,
Thanks @geertijewski for this PR!
with php8 count on uncountable object trigger an error and not a warning
testing var with is_countable before count is now mandatory

Hi @nono303 thanks I was not aware of this. However I already removed the invocation of count() and replaced it with is_array(), so unless I missed something I don't think I need to amend this Pull Request

Hi @geertijewski,
you don't need to amend the PR
is_countable is nicer before count but require >=7.3 https://php.watch/versions/7.3#new-is_countable-function


Getting errors in terminal related to modeline vcs and checker (and related crash?)

What did you expect to happen?
...
No error messages in Messages buffer
I'll add more info when I try again.
What actually happened?
...
Continuous messages being generated, and eventual crash, although crash may not be related
Error during redisplay: (eval (doom-modeline-segment--checker)) signaled (wrong-type-argument plistp doom-modeline-urge$
Error during redisplay: (eval (doom-modeline-segment--vcs)) signaled (wrong-type-argument plistp doom-modeline-info)
Error during redisplay: (eval (doom-modeline-segment--checker)) signaled (wrong-type-argument plistp doom-modeline-urge$
Error during redisplay: (eval (doom-modeline-segment--vcs)) signaled (wrong-type-argument plistp doom-modeline-info)
Error during redisplay: (eval (doom-modeline-segment--checker)) signaled (wrong-type-argument plistp doom-modeline-warn$
Error during redisplay: (eval (doom-modeline-segment--vcs)) signaled (wrong-type-argument plistp doom-modeline-info)
Error during redisplay: (eval (doom-modeline-segment--checker)) signaled (wrong-type-argument plistp doom-modeline-warn$

Additional details:

Include a link to your private config:  git@gitlab.com:asyncmind/doom.git

I cannot reproduce this. Not enoughinformat
In the future, please do not ignore the issue template. It asks you to include steps to, at the very least, include your M-x doom/info (alternatively, the output of ~/.emacs.d/bin/doom info). Please edit your original post or this thread will be automatically deleted after a week of inactivity.
Also, https://gitlab.com/asyncmind/doom does not exist.

SYSTEM  type       gnu/linux
│       config     x86_64-pc-linux-gnu
│       shell      /bin/zsh
│       uname      Linux 5.4.69-1-lts #1 SMP Thu, 01 Oct 2020 20:40:59 +0000 x86_64
│       path       (~/.emacs.d/bin/ ~/.npm/bin ~/.local/bin ~/.emacs.d/bin/ ~/.bin ~/.zsh/bin/ ~/.gem/ruby/1.9.1/bin /usr/sbin/ /sbin/ ~/.npm/bin/ ~/.gem/ruby/2.2.0/bin /usr/local/bin /usr/local/sbin ~/.npm/bin /usr/bin /opt/cuda/bin /usr/lib/jvm/default/bin /usr/bin/site_perl /usr/bin/vendor_perl /usr/bin/core_perl
/var/lib/snapd/snap/bin /usr/lib/emacs/27.1/x86_64-pc-linux-gnu/)
EMACS   dir        ~/.emacs.d/
│       version    27.1
│       build      Aug 29, 2020
│       buildopts  --prefix=/usr --sysconfdir=/etc --libexecdir=/usr/lib --localstatedir=/var --with-x-toolkit=gtk3 --with-xft --with-wide-int --with-modules --with-cairo --with-harfbuzz 'CFLAGS=-march=x86-64 -mtune=generic -O2 -pipe -fno-plt' CPPFLAGS=-D_FORTIFY_SOURCE=2
LDFLAGS=-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now
│       features   XPM JPEG TIFF GIF PNG RSVG CAIRO SOUND GPM DBUS GSETTINGS GLIB NOTIFY INOTIFY ACL GNUTLS LIBXML2 FREETYPE HARFBUZZ M17N_FLT LIBOTF ZLIB TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM MODULES THREADS LIBSYSTEMD JSON PDUMPER LCMS2 GMP
│       traits     (daemon server-running envvar-file)
DOOM    dir        ~/.doom.d/
│       version    2.0.9
│       build      HEAD -> refs/heads/develop 8ba2b9eaf 2020-10-15 18:13:42 -0400
│       elc-files  0
│       modules    (:completion company helm ivy :ui doom doom-dashboard doom-quit hl-todo indent-guides ligatures minimap modeline nav-flash ophints (popup +defaults) unicode vc-gutter vi-tilde-fringe workspaces :editor (evil +everywhere) file-templates fold snippets :emacs dired electric undo vc :term vterm
:checkers syntax spell grammar :tools ansible docker (eval +overlay) gist lookup lsp magit pass tmux :os tty :lang data elixir emacs-lisp erlang json (java +meghanada) javascript markdown nix org plantuml qt rst (ruby +rails) rust scala scheme sh swift web yaml :email notmuch :config (default +bindings +smartparens))
│       packages   ((helm) (helm-rg) (rg) (projectile) (helm-projectile) (lsp-pyright) (pipenv) (gherkin-mode) (xclip))
│       unpin      (n/a)
│       elpa       (xclip s pyvenv projectile popup pkg-info pipenv helm-projectile helm-core helm gherkin-mode f epl dash async)

https://gitlab.com/asyncmind/doom.git

Does not exists or is private

I am also getting at least related errors in terminal emacs as well. When I run emacs as emacsclient -t .zprofile for example, I receive many of these errors:
Error during redisplay: (eval (doom-modeline-segment--checker)) signaled (wrong-type-argument plistp doom-modeline-info) [8 times]
Error during redisplay: (eval (doom-modeline-segment--checker)) signaled (wrong-type-argument plistp doom-modeline-info) [2 times]
evil-line-move: End of bufferError during redisplay: (eval (doom-modeline-segment--checker)) signaled (wrong-type-argument plistp doom-modeline-info)
Error during redisplay: (eval (doom-modeline-segment--checker)) signaled (wrong-type-argument plistp doom-modeline-info)
Error during redisplay: (eval (doom-modeline-segment--checker)) signaled (wrong-type-argument plistp doom-modeline-info) [4 times]
Error during redisplay: (eval (doom-modeline-segment--checker)) signaled (wrong-type-argument plistp doom-modeline-info) [2 times]
Error during redisplay: (eval (doom-modeline-segment--checker)) signaled (wrong-type-argument plistp doom-modeline-info) [2 times]

Here is my doom config: https://gitlab.com/burrhole/doom-config
This is the output of M-x doom/info:
SYSTEM  type       gnu/linux
        config     x86_64-pc-linux-gnu
        shell      /bin/zsh
        uname      Linux 5.9.1-arch1-1 #1 SMP PREEMPT Sat, 17 Oct 2020 13:30:37 +0000 x86_64
        path       (~/.config/emacs/bin/ ~/.config/emacs/bin/ ~/.config/emacs/bin/ ~/.config/emacs/bin/ ~/.config/emacs/bin/ /usr/local/sbin /usr/local/bin /usr/bin ~/.local/share/flatpak/exports/bin /usr/bin/site_perl /usr/bin/vendor_perl /usr/bin/core_perl ~/.local/bin/cron ~/.local/bin/statusbar ~/.local/bin/ ~/.config/emacs/bin /var/lib/flatpak/exports/share ~/.local/share/flatpak/exports/share /usr/lib/emacs/27.1/x86_64-pc-linux-gnu/)
EMACS   dir        ~/.config/emacs/
        version    27.1
        build      Aug 28, 2020
        buildopts  --prefix=/usr --sysconfdir=/etc --libexecdir=/usr/lib --localstatedir=/var --with-x-toolkit=gtk3 --with-xft --with-wide-int --with-modules --with-cairo --with-harfbuzz 'CFLAGS=-march=x86-64 -mtune=generic -O2 -pipe -fno-plt' CPPFLAGS=-D_FORTIFY_SOURCE=2 LDFLAGS=-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now
        features   XPM JPEG TIFF GIF PNG RSVG CAIRO SOUND GPM DBUS GSETTINGS GLIB NOTIFY INOTIFY ACL GNUTLS LIBXML2 FREETYPE HARFBUZZ M17N_FLT LIBOTF ZLIB TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM MODULES THREADS LIBSYSTEMD JSON PDUMPER LCMS2 GMP
        traits     (daemon server-running envvar-file)
DOOM    dir        ~/.config/doom/
        version    2.0.9
        build      HEAD -> develop 3907e75c8 2020-10-21 17:16:59 -0400
        elc-files  0
        modules    (:completion company ivy :ui doom doom-dashboard (emoji +unicode) hl-todo indent-guides ligatures modeline ophints (popup +defaults) treemacs unicode vc-gutter vi-tilde-fringe workspaces :editor (evil +everywhere) file-templates fold snippets word-wrap :emacs dired electric undo vc :term vterm :checkers syntax spell :tools (eval +overlay) lookup magit pdf :os tty :lang emacs-lisp lua org python sh :config literate (default +bindings +smartparens))
        packages   ((elpher) (eterm-256color) (simple-mpc))
        unpin      (n/a)
        elpa       (n/a)

Finally tracked down the issue. As of adff1aa this issue should be resolved. Let me know if that isn't the case and I'll reopen it. Thanks for bringing it to my attention!

Got another error now
my config repo is public now
Starting Emacs daemon.
mapc: Loading file /home/steven/.emacs.d/.local/straight/build/with-editor/with-editor.elc failed to provide feature ‘with-editor’
Package cl is deprecated
When done with this frame, type SPC q f
Loading /home/steven/.emacs.d/.local/cache/recentf...done
condition-case: Error in a Doom startup hook: global-git-commit-mode, (error "Loading file /home/steven/.emacs.d/.local/straight/build/with-editor/with-editor.elc failed to provide feature ‘with-editor’")
Beginning of buffer [2 times]
Mark set


Updated switch particles prefab

Updated SwitchParticles prefab because it was broken. To test the prefab:

Delete SwitchParticles in the Level1.scene inside the player GO and load the prefab there.
This prefab's root node has a script that needs the player GO and all the particles effects (SwitchParticles' children)
Assign the SwitchParticles GO to the player script

It's not 100% as we had it but it is close and now we have a feedback when changing characters (Nil can modify it later to be more accurate)

Task linked: CU-nh9vua Switch particles broken


Tool needed: Matrix notification from crates.io

Describe your problem and idea for the tool
From https://github.com/paritytech/devops/issues/192

It'd be nice to have a "crate published" notification in a Riot-channel whenever a crate owned by paritytech:core-devs gets updated.

How do you solve this problem now?
We don't :(
Who's going to use it?

Yes, quite usefull at least for security audit purposes.
Crates.io ACLs suck, basically anyone can publish anything; so we'd better to at least be aware when this happens.

So this would be at least for the security team, and @dvdplm from the other tooling team actually was the original requestor.
Importance

[x] Would be nice to have it at some point in the future
[ ] Lack of this instrument really prevents some people from working effectively
[ ] If we don't make it by the deadline — we're doomed!

Deadline
None, the devops issue has been open for a long time already :sweat_smile:
Additional info
Potentially could become a part of a multi-purpose Parity matrix bot our bot suite. We have a lot of general ideas for matrix bots but really don't have any current tooling that uses them effectively beyond general monitoring webhooks etc. I can highly recommend using the matrix-nio API as it supports e2ee, and the template is a great place to start: https://github.com/anoadragon453/nio-template

Yan suggests to take a look at https://newreleases.io/ for this as potential way to solve it


References in stateless components don't work (but do in full react)

I encountered this when trying to use drop down menus from http://blueprintjs.com.  Here is a small example:
class Wrapper extends React.Component {
   render() {
       return
<Stateless />
;
  }
}

function setRef(c) {
   console.log("ref from a stateless component", c);
}

const Stateless = (props) =>
Hello, world
;

When rendered, the console log displays in full react but not react-lite.
Looking at the code, I can fix it by doing the following, but I'm not sure if this is the best solution or has any other impacts:

Remove the velem.refs check in initVelem
in attachRef and detachRef move the refs check to the bottom of the method (eg just before they are used).

Thx feedback, I think you are totally right.
And fixed it at this commit, now the version 0.15.28 of react-lite support this pattern.

Thanks!  Works for me now.


Import-export doesn't work with certain people

I've been testing the plugin, and it works perfectly on my computer and different files and with some other people. However, it doesn't work with one of my colleagues. He has the same sketch version that I have, no other plugins and he has tried to reinstall both sketch and your plugin with no success.
He's able to import his own files (test-text-styles-1.json) but when I take his file, do a modification (test-text-styles-2.json), and he tries to import it then it doesn't work.
json.zip

same here, as well as on collegues machine, same version latest everything

This is weird, I'll see what I can do.

we can give you more info or logs if you tell us what to do.


training on groups and values

I am trying to improve the age estimation model by first training different age group models and then combine into 1 final model to predict 101 classes. The code is below:
modelA = load_model('/content/8-base_model_1.h5')
modelA.load_weights('/content/8-weights.002-0.991-0.406.hdf5')
modelB = load_model('/content/30-base_model_1.h5')
modelB.load_weights('/content/30-weights.010-0.499-0.374.hdf5')
modelC = load_model('/content/80-base_model_1.h5')
modelC.load_weights('/content/80-weights.002-1.064-0.305.hdf5')
for layer in modelA.layers:     layer.trainable = False     layer.name = layer.name + str("_1")
for layer in modelB.layers:     layer.trainable = False     layer.name = layer.name + str("_2")
for layer in modelC.layers:     layer.trainable = False     layer.name = layer.name + str("_3")
mod_1 = concatenate([modelA.layers[-1].output,       modelB.layers[-1].output,modelC.layers[-1].output])
mod_1 = Dropout(0.2)(mod_1)
mod_1 = Dense(15, activation='relu')(mod_1)
predictions = Dense(9, activation='softmax', name='pred_age')(mod_1)
top_model = Model(inputs=[modelA.input, modelB.input, modelC.input], outputs=predictions)
opt = get_optimizer('adam', lr_rate)
 top_model.compile(loss=['categorical_crossentropy'],           optimizer=opt,           metrics=['accuracy','mae'])
model.fit_generator(inputgenerator,                            epochs=EPOCHS,                            steps_per_epoch = STEPS_PER_EPOCH,                            validation_data=validgenerator,                            validation_steps = VALIDATION_STEPS,                            verbose=1,                            callbacks=callbacks)
I first trained 3 different models, for example:
model 1 (age group 0-29) = 3 labels (0,1,....,8,9,10....), contains labels from 0 until 29
model 2 (age group 30-79)= 2 labels (30,31.....), contains labels from 30 until 79
model 3 (age group 80-100) = 4 labels (80,81,82,83.....), contains labels from 80 until 100
I would like to combine these 3 models into a final model with 1 output containing the 101 labels (0,1,2, ... ,100). However the model expects 3 inputs instead of 1. For the final model would just like to give 1 image.
The final input needs to get 1 input image instead of 3 images. But I still get stuck. I am building a kind of hierarchy here to improve the accuracy.
Finetune imagenet with age_groups > fine-tune age_groups model with 101 images (combined N models).
I am doing this to increase the age estimation model for older age groups while these are predicted very young for older people. Another question is about the fine-tuning, when i fine-tune a trained model, the weights of the trained model are forgotten, which I do not want it.

Sorry, I did not understand what you want.
You might have to estimate age group (out of three groups) using the other model first.

For example,
I have an image with age label 36. I want to train a hierarchical cnn, where I first predict the age_group and then the apparent age.
image label 36 > layers > softmax classification age_group > softmax classification apparant age.

You might have to estimate age group (out of three groups) using the other model first.

So, you mean first predict the age group model. And for example if age group is 30-40, then select the model of (30-40) and predict ages between 30-40. right?

So, you mean first predict the age group model. And for example if age group is 30-40, then select the model of (30-40) and predict ages between 30-40. right?

Yes.
There is no need to put all functions into a single model.
(Anyway, I'm not sure this approach can improve the accuracy)


Chemical Trend Tool - Displaying same number for Items and Cost

From doorbell

the item value and the spend value seems to be the same figure when I search using the Trend tool. Could you tell me if this is an issue at my end or with the tool please?

This does appear to be the case.
For example on Lactulose the items in July = 238 440 and the spend = £238 440.
The same issue is happening on the BNF sections trend tool.
See Proton Pump Inhibitors page

The API is returning something sensible, eg https://openprescribing.net/api/1.0/spending/?format=json&code=0101010C0.
So I suppose the problem is with the JS.

I can't reproduce this locally.

I can't spend any more time on this right now.

I can reproduce this locally -- I hadn't rebuilt the JS.

This was introduced in e1890111e0cf8315d7b4a43733ff0af40c9313cb (Upgrade to Highcharts 6.1.3 ).  Going by the timestamps on the filesystem on largeweb2, this has been broken since 2018-09-27.
I will spend a little bit of time investigating, but may need to wait for somebody who knows what they're doing with JS.

When you change from viewing items to viewing spending, barChart.updateChart gets called as expected, which calls barChart.getYValueOfData, which returns its input, updated with the y property set correctly.  This all seems to work correctly.
But then either chart.series[0].setData(newData, false) doesn't update the chart's data, or chart.redraw() doesn't redraw the chart.
I don't have any more time today.  @evansd can you take a look tomorrow?

@inglesp I certainly can. It looks like you've done 80% of the hard work already. Thanks!


ScriptManager.shared.addResolver() to cause "Property 'webpack_require' doesn't exist"

Environment
Macbook M2 Max - MacOS Ventura 13.3 (and Macbook 2016 - MacOS Monrerey 12.6.3)
Metro v0.73.9
"react": "18.2.0",
"react-native": "0.71.7"
"@callstack/repack": "^3.1.1",
"webpack": "^5.80.0"
Description
ScriptManager.shared.addResolver() to cause "Property 'webpack_require' doesn't exist"
Reproducible Demo
1.) plugin terminal: npx react-native@latest init plugin ; cd plugin
2.) host terminal: npx react-native@latest init host ; cd host
3.) plugin and host terminal:
npm i -D webpack terser-webpack-plugin babel-loader @callstack/repack
react-native.config.js add:
module.exports = {             commands: require('@callstack/repack/commands')         }
webpack.config.mjs: from https://github.com/callstack/repack/blob/main/templates/webpack.config.mjs
android/app/build.gradle add:
react {             bundleCommand = "webpack-bundle"
babel.config.js add:
comments: true,
4.) plugin project: webpack.config.mjs add:
  plugins: [
      new Repack.plugins.ModuleFederationPlugin({
        name: 'plugin',
        shared: {
          react: {
            ...Repack.Federated.SHARED_REACT,
            eager: false,
          },
          'react-native': {
            ...Repack.Federated.SHARED_REACT_NATIVE,
            eager: false,
            requiredVersion: '0.68.2',
          },
        },
      }),

5.) plugin terminal: npx react-native webpack-bundle --platform android --entry-file ./index.js --bundle-output ../plugin.bundle --config ./webpack.config.mjs
6.) plugin: generated bundle insert to production web server
7.) host project: webpack.config.mjs:
  plugins: [
      new Repack.plugins.ModuleFederationPlugin({
        name: 'host',
        // remotes: {
        //  plugin: 'plugin@dynamic',
        // },
        shared: {
          react: {
            ...Repack.Federated.SHARED_REACT,
            requiredVersion: '17.0.2',
          },
          'react-native': {
            ...Repack.Federated.SHARED_REACT_NATIVE,
            requiredVersion: '0.68.2',
          },
        },
      }),

8.) host project: inde.js add before AppRegistry:
	import { ScriptManager, Script, Federated } from '@callstack/repack/client';

	const resolveURL = Federated.createURLResolver({
	    containers: {
                plugin: 'http://....my production server.../plugin.bundle',
	    },
	});

	ScriptManager.shared.addResolver(async (scriptId, caller) => {
	    let url;
	    if (caller === 'main') url = Script.getDevServerURL(scriptId);
	    else url = resolveURL(scriptId, caller);

	    if (!url) return undefined;

	    return {
	        url,
	        cache: false, // For development
	        query: {
	            platform: Platform.OS,
	        },
	    };
	});

9.) host terminal: npx react-native webpack-start
10.) react native terminal: npx react-native start a
retult:
BUILD SUCCESSFUL in 2s
57 actionable tasks: 2 executed, 55 up-to-date
info Connecting to the development server...
info Starting the app on "emulator-5554"...
Starting: Intent { cmp=com.host/.MainActivity }
 BUNDLE  ./index.js

 ERROR  ReferenceError: Property '__webpack_require__' doesn't exist, js engine: hermes
 LOG  Running "host" with {"rootTag":11}
 ERROR  Invariant Violation: "host" has not been registered. This can happen if:
* Metro (the local dev server) is run from the wrong folder. Check if Metro is running, stop it and restart it in the current project.
* A module failed to load due to an error and `AppRegistry.registerComponent` wasn't called., js engine: hermes

Source files:
https://uloz.to/file/Iayw2zOqV00W/module-federation-bug-zip#!ZGN4ZwR2AJVjMQWzZJLmLwH3AwL1Z1pjZSMlMaqDYJIeHGEzAt==
NOTE:

It is not clear from your documentation where to place AAA Repack.plugins.ModuleFederationPlugin. I had to look at an example from: https://github.com/callstack/repack-examples/tree/main/module-federation
Alternative without ScriptManager.shared.addResolver() not working:

      new Repack.plugins.ModuleFederationPlugin({
        name: 'host',
        remotes: {
          plugin: 'plugin@http://....my production server.../plugin.bundle',
        },
        shared: {
          react: {
            ...Repack.Federated.SHARED_REACT,
            requiredVersion: '17.0.2',
          },
          'react-native': {
            ...Repack.Federated.SHARED_REACT_NATIVE,
            requiredVersion: '0.68.2',
          },
        },
      }),

Hey @ObscurusGrassator, There a limitation in Repack: A Host App cannot use remotes to consume other containers (plugin in your case), for more info please refer to this part of the docs: https://re-pack.netlify.app/docs/module-federation#host-application-cant-use-remotes

I have already read the whole web page 5x (https://re-pack.netlify.app/docs/module-federation) and I still don't understand how to connect it all together.
I need my app to load dynamically remote modules. Is it possible through Re.pack to achieve? If yes, what do I have to change?
If a host app cannot use remotes to consume other containers, so what is the meaning of Module Federation? And how else can plugin containers can be loaded to host app?
In this example:
https://github.com/callstack/repack-examples/blob/main/module-federation/host/index.tsx
are used external urls in host app.

@ObscurusGrassator we recently released a more comprehensive example of how you can setup module federation for, what we call, a super app: https://github.com/callstack/super-app-showcase. Please check how this works for you

Hi @thymikee, I install super-app-showcase, but I guess I'm doing something wrong.
MacBook-Pro:super-app-showcase xxx$ yarn install
➤ YN0009: │ shell@workspace:packages/shell couldn't be built successfully (exit code 127, logs can be found here: /private/var/folders/zd/fmk31xcn1q3b09jlpc14k3g00000gn/T/xfs-89759b04/build.log)
➤ YN0009: │ host@workspace:packages/host couldn't be built successfully (exit code 127, logs can be found here: /private/var/folders/zd/fmk31xcn1q3b09jlpc14k3g00000gn/T/xfs-49105698/build.log)
➤ YN0009: │ dashboard@workspace:packages/dashboard couldn't be built successfully (exit code 127, logs can be found here: /private/var/folders/zd/fmk31xcn1q3b09jlpc14k3g00000gn/T/xfs-41dca92d/build.log)
➤ YN0000: └ Completed in 21s 930ms
➤ YN0000: Failed with errors in 56s 788ms

cat /private/var/folders/zd/fmk31xcn1q3b09jlpc14k3g00000gn/T/xfs-89759b04/build.log
# This file contains the result of Yarn building a package (shell@workspace:packages/shell)
# Script name: postinstall

yarn start
[catalog] [CatalogServer] Server listening at port 3000
[host] ℹ [07:56:05.303Z][DevServer] Server listening at http://[::1]:8081
[booking] ℹ [07:56:05.312Z][DevServer] Server listening at http://[::1]:9000
[shopping] ℹ [07:56:05.564Z][DevServer] Server listening at http://[::1]:9001
[auth] ℹ [07:56:05.836Z][DevServer] Server listening at http://[::1]:9003
[dashboard] ℹ [07:56:05.980Z][DevServer] Server listening at http://[::1]:9002
[host] ℹ [07:56:07.043Z][DevServer] Hermes device connected { deviceId: 0 }
[host] ℹ [07:56:07.045Z][DevServer] Hermes device connected { deviceId: 1 }

yarn run:shell:android
[host] ⚠ [07:33:10.134Z][LoggerPlugin] Warning in<EMAIL_ADDRESS>Module not found: Error: Can't resolve '@react-native-masked-view/masked-view' in '/Users/xxx/Desktop/super-app-showcase/packages/host/node_modules/@react-navigation/elements/src'
[host] ℹ [07:33:10.558Z][Console] Running "shell" with {"rootTag":11}
[host] ✖ [07:33:10.558Z][Console] Invariant Violation: "shell" has not been registered. This can happen if:
[host] * Metro (the local dev server) is run from the wrong folder. Check if Metro is running, stop it and restart it in the current project.
[host] * A module failed to load due to an error and `AppRegistry.registerComponent` wasn't called., js engine: hermes

yarn run:host:android
[host] ✖ [07:56:31.091Z][Console] [ScriptManager] Failed while resolving script locator: {"caller": undefined, "scriptId": "auth"} {"originalError": [TypeError: Network request failed]}
[host] ✖ [07:56:31.134Z][Console] Error: Unhandled error. ([ScriptManager] Failed while resolving script locator:)
[host]
[host] This error is located at:
[host]     in Lazy (created by App)
[host]     in Suspense (created by App)
[host]     in ErrorBoundary (created by App)
[host]     in App
[host]     in RCTView (created by View)
[host]     in View (created by AppContainer)
[host]     in RCTView (created by View)
[host]     in View (created by AppContainer)
[host]     in AppContainer
[host]     in host(RootComponent), js engine: hermes
[host] ℹ [07:56:31.135Z][Console] [Error: Unhandled error. ([ScriptManager] Failed while resolving script locator:)] {"componentStack": "

Or NPM:
MacBook-Pro:shell xxx$ npm i
npm ERR! code ERESOLVE
npm ERR! ERESOLVE unable to resolve dependency tree
npm ERR!
npm ERR! While resolving<EMAIL_ADDRESS>npm ERR! Found<EMAIL_ADDRESS>npm ERR! node_modules/react-native-paper
npm ERR<EMAIL_ADDRESS>from<EMAIL_ADDRESS>npm ERR!   packages/shell
npm ERR<EMAIL_ADDRESS>npm ERR!     node_modules/shell
npm ERR!       workspace packages/shell from the root project
npm ERR!
npm ERR! Could not resolve dependency:
npm ERR! peer react-native-paper@">= 3.0.0" from<EMAIL_ADDRESS>npm ERR! node_modules/@react-navigation/material-bottom-tabs
npm ERR<EMAIL_ADDRESS>from<EMAIL_ADDRESS>npm ERR!   packages/shell
npm ERR<EMAIL_ADDRESS>npm ERR!     node_modules/shell
npm ERR!       workspace packages/shell from the root project

MacBook-Pro:shell xxx$ npm run android

><EMAIL_ADDRESS>android
> react-native run-android --no-packager

info Installing the app...

FAILURE: Build failed with an exception.

* Where:
Settings file '/Users/xxx/Desktop/super-app-showcase/packages/shell/android/settings.gradle' line: 2

* What went wrong:
A problem occurred evaluating settings 'shell'.
> Could not read script<EMAIL_ADDRESS>as it does not exist.

Looks like your app on macOS m2 not working via npm or yarn

#RafikiTiki
I comment "remotes" property in plugin and error is identical.
I test it in intel macOS and error is identical.
Where in the described procedure do I have a mistake?
I need my app to load dynamically remote modules. Is it possible through Re.pack to achieve? If yes, what do I have to change?

@ObscurusGrassator
In the super-app-showcase, please run yarn bootstrap in the root of the monorepo in order to install dependencies & pods for all of the packages inside of it.

@jbroma A couple relevant things, the latter especially to this post's concern:
super-app-showcase needs to mention that it needs an rvm install of the required ruby version, and on top of that, a gem install cocoapods at the required version, or it simply won't bootstrap correctly.  This will lead to the errors posted above.
With respect to this issue regarding webpack_require and "host," I am indeed having that issue in the current version of the showcase app.  It is isolated to Android... the app runs fine with yarn:run:ios.  When I do yarn:run:android, I see no activity in the console window that has yarn start... no attempts to access any of the URLs.  This is the case with a fresh Android device, a wiped android device, and before and after running adbreverse.


Also add inreg to sreg that's added by CreateCallOrInvoke

This case was missed by https://github.com/swiftlang/swift/pull/76159.
This is a partial fix for https://github.com/swiftlang/swift/issues/74866

@swift-ci please test

@swift-ci please test macOS platform

@swift-ci please test macOS platform

@swift-ci please test macOS platform

@swift-ci please test macOS platform

@swift-ci please test macOS platform

Do you mean why all attributes aren't added in SignatureExpansion::expandExternalSignatureTypes?
I wonder that, too. I don't know this code enough to know off hand.
Would looking at these commits, which seem to have added or touched this code, help to clarify?
https://github.com/swiftlang/swift/commit/0e31acb18c109610898a282e69a53ca68c8550dc
https://github.com/swiftlang/swift/commit/b3f302b96b5e099d26362804e750e0a926be6361

@egorzhdan Can you help out here?  Why does the FunctionPointer not just have the right attributes for these calls?

I'm afraid I don't know the reason for the missing attribute. Perhaps this sret/noreg logic should be moved to SignatureExpansion::expandExternalSignatureTypes?
@ahatanaka WDYT?

@rjmccall WDYT? I think I will see what I can figure out. If there are some hints or places to look at, it'd be great if you can point me to them.

Rebased. @swift-ci please test

@ahatanaka ping

@swift-ci please test

It looks like it's possible to add sret and inreg in SignatureExpansion::expandExternalSignatureTypes, but special care must be taken to handle noreturn functions correctly as SILGen changes the return type to Never.
For example, this is the SIL for the call to Large largeStructNoReturn()  __attribute__((noreturn)) in testIndirectReturnNoReturn in test/IRGen/noreturn.swift.
  %0 = function_ref @largeStructNoReturn : $@convention(c) () -> Never // user: %1
  %1 = apply %0() : $@convention(c) () -> Never

@ahatanaka So it sounds like that was the reason to add sreg in IRBuilder::CreateCallOrInvoke?
@rjmccall What do you think?

SILGen should absolutely not be producing functions with a return type of Never just because they're noreturn.  I guess it would work if we reliably had a Clang type available on the SILFunctionType in IRGen, but otherwise, no; we need the actual Clang type in order to emit the call correctly.  As a workaround, trying to retroactively paper over the problem will often work because most noreturn calls will be direct, but it'll blow up if they're ever indirect, and Clang does support noreturn function pointers, ObjC methods, virtual methods, and so on.

ClangImporter::Implementation::importFunctionParamsAndReturnType is setting the result type:
  if (clangDecl->isNoReturn())
    swiftResultTy = SwiftContext.getNeverType();

Okay.  That's not an unreasonable decision for the surface language, but it's incorrect for SIL, at least without a guarantee of carrying a C function type in the SIL function type.  We need to know the actual C calling convention of the function in order to call it correctly, and in general that includes knowing its true declared return type.  In theory, the calling conventions for void (int) and float (int) could be completely different.
Now, on most platforms, we wouldn't actually need to know the exact return type.  The call doesn't return, so anything in the ABI that only matters on the return path is irrelevant, which means that e.g. we can absolutely call a float (int) as if it were a void (int). In every ABI for machine-code platforms that I know of, the return type only affects the non-return path when the return value is returned indirectly, so maybe we could represent that in SIL function types, e.g. as @out Never (which is indirect) vs. Never (which is direct).
However:

That doesn't really work with how we lower foreign calls with clang, because we'd need to use a return type that forced Clang to return indirectly, which we don't have a way to do; we just assume clang will generate an indirect parameter/return convention for a parameter/return type that's @in / @out in SIL.
That's an unportable assumption because, as noted, the C ABI is absolutely allowed to use completely different calling conventions for different function signatures.
I believe WebAssembly does a runtime check that the function signature of a call exactly matches the signature of the callee, at least up to pointers.  So IIUC calling a float (int) as a void (int) in WASM does not work, even if the function never returns.  Therefore, we really do need the exact C return type in order to do the call correctly.

The cleanest solution would simply be to allow the SIL function type to lie arbitrarily about the return type as long as it preserves the C function type.  IRGen would then just have to recognize when a Never return type actually matches a non-trivial return convention in the CGFunctionInfo and e.g. fake an indirect argument as appropriate.  But that would require us to actually preserve C function types in SIL function types.
Unfortunately, the alternatives are much more invasive.  We could recognize a C function with a supposed Swift return type of Never and try to restore the original return type in that function's SIL type, but for that SIL type, we'd actually need a Swift type that corresponds to that original C return type, which is not something that SIL type lowering knows how to do in general.  And using a SIL return type other than Never would actually mess up our SIL analyses, because we'd lose the information that the call didn't return.  And the conservative solution of just not importing non-void noreturn functions as returning Never would break source compatibility for similar reasons.
Maybe there's some viable path for preserving the Clang function type in the SIL function type of just these functions, so that we don't have to tackle the general problem just to fix this bug?

Don't we store the clang type in SILFunctionType already? https://github.com/swiftlang/swift/blob/0d747d4d2b1d6eac0065ea3fc0f3f32d5cf806c1/include/swift/AST/Types.h#L5395

I believe that, just like unlowered function types, it's not reliably there unless you pass the clang-function-types flag.

@rjmccall What do you think of getting this PR merged and moving this discussion to a github issue? It aims to address an orthogonal issue of adding a missing inreg to the sret which seems needed regardless even if we decided to add sret in expandExternalSignatureTypes instead or elsewhere.
This is one of the PRs that are needed to resurrect the Windows ARM64 Swift compiler and while I don't fully follow the discussion here but it sounds like there's more general issue here about noreturn functions and preserving the Clang function type in the SIL function type.
CC: @ahatanaka  @eeckstein @compnerd

In my experience, it's important to resist solving problems the wrong way in a compiler because the hacks (and bugs) just keep piling up.  In this case, there are a large number of IR attributes that are necessary on various platforms in order to generate calls correctly.  We can already see that this little hack of copying attributes from the callee has been slowly accreting support for more and more attributes.  Each of those accretions presumably represents a separate, slowly-debugged miscompile, each potentially paired with a second and still-latent miscompile when the call actually goes through a function pointer.  There is a central place in IRGen that adds attributes to C-convention calls and declarations based on their type.  Fixing bugs there, and making sure that it's used properly, is how we avoid having to fix bugs ten different ways for ten slightly-different inputs.

@rjmccall In general I agree with what you said. But @hjyamauchi is probably not the right person to do this kind of (I assume) rather big rewrite. We need to get this unblocked. And what @hjyamauchi suggested is the right way to do so. After filing a github issue, whoever owns - and fully understands - the code can do a proper fix as a follow-up.
So, if there isn't any strong objection (e.g. this change would introduce another problem), I would suggest to merge this PR and do the proper fix in a follow-up.

It's not clear to me whether @ahatanaka's observation about Never substitution is actually a blocker for fixing this properly in signature expansion.  What happens if we just try to fix it there?

We don't have the exact C function type, but we deconstruct the SILFunctionType to create a CGFunctionInfo, which should be good enough to recognize that we need to apply inreg.  The problem you identified with Never is serious but is an existing miscompile that we at least wouldn't be regressing.

Ah I see. If we don't care about noreturn functions, that should work.

Yeah, I think we clearly need a deeper solution for noreturn functions, with the most elegant option being to propagate down the Clang function type.

@rjmccall @ahatanaka
I examined the test case (included in this PR) and it seems that

The function getFirstBlock@BridgedFunction@@QEBA?AUOptionalBridgedBasicBlock@@XZ in question doesn't look indirect-return or have an indirect return parameter at the time of expandExternalSignatureTypes hence for example it doesn't take the path into the returnInfo.isIndirect() block (here) and no sret (nor inreg) was added.

After expandExternalSignatureTypes, it's made indirect-return (an indirect return parameter is added) in SyncCallEmission::setArgs (here) in the passIndirectResults lambda becausefnConv.getNumIndirectSILResults() > 0.

Later in CreateCallOrInvoke, when it compares the args to the callee, it finds that the parameter is missing sret (and inreg) and adds them. This is where this PR currently adds inreg and we are discussing whether we could instead do that in expandExternalSignatureTypes.

I did some printf debugging around CGFunctionInfo in expandExternalSignatureTypes but it seemed that CGFunctionInfo doesn't have enough info indicating that it is an indirect return or we could add an sret there. This is what I got printed:
expandExternalSignatureTypes on
(sil_function_type type="@convention(cxx_method) (@in_guaranteed BridgedFunction) -> OptionalBridgedBasicBlock"
  (input=struct_type decl="InRegSRet.(file).BridgedFunction")
  (result=struct_type decl="InRegSRet.(file).OptionalBridgedBasicBlock")
  (substitution_map null_generic_signature)
  (substitution_map null_generic_signature))
FnType->getRepresentation() 12
SILResultTy $OptionalBridgedBasicBlock
clangResultTy RecordType 0x59a82d0 'struct OptionalBridgedBasicBlock' imported
`-CXXRecord 0xfe04a8 'OptionalBridgedBasicBlock'
returnInfo.isIndirect 0
getSILFuncConventions().getNumIndirectSILResults() 0
getSILFuncConventions().getNumIndirectSILResults() > 0 0
returnInfo.getKind() 4
returnInfo.isInAlloca() 0
CreateCallOrInvoke: found a missing inreg
declare dso_local void @"?getFirstBlock@BridgedFunction@@QEBA?AUOptionalBridgedBasicBlock@@XZ"(ptr noundef nonnull align 1 dereferenceable(1) %0, ptr inreg sret(%struct.OptionalBridgedBasicBlock) align 1 %1) #0

Notably returnInfo.isIndirect is false, getSILFuncConventions().getNumIndirectSILResults() is 0, returnInfo.getKind() is ABIArgInfo::Ignore, (and returnInfo.getInReg() is false or would cause an assert failure.)
Here's part of the debug print code I added:
In expandExternalSignatureTypes
    llvm::dbgs() << "expandExternalSignatureTypes on\n";
    FnType->dump();
    llvm::dbgs() << "FnType->getRepresentation() " << (uint32_t)(FnType->getRepresentation()) << "\n";
    llvm::dbgs() << "SILResultTy ";
    SILResultTy.dump();
    llvm::dbgs() << "clangResultTy ";
    clangResultTy.dump();
    llvm::dbgs() << "returnInfo.isIndirect " << returnInfo.isIndirect() << "\n";
    llvm::dbgs() << "getSILFuncConventions().getNumIndirectSILResults() "
                 << getSILFuncConventions().getNumIndirectSILResults() << "\n";
    llvm::dbgs() << "getSILFuncConventions().getNumIndirectSILResults() > 0 "
                 << (getSILFuncConventions().getNumIndirectSILResults() > 0) << "\n";
    llvm::dbgs() << "returnInfo.getKind() " << (uint32_t)(returnInfo.getKind()) << "\n";
    llvm::dbgs() << "returnInfo.isInAlloca() " << returnInfo.isInAlloca() << "\n";
    if (returnInfo.isInAlloca()) {
      llvm::dbgs() << "returnInfo.getInAllocaSRet() "
                   << returnInfo.getInAllocaSRet()
                   << "\n";
      llvm::dbgs() << "returnInfo.getInAllocaIndirect() "
                   << returnInfo.getInAllocaIndirect()
                   << "\n";
    }
    if (returnInfo.isDirect() || returnInfo.isExtend() || returnInfo.isIndirect())
      llvm::dbgs() << "returnInfo.getInReg() " << returnInfo.getInReg() << "\n";

In CreateCallOrInvoke
        if (func->hasParamAttribute(argIndex, llvm::Attribute::InReg) &&
            !attrs.hasParamAttr(argIndex, llvm::Attribute::InReg)) {
          llvm::dbgs() << "CreateCallOrInvoke: found a missing inreg\n";
          func->dump();
        }

I see that in SyncCallEmission::setARgs, getNumIndirectSILResults in passIndirectResults is returning 1 here, as opposed to 0 in expandExternalSignatureTypes. So something may be different there, but I have yet to understand that code.

Ah. Interesting. I am going to experiment with arrangeCXXMethodType

I used arrangeCXXMethodCall and it appears to be working now! The returnInfo says it's indirect and sret/inreg are added in expandExternalSignatureTypes.
expandExternalSignatureTypes on
(sil_function_type type="@convention(cxx_method) (@in_guaranteed BridgedFunction) -> OptionalBridgedBasicBlock"
  (input=struct_type decl="InRegSRet.(file).BridgedFunction")
  (result=struct_type decl="InRegSRet.(file).OptionalBridgedBasicBlock")
  (substitution_map null_generic_signature)
  (substitution_map null_generic_signature))
FnType->getRepresentation() 12
SILResultTy $OptionalBridgedBasicBlock
clangResultTy RecordType 0x81031d0 'struct OptionalBridgedBasicBlock' imported
`-CXXRecord 0xfe48a8 'OptionalBridgedBasicBlock'
returnInfo.isIndirect 1
getSILFuncConventions().getNumIndirectSILResults() 0
getSILFuncConventions().getNumIndirectSILResults() > 0 0
returnInfo.getKind() 2
returnInfo.isInAlloca() 0
returnInfo.getInReg() 1
addIndirectResultAttributes : added sret
addIndirectResultAttributes : added inreg

The diff currently looks like the following:
In expandExternalSignatureTypes
-  auto &FI = clang::CodeGen::arrangeFreeFunctionCall(IGM.ClangCodeGen->CGM(),
-                                             clangResultTy, paramTys, extInfo,
-                                             clang::CodeGen::RequiredArgs::All);
+  bool isCXXMethod =
+      FnType->getRepresentation() == SILFunctionTypeRepresentation::CXXMethod;
+  auto &FI = isCXXMethod ?
+      clang::CodeGen::arrangeCXXMethodCall(IGM.ClangCodeGen->CGM(),
+                                           clangResultTy, paramTys, extInfo,
+                                           clang::CodeGen::RequiredArgs::All) :
+      clang::CodeGen::arrangeFreeFunctionCall(IGM.ClangCodeGen->CGM(),
+                                              clangResultTy, paramTys, extInfo,
+                                              clang::CodeGen::RequiredArgs::All);

In the llvm-project repo
diff --git a/clang/include/clang/CodeGen/CodeGenABITypes.h b/clang/include/clang/CodeGen/CodeGenABITypes.h
index 0ba6c2019285..0d38766cac34 100644
--- a/clang/include/clang/CodeGen/CodeGenABITypes.h
+++ b/clang/include/clang/CodeGen/CodeGenABITypes.h
@@ -75,6 +75,12 @@ const CGFunctionInfo &arrangeCXXMethodType(CodeGenModule &CGM,
                                            const FunctionProtoType *FTP,
                                            const CXXMethodDecl *MD);

+const CGFunctionInfo &arrangeCXXMethodCall(CodeGenModule &CGM,
+                                           CanQualType returnType,
+                                           ArrayRef<CanQualType> argTypes,
+                                           FunctionType::ExtInfo info,
+                                           RequiredArgs args);
+
 const CGFunctionInfo &arrangeFreeFunctionCall(CodeGenModule &CGM,
                                               CanQualType returnType,
                                               ArrayRef<CanQualType> argTypes,
diff --git a/clang/lib/CodeGen/CodeGenABITypes.cpp b/clang/lib/CodeGen/CodeGenABITypes.cpp
index a6073e1188d6..283aa8425867 100644
--- a/clang/lib/CodeGen/CodeGenABITypes.cpp
+++ b/clang/lib/CodeGen/CodeGenABITypes.cpp
@@ -59,6 +59,17 @@ CodeGen::arrangeCXXMethodType(CodeGenModule &CGM,
   return CGM.getTypes().arrangeCXXMethodType(RD, FTP, MD);
 }

+const CGFunctionInfo &
+CodeGen::arrangeCXXMethodCall(CodeGenModule &CGM,
+                              CanQualType returnType,
+                              ArrayRef<CanQualType> argTypes,
+                              FunctionType::ExtInfo info,
+                              RequiredArgs args) {
+  return CGM.getTypes().arrangeLLVMFunctionInfo(returnType,
+                                                FnInfoOpts::IsInstanceMethod,
+                                                argTypes, info, {}, args);
+}
+
 const CGFunctionInfo &
 CodeGen::arrangeFreeFunctionCall(CodeGenModule &CGM,
                                  CanQualType returnType,

I added arrangeCXXMethodCall in CodeGenABITypes.h instead of using the existing arrangeCXXMethodType to keep the same set of arguments as arrangeFreeFunctionCall.
Is there a reason to not to add arrangeCXXMethodCall and to use arrangeCXXMethodType? If so, how? @rjmccall
I also need to test the whole win/arm64 toolchain bootstrap works with this.

I added arrangeCXXMethodCall in CodeGenABITypes.h instead of using the existing arrangeCXXMethodType to keep the same set of arguments as arrangeFreeFunctionCall.
Is there a reason to not to add arrangeCXXMethodCall and to use arrangeCXXMethodType? If so, how? @rjmccall

No, it's abstractly the right thing to do.  It shouldn't matter practically because IIRC declaration vs. type arrangement only matters for variadic calls, which Swift currently never emits, but I do support doing it.  Please upstream that patch to LLVM if you don't mind.

Yeah, will upstream the patch to LLVM. ty!

@swift-ci please test

An update:
I merged the spun-off Clang PR: https://github.com/llvm/llvm-project/pull/111597. Thanks for reviewing, John. I'm working on cherrypicking it into swiftlang.
Re: the above refactoring to do with isSRetAfterThis, I attempted this refactoring and it worked up to some extent but I hit a few cases in the swift tests where the use of isSRetAfterThis didn't preserve the existing behaviors.
I changed the four code locations in GenCalls.cpp (I attach the patch/diff at the bottom.) and replaced the existing conditions that check for the windows build and the C++ method, etc. with the new ones that use isSRetAfterThis and also inserted asserts that check the equivalence between the old and the new conditions. But I encountered a few assert failure cases in particular with some C++ operator+='s. It seems like they use an sret argument but their CGFunctionInfo don't indicate that they are indirect-return (i.e., isIndirect()==false). (Note isSRetAfterThis can be called only when isIndirect()==true.) @rjmccall Do you see what I'm missing here?
The asserts failed in the following two tests along with some debug prints:
Test 1: Interop/Cxx/class/method/msvc-abi-return-indirect-trivial-record.swift
llvm::FunctionType
void (ptr, ptr, i64)
OrigFunctionType
(sil_function_type type="@convention(cxx_method) (LoadableIntWrapper, @inout LoadableIntWrapper) -> ()"
  (input=struct_type decl="MsvcUseVecIt.(file).LoadableIntWrapper")
  (input=struct_type decl="MsvcUseVecIt.(file).LoadableIntWrapper")
  (substitution_map null_generic_signature)
  (substitution_map null_generic_signature))
SubstFunctionType
(sil_function_type type="@convention(cxx_method) (LoadableIntWrapper, @inout LoadableIntWrapper) -> ()"
  (input=struct_type decl="MsvcUseVecIt.(file).LoadableIntWrapper")
  (input=struct_type decl="MsvcUseVecIt.(file).LoadableIntWrapper")
  (substitution_map null_generic_signature)
  (substitution_map null_generic_signature))
IGF.IGM.Triple.isWindowsMSVCEnvironment() 1
getCallee().getRepresentation() == SILFunctionTypeRepresentation::CXXMethod 1
substResultType.isVoid() 1
getCallee().getFunctionPointer().getRawPointer() 1
FI != nullptr 1
FI->getReturnInfo().isIndirect() 0
FI->getReturnInfo().getKind() 4
Assertion failed: (IGF.IGM.Triple.isWindowsMSVCEnvironment() && getCallee().getRepresentation() == SILFunctionTypeRepresentation::CXXMethod && substResultType.isVoid() && getCallee().getFunctionPointer().getRawPointer()) == (FI && FI->getReturnInfo().isIndirect() && FI->getReturnInfo().isSRetAfterThis() && substResultType.isVoid() && getCallee().getFunctionPointer().getRawPointer()), file S:\SourceCache\swift\lib\IRGen\GenCall.cpp, line 3623

The operator+= in source code.
struct LoadableIntWrapper {
    int value;

    LoadableIntWrapper operator+=(LoadableIntWrapper rhs) {
        value += rhs.value;
        return *this;
    }
};

Test 2: Interop/Cxx/operators/member-inline.swift
llvm::FunctionType
void (ptr, ptr, i64)
OrigFunctionType
(sil_function_type type="@convention(cxx_method) (LoadableIntWrapper, @inout DerivedFromLoadableIntWrapperWithUsingDecl) -> ()"
  (input=struct_type decl="MemberInline.(file).LoadableIntWrapper")
  (input=struct_type decl="MemberInline.(file).DerivedFromLoadableIntWrapperWithUsingDecl")
  (substitution_map null_generic_signature)
  (substitution_map null_generic_signature))
SubstFunctionType
(sil_function_type type="@convention(cxx_method) (LoadableIntWrapper, @inout DerivedFromLoadableIntWrapperWithUsingDecl) -> ()"
  (input=struct_type decl="MemberInline.(file).LoadableIntWrapper")
  (input=struct_type decl="MemberInline.(file).DerivedFromLoadableIntWrapperWithUsingDecl")
  (substitution_map null_generic_signature)
  (substitution_map null_generic_signature))
IGF.IGM.Triple.isWindowsMSVCEnvironment() 1
getCallee().getRepresentation() == SILFunctionTypeRepresentation::CXXMethod 1
substResultType.isVoid() 1
getCallee().getFunctionPointer().getRawPointer() 1
FI != nullptr 1
FI->getReturnInfo().isIndirect() 0
FI->getReturnInfo().getKind() 4
Assertion failed: (IGF.IGM.Triple.isWindowsMSVCEnvironment() && getCallee().getRepresentation() == SILFunctionTypeRepresentation::CXXMethod && substResultType.isVoid() && getCallee().getFunctionPointer().getRawPointer()) == (FI && FI->getReturnInfo().isIndirect() && FI->getReturnInfo().isSRetAfterThis() && substResultType.isVoid() && getCallee().getFunctionPointer().getRawPointer()), file S:\SourceCache\swift\lib\IRGen\GenCall.cpp, line 3623

The operator+= in source code.
struct LoadableIntWrapper {
  int value;
  LoadableIntWrapper operator-(LoadableIntWrapper rhs) {
    return LoadableIntWrapper{.value = value - rhs.value};
  }

  LoadableIntWrapper operator+=(LoadableIntWrapper rhs) {
    value += rhs.value;
    return *this;
  }
  ...
};
struct DerivedFromLoadableIntWrapperWithUsingDecl : private LoadableIntWrapper {
  using LoadableIntWrapper::operator-;
  using LoadableIntWrapper::operator+=;
 ...
};

The patch/diff
The four code locations are marked with: *** (1) ***, *** (2) ***, etc. The above two failures happen in *** (4) ***.
diff --git a/lib/IRGen/Callee.h b/lib/IRGen/Callee.h
index 66199d8819f..6232d9436f8 100644
--- a/lib/IRGen/Callee.h
+++ b/lib/IRGen/Callee.h
@@ -563,7 +563,7 @@ namespace irgen {

     const FunctionPointer &getFunctionPointer() const { return Fn; }

-    llvm::FunctionType *getLLVMFunctionType() {
+    llvm::FunctionType *getLLVMFunctionType() const {
       return Fn.getFunctionType();
     }

diff --git a/lib/IRGen/GenCall.cpp b/lib/IRGen/GenCall.cpp
index 660e8d22f9f..39ef485b27b 100644
--- a/lib/IRGen/GenCall.cpp
+++ b/lib/IRGen/GenCall.cpp
@@ -1462,9 +1462,15 @@ void SignatureExpansion::expandExternalSignatureTypes() {
   // Generate function info for this signature.
   auto extInfo = clang::FunctionType::ExtInfo();

-  auto &FI = clang::CodeGen::arrangeFreeFunctionCall(IGM.ClangCodeGen->CGM(),
-                                             clangResultTy, paramTys, extInfo,
-                                             clang::CodeGen::RequiredArgs::All);
+  bool isCXXMethod =
+      FnType->getRepresentation() == SILFunctionTypeRepresentation::CXXMethod;
+  auto &FI = isCXXMethod ?
+      clang::CodeGen::arrangeCXXMethodCall(IGM.ClangCodeGen->CGM(),
+          clangResultTy, paramTys, extInfo, {},
+          clang::CodeGen::RequiredArgs::All) :
+      clang::CodeGen::arrangeFreeFunctionCall(IGM.ClangCodeGen->CGM(),
+          clangResultTy, paramTys, extInfo, {},
+          clang::CodeGen::RequiredArgs::All);
   ForeignInfo.ClangInfo = &FI;

   assert(FI.arg_size() == paramTys.size() &&
@@ -1586,9 +1592,11 @@ void SignatureExpansion::expandExternalSignatureTypes() {
   if (returnInfo.isIndirect()) {
     auto resultType = getSILFuncConventions().getSingleSILResultType(
         IGM.getMaximalTypeExpansionContext());
-    if (IGM.Triple.isWindowsMSVCEnvironment() &&
-        FnType->getRepresentation() ==
-            SILFunctionTypeRepresentation::CXXMethod) {
+    // *** (1) *** This passes tests.
+    assert((IGM.Triple.isWindowsMSVCEnvironment() &&
+            FnType->getRepresentation() == SILFunctionTypeRepresentation::CXXMethod) ==
+           returnInfo.isSRetAfterThis());
+    if (returnInfo.isSRetAfterThis()) {
       // Windows ABI places `this` before the
       // returned indirect values.
       emitArg(0);
@@ -2555,11 +2563,22 @@ public:

         // Windows ABI places `this` before the
         // returned indirect values.
-        bool isThisFirst = IGF.IGM.Triple.isWindowsMSVCEnvironment();
-        if (!isThisFirst)
+        // *** (2) *** This passes tests.
+        assert(getCallee().getForeignInfo().ClangInfo != nullptr);
+        auto &returnInfo =
+            getCallee().getForeignInfo().ClangInfo->getReturnInfo();
+        if (fnConv.getNumIndirectSILResults() > 0) {
+          if (!returnInfo.isIndirect()) {
+            llvm::dbgs() << "returnInfo.isIndirect() " << returnInfo.isIndirect() << "\n";
+            llvm::dbgs() << "fnConv.getNumIndirectSILResults() " << fnConv.getNumIndirectSILResults() << "\n";
+          }
+          assert(returnInfo.isIndirect());
+          assert(returnInfo.isSRetAfterThis() == IGF.IGM.Triple.isWindowsMSVCEnvironment());
+        }
+        if (returnInfo.isIndirect() && !returnInfo.isSRetAfterThis())
           passIndirectResults();
         adjusted.add(arg);
-        if (isThisFirst)
+        if (returnInfo.isIndirect() && returnInfo.isSRetAfterThis())
           passIndirectResults();
       }

@@ -3148,9 +3167,46 @@ void CallEmission::emitToUnmappedMemory(Address result) {
   assert(LastArgWritten == 1 && "emitting unnaturally to indirect result");

   Args[0] = result.getAddress();
-  if (IGF.IGM.Triple.isWindowsMSVCEnvironment() &&
-      getCallee().getRepresentation() ==
-          SILFunctionTypeRepresentation::CXXMethod &&
+
+  // *** (3) ***
+  auto *FI = getCallee().getForeignInfo().ClangInfo;
+  if ((IGF.IGM.Triple.isWindowsMSVCEnvironment() &&
+       getCallee().getRepresentation() ==
+           SILFunctionTypeRepresentation::CXXMethod &&
+       Args[1] == getCallee().getCXXMethodSelf()) !=
+      (FI && FI->getReturnInfo().isIndirect() &&
+       FI->getReturnInfo().isSRetAfterThis() &&
+       Args[1] == getCallee().getCXXMethodSelf())) {
+    llvm::dbgs() << "IGF.IGM.Triple.isWindowsMSVCEnvironment() "
+                 << IGF.IGM.Triple.isWindowsMSVCEnvironment() << "\n";
+    llvm::dbgs() << "getCallee().getRepresentation() == SILFunctionTypeRepresentation::CXXMethod "
+                 << (getCallee().getRepresentation() == SILFunctionTypeRepresentation::CXXMethod) << "\n";
+    llvm::dbgs() << "FI != nullptr " << (FI != nullptr) << "\n";
+    if (FI) {
+      llvm::dbgs() << "FI->getReturnInfo().isIndirect() "
+                   << FI->getReturnInfo().isIndirect() << "\n";
+      llvm::dbgs() << "FI->getReturnInfo().getKind() "
+                   << (unsigned)FI->getReturnInfo().getKind() << "\n";
+      if (FI->getReturnInfo().isIndirect()) {
+        llvm::dbgs() << "FI->getReturnInfo().isSRetAfterThis()"
+                     << FI->getReturnInfo().isSRetAfterThis() << "\n";
+      }
+    }
+    llvm::dbgs() << "Args.size() " << Args.size() << "\n";
+    if (Args.size() >= 2) {
+      llvm::dbgs() << "Args[1] == getCallee().getCXXMethodSelf() "
+                   << (Args[1] == getCallee().getCXXMethodSelf()) << "\n";
+    }
+  }
+  assert((IGF.IGM.Triple.isWindowsMSVCEnvironment() &&
+          getCallee().getRepresentation() ==
+              SILFunctionTypeRepresentation::CXXMethod &&
+          Args[1] == getCallee().getCXXMethodSelf()) ==
+         (FI && FI->getReturnInfo().isIndirect() &&
+          FI->getReturnInfo().isSRetAfterThis() &&
+          Args[1] == getCallee().getCXXMethodSelf()));
+  if (FI && FI->getReturnInfo().isIndirect() &&
+      FI->getReturnInfo().isSRetAfterThis() &&
       Args[1] == getCallee().getCXXMethodSelf()) {
     // C++ methods in MSVC ABI pass `this` before the
     // indirectly returned value.
@@ -3515,10 +3571,58 @@ void CallEmission::emitToExplosion(Explosion &out, bool isOutlined) {
       emitToMemory(temp, substResultTI, isOutlined);
       return;
     }
-    if (IGF.IGM.Triple.isWindowsMSVCEnvironment() &&
-        getCallee().getRepresentation() ==
-            SILFunctionTypeRepresentation::CXXMethod &&
-        substResultType.isVoid()) {
+
+    // *** (4) ***
+    // Didn't work due to operator+=() that returns void but CGFunctionInfo doesn't say it's indirect
+    // Tests:
+    // Interop/Cxx/class/method/msvc-abi-return-indirect-trivial-record.swift
+    // Interop/Cxx/operators/member-inline.swift
+    auto *FI = getCallee().getForeignInfo().ClangInfo;
+    if ((IGF.IGM.Triple.isWindowsMSVCEnvironment() &&
+         getCallee().getRepresentation() ==
+             SILFunctionTypeRepresentation::CXXMethod &&
+         substResultType.isVoid() &&
+         getCallee().getFunctionPointer().getRawPointer()) !=
+        (FI && FI->getReturnInfo().isIndirect() &&
+         FI->getReturnInfo().isSRetAfterThis() &&
+         substResultType.isVoid() &&
+         getCallee().getFunctionPointer().getRawPointer())) {
+      llvm::dbgs() << "llvm::FunctionType " << "\n";
+      getCallee().getLLVMFunctionType()->dump();
+      llvm::dbgs() << "OrigFunctionType " << "\n";
+      getCallee().getOrigFunctionType()->dump();
+      llvm::dbgs() << "SubstFunctionType " << "\n";
+      getCallee().getSubstFunctionType()->dump();
+      llvm::dbgs() << "IGF.IGM.Triple.isWindowsMSVCEnvironment() "
+                   << IGF.IGM.Triple.isWindowsMSVCEnvironment() << "\n";
+      llvm::dbgs() << "getCallee().getRepresentation() == SILFunctionTypeRepresentation::CXXMethod "
+                   << (getCallee().getRepresentation() == SILFunctionTypeRepresentation::CXXMethod) << "\n";
+      llvm::dbgs() << "substResultType.isVoid() " << substResultType.isVoid() << "\n";
+      llvm::dbgs() << "getCallee().getFunctionPointer().getRawPointer() "
+                   << (getCallee().getFunctionPointer().getRawPointer() != 0) << "\n";
+      llvm::dbgs() << "FI != nullptr " << (FI != nullptr) << "\n";
+      if (FI) {
+        llvm::dbgs() << "FI->getReturnInfo().isIndirect() "
+                     << FI->getReturnInfo().isIndirect() << "\n";
+        llvm::dbgs() << "FI->getReturnInfo().getKind() "
+                     << (unsigned)FI->getReturnInfo().getKind() << "\n";
+        if (FI->getReturnInfo().isIndirect()) {
+          llvm::dbgs() << "FI->getReturnInfo().isSRetAfterThis() "
+                       << FI->getReturnInfo().isSRetAfterThis() << "\n";
+        }
+      }
+    }
+    assert((IGF.IGM.Triple.isWindowsMSVCEnvironment() &&
+            getCallee().getRepresentation() ==
+                SILFunctionTypeRepresentation::CXXMethod &&
+            substResultType.isVoid() &&
+            getCallee().getFunctionPointer().getRawPointer()) ==
+           (FI && FI->getReturnInfo().isIndirect() &&
+            FI->getReturnInfo().isSRetAfterThis() &&
+            substResultType.isVoid() &&
+            getCallee().getFunctionPointer().getRawPointer()));
+    if (FI && FI->getReturnInfo().isIndirect() &&
+        FI->getReturnInfo().isSRetAfterThis() && substResultType.isVoid()) {
       // Some C++ methods return a value but are imported as
       // returning `Void` (e.g. `operator +=`). In this case
       // we should allocate the correct temp indirect return

That seems like a larger problem — that function obviously should not be given a () return type in SIL when it clearly returns a non-trivial value.  We could fix that in SIL by "bridging" non-void return values into () (by ignoring them), but in this case, I feel like we should just be importing the function as having a real return value.  Since it's a (compound) assignment operator, it seems conventionally correct to infer @discardableResult, though.

@rjmccall
CGFunctionInfo is from the Clang code. So, that's just how Clang handles operator+= and not a bug? If so, I'm curious how this isn't a problem in Clang, which may need to do similar isIndirect and isSRetAfterThis checks.
Is "importing the function as having a real return value" referring to happening in Clang or Swift? If Swift, it isn't a right assumption that we can solely rely on CGFunctionInfo to detect an indirect return?
CC @eeckstein @compnerd @ahatanaka @hyp Any thoughts?

@swift-ci please test

@rjmccall
CGFunctionInfo is from the Clang code. So, that's just how Clang handles operator+= and not a bug? If so, I'm curious how this isn't a problem in Clang, which may need to do similar isIndirect and isSRetAfterThis checks.

It's not an issue in Clang because Clang is emitting the call using the actual signature of the function.  Swift could do the same if it propagated the Clang function type in the SILFunctionType.  Because it doesn't do that, Swift has to regenerate the Clang function type from the components of the SILFunctionType, which is why it's a problem that the SILFunctionType is incorrectly saying that the function doesn't have a return value.
We're importing this operator at the AST level as if it returned void; that's then just carrying through into the SIL type.  I assume the importer is doing a blanket suppression of the return value of assignment operators.  @egorzhdan probably knows why, but I imagine it's because assignment operators usually return references and people usually want to ignore them.  That's definitely biting us here, though.  It's okay on most platforms to call a pointer-returning function as a void-returning function, but that's not a totally portable guarantee (e.g. I imagine it blows up on wasm), and it's definitely not true for arbitrary return types — for example, on i386, float functions return on the x87 stack, and calling them as void functions will leave the stack unbalanced. In this case, we're both missing an sret and potentially leaking a value (if the return type were non-trivial).  As I said before, the right fix is probably just to import compound-assignment operators with their actual return type, maybe decorated as @discardableResult for usability reasons.

@swift-ci please test

@rjmccall Okay. Would you elaborate on "import compound-assignment operators with their actual return type, maybe decorated as @discardableResult for usability reasons" (like what code to modify, etc. if you are familiar)? I'll see what I can do if you can provide guidance. @egorzhdan I'd appreciate your thoughts.

@rjmccall Okay. Would you elaborate on "import compound-assignment operators with their actual return type, maybe decorated as @discardableResult for usability reasons" (like what code to modify, etc. if you are familiar)? I'll see what I can do if you can provide guidance. @egorzhdan I'd appreciate your thoughts.

Looks like this is the line:
https://github.com/swiftlang/swift/blob/940730efb66c22439b32feaf585ba5348fcff907/lib/ClangImporter/ImportType.cpp#L2205

I'd say that this should check that the result is a reference (and maybe a reference specifically to the this type?) before dropping it.  I believe we automatically add @discardableResult to imported declarations with non-Void return types, so that much is handled already.

As suggested, dropping the return type from the operator+= etc. only when the return type is of a reference type seems to make the above assertion failures go away. I also tried to limit this to the this pointer case but it seems to cause quite a few other test failures maybe because the conformance checking relies on the return type being void (the details at the bottom). I think it's sufficient to drop the return type only when the return type is a reference type because that's probably what the original intention of the return-type-dropping code and a reference-type return type means it's unlikely an indirect return, which solves this CGFunctionInfo/operator+= issue with the above assert failures. Please take another look @rjmccall
The test failures (with restricting to this pointer)
******************** TEST 'Swift(windows-x86_64) :: Interop/Cxx/class/protocol-conformance-typechecker.swift' FAILED ********************
S:\SourceCache\swift\test\Interop\Cxx\class\protocol-conformance-typechecker.swift:48:15: error: unexpected note produced: protocol requires function '+=' with type '(inout HasOperatorPlusEqual<CInt>, Int32) -> ()'
  static func +=(lhs: inout Self, x: Int32)
              ^
S:\SourceCache\swift\test\Interop\Cxx\class\protocol-conformance-typechecker.swift:51:1: error: unexpected error produced: type 'HasOperatorPlusEqual<CInt>' does not conform to protocol 'HasOperatorPlusEqualProtocol'
extension HasOperatorPlusEqualInt : HasOperatorPlusEqualProtocol {}
^
S:\SourceCache\swift\test\Interop\Cxx\class\protocol-conformance-typechecker.swift:51:1: error: unexpected note produced: add stubs for conformance
extension HasOperatorPlusEqualInt : HasOperatorPlusEqualProtocol {}
^

******************** TEST 'Swift(windows-x86_64) :: Interop/Cxx/stdlib/msvcprt-module-interface.swift' FAILED ********************
S:\SourceCache\swift\test\Interop\Cxx\stdlib\msvcprt-module-interface.swift:15:18: error: CHECK-STRING: expected string not found in input
// CHECK-STRING: struct basic_string<CChar, char_traits<CChar>, allocator<CChar>> : CxxRandomAccessCollection {
                 ^
<stdin>:23948:54: note: scanning from here
 static func to_wstring(_ _Val: Int32) -> std.wstring
                                                     ^

Input file: <stdin>
Check file: S:\SourceCache\swift\test\Interop\Cxx\stdlib\msvcprt-module-interface.swift

******************** TEST 'Swift(windows-x86_64) :: Interop/Cxx/stdlib/overlay/custom-collection-module-interface.swift' FAILED ********************
S:\SourceCache\swift\test\Interop\Cxx\stdlib\overlay\custom-collection-module-interface.swift:17:11: error: CHECK: expected string not found in input
// CHECK: struct SimpleCollectionReadOnly : CxxRandomAccessCollection {
          ^
<stdin>:167:2: note: scanning from here
}
 ^
<stdin>:168:1: note: possible intended match here
struct SimpleCollectionReadOnly : CxxConvertibleToCollection {
^

Input file: <stdin>
Check file: S:\SourceCache\swift\test\Interop\Cxx\stdlib\overlay\custom-collection-module-interface.swift

******************** TEST 'Swift(windows-x86_64) :: Interop/Cxx/stdlib/overlay/custom-collection.swift' FAILED ********************
S:\SourceCache\swift\test\Interop\Cxx\stdlib\overlay\custom-collection.swift:32:17: error: value of type 'SimpleCollectionReadOnly' has no member 'first'
30 | CxxCollectionTestSuite.test("SimpleCollectionReadOnly as Swift.Collection") {
31 |   let c = SimpleCollectionReadOnly()
32 |   expectEqual(c.first, 1)
   |                 `- error: value of type 'SimpleCollectionReadOnly' has no member 'first'
33 |   expectEqual(c.last, 5)
34 |

******************** TEST 'Swift(windows-x86_64) :: Interop/Cxx/stdlib/overlay/custom-iterator-module-interface.swift' FAILED ********************
S:\SourceCache\swift\test\Interop\Cxx\stdlib\overlay\custom-iterator-module-interface.swift:22:11: error: CHECK: expected string not found in input
// CHECK: struct ConstRACIteratorRefPlusEq : UnsafeCxxRandomAccessIterator, UnsafeCxxInputIterator {
          ^
<stdin>:70:2: note: scanning from here
}
 ^
<stdin>:72:1: note: possible intended match here
struct ConstRACIteratorRefPlusEq : UnsafeCxxInputIterator {
^

Input file: <stdin>
Check file: S:\SourceCache\swift\test\Interop\Cxx\stdlib\overlay\custom-iterator-module-interface.swift

******************** TEST 'Swift(windows-x86_64) :: Interop/Cxx/stdlib/use-std-span.swift' FAILED ********************
S:\SourceCache\swift\test\Interop\Cxx\stdlib\use-std-span.swift:185:18: error: cannot use mutating getter on immutable value: 's' is a 'let' constant
180 |   var arr: [Int32] = [1, 2, 3]
181 |   arr.withUnsafeMutableBufferPointer{ ubpointer in
182 |     let s = initSpan(ubpointer.baseAddress!, ubpointer.count)
    |     `- note: change 'let' to 'var' to make it mutable
183 |     expectEqual(s.size(), 3)
184 |     expectFalse(s.empty())
185 |     expectEqual(s[0], 1)
    |                  `- error: cannot use mutating getter on immutable value: 's' is a 'let' constant
186 |     expectEqual(s[1], 2)
187 |     expectEqual(s[2], 3)

******************** TEST 'Swift(windows-x86_64) :: Interop/Cxx/stdlib/use-std-string.swift' FAILED ********************
S:\SourceCache\swift\test\Interop\Cxx\stdlib\use-std-string.swift:372:14: error: for-in loop requires 'std.string' (aka 'std.basic_string<CChar, char_traits<CChar>, allocator<CChar>>') to conform to 'Sequence'
370 |     let cxx1 = std.string()
371 |     var iterated = false
372 |     for _ in cxx1 {
    |              `- error: for-in loop requires 'std.string' (aka 'std.basic_string<CChar, char_traits<CChar>, allocator<CChar>>') to conform to 'Sequence'
373 |         iterated = true
374 |     }

******************** TEST 'Swift(windows-x86_64) :: Interop/Cxx/stdlib/use-std-vector.swift' FAILED ********************
S:\SourceCache\swift\test\Interop\Cxx\stdlib\use-std-vector.swift:32:20: error: cannot convert value of type '[Any]' to expected argument type 'std.allocator<CInt>'
 30 |
 31 | StdVectorTestSuite.test("VectorOfInt.init(sequence)") {
 32 |     let v = Vector([])
    |                    `- error: cannot convert value of type '[Any]' to expected argument type 'std.allocator<CInt>'
 33 |     expectEqual(v.size(), 0)
 34 |     expectTrue(v.empty())

S:\SourceCache\swift\test\Interop\Cxx\stdlib\use-std-vector.swift:155:15: error: value of type 'Vector' (aka 'std.vector<CInt, allocator<CInt>>') has no member 'map'
153 |     fill(vector: &v)
154 |
155 |     let a = v.map { $0 + 5 }
    |               `- error: value of type 'Vector' (aka 'std.vector<CInt, allocator<CInt>>') has no member 'map'
156 |     expectEqual(a, [6, 7, 8])
157 | }

S:\SourceCache\swift\test\Interop\Cxx\stdlib\use-std-vector.swift:174:14: error: for-in loop requires 'VectorSubclass' to conform to 'Sequence'
172 |
173 |     var count: CInt = 1
174 |     for e in v {
    |              `- error: for-in loop requires 'VectorSubclass' to conform to 'Sequence'
175 |         expectEqual(e, count)
176 |         count += 1

******************** TEST 'Swift(windows-x86_64) :: Interop/CxxToSwiftToCxx/span/span-execution.cpp' FAILED ********************
S:\b\5\tools\swift\test-windows-x86_64\Interop\CxxToSwiftToCxx\span\Output\span-execution.cpp.tmp/use-span.swift:12:15: error: cannot use mutating getter on immutable value: 'sp' is a 'let' constant
10 |
11 | public func printSpan(_ sp: Span) {
12 |   print("{\(sp[0]), \(sp[1]), \(sp[2])}")
   |               `- error: cannot use mutating getter on immutable value: 'sp' is a 'let' constant
13 | }
14 |

S:\b\5\tools\swift\test-windows-x86_64\Interop\CxxToSwiftToCxx\span\Output\span-execution.cpp.tmp/use-span.swift:28:19: error: value of type 'Span' (aka 'std.span<CInt, _CUnsignedLongLong_-1>') has no member 'map'
26 |
27 | public func mapSpan(_ sp: Span) {
28 |   let result = sp.map { $0 + 3 }
   |                   `- error: value of type 'Span' (aka 'std.span<CInt, _CUnsignedLongLong_-1>') has no member 'map'
29 |   print(result)
30 | }

@swift-ci please test

@swift-ci please test

Thanks, John! I will look into the test failures.

@swift-ci please test

@swift-ci please test Windows Platform

@swift-ci please test Windows Platform

@swift-ci please test Windows Platform

@swift-ci please test Windows Platform

@rjmccall Can you take a quick look after the test fix? (for the return type change, additional attributes like noalias, nocapture and a C++ struct type name change as a consequence of the code gen change.) The CI all passed.

Thanks, John! I am not sure why the 16-byte alignment there, either 🤔 but I agree with you. If I have a chance, I'll think about this. I'll merge this, and move onto cherrypicking this along with the other commits to 6 and 5.10 to make progress in resurrecting the windows arm64 compiler.