Refactor EIP Operator

    - transition eip/pod watchers to mostly adjust tags
    - update logic of eip controller to:
      1. validate current claim or remove
      2. detatch if attatched and unclaimed 3. find a pod/node to claim 4. attach to node or pod node 5. update status to reflect the current state

Refactor EIP Operator to better handle multiple selector matches
I believe this change set represents the changes discussed here:
node:
reconciliation
-  search through eip , find ones that match labels and are unclaimed
-  if no matches nothing
- if matches claim with node uuid

cleanup
  - remove claim from eip

EIP:
reconciliation:
  - if claimed
     - check node/pod conditions - disconnect from terminating resoruces
     - if still claimed
	associate with claimed node/pod
	return

  - if unclaimed
      - detach if attached to something
	  - search through nodes / pods
	  - find ones that match and are not terminating
		attatch and update status
		return

Pods:
basically the same as nodes (except for handling target annotations and auto-create eip)

Concerns:

I'm not sure I understand fully how this interacts with the pod controllers dns_target_annotations.
I believe we've walked through all of the scenarios we can think of there are potentially some we hadn't thought of. Hopefully those get spotted during test/review.
In the eip controller, I'm using a mutable status spec to update the resource status as I operate and I patch at the end. I think this works, but I have some concerns that some action will be performed and an error down the line will lead to a an unpatched status. Just wanted to get some thoughts on this.

Testing:
This passed tests on the first test run which is either very surprising or an indicating that the things I broke our outside our local test scope. Either way I'd like to test this on a personal MZ stack, I'll reach out for some pointers and update this with results.
** update **
I've tested this on a personal stack.

For pods, I deleted envd pods, and added/removed environments, these all seem to work great.
For nodes, I deleted egress nodes, and upped the replica count on the egress no-op replica set so two nodes were competing for the same eip. This seems to work great.  In the two nodes case, we wont' swap over until the node with the existing claim removes its claim, likely by termination, although failing to matche eip is a possible reason. In both cases the failover happens super quick and appears stable.

I believe I've also handled the transition case to this new method.
To test this I patched the eip resource status to set claims to None, and ensured reconciliation of the EIP correctly maintained the association and only added the new claim value to the status.
Any happy to do any other recommended testing.

Side note here:
There's a larger issue with the autocreate_eip label.. If an EIP is created and matches a pod we'll remove it, since pods are unique this is probably fine, but if there are two eips that match a pod it'll just delete one kind of at random.. along with this if an eip is auto created and then modified to point at a different pod it will not be deleted, this one is probably fine, especially if we only ever allow matching by pod name, but I'm not sure these edge cases are well documented or entirely thought through. I don't plan on handling either of these cases.


Deploy to AWS Lambda

Hi,
Would this bot work on a serverless architecture such as AWS Lambda?
Thanks

Though I haven't tried myself, I found this guide https://aws.amazon.com/blogs/compute/nodejs-packages-in-lambda/ which suggests that running a smooch-bot this way should be easy. You'd have to include the npm module dependencies as part of your Lambda deployment as the guide shows.

Thanks for the response @alavers - I gave it a go after posting actually. I got it working using this new setup from AWS labs - https://github.com/awslabs/aws-serverless-express - it was actually fairly simple 👍
The only problem is that the bot responses are broken, the first response is correct but then it seems to double up after or just get confused.
Any thoughts?

I assume you only clicked Yes once? :-) If you link me to where your bot is running I can take a peek.

Haha yep only clicked Yes once. Unfortunately I've taken it offline now as I decided to go with another bot solution. Thanks for your help though :)


Use GitHub Actions.

Switch to GitHub Actions workflow.
Update badges.
Add coverage testing.

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@7dd27c0). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master      #23   +/-   ##
=========================================
  Coverage          ?   84.04%
=========================================
  Files             ?        5
  Lines             ?       94
  Branches          ?        0
=========================================
  Hits              ?       79
  Misses            ?       15
  Partials          ?        0

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7dd27c0...b9a22a9. Read the comment docs.

Yeah, I just did some quick hacks to deal with linting errors.  What's the preferred way to handle unused params?  Use the pragma or comment it out?  Some of the other warnings just need a bit of love to fix up.  That it() continuation hack with an empty string is gross. I'm not sure how to fix that without the worse idea of indenting every function body.

eslint-disable directives are my preferred solution.


InvalidSignatureException

Hi,
Registration works fine but when I try to get login I am getting the below error. Can you please guide is there anything I am missing?
{
"status": "error",
"message": {
"headers": {},
"original": {
"error": "InvalidSignatureException",
"message": "InvalidSignatureException"
},
"exception": null
}
}

After following the IAM User Roles creation steps I have recreated the access key. I get the below error now. Please help
{
"status": "error",
"message": {
"headers": {},
"original": {
"error": "UnrecognizedClientException",
"message": "UnrecognizedClientException"
},
"exception": null
}
}

You need to save the user details, specially username, password, in your local Database under table name 'users'. I have also got into this issue, solved by this. Now I am getting accesstoken, expire_time, type, refresh_token and IdToken.

You need to save the user details, specially username, password, in your local Database under table name 'users'. I have also got into this issue, solved by this. Now I am getting accesstoken, expire_time, type, refresh_token and IdToken.

You need to have the username in DB and Cognito to be alike. You should avoid storing passwords in DB, as Cognito manages the same for you.

Read me is updated. Added a working code with a demo link.


Add runtime feature to examples/postgres/todos/Cargo.toml

Add the runtime to allow the example to compile.
(other examples might need to be updated also)

CI failure doesn't look related. Thanks for the fix. 🙌

On second thought, it looks like this got fixed in another PR that was recently merged. Thanks regardless for pushing this up.


update codecov settings

Create a codecov.yml configuration to change the codecov settings:

less unnecessary precision
tolerate a 1% difference

Codecov Report

Merging #846 into develop will decrease coverage by 0.6%.
The diff coverage is n/a.

@@            Coverage Diff            @@
##           develop    #846     +/-   ##
=========================================
- Coverage     83.1%   82.6%   -0.6%
=========================================
  Files           47      47
  Lines         5524    5326    -198
=========================================
- Hits          4592    4398    -194
+ Misses         932     928      -4

Impacted Files
Coverage Δ

stellargraph/core/graph.py
97.7% <0%> (ø)
:arrow_down:

stellargraph/core/convert.py
100% <0%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update de2c9b7...9db089a. Read the comment docs.


CI: use pytest-timeout

Try to avoid appveyor hanging, by limiting test runtime.

Coverage increased (+0.1%) to 87.024% when pulling 9f8d70eef78714342f84f7a8416c541cad175654 on pv:ci-fixes into 68ff232cb49c5ade3b74665351df6501735ffff2 on spacetelescope:master.


Add patch to write api

Add a patch route to the Backdrop Write API to allow one or more values in a data set document to be updated e.g.
curl --request PATCH -H 'Content-type: application/json' -H 'Authorization: Bearer ' http://<IP_ADDRESS>:3039/data/find-pension-contact-details/digital-take-up/<_id of document in a data set collection>' -d '{"count": 200}'
Multiple key-value pairs can be supplied via the -d argument

👍  This is such a good idea!


⚠ SceneTime has degraded performance

In bdf8106, SceneTime ($ST) experienced degraded performance:

HTTP code: 200
Response time: 3730 ms

Resolved: SceneTime performance has improved in 95c8396 after 6 minutes.


[AutoPR containerregistry/resource-manager] Azure Container Registry Auto Build Feature Swagger and examples

Created to sync https://github.com/Azure/azure-rest-api-specs/pull/2378

This PR has been merged into https://github.com/Azure/azure-sdk-for-go/pull/1689


The request for path /MS/DEV/signalr/negotiate failed with code InternalServerError. Reason: Internal Server Error

AL Extension Version: <IP_ADDRESS>991

Server Version: 14

Even when you try with a blank project from AL:GO is the same behaviour.
[2019-06-27 18:43:57.80] Publishing AL application using launch configuration 'Microsoft cloud sandbox'.
[2019-06-27 18:43:57.84] Authenticating...
[2019-06-27 18:43:58.58] Authenticated as user<EMAIL_ADDRESS>Please note that these credentials are cached. Clear the credentials cache to authenticate as another user.
[2019-06-27 18:43:58.58] Targeting Dynamics 365 Business Central.
[2019-06-27 18:48:08.20] The request for path /MS/DEV/signalr/negotiate failed with code InternalServerError. Reason: Internal Server Error

Hi! Let's get the issue to a team who will fix and backport it, as this repository is only for issues related to the AL compiler and developer tools of the latest developer preview environment for Dynamics 365 Business Central. We suggest that you open a support case to ensure that all Business Central users benefit from your catch as soon as possible.
To open a support case, you can:

Open a support request to CSS through the PartnerSource portal

Contact your Service Account Manager (SAM) in your local subsidiary to understand what is included in your contract as of support incident and PAH (Partner Advisory Hours). Your SAM might also direct you "step by step" how to open a support request or how to get credentials, if this is the first time for you or your company


Windows installer does not work

Hi There,
Several people have reported this problem, so I'm creating a dedicated issue.
The installer Windows fail without any notification.
In result, only the uninstaller is present in the "C:\Users\Benjamin\AppData\Local\Programs\ove-electron".
The app from MS store works fine.
But unfortunately, the use of the MS store is not possible for us (security restriction)
This occurs both on:
Windows 10 Pro - 22H2 (Build 19045.2913).
Windows 10 Education - 22H2 (Build 19045.2846).

Is there a debug mode for the installer?
Edit:
Looking into "OpenVectorEditor-Setup-1.5.5.exe", the $PLUGINSDIR dir contain "app-arm64.7z", Is this a mistake or is it deliberate? (there is no other binary in the installer, except the uninstaller)

Hi @Norde thanks for the issue. I haven't had a ton of bandwidth to look into the installer issues recently. I'll let you know when I get a moment to dig in further.

Hi @tnrich, Thanks for your concern!
I stay tuned.

Hi @Norde thanks for the issue. I haven't had a ton of bandwidth to look into the installer issues recently. I'll let you know when I get a moment to dig in further.

Hi @tnrich, did you have some time to give a look on the installer issue?

Hi @Norde I haven't looked into this issue still. Sorry


lbmap: Initialize maps before test suite runs

This commit adds the missing initialization needed for BPF maps à la
d9fa628ea ("daemon, lbmap: Avoid premature init of BPF maps"). This
fixes the following test failure in the privileged tests:
$ sudo -E make -j $(nproc) TESTPKGS=pkg/maps/lbmap tests-privileged
...
=== RUN   Test
START: maglev_test.go:43: MaglevSuite.SetUpSuite
PASS: maglev_test.go:43: MaglevSuite.SetUpSuite 0.000s

START: maglev_test.go:73: MaglevSuite.TestInitMaps
level=info msg="Deleting Maglev outer map due to different M or empty map" bpfMapName=cilium_lb4_maglev subsys=map-lb
level=info msg="Deleting Maglev outer map due to different M or empty map" bpfMapName=cilium_lb4_maglev subsys=map-lb
level=info msg="Deleting Maglev outer map due to different M or empty map" bpfMapName=cilium_lb4_maglev subsys=map-lb
... Panic: runtime error: invalid memory address or nil pointer dereference (PC=0x4399D8)

/usr/lib/go/src/runtime/panic.go:969
  in gopanic
/usr/lib/go/src/runtime/panic.go:212
  in panicmem
/usr/lib/go/src/runtime/signal_unix.go:742
  in sigpanic
/home/chris/code/cilium/cilium/pkg/bpf/map_linux.go:464
  in Map.OpenOrCreate
lbmap.go:551
  in updateServiceEndpoint
lbmap.go:119
  in LBBPFMap.UpsertService
maglev_test.go:103
  in MaglevSuite.TestInitMaps
/usr/lib/go/src/reflect/value.go:337
  in Value.Call
/usr/lib/go/src/runtime/asm_amd64.s:1374
  in goexit
PANIC: maglev_test.go:73: MaglevSuite.TestInitMaps

OOPS: 0 passed, 1 PANICKED
--- FAIL: Test (0.11s)
FAIL
coverage: 10.4% of statements in github.com/cilium/cilium/pkg/maps/lbmap
FAIL    github.com/cilium/cilium/pkg/maps/lbmap 0.123s
FAIL
make: *** [Makefile:146: tests-privileged] Error 1

Fixes: d9fa628ea
Signed-off-by: Chris Tarazi<EMAIL_ADDRESS>

retest-runtime

Why we didn't see this failure in the CI? Because the maps were initialized by other test suite?

Why we didn't see this failure in the CI? Because the maps were initialized by other test suite?

Not sure, I was wondering the same. There was discussion about it here: https://cilium.slack.com/archives/C7PE7V806/p1612302871017900

At this point, we have code owner approval and we only needed the runtime tests to pass. Marking as ready to merge.


Fixed 'is undefined' error and prop checking in Counter example

Added method binding in constructor + prop checking.

Thanks!

@Krizzu Just out of curiosity, why is binding in constructor better?

It's not like it's better, but changed it because eslint was throwing 'undefined` errors.

This is actually caused by the upgrade to eslint 4.0 at the top level and CRA using the wrong version: https://github.com/facebookincubator/create-react-app/issues/2604#issuecomment-311168618.
I'll likely revert this once https://github.com/facebookincubator/create-react-app/issues/2644 lands.

Ok, thanks!


Can the Ecospold version be imported into SimaPro directly?

Hello,
Is it possible to import the ecospold version directly into the SimaPro? Is there a corresponding mapping file?
https://github.com/uslci-admin/uslci-content/blob/dev/downloads/uslci_fy20_q2_01_olca1_10_3_ecospold2.zip
Thanks,
Xun

USLCI admin does not have access to a SimaPRO license to determine if a mapping file is required or to create an associated mapping file. I am contacting Long Trail (ITexperts) to inquire. I will post their feedback here.

@xliao2010 : I am still waiting on Pre to respond to my inquiries. However, NREL did receive the following notice regarding the publishing of DataSMART 2020 has been published:

The DATASMART 2020 Update is now available!
The DATASMART 2020 Update is now available!

The DATASMART 2020 Update is now available!

The DATASMART 2020 Update includes 11 new processes, 75 updated processes, and 11 corrected processes. This includes new molybdenum data from the International Molybdenum Association. Electricity mixes for Canadian provinces and territories, the Canadian country average, U.S. eGRID, U.S. state and the U.S. country average were also updated to incorporate the most recently published data. Two new waste scenarios for general waste and packaging waste were created based on the most recent publications from the U.S. Environmental Protection Agency. The LTS Method was updated with the latest ReCiPe 2016 version 1.04 method. Finally, corrected water flows were corrected for Chinese textile processing system level data. Learn more about these updates in a free webinar on November 18th from noon-1pm ET. Email us at<EMAIL_ADDRESS>to register.
The DATASMART 2020 Update includes 11 new processes, 75 updated processes, and 11 corrected processes. This includes new molybdenum data from the International Molybdenum Association. Electricity mixes for Canadian provinces and territories, the Canadian country average, U.S. eGRID, U.S. state and the U.S. country average were also updated to incorporate the most recently published data. Two new waste scenarios for general waste and packaging waste were created based on the most recent publications from the U.S. Environmental Protection Agency. The LTS Method was updated with the latest ReCiPe 2016 version 1.04 method. Finally, corrected water flows were corrected for Chinese textile processing system level data. Learn more about these updates in a free webinar on November 18th from noon-1pm ET. Email us at<EMAIL_ADDRESS>to register.

The DATASMART 2020 Update includes 11 new processes, 75 updated processes, and 11 corrected processes. This includes new molybdenum data from the International Molybdenum Association. Electricity mixes for Canadian provinces and territories, the Canadian country average, U.S. eGRID, U.S. state and the U.S. country average were also updated to incorporate the most recently published data. Two new waste scenarios for general waste and packaging waste were created based on the most recent publications from the U.S. Environmental Protection Agency. The LTS Method was updated with the latest ReCiPe 2016 version 1.04 method. Finally, corrected water flows were corrected for Chinese textile processing system level data. Learn more about these updates in a free webinar on November 18th from noon-1pm ET. Email us at<EMAIL_ADDRESS>to register.
Update Now | Update Now
Update Now
The DATASMART 2020 update is now available for you to download! Please find your credentials and instructions below, and the DATAMSART What's New, update/installation manual and manual attached.  1.    Log-on to our website at http://ltsexperts.com/wp-admin        User name: NREL        Password: z&8391fTrC(uZ@0t6gqsZk%2. Then go to https://ltsexperts.com/download/datasmart-download/3. Next, enter LTS2020DS for the password to the right of "DATASMART2020" and click the green download button.4. Follow the instructions attached to restore the DATASMART SimaPro backup file and import INTO your database. | The DATASMART 2020 update is now available for you to download! Please find your credentials and instructions below, and the DATAMSART What's New, update/installation manual and manual attached.  1.    Log-on to our website at http://ltsexperts.com/wp-admin        User name: NREL        Password: z&8391fTrC(uZ@0t6gqsZk%2. Then go to https://ltsexperts.com/download/datasmart-download/3. Next, enter LTS2020DS for the password to the right of "DATASMART2020" and click the green download button.4. Follow the instructions attached to restore the DATASMART SimaPro backup file and import INTO your database.
The DATASMART 2020 update is now available for you to download! Please find your credentials and instructions below, and the DATAMSART What's New, update/installation manual and manual attached.  1.    Log-on to our website at http://ltsexperts.com/wp-admin        User name: NREL        Password: z&8391fTrC(uZ@0t*6gqsZk%2. Then go to https://ltsexperts.com/download/datasmart-download/3. Next, enter LTS2020DS for the password to the right of "DATASMART2020" and click the green download button.4. Follow the instructions attached to restore the DATASMART SimaPro backup file and import INTO your database.
Submit Data | Submit Data
Submit Data
Are you interested in having your data in DATASMART?You can choose to publish your data at the unit or system level (we will need to review the unit level), and have the option to be listed as the generator (or not!). Follow these procedures to get started. Please email us at<EMAIL_ADDRESS>if you have any questions.  Long Trail Sustainability |<PHONE_NUMBER> | info@ltsexperts.comhttps://ltsexperts.com/ | Are you interested in having your data in DATASMART?You can choose to publish your data at the unit or system level (we will need to review the unit level), and have the option to be listed as the generator (or not!). Follow these procedures to get started. Please email us at<EMAIL_ADDRESS>if you have any questions.  Long Trail Sustainability |<PHONE_NUMBER> |<EMAIL_ADDRESS>https://ltsexperts.com/
Are you interested in having your data in DATASMART?You can choose to publish your data at the unit or system level (we will need to review the unit level), and have the option to be listed as the generator (or not!). Follow these procedures to get started. Please email us at<EMAIL_ADDRESS>if you have any questions.  Long Trail Sustainability |<PHONE_NUMBER> | info@ltsexperts.comhttps://ltsexperts.com/

After discussion with Pre, the developers of SimaPRO, I understand that special mapping files ARE required to import the current, FEDEFL-adapted version of the USLCI directly into SimaPRO. Pre intends to facilitate updating SimaPRO for this capacity in the coming year. Also, the GLAD nomenclature working group is currently working on bi-directional mapping harmonization for 4 key flow lists (ILCD, ecoinvent, IDEA, FEDEFL); see this GLAD progress article. Having the GLAD bi-directional mappings may accelerate the adaptation of the FEDEFL, and thus, updated USLCI into SimaPRO.


setup plotter demo

This PR adds a lona-chartjs demo showing a function plotter. This demo was originally contributed by @markusgritsch in #367

@markusgritsch: I created a commit as you for the original code, so the attribution does not get lost. I only did some basic changes, so the linter is happy. Then i ported the code to lona-picocss, and created a small clip of the demo.
Let me know whether you are ok with these changes.

Codecov Report
Patch coverage: 100.00% and project coverage change: -0.44 :warning:

Comparison is base (be4c722) 69.00% compared to head (b2548d4) 68.57%.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #369      +/-   ##
==========================================
- Coverage   69.00%   68.57%   -0.44%
==========================================
  Files          68       68
  Lines        5166     5168       +2
  Branches     1164     1156       -8
==========================================
- Hits         3565     3544      -21
- Misses       1325     1351      +26
+ Partials      276      273       -3

Impacted Files
Coverage Δ

lona/html/nodes.py
89.43% <100.00%> (+0.17%)
:arrow_up:

... and 13 files with indirect coverage changes
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

I get this traceback:
Traceback (most recent call last):
  File "C:\Users\mgr\Desktop\lona-test\pl2.py", line 3, in <module>
    from lona_picocss.html import TextInput, Label, HTML, Grid, Sub, H1
ImportError: cannot import name 'Sub' from 'lona_picocss.html'```

Thanks for the honor of adding the example to Lona :)


Upgrade python and tools to recent version

Unsure if we really want to do this for ST3, but this PR will track the impact:

[x] flake8: E252 missing whitespace around parameter equals on argument default values.
[x] mypy: two small issues (incorrect format() call and new need for a type annotation.
[x] unittest (mock) was using non-existing methods "assert_called_once" and "assert_not_called" (and silently passing)
[x] Checking formatting correctness with yapf --diff works

Coverage decreased (-0.05%) to 38.06% when pulling 52de49bb00b12d6ceb2987eec556a4bb8c62af51 on update-travis-python into 832390056b5bb8e0cf646b2179a47ab9a4bacab0 on master.

I think the updates to mock and mypy has already caught more mistakes than expected, so I'd like to merge this shortly

What about tox.ini? Does it need any updates? I like to have my local dev environment in sync with travis as much as possible.

Good point, updated and tested tox ( I don't use it myself much)


Parse SDK sending POST instead of GET when trying to fetch objects

Hi There,
I am trying to integrate the build posted here: https://github.com/ParsePlatform/Parse-SDK-dotNET/issues/166#issuecomment-217977259
I am having issues with fetching a list of objects, I do it like this:
var query = ParseObject.GetQuery("HiScore").Limit(100).OrderByDescending("score").WhereGreaterThanOrEqualTo("createdAt", time);

query.FindAsync().ContinueWith(t => {
	IEnumerable<ParseObject> scores = t.Result;
	cache.Clear();

	if(scores != null) {
		// Map the scores to ScoreEntry objects
		foreach(ParseObject score in scores) {
			if(Manager.debug) Debug.Log ("Parse loop -> " + score["coins"] + " - " + score["kills"] + " - " + score["nickName"]);
		}
	}
});

But I dont get any kind of reply or callback in this example. What's weirder is that my Server logs tell me this:
141.101.x.x - - [27/Jul/2016:10:10:52 +0200] "POST /parse/classes/HiScore?order=-score&limit=100 HTTP/1.1" 301 178 "-" "UnityPlayer/5.3.6f1 (http://unity3d.com)"

It is doing a POST request while I THINK it should be doing a GET request (because I try to find objects)
Is there anyone who know what's going on? Thanks!

Fetching a list seems to work fine for me. Posted my code snippet below if that helps
`
IEnumerable leaders = new List();
	ParseQuery<ParseObject> parseData = ParseObject.GetQuery ("GameName").OrderByDescending ("HighScore").Limit(10);
	Task queryTask = parseData.FindAsync().ContinueWith (t => {
		leaders = t.Result;

	});

	while (!queryTask.IsCompleted) yield return null;

	if (queryTask.IsFaulted) {
		//Data Unreachable; Show Last stored Leaderboard
	}
	else if (!queryTask.IsCanceled) {

                //Do Something with retrieved Leaderboard here
	}

`
I am using a forked version of the 1.7 Unity SDK and parse-server.
What version of the SDK are you using?

@amplebrain Hi there, I didn't have the luxury of building the SDK myself (humble linux user) so some good people have done it for me, see: https://github.com/ParsePlatform/Parse-SDK-dotNET/issues/166#issuecomment-217832665 and https://github.com/ParsePlatform/Parse-SDK-dotNET/issues/166#issuecomment-217834669
Neither worked for me, I'll try your example soon, maybe it will work suddenly :)

@amplebrain really weird, but today I'm not managing to even fetch anything at all! Gosh, ever since parse went down it's nothing but trouble for me. I never managed to get the server and the unity client to work with eachother. It's a disaster and am considering rolling my own backend since it's just 1 project using parse :(

What error does Unity throw? Can you try to catch the ParseExceptions and see if they tell you anything?
What version of parse server are you running?
Do you get any response when you try to CURL?
Can you post your server config here?
So far, parse-server has been fairly stable my test deployment but I havent migrated most of my users yet.

@amplebrain It doesnt even request anything at all. It used to work a little (as in: I could see the Unity SDK doing requests to my server).
But none of it matters now as I rolled my own simple PHP script and replaced Parse functions with simple use of WWW objects to make requests to them.
I know it's not really solving the problem but at least now I know for sure it works. I just didn't want to spend any more hours on it.
But thanks for the help, I'm glad it works for you! :)


Proposal to use basic auth in headers instead of body for console API

The C# client code generator produces incorrect code from the console API because it doesn't know how to generate a method for the /v2/console/authenticate operation.
This happens because the operation in the console.swagger.json omits BasicAuth as a security option and opts to authenticate via a body param in lieu of the headers.
https://github.com/heroiclabs/nakama/blob/master/console/console.swagger.json#L737
The following would make the client generate a successful method:
(1) Remove the consoleAuthenticateRequest parameter.
(2) Use BasicAuth as the security option.
Of course, the client code generator could be modified to accommodate this, but frankly it would make the code a lot more brittle. If this is something that is not a big deal to do on the backend, that would be better. But always up for thinking about alternatives.

@lugehorsam I think this is a good move especially if it standardizes out the differences when we utilize our codegen tools. I think we go for it but must make the console UI changes to accommodate it too.

Great, I can do it next week.


feat: use json schema from openapi doc and recover in case of assistant protocol error

/deploy-review

/deploy-review

/deploy-review


May I ask what is wrong in the picture? There is no error in compiling the installation, but there is an error in installing extension. The Linux version is Centos7.9.

Contact Details
<EMAIL_ADDRESS>What's the question?
error pictur

e
Relevant log output or information
No response
Code of Conduct

[X] I agree to follow this project's Code of Conduct

Hi @weiya1990 , can you share more details about the environment and the steps done to reproduce this issue?


Create dedicated docs for "Stablecoins on Hedera"

### Tasks
- [ ] https://github.com/hashgraph/hedera-docs/issues/422
- [ ] https://github.com/hashgraph/hedera-docs/issues/423
- [ ] https://github.com/hashgraph/hedera-docs/issues/425
- [ ] Get feedback from the SME's for stablecoin section
- [ ] Review and publish documentation

Duplicate


Goal conditioning via FiLM?

Authors,
First, I greatly appreciate your insightful paper and well-written code!
My questions is in regards to goal-conditioning.  In section 3.1 of your paper (Network Architecture Options) when discussing the CNN-based Diffusion Policy, you mention:
"However, goal conditioning is still possible with the same FiLM conditioning method used for observations."
I wonder if you could comment along these lines a bit?  As the trajectory sampled is the conditional probability p(A|O), would one simply encode the goal observation and concat it to the initializing observations Ot?  Or do you mean something else, like a restructuring to sample the joint trajectory p(A, O)?  I'm sorry, I think I'm missing something.
Thanks very much for your time,
Robert Mash

Hi! I'm not the author, but I can offer my understanding.
I believe your first guess is correct: We model p(A|O,g), where g is the goal. In the case of a goal-reaching task, g would be the coordinates for the goal position.
Concretely, in the case of their U-Net, this would mean appending the goal-tensor to the observations and passing the whole tensor to the FiLM conditioning. Please let me know if I'm wrong here :)


[Spotify-Controls] Display notification when track changes

Description

Allows user to enable notifications for Next Track and Previous Track commands
Screencast

Checklist

[x] I read the extension guidelines
[x] I read the documentation about publishing
[x] I ran npm run build and tested this distribution build in Raycast
[x] I checked that files in the assets folder are used by the extension itself
[x] I checked that assets used by the README are placed outside of the metadata folder

Do I have to add new command preferences to CHANGELOG.md? I think so. 😳

Hey thanks for the contribution. How about changing the notifcation to a HUD? That feels better integrated with the Raycast UI.

I think it's good practice to add a changelog entry. It's a new feature that we should highlight properly. Otherwise existing users might not know about it.

Thanks for the addition! Looking forward to see my next tracks :-)


LSTM/GRU performance discrepancy

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): yes
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Fedora32/Colab/Ubuntu 19.04
TensorFlow installed from (source or binary): binary
TensorFlow version (use command below): 2.2.0rc4/2.2.0rc3/2.1.0
Python version: 3.7.6/3.7.4/3.6.9
CUDA/cuDNN version: 10.1
GPU model and memory: GTX 970 4GB/Colab GPU/NVIDIA Tesla P100 16GB

Describe the current behavior
I'm experimenting with models that require the state of LSTMs to be processed after after each time step. So I tried unrolling the LSTM layer in a loop, iterating over the time steps. That's were I noticed a huge increase in time per training step. Uppon further investigation I noticed that this is also the case in some other configuration featuring LSTMs/GRUs. Different versions of TensorFlow are effected differently. See below:
Some results:

Colab
Compute Cloud
Local

Keras Sequential with LSTMs
17ms/step
15ms/step
17ms/step

Keras Sequential with GRUs
17ms/step
14ms/step
16ms/step

Custom Model with LSTMs
17ms/step
14ms/step
17ms/step

Custom Model with GRUs
88ms/step
14ms/step
40ms/step

Custom Model with LSTMs in loop
120ms/step
173ms/step
81ms/step

Custom Model with GRUs in loop
107ms/step
153ms/step
66ms/step

Custom Model with compat LSTMs
19ms/step
15ms/step
17ms/step

Custom Model with compat GRUs
20ms/step
15ms/step
17ms/step

Custom Model with compat LSTMs in loop
87ms/step
108ms/step
62ms/step

Custom Model with compat GRUs in loop
78ms/step
86ms/step
49ms/step

Colab: TF 2.2.0rc3, Python 3.6.9, with GPU
Compute Cloud: TF 2.1.0, Python 3.7.4, Cuda 10.1, Nvidia Tesla P100, Ubuntu 19.04
Local: TF 2.2.0rc4, Python 3.7.6, Cuda 10.1, Nvidia GTX 970, Fedora 32

Describe the expected behavior
Time per step should be more or less similar.
Standalone code to reproduce the issue
Colab Notebook

@AwekasGerald
I ran the code shared and face an error, please find the gist here

Please make sure to use a colab runtime with GPU!
(runtime -> change runtime type -> select "GPU" as hardware accelerator)

@AwekasGerald
I ran the code on gpu, please find the gist here and let us know if this confirms your issue.

Yes, it does.

Thanks for reporting the issue. I think the performance discrepancy is expected between a python while loop vs a fused LSTM/GRU layer.
In TF2, we use cudnn kernels in tf.keras.layers.LSTM/GRU, which will execute the whole timesteps in one op. This is usually much more fast than an unrolled for loop, which will execute the individual timesteps one by one (while will be slower than tf.while).


More animations

Add an option where we can change animation types.
Ideas:

Hourglass, where speed of animation is set by time from start to finish
Video example: link

Also, found some great animations that could be used: link, like if you're a chemist, you may like a beaker, if you're a mechanic, you may like a gear etc... Hourglass on page 12

This is not something I'm personally interested in -- the Cuppa Mobile app and macOS apps are companion apps sharing the same teacup graphics and tea-darkening animation, and I prefer to keep it that way. I also do not have plans to add other animations to the macOS app. I have to focus my time and energy elsewhere. Sorry about that!


Install fails because uamqp wheel build fails

I want to install azure-iot-hub in a virtual environment (pip install azure-iot-hub), but the build of uamqp wheel fails.
Is there something wrong with the installation method?
M2 Pro
MacOS 14.4.1
python 3.9.17
      uamqp/c_uamqp.c:90791:109: warning: passing 'const IO_INTERFACE_DESCRIPTION *' (aka 'const struct IO_INTERFACE_DESCRIPTION_TAG *') to parameter of type 'IO_INTERFACE_DESCRIPTION *' (aka 'struct IO_INTERFACE_DESCRIPTION_TAG *') discards qualifiers [-Wincompatible-pointer-types-discards-qualifiers]
        __pyx_t_2 = ((struct __pyx_vtabstruct_5uamqp_7c_uamqp_XIO *)__pyx_v_xio->__pyx_vtab)->create(__pyx_v_xio, __pyx_v_interface, ((PyObject *)__pyx_v_io_config), (&__pyx_v_io_config->_c_value)); if (unlikely(!__pyx_t_2)) __PYX_ERR(24, 39, __pyx_L1_error)
                                                                                ^~~~~~~~~~~~~~~~~
      uamqp/c_uamqp.c:111209:3: warning: unused label 'bad' [-Wunused-label]
        bad:
        ^~~~
      35 warnings and 1 error generated.
      error: command '/usr/bin/clang' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for uamqp
Failed to build uamqp
ERROR: Could not build wheels for uamqp, which is required to install pyproject.toml-based projects

azure-iot-hub depends on azure-uamqp-python package which is deprecated and fails to build on certain systems including Mac ARM and Linux ARM on Python 3.12+.
As a stopgap, I developed an small alternative package azure-iot-hub-api that has basic functionalities to use in my own project until the azure-iot-hub receive necessary updates. I tired to keep it as similar as possible to azure-iot-hub.

How is there still no official solution? @cartertinney , is this SDK discontinued?

this package is not maintained...
I took advantage of this package’s functionality by ignoring build errors. Because only some features were needed.
CFLAGS="-Wno-error=incompatible-function-pointer-types" pip install azure-iot-hub

Thank you @sugawarayss . Are you aware which features are affected?

@timozerrer
I don’t know if this is correct, but I think it probably affects the part where C2D messages are sent. However, I was able to send C2D messages even if I ignored the build error.

This workaround does not seem to work on Python 3.12: https://github.com/Azure/azure-uamqp-python/issues/386#issuecomment-2051754769
I can confirm build is still not running using Python 3.12 and M3

fmmm...
i use Python 3.9.17 on M2 MacOS device...

Is there any official update this issue. This is really big gap for anyone using Mac as a development machine. Python SDKs are supposed to be fully cross platform.


Snippets are deprecated

Type: Performance Issue
Resolve this problem of snippets.
VS Code version: Code 1.93.1 (38c31bc77e0dd6ae88a4e9cc93428cc27a56ba40, 2024-09-11T17:20:05.685Z)
OS version: Windows_NT x64 10.0.22631
Modes:

Thank you for reaching VS Marketplace Support Team. We are following up to determine how best to address it.
-VS Marketplace Team

Thank you for opening the issue. Could you please share more details about your concern?
It appears to be specific to VS Code editor and tools if we understand correctly. It is best to reach out VS Code team at  https://github.com/microsoft/vscode/issues.
Regards,
VS Marketplace Team


[VS Code] 'Create new mapping' button is not working inside VS Code UI

open AtlasMap
import a source (from an xml file for instance)
import a target (from the same xml file for instance)
on one of the field in source, click on the top-right button for it Create new mapping
--> nothing happens, the right panel is not opening, there is no error in console log

it is working using Chrome.
I reproduced with this example https://github.com/atlasmap/atlasmap/blob/master/standalone/src/test/resources/json-schema-source-to-xml-schema-target.adm used in AtlasMap tests

workaround:

click on the line
navigate with tab to the "mapping" button
click enter

I spent a good chunk of time on this today and I can say that the issue is in the iframe nesting that is causing some issues:

there is no console.log, and the dev tools inspector can't inspect AtlaMap's iframe at all
single clicking on the fields icons doesn't work (but double-clicking them works for reasons)
drag & drop, you can start dragging but it stops immediately

I have the feeling something else could be broken.
I then tried loading AtlasMap by injecting the required js files directly and it works, but because of how the standalone app is packaged it's not a straight-forward change.
I'll update this or open a PR when I have news.


KivyMD version(0.104.1) dropdown in not working

hello guys I have updated my kivyMD version(0.104.0) a few days ago and my MDdropdown was working fine before updating but in latest MD version(0.104.1) it is not working so do you guys having any solutions for it? please fix it because it is not working on kivyMD demo app too
below is the same code which I am using
https://pastebin.com/GDPEXmGk

@navjotcis https://kivymd.readthedocs.io/en/latest/components/menu/index.html#center-position

@HeaTTheatR, I have tried using above giving URL for the dropdown problem and it makes the dropdown working but it have some problems so do you guys have any permanent fix for it

These errors appears multiple times when someone try to open the dropdown.


Allow general discrete grids

In this PR, I add the functionality to allow general discrete grids.
Previously, only index grids were allowed. Here we add an updating step, which takes a model containing an arbitrary discrete grid, overwrites the grid with a corresponding index grid, and adds a function to the model that transforms the index values to the correct label. This means that the user can now use arbitrary grids, while internally, we can still use the convenient indexer grids.
Closes #82.
ToDo

[x] Implement model updating and add first tests
[x] Refactor code in process_model.py into single model-updating call
[x] Add unit tests
[x] Test simulation
[x] Check if we can integrate #87.
-> This is possible, but will mainly be an interface problem. I will write up a few ideas in another LEP.

Should provide only the following option to create discrete grids:
@dataclass
class RetirementStatus:
    working: int = 0
    retired: int = 1

DiscreteGrid(RetirementStatus)


Trouble with Raspiblitz setup

I am having two issues.  First, I have tried two screens and they both only turn on and display white.  I setup the raspiblitz anyway via SSH.  However for 2 1/2 days it has been stuck at:
Node is Syncing (0)
Blockchain Progress: 0.00%
Lightning Progress: .01%
I checked the error log and it appears that it is not connecting.  I forwarded port 8333 and rebooted but it does not appear to have made a difference.
From the log:
2020-12-13T23:03:46Z Potential stale tip detected, will try using extra outbound                                                                                peer (last tip update: 1891 seconds ago)
2020-12-13T23:14:16Z Potential stale tip detected, will try using extra outbound                                                                                peer (last tip update: 2521 seconds ago)
2020-12-13T23:24:46Z Potential stale tip detected, will try using extra outbound                                                                                peer (last tip update: 3151 seconds ago)
2020-12-13T23:27:29Z Socks5() connect to uz3pvdhie3372vxw.onion:8333 failed: gen                                                                                eral failure
2020-12-13T23:35:16Z Potential stale tip detected, will try using extra outbound                                                                                peer (last tip update: 3781 seconds ago)
2020-12-13T23:45:46Z Potential stale tip detected, will try using extra outbound                                                                                peer (last tip update: 4411 seconds ago)
2020-12-13T23:47:20Z Socks5() connect to uz3pvdhie3372vxw.onion:8333 failed: gen

From the log:
2020-12-13T23:03:46Z Potential stale tip detected, will try using extra outbound                                                                                peer (last tip update: 1891 seconds ago)
2020-12-13T23:14:16Z Potential stale tip detected, will try using extra outbound                                                                                peer (last tip update: 2521 seconds ago)
2020-12-13T23:24:46Z Potential stale tip detected, will try using extra outbound                                                                                peer (last tip update: 3151 seconds ago)
2020-12-13T23:27:29Z Socks5() connect to uz3pvdhie3372vxw.onion:8333 failed: gen                                                                                eral failure
2020-12-13T23:35:16Z Potential stale tip detected, will try using extra outbound                                                                                peer (last tip update: 3781 seconds ago)
2020-12-13T23:45:46Z Potential stale tip detected, will try using extra outbound                                                                                peer (last tip update: 4411 seconds ago)
2020-12-13T23:47:20Z Socks5() connect to uz3pvdhie3372vxw.onion:8333 failed: gen

Please close this issue if solved.


tests: add system tests for python-compute

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

[ ] Make sure to open an issue as a bug/issue before writing your code!  That way we can discuss the change, evaluate designs, and agree on the general idea
[x] Ensure the tests and linter pass
[ ] Code coverage does not decrease (if any source code was changed)
[ ] Appropriate docs were updated (if necessary)

Fixes #<issue_number_goes_here> 🦕

@georgiyekkert could you remove the tests/test_system.py as well? It is just a dummy test for testing mtls before, now it is no longer needed.


Windows wallet: voting choice display truncates information and refuses to use available space.

Voting UI linebreaks at ~32 characters thus ignoring a large amount of available space available to the interface.    Menu choices should wordwrap according to user window selected size.
Screenshot as to why this became an issue on one of the last votes.

This will be fixed with the new voting UI coming up in the next version.


latestMajorVersionForPackage fails if url has the form ~/Documents/somepath/Package

Marathon currently does not support local packages with a tilde in the path:
import HTMLParser // marathon:~/Documents/somepath/HTMLParser

I am not quite sure where the patch should  land, in Marathon itself:
class PackageManager {
    private func latestMajorVersionForPackage(at url: URL) throws -> Int {
        ....
        let releases = try perform(Releases.versions(for: url.expandPath).withoutPreReleases(),
                                   orThrow: Error.failedToResolveLatestVersion(url))
        ....
    }
}

or in the Releases package:
class Releases {
    private static func string(from url: URL) throws -> String {
        ....
        let path = url.absoluteString.replacingOccurrences(of: "file://", with: "").expandPath
        ....
    }
}

I tried both patches, both work for me.
extension String {

    fileprivate var _expandPath: String? {
        guard hasPrefix("~/"), let home = ProcessInfo.processInfo.environment["HOME"] else { return nil }
        return self.replacingOccurrences(of: "~/", with: home + "/")
    }

    var expandPath: String {
        return _expandPath ?? self
    }
}

extension URL {

    var expandPath: URL {
        guard let expanded = absoluteString._expandPath, let url = URL(string: expanded) else { return self }
        return url
    }
}

Hi @jazzbox this looks great.
I think it might need to be done with the Files API though, which is currently a dependency for marathon.

@jazzbox , does it actually support local libraries without the tilde? In my tests it does not seem to work and I couldn't find documentation about it.

@lf-araujo yes, it works. You have to specify the full path like here
import HTMLParser // marathon:/Users/box/Documents/dev/myprojects/Marathon/HTMLParser
And you should test that your library can be imported in a small demo SwiftPM project. If you have made a mistake (for example you forgot to version tag your library) Marathon gives total misleading and unhelpful error messages.
That and the fact, that marathon is and will be always lagging behind SwiftPM, I think that it is not worth the effort. I am using Marathon for proof-of-concept and prototyping now, but when I'm finished, I'm moving to a SwiftPM project (and version control of course).

@jazzbox not worth the effort? a little respect goes a long way. Not everyone deserves respect but I think John has earned it don’t you think?
Marathon might not be as reliable as main stream services but it’s free of charge.
you was using marathon in your release/production workflow? I don’t recommend using relatively unpopular and experimental open source ideas like this in production. Just use this for a bit of scripting here or there.

That and the fact, that marathon is and will be always lagging behind SwiftPM, I think that it is not worth the effort. I am using Marathon for proof-of-concept and prototyping now, but when I'm finished, I'm moving to a SwiftPM project (and version control of course).

I mean, marathon makes small scripts very easy to test and run, that is what I use it for. Perhaps you are using it to a too large of a project, where SPM may perform better.
@rob-nash In Linux the directory address is not working is it perhaps related to the fact that marathon tries to use -C, which is now deprecated?
warning: '--chdir/-C' option is deprecated; use '--package-path' instead
Thanks.

@rob-nash Did we had a conversation? And who are you that you can tell me how I have to structure my work?
And wow "experimental idea", that is really disrespectful.

Hi @lf-araujo not entirely sure about linux 🙄 sorry
@jazzbox relax. you have done some good work here. if you want to use marathon in a production environment, then go ahead.


Compile for iOS

I need to compile this library for iPhone. Target architecture is arm64 and armv7. I set default clang compiler to use and arm64 arch.
make CC=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang ARCH=arm64

It fails with error: make[1]: *** No rule to make target "../kernel/arm64/amax.S", needed by "samax_k.o".  Stop.
It's successfully compile if I do not set ARCH=arm64. Can you give advice how to compile library with arm64 and armv7 support.

Also if I add arch flag to compiler, so compile going like so:
Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -O2 -arch arm64 -miphoneos-version-min=5.0 -DMAX_STACK_ALLOC=2048 -DEXPRECISION  -Wall -m64 -DF_INTERFACE_GFORT -fPIC -DSMP_SERVER -DNO_WARMUP -DMAX_CPU_NUMBER=8 -DASMNAME=_saxpy -DASMFNAME=_saxpy_ -DNAME=saxpy_ -DCNAME=saxpy -DCHAR_NAME=\"saxpy_\" -DCHAR_CNAME=\"saxpy\" -DNO_AFFINITY -I.. -I. -UDOUBLE  -UCOMPLEX -c axpy.c -o saxpy.o

I start get error like this.
/usr/include/sys/cdefs.h:680:2: error: Unsupported architecture
/usr/include/sys/cdefs.h:680:2#error Unsupported architecture:
/usr/include/sys/cdefs.h:680:2: error: Unsupported architecture
/usr/include/sys/cdefs.h:680:2: error: Unsupported architecture
unknown type name '__uint32_t'; did you mean '__uint128_t'?
error: unknown type name '__darwin_va_list'
error: unknown type name '__darwin_size_t'
// and others

Thank you.

Closed ticket #523 looks relevant. Also please try the develop branch if you are currently using one of the releases - some ARM-related fixes went in after 0.2.14

@AlexandrGraschenkov , I didn't try to compile OpenBLAS for iOS before.

Thank you for reply. I've try to make TARGET=ARMV8 but get error on compile:
symm.c:388:36: error: use of undeclared identifier 'QGEMM_DEFAULT_P'
gemm.c:397:37: error: use of undeclared identifier 'QGEMM_DEFAULT_P'
gemm.c:397:46: error: use of undeclared identifier 'QGEMM_DEFAULT_Q'
// and so on

With make TARGET=ARMV8 ARCH=arm64 more code get compiled but still get errors:
../kernel/arm64/../arm/scal.c:55:2: error: non-void function 'sscal_k' should return a value [-Wreturn-type]

After change to return 0; get errors:
../kernel/arm64/sgemm_kernel_4x4.S:759:19: error: unknown token in expression
        add sp,sp,#-(5*16)
                  ^
../kernel/arm64/sgemm_kernel_4x4.S:760:19: error: brackets expression not supported on this target
        stp d8,d9,[sp,#(0*16)]
                  ^
../kernel/arm64/sgemm_kernel_4x4.S:761:21: error: brackets expression not supported on this target
        stp d10,d11,[sp,#(1*16)]
                    ^
../kernel/arm64/sgemm_kernel_4x4.S:762:21: error: brackets expression not supported on this target
        stp d12,d13,[sp,#(2*16)]
                    ^
../kernel/arm64/sgemm_kernel_4x4.S:763:21: error: brackets expression not supported on this target
        stp d14,d15,[sp,#(3*16)]
                    ^
../kernel/arm64/sgemm_kernel_4x4.S:764:21: error: brackets expression not supported on this target
        stp d16,d17,[sp,#(4*16)]
                    ^
../kernel/arm64/sgemm_kernel_4x4.S:766:22: error: unexpected token in argument list
        mov x12, v0.d[0]
// and so on

Googling for the "brackets expression..." error message finds some cases where the cause appeared to be mis-detection of the target architecture, e.g.
https://github.com/project-imas/security-check/issues/3
(I assume you did issue "make clean" between your various attempts ?)

Hi Alexander,
Have you solved the compilation problem? I have exactly the same issue.

Hi @tchernitski
I'm not solved the problem. I compile one project witch uses Open-BLAS. I decide to complie project without library, and change code usage of Open-BLAS to Accelerate framework functions. But project successfully compile for iOS, and I stop dig into this problem.

Thanks for answer, but I would like to use something that is less platform-dependend

Hello,
Do you have any progress of this issue?

Hi,
I replaced OpenBlas by Eigen in my project

Hello,
Not sure if this still actual but here is how I did build for ARM64 arch.

In file c_check add following line for $cross_suffix:
$cross_suffix = "";
if (dirname($compiler_name) ne ".") {
$cross_suffix .= dirname($compiler_name) . "/";
}
if (basename($compiler_name) =~ /([^\s]-)(.)/) {
$cross_suffix .= $1;
}
$cross_suffix = "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/";

You need this because we will provide CC with arguments and Perl function for extracting directory doesn't work well.

Create file with following content.
TOOLCHAIN_PATH=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
SYSROOT_PATH=/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS10.2.sdk
make TARGET=ARMV8 BINARY=64 HOSTCC=clang CC="$TOOLCHAIN_PATH/clang -isysroot $SYSROOT_PATH -arch arm64" NOFORTRAN=1 libs

Now you can build library.

Thank you very much for this. I wonder if your addition to the c_check file can be made conditional on something like
if ( $data =~ /OS_DARWIN/ && $data =~ /ARCH_ARM/)
which should allow adding it to the current code without breaking other platforms ? Then users would only need to create your build script (which I think could be added as "quickbuild.ios", if the toolchain path can be expected to be fairly constant). Unfortunately I do not have any Apple hardware to try this myself.

I guess we need to find how get directory name from path like this: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS10.2.sdk -arch arm64. Functions like dirname() or basename() don't work well for such path.

I understand the sysroot path will contain a version number and/or target platform name, so your build script would always need to be edited by the user. Can we rely on the toolchain path that you put in c_check to be constant ? Then at least the user would not have to change any files from OpenBLAS if that line was present by default - less chance to get something wrong.

Copied your text to the wiki as https://github.com/xianyi/OpenBLAS/wiki/How-to-build-OpenBLAS-for-iPhone-iOS

@sstepashka
No, I tried to build for armv7 arch but got error about CPU/FPU features.

hi, all, I build the develop branches to use the solution, it has no error, but just generate .a library(the size only 3m) ，when I use "make" to build the x86 library，it generate 2 library(.a and .dylib, the size both more than 10m),  I want to use the dylib
by the way, it has a warning:"clang: warning: using sysroot for 'iPhoneOS' but targeting 'MacOSX'"
it's right?

@tamworth   Did you build for armv7?

@tamworth Did you build for armv7?

I've found Accelerate.framework with BLAS functions. Maybe you shouldn't use OpenBLAS for iOS?
See https://developer.apple.com/reference/accelerate/blas

Thanks,I kown Accelerate.framework.Now we want to test the Openblas and Accelerate.framework,which one is faster.

@ccy022364  I build for arm64

@tamworth googling for the error message led me to https://curl.haxx.se/mail/lib-2016-10/0094.html which seems to suggest you may have "-mmacosx-version-min" or similar set somewhere in your default clang options (but I know nothing about cross-compiling for iOS on OSX)
And if someone of you who tried to build this for armv7 could post the error message (about wrong or missing CPU features apparently ?) maybe this will provide a clue for the ARM experts here

When I use arm64 library,it failed with error,
Undefined symbols for architecture arm64:
"_sgemm_kernel", referenced from:
_sgemm_nn in libopenblas_armv8p-r0.2.20.dev.a(sgemm_nn.o)
_sgemm_nt in libopenblas_armv8p-r0.2.20.dev.a(sgemm_nt.o)
_inner_thread in libopenblas_armv8p-r0.2.20.dev.a(sgemm_thread_nn.o)
_inner_thread in libopenblas_armv8p-r0.2.20.dev.a(sgemm_thread_nt.o)
_inner_thread in libopenblas_armv8p-r0.2.20.dev.a(sgemm_thread_tn.o)
_inner_thread in libopenblas_armv8p-r0.2.20.dev.a(sgemm_thread_tt.o)
_sgemm_tn in libopenblas_armv8p-r0.2.20.dev.a(sgemm_tn.o)
...
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
thank you

@martin-frbg Thank you for your reply, it works, this is my script:
TOOLCHAIN_PATH=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin SYSROOT_PATH=/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS10.2.sdk rm -f /libs/* make clean make TARGET=ARMV8 BINARY=64 HOSTCC=clang CC="$TOOLCHAIN_PATH/clang -isysroot $SYSROOT_PATH -arch arm64 -miphoneos-version-min=8.0 -O2" NOFORTRAN=1  libs

@martin-frbg Thank you for your reply, it works, this is my script:
SYSROOT_PATH=/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS10.2.sdk
make clean
make TARGET=ARMV8 BINARY=64 HOSTCC=clang CC="$TOOLCHAIN_PATH/clang -isysroot $SYSROOT_PATH -arch arm64 -miphoneos-version-min=8.0 -O2" NOFORTRAN=1  libs

@ccy022364 This is my test project, It working on arm64 device https://github.com/tamworth/BLASCompare
In this project, I use same function to make a test between openblas and Accelerate.framewrok, It seems that openblas is far slower than Accelerate.framewrok, I'm a newbie of openblas, I don't know the results whether it's right

Try building OpenBLAS for TARGET=CORTEXA57 and compare the the performance on arm64.
Also what APIs are you comparing??

@ashwinyes I comparing with Accelerate.framewrok, it's Apple's cblas library,
I run 10000000 times on iPhone 5S with:
    double A[6] = {1.0,2.0,1.0,-3.0,4.0,-1.0};
    double B[6] = {1.0,2.0,1.0,-3.0,4.0,-1.0};
    double C[9] = {.5,.5,.5,.5,.5,.5,.5,.5,.5};
    cblas_dgemm(CblasColMajor, CblasNoTrans, CblasTrans,3,3,2,1,A, 3, B, 3,2,C,3);

It took 9s on openblas, and it only took 2s on Accelerate.framewrok
I have try to build withTARGET=CORTEXA57, It caused so many error, And I'm not sure the CORTEXA57 is applicable to iPhone

Thanks for testing.
CORTEXA57 should work on all arm64 processors having Advanced SIMD support which I believe should be there for iphone.
Could you share the compile errors that you are seeing for CORTEXA57 target ??
FYI. TARGET=ARM64 just uses the generic C files. So not much performance is to be expected. On actual cortexa57 processor, TARGET=CORTEXA57 is 3-4x times faster than TARGET=ARM64 if i remember correctly.

Also I believe the matrix sizes that you are using is very small and not appropriate for benchmarking OpenBLAS as it is optimized for larger matrices.
Would request you to try with larger matrix sizes (starting from 100x100) with lesser number if iterations.

@ashwinyes Here is the error message
<instantiation>:4:21: error: unexpected token in argument list
 beq KERNEL_F1_NEXT_\@
                    ^
<instantiation>:1:1: note: while in macro instantiation
KERNEL_F1
^
../kernel/arm64/nrm2.S:150:2: note: while in macro instantiation
 KERNEL_F8
 ^
<instantiation>:7:27: error: unexpected token in argument list
 bge KERNEL_F1_SCALE_GE_X_\@
                          ^
<instantiation>:1:1: note: while in macro instantiation
KERNEL_F1
^
../kernel/arm64/nrm2.S:150:2: note: while in macro instantiation
 KERNEL_F8
 ^
<instantiation>:13:19: error: unexpected token in argument list
 b KERNEL_F1_NEXT_\@
                  ^
<instantiation>:1:1: note: while in macro instantiation
KERNEL_F1
^
../kernel/arm64/nrm2.S:150:2: note: while in macro instantiation
 KERNEL_F8
 ^
<instantiation>:14:22: error: unknown token in expression
KERNEL_F1_SCALE_GE_X_\@:
                     ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS10.2.sdk -arch arm64 -miphoneos-version-min=8.0 -O2 -c -O2 -DMAX_STACK_ALLOC=2048 -Wall -DF_INTERFACE_GFORT -fPIC -DNO_LAPACK -DNO_LAPACKE -DSMP_SERVER -DNO_WARMUP -DMAX_CPU_NUMBER=4 -march=armv8-a+crc+crypto+fp+simd -mtune=cortex-a57 -DASMNAME=_saxpby_k -DASMFNAME=_saxpby_k_ -DNAME=saxpby_k_ -DCNAME=saxpby_k -DCHAR_NAME=\"saxpby_k_\" -DCHAR_CNAME=\"saxpby_k\" -DNO_AFFINITY -I.. -UDOUBLE  -UCOMPLEX -UCOMPLEX -UDOUBLE ../kernel/arm64/../arm/axpby.c -o saxpby_k.o
<instantiation>:1:1: note: while in macro instantiation
KERNEL_F1
^
../kernel/arm64/nrm2.S:150:2: note: while in macro instantiation
 KERNEL_F8
 ^
<instantiation>:14:22: error: invalid operand
KERNEL_F1_SCALE_GE_X_\@:
                     ^
<instantiation>:1:1: note: while in macro instantiation
KERNEL_F1
^
../kernel/arm64/nrm2.S:150:2: note: while in macro instantiation
 KERNEL_F8
 ^
../kernel/arm64/nrm2.S:87:16: error: unknown token in expression
KERNEL_F1_NEXT_\@:
               ^
<instantiation>:1:1: note: while in macro instantiation
KERNEL_F1
^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
../kernel/arm64/nrm2.S:87:16: error: invalid operand
KERNEL_F1_NEXT_\@:
               ^
<instantiation>:1:1: note: while in macro instantiation
KERNEL_F1
^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
<instantiation>:4:21: error: unexpected token in argument list
 beq KERNEL_F1_NEXT_\@
                    ^
<instantiation>:2:2: note: while in macro instantiation
 KERNEL_F1
 ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
<instantiation>:7:27: error: unexpected token in argument list
 bge KERNEL_F1_SCALE_GE_X_\@
                          ^
<instantiation>:2:2: note: while in macro instantiation
 KERNEL_F1
 ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
<instantiation>:13:19: error: unexpected token in argument list
 b KERNEL_F1_NEXT_\@
                  ^
<instantiation>:2:2: note: while in macro instantiation
 KERNEL_F1
 ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
<instantiation>:14:22: error: unknown token in expression
KERNEL_F1_SCALE_GE_X_\@:
                     ^
<instantiation>:2:2: note: while in macro instantiation
 KERNEL_F1
 ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
<instantiation>:14:22: error: invalid operand
KERNEL_F1_SCALE_GE_X_\@:
                     ^
<instantiation>:2:2: note: while in macro instantiation
 KERNEL_F1
 ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
../kernel/arm64/nrm2.S:87:16: error: unknown token in expression
KERNEL_F1_NEXT_\@:
               ^
<instantiation>:2:2: note: while in macro instantiation
 KERNEL_F1
 ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
../kernel/arm64/nrm2.S:87:16: error: invalid operand
KERNEL_F1_NEXT_\@:
               ^
<instantiation>:2:2: note: while in macro instantiation
 KERNEL_F1
 ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
<instantiation>:4:21: error: unexpected token in argument list
 beq KERNEL_F1_NEXT_\@
                    ^
<instantiation>:3:2: note: while in macro instantiation
 KERNEL_F1
 ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
<instantiation>:7:27: error: unexpected token in argument list
 bge KERNEL_F1_SCALE_GE_X_\@
                          ^
<instantiation>:3:2: note: while in macro instantiation
 KERNEL_F1
 ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
<instantiation>:13:19: error: unexpected token in argument list
 b KERNEL_F1_NEXT_\@
                  ^
<instantiation>:3:2: note: while in macro instantiation
 KERNEL_F1
 ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
<instantiation>:14:22: error: unknown token in expression
KERNEL_F1_SCALE_GE_X_\@:
                     ^
<instantiation>:3:2: note: while in macro instantiation
 KERNEL_F1
 ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
<instantiation>:14:22: error: invalid operand
KERNEL_F1_SCALE_GE_X_\@:
                     ^
<instantiation>:3:2: note: while in macro instantiation
 KERNEL_F1
 ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
../kernel/arm64/nrm2.S:87:16: error: unknown token in expression
KERNEL_F1_NEXT_\@:
               ^
<instantiation>:3:2: note: while in macro instantiation
 KERNEL_F1
 ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
../kernel/arm64/nrm2.S:87:16: error: invalid operand
KERNEL_F1_NEXT_\@:
               ^
<instantiation>:3:2: note: while in macro instantiation
 KERNEL_F1
 ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
<instantiation>:4:21: error: unexpected token in argument list
 beq KERNEL_F1_NEXT_\@
                    ^
<instantiation>:4:2: note: while in macro instantiation
 KERNEL_F1
 ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
<instantiation>:7:27: error: unexpected token in argument list
 bge KERNEL_F1_SCALE_GE_X_\@
                          ^
<instantiation>:4:2: note: while in macro instantiation
 KERNEL_F1
 ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
<instantiation>:13:19: error: unexpected token in argument list
 b KERNEL_F1_NEXT_\@
                  ^
<instantiation>:4:2: note: while in macro instantiation
 KERNEL_F1
 ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
<instantiation>:14:22: error: unknown token in expression
KERNEL_F1_SCALE_GE_X_\@:
                     ^
<instantiation>:4:2: note: while in macro instantiation
 KERNEL_F1
 ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
<instantiation>:14:22: error: invalid operand
KERNEL_F1_SCALE_GE_X_\@:
                     ^
<instantiation>:4:2: note: while in macro instantiation
 KERNEL_F1
 ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
../kernel/arm64/nrm2.S:87:16: error: unknown token in expression
KERNEL_F1_NEXT_\@:
               ^
<instantiation>:4:2: note: while in macro instantiation
 KERNEL_F1
 ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
../kernel/arm64/nrm2.S:87:16: error: invalid operand
KERNEL_F1_NEXT_\@:
               ^
<instantiation>:4:2: note: while in macro instantiation
 KERNEL_F1
 ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
<instantiation>:4:21: error: unexpected token in argument list
 beq KERNEL_F1_NEXT_\@
                    ^
<instantiation>:5:2: note: while in macro instantiation
 KERNEL_F1
 ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
<instantiation>:7:27: error: unexpected token in argument list
 bge KERNEL_F1_SCALE_GE_X_\@
                          ^
<instantiation>:5:2: note: while in macro instantiation
 KERNEL_F1
 ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
<instantiation>:13:19: error: unexpected token in argument list
 b KERNEL_F1_NEXT_\@
                  ^
<instantiation>:5:2: note: while in macro instantiation
 KERNEL_F1
 ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
<instantiation>:14:22: error: unknown token in expression
KERNEL_F1_SCALE_GE_X_\@:
                     ^
<instantiation>:5:2: note: while in macro instantiation
 KERNEL_F1
 ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
<instantiation>:14:22: error: invalid operand
KERNEL_F1_SCALE_GE_X_\@:
                     ^
<instantiation>:5:2: note: while in macro instantiation
 KERNEL_F1
 ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
../kernel/arm64/nrm2.S:87:16: error: unknown token in expression
KERNEL_F1_NEXT_\@:
               ^
<instantiation>:5:2: note: while in macro instantiation
 KERNEL_F1
 ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
../kernel/arm64/nrm2.S:87:16: error: invalid operand
KERNEL_F1_NEXT_\@:
               ^
<instantiation>:5:2: note: while in macro instantiation
 KERNEL_F1
 ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
<instantiation>:4:21: error: unexpected token in argument list
 beq KERNEL_F1_NEXT_\@
                    ^
<instantiation>:6:2: note: while in macro instantiation
 KERNEL_F1
 ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
<instantiation>:7:27: error: unexpected token in argument list
 bge KERNEL_F1_SCALE_GE_X_\@
                          ^
<instantiation>:6:2: note: while in macro instantiation
 KERNEL_F1
 ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
<instantiation>:13:19: error: unexpected token in argument list
 b KERNEL_F1_NEXT_\@
                  ^
<instantiation>:6:2: note: while in macro instantiation
 KERNEL_F1
 ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
<instantiation>:14:22: error: unknown token in expression
KERNEL_F1_SCALE_GE_X_\@:
                     ^
<instantiation>:6:2: note: while in macro instantiation
 KERNEL_F1
 ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
<instantiation>:14:22: error: invalid operand
KERNEL_F1_SCALE_GE_X_\@:
                     ^
<instantiation>:6:2: note: while in macro instantiation
 KERNEL_F1
 ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
../kernel/arm64/nrm2.S:87:16: error: unknown token in expression
KERNEL_F1_NEXT_\@:
               ^
<instantiation>:6:2: note: while in macro instantiation
 KERNEL_F1
 ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
../kernel/arm64/nrm2.S:87:16: error: invalid operand
KERNEL_F1_NEXT_\@:
               ^
<instantiation>:6:2: note: while in macro instantiation
 KERNEL_F1
 ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
<instantiation>:4:21: error: unexpected token in argument list
 beq KERNEL_F1_NEXT_\@
                    ^
<instantiation>:7:2: note: while in macro instantiation
 KERNEL_F1
 ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
<instantiation>:7:27: error: unexpected token in argument list
 bge KERNEL_F1_SCALE_GE_X_\@
                          ^
<instantiation>:7:2: note: while in macro instantiation
 KERNEL_F1
 ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
<instantiation>:13:19: error: unexpected token in argument list
 b KERNEL_F1_NEXT_\@
                  ^
<instantiation>:7:2: note: while in macro instantiation
 KERNEL_F1
 ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
<instantiation>:14:22: error: unknown token in expression
KERNEL_F1_SCALE_GE_X_\@:
                     ^
<instantiation>:7:2: note: while in macro instantiation
 KERNEL_F1
 ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
<instantiation>:14:22: error: invalid operand
KERNEL_F1_SCALE_GE_X_\@:
                     ^
<instantiation>:7:2: note: while in macro instantiation
 KERNEL_F1
 ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^                            pplications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS10.2.sdk -arch arm64 -miphoneos-version-min=8.0 -O2 -c -O2 -DMAX_STACK_ALLOC=2048 -Wall -DF_INTERFACE_GFORT -fPIC -DNO_LAPACK -DNO_LAPACKE -DSMP_SERVER -DNO_WARMUP -DMAX_CPU_NUMBER=4 -march=armv8-a+crc+crypto+fp+simd -mtune=cortex-a57 -DASMNAME=_sgemv_n -DASMFNAME=_sgemv_n_ -DNAME=sgemv_n_ -DCNAME=sgemv_n -DCHAR_NAME=\"sgemv_n_\" -DCHAR_CNAME=\"sgemv_n\" -DNO_AFFINITY -I.. -UDOUBLE  -UCOMPLEX -UDOUBLE -UCOMPLEX  -UTRANS ../kernel/arm64/gemv_n.S -o sgemv_n.o
31merror: unknown token in expression
KERNEL_F1_NEXT_\@:
               ^
<instantiation>:7:2: note: while in macro instantiation
 KERNEL_F1
 ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
../kernel/arm64/nrm2.S:87:16: error: invalid operand
KERNEL_F1_NEXT_\@:
               ^
<instantiation>:7:2: note: while in macro instantiation
 KERNEL_F1
 ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
<instantiation>:4:21: error: unexpected token in argument list
 beq KERNEL_F1_NEXT_\@
                    ^
<instantiation>:8:2: note: while in macro instantiation
 KERNEL_F1
 ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
<instantiation>:7:27: error: unexpected token in argument list
 bge KERNEL_F1_SCALE_GE_X_\@
                          ^
<instantiation>:8:2: note: while in macro instantiation
 KERNEL_F1
 ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
<instantiation>:13:19: error: unexpected token in argument list
 b KERNEL_F1_NEXT_\@
                  ^
<instantiation>:8:2: note: while in macro instantiation
 KERNEL_F1
 ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
<instantiation>:14:22: error: unknown token in expression
KERNEL_F1_SCALE_GE_X_\@:
                     ^
<instantiation>:8:2: note: while in macro instantiation
 KERNEL_F1
 ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
<instantiation>:14:22: error: invalid operand
KERNEL_F1_SCALE_GE_X_\@:
                     ^
<instantiation>:8:2: note: while in macro instantiation
 KERNEL_F1
 ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
../kernel/arm64/nrm2.S:87:16: error: unknown token in expression
KERNEL_F1_NEXT_\@:
               ^
<instantiation>:8:2: note: while in macro instantiation
 KERNEL_F1
 ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
../kernel/arm64/nrm2.S:87:16: error: invalid operand
KERNEL_F1_NEXT_\@:
               ^
<instantiation>:8:2: note: while in macro instantiation
 KERNEL_F1
 ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:146:2: note: while in macro instantiation
 KERNEL_F8
 ^
<instantiation>:4:21: error: unexpected token in argument list
 beq KERNEL_F1_NEXT_\@
                    ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:159:2: note: while in macro instantiation
 KERNEL_F1
 ^
<instantiation>:7:27: error: unexpected token in argument list
 bge KERNEL_F1_SCALE_GE_X_\@
                          ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:159:2: note: while in macro instantiation
 KERNEL_F1
 ^
<instantiation>:13:19: error: unexpected token in argument list
 b KERNEL_F1_NEXT_\@
                  ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:159:2: note: while in macro instantiation
 KERNEL_F1
 ^
<instantiation>:14:22: error: unknown token in expression
KERNEL_F1_SCALE_GE_X_\@:
                     ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS10.2.sdk -arch arm64 -miphoneos-version-min=8.0 -O2 -c -O2 -DMAX_STACK_ALLOC=2048 -Wall -DF_INTERFACE_GFORT -fPIC -DNO_LAPACK -DNO_LAPACKE -DSMP_SERVER -DNO_WARMUP -DMAX_CPU_NUMBER=4 -march=armv8-a+crc+crypto+fp+simd -mtune=cortex-a57 -DASMNAME=_sgemv_t -DASMFNAME=_sgemv_t_ -DNAME=sgemv_t_ -DCNAME=sgemv_t -DCHAR_NAME=\"sgemv_t_\" -DCHAR_CNAME=\"sgemv_t\" -DNO_AFFINITY -I.. -UDOUBLE  -UCOMPLEX -UDOUBLE -UCOMPLEX  -DTRANS  ../kernel/arm64/gemv_t.S -o sgemv_t.o
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:159:2: note: while in macro instantiation
 KERNEL_F1
 ^
<instantiation>:14:22: error: invalid operand
KERNEL_F1_SCALE_GE_X_\@:
                     ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:159:2: note: while in macro instantiation
 KERNEL_F1
 ^
../kernel/arm64/nrm2.S:87:16: error: unknown token in expression
KERNEL_F1_NEXT_\@:
               ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:159:2: note: while in macro instantiation
 KERNEL_F1
 ^
../kernel/arm64/nrm2.S:87:16: error: invalid operand
KERNEL_F1_NEXT_\@:
               ^
/var/folders/d4/hc88wzv923s4b0ck9vpwrlm80000gp/T/nrm2-afe453.s:159:2: note: while in macro instantiation
 KERNEL_F1
 ^
make[1]: *** [snrm2_k.o] Error 1
make[1]: *** Waiting for unfinished jobs....
make: *** [libs] Error 1

I build script with:
TOOLCHAIN_PATH=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
SYSROOT_PATH=/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS10.2.sdk
make clean
make TARGET=CORTEXA57 BINARY=64 HOSTCC=clang CC="$TOOLCHAIN_PATH/clang -isysroot $SYSROOT_PATH -arch arm64 -miphoneos-version-min=8.0 -O2" NOFORTRAN=1  libs

Thank you

Thanks for the logs.
Looks like the only files that are causing the errors are kernel/arm64/nrm2.S and kernel/arm64/znrm2.S. clang dont seem to understand the @ token in macros used to create unique labels.
You can comment out the following lines in kernel/arm64/KERNEL.CORTEXA57 and do a clean make to get rid of the compilation errors.
SNRM2KERNEL  = nrm2.S
DNRM2KERNEL  = nrm2.S
CNRM2KERNEL  = znrm2.S
ZNRM2KERNEL  = znrm2.S
Thanks

@ashwinyes Thank you for your reply, The library is successful compile with your solution, But when I run with Xcode, It caused some error:
Undefined symbols for architecture arm64:
  "_dgemm_incopy", referenced from:
      _inner_thread in libopenblas_cortexa57p-r0.2.20.dev.a(dgemm_thread_tn.o)
      _inner_thread in libopenblas_cortexa57p-r0.2.20.dev.a(dgemm_thread_tt.o)
      _dgemm_tn in libopenblas_cortexa57p-r0.2.20.dev.a(dgemm_tn.o)
      _dgemm_tt in libopenblas_cortexa57p-r0.2.20.dev.a(dgemm_tt.o)
  "_dgemm_otcopy", referenced from:
      _dgemm_nt in libopenblas_cortexa57p-r0.2.20.dev.a(dgemm_nt.o)
      _inner_thread in libopenblas_cortexa57p-r0.2.20.dev.a(dgemm_thread_nt.o)
      _inner_thread in libopenblas_cortexa57p-r0.2.20.dev.a(dgemm_thread_tt.o)
      _dgemm_tt in libopenblas_cortexa57p-r0.2.20.dev.a(dgemm_tt.o)
  "_dgemm_oncopy", referenced from:
      _dgemm_nn in libopenblas_cortexa57p-r0.2.20.dev.a(dgemm_nn.o)
      _inner_thread in libopenblas_cortexa57p-r0.2.20.dev.a(dgemm_thread_nn.o)
      _inner_thread in libopenblas_cortexa57p-r0.2.20.dev.a(dgemm_thread_tn.o)
      _dgemm_tn in libopenblas_cortexa57p-r0.2.20.dev.a(dgemm_tn.o)
  "_dgemm_itcopy", referenced from:
      _dgemm_nn in libopenblas_cortexa57p-r0.2.20.dev.a(dgemm_nn.o)
      _dgemm_nt in libopenblas_cortexa57p-r0.2.20.dev.a(dgemm_nt.o)
      _inner_thread in libopenblas_cortexa57p-r0.2.20.dev.a(dgemm_thread_nn.o)
      _inner_thread in libopenblas_cortexa57p-r0.2.20.dev.a(dgemm_thread_nt.o)
  "_dgemm_kernel", referenced from:
      _dgemm_nn in libopenblas_cortexa57p-r0.2.20.dev.a(dgemm_nn.o)
      _dgemm_nt in libopenblas_cortexa57p-r0.2.20.dev.a(dgemm_nt.o)
      _inner_thread in libopenblas_cortexa57p-r0.2.20.dev.a(dgemm_thread_nn.o)
      _inner_thread in libopenblas_cortexa57p-r0.2.20.dev.a(dgemm_thread_nt.o)
      _inner_thread in libopenblas_cortexa57p-r0.2.20.dev.a(dgemm_thread_tn.o)
      _inner_thread in libopenblas_cortexa57p-r0.2.20.dev.a(dgemm_thread_tt.o)
      _dgemm_tn in libopenblas_cortexa57p-r0.2.20.dev.a(dgemm_tn.o)
      ...
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Now this is a tougher problem.
All the above undefined references are implemented in assembly file for TARGET=CORTEXA57, but are implemented in C files for TARGET=ARMV8. So, I think the linker is unable to find any symbols implemented in assembly files.
I believe you will face the same issue if you try to use SGEMM in TARGET=ARMV8 as its implemented in assembly file.
Looks like OpenBLAS is lacking support for iOS compilation and this needs to be added. I don't have much idea how compilation/linking works in iOS. So will take time for me to figure out as I dont even have a development environment for iOS.
If any iOS experts are here, they will be able to figure this out quickly.
Thanks

A quick fix that you can try is in the following branch.
https://github.com/ashwinyes/OpenBLAS/commits/ios_experiment_20170312
The above is based on http://stackoverflow.com/questions/28109826/arm64-using-gas-on-ios
Also, please note that the iOS ARM64 ABI has some divergences from the Standard Procedure Call Standard for the ARM 64-bit Architecture (AAPCS64) . Please refer below link.
https://developer.apple.com/library/content/documentation/Xcode/Conceptual/iPhoneOSABIReference/Articles/ARM64FunctionCallingConventions.html
This may result in unexpected errors as OpenBLAS assembly implementations for ARM64 does not follow the iOS ABI.
Thanks

I wonder if this is only a problem with clang's built-in assembler, i.e. if adding "-no-integrated-as" to the CFLAGS would help in any way ?

@martin-frbg  No matter ARMV8 or CORTEXA57 also error caused When I used -no-integrated-as.
If I used CORTEXA57 and didn't comment out like this:  https://github.com/ashwinyes/OpenBLAS/commit/db92f4ab791278a96c0a6ff2fe4c444f8f46dc80#diff-6b7ad97b93aa3fd3e259f94aafba00fb
the error is:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/as: can't specifiy -Q with -arch arm64
clang: error: assembler command failed with exit code 1 (use -v to see invocation)
make[1]: *** [scopy.o] Error 1
make[1]: *** Waiting for unfinished jobs....
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/as: can't specifiy -Q with -arch arm64
clang: error: assembler command failed with exit code 1 (use -v to see invocation)
make[1]: *** [sscal.o] Error 1
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/as: can't specifiy -Q with -arch arm64
clang: error: assembler command failed with exit code 1 (use -v to see invocation)
make[1]: *** [sswap.o] Error 1
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/as: can't specifiy -Q with -arch arm64
clang: error: assembler command failed with exit code 1 (use -v to see invocation)
make[1]: *** [saxpy.o] Error 1
make: *** [libs] Error 1

@ashwinyes I successfully build and run in ARMV8.
I used to print all functions with 'nm' at ARMV8 and CORTEXA57,  Not any undefined symbols function in ARMV8, but in CORTEXA57 can be found like this:
U _dgemm_incopy

Please see the stackoverflow link that I posted in my previous comment.
iOS has an additional underscore at the beginning of the function name which is not followed in the arm64 assembly files. Please try the fix that I posted which adds this underscore to the function names in the assembly files.

@tamworth Did you get the opportunity to test the code at https://github.com/ashwinyes/OpenBLAS/commits/ios_experiment_20170312
If yes, please let me know whether it worked for you.
Thanks

@ashwinyes @tamworth  Is it successful?
Thanks

Hi, I am trying to make Openblas library to work on my project:
@ccy022364 Did you manage to resolve your issue:
Showing All Messages
ld: symbol(s) not found for architecture arm64
"_sgemm_kernel", referenced from ...
I have following issue:
My project uses dlib with BLAS but linking fails with the same message.
Weird is that I used https://github.com/tamworth/BLASCompare project and Openblas library that I built behaves the same way as the one in that XCode project, it works without issues (also prebuilt Openblas library from that project fails to link in my project). Does anyone knows what might be the issue.

Hello,
Not sure if this still actual but here is how I did build for ARM64 arch.

In file c_check add following line for $cross_suffix. $cross_suffix = "";

 if (dirname($compiler_name) ne ".") {
     $cross_suffix .= dirname($compiler_name) . "/";
 }

 if (basename($compiler_name) =~ /([^\s]*-)(.*)/) {
     $cross_suffix .= $1;
 }

 $cross_suffix = "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/";

You need this because we will provide CC with arguments and Perl function for extracting directory doesn't work well.

Create file with following content. TOOLCHAIN_PATH=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
 SYSROOT_PATH=/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS10.2.sdk
 make TARGET=ARMV8 BINARY=64 HOSTCC=clang CC="$TOOLCHAIN_PATH/clang -isysroot $SYSROOT_PATH -arch arm64" NOFORTRAN=1 libs

Now you can build library.

Thank you so much


Hotplug network ifaces based on the current domain info, rather than on the interface status

What this PR does / why we need it:
We were currently hotplugging interfaces based on the interfaces present
in the VMI.Spec.Interfaces, but not present in the
VMI.Status.Interfaces, having its PodConfigDone attribute enabled.
This commit changes this, by hotplugging based on interfaces present in
the VMI.Spec.Interfaces, but not presently available in the
domain, which is more accurate.
Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Special notes for your reviewer:
/sig network
Release note:

Compute the interfaces to be hotplugged based on the current domain info, rather than on the interface status.

/test pull-kubevirt-e2e-k8s-1.24-sig-network
/test pull-kubevirt-e2e-k8s-1.25-sig-network
/test pull-kubevirt-e2e-k8s-1.26-sig-network
/test pull-kubevirt-unit-test

@EdDev I think you'll like this.
... although this PR probably should be split in 4:

one for replacing the attach device call
one for fixing the virtIO net prohibited bug
one for refactoring to allow unit tests
final one for replacing the hotplug criteria from the launcher's perspective (take into account the running domain).

/test pull-kubevirt-e2e-k8s-1.24-sig-network
/test pull-kubevirt-e2e-k8s-1.25-sig-network
/test pull-kubevirt-e2e-k8s-1.26-sig-network

/test pull-kubevirt-e2e-k8s-1.24-sig-network
/test pull-kubevirt-e2e-k8s-1.25-sig-network
/test pull-kubevirt-e2e-k8s-1.26-sig-network

/test pull-kubevirt-e2e-k8s-1.24-sig-network
/test pull-kubevirt-e2e-k8s-1.25-sig-network
/test pull-kubevirt-e2e-k8s-1.26-sig-network
Worked 3 times before.

/test pull-kubevirt-e2e-k8s-1.24-sig-network
/test pull-kubevirt-e2e-k8s-1.25-sig-network
/test pull-kubevirt-e2e-k8s-1.26-sig-network
Worked 4 times before.

/test pull-kubevirt-e2e-k8s-1.24-sig-network
/test pull-kubevirt-e2e-k8s-1.25-sig-network
/test pull-kubevirt-e2e-k8s-1.26-sig-network
Worked 5 times before.

/test pull-kubevirt-e2e-k8s-1.24-sig-network
/test pull-kubevirt-e2e-k8s-1.25-sig-network
/test pull-kubevirt-e2e-k8s-1.26-sig-network
Worked 6 times before.

/test pull-kubevirt-e2e-k8s-1.26-sig-network-multus-v4

/test pull-kubevirt-unit-test

/approve

@maiqueb regarding [1] and [2], I hoped we could save getting back to it and do these changes when new code is introduced, but I don't know how much work it will take so I guess it can be improved later on.
/lgtm


The DataStream data downloaded in the Export has wrong JSON format and missing data

When exporting data, in the data-streams.json file I wold expect to get correctly formatted JSON according to the CARP Core domain model. But - alas - there are issues.
Here is an example of some JSON exported:
data-streams.json

  {
    "id": 4961,
    "data_stream_id": 631,
    "snapshot": {
      "syncPoint": {
        "synchronizedOn": "1970-01-01T00:00:00Z",
        "relativeClockSpeed": 1.0,
        "sensorTimestampAtSyncPoint": 0
      },
      "triggerIds": [
        4
      ],
      "measurements": [
        {
          "data": {
            "__type": "dk.cachet.carp.freememory",
            "freeVirtualMemory":<PHONE_NUMBER>,
            "freePhysicalMemory": 164835328
          },
          "sensorStartTime":<PHONE_NUMBER>872632
        },
        {
          "data": {
            "__type": "dk.cachet.carp.freememory",
            "freeVirtualMemory":<PHONE_NUMBER>,
            "freePhysicalMemory": 117329920
          },
          "sensorStartTime":<PHONE_NUMBER>857247
        },
        {
          "data": {
            "__type": "dk.cachet.carp.freememory",
            "freeVirtualMemory":<PHONE_NUMBER>,
            "freePhysicalMemory": 124141568
          },
          "sensorStartTime":<PHONE_NUMBER>870355
        },
        {
          "data": {
            "__type": "dk.cachet.carp.freememory",
            "freeVirtualMemory":<PHONE_NUMBER>,
            "freePhysicalMemory": 113905664
          },
          "sensorStartTime":<PHONE_NUMBER>889803
        }
      ]
    },
    "first_sequence_id": 29,
    "last_sequence_id": 32,
    "created_by": "20e2093c-e510-4e79-9385-478a19dc4723",
    "created_at": "2024-10-25T13:54:43.080504Z",
    "updated_by": "20e2093c-e510-4e79-9385-478a19dc4723",
    "updated_at": "2024-10-25T13:54:43.081204Z"
  },
  {
    "id": 4605,
    "data_stream_id": 655,
    "snapshot": {
      "syncPoint": {
        "synchronizedOn": "1970-01-01T00:00:00Z",
        "relativeClockSpeed": 1.0,
        "sensorTimestampAtSyncPoint": 0
      },
      "triggerIds": [
        9
      ],
      "measurements": [
        {
          "data": {
            "__type": "dk.cachet.carp.triggeredtask",
            "control": "Start",
            "taskName": "Task #17",
            "triggerId": 9,
            "destinationDeviceRoleName": "Polar HR Sensor"
          },
          "sensorStartTime":<PHONE_NUMBER>592930
        }
      ]
    },
    "first_sequence_id": 1,
    "last_sequence_id": 1,
    "created_by": "20e2093c-e510-4e79-9385-478a19dc4723",
    "created_at": "2024-10-24T13:31:24.977530Z",
    "updated_by": "20e2093c-e510-4e79-9385-478a19dc4723",
    "updated_at": "2024-10-24T13:31:24.979868Z"
  },
...

Compared to the CARP Core Domain data model there are several issues with this JSON:

The JSON uses a mixture of camelCase and snake_case serialization. Default JSON in CARP Core is always camelCase. For example, first_sequence_id should be firstSequenceId (see DataStreamSequence).
The data_stream_id (which should be dataStreamId) is a JSON object (and not a number). See DataStreamId for details.
The JSON contains data that is not part of the CARP Core domain model. For example id, last_sequence_id, created_by, etc.
The snapshot is not part of the DataStreamSequence domain model. However, the SyncPoint is but is embedded in this weird "snapshot" thing.

@yuanchen233 - I think the main problem here is that data is exported from the database directly rather than from the DataStreamService. We have talked about this before - the approach of bypassing the domain logic in CAPR Core, and I really do not like this. It goes against everything that DDD recommends.
I am sure @Whathecode will agree ;-)

Yup. The endpoints on that service are severely lacking, however, but this should be a prompt to add/work on those.

The JSON uses a mixture of camelCase and snake_case serialization. Default JSON in CARP Core is always camelCase. For example, first_sequence_id should be firstSequenceId (see DataStreamSequence).

This is caused by JsonNaming is set to SnakeCaseStrategy, and all column generated from Auditable is using the snake naming schema. It is unsure to me if those extra field is still needed.

The data_stream_id (which should be dataStreamId) is a JSON object (and not a number). See DataStreamId for details.
The JSON contains data that is not part of the CARP Core domain model. For example id, last_sequence_id, created_by, etc.
The snapshot is not part of the DataStreamSequence domain model. However, the SyncPoint is but is embedded in this weird "snapshot" thing.

This is cuased by webservices' wired design of dataStream database table/entity, we have this 3 table and the notion of snapshot inherited from old Java project, and also plan to change it at #137

I think the main problem here is that data is exported from the database directly rather than from the DataStreamService. We have talked about this before - the approach of bypassing the domain logic in CAPR Core, and I really do not like this. It goes against everything that DDD recommends.

Agreed. Currently, the export endpoint ( the export button on the portal ) performs a database dump. It does not utilize the API of each ApplicationService to retrieve data, and this should be changed. This change was proposed but postponed due to the need to maintain a consistent format for the exported data of active studies.

Yup. The endpoints on that service are severely lacking, however, but this should be a prompt to add/work on those.

Most of the issues are caused by the current design of database, given that they works, retriving data through core endpoints like DataStreamServicesRequest.GetDataStream should work properly without exposing those extra information, despite the fact that the database do need some re-design.

In short, I think there are a few things we could/should do, some of them might be off topic here:

[ ] Use core dataStream endpoint to retrieve data, not export
[ ] Export feature in webservice should also use core endpoint to retrieve data, instead of a database dump
[ ] Re-design dataStream database, stop using this added snapshot
[ ] Revisit Auditable and decided if it's still needed for the current implementation of webservices
[ ] At somepoint, a re-design of all database is also desired. Currently all data is saved as snapshots which is created using core Json serializer. This is not very efficient and does not utilize any advantage a database provide, and the process time is way too long when the number of instances go up. (e.g. ReruitmentSnapshot keeps a list of all participantGroup, when we have 20000 participantGroups and a new participantGroup is created, it takes huge amount of time to build the Recruitment object, updata it and convert it back to Json and save it)

If we decide to re-design the database later, some word of suggestion from @Whathecode would be very helpful and much appreciated.

Current webservices' dataStream database implementation: