Partial parsing

Here is an attempt at providing partial parsing support. It's orthogonal to the event parsing branch I proposed before so those changes are not included here.
There are still problems for me with this branch:

7ed92d18a4404d12119b3ace116d3df7f19d48f1: I'm coming from a C programming background, so I'm still not used to the C++ way of doing things. I'd like to modify the stacked value in place but for some reason it's not working so I pop them and push them back again, I suppose there is a better way.

Testing: I did some test locally but I did not modify the test.cpp, on purpose. It's a bit of a mess and I wanted to know how and what you'd want me to add as tests.

I also suppose you'll have other remarks and they'd be welcomed.

Automated message from Dropbox CLA bot
@canatella, thanks for the pull request! It looks like you haven't yet signed the Dropbox CLA. Please sign it here.

This PR is to go along with PR https://github.com/dropbox/json11/pull/83.
We develop mobile applications. We use json11 and djinni to connect to REST apis. We have a lot of data to transfert at first application startup on possibly bad connection (mobile data),  and we want to be able to display the data ASAP. This PR allows parsing the json body without having to buffer the entire HTTP response body. As soon as there is data, you can feed the parser with it by calling consume and the parser will continue where it left. Used in conjunction with PR PR https://github.com/dropbox/json11/pull/83, it will trigger event as soon as any JSON value is available. With these two PR, we could then also make the parser drop already parsed code to limit memory usage when parsing large json strings, which is part of my use case. They could also allow parsing an io directly without having to buffer the entire string in memory.
From an architectural point of view, we need to be able to restart parsing from where we left at any point. For that,

we need to keep the full parser state up to date and self contained
we need to make the parsing loop use the state information.

At the api level, this implies that we expose a parser object with the aforementioned state information that the user can manage by itself, and a function to feed data to the parser.
The first commit deals with this api change, it creates a public parser class which hold the current parser implementation as private member and has a consume method.
The next commits are mostly changes needed to store the full parsing state, besides commit https://github.com/dropbox/json11/pull/88/commits/eb177d0053b0b69a3a510d1958b27768de430283 which uses the introduced state information to provide the depth limit check.
Finally, the last commit is the one making the parsing loop explicit.
I'll add some test that feed the parser with one character at a time and some documentation and examples in the header file.

Okay, I understand your intent, and how this fits together with your other PR.  I'm a bit iffy on merging this feature on it's own since it feels somewhat half-baked without the other pieces you describe to make a useful system.  I'd be okay with it as a half-way step to a final good state, though, barring any concerns about safety or performance I might have on deeper review.
I'll wait for your next update with docs and examples before doing the full review.

I can continue the patch series with the freeing of already parsed string and the event parsing and the doc and example for everything if you prefer.

I could go either way.  I see the value of bite-sized pieces for me to review, and to built/tested separately.  But I also prefer the final public result is something worth using.  Possible compromises might be to do multiple stages of review in the same PR (I'm not sure how well GItHub supports this flow) or to merge multiple PRs onto a public branch then merge into master when all the planned work is done.  The branch might be the easiest option for us to collaborate, if you want to make this a multi-part project.
My overall philosophy on PR merges/reviews (and on the internal code-reviews I do at Dropbox): Strictly, each public merge should not be a step backwards.  No merging code which is broken, messy, unperformant, etc. with promises to fix it later.  Less strictly, I prefer if each public merge is a step forward, meaning no half-baked features which aren't really ready for use yet, but there can be exceptions to this.  The point so ensure that users who pull for the first time don't get a broken/confusing intermediate state, and that the code isn't left in the bad state if an author gets busy and doesn't finish the next step.

I added some documentation in the README.md
I added two tests where I used the already existing test results and compare them to the result of parsing the stream one char at a time. I also feeded a big (+/- 300) corpus of various valid json object to it from https://github.com/jdorfman/awesome-json-datasets without problems.
I added some other fixes detected by testing parsing json with comments enabled.
This patch set can be applied on his own has it does not change anything for current users: api and behaviour are the same except for the chunked parsing addition. So you could either apply it to master where it would get a bit of exposure, or apply it to another branch. I should be able to provide fixes in cases of trouble in the next weeks but I think I won't have the time to update the event parsing PR and the freeing of already parsed data quickly as I have more urgent stuff now.
Speaking of which, what would be the best way to free up the already parsed strings, and at which point would it be best to execute ? I could see the use for a circular buffer here but there are no implementation in std, and it might add too much complexity. Other solution I see is using a list of string, that way we can easily drop parsed strings, or replace the string at some point with a substring,  but that would entail a lot of memory operations I think.

Some high-level feedback after the deeper (but still not complete) code-review.  It seems like your long-term goal is a good one, but the usability of this current step on its own isn't clear yet.  It also seems that you've already diverged quite far from the original json11 code (it's not a simple 1:1 translation which is obviously correct, for instance), and will have gone far further down that path by the time you're done.  In particular you're moving away from the "tiny library" descriptor which the README starts with.
By the time you're done I feel the library you'll be left with will be a newly-written library inspired by json11, so I wonder what the value is of calling it json11 at that point, vs. forking off and publishing your own thing under another name.  I don't feel like I'll give much value as gatekeeper on your efforts, given I/Dropbox isn't motivated to use them.  I want to encourage you to think about just going your own way and building the advanced parsing engine you want without worrying about what fits in with our needs.
One possible more limited integration would be if you might want to provide a more advanced parser which produces json11 objects as output, rather than replacing the original parser outright.  In that case your parser could be a separate add-on library of its own, or maybe eventually a separate optional class/file in the json11 repo.  If there are aspects of the json11 data which hold you back from doing so (like a missing interface for building the inner value types) I would be open to PRs which make json11 more open to replaceable parsers.  I.e. make small changes to json11 to enable you and others to build on top of it freely, rather than making large changes to json11 to do exactly what you want.

Thank you for the review. I have a bit more time now but before going further, I want to say that if you don't want to pursue on this, that's no problem for me, the code is there and it's working for my use case. I wanted to share back the code but if you feel it's a burden for you, I can just keep it in my repository. I just don't want to end up going through all the review process to finally have the PR rejected.
So to make things clear, I'm willing to go forward with the review process if it's going to be accepted and from your previous comment it seems that it's more for you to say that.

I'd be interested to hear from the original author @j4cbo on his interest in such a submission.  From my point of view, Dropbox doesn't have a need for this, so I'd have to take on the CR and validation of this as a side project, and if this is going beyond Dropbox territory into community/side-project territory I feel like @j4cbo has more ownership here than I do.


Dropzone doesn’t work on IE, shows obj error.

Hi,
I was working with Dropzone and found that it doesn’t work on Internet Explorer(IE).
I have installed vue2-dropzone version 3.5.9 and on running it on IE (version 11.192.16299.0), it shows the following error:
`[object Error]
SyntaxError: Expected ':'
{
[functions]: ,
proto: { },
description: "Expected ':'",
message: "Expected ':'",
name: "SyntaxError",
number: -2146827285,
stack: "SyntaxError: Expected ':'
at ./node_modules/vue2-dropzone/dist/vue2Dropzone.js (http://localhost:8080/47.js:10:1)
at webpack_require (http://localhost:8080/app.js:767:12)
at fn (http://localhost:8080/app.js:130:13)
at eval code (eval code:2:22)
at ./node_modules/cache-loader/dist/cjs.js?!./node_modules/babel-loader/lib/index.js!./node_modules/cache-loader/dist/cjs.js?!./node_modules/vue-loader/lib/index.js?!./src/components/Widgets/Dropzone.vue?vue&type=script&lang=js& (http://localhost:8080/9.js:11:1)
at webpack_require (http://localhost:8080/app.js:767:12)
at fn (http://localhost:8080/app.js:130:13)
at eval code (eval code:2:22)
at ./src/components/Widgets/Dropzone.vue?vue&type=script&lang=js& (http://localhost:8080/9.js:71:1)
at webpack_require (http://localhost:8080/app.js:767:12)",
Symbol()_m.5oe50ldzm01: undefined,
Symbol()_n.5oe50ldzm01: undefined,
Symbol()_o.5oe50ldzm01: undefined,
Symbol()_p.5oe50ldzm01: undefined,
Symbol()_q.5oe50ldzm01: undefined,
Symbol()_u.5oe50ldzm01: undefined,
Symbol(foo)_v.5oe50ldzm01: undefined,
Symbol(Symbol.dispose)_s.5oe50ldzm01: undefined,
Symbol(Symbol.patternMatch)_t.5oe50ldzm01: undefined,
Symbol(test)_r.5oe50ldzm01: undefined
}`
Thanks in advance. Any kind of help to resolve this issue is appreciable.

Try dropping back to version 3.5.2, there are some know IE compatibility problems (outlined in #439 , #443) with the current release.


cloud-init 0.7.6 instead of 0.7.9 since november

and no fix or no mention on the blog page

Latest documentation on cloud-init is now version 0.7.7, that is another reason to update it.

What's the exact problem here? Just the version numbers?

@StefanScherer the cloud-config (user-data) file refers to 0.79 for the documentation.
From what @bootsektor writes I understand that the November version had cloud-init 0.79 installed.
The actual version of cloud-init on the hypriot image (hypriotos-rpi-v1.7.1.img) is 0.76.

Just flash a 1.7.1 image. Yes, cloud-init package is 0.7.6 due to Raspbian Jessie. We're preparing a Raspbian Stretch release with HypriotOS 1.8.0, there will be cloud-init 0.7.9 installed.


Add troubshooting section for OMI 100% CPU

Replicating content here: https://github.com/microsoft/OMS-Agent-for-Linux/pull/951

@aliabuckner : Thanks for your contribution! The author, @mgoedtel, has been notified to review your proposed change.

Reviewed the proposed changes and #sign-off to approve it for release.

hi @aliabuckner - Microsoft authors and contributors need to make content contributions through the private repository so the content can be staged and validated by the current validation rules.
This PR is almost too large to review without the tools we have in the private repo, but as a courtesy I've done that and we'll merge this PR here.
In the future, however, can you submit changed to the private repos?
In private, after you review staging and fix all validation issues, you can tag an author for review or add the sign-off comment to let PR reviewers know that the updates are ready to be merged.
Read about authors working in the public repo for more information.
If you need help getting set up to work in the private repo, see Set up Git repository locally for documentation.
Thanks.

Hey @Jak-MS - Sorry about that! I wasn't aware there was a difference, I'll use private in the future! Thanks for the heads up and appreciate you merging anyways :)

@Jak-MS In your linked instructions on how to get to the private repo, it says to simply add -pr to repo name in the URL string. However when I do this (or even click on the link provided in that doc as an example), I get a 404 error saying the page doesn't exist.

Hi @aliabuckner - does any of this apply?
from https://review.docs.microsoft.com/en-us/help/contribute/contribute-authors-in-public-repo?branch=master#why-work-in-the-private-repo

The private repositories require you to sign in to GitHub with your account and have access to the repo. If you get a 404 error when browsing to the private repository that you know exists, try signing in to GitHub first. If you are already signed in, that 404 error typically means you do not have access. To get access, your GitHub account needs to be configured with two-factor authentication, and joined to the right the organization (such as MicrosoftDocs). Follow the steps in the GitHub account setup article.

If you need more assistance, you can contact the peer help alias at<EMAIL_ADDRESS>Thanks.


Throws "Expected a type" on enum object "OMEMOModuleNamespace"

I followed the given instructions on my custom project.
I am getting the following error on building the application on XCODE.

But this error gone when I imported the missing file on the following file given below.

But now I am getting following errors on the Enumerated objects on the method declarations on the following file give below.

I tried with 3.7 earlier. But still getting error : 'XMPPFramework.h' file not found

Have you added this module to Podfile?

6 июня 2018 г., в 11:13, AAGSICON<EMAIL_ADDRESS>написал(а):
I tried with 3.7 earlier. But still getting error : 'XMPPFramework.h' file not found
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

My podfile is as follows
Uncomment the next line to define a global platform for your project
platform :ios, '9.3'
target 'XmppDemo' do
Uncomment the next line if you're using Swift or would like to use dynamic frameworks
use_frameworks!
pod 'XMPPFramework', '~> 3.7.0'
pod 'KissXML', :git => "https://github.com/aksonov/KissXML.git", :branch => '5.1.4'
end

My podfile is as follows
platform :ios, '9.3'
target 'XmppDemo' do
use_frameworks!
pod 'XMPPFramework', '~> 3.7.0'
pod 'KissXML', :git => "https://github.com/aksonov/KissXML.git", :branch => '5.1.4'
end

I already have this issue.
Solved by following way

Manually placed the folder RNXMPP on my project
Then install XMPPFramework, only via pod

Now it works. The issue is related to the RNXMPP package.


Email verification link already used when user receives email

I have an issue with some users, that when they receive their link in an email, it is already used.
In the web server logs i can see the link is used from a completely different ip, than the user has.
My guess is that some email scanner service visits the link to check for malware.
I have a setup where I create a user with a blank password, and no password is sent in the email, when they use the verification link, i redirect them to change their password, in that way i force the user to pick there own password.
One solution could be to allow multi use of the link, but that would degrade the security.
Any thought about how to solve this?
This could be the same issue as #482 is experiencing.

Make the page POST back with JS -- the automated scanners will not submit the page.

thanks, i will try to implement that.


Update linux.md

text editor kde

Hello @Arnab11
Thanks for your contribution.
Let's connect ->
GitHub -> @Suryansh1720001 (Follow)
LinkedIn -> https://www.linkedin.com/in/suryanshprajapati17
Also, Star the repo ⭐

Thanks for accepting the merged request. followed you on github and linkedin


[Performance regression] Performance regression in Typescript 5.1 on large files

🔎 Search Terms
performance regression, Typescript 5.1
🕗 Version & Regression Information

This changed between versions 5.0.4 and 5.1
This changed in commit 511921e1e205f52304559a35e58e16eb9213ecb3 (detected via every-ts)
This is the behavior in every version I tried after 5.1, including the latest 5.6 version

⏯ Playground Link
https://github.com/moufmouf/typescript-perf-issue
💻 Code
See Github repo for the code sample (the issue is that very big files take wayyyyy longer to be analyzed)
🙁 Actual behavior
The issue is that the compilation time of some very large files (22k lines) has increased significantly between v5.0 and v5.1 (and is still very long in the latest 5.6 release)
The file I'm trying to compile is 22k lines long.
Compilation time in v5.0: 6s.
Compilation time in v5.1: 110s.
This is a 18x increase.
I generated trace files but I don't know how to interpret them.
Tracefile for 5.0: traceDir_5.0.log
Tracefile for 5.1: traceDir_5.1.log
🙂 Expected behavior
The analysis time would not greatly differ between 5.0 and recent releases.
Additional information about the issue
The file that is suddenly taking very long to analyze was generated using ts-proto.
I could perfectly disable the analysis for this file (since it was autogenerated) but there is probably an underlying issue.
For the record, the original file comes from the WorkAdventure project (https://github.com/workadventure/workadventure)

After update<EMAIL_ADDRESS>the Typescript IntelliSense is so slow😥. I'm not sure is that related to this issue.

@s3xysteak  TypeScript 5.1 was ages ago. If things got slower when updating to this month's VS Code version, your issue is definitely not this issue.

Bisects to #53435

We're spending huge amounts of time analyzing functions like this one https://github.com/moufmouf/typescript-perf-issue/blob/master/src/messages.ts#L6122 which have extremely deep control flow graphs with possible narrowing effects due to the conditional assignments.
If this is generated code, it's a huge waste of time to be typechecking these bodies (which would almost certainly be written differently by a human, i.e. in a way that is more amenable to typechecking). It should really be using any in all function bodies, since any is effectively free from a perf perspective. I would also question why it's not using switch, e.g. here it would be much shorter and also efficient
    if (message.message?.$case === "batchMessage") {
      obj.batchMessage = BatchMessage.toJSON(message.message.batchMessage);
    }
    if (message.message?.$case === "errorMessage") {
      obj.errorMessage = ErrorMessage.toJSON(message.message.errorMessage);
    }
    if (message.message?.$case === "roomJoinedMessage") {
      obj.roomJoinedMessage = RoomJoinedMessage.toJSON(message.message.roomJoinedMessage);
    }
    if (message.message?.$case === "webRtcStartMessage") {
      obj.webRtcStartMessage = WebRtcStartMessage.toJSON(message.message.webRtcStartMessage);
    }
    if (message.message?.$case === "webRtcSignalToClientMessage") {
      obj.webRtcSignalToClientMessage = WebRtcSignalToClientMessage.toJSON(message.message.webRtcSignalToClientMessage);
    }

It's possible there's a fix here but it's always going to be possible (but unlikely for a human) to write over-deep CFA graphs like this.
I'm open to a PR on this but overall I would say this is taking an expectable amount of time for the amount of CFA graph complexity the code is presenting; the 5.0 behavior was fast because it was also wrong in many cases.


New GC strategy.

This PR adds a new GCStrategy, that treats any pointer with address space other than 0 as a pointer to managed memory.
We extent class GCStrategy in llvm/IR/GCStrategy.h, and register the new GCStrategy.

I tested that the strategy named gcs-llvm-backend exists in the generated plugin as follows:
Using the sample llvm program
define void @f() gc "gcs-llvm-backend" {
entry:
  ret void
}

I ran
llvm-as-16 < sample.ll | llc-16

which fails not finding the gc strategy
LLVM ERROR: unsupported GC: gcs-llvm-backend

and
llvm-as-16 < sample.ll | llc-16 -load build/install/lib/kllvm/libKLLVMPass.so

which succeeds.

To test if the gc strategy is correctly loaded as needed, I then added the gc strategy to the function generated by make_function, using
apply_rule->setGC("gcs-llvm-backend");

and edited llvm-kompile-clang to load the pugin to llc
      run @LLC@ \
        "$modopt" -mtriple=@BACKEND_TARGET_TRIPLE@ \
        -load="$passes" \
        -filetype=obj "$llc_opt_flags" "${llc_flags[@]}" -o "$modasm"

I used test/defn/imp-sum.kore for testing.
Testing with --emit-ir
llvm-kompile-testing test/defn/imp-sum.kore main --emit-ir -v -o imp-sum-interpreter

This worked only when I emit textual IR, not binary IR (I edited llvm-compile for that). Otherwise, the following error was generated.
/usr/lib/llvm-16/bin/llc: error: /usr/lib/llvm-16/bin/llc: tmp.tmZpzEAPip: error: Invalid record (Producer: 'LLVM16.0.6' Reader: 'LLVM 16.0.6')

This was the case in develop branch as well if --emir-ir is used. I am not sure if there is something wrong with my setup.
Testing with --emit-ir
llvm-kompile-testing test/defn/imp-sum.kore main -v -o imp-sum-interpreter

llvm-kompile-codegen fails with
LLVM ERROR: unsupported GC: gcs-llvm-backend

Okay so this works when loaded into LLC but it doesn't seem to find it when just running llvm-kompile-codegen normally. I believe in the latter case, it is linked via a static library, so it's possible that the TU containing the registration object is not getting linked in because none of its symbols are required. Try making some trivial modification to the main method of llvm-kompile-codegen to ensure that linking in that TU occurs.

To do that, you can probably just construct an instance of the strategy and just not use it.

Your suggestion solved the issue, the test I described above worked.


[LintDiff] pipeline is showing false positive on LRO of delete

open this issue from this comment: https://github.com/Azure/azure-rest-api-specs/pull/23085#issuecomment-1487569177
This delete operation has the x-ms-long-running-operation: true with 200 and 204, but reported by the linter to require a 200 or 204 which is a false positive.

Thank you @ArcturusZhang for reporting this. Until this is solved, please proceed with merging PRs in spite of this false positive.

Task 22896654: Fix LongRunningResponseStatusCode


Current performance

Fixes #128
Challenges.findAllChallengeablePeopleForUser checks for performance id and doesn't make a query if not given one.
Result.findAllForEval query joins on performances with a closed challenge window

[ ] Checked On Staging


Help! Exception NoSuchMethodError

On Manjaro Linux, Java 8, when trying to run the jar.
Exception in thread "main" java.lang.NoSuchMethodError: java.lang.Class.getPackageName()Ljava/lang/String; 	at tv.dotstart.beacon.core.util.VersionMetadata.getId(VersionMetadata.kt:45) 	at tv.dotstart.beacon.core.util.VersionMetadata.<init>(VersionMetadata.kt:57) 	at tv.dotstart.beacon.ui.BeaconUiMetadata.<init>(BeaconUiMetadata.kt:26) 	at tv.dotstart.beacon.ui.BeaconUiMetadata.<clinit>(BeaconUiMetadata.kt:26) 	at tv.dotstart.beacon.ui.BeaconCli.<clinit>(Beacon.kt:134) 	at tv.dotstart.beacon.ui.BeaconKt.main(Beacon.kt:273)
Do I need some packages that I'm missing?

 Beacon 2.4 requires Java 9 while Beacon 3.x even requires Java 11 hence the error related to Class#getPackageName (which was introduced in Java 9).
Generally, I currently recommend sticking to the LTS versions of Java when running Beacon as there have been various issues reported due to the changes in architecture the Java APIs are currently going through. They're the only versions I actively validate when releasing a new version. The latest validated version is Java 11 (as 17 has just been released a few weeks ago).
That being said: Version 2.4 was lacking a check for the minimum required Java version within the application itself as the target audience are mostly Windows users (The assumption is that somebody who runs Linux will probably prefer to configure their router themselves). Version 3.x changes that fact but its checks have not been updated to reflect the Java 11 requirement yet (as it is still an Alpha release, that's somewhat acceptable).


[SPARK-19520][streaming] Do not encrypt data written to the WAL.

Spark's I/O encryption uses an ephemeral key for each driver instance.
So driver B cannot decrypt data written by driver A since it doesn't
have the correct key.
The write ahead log is used for recovery, thus needs to be readable by
a different driver. So it cannot be encrypted by Spark's I/O encryption
code.
The BlockManager APIs used by the WAL code to write the data automatically
encrypt data, so changes are needed so that callers can to opt out of
encryption.
Aside from that, the "putBytes" API in the BlockManager does not do
encryption, so a separate situation arised where the WAL would write
unencrypted data to the BM and, when those blocks were read, decryption
would fail. So the WAL code needs to ask the BM to encrypt that data
when encryption is enabled; this code is not optimal since it results
in a (temporary) second copy of the data block in memory, but should be
OK for now until a more performant solution is added. The non-encryption
case should not be affected.
Tested with new unit tests, and by running streaming apps that do
recovery using the WAL data with I/O encryption turned on.

@tdas @zsxwing

Test build #72614 has started for PR 16862 at commit bdbe267.

Test build #72614 has finished for PR 16862 at commit bdbe267.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72614/
Test FAILed.

Merged build finished. Test FAILed.

@liancheng FYI

Kafka test error (hopefully unrelated?). Retest this please

Test build #72654 has started for PR 16862 at commit bdbe267.

Test build #72654 has finished for PR 16862 at commit bdbe267.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72654/
Test PASSed.

Looks good.

Test build #72667 has started for PR 16862 at commit 42d5d10.

Test build #72667 has finished for PR 16862 at commit 42d5d10.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72667/
Test PASSed.

We can remove the putBytes workaround when I (or someone else) get around to fixing SPARK-19556.
For now, I'm merging this to master and 2.1.


Lodash duplication

🐛 Bug report
In our bundle analyser we have picked up that both lodash and lodash-es are being imported into Formik.
https://github.com/formik/formik/blob/c238e9881e2473dccadf58246c6183ff5c17ff6b/packages/formik/package.json#L46-L47
This unnecessarily increases the file size.
Suggested solution(s)
Remove lodash-es and ensure only the individual lodash utilities are imported.
Your environment

Software
Version(s)

Formik
2.1.4

React
16.8.4

TypeScript
3.7.4

Browser
Version 83.0.4103.61 (Official Build) (64-bit)

npm/Yarn
6.9.0 / 1.17.3

Operating System
Mac OS Catalina

This is a pretty common issue, but the issue is that lodash itself exports CommonJS modules which webpack wraps with extra code to incorporate into your ES module environment.
Formik uses lodash-es because it doesn't require this wrapping, meaning it is smaller. If you are using lodash in an ESM environment, you should instead do this:
// webpack.config.js or whatever
webpackConfig.resolve.alias = {
  lodash: 'lodash-es',
};

Now, every single time your project or any of its dependencies imports lodash, you'll get lodash-es (just like Formik), and lodash will never be used in your build, resulting in the smallest possible build of all the options.
// with the above alias, this actually imports `lodash-es`,
// and _only_ the shuffle function and any dependent functions
// the rest of `lodash-es` will be tree-shaken unless it is used in dependencies like Formik
import { shuffle } from 'lodash';

shuffle([]);

https://github.com/formik/formik/issues?q=is%3Aissue+lodash-es
If we did not preoptimize this, anyone who uses Formik in ESM but does not use Lodash themselves would have an oversized lodash import.


Map is not defined

Sometimes, we are getting the following error, while using anime.js. it is bundled inside browserify, hence the bundle reference.
We are not sure how to reproduce, or how to get into this error, but we keep on getting this:
ReferenceError: Map is not defined
at getElementTransforms(/bundle.js:1:437032)
at ? (/bundle.js:1:439931)
at Array.map(/Cat/76)
at getAnimatables(/bundle.js:1:439860)
at ? (/bundle.js:1:443734)
at anime(/bundle.js:1:443956)
at Object.t.add(/bundle.js:1:449041)
at Nanocomponent._$circle_171.load(/bundle.js:1:462466)
at Nanocomponent.(anonymous function) as _load
at ? (/bundle.js:1:452651)
at o(/bundle.js:1:66264)

Hi @zoharit are you still encountering this issue?


Update PPP

Pull Request Test Coverage Report for Build 197

0 of 0 (NaN%) changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 70.747%

Totals

Change from base Build 196:
0.0%

Covered Lines:
1988

Relevant Lines:
2810

💛  - Coveralls

you forgot to remove 1 line " *** Wrote additional tests for existing features to increase coverage from 88% to 92% (Pull requests https://github.com[#36], https://github.com[#38])" either that or u forgot to link it up


Messed up Principal Ref

I think I have  a messed up principal reference somewhere. I'm using vite and am trying to integrate Stoic.
Error:
Uncaught (in promise) Error: Attempt to hash a value of unsupported type: q4eej-kyaaa-aaaaa-aaaha-cai
    at hashValue (vendor.90183b1f.js:formatted:13276)
    at vendor.90183b1f.js:formatted:13282
    at Array.map (<anonymous>)
    at requestIdOf (vendor.90183b1f.js:formatted:13282)
    at vendor.90183b1f.js:formatted:18634
    at new Promise (<anonymous>)
    at StoicIdentity.transformRequest (vendor.90183b1f.js:formatted:18629)
    at HttpAgent.call (vendor.90183b1f.js:formatted:8054)
    at async r (vendor.90183b1f.js:formatted:8632)
    at async te.i.loggedIn (index.e2d5eb40.js:1)

This is how I set things up:
canisterID is a Text
agent is let stoicIdentity = await StoicIdentity.load();
myCanister = Actor.createActor(canister_idl, { agent, canisterId: canisterID });
The following are the results from the console when I put a break point at
if (A instanceof Principal)
        return hash$3(A.toUint8Array());

A instanceof Principal - false
A instanceof Principal$2 - true
I end up hitting:
throw Object.assign(new Error(`Attempt to hash a value of unsupported type: ${A}`), {
        value: A
    })

Principal$2
{     "_arr": {         "0": 0,         "1": 0,         "2": 0,         "3": 0,         "4": 0,         "5": 0,         "6": 0,         "7": 14,         "8": 1,         "9": 1     },     "_isPrincipal": true }
Do I need to wrap my canister id in something? Is there a ref inside the stoic Identity I can use?

It looks like stoic-identity puts some node_modules underneath it with another copy of the agent libraries. I guess because of this it creates Principal and Principal$1 and Principal$2 as different object types.  There fore the instanceof Principal doesn't work.
If I add a branch to hash_value in request_id.js I can get it to work:
else if (typeof value === 'object' &&
        value !== null &&
        value._isPrincipal == true &&
        typeof value.toUint8Array === 'function') {
        return hashValue(value.toUint8Array());

    }

I'll make an issue on the agent project to include this branch. It is possible this was caused because I originally had 0.9.7 for my project and npm installed stoic-identity referencing agent 0.10.0 and now npm can't unwind things.

Thanks - yeah @jorgenbuilder came across the same sorts of things (as I have I in the past), definitely because of a mismatch I believe. Will try to see if we can make a fix from our end too incase they don't merge from the agent repo.

Kyle is going to try to get the fix into next version of agent-js: https://forum.dfinity.org/t/mismatching-dfinity-agent-versions-can-cause-hashing-issues/7359/4
Not sure if @aramakme already found a temporary fix, but this works for me: https://github.com/aviate-labs/dots/commit/136575e21b10015932916f8e3f2b3d3c5dc2f7d5
tldr: instanceof operator doesn't work well across modules in npm.


ENH: stats: override stats.gumbel_r.fit and stats.gumbel_l.fit with system of equations for MLE

Reference issue
 from the gh-11782 main issue
What does this implement/fix?

Implements MLE system of equations equations for gumbel_r distribution according to the equation found in the text Evans, Hastings, and Peacock (2000), Statistical Distributions, 3rd. Ed., John Wiley and Sons.pdf, page 101.
Additional information

A runtime warning is sometimes raised from within optimize.root because the guess causes an overflow from system of equations.

https://github.com/scipy/scipy/pull/12097#issuecomment-653731803

@WarrenWeckesser, We were hoping that you could advise on an issue that is being encountered with this distribution.
It is that if the the scale is close enough to zero for the input data, the system of equations runs into an overflow, causing warnings. Considering this situation seems to be identifiable by noticing that the standard deviation of the data is very small, @mdhaber and I took a look at the two simultaneous equations with the limit b->0, where b is the scale.
heuristically, it would seem that as b approaches zero, the location, a is the mean/mode of the data, as with such small scale all of the values are near identical.  But the limits are complicated and a bit out of my skill level.

So, the real question is how to identify / and or treat this situation. In the least we could try to identify data with little to no deviations and pass them to the super method. (in addition to falling back on the super method if optimization over the system of equations fails). Another option may be to find solutions with the limits, if they are indeed computable. As you may also see, tests are failing because of attempts to suppress the warnings created during optimization, and a test exists to prohibit warning suppression. If there is an acceptable way to suppress warnings let me know.

Where the optimization fails due to overflow.

if x is a constant value
x0 = x[0]  # == x.mean()

b = x.mean() - sum(xi * exp(-xi/b)) / sum(exp(-xi/b))
b = x0 - sum(x0 * exp(-x0/b)) / sum(exp(-x0/b))
b = x0 - (x0 * exp(-x0/b)) / exp(-x0/b)
b = x0 - x0
b = 0

a = -b * log(1/n * sum(exp(-x0 / b))
a = -b * log(1/n * n * exp(-x0/b))
a = -b * log(exp(-x0/b))
a = -b * -x0 / b
a = x0

The fit method for gumbel_r can be repurposed to for use with gumbel_l. The process is to use gumbel_r.fit with data * -1 and then use the negative of the resulting location along with the unmodified scale:

from scipy.stats import gumbel_r, gumbel_l
data = gumbel_l.rvs(size=1000, scale=2, loc=.55)

loc_r, scale_r = gumbel_r.fit(data * -1)
loc_l, scale_l = (- loc_r, scale_r)

loc_super, scale_super = gumbel_l.fit(data)  # gumbel_l is not overriden, so it uses rv_continuous.

print(f"using system of equations: { scale_l:>5} \nusing super method:        { scale_super:>5}\n")

args = [data, (gumbel_l._fitstart(data),)]
func = gumbel_l._reduce_func(args, {})[1]

print("log-likelihood results:")
print(func((loc_l, scale_l), data))
print(func((loc_super, scale_super), data))

print(f"is system of equations less than super method?: {func((loc_l, scale_l), data) < func((loc_super, scale_super), data)}")

I believe this is a result of gumbel_r and gumbel_l being reflections of each other over their respective locations, observationally (charts are generated using code from distribution documentation page + added title):

@sethtroisi Thanks for taking a look - I know that the warning suppression is causing that failure and am still trying to figure out the best way to avoid it.
Benchmarks are behaving as expected; Two free parameters is fitted quickly, fixing either one is still slow:
(header is cut off in the screenshot, so here it is):
distribution   |    loc_fixed     |     scale_fixed    |      shape1_fixed     |     True / True     |     True / False     |     False / True   False / False

I was hoping that @WarrenWeckesser would have an opinion on it; Whether it is better to try to perform the optimization and figure out some way to catch the warning (and then decide how to proceed if the optimization was successful) or to calculate in advance whether or not we expect there to be a warning by checking if there is low standard deviation.

The way I see it - there are essentially two roads to go down.

try to catch the type of scenarios that will causes warnings to come up
suppress the warnings during optimization in a way that does not fail tests that check for warning suppression

I lean towards finding a way to perform the optimization inside the overridden fit method, while suppressing warnings that may arise, and then checking for success of the optimization. It seems a bit unsure to me to try to determine this unless we can find the exact threshold that causes the overflow and how to test for it. Therefore, I am wondering if there is a way to get past scipy._lib.tests.test_warnings.FindFuncs.

I created a PR on swallan's repo with some suggested changes for this PR: https://github.com/swallan/scipy/pull/2

This needs to be updated to use _assert_less_or_close_loglike instead of _assert_lessthan_loglike (see https://github.com/scipy/scipy/pull/12882).  Maybe rebase on master and update the test?

Looks like the fit method of the rayleigh distribution needs some work.  I've seen failures locally like those in CI, and it might just be luck that we haven't seen those failures in the master branch.
I think I know what the problem is; I'll work on a fix for rayleigh.fit today.

gh-12968 should fix the failures that we see in this PR.  If we get that PR (or something like it) merged, I expect this PR will run the test suite with no problems.

This should be OK to merge once the tests finish.

CircleCI finished successfully, even though it didn't report to GitHub properly. Travis failures are unrelated (arpack, minimum_phase)

Test failures were unrelated to this PR.
Merged; thanks @swallan and all reviewers.

We need to revisit this.
import numpy as np
from scipy import stats

np.random.seed(0)

dist = stats.gumbel_r
# dist = stats.gumbel_l
x = dist.rvs(loc=1, scale=2, size=100)

fparams = {'floc': 1}
afit = dist.fit(x, **fparams)  # subclass fit method
nfit = super(type(dist), dist).fit(x, **fparams)  # superclass fit method

print(dist.nnlf(nfit, x) - dist.nnlf(afit, x))  # 0

So far so good. As we expect, if one parameter is given, fall back to the superclass fit method.
When we do the same thing with dist = stats.gumbel_l:
print(dist.nnlf(nfit, x) - dist.nnlf(afit, x))  # -24.162130298986284

I notice that afit is (-1, 3.01820809418259) (fitted loc is negative 1) so it has something to do with the way gumbel_l is the reflection of gumbel_r.
But also, we thought before that if one parameter is given, we couldn't rely on the optimal fit values given in the reference. That's not quite true.  If the fscale is given, we can still use the same formula for loc. If the floc is given, there is an equation for scale, although it's a little different than the one in the book when both are free. (I think that it simplifies when both are free? In any case, it's just the first-order condition corresponding with the scale.)


Windows job update

Change location for Azure VMs created by Windows Periodic Job. Also change ServerKeepAlive interval to prevent ssh timeouts.
Related to https://github.com/containerd/containerd/issues/5549

Check original issue for this PR, source of problem was different, no need for this anymore.


Add ARMv6 / ARMv7 support (hard float)

The ABI is a rework from the work @ssvb did (port from Rust) to which I merely added support for the LLVM Array type.
The arm-unknown-linux-gnueabihf target is for the Raspberry Pi (Raspbian). I only tested in QEMU emulating the ARM1176 CPU, not on a real hardware.
The fiber switch context took quite some time, but seems to be working.
I could cross-compile Crystal itself (non release mode), as well as my posix project, which was capable to regenerate the C bindings (in QEMU). I didn't try to build Crystal with Crystal (QEMU is very slow) but Crystal was capable to build simple programs.
The one thing NOT working in unwind. Trying to unwind to generate the stacktrace segfaults, raising an exception fails (phase1 fatal error). Maybe this is because of libgcc_s, which is always linked in, but misses some symbols: _Unwind_GetIP, _Unwind_GetGR and _Unwind_SetGR (despite LSB stating they must be available). Linking against libunwind passes, but maybe it's causing conflicts?

I ran into some issues using this PR to cross-compile to a Raspberry Pi:
Firstly, something was missing that caused code using the SecureRandom module to not compile properly. The minimal example
require "secure_random"

puts SecureRandom.urlsafe_base64(18)

led to the following error: https://gist.github.com/meew0/daf2c57b0de3878ae85e8e38bc6b0282
Fortunately, this wasn't a large problem for me, as SecureRandom wasn't actually necessary in my particular code. After replacing it and building again, I ran into the following segfault (here with the compiler -s flag): https://gist.github.com/meew0/07a8a063a31837cfd7fe275d54fa8f56
Note that this only occurred with the --release flag; without it the compilation worked fine. Other than that, thanks a lot for this PR; it came just in time when I needed it!

Thanks for testing! There must be some tweaks to do in the stdlib, like you found out for SecureRandom. And certainly some issues to be debugged, too (maybe the fiber context switch is a bit wrong).
How did it crash? You cross compiled the compiler, then tried to build your app? I got a segfault with the compiler built in release mode, too (at the same pass), but my linux has LLVM 3.8 when the QEMU image has LLVM 3.5. We should verify with the exact LLVM version.
You can try to cross compile your app directly, and verify if it passes when compiled with --release, and see how it goes.

Sorry for being unclear there; I compiled the compiler with ARM support locally, then used that to cross compile my app to ARM. Should I try cross-compiling the compiler in debug mode and then using that compiler to compile my app directly on the RasPi?

After some more investigating, the Unwind issue is too fold:

libgcc_s:

is capable to throw exceptions (__crystal_personality is called),
is missing symbols (_Unwind_GetIP, _Unwind_SetIP and _Unwind_GetGR);

libunwind:

has all symbols;
is capable to unwind the stack;
is incapable to raise an exception because it can't find an exception handler (_Unwind_RaiseException returns END_OF_STACK).

This kinda looks like a dead end. We could use libunwind to unwind the stack, and libgcc_s to raie the exception ... but we need SetIP and GetGR in the exception handler so it won't work.
Maybe using LLVM's libunwind would help. Maybe using the latest GNU's libunwind could help, too.

@waj I give up the unwind stuff. This is too complicated for me to fix :sob: Maybe I'll try on Alpine Linux armhf but I think I'll have the same result.

I understood why __crystal_personality caused segfaults (using libgcc_s). It's being called with UNW_UA_FORCE_UNWIND (_Unwind_Backtrace) or UNW_UA_END_OF_STACK  (_Unwind_RaiseException), but calling _Unwind_GetLanguageSpecificData in those cases segfaults because context is invalid.
Prepending the following to __crystal_personality fixes the segfaults:
if actions.end_of_stack?
  return LibUnwind::ReasonCode::END_OF_STACK
end

We end up with the following:
CallStack.unwind()
<collects the backtrace>
__crystal_personality(8, (END_OF_STACK),<PHONE_NUMBER>30632, Pointer(LibUnwind::Exception).null, Pointer(Void).null)

__crystal_raise(Pointer(LibUnwind::Exception)@0x96f30)
__crystal_personality(0, (END_OF_STACK),<PHONE_NUMBER>30680, Pointer(LibUnwind::Exception)@0xffffffff, Pointer(Void)@0x96f30)
Failed to raise an exception: 9
[244224] *CallStack::print_backtrace:Int32 +-1090522800
[143892] __crystal_raise +-1090522800
[151176] *raise<Exception>:NoReturn +-1090522800
[150888] *raise<String>:NoReturn +-1090522800
[238912] *failure:NoReturn +-1090522800
__crystal_personality(8, (END_OF_STACK),<PHONE_NUMBER>30632, Pointer(LibUnwind::Exception).null, Pointer(Void).null)
[142324] ???
[Inferior 1 (process 5620) exited with code 011]

Note that unwind_ex (at 0x963f0) eventually ends up being the 5th parameter to __crystal_personality when it's supposed to be the 4th ... i.e. there is something fishy.

Hey @ysbaddaden, this is really awesome!!
Do you think we can still merge this anyway? In this way it doesn't get outdated and trying to make the exception-handling work is a bit easier (no need to work in a separate branch and constantly rebase against master). What do you think?

@ysbaddaden

The ABI is a rework from the work @ssvb did (port from Rust) to which I merely added support for the LLVM Array type.

I had omitted the LLVM Array type support on purpose, because it did not seem to be used by Crystal in any way. I generally don't feel very comfortable about having untested code branches. Missing this particular functionality is perfectly fine because in the case if Crystal ever needs it, the error message would be very clear and explicit. The rest of the ARM ABI code paths are more or less validated. Some of them even used a minimalistic standalone test program for this as mentioned in https://github.com/crystal-lang/crystal/issues/324#issuecomment-100508055
Thanks a lot for taking care of the fiber context switch code. The ancient Crystal did not need it, so we could get away without this functionality.
Now the remaining bits are the support for raising exceptions (the libunwind stuff) and LLVM code generation problems (enabling/disabling optimizations was a bit fragile). I haven't tested your branch yet, but maybe will try to find some time for this.

@asterite

Do you think we can still merge this anyway? In this way it doesn't get outdated and trying to make the exception-handling work is a bit easier (no need to work in a separate branch and constantly rebase against master). What do you think?

I fully support this! And I'm glad that you changed your mind. My impressions was that you had the "all or nothing" approach, so that the ARM ABI bits could not land before we have the exceptions handling working too and before the ARM port is fully passing the Crystal test suite.

By the way, ARM development boards are very cheap nowadays. If the Raspberry Pi is too expensive, then I can recommend one of the Orange Pi boards. The shipping cost from China is also very low. Also there are board variants with 2GB of RAM (this might be important for Crystal) and on-board WIFI / eMMC for a little bit higher price.

@ssvb I needed the array type for my posix project, and the x86_64 ABI do references it, now.
I don't have a Raspberry yet, because I'm not sure which one to get. I wish there was a bunch of Zero with different processors (big or little endian, hard or soft float).

@ysbaddaden OK, thanks. If the array type is used now, then it makes sense.
As for big/little endian and hard/soft float, these are purely software things and you can try all of them on the same ARM processor. But big endian is a very unpopular choice and pretty much immature at the moment. As far as I know, NetBSD tried the big endian mode for fun (but I haven't tested it myself). Nowadays it may be a good idea to get a 64-bit ARMv8 board though.

Unwind is kinda different on ARM EHABI. The exception struct is different, the personality signature is different, the software is required to unwind the stack frame, ...
Anyway, exceptions are now working on ARM EHABI. At least a begin/raise/rescue did work.

Unwind is almost working. Catch all rescues do catch the raised exceptions, but trying to catch a specific exception class doesn't. For example the following snippet doesn't work. It prints "Catch-All: Bar" instead of "Gotcha: Bar":
class Bar < Exception
end

def foo
  raise Bar.new("oh no")
end

begin
  foo
rescue ex : Bar
  puts "Gotcha: #{ex.class}"
rescue ex
  puts "Catch-All: #{ex.class}"
end

Fixed!
Now, I have a segfault in spec/std/channel_spec.cr, more specifically in can be closed from different fiber. An exception is raised, but _Unwind_Backtrace segfaults when trying to collect the callstack. It's most likely caused by an issue in the fiber context switch.

After the updates to this PR I've revisited the cross compilation I was doing before (see my earlier comments), here are my results:
The SecureRandom thing now works perfectly. Great, thanks!
The x86 compiler crash still exists, and the backtrace I get doesn't seem to have changed. The problem doesn't affect me that much and it may be unrelated to this PR, but I'd still be happy to provide any information necessary in fixing it.
I've also now tried to do the actual linking step on the Raspberry Pi itself, with the command output by the compiler on x86; and after installing and updating some missing libraries, building libcrystal.a and editing the command to point to it, I get the following output:
/usr/bin/ld: error: elgyem uses VFP register arguments, elgyem.o does not
/usr/bin/ld: failed to merge target specific data of file elgyem.o
collect2: error: ld returned 1 exit status

(In case there is confusion, elgyem is the name of the project.) From some Google searches, this should be fixable by adding -mfloat-abi=softfp (or soft or hard, I've tried all three) to the output, but nothing I've tried worked.
Apologies if this is something obvious I'm missing (I'm not very experienced in cross compilation) or if this problem is unrelated to the PR, but I feel like if this problem is fixable by adding a compiler flag, then that compiler flag should probably be added to the output when building on x86.

Thanks for testing. Please do!
I didn't tested on a real Pi. I only did on QEMU with the versatilepb machine, arm1176 CPU, 256M of RAM and the latest Raspbian image, which targets ARMv6 with hard float. That is a Pi A or Pi Zero (I'm waiting for thepihut to stock some zeros to test on real hardware).
That being said, I never got a soft/hard float issue. I cross compile with --cross-compile --target=arm-unknown-linux-gnueabihf to get an object file, then link it in Raspbian/QEMU. Maybe toying with the target triple, so LLVM knows what features to use could help?

Ah, it seems like using arm-unknown-linux-gnueabihf works; I've only been using arm-linux-gnueabihf previously. Now it compiles and links successfully, thanks!
However I'm now getting a segfault at runtime:
Invalid memory access (signal 11) at address 0x1c
[1519956] ???
[475236] __crystal_sigfault_handler +64

This trace alone probably isn't of much use to you; tell me if you need any more information. I've been able to trace it down to a minimal example that simply makes an HTTP request: https://gist.github.com/meew0/c0683f76f603f8c8da135f9264245ffc
(Note that this specific example produces a slightly different segfault, however I don't think that matters much.)

@meew0 you may comment out LibExt.setup_sigfault_handler in src/signal.cr and compile with --debug then use gdb to get a backtrace. Yet, I have a feeling the fiber context switch ASM is the culprit.

This is what I got when running the minimal example I posted above, with my very limited gdb experience:
(gdb) run
Starting program: /home/pi/minimal2
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1".

Program received signal SIGSEGV, Segmentation fault.
0x00000000 in ?? ()
(gdb) backtrace
#0  0x00000000 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

I found in the D runtime the answer to my unwind crash (I believe), and also notes on which registers should actually be saved (r4-r11, lr, d8-d15):
https://github.com/dlang/druntime/blob/master/src/core/threadasm.S#L438
I'm working on that, and need to get my head around registers, stacks, etc.

@meew0 I just fixed the fiber context switch segfault.
I can now run the std spec suite without any crash. It merely hangs a bit when computing the BCrypt digests, which is unavoidable.

./std_spec
Finished in 1:07 minutes
3451 examples, 187 failures, 69 errors, 1 pending

After fixing the path at which std specs expect some files to be, I now have:

4466 examples, 194 failures, 5 errors, 3 pending

Of which there is:

1 unwind error (failed to raise an XML::Error exception);
8 socket errors because my QEMU machine lack IPv6 support;
(almost) everything else are Float errors: the generated code is failing on every FPU calculations. I don't know why, yet.

Hm, when trying to cross-compile, I now get the following error (on x86):
Codegen (crystal):                 00:00:00.0136101 (  35.99MB)
<inline asm>:4:9: error: instruction requires: VFP2
        vstmdb sp!, {d8-d15}
        ^
<inline asm>:7:9: error: instruction requires: VFP2
        vldmia sp!, {d8-d15}
        ^
LLVM ERROR: Error parsing inline asm

It happens with anything I'm trying to cross-compile, even an empty file. Is it something with the target triple again? It happens with both arm-unknown-linux-gnueabihf and arm-linux-gnueabihf.

No, it's the fiber context switch. I now push/pop the FPU registers too. LLVM complained for me, too, but I added .fpu vfp to the ASM (see) and it stopped complaining. I compiled Crystal against LLVM 3.9 (with ARM codegen fixes) by-the-way. Or maybe we must specify another version, like .fpu vfpv2?

@asterite the FPU is working. LLVM intrinsics are fine with both Float32 and Float64, Crystal is fine with Float32, but fails when dealing with Float64. Note that ARM VFP (vector-floating-point) uses double word registers.
 Could LLVM fail to understand that self is a double primitive, here?

If I compile with --release, then the bug disappears for something as simple as ceil, because the calls are inlined (no more Float64#ceil symbol, nor bl <ceil> in the disassembly).
The std spec is down to:

4466 examples, 96 failures, 4 errors, 3 pending

But the issue is still there for methods of Math that aren't inlined. For example expm1, log2 or acosh.

After some messing around with LLVM versions (I was on 3.5 previously), everything works perfectly now! The x86 segfault when compiling in release mode seems to be gone as well. Thanks a lot!

@ysbaddaden Does it work if you compile with --mcpu=cortex-a9?
That question has a reason, I'll leave the original text I was going to write before finding the "answer":

Findings so far...
I'm using this program, which can be compiled with --prelude=empty:
{% if flag?(:linux) || flag?(:freebsd) || flag?(:openbsd) %}
  @[Link("m")]
{% end %}
lib LibC
  fun printf(x0 : UInt8*, ...) : Int32
end

lib LibM
  fun ceil_f64 = "llvm.ceil.f64"(value : Float64) : Float64
end

class String
  def to_unsafe
    pointerof(@c)
  end
end

def foo(b)
  LibM.ceil_f64(b)
end

b = 123.456_f64
b = foo(b)
LibC.printf "%g\n", b

I made a C program and compiled it with clang to see what it generated (this is with clang 3.0 on a real raspberry pi 2):
#include <stdio.h>
#include <math.h>

double doit(double x) {
  return ceil(x);
}

int main() {
  double x = 123.456;
  x = doit(x);
  printf("%g\n", x);
  return 0;
}

The LLVM IR for that, on the raspberry, is:
; ModuleID = 'c.c'
target datalayout = "e-p:32:32:32-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:32:64-f32:32:32-f64:32:64-v64:32:64-v128:32:128-a0:0:32-n32-S32"
target triple = "armv4t-unknown-linux-gnueabihf"

@.str = private unnamed_addr constant [4 x i8] c"%g\0A\00", align 1

define double @doit(double %x) nounwind {
  %1 = alloca double, align 4
  store double %x, double* %1, align 8
  %2 = load double* %1, align 8
  %3 = call double @ceil(double %2) nounwind readnone
  ret double %3
}

declare double @ceil(double) nounwind readnone

define i32 @main() nounwind {
  %1 = alloca i32, align 4
  %x = alloca double, align 8
  store i32 0, i32* %1
  store double 1.234560e+02, double* %x, align 8
  %2 = load double* %x, align 8
  %3 = call double @doit(double %2)
  store double %3, double* %x, align 8
  %4 = load double* %x, align 8
  %5 = call i32 (i8*, ...)* @printf(i8* getelementptr inbounds ([4 x i8]* @.str, i32 0, i32 0), double %4)
  ret i32 0
}

declare i32 @printf(i8*, ...)

One difference: the target triple is "armv4t-unknown-linux-gnueabihf".
However, then I try to do this on my machine:
$ bin/crystal build --cross-compile -D arm --target arm-unknown-linux-gnueabihf --prelude=empty foo.cr --link-flags="-mcpu=cortex-a9"
cc foo.o -o foo -mcpu=cortex-a9 -rdynamic  -lm -L/usr/lib -L/usr/local/lib

(the -mcpu=cortex-a9 seems to be needed, explanation below)
Now I copy that foo.o and link it and run it:
$ cc foo.o -o foo -mcpu=cortex-a9 -rdynamic  -lm -L/usr/lib -L/usr/local/lib && ./foo
0

Nope :-(
However, if I generate an LLVM IR file for foo.cr:
$ bin/crystal build -D arm --target armv4t-unknown-linux-gnueabihf --prelude=empty foo.cr --emit llvm-ir

(it gives a warning, but no problem)
Now the LLVM IR is this:
; ModuleID = 'main_module'
source_filename = "main_module"
target datalayout = "e-m:e-p:32:32-i64:64-v128:64:128-a:0:32-n32-S64"
target triple = "armv4t-unknown-linux-gnueabihf"

%String = type { i32, i32, i32, i8 }

@ARGC_UNSAFE = internal global i32 0
@ARGV_UNSAFE = internal global i8** null
@"'%g\0A'" = private constant { i32, i32, i32, [4 x i8] } { i32 1, i32 3, i32 3, [4 x i8] c"%g\0A\00" }

define internal i32 @__crystal_main(i32 %argc, i8** %argv) {
alloca:
  %b = alloca double
  br label %entry

entry:                                            ; preds = %alloca
  store i32 %argc, i32* @ARGC_UNSAFE
  store i8** %argv, i8*** @ARGV_UNSAFE
  store double 1.234560e+02, double* %b
  %0 = load double, double* %b
  %1 = call double @"*foo<Float64>:Float64"(double %0)
  store double %1, double* %b
  %2 = call i8* @"*String#to_unsafe:Pointer(UInt8)"(%String* bitcast ({ i32, i32, i32, [4 x i8] }* @"'%g\0A'" to %String*))
  %3 = load double, double* %b
  %4 = call i32 (i8*, ...) @printf(i8* %2, double %3)
  ret i32 %4
}

declare i32 @printf(i8*, ...)

; Function Attrs: uwtable
define i32 @main(i32 %argc, i8** %argv) #0 {
entry:
  %0 = call i32 @__crystal_main(i32 %argc, i8** %argv)
  ret i32 0
}

; Function Attrs: uwtable
define internal double @"*foo<Float64>:Float64"(double %b) #0 {
entry:
  %0 = call double @llvm.ceil.f64(double %b)
  ret double %0
}

; Function Attrs: nounwind readnone
declare double @llvm.ceil.f64(double) #1

; Function Attrs: uwtable
define internal i8* @"*String#to_unsafe:Pointer(UInt8)"(%String* %self) #0 {
entry:
  %0 = getelementptr inbounds %String, %String* %self, i32 0, i32 3
  ret i8* %0
}

; Function Attrs: nounwind
declare void @llvm.stackprotector(i8*, i8**) #2

attributes #0 = { uwtable }
attributes #1 = { nounwind readnone }
attributes #2 = { nounwind }

I link it:
$ llc foo.ll -filetype=obj -o foo.o -float-abi=hard -mcpu=cortex-a9

Then copy foo.o to the raspberry and link it and run it again:
$ cc foo.o -o foo -rdynamic  -lm -L/usr/lib -L/usr/local/lib && ./foo
124

It... works??!
I don't know why is there a difference, but maybe it helps to figure out the problem.
About -mcpu=cortex-a9: if I don't put it in the llc command then it doesn't work, it prints 0 again.
Another maybe notable thing is that clang for ARM moves the double parameter to an alloca that has align 4, and stores/loads from it using align 8. I don't know if it is relevant, I spent some time thinking that was it, but apparently not.
Yet another difference is that clang uses ceil while we use llvm.ceil.f64, but again it doesn't seem to make a difference.
FINALLY: I then remember that you can specify the cpu with --mcpu, so the --link-flags wasn't needed, and in fact wasn't working (the cpu is used in the object generation in our code). With that:
$ bin/crystal build --cross-compile --target armv4t-unknown-linux-gnueabihf --prelude=empty foo.cr --mcpu=cortex-a9
cc foo.o -o foo  -rdynamic  -lm -L/usr/lib -L/usr/local/lib

Copy over the raspberry pi:
$ cc foo.o -o foo -mcpu=cortex-a9 -rdynamic  -lm -L/usr/lib -L/usr/local/lib && ./foo
124

YAY!! :-)
So using this mcpu seems to fix it, but no idea why... (I read it here)

Also: the target triple difference doesn't seem to matter, with arm-unknown-linux-gnueabihf and with the mcpu setting it works well too.

@asterite You rock! Compiling with --mcpu=arm1176jzf-s (Pi A, B1, Zero) fixed the issue.
It means that to compile for the Rapsberry Pi family, we must pass:

--mcpu=arm1176jzf-s (A, B, Zero);
--mcpu=cortex-a7 (B2);
--mcpu=cortex-a53 (B3);

The complete definitions are in llvm/Support/ARMTargetParser.def.

As for the reason, if you look at ARMTargetParser.dev, you'll see that the ARM definitions do specify an FPU to use or not. Just like there are many ARM instruction sets, there are many FPU instruction sets, that may be available, or not. If you look at the Cortex A family, a FPU is attached to the CPU (e.g. A9: NEON_VFPV4, A7: NEON_FP16). If you look at the ARM11 family (or older), half of them have a FPU (e.g. arm1176jzf-s or arm1156t2f-s have a VFPV2). By default, I assume LLVM doesn't enable a FPU for the ARM target.
Setting a specific ARM CPU enables a particular FPU (hard-float), otherwise we're downgraded to soft-float. It works when the program runs in isolation, but it breaks when we interact with a libc compiled with hard-float support, since we don't use the floating-point registers, but the libc does. At least, this is how I interpret this.
If we could ask LLVM whether the target has a FPU, or not, we could support soft-float in addition to hard-float for the ARM target. The single change is the fiber context switch: don't push/pop the d8-d15 registers on soft-float.

@ysbaddaden Regarding the failing Base64 spec, I was able to reduce it to this:
array = uninitialized UInt8[5]
array[4] = 1_u8
puts array # => StaticArray[126, 212, 8, 2, 184]
puts array[4] # => 1

I have no idea where the problem could be. It's clear that the value is put in the array, but then garbage is obtained when iterating it in puts. I also tried puts array.to_s or array.to_s(STDOUT) but those work well. Either it's a bug on our side because of some assumption we make which doesn't hold on ARM, or there's a bug somewhere else... Maybe we should try to reduce it further, probably without using the std (with --prelude=empty and LibC.printf).

Also found this: https://llvm.org/bugs/show_bug.cgi?id=8040
It says it's resolved but by the comments it doesn't look like it... I'll try to remove byval and see what happens (it's not trivial, though)

This is now fixed for Darwin only

Cool, it works on iOS but fails everywhere else...

Update: so, not using byal does seem to fix it, though I didn't manage to get everything working yet. byval simplified a lot of things for us, basically treating everything as pointers, so now a lot of logic needs to change. In any case I remember byval gave us a bit of trouble before so it might be a good idea to get rid of it. I'll try to slowly do this on master. I don't know if it's a good idea to wait until they fix it in LLVM...

On the other hand, the XML::XPathContext spec still fails, so that must be something related to unwind.

More info: https://github.com/ldc-developers/ldc/issues/1283#issuecomment-186296423

@asterite this is great to hear! If the ARM can improve Crystal, this is all win :-)
Yes, the failing XML:Path spec is an unwind error. I done know why, but it unwinds the stack successfully but fails immediately trying to raise. The LCD link has some references to unwind and CFA, I'll look closer in a few days.

I've tested on a raspberry pi v3, and had success compiling and linking the compiler iself, once I got the llvm version on my pi and desktop to match.
CRYSTAL_CONFIG_PATH=`pwd`/src bin/crystal build --cross-compile -D arm --target arm-unknown-linux-gnueabihf --mcpu=cortex-a53 src/compiler/crystal.cr -D without_openssl -D without_zlib -o .build/crystal-arm

was the command I used.
Unfortunately, once I executed the binary, I got SIGILL. It seems like llvm.readcyclecounter hasn't been implemented for cortex-a53 yet:

After fixing the problem
  def self.new_seed : UInt32
    # Chosen by fair dice roll.
    # Guaranteed to be random.
    4_u32
  end

the compiler ran!
However when trying to compile something on the pi 3, I get:
pi@raspberrypi:~/crystal $ LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib make std_spec
Using /usr/bin/llvm-config-3.5 [version=3.5.0]
./bin/crystal build  -o .build/std_spec spec/std_spec.cr
Using compiled compiler at .build/crystal
<inline asm>:4:9: error: instruction requires: VFP2
        vstmdb sp!, {d8-d15}
        ^
<inline asm>:7:9: error: instruction requires: VFP2
        vldmia sp!, {d8-d15}
        ^
LLVM ERROR: Error parsing inline asm

Makefile:75: recipe for target '.build/std_spec' failed
make: *** [.build/std_spec] Error 1

I've tried adding the full cross-compile flags but it still doesn't work.
pi@raspberrypi:~/crystal $ LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib bin/crystal build --cross-compile --target arm-unknown-linux-gnueabihf --mcpu=cortex-a53 -D arm spec/std_spec.cr
Using compiled compiler at .build/crystal
<inline asm>:4:9: error: instruction requires: VFP2
        vstmdb sp!, {d8-d15}
        ^
<inline asm>:7:9: error: instruction requires: VFP2
        vldmia sp!, {d8-d15}
        ^
LLVM ERROR: Error parsing inline asm

By the way, all of this testing has been with #3424, #3474, and #3471 merged on both the desktop and the pi.

@RX14 Cool, thank you for testing!! What happens if you test it without merging #3474 ? I want to know if #3474 is worth it or not. It doesn't produce a change on non-ARM machines, but I don't know if it does indeed fix some issues on ARM.

@RX14 About the error you get about "error: instruction requires: VFP2", it's probably this

@asterite I saw that comment, but I'm not quite sure what @suggested. I cross-compiled using llvm 3.5, so if it is llvm version, I'm not sure why it wouldn't work when compiling natively on arm.

@RX14 Sorry, I mean this comment. It seems that depending on the mcpu option we would need to use different asm instructions (at least that's what I understood). Right now the asm is hardcoded for al mcpus, which doesn't seem to be OK.

@asterite so then why does it cross-compile correctly? I can cross-compile the std specs just fine, and i'm pretty sure they need fibers so will include the fiber ASM. Unfortunately I can't link the std_spec, sue to openssl on raspbian being only  1.0.1 but that llvm error happens before the linking stage so i'm pretty sure it'll work.

Ok, compiled openssl and linked std_spec.
Running specs without the byval patch:
pi@raspberrypi:~/crystal $ LD_LIBRARY_PATH=$LD_BRARY_PATH:/usr/local/lib ./std_spec
Invalid memory access (signal 11) at address 0xfffffff8
[9274164] ???
[1765800] __crystal_sigfault_handler +64

(gdb) run
Starting program: /home/pi/crystal/std_spec
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1".

Program received signal SIGSEGV, Segmentation fault.
0x001b2864 in *OptionParser#on<String, String, String, &Proc(String, Nil)>:Array(OptionParser::Handler) ()
(gdb) bt
#0  0x001b2864 in *OptionParser#on<String, String, String, &Proc(String, Nil)>:Array(OptionParser::Handler) ()

Running specs with the byval patch:
pi@raspberrypi:~/crystal $ LD_LIBRARY_PATH=$LD_BRARY_PATH:/usr/local/lib ./std_spec
................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................E...........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................*........................................................................................................................................................................................................................................F........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................GC Warning: Repeated allocation of very large block (appr. size 135168):
	May lead to memory leak and poor performance
....GC Warning: Repeated allocation of very large block (appr. size 266240):
	May lead to memory leak and poor performance
.......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................*................................................................................................................................................Failed to raise an exception: FAILURE
[9348276] ???
[1562224] __crystal_raise +108
[1777364] ???
[1777200] ???

I added a --mattr option so one may now enable or disable some features (eg: a specific FPU or ARMvX intructions).
An armhf flag is added for some targets (i.e. only gnueabihf for now).
I enabled the vfp2 feature by default for gnueabihf targets, so passing --mcpu isn't required anymore. vfp2 is a conservative choice, so I hope it will work for other CPUs than the ARM1176 (i.e., Cortex A). This isn't set when --mcpu is set or when --mattr contains fp.
The fiber context switch now saves/restores the FPU registers for armhf targets only.

The XML::Xpath error originates from trying to raise from the C callback set by xmlSetGenericErrorFunc. GDB complains that it can't unwind the stack:
#1  0x0002a1b0 in raise () at /home/github/crystal/src/raise.cr:183
#2  0x0002a110 in ~procProc(Pointer(Void), Pointer(UInt8), NoReturn)@./src/xml/error.cr:22 () at /home/github/crystal/src/pointer.cr:80
#3  0xb6d2bf28 in ?? () from /usr/lib/arm-linux-gnueabihf/libxml2.so.2
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Actually, no unwind information can be found in /usr/lib/arm-linux-gnueabihf/libxml2.so.2.9.1 because it's missing the ARM specific .ARM.extab and .ARM.exidx sections.
The following patch avoids the issue, by delaying the raise after calling xmlXPathEvalExpression, where unwind tables are available. XML specs are passing, but it's possible that errors may pop up, since we now ignore errors when we used to raise; some XML functions may now return a null pointer.
diff --git a/src/xml/error.cr b/src/xml/error.cr
index aab0969..f7df0d5 100644
--- a/src/xml/error.cr
+++ b/src/xml/error.cr
@@ -21,7 +21,7 @@ class XML::Error < Exception

   LibXML.xmlSetGenericErrorFunc nil, ->(ctx, fmt) {
     # TODO: use va_start and va_end
-    raise XML::Error.new(String.new(fmt).chomp, 0)
+    #raise XML::Error.new(String.new(fmt).chomp, 0)
   }

   # :nodoc:
diff --git a/src/xml/xpath_context.cr b/src/xml/xpath_context.cr
index dc79c02..9b4066a 100644
--- a/src/xml/xpath_context.cr
+++ b/src/xml/xpath_context.cr
@@ -6,7 +6,7 @@ struct XML::XPathContext

   def evaluate(search_path : String)
     xpath = LibXML.xmlXPathEvalExpression(search_path, self)
-    raise XML::Error.new("error in '#{search_path}' expression", 0) unless xpath.value
+    raise XML::Error.new("error in '#{search_path}' expression", 0) unless xpath && xpath.value

     case xpath.value.type
     when LibXML::XPathObjectType::STRING

Compiling libxml2 manually could fix the problem, yet, I'm hardly motivated to build libxml2 on my slow QEMU machine...

Compiling libxml2 with -fexceptions -funwind-tables will create the .ARM.extab and .ARM.exidx sections which fixes the problem! We can either deal with the problem by ignoring errors then checking return values (above patch) but we lose some context, or forcing to recompile libxml2.
The actual command I used: ./configure --with-python=no --with-lzma=no CFLAGS="-fexceptions -funwind-tables" && make

Down to:

4467 examples, 3 failures, 4 errors, 2 pending

crystal spec spec/std/socket_spec.cr:285 # TCPServer fails when port is in use
crystal spec spec/std/socket_spec.cr:374 # TCPSocket fails when connection is refused
crystal spec spec/std/socket_spec.cr:295 # TCPSocket sends and receives messages
crystal spec spec/std/socket_spec.cr:392 # UDPSocket sends and receives messages by reading and writing
crystal spec spec/std/socket_spec.cr:444 # UDPSocket sends and receives messages by send and receive over IPv6
crystal spec spec/std/socket_spec.cr:462 # UDPSocket broadcast messages

I also have the following one when running the std_spec, but not when running the spec file alone:
crystal spec spec/std/file_spec.cr:439 # File real_path expands paths of symlinks

Great!!
Another solution for XML: put the errors in @@errors and later check. Right now the only place this happens is in XML::XPathContext#evaluate, though maybe other places should checked, but we can do this progressively. And this way we don't need to ship a libxml2 compiled with some specific flags.
Diff:
diff --git a/src/xml/error.cr b/src/xml/error.cr
index aab0969..9c1ae53 100644
--- a/src/xml/error.cr
+++ b/src/xml/error.cr
@@ -20,15 +20,24 @@ class XML::Error < Exception
   }

   LibXML.xmlSetGenericErrorFunc nil, ->(ctx, fmt) {
-    # TODO: use va_start and va_end
-    raise XML::Error.new(String.new(fmt).chomp, 0)
+    # TODO: use va_start and va_end to
+    @@errors << XML::Error.new(String.new(fmt).chomp, 0)
   }

   # :nodoc:
   def self.set_errors(node)
-    unless @@errors.empty?
-      node.errors = @@errors.dup
+    if errors = self.errors
+      node.errors = errors
+    end
+  end
+
+  def self.errors
+    if @@errors.empty?
+      nil
+    else
+      errors = @@errors.dup
       @@errors.clear
+      errors
     end
   end
 end
diff --git a/src/xml/xpath_context.cr b/src/xml/xpath_context.cr
index dc79c02..1b1a56d 100644
--- a/src/xml/xpath_context.cr
+++ b/src/xml/xpath_context.cr
@@ -6,6 +6,14 @@ struct XML::XPathContext

   def evaluate(search_path : String)
     xpath = LibXML.xmlXPathEvalExpression(search_path, self)
+    unless xpath
+      if errors = XML::Error.errors
+        raise errors.last
+      else
+        raise XML::Error.new("error in '#{search_path}' expression", 0)
+      end
+    end
+
     raise XML::Error.new("error in '#{search_path}' expression", 0) unless xpath.value

     case xpath.value.type

Looking at other bindings:

Nokogiri has a single use of xmlSetGenericErrorFunc for libxml2 (raises), a second one for libxslt1 (noop), and... that's it. It also sets the callback just before calling and removes it immediately after. I don't know why.
libxml++ they don't bother to set the callback. If xmlXPathEvalExpression returns null, then they simply raise an "invalid xpath" exception.
lxml (python bindings): noop.

I guess there aren't much usages, except for xmlXPathEvalExpression. I applied your patch, but limited to arm targets, since it works on X86 by default (the .eh_frame section is created).

@asterite merge?

@ysbaddaden Let's do it!! Thank you so much for all of your hard work! ❤

Ah, mmm... I'll let you merge it so you can decide whether to merge, squash or rebase this (I never know :-P)

Let's merge, I just rebased, each commit is self explanatory, and we'll keep the reference to the pull request, with comment history :)

Can you explain how to compile crystal for arm?  I'm getting (on mac) Error: llc: No available targets are compatible with this triple.

@zeiv are you sure you're running crystal built from master? This PR isn't in the latest release.

@zeiv you must first compile the latest master of Crystal, then you'll be able to target ARM (or wait for the 0.20 to be released). Also, do not specify an empty prelude, std and core lib are fine, and they're required to compile Crystal.
You may either cross compile Crystal itself (again, latest master). Be sure to have the same LLVM version on both the host and the guest. You'll may also want to install/compile a recent LLVM version (eg: 3.8 or 3.9). Be sure to not pass an empty prelude (compilation with fail).
bin/crystal build src/compiler/crystal.cr \
  --cross-compile --target arm-unknown-linux-gnueabihf \
  --release -Dwithout_openssl -Dwithout_zlib

Or you may cross-compile your applications for ARM:
bin/crystal build myapp.cr \
  --cross-compile --target arm-unknown-linux-gnueabihf

Once an object file is created, copy it to the the ARM board (or VM), but before linking it by copy/pasting the printed cc command, you'll need a bunch of libraries to be installed on the ARM board. Please refer to Required Libraries for an exhausted list. You'll have to compile Boehm GC 7.6; clone crystal and run make deps to compile the src/llvm/ext/llvm_ext.o and src/ext/libcrystal.a objects.
Eventually tweak the copied cc command to refer correctly to llvm_ext.o and libcrystal.a, and you should have a working Crystal compiler or application running on ARM.

@ysbaddaden Thanks for the reply.  That's what I was missing, I was using bin/crystal from master without actually compiling the entire source.  I'm trying to compile master on my mac and am now running into the following error:
$ make LLVM_CONFIG=/usr/local/Cellar/llvm36/3.6.2/bin/llvm-config-3.6
Using /usr/local/Cellar/llvm36/3.6.2/bin/llvm-config-3.6 [version=3.6.2]
c++ -c  -o src/llvm/ext/llvm_ext.o src/llvm/ext/llvm_ext.cc `/usr/local/Cellar/llvm36/3.6.2/bin/llvm-config-3.6 --cxxflags`
cc -fPIC    -c -o src/ext/sigfault.o src/ext/sigfault.c
ar -rcs src/ext/libcrystal.a src/ext/sigfault.o
CRYSTAL_CONFIG_PATH=`pwd`/src ./bin/crystal build  -o .build/crystal src/compiler/crystal.cr -D without_openssl -D without_zlib
Undefined symbols for architecture x86_64:
  "_LLVMInitializeARMAsmParser", referenced from:
      _*LLVM::init_arm:Nil in L-L-V-M-.o
  "_LLVMInitializeARMAsmPrinter", referenced from:
      _*LLVM::init_arm:Nil in L-L-V-M-.o
  "_LLVMInitializeARMTarget", referenced from:
      _*LLVM::init_arm:Nil in L-L-V-M-.o
  "_LLVMInitializeARMTargetInfo", referenced from:
      _*LLVM::init_arm:Nil in L-L-V-M-.o
  "_LLVMInitializeARMTargetMC", referenced from:
      _*LLVM::init_arm:Nil in L-L-V-M-.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Error: execution of command failed with code: 1: `cc -o "/Users/xavier/OneDrive/Documents/Dev/crystal/crystal/.build/crystal" "${@}"  -rdynamic  /Users/xavier/OneDrive/Documents/Dev/crystal/crystal/src/llvm/ext/llvm_ext.o `/usr/local/Cellar/llvm36/3.6.2/bin/llvm-config-3.6 --libs --system-libs --ldflags 2> /dev/null` -lstdc++ -lpcre -lgc -lpthread /Users/xavier/OneDrive/Documents/Dev/crystal/crystal/src/ext/libcrystal.a -levent -liconv -ldl -L/usr/lib -L/usr/local/lib`
make: *** [.build/crystal] Error

UPDATE
I was able to successfully cross-compile the compiler WITHOUT the --release option using Ubuntu.
I was compiling on a digital ocean droplet.  When I first tried to compile with --release, this happened...
Using compiled compiler at .build/crystal

Invalid memory access (signal 11) at address 0x18
[9183605] *CallStack::print_backtrace:Int32 +117
[9062216] __crystal_sigfault_handler +56
[35486375] sigfault_handler +40
[140131403375584] ???
[26166049] _ZN4llvm12SelectionDAG7getNodeEjNS_5SDLocENS_3EVTENS_7SDValueE +65
[26947182] ???
[26949563] ???
[26603809] ???
[26604895] _ZN4llvm12SelectionDAG13LegalizeTypesEv +943
[26456599] _ZN4llvm16SelectionDAGISel17CodeGenAndEmitDAGEv +215
[26470802] _ZN4llvm16SelectionDAGISel20SelectAllBasicBlocksERKNS_8FunctionE +1058
[26476045] _ZN4llvm16SelectionDAGISel20runOnMachineFunctionERNS_15MachineFunctionE +669
[23793557] ???
[34592087] _ZN4llvm13FPPassManager13runOnFunctionERNS_8FunctionE +487
[34592747] _ZN4llvm13FPPassManager11runOnModuleERNS_6ModuleE +43
[34591156] _ZN4llvm6legacy15PassManagerImpl3runERNS_6ModuleE +772
[33208958] ???
[33210344] LLVMTargetMachineEmitToFile +456
[21001963] *LLVM::TargetMachine#emit_to_file<LLVM::Module, String, LLVM::CodeGenFileType>:Bool +91
[21001851] *LLVM::TargetMachine#emit_obj_to_file<LLVM::Module, String>:Bool +11
[17811350] *Crystal::Compiler#cross_compile<Crystal::Program, Array(Crystal::Compiler::CompilationUnit), String, String>:Nil +326
[17810546] *Crystal::Compiler#codegen<Crystal::Program, Crystal::ASTNode+, Array(Crystal::Compiler::Source), String>:(Array(String) | Nil) +1746
[17800966] *Crystal::Compiler#compile<Array(Crystal::Compiler::Source), String>:Crystal::Compiler::Result +150
[17825657] *Crystal::Command::CompilerConfig#compile<String>:Crystal::Compiler::Result +57
[17825570] *Crystal::Command::CompilerConfig#compile:Crystal::Compiler::Result +34
[10002466] *Crystal::Command#build:Crystal::Compiler::Result +290
[9999371] *Crystal::Command#run:(Array(Crystal::ImplementationTrace) | Array(Crystal::Init::View+:Class) | Array(String) | Bool | Crystal::Compiler::Result | Hash(String, String) | IO::FileDescriptor | Nil) +491
[9998603] *Crystal::Command::run<Array(String)>:(Array(Crystal::ImplementationTrace) | Array(Crystal::Init::View+:Class) | Array(String) | Bool | Crystal::Compiler::Result | Hash(String, String) | IO::FileDescriptor | Nil) +27
[9998537] *Crystal::Command::run:(Array(Crystal::ImplementationTrace) | Array(Crystal::Init::View+:Class) | Array(String) | Bool | Crystal::Compiler::Result | Hash(String, String) | IO::FileDescriptor | Nil) +25
[8987984] ???
[9061913] main +41
[140131383912496] __libc_start_main +240
[8984489] _start +41
[0] ???

I thought it might be a memory leak or something, so I resized my droplet to their super-duper 64GB / 16 core option and tried again, but it crashed again.  (I was using screen and it terminated when it died, so I don't have a trace unfortunately).

@zeiv homebrew is missing some targets by default. AArch64 isn't enabled for 3.9 and it seems ARM wasn't in 3.6 :-(
I'll try to reproduce the crash with --release. It's a crash in LLVM.

That segfault looks like the one I got; it was fixed by upgrading LLVM to 3.9.

Hi, I am super exited about running Crystal on Raspi. I am trying to do Docker playground for compiling.
 When you have docker installed and running, just save this file as Dockerfile anywhere and then in same directory run docker build . -t crystal
Unfortunately it is not 100% yet and there is few problems. Like installing  Boehm GC 7.6 throws error when calling: ./configure
...
checking sys/dg_sys_info.h presence... no
checking for sys/dg_sys_info.h... no
./configure: line 16943: syntax error near unexpected token `ATOMIC_OPS,'
./configure: line 16943: `   PKG_CHECK_MODULES(ATOMIC_OPS, atomic_ops, ,'
The command '/bin/sh -c cd /tmp/bdwgc && ./configure' returned a non-zero code: 2

@schovi Did you install autotools (as per https://github.com/ivmai/bdwgc/issues/69#issuecomment-116224201)?

@schovi just use gc 7.6.0 and libatomic_ops 4.4.0 tarballs, instead of cloning the Git master.

@schovi ran into that as well, seems the work around was to install and run 'pkg-config' once there was a note added to the README recently about it I think. GL!

Sorry if this has been answered I've read through this whole thread a few times though. I'm trying to install crystal on a raspberry pi 3. @rx14 helped me over on StackOverflow, and I feel like I followed the instructions exactly. I installed all of the required files for both OSX and Raspbian.
LLVM 3.9.1 was the latest version I could find on debian for armhf. So I edited my homebrew formula to install 3.9.1 instead of 4.0. After that I ran ./bin/crystal build src/compiler/crystal.cr --cross-compile --target arm-unknown-linux-gnueabihf --release -s -D without_openssl -D without_zlib which successfully created a crystal.o file. I copied that into my crystal folder on the raspi3 and ran cc crystal.o -o crystal  -rdynamic  /home/pi/crystal/src/llvm/ext/llvm_ext.o /usr/local/bin/llvm-config --libs --system-libs --ldflags 2> /dev/null -lstdc++ -lpcre -lm -lgc -lpthread /home/pi/crystal/src/ext/libcrystal.a -levent -lrt -ldl -L/usr/lib -L/usr/local/lib which I edited to point to llvm_ext.o and crystal.a on my raspi3. But instead of working it immediately gave me this error.
crystal.o: In function `~LLVM::Attribute::kind_ids:init':
main_module:(.text+0x26a84): undefined reference to `LLVMGetEnumAttributeKindForName'
main_module:(.text+0x26acc): undefined reference to `LLVMGetEnumAttributeKindForName'
main_module:(.text+0x26b14): undefined reference to `LLVMGetEnumAttributeKindForName'
main_module:(.text+0x26b5c): undefined reference to `LLVMGetEnumAttributeKindForName'
main_module:(.text+0x26ba4): undefined reference to `LLVMGetEnumAttributeKindForName'
crystal.o:main_module:(.text+0x26bec): more undefined references to `LLVMGetEnumAttributeKindForName' follow
crystal.o: In function `default_target_triple':
/Users/isaac/workspace/crystal/recompile/crystal/src/llvm.cr:77: undefined reference to `LLVMGetDefaultTargetTriple'
crystal.o: In function `string_and_dispose':
/Users/isaac/workspace/crystal/recompile/crystal/src/llvm.cr:93: undefined reference to `LLVMDisposeMessage'
crystal.o: In function `default_target_triple':
/Users/isaac/workspace/crystal/recompile/crystal/src/llvm.cr:77: undefined reference to `LLVMGetDefaultTargetTriple'
crystal.o: In function `string_and_dispose':
/Users/isaac/workspace/crystal/recompile/crystal/src/llvm.cr:93: undefined reference to `LLVMDisposeMessage'
/Users/isaac/workspace/crystal/recompile/crystal/src/llvm.cr:93: undefined reference to `LLVMDisposeMessage
.........

The message goes on forever but doesn't seem to give me anything useful other than the fact that everything seems to be undefined. One thing I find interesting is that the paths for the files are still the same as they were on my mac when I compiled. I guess that makes sense though for crystal.o. Anyway any help would be really appreciated. Thanks.

When trying to use the version of crystal I compiled on my raspi I get the following error when trying to compile a simple hello world app. Error in hello.cr:1: while requiring "prelude": can't find file 'prelude' relative to '/home/pi/crystal' I've been using crystal for a couple years on OSX and Linux (x86) and I've never seen this before. What did I do wrong?

Is the prelude.cr file there? Everything in src/ must be available to the compiler at runtime.

I just checked crystal out from master and didn't delete any files. Are you
saying that I need all crystal-lang source files in the folder with my
crystal binary or every one of my crystal projects?
On Mar 15, 2017 6:47 PM, "Guilherme Bernal"<EMAIL_ADDRESS>wrote:

Is the prelude.cr file there? Everything in src/ must be available to the
compiler at runtime.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/crystal-lang/crystal/pull/3424#issuecomment-286924650,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAHmpBMMbZBHuwt5RfWtD7Gjft_3XKA2ks5rmIalgaJpZM4KX2Dy
.

I moved my hello.cr into crystal/src which I verified did have prelude.cr. When I then ran pi@raspberrypi:~/crystal/src $ ../bin/crystal build hello.cr
but still got this error:
Error in hello.cr:1: while requiring "prelude": can't find file 'prelude' relative to '/home/pi/crystal/src'

You need to set CRYSTAL_PATH=lib:/path/to/crystal/source/src. I've updated my stackoverflow answer here.

@ysbaddaden I finally got crystal working on my raspberry pi. 👍  When I try to compile with it though I get this error https://dl.dropboxusercontent.com/s/m5mvghbb8q043be/2017-03-16 at 12.19 PM.png
@RX14 Advised me to change crystal build hello.cr to crystal build --target arm-unknown-linux-gnueabihf hello.cr which worked. He also said that it was a bug and I should report it here. Thanks.

@elorest you copied the binary over to the pi, right? The compiler needs a bash script that sets CRYSTAL_PATH and eventually delegates to the binary —see any Linux or macOS distribution.
Default target issue: how did you install or compile LLVM for the rpi? Crystal asks LLVM what the default host target is. You can check with llvm-config --host-target; it should be arm-unknown-linux-gnueabihf but I guess it's not?

@ysbaddaden I got @elorest to fix it by setting CRYSTAL_CONFIG_PATH.

I installed in from a debian repo that i needed to add since the newest stable version was 3.5. It installed it as llvm-config-3.9 which I specified when I compiled on my arm machine from cyrstal.o.
It returns armv7l-unknown-linux-gnueabihf from --host-target.

The issue may be that Crystal doesn't deal correctly with the armv7l part, and searches for a lib_c/armv7l-linux-gnueabihf which I'm not really sure how to fix. Maybe force it to arm-....

@elorest Does the fix in #4167 fixes the target issue? are Crystal specs passing on your RPI?

@ysbaddaden Yes they pass when I pass arm-unknown-linux-gnueabih as the target. I've since messed something up on the raspberry pi system I was using so can't try that latest fix yet. I'll let you know when I check that.


Add translation in English for German Unity Day

Add translation for German Unity Day in English. Please give me feedback.

You don't need to specify en_US and en_GB  separately. Just specify en – this will work for all English variants.

Hello, thank you for your feedback
I tried using 'en' but I couldn't make it work. I noticed that in other translations there are 'en_US' etc. specified separately.
Can you direct me where to look to solve this?

@snikoletopoulos Could you share perhaps your code what you are trying to do? I used the following script and works fine using only the en locale.
<?php

declare(strict_types=1);

require 'vendor/autoload.php';

$holidays = Yasumi\Yasumi::create('Germany', 2022);

$unityDay = $holidays->getHoliday('germanUnityDay');

echo $unityDay->getName(['de']).PHP_EOL;
// prints "Tag der Deutchsen Einheit"

echo $unityDay->getName().PHP_EOL;
// prints "German Unity Day"


[patch] fix AUS ACT

I have no clue why this works but it seems to. Can anyone enlighten me? It scraped 0 last day.

How to fix the erroneous 0 from history?

If you mean yesterday's timeseries, it should be removed the next time we update the site as long as the scraper works

@qgolsteyn are we able to merge?


Has Errors when i try to start ples help

IDK I JUST NEED HELP

@899475 Please describe your problem.

it says this when i try to debug it
System.InvalidOperationException: ''arc' name cannot be found in the name scope of 'Invisible_Man.MainWindow'.'

@899475 Did you remove any objects especially arc from the windows form?

i didnt touch any code at all

https://www.microsoft.com/en-us/download/details.aspx?id=10801  Expression Blend Software Development Kit (SDK)
Add reference
C:\Program Files (x86)\Microsoft SDKs\Expression\Blend.NETFramework\v4.0\Libraries\Microsoft.Expression.Drawing.dll


Add email title to be acknowledged by email-alert-check

The title for this content was changed after e-mails were sent, causing the email-alert-check to fail
https://deploy.blue.production.govuk.digital/job/email-alert-check/1496/console
This adds the email title to the acknowledged list, following guidance here
https://docs.publishing.service.gov.uk/manual/alerts/email-alerts.html#troubleshooting-failed-checks

The title for this content was changed after e-mails were sent, causing the email-alert-check to fail

At least for me, this would be useful information to have in the commit message.

The title for this content was changed after e-mails were sent, causing the email-alert-check to fail

At least for me, this would be useful information to have in the commit message.

commit and PR comment updated with more info


Nav ng-show/hide

Started working on ng-show and hide functionality on login and logout.

awesome! looks good


feat: use sane numbering and separation

Traveling through the post folder always made me a bit insane (names with different number of digits, broken sorting, etc). I renamed the file names to make it easier to navigate, c&p and so on.
Would people be fine to take this PR?

@Stebalien it doesn't I tested them. They use the url value inside the post itself, not the filename

How do we sort these? By date or by URL? I'm still not sure if cleaning this up is worth the confusion (urls not matching file names) but if we sort by date, I have no real objections.
Note: I agree this is a mess, it's just consistently messy.

but if we sort by date, I have no real objections.

We sort by date on the blog :)


Correct domainsList in documentation

Values are searched for in domainsList, not domainList

What this PR does / why we need it:
The readme specifies an incorrect configuration acme.domainList. This only updates part of the file to the correct values - they're listed correctly in other places.
The specific code that looks for domainsList is here and here.
Special notes for your reviewer:
Checklist
[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

[ ] DCO signed
[ ] Chart Version bumped
[X] Variables are documented in the README.md

/assign @emilevauge

@tomwaddington Could you please rebase your branch on master.

/assign @mmatur


Test fails on MacOS with Intel MKL

I managed to reproduce the problem locally with gcc. It seems that using the Single Dynamic Library to build the tests causes problems. This is a sinppet that shows the problem:
#include <iostream>
#include <numeric>
#include <shogun/mathematics/Statistics.h>
#include <shogun/lib/SGMatrix.h>

int main()
{
	shogun::init_shogun();

	index_t num_samples = 100000;
	shogun::SGMatrix<float64_t> samples(2, num_samples);
	std::iota(samples.matrix, samples.matrix+num_samples, 1);

	auto emp_cov = shogun::CStatistics::covariance_matrix(samples, false);

	for (index_t i=0; i < emp_cov.num_rows*emp_cov.num_cols; ++i)
		std::cout << emp_cov.matrix[i] << '\n';
}

This piece of code when compiled using
g++  main.cpp -o out -lshogun

or with
g++  main.cpp -o out -I/home/theartful/anaconda3/include/ -L/home/theartful/anaconda3/lib  -lshogun -lmkl_rt

produces different wrong results on each run. And it works fine when compiled using:
g++  main.cpp -o out -I/home/theartful/anaconda3/include/ -L/home/theartful/anaconda3/lib  -lshogun -lmkl_intel_lp64 -lmkl_gnu_thread -lmkl_core -lgomp -lpthread -lm -ldl

Replacing the single dynamic mkl library to build the tests seems to fix the problem.
I also tried to write code that uses Eigen using mkl. It works fine using -lmkl_rt, but if I also linked shogun -lshogun (without even using it in code) it behaves unpredictably, like the previous code sinppet.

Nvm what I said. The problem was solved locally by compiling against Intel OpenMP, instead of GNU OpenMP. Apparently MKL and gomp don't work together well. https://stackoverflow.com/questions/25986091/telling-gcc-to-not-link-libgomp-so-it-links-libiomp5-instead

Fixed by #4656.