Handling Nulls in Database Join Conditions

This task is automatically imported from the old Task Issue Board and it was originally created by Radosław Waśko.
Original issue is here.

We planned to implement operations in the Table library so that Nothing == Nothing evaluates to True (unlike SQL preferred approach which would yield NULL/Undefined for NULL-NULL comparisons).
We also want the Database.Table to be consistent with that. We can achieve that by replacing the a = b operator with one of:

a IS NOT DISTINCT FROM b (works in Postgres) or a IS b (only works in SQLite),
a <=> b (only works in MySQL)
COALESCE(a = b, a IS NULL AND b IS NULL) - should work everwhere.

This however has some performance implications - all approaches I tried in Postgres were making it unable to exploit indexes set on the column on which the join was performed - which can be very bad for performance.
It seems that using the preferred operator (<=> for MySQL, IS for SQLite) did allow the database to use the existing index. But for Postgres I could find no way to make it use the index and handle nulls the way we want.
We need to discuss how we want to approach this null-handling - as there is a potential performance price for NULL=NULL, at least in some databases (Postgres). We also need to make sure that In-Memory table operations (both as Join_Condition.Equals and as Column Operation ==) are consistent with what we do for the database (or if we are not consistent, we need to ensure it is very clearly described to the user so that there are no surprises).
Comments:
References: the db fiddle I used for most of the testing
https://www.db-fiddle.com/f/sehxaqWyQroFmghUE4A4Pm/1
(Radosław Waśko - Dec 27, 2022)

Probably related to this is that we may want stuff like Filter_Condition.Not_Equal 100 to be consistent between in-memory and Database.
Currently however, if a column contains a Nothing, in in-memory Not_Equal 100 will evaluate to True on that Nothing in memory, but to Undefined/NULL in Database.
So
t = table_builder [["X", [100, 50, Nothing]]]
r = t.filter "X" (Filter_Condition.Not_Equal 100)

will have different results in both backends (r.at "X" . to_vector will be [50, Nothing] in in-memory but just [50] in the Database backends).
However, since != does not take part in joins, we can probably hack it around by changing [X] != ? into [X] IS NULL OR [X] != ? for ? != Nothing.
But first I'd like to understand if we want this. For now I'm adding tests requiring that but marking them as pending and linking to this issue.
I encountered this a bit randomly when adapting some tests.

Note that allowing NULL = NULL can result in some unexpected results.
Let's say we have two tables:
t1:

id
 col1

1
 A

2
 B

Nothing
 C

Nothing
 D

t2:

id
 col2

3
 X

2
 Y

Nothing
 Z

Nothing
 W

Then t1.join t2 Join_Kind.Inner on="id", will result in:

id
 col1
 col2

2
 B
 Y

Nothing
 C
 Z

Nothing
 D
 Z

Nothing
 C
 W

Nothing
 D
 W

Because the NULL rows will all match with each other. This may sometimes be desirable, although it seems not necessarily good behaviour, as it can create a lot of matches when there are lots of NULLs in the key column.

Also note that this is the current behaviour of the in-memory backend. But whether this is desirable is another thing and remains to be discussed as part of this overall issue.
The example was inspired by confusion we encountered in @farmaazon's solution to 08.08 bookclub.

Agreed that we will match database behaviour here. So closing this.

Superseded by https://github.com/enso-org/enso/issues/5156


[SPARK-33070][SQL] Optimize higher order functions

What changes were proposed in this pull request?

Add optimization rules to handle some common higher order functions on collection types.
Why are the changes needed?

Combining and reordering expressions can improve the performance and enable further optimizations by other rules.
Does this PR introduce any user-facing change?

No
How was this patch tested?

UTs

Test build #133387 has started for PR 30930 at commit c8048d4.

Test build #133387 has finished for PR 30930 at commit c8048d4.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/133387/

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37978/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37978/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37978/

Test build #133652 has started for PR 30930 at commit 027abad.

Test build #133652 has started for PR 30930 at commit 027abad.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38241/

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38241/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38241/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38241/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38241/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38241/

Test build #133652 has finished for PR 30930 at commit 027abad.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test build #133652 has finished for PR 30930 at commit 027abad.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/133652/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/133652/

cc: @hvanhovell @ueshin

cc: @hvanhovell @ueshin

Test build #133852 has started for PR 30930 at commit 42ee890.

Test build #133852 has started for PR 30930 at commit 42ee890.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38441/

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38441/

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38441/

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38441/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38441/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38441/

Test build #133852 has finished for PR 30930 at commit 42ee890.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test build #133852 has finished for PR 30930 at commit 42ee890.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/133852/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/133852/

Test build #136770 has started for PR 30930 at commit 150c33d.

Kubernetes integration test unable to build dist.
exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41353/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41353/

K, looks fine to me


Custom ICanvas implementation for PDFsharp

This pull request is related to https://github.com/manuelbl/SwissQRBill.NET/issues/3
Here is an initial ICanvas implementation using PDFsharp.
This is not production ready yet. These are the issues:

[x] One of the scissor symbols not drawn correctly.
[x] Cross missing in QR code.
[x] Individual lines in a path are connected.
[x] Implementations of FillPath and StrokePath do not respect the color yet. I don't think this is needed, as the QR bill specifications allows only black anyway.

One challenge was that PDFsharp uses a coordinate system with its origin in the top left corner and the y axis pointing downwards. So I needed to apply a transformation to mirror the y axis. But this caused to mirror the actual text output as well. So as a workaround I had to apply additional transformations in PutText. Text looks ok with this workaround, but I'm not sure if the erroneous scissor symbol is related to this coordinate system mess.
I tried a couple of things, but I didn't find a way to resolve any of the issues 1-3 above.
@manuelbl maybe you have some ideas?

I tried solving issue 2 above by playing around with CurrentPath.FillMode, but no luck so far.
Issue 3 above can probably be solved by calling CurrentPath.StartFigure() and CurrentPath.CloseFigure() at the appropriate places. But I didn't find the solution yet.

Ok I realized that the cross is drawn in white color, so it is related to issue 4 above. Issues 2 and 4 are now fixed.

Issue 1 above was a rad vs. degree problem. Fixed.

I was able to fix issue 3 above as well by calling CurrentPath.StartFigure() right before AddLine, and CurrentPath.CloseFigure() right after AddLine in the LineTo implementation.
It feels wrong to wrap every individual line segment in Start-/CloseFigure. I thought that dealing with Start-/CloseFigure in CloseSubpath would be more appropriate (@manuelbl do you agree?), but I couldn't get it to work that way.
Anyway, the result looks fine now :)
I notice that the scissor symbol looks slightly different when comparing the output of my new implementation with the output of the existing implementation, and I'm not sure why. But that doesn't bother me too much.

Thanks a lot. I'll into it later today...

The code looks good but it brings back bad memories. I've wasted time with PDFsharp's XGraphicsPath before because it's broken and I couldn't find a workaround. It's basically impossible to implement this sequence of drawing commands, which would perfectly fit the graphics model of PDF:

Move to x1, y1
Line to x2, y2
Move to x3, y3
Line to x4, y4
Stroke path

With XGraphicsPath (and System.Drawing's GraphicsPath), it would translate to:
path.AddLine(x1, y1, x2, y2);
path.StartFigure();
path.AddLine(x3, y3, x4, y4);
graphics.DrawPath(pen, path);

But as it turns out, StartFigure() is broken. The implementation is (see XGraphicsPath.cs):
        // TODO: ???

In the context of QR bill, your workaround mostly works except line joins are incorrect as each line segment is a separate subpath. It's visible in the corner marks:

The implementation would certainly break for more complex paths. They would be incorrectly filled. It seems like pure luck already that the scissors aren't affected. They look identical to other implementations.
I wish I could recommend a better implementation. But as mentioned above, I failed to find a solution. If you can find a better workaround, that would be great.
Regarding the top-down coordinate system: This also affects the PNG and SVG canvas implementation. And I also found that a transformation matrix alone is not sufficient as the text will be mirrored. I've solved it by mirroring the y coordinates in each method. Your solution seems to work just fine. So no need to change.
The only thing that should be changed is to add a license in the header of the new files, preferably the MIT License. As an example see https://github.com/manuelbl/SwissQRBill.NET/blob/master/Generator/QRBill.cs. I am aware that not all my files have a license. I will add them soon.
Once you have added the license, the PR can be merged.

Thanks for your feedback. I will add MIT license information and update the PR tomorrow.
Regarding the path related problems: Would it be possible to draw the QR bills without using subpaths/moveto, but just using connected paths? So for example to create the "amount box" you would create two separate paths instead of a single path with a gap?
If this is possible from a design perspective, we could maybe change the interface in the following way, to enforce that limitation on the consumer side of the interface:

Remove ICanvas.MoveTo
Remove ICanvas.CloseSubpath
Add x/y parameters to ICanvas.StartPath, to provide a start position for the path.

This interface would only allow to draw connected paths, and it should probably be possible to implement such a simplified interface using PDFsharp without hitting its limitations.
On the one hand it is unfortunate to adjust the interface because of a bug in some other library. But on the other hand a simplification of the interface could make other future implementations easier as well. Furthermore, I wouldn't hold my breath for PDFsharp to fix this bug anytime soon, as it does not seem to be very actively maintained. Also there are many forks of PDFsharp out there, and I'm forced to use one of these forks because to official build does not yet support .NET Core.

I guess I've finally found a workaround for the PDFsharp bug. Basically, all path operations need to be recorded. Once it is know if the path will be stroked or filled, they are replayed. For filled path, each subpath is closed. For stroked paths, each subpath is drawn separately.
BTW: Paths with subpaths are required for the scissors with the hole in the handle.

Perfect!
Now the output looks nice even with zoom level 600% :)
Thank you!


chore: Add custom browserslist

Now that we have an idea what browsers our vendors use, we can add a browserslist config to help catch incompatibilities. I added this to ESLint and StyleLint so CI can help us find issues.

To see the specific tasks where the Asana app for GitHub is being used, see below:

https://app.asana.com/0/0/1207719326862934

I don't see a change to our Webpack config file here; we have this in there:
["@babel/preset-env", { targets: "> 0.25%" }]

I assume we can now at least remove the targets configuration here and let browserslist take care of it — does present-env read that by default?


Use webpack hooks

In webpack 4, Tapable#apply and Tapable#plugin raise a deprecation notice.

Tapable#apply(plugin) must be replaced by a call to plugin.apply
Tapable#plugin(hook, fn) must be replaced by a call to hook.tap (or hook.tapAsync)

Note: This is breaking change because it drops webpack < 4 compatibility.

Note: This is breaking change because it drops webpack < 4 compatibility.

Unfortunately this isn't an option. It should support versions 2-4. That's what v5 will be supporting.

No problem. I’ll update the PR to support both APIs.

Okay, sounds good, thanks!

I don't understand the CI failure. Tests are fine when ran locally.

It's travis issue, it's just too slow to handle browser tests and they fail with timeout.

Anyway, I guess if tests pass, then everything works 🙃 Merging. Thank you for the PR and follow up updates regarding previous versions support! 👏🎉


Mock test a contract using SafeERC20 library with smock

When I want to test my contract using smock framework:
contract TestSafeERC20 {
    using SafeERC20 for IERC20;
    address public token;

    constructor(address tkn) {
        token = tkn;
    }

    function safeTransfer(address from, uint256 value) external {
        IERC20(token).safeTransferFrom(from, address(this), value);
    }
}

I can test it as:
let fakeSafe;
let mockSafeFactory;
let mockSafe;

 beforeEach(async() => {
        fakeSafe = await smock.fake("TestSafeERC20");

        mockSafeFactory = await smock.mock<TestSafeERC20factory>("TestSafeERC20");
        mockSafe = await mockSafeFactory.deploy(tokenAddress);
})

 describe.only("safe tests", async () => {
        it ("should run this test", async () => {
            // this txn is successful
            await fakeSafe.connect(someSigner).safeTransfer(xaddress, toBN(100))

        })

        it ("should run this too right", async() => {
            // this reverts with `SafeERC20: ERC20 operation did not succeed`
            await mockSafe.connect(someSigner).safeTransfer(xaddress, toBN(100))
        })
    })

Shouldn't mock just behave like fakes in this case?

SafeERC20 is an internal library of YOUR contract, in the end, when you mock the ERC20 it'll be called with just transfer, just make sure to return a true.

SafeERC20 is an internal library of YOUR contract, in the end, when you mock the ERC20 it'll be called with just transfer, just make sure to return a true.

Ah okay, makes sense, and I get it. Thank you very much!


[Feature Request] Back Button

It would be great to get the back button, like who it is in a web browser in the top right.
I used always click into stuff and wasn't able to go to my previous page.
Just noticed as I'm writing this you can go back in the history apple nav section, but I think having a button would be nice as well.

I don't see this as an absolutely necessary feature as

SoundCloud very often has a link to "where you came from" eg. on a track, there is a link to the user who published the track, etc.
You can use the History menu
You can use Cmd and  left and right arrow keys
You can use two finger left and right swipe

Having said that, if someone contributes back/forward buttons with a decent looking user interface, I am not opposed to adding them.

Didn't know about the CMD arrow options. Those work well! Thanks for sharing.

i created a PR that fixes the context menu, which also includes a back and forward menu item


No module named 'aim.sdk.session

Hi, thanks for your nice work.
However, when I ran the follow code you provide:
for DATASET in 'cola' 'sst2' 'mrpc' 'qqp' 'stsb' 'mnli' 'rte' 'wnli' 'qnli'
do
export HPARAMS='{
"dataset": "'$DATASET'",
"lr": 0.0001,
"num_epochs": 20,
"prompts": [],
"reorder_optimized": false,
"max_batch_size": 8,
"max_tokens_sq": 262144, "on_logits":  false, "pooling_index":  null, "seed":  1}'
python3 -m allennlp train
-s .aim/baseline-linear-${DATASET} configs/warp.jsonnet
done
,
I encountered the follow problem:
ModuleNotFoundError: No module named 'aim.sdk.session'.
And I check the version of package "aim = 3.0.2" and the functions in aim.sdk. I did not find the function "session".
Do I get the wrong aim version?

Yes, it does not work with aim v3 yet. Please make sure installing aim v2.

Hi, I'm still confused. I process the "pip install aim==2", yet it cannot install the aim v2.

Hi, I'm still confused. I process the "pip install aim==2", yet it encounters the PackageNotFoundError. I track the documentation about the aim package (https://aimstack.readthedocs.io/en/latest/overview.html), and I cannot find the any information about the function "aim.sdk.session".

Hi, I fix this problem by pip install aim=2.3.
And I closed this issue. Thanks for your answers.


JS syntax highlighting thinks quotes inside of a regular expression should be treated as string quotes

Description of the problem or steps to reproduce
If you have a quote in a regular expression, it treats the quote as the opening of a string, and makes the rest of the file unreadable.
Specifications
Here's an image:

Here's some example code you can use:
console.log("great!");
if (`Does this string have quotes? Let's find out`).match(/['"]/)) {
  console.log("Looks like it does");
}

Commit hash: af520cf
OS: macOS High Sierra
Terminal: iTerm2

Actually the JS highlighting doesn't support regexp at all, it just highlights the expression just like normal tokens

Will making regular expressions a constant override strings? A region could be added, then

(/.*?/[gimuy]?) could be used as a regular expression to test this. However, making my own syntax file (for a different, dummy extension) and putting that in (under line 29) proved not to work as the file was highlighted in exactly the same way. Amy I doing something wrong here?

this regex is actually wrong, because:

unescaped forward slashes
the ? 'optional' quantifier matches 0 or 1, so this is wrong
try this regex instead

/\/.*\/[gimuy]*/g

On Tue, May 29, 2018, 02:20 Victor Veytsman<EMAIL_ADDRESS>wrote:

(/.*?/[gimuy]?) could be used as a regular expression to test this.
However, making my own syntax file (for a different, dummy extension) and
putting that in (under line 29) proved not to work as the file was
highlighted in exactly the same way. Amy I doing something wrong here?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/zyedidia/micro/issues/1110#issuecomment-392621129,
or mute the thread
https://github.com/notifications/unsubscribe-auth/APp7vuJqjm07mbVBwBZcdnQ-EuQjj-Uxks5t3JRBgaJpZM4UHrKY
.

I put *?, using ? to make * lazy instead of greedy, because in a line like
console.log(/abc/, /abc/);

, it would match
/abc/, /abc/

all together. Although, you're right about the flags. Finally, I put in the () because in micro's syntax highlighting it looks to me like the highlighted region is the first capture group. Although, looking at it again, I'm probably wrong.


Sourcing .zshrc again, when using zsh-autosuggestions in conjunction with zsh-syntax-highlighting causes zsh to exit

@zsh-users
This is a curios bug that only occurs in a particular use case. I will list the steps to reproduce.
Environment details:

Kernel Version: Darwin 15.5.0
System Version: OS X 10.11.5
Zsh Version: zsh 5.2 (x86_64-apple-darwin15.0.0)

Pre-requisites:

Mac OS X El Capitan (version 10.11) machine
Zsh installed via brew (and set as default shell)

Instructions for re-producing bug:

Clone the zsh-syntax-highlighting plugin and the zsh-autosuggestions plugin:

git clone https://github.com/zsh-users/zsh-autosuggestions ~/zsh-autosuggestions
git clone https://github.com/zsh-users/zsh-syntax-highlighting ~/zsh-syntax-highlighting

Source both plugins in ~/.zshrc, by adding the following lines in that file:

source ~/zsh-autosuggestions/zsh-autosuggestions.zsh
source ~/zsh-syntax-highlighting/zsh-syntax-highlighting.zsh

Run source ~/.zshrc
After running the command above, explicitly press the Enter key again, and zsh should exit.

Changing the order doesn't make a difference. To be clear, this doesn't happen if you don't explicitly source ~/.zshrc during the current session. I have seen a somewhat similar discussion in the prezto project, but I am not sure if it's the same problem. The factors that I am seeing are slightly different (part of the discussion there revolves around zle reset-prompt which is not referenced anywhere in this project).
I don't have a lot of expertise with zsh and these plugins, but I am happy to help debug this. I have a vanilla Mac OS X 10.11 environment in vagrant setup for testing.
Keep up the great work!

This is a duplicate of #126 so I will be closing this ticket and follow along there. Thanks!


fix: 修正一些对内的地址

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.alex.mm seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


Memory Allocation Fault in LU factorization

This issue was encountered here in Scikit. In short, when trying to perform linear dimensionality reduction for data analysis purposes, the script fails abruptly with a memory allocation error.
After some debugging, I managed to pin down the possible source of the problem to the LU factorization on line line 258 in utils/extmath.py. The fault occurs inside the decomposition in linalg.lu (specifically in line 185 of scipy/linalg/decomp_lu.py): the execution flow passes through the f2py wrapper of dlu_c and fails somewhere inside dgetrf (line 27).
Below is a simple reproducing snippet (on OS X 10.10.5), although LU-decomposing a matrix such as this makes little sense:
import numpy as np

from scipy import linalg

# changing 46336 to 46335 avoids the fault
obj = np.ones((46336, 110), dtype=np.float)
Q, _ = linalg.lu(obj, permute_l=True)

In python 3.6.0 (and in 2.7.13) it fails with
python(723,0x7fff77404300) malloc: *** mach_vm_map(size=18446744065123717120) failed (error code=3)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug

The value<PHONE_NUMBER>5123717120 is FFFF FFFE 003E 9000 in hex looks like a signed extension of an unsigned int.
I used the following setup: python 3.6.0 (default, Dec 24 2016, 08:02:28) (in virtualenv), default numpy ('1.12.0') from pip. I tested both the latest build from git and the pip version ('0.18.1') of scipy.
System information:
System Version: OS X 10.10.5 (14F2109)
Kernel Version: Darwin 14.5.0

No manual linkage to specific linear algebra libraries was done, since according to the reference numpy/scipy automatically links with the Apple's Accelerate/vecLib linear algebra libraries.
Judging from the crash report form python 2.7.13 (crash_report.txt), something odd is going on inside Apple's Accelerate.
Python 2.7.13 (default, Dec 18 2016, 07:03:34)
[GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)] on darwin

cc: @lesteve

Assuming the data is real valued, could you please pass the same data directly to linalg.lapack.dgetrf?

Yes, np.ones(...) creates a matrix of real 1.0 values (double). This example fails with exactly the same fault:
import numpy as np

from scipy import linalg

# changing 46336 to 46335 avoids the fault
obj = np.ones((46336, 110), dtype=np.float64, order="F")
lu_, piv_, info_ = linalg.lapack.dgetrf(obj)

Changing np.float64 to np.float32 does not solve the issue.
The last 3 calls before the fault in the stack traces in the new crash report are identical to the ones in the previous crash report.

In case it helps, from the scikit-learn issue the problem was not happening within a conda environment using MKL, so it may be an OpenBLAS problem on OSX.

Indeed, on my machine I have no issues.

Yeah this is a OSX-specific problem than only happens when using OpenBLAS. I have to say I am not sure how to further debug the problem and I am hoping someone here will have some suggestions.

I am sure this has everything to do with Apple's accelerate, but I am unable to go any further. The following C++ code behaves identically to the python snippets above.
#include <math.h>

// include the clapack headers
#include <Accelerate/Accelerate.h>

int main (int argc, char const *argv[])
{

    int num_rows=46336, num_cols=110;

    // create data array for our matrix. Will be a vector that is num_rows*num_cols elements
    // to access A(r,c) use A[r+c*num_rows], in otherwords major column ordering. {r1c1,r2c1,r3c1...}
    double *A = new double[num_rows*num_cols];
    for(int row=0; row<num_rows; row++) {
        for(int col=0; col<num_cols; col++) {
            A[row+col*num_rows] = 1.0;
        }
    }

    __CLPK_integer m, n, lda, info;
    m=lda=num_rows;
    n=num_cols;

    __CLPK_integer * ipiv = new __CLPK_integer[(num_rows>num_cols?num_cols:num_rows)];

    dgetrf_(&m, &n, A, &lda, ipiv, &info);

    return 0;
/* g++ file.cpp -o file -framework accelerate */
}

This should be compiled with g++ file.cpp -o file -framework accelerate on a recent Mac. Changin 46336 to 46335 agains removes the fault.

Then it's likely an integer overflow inside Accelerate. Only Apple can
fix bugs in Accelerate, so it should be reported to them, and this issue
can be closed.

Closing (looks like accelerate bug).

I guess the simplest fix is to not use accelerate.

I was able to reproduce the original scikit-learn issue using OpenBLAS within an Anaconda environment. Was I somehow still using accelerate?

Sorry, I understood your comment above as that it did not occur with non-accelerate environment.
You can see where the problem is by running it under debugger and seeing whether the backtrace points to openblas or accelerate.
Nevertheless, you should try a similar minimal program as above. If it fails, then it's openblas/lapack issue.

I assumed that the scipy wheels available on PyPI were built against OpenBLAS as is the case on Linux. I was wrong, on OSX the wheels are built against Accelerate, at least if I believe the output of otool -L on scipy/linalg/_fblas*.so. Pinging @matthew-brett just to confirm this.
I could run the Python snippet with OpenBLAS using conda-forge without error.
All in all it seems like an Accelerate-specific problem.

Yes, the OSX wheels are build with Accelerate.   What version of OSX are you running?

Yes, the OSX wheels are build with Accelerate

Thanks for confirming this, which I was not aware of.
@ivannz has opened a bug with Apple and will let us know when he gets an answer.
For completeness:
The version of OSX I am using: OSX 10.12.1, Darwin 16.1.0.
@ivannz's version is different though, so it looks like it is not restricted to only one OSX version:
System Version: OS X 10.10.5 (14F2109)
Kernel Version: Darwin 14.5.0

Same output on
System Version: OS X 10.9.5 (13F1911)
Kernel Version: Darwin 13.4.0

and
      System Version: OS X 10.11.5 (15F34)
      Kernel Version: Darwin 15.5.0

and
      System Version: OS X 10.11.6 (15G1217)
      Kernel Version: Darwin 15.6.0

but no such error on:
      System Version: Mac OS X 10.6.8 (10K549)
      Kernel Version: Darwin 10.8.0

@ivannz - would you mind maintaining a copy of your report and any replies from Apple over at http://www.openradar.me ?

Certainly! Here is the link to Open Radar rdar://30868533.

Is it possible to build the macOS wheels without the Accelerate framework? I mean, are there any alternative libraries pre-installed such as OpenBLAS? Or are we really stuck with Apple's ancient and buggy numerical libraries (with no chance for a quick fix)?

I'm hopeful that once the discussion in #6051 is finalized, we will be back on track with the freedom of choosing BLAS or LAPACK flavor. But it seems that we are stuck until v1.0 with Accelerate

Has anyone tested if the issue still exists in the latest macOS High Sierra beta? I'm hoping that Apple has noticed the Accelerate bug and fixed it silently...

Seems this was closed incorrectly, so reopening.
The two examples don't crash for me on OS X 10.12 with scipy master built against Accelerate. Not sure if that's a change in scipy or in Accelerate.

I just built numpy and scipy from their master branches, and the code given in the description above crashes on my late 2013 Macbook running OS X 10.9.5, with Python 3.5.2:
In [1]: import numpy as np

In [2]: np.__version__
Out[2]: '1.14.0.dev0+9afb77a'

In [3]: import scipy

In [4]: scipy.__version__
Out[4]: '1.0.0.dev0+85287f9'

In [5]: from scipy import linalg

In [6]: obj = np.ones((46336, 110), dtype=np.float)

In [7]: Q, _ = linalg.lu(obj, permute_l=True)
python(15120,0x7fff74a9d310) malloc: *** mach_vm_map(size=18446744065123717120) failed (error code=3)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
Segmentation fault: 11

If I use numpy from Anaconda (numpy 1.12.1), I don't get the crash.

@rgommers the C++ example as well as all Python examples (using scipy master) still crash on macOS 10.12.6 (latest version released on July 19). If anyone has the preview of macOS 10.13 (High Sierra), it would be great to hear if the problem has been fixed by Apple.
In any case, I suggest to add a test to scipy to test against this issue - WDYT?

@cbrnr Could you also please add the result of scipy.linalg.lapack.ilaver()

It looks to me the diagnosis of the issue was correct, and was
reproduced with a standalone C program --- there clearly is a bug in
Accelerate, and the only way Scipy can fix it is by not using Accelerate.

@pv yes - unless Apple fixes the bug, which might have already happened in 10.13 Preview...

and the only way Scipy can fix it is by not using Accelerate.

@pv Ahem... #6051 anyone ? :-D

>>> scipy.linalg.lapack.ilaver()
(3, 2, 1)

If we add a test, we should add a check in setup.py that raises an error
if trying to build against Accelerate. Otherwise, you'll just get a
segfaulting test suite.
.
I would just suggest dropping Accelerate. There's no point in spending
resources of an open source project in trying to debug software provided
by one of the world's biggest and most profitable companies with
hundreds of thousands of employees, who seem not even willing to
communicate publicly about defects in their software products.

Yep, adding this test basically means that scipy is broken on macOS - I don't know how to best handle this. I guess such questions had better be discussed in #6051...

I think #6051 is saturated and does not reflect the intent the discussion happened towards the end. I don't have any experience with Accelerate to construct meaningful sentences.
@cbrnr Would you be interested in making separate issue for this ? (By the way, ouch for the LAPACK version)
I can also try to summarize that thread if nobody wants to take it on

@ilayn that would be great. What I would suggest is not to do that in a comment but in a short markdown/rest doc (put it in a gist maybe or on the wiki). That way others can contribute and we can get something with the pros/cons for each option properly summarized.

@rgommers @cbrnr Let me try.

@rgommers Is this gist  something that you can work with? I hope I understood correctly.

Thanks @ilayn, that's a good start. Motivation is good as is, the second half is a bit incomplete. I would suggest this structure, working it out in a bit more detail):
Options
--------

Keep supporting Accelerate as we do now
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
*Pros*: least effort, building/installing SciPy on OS X is easy.

*Cons*: cannot make use of recently introduced functionality in LAPACK (e.g. gh-xxxx, ) or fix bugs that are due to Accelerate (e.g. gh-7131)

Drop Accelerate support completely
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
*Pros*: can move to LAPACK 3.y.z, can remove Accelerate patching in ``scipy._build_utils``

*Cons*: some people want to use Accelerate for performance reasons (link), we force users to install another BLAS/LAPACK which is not easy right now. This could be addressed by packaging OpenBLAS as a wheel, but that's a significant amount of work too.

In particular, these are the questions to address (copy from https://github.com/scipy/scipy/pull/6051#issuecomment-290933088)

Keep using Accelerate BLAS but shadow LAPACK
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
this one is a bit harder to summarize, need to read through gh-6051. IIRC can be done in a couple of ways
- Using ``vecLibFort``, see https://github.com/scipy/scipy/pull/6051#issuecomment-225431271
- Just bundling missing + broken routines: https://github.com/scipy/scipy/pull/6051#issuecomment-225431600
- <I think there's more flavors>

Other considerations
---------------------

Which minimum LAPACK version to support
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
See https://github.com/scipy/scipy/pull/6051#issuecomment-265656371

ABI issues with shadowing
~~~~~~~~~~~~~~~~~~~~~
MKL also uses a g77 ABI, see https://github.com/scipy/scipy/pull/6051#issuecomment-225431914

Rejected alternatives
---------------------
Dropping all of LAPACK into SciPy (see https://github.com/scipy/scipy/pull/6051#issuecomment-266380190).

Sorry I'm boarding a plane in an hour and will probably not have time for more than drive-by commenting in the next few weeks. But I think if you can work things out as far as possible along these lines, then some of the people on gh-6051 will be happy to fill in the remaining blanks.

While I would gladly maintain this doc somewhere, I'm really not into this topic at the required level. So I will leave it in an issue and maintainers with edit priviledges can fix/modify further.

It would be great if everyone could just add comments and edits instead of you having to edit the document all the time. Maybe a separate issue is a good starting point for that. I'd just highlight that scipy is currently broken on macOS due to it depending on Accelerate by default. Other open-source alternatives (OpenBLAS) should be highlighted more because these issues just can't happen. MKL would be another alternative, but since it is also proprietary the same issues might occur (and probably already have occurred at some point).

Suggest the wiki (https://github.com/scipy/scipy/wiki). You should be able to make a new page there - much easier for multiple people to edit.

Ah perfect. Wiki it is.

Initial Wiki page is created at https://github.com/scipy/scipy/wiki/Dropping-support-for-Accelerate

Sad as it is, it is time to drop Accelerate. Integer overflow is even present in fundamental BLAS functions like *GEMM. This means that even the proposed solutions will never work properly. Even if we build LAPACK on top of Accelerate's CBLAS it will still be unstable. There is no chance of Apple ever updating Accelerate or fixing the known bugs. Dropping Accelerate support is the only sane solution, even for NumPy. We need to focus on supporting MKL, OpenBLAS, and ATLAS.

@sturlamolden I've added your example to the wiki document. That's indeed unfortunate. Should I also create an issue on the NumPy side too?

Here is an example of NumPy failing due to integer overflow in Accelerate *GEMM: https://github.com/numpy/numpy/issues/5533

Also note that ATLAS has this problem, it is not Accelerate specific. It is due to the helper functions catlas_dset et al.

Here is an example of scikit-learn running into this issue:
https://github.com/scikit-learn/scikit-learn/issues/4197

I am not completely sure that I've read the thread carefully enough, but just in case this is useful:
On macOS 10.12.6 (i.e., latest non-beta Sierra, as of 24 July 2017), with Python 2.7.13 or Python 3.6.2 framework builds from python.org and pip-installed numpy and scipy (which should therefore use Accelerate) the original python script in the message from @ivannz at the top of this thread runs without crashing.
Similarly, the g++ code works fine when compiled with Xcode g++ [really Apple LLVM version 8.1.0 (clang-802.0.42)].
I came to this following the thread suggesting dropping accelerate which seems like a terrible to me... I think you will lose a lot of users in the scientific community who need this functionality but don't have the time to deal with extra installations...

@defjaf Please have a look at the arguments in the wiki page --> https://github.com/scipy/scipy/wiki/Dropping-support-for-Accelerate
There are many problems provided which is due to this framework and just look at the examples given by sturlamolden above your comment. It is not an installation problem. OSX support is not being dropped. It's the library that the build will link to, that changes. I don't know who we are or which users will be lost but all will benefit from this.
As being an ex-academic all the way, scientific community has all the time in the world to download <50 MBs and run two more lines of commands on the terminal to have a more stable environment.

@ilayn, sorry, I misunderstood whether @sturlamolden was citing new incidents of the original issue cropping up rather than unrelated issues.
However, for what it's worth, numpy issue 5533 does not crash my machine with the setup as above (I did not read that thread further than the first message, so apologies if that is not the "real" problem.)
In any case, I do agree that we should have an overall working set of routines...
On the other hand, I can tell you from first-hand experience (both with my own machine and attempting to debug others' setups) that external dependences are very often a PITA to handle, and are rarely trivial, due to conflicts, upgrades, permissions, environment variables, etc. And, yes, scientists should be savvy enough to deal with this but often do not have the requisite mental bandwidth available after the first failure... So if we do abandon Accelerate, please do make the outcome as frictionless as possible...

I'm building scipy on Windows machines or rather failing at it continuously I should say 😃 So, please trust me when I say that I share your concerns wholeheartedly.

For what it is worth, the LU code in question does not fail on my system either (macOS 10.12.6, Python 3.6.1, scipy 0.18.1). This illustrates another issue with Accelerate, that fixes to the framework are distributed as OS upgrades, and not backported to previous OS versions.
However, it is still the case that the LAPACK version is old because the interface is frozen on CLAPACK which is abandonware.
I do not have enough RAM to check if the *GEMM integer overflow due to code from ATLAS is still present, but is has a simple fix: raise an exception is arrays have more than<PHONE_NUMBER> and we are using Accerate or ATLAS.
I do not advocate replacing Accelerate with ATLAS for the macOS wheels, though. ATLAS is equally buggy as Accelerate, though less performant. It will have to be OpenBLAS. Or, we could accept the quirks of Accelerate but build a recent LAPACK on top of it.

@defjaf @sturlamolden I have macOS 10.12.6 and the C++ program (compiled with Apple LLVM version 8.1.0 (clang-802.0.42)) does not work (i.e. it crashes with a segfault). Same for the Python examples.

Oh, this is embarassing. It seems my default SciPy was built with MKL... That is why it didn't segfault...
Rebuilding with clang and Accelerate and it does indeed fail.
(Building with GCC does not even work, because of errors in veclib.h. I guess that must have changed after the supgrade to macOS Sierra...)

Incidentally, openblas is now available in core Homebrew.

OK, I'm confused. My installation definitely doesn't crash -- I use the wheels that get downloaded and installed with pip install numpy scipy (either pip2 or pip3) -- I know that there is a way to check how numpy and scipy are compiled but I don't remember the incantation...

Since the underlying issue is with Accelerate, the C example by @ivannz should also crash - it is strange that this is working for you. Could you double-check?

@cbrnr Yes, it absolutely runs on my machine, which has the latest 10.12.6 and Xcode (as I think @sturlamolden points out, the issue may be that Apple distributes fixes through OS updates). I have tried g++, clang++ and even some homebrew llvm versions.

But does Apple distribute fixes only to certain users/machines? I'm on the same macOS and Xcode versions...

Just to rule one thing out, do you have Anaconda installed? If so, please remove it (or at least everything on the path). You can never be sure what they include in their distribution (I wouldn't be surprised if they included compilers).

No, this is the latest Python.org framework installs of 2.7.* and 3.6.* with pip installation of packages.

which g++?

$ g++ --version
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer//usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 8.1.0 (clang-802.0.42)
Target: x86_64-apple-darwin16.7.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

Same here (only without the strange double slash in the prefix argument). Maybe for whatever reason this particular number 46336 does not cause an overflow on your machine, but some other number does. It is pretty unlikely that you have a different version of Accelerate if we're on the same OS and Xcode versions.
The whole back and forth only goes to show that Accelerate (and ATLAS) seem to have a lot of bugs that are difficult to reproduce (e.g. NumPy 5533 works for me) and/or won't be fixed in a while, so choosing OpenBLAS or MKL as primary options seems to be a good idea. And maybe I should just install Linux on my MacBook...

@defjaf That is not really g++ but actually a symlink to clang++.  Here is roughly what you should see with a real g++:
sturla$ which g++
/usr/local/gfortran/bin/g++

sturla$ g++ --version
g++ (GCC) 6.3.0
Copyright (C) 2016 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

@sturlamolden Apple doesn't ship gcc - all gcc binaries point to clang alternatives.

@cbrnr Yes, that is what I meant. GCC must be installed separately.

OK, but on my machine the example crashes even when using clang. How is the compiler choice relevant to this discussion (sorry, the thread is getting longer and longer)?

sturla$ clang++ file.cpp -o file -framework accelerate
sturla$ ./file
file(23769,0x7ffff128e3c0) malloc: *** mach_vm_map(size=18446744065123717120) failed (error code=3)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
Segmentation fault: 11

Changing number of rows to 46335:
sturla$ clang++ file.cpp -o file -framework accelerate
sturla$ ./file
sturla$ _

Compiler version:
sturla$ clang++ --version
Apple LLVM version 8.1.0 (clang-802.0.42)
Target: x86_64-apple-darwin16.7.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

Yes, I thought this was the point - it crashes with the default compiler (but not on @defjaf's setup, which also uses clang). What about gcc in this context? It doesn't work with accelerate and spits out compilation errors.

Edit: OK, it works with -flax-vector-conversions. No segfault.

What did I just do? Now the C example works with both clang++ and g++... 😱  Did I install any library update with Homebrew? Could this be not related to Accelerate after all?

@cbrnr what about the python version?

sturla$ clang++ -o file file.cpp -flax-vector-conversions -framework accelerate
sturla$ ./file
file(24274,0x7ffff128e3c0) malloc: *** mach_vm_map(size=18446744065123717120) failed (error code=3)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
Segmentation fault: 11

@defjaf Python works too. 😕  I have no idea what's going on...
@sturlamolden I meant -flax-vector-conversions is required to compile with g++.

Perhaps this verbose compiler output might help find any differences in our setups? (Note that the interesting lines are the long ones starting with quote marks where the compilation and linking actually happens...)
$ clang++ -v ./accelcrash.cpp -o accelcrash -flax-vector-conversions -framework accelerate
Apple LLVM version 8.1.0 (clang-802.0.42)
Target: x86_64-apple-darwin16.7.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
 "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang" -cc1 -triple x86_64-apple-macosx10.12.0 -Wdeprecated-objc-isa-usage -Werror=deprecated-objc-isa-usage -emit-obj -mrelax-all -disable-free -disable-llvm-verifier -discard-value-names -main-file-name accelcrash.cpp -mrelocation-model pic -pic-level 2 -mthread-model posix -mdisable-fp-elim -masm-verbose -munwind-tables -target-cpu penryn -target-linker-version 278.4 -v -dwarf-column-info -debugger-tuning=lldb -resource-dir /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/8.1.0 -stdlib=libc++ -fdeprecated-macro -fdebug-compilation-dir /Users/jaffe/home/python -ferror-limit 19 -fmessage-length 80 -stack-protector 1 -fblocks -fobjc-runtime=macosx-10.12.0 -fencode-extended-block-signature -fcxx-exceptions -fexceptions -fmax-type-align=16 -fdiagnostics-show-option -fcolor-diagnostics -o /var/folders/nc/n7d98d0x46g0h5yyb4bc1gv40000gv/T/accelcrash-3fb9a3.o -x c++ ./accelcrash.cpp
clang -cc1 version 8.1.0 (clang-802.0.42) default target x86_64-apple-darwin16.7.0
ignoring nonexistent directory "/usr/include/c++/v1"
#include "..." search starts here:
#include <...> search starts here:
 /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1
 /usr/local/include
 /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/8.1.0/include
 /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include
 /usr/include
 /System/Library/Frameworks (framework directory)
 /Library/Frameworks (framework directory)
End of search list.
 "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld" -demangle -lto_library /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/libLTO.dylib -no_deduplicate -dynamic -arch x86_64 -macosx_version_min 10.12.0 -o accelcrash /var/folders/nc/n7d98d0x46g0h5yyb4bc1gv40000gv/T/accelcrash-3fb9a3.o -framework accelerate -lc++ -lSystem /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/8.1.0/lib/darwin/libclang_rt.osx.a

@defjaf my output looks identical. Note that now all of a sudden it works (just like on your machine) and I can't go back to when it didn't.

Yep, this was more directed @sturlamolden: what about you, since you're still seeing the crash(es)?...

Apparently, you have accidentally Microsoft-ified your machines. That's my default daily headscratching position. Things work, based on planets' alignment and weather forecast.

sturla$ $ clang++ -v ./file.cpp -o file -flax-vector-conversions -framework accelerate
-bash: $: command not found

@sturlamolden You (accidentally?) copied my bash prompt character, $! Just start with clang++ .... (And as before neither of us need -flax-vector-conversions)

Oops...
sturla$ clang++ -v -o file file.cpp -flax-vector-conversions -framework accelerate
Apple LLVM version 8.1.0 (clang-802.0.42)
Target: x86_64-apple-darwin16.7.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
 "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang" -cc1 -triple x86_64-apple-macosx10.12.0 -Wdeprecated-objc-isa-usage -Werror=deprecated-objc-isa-usage -emit-obj -mrelax-all -disable-free -disable-llvm-verifier -discard-value-names -main-file-name file.cpp -mrelocation-model pic -pic-level 2 -mthread-model posix -mdisable-fp-elim -masm-verbose -munwind-tables -target-cpu penryn -target-linker-version 278.4 -v -dwarf-column-info -debugger-tuning=lldb -resource-dir /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/8.1.0 -stdlib=libc++ -fdeprecated-macro -fdebug-compilation-dir /Users/sturla/development/lutest -ferror-limit 19 -fmessage-length 80 -stack-protector 1 -fblocks -fobjc-runtime=macosx-10.12.0 -fencode-extended-block-signature -fcxx-exceptions -fexceptions -fmax-type-align=16 -fdiagnostics-show-option -fcolor-diagnostics -o /var/folders/yl/_1z15lyj0w95_0tpv51sh47m0000gn/T/file-b0d43d.o -x c++ file.cpp
clang -cc1 version 8.1.0 (clang-802.0.42) default target x86_64-apple-darwin16.7.0
ignoring nonexistent directory "/usr/include/c++/v1"
#include "..." search starts here:
#include <...> search starts here:
 /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1
 /usr/local/include
 /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/8.1.0/include
 /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include
 /usr/include
 /System/Library/Frameworks (framework directory)
 /Library/Frameworks (framework directory)
End of search list.
 "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld" -demangle -lto_library /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/libLTO.dylib -no_deduplicate -dynamic -arch x86_64 -macosx_version_min 10.12.0 -o file /var/folders/yl/_1z15lyj0w95_0tpv51sh47m0000gn/T/file-b0d43d.o -framework accelerate -lc++ -lSystem /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/8.1.0/lib/darwin/libclang_rt.osx.a

@ilayn 🤣  in reality, we should all be using Linux for scientific computing (and actually I am, but just not on my MacBook)...

@sturlamolden No differences other than filenames...

@ilayn It certainly feels that way after the upgrade to Sierra. Now my gfortran does not work because as does not understand the -m flag.

@sturlamolden plain -m? This flag does not exist on Linux either.
Anyway, this discussion is digressing too far from the main issue here. But it's clear that there are some major issues popping up here and there. I'm not sure this is helping in the decision how SciPy should proceed with official macOS builds.

@sturlamolden @cbrnr I agree (but nb. gfortran from homebrew works fine on my machine... perhaps this points to something brittle in your setup?)

I guess Apple hasn't fixed the issue. On the latest Sierra
      System Version: macOS 10.12.6 (16G29)
      Kernel Version: Darwin 16.7.0

I get the segfault for these new constants (just used the binary search from 46335 upward to find a new failure threshold):
    int num_rows=46344, num_cols=110;

Changing 46343 to 46344 or any higher value also causes a segfault. Changing 110 to a higher value, while keeping num_rows = 46343 does not cause a segfault.
I replicated this behaviour both with the standalone C code and the python example at the top of the thread.
PS: There has been no activity around my bug report to Apple so far.

Closing, there's nothing we can do aside from dropping Accelerate.

I just upgraded to macOS High Sierra (10.13) and the bug seems to have been fixed by Apple. All examples above (both Python and C++) work! 🎉

Today I got an email from the developers, asking to test if the bug persists in High Sierra 10.13 GM.

This is a follow-up regarding regarding Bug ID# 30868533.
Please verify this issue with the macOS 10.13 GM and update your bug report at https://bugreport.apple.com/ with your results.
macOS High Sierra 10.13 GM (17A365)
https://developer.apple.com/download/
Posted Date: Sep 25, 2017
If the issue persists, please attach a new sysdiagnose captured in the latest build and attach it to the bug report.
macOS sysdiagnose Instructions:
https://developer.apple.com/services-account/download?path=/OS_X/OS_X_Logs/sysdiagnose_Logging_Instructions.pdf

Unfortunately, I won't be able to test this issue in the near future myself.
@cbrnr Could I ask you, please, if it is not too inconvenient, to try increasing
the num_rows variable here
int num_rows=46344, num_cols=110;

For example, could you try 60000 or higher and if it ever fails use, perhaps,
binary search to pinpoint the threshold of failure? Again, I apologise for not
being able to do this test myself.

Downloading High Sierra now, still half an hour to go. :)

Indeed, the bug seems to be fixed in High Sierra 😃

@ivannz @sturlamolden it works up to about num_rows=5925000, but a little higher and it crashes due to a malloc error. I think this is expected on my 16GB machine, right?

Hm, I can confirm that raising num_rows triggers the error again. At this point the data array is a little less than 5 GB.

And with MKL it actually works with these large matrices... So 👎  for this "fix"...

@cbrnr how do you use MKL on macos sierra 10.12.6? I try to install it with pip install numpy-mkl but pip return an error Could not find a version that satisfies the requirement numpy-mkl. It's weird because when I search numpy-mkl with pip is listed 😖

I use it with Anaconda, but I'd be highly interested if it is possible to install it so that I can use MKL with my Homebrew Python.

@ivannz and @matthew-brett the sample C code provided with this issue shows no problem on the Apple M1. The Apple Accelerate frameworks are able to leverage the dedicated AMX hardware on the M1. This can provide superior performance for some functions relative to openBLAS.
Will SciPy and numpy ever reconsider using Accelerate? I think the strongest argument on the SciPy page is the opaque support Apple has for their tools, making discovery and resolution of bugs hard to track. However, it would be great if the Apple Developer Ecosystem Engineering team could be engaged with core tools like SciPy and numpy in the same way they have helped port Intel's Embree to Apple Silicon.

@ivannz and @matthew-brett the sample C code provided with this issue shows no problem on the Apple M1. The Apple Accelerate frameworks are able to leverage the dedicated AMX hardware on the M1. This can provide superior performance for some functions relative to openBLAS.
Will SciPy and numpy ever reconsider using Accelerate? I think the strongest argument on the SciPy page is the opaque support Apple has for their tools, making discovery and resolution of bugs hard to track. However, it would be great if the Apple Developer Ecosystem Engineering team could be engaged with core tools like SciPy and numpy in the same way they have helped port Intel's Embree to Apple Silicon.

I'd be 💯 for this! Can we tag someone at Apple who might be able to chime in?

I'd be 💯 for this! Can we tag someone at Apple who might be able to chime in?

Will SciPy and numpy ever reconsider using Accelerate?

If Apple is interested and addresses the issues outlined at https://github.com/scipy/scipy/wiki/Dropping-support-for-Accelerate, then of course we can reconsider. We support MKL as well which is not open source, because Intel maintains it well and when there's an issue we have actual humans to talk to.

think the strongest argument on the SciPy page is the opaque support Apple has for their tools, making discovery and resolution of bugs hard to track.

Accelerate being stuck on a too-low version of LAPACK (3.2.1 last time I checked) is also a real blocker. Our minimum is 3.4.1 at the moment I believe, and we may gradually increase that minimum. A lowest version that's from 2012 isn't too much to ask.

However, it would be great if the Apple Developer Ecosystem Engineering team could be engaged with core tools like SciPy and numpy in the same way they have helped port Intel's Embree to Apple Silicon.

I'd be 💯 for this! Can we tag someone at Apple who might be able to chime in?

@lawrence-danna-apple given that you worked on M1 support for Python packaging, maybe you have an opinion on this topic or know who to ping?

Will SciPy and numpy ever reconsider using Accelerate?

If Apple is interested and addresses the issues outlined at https://github.com/scipy/scipy/wiki/Dropping-support-for-Accelerate, then of course we can reconsider. We support MKL as well which is not open source, because Intel maintains it well and when there's an issue we have actual humans to talk to.

think the strongest argument on the SciPy page is the opaque support Apple has for their tools, making discovery and resolution of bugs hard to track.

Accelerate being stuck on a too-low version of LAPACK (3.2.1 last time I checked) is also a real blocker. Our minimum is 3.4.1 at the moment I believe, and we may gradually increase that minimum. A lowest version that's from 2012 isn't too much to ask.

However, it would be great if the Apple Developer Ecosystem Engineering team could be engaged with core tools like SciPy and numpy in the same way they have helped port Intel's Embree to Apple Silicon.

I'd be 💯 for this! Can we tag someone at Apple who might be able to chime in?

@lawrence-danna-apple given that you worked on M1 support for Python packaging, maybe you have an opinion on this topic or know who to ping?

The mains issues are:

Accelerate is not fork safe
Accelerate does not support 64-bit BLAS
LAPACK version is not really an issue, as we can build LAPACK from Fortran sources on top of BLAS (like we do for OpenBLAS).

The mains issues are:

Accelerate is not fork safe
Accelerate does not support 64-bit BLAS
LAPACK version is not really an issue, as we can build LAPACK from Fortran sources on top of BLAS (like we do for OpenBLAS).

While I wholeheartedly agree that M1 users should be able to join the number crunching community if accelerate is overhauled, however, I am not sure whether apple would bother with it since it doesn't just have blas/lapack but all kinds of extras in that framework that needs redeveloping.
Since even LAPACK devs joined the fast GitHub track and started a much quicker release cycle, a turtle-speed and coldwar-responses won't cut it for us to enable a maintenance effort so it has many meta issues as well as technical ones.

While I wholeheartedly agree that M1 users should be able to join the number crunching community if accelerate is overhauled, however, I am not sure whether apple would bother with it since it doesn't just have blas/lapack but all kinds of extras in that framework that needs redeveloping.
Since even LAPACK devs joined the fast GitHub track and started a much quicker release cycle, a turtle-speed and coldwar-responses won't cut it for us to enable a maintenance effort so it has many meta issues as well as technical ones.

Given the divergence described with Accelerate, a simpler option might be for the Apple Developer Ecosystem Engineering team to provide patches to optimize OpenBLAS. This would parallel their support of open source projects like Embree and Clang/LLVM. Support could be added piecemeal and tested against the existing functional but not optimized code. I do think direct help from Apple would help, though some have begun to reverse engineer the AMX instructions.

Given the divergence described with Accelerate, a simpler option might be for the Apple Developer Ecosystem Engineering team to provide patches to optimize OpenBLAS. This would parallel their support of open source projects like Embree and Clang/LLVM. Support could be added piecemeal and tested against the existing functional but not optimized code. I do think direct help from Apple would help, though some have begun to reverse engineer the AMX instructions.

I honestly do not think Apple could care less about Accelerate or OpenBLAS. They want everyone to use Metal 2 for any serious number crunching. They have already deprecated OpenCL. Personally I am wainting for them to do away with Accelerate and GCD as well. Either we jump on the Metal bandwagon or we forget about Mac as a platform for serious number crunching and 3D graphics. That is the options we are given, and honestly it is an easy choice: just ignore Apple. I thought I would stick to Mac because that is what I like to use, but now I am considering to run Linux on my next desktop computer. It seems that is the way scientific computing is heading.

I honestly do not think Apple could care less about Accelerate or OpenBLAS. They want everyone to use Metal 2 for any serious number crunching. They have already deprecated OpenCL. Personally I am wainting for them to do away with Accelerate and GCD as well. Either we jump on the Metal bandwagon or we forget about Mac as a platform for serious number crunching and 3D graphics. That is the options we are given, and honestly it is an easy choice: just ignore Apple. I thought I would stick to Mac because that is what I like to use, but now I am considering to run Linux on my next desktop computer. It seems that is the way scientific computing is heading.

The thing with Apple is, once we have fully embraced Metal, they will come up with something new and deprecate it.   At some point we just have to tell them they will be irrelevant if they continue to try to make themselves irrelevant.

The thing with Apple is, once we have fully embraced Metal, they will come up with something new and deprecate it.   At some point we just have to tell them they will be irrelevant if they continue to try to make themselves irrelevant.

Well we can't ignore it, even if we're annoyed with Accelerate's (lack of) maintenance historically. We have enough users (and devs, like me) who use macOS that we want to provide both from-source build options and wheels with decent performance. Metal isn't really useful until there's a BLAS/LAPACK library for it.
There's also Intel MKL on macOS - it's not open source so not good for wheels, but it's fine to develop with. We'll take what we can get.

Well we can't ignore it, even if we're annoyed with Accelerate's (lack of) maintenance historically. We have enough users (and devs, like me) who use macOS that we want to provide both from-source build options and wheels with decent performance. Metal isn't really useful until there's a BLAS/LAPACK library for it.
There's also Intel MKL on macOS - it's not open source so not good for wheels, but it's fine to develop with. We'll take what we can get.

@sturlamolden as I note on my review Metal has several limitations for matrix mathematics: limited to single precision, limited to maximum dimension of 65535, requires dimensions divisible by 8 for decent performance. Apple AMX removes these limitations and outperforms the Metal MPSMatrixMultiplication. You can demonstrate this for yourself by compiling the projects here. If Apple really felt Metal was the preferred solution, why would they waste the die space to implement the matrix co-processor?

@sturlamolden as I note on my review Metal has several limitations for matrix mathematics: limited to single precision, limited to maximum dimension of 65535, requires dimensions divisible by 8 for decent performance. Apple AMX removes these limitations and outperforms the Metal MPSMatrixMultiplication. You can demonstrate this for yourself by compiling the projects here. If Apple really felt Metal was the preferred solution, why would they waste the die space to implement the matrix co-processor?

@sturlamolden given how much faster M1 perform it would be a shame to ignore Apple. A lot of people seem to be going back to Macs just because of this when they previously would have rather bought Dell/... running Linux or even Windows (WSL2). @rgommers Intel MKL won't work on M1 which is clearly the future for Macs.

@sturlamolden given how much faster M1 perform it would be a shame to ignore Apple. A lot of people seem to be going back to Macs just because of this when they previously would have rather bought Dell/... running Linux or even Windows (WSL2). @rgommers Intel MKL won't work on M1 which is clearly the future for Macs.

I don't have any strong opinion about Apple but I am leaning towards force Apple's hand instead of us (OSS in general) are running around just to support their stuff. Unfortunately that's what you get if you want to pay for M1. Performance means nothing if notihng supports it.

I don't have any strong opinion about Apple but I am leaning towards force Apple's hand instead of us (OSS in general) are running around just to support their stuff. Unfortunately that's what you get if you want to pay for M1. Performance means nothing if notihng supports it.

By the way should we get to a brand new issue instead of hiding inside an old  linalg issue?

By the way should we get to a brand new issue instead of hiding inside an old  linalg issue?

The only reason I re-ignited this here is that the specific issue raised by this issue seems to be resolved with the M1 (or perhaps the latest versions of accelerate). Naively, I thought this was one of the few blockers to using the accelerate framework. There are clearly a lot more issues that I was not aware of. This is well outside my expertise. I do appreciate the insightful and rapid feedback from SciPy developers. This suggests a vibrant community.
My sense is that this question dies here unless someone closely involved with either SciPy or Apple takes the lead. If anyone fits that criteria, I encourage them to start a new issue.

The only reason I re-ignited this here is that the specific issue raised by this issue seems to be resolved with the M1 (or perhaps the latest versions of accelerate). Naively, I thought this was one of the few blockers to using the accelerate framework. There are clearly a lot more issues that I was not aware of. This is well outside my expertise. I do appreciate the insightful and rapid feedback from SciPy developers. This suggests a vibrant community.
My sense is that this question dies here unless someone closely involved with either SciPy or Apple takes the lead. If anyone fits that criteria, I encourage them to start a new issue.

I do appreciate the insightful and rapid feedback from SciPy developers. This suggests a vibrant community.

glad to hear that. we try to create a good community and be as responsive as we can:)

My sense is that this question dies here unless someone closely involved with either SciPy or Apple takes the lead. If anyone fits that criteria, I encourage them to start a new issue.

Let me just link to https://github.com/numpy/numpy/issues/18143, which is the immediate bigger issue here. NumPy needs to provide universal2 wheels (and macos-arm64 thin wheels), which includes figuring out what BLAS/LAPACK to use for the native M1 build. Once that's done, following suit for SciPy should be not too difficult.

I do appreciate the insightful and rapid feedback from SciPy developers. This suggests a vibrant community.

glad to hear that. we try to create a good community and be as responsive as we can:)

My sense is that this question dies here unless someone closely involved with either SciPy or Apple takes the lead. If anyone fits that criteria, I encourage them to start a new issue.

Let me just link to https://github.com/numpy/numpy/issues/18143, which is the immediate bigger issue here. NumPy needs to provide universal2 wheels (and macos-arm64 thin wheels), which includes figuring out what BLAS/LAPACK to use for the native M1 build. Once that's done, following suit for SciPy should be not too difficult.


BUG? Index doesn't have an .empty attribute

Code Sample, a copy-pastable example if possible
In [108]: i = pd.Index([])

In [109]: bool(i)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-109-43b3c3723ad4> in <module>()
----> 1 bool(i)

C:\Users\Chris\Anaconda\envs\py35\lib\site-packages\pandas\indexes\base.py in __nonzero__(self)
   1227         raise ValueError("The truth value of a {0} is ambiguous. "
   1228                          "Use a.empty, a.bool(), a.item(), a.any() or a.all()."
-> 1229                          .format(self.__class__.__name__))
   1230
   1231     __bool__ = __nonzero__

ValueError: The truth value of a Index is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().

In [110]: i.empty
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-110-1524e0ba750e> in <module>()
----> 1 i.empty

AttributeError: 'Index' object has no attribute 'empty'

yeah this could be added (or more likely the Series one moved to the Mixin)

@jreback I would like to work on this. Probably add an attribute to Index, perhaps in the pandas/indexes/base.py file?

@pfrcks - I think you could add it to the IndexOpsMixIn class https://github.com/pydata/pandas/blob/master/pandas/core/base.py#L794

@chris-b1 @jreback I've implemented the empty attribute and added unittests for it too. Kindly take a look.


addAccount API to support adal-node to msal-node migration

Currently, acquireTokenByRefreshToken API will not cache the response from service if there exists no corresponding account entity in the cache already. This diminishes the usefulness of acquireTokenByRefreshToken for apps migrating from adal-node to msal-node as it prevents them from making use of their still valid refresh tokens without prompting their users.
This PR proposes an addAccount API to msal-node TokenCache class, which takes in a raw ID token string and optionally a base64 encoded clientInfo object to create an AccountEntity and the relevant cache record in the cache. This in turn enables acquireTokenByRefreshToken calls to successfully cache RT flow response in migration scenarios.
This also adds a sample to demonstrate usage.
todo:

unit tests

cc @tnorling

Codecov Report

Merging #4917 (6b4d7e4) into dev (b315d95) will decrease coverage by 0.10%.
The diff coverage is 26.31%.

Flag
Coverage Δ

*Carryforward flag

msal-angular
96.42% <ø> (ø)

Carriedforward from b315d95

msal-browser
86.87% <ø> (ø)

Carriedforward from b315d95

msal-common
84.44% <ø> (ø)

Carriedforward from b315d95

msal-core
82.67% <ø> (ø)

Carriedforward from b315d95

msal-node
80.45% <26.31%> (-1.49%)
:arrow_down:

msal-node-extensions
76.03% <ø> (ø)

Carriedforward from b315d95

msal-react
94.50% <ø> (ø)

Carriedforward from b315d95

node-token-validation
86.92% <ø> (ø)

Carriedforward from b315d95

*This pull request uses carry forward flags. Click here to find out more.

Impacted Files
Coverage Δ

lib/msal-node/src/client/ClientApplication.ts
73.00% <ø> (ø)

lib/msal-node/src/cache/TokenCache.ts
64.64% <26.31%> (-10.06%)
:arrow_down:

We have discussed this offline and decided that it introduces unnecessary complexity. Closing.


Added endSelection event

This is needed because even though startSelection always fires from the MouseTracker, the cancel event in Tool is basically ignored. This makes cancelSelected and createSelection work unreliably as indicators that drawing of a very small rectangle has ended.
This new event only fires if startSelection has previously fired.
TODO: Add documentation
It could be that this PR belongs in the base Annotorious repo. I need it in my fork of Annotorious-openseadragon, but I'll be happy to open it there also.

Hi,
thanks for the message! Just to confirm: this will always fire when a selection has ended, correct? Regardless of whether the selection has ended by creating a new shape, or by canceling the selection (because the rectangle was too small). And in the case a new selection is created, the event sequence would be startSelection -> endSelection -> selectionCreated?

To be honest, I forgot everything about this codebase. I'll have to get back into it this week.

After looking at the code again, yes, it seems that the endSelection event gets triggered always. The chain of events would actually be startSelection-> endSelection and then either complete, which triggers selectionCreated, or cancel, which does nothing as far as I understand.

Merged this now - thanks!

Great, but we shouldn't forget to add the documentation!


Support ChainerX in text classification example

Part of #6661
~Merge #6768 first.~

Rebased.

LGTM but I left some nitpicks. Could you take a look?

Jenkins, test this please.

Jenkins CI test (for commit f7f0266e526036dc1da3b22fd494a69d46f1d916, target branch master) failed with status FAILURE.

Could you update the branch?

Jenkins, test this please.

Jenkins CI test (for commit f7f0266e526036dc1da3b22fd494a69d46f1d916, target branch master) failed with status FAILURE.

Jenkins, test this please.

Jenkins CI test (for commit f7f0266e526036dc1da3b22fd494a69d46f1d916, target branch master) failed with status FAILURE.

Failures are unrelated. LGTM!


Add hmmeral to Contributors list

@hmmeral, I'm quite elated about your PR. I wanna evolve this project to addresses various problems faced by first-time contributors. I'd love to learn about your journey in open source community, the problems, pain points you had etc.
Could you explain how you felt when you went through the tutorial, made a PR and learned that I merged it?


findIndex Method Problem

Hi,
We test the plugin with IE 11 and getting error about findIndex method in bulk-mover.js. I think IE 11 doesn't support the method.
Thank you.

@kurtiyil Thank you for your bug report. You are right, according to MDN, Internet Explorer doesn't support the findIndex.
Backports to ES5 compatible browsers are made with babel, but it seems that we only backport ES2015 features, instead of all of them (ES2015, ES2016, ES2017).
Solution
Replace npm package babel-preset-es2015 with babel-preset-env and update its usage in webpack.config.js

Sorry for the late reply, I struggled with the migration to babel-preset-env. I will try this as soon as possible again, hopefully it works.


Remove uuid dependency

Motivation
Fails on Python 3.7 and is not used in any check (just ddev and one test file)
Additional Notes
Last release was in 2006 b/c it became part of the standard library

@ofek found these two imports:
datadog_checks_dev/datadog_checks/dev/tooling/create.py:from uuid import uuid4
tokumx/datadog_checks/tokumx/vendor/bson/binary.py:from uuid import UUID

Ok seems that package was a 2006 thing and since then a uuid package has been included in the standard library?


fix(diff): warn about generated drop statements

What kind of change does this PR introduce?
Bug fix https://github.com/supabase/cli/issues/1721
What is the new behavior?
Print warnings when drop statements are generated.
Additional context
Add any other context or screenshots.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

13 of 20 (65.0%)  changed or added relevant lines in 1 file are covered.
5 unchanged lines in 1 file lost coverage.
Overall coverage decreased (-0.02%) to 58.362%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

internal/db/diff/migra.go
13
20
65.0%

Files with Coverage Reduction
New Missed Lines
%

internal/gen/keys/keys.go
5
11.76%

Totals

Change from base Build<PHONE_NUMBER>:
-0.02%

Covered Lines:
6006

Relevant Lines:
10291

💛  - Coveralls


Publishing profiles should not be stored in appsettings.json

Describe the bug
Publishing profiles contain unencrypted secrets, appsettings.json is normally checked into source control and should not contain sensitive information.
Version
Version:  1.4.0-nightly.237504.97571e0
Electron: 8.2.4
Chrome: 80.0.3987.165
NodeJS: 12.13.0
V8: 8.0.426.27-electron.0
To Reproduce
Steps to reproduce the behavior:

Create a publishing profile in composer
Open appsettings.json and see your bot password and other secrets there

Expected behavior
Publishing profiles should not be stored in appsettings.json
Additional context
appsettings.json is normally not in gitignore and it is used by developers to share common configuration settings (excluding secrets)

Known issue. The decision was to punt until we take on the KeyVault worksteam, something not funded in Cobalt.

Moving out of R13 and putting on project board in the R14 backlog to consider.

I'd like to just bring this to attention, as I was inundated with "problems with secrets in repositories" from our secrets police and a lot of them were that I just stored my bot composer projects as is.  We should rectify this, since if it's not good enough for our internal policies, it should not be good enough for our customers, either.

I'm wondering if this will be worked on soon?
In my company we have to get fancy on githooks and file parsing, just to make sure none of our sensitive (microsoft and others) credentials end up in github / bitbucket.
Normally all dev solutions bring some way to omit this, be it env vars, key vaults, or whatever - in an age where digital security is a real issue and threat, this is kinda must have I would say!


Where is the version for 4.2 ?

i have added require...... into my composer.json and when I run composer update I got follwoing message
Problem 1
- Conclusion: remove laravel/framework v4.2.17
- Conclusion: don't install laravel/framework v4.2.17
- Conclusion: don't install laravel/framework v4.2.16
- Conclusion: don't install laravel/framework v4.2.15
- Conclusion: don't install laravel/framework v4.2.14
- Conclusion: don't install laravel/framework v4.2.13
- Conclusion: don't install laravel/framework v4.2.12
- Conclusion: don't install laravel/framework v4.2.11
- Conclusion: don't install laravel/framework v4.2.10
- Conclusion: don't install laravel/framework v4.2.9
- Conclusion: don't install laravel/framework v4.2.8
- Conclusion: don't install laravel/framework v4.2.7
- Conclusion: don't install laravel/framework v4.2.6
- Conclusion: don't install laravel/framework v4.2.5
- Conclusion: don't install laravel/framework v4.2.4
- Conclusion: don't install laravel/framework v4.2.3
- Installation request for mattbrown/laracurl dev-master -> satisfiable by mattbrown/laracurl[dev-master].
- Conclusion: don't install laravel/framework v4.2.2
- Conclusion: don't install laravel/framework v4.2.1
- mattbrown/laracurl dev-master requires illuminate/support 5.* -> satisfiable by illuminate/support[v5.0.0, v5.0.22, v5.0.4].
- don't install illuminate/support v5.0.0|don't install laravel/framework v4.2.0
- don't install illuminate/support v5.0.22|don't install laravel/framework v4.2.0
- don't install illuminate/support v5.0.4|don't install laravel/framework v4.2.0
- Installation request for laravel/framework 4.2.* -> satisfiable by laravel/framework[v4.2.0, v4.2.1, v4.2.10, v4.2.11, v4.2.12, v4.2.13, v4.2.14, v4.2.15, v4.2.16, v4.2.17, v4.2.2, v4.2.3, v4.2.4, v4.2.5, v4.2.6, v4.2.7, v4.2.8, v4.2.9].

Hello,
I've updated the package tags.  For Laravel 4.2 version:
"mattbrown/laracurl": "1.0.0"


fix wrong uniform for loaded textures

If you are trying to submit to the v2 maintenance branch, please submit to v2.x. Otherwise submit to dev for v3 changes.


Link leads to empty page

Link: https://app2.test.dasch.swiss/resource/http%253A%252F%252Frdfh.ch%252F0803%252F5e77e98d2603
when I click (Links: Lateinische Übersetzung) I get an empty page, see screenshot.

Desktop (please complete the following information):

OS: Windows 10 Pro N, 1903 - 18362.418

Browser Google Chrome, 78.0.3904.87

Version Knora App, 1.6.2

Additional context
Add any other context about the problem here.

@kilchenmann this situation has happened due to missing data, hasn't it? should we keep this issue to handle the potential case (add warning message for instance)? or it will never happened on production?

@flavens Yes it's an issue in the data and has nothing to do the app design directly. But in this case, it would be probably nice to display something like "empty object" or "This resource doesn't contain any data (yet)". A kind of descriptive message. Additional we can list the users of the project or the institute to contact them. This would be a nice feature, but at the moment we don't have all necessary information in the project and I don't know about the privacy rules 😉

...but in the new resource viewer, we will link to the project, where all necessary information should be available. But a "empty resource" message would be nice.

@kilchenmann yes, at least a message should be displayed.
Regarding the listing of project members, only the project admin should be contacted. But this is another feature..

Yes, exactly:

this is another feature..

this issue has been moved to YouTrack: DSP-307


Plugin not working (but ACT is parsing)

The parsing is working and I change the DPS update interval to 3, I can't get any results in other windows other than Mini Parse, I'm using the plugin <IP_ADDRESS>

Can you take a screenshot of your overlay plugin configuration and share it here?

If this is still happening, please let me know.


Using same end-time in different positions

I define an ending time in AngularJS as follows:
  var today = new Date();
  $scope.endingTime = today.setHours(today.getHours() + 3);

and then I use this in HTML, the first usage is working perfectly:
 <timer style="color: red;" end-time="endingTime" max-time-unit="'hour'"> {{hours}} Std. {{minutes}} Min. {{seconds}} Sek. </timer>

But the second one does not work, the $scope.endingTime is just the same:
 Angebot gilt noch <timer style="color: red;" end-time="endingTimer" max-time-unit="'hour'"> {{hours}} Std. {{minutes}} Min. {{seconds}} Sek. </timer>

@tmundt In the second instance, you have endingTimer, not endingTime


add support for num_return_sequences to BLIP_Decoder.generate ?

Congrats on creating such an effective image captioner!  I'm using it in a project, and noticing that it's popping up in quite a few other places as well.
Inspired in part by this paper – https://arxiv.org/pdf/2205.10747.pdf – I was wondering if it would be possible to generate multiple candidate captions instead of just one?
It looks like it is possible, at least using the nucleus sampling method, but the value of num_return_sequences is hard-coded = 1 in the Blip_Decoder generate method – see here: https://github.com/salesforce/BLIP/blob/main/models/blip.py#L149
Would it be possible to add num_return_sequences to the generate method arguments and thus get multiple candidate captions?
Thank you!

Thanks for the suggestion. It is definitely possible to generate multiple sentences. We do not plan to change the code now in case it break other codes, but feel free to clone the code and change num_return_sequences for yourself.

Thank you!  It seems this only works for nucleus sampling method – is that right?  Thanks again.  :)

You can also return multiple sentences with beam search, please refer to the generate function description here: https://huggingface.co/docs/transformers/main/en/main_classes/text_generation


Initial pull request

Create objects to store data and get data from json file to fill objects.

You should try to avoid the global functions. Wrap them in the doc.ready


Can't update dashboard settings anymore

Since last changes related to #287, I can't update dashboard settings anymore... The form gets redirected from /dashboard/settings to /dashboard/settings/setup, and thus the POST request is gone, meaning it does not submit the form.

@willdurand can you try #302 for me too please.

This works under MySQL, so I'm inclined to think that it's related to PgSQL again. I'll have a look.

@willdurand I've deployed a dev version and I'm able to update settings.

Heroku btw.

Oh, hangon... I've tried to change a theme setting and now get an error.

mmh interesting..

The main /dashboard/settings/setup works for me.

Well. It does not work for me. Second time you're telling me that it works for you. Honestly, I am really glad for you BUT, I really have an issue, and that one is described as much as possible here.
POST'ing a form to update Application Setup gets redirected with a 302 and changes are not persisted. I don't get any feedback, and looking at the code, there are 5 possible redirections.

Commit deployed: 3e74d28ff4d7b71fd0fec72f4c3ee7311ce55cfa
Buildpack used is the recommended one
No errors in logs

@willdurand sorry, but I don't appreciate your tone (yes, I saw your tweet too). I'm investigating the issue as I type this (pushing to Heroku, plus I've just pushed another change which could help). The reason for me mentioning that it works is for me to dump my thoughts. It works in MySQL, so now I know that it's again a PgSQL issue.
I'll continue to update this issue as I go through it.

I'm sorry that there is an issue for you, I'm not singling you out, it's an issue that will be affecting everybody. I'm doing my best to get that resolved for you and everyone.

@willdurand please try upgrading to 3cbb500de7520c1af7f41d4d962cfe821bc1e732 - I've pushed to Heroku (https://cachet-dev.herokuapp.com) and that has allowed me to update the settings.

I don't really appreciate your tone either, so let's go ahead ;-)

Ok, upgrading..

@willdurand I've not been rude to you, I've done nothing but apologise and tried to fix the issues. I'm sorry if you think I'm being rude.
Let me know if it goes whacky.

@jbrooksuk it works like a charm, thank you.
Updating all settings now works again \o/

:boom:
Thanks for the bug report and helping me test this.


fix: Divide by zero should fail to satisfy constraints for Field and ints

Description
Problem*
Resolves #2149
Summary*
The issue was posted using nargo 0.9.0 and there has since been some changes, although the bug still exists. let a: Field = 3 / 0; still unexpectedly prints zero while let a: u8 = 3 / 0; actually now panics during instruction simplification for attempting to divide by zero. let a: i8 = 3 / 0; fails in the same cases as u8.
When the dividend comes from inputs to main, Field behaves the same way, while u8 panics inside of execution of the Brillig VM. The Brillig VM should return an error for division by zero rather than panic'ing but this is a separate issue.
The following changes were made:

For a field division, we have a check in ACIR gen for whether the brillig inverse is valid. However, we were missing this check for constants, this has now been included.
I have also introduced constraints to check whether we are dividing by zero in euclidean division. If we are dividing by zero, the execution of the program will fail to satisfy the constraints.
I had to remove a simplification for when we have an rhs of zero with a division binary op. Now, instead of a panic on let a: u8 = 3 / 0 we will get a constraint failure that points to the span 3 / 0.

All variants of division by zero (whether by constant or witness) now trigger a failed constraint.
Documentation

[ ] This PR requires documentation updates when merged.

[ ] I will submit a noir-lang/docs PR.

[ ] I will request for and support Dev Rel's help in documenting this PR.

Additional Context

PR Checklist*

[X] I have tested the changes locally.
[X] I have formatted the changes with Prettier and/or cargo fmt on default settings.

 If anything this will give them the correct result now no?


Use extract-zip-fork to fix unzip issues

The extract-zip module sometimes has an issue properly detecting zip file entries as directories. This commit switches to a forked version with a fix for that issue.

Why do they not accept the PR of the fork instead? I feel saver with having the official version as a dependency.

I have no idea, the issue is open since over half a year there. Also the fork was made for and is used in https://github.com/TryGhost/Ghost so it should be good for now. I also thought about kicking out extract-zip and use the underlying yauzl library directly but that seemed a little bit of an overkill.


Add a Gitter chat badge to README.md

uber/piranha now has a Chat Room on Gitter
@mkruber has just created a chat room. You can visit it here: https://gitter.im/uber/piranha.
This pull-request adds this badge to your README.md:

If my aim is a little off, please let me know.
Happy chatting.
PS: Click here if you would prefer not to receive automatic pull-requests from Gitter in future.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


partial translation of basic-javascript

[x] I have read freeCodeCamp's contribution guidelines.
[ ] My pull request has a descriptive title (not a vague title like Update index.md)
[ ] My pull request targets the master branch of freeCodeCamp.
[ ] All the files I changed are in the same world language, for example: only English changes, or only Chinese changes, etc.

Regarding #40480 , translation of basic-javascript is partially done

@S1ngS1ng Thank you for your contribution.

@S1ngS1ng Thank you for your contribution.


Updated DevC Hyderabad details -  DevCGlobalDirectory.md

Thanks @NavyaTatikonda for your commit! Can you please sign fb cla ? other than that LGTM 👍

Thanks @NavyaTatikonda for your commit! Can you please sign fb cla ? other than that LGTM 👍


Notes with quoted strings in their name are synced as if they are deleted

Abstract
Notes with quoted strings are synced as if they are deleted?
for example when making a note for someone with a nickname Dwayne "The Rock" Johnson
Expected behaviour

The note name updates accordingly instead of deleting the note

Actually happened

The note gets deleted on all devices that are on the receiving end of the sync

Reproducing procedure

Make a new note
See that said new note is synced with another device with the name: Untitled
Change the note's name to: Dwayne "The Rock" Johnson
See the note dissapear on the receiving device's end

I had not thought of actual OS limitations, This would type of title not work for me as I also sync with Windows.
I was using a Linux VM at the time which caused the characters to be accepted.
From a user perspective, instead of only preventing errors on the backend, it would be nice to be shown a popup with something along the lines of "File X contains problem character Y"
With a button below it to remove the character (could also be a setting to do this automatically), or to have the current non-sync situation and fix it manually.
This could also be extended by showing the platform that the character isn't accepted on, and an option to ignore that platform.
I don't use Mac for example and don't care about characters that only Mac cannot handle (if they exist).


Aetros not running on Windows Server

When starting the aetros cli it produces:
[0m aetros-server[3912] ERROR Connection error during connecting to trainer.aetros.com: preexec_fn is not supported on Windows platforms
Exception in thread Thread-3:
Traceback (most recent call last):
  File "c:\anaconda\lib\threading.py", line 801, in __bootstrap_inner
    self.run()
  File "c:\anaconda\lib\threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "c:\anaconda\lib\site-packages\aetros\backend.py", line 347, in thread_write
    if not self.connect():
  File "c:\anaconda\lib\site-packages\aetros\backend.py", line 210, in connect
    stdout=subprocess.PIPE, stdin=subprocess.PIPE, stderr=subprocess.PIPE)
  File "c:\anaconda\lib\subprocess.py", line 343, in __init__
    raise ValueError("preexec_fn is not supported on Windows "
ValueError: preexec_fn is not supported on Windows platforms

This is because Windows is currently not supported, but we're working on it. In the next release probably end of the week we release some fixes for windows issue like that.


Add preference pane for changing the layout

Description: If you start it as an app (or double click on the jar on windows), flags are not taken into consideration. This means whatever the default is will be used. Thus you can only use a different layout, when you start it from the command line. I suggest to use a preference pane for changing the layout.
Comment by @ThomasDaheim:
I have thought about that as well - is on my list of things still to add :-) Main point is that you then need to be able to switch layouts on the fly and also to store the info somewhere

I have thought about that as well - is on my list of things still to add :-) Main point is that you then need to be able to switch layouts on the fly and also to store the info somewhere

Not necessarily. You can always add a text that states "restart required".
Also, you already store the path of the notes. Once you have selected it via the GUI, the app always starts with that path, which means it has to be stored somewhere.

Have played around with this a bit. Unfortunately, you can't have a tooltip @ a menuitem in javafx - so I had to add a disabled menuitem to give the hint... (and yes, i have noticed that disabled menuitems are hard to read :-))

I don't understand your comment. What does it have to do with a preference pane/window?

For simplicity, I have added this as a menu and not a new pane - and I wanted to show "restart required" as tooltip

aha. got it. thank you.