da.from_array doesn't work as expected for scalar masked numpy array inputs

Hello,
I've noticed some unexpected behaviour when passing scalar masked numpy arrays to da.from_array, e.g. x = da.from_array(np.ma.array(9, mask=True)). When x is computed, we don't get the same numpy array input back, rather we get the np.ma.masked constant. This is problematic for because

np.ma.masked is a float, which may be a different dtype from the dask array
np.ma.masked  can not be sliced, unlike a numpy scalar array

Reproducer
import numpy as np
import dask as da
a = np.ma.array(9, mask=True)  # 0-d
x = da.from_array(a)

print('\nnumpy input =', repr(a))
print('\ndask =', repr(x))
print('\ndask dtype =', x.dtype)
print('\ndask computed =', repr(x.compute()))
print('\ndask computed dtype =', x.compute().dtype)

Gives output:
numpy input = masked_array(data=--,
             mask=True,
       fill_value=999999,
            dtype=int64)

dask = dask.array<array, shape=(), dtype=int64, chunksize=(), chunktype=numpy.MaskedArray>

dask dtype = int64

dask computed = masked

dask computed dtype = float64

Other size one masked arrays are not affected
import numpy as np
import dask as da
a = np.ma.array([9], mask=[True])  # 1-d
x = da.from_array(a)

print('\nnumpy input =', repr(a))
print('\ndask =', repr(x))
print('\ndask dtype =', x.dtype)
print('\ndask computed =', repr(x.compute()))
print('\ndask computed dtype =', x.compute().dtype)

Gives output:
numpy input = masked_array(data=[--],
             mask=[ True],
       fill_value=999999,
            dtype=int64)

dask = dask.array<array, shape=(1,), dtype=int64, chunksize=(1,), chunktype=numpy.MaskedArray>

dask dtype = int64

dask computed = masked_array(data=[--],
             mask=[ True],
       fill_value=999999,
            dtype=int64)

dask computed dtype = int64

Possible solution
This seems to occur because the scalar masked array input is sliced with () during it's passage through da.from_array:
>>> x = da.from_array(np.ma.array(9, mask=True))
>>> dict(x.dask)
{'original-array-27cd77571adb3b50d0abe2275510329d': masked_array(data=--,
              mask=True,
        fill_value=999999,
             dtype=int64),
 ('array-27cd77571adb3b50d0abe2275510329d',): (subgraph_callable-98ef2ccb-7037-4a4a-9d58-9417f3808eb6,
  'original-array-27cd77571adb3b50d0abe2275510329d',
  ())}

In fact, the 1-d masked case is also sliced, but with slice(0, 1) so that everything works as expected:
>>> x = da.from_array(np.ma.array([9], mask=[True])) # 1-d
>>> dict(x.dask)
{'original-array-5534b77f07bda580d7a7709dec8ce5af': masked_array(data=[--],
              mask=[ True],
        fill_value=999999,
             dtype=int64),
 ('array-5534b77f07bda580d7a7709dec8ce5af',
  0): (subgraph_callable-1334e14f-ad05-4fdb-9dcb-54a8c20644e4, 'original-array-5534b77f07bda580d7a7709dec8ce5af', (slice(0, 1, None),))}

I wonder if the reason behind this is that numpy masked arrays do not have type np.ndarray, rather they are np.ma.core.MaskedArray:
>>> b = np.ma.arange(9)
>>> type(b) is np.ndarray
False
>>> type(b) is np.ma.core.MaskedArray
True

and this means that from_array's is_ndarray variable (https://github.com/dask/dask/blob/2022.03.0/dask/array/core.py#L3293) is currently  False for these cases, rather than it being True so that we end up in the # No slicing needed clause (https://github.com/dask/dask/blob/2022.03.0/dask/array/core.py#L3306).
A solution could be to replace  is_ndarray = type(x) is np.ndarray with
    is_ndarray = type(x) in (np.ndarray, np.ma.core.MaskedArray)

I have tested this and it solves the use case I've described, and doesn't seem to make anything else fail.
If you think that this is all above board, I'm happy to make a PR with new unit tests.
Many thanks,
David

@davidhassell Thank you for opening this! @jcrist @jsignell what do you think about the proposed solution?


adding squash operation + tests

Initial implementation for review.
@porcelli I didn't want to go deeper in the testing before checking the initial approach.

@Salaboy IMO, we should not create PR's just for initial implementations.
That only creates noise and you can have this conversation sending a PR to @porcelli  forks.

I agree with what @ederign. We should only create PRs against "blessed" repositories where the code is ready for review and merge.

I also agree with both Eden and Petr.

@ederign @psiroky @manstis I got the feeling that the issue was the word "initial" because the operation is complete and it is working. Do you see any other issue with the content of the PR? I think that the code is ready for review and merge, I was looking for that kind of feedback. I will avoid the word "initial" in my future PRs.

@Salaboy Yes, you are right. For me the word "initial" looked like something that is not yet ready.
Also when mentioning "I didn't want to go deeper in the testing...", does that mean more tests are on the way?

@psiroky they will come in the future if they are needed, now the test provided check that the behaviour is the expected one

@porcelli is this one good to go?

@csadilek still on low priority, after BZs i'll be back to it

Can one of the admins verify this PR? Comment with 'ok to test' to start the build.

@porcelli do you still need this PR? I would like to keep this list clean! :)

Yes, keep it there for now please
Sent from my iPhone

On Jun 29, 2016, at 5:31 PM, Eder Ignatowicz<EMAIL_ADDRESS>wrote:
@porcelli do you still need this PR? I would like to keep this list clean! :)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

@adrielparedes is working on it, and will submit a new PR soon. finally we can close this ;)


Avast says maybe risk in 2.7.1

I download version 2.7.1
Avast says maybe risk, and ScreenToGif crash when I open again.
But then I try 2.7
Everything is ok.

Maybe because v2.7.1 is a recently created binary. Try adding it to the avast exception list.

Did it work out?

Hi, I didn't try the 2.7.1
Because the 2.7 is all ok, so I used the version 2.7.
Thank for create this amazing app ^_^

Oh, ok. Thank you. :)


How to hide Value field from Top bar ?

This is not an issue but some help needed...
My source csv is simple, only 3 columns:-
Country, Product, Revenue.
Revenue is value field and we are interested in sum of it so the default is
Integer Sum > Revenue
The PivotUI is displaying nicely but at the same time the Revenue button also appear on top horizontal bar.
Some user drag this Revenue to side bar causing it crash on use.
Is it possible to hide the Revenue button on top ?
I tried hiddenAttributes: ["Revenue"] it then disappear from top bar but at the same time it disappears from selection box beneath Integer Sum.
How to only hide it from top horizontal bar ? Thanks

There is no way to do this at the moment, although there is an open pull request which I may merge that does this: #408

I've just merged the changes from PR #408 so you can now use hiddenFromDragDrop to accomplish what you're looking for, or hiddenFromAggregators for the opposite.

Yes this is great. Thanks... making it even more perfect now.

Great. Please close this issue if you don't need any further help.
On Sun, Sep 24, 2017 at 07:57 hkbeer<EMAIL_ADDRESS>wrote:

Yes this is great. Thanks... making it even more perfect now.
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/nicolaskruchten/pivottable/issues/778#issuecomment-331705223,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAMbA-lta2eMx1nn4HUmlEtvK77Oixzoks5slkO6gaJpZM4PaYLO
.


HV: remove tinycrypt

Remove tinycrypt files:
Currently, hkdf_sha256 (based on Tinycrypt) is used by Trusty seed
derivation. And we plan to remove Tinycrypt from ACRN hypervisor
now, and replace it with another crypto lib in the future.
Signed-off-by: Qi Yadong<EMAIL_ADDRESS>

PR to upstream: https://github.com/projectacrn/acrn-hypervisor/pull/221
close this one.


Wrong link syntax

There you go, These were way easier to find when browsing http://php-http.readthedocs.org/en/latest/

Good catch.


Improvement in UI of PowerToy Run

Description of the new feature / enhancement

The first search box is visually better than the current PowerToy run's UI.
Scenario when this would be used?
Eyecandy UI will make application more user friendly
Supporting information
First UI is packed with latest version of MS edge. Press Ctrl + Q to see the dialog.

/dup #12182


ML : Restaurant Revenue Prediction

Description
📌 Issues for Week 9 - Basic Projects of Machine Learning
🔴 Project Title : Restaurant Revenue Prediction
🔴 Aim of the Project : Machine learning model to predict the revenue of restaurant.
🔴 Dataset : https://www.kaggle.com/c/restaurant-revenue-prediction/data
🔴 Domain : Machine Learning
🔴 Special Credit : Try to implement more than three machine learning algorithms and make a comparison between them, then conclude about the best fit algorithm to create the model. All the algorithms must be checked through the accuracy scores to determine the best fitted model
Welcome to 'ML' Team, good to see you here, in this week we'll be creating several basic projects of this Machine Learning Domain.

📍 Follow the Guidelines to Contribute in the Project :

You need to create a separate folder named as the Project Title.
Inside that folder, there will be four main components.

Images - To store the required images.
Dataset - To store the dataset or, information/source about the dataset.
Model - To store the machine learning model you've created using the dataset.
requirements.txt - This file will contain the required packages/libraries to run the project in other machines.

Inside the Model folder, the README.md file must be filled up properly, with proper visualizations and conclusions.
A Demo Project has been created to guide you about the whole structure of presenting the Project in this repository. Here's the link of the Demo Project - https://github.com/girlscript/winter-of-contributing/tree/Machine_Learning/Machine_Learning/Basic_Projects_of_Machine_Learning/Demo_Project

Take the Reference from these two projects attached here,

https://github.com/prathimacode-hub/ML-ProjectKart/tree/main/Airbnb Price Prediction
https://github.com/prathimacode-hub/ML-ProjectKart/tree/main/Bike Crash Analysis

:red_circle::yellow_circle: Points to Note :

The issues will be assigned on a first come first serve basis, 1 Issue == 1 PR.
"Issue Title" and "PR Title should be the same. Include issue number along with it.
Changes should be made inside the Machine Learning/Machine_Learning/Basic_Projects_of_Machine_Learning Branch.
Follow Contributing Guidelines & Code of Conduct before start Contributing.
This issue is only for 'GWOC' contributors of 'Machine Learning' domain.

:white_check_mark: To be Mentioned while taking the issue :

Full name :
Batch Number :
GitHub Profile Link :
Approach for this Project :
Are you a participant of GWoC21?

[ ] YES
[ ]  No

Happy Contributing 🚀
All the best. Enjoy your open source journey ahead. 😎
Domain
Machine Learning
Type of Contribution
Others
Code of Conduct

[X] I follow Contributing Guidelines & Code of conduct of this project.

Hello, I am Deepthi M with serial number:198, Batch-2. I would like to do documentation on this issue.my approach: a detailed explanation

@deepthi1107 Assigned to you, Create the PR in 3 days.

Full name : Deepthi M
Batch Number : 02
GitHub Profile Link : https://github.com/deepthi1107
Approach for this Project : EDA, SVM, logistic regression, knn
Are you a participant of GWoC21?
YES

@deepthi1107 Progress?

By tomorrow I will merge it mam, its in the final process mam.


[nRF52840] fix alarm problem on timer overflow

Millisecond and microsecond timers used by OpenThread can overflow during
application run. This commit sets RTC Compare Channel to correct value
after overflow.

We've been seeing this problem in our environment, too, and this PR apparently fixes it.


Integration with Github's Pull Request Templates

Though #1187 already offers PR template functionality, I was surprised to see that hub doesn't automatically look for templates as specified in Creating a pull request template for your repository.
Edit: Sorry, the code seems to suggest that this is supported. Am I missing something? I'm using Hub 2.2.9 on OS X 10.12.6, and I have a .github directory with a file called PULL_REQUEST_TEMPLATE.md, but it doesn't get picked up when I run hub pull-request.

This hasn't been yet released as a stable release, but you can pick up one of the prereleases from the Releases page, or use Homebrew: brew upgrade hub --devel.

My bad. Thanks for the prompt response.
– P
On Mon, Sep 25, 2017 at 5:50 PM, Mislav Marohnić<EMAIL_ADDRESS>wrote:

Closed #1577 https://github.com/github/hub/issues/1577.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/github/hub/issues/1577#event-1263257297, or mute the
thread
https://github.com/notifications/unsubscribe-auth/AA116BOzn17XeOti1f4vZ8xx8aJJm8Jpks5sl2lsgaJpZM4PiNsI
.


Add snippets for julia and delete pi and phi

Removing phi and pi as it can cause conflicts and problems in julia and latex modes (also in org-mode) and other programming modes, see https://github.com/hlissner/doom-snippets/issues/39.
Also add snippets for julia-mode, some snippets where taken from https://github.com/AndreaCrotti/yasnippet-snippets, I left the contributors name in these snippet files.

implemented the requested changes. Before you merge could you tell me what this implicit selection does? I did not find it https://joaotavora.github.io/yasnippet/snippet-development.html nor by a quick search in doom emacs.
Also I am not completely sure how YASnippet works so would it be better to insert blocks such that $0 is after the block or like how I did it right now?

Quoting the relevant section of yasnippet's docs

From version 0.6, snippets expansions are run with some special Emacs-lisp variables bound. One of this is yas-selected-text. You can therefore define a snippet like:
for ($1;$2;$3) {
yas-selected-text$0
}
to "wrap" the selected region inside your recently inserted snippet. Alternatively, you can also customize the variable yas-wrap-around-region to t which will do this automatically.

This snippets library aliases yas-selected-text to % here for convenience.
Hope that helps!

Also: there are two unresolved comments in the last review. See above.

Ups, forgot to push this change and thanks for the reference.

No problem! Thanks for the help!


Switch flyteidl output to be commonjs

flyteidl is currently being output as an es6 module, which makes it incompatible with NodeJS unless it is run through webpack first. There's no real reason to do it that way, and protobufjs supports commonjs output, so we should switch to that.

Bug bankruptcy.

Bug bankruptcy.


the meaning of neighborhood_limits

Hello, HuguesTHOMAS.
Thanks for your amazing work! In every calibration function of all the sampler classes, you remarked that：
Neighbors calibration: Set the "neighborhood_limits" (the maximum number of neighbors allowed in convolutions)
so that 90% of the neighborhoods remain untouched. There is a limit for each layer.
I'dont understand why some parts of the neighborhood points should remain untouched. And I want to use the KPConv module in my experiment and try to build a simpler dataloader, I'm wandering that whether it necessary to add a calibration function to gain the batch_limit and neighborhood_limits or I could just set the batch_num and neighborhood_num fixed with fixing number of downsampled points.

You definitely can. The calibration function is here to simplify the parameter tuning by choosing an appropriate value depending on the point density. But you can set it yourself absolutely.

Thanks for your reply! One more thing, could you explain why the calibration function needs the 90% of the neighborhoods remain untouched ? I don't understand.

It does not "need" it. It is just a value you can decide for yourself. In my experiments, I noticed this was a good value as it already reduced the computations by a lot, while not losing any performance

Thanks for your reply！


CI: Setup dev workflow

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:
:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.


Use bluebird promise implementation with async/await

I use bluebird a lot in my project. It has benefits over standard Promise implementation, such as better api and performance. Alwo bluebird's promises fully backward compatible with standard Promise interface, so, it can be used as replacement of standard Promise.
But, trying to use async / await functions with Bluebird I got an error about impossibility to use Bluebird as return of async function. Consider the example:
async function readFile(fileName: string): Bluebird<string> {
}

As for now I should explicitly write something like that:
function  readFile(fileName: string): Bluebird<string> {
  return Bluebird.try(async () => {

  })
}

This extra wrapper makes me sad :( While I can force to use bluebird's promise implementation by overloading global.Promise or hoook my transpiler (babel or somewhat else), I can't use it with typescript :(

Works for me. You're doing something wrong.

@Strate Are you using "target": "es5", or "target": "es2015"
In the former case, this works fine:
import Bluebird from 'bluebird';

export async function getString(): Bluebird<string> {
  return await 'hello';
}

export async function useString(): Bluebird<void> {
  console.log(await getString());
}

In the latter case, the above code will issue an error stating

The return type of an async function or method must be the global Promise type.

This is required by the ECMAScript spec which mandates that async function return instances of the global Promise function. You can get around this with by writing the following
// bootstrap.ts
import Bluebird from 'bluebird';
Promise = Promise || Bluebird as any;

// example.ts
export async function getString() {
    return await 'hello';
}

export async function useString() {
    console.log(await getString());
}

But your code will not be spec compliant if you are targeting a runtime that provides Promises.
This has nothing specifically to do with TypeScript.

Thanks @aluanhaddad for patient explanation! Learn a lot from your comment.
I peruse the TC39's proposal, specifically in the section abstract-async-function-start, where the return behavior is specified, it reads:
If result.[[type]] is normal, then
 　　Perform ! Call(promiseCapability.[[Resolve]], undefined, «undefined»).

After another lengthy search, the promiseCapability.[[Resolve]] is promise's native resolve, which is defined here
8. Let then be Get(resolution, "then").
9. If then is an abrupt completion, then

    a. Return RejectPromise(promise, then.[[value]]).

10. Let thenAction be then.[[value]].
11. If IsCallable(thenAction) is false, then

    b. Return FulfillPromise(promise, resolution).

12. Perform EnqueueJob ("PromiseJobs", PromiseResolveThenableJob, «promise, resolution, thenAction»)

If I read the spec correctly, it means every async function will return a native promise. But in the async body, every promise compatible object can be awaited or returned.

Thanks @aluanhaddad for patient explanation! Learn a lot from your comment.

I am pretty sure I learned it from someone else, perhaps a TypeScript team member, but I forget. Regardless, you are most welcome 😉

If I read the spec correctly, it means every async function will return a native promise. But in the async body, every promise compatible object can be awaited or returned.

I believe you are reading that correctly. It does seem to suggest that it is indeed only top level async bodies which are restricted to returning instances of the global Promise type.
The behavior of TypeScript's implementations matches your reading.
With "target": es2015"
TypeScript
import Bluebird from 'bluebird';

function getValue() { // The return type is inferred as Bluebird<number>.
  return Bluebird.resolve(4);
}
async function getValue() { // The return type is inferred as Promise<number>.
  return await Bluebird.resolve(4);
}
async function getValue1() { // The return type is also inferred as Promise<number>.
  return Bluebird.resolve(4);
}

Interestingly with "target": "es5" and "lib": "[ "es2015" ]" we are allowed to mix and match with async functions returning the result type of propagating their awaitable but only when explicitly specified.
TypeScript
import Bluebird from 'bluebird';

// The return type annotation changes the emit. This doesn't happen anywhere else in TypeScript.
// If we do not have a `Promise` declaration in scope, like if we set `"lib": [ "es5" ]`,
// we will get a compiler error unless we specify the return type explicitly.
async function getValue(): Bluebird<number> { // The return type is inferred as Promise<number>.
    return await Bluebird.resolve(4);
}

ES5.1
import Bluebird from 'bluebird';

function getValue() {
    return __awaiter(this, void 0, Bluebird, function () {
        return __generator(this, function (_a) {
            switch (_a.label) {
                case 0: return [4 /*yield*/, Bluebird.resolve(4)];
                case 1: return [2 /*return*/, _a.sent()];
            }
        });
    });
}

I think I ran into this too - or possibly a related issue - because I wanted to use mocha-typescript and to have my test suite import my app code, while having my test project be its own typescript project.
This prompted me to add declaration: true to my src project's tsconfig.json so my test project can consume/reference it.
And my main application's entry point exports an async function as its default export like this:
import * as Promise from 'bluebird';
/* ... all imports and stuff... */
import * as feathers from 'feathers';

const feathersApp = feathers();

const configApp = async (app: feathers.Application) => {
  /*... await some stuff.... await more stuff... */
  return app;
}

export default configApp(feathersApp); /* ERR with bluebird as Promise (w/ declarations) */

Getting error - which I suppose I could sidestep with the aforementioned wrapper? Or possible a declaration...?
declare var Promise: bluebird<any>;

with import * as bluebird from 'bluebird' instead of importing bluebird as promise?
Version deets
node.js: 8.0.0
npm: 5.0.2
tsc: 2.3.4
VSCode: 1.13.0
Both tsconfig targets es2017.

Seems like most people got this sorted out and native Promise implementations are everywhere now, so I don't think there's any further action warranted from our side. Thanks everyone for being a helping community!


add recode flag to V/BCF importer

Following the design of PGDSpider2 output, the V/BCF importer should have an optional flag recode::Bool = false (name pending) to rename all loci a simple generic name like SNP_1...SNP_n.

Should be added with https://github.com/pdimens/PopGen.jl/commit/69aae3e83ea91167edc586f82fc5e35c3d9282bc


C++11 inline initializers

Description
Can't initialize static const arrays inside structs
Steps to Reproduce
https://godbolt.org/z/eqzY39oov
template <int Order>
struct GaussLegendreValues
{};

template <>
struct GaussLegendreValues<2>
{
    const static double wi[2] = { 1.0, 1.0 };
    const static double xi[2] = { -0.5773502691896257, 0.5773502691896257 };
};

Actual Behavior
The snippet above won't compile

static data member of type 'const double [2]' must be initialized out of line

Environment
Whatever godbolt DXC(trunk) compiler runs on.

this is quite funny because when its not an array, everything is fine.

That's because HLSL unintentionally inherited a GNU C++ extension that allowed inline specification of floating point and integral constants. That extension did not support arrays (or any other type requiring an initialization list).
C++11 adopted the GNU extension and extended it to support for types defined by constexpr expressions (initializer lists or constexpr functions).
@pow2clk reopened this so that I could further investigate, but I think we made the right call in the first place that this isn't a DXC bug. Adding this correctly to the language relies on constexpr (which we are tracking in #74). We don't have a separate request for C++11 inline initializers, so I'm going to move this over to hlsl-specs to track a feature request for that.

just make sure https://github.com/microsoft/DirectXShaderCompiler/issues/5823 doesn't get closed, as we'd like an alternative to this.

This is starting to become a blocker because we've added the equivalent of std::complex<ScalarT> to our HLSL STL and now we have a problem specializing our std::plus counterparts in functional.hlsl
(we're working on HLSL FFT workgroup and subgroup utils)
I need an identity static const member, so we try to specialize it
template<typename Scalar>
struct multiplies<complex_t<Scalar> >
{
...

   const static complex_t<Scalar> identity = {Scalar(1),Scalar(0)};
};

Now obviously I get the error: static data member of type 'const complex_t<Scalar>' must be initialized out of line
And the only way is to do out-of-line initialization but that can't be partially specialized.
So you literally have to explicitly specialize every combination of float16_t, through to float64_t4
template<>
const static complex_t<float32_t> multiplies<complex_t<float32_t> >::identity = {1.f,0.f};


Refactor names of send and receive related types

After the refactoring of #158 some type and file names are a little bit confusing. I tried to improve the situation (hopefully for the last time):
Here the new structure

client.rs contains ClientFlow
client_receive.rs contains ClientReceiveState
client_send.rs contains ClientSendState
server.rs contains ServerFlow
server_receive.rs contains ServerReceiveState
server_send.rs contains ServerSendState
receive.rs contains ReceiveState which is used by both ClientReceiveState and ServerReceiveState

Also I tried to replace the specific terms command and response with the less specific term message in many places because the more specific term was sometimes not correct anymore (e.g. ServerSendState handles both Response and Greeting, so the term message is more appropriate than response).
I split this refactoring in many small commits, so it should be easy to understand the changes.
No public API has changed.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

128 of 205 (62.44%)  changed or added relevant lines in 6 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.008%) to 52.67%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/server_send.rs
66
67
98.51%

src/client_receive.rs
8
10
80.0%

src/server.rs
7
11
63.64%

src/client.rs
7
16
43.75%

src/server_receive.rs
9
34
26.47%

src/client_send.rs
31
67
46.27%

Totals

Change from base Build<PHONE_NUMBER>:
0.008%

Covered Lines:
730

Relevant Lines:
1386

💛  - Coveralls


Add approx traits

It would be nice if the sparse matrices support the traits from approx, which allows checking whether two matrices containing floats are approximately equal. This to aid in testing of code, both internally in this crate and outside.

That would be nice indeed. I guess there would first be a structure check, and then a check to see if the norm of the difference between the data vectors is small enough.


Why is there no intrinsics for Type-Convert from 'vfloat32m1_t' to 'vfloat16mf2_t' in Narrowing Floating-Point Type-Convert intrinsics?

I find that there is no Narrowing Floating-Point Type-Convert intrinsics for from 'vfloat32m1_t' to 'vfloat16mf2_t' in Intrinsic Functions List
Like this:
vfloat16mf2_t vfncvt_f_f_w_f16mf2 (vfloat32m1_t src);
Is it missing? Or is there no such intrinsic for some reason?

We have :

'vfloat32mf2_t' to 'vfloat16mf4_t'
'vfloat32m2_t' to 'vfloat16m1_t'
'vfloat32m4_t' to 'vfloat16m2_t'
'vfloat32m8_t' to 'vfloat16m4_t'

but there is no 'vfloat32m1_t' to 'vfloat16mf2_t'  and 'vfloat32mf4_t' to 'vfloat16mf8_t'  so I believe it's mistake :P Thanks your report!

Thanks for reply.


test story

Howdy! I was trying to have a world with only one location, and it didn't go so well, so I started working on this bit.
These tests don't pass and are gibberish, so I'll be updating this PR as I go, but for now, questions:

would it be alright to setup Travis on this repository so that the test status is displayed on PRs? It's free for open source and I'm happy to add the travis.yml
is it alright with you that I keep breaking out methods like this for individually testing?
is it alright if I start using jsdoc to add inline documentation?
more questions in with the code 😄

As for your questions above, Travis is a great idea, and I don't object to breaking out functions for testing. Some things I have been thinking about are the random story generation. I don't think just randomly generating a story every time is the best thing. Some rules are really interesting but they are so specific that the chances of them happening are really low.
To improve this situation I think I will add a feature where you set certain specific events to happen at certain time steps. As I mentioned above, it is also probably a good idea to let things interact with their locations instead of just moving between them. That would fix your single location problem. But right now locations are assumed to be added for the purpose of cause "transitions" which is no good. Finally, the question of how to run story-graph with these various features turned on or off is another question. Let me go make some tickets :-)

@incrediblesound Thanks so much for the feedback 😄 I like the way you're thinking about new features and was having similar ideas.
I'll update soon with adjusted codes 👍

I think this is ready for review! Feels like the tests could be DRY'd out a bit, but it's not coming to me at the moment, so...

Awesome! I'll check it out soon.

Good stuff!


Validate if message is set in AccountAuthView

Hi,
We're using the messages framework throughout our oscar application however the "welcome back" notice when a user logs in isn't one we would like to display.
Is it possible to have the view function check if a message is set before actually setting the message this way we can use the get_login_success_message to return False.
https://github.com/django-oscar/django-oscar/blob/master/src/oscar/apps/customer/views.py#L161

Closing since related PR already merged.


Locking is slow (and performs redundant downloads)

Is this an issue with my installation?  It happens on all of my machines...  Is there anything I/we can do to speed it up?
I install one package and the locking seems to take minutes.
Locking [packages] dependencies…

$ python -m pipenv.help output
Pipenv version: '2018.05.18'
Pipenv location: '/Users/colllin/miniconda3/lib/python3.6/site-packages/pipenv'
Python location: '/Users/colllin/miniconda3/bin/python'
Other Python installations in PATH:

2.7: /usr/bin/python2.7

2.7: /usr/bin/python2.7

3.6: /Library/Frameworks/Python.framework/Versions/3.6/bin/python3.6m

3.6: /Library/Frameworks/Python.framework/Versions/3.6/bin/python3.6

3.6: /Users/colllin/miniconda3/bin/python3.6

3.6: /Users/colllin/.pyenv/shims/python3.6

3.6: /usr/local/bin/python3.6

3.6.3: /Users/colllin/miniconda3/bin/python

3.6.3: /Users/colllin/.pyenv/shims/python

2.7.10: /usr/bin/python

3.6.4: /Library/Frameworks/Python.framework/Versions/3.6/bin/python3

3.6.3: /Users/colllin/miniconda3/bin/python3

3.6.4: /Users/colllin/.pyenv/shims/python3

3.6.4: /usr/local/bin/python3

PEP 508 Information:
{'implementation_name': 'cpython',
 'implementation_version': '3.6.3',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '17.5.0',
 'platform_system': 'Darwin',
 'platform_version': 'Darwin Kernel Version 17.5.0: Mon Mar  5 22:24:32 PST '
                     '2018; root:xnu-4570.51.1~1/RELEASE_X86_64',
 'python_full_version': '3.6.3',
 'python_version': '3.6',
 'sys_platform': 'darwin'}

System environment variables:

TERM_PROGRAM
NVM_CD_FLAGS
TERM
SHELL
TMPDIR
Apple_PubSub_Socket_Render
TERM_PROGRAM_VERSION
TERM_SESSION_ID
NVM_DIR
USER
SSH_AUTH_SOCK
PYENV_VIRTUALENV_INIT
PATH
PWD
LANG
XPC_FLAGS
PS1
XPC_SERVICE_NAME
PYENV_SHELL
HOME
SHLVL
DRAM_ROOT
LOGNAME
NVM_BIN
SECURITYSESSIONID
_
__CF_USER_TEXT_ENCODING
PYTHONDONTWRITEBYTECODE
PIP_PYTHON_PATH

Pipenv–specific environment variables:
Debug–specific environment variables:

PATH: /Library/Frameworks/Python.framework/Versions/3.6/bin:/Users/colllin/miniconda3/bin:/Users/colllin/.pyenv/plugins/pyenv-virtualenv/shims:/Users/colllin/.pyenv/shims:/Users/colllin/.pyenv/bin:/Users/colllin/.nvm/versions/node/v8.1.0/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
SHELL: /bin/bash
LANG: en_US.UTF-8
PWD: /Users/.../folder

Contents of Pipfile ('/Users/.../Pipfile'):
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
gym-retro = "*"

[dev-packages]

[requires]
python_version = "3.6"

@colllin Have you checked to see whether pip commands that contact the server - like pip search (I think) - are also slow?
I see similar behavior, but it's kind of a known issue and network dependent. For some reason, access to pypi.org from my work network is incredibly slow but it is normally fast from my home network. I think locking does a lot of pip transactions under the hood, so slow access to the server slows the operation a lot.

Thank you for the thoughtful response.
pip search isn't especially fast or slow for me... ~1 second?
Forgive me for my lack of domain knowledge: Does it really need to pip search?  Didn't it just install everything?  Doesn't it just need to write down what is already installed?  Or... since it ensures the existence of the lock file anyway, could it do this as it installs the packages, or before?
I'm guessing... pipenv uses pip under the hood? so the installation process is a black box, and it can't know the dependency graph of what was/will be installed without doing a pip search?

I'm not a contributor to the project and at the moment I don't know all the specifics, but my understanding is that the locking phase is where all of the dependencies get resolved and pinned. So if you have one top-level package with ~65 dependencies, it's during the locking phase that all of the dependencies of that first package are (recursively) discovered, and then the dependency tree is used to resolve which packages need to be installed and (probably) in what rough order they should be installed in. Not as sure about the last part.
If you pip install from a Pipfile without a lockfile present, you'll notice that it does the locking phase before installing the packages into the venv. Similarly if you have a lockfile but it's out of date. I suspect having a lockfile and installing using the --deploy option would be faster, as would the --no-lock option; in the former case you get an error if the lockfile is out of date, in the latter you lose the logical splitting of top-level packages (declared environment) and the actual installed (locked) environment of all packages. At least this is how I understand it.
Whether or not pipenv uses pip under the hood - I think it does - it still needs to get the information from the pypi server(s) about package dependencies and the like, so my question about pip search was more a proxy for how fast or slow your path to the pypi server is than a direct implication about the mechanism by which pipenv does its thing.
An interesting experiment might be to compare the time required for locking the dependency tree in pipenv, and installing requirements into a new venv using pip install -r requirements.txt. I think they should be doing pretty similar things during the dependency resolution phase.

Have we established somewhere that there are redundant downloads happening btw? I suspect that is the case but proving it would be really helpful
FYI comparing pip install -r requirements.txt to the time it takes to lock a dependency graph is not going to be informative as a point of comparison.  Pip doesn't actually have a resolver, not in any real sense.  I think I can describe the difference.  When pip installs your requirements.txt, it follows this basic process:

Find the first requirement listed

Find all of its dependencies
Install them all

Find the second requirement listed

Find all of its dependencies
Install them all

Find the third requirement listed

Find all of its dependencies
Install them all

This turns out to be pretty quick because pip doesn't really care if the dependencies of package 1 conflicted with the dependencies of package 3, it just installed the ones in package 3 last so that's what you get.
Pipenv follows a different process -- we compute a resolution graph that attempts to satisfy all of the dependencies you specify, before we build your environment. That means we have to start downloading, comparing, and often times even building packages to determine what your environment should ultimately look like, all before we've even begun the actual process of installing it (there are a lot of blog posts on why this is the case in python so I won't go into it more here).
Each step of that resolution process is made more computationally expensive by requiring hashes, which is a best practice.  We hash incoming packages after we receive them, then we compare them to the hashes that PyPI told us we should expect, and we store those hashes in the lockfile so that in the future, people who want to build an identical environment can do so with the contractual guarantee that the packages they build from are the same ones you originally used.
Pip search is a poor benchmark for any of this, in fact any of pip's tooling is a poor benchmark for doing this work -- we use pip for each piece of it, but putting it together in concert and across many dependencies to form and manage environments and graphs is where the value of pipenv is added.
One point of clarification -- once you resolve the full dependency graph, installation order shouldn't matter anymore.   Under the hood we actually pass --no-deps to every installation anyway.
As a small side-note, pip search is currently the only piece of pip's tooling that relies on the now deprecated XMLRPC interface, which is uncacheable and very slow.  It will always be slower than any other operation.

Locking numpy (and nothing else) takes 220 s on my machine (see below). Most of the time seems to be spent downloading more than 200MB of data, which is quite puzzling given that the whole numpy source has 4 MB. Though clearly even if that was instant, there's still 25 s of actual processing, and even that seems excessive to calculate a few hashes.
11:46 ~/Co/Ce/torchdft time pipenv install
Creating a virtualenv for this project…
Using /usr/local/Cellar/pipenv/2018.5.18/libexec/bin/python3.6 (3.6.5) to create virtualenv…
⠋Already using interpreter /usr/local/Cellar/pipenv/2018.5.18/libexec/bin/python3.6
Using real prefix '/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6'
New python executable in /Users/hermann/.local/share/virtualenvs/torchdft-mABBUp_t/bin/python3.6
Also creating executable in /Users/hermann/.local/share/virtualenvs/torchdft-mABBUp_t/bin/python
Installing setuptools, pip, wheel...done.

Virtualenv location: /Users/hermann/.local/share/virtualenvs/torchdft-mABBUp_t
Creating a Pipfile for this project…
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Updated Pipfile.lock (ca72e7)!
Installing dependencies from Pipfile.lock (ca72e7)…
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 0/0 — 00:00:00
To activate this project's virtualenv, run the following:
 $ pipenv shell
        7.81 real         6.39 user         1.64 sys
11:46 ~/Co/Ce/torchdft time pipenv install numpy --skip-lock
Installing numpy…
Collecting numpy
  Using cached https://files.pythonhosted.org/packages/f6/cd/b2c50b5190b66c711c23ef23c41d450297eb5a54d2033f8dcb3b8b13ac85/numpy-1.14.5-cp36-cp36m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
Installing collected packages: numpy
Successfully installed numpy-1.14.5

Adding numpy to Pipfile's [packages]…
        4.97 real         2.88 user         1.81 sys
11:46 ~/Co/Ce/torchdft time pipenv lock --verbose
Locking [dev-packages] dependencies…
Using pip: -i https://pypi.org/simple

                          ROUND 1
Current constraints:

Finding the best candidates:

Finding secondary dependencies:
------------------------------------------------------------
Result of round 1: stable, done

Locking [packages] dependencies…
Using pip: -i https://pypi.org/simple

                          ROUND 1
Current constraints:
  numpy

Finding the best candidates:
  found candidate numpy==1.14.5 (constraint was <any>)

Finding secondary dependencies:
  numpy==1.14.5 not in cache, need to check index
  numpy==1.14.5             requires -
------------------------------------------------------------
Result of round 1: stable, done

Updated Pipfile.lock (4fccdf)!
      219.24 real        25.14 user         5.77 sys

Numpy should be substantially faster now (I have been using your example as a test case in fact!).  As of my most recent test, I had it at ~30s on a cold cache on a vm.
Can you confirm any improvements with the latest release?

It has improved substantially for me as well. I'm now sitting on a very fast connection, and got as low as 14 s, but that was when the downloading went at 30 MB/s. What is being download besides a single copy of the source code of numpy?

I think we’re downloading redundant wheels (not sure). We’re already evaluating the situation.

I changed my Pipfile.lock drastically by uninstalling a fe changes and now deploying that on a different machine is freezing. Any particular fix for this?

It’s not recommended that you manually edit your lockfile. Without more information it’s not possible to help. Please open s separate issue.

If you want to benchmark the performance of pipenv lock, you should try to add pytmx to your dependencies...
pipenv lock used to take 1 hour or more for us (we have a pretty slow internet), and after removing pytmx, we got down to about 5 minutes and finally pipenv is more usable.
I know pytmx is a large package because it's a big monolithic lib and depends on opengl/pygame and other related things, but it shouldn't take 1 hour to pipenv lock no matter how big the package

It doesn’t take one hour for me
$ cat Pipfile
[packages]
pytmx = "*"

$ time pipenv lock --clear
Locking [dev-packages] dependencies...
Locking [packages] dependencies...
Updated Pipfile.lock (eb50ab)!

real	0m2.827s
user	0m2.287s
sys	0m0.390s

you're right it's smaller than I thought it does not depend explicitly on pygame and such, not sure why it was taking so long then !
I will try to find more information but i have a top CPU and SSD so I still think the issue is related to our slow internet

@techalchemy I didn't edit the file manually. I uninstalled a lot of dependencies using pipenv uninstall package_name and afterwards ran it on the server. It stayed in the lock state for a very long time.

I am not interested in spending energy on this discussion with random shots in the dark.  Please provide a reproducible test case.

Here's what I hope is a reproducible test case
https://github.com/Mathspy/tic-tac-toe-NN/tree/ab6731d216c66f5e09a4dabbe383df6dc745ba18
Attempting to do
pipenv install
in this lock-less repository have so far downloaded over 700MBs or so while it displayed
Locking [packages] dependencies...
Will give up in a bit and rerun with --skip-lock until it's fixed

I noticed that lock was really slow and downloaded huge amount of data from files.pythonhosted.org, more than 800MB for a small project that depends on scipy flask etc.
So I sniffed the requests made to files.pythonhosted.org, and it turns out that pip or pipenv were doing completely unnecessary downloads, which makes lock painfully slow.

For example, same version numpy had been downloaded several times in full. And it downloaded wheels for windows / linux, although I was using a Mac.
My setup:
$ pipenv --version
pipenv, version 2018.05.18

$ pip -V
pip 18.0 from /usr/local/lib/python2.7/site-packages/pip (python 2.7)

are additional Pipfiles helpful for debugging here?

Most likely @AlJohri, also any info about running processes / locks / io would help

Been stuck here for about 5 minutes already. First thought it might have been some sort of pip install issues and reinstalled everything fresh via Homebrew, but still the same problem. Any ideas why?

Finally finished after about 6 - 7 minutes. Pretty new to Python and Pipenv, so a little help about where to find the necessary files for debugging would be great! :)

this is pretty bad to the point I am afraid to install new python libs or upgrade existing ones.

After watching one of the talks from the creator, I decided to use pipenv. But it is too slow.

Thanks for your insightful feedback.

@techalchemy If there is something I could do to help and fix this. I am very happy to contribute.

Any news on this? Any way to help?

I find it install quickly and locks slowly, so as soon as you get the Installation Succeeded message your good to continue working... unless you want to install something else...

... or need to push the lock file into some repo.

Should install be preforming lock anyway, seeing that lock is already a separate command? In the meanwhile the install option description should specify that locking also takes place, and maybe even recommend --skip-lock.
Also, how about pinning this issue?

Pipenv is a really wonderful tool and I used to recommend it, but a project with 8 modules can't lock... it just times out. There doesn't seem to be any interest in solving this issue and that is very frustrating.

How is this not the main priority for this project? Pipenv is so slow its pretty much unusable. An not only in some uncommon side uses cases its always super slow.

Without knowing too much about what's going on under-the-hood, I was thinking that this might be solved nicely with a local cache of the dependency graph.
Think about it: if we cache that package x-1.2.3 depends on y>=3.4, we can locally do a lot of the work that's currently done in downloading packages one at a time, expanding them, and checking dependencies.  The lock phase would then be as simple as:

Compare the Pipfile to the cache and make sure we have everything we need.
If not, download anything new and calculate dependencies there

Cache the changes

Install.

In every case, while the first time might be slow, subsequent locks would be painless, no?

I just decided to use pipenv instead of pip for a small project. First thing I did was pipenv install -r requirements.txt. It's been locking the dependencies for about 10 minutes now. Therefore, I'm gonna go back to pip.
Guys, this issue is costing you a lot of users. I propose to address it quickly.

It's really frustrating using this plugin. In my case installing the dependencies on the server hangs the server for hours. I'm using AWS EC2 install t2.micro with 1 GB RAM. This much RAM is enough for a single application with few dependencies but the installation take all the memory and there is only one way to get it to work by restarting the server.
This issue is pending for so long years and no fix has been made for this. I see multiple issues being closed without any resolution.
I'm considering switching to something else better than pipenv.

Such a nice tool getting neglected. I will be unsubscriing from this issue as I see it will never get resolved. Will be sticking to something like conda or do it manually using virtualenv.

Guess I'll give https://github.com/sdispater/poetry a shot :|

Can an admin kindly close this thread to comments? It looks like no helpful additional content is being added to the discussion.
I'd be happy to subscribe a ticket tracking the work towards fixing the issue.
Thanks!

I suspect 99% of folks using this tool are programmers. Instead of whining, why not submit a PR.

Hello @idvorkin,
I've tried once.
It took weeks to achieve merging of the trivial fix.
Just compare the amount of discussions with the actual fix size.
I definitely do not want to submit any more fixes to this project.
So your advice is not as viable as you can assume.

@Jamim on behalf of the many users (and I suspect the admins as well), thank you for your contributions.  I read your PR, and I could empathize with the frustration.  However, I have to agree w/ @techalchemy  on this one:

Of course we care about the library we maintain, but I would suggest that the phrasing is probably not the most effective way to have a positive interaction.

I've never met the admins, but if they're anything like me (and maybe you) they are humans with busy lives whose  lives are packed with commitments even before they have energy to spend on this project.
Similarly, I bet if you (or anyone else) fixed the performance problem, you'd have slews of people who'd help you develop, test, merge it, or if required (and I highly doubt it would be) create a fork.

I'm thankful for the time the developers of this project are spending on this, but I suggest that it should be warned in bold that this project is not yet production ready right above the user testimonials in README.md, currently it's misleading people to spend precious time to replace their current pip/virtualenv stack with pipenv until they find out about this slow locking and they understand they can't use it.

until they find out about this slow locking and they understand they can't use it.

While I'd very happy to get a speed up as it is indeed very slow, there is no need for such hyperbole.
I'm using pipenv just fine every day. The workflow improvements it provides greatly outweight the occasional slowness. Locking is just not something I do all the time, as opposed to running scripts for example.
How often do you lock that it actually becomes such a problem you feel you can't use it? :open_mouth:

@bochecha my statement may be hyperbole in your opinion but it's a fact based on my experience, I heard about pipenv from some coworkers, today I tried to update an old project, updating its dependencies, etc I thought lets update from pip/virtualenv to pipenv as part of the update process. I had to update a dependency, check how things work with it, update parts of code if needed and then update another dependency, each time I ran pipenv install <something> I had to wait a ridiculously long time, first I thought it's calculating something and it'll cache it for future as I couldn't believe it's a problem in a claiming to be production ready package manager. After installing ~10th package I started searching about it and I found this thread, I removed Pipfile and Pipfile.lock and went back to my pip/virtualenv workflow. I was tempted to try poetry but I couldn't risk another hour.
This things happen in JS community for example but I don't expect it in a Python community, we don't have this kind of problems in our community and we should try to avoid it, a disclaimer in README.md can avoid this inconvenience so I suggested it in my comment. It could save my time today and I think it'll save time for other newcomers and they won't have a bad experience with this project so they may stay as potential future users.

I kinda agree with sassanh. Not everyone is equally affected by the issue but some of us were affected pretty bad. I have made open source projects that were not really fully functional or production ready and when it was the case I put a disclaimer on it so I don't waste people's time if they are not ready for the bumps.
I am not mad at the the people who work on this project but I am kinda mad at the person who made a public talk about it, selling it as a great tool with 0 disclaimer. As a result, I wasted quite a lot of my precious time trying to make a tool work, hoping to save time in the long run, but I ended up having to go back to pip and my own script, because pipenv didn't work in my time and bandwidth constrained environment.

each time I ran pipenv install

Did you know about pipenv install -r requirements.txt to lock/install only once from your existing project when you try moving it to pipenv? If you don't, the problem might be one of documentation?

Before anything I'm positive that pipenv will be a good replacement for pip/virtualenv workflow, I guess we all know that we need it and I think the day pipenv is production ready it'll be a great help for many people/projects.
@bochecha as I explained I had to install a newer version of a package, do some stuff and then go with the next package, maybe if I did this process with pip and then migrated to pipenv I wouldn't notice the problem at all, but I first migrated to pipenv and then did the package updates one by one and it was really annoying. I'm glad it works for your usecase, but I'm sure it doesn't work for many people like me (take a look at the comments above). It should be mentioned in the README.md, at least it should be mentioned that "each locking may take a long time, so if your usecase includes installing/removing lots of packages quickly you should avoid using pipenv until this issue is resolved" (again in bold, and on top of testimonials) if you announce the problems before anyone is affected by it, everyone will be thankful, if the problems affect others and you didn't warn them, everyone will get angry.

Agreed. I started looking into poetry and even though of adding another user to my OS per project instead of using pipenv again. If it's not working fine for casual use cases with the default settings it's broken imho.
It's a super useful tool! There's just this one thing that makes it super useless to me. And sadly I have little time to contribute :|

Sure, it is annoying waiting for a lock when having to do multiple installs mid dev session, but this can be managed.
The important thing is that a lock file is generated before pushing local changes to repo. I make judicious use of the —skip-lock flag during dev sessions, and pipenv lock once at the end before I commit.

Thanks for the project. But,
Locking is verrrrrrrrrrrrrrrrrrrrrrrrrrrrrry slowwwwwwwwwwwwwwwwwwwwwwwwwwwww.

PIP_NO_CACHE_DIR=off
This env makes locking more fast

Hi @yssource and everyone,

Thanks for the project. But,
Locking is verrrrrrrrrrrrrrrrrrrrrrrrrrrrrry slowwwwwwwwwwwwwwwwwwwwwwwwwwwww.

This project seems to be dead, so if you want to eliminate the speed issue please consider migrating to Poetry which is significantly faster.

@Jamim , thanks for suggesting Poetry. Personally, for some reason I did not come across it. After reading its readme it seems worth trying. It lists some benefits over Pipenv as well (https://github.com/sdispater/poetry/#what-about-pipenv).
Having said that, the project being dead is a gross overstatement, and if I were in pipenv authors' shoes, I would find it disrespectful. The author replied in the issues section just yesterday. It's just this locking issue being overlooked, probably because it is hard to fix.

For the record, Poetry suffers from performance issues as well:
https://github.com/sdispater/poetry/issues/338

I have the same problem in all of my projects.
The cause seems to be pylint.
Pipenv (pip) can install it successfully, but locking takes forever!
pipenv, version 2018.11.26

Minimal working example
djbrown@DESKTOP-65P6D75:~$ mkdir test
djbrown@DESKTOP-65P6D75:~$ cd test
djbrown@DESKTOP-65P6D75:~/test$ pipenv install --dev pylint --verbose
Creating a virtualenv for this project…
Pipfile: /home/djbrown/test/Pipfile
Using /usr/bin/python3 (3.6.9) to create virtualenv…
⠸ Creating virtual environment...Already using interpreter /usr/bin/python3
Using base prefix '/usr'
New python executable in /home/djbrown/.local/share/virtualenvs/test-PW-auWy_/bin/python3
Also creating executable in /home/djbrown/.local/share/virtualenvs/test-PW-auWy_/bin/python
Installing setuptools, pip, wheel...done.

✔ Successfully created virtual environment!
Virtualenv location: /home/djbrown/.local/share/virtualenvs/test-PW-auWy_
Creating a Pipfile for this project…
Installing pylint…
⠋ Installing...Installing 'pylint'
$ ['/home/djbrown/.local/share/virtualenvs/test-PW-auWy_/bin/pip', 'install', '--verbose', '--upgrade', 'pylint', '-i', 'https://pypi.org/simple']
Adding pylint to Pipfile's [dev-packages]…
✔ Installation Succeeded
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
⠇ Locking...

I heard about pipenv alot and tried it today,
the locking is also very slow for me. It's been already around 2 minute, still stuck on locking.
Downloading is pretty fast, but issue is with locking.
Is this issue resolved?
I am using Pop os 19.10, pipenv, version 11.9.0 from apt, python 3.7.5.

I want to draw attention to this excellent comment from #1914 on the same topic https://github.com/pypa/pipenv/issues/1914#issuecomment-457965038 which suggests that downloading and executing each dependency is not necessary any longer.
I wonder if any devs could comment on the feasibility of this approach.

I've noticed that it's actually faster to remove the environment and recreate it from scratch to update the lockfile.
This is true both for running pipenv lock and pipenv install some-package

I really like pipenv but not as much as I like my bandwidth and time. So I end up solving the issue using:
$ pipenv --rm
$ virtualenv .
$ source bin/activate
$ # Create a requirement file (Cause pipenv lock -r > requirements.txt... you know!)
$ pip install -r requirement.txt

Wish the developers best of luck...

@ravexina thanks for the suggestion, I'll try for sure

I would just like to note that this was never a problem when I was in a major city with awesome internet (it took like 3 seconds) and now I'm in a rural area and locking the EXACTLY SAME PROJECT takes at least 4 minutes.

I tried installing texttable packages, and waited for 30 minutes, still locking.
Did pipenv --rm and reinstalled the venv, didn't work either. I had to skip lock, and I am facing lock issues which is what I need.
Please guide me.

I really like pipenv but not as much as I like my bandwidth and time. So I end up solving the issue using:
$ pipenv --rm
$ virtualenv .
$ source bin/activate
$ # Create a requirement file (Cause pipenv lock -r > requirements.txt... you know!)
$ pip install -r requirement.txt

Wish the developers best of luck...

Try out py-poetry - more features and faster locking.

In reply to the last comment, poetry might be faster, but still depressingly slow. How much time do you think it takes to install a dummy package like https://pypi.org/project/insecure-package/ ? Here is the answer:
# poetry add insecure-package
Skipping virtualenv creation, as specified in config file.
Using version ^0.1.0 for insecure-package

Updating dependencies
Resolving dependencies... (594.2s)

Writing lock file

(Disclaimer: installation time depends on the size of your project, but I wouldn't call ours big).
The results above are obtained with
# poetry --version
Poetry version 1.1.5

which has "improved build speed" as per these issues:

https://github.com/python-poetry/poetry/issues/338
https://github.com/mozilla/experimenter/issues/3577

So don't expect Poetry to solve all of your problems. Package management in Python is still a PITA, at least in comparison with NodeJS / npm / yarn.

But it's still slower, and my pipenv doesnt ever stop locking, unlike others :(

My comment from another related pipenv issue:
pipenv install flamegraph (generated with python-flamegraph 0.1 statistical profiler for the main function of the resolver.py), see below (scroll down the svg)

This was generated on a poor vpn network connection. most of the time is spent in SSL.py read()
Looks like finding "candidates" is fast, only downloading is slow. Limit unnecessary downloads  is where we could gain something. I guess this is used for generating hashes?

I wonder which version of Pipenv are you using, generating hashes no longer downloads the artifacts, as long as SHA256 hash is included in the URL(True for most packages on pypi.org). So what package index are you using?

@frostming  pipenv           2020.11.15

It's sad to see all the bashing on this issue, constructive ideas make progress!
This might not work for everyone, but this workaround seems to speed things up a bit: pipenv --rm && pipenv install.
If that doesn't help, you can try to remove the lock file before running the above command.

I don't believe this exact issue is still relevant to latest versions of pipenv, such as the one released yesterday pipenv==2022.1.8.  I am closing this multi-year issue (that is kind of hard to parse) in hopes that we can get any new detailed bug reports that pertain to the latest version of pipenv filed instead.

Just to understand, was this issue resolved by another PR or simply closed?

I believe it was closed because there were many conflicting reports that could have been separate issues. Some of the issues may be resolved by time of the latest release, but others may still be relevant - in which case we should open new issues to track those issues.

@JoshuaPoddoku This issue it was determined that the current design uses hashes from the API and so there should not be redundant downloads in performing locking, much has changed since the original report was opened.  When looking for PRs that seemed relevant I performed this search:  https://github.com/pypa/pipenv/pulls?q=is%3Apr+is%3Amerged+hash
However I don't have a link to an exact PR, but frostming above had said:

generating hashes no longer downloads the artifacts, as long as SHA256 hash is included in the URL(True for most packages on pypi.org)

Basically @meichthys has it correct that any thing still relevant to the latest version of pipenv should be reported as new issues should it not match an existing open issue.   Closing this was part of a larger effort to evaluating/triaging the backlog of issues to achieve a more manageable and accurate backlog for this project.


sf org create user returns error after successfully creating user when using JWT auth

Summary
When using sf org create user to create a user in a scratch org command returns error when using JWT authentication even if user is successfully created.
Steps To Reproduce

Auth using sf org login jwt

sf org login jwt `
        --username $(orgUsername) `
        --jwt-key-file $(jwtKeyFile) `
        --client-id $(clientId) `
        --alias scratchOrgAlias `
        --instance-url $(orgLoginUrl)

Create user

    sf org create user --target-org scratchOrgAlias --definition-file $def --set-unique-username

Expected result

If user is created command should exit normally

Actual result

User is successfully created in org
CLI returns below

Errors encountered:
user hasn't approved this consumer
user hasn't approved this consumer
user hasn't approved this consumer
user hasn't approved this consumer
user hasn't approved this consumer
    at SfError.wrap (/home/vsts/.config/yarn/global/node_modules/@salesforce/core/lib/sfError.js:79:20)
    at catchCreateUser (/home/vsts/.config/yarn/global/node_modules/@salesforce/plugin-user/lib/commands/org/create/user.js:255:30)
    at getNewUserAuthInfo (/home/vsts/.config/yarn/global/node_modules/@salesforce/plugin-user/lib/commands/org/create/user.js:238:16)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async CreateUserCommand.run (/home/vsts/.config/yarn/global/node_modules/@salesforce/plugin-user/lib/commands/org/create/user.js:43:33)
    at async CreateUserCommand._run (/home/vsts/.config/yarn/global/node_modules/@salesforce/sf-plugins-core/node_modules/@oclif/core/lib/command.js:117:22)
    at async Config.runCommand (/home/vsts/.config/yarn/global/node_modules/@salesforce/cli/node_modules/@oclif/core/lib/config/config.js:400:25)
    at async run (/home/vsts/.config/yarn/global/node_modules/@salesforce/cli/node_modules/@oclif/core/lib/main.js:85:16); exitCode=1; warnings=System.Object[]}
@{code=1; context=CreateUserCommand; commandName=CreateUserCommand; message=Error authenticating with JWT.

System Information
Ubuntu - pwsh
{
  "architecture": "linux-x64",
  "cliVersion"<EMAIL_ADDRESS>  "nodeVersion": "node-v18.18.2",
  "osVersion": "Linux 6.2.0-1015-azure",
  "rootPath": "/home/vsts/.config/yarn/global/node_modules/@salesforce/cli",
  "shell": "bash",
  "pluginVersions": [
    "@oclif/plugin-autocomplete 3.0.1 (core)",
    "@oclif/plugin-commands 3.0.6 (core)",
    "@oclif/plugin-help 6.0.5 (core)",
    "@oclif/plugin-not-found 3.0.3 (core)",
    "@oclif/plugin-plugins 4.1.7 (core)",
    "@oclif/plugin-search 1.0.6 (core)",
    "@oclif/plugin-update 4.1.3 (core)",
    "@oclif/plugin-version 2.0.6 (core)",
    "@oclif/plugin-warn-if-update-available 3.0.2 (core)",
    "@oclif/plugin-which 3.0.8 (core)",
    "@salesforce/cli 2.17.14 (core)",
    "apex 2.3.20 (core)",
    "auth 2.8.26 (core)",
    "data 2.6.3 (core)",
    "deploy-retrieve 1.20.0 (core)",
    "info 3.0.1 (core)",
    "limits 3.0.1 (core)",
    "login 2.0.1 (core)",
    "marketplace 1.0.2 (core)",
    "org 2.11.8 (core)",
    "schema 3.0.1 (core)",
    "settings 2.0.2 (core)",
    "sobject 0.2.15 (core)",
    "source 2.11.0 (core)",
    "telemetry 3.1.0 (core)",
    "templates 55.5.18 (core)",
    "trust 3.0.2 (core)",
    "user 2.3.42 (core)"
  ]
}

Additional information

After the user is created, the CLI is going to try to auth the new user with the same JWT connected app.  Can you confirm that the created user will have access to do that (the Connected App allows their assigned Profile?)

the purpose of org create user is to create users that are auth'd via the CLI so you can do CLI stuff as that user.  Auth is built into the process.  I understand why that's confusing and maybe we could fix it.
Scratch org creation replicates the ConnectedApp from the DevHub into the scratch org.
If you just want to create a user as a part of a script, I'd suggest either

use sf data create record and pass the fields it needs
create a user via an apex script executed by sf apex run -f <your file>

I think this command doesn't support the jwt authentication method.

@nwcm it works if the user being created has access to the ConnectedApp (ex: SystemAdmin or some other profile is allowed for the ConnectedApp, and the user being created is assigned that Profile).  I'd guess that if you created a user that gets assigned the SystemAdmin profile it'd auth correctly.
That's the trick

Profile/PermSet has access to connectedApp in devHub
ConnectedApp is replicated to scratch org (that's how the Admin user gets access to the scratch org)
User is created that has one of the Profile/PermSets that can use the ConnectedApp.

If not, please let me know what command you're using and screenshot your ConnectedApp settings (permissionSets/Profiles) to make sure you've got everything correct.
You definitely can't set a password for a user that the CLI is not authenticated with.  (That API/permission is called selfSetPassword, meaning an admin can't do it for other users, on for themselves).  That's why the org password generate uses --on-behalf-of as a user that the CLI has auth'd and not just anybody in the org.
[there is an API to initiate the password reset process for a user that's not yourself, but they get an email link, etc to do the actual process]

@mshanemc this problem looks to be directly related to this known issue as the steps to reproduce mirror ours ( https://issues.salesforce.com/issue/a028c00000j5kSUAAY/an-error-message-returned-when-running-forceusercreate-for-scratch-org-of-hyperforce-using-oauth-20-jwt-bearer-flow )
However the issue is marked as working as intended with no reference to a document explaining why it's expected for this not to work.

Ahh, that was the missing step.  I should have put together that you're likely on hyperforce trying this, had forgotten about that "feature".

I'm going to put an error message in there so people know why this isn't working.  Marking that as a feature.

Right, so if i'm understanding correctly. Any CI/CD using hyperforce orgs can only ever use the original user created with the scratch org.

That's overly broad...you could use sfdx-url auth instead of JWT.  Those work fine on hyperforce.
I think there's also a really complex path to

create a new ConnectedApp in the scratch org (and keep the cert handy!)
re-auth the admin user using that
org create user

It's never going to work on hyperforce wth JWT.  We're changing the command to throw an error before it even tries when it knows you're both JWT and hyperforce.
If you need to automate this kind of thing, you'll need to do a non-JWT auth (ex: use sfdx-url).  That's how we do our automated tests.

Okay got it working. @mshanemc I think I have missed or misunderstood the proposed solutions in the different threads.
Here is a step-by-step of what worked for me to move from JWT to sfdx-url authentication:

In the SF cli, logout from all authenticated accounts including the dev-hub account.
Use sf org login web to link back your dev-hub account previously using JWT.
Now you can get the sfdx auth url from sf org display --target-org TestOrg1 --verbose
In the cli, logout again from the dev-hub account.
Again link the dev-hub account but now using the previously fetched Sfdx Auth Url with sf org login sfdx-url

Once these steps are done, creating scratch orgs and users will work. If you need to re-authenticate to existing scratch orgs or users, make sure to persist their unique sfdx auth urls using sf org display --target-org TestOrgOrUser --verbose when initially creating these.
So leaving these here and hope that helps someone else 😄

This issue is addressed in 2.21.7 (Dec 13, 2023). Thank you!


[UI] Scrolling irregularities in Meshery Docs

Current Behavior

Scrolling irregularities in Meshery Docs in Mobile and Tablet view. Won't able to scroll through whole content.
Desired Behavior

Screenshots/Logs

https://github.com/meshery/meshery/assets/65087356/445aa419-5bda-4d68-b8fe-744055845bc9
Environment

Browser: Chrome Safari Firefox
Host OS: Mac Linux Windows
Meshery Server Version: stable-v
Meshery Client Version: stable-v
Platform: Docker Kubernetes

Contributor Guides and Handbook

🎨 Wireframes and designs for Meshery UI in Figma (open invite)
🖥 Contributing to Meshery UI
🙋🏾🙋🏼 Questions: Discussion Forum and Community Slack

Can I work on this issue?

Hi, I will work on this issue. Please assign it to me.

As discussed in the meeting @Aayush987, go ahead with this one 🚀


AMD + signature js failed

can You please help me how to use signature js with amd approach. as I am new.

Sorry, I can't help you out. I don't know anything about AMD.


add some tests

I added some tests. I basically took some code from the examples and adapted it to the standard testing infrastructure of Julia. We can (and should) add more tests like these in the future when the interface stabilizes.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (3fad782) 97.40% compared to head (8e6b922) 97.40%.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@           Coverage Diff           @@
##             main       #9   +/-   ##
=======================================
  Coverage   97.40%   97.40%
=======================================
  Files           3        3
  Lines         193      193
=======================================
  Hits          188      188
  Misses          5        5

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Added an implementation to view the readme.

This change adds an implementation for the readme button. I've tried to keep the coding style from the repo intact.

Still need to implement a way to get non-downloaded readmes, but this is fine.


[RFC]: add math/base/special/asecf

Description
This RFC proposes adding math/base/special/asecf, which would be the single-precision equivalent of math/base/special/asec.
float stdlib_base_asecf( const float x )

Related Issues
None.
Questions
No.
Other
No.
Checklist

[X] I have read and understood the Code of Conduct.
[X] Searched for existing issues and pull requests.
[X] The issue name begins with RFC:.

I would like to work on this.


Removes legacy SPIFFE TLS clients and servers in favour of the new SPIRE TLS clients and servers.

E2E tests are currently failing on N-2 version skew, though it was my understanding that this was best effort and not guaranteed. Perhaps we need to be skipping this test for now.

/test-version-skew

/test-version-skew

E2E tests are currently failing on N-2 version skew, though it was my understanding that this was best effort and not guaranteed. Perhaps we need to be skipping this test for now.

The failures seem related to some setup steps, not code changes?

/test-version-skew

/test-version-skew

/test-version-skew

/test-version-skew


Add method to print backtrace

Implements https://github.com/godotengine/godot-proposals/issues/3062 but only for Linux now.
I'm not sure if OS is best class for it, but this works fine for now.
Not sure if I will finish this, so feel free to fork it.
This is quite slow(full backtrace prints ~5-10s on my OS).

Why close?

PR hasn't been updated for almost 2 years and no one has attempted to finish it and I don't quite know how it could be implemented for other systems, so I decided to just close it because I'm not going to work on issue anymore.

Pity - it would be a good thing to have


Adding specific targets to CI for testing wasm and no_std

This should ensure that if we make a change that prevents a successful compilation wasm, our CI should catch it. Same with no_std compatibility.

No longer needed, closing


ReferenceError: Blob is not defined

Can i know the reason and a fix for this?
thank you

https://github.com/FlowiseAI/Flowise/issues/57


bug with ReferencedList

I've had some issues with ReferencedList not being able to generate the correct REST request when showing a record. moreover when it happens ng-admin just hangs without showing any error message (and it seems to happen in other places when the server returns something different from a 200 status code)
this little patch seems to have solved the problem, but I'm not sure it is the correct or intended way to fix it.

diff --git a/src/javascripts/ng-admin/Crud/repository/RetrieveQueries.js b/src/javascripts/ng-admin/Crud/repository/RetrieveQueries.js
index dba756e..1baa2ec 100644
--- a/src/javascripts/ng-admin/Crud/repository/RetrieveQueries.js
+++ b/src/javascripts/ng-admin/Crud/repository/RetrieveQueries.js
@@ -101,7 +101,7 @@ define(function (require) {
                 filterName;
             params._filters = {};
             for (filterName in filters) {
-                if (filterFields[filterName].hasMaps()) {
+                if (filterFields[filterName] && filterFields[filterName].hasMaps()) {
                     angular.extend(params._filters, filterFields[filterName].getMappedValue(filters[filterName]));
                 } else {
                     params._filters[filterName] = filters[filterName];
@@ -209,7 +209,7 @@ define(function (require) {
             filter = {};
             filter[referenceList.targetReferenceField()] = entityId;

-            calls.push(self.getRawValues(referenceList.getReferencedView(), 1, null, sortField, sortDir, filter));
+            calls.push(self.getRawValues(referenceList.getReferencedView(), 1, filter, sortField, sortDir));
         }

         return this.$q.all(calls)

Thanks for your report !
It will be fixed in #233

thanks for your support :)


Segmentation fault when producing object files on linux

System:
WSL2/Debian “bullseye” x86_64
Version used:
prebuild codon-linux-x86_64.tar.gz
deployed in ~/.local directory
Reproducer
$ codon --version
0.15.5
$ cat test.py
@export
def main():
    return 1
$ codon build -o test.o test.py
Segmentation fault (core dumped)

Core dump attached

Thanks for the report -- this seems to be because the function is named main, which clashes with the generated main. Will fix ASAP; for now the workaround would be to use a different function name.

Fixed in 0.16.1!


IC Code harmonization with the PromotionalVideo

Refactoring - unimportant changes

No more commits are expected. Consider it done.


Update custom-resource-definition-versioning.md

Since there can be only one version that needs to be stored and storage option is required else the parsing fails. Adding the storage option and setting it false.
Schema Vaildation fails with error: error validating data: ValidationError(CustomResourceDefinition.spec.versions[1]): missing
required field "storage" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1.CustomResourceDefinitionVersion; if you choose to
ignore these errors, turn validation off with --validate=false

/assign @jlbutler

From kubectl explain CustomResourceDefinition.spec.storage

storage       -required-
storage indicates this version should be used when persisting custom
resources to storage. There must be exactly one version with storage=true

/lgtm

/lgtm

There's 3 lgtms
/approve


Point extend

abc
023ef30130654689a64c864d6dd38760481c55fc525e2c6c7084e2d2d3d4d51be9
043ef30130654689a64c864d6dd38760481c55fc525e2c6c7084e2d2d3d4d51be9f7d86b288c09ddb5311f292285168000e43e4b62201bd8de23a391daa8e00ce8
def
036a6e1dc6f203f7fdd97965892301e5fb995a37318c410543835f0edcd3456c49
046a6e1dc6f203f7fdd97965892301e5fb995a37318c410543835f0edcd3456c492a072b9898b93e9eb05f9ad86a97546d83b579bf6efd3482f93baca13784496b
abcdef
0312faae608bd6562562b8f85564664cd1fdcd667f6b24b2b221ef86b9231f4d74
0412faae608bd6562562b8f85564664cd1fdcd667f6b24b2b221ef86b9231f4d74512ee8cd9b34331afd05ccb8d81d1393c150c73ec5695845b731f7e6e0086719
here abc is 1 point, and 2nd point is def
if we can join or merge like abc add to def
where result would be abcdef point
have you study and experiance this
maybe some help here
http://web.math.princeton.edu/swim/SWIM 2010/Yao-Zhan Presentation SWIM 2010.pdf
hope you will add this func too

point 1
point 2
a =1+2 = 3
more advance looking func
point 1
point 2
a =1+2 = 12

What is this function supposed to do?

as above pubkey abc add(line extend) with pubkey def
and result of pubkey of abcdef

as add func inside your curve.py, bring
abc+def = 18AB
one new func could be inside
like abc+def = abcdef
like point +point = pp ( line extended) ( not 2p)

You can’t just concatenate coordinates (if that’s the request) on a curve to make new coordinates. I’m not sure what the use case would be but the result would not be a valid curve point.

if both point are under F, then hope could be valid, and if range goes out, then error will apear, try some fomula's, maybe we reach at Goal

I still don’t think i understand the intent or use is here. Unless you can provide a detailed example of how you expect this to work, along with use cases for why this is needed, I’m going to close this ticket.

during play with ecc, add, substract, multiply, div, i check all ways working, and suddenly apear in mind, could be point addition like point concatenate
like as above example of abc is hex and pubkey, add with def is hex and pubkey
if it works, then further calc will help me to research inside results

I don't think there's a reason to include concatenation as part of this library since not a standard ECC operation and doesn't have any practical applications i.e. cryptographic schemes that use it. You should be able to implement this fairly quickly if you want to experiment with key concatenation but I don't see any reason to support it in this codebase.

you can create separate codebase for beta testing share with me personal or publicly , and let world test and use in different stages,
no one can predict before , till some idea's come to know in practical,

Feel to free to fork this repo and implement any experiments as you see fit.


just rename file

just rename the file, without moving the file to another folder.
Is there a way to do that?

if i add 5 files, just 1 file can not find data. Move and rename file feature will not work. I think we should move or rename file right after finishing 1 file. it will not be affected by other files process.


Minor fixes

on line 73: Wallet fils or Wallet fills?

@kant thank you so much for submitting your fixes.


Unexpected issue with deployment

Just tried uploading a new app to the webserver and encountered this issue:
http://pastebin.com/raw.php?i=1gdhEcBg
If you need any additional details let me know
mups logs -f:
Error: listen EADDRINUSE
at errnoException (net.js:905:11)
at Server._listen2 (net.js:1043:14)
at listen (net.js:1065:10)
at net.js:1147:9
at dns.js:72:18
at process._tickCallback (node.js:442:13)
error: Forever detected script exited with code: 8

Running version 0.10.6


Rename taxonomy kinds from taxonomy to term, taxonomyTerm to taxonomy

And we have taken great measures to limit potential site breakage:

For disableKinds and outputs we try to map from old to new values if possible, if not we print an ERROR that can be toggled off if not relevant.
The layout lookup is mostly compatible with more options for the new term kind.

That leaves:

Where queries in site.Pages using taxonomy/taxonomyTerm Kind values as filter.
Other places where these kind value are used in the templates (classes etc.)

Fixes #6911
Fixes #7395

OK, I have revisited the "pages from data" which requires a fair amount of precision in this department -- and that went out with the bath water when it came to the taxonomies ...
I have taken great care to make this as bakwards compatible as possible (ref. the new ignoreErrors setting etc.) and I don't see any issues in any of the many sites that I somehow "manage". I will take the theme sites for a spin once this is in master... There will obviously be some odd site out there that will have an issue, but we really need to fix this.

This is a welcome change. Thank you. Here are two examples of changes that we needed to make:

CSS selectors based on kind in body classes.
A custom meta description based on kind.

Both were trivial.


typo: web/resource-manager/Microsoft.Web

nuget -> NuGet
mimuntes -> minutes
repesenting -> representing
souce -> source
containting -> containing
databses -> databases
inclue -> include
Priviledged -> Privileged

Automation for azure-sdk-for-ruby

Automation for azure-sdk-for-python

Can one of the admins verify this patch?

Automation for azure-sdk-for-js

Automation for azure-sdk-for-go

Automation for azure-sdk-for-java

Automation for azure-sdk-for-node


Restrict license checks to staged/committed files only

Why are these changes needed?
In my previous implementation of the license check, #90, when running the check locally with pre-commit run it would check all python files, irrespective of whether they were staged or not.
This update fixes that.
Also, when this runs as a GitHub action it will check against the base branch.
Related issue number
N/A
Checks

[ ] I've included any doc changes needed for https://ag2ai.github.io/ag2/. See https://ag2ai.github.io/ag2/docs/Contribute#documentation to build and test documentation locally.
[ ] I've added tests (if relevant) corresponding to the changes introduced in this PR.
[X] I've made sure all auto checks have passed.

There is code formating problem.

There is code formatting problem.

Yep, thanks - let me make sure it goes through as the code covers both local use and GitHub action use - so need to make sure it's working in GitHub too :)

Okay, passed!


Geo Distance Query

Hi!
Which version of Elastic are you using?
[ ] elastic.v3 (for Elasticsearch 2.x)
I created an index "findmystore" and two types "file" (that doesn't matter) and "store".
"store" is composed of a location (struct with a lat and a lon), an address, a postal and a city. For example, I get this as a description when I look at one element of type "store" :
"_source": {
    "location": {
        "lat": 50.6341605,
        "lon": 3.0656515
    },
    "address": "40 rue Des Tanneurs",
    "postal": 59000,
    "city": "LILLE"
},

I'm using currently using the following code to make the query and execute it :
distanceQuery := elastic.NewGeoDistanceQuery("store.location")
distanceQuery = distanceQuery.Lat(48.81433999999999)
distanceQuery = distanceQuery.Lon(2.3204906)
distanceQuery = distanceQuery.Distance("5km")

boolQuery := elastic.NewBoolQuery()
boolQuery = boolQuery.Must(elastic.NewMatchAllQuery())
boolQuery = boolQuery.Filter(distanceQuery)

searchResult, err := client.Search().Index("findmyfpstore").Type("store").Query(boolQuery).Do()

As store seems to be already specified in the first line of the code, I tried removing it in either the first or the last line but it doesn't seem to change much as I keep getting this error : elastic: Error 400 (Bad Request): all shards failed [type=search_phase_execution_exception]
I also tried to get the source of the boolQuery and execute it on elasticsearch head but it returns me the same error.
Thanks in advance!

Can you please enable the trace log and send me the output of your request and response. See here on how to enable the trace log.

Thanks for your reactivity
I'm looking to enable and get the trace log.
While I'm doing this, I have more information from elasticsearch head, maybe it can help as it is the same error I get on the console when I execute my script :
{
    "error": {
        "root_cause": [
            {
                "type": "search_parse_exception",
                "reason": "failed to parse search source. unknown search element [bool]",
                "line": 1,
                "col": 9
            }
        ],
        "type": "search_phase_execution_exception",
        "reason": "all shards failed",
        "phase": "query",
        "grouped": true,
        "failed_shards": [
            {
                "shard": 0,
                "index": "findmyfpstore",
                "node": "1d-Qdj0WQla_w6ZpvkhShQ",
                "reason": {
                    "type": "search_parse_exception",
                    "reason": "failed to parse search source. unknown search element [bool]",
                    "line": 1,
                    "col": 9
                }
            }
        ]
    },
    "status": 400

}

Yes. Seems like your query is malformed. It's the same issue as this. I need to look at your code to find out what's wrong. The code sample you submitted above looks okay.

The whole code looks like that, there are not much things happening aside from connecting to ElasticSearch and printing the results:
package main

import (
	elastic "gopkg.in/olivere/elastic.v3"
	"encoding/json"
	"fmt"
)

type Location struct {
	Lat			float64 	`json:"lat"`
	Lon			float64		`json:"lon"`
}

type Store struct {
	Loc			Location	`json:"location"`
	Address		string		`json:"address"`
	Postal		int			`json:"postal"`
	City		string		`json:"city"`
}

type Coord struct {
	RequestId	int			`json:"request_id"`
	Lat			float64		`json:"lat"`
	Lon			float64		`json:"lon"`
}

func main() {
	client, err := elastic.NewClient()
	CheckErr(err)

	distanceQuery := elastic.NewGeoDistanceQuery("store.location")
	distanceQuery = distanceQuery.Lat(48.81433999999999)
	distanceQuery = distanceQuery.Lon(2.3204906)
	distanceQuery = distanceQuery.Distance("10km")

	boolQuery := elastic.NewBoolQuery()
	boolQuery = boolQuery.Must(elastic.NewMatchAllQuery())
	boolQuery = boolQuery.Filter(distanceQuery)

	src, err := boolQuery.Source()
	CheckErr(err)

	data, err := json.Marshal(src)
	CheckErr(err)

	got := string(data)

	fmt.Println(got)

	searchResult, err := client.Search().Index("findmyfpstore").Type("store").Query(boolQuery).Do()
	CheckErr(err)

	if searchResult.Hits.TotalHits > 0 {
		fmt.Printf("Found a total of %d stores\n", searchResult.Hits.TotalHits)

		for _, hit := range searchResult.Hits.Hits {
			var s Store
			err := json.Unmarshal(*hit.Source, &s)
			if err != nil {
				// deserialization failed
			}

			fmt.Printf("Store from - Address : " + s.Address + " ; City : " + s.City)
		}
	} else {
		fmt.Print("Found no stores\n")
	}
}

func CheckErr(err error) {
	if err != nil {
		fmt.Println(err)
	}
}

I get this after enabling the trace log :
ELASTIC 2016/09/07 12:50:56 POST /findmyfpstore/store/_search HTTP/1.1
Host: <IP_ADDRESS>:9200
User-Agent: elastic/3.0.50 (linux-amd64)
Transfer-Encoding: chunked
Accept: application/json
Content-Type: application/json
Accept-Encoding: gzip

1
{
94
"query":{"bool":{"filter":{"geo_distance":{"distance":"10km","store.location":{"lat":48.81433999999999,"lon":2.3204906}}},"must":{"match_all":{}}}}}
0

elastic: Error 400 (Bad Request): all shards failed [type=search_phase_execution_exception]

This one works fine: https://gist.github.com/olivere/2804a338bd743ac09942180000c6b325
Watch out! It will delete your index and recreate it with a mapping on every execution!

Thanks for your solution.
I used to make a search query in an older version to retrieve all the existing stores but it doesn't seem to work very well with the mapping (I corrected the mapping a bit as there was 2 address properties) :
searchResult, err := client.Search().
Index(indexName).
Type(storeType).
Do()
CheckErr(err)
It returns an error 503 :
POST /findmyfpstore/store/_search HTTP/1.1
Host: <IP_ADDRESS>:9200
User-Agent: elastic/3.0.50 (linux-amd64)
Transfer-Encoding: chunked
Accept: application/json
Content-Type: application/json
Accept-Encoding: gzip
1
{
1
}
0
elastic: Error 503 (Service Unavailable): all shards failed [type=search_phase_execution_exception]
panic: runtime error: invalid memory address or nil pointer dereference
I switched the number of shards to 5 and it seems to work better, I'm continuing to test!

The code above does not execute any search query. The Query part is
simply missing.
On 7 September 2016 at 14:50, William Ung<EMAIL_ADDRESS>wrote:

Thanks for your solution.
I used to make a search query in an older version to retrieve all the
existing stores but it doesn't seem to work very well with the mapping (I
corrected the mapping a bit as there was 2 address properties) :
searchResult, err := client.Search().
Index(indexName).
Type(storeType).
Do()
CheckErr(err)
It returns an error 503 :
POST /findmyfpstore/store/_search HTTP/1.1
Host: <IP_ADDRESS>:9200
User-Agent: elastic/3.0.50 (linux-amd64)
Transfer-Encoding: chunked
Accept: application/json
Content-Type: application/json
Accept-Encoding: gzip
1
{
1
}
0
elastic: Error 503 (Service Unavailable): all shards failed
[type=search_phase_execution_exception]
panic: runtime error: invalid memory address or nil pointer dereference
I switched the number of shards to 5 and it seems to work better, I'm
continuing to test!
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/olivere/elastic/issues/351#issuecomment-245270130,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAIOwHODgwoNkizS-k9AsXymnVb7JD18ks5qnrL-gaJpZM4J2wfN
.

Thanks for your replies, I think it's working well! One more question, why do you delete the index and recreate it on every execution ? Is this necessary ?

No. Of course it's not. But that makes the example idempotent: It does
behave exactly the same every time you call it.
On 7 September 2016 at 15:35, William Ung<EMAIL_ADDRESS>wrote:

Thanks for your replies, I think it's working well! One more question, why
do you delete the index and recreate it on every execution ? Is this
necessary ?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/olivere/elastic/issues/351#issuecomment-245282174,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAIOwGIXKwez_1Rpr52jl7FQwYVmdTujks5qnr26gaJpZM4J2wfN
.

I understand, thanks again!
I finally found the combination of funcs I needed to find the closest store for a specified location, I'll leave this here in case it might help someone:
	geoDistanceSorter := elastic.NewGeoDistanceSort("location").
		Point(48.81433999999999, 2.3204906).
		Unit("km").
		GeoDistance("plane").
		Asc()

	matchAllQuery := elastic.NewMatchAllQuery()
	searchResult, err := client.Search().
		Index(indexName).
		Type(typeName).
		Query(matchAllQuery).
		Size(1).
		SortBy(geoDistanceSorter).
		Do()

Closing the issue then.


Port Request: minhook

Minhook - The Minimalistic x86/x64 API Hooking Library for Windows.
It would be great to have a port for it.

trying to support it with the upstream in the first.
https://github.com/TsudaKageyu/minhook/pull/58

Looks like maintainer of minhook project do not merge it there. Can CMake be added as patch from VCPKG side?

Hi @iskiselev, thanks for posting this issue. The port has been merged to the master. Please try to use it.


Volume controls are working in reverse

When I move the volume slider up, the volume goes down and vice versa. However, if you move the volume slider all the way to the left (no volume), it still mutes the system. Brightness slider works as expected. This also happens when I use keyboard shortcuts for volume (volume up decreases volume etc.)

This is the same as this one: https://github.com/MonitorControl/MonitorControl/issues/305
I'll close this. Keep following the other thread. If I have the time I'll implement reversing the controls as an advanced setting as it seems to be an issue affecting several displays with weird firmwares.


Web is not building [isar 2.5.0]

My project is not building when target web. A lot of errors appears on the console, for example:
: Error: Type 'WorkContractFinalizer' not found.
../…/working/work_contract.g.dart:2158

 QueryBuilder<WorkContract, List<WorkContractFinalizer>, QQueryOperations>

Web works fine for 2.5.0. I suggest doing flutter clean & flutter pub upgrade. Still facing, then flutter pub cache repair.


Fix pattern of output for coffeelint

Sometimes output of coffeelint contains comma , like max_line_length error below.
Line exceeds maximum allowed length Length is 122, max is 120.

This PR fixes it.

Yep, looks fine to me. Cheers! :beers:


Anisotropic diffusion coefficient

Some applications may require an anisotropic diffusion coefficient.
It may be worth trying to implement that in FESTIM.
Some useful links:

https://fenicsproject.discourse.group/t/anisotropic-heterogeneous-diffusion/10826

@RemDelaporteMathurin
What about other material properties, say, thermal conductivity?

@KulaginVladimir there you go I adapted it to thermal conductivity
https://github.com/RemDelaporteMathurin/FESTIM/blob/anisotropy/anisotropy_thermal.ipynb


Crash on Reactant.compile

When trying to run the test on my aarch64 macOS, I'm getting the following fatal error on line https://github.com/EnzymeAD/Reactant.jl/blob/6f9c9a141e40243e097e9fe6d29bb2b9c715a2dc/test/basic.jl#L15-L15
Precompiling project...
  7 dependencies successfully precompiled in 20 seconds. 122 already precompiled.
     Testing Running tests...

Yeah that’s a misconfigure on Yggdrasil which I’ve made a fix for but not
released a jll. You should use Linux or x86 for now
On Sat, May 4, 2024 at 3:14 PM Sergio Sánchez Ramírez <
@.***> wrote:

When trying to run the test on my aarch64 macOS, I'm getting the following
fatal error on line
https://github.com/EnzymeAD/Reactant.jl/blob/6f9c9a141e40243e097e9fe6d29bb2b9c715a2dc/test/basic.jl#L15-L15
Precompiling project...
7 dependencies successfully precompiled in 20 seconds. 122 already precompiled.

You are receiving this because you are subscribed to this thread.Message
ID: @.***>

Continues in #33


Peppy.Osu! version 2021.612.0

[ ] Have you signed the Contributor License Agreement?
[ ] Have you checked that there aren't other open pull requests for the same manifest update/change?
[ ] Have you validated your manifest locally with winget validate --manifest <path>?
[ ] Have you tested your manifest locally with winget install --manifest <path>?
[ ] Does your manifest conform to the 1.0 schema?

Note: <path> is the name of the directory containing the manifest you're submitting.

Microsoft Reviewers: Open in CodeFlow

wait still not updated :/


Fully document environment variables

rustic reads various environment variables at runtime, that also interfere with configuration settings.
For a better overview, we should therefore document:

[ ] which environment variables there are
[ ] what is the priority these values are applied (cli > env > config?)

what is the priority these values are applied (cli > env > config?)

Exactly like this! But with restrictions: You can overwrite values, but for many,you cannot "unset" them.


Update CHANGES.md with Python prism note.

Release branch equivalent to https://github.com/apache/beam/pull/33262 which updates the blog as well.
The note has already been added to the github release blurb: https://github.com/apache/beam/releases/tag/v2.61.0
Back updating the CHANGES for something that made the 2.61 release, but wasn't noted in the CHANGES

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

R: @damccorm


How do I add vipAddress in httpTask for Netflix Conductor?

I have the microService registered on eureka discovery service with service registered under hostname:service_name: port. How do I add it as VipAddress for a http_task in my conductor service?

@Prajsp You can use vipAddress in http_request, and then uri to provide the path. Checkout docs for http task for an example: https://netflix.github.io/conductor/configuration/systask/#http

Can anyone provide a simple example? Document is not very explanatory. I appreciate any help on this. For eg: my microService is registered in eurekha, by the name "helloService". Now how I form my HTTP request to from HTTP system task?