Add mbstring dependency to composer.json

Parsedown requires PHP's mbstring extension due to
https://github.com/erusev/parsedown/blob/d638fd8a25ba1a04106a663d80442afe5bc8d8a0/Parsedown.php#L170
Parsedown never worked without the mbstring extension and users were experiencing PHP errors. Most PHP installations have mbstring enabled (even though it's not one of PHP's default extensions, it's a default extension on virtually any Linux distribution), so IMO there's no need for action here besides making this implicit dependency explicit. Users which can't install mbstring on their platform can still use a polyfill like https://github.com/symfony/polyfill-mbstring, but since it's comparibly slow and users should rather install the mbstring extension whenever possible, we shouldn't make the polyfill the default. Letting users know about this is IMO the best way to deal with it.
Also see #541 - what should be closed IMO

I don't contest that Parsedown should declare the dependencies it uses, but I wonder whether we have an opportunity to make Parsedown a little more compatible here.
Given that we only use mb_strlen from mbstring, I wonder whether Parsedown should implement this single function as an internally available polyfill to avoid needing the dependency. (The implementation in https://github.com/symfony/polyfill-mbstring is pretty short for what its worth, and looks very similar to something already suggested).
Of course, in the current version this dependency should be explicit. I just wonder whether a better solution would be to support vanilla PHP instead of declaring a dependency that just makes Parsedown a little faster?

The problem is not code complexity, but performance. The polyfill takes 38 times (!) longer than native mbstring. When calling native mb_strlen() using Parsedown's code above and the first paragraph of Lorem ipsum as input text takes around 0.058 seconds for 10,000 rounds (i.e. 10,000 lines), but 2.204 seconds using the mb_strlen() polyfill (testing on my local machine with PHP 5.6). Naturally this isn't a credible performance test, but there's a reason why the creators of the polyfill tell people to rather install native mbstring :wink: The polyfill is intended as last resort solution, no more.
And there's a second catch: Even when using the polyfill we still require the iconv extension. iconv is one of PHP's default extensions, but still can be disabled, so we definitly still need a explicit dependency. There's virtually no difference to requiring mbstring, mbstring is available virtually anywhere (I'd even say that not having mbstring enabled is rather a misconfiguration of PHP, virtually any web application depends on mbstring and that's the reason why all Linux distributions enable mbstring by default).
Parsedown is a library, not an end-user application. We should leave this decision up to (1) the application developer (as soon as they require the polyfill the ext-mbstring requirement is fulfilled), and (2) the developer should IMO leave this decision up to the user (but there might be application developers with a different opinion on this, but this is nothing we should decide as library developers). Using the polyfill is a bad idea in general: If you don't have native mbstring, install native mbstring. The polyfill really is just a last resort solution - and the user should be aware of the fact that the polyfill is a last resort solution and he sacrifices one of Parsedown's main goals: performance. Silently using this polyfill IMO is a bad idea.

I think the performance impact doesn't really matter, because the performance impact would only be felt if the slow polyfill implementation was actually used. If mb_strlen is available then obviously we shouldn't use the alternative (slower) implementation ;p
i.e. If don't have mbstring then Parsedown would be slower, but would at least work. If you do have mbstring installed then there is no difference.
Would be something like
protected static function mb_strlen(string $s, $enc = null)
{
    static $polyfillNeeded;

    if (!isset($polyfillNeeded)) { $polyfillNeeded = !function_exists('mb_string'); }

    return $polyfillNeeded ? self::slower_mb_strlen($s, $enc) : mb_strlen($s, $enc);
}

This would still give the end user a decision: if they want to (and can?) install mbstring then Parsedown will be faster, if not then at least it'll still work.
You do make a good point about iconv also being an extension though (albeit being a core/default one), so relying on this instead of mbstring probably doesn't gain too much.
Maybe (as you say) it is just simpler to require the dependency we actually need, not having mbstring enabled is a pretty weird situation anyway ;p (and probably doesn't happen too often?)

and probably doesn't happen too often?

One in a million or something like that? This problem virtually doesn't exist for production servers. We recently learned that there are millions of Parsedown installations out there - and there are exactly two Issues about this, this one count in :laughing:

Maybe (as you say) it is just simpler to require the dependency we actually need

Exactly.

This would still give the end user a decision

Simply put, it's opt-in vs. opt-out. I'd strongly vote for opt-in - not because I think that this makes Parsedown slower for users with native mbstring (it won't), but because I think that using the polyfill is never a good idea. By using a opt-in strategy users are forced to think about the problem and make a decision (and the decision should always be "install native mbstring" in the end :laughing: and the "one in a billion" user who really wants to use the polyfill can still run composer require symfony/polyfill-mbstring and the warning disappears).  It's because of a misconfiguration, but this isn't obvious. Since Parsedown is a library I assume that everyone here knows what to do when a error message like "PHP extension mbstring is missing" is shown :laughing:


Added Poland Stock Exchange XWAR calendar

Added Poland Stock Exchange XWAR calendar with Holidays

Turns out some of these holiday declarations are broken, I'm working on fixes and tests in https://github.com/quantopian/trading_calendars/pull/74


mysql insert execute

There is an error when I try to insert data to the database.
Warning: mysql_num_rows() expects parameter 1 to be resource, boolean given in sparrow.php on line 809
<?php
require 'sparrow.php';

$db = new Sparrow();
$db->setDb('mysql://user:pass@localhost/database');

$data = array('name'=>'Abe');
$db->from('users')->insert($data)->execute();

Mac OS X 10.6.8
PHP 5.3.8
Apache 2.2.20
MySQL 5.5.15

I found this on the PHP manual - mysql_query

For SELECT, SHOW, DESCRIBE, EXPLAIN and other statements returning resultset, mysql_query() returns a resource on success, or FALSE on error.

For other type of SQL statements, INSERT, UPDATE, DELETE, DROP, etc, mysql_query() returns TRUE on success or FALSE on error.

And on the line 809 you're expecting a 'resultset' and is receiving a 'boolean' when I try to insert.
$this->num_rows = mysql_num_rows($result);

And I found this on mysql_num_rows

Retrieves the number of rows from a result set. This command is only valid for statements like SELECT or SHOW that return an actual result set. To retrieve the number of rows affected by a INSERT, UPDATE, REPLACE or DELETE query, use mysql_affected_rows().

Looks like you need and if to detect a select or show and inserts, updates, replaces or deletes to use only one function.

Thanks. I merged your pull request.


Error: Get config: Unable to establish a connection to query-engine-node-api library

$/prisma/starter$ npx prisma generate
Prisma schema loaded from prisma/schema.prisma
Error: Get config: Unable to establish a connection to query-engine-node-api library

Prisma CLI Version : 3.15.1-dev.1

How to Reproduce
OS: 22.04 Ubuntu
Node: 16.15.1
Follow these instructions. https://www.prisma.io/docs/getting-started/quickstart#1-download-the-starter-project
Switch to latest possible release: 3.15.1-dev.1 as per today.
What have i tried so far
Switching databases sqlite, postgres
Switching node from 16 to 17.
using 3.13 3.14 gives me this error
/snap/core20/current/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by /lib/x86_64-linux-gnu/libcrypto.so.3)

in ubuntu i could not figure out how to install GLIBC_2.33
tying with npm and yarn

Error message is from our new error reporting code from @jkomyno: https://github.com/prisma/prisma/blob/a490f5a82baeccf985c2cc30476f45748ed0fcad/packages/sdk/src/engine-commands/queryEngineCommons.ts#L80

To confirm:
Using 3.14.0 you are getting a different error message around GLIBC?

I got this error when using 3.14.0 and 3.13.0
libssl.so.1.1: cannot open shared object file: No such file or directory

i confirm using version 3.13.x 3.14.x gives this error
/snap/core20/current/lib/x86_64-linux-gnu/libc.so.6: version GLIBC_2.33' not found (required by /lib/x86_64-linux-gnu/libcrypto.so.3)
`
when using 3.15 gives the Config Error described above.
in bouth cases making prisma work on ubuntu 22.04 is not possible.

@irfanabdnst Please create a new issue providing all the info the bug issue template asks for. Thanks.

Looks like you are using snap to either install Node.js or openssl or maybe even glibC. Since snaps are sandboxes we can't dynamically link to required runtime libraries and so it will throw out that error. So please try installing these libs using apt:
sudo apt install openssl openssl-dev libc6

https://github.com/nodesource/distributions/blob/master/README.md
# Using Ubuntu
curl -fsSL https://deb.nodesource.com/setup_16.x | sudo -E bash -
sudo apt-get install -y nodejs

Ok the snap version of node is not compatible with prisma.
Its going to trap ubuntu users who install almost everything with snap.

We want to keep this open for the bug with the error reporting that you also reported. Thanks for your help in confirming that this was caused by the snap thingie.

Hi @PiecePaperCode, to summarise, this issue was due an incompatibility between your system's snap, as highlighted in this comment.
A related issue is that the current error message can be somewhat obscure and misleading. We closed it with this PR.
Tomorrow we will release a new major version that will contain improvements such as the following error message that tries to explain which connecting to the query-engine-node-api library failed:
Environment variables loaded from .env
Prisma schema loaded from prisma/schema.prisma
Error: Get Config: Unable to establish a connection to query-engine-node-api library. It seems there is a problem with your OpenSSL installation!
Details:Unable to require(`/home/prisma/.nvm/versions/node/v16.15.1/lib/node_modules/prisma/libquery_engine-debian-openssl-1.1.x.so.node`)
 libssl.so.1.1: cannot open shared object file: No such file or directory

Prisma CLI Version : 3.16.0-dev.102

Please let us know if we can close this issue then :)

I am also getting this error when running prisma generate though docker using the FROM node:16.15-alpine3.16 image and installing openssl through apk:
FROM node:16.15-alpine3.16
RUN apk add --no-cache git curl python3 make gcc libc-dev g++ libc6-compat openssl openssl-dev \
    && curl -sL https://unpkg.com/@pnpm/self-installer | node
RUN ln -sf python3 /usr/bin/python

Here is my example repo

What error exactly @cmwhited?
Can you try with a prisma@dev and @prisma/client@dev (or version 4.0.0 which will be release tomorrow) please and post the full output you are getting? That would help us to make sure you ar egetting the same error indeed.

What error exactly @cmwhited? Can you try with a prisma@dev and @prisma/client@dev (or version 4.0.0 which will be release tomorrow) please and post the full output you are getting? That would help us to make sure you ar egetting the same error indeed.

Updated to v4.0.0 and it still fails. I think it is actually an issue with pnpm. Testing now

Your error is different @cmwhited, which is now nicely visible with the new error message. Can you copy your last message into a new issue please? Thanks. Happy to investigate what is going on there.
Follow up question 1 (please reply in new issue): Does /app/node_modules/prisma/libquery_engine-linux-arm64-openssl-1.1.x.so.node exist?

https://github.com/prisma/prisma/issues/14043

Error: Get config: Unable to establish a connection to query-engine-node-api library. -- I got this error when i try to connect to SQLite Database using Prisma

Hi @PiecePaperCode, to summarise, this issue was due an incompatibility between your system's snap, as highlighted in this comment. A related issue is that the current error message can be somewhat obscure and misleading. We closed it with this PR.
Tomorrow we will release a new major version that will contain improvements such as the following error message that tries to explain which connecting to the query-engine-node-api library failed:
Environment variables loaded from .env
Prisma schema loaded from prisma/schema.prisma
Error: Get Config: Unable to establish a connection to query-engine-node-api library. It seems there is a problem with your OpenSSL installation!
Details: Unable to require(`/home/prisma/.nvm/versions/node/v16.15.1/lib/node_modules/prisma/libquery_engine-debian-openssl-1.1.x.so.node`)
 libssl.so.1.1: cannot open shared object file: No such file or directory

Prisma CLI Version : 3.16.0-dev.102

Please let us know if we can close this issue then :)

I still have this issue, which new version can fix it?

There is no new version that can fix, as this is a problem on your system. See the instructions on how to fix here: https://github.com/prisma/prisma/issues/13717#issuecomment-1151080267

$ npm-run-all build-tracker build-geo build-db build-app
$ rollup -c rollup.tracker.config.js
tracker/index.js → public/umami.js...
created public/umami.js in 359ms
$ node scripts/build-geo.js
Saved geo database: E:\my\umami-master\umami-master\public\geo\GeoLite2-Country.
mmdb
$ npm-run-all copy-db-files build-db-client
$ node scripts/copy-db-files.js
internal/modules/cjs/loader.js:818
throw err;
^
Error: Cannot find module 'fs-extra'
Require stack:

E:\my\umami-master\umami-master\scripts\copy-db-files.js
at Function.Module._resolveFilename (internal/modules/cjs/loader.js:815:15)
at Function.Module._load (internal/modules/cjs/loader.js:667:27)
at Module.require (internal/modules/cjs/loader.js:887:19)
at require (internal/modules/cjs/helpers.js:74:18)
at Object. (E:\my\umami-master\umami-master\scripts\copy-db-files
.js:2:13)
at Module._compile (internal/modules/cjs/loader.js:999:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10)
at Module.load (internal/modules/cjs/loader.js:863:32)
at Function.Module._load (internal/modules/cjs/loader.js:708:14)
at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js
:60:12) {
code: 'MODULE_NOT_FOUND',
requireStack: [ 'E:\my\umami-master\umami-master\scripts\copy-db-files.js
' ]
}
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this comm
and.
ERROR: "copy-db-files" exited with 1.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this comm
and.
ERROR: "build-db" exited with 1.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this comm
and.

Prisma schema loaded from prisma\schema.prisma
Error: Get config: Unable to establish a connection to query-engine-node-api lib
rary
Prisma CLI Version : 3.15.2
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this comm
and.
ERROR: "build-db-client" exited with 1.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this comm
and.
ERROR: "build-db" exited with 1.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this comm
and.

There is no new version that can fix, as this is a problem on your system. See the instructions on how to fix here: #13717 (comment)

It turns out that cannot generate scheme without root permission in linux. I finally can generate it with root. I don't know how to make it work without root permission.

@aniwn88480857 Your first post looks completely unrelated to Prisma. Your second one is an old version that does not show the full error message, please upgrade to Prisma 4.0.0.
@shtse8 That is not a known limitation, and probably has to do with how you installed your node or application. If the error persists, please open a discussion here in Github and provide more information so Support can take a look. Thanks.

@kaykhan #9975 (comment) check out this solution, it may fix you issue too. install MSVCR on windows..

No~ im on ubuntu 22.04

Same issue in Docker
FROM node:16.13-alpine
...
RUN apk add --update openssl openssl-dev libc6-compat
...
Get Config: Unable to establish a connection to query-engine-node-api library. It seems there is a problem with your OpenSSL installation!

If someone finds a solution for Alpine images – please share.

@ivan-kleshnin There should be further output with the actual error message that provides the details what is going wrong in your case.

@janpio here's the rest of the message, thank you for responding:
#0 347.2 Details: Unable to require(`/usr/src/app/node_modules/prisma/libquery_engine-linux-arm64-openssl-1.1.x.so.node`)
#0 347.2  Error relocating /usr/src/app/node_modules/prisma/libquery_engine-linux-arm64-openssl-1.1.x.so.node:
__res_init: symbol not found
#0 347.2 Prisma CLI Version : 4.1.1
#0 347.2 error Command failed with exit code 1.

Ugh, that one is new. Can you please open a new issue for this and fill the full template? We should try to reproduce this and understand what exactly is going on.

i was having this simple problem, >>> sudo su npx migrate dev....

Ugh, that one is new. Can you please open a new issue for this and fill the full template? We should try to reproduce this and understand what exactly is going on.

While I don't have a template for you unfortunately, I was encountering this error in my CI.
In my project, I build a docker image using node:16-slim as a base image, where I generate my prisma types.
For my runner layer, I was using node:16-buster-slim. I think there's something missing in the node:16-slim image. I have many checks from CI that are green with the same images, but have failed since I started including the prisma files on July 10th.
I wasn't using prisma before then, so I assume that on June 8th this hasn't worked. For docker users, it seems like using node:16 or your favorite flavor of node should do the trick.

Please open new issues with additional information if you still encounter this problem. Thanks.


ERROR:Installation Failed! HRESULT:0x80370102

Not able to install after updating to wsl2.
running windows build 19041.207
1st Attempt - Only enabled WSL & Virtual Machine Platform = Failed
2nd Attempt - Enabled Hypervisor and Tools = Failed
Checked Event Viewer - error with database format feature version?
What can do to get this resolved?

run PowerShell as Admin, Then type this command
Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Windows-Subsystem-Linux

Type yes

Click on the executable and enjoy.

But if you notice my picture i attached FeatureName Microsoft-Windows-Subsystem-Linux is already activated.

But if you notice my picture i attached FeatureName Microsoft-Windows-Subsystem-Linux is already activated.

even tho i had the same problem and that was my fix for it

Is hypervisor enabled?
Please see BIOS settings and enable Intel VT-X/AMD-V.

I could reproduce the issue following those steps:

Download the Centos7.zip file to C:\users\myuser\Download
Unzip it (C:\users\myuser\Download\Centos7)
Execute Centos7.exe ---> works
Remove the folder without clean it
Move to a definitive folder (C:\tools\centos7)
Execute again

You are not able to clean neither install it, showing this error message.

I resolved the issue raised by @FROSADO by following the solution provided by Alword Artyom Slepushko in https://github.com/yuk7/CentWSL/issues/50

Hi... I'm having the same problem and none resolve ... anyone have the solution.... thanks

I could make it work finally.
https://github.com/microsoft/WSL/issues/5964#issuecomment-696619923

I ran the command:
wsl --set-default-version 2
before attempting the install.  After I received the same error you received I toggled back to version 1 and it worked by executing:
wsl --set-default-version 1
<DISTRO.EXE>

Any support for WSL1 users with this error?  I use virtualbox and cant use hyperv, so no wsl 2

I already had Docker and Ubuntu running with WSL2, but my default was set to 1. CentOS7 installed fine as WSL1, and I was able to convert it via wsl --set-version CentOS7 2 just fine.


Change Language & Change Theme

[ ] Change Language
[ ] Change Theme

Assigned to @hadiyarajesh team as Loveleen left the team.


Define the library we want to support first

As a milestone to work towards, we can look at PyPl's most popular package and try to make it work.
So this issue is to investigate which package should we include, based on

popularity
number of dependencies (the less, the easier to make it work)

I'd highly suggest using urllib3. It's among the top 20 most downloaded from PyPI. It is depended on directly or indirectly for a significant number of well-used packages (requests and pip to name a couple from the top 20 downloaded). Its base install has no dependencies, IIRC.

Another suggestion: youtube-dl

Suggested in #344  unittest or pytest.

even as a pytest maintainer, i would propose to go for unittest first, since its used for the stdlib testing, so having it would be of great benefit for the direct development of this python impl

+1 for unittest

unittest is part of the stdlib. This issue seemed to be asking what 3rd-party library to try as RustPython's functionality and stdlib came together. Any of the major stdlib modules use a number of other stdlib modules. In unittest's case, 20 or so from a quick count. Really, it will just be a lot of parallel additions for any of them. A better question as far as the stdlib modules go is the most efficient order in which to add them to make it to the end.

To put it another way: by the time RustPython is ready to handle the stdlib modules with more sprawling dependencies, it will likely also be ready to be testing some major 3rd-party libraries.

@skinny121 could we add here a task list with the missing parts needed for importing unittest? I would love to help in that and it is kind of hard to understand from #824 and #829 what is missing.

@palaviv
The major blockers for importing unittest and it's dependencies are:

printf/percent style formatting
relative imports - which will be fixed by importlib
async/await functionality, mostly for _collections_abc.py
namedtuple's which @youknowone is working on.
_signals module

There is probably a lot of other small things that are also required.

importlib issues: #958
The core rust bugs of namedtuple related issues are fully merged. Now only operator and iterator modules are remained.

operator.py: bringing CPython operator.py is ok. So basically no problem.
iterator: A rust module is working on PR. But when this version has any problem, pure python modules will work too. #824 used ouroboros one.

numpy
pandas
matplotlib (if too complicated, some other chart/graphing libraries?)

The motivation is to support some stuff most commonly used in "data science".

@mireille-raad Are you thinking about the packages in native rustpython or wasm? numpy for rustpython already is a challenging goal, but for wasm will be even more. It means we need cpyext for rustpython and cpyext for wasm.

#1983 is the current major goal

I don't know if HPy will become "The One API to Rule Them All", I hope it does because then the work would be implement an HPy layer. I would suggest focusing on a stdlib or builtin lib that is missing. I've been poking around the CPython's ctypes lib on my free time for a while now, I hope I can learn more about it and make a pure rust version with pyo3 and then contribute to an implementation for RustPython.
With ctypes/mmap I believe it's easier to support things that have binaries, if they don't mess with CPython C API. A big thing would be cffi, though.

numpy would lay the foundation for other libraries like pandas and tensorflow.

#4055 setuptools will be the next goal


ValueError: We don't support continuous values on this synthesizer.  Please discretize values.

is there some discretizer to prefer? How do we deal with data after synthetozation (to transform bins into numerical values again with regard to the correlations)?

Note that we will soon reintroduce support for continuous values on the GAN-based synthesizers.  However, MWEM will still support only discretized value, and we may have future synthesizers that require discretization.
To group numeric items into bins, np.histogram works well.  However, you will want to take care to ensure that bins don't leak private information.  In general, this means that the bin boundaries should be determined in a way that does not require looking at the data.  For example, if the data curator or domain expert has determined that the age column will have values that range only between 0 and 100, you can safely make n bins by dividing the range by the number of buckets desired, and make n equal sized bins.
In the case of something like age, you could plausibly used 100 bins, with each integer age being its own bin.  The key is that the min and max value should be provided independent of the data; i.e. without being derived from the actual data.  For fields with a much larger range, you can still safely bin, as long as you have a way of obtaining a data-independent min and max.  In cases where you don't have a public min/max available, you could compute a differentially private min and max, and then divide that range into equally sized bins.  In this case, you will spend some extra epsilon to safely learn the min and max.
There isn't a one-size-fits-all answer to converting bins back into numerical values.  Some people treat the binning as a pre-processing step, and just use the binned data output from the synthesizer.  If the values in the binned column are uniformly distributed before binning, you can invert them by choosing the midpoint of the bin edges, or by uniformly drawing from between the bin edges.  It's relatively uncommon for data to be uniform in this manner, though, so choosing the midpoint will result in some loss of fidelity (for example, if data are gaussian or exponential such as zipf).
You could plausibly improve on the midpoint if you know that data are roughly exponential or gaussian, but in this case you would want to be careful not to leak private information with the binning/reversing procedure.  For example, let's say that you are binning an income column, and we know without looking at the data that incomes typically follow a power law distribution.  If you choose a "safe" min and max, and divide the incomes into 30 equally-sized bins, the records will be more heavily concentrated in the bins toward the bottom of the income distribution, and more importantly, the records within each bin will be skewed left.  So, showing the midpoint of each bin will cause the final results to be biased upwards.  If you had some estimate of the exponent of the distribution, you could instead choose a point in each bin, slightly to the left, which would a good guess for the midpoint of the mass of each bin (presumably the exponent is consistent across bins, so the point chosen would be the same for each bin).  The key point is that the exponent is itself a summary statistic, and would need to be computed in a differentially private way (or from public knowledge).
Also note that people will sometimes use non-uniform bin sizes when dealing with exponentially distributed data like income.  For example, each bin is double the width of the bin that falls below it.  As long as this increase factor (e.g. 2X, 1.23X, etc.) is derived from public knowledge or computed in a privacy-preserving way, this can give bins that are more balanced in membership.

Tagging @AprilXiaoyanLiu re: continuous

This is fixed in #463


Set up specs with a mock global store

The global store export causes issues between specs
This could be fixed by exporting the Store class, for spec purposes only,
the spec here is just an example of how we could test modules independently outside of main.

I was messing around with specs and ran into issues w/ the global store. This was the best way I found to avoid the issue and make it easier to do independent tests.
This is more of a "look at this request", but maybe it's a good idea?


[ENG-774] A Naive Solution for Waterbutler API Rate-limiting with Redis

Ticket
https://openscience.atlassian.net/browse/ENG-774
Related Tickets
The parent EPIC ticket: https://openscience.atlassian.net/browse/ENG-451
The previous Analysis / POC ticket: https://openscience.atlassian.net/browse/ENG-468
The next ticket: https://openscience.atlassian.net/browse/ENG-775
Purpose
Add an naive solution to rate-limit Waterbutler API V1 using Redis.
Here is the OSF side support for local development: OSF-PR#9199.
DevOps tasks are out of scope for now.
Changes
Preparation

Rate-limiting algorithm: fixed Window.

Here is an article that explains different rate-limiting algorithms as well as their pros/cons.
This same algorithm is used by GitHub, as mentioned in its API Overview -> Rate Limiting.

Maintaining state: using a Redis server

Redis is an ideal choice for rate-limiting. It's fast (in-memory) and simple to use as a dictionary. Here are the commands we use: INCR, EXPIRE and TTL. Here is how a simple rate-limiter is done using Redis.
Server-side:  run it using docker-compose as configured in OSF-PR#9199.
Client-side: use redis-py. Here is the API Docs and here is an example on how to use it.

What and How To Rate Limit

Window size is 1 hour and the limit is 3600 requests
Rate-limit applies to requests based on token auth, basic auth and no auth
Rate-limit does not apply to cookie auth requests (which are sent by the OSF API server)
Redis keys (token, cookie and base64 encoded username/password and IP address) are hashed for better security.

What's not included in this naive solution

What will WB do if Redis is down or Redis call returns an exception?

Naive: WB catches it and returns a 503

Applies stricter limits to no-auth and less strict ones to token and basic auth

Naive: same limit for all

Return X-WB-RL headers for every request so benign API users can back-off

Naive: only when a request hits the limit

Remove INFO level log

Naive: INFO level log are kept for convenience

Side effects
Please refer to Changes
Dev / QA Notes

assert needQa is False
The flag is turned off by default to make Travis happy

Deployment Notes

Merge into a feature branch instead of develop or next release
This feature should not go to production

Coverage decreased (-0.7%) to 91.1% when pulling<PHONE_NUMBER>01d2c5792b7c3d9ed9508097f47dcd on cslzchen:feature/wb-rate-limiting-with-redis into 903f8459e815bb16a934885c1c439f9a4318d1e9 on CenterForOpenScience:develop.

LGTM!  Added some minor changes / docs / refactors, but nothing earth-shattering.  Shall merge now!


update VulkanMemoryAllocator to 2.1.0

I've followed this guide
step by step carefully. **[Yes

I've tested this package remotely and have excluded all broken builds.
Here is the links to the Travis/AppVeyor with status "All passed":

https://ci.appveyor.com/project/jhs67/hunter-vfmrx/build/1.0.26
https://travis-ci.com/jhs67/hunter/builds/85918834

Testing:

https://ci.appveyor.com/project/ingenue/hunter/build/1.0.4438
https://travis-ci.org/ingenue/hunter/builds/433144997?utm_source=github_status&utm_medium=notification

Uploading:

https://ci.appveyor.com/project/ingenue/hunter/build/1.0.4439
https://travis-ci.org/ingenue/hunter/builds/433151599?utm_source=github_status&utm_medium=notification


How to use utilityProcess in this template?

I'm trying to use the utility process for some background jobs but having trouble getting it to work using the example.
How do I tell vite config to include another file that will be used to spawn the utility process?

I'm sorry for replying so late. Can you provide an example of using it as a minimum? I haven't tried using it before, so I don't know how to meet your needs. Thank you.

vite.config.ts
// Use Vite multi-entry
export default {
  plugins: [
    electron({
      entry: {
        // Main-Process entry file of the Electron App.
        main: 'electron/main.ts'
        test: 'electron/test.js',
      },
    }),
  ],
}

electron/main.ts
// Main process
const { port1, port2 } = new MessageChannelMain()
const child = utilityProcess.fork(path.join(__dirname, 'test.js'))
child.stdout.on('data', (data) => {
  console.log(`Received chunk ${data}`)
})

vite.config.ts
// Use Vite multi-entry
export default {
  plugins: [
    electron({
      entry: {
        // Main-Process entry file of the Electron App.
        main: 'electron/main.ts'
        test: 'electron/test.js',
      },
    }),
  ],
}

electron/main.ts
// Main process
const { port1, port2 } = new MessageChannelMain()
const child = utilityProcess.fork(path.join(__dirname, 'test.js'))
child.stdout.on('data', (data) => {
  console.log(`Received chunk ${data}`)
})

Thanks, I'll try this out.


Multi-Arch DockerHub builds [🚀]

Following the blog post by @everflux in https://www.trion.de/news/2019/10/14/docker-multi-arch-dockerhub.html I've implemented a multi-arch Docker build process for Juice Shop. There are some unsolved issues and some missing functionality still, but I'm not sure my Docker skills are sufficient to address these.

:bug: The snapshot-arm64v8 tag is downloading the x64 modules for iltorb, libxml2js and sqlite3 because it seems to think it is running on an x64 architecture. This is a dealbreaker as the resulting image simply cannot work on e.g. a RasPi3 or other ARM-chip computer.
💡 Currently the manifest is only configured for the snapshot builds from develop branch. If 1. gets eventually fixed, building latest from master and vX.Y.Z from Git tags would be needed. @everflux recommended to stick to one manifest file and replace the tag names e.g. via sed in the pre_build hook.

The additional hooks, Dockerfile for Arm64v8 and manifest file are available on https://github.com/bkimminich/juice-shop/tree/develop for anyone who wants to assist in this endeavor.

I guess the best / most stable thing to do is do run the docker builds on travis, as we actually have access to ARM CPU's there.

So, introducing a new build stage "docker" and once that works switch off the Automated Builds on DockerHub?

  - stage: docker
    if: "(branch = develop) AND NOT type = pull_request"
    install: docker build --build-arg VCS_REF=`git rev-parse --short HEAD` --build-arg
      BUILD_DATE=`date -u +”%Y-%m-%dT%H:%M:%SZ” -t bkimminich/juice-shop:snapshot-amd64
      .
    script:
    - echo "$DOCKER_ACCESS_TOKEN" | docker login -u bkimminich --password-stdin
    - docker push bkimminich/juice-shop:snapshot-amd64
    after_success:
    - curl -Lo manifest-tool https://github.com/estesp/manifest-tool/releases/download/v0.9.0/manifest-tool-linux-amd64
    - chmod +x manifest-tool
    - "./manifest-tool push from-spec multi-arch-manifest.yaml"

This attempt covers the regular Docker build plus

injecting the Git ref/date meta data (previously done via pre_build hook)
login to DockerHub and pushing the image
Downloading the manifest tool and pushing it to DockerHub (previously via post_push hook)

All that currently in a separate stage (which limits it to the default matrix job (Node 14 on Amd64 Linux) and only for develop branch. Once this works, it could be extended to also run on master, tag pushes and finally on Node 14 on Arm64 matrix job.

The arm64 builds have terrible performance on Travis-CI unfortunately, so bad that it often doesn't even finish to install the Docker service before the default 10min timeout. Unfortunately I cannot influence the timeout, because this happens before any build step that I could prefix with travis_wait 30 ...
Fingers crossed for https://travis-ci.com/github/bkimminich/juice-shop/builds/170994720 to make it ...

If it keeps failing I might have to install Docker manually as a before_install step, but having to do a thing like that would kind of hurt my feelings... 😢

Mh thats weird.
This is just on arm, right?
Have used travis many times before to build docker images, always worked without any problems, but never on arm. Would have hoped it would just work 😕

On amd64 the entire build takes 10min, which makes it even a lot faster than it was on DockerHub... Only arm64 is causing trouble. I switched to manually installing Docker and try to increase the timeout for the long running steps on ARM now...

Hell, this is frustrating... while most builds just timed out before even doing anything this one at least got the server-side built successfully and then timed out before/while doing the Angular compile step... 😭 ... https://travis-ci.com/github/bkimminich/juice-shop/jobs/348091021

Hell, this is frustrating... while most builds just timed out before even doing anything this one at least got the server-side built successfully and then timed out before/while doing the Angular compile step... 😭 ... https://travis-ci.com/github/bkimminich/juice-shop/jobs/348091021

I've previously had problems with the angular build on my raspberry.
I think the problem is the --max-old-space-size=8192 in https://github.com/bkimminich/juice-shop/blob/master/package.json#L55
This seems to tries to allocate 8gigs of ram. When i removed this flag the build worked flawlessly on my raspberry. Might be a similar problem here.

I removed the RAM allocation completely from the build script in package.json, but the job still times out... I'm giving it one more try with travis_wait but then I'm out of ideas.

I'm officially out of ideas... ❌

But: Amd64 builds work like a charm, though, so even for that it might be worth switching away from DockerHub auto-builds.


Failed to register account provider

Azure Data Studio Version:
Version: 1.18.0
Commit: 9ca6200018fc206d67a47229f991901a8a453781
Date: 2017-12-15T12:00:00.000Z
VS Code: 1.45.0
Electron: 7.2.2
Chrome: 78.0.3904.130
Node.js: 12.8.1
V8: 7.8.279.23-electron.0
OS: Darwin x64 19.4.0
Steps to Reproduce:

Startup Azure Data Studio
Open developer  tools -> console.

The following error is seen:
/Users/arvranmac/git/ads/node_modules/zone.js/dist/zone-node.js:2280 [Extension Host] Failed to register account provider: TypeError: this.metadata.settings.scopes is not iterable (at AzureAccountProviderService.registerAccountProvider (/Users/arvranmac/git/ads/extensions/azurecore/out/account-provider/azureAccountProviderService.js:125:25))
console. @ /Users/arvranmac/git/ads/node_modules/zone.js/dist/zone-node.js:2280
Visually from developer tools console:
PastedGraphic-1.png

Are you seeing this issue in insiders?

i observed it on a dev build but I am ~2 days behind the master.

I'm not seeing this in the insiders build or my dev environment.  Please reactive if you see this in an official build, otherwise, I suspect it's something with the local dev build.


Allow variable override to work with OS defaults

Fixes #18
Avoids warning printed when server-specific nrpe plugins directory doesn't exist.
Avoids looping with the copy module when recursive copy works fine.
Allows nagios_nrpe_server_plugins_src_dir to be overridden so this is useful when deploying module from galaxy.

Nice one, thanks Baggy!


Add mechanism to show that field was saved

For example:

if _handleFormUpdates run successfully, add wos-saved class
if _handleFormUpdates errors, add wos-error class and trigger an event named options-sync:error on the field
when user starts typing, remove both classes

Invisible form auto-saving is great and "native", but feedback is good when things don't work well.
Maybe just the error needs to be handled.

When the save is successful, a options-sync:form-synced event is fired on the form

For GitHub:basecampy:

if _handleFormUpdates run successfully, add wos-saved class

-– if _handleFormUpdates {fix _ error'string< add <wos-mecy890413< class and trigger an event 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 options-sync:wos on the field

when user starts, remember both classes

every property's owned form auto-saving is great and "C#";"native"; but feedback is good when things done work well.
almighty I*.YAIR IVAN MEDINA COTA {fix_ 'string} just the error'string message letting needs… to be handled by "c001879"

Success and failure events are now fired on the field and documented in the readme.
Actually informing the user isn't straightforward because there's no native way to properly do so (particularly on textareas) and I don't want to venture into UI just yet.
I thought about using setCustomValidity on the field, but it shows off screen on large textareas, where it's most needed.
It's unclear what to do when the form stops being syncable due to storage limitations, you should probably show a notification telling the user to limit the size of whatever large field input you have.
Once the storage reaches the limit, even just checking a checkbox might not be saved anymore.

Some code and suggested wording may be found in https://github.com/refined-github/refined-github/pull/8061


Add 'config=scalar' build flag

This allows building codecbench with no SIMD code but still with optimizations enabled. @zeux, there is likely a clearer way to do this so feel free to ask for changes or commit to this branch! The rationale for this is to build an optimized, scalar version of codecbench.

Is the idea to be able to profile native binaries with scalar implementations?

Yeah, it would be nice to see the speedup caused by enabling SIMD but that only makes sense if the baseline (non-SIMD) is actually optimized.

Ok - cool. I was thinking at some point that a configuration option to disable SIMD might be good in case there's any platform or compiler specific issues, so users don't need to edit source code.
It would be good to see a couple changes for this though:

The define should be called MESHOPTIMIZER_NO_SIMD so that it can be declared externally as a config option without clashes
It would be good to, instead of undef-ing existing macros, just wrap the entire autodetection section at the top of vertexcodec.cpp & vertexfilter.cpp in #ifndef MESHOPTIMIZER_NO_SIMD. That way this define can work in cases where, for example, there's a problem with a platform-provided SIMD intrinsic header so even including it is problematic.

That way you also wouldn't need to change the dispatch logic (I think), just the absence of any SIMD_ macros will be enough.

@zeux, updated. Let me know if you want different code style for #if/#endif (I ended up using define(...) since that is what I saw in the rest of the code).

Thanks!


Add playframework example to swagger-api/swagger-samples

This depends on swagger-play first supporting swagger spec 2.0 (see https://github.com/swagger-api/swagger-play/issues/13) since the swagger-samples only maintains examples for 2.0.
Old versions of samples can be seen at https://github.com/swagger-api/swagger-core/tree/92ead0ffc7b36691134b02de6c217f7f8f93d137/samples

There's a branch for this now: https://github.com/swagger-api/swagger-samples/tree/play-2.4

PR for this here: https://github.com/swagger-api/swagger-samples/pull/36

Done.


Add $ref to externalSponsors and internalSponsors

$ref should be added to externalSponsors and internalSponsors in Entitlement Management connected organizations for POST and DELETE.
e.g.
POST /identityGovernance/entitlementManagement/connectedOrganizations/{id}/externalSponsors should be POST /identityGovernance/entitlementManagement/connectedOrganizations/{id}/externalSponsors/$ref
See docs https://docs.microsoft.com/en-us/graph/api/connectedorganization-post-externalsponsors?view=graph-rest-beta&tabs=http.
Related to https://github.com/microsoftgraph/msgraph-sdk-powershell/issues/516.
AB#7825

The docs don't match the metadata.  This is probably an issue with the metadata /cc @pixieofhugs

@peombwa please reopen this if you still think its an issue.


Issue with injected language

Hey, have noticed that if you have a cursor inside string such as (lua file example)
vim.cmd 'stopinsert!'

commanding gcc leads to incorrectly commented out line:
" vim.cmd 'stopinsert!'

I am not quite sure if I would expect it to turn to
vim.cmd '" stopinsert!'

or simply do nothing (perhaps it should comment out injected code, as it would make sense within multilane blocks). But correct behaviour is surely wrong.
(thanks for great plugin!)

This is kinda similar to #135
Could you try to recreate the same with vim.cmd[[stopinsert!]] or vim.cmd('stopinsert!')?

Both work fine, as in, comments out entire line with -- .

As I suspect, the thing with gcc is that it takes the range from the current cusor possition to the end of line and uses that to detects the language.
So with vim.cmd 'stopinsert!', gcc was only able to detect vim and used " as the commentstring. But with vim.cmd('stopinstert!'), the last char i.e ) is Lua so gcc then used ---
This situation, two languages on the same line, will always be ambiguous. And I don't even know whether is this something that needs to be fixed or not.

I am closing this for now. As I don't know a solution that'll work in every case. If anyone figures this out feel free to reply or even raise a PR :)


Will not load on Blender 2.80 on Mac

I am trying to load the add-on for Blender 2.8 running on Mac but I'm getting a error about using international fonts

It's fixed in the development version of cats, use the updater to update.


Easy label

Intermediate
Hard
easy

What issue is this @Shiv10122003

Hi
There is a issue regarding the labels

which being ?


has_filter()/has_action() should also work without callback argument

Native WP has_filter()/has_action() can be called without arguments to check if anything is attached to a hook.
Monkey implementation doesn't allow this and insist on checking for a specific callback. This makes it impossible to assert hook has nothing attached to it.
My use case: I temporarily override time zone option by hooking before calling WP API function and unhooking after. The more thorough/broad assertion is to check that I have no leftovers of any kind hooked.
Check for specific callback is possible but it's more prone to error (for example if I make mistake in my assertion and check for non–existing callback it will always pass).
// works
$this->assertFalse( has_filter( 'pre_option_timezone_string', 'DateTimeZone->getName()', 10, 0 ) );

// InvalidArgumentException: A callback is required to add a filter.
$this->assertFalse( has_filter( 'pre_option_timezone_string' ) );

This is done with 7dcd5e3 and will be part of 1.4.2 release. Thanks for reporting! @Rarst


ci: build all

this will reduce compilation time?

this will reduce compilation time?

I don't think so? Presumably it was being built on the first test only. I guess we'll find out.

cc @digorithm


UHC Voter Bug Fix

Description

This PR fixes the bug introduced when AHR updated its dataset, and it refactors and removes unneeded code.

AHR's dataset has a typo, where the measure name for the 65+ category has an extra space at the end.

This caused a bug as our code looks for the 65+ string without extra spacing. This is specific only to the 65+ voter participation line. The other measure names we use from AHR's dataset do not have this issue.

This PR will replace the measure name on the data frame with the correct format.

Removed get_average_determinate_value function. Suggest that we only display breakdowns for presidential elections so that our data is more uniform since AHR no longer provides demographic breakdowns for midterm elections.

Overall Issue:
When AHR updates their dataset, sometimes they change the spacing in the names of the measures. When we check for matched rows, extra spacing in the measure name will cause the data frame to leave out those rows because we check for string literals. This will create unexpected null values. We should introduce checks to counter this.
Motivation and Context

fixes #1732
How has this been tested?

I tested this locally using pytests and on glcloud.
Screenshots (if appropriate):
This created a null value specifically for the 65+ category:

No more demographic breakdowns for the Midterm section:

Types of changes

Bug fix (non-breaking change which fixes an issue)

@eriwarr Good work Eric.
I think we still need the get_average_determinate_value function as it is used to generate a single voter_participation value for each demographic group. The way the function works is that if there are two values it averages them, otherwise, like in the case for 65+ it only uses the one value that we have.

@eriwarr Good work Eric.
I think we still need the get_average_determinate_value function as it is used to generate a single voter_participation value for each demographic group. The way the function works is that if there are two values it averages them, otherwise, like in the case for 65+ it only uses the one value that we have.
The thing is, the tests should be failing, I am not really sure why they still passed without the function

Thanks, Josh! I think the tests are still passing because the calculation isn't done until after the row is matched with the demographic group. The function was already bypassing missing midterm values, so I just removed the function altogether and only used the presidential values.  I can add the function back to the All portion of the code, but I think it could still be left off the demographic breakdown section. However, I could see the argument for keeping it there if AHR  decides to include demographic midterms breakdowns in the future.

@eriwarr Good work Eric.
I think we still need the get_average_determinate_value function as it is used to generate a single voter_participation value for each demographic group. The way the function works is that if there are two values it averages them, otherwise, like in the case for 65+ it only uses the one value that we have.
The thing is, the tests should be failing, I am not really sure why they still passed without the function

Thanks, Josh! I think the tests are still passing because the calculation isn't done until after the row is matched with the demographic group. The function was already bypassing missing midterm values, so I just removed the function altogether and only used the presidential values. I can add the function back to the All portion of the code, but I think it could still be left off the demographic breakdown section. However, I could see the argument for keeping it there if AHR decides to include demographic midterms breakdowns in the future.

I see what you mean, Yeah I think we should add it back for the ALL calculation.


lcd: Fix npixels parameter in calls to getrun/putrun

Summary
This PR intends to fix a regression in LCD Character Device Driver introduced in #6837.
The npixels parameter in some calls to getrun and putrun was being passed in number of bytes, but the correct should be the number of pixels.
Impact
Fix for applications relying on the LCD character device driver, e.g. LVGL demos.
Testing
esp32-wrover-kit:lvgl now properly executing the lvgldemo application.

Thank you, @gustavonihei . Actually, it affects the LCD display drivers that don't implement the get/putarea method. Thanks for noticing that!


Allegheny County availability of translation services in police stops

Someone has requested help on a data project:
Submission notes
Are there data on the actual availability of translation services in police stops?
Location
Allegheny County, PA
Requirements
Data location and analysis
How to help
Leave a comment if you'd like to help. Do you have ideas, questions, or need any more information to get started?

There doesn't seem to be a source of information about how available translation services are.


Get all grant receipt events between project wallets and grantmaking addresses

See here for list of tracked grantmaking addresses on both Ethereum and Optimism.
See also this issue about getting the projects' Ethereum addresses

https://github.com/hypercerts-org/hf-private/issues/31

Related to #149


Expose button style for customisation

Issue:
What I did
Cannot modify button.topRight style from storybook config.js
example:
import { setDefaults } from '@storybook/addon-info';
import deepmerge from 'deepmerge';

setDefaults({
  styles: (defaultStyles) => {
    return deepmerge(defaultStyles, {
      button: {
        topRight: {
          right: 'auto',
          borderRadius: '0 0 5px 0',
          left: 0,
        },
      },
    });
  },
});

How to test
Is this testable with jest or storyshots?
Does this need a new example in the kitchen sink apps?
Does this need an update to the documentation?
If your answer is yes to any of these, please make sure to include it in your PR.

Thanks, totally makes sense. Can you please do the same in rest of the places in this component?
https://github.com/baktiaditya/storybook/blob/961e1b91439bb42d4c2ccc5a4dd108660a683cd9/addons/info/src/components/Story.js#L159
https://github.com/baktiaditya/storybook/blob/961e1b91439bb42d4c2ccc5a4dd108660a683cd9/addons/info/src/components/Story.js#L218

@Hypnosphi ok, I'll inform you again when it's done

@Hypnosphi 👍 done


Added incremental MenuComplete support

#485
Add new Option and support for Incremental MenuComplete (Default = $false for seamless intergration)
Mimic PowerTab behaviour on MenuComplete - Enter finishes completition, Space finishes completition and add a space after.

This is an option mostly because it change behavoiur of Enter key - it can be a pain for other users ( not me )
I use point and mark because it a common way to suggest things in other editors and because every cursor move drop this mark, but I agree that saving it before and restoring after will be a good thing
and yes, I still can't feel that I fully understand how Tab Completition work and I decided: lets PSReadLine think for me :) I will see what I can do with it with minimal changes

Still leave Option intact because of reasons above. If other changes will be approved it can be removed faster than inserted :)
Still use mark for suggestions because can't find how original version can use it, Marking always dropped after using menu.
Now CompletionMatches filtered by remove because of strange error with CommandCompletion constructor. This code stay commented out.

@lzybkr , rfc for latest commit :)
mostly I want to know your opinions about commented out code with new CommandCompletion(), why the compiler say 'Invalid Constructor'
the powershell can create such object
D:\> $filtered = New-Object 'System.Collections.ObjectModel.Collection[System.Management.Automation.CompletionResult]'
D:\> $compl = New-Object System.Management.Automation.CommandCompletion $filtered,0,0,10
D:\> $compl
CurrentMatchIndex ReplacementIndex ReplacementLength CompletionMatches
---------------- ---------------- ----------------- -----------------
               0                0                10 {}
D:\>

Sorry, I've been pretty busy and haven't had a chance to look closely at this yet.
If you are getting a compiler error, it's most likely because I build against the PowerShell V3 reference assemblies and a new constructor was added since then.
I will consider dropping V3 support in the next release though - because I want to make the module portable between Windows PowerShell and PowerShell Core - and that means I'll need to require .Net 4.5 at a minimum.  Windows PowerShell V3 does not require .Net 4.5, so it might be confusing if PSReadline started requiring a newer runtime.

latest commit use CopyTo() instead of building new CommandCompletion()
this is more universal and does not use expensive Remove()
may be tomorrow I try to save/restore user mark
and what with option ? remove it ?

I strongly prefer no option - so we'll refine the experience so it feels natural.

here it is

There still problems with keys like Home/End/PgUp/PgDn.
I don't know, may be I need to rework current concept into handling all keys inside PossibleCompletionsImpl() without leaving until Enter/Space/Esc pressed and so without handling dispatchTable handlers ?

I hope now I foresaw all the cases with completions filtering...

@lzybkr , please tell what story lies behind consistentQuoting code in Completion.cs around line 150+
It does not work for the case when & inserted before CompletionText
PS C:\Users\User> $r = [System.Management.Automation.CommandCompletion]::CompleteInput('.\nt',1,$null)
>> $r.CompletionMatches
CompletionText                                                                                ListItemText
--------------                                                                                ------------
.\NTUSER.DAT                                                                                NTUSER.DAT
& '.\NTUSER.DAT{d51862bb-b663-11e6-9335-dfe3e1845d6f}.TM.blf'                                      NTUSER.DAT{d51862...
.\ntuser.ini                                                                                ntuser.ini

I would like to add similar feature inside IncrementalCompletion (Tab advance IncrementalCompletion to the longest unambiguous prefix) and reuse the code but not fully understanded why it was implemented in a such way

IIRC, the scenario was cd c:\Pro<TAB>.
Without that code, the unambiguous prefix was the empty string because a directory C:\ProgramData which did not need quotes, and C:\Program Files which did.
It seems I missed the case where & is also added when the completion is a command name.

I see but why you calculate consistentQuoting itself  instead of just
var replacementText = GetUnquotedText(firstResult.CompletionText, false); ? some speedup ?
what if I correct IsQuoted for & " case as I did in my code block ?

@lzybkr , while I improving my implementation I discovered incorrect menu displaying with completions text longer than buffer width. For example long file names.
Among bug in ShowTooltips code branch which I fixed there is weird menu display in both modes and screen pollution.
What your opinions on my code ?
Should I try to fix menu displaying thru shortening text of menu items( ... in the middle) or thru recalculating menu rows count ?

Sorry I haven't been too responsive lately, just busy on other things. I also have some significant work in progress on PSReadline to rewrite the rendering so it's fast on Linux and handles resized windows correctly, and I'm avoiding thinking about how your changes will impact that work.
I just built your changes and tried it for the first time, great first impression, I'm now very motivated to get this merged.
Regarding the consistent quotes code - I don't remember specifics - the important part is the user experience. I'll play with your changes and if I don't notice problems, we'll call it good.
I'm totally open to big changes to the ShowToolTips code path - what I have now was quick and easy to see if it was useful. I think it's probably not too useful because the tip gets truncated or worse too often.
Showing the tooltip for just the current selection either above or below the menu is probably worth trying - maybe with a different background. Then we would always show the tooltip or possibly after a short delay like what happens in editors like the ise.

First of all I'm interested in the opinion of processing long names, what to do with them. Tips for long names now do not cause a fall - and this is the main thing :)

After playing with what you have right now, truncation is promising. The menu doesn't crash and is somewhat useless, but because you can see the expanded filename for the selected item, it actually works just fine.
So I'd go with truncation in the menu - it seems the simplest to implement, and should be a fine experience too.

Thanks for your comments, I'll try to implement it on then next week, I think, right after shortening long names.
btw, How you think, in current implementation, may be I should turn off Tooltips when it will be shorter than 5 symbols ? they are still useless

I did hit a small usability issue - when I type something that doesn't match, I wonder if we should assume that's what they wanted.
Before your changes, an unexpected key accepted the current selection and was applied as though the key was pressed after the selection completed

if (nextKey == Keys.Delete)

I'm not sure this case is a good idea.

my way simplifies _mark restoration :)
I try to implement it other way, more closely to previous behavior. But I want to keep 'Delete' code, may be on the other chord, because sometimes you already typed loooong command and want to keep it with small change and automatically use  completion... you can type space+backspace, but it happen so much...

When completing methods, maybe we should accept '(' as an ending character.

This block widely open for additions.

Maybe I'm not normal, but I use End a lot when I'm done with the menu

Yes, after implementing this I also realize than I use End more often to navigate to end of line, than to end of menu :)

I'm building with C# 7 now, so maybe introduce a local function so you can avoid the ContainsKey call and just use TryGetValue instead.

I'm not sure I'm familiar with these changes. Can you show some sample anywhere

           //consider possible '& ' prefix

A space is not required, e.g. I could have typed &C:\Pro. I doubt people do that much though.

No, '& ' can appear only in CompletionMatches text. and it always have space. If user typed it yourself it not counted in completions
PS C:\> $r = [System.Management.Automation.CommandCompletion]::CompleteInput('C:\Pro',6,$null)
PS C:\> $r.CompletionMatches

CompletionText             ListItemText               ResultType ToolTip
--------------             ------------               ---------- -------
& 'C:\Program Files'       Program Files       ProviderContainer C:\Program Files
& 'C:\Program Files (x86)' Program Files (x86) ProviderContainer C:\Program Files (x86)
C:\ProgramData             ProgramData         ProviderContainer C:\ProgramData

PS C:\> $r = [System.Management.Automation.CommandCompletion]::CompleteInput('&C:\Pro',7,$null)
PS C:\> $r.CompletionMatches

CompletionText           ListItemText               ResultType ToolTip
--------------           ------------               ---------- -------
'C:\Program Files'       Program Files       ProviderContainer C:\Program Files
'C:\Program Files (x86)' Program Files (x86) ProviderContainer C:\Program Files (x86)
C:\ProgramData           ProgramData         ProviderContainer C:\ProgramData

PS C:\> $r = [System.Management.Automation.CommandCompletion]::CompleteInput('&"C:\Pro',8,$null)
PS C:\> $r.CompletionMatches

CompletionText           ListItemText               ResultType ToolTip
--------------           ------------               ---------- -------
"C:\Program Files"       Program Files       ProviderContainer C:\Program Files
"C:\Program Files (x86)" Program Files (x86) ProviderContainer C:\Program Files (x86)
"C:\ProgramData"         ProgramData         ProviderContainer C:\ProgramData

else if (nextKey == Keys.Tab)

Could you also add comments to this code? I'm not sure what's going on, and whatever the intent, I'm confused because I don't see similar code in the Shift+Tab case when if feels like the inverse probably needs similar treatment.

I try to mimic <Tab> behavior like in CompleteImpl() when first  completes until first common prefix
So I can for example type Get-ProcessOwner like Get-Pr<tab>oc<Tab>o<space>
I doesn't implement it in <ShiftTab> because it never used in a such way

I am wondering if you're trying to fix a long-standing bug where I show two identical items in the menu but they have different replacement text. If that's the case, this doesn't feel like the right fix.

Not sure which bug you mean
Now It works when Completions constructed from different sources like commands from different modules:
PS C:\> $r = [System.Management.Automation.CommandCompletion]::CompleteInput('get-priv',1,$null)
PS C:\> $r.CompletionMatches

CompletionText     ListItemText  ResultType ToolTip
--------------     ------------  ---------- -------
Get-Privilege      Get-Privilege    Command ...
Pscx\Get-Privilege Get-Privilege    Command Get-Privilege...

or for cases when user type some text and completions have this text not from the beginning of a line

PS C:\> $r = [System.Management.Automation.CommandCompletion]::CompleteInput('pro',1,$null)
PS C:\> $r.CompletionMatches

CompletionText                 ListItemText               ResultType ToolTip
--------------                 ------------               ---------- -------
& '.\Program Files'            Program Files       ProviderContainer C:\Program Files
& '.\Program Files (x86)'      Program Files (x86) ProviderContainer C:\Program Files (x86)
.\ProgramData                  ProgramData         ProviderContainer C:\ProgramData

@lzybkr, I think that's all I can do without your attention.

Thanks for the update - I've been using your latest today and so far it seems pretty good. I'll review the code more closely when I get a chance.

I might have hit a bug in your changes - see if you can repro:
Exception:
System.Management.Automation.PSArgumentNullException: Cannot process argument because the value of argument "completionText" is null. Change the value of argument "completionText" to a non-null value.
   at System.Management.Automation.CompletionResult..ctor(String completionText, String listItemText, CompletionResultType resultType, String toolTip)
   at Microsoft.PowerShell.PSConsoleReadLine.PossibleCompletionsImpl(CommandCompletion completions, Boolean menuSelect)

The single line where new CompletionResult() created - its line 745.
and I can repro this:  Get-Process <Tab><Backspace><Backspace>
It happen when you exit from Completion loop through  double <backspace> with empty completion.
I think it can be fixed but fix will be not easy as it should :)
In this case I call DoReplacementForCompletion(replacement, completions) with empty replacement text. And seems it can't be created by standard method. May be it is need to write overload for DoReplacementForCompletion with first parameter as string or make direct call to Replace() (it exiting loop anyway)

After I slept with this thought, I realized that for the user its look like undo, so I make it through undo.

Thanks for all the fixes, it's probably getting close to be ready to merge.
I hit another minor issue, it's a little weird. The cursor is moving outside the command line.
To reproduce:
PS> 1..100 # Doesn't reproduce at the top of the screen
PS> Set-PSReadlineOption -EditMode Emacs
PS> [pscred<TAB><CTRL+SPACE>

When you've done this, the screen will look like:
PS C:\Users\jason> [pscred
PSCredential         PSCredentialTypes      PSCredentialUIOptions
PS C:\Users\jason> [pscredential
PSCredential         PSCredentia**l**Types      PSCredentialUIOptions

And the cursor will be on the l that I added ** around. (I'd paste a screen shot, but the cursor disappears when the window doesn't have focus, and any keybinding I try to get the screen shot also changes the focus to lose the cursor.
I don't see any real problems with this bug, but it's weird enough I'd like it fixed.

I see. The original version have similar bug with end of buffer.
I tried to fix it but seems broke other case :)

What you think about double call to GetUnambiguousPrefix() ?
If you accept it as solution for PS C:\>prog<<CTRL+SPACE> cases may be I should do it inside method itself and eliminate double call ?
It change nothing in my code but extend Unambiguous search style to CompleteImpl() method (Emacs  completion, line 202 )
For example if I have program.exe somewhere in search path, CompleteImpl() doesn't find program prefix as ambiguous but MenuComplete find it and extend prog to program

That does sound like an improvement - I think I'm fine with that.

How it works now
PS C:\> Set-PSReadlineOption -EditMode Emacs
PS C:\> prog<tab> #Ding here and line changed to
PS C:\> Program
PS C:\> #Second <tab>
Program Files        - C:\Program Files
Program Files (x86)  - C:\Program Files (x86)
ProgramData          - C:\ProgramData
program.exe          - C:\Windows\system32\program.exe

the only thing that can be named 'prblem' is that first letter case changes to upper

I think this is ready to merge, but I'd like the history cleaned up a bit.
If you're fine with me squashing and editing the commit message as I see fit, then I'll go ahead and do that, but I thought I'd give you a chance to do that before I merge.
If you feel like adding some tests, that'd be nice too, but I'd merge without tests, this functionality will be heavily used so regressions would be quickly noticed.

I'm fine with commit messages editing.
Basically they are needed in the process of coding and not all have any long-term meaning, especially early.
About the tests, to be honest, I do not know how to write them correctly in this case.

about last fix:
PS C:\>Get-PR<tab>
Get-Privilege
...
Get-Process
...
PScx\Get-Privilege

with 'optimization', ListItemText comparing starts on last item, and return Get-Pri prefix

Thanks so much for your contribution and your patience with me, this will be a popular feature.
I don't have many completion tests (partly because it's a bit painful), see https://github.com/lzybkr/PSReadLine/blob/master/UnitTestPSReadLine/CompletionTest.cs. There is a useful sort of dsl to make writing tests easier, but it's still tedious for the menu completion. I'll gladly accept a future PR with just tests if you're so inclined.

@lzybkr , I tried to write a small test, but I was stuck with exception Shouldn't call ReadKey when there are no more keys, seems, because of menu cycle. I should mock ReadKey inside it, but it is beyond my current abilities :)
I never wrote such tests before.
If you show me how can I move forward, I will add some tests, which I'm doing manually now.


ProcessGroupNCCL.cpp, unhandled system error, Software caused connection abort

File "/opt/conda/lib/python3.10/site-packages/torch/distributed/distributed_c10d.py", line 1906, in broadcast 10: work = default_pg.broadcast([tensor], opts) 10: torch.distributed.DistBackendError: NCCL error in: /opt/conda/conda-bld/pytorch_1696588393313/work/torch/csrc/distributed/c10d/ProcessGroupNCCL.cpp:1331, unhandled system error (run with NCCL_DEBUG=INFO for details), NCCL version 2.18.5 10: ncclSystemError: System call (e.g. socket, malloc) or external library call failed or device error. 10: Last error: 10: socketStartConnect: Connect to <IP_ADDRESS><45069> failed : Software caused connection abort

Print the NCCL env vars and compare against this list:  https://github.com/aws-samples/awsome-distributed-training/blob/main/1.architectures/efa-cheatsheet.md
If you are using a p5.48xlarge (H100) cluster instance type, you can specify the following to fix this issue:
NCCL_SOCKET_IFNAME=enp

See https://github.com/aws-samples/awsome-distributed-training/pull/168/files for more information.


Build not installing on emulator or on device

My emulator is open and in console it is showing build successful but it is not installing build on emulator. I even connected my phone to check if issue is with emulator but same issue with phone, It is just stuck after the following message.

Hi, did you find anything suggesting it's a problem with this library?


Not an issue but a SUCCESS!

GitHub doesn't have a congratulations or great work tab. So forgive me for posting here. Keep up the good work.

S22 Ultra on AVHF

S22 Ultra on AVF1.

Glad to hear that!


[SPARK-33354][DOC] Remove an unnecessary quote in doc

What changes were proposed in this pull request?

Remove an unnecessary quote in the documentation.
Super trivial.
Why are the changes needed?

Fix a mistake.
Does this PR introduce any user-facing change?

No
How was this patch tested?

Just doc

Merged to master and branch-3.1.


Fix composer command README 2.0

Changes composer install to the correct composer require

Hi @Iandenh this branch is maintained from the mono-repo and edits there will overwrite this. I can edit is there or you can send a PR on that branch if you like. Let me know, it's all the same to me 👍

Thanks for the feedback I made a PR on the main repo:
thephpleague/flysystem#1240
Saw that the same typo was on more README files.

Thank you! It's merged 👍


Link "Configure tables and attributes for auditing" leads to page not found

[Enter feedback here]
The link with the text "Configure tables and attributes for auditing" leads to a 404 page not found.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: a5c391f2-d677-6d8b-6bfc-aac9796670f4
Version Independent ID: 09fe5f18-2a69-2c05-2a86-5b243a75bd9e
Content: Work with any data - Power Apps
Content Source: powerapps-docs/maker/data-platform/work-with-any-data.md
Service: powerapps
Sub-service: dataverse-maker
GitHub Login: @matp
Microsoft Alias: matp

@kannan-balasubramanian - Thank you for pointing this out. The link has been updated to point to the correct article on configuring auditing for Dataverse table records.
Closing issue.


Added link in README to countdown project

I added a link to my countdown project in the Bubble Tea in the Wild section of the README.

Oh this looks great. Thanks for the PR!


Unable to run Kubevela in Kubernetes v1.29

Describe the bug
I am trying to run Kubevela in Kubernetes v1.29, during which even though the vela-core pods start but the errors continue for all the CRDs resources.
To Reproduce

Install kubevela on Kubernetes v1.29 using 'Kubernetes' installation documentation.
During the installation, vela-core pod logs exhibit the above error.
See error

...
revison.go:349] "Could not create DefinitionRevision" err="the server could not find the requested resource (post definitionrevisions.core.oam.dev)"
traitdefinition_controller.go:64] "Reconcile traitDefinition" traitDefinition="vela-system/sidecar"
...

Expected behavior
There should be no error related to CRDs and the resources should be available for usage.
Screenshots

KubeVela Version
CLI Version: 1.9.10
Core Version: 1.9.10
GitRevision: 24756fc507665bff5cbd6108e63391a3e50fa96b
GolangVersion: go1.22.1

Cluster information
System Info:
  Machine ID:                 7d6394eb-ac2b-4be8-88fe-26e629db961d
  System UUID:                7d6394eb-ac2b-4be8-88fe-26e629db961d
  Boot ID:                    e44882f7-7c17-4f08-ac48-179e55cd2ed0
  Kernel Version:             6.6.12-linuxkit
  OS Image:                   Docker Desktop
  Operating System:           linux
  Architecture:               arm64
  Container Runtime Version:  docker://25.0.2
  Kubelet Version:            v1.29.1
  Kube-Proxy Version:         v1.29.1

Additional context
The latest Kubevela version seems to be limited to Kubernetes v1.26 which might be the root of this issue. Need further information about how Kubevela can be upgraded to Kubernetes v1.29.

Same error here.

Kubevela officially support Kubernetes cluster >= v1.19 && <= v1.26. that's why it is not working properly in 1.29

I was able to reproduce the issue performing the installation using vela install on Kubernetes v1.26.0, v1.26.14, v1.27.11, v1.28.7, and v1.29.2. And it seems that the issue may be on the vela CLI itself as using the helm approach on v1.29.2 seems to be working. I will suggest trying this approach while the issue is investigated on vela:
helm repo add kubevela https://charts.kubevela.net/core
helm repo update
helm install --create-namespace -n vela-system kubevela kubevela/vela-core --wait

$ k apply -f basic_app.yaml
application.core.oam.dev/first-vela-app created

$ k get pods
NAME                              READY   STATUS    RESTARTS   AGE
express-server-565ccffd86-lg2qh   1/1     Running   0          9s

$ k get app
NAME             COMPONENT        TYPE         PHASE     HEALTHY   STATUS      AGE
first-vela-app   express-server   webservice   running   true      Ready:1/1   12s

Hi @dhiguero,
I tried the above approach by using Helm earlier. The vela-core app was failing with the same error. I tried to install kubevela again using Helm commands that you mentioned above (adding below) in a fresh namepsace velaa.  I am currently using Kubernetes v1.29.1 (installed using Docker Desktop).
helm repo add kubevelaa https://charts.kubevela.net/core
helm repo update
helm install --create-namespace -n velaa kubevelaa kubevela/vela-core --wait

The vela-core pods is in Running state as in the screenshot above. However, the pod logs represent a different scenario with same errors again.
I0413 09:58:07.297014       1 componentdefinition_controller.go:64] "Reconcile componentDefinition" componentDefinition="velaa/worker"
E0413 09:58:07.297537       1 revison.go:349] "Could not create DefinitionRevision" err="the server could not find the requested resource (post definitionrevisions.core.oam.dev)"
I0413 09:58:09.065582       1 policydefinition_controller.go:62] "Reconciling PolicyDefinition..." Name="apply-once" Namespace="velaa"
I0413 09:58:09.065750       1 policydefinition_controller.go:62] "Reconciling PolicyDefinition..." Name="envbinding" Namespace="velaa"
I0413 09:58:09.065950       1 policydefinition_controller.go:62] "Reconciling PolicyDefinition..." Name="garbage-collect" Namespace="velaa"
I0413 09:58:09.067557       1 policydefinition_controller.go:62] "Reconciling PolicyDefinition..." Name="override" Namespace="velaa"
E0413 09:58:09.075152       1 revison.go:349] "Could not create DefinitionRevision" err="the server could not find the requested resource (post definitionrevisions.core.oam.dev)"
E0413 09:58:09.076257       1 revison.go:349] "Could not create DefinitionRevision" err="the server could not find the requested resource (post definitionrevisions.core.oam.dev)"
E0413 09:58:09.076729       1 revison.go:349] "Could not create DefinitionRevision" err="the server could not find the requested resource (post definitionrevisions.core.oam.dev)"
E0413 09:58:09.077156       1 revison.go:349] "Could not create DefinitionRevision" err="the server could not find the requested resource (post definitionrevisions.core.oam.dev)"
0413 09:58:09.078926       1 controller.go:329]  "msg"="Reconciler error" "error"="the server could not find the requested resource (patch policydefinitions.core.oam.dev override)" "PolicyDefinition"={"name":"override","namespace":"velaa"} "controller"="policydefinition" "controllerGroup"="core.oam.dev" "controllerKind"="PolicyDefinition" "name"="override" "namespace"="velaa" "reconcileID"="4f902650-f898-421d-82ec-ffadc416d267"
I0413 09:58:09.079030       1 policydefinition_controller.go:62] "Reconciling PolicyDefinition..." Name="read-only" Namespace="velaa"
E0413 09:58:09.080864       1 revison.go:349] "Could not create DefinitionRevision" err="the server could not find the requested resource (post definitionrevisions.core.oam.dev)"

Just to test the application resource, I also create a basic app using https://kubevela.io/docs/quick-start/#deploy-a-classic-application.  I am able to see the application created but not able to see anything in the pod list.
% kubectl apply -f basic_app.yaml
application.core.oam.dev/first-vela-app created
% kubectl get velaapp --all-namespaces
NAMESPACE   NAME             COMPONENT        TYPE         PHASE   HEALTHY   STATUS   AGE
default     first-vela-app   express-server   webservice                              2m2s
% kubectl get pods
No resources found in default namespace.

On inspecting further, below are the details when describing the velaapp. It shows no events for the app.
% kubectl describe velaapp first-vela-app
Name:         first-vela-app
Namespace:    default
Labels:       <none>
Annotations:  <none>
API Version:  core.oam.dev/v1beta1
Kind:         Application
Metadata:
  Creation Timestamp:  2024-04-13T10:02:47Z
  Generation:          1
  Resource Version:    1315418
  UID:                 e5149fa4-204a-4ff7-9c88-bca3a55ed35c
Spec:
  Components:
    Name:  express-server
    Properties:
      Image:  oamdev/hello-world
      Ports:
        Expose:  true
        Port:    8000
    Traits:
      Properties:
        Replicas:  1
      Type:        scaler
    Type:          webservice
  Policies:
    Name:  target-default
    Properties:
      Clusters:
        local
      Namespace:  default
    Type:         topology
    Name:         target-prod
    Properties:
      Clusters:
        local
      Namespace:  prod
    Type:         topology
    Name:         deploy-ha
    Properties:
      Components:
        Traits:
          Properties:
            Replicas:  2
          Type:        scaler
        Type:          webservice
    Type:              override
  Workflow:
    Steps:
      Name:  deploy2default
      Properties:
        Policies:
          target-default
      Type:  deploy
      Name:  manual-approval
      Type:  suspend
      Name:  deploy2prod
      Properties:
        Policies:
          target-prod
          deploy-ha
      Type:  deploy
Events:      <none>

Adding further,
I have installed a new Kubernetes cluster in local using kind utility and setup vela installation using Helm commands as in last comment. The error persists for Kubernetes v1.29.2.
kind create cluster --image=kindest/node:v1.29.2 --name=kind29-2

% kubectl version
Client Version: v1.29.3
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.29.2
% kubectl -n velaa get pods
NAME                                         READY   STATUS    RESTARTS   AGE
kubevelaa-cluster-gateway-5bb6f78c64-82rh7   1/1     Running   0          9m18s
kubevelaa-vela-core-5f88c87bff-qtfrq         1/1     Running   0          9m18s

E0413 10:45:53.716150       1 revison.go:349] "Could not create DefinitionRevision" err="the server could not find the requested resource (post definitionrevisions.core.oam.dev)"
E0413 10:45:53.717433       1 controller.go:329]  "msg"="Reconciler error" "error"="the server could not find the requested resource (patch workflowstepdefinitions.core.oam.dev list-config)" "WorkflowStepDefinition"={"name":"list-config","namespace":"velaa"} "controller"="workflowstepdefinition" "controllerGroup"="core.oam.dev" "controllerKind"="WorkflowStepDefinition" "name"="list-config" "namespace"="velaa" "reconcileID"="df5e26cc-846c-4fd6-be3d-faee6c852024"
E0413 10:45:53.721594       1 controller.go:329]  "msg"="Reconciler error" "error"="the server could not find the requested resource (patch traitdefinitions.core.oam.dev resource)" "TraitDefinition"={"name":"resource","namespace":"velaa"} "controller"="traitdefinition" "controllerGroup"="core.oam.dev" "controllerKind"="TraitDefinition" "name"="resource" "namespace"="velaa" "reconcileID"="1d99f3a8-8b9d-4855-8a8f-1ef426a1f2dc"
I0413 10:45:53.811156       1 workflowstepdefinition_controller.go:65] "Reconciling WorkflowStepDefinition..." Name="clean-jobs" Namespace="velaa"
E0413 10:45:53.811872       1 revison.go:349] "Could not create DefinitionRevision" err="the server could not find the requested resource (post definitionrevisions.core.oam.dev)"
E0413 10:45:53.813681       1 controller.go:329]  "msg"="Reconciler error" "error"="the server could not find the requested resource (patch workflowstepdefinitions.core.oam.dev clean-jobs)" "WorkflowStepDefinition"={"name":"clean-jobs","namespace":"velaa"} "controller"="workflowstepdefinition" "controllerGroup"="core.oam.dev" "controllerKind"="WorkflowStepDefinition" "name"="clean-jobs" "namespace"="velaa" "reconcileID"="62eae767-2015-45a9-bb23-ad4b538d9ea9"
I0413 10:45:53.912814       1 workflowstepdefinition_controller.go:65] "Reconciling WorkflowStepDefinition..." Name="build-push-image" Namespace="velaa"

facing same issue for our  setup as well :
% kubectl version
Client Version: v1.28.3
Server Version: v1.28.6-eks-508b6b3

helm list -n vela-system
NAME    	NAMESPACE  	REVISION	UPDATED                            	STATUS  	CHART           	APP VERSION
kubevela	vela-system	1       	2024-04-15 21:57:27.85141 +0530 IST	deployed	vela-core-1.9.10	1.9.10

E0415 16:31:42.806599       1 controller.go:329]  "msg"="Reconciler error" "error"="the server could not find the requested resource (patch traitdefinitions.core.oam.dev ingress)" "TraitDefinition"={"name":"ingress","namespace":"vela-system"} "controller"="traitdefinition" "controllerGroup"="core.oam.dev" "controllerKind"="TraitDefinition" "name"="ingress" "namespace"="vela-system" "reconcileID"="05210090-00a4-4961-9cd3-3d770ab73a2c"
I0415 16:31:42.902780       1 traitdefinition_controller.go:64] "Reconcile traitDefinition" traitDefinition="vela-system/scaler"
I0415 16:31:42.903218       1 workflowstepdefinition_controller.go:65] "Reconciling WorkflowStepDefinition..." Name="apply-component" Namespace="vela-system"
E0415 16:31:42.904018       1 revison.go:349] "Could not create DefinitionRevision" err="the server could not find the requested resource (post definitionrevisions.core.oam.dev)"
E0415 16:31:42.904324       1 revison.go:349] "Could not create DefinitionRevision" err="the server could not find the requested resource (post definitionrevisions.core.oam.dev)"
E0415 16:31:42.907039       1 controller.go:329]  "msg"="Reconciler error" "error"="the server could not find the requested resource (patch traitdefinitions.core.oam.dev scaler)" "TraitDefinition"={"name":"scaler","namespace":"vela-system"} "controller"="traitdefinition" "controllerGroup"="core.oam.dev" "controllerKind"="TraitDefinition" "name"="scaler" "namespace"="vela-system" "reconcileID"="e24d5a21-77d6-4625-832d-ac08a75f0490"
E0415 16:31:42.907214       1 controller.go:329]  "msg"="Reconciler error" "error"="the server could not find the requested resource (patch workflowstepdefinitions.core.oam.dev apply-component)" "WorkflowStepDefinition"={"name":"apply-component","namespace":"vela-system"} "controller"="workflowstepdefinition" "controllerGroup"="core.oam.dev" "controllerKind"="WorkflowStepDefinition" "name"="apply-component" "namespace"="vela-system" "reconcileID"="af342392-8810-4376-9b4e-dbc6f2ad4368"

Adding further,
I have installed a new Kubernetes cluster in AWS and setup kubevela installation using Helm with vela v1.9.9. Apparently everything is fine for Kubernetes v1.29.1 em vela 1.9.9.
KubeVela Version
CLI Version: 1.9.9
Core Version: 1.9.9

% kubectl version
Client Version: v1.28.3
Server Version: v1.29.1-eks-b9c9ed7

% kubectl get pods -n vela-system
NAME                                         READY   STATUS    RESTARTS   AGE
kubevelaa-cluster-gateway-0o47f78c64-64st8   1/1     Running   0          4m11s
kubevelaa-vela-core-as9x87bff-98xo9         1/1     Running   0          4m11s

Additional context
I didn't visualize errors on vela-core and i was able to create an kubevela application, running a nginx for tests

This is caused by the bug of v1.9.10.
I created a PR for fixing the client bug for v1.9.10, once it's merged, we'll release v1.9.11 and it should be fixed.


chore(ci): don't upload micro-bench results for v1.0

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.
What's changed and what's your intention?
Don't upload the results from this micro-bench. Otherwise our daily microbench will fluctuate wildly.

Checklist

[ ] I have written necessary rustdoc comments
[ ] I have added necessary unit tests and integration tests
[ ] I have added fuzzing tests or opened an issue to track them. (Optional, recommended for new SQL features #7934).
[ ] My PR contains breaking changes. (If it deprecates some features, please create a tracking issue to remove them in the future).
[ ] All checks passed in ./risedev check (or alias, ./risedev c)
[ ] My PR changes performance-critical code. (Please run macro/micro-benchmarks and show the results.)

[ ] My PR contains critical fixes that are necessary to be merged into the latest release. (Please check out the details)

Documentation

[ ] My PR contains user-facing changes.

Click here for Documentation
Types of user-facing changes
Please keep the types that apply to your changes, and remove the others.

Installation and deployment
Connector (sources & sinks)
SQL commands, functions, and operators
RisingWave cluster configuration changes
Other (please specify in the release note below)

Release note

I guess the intention is to upload results only from main branch?
wonder if we can judge if this is on the main branch to enable/disable the uploading step

In favour of https://github.com/risingwavelabs/risingwave/pull/11218


seeking structural types

This is apparently already supported (yay!), but I am too dumb and couldn't easily find out via web / issue searching, so I'm adding this here for seo purposes, if that is considered ok?
/* @flow */
type T = {a:string};
function foo(x:T) {
    console.log( x );
    return x.a;
}
function bar(x:{b:number}) {
    console.log( x );
    return x.b;
}
foo({a:'Hello, world!'});
foo({a:'Hello, world!',b:'foobar'});
//foo({b:'foobar'});
//bar({a:'Hello, world!'});
bar({a:'Hello, world!',b:42});
bar({b:42});

I'm not sure what's going on here. Are you reporting an issue with Flow? Is this an attempt to add documentation? I don't think an issue on GitHub is a great place to add information—people rarely read this. If you want to add documentation, your best bet is to clone the repo and send a PR to the gh-pages branch. If you aren't able to do that, please be more specific about what documentation is missing, so we can track that.

Apologies. Yes this was a stab at documentation. I can try to find time to do the whole PR thing instead, of course. Thanks for everybody's efforts on Flow.


Spike: Selective query response

Proposal:
Optimize query responses to minimize re-reading the same data
Current behavior:
We are incurring large data charges in our clouds, because we re-fetch all the data, rather than just the recent data, on each dashboard refresh.
Desired behavior:
Optimize the query to only fetch the newer data, and merge it in with the previously fetched data.
Alternatives considered:
Need to research whether this is better done in the query engine, or in the UI, or in some mix of both.
Use case:
This is important to save money on data access.

Will be addressed by flux and delorean.


Mui v5

Any chance there will be support for v5?
Getting an error trying to compile.
Module not found: Error: Can't resolve '@material-ui/core'
Which makes sense since @material-ui/core has been moved to @mui/material.
Thanks!

I can't directly support v5 https://stackoverflow.com/questions/70217864/how-to-create-a-package-for-a-react-component-that-supports-material-ui-versions?noredirect=1#comment124127105_70217864
I deployed a version on npm that supports mui v5: https://www.npmjs.com/package/material-duration-picker/v/1.4.5-muiv5
Install it using
npm i<EMAIL_ADDRESS>
I have not tested it myself, so their might be some ui bugs from the migration to mui v5. Let me know if you find any!


Static page update

Jira Ticket:
BRS-384
Jira Ticket URL:
https://bcparksdigital.atlassian.net/browse/BRS-384
Description:

Updated static page template
Added new PageContent folders for components and styles to house page-component specific code
Added PageHeader and PageSection components
Created sticky page section menus for desktop and mobile
Updated calc of page title to pull from page (when it exists in the future), or from the PageHeader component, if it exists
Removed unnecessary MUI components, cleaned up related scss files - both efforts need further work
Created variables.scss to import from, will need to import this to other files as well

Note that not every aspect of the design has been met, as the design is still in flux, and some parts rely on other branches. In the ticket I have listed the parts still to do.
To test, create or update a page that includes a PageHeader component and several PageSection components.
You will need an image url for the header, here is one for testing purposes
https://nrs.objectstore.gov.bc.ca/kuwyyf/haida_trail_marker_installation_1_5319194a95.jpg

Thanks for the review. I'll make a few minor updates.

The menu on my local is appearing like this. Probably not intended?

I was confused because I wasn't seeing thing but it looks like it's caused by short titles. Needs a width: 100% or similar.

Looks good to me other than the two points already noted.


check http response code because safely is not the correct check

even after a failure during the upgrade CI the job continues to run
(finally into a timeout)

+1 code looks good. did you test it in both the positive and negative case?

did you test it in both the positive and negative case?

Tested only manually. I want to run a separate upgrade gating (at least the negative should be covered then)

@MaximilianMeister why not simply using "safely curl -f " ? would be much more readable imho

i didn't know that, thanks for the hint. the only thing is that it will fail silently (i guess no return code, which could be useful to know) so i am not sure if it is better here in this case
it also says
This  method  is  not  fail-safe and there are occasions where non-successful response
              codes will slip through, especially when authentication is  involved  (response  codes
              401 and 407).

+1

@MaximilianMeister you're not using authentication.. so that doesn't affect it.

you're not using authentication.. so that doesn't affect it.

i know, but are you 100% sure that this only happens when authentication is involved? in other words that it is fail-safe? the statement about fail safety in the man page is too vague IMO

just saw that we use digest auth here... not sure if that is covered, but still, i prefer to have sth verbose and fail safe. @jdsn any opinion on this?

actually I prefer the fail-safe option - otherwise other errors might slip through $somewhen
The refactoring I also discussed with Maximilian, but the other places use GET requests.
So finding a generic way of doing this should imho be a separate PR.
So I would recommend to merge this now to be able to continue the testing on the upgrade workflow.
The upgrade-gating job failing is what is wanted with this PR as it hid a bug. But the default gating job should not fail.
When this is fixed I am :+1:

rebased (to trigger new gating)

:+1:  gating is green (upgrade gating red is wanted - so red is green in this case)

+1


AnimatedTriangle.gltf does not rotate correctly

Describe the bug
I tried to create a display sample of glTF model by filament v1.4.0 with gltf-test.
However, the model does not seem to rotate as expected.
To Reproduce
glTF/AnimatedTriangle.gltf + filament result:

Expected behavior

This model is expected to rotate seamlessly.
https://github.com/KhronosGroup/glTF-Tutorials/blob/master/gltfTutorial/gltfTutorial_006_SimpleAnimation.md
Desktop (please complete the following information):

OS: Windows 10 + Chrome 77
GPU: Intel HD Graphics 520
Backend: WebGL/ANGLE (Intel(R) HD Graphics 520 Direct3D11 vs_5_0 ps_5_0)

Thanks!  This is probably occurring in the last .25 seconds, when we slerp between the last quat and the first quat -- we aren't taking the shortest path between these two orientations.
This is my fault: when I fixed a NaN issue, I probably broke this corner case.  We take an abs at the top of our slerp implementation, which obviates the check at the bottom.  Will fix and add a unit test.

Thanks, I have confirmed that AnimatedTriangle.gltf rotates correctly with the latest version of Filament.
glTF/AnimateTriangle.gltf + filament result:

I updated the status of gltf-test.


Add support for Swift Package Manager

Summary
IOS developers have a need to install a pan modal through SPM. Added the ability to connect a pan modal through SPM.
Requirements (place an x in each [ ])

[x] I've read and understood the Contributing Guidelines and have done my best effort to follow them.
[x] I've read and agree to the Code of Conduct.

Any update on this?

cc @nikitskynikita ❤

Just merged this change + updated for Swift 5.0.
Not as familiar with SPM yet so let me know if there are any issues still.
https://github.com/slackhq/PanModal/releases/tag/1.2.6

@nikitskynikita Thanks so much for your PR. That was a great help & apologies for the delay in merging.


Usage:
  gort user [command]

Available Commands:
  create      Create a new user
  delete      Delete an existing user
  info        Retrieve information about an existing user
  list        List all existing users
  update      Update an existing user

Flags:
  -h, --help   help for user

Global Flags:
  -P, --profile string   Gort profile to use

Use "gort user [command] --help" for more information about a command.

While the next execution (with nothing else having changed) returns only the first line:
$ gort group --help

GortAPP  9:57 AM
Allows you to perform group administration.

Fixed by https://github.com/getgort/gort/pull/39