Awk: Fix stripping of CR

Fixes https://github.com/git-for-windows/git/issues/1524

@orgads excellent analysis and work on fixing the issue! I do think, though, that you will want to squash git-for-windows/MSYS2-packages@e65e181a145e940fa00185bc4534be350f535e5a in ;-)

Done. I don't have much experience with makepkg, as you can see ;)

@orgads move configure to build step

Done

I opened a discussion in awk and cygwin mailing lists. Looks like this was a deliberate change:
http://www.cygwin.com/ml/cygwin/2018-02/msg00280.html

move configure to build step

Why, oh why? 😦

Why, oh why? 😦

I guess it is inherited from Pacman's peculiar interpretation of the words "prepare" and "build": https://wiki.archlinux.org/index.php/Creating_packages#prepare.28.29 & https://wiki.archlinux.org/index.php/Creating_packages#build.28.29 ☹ ☹ ☹

@dscho it's Arch standard and makes easier to quickly test changes.
makepkg -e will skip prepare() so one can modify source and then build it.

@dscho also when you build for multiple arches you can run prepare only once

This is NOT a bug - and this pull should be reverted ASAP
http://cygwin.com/ml/cygwin/2017-08/msg00033.html

@Alexpux this is NOT a bug - and this pull should be reverted ASAP
http://cygwin.com/ml/cygwin/2017-08/msg00033.html

it's Arch standard and makes easier to quickly test changes.

@mati865 hah... well, I don't know about you, but on my machine, the ./configure run often takes way longer than the entire make run.

also when you build for multiple arches you can run prepare only once

@Alexpux okay, that's a valid point. Except that we don't use that feature, do we? When we call makepkg-mingw, we still call makepkg twice... or am I wrong?
BTW @Alexpux thanks for the quick turnaround on unbreaking our users.

@Alexpux this is NOT a bug - and this pull should be reverted ASAP
http://cygwin.com/ml/cygwin/2017-08/msg00033.html

@svnpenn: Since @dscho locked the issue, I'll reply here.
I do agree that the applied changes in cygwin are "correct". Still, let me try to explain why this should not be reverted ASAP.
This kind of change can be accepted, but it must come after a decent period of time, after notifying potential users about the implications, and letting them adapt to the change.
I spent a lot of time investigating this issue, and I really doubt I'm the only one that got hit by this.
At work, we have an awk script that in order to be portable does something like:
len = length(substr(string, 2))
if (win) len += 1 # win = match(ENVIRON["OS"], "Windows")
var = substr(string, 2, len-1)

I know this is really bad scripting, but I didn't write this script, and it's there - works as expected.
Following this behavior change, I replaced this with proper stripping of \r.
BUT we have in this repository about 5-6 active official branches, so we need to cherry-pick or merge this change to all of them. Apart from that, there are about 7-8 collaboration branches. We have users that work on all of these branches, each of them typically has about 2-5 local branches, based on different commits from different branches.
Now I can cherry-pick the change to all of the branches, and announce this change and tell all the users to rebase their local branches. Most of them will not do this immediately, and even if they do, some of them will only rebase the current branch and will forget this issue when they switch to another branch. Each user that will face this problem might spend about 30-40 minutes trying to build, rebuild, call me, pull, update git. I'm talking about ~30 users I'm aware of.
Now let's consider the gerrit issue. The commit-msg hook did a seemingly right assumption that /^$/ is an empty line. This is broken with the new behavior, because RS was not set. Gerrit's commit-msg hook is installed once, and used for every commit. Users don't update this hook on a regular basis. There was an update to the qt-project gerrit hook about 5 years ago, and I still notice users with the previous version from time to time (see for example here, and this is one of the primary maintainers of the project. Probably used an old laptop or something).
I applied the needed fix in gerrit today, for version 2.14. It can take a few years until users will upgrade to this version (the qt-project for example is stuck with 2.7, and unlikely to upgrade any time soon...).  I assume that there are at least tens of thousands of gerrit users around the globe, not sure how many on Windows. All of them must update the commit-msg to get correct commit messages.
And these 2 small examples are only those that I personally hit. Who knows how many scripts out there rely on this so-called broken behavior?
To summarize, I'm not opposed to the behavior change, but I strongly disagree with just throwing it at the users without proper announcement (and the cygwin mailing list is not enough. Release notes of MSYS2 and Git for Windows must contain this). So :-1: to ASAP. After announcing it, you should give the users at least 6 months (or even a year) to adapt their scripts. Then you can revert this change, and hope for good.
And if the same should be done for sed and grep, so be it.

@orgads using this command:
printf 'hello world\r\n' | awk 1

previously stripped carriage returns, which you were happy with and relied on.
however what about people who werent happy with that, and didnt rely on that? If
one didnt want Awk to do that, you had to use a command like this:
$ unset POSIXLY_CORRECT
$ printf 'hello world\r\n' | awk -vBINMODE=1 1

and this command doesnt work for Awk scripts, as BINMODE is not recognized even
in the BEGIN section. This error I describe was fixed by the change you are
complaining about.
The pull request here that you tricked @Alexpux into committing reintroduces
this error. Now Awk users have no way to write portable scripts because of your
actions. so yes, while it is certainly inconvenient to do this:
something | tr -d '\r' | awk something

it is better than breaking portability altogether, which is what you have done.

This discussions is going to nowhere.
For now there should be environment variable to let you switch between old vs portable behaviour without using binary mode.

It could be set to old behaviour by default with some sane deadline announced (it was considered as feature for long time and big amount of tools depending on that shouldn't unexpectedly get broken).

Alternatively MSYS2 and Git for Windows could define variable to maintain old behaviour by default  for some time.

@mati865: An environment variables makes sense.
By the way, tr -d '\r' (and also RS = "\r?\n") is also not portable, because \r is used as line break (without \n) on macOS. I think that the assumption that a script works on the same binary file on all platforms is too inclusive.

@orgads you keep putting RS = "\r?\n", and maybe you are not getting that is not portable either. POSIX awk doesnt support RegEx for RS, and it doesnt support multi character RS

By the way, tr -d '\r' (and also RS = "\r?\n") is also not portable, because
\r is used as line break (without \n) on macOS. I think that the assumption
that a script works on the same binary file on all platforms is too inclusive.

@orgads you keep putting RS = "\r?\n", and maybe you are not getting that is
not portable either. POSIX awk doesnt support RegEx for RS, and it doesnt
support multi character RS
true portability is not something that can be reached. it is a goal, it is
something everyone should strive for. what youve done here is say "no thats not
important, what is important is that i dont have to rewrite any non-portable
scripts ever". youve walked away from that goal, shame on you.

By the way, tr -d '\r' (and also RS = "\r?\n") is also not portable, because
\r is used as line break (without \n) on macOS. I think that the assumption
that a script works on the same binary file on all platforms is too inclusive.

@orgads you keep putting RS = "\r?\n", and maybe you are not getting that is
not portable either. POSIX awk doesnt support RegEx for RS, and it doesnt
support multi character RS
true portability is not something that can be reached. it is a goal, it is
something everyone should strive for. its the whole reason the POSIX
(Portable Operating System Interface) was created.
what youve done here is say "no thats not important, what is important is that i
dont have to rewrite any non-portable scripts ever". youve walked away from that
goal, shame on you.

[...] shame on you.

No, shame on you for completely missing the opportunity to realize how your rudeness makes everything worse.
This is not helpful. Stop it.

@dscho you stop it? Youre responding to a comment from 2 days ago that doesnt even exist in the form that you are quoting. This isnt your repository and I dont like being bosses around. My arguments are strong and are backed by sources and merit.
If you wish to continue off topic bickering that is your mistake.

https://github.com/Alexpux/MSYS2-packages/issues/1138

@svnpenn Your arguments constantly ignore the fact that you would break tons of users' setups, and thereby cause no little harm in terms of time and money.
Also, a very important thing to remember is that text (which is what gawk processes, not binary files) is platform dependent, due to the end-of-line conventions. This has also already been pointed out by @orgads.
If you want to turn this into a sober discussion again, do not even post those incorrect claims, and also refrain from insulting people (even if you then delete the comment, to pretend that it never happened).

@dscho you dont have the authority to tell me to post and what not to post.  This issue has been raised and rejected by the Cygwin
community:

http://cygwin.com/ml/cygwin/2018-02/msg00280.html
http://cygwin.com/ml/cygwin/2017-09/msg00130.html
http://cygwin.com/ml/cygwin/2017-08/msg00032.html
http://cygwin.com/ml/cygwin/2017-06/msg00040.html
http://cygwin.com/ml/cygwin/2017-05/msg00161.html

It was likely passed here because @orgads shielded the history of the change
from @Alexpux. Notice carefully that he made no mention in either repository of
the extensive discussion i have linked above. he either didnt care about it or
didnt know about it.
i am guessing the latter because he posted a new thread on Cygwin when at least
4 other threads already existed.
the final word is this. while it may be a (big) hinderance for some to go back
and change scripts to santize input going into Grep/Sed/Awk, its a necessary
evil. You can sacrifice portability in such a blatant and obvious way as this
pull request does with the only justification being that people can keep their
old scripts.

@svnpenn You guessed right, I wasn't aware of these discussions. But neither were @dscho or @Alexpux. Like I said, this change can be accepted, but it will take (a lot of) time. You must give users enough time to adapt to it, and even after accepting it, there has to be a way to force the current behavior. An environment var, like suggested by @mati865, makes a lot of sense. Its default can be changed when decided. After all, this portability issue existed since ever, and you lived with it.
You're right that we can't tell you what to do, but with your attitude you can expect very low cooperation. I consider myself a cooperative person (as you can see from over 2,600 commits I contributed to Qt Creator, and my activity here on github), but because of your attitude I have very low interest in helping out.
I suggest that you propose a pull request that introduces an environment variable which you can use, and we'll consider this issue closed.

The argument that MSYS2 should follow Cygwin is probably based on a misunderstanding: for Cygwin, the line ending is LF-only. For MSYS2, which aims to integrate better with Windows, it is the Windows line ending: CR/LF.
I guess that @Alexpux has already come to that conclusion.

I think some people also misunderstand the importance of portability in scripts


Add 3d heatmap demo

Add 3d heatmap demo using hexagonLayer
Use UK traffic accident data
Create a standalone app and import it from demo

I guess that in the future we'll always have to store our large data files twice... once in the example folder and once in the demo folder...

The idea was that the files in the example folder would be small subsets, quick to load uncompressed.
For the big data files in the demo repository, we really need to get the out of this repository. We can't let deck.gl become a 1GB repo that takes an hour to clone.
I'll open an issue on this.


Compatibility with imagemin-svgo version 9

Package imagemin-svgo released a new major version 9.0.0 a few days ago and it is no longer compatible with this plugin.
When trying to build my Next.js project I get an error like this:
Failed to compile.

./xxxxxxxxx/xxxxxxxxxx.svg
Error: Plugin name should be specified
    at Array.map (<anonymous>)

> Build error occurred

Downgrade to 8.0.0 and the project compiles fine. This might be the clue, but I'm not sure how to apply the change here.

I'm having a very similar problem, except my project doesn't use this plugin. This might be more related to nextjs than to this plugin, unless if NextJs 10 suddenly uses this plugin by default and I missed something.
Error: /..../index.tsx: Plugin name should be specified
    at Array.map (<anonymous>)```

I'm getting no clues whatsoever what is causing this. The message isn't very helpful. The file it's refering to is importing a couple of .svgs as a helper function.

I agree with @stijn-gravity
Seems NextJS still struggling with SVGs. Also the error message can be improved.


ACCUMULO-4086 Improve volume chooser fallback

Implement a better fallback mechanism when the user's selected
VolumeChooser implementation cannot be loaded or is not specified.
Handles all such cases, including non-table scopes (logger), and
per-table scopes.

Note: this isn't quite ready for merging, but since I'm trying to proxy code written by somebody else, I think it'd be helpful to have it inspected/reviewed by more eyes. I tried to run the tests, but they hung (in Eclipse, at least). Keep in mind that this has been rebase'd from an older version of Accumulo, and probably needs a bit more work.
The goal of this is described in ACCUMULO-4086 and its parent task, but all I have is the implementation, and it's not entirely clear what the best design is, in order to make sure this particular implementation matches that design.
Any and all feedback/assistance is helpful. However, I'm already aware documentation is significantly lacking.... will address that once the design and implementation is hammered out... and that's a bit fuzzy to me right now. For example, I see that the patch modifies the default configs, and it's not clear to me what design goal that achieves.

I spoke with Matt, who wrote the original code for this PR, to get a sense of his design intentions. He clarified that the overall goal was to try to avoid RandomVolumeChooser as much as possible, no matter what went wrong (typos, etc.) when a user mis-typed a per-table chooser class or the class path was not set up correctly to find the user's chooser class.
Currently (without this PR), Accumulo has hard-coded defaults on RandomVolumeChooser in several places. This can be problematic, because it can result in filling up volumes reserved for certain tables or the WALs. Further, it seems to select the RandomVolumeChooser mostly silently.
This PR changes things so that if the user has configured the PerTableVolumeChooser, then when that fails to load a particular table's chooser (table.volume.chooser from a TableConfiguration), it will not fall back to the RandomVolumeChooser, but instead fall back to the system-wide setting (SystemConfiguration) for table.volume.chooser (for example, appearing in accumulo-site.xml, outside of any per-table context). If the user did not specify a table.volume.chooser, then then the PerTableVolumeChooser would simply fail. This ensures that RandomVolumeChooser is never used accidentally when using the PerTableVolumeChooser. A consequence of the change in this PR is that the defaults have changed, but this will only affect users who were relying on general.volume.chooser being the default and overriding table.volume.chooser (or vice-versa).
The following is how I'm thinking it should probably work:
Use the RandomVolumeChooser as the new default for general.volume.chooser and empty string (or null) for the default table.volume.chooser, as they are in this PR (see last sentence of previous paragraph). Document VERY WELL (especially in the release notes) that if the PerTableVolumeChooser is required by the user (e.g. they were relying on it being the default), then the user must set the general.volume.chooser to the PerTableVolumeChooser AND must either set a default value for table.volume.chooser in the system-wide configuration (system-wide ZK settings, or site file) OR ensure table.volume.chooser is set for every table (or namespace). If loading a per-table chooser or the general chooser fails at any point, we simply throw an exception, rather than trying to find a fall-back chooser to use.
If we simply fail, rather than trying to fall back to some system-wide working configuration, we can still run into problems loading the class defined at that level, and it's not clear that is what the user wishes to occur. So, we should just fail instead. This would simplify this PR a bit.

I concur that failing if a configured class fails to load is better then falling back to something else.  If a user went through the trouble of configuring something and they did it incorrectly, then the system should never fall back to something else because that may have unexpected and perhaps bad consequences.  Instead the user should be forced to fix the configuration.

Do the test cases hang when run via mvn?

@ivakegg The test cases hung for me in Eclipse. I didn't try on the command-line yet. I'm still trying to understand what the tests are actually trying to do, and how we should change them if we switch to hard fail.

@ctubbsii  I am attempting to resolve the issues within this pull request.  I am creating a separate branch under my account.  Shall I subsequently create a pull request from my branch to yours so that you can update this pull request or is there another way to do this?

@ivakegg if you do a PR against my branch, I'll merge it. Probably the easiest way forward.

I did another PR against this branch to make this compatible with the master.  Your change to the configuration factory object methods require changes to this PR.

@ivakegg  Rather than merge your last PR, I simply rebased, and resolved the errors, and cleaned up some unused variables. Have you been able to get VolumeChooserForNonTableScopeIT to pass? It always hangs when I try.

As it turns out the VolumeChooserForNonTableScopeIT is no longer a valid test.  It is explicitly configuring to use the PreferredVolumeChooser but is not setting the volumes leaving the system badly configured.  Hence when the MiniAccumulo is started up and a table is created, it promptly falls over leaving the test case hanging on the createTable call.  I will try to figure out if I can get the test case to detect the failure in the MiniAccumulo.  If that is not possible then I will simply remove this test.  I think I need to fix up VolumeChooserIT.java as well.

I rebase'd again, and updated to add an enum for scopes and to simplify VolumeChooserEnvironment a bit. Other work that needs to be completed:

per-table scope needs to use the table context for class loading.
need to fail if volume chosen is not in instance.volumes (it should always choose among the configured volumes)
need to support a global default in case per-table or scope-specific property is not set anywhere. if it is set, and fails to load, then it should simply fail. if it's not set, then it should try the global default (using the ChooserScope.DEFAULT and whatever property corresponds to that scope; property name depends on the chooser)
clean up / simplify property names

I've kept this branch rebase'd. It is ready, but I want to simplify the unit tests a bit before merging it in, and to ensure they have full coverage of the different scopes and fallback cases. I'll be working on that tomorrow.


Typo mistake in write up.

https://developers.sitecore.com/downloads/xm-cloud#xm-to-xm-cloud-content-migration-tool

Mentioned items not available on XM Cloud but exist in XM.
Document Details
⚠ Do not edit this section. It is required for developer.sitecore.com.com ➟ GitHub issue linking.

Page: XM Cloud Tools and Resources
Source: markdown/pages/downloads/xm-cloud/index.md

Thank you @rinku79  ! I've just fixed this and deployed the typo fix to the site.

Great, thanks


Update README.md

Correct the json in README.md to be a valid json.

@cloudbow Can you provide more details of what you are actually fixing here?

The tip where to find the output was great! Should accept this PR for that alone.

LGTM. Thanks @cloudbow


Fixing generate qa tests

Fixing
Traceback (most recent call last):
  File "/opt/conda/envs/quicklook/lib/python3.5/threading.py", line 914, in _bootstrap_inner
    self.run()
  File "/opt/conda/envs/quicklook/lib/python3.5/threading.py", line 862, in run
    self._target(*self._args, **self._kwargs)
  File "/app/framework/bin/qlf_pipeline.py", line 235, in ingest_parallel_qas
    qa_tests=self.generate_qa_tests()
  File "/app/framework/bin/qlf_pipeline.py", line 253, in generate_qa_tests
    for camera in self.data.get('cameras'):
  File "<string>", line 2, in __getitem__
  File "/opt/conda/envs/quicklook/lib/python3.5/multiprocessing/managers.py", line 716, in _callmethod
    conn.send((self._id, methodname, args, kwds))
  File "/opt/conda/envs/quicklook/lib/python3.5/multiprocessing/connection.py", line 206, in send
    self._send_bytes(ForkingPickler.dumps(obj))
  File "/opt/conda/envs/quicklook/lib/python3.5/multiprocessing/connection.py", line 404, in _send_bytes
    self._send(header + buf)
  File "/opt/conda/envs/quicklook/lib/python3.5/multiprocessing/connection.py", line 368, in _send
    n = write(self._handle, buf)

Problem will be solved by verifying the process is still running


Doubled error message

Part of the karma.config:
reporters: ["progress", "coverage", "karma-remap-istanbul"],

SlimerJS App component should build without a problem FAILED
        Failed: fixture.nativeElement.innerText is undefined
        @build/js/components/app/components/app/app.component.spec.ts:25:16 <- build/js/components/app/app.component.spec.js:22:13
SlimerJS: Executed 8 of 14 (1 FAILED) (skipped 1) (0 secs / 0.268 secs)
SlimerJS App component should build without a problem FAILED
        Failed: fixture.nativeElement.innerText is undefined
SlimerJS: Executed 8 of 14 (1 FAILED) (skipped 6) (0.283 secs / 0.268 secs)

If I will remove karma-remap-istanbul from reporters, the result will be:
SlimerJS App component should build without a problem FAILED
        Failed: fixture.nativeElement.innerText is undefined
        @build/js/components/app/components/app/app.component.spec.ts:25:16 <- build/js/components/app/app.component.spec.js:22:13
SlimerJS: Executed 8 of 14 (1 FAILED) (skipped 6) (0.315 secs / 0.263 secs)

Workaround - separated gulp task.
    var remapIstanbul = require("remap-istanbul/lib/gulpRemapIstanbul");
    gulp.task("coverage", function() {
        return gulp.src(".coverage/**/coverage.json")
            .pipe(remapIstanbul({
                basePath: "./",
                reports: {
                    "html": ".coverage/html-report",
                    "json": ".coverage/report.json",
                    "lcovonly": ".coverage/lcov-report.info"
                }
            }));
    });

I think the issue here was that karma-coverage is not actually needed when using this lib, using reporters: ["progress", "karma-remap-istanbul"] and adding the reporters in the karma.conf.js like so:
remapIstanbulReporter: {
      reports: {
        html: 'coverage/html',
        'text-summary': null
      }
    },

should be enough to prevent the double output. If this doesn't work for you please let me know!

Hi,
I also have this issue. I only have progress and karma-remap-istanbul as reporters. I have similar setup as this project: https://github.com/gund/ng-http-interceptor and it also happens there.

I don't have time to actually test it, but try setting this to only instrument .ts files: https://github.com/gund/ng-http-interceptor/blob/master/webpack.config.js#L65 and remove the inline sourcemap option as it's already handled by the SourceMapDevToolPlugin below: https://github.com/gund/ng-http-interceptor/blob/master/webpack.config.js#L8
Alternatively try using this loader with this reporter instead of sourcemap-istanbul-instrumenter-loader and karma-remap-istanbul.

I tried all of your suggestions, but none of them seem to have any effect on the doubled failed test reporting.

Oh maybe it's because of the entry in the webpack config, this isn't required if using karma-webpack, try deleting this: https://github.com/gund/ng-http-interceptor/blob/master/webpack.config.js#L14-L16

It isn't required - but removing is again has no effect :P


AnkiOCR error

Hi, I've encountered an error message

seems like it tried to show  "\U0001F50A", which is a speaker emoji in html form but failed.

Maybe "U0001F50A" and "U0001f50a" are not the same thing but I know nothing about coding.


See also removed file 96 to 100

partial fix for SEE ALSO section under content/zh/docs/reference/setup-tools/kubeadm/generated/ need to be deleted #18411
see also removed from file 96 to 100

/lgtm

/cc @xichengliudui


Fix contibution guide link

Currently, the link in README.md is broken.

LGTM. Thank you very much!


i18n(es): Removing dead link from wordpress docs

Description (required)

Removing a dead link from the Wordpress guide (spanish)

Related issues & labels (optional)

Closes #
Suggested label:

Lunaria Status Overview
🌕 This pull request will trigger status changes.

Learn more
By default, every PR changing files present in the Lunaria configuration's files property will be considered and trigger status changes accordingly.
You can change this by adding one of the keywords present in the ignoreKeywords property in your Lunaria configuration file in the PR's title (ignoring all files) or by including a tracker directive in the merged commit's description.

Tracked Files

File
Note
Locale

src/content/docs/es/guides/cms/wordpress.mdx
Localization changed, will be marked as complete.
es

Warnings reference

Icon
Description

🔄
The source for this localization has been updated since the creation of this pull request, make sure all changes in the source have been applied.


Bug: could not execute VirtualBox

My System Spec
OS : Windows 8.1 EnterPrise K (64bit)
RAM : 4GB
Hain version : v0.0.4
Symptom

To search for "VirtualBox" is good.

After 'enter' key, Hain could not execute the "VirtualBox" program.
If needed, I will append more information about this issue.

Maybe it's broken link. Could you check it?

I also have a few program not launched via Hain :

Oracle VM VirtualBox
XnConvert
Rainmeter
SketchUp
...

All x64 apps.
All shortcuts are valid

I also couldnt launch Snipping Tool from Windows 10 64

Oracle VM VirtualBox.zip
Please refer to the VirtualBox lnk file.
I copied it from ProgramData folder to DeskTop(바탕화면) folder in order to zip it.
After copying it, the icon picture was changed.
This seems to be the reason why Hain couldn't execute this program.

I found that shortcut which is owned by SYSTEM not a user couldn't be launched by Electron shell.openItem. I'm finding a way to solve this problem.

@SDRLurker I think it's fixed at v0.1.0, Can you try this?

@appetizermonster I have tried to execute Virtual Box. It's a success to execute Virtual Box. Thank you. :+1:

@SDRLurker Good!, I close this issue


Newsroom pages aren't updated with the latest news

We see that neither the no or the en newsroom is updated with the latest news.
I published a news today that is visible on the homepages, but not on the newsroom pages. The English newsroom is just missing the latest news (from today), but the Norwegian newsroom is missing the one from today and two more.

@meols This is fixed. I missed updating the webhook after i18n upgrade for production.

Fixed this for all news that went missing. Will see if there are any issues with the new ones , if not we are good to close. I have updated the webhook which I missed to update after i18n.

28 th Aug news publish did not trigger webhook, so the news is not indexed. So manually trigger the indexer function.

11th sep news is not indexed. Modified the webhook to trigger on create not on update.

Confirmed that the news is indexed on 17 th sep

@meols No specific criteria to test as this is happened in prod. Two things were done to fix

I updated the old _lang field with the new lang field in the webhook that I missed during i18n upgrade.
Next, I enabled the webhook trigger on publish along with edits. This was necessary after the i18n upgrade.

I observed the newsroom for the last two news articles published and they were indexed immediately.
I suggest we can close this now as this is due to wrong configuration of the webhook and not a bug in the code. Please create a new one if an issue arises in future.

Thanks @padms - I agree, this can be closed.


[DEV] The check port script fails on it's own and does not check if the process running on the port is a metro bundler

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

1. Script Fails on it's own
Action Performed:

Make sure you are not running anything on port 8081
Try to run npm run android

Expected Result:
The check port script passes and continues execution
Actual Result:
The script reports that port 8081 is used. Possibly because it tries to start 3 servers on the same port at the same time.
Workaround:
Run npx react-native run-android manually to avoid running the check port script
Platform:

Where is this issue occurring?

GNU/Linux (Arch Linux)

2. Script does not check if port is used by bundler
Action Performed:

Start metro bundler npm start
npm run android

Expected Result:
The check port scripts doesn't error out if port 8081 is already used by metro bundler
Actual Result:
The script reports that port 8081 is used and prevents further execution
Workaround:
Run npx react-native run-android manually to avoid running the check port script
Platform:

Where is this issue occurring?

GNU/Linux (Arch Linux)
macOS Big Sur

Version Number:
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation
Expensify/Expensify Issue URL:
View all open jobs on GitHub

Triggered auto assignment to @Christinadobrzyn (AutoAssignerTriage), see https://stackoverflow.com/c/expensify/questions/4749 for more details.

Proposal
According to https://github.com/Expensify/App/pull/3718, the intention behind that script is to improve the error message when attempting to run the iOS/android app alongside the desktop and web in development, but while it does that to a certain extent, it removes some of the functionality baked into any react-native project by default. To be more specific, the ability to start the bundler manually (issue 2 in the original post).
Why is it important? The npx react-native run-{ios,android} commands can't spawn the window with a sub-process in every environment. There is this issue for example https://github.com/facebook/react-native/issues/28807 and while it works for me on macos in tmux, I would assume that it won't be the case with WSL. So in cases like this it is recommended to start metro bundler manually.
I have a proposal that both allows you to keep that fancy error message and brings back the default functionality.
The original check port script tries to start 3 servers on the port at the same time. I think it fails for me because it tries to start one of it's servers while another one is still occupying the port. Even if we are to fix this issue, we still need to check if the port is used by metro bundler and it's not possible with this approach.
I propose using a function isPackagerRunning from @react-native-community/cli-tools. You can find the source here: https://github.com/react-native-community/cli/blob/master/packages/tools/src/isPackagerRunning.ts
Basically it has 3 possible returns:

running - if metro bundler is already on that port
not_running - if nothing is running on that port
unrecognized - if there is a server that is not metro bundler on that port

So we can replace the entire start-three-servers-at-the-same-time code with this single call. It should looks like this:
isPackagerRunning().then((result) => {
    if (result === 'unrecognized') {
        console.error(
            'The port 8081 is currently in use.',
            'You can run `lsof -i :8081` to see which program is using it.\n',
        );
        process.exit(1);
    }
});

If you still want to use it for any port you can pass a port to it:
// The code from original implementation
const inputPort = Number(process.argv.slice(2)[0]);
if (Number.isNaN(inputPort) || inputPort < 0 || inputPort > 65535) {
    console.error('This command requires an argument that must be a number [0-65535]\n');
    process.exit(1);
}

isPackagerRunning(inputPort).then((result) => {
    if (result === 'unrecognized') {
        console.error(
            `The port ${inputPort} is currently in use.`,
            `You can run \`lsof -i :${inputPort}\` to see which program is using it.\n`,
        );
        process.exit(1);
    }
});

With this implementation you still get the same error if port is used by anything that doesn't look like metro bundler and you don't have to kill everything on port 8081 before running npm run {android,ios}.

Oh, and since this is a dev-only issue that nobody has complained about since that check-port script was merged, I am willing to contribute this one without an upwork contract. It doesn't really make sense for you to pay for something that only apparently affects one person.
Thanks.

Triggered auto assignment to @tgolen (Engineering), see https://stackoverflow.com/c/expensify/questions/4319 for more details.

Could we confirm the --port parameter works correctly with iOS? I remember changing the metro port, but it didn't work with iOS because the 8081 is hardcoded in some iOS-specific files.

@HorusGoul
You are right. It is hardcoded: https://github.com/facebook/react-native/issues/9145
The issue is closed, but I can still reproduce it.
Neither the env var RCT_METRO_PORT nor the --port option/flag seem to do the job for me.
So I guess this leaves us with Solution 1?

The issue has seen no discussion last week, so I opened a PR with my best take on it.
I have no env setup for windows testing, so it would be nice if someone was to validate the PR on windows.
Thanks.

None of us have windows environments either, so I think we are OK with just trying out best. Thanks!

PR submitted - under review

:hand: This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

🚀 Deployed to staging by @marcaaron in version: 1.0.95-2 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

Hold for payment set to [HOLD FOR PAYMENT Sept 17]

🚀 Deployed to production by @roryabraham in version: 1.0.96-0 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

7 days since PR deployed - paid @dklymenk in Upwork (including $250 bonus for finding and fixing the bug) and closed the job.


Firebase_Auth check email verfied before sign in

There must be a way to check if a user is email verified before firebase signs in the user with email and password!
Currently on the API the method signs in the user without check if it's verified. If I use authStateChanges() to control which page to show to the user and when the user signs in but not email verified, I have to manually sign them out, and that cause problem
"
Future signInWithEmailAndPassword (
{@required String email,
@required String password}
)
Attempts to sign in a user with the given email address and password.
If successful, it also signs the user in into the app and updates any authStateChanges, idTokenChanges or userChanges stream listeners.
"

This is how the Firebase API works, you have to sign in to know the status of the user, so this is working as intended. Unless you have specific reasons for doing so - you don't need to sign your user out, you could just block interactions and request that they verify their account.


Exposed some backend information; useful for debugging

Looks good and read only.

Ok VkGraphicsDevice got full of whitespace changes so sorry about that.
Luckily there is a feature for mitigating that problem, so if you don't think it's necessary to fix the whitespaces you can use this:

@TechnologicalPizza I'm looking at this again and it seems like a lot of unrelated changes got mixed into this branch which need to be pulled out.

Oh yeah how strange. I'll try fixing it.

Looks good to me, merged it to master.


Fix Tokenizer for Correct Tokenization of Texts with Emojis

During the error analysis of Product Review Sentiment Model I saw that the reviews having emojis where tokenized incorrectly. One such example of this incorrect tokenization is 👍basarili ve kaliteli bir urun .  > [👍b, asarili, v, e, k, aliteli, b, ir, u, run, .]. There are various cases similar to this one.
I will be exploring this issue to understand the underlying cause and try to fix it accordingly.

For this type of bugs we have implemented exception rule handling. Simple solution might be to create user.ini file into ~/.sadedegel directory and update emoji=true in [tokenizer] section. Once done this is the output
In [1]: from sadedegel.about import __version__

In [2]: __version__
Out[2]: '0.20.1'

In [3]: from sadedegel import Doc

In [4]: Doc("👍basarili ve kaliteli bir urun .").tokens
Out[4]: [👍, basarili, ve, kaliteli, bir, urun, .]


Updating doc urls

Summary
Updating doc urls in the readme.

This PR can be closed as it was created before the readme overhaul when updating Full Stack to legacy.


The response time is quite high

I have uninstall Nginx

I test is on my local computer, everything is fine. It's really strange.

After I uninstall and reinstall nginx, this problem is disappered.


[cf-forms] Missing @input-disabled definition

https://github.com/cfpb/capital-framework/blob/prerelease-ver-4/src/cf-forms/src/cf-forms.less#L118 is not defined.
Also, the color variables at the top of that file are undefined.

Addressed in #424


NullPointer when WeakReference cached type is GC'd

Hello,
We are currently having intermittent problems during unit tests in our build that breaks because of a NPE in Orika.
Here is my fix: https://github.com/orika-mapper/orika/pull/342/files
I investigated the issue, seems to be related to TypeFactory cache, that holds a WeakReference for the type that is not properly checked for nulls, and hence will return Null as Type.
Unfortunately there is no easy way to reproduce a WeakReference being GC'd, so I wasn't able to think of a unit test to share reproducing the problem.

Please try to reproduce it with a test.

@fernandosiq-dh Please close the issue since the pull request is now merged. Thx.


Allow importing redirects from a JSON file

The package features a JSON importer, but it hasn't been fully implemented yet.

This was implemented as part of the v4.0.0 release.


Please mention adding Power BI component into solution as Prerequisites

I have received a query from customer that customer cannot use this feature because they have not added Power BI components into a solution.
https://learn.microsoft.com/en-us/power-apps/maker/model-driven-apps/powerbi-control
Could you please mention following article in Prerequisites?
This article is located in a different section, so customer sometimes does not notice this.
https://learn.microsoft.com/en-us/power-apps/maker/model-driven-apps/create-edit-powerbi-report-dataset-components#create-power-bi-report-and-dataset-components

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 218a219e-4e18-5f84-29b4-035e410c5681
Version Independent ID: 218a219e-4e18-5f84-29b4-035e410c5681
Content: Use the Power BI report control to add a report to a form with Power Apps - Power Apps
Content Source: powerapps-docs/maker/model-driven-apps/powerbi-control.md
Service: powerapps
Sub-service: mda-maker
GitHub Login: @Mattp123
Microsoft Alias: matp

Thanks for your comment, @yokoshima-ms. This issue has been assigned to the doc author for review.


Create & Configure Slack Integration For Critical Alerts

Create & Configure Slack Integration For Critical Alerts

Wazuh monitoring , and alerting in place via slack, Critical alerts will alert directly into the new secure/private slack channel, and a monitor is running, which will notify the volume of Serious alerts every 3 hours.


d.ts not updated

The d.ts of the project is not updated.
The main problem is the inclusion of this line:

import React = __React;

When using @types/react there is no global variable named __React and so this file breaks the build.
The solution is kind of easy - omit the d.ts file from the project.
This way, whoever wants to use the d.ts will use @types/react-custom-scrollbars which don't use the global __React variable.
(In a way, inserting the d.ts was a breaking change for everyone who used @types/react).
Let me know what you think.

+1 the .d.ts file in project breaks usage.

I'm not quite familiar with TypeScript, would be cool if someone made a PR to fix this!

remove the index.d.ts file,
and delete
"typings": "index.d.ts",
section in package.json.

https://github.com/malte-wessel/react-custom-scrollbars/pull/135

Fixed by #135


Add types to getDatabaseRef

Add types to getDatabaseRef function, and restructure code to take advantage of the types.
Tested locally.

Published in beta tag under<EMAIL_ADDRESS>


Make types more general

lazyBytes exits from the computation, so these return calls are never evaluated. Removing them allows the type signatures to be more general.

Okay, makes sense.

Uploaded Spock-lucid-0.4 on Hackage.


TypeScript Organize Imports inserts extra newline

TS Template added by @mjbvz
TypeScript Version: 3.8.0-dev.20200108
Search Terms

Organize imports

In TypeScript / JavaScript, the command Organize Imports inserts up to two newlines if export { ... }; is at the end of a file. This does not happen with other forms of export.

VSCode Version: 1.41.1
OS Version: Windows 10 (1803)

Steps to Reproduce:

Have a list / renamed export as the last line of your file, e.g.:

import { Router } from 'express';
const router = Router();
// ... other code
export { router as namedRouter }; // also happens with `export { router };`

Run the Organize Imports command. It will insert a newline after the export { ... };
Run the Organize Imports command again. It will insert another newline, resulting in two empty lines at the end of the file.

Expected behavior
Only one newline is inserted / kept after the export { ... };.
Does this issue occur when all extensions are disabled?: Yes

Can I pick it up?

Yes

(Unless @elibarzilay has already started, but I assume you can go ahead)

@DanielRosenwasser if @elibarzilay has already started then I will only follow this. Anyway I don't know where to start

I believe you'll need to fix coalesceExports in src/services/organizeImports.ts

@Kinrany Thank you

@Kinrany do you know how to debug this file? I've tried to put console.log, but it is throws an error

@DanielRosenwasser @elibarzilay @Kinrany
I'm able to reproduce the bug, but I'm unable to see any of my logs.
I've tried:

link typescript local build to my project (npm link)
change  "typescript.tsdk":"D:/js/typescript/TypeScript-master/lib"
add logs directly into node_modules  source code (without any links)
to use console.log and logger.log
it did not help :/

How should I do it right?

@captain-yossarian I'm new so little experience with trying to run things live, instead I start with a test case for the problem and then use the debugger.

Do you have any auto formatters installed currently? Something like prettier may be adding the new line.

If someone out there is still having this issue, try disabling the "EditorConfig for VS Code" extension or removing insert_final_newline = true rule from .editorconfig.

I did set "source.insertFinalNewline": false. Nothing changed.
"source.organizeImports": true still inserts a newline at the end of imports.

@MM1132 This issue is about a problem where VS Code inserts two newlines at the end of imports, which can be solved by disabling insertFinalNewline. This solution will not make VS Code stop inserting a single newline at the end of imports, it just fixes the duplicate newline problem.

insertFinalNewline changes absolutely nothing. For now, disabling source.organizeImports is the only thing that makes VsCode not insert double newline after imports.

I didn't test it extensively, but for me adding the following to settings.json worked so far:
"files.insertFinalNewline": true,
"files.trimFinalNewlines": true,

With that, multiple newlines at the end of a file are being trimmed to only have a single newline, and it seems that this setting runs last.


⚠ Bofrost.at has degraded performance

In 8db0624, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 824 ms

Resolved: Bofrost.at performance has improved in ac02bd2 after 304 days, 1 hours, 2 minutes.


changes on the Watson Assistant configuration

@kephart I have updated the readme with the issues that Melina had on the installation and configuration with Watson Assistance. Please could you give me write access to close the pull request?
Thanks


Add redis built into Gitea in Docker image by default

As title

Codecov Report

Merging #3937 into master will increase coverage by 0.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #3937      +/-   ##
==========================================
+ Coverage   20.26%   20.27%   +0.01%
==========================================
  Files         146      146
  Lines       29316    29316
==========================================
+ Hits         5942     5945       +3
+ Misses      22472    22470       -2
+ Partials      902      901       -1

Impacted Files
Coverage Δ

modules/process/manager.go
73.91% <0%> (+4.34%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 60424b7...0f812b5. Read the comment docs.

@lunny my understanding is that when docker builds it takes the environment variables set and passes them to the Makefile and builds the binary which adds it to the image. My change is to add another tag to the build.
Or do you mean the build label for this PR?

@techknowlogick I mean which source file will use this build tag redis? I didn't find the build tag on Gitea's source files maybe it's in macaron's source files?

@lunny It's mentioned in our docs (https://docs.gitea.io/en-us/config-cheat-sheet/ - in section Cache)

@daviian It seems the docs is too old. see https://github.com/go-gitea/gitea/blob/master/vendor/github.com/go-macaron/cache/redis/redis.go, no redis tag there.

@lunny You are totally right. I will send a PR updating the docs. So this PR is not necessary at all to use redis in docker ;-)

Than probably this PR can be closed than just like issue that redis does not work in docker

Closing this per above discussion. Thanks all for the above information.


Next version is 3.0.1.FInal-SNAPSHOT

Signed-off-by: Jeff Mesnil<EMAIL_ADDRESS>

@jamezp I'm not sure if that was intentional to drop the Final from the version.


Starting a container with a previous-used name returns: a container has already joined the endpoint

Hi,
This seems related to #14169.  In that issue, the conflict is between two containers with the same service name and is described as 'by design'.  In this case, I am not defining a service name, just a container name.
Description of problem:
Container fails to start if it has the same name as an existing container, even if the existing container has been renamed to something else.
docker version:
Client:
 Version:      1.8.1
 API version:  1.20
 Go version:   go1.4.2
 Git commit:   d12ea79
 Built:        Thu Aug 13 02:35:49 UTC 2015
 OS/Arch:      linux/amd64

Server:
 Version:      1.8.1
 API version:  1.20
 Go version:   go1.4.2
 Git commit:   d12ea79
 Built:        Thu Aug 13 02:35:49 UTC 2015
 OS/Arch:      linux/amd64

docker info:
Containers: 2
Images: 4
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 8
 Dirperm1 Supported: false
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 3.13.0-62-generic
Operating System: Ubuntu 14.04.3 LTS
CPUs: 1
Total Memory: 1.955 GiB
Name: vagrant-ubuntu-trusty-64
ID: COWL:55FE:IHYG:AVJ6:AE7B:U4E2:H77K:E3ON:WKRS:CODI:6OM6:RJFR
WARNING: No swap limit support

uname -a:
Linux vagrant-ubuntu-trusty-64 3.13.0-62-generic #102-Ubuntu SMP Tue Aug 11 14:29:36 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

Environment details (AWS, VirtualBox, physical, etc.):
Ubuntu Trusty 64-bit running in VirtualBox (via vagrant)

How reproducible:
100% of the time.
Steps to Reproduce:

start container named foo:
 docker run -d --name=foo ubuntu:trusty bash -c 'while true; do sleep 1; done'

rename container to bar:
 docker rename foo bar

start another container named foo:
 docker run -d --name=foo ubuntu:trusty bash -c 'while true; do sleep 1; done'

Actual Results:
New container foo gets created but an error occurs when trying to start it.
$ docker run -d --name=foo ubuntu:trusty bash -c 'while true; do sleep 1; done'
Error response from daemon: Cannot start container 27e3756a16d06c4c0172e52538d5046c584277a10429fab6bed53041e9fad25c: a container has already joined the endpoint

$ docker ps -a

CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS               NAMES
27e3756a16d0        ubuntu:trusty       "bash -c 'while true;"   2 seconds ago       Created                                 foo
183810fd902f        ubuntu:trusty       "bash -c 'while true;"   13 seconds ago      Up 12 seconds                           bar

Expected Results:
New container foo should be created and started with no errors.
Additional info:

docker.log output:
DEBU[0195] Calling POST /containers/create
INFO[0195] POST /v1.20/containers/create?name=foo
DEBU[0195] Calling POST /containers/{name:.}/start
INFO[0195] POST /v1.20/containers/7ec6ad9b07af7e942248577d88394d791041874f0a454a9d8e576d18b883b121/start
DEBU[0201] Calling POST /containers/{name:.}/rename
INFO[0201] POST /v1.20/containers/foo/rename?name=bar
DEBU[0204] Calling POST /containers/create
INFO[0204] POST /v1.20/containers/create?name=foo
DEBU[0204] Calling POST /containers/{name:.}/start
INFO[0204] POST /v1.20/containers/35fb82ca807425dee3b809af98c64fd82e4d1d02ac6ee7c0d229b9ec2437ac05/start
ERRO[0204] Leave all failed for  35fb82ca807425dee3b809af98c64fd82e4d1d02ac6ee7c0d229b9ec2437ac05: could not find sandbox for container id 35fb82ca807425dee3b809af98c64fd82e4d1d02ac6ee7c0d229b9ec2437ac05
ERRO[0204] Handler for POST /containers/{name:.}/start returned error: Cannot start container 35fb82ca807425dee3b809af98c64fd82e4d1d02ac6ee7c0d229b9ec2437ac05: a container has already joined the endpoint
ERRO[0204] HTTP Error                                    err=Cannot start container 35fb82ca807425dee3b809af98c64fd82e4d1d02ac6ee7c0d229b9ec2437ac05: a container has already joined the endpoint statusCode=500

docker inspect output for both containers

@pbitty the design is related to https://github.com/docker/docker/issues/14169, but not handling the renaming case is a bug. Added 1.9.0 milestone.

@jfrazelle @icecrime is this a critical requirement for 1.8.2 ? with the support for multi-host networking in 1.9.0, there will be a few changes to the back-end implementation of using --name as endpoint name. I prefer not to add a patch now to fix this and redo it for 1.9.0. Since the rename feature is not being used much  I dont see the compelling need to force this fix into 1.8.2. WDYT ?

not critical for 1.8.2 i dont think because it has been broken since 1.7.0
and we just are hearing about it now
On Tue, Sep 1, 2015 at 8:38 AM, Madhu Venugopal<EMAIL_ADDRESS>wrote:

@jfrazelle https://github.com/jfrazelle @icecrime
https://github.com/icecrime is this a critical requirement for 1.8.2 ?
with the support for multi-host networking in 1.9.0, there will be a few
changes to the back-end implementation of using --name as endpoint name. I
prefer not to add a patch now to fix this and redo it for 1.9.0. Since the
rename feature is not being used much I dont see the compelling need to
force this fix into 1.8.2. WDYT ?
—
Reply to this email directly or view it on GitHub
https://github.com/docker/docker/issues/15727#issuecomment-136763736.

Thanks @jfrazelle . Also, I confirmed that docker rename feature is not compatible with link in 1.6 & earlier. And that is exactly the service discovery issue that i was worried about for multi-host networking.
Is it a stretch to prevent rename for some conditions (such as multi-host service discovery or --link) ?

I'd like to understand why changing the container name breaks networking. We should use IDs there for everything.

@calavera its the same reason why --link fails when we rename a container. Its to update the /etc/hosts with the name to ip-address mapping.
When the container name is changed, all the linked containers need to be updated. It was never taken care of --link feature. But, in the new CNM model, we implicitly create an endpoint with the container name. a Rename doesnt rename the endpoint and hence when a new container reuses the name, it is unable to reuse the endpoint which is already being used.
A quick fix is to just rename the endpoint. But it will result in the same issue as in --link.

@mavenugo why are endpoints using container names as IDs in the first place? This makes me think that if you rename a container and then remove it, the endpoint and net_ns will be leaked because we won't be able to find it, is this correct?

Closed by #17390

@tiborvass @calavera is there any chance of getting this backported to 1.8? We are using 1.8.3 and realized two of our servers weren't on docker 1.8. After upgrading those docker 1.6 boxes we noticed this bug (as you know a regression from docker 1.6). A bit outside of your control, but unfortunately coreOS still doesn't have docker 1.9 in stable and while I'd love to just upgrade it would prove a lot more difficult.


cl_khr_fp16 is poorly tested

Missing tests in most of the suites:

[ ] bruteforce
[ ] conversions
[ ] relationals
... and probably others

Trying to figure out exactly what folders would be the most work to properly test this extension.
Any insights welcome.
Bruteforce
Rather than reimplementing all the reference functions with a 32-bit single precision overload to verify half against, we could use Ulp_Error_Half with the existing double references. Would that be admissible?
For functions which take a single parameter we can resonably test every possible 16-bit input. How many inputs should we test for functions which take multiple parameters? Not sure how this is currently done.
Geometric
Looks a pain, seperate ~1000 line files for single and double precision, with a lot of duplication. C++ files rather than C, so scope for some refactoring but could be a fair bit of work.
Relationals
Same as geometrics but not as large, seperate files for float and double which could be cleaned up into a single file for all floating point types.
Verification functions on host won't be able to use half, convert to float with half2float and do comparison there instead.
Conversions
Already a mass of function definitions enumerating all the possibilites that will need to be added to. Definetelty scope for refactoring this code to use templates.
This test already takes a long time, could be an issue if increase in combinations makes this worse
Commonfns
Validation functions use type of same precision on, but host code won't be able to use half, upcast to float with half2float and do comparison there instead.
Basic

async_work_group_strided_copy
async_work_group_copy
vload/store
astype

Select
Glance through suggests not much to add and we can us cl_ushort & cl_short in reference functions.
Printf
Doesn't look like too much work other than adding the format specifiers
SPIR
Regenerate zips and update khr.csv?
SPIRV New
Don't know much about these.

Thanks for the initial analysis, that's great input.
I think we've reached the point where we need to take a step back and figure out how we want the code of these individual tests to be structured overall. We can't keep copy/pasting and adding special cases :). If there are specific refactorings you have in mind that you think would make it easier to add FP16 support, feel free to suggest them here, create individual issues or send PRs directly.

Addressing the copy/paste situation definitely sounds like a good approach long term. The worst offender seems to be conversions, in particular basic_test_conversions.c, so I've come up with an idea about that.
I think we need to change this from a C file to C++ so we can use templates to reduce the duplication, since this code doesn't use any of our C entry points I don't think that should be a problem.
When I wrote some unit testing of conversions internally for half, I used a templated struct resembling the below, with Strong Types to differentiate between cl_half and cl_ushort in template instantiations.
template <typename StrongFrom, typename StrongTo>
struct ConvertHelper {
  using WeakTo = typename StrongTo::WrappedT;
  using WeakFrom = typename StrongFrom::WrappedT;

  // Reference function
  static WeakTo reference(const WeakFrom x, const RoundingMode rounding, const bool saturated);

  // Check for undef behaviour if validation with reference failed
  static bool undef(const weakFrom, const bool saturated);

  // Used to check for FTZ
  static bool denormal(const weakFrom x);
};

Then defined various partial specializations for different cases like int->float, smaller float->larger float, etc. Which is where we'd add the fp16 conversion references.
Using saturated conversions as an example from the CTS if we use a similar technique, all the foo2bar_sat_many definitions could be replaced with a function like
template<typename StrongFrom, typename StrongTo>
convert_test(void * out, void* in, size_t n) {
  StrongFrom::WrappedT* in_ptr = (StrongFrom::WrappedT*) in;
  StrongTo::WrappedT* out_ptr = (StrongTo::WrappedT*) out;

  for (size_t i = 0; i < n; i++) {
    ConvertHelper<StrongFrom, StrongTo>::reference(out_ptr + i, in_ptr + i, RoundingMode::default, true); }
  }
}

Which we'd instate in gSaturatedConversions
Convert gSaturatedConversions[kTypeCount][kTypeCount] = {
  convert_test<UChar, UChar>, ..., convert_test<Long, Long>
};

Hacking together a prototype would be the best way to see if this is really suitable, but finding the time for that is a different matter.

Regrading Bruteforce I pushed our proposal #529 for some functions and fp16. We have more changes in other files:
binary_two_results
binaryOperator
ternary
unary_two_results
unary_two_results_i
However we need some more time to debug and fix errors we see.

Request suggested test plan from Mobica before work start


Terminating request after handling an error

I've been tinkering with file upload streams and came across such scenario:

something causes an error and this error is passed down to the custom error handler
the custom error handler invokes res.status(500).send(message)
request is still not terminated
This is caused by data waiting on the request to be handled. I want the request to terminated soon after 2 happens.

I've looked into the express code and noticed that there is a last handler I could pass the error to. The problem is, in "production" mode a generic error message would be sent to the user, whereas I'd prefer a custom message. Tampering with http.STATUS_CODES is out of question for me.
After looking a bit further and experimenting, I've created such a snippet:
app.use((err, req, res, next) => {
  // Print error
  console.error(err.stack);

  function write() {
    res.status(500).send('Internal server error - see logs for details');
  }

  if (req.complete) {
    return write();
  }

  // Terminate request if there is unsent data
  req.unpipe();
  req.on('end', write);
  req.resume();
});

The bottom part is obviously copied from the finalhandler lib.
My question (at last) is: can this be done a bit cleaner? Getting there wasn't that easy and the result looks a bit complicated. Did I miss something in the docs?

No, you did not miss anything. The issue here is a Node.js core issue. The finalhandler contains a workaround for the Node.js core issue, but it is not fool proof and can have some nasty effects, so I don't recommend using it if you want to be sure the client gets your message.
You would have the exact same problem if you were just using the "http" module from Node.js core directly and not Express.
Typically, the suggested pattern to get around this problem is two things 1) open an issue with Node.js to see what can be done or 2) at the site you are reading from the request, even if you have an error, finish reading the entire request before calling next(error) (this is what the body-parser and multiparty modules do).

Thanks for the quick response :) That clarifies things a bit; I will look further into this issue.
About 2 - it would be better to terminate the request immediately, since there may be a lot of data awaiting. If the error occurs at 5% of upload, then keeping user waiting for the next 95% is undesirable for me.
Could you write more about those "nasty effects"? Is this about memory leaks or something else?

would be better to terminate the request immediately, since there may be a lot of data awaiting. If the error occurs at 5% of upload, then keeping user waiting for the next 95% is undesirable for me.

Right, but using the pattern in finalhandler, you'll just end up waiting for that 95% anyway, so you are not saving any time by copying that pattern :) The only successful way Node.js give you to terminate at the 5% is using req.destroy(), which won't send your response (which leads back to possibility a bug in Node.js core).

Could you write more about those "nasty effects"? Is this about memory leaks or something else?

It is with things like calling .unpipe() in a context where you do not know what is piped--that destination may now never close some related file descriptor and you'll end up with a fd leak or other really hard to notice things. That's why I would recommend handling this condition at the site you are reading from the request, since you'll know what you need to clean up.

Right, but using the pattern in finalhandler, you'll just end up waiting for that 95% anyway, so you are not saving any time by copying that pattern :)

I've missed that :/ Just tested this with a 3.5GB file, which takes 14s to upload. When I invoke an error after 1GB is uploaded, it is printed immediately (after 4s or so), but the request still takes full 14s to end.
It seems that the best way for me is to save an info about error somewhere on the server side, then destroy the request, and handle this on client side by querying the server with a request id or something.
Or just abandon the whole "immediate termination" idea and just let users reap what they've sown by trying to upload too much data.
Thanks again for help!

It's no problem! I'm going to close this issue for now, but please feel free to reach out! Also, let me know if there is something you wanted to keep this issue for and I can definitely reopen.


RESTEASY-1700

Cleaned up version of https://github.com/resteasy/Resteasy/pull/1244

@NicoNes , thanks for your feedback. I would have to verify, but I think the "Content-Length" header is set at the web/undertow level, same for the "Transfer-Encoding: chunked".
My proposal would be to implement the changes here differently, that is by tracking when the inputstream is actually read and using that to throw the exception. Also note that in case the stream is closed, the response is also marked as closed automatically, hence the call to getEntity will fail. Please check my latest changes here (I also had to rebase the PR) and let me know what you think.

Thanks @NicoNes


failing on protractor test/protractor-conf.js

Hello -
So I'm new to development in general and the setup for this was a trip. Looked like I got it running - I could use [npm start]  and [npm test] but [npm run protractor] fails as shown.
I have an updated chrome, and the server and Karma will both run correctly (no visible errors and Karma is green).
I'm running this in Cygwin on 64-bit Windows 8.
The debug log  will follow.
Here are my commands and the resulting script:
$ npm run protractor

<EMAIL_ADDRESS>preprotractor C:\angular-phonecat
npm run update-webdriver
<EMAIL_ADDRESS>preupdate-webdriver C:\angular-phonecat
npm install
<EMAIL_ADDRESS>postinstall C:\angular-phonecat
bower install
<EMAIL_ADDRESS>update-webdriver C:\angular-phonecat
webdriver-manager update

selenium standalone is up to date.
chromedriver is up to date.

<EMAIL_ADDRESS>protractor C:\angular-phonecat
protractor test/protractor-conf.js

PID: 12540 (capability: chrome #1)
Using ChromeDriver directly...
[launcher] Runner Process Exited With Error Code: 1
npm ERR<EMAIL_ADDRESS>protractor: protractor test/protractor-conf.js
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>protractor script.
npm ERR! This is most likely a problem with the angular-phonecat package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     protractor test/protractor-conf.js
npm ERR! You can get their info via:
npm ERR!     npm owner ls angular-phonecat
npm ERR! There is likely additional logging output above.
npm ERR! System Windows_NT 6.2.9200
npm ERR! command "C:\Program Files\nodejs\node.exe" "C:\Program Files\nodejs\node_modules\npm\bin\npm-cli.js" "run" "protractor"
npm ERR! cwd C:\angular-phonecat
npm ERR! node -v v0.10.28
npm ERR! npm -v 1.4.9
npm ERR! code ELIFECYCLE
debug log:
0 info it worked if it ends with ok
1 verbose cli [ 'C:\Program Files\nodejs\node.exe',
1 verbose cli   'C:\Program Files\nodejs\node_modules\npm\bin\npm-cli.js',
1 verbose cli   'run',
1 verbose cli   'protractor' ]
2 info using<EMAIL_ADDRESS>3 info using<EMAIL_ADDRESS>4 verbose run-script [ 'preprotractor', 'protractor', 'postprotractor' ]
5 info preprotractor<EMAIL_ADDRESS>6 verbose unsafe-perm in lifecycle true
7 info protractor<EMAIL_ADDRESS>8 verbose unsafe-perm in lifecycle true
9 info<EMAIL_ADDRESS>Failed to exec protractor script
10 error<EMAIL_ADDRESS>protractor: protractor test/protractor-conf.js
10 error Exit status 1
11 error Failed at the<EMAIL_ADDRESS>protractor script.
11 error This is most likely a problem with the angular-phonecat package,
11 error not with npm itself.
11 error Tell the author that this fails on your system:
11 error     protractor test/protractor-conf.js
11 error You can get their info via:
11 error     npm owner ls angular-phonecat
11 error There is likely additional logging output above.
12 error System Windows_NT 6.2.9200
13 error command "C:\Program Files\nodejs\node.exe" "C:\Program Files\nodejs\node_modules\npm\bin\npm-cli.js" "run" "protractor"
14 error cwd C:\angular-phonecat
15 error node -v v0.10.28
16 error npm -v 1.4.9
17 error code ELIFECYCLE
18 verbose exit [ 1, true ]
What is going on? Is it something simple?

./node_modules/.bin/protractor test/protractor-conf.js
Worked for me in mac

I meet the same problem as well  when following angularjs tutorial at step 3 :
https://docs.angularjs.org/tutorial/step_03
And I solve the problem by @hawaiikaos 's method
./node_modules/.bin/protractor test/protractor-conf.js
But why ?

OSX 10.95
The following solution help me:
https://github.com/angular/protractor/issues/906

Yeah i got the same problem but what i did is just unzipping the downloaded Chromedriver and i run it directly (chromedriver.exe)... This solved my problem :)

I am running the angular-phonecat example on Mac and get this error for 'npm run protractor' command
[launcher] Running 1 instances of WebDriver
[launcher] Process exited with error code 1
/Users/macadmin/angular-phonecat/node_modules/protractor/node_modules/selenium-webdriver/lib/goog/async/nexttick.js:39
goog.global.setTimeout(function() { throw exception; }, 0);
^
Error: Timed out waiting for the WebDriver server at http://IP/wd/hub
at Error (native)
at onResponse (/Users/macadmin/angular-phonecat/node_modules/protractor/node_modules/selenium-webdriver/http/util.js:87:11)
at /Users/macadmin/angular-phonecat/node_modules/protractor/node_modules/selenium-webdriver/http/util.js:42:21
There is this forum (Corrupted selenium server jar #656) but don't see a precise solution.
I am also getting different ports in IP.

in protractor-conf.js
change chromeOnly to directConnect

It might help if you add "sudo" to the command
sudo npm run protractor

In my case I had just got ubuntu 16.04 installed on my system and had not installed the jre package. Once I did, the protractor started working. As people mentioned earlier, please run npm start and npm protractor start in a different window. The instructions on quick start guide seems outdated.
Please check your java path and if it is accessible to start a selenium web server as well.
Hope it helps someone.

The instructions on quick start guide seems outdated.

@bharathkallurs, what guide are you referring to? If something is outdated, we should fix it :smiley:

Sure Georgios.
I am referring to
https://docs.angularjs.org/tutorial
Pardon me for mentioning outdated, instead of not highlighted for npm start.
The point I was talking about is simultaneously starting npm start and npm
run protractor.
It is mentioned under "Note" section in the guide which might go a miss at
times.
I was thinking if it could be added directly in one of those paragraph text?
Please let me know if there is anything I could do to fix it? :)
On Mon, Nov 21, 2016 at 1:41 PM, Georgios Kalpakas<EMAIL_ADDRESS>
wrote:

The instructions on quick start guide seems outdated.
@bharathkallurs https://github.com/bharathkallurs, what guide are you
referring to? If something is outdated, we should fix it 😃
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/angular/angular-phonecat/issues/168#issuecomment-261870364,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACH_Zv7bUbUz1UQljprCO72Md069JVoyks5rAVIkgaJpZM4CEEmP
.

--
With Best Regards
Bharath S Kallur

I see. We were trying to keep the flow simple (without clattering it with everything that might go wrong), but still provide a 'Troubleshooting" section with known issue. But I guess, since JDK is a pre-requisite in this setup, we could mention it in the "Environment setup section".
The tutorial content lives in the main AngularJS repo, so you can submit a pull request with any improvements you can come up with. There is also a useful "Improve this doc" button on the top-right of each page that will set you up for quick edits automatically.
I will also add a note in README file here. Thx for pointing this out.

Thanks for addressing Georgios. I will check the points you mentioned.
Appreciate the help.
On Mon, Nov 21, 2016, 15:50 Georgios Kalpakas<EMAIL_ADDRESS>wrote:

I see. We were trying to keep the flow simple (without clattering it with
everything that might go wrong), but still provide a 'Troubleshooting"
section with known issue. But I guess, since JDK is a pre-requisite in this
setup, we could mention it in the "Environment setup section".
The tutorial content lives in the main AngularJS repo, so you can submit a
pull request with any improvements you can come up with. There is also a
useful "Improve this doc" button on the top-right of each page that will
set you up for quick edits automatically.
I will also add a note in README file here. Thx for pointing this out.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/angular/angular-phonecat/issues/168#issuecomment-261898213,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACH_ZuVcHnu8OO8VPSUVUTdT0q2yd59uks5rAXBdgaJpZM4CEEmP
.

For reference: 788ddca

I got the same problem and solved it this way:
npm install -g protractor
webdriver-manager update
protractor testfolder/protractor.conf.js


Unable to install curl_cffi

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

[X] I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

[X] I'm asking a question and not reporting a bug or requesting a feature
[X] I've looked through the README
[X] I've verified that I have updated yt-dlp to nightly or master (update instructions)
[X] I've searched known issues and the bugtracker for similar questions including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue

Please make sure the question is worded well enough to be understood
I am trying to install curl_cffi in order to get around Vimeo's new TLS fingerprinting anti-bot protection. I have run the command pipx install 'yt-dlp[default,curl_cffi]' --force', which gives the output:
Installing to existing venv 'yt-dlp'
⚠  Note: yt-dlp was already on your PATH at /opt/homebrew/bin/yt-dlp
  installed package yt-dlp 2024.7.2, installed using Python 3.12.4
  These apps are now globally available
    - yt-dlp
  These manual pages are now globally available
    - man1/yt-dlp.1
⚠  Note: '/Users/username-hidden/.local/bin' is not on your PATH environment variable. These apps will not be globally accessible until your PATH is updated. Run `pipx ensurepath` to automatically add it,
    or manually modify your PATH in your shell's config file (e.g. ~/.bashrc).
done! ✨ 🌟 ✨

From this output, I understand that curl_cffi would have been installed. However, running yt-dlp --list-impersonate-targets -vU does not show it.
I intend to use --impersonate chrome but I am stuck at curl_cffi installation. Any help would be greatly appreciated. Thank you.
Provide verbose output that clearly demonstrates the problem

[X] Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
[X] If using API, add 'verbose': True to YoutubeDL params instead
[X] Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output
[debug] Command-line config: ['--list-impersonate-targets', '-vU']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version<EMAIL_ADDRESS>from yt-dlp/yt-dlp [93d33cb29] (pip)
[debug] Python 3.12.4 (CPython arm64 64bit) - macOS-14.5-arm64-arm-64bit (OpenSSL 3.3.1 4 Jun 2024)
[debug] exe versions: ffmpeg 7.0.1 (setts), ffprobe 7.0.1
[debug] Optional libraries: Cryptodome-3.20.0, brotli-1.1.0, certifi-2024.06.02, mutagen-1.47.0, requests-2.32.3, sqlite3-3.46.0, urllib3-2.2.2, websockets-12.0
[debug] Proxy map: {}
[debug] Request Handlers: urllib, requests, websockets
[debug] Loaded 1831 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version<EMAIL_ADDRESS>from yt-dlp/yt-dlp
yt-dlp is up to date<EMAIL_ADDRESS>from yt-dlp/yt-dlp)
[info] Available impersonate targets
Client   OS   Source
---------------------------------------
Chrome   -    curl_cffi (not available)
Edge     -    curl_cffi (not available)
Safari   -    curl_cffi (not available)

⚠  Note: yt-dlp was already on your PATH at /opt/homebrew/bin/yt-dlp
⚠  Note: '/Users/username-hidden/.local/bin' is not on your PATH environment variable. These apps will not be globally accessible until your PATH is updated. Run pipx ensurepath to automatically add it, or manually modify your PATH in your shell's config file (e.g. ~/.bashrc).

 Did you try running pipx ensurepath like the note says?
Another alternative is the macos pyinstaller-bundled binary, which includes curl_cffi

Thanks for your response, @bashonly. How do I check which installation is being called when I run yt-dlp?
Yes, I did run pipx ensurepath, and it returned the following output.
/Users/username-hidden/.local/bin has been been added to PATH, but you need to open a new terminal or re-login for this PATH change to take effect. Alternatively, you can source your shell's config file
    with e.g. 'source ~/.bashrc'.

You will need to open a new terminal or re-login for the PATH changes to take effect. Alternatively, you can source your shell's config file with e.g. 'source ~/.bashrc'.

I ran yt-dlp --list-impersonate-targets -vU in a new Terminal window thereafter but it still returned the same result.
It might be that Homebrew's installation in being called when I run yt-dlp. Is there a way I can check? Should I uninstall Homebrew's version of yt-dlp and try running the command?
Thank you for your help.

How do I check which installation is being called when I run yt-dlp?

If you run which yt-dlp should show the path of the executable that's called when you run yt-dlp.  (If which isn't available on your system, try command -v yt-dlp)

Should I uninstall Homebrew's version of yt-dlp and try running the command?

If you can't get the PATH thing figured out, this would be a surefire way of solving it

Thank you once again for the reply, @bashonly.
Yes, Homebrew's version of yt-dlp is being called when I run yt-dlp. which yt-dlp gives me:
/opt/homebrew/bin/yt-dlp

Can I change yt-dlp to run from pipx instead of Homebrew? I am not sure how to do this (I guess it requires mentioning the absolute path of pipx's version of yt-dlp when calling yt-dlp, but I don't know how to find this path.) Could you please help me out?
Thank you!

Can I change yt-dlp to run from pipx instead of Homebrew? I am not sure how to do this (I guess it requires mentioning the absolute path of pipx's version of yt-dlp when calling yt-dlp, but I don't know how to find this path.) Could you please help me out?

If pipx ensurepath isn't working, then you'll have to manually modify your PATH environment variable.
First, you need to figure out which shell your terminal uses by default. On newer versions of macOS, it's zsh. On older macOS, it's an ancient version of bash. Run echo $SHELL to determine which.
If you're using zsh, then run this command:
echo 'export PATH="${HOME}/.local/bin:${PATH}"' >> ~/.zshrc

If bash, then this:
echo 'export PATH="${HOME}/.local/bin:${PATH}"' >> ~/.bash_profile

Make sure to copy/paste verbatim, since the quoting etc needs to be exact.
Then log out and log back in to make sure the changes take effect.

This did it! Thank you very much, @bashonly!
I am using Z shell, so copying and pasting the Z shell command did it for me.
Could you please explain what the command is doing? I understand that the command is modifying the PATH variable in the Z shell source file. You are adding /Users/username-hidden/.local/bin to PATH. Is that right?
Is this (/Users/username-hidden/.local/bin) where pipx's yt-dlp resides? I had a look inside the directory (ls) and it contains yt-dlp.
Thank you for your help! 😊

Understood, @bashonly. Thank you very much. 🙂


[BUG] Capacity overflowing uint32

Description
I am trying to use bytes as my cost function, but the policy is silently casting the capacity from an int to a uint32. This effectively makes the max cache size 4GB, where values over that overflow and get a ~random size
To Reproduce
Set the capacity to math.MaxUint32 + 1 every entry will be rejected.
Expected behavior
Normal caching
Environment

OS: macos

Additional context

https://github.com/maypok86/otter/blob/9d61c676ef185402bac4a291b46ee6b72c83bebf/internal/core/cache.go#L142

Okay, I'll fix it a bit later. But I should immediately note that storing 4 GB of data in an onheap cache is hardly a good idea.

I have 32GB of RAM to use as a ~block cache for s3/gcs, why is >4GB a bad idea?

Check it out, but there is a possibility that the garbage collector will run with long pauses due to the many entries that it has to scan every cycle. Usually onheap caches are not very large. Up to 10 million entries or 1 GB (approximate numbers).

You can store more, but it's worth checking the behavior on your data.

My chunk size is 4-8MB, so 4GB ends up being ~1k entries of ~[]bytes.

being ~1k entries of ~[]bytes.

Oh, well, then everything should be fine. I should have warned you, sorry.

@Alfus I believe I've fixed it. Please try it from the main branch, if everything is fine, then I will release a new version tomorrow.

I have published a release with a fix. If something is wrong, then write.


Android: Scrolling not working on Hotspot Checklist

When viewing the hotspot checklist on Andriod 11 on a pixel 4, you cannot scroll left and right to see completed tasks.

Reproducible on Pixel 3.

The fix for this is not working as expected, will need further debugging today. Also noted that the interim fix is to switch from carousel to BottomSheetFlatlist, which does not accept opacity with svgs. The second interim fix here is to use solid svgs, then regenerate opacities with pngs.


Update requirements.txt

i forgot to remove it as the main domain of Turkish123 used Cloudflare to let the browser late 5 so i tested cloudscraper but it didn't work too so i just checked some different domains of Turkish123 and it worked so this was just testing that i forgot to remove.


new: Support PowerShell profiles.

@rotu Is this how you use pwsh?

@rotu Is this how you use pwsh?

Roughly, yes.

I only use Documents\PowerShell\Profile.ps1, not Microsoft.PowerShell_profile.ps1. For example, the latter is not respected by PowerShell running in VSCode.
You may want to use [IO.Path]::PathSeparator instead of a literal ; or :.
I think env::var("PSHOME") is not right. Powershell sets $PSHOME, and I think you’re looking at $env:PSHOME.

1/3 - Basing on this doc: https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_profiles?view=powershell-7.4#profile-types-and-locations
2 - I may be blind, but I couldn't find a constant in Rust std for the path delimiter.

1/3 - Basing on this doc: https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_profiles?view=powershell-7.4#profile-types-and-locations

Careful. “Host” there means “Application hosting the shell”, not “host machine”
Also, in PowerShell, $PSHOME and $HOME are variables but not environment variables. $env:PSHOME and $env:HOME are environment variables. I would not expect them to be set (maybe they are but that page doesn’t tell me)

2 - I may be blind, but I couldn't find a constant in Rust std for the path delimiter.

I’m saying don’t split the Windows and non-Windows cases. Leave is as [IO.Path]::PathSeparator in the shell script and let PowerShell resolve this to the particular character. https://powershell.org/2019/02/tips-for-writing-cross-platform-powershell-code/

Careful. “Host” there means “Application hosting the shell”, not “host machine”
Also, in PowerShell, $PSHOME and $HOME are variables but not environment variables. $env:PSHOME and $env:HOME are environment variables. I would not expect them to be set (maybe they are but that page doesn’t tell me)

Well that's super confusing. I don't see any clarification in the docs either, but not supporting PSHOME is probably fine.

I’m saying don’t split the Windows and non-Windows cases. Leave is as [IO.Path]::PathSeparator in the shell script and let PowerShell resolve this to the particular character. https://powershell.org/2019/02/tips-for-writing-cross-platform-powershell-code/

Ah in the PS code, thought you meant Rust side.

This is what the relevant part of my PowerShell profile looks like:
$env:PROTO_HOME = Join-Path $HOME '.proto'
$env:PATH = @(
    (Join-Path $env:PROTO_HOME 'shims'),
    (Join-Path $env:PROTO_HOME 'bin'),
    $env:PATH
) -join [IO.PATH]::PathSeparator

$env:PATH = ($env:PATH.split([IO.Path]::PathSeparator) | Select-Object -Unique) -join [IO.PATH]::PathSeparator

@rotu Thanks thats super useful. Updated.


check pandas on the notebook interpreter

For #8227

[x] Pull request represents a single change (i.e. not fixing disparate/unrelated things in a single PR)
[x] Title summarizes what is changing
[x] Has a news entry file (remember to thank yourself!)
[ ] Appropriate comments and documentation strings in the code
[ ] Has sufficient logging.
[ ] Has telemetry for enhancements.
[ ] Unit tests & system/integration tests are added/updated
[ ] Test plan is updated as appropriate
[ ] package-lock.json has been regenerated by running npm install (if dependencies have changed)
[ ] The wiki is updated with any design decisions/details.

Codecov Report

Merging #8863 into master will increase coverage by 0.29%.
The diff coverage is 6.66%.

@@            Coverage Diff             @@
##           master    #8863      +/-   ##
==========================================
+ Coverage   57.63%   57.92%   +0.29%
==========================================
  Files         527      527
  Lines       27480    27480
  Branches     4126     4132       +6
==========================================
+ Hits        15839    15919      +80
+ Misses      10770    10690      -80
  Partials      871      871

Impacted Files
Coverage Δ

...ent/datascience/data-viewing/dataViewerProvider.ts
23.07% <6.66%> (ø)
:arrow_up:

src/client/language/characters.ts
37.14% <0%> (+2.85%)
:arrow_up:

src/client/common/utils/version.ts
81.48% <0%> (+3.7%)
:arrow_up:

src/client/providers/providerUtilities.ts
88.88% <0%> (+5.55%)
:arrow_up:

src/client/datascience/cellFactory.ts
71.25% <0%> (+12.5%)
:arrow_up:

src/client/api.ts
92.85% <0%> (+14.28%)
:arrow_up:

...cience/jupyter/jupyterDebuggerNotInstalledError.ts
66.66% <0%> (+16.66%)
:arrow_up:

src/client/common/utils/icons.ts
100% <0%> (+16.66%)
:arrow_up:

...ience/jupyter/jupyterDebuggerRemoteNotSupported.ts
80% <0%> (+20%)
:arrow_up:

...t/datascience/jupyter/jupyterDataRateLimitError.ts
75% <0%> (+25%)
:arrow_up:

... and 4 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d0fbdfd...98f774a. Read the comment docs.


Reduce number of NumberFormatter and EtherNumberFormatter defined, if possible

Do a sweep, especially in CurrencyFormatter.swift and EtherNumberFormatter.swift.
Generally, we want to round down to 4 decimal places for crypto balances (but for transaction amounts, we display the full value) and round down to 2 decimal places for fiat.
Can you extract the 4 decimal places to a constant? (the other project fork will want 5 instead)
Take care with refactoring because:

amounts are sensitive
the other project fork is affected by this too. We'll probably copy these changes over to the other repo.

Add test cases with something like this:
        let values: [(BigInt, String)] = [
            (BigInt("1123450000000000000"), "1.1234"),
            (BigInt("1123440000000000000"), "1.1234"),
            (BigInt("1123490000000000000"), "1.1234"),
            (BigInt("1123499000000000000"), "1.1234"),
            (BigInt("1123990000000000000"), "1.1239"),

            (BigInt("1123850000000000000"), "1.1238"),
            (BigInt("1123840000000000000"), "1.1238"),
            (BigInt("1123890000000000000"), "1.1238"),
            (BigInt("1123899000000000000"), "1.1238"),
        ]
        for each in values {
            //EtherNumberFormatter.short.string(from: each.0, decimals: 18)
        }

@hboon Related to CurrencyFormatter you mean to move this
let formatter = NumberFormatter()
formatter.minimumFractionDigits = 4
formatter.maximumFractionDigits = 4
formatter.roundingMode = .down

to constant and other formatters in CurrencyFormatter should use it as base?

@hboon  what do i need to sweep in EtherNumberFormatter? there is only full/short formatter

@hboon what do i need to sweep in EtherNumberFormatter? there is only full/short formatter

I supposed you just did and checked that there is no duplicate and unnecessary formatters in those 2 files :)

@hboon Related to CurrencyFormatter you mean to move this
let formatter = NumberFormatter()
formatter.minimumFractionDigits = 4
formatter.maximumFractionDigits = 4
formatter.roundingMode = .down

to constant and other formatters in CurrencyFormatter should use it as base?

Just extract the 4 in EtherNumberFormatter.swift into a constant and the 2 in CurrencyFormatter. No need to change the formatters themselves.
There's a CurrencyFormatter.shortFormatter which seems to be used for tokens/crypto, can you see if that can be replaced with something from EtherNumberFormatter or otherwise moved into EtherNumberFormatter?
I guess the goal is make this easier and harder to miss a spot:

if we want to change 4 -> 5 for ether/tokens
change how fiat is displayed

@hboon What if i create factory extension for NumberFormatter with formatters for usd/short eth. and then i remove shortFormatter, usdFormatter, instead i will use CurrencyFormatter.formatter.usd or CurrencyFormatter.formatter.short. does it make sens?

making this we even can remove CurrencyFormatter and do formatting just create let x = NumberFormatter().usd


add "create path" operation to Azure Storage data_lake (using Pipeline architecture)

This adds create_path operation to the data_lake crate using the pipeline architecture and adds examples.
Any early feedback appreciated.
Issue #356

Doesn't compile yet

Todo list:

Fix all my Rust issues (this PR) - calling all Rust experts
Make sure use of pipeline architecture matches other crates, e.g. Cosmos (this PR)
Take TokenCredential instead of bearer token directly in client (this PR or follow-up PR)
Expose separate file and directory clients with high-level operations like 'rename file' (follow-up PR)

Follow-up PRs: #451 and #491


[opencv] build failure

Operating system
Windows 11 23H2 x64
Compiler
Visual Studio 2022 17.9.6
Steps to reproduce the behavior
.\vcpkg install opencv[contrib,cuda,cudnn,eigen,ffmpeg,opengl,openmp]:x64-windows

Failure logs
PS D:\Data\vcpkg> .\vcpkg install opencv[contrib,cuda,cudnn,eigen,ffmpeg,opengl,openmp]:x64-windows
Computing installation plan...
The following packages will be built and installed:
<EMAIL_ADDRESS>
<EMAIL_ADDRESS>Additional packages (*) will be modified to complete this operation.
Detecting compiler hash for triplet x64-windows...
-- Automatically setting %HTTP(S)_PROXY% environment variables to "<IP_ADDRESS>:10809".
Compiler found: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.39.33519/bin/Hostx64/x64/cl.exe
Restored 0 package(s) from C:\Users\CYH\AppData\Local\vcpkg\archives in 1.27 ms. Use --debug to see more details.
Installing 1/2<EMAIL_ADDRESS>Building<EMAIL_ADDRESS>-- Using cached opencv4-e5e1a3bfdea96bebda2ad963bc8f6cf17930aef7.patch.
-- Using cached opencv-opencv-4.8.0.tar.gz.
-- Cleaning sources at D:/Data/vcpkg/buildtrees/opencv4/src/4.8.0-2bf495557d.clean. Use --editable to skip cleaning for the packages you specify.
-- Extracting source D:/Data/vcpkg/downloads/opencv-opencv-4.8.0.tar.gz
-- Applying patch 0001-disable-downloading.patch
-- Applying patch 0002-install-options.patch
-- Applying patch 0003-force-package-requirements.patch
-- Applying patch 0004-fix-eigen.patch
-- Applying patch 0005-fix-policy-CMP0057.patch
-- Applying patch 0006-fix-uwp.patch
-- Applying patch 0008-devendor-quirc.patch
-- Applying patch 0009-fix-protobuf.patch
-- Applying patch 0010-fix-uwp-tiff-imgcodecs.patch
-- Applying patch 0011-remove-python2.patch
-- Applying patch 0012-fix-zlib.patch
-- Applying patch 0015-fix-freetype.patch
-- Applying patch 0017-fix-flatbuffers.patch
-- Applying patch 0019-missing-include.patch
-- Applying patch 0020-fix-compat-cuda12.2.patch
-- Applying patch 0021-static-openvino.patch
-- Applying patch D:/Data/vcpkg/downloads/opencv4-e5e1a3bfdea96bebda2ad963bc8f6cf17930aef7.patch
-- Applying patch 0022-fix-supportqnx.patch
-- Using source at D:/Data/vcpkg/buildtrees/opencv4/src/4.8.0-2bf495557d.clean
-- Using cached opencv-cache/tiny_dnn/adb1c512e09ca2c7a6faef36f9c53e59-v1.0.0a3.tar.gz.
-- Using cached opencv-cache/nvidia_optical_flow/a73cd48b18dcc0cc8933b30796074191-edb50da3cf849840d680249aa6dbef248ebce2ca.zip.
-- Using cached opencv-opencv_contrib-4.8.0.tar.gz.
-- Cleaning sources at D:/Data/vcpkg/buildtrees/opencv4/src/4.8.0-8d756cdf2d.clean. Use --editable to skip cleaning for the packages you specify.
-- Extracting source D:/Data/vcpkg/downloads/opencv-opencv_contrib-4.8.0.tar.gz
-- Applying patch 0007-fix-hdf5.patch
-- Applying patch 0016-fix-freetype-contrib.patch
-- Applying patch 0018-fix-depend-tesseract.patch
-- Applying patch 0019-fix-ogre-dependency.patch
-- Using source at D:/Data/vcpkg/buildtrees/opencv4/src/4.8.0-8d756cdf2d.clean
-- Using cached opencv-cache/wechat_qrcode/238e2b2d6f3c18d6c3a30de0c31e23cf-detect.caffemodel.
-- Using cached opencv-cache/wechat_qrcode/cbfcd60361a73beb8c583eea7e8e6664-sr.caffemodel.
-- Using cached opencv-cache/wechat_qrcode/6fb4976b32695f9f5c6305c19f12537d-detect.prototxt.
-- Using cached opencv-cache/wechat_qrcode/69db99927a70df953b471daaba03fbef-sr.prototxt.
-- Using cached opencv-cache/xfeatures2d/boostdesc/0ea90e7a8f3f7876d450e4149c97c74f-boostdesc_bgm.i.
-- Using cached opencv-cache/xfeatures2d/boostdesc/232c966b13651bd0e46a1497b0852191-boostdesc_bgm_bi.i.
-- Using cached opencv-cache/xfeatures2d/boostdesc/324426a24fa56ad9c5b8e3e0b3e5303e-boostdesc_bgm_hd.i.
-- Using cached opencv-cache/xfeatures2d/boostdesc/202e1b3e9fec871b04da31f7f016679f-boostdesc_binboost_064.i.
-- Using cached opencv-cache/xfeatures2d/boostdesc/98ea99d399965c03d555cef3ea502a0b-boostdesc_binboost_128.i.
-- Using cached opencv-cache/xfeatures2d/boostdesc/e6dcfa9f647779eb1ce446a8d759b6ea-boostdesc_binboost_256.i.
-- Using cached opencv-cache/xfeatures2d/boostdesc/0ae0675534aa318d9668f2a179c2a052-boostdesc_lbgm.i.
-- Using cached opencv-cache/xfeatures2d/vgg/e8d0dcd54d1bcfdc29203d011a797179-vgg_generated_48.i.
-- Using cached opencv-cache/xfeatures2d/vgg/7126a5d9a8884ebca5aea5d63d677225-vgg_generated_64.i.
-- Using cached opencv-cache/xfeatures2d/vgg/7cd47228edec52b6d82f46511af325c5-vgg_generated_80.i.
-- Using cached opencv-cache/xfeatures2d/vgg/151805e03568c9f490a5e3a872777b75-vgg_generated_120.i.
-- Using cached opencv-cache/data/7505c44ca4eb54b4ab1e4777cb96ac05-face_landmark_model.dat.
-- Found external ninja('1.11.0').
-- Configuring x64-windows
-- Building x64-windows-dbg
CMake Error at scripts/cmake/vcpkg_execute_build_process.cmake:134 (message):
Command failed: D:/Data/vcpkg/downloads/tools/cmake-3.29.0-windows/cmake-3.29.0-windows-i386/bin/cmake.exe --build . --config Debug --target install -- -v -j9
Working Directory: D:/Data/vcpkg/buildtrees/opencv4/x64-windows-dbg
See logs for more information:
D:\Data\vcpkg\buildtrees\opencv4\install-x64-windows-dbg-out.log

Call Stack (most recent call first):
installed/x64-windows/share/vcpkg-cmake/vcpkg_cmake_build.cmake:74 (vcpkg_execute_build_process)
installed/x64-windows/share/vcpkg-cmake/vcpkg_cmake_install.cmake:16 (vcpkg_cmake_build)
ports/opencv4/portfile.cmake:479 (vcpkg_cmake_install)
scripts/ports.cmake:175 (include)
error: building opencv4:x64-windows failed with: BUILD_FAILED
Elapsed time to handle opencv4:x64-windows: 18 min
Please ensure you're using the latest port files with git pull and vcpkg update.
Then check for known issues at:
https://github.com/microsoft/vcpkg/issues?q=is%3Aissue+is%3Aopen+in%3Atitle+opencv4
You can submit a new issue at:
https://github.com/microsoft/vcpkg/issues/new?title=[opencv4]+Build+error+on+x64-windows&body=Copy+issue+body+from+D%3A%2FData%2Fvcpkg%2Finstalled%2Fvcpkg%2Fissue_body.md
install-x64-windows-dbg-out.log
Additional context
No response

install-x64-windows-dbg-out.log

@cyh19910216, I can not reproduce your error on my local machine:
Elapsed time to handle opencv:x64-windows: 765 ms
Total install time: 3.1 h
opencv provides CMake targets:

  # this is heuristically generated, and may not be correct
  find_package(OpenCV CONFIG REQUIRED)
  # note: 59 additional targets are not displayed.
  target_link_libraries(main PRIVATE opencv_ml opencv_dnn opencv_dpm opencv_hdf)

Please make sure you are using the latest port files with git pull and vcpkg update. Delete all opencv caches and try again.

@cyh19910216, I can not reproduce your error on my local machine:@cyh19910216，我无法在本地计算机上重现您的错误：
Elapsed time to handle opencv:x64-windows: 765 ms
Total install time: 3.1 h
opencv provides CMake targets:

  # this is heuristically generated, and may not be correct
  find_package(OpenCV CONFIG REQUIRED)
  # note: 59 additional targets are not displayed.
  target_link_libraries(main PRIVATE opencv_ml opencv_dnn opencv_dpm opencv_hdf)

Please make sure you are using the latest port files with git pull and vcpkg update. Delete all opencv caches and try again.

Thank you! But I fail again.
I have used "git pull" and "vcpkg update", and deleted the folders "opencv", "opencv4", "opencv-cache" and "temp".
But it fails again.
CPU: Intel Core i5-1155G7
GPU: NVIDIA GeForce MX450
CUDA: 12.4

The latest version of OpenCV available in vcpkg is 4.8.0.
OpenCV version 4.8.0 supports the latest CUDA version, which is 12.0.
so, use CUDA  12.0