Bugfix and readability improvement

A combination of small updates I have been testing over the past few days.
This PR includes the removal of set_pythonpath, which has been replaced by hooks #81. No users should be negatively affected.
This is not ready to merge yet. I will use this for a few more days. Any feedback is welcome!

I have not encountered any changes in behaviour after using it for a while. I think it is ready for review.

I happen to be reading this comment in the nvim-lspconfig repo. https://github.com/neovim/nvim-lspconfig/issues/299#issuecomment-718458638
I think Pylance has not been working outside of VSCode since 2020, so I assume it is safe to remove entirely?

Seems to work fine for me also, did a testrun here and didnt notice anything breaking.
And yes, please remove Pylance when you are at it. :)

I rebased to clean up some commits, and I think it can be merged now if everything looks good.
I am still looking at a few small things, but those can be added in another PR.

I just noticed that I forgot to update the documentation. Will do that first.

Now the docs should also be up-to-date.

Ok thank you. What do you say, we merge this in or do you want to add more changes?

I think you can merge it. 👍


Change Zookeeper path defaults

Currently the default leader path is "/leader" and the default election path is "/election". These defaults make sense when Zookeeper is being used for Alluxio only, but in real-world deployments, there will often be multiple services making use of Zookeeper. Change the defaults to something like "/alluxio/leader" and "/alluxio/election" so that there's a better chance that users will not have to change them. It is especially annoying to use non-default paths because clients need to be configured with the leader path, and this configuration can't be specified through a centralized cluster config, or by adding something to the URI.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/3697/
Test PASSed.

alluxio-bot, merge this please


Android support

example doesn't work in android.

Yes , seems like the plugin does not work on Android at all..using RN 0.22...

Hey guys, I'm tied down with work for the next few weeks so won't be able to work on adding android support. There shouldn't be much getting in the way of android support working though as it's all js. PRs welcome :)

Hey @rick-li,
I just accepted and released a PR that ads android support. Note that the module relies on LayoutAnimation, which needs to be enabled on Anrdroid with:
var {UIManager} = require('react-native');
UIManager.setLayoutAnimationEnabledExperimental && UIManager.setLayoutAnimationEnabledExperimental(true);```


Language-specific guides request

Problem description

Problem location

I saw a problem on the following URL:

I couldn't find the information I wanted. I expected to find it near the following URL:

Other:

Project version(s) affected

Suggestions for a fix

would you mind providing any details?

closing as we need more details


Consider updating tags from EXIF when they change on source file

Would be nice to update ALL tags on replacePhoto and not only the mandatory checksum tag as FLICKR maintains the tags from the first load.
It would require accessing EXIF information from file...

I'm surprised flickr does not update tags based on contents of replaced file? If not, seems like this functionality is practically required if one is to update pictures in-place. To update a photo with changed tags locally and not have flickr mirror that would be sad. If there's no convenient EXIF/IPTC/XMP reading library for python, one could always parse exiftool output, which I see the documentation says the optional raw file conversion option uses already).
Confusingly I find that between digikam and darktable, my tags appear in jpeg files repeated under the following six exiftool output labels: Subject, Tags List, Last Keyword XMP, Hierarchical Subject, Catalog Sets, Keywords. Of these, it seems like the "Keywords" IPTC field is the most standard. And/or possibly the XMP "subject" field.

@timtoo

Thanks for the heads up! Will try to work on it in the coming month or so.
Tried to place the following remark on your repository timtoo/FlickrBox but it's not accepting Issues. So here hoes:

As the timtoo/FlickrBox project is in its infancy take a look at two alternatives where I'd invite you to contribute to:

flickr-uploader ... on its version 2.8.6 using flickrapi with plenty of functionality
flickr-rsync ... more towards the line of the concept of FlicrkBox of copy files over. It follows the idea of rsync (remore sync two folders)... I kicked off the migration work to flickrapi and compatibility with Python 3... but stil work in progress.


email_mirror: Create attachments as the message sender.

When the email mirror gateway is sending messages "as" a user (as triggered by having access to the missed-message email address), attachments were still created as the Email Gateway bot.  Since the sender (the end-user) was not the owner of those attachments (the gateway bot), nor were they referenced yet anywhere, this resulted in the attachments being "orphaned" and not allowed to be accessed by anyone -- despite the attachment links being embedded in the message. This was accompanied by the error:
WARN [] User 12345 tried to share upload 123/3LkSA4OcoG6OpAknS2I0SFAQ/example.jpf in message 123456, but lacks permission
INFO [zerver.lib.email_mirror] Successfully processed email from user 12345 to example-stream

We solve this by creating attachment objects as the users the message will be sent from.

There's a bit of a question of if we try to backfill this for the orphaned attachments in the history.  Doing this would require finding all of the orphaned attachments owned by the gateway bot (which is straightforward) and then, for each, trying to match them up with message contents which were sent to that realm, with a client_id of Internal, and not has_attachment, and has content containing that path-id.  That is doable, it's just not likely to be a well-indexed query at all so the migration will be slow -- for those that use the mail gateway, which is probably not that many deploys.

Screenshots and screen captures:

Self-review checklist

[ ] Self-reviewed the changes for clarity and maintainability
(variable names, code reuse, readability, etc.).

Communicate decisions, questions, and potential concerns.

[ ] Explains differences from previous plans (e.g., issue description).
[ ] Highlights technical choices and bugs encountered.
[ ] Calls out remaining decisions and concerns.
[ ] Automated tests verify logic where appropriate.

Individual commits are ready for review (see commit discipline).

[ ] Each commit is a coherent idea.
[ ] Commit message(s) explain reasoning and motivation for changes.

Completed manual review and testing of the following:

[ ] Visual appearance of the changes.
[ ] Responsiveness and internationalization.
[ ] Strings and tooltips.
[ ] End-to-end functionality of buttons, interactions and flows.
[ ] Corner cases, error conditions, and easily imagined bugs.

Tangentially related to #20932 and #21580.

lgtm, but I'd like @mateuszmandera to review carefully as well.

Does this mean including attachments in these missed message emails was completely broken all along or am I missing something?

AFAICT, yes.

Tangential question - do we set an explicit limit for the size of attachments we will accept via email? Just thinking whether we should be making sure to enforce it to be consistent with the standard upload size limit, e.g. to be able to safely make assumptions about the size of any file we have in uploads.

We don't set a file size limit for individual files, only a realm upload limit.  The individual file upload limit is very much a technical "it starts breaking somewhere around here" limit, and my sense is not meant to be an enforcement mechanism.

Don't we hard-limit the upload size at least via nginx? client_max_body_size 80m;

AFAICT, yes.

Hmm, that seems surprising -- it seems like someone would have reported it long ago. I guess uploads sent to public streams would normally get the is_realm_public flag sent, and end up working... but that code only run when the attachment gets associated with the message, and that's the thing that's not working here.
(We added support for uploaded files in these back in 2013)

Merged, thanks @alexmv!
I'd like us to confirm it in production before we backport it to 6.x.

#24128 is the follow-up migration to fix old attachments and messages.

Backported to 6.x as 8bc6ad7a469b8dde1f41cfd9010f639c7909d6ae.


Added CONTRIBUTING.md

Added the CONTRIBUTING.md file. This fixes #52.

Perfect 👍

@charithsoori Would you like me to change anything in order for this PR to be merged or should I just close this PR?

Closing because of inactivity.


Slow GPU training with CUDA10.2 (ParagraphVectors)

Training model based on ParagraphVectors with GPU is very slow. At the same time, CPU training faster 440 times than GPU (at fixed period, see below).
After 20 minutes of training on GPU vectorized only 150 lines
GPU utilization while training - 30-40%
CPU utilization while training - 50-80%
At them same time, training on CPU is faster, after 20 minutes of training vectorized 65185 lines. While CPU training, CPU utilization is 100%.
cuDNN also installed.
Garbage collection is very rare during training.
What's wrong with my configuration / code? Why it's so slow?
Versions:
CUDA - 10.2
DL4J - 1.0.0-beta6
OS - Win7 (64bit)
NVIDIA driver version - 441.41
NVIDIA card - GeForce GTX 1660 SUPER
POM.xml
https://gist.github.com/AlikJoke/89137d9491fb1c727879bcaa759e9657
ParagraphVectors & CudaEnvironment configuration
https://gist.github.com/AlikJoke/ea2204ff0212f7e8b279de0b1258b4f8
JVM args:
-verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -Xms4G -Xmx4G -XX:+UseG1GC -XX:MetaspaceSize=2G
Initialization and computing logs (GPU):
https://gist.github.com/AlikJoke/1102e430f435fd55b204c3b6d624b3db
Initialization logs (CPU):
https://gist.github.com/AlikJoke/8a605598ac930043e0ad4d5b7c7a9b40

Due to GC I suppose. Unfortunately family of skipgram algorithms never was a priority for gpus for us.
With best regards,
raver119

От: Alik<EMAIL_ADDRESS>Отправлено: вторник, декабря 17, 2019 15:30
Кому: eclipse/deeplearning4j
Копия: Subscribed
Тема: [eclipse/deeplearning4j] Slow GPU training with CUDA10.2 (ParagraphVectors) (#8544)
Training model based on ParagraphVectors with GPU is very slow. At the same time, CPU training faster 100 times than GPU (at fixed period, see below).
After 20 minutes of training on GPU vectorized only 150 lines
GPU utilization while training - 30-40%
CPU utilization while training - 50-80%
At them same time, training on CPU is faster, after 20 minutes of training vectorized 65185 lines. While CPU training, CPU utilization is 100%.
cuDNN also installed.
Garbage collection is very rare during training.
What's wrong with my configuration / code? Why it's so slow?
Versions:
CUDA - 10.2
DL4J - 1.0.0-beta6
OS - Win7 (64bit)
NVIDIA driver version - 441.41
NVIDIA card - GeForce GTX 1660 SUPER
POM.xml
https://gist.github.com/AlikJoke/89137d9491fb1c727879bcaa759e9657
ParagraphVectors & CudaEnvironment configuration
https://gist.github.com/AlikJoke/ea2204ff0212f7e8b279de0b1258b4f8
JVM args:
-verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -Xms4G -Xmx4G -XX:+UseG1GC -XX:MetaspaceSize=2G
Initialization and computing logs (GPU):
https://gist.github.com/AlikJoke/1102e430f435fd55b204c3b6d624b3db
Initialization logs (CPU):
https://gist.github.com/AlikJoke/8a605598ac930043e0ad4d5b7c7a9b40
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://github.com/eclipse/deeplearning4j/issues/8544?email_source=notifications&email_token=AC5O57YK2GZVLOIVSFSMMX3QZDIAHA5CNFSM4J33O552YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4IBBQNHA, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AC5O575GXHW6N6ZV27OVQ43QZDIAHANCNFSM4J33O55Q.

GC is very rare during training. STW time is very-very small. There are almost no major (full) GC (mostly minor).

Ok, we'll take a closer look then during current sprint. Thanks for highlighting this problem
With best regards,
raver119

От: Alik<EMAIL_ADDRESS>Отправлено: среда, декабря 18, 2019 10:53
Кому: eclipse/deeplearning4j
Копия: raver119; Comment
Тема: Re: [eclipse/deeplearning4j] Slow GPU training with CUDA10.2 (ParagraphVectors) (#8544)
GC is very rare during training. STW time is very-very small. There are almost no major (full) GC (mostly minor).
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com/eclipse/deeplearning4j/issues/8544?email_source=notifications&email_token=AC5O57YOEWXZXALK6MB5SD3QZHQJ5A5CNFSM4J33O552YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHFMTZQ#issuecomment-566938086, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AC5O576HDR5V6AU2IODDFQ3QZHQJ5ANCNFSM4J33O55Q.