How can I get the original projection data via dicom image by calling rtk via python and reconstruct it again

Dear Code Contributors：
I have installed the RTK python library correctly!
I now have in my possession a dicom image of a patient obtained by CBCT scanning，But now I want to get its original projection data from this image and use this projection data for FDK reconstruction to get the image and make sure that the reconstructed image is consistent with the original image.
I have read some of the example.py  but get less information, if you could give some hints on the code I would be very grateful!

Not a code issue, use the mailing list please.
The CBCT dcm image does not contain the raw data. you need to check whether you can access it on your scanner. What you can do is only reproject the current CBCT but that is not the same.
You can try rtkfdk for reconstructing the raw data if you get it but that is unlikely that the result will be consistent because each manufacturer has specific steps which are kept secret. The full procedure, in particular how to obtain the geometry file) will depend on the CBCT scanner.


Make Loop.checkpoint file_pattern function more flexible

Currently in the file_pattern function of Loop.checkpoint only epoch and iteration get passed through. This limitation makes it impossible to include metrics like loss in the filename which can be helpful.
To solve this we can pass the whole loop state to the file_pattern function.
With this change we can specify the file_pattern like this.
Axon.input({nil, 1})
|> Axon.dense(1)
|> Loop.trainer(:binary_cross_entropy, :sgd, log: 0)
|> Loop.checkpoint(file_pattern: fn state -> "epoch_#{state.epoch}_#{state.metrics["loss"]}.ckpt" end)
|> Loop.run([{Nx.tensor([[1]]), Nx.tensor([[2]])}], epochs: 3)

This PR also adds tests for checkpoint.

This is awesome! The Loop API is also one of those APIs that I have not been able to test as thoroughly as I'd like, so I really appreciate you taking the time to add those tests in! Thank you!


Tensorflow would sometimes get rather slow during training

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): No
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 16.04
TensorFlow installed from (source or binary): pip
TensorFlow version (use command below): 1.4
Python version: 3.5
CUDA/cuDNN version: CUDA 8.0 / cuDNN 6.0
GPU model and memory: GTX1060 6GB

Describe the problem
When I train my network, at first it works properly, however sometimes it gets rather slower (about 10 times slower than normal speed, shown below). When I type nvidia-smi to check the GPU state, I find the GPU-Util is rather low (2-3%)
The normal speed is about 1 minute per 100 steps while it would sometimes cost 10 minminutes per 100 steps.
I used to use Tensorflow v1.1 and did not find this problem. However I updated Tensorflow to v1.4 several days ago and this problem occurred.
 Could you please share that and then might be able to help you better.

@rohan100jain The model I run is Inception_v1 provided here https://github.com/tensorflow/models/blob/master/research/slim/nets/inception_v1.py
The code is like:
        with tf.name_scope('input'):
		tra_image_batch, tra_label_batch = input_data3.getbatch(BATCH_SIZE, 'inception_v1',
			224, True, 0)
		val_image_batch, val_label_batch = input_data3.getbatch(test_batchsize, 'inception_v1',
			224, False, 0)
	x = tf.placeholder(tf.float32, shape=[None, IMG_W, IMG_H, 3])
	y_ = tf.placeholder(tf.int32, shape=[None])
	is_training = tf.placeholder(tf.bool, shape=[])
	with slim.arg_scope(inception_v1.inception_v1_arg_scope()):
		logits, end_points = inception_v1.inception_v1(x, 1001, is_training=is_training)
	loss = tools.loss(logits, y_)
	accuracy = tools.accuracy(logits, y_)
	top5_accuracy = tools.top5_accuracy(logits, y_)
        train_op = tools.optimize_m(loss, learning_rate1, learning_rate2, my_global_step)
	try:
                for step in np.arange(MAX_STEP):
			if coord.should_stop():
					break
			tra_images,tra_labels = sess.run([tra_image_batch, tra_label_batch])
			_, tra_loss, tra_acc = sess.run([train_op, loss, accuracy], feed_dict={x:tra_images, y_:tra_labels, is_training:True})
			if step % 100 == 0 or step == MAX_STEP:
				nowtime = time.strftime("%Y-%m-%d %H:%M:%S", time.localtime())
				print (nowtime+'  Step: %d, loss: %.4f, accuracy: %.4f%%' %(step+global_step, tra_loss, tra_acc))

I have same issue, training become very slow after updating to TF 1.4. But when I update tensorflow-gpu to version 1.4.0 too (pip install tensorflow-gpu==1.4.0), it works.

@tfboyd @sguada Is that something we know about? Any way to debug?

It has been 14 days with no activity and this issue has an assignee.Please update the label and/or status accordingly.

Nagging Assignee: It has been 14 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee: It has been 14 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @tfboyd: It has been 14 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @tfboyd: It has been 14 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @tfboyd: It has been 14 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @tfboyd: It has been 171 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @tfboyd: It has been 186 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @tfboyd: It has been 201 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @tfboyd: It has been 216 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @tfboyd: It has been 231 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

To close this and likely not with a great end.  I wrote this script a long time ago and it watches nvidia-smi for any type of GPU slow down.  I doubt that is the issue, but it was my issue yesterday and I was reminded about this ticket.  All it does is run NVIDIA-SMI, grep for a very basic Hardware slowdown as well as keep a log of each full nvidia-smi result.
https://github.com/tfboyd/tf-tools/blob/master/benchmark/multi_gpu/stats_monitor.sh


Add inmem key repository

Signed-off-by: Bogdan Vaneev<EMAIL_ADDRESS>

Description of the Change

To implement peerbook we need:

[x] key repository
[ ] addr repository
[ ] metadata repository

This PR adds inmem_key_repository - a DAO (data access object), which is essentially a
map[peer id] => set(public key)
map[peer id] => set(keypair)

Storage is in-memory.
Benefits

Ability to store cryptographic material of observed peers.
Possible Drawbacks

None.

Codecov Report

Merging #88 into master will decrease coverage by 0.15%.
The diff coverage is 87.24%.

@@            Coverage Diff             @@
##           master      #88      +/-   ##
==========================================
- Coverage   85.62%   85.46%   -0.16%
==========================================
  Files         165      168       +3
  Lines        4695     4685      -10
==========================================
- Hits         4020     4004      -16
- Misses        675      681       +6

Impacted Files
Coverage Δ

core/libp2p/peer/errors.cpp
0% <0%> (ø)

test/core/libp2p/crypto/marshaller_test.cpp
100% <100%> (ø)
:arrow_up:

core/libp2p/multi/multihash.hpp
100% <100%> (ø)

test/core/libp2p/crypto/keys_test.cpp
100% <100%> (ø)
:arrow_up:

core/common/buffer.hpp
100% <100%> (ø)
:arrow_up:

core/libp2p/crypto/marshaller/key_marshaller.cpp
92.85% <100%> (ø)
:arrow_up:

core/libp2p/crypto/private_key.hpp
100% <100%> (ø)
:arrow_up:

...core/libp2p/muxer/yamux/yamux_integration_test.cpp
100% <100%> (ø)
:arrow_up:

...peer/inmem_key_repository/inmem_key_repository.hpp
100% <100%> (ø)

core/libp2p/peer/key_repository.hpp
100% <100%> (ø)

... and 22 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c09e363...578e8ec. Read the comment docs.


ASU-1421 | Add state change event data to customer reservation api

Codecov Report

Merging #196 (4b077d2) into develop (9301324) will increase coverage by 0.12%.
The diff coverage is 97.40%.

@@             Coverage Diff             @@
##           develop     #196      +/-   ##
===========================================
+ Coverage    94.81%   94.94%   +0.12%
===========================================
  Files          101      103       +2
  Lines         3182     3321     +139
  Branches       262      276      +14
===========================================
+ Hits          3017     3153     +136
- Misses         117      120       +3
  Partials        48       48

Impacted Files
Coverage Δ

apartment/enums.py
91.66% <88.23%> (-8.34%)
:arrow_down:

application_form/services/export.py
96.22% <96.22%> (ø)

apartment/api/sales/serializers.py
100.00% <100.00%> (ø)

apartment/api/views.py
100.00% <100.00%> (ø)

apartment/elastic/queries.py
100.00% <100.00%> (ø)

apartment/urls.py
100.00% <100.00%> (ø)

application_form/api/sales/serializers.py
97.05% <100.00%> (+0.83%)
:arrow_up:

application_form/api/sales/views.py
97.77% <100.00%> (ø)

application_form/api/serializers.py
100.00% <100.00%> (+0.81%)
:arrow_up:

application_form/services/reservation.py
100.00% <100.00%> (ø)

... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8bd90e3...4b077d2. Read the comment docs.


how to generated options.html & filters.html

Steps to reproduce the problem:

git clone https://github.com/mitmproxy/mitmproxy.git
2.cd docs
yarn
modd

Any other comments? What have you tried so far?
11:12:11: prep: ./node_modules/.bin/node-sass -o ./src/themes/mitmproxydocs/static/css ./style/style.scss --error-bell
11:12:11: daemon: cd src; hugo server -D
>> starting...
Building sites … ERROR 2018/11/26 11:12:11 "/Users/dc/Documents/interest/gitHub/mitmproxy/docs/src/content/concepts-options.md:47:5": failed to render shortcode "readfile": "/Users/dc/Documents/interest/gitHub/mitmproxy/docs/src/layouts/shortcodes/readfile.html:5:12": execute of template failed: template: shortcodes/readfile.html:5:12: executing "shortcodes/readfile.html" at <readFile>: error calling readFile: stat /Users/dc/Documents/interest/gitHub/mitmproxy/docs/src/content/generated/options.html: no such file or directory
Total in 51 ms
Error: Error building site: "/Users/dc/Documents/interest/gitHub/mitmproxy/docs/src/content/concepts-filters.md:13:5": failed to render shortcode "readfile": "/Users/dc/Documents/interest/gitHub/mitmproxy/docs/src/layouts/shortcodes/readfile.html:5:12": execute of template failed: template: shortcodes/readfile.html:5:12: executing "shortcodes/readfile.html" at <readFile>: error calling readFile: stat /Users/dc/Documents/interest/gitHub/mitmproxy/docs/src/content/generated/filters.html: no such file or directory
exited: exit status 1
>> sleeping...<PHONE_NUMBER>

I just delete {{< readfile file="/generated/options.html" >}} in mitmproxy/docs/src/content/concepts-options.md:47:5 and {{< readfile file="/generated/filters.html" >}} in mitmproxy/docs/src/content/concepts-filters.md:13:5 and it will work fine but it miss something.
System information
Mitmproxy: 4.0.4
Python:    3.7.0
OpenSSL:   OpenSSL 1.1.0i  14 Aug 2018
Platform:  Darwin-18.0.0-x86_64-i386-64bit

#3667
I solved the issue by running the ./build-current script

Thanks @jannst!


chore: bdd-sbhg-import-1586992788 to 0.0.1

chore: Promote bdd-sbhg-import-1586992788 to version 0.0.1

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approvers:
If they are not already assigned, you can assign the PR to them by writing /assign  in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


hash_ty is a bad name

The ecdsa::Signature field names need some love before v1.0

hash_ty is not great, ty so as not to use reserved keyword type?
sig makes usage ugly: we often end up with sig.sig

Possible solution:
pub struct Signature {
    /// The underlying ECDSA signature.
    pub inner: secp256k1::ecdsa::Signature,
    /// The corresponding sighash type.
    pub sighash_type: EcdsaSighashType,
}

inner is not that great either, ideas?

Maybe ecdsa_sig? Or just sig? It will result in some stutterning sig.sig but that's fairly common with crypto types.

AFAIR inner is rather standard name to use, and stdlib is full of these: https://doc.rust-lang.org/src/std/fs.rs.html#100-102

Why not just call it signature? Yeah, if someone has signature.signature it looks a bit funny but I've done it and it's not really bad IMO. Maybe raw_sig if you really want to avoid it.
@dpc not that much in public API where it's get_ref, get_mut or into_inner - that's the closes one I've seen.

Perhaps signature is best. Local var usage can then be sig.signature which looks better to me than sig.sig.


Can spoof inbound mail on local network

Classification

[X] Question

Reproducibility

[X] Always

Docker information
Containers: 4
 Running: 4
 Paused: 0
 Stopped: 0
Images: 4
Server Version: 18.09.7
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version:
runc version: N/A
init version: v0.18.0 (expected: fec3683b971d9c3ef73f284f176672c44b448662)
Security Options:
 apparmor
 seccomp
  Profile: default
Kernel Version: 4.15.0-58-generic
Operating System: Ubuntu 18.04.3 LTS
OSType: linux
Architecture: x86_64
CPUs: 1
Total Memory: 985.5MiB
Name: mail
ID: GJNY:4YWL:KHXA:QAHB:FRZR:3H6N:5ZVS:R3XC:5SE7:EU5X:RELW:GOLD
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
 <IP_ADDRESS>/8
Live Restore Enabled: false

WARNING: No swap limit support
root@mail:~/mailserver# docker images hardware/mailserver --digests --filter "dangling=false"
REPOSITORY            TAG                 DIGEST                                                                    IMAGE ID            CREATED             SIZE
hardware/mailserver   1.1-stable          sha256:4875f76004ef9976099bd12e1f16671776d7d20f788d59cc279a17d63add73ac   c4513232af37        12 days ago         375MB

Description
I can spoof email when communicating with the mailserver on local network addresses (both <IP_ADDRESS> and <IP_ADDRESS>/x network addresses work, for reference my mailserver and host are on 10.x  network with a /24 subnet, with both hosts in the same subnet)
Steps to reproduce
import smtplib
import socket

def self_spoof():
    fromaddr =<EMAIL_ADDRESS>    toaddrs  = ["victim@mydomain", ]
    msg = ("From: %s\r\nSubject: HelloWorld\r\nTo: %s\r\n\r\n"
           % (fromaddr, ", ".join(toaddrs)))
    msg += "Hello World"
    server = smtplib.SMTP("localhost")
    server.starttls()
    server.set_debuglevel(1)
    server.sendmail(fromaddr, toaddrs, msg)
    server.quit()
self_spoof()

replace toaddrs with a valid email which is hosted by the mailserver on localhost
Optionally replace localhost with the local IP of the system and run from an adjacent host on the same network
python3 spoof.py
The email will not be marked as spam

In contrast, replace localhost with the FQDN, the email will be marked as spam
Expected results
Email should be flagged as spam, the same email with the hosts' FQDN receives a spam score of 9.49/15.
X-Spamd-Result: default: False [9.49 / 15.00];
	 ARC_NA(0.00)[];
	 GREYLIST(0.00)[pass,body];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_FROM(0.00)[gmail.com];
	 MISSING_DATE(1.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_DN_NONE(0.00)[];
	 R_SPF_SOFTFAIL(0.00)[~all:c];
	 RCPT_COUNT_ONE(0.00)[1];
	 IP_SCORE_FREEMAIL(0.00)[];
	 MISSING_MID(2.50)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com];
	 FROM_NO_DN(0.00)[];
	 IP_SCORE(0.00)[country: US(-0.01)];
	 VIOLATED_DIRECT_SPF(3.50)[];
	 RCVD_COUNT_ZERO(0.00)[0];
	 FROM_EQ_ENVFROM(0.00)[];
	 R_DKIM_NA(0.00)[];
	 FREEMAIL_ENVFROM(0.00)[gmail.com];
	 ASN(0.00)[asn:14061, ipnet:<IP_ADDRESS>/20, country:US];
	 TAGGED_FROM(0.00)[];
	 HFILTER_HOSTNAME_UNKNOWN(2.50)[];
	 DMARC_POLICY_SOFTFAIL(0.10)[gmail.com : No valid SPF, No valid DKIM,none]

Actual results
Email is not marked as spam, it receives a score of 3.40/15
X-Spamd-Result: default: False [3.40 / 15.00];
	 ARC_NA(0.00)[];
	 FREEMAIL_FROM(0.00)[gmail.com];
	 MISSING_DATE(1.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_DN_NONE(0.00)[];
	 RCPT_COUNT_ONE(0.00)[1];
	 MISSING_MID(2.50)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 FROM_NO_DN(0.00)[];
	 RCVD_COUNT_ZERO(0.00)[0];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_ENVFROM(0.00)[gmail.com]

I'm unsure if this is expected behaviour, I wouldn't care if it were limited to localhost, but a host on the same subnet is also able to reliably spoof email.
Debugging information
docker logs mailserver

Configuration (docker-compose.yml, traefik.toml...etc)
services:
  mailserver:
    image: hardware/mailserver:${MAILSERVER_DOCKER_TAG}
    restart: ${RESTART_MODE}
    domainname: ${MAILSERVER_DOMAIN}
    hostname: ${MAILSERVER_HOSTNAME}
    ports:
      - "25:25"
      - "143:143"
      - "587:587"
      - "<IP_ADDRESS>:11334:11334"
    environment:
      - DBPASS=${DATABASE_USER_PASSWORD}
      - RSPAMD_PASSWORD=${RSPAMD_PASSWORD}
      - DISABLE_CLAMAV=true
      - PASSWORD_SCHEME=PLAIN
    volumes:
      - ${VOLUMES_ROOT_PATH}/mail:/var/mail
    depends_on:
      - mariadb
      - redis
    networks:
      - mail_network
      - http_network

Just because it's stale doesn't mean it's not an issue ;)


How would you go about instruction finetuning?

How would you finetune in this style with an instruction finetuning data set like Open-Orca?

Hi, thanks for an excellent question and suggestion of the dataset! We are planning to provide an example of fine-tuning our models using the huggingface API, which would include instruction tuning.

Following on this as well! Would love to try an Instruct-LongLlama

We are planning to release instruction tuning code in pytorch & checkpoints  & examples early next week. Stay tuned!

We are planning to release instruction tuning code in pytorch & checkpoints & examples early next week. Stay tuned!

Thank you! Super excited now!

In case you haven't seen, the instruction code is already there! see https://twitter.com/s_tworkowski/status/1687620785379360768 and READMEs in this repo for more details

In case you haven't seen, the instruction code is already there! see https://twitter.com/s_tworkowski/status/1687620785379360768 and READMEs in this repo for more details

Hey thanks again for sharing this! Just curious, would this training also work for much smaller models like Pythia 160m or does this only work for the LongLLama model yall released at the moment?

In terms of instruction finetuning, I personally don't think that it makes much sense to do SFT on models below 3B.
But for the FoT pretraining, you're encouraged to look at our paper https://arxiv.org/abs/2307.03170 in particular section 5 where all experiments are done on a ~150M parameter scale. We initially developed this method on such a scale and then scaled it to billion parameter models, so it definitely should work. There is currently no pretraining codebase available though, we are planning to open source one in JAX after LongLLaMA v2 release, but there are no plans to write a pytorch codebase on our side.

In terms of instruction finetuning, I personally don't think that it makes much sense to do SFT on models below 3B - I mean 3B models are not very capable and the thing you want to achieve with SFT is to actually have a useful model to interact with. Our code is technically just a bunch of optimizations to fit a 3B model into an A100, so it's not very useful for a 160m model I presume.
But for the FoT pretraining if you want to test hypotheses by measuring perplexity, you're encouraged to look at our paper https://arxiv.org/abs/2307.03170 in particular section 5 where all experiments are done on a ~150M parameter scale. We initially developed this method on such a scale and then scaled it to billion parameter models, so it definitely should work. There is currently no pretraining codebase available though, we are planning to open source one in JAX after LongLLaMA v2 release, but there are no plans to write a pytorch codebase on our side.

Yeah right now I'm testing a hypothesis similar to the Tiny Stories and the Textbooks are All You Need paper. Where the goal is to see how far we can push small models by using more data and training then longer than you normally would. At the moment I have a dataset of 12 million reasoning style instructions and I'm planning on training a few models on that for up to 20 epochs.
But I'm curious is there a way to make an off the shelf model work with FOT, or does it have to be pretrained in the way y'all set it up, and then finetuned further to follow instructions?
Thanks again for the help!

I see, I see, I think now I understand your question. There is no requirement for a model to be pre-trained with FoT. Both in the paper and LongLLaMA we take models trained in a vanilla way, and the fine-tune with FoT. My doubt is actually what do you mean by FoT here, it has 2 components: training on negative examples, and architecture. I don't think training on negative examples makes much sense in the context of instruction tuning, but i haven't tried that, just seems unreasonable. Architecturally, there is no problem in fine-tuning a model with FoT architecture (basically having a subset of layers responsible for handling long-range dependencies and disabling positionals)

Oh I think I understand! Sorry for the confusion. So in order to get an off the shelf model to work like the LongLlama models y'all released I need to pretrain it for longer using y'all's pre training code and then if I wanted to follow instructions for long context, that's where y'all's latest finetuning code would come in?
If that's correct, do y'all have any plans to release either the pretraining code and data or some of the smaller models you worked with?
Thanks again!

Also would y'all be willing to get on a zoom call or a Google meet to discuss it a little bit more? If not that's totally cool!

So in order to get an off the shelf model to work like the LongLlama models y'all released I need to pretrain it for longer using y'all's pre training code and then if I wanted to follow instructions for long context, that's where y'all's latest finetuning code would come in?

That's it, basically. If you want to achieve long context capability just by instruction finetuning, honestly you'd better try sth like longchat. But their training data is not public as well, and you definitely need to have long-context instruction tuning data which is not there. That's where our bet on continued long-context pretraining with the FoT objective and then instruction fine-tuning on short context to follow instructions comes from.
By the way, it is an interesting hypothesis to test if, similarly to generalizing to multiple languages just after instruction tune in pure English, LLMs could generalize to longer instructions without seeing them at train time. I think it just requires some playing with inference with our current instruct model to actually test that hypothesis - not exactly sure how to formulate it rigorously.
For the release of the pretraining code, see this tt thread: https://twitter.com/4evaBehindSOTA/status/1687757769120862209?s=20
Note that our current code is in JAX and it would require substantial engineering work on your side to understand it, and rewrite into pytorch, if you wanted. If you are willing to undertake this work (which would probably be quite valuable for the community if you then publish), please let me know and we could set up a call or sth.


Unclassified error occured while trying to activate license

Bug description

It throws an error even though I followed this document for the activation:
---------- git config --list --show-origin -------------
file:/etc/gitconfig	filter.lfs.required=true
file:/etc/gitconfig	filter.lfs.clean=git-lfs clean -- %f
file:/etc/gitconfig	filter.lfs.smudge=git-lfs smudge -- %f
file:/etc/gitconfig	filter.lfs.process=git-lfs filter-process
Changing to "/github/workspace/_activate-license~" directory.
/github/workspace/_activate-license~ /github/workspace
Requesting activation (personal license)
Unclassified error occured while trying to activate license.
Exit code was: 1
Error: The process '/usr/bin/docker' failed with exit code 1

How to reproduce

Run this workflow:
    - name: Build Unity project
      uses: game-ci/unity-builder@v3
      env:
        UNITY_LICENSE: ${{ secrets.UNITY_LICENSE }}
        UNITY_EMAIL: ${{ secrets.UNITY_EMAIL }}
        UNITY_PASSWORD: ${{ secrets.UNITY_PASSWORD }}
      with:
        targetPlatform: WebGL
        unityVersion: 2022.3.7f1
        projectPath: /path/to/project

Expected behavior

Successfully built the Unity project
Additional details

I used a personal license.
My password for the Unity account only contains alphabets and numbers (no special characters such as ! and #)

My bad, I didn't read the following line carefully and I only copied the serial number in the license file rather than the entire content of the file

UNITY_LICENSE - (Copy the contents of your license file into here)

https://game.ci/docs/github/activation/ - activation that was the trick the section "CAUTION" it either has been updated recently or I missed it. Now I can see personal due to CAUTION, after regenerating license it works, thanks.


Can't show anything

Bug

Environment info

React native info output:
React Native Environment Info:
System:
OS: macOS 10.14.5
CPU: (4) x64 Intel(R) Core(TM) i5-7267U CPU @ 3.10GHz
Memory: 1.54 GB / 8.00 GB
Shell: 5.3 - /bin/zsh
Binaries:
Node: 12.4.0 - /usr/local/bin/node
Yarn: 1.16.0 - /usr/local/bin/yarn
npm: 6.9.0 - /usr/local/bin/npm
Watchman: 4.9.0 - /usr/local/bin/watchman
SDKs:
iOS SDK:
Platforms: iOS 12.2, macOS 10.14, tvOS 12.2, watchOS 5.2
Android SDK:
API Levels: 27
Build Tools: 27.0.3
IDEs:
Android Studio: 3.3 AI-182.5<IP_ADDRESS>99772
Xcode: 10.2.1/10E1001 - /usr/bin/xcodebuild
npmPackages:
react: 16.8.3 => 16.8.3
react-native: 0.59.9 => 0.59.9
npmGlobalPackages:
react-native-cli: 2.0.1
 // paste it here

Library version: x.x.x
Steps To Reproduce

import {
    Surface,
    Group,
    Shape,
    LinearGradient,
    Path,
    Transform,
    ClippingRectangle,
} from '@react-native-community/art';

export class ARTPage extends Component {
    render() {
        const path = new Path();
        path.moveTo(1,1); //将起始点移动到(1,1) 默认(0,0)
        path.lineTo(300,1); //连线到目标点(300,1)
        return (
            <View style={this.props.style}>
                <Surface width={500} height={500}>
                <Shape d={new Path().moveTo(0, 0).lineTo(200, 200)} fill="#d39494" />
                </Surface>
            </View>

        )
    }
}

...
Describe what you expected to happen:

Reproducible sample code

Hey @xclidongbo
I'm having the same issue, would you mind sharing on the steps to fix this?

i get the same problem


Support multi-arch images

This change log is obsolete. Please see https://github.com/ceph/ceph-csi/pull/707#issuecomment-558032838 for latest change log.
Support multi-arch images
Add jobs to build arm64 image and support multi-arch docker image.
By multi-arch it means "docker pull cephcsi/cephcsi:canary" will
download amd64 or arm64 image per host archtecture automatically.
Architecture dependent images are named as cephcsi/cephcsi-amd64
and cephcsi/cephcsi-arm64.

Major changes:
- Add travis job "build arm64 image" which runs on travis arm64 nodes
  to build cephcsi image for arm64.
- Add travis job "push image manifest" which pushes multi-arch image
  manifest to docker registry.
- Update Makefile to tag and push architecture dependent image with
  architecture appended in name (e.g, cephcsi/cephcsi-amd64:canary)
- Add Makefile target "push-manifest-cephcsi" to push image manifest
  and support multi arch.
- Add ENV_DOCKER_REGISTRY variable to configure docker registry
  for hosting built image, rather than hard coded quay.io.

NOTES:
- CI test jobs are disabled on Arm now build due to minikube issue.
- Job "push-manifest-cephcsi" runs for every PR. It should only run
  when patch merges. Though it does nothing for normal PR (deploy.sh
  checks ${TRAVIS_PULL_REQUEST}), it still needs booting, installing
  packages, etc, which costs about 1 minute.
- In current jobs, docker image is pushed if one test succeeds and
  runs deployment, even if other tests fails. After applying this
  patch, architecure image is handled same, pushed when any test
  succeeds. But multi-arch manifest is pushed only after all tests
  succeed, any broken test will block pushing multi-arch manifest.

CI test log
https://travis-ci.org/cyb70289/ceph-csi/builds/608636375

Major changes:

Add travis job "cephcsi on arm64" which runs on travis arm64 node to build and test cephcsi image for arm64.
Update deploy.sh and Makefile to build and push multi-arch images.

NOTE: arm64 test job is disabled by default due to two reasons

Kubernetes-csi sidecar images for arm64 are missing
Instead of virtual machine, travis arm64 job runs in LXD, and it denies privileged filesystem access. It causes trouble to launch minikube cluster.

deploy.sh is not run for normal PR. To see full test logs, please check my personal CI job:

Test log with deploy.sh https://travis-ci.org/cyb70289/ceph-csi/builds/616541161
Test image pushed to https://hub.docker.com/r/cyb70289/cephcsi/tags

@Madhu-1 I've updated this patch. Would you review it? Thanks.

Major changes:

Add travis job "cephcsi on arm64" which runs on travis arm64 node to build and test cephcsi image for arm64.
Update deploy.sh and Makefile to build and push multi-arch images.

NOTE: arm64 test job is disabled by default due to two reasons

Kubernetes-csi sidecar images for arm64 are missing
Instead of virtual machine, travis arm64 job runs in LXD, and it denies privileged filesystem access. It causes trouble to launch minikube cluster.

how can we test amd64 images now?

deploy.sh is not run for normal PR. To see full test logs, please check my personal CI job:

Test log with deploy.sh https://travis-ci.org/cyb70289/ceph-csi/builds/616541161
Test image pushed to https://hub.docker.com/r/cyb70289/cephcsi/tags

Major changes:

Add travis job "cephcsi on arm64" which runs on travis arm64 node to build and test cephcsi image for arm64.
Update deploy.sh and Makefile to build and push multi-arch images.

NOTE: arm64 test job is disabled by default due to two reasons

Kubernetes-csi sidecar images for arm64 are missing
Instead of virtual machine, travis arm64 job runs in LXD, and it denies privileged filesystem access. It causes trouble to launch minikube cluster.

how can we test amd64 images now?

 To see full test logs, please check my personal CI job:

Test log with deploy.sh https://travis-ci.org/cyb70289/ceph-csi/builds/616541161
Test image pushed to https://hub.docker.com/r/cyb70289/cephcsi/tags

Major changes:

Add travis job "cephcsi on arm64" which runs on travis arm64 node to build and test cephcsi image for arm64.
Update deploy.sh and Makefile to build and push multi-arch images.

NOTE: arm64 test job is disabled by default due to two reasons

Kubernetes-csi sidecar images for arm64 are missing
Instead of virtual machine, travis arm64 job runs in LXD, and it denies privileged filesystem access. It causes trouble to launch minikube cluster.

how can we test amd64 images now?

I tested ceph-csi arm64 image in a local k8s cluster running inside VM, and with all necessary sidecar images manually built.

how can we test  it on CI?

deploy.sh is not run for normal PR. To see full test logs, please check my personal CI job:

Test log with deploy.sh https://travis-ci.org/cyb70289/ceph-csi/builds/616541161
Test image pushed to https://hub.docker.com/r/cyb70289/cephcsi/tags

Major changes:

Add travis job "cephcsi on arm64" which runs on travis arm64 node to build and test cephcsi image for arm64.
Update deploy.sh and Makefile to build and push multi-arch images.

NOTE: arm64 test job is disabled by default due to two reasons

Kubernetes-csi sidecar images for arm64 are missing
Instead of virtual machine, travis arm64 job runs in LXD, and it denies privileged filesystem access. It causes trouble to launch minikube cluster.

how can we test amd64 images now?

I tested ceph-csi arm64 image in a local k8s cluster running inside VM, and with all necessary sidecar images manually built.

how can we test it on CI?

I'm afraid there's no easy way to test ceph-csi fully in arm64 CI now. The major barrier is travis arm64 node runs in non-priviledged LXD which prohibits launching minikube cluster. Without actual k8s environment, I think ceph-csi cannot be tested thoroughly.

deploy.sh is not run for normal PR. To see full test logs, please check my personal CI job:

Test log with deploy.sh https://travis-ci.org/cyb70289/ceph-csi/builds/616541161
Test image pushed to https://hub.docker.com/r/cyb70289/cephcsi/tags

Can't we use https://aws.amazon.com/blogs/aws/new-ec2-instances-a1-powered-by-arm-based-aws-graviton-processors/ ?

Or CentOS CI?
Or Ceph CI has some ARM support?

CC @nixpanic

Or CentOS CI?
Or Ceph CI has some ARM support?

Ceph-csi is using travis CI. To leverage machines from amazon or other CI, we have to add a new set of CI code to run in parallel with travis. Looks no good.

we have a plan to use centos CI for ceph-csi, CC'ed @nixpanic to provide more information

Posted a travis issue to try replacing LXD with VM:
https://travis-ci.community/t/arm64-unprivileged-lxd-blocks-kubernetes-tests/6272

On Tue, Dec 03, 2019 at 02:05:18AM -0800, Yaniv Kaul wrote:

Or CentOS CI?
Or Ceph CI has some ARM support?

The CentOS CI does not have ARM hardware available at this time:

https://wiki.centos.org/QaWiki/PubHardware

I do not know about the Ceph infrastructure.
Travis does seem to have some support for ARM.

Isnt it good to target this for v2.0.0 ?

@humblec we cannot test ARM images on Travis CI, we have two options 1) we need a CI to test arm images 2) declare ARM images as experimental(not tested)

It's important to make this feature available, even if it is marked experimental initially. If the only reason is that the CI doesn't cover it, the risk with golang is fairly low at least from what we've seen in rook.

It's important to make this feature available, even if it is marked experimental initially. If the only reason is that the CI doesn't cover it, the risk with golang is fairly low at least from what we've seen in rook.

in that case, will take it up for 2.0.0 release  @humblec @nixpanic @ShyamsundarR PTAL

Sure.  I am fine to take this in for release-2.0.0. The only thing which I could suggest here is, updating the documentation that its experimental. @cyb70289 can you please update the README with this ?

@cyb70289 can you also mention the subjected minikube issue - CI test jobs are disabled on Arm now build due to minikube issue. referred here in the code or source comments ?

@humblec Cool! Will update soon.

diffs to last commit:

add note "ceph csi Arm64 support is experimental" to README.md
add reasons why disabling Arm64 test job in .travis.yml

Thanks @cyb70289

@Madhu-1 @nixpanic ptal.

@Madhu-1 sure.. I will revert once I verify this.

Note that we can ask CNCF / Packet for an arm node to help with the CI.
Let me know if I can help coordinate that.
https://github.com/rook/rook/issues/4051#issuecomment-569386390

@cyb70289 I was testing this in my system and facing below issue:
[root@localhost ceph-csi]# docker manifest create --amend $REPO/busybox:latest $REPO/busybox-amd64:latest $REPO/busybox-arm64:latest
unsupported manifest media type and no default available: application/vnd.docker.distribution.manifest.v1+prettyjws
[root@localhost ceph-csi]# docker version
Client: Docker Engine - Community
 Version:           19.03.5
 API version:       1.40
 Go version:        go1.12.12
 Git commit:        633a0ea838
 Built:             Wed Nov 13 07:26:10 2019
 OS/Arch:           linux/amd64
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          19.03.5
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.12
  Git commit:       633a0ea838
  Built:            Wed Nov 13 07:24:42 2019
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.2.10
  GitCommit:        b34a5c8af56e510852c35414db4c1f4fa6172339
 runc:
  Version:          1.0.0-rc8+dev
  GitCommit:        3e425f80a8c931f88e6d94a8c831b9d5aa481657
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683
[root@localhost ceph-csi]#

Any idea whats causing this ? I came across this [1] https://github.com/docker/cli/issues/1724
Btw, below commands were successful:

docker pull busybox
docker pull arm64v8/busybox

docker tag busybox:latest $REPO/busybox-amd64:latest
docker tag arm64v8/busybox:latest $REPO/busybox-arm64:latest

docker push $REPO/busybox-amd64:latest
docker push $REPO/busybox-arm64:latest

If this is good enough, please let me know we can go ahead .

@humblec @Madhu-1 From the error message, looks quay.io/cephcsi does not support multiarch manifest. Same commands run correctly on dockerhub. Maybe I'm missing something but current patch will break cephcsi image pushing process. We should NOT go ahead.
If we cannot switch to a docker registry with multiarch support(quay.io v3 or dockerhub) recently, I would recommend a temporary solution(similar approach used by flannel):

build and push x86 image quay.io/cephcsi/cephcsi:v2.0.0-amd64
build and push arm image quay.io/cephcsi/cephcsi:v2.0.0-arm64
re-tag and push x86 image as quay.io/cephcsi/cephcsi:v2.0.0

So the default image is still x86, user can pull arm image by appending "-arm64" after the tag.

build and push x86 image quay.io/cephcsi/cephcsi:v2.0.0-amd64
build and push arm image quay.io/cephcsi/cephcsi:v2.0.0-arm64
re-tag and push x86 image as quay.io/cephcsi/cephcsi:v2.0.0

So the default image is still x86, user can pull arm image by appending "-arm64" after the tag.

@cyb70289 ACK. lets do that !

So we won't be able to have multi-arch images, but instead specifically-named images for certain types?

So we won't be able to have multi-arch images, but instead specifically-named images for certain types?

@cyb70289 yes that a temp solution for now till quay supports it

So we won't be able to have multi-arch images, but instead specifically-named images for certain types?

@cyb70289 yes that a temp solution for now till quay supports it

Thanks for the clarification.  I would suggest re-naming this PR to indicate that what is being delivered are multiple different arch images and not multi-arch images.  These are very much different things and it would be important to note the distinction for those who are expecting to be able to run these images in a multiple-arch cluster.
Also, it looks like quay 3.0 started supporting multi-arch images based on these release notes but not sure what are the mechanics of using this new version for cephcsi.

So we won't be able to have multi-arch images, but instead specifically-named images for certain types?

@cyb70289 yes that a temp solution for now till quay supports it

Thanks for the clarification. I would suggest re-naming this PR to indicate that what is being delivered are multiple different arch images and not multi-arch images. These are very much different things and it would be important to note the distinction for those who are expecting to be able to run these images in a multiple-arch cluster.

Yes PR title needs to be changed,  we  will update doc to use this image if you are running in  different architecture

Also, it looks like quay 3.0 started supporting multi-arch images based on these release notes but not sure what are the mechanics of using this new version for cephcsi.

AFAIK ceph-csi uses free service provided by the quay, so we  cant use multi-arch support for now

AFAIK ceph-csi uses free service provided by the quay, so we cant use multi-arch support for now

This is not relevant. Quay.io does not support multi-arch images for now, it is a feature that will be added later. Reply from Quay Support on the topic:

Mutliarch images are supported through the Docker v2 schema 2, which is yet not generally enabled. This feature is still under
testing and will available for usage after next general roll out.

@humblec PTAL

Thanks @Madhu-1 @humblec
I see cephcsi arm64 image is pushed, great !
https://quay.io/repository/cephcsi/cephcsi?tab=tags

@cyb70289 Thanks for the work!


AdminLTE layout-top-nav not working

I added the class 'layout-top-nav' in body tag  but the layout not changed. The logo not show and the menu not change to top continuing the same place. Any fix to that?

@jhosepharaujo Which version you using?

question related to adminlte https://adminlte.io/docs/2.4/layout

In this case, everything will be already published in your code. so you can better find it and fix it. Not a generator issue.

@jhosepharaujo Which version you using?

I using 5.7 version.


SourceLink.exe srctoolx

The goal is to add a SourceLink.exe srctoolx command that has the same output as running srctool -x and also using the same native library. In this case, it would use SourceLink.SymbolStore. srcsrv.dll would probably need to be redistributed as well, which is permitted.
On the Mozilla mailing list from 2013-09:
I should also point out that the versions of dbghelp.dll, symsrv.dll, and srcsrv.dll that are included with Debugging Tools for Windows are redistributable. See http://msdn.microsoft.com/en-us/library/windows/desktop/ms679294(v=vs.85).aspx for more information.

tested with:
srctool.exe -x .\packages\FAKE\tools\FakeLib.pdb
.\Exe\bin\Debug\SourceLink.exe srctoolx -p .\packages\FAKE\tools\FakeLib.pdb
.\bin\SourceLink.0.5.0-ci1503142136\tools\SourceLink.exe srctoolx -p .\packages\FAKE\tools\FakeLib.pdb

All output matches.


Change Backup and Import Options in settings screen on UI/UX

[ ] Update transaction history screen to show different tabs for USD and BTC transactions
[ ] Change default memo to Pay to Flash Wallet User (username)


Add Clipboard as an optional output of the ObjectDumper extension

Source: Visual Studio Marketplace Q&A
Denis Pujdak:

If there was an option to just simply copy Json to the clipboard (without opening a window) it would be really cool.

Me:

Good idea. How it can be implemented better:

As a setting, where to output the result - a new document tab or the clipboard.
As a separate set of commands.
Or somehow else?
Welcome to open a feature request.

Denis Pujdak:

For me the option would be more convenient. I think people would use one over other but not both at the same time. But I could be wrong. By default it would be the window one.

Me:

One more question. The opened window naturally shows that process has been finished. What should indicate the same for clipboard output?

Denis Pujdak:

Not sure which options are available. I believe there could be some notification popup, but it should not require any user interaction (like modal popup which requires to close it). Or as a result it could be simply a context menu disappearing. I cannot see a problem if there is no indication at all.

Blocked by https://github.com/ycherkes/ObjectDumper/issues/68


[Improvement][Logger]Logger server integrate into worker server

Purpose of the pull request
Because Logger Server is a separate process, it increases the cost of deployment and maintenance, so integrate Logger server into Worker server and update the corresponding deployment scripts, Docker, K8s, etc
Brief change log
Logger server integrate into Worker server
Remove the Logger Server from the deployment script, docker,docker-swarm,K8s

Verify this pull request
This pull request is already covered by existing tests

It seem that you totally remove logger server. why are you going to do that? Did you missing something unexpect in your env, or just think logger server is not unnecessary.

Hi @aaronlinv , please resolve the conflicts, and add the issue for this pr, thxs.

@caishunfeng Okay, I'll submit it later

@caishunfeng Okay, I'll submit it later

Looking forward to this PR. you coould ping ShunFeng to review after you fix the conflict, we would review ASAP.

@caishunfeng @zhongjiajie Because of the huge changes in 2.0 architecture, so I reworked and submitted the code, the deployment and documentation part I will submit the relevant commit after the merging, I hope to review and merge as soon as possible, so as to continue to follow up the subsequent iterations

Codecov Report

Merging #6894 (a765f21) into dev (d353385) will decrease coverage by 0.14%.
The diff coverage is 37.50%.

@@             Coverage Diff              @@
##                dev    #6894      +/-   ##
============================================
- Coverage     41.26%   41.12%   -0.15%
+ Complexity     3673     3663      -10
============================================
  Files           641      640       -1
  Lines         26473    26457      -16
  Branches       2961     2961
============================================
- Hits          10925    10881      -44
- Misses        14523    14552      +29
+ Partials       1025     1024       -1

Impacted Files
Coverage Δ

...e/dolphinscheduler/server/worker/WorkerServer.java
0.00% <0.00%> (ø)

...erver/worker/processor/LoggerRequestProcessor.java
20.31% <ø> (ø)

...nscheduler/api/service/impl/LoggerServiceImpl.java
66.15% <100.00%> (ø)

...er/master/dispatch/host/assign/RandomSelector.java
77.77% <0.00%> (-5.56%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d353385...a765f21. Read the comment docs.

I just approve to run PR CI, PTAL after CI green @caishunfeng

PTAL @lenboo

@aaronlinv For the task Condition, SubProcess, and Dependent, integrating the logger into the master also needs to be considered. At the same time, do not concat the magic number 1234 or 5678, you should directly call processInstance.getHost() or taskInstance.getHost()

@chengshiwen @lenboo Logger Server has been integrated into the Master, so that other tasks can view logs normally, and the IP and port are read directly by TaskInstance.

Please remember change our dev doc after this PR merged.

@aaronlinv  I add label miss document for this PR, please change doc in https://dolphinscheduler.apache.org/en-us/docs/dev/user_doc/guide/installation/pseudo-cluster.html dev dir. our website repo is in https://github.com/apache/dolphinscheduler-website

@aaronlinv +1


File "/home/xxx/.conda/envs/tensorflow/lib/python2.7/site-packages/gpflow/core/compilable.py", line 110     class ICompilable(metaclass=AutoBuild):       ^ SyntaxError: invalid syntax

installed tensorflow 1.4.0 with Anaconda  on Ubuntu 16.4  ,  testing ok for tensorflow .
and  type "pip install ."  or "python setup.py develop" in the virtual env named tensorflow  ,  there is the same error like "
File "gpflow/init.py", line 28, in
from . import transforms
File "gpflow/transforms.py", line 22, in
from .core.base import ITransform
File "gpflow/core/init.py", line 17, in
from .compilable import Build
File "gpflow/core/compilable.py", line 110
class ICompilable(metaclass=AutoBuild):
^
SyntaxError: invalid syntax
How to handle this ?  Any help is appreciated .  Thank you very much .

Hi, what's the output of python --version? GPflow now supports python3 only. If you are still using python2, I would strongly suggest switching to python3, but if that's not possible, you can still use the previous release of GPflow: https://github.com/GPflow/GPflow/tree/0.5.0

@st--  thank you .  Yes , I still use python2 . And I switch to python 3.5 with anaconda and tensorflow 1.4.0 , then it works .


Recontribute our updated chai.d.ts to DefinitelyTyped

This is a distraction, but I had to update chai.d.ts in order to reproduce the pouchdb tests. Really it needs contributing back to DefinitelyTyped, but it lookes like a fair bit of work to update their chai-tests.ts file and I don't have time right now.

Complete: This was done on 24th May 2015: borisyankov/DefinitelyTyped#4372


Create synthetic data generator script

Description of change
This should be used when enabling a new test environment and new
synthetic data is needed. It can create Companies, Contacts, Advisers,
Interactions which are connected with each other.
Checklist

[ ] Has a new newsfragment been created? Check changelog/README.md for instructions
[ ] Do any added or updated endpoints appear in the API documentation? See docs/Maintaining the API documentation.md for more details
[ ] Have any relevant search models been updated?
[ ] Have any relevant fixtures (fixtures/test_data.yaml) been updated?
[ ] Have any relevant select-/prefetch-related field lists in the views and search apps been updated?
[ ] Has the admin site been updated (for new models, fields etc.)?
[ ] Has the README been updated (if needed)?

Currently on hold due to another feature being prioritised. Do get in touch with me if you want to use it, as it's usable, just not having the necessary test coverage and documentation.


GROUP_CONCAT With string and not identifier

Hello,
It seems to be not possible to use your GROUP_CONCAT DQL method with simple strings args :
For example, i want to generate JSON string with this instruction  : GROUP_CONCAT('{report:"',r.label,'", expiration_date:"', ur.expiration_date, '"}').

Hi, @Gawel1
In this case CONCAT must be used.
Group concat is equal to http://dev.mysql.com/doc/refman/5.7/en/group-by-functions.html#function_group-concat

The problem is there can be more than one line :
Example :
a table with :
a | b | c
a | d | e
I want in output : a | [{label1 : b, label2 : c},{label1 : e, label2 : e}] and group_concat works fine for do that.

Please, provide your DQL

This MYSQL request give the result i want :
SELECT u.email, CONCAT(u.first_name,' ', u.last_name) AS full_name, ur.is_active AS activated_licence, CONCAT('[', GROUP_CONCAT('{report:"',r.label,'", expiration_date:"', ur.expiration_date, '"}'), ']') AS licences FROM user u left join user_report ur ON ur.user_id=u.id left join report r ON ur.report_id=r.id WHERE u.company_id=%d AND u.status =  AND u.role != %s GROUP BY u.email ORDER BY u.email;

Hi, @Gawel1
Issue was found and fixed. Thank you for patience


fix #103; fix #96; Coerce checkbox values (without allowing booleans)

@pvrobays here's the other PR for the checkboxes, if you'd prefer to explicitly disallow invalid values as defaults, just merge one of these and close the other :)

Closing this PR as we've chosen another route: we'll allow single boolean values for checkboxes. See pull #104


Update microsoft-365-security-center-mde.md

Learn Build status updates of commit 74333dd:
:bulb: Validation status: suggestions

File
Status
Preview URL
Details

microsoft-365/security/defender/microsoft-365-security-center-mde.md
:bulb:Suggestion

Details

microsoft-365/security/defender/microsoft-365-security-center-mde.md

Line 74, Column 53: [Suggestion: preserve-view-not-set - See documentation] You've pinned this link to a specific version of content with the view parameter. It's recommended not to pin a version unless that version is A) not the default view and B) the context is about that version specifically. To proceed with pinning a version add the &preserve-view=true to the URL.  Otherwise, remove the view parameter. URL: /microsoft-365/security/defender-endpoint/gov?view=o365-worldwide

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
Note: Your PR may contain errors or warnings or suggestions unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
For any questions, please:Try searching the learn.microsoft.com contributor guidesPost your question in the Learn support channel

Learn Build status updates of commit f46ca24:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

microsoft-365/security/defender/microsoft-365-security-center-mde.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the learn.microsoft.com contributor guidesPost your question in the Learn support channel

Learn Build status updates of commit 1b38971:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

microsoft-365/security/defender/microsoft-365-security-center-mde.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the learn.microsoft.com contributor guidesPost your question in the Learn support channel

Learn Build status updates of commit 9285788:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

microsoft-365/security/defender/microsoft-365-security-center-mde.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the learn.microsoft.com contributor guidesPost your question in the Learn support channel


fix(fontupload): Allow user uploaded fonts and apply

What this PR does 📖
-Adds save files specifically for font uploads. Creates short lived blob urls to handle files in indexdb and keeps track locally in svelte Stores.
Which issue(s) this PR fixes 🔨

Resolve #170

Special notes for reviewers 🗒
Ticket was originally for uploading emoji but this involves some more discussion as pointed out by Shelly, uploaded emoji would only work for user that uploaded and would be broken when sent to another user, (unless the receiver has the same installed emoji)
Current functionality allows upload of specific types of files (we can add extra but these are the ones I found), .woff, .woff2, .otf, and .ttf
Some notes on current functionality:
uploaded fonts should load in and set automatically on user upload
user uploads should persist across the app on navigation but also page refreshes
Additional comments 🎤
Currently there is no error (toast) notifications if a user uploads a non-font file within font upload, (noted above in excepted file types) I am working on this currently

@Jekrimo The automated tests failing are related to the change on the following UI locator. I went ahead and added a pr in testing repo to handle this change in tests: https://github.com/Satellite-im/automated-tests-web/pull/160

Thanks @luisecm I will keep these in mind in the future for future pr's so I don't give you more work!

This looks great, the only thing I'd say we should change is let's remove the theme code since that is introduced in another PR

@InfamousVague Do you think we should hide the file upload buttons for these for now?

I haven't checked out the identicon/theme needs for the app in terms of file type, limitations, and such,  but we had talked in the meeting about the emoji not working for the receiver and the issue that can cause having the glyphs come through as gobblygook for said reciever.

Found one bug introduced
When you go to Settings Customizations for the first time it looks like this in your branch:

Moon theme button appears as "clear button". Also "Default" is not showing in dropdown selected. After that, if you click it, button changes to moon theme
Currently on dev, when going for the first time to screen, moon button appears and Default appears as dropdown value

That was one billion percent my fault. I was trying to type our some stuff for themes but after I broke it I missed some stuff, my bad! I removed all the stuff that was messing it up and it should be good now! Thanks @luisecm !

@Jekrimo all good now, thanks for fixing it!

Added a couple extra checks today to weed out too large of files and unsupported files


Keep Pandas-like Series original name

What type of PR is this? (check all applicable)

[ ] 💾 Refactor
[ ] ✨ Feature
[x] 🐛 Bug Fix
[ ] 🔧 Optimization
[ ] 📝 Documentation
[ ] ✅ Test
[ ] 🐳 Other

Related issues
As discussed in #204, @MarcoGorelli is it what you had in mind?
Checklist

[x] Code follows style guide (ruff)
[x] Tests added
is it the correct place? Or should we create a pandas_like/test_series.py file?
[ ] Documented the changes

If you have comments or can explain your changes, please do so below.
Should we also add a line in the docs that say that certain operations assume str column names and may not work (e.g. DataFrame.__getitem__) ?
Or maybe we can have a warning that is triggered when the column name is not str (both in Series and DataFrame)

thanks @EdAbati ! could you run pytest narwhals --doctest-modules too? there's a doctest that needs fixing up

Should we also add a line in the docs that say that certain operations assume str column names and may not work (e.g. DataFrame.getitem) ?
Or maybe we can have a warning that is triggered when the column name is not str (both in Series and DataFrame)

tbh i think it's ok as-is, a warning risks being annoying. people using narwhals will probably pass in series/dataframe names taken from objects their users pass anyway, so using whatever the user passes should be fine


CustomDragLayer slow performance

Is there any reason why the CustomDragLayer for me is giving me bad performance? Its like theres low fps, or rather, the arbitrary object that I am dragging around is lagging and not looking smooth

I also experienced this. The problem here is, that even if you render pure components that don't change in the drag layer, the offset change triggers a at least a react reconciliation on every single mouse move. This is very inefficient if one has the common case, where the drag layer should just display a non-changing custom preview element.
To work around that, I duplicated the DragLayer.js the does the offset rendering for me and this in a high performant manner, meaning changing the style of the container that moves around the preview directly.
// PerformantDragLayer.js
/* eslint-disable */
import React, { Component, PropTypes } from 'react';
import shallowEqual from 'react-dnd/lib/utils/shallowEqual';
import shallowEqualScalar from 'react-dnd/lib/utils/shallowEqualScalar';
import isPlainObject from 'lodash/isPlainObject';
import invariant from 'invariant';
import checkDecoratorArguments from 'react-dnd/lib/utils/checkDecoratorArguments';
import hoistStatics from 'hoist-non-react-statics';

function layerStyles(isDragging) {
  return {
    position: 'fixed',
    pointerEvents: 'none',
    zIndex: 1,
    left: 0,
    top: 0,
    width: isDragging ? '100%' : 0,
    height: isDragging ? '100%' : 0,
    opacity: isDragging ? 1 : 0
  };
}

export default function DragLayer(collect, options = {}) {
  checkDecoratorArguments('DragLayer', 'collect[, options]', ...arguments);
  invariant(
    typeof collect === 'function',
    'Expected "collect" provided as the first argument to DragLayer ' +
    'to be a function that collects props to inject into the component. ',
    'Instead, received %s. ' +
    'Read more: http://gaearon.github.io/react-dnd/docs-drag-layer.html',
    collect
  );
  invariant(
    isPlainObject(options),
    'Expected "options" provided as the second argument to DragLayer to be ' +
    'a plain object when specified. ' +
    'Instead, received %s. ' +
    'Read more: http://gaearon.github.io/react-dnd/docs-drag-layer.html',
    options
  );

  return function decorateLayer(DecoratedComponent) {
    const { arePropsEqual = shallowEqualScalar } = options;
    const displayName =
      DecoratedComponent.displayName ||
      DecoratedComponent.name ||
      'Component';

    class DragLayerContainer extends Component {
      static DecoratedComponent = DecoratedComponent;

      static displayName = `DragLayer(${displayName})`;

      static contextTypes = {
        dragDropManager: PropTypes.object.isRequired
      }

      getDecoratedComponentInstance() {
        return this.refs.child;
      }

      shouldComponentUpdate(nextProps, nextState) {
        return !arePropsEqual(nextProps, this.props) ||
          !shallowEqual(nextState, this.state);
      }

      constructor(props, context) {
        super(props);
        this.handleOffsetChange = this.handleOffsetChange.bind(this);
        this.handleStateChange = this.handleStateChange.bind(this);

        this.manager = context.dragDropManager;
        invariant(
          typeof this.manager === 'object',
          'Could not find the drag and drop manager in the context of %s. ' +
          'Make sure to wrap the top-level component of your app with DragDropContext. ' +
          'Read more: http://gaearon.github.io/react-dnd/docs-troubleshooting.html#could-not-find-the-drag-and-drop-manager-in-the-context',
          displayName,
          displayName
        );

        this.state = this.getCurrentState();
      }

      componentDidMount() {
        this.isCurrentlyMounted = true;

        const monitor = this.manager.getMonitor();
        this.unsubscribeFromOffsetChange = monitor.subscribeToOffsetChange(
          this.handleOffsetChange
        );
        this.unsubscribeFromStateChange = monitor.subscribeToStateChange(
          this.handleStateChange
        );

        this.handleStateChange();
      }

      componentWillUnmount() {
        this.isCurrentlyMounted = false;

        this.unsubscribeFromOffsetChange();
        this.unsubscribeFromStateChange();
      }

      handleOffsetChange() {
        if (!this.isCurrentlyMounted) {
          return;
        }

        const monitor = this.manager.getMonitor();
        const offset = monitor.getSourceClientOffset();
        const offsetDiv = this.refs.offset;
        if (offset && offsetDiv) {
          offsetDiv.style.transform = `translate(${offset.x}px, ${offset.y}px)`;
        }
      }

      handleStateChange() {
        if (!this.isCurrentlyMounted) {
          return;
        }

        const nextState = this.getCurrentState();
        if (!shallowEqual(nextState, this.state)) {
          this.setState(nextState);
        }
      }

      getCurrentState() {
        const monitor = this.manager.getMonitor();
        return {
          collected: collect(monitor),
          isDragging: monitor.isDragging()
        };
      }

      render() {
        return (

              {this.state.isDragging && <DecoratedComponent {...this.props} {...this.state.collected} ref='child' />}

        );
      }
    }

    return hoistStatics(DragLayerContainer, DecoratedComponent);
  };
}

This can then be used the following way:
// MyCustomDragLayer.js
import PerformantDragLayer from './PerformantDragLayer'
import React, {PropTypes} from 'react'

class CustomDragLayerRaw extends React.Component {
  static propTypes = {
    item: PropTypes.any,
    itemType: PropTypes.string
  }

  render () {
    const {item, itemType} = this.props
    return
{itemType}

  }
}

export default PerformantDragLayer((monitor) => ({
  item: monitor.getItem(),
  itemType: monitor.getItemType()
})(CustomDragLayerRaw)

The performance improvment is very noticable:

Of course the default DragLayer is still more flexible. My more performant implementation is just faster, because it handles a special case. But I guess this special case is very common.
@gaearon Is such a specialized but way more performant implementation interesting for you to be integrated as alternative into react-dnd?

@choffmeister Hi. Thanks for your post. I had a brief look at it a day ago but only looked at it in depth just then. The code you posted seems very specialised within your implementation, could you please perhaps point out the important aspects needed for this?
I am not sure if all of those plugins, libraries and other code you have is part of the solution or not.
Thanks

I have experimented around and managed to make it work. It seems that the content you want in your "drag ghost thing" should be in the 'offset' div. I don't know the optimal way to do this, but in my test, I just put a similar div to the dragged content. I've yet to make the drag layer item the same as the dragged item, but passing in some data and changing the drag layer item's state isn't too hard to do.
The performance is very good, though it only works when dragging outside of the container. If you drag in circles around the item, the performance is also as flawless as the previous case (i.e. you have a list of 4 items, dragging and keeping the mouse in the area of the 4th item). However, if you draw circles with your mouse, dragging, across all the 4 elements, it still lags, like before, but with a slight fps improvement (nonetheless its still laggy). But this implementation is a good start. At least for most common cases when you want to drag from one container to another, then this will definitely keep the UX good.

What exactly do you mean by laggy:

Is the FPS to low so that it looks stuttering, or
is the element moving smoothly, but while moving it it always some distance behind the mouse movement?

I'd say both

Upon further usage, I realise that the solution doesn't really eliminate lag altogether. I have a card div that has quiet a bit of features in it. Your solution did initially provide a huge fps boost compared to the example in the docs, however, as I make my div more complicated, it starts to lag like I didn't have this.
It seems at this point that react-dnd isn't the best option for very fancy looking UX. I believe that no other react library comes close to having a proper drag and drop feature, so I am most likely going to try out some existing jQuery libraries.
Also if anyone would like to try and see if it also lags for them, I had used a Semantic UI card, with lots of elements inside of it. My placeholder card had 2 titles, 5 semantic "label"s, 3 semantic "icon"s, and an avatar image.

After further testing with this different example at: http://react-trello-board.web-pal.com/ (its implementation is very similar to what choffmeister had posted earlier), I still obtain lag when trying to drag my complicated div. I'm glad that this is the conclusion I have come to, as I don't need to do some more experimenting with code; the problem lies with my div, which can be easily fixed.

I'm still experiencing this pretty badly using VERY similar code to the example in the docs. It must be the transform calculation having every tiny mouse move?
Has anyone else found a solution to this?

I've had a performance problem, even with PureComponents. Using the 'Highlight Updates' feature in the Chrome React Dev plug-in to diagnose seems like all my components were being updated when the currentOffset prop was getting changed. To mitigate I made sure that the DragLayer that passes on this prop only contains the drag preview itself, even though there was already a DragLayer at the top-level of the app to track what item is being dragged.
I haven't tried adapting @choffmeister's solution but it looks like it would also solve the issue.  I think perhaps something like that should be considered for the core library because I can only imagine that this is a common problem for anyone implementing drag previews.

I made sure that the DragLayer that passes on this prop only contains the drag preview itself, even though there was already a DragLayer at the top-level of the app

I'm not sure I follow! I only have the one custom drag layer in my app, I believe!

It's a problem specific to my use case – a drag layer was already being used before a custom drag preview was needed, in order to track what item was being dragged. I had initially thought it made sense to use the same drag layer to do the custom drag preview when I added it but, since this was at the top level of my application, changing the props for this on every slight movement was causing a lot of extra component updates.

It seems like the react-dnd-html5-backend has terrible performance with a custom DragLayer whereas the react-dnd-touch-backend has OK performance.

Kinda obvious, but it helps:
let updates = 0;
@DragLayer(monitor => {
  if (updates++ % 2 === 0) {
    return {
      currentOffset: monitor.getSourceClientOffset()
    }
  }
  else {
    return {}
  }
})

Thanks @aks427 I'll try that and see if it improves further (using a custom drag layer like above too which helped a lot in MOST cases but not all)

@dobesv  change to react-dnd-touch-backend solved the problem. I also tried react-throttle-render, but still seems not good.

Yeah, I know the touch backend works, but it doesn't support drag and drop file uploads, which I want to use.

@dobesv we use https://react-dropzone.js.org/ for drag and drop files  :)  maybe it's good for you as well.

Kind of a no brainer, but I used the performant drag layer fix and spent a
LOT of time looking at my code to limit the number of renders that were
being called by the app (lots of switching Component -> PureComponent) and
in the end I didn't even need to use the react-throttle-render.
I DEFINITELY recommend looking into using PureComponent and trying to
maximize your code instead of looking for an easy fix on this thread if
your render is slow! I'll post more if we improve the Performant model at
all though!

@framerate Even using PureComponents it's an issue because the reconciliation algo is expensive to run on every mouse move. Just profile ur app with Chrome DevTools and CPU throttling to 4X, which is a realistic slowdown for mid-end mobile devices.

For posterity and anyone else who is struggling with the drag performance, a workaround is to use a combination of mouse events and PureComponent:
export default DragLayer(monitor => ({
  itemBeingDragged: monitor.getItem(),
  componentType: monitor.getItemType(),
  initialClientOffset: monitor.getInitialClientOffset(),
  beingDragged: monitor.isDragging()
}))(
  class CustomDragLayer extends React.PureComponent {
    constructor(props) {
      super(props);
      this.onMouseMove = this.onMouseMove.bind(this);
    }

    onMouseMove(event) {
      const { x, y } = this.props.initialClientOffset;
      const { clientX, clientY } = event;
      const transform = `translate(${clientX - x}px, ${clientY - y}px)`;
      this.rootNode.style["transform"] = transform;
      this.rootNode.style["-webkit-transform"] = transform;
    }

    componentDidUpdate(prevProps) {
      if (!prevProps.beingDragged && this.props.beingDragged) {
        document.addEventListener("mousemove", this.onMouseMove);
      } else if (prevProps.beingDragged && !this.props.beingDragged) {
        document.removeEventListener("mousemove", this.onMouseMove);
      }
    }

    render() {
      if (!this.props.beingDragged) return null;
      return (

 (this.rootNode = el)}
          style={{
            position: "fixed",
            pointerEvents: "none",
            top: this.props.itemBeingDragged.top,
            left: this.props.itemBeingDragged.left
          }}
        >
          {renderComponent(
            this.props.componentType,
            this.props.itemBeingDragged
          )}

      );
    }
  }
);

I used @stellarhoof 's solution but I needed to make props available in my onOffsetChange function.  I was able to accomplish it with this collect function:
let unsubscribe = null;
let lastPropsSeen = null;

export default DragLayer((monitor, props) => {
    if (!subscribedToOffsetChange) {
        unsubscribe = monitor.subscribeToOffsetChange(onOffsetChange(monitor, props));
        subscribedToOffsetChange = true;
        lastPropsSeen = props;
    } else if (lastPropsSeen != props && unsubscribe != null) {
        unsubscribe();
        unsubscribe = monitor.subscribeToOffsetChange(onOffsetChange(monitor, props));
        lastPropsSeen = props;
    }

    return {
        item: monitor.getItem(),
        itemType: monitor.getItemType(),
        isDragging: monitor.isDragging(),
    };
})(MyComponent);

Be aware it is doing a shallow equals.

@stellarhoof  Thank you for great answer! Unfortunately the solution does not work on IE11 for me. subscribeToOffsetChange does not seem to call the callback we pass into it. Fortunately, I was able to fix it by not using subscribeToOffsetChange, but instead just setting the translations inside of the collector like so:
let dragLayerRef: HTMLElement = null;

const layerStyles: React.CSSProperties = {
  position: 'fixed',
  pointerEvents: 'none',
  zIndex: 100,
  left: 0,
  top: 0,
  width: '100%',
  height: '100%',
};

const dragLayerCollector = (monitor: DragLayerMonitor) => {
  if (_.isObject(dragLayerRef)) {
    const offset = monitor.getSourceClientOffset() || monitor.getInitialClientOffset();

    if (_.isObject(offset)) {
      dragLayerRef.style["transform"] = `translate(${offset.x}px, ${offset.y}px)`;
    } else {
      dragLayerRef.style["display"] = `none`;
    }
  }

  return {
    item: monitor.getItem(),
    isDragging: monitor.isDragging(),
  };
};

export default DragLayer(dragLayerCollector)(
  (props): JSX.Element => {
    if (!props.isDragging) {
      return null;
    }

    return (

 dragLayerRef = ref }>test

    );
  }
);

@framerate https://github.com/stellarhoof/trello-clone/blob/48d8b8f5277cfc4d50ebf0c517562d3bf9c7ac7a/src/app/DragPreview.js

@stellarhoof Thank you for the great answer! Unfortunately the solution does not work on IE11 for me. subscribeToOffsetChange does not seem to call the callback we pass into it. Fortunately, I was able to fix it by not using subscribeToOffsetChange, but instead just setting the translations inside of the collector like so:
let dragLayerRef: HTMLElement = null;

const layerStyles: React.CSSProperties = {
  position: 'fixed',
  pointerEvents: 'none',
  zIndex: 100,
  left: 0,
  top: 0,
  width: '100%',
  height: '100%',
};

const dragLayerCollector = (monitor: DragLayerMonitor) => {
  if (dragLayerRef) {
    const offset = monitor.getSourceClientOffset() || monitor.getInitialClientOffset();

    if (offset) {
      dragLayerRef.style["transform"] = `translate(${offset.x}px, ${offset.y}px)`;
    } else {
      dragLayerRef.style["display"] = `none`;
    }
  }

  return {
    item: monitor.getItem(),
    isDragging: monitor.isDragging(),
  };
};

export default DragLayer(dragLayerCollector)(
  (props): JSX.Element => {
    if (!props.isDragging) {
      return null;
    }

    return (

 dragLayerRef = ref }>test

    );
  }
);

This worked for me!
I'm using this version: 9.4.0
The others solutions seems to work only for older versions.

For people that uses hooks (useDragLayer hook) and lands here: I opened a ticket specifically about the hook implementation, with a workaround proposal here : https://github.com/react-dnd/react-dnd/issues/2414

Agreed, hooks implementation is a little optimistic in regards to performance. This seems to work like a charm for me.
export default function DragLayer() {
  const dragDropManager = useDragDropManager();
  const monitor = dragDropManager.getMonitor();
  const [item, setItem] = React.useState();

  const updateState = React.useCallback(() => setItem(monitor.getItem()), [monitor]);
  React.useEffect(() => monitor.subscribeToStateChange(updateState), [monitor, updateState]);

  const previewRef = React.useRef();
  const updateOffset = React.useCallback(() => {
    if (previewRef.current) {
      const offset = monitor.getClientOffset();
      if (offset) {
        previewRef.current.style.left = `${offset.x}px`;
        previewRef.current.style.top = `${offset.y}px`;
        previewRef.current.style.visibility = "";
      } else {
        previewRef.current.style.visibility = "hidden";
      }
    }
  }, [monitor]);
  React.useEffect(() => monitor.subscribeToOffsetChange(updateOffset), [monitor, updateOffset]);

  if (!item) return null;

  // implement render logic here
  if (item.type === "IMAGE") {
    return (

    );
  }

  return null;
}

Hi guys. I stumbled upon the same problem but didn't like the hackish solutions with the direct access / event handling above.
The rule of thumb for efficient hooks to prevent re-renders and reconciliation is: Do not output more than necessary from a hook returning state
This is what the original solution everyone copies doesn't honor, creating a new object with very volatile properties each time, that is your problem (as many properly pointed out).
What I did was rather simple ... I output only the transform part as a string (not a new object). In addition to that, I simply round the transform coordinates in it to reduce the number of rerenders that are actually needed.
interface IDragPreviewProps {
  readonly renderPreview: () => React.ReactNode;
  readonly sourceId: string;
  readonly width?: number;
}

const DragPreview: React.FC<IDragPreviewProps> = ({
  renderPreview,
  sourceId,
  width,
}) => {
  const getPreviewTransform = useCallback(
    (monitor: DragLayerMonitor): string | null => {
      const item = monitor.getItem();
      const initialOffset = monitor.getInitialSourceClientOffset();
      const currentOffset = monitor.getSourceClientOffset();
      const isDragging = monitor.isDragging();

      if (!isDragging || (item.sourceId !== sourceId) || !initialOffset || !currentOffset) {
        return null;
      }

      const { x, y } = currentOffset;

      return `translate(${Math.round(x)}px, ${Math.round(y)}px)`;
    },
    [sourceId],
  );
  const transform = useDragLayer(getPreviewTransform);

  if (!transform) {
    return null;
  }

  return (
    <div
      className="drag-layer"
      style={width ? { width: `${width}px` } : undefined}
    >

        {renderPreview()}

  );
};

.drag-layer {
	position: fixed;
	top: 0;
	left: 0;
	z-index: 100;
	width: 100%;
	height: 100%;
	pointer-events: none;
	opacity: 0.7;
}

No hacks with direct element access, no additional events, works like a charm.
For further optimization, I could imagine further debouncing or rounding to more rough values accompanied
Note that I place render this preview component directly to the component being dragged, so I match the sourceId from my item and sync width of the preview based on the source width (using useResizeObserver)

@kenticomartinh Thanks for posting this! Makes sense to me as a simple solution to optimizing performance here.
When I use useDragLayer, I find that when you release the mouse when not over a drop target, there is a delay before isDragging returns false.
Is this a related performance issue? Or am I missing something about how this hook works.

Hi @jamesopti , I can't reproduce it on your sandbox, it drops immediately. Are you still experiencing that issue?

Hi guys!
Recently I've also ran in this issue and after a while I came up to the solution with direct access to ref
It is kind of a summary of what I read above, so I  just moved logic to a hook that could be reused wherever dnd preview is necessary
And as an optimization of numerous DOM updates I'm using requestAnimationFrame
const useDragPreview = () => {
  const previewRef = useRef();
  const monitorRef = useRef();
  const [item, setItem] = useState(null);
  const dragDropManager = useDragDropManager();
  const updateItem = useCallback(() => setItem(monitorRef.current.getItem()), []);
  const updateOffset = useCallback(() => requestAnimationFrame(() => {
    if (previewRef.current) {
      const offset = monitorRef.current.getSourceClientOffset() || {};
      previewRef.current.style.transform = `translate(${offset.x}px, ${offset.y}px)`;
    }
  }), [horizontal, vertical]);

  useEffect(() => {
    let unsubscribeToStateChange;
    let unsubscribeToOffsetChange;

    if (!monitorRef.current) {
      monitorRef.current = dragDropManager.getMonitor();
      unsubscribeToStateChange = monitorRef.current.subscribeToStateChange(updateItem);
      unsubscribeToOffsetChange = monitorRef.current.subscribeToOffsetChange(updateOffset);
    }

    return () => {
      if (typeof unsubscribeToStateChange === 'function') unsubscribeToStateChange();
      if (typeof unsubscribeToOffsetChange === 'function') unsubscribeToOffsetChange();
    };
  }, [dragDropManager, updateItem, updateOffset]);

  return [item, previewRef];
};

and then I apply it in the preview component
const DragPreview = () => {
  const [item, previewRef] = useDragPreview();
  const showPreview = item && item.type === DRAG_TYPE.GROUP;

  return (

      {showPreview && (
        <VendorTypeGroupView
          ref={previewRef}
          {...item}
        />
      )}

  );
};

It works nice and looks pretty easy - subscribe and set position in the callback
PS. I might be missing something, so please let me know if something does not look good

@AliakseiYakubuk  I actually ended up with something similar (directly modifying the style) working with preview components with larger DOM trees which were slow even on React reconciliation.
What you may consider:

I am using an extra isBusy flag as the updates may pile up still causing delays and a queue of future CPU allocation on slower machines (not sure if your solution has some throttling). The isBusy flag is throttling it without the overhead of setTimeout.
I render the preview to body to achieve proper z-index placement over everything else with z-index. I also observed that using position: absolute instead of fixed gives better frame time in Chrome. Don't ask me why. So I use that (I can, because of the placement in body).

const DragPreview: React.FC<IDragPreviewProps> = ({
  renderPreview,
  sourceId,
  sourceRef,
}) => {
  const previewRef = useRef<HTMLDivElement>(null);
  const lastTransform = useRef<string | null>(null);
  const isBusy = useRef(false);

  const getShouldRenderPreview = useCallback(
    (monitor: DragLayerMonitor): boolean => {
      const isDragging = monitor.isDragging();
      const item = monitor.getItem();
      if (!isDragging || (item.sourceId !== sourceId)) {
        lastTransform.current = null;
        isBusy.current = false;
        return false;
      }

      if (isBusy.current) {
        return !!lastTransform.current;
      }

      const initialOffset = monitor.getInitialSourceClientOffset();
      const currentOffset = monitor.getSourceClientOffset();

      if (!initialOffset || !currentOffset) {
        lastTransform.current = null;
        isBusy.current = false;
        return false;
      }

      const { x, y } = currentOffset;

      const newTransform = `translate(${Math.round(x)}px, ${Math.round(y)}px)`;
      if (newTransform !== lastTransform.current) {
        isBusy.current = true;
        lastTransform.current = newTransform;
        requestAnimationFrame(() => {
          if (previewRef.current) {
            previewRef.current.style.transform = newTransform;
            requestAnimationFrame(() => isBusy.current = false);
          }
        });
      }
      return true;
    },
    [sourceId],
  );

  const shouldRenderPreview = useDragLayer(getShouldRenderPreview);
  if (!shouldRenderPreview) {
    return null;
  }

  const width = sourceRef.current?.offsetWidth;

  return ReactDOM.createPortal(
    (
      <div
        className="drag-preview"
        style={{
          ...(width ? { width: `${width}px` } : undefined),
          ...(lastTransform.current ? { transform: lastTransform.current } : undefined),
        }}
        ref={previewRef}
      >
        {renderPreview()}

    ),
    document.body,
  );
};

@fromi Sorry for late response, I had GitHub notifications redirected to a side folder due to too much spam from our internal repos, but the rule was not great and even mentions ended up there :-(
Anyway, yes, requestAnimationFrame totally helps, I also used it. See my current version with some commentary above.

@kenticomartinh Your version contains a nice idea with not collecting when it is busy.
I integrated it in my solution to get a very generic and reusable hook:
import {useRef, useState} from 'react'
import {DragLayerMonitor, useDragLayer} from 'react-dnd'

export default function useEfficientDragLayer<CollectedProps>(collect: (monitor: DragLayerMonitor) => CollectedProps): CollectedProps {
  const [requestID, setRequestID] = useState<number>()
  const collected = useDragLayer(monitor => requestID === undefined ? collect(monitor) : undefined)
  const result = useRef(collected)
  if (requestID === undefined && result.current !== collected) {
    result.current = collected
    setRequestID(requestAnimationFrame(() => setRequestID(undefined)))
  }
  return result.current!
}

I really think this should be the default implementation for "useDragLayer" actually.

Hi @fromi and @kenticomartinh , currently i am using "usePreview hook from "react-dnd-multi-backend" to create a custom drag layer but i am facing some performance issue.

if you came here because your preview element moves worse than mouse pointers then check the transition property in your CSS file. This can save you several hours of no search results.

Здарова если ты пришёл сюда из за того что твой превью элемент двигается хуже указатели мыши то проверь свойство transition в твоем CSS файле. Это может спасти тебе несколько  часов без результатных поисков.

Здорова якщо ти прийшов сюди через те, що твій превью елемент рухається гірше покажчики миші то перевір властивість transition у твоєму CSS файлі. Це може врятувати тобі кілька годин без результатів.

Здарова калі ты прыйшоў сюды з-за таго што твой прэв'ю элемент рухаецца горш паказальнікі мышы то правер уласцівасць transition у тваім CSS файле. Гэта можа выратаваць табе некалькі гадзін без рэзультатных пошукаў.

const DragPreview = () => {
  const [item, previewRef] = useDragPreview();
  const showPreview = item && item.type === DRAG_TYPE.GROUP;

  return (

      {showPreview && (
        <Preview
          ref={previewRef}
          {...item}
        />
      )}

  );
};

Can you share all files? I couldn't understand what horizontal and vertical are.

@kenticomartinh Thanks for posting this! Makes sense to me as a simple solution to optimizing performance here.
When I use useDragLayer, I find that when you release the mouse when not over a drop target, there is a delay before isDragging returns false.
Is this a related performance issue? Or am I missing something about how this hook works.
EDIT: The issue I'm experiencing can be seen in this demo codesandbox.
If this is a separate issue, I can file a separate ticket.

@jamesopti did you ever find a solution to this? i'm seeing the same issue!

Can you share all files? I couldn't understand what horizontal and vertical are.

Hi, @corskaya!
I am sorry, updateOffset should have an empty array of dependencies.
I forgot to remove those two parameters: vertical and horizontal,  they were
used to enable/disable changing offset.x or offset.y


Web3Modal API v2 spec

Context
See: https://www.notion.so/walletconnect/Web3Modal-API-v2-Proposal-a2fc6323156d49f79d8092f5c39b8cda
Scope

Problem 4 & 5 from the proposal are out-of-scope for the spec, since they are implementation details.

TODO

[x] Move current (v1) spec into a separate markdown file where we can back-reference it
[x] Account for the endpoints that were added previously without updating the v1 spec
[ ] Finalize spec

@chris13524 I added an affordance for granular image size control as discussed and added by you to the proposal doc 👍


How tune proxy ipv6?

Hello to all!
I can not configure a private proxy.
Please, help.
ipv6
ip - 2a03:c011:169b:2f13:9eb7:65ad:1714:0d22
HTTP(s)-server: <IP_ADDRESS>:13200
login: 0zzzzh
pass: KDicnq1
type: HTTPS_
I write to the proxy parameter (-proxy 'htts://username:password@ip_address:port'), but Failed to parse written every time.

many thanks,
http://0zzzzh:KDicnq1@<IP_ADDRESS>:13200
work ok


VM Service removes breakpoints during a hot-restart when the app is paused

Steps to repro:

Set a breakpoint (or several) in Dart DevTools
Hit one of the breakpoints
While paused, perform a hot-restart
For each breakpoint that was set, DevTools receives a kBreakpointRemoved event: https://github.com/flutter/devtools/blob/2859d24b610d4dcf9d8e3248a9ae4eff862186ad/packages/devtools_app/lib/src/screens/debugger/breakpoint_manager.dart#L286

Expected result: There should be no kBreakpointRemoved events being sent.
I have verified that Dart DevTools is not sending a request to remove these breakpoints itself.
Note that when Dart DevTools triggers a hot restart while the app is not paused,  then there are not kBreakpointRemoved events sent.
FYI @bkonyi

Thinking about this more, I'm guessing it's because the vm_service first resumes the isolate before starting the hot-restart. Since we don't want to hit any breakpoints while resuming, it makes sense that the vm_service would first remove them all.

So it looks like this isn't a VM issue, but behavior in Flutter tools. So it looks like this is expected.


Fixing #14 (Options Dialog update)

This pull request is not ready to be merged yet. When ready, it should include a rewritten options dialog, together with support for integration with the Brackets' preferences system.

Rebased pull request in #22.


bd not working #Ubuntu16.04

I can't quite get this "bd" thing going, what wrong did I do, see code...
ousodanos@ousodanos-UX360CA:/usr/bin$ chmod +rx bd
chmod: changing permissions of 'bd': Operation not permitted
ousodanos@ousodanos-UX360CA:/usr/bin$ sudo chmod +rx bd
ousodanos@ousodanos-UX360CA:/usr/bin$ sudo echo 'alias bd=". bd -s"' >> ~/.bashrc
ousodanos@ousodanos-UX360CA:/usr/bin$ source ~/.bashrc
ousodanos@ousodanos-UX360CA:/usr/bin$ cd
ousodanos@ousodanos-UX360CA:~$ sudo wget -o /etc/bash_completion.d/bd https://raw.github.com/vigneshwaranr/bd/master/bash_completion.d/bd
ousodanos@ousodanos-UX360CA:~$ sudo source /etc/bash_completion.d/bd
sudo: source: command not found
ousodanos@ousodanos-UX360CA:~$ source /etc/bash_completion.d/bd
--2017-04-15: command not found
bash: /etc/bash_completion.d/bd: line 2: syntax error near unexpected token (' bash: /etc/bash_completion.d/bd: line 2: Resolving raw.github.com (raw.github.com)... <IP_ADDRESS>'
ousodanos@ousodanos-UX360CA:~$ cd Downloads/LibreOffice_<IP_ADDRESS>_Linux_x86-64_deb/DEBS/
ousodanos@ousodanos-UX360CA:~/Downloads/LibreOffice_<IP_ADDRESS>_Linux_x86-64_deb/DEBS$ bd Downloads
--2017-04-15: command not found
bash: /usr/bin/bd: line 2: syntax error near unexpected token (' bash: /usr/bin/bd: line 2: Resolving raw.github.com (raw.github.com)... <IP_ADDRESS>'
ousodanos@ousodanos-UX360CA:~/Downloads/LibreOffice_<IP_ADDRESS>_Linux_x86-64_deb/DEBS$
I'd really appreciate a quick response here, thanks

Jesus, are you sure you even know enough to be using a tool like this?
First off, ignore the author's instructions and copy the bd binary into /usr/local/bin.  Never touch /usr/* other than through a package manager.
Second, do you even understand what sudo does?  You should not be using it to write to your own config files, nor to source other bash scripts running as your own user.  source is a bash builtin command, not an executable, that's why you can't run it with sudo.
Third, you are using -o instead of -O (capitalized) as your argument to wget, which means you are saving wget's output to that file, and then trying to source it as a bash script.
I don't mean to be so harsh, and I hope you're learning something from all of this, but this is seriously pretty basic stuff.  bd is the kind of thing that you really should only have the need to use once you've already mastered all the basics.  Good luck.

@hackel No, you're not harsh, you're merely stating what he's doing wrong; it's obvious he doesn't know what he's doing, but we all were beginners.

Ok, let's fix what mistakes were made. You need to copypaste directly so you won't make a mistake, and remember that the letter case and spacing matters:

Delete the .bashrc created to filesystem root: sudo rm --verbose /.bashrc. Even if you made the mistake of having a space after / with that command, the system won't be touched since the filesystem root, represented by /, is considered a folder, which rm refuses to touch unless you provide either -r or --recursive (Bash should complain about such mistake, but there's no guaranteeing it will).
Delete the Bash completion file that's now a log file: sudo rm --verbose /etc/bash_completion.d/bd
Remove the alias from the currently open terminal (either close all terminal windows or run this in all of them to rid of it): unalias bd

Normally it's not advisable, but given this script doesn't look like it'd get updated anytime soon, and since it's not packaged for Ubuntu yet, but Ubuntu is based on Debian Unstable anyway, downloading the DEB from Debian should be the best way. Debian packages are compatible with Ubuntu, but in some cases not the other way around.

Download the DEB to your downloads folder (it's a single line of code):

wget -O ~/Downloads/bd_1.02-1_all.deb http://ftp.de.debian.org/debian/pool/main/b/bd/bd_1.02-1_all.deb

(it's one of the links from https://packages.debian.org/sid/all/bd/download, the first European one I pick)
2) Install the DEB: sudo dpkg -i ~/Downloads/bd_1.02-1_all.deb
Now you can uninstall and update bd easily.
On another note
You have a LibreOffice download in there, which is also doing things the wrong way. On a Linux distro like Ubuntu where most stuff is installed from the repos, you're supposed to use that before downloading from a site. Delete the LibreOffice download and use a program to install what you need. Ubuntu's own software center is easiest for you, if not the highest in functionality. Otherwise use Synaptic.

@hackel Thanks for the quick response. Appreciate it!
I've updated the Readme changing the installation directory to /usr/local/bin.

@rautamiekka Thanks for helping out @ousodaniel
BTW a package for Ubuntu is available in their "Universe" repository
https://launchpad.net/ubuntu/+source/bd
I am on a Mac. I am yet to install an Ubuntu VM to test the package and update the Readme with the installation instructions. Might do it before next weekend :)

@vigneshwaranr Seems I failed to check packages.ubuntu.com for it when working on multiple things at once, as bd is found there. Updated.


SQLitePCL.Batteries_V2.Init() not working in UWP with SQLitePCLRaw.bundle_sqlcipher v1.1.5

Hi. Is SQLCipher supported in UWP?
I tried this code
SQLitePCL.Batteries_V2.Init();
sqlite3 db;
var rc = raw.sqlite3_open(":memory:", out db);
Debug.Assert(rc == raw.SQLITE_OK);
in a blank UWP app created in VS 2017 after installing SQLitePCLRaw.bundle_sqlcipher v1.1.5 via NuGet, but I got the exception below:
System.Exception occurred
HResult=0x80131500
Message=This is the 'bait'.  You probably need to add one of the SQLitePCLRaw.bundle_* nuget packages to your platform project.
Source=Cannot evaluate the exception source
StackTrace:
at SQLitePCL.Batteries_V2.Init()
...
If I replace SQLitePCLRaw.bundle_sqlcipher with SQLitePCLRaw.bundle_green v1.1.5, the code works as expected after replacing SQLitePCL.Batteries_V2.Init(); with SQLitePCL.Batteries.Init();.

Thanks for your reply. From your experience, would it be relatively straightforward to add support for UWP? I'm considering doing the work and creating a pull request at some stage.

I am currently using the sqlcipher builds managed by couchbase.  There is a sub module link to their repo in the tree.  They plan to support UWP, so sometime after they do, I will.

Many thanks. That'll be great! 👍 I'll wait patiently.

FWIW, version 1.1.8 (just pushed to nuget) has a new package called bundle_zetetic which works with the official SQLCipher builds from Zetetic, and they do support UWP.

Version 1.1.9 includes sqlcipher builds for UWP.


Add new test fixtures to CudaBinaries project resolves #1394

Fixed, thanks @NiklasGustafsson I'll remember this in the future. I assume no need to update release notes on this one.

Merged and released.

@NiklasGustafsson does that mean I should see it on nuget.org? I don't see it, and readme still shows v0.103.0. Also the latest build looks like it did complete the native build but skipped cuda packaging, and didn't push packages: https://dotnet.visualstudio.com/TorchSharp/_build/results?buildId=110856&view=results

Once it's indexed by NuGet.

Ok that's the patience thing I'm working on. Got it. I'll check tomorrow!

Ok that's the patience thing I'm working on. Got it. I'll check tomorrow!

Not that long. Give it another 20-30 minutes.

Aha! Certificate problem holding it up...

Available on NuGet now.

@NiklasGustafsson

Am now using the new scope changes with packed sequences. Working great!
I appreciate your quick responses and good code review feedback. Being the PR guy can be thankless sometimes, so THANK YOU!

It's the opposite of thankless when community members send PRs my way... :-)


training code for VQVAE

thanks for the excellent work, we are looking forward to the training code for the VAE model, thanks！

+1

+1

please refer to this https://github.com/FoundationVision/vaex
we will supplement the detailed tutorial later