fix: check response content-length other than request content-length

What are you trying to achieve with this change?

What are you trying to achieve with this change?

Just like the logic written in comments: Check that the server doesn't send more data in DATA frames than indicated by the Content-Length header (if set).

A simple example to reproduce the bug:
package main

import (
	"github.com/quic-go/quic-go/http3"
	"io"
	"net/http"
	"strings"
)

func main() {
	client := &http.Client{
		Transport: &http3.RoundTripper{},
	}
	req, err := http.NewRequest(http.MethodPost, "https://www.cloudflare.com/", strings.NewReader("test"))
	if err != nil {
		panic(err)
	}
	req.Header.Set("Content-Length", "4")
	res, err := client.Do(req)
	if err != nil {
		panic(err)
	}
	_, err = io.ReadAll(res.Body)
	if err != nil {
		panic(err)
	}
}

Run the code:
$ go run .
panic: peer sent too much data

goroutine 1 [running]:
main.main()
        /Users/roc/dev/test/quic-go/main.go:27 +0x2d4
exit status 2

Current logic: Check if response body content-length if greater than Content-Length header in request.
Should change to: Check if response body content-length if greater than Content-Length header in response.

Oh, I see! I missed that you're using res and not req. I don't think the textproto conversion is needed though, headers are already normalized after we retrieve them from qpack.

Oh, I see! I missed that you're using res and not req. I don't think the textproto conversion is needed though, headers are already normalized after we retrieve them from qpack.

OK, textproto is removed.


Last publish release action is not included in the manifest

Hey @jjw24 found a bug in the notion search plugin. It was difficult to test the plugin in dev because it can't call the main FlowLauncher class. I ended up doing testing by editing the code in the app's plugin folder.
I pushed the fix and v0.1.5 is the last release. But the manifest still says 0.1.4. Is there anything else I need to do?
https://github.com/danielduckworth/Flow.Launcher.Plugin.NotionSearch
Thanks,
Daniel.

That's because the plugin manifest  repository automatically updates on a schedule of every 3 hours.
I can see it has updated now though.


[Feature] Add whatsapp linking

Description
We need command /link which will link Whatsapp to the Telegram account.
User will be asked with prompt:

Select preferred way to log in:

QR Code
Pairing Code

[!NOTE]
Selection could be replaced with buttons.

And this will "create" session for telegram account. Then user could easily select account when "injecting" into forum.

This would require database to store:

Telegram ID
Randomly generated UUID related to Whatsapp session
Creation timestamp
Modification timestamp


Trev8939/line of sight geo element

Thanks for the other review. Can I trouble you for one more @priy1289?
Thanks!

Good idea to make that int variable. Thanks for the review @priy1289!


Distribution over multiple GPUs

Hi,
I ran alphafold on a 2k sequence using 2x V100 (32 GB) GPU. Like for shorter sequences, 29 GB are allocated on the first GPU and 300 MB on the other from the start. After hhblits I got an out-of-memory error from JAX that 26 GB couldn't be allocated. Does everything need to be loaded to a single GPU or should it in theory be possible to use the next GPU if the first one runs out of memory? Why are 300 MB allocated to the second GPU from the start?

Same error here, except an error is raised trying to allocate 8GB on a ~1600aa sequence, when working with a 4x A100 setup (40GB).
I0719 14:52:43.337913<PHONE_NUMBER>84352 run_docker.py:180] 2021-07-19 12:52:43.337083: E external/org_tensorflow/tensorflow/stream_executor/cuda/cuda_driver.cc:794]
failed to alloc<PHONE_NUMBER> bytes on host: CUDA_ERROR_OPERATING_SYSTEM: OS call failed or operation not supported on this OS

This is inside the docker container.

Hi, we don't parallelize the model itself in JAX over multiple GPUs, but we do enable unified memory which should (CUDA might be smarter) at least be able to use the host RAM as well.
Can you see any logging statements saying how much device memory JAX has access to?
How much RAM does your GPU host have?

Hi,
I ran it again with a 2.7k sequence on a 48 GB GPU and asked for 80GB RAM on the server.
I get the following error:
e_aatype': (4, 4, 2799), 'template_all_atom_masks': (4, 4, 2799, 37), 'template_all_atom_positions': (4, 4, 2799, 37, 3), 'template_sum_probs': (4, 4, 1), 'is_distil
lation': (4,), 'seq_mask': (4, 2799), 'msa_mask': (4, 508, 2799), 'msa_row_mask': (4, 508), 'random_crop_to_size_seed': (4, 2), 'template_mask': (4, 4), 'template_ps
eudo_beta': (4, 4, 2799, 3), 'template_pseudo_beta_mask': (4, 4, 2799), 'atom14_atom_exists': (4, 2799, 14), 'residx_atom14_to_atom37': (4, 2799, 14), 'residx_atom37
to_atom14': (4, 2799, 37), 'atom37_atom_exists': (4, 2799, 37), 'extra_msa': (4, 5120, 2799), 'extra_msa_mask': (4, 5120, 2799), 'extra_msa_row_mask': (4, 5120), 'b
ert_mask': (4, 508, 2799), 'true_msa': (4, 508, 2799), 'extra_has_deletion': (4, 5120, 2799), 'extra_deletion_value': (4, 5120, 2799), 'msa_feat': (4, 508, 2799, 49)
, 'target_feat': (4, 2799, 22)}
2021-07-19 16:40:55.656793: W external/org_tensorflow/tensorflow/core/common_runtime/bfc_allocator.cc:457] Allocator (GPU_0_bfc) ran out of memory trying to allocate
45.86GiB (rounded to<PHONE_NUMBER>2)requested by op
2021-07-19 16:40:55.661892: W external/org_tensorflow/tensorflow/core/common_runtime/bfc_allocator.cc:468] *******************______________________________________

2021-07-19 16:40:55.662105: E external/org_tensorflow/tensorflow/compiler/xla/pjrt/pjrt_stream_executor_client.cc:1981] Execution of replica 0 failed: Resource exhau
sted: Out of memory while trying to allocate<PHONE_NUMBER>0 bytes.
[...]
alphafold-conda/lib/python3.6/site-packages/jax/interpreters/xla.py", line 893, in _execute_compiled
out_bufs = compiled.execute(input_bufs)
jax._src.traceback_util.UnfilteredStackTrace: RuntimeError: Resource exhausted: Out of memory while trying to allocate<PHONE_NUMBER>0 bytes.
The stack trace below excludes JAX-internal frames.
The preceding is the original exception that occurred, unmodified.
[...]
.../alphafold-conda/lib/python3.6/site-packages/jax/interpreters/xla.py", line 893, in _execute_compiled
out_bufs = compiled.execute(input_bufs)
RuntimeError: Resource exhausted: Out of memory while trying to allocate<PHONE_NUMBER>0 bytes.
It seems the maximum RAM usage for this job was 20GB (according to slurm).
I am running alphafold from a conda installation (not the container) and I am using jaxlib==0.1.67+cuda111

You need to be using jaxlib 0.1.69 (as set in the Dockerfile) for CUDA unified memory to work, this was recently added. Can you try that?

Thanks a lot for the hints, installing jaxlib 0.1.69 and setting the environment variables TF_FORCE_UNIFIED_MEMORY=1, XLA_PYTHON_CLIENT_MEM_FRACTION=4.0 helped to get it to the minimization step. Here it's running now many minimization iterations for the same prediction/model with the following error messages between iterations:
2021-07-20 11:18:29.326371: E external/org_tensorflow/tensorflow/stream_executor/cuda/cuda_driver.cc:794] failed to alloc<PHONE_NUMBER>8 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2021-07-20 11:18:29.326403: W external/org_tensorflow/tensorflow/core/common_runtime/device/device_host_allocator.h:46] could not allocate pinned host memory of size:<PHONE_NUMBER>8
2021-07-20 11:18:29.326444: W external/org_tensorflow/tensorflow/core/common_runtime/bfc_allocator.cc:457] Allocator (xla_gpu_host_bfc) ran out of memory trying to allocate 5.72GiB (rounded to<PHONE_NUMBER>)requested by op
However the iterations seem to complete since it prints Efinal. For another prediction I saw that it also ran many iterations for the first model (without the above mentioned error/warning messages).

@RodenLuo to run different jobs on different GPUs you'll need to use the --gpu_devices devices flag on run_docker, or equivalently set the NVIDIA_VISIBLE_DEVICES environment variable if you're not using docker. From your nvidia-smi, you'd want to set them to "0", "1", "2", and "3" to run four jobs in parallel on different cards.

Thanks, this is great!
I read the doc at home page again, it says "By default, Alphafold will attempt to use all visible GPU devices." It seems not the case for me? So if I want a job to use two GPUs, I would use --gpu_devices=2,3, and would expect it will use two?

So, I tested it and found that it can at most use one GPU per job. Is this only happening to Ampere cards? Thanks.
$ python3 docker/run_docker.py --fasta_paths=predictions/T4/37_trimer.fasta --max_template_date=2021-08-01 --pr
eset=casp14 --gpu_devices=2,3

$ nvidia-smi
Thu Aug  5 22:20:57 2021
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.57.02    Driver Version: 470.57.02    CUDA Version: 11.4     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA RTX A6000    Off  | 00000000:18:00.0 Off |                  Off |
| 55%   79C    P2   202W / 300W |  48682MiB / 48685MiB |    100%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   1  NVIDIA RTX A6000    Off  | 00000000:3B:00.0 Off |                  Off |
| 30%   38C    P8     7W / 300W |   2629MiB / 48685MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   2  NVIDIA RTX A6000    Off  | 00000000:86:00.0 Off |                  Off |
| 59%   83C    P2   298W / 300W |  48681MiB / 48685MiB |    100%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   3  NVIDIA RTX A6000    Off  | 00000000:AF:00.0 Off |                  Off |
| 30%   28C    P8    17W / 300W |   1011MiB / 48685MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A     31505      C   python                           1095MiB |
|    1   N/A  N/A     27949      C   python                            259MiB |
|    1   N/A  N/A     31505      C   python                            447MiB |
|    2   N/A  N/A      9729      C   python                           1093MiB |
|    2   N/A  N/A     31505      C   python                            447MiB |
|    3   N/A  N/A      9729      C   python                            447MiB |
|    3   N/A  N/A     31505      C   python                            447MiB |
+-----------------------------------------------------------------------------+

@RodenLuo to run different jobs on different GPUs you'll need to use the --gpu_devices devices flag on run_docker, or equivalently set the NVIDIA_VISIBLE_DEVICES environment variable if you're not using docker. From your nvidia-smi, you'd want to set it to "0", "1", "2", and "3" for each job to run four jobs in parallel on different cards.

@tfgg NVIDIA_VISIBLE_DEVICES environment variable need to  written in run_alphafold.py , if I use no-docker for multiple gpus?

@zhoujingyu13687306871 Yes, you need to set that variable yourself if not using Docker.
Closing this issue for now, feel free to reopen if this is still a problem and using --gpu_devices doesn’t solve it.


Store settings in database instead settings file

Hi there crew,
i'm wondering if it is possible to store the settings inside a model of the database instead of settings.py, I think we cannot because the "link" to the database is done after settings loads, anybody has tried that?
Thank you!

Hi!
Have you checked out this awesome project by @sergei-maertens? https://github.com/isprojects/django-auth-adfs-db

No, i didnt!
Onn my way :P

Closing for inactivity. The project above also solves DB settings, so we will not implement it in this repo.


Replace unicode emoji characters, allowing them to be customized by renderer rules

A very straightforward implementation of #43.  "It just works!"
I'm sure it's possible to provide an option to control this behavior, but it doesn't seem necessary.

I added some tests to cover this new behavior, and fixed the linter errors in my previous commit.
I'm happy to squash into a single commit, if you'd prefer.  Just let me know.


fix: deadlock when sending ping

See https://github.com/libp2p/rust-libp2p/issues/5410 and https://github.com/libp2p/rust-yamux/pull/112
The problem might be that we try to send a ping before reading pending frames. If the receive window is exhausted, no progress can be made.

CC @mxinden

The problem might be that we try to send a ping before reading pending frames. If the receive window is exhausted, no progress can be made.

Can you expand how that would lead to a dead-lock? Once the local node has sent the ping, it would then proceed to reading pending frames. Also who's receive window?

The problem might be that we try to send a ping before reading pending frames. If the receive window is exhausted, no progress can be made.

Can you expand how that would lead to a dead-lock? Once the local node has sent the ping, it would then proceed to reading pending frames. Also who's receive window?

I don't know much about the internals, but my theory is that when the remote receive window has been exhausted, we try to send a ping before reading the frame which increases our credit again. If both peers do this at the same time, the deadlock happens as shown here https://github.com/libp2p/rust-yamux/pull/193#discussion_r1634544258. I couldn't reproduce the deadlock anymore after reading the frame before sending the ping. I apologize, but this PR is mostly empirical, I don't have much knowledge about the system yet to make a better analysis. I hope it can be somehow helpful.

any update on this?
CC @diegomrsantos

I am not sure I fully understand the deadlock. It is at the TCP flow-control level?

We were already privileging clearing pending_read_frames over pending_write_frames, this change makes it so that sending a Ping has a lower priority than the those.

Prioritizing Ping frames is an optimization, leading to better RTT measurements only.
The optimization should not be prioritized over correct behavior. In other words, if this can indeed lead to a dead-lock, then yes, your change to the order makes sense.

Sadly I don't have a reliable reproducible case for this, I tried or quite a while.
I just found that this would semi-reliably happen when using yamux 1.3 - so it's some kind of edge case.
I'll try again maybe I'll find a reproducible case, tho I am not filled with confidence 😅

@dariusc93 was able to found a reproducible example that I pushed to https://github.com/jxs/rust-libp2p/tree/yamux-deadlock,
to reproduce it, one has to checkout that branch, yamux-deadlock and run the stream example, both the listener instance and the connecting instance.
Contrary to the example on master the one on yamux-deadlock eventually stops echoing.

@dariusc93 was able to found a reproducible example that I pushed to https://github.com/jxs/rust-libp2p/tree/yamux-deadlock, to reproduce it, one has to checkout that branch, yamux-deadlock and run the stream example, both the listener instance and the connecting instance. Contrary to the example on master the one on yamux-deadlock eventually stops echoing.

I ran it on macOS and I don't see any echo. Could you confirm this behavior is caused by the deadlock? I'm not entirely sure I'm testing it correctly, but I still can't see any echo after applying the fix in this PR.

The potential solution proposed here doesn't fix the issue, as the cause doesn't seem related to the ping.
The code here https://github.com/jxs/rust-libp2p/blob/yamux-deadlock/examples/stream/src/main.rs successfully reproduces the issue. It happens when the client streams around 500 KB of data to a server which echoes it back to the client.
After a deeper investigation, @jxs and I found the cause seems to be related to flow control, specifically these lines https://github.com/diegomrsantos/rust-yamux/blob/master/yamux/src/connection/stream/flow_control.rs#L52:L54.
The server decides no window update needs to be sent, as the condition is satisfied, but right after neither the client nor the server has more credits left.
It's still unclear what that's the case, but it should probably be addressed in a new PR. Thanks to everyone who helped to clarify the issue.


Update profile

@Jcardif - there's a requirement that the images are equal dimensions (height and width) - could you resize the image to be a square?


save/restore window maximize state

Windows can be maximized or minimized with maximize

This does not seem possible to track the state, but it is possible to restore to maximized state


Issue with k8s.io/docs/concepts/architecture/nodes/

This is a...

[ ] Feature Request
[x] Bug Report

Problem:
Ambiguous wording:

MemoryPressure  True if node has no memory pressure, otherwise False

Could be understood as if node as a "no-memory" pressure (which is what I understand from context) of if node does not have a memory pressure (which is what I understand without the context)
So it is not clear if this means the memory is low or not...
Proposed Solution:
Pick up the appropriate wording (I don't know for sure what is true).
Same for disk BTW
Page to Update:
https://kubernetes.io/docs/concepts/architecture/nodes/

Looking for small fixes as a better entry point to the community -- I'll take this one


Random string generation extensions

This Pull Request adds extensions which add the capability of generating random Strings into the library.
Random Strings can be generated using one of the following methods:
// generates a random string containing the specified characters only ranging from 16 to 32 in length.
let some: String = .random(withCharactersInString: "abc", minimumLength: 16, maximumLength: 32)

// generates a random string containing alphanumeric characters ranging in length from 16 to 32.
let some: String = .random(minimumLength: 16, maximumLength: 32)

// generates a random 16 character long string containing alphanumeric characters
let some: String = .random(ofLength: 16)

// generates a random 16 character long string containing the specified characters
let some: String = .random(withCharactersInString: "abc", ofLength: 16)

This PR also includes Unit Tests for all the new methods it introduces and updates to the README.md file documenting the new methods.

@thellimist I have now made the requested amendments to this pull request, with one comment regarding the unit test assertion you suggested. Please review accordingly.
Thanks :)

@jaderfeijo it looks good, thanks!
I've added another comment but for some reason, github shows my comment as outdated..

@thellimist yeah, I just saw that, I'm amending it now.

@thellimist tabs issue should be sorted now

Thanks 😊


Check description of CommandBar please

Issue type
enhancement
Which version of the app?
WinUI 3 Gallery
Description
App says

[...] When you need an app bar that contains only AppBarButton,AppBarToggleButton , and AppBarSeparator controls, use a CommandBar. If you need more complex content, such as images, progress bars, or text blocks, use an AppBar control."

MS Learn says

[!IMPORTANT]
You should use the AppBar only when you are upgrading a Universal Windows 8 app that uses the AppBar, and need to minimize changes. For new apps in Windows 10, we recommend using the CommandBar control instead.

https://learn.microsoft.com/uwp/api/windows.ui.xaml.controls.appbar?view=winrt-22621

Also, what's with "primary on the right and secondary on the left" ?

@jwmsft Are you able to weigh in?

@Jay-o-Way, thanks for letting us know about this issue.
It looks like CommandBar still has the description from the original Windows 8 version of the control, and it wasn't updated for the design and behavior changes that happened in Windows 10. (If you're wondering how the app ever got the Windows 8 descriptions in the first place, the WinUI Gallery is an evolution of a sample that was originally written for the Windows 8 docs.)
@karkarl, the description should be updated to reflect the current documentation.


t2m checkpoint model size error

I've finished setup with py39 and torch 1.12.1 and ran
python demo.py --cfg ./configs/config_mld_humanml3d.yaml --cfg_assets ./configs/assets.yaml --example ./demo/example.txt

But there is an error in loading the pre-trained weight
Length-50: a man kicks with something or someone with his left leg.
Length-100: A person is skipping rope.
Length-100: a person walks backward slowly.
Pointer Pointing at 0
`text_config_dict` is provided which will be used to initialize `CLIPTextConfig`. The value `text_config["id2label"]` will be overriden.
/home/hangyu5/anaconda3/envs/t2m_mld/lib/python3.9/site-packages/torch/cuda/__init__.py:146: UserWarning:
NVIDIA GeForce RTX 3090 with CUDA capability sm_86 is not compatible with the current PyTorch installation.
The current PyTorch install supports CUDA capabilities sm_37 sm_50 sm_60 sm_70.
If you want to use the NVIDIA GeForce RTX 3090 GPU with PyTorch, please check the instructions at https://pytorch.org/get-started/locally/

  warnings.warn(incompatible_device_warn.format(device_name, capability, " ".join(arch_list), device_name))
Traceback (most recent call last):
  File "/home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/AIGC/Text2Motion/motion-latent-diffusion/demo.py", line 337, in <module>
    main()
  File "/home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/AIGC/Text2Motion/motion-latent-diffusion/demo.py", line 105, in main
    model = get_model(cfg, dataset)
  File "/home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/AIGC/Text2Motion/motion-latent-diffusion/mld/models/get_model.py", line 7, in get_model
    return get_module(cfg, datamodule)
  File "/home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/AIGC/Text2Motion/motion-latent-diffusion/mld/models/get_model.py", line 17, in get_module
    return Model(cfg=cfg, datamodule=datamodule)
  File "/home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/AIGC/Text2Motion/motion-latent-diffusion/mld/models/modeltype/mld.py", line 86, in __init__
    self._get_t2m_evaluator(cfg)
  File "/home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/AIGC/Text2Motion/motion-latent-diffusion/mld/models/modeltype/mld.py", line 175, in _get_t2m_evaluator
    self.t2m_moveencoder.load_state_dict(
  File "/home/hangyu5/anaconda3/envs/t2m_mld/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1604, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for MovementConvEncoder:
	size mismatch for main.0.weight: copying a param with shape torch.Size([512, 259, 4]) from checkpoint, the shape in current model is torch.Size([512, 247, 4]).

Is it an known issue?

Hi @yhyu13
From the error log, it seems that the input size  of evaluators is not correct which indicates the feature size of the dataset is not correct.
https://github.com/ChenFengYe/motion-latent-diffusion/blob/c28a06435077800fde4d76ef93eb2a4016a5120c/mld/models/modeltype/mld.py#L157-L161
Maybe the problem is caused by the incorrect dataset. The  feature size of the HumanML3D data format is 263 and the feature size of the KIT-ML data format is 251. From the log it seems that your HumanML3d dataset feature size is 251 so maybe you are loading the KIT-ML data for the HumanML3D config.

First you can check whether the path of HumanML3D is correct. https://github.com/ChenFengYe/motion-latent-diffusion/blob/c28a06435077800fde4d76ef93eb2a4016a5120c/configs/assets.yaml#L13-L14
Then you can use NumPy to load the npy files in the HumanML3D folder manually and check whether the shape is [motion_length, 263] which 263 is the feature size of  HumanML3D data format.

You can refer to  HumanML3D for more details about the HumanML3D dataset.

i have the same error
@yhyu13  how can you solve this ? are you remember ?

Hi @yhyu13
From the error log, it seems that the input size of evaluators is not correct which indicates the feature size of the dataset is not correct.
https://github.com/ChenFengYe/motion-latent-diffusion/blob/c28a06435077800fde4d76ef93eb2a4016a5120c/mld/models/modeltype/mld.py#L157-L161
Maybe the problem is caused by the incorrect dataset. The feature size of the HumanML3D data format is 263 and the feature size of the KIT-ML data format is 251. From the log it seems that your HumanML3d dataset feature size is 251 so maybe you are loading the KIT-ML data for the HumanML3D config.

First you can check whether the path of HumanML3D is correct. https://github.com/ChenFengYe/motion-latent-diffusion/blob/c28a06435077800fde4d76ef93eb2a4016a5120c/configs/assets.yaml#L13-L14
Then you can use NumPy to load the npy files in the HumanML3D folder manually and check whether the shape is [motion_length, 263] which 263 is the feature size of  HumanML3D data format.

You can refer to HumanML3D for more details about the HumanML3D dataset.

Hello, I believe I have correctly prepared the HumanML3D dataset, as I'm able to obtain the expected shape. However, I'm still encountering persistent shape mismatching errors.

Hi @yhyu13
From the error log, it seems that the input size of evaluators is not correct which indicates the feature size of the dataset is not correct.
https://github.com/ChenFengYe/motion-latent-diffusion/blob/c28a06435077800fde4d76ef93eb2a4016a5120c/mld/models/modeltype/mld.py#L157-L161
Maybe the problem is caused by the incorrect dataset. The feature size of the HumanML3D data format is 263 and the feature size of the KIT-ML data format is 251. From the log it seems that your HumanML3d dataset feature size is 251 so maybe you are loading the KIT-ML data for the HumanML3D config.

First you can check whether the path of HumanML3D is correct. https://github.com/ChenFengYe/motion-latent-diffusion/blob/c28a06435077800fde4d76ef93eb2a4016a5120c/configs/assets.yaml#L13-L14
Then you can use NumPy to load the npy files in the HumanML3D folder manually and check whether the shape is [motion_length, 263] which 263 is the feature size of  HumanML3D data format.

You can refer to HumanML3D for more details about the HumanML3D dataset.

Hello, I believe I have correctly prepared the HumanML3D dataset, as I'm able to obtain the expected shape. However, I'm still encountering persistent shape mismatching errors.

This is caused by loading kit ckpt for humanml3d!


Unable to load the requested language file: language/english/pagination_lang.php

I tried to hosting my application using codeigniter framework, then I get this error message
Unable to load the requested language file: language/english/pagination_lang.php
what should i do ????

Post on our forums; this is a bug tracker.


271 - Implement error handling for API resource requesets for the search results page

Issue Number
NOTE: The changes in 10 files were simply the result of file/variable name renaming, import adjustments, and minor style changes. The env vars names were updated on Vercel  (instead of _PATH, _HOST suffix is now used).
Closing #271
Purpose/Implementation Notes
Implement error handling for the search results page and render an error view based on the status code when error occurred.
Please see the latest UI here.
Types of changes

Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist

[x] Lint and unit tests pass locally with my changes
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] I have added necessary documentation (if appropriate)
[ ] Any dependent changes have been merged and published in downstream modules

I think this looks good to me. My only concern is around the renaming of API_HOST and API_PATH.
Existing convention:
* `API_HOST`: The domain of our API, localhost, staging.refine.bio, refine.bio

* `API_VERSION`: The path partial of the API where API endpoints are nested.

* `API_PATH`: Root path of all API endpoints, this is the concatenated host and version.

I am sorry for the typo. I just fixed the value of  the apiPath variable in sitemap.config (9c1694ceade37bdbff2e691cca2f89c461fd22b0) and this was the reason for renaming env name (to API_HOST from API_PATH).

I think removing the protocol from API_HOST would be something to be considered but I do not understand the intent behind this change in this PR.

I cleaned up the declaration of STAGE_API_PATH and STAGE_API_VERSION variables from the docker-compose.env file (051589656bc0c69b881fc3da88b89b9370075f9b) since currently we are not using them to compose docker container.  But I kept the protocol (https) and was added in all env variables in Vercel.
Also I ran the deploy scripts locally since the project counts changed for sitemap etc for Vercel preview deploys (02d87b9f92cdf0ff15b48ff39c95d90cb94a149b). Thank you!


IRGen: repair the windows build of Foundation

Foundation imports free, importing it as dllimport dso_local which
is not permitted.  Clean up the IRGen to ensure that the two are not
emitted together.

Replace this paragraph with a description of your changes and rationale. Provide links to external references/discussions if appropriate.

Resolves SR-NNNN.

@swift-ci please smoke test

@swift-ci please test windows platform


add stats-map

@sindresorhus something like this? Or do you want to place it somewhere else?

I was thinking a Tip section with a link and a example.

Fixed it. But let's first discuss the issue I have currently in #5 before merge.


[WIP] Speculative decoding using a draft model

Speculative decoding
This PR adds speculative decoding to vLLM using the draft model approach. This was first explored in these papers:

Accelerating Large Language Model Decoding with Speculative Sampling
Fast Inference from Transformers via Speculative Decoding

As a simplified overview, this type of speculative decoding runs a smaller draft model to guess what the larger target model will emit. The target model then verifies the guesses, and emits them if they pass verification. This yields a latency reduction as the verification of many tokens can happen in a single forward pass of the target model.
Running this at Anyscale, I see a 30-50% latency reduction depending on the draft model, target model and dataset.
Usage
The usage looks like this:
llm = LLM(
    model="meta-llama/Llama-2-13b-chat-hf",
    speculative_model="JackFram/llama-68m", # The draft model. Must have same vocabulary as target model.
    tensor_parallel_size=4,
    speculative_model_uses_tp_1=True, # Whether the draft model should use TP=1 or same TP as target model.
    num_speculative_tokens=3, # The number of speculative tokens to score.
)

Feature list

Any vLLM model can be used as the draft model as long as it has the same vocabulary as the target model.
Rejection sampling to guarantee target model distribution.
The draft model can use tensor-parallel degree of 1, or the same tensor-parallel degree as the target model. This reduces draft model latency and contributes to the overall speedup.
The vLLM scheduler has been improved to support >1 token per step.
Tests for rejection sampling, various workers, scheduler, and e2e speculative decoding.

Future work (not implemented)

Top-k speculative decoding ("Tree attention") to increase draft hitrates
Optimized PagedAttention (multi-query attention) for scoring
Beam-sampling + speculative decoding

This PR is marked as draft as there is nontrivial work required to get this into a mergable state.
Guide for reviewers
The following are key files for understanding this speculative decoding implementation:

Rejection sampler. Use rejection sampling to approximate the target distribution using samples from the draft distribution.
Draft target worker. A worker which contains both the draft and target models. This orchestrates the drafting of speculative continuations, scoring, and accepting speculative tokens based on rejection sampling.
Multi-step worker. This runs the draft model several times, without invoking the vLLM scheduler each time.
Single-TP worker. This allows the DraftTargetWorker to run the draft model with TP=1, while the target model is TP=2, 4 or 8. This reduces latency of the draft model.
Key scheduler change. The scheduler and block manager now have the notion of "preallocated slots". This allows them to schedule KV block space sufficient for the worker to run several steps before the next scheduler invocation.
Key sequence changes. Sequences now track "processed tokens". A processed token is one that has been processed by the model such that the KV activations have been saved to cache.
Performance optimizations. Significant modifications have been made to the sampler so that the draft model can run several steps without CPU synchronization. This plus the SingleTPWorker reduced draft model latency from 5ms to 1.5ms.

How is the current progress? May I use your method to accelerate the llama2 70b on vLLM for now?

I'm very fortunate to witness such great work from you. How is the current progress? May I use your method to accelerate the llama2 70b on vLLM for now?

hi @Lvjinhong. The current PR requires some work to get into a working state on the public vllm repo. I will start on this later this week but given US holidays I expect to finish early january.

Very glad to see this work. I also have a wip version of speculative decoding and looking forward to using this feature.

Created a plan to break this PR into separate pieces. Pending review from vLLM original authors, I will start on it this week. https://docs.google.com/document/d/1rE4pr3IdspRw97XbImY4fS9IWYuJJ3HGtL7AdIKGrw8/edit

So from the little I understood I think there is a way to avoid having a draft model for your vocabulary by using directly the model's ngrams? It should probably be the subject of a separate PR but I thinks this is the way forward for everyone to easily get the benefits of speculative decoding.
What do you think @cadedaniel, do you think its as straigh forward as @simon-mo puts it?

I would imagine after this PR, ngram support should be very straightforward.

Yep. This PR builds the framework for scoring and verifying draft tokens, independent of whether they come from a draft model or something like Medusa, or lookup ngrams like prompt lookup or RAG lookup.

I think I understand a bit better know thank you, @cadedaniel , again for this incredible work!

@cadedaniel Hi! Thanks for your great work!
As I understand it, in each iteration, we are going to run the draft model several times and the target model once. Will this increase the first token latency greatly?

Yes, the time to first token is a few milliseconds higher for a draft model of size 68m. It can be optimized in future versions, e.g. with Medusa/EAGLE where draft tokens are generated without independent kv cache.

@cadedaniel Thanks for your reply! I also want to confirm my understanding regarding the decoding step's impact on first token latency:

Generating k draft tokens per iteration introduces a delay of k * d milliseconds, where d is the time to run the draft model once.
Additionally, the target model's execution time of t milliseconds further contributes to the overall latency.
This means the total decoding time is approximately k * d + t milliseconds. The cost of sampling is ignored.
Consequently, newly arriving prefill requests might experience a wait of up to k * d + t milliseconds before execution.

I'm wondering if it helps to break the decoding step into 2 sub-steps, drafting and verifying. In this way, newly arriving prefill requests wait up to k * d milliseconds. And, if the newly arriving prompts are short, they can be batched with the verifying step.
What do you think?

@UranusSeven could you ask your question in a discussion post? happy to answer there

Hi! Do you have some speed tps benchmarks?

Out of curiosity, does the proposal support separate deployment for the draft and target model? Asking because in production these two likely have different QPS and computing resource requirements.

This PR is about drinking fake wine?

Hi everyone, want to provide a quick update here:

Last few weeks I've prioritized optimizing Mixtral latency https://github.com/vllm-project/vllm/pull/2913
Now I will focus on getting this merged full-time. I am aiming to finish merges by February with @LiuXiaoxuanPKU's help reviewing.
After the correctness tests are merged, I will accept any optimizations that pass correctness tests. I will list out some major optimizations that people can take on (and already some tech discussions happening on MQA cc @ymwangg @robertgshaw2-neuralmagic ).

Thank you for your update and your efforts!
On Tue, Feb 20, 2024 at 10:49 AM Cade Daniel @.***>
wrote:

Hi everyone, want to provide a quick update here:

Last few weeks I've prioritized optimizing Mixtral latency #2913
https://github.com/vllm-project/vllm/pull/2913
Now I will focus on getting this merged full-time. I am aiming to
finish merges by February with @LiuXiaoxuanPKU
https://github.com/LiuXiaoxuanPKU's help reviewing.
After the correctness tests are merged, I will accept any
optimizations that pass correctness tests. I will list out some major
optimizations that people can take on (and already some tech discussions
happening on MQA cc @ymwangg https://github.com/ymwangg
@robertgshaw2-neuralmagic https://github.com/robertgshaw2-neuralmagic
).

—
Reply to this email directly, view it on GitHub
https://github.com/vllm-project/vllm/pull/2188#issuecomment-1954858949,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAACPY5KTS4UPZU6NU3Y53DYUTV5LAVCNFSM6AAAAABA2LBWYCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNJUHA2TQOJUHE
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

Thanks for the update! Looking forward to seeing how to incorporate our work once your PRs are out.

Thanks @cadedaniel for truly inspiring work! 🙏
Would Speculative decoding work with vLLM's continuous batching as well? Would that be the step Integrate speculative decoding with LLMEngine in our proposed design doc?
https://docs.google.com/document/d/1rE4pr3IdspRw97XbImY4fS9IWYuJJ3HGtL7AdIKGrw8/edit

Thanks @cadedaniel for truly inspiring work! 🙏
Would Speculative decoding work with vLLM's continuous batching as well? Would that be the step Integrate speculative decoding with LLMEngine in our proposed design doc?
https://docs.google.com/document/d/1rE4pr3IdspRw97XbImY4fS9IWYuJJ3HGtL7AdIKGrw8/edit

Yep! Once the e2e correctness tests pass you can use it with continuous batching.

Thanks for the great work you are doing. I wonder what is the timeline looking like. thanks

Hi @cadedaniel ,
thanks for your great contribution!
Since I'm also very keen on this feature, can I support you somehow?

Thanks! It would be helpful if you could test it out and add features or optimizations. Once https://github.com/vllm-project/vllm/pull/3951 is merged it will be correct but not yet fast; I'll post more about optimizing it this week.

will AssertionError: Speculative decoding not yet supported for RayGPU backend. error go away if this PR is merged? Ref: https://github.com/vllm-project/vllm/issues/4358

Hi guys,
First of all great to see your work on speculative decoding.
However, there is a new paper, Draft & Verify: Lossless Large Language Model Acceleration via
Self-Speculative Decoding (https://arxiv.org/abs/2309.08168), which talks about how we can create a draft model, by skipping layers from the original LLM.
There's another paper recently by Meta, that talks about it, LayerSkip: Enabling Early Exit Inference and
Self-Speculative Decoding - https://arxiv.org/pdf/2404.16710.
This method leads further optimization of memory as it is using the same model again. Would love to see this implemented in vLLM, as I am keen on this feature, can I support you somehow?

Hi @HimanshuJanbandhu, I would like also mention my recent work S3D (https://arxiv.org/abs/2405.20314). It is very similar to your mentioned Self-Speculative Decoding work which is simple to implement and easy to be integrated to existing stacks. But we have achieved better efficiency in general (compared to Self-Spec), our method combines layer-skipping with multiple next-token generation/unmasking. Although ours requires a bit training, but it should be straightforward just like training a Transformer decoder like BERT.

We welcome a self-speculative implementation!

Thanks! It would be helpful if you could test it out and add features or optimizations. Once #3951 is merged it will be correct but not yet fast; I'll post more about optimizing it this week.

Great, I'll start tomorrow (CET)

by the way: sorry for volunteering and never coming back to you. Unfortunately, I am and will be reaaally busy until beginning of August ✌🏻

Hello, Teacher, it is a great honor to witness your magnificent work. I have been studying you SpecDecode  work recently, and I have a question that I hope you can guide me on. Why does SpecDecodeWorker inherit from LoraNotSupportedWorkerBase? Why doesn't the current SpecDecode support Lora? @cadedaniel

Hi @skylee-01 . See https://github.com/vllm-project/vllm/issues/6912 for the work required to add LoRA + spec decode.

Thanks for the interest everyone! With https://github.com/vllm-project/vllm/issues/4630#issuecomment-2269618262, all the work in this PR has been merged.


chore(deps): upgrade dependencies

Upgrades project dependencies. See details in workflow run.

Automatically created by projen via the "upgrade-master" workflow

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 98.27%. Comparing base (5e70d5a) to head (1ce3118).

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #115   +/-   ##
=======================================
  Coverage   98.27%   98.27%
=======================================
  Files           1        1
  Lines          58       58
  Branches       10       10
=======================================
  Hits           57       57
  Misses          1        1

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


[Ray Cluster] A crash problem occurred when using modin to process large data sets on the ray cluster

Problem description

ray 1.4.1
modin 0.10.0

I used Kubernetes to build a Ray cluster with 10 nodes, and tried to use modin to process large-scale data sets on the cluster, but encountered an inexplicable crash problem. I don't understand, is this a modin problem or a ray problem?

The configuration of the cluster is as follows. I believe that this configuration can theoretically handle this data set.

Node Type
Number of Nodes
CPU
Memory
Plasma

Head
1
8
16 GB
9 GB

Worker
10
8
16 GB
9 GB

If part of the data set is used, it can run successfully. But using all the data sets, the above error occurred. No overflow was observed on the ray dashboard. It seems that the crash happened suddenly for no reason.

@youthmrg , thanks for the report! It seems to me there are no enough resources in your cluster nodes to process the data.
cc @modin-project/modin-ray , any comments?

@youthmrg , how much large your data is? And also, could you try installing Modin 0.10.1 or from sources (installing instruction) to see if the issue persists?

@YarShev The size of the dataset has 10 million rows and 41 columns, and the disk occupies about 3GB in the PostgreSQL database. This is a small dataset for testing, but this error has already occurred

I assume https://github.com/ray-project/ray/pull/17234 fixed this issue because it's quite old and no one else has commented here. @briancpark please comment here if you need more help.


Enforce resize bounds in rootston

wl_shell doesn't have resize bounds.
Test plan: try to resize a xdg shell surface or a xwayland surface, check it doesn't get smaller than the minimum size.
Fixes #346

Thanks!


Add navigation sections/heading to page

This page is very long, it would be easier to navigate if there were additional headings in the left or right side page navigation.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 3b3240d3-6a7d-fc4b-62fd-d3a5375d8a4a
Version Independent ID: 561212df-4dab-92b7-b634-57d53d313c6f
Content: PnP PowerShell overview
Content Source: sharepoint/docs-conceptual/sharepoint-pnp/sharepoint-pnp-cmdlets.md
Service: unspecified
GitHub Login: @Techwriter40
Microsoft Alias: kirks

@DeanGross Thank you for this feedback
We want to make sure your PnP issue is handled appropriately.
We have a separate repo for the PnP PowerShell issues. We want your issue to get the proper attention it deserves.
Please open your issue in this repo: https://github.com/SharePoint/PnP-PowerShell
I am going to close this issue so that we can track to correct one in the PnP PowerShell repo.


Icons in contact dropdown?

See experimental site for demo.

Icons look great. +1.

Merged into main.


Correct imports and typecast in Greeter deploy task

On a fresh clone of the repo, I couldn't get the greeter contract to deploy with the commands in the README. I needed to tweak two of the import paths and typecast one of the variables.
To reproduce:

generate a new project from the existing template
follow the steps in the README
run yarn deploy --greeting "Bonjour, le monde"

What happens:

you get a TypeScript import error "Cannot find module..."

What should happen:

the sample contract is deployed to Hardhat

Good catch!
Sorry, I accidentally closed this PR. I applied your changes via #77.


How can i take my ssr-project to nginx?

Now i have complete a project,but there is a problem,what should i do to take it to nginx

Reverse proxy to the rescue! Here is a simple example:
    location / {
        proxy_pass http://<IP_ADDRESS>:3000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }

It's very basic but may help you to move forward.

Correct me if I'm wrong, but any NodeJS tutorial should work for this. Are you deploying to a single VPS (how many cores?) or are you deploying to an autoscaling environment like AWS EC2 + CodeDeploy or Heroku or Zeit Now?


MSG_PEEK support

As far as I can tell MSG_PEEK is not supported by TCP sockets.
It would be great to add support for this feature.
This flag can be very useful when the size of the incoming packet is unknown, and it is encoded in the packet itself.
You can "peek" into the data, determine the size of the packer, allocate the needed space, and then perform the actual read.
A great example of this is MQTT.
In MQTT you have a header byte, 1 to 4 bytes that encode the size of the payload, and then the payload.
Now, these bytes have to be read one-by-one, allocate the space and then read the payload.
This is not optimal, because if the packet has not been received yet completely (or malloc fails), it is difficult to try again later.
(You have to keep somewhere the already read data to reuse later - complicated).
With MSG_PEEK you can read bytes 1-4 to determine the size, check that everything is correct, and then read in one go.
If everything goes wrong, just try again later. Everything is still in the buffer, untouched.

Look it's useful.

This was addressed in #9791.


Adding unit tests to test and reproduce several scenarios with readin…

Description
Thanks for contributing this Pull Request. Make sure that you submit this Pull Request against the develop branch of this repository, add a brief description, and tag the relevant issue(s) and PR(s) below.

Relevant Issues : https://github.com/intuit/karate/issues/1373#issuecomment-761610722
Relevant PRs : (optional)
Type of change :

[ ] New feature
[x] Bug fix for existing feature
[ ] Code quality improvement
[x] Addition or Improvement of tests
[ ] Addition or Improvement of documentation

@ptrthomas minimal scenario to replicate https://github.com/intuit/karate/issues/1373#issuecomment-761610722 is the js-read-4.feature :
js-read-4.feature
Feature:

Background:

Scenario:
    * def params = { 'foo': 'bar' }
    * call read('js-read-called-2.feature') params

js-read-called-2.feature
Feature:

Background:
    * call read('../utils-reuse-common.feature')

Scenario:
    * print 'arg: ' + __arg
    * match __arg == "#present"
    * match __arg == "#notnull"

Issue happens when the called feature calls and the called feature calls another feature in background adding it's variables.

@ptrthomas minimal scenario to replicate https://github.com/intuit/karate/issues/1373#issuecomment-761610722 is the js-read-4.feature :
js-read-4.feature
Feature:

Background:

Scenario:
    * def params = { 'foo': 'bar' }
    * call read('js-read-called-2.feature') params

js-read-called-2.feature
Feature:

Background:
    * call read('../utils-reuse-common.feature')

Scenario:
    * print 'arg: ' + __arg
    * match __arg == "#present"
    * match __arg == "#notnull"

Issue happens when the called feature calls and the called feature calls another feature in background adding it's variables.

After checking this in I realized I could try this:
js-read-called-2.feature
Feature:

Background:
    * def called = call read('../utils-reuse-common.feature')

Scenario:
    * print 'arg: ' + __arg
    * match __arg == "#present"
    * match __arg == "#notnull"

And it works. The difference is that on the the called feature we are assigning the result of another called feature to a variable. So it's probably the piece of code that "adds" the content/variables of the called feature in the background that is somehow overriding/removing the __arg .
I haven't tried other keywords yet like __row , __num for Outlines

After checking this in I realized I could try this:
js-read-called-2.feature
Feature:

Background:
    * def called = call read('../utils-reuse-common.feature')

Scenario:
    * print 'arg: ' + __arg
    * match __arg == "#present"
    * match __arg == "#notnull"

And it works. The difference is that on the the called feature we are assigning the result of another called feature to a variable. So it's probably the piece of code that "adds" the content/variables of the called feature in the background that is somehow overriding/removing the __arg .
I haven't tried other keywords yet like __row , __num for Outlines

@joelpramos I'm lost. any way to simplify - just focus on 1 case at a time instead of so many tests. also I see a build failure. will look at this later

@joelpramos I'm lost. any way to simplify - just focus on 1 case at a time instead of so many tests. also I see a build failure. will look at this later

@joelpramos no worries. I just may have realized what the problem could me. just making one commit before logging off - hope that solves !

@joelpramos no worries. I just may have realized what the problem could me. just making one commit before logging off - hope that solves !

I got it - was refactoring now so that the magicVariables stay only within the scope of the ScenarioRuntime (ie don't add to the vars Map) but are added to the JS Engine. Let me know if that's what you're working towards.

I got it - was refactoring now so that the magicVariables stay only within the scope of the ScenarioRuntime (ie don't add to the vars Map) but are added to the JS Engine. Let me know if that's what you're working towards.

@ptrthomas made the changes for the magicVariables being available only within the Scenario or Scenario Outline being executed - ie if a scenario is called that scenario won't have changes to __arg (for e.g.) even if in shared score.
When reviewing let me know if this would be an intended design that aligns with your thoughts. Also check out
ScenarioRuntimeTest.testCallFromJs()
ScenarioRuntimeTest.testCallKarateFeature()
Especially the later. That was an unintended consequence in which those variables are now not getting returned when assigning the result of a called feature (diff in the PR should be obvious). I can give it some thought into how to work around it / make it work as before if you think it's important (maybe adding in the scenario result when the execution of the scenario is over could be an option but my concern is it's complex as we should "pollute" the variables of the caller).

@ptrthomas made the changes for the magicVariables being available only within the Scenario or Scenario Outline being executed - ie if a scenario is called that scenario won't have changes to __arg (for e.g.) even if in shared score.
When reviewing let me know if this would be an intended design that aligns with your thoughts. Also check out
ScenarioRuntimeTest.testCallFromJs()
ScenarioRuntimeTest.testCallKarateFeature()
Especially the later. That was an unintended consequence in which those variables are now not getting returned when assigning the result of a called feature (diff in the PR should be obvious). I can give it some thought into how to work around it / make it work as before if you think it's important (maybe adding in the scenario result when the execution of the scenario is over could be an option but my concern is it's complex as we should "pollute" the variables of the caller).

@joelpramos ok so I made one change which puts the background in the engine variables. which hopefully solves some of the problems you outlined - but I'm finding it hard to understand some of it. btw I don't handle large diffs well. and this is one of the most complex areas of karate. some thoughts below

I've never encouraged calling other features. in my mind it should only be used one-level and the called feature should IMO never do any outline business especially dynamic
so if some of the variables get over-written, especially in "shared scope" scenarios I'm fine with it. leave it to user to handle
I do not want to add complexity to setVariables() which can be called during a scenario execution. any complexity should be in the constructor or "once only" init()
note that when you add callonce or callSingle() into the mix - it may open up a whole bunch of edge cases on top of these (background, dynamic, outline, call, shared scope, passed arguments)

so I propose
a) see if my fix solves 80% of the problems
b) please create ONE test to show the next most high-priority issue in your opinion and start a new PR if needed and we can work through one at a time

@joelpramos ok so I made one change which puts the background in the engine variables. which hopefully solves some of the problems you outlined - but I'm finding it hard to understand some of it. btw I don't handle large diffs well. and this is one of the most complex areas of karate. some thoughts below

I've never encouraged calling other features. in my mind it should only be used one-level and the called feature should IMO never do any outline business especially dynamic
so if some of the variables get over-written, especially in "shared scope" scenarios I'm fine with it. leave it to user to handle
I do not want to add complexity to setVariables() which can be called during a scenario execution. any complexity should be in the constructor or "once only" init()
note that when you add callonce or callSingle() into the mix - it may open up a whole bunch of edge cases on top of these (background, dynamic, outline, call, shared scope, passed arguments)

so I propose
a) see if my fix solves 80% of the problems
b) please create ONE test to show the next most high-priority issue in your opinion and start a new PR if needed and we can work through one at a time

@joelpramos ok I think I have a solution for all the problems, explained / details in this commit: https://github.com/intuit/karate/commit/426e9c0e0695a9bb1219b971682c967dcfa235ca
so maybe all we need is some extra tests that you see fit !

@joelpramos ok I think I have a solution for all the problems, explained / details in this commit: https://github.com/intuit/karate/commit/426e9c0e0695a9bb1219b971682c967dcfa235ca
so maybe all we need is some extra tests that you see fit !

@ptrthomas Yeah those changes would work (although I haven't run the unit tests yet).
I'd suggest one more thing - see the changes I did at ScenarioRuntime:initMagicVariables().
In that method we are adding to the map the variables from the caller feature and the variables from the Examples table. With the Map stored on that variable we will iterate add set those variables as hidden variables (your code change in the Scenario Engine).
I haven't tried your PR yet with the tests but as a principle I'd suggest we initialize only those special variables in that method and do a setVariables() for both caller and examples table variables to the ScenarioRuntime vars property in the Engine. See here: ScenarioEngine.java:init() in my PR.
Proposed additional snippet in the init() method below:
        runtime.magicVariables.forEach((k, v) -> setHiddenVariable(k, v)); // your changes

        // proposed extra code
        if(this.runtime.scenario.isOutlineExample()) {
            Map<String, Object> exampleData = runtime.scenario.getExampleData();
            this.setVariables(exampleData);
        }
        if (this.runtime.caller.arg != null && this.runtime.caller.arg.isMap()) {
            this.setVariables(this.runtime.caller.arg.getValue());
        }

My gut feeling is that your PR works completely because there's a line here  that sets all the magicVariables from the caller and so it doesn't really make a difference but as a design principle I'd keep that function initMagicVariables() only with the responsibility of keeping those indeed magical variables.
The responsibility of setting variables as part of the shared context or not is already delegated by the caller in which the vars map is either passed as reference or a deep copy (which come to think of it passing those variables in the initMagicVariables() might actually give "write" access to called features in when context is not shared).

@ptrthomas Yeah those changes would work (although I haven't run the unit tests yet).
I'd suggest one more thing - see the changes I did at ScenarioRuntime:initMagicVariables().
In that method we are adding to the map the variables from the caller feature and the variables from the Examples table. With the Map stored on that variable we will iterate add set those variables as hidden variables (your code change in the Scenario Engine).
I haven't tried your PR yet with the tests but as a principle I'd suggest we initialize only those special variables in that method and do a setVariables() for both caller and examples table variables to the ScenarioRuntime vars property in the Engine. See here: ScenarioEngine.java:init() in my PR.
Proposed additional snippet in the init() method below:
        runtime.magicVariables.forEach((k, v) -> setHiddenVariable(k, v)); // your changes

        // proposed extra code
        if(this.runtime.scenario.isOutlineExample()) {
            Map<String, Object> exampleData = runtime.scenario.getExampleData();
            this.setVariables(exampleData);
        }
        if (this.runtime.caller.arg != null && this.runtime.caller.arg.isMap()) {
            this.setVariables(this.runtime.caller.arg.getValue());
        }

My gut feeling is that your PR works completely because there's a line here  that sets all the magicVariables from the caller and so it doesn't really make a difference but as a design principle I'd keep that function initMagicVariables() only with the responsibility of keeping those indeed magical variables.
The responsibility of setting variables as part of the shared context or not is already delegated by the caller in which the vars map is either passed as reference or a deep copy (which come to think of it passing those variables in the initMagicVariables() might actually give "write" access to called features in when context is not shared).

As per unit tests - I'm pleased with the ones on this PR, once we finish the work in the develop branch I'll close this one and open a new one with just the tests. I'll do a review to confirm whether I've covered all the scenarios with callonce as well (probably not as you can see there were a lot of different cases I could think of :) )

As per unit tests - I'm pleased with the ones on this PR, once we finish the work in the develop branch I'll close this one and open a new one with just the tests. I'll do a review to confirm whether I've covered all the scenarios with callonce as well (probably not as you can see there were a lot of different cases I could think of :) )

@joelpramos

might actually give "write" access to called features in when context is not shared

the "hidden" variable approach solves this problem nicely they will not be "mutated" by called features to any depth. which also may be the reason why your other concerns may not be valid. do have a closer look and see what you think, I felt it is reasonably elegant.
I think we will have cases where users make a call (just after landing from another call) and expect variables to be in scope (even though I hate this) hence the need for copying parent variables over. lets see if you can find any edge cases via your tests

@joelpramos

might actually give "write" access to called features in when context is not shared

the "hidden" variable approach solves this problem nicely they will not be "mutated" by called features to any depth. which also may be the reason why your other concerns may not be valid. do have a closer look and see what you think, I felt it is reasonably elegant.
I think we will have cases where users make a call (just after landing from another call) and expect variables to be in scope (even though I hate this) hence the need for copying parent variables over. lets see if you can find any edge cases via your tests

Closed as per discussion. See unit tests at this PR https://github.com/intuit/karate/pull/1438

Closed as per discussion. See unit tests at this PR https://github.com/intuit/karate/pull/1438


v0.8.0

Features

[x] Add pass, fail and error functions for use in tests (#74)
[x] Add is_substring_of and is_not_substring_of assertion methods (#64)
[x] Add is_positive, is_negative, is_greater_than and is_less_than assertion methods (#75)
[x] Add --verbose option to print full test output to stdout (#65)

Enhancements

[x] Remove color dependency (#73)

Bugfixes

[x] Fix time output at end of tests (#66)

Chores

[x] Tweak .travis.yml to improve CI build times (#68)
[x] Separate test and deploy steps into build stages
[x] Repository migrated from molovo/zunit to zunit-zsh/zunit

[x] Update Documentation (molovo/zunit-www#3)
[x] Bump Version

Fix #63, #69, #76

I've had no bug reports, and tests have all gone well for me, so I'm releasing v0.8.0.


feat: add group_by to fetch_page

feature is required by #2139

Can we please have one small test for GROUP BY.

added tests


Added NoUselessReturnFixer

replaces:
https://github.com/FriendsOfPHP/PHP-CS-Fixer/pull/1591

Diff. of Fixer behavior is minimal, biggest change is the clear and merge logic added to Tokens.
This change will great an option to start finishing some other PRs.

Please, fix Travis

3 comments, all the rest look OK for me

Needs squashing to 1 commits per author.

:+1: (after squashing)

(after squashing)

squashed to 1 commit by author, is this ok?

Please fix your commit log to sth more meaningful ;)

ping @SpacePossum

git checkout 538dd8d
git pick --no-commit d999763
git commit --amend -m "Added UselessReturnFixer"
git pick c84039ba4d4a20389099666dcc2a0315fa8dec9e
git push origin 1_12_clear_tokens -f

thanks for hints, I did a little detour but I think this should be OK-ish now

btw, just a thought, what about no_useless_return ? as in fact we are removing the statement ?

no_useless_return

:+1: will rename and do the squashing thing again later today or maybe tomorrow morning.
@ all let me know if there are other things to be fixed in this PR as well :)

?

github auto closes the PR if I reset the branch back to 0 commits, than force push. After that I do the picking and merging and have to re-open the PR by hand. Not sure how to prevent this :/

don't push before picking.

You need to regenerate readme

d'oh, no time for the squashing, will do later tonight

I am still wondering how you squash... a lot of time and PR closing/opening.

@SpacePossum , just copy-execute:
D:\GIT\fork\PHP-CS-Fixer (2.x-async_linting)
λ git co 9656a82af22011ada04629618c82e96d3e902f02
Note: checking out '9656a82af22011ada04629618c82e96d3e902f02'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b new_branch_name

HEAD is now at 9656a82... Added NoUselessReturnFixer

D:\GIT\fork\PHP-CS-Fixer ((detached from 9656a82))
λ git cherry-pick --no-commit cdfe52d18e87e1466054fd154f98f546270a3c6f

D:\GIT\fork\PHP-CS-Fixer ((detached from 9656a82))
λ git commit --amend --no-edit
[detached HEAD 9e4d4e3] Added NoUselessReturnFixer
 Author: Possum<EMAIL_ADDRESS> 19 files changed, 684 insertions(+), 20 deletions(-)
 create mode 100644 Symfony/CS/Fixer/Contrib/NoUselessReturnFixer.php
 create mode 100644 Symfony/CS/Tests/Fixer/Contrib/NoUselessReturnFixerTest.php
 create mode 100644 Symfony/CS/Tests/Fixtures/Integration/misc/empty_return,useless_return,whitespacy_lines,extra_empty_lines.test
 create mode 100644 Symfony/CS/Tests/Fixtures/Integration/priority/duplicate_semicolon,no_useless_return.test
 create mode 100644 Symfony/CS/Tests/Fixtures/Integration/priority/empty_return,no_useless_return.test
 create mode 100644 Symfony/CS/Tests/Fixtures/Integration/priority/no_useless_return,extra_empty_lines.test
 create mode 100644 Symfony/CS/Tests/Fixtures/Integration/priority/no_useless_return,return.test
 create mode 100644 Symfony/CS/Tests/Fixtures/Integration/priority/no_useless_return,whitespacy_lines.test

D:\GIT\fork\PHP-CS-Fixer ((detached from 9656a82))
λ git cherry-pick 349f754215b201ab505d20e2ed86f46dcbf7ca0d
[detached HEAD bb4bc6b] Tokens::clearTokenAndMergeSurroundingWhitespace - code grooming
 4 files changed, 36 insertions(+), 41 deletions(-)

D:\GIT\fork\PHP-CS-Fixer ((detached from 9656a82))
λ git branch -D 1_12_clear_tokens
error: branch '1_12_clear_tokens' not found.

D:\GIT\fork\PHP-CS-Fixer ((detached from 9656a82))
λ git checkout -b 1_12_clear_tokens
Switched to a new branch '1_12_clear_tokens'

D:\GIT\fork\PHP-CS-Fixer (1_12_clear_tokens)
λ git push origin 1_12_clear_tokens -f
Counting objects: 229, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (49/49), done.
Writing objects: 100% (51/51), 9.62 KiB | 0 bytes/s, done.
Total 51 (delta 41), reused 0 (delta 0)
To git@github.com:keradus/PHP-CS-Fixer.git
 * [new branch]      1_12_clear_tokens -> 1_12_clear_tokens

thanks so much @keradus !
all done, lets see what travis and co. think :)

Thank you @SpacePossum.

Thanks all for reviewing :)
Thanks @keradus for the assists with the git stuff and for merging, this opens up a lot of new options for new and better fixers :D

@SpacePossum Not sure of that, but should not be this fixer marked as risky since PHP 7.1 and the void/null return difference?

@Soullivaneuh thanks for the heads up.
At the time of writing the void type was only a thing talked about, but now we should indeed update the fixer and/or mark it risky.
More discussion in detail about what people think would be a good way;
https://github.com/FriendsOfPHP/PHP-CS-Fixer/issues/1638#issuecomment-205916520
https://github.com/FriendsOfPHP/PHP-CS-Fixer/issues/1916


docs: show simple example of migration from wapi-python

To show the most typical changes.

I know this is just an example. But I still did a code review like it is gonna be a production code. 🙂 Would be good to set a good example for clients.

Making a good example of how to write a prod script is beyond the scope of this example.


Private Key Path for Hopping

Hi liximomo,
Thanks for the great extension. But when I try to connect to a server that requires a single hopping like the following:
local -(password)-> hop -(pubkey)->host
How do I specify the location of private key? Also, should I specify it in the config of the hop or the host?
Thanks in advance.

Specify it in the hop.


Hi, Italian language support

Hi, I have translated it.yaml to italian in order to have italian language support.
all the best

@raffdoc your PR contains changes for 2 files: nl.yaml and it.yaml. Please remove the nl.yaml from your PR. Ideally, please sync your master with the official master, create a new branch (e.g. add-italian), check that the only difference in your branch is new it.yaml file, and create a new PR only for Italian language. Thanks.

Also, the language.yaml file would require updating, similar to how the Portuguese translation as added ( https://github.com/gcushen/hugo-academic/commit/6a2fd0075af19ba7507fbfc8070bc9467a1f3d63 ).

Closing as no fix has been submitted and Italian has since been added in #249.


Fix incorrect variable name and log message

What type of PR is this?
cleanup
What this PR does / Why we need it?
In #428 I mistakenly changed the variable name to proxyHealthCheckURL, but it was correct before. config.URL holds the backplane URL, not the proxy URL.
Which Jira/Github issue(s) does this PR fix?
OSD-23471
Special notes for your reviewer
Pre-checks (if applicable)

[x] Ran unit tests locally
[ ] Validated the changes in a cluster
[ ] Included documentation changes with PR

Codecov Report
Attention: Patch coverage is 66.66667% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 44.49%. Comparing base (ca7d234) to head (7a2015b).

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #429   +/-   ##
=======================================
  Coverage   44.49%   44.49%
=======================================
  Files          63       63
  Lines        5286     5286
=======================================
  Hits         2352     2352
  Misses       2617     2617
  Partials      317      317

Files
Coverage Δ

pkg/cli/config/config.go
80.73% <66.66%> (ø)

/lgtm


BLD: bump OpenBLAS to v0.3.20-571-g3dec11c6 [wheel build]

v0.3.20-571-g3dec11c6 should contain fixes for https://github.com/scipy/scipy/issues/16949

@mattip all the windows entries are failing with the OpenBLAS build thats available on anaconda.
In the build log is the error message:
    c:/rtools40/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/10.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot find -lopenblas
    collect2.exe: error: ld returned 1 exit status

which is unusual because meson said it found openblas. However, I then had a look at openblas.pc from the openblas-v0.3.20-571-g3dec11c6-win_amd64-gcc_10_3_0.zip:
libdir=/c/opt/64/lib
libsuffix=
includedir=/c/opt/64/include
openblas_config= USE_64BITINT= DYNAMIC_ARCH=1 DYNAMIC_OLDER= NO_CBLAS= NO_LAPACK= NO_LAPACKE= NO_AFFINITY=1 USE_OPENMP= SKYLAKEX MAX_THREADS=2
version=0.3.21.dev
extralib=-defaultlib:advapi32 -lgfortran -defaultlib:advapi32 -lgfortran
Name: openblas
Description: OpenBLAS is an optimized BLAS library based on GotoBLAS2 1.13 BSD version
Version: ${version}
URL: https://github.com/xianyi/OpenBLAS
Libs: -L${libdir} -lopenblas${libsuffix}
Libs.private: ${extralib}
Cflags: -I${includedir}

I think meson uses Libs in the link step. Since the libsuffix is empty I think the library it's looking for in the link step is -lopenblas. However, that library isn't present, only libopenblas_v0.3.20-571-g3dec11c6-gcc_10_3_0 is available. i.e. libsuffix needs to be fixed.
In comparison openblas.pc for v0.3.21 is Libs: -L${libdir} -lopenblas_v0.3.21-gcc_10_3_0, and that library IS present in lib.

The macos-meson entry is failing because conda-forge is used to supply openblas, and it packages the 0.3.21 release, not the more updated version here.

I got the windows wheels a bit further by using CMAKE_PREFIX_PATH (the cmake config files aren't broken like the pc files are). However, the windows wheel builds then fail with:
    c:/rtools40/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/10.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: scipy/special/_ellip_harm_2.cp38-win_amd64.pyd.p/meson-generated__ellip_harm_2.c.obj:_ellip_harm_2.c:(.text+0xcec): undefined reference to `dstevr_'

I wonder if there's a compiler mismatch between multibuild and the windows compiler?

I need to fix the windows meson job. The windows .pc file issue was fixed because @mattip made a new build.

The windows openblas.pc problem was fixed in openblas-libs, xref MacPython/openblas-libs#95 and MacPython/openblas-libs#96

Feel free to merge this as is and leave the other fix to a follow-up if you prefer that.

I'll fix here and merge if it works. Can we rationalise a few steps in windows.yml to just call python dev.py test, or do you prefer to keep the meson-configure, meson-install, build-path, post-install, prep-test, test as individual steps?

I think that's fine to clean up a bit by now. Keeping dev.py build and dev.py test as separate stages is nice though - makes it much easier to read the logs.

I tried repeatedly yesterday to do python dev.py build --win-cp-openblas and python dev.py test on Win, but couldn't get it to work on CI or remotely. This is why I've only got the build step here. After coming back to this today I can now get both steps to work remotely. However, I'm not going to change the existing commits after all the effort getting windows.yml to work. I may come back to this later.


[stable/concourse] Upgrade to 5.0.0

What this PR does / why we need it:
This PR is all about bringing the necessary changes for accomodating the 5.0 release of Concourse. Here you can find the release notes: https://concourse-ci.org/download.html#v500
From the commit message, here's a breakdown of the changes, sorted by three categories: breaking, deprecations, and enhancements.
BREAKING:

concourse.web.auth.mainTeam.allowAllUsers, which was previously used
to allow any logged user to be part of main, has now been removed;

concourse.worker.bindDebugPort is now
concourse.worker.debugBindPort;

concourse.worker.baggageclaim.bindDebugPort is now
concourse.worker.baggageclaim.debugBindPort;

concourse.worker.garden.* variables that would previously turn into
environment variables that got translated to gdn-specfic tunings are
not values that can be set through concourse.worker.garden anymore.
These can still be provided through environment variables though -
CONCOURSE_GARDEN_... works as before.

the new image is packaged in a slightly different way (see release notes or the docs repo directly: new binary distribution), meaning that the concourse binary does not live in the same place as before (now it's under /usr/local/concourse/bin/concourse) - see https://concourse-ci.org/install.html.

DEPRECATIONS:

concourse.worker.peerIp is not used anymore;
worker.fatalErrors are not used anymore in favor of worker
healthchecks through a specific port (worker-hc) in the worker
container.

ENHANCEMENTS:

reduced complexity in the liveness probe for the worker, removing the
need for custom scripts to verify its health;
bitbucket cloud is now supported through
secrets.bitbucket-cloud-client* and other concourse.web.*
configurations;
AWS region settings can now be configured at environment-var level
instead of arguments;
garden configuration can be specified through ini files that can be
passed through configmaps;
RBAC configuration for local users can be specified through
configmaps
ability to opt-out of the automatic removal of
concourse.worker.workDir contents
ability to specify whether landing or retireing should be
performed when terminating the worker pod (through the configuration
of worker.shutdownSignal);
ability to specify custom readinessProbe for the worker;
more consistent values.yaml examples and documentation.

By the way, README might need some updates as well - I'll review that soon 😁
Special notes for your reviewer:
To test it, make sure you properly set image and imageTag:
helm upgrade \
  --install \
  --set=image=concourse/concourse-rc \
  --set=imageTag=latest \
  dev .

That's needed because concourse/concourse:5.0.0 is not released yet, and we've not been pushing tags for concourse/concourse-rc (5.0 sits in latest at this moment - https://github.com/concourse/concourse/issues/3332)
Checklist
[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

[x] DCO signed
[x] Chart Version bumped
[ ] Variables are documented in the README.md  (not sure yet)

/hold

/retest

Not a draft anymore as 5.0.0 has been published 😁  See release notes here: https://concourse-ci.org/download.html#v500
Any thoughts on it @william-tran?

/assign

The readme certainly needs an update, but other than that, LGTM!

Hey @william-tran,
I just updated the README, now values.yaml and README.md's configuration section are on-par in terms of non-concourse.* variables.
Using charts-values-check:
charts-values-check -file ./README.md -type readme | \
  awk '!/concourse\..*/{ print $1 }' | \
  sort -u > /tmp/actual
charts-values-check -file ./values.yaml -type values | \
  awk '!/concourse\..*/{ print $1 }' | \
  sort -u > /tmp/expected
git diff --no-index /tmp/actual /tmp/expected

Thanks!

/hold cancel

/lgtm


Permissions selecting Admin:All does not add permissions for all actions

Choosing Admin:All does not allow a user to add/delete events or files. These have to be chosen explicitly.

So being a part of the admin group as we normally set it up by default
doesn't do the job? Wouldn't that have prevented anybody from working with
events until now?
On Thu, Sep 8, 2016 at 11:15 AM, Kylee Acker<EMAIL_ADDRESS>wrote:

Choosing Admin:All does not allow a user to add/delete events or files.
These have to be chosen explicitly.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/punkave/apostrophe/issues/650, or mute the thread
https://github.com/notifications/unsubscribe-auth/AAB9fcMz3ssDOvHAB7GqVsZUMKmtKT2gks5qoCapgaJpZM4J4HIs
.

--
*THOMAS BOUTELL, *SUPPORT LEAD
P'UNK AVENUE |<PHONE_NUMBER>  |  punkave.com

(eg until recently when we got the group editor working.)
On Thu, Sep 8, 2016 at 1:43 PM, Tom Boutell<EMAIL_ADDRESS>wrote:

So being a part of the admin group as we normally set it up by default
doesn't do the job? Wouldn't that have prevented anybody from working with
events until now?
On Thu, Sep 8, 2016 at 11:15 AM, Kylee Acker<EMAIL_ADDRESS>wrote:

Choosing Admin:All does not allow a user to add/delete events or files.
These have to be chosen explicitly.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/punkave/apostrophe/issues/650, or mute the thread
https://github.com/notifications/unsubscribe-auth/AAB9fcMz3ssDOvHAB7GqVsZUMKmtKT2gks5qoCapgaJpZM4J4HIs
.

--
*THOMAS BOUTELL, *SUPPORT LEAD
P'UNK AVENUE |<PHONE_NUMBER>  |  punkave.com

--
*THOMAS BOUTELL, *SUPPORT LEAD
P'UNK AVENUE |<PHONE_NUMBER>  |  punkave.com

Right. In 0.5, if I'm right, you could select Admin:all and that would allow that group to have admin privileges for all the things, yeah?
The reason they've been able to add events is because they went into the group and checked 'Admin:file' and 'Admin:event' in addition to 'Admin:all' which solved it (I didn't know that until now).

I just think we're missing something because we have lots of 2.0 sites that
don't even have an editable permission list, they just have one group
hardcoded to the permission "admin."
I will try to reproduce the issue though!
On Thu, Sep 8, 2016 at 1:52 PM, Kylee Acker<EMAIL_ADDRESS>wrote:

Right. In 0.5, if I'm right, you could select Admin:all and that would
allow that group to have admin privileges for all the things, yeah?
The reason they've been able to add events is because they went into the
group and checked 'Admin:file' and 'Admin:event' in addition to 'Admin:all'
which solved it (I didn't know that until now).
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/punkave/apostrophe/issues/650#issuecomment-245681909,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAB9fSDrqHAL60A3hlcvD83-pO6B2qMuks5qoEtwgaJpZM4J4HIs
.

--
*THOMAS BOUTELL, *SUPPORT LEAD
P'UNK AVENUE |<PHONE_NUMBER>  |  punkave.com

I am closing this item because the issue has been idle and we have had no complaints of the "admin:all" permission not working as expected since.


Little bit of August's translations for UA locale

Thanks!

You are welcome, but it is not the end from my side ;) i will try to to spend more time for translations


Rendering links in posts

Sometimes breaks: https://beta.unsweets.net/posts/78139

Links also try to open in the web page.
https://beta.unsweets.net/posts/78168
The link should be https://github.com/bazbt3/pnut_io_stuff/wiki/WednesdayChallenge, but it tries to open https://beta.unsweets.net/bazbt3/pnut_io_stuff/wiki/WednesdayChallenge

Fixed.


Test | Added connection Open check to DataUtility methods for Db entity cleanups

Fix for Intermittent failure for CollatedDataReaderTest. (https://dev.azure.com/sqlclientdrivers-ci/sqlclient/_build/results?buildId=46069&view=ms.vss-test-web.build-test-results-tab&runId=497922&resultId=100031&paneView=debug).
The connection closes before cleanup in finally block. I updated the DataUtility methods to open connection to do mandatory cleanup if there is ever connection failure and wrapped the command execution in try catch block as they are mostly invoked from finally block.

No longer needed after changes from #1658.


Maintenance status?

I was wondering if this project is still being maintained.
I'm considering switching to Phaxio from Twilio as they are shutting down their fax service. But, it doesn't look like there's been any activity for a little while. Of course, the project could just be stable which is a good thing. :)

Hi Dominic.  Yes, this is the official Phaxio Node SDK and we do maintain it to ensure it is operational and secure.  If you have any specific questions about implementation or the sdk, we'd be happy to chat!  Hit me up at<EMAIL_ADDRESS>Thanks!

You rock! Thanks for the quick response. It sounds like I found a good place to migrate from Twilio to. :)

You rock! Thanks for the quick response. It sounds like I found a good place to migrate from Twilio to. :)


Escape single quotes in test names on Linux (#81)

Tested manually on Ubuntu. Fixes examples like "tester's test" and other variations.
I wanted to add unit tests but I couldn't get them to run:
node_modules/jest-editor-support/index.d.ts:35:2 - error TS1036: Statements are not allowed in ambient contexts.

35 };
    ~

thx @kevin-brotcke. i also ran into this issue. i think you just have to delete the semicolon in the typing of jest-editor-support.

Got it to run by removing the semicolon and current tests pass. I can add a test containing a single quote but I don't think it's that useful because what I actually want to test is buildJestCommand method but it's private and looks like it'll be a little refactoring work to get it under test. I think this change is safe to merge without unit tests (for now) and we can add coverage later as part of #117 .

Can we merge this?

yes sir


Make password length configurable

I'm not entirely sure about this, but it was brought up in the mailing list (see discussion).
The gist of it is: in LDAP environments, passwords might not match the minimum length required by Portus (and Portus administrators might not be in charge of the LDAP server), so trying to login with Portus with valid LDAP credentials is simply not possible. Moreover, in this specific case, the Portus instance is run internally, so security concerns shouldn't be that important.
Of course, this would be really discouraged but it would solve this scenario. I'm not entirely sure about adding this, and that's why I'm adding the discussing label.

Security is important, but if we decide to authenticate to ldap, maybe we should let ldap to check the password policy, portus should do the check only when using built-in authentication.

:+1: I'm currently having issues with this :)
I think that while Portus is right to enforce a password policy (that should be configurable) for internal users, I can't enforce one for external users, the LDAP directory should

When delegating authentication to LDAP it really stops being Portus' responsibility to enforce password policies. I dug around in devise and found the responsible validation in the validatable module here.
What do you think about disabling the validatable module in the user model (see gist here) in LDAP setups?

I am wondering why password policy is enforced at logon time. Usually a policy is enforced at password generation/change time, eventually accompanied by the possibility for an admin user to override this policy.

I skimmed the documentation but to no avail. What if we asked the generous folks to give us devise to explain their rationale?

Fixed with #1064


ValueError: hidden size is called d_model

I tried to use longformer-base-4096 with LongformerEncoderDecoderConfig.
ValueError: hidden size is called d_model

ah, that won't work. The encoder-decoder work is still in progress and we didn't post it on the HF repo yet, You will need to download the checkpoint manually (check readme for links) https://github.com/allenai/longformer/tree/encoderdecoder

Ok, i download this checkpoint-base: https://ai2-s2-research.s3-us-west-2.amazonaws.com/longformer/longformer-encdec-base-16384.tar.gz:
RuntimeError: Error(s) in loading state_dict for LongformerEncoderDecoderForConditionalGeneration:
size mismatch for model.encoder.embed_positions.weight: copying a param with shape torch.Size([16386, 768]) from checkpoint, the shape in current model is torch.Size([1026, 768]).

Please make sure you are using my fork of the HF/transformers repo (listed here: https://github.com/allenai/longformer/blob/encoderdecoder/requirements.txt#L1)

Ok, i download this checkpoint-base: https://ai2-s2-research.s3-us-west-2.amazonaws.com/longformer/longformer-encdec-base-16384.tar.gz:
RuntimeError: Error(s) in loading state_dict for LongformerEncoderDecoderForConditionalGeneration:
size mismatch for model.encoder.embed_positions.weight: copying a param with shape torch.Size([16386, 768]) from checkpoint, the shape in current model is torch.Size([1026, 768]).

hi bro, did u fixed the probelm yet? I also met the same problem.

Ok, i download this checkpoint-base: https://ai2-s2-research.s3-us-west-2.amazonaws.com/longformer/longformer-encdec-base-16384.tar.gz:
RuntimeError: Error(s) in loading state_dict for LongformerEncoderDecoderForConditionalGeneration:
size mismatch for model.encoder.embed_positions.weight: copying a param with shape torch.Size([16386, 768]) from checkpoint, the shape in current model is torch.Size([1026, 768]).

hi,I also meet the error in HG/transformers of version 4.4.2, in version 3.x, the code can run


MTV-262 Verify and update prerequisites for MTV 2.3

MTV-262 Verify and update prerequisites as needed
MTV 2.3
https://issues.redhat.com/browse/MTV-262

LGTM


Added a rate us page

I have done the necessary changes and made the page responsive.
https://github.com/anuragverma108/SwapReads/assets/140010647/341311dc-c64b-46ea-8d31-825c72adf178

Good work @Aashi19234


specify datum for location coordinates

Although the GPS system's use of EPSG:4326/WGS84 has made it a more-or-less standard choice, there are numerous American authorities that still use NAD83 or even more region-specific datums. This should be specified to avoid ambiguity or imprecisely submitted coordinates. (context)

What if we reference the geoJson spec which defines the coordinate reference system you want....

Coordinate Reference System

The coordinate reference system for all GeoJSON coordinates is a
geographic coordinate reference system, using the World Geodetic
System 1984 (WGS 84) [WGS84] datum, with longitude and latitude units
of decimal degrees.  This is equivalent to the coordinate reference
system identified by the Open Geospatial Consortium (OGC) URN
urn:ogc:def:crs:OGC::CRS84.  An OPTIONAL third-position element SHALL
be the height in meters above or below the WGS 84 reference
ellipsoid.  In the absence of elevation values, applications
sensitive to height or depth SHOULD interpret positions as being at
local ground or sea level.
Note: the use of alternative coordinate reference systems was
specified in [GJ2008], but it has been removed from this version of
the specification because the use of different coordinate reference
systems -- especially in the manner specified in [GJ2008] -- has
proven to have interoperability issues.  In general, GeoJSON
processing software is not expected to have access to coordinate
reference system databases or to have network access to coordinate
reference system transformation parameters.  However, where all
involved parties have a prior arrangement, alternative coordinate
reference systems can be used without risk of data being
misinterpreted.

It also address precision....
https://tools.ietf.org/html/rfc7946#section-3.1.10
3.1.10.  Uncertainty and Precision
As in [RFC5870], the number of digits of the values in coordinate
positions MUST NOT be interpreted as an indication to the level of
uncertainty.

@DeraldDudley I like it. If GeoJSON PRs get merged for affected data fields then I agree that specifying datum is redundant.

Implemented with GeoJSON in #53


Incompatiblity with rules_rust + rust version >= 1.70.0

I'm seeing this error:
home/dhash/.cache/bazel/_bazel_dhash/d01f478a72a2d3121d654f978507dcd9/external/crate_index__async-trait-0.1.71/BUILD.bazel:22:16: Compiling Rust proc-macro async_trait v0.1.71

<snip>

error: linking with `external/zig_sdk/tools/x86_64-linux-gnu.2.28/c++` failed: exit status: 1
  |
  = note: LC_ALL="C" PATH="/home/radix/rbe/worker/build/27ac6bc9a01363d5/root/bazel-out/k8-opt-exec-2B5CBBC6/bin/external/rust_linux_x86_64__x86_64-unknown-linux-gnu__stable_tools/rust_toolchain/lib/rustlib/x86_64-unknown-linux-gnu/bin" VSLANG="1033" "external/zig_sdk/tools/x86_64-linux-gnu.2.28/c++" "-Wl,--version-script=/tmp/rustcHJc8ST/list" "-Wl,--no-undefined-version"

<snip>

= note: error: unsupported linker arg: --no-undefined-version

which is present due to, as far as I can tell, that flag being flipped on by rust in that release by default. It's only for proc-macros, and as a first pass solution i've put in https://github.com/ziglang/zig/pull/17326. If there's a better way to do this, that would be cool.

Fixed upstream in https://github.com/ziglang/zig/pull/18486 , you can pick up this change with hermetic_cc_toolchain v2.2.0.


Ref use of xtgeo.create_box_grid(). Latest version of xtgeo causes lots of warnings in the tests now, and probably requires a slightly larger cleanup. Will create separate issue on this.

Ref use of xtgeo.create_box_grid(). Latest version of xtgeo causes lots of warnings in the tests now, and probably requires a slightly larger cleanup. Will create separate issue on this.
Originally posted by @perolavsvendsen in https://github.com/equinor/fmu-dataio/issues/146#issuecomment-995681288

Also other tests fail now due to something in XTgeo. Should do a thorough synchronization.


Mobile Wallet, creating new instance

Creating new wallet, Unable to see the seed.
@vinayaga07 , Update:   Seems it is a blocker. That workaround bug probably was fixed

02-19 12:37:52.094  6956  6980 D mwc-qt-wallet: Processing task ' "TaskInit" '
02-19 12:37:52.094  6956  6980 D mwc-qt-wallet: TaskInit::processTask:  "Evt(T=S_WELCOME, msg=4.1.0), Evt(T=S_LINE, msg=Welcome to wallet713 for MWC v4.1.0), Evt(T=S_LINE, msg=Initialising a new wallet), Evt(T=S_LINE, msg=Set an optional password to secure your wallet with. Leave blank for no password.), Evt(T=S_LINE, msg=Your recovery phrase is:), Evt(T=S_LINE, msg=reject steel elev...... e dish border camp), Evt(T=S_LINE, msg=Please back-up these words in a non-digital format.), Evt(T=S_INIT_WANT_ENTER, msg=)"
02-19 12:37:52.094  6956  6980 D mwc-qt-wallet: Get a passphrase, it has words:  24
02-19 12:37:52.096  6956  6980 W mwc-qt-wallet: qrc:/windows_mobile/NewSeed.qml:9: Error: Invalid write to global property "seedText"
The issue happens because of that:
mwc-qt-wallet: qrc:/windows_mobile/NewSeed.qml:9: Error: Invalid write to global property "seedText"

With Today's build, able to reproduce this issue.


Support Rail's importmap

Is there an existing issue for this?

[X] I have searched the existing issues

Feature description
Currently there is no package ecosystem support for importmap-rails.
I have searched importmap both in the Issues and Pull requests, with no result returned.

https://github.com/dependabot/dependabot-core/pulls?q=is%3Apr+importmap
https://github.com/dependabot/dependabot-core/issues?q=is%3Aissue+importmap

There has been discussion in importmap-rails about addressing outdated pins, which led to this result.
It would be great for the Ruby on Rails community to have dependabot support for importmap.

Would like to see this support :)

Would love it !

We rely on dependabot so much, cause we love it! :P But missing the PRs for the frontend now 😢


Release 0.11

It has been 2 month since the release of 0.10 already, so we should start thinking about the next release.
Is there anything specific that should go into 0.11?

So, I'll try again to get some opinions on a 0.11 release. What do you think should go into the release? When approximately should we schedule a release?

@stefankoegl There are a few bug fixes that I think it would be nice to get out there. So if we are willing to do just that we could release now.
However, I do think the work in #296 would be worth pursuing and try to get in the next release. The only thing is that it is not done. I need feedback from people on whether they think it is a good idea or not. Provided I get the feedback, I think I should find time to finish it within the next couple of weeks and we could then schedule a release there.
On another note, I don't think we need to wait for #262, even though it would be nice. It is my feeling that it still requires more work and that @lawrenceakka is tied up with real life.

OK, then let's wait for #296. I should find time to do a review in the next days.

@stefankoegl It's actually more a "yes" or "no" on the general idea rather than a formal review, so should not take so long

OK, it seems we have everything for the release.. If there are no objections I'd prepare the release most likely tomorrow afternoon/evening (UTC).

@stefankoegl I think there are a few more issues marked for 0.11:

#225 Which I think is good to go, I'm just waiting for the go a head from @dajobe
#276 Which I don't think there is any progress on, so probably just postpone it.

I have created the proposed release notes in #311, already including #225.
If there are no objections I'd publish the release after #225 is merged.

Sounds great.

I have just released SoCo 0.11!

PyPI
GitHub release
Release notes
Change Log

Thanks everyone for their contributions! @simonalpha @trollkarlen @lawrenceakka @dajobe @robwebset @gutworth @poirier @michaelotto @hildroid @KennethNielsen

Cheers

and thanks for the effort


Provide support for resource provider profiles

Is your feature request related to a problem? Please describe.
Currently you need to specify the api version for each resource type.
This makes it very cumbersome to maintain and to keep it up to date.
Describe the solution you'd like
A 'provider' resource could append the correct api version to each resource.
Provider resources could follow the concept of profiles in azure stack hub

Is this equivalent to #16?
#364 also seems related

As per Brian's comment to #16, I do think so, but I would leverage an already existing concept. As edge/ onpremises can easily be included.
As per #364, it would perfectly relate.
Hypotetical example:
Resource provider azure.profile = {
Name = '2019-03-01-hybrid'
}
resource stg 'Microsoft.Storage/storageAccounts= {
name: 'uniquestorage001' // must be globally unique
location: 'eastus'
kind: 'Storage'
sku: {
name: 'Standard_LRS'
}
}
The api version could easily be append to each resource during build based on the already existing profiles and easily eliminates the micro management need in api profiles.
If a resourced referenced a specific api, it supersedes the defined top level profile.


Fix mocking for modules with folders on windows

Summary
Fixing #4014: mocks for packages with folders (like lodash/fp/sum) on Windows.
Questions

what is the better way to write tests for that?
Is this change enough? Doesn't it break something else on windows? There are not a lot for tests

This works for me, do you mind running yarn lint -- --fix and pushing to this branch again?

@cpojer sure, done


chore(@toss/date): use relational operators to compare dates

Overview

Use relational operators to compare dates.
From issue #463
PR Checklist

[x] I read and included theses actions below

I have read the Contributing Guide
I have written documents and tests, if needed.

Thanks for your contribution!
While it is true that dates can be compared with relational operators, it is because dates are implicitly converted numbers before they are being compared.
Personally, I prefer to use more explicit functions to compare dates; this is why we are using functions from date-fns.
Closing the pull request for this reason. But we appreciate your pull request!


Add new kafka topics and samza job properties for druid

@anandp504 Please review

Please raise this PR against release-1.15 branch.