[Bug] Quantized Model Incompatibility with TurboMind Inference Engine

Checklist

[X] 1. I have searched related issues but cannot get the expected help.
[X] 2. The bug has not been fixed in the latest version.

Describe the bug
I am attempting to quantize a model and use it with the TurboMind inference engine. I have successfully quantized the model following the provided instructions. The  documentation seems to imply that the quantize version only works with the pytorch backend?  If that is the case I  think it would be good to make that explicit in the documentation.
When I try to run the quantized version with TurboMind, I encounter a KeyError (see below).
I tested both latest and nightly.
Reproduction
lmdeploy lite smooth_quant meta-llama/Meta-Llama-3-8B-Instruct --work-dir ./lmdeploy_llama3_8b_q8
lmdeploy serve api_server ./lmdeploy_llama3_8b_q8 --server-port 23333 # does not work

Environment
sys.platform: linux
Python: 3.10.12 (main, Jun 11 2023, 05:26:28) [GCC 11.4.0]
CUDA available: True
MUSA available: False
numpy_random_seed:<PHONE_NUMBER>
GPU 0: NVIDIA A100-PCIE-40GB MIG 7g.40gb
CUDA_HOME: /usr
NVCC: Cuda compilation tools, release 11.5, V11.5.119
GCC: x86_64-linux-gnu-gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
PyTorch: 2.2.2+cu121
PyTorch compiling details: PyTorch built with:
  - GCC 9.3
  - C++ Version: 201703
  - Intel(R) oneAPI Math Kernel Library Version 2022.2-Product Build 20220804 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v3.3.2 (Git Hash 2dc95a2ad0841e29db8b22fbccaf3e5da7992b01)
  - OpenMP 201511 (a.k.a. OpenMP 4.5)
  - LAPACK is enabled (usually provided by MKL)
  - NNPACK is enabled
  - CPU capability usage: AVX2
  - CUDA Runtime 12.1
  - NVCC architecture flags: -gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_90,code=sm_90
  - CuDNN 8.9.2
  - Magma 2.6.1
  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=12.1, CUDNN_VERSION=8.9.2, CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=0 -fabi-version=11 -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=pedantic -Wno-error=old-style-cast -Wno-missing-braces -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=2.2.2, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=1, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF,

TorchVision: 0.17.2+cu121
LMDeploy: 0.5.0+
transformers: 4.42.4
gradio: Not Found
fastapi: 0.111.1
pydantic: 2.8.2
triton: 2.2.0

Error traceback
(.venv) ➜  benchmark_llm_inference lmdeploy serve api_server ./lmdeploy_llama3_8b_q4 --server-port 23333
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Traceback (most recent call last):
  File "/home/ubuntu/jan/benchmark_llm_inference/.venv/bin/lmdeploy", line 8, in <module>
    sys.exit(run())
  File "/home/ubuntu/jan/benchmark_llm_inference/.venv/lib/python3.10/site-packages/lmdeploy/cli/entrypoint.py", line 43, in run
    args.run(args)
  File "/home/ubuntu/jan/benchmark_llm_inference/.venv/lib/python3.10/site-packages/lmdeploy/cli/serve.py", line 310, in api_server
    run_api_server(args.model_path,
  File "/home/ubuntu/jan/benchmark_llm_inference/.venv/lib/python3.10/site-packages/lmdeploy/serve/openai/api_server.py", line 1197, in serve
    VariableInterface.async_engine = pipeline_class(
  File "/home/ubuntu/jan/benchmark_llm_inference/.venv/lib/python3.10/site-packages/lmdeploy/serve/async_engine.py", line 189, in __init__
    self._build_turbomind(model_path=model_path,
  File "/home/ubuntu/jan/benchmark_llm_inference/.venv/lib/python3.10/site-packages/lmdeploy/serve/async_engine.py", line 234, in _build_turbomind
    self.engine = tm.TurboMind.from_pretrained(
  File "/home/ubuntu/jan/benchmark_llm_inference/.venv/lib/python3.10/site-packages/lmdeploy/turbomind/turbomind.py", line 342, in from_pretrained
    return cls(model_path=pretrained_model_name_or_path,
  File "/home/ubuntu/jan/benchmark_llm_inference/.venv/lib/python3.10/site-packages/lmdeploy/turbomind/turbomind.py", line 144, in __init__
    self.model_comm = self._from_hf(model_source=model_source,
  File "/home/ubuntu/jan/benchmark_llm_inference/.venv/lib/python3.10/site-packages/lmdeploy/turbomind/turbomind.py", line 237, in _from_hf
    output_model = OUTPUT_MODELS.get(output_model_name)(
  File "/home/ubuntu/jan/benchmark_llm_inference/.venv/lib/python3.10/site-packages/lmdeploy/turbomind/deploy/target_model/w4.py", line 80, in __init__
    super().__init__(input_model, cfg, to_file, out_dir)
  File "/home/ubuntu/jan/benchmark_llm_inference/.venv/lib/python3.10/site-packages/lmdeploy/turbomind/deploy/target_model/base.py", line 168, in __init__
    self.cfg = self.get_config(cfg)
  File "/home/ubuntu/jan/benchmark_llm_inference/.venv/lib/python3.10/site-packages/lmdeploy/turbomind/deploy/target_model/w4.py", line 92, in get_config
    w1s, _, _ = bin.ffn_scale(i)
  File "/home/ubuntu/jan/benchmark_llm_inference/.venv/lib/python3.10/site-packages/lmdeploy/turbomind/deploy/source_model/llama_awq.py", line 52, in ffn_scale
    return ensure_fp16orint32(self._ffn(i, 'scales'))
  File "/home/ubuntu/jan/benchmark_llm_inference/.venv/lib/python3.10/site-packages/lmdeploy/turbomind/deploy/source_model/llama.py", line 103, in _ffn
    tensor = self.params[
KeyError: 'model.layers.0.mlp.gate_proj.scales'

W8A8 is only supported on PyTorch Engine for now.

hello, i met the problem too. I use lmdeploy to quant a model in kv8 model and use lmdeploy serve api_server .Meta-Llama-3-8B-kv8-hf/  to try to run the service and the same error occured. Just like KeyError: 'model.layers.0.mlp.gate_proj.scales'. Although i found in 0.4.* version llmdeploy has already remove the offline quant but i want to benchmark the difference between online mode and offline mode. Is there anyone can help me. thanks a lot

oh, i found the comment and added the --backend pytorch and it works.


Enhancement: Add single series selector on mouse-over for multi-series line charts.

Feature Description
This is sparked from the following chart on fivethirtyeight.com

On mouse over, there is an indicator and after 1-2 second pause, it highlights the specific point in the line chart.
Initial mouse over:

after 1-2 second delay:

Goal of Feature
This feature would be a very nice design element to the line charts when using multi-series.
Current Solution
There is no way to do this currently (although I guess I could use custom charts). I have an example of the short comings in the mds in a box project: Advanced analytics page.
Currently multi series line chart mouse over looks like this:

Are you interested in contributing this feature?
I can QA feature but I am not a front end guy!

The scatterplot ALMOST gets there, so "enabling lines" on a scatterplot and a little indicator might work.


[SPARK-48586][SS][3.5] Remove lock acquisition in doMaintenance() by making a deep copy of file mappings in RocksDBFileManager in load()

Backports #46942 to 3.5
What changes were proposed in this pull request?
When change log checkpointing is enabled, the lock of the RocksDB state store is acquired when uploading the snapshot inside maintenance tasks, which causes lock contention between query processing tasks and state maintenance thread. This PR fixes lock contention issue introduced by https://github.com/apache/spark/pull/45724.
The changes include:

Removing lock acquisition in doMaintenance()
Adding a copyFileMappings() method to RocksDBFileManager,  and using this method to deep copy the file manager state, specifically the file mappings versionToRocksDBFiles and localFilesToDfsFiles,  in load()
Capture the reference to the file mappings  in commit().

Why are the changes needed?
We want to eliminate lock contention to decrease latency of streaming queries so lock acquisition inside maintenance tasks should be avoided. This can introduce race conditions between task and maintenance threads. By making a deep copy of versionToRocksDBFiles and localFilesToDfsFiles in RocksDBFileManager, we can ensure that the file manager state is not updated by task thread when background snapshot uploading tasks attempt to upload a snapshot.
Does this PR introduce any user-facing change?
No
How was this patch tested?

Added unit test cases.
Was this patch authored or co-authored using generative AI tooling?
No

Looks like the CI failed with infra side issue. I'll merge once the build without tests is passing, as I would have done that if there wasn't merge conflict.

Locally verified the build pass. Thanks! Merging to 3.5!


[Chore]: Check if we are vulnearable to ChainJacking

Background
There is a vulnerability of golang modules called chain jacking. The idea is that if you depend on a go module from github that is hosted from a regular github user (not an org) and the user changes their handle, the old handle becomes available and a malicious user can take it and host a modified version of the code, thus attacking your app.
Action to take
Let's try the chain jacking tool to detect if we have any depencencies that are vulnerable to this attack. We should also consider adding a github action that checks that when a PR comes in (there is an example for that in the chain jacking repo.
Impact
We are not vulnerable to chain jacking
Dev Notes
No response

~/workspace/cf-k8s-controllers main
❯ python3 -m chainjacking -gt "..."
✅ Dependencies analyzed and found to be safe from ChainJacking attacks


pivot --example wiki not working on Mac

i dont know why but following the How-to guide, this command is not working: (im on MAC OS)
pivot --example wiki
bash: pivot: command not found

Hi! Are you issuing the command from the base pivot directory? If so could you try
bin/pivot --example wiki

Ups! thanks! my fault


Added new styles to mark branches as pre releases

This adds two new styles pep440-branch and pep440-post-branch which
mark all versions generated from non master branches with a version
compatible with pip --pre.
These formats are useful if you want to publish all CI builds to a pypi
repository but don't want feature branch builds to be pulled
automatically ahead of the master branch with pip install.

@warner  Do you think this could be merged ? Tests are passing and feature is documented.

We're working on reviving this project. Can you rebase/merge master to resolve conflicts and refresh the CI?

@effigies all set

Sorry, this is conflicting with another recent merge. Could you try another rebase?

@effigies all set

Thanks!


Datadog: customizable Logs API endpoint

Is your feature request related to a problem? Please describe.
The API endpoint where the logs are sent to is statically defined in the code. It should be customizable.
Datadog has a couple API endpoints, continent-based. The data must be sent to the appropriate endpoint to get ingested, it's
not only a latency/networking concern.
Describe the solution you'd like
Be able to set a custom API endpoint FQDN in the consumer options. If not provided, the default one (http-intake.logs.datadoghq.com) should be used.
Alternatively (as those endpoint FQDN seem to be quite static), the selection of the site (US or EU) could do it.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
e need to use the EU site endpoint. FQDN=http-intake.logs.datadoghq.eu

Thank you for your feedback. I have added AUTOTOOL-2752 to our internal product backlog to add region selection to the Datadog consumer.

This has been resolved in the 1.24.0 release.


Rawtools: new Galaxy tool

RawTools is an open-source and freely available package designed to perform scan data parsing and quantification, and quality control analysis of Thermo Orbitrap raw mass spectrometer files.
RawTools galaxy tool is on working condition, postponing the implementation of Quality Control analysis.
There are currently 2 main issues about which it would be great to have galaxy-p team feedback:

Maximum size of a Galaxy tool. In order to develop tests, we need to include into the tool, at least, one .raw file and .mgf file; first ones are specially big (I have been reported that the minimum raw file available in my lab is about 66mb), and we do not know which could be an acceptable maximum size for a galaxy tool.
Current consequences of this doubt is tests have not been implemented yet (next issue implies another reason for this).

We are having similar problems with raw files than the ones I had using moFF. When uploading our .raw files to Galaxy (my Galaxy version is 17_09, but we cannot update it for some time yet) , it shows an error with this message:

WARNING:galaxy.model:Datatype class not found for extension 'raw'
I think that, in theory, thermo.raw file type supports .raw extension, but there is some problem with this extension that does not properly support our Thermo raw files with .raw extension.
I tried to upload the same file to usegalaxy.org and it also showed the same error, but the history item finished green instead of red... it would be great to have the same behavior (or no error at all, of course) here...
In order to bypass this problem on our server I had to install "proteomics_datatypes" tool, which adds "raw" extension support and it allows me to upload my .raw files successfully.
But, this tool is deprecated and it seems that it is not currently the correct approach to solve this.
By the way, even with this tool installed and a "tool_dependencies.xml" file with proteomics_datatypes repo, I get an error when executing the first test I am trying to develop on my local (the one commented on the uploaded xml):
ERROR

======================================================================
ERROR: Raw Tools ( rawtools ) > Test-1
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/parallels/GitHub/galaxy/test/functional/test_toolbox.py", line 317, in test_tool
    self.do_it(td)
  File "/home/parallels/GitHub/galaxy/test/functional/test_toolbox.py", line 48, in do_it
    stage_data_in_history(galaxy_interactor, testdef.test_data(), test_history, shed_tool_id)
  File "/home/parallels/GitHub/galaxy/test/base/interactor.py", line 44, in stage_data_in_history
    upload_wait()
  File "/home/parallels/GitHub/galaxy/test/base/interactor.py", line 314, in wait
    while not self.__history_ready(history_id):
  File "/home/parallels/GitHub/galaxy/test/base/interactor.py", line 336, in __history_ready
    return self._state_ready(state, error_msg="History in error state.")
  File "/home/parallels/GitHub/galaxy/test/base/interactor.py", line 406, in _state_ready
    raise Exception(error_msg)
Exception: History in error state.
-------------------- >> begin captured stdout << ---------------------
Problem in history with id<PHONE_NUMBER>fa2d5a - summary of datasets below.
--------------------------------------
| 1 - qExactive01819.thermo.raw (HID - NAME)
| Dataset State:
|  error
| Dataset Blurb:
|  error
| Dataset Info:
|  uploaded thermo.raw file
|  WARNING:galaxy.model:Datatype class not found for extension 'raw'
|  WARNING:galaxy.model:Datatype class not found for extension 'raw'
|  WARNING:galaxy.model:Datatype class not found for extension 'raw'
|  WARNING:galaxy.model:Datatype c
| Peek:
|  None
| Dataset Job Standard Output:
|  *Standard output was empty.*
| Dataset Job Standard Error:
|  WARNING:galaxy.model:Datatype class not found for extension 'raw'
|  WARNING:galaxy.model:Datatype class not found for extension 'raw'
|  WARNING:galaxy.model:Datatype class not found for extension 'raw'
|  WARNING:galaxy.model:Datatype class not found for extension 'raw'
|  WARNING:galaxy.model:Datatype class not found for extension 'raw'
|
--------------------------------------

--------------------- >> end captured stdout << ----------------------

----------------------------------------------------------------------
XML: /tmp/tmpo3rKaI/xunit.xml
----------------------------------------------------------------------
Ran 1 test in 27.905s

FAILED (errors=1)
2019-03-05 12:10:57,092 INFO  [test_driver] Shutting down
2019-03-05 12:10:57,093 INFO  [test_driver] Shutting down embedded galaxy web server
2019-03-05 12:10:57,097 INFO  [test_driver] Embedded web server galaxy stopped
2019-03-05 12:10:57,097 INFO  [test_driver] Stopping application galaxy
2019-03-05 12:10:58,594 INFO  [galaxy.jobs.handler] sending stop signal to worker thread
2019-03-05 12:10:58,594 INFO  [galaxy.jobs.handler] job handler queue stopped
2019-03-05 12:10:58,594 INFO  [galaxy.jobs.runners] TaskRunner: Sending stop signal to 2 worker threads
2019-03-05 12:10:58,595 INFO  [galaxy.jobs.runners] LocalRunner: Sending stop signal to 4 worker threads
2019-03-05 12:10:58,595 INFO  [galaxy.jobs.handler] sending stop signal to worker thread
2019-03-05 12:10:58,595 INFO  [galaxy.jobs.handler] job handler stop queue stopped
2019-03-05 12:10:58,596 INFO  [test_driver] Application galaxy stopped.
2019-03-05 12:10:58,597 INFO  [galaxy.jobs.handler] sending stop signal to worker thread
2019-03-05 12:10:58,598 INFO  [galaxy.jobs.handler] job handler queue stopped
2019-03-05 12:10:58,598 INFO  [galaxy.jobs.runners] TaskRunner: Sending stop signal to 2 worker threads
2019-03-05 12:10:58,598 INFO  [galaxy.jobs.runners] LocalRunner: Sending stop signal to 4 worker threads
2019-03-05 12:10:58,598 INFO  [galaxy.jobs.handler] sending stop signal to worker thread
2019-03-05 12:10:58,598 INFO  [galaxy.jobs.handler] job handler stop queue stopped
Testing complete. HTML report is in "/media/psf/shared/rawtools_for_tests/tool_test_output.html".
There were problems with 1 test(s) - out of 1 test(s) executed. See /media/psf/shared/rawtools_for_tests/tool_test_output.html for detailed breakdown.
Traceback (most recent call last):
  File "/home/parallels/Programs/Bio/planemo/.venv/bin/planemo", line 11, in <module>
    sys.exit(planemo())
  File "/home/parallels/Programs/Bio/planemo/.venv/local/lib/python2.7/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/parallels/Programs/Bio/planemo/.venv/local/lib/python2.7/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/parallels/Programs/Bio/planemo/.venv/local/lib/python2.7/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/parallels/Programs/Bio/planemo/.venv/local/lib/python2.7/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/parallels/Programs/Bio/planemo/.venv/local/lib/python2.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/home/parallels/Programs/Bio/planemo/.venv/local/lib/python2.7/site-packages/click/decorators.py", line 64, in new_func
    return ctx.invoke(f, obj, *args[1:], **kwargs)
  File "/home/parallels/Programs/Bio/planemo/.venv/local/lib/python2.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/home/parallels/Programs/Bio/planemo/.venv/local/lib/python2.7/site-packages/planemo/cli.py", line 178, in handle_blended_options
    return f(*args, **kwds)
  File "/home/parallels/Programs/Bio/planemo/.venv/local/lib/python2.7/site-packages/planemo/commands/cmd_test.py", line 77, in cli
    return_value = run_in_config(ctx, config, **kwds)
  File "/home/parallels/Programs/Bio/planemo/.venv/local/lib/python2.7/site-packages/planemo/galaxy/test/actions.py", line 115, in run_in_config
    kwds=kwds
  File "/home/parallels/Programs/Bio/planemo/.venv/local/lib/python2.7/site-packages/planemo/galaxy/test/actions.py", line 129, in handle_reports_and_summary
    **kwds
  File "/home/parallels/Programs/Bio/planemo/.venv/local/lib/python2.7/site-packages/planemo/galaxy/test/actions.py", line 220, in _handle_summary
    **kwds
  File "/home/parallels/Programs/Bio/planemo/.venv/local/lib/python2.7/site-packages/planemo/galaxy/test/actions.py", line 232, in _summarize_tests_full
    _summarize_test_case(test_case_data, **kwds)
  File "/home/parallels/Programs/Bio/planemo/.venv/local/lib/python2.7/site-packages/planemo/galaxy/test/actions.py", line 250, in _summarize_test_case
    get_dict_value("data", structured_data)
  File "/home/parallels/Programs/Bio/planemo/.venv/local/lib/python2.7/site-packages/planemo/test/results.py", line 113, in get_dict_value
    raise KeyError("No key [%s] in [%s]" % (key, data))
KeyError: 'No key [status] in [None]'

So, at the end, this problem also prevents me to create proper tests...
I would be very thankful if someone could orientate us about the best way to focus these issues, please.
Greetings,
Carlos

can you try thermo.raw see here: https://github.com/galaxyproject/galaxy/blob/dev/config/datatypes_conf.xml.sample#L234

can you try thermo.raw see here: https://github.com/galaxyproject/galaxy/blob/dev/config/datatypes_conf.xml.sample#L234

Um... when changing the format to "thermo.raw" in the input field this problem still occurs...
<param format="thermo.raw" name="input_raw" type="data" label="Raw file" multiple="true"
            help="Select the raw file from history"/>

@CarlosHorro, my lab shared a few very small Thermo files for this sort of thing: https://github.com/HegemanLab/small-thermo-raw-file-examples
We could add MSMS files if needed too.

@eslerm, thanks for sharing the files. RawTools is specifically intended for data-dependent acquisition (DDA) .raw files from Orbitrap intruments (both MS2 and SPS-MS3 DDA is fine). Does your lab have any small files of that variety?
Kevin

@kevinkovalchik added https://github.com/HegemanLab/small-thermo-raw-file-examples/issues for this
I'll talk to the instrument operator.

@eslerm Okay, thanks. I might be able to take care of it on my end by taking a slice out of a larger file. If I succeed would you like a copy to host with your other test files?

@eslerm I was able to create a small file by slicing up a larger file.

Nice @kevinkovalchik :)
How did you do it?

There is a set of tools from Thermo called FT Programs, and in one of those (RecalOffline or something) you can save slices of raw files. It requires an installation of XCalibur to work.

Can you enable tests, I will have a look at this then.

Can you enable tests, I will have a look at this then.

Yes, I'm working on them.

As RawTools can execute multiple files at the same time (and also return multiple files of the same type), I think I should use collections in all outputs.
I've been trying to start to do it with output_mgf, but I'm not able to do it. What is wrong if I do something like this?
<collection name="output_mgf" type="list" label="${tool.name} - ${on_string}: MGF">
            <filter>pq_options['mgf_conditional']['mgf_select'] == "yes"</filter>
            <discover_datasets pattern="(\S+)\.mgf" ext="mgf" directory="output"/>
        </collection>

I think it should work, but the history item in Galaxy stays infinitely yellow waiting to find a dataset which fits to the discover_datasets params...
MGF files generated finish with .mgf extension and the pattern is like the one specified there. This is the content of the "output" folder:
-rw-r--r-- 1 root root   2447597 Mar 25 12:22 ch_23Aug2018_HeLa_Std_1.raw_Matrix.txt
-rw-r--r-- 1 root root       737 Mar 25 12:22 ch_23Aug2018_HeLa_Std_1.raw_Metrics.txt
-rw-r--r-- 1 root root 137754912 Mar 25 12:22 ch_23Aug2018_HeLa_Std_1.raw.mgf
-rw-r--r-- 1 root root   2558440 Mar 25 12:25 qExactive01819.raw_Matrix.txt
-rw-r--r-- 1 root root       733 Mar 25 12:25 qExactive01819.raw_Metrics.txt
-rw-r--r-- 1 root root  29672634 Mar 25 12:25 qExactive01819.raw.mgf

The delay with the tests is mainly due to the work necessary to obtain a .raw file as small as possible to be included into the tool for executing tests.
Some days ago I got a .raw file from my lab of about 70mb, which is about 7mb when converted to mgf. Are these sizes (about 80mb in total) valid to be included into the tool @bgruening ?

Sorry for the delay on the small files. I've had a lot on my plate and have
been putting it off. I should be able to generate some files smaller than
70 Mb that will work with RawTools. Probably in the 10-20 Mb range. I'll
look into it again today.
Kevin
On Mon, Mar 25, 2019, 5:54 AM Carlos<EMAIL_ADDRESS>wrote:

The delay with the tests is mainly due to the work necessary to obtain a
.raw file as small as possible to be included into the tool for executing
tests.
Some days ago I got a .raw file from my lab of about 70mb, which is about
7mb when converted to mgf. Are these sizes (about 80mb in total) valid to
be included into the tool @bgruening https://github.com/bgruening ?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/galaxyproteomics/tools-galaxyp/pull/355#issuecomment-476184030,
or mute the thread
https://github.com/notifications/unsubscribe-auth/APT50PSq-Ghr1uEeBJE2Dd1lnigtZG2Iks5vaMcdgaJpZM4beeMP
.

@CarlosHorro :( 70MB are something like 69MB too much. We try to keep test data really small to not waste the space on github, the toolshed and all our deployments.
I guess there is no other way around? If so we will hack this into travis. We will download the big test data from somewhere during testing ... huge hack but hopefully just for 1-2 tools.

@CarlosHorro :( 70MB are something like 69MB too much. We try to keep test data really small to not waste the space on github, the toolshed and all our deployments.
I guess there is no other way around? If so we will hack this into travis. We will download the big test data from somewhere during testing ... huge hack but hopefully just for 1-2 tools.

Hi,
@kevinkovalchik has done a great job and got a 1mb raw file which seems to work with RawTools :-)

Awesome!!! Let me know when you need a final review!

Hi,
everything seems to be ready and working, apart from what I asked in
https://github.com/galaxyproteomics/tools-galaxyp/pull/355#issuecomment-476165225
related to the possibility of managing more than 1 file per execution.

can you try pattern="(?P<designation>.+)\.mgf". Does that work?

can you try pattern="(?P<designation>.+)\.mgf". Does that work?

Thank you very much @bgruening ! It has work magically! :-)

It is quite weird, the error executing test 2 :
JobOutputsError: 'ascii' codec can't encode character u'\ufeff' in position 78: ordinal not in range(128)
has been happening in my local for many days, but it did not happen here until now (I just supposed I had something misconfigured in my planemo)... I have not been able to find out how to solve it...

The Unicode character U+FEFF is the byte order mark, or BOM, and is used to tell the difference between big- and little-endian UTF-16 encoding. mh ...

Mh ... one other guess, can it be that on stderr or stdout is some UTF symbol or anything like that only if you run the tool with specific settings ...

Mh ... one other guess, can it be that on stderr or stdout is some UTF symbol or anything like that only if you run the tool with specific settings ...

I don't know... whatever it is , it seems to be related to the test platform as the mgf generation works well (with the same options) using the tool interface. I don't see either any suspicious character in the stderr or stdout when executing it...
I've also being testing some things I've seen across internet like:
#import sys
#reload(sys)
#sys.setdefaultencoding('utf-8')
#set $myencoding = sys.getdefaultencoding()
echo "myencoding: " &&
echo $myencoding &&

By default it is "ascii" and I wanted to try changing it to utf-8, but it seems something fails when setting the default encoding because all tests return empty collections:
Failed to find identifier [really_small] for testing, tool generated collection elements [[]]

As I don`t find any way to avoid this problem (which seems to be specific of the testing functionality), I think I am going to remove the second test.

@CarlosHorro the error is actually a valid one just cryptic.
I only get these results:
bag@bag:~/projects/code/tools-galaxyp/tools/rawtools$ grep PEPMASS /tmp/tmpc6clwG/job_working_directory/000/2/working/output/really_small.raw.mgf
PEPMASS=675,248779296875
PEPMASS=407,919311523438
PEPMASS=436,341217041016
PEPMASS=594,222717285156
PEPMASS=511,47119140625
PEPMASS=834,349731445313
PEPMASS=741,7568359375
PEPMASS=899,80419921875
PEPMASS=667,705932617188
PEPMASS=860,806823730469
PEPMASS=905,804077148438
PEPMASS=791,7453928975
PEPMASS=1024,85485839844
PEPMASS=661,706604003906
PEPMASS=1088,44177246094
PEPMASS=915,798645019531
PEPMASS=1018,85278320313
PEPMASS=786,755065917969
PEPMASS=965,25830078125
PEPMASS=853,323913574219
PEPMASS=1137,90335188188
PEPMASS=887,262299635781
PEPMASS=803,747468092812
PEPMASS=1056,93298339844
PEPMASS=623,21337890625
PEPMASS=694,163146972656
PEPMASS=678,697814941406
PEPMASS=630,706604003906
PEPMASS=979,858063795937
PEPMASS=686,680358886719
PEPMASS=805,729125976563
PEPMASS=960,893493652344
PEPMASS=824,2004710225
PEPMASS=1072,44873046875

So your asserts are being false. Also please note the , instead of a .. So your tools seems to take the locale settings into account, which is not good I think.
I tried adding LC_ALL=C to the tool, but this did not work.

Ah. @CarlosHorro are you using v1.4.1? In 1.4.2 we made a change to ignore local culture settings, to prevent this exact issue.

https://github.com/bioconda/bioconda-recipes/pull/14335

@CarlosHorro I think you probably need to adopt the asserts.
@kevinkovalchik thanks!!!

Thank you very much! :-)  I was quite lost with this last problem but it must work now :-)

Awesome!

@CarlosHorro, @kevinkovalchik: https://github.com/usegalaxy-eu/usegalaxy-eu-tools/pull/171

Hi @bgruening ,
Something weird is happening when installing Rawtools from galaxyp. The installation of the tool in Galaxy seems to go well, but running it just returns empty collections or execution never stops.
When looking into the package details, I see the dependency has not been properly managed:
Dependency | Version | Resolver | Exact version | Current Installation Status |
rawtools | None | None | True |
ie, it does not detect even we manage a Conda dependency, but the installation status says "True".
And, when executing the tool, I see this in the logs:
galaxy.jobs.output_checker INFO 2019-04-05 13:47:52,250 Job 822: Fatal error: Exit code 127 ()
galaxy.jobs.output_checker DEBUG 2019-04-05 13:47:52,250 Tool exit code indicates an error, failing job.

Do you know which could be the reason for not detecting the Conda dependency?
Greetings,
Carlos

Well I think I found it:
galaxy.tools.deps.conda_util DEBUG 2019-04-05 14:08:04,725 Executing command: /home/parallels/GitHub/galaxy/database/dependencies/_conda/bin/conda create -y --override-channels --channel iuc --channel bioconda --channel conda-forge --channel defaults --channel r --name<EMAIL_ADDRESS>rawtools=1.4.2
Fetching package metadata .................
Solving package specifications:

PackageNotFoundError: Packages missing in current channels:

  - rawtools 1.4.2* -> mono >=5.0.0 -> libgcc-ng >=7.3.0
  - rawtools 1.4.2* -> mono >=5.0.0 -> libstdcxx-ng >=7.3.0

We have searched for the packages in the following channels:

  - https://conda.anaconda.org/iuc/linux-64
  - https://conda.anaconda.org/iuc/noarch
  - https://conda.anaconda.org/bioconda/linux-64
  - https://conda.anaconda.org/bioconda/noarch
  - https://conda.anaconda.org/conda-forge/linux-64
  - https://conda.anaconda.org/conda-forge/noarch
  - https://repo.continuum.io/pkgs/free/linux-64
  - https://repo.continuum.io/pkgs/free/noarch
  - https://repo.continuum.io/pkgs/r/linux-64
  - https://repo.continuum.io/pkgs/r/noarch
  - https://repo.continuum.io/pkgs/pro/linux-64
  - https://repo.continuum.io/pkgs/pro/noarch
  - https://conda.anaconda.org/r/linux-64
  - https://conda.anaconda.org/r/noarch

galaxy.tools.deps.resolvers.conda DEBUG 2019-04-05 14:08:15,680 Removing failed conda install of [CondaTarget[rawtools,version=1.4.2]]

I think your channel order is incorrect. Which Galaxy version is that?
It's wired since it worked here and was installed correctly?

I think your channel order is incorrect. Which Galaxy version is that?
It's wired since it worked here and was installed correctly?

It`s 17.09

Uha ... can you try to update your conda version? What is your conda version.

Uha ... can you try to update your conda version? What is your conda version.

Yep I am working on that... actual one is 4.5.12...

After updating conda and stablishing conda channels like here:
https://github.com/bioconda/bioconda-recipes/blob/master/config.yml
it did not work either, and I had to add "anaconda" channel like this:
conda_ensure_channels = conda-forge,bioconda,defaults,r,anaconda
in order it properly gets both packages from the anaconda channel :-)
Thank you very much,
Carlos

wow, that is wired, anaconda == defaults ... but glad that it is working ... But hey ... update!!!! :)
Carlos ... it would be great if you could join us at https://galaxyproject.org/events/gcc2019/!

wow, that is wired, anaconda == defaults ... but glad that it is working ... But hey ... update!!!! :)
Carlos ... it would be great if you could join us at https://galaxyproject.org/events/gcc2019/!

I can assure you I would really like to update our version hehe, but it's quite complicated until we finish the current development we are working on...
I would also love to go to the gcc but I'm afraid it's currently impossible... there will be a new member in my family very soon ;-) and I'm going to be extra-busy for the next months :-)


Rain gauge

Add rain gauge

Something went wrong with syncing the fork, will make a new pull request

I am amazed by the quantity of sensors you have :)

Haha, well that's about it I think ;)

@gerard33 which script do you use in Domoticz to get rain values - bit of topic but nice to know ;)

@Sneeuwtijger I am using this device https://www.weerstationkopen.nl/accessoires-weerstations/homewizard/homewizard-regenmeter-ts906.
And I also have 2 rain gauges with data from Weather Underground and Forecast.io.

@gerard33 thx for the info - I am using the rain data from Weather Underground, have to check it to the real conditions, it has not rained when I was awake after I added the widget to the dashboard ;)

@Sneeuwtijger, check also issue #54. I have to wait for some rain as well to see if it's working correctly :)

@gerard33 type, saw #54 later - will put info there if available


Removed default configuration of deprecated rules.

Fixes #410.
I could have set all deprecated rules to "false".  But instead, I have dropped them completely.  I did not want to interfere with any rule configurations that the user might have customized elsewhere.

Thank you for the fix! :)
Well, it would be nice to have a release, because we would be able to get rid of the warnings in our build.  It would fix #410 basically.
Are you planning to release 5.0.3 soon?

Is there any news about 5.0.3 release ? I'm waiting for version 5.0.3 ~ 😃

me too :-) I notice I have the following being displayed. I presume that this commit fixes that - right ?
Warning: no-duplicate-case rule is deprecated. Replace your usage with the TSLint no-duplicate-switch-case rule.
Warning: no-stateless-class rule is deprecated. Replace your usage with the TSLint no-unnecessary-class rule.
Warning: no-var-self rule is deprecated. Replace your usage with the TSLint no-this-assignment rule.
Warning: valid-typeof rule is deprecated. Replace your usage with the TSLint typeof-compare rule.

@vreality64 and @appsolutegeek: Version 5.0.3 has been released.  And you are right, this commit (and the associated release) fixes the warnings you are seeing.


Import node into current document to prevent WrongDocumentError in IE

I stumbled upon this error in Edge and IE 11. I found a solution for it in the following pull request https://github.com/iconic/SVGInjector/pull/46/commits/860a89dd3c7b40ce9b20d3cb390f57e93d273baa. More infos about the fix can be found in the following stackoverflow thread http://stackoverflow.com/questions/7981100/how-do-i-dynamically-insert-an-svg-image-into-html#7986519

@davidspiess good work, thanks!

@davidspiess published as 0.0.23


Update the names of release packages

For 3.13 and later, the names of the binary packages we produce has changed.
Fixes dafny-lang/dafny#3653.

Is there any way to test this ahead of time?

I've tested locally that it correctly downloads the latest nightly build. I can't test it with the 3.13 or 4.0 releases yet, of course. :)


rocksdb: optimize sub compaction

This is a cherry-pick from pingcap/rocksdb/release.

Do you edit it manually or make update-rocksdb?


chore: remove autoImplicit in NeZero.lean

I was unaware there was some of these still lying around - should we tidy these up?

I was unaware there was some of these still lying around

I didn't make any attempt to clean them up; only to mark the files that used the feature

should we tidy these up?

I think in places like this where it's just a single declaration that uses autoImplicits, its a good idea. Files that use them throughout (where it's clearly a conscious decision) should probably be left as is.
bors d+

bors r+


Deprecation warnings in SQLalchemy

=========================================================== warnings summary ===========================================================
tests/app_tests/api/test_routes.py::test_save_editor_data[query0-data0-400-expected_response0]
  /usr/local/lib/python3.7/site-packages/sqlalchemy/orm/relationships.py:1917: SAWarning: Setting backref / back_populates on relationship RepositoryFileComments.commit to refer to viewonly relationship VulnerabilityGitCommits.comments will be deprecated in SQLAlchemy 1.4, and will be disallowed in a future release.  viewonly relationships should not be mutated (this warning may be suppressed after 10 occurrences)
    (self, other),

tests/app_tests/api/test_routes.py::test_save_editor_data[query0-data0-400-expected_response0]
  /usr/local/lib/python3.7/site-packages/sqlalchemy/orm/relationships.py:1917: SAWarning: Setting backref / back_populates on relationship VulnerabilityGitCommits.comments to refer to viewonly relationship RepositoryFileComments.commit will be deprecated in SQLAlchemy 1.4, and will be disallowed in a future release.  viewonly relationships should not be mutated (this warning may be suppressed after 10 occurrences)
    (self, other),

tests/app_tests/api/test_routes.py::test_save_editor_data[query0-data0-400-expected_response0]
  /usr/local/lib/python3.7/site-packages/sqlalchemy/orm/relationships.py:1917: SAWarning: Setting backref / back_populates on relationship RepositoryFileMarkers.commit to refer to viewonly relationship VulnerabilityGitCommits.markers will be deprecated in SQLAlchemy 1.4, and will be disallowed in a future release.  viewonly relationships should not be mutated (this warning may be suppressed after 10 occurrences)
    (self, other),

tests/app_tests/api/test_routes.py::test_save_editor_data[query0-data0-400-expected_response0]
  /usr/local/lib/python3.7/site-packages/sqlalchemy/orm/relationships.py:1917: SAWarning: Setting backref / back_populates on relationship VulnerabilityGitCommits.markers to refer to viewonly relationship RepositoryFileMarkers.commit will be deprecated in SQLAlchemy 1.4, and will be disallowed in a future release.  viewonly relationships should not be mutated (this warning may be suppressed after 10 occurrences)
    (self, other),

-- Docs: https://docs.pytest.org/en/latest/warnings.html

Solved by a689c063914ae422346aa77bde1f92382ee4c202.


Connect replications to /metrics

We want our /metrics output to include the following information per replication:

Current size (or maybe fullness %?)
Error codes returned by the remote (maybe as a histogram?)
Bytes sent (maybe per time unit?)
Bytes dropped (per time unit?)

Lines of line protocol would be nice in addition to bytes sizing.


Deprecated code in examples

As I'm following the examples on the examples page, I noticed that the sample code tosses deprecation warning. Should the examples be updated so it no longer shows deprecated code?

Specifically include row(table) with display context passed throws a warning (and refers to using a display() mixin which i cant seem to find?)

It looks like this is no more a problem. Examples looks good to me.
The display mixin mentionned should be display-context. PR #365 address it.


Add device tests for ex14 and ex14p

Resolves #4308

I think in this form the tests that will be run are:
./ex14 -no-vis -d cuda
mpirun -np 4 ./ex14p -no-vis -d cuda

Instead of this, we probably want something more in line with the device sample runs in the source files -- e.g. enable PA and make sure we are not running big problems with long runtimes.

Is there an example of something similar, e.g. for ex1?

Is there an example of something similar, e.g. for ex1?

In the makefile, the sample runs for the specific device examples, ex14 and ex14p, can be specialized by defining the targets ex14-test-seq-{cuda,hip} and ex14p-test-par-{cuda,hip} to add more options to the default rules for all device examples: https://github.com/mfem/mfem/blob/7c296d00d8a770e9f569e4db9d7e6d415902a886/examples/makefile#L119-L126
So far, we've only had specializations for non-device tests, e.g. here: https://github.com/mfem/mfem/blob/7c296d00d8a770e9f569e4db9d7e6d415902a886/examples/makefile#L137-L140

For CMake, for non-device tests we define specialized sets of arguments like this: https://github.com/mfem/mfem/blob/7c296d00d8a770e9f569e4db9d7e6d415902a886/examples/CMakeLists.txt#L124-L132
We can use the same approach in the loop for setting up all device tests here: https://github.com/mfem/mfem/blob/7c296d00d8a770e9f569e4db9d7e6d415902a886/examples/CMakeLists.txt#L160-L161

Something like this for the makefile build?
diff --git a/examples/makefile b/examples/makefile
index d84f04ecc..af055004c 100644
--- a/examples/makefile
+++ b/examples/makefile
@@ -138,6 +138,14 @@ ex10-test-seq: ex10
 	@$(call mfem-test,$<,, Serial example,-tf 5)
 ex10p-test-par: ex10p
 	@$(call mfem-test,$<, $(RUN_MPI), Parallel example,-tf 5)
+ex14-test-seq-cuda: ex14
+	@$(call mfem-test,$<,, Serial CUDA example,-r 2 -pa -d cuda)
+ex14p-test-par-cuda: ex14p
+	@$(call mfem-test,$<, $(RUN_MPI), Parallel CUDA example,-rs 2 -rp 0 -pa -d cuda)
+ex14-test-seq-hip: ex14
+	@$(call mfem-test,$<,, Serial HIP example,-r 2 -pa -d hip)
+ex14p-test-par-hip: ex14p
+	@$(call mfem-test,$<, $(RUN_MPI), Parallel HIP example,-rs 2 -rp 0 -pa -d hip)
 ex15-test-seq: ex15
 	@$(call mfem-test,$<,, Serial example,-e 1)
 ex15p-test-par: ex15p

Something like this for the makefile build?

These look good to me -- same as one of the device sample runs in the source files but without the -o 3.

OK, I added something like that in 14b1c27dc.

This PR is now under review (see the table in the PR description). To help with the review process, please do not force push to the branch.

Merged in next for testing...


Should bang ignore be consider.

I would be nice to have suppose for bang ignore locations.
sass(['wwwroot/**/*.sass', '!wwwroot/lib/**/*.sass'], config)
	.on('error', sass.logError)
	.pipe(gulp.dest(destination));

That's being removed from the glob package (which this relies on) so I
decided not to include here. If glob introduces a way to exclude files I'll
make it available here too.
On Saturday, October 3, 2015, roydukkey<EMAIL_ADDRESS>wrote:

I would be nice to have suppose for bang ignore locations.
sass(['wwwroot//*.sass', '!wwwroot/lib//*.sass'], config)
.on('error', sass.logError)
.pipe(gulp.dest(destination));
—
Reply to this email directly or view it on GitHub
https://github.com/sindresorhus/gulp-ruby-sass/issues/269.

--
Rob Wierzbowski
@robwierzbowski http://twitter.com/#!/robwierzbowski
http://github.com/robwierzbowski
http://robwierzbowski.com

@robwierzbowski Just use globby instead of node-glob. globby supports the ignore bang.

I'll take a look at that and at the node-glob/minimatch exclude option.
Leaving this minimatch comment here, I think where I first thought that ! excludes were going to be deprecated: https://github.com/isaacs/minimatch/issues/65#issuecomment-118652817

@robwierzbowski It is. In globby we convert ! to the node-glob ignore option, so it's just as performant as node-glob.

Awesome! Will do.
On Saturday, October 3, 2015, Sindre Sorhus<EMAIL_ADDRESS>wrote:

@robwierzbowski https://github.com/robwierzbowski It is. In globby we
convert ! to the node-glob ignore option, so it's just as performant as
node-glob.
—
Reply to this email directly or view it on GitHub
https://github.com/sindresorhus/gulp-ruby-sass/issues/269#issuecomment-145254674
.

--
Rob Wierzbowski
@robwierzbowski http://twitter.com/#!/robwierzbowski
http://github.com/robwierzbowski
http://robwierzbowski.com

Took a look at this today. It's a little more complicated than it seems because of the way I'm getting the  base paths for the files from each individual source. I'll still try to implement when I get an extra day to work on the plugin.

Hey, sorry, this project is now deprecated because Ruby Sass is deprecated.
See: https://github.com/sindresorhus/gulp-ruby-sass#deprecated


IndexSizeError in Internet Explorer 11

The jQuery.jeyframes.min.js throws an IndexSizeError when running in IE11. Is this a known issue or am I the only one experiencing this.

it seems to occur here:
var ruleIndex = Keyframes.sheet.insertRule(css);

@justferfernandez any chance you could produce an example on codepen or jsfiddle? I would really appreciate it.

Hi! I'm just checking back if there's anything new here. :) Thank you!

Hi, I have the same problem with IE11 IndexSizeError. I'm using jquery 3.3.1 and seems it has a problem with jquery SEC7131: Security of a sandboxed iframe is potentially compromised by allowing script and same origin access.
SCRIPT5022: IndexSizeError
jquery-3.3.1.min.js (3827,3)
SEC7131: Security of a sandboxed iframe is potentially compromised by allowing script and same origin access.

Hi! I'm just checking back if there's anything new here. :) Thank you!

Hi, I used this script https://raw.githubusercontent.com/Keyframes/jQuery.Keyframes/ie1011fix/jquery.keyframes.js and for me is working on ie 11

Thanks! This really helps! :D

Well, I investigated a bit.
There are two things that need to be done to get this running in Internet Explorer 10 and 11:

Some polyfill for Object.assign(), see:

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/assign#Polyfill
since this is using babel maybe add it this way: https://stackoverflow.com/questions/44244468/method-assign-not-supported-in-ie-what-to-do
or this way: https://www.npmjs.com/package/es6-object-assign

insertRule requires adding index although it's optional per specification (see: https://developer.mozilla.org/en-US/docs/Web/API/CSSStyleSheet/insertRule#Syntax), so just replace
var ruleIndex = Keyframes.sheet.insertRule(css);
with 0 as default value for index so it becomes
var ruleIndex = Keyframes.sheet.insertRule(css, 0);
in the generate function.

These two things need to be done in the other repo.
@krazyjakee Should I create some pull request or are you against polyfills or how to get this into here for the benefit of others?

Sorry for the lurking, life is majorly in the way right now. I am not against a polyfill and would follow up with any pull request. Thanks.

This solved it as far as I can see: https://github.com/Keyframes/Keyframes/pull/4/commits/c1fea19c910b178687bd0bfd91b6e0175fde42dd

@su-ex quick question, why es6-object-assign/auto over @babel/plugin-transform-object-assign?

Because I didn't get it to run with babel in the final dist version of jQuery.Keyframes (it always used the src probably on include, even if I added the babel in this repo and built it). The dist version of the Keyframes core were ok with it.

@su-ex ok, nice catch!

Yeah, finally, after two days I finally got my websites running in an acceptable way in Internet Explorer, too. -.-

Ok, this is deployed in 1.0.6 https://www.npmjs.com/package/jquerykeyframes
Please let me know if this has solved this issue and we can close it.

Tested with the deployed version and seems working.

awesome!


Add support for storing isolated bloqs, like GlobalPhaseGate, in a CompositeBloq

Fixes https://github.com/quantumlib/Qualtran/issues/422
Does the following:

Updates CompositeBloq to store both nodes and edges of the compute graph instead of only storing the nodes
Updates cbloq_to_quimb to add logic for adding isolated bloqs to the tensor networks
Updates CirqGateAsBloq to have an empty signature when cirq.num_qubits(self.gate) == 0.
Updates cirq_optree_to_cbloq to correctly operations acting on 0 qubits.
Tests the entire end to end workflow by wrapping cirq.GlobalPhaseGate in a bloq and using tensor contraction to verify that the resulting unitary is identical to the expected cirq unitary.

This is needed for porting over hubbard model implementation from Cirq-FT to Qualtran.

What are your concerns? Why do you think we should not support scalars as Bloqs?
Also, we can not just drop the global phase, so we need to come up with an alternative solution if you think this is not the right way. Do you have alternative proposals in mind?

Some historical discussion:
https://github.com/quantumlib/Cirq/pull/4697
https://github.com/quantumlib/Cirq/pull/1743

I don't know what you mean by "drop the global phase". Every bloq and composite bloq currently has a well-defined global phase.

I don't know what you mean by "drop the global phase". Every bloq and composite bloq currently has a well-defined global phase.

Okay, let me try to explain this differently. The use case is that often times it is possible that I want to implement a unitary U but, using the existing primitives, I have a way of implementing a unitary Q where Q and U are equal up to global phase. Having access to a global phase gate let's me define a decomposition of U in terms of simpler primitives + GlobalPhaseGate; which is convenient. It is important to ensure that the decomposition has the same global phase as U, or else the auto generated decomposition for a controlled invocation of U will result in a wrong unitary.
Not having the ability to add GlobalPhaseGate would make it annoying for us to find ways to shoehorn this global phase into one of the other top level bloqs used in the decomposition; or use an additional qubit in |1> state and apply a Z gate to it -- both of these are wasteful / inconvenient.

I want to implement a unitary U but, using the existing primitives, I have a way of implementing a unitary Q where Q and U are equal up to global phase.

Ok, I'm open to this. We have other bookkeeping bloqs.
I suspect this is still going to cause breakages elsewhere, but I guess we'll cross that bridge when we get there.
Does the drawing code work?

I suspect this is still going to cause breakages elsewhere, but I guess we'll cross that bridge when we get there.

Yeah, the good thing is that all the tests pass and drawing code works (see below) and the quimb simulation works (as shown in the test). For whatever is remaining, we'll fix it as we discover it.

Does the drawing code work? -

Yep!


ISSUE #1700 (solved) Added new project

This pull request introduces a Voice-Controlled Flashcards Quiz application using the Web Speech API. The quiz allows users to answer questions verbally and provides real-time feedback, creating a hands-free and interactive learning experience.

fix #1700


ProVerif goal: Noise protocol example extraction

In a branch of hacspec/specs there's a hacspec version of the KKpsk0 noise protocol pattern that @karthikbhargavan was working on. It's already pretty much in the code style we would expect the ProVerif backend to be able to handle, which makes it an ideal intermediate goal before moving from toy example to more complicated code bases.

[x] Sanity: extraction works
[x] Translate early returns and ? to failures
Separate Protocol flow from party internals
[ ] generate top-level process; easier in this case since there always two parties (#426)

We split the remaining work into the following subtasks:

[x] #446
[ ] #447
[ ] #448
[ ] #449
[ ] #450

Closed in favour of the sub-issues in https://github.com/hacspec/hax/issues/414#issue-2063953621


🐛 Mobile | Add bicep script for Azure Maps

What triggered this change? (PBI link, Email Subject, conversation + reason, etc)

✏ #791

What was changed?

✏ Created a bicep script which would create an Azure Maps account and save the key to key vault

Did you do pair or mob programming?

✏ No

Can we also add this reference where required? (Referencing the KV Secret in the API Appsettings)?


TreeView.message doesn't refresh correctly when cleared

vscode 1.41.0
In my extension, if the tree is unpopulated I set a message to encourage the user what to do. If I add an item to the tree and then clear the message it is removed correctly.
The problem is when I then remove the item from the tree and reset the message, the message doesn't appear.
I have tried setting the message to "" and also undefined.

@Gruntfuggly since you have an extension can you point me to the repository with steps so I can clone and reproduce the issue myself?

Hi - I haven't published it in the store yet, but you can find it here:
https://github.com/Gruntfuggly/remembrall
The master branch should exhibit the behaviour.
Line 121 of extension.js is where the message is set. If I change it to a non-empty string when the tree has content it works as expected. With an empty string or undefined, it doesn't.

Nice bug, thanks for filing and providing a repo! The issue was that if you remove the message and then put back the same message from before you removed then we wouldn't correctly show the message.


Separate productization process

The product builds of SmallRye are currently performed as part of Thorntail product work.
We need to separate it into its own set of builds so that we can spin new versions at any time for Thorntail, EAP, or Quarkus.
We also need to document the process for everyone

I believe this was sufficiently addressed, though there are some improvements still planned/ongoing. Let's close this -- if anyone disagrees, we can easily reopen.
Also would like to honourably mention that @jmartisk has done a lot of work there! :-)


pip-compile expands environment variables

Environment variables get expanded by pip-compile. This is especially unwanted when the variable describes a password or local path.
Environment Versions

Ubuntu 18.04
Python version: 3.7.4
pip version:  19.3.1
pip-tools version: 4.2.0

Steps to replicate
requirements.in
-e ${MY_PROJECT_DIR}/myapp
-e<EMAIL_ADDRESS>
Expected result
requirements.txt
-e ${MY_PROJECT_DIR}/myapp
-e<EMAIL_ADDRESS>
Actual result
The requirements.txt contains the expanded variables.

Hello @cbows,
Thanks for the issue! How it works on pip side:

pip expands the variables during parse a requirements file
substitutes variables by values on the fly

Thus, there is no way to distinguish whether the credentials come from env vars or not using pip API. Possible solutions could be:

match lines in requirements.in with parsed URL, and inject there variables
add some heuristics: guess whether the credentials come from env by searching env variables by values

Don't like either of them, though, and I'd be intuitively against implementing such things outside of pip API.
Consider the alternatives:

ssh keys
netrc
a kung-fu with GIT_ASKPASS, see gitcredentials

Hello @atugushev,
I agree there is no clean solution applicable outside of pip. I guess for credentials there are, as you mentioned, ways around it, and they are probably better.
This still leaves us with local paths though;
Let's say you want to specify a base directory for some app -e ${MY_PROJECT_DIR}/myapp that varies from machine to machine (could be $HOME for development). This gives you the full path in requirements.txt, which is not distributable.

@cbows

Maybe there is a way to parse only file:// urls and see if they come from env var?
Heuristics could be to take a diff of the expanded and original line in requirements and see if the diff matches exactly a value in environment?

Maybe... It could be quite tricky, I imagine, and I think we need to fix #204 first 😊

@cbows Once #204 is fixed (there is currently a PR pending review), you could avoid using environment variables for local paths by using symlinks and relative paths, like
-e ./myapp

Where myapp is a link to wherever.


Allow editing of Node labels for GKE clusters

Is your feature request related to a problem? Please describe.
With changes done in https://github.com/rancher/gke-operator/issues/85, user must be now be allowed to edit Node labels on GKE cluster while performing Edit Cluster

Issue persists on v2.9.2


make docker daemon initiated container (re)start work properly

The docker daemon can decide to (re)start containers, e.g. after a reboot or when a --restart policy was specified for the container.
We want this to work properly for a) the weave container and any similar weave infrastructure containers (such as weavedns), b) application containers. In the latter case this would entail re-attaching to the weave network, and re-registering with weavedns.
This does require bits of #230/#312/#117.

It would be nice if the container came back with the same IP address after a restart, as described in #1047.

Presumably any containers started via the proxy will hang in weavewait when restarted by docker.
The proxy, or a separate container, could take care of re-attaching containers. This won't work on docker restart though, since the proxy may not be running yet. Though I guess we could scan all containers on proxy startup and attach any that are weavewait-ing.

This won't work on docker restart though, since the proxy may not be running yet

The Docker Event mechanism allows clients to fetch events from before the time they were started, so this is not an issue.
Doing that brings a different issue: what if the proxy itself is restarted, and queries for past events, and thus gets told of the start of containers that it already attached to weave (before the proxy was restarted)?
Maybe we can rely on attachment being idempotent, or maybe we can check before attaching.

The fix in #1210 does not do (a) in the original description - dealing with restart of weave itself.


Update Comments in Database, Single Comment Crash, No comment crash

Update the state of the comment table when saving and viewing edits.

When only a single comment is made, when viewing the edits again the page crashes due to the "split".
When no comments are made but an overall comment is and is submitted it crashes since the code is expecting comments.

Commentviewer, if edit are made and saved but not submitted, when the article is reopened all the original comments are there still visible.

Commenteditored, multi line comments delete selected text.


Lint with tern & vim

CodeMirror, Eclipse, Emacs and Atom have implemented marker to support JavaScript validation (static type checker like http://flowtype.org/) with tern-lint . If you are interested you could implement marker with vim to use the tern plugin lint.
Don't hesitate to contact me if you need some info about the tern lint.

I know you're doing a Tern linter, and that's great, but I really don't want to get involved with that.


tdmpc2 struggles to learn very simply Mujoco env: Inverted-Pendulum

I am trying to train TDMPC-2 as a benchmark on my modified JAX-based differentiable Mujoco Environment (MJX). To verify that my environment implementation is correct, I am trying to train TDMPC-2 on a very simple MJX environment: Inverted-Pendulum. However, for some reason, it cannot learn well and perform poorly despite the environment's simplicity.
Specifically, after seed pretraining, the model learned decently well after a few steps, reaching a score of 200/1000 after ~30k steps. However, its performance severely dropped afterward and forever stuck at an average reward of 5-10. I have tried training PPO in my implemented environment, and it had no trouble converging into a good policy. Therefore, I am quite confident that the issue TDMPC-2 is having is not because of the environment implementation.
Could you let me know if you encounter the same issue for the Mujoco Inverted Pendulum environment? My environment is identical to the original Mujoco environment in terms of observation space, action space, reward function, and termination conditions. The only key difference is the environment differentiability, which I did not use for training TDMPC-2.

Hi @anh-nn01, thanks for reaching out! I don't know exactly what the characteristics are for this task, but a couple of things that could make learning tricky: (1) episode length and discount factor (this may or may not be different from your PPO implementation), (2) action repeat (off-policy algos often require more temporal abstraction than on-policy algos), and (3) make sure to either handle early termination appropriately or disable it (our main branch does not consider termination signals, but branch episodic-rl does).

Hi @nicklashansen , thank you so much for your response! I have resolved the issue by (1) clipping the range of action space and (2) reducing the simulation timestep. These changes significantly helped stabilize the simulation and made learning easier.


Remove support for entity-signed node registrations

Closes #3661.
TODO:

[x] fix unit tests.
[x] fix e2e tests.

Thanks for the comments, this is now ready for re-review :)


Setup build&release automaton for osrm-backend-dev docker

Have implemented Build Docker - osrm-backend-dev and Release Docker - osrm-backend-dev.
Refer to more details in https://github.com/Telenav/osrm-backend/wiki/Automation-by-GitHub-Actions


feat: parsing rules

@giacomociti you should not have removed your comment. It forced me to investigate and I found that we do not need that transform!
n3 quad defines properties as getters
rdf-canonize had a bug in how it created shallow copy of quads. Newer version does not have this problem any more :)


Need some advice on which models to use.

Hello:
I want to upscale some videos (in MP4 format), most of those videos have low resolution, like: 640x480.  If I want to upscale them to at least 720P (1280x720), then which models I should download?  I am using C# on Windows 10.
I searched the following 3 pages:

https://upscale.wiki/wiki/Model_Database
This page has too many different models, which ones can meet my requirement?
https://nmkd.de/?esrgan-list
I got 404 error:
What you are looking for doesn't exist anymore or has never existed.
Make sure there are no typos in the address.
For discord, I try to know how to find useful information from it now.
Therefore, now I can only look for model from the first web page.
But due to the high number of models available, please advise on which one best fit my needs.
Thanks,

Try them

Correct link is https://nmkd.de/?esrgan

You have not mentioned what you want to achieve so I can't tell you which models are best lol


Dispose not working

I noticed a big memory leak (probably not tied to this actual case, lot of charts and adding new/removing old) and then started to search for probable causes. My code previously used SciChartBuilder.Dispose(), but I suspected it may not work as thought so I changed it to async version of it and now I will get errors.
My code:
    public async void Dispose()
    {
        await _chartBuilder.DisposeAsync();
        GC.SuppressFinalize(this);
    }

Error:
Error: Could not find 'sciChartBlazorJson.unregister' ('unregister' was undefined).
at https://<IP_ADDRESS>/_framework/blazor.webview.js:1:328
at Array.forEach (<anonymous>)
at s.findFunction (https://<IP_ADDRESS>/_framework/blazor.webview.js:1:296)
at S (https://<IP_ADDRESS>/_framework/blazor.webview.js:1:2437)
at https://<IP_ADDRESS>/_framework/blazor.webview.js:1:3325
at new Promise (<anonymous>)
at beginInvokeJSFromDotNet (https://<IP_ADDRESS>/_framework/blazor.webview.js:1:3306)                                                                                at https://<IP_ADDRESS>/_framework/blazor.webview.js:1:40711
at EventTarget.<anonymous> (<anonymous>:7:62)
at EmbeddedBrowserWebView.<anonymous> (<anonymous>:1:40673)
at Microsoft.JSInterop.JSRuntime.InvokeAsync[TValue](Int64 targetInstanceId, String identifier, Object[] args)
at Microsoft.JSInterop.JSRuntimeExtensions.InvokeVoidAsync(IJSRuntime jsRuntime, String identifier, Object[] args)
at SciChartBlazor.SciChartBuilder.DisposeAsync()

Any recommendations what to do/try? Create a new method inside javascript and use that and make PR?

Yeah! I think you're right about not handling disposal correctly. I can take a look at this.
The error you're getting is because the JS file doesn't have an unregister function in it. This could be because npm run build didn't run, but can also be because chrome (and others) have cached the sciChartBlazorJson.js file. you can fix this by hard reloading and emptying the cache.

Looking into this more. That function just doesn't exist!
Add this to Builder.js and you should be good to go!
export async function unregister(element) {     const { sciChartSurface, wasmContext } = resolveContext(element);     sciChartSurface.delete(); }


Torch fixes

This PR fixes two issues with torch behavior on iOS.

Fixes https://github.com/lwansbrough/react-native-camera/issues/123 (can now initialize the component with the torch turned on).
Fixes an issue where the torch is on but then turns itself off as soon as a video recording starts.

Note that this PR seems really big, but that's just because it's based on https://github.com/lwansbrough/react-native-camera/pull/260. The actual changes related to this PR are in the second of the two commits on this branch, so reviewers can just look at that.

I pulled this into v1 and PR'ed back to master here: https://github.com/lwansbrough/react-native-camera/pull/270
👍


Incorrect message: Provided schema version 4 is less than last set version 3.

The message
m_what = "Provided schema version " + std::to_string(old_version) + " is less than last set version " + std::to_string(new_version) + ".";

is not always correct, because it can happen, e.g. when switching branches, that the Realm migration version is already higher than the last in code, where the message displays as follows:
Provided schema version 4 is less than last set version 3.

My CPP skill are absent so I didn't try to fix it myself, but I guess a simple comparison should be enough:
joiner = old_version > new_version ? "greater" : "less"

https://github.com/realm/realm-cocoa/blob/facde33440f2d46ece3e32ca60ff58f2a9028080/Realm/ObjectStore/object_store.cpp#L504

Turns out that the exception is just thrown at one place, so that old_version is always greater than new_version. So I guess that we just swapped old and new version in the error message by accident.


⚠ File access has degraded performance

In 6d507a6, File access (https://fyle.uk/file/fish) experienced degraded performance:

HTTP code: 200
Response time: 5705 ms

Resolved: File access performance has improved in d3cae4a after 20 minutes.


Turnstile ontology - typo in StructualCoverageAnalysis (sic!)

The Turnstile development plan ontology defines an activity called StructualCoverageAnalysis rather than StructuralCoverageAnalysis.

Fixed. Thanks for catching that!


Emoji-with-rolling-eyes-animaiton

Hii I'm a GSSoc22 contributor and I'm looking to make a pr request in this repo regarding my CSS animation project as Square shimmer loading animation.
The idea behind this project:
There is a specific idea behind this CSS animation I'm just looking to play with emoji in real view
How does it work :
while loading the webpage on the browser in the middle of the webpage there will be yellow color emoji and it will roll his eyes from left to right and vice versa while loading.
for more details refers to this image

for animation, view refer to this video
https://user-images.githubusercontent.com/73521123/164494800-557e7ee7-fdb4-4591-a198-97c57f20e72e.mp4

Can we make it more interesting and fun? Like on one side we will have two emojis (girl and boy saying hi) and on other this side we will have this smiley. Whenever user clicks on girl or boy, a bubble will appear telling "Hey Smiley!" and then the smiley will roll eyes. 😂 What say?

@VidhiBhatt01 that's awesome idea, definitely I will work on it  but for not in this issue, as this should be a another animation if I'm not wrong will it be possible to create a sperate issue or please refer me a name for your ideology project 😀.

No need for other issue, add it in this only! Keep the name same only ie. Rolling eye animation

Can we make it more interesting and fun? Like on one side we will have two emojis (girl and boy saying hi) and on other this side we will have this smiley. Whenever user clicks on girl or boy, a bubble will appear telling "Hey Smiley!" and then the smiley will roll eyes. 😂 What say?

🤣🤣

@Kumar-Ankit56 any updates?


Subpackages do not import properly in a clone outside of GOPATH

I have a program with import path go.company.com/team/prog1 which has "subpackages" inside it, committed to its repo as e.g. go.company.com/team/prog1/helper. I've vendored all the external (other repo) dependencies with godep. Now I'm trying to automate the build and test steps in an external build tool, and prog1's dependencies on its own subpackages cause build failures when cloned outside of a GOPATH:
$ unset GOPATH # only here for clarity, it was actually never set in the first place
$ git clone go.company.com/team/prog1
$ cd prog1
$ godep go test ./...

sourcefile.go:13:2: cannot find package "go.company.com/team/prog1/helper" in any of:
    /usr/lib/golang/src/pkg/go.company.com/team/prog1/helper (from $GOROOT)
    /home/user/prog1/Godeps/_workspace/src/go.company.com/team/prog1/helper (from $GOPATH)
...repeated likewise for all the other subpackage imports...

I've worked around this issue by instead doing this:
$ mkdir -p src/go.company.com/team
$ git clone go.company.com/team/prog1 src/go.company.com/team/prog1
$ export GOPATH=`pwd`
$ cd src/go.company.com/team/prog1
$ godep go test ./...

But this feels like an unneccessary and unintended complication, especially since the exact same script works perfectly fine for a program that does not import any of its own "subpackages".
Before godep was around (near Go 1.0), the build script I wrote for another project to do similar things would copy the repo's contents inside the vendoring workspace, which allowed these "subpackages" to work. That doesn't seem reasonable for godep though, since committing your workspace is the intended use case...

godep relies on the go tool and by extension things being compatible with a $GOPATH workspace layout.


Legal disclaimers - Doorway

For Doorway, we need to introduce some new legal language / disclaimers to both Partners and Public.
Notion Ticket
Legal terms copy
Note: Translations for these were added to the document doorway is sharing with their translations vendor.
AC:
Upon creation of an account:

Partners will be required to scroll through and accept terms of use
Applicants will see the terms of use, but not be required to scroll through. They will still need to accept to proceed.

Upon submittal of an Application by an Applicant:

Additional language relating to their age needs to be added + agreed to before application can be submitted.

Upon downloading any applications, or Excel or CMS files with PII, by a Professional Partner or Local Government:

They will need to accept terms of use (listed in doc & designs) before proceeding.

We have a few questions on this open with Barry. I don't think these should stop us from getting started let's discuss!
See the following:
Account creation:

We assumed we would use the Doorway terms of use on the MTC site for this. Is that correct?
Are applicants and partners subject to the same terms of use upon account creation?
Do partners need to scroll through the entirety of the terms of use upon account creation, or is there a particular section we should include in the modal? We noticed that this is quite a bit of text - alternatively, could we link to the terms of use in this modal?

Application Submission for housing seekers:

When a housing seeker submits an application, should they see the same terms of use they see at account creation?

Question

do we need to add a disclosure to users export?

Question for eng:

link to bottom - feasible? EJ: Yes, feasible
aria labels on disabled button (SR should know it's disabled, and then should indicate when it's enabled)

Break into three tickets, one for each term:

Account creation: 2
(assume we will need scrolling mechanism)
Applicant age:  1
Export: 3


File/folder in the recycle bin can be added to other folders in it

Description
I can take a file or folder from the recycle bin and then added it to a folder in there. It doesn't permanently delete them it just adds them like a normal folder would.
Steps To Reproduce
No response
Expected behavior
Moving file/folder into others in the recycle bin should be disabled. External files/folders added should go directly into the root of the recycle bin when dragged and dropped.
Files Version
<IP_ADDRESS>
Windows Version
10.19043.1466
Relevant Assets/Logs
debug.log
debug_fulltrust.log

Why do you interfear with someone learning proper
Handling of coder s purps,
Placing problems on others.
Interupting a person an persons
Life. Your taking soneones files
That have been deleted and
Trying to have those issues
Placed on deleter because its away to make themselves a better person ? Some times
You look at something does not mean you wAnt it downloaded. So delete is nessary to get that issue out of your phone .
You are wrong for placing issues delete back on them .!

what on earth are you on about linnjason678

This issue should be resolved with #9323


how to trigger backdoor mode?

Hello , I needed help about backdoor mode. How to trigger it , should an operator add password="password" on any http/tcp request?
or is there any other technique to send a specific type of packet? thx

Hello,
You can trigger the backdoor mode using a crafter TCP request.
You can use scapy with the following command:
send(IP(dst="<IP_ADDRESS>") / TCP(sport=51337,dport=443,flags="SR",seq=1000) / (bytes(b"password")))


Expose maintenance interval as a blueprint setting for node pools in GKE

Request
Expose host_maintenance_interval as a user configurable setting in the blueprint
Approach
Expose as an optional terraform variable. If passed, we will set this under node_pool.node_config, using a dynamic block. Possible values are PERIODIC and AS_NEEDED
Test plan

[x] Update hpc-gke.yaml example with settings.host_maintenance_interval = PERIODIC. Verified this is reflected in Terraform plan and on gcloud cli
[x] Update hpc-gke.yaml example with settings.host_maintenance_interval = AS_NEEDED. Verified this is reflected in Terraform plan and on gcloud cli
[x] Keep hpc-gke.yaml as is (no settings.host_maintenance_interval key in yaml). Verified host maintenance interval is unset on gcloud cli

Deployment
Merge to develop. No additional action needed as of now
Submission Checklist
Please take the following actions before submitting this pull request.

[x] Fork your PR branch from the Toolkit "develop" branch (not main)
[x] Test all changes with pre-commit in a local branch #
[x] Confirm that "make tests" passes all tests
[ ] Add or modify unit tests to cover code changes
[x] Ensure that unit test coverage remains above 80%
[x] Update all applicable documentation
[x] Follow Cluster Toolkit Contribution guidelines #

I've tested all scenarios (see description for details). Please review @ankitkinra

Added maintenance policy to a daily test. Verified that the changes were reflected in the created node pool (ubuntu)


best way to compose styles

Let's say I want to create a custom function called size. The value passed to it should be applied in the same way tailwind does: sets width and height.
I tried to do this:
pub fn size(size: Size) -> Style(a, b) {
  sketch.class([
    sketch.width(size),
    sketch.height(size),
  ])
  |> sketch.compose()
}

But it ended memoizing the function and always returned the same class on lustre.
Returning a list and spreading/appending is also a possible way. But I did it like this:
pub fn size(size: Size) -> Style(a, b) {
  sketch.dynamic("size-" <> size.to_string(size), [
    sketch.width(size),
    sketch.height(size),
  ])
  |> sketch.compose()
}

// Then we can sketch.class([some_style, size(px(10))])

How bad is it? Is there a better way to do it? I think we should have some compose examples on the docs.

Your second option is the right way to go in your case (if you don't want to change the size of the node in the future)! Unfortunately, due to the fact that I can't patch the lustre renderer (yet? I'm considering this option seriously), a call to class will always compile something static. You should think of class exactly like a class declaration in CSS: immutable. That's why the package exposes a dynamic function, which can render a custom class, which changes every time the render turns, but at the cost of defining an ID, to avoid duplicated classes in the stylesheet.
For your specific problem, I would have define the styles differently:
fn base_style() {
  sketch.class([some_style])
}

pub fn size(size: Size) -> Class {
  sketch.dynamic("size-" <> size.to_string(size), [
    sketch.width(size),
    sketch.height(size),
    sketch.compose(base_style),
  ])
}

// Then you can use html.div([sketch.to_lustre(size(s))], []).
// That will output
//

That way, you inherit from static styling (base_style) and can still modify the size afterwards.

I'll keep it open few days if you're not against it, before doing a meta issue for the documentation


MM-44650_Turn off Use Case Based Onboarding for Self-Managed while A/B Test runs on Cloud

Summary
While the A/B test to hide the Use Case Based Onboarding is running on Cloud the experience should be hidden by default for Self-Managed workspaces.
Ticket Link
https://mattermost.atlassian.net/browse/MM-44650
Release Note
NONE

/e2e-test

Successfully triggered e2e testing!
https://git.internal.mattermost.com/qa/cypress-ui-automation/-/pipelines/201475

/e2e-test

Successfully triggered e2e testing!
https://git.internal.mattermost.com/qa/cypress-ui-automation/-/pipelines/201832

Cherry pick is scheduled.


Content on how to Review a PR

I hope it isn't too meta to review this PR on reviewing a PR
Closes #287

Thanks for your review @dcamron Getting to these points now.

Thanks @ktyle
Just addressed these typos and improvements.

@dcamron will you take one last look at this before we merge!

@dcamron will you take one last look at this before we merge!

Yep, though it might be mid-afternoon today before so. Thanks for the nudge!


Go build error for linux/386 nocgo

Building a Go binary for linux/386 nocgo fails
go build -buildvcs=false -ldflags=-w -o dist/rain-v1.8.2-test3_linux-i386-nocgo/ ./cmd/rain

# github.com/apple/pkl-go/pkl
/go/pkg/mod/github.com/apple/pkl-go@v0.6.0/pkl/values.go:111:29: Second * 60 (constant<PHONE_NUMBER>0 of type DurationUnit) overflows int
/go/pkg/mod/github.com/apple/pkl-go@v0.6.0/pkl/values.go:238:7: Terabytes (untyped int constant<PHONE_NUMBER>000) overflows int
/go/pkg/mod/github.com/apple/pkl-go@v0.6.0/pkl/values.go:240:7: Tebibytes (untyped int constant<PHONE_NUMBER>776) overflows int
/go/pkg/mod/github.com/apple/pkl-go@v0.6.0/pkl/values.go:242:7: Petabytes (untyped int constant<PHONE_NUMBER>000000) overflows int
/go/pkg/mod/github.com/apple/pkl-go@v0.6.0/pkl/values.go:244:7: Pebibytes (untyped int constant<PHONE_NUMBER>842624) overflows int
/go/pkg/mod/github.com/apple/pkl-go@v0.6.0/pkl/values.go:279:10: cannot use Terabytes (untyped int constant<PHONE_NUMBER>000) as DataSizeUnit value in return statement (overflows)
/go/pkg/mod/github.com/apple/pkl-go@v0.6.0/pkl/values.go:281:10: cannot use Tebibytes (untyped int constant<PHONE_NUMBER>776) as DataSizeUnit value in return statement (overflows)
/go/pkg/mod/github.com/apple/pkl-go@v0.6.0/pkl/values.go:283:10: cannot use Petabytes (untyped int constant<PHONE_NUMBER>000000) as DataSizeUnit value in return statement (overflows)
/go/pkg/mod/github.com/apple/pkl-go@v0.6.0/pkl/values.go:285:10: cannot use Pebibytes (untyped int constant<PHONE_NUMBER>842624) as DataSizeUnit value in return statement (overflows)
Error: Process completed with exit code 1.

This was fixed in https://github.com/apple/pkl-go/pull/32.
For now, you should be able to fix your build errors by pinning to the main branch.

Confirmed that installing from main fixes it.


SectionList types do not include ListFooterComponentStyle

Description
SectionListProps type should contain the relevant types from VirtualizedList. The property itself works, but the type definitions are out of sync with the code.
React Native version:
System:
    OS: macOS 10.15.7
    CPU: (12) x64 Intel(R) Core(TM) i9-8950HK CPU @ 2.90GHz
    Memory: 1.11 GB / 32.00 GB
    Shell: 5.7.1 - /bin/zsh
  Binaries:
    Node: 14.0.0 - /usr/local/bin/node
    Yarn: 1.19.1 - /usr/local/bin/yarn
    npm: 6.14.4 - /usr/local/bin/npm
    Watchman: 4.9.0 - /usr/local/bin/watchman
  Managers:
    CocoaPods: 1.9.3 - /Users/cjpitt/.rvm/gems/ruby-2.4.6/bin/pod
  SDKs:
    iOS SDK:
      Platforms: iOS 14.0, DriverKit 19.0, macOS 10.15, tvOS 14.0, watchOS 7.0
    Android SDK:
      API Levels: 23, 26, 27, 28, 29
      Build Tools: 27.0.3, 28.0.0, 28.0.2, 28.0.3, 29.0.2, 29.0.3
      System Images: android-22 | Google APIs Intel x86 Atom, android-24 | Google Play Intel x86 Atom, android-27 | Google APIs Intel x86 Atom, android-28 | Google APIs Intel x86 Atom, android-29 | Google APIs Intel x86 Atom, android-29 | Google Play Intel x86 Atom
      Android NDK: Not Found
  IDEs:
    Android Studio: 4.0 AI-193.69<IP_ADDRESS>14223
    Xcode: 12.0.1/12A7300 - /usr/bin/xcodebuild
  Languages:
    Java: 1.8.0_222 - /usr/bin/javac
    Python: 2.7.16 - /usr/bin/python
  npmPackages:
    @react-native-community/cli: Not Found
    react: 16.13.1 => 16.13.1
    react-native: 0.62.2 => 0.62.2
    react-native-macos: Not Found
  npmGlobalPackages:
    *react-native*: Not Found

Steps To Reproduce
N/A

@cjpete Are the VirtualizedList props not already included in the prop definition here?

@cjpete Are the VirtualizedList props not already included in the prop definition here?


Polygonal editing - bug

When the polygon editor, the current point reaches 500+, it is impossible to drag and drag the point. Check the relevant data. Except for the same point as the first point and the last point, there is no abnormal.

The polygon is not edited as a whole, but you can delete the point above the red box to edit it

@Thunder7991 thanks for raising an issue - I'm not sure I fully understand the first issue you raised. Could you try rephrasing it for me?
The second image you shared shouldn't be possible. Can you share how you achieved that?

The current map environment is mapbox, the current graphic cannot be edited, Its point cannot be moved.
I must delete the abnormal points to make them a regular polygon (without intersecting positions), and the top image is the one I can edit after deleting the abnormal points

When intersecting polygons appear during the drawing process, moving a vertex to release this state reveals that it cannot be moved.


Fix/module name change

[x] The purpose of this PR is explained in this or a referenced issue.
[x] The PR does not update generated files.

These files are managed by the codegen framework at Azure/autorest.go.

[x] Tests are included and/or updated for code changes.
[x] Updates to module CHANGELOG.md are included.
[x] MIT license headers are included in each file.

There should ideally be one commit showing here which should be the last one, right?

Corrected the chageset in #22273


Just leave space in between when vector grows with insert

See https://github.com/cppfastio/fast_io/issues/655


Fix misleading example boxplot and change default

Fixes misleading plot (title says colored by cylinder, but the coloring looks random) https://github.com/ioam/holoviews/issues/2974
Also, changes the default of box and violin's fill color from Cycle() to 'lightgray' because the colors don't really add any value unless there are multiple categories. See https://serialmentor.com/dataviz/color-pitfalls.html Figure 15.3
This has a misleading title

This is correct but looks a bit ugly to me

This looks the cleanest to me

Makes sense to me and thanks for these improvements!


Adding multiple packages from different server

There is currently no way (that I'm aware of) for transferring multiple packages from one server to another.
This is why I created this tool bulk-npm-publish for publishing multiple packages from 1 verdaccio server to another by bulk publishing

Hi, nop, there is no way. Your tool seems to be a solution. I haven’t tried but I will take a look.
Thanks for sharing I will share my thoughts later on 👍

@rluvaton I'm having a hard time understanding how to use it. Could you give me some examples, what to run ? Readme is good but I don't fully understand how to run it.

@rluvaton I'm having a hard time understanding how to use it. Could you give me some examples, what to run? Readme is good but I don't fully understand how to run it.

Steps

I assume you installed the package globally

1 PC

The common use case is that you will get the storage from another PC, we just emulating new storage

Open the verdaccio config file
Change the storage path to new folder (for example ./temp-storage)
Run verdaccio
Install some package
Stop verdaccio
Change back the storage path to the original one
Continue in 2 PCs

2 PCs

Run verdaccio
Open terminal and type bulk-npm-publish
Write at the first question your folder that contains the packages from another PC

if you continued from 1 PC this is the folder you wrote in step 2

Type your publish script path that will be created or just press enter for default
Type the verdaccio registry URL (no need if in the .npmrc file the registry URL is the verdaccio one)
If you want to add only new packages press enter and type your current storage path
Run the batch file that created (it’s in the path you provided in step 4)

I hope it helps 👍🏻

Unfortunately is failing the latest version<EMAIL_ADDRESS>and nvm v13.1.0
➜ bulk-npm-publish
 line 1: use strict: command not found
 line 2: syntax error near unexpected token `('
 line 2: `var __awaiter = (this && this.__awaiter) || function (thisArg, _arguments, P, generator) {'

Unfortunately is failing the latest version<EMAIL_ADDRESS>and nvm v13.1.0
➜ bulk-npm-publish
 line 1: use strict: command not found
 line 2: syntax error near unexpected token `('
 line 2: `var __awaiter = (this && this.__awaiter) || function (thisArg, _arguments, P, generator) {'

Can you please open an issue at the repo so we can further discuss it there, and trying to solve it

Done https://github.com/rluvaton/bulk-npm-publish/issues/33


Issues with llama-2-7b-f

System Info
latest versions for python and langchain
Who can help?
No response
Information

[ ] The official example notebooks/scripts
[ ] My own modified scripts

Related Components

[X] LLMs/Chat Models
[ ] Embedding Models
[ ] Prompts / Prompt Templates / Prompt Selectors
[ ] Output Parsers
[ ] Document Loaders
[ ] Vector Stores / Retrievers
[ ] Memory
[ ] Agents / Agent Executors
[ ] Tools / Toolkits
[X] Chains
[X] Callbacks/Tracing
[ ] Async

Reproduction
I used the demo code to run Sagemaker using Langchain for llama-2-7b-f
However, I'm getting the following issues:
ValueError: Error raised by inference endpoint: An error occurred (ModelError) when calling the InvokeEndpoint operation: Received client error (424) from primary with message "{
 "code":424,
 "message":"prediction failure",
 "error":"string indices must be integers"
}

This is what is shown from AWS Logs:
[INFO ] PyProcess - W-80-model-stdout: [1,0]<stdout>:TypeError: string indices must be integers
How do I resolve this
Expected behavior
Shown:
ValueError: Error raised by inference endpoint: An error occurred (ModelError) when calling the InvokeEndpoint operation: Received client error (424) from primary with message "{
 "code":424,
 "message":"prediction failure",
 "error":"string indices must be integers"
}

@dosu-bot
The issue is that the "string indices must be indices", but your transformation does not deal with that. Here is the curretn code, please see below and change to avoid this error:
from typing import Dict
from langchain import PromptTemplate, SagemakerEndpoint
from langchain.llms.sagemaker_endpoint import LLMContentHandler
from langchain.chains.question_answering import load_qa_chain
import json

example_doc_1 = """
string
"""

docs = [
    Document(
        page_content=example_doc_1,
    )
]

query = """
prompt
"""

prompt_template = """Use the following pieces of context to answer the question at the end.

{context}

Question: {question}
Answer:"""
PROMPT = PromptTemplate(
    template=prompt_template, input_variables=["context", "question"]
)

class ContentHandler(LLMContentHandler):
    content_type = "application/json"
    accepts = "application/json"

    def transform_input(self, prompt: str, model_kwargs: Dict) -> bytes:
        input_dict = {"inputs": prompt, "parameters": model_kwargs}
        return json.dumps(input_dict).encode('utf-8')

    def transform_output(self, output: bytes) -> str:
        response_json = json.loads(output.read().decode("utf-8"))
        print("output: ", response_json[0])
        return response_json[0]["generation"]

content_handler = ContentHandler()

chain = load_qa_chain(
    llm=SagemakerEndpoint(
        endpoint_name="endpointname",
        credentials_profile_name="profilename",
        region_name="us-east-1",
        model_kwargs={"temperature": 1e-10, "max_length":500},
        endpoint_kwargs={"CustomAttributes": "accept_eula=true"},
        content_handler=content_handler,
    ),
    prompt=PROMPT,
)

chain({"input_documents": docs, "question": query}, return_only_outputs=True)```

We are having this issue to. llama-2 works but llama-2 chat does not which is one of the most popular models.
This should be on your automated testing so you know when the integration breaks.
Discussing with AWS and this and other issues are why people are moving away from langchain to just writing the integrations themselves.
Trying to hold on but seems like everyday we are dealing with langchain issues instead of building the app.

@maggarwal25
try changing the Content handler as follows:
def transform_input(self, prompt: str, model_kwargs: Dict) -> bytes:
       def split_sentence(sentence, delimiters):
           for d in delimiters:
               sentence = sentence.replace(d, '|')
           return sentence.split('|')
       delimiters = ["--Question--:","--Answer--:"]
       system, user , _= split_sentence(prompt, delimiters)

       input_dict = {'inputs': [[{'role': 'system',
                     'content': f"{system}Question: \n"},
                    {'role': 'user','content':f'{user}\nAnswer:'}]
                   ],
        'parameters': model_kwargs}

       return json.dumps(input_dict).encode('utf-8')

def transform_output(self, output: bytes) -> str:
     response_json = json.loads(output.read().decode("utf-8"))
     print("output: ", response_json[0])
     return response_json[0]["generation"]["content"]

a bit ugly I suppose.

@maggarwal25 Were you able to solve this issue?

If Anyone facing the same issue, try this
class ContentHandler(LLMContentHandler):
    content_type = "application/json"
    accepts = "application/json"

    def transform_input(self, prompt: str, model_kwargs: dict) -> bytes:
        payload = {
            "inputs": [
                [
                    {"role": "user", "content": prompt},
                ],
                ],
                "parameters": {"max_new_tokens": 1000, "top_p": 0.6, "temperature": 0.1},
        }

        input_str = json.dumps(
            payload,
        )

        return input_str.encode("utf-8")

    def transform_output(self, output: bytes) -> str:
        response_json = json.loads(output.read().decode("utf-8"))
        content = response_json[0]["generation"]["content"]
        return content

@prkhrv , I tried your solution, and I don't get an error, but the content comes back empty
I try the same prompt with the SageMaker SDK, and it works fine. Very strange

@prkhrv , nevermind, the actual issue was setting temperature to "0.0".
I have never seen this behavior with other models, but Llama 2 70b chat was consistently giving me empty responses with that temprature
Your fix works, thanks for posting

@prkhrv I am facing the same issue while trying to use a Llama model hosted on GCP (Vertex AI - Model Garden).
I am using the VertexAIModelGarden module to create the LLM and when calling the LLM for inference the same error is returned.
Do I have also to create a ContentHandler class? And how can I make the LLM use that class?


Support umbrella apps

Fix root path detection for umbrella apps, plus two small cleanups.

This is awesome! 🎉
If you have time to add both Elixir versions (1.7 and 1.8) to .travis.yml so they are both officially tested, I’ll merge it right after.
If you don’t have time, no worries, I’ll merge it right now and edit .travis.yml afterwards.
Thank you!

This is great, thank you Tomáš! I’ll cut the 0.2.0 right after this.

Thanks for merging.
BTW, I'm new to Elixir and its community and I've been wondering if merging via squashing is the standard here because this is my second Elixir-related PR that was merged in that way and it's making me somewhat uncomfortable as there's now a squashed commit authored by me in some git history somewhere and I always strived to make proper commits that don't mix unrelated changes... But if this is standard in the Elixir community, I can live with that, I guess. :-)

BTW, I'm new to Elixir and its community and I've been wondering if merging via squashing is the standard here because this is my second Elixir-related PR that was merged in that way

I think it’s just a coincidence :)

It's making me somewhat uncomfortable as there's now a squashed commit authored by me in some git history somewhere and I always strived to make proper commits that don't mix unrelated changes...

This is just my personal preference as an open-source project maintainer. I always try to keep a clean git history in the master branch. For instance when you look at the history for this repo, you see a Support umbrella apps commit, not all the individual commits that amounted to the whole feature (eg. refactors from PR comments, minor stuff like formatting, etc.).
Thank you for contributing!

Okay, thanks for clarifying. :-)


Some Premium Question are unable to open

Describe the bug
A clear and concise description of what the bug is.
To Reproduce
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error

Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):

Browser [e.g. chrome, safari]

Additional context
Add any other context about the problem here.

@surbhi498 bro explain it in a new issue. I CANT do anything with it. Explain where, when, how and which of the "some" isn't working. I'm not gonna be able to fix the "some question", without knowing which ones, and when.
Open a new issue, with a detailed issue.


examples

Is it possible to add a readme how you can use this ?

Added README + link to examples

Thank you @JonasMH !


added LockerService detection and warning code.

@dospromptman this adds a simple check for LockerService and displays a warning if its activated.  I tested this under my DE org where I can turn LockerService on and off.

@dospromptman converted addresses in address mgr to objects, to avoid inconsistent handling of namespace prefixing by lightning code.

@dospromptman converted other typed sobjects to also be objects to avoid other namespace prefixing issues.

@dospromptman this commit also now includes changing the Delete Contact functionality to be Remove (from Household).

I'm merging this with the doc string review outstanding, since that can easily be managed in its own PR


Angular Universal error

This is a (multiple allowed):

[x] bug

[ ] enhancement

[ ] feature-discussion (RFC)

Swiper Version: 6.4.5.

Platform/Target and Browser Versions: macOS, Windows, iOS, ANDROID, CHROME

Live Link or JSFiddle/Codepen or website with isssue: PREFERABLY (IF YOU WANT YOUR ISSUE TO BE RESOLVED ASAP).

What you did
Create app and run in SSR mode
Expected Behavior
No render errors
Actual Behavior
Nodejs render error: Cannot read property 'swiperSlideIndex' of undefined

can you provide demo?

can you provide demo?

can you provide demo?

Where I can create to render in SSR mode?

can you provide demo?

Where I can create to render in SSR mode?

Fix will be included in the next release.
@neilhem give an update if it works for you now.

Fix will be included in the next release.
@neilhem give an update if it works for you now.

v6.4.6?

v6.4.6?

@neilhem v6.4.7

@neilhem v6.4.7

Yeah, no more error. Works fine! Thanks

Yeah, no more error. Works fine! Thanks


Rust 2018 macro export pain

We need to add #[macro_export(local_inner_macros)] so in Rust 2018, this syntax works:
use error_chain::error_chain;
see https://github.com/rust-lang/rust/issues/57013

https://github.com/rust-lang-nursery/rust-cookbook/pull/565

This appears to have been resolved in https://github.com/rust-lang-nursery/error-chain/commit/e6d0b2c017e70ae92d52ef833b336705917c0ae3

I was seeing 2018 edition problems in a v 0.11 use case.  Upgrading to 0.12 resolved this problem.