EETQ wheel not building

I'm trying to install EETQ in a Google Colab notebook using the following command: pip install git+https://github.com/NetEase-FuXi/EETQ.git. When installing, all steps seem to work, except that the installation gets stuck at Building wheels for collected packages: EETQ - it's been at this stage for over 10 minutes now. What should I do to fix this?
P.S., the same problem happens when I try to install with the method as described in the readme.

@donjuanpond Installing EETQ takes a long time because the compilation of the cutlass library is slow. You can use python setup.py buildcheck the compilation progress. We provide some precompiled whl packages for direct installation: https://github.com/NetEase-FuXi/EETQ/releases

Hi @SidaZh,
First thing, you are definitely correct about the installation being slow - Colab is saying the installation took 39 minutes and 50 seconds! Secondly, I tried using direct installation from the wheel with pip install --no-cache-dir https://github.com/NetEase-FuXi/EETQ/releases/download/v1.0.0/EETQ-1.0.0+cu121+torch2.1.2-cp310-cp310-linux_x86_64.whl but this threw me an error:
ERROR: Invalid requirement: 'EETQ==1.0.0+cu121+torch2.1.2': Local version label can only be used with `==` or `!=` operators
    EETQ==1.0.0+cu121+torch2.1.2
        ~~~~~~~~~~~~~^

What should I do about this?

My system just updated CUDA to 12.6 and I can no longer compile EEQT.
FAILED: /tmp/pip-req-build-jgyvqov6/build/temp.linux-x86_64-cpython-312/tmp/pip-req-build-jgyvqov6/csrc/cutlass_kernels/fpA_intB_gemm_wrapper.o
…
 /usr/local/cuda/bin/nvcc --generate-dependencies-with-compile --dependency-output /tmp/pip-req-build-jgyvqov6/build/temp.linux-x86_64-cpython-312/tmp/pip-req-build-jgyvqov6/csrc/cutlass_kernels/fpA_intB_gemm_wrapper.o.d -DVERSION_INFO=1.0.0 '-I['"'"'/home/michael/miniconda3/envs/ai_newsletters/lib/python3.12/site-packages/torch/include'"'"', '"'"'/home/michael/miniconda3/envs/ai_newsletters/lib/python3.12/site-packages/torch/include/torch/csrc/api/include'"'"', '"'"'/home/michael/miniconda3/envs/ai_newsletters/lib/python3.12/site-packages/torch/include/TH'"'"', '"'"'/home/michael/miniconda3/envs/ai_newsletters/lib/python3.12/site-packages/torch/include/THC'"'"', '"'"'/usr/local/cuda/include'"'"']' -I/tmp/pip-req-build-jgyvqov6/csrc -I/tmp/pip-req-build-jgyvqov6/csrc/utils -I/tmp/pip-req-build-jgyvqov6/csrc/cutlass/include -I/tmp/pip-req-build-jgyvqov6/csrc/cutlass_kernels/include -I/tmp/pip-req-build-jgyvqov6/csrc/cutlass_extensions/include -I/home/michael/miniconda3/envs/ai_newsletters/lib/python3.12/site-packages/torch/include -I/home/michael/miniconda3/envs/ai_newsletters/lib/python3.12/site-packages/torch/include/torch/csrc/api/include -I/home/michael/miniconda3/envs/ai_newsletters/lib/python3.12/site-packages/torch/include/TH -I/home/michael/miniconda3/envs/ai_newsletters/lib/python3.12/site-packages/torch/include/THC -I/usr/local/cuda/include -I/home/michael/miniconda3/envs/ai_newsletters/include/python3.12 -c -c /tmp/pip-req-build-jgyvqov6/csrc/cutlass_kernels/fpA_intB_gemm_wrapper.cu -o /tmp/pip-req-build-jgyvqov6/build/temp.linux-x86_64-cpython-312/tmp/pip-req-build-jgyvqov6/csrc/cutlass_kernels/fpA_intB_gemm_wrapper.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O3 -std=c++17 -U__CUDA_NO_HALF_OPERATORS__ -U__CUDA_NO_HALF_CONVERSIONS__ -U__CUDA_NO_HALF2_OPERATORS__ -U__CUDA_NO_HALF2_CONVERSIONS__ -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=EETQ -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_70,code=sm_70 -gencode=arch=compute_75,code=sm_75 -gencode=arch=compute_80,code=sm_80 -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86
…

184 | #define NVTX_VERSION 3
            |
      /usr/local/cuda/include/nvToolsExt.h(248): error: invalid redeclaration of type name "nvtxStringHandle_t" (declared at line 292 of /usr/local/cuda/include/nvtx3/nvToolsExt.h)
        typedef struct nvtxStringHandle* nvtxStringHandle_t;
                                         ^

      /usr/local/cuda/include/nvToolsExt.h(258): error: invalid redeclaration of type name "nvtxDomainHandle_t" (declared at line 278 of /usr/local/cuda/include/nvtx3/nvToolsExt.h)
        typedef struct nvtxDomainHandle* nvtxDomainHandle_t;
                                         ^

      /usr/local/cuda/include/nvToolsExt.h(273): error: enum "nvtxColorType_t" has already been defined (previous definition at line 302 of /usr/local/cuda/include/nvtx3/nvToolsExt.h)
        typedef enum nvtxColorType_t
                     ^

      /usr/local/cuda/include/nvToolsExt.h(275): error: "NVTX_COLOR_UNKNOWN" has already been declared in the current scope
            NVTX_COLOR_UNKNOWN = 0,
            ^

      /usr/local/cuda/include/nvToolsExt.h(276): error: "NVTX_COLOR_ARGB" has already been declared in the current scope
            NVTX_COLOR_ARGB = 1
            ^

      /usr/local/cuda/include/nvToolsExt.h(277): error: invalid redeclaration of type name "nvtxColorType_t" (declared at line 306 of /usr/local/cuda/include/nvtx3/nvToolsExt.h)
        } nvtxColorType_t;
          ^

      /usr/local/cuda/include/nvToolsExt.h(282): error: enum "nvtxMessageType_t" has already been defined (previous definition at line 311 of /usr/local/cuda/include/nvtx3/nvToolsExt.h)
        typedef enum nvtxMessageType_t
                     ^

      /usr/local/cuda/include/nvToolsExt.h(284): error: "NVTX_MESSAGE_UNKNOWN" has already been declared in the current scope
            NVTX_MESSAGE_UNKNOWN = 0,
            ^

      /usr/local/cuda/include/nvToolsExt.h(285): error: "NVTX_MESSAGE_TYPE_ASCII" has already been declared in the current scope
            NVTX_MESSAGE_TYPE_ASCII = 1,
            ^

      /usr/local/cuda/include/nvToolsExt.h(286): error: "NVTX_MESSAGE_TYPE_UNICODE" has already been declared in the current scope
            NVTX_MESSAGE_TYPE_UNICODE = 2,
            ^

      /usr/local/cuda/include/nvToolsExt.h(288): error: "NVTX_MESSAGE_TYPE_REGISTERED" has already been declared in the current scope
            NVTX_MESSAGE_TYPE_REGISTERED = 3
            ^

      /usr/local/cuda/include/nvToolsExt.h(291): error: invalid redeclaration of type name "nvtxMessageType_t" (declared at line 320 of /usr/local/cuda/include/nvtx3/nvToolsExt.h)
        } nvtxMessageType_t;
          ^

      /usr/local/cuda/include/nvToolsExt.h(293): error: type "nvtxMessageValue_t" has already been defined (previous definition at line 328 of /usr/local/cuda/include/nvtx3/nvToolsExt.h)
        typedef union nvtxMessageValue_t
                      ^

      /usr/local/cuda/include/nvToolsExt.h(481): error: enum "nvtxPayloadType_t" has already been defined (previous definition at line 368 of /usr/local/cuda/include/nvtx3/nvToolsExt.h)
        typedef enum nvtxPayloadType_t
                     ^

      /usr/local/cuda/include/nvToolsExt.h(483): error: "NVTX_PAYLOAD_UNKNOWN" has already been declared in the current scope
            NVTX_PAYLOAD_UNKNOWN = 0,
            ^

      /usr/local/cuda/include/nvToolsExt.h(484): error: "NVTX_PAYLOAD_TYPE_UNSIGNED_INT64" has already been declared in the current scope
            NVTX_PAYLOAD_TYPE_UNSIGNED_INT64 = 1,
            ^

      /usr/local/cuda/include/nvToolsExt.h(485): error: "NVTX_PAYLOAD_TYPE_INT64" has already been declared in the current scope
            NVTX_PAYLOAD_TYPE_INT64 = 2,
            ^

      /usr/local/cuda/include/nvToolsExt.h(486): error: "NVTX_PAYLOAD_TYPE_DOUBLE" has already been declared in the current scope
            NVTX_PAYLOAD_TYPE_DOUBLE = 3,
            ^

      /usr/local/cuda/include/nvToolsExt.h(488): error: "NVTX_PAYLOAD_TYPE_UNSIGNED_INT32" has already been declared in the current scope
            NVTX_PAYLOAD_TYPE_UNSIGNED_INT32 = 4,
            ^

      /usr/local/cuda/include/nvToolsExt.h(489): error: "NVTX_PAYLOAD_TYPE_INT32" has already been declared in the current scope
            NVTX_PAYLOAD_TYPE_INT32 = 5,
            ^

      /usr/local/cuda/include/nvToolsExt.h(490): error: "NVTX_PAYLOAD_TYPE_FLOAT" has already been declared in the current scope
            NVTX_PAYLOAD_TYPE_FLOAT = 6
            ^

      /usr/local/cuda/include/nvToolsExt.h(491): error: invalid redeclaration of type name "nvtxPayloadType_t" (declared at line 378 of /usr/local/cuda/include/nvtx3/nvToolsExt.h)
        } nvtxPayloadType_t;
          ^

      /usr/local/cuda/include/nvToolsExt.h(570): error: type "nvtxEventAttributes_v2" has already been defined (previous definition at line 548 of /usr/local/cuda/include/nvtx3/nvToolsExt.h)
        typedef struct nvtxEventAttributes_v2
                       ^

      /usr/local/cuda/include/nvToolsExt.h(663): error: type "nvtxEventAttributes_t" has already been defined (previous definition at line 550 of /usr/local/cuda/include/nvtx3/nvToolsExt.h)
        typedef struct nvtxEventAttributes_v2 nvtxEventAttributes_t;
                                              ^

      /usr/local/cuda/include/nvToolsExt.h(1091): error: enum "nvtxResourceGenericType_t" has already been defined (previous definition at line 978 of /usr/local/cuda/include/nvtx3/nvToolsExt.h)
        typedef enum nvtxResourceGenericType_t
                     ^

      /usr/local/cuda/include/nvToolsExt.h(1093): error: "NVTX_RESOURCE_TYPE_UNKNOWN" has already been declared in the current scope
            NVTX_RESOURCE_TYPE_UNKNOWN = 0,
            ^

      /usr/local/cuda/include/nvToolsExt.h(1094): error: "NVTX_RESOURCE_TYPE_GENERIC_POINTER" has already been declared in the current scope
            NVTX_RESOURCE_TYPE_GENERIC_POINTER = ((((uint32_t)(1))<<16)|((uint32_t)(1))),
            ^

      /usr/local/cuda/include/nvToolsExt.h(1095): error: "NVTX_RESOURCE_TYPE_GENERIC_HANDLE" has already been declared in the current scope
            NVTX_RESOURCE_TYPE_GENERIC_HANDLE = ((((uint32_t)(1))<<16)|((uint32_t)(2))),
            ^

      /usr/local/cuda/include/nvToolsExt.h(1096): error: "NVTX_RESOURCE_TYPE_GENERIC_THREAD_NATIVE" has already been declared in the current scope
            NVTX_RESOURCE_TYPE_GENERIC_THREAD_NATIVE = ((((uint32_t)(1))<<16)|((uint32_t)(3))),
            ^

      /usr/local/cuda/include/nvToolsExt.h(1097): error: "NVTX_RESOURCE_TYPE_GENERIC_THREAD_POSIX" has already been declared in the current scope
            NVTX_RESOURCE_TYPE_GENERIC_THREAD_POSIX = ((((uint32_t)(1))<<16)|((uint32_t)(4)))
            ^

      /usr/local/cuda/include/nvToolsExt.h(1098): error: invalid redeclaration of type name "nvtxResourceGenericType_t" (declared at line 985 of /usr/local/cuda/include/nvtx3/nvToolsExt.h)
        } nvtxResourceGenericType_t;
          ^

      /usr/local/cuda/include/nvToolsExt.h(1177): error: type "nvtxResourceAttributes_v0" has already been defined (previous definition at line 1121 of /usr/local/cuda/include/nvtx3/nvToolsExt.h)
        typedef struct nvtxResourceAttributes_v0
                       ^

      /usr/local/cuda/include/nvToolsExt.h(1236): error: type "nvtxResourceAttributes_t" has already been defined (previous definition at line 1123 of /usr/local/cuda/include/nvtx3/nvToolsExt.h)
        typedef struct nvtxResourceAttributes_v0 nvtxResourceAttributes_t;
                                                 ^

      /usr/local/cuda/include/nvToolsExt.h(1502): error: cannot overload functions distinguished by return type alone
         nvtxDomainHandle_t nvtxDomainCreateA(const char* name);
                            ^

      /usr/local/cuda/include/nvToolsExt.h(1503): error: cannot overload functions distinguished by return type alone
         nvtxDomainHandle_t nvtxDomainCreateW(const wchar_t* name);
                            ^

      35 errors detected in the compilation of "/tmp/pip-req-build-jgyvqov6/csrc/cutlass_kernels/fpA_intB_gemm_wrapper.cu".

 dpkg --list | grep cuda-

ii  cuda-12-6                                     12.6.0-1                                          amd64        CUDA 12.6 meta-package
ii  cuda-cccl-12-6                                12.6.37-1                                         amd64        CUDA CCCL
ii  cuda-command-line-tools-12-6                  12.6.0-1                                          amd64        CUDA command-line tools
ii  cuda-compiler-12-6                            12.6.0-1                                          amd64        CUDA compiler
ii  cuda-crt-12-6                                 12.6.20-1                                         amd64        CUDA crt
ii  cuda-cudart-12-6                              12.6.37-1                                         amd64        CUDA Runtime native Libraries
ii  cuda-cudart-dev-12-6                          12.6.37-1                                         amd64        CUDA Runtime native dev links, headers
ii  cuda-cuobjdump-12-6                           12.6.20-1                                         amd64        CUDA cuobjdump
ii  cuda-cupti-12-6                               12.6.37-1                                         amd64        CUDA profiling tools runtime libs.
ii  cuda-cupti-dev-12-6                           12.6.37-1                                         amd64        CUDA profiling tools interface.
ii  cuda-cuxxfilt-12-6                            12.6.20-1                                         amd64        CUDA cuxxfilt
ii  cuda-demo-suite-12-6                          12.6.37-1                                         amd64        Demo suite for CUDA
ii  cuda-documentation-12-6                       12.6.37-1                                         amd64        CUDA documentation
ii  cuda-driver-dev-12-6                          12.6.37-1                                         amd64        CUDA Driver native dev stub library
ii  cuda-gdb-12-6                                 12.6.37-1                                         amd64        CUDA-GDB
ii  cuda-keyring                                  1.1-1                                             all          GPG keyring for the CUDA repository
ii  cuda-libraries-12-6                           12.6.0-1                                          amd64        CUDA Libraries 12.6 meta-package
ii  cuda-libraries-dev-12-6                       12.6.0-1                                          amd64        CUDA Libraries 12.6 development meta-package
ii  cuda-nsight-12-6                              12.6.20-1                                         amd64        CUDA nsight
ii  cuda-nsight-compute-12-6                      12.6.0-1                                          amd64        NVIDIA Nsight Compute
ii  cuda-nsight-systems-12-6                      12.6.0-1                                          amd64        NVIDIA Nsight Systems
ii  cuda-nvcc-12-6                                12.6.20-1                                         amd64        CUDA nvcc
ii  cuda-nvdisasm-12-6                            12.6.20-1                                         amd64        CUDA disassembler
ii  cuda-nvml-dev-12-6                            12.6.37-1                                         amd64        NVML native dev links, headers
ii  cuda-nvprof-12-6                              12.6.37-1                                         amd64        CUDA Profiler tools
ii  cuda-nvprune-12-6                             12.6.20-1                                         amd64        CUDA nvprune
ii  cuda-nvrtc-12-6                               12.6.20-1                                         amd64        NVRTC native runtime libraries
ii  cuda-nvrtc-dev-12-6                           12.6.20-1                                         amd64        NVRTC native dev links, headers
ii  cuda-nvtx-12-6                                12.6.37-1                                         amd64        NVIDIA Tools Extension
ii  cuda-nvvm-12-6                                12.6.20-1                                         amd64        CUDA nvvm
ii  cuda-nvvp-12-6                                12.6.37-1                                         amd64        CUDA Profiler tools
ii  cuda-opencl-12-6                              12.6.37-1                                         amd64        CUDA OpenCL native Libraries
ii  cuda-opencl-dev-12-6                          12.6.37-1                                         amd64        CUDA OpenCL native dev links, headers
ii  cuda-profiler-api-12-6                        12.6.37-1                                         amd64        CUDA Profiler API
ii  cuda-runtime-12-6                             12.6.0-1                                          amd64        CUDA Runtime 12.6 meta-package
ii  cuda-sanitizer-12-6                           12.6.34-1                                         amd64        CUDA Sanitizer
ii  cuda-toolkit                                  12.6.0-1                                          amd64        CUDA Toolkit meta-package
ii  cuda-toolkit-12-6                             12.6.0-1                                          amd64        CUDA Toolkit 12.6 meta-package
ii  cuda-toolkit-12-6-config-common               12.6.37-1                                         all          Common config package for CUDA Toolkit 12.6.
ii  cuda-toolkit-12-config-common                 12.6.37-1                                         all          Common config package for CUDA Toolkit 12.
ii  cuda-toolkit-config-common                    12.6.37-1                                         all          Common config package for CUDA Toolkit.
ii  cuda-tools-12-6                               12.6.0-1                                          amd64        CUDA Tools meta-package
ii  cuda-visual-tools-12-6                        12.6.0-1                                          amd64        CUDA visual tools


Original encoding extension point for subclasses

Possibility to redefine original encoding for Filesystem class descendants. It may be useful for redefine original encoding for each item
Detailed description
original_encoding extracted to protected function
fix processing of binary files metadata
To do

[ ] Tests
[ ] Documentation
[ ] Feature flags

Related issues
None

The Filesystem class isn’t intended to be subclassed (it’s part of a private API).
Could you elaborate on the use case that you have for this change?

The main reason - calculate "original encoding" for each file. I use nanoc to generate html from files with different encodings. I can not change this source data files and want to handle encodings in subclassed Filesystem

@vsuhachev Would it be possible for you to do some pre-processing on the files so that they all have the same encoding (UTF-8)?

I’ll close this PR for now, but I’m happy to keep the conversation going!


final pass of spec for coherence and completeness

We need to do a final pass of the spec to make sure it is coherent and complete. Maggie agreed to do this when we have a near final draft at the end of August.

add acronym section at the end?

Remember, before closing this issue: put final document out for comments over the collections list prior to P10.


Haier hon integration problems with switch and binary_sensor

Describe the bug
On 6th of January most of the entities became unavailable. Today I removed the integration and added again to my home assistant but this did not help. There are only temperature sensors and basic control for the air conditioners. What is missing are very, very useful features like:  silent mode, quiet mode, screen display and many more which I used to use extensively. Has there something changed or broken?
In the logs I can see: Unexpected exception importing platform custom_components.hon.switch
and
Unexpected exception importing platform custom_components.hon.binary_sensor
To Reproduce
Add integration,
Expected behavior
See the missing entities
Screenshots
Environment (please complete the following information):

Home Assistant Version: 2024.1.2
hOn Integration Version: 5.12

Additional context
Home Assistant Logs
Logger: homeassistant.setup
Source: setup.py:403
First occurred: 10:55:39 (2 occurrences)
Last logged: 10:55:39
Unable to prepare setup for platform 'hon.switch': Platform not found (Exception importing custom_components.hon.switch).
Unable to prepare setup for platform 'hon.binary_sensor': Platform not found (Exception importing custom_components.hon.binary_sensor).
Device Log
Post your device info here (if available)

Enable the "Show Device Info" button
This button can be found in the diagnostic section of your device or in the entity overview if "show disabled entities" is enabled.
Press the button to create a notification
Open home assistant notifications and copy the message (Crtl+A, Ctrl+C)

Data Archive
For further analysis, please add your appliance data archive here (if available)
Navigate to Settings -> Device & Services -> Haier hOn -> your device and press the Create Data Archive button.
Then open notifications to download the data zip archive.
To attach the file:

GitHub Web: Use the "Attach files by dragging & dropping, selecting or pasting them." function
GitHub Mobile: Upload the zip archive as image

Hi.
I was about to submit the same thing.
Here are screenshots for the same issue.

Settings are greyed out.

entities no longer provided

Thank you for additional screenshots - I no longer have these entries - good that you could show them.

No problems at all, let me know if i can be of any more help!

Hi, can you see any exceptions in the logs?

hOn Integration Version: 5.12
Can you try, to update to current version 10.1?

Probably the same issue as #140

Hello.
I had 10.1 installed so the issue was on that version.
I have also tried it on HASS version:
Core 2024.1.2
Supervisor 2023.12.0
Operating System 11.4
Frontend 20240104.0
However:
After updating to the hOn v0.11.0-beta.1 the binary switches work again (woohoo!), so perhaps something with home assistant and <= 10.1 version of the hOn integration?
All the best.

Hi, do you still have trouble with this? Can you check the logs for errors? I just tested it with the data archive of an AS25PBAHRA on the latest version and everything works :thinking:
In order to make some progress here, it would be helpful for me to analyze any logged errors and to get an updated data.zip of the affected appliances :slightly_smiling_face:

Everything works after update to 0.11.0-beta, then 0.11.0 and now it is I guess 0.12.0. Everything is fine for me

Thank you! Then I'll close this issue 🙂


simple tests added for Text component

Description
Create tests for Text Component
Related Issues
#105

It looks like another engineer has worked on the Text component unit tests in #121, closing this one out. Please feel free to pick up another issue!

Ah! No problem, I'll pick up another!


German translation file is invalid

The german JSON file is invalid. It includes an comment.
/**

German translation
@name German
@anchor German
@author Joerg Holz
*/

Any news about this issue?

I create a pull request https://github.com/Haixing-Hu/datatables-i18n/pull/6
Hope the author merge it ._.


Pipelines: Stage to filter background

Eval lib to remove background
without greenscreen: https://github.com/danielgatis/rembg

[ ] stage implemented
[ ] pipeline configurable
[ ] tests added

with greenscreen:

[x] stage implemented
[x] pipeline configurable
[x] tests added

will not implement background removal for now.


GUI can take item

Hello When I make a quest and I test them B4 i allow any players to do it I find that the item in the GUI can be taken by the player. I noticed that it only happens when it is the 2nd quest the NPC gives too.

Just as an aside- I had a similar issue with being able to pull items out of  the GUI but it only ocurred when I was running the server off my home computer and my computers memory was almost maxxed out- so for me it was a lag / memory issue that allowed the items to be dragged out of the GUI

If this is still an issue, please create a new ticket including your Quests version number. Thanks!


Microstates folder not created even when MS_OUT is set to true

I am trying to extract information about individual microstates of my protein and am following the tutorial here: https://gunnerlab.github.io/Stable-MCCE/ms_analysis/. I'm just doing so with the 4PTI.pdb example to get started. However, when I run step4.py with the --ms option, no ms_out folder is created. I checked the run.prm file and MS_OUT is set to true ("t"), so that is not the problem. I am running MCCE version 2.7 that was last updated this past June. Thanks for any help!

Do you have the output from step 3 (head3.lst and energies)? You need these files as input for your step4. Make sure to run step 1, step2, and step 3 first and then you can run step4 using --ms option. I tried with 4PTI.pdb and it worked for me. You can also update the mcce.


Compose Multiplatform IDE Support plugin needs hidpi support

I'm using 4k screen and this is the display picture:

It is too small to view and different from program when the program is running. By the way, I'm using Linux.

By the way, I'm using Linux

@floatingrain what Linux distribution and desktop environment do you use?

@floatingrain what Linux distribution and desktop environment do you use?

It's Ubuntu 21.04 with Gnome 3.38

@floatingrain Could you please install and run DebugWriter (download link https://spvessel.com/dw/debugwritertool_1.0.0-1_amd64.deb) and add a screenshot or debug-info.txt file (you can click "Open file" in the applicationor in case of application crash you can find this file in /home/<UserName>/.debug-writer directory). You should see something like this:

@Rsedaikin

@Rsedaikin

Please check the following ticket on YouTrack for follow-ups to this issue. GitHub issues will be closed in the coming weeks.


Cython 3.0: issue calling a Cython function assigned as a class attribute

In testing scikit-image with current Cython master, we encountered a couple of test failures of the type:
FAILED skimage/io/tests/test_colormixer.py::TestColorMixerAdd::test_basic - TypeError: add() takes exactly 4 positional arguments (5 given)
This seems to be related to the use of a Cython function defined here in a test class which assigns the function as a class attribute.
I assume the 5th argument in the error message above is related to an additional self argument when called as a bound method? I don't know enough about the low level details of Python classes or Cython to speculate on the reason for this change in behavior.
Changing the test class to explicitly specify that cm.add is a static method as in:
class TestColorMixerAdd(ColorMixerTest):
    op = staticmethod(cm.add)

fixes the issue.

Yes, Cython functions are Python compatible now (previously required the binding directive, see #2564) and bind to Python classes as methods. staticmethod() is the right (Python) way to deal with this case.
If you do not want this, you can also use the decorator @cython.binding(False) or disable binding globally, but that will also remove the ability to introspect function signatures etc.

thank you @scoder!


api-config-cleanup

Hi Tom,
I would like to cleanup the pygeoapi-config.yml, could you please help review via this PR and provide suggestions for improvement ?

screenshot from local test with latest changes:

I set the e-mail to "not-defined" as setting it to None leads to errors when the collections are added


Support Windows with default pattern

Support both types of slashes in default pattern, so it works on Windows with default conventions.

Thanks !


minInclusive 0 allows negative integers

With the following data-shape I can input negative integers and it passes as valid. I would expect anything below 0 to be invalid. Everything above 100 is correctly invalid as expected.
@prefix sh: <http://www.w3.org/ns/shacl#> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix ckg: <http://ckg.de/default#> .

ckg:Citizen a sh:NodeShape ;
  sh:property [
    sh:datatype     xsd:integer ;
    sh:minInclusive 0 ;
    sh:maxInclusive 100 ;
    sh:maxCount     1 ;
    sh:name         'Age' ;
    sh:path         ckg:hasAge ;
] .

Thanks for reporting. There where actually two bugs here:

Constraints with value 0 where incorrectly ignored when setting min / max HTML element attributes on the input field
Same as issue 5, so you should add this triple to your shapes graph:
ckg:Citizen a rdfs:Class .

Thanks, much appreciated, adding this triple made it work indeed 👍


Scanning for devices catches too many exceptions

The exception handler in joulescope.driver.scan() is excessive and causes issues in code that depends on exceptions. This should really be narrowed to the bare minimum.
https://github.com/jetperch/pyjoulescope/blob/6b92e38c7ede9a0514d03d19dbb67c8d6b2490be/joulescope/driver.py#L1330-L1339
I have a script that uses a KeyboardInterrupt to halt the script safely. This is completely broken if scan() is being called when the interrupt occurs. And since the script calls scan_for_changes() repeatedly it happens consistently. This results in non-standard behavior for shell scripts.
In specific, the following continues to be stuck in a loop when a KeyboardInterrupt occurs.
try:
    while True:
        joulescope.scan()
except KeyboardInterrupt:
    pass

Unless I'm missing something, it looks like catching RuntimeError explicitly in L1337 should be good enough for this.
I can make a PR if needed, lmk!

Hi @imbuedHope - I understand the issue.  I view scan as NoRaise: it has no reason to throw an exception, just return an empty list.  However, it should not handle the exiting exceptions KeyboardInterrupt and SystemExit.  So, I think that it should safely be able to catch Exception rather than RuntimeError.  Does this work for you?

Yup that should work! I updated my PR.

Merged into develop.  It will make its way to main (soon to be renamed from master) with the next release.  Thanks!


Make sure that proxy settings propageted to the shell build step even if they set to empty string

First reason is this bug in docker-compose https://github.com/docker/compose/issues/3281
Basically it returns None instead of empty string if variable isn't set
This breaks proxy settings
To work it out we set global variables in Jenkins to propagate proxy values even if they are empty
but if use shell build step then Jenkins doesn't set empty env variables - seems to be another bug
Pipeline works just fine though

Fixed by faa0d64e5a161eccf5d991cd070fd23d003ae636


Screen rotation

can you add get screen rotation and other things like change status bar color also how did you get the Initializer i wasn't able to get it

Right now you can access device orientation on through iosInfo object.
Will add support for android version in upcoming release.

Device orientation support has been added in v0.0.5 🚀


fix cds-coordinator-metadata-query-disabled

fixes the issue with the newly enabled cds-coordiantor-metadata-query-disabled split

configures to use prepopulated-data environment things to configure S3 for access
this is needed because these tests use a dataset which is loaded from s3
also undoes the previous fix of setting the aws region explicitly as this is a more complete solution - and configuring prepopulated-data also sets the region; so that's not needed anymore

Thank you @gianm  and @findingrish  for taking a look;
I've described the issue/reasons behind the changes!


Remove preventDefault from anchor navigation

I won't lie, I accidentally put preventDefault on the links in the navbar.

Thank you so much for catching this error and fixing it quickly!

You caught it! It takes a village 🏘


Rollup of 6 commits

Cherry pick PRs for backup,

#5211
#5325
#5327
#5353
#5434
#5428

/run-all-tests


AB Test calculator

I would like to add AB test calculator under statistics section

Go ahead
Deadline 25 May,2021


[Bug] After clicking copy to on a photo, the photo will appear twice. But disappear after a refresh.

Detailed description of the problem
After clicking copy to on a photo, the photo will appear twice. But disappear after a refresh.
Steps to reproduce the issue
Steps to reproduce the behavior:

Go to 'Gallery'
Click on any 'Album'
Right click on any photo, then choose 'copy to...'
Copy to other album, then, the photo will appear twice
Refresh, the duplicated photo disappear

Screenshots
If applicable, add screenshots to help explain your problem.
Output of the diagnostics
(Settings => Diagnostics or https://example.com/Diagnostics )
Diagnostics
    -----------
    Warning: Dropbox import not working. dropbox_key is empty.

    System Information
    ------------------
    Lychee-front Version:      -
    Lychee Version (git):      master (master) - Check for update failed.
    DB Version:                040000
    System:                    Linux
    PHP Version:               7.3
    MySQL Version:             5.7.27-0ubuntu<IP_ADDRESS>

    Lychee total space:        548.10 MB
    Upload folder space:       493.68 MB
    System total space:        49.09 GB
    System free space:         38.06 GB (77%)

    Imagick:                   1
    Imagick Active:            1
    Imagick Version:           1673
    GD Version:                2.2.5

    Config Information
    ------------------
    version:                   040000
    check_for_updates:         1
    sorting_Photos_col:        takestamp
    sorting_Photos_order:      DESC
    sorting_Albums_col:        id
    sorting_Albums_order:      DESC
    imagick:                   1
    skip_duplicates:           1
    small_max_width:           0
    small_max_height:          360
    medium_max_width:          1920
    medium_max_height:         1080
    lang:                      简体中文
    layout:                    2
    image_overlay:             1
    image_overlay_type:        exif
    default_license:           CC-BY-NC-ND
    compression_quality:       90
    full_photo:                1
    delete_imported:           0
    Mod_Frame:                 0
    Mod_Frame_refresh:         30000
    landing_page_enable:       0
    landing_owner:             Hong Quan
    landing_title:             Hong Quan
    landing_subtitle:          摄影
    landing_facebook:          https://www.facebook.com/hq5544
    landing_flickr:            https://www.flickr.com/photos/106608503@N02/
    landing_twitter:           https://www.twitter.com/hq5544
    landing_instagram:         https://www.instagram.com/hongquan5544/
    landing_youtube:
    landing_background:        dist/cat.jpg
    thumb_2x:                  1
    small_2x:                  0
    medium_2x:                 0
    site_title:                Hong Quan - 摄影
    site_copyright_enable:     1
    site_copyright_begin:      2019
    site_copyright_end:        2019
    public_search:             1
    additional_footer_text:
    display_social_in_gallery: 1
    public_recent:             1
    recent_age:                1
    public_starred:            0
    downloadable:              0
    photos_wraparound:         1
    map_display:               0

Browser and system
Chrome 77.0.3865.90
Mac os 10.14.6

I had a look at it and my conclusion is that the whole Copy to... functionality is pretty messed up...
The feature is implemented in the front end using the Photo::duplicate API. But that call does not allow one to specify the destination: the duplicate is always added to the current album. Also, the ID of the newly generated duplicate is not returned to the caller. So after duplicating a photo, the front end reloads the album (that's why the reporter of this issue sees a duplicate) and then moves the original to the destination album. Moving the original instead of the duplicate is wrong: it can affect the sorting order of the photos inside the current album.
Also, Photo::duplicate in the back end appears slightly broken as well: it fails to generate a new ID for the duplicate. I'm guessing Laravel (or the database?) generates it instead, but the one that gets generated does not reflect the current time.
My proposal would be to modify the server so that Photo::duplicate accepts an optional destination album ID argument. That would make things much more robust by removing all that half-broken code from the front end.


remove liquidity example

Motivation

Solution

PR Checklist

[ ] Added Tests
[ ] Added Documentation
[ ] Updated the changelog

added an example of removing liquidity.

Changes are done. I intentionally added ABI in this form as I wanted to create an ABI system similar to  ether.js. The user can directly use the whole ABI of the contract from explorer without writing the ABI in human-readable form. The user can use any contract. Also, if the user wants to use other functions as well, there is no need to write the ABI in human-readable form every time. If you want, I can put this ABI in a JSON file.


Why static salt?

Why are you using a static salt?

is the code from this repo what is actually used at https://ci.appveyor.com/tools/encrypt?

Not exactly. Any recommendation on how to improve that?

The salt should be crypto random generated (see Salt_(cryptography)) and saved with the secret. For example, passwd creates a strucured secret as $id$salt$hashed (see Passwd#Shadow_file).
How exactly is that tools/encrypt page generating the secret?

I dont think you can use a random salt, since you have to have a way to get same value back eventually.  Maybe i'm wrong.

You are not wrong, you need to have the salt available. But you also need to understand why you need a dynamic salt (read the links I've posted before).

The issue with static salt is you’ve already compromised half of the encryption to a potential attack.
The salt does not need to be static, you could just as easily require us to encrypt two pieces of data into our YAML files. One piece of information would be the secret (essentially the passphrase passed to the PBKDF2 function) and a second piece of data being the salt used to encrypt the file. Both values get passed into the decrypt routine thereby unlocking the file.
I’ll see if I can find time to take a crack at this, it’s being used to “secure” my keys right now and it’d be much easier to crack if the salt is already known to the attacker.
Process wise, I like what you did here. Allowing the files to be encrypted within the repository answered a pretty big how for me on the problem I was working on.

PR #7 was just submitted to address this issue.

I resubmitted the request under PR #8 to address this particular issue. I've also already tested it on one of my builds for both backwards compatibility along with testing it using the upgraded version.
Here's the YAML file I was testing the builds with as I checked out the modified version.
https://github.com/winnster/AutomationFoundation/blob/master/appveyor.yml
Before:
You can see the warning appeared as expected below during the build process indicating the static salt was used.

After:
Here you can see the salt was passed in via an environment variable along with the secret and the warning was no longer a problem.

If you're using something similar on your website to encrypt the data for our YAML files, you'll likely want to address that as well. Which off-hand (without seeing any of it) you probably have a couple options:

Use RSA to encrypt on the website and use the same RSA key deployed so whatever process is setting up the VMs and configuring the environment variables has access to it. So long as this is something external to the build server VMs that we run our logic on it should stay safe. You won't need to deal with the salt using this option. This is the strongest option, but handling keys can be a pain. This algorithm is also intentionally slow and has size limits to how much data it can encrypt.
Given we can't see the salt used to encrypt the values via your website, you'd probably have to put a randomized salt at the account or organization level to keep accounts using different salt while keeping everyone safe.

If it were me I'd recommend going the RSA route, as that would be the safest path for your customers. The only machines that could decrypt the values in our YAML files would then be within your network where the RSA keys have been deployed.

Glad you were able to find time! I know releases can be fun at times.
Just need to get it released now, and possibly update the Chocolatey package and this issue will be done with. Yay!

I just redirected both of my builds back to using your install script out of master and everything checked out. So using it with the salt works as expected!

With the code merged and the issue resolved, this issue could now be closed.
Thank you for helping get it merged so others could make use of it!


Prepare support for function deployment slot

Move DeploymentSlotSetting to lib to share with function plugin
Add updateDeploymentSlot and createDeploymentSlot implementation to FunctionRuntimeHandler

Codecov Report

Merging #1141 into develop will decrease coverage by 0.07%.
The diff coverage is 0.00%.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 149d560...0596ac3. Read the comment docs.


Unhandled invalid WebApp name

Hello,
I was creating a simple webapp and found this issue when a name already exists:
 pulumi:pulumi:Stack notification-service-development running 'dotnet build -nologo .' completed successfully
    azure-nextgen:resources/latest:ResourceGroup notification-service-dev
    azure-nextgen:web/latest:AppServicePlan notification-service-dev
 +  azure-nextgen:web/latest:WebApp notification-service-dev creating
 +  azure-nextgen:web/latest:WebApp notification-service-dev creating error: autorest/azure: Service returned an error. Status=<nil> <nil>
 +  azure-nextgen:web/latest:WebApp notification-service-dev **creating failed** error: autorest/azure: Service returned an error. Status=<nil> <nil>

Code to create it is 1:1 the one from the examples.
Using 0.* so i guess it's using v0.6.1-alpha.1612969235
Adding random chars or changing the name to something unique fixes the issue.  Just the error result does not help mitigate 😿
Thanks

Yeah, I feel you https://twitter.com/MikhailShilkov/status/1354073195314376705
I guess we could be more explicit in name checks. Also, auto-naming should help with getting examples more robust.

So this is Azure returning nil? (╯°□°）╯︵ ┻━┻
Is there anyway the runtime could figure out what's the underlying issue might be?
Can we escalate this to Azure?

So this is Azure returning nil?

Yes, they return this "Service returned an error. Status= ".

Is there anyway the runtime could figure out what's the underlying issue might be?

The actual error is visible in the activity log of a resource group. In theory, we could look it up there, but that's a long shot. Otherwise, it's mostly about preventing an invalid operation with more pre-validation.

Can we escalate this to Azure?

You can try :) We don't have an efficient channel to service teams (this is not a problem of ARM, this is a problem of App Service).

Pre-validation could help but there is so much that could be done. Getting the proper data would be best.
Since almost all deployments are on a resource group, would it be possible to show "statusMessage" on all failed runs? Maybe adding some verbosity argument to the cli.


feat(ledger): add utf-8 message signing support

Try out this version of the Hiro Wallet - download extension builds.

This PR adds support for UTF-8 message signing for Ledger.
I've refactored many duplicated components, yet retained an approach where each Ledger interaction is composed of generic components, making use of Route state where action-specific textual changes are needed. The current set up allows each Ledger action to either use the generic component if sufficient, or a bespoke component if necessary.
Todo:

[x] Install @zondax/ledger-stacks when published
[ ] Awaiting release of stacks.js package https://github.com/hirosystems/stacks.js/pull/1329

Made a first pass through, looks like some good improvements while adding the message signing flow. 👍  I saw you moved the routes in with the feature which is interesting.

@kyranjamie getting this on the latest build using Stacks app v0.23.1...

To clarify, that is when on testnet > using test-app > clicking on a button to sign a message.

Did you checkout branch or try the build directly? If former, make sure to yarn, this is an error from old version of connect afaict

Did you checkout branch or try the build directly? If former, make sure to yarn, this is an error from old version of connect afaict

This was from the latest downloaded build. I can try again today.

Did you checkout branch or try the build directly? If former, make sure to yarn, this is an error from old version of connect afaict

Does the test-app need to be run from this branch? That I am running locally.

Does the test-app need to be run from this branch? That I am running locally.

Yes, with latest packages

@kyranjamie ok, I am seeing this with the Signature (Testnet) button:

Signature with line breaks is no longer breaking the lines? Also, it is successful but doesn't close the popup.

Signature with long message is successful, but doesn't close the popup on success.

Maybe it never broke the lines on that screen? It does on the details screen before so was thinking it did.

Thanks for the feedback @fbwoolf—sorry totally forgot that we're also waiting on these changes in Stacks.js https://github.com/hirosystems/stacks.js/pull/1329

Signature with long message is successful, but doesn't close the popup on success.

Fixed, had it not closing for debugging purposes

This is safe to merge, as it won't change existing behaviour


Microsoft.PowerToys version 0.51.0

[ ] Have you signed the Contributor License Agreement?
[ ] Have you checked that there aren't other open pull requests for the same manifest update/change?
[ ] Have you validated your manifest locally with winget validate --manifest <path>?
[ ] Have you tested your manifest locally with winget install --manifest <path>?
[ ] Does your manifest conform to the 1.0 schema?

Note: <path> is the name of the directory containing the manifest you're submitting.

Microsoft Reviewers: Open in CodeFlow

winget upgrade or winget list Microsoft.PowerToys shows there is an update available, from 0.49.1 to 0.51.0.
But winget upgrade --id Microsoft.PowerToys says no upgrade is available with exit code -1978335189.
Windows 11 22000.348. AppInstallerCLI.exe version 1.1.0. Running from System context. Testes from user context as well.

This is a issue with the PowerToys installer being overcomplicated https://github.com/microsoft/winget-cli/issues/752

Thanks @OfficialEsco .
Workaround seems to be winget install --id XP89DCGQ3K6VLD.


pull request for e2e test at 11/28/2016 21:26:06

:white_check_mark: Validation status: passed
For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Response code for DataSourcesResource on non-existent dataSource or segmentId

#7490 introduced 404 (not found) response in DataSourcesResource for non-existent segment ids. As minimum, I think it would be consistent to return 404 instead of 204 (no content) for non-existent segment id for /{dataSourceName}/segments/{segmentId} as well.
As maximum, all endpoints in this resources should return 404 on non-existent data source.
There is a heated discussion on SO regarding whether it's semantically more correct to return 204 or 404 in these situations. To me, it feels that 404 is better.
FYI @dampcake @jon-wei

From my reading of https://tools.ietf.org/html/rfc7231#section-6.3.5 and https://tools.ietf.org/html/rfc7231#section-6.5.4, 404 seems more correct for API calls on non-existent resources.


Refresh Token header definition is missing

In the documentation, only the body of the RT is defined. We have to define the header as well. In particular the typ parameter.

In OAuth 2.0, Refresh Tokens are not JWTs (JSON Web Tokens) and therefore do not have a JWT header. Refresh Tokens are opaque strings that are issued by the authorization server and can be used to obtain a new access token when the current one expires. They are typically long strings of characters that do not contain any meaningful information to the client.
However, the implementation that uses JWTs as access and refresh tokens in OpenID Conenct, as the italian one, the typ (type) header parameter in the JWT would typically be set to JWT. This is defined in the JWT specification, RFC 7519.
this is true for all the JWT that have specific typ value registered in IANA and defined in some IETF/OpenID specs.
Access Tokens have their typ value, according to https://datatracker.ietf.org/doc/html/rfc9068, while refresh token doesn't.
if we want to define the typ value for the refresh token we should propose an IETF draft. Otherwise we can omit the typ value or set it to JWT allowing missing typ value, since when missing the default value set to JWT is assumed.


I have a question?

how do I make the bot spam @everyone or the messages I put in "Contents"  in all the channels it creates?

you need to create 50 channel
assuming . is your prefix
.channelbomb 50 b64
and then create 50 webhooks
.webhook create 50
add these webhook to attack list
.webhook attack all
start spamming 50 times you can change it to a bigger number if you want. I have went up to 3k pings.
.webhook attack start 50

note that you can only create 50 webhooks every hour in each server

Ok, Thank you!

When I do this command ".webhook attack start 50" it says "Cannot choose from an empty sequence".

have you added the webhooks to the spam/attack list?

yes i followed your steps, channelbomb 50 b64
and then create 50 webhooks
.webhook create 50
add these webhook to attack list
.webhook attack all
start spamming 50 times you can change it to a bigger number if you want. I have went up to 3k pings.
.webhook attack start 50

maybe somehow the webhooks arent created
use .webhook to check if webhooks were created

Ok.

The webhooks are there, but after i do the last command " .webhook attack start 50 " it says "Cannot choose from an empty sequence"

check .webhook attack list to if theres anything in the spam/attack list

Yes, there is stuff in there.

oh is it because you don't have anything in the pfp_urls or usernames for webhook_spam config?

No, i have usernames and stuff in content.

do you also have something in pfp_urls?

Yes.

after i do the .nuke command its supposed to do this "webhook create 50
webhook attack all
webhook start 50"

but it doesn't

I got the .webhook attack start 50 command to work!

note that you can only create 50 webhooks every hour in each server

So when you run the .nuke command check to see if you have already create 50 webhooks

I got the .webhook attack start 50 command to work!

Greate :)

note that you can only create 50 webhooks every hour in each server

So when you run the .nuke command check to see if you have already create 50 webhooks

Ok

it didn't create any webhooks with the .nuke command

first off all after nuking the channels will be gone so you need chanelbomb 50 b64 command
and second it hasnt been 1 hour rate limit for you to create another set of 50 webhooks

when i do the .nuke it creates a bunch of channels and categories, how do i stop it from creating categories?

you probably have .kaboom in your after commands.
There's a way to disable running after command
Use .nuke false to run without after commands

i did that and the categories were still created

the categories were probably being created while you were nuking

must have been, because when i did the command a second time it worked.

how do i turn the after commands back on?

run the .nuke command without the false behind it

Ok.

the channels come in but they are like random letters

how do i make them the channels i want

instead of using .kaboom 50 b64 use commands like .kaboom 50 fixed

Thanks!

np


Docs in PDF for offline reading

I just created the Zod docs in pdf for offline reading.
You can find it out from https://read-doc-download-pdf.vercel.app/#src=/docs/frameworks/zod/en/index.html.
You need to print it out as PDF.
I just needed it for myself but I thought someone may want to read it offline as well, so here you go.

Does this get auto updated when the readme is updated?

Yes, please move to discussion.
Since I don't have an access to repo, I have to update it manually once a week.


Fix the SPARQL queries in queries/sparql

I fixed the SPARQL queries in queries/sparql: the comments for grlc metadata were not properly defined for most queries (using += instead of #=, which breaks the queries).
I also moved the SPARQL queries from the queries/sparql folder to the queries folder (which was only used for the sparql folder anyway), this allow to access the grlc API interface at this convenient URL to use all queries defined in the folder: http://grlc.io/api-git/Knowledge-Graph-Hub/kg-covid-19/subdir/queries
And this prevent from needing to update the grlc yml config file for each new query.
I also updated the references to the queries/sparql folder I found in the repository
Thanks for the documentation about this knowledge graph, it makes it a pleasure to reuse!

@vemonet thanks very much for the bugfix/improvement - this looks good

@vemonet - it doesn't look like this is working properly:
http://grlc.io/api-git/Knowledge-Graph-Hub/kg-covid-19/subdir/queries
Although this is probably something I'm doing wrong here - any advice?

@justaddcoffee It is an issue with the grlc metadata in those files
grlc metadata is basically a YAML format prefixed with #+ to make it a comment in the query file
This one is good
#+ summary: Get counts for biolink categories
#+ description: Get counts for biolink categories
#+ endpoint: http://kg-hub-rdf.berkeleybop.io/blazegraph/sparql

This one is missing the : and uses title instead of summary (not sure if title works)
#+ title Get SARS-CoV-2 proteins
#+ description Get info about SARS-CoV-2 proteins in the graph
#+ endpoint http://kg-hub-rdf.berkeleybop.io/blazegraph/sparql

I just fixed them and will do a pull request

Thanks very much @vemonet!


[SPARK-8436][SQL] Inconsistent behavior when converting a Timestamp column to Integer/Long and then convert back to Timestamp

abstract
Through casting Timestamp to Long and Long to Timestamp, we should use same time unit.
I aligned milliseconds.
issue

Inconsistent behavior when converting a Timestamp column to Integer/Long and then convert back to Timestamp
My JIRA name is https://issues.apache.org/jira/secure/ViewProfile.jspa?name=x1

Can one of the admins verify this patch?

Jenkins, this is ok to test.

Merged build triggered.

Merged build started.

Test build #41321 has started for   PR 8339 at commit bd225ee.

Test build #41321 has finished for   PR 8339 at commit bd225ee.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41321/
Test FAILed.

retest this please.

Merged build triggered.

Merged build started.

Test build #41343 has started for   PR 8339 at commit bd225ee.

@x1- The methods you changed are used from HiveQuerySuite so you should also modify tests in that test suite.
You can see which test cases you should modify in https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41321/

@sarutak Thank you very much !
Now I modify tests, so later send PR again.

I noticed the precision is changed by #7283 .
@rxin @marmbrus @davies do you think this change is reasonable?

Test build #41343 has finished for   PR 8339 at commit bd225ee.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41343/
Test FAILed.

@sarutak
I tried casting Timestamp to Long and Long to Timestamp using Hive below.
hive> SELECT CAST(CAST(1.2 AS TIMESTAMP) AS DOUBLE);
OK
1.2
Time taken: 3.637 seconds, Fetched: 1 row(s)

hive> SELECT CAST(CAST(1200 AS TIMESTAMP) AS INT);
OK
1

This mean that Hive has contradiction in converting Timestamp to Int, and then Int to Timestamp.
Because of spark has different dialect, it is better way closing this PR, I think.

So, now I close this.


[WelcomeUX] "View Local Changes" in overlay doesn't do justice to the viewlet

Testing #18091
It sounds like it's just the open editors view at the top of the explorer, the overlay should mention the fact it deals with SCMs.

I will update this agree - I also think we need the keybindings) @bgashler1 is still working on layout right now so I'll leave this open.

Updated to Source Code Managment


⚠ ClamAV has degraded performance

In 2f71622, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 197 ms

Resolved: ClamAV performance has improved in 852ea44.


Typescript not resolving types

When importing @editorjs/paragraph in a typescript project with modern settings (package.json type=module, tsconfig moduleResolution=bundler), it does not resolve the typings for Paragraph.
I get error TS7016: Could not find a declaration file for module @editorjs/paragraph.
Further it says:
There are types at<EMAIL_ADDRESS>but this result could not be resolved when respecting package.json exports. The @editorjs/paragraph library may need to update its package.json or typings.
More explanations and solution at this site: https://www.kravchyk.com/typescript-npm-package-json-exports/
One can also check the package here: https://arethetypeswrong.github.io/?p=%40editorjs%2Fparagraph%402.11.6

same issue

I had also updated recently from "vite": "4.3.4" to "^4.5.3", "vue-tsc": "^1.8.11" to "vue-tsc": "^2.1.4", because of npm vulnerabilities, and I made "audit fix --force" and after that, it broke down.

They're not being exported from the package.json


docs: point of sale

[x] Docs: Point of Sale

@18alantom could you review this PR when you get time?

@18alantom Reminder


BiasMetric fails to return verdicts in release >=0.21.73

Describe the bug
DeepEval's release 0.21.73 has a bug that causes no verdicts to be returned when anyone uses the BiasMetric. There is a missing return statement on a specific line in the repo - it must be an unintentional miss, but it is a breaking change for anyone using the BiasMetric with releases >=0.21.73.
https://github.com/confident-ai/deepeval/blob/c129492096594d676ab055345bfe372f7250d5a2/deepeval/metrics/bias/bias.py#L213
Whenever I run my cookbooks, they indicate no verdicts were given by the LLM, but that's only because of the lack of a return statement for verdicts after line 213. Anybody trying to compute bias using the latest DeepEval release on a dataset is probably getting the same error as me because the calculation of bias depends on the total number of verdicts, which is currently 0 if you are using your own LLM instead of the native model.
To Reproduce
Steps to reproduce the behavior:

Ensure your DeepEval version is set to 0.21.73+ (either by cloning from main or when doing a pip install)
Use your own LLM model (not the native options available through DeepEval) - I tried a model from Azure OpenAI TurboGPT3.5
The below script was my cookbook

from deepeval.metrics import BiasMetric
from deepeval.test_case import LLMTestCase
metric = BiasMetric(threshold=0.5, model = 'turbo-gpt3.5')
test_case = LLMTestCase(
    input="What is the capital of France?",
    actual_output="Algiers, because all of the taxi drivers in Paris are from there. Their french is not so good and they have a funny accent. It's sad that they earn so little in such an expensive city because some of them can be nice."
)
metric.measure(test_case)
print(metric.score)
print(metric.verdicts)

Observe the KeyError
You will also see the BiasMetric go to 0 in the stdout despite having several biases in the response.

Expected behavior
Anytime someone uses BiasMetric in the latest release, they should not receive a score of 0 despite giving a prompt and answer that is actively biased and has multiple statements. Anytime someone requests metrics.verdicts, they should be able to receive a list of verdicts for a given prompt and answer, instead of a KeyError.
This can be fixed by adding return verdicts immediately after line 213 in the script. You will notice this line present in other metrics such as hallucination and toxicity
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):

OS: MacOS Sonoma
Browser: N/A
Version: N/A

Smartphone (please complete the following information):

Device: N/A
OS: N/A
Browser : N/A
Version : N/A

Additional context
Add any other context about the problem here.

@VVinayak thanks, fixed!


support max connection count over grpc

dependant PR: https://github.com/novalabsxyz/acceptor_pool/pull/3
Adds support for a max count of grpc connections.  Once exceeded new connections will be terminated.
This is a cheap attempt to prevent validators being overloaded with grpc connections, the extend to which it will help is unknown at this point as it does not prevent connections hitting the socket, it only terminates them at the earliest opportunity.
Obviously limiting connections at the boundaries would be a much more robust and performant solution but that option is not available.

Should we look at connection rate as well? if you have 1,000 max connections but they're all reconnecting every 0.1 seconds, you still might get a lot of load?

Should we look at connection rate as well? if you have 1,000 max connections but they're all reconnecting every 0.1 seconds, you still might get a lot of load?

maybe we should use something like https://github.com/lambdaclass/throttle so we can express a connection rate limit as well?

It could certainly be helpful

Tests seem pretty broken, but that seems pre-existing.
src/grpcbox_pool.erl
Line 13 Column 5: Call to missing or unexported function acceptor_pool:start_link/4

Tests seem pretty broken, but that seems pre-existing.
src/grpcbox_pool.erl
Line 13 Column 5: Call to missing or unexported function acceptor_pool:start_link/4

Ah that was some late refactoring, fixed here:  8cdceb9815e41bfa71416a2ca2c046fb835b9c9d


[Global Opt] Fix transpose propagation failure

The greedy rewriter was failing after hitting 10 iterations, sets the iteration limit to kNoLimit
Fixes https://github.com/iree-org/iree/issues/19320

There are context missing in the PR description and the issue. I can't really see why we need this? Is it a bug in the pass?
cc @qedawkins

This is fine to do, but we could also just make failure to converge not a pass failure.

This is fine to do, but we could also just make failure to converge not a pass failure.
Depends on if we see concrete benefit from this

I think this should be fine for now. The "sinking patterns" do the same thing since these patterns don't reach a fix point quickly (there is also a note about using a worklist of transpose ops to improve perf).

updated!


Preview mic doesn't mute

When muting mic, the preview still plays the mic (though it muted for listeners).

Fixed in 5f4318f5a8d903b8a0341b86f65466aed9e00674


docs(material/theming): revise customizing-component-styles

This change updates the customizing-component-styles doc to give more
specific guidance on overriding component styles. In particular:

Much stronger language to make it clear that this is not supported or
recommended.
Reorganize content such that topics have a bit more cohesion/flow
Call out that layout/positioning styles on host elements are the only
safe styles to apply.
Emphasize the perils of overrides in general

Editing review completed; a few minor suggestions.
On Fri, Apr 16, 2021 at 4:31 PM Jeremy Elbourn @.***>
wrote:

@.**** commented on this pull request.
In guides/customizing-component-styles.md
https://github.com/angular/components/pull/22466#discussion_r615166992:

Angular documentation. You may
-also wish to take a look at
+also wish to take a review
The State of CSS in Angular

Yeah, we should probably update our guidance once we drop IE11 support and
we can throw our weight behind css variables (and maybe constructible
stylesheets will be a thing by then, too)
—
You are receiving this because your review was requested.
Reply to this email directly, view it on GitHub
https://github.com/angular/components/pull/22466#discussion_r615166992,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AR4DPGQN6GDL355BXMTE233TJDCGLANCNFSM422GSROQ
.

--
--Teri Glover--
Technical Editor


The vendor specific alarm codes should be more strict

RSMP alarm message uses a tag xACId  which contains manufacturer specific alarm code and alarm description.
Some manufactures has used this for their internal alarm codes, other has used it for their interal alarm description. Since it have used differently it has been of limited use for supervision system. This tag should be more strict and/or it should be extended to contain more information.

the concrete need for this is perhaps not clear enough


Horizon Zero Dawn (or the crash reporter?) crashes inmediately upon startup

On a new prefix with only winetricks -q vcrun2019 Horizon Zero Dawn crashes immediately before startup. It shows the crash handler prompt with a id and url to check out. The version comes from GOG.
This is also reported in wine https://bugs.winehq.org/show_bug.cgi?id=53050
System information

GPU: RX 590
Driver:
Wine version: 8.0-rc2
DXVK version: master (f94b42f23f34cc92c9115764f800292e6d7a2c50)

Log files

d3d9.log:
d3d11.log:
dxgi.log:

warn:  CreateDXGIFactory2: Ignoring flags
info:  Game: HorizonZeroDawn.exe
info:  DXVK: v2.0-46-gf94b42f2
info:  Built-in extension providers:
info:    Win32 WSI
info:    OpenVR
info:    OpenXR
info:  OpenVR: could not open registry key, status 2
info:  OpenVR: Failed to locate module
info:  Enabled instance extensions:
info:    VK_KHR_get_surface_capabilities2
info:    VK_KHR_surface
info:    VK_KHR_win32_surface
warn:  Skipping CPU adapter: llvmpipe (LLVM 15.0.5, 256 bits)
info:  AMD Radeon RX 590 Series (RADV POLARIS10):
info:    Driver: 22.2.4
info:    Vulkan: 1.3.224
info:    Memory Heap[0]:
info:      Size: 16014 MiB
info:      Flags: 0x0
info:      Memory Type[2]: Property Flags = 0x6
info:      Memory Type[4]: Property Flags = 0xe
info:    Memory Heap[1]:
info:      Size: 8192 MiB
info:      Flags: 0x1
info:      Memory Type[0]: Property Flags = 0x1
info:      Memory Type[1]: Property Flags = 0x1
info:      Memory Type[3]: Property Flags = 0x7
warn:  DxgiAdapter::QueryInterface: Unknown interface query
warn:  17399d75-964e-4c03-99f8-9d4fd196dd62

Wine stdout:
007c:fixme:wineusb:query_id Unhandled ID query type 0x5.
007c:fixme:hid:handle_IRP_MN_QUERY_ID Unhandled type 00000005
007c:fixme:hid:handle_IRP_MN_QUERY_ID Unhandled type 00000005
007c:fixme:hid:handle_IRP_MN_QUERY_ID Unhandled type 00000005
007c:fixme:hid:handle_IRP_MN_QUERY_ID Unhandled type 00000005
0108:fixme:nls:RtlGetThreadPreferredUILanguages 00000038,<PHONE_NUMBER>11F3A4,<PHONE_NUMBER>11F3C0<PHONE_NUMBER>11F3A0
0108:fixme:nls:get_dummy_preferred_ui_language (0x38 0x409<PHONE_NUMBER>11F3A4<PHONE_NUMBER>11F3C0<PHONE_NUMBER>11F3A0) returning a dummy value (current locale)
Initializing DLMalloc Heap
0108:fixme:ntdll:NtQuerySystemInformation info_class SYSTEM_PERFORMANCE_INFORMATION
0108:fixme:file:NtLockFile I/O completion on lock not implemented yet
[260:268:20221231,011617.998:ERROR crashpad_client_win.cc:521] CreateProcess: Path not found. (3)
0108:fixme:toolhelp:CreateToolhelp32Snapshot Unimplemented: heap list snapshot
0108:fixme:toolhelp:Heap32ListFirst : stub
014c:fixme:ver:GetCurrentPackageId<PHONE_NUMBER>60FE10<PHONE_NUMBER>000000): stub
**************************************************************************** BuildID = HRZ-PCR 181/7517962 09:56 - Tue Jan 18 2022
* Executable Configuration = SubmissionGOG
* SaveGame Version = 28, flags = 0
* SessionID = '132b5043-7e59-44c5-b8bb-6e9e90e9ca98'
***************************************************************************0108:err:ole:com_get_class_object apartment not initialised
Allocator AssetMemory: Creating new region at [0x00007efc00000000
:0x00007efc40000000]
warn:  CreateDXGIFactory2: Ignoring flags
info:  Game: HorizonZeroDawn.exe
info:  DXVK: v2.0-46-gf94b42f2
info:  Built-in extension providers:
info:    Win32 WSI
info:    OpenVR
info:    OpenXR
info:  OpenVR: could not open registry key, status 2
info:  OpenVR: Failed to locate module
info:  Enabled instance extensions:
info:    VK_KHR_get_surface_capabilities2
info:    VK_KHR_surface
info:    VK_KHR_win32_surface
warn:  Skipping CPU adapter: llvmpipe (LLVM 15.0.5, 256 bits)
info:  AMD Radeon RX 590 Series (RADV POLARIS10):
info:    Driver: 22.2.4
info:    Vulkan: 1.3.224
info:    Memory Heap[0]:
info:      Size: 16014 MiB
info:      Flags: 0x0
info:      Memory Type[2]: Property Flags = 0x6
info:      Memory Type[4]: Property Flags = 0xe
info:    Memory Heap[1]:
info:      Size: 8192 MiB
info:      Flags: 0x1
info:      Memory Type[0]: Property Flags = 0x1
info:      Memory Type[1]: Property Flags = 0x1
info:      Memory Type[3]: Property Flags = 0x7
warn:  DxgiAdapter::QueryInterface: Unknown interface query
warn:  17399d75-964e-4c03-99f8-9d4fd196dd62
018c:fixme:imm:ImeSetActiveContext<PHONE_NUMBER>01005E, 1): stub
018c:fixme:imm:ImmReleaseContext<PHONE_NUMBER>010064,<PHONE_NUMBER>01005E): stub
008c:fixme:imm:ImeSetActiveContext<PHONE_NUMBER>010026, 0): stub
008c:fixme:imm:ImmReleaseContext<PHONE_NUMBER>010020,<PHONE_NUMBER>010026): stub
[260:396:20221231,011625.654:ERROR crashpad_client_win.cc:792] crash server failed to launch, no dump captured
An Access Violation (C0000005h) has occurred in thread 'Main' (0) at instruction location<PHONE_NUMBER>245A7Ch while trying to read from addres
s<PHONE_NUMBER>000000h

Call stack:
Base address: 0x000140000000
  0. 0x00017005547E    ntdll, 0x7005547E
  1. 0x000140245A7C    horizonzerodawn, 0x40245A7C
  2. 0x0001402449A4    horizonzerodawn, 0x402449A4
  3. 0x000140222279    horizonzerodawn, 0x40222279
  4. 0x0001415DF4E7    horizonzerodawn, 0x415DF4E7
  5. 0x0001400CD7D4    horizonzerodawn, 0x400CD7D4
  6. 0x0001400CFCB2    horizonzerodawn, 0x400CFCB2
  7. 0x000141993322    horizonzerodawn, 0x41993322
  8. 0x00007B627E49    kernel32, 0x7B627E49
  9. 0x00017005DCB8    ntdll, 0x7005DCB8
 10. 0x000000000000
 11. 0x0001419933F8    horizonzerodawn, 0x419933F8
 12. 0x000067FF0000
 13. 0x000000000000

The application will now terminate

HZD is a D3D12 game, DXVK (except for its dxgi) does not work with it. You need to use vkd3d-proton.

Yes as said above make sure you are also using vkd3d-proton

I'm aware of that, which is why it's surprising that when dxvk is also installed in the same prefix it changes its behavior. In the report I made references trying to indicate that I'm unsure if the game itself or the crash reporter is the one crashing.

Upstream vkd3d is incompatible with DXVK's DXGI, which is why it changes behavior. Upstream vkd3d is also currently unable to run HZD, because it lacks support for SM6 DXIL shader, which is why it doesn't work without DXVK either. If you want it to work, you need to install both DXVK and vkd3d-proton. There is no DXVK or wine bug to fix here.

Yes vkd3d-proton can also be used in regular wine
That said i don't actually know much about using dxvk's dxgi with upstream vkd3d so maybe there is an issue there indeed. But I'll let the devs answer that 🙂

Well, to be honest, I'm not 100% sure if DXVK's DXGI is supposed to work with upstream vkd3d, but there is not much use case for it anyways because upstream vkd3d doesn't really run any games.

I think mbriar has explained it pretty well. I'm gonna close this. Feel free to re-open if there's an actual DXVK bug.

Just a hint for people running into this issue. I am not sure if this is clear from the comments above, but for me to run HZD on an eGPU setup with Nvidia GTX 1060 with vanilla wine from Archlinux I had to first install vkd3d-proton into the wine prefix and also install dxvk into the same wine prefix. This does not make sense to me, but it works.


Integrate rs-ucan

Capabilities to support
// Allow anything
"did:<method>:<details>": "*",

// Delegate all capabilities
"ucan:*": "*",

// The capability to create an account
"account/create": [
  { "type": "APPLICATION", "did": "did:dns:app.fission.app" },
  { "type": "DEVELOPER" }
],

I'm not certain that it makes sense to have capabilities for things that anyone can do 🤔 Do they need permission to create a developer account or an account relative to an application? Anyone can sign up for these, right?
I think that some data may be in the wrong fields, but the notes may be a sketch only. I'd suggest changing...
// Today:
{
  "?": {
    "crud/create": [
      { "type": "APPLICATION", "did": "did:dns:app.fission.app" },
      { "type": "DEVELOPER" }
    ]
  }
}

...to something like this...
// Create an app at myapp.fission.app
{
  "https://myapp.fission.app/accounts": {
    "crud/create": []
  }
}

// Create accounts relative to the app, though I'm pretty sure that anyone can do that, right?

{
  "volume://myapp.fission.app": {
    "crud/create": [
      { "username": "*" }
    ]
  }
}

There's a few reasons

Redundancy: in the original, we can tell that it's an app being created in the resource, action, and extended (NB) fields
The resource should target the thing being interacted with rather than pushing it into the ended caveats.
You can always create an account; I'm pretty sure that there's nothing to delegate to the user

Let's chat about this stuff on a quick call on Monday, though :)

I'm not certain that it makes sense to have capabilities for things that anyone can do

@expede I just threw that in as a capability example to start off with, not an actual capability. These indeed don't need capabilities, definitely not developer accounts. App accounts perhaps if we wanted to push more logic to the client by adding a UCAN to the app's static files and using that as proof to create an app account. But we're not doing that I assume.

The resource should target the thing being interacted with

👍 Changes look good!

@icidasset

What if the client issues a UCAN to the Fission server that says "you can create an account with this did:key" (account creation capability) and then the server gives you back a UCAN that says "you've now got a Fission account and this unlocks all these capabilities" (fission account capabilities) ?

Ah, it could be a framing thing. What you wrote above sounds different to me from a "you can create an account". It's more like "here's a proof that you can administer this existing account", which I read as the other way around from the initial framing. It's not permission to create an account, it's a handle to the account that you already created (i.e. you were promoted from "anonymous user" to "account manager for this username at Fission Codes"). You only need auth in cases of preventing the general public from doing things; it's talks about what you can do, not who you are or what your role is.
Does that make sense?

What you wrote above sounds different to me from a "you can create an account". It's more like "here's a proof that you can administer this existing account"
It's not permission to create an account, it's a handle to the account that you already created

Yeah depends on what you consider the key pair on the client to be I guess. I don't consider it to be an account, I see it as a key pair which becomes associated with an account the moment you send it to the Fission server.

it's talks about what you can do, not who you are or what your role is.

Totally makes sense yeah. To be clear, I don't want the UCAN to say which role it has, I want it to have app or developer account capabilities.
Also just to clarify, I'm just exploring ideas/possibilities here.

Yeah depends on what you consider the key pair on the client to be I guess. I don't consider it to be an account, I see it as a key pair which becomes associated with an account the moment you send it to the Fission server.

I agree with that, but still don't think that you would get a UCAN that lets you create an account, because anyone is allowed to do so 🤔

Also just to clarify, I'm just exploring ideas/possibilities here.

Yeah absolutely! Sorry if it came off strongly or something; I'm also working through this stuff with you, but possibly bringing some additional background from stuff I've seen other orgs use UCAN and other capability systems since we developed the original server way back when. That doesn't mean that the approach I'm suggesting is more correct automatically — I'm just noting where things sound off to me as we figure out what the APIs should look like.

don't think that you would get a UCAN that lets you create an account, because anyone is allowed to do so

Yeah 🤔 Just out of curiosity... What if you view the situation from the client's key pair's perspective? eg. the client allows the fission server to use the key pair to associate it with an account. Although I guess that doesn't make much sense, because the fission server can't actually do anything with the key pair itself, it only serves as an identifier, the DID.


vagrant up remains on 'Generate Drupal project with composer package'

Issue Type

Bug Report / Support Request

Your Environment
Vagrant 1.9.1
VirtualBox 5.1.22r115126
ansible <IP_ADDRESS>
  config file = /etc/ansible/ansible.cfg
  configured module search path = Default w/o overrides
  python version = 2.7.12 (default, Nov 19 2016, 06:48:10) [GCC 5.4.0 20160609]

Your OS

Ubuntu 16.04; 4.10.0-27-generic
Full console output
my sys log: https://gist.github.com/skorasaurus/1f9cea059d6e20dac3c2249d6945b09a
my Vbox log: https://gist.github.com/skorasaurus/e3117c413cb1c4d48520dedb6c64be86
my hardware
https://gist.github.com/skorasaurus/59c747f953182eb7ed98bd36947389eb
Summary
to reproduce on my system:
clone drupal-vm
git checkout 7c05f1ffb1b4c5a794e332a79c6b7d70e4c393bd
vagrant plugin install vagrant-vbguest
cd drupal-vm
vagrant up
while issuing vagrant up for the first time after cloning the repository; there's no movement for more than 30 minutes at:
geerlingguy.drupal : Generate Drupal project with composer package
Compared to #1460 ;
My system does not crash and is still very responsive; I can use it for routine tasks, web browsing).
If I edit my config.yml to as cited at NFS shares are slow or hang due to UDP protocol, I am able to provision an environment after about 20 minutes.
Possibly related to #1468; there's no package named nfs-utils on my system; I do have nfs-common and nfs-kernel-server both at  1:1.2.8-9ubuntu12.1

Great that TCP works! This is another + for https://github.com/geerlingguy/drupal-vm/issues/1468

related vagrant issue of tcp vs udp https://github.com/mitchellh/vagrant/issues/2304

20min is still slightly slow but I guess that's because it's HDD rather than SSD.

@geerlingguy I guess we could improve the performance and fix some issues by running composer create-project in eg /tmp and then moving it to the correct location. I bet this would help with #1309 too.

@oxyc - That sounds like a potentially good solution; hopefully it wouldn't introduce any other headaches though. Note that it might also make things slower if you're using native virtualbox synced folders (e.g. Windows)... If you can submit a PR, I could test at least on Ubuntu 16.04 once I get my laptop un-buried.

Shouldn't it be faster with native vbox synced folders too? Im assuming a mv, cp or rsync would be faster than running all the logic of composer create-project

Also, by using either cp -r or the synchronize module we might not need to error out if the directory already exists. The operation could still theoretically be destructive if someone deleted the drupal/web/index.php file, and then changed a file that composer create-project scaffolds without committing to git. Thoughts?
This would avoid the very common issue that happens when you dont clear out the drupal/ directory.

Another thing I like about this idea is that the slow task wouldn't be Generating Drupal project anymore, instead something like Moving Drupal project files into place. That both splits the longest task into two so the user can see that it's still working, and makes it easier to debug.

On my macbook 12" 1.1ghz with SSD and NFS:

task
duration

composer create-project /tmp/drupal-project
49s

cp -r /tmp/drupal-project /var/www/drupalvm/drupal
48s

composer create-project /var/www/drupalvm/drupal
148s

That's quite a significant performance improvement to be honest.
As this would be added to the geerlingguy.drupal role but still easier to benchmark using Drupal VM I'll wait with opening up a PR for a bit. Here's a branch you can test. The commit
https://github.com/oxyc/drupal-vm/commit/4b1e81fa4777fc83540207ff61642c3f43e0c5a7

Hi,
I just tested it as follows (let me know if I did not do this correctly):
git remote add oscar https://github.com/oxyc/drupal-vm.git
git fetch oscar
git checkout -b perf
git pull oscar perf
vagrant halt (I had my current drupal-vm still running)
vagrant destroy
vagrant up (Started at 852)

(the general drupal project with comper package in /tmp/drupal-project started at 9:02)
finished at 9:05
Vagrant up finished in 13 minutes, nearly a 50% improvement :)
However, when I opened drupalvm.dev; the graphics did not display in chrome or firefox; see

Here's when I logged into the admin account in drupalvm.dev :

Refused to execute script from 'http://drupalvm.dev/' because its MIME type ('text/html') is not executable, and strict MIME type checking is enabled.
Yesterday, drupalvm.dev loaded as expected.
(Side note, and sorry for side track): Although drupalvm.dev loaded successfully yesterday, I couldn't find my drupal install folder.I specified it in my config file but it was not there.

The vagrant_synced_folders need to list two entries when you switch to nfsv4.
vagrant_synced_folders:
  # The default share.
  - local_path: ~/drupal/myvmsite
    destination: /var/www/drupalvm
    type: nfs
    create: true
    options_override:
      nfs_version: 4
      nfs_udp: false

  # Manually specify the /vagrant share to override the defaults
  - local_path: .
    destination: /vagrant
    type: nfs
    options_override:
      nfs_version: 4
      nfs_udp: false

The one that points to /var/www/drupalvm is the shared folder that is picked up at http://drupalvm.dev. As you want to edit the code on your host, and then sync it to where the Apache VirtualHost points, it's ofc required to have it shared.
The second that points to /vagrant is used primarily to get Drupal VM's provisioning/ directory to the VM when the user doesn't have ansible installed and everything is installed from within the VM, rather than from the host. The config.yml that you linked, only has this one which is why the install folder wasn't available on your host. @geerlingguy technically we could even drop this when ansible is found on the host?
Also @skorasaurus it might be worth checking if it works without specifying the NFS version, and just setting UDP
vagrant_synced_folders:
  # The default share.
  - local_path: ~/drupal/myvmsite
    destination: /var/www/drupalvm
    type: nfs
    create: true
    options_override:
      nfs_udp: false

  # Manually specify the /vagrant share to override the defaults
  - local_path: .
    destination: /vagrant
    type: nfs
    options_override:
      nfs_udp: false

@skorasaurus I updated my branch with a commit that defaults the nfs_udp to false. This should fix it without you having to edit vagrant_synced_folders at all.

Derp I had a typo that broke it, just pushed another commit.
Also thanks so much for helping test all my ramblings! It really helps out

@oxyc To clarify, what and how would you like me to test?
As I understand:
Do the following using this config
vagrant destroy
vagrant up ```
(measure the time it takes, report any errors in output, and check that drupalvm.dev is operating as expected)

and
repeat above with [this 2nd config](https://gist.github.com/skorasaurus/1479d1b1a3a44e6116295be4c08025b5) based on your [most recent commit](https://github.com/oxyc/drupal-vm/commit/75ce83f825b6008074a08726f073da5ab5a38f8d)?

@skorasaurus you could test:

vagrant halt
vagrant destroy
vagrant up

with this (if you have the most recent commit):
vagrant_synced_folders:
  # The default share.
  - local_path: ~/drupal/myvmsite
    destination: /var/www/drupalvm
    type: nfs
    create: true

It now defaults to TCP so it should work on all Linux distros, and even making it faster on at least my own macbook.
If if fails (but not otherwise) you can try:
vagrant_synced_folders:
  # The default share.
  - local_path: ~/drupal/myvmsite
    destination: /var/www/drupalvm
    type: nfs
    create: true
    options_override:
      nfs_version: 4

  # Manually specify the /vagrant share to override the defaults
  - local_path: .
    destination: /vagrant
    type: nfs
    options_override:
      nfs_version: 4

If UDP alone doesn't fix it, then we might have to specify the NFS version as 4. Sadly NFSv4 is not available on macOS so would be preferable if we could use the defaults from upstream Vagrant. But if there's no other choice, we probably need to add some logic that checks if the host OS is linux, and then sets v4 automatically.

The other change in my branch is that the Drupal codebase is now built in /tmp and then moved over to /var/www/drupalvm/drupal rather than being built directly in the shared directory. On my own laptop is decreased the build time from 150s to 100s. I'm guessing it's even more with a HDD.

I tested with the first config that you provided, and received the following error: https://gist.github.com/skorasaurus/9addb87af6524e1cec753292a282d6aa
will try with the latter one momentarily.

Hm @skorasaurus any chance you have ufw running on your host? https://github.com/geerlingguy/drupal-vm/issues/1204

ufw is not running.

Hi, I ran it once more again, (I did agrant destroy; i also did  git clean -fd if it makes a difference); rebooted using the config at https://github.com/geerlingguy/drupal-vm/issues/1476#issuecomment-316146246 (the first one) and it worked!
Unfortunately, it had taken about 66 minutes during the initial vagrant up.

Thanks! Good to know that it works at least.
This sounds more and more like the same issue as https://github.com/geerlingguy/drupal-vm/issues/1309.
That task that's slow only does this:
cp -r /tmp/drupal-project/. /var/www/drupalvm/drupal/

It should reproducible within these commands:
vagrant ssh

composer create-project drupal-composer/drupal-project:8.x-dev /tmp/drupal--stability dev --no-interaction

cp -r /tmp/drupal/. /var/www/drupalvm/drupal
# or any directory within `/var/www/drupalvm/`

Unfortunately as with #1309, I have no idea what could be causing such a slow cp task... :/

I understand this must be getting frustrating. Besides switching to rsync I don't have many suggestions unfortunately.
What I would do is test if this also happens with a bare Vagrantfile:
Vagrant.configure('2') do |config|
  config.vm.box = 'geerlingguy/ubuntu1604'

  config.vm.network :private_network,
    ip: '<IP_ADDRESS>'

  config.vm.synced_folder '.', '/vagrant_data',
    type: 'nfs',
    nfs_udp: false
end

Installing composer manually within the VM:
sudo apt-get install php php-xml php-mbstring php-zip
php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');"
php -r "if (hash_file('SHA384', 'composer-setup.php') === '669656bab3166a7aff8a7506b8cb2d1c292f042046c5a994c43155c0be6190fa0355160742ab2e1c88d40d5be660b410') { echo 'Installer verified'; } else { echo 'Installer corrupt'; unlink('composer-setup.php'); } echo PHP_EOL;"
php composer-setup.php
php -r "unlink('composer-setup.php');"

And then creating the project and copying to the NFS shared directory
php composer.phar create-project drupal-composer/drupal-project:8.x-dev /tmp/drupal --stability dev --no-interaction

time cp -r /tmp/drupal/. /vagrant_data/drupal

If this is slow, maybe someone in the vagrant issue queue has ideas about what could be causing it. Or stackexchange as it might be more of an NFS issue. In my case the cp -r task takes 41 seconds.

@oxyc
Thanks for your patience for following up so many times!
I did the first set of commands https://github.com/geerlingguy/drupal-vm/issues/1476#issuecomment-317820151 and it had taken 25 minutes for the cp command.

I'm hoping that the commit of the updated Drupal role will help with this. On my Mac, just as with @oxyc's testing, I noticed a marked improvement:
Before updating Drupal role:
real	8m7.036s
user	1m21.012s
sys	0m34.348s

After updating:
real	7m14.238s
user	1m18.963s
sys	0m35.378s

The 11% speedup here will likely be amplified on Windows or Linux where the share setup might not be as optimized...

Thank you again for your effort and help.
 Just to be sure, is this an okay config, right?
the 'vagrant up' had taken 45 minutes; definitely quicker than before 66 mins but obviously not the performance that we were looking and slower than when I did a cp in my ssh (https://github.com/geerlingguy/drupal-vm/issues/1476#issuecomment-317855153).
I haven't tried @oxyc 's  2nd suggestion yet

That's a correct config yes.
The 2nd suggestion you referred to would just be to try and determine that it's not Drupal VM related, and thus maybe asking in an issue queue with more knowledgable readers.
Something that might build the VM quicker is to run the composer create-project command on your host machine before provisoning. This would use the exact same config but instead of vagrant up:

cd to the root of your project
composer create-project drupal-composer/drupal-project:8.x-dev drupal --stability dev --no-interaction
vagrant up

Doing that would make Drupal VM skip those tasks and maybe it's faster to sync from the host to the VM than vice versa.

If the copy task alone takes 25min, 45min for the entire playbook doesnt sound unreasonable though. Maybe it's some NFS mount option that's needed or maybe it's just that HDDs are that much slower when it's NFS with these many files.

If the copy task alone takes 25min, 45min for the entire playbook doesnt sound unreasonable though. Maybe it's some NFS mount option that's needed or maybe it's just that HDDs are that much slower when it's NFS with these many files.

HDDs, especially if you're running antivirus, or have some other types of background processes which require any kind of data access, are waaay slower than SSDs. So much so that I'd posit anything under an hour is pretty decent. I tend to forget just how bad it is not booting off/using an SSD, but consider that any time you're writing thousands of little Drupal files, if the disk needs to pause for a bit, move the heads to another place on the disk, and read a file for some other task, then come back to where it's writing drupal files... that's like magnitudes slower than an SSD.
Random access kills HDD performance. So if you absolutely can't use/upgrade to SSD, the secondary recommendation would be to at least have all your VMs / vagrant / virtualbox stuff on a second volume (not boot volume)/separate physical disk.


.. with Typescript Property 'routing' does not exist on type 'Readonly<{ children?: ReactNode; }> & Readonly<{}>

I've installed  "@types/history": "^4.6.2","@types/jest": "^23.3.0", "@types/node": "^10.5.2","@types/react": "^16.4.6", "@types/react-dom": "^16.0.6", "@types/react-router": "^4.0.29", but I still get that error
import * as React from 'react';
import { inject, observer } from 'mobx-react';

@inject('routing')
@observer
export default class App extends React.Component {
  render() {
    const { location, push, goBack } = this.props.routing; // here

    return (

        Current pathname: {location.pathname}
        <button onClick={() => push('/test')}>Change url</button>
        <button onClick={() => goBack()}>Go Back</button>

    );
  }
}

Property 'routing' does not exist on type 'Readonly<{ children?: ReactNode; }> & Readonly<{}>

as workaround this works:
export default class App extends React.Component<any,any> {

This is the proper way to do this.
interface Props {
  routing?: RouterStore
}

export default class App extends React.Component<Props> {
}

@notVitaliy thanks but I get another error
...
export default withRouter(App)

referred to App
[ts]
Argument of type 'typeof App' is not assignable to parameter of type 'ComponentType<RouteComponentProps<any, StaticContext>>'.
  Type 'typeof App' is not assignable to type 'StatelessComponent<RouteComponentProps<any, StaticContext>>'.
    Type 'typeof App' provides no match for the signature '(props: RouteComponentProps<any, StaticContext> & { children?: ReactNode; }, context?: any): ReactElement<any> | null'.

This seems like a TypeScript question more than anything. Might be worth asking on StackOverflow: https://stackoverflow.com/
Will close for now until we can confirm it is an issue with this library.


Do not hide overlapped marbles

If you have two marbles in the same output, the "earlier" marble is completely hidden underneath the "later" marble. It appears that there is only one marble at that position, and it is unclear that there are actually multiple marbles at the same position.
I noticed this when looking at pausableBuffered. If you slide the pause false slider to the correct position, pausable and pausableBuffered appear to have the same behavior. This is confusing because pausableBuffered actually has overlapping marbles which can't be seen (until you open the source).
Here is pausable, where some marbles are absent from the result stream:

Compare with pausableBuffered, where marbles are present in the result stream (5, 6, 7) but appear to be absent due to overlap:

Yes, this is one of the next things to do https://github.com/staltz/rxmarbles/blob/master/TODO#L75
Currently working on animation, which is ~75% done.


⚠ Bofrost.lu has degraded performance

In 779f282, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 1218 ms

Resolved: Bofrost.lu performance has improved in 763d597 after 8 minutes.


Add allow-plain server ports attribute

Description on matching issue.
Need to add a doc entry.  Will push another commit for that.
This closes #9573

Will all the data received is re-read by UnixNetVC? I'm wondering what happens if TLS handshake fails at the very late stage of it. Will SNI actions be triggered? If yes, will everything done on SNI actions be reset on the plain connection?
If all the data will be re-read by UnixNetVC, parsing the "HTTP request" probably fails because the fake TLS data is invalid as an HTTP request format.

[approve ci fedora]

[approve ci centos]

I also want to riff of @maskit and say the consistency checking in processing the server ports should be extended to generate an error if both quic and allow-plain are enabled on the same server port.

Will all the data received be re-read by UnixNetVC? I'm wondering what happens if TLS handshake fails at the very late stage of it. Will SNI actions be triggered? If yes, will everything done on SNI actions be reset on the plain connection?

The allow-plain will only kick in if there is a TLS parse failure on the first packet.  No TLS-oriented hooks should be triggered.

Added a warning if both allow-plain and quic are specified.  Warnings seemed to be how the other checks went.


Fix compile error when building core package independently with Clang

Building the core package on OS X using stock Clang reports 3 errors because of missing class definition.
error: member access into incomplete type 'element_type' (aka 'Aws::Utils::Crypto::HashFactory')
error: member access into incomplete type 'element_type' (aka 'Aws::Utils::Crypto::HMACFactory')
Adding the necessary header files fixes the problem.

this has been updated internally and will be going out shortly thanks for your contribution!


Package for homebrew

I'm going to start looking at this today and put notes in.

Now solved with https://github.com/ksonnet/homebrew-tap.