live stream from youtube

Hello, I'm trying to read live stream from youtube but after 10 seconds the streaming blocks.
I tried the following URL : https://www.youtube.com/watch?v=d29O7DT6NHQ - https://www.youtube.com/watch?v=1EiC9bvVGnk.
I red your FAQ & Troubleshooting but I don't find a solution.
I'm working with the last version of Python (3.8 64 bits) and the version of Camgear on Windows 10.
Your code is very simple and works with simple youtube video and RSTP video without problem->
# import libraries
from vidgear.gears import CamGear
import cv2
stream = CamGear(source='https://www.youtube.com/watch?v=d29O7DT6NHQ', y_tube =True).start() # YouTube Video URL as input
# infinite loop
while True:
    frame = stream.read()
    # read frames
    # check if frame is None
    if frame is None:
        #if True break the infinite loop
        break
    # do something with frame here
    cv2.imshow("Output Frame", frame)
    # Show output window
    key = cv2.waitKey(1) & 0xFF
    # check for 'q' key-press
    if key == ord("q"):
        #if 'q' key-pressed break out
        break
cv2.destroyAllWindows()
# close output window
stream.stop()
# safely close video stream.

@Smart-Stadium This is a bug in OpenCV's FFmpeg backend which cannot handle hls/m3u8 streams provided by youtube-dl or pafy. Thereby, the real culprit here is FFmpeg which has following hls/m3u8 bug:
https://trac.ffmpeg.org/ticket/5348
https://trac.ffmpeg.org/ticket/6287
https://trac.ffmpeg.org/ticket/7804
Steps to Reproduce Bug:
First, Use youtube-dl to grab Youtube Video HLS m3u8 stream:
youtube-dl -g "https://youtu.be/d29O7DT6NHQ"

Then play that URL using FFmpeg's FFplay, it stops after few seconds:
ffplay "https://manifest.googlevideo.com/api/manifest/hls_playlist/expire/1591217239/ei/97fXXv3rA4mywgOv4brYCA/ip/<IP_ADDRESS>/id/d29O7DT6NHQ.906/itag/96/source/yt_live_broadcast/requiressl/yes/ratebypass/yes/live/1/goi/160/sgoap/gir%3Dyes%3Bitag%3D140/sgovp/gir%3Dyes%3Bitag%3D137/hls_chunk_host/r3---sn-bpb5oupj-qxae.googlevideo.com/playlist_duration/30/manifest_duration/30/vprv/1/playlist_type/DVR/initcwndbps/5040/mh/hQ/mm/44/mn/sn-bpb5oupj-qxae/ms/lva/mv/m/mvi/2/pl/24/dover/11/keepalive/yes/mt/1591195523/disable_polymer/true/sparams/expire,ei,ip,id,itag,source,requiressl,ratebypass,live,goi,sgoap,sgovp,playlist_duration,manifest_duration,vprv,playlist_type/sig/AOq0QJ8wRQIhAP_SCb7nBIMiIv3KvXim1H54wNHBFPVmCscvz4VieJp0AiBZX03sFC1cD3YYTBPnt2-sjxsZ_HnrFrG8s7XBg1tppQ%3D%3D/lsparams/hls_chunk_host,initcwndbps,mh,mm,mn,ms,mv,mvi,pl/lsig/AG3C_xAwRQIgcc8v2_DuEfBByN_g0YiCfRxLmDEmMH2u04G7WM_rXGYCIQCFHnW8SjP2drb0nENv9SIeReNXOygJXbMvO6xYfBhtRA%3D%3D/playlist/index.m3u8"

But if you play the same hls/m3u8 stream with MPV media player using command:
mpv "https://manifest.googlevideo.com/api/manifest/hls_playlist/expire/1591217239/ei/97fXXv3rA4mywgOv4brYCA/ip/<IP_ADDRESS>/id/d29O7DT6NHQ.906/itag/96/source/yt_live_broadcast/requiressl/yes/ratebypass/yes/live/1/goi/160/sgoap/gir%3Dyes%3Bitag%3D140/sgovp/gir%3Dyes%3Bitag%3D137/hls_chunk_host/r3---sn-bpb5oupj-qxae.googlevideo.com/playlist_duration/30/manifest_duration/30/vprv/1/playlist_type/DVR/initcwndbps/5040/mh/hQ/mm/44/mn/sn-bpb5oupj-qxae/ms/lva/mv/m/mvi/2/pl/24/dover/11/keepalive/yes/mt/1591195523/disable_polymer/true/sparams/expire,ei,ip,id,itag,source,requiressl,ratebypass,live,goi,sgoap,sgovp,playlist_duration,manifest_duration,vprv,playlist_type/sig/AOq0QJ8wRQIhAP_SCb7nBIMiIv3KvXim1H54wNHBFPVmCscvz4VieJp0AiBZX03sFC1cD3YYTBPnt2-sjxsZ_HnrFrG8s7XBg1tppQ%3D%3D/lsparams/hls_chunk_host,initcwndbps,mh,mm,mn,ms,mv,mvi,pl/lsig/AG3C_xAwRQIgcc8v2_DuEfBByN_g0YiCfRxLmDEmMH2u04G7WM_rXGYCIQCFHnW8SjP2drb0nENv9SIeReNXOygJXbMvO6xYfBhtRA%3D%3D/playlist/index.m3u8"

It plays just fine.

Workaround for this Bug:
Use different backend if available like Gstreamer:

But first check your OpenCV to be built with GStreamer support. You can easily check it by running print(cv2.getBuildInformation()) python command and see if output contains something similar as follows:
Video I/O:
 ...
     GStreamer:
       base:                      YES (ver 1.8.3)
       video:                     YES (ver 1.8.3)
       app:                       YES (ver 1.8.3)
...
```sh
---

Then, if Gstreamer support is present, then use this code:
```python
# import required libraries
from vidgear.gears import CamGear
import cv2

# Add YouTube Video URL as input source of your choice and change backend
stream = CamGear(source='https://youtu.be/d29O7DT6NHQ', y_tube=True, logging=True, backend=cv2.CAP_GSTREAMER).start()

# loop over
while True:

    # read frames from stream
    frame = stream.read()

    # check for frame if Nonetype
    if frame is None:
        break

    # {do something with the frame here}

    # Show output window
    cv2.imshow("Output Frame", frame)

    # check for 'q' key if pressed
    key = cv2.waitKey(1) & 0xFF
    if key == ord("q"):
        break

# close output window
cv2.destroyAllWindows()

# safely close video stream
stream.stop()
# safely close video stream.

And problem is gone. Goodluck!

I tried the sample code provided above with a different url: https://www.youtube.com/watch?v=_fqpYLM6SCw&ab_channel=TheAIGuy
I have Gstreamer installed and the version(1.18.1) is displayed when I typed cv2.getBuildInformation()
This is the error I'm experiencing:

@imranilyas First of all, you don't need to enable Gstreamer backend for non-live streams. Being said that there's a bug in CamGear which recently got fixed in testing branch. Therefore, To eliminate this error, you can install the latest testing branch as follows:
Remember on Windows OS: You'll also need to install Git for Windows (if you haven't already): https://gitforwindows.org/
  # first update remember to  update `youtube-dl`
  python -m  pip install -U youtube-dl

  # clone the repository and get inside
  git clone https://github.com/abhiTronix/vidgear.git && cd vidgear

  # checkout the latest testing branch
  git checkout testing

  # OR install with asyncio support
 python -m  pip install .

Then you can rerun your example again and it should work. Good luck.

@abhiTronix I tried what you suggested and I am still experiencing the same error I was before when testing with a YouTube livestream. This only occurs when I have backend=cv2.CAP_GSTREAMER. When I remove it, the normal timed videos run fine. However, I need the livestreams which is why I need Gstreamer. Is there something I have to do with Gstreamer in order to get it piped in?
Livestream url: https://www.youtube.com/watch?v=my5iauCqerU&ab_channel=ComedyCentral

@imranilyas What's the output of python -c "import vidgear; print(vidgear.__version__)" command?

Output is 0.1.9
When I check pip list, it says 0.2.0.dev2

@imranilyas There are two pip package installation sites installed on your computer. How did you installed vidgear, and did you used sudo?

@abhiTronix I installed vidgear through pip install. I did not use sudo.

@abhiTronix I installed vidgear through pip install. I did not use sudo.
In python37/Lib/site-packages/vidgear, the version.py file says 0.1.9 which is where I believe it's pulling the version number from.
In python37/Lib/site-packages there is a vidgear folder and a vidgear-0.2.0.dev2.dist-info folder.

@imranilyas Sorry, Yeah I was wrong, and unintentionally I gave you half instructions to solve this problem. You need  to use options={"FORCE_YTUBE_VAMODE":True} flag for these special kind of YouTube videos _(that doesn't have 1080p video+audio streams) _, as follows:
# import required libraries
from vidgear.gears import CamGear
import cv2

# enable special mode
options={"FORCE_YTUBE_VAMODE":True}
# Add YouTube Video URL as input source of your choice and change backend
stream = CamGear(source='https://www.youtube.com/watch?v=_fqpYLM6SCw&ab_channel=TheAIGuy', y_tube=True, logging=True, backend=cv2.CAP_GSTREAMER, **options).start()

# loop over
while True:

    # read frames from stream
    frame = stream.read()

    # check for frame if Nonetype
    if frame is None:
        break

    # {do something with the frame here}

    # Show output window
    cv2.imshow("Output Frame", frame)

    # check for 'q' key if pressed
    key = cv2.waitKey(1) & 0xFF
    if key == ord("q"):
        break

# close output window
cv2.destroyAllWindows()

# safely close video stream
stream.stop()

Try this code and this must solve your problem. Please report your result here too,

@abhiTronix I tried the above solution with the same YouTube video link and this is the error I receive, the same error as before. When I remove backend=cv2.CAP_GSTREAMER and have a YouTube livestream as the source, I get these errors after the livestream runs for a couple of seconds.
[tls @ 000001d3aac4c880] Unable to read from socket
[tls @ 000001d3aac6d980] Failed to send close message

I tried the above solution with the same YouTube video link and this is the error I receive, the same error as before.

@imranilyas It is working all fine here, even tested on different machines. I can even record everything for your convenience if you like.

When I remove backend=cv2.CAP_GSTREAMER and have a YouTube livestream as the source, I get these errors after the livestream runs for a couple of seconds.
[tls @ 000001d3aac4c880] Unable to read from socket
[tls @ 000001d3aac6d980] Failed to send close message

Yeah, that's why we are working on issue #160. You can wait for it as it will solve it.

@imranilyas Can you do following and try again:
sudo add-apt-repository ppa:gstreamer-developers/ppa
sudo apt-get update
sudo apt-get install gstreamer1.0*

@abhiTronix Sorry for the late response. I'm using Git/Bash which neither supports sudo nor apt-get. I upgraded by downloading from the source for both gstreamer(1.18.2) and OpenCV(4.5.0) and compiled OpenCV using Visual Studio 2017. However, it still did not work.
Then, I tried using the command prompt and it WORKED! Thank you for all your help. I'm still not sure why it works on the Command Prompt but doesn't on Git/Bash, but it 100% worked. Thank you again! I really appreciate your quick responses.

This issue cannot be resolved at this moment as vidgear as we need to patch FFmpeg itself first to progress further, thereby need we'll work on this after #148 is done. The required patch can be found here: http://git.videolan.org/?p=ffmpeg.git;a=commitdiff;h=a606f27f4c610708fa96e35eed7b7537d3d8f712

Temporarily Resolved in commit https://github.com/abhiTronix/vidgear/commit/bf1b8aabd0a21543570457e8aeb7775339611b18 and now no need to specify GStreamer backend(cv2.CAP_GSTREAMER) manually, CamGear automatically enforces it.

@Smart-Stadium I think #274 resolves this issue too.
Clone and Install development branch:
  # clone the repository and get inside
  git clone https://github.com/abhiTronix/vidgear.git && cd vidgear

  # checkout the latest testing branch
  git checkout development

  # install normally
  pip install .[core]

  # OR install with asyncio support
  pip install .[asyncio]

Test your code:
# import required libraries
from vidgear.gears import CamGear
import cv2

options = {"STREAM_RESOLUTION": "720p"}

# Add YouTube Video URL as input source (for e.g https://youtu.be/bvetuLwJIkA)
# and enable Stream Mode (`stream_mode = True`)
stream = CamGear(
    source="https://youtu.be/j1GLs_fMn1s", stream_mode=True, logging=True, **options
).start()

# loop over
while True:

    # read frames from stream
    frame = stream.read()

    # check for frame if Nonetype
    if frame is None:
        break

    # {do something with the frame here}

    # Show output window
    cv2.imshow("Output", frame)

    # check for 'q' key if pressed
    key = cv2.waitKey(1) & 0xFF
    if key == ord("q"):
        break

# close output window
cv2.destroyAllWindows()

# safely close video stream
stream.stop()

Successfully resolved and merged in commit: https://github.com/abhiTronix/vidgear/commit/dc26c00bcb4ec6e40645b4ad1aa1aedf4f0c6b43
Usage docs are available at: https://abhitronix.github.io/vidgear/v0.2.4-dev/gears/camgear/usage/#using-camgear-with-youtube-videos

Thank you for your job. My project is finished but as soon as I have a little bit time, I'll test.

I'm getting a Invalid NAL unit 0, skipping error. I have
stream = CamGear(
source="https://youtu.be/j1GLs_fMn1s", stream_mode=True, logging=True, **options
).start()
and sometimes the video will play for a couple seconds and then freeze and it won't play any more. The goal is I'm trying to get a live stream from youtube using openCV and then apply some background segmentation.

@abhiTronix ok sounds good! Thank you! Do you know how long it might take?

@abhiTronix ok sounds good! Thank you! Do you know how long it might take?

That might take a while. But you can use Deffcode right now as follows with CamGear, it's pretty simply actually:
# import required libraries
from vidgear.gears import CamGear
# import the necessary packages
from deffcode import FFdecoder

# Add YouTube Video URL as input source (for e.g https://youtu.be/uCy5OuSQnyA)
# and enable Stream Mode (`stream_mode = True`)
stream = CamGear(
    source="https://youtu.be/j1GLs_fMn1s", stream_mode=True, logging=True
).start()

# get Video's metadata as JSON object
video_metadata =  stream.ytv_metadata

# get data like `url`
source_url = video_metadata["url"]

# close CamGear
stream.stop()

# initialize and formulate the decoder for BGR24 output
decoder = FFdecoder(source_url, frame_format="bgr24").formulate()

# grab the BGR24  frame from the decoder
for frame in decoder.generateFrame():

    # check if frame is None
    if frame is None:
        break

    # Do here whatever you want

    # Show output window
    cv2.imshow("Output", frame)

    # check for 'q' key if pressed
    key = cv2.waitKey(1) & 0xFF
    if key == ord("q"):
        break

# close output window
cv2.destroyAllWindows()
# terminate the decoder
decoder.terminate()

Make sure you install deffcode and FFmpeg correctly on your machine. You can find related docs here: https://abhitronix.github.io/deffcode/latest/installation/

@Teagueporter This code will work for the time being. Please report any issues if found. Goodluck.

Thank you!

@abhiTronix Sorry for so many questions but does GStreamer have to be working? I have deffcode and CamGear and i'm using the same layout as the code you posted above but I'm still getting an Invalid Nal unit 0 after 3ish seconds

@Teagueporter I'm not seeing any deffcode logs except version. Paste the complete logs with verbose=True in FFdecoder.

@abhiTronix Sorry I get the error in my terminal, here is the screen shot of that error

.
What happens is the video will play for about 4 to 5 seconds but the invail nal unit error will happen almost as soon as I begin the program


Visualizing currently unvisualized properties of messages

Currently the listed properties of the messages below aren't visualized in any way. Visualizing this information may be quite useful, but I didn't get around to coming up with specs for visualizing them. Should think of how to visuliaze.

Goalpost.msg

team
side

Robot.msg

state
facing

Some spontaneous ideas on how to visualize these properties:

Goalpost.msg

side: Write Letter L or R hovering above the post
team: Color this letter or the post in the corresponding team color

Robot.msg

state: Write (abbreviated) state hovering above the marker
facing: Add a arrow representing the direction or color the marker boxes in two different colors/patterns
This is probably a bit harder, as we would need information about the own robot's orientation for this…


Red Hat - Associate sponsor

Review the content
Riseup Pad Link: TBD
Logo: TBD
Poster: TBD
Tasks

[ ] Review of content, @sayanchowdhury @rayandas @palnabarun @Sc0rpi0n101 @harisibrahimkv
[ ] Blog Creation, @Sc0rpi0n101 @harisibrahimkv
[ ]  Posting of blog on website, @bhansa @vamc-k
[ ]  Poster Creation, @iamaravindsekar

Timeline
Pre 4PM

[ ] Review of content, @sayanchowdhury @rayandas @palnabarun @Sc0rpi0n101 @harisibrahimkv
[ ] Blog Creation, @Sc0rpi0n101 @harisibrahimkv
[ ]  Posting of blog on website, @bhansa @vamc-k
[ ]  Poster Creation, @iamaravindsekar

Post 5PM

[ ] posting content and mailing, @WatchfulHitman @tankala

Communication Channel (if needed for your task)

[ ] Mailing List

[ ] InPyCon
[ ] X-Posts

[ ] All Local Python User Groups

[ ] Social Media

[ ]  Twitter
[ ]  Linkedin
[ ]  Mastodon
[ ]  Instagram
[ ]  Facebook

https://github.com/pythonindia/inpycon2024-tasks/issues/80

This is done.


[Version 1.0.3] Required rule does not work with custom validation rule

I noticed that when you build a validation rule using the function addovcustomrules, and you use the custom rule with a required rule on an element, the script will not validate the required rule. But will validate your custom rule skipping the required rule

This issue has been resolved in v1.0.4


Derive PartialEq trait except for streaming fields

I'd like to add PartialEq support to request/response structs for exactly the same reason with #932 .
(This changeset is based on #1059's re-format)

Thanks for organizing this PR! It's good to know that someone else has a use case for it.

Makes sense, thanks!


idlelib3.ToolTip error in Python 3.6

Hi Grant, thanks a lot for this cool script. I tried to run the script a lot of times with Python 3.6 from my mac terminal, but I have this error:
File "main.py", line 11, in
from idlelib.ToolTip import ToolTip
ModuleNotFoundError: No module named 'idlelib3'
With Python 2.7 the error was:
File "main.py", line 7, in
import queue
ImportError: No module named queue
How can I solve it?
Thanks!

Hi, they changed the names of the idlelib modules from uppercase to lower in 3.6
Just change:
from idlelib.ToolTip import ToolTip
to
from idlelib.tooltip import ToolTip

Thanks a lot Grant! Now it works.
But now the problem is about login: all my credentials are corrects, I don’t know why I have this error.

On 22 Jun 2017, at 16:49, Grant Williams<EMAIL_ADDRESS>wrote:
Hi, they changed the names of the idlelib modules from uppercase to lower in 3.6
Just change:
from idlelib.ToolTip import ToolTip
to
from idlelib.tooltip import ToolTip
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/grantwilliams/wg-gesucht-crawler/issues/3#issuecomment-310403346, or mute the thread https://github.com/notifications/unsubscribe-auth/AV80M0pF7daJng5NWJ4s8o-TlLUjBTupks5sGn8AgaJpZM4OCVqe.

Hi, you have the same error during login, or a different one? I tested with Ubuntu 16.04 with python 3.6, only changing the above line and it worked.

Hi Grant,
thanks for reply. When I try to login with the correct email and password I receive the message “WG-Gesucht website timed out, please try again later.”. Tried a lot of times but it doesn’t work. Another question: why the mobile number is required for login?
I’m using OSX 10.12.5 with Python 3.6.

On 24 Jun 2017, at 01:34, Grant Williams<EMAIL_ADDRESS>wrote:
Hi, you have the same error during login, or a different one? I tested with Ubuntu 16.04 with python 3.6, only changing the above line and it worked.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/grantwilliams/wg-gesucht-crawler/issues/3#issuecomment-310795017, or mute the thread https://github.com/notifications/unsubscribe-auth/AV80M2U5YoN5XN-WeXqwXvjuZvhk_Y_Xks5sHEuBgaJpZM4OCVqe.

Hi Grant,
I know that I’m annoying you with this stuff but it’s really important for me the tool you created! Could you please help me? Also: during the week I’m working at Factory, are you too? We can take a beer / coffee together, I want to say thanks to you.
Best,
Francesco

On 24 Jun 2017, at 11:39, Francesco Calabretta<EMAIL_ADDRESS>wrote:
Hi Grant,
thanks for reply. When I try to login with the correct email and password I receive the message “WG-Gesucht website timed out, please try again later.”. Tried a lot of times but it doesn’t work. Another question: why the mobile number is required for login?
I’m using OSX 10.12.5 with Python 3.6.

On 24 Jun 2017, at 01:34, Grant Williams<EMAIL_ADDRESS><EMAIL_ADDRESS>wrote:
Hi, you have the same error during login, or a different one? I tested with Ubuntu 16.04 with python 3.6, only changing the above line and it worked.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/grantwilliams/wg-gesucht-crawler/issues/3#issuecomment-310795017, or mute the thread https://github.com/notifications/unsubscribe-auth/AV80M2U5YoN5XN-WeXqwXvjuZvhk_Y_Xks5sHEuBgaJpZM4OCVqe.

Hi there,
If it's saying the website timed out, maybe wg-gesucht was down then, have you tried since? It's working for me on Windows10 and Ubuntu, but I don't have a Mac to try it out on...
And the phone number is just passed on when it applies for an apartment, you can just write a '0' or something if you don't want to give them your phone number.

Hi Grant,
What I discovered is that: with my flat connection (60mb) every time I receive the error of time-out, with Factory connection (100mb) I don’t have problem. Actually I don’t know why. Do you think that maybe I have to change some parameters in the script to avoid this error?
Best,
Francesco

On 4 Jul 2017, at 20:50, Grant Williams<EMAIL_ADDRESS>wrote:
Hi there,
If it's saying the website timed out, maybe wg-gesucht was down then, have you tried since? It's working for me on Windows10 and Ubuntu, but I don't have a Mac to try it out on...
And the phone number is just passed on when it applies for an apartment, you can just write a '0' or something if you don't want to give them your phone number.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/grantwilliams/wg-gesucht-crawler/issues/3#issuecomment-312934407, or mute the thread https://github.com/notifications/unsubscribe-auth/AV80M5zdwVDJJ9a-23CC4iWfuwWgOE4yks5sKoljgaJpZM4OCVqe.

Heya,
Not sure if you still need to use this, but I've changed it quite a bit, so it doesn't use Selenium anymore, maybe it works better for you now?


Styling via data attributes has broken

I recently updated a project from formsnap 0.2.0 to latest on npm as of today.  Styling via the data-fs labels no longer worked.  (https://www.formsnap.dev/docs/styling)  Reverting to version 0.2.0 resolved the issue.

It looks like data-fs-select is applied in "form-input.svelte", "form-radio.svelte", "form-select.svelte" (this one makes sense) and "form-textarea.svelte"  I was successfully able to apply styles to my input boxes using the data-fs-select data attribute, but I doubt that is intended :)


hidding / showing panels

You can hide the panels by clicking twice on the separator.
When clicking only once (to open it again) you get a strange effect. The place for panels is there but no panels is displayed
I think we can use only one click to close / open the panels

@lpatiny
it must be done using double click and not single click because of drag and drop events and I can not modify the way how the slider work.....


[MRG] Remove channel renaming & channel type settings; support latest MNE-BIDS

Channel names and types should all be handled via BIDS!

@jasmainak @agramfort feel free to merge :)

Thx @hoechenberger


Move to using ruamel.yaml

pyyaml, the library we currently use, froze development some years ago and only supports Yaml 1.1 - but the current standard, Yaml 1.2, is almost ten years old
This fork: https://yaml.readthedocs.io/en/latest/pyyaml.html fixes one issue #753, two issue ("YAML 1.2 no longer supports sexagesimals, so the string scalar 12:34:56 doesn’t need quoting"/"correct parsing of floating point scalars with exponentials") that I didn't know we had.

Interesting... I was looking at that recently but thought it was the same
thing. Yes let's do that. Will do as my in progress PR
On Mon, May 28, 2018, 2:42 PM Tom Swirly<EMAIL_ADDRESS>wrote:

pyyaml, the library we currently use, froze development some years ago and
only supports Yaml 1.1 - but the current standard, Yaml 1.2, is almost ten
years old
This fork: https://yaml.readthedocs.io/en/latest/pyyaml.html fixes one
issue #753 https://github.com/ManiacalLabs/BiblioPixel/issues/753, two
issue ("YAML 1.2 no longer supports sexagesimals, so the string scalar
12:34:56 doesn’t need quoting"/"correct parsing of floating point scalars
with exponentials") that I didn't know we had.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/ManiacalLabs/BiblioPixel/issues/838, or mute the
thread
https://github.com/notifications/unsubscribe-auth/AA6a6oRun2mVs0sOelCaCf791JuUBVQuks5t3EUOgaJpZM4UQgNx
.

That's the one. Though on Travis other tests are failing of course
On Mon, May 28, 2018, 3:24 PM Tom Swirly<EMAIL_ADDRESS>wrote:

Ah - is it just udp_test?
I find that test weirdly flakey. Sometimes it works for a week in a row -
other times it fails every time.
I need to deal with that.
On Mon, May 28, 2018 at 9:23 PM, Adam Haile<EMAIL_ADDRESS>wrote:

Fair...still need to figure out why the tests hang on my new laptop :P
On Mon, May 28, 2018, 3:21 PM Tom Swirly<EMAIL_ADDRESS>wrote:

I'm on that PR actually - so stay tuned. :-)
On Mon, May 28, 2018 at 9:14 PM, Adam Haile<EMAIL_ADDRESS>wrote:

Interesting... I was looking at that recently but thought it was the
same
thing. Yes let's do that. Will do as my in progress PR
On Mon, May 28, 2018, 2:42 PM Tom Swirly<EMAIL_ADDRESS>wrote:

pyyaml, the library we currently use, froze development some years
ago
and
only supports Yaml 1.1 - but the current standard, Yaml 1.2, is
almost
ten
years old
This fork: https://yaml.readthedocs.io/en/latest/pyyaml.html fixes
one
issue #753 <https://github.com/ManiacalLabs/BiblioPixel/issues/753
,
two
issue ("YAML 1.2 no longer supports sexagesimals, so the string
scalar
12:34:56 doesn’t need quoting"/"correct parsing of floating point
scalars
with exponentials") that I didn't know we had.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/ManiacalLabs/BiblioPixel/issues/838, or mute
the
thread
<https://github.com/notifications/unsubscribe-auth/
AA6a6oRun2mVs0sOelCaCf791JuUBVQuks5t3EUOgaJpZM4UQgNx>
.

—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
<
https://github.com/ManiacalLabs/BiblioPixel/issues/838#issuecomment-
392589335
,
or mute the thread
<
https://github.com/notifications/unsubscribe-auth/
AAPdstRgcmaTdY8Be4d05gN1s0uC_Qlyks5t3EyxgaJpZM4UQgNx
.

--
/t
https://tom.ritchford.com https://tom.ritchford.com
https://tom.swirly.com https://tom.swirly.com
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<https://github.com/ManiacalLabs/BiblioPixel/issues/838#issuecomment-
392590204>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/
AA6a6mSB8HhUV1f7LsNs4yqr4CSrApeuks5t3E5LgaJpZM4UQgNx>

.

—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
<
https://github.com/ManiacalLabs/BiblioPixel/issues/838#issuecomment-392590355
,
or mute the thread
<
https://github.com/notifications/unsubscribe-auth/AAPdspn2WaAEumCb2IbNjzxxlJcSl2vEks5t3E6WgaJpZM4UQgNx
.

--
/t
https://tom.ritchford.com https://tom.ritchford.com
https://tom.swirly.com https://tom.swirly.com
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/ManiacalLabs/BiblioPixel/issues/838#issuecomment-392590548,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AA6a6vLgtahML1Yx8-SheKBtyrc0VkP_ks5t3E7rgaJpZM4UQgNx
.

I'm mostly done on a revamp of that pull... :-)

Ah, ignore that.  :-)

Ignore what?

The previous comment was a duplicate.
On Tue, May 29, 2018 at 12:51 AM, Adam Haile<EMAIL_ADDRESS>wrote:

Ignore what?
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
https://github.com/ManiacalLabs/BiblioPixel/issues/838#issuecomment-392613637,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAPdsml2bOOKwKcKEw21zXUlQ-bB58VBks5t3H96gaJpZM4UQgNx
.

--
/t
https://tom.ritchford.com https://tom.ritchford.com
https://tom.swirly.com https://tom.swirly.com

Another candidate is https://github.com/crdoconnor/strictyaml which is referenced from here: https://arp242.net/weblog/yaml_probably_not_so_great_after_all.html

Unfortunately, installing strictyaml installs ruamei.yaml and that fails with an error that strongly appears to be a bug in ruamei's setup.py.
Putting this down for a bit, so much for "quickly fix this".


Improve the course on libraries

What is the standard structure of the exportable source-code library?
How the libraries are usually managed?
How the libraries are usually integrated into the programs?
What is the standard application-level package manager (e.g. Maven, NuGet)?

Two years later I came to the conclusion that openness was never a value here. The article Modern COBOL: Package Tutorial is still counting reads but only one person tried the package manager so far.


Added zIndex for timeline dropdown menu

Fixes #1268
Changes proposed in this pull request:

Added z-index for the sub-menu-wrapper

@sasicodes Can we use the tailwind utility class for that particular dropdown instead of adding new CSS?
@samuveth WDYT?


Removing the header

I would like to not have any headers in my table (only data itself), but the following code is leaving the first row of the table as blank (expecting the headers):
<BootstrapTable data={this.state.serviceData}
                                            striped
                                            hover
                                            keyField='id'
                                            className="col-hidden">
                                {serviceCols.map( column => {
                                        return (
                                            <TableHeaderColumn
                                                dataField={column}
                                                dataFormat={this.formatter}
                                                className="col-hidden"
                                            ></TableHeaderColumn>
                                        );
                                    })
                                }
                            </BootstrapTable>

Is there a way i could remove the header row at all ? (Issue #755) is not helping

why close it? does need any help for this?

@AllenFang figured out :). Similar to #755, .css file needed to imported too.


New profile page

You really put a lot of effort into this description! ;-)
I'd do something simple that we can quickly implement to have something. Then we can improve it over time.
So for v1, we need to see (in that order)

user avatar/logo (depending if it's a user or a company)
collectives he/she is part of
collectives he/she is supporting

The big question is what are we going to do with the namespace. Let's create a separate issue to discuss this.

lol i typed it but i closed the window without commenting :/

Should we add a way to store the user payment info from here (paypal/stripe) ?

Not for v1. But yes it would be good to have from there a way to edit avatar, bio, website and connected accounts in v2. We will add payment methods in v3. Step by step.
As a side note, I wonder if we can come up with something creative with the rounded avatar and a network of collectives. A bit like the default background image we have on the collective page. Not important for now but just a thought.

@xdamman On your side note: First thing that popped into my mind was D3.js, heres is an example of what I mean http://bl.ocks.org/jose187/4733747, of course instead of dots they would be rounded avatars.
Here's a more advance example, it looks a lot like the default background image we have on the collective page.
http://projects.flowingdata.com/tut/interactive_network_demo/

There might be interesting things to do with that in the future. A collective is a network of people. There is no organizational chart, it's instead an organizational web. Food for thoughts!

this is the public view, working on the editable one
Profile page with a lot of data

Profile page with a lot of data

This is the public view, working on the editable one
Profile page with a lot of data ( for a regular user)

Profile page with a lot of data (for organizations)

Profile page with little data

Profile page with no data

i really like this!
P.
piamancini.com
t: @piamancini
s: piamancini
OpenCollective https://opencollective.com
On Thu, Apr 28, 2016 at 4:19 PM, Ericku<EMAIL_ADDRESS>wrote:

This is the public view, working on the editable one
Profile page with a lot of data ( for a regular user)
[image: profile page full]
https://cloud.githubusercontent.com/assets/18429104/14904388/89a12eb2-0d6d-11e6-84bc-50104fccf5e8.png
Profile page with a lot of data (for organizations)
[image: profile page org]
https://cloud.githubusercontent.com/assets/18429104/14904403/a65aeeb2-0d6d-11e6-8a3f-6bd299d32072.png
Profile page with little data
[image: profile page little data]
https://cloud.githubusercontent.com/assets/18429104/14904413/b1446f92-0d6d-11e6-9a9a-5464f762fab0.png
Profile page with no data
[image: profile page no data at all]
https://cloud.githubusercontent.com/assets/18429104/14904426/bbb92058-0d6d-11e6-9636-5892cc561fe8.png
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
https://github.com/OpenCollective/OpenCollective/issues/53#issuecomment-215590576

Ship it!

I'm leaving the branch here for quick reference
https://github.com/OpenCollective/opencollective-website/tree/profile
Its not ready the backend is not implemented yet, @asood123 is taking care of this

@xdamman picked it up from me
On Wed, Jun 1, 2016 at 6:24 PM Ascari<EMAIL_ADDRESS>wrote:

I'm leaving the branch here for quick reference
https://github.com/OpenCollective/opencollective-website/tree/profile
Its not ready the backend is not implemented yet, @asood123
https://github.com/asood123 is taking care of this
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/OpenCollective/OpenCollective/issues/53#issuecomment-223143389,
or mute the thread
https://github.com/notifications/unsubscribe/AF0vPeAbcA9qHOyikDqWls5KAkw17tTRks5qHgangaJpZM4IRZa9
.


Add support for ON CONFLICT as a support for upserts in PostgreSQL

PostgreSQL supports upserts by ON CONFLICT clause provided
in versions 9.5 and above. This should fix issue #333

This is very nice feature. As we are planning to use latest version of postgres this will be really useful
@Team when can I expect it in trunk?

Awesome work! Sorry for taking a while to get to this. Pushing a new minor version out to npm today!


Support Rails CSS and JS include helpers

Override stylesheet_link_tag and javascript_include_tag helpers to delegate to Proscenium::SideLoad.append, which will improve the DX.

Resolved in 3c1f8b993a5767161935bb51bb60657a694c0540


Resource - Color Palette

Need Color palette for Resource page in Fundamental UI Website.

@jeannewalters
https://zpl.io/aNNPJea

migrate with better description


image is hazy after generate

I followed the guidance Fixing images that already exist, everything is set to default, and then I click "generate", but the image has a grayish and hazy feeling. Why is that?

@TITC
When using the Face Editor with img2img, the entire image will be rewritten in the following cases:

Denoising strength for the "img2img" is greater than 0.
When the value of "Denoising strength for the entire image" (Advanced Options - (5) Blend the entire image) in Face Editor is greater than 0.

The screen capture shows that the Denoising strength of img2img is set to 0, so it is possible that the value of "Denoising strength for the entire image" in the Face Editor is set to a value greater than 0.
Could you please check that setting?
Also, in img2img, the prompt needs to be specified. If you transfer the image from the PNG Info tab to img2img, it will paste the prompt used when it was generated by txt2img. You can use that as is, or if you do not rewrite the entire image, you can specify a prompt specifically for redrawing the face.
I think it is very likely that the reason the generated image is hazy is that you have not specified any prompts.
Thanks.

Thanks for your response.
The Denoising strength for the entire image is greater than 0 and I have not specified any prompts, that's true. The image is regenerated from an avatar with ControlNet canny type, it hasn't any prompts even I transfer the image from the PNG Info tab to img2img.

Now I simply copy and paste the prompt from image, and modify Denoising strength for the entire image=0,but seems still hazy at all image and there is a square area cover the face.

high quality, face portrait photo of 26 y.o european woman, wearing black dress, serious face, detailed face, skin pores, cinematic shot, dramatic lighting

Negative prompt: (deformed iris, deformed pupils, semi-realistic, cgi, 3d, render, sketch, cartoon, drawing, anime), text, cropped, out of frame, worst quality, low quality, jpeg artifacts, ugly, duplicate, morbid, mutilated, extra fingers, mutated hands, poorly drawn hands, poorly drawn face, mutation, deformed, blurry, dehydrated, bad anatomy, bad proportions, extra limbs, cloned face, disfigured, gross proportions, malformed limbs, missing arms, missing legs, extra arms, extra legs, fused fingers, too many fingers, long neck, BadDream, UnrealisticDream

Steps: 97, Sampler: DPM++ 2S a Karras, CFG scale: 7, Seed: 798713878, Size: 512x520, Model hash: a757fe8b3d, Model: chilloutmix_, Denoising strength: 0, Clip skip: 2, Version: v1.5.1

I am not imagining why the entire image is rewritten with this setting :thinking: , so can you give me the following 4 pieces of information?
1. Information about the image on the right side (the image after Face Editor has processed it) when "Show original image" is enabled.
There should be more information there than in the image on the left (the image generated by img2img).
 The left image with "Show original image" enabled.
This is the image generated by img2img. I would like to compare the difference between the image on the right side and the image on the left side.
3. Information displayed in the console at runtime.
For example:
number of faces: 1.                                                | 0/1 [00:00<?, ?it/s]
prompt for the face: high quality, face portrait photo of 26 y.o european woman, wearing black dress, serious face, detailed face, skin pores, cinematic shot, dramatic lighting

100%|██████████████████████████████████████████████████████| 9/9 [00:01<00:00,  5.93it/s]
Total progress: 10it [00:02,  4.58it/s]
Total progress: 10it [00:02,  5.51it/s]

4. The image generated when the Face Editor is disabled.

The problem seems solved after I changed the base model from chilloutmix to majicmixRealistic_v6, feel free to close the issue at any time, and thanks for your time to help me solve the problem.

Information about the image on the right side (the image after Face Editor has processed it) when "Show original image" is enabled.

high quality, face portrait photo of 26 y.o european woman, wearing black dress, serious face, detailed face, skin pores, cinematic shot, dramatic lighting

Negative prompt: Negative prompt: (deformed iris, deformed pupils, semi-realistic, cgi, 3d, render, sketch, cartoon, drawing, anime), text, cropped, out of frame, worst quality, low quality, jpeg artifacts, ugly, duplicate, morbid, mutilated, extra fingers, mutated hands, poorly drawn hands, poorly drawn face, mutation, deformed, blurry, dehydrated, bad anatomy, bad proportions, extra limbs, cloned face, disfigured, gross proportions, malformed limbs, missing arms, missing legs, extra arms, extra legs, fused fingers, too many fingers, long neck, BadDream, UnrealisticDream

Steps: 97, Sampler: DPM++ 2S a Karras, CFG scale: 7, Seed: 798713878, Size: 512x512, Model hash: a757fe8b3d, Model: chilloutmix_, Denoising strength: 0, Clip skip: 2, face_editor_enabled: True, face_editor_face_margin: 1.6, face_editor_confidence: 0.97, face_editor_strength1: 0.25, face_editor_strength2: 0, face_editor_max_face_count: 20, face_editor_mask_size: 1, face_editor_mask_blur: 3, face_editor_prompt_for_face: "", face_editor_apply_inside_mask_only: True, face_editor_apply_scripts_to_faces: False, face_editor_face_size: 512, face_editor_use_minimal_area: True, face_editor_ignore_larger_faces: True, face_editor_affected_areas: Face;Hair, face_editor_workflow: "{\n \"face_detector\": \"RetinaFace\",\n \"rules\": {\n \"then\": {\n \"face_processor\": \"img2img\",\n \"mask_generator\": {\n \"name\": \"BiSeNet\",\n \"params\": {\n \"fallback_ratio\": 0.1\n }\n }\n }\n }\n}", face_editor_upscaler: None, face_editor_tilt_adjustment_threshold: 20, Version: v1.5.1

The left image with "Show original image" enabled.
upload image->left image->image with face mask

Information displayed in the console at runtime.

number of faces: 1.                                                                                | 0/1 [00:00<?, ?it/s]
prompt for the face: high quality, face portrait photo of 26 y.o european woman, wearing black dress, serious face, detailed face, skin pores, cinematic shot, dramatic lighting

100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 25/25 [00:06<00:00,  3.69it/s]
Total progress: 26it [00:07,  3.46it/s]
Total progress: 26it [00:07,  3.60it/s]

The image generated when the Face Editor is disabled.


"Read the code" button nukes document contents

Only the user who clicked it can undo it (until they leave the page), otherwise contents are irrecoverable.
Surely this should create a new document or just link to the github file? It does not sound like a modify button, so it is a negative surprise for a user who joins an in-progress pad.

I understand your inconvenience but I also don't want to redesign this 3 years later

That's a disappointing response, tbh. I can offer some suggestions so the amount of redesign work on your end is low. It was easy enough to find where to change, I could maybe submit a simple PR:
https://github.com/ekzhang/rustpad/blob/cbc5df8b4b2914fa1f13e328b0acd7649985c723/src/App.tsx#L285-L288
So my potential ideas for this are:

Change the text to "Edit the code" or "Edit our code" or "Try our code sample" or something
Remove the button entirely.
Replace the button onclick with a link to a new pad with default contents (don't know if this is possible).

Are there any of these you would consider accepting?

If you had made this suggestion 3 years earlier I probably would've taken it, but honestly nothing has changed since then, and I don't really want to nitpick my past work.

I don't understand what you mean by past work given that you are still hosting rustpad.io and accepting PRs and issues. But ok.


Mouse click no response

VSCode Version: Code 1.18.1 (929bacba01ef658b873545e26034d1a8067445e9, 2017-11-16T18:32:36.023Z)

OS Version: Windows_NT x64 10.0.16299
Extensions:

Extension
Author (truncated)
Version

Bookmarks
ale
0.18.0

bracket-pair-colorizer
Coe
0.10.14

vscode-markdownlint
Dav
0.12.0

vscode-instant-markdown
dba
1.3.0

githistory
don
0.2.3

vscode-great-icons
emm
2.1.20

code-python-isort
fre
0.0.3

output-colorizer
IBM
0.1.2

easy-icons
jam
0.3.1

vscode-JS-CSS-HTML-formatter
lon
0.2.3

start-git-bash
McC
1.2.1

vscode-filesize
mkx
1.0.0

python
ms-
0.8.0

azure-account
ms-
0.2.2

team
ms-
1.122.0

vscode-docker
Pet
0.0.22

vscode-icons
rob
7.18.1

code-settings-sync
Sha
2.8.6

guides
spy
0.9.1

html-preview-vscode
tht
0.1.1

python
tht
0.2.3

bootstrap-3-snippets
wcw
0.1.0

(11 theme extensions excluded)

Steps to Reproduce:

Reproduces without extensions: Yes/No

Check out the part about submiting good issues
https://github.com/Microsoft/vscode/wiki/Submitting-Bugs-and-Suggestions

The selected content can not be clicked to cancel the selection

What does some time mean? Please file reproducable steps as we can not get this behavior on our machines.
Can you also reproduce without extensions?

@illinux
I cannot repro either.
Run code --disable-extensions and see if you can still repro this way with all extensions disabled.
Also, try our Insiders build from https://code.visualstudio.com/insiders where the issue might already be fixed.


It's Can working on win10 right?

Because this security tool calls win32 API in the execution process, so it will be windows defender or other antivirus software as a virus, I will provide a simple way to bypass in version 0.26, and then no longer provide the corresponding bypass strategy. The core code is open source, you can compile your own.

Because this security tool calls win32 API in the execution process, so it will be windows defender or other antivirus software as a virus, I will provide a simple way to bypass in version 0.26, and then no longer provide the corresponding bypass strategy. The core code is open source, you can compile your own.

how to use it do have course？

Give me more details.


Aaf

Created dataset and data loader for AAF dataset

To keep the repo / commit history clean, could you do the following two things before merging?

Squash all your commits into one, then rebase onto main so we only add one commit to the repo history. You can accomplish this with git reset <commit-sha> (choose the commit sha based on the last commit from main that's in your git history before your branching changes occurred) and then re-staging all your changes and committing them once.
Run the linters on your changes (see the pre-commit section in README.md in the latest version of main) to auto-format and standardize imports, catch small bugs, etc.


updated the landing page with particles effect

https://github.com/mdazfar2/HelpOps-Hub/assets/138752385/5149b4d6-d946-4731-b00b-9c2b8990e931
Added the effect on the landing page.

Hey @patelkiran185, please visit our website. Someone added this effect very well. By the way, thanks for making HelpOps-Hub better.

Don't forget to join our official Discord server so we can discuss more about HelpOps-Hub enhancement.
And also follow us on Linkedin
And please ⭐ our repo


Connect Motor PIN

I Have a Questions..
I not find where i need to connect Motor Driver ?
On RPi or Arduino ?

Can u say what pin i need to use for servo?

@annak1n Arduino is connected to the controller, see rc_keyboard_control.ino for actual pin numbers used in this project.


Opt-out of localForage

It seems like there is no need in third-party storage because chrome allows using unlimited storage.
So it's better to opt-out of localforage and uses standard chrome storage instead.

I think that it's better to create a separate module storage.js which will provide simple API to the outer world.
Something like this:
storage.get()
storage.set()

We can use whatever we want as a storage backend. It can be localForage or chrome.storage or whatever, we don't care, because storage.js provides only two functions and hides details of implementation, as I mentioned above.
`

Something like a new one database.js?

Actually we can use module database.js. The filename does not have to be storage.js.
I think that we better focus on other issues right now, but in a feature, we will use chrome.storage I think. It is always good when you can opt-out of some third-party libraries.


[W5.3b][W14-A1]Wong Peng Fai Shannon

@shanwpf
Some comments added. Please ack & close the PR after reading comments.


Feature Request: Ensure useUser Always Returns a User

Feature Request: Ensure useUser Always Returns a User
Description
Allow to have a useUser to always return a valid user object in certain context. This is particularly useful in authenticated pages where authentication is verified at the route level. In such cases, the presence of a logged-in user is assured, making additional checks for user existence unnecessary.

For now my "workaround" is to use the Tanstack Router context.
I have my restricted routes under a _restricted folder with this code :
export const Route = createFileRoute("/_restricted")({
  beforeLoad: async ({ context, location }) => {
    if (!context.user) {
      throw redirect({ search: { redirect: location.href }, to: "/login" });
    }
    const { user } = context;
    return { ...context, user };
  },
  component: () => <RestrictedRoot />,
});

And at the App level I have something like that :
const user = useUser();
return (
  <RouterProvider
    router={router}
    context={{
      queryClient,
      user: user.data ?? undefined,
    }}
  />
);

This allow me to narrow down the user inside my restricted route. I can then access the use with :
const { user } = Route.useRouteContext();

Instead of
const { user } = useUser();

And then I'm sure user is defined.

Hey @Ganbin, thanks for sharing! I don't know which version of the library you are using, but it is not relying on context anymore. useUser uses useQuery now and since the user can be null, it is safer to keep that in mind and treat it that way. However, if you want to modify something, feel free to just copy the source code and modify it in your project, it's pretty straigthforward.


Tweaks to improve cleanliness of code; removes warnings that appear in Eclipse

-Surrounded Scanner in runInteractive() in try blocks
-Tweaked precomputeFreqsCis() and removed code that doesn't appear to do anything (triggers a warning in Eclipse)
-Added @SuppressWarnings("preview") to loadModelImpl()

Alright, sorry. This was the first time I'd ever done a pull request since I usually just build my own tools. I'll fix it up when I have time.

Alright, I've adjusted it based on your feedback. That said, this version does include a git ignore file, but that's specifically so it won't push the classpath, project file, and other unwanted items. I think you can delete that on your end though.
Sorry for all of the mistakes! I hope this is useful to you.


drawing one way symbol

Right there does not seem to be a way to draw oneway symbol. The issue is that we need a way to tell if we want to draw text left to right or right to left. This is true for text and marker with marker-placement: line or text-placement: line;

There is no option to 'flip' markers or texts, you need 2 images of the arrow. If I remember correctly you could create street signs with something like this:
#road::arrow {
   [oneway=-1], [oneway=1] {
        marker-placement: line;
        marker-file: [oneway]=1 ? url('icon/arrow-right.png') : url('icon/arrow-left.png');
        marker-spacing: 180;
        marker-width: 11;
        marker-height: 5;
   }
}

@mtehver tried to use your code. Seems to work in a way.
But the marker does not always follow the line

Also there seems to be an issue with marker-spacing: 180; (same for text and others)
With it i almost see none

Thanks for reporting this. Should be fixed in develop branch (mobile-libs) now.

I can confirm (late though) that this is fixed


[Feat]: Allow a custom button order

Description
If all buttons are enabled in the consent and preferences modal, they are ordered like this:

Reject all
Accept all
Manage preferences (consent modal) / Save preferences (preferences modal)

I can use flipButtons to invert the order of those buttons. But what is not possible, as far as I can tell, is ordering the buttons like this:

Accept all
Reject all
Manage preferences (consent modal) / Save preferences (preferences modal)

Is this possible? If not, this would be great. In the default order, having reject all first isn't optimal, because the suggested action (from the website owner's perspective) should be accept all (while displaying both options with equal weight – equalWeightButtons is great!). But when the order is inverted, the secondary action is displayed first, which isn't ideal.
I know there are multiple layouts to chose from, but all of those could benefit from allowing for a custom button order.
Proposed solution
Some ideas:

Extend flipButtons to support an object or array that defines the button order (in addition to the existing behaviour).
Create a separate config under guiOptions for the order and allow flipButtons to invert it.

Additional details
No response

While I get your point, I also think that this is a very optional feature. I'd rather change the current default order to keep the size of the plugin as small as possible.

@orestbida Thanks for the reply – you're right about keeping the size (and also number of options small).
As a workaround, I can reorder the buttons using grid-colum-start (side-note: any reason you're using margin-top instead of row-gap to space those buttons?). That isn't ideal for accessibility, though, since the tab order will be different than the visual order.
I'm in favour of changing the default order. This will allow developers to put the Accept all button either at the top or the bottom using the flipButtons config, depending on the surrounding layout system. I don't think it ever makes sense to have the primary/suggested button in the middle.

Instead of flipButtons: true, maybe something like this buttonOrder: ['accept', 'reject',  'preferences'] so we can customize the order easily

In v3, I notice we can do these 2 layouts

But not this 3rd option which seems the most logical to me:

I'll open a new issue, it will be easier to track


Issue in documentation of Planner Connector

Hi Team,
We have followed  Document  and now able to building the connector and found where the issue is present .
It was mentioned in the above  Document which was incorrect,

For API permissions, among other required permissions under "Microsoft Graph", please make sure "User.Read" is added. "User.Read" permission is required in order to login the current user and retrieve relevant information.

Solution(Document need to be updated)
We need to have any one of Group.Read.All, Group.ReadWrite.All permission and grant permission for APP to use planner as per MS Document

Thanks for reporting!

@yshkrishna We have taken in this documentation bug and will prioritize accordingly to ensure that it is completed.


Closes #94 Add styles to hostile sink to reset Browser Sync notificat…

…ion styles

Coverage remained the same at 70.483% when pulling b747949e92b46e86c09ad94806ab7e397f8f4ea5 on feature/ESDS-BUILD-94--browser-sync-notification-sink-page-styles into fb4f322494d05cb11299748d3034d1249725856e on master.


Long term acquisition bug might still be there.

Hi,
While the last fix clearly made things better, I think there is still an issue with the time required to do the write of the NFTiff store increasing over the duration of a long acquisition.  I have put timers around the call to dataset._add_index_entry in the storage management thread and it seems to be steadily increasing.
The test case is pretty simple... I startup a headless micromanager using the demo driver and just do a big acquisition against it.  You should see increase with an event size of 50K.
I didn't see this before because my 1 hour long test runs fine.  I tried a two hour long acqusision and the problem returned.
Thanks,
Carl

Looks like this is the offending code in _add_index_entry in ndtiff:
        if axis_name not in self.axes.keys():

            self.axes[axis_name] = []

            self.axes_types[axis_name] = type(axes[axis_name])

            self.axes[axis_name].append(axes[axis_name])

      self.axes[axis_name] = sorted(list(set(self.axes[axis_name])))

Thanks for tracking that down. That makes sense why it eventually causes a problem and I agree that it should be constant time.
I believe the reason they are sorted is for the as_array function to put all the individual images in the correct order. I'm not recalling why they also have to be in a list instead of a set...but I don't think that's just extraneous. So it would seem the requirements as the code is written now is that they are 1) unique 2) sorted 3) in a list. Can bisect make this work with these requirements, without assuming that all the values are sequential?

There is also: https://grantjenks.com/docs/sortedcontainers/
This seems to be popular, but now unclear if its still in active development.
The set and sort on read approach would seem to me to be the closest to constant time.
Carl
On Jun 12, 2023, at 9:23 AM, Carl Kesselman @.***> wrote:
One option would be to put the sorting in as_array and you could cache the result?  Then you could just keep everything as a set and it would be better?
I’ve been thinking about this.. .the set is implemented as a hash table, which I think would be suboptimal for the time access. A more complicated result would be to detect if you are looking at a monotonically increasing sequence, store that as a list using just append, and convert to the set version if you violate the condition.
I think bisect will do the right thing. Rather than convert to a set, I would just use index to look up if the element already exists (log(N)) and the just insert (log(N))
Carl
On Jun 12, 2023, at 8:55 AM, Henry Pinkard @.***> wrote:
Thanks for tracking that down. That makes sense why it eventually causes a problem and I agree that it should be constant time.
I believe the reason they are sorted is for the as_array function to put all the individual images in the correct order. I'm not recalling why they also have to be in a list instead of a set...but I don't think that's just extraneous. So it would seem the requirements as the code is written now is that they are 1) unique 2) sorted 3) in a list. Can bisect make this work with these requirements, without assuming that all the values are sequential?
—
Reply to this email directly, view it on GitHubhttps://urldefense.us/v2/url?u=https-3A__github.com_micro-2Dmanager_pycro-2Dmanager_issues_615-23issuecomment-2D1587615419&d=DwMCaQ&c=qzHnJIRvjI6L-clJH8JwLQvf_Iq43fzikf6aoxZgMb8&r=sGCma2ufaUVT-N141kRIZQ&m=1XlPrjcykzNs0TsCujwQdJL9uVvudz8Ob3qDMc6nRSs2rrS0FoyUMuE_V-XyuGeI&s=65on-v0JwgcZrpEMvGNaenYtgWsicl_TBBbdUdcE_fs&e=, or unsubscribehttps://urldefense.us/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AA3OGXU7CYCJYVTCZZTFSMTXK43WXANCNFSM6AAAAAAZCAHNT4&d=DwMCaQ&c=qzHnJIRvjI6L-clJH8JwLQvf_Iq43fzikf6aoxZgMb8&r=sGCma2ufaUVT-N141kRIZQ&m=1XlPrjcykzNs0TsCujwQdJL9uVvudz8Ob3qDMc6nRSs2rrS0FoyUMuE_V-XyuGeI&s=JDbLF-s1j_MdAk-MSBI7A48zaTpg7tUZmy0QYFZ0WWw&e=.
You are receiving this because you authored the thread.Message ID: @.***>

I think bisect will do the right thing. Rather than convert to a set, I would just use index to look up if the element already exists (log(N)) and the just insert (log(N))

This seems like the most promising option. Do you want to try it out and see if it gives sufficient performance?

Sure… I’ll give it a shot.
Carl
On Jun 12, 2023, at 9:43 AM, Henry Pinkard @.***> wrote:
I think bisect will do the right thing. Rather than convert to a set, I would just use index to look up if the element already exists (log(N)) and the just insert (log(N))
This seems like the most promising option. Do you want to try it out and see if it gives sufficient performance?
—
Reply to this email directly, view it on GitHubhttps://urldefense.us/v2/url?u=https-3A__github.com_micro-2Dmanager_pycro-2Dmanager_issues_615-23issuecomment-2D1587690416&d=DwMCaQ&c=qzHnJIRvjI6L-clJH8JwLQvf_Iq43fzikf6aoxZgMb8&r=sGCma2ufaUVT-N141kRIZQ&m=UhQ8NQLtiOrf80WG3YWIyrsxtZfB8vPsSD_topTZlAkrze3qG4-64e_nWPMdQ4AF&s=Qwfb75JZlnwDtlY4CtIJ1goCx9jdag5EzRiZvTkh7us&e=, or unsubscribehttps://urldefense.us/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AA3OGXXWQJOQRDHOSLGSZH3XK5BLPANCNFSM6AAAAAAZCAHNT4&d=DwMCaQ&c=qzHnJIRvjI6L-clJH8JwLQvf_Iq43fzikf6aoxZgMb8&r=sGCma2ufaUVT-N141kRIZQ&m=UhQ8NQLtiOrf80WG3YWIyrsxtZfB8vPsSD_topTZlAkrze3qG4-64e_nWPMdQ4AF&s=GtnhVTirkInuZxMUrrl22PCCrkRJyaMpE6jT_subdXA&e=.
You are receiving this because you authored the thread.Message ID: @.***>

Ok…
If you are willing to use the sorted containers library, then there is an order log(N) solution that requires very few code changes.
Basically store index as a SortedSet, add elements one at a time, and then in one place in as_array, you need to add a list() function.
Carl
On Jun 12, 2023, at 10:23 AM, Carl Kesselman @.***> wrote:
Actually, I take it back…..
bisect.insort_right(a, x, lo=0, hi=len(a), *, key=None)
bisect.insort(a, x, lo=0, hi=len(a), *, key=None)
Similar to insort_left(), but inserting x in a after any existing entries of x.
This function first runs bisect_right() to locate an insertion point. Next, it runs the insert() method on a to insert x at the appropriate position to maintain sort order.
To support inserting records in a table, the key function (if any) is applied to x for the search step but not for the insertion step.
Keep in mind that the O(log n) search is dominated by the slow O(n) insertion step.
So while this would be several times better than the current approach, it will still be order N in the number of axis values, which I think is going to be a problem for the time axis.  I think the simplest option is to keep this as a set, which will support constant time insertion and then you pay the price when you want as_array.  This will shift the cost from creation to reading, but in total, the cost for doing the conversion of set to list and than sorting once if as_array is called, will still be a lot less than what is currently there.
Otherwise, I think the only viable solution is to use the proper SortedList library which will use a btree, or some such thing while providing a list interface.
Carl
On Jun 12, 2023, at 10:07 AM, Carl Kesselman @.***> wrote:
Sure… I’ll give it a shot.
Carl
On Jun 12, 2023, at 9:43 AM, Henry Pinkard @.***> wrote:
I think bisect will do the right thing. Rather than convert to a set, I would just use index to look up if the element already exists (log(N)) and the just insert (log(N))
This seems like the most promising option. Do you want to try it out and see if it gives sufficient performance?
—
Reply to this email directly, view it on GitHubhttps://urldefense.us/v2/url?u=https-3A__github.com_micro-2Dmanager_pycro-2Dmanager_issues_615-23issuecomment-2D1587690416&d=DwMCaQ&c=qzHnJIRvjI6L-clJH8JwLQvf_Iq43fzikf6aoxZgMb8&r=sGCma2ufaUVT-N141kRIZQ&m=UhQ8NQLtiOrf80WG3YWIyrsxtZfB8vPsSD_topTZlAkrze3qG4-64e_nWPMdQ4AF&s=Qwfb75JZlnwDtlY4CtIJ1goCx9jdag5EzRiZvTkh7us&e=, or unsubscribehttps://urldefense.us/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AA3OGXXWQJOQRDHOSLGSZH3XK5BLPANCNFSM6AAAAAAZCAHNT4&d=DwMCaQ&c=qzHnJIRvjI6L-clJH8JwLQvf_Iq43fzikf6aoxZgMb8&r=sGCma2ufaUVT-N141kRIZQ&m=UhQ8NQLtiOrf80WG3YWIyrsxtZfB8vPsSD_topTZlAkrze3qG4-64e_nWPMdQ4AF&s=GtnhVTirkInuZxMUrrl22PCCrkRJyaMpE6jT_subdXA&e=.
You are receiving this because you authored the thread.Message ID: @.***>

Two hour run… rock solid.  Not a frame dropped, no increase in add_index time.
I can clone and do a PR on NDTiff if you like.  Changes are really small.
Carl
On Jun 12, 2023, at 3:13 PM, Carl Kesselman @.***> wrote:
Ok…
If you are willing to use the sorted containers library, then there is an order log(N) solution that requires very few code changes.
Basically store index as a SortedSet, add elements one at a time, and then in one place in as_array, you need to add a list() function.
Carl
On Jun 12, 2023, at 10:23 AM, Carl Kesselman @.***> wrote:
Actually, I take it back…..
bisect.insort_right(a, x, lo=0, hi=len(a), *, key=None)
bisect.insort(a, x, lo=0, hi=len(a), *, key=None)
Similar to insort_left(), but inserting x in a after any existing entries of x.
This function first runs bisect_right() to locate an insertion point. Next, it runs the insert() method on a to insert x at the appropriate position to maintain sort order.
To support inserting records in a table, the key function (if any) is applied to x for the search step but not for the insertion step.
Keep in mind that the O(log n) search is dominated by the slow O(n) insertion step.
So while this would be several times better than the current approach, it will still be order N in the number of axis values, which I think is going to be a problem for the time axis.  I think the simplest option is to keep this as a set, which will support constant time insertion and then you pay the price when you want as_array.  This will shift the cost from creation to reading, but in total, the cost for doing the conversion of set to list and than sorting once if as_array is called, will still be a lot less than what is currently there.
Otherwise, I think the only viable solution is to use the proper SortedList library which will use a btree, or some such thing while providing a list interface.
Carl
On Jun 12, 2023, at 10:07 AM, Carl Kesselman @.***> wrote:
Sure… I’ll give it a shot.
Carl
On Jun 12, 2023, at 9:43 AM, Henry Pinkard @.***> wrote:
I think bisect will do the right thing. Rather than convert to a set, I would just use index to look up if the element already exists (log(N)) and the just insert (log(N))
This seems like the most promising option. Do you want to try it out and see if it gives sufficient performance?
—
Reply to this email directly, view it on GitHubhttps://urldefense.us/v2/url?u=https-3A__github.com_micro-2Dmanager_pycro-2Dmanager_issues_615-23issuecomment-2D1587690416&d=DwMCaQ&c=qzHnJIRvjI6L-clJH8JwLQvf_Iq43fzikf6aoxZgMb8&r=sGCma2ufaUVT-N141kRIZQ&m=UhQ8NQLtiOrf80WG3YWIyrsxtZfB8vPsSD_topTZlAkrze3qG4-64e_nWPMdQ4AF&s=Qwfb75JZlnwDtlY4CtIJ1goCx9jdag5EzRiZvTkh7us&e=, or unsubscribehttps://urldefense.us/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AA3OGXXWQJOQRDHOSLGSZH3XK5BLPANCNFSM6AAAAAAZCAHNT4&d=DwMCaQ&c=qzHnJIRvjI6L-clJH8JwLQvf_Iq43fzikf6aoxZgMb8&r=sGCma2ufaUVT-N141kRIZQ&m=UhQ8NQLtiOrf80WG3YWIyrsxtZfB8vPsSD_topTZlAkrze3qG4-64e_nWPMdQ4AF&s=GtnhVTirkInuZxMUrrl22PCCrkRJyaMpE6jT_subdXA&e=.
You are receiving this because you authored the thread.Message ID: @.***>


[discuss] Configuration and API Key documentation

Question
The Configuring page talks about the precedence of configuration sources, and how to generate the config file, but doesn't explain where to get the API key that is needed for ecctl init (this page for ESS and this section for ECE). It might be good to mention that above the ecctl init steps. For ESS we can link directly to https://cloud.elastic.co/deployment-features/keys .
I also think we'd prefer to de-emphasize the --user and --pass options (which I don't think work at all for ESS). But I didn't want to make those changes without discussing it first.

We should fix the lack of mentions on how to get the API key for sure.
Regarding the --user and --pass flags, I believe that's still something that's very relevant for our ECE customers and also something that is used for other internal tools that rely on the username and password still.
What ideas did you have in mind to de-emphasize the --user and --pass flags?

What ideas did you have in mind to de-emphasize the --user and --pass flags?

I was thinking that we could find different examples of flags to use under the "Before you begin" section, that apply to both ESS and ECE users. Looking at the output of ecctl help, I'm not sure what those would be. Maybe just add --api-key as well?
I just got confused initially about whether I would need a username and password, since the API key wasn't mentioned until I got to the Paste your API Key step.

I was thinking that we could find different examples of flags to use under the "Before you begin" section, that apply to both ESS and ECE users. Looking at the output of ecctl help, I'm not sure what those would be. Maybe just add --api-key as well?

We could add --output and --api-key as examples in that document.
I think we could possibly add a link to https://cloud.elastic.co/deployment-features/keys in the ecctl init command after the ESS option has been select to ease things out.
$ ecctl init
Welcome to Elastic Cloud Control (ecctl)! This command will guide you through authenticating and setting some default values.

Missing configuration file, would you like to initialise it? [y/n]: y

Select which type of Elastic Cloud offering you will be working with:
  [1] Elasticsearch Service (default).
  [2] Elastic Cloud Enterprise (ECE).
  [3] Elasticsearch Service Private (ESSP).

Please enter your choice: 1

Using "https://api.elastic-cloud.com" as the API endpoint.

Select a region you would like to have as default:

  GCP
  [1] us-central1 (Iowa)
  [2] us-east1 (S. Carolina)
  [3] us-east4 (N. Virginia)
  [4] us-west1 (Oregon)
  [5] northamerica-northeast1 (Montreal)
  [6] southamerica-east1 (São Paulo)
  [7] australia-southeast1 (Sydney)
  [8] europe-west1 (Belgium)
  [9] europe-west2 (London)
  [10] europe-west3 (Frankfurt)
  [11] asia-northeast1 (Tokyo)
  [12] asia-south1 (Mumbai)
  [13] asia-southeast1 (Singapore)

  AWS
  [14] us-east-1 (N. Virginia)
  [15] us-west-1 (N. California)
  [16] us-west-2 (Oregon)
  [17] eu-central-1 (Frankfurt)
  [18] eu-west-2 (London)
  [19] eu-west-1 (Ireland)
  [20] ap-northeast-1 (Tokyo)
  [21] ap-southeast-1 (Singapore)
  [22] ap-southeast-2 (Sydney)
  [23] sa-east-1 (São Paulo)

  Azure
  [24] eastus2 (Virginia)
  [25] westus2 (Washington)
  [26] westeurope (Netherlands)
  [27] uksouth (London)
  [28] japaneast (Tokyo)
  [29] southeastasia (Singapore)

Please enter your choice: 2

Create a new ESS API Key (https://cloud.elastic.co/deployment-features/keys) or
Paste your API Key and press enter:

I think we could possibly add a link to cloud.elastic.co/deployment-features/keys in the ecctl init command after the ESS option has been select to ease things out.

This would be great! Even if people aren't reading the docs, they'll know what to do.


Rename TaskResult to TaskRun

Following up on https://github.com/RealOrangeOne/django-tasks/discussions/19, I figured I would go through and try renaming things to see how it feels in comparison (as per the Django docs, "code talks").
I don't love TaskRunStatus, it might be better as just TaskStatus, but that loses the association with the TaskRun/TaskResult/whatever else it might end up being named.
Mainly just opening to see what everyone thinks!

TaskRunStatus is a bit wordy, but overall I still find it easier to reason about than TaskResult.

Closing as the naming is going in another direction (per #19)


USB Mapping

Hi there,
I was looking through the Kexts and I noticed you included 3 usb mapping kexts. Do I need all 3 (USBMap.kext, UTBMap.kext, and USBToolBox.kext)?
I'm asking as I'm trying to get my usb c 3 hub working and it sometimes work. HDMI out seems to be working. The hub usb ports works somewhat.
I did disable Thunderbolt 3 in the bios to get hot plugging working however the usb ports some what work.

yes, u need all three, u can make ur own using the dortania guide, if u do, do not include the dummy map kext as this will hinder booting

Do you know if I have to map the usb ports on my usb c hub?

No, you shouldn't they should come up automatically, all you need to do is map all ports on the computer

Oh ok got it.


you should be able to manually swipe a snack bar off-screen

https://www.google.com/design/spec/components/snackbars-toasts.html#snackbars-toasts-usage
"They exit by being swiped off-screen"

Maybe the snack bar should be a Dismissable.


[CAMEL-11798] Connector API assumes flat classpath

Thanks for the PR it has been merged. Do you mind closing this?


logging: parametrize workspace sku and resource specific tags

Overview/Summary
I trying to get existing ALZ production environment to be managed with Bicep and do it on way that WhatIf functionality would result tell that code is in sync with environment.
This PR fixes some Get-AzResourceGroupDeploymentWhatIfResult results similarly than #190 on environment where workspace sku pergb2018 is used and where automation account and analytics workspace have their own tags which are not used in elsewhere.
Related issues which are not fixed by this one:

https://github.com/Azure/arm-template-whatif/issues/176
https://github.com/Azure/arm-template-whatif/issues/251

This PR fixes/adds/changes/removes

Add parameter parLogAnalyticsWorkspaceSkuName to logging module and default to PerNode
Add parameter parAutomationAccountTags to logging module and default to same value than parTags
Add parameter parLogAnalyticsWorkspaceTags to logging module and default to same value than parTags

Breaking Changes
N/A
Testing Evidence
Before this PR:
Get-AzResourceGroupDeploymentWhatIfResult ...

The deployment will update the following scope:

Scope: /subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-euw-shared-analytics

  ~ Microsoft.Automation/automationAccounts/aauto-euw-masterWorkspace-lalinked [2019-06-01]
    - tags.dataclassification: "confidential"
    x properties.sku.name:     "Basic"

  ~ Microsoft.OperationalInsights/workspaces/law-euw-masterWorkspace-476b2b [2020-08-01]
    - tags.sourceproject:     "Management"
    ~ properties.sku.name:    "pergb2018" => "PerNode"

After this PR:
Those changes was not shown.
As part of this Pull Request I have

[x] Read the Contribution Guide and ensured this PR is compliant with the guide
[x] Checked for duplicate Pull Requests
[ ] Associated it with relevant GitHub Issues
[ ] (ALZ Bicep Core Team Only) Associated it with relevant ADO Items
[x] Ensured my code/branch is up-to-date with the latest changes in the main branch
[x] Performed testing and provided evidence.
[ ] Updated tests (if required) Unit - Linting - E2E (End-To-End)
[x] Updated relevant and associated documentation (e.g. Contribution Guide, Module READMEs, Wiki Docs etc.)
[ ] If relevant, created or updated Code Tours here

@jtracey93 thanks for good comments. Updated now.

@jtracey93 ah sorry, missed that comment. Now also parameters files are updated.
However I was wondering that is that actually considered as breaking change that now analytics workspace default sku will change from earlier hardcoded value PerNode to new default PerGB2018 ?

@jtracey93 Jack Tracey FTE ah sorry, missed that comment. Now also parameters files are updated.
However I was wondering that is that actually considered as breaking change that now analytics workspace default sku will change from earlier hardcoded value PerNode to new default PerGB2018 ?

Thanks. Yup, this will be a breaking change 😔

/azp run e2e


Dealing multiple cards only shows the last 1

I love this module!
I have a card stack that all players can see - the idea is there are 4 cards face up on the table.
When I deal from the deck to this stack, on the dealer side all 4 cards are displayed, but on the player side only the last card is displayed. I would have expected the same - 4 cards displayed.
Module: 0.1.8
Foundry: 11.315

Been busy with actual job work, but wanted to leave a comment to let you know I have seen your issue, and I do intend to figure out a solution.  This has been on my TODO list, in fact, but it's going to be a hairy fix, so I've been kicking this can down the road for a while.  By the token, I'm also open to ideas?
Thanks for the callout - will fix.

This isn't pressure, just want to add a +1 for this request. This module is amazing and has such great potential.  So thank you. I wouldn't even consider using the cards in foundry if it weren't for this mod. I can imagine there are a lot of challenges and variables for all type of use cases. In my case this is the same issue I'm running into. I'm running Curse of Strahd and if there's way to show the players 5 face down cards that I control and reveal one by one it would be amazing. I have zero coding experience so I don't have any solutions unfortunately, but I fully support this fix / add. And again thanks for working on this stuff.

@joomux The latest release should take care of this (well, the original bug noted here), as it groups multiple displayed cards into one view, and likewise shares them in one view.
@davidhawbecker I've got an item on my personal Stretch Goals list that I've named "Watch me Flip", which I believe is what you're suggesting here.  You'll be pleased to hear that, other than important bug fixes (if they arise), this feature is essentially my next priority :)


Support sort pushdown

Is your feature request related to a problem or challenge?
It seems there is no pushdown for sort order yet in the TableProvider. It seems this could also lead to wrong results if filters and limit is used without specifying the sort order.
Describe the solution you'd like
Either extend supports_filters_pushdown to allow indicating if sorting is supported or add a new negotiation method for sorting specifically. I'm looking for guidance on a solid approach.
Describe alternatives you've considered
No response
Additional context
I wonder if there has been thinking on other pushdown options that could be interesting? Down the line, full query/Substrait pushdown also seems interesting. However, it's less clear how to negotiate partial support in that case.

Thank you for bringing this up @backkem
Sort Pushdown?
When you say "sort pushdown" does that mean providing a sort order to TableProvider::scan?
Is the idea that the table providers have some faster way to sort than what is built into DataFusion?
Most of the sort based optimizations are done after TableProvider::scan has been called, so I am not sure how useful a pushed down sort would be
Existing ordering
There is a similar idea (maybe it is what you mean by sort order push down) where sources can tell DataFusion about any pre existing sort orders the data may have (e.g. because a parquet file was written with some sort order)
The TableProvider trait does not have any way to communicate this information directly as it is not used until physcal planning, but the ExecutionPlan returned by TableProvider::scan does allow setting this via  ExecutionPlan::output_ordering
You can see an example of how this is hooked up in the built-in  ListingTable TableProvider (via ListingTableConfig via ListingOptions via file_sort_order

I think there is merit to both cases. My use-case falls more under the former. My table provider wraps a remote DB and I want to fetch only a part of the table, for a simple pagination case. This combines a filter and limit to significantly reduce how much data needs to be transferred from the DB to be combined with other data in-process. Without limit, I exceed latency goals. In this case, the rows that should be returned are dependant on the sort order since the limit statement is restrictive enough that the result otherwise becomes arbitrary.
I know this may be somewhat atypical of a use-case. Happy to hear if this can be accommodated or not.
When working with remote DBs there are also other pushdown opportunities. For example: pushdown of merges across tables in the same DB. Hence why I was pondering about in the direction of 'execution plan federation'. I've seen similar ideas mentioned in a Substrait talk.

I just ran across #970 which touches on query federation. Though, while I'd be happy to explore that further, support for sort would already be a decent start.

The sort/limit usecase makes sense. Thank you for the explanation
One thing you could do is create a add custom optimizer passe (OptimizerRule and PhysicalOptimizerRule) that propagates the sort/limit information down into your scan.

It looks like we'll need to to support Joins over remote sources after all. Maybe we'll try to implement an ExecutionPlan based on the DataLoader concept. Hopefully that'll help us meet latency requirements when a restrictive limit is set.
Regarding Sort: I do agree (reading between the lines) that this pushdown direction feels like a slippery slope/feature creep. The query federation approach mentioned above seems more flexible. I'll close this for now.

The TableProvider trait does not have any way to communicate this information directly as it is not used until physcal planning, but the ExecutionPlan returned by TableProvider::scan does allow setting this via ExecutionPlan::output_ordering

@alamb  ExecutionPlan::output_ordering seems can not support both asc and desc for one column, because TableProvider doesn't pass sort option to scan execution plan.
#[async_trait]
pub trait TableProvider: Sync + Send {
    async fn scan(
        &self,
        state: &SessionState,
        projection: Option<&Vec<usize>>,
        filters: &[Expr],
        limit: Option<usize>,
        sort: &[Expr],
    ) -> Result<Arc<dyn ExecutionPlan>>;
}

We could pass sort exprs to scan, then users can construct ExecutionPlan based on sort exprs.

We could pass sort exprs to scan, then users can construct ExecutionPlan based on sort exprs.

I think the challenge is that DataFusion currently treats the sort order from an ExecutionPlan like "if it has a sort order, I will try and use it" rather than "I will try and push Sort into the scan"
Instead, DataFusion will introduce SortExec to resort the data if that is necessary to answer the query.
In order to "push" sorts into ExecutionPlans / scans, we would need some way to help DataFusion figure out if it should push the sort into the scan, or use a Sort Exec afterwards
For example, it is not clear which of the following plans is better as it depends on how the Sort within ExecutionPlan was implemented
SortExec
  Filter
   Scan (no sort)

vs
  Filter
   Scan (Sort in the Scan)

Depending on how selective the filter, it may be better to do the scan / filter and then sort.
of course in this case the filter is likely pushed down to the scan too, but I think in general the same issue still applies
For this usecase, I suggest adding a custom optimizer pass that does the sort pushdown you want and can take advantage of the details of what the underlying source is to make these choices

Your analysis make sense, thanks for explanation!

We have a similar use-case for pushing down TopK down to a column.
The idea is to push down TopK to a column, so we can reduce the scan by only fetching/decoding rows matching the TopK rows.
   TopK
      Scan

->
   Scan (RowFilter column in (TopK column))

The core problem here is: how far do you want to go with allowing the TableProvider to express compute ability versus a more full-blown form of query federation. The question is: where do you draw the line in expanding the TableProvider trait. Do we allow pushing down sort, aggregation, UDFs, etc.. ?
It may be a good idea to come up with a clear rule of where to draw this line.
One option would be to fully lean into the query federation idea: provide a good framework for that, provide basic implementation with filter/limit/sort pushdown out of the box and foster development of more complex federation cases (such as for remote DBMSs) out of tree.
One notable case regarding the complexity of this kind of pushdown is when you start combining joins with limit/sort pushdown. Not all tables may have the needed columns to sort/filter on. In that case you'd need something like what the Velox docs call Dynamic Filter Pushdown to avoid full table scans.

The core problem here is: how far do you want to go with allowing the TableProvider to express compute ability versus a more full-blown form of query federation. The question is: where do you draw the line in expanding the TableProvider trait. Do we allow pushing down sort, aggregation, UDFs, etc.. ?
It may be a good idea to come up with a clear rule of where to draw this line.
One option would be to fully lean into the query federation idea: provide a good framework for that, provide basic implementation with filter/limit/sort pushdown out of the box and foster development of more complex federation cases (such as for remote DBMSs) out of tree.

I agree that this sounds like a good idea to explore further

Is the idea that the table providers have some faster way to sort than what is built into DataFusion?

As I understand @backkem wanted to load data from an external datasource and delegate some sorting operations to it.
An executing query like ORDER BY col LIMIT 10 will result in load all the data from an external DB into DataFusion.
And it could be avoided by providing an information about sorting

For now I'm experiencing with another approach in datafusion-federation.
@backkem Could you please provide more details of how did you handle sort issue using it?

The federation repo turns (part of) the query plan back into SQL. In the simple case, the query only uses table providers of one remote DBMS. In that case the entire query will be forwarded. In more complex cases, the federation repo splits the plan in pieces that can be sent to each corresponding remote DB. DataFusion pieces together the results to resolve the final query.

A usecase from discord https://discord.com/channels/885562378132000778/1166447479609376850/1261096613565304884
Basically if you have multiple indexes that can provide the data in sorted order, there is no way to have datafusion tell you what a "useful order" might be


Fix typo

At the same time, we fix the build process on Netlify.
For now, we have this:

Even with this change, we have the same issue: fetch() fails on the build because we get a URL with a non-supported protocol. Why? We are building the URL using import.meta.url as a base: https://github.com/color-js/color.js/blob/3e20e82d774641871bc13780d1cb13a03108f756/data/spaces.js#L8 In Node, it returns the protocol that fetch() doesn't support.
Even if the site works fine for now, it's because the previous build is being served. All the builds starting from this commit will fail.

Since this PR doesn't fix the build on Netlify, I will close it in favor of #600, which fixes both issues.


Fixed Resolve error message

Updated the error message for resolving log associated with Guard record.
Defect: https://jazz07.rchland.ibm.com:13443/jazz/web/projects/CSSD#action=com.ibm.team.workitem.viewWorkItem&id=668412


runtime-safe webhook handlers

What I'm trying to achieve
Build a Saleor app with webhooks in runtime-safe manner.
Describe a proposed solution
Type-safe code is achieved with import type from the generated GraphQL code.
Allow runtime-safe code with zod schemas generated as part of saleor-app-sdk.
import { zodOrderPayload } from "@saleor/app-sdk/handlers/payloads";
// ...api handler declaration...
// within the first lines of the API handler:
const orderPayload = zodOrderPayload.parse(body);
// now orderPayload is both type-safe and runtime-safe

P.S. If payload from API means the query field of the webhook and is hard to pregen and bake into saleor-app-sdk then at least describing the best/common/fast/easy way to generate those zod schemas would already improve DX somewhat.
PP.S. Tree-shaking should help shaking off zod if the schemas are not used. Or at least the schemas could be put in a separate saleor-app-sdk-zod

Hey
I don't think we will follow this direction:

GraphQL guarantees proper structure. The only issues you need to care about is that some fields can be nullable, but it doesn't need Zod.
Apps are shipped with their own codegen. SDK is a set of tools, queries, mutations -> are part of client (app) codebase. We can't predict what queries will be used and there can be an infinite number of possible zod schemas

I'm very much into "making everything safe", but SDK looks like just a wrong place to do it.
I suggest you implement such thing in your own codebase. You can look into Typescript->Zod or GraphQL->Zod converters to build schemas once your graphQL definitions change


Uncaught exception 'Illuminate\Contracts\Container\BindingResolutionException'

Hey,
when I install a fresh version of Laravel 5.1 through the Installer and run phpunit I get:
Fatal error: Call to undefined method PHPUnit_Framework_TestResult::warningCount() in /home/vagrant/.composer/vendor/phpunit/phpunit/src/TextUI/ResultPrinter.php on line 185
PHP Fatal error:  Uncaught exception 'Illuminate\Contracts\Container\BindingResolutionException' with message 'Target [Illuminate\Contracts\Debug\ExceptionHandler] is not instantiable.' in /home/vagrant/Private/test/vendor/laravel/framework/src/Illuminate/Container/Container.php:745
Stack trace:
#0 /home/vagrant/Private/test/vendor/laravel/framework/src/Illuminate/Container/Container.php(627): Illuminate\Container\Container->build('Illuminate\\Cont...', Array)
#1 /home/vagrant/Private/test/vendor/laravel/framework/src/Illuminate/Foundation/Application.php(674): Illuminate\Container\Container->make('Illuminate\\Cont...', Array)
#2 /home/vagrant/Private/test/vendor/laravel/framework/src/Illuminate/Foundation/Bootstrap/HandleExceptions.php(154): Illuminate\Foundation\Application->make('Illuminate\\Cont...')
#3 /home/vagrant/Private/test/vendor/laravel/framework/src/Illuminate/Foundation/Bootstrap/HandleExceptions.php(79): Illuminate\Foundation\Bootstrap\HandleExceptions->getExceptionHandler()
#4 /home/vagrant/Private/test/vendor/ in /home/vagrant/Private/test/vendor/laravel/framework/src/Illuminate/Container/Container.php on line 745

Same error on php7.0 and php5.6.

What version of php?

Only 4.x, and 5.0 are known to work with laravel.

5.1+ don't work correctly atm.

php 5.6.13 and laravel 5.1.23

What version of phpunit?

PHPUnit 4.8.19

Hmmm. Thanks for reporting. Ping @taylorotwell.

when I install a fresh version of Laravel 5.1 through the Installer

The installer doesn't generate .env, and most likely due to APP_KEY not being set due to the missing file, PHPUnit throws this exception.
Could you try running:
composer run-script post-root-package-install
php artisan key:generate

and run phpunit again.

@crynobone the .env file was generated with the installer, see:
Phillipps-MacBook-Pro ~/Private/Code/test$ cat .env
APP_ENV=local
APP_DEBUG=true
APP_KEY=V4Y9bY6YFVHzPQEALF2qzfz3IqbUAuDW

DB_HOST=localhost
DB_DATABASE=homestead
DB_USERNAME=homestead
DB_PASSWORD=secret

CACHE_DRIVER=file
SESSION_DRIVER=file
QUEUE_DRIVER=sync

MAIL_DRIVER=smtp
MAIL_HOST=mailtrap.io
MAIL_PORT=2525
MAIL_USERNAME=null
MAIL_PASSWORD=null
MAIL_ENCRYPTION=null

Works fine with composer create-project.

Weird, tried it with 'composer create-project' and got the same error.

When i try it inside the vagrant vm, then it works, lol. But the version installed by the laravel Installer still fails.

could it be something install in your composer global that could trigger the problem. That would explain why it work fine in vagrant vm.

I dont think so because the installer version fails and the composer create-project doesn't fail in the vm. But what's the difference?

Another weird thing is: I have to use 'sudo' on the laravel installer one when i run phpunit.

I solved it!
First I had to remove the vendor folder and install everything again, then I had to remove phpunit globally and set an alias for phpunit so it uses the local phpunit from the vendor folder. Now everything work.

The removal of vendor folder did not work for me. However, I was able to resolve the issue by upgrading phpunit from 4.8.19 to 5.0.

@husonghua I had to remove it because of wrong permissions on the binary. Reinstalling through composer set the right permissions then.

@phillippOhlandt, what version of PHPUnit are you using then? I'm having this problem when I run unit tests.
Laravel 5.1, PHPUnit 5.1.

Looks like a project my project lives inside had an autoloader which used their version of PHPUnit's files which were PHPUnit 5.0. Upgraded them to PHPUnit 5.1 and my issues are gone.

I solved this issue in my computer changing the phpunit version in my composer.json file.
(I have composer installed in my home directory, so I edited ~/.composer/composer.json file)
"require": {
...
"phpunit/phpunit": "~4.0"
}

Cause is the default version when you create a project (via composer or laravel new)
Then:
composer update

And done

I use Debian with phpunit 5.1.3 installed as a command in my system and I have the same problem:
Call to undefined method PHPUnit_Framework_TestResult::warningCount() in /usr/share/php/PHPUnit/TextUI/ResultPrinter.php on line 185
How can I solve?

I had the same problem. Updating to 5.1.3 fixed it for me.

I had the same problem and solved it by upgrading to phpunit 5.3.2. I suspect it has less to do with the version than with re-installing phpunit after maybe composer missed something the first time.

is there no real fix for this error yet?

ResultPrinter.php issue has been fixed in newer versions. My issue is fixed after updating version of phpunit to 5.0 in composer.json and composer update.
"phpunit/phpunit": "~5.0",

I can confirm that updating in composer.json works.
My issue was:

laravel 5.2.* application with homestead
composer.json defined phpunit version to be ~4.0
local (host) system: linux, phpunit version 4.8.23
vagrant (guest) system: phpunit version 5.1.3 (installed with sudo apt-get install -y phpunit in {project-root}/after.sh)

phpunit would work when called on my local system, but not inside vagrant vm. Inside the vm it would fail with the message PHP Fatal error:  Uncaught Illuminate\Contracts\Container\BindingResolutionException: Target [Illuminate\Contracts\Debug\ExceptionHandler] is not instantiable.
After changing composer.json to the below and running composer update, everything works fine again. Remember that I didn't touch my installed phpunit versions.
Update composer.json:
"require-dev": {
   [...]
   "phpunit/phpunit": "~5.0",
   [...]
}

Thanks for your hints. Helped me a lot.

@GrahamCampbell another example of why this issue still exists, people still running into it way after you've closed this.  I even have this duplicated with phpunit 6


add publishBlueSky.js file

Description
This PR tries to resolve issue #21 by removing facebook and twitter code for posting and adding code for BlueSky.

@bkimminich sir Could you pls review the pr and changes made

Nice! I'll give it a try over the next few days! 👍

First glance feedback, it seems you took the Reddit mechanism as a template? Could you look at the Twitter mechanism instead, where the message text is randomized from a set of options? The Reddit mechanism does the same, but additionally gets a fixed title passed in for the post. BlueSky and Twitter don't have titles, so you can just stick to the message text randomly chosen.


1password: 1.9.1 -> 1.11.2

(This is my first nixpkgs PR, so feel free to correct any errors I make in drafting this PR.)
This PR updates 1password from version 1.9.1 to version 1.11.2. As you can see in the release notes, since version 1.11.1 1password has been distributing a binary for apple_universal instead of darwin_amd64 to support M1 Macs. If that is a problem, we can revert this to version 1.11.0, which is the last version that has darwin_amd64. I don't have a Mac machine handy to test this on MacOS, unfortunately.
I fetched the SHAs with
nix-prefetch-url --unpack https://cache.agilebits.com/dist/1P/op/pkg/v1.11.2/op_linux_386_v1.11.2.zip
nix-prefetch-url --unpack https://cache.agilebits.com/dist/1P/op/pkg/v1.11.2/op_linux_amd64_v1.11.2.zip
nix-prefetch-url --unpack https://cache.agilebits.com/dist/1P/op/pkg/v1.11.2/op_linux_arm_v1.11.2.zip
nix-prefetch-url --unpack https://cache.agilebits.com/dist/1P/op/pkg/v1.11.2/op_apple_universal_v1.11.2.pkg

and got
path is '/nix/store/wifdcgnjn9p5ljh3vlgdfbfm2vsg3r9j-op_linux_386_v1.11.2.zip'
0rh5bakj9qd43cf6wj5v46a3h98kcwqyc0f1yw72wvcacvjycyjz
path is '/nix/store/28z2c9zw03pfq7cadwcfm3fhj3rq3956-op_linux_amd64_v1.11.2.zip'
00nf0cb8cxk1pvzr1wq778wvikzrlzy38r3rzkq44whdpdj50jzx
path is '/nix/store/fpklzjc39wiw24jc1h2q2jyvsyk1zr03-op_linux_arm_v1.11.2.zip'
1gv282z49bj3ln5na4wb1z5455a64cyd54fp5i96k8shaxd0apxf
path is '/nix/store/qydgwga60z8w3p37ml2csh92505ic76y-op_apple_universal_v1.11.2.pkg'
14j9071ckd7dy7b8687231rbgyxb96iwwdlpxwyngcq57vhl7n7j

Motivation for this change
Upgrade the 1password CLI to the latest version (I needed some new features for my dotfiles).
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[x] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)david@nixos:~/git/nixpkgs/ > ./result/bin/op --version
1.11.2

21.11 Release Notes (or backporting 21.05 Relase notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 131497 at 12db3df2 run on x86_64-linux 1

1 package built successfully:

_1password

1 suggestion:

warning: missing-phase-hooks
installPhase should probably contain runHook preInstall and runHook postInstall.
Near pkgs/applications/misc/1password/default.nix:31:3:
   |
31 |   installPhase = ''
   |   ^

Hey @marsam and @joelburget, is there anything else I need to do to move this forward?

Sorry for the delay @jdreaver -- looks good to me

Awesome, thanks @joelburget! There's no rush here, I was just worried I was misunderstanding one of these status checks and you all weren't actually getting pinged for a review yet.

Thanks for checking that SHA @marsam! I checked it on Linux. I wonder why it is different when run on Darwin. I committed your suggested change.


First bunch of localization commits

It's glad to see you PR @fhebel, I was waiting for yesterday in Gitter after I read your email

Before I review the code, could you please mention what you are trying to fix or enhance, if there more than one thing small PRs will be better

The code is pretty good. The change can be separate into to PRs: one for contact areas translation and one for the remaining. But it is ok, I can understand all the changes.

@thiennn let me review it then ...

@thiennn @hishamco I believe I resolved all issues ...

Thank you so much

You're welcome. As soon as I my shop is in production I'll share more ...


complete peakcan driver

Rewrote the pcan driver with all its functionalities. But someone needs to test its functionality, cause I dont own a pcan.

maybe someone with a pcan can test if its working.

I do have PCAN, so I may be able to try this out


Fix unit test failures

Add tag to tensor_float_32_test to prevent its running on pip where it will fail due to being unable to import xla_test from tensorflow.compiler.tests

Fixes test failure introduced with https://github.com/tensorflow/tensorflow/commit/ce26e300f41d831606bf31b9095ef769486c23cf