[WIP] tooling: Add all pytests checker

Signed-off-by: Ryan Northey<EMAIL_ADDRESS>Commit Message: tooling: Add all pytests checker
Additional Description:
another breakout from #15833
Risk Level:
Testing:
Docs Changes:
Release Notes:
Platform Specific Features:
[Optional Runtime guard:]
[Optional Fixes #Issue]
[Optional Deprecated:]
[Optional API Considerations:]

coverage is here https://storage.googleapis.com/envoy-pr/16031/tooling/index.html


Job Superseding

Update the job model to handle the concept of jobs superseding one another.
Add root_id, is_superseded, superseded_id, superseded_by_id, and delete_superseded as fields to the job model.
Update JobManager.create_job() to accept a job that is being superseded by the new job.

Closed due to being merged into master


feat(components): slider

tests are coming.

the current commit implements only the slider widget from FAST and does not use the slider-label.
it mimics the CSS-based vivid-2 component markers instead.
there is no pin (i.e., the tooltip displaying the current value on top of the thumb)


Documentation cleanup

use chassis consistently
run as root user for first time user

Signed-off-by: Hui Kang<EMAIL_ADDRESS>

partially address #119

Once you line-wrap the last line I commented on, I will merge this. Thanks @huikang!

@mestery thanks for your review.


some font or sign can not display

some font or sign can not display
version 3.1.0-beta.1

.enableAnnotationRendering(true) // render annotations (such as comments, colors or forms)

I was getting invalid characters like  i, Ç, Ş, Ö, Ü etc. I tried above solution but still this issue was coming. Does anyone knows how to fix this?

.enableAnnotationRendering(true) // render annotations (such as comments, colors or forms)

resolved


FAQ, working with LLM endpoints and explaining why config-list is a list

Making the docs clearer about how LLM API endpoint config works.
Related issue number
Closes #749 and #97
Checks

[x] I've included any doc changes needed for https://microsoft.github.io/autogen/. See https://microsoft.github.io/autogen/docs/Contribute#documentation to build and test documentation locally.
[N/A] I've added tests (if relevant) corresponding to the changes introduced in this PR.
[x] I've made sure all auto checks have passed.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (6cf5bb0) 33.19% compared to head (2936566) 32.96%.
 Consider uploading reports for the commit b2d6e99 to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1451      +/-   ##
==========================================
- Coverage   33.19%   32.96%   -0.23%
==========================================
  Files          41       41
  Lines        4950     4950
  Branches     1135     1135
==========================================
- Hits         1643     1632      -11
- Misses       3177     3186       +9
- Partials      130      132       +2

Flag
Coverage Δ

unittests
32.96% <ø> (-0.19%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

@AaronWard , @sonichi please check the update


Re: Download livestream from the start

ins_v2.txt
Fill in this template completely. Issues not following this template will be closed and ignored.
Check the boxes below by filling [ ] with an x so it looks like [x].
Use the Preview button to ensure the template is filled in correctly.

[x] I am using the latest version of PyInstaLive: 3.2.4.
[x] I have installed either Python 2.7.x or 3.5+: 3.9
[ ] I have read the README.
[ ] I have read the FAQ.
[ ] I have checked all existing issues, none of which could solve my issue.

I am using:

[x] Windows 10
[ ] Windows 8.1
[ ] Windows 8
[ ] Windows 7
[ ] Linux (distribution: )
[ ] macOS (version: )
[ ] Other (device & OS name: )

Question
For the previous request of download the previous seconds or minutes of an ongoing instagram live (Download livestream from the start #24), I have written a python script, which I uploaded, to accomplish this by reading the "*downloads.json" file. The instalive stream seems to be started from some "000900" ,  "000934" or "001001" segment. Then I merge downloaded files manually. Would you be possible to embed such function in your code?

If this works and that @dvingerh can make a quick integration, it would be fantastic👍
Especially with Instagram rate limits changes since some months that have complicated the permanent scripts.

This is an ffmpeg-related issue. Unless the user saves their stream as a replay, DASH stream rewinding is not possible

Nice. It works, but it is a bit slow.
Final merge command is something like this:
cat init.m4v $(ls -vx  <stream_id>_0-*.m4v) > video.mp4 &&
cat init.m4a $(ls -vx  <stream_id>_0-*.m4a) > audio.m4a &&
ffmpeg -i video.mp4 -i audio.m4a -c copy output.mp4

Nice. It works, but it is a bit slow.

I'm not sure if linear search can use multithread.
I have found that when a user join the livestream, the timestep can be less than 2000 and it will sucks.
If this situation is included, the parse will be much slower.

Oh wow this is wonderful, I never knew this was possible with IG Lives. This is would amazing to have for livestream downloading via PyInstaLive. Do you know when dash_playback_url expires? Is it immediately after the IG Live or 24 hours later? I know for a fact it expires after over 48-72 hours.

insaio.txt
An aiohttp version of the script.
It checks about 150 urls per seconds.
Obviously it will take more than hours if you miss thousands of segments.

@cykac04
Once you find the first segment you can find all subsequent segments by the following method:

Download init.m4v

Find the first segment somehow, i.e. 1200.m4v
concat init.m4v and 1200.m4v > init_1200.m4v
run: ffprobe -v error -show_entries stream=duration_ts -of default=nw=1:nk=1 init_1200.m4v
output: 2401

Download 2401.m4v
concat init.m4v and 2401.m4v > init_2401.m4v (there is no need to concat earlier segments)
run: ffprobe -v error -show_entries stream=duration_ts -of default=nw=1:nk=1 init_2401.m4v
output: 2499

repeat

There may be an even simpler way, but this works. The trick is finding the first segment. I've been researching the dash manifests and there may be a way to get the earliest manifest by including a "start=" query string in the manifest URL, but their time keeping seems wonky. I'll keep looking into it.

@cykac04
Once you find the first segment you can find all subsequent segments by the following method:

Download init.m4v
Find the first segment somehow, i.e. 1200.m4v
concat init.m4v and 1200.m4v > init_1200.m4v
run: ffprobe -v error -show_entries stream=duration_ts -of default=nw=1:nk=1 init_1200.m4v
output: 2401
Download 2401.m4v
concat init.m4v and 2401.m4v > init_2401.m4v (there is no need to concat earlier segments)
run: ffprobe -v error -show_entries stream=duration_ts -of default=nw=1:nk=1 init_2401.m4v
output: 2499
repeat

There may be an even simpler way, but this works. The trick is finding the first segment. I've been researching the dash manifests and there may be a way to get the earliest manifest by including a "start=" query string in the manifest URL, but their time keeping seems wonky. I'll keep looking into it.

17900932982065128.zip
I have a stream getting error.
I get 2400 from ffprobe but the correct segment is 2501

cat init.m4v $(ls -vx  <stream_id>_0-*.m4v) > video.mp4 &&
cat init.m4a $(ls -vx  <stream_id>_0-*.m4a) > audio.m4a &&
ffmpeg -i video.mp4 -i audio.m4a -c copy output.mp4

Anyone now how can merge this ? I tried but i think it's not working on windows 10.
18129352513204028.zip

cat init.m4v $(ls -vx  <stream_id>_0-*.m4v) > video.mp4 &&
cat init.m4a $(ls -vx  <stream_id>_0-*.m4a) > audio.m4a &&
ffmpeg -i video.mp4 -i audio.m4a -c copy output.mp4

Anyone now how can merge this ? I tried but i think it's not working on windows 10.
18129352513204028.zip

Keep in mind you can't run the commands in Windows CMD or Powershell as is because some of those commands and the syntax is not supported. The easiest thing to do would be to download Git and install it accordingly. Open up Git Bash and make sure your working directory within Git Bash is the same location where those m4v and m4a files are. Then you could run the following snippet, replacing stream ID with your stream id (its the folder name).
cat init.m4v $(ls -vx<PHONE_NUMBER>3204028_0-*.m4v) > video.mp4 &&
cat init.m4a $(ls -vx<PHONE_NUMBER>3204028_0-*.m4a) > audio.m4a &&
ffmpeg -i video.mp4 -i audio.m4a -c copy output.mp4

cat init.m4v $(ls -vx  <stream_id>_0-*.m4v) > video.mp4 &&
cat init.m4a $(ls -vx  <stream_id>_0-*.m4a) > audio.m4a &&
ffmpeg -i video.mp4 -i audio.m4a -c copy output.mp4

Anyone now how can merge this ? I tried but i think it's not working on windows 10.
18129352513204028.zip

Keep in mind you can't run the commands in Windows CMD or Powershell as is because some of those commands and the syntax is not supported. The easiest thing to do would be to download Git and install it accordingly. Open up Git Bash and make sure your working directory within Git Bash is the same location where those m4v and m4a files are. Then you could run the following snippet, replacing stream ID with your stream id (its the folder name).
cat init.m4v $(ls -vx<PHONE_NUMBER>3204028_0-*.m4v) > video.mp4 &&
cat init.m4a $(ls -vx<PHONE_NUMBER>3204028_0-*.m4a) > audio.m4a &&
ffmpeg -i video.mp4 -i audio.m4a -c copy output.mp4

Thank you. I already have git and It's working perfectly right now.

@cykac04
That is odd.
How much time elapsed after the live ended before you attempted to download the segments?
Did the ffprobe method work for the other segments?
Did the 2400.mkv link return 404?
If downloaded, was it a 0 byte file?
Did you try slightly less or more than 2400? Such as 2399? 2398?
I'll try to keep an eye out for this problem and see if I can learn more. Until then you can use the ffprobe method as a first resort and fall back on brute force if you encounter problem segments.

How much time elapsed after the live ended before you attempted to download the segments?
It started with pyinstalive.
Did the ffprobe method work for the other segments?
Yes.
Did the 2400.mkv link return 404?
Yes.
Did you try the 2400 link without leading 0s? i.e. 2400 instead of 0002400
No, but it should be the same.
If downloaded, was it a 0 byte file?
Yes.
Did you try slightly less or more than 2400? Such as 2399? 2398?
I brute force it, nothing found.
I'll try to keep an eye out for this problem and see if I can learn more. Until then you can use the ffprobe method as a first resort and fall back on brute force if you encounter problem segments.

@cykac04
After 2400 failed did your brute force method begin at 2401 or go back and start at 935?
When you merged the files, was there a jump in video at the 934-2501 boundary?
Thanks for answering all my questions.

@cykac04

all numbers until the start of the recording

Forgive me for being confused on this point, but you brute forced it backwards? Or, when 2400 failed, did you brute force it from 935 onwards?

I don't realize any gap.

I guess there's audio playing? I don't have the audio segments, so I can't tell. But if it's just black and silent then there's no way to know. If the error happens while there's video or audio activity then it'll be something to look for.

cat init.m4v $(ls -vx  <stream_id>_0-*.m4v) > video.mp4 &&
cat init.m4a $(ls -vx  <stream_id>_0-*.m4a) > audio.m4a &&
ffmpeg -i video.mp4 -i audio.m4a -c copy output.mp4

Anyone now how can merge this ? I tried but i think it's not working on windows 10.
18129352513204028.zip

Keep in mind you can't run the commands in Windows CMD or Powershell as is because some of those commands and the syntax is not supported. The easiest thing to do would be to download Git and install it accordingly. Open up Git Bash and make sure your working directory within Git Bash is the same location where those m4v and m4a files are. Then you could run the following snippet, replacing stream ID with your stream id (its the folder name).
cat init.m4v $(ls -vx<PHONE_NUMBER>3204028_0-*.m4v) > video.mp4 &&
cat init.m4a $(ls -vx<PHONE_NUMBER>3204028_0-*.m4a) > audio.m4a &&
ffmpeg -i video.mp4 -i audio.m4a -c copy output.mp4

Thanks for the help I used to log on a linux machine just to merge them.
I have one issue, the command as it stands
"cat init.m4v $(ls -vx<PHONE_NUMBER>7491353_0-.m4v) > video.mp4 &&  cat init.m4a $(ls -vx<PHONE_NUMBER>7491353_0-.m4a) > audio.m4a &&  ffmpeg -i video.mp4 -i audio.m4a -c copy output.mp4"
will end up with this:

I can only merge if I use the 3 lines separately. Do you know if there is a way to make it work in one go?
Also is there a possibility for omitting the stream ID and just let it merge .m4v and .m4a files in the order its listen on the file system?

I am willing to implement this if you can figure out a more efficient way to find and download the missing segments instead of brute-forcing thousands of URLs. I have a testing branch which authenticates using and interacts with the Instagram web API and no longer uses the very much outdated app API dependency. Its extension module is still used to handle downloading of segments but I don't expect the underlying logic to change in the foreseeable future

Am I using this script correctly? Just place it in a folder with the relevant .json and let it run?

This issue has ran its course for now I will be closing it. @Testing2134 For any questions please create a new issue.

You can test with pip install git+https://github.com/dvingerh/PyInstaLive.git@testing --upgrade

Hi sorry to disturb you just a little question about this testing version:
You are running PyInstaLive 3.3.0 using Python 3.8.10
The final file/live have a low quality compared to the old/stable version, it's suppose to be this way or I am doing something wrong here ?

You can test with pip install git+https://github.com/dvingerh/PyInstaLive.git@testing --upgrade

Hi sorry to disturb you just a little question about this testing version:
You are running PyInstaLive 3.3.0 using Python 3.8.10
The final file/live have a low quality compared to the old/stable version, it's suppose to be this way or I am doing something wrong here ?

There should be no difference since the way livestreams are downloaded are not changed. Unless all new downloaded streams are lower quality suddenly i would chalk it up to coincidence
In any case more testing is needed and this these new changes are unlikely to get pushed out as an official new version any time soon, if ever

There should be no difference since the way livestreams are downloaded are not changed. Unless all new downloaded streams are lower quality suddenly i would chalk it up to coincidence

It's weird the old version running is like 10 times more big the file this testing it's like low quality than the normal I am testing with both versions the same livestream. Eg this is from the testing version the length is little less due I forgot to start at the same time but it's just few minutes less:

This one is from the old version (You are running PyInstaLive 3.2.4 using Python 3.6.9):

There is any place to change the quality in this testing version ? Right now are recording a livestream and I can see from both the temporary files size differences old version using du -sh:
531M    ../IG/
and the testing version:
96M     ../IG/
Maybe it's just me and I need to test the same version on this machine then to see if will be fine.

@cykac04
I'd like to ask you something about the script which is not appropriate for discussion in public.
Because I can't find your contact information, could you contact me by email (you can find it on my profile)?

[W] No matching JSON file found for the segment directory, trying to continue without it.
[E] Could not create video file: 'broadcast_dict'
when trying to merge? Is this because it was before I upgraded to testing branch?

Old files including JSON files are not compatible with the testing branch
As it stands the master branch still works just fine so I would only switch to the testing branch if major functions stop working

inpst.txt ins+aio.txt
@dvingerh @coder39248583
Hi all, I've make some updates. Now the scripts will get the PPID, extract the user, and open the corresponding json.
run_at_start = python3 ins+aio.py The "start" script only use brute force when the ffprobe method does not work. I've test it can parse up to "7000XXX" segments in much shorter time.
run_at_finish = python3 inpst.py The "finish" script will merge the missing segment to the downloaded stream automatically. Else, it would redownload the pst quality video when pyinstalive sometimes does not download it.
Please change those "path" and "directory" to what you use.

@cykac04
I've mostly gotten this to work but I'm getting an error with the inpst.py
[I] Launched finish command: python inpst.py Traceback (most recent call last):   File "inpst.py", line 44, in <module>     folder = sorted(glob.glob(rf'{path}/instalives/*{user}*downloads'), key=os.path.getctime)[-1] IndexError: list index out of range
Seems like it's having trouble finding the {user} name, as the path it's looking at is correct and is the same in the other script. In my instalives I have a 20211104_<username>_17920175195000447_1636007477_live_downloads folder and also a matching<PHONE_NUMBER>5000447 folder. right now it creates a full Live file for the stream I captured, and I have the "beginning of stream" files, but it's not combining them.
If I manually change {user} in the errored line of the script to the actual IG username and run it, it does work and creates a full *_live_pst.mp4 file which seems fine. I'm not sure why it's not working automatically though. atm I have to change it manually for each different user I get a stream for so it will combine them.

inpst.txt ins+aio.txt
@dvingerh @coder39248583
Hi all, I've make some updates. Now the scripts will get the PPID, extract the user, and open the corresponding json.
run_at_start = python3 ins+aio.py The "start" script only use brute force when the ffprobe method does not work. I've test it can parse up to "7000XXX" segments in much shorter time.
run_at_finish = python3 inpst.py The "finish" script will merge the missing segment to the downloaded stream automatically. Else, it would redownload the pst quality video when pyinstalive sometimes does not download it.
Please change those "path" and "directory" to what you use.

@cykac04 I've mostly gotten this to work but I'm getting an error with the inpst.py
[I] Launched finish command: python inpst.py Traceback (most recent call last): File "inpst.py", line 44, in <module> folder = sorted(glob.glob(rf'{path}/instalives/*{user}*downloads'), key=os.path.getctime)[-1] IndexError: list index out of range
Seems like it's having trouble finding the {user} name, as the path it's looking at is correct and is the same in the other script. In my instalives I have a 20211104_<username>_17920175195000447_1636007477_live_downloads folder and also a matching<PHONE_NUMBER>5000447 folder. right now it creates a full Live file for the stream I captured, and I have the "beginning of stream" files, but it's not combining them.
If I manually change {user} in the errored line of the script to the actual IG username and run it, it does work and creates a full *_live_pst.mp4 file which seems fine. I'm not sure why it's not working automatically though. atm I have to change it manually for each different user I get a stream for so it will combine them.

It is probably the the ps command does not work.
'ps -p {os.getppid()} -o cmd='
This works in linux, and I have got a reply that 'cmd' key does not work in OSX.
Of course, windows does not have ps at all.
You may have to find the alternative command or just paste the folder as argument.

inpst.txt ins+aio.txt
@dvingerh @coder39248583
Hi all, I've make some updates. Now the scripts will get the PPID, extract the user, and open the corresponding json.
run_at_start = python3 ins+aio.py The "start" script only use brute force when the ffprobe method does not work. I've test it can parse up to "7000XXX" segments in much shorter time.
run_at_finish = python3 inpst.py The "finish" script will merge the missing segment to the downloaded stream automatically. Else, it would redownload the pst quality video when pyinstalive sometimes does not download it.
Please change those "path" and "directory" to what you use.

@cykac04 I've mostly gotten this to work but I'm getting an error with the inpst.py
[I] Launched finish command: python inpst.py Traceback (most recent call last): File "inpst.py", line 44, in <module> folder = sorted(glob.glob(rf'{path}/instalives/*{user}*downloads'), key=os.path.getctime)[-1] IndexError: list index out of range
Seems like it's having trouble finding the {user} name, as the path it's looking at is correct and is the same in the other script. In my instalives I have a 20211104_<username>_17920175195000447_1636007477_live_downloads folder and also a matching<PHONE_NUMBER>5000447 folder. right now it creates a full Live file for the stream I captured, and I have the "beginning of stream" files, but it's not combining them.
If I manually change {user} in the errored line of the script to the actual IG username and run it, it does work and creates a full *_live_pst.mp4 file which seems fine. I'm not sure why it's not working automatically though. atm I have to change it manually for each different user I get a stream for so it will combine them.

It is probably the the ps command does not work. 'ps -p {os.getppid()} -o cmd=' This works in linux, and I have got a reply that 'cmd' key does not work in OSX. Of course, windows does not have ps at all. You may have to find the alternative command or just paste the folder as argument.

I'm using Powershell to SSH into an Ubuntu VM and running scripts there so seems odd.

inpst.txt ins+aio.txt
@dvingerh @coder39248583
Hi all, I've make some updates. Now the scripts will get the PPID, extract the user, and open the corresponding json.
run_at_start = python3 ins+aio.py The "start" script only use brute force when the ffprobe method does not work. I've test it can parse up to "7000XXX" segments in much shorter time.
run_at_finish = python3 inpst.py The "finish" script will merge the missing segment to the downloaded stream automatically. Else, it would redownload the pst quality video when pyinstalive sometimes does not download it.
Please change those "path" and "directory" to what you use.

@cykac04 I've mostly gotten this to work but I'm getting an error with the inpst.py
[I] Launched finish command: python inpst.py Traceback (most recent call last): File "inpst.py", line 44, in <module> folder = sorted(glob.glob(rf'{path}/instalives/*{user}*downloads'), key=os.path.getctime)[-1] IndexError: list index out of range
Seems like it's having trouble finding the {user} name, as the path it's looking at is correct and is the same in the other script. In my instalives I have a 20211104_<username>_17920175195000447_1636007477_live_downloads folder and also a matching<PHONE_NUMBER>5000447 folder. right now it creates a full Live file for the stream I captured, and I have the "beginning of stream" files, but it's not combining them.
If I manually change {user} in the errored line of the script to the actual IG username and run it, it does work and creates a full *_live_pst.mp4 file which seems fine. I'm not sure why it's not working automatically though. atm I have to change it manually for each different user I get a stream for so it will combine them.

It is probably the the ps command does not work. 'ps -p {os.getppid()} -o cmd=' This works in linux, and I have got a reply that 'cmd' key does not work in OSX. Of course, windows does not have ps at all. You may have to find the alternative command or just paste the folder as argument.

I'm using Powershell to SSH into an Ubuntu VM and running scripts there so seems odd.

Is the run_at_start script work? Only the finish script is in trouble?

inpst.txt ins+aio.txt
@dvingerh @coder39248583
Hi all, I've make some updates. Now the scripts will get the PPID, extract the user, and open the corresponding json.
run_at_start = python3 ins+aio.py The "start" script only use brute force when the ffprobe method does not work. I've test it can parse up to "7000XXX" segments in much shorter time.
run_at_finish = python3 inpst.py The "finish" script will merge the missing segment to the downloaded stream automatically. Else, it would redownload the pst quality video when pyinstalive sometimes does not download it.
Please change those "path" and "directory" to what you use.

@cykac04 I've mostly gotten this to work but I'm getting an error with the inpst.py
[I] Launched finish command: python inpst.py Traceback (most recent call last): File "inpst.py", line 44, in <module> folder = sorted(glob.glob(rf'{path}/instalives/*{user}*downloads'), key=os.path.getctime)[-1] IndexError: list index out of range
Seems like it's having trouble finding the {user} name, as the path it's looking at is correct and is the same in the other script. In my instalives I have a 20211104_<username>_17920175195000447_1636007477_live_downloads folder and also a matching<PHONE_NUMBER>5000447 folder. right now it creates a full Live file for the stream I captured, and I have the "beginning of stream" files, but it's not combining them.
If I manually change {user} in the errored line of the script to the actual IG username and run it, it does work and creates a full *_live_pst.mp4 file which seems fine. I'm not sure why it's not working automatically though. atm I have to change it manually for each different user I get a stream for so it will combine them.

It is probably the the ps command does not work. 'ps -p {os.getppid()} -o cmd=' This works in linux, and I have got a reply that 'cmd' key does not work in OSX. Of course, windows does not have ps at all. You may have to find the alternative command or just paste the folder as argument.

I'm using Powershell to SSH into an Ubuntu VM and running scripts there so seems odd.

Is the run_at_start script work? Only the finish script is in trouble?
ins_aio.log

Yes the start script works, I've got the {id} folder with the missed stream files. Only the finish script errors.

Yes the start script works, I've got the {id} folder with the missed stream files. Only the finish script errors.

It sounds weird as the both scripts are the same.
May be you can try this.
run_at_finish = bash -c 'python3 -u /home/madoka/Command/inpst.py $(ps -p $PPID -o cmd=)'
if arg == 1:
    folder = f'{path}/instalives/{sys.argv[1]}'
elif arg == 9:
    #output = check_output(f'ps -p {os.getppid()} -o cmd=', shell=True, encoding='UTF-8')
    user = sys.argv[4]#re.search(r'(?<=-d\s)[^\s]+', output).group(0)
    folder = sorted(glob.glob(rf'{path}/instalives/*{user}*downloads'), key=os.path.getctime)[-1]
    print(f'Getting info for \'{user}\' successful.\n')
else:
    exit()

But remind that arg ==9 is only for using -df option.
python3 pyinstalive -d <user> -cp pyinstalive.ini -dp instalives/ --no-replays
You have to change it according to what command you use.

Yes the start script works, I've got the {id} folder with the missed stream files. Only the finish script errors.

It sounds weird as the both scripts are the same. May be you can try this. run_at_finish = bash -c 'python3 -u /home/madoka/Command/inpst.py $(ps -p $PPID -o cmd=)'
if arg == 1:
    folder = f'{path}/instalives/{sys.argv[1]}'
elif arg == 9:
    #output = check_output(f'ps -p {os.getppid()} -o cmd=', shell=True, encoding='UTF-8')
    user = sys.argv[4]#re.search(r'(?<=-d\s)[^\s]+', output).group(0)
    folder = sorted(glob.glob(rf'{path}/instalives/*{user}*downloads'), key=os.path.getctime)[-1]
    print(f'Getting info for \'{user}\' successful.\n')
else:
    exit()

But remind that arg ==9 is only for using -df option. python3 pyinstalive -d <user> -cp pyinstalive.ini -dp instalives/ --no-replays You have to change it according to what command you use.

Your comment made me check the other script and I realized that my user = re.search(r'(?<=-d\s)[^\s]+', output) line was missing the .group(0) in the finish script. But now if I try to run it manually I get-
Traceback (most recent call last): File "inpst.py", line 43, in  user = re.search(r'(?<=-d\s)[^\s]+', output).group(0) AttributeError: 'NoneType' object has no attribute 'group'
I also have to remove the .group(0) part of the line for the script to work when I replace {user} with the  directly, or it just gives the same error message.
I just had another livestream today I was recording and the run_at_finish script errored with this again. I'm not sure where exactly this line is looking to find the username. My command is simply pyinstalive -d .
I have not tried the new script you posted yet.

@mb9023 it sounds like ps does not work to get the command.
I am using this version.
ps -V ps from procps-ng UNKNOWN
You have check whether your ps allows -p option and cmd key.

@mb9023 It sounds like ps does not work well get the command. I am using this version. ps -V ps from procps-ng UNKNOWN You have check whether your ps allows -p option and cmd key.
@cykac04
Mine is the same as yours.

I re-downloaded the inspst.txt file from this thread and replaced my script and it's partly working now, so I must have messed something up before.
However, I am not getting the final _pst.mp4 file. I am getting a *_live.mp4 and a *_live-.mp4. The *user*live_downloads folder is getting deleted, I also do not see an {id}_pst folder that should be created. It seems like the second half of the script is not doing anything. There are no error messages.
My ins_pst.log looks like this
Getting info for {user} successful. PST stream was already downloaded. Merging downloaded files.

@mb9023 It sounds like ps does not work well get the command. I am using this version. ps -V ps from procps-ng UNKNOWN You have check whether your ps allows -p option and cmd key.

@cykac04 Mine is the same as yours.
I re-downloaded the inspst.txt file from this thread and replaced my script and it's partly working now, so I must have messed something up before.
However, I am not getting the final _pst.mp4 file. I am getting a *_live.mp4 and a *_live-.mp4. The *user*live_downloads folder is getting deleted, I also do not see an {id}_pst folder that should be created. It seems like the second half of the script is not doing anything. There are no error messages.
My ins_pst.log looks like this
Getting info for {user} successful. PST stream was already downloaded. Merging downloaded files.

Well, you should read the words.
The api got pst quality so it will just merge the segments.

Well, you should read the words. The api got pst quality so it will just merge the segments.

@cykac04 I don't know what pst quality is, I assumed that was supposed to just be the final output. Is it not supposed to merge together the missed stream files with the live captured stream files to create one final? I am getting two files and I have to manually combine them to get a full live still

Well, you should read the words. The api got pst quality so it will just merge the segments.

@cykac04 I don't know what pst quality is, I assumed that was supposed to just be the final output. Is it not supposed to merge together the missed stream files with the live captured stream files to create one final? I am getting two files and I have to manually combine them to get a full live still

PST is the original quality with highest bitrate, but may not be the highest resolution.
*_live.mp4 is what pyinstalive downloaded.
*_live-.mp4 is missing segments + *_live.mp4.

PST is the original quality with highest bitrate, but may not be the highest resolution. *_live.mp4 is what pyinstalive downloaded. *_live-.mp4 is missing segments + *_live.mp4.

My *_live-.mp4 is missing segments ONLY. It's 16 seconds long while the full test stream was almost 3 minutes.

PST is the original quality with highest bitrate, but may not be the highest resolution. *_live.mp4 is what pyinstalive downloaded. *_live-.mp4 is missing segments + *_live.mp4.

My *_live-.mp4 is missing segments ONLY. It's 16 seconds long while the full test stream was almost 3 minutes.

Oops. Just add something to the old code.
import time
and
time.sleep(300) before print('PST stream was already downloaded.\nMerging downloaded files.')
It becase it merged before "pyinstalive" finished everything.

I got both errors as @mb9023
Traceback (most recent call last):
  File "ins+aio.py", line 92, in <module>
    j_file = sorted(glob.glob(rf'{path}/instalives/*{user}*downloads.json'), key=os.path.getctime)[-1]
IndexError: list index out of range

Traceback (most recent call last):
  File "inpst.py", line 44, in <module>
    folder = sorted(glob.glob(rf'{path}/instalives/*{user}*downloads'), key=os.path.getctime)[-1]
IndexError: list index out of range

so i looked up the scripts and realized my pyinstalive didn't create the instalives subdirectory while downloading live streams. There's no such option in pyinstalive.sh why do you have this subdirectory?
Then I edited all paths in the scripts and manually ran ins+aio.py while the stream was still live and got something similar to

Traceback (most recent call last): File "inpst.py", line 43, in  user = re.search(r'(?<=-d\s)[^\s]+', output).group(0) AttributeError: 'NoneType' object has no attribute 'group'

(not exactly the same cuz it's an ins+aio.py error )
How do I solve this problem and can I still download the whole stream? The livestream is over now.
I'm running the pyinstalive and the script on Ubuntu 20.04 and

You are running PyInstaLive 3.2.4 using Python 3.8.10

Testing on short streams seems to be working now! I'm really not sure what happened to the script I had that gave me so many issues.. I basically ended up putting everything back to how it was originally as well as adding the time.sleep seemed to help (though I lowered it to 10 seconds instead of 300).
@yoshyv I just created the instalives folder myself and pointed my pyinstalive.ini script to save streams there.
The other errors I think you're basically just always going to get those running manually as it's looking for the active stream ID. I had to use a second IG account to go live on to keep testing my captures. I'm not sure about capturing streams that have ended.

@yoshyv run the script manually after you confirm the path is correct.
python3 ins+aio.py 20211108_felonyrose__n_17953574011501851_1636370890_live_downloads
Definitely all your errors are not finding user from ps command.

Testing on short streams seems to be working now! I'm really not sure what happened to the script I had that gave me so many issues.. I basically ended up putting everything back to how it was originally as well as adding the time.sleep seemed to help (though I lowered it to 10 seconds instead of 300).
@yoshyv I just created the instalives folder myself and pointed my pyinstalive.ini script to save streams there. The other errors I think you're basically just always going to get those running manually as it's looking for the active stream ID. I had to use a second IG account to go live on to keep testing my captures. I'm not sure about capturing streams that have ended.

300 is actually for long instalive which may merge for more minutes.

For those who get errors can try on this.

Change the path path = 'Your download path'.
Test if ps -p {pid} -o cmd= works in terminal.
Check if instalive folder exists.
Modify ini.file
run_at_start = bash -c 'python3 -u instart.py $(ps -p $PPID -o cmd=)' run_at_finish = bash -c 'python3 -u insfinish.py $(ps -p $PPID -o cmd=)'
insfinish.txt
instart.txt

@yoshyv It works if your download.json and source file are there.

I'm still confused why do you all have instalives subdirectories while my pyinstalive doesn't create it.
@cykac04 what's the commands if the stream is ended?
python3 ins+aio.py 20211108_userid_17927371057781885_1636364142_live_downloads.json
then
python3 inpst.py 20211108_userid_17927371057781885_1636364142_live_downloads.json
??

I double checked and edited the script and finally managed to get instart.py to work.
id:<PHONE_NUMBER>7781885
quality: live-hd2-v
end time: 52968

Starting loop 0-2000.
50966
Downloading audio files.

and a folder with video/audio files is generated.
| a.mp4 | 2021-11-08 23:04 | 316K |
  | init.m4a | 2021-11-08 23:04 | 598 |
  | init.m4v | 2021-11-08 23:04 | 659 |
  | v.mp4
However, the v&a files are not merged and
python insfinish.py 20211108_userid_17927371057781885_1636364142_live_downloads

downloaded:
| live-hd2-v |  720 | 1280 |
source:
| live-pst-v |  720 | 1280 |
Creating download list.
Start downloading source stream.
Merging downloaded files.
/bin/sh: 1: cannot create /home/user/live/20211108_userid_17927371057781885_1636364142_live_downloads/audio.mp4: Directory nonexistent
/bin/sh: 1: cannot create /home/user/live/20211108_userid_17927371057781885_1636364142_live_downloads/video.mp4: Directory nonexistent
/home/user/live/20211108_userid_17927371057781885_1636364142_live_downloads/video.mp4: No such file or directory

Does insfinish.py have to be executed before the pyinstalive files being merged?
Is it safe to manually mux v.mp4 and a.mp4 downloaded via the script and merge them with pyinstalive file?
BTW. There's import telegram_send code in the script.
How does it work?

@dvingerh My account was flagged and comments were hidden but now it's finally back to normal again.
I tried the new scripts and they seems to have some trivial path-related (I suppose) problem.
 % /bin/sh: 1: cannot create /home/user/live/20211116_dazn_jpn_18010227361360435_1637071282_liv
e_downloads/audio.mp4: Directory nonexistent
/bin/sh: 1: cannot create /home/user/live/20211116_dazn_jpn_18010227361360435_1637071282_live_d
ownloads/video.mp4: Directory nonexistent
/home/user/live/20211116_dazn_jpn_18010227361360435_1637071282_live_downloads/video.mp4: No such file or directory
/home/user/live/20211116_dazn_jpn_18010227361360435_1637071282_live_downloads/audio.mp4: No such file or directory

My guess is that pyinstalive deleted the audio.mp4 and video.mp4 before the scripts read them.
 I only missed about 10 seconds. Is there a way to restore the ten seconds and get the whole video in pst format (higher bit rate). Also, do I need to edit the path in the scripts?
Thank you.

For those who get errors can try on this.

Change the path path = 'Your download path'.
Test if ps -p {pid} -o cmd= works in terminal.
Check if instalive folder exists.
Modify ini.file
run_at_start = bash -c 'python3 -u instart.py $(ps -p $PPID -o cmd=)'
run_at_finish = bash -c 'python3 -u insfinish.py $(ps -p $PPID -o cmd=)'
insfinish.txt
instart.txt

@yoshyv It works if your download.json and source file are there.

Hi.
These two new scripts especially the finish one, don't download audio files and only download pst version of video segments.
Could you please take a look at the script and correct it?
Thanks.


[sending] Remove transferring.

Out of an abundance of caution, we:

Left in parsing support for transferring but internally made it rely on the internals of sending.

Added a warning to tell people that transferring was going to be removed very soon.

Now that we have given people some time, remove support for parsing transferring.
rdar://130253724

I need to do the swift parser part of this

https://github.com/apple/swift-syntax/pull/2692
@swift-ci smoke test

https://github.com/apple/swift-syntax/pull/2692
@swift-ci smoke test


The requested module ... does not provide an export named 'useMachine'

⚠ IMPORTANT ⚠ Please check the following list before proceeding. If you ignore this issue template, your issue will be directly closed.

[x] Read the docs.
[x] Use Vite >=2.0. (1.x is no longer supported)

Describe the bug
When using the @xstate/vuelibrary, which has a named export called useMachine, Vite will throw an error saying it can't find the named export.
Reproduction
https://github.com/TheDutchCoder/oms-vite
System Info

vite version: 2.0.0-beta.48
Operating System: MacOS 11.1
Node version: 12.19.0
Package manager (npm/yarn/pnpm) and version: npm 6.14.10

Tested with latest commit and it's working, should be fixed in next release.

Tested with latest commit and it's working, should be fixed in next release.


Missed amount on received transaction

In the transaction details page it is shown.

It could got shifted to the line below and thus became hidden. Could you please do the following: make system wide font size smaller than it is now and open an app again, will payment sum be there?

There is already an instruction to ellipsize a text if it overflows but apparently not all models respect it. I've made such that sum has more room in https://github.com/btcontract/lnwallet/commit/1583872236f2744c2d8f484fe7bbbe7586e43344, please let me know if that fixes an issue after next 0.2-77 update.

Has it been fixed in recent updates?

Has it been fixed in recent updates?

Yes


Work around nvFuser's current limitation that fd.ops.linear must be 2D.

Tests are failing due to another known limitation in matmul_expr_eval. However, this draft PR can still serve as a reproducer for @Priya2698 .

I'm closing this because @Priya2698 will soon send out the right fix. Feel free to reuse the tests here.


Add an Entry API for HashSet

This adds HashSet::entry and rustc_entry, mimicking the API of HashMap. The "rustc" variants are without the S type parameter, just like the difference on maps. Set entries have fewer methods since there's no mutable value like in maps, but the basic insert/remove/replace functionality is there.
Resolves #274.

Could we avoid introducing another rustc_entry? This was a hack for compatibility with std which didn't have a S parameter on the Entry type. I would prefer if we did this "correctly" and had a S parameter for the version used by std.

I expect that std::collections::hash_set::Entry<T> should look like a simple wrapper on hash_map::Entry<T, ()>, even if that's not really direct because it goes through hashbrown. I think it would be unfortunate if std were inconsistent about whether these Entry types have an S parameter, but that's ultimately a libs-api decision.
If you still want to push for that S, then I'll drop rustc_entry here and keep it in a branch on the side, just in case. :)

I'll repeat that Entry not having S was a mistake that we shouldn't carry over to new APIs. The rustc_entry in the hashbrown crate is actually slightly less efficient because it has to reserve space for a value ahead of time before knowing whether a value will be inserted into the map. It is better to defer the reservation until a value is explicitly inserted, which requires the S parameter.

OK, I've dropped the S-less one.

@bors r+

:pushpin: Commit af67f7d53271d1aec4fc953a35b1007a04ece0d0 has been approved by Amanieu

:hourglass: Testing commit af67f7d53271d1aec4fc953a35b1007a04ece0d0 with merge e54c1608192b92206ac95009c1afcfc99bb635e7...

:sunny: Test successful - checks-actions
Approved by: Amanieu
Pushing e54c1608192b92206ac95009c1afcfc99bb635e7 to master...

Any idea when this will be released?

I just published a release.

Thanks!


Random sampling to obtain segment posterior summary statistics from HMM segmenters

The math is worked out -- need to implement (will be done after germline evaluations).

Issue moved to broadinstitute/gatk #2894 via ZenHub


Add namespace to prometheus metrics

What this PR does / why we need it:
This PR adds a namespace ("istio") to all metrics created and exposed by the prometheus adapter for Mixer. This is needed to clearly distinguish metrics generated by istio (and avoid collisions with service-defined metrics moving forward).
cc: @nmnellis
Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #
Special notes for your reviewer:
Release note:

Add "istio" namespace to all istio-generated prometheus metrics.

@douglas-reid: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/e2e-suite-rbac-auth.sh
4d44e2d8b63a6c7dc863d16e140961118f16267f
link
/test e2e-suite-rbac-auth

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@douglas-reid: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/e2e-suite-rbac-auth.sh
4d44e2d8b63a6c7dc863d16e140961118f16267f
link
/test e2e-suite-rbac-auth

prow/e2e-suite-rbac-no_auth.sh
4d44e2d8b63a6c7dc863d16e140961118f16267f
link
/test e2e-suite-rbac-no_auth

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@douglas-reid: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/e2e-suite-rbac-auth.sh
4d44e2d8b63a6c7dc863d16e140961118f16267f
link
/test e2e-suite-rbac-auth

prow/e2e-suite-rbac-no_auth.sh
4d44e2d8b63a6c7dc863d16e140961118f16267f
link
/test e2e-suite-rbac-no_auth

prow/e2e-cluster_wide-auth.sh
4d44e2d8b63a6c7dc863d16e140961118f16267f
link
/test e2e-cluster_wide-auth

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

Is it still WIP?
I think this is a good default and let’s us bake a grafana dashboard.
However shouldn’t namespace be a configuration option?

@mandarjog I don't think that we should make the namespace a config option, at least right now. It makes the dashboard stuff a bit more brittle and makes prometheus config a bit more verbose and confusing than absolutely needed. we don't, at the moment, even allow it to be set, let alone configured, so this isn't a loss in functionality. we can introduce config for it when needed.

@douglas-reid: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/e2e-suite-rbac-auth.sh
4d44e2d8b63a6c7dc863d16e140961118f16267f
link
/test e2e-suite-rbac-auth

prow/e2e-suite-rbac-no_auth.sh
4d44e2d8b63a6c7dc863d16e140961118f16267f
link
/test e2e-suite-rbac-no_auth

prow/e2e-cluster_wide-auth.sh
4d44e2d8b63a6c7dc863d16e140961118f16267f
link
/test e2e-cluster_wide-auth

prow/istio-presubmit.sh
f17493dbba28dd636f7b32b8f81449bc266b3314
link
/test istio-presubmit

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@douglas-reid: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/e2e-cluster_wide-auth.sh
557bde5ec6de682d5cdf800777dac519980f576e
link
/test e2e-cluster_wide-auth

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.


PNG Compression Parameter

Hey there;
Generating PNG's with compression hard-coded to 0 makes for HUGE files (easily 10 times the size of the source PNG).  This isn't going to work for my application so I made compression a parameter.  PNG compression is lossless so this actually works great.
I don't know symphony at all so I didn't use the symphony options thing -- instead, I cherry pick out the option I need and remove it from the array so it doesn't make symphony complain downstream.  If there's a better way to do it, by all means ...
Thanks!

Coverage increased (+0.1%) to 96.354% when pulling ebbcba708e8eabedb793641231d3594a224b3371 on Tigerdile:master into d60fdf365679a1ed5dcca6e3a4741df245c16bd6 on kzykhys:master.


replace NOT_IMPLEMENTED w/ NOT_REACHED

We now have NOT_REACHED macro. We should grep for NOT_IMPLEMENTED and switch to NOT_REACHED in places where that is applicable.
cc @jamessynge

@htuch ^ this is another good starter one for your new grad.

@hennna

Working on it.
Grep return 17 instances of "NOT_IMPLEMENTED" and 1 instance of "not implemented"

FYI, https://github.com/lyft/envoy/blob/master/source/server/config/network/http_connection_manager.cc#L168 is an example of one that should be NOT_REACHED() (since the switch cases cover all possibiilties) and https://github.com/lyft/envoy/blob/1d5035fd0b7bc8e721d059720736a31eb005e329/source/common/http/http2/codec_impl.cc#L350 is a genuine NOT_IMPLEMENTED() (since it's flagging something that is genuinely not implemented today).


i can haz path?

omg this plugin is awesome u guys...

❤❤❤

Thanks!  I'm glad you're finding it useful.  :)


Update BaseGroupApprovalTaskStateEmailNotifier to check for AbstractGroupApprovalTask instances

#11653 added AbstractGroupApprovalTask and GroupApprovalTask inherits from it.
howevere the BaseGroupApprovalTaskStateEmailNotifier still checks GroupApprovalTask - https://github.com/wagtail/wagtail/blob/6affa04d320f50b6a3babd8afc1a590e02e84a5d/wagtail/admin/mail.py#L368-L373
This means any custom group tasks derived from AbstractGroupApprovalTasks require adding custom notifiers and wiring in a signal handler

I like to work on this issue, please assign.

We don't assign issues to anyone outside the core team. Feel free to work on this.
I suggest reading https://docs.wagtail.org/en/stable/contributing/first_contribution_guide.html

@zerolab I have attempted to solve the issue, please have a look and suggest me the changes required


[Question] How purchaseProductForUser really works?

I've implemented the purchaseProductForUser for a monthly subscription, and I'm passing firebase user UID.
But when I change the user in app (but keep the same iTunes account on device), which has another UID, I can restore the previous user's purchases.
The purchaseProductForUser and restorePurchasesForUser methos works per iTunes account?
Or does passing different UID to the same iTunes account should work properly?

Some one?

The purchaseProductForUser and restorePurchasesForUser methos works per iTunes account?

As far as I know, purchases are linked to iTunes account.

Then what is the purpose of the purchaseProductForUser if using different username will get products from same account without restriction?

@lucianomlima I'm seeing the same issue. restorePurchasesForUser restores all purchases and not just the user's.

@RalfNieuwenhuizen is this still working for you? I can see that you added this feature.


fix(contains): find objects

Previously, contains would not be able to handle searches for object instances within the collection, because the element being searched for was incorrectly treated as an expression.
Fixes #182

@a8m looks good to me :+1:

Awesome @oliversalzburg, thanks.


Update UI documentation references links

Reference: https://github.com/kubev2v/forklift-console-plugin/issues/781
Update all UI documentation fields help text references links by pointing to  HELP_LINK_HREF and by adding DOCUMENTATION_VERSION as an internal parameter. So that it will be configurable for next version bumps. The doc link is pointed to either u/s or d/s doc, based on the BRAND_TYPE env. variable.
This is currently relevant for both providers details tab and overview settings card.

As disccused offline, the links will be changed, once we agree on the best way to do it, and after doc team will have docs for 2.6, until then we will use the downstream docs

The current status, before this fix, is that we use u/s docs on few places that points to a broken link  (e.g., empty state msgs)  and u/s docs on other places which points to 2.4 instead of 2.5.
If not important to fix to 2.5.x, it can wait to the new 2.6 docs of course

is that we use u/s docs on few places that points to a broken link

we will fix the upstream docs once the doc team will create some, AFAIK forklift project does not have upstream docs yet.

on other places which points to 2.4 instead of 2.5.

we are currently working on 2.6, once the 2.6 docs will be ready, we will need to collaborate with release and docs teams to come up with a release mechanism / automation to manage the links in collaboration with all stake holders, e.g. dev that set up the links, docs that publish the docs and release that set up the downstream package so when we release 2.6 the docs will be as up to date as possible
cc:// @ahadas @RichardHoch - cc because we currently don't have any automation/manual-flow to update the docs links once they are published by the docs team for specific downstream version, we will need to agree on some flow and then try to automate it, see: https://github.com/kubev2v/forklift-console-plugin/issues/781 .

@anarnold97 Would appreciate your insights.

remove label/ stale

closing in favor of:
https://github.com/kubev2v/forklift-console-plugin/pull/817


Tag a release / Semantic Versioning

Can we tag a release, it would make it easier for consumers of this SDK to stick to a version and it will make compatibility with other libraries much easier in the future.
I'd recommend semantic versioning: http://semver.org/

@benjy have tagged v0.1.0. Will keep it under 1.0.0 for a bit until we feel it's stable enough. At least this gives you a fixed point to require. That work for you?

Awesome, thank you.


Add compatibility for libraries installed via homebrew on M1 macs

As of version 2.6.0 of the homebrew package manager, native
libraries on M1 macs are installed to /opt/homebrew/lib to
avoid comflict with intel compiled libraries installed into
/usr/local/lib (See release notes here:
https://brew.sh/2020/12/01/homebrew-2.6.0/ ).
This patch allows findlib to locate libraries installed in that
location. This allows, for example, this cfgrib package to find
the eccodes library when installed via brew install eccodes on
an M1 mac.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

After a lot of manual digging I've just had to apply this patch manually to fix cfgrib and eccodes on my M1 mac, it would be great to get this merged and released as its very confusing to have no way to find a valid eccodes install otherwise.


TigerGraph implementation of LDBC ACID tests

@yczhang1017 I'm following the instructions in the README. Any ideas why I run into this?
docker exec --user tigergraph -it acid-tg bash -c "export PATH=/home/tigergraph/tigergraph/app/cmd:\$PATH; gadmin start all; gsql /tigergraph/setup.gsql"
[   Info] Starting EXE
[   Info] Starting CTRL
[   Info] Starting ZK ETCD DICT KAFKA ADMIN GSE NGINX GPE RESTPP KAFKASTRM-LL KAFKACONN TS3SERV GSQL TS3 IFM GUI
Connection refused.

Your command seems to be missing the port mappings

I'm on MacOS High Sierra (cos of weird uni restrictions). The docker container is starting fine via run. I'll try on my other machine, that runs Monterey.

@yczhang1017 could you please tick "Allow edits by maintainers"?

@yczhang1017 I have a problem with the io.github.karol_brejna_i.tigergraph:restpp-client:1.0.0 dependency, gradle can't seem to find it on jcenter. I fixed it by installing locally but now it causes CI to fail in my PR #10

No I didn't, however that wouldn't explain why the CI is failing: https://app.circleci.com/pipelines/github/ldbc/ldbc_acid/97/workflows/ad6e985b-5286-4857-8d1b-f4458cfa88ba/jobs/98?invite=true#step-106-80
> Could not find io.github.karol_brejna_i.tigergraph:restpp-client:1.0.0.
     Searched in the following locations:
- https://repo.maven.apache.org/maven2/io/github/karol_brejna_i/tigergraph/restpp-client/1.0.0/restpp-client-1.0.0.pom
       - https://repo.maven.apache.org/maven2/io/github/karol_brejna_i/tigergraph/restpp-client/1.0.0/restpp-client-1.0.0.jar
       - file:/home/circleci/.m2/repository/io/github/karol_brejna_i/tigergraph/restpp-client/1.0.0/restpp-client-1.0.0.pom
       - https://jcenter.bintray.com/io/github/karol_brejna_i/tigergraph/restpp-client/1.0.0/restpp-client-1.0.0.pom
       - file:/home/circleci/project/lib/restpp-client-1.0.0.jar
       - file:/home/circleci/project/lib/restpp-client.jar

Fixed the dependency issue by adding this repository to the Gradle config: https://github.com/ldbc/ldbc_snb_interactive_impls/blob/63627976e45fb549c66a95ee66fb40f2891ac9a7/tigergraph/pom.xml#L47-L58
The  build is still failing, looking into it ...

@yczhang1017 @karol-brejna-i the build is failing as it's unable to resolve this dependency:
    compile "io.github.karol_brejna_i.tigergraph:restpp-client:1.0.0"

This seems a stable version while the Sonatype repository only has SNAPSHOT versions:
https://s01.oss.sonatype.org/content/repositories/snapshots/io/github/karol-brejna-i/tigergraph/restpp-client/
Could you please clarify which version to use and where to obtain it from? Thanks!

For now, there is no release version of the dependency.
Only SNAPSHOT is available.
@yczhang1017 Let's discuss if new/stable version should be published.

Thanks! I added this to the Gradle config and now the build is green. Will merge this as part of #10


Properly sanitize STUN urls

Currently we error if a STUN URL passed to Configuration.ICEServers
contained a query parameter. Fix this by using the getICEServers
function that sanitizes these for us.

Codecov Report

Merging #939 into master will decrease coverage by 0.28%.
The diff coverage is 61.53%.

@@            Coverage Diff             @@
##           master     #939      +/-   ##
==========================================
- Coverage   76.41%   76.13%   -0.29%
==========================================
  Files          59       59
  Lines        4351     4354       +3
==========================================
- Hits         3325     3315      -10
- Misses        742      756      +14
+ Partials      284      283       -1

Impacted Files
Coverage Δ

configuration_common.go
100% <100%> (ø)
:arrow_up:

peerconnection.go
72.85% <37.5%> (-0.22%)
:arrow_down:

sctptransport.go
75.38% <0%> (-3.59%)
:arrow_down:

datachannel.go
78.83% <0%> (-2.05%)
:arrow_down:

icegatherer.go
82.78% <0%> (+0.81%)
:arrow_up:

dtlstransport.go
72.77% <0%> (+1.04%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e9ac852...2800459. Read the comment docs.


Remove version: OPAutoClicker.OPAutoClicker version 3.0

[Automated] It returns code over 400 in all urls
Microsoft Reviewers: Open in CodeFlow

URL: https://sourceforge.net/projects/orphamielautoclicker/files/autoclicker-3.0/AutoClicker-3.0.exe/download
OK
(Automated message - build 923)

URL: https://sourceforge.net/projects/orphamielautoclicker/files/autoclicker-3.0/AutoClicker-3.0.exe/download
Status Code: 200
(Automated message - build 904)

Close with reason: Removal appears to be automated and in error;


Event System Improvements

@brianwatling following discussion here so we don't lose it. Thoughts on this? Goal is to consolidate most event systems into just one nk_console_event_handler* events.

Make a enum nk_console_event_type with DESTROY, CHANGE, and CLICK
Add a enum nk_console_event_type to nk_console_event_handler
Have nk_console_event_handler* events in nk_console as a cvector for all events
Move nk_console_button_data::onchange to the new events pattern
Move nk_console::destroy to the new events pattern
Keep render outside of the event system to keep performance a priority
Re-work the invoke events to take a enum nk_console_event_type
Rather than nk_console_onchange(widget), call nk_console_invoke(widget, NK_CONSOLE_EVENT_TYPE_CHANGE)... or something?
Update nk_console_set_onchange_handler() to nk_console_add_event_handler()
Update nk_console_set_onchange() to nk_console_add_event

I think your suggestion to have an enum for the event type seems reasonable. That'll help reduce the API surface for event handling.

Have nk_console_event_handler* events in nk_console as a cvector for all events

Are you suggesting all events are stored in a single array that's scanned when an event is triggered? Seems like a clean approach and performance shouldn't be a concern given events are triggered relatively infrequently - so I think this is decent way to implement this


Previewing a song in the app drastically lowers the app's volume.

When you preview a song in the app, the volume for the app is dramatically reduced, going from 100 to usually around 20. No other apps have their volume affected by this bug but it's a little annoying having to go into my volume mixer and raise the volume back to normal every time I want to test a song.

That's a known issue. It should work better if you disable [EXP] In-Game volume for now.
This feature is mostly used to balance the audio volume between songs as it would sound in game.


onError isn't working...

<CachedImage source={{uri: 'http://this.is.a.bad.url'}}
onError={() => console.log('load error')}/>
onError isn't working...

What version you use?
It works for me, are you sure this is rendered?

@chrusart
"react-native-fetch-blob": "^0.10.8",
"react-native-img-cache": "^1.4.0",
Image can display if uri is correct.


Fix billing download acceptance test

Fixes. TestMwsAccUsageDownload. The timeout is increased when the testing client is created because doing so in the test itself makes the transpilation test (TestLoadsFolder) fail

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (9632fb2) 18.24% compared to head (06674e4) 18.24%.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #764   +/-   ##
=======================================
  Coverage   18.24%   18.24%
=======================================
  Files         109      109
  Lines       14810    14810
=======================================
  Hits         2702     2702
  Misses      11886    11886
  Partials      222      222

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


ENH: Python wrapped ConvertTubesToImage

@thewtex @aylward Maybe we should write python tests and use one of the python testing frameworks

the native unittest framework
nose
py.test

That itself could act as nice documentation for people wanting to use python interface of TubeTK

Dont know which of the python testing frameworks plays better with CMake

@jcfr @thewtex @aylward
See this part in our large_image repo where they are adding a bunch of python tests.
The .md5 files are here.
And Here is some test written using that data. See here where they are referring to one of those images.
Some similar way to get the data so we can write python tests for python wrapped TubeTK applications would be nice.

@jcfr @thewtex @aylward
Found a cleaner way to write a python test that would serve two purposes:

Test python wrapping of the CLI
Document/show how to use the python wrapped interface of the CLI

It uses ctk_cli -- the python equivalent of GenerateCLP that me and @jcfr developed recently to do command line argument parsing for slicer CLIs written in python.
We have to find a way to get CMake to

add TubeTK and ITK to PYTHONPATH while running the test so i dont have to do it in the test file as shown here.
Change ${PROJ_EXE} in Midas3FunctionAddTest to python <relative-path-to-python-test-cli>

We have to find a way to get CMake to

add TubeTK and ITK to PYTHONPATH while running the test so i dont
have to do it in the test file as shown here
https://github.com/cdeepakroy/TubeTK/blob/master/Applications/ConvertTubesToImage/Testing/ConvertTubesToImageTest.py#L5-L11
.

The WrapITK.pth file could be added to the Python site-packages directory
as described here:
http://itk.org/ITKSoftwareGuide/html/Book1/ITKSoftwareGuide-Book1ch3.html#x34-410003.7

Merged as part of PR #702


CTD when opening a ProjectE transmutation tablet GUI with refined storage items present.

crashreport.txt

I'll look into it, but this error is caused by ProjectE not correctly initializing storage disks. However I can work around that


quote should quote dicts too

Closes #5904.

[x] Tests added / passed
[x] Passes black dask / flake8 dask

Thanks @bmerry !


Make attohttpc optional for credentials

If an application is simply constructing a Credentials struct from existing values for rust-s3, it doesn't need an http client bundled into the library. This change makes the http client in aws-creds optional behind a feature flag, but still enabled by default.

Good stuff, thank you!


Support for private registries?

Will support for scanning private registries be added with the command line tools eventually?  I'd like to be able to scan my OpenShift Registry.

The command line tools support scanning any docker v2 compatible registry that you have local access to. So if you install the CLI on a rhel/centos/fedora/debian/ubuntu machine/vm then you can pull the image and analyze it.
docker pull myregistry:5000/myrepo/app:latest
anchore analyze --image=myregistry:5000/myrepo/app:latest
You can also ask questions on slack here: https://anchore.com/opensource/community/#slack


Bun/Deno Support? A way to add stuff ourselves. Subprocess support.

Or at least add a way to install other things. Bun natively supports Apple Silicon, and all I need to install it is bash (I think, it may still not work with bash).
We need a way to install things and have custom commands.
Here's my idea for subprocess support.
Reroute subprocess to instead run asynchronously to the main program.
If you could add those 3 things, this would be the best iPad ide.

What I meant was building it from source on computer and bundle it into the
app like what I did with Node and Python.
On Mon, 11 Jul 2022 at 23:06, hg0428 @.***> wrote:

I think it's possible to build Bun/Deno for iOS. Subprocess is another
story though.
Currently you can install custom commands using npm or pip.
How could I get Bun?
What's your idea?
The current install process for Bun requires bash and unzip
—
Reply to this email directly, view it on GitHub
https://github.com/thebaselab/codeapp/issues/516#issuecomment-1180528786,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AJE6T2YHGNFZF65QTWVSEFTVTQZ7DANCNFSM53HYUVGQ
.
You are receiving this because you commented.Message ID:
@.***>

What I meant was building it from source on computer and bundle it into the
app like what I did with Node and Python.
On Mon, 11 Jul 2022 at 23:06, hg0428 @.***> wrote:

I think it's possible to build Bun/Deno for iOS. Subprocess is another
story though.
Currently you can install custom commands using npm or pip.
How could I get Bun?
What's your idea?
The current install process for Bun requires bash and unzip
—
Reply to this email directly, view it on GitHub
https://github.com/thebaselab/codeapp/issues/516#issuecomment-1180528786,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AJE6T2YHGNFZF65QTWVSEFTVTQZ7DANCNFSM53HYUVGQ
.
You are receiving this because you commented.Message ID:
@.***>

That would be great! Please do. Please, hopefully its not too hard...
Thats actually the same thing the Bun devs recommended.
I've been wanting to use Bun/Deno for a while now actually.


Socks Broken W/ Server Side Error

When trying to start a socks proxy on port 7004 I get the following server side error from the socks.py agent_functions script. Using a fresh clone of Athena and Mythic 3.0.0.

[STDOUT]:
[STDERR]:
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/mythic_container/agent_utils.py", line 331, in createTasking
createTaskingResponse = await cmd.create_tasking(task=task)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Mythic/athena/mythic/agent_functions/socks.py", line 60, in create_tasking
stop_res = await SendMythicRPCProxyStopCommand(port=task.args.get_arg("port"), task_id=task.id)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: SendMythicRPCProxyStopCommand() got an unexpected keyword argument 'port'

Apollo agent's socks is working as expected and calls the SendMythicRPCProxyStartCommand differently than the athena script.
https://github.com/MythicAgents/Apollo/blob/master/Payload_Type/apollo/apollo/mythic/agent_functions/socks.py
    resp = await SendMythicRPCProxyStartCommand(MythicRPCProxyStartMessage(
        TaskID=taskData.Task.ID,
        PortType="socks",
        LocalPort=taskData.args.get_arg("port")
    ))

yeah socks is currently somewhat borked, I'm working on it at this moment actually lol, hoping to have it fixed soon

go ahead and check out the socks-update branch and let me know if that helps

yolo merged into main lmk if you have issues

SOrry running behind. Blew away mythic and rebuild fresh,  checked out the update made to the master branch, built an exe payload, ran a few commands to make sure all was sell, ran "socks {"port":"7001"}", received error "Failed to start listening on port 0, it's not exposed through docker".   It seems that the input is not being passed through. This is the same error I got when trying to fix myself.   The error comes form "/mythic-docker/src/rabbitmq/utils_proxy_traffic.go".
That function is looking for if isPortExposedThroughDocker(p.LocalPort) {
But we are sending Port
    resp = await SendMythicRPCProxyStartCommand(MythicRPCProxyStartMessage(
        TaskID=taskData.Task.ID,
        PortType="socks",
        Port=taskData.args.get_arg("port")
    ))

But, if you send LocalPort then the mythic API complains about not getting the Port variable.
TypeError: MythicRPCProxyStartMessage.init() missing 1 required positional argument: 'Port'
I noticed apollo is using a different image, maybe their API changed on the backend?
https://github.com/MythicAgents/Apollo/blob/master/Payload_Type/apollo/Dockerfile  "FROM itsafeaturemythic/mythic_python_dotnet:latest"
https://hub.docker.com/r/itsafeaturemythic/mythic_base_image - this image is deprecated.

reopening to take a closer look, I had been testing on my local machine just fine but maybe I'm running an out of date mythic. I'll start a fresh instance in a bit

To be clear in your .env file the parameter MYTHIC_SERVER_DYNAMIC_PORTS contains 7000-7010 right?

Okay, I've updated main with the new docker image, the updated mythiccontainer, and the appropriate socks.py. Give that a try and lmk. I'll keep this open in the mean time.

To be clear in your .env file the parameter MYTHIC_SERVER_DYNAMIC_PORTS contains 7000-7010 right?

correct

Confirmed working on my end. Thank you very much for your help!!