qBittorrent-Unrar-Script in combination with SMA-MOD. (Originally: Typo?)

pythonpath=/usr/local/sma /ven/ bin/python3
Should be pythonpath=/usr/local/sma/venv/bin/python3
https://github.com/mdhiggins/sickbeard_mp4_automator/wiki/qBittorrent-Unrar-Script

Fixed, thanks

Is this accurate in the script? Or should I replace it with another value? /opt/qbittorrent/unrarall/unrarall

Yes, it's accurate. You have to pull that down from https://github.com/arfoll/unrarall. You can place it wherever you want, I just had it loaded there since qbit was the only thing that required it.
I can't remember if I had to load unrar as well, but if so I used the 'Nux Desktop EL7' repo from http://li.nux.ro/repos.html.

Awesome. Thanks alot for clarifying :)
On 3 May 2021, 13:22 +0200, John Chambers @.***>, wrote:

Yes, it's accurate. You have to pull that down from https://github.com/arfoll/unrarall. You can place it wherever you want, I just had it loaded there since qbit was the only thing that required it.
I can't remember if I had to load unrar as well, but if so I used the 'Nux Desktop EL7' repo from http://li.nux.ro/repos.html.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

Updated to include that information in the wiki and moved the unrar path to the environment variable unrarpath

@johnwchambers If I may ask, assuming you have tested it with both *arrs. How did you get the *arrs to pick up the converted files?

The script calls qBittorrentPostProcess.py which notifies the downloader to process the directory as part of its usual function

The script calls qBittorrentPostProcess.py which notifies the downloader to process the directory as part of its usual function

Sweet! Thanks.

![image](https://user-images.githubusercontent.com/57449785/116879679-38191c00-ac21-11eb-9a73-2574889cb90f.png)
Isn't requests installed with the sma-mod?

Yes but only within the virtual environment, are you using that as your pythonpath?

Yes but only within the virtual environment, are you using that as your pythonpath?

Yes, I guess so. This is what my script looks like:
#!/bin/sh

torrentid=$6
torrentname=$5
torrentpath=$4
torrentrootpath=$3
torrenttracker=$2
torrentcategory=$1

pathextraction=/mnt/completed/torrent/extracted
pathconversion=/mnt/completed/torrent/converted

pythonpath=/usr/local/sma/venv/bin/python3
smapath=/usr/local/sma
unrarpath=/opt/appdata/qbittorrent/unrarall/unrarall

if [ "$torrentcategory" = "bypass" ]; then
    #echo "skipping this since it's in the bypass category"
    exit 0
fi

if [[ -d "$torrentrootpath" ]]; then
    #echo "$torrentrootpath" + " is a directory"
    mkdir "$pathextraction/$torrentname"
    mkdir "$pathconversion/$torrentname"
    "$unrarpath" -v -o "$pathextraction/$torrentname" "$torrentrootpath"
    extracted=$?
    wait $!
    if [ $extracted -eq 0 ]; then
        if [[ -n $(find "$pathextraction/$torrentname" -type f -size +50M) ]]; then
            "$pythonpath" "$smapath/qBittorrentPostProcess.py" "$torrentcategory" "$torrenttracker" "$pathextraction/$torrentname" "$pathextraction/$torrentname" "$torrentname" "$torrentid"
        else
            "$pythonpath" "$smapath/qBittorrentPostProcess.py" "$torrentcategory" "$torrenttracker" "$torrentrootpath" "$torrentpath" "$torrentname" "$torrentid"
        fi
    else
        "$pythonpath" "$smapath/qBittorrentPostProcess.py" "$torrentcategory" "$torrenttracker" "$torrentrootpath" "$torrentpath" "$torrentname" "$torrentid"
    fi
    rm -rf "$pathextraction/$torrentname"
elif [[ -f "$torrentpath" ]]; then
    #echo "$torrentpath" + " is a file"
    "$pythonpath" "$smapath/qBittorrentPostProcess.py" "$torrentcategory" "$torrenttracker" "$torrentrootpath" "$torrentpath" "$torrentname" "$torrentid"
fi

exit 0

Yeah should be installed then, if you delete your SMA folder and restart the container that should rerun the installation routine and maybe fix it

I'll try that. Thanks!

Yeah should be installed then, if you delete your venv folder inside the SMA folder and restart the container that should rerun the installation routine and maybe fix it

I tried that now and I also recreated the qbit container. This is what I'm getting now:

[cont-init.d] 10-adduser: exited 0.

[cont-init.d] 30-config: executing...

[cont-init.d] 30-config: exited 0.

[cont-init.d] 90-sma-config: executing...

E: Unable to locate package python-pymediainfo

Traceback (most recent call last):

  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main

    return _run_code(code, main_globals, None,

  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code

    exec(code, run_globals)

  File "/config/.local/lib/python3.8/site-packages/pip/__main__.py", line 21, in <module>

    from pip._internal.cli.main import main as _main

  File "/config/.local/lib/python3.8/site-packages/pip/_internal/cli/main.py", line 8, in <module>

    from pip._internal.cli.autocompletion import autocomplete

  File "/config/.local/lib/python3.8/site-packages/pip/_internal/cli/autocompletion.py", line 9, in <module>

    from pip._internal.cli.main_parser import create_main_parser

  File "/config/.local/lib/python3.8/site-packages/pip/_internal/cli/main_parser.py", line 7, in <module>

    from pip._internal.cli import cmdoptions

  File "/config/.local/lib/python3.8/site-packages/pip/_internal/cli/cmdoptions.py", line 20, in <module>

    from pip._vendor.packaging.utils import canonicalize_name

  File "/config/.local/lib/python3.8/site-packages/pip/_vendor/packaging/utils.py", line 9, in <module>

    from .tags import Tag, parse_tag

  File "/config/.local/lib/python3.8/site-packages/pip/_vendor/packaging/tags.py", line 7, in <module>

    import distutils.util

ModuleNotFoundError: No module named 'distutils.util'

Traceback (most recent call last):

  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main

    return _run_code(code, main_globals, None,

  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code

    exec(code, run_globals)

  File "/config/.local/lib/python3.8/site-packages/pip/__main__.py", line 21, in <module>

    from pip._internal.cli.main import main as _main

  File "/config/.local/lib/python3.8/site-packages/pip/_internal/cli/main.py", line 8, in <module>

    from pip._internal.cli.autocompletion import autocomplete

  File "/config/.local/lib/python3.8/site-packages/pip/_internal/cli/autocompletion.py", line 9, in <module>

    from pip._internal.cli.main_parser import create_main_parser

  File "/config/.local/lib/python3.8/site-packages/pip/_internal/cli/main_parser.py", line 7, in <module>

    from pip._internal.cli import cmdoptions

  File "/config/.local/lib/python3.8/site-packages/pip/_internal/cli/cmdoptions.py", line 20, in <module>

    from pip._vendor.packaging.utils import canonicalize_name

  File "/config/.local/lib/python3.8/site-packages/pip/_vendor/packaging/utils.py", line 9, in <module>

    from .tags import Tag, parse_tag

  File "/config/.local/lib/python3.8/site-packages/pip/_vendor/packaging/tags.py", line 7, in <module>

    import distutils.util

ModuleNotFoundError: No module named 'distutils.util'

Traceback (most recent call last):

  File "/usr/lib/python3.8/runpy.py", line 185, in _run_module_as_main

    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)

  File "/usr/lib/python3.8/runpy.py", line 144, in _get_module_details

    return _get_module_details(pkg_main_name, error)

  File "/usr/lib/python3.8/runpy.py", line 111, in _get_module_details

    __import__(pkg_name)

  File "/config/.local/lib/python3.8/site-packages/virtualenv/__init__.py", line 3, in <module>

    from .run import cli_run, session_via_cli

  File "/config/.local/lib/python3.8/site-packages/virtualenv/run/__init__.py", line 14, in <module>

    from .plugin.creators import CreatorSelector

  File "/config/.local/lib/python3.8/site-packages/virtualenv/run/plugin/creators.py", line 6, in <module>

    from virtualenv.create.via_global_ref.builtin.builtin_way import VirtualenvBuiltin

  File "/config/.local/lib/python3.8/site-packages/virtualenv/create/via_global_ref/builtin/builtin_way.py", line 7, in <module>

    from virtualenv.create.creator import Creator

  File "/config/.local/lib/python3.8/site-packages/virtualenv/create/creator.py", line 15, in <module>

    from virtualenv.discovery.cached_py_info import LogCmd

  File "/config/.local/lib/python3.8/site-packages/virtualenv/discovery/cached_py_info.py", line 16, in <module>

    from virtualenv.discovery.py_info import PythonInfo

  File "/config/.local/lib/python3.8/site-packages/virtualenv/discovery/py_info.py", line 16, in <module>

    from distutils import dist

ImportError: cannot import name 'dist' from 'distutils' (/usr/lib/python3.8/distutils/__init__.py)

/var/run/s6/etc/cont-init.d/90-sma-config: line 25: /usr/local/sma/venv/bin/pip: No such file or directory

/var/run/s6/etc/cont-init.d/90-sma-config: line 74: /usr/local/sma/venv/bin/python3: No such file or directory

[cont-init.d] 90-sma-config: exited 127.

[cont-init.d] 99-custom-scripts: executing...

[custom-init] no custom files found exiting...

[cont-init.d] 99-custom-scripts: exited 0.

[cont-init.d] done.

[services.d] starting services

[services.d] done.

Hm looks like that package isn't available on focal, I'm gonna fix that and update the container now, just wait a few minutes for the container to rebuild

Done, rebuild container and try again

Done, rebuild container and try again

It finally executes the script. But it fails when trying to call PYTHON-QBITTORRENT.
All OK

ok

1 rar files found and extracted

2021-05-03 16:18:02 - qBittorrentPostProcess - INFO - qBittorrent post processing started.

2021-05-03 16:18:02 - resources.readsettings - INFO - /usr/local/sma/venv/bin/python3

2021-05-03 16:18:02 - resources.readsettings - INFO - Loading config file /usr/local/sma/config/autoProcess.ini.

2021-05-03 16:18:02 - qBittorrentPostProcess - ERROR - Python module PYTHON-QBITTORRENT is required. Install with 'pip install python-qbittorrent' then try again.

Traceback (most recent call last):

  File "/usr/local/sma/qBittorrentPostProcess.py", line 57, in <module>

    from qbittorrent import Client

ModuleNotFoundError: No module named 'qbittorrent'

2021-05-03 16:18:02 - qBittorrentPostProcess - ERROR - Unexpected exception.

Traceback (most recent call last):

  File "/usr/local/sma/qBittorrentPostProcess.py", line 57, in <module>

    from qbittorrent import Client

ModuleNotFoundError: No module named 'qbittorrent'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "/usr/local/sma/qBittorrentPostProcess.py", line 60, in <module>

    sys.exit(1)

SystemExit: 1```

Added those missing dependencies as well for both qbittorrent and deluge

Woohoo! Let's test :)
On 3 May 2021, 18:29 +0200, Michael Higgins @.***>, wrote:

Added those missing dependencies as well for both qbittorrent and deluge
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

This is how far I got. Not sure what it's missing this time.
2021-05-03 19:42:43 - qBittorrentPostProcess - INFO - qBittorrent post processing started.,
2021-05-03 19:42:43 - resources.readsettings - INFO - /usr/local/sma/venv/bin/python3,
2021-05-03 19:42:43 - resources.readsettings - INFO - Loading config file /usr/local/sma/config/autoProcess.ini.,
2021-05-03 19:42:43 - qBittorrentPostProcess - INFO - Performing conversion,
2021-05-03 19:42:44 - qBittorrentPostProcess - ERROR - No output files generated for the entirety of this mutli file torrent, aborting.,
2021-05-03 19:42:44 - qBittorrentPostProcess - ERROR - Unexpected exception.,
Traceback (most recent call last):,
  File "/usr/local/sma/qBittorrentPostProcess.py", line 132, in <module>,
    sys.exit(1),
SystemExit: 1,

On 3 May 2021, 18:53 +0200, Morgan Zero @.***>, wrote:

Woohoo! Let's test :)
On 3 May 2021, 18:29 +0200, Michael Higgins @.***>, wrote:

Added those missing dependencies as well for both qbittorrent and deluge
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

No files are being converted for some unknown reason, I would make sure your FFMPEG binaries can be executed (ffmpeg and ffprobe) and then trying running a file through manual.py to see if you can get an error

No files are being converted for some unknown reason, I would make sure your FFMPEG binaries can be executed (ffmpeg and ffprobe) and then trying running a file through manual.py to see if you can get an error

I bound then like this in the container.

And this is the autoProcess.ini paths for them
[Converter]
ffmpeg = /usr/local/bin/ffmpeg
ffprobe = /usr/local/bin/ffprobe

Looks correct?

It may be a permissions issue, you don't have to bind FFMPEG its downloaded, installed, and permissions are automatically adjusted as part of sma-mod though, but if it detects that you already have ffmpeg installed it won't go through that process so you may be inadvertently messing it up by binding it

Reason I did that was because container logs said it couldn't find ffmpeg/ffprobe. I pasted logs here.
[cont-init.d] 90-sma-config: executing... ,
/var/run/s6/etc/cont-init.d/90-sma-config: line 62: wget: command not found,
tar (child): /tmp/ffmpeg.tar.xz: Cannot open: No such file or directory,
tar (child): Error is not recoverable: exiting now,
tar: Child returned status 2,
tar: Error is not recoverable: exiting now,
chown: cannot access '/usr/local/bin/ffmpeg': No such file or directory,
chown: cannot access '/usr/local/bin/ffprobe': No such file or directory,
chmod: cannot access '/usr/local/bin/ffmpeg': No such file or directory,
chmod: cannot access '/usr/local/bin/ffprobe': No such file or directory,
[cont-init.d] 90-sma-config: exited 0.,

On 3 May 2021, 20:16 +0200, Michael Higgins @.***>, wrote:

It may be a permissions issue, you don't have to bind FFMPEG its downloaded, installed, and permissions are automatically adjusted as part of sma-mod though, but if it detects that you already have ffmpeg installed it won't go through that process so you may be inadvertently messing it up by binding it
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

That might have been from earlier when the apt-get install command was failing, wget should be included in that initial package request, is it still giving you that error?

Yeah. This one I got just before I posted it for you. I had removed bindings and rebuilt.
On 3 May 2021, 20:26 +0200, Michael Higgins @.***>, wrote:

That might have been from earlier when the apt-get install command was failing, wget should be included in that initial package request, is it still giving you that error?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

Hm that doesn't make any sense, does wget work if you just try it from the command line?

Is it ffmpeg.tar.xz or GZ?
On 3 May 2021, 20:31 +0200, Michael Higgins @.***>, wrote:

Hm that doesn't make any sense, does wget work if you just try it from the command line?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

Should matter, the command that's failing is wget
wget: command not found,

From inside the container I get:
bash: wget: command not found
On 3 May 2021, 20:34 +0200, Michael Higgins @.***>, wrote:

Shouldn't matter, the command that's failing is wget
wget: command not found,
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

Does running apt-get install wget do anything?

Looks like it doesn't wanna be friends with wget.
***@***.***:/# apt-get install wget
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package wget

On 3 May 2021, 20:37 +0200, Michael Higgins @.***>, wrote:

Does running apt-get install wget do anything?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

https://packages.ubuntu.com/focal/wget
Looks like wget should be available. If you run apt-get update first then try apt-get install wget does that help?

Yeah I just ran this
apt-get update && apt-get install apt-file -y && apt-file update && apt-get install wget -y
And it seems to be working now. Going to try and execute the script once again and see if it works now :)
On 3 May 2021, 20:43 +0200, Michael Higgins @.***>, wrote:

https://packages.ubuntu.com/focal/wget
Looks like wget should be available. If you run apt-get update first then try apt-get install wget does that help?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

The question is how did this fail in the first place. The 90-sma-config includes the apt-get update and apt-get install wget options so somehow they must have been skipped, which makes me worry you might be missing other dependent packages as well
The script checks to see if the virtual environment has been created to determine if it should run these steps, I'm going to decouple a few of the dependency checks from this and update the container

Okay, so everytime I rebuild the container wget stops working. Perhaps adding in apt-get update to your mod is necessary?
On 3 May 2021, 20:45 +0200, Morgan Zero @.***>, wrote:

Yeah I just ran this
apt-get update && apt-get install apt-file -y && apt-file update && apt-get install wget -y
And it seems to be working now. Going to try and execute the script once again and see if it works now :)
On 3 May 2021, 20:43 +0200, Michael Higgins @.***>, wrote:

https://packages.ubuntu.com/focal/wget
Looks like wget should be available. If you run apt-get update first then try apt-get install wget does that help?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

This is the log if it helps
[cont-init.d] 10-adduser: exited 0.

[cont-init.d] 30-config: executing...

[cont-init.d] 30-config: exited 0.

[cont-init.d] 90-sma-config: executing...

/var/run/s6/etc/cont-init.d/90-sma-config: line 62: wget: command not found

tar (child): /tmp/ffmpeg.tar.xz: Cannot open: No such file or directory

tar (child): Error is not recoverable: exiting now

tar: Child returned status 2

tar: Error is not recoverable: exiting now

chown: cannot access '/usr/local/bin/ffmpeg': No such file or directory

chown: cannot access '/usr/local/bin/ffprobe': No such file or directory

chmod: cannot access '/usr/local/bin/ffmpeg': No such file or directory

chmod: cannot access '/usr/local/bin/ffprobe': No such file or directory

[cont-init.d] 90-sma-config: exited 0.

[cont-init.d] 99-custom-scripts: executing...

[custom-init] no custom files found exiting...

[cont-init.d] 99-custom-scripts: exited 0.

[cont-init.d] done.

[services.d] starting services

[services.d] done.

On 3 May 2021, 20:49 +0200, Michael Higgins @.***>, wrote:

The question is how did this fail in the first place. The 90-sma-config includes the apt-get update and apt-get install wget options so somehow they must have been skipped, which makes me worry you might be missing other dependent packages as well
The script checks to see if the virtual environment has been created to determine if it should run these steps, I'm going to decouple a few of the dependency checks from this and update the container
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

apt-get update is included its one of the first lines, it's probably not getting executed though and I just updated the conditions to hopefully help, the container rebuild just finished

Let's try again :)
On 3 May 2021, 20:53 +0200, Michael Higgins @.***>, wrote:

apt-get update is included its one of the first lines, it's probably not getting executed though and I just updated the conditions to hopefully help, the container rebuild just finished
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

Container just got rebuilt. Took a while, longer than usual.
Just thought I'd share the logs with you with the warnings I got before I try and execute the script.
[mod-init] Attempting to run Docker Modification Logic

[mod-init] Applying mdhiggins/sma-mod:latest files to container

[s6-init] making user provided files available at /var/run/s6/etc...exited 0.

[s6-init] ensuring user provided files have correct perms...exited 0.

[fix-attrs.d] applying ownership & permissions fixes...

[fix-attrs.d] done.

[cont-init.d] executing container initialization scripts...

[cont-init.d] 01-envfile: executing...

[cont-init.d] 01-envfile: exited 0.

[cont-init.d] 10-adduser: executing...

-------------------------------------

         _         ()

        | |  ___   _    __

        | | / __| | |  /  \

        | | \__ \ | | | () |

        |_| |___/ |_|  \__/

Brought to you by linuxserver.io

-------------------------------------

To support LSIO projects visit:

https://www.linuxserver.io/donate/

-------------------------------------

GID/UID

-------------------------------------

User uid:    1000

User gid:    1000

-------------------------------------

[cont-init.d] 10-adduser: exited 0.

[cont-init.d] 30-config: executing...

[cont-init.d] 30-config: exited 0.

[cont-init.d] 90-sma-config: executing...

WARNING: The directory '/config/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.

WARNING: Running pip as root will break packages and permissions. You should install packages reliably by using venv: https://pip.pypa.io/warnings/venv

WARNING: The directory '/config/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.

WARNING: Running pip as root will break packages and permissions. You should install packages reliably by using venv: https://pip.pypa.io/warnings/venv

[cont-init.d] 90-sma-config: exited 0.

[cont-init.d] 99-custom-scripts: executing...

[custom-init] no custom files found exiting...

[cont-init.d] 99-custom-scripts: exited 0.

[cont-init.d] done.

[services.d] starting services

[services.d] done.

On 3 May 2021, 20:53 +0200, Michael Higgins @.***>, wrote:

apt-get update is included its one of the first lines, it's probably not getting executed though and I just updated the conditions to hopefully help, the container rebuild just finished
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

Looks like things worked this time without any serious errors, good sign

I don't think you understand how happy these lines make me 😂
2021-05-03 21:16:05 - resources.mediaprocessor - INFO - Starting conversion.

2021-05-03 21:16:05 - resources.mediaprocessor - INFO - FFmpeg command:

2021-05-03 21:16:05 - resources.mediaprocessor - INFO - ======================

2021-05-03 21:16:05 - resources.mediaprocessor - INFO - /usr/local/bin/ffmpeg -fix_sub_duration -i /mnt/completed/torrent/extracted/video.mkv

Although, I cannot see the process bar so I don't really know if it's converting.
2021-05-03 21:16:05 - resources.mediaprocessor - INFO - ======================

Possible to add?
On 3 May 2021, 21:14 +0200, Michael Higgins @.***>, wrote:

Looks like things worked this time without any serious errors, good sign
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

Nah the technique that's used to write/rewrite the line for the progress bar doesn't translate over nicely into logs for other applications, but glad its working!

Thanks alot for your support!
On 3 May 2021, 21:24 +0200, Michael Higgins @.***>, wrote:

Nah the technique that's used to write/rewrite the line for the progress bar doesn't translate over nicely into logs for other applications, but glad its working!
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

Would this exact process and script work if I  switched out $smapath/qBittorrentPostProcess.py to $smapath/delugePostProcess.py ?

Not sure if this is a big issue. It successfully competed the conversion, however it failed at deleting the temporary folder.
2021-05-03 22:29:24 - resources.mediaprocessor - INFO - ======================

2021-05-03 22:38:52 - resources.mediaprocessor - INFO - /mnt/completed/torrent/radarr/785361a14565b462ab412812ba57da48eafa5bd0-convert/video.mkv created.

2021-05-03 22:38:52 - qBittorrentPostProcess - INFO - Passing /mnt/completed/torrent/radarr/785361a14565b462ab412812ba57da48eafa5bd0-convert directory to Radarr.

2021-05-03 22:38:52 - autoprocess.radarr - INFO - Radarr notifier started.

2021-05-03 22:38:52 - autoprocess.radarr - INFO - Requesting Radarr to scan directory '/mnt/completed/torrent/radarr/785361a14565b462ab412812ba57da48eafa5bd0-convert'.

2021-05-03 22:38:54 - autoprocess.radarr - INFO - Radarr response: queued.

2021-05-03 22:38:54 - qBittorrentPostProcess - ERROR - Unable to delete temporary directory

Traceback (most recent call last):

 File "/usr/local/sma/qBittorrentPostProcess.py", line 200, in <module>

   os.rmdir(delete_dir)

OSError: [Errno 39] Directory not empty: '/mnt/completed/torrent/radarr/785361a14565b462ab412812ba57da48eafa5bd0-convert

On 3 May 2021, 21:26 +0200, Morgan Zero @.***>, wrote:

Thanks alot for your support!
On 3 May 2021, 21:24 +0200, Michael Higgins @.***>, wrote:

Nah the technique that's used to write/rewrite the line for the progress bar doesn't translate over nicely into logs for other applications, but glad its working!
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

Found a duplicate entry in line 31. Should one of them be $pathconversion/$torrentname?. Cause right now, the script converts the file inside of the extraction path and calls *arr to scan the extraction path as well.
"$pythonpath" "$smapath/qBittorrentPostProcess.py" "$torrentcategory" "$torrenttracker" "$pathextraction/$torrentname" "$pathextraction/$torrentname" "$torrentname" "$torrentid"

No, it's not a duplicate. There is a check (line 30) to see if there is a file greater than 50MB in size before passing it to either arr application. If after the extraction there is no file greater than 50MB, we're assuming nothing valid was extracted because either there was no archive in the first place or because what was extracted was too small.
So line 31 is if there was a "successful" extraction and line 33 is if there wasn't one and to continue trucking along.

*** Sorry for the edits, long day at the office and want to make sure my wording is right lol***
No, it's not a duplicate.
Lines 29-35 are for when unrarall successfully extracted something. There is a check (line 30) to see if there is a file greater than 50MB in size before passing the extracted path to either arr application. If after the extraction there is no file greater than 50MB, just pass the original path to either arr application. We're assuming that even though an extraction occurred that since the file is so small it's probably wrong anyway.
Lines 36 - 37 are if unrarall did NOT successfully extract anything. So just pass along the original paths.

Oh, no worries @johnwchambers , I'm an edit God lol
That makes sense. Thank you for clarifying mate. I do wonder because I get ERROR - Unable to delete temporary directory after a successful conversion, where it seems it has been converting in extraction path rather than conversion path. Or have I misunderstood the process? :)
And also, is it possible to remove the "-convert" flag from the end of the filename after a successful conversion? Cause *arr puts "-convert" as a release group lol

It probably has to do with the qbit section where I have the output directory already defined to somewhere else. My setup is not super complicated but not a peach to navigate when I was first getting it going lol. If you don't have the output-directory set to something different than the extracted directory then it will convert the file in place and add the "-convert" to the names of folders and files if I recall.
However, if you have that output-directory defined, conversions of extracted (or unextracted) files are done on the fly to this spot. This path is what eventually gets passed to the arr application for ingest. This way my original qbit files will seed for as long as I see fit and any extracted files etc won't mess up their original folder/file structures.
Extracted folders are always cleaned up successfully, the converted folder sometimes has leftover hanging fruit due to more than just the converted file being leftover. This is where I see some errors about deleting the directory as well but I just look past it. This is also a good thing too since I can review what's left if I notice something wasn't pulled in when it should have been. Sometimes Korean TV shows give me issues due to how they name their releases so in the end it works out for me this way.
A copy of my full autoProcess.ini is below as well for comparison. Feel free to email me directly (my email is on my github page) if you want to take this conversation offline. It's been a pleasure thus far! ^^
[Converter]
ffmpeg = /usr/bin/ffmpeg
ffprobe = /usr/bin/ffprobe
threads = 0
hwaccels = dxva2, cuvid, qsv, d3d11va, cuda
hwaccel-decoders = h264_cuvid, h265_cuvid, hevc_cuvid, mjpeg_cuvid, mpeg1_cuvid, mpeg2_cuvid, mpeg4_cuvid, vc1_cuvid, hevc_qsv, h264_qsv
output-directory =
output-format = mp4
output-extension = mp4
temp-extension =
ignored-extensions = nfo, ds_store
copy-to =
move-to =
delete-original = True
sort-streams = False
process-same-extensions = True
force-convert = False
post-process = False
preopts =
postopts = -max_muxing_queue_size, 9999
minimum-size = 0
detailed-progress = False
hwdevices = vaapi:/dev/dri/renderD128
hwaccel-output-format = vaapi:vaapi
wait-post-process = True
bypass-if-copying-all = False
opts-separator = ,
regex-directory-replace = [^\w\-_\. ]

[Permissions]
chmod = 0777
uid = -1
gid = -1

[Metadata]
relocate-moov = True
full-path-guess = True
tag = True
tag-language = eng
download-artwork = False
preserve-source-disposition = True
sanitize-disposition =
strip-metadata = False

[Video]
codec = h264_nvenc, h264, x264, h265_nvenc, hevc, h265, x265
bitrate = 0
crf = -1
crf-profiles = 0:23:4.5M:12M,4100:21:8M:24M,8000:17:10M:30M
max-width = 0
profile =
max-level = 0.0
pix-fmt =
max-bitrate = 10000
filter =
force-filter = False
codec-parameters =
dynamic-parameters = True
preset =
bitrate-ratio =

[Audio]
codec = ac3
languages = eng, kor, jpn, chi, zho, deu, ger
default-language =
channel-bitrate = 128
max-channels = 0
prefer-more-channels = True
filter =
sample-rates =
copy-original = False
first-track-of-language = False
allow-language-relax = True
aac-adtstoasc = False
ignore-truehd = mp4, m4v
first-stream-of-language = False
default-more-channels = True
max-bitrate = 0
force-filter = False
ignored-dispositions =
unique-dispositions = False

[Universal Audio]
codec = aac
first-track-only = False
move-last = False
filter =
first-stream-only = False
move-after = False
channel-bitrate = 128
force-filter = False

[Subtitle]
codec = srt
codec-image-based =
languages = eng, kor
default-language =
encoding = UTF-8
burn-subtitles = False
embed-subs = False
embed-image-subs = False
embed-only-internal-subs = False
attachment-codec =
first-stream-of-language = False
ignore-embedded-subs = False
burn-dispositions =
filename-dispositions = forced
ignored-dispositions =
unique-dispositions = False

[Sonarr]
host = localhost
port = 8989
apikey = asdfasdfasdf
ssl = False
webroot =
force-rename = False
rescan = True
block-reprocess = False

[Radarr]
host = localhost
port = 7878
apikey = asdfasdfasdf
ssl = False
webroot =
force-rename = False
rescan = True
block-reprocess = False

[Sickbeard]
host = localhost
port = 8081
ssl = False
apikey =
webroot =
username =
password =

[Sickrage]
host = localhost
port = 8081
ssl = False
apikey =
webroot =
username =
password =

[CouchPotato]
host = localhost
port = 5050
username =
password =
apikey =
delay = 65
method = renamer
delete-failed = False
ssl = False
webroot =

[SABNZBD]
convert = True
sickbeard-category = sickbeard
sickrage-category = sickrage
couchpotato-category = couchpotato
sonarr-category = tv
radarr-category = movie
bypass-category = bypass
output-directory = None
path-mapping =

[Deluge]
couchpotato-label = couchpotato
sickbeard-label = sickbeard
sickrage-label = sickrage
sonarr-label = tv
radarr-label = movie
bypass-label = bypass
convert = True
host = localhost
port = 58846
username = mp4automator
password = mp4automator
output-directory = /dwnldr/tor/converted
remove = False
path-mapping =

[qBittorrent]
couchpotato-label = couchpotato
sickbeard-label = sickbeard
sickrage-label = sickrage
sonarr-label = tv
radarr-label = movie
bypass-label = bypass
convert = True
action-before =
action-after =
host = localhost
port = 8080
ssl = False
username =
password =
output-directory = /dwnldr/tor/converted
path-mapping =

[uTorrent]
couchpotato-label = couchpotato
sickbeard-label = sickbeard
sickrage-label = sickrage
sonarr-label = sonarr
radarr-label = radarr
bypass-label = bypass
convert = True
webui = False
action-before = stop
action-after = removedata
host = localhost
ssl = False
port = 8080
username =
password =
output-directory =
path-mapping =

[Plex]
host = localhost
port = 32400
refresh = False
token = asdfasdf

[HDR]
space = bt2020nc
transfer = smpte2084
primaries = bt2020
filter =
force-filter = False
codec-parameters =
pix-fmt =
preset =
codec =
profile =

[Subtitle.Subliminal]
download-subs = False
download-hearing-impaired-subs = False
providers =

[Audio.ChannelFilters]
6-2 = pan=stereo|FL=0.5*FC+0.707*FL+0.707*BL+0.5*LFE|FR=0.5*FC+0.707*FR+0.707*BR+0.5*LFE

[Subtitle.Subliminal.Auth]
opensubtitles =
tvsubtitles =

It probably has to do with the qbit section where I have the output directory already defined to somewhere else. My setup is not super complicated but not a peach to navigate when I was first getting it going lol. If you don't have the output-directory set to something different than the extracted directory then it will convert the file in place and add the "-convert" to the names of folders and files if I recall.
However, if you have that output-directory defined, conversions of extracted (or unextracted) files are done on the fly to this spot. This path is what eventually gets passed to the arr application for ingest. This way my original qbit files will seed for as long as I see fit and any extracted files etc won't mess up their original folder/file structures.
Extracted folders are always cleaned up successfully, the converted folder sometimes has leftover hanging fruit due to more than just the converted file being leftover. This is where I see some errors about deleting the directory as well but I just look past it. This is also a good thing too since I can review what's left if I notice something wasn't pulled in when it should have been. Sometimes Korean TV shows give me issues due to how they name their releases so in the end it works out for me this way.
A copy of my full autoProcess.ini is below as well for comparison. Feel free to email me directly (my email is on my github page) if you want to take this conversation offline. It's been a pleasure thus far! ^^
[Converter]
ffmpeg = /usr/bin/ffmpeg
ffprobe = /usr/bin/ffprobe
threads = 0
hwaccels = dxva2, cuvid, qsv, d3d11va, cuda
hwaccel-decoders = h264_cuvid, h265_cuvid, hevc_cuvid, mjpeg_cuvid, mpeg1_cuvid, mpeg2_cuvid, mpeg4_cuvid, vc1_cuvid, hevc_qsv, h264_qsv
output-directory =
output-format = mp4
output-extension = mp4
temp-extension =
ignored-extensions = nfo, ds_store
copy-to =
move-to =
delete-original = True
sort-streams = False
process-same-extensions = True
force-convert = False
post-process = False
preopts =
postopts = -max_muxing_queue_size, 9999
minimum-size = 0
detailed-progress = False
hwdevices = vaapi:/dev/dri/renderD128
hwaccel-output-format = vaapi:vaapi
wait-post-process = True
bypass-if-copying-all = False
opts-separator = ,
regex-directory-replace = [^\w\-_\. ]

[Permissions]
chmod = 0777
uid = -1
gid = -1

[Metadata]
relocate-moov = True
full-path-guess = True
tag = True
tag-language = eng
download-artwork = False
preserve-source-disposition = True
sanitize-disposition =
strip-metadata = False

[Video]
codec = h264_nvenc, h264, x264, h265_nvenc, hevc, h265, x265
bitrate = 0
crf = -1
crf-profiles = 0:23:4.5M:12M,4100:21:8M:24M,8000:17:10M:30M
max-width = 0
profile =
max-level = 0.0
pix-fmt =
max-bitrate = 10000
filter =
force-filter = False
codec-parameters =
dynamic-parameters = True
preset =
bitrate-ratio =

[Audio]
codec = ac3
languages = eng, kor, jpn, chi, zho, deu, ger
default-language =
channel-bitrate = 128
max-channels = 0
prefer-more-channels = True
filter =
sample-rates =
copy-original = False
first-track-of-language = False
allow-language-relax = True
aac-adtstoasc = False
ignore-truehd = mp4, m4v
first-stream-of-language = False
default-more-channels = True
max-bitrate = 0
force-filter = False
ignored-dispositions =
unique-dispositions = False

[Universal Audio]
codec = aac
first-track-only = False
move-last = False
filter =
first-stream-only = False
move-after = False
channel-bitrate = 128
force-filter = False

[Subtitle]
codec = srt
codec-image-based =
languages = eng, kor
default-language =
encoding = UTF-8
burn-subtitles = False
embed-subs = False
embed-image-subs = False
embed-only-internal-subs = False
attachment-codec =
first-stream-of-language = False
ignore-embedded-subs = False
burn-dispositions =
filename-dispositions = forced
ignored-dispositions =
unique-dispositions = False

[Sonarr]
host = localhost
port = 8989
apikey = asdfasdfasdf
ssl = False
webroot =
force-rename = False
rescan = True
block-reprocess = False

[Radarr]
host = localhost
port = 7878
apikey = asdfasdfasdf
ssl = False
webroot =
force-rename = False
rescan = True
block-reprocess = False

[Sickbeard]
host = localhost
port = 8081
ssl = False
apikey =
webroot =
username =
password =

[Sickrage]
host = localhost
port = 8081
ssl = False
apikey =
webroot =
username =
password =

[CouchPotato]
host = localhost
port = 5050
username =
password =
apikey =
delay = 65
method = renamer
delete-failed = False
ssl = False
webroot =

[SABNZBD]
convert = True
sickbeard-category = sickbeard
sickrage-category = sickrage
couchpotato-category = couchpotato
sonarr-category = tv
radarr-category = movie
bypass-category = bypass
output-directory = None
path-mapping =

[Deluge]
couchpotato-label = couchpotato
sickbeard-label = sickbeard
sickrage-label = sickrage
sonarr-label = tv
radarr-label = movie
bypass-label = bypass
convert = True
host = localhost
port = 58846
username = mp4automator
password = mp4automator
output-directory = /dwnldr/tor/converted
remove = False
path-mapping =

[qBittorrent]
couchpotato-label = couchpotato
sickbeard-label = sickbeard
sickrage-label = sickrage
sonarr-label = tv
radarr-label = movie
bypass-label = bypass
convert = True
action-before =
action-after =
host = localhost
port = 8080
ssl = False
username =
password =
output-directory = /dwnldr/tor/converted
path-mapping =

[uTorrent]
couchpotato-label = couchpotato
sickbeard-label = sickbeard
sickrage-label = sickrage
sonarr-label = sonarr
radarr-label = radarr
bypass-label = bypass
convert = True
webui = False
action-before = stop
action-after = removedata
host = localhost
ssl = False
port = 8080
username =
password =
output-directory =
path-mapping =

[Plex]
host = localhost
port = 32400
refresh = False
token = asdfasdf

[HDR]
space = bt2020nc
transfer = smpte2084
primaries = bt2020
filter =
force-filter = False
codec-parameters =
pix-fmt =
preset =
codec =
profile =

[Subtitle.Subliminal]
download-subs = False
download-hearing-impaired-subs = False
providers =

[Audio.ChannelFilters]
6-2 = pan=stereo|FL=0.5*FC+0.707*FL+0.707*BL+0.5*LFE|FR=0.5*FC+0.707*FR+0.707*BR+0.5*LFE

[Subtitle.Subliminal.Auth]
opensubtitles =
tvsubtitles =

Ah, thanks man I do appreciate you guys. I'm so happy I got torrents working again. Have been trying for over a year without success lol.
Thanks again!

Nah the technique that's used to write/rewrite the line for the progress bar doesn't translate over nicely into logs for other applications, but glad its working!

I see what you mean lol

But did you change something? Cause for some reason the progress bar started showing.

The "resume" function seems broken for qBit in the autoProcess.ini. I have tried with and without SSL. Any thoughts?
[#####               ] 24% 2021-05-05 11:21:41 - autoprocess.radarr - INFO - Radarr response: queued.
2021-05-05 11:21:41 - qBittorrentPostProcess - ERROR - Unexpected exception.
Traceback (most recent call last):
  File "/usr/local/sma/qBittorrentPostProcess.py", line 187, in <module>
    qb.resume(torrent_hash)
  File "/usr/local/sma/venv/lib/python3.8/site-packages/qbittorrent/client.py", line 494, in resume
    return self._post('torrents/resume', data={'hashes': infohash.lower()})
  File "/usr/local/sma/venv/lib/python3.8/site-packages/qbittorrent/client.py", line 64, in _post
    return self._request(endpoint, 'post', data, **kwargs)
  File "/usr/local/sma/venv/lib/python3.8/site-packages/qbittorrent/client.py", line 88, in _request
    request.raise_for_status()
  File "/usr/local/sma/venv/lib/python3.8/site-packages/requests/models.py", line 943, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: http://qbit:8098/api/v2/torrents/resume

Double-check your Web UI settings in qbit and make sure the username and password you have defined in your autoProcess.ini match. If all else fails you can bypass authentication if qbit is on the same system as the sma stuff or let it be bypassed by IP.
Reference qbit Web UI settings for my setup is below.

Thanks pal, I’ll test that out :)
On 5 May 2021, 14:22 +0200, John Chambers @.***>, wrote:

Double-check your Web UI settings in qbit and make sure the username and password you have defined in your autoProcess.ini match. If all else fails you can bypass authentication if qbit is on the same system as the sma stuff or let it be bypassed by IP.
Reference qbit Web UI settings for my setup is below.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

https://github.com/linuxserver/docker-mods/issues/160
Seems like a fix is in the work and will make its way out to the various linuxserver images shortly


Enable empty tabs to be clickable to promote contributions

This will require design and copy but the idea is, if a tab has no component, we'd allow the user to click on the tab and they could see a  CTA of  how they can contribute to the pack in the case if a pack lacks a dashboard, synthetic, alert, nerdpack.
Acceptance Criteria

[x] Work with Kike Alonzo on our design approach to encourage contributions on these tabs
[ ] Work with @Calya Budhev on copy for this page - @jpvajda
[ ] Content can be found in Google Drive> UIWRITE-1212: [Observability packs] Write tab intros for the pack details pages
[ ] Default empty state content is show on each tab
[ ] view repo link would work the same as #1447

Content Block
TBD

Hi!
I need some help with the copy, but here there is an early proposal for the empty case. I'd prefer to have a nice illustration there, but I added an icon as it would require more time to do it. Let me know what you think @jpvajda.

view repo link would the same as #1447

I'm confirming with Cayla what content we can use for empty state. I believe there is a draft of it in the this doc: Content can
Google Drive> UIWRITE-1212: [Observability packs] Write tab intros for the pack details pages


Error: generating index SBOM: generating spdx sbom (pull 1441)

I encountered an error during the pipeline execution with APKO after a new build of the repository. It appears that a recent change in the container image is causing the issue. The error is as follows:
2024/12/17 15:36:58 INFO built index file as /tmp/apko-temp-675144136/index.json
Error: generating index SBOM: generating spdx sbom: unable to render index sbom, no architecture images found
2024/12/17 15:36:58 INFO error during command execution: generating index SBOM: generating spdx sbom: unable to render index sbom, no architecture images found

Change Reference:
The issue seems to coincide with the following recent change: [https://github.com/chainguard-dev/apko/pull/1441]
This is the comand to build:
docker run --rm -v "${PWD}":/work -w /work \
       cgr.dev/chainguard/apko build \
       --arch amd64 apko.yaml \
       myapp:apko "/work/myapp.tar" -k melange.rsa.pub

this is the log of execution after pull 1441:

Unable to find image 'cgr.dev/chainguard/apko:latest' locally
latest: Pulling from chainguard/apko
c6641a5e2e67: Pulling fs layer
c6641a5e2e67: Verifying Checksum
c6641a5e2e67: Download complete
c6641a5e2e67: Pull complete
Digest: sha256:960fa7d713be9529c5ec52b3c79af2ca3fcbcef3c579a4f2005291e309b51e98
Status: Downloaded newer image for cgr.dev/chainguard/apko:latest
2024/12/17 14:58:17 INFO Building images for 1 architectures: [amd64]
2024/12/17 14:58:17 INFO detected<EMAIL_ADDRESS>as VCS URL
2024/12/17 14:58:18 [DEBUG] GET https://dl-cdn.alpinelinux.org/alpine/edge/community/apk-configuration
2024/12/17 14:58:19 [DEBUG] GET https://dl-cdn.alpinelinux.org/alpine/edge/main/apk-configuration
2024/12/17 14:58:19 INFO setting apk repositories: [@local ./packages https://dl-cdn.alpinelinux.org/alpine/edge/community https://dl-cdn.alpinelinux.org/alpine/edge/main]
2024/12/17 14:58:21 [DEBUG] GET https://dl-cdn.alpinelinux.org/alpine/edge/community/apk-configuration
2024/12/17 14:58:21 [DEBUG] GET https://dl-cdn.alpinelinux.org/alpine/edge/main/apk-configuration
2024/12/17 14:58:21 INFO setting apk repositories: [@local ./packages https://dl-cdn.alpinelinux.org/alpine/edge/community https://dl-cdn.alpinelinux.org/alpine/edge/main]
2024/12/17 14:58:21 INFO built index file as /tmp/apko-temp-3854543332/index.json
Error: generating index SBOM: generating spdx sbom: unable to render index sbom, no architecture images found
2024/12/17 14:58:21 INFO error during command execution: generating index SBOM: generating spdx sbom: unable to render index sbom, no architecture images found

and this is the log of the version just before pull 1441

2024/12/17 14:43:31 INFO Building images for 1 architectures: [amd64]
2024/12/17 14:43:31 INFO detected<EMAIL_ADDRESS>as VCS URL
2024/12/17 14:43:31 ERRO request failed error="Get<EMAIL_ADDRESS>unsupported protocol scheme \"\"" method=GET<EMAIL_ADDRESS>2024/12/17 14:43:31 WARN ignoring missing keys for @local ./packages: failed to perform key discovery: Get<EMAIL_ADDRESS>GET @local%20./packages/apk-configuration giving up after 1 attempt(s): Get<EMAIL_ADDRESS>unsupported protocol scheme ""
2024/12/17 14:43:32 INFO setting apk repositories: [@local ./packages https://dl-cdn.alpinelinux.org/alpine/edge/community https://dl-cdn.alpinelinux.org/alpine/edge/main]
2024/12/17 14:43:33 INFO installing musl (1.2.5-r8)
2024/12/17 14:43:33 INFO installing libcrypto3 (3.3.2-r4)
..........
2024/12/17 14:43:43 INFO installing python3-dev (3.12.8-r1)
2024/12/17 14:43:43 INFO setting apk repositories: [@local ./packages https://dl-cdn.alpinelinux.org/alpine/edge/community https://dl-cdn.alpinelinux.org/alpine/edge/main]
2024/12/17 14:43:43 WARN /etc/os-release is missing
2024/12/17 14:43:47 INFO built image layer tarball as /tmp/apko-temp-1447558923/apko-x86_64.tar.gz
2024/12/17 14:43:47 INFO OCI layer digest: sha256:f1d6...20da arch=x86_64
2024/12/17 14:43:47 INFO OCI layer diffID: sha256:1a7b...d8f2 arch=x86_64
2024/12/17 14:43:47 INFO built index file as /tmp/apko-temp-1447558923/index.json

I found the problem. Since it is getting the original package list from the configuration, it cannot correctly get the diff here. I sent a PR but probably there is a better solution to it.


Invalid IP Address in Step 3 - <IP_ADDRESS>.254

It's likely that the intent of Step 3 (below) was to have two sets of zeros (0.0.) instead of 3 (0.0.0).
--
3.	Azure added this route when a user-defined route for the <IP_ADDRESS>/24 address prefix was associated to the Subnet1 subnet. Traffic destined for addresses between <IP_ADDRESS> and <IP_ADDRESS>.254 remains within the subnet

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 38bd69ac-a1b5-a401-184d-37b55188a7c4
Version Independent ID: ef976d45-36eb-8a32-4b29-ae03412165d2
Content: Azure virtual network traffic routing
Content Source: articles/virtual-network/virtual-networks-udr-overview.md
Service: virtual-network
GitHub Login: @jimdial
Microsoft Alias: jdial

@dionjr You are correct, thanks for catching. I've fixed the article. It will publish later today. #please-close


Parameterizable cache size

Currently, the cache is fixed into 8-way 64-set 512 bit (32 kB) size, which is quite large.  The parameters are all there in the cache to change this size, but permutations are untested and most likely have subtle bugs.  It would be nice to enable DSE with the cache, bearing in mind that this is a VM-enabled memory system so the homonym and synonym problems apply (http://www.cse.unsw.edu.au/~cs9242/02/lectures/03-cache/node8.html)

Parameterize D$ 32kB / 8 way, 16kB / 4 way, 8 kB / 2 way.
I$ can possibly get away with less associativity due to it being readonly -- investigate
Cache line size is a much larger change (specifically changing the cache line size to be different between the CCE and the LCE), although this will be necessary to support coherent accelerators

This is now supported in dev


bifacial-radiance installs to wrong directory

I followed the YouTube installation guide and managed to get as far as installing bifacial radiance.
It seemed to install correctly using "pip install -e ."
However I get a run error when trying to use radiance in Spyder and I've worked out that bifacial-radiance has somehow installed in the wrong directory.
Instead of installing in:
"C:\Users[USER]\AppData\Local\Continuum\anaconda3\Lib\site-packages"
where I thought it would install, it has rather just installed in the folder where I downloaded the package:
"C:\Users[USER]\Downloads"
Also when checking the version of bifacial-radiance using "pip list" it just shows "0+unknown"
I am new to programming so I might be asking the the wrong people as it might be a python issue, but I thought I'd try here in case.
I'm using Python 3.7.4 on Windows 10 Pro

When you donwload the bifacial_radiance files, it does not install into the Anaconda environment/folder, but rather wherever you have placed your software files. So yes, if you ran the installation in the donwloads folder that's where it will point to.
(I'm not so sure of the pip install bifacial_radiance, maybe that one does go into the anaconda folder? @cdeline might now ~)
Well  I think the steps to fix this should be:

extract the contents of your donwloaded bifacial_radiance-master into the folder where you want it. (i.e. documents\ bifacial_radiance).
Delete downloads\bifacial_radiance-master
2.5) you might try to do a pip uninstall bifacial_radiance ~ I haven't tried it but if it doesn't work, in any case, 3 should redirect to the right folder:
Reinstall from inside documents\bifacial_radiance\      with the pip install . or pip install -e .

Let me know if this works?

Closing this issue - the installation process has been significantly streamlined and explained since this comment.


[TCGC] Missing API version in operation group client when there is scope on parent client

When SDK language specified in scope of @client, generated operation group will miss API version.
client.tsp
@client({
name: "FaceServiceClient",
service: Face,
}, "csharp,python")
@client({
name: "administration.FaceServiceClient",
service: Face,
}, "java")
namespace FaceServiceClient {
@operationGroup
@clientName("FaceListClientImpl", "csharp")
interface FaceList {
...
}
...
}
The client parameter apiVersion was missing in sub-client.
If there is no scope of @client, the apiVersion is client parameter.

Root cause, the apiVersion is path parameter of url, but when there is a scope, the client Initialization of sub-client missing the apiVersion parameter.


CdkDropList breaks when data changes during draging

Reproduction
Use StackBlitz to reproduce your issue: https://stackblitz.com/fork/components-issue
Steps to reproduce:

Make cdkDropList
Add cdkDrag
Set Interval to change data every few seconds

Expected Behavior
What behavior were you expecting to see?
It should continue the dragging and choose the new respective data for that index.
Actual Behavior
What behavior did you actually see?
It reset the dragging and made it impossible to do any further dragging/dropping essentially removing drag/drop till i reload the page
Environment

Angular: 8.2.6
CDK/Material: 8.2.0
Browser(s): Chrome
Operating System (e.g. Windows, macOS, Ubuntu): Windows 10

https://stackblitz.com/edit/angular-dbvkgq

This should work if you add a trackBy to the ngFor. The problem is that there's no way of knowing whether the data was changed because it was removed, or because it was swapped out.

Sorry for the late update, it seems to resolve the issue.
Thanks again :)


Limit doesn't affect count

Is this the correct behaviour?
[59] pry(main)> EventsIndex::Event.all.limit(10).count
=> 15
[60] pry(main)> EventsIndex::Event.all.limit(10).to_a.length
  EventsIndex::Event Search (15.1ms) {:index=>["events"], :type=>["event"], :body=>{:size=>10}}
=> 10

I thought total_count would return the total number of records and count would return the number of records in the current query?

I would not say correct or wrong behavior, but unexpected behavior.
count is meant to use Count API which do not have support to from and size parameters, so limit is ignored.
On the other way, when you call to_a over a query, we use the search API to retrieve data, and then we are able to fetch results accordingly to limit and then value should match (or be smaller than if there isn't that many records at ES).
I agree that this result seems uncanny, but also I don't see use for such use case, so I'll close this issue unless you can give me some other opinion about it, then I'll reopen it.
Thanks


Menu to create/add/remove node can have hidden entry when elements is at the bottom

Describe the Bug

Steps to Reproduce the Bug or Issue

create route with several nodes
click on the 3 dots next to one close to the bottom

Screenshots or Videos
No response
Platform

OS: Linux
Browser: Firefox
Version: online version

Raised an issue for this in react-topology project - https://github.com/patternfly/react-topology/issues/120


Pop implementation

As discussed in https://github.com/younatics/Highlighter/issues/2 here is the POP implementation.
This should be a major version change since it contains breaking changes to the syntax.
Please give it a quick test before merging.
🤘

I will test some data and merge this request. Thanks!

Sorry for late PR :(
Anyway, Thanks!


Feature to load more indexes on indexes page

Hi, first thank you for your package.
I want to know if it's possible to add a pagination or a scrolling feature to load more indexes data like on the task page.
Thank you

@TheOneWhoKnokks Hi,
Do u mean search result data of index?

@TheOneWhoKnokks Thanks!
Updated index list limit up to 1000 in new release version v0.6.12.
It will take some time to run ci, you can try online demo & latest docker image after ci finished.

Thank you @riccox


Superb!! This is what I've long wanted to do on my own since 25 more years ago.

Many customers of mine must have long wanted this to be realized much sooner.  However, I'm now turning 62 and almost retired as my every day is now almost like everybody's holiday ;)

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 17d27a0d-c072-8c9f-7e31-3de60bee9ae8
Version Independent ID: e0c09794-e712-19ff-cb9f-12648d60d4f6
Content: Asynchronous programming in C#
Content Source: docs/csharp/programming-guide/concepts/async/index.md
Product: dotnet-csharp
GitHub Login: @BillWagner
Microsoft Alias: wiwagn

Thank you for the compliments.
Closing as kudos.


Feat: Support custom API calls via Management client

Motivation
If new API endpoint is added / changed, it may take some time for this SDK to reflect the changes. For that reason, there should be an ability for developers to use SDK for custom MAPI calls.

fixed by https://github.com/Kentico/kontent-management-sdk-js/issues/16


Add overview paragraph

I didn't change the text in home.jinja2 significantly because I see it distinguishes between users who are logged in or not, with useful information in both cases.  It could perhaps be shortened a bit by omitting the suggestion to look below, especially if we use a smaller font so that "below" is not very far.

Ah, I was just working on #37 (PR just opened) to clear up a space in the home page for the overview text. After #37 is merged , the overview text can go to overview.md and it will appear side-by-side with "Getting Started".

Ah, I was just working on #37 (PR just opened) to clear up a space in the home page for the overview text. After #37 is merged , the overview text can go to overview.md and it will appear side-by-side with "Getting Started".

I've modified the text concerning sizes.
It's not clear to me that putting the two sections side by side is better than having the overview on top.  Most people coming to the page for the first time should read that paragraph first, but it's short enough that people just wanting to look up the details can skip right to them.  What are the advantages of side-by-side?

I've modified the text concerning sizes.
It's not clear to me that putting the two sections side by side is better than having the overview on top.  Most people coming to the page for the first time should read that paragraph first, but it's short enough that people just wanting to look up the details can skip right to them.  What are the advantages of side-by-side?

It's just style. Right now this PR would result in a very long left column and a very short right column, which do look very balanced.
The overview text needs some more edits. The cosmoDC2 files are in HDF5.
Since the overview text will appear on the front page, does it still make sense to refer to specific catalogs?

It's just style. Right now this PR would result in a very long left column and a very short right column, which do look very balanced.
The overview text needs some more edits. The cosmoDC2 files are in HDF5.
Since the overview text will appear on the front page, does it still make sense to refer to specific catalogs?

Note that the Getting Started page is not linked from the home page, but directly displayed in the home page.

Note that the Getting Started page is not linked from the home page, but directly displayed in the home page.

I know that the old design was to include getting_started.md and it appears your new design includes both overview and getting_started in adjacent columns.   I'm suggesting we stick with the original arrangement: just one column, with the overview paragraph preceding the list.
Including the CosmoDC2 certainly does complicate things.  Once we merge in some version of the currrent changes under consideration I suspect we ought to rethink the whole organization. Most of the information applies to both, but not all.  Perhaps we need a subsection specific to DC2 data, maybe another specific to CosmoDC2.  It depends on how many places we need to say something different about one or the other.

I know that the old design was to include getting_started.md and it appears your new design includes both overview and getting_started in adjacent columns.   I'm suggesting we stick with the original arrangement: just one column, with the overview paragraph preceding the list.
Including the CosmoDC2 certainly does complicate things.  Once we merge in some version of the currrent changes under consideration I suspect we ought to rethink the whole organization. Most of the information applies to both, but not all.  Perhaps we need a subsection specific to DC2 data, maybe another specific to CosmoDC2.  It depends on how many places we need to say something different about one or the other.


Modal transitions bugged by programmatic show/hide

http://getbootstrap.com/javascript/#modals
$('#myModal').modal('show');
$('#myModal').modal('hide');
$('#myModal').modal('show');
$('#myModal').modal('hide');
backdrop not correctly cleaned up due to new backdrop created each time show is called

As noted in the documentation, those methods are asynchronous and return before the modal has actually been shown/hidden. So it doesn't make sense to call them directly one after another like that. You need to listen for the appropriate event (http://getbootstrap.com/javascript/#modals-events ) to fire, indicating that the showing/hiding has completed.


WIP: More fixes

some targeted fixes on the logic that I'm pretty sure are correct. Still doing more testing...marking this as WIP

Abandoning this PR. Found a much more deep issue in the ISR/OSR forms that requires enough rework that it won't fit into the current top-level file interface.


update amqp stubs from "pdezwart/php-amqp"

Hello @wenpeng,
Thank you for your contribution, but it's required to perform some steps before your PR can be approved:
1)Please revert all code-style related changes and leave only notable ones (please see https://github.com/JetBrains/phpstorm-stubs/blob/master/CONTRIBUTING.md for additional notes)
2)Please note that after your changes tests started to fail. Please update your changes so that tests pass again to be able to merge your PR

@isfedorov Hi, Thank you for your review, it's has been updated.


biagent geo_search "breast cancer"  error

when i run biagent geo_search "breast cancer" command, i get this error
def get_geo(geo_id, return_gse=False) -> GSM | tuple[GSM, GSE]:
TypeError: unsupported operand type(s) for |: 'type' and 'types.GenericAlias'
how to fix this?

Hi Di,
this error was caused by the python version (< 3.10). The syntax is only supported in 3.10 or later. You can either upgrade your python version to 3.10 or just simply update the source code from def get_geo(geo_id, return_gse=False) -> GSM | tuple[GSM, GSE]: to def get_geo(geo_id, return_gse=False): to fix that.
Cheers,
Quyu

think you, I have checked my python version and it is version 3.9. I tried to change the source code but got more errors. I also found that the installation was only successful with python 3.9


This does not close the window when form opened via Xrm.Navigation.openForm

Title, this does not close the window. The window acts as if a save action occurred.
The console output states "Offset supplied makes index less than zero."
Code to open a form where Save & Close does not close the window:
                var entityFormOptions = {};
                entityFormOptions["entityName"] = "crm_entitylogicalname";
                entityFormOptions["formId"] = "5C50977D-9654-4DBF-A3B7-3E2F523142FA";
                entityFormOptions["openInNewWindow"] = true;
                entityFormOptions["navbar"] = "off";
                entityFormOptions["cmdbar"] = true;
                entityFormOptions["createFromEntity"] = formContext.data.entity.getEntityReference()

                var formParameters = {};

                // Open the form.
                Xrm.Navigation.openForm(entityFormOptions, formParameters).then(

                function (success)
                {
                    console.log(success);
                },

                function (error)
                {
                    console.log(error);
                });

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: b662c9e4-9905-b2bb-96eb-55093a371f80
Version Independent ID: 1569aa19-e370-40af-6238-469ab1a988a8
Content: close (Client API reference) in model-driven apps - PowerApps
Content Source: powerapps-docs/developer/model-driven-apps/clientapi/reference/formContext-ui/close.md
Service: powerapps
GitHub Login: @KumarVivek
Microsoft Alias: kvivek

@Tim-Townsend Can you confirm if you still see this issue?

@Nkrb Issue no longer occurs, thank you.


IIS - Internet Information Services

Full and short name of product

IIS
Internet Information Services
related to: Microsoft, Windows Server

Does this product have LTS versions? What are the intervals between each LTS version?
Not sure.
What is the website for the product and for its version information?
https://learn.microsoft.com/en-us/lifecycle/products/internet-information-services-iis
Additional context
We already have nginx, Apache, but not IIS.

Duplicates #2495.


Submodule for bootstrap not included correctly

Currently, the bootstrap submodule is not correctly included:
$ git submodule
-beb7de2dd69aa75f4b97e5a44e33deda09a34199 docker-qgis/libqfieldsync

It is missing as reference. In .gitmodules it is mentioned.
Further, the branch v4.6.0 is not existing anymore in the branch.

Hey, @stcz , thanks for digging in and finding this archeology sample. It needs to be removed, see #582 .


Improvements and fixes

Add support of almost all fluid tanks

Add highlighting of dropped items (configurable, enabled by default)

Block highlight box now equal to block selection shape (if it is big enough) + special handling of double chests

Fix nearest block calculation to look at

Fix slot highlight render with Angelica (it was affected by gl lighting)

Add config option to search items & fluid in GT pipes (disabled by default)

Code refactoring

Have you tested this in the full pack?

Have you tested this in the full pack?

Yes. All screenshots were taken from GTNH

Can you add a screenshot of it finding a fluid in GT pipes?

Can you add a screenshot of it finding a fluid in GT pipes?

Can you add a screenshot of it finding a fluid in GT pipes?

maybe i should also add config option for fluid conduits...

Fluid Conduits fixed

Have you tested this in the full pack with angelica?
Specifically the latest nightly

nightly

Tested on one of the latest 2.7.0 nightly builds with Angelica-alpha50. Everything works fine.

Have you tested this in the full pack with angelica? Specifically the latest nightly

Tested on one of the latest 2.7.0 nightly builds with Angelica-alpha50.

Looks like it works fine on dropped items but is offset for dropped blocks.


Upgrade issue

On my test server the upgrade from 140 to 150 ran fine.
On the production server I did the same steps, however after the upgrade I do not see any of the new features of 150 (eg labels, tiles for images). What I see is still the same as what I had in 140.
However, the About shows the following text: Version v1.5.0-d59c257e
What can I do to check this further out?

I decided not to wait and do a clean re-install and container image reload.
That means I cannot reproduce the issue anymore, I close it.


Jan 19 11:14:01 AM  info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
Jan 19 11:14:01 AM  rake aborted!

** Invoke assets:precompile (first_time)
** Invoke assets:environment (first_time)
** Execute assets:environment
** Invoke environment (first_time)
** Execute environment
** Invoke javascript:build (first_time)
** Execute javascript:build
yarn install v1.22.19
[1/4] Resolving packages...
success Already up-to-date.
Done in 0.21s.
yarn run v1.22.19
$ node esbuild.config.js
✘ [ERROR] Could not resolve ""

resolveDir /app/app/javascript/controllers
path ./**/*_controller.js
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
rake aborted!
jsbundling-rails: Command build failed, ensure yarn is installed and `yarn build` runs without errors
/usr/local/bundle/gems/jsbundling-rails-1.1.1/lib/tasks/jsbundling/build.rake:5:in `block (2 levels) in <main>'
/usr/local/lib/ruby/gems/3.2.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `block in execute'
/usr/local/lib/ruby/gems/3.2.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `each'
/usr/local/lib/ruby/gems/3.2.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `execute'
/usr/local/lib/ruby/gems/3.2.0/gems/rake-13.0.6/lib/rake/task.rb:219:in `block in invoke_with_call_chain'
/usr/local/lib/ruby/gems/3.2.0/gems/rake-13.0.6/lib/rake/task.rb:199:in `synchronize'
/usr/local/lib/ruby/gems/3.2.0/gems/rake-13.0.6/lib/rake/task.rb:199:in `invoke_with_call_chain'
/usr/local/lib/ruby/gems/3.2.0/gems/rake-13.0.6/lib/rake/task.rb:243:in `block in invoke_prerequisites'
/usr/local/lib/ruby/gems/3.2.0/gems/rake-13.0.6/lib/rake/task.rb:241:in `each'
/usr/local/lib/ruby/gems/3.2.0/gems/rake-13.0.6/lib/rake/task.rb:241:in `invoke_prerequisites'
/usr/local/lib/ruby/gems/3.2.0/gems/rake-13.0.6/lib/rake/task.rb:218:in `block in invoke_with_call_chain'
/usr/local/lib/ruby/gems/3.2.0/gems/rake-13.0.6/lib/rake/task.rb:199:in `synchronize'
/usr/local/lib/ruby/gems/3.2.0/gems/rake-13.0.6/lib/rake/task.rb:199:in `invoke_with_call_chain'
/usr/local/lib/ruby/gems/3.2.0/gems/rake-13.0.6/lib/rake/task.rb:188:in `invoke'
/usr/local/lib/ruby/gems/3.2.0/gems/rake-13.0.6/lib/rake/application.rb:160:in `invoke_task'
/usr/local/lib/ruby/gems/3.2.0/gems/rake-13.0.6/lib/rake/application.rb:116:in `block (2 levels) in top_level'
/usr/local/lib/ruby/gems/3.2.0/gems/rake-13.0.6/lib/rake/application.rb:116:in `each'
/usr/local/lib/ruby/gems/3.2.0/gems/rake-13.0.6/lib/rake/application.rb:116:in `block in top_level'
/usr/local/lib/ruby/gems/3.2.0/gems/rake-13.0.6/lib/rake/application.rb:125:in `run_with_threads'
/usr/local/lib/ruby/gems/3.2.0/gems/rake-13.0.6/lib/rake/application.rb:110:in `top_level'
/usr/local/lib/ruby/gems/3.2.0/gems/rake-13.0.6/lib/rake/application.rb:83:in `block in run'
/usr/local/lib/ruby/gems/3.2.0/gems/rake-13.0.6/lib/rake/application.rb:186:in `standard_exception_handling'
/usr/local/lib/ruby/gems/3.2.0/gems/rake-13.0.6/lib/rake/application.rb:80:in `run'
/usr/local/lib/ruby/gems/3.2.0/gems/rake-13.0.6/exe/rake:27:in `<top (required)>'
/usr/local/bin/rake:25:in `load'
/usr/local/bin/rake:25:in `<top (required)>'
/usr/local/bundle/gems/bundler-2.4.1/lib/bundler/cli/exec.rb:58:in `load'
/usr/local/bundle/gems/bundler-2.4.1/lib/bundler/cli/exec.rb:58:in `kernel_load'
/usr/local/bundle/gems/bundler-2.4.1/lib/bundler/cli/exec.rb:23:in `run'
/usr/local/bundle/gems/bundler-2.4.1/lib/bundler/cli.rb:491:in `exec'
/usr/local/bundle/gems/bundler-2.4.1/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
/usr/local/bundle/gems/bundler-2.4.1/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
/usr/local/bundle/gems/bundler-2.4.1/lib/bundler/vendor/thor/lib/thor.rb:392:in `dispatch'
/usr/local/bundle/gems/bundler-2.4.1/lib/bundler/cli.rb:34:in `dispatch'
/usr/local/bundle/gems/bundler-2.4.1/lib/bundler/vendor/thor/lib/thor/base.rb:485:in `start'
/usr/local/bundle/gems/bundler-2.4.1/lib/bundler/cli.rb:28:in `start'
/usr/local/bundle/gems/bundler-2.4.1/exe/bundle:45:in `block in <top (required)>'
/usr/local/bundle/gems/bundler-2.4.1/lib/bundler/friendly_errors.rb:117:in `with_friendly_errors'
/usr/local/bundle/gems/bundler-2.4.1/exe/bundle:33:in `<top (required)>'
/usr/local/bundle/bin/bundle:25:in `load'
/usr/local/bundle/bin/bundle:25:in `<main>'
Tasks: TOP => assets:precompile => javascript:build

@gazayas
The problem lies within the bundle exec rake assets:precompile command which triggers node esbuild.config.js which produces the error [ERROR] Could not resolve "".
I've deployed about 5 times with the same error.

@vsnguyen Thanks for opening your PR at #627. For what it's worth, I just deployed an app to Render with our one-click button from the readme, and all I had to do was set the BASE_URL manually to get it working, so I'm not entirely sure where this is coming from. I haven't done this with Docker yet though, so maybe I'm overlooking a configuration setting there.

@gazayas
Basically, when the env THEME isn't set, the variable themeFile is defaulted to an empty string.  themeFile is then used downstream on esbuild entrypoints which blows up because themeFile is an empty string.

Fixed in https://github.com/bullet-train-co/bullet_train/pull/627


fix cigar case visuals

This fixes an issue where the cigars inside a case would be visible despite the case being closed.
The issue was that some appearance data was only set client side, and was overridden by the server. This code is now server side, which also means that if another user opens the case, then the sprite will properly update.
:cl:

fix: Fixed cigar case visuals. Can no longer see the cigars when case is closed.

That's strange. I was sure this issue was fixed when sloth fixed the visualizer double call. The fact it happened again, is troubling.

What was the issue that was causing the double call?
Because here the root problem is just that if the client's AppearanceComponent data dictionary differs at all from the severs, then it just uses the servers' dictionary, in the process it discards any client-exclusive appearance data and double-calls all the visualizers. Basically client-side appearance data is not supported atm.
I thought about changing appearance component, but then I realised that this wouldn't fix the issue of other players not seeing the case as open if someone else is rummaging through it,

Anyway can confirm, the bug was present on master and this fixes it.


Allow slashes in path parameters

If a path parameter has a slash in it, the request object isn't populated correctly.

Please check : too, I noticed that it had the same effect as / for me.

@urmuzov if you remove ConfigFilter from web.xml does this issue still occur? I believe this issue is actually in the management code.

@tangiel I don't have ConfigFilter in web.xml. I'm on AppEngine Flex Java with ESP

What happens if you disable ESP?

Without ESP it works as expected.
Also checked on AppEngine Standard - it works as expected too.
By expected I mean: if you enter value arn:aws:bla-bla-bla/param1/param2 in API Explorer on pass it though generated client library, the URL will look like /_ah/api/echo/v1/echo2/arn%3Aaws%3Abla-bla-bla%2Fparam1%2Fparam2 and in application code you will see it as arn:aws:bla-bla-bla/param1/param2 again

@urmuzov is the parameter you're using in the middle of your path template or at the end?

@tangiel I'm 100% sure that it failed when I used path like this /path1/{path2}/path3.
I also have paths like this /path1/{path2}, but I can't recall if I caught any exceptions on thiese paths. I went with replacing all path parameters that might have / or : with query parameters as temporary workaround.
If it's critical for your investigation I can test it

I simply ask because this behavior is actually by design. When you have {variable} then it doesn't accept the slash. You can do {variable=**} to accept them, however it has to be the last path fragment in the template for ESP to accept it. We can generate it for the last fragment so you don't have to modify it, but it sounds like your use case falls outside of this use case.

I still finding it very hard to understand why If I call URL like this:
/_ah/api/echo/v1/echo2/arn%3Aaws%3Abla-bla-bla%2Fparam1%2Fparam2/something
ESP (or endpoints) trying to decode all %3A and %2F first and then parse the string? Why not the opposite way?
I also found this behaviour only on AppEngine Flex Java + Endpoints v2 + ESP.
I tested these combinations previously and they all have different way to handle path parameters (the way that I'm used to):

AppEngine Standard + Endpoints v2 (currently supported combination)
Old-style Managed VMs (java-compat) + Endpoints v2 (no ESP)
Old-style Managed VMs (java-compat) + Endpoints v1
AppEngine Standard + Endpoints v1

I assume that if you use same framework on Standard or Flex you should get the same results, but looks like they differ in this regard at least. (and couple other things that are out of scope of this issue)

This is Endpoints v2 on AppEngine Standard Java:

@urmuzov perhaps I spoke too soon about it being by design--I may have misread some documents. I'm following up with the ESP team now to see what they think.

@urmuzov the ESP team will take up this bug. I'm adding the owner of the bug here.

@qiwzhang any updates on this issue?

I just release ESP 1.2.0 (5 minutes ago) with fix.   As long as you are not using FLEX,  you can re-deploy your service to get the fix.  For FLEX, you may have to wait for a couple weeks.

Great news! Is there a way for me to get ESP version my services are running on? And if there is an update available?
PS I'm on flex

Unfortunately, Flex team builds its own docker image in their release circle (a couple of weeks.)  You have to wait.

@urmuzov is this issue fixed for you yet? I've not had the time to test on Flex.

Not,  Flex next proxy image will carry this fix.  Not sure when they will
build it.
On Mon, May 8, 2017 at 1:04 PM, Daniel T<EMAIL_ADDRESS>wrote:

@urmuzov https://github.com/urmuzov is this issue fixed for you yet?
I've not had the time to test on Flex.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/cloudendpoints/endpoints-java/issues/78#issuecomment-299975187,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ALPph-MgB50e0ubPTwiLRsH3znueyYOiks5r33VjgaJpZM4Mr8kP
.

@tangiel @qiwzhang Checked, the issue has been fixed!
Thanks!

@tangiel @qiwzhang I was celebrating too early.
Your fix will fork for API Explorer and direct HTTP calling, if you use generated library that uses google-http-client:1.22.0 and google-api-client:1.22.0 you can't get it working at all.
If I use value that contains : it will go through: com.google.api.client.http.UriTemplate

and com.google.api.client.util.escape.CharEscapers

and finally com.google.api.client.util.escape.PercentEscaper

which will NOT encode it.
If I pre-encode it myself to %3A it will be double encoded to %253A and will not work either.
I assume that it's almost impossible to make changes in google-http-client:1.22.0 and google-api-client:1.22.0 because so much artifacts depend on them, and possibly you guys should become compatible with these libraries. Which means to allow both variants : as : and as %3A
What do you think?

I emailed the owner of the Java client, but you may also want to file a bug against the repository at https://github.com/google/google-api-java-client.

ESP supports custom verb.  such as :verb.  If : is at last part, after last slash,  it will be treated as a verb.

@qiwzhang And how to disable them? There are no "custom verbs" in java cloud endpoints framework https://cloud.google.com/endpoints/docs/frameworks/java/annotations so everybody on this framework doesn't need them in ESP, because they can't use them.
I suppose design decision has been made a while ago, but doing custom verbs via ...:verb, effectively bans perfectly fine symbol from using in url. And breaks compatibility with google-api-client.
And why not check if there is such verb in openapi.json first and if not - then pass whole url to the original verb?

@urmuzov I just want to make sure that disabling custom verb will indeed solve your problem.  Only if ":" is in the last path segment, ESP will treat it as custom verb.  For your specific use cases, could you help to test if ":" in the middle works for you?  If no, it could be other problems.
If only when ":" is in the last segment, it doesn't work.  then it could be caused by custom verb.
In either cases, could you show us your service config and the paths you used to test?

@qiwzhang Checked, confirmed. The problem exists when ":" is after last "/", in other parts of URL it works.
Method 1:
https://github.com/cloudaware/cloudmine-amazon/blob/885274ebd347db342a559687a47d38ff5ace7426/src/main/java/com/cloudaware/cloudmine/amazon/codestar/CodeStarApi.java#L125
/_ah/api/codestar/v1/us-east-1:asd/user-profiles - goes into the Application code
Method 2:
https://github.com/cloudaware/cloudmine-amazon/blob/885274ebd347db342a559687a47d38ff5ace7426/src/main/java/com/cloudaware/cloudmine/amazon/codestar/CodeStarApi.java#L142
/_ah/api/codestar/v1/us-east-1/user-profiles/aws:asd - method not found
Service config: cloudmine-amazon.json.zip

Thanks for your help.  It is indeed custom verb caused problem. ESP will
fix it in the next release.
On Tue, Aug 29, 2017 at 12:33 PM, Alexander A. Urmuzov <
<EMAIL_ADDRESS>wrote:

@qiwzhang https://github.com/qiwzhang Checked, confirmed. The problem
exists when ":" is after last "/", in other parts of URL it works.
Method 1:
https://github.com/cloudaware/cloudmine-amazon/blob/
885274ebd347db342a559687a47d38ff5ace7426/src/main/java/com/
cloudaware/cloudmine/amazon/codestar/CodeStarApi.java#L125
/_ah/api/codestar/v1/us-east-1:asd/user-profiles - goes into the
Application code
Method 2:
https://github.com/cloudaware/cloudmine-amazon/blob/
885274ebd347db342a559687a47d38ff5ace7426/src/main/java/com/
cloudaware/cloudmine/amazon/codestar/CodeStarApi.java#L142
/_ah/api/codestar/v1/us-east-1/user-profiles/aws:asd - method not found
Service config: cloudmine-amazon.json.zip
https://github.com/cloudendpoints/endpoints-java/files/1261504/cloudmine-amazon.json.zip
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/cloudendpoints/endpoints-java/issues/78#issuecomment-325774579,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ALPph57WoJCzKv6SF1tb1yvJzb4mMwrNks5sdGeigaJpZM4Mr8kP
.


Sitemap no more optional in 2.0 ?

With this commit https://github.com/BitBagCommerce/SyliusCmsPlugin/commit/5a651cb5e72528e3669b1ab770cf9bbcf02fe8ab
in 2.0, the sitemap is always defined, so if you don't have the sylius sitemap plugin it will fire an error :
  The service "bitbag_sylius_cms_plugin.sitemap_provider.page" has a dependency on a non-existent service "sylius.sitemap_url_factory".

Is it a mistake ? @bitbager

In 2.0 CMS will use the Sitemap plugin by default. We will mention it in the README.


update README.md link

Codecov Report

Merging #553 into master will increase coverage by <.01%.
The diff coverage is 50%.

@@            Coverage Diff             @@
##           master     #553      +/-   ##
==========================================
+ Coverage   80.13%   80.13%   +<.01%
==========================================
  Files         328      328
  Lines       21842    21843       +1
==========================================
+ Hits        17502    17503       +1
  Misses       4340     4340


python3Packages.spyder broken

Describe the bug
Spyder won't build from master (c1741fc12a3329cf4c4509f22c60dfd4d54a192f).
To Reproduce
nixpkgs> nix-build -A python3Packages.spyder
<snipped>
installing
Executing pipInstallPhase
/build/spyder-4.0.0/dist /build/spyder-4.0.0
Processing ./spyder-4.0.0-py3-none-any.whl
ERROR: Could not find a version that satisfies the requirement intervaltree (from spyder==4.0.0) (from versions: none)
ERROR: No matching distribution found for intervaltree (from spyder==4.0.0)
builder for '/nix/store/sbrpfrncamjnhn8bvnx6ryw0b9xx490l-python3.7-spyder-4.0.0.drv' failed with exit code 1
error: build of '/nix/store/sbrpfrncamjnhn8bvnx6ryw0b9xx490l-python3.7-spyder-4.0.0.drv' failed

Additional context
A batch update of python packages bumped spyder from 3.3.6 to 4.0.0, but didn't add the new dependencies such as intervaltree.
https://github.com/NixOS/nixpkgs/blob/c1741fc12a3329cf4c4509f22c60dfd4d54a192f/pkgs/development/python-modules/spyder/default.nix
I'm confused why this version bump was done. Unless I'm missing something huge, I think this derivation would never have successfully built.
I think I can fix this all myself, I'll open up a draft PR as I work on it.
Maintainer information:
# a list of nixpkgs attributes affected by the problem
attribute:
  - python3Packages.spyder
# a list of nixos modules affected by the problem
module:

@marcus7070 did you run into any issues with code completions with 3.3.x?
We won't be able to fulfill the dependency requirements for 4.x (specifically, the jedi package version), so it is probably best to revert to 3.3.x. However, that constraint was added because of issues with newer versions. Given the 3.3.x also needs jedi, I would expect those issues to exist there as well. If that is the case, reverting is pointless and we should just disable the version constraint.

Thanks for having a look @FRidh.

did you run into any issues with code completions with 3.3.x?

I don't use Spyder standalone, it's a component of cq-editor which I maintain. But even in cq-editor, I still don't use code completion. So I can't shed any light on what the Spyder devs mean when they write:
    # This is here until Jedi 0.15+ fixes completions for
    # Numpy and Pandas
    'jedi==0.14.1',

I couldn't find and issues discussing the Jedi problems in either Spyder or Jedi, and python-language-server uses Jedi and was passing tests specifically with numpy and pandas, so I patched the version requirement for nixpkgs in my PR.
But I've chased my tail in a complete circle anyway. Turns out cq-editor doesn't work with Spyder 4.x, so I would have to leave Spyder 3.x in nixpkgs. So reverting to 3.3.x is a good solution to me.
Do you want to revert it or should I do it in my PR?
Note that spyder-kernels would need to be reverted as well. https://github.com/spyder-ide/spyder/blob/47e0d1dc1486755602a5800a90aa964b23bd3e0a/setup.py#L241

A new release came out a few hours ago: https://github.com/spyder-ide/spyder/releases/tag/v4.1.1
Jedi is pinned to 15.2 in this release.
I'll try to fix this myself, but I won't have any time to do it in the next few days.


clippy errors

Just curious what is wrong with clippy in the CI?
e.g. https://github.com/ivmarkov/rust-esp32-std-demo/runs/5257226106?check_suite_focus=true
It seems to be complaining about errors in embedded-svc.
But my understanding is that as we use the --no-deps option we shouldn't be checking for errors in dependencies.
Also if I run the same command locally it works.

Issues with latest Rust nightly and GATs.

Looking at it since an hour or so, but not sure what is going on. Might be a regression in the compiler itself, as the error looks bogus.

Apparently they fixed something because since this morning, we get clean CIs again.
Perhaps this one.


Resolve dependencies via importmap when bundling

This PR wraps the nodeResolve plugin and overrides the resolveId.handler in order to resolve @endo/o and other specifiers that are manually configured in the rollup.config.
While this is not addressed in this PR, it is possible build on this PR to automatically populate the importmap from  index.html.

(I'm all for landing this when you're ready.  No need for fancy index.html importmap auto-population at this time.)


Add images for replacing data connection.

To handle switching between temporary version of connector and published version.

So glad there turns out to be an easy answer. 👍


BossBar doesn't delete after cycling

@TiTo-418 only happens on /cycle 0

I'm assuming this is because the wither spawn packet is sent after the wither despawn packet

@iEli2tyree011 Can we please remove support for /cycle 0? It drives me crazy when people do that.

@dentmaged  No, that's pointless.

maybe /cycle 0 should be a /cycle 1 tick, that may fix the issue

0e1a349

Fixed


[docs] Add cell guide notebook for queries on cell type descendants

Fixes #1029

@nayib-jose-gloria @Bento007 - would you take a quick look?


Question about change voxel_feature_extractor in the config

I want to change the voxel_feature_extractor from "SimpleVoxel" to "VoxelFeatureExtractor"(or V2) in the config file, and use the voxel size in the SECOND paper: [0.2, 0.2, 0.4], max_num_per_voxel = 35.
I modified the num_input_feature to 128 in the middle conv layer, is that right?
the errors:
Traceback (most recent call last):
File "./pytorch/train.py", line 306, in train
ret_dict = net_parallel(example_torch)
File "/home/helanyi/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "/data/second.pytorch.dis/second.pytorch.baseline/second/pytorch/models/voxelnet.py", line 367, in forward
preds_dict = self.network_forward(voxels, num_points, coors, batch_size_dev)
File "/data/second.pytorch.dis/second.pytorch.baseline/second/pytorch/models/voxelnet.py", line 332, in network_forward
voxel_features, coors, batch_size)
File "/home/helanyi/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "/data/second.pytorch.dis/second.pytorch.baseline/second/pytorch/models/middle.py", line 203, in forward
ret = self.middle_conv(ret)
File "/home/helanyi/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "/home/helanyi/anaconda3/lib/python3.7/site-packages/spconv/modules.py", line 130, in forward
input = module(input)
File "/home/helanyi/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "/home/helanyi/anaconda3/lib/python3.7/site-packages/spconv/conv.py", line 177, in forward
use_hash=self.use_hash)
File "/home/helanyi/anaconda3/lib/python3.7/site-packages/spconv/ops.py", line 93, in get_indice_pairs
stride, padding, dilation, out_padding, int(subm), int(transpose), int(use_hash))
RuntimeError: /data/spconv/include/tensorview/helper_launch.h 17
N > 0 assert faild. CUDA kernel launch blocks must be positive, but got N= 0
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "./pytorch/train.py", line 663, in
fire.Fire()
File "/home/helanyi/anaconda3/lib/python3.7/site-packages/fire/core.py", line 138, in Fire
component_trace = _Fire(component, args, parsed_flag_args, context, name)
File "/home/helanyi/anaconda3/lib/python3.7/site-packages/fire/core.py", line 471, in _Fire
target=component.name)
File "/home/helanyi/anaconda3/lib/python3.7/site-packages/fire/core.py", line 675, in _CallAndUpdateTrace
component = fn(*varargs, **kwargs)
File "./pytorch/train.py", line 421, in train
print(json.dumps(example["metadata"], indent=2))
File "/home/helanyi/anaconda3/lib/python3.7/json/init.py", line 238, in dumps
**kw).encode(obj)
File "/home/helanyi/anaconda3/lib/python3.7/json/encoder.py", line 201, in encode
chunks = list(chunks)
File "/home/helanyi/anaconda3/lib/python3.7/json/encoder.py", line 429, in _iterencode
yield from _iterencode_list(o, _current_indent_level)
File "/home/helanyi/anaconda3/lib/python3.7/json/encoder.py", line 325, in _iterencode_list
yield from chunks
File "/home/helanyi/anaconda3/lib/python3.7/json/encoder.py", line 405, in _iterencode_dict
yield from chunks
File "/home/helanyi/anaconda3/lib/python3.7/json/encoder.py", line 438, in _iterencode
o = _default(o)
File "/home/helanyi/anaconda3/lib/python3.7/json/encoder.py", line 179, in default
raise TypeError(f'Object of type {o.class.name} '
TypeError: Object of type ndarray is not JSON serializable

the config is here, the Bold Font are what I modified
model: {
second: {
network_class_name: "VoxelNet"
voxel_generator {
point_cloud_range : [0, -40, -3, 70.4, 40, 1]
# point_cloud_range : [0, -32.0, -3, 52.8, 32.0, 1]
voxel_size : [0.2, 0.2, 0.4]
max_number_of_points_per_voxel : 35
}
voxel_feature_extractor: {
  **module_class_name: "VoxelFeatureExtractorV2"**
  num_filters: [16]
  with_distance: false
  num_input_features: 4
}
middle_feature_extractor: {
  module_class_name: "SpMiddleFHD"
  # num_filters_down1: [] # protobuf don't support empty list.
  # num_filters_down2: []
  downsample_factor: 8
  **num_input_features: 128**
}
rpn: {
  module_class_name: "RPNV2"
  layer_nums: [5]
  layer_strides: [1]
  num_filters: [128]
  upsample_strides: [1]
  num_upsample_filters: [128]
  use_groupnorm: false
  num_groups: 32
  num_input_features: 128
}
loss: {
  classification_loss: {
    weighted_sigmoid_focal: {
      alpha: 0.25
      gamma: 2.0
      anchorwise_output: true
    }
  }
  localization_loss: {
    weighted_smooth_l1: {
      sigma: 3.0
      code_weight: [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]
    }
  }
  classification_weight: 1.0
  localization_weight: 2.0
}
num_point_features: 4 # model's num point feature should be independent of dataset
# Outputs
use_sigmoid_score: true
encode_background_as_zeros: true
encode_rad_error_by_sin: true
sin_error_factor: 1.0

use_direction_classifier: true # this can help for orientation benchmark
direction_loss_weight: 0.2 # enough.
num_direction_bins: 2
direction_limit_offset: 1

# Loss
pos_class_weight: 1.0
neg_class_weight: 1.0

loss_norm_type: NormByNumPositives
# Postprocess
post_center_limit_range: [0, -40, -2.2, 70.4, 40, 0.8]
nms_class_agnostic: false # only valid in multi-class nms

box_coder: {
  ground_box3d_coder: {
    linear_dim: false
    encode_angle_vector: false
  }
}
target_assigner: {
  class_settings: {
    anchor_generator_range: {
      sizes: [1.6, 3.9, 1.56] # wlh
      anchor_ranges: [0, -40.0, -1.00, 70.4, 40.0, -1.00] # carefully set z center
      rotations: [0, 1.57] # DON'T modify this unless you are very familiar with my code.
    }
    matched_threshold : 0.6
    unmatched_threshold : 0.45
    class_name: "Car"
    use_rotate_nms: true
    use_multi_class_nms: false
    nms_pre_max_size: 1000
    nms_post_max_size: 100
    nms_score_threshold: 0.3 # 0.4 in submit, but 0.3 can get better hard performance
    nms_iou_threshold: 0.01

    region_similarity_calculator: {
      nearest_iou_similarity: {
      }
    }
  }
  # anchor_generators: {
  #   anchor_generator_stride: {
  #     sizes: [1.6, 3.9, 1.56] # wlh
  #     strides: [0.4, 0.4, 0.0] # if generate only 1 z_center, z_stride will be ignored
  #     offsets: [0.2, -39.8, -1.00] # origin_offset + strides / 2
  #     rotations: [0, 1.57] # DON'T modify this unless you are very familiar with my code.
  #     matched_threshold : 0.6
  #     unmatched_threshold : 0.45
  #   }
  # }
  sample_positive_fraction : -1
  sample_size : 512
  assign_per_class: true
}

}
}

Have you solved it? I meet the same problem.

@hly92711 did you solve it yet? I ran into the exact same problem. Would be very nice if you could share your insights, please :)

ah, I ran into the same issue in a different repo.
I think the problem is with you changing the voxel size in the z axis. If you look at the SpMiddleFHD code, you see that it's tailored to voxel z size 0.1. Since your voxel size is much bigger, the z-dimension probably collapsed to 1 or 0 somehow, and that caused issues.
You should probably change the SpMiddleFHD layers or adjust back your voxel size


1.74 serialization/list.hpp does not compile

Error message:
c:\lang\boost\1_74_0\boost/serialization/list.hpp(53,33): error C2039: 'library_version_type': is not a member of 'boost::serialization'
It appears that boost::serialization::library_version_type should be changed to boost::archive::library_version_type.

Sorry, wrong component. Ignore.


Init version

Should the Sterling OMS helm reference one of the official helm charts at https://github.com/IBM/charts/blob/master/repo/ibm-helm/ instead of embedding it inside this repo?


Add Odyssey Engine

SteamDB app page links to a few games using this
https://steamdb.info/app/7110/
https://steamdb.info/app/208580/
https://steamdb.info/app/32370/
Brief explanation of the change
Detect Odyssey Engine by "chitin.key".
ToDo: Write description file.

missing non matching tests and description


resource_metering: introduce pub/sub mode

Signed-off-by: Zhenchi<EMAIL_ADDRESS>Make the diagnostic node dependent on TiKV instead of TiKV dependent on the diagnostic node to reverse the dependency to a normal form.
Depends on: https://github.com/pingcap/kvproto/pull/827
Related: https://github.com/pingcap/tidb/pull/29020
What is changed and how it works?

Deprecate config resource-metering.enabled

If exists external subscribers, the recorder will be enabled. If no external subscribers, the recorder will be paused. The config resource-metering.enabled takes no effect anymore.

Provide gRPC service kvproto::resource_usage_agent_grpc::ResourceMeteringPubSub

Clients can subscribe to resource metering records through this service.

Modify trait resource_metering::Client

Add methods is_pending and is_closed. Reporter depends on them to enable/disable the recorder.
Add a new implementation PubClient for connecting the reporter and external subscribers.

Check List
Tests

Integration test
Manual test (add detailed scripts or steps below)

Side effects

Performance regression

Consumes more CPU
Consumes more MEM

Release note
Add resource metering pub/sub gRPC service for publishing records

/cc

Maybe we can hold the change of this PR until the change of Pub/Sub at TiDB side comes to an agreement since mostly they are similar designs.


Change default parameter of the ICC method

This should increase runtime without much loss of accuracy.

decrease runtime?

decrease runtime?

Oh yes, meant decrease :) Edited


Rename clusterOperation flag "reconcilePaused" to "reconciliationPaused"

Rename clusterOperation flag "reconcilePaused" to "reconciliationPaused"
This change should be backported to 23.7 and 23.4.

Is this an actual breaking change or a case of the docs mismatching the implementation?

Is this an actual breaking change or a case of the docs mismatching the implementation?

The documentation was incorrect. The implementation always used reconciliationPaused.

LGTM as this only fixes the docs. Do you take care of back-porting this?

backported to 23.7 and 23.4


Naming Inconsistency with Topo Blazor

Hey, team.
When you generate a sign-in sheet for an activity with Topo Blazor, the file name that it generates with is different to if you generate it with normal Topo.
It doesn't have the name of the activity or the date, which causes problems when generating a bunch of sign-in sheets for different activities, as they all end up with the same name.

Topo Blazor (Left) vs Normal Topo (Right)

Hi,
Thanks for that. My Group Leader brought that to my attention too.
It has been fixed and will be in the next update to Topo Blazor. (I'm still not sure what to call this, but most of the new work will be in the blazor version, so it will just become Topo in the end)