fedora docker build fails

I have tried the docker build on Fedora31 (5.8.13-100.fc31.x86_64, gcc 9.3.1) with a small modification to the docker incantation: append :Z to -v and add -u root:root to share managarm dir.
xbstrap fails with configuring vim (wide char issue)
checking size of wchar_t is 2 bytes... configure: error: failed to compile test program
xbstrap: Action configure of package vim failed

and xbstrap install --all --keep-going ends with
xbstrap: Skipping action configure of base-devel due to failed prerequisites [264/266]
xbstrap: Skipping action build of base-devel due to failed prerequisites [265/266]
xbstrap: Skipping action install of base-devel due to failed prerequisites [266/266]
xbstrap: The following steps failed:
    configure      coreutils
    build          coreutils (prerequisites failed)
    install        coreutils (prerequisites failed)
    configure      vim
    build          vim (prerequisites failed)
    install        vim (prerequisites failed)
    configure      base (prerequisites failed)
    build          base (prerequisites failed)
    install        base (prerequisites failed)
    configure      base-devel (prerequisites failed)
    build          base-devel (prerequisites failed)
    install        base-devel (prerequisites failed)
xbstrap: Plan failed

I have never used xbstrap before and I'd appreciate some pointer. Thanks!

Thanks you for the report. For clarification: you built the docker image from our Dockerfile and only modified the docker command line, right?

Yes I followed https://github.com/managarm/bootstrap-managarm#creating-docker-image-and-container
That gave me permissions error because of host/docker sharing of managarm, and I amended the docker command line.
docker run --group-add root -v $(realpath "$MANAGARM_DIR"):/home/managarm_buildenv/managarm:Z -itu root:root  managarm_buildenv

So I think I know why vim fails, and I’ll try to fix that tomorrow. I’m curious why coreutils failed for you, do you have a error for that as well?

: ~/m/build 12 ; docker --version
Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.
podman version 2.1.1

.... # start docker

root@88d6f33f8eba:/home/managarm_buildenv# ls -la
total 28
drwxr-xr-x. 3 managarm_buildenv managarm_buildenv 4096 Oct 10 21:02 .
drwxr-xr-x. 3 root              root              4096 Oct 10 21:02 ..
-rw-r--r--. 1 managarm_buildenv managarm_buildenv  220 Apr 18  2019 .bash_logout
-rw-r--r--. 1 managarm_buildenv managarm_buildenv 3526 Apr 18  2019 .bashrc
-rw-r--r--. 1 managarm_buildenv managarm_buildenv   70 Oct  9 21:56 .gitconfig
-rw-r--r--. 1 managarm_buildenv managarm_buildenv  807 Apr 18  2019 .profile
drwxr-xr-x. 4 root              root              4096 Oct 10 00:13 managarm
root@88d6f33f8eba:/home/managarm_buildenv# xbstrap install --rebuild coreutils
...
checking for mkfifo... yes
checking whether mkfifo rejects trailing slashes... yes
checking whether mknod can create fifo without root privileges... configure: error: in `/home/managarm_buildenv/managarm/build/pkg-builds/coreutils':
configure: error: you should not run configure as root (set FORCE_UNSAFE_CONFIGURE=1 in environment to bypass this check)
See `config.log' for more details
xbstrap: Action configure of package coreutils failed

https://gist.github.com/jaeheum/544c72f5fc6bdfd46d9f9b30a4c7dc47 has the config.log
I don't like running as root even inside a docker image either... can we get the managarm chown'ed at the image creation time?

Ah, so for the coreutils issue I think you need @avdgrinten. I don’t think we need root at any time during the build (image creation excluded, and even that can be done without due to libguestfs). I’ll need to verify the vim fix, I’m planning on having that done tomorrow

As of 19a3074 the vim configure issue should be fixed, leaving this open for the docker issue that @avdgrinten is looking at

just tried running xbstrap with commit c2266947f5e1eacc6c7dba464d7c5a340bc703ae (HEAD -> master, origin/master, origin/HEAD)

xbstrap install --rebuild coreutils runs into the same problem still.
something in tzdata is also wonky

xbstrap: checkout tzdata [164/287]
Traceback (most recent call last):
  File "/usr/local/bin/xbstrap", line 484, in <module>
    do_install(args)
  File "/usr/local/bin/xbstrap", line 358, in do_install
    plan.run_plan()
  File "/usr/local/lib/python3.7/dist-packages/xbstrap/base.py", line 2297, in run_plan
    checkout_src(self._cfg, subject, self._settings)
  File "/usr/local/lib/python3.7/dist-packages/xbstrap/base.py", line 1365, in checkout_src
    prefix = source['extract_path'] + '/'
KeyError: 'extract_path'
root@df8f7b7270dc:/home/managarm_buildenv/managarm/build#

Seems like you have an old xbstrap version, can you try upgrading xbstrap with pip3 install -U xbstrap and try it again?

The docker root issue is indeed not yet fixed, pinging @avdgrinten for that again

is xbstrap 0.16 reasonably up-to-date? (it is giving me the same error)
with src/ at 3ad1f4b84049869ff1e61be860980accbdc54909 xbstrap isn't too happy with the yml files anyways
xbstrap: Failed to validate boostrap.yml
         * Error in file: ../src/bootstrap.yml, YAML element: general
           Additional properties are not allowed ('everything_by_default' was unexpected)
         * Error in file: ../src/bootstrap.yml, YAML element: packages/26
           Additional properties are not allowed ('stability_level' was unexpected)
xbstrap: Validation issues will become hard errors in the future
xbstrap: Failed to validate boostrap.yml
         * Error in file: ../src/bootstrap.d/app-admin.yml, YAML element: packages/0
           Additional properties are not allowed ('stability_level' was unexpected)
xbstrap: Validation issues will become hard errors in the future
xbstrap: Failed to validate boostrap.yml
         * Error in file: ../src/bootstrap.d/dev-libs.yml, YAML element: packages/7
           Additional properties are not allowed ('stability_level' was unexpected)
xbstrap: Validation issues will become hard errors in the future
xbstrap: Failed to validate boostrap.yml
         * Error in file: ../src/bootstrap.d/dev-util.yml, YAML element: packages/2
           Additional properties are not allowed ('stability_level' was unexpected)
xbstrap: Validation issues will become hard errors in the future
xbstrap: Failed to validate boostrap.yml
         * Error in file: ../src/bootstrap.d/sys-apps.yml, YAML element: packages/6
           Additional properties are not allowed ('stability_level' was unexpected)
         * Error in file: ../src/bootstrap.d/sys-apps.yml, YAML element: packages/8
           Additional properties are not allowed ('stability_level' was unexpected)
xbstrap: Validation issues will become hard errors in the future
xbstrap: Failed to validate boostrap.yml
         * Error in file: ../src/bootstrap.d/sys-devel.yml, YAML element: packages/1
           Additional properties are not allowed ('stability_level' was unexpected)
         * Error in file: ../src/bootstrap.d/sys-devel.yml, YAML element: packages/2
           Additional properties are not allowed ('stability_level' was unexpected)
         * Error in file: ../src/bootstrap.d/sys-devel.yml, YAML element: packages/3
           Additional properties are not allowed ('stability_level' was unexpected)
xbstrap: Validation issues will become hard errors in the future
xbstrap: Failed to validate boostrap.yml
         * Error in file: ../src/bootstrap.d/x11-misc.yml, YAML element: packages/0
           Additional properties are not allowed ('stability_level' was unexpected)
xbstrap: Validation issues will become hard errors in the future

xbstrap 0.16 is up to date. It should not result in these validation errors. Is xbstrap inside the Docker image up to date?
Regarding the configure error in binutils: it seems that the configure script is not happy when run as root. Is there any specific reason why you run as root in the container and not as a normal user?

Running the documented docker build and run incatations on Fedora ends with ~/managarm that is chown root:root. (Sigh) I think I have a solution, but it will be at least a few days before I can document it accurately. Building everything takes a long time...

post the exact commands you ran. the docker image has been used for development and in production for building for a few years, it should not result in bad file ownership if your users id is 1000

On Fedora 31, docker is an alias for podman ("rootless docker") and running the documented
docker build -t managarm_buildenv src/docker
docker run -v $(realpath "$MANAGARM_DIR"):/home/managarm_buildenv/managarm -it managarm_buildenv

ends with ~/managarm chown root:root.
If I modify the docker/podman incantation to
sudo docker build -t managarm_buildenv src/docker
sudo docker run -v $(realpath "$MANAGARM_DIR"):/home/managarm_buildenv/managarm:Z -itu managarm_buildenv:managarm_buildenv  managarm_buildenv

then ~/managarm chown managarm_buildenv:managarm_buildenv and xbstrap runs ok.
On Arch, I prefixed the documented docker incantation with sudo.

Thank you for figuring this out. Strangely, on Debian (with Docker CE instead of podman), sudo is not required. We should converge to a solution that does not require sudo on any distribution.

I assume this is a breakage in podman at the end of the day. I will have to investigate personally at some point. podmans intention clearly is to stay compatible with docker, but rootless, so this would seem like a problem.
as for whether sudo is required, it is quite simple: /var/run/docker.sock is how docker communicates with it's daemon, access to this file is access to all of docker itself. it tends to be owned by a docker group (example from my system: Access: (0660/srw-rw----)  Uid: (    0/    root)   Gid: (  994/  docker)), so adding yourself to the docker group is sufficient. I do not think there needs to be any method change here to accommodate for that.
perhaps the reason why this worked with, but not without sudo, if you have actual docker installed, is because sudo will, for reasons obvious, not account for shell aliases, so sudo docker invokes actual docker.


Integration Test - Release - 2022/01/07 06:45:40

Auto create for integration test. IssueCommandFlow

#pri1

Just type '#pri3'

#pri2

#pri3

#assign PRMergerTest1

#assign NonAllowedUser

The following users don't have valid GitHub IDs, or are not collaborators on this repo: nonalloweduser

#reassign PRMergerTest2

#reassign NonAllowedUser

The following users don't have valid GitHub IDs, or are not collaborators on this repo: nonalloweduser

#please-close

GitHub user PRMergerTest2 has been removed from the MicrosoftDocs organization, so they were automatically removed as an assignee.

#unassign PRMergerTest1

#unassign NonAllowedUser


Suggest to apply config at import

When importing a project, it would be nice to detect the import of a .editorconfig file, and when preferences are not in sync with it, to ask user whether they want those settings to be applied.

I've updated the readme to better explain how this plugin currently works - basically by clobbering the global whitespace settings for all supported editor types whenever any file is opened. The way I see it, this approach makes trying to sync up settings with the editorconfig pointless.
There is definitely room for improvement around handling mixtures of projects with and without editorconfig (see #14).
Do you still see this issue as applicable?

Can the support of editorconfig be disabled for a given project?

No, but that's a good idea. If a project does not have editorconfig then it should use the project or workspace settings. Unfortunately the workspace settings have already been clobbered so we need a way of unclobbering them.


jupyter-sparkmagic chart uses k8s APIs Removed In 1.16

Cannot install jupyter-sparkmagic 1.0.0 chart on k8s 1.16+
Error occurred: no matches for kind "Deployment" in version "apps/v1beta2"
https://github.com/jahstreet/spark-on-kubernetes-helm/blob/96f2521b0e20f30fe2dcede5c81239698bd01fb0/charts/jupyter-sparkmagic/templates/deployment.yaml#L2
Deprecated APIs Removed In 1.16: https://kubernetes.io/blog/2019/07/18/api-deprecations-in-1-16/

Thanks for reporting the issue.
1.0.0 charts are built for Spark v2.4.5- (which depends on fabric8 Java Kubernetes client 4.6.1 compatible with K8s API v1.15.3-). I'm preparing charts 2.0.0 for Spark 3.x, which is going to address such issues and extend the supported K8s API versions.

Going to be solved in #39 .

@jahstreet, have you decided not to support jupiter anymore? I see you released new versions for other charts, but jupiter-sparkmagic is missing.

@pdambrauskas , yeah, I've dropped it since spark-cluster contains the JupyterHub, in favour of which I would love to continue support. Also JupyterHub seems to be better alternative to standalone Jupyter on K8s.


MIPS R_MIPS_REL32 relocation not supported

Version and Platform (required):
3.5
Bug Description:
R_MIPS_REL32 relocation not supported.
Steps To Reproduce:
Open the attached binary: libhdk-mips-elf.zip
Go to address 2C014 and see 0C 8F 01 00 (address 0x18F0C) which references string "POST". Now rebase binary to 0x1000. Address 2C014 still points at old address, reference to "POST" is lost.
You will also see these in log:
Unsupported relocation type: 3 (R_MIPS_REL32) @0x2C014
Unsupported relocation type: 3 (R_MIPS_REL32) @0x2C01C
Unsupported relocation type: 3 (R_MIPS_REL32) @0x2C030
Unsupported relocation type: 3 (R_MIPS_REL32) @0x2C054
Unsupported relocation type: 3 (R_MIPS_REL32) @0x2C05C
Unsupported relocation type: 3 (R_MIPS_REL32) @0x2C064
Unsupported relocation type: 3 (R_MIPS_REL32) @0x2C06C
...

Expected Behavior:
Reference to "POST" is maintained after rebase.
Log does not show unsupported relocation messages.

Fixed in arch-mips: https://github.com/Vector35/arch-mips/commit/5c94e6aff1e2e6213f334904e1be3f2b8a925517
BinaryNinja will point to new version soon.


TableTools Extension

Hello budy, I'm working with DataTables in a Yii2 Advanced Application, and I was wondering If I could integrate TableTools Extension with your plug-in. Do you know how to do it? If there is another way to do data export (PDF, CSV, ...) I will be grateful if you give me an example.
Here is the extension doc. https://www.datatables.net/extensions/tabletools/
I'm looking forward to hearing from you!

Hi @nadrianm,
It should be pretty straightforward. We should add a new asset to yii2-widget-datatables with the TableTools plugin (like the bootstrap asset added in v1.1) and then you could use it using the clientOptions array. I will try to add the asset ASAP or feel free to add a PR.
Cheers

Closed via 3e0d4350f49405d2e6cd08e7b7e553eba0529963


Fix building status overlays overlapping with other UI elements

(Feat) Depot Priority
(Feat) Disable Building

The icons on the map are on the same overlay as their associated buttons (which buttons don't conflict by themselves as they are not shown at the same time for now.)
I'll probably have to put each *Overlay GameObject with a separate canvas and render sorting order below the buttons' canvases.

Fixed.


Make this a separate issue, this is out of scope for this issue

Fixed in https://github.com/Goob-Station/Goob-Station/pull/309.


Question: when is Black's next release?

I'm interested in some bugfixes which have merged to master, didn't see any info about release process in the docs I checked. Thanks in advance!

Very soon now—sorry for the long wait!

I'll plan to reopen this in a month if there hasn't been a release :-). No worries -- thanks very much for your hard work on Black!


Build fails on big-endian

Hey!
We're currently running into the following (as expected) on big-endian machines:
   |
10 | / compile_error!(
11 | |     r#"
12 | | This crate doesn't support big-endian targets, since I didn't
13 | | have one to test correctness on.  If you're seeing this message,
14 | | please file an issue!
15 | | "#
16 | | );
   | |__^

error: aborting due to previous error

error: could not compile `merlin`.

FYI, there is a pretty complete tutorial on how to test endian aware code through Qemu https://create.stephan-brumme.com/big-endian/

@n4ss Cool, if you'd like to write a PR that adds endianness testing to CI (could be Github Actions instead of Travis, whichever is easier), I'd be happy to merge it!

Hello,
We are building https://github.com/paritytech/polkadot  on s390x platform, which is a Big Endian. Polkadot is using merlin and it is failing with the same error that has been reported in this.
To get going, I have commented that block and got the polkadot compiled. I have not tested that yet, but  because I commented this, I am foreseeing some issues to come while running.
Wonder if you there is any plan on supporting Big Endian platforms ?
Thanks,
!! Vishwa !!


Saving tmpenv

Hi,
I often pew mktmpenv to run some code. After a while, I sometimes realize I actually want to keep this venv. Is there a way to save it under some name afterwards (could not find it)? Else, I think it would be a great new feature!
Thanks!

Thanks, indeed this use case hasn't been thought about before
You can anyhow accomplish what you asked with pew cp:
pew  dario@feynman ~/P/pew (master)> pew mktmpenv --python=python3.5
Running virtualenv with interpreter /home/dario/Applications/bin/python3.5
Using base prefix '/home/dario/.pythonz/pythons/CPython-3.5.0'
New python executable in 4ff159190479760/bin/python3.5
Also creating executable in 4ff159190479760/bin/python
Installing setuptools, pip, wheel...done.
This is a temporary environment. It will be deleted when you exit
Launching subshell in virtual environment. Type 'exit' or 'Ctrl+D' to return.
4ff159190479760  dario@feynman ~/P/pew (master)> pip install inquirer
Collecting inquirer
 Downloading inquirer-2.1.6.tar.gz
Collecting blessings>=1.5.1 (from inquirer)
 Downloading blessings-1.6.tar.gz
Collecting readchar==0.7 (from inquirer)
 Downloading readchar-0.7.tar.gz
Building wheels for collected packages: inquirer, blessings, readchar
 Running setup.py bdist_wheel for inquirer
 Stored in directory: /home/dario/.cache/pip/wheels/b9/8c/30/94325753c9adca00b0d41eebbcb24483e89162958a9dda9778
 Running setup.py bdist_wheel for blessings
 Stored in directory: /home/dario/.cache/pip/wheels/f3/c9/62/5c19370a7e3b1a8221411d67be2f483e4b1a44b0a9fb2d1217
 Running setup.py bdist_wheel for readchar
 Stored in directory: /home/dario/.cache/pip/wheels/fd/b6/26/319a717eaf7c5c9a9cf8215e0157135cebbb615a359f644a7d
Successfully built inquirer blessings readchar
Installing collected packages: blessings, readchar, inquirer
Successfully installed blessings-1.6 inquirer-2.1.6 readchar-0.7
You are using pip version 7.1.2, however version 8.0.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
4ff159190479760  dario@feynman ~/P/pew (master)> pew cp 4ff159190479760 not-temp
Copying /home/dario/.local/share/virtualenvs/4ff159190479760 in not-temp
Launching subshell in virtual environment. Type 'exit' or 'Ctrl+D' to return.
not-temp  dario@feynman ~/P/pew (master)> exit

4ff159190479760  dario@feynman ~/P/pew (master)> exit

pew  dario@feynman ~/P/pew (master)> pew workon not-temp
Launching subshell in virtual environment. Type 'exit' or 'Ctrl+D' to return.
not-temp  dario@feynman ~/P/pew (master)> python
Python 3.5.0 (default, Sep 14 2015, 09:43:40)
[GCC 4.9.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import inquirer
>>> inquirer.__file__
'/home/dario/.local/share/virtualenvs/not-temp/lib/python3.5/site-packages/inquirer/__init__.py'
>>>

pew cp relies con virtualenv-clone which is known to have problems on Windows and might have other latent bugs.
Let me know if it doesn't work for you, and I can either see to fix it or maybe add some other command

Ok. Thanks! Having an alias for it would be useful though :)


Add bitbucket plugin

wondering if we need reviews here @mariusandra - or rather if the review here should consist of a quick look at the plugin itself

wondering if we need reviews here @mariusandra - or rather if the review here should consist of a quick look at the plugin itself

Other than the name filed in plugin.json in the plugin's repo itself, all seems good here. See here for the "plugin cleanup" discussion.

Other than the name filed in plugin.json in the plugin's repo itself, all seems good here. See here for the "plugin cleanup" discussion.

Ah yes force of habit - I copied over from the GitLab one, will update all of them

Ah yes force of habit - I copied over from the GitLab one, will update all of them


Upload taq to various repositories

Probably PyPI, but also anaconda.org. @davclark can help with conda packaging.

@davclark I would like to lay out everything that we need to do prior to getting this issue resolved. I'll bring up during our meeting later today.


interactive menu in config reset command

#91

Codecov Report

Merging #96 (8dbc64a) into main (c64c3af) will increase coverage by 1.26%.
The diff coverage is 72.72%.

@@            Coverage Diff             @@
##             main      #96      +/-   ##
==========================================
+ Coverage   47.61%   48.87%   +1.26%
==========================================
  Files           8        8
  Lines         733      755      +22
==========================================
+ Hits          349      369      +20
- Misses        384      386       +2

Impacted Files
Coverage Δ

shellfirm/src/config.rs
83.96% <72.72%> (+0.44%)
:arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


fire spreads when being placed on the side of a claim

I am using sponge 7.1.0 and using the griffin prevention plugin, however when putting lava on the side of a claim the lava comes in and spreads fire 1.12.2-4.30.689 the plugin version

GP <IP_ADDRESS>9+ requires SpongeForge 7.1.6 RC3610+(Or SpongeVanilla 7.1.6-RC156+). As of this post, SpongeForge 7.1.6 RC3616(Or SpongeVanilla 7.1.6-RC156) is what we recommend using, and like always the latest GP version available.


[SPARK-25666][PYTHON] Internally document type conversion between Python data and SQL types in normal UDFs

What changes were proposed in this pull request?
We are facing some problems about type conversions between Python data and SQL types in UDFs (Pandas UDFs as well).
It's even difficult to identify the problems (see https://github.com/apache/spark/pull/20163 and https://github.com/apache/spark/pull/22610).
This PR targets to internally document the type conversion table. Some of them looks buggy and we should fix them.
import sys
import array
import datetime
from decimal import Decimal

from pyspark.sql import Row
from pyspark.sql.types import *
from pyspark.sql.functions import udf

if sys.version >= '3':
    long = int

data = [
    None,
    True,
    1,
    long(1),
    "a",
    u"a",
    datetime.date(1970, 1, 1),
    datetime.datetime(1970, 1, 1, 0, 0),
    1.0,
    array.array("i", [1]),
    [1],
    (1,),
    bytearray([65, 66, 67]),
    Decimal(1),
    {"a": 1},
    Row(kwargs=1),
    Row("namedtuple")(1),
]

types =  [
    BooleanType(),
    ByteType(),
    ShortType(),
    IntegerType(),
    LongType(),
    StringType(),
    DateType(),
    TimestampType(),
    FloatType(),
    DoubleType(),
    ArrayType(IntegerType()),
    BinaryType(),
    DecimalType(10, 0),
    MapType(StringType(), IntegerType()),
    StructType([StructField("_1", IntegerType())]),
]

df = spark.range(1)
results = []
count = 0
total = len(types) * len(data)
spark.sparkContext.setLogLevel("FATAL")
for t in types:
    result = []
    for v in data:
        try:
            row = df.select(udf(lambda: v, t)()).first()
            ret_str = repr(row[0])
        except Exception:
            ret_str = "X"
        result.append(ret_str)
        progress = "SQL Type: [%s]\n  Python Value: [%s(%s)]\n  Result Python Value: [%s]" % (
            t.simpleString(), str(v), type(v).__name__, ret_str)
        count += 1
        print("%s/%s:\n  %s" % (count, total, progress))
    results.append([t.simpleString()] + list(map(str, result)))

schema = ["SQL Type \\ Python Value(Type)"] + list(map(lambda v: "%s(%s)" % (str(v), type(v).__name__), data))
strings = spark.createDataFrame(results, schema=schema)._jdf.showString(20, 20, False)
print("\n".join(map(lambda line: "    # %s  # noqa" % line, strings.strip().split("\n"))))

This table was generated under Python 2 but the code above is Python 3 compatible as well.
How was this patch tested?
Manually tested and lint check.

cc @cloud-fan, @viirya and @BryanCutler, WDYT?

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3744/
Test PASSed.

Test build #97046 has started for PR 22655 at commit 3084be1.

Test build #97046 has finished for PR 22655 at commit 3084be1.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97046/
Test PASSed.

it's useful to have this table, thanks!
Shall we discuss the expected behavior here or in another JIRA ticket?

Let me make this table for Pandas UDF too and then open another JIRA (or mailing list) to discuss about this further. I need more investigations to propose the desired behaviour targeting 3.0.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3783/
Test PASSed.

Test build #97097 has started for PR 22655 at commit 3aa0103.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3784/
Test PASSed.

Test build #97098 has started for PR 22655 at commit 6ee69a3.

Test build #97097 has finished for PR 22655 at commit 3aa0103.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97097/
Test PASSed.

Merged build finished. Test PASSed.

Test build #97098 has finished for PR 22655 at commit 6ee69a3.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97098/
Test PASSed.

I am getting this in. This is an ongoing effort and it just documents them internally for now.

Merged to master.

@viirya and @BryanCutler, do you guys have some time to go for Pandas one? I think I wouldn't have some time within a couple of weeks. If you guys have some time, I would appreciate if you could go ahead. Otherwise, I will start this one after a couple of weeks.

@HyukjinKwon I can take some time to do similar for Pandas UDF.

Thanks @viirya !

@HyukjinKwon Cool! Thanks!


Add TMDB images quality (tmdb_images_quality) option.

Now user can choose TMDB images quality (tmdb_images_quality).
depends on https://github.com/elgatito/plugin.video.elementum/pull/990
fixes https://github.com/elgatito/plugin.video.elementum/issues/945
links:
https://www.themoviedb.org/talk/5abcef779251411e97025408
https://github.com/jurialmunkey/plugin.video.themoviedb.helper/pull/638#issuecomment-1031073847
https://github.com/jurialmunkey/plugin.video.themoviedb.helper/blob/efee395ac195cd4d03cc0538408d7318e11b3bf6/resources/tmdbhelper/lib/addon/consts.py#L41
$ curl -s https://api.themoviedb.org/3/configuration?api_key=xxx | jq .images
{
  "base_url": "http://image.tmdb.org/t/p/",
  "secure_base_url": "https://image.tmdb.org/t/p/",
  "backdrop_sizes": [
    "w300",
    "w780",
    "w1280",
    "original"
  ],
  "logo_sizes": [
    "w45",
    "w92",
    "w154",
    "w185",
    "w300",
    "w500",
    "original"
  ],
  "poster_sizes": [
    "w92",
    "w154",
    "w185",
    "w342",
    "w500",
    "w780",
    "original"
  ],
  "profile_sizes": [
    "w45",
    "w185",
    "h632",
    "original"
  ],
  "still_sizes": [
    "w92",
    "w185",
    "w300",
    "original"
  ]
}

@antonsoroko

as stated in elgatito/plugin.video.elementum#998 - this works good and it is ready.

You mean as stated in https://github.com/elgatito/plugin.video.elementum/issues/945
We are still hoping for this :-) https://github.com/elgatito/plugin.video.elementum/issues/998

@matke-84 please try
https://disk.yandex.by/d/5DIzxHtT9P3YEg
https://disk.yandex.by/d/B94U7E4HVfCGbw

@antonsoroko @elgatito
Not good. Now TV Shows have no clearlogo.
Also, is there any possibility that there is an option for fallback artwork? I don't like this at all because the plugin determines what the fallback will be. Don't get me wrong, Kodi's Estuary skin is a spartan skin with few options just to make it as fast as possible.  So who wants fallback through the plugin and who uses these simple skins, click one button to turn it on.
Here is one an example. Elementum, if there is no landscape artwork, posts fanart by itself. My skin, like most others, when there is no landscape image, it puts fanart, but over it with a clearlogo, so that you get what you need. In this case it won't do it because Kodi still sees the landscape artwork. I know, this was already from before, and it was not included in the last update, but I mentioned it as a good example. And that the poster changes the backdrop and vice versa is not a good idea in my opinion. I already explained.
Simply, the plugin should deliver artwork but give skinners the opportunity to decide which type they will use and not force their own.
But of course, you decide, I'm here to test and suggest something. ;-)
Elementum

TMDb Helper

I also do not like the idea of using poster as backdrop and backdrop as poster. It will look awful because of differences in dimensions.
Also as I can see skin developers use a lot of custom logic to make skin look good if some art is missing.
But we try to outsmart them. But they deal with art for a long time, and for us it is just a "theory".

Simply, the plugin should deliver artwork but give skinners the opportunity to decide which type they will use and not force their own.

I like this statement.
We get art from tmdb, we set appropriate (only appropriate) art. If art is missing in tmdb - then we let skin to choose how to replace it.
Also, with all these fallbacks code looks lame and it is harded to follow it logic :-)

Not good. Now TV Shows have no clearlogo.

I made a typo for shows in on of "refactoring" commit. (Clearlogo works for movies) .

when there is no landscape image, it puts fanart, but over it with a clearlogo

I guess we can set landscape only if we found fanart with text. If elgatito agrees.

I also do not like the idea of using poster as backdrop and backdrop as poster. It will look awful because of differences in dimensions. Also as I can see skin developers use a lot of custom logic to make skin look good if some art is missing. But we try to outsmart them. But they deal with art for a long time, and for us it is just a "theory".

If size is different, or the format - we should not use it as a fallback probably.
I am only interested in thumbnails, posters, and background images, as they are used in Estuary, everything else it up to @antonsoroko  and @matke-84 . I believe in you 🥇

Also, with all these fallbacks code looks lame and it is harded to follow it logic :-)

I think we should try to set basic images, used in default skins. Then everything else is used only if available from TMDB/Fanart.tv?
At some point @matke-84 was happy with arts in this PR? What is different from what we have now?

@antonsoroko @elgatito
OK. I've been thinking about this for a while. Here's a suggestion.
I would make an option: "Disable Elementum artwork fallback"
By default, Elementum would do the fallback image in the following way (for TMDb artworks):
Movies/TV Shows
Poster - default poster > cannot have a fallback, we talked about that
Landscape - default backdrop with text > fallback backdrop without text
Fanart - default backdrop without text > fallback backdrop with text
Clearlogo - default clearlogo > cannot have a fallback
TV Show Seasons
Poster - default poster of the Season > fallback main TV Show poster
Landscape - default main TV Show backdrop with text > fallback main TV Show backdrop without text (as far as I know TMDb has nothing but Posters for Seasons, that's why I put the main TV Show backdrop)
Fanart - default main TV Show backdrop without text > fallback main TV Show backdrop with text (as far as I know TMDb has nothing but Posters for Seasons, that's why I put the main TV Show backdrop)
Clearlogo - default clearlogo > cannot have a fallback
TV Show Episodes
Poster - default poster of the Season > fallback main TV Show poster
Landscape - default still > fallback main TV Show backdrop with text > fallback main TV Show backdrop without text (same reason as for seasons)
Fanart - default main TV Show backdrop without text > fallback main TV Show backdrop with text > fallback still (same reason as for seasons)
Clearlogo - default clearlogo > cannot have a fallback
Thumbnail - default still > fallback main TV Show backdrop with text > fallback main TV Show backdrop without text
For the other thumbnails (movies/tv shows/seasons), I would go with that hack: default backdrop without picture, but fallback would put backdrop with picture not poster like you set few days ago. @antonsoroko I don't know if this hack also affects Episodes?
I think I covered everything. :-)
So this would be the default in Elementum.
If we would turn on the option: "Disable Elementum artwork fallback", Elementum would only deliver the default artworks without forcing a fallback, and the skin could then use its own fallback, logic or whatever. And so just satisfied both parties. :-) What do you think about this?
P.S. This is the logic only for TMDb artworks. I don't know if we have to immediately include the Fanart.tv logic, especially since we are also thinking about the combination of TMDb+fallback Fanart.tv. Because for example, if we had this option, then we could use landscape from Fanart.tv instead of the main TV Show backdrop from TMDb, since Fanart.tv supplies such artwork.

@matke-84 @antonsoroko
I feel that I ruined your work in this PR :)
You had everything complete and now it is getting more complex and away from simplification attempts that we wanted to achieve here.
Let's not use all those special fallbacks. And try to use items in rights places, with right destinations.
Then if something would dissapear - we can tackle that afterwards (in another PR/version).
Just make sure to check movie/show/season/episode arts with Fanart enabled and without. That would be enough.

@elgatito
Hahaha. I wrote a novel because of you. :-)
If you ask me and @antonsoroko, we would be the simplest. One artwork for each category and the skin decides further.
I think the solution is to change your skin. :-)

@elgatito

I guess we can set landscape only if we found fanart with text. If elgatito agrees.

Do you agree with this?

Yes, go on.
пт, 8 мар. 2024 г., 18:06 matke-84 @.***>:

@elgatito https://github.com/elgatito
I guess we can set landscape only if we found fanart with text. If
elgatito agrees.
Do you agree with this?
—
Reply to this email directly, view it on GitHub
https://github.com/elgatito/elementum/pull/111#issuecomment-1986079334,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAJQQCSHNJP4DMKU7G5WCB3YXHV23AVCNFSM6AAAAABEBX67MSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOBWGA3TSMZTGQ
.
You are receiving this because you were mentioned.Message ID:
@.***>

@antonsoroko
Ok. Most of it is done.
This should then be returned to the old way, without fallback
https://github.com/elgatito/elementum/pull/111/commits/390b78deeed94a2cab1ceff9507eafb91eed6182
And set landscape as a backdrop with text as we talked.
And then let's test...

@matke-84
1.

And set landscape as a backdrop with text as we talked

Only if such image exists, right? Because by default tmdb has image without text, and then it might have images with text.

What about fallback for thumbnail? Should we try to use poster instead of backdrop, if backdrop is not available? Thumbnail should have landscape/horizontal orientation, and backstop has horizontal orientation, but poster has portrait/vertical orientation, which means that it will look strange.
But if skin does not have its own fallback logic for thumbnail - then we will not have thumbnail at all.
In 0.1.99 we use poster as thumbnail. Now we use backdrop, but i also added fallback logic recently, but I am not if it is needed.
So I need the option of skin maker about this.

@elgatito
Thanks for your understanding :-) I was afraid that in the end we will end up with another tangled logic :-)

This is what tmdb provides in term of images:
Movie/show:
Default: poster, backdrop (without text)
Optional: posters, backdrops (with and without text), logos.
Season:
Default: Poster
Optional: Posters
Episode:
Default: still (screenshot)
Optional: stills (screenshots)
Image type that has text (logo, poster, some backdrops) can be in different languages.
So any other Kodi art type apart from poster (= poster), clear logo (=logo), thumbnail (=still), fanart (=backdrop without text), and sometimes landscape (=backdrop with text) - do not have have match with tmdb image types.
That's why i think we should not try to re-invent complicated fallback logic that skins already invented.

@antonsoroko

Yes, only if exists backdrop with text. If it doesn't exist, it takes over the skin.
I would go without fallback for now. Titan and some other skins treat the thumbnail similarly to the backdrop, that is, as a horizontal image.
I agree. Only in special cases, the skin cannot determine the artwork, for example, if there is no poster of the season, the fallback should be the main poster of the TV Show. The plugin has to do that.
Let's try this way, and if there are any problems, we will make a PR with a fix.

@matke-84
1 2 Ok.
3

if there is no poster of the season, the fallback should be the main poster of the TV Show. The plugin has to do that.

As elgatito mentioned above there is
ListItem.Art(tvshow.poster) | Returns the tv show poster of the parent container. - so for season it will return poster of show, if I understood it correctly. (Although, a link to the source of this statement would be nice.)
I saw that skins uses it, but I am not sure if we should set it or Kodi is smart enough to dynamically calculate it. I hope Kodi calculates it automatically so plugins do not need to set it, it would be logical. Anyway , an investigation is needed.
But this should go to separate "issue".

@antonsoroko
I think it does, not sure. I will test in detail when you send me the test samples. I will test all with popular skins and with the default Kodi skin to see the results.

@matke-84 @elgatito
if there is no translated/localized version of backdrop (for landscape) or logo (for clearlogo) - should we use English version? (we always get English images from TMDB in addition to user's language).
I hesitate because:

if we will not set landscape, then if there is localized logo - skin will use backdrop+logo.
if we set landscape to image with english text, then even if there is localized logo - skin will not use it.

but on the other hand - english landscape (made by person) might looks better that auto generated (by skin) landscape.

and again: for logos - it is better to have english logo or nothing? (or maybe skin will use just "text", but it will be localized text, not in english)

@antonsoroko
Yes, this is a bit tricky. TMDb helper solved it with an option. Can we do something similar?

@matke-84 this option does not do what i speak about. it only controls if plugin gets english images in addition to user's language images.
and it used for available/additional artwork, not for "default" artwork. in tmdb helper "default" art (except poster) always in english (as it returned by tmdb by default), because tmdb returns only localized poster by default, but not backdrop or logo. and tmdb helper does not specifically search for localized landscape or logo, it just uses first images in list.
open https://api.themoviedb.org/3/movie/787699?api_key=29a551a65eef108dd01b46e27eb0554a&append_to_response=images&include_image_language=ru,en,null&language=ru and see poster_path, backdrop_path and "images":{ values.
and then see
landscape is in english

but there is localized landscape https://www.themoviedb.org/movie/787699-wonka/images/backdrops?image_language=ru
same for logo

https://www.themoviedb.org/movie/787699-wonka/images/logos?image_language=ru
but i guess it is just a bug in tmdb helper.
https://github.com/search?q=repo%3Ajurialmunkey%2Fplugin.video.themoviedb.helper iso_639_1&type=code

on the contrary - we always try to set "default" art to localized image.
and we always get english images. so i though that we can also always fallback to english.
i mean i can make an option, but i just do not want to make too many options :-)

@matke-84 btw, should we use different quality for landscape? (different from fanart)

@antonsoroko
The tmdb helper works a little differently, if you want to localize to your language, you have to change the settings in the tmdb helper itself. :-)

btw, should we use different quality for landscape? (different from fanart)

Well, I don't know. Maybe I would go with w1280, w780, w300 (high, medium, low). I don't believe that anyone will use landscape images over the entire screen, they are mostly tiles in all skins.
Also, since the thumb with the hack is now also the backdrop, then we should change the size there as well, since it is currently set for a poster size. This is how it should look like.
//                                                           Original    High    Medium  Low
	ImageQualitiesPoster    = []ImageQualityIdentifier{"original", "w780", "w500", "w342"}
	ImageQualitiesFanArt    = []ImageQualityIdentifier{"original", "w1280", "w1280", "w780"}
	ImageQualitiesLogo      = []ImageQualityIdentifier{"original", "w500", "w500", "w300"}
	ImageQualitiesThumbnail = []ImageQualityIdentifier{"original", "w1280", "w780", "w300"}

What do you think? Do you have any suggestion?

@matke-84

you have to change the settings

i obviously have it set to my language. you can see it on my screenshots - title/genres/etc are translated.
they just do not have localization logic for images, only for text.
i already posted link with proof https://github.com/search?q=repo%3Ajurialmunkey%2Fplugin.video.themoviedb.helper iso_639_1&type=code
:-)
anyway, it is not our issue.

@matke-84 i added landscape sizes since it can speed up thing by downloading smaller images.
for thumbnails - ok, i increased sizes.
thanks.

@matke-84
please test this build: https://disk.yandex.by/d/fCEhDmJ2szprrw
it has all changes we discussed above.
NOTE: you need to check on videos that are NOT in your Kodi library, otherwise you will see images selected by Kodi's scraper you use (e.g. tmdb info plugin).

@antonsoroko
If I can ask you to compile it for Android as well. I should check it on Android as well. But before that, just please change for landscape and thumbnail on w780 for low. I was immediately interested in that to see how it looked. It looks awful on w300. I didn't know it would look so bad. It's a pity that we only have three dimensions for the backdrop. W300 is unusable.
And when you make it for Android, make it again with this change for win64. Sorry and thanks. :-)
ImageQualitiesThumbnail = []ImageQualityIdentifier{"original", "w1280", "w780", "w780"}
ImageQualitiesLandscape = []ImageQualityIdentifier{"original", "w1280", "w780", "w780"}

@matke-84

It's a pity that we only have three dimensions for the backdrop.

it is the same as in tmdb helper - original,high,medium,low.
do you propose to add another option? i am not sure if we need more options.

I should check it on Android as well.

technically, it must look the same on android as on windows. but ok.

change for landscape and thumbnail on w780 for low.

ok.
also, is w300 ok for logo and w342 for poster?
we can use w500  for any image, by the way.
maybe it is better choice than w780.
build with w500:
https://disk.yandex.by/d/r8bE3FAH1ggAPQ
https://disk.yandex.by/d/t_0mcr1GskJldA
build with w780:
https://disk.yandex.by/d/YoQzlunfxOti0Q
https://disk.yandex.by/d/zgCC7lKPkQ-3jw

it is the same as in tmdb helper - original,high,medium,low.
do you propose to add another option? i am not sure if we need more options.

No no. You did not understand me. I meant the backdrop size selection from tmdb. I didn't know that the backdrop could be w500. I followed their table and there it said w300, w780 and w1280. That's why I said three dimensions.

technically, it must look the same on android as on windows. but ok.

I know it's the same, but I'm interested in the opening speed on Android. :-)

also, is w300 ok for logo and w342 for poster?
we can use w500 for any image, by the way.
maybe it is better choice than w780.

The rest of the artworks are not so bad on first look on low, but I will look in detail with more views and skins. Only landscape and thumbnail was terrible on w300. I'll try both. Thanks. ;-) I'm reporting results.

I followed their table and there it said w300, w780 and w1280.

In theory, but in reality they allow to use all supported sizes.

@antonsoroko
First a small suggestion. When a language other than English is used, the posters that exist in that language are displayed ok, and the fallback by default, regardless of the new setting, is English. Maybe it is a better option to go with no language poster, since it is a poster without text. And then when the English fallback option is turned on to display English. Only for seasons it can happen that there is no language poster, but it should take over the main TV show poster. I just don't know if this is possible?
This new option works well for clearlogo and landscape as far as I can tell, I generally use Kodi on English.
w500 for Low quality for Thumbnail and Landscape - yes, it's better for low, I see that you have already taken care of it, great.
In general, for low settings, everything is ok, the purpose is achieved, the quality of all artwork is solid, as it should be with low settings in all programs, the highest speed and solid quality.
I will test some more but I think we got all the settings right. Even with default Kodi skin looks fine for now. :-)
Just one question, do all these new settings also affect the artwork in the OSD menus when the movie is played?

@matke-84
1.

When a language other than English is used, the posters that exist in that language are displayed ok, and the fallback by default, regardless of the new setting, is English.

i do not change poster, since localized poster is supplied by TMDB by default (if it exist, of course), otherwise TMDB uses "default" poster. so we do not change anything - we use logic of TMDB.

Only for seasons it can happen that there is no language poster

it also can happen for "not popular in region" movies and shows.
here is screenshot of https://www.themoviedb.org/movie

so as you can see if TMDB has localized poster - it uses it. otherwise it uses default poster (with language for video poster).
same logic is used for API.
read official explanation here - https://developer.themoviedb.org/docs/image-languages
i mean - i can do what you said - it is a small change.
but i am just not sure if we should override TMDB logic.
also i am not sure if it is ok for skins: for example - if you use wall of posters (no text title) - then you will not see title of movie (since we will use poster without text).
can you please give me some examples when using poster w/o text will be beneficial?

Just one question, do all these new settings also affect the artwork in the OSD menus when the movie is played?

I can't say. I have not touched that part so i am unfamiliar how this works. maybe @elgatito knows.

The only thing I thought was that the languages don't mix by default.

i mean, TMDB mixes them.
but we can make our logic to try to use only "user language". i am just not sure if poster without text is better choice than poster in original language in terms of how it looks in Kodi interface.

@matke-84

I didn't understand this very well. Let's say if I call it with the TMDb helper, there is that possibility, what is the difference from before this change.

I do not know how TMDB Helper is launching Elementum's Player.
There are 2 ways to launch:

Asking to play some torrent/magnet, without specifying TMDB ID, then Elementum will get ListItem, that you clicked in that plugin and use all the Infolabels from it.
Asking to play some torrent/magnet, with specific TMDB and specified media type, then Elementum will get TMDB information and construct ListItem on it's own, so does not matter what was the ListItem that you clicked on.

Correction was about 1), because we were not properly passing ListItem.Art values into Player, so you had information about title/plot, but no arts.

@matke-84
for simplicity and consistency i will use for art (but only for art that must have text) the same fallback logic as for text: we use user's first language, then we use user's second language (which is right now is always English).
@elgatito btw, do you plan to make this SecondLanguage configurable?
this is the latest version of "Add TMDB images quality" plus latest changes from development branch (this is important b/c many changes were made while we were polishing this PR so something can interfere with our changes)
https://disk.yandex.by/d/4srg1Drs-PnogQ
https://disk.yandex.by/d/akKzjD0ySmt14w
please test if you have time. thanks.

@antonsoroko

please test if you have time. thanks.

Of course. I'm reporting the results. ;-)

Is this PR ready for final review? Everything is resolved?

@elgatito btw, do you plan to make this SecondLanguage configurable?

@antonsoroko Now I realize it makes sense to make it selectable. There are also few things about that I need to cover.

@antonsoroko @elgatito
It seems to me that everything is ok. Visually it is better, and we also gained in terms of performance which is even more important. I think users will be satisfied with this changes. :-)
P.S. Take a look at this. https://github.com/elgatito/plugin.video.elementum/issues/1008

Conflicting files
tmdb/tmdb.go

this makes me sad :-D
okay, i will fix this conflict that arose after adding TagLine and then I think we can finally merge this PR.
to note: in comparison with 0.1.99 we will not lose any previous art (poster,fanart,thumbnail), but we should receive new art (logo, landscape). at least i do not see such loses.

this makes me sad :-D

Yes, yes. I had to manually resolve conflicts for every .pot file to add new translation...

@antonsoroko Latest code now has selection of second language in Kodi settings.
There is another interesting endpoint: https://developer.themoviedb.org/reference/configuration-primary-translations
That adds all available variations of translations. So it is language+region.
But it would be hard to use it directly.

@matke-84
hopefully this is the final build. it also has fix for TagLine.
https://disk.yandex.by/d/vjoOOt8eohaQSw
https://disk.yandex.by/d/fYo_N94UtUp76w

@antonsoroko
Select interface and fallback language doesn't work.
Tagline for TV Show work.
I just noticed now that there is a fallback for landscape artwork, backdrop without text. Is that how it should be? Didn't we say that it only shows landscape (backdrop with text)?

@antonsoroko

I just noticed now that there is a fallback for landscape artwork, backdrop without text. Is that how it should be? Didn't we say that it only shows landscape (backdrop with text)?

Sorry my bad. Everything is OK.

@matke-84

Select interface and fallback language doesn't work.

needed commit was added after i made the build. this is not connected to this PR , but here is the new build
https://disk.yandex.by/d/vjoOOt8eohaQSw
https://disk.yandex.by/d/fYo_N94UtUp76w
@elgatito
for test i set the "first language" to Bulgarian and the "second language" to German and it works fine with Art:

and for movie that does not have both those languages it took "original language" poster (Chinese):
https://www.themoviedb.org/movie/1046090
https://image.tmdb.org/t/p/w780/eNvuf4PHepy0nlXfludjGCJGG59.jpg

it is Chinese poster chosen by API, see https://www.themoviedb.org/movie/1046090/images/posters?image_language=zh and if you set languages on website that do not have their own poster (e.g. Bulgarian & German) - you will see this image as well:

@matke-84  @antonsoroko
I am fine with the code. Waiting for last sign off from @matke-84

I just noticed now that there is a fallback for landscape artwork, backdrop without text. Is that how it should be? Didn't we say that it only shows landscape (backdrop with text)?

This is not an issue?

@elgatito see next comment from matke-84
https://github.com/elgatito/elementum/pull/111#issuecomment-1999740005

@elgatito @antonsoroko
Does this have to be split into two buttons? I think it will confuse people. Why not put only button "Interface language" and "interface fallback language".

Are these two options necessary now? The first changes only the title of the movie but not the clearlogo, and the second, as far as I can understand, has no function now, or I didn't notice it.

@antonsoroko @elgatito
About languages. As far as I can see, if there is neither the first inteface language nor the fallback language, the plot, tagline... is taken from the original language too. Is that how it should be?

@matke-84

I do not know why it is done like this - i guess some technical limitation. elgatito can answer better.
But i must say that this is not connected to "TMDB images quality" change, so it should not affect this PR :-)

a. this is from https://github.com/scakemyer/plugin.video.quasar/issues/200 . yes, it is only for movie/show titles. and it is disabled by default. i guess we can delete it, but i do not know if people use it or not. maybe some people prefer to see "original" title, maybe not.
and yes, it forces exactly "original language", because it make sense only with original language, since by default we already use "user's language" -> "second language" -> "original language". so this option override this behavior.
b. it uses https://github.com/elgatito/plugin.video.elementum/blob/master/fanart.jpg as fanart for items that does not have art, like some menus.
I used this option with the best skin in my life :-) - (fuse)neue
(it had the best controls layout for video player, in kodi world, of course. nowadays "regular" video players has much better UX for control elements than Kodi.)
But nowadays I have not seen skins that need such workaround, but i used only few skins.

@antonsoroko @elgatito

@antonsoroko @elgatito About languages. As far as I can see, if there is neither the first inteface language nor the fallback language, the plot, tagline... is taken from the original language too. Is that how it should be?

And this? Is this ok? I think based on the previous answer that it is, but I'd better ask. :-)

@antonsoroko

The real issue with this option is Asian videos - you get hieroglyphs.

Yes, Chinese, Japanese, Arabic, Greek... all languages with non-standard scripts.

b. it uses https://github.com/elgatito/plugin.video.elementum/blob/master/fanart.jpg as fanart for items that does not have art, like some menus.

Ohhhhh, that's what it's for. :-) I didn't know that.

@matke-84

About languages. As far as I can see, if there is neither the first inteface language nor the fallback language, the plot, tagline... is taken from the original language too. Is that how it should be?

And this? Is this ok? I think based on the previous answer that it is, but I'd better ask. :-)

As i already answered

and yes, it forces exactly "original language", because it make sense only with original language, since by default we already use "user's language" -> "second language" -> "original language". so this option override this behavior.

This is not about ok or not ok, but about correctness of logic. Logic is correct, in my opinion. It does exactly what it says. If user do not want to see original title - then user do not enable this option (it disabled by default).
Is it useful nowadays? I think not.
Should we delete this option or not - that's a discussion for a separate issue.

@antonsoroko
You didn't understand me. I'm not talking here about the option that existed for the original title, but about the new display method. If I set as the first language, for example, Russian, as fallback German, if there is no information about movie in either of these two languages, it gives me artwork from the original language, which is ok, but also the title, plot, tagline... from the original language, and if it's a chinese movie i get hieroglyphs.

@matke-84 So, let's try to start from scratch:

but also the title, plot, tagline... from the original language, and if it's a chinese movie i get hieroglyphs.

As I mentioned, with latest changes from elgatito for text (like title, summary, etc):

by default we use "user's language" -> "second language" -> "original language"

But looks like you do not like the fallback of text to original language, right?
Now when I think about it, I guess it is a useless fallback, since user most likely will not be able to read such text. Is it what you mean?
CC: @elgatito

There is nothing to discuss. If there is no text/arts for primary language and nothing for a second language then we show original, bacause that exists for sure.
There are only 2 options: Show original ones or show nothing.
I prefer first.

@antonsoroko @elgatito
OK. Everything is clear. I thought it might be better to use English as the international language for text (not artworks) instead of the original language if there is no primary and second language text. But if it has to be like this, then it's better than nothing.

@matke-84 You mean if you set non-EN as primary and non-EN as secondary language, then instead of "original" it should be EN as the third option and only then "original"?
Because now fallback is always EN, unless you change it in the options, but if you set primary/secondary to non-EN, then EN will not be used at all, true.

@elgatito

You mean if you set non-EN as primary and non-EN as secondary language, then instead of "original" it should be EN as the third option and only then "original"?

Well, yes. I think that's good logic, of course if it's possible. What you think about it and @antonsoroko?

It is possible of course :-) but it is not related to this PR as well :-)
Also, we still need to discuss if we should remove 'prefer original title' and possibility to use only 2 buttons for languages instead 2 buttons + 2 text entries.
But I am sure that we should discuss all these topics in  new issue, and we should finally merge and close this PR.

@antonsoroko

Although, I can add such logic for images, if we agree that we will do the same for text

Yes, you are right. If we use it for text, then it can also be used for artwork.
Do you want to open a new issues for the rest?

Do you want to open a new issues for the rest?

i think 1 issue is enough. btw - all those questions (usefulness of original title option, 2 buttons, english text fallback ) are to elgatito/

@antonsoroko
I just didn't understand. Do you want to do it or do you want me to do it? Let's not both make the same issue. :-)

@matke-84

do you want me to do it?

this. since those are your questions originally.

@antonsoroko @elgatito
We can continue the discussion here.
https://github.com/elgatito/plugin.video.elementum/issues/1009

@matke-84 here is the new build with english as intermediate fallback for images.
https://disk.yandex.by/d/lmCq97yF1jig3g
To find localized image we use: user's language -> second language -> English.
Only for poster it also will have "original language" as the final fallback (since TMDB api returns "original language" poster if there is nothing else).
good video for test - https://www.themoviedb.org/movie/1046090/images/posters

@antonsoroko
I was just about to say that @elgatito mentioned that he would put the original language after English, because the original language is better than nothing (maybe he only thought of the text, plot, tagline...), but now you have clarified everything. :-)
This seems ok to me. A lot of artwork is missing for many languages and we all understand English, more or less. I don't know what @elgatito means?
Now we just need to fix the text https://github.com/elgatito/plugin.video.elementum/issues/1009

@matke-84 text should be fixed in other PR, in order not to mix things.
I think we done with images.

@antonsoroko

text should be fixed in other PR, in order not to mix things.

Yep, that's why I put a link to the new issue. ;-)

i made some cosmetic changes to reduce amount of code. nothing changed in terms of logic.
still ready to merge.

Let's merge this one and fix any follow up issues separately.
Thanks everyone for taking part in this MR.

@elgatito
No, thank you and @antonsoroko for continuing to maintain the best plugin for movies. ;-)


RC Test: wave-interference/2.0.0-rc.1 and waves-intro/1.0.0-rc.3

@ariel-phet @KatieWoe @arouinfar @kathy-phet, wave-interference/2.0.0-rc.1 and waves-intro/1.0.0-rc.3 are ready for RC testing. We would like this to be published before the beginning of school. Document issues in https://github.com/phetsims/wave-interference/issues and link to this issue.
Assigning to @ariel-phet for prioritization.

General RC Test

What to Test

Click every single button.
Test all possible forms of input.

Test all mouse/trackpad inputs.
Test all touchscreen inputs.

If there is sound, make sure it works.
Make sure you can't lose anything.
Play with the sim normally.
Try to break the sim.
Test all query parameters on all platforms. (See QA Book for a list of query parameters.)
Download HTML on Chrome and iOS.
Make sure the iFrame version of the simulation is working as intended on all platforms.
Make sure the XHTML version of the simulation is working as intended on all platforms.
Complete the test matrix.
Don't forget to make sure the sim works with Legends of Learning.
Check this LoL spreadsheet and notify AR or AM if it not there.
This is the first RC (even though one is marked rc.3), so please run a memory test.
When making an issue, check to see if it was in a previously published version
If there is a console available, check for errors and include them in the Problem Description.

Focus and Special Instructions
Please be aware of prior dev testing which completed here: https://github.com/phetsims/QA/issues/389
Issues to make sure credits are correct are https://github.com/phetsims/wave-interference/issues/428 and
https://github.com/phetsims/waves-intro/issues/9
Note that build failures bumped the version number of waves-intro to rc.3 even though it is the first RC to be tested.

Link(s)
Wave Interference

Simulation
iFrame
XHTML
Test Matrix
Legends of Learning Harness

Waves Intro

Simulation
iFrame
XHTML
Test Matrix
Legends of Learning Harness

FAQs for QA Members

There are multiple tests in this issue... Which test should I do first?
Test in order! Test the first thing first, the second thing second, and so on.

How should I format my issue?

We typically assign the developer who opened the issue in the QA repository.

My question isn't in here... What should I do?
You should:

Consult the QA Book.
Google it.
Ask Katie.
Ask a developer.
Google it again.
Cry.

High priority @KatieWoe but after #404

Wave Interference Memory Test Results

Waves Intro memory test:

Large amount of lag on 4th screen in Edge, especially the slider that controls light color. I'm guessing we're not concerned enough for an issue @ariel-phet?

QA is done

All issues have been discussed, deferred or addressed.  I'll continue in https://github.com/phetsims/wave-interference/issues/441 to create the next RC.


Adding contact info to jwmdev node address

@jwmdev please send a PR and add your contact info as commentin config.

@jwmdev, if you can't share your contact information, we have no choice but to remove your node information as a bootstrap node.

@b00f kindly remove the node info as a bootstrap node.

@kehiy FYI


[Rhodonite Editor] Wrong material rendered for VRM model

Describe the bug
Wrong material rendered for VRM model
Expected:

Actual:

How to reproduce

Download xbot.0.x.vrm
Drag & Drop onto https://editor.librn.com/

@infosia Thank you for the issue.
I'm now investigating this.

@infosia Now fixed. Please check again: https://editor.librn.com/

Nice! Yes it's fixed for me too 👍


chore: add doc comments

Description:
Add more doc comment for variables.
Related Issues:
N/A
Checklist:

[x] Review the CONTRIBUTING.md documentation.
[x] Add a reference to a related issue in the repository.
[x] Add a description of the changes proposed in the pull request.
[x] Add unit tests if applicable.
[x] Update documentation if applicable.
[x] Add a note in the CHANGELOG.md if applicable.
[x] Sign the Google CLA.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 43.20%. Comparing base (ded3a72) to head (9071a41).
Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main      #26   +/-   ##
=======================================
  Coverage   43.20%   43.20%
=======================================
  Files           7        7
  Lines         810      810
=======================================
  Hits          350      350
  Misses        404      404
  Partials       56       56

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Helm install/upgrade fails when setting router deployment annotation in values.yaml

Client Version: version.Info{Major:"1", Minor:"5", GitVersion:"v1.5.1", GitCommit:"82450d03cb057bab0950214ef122b67c83fb11df", GitTreeState:"clean", BuildDate:"2016-12-14T00:57:05Z", GoVersion:"go1.7.4", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"5", GitVersion:"v1.5.1", GitCommit:"82450d03cb057bab0950214ef122b67c83fb11df", GitTreeState:"clean", BuildDate:"2016-12-14T00:52:01Z", GoVersion:"go1.7.4", Compiler:"gc", Platform:"linux/amd64"}

Client: &version.Version{SemVer:"v2.1.3", GitCommit:"5cbc48fb305ca4bf68c26eb8d2a7eb363227e973", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.1.3", GitCommit:"5cbc48fb305ca4bf68c26eb8d2a7eb363227e973", GitTreeState:"clean"}

# inside the values.yaml
  deployment_annotations:
    router.deis.io/nginx.defaultWhitelist: "<IP_ADDRESS>, <IP_ADDRESS>"

Executing the following command:
$ helm upgrade deis-workflow-2 deis/workflow -f values.yaml
Error: UPGRADE FAILED: unable to decode "": [pos 236]: json: expect char '"' but got char 't'

Removing the annotation in the values.yaml allows the upgrade to pass. Am I perhaps doing something wrong?

can you provide your full values.yaml with sensitive information redacted? Seems similar to https://github.com/deis/builder/issues/460.

ping, is this still relevant now that deis/builder#460 has been resolved?

This is still relevant. I will provide an example configuration file.

Are you sure error is relevant to annotations?
Check position 236 in values.yaml.
I always have this error in kubectl apply -f ... when my manifest contains something like:
some_key: true

Changing true to "true" (with explicit quotes) works like a charm in this cases.

Problem persists. I just upgraded another DeisWF installation from 2.9.0 tot 2.10.0. This time a Kubernetes 1.4.8 cluster. Below are the values with REDACTED configuration. The deployment only succeeds if I comment out the deployment_annotations. Quoting/unquoting key/values does not help. I have tried removing existing annotations from the deploy/deis-router, that didn't help. I tried upgrading with only a single annotation in the values.yaml, but no matter how many I enable/disable the upgrade will fail. Fortunately, upgrading without annotations does not clear existing annotations on the router deployment object.
---
# This is the global configuration file for Workflow

global:
  # Set the storage backend
  #
  # Valid values are:
  # - s3: Store persistent data in AWS S3 (configure in S3 section)
  # - azure: Store persistent data in Azure's object storage
  # - gcs: Store persistent data in Google Cloud Storage
  # - minio: Store persistent data on in-cluster Minio server
  storage: gcs

  # Set the location of Workflow's PostgreSQL database
  #
  # Valid values are:
  # - on-cluster: Run PostgreSQL within the Kubernetes cluster (credentials are generated
  #   automatically; backups are sent to object storage
  #   configured above)
  # - off-cluster: Run PostgreSQL outside the Kubernetes cluster (configure in database section)
  database_location: "off-cluster"

  # Set the location of Workflow's logger-specific Redis instance
  #
  # Valid values are:
  # - on-cluster: Run Redis within the Kubernetes cluster
  # - off-cluster: Run Redis outside the Kubernetes cluster (configure in loggerRedis section)
  logger_redis_location: "on-cluster"

  # Set the location of Workflow's influxdb cluster
  #
  # Valid values are:
  # - on-cluster: Run Influxdb within the Kubernetes cluster
  # - off-cluster: Influxdb is running outside of the cluster and credentials and connection information will be provided.
  influxdb_location: "on-cluster"
  # Set the location of Workflow's grafana instance
  #
  # Valid values are:
  # - on-cluster: Run Grafana within the Kubernetes cluster
  # - off-cluster: Grafana is running outside of the cluster
  grafana_location: "on-cluster"

  # Set the location of Workflow's Registry
  #
  # Valid values are:
  # - on-cluster: Run registry within the Kubernetes cluster
  # - off-cluster: Use registry outside the Kubernetes cluster (example: dockerhub,quay.io,self-hosted)
  # - ecr: Use Amazon's ECR
  # - gcr: Use Google's GCR
  registry_location: "gcr"
  # The host port to which registry proxy binds to
  host_port: 5555
  # Prefix for the imagepull secret created when using private registry
  secret_prefix: "private-registry"

s3:
  # Your AWS access key. Leave it empty if you want to use IAM credentials.
  accesskey: ""
  # Your AWS secret key. Leave it empty if you want to use IAM credentials.
  secretkey: ""
  # Any S3 region
  region: "us-west-1"
  # Your buckets.
  registry_bucket: "your-registry-bucket-name"
  database_bucket: "your-database-bucket-name"
  builder_bucket: "your-builder-bucket-name"

azure:
  accountname: "YOUR ACCOUNT NAME"
  accountkey: "YOUR ACCOUNT KEY"
  registry_container: "your-registry-container-name"
  database_container: "your-database-container-name"
  builder_container: "your-builder-container-name"

gcs:
  # key_json is expanded into a JSON file on the remote server. It must be
  # well-formatted JSON data.
  key_json: "REDACTED"
  registry_bucket: "REDACTED"
  database_bucket: "REDACTED"
  builder_bucket: "REDACTED"

swift:
  username: "Your OpenStack Swift Username"
  password: "Your OpenStack Swift Password"
  authurl: "Swift auth URL for obtaining an auth token"
  # Your OpenStack tenant name if you are using auth version 2 or 3.
  tenant: ""
  authversion: "Your OpenStack swift auth version"
  registry_container: "your-registry-container-name"
  database_container: "your-database-container-name"
  builder_container: "your-builder-container-name"

# Set the default (global) way of how Application (your own) images are
# pulled from within the Controller.
# This can be configured per Application as well in the Controller.
#
# This affects pull apps and git push (slugrunner images) apps
#
# Values values are:
# - Always
# - IfNotPresent
controller:
  app_pull_policy: "IfNotPresent"
  # Possible values are:
  # enabled - allows for open registration
  # disabled - turns off open registration
  # admin_only - allows for registration by an admin only.
  registration_mode: "admin_only"

database:
  # The username and password to be used by the on-cluster database.
  # If left empty they will be generated using randAlphaNum
  username: ""
  password: ""
  # Configure the following ONLY if using an off-cluster PostgreSQL database
  postgres:
    name: "REDACTED"
    username: "REDACTED"
    password: "REDACTED"
    host: "REDACTED"
    port: "5432"

redis:
  # Configure the following ONLY if using an off-cluster Redis instance for logger
  db: "0"
  host: "redis host"
  port: "redis port"
  password: "redis password" # "" == no password

fluentd:
  syslog:
    # Configure the following ONLY if using Fluentd to send log messages to both
    # the Logger component and external syslog endpoint
    # external syslog endpoint url
    host: ""
    # external syslog endpoint port
    port: ""

monitor:
  grafana:
    user: "admin"
    password: "admin"
  # Configure the following ONLY if using an off-cluster Influx database
  influxdb:
    url: "my.influx.url"
    database: "kubernetes"
    user: "user"
    password: "password"

registry-token-refresher:
  # Time in minutes after which the token should be refreshed.
  # Leave it empty to use the default provider time.
  token_refresh_time: ""
  off_cluster_registry:
    hostname: ""
    organization: ""
    username: ""
    password: ""
  ecr:
    # Your AWS access key. Leave it empty if you want to use IAM credentials.
    accesskey: ""
    # Your AWS secret key. Leave it empty if you want to use IAM credentials.
    secretkey: ""
    # Any S3 region
    region: "us-west-2"
    registryid: ""
    hostname: ""
  gcr:
    key_json: "REDACTED"
    hostname: "eu.gcr.io"

router:
  dhparam: ""
  # Any custom router annotations(https://github.com/deis/router#annotations)
  # which need to be applied can be specified as key-value pairs under "deployment_annotations"
  deployment_annotations:
    router.deis.io/nginx.defaultWhitelist: "REDACTED"
    router.deis.io/nginx.enforceWhitelists: "true"
    router.deis.io/nginx.platformDomain: "REDACTED"
    router.deis.io/nginx.proxyRealIpCidrs: "REDACTED"
    router.deis.io/nginx.serverNameHashBucketSize: "128"
    router.deis.io/nginx.whitelistMode: "extend"

  # Any custom annotations for k8s services like http://kubernetes.io/docs/user-guide/services/#ssl-support-on-aws
  # which need to be applied can be specified as key-value pairs under "service_annotations"
  service_annotations:
    #<example-key>: <example-value>

  # Enable to pin router pod hostPort when using minikube or vagrant
  host_port:
    enabled: true

  # Service type default to LoadBalancer
  service_type: NodePort

workflow-manager:
  versions_api_url: https://versions.deis.com
  doctor_api_url: https://doctor.deis.com

@mhulscher
# Enable to pin router pod hostPort when using minikube or vagrant
host_port:
  enabled: true

Unquoted true

Booleans are a type in YAML. Also quoting the "true" causes the deis-upgrade job to fail with a ErrImagePullBackoff

I mean something like this:
    apiVersion: extensions/v1beta1
    kind: Deployment
    metadata:
      annotations:
        deployment.kubernetes.io/revision: "15"
        prometheus.io/scrape: "true"

Unquoted true here will fire an error with kubectl apply

So in fact helm upgrade just generates proper manifest from templates and applies them.
You can see templates with helm fetch deis/workflow --untar. But main problem is we don't know which manifest precisely is failing to apply...

helm lint shows the error with values.yaml:
><> helm fetch deis/workflow --untar
><> vim workflow/values.yaml
><> helm lint workflow/
2017/01/16 15:44:33 warning: skipped value for deployment_annotations: Not a table.
2017/01/16 15:44:33 warning: skipped value for deployment_annotations: Not a table.

This is an actually interesting issue. It turns out that the downstream router chart's values.yaml mangles this:
><> cat workflow/charts/router/values.yaml
org: "deis"
pull_policy: "IfNotPresent"
docker_tag: v2.8.1
platform_domain: ""
dhparam: ""
# limits_cpu: "100m"
# limits_memory: "50Mi"

# Any custom router annotations(https://github.com/deis/router#annotations)
# which need to be applied can be specified as key-value pairs under "deployment_annotations"
deployment_annotations:
  #<example-key>: <example-value>

# Any custom annotations for k8s services like http://kubernetes.io/docs/user-guide/services/#ssl-support-on-aws
# which need to be applied can be specified as key-value pairs under "service_annotations"
service_annotations:
  #<example-key>: <example-value>

# Enable to pin router pod hostPort when using minikube or vagrant
host_port:
  enabled: true

# Service type default to LoadBalancer
# service_type: LoadBalancer

If I remove the deployment_annotations (and service_annotations) field completely from that values.yaml, helm lint passes. Can you try that and try re-deploy? I'm on a plane at the moment so I don't have access to a kube cluster handy to test.
ping @technosophos, do you know if this is a known helm bug or something we should file a bug and a fix for?

Removing deployment_annotations and service_annotations still allows for deployment. Also doesn't strip existing annotations from the deployment object.


Enable FxCop Analyzers

Building on #7467, we should enable FxCop analyers: https://docs.microsoft.com/visualstudio/code-quality/install-fxcop-analyzers. Not only is this recommended by TwC but it allows us to catch things like catch (Exception ex) { /* do stuff */ throw ex; } (CA2200).
To note, some of this can be enabled via our existing .editorconfig: https://docs.microsoft.com/dotnet/fundamentals/code-analysis/code-quality-rule-options

We might start with .NET's https://github.com/dotnet/runtime/blob/master/eng/CodeAnalysis.ruleset since we started with their .editorconfig as well (with 1 small tweak I don't remember).

Update the package to the latest version.


Remove mappings in page cache lazily

Problem
Right now, there are mapping removals for the page cache in the Drop destructors in two places:

PageWriteGuard for pages with valid == false that haven't been written to yet
EphemeralFile for all pages associated with the file being dropped

Both these mapping removals lock slots in the page cache and then free up page slots related to the datastructures that are being dropped. In #5023, we want to make these slot locks async, and as we can't make the Drop destructors async themselves, we need to find alternatives.
One possibility would be just not do any removals, which would still be perfectly correct, but the pages would now be "leaked" and eviction code would treat them the same as currently used pages. This would obviously be quite unfortunate.
That's why we now instead of eagerly removing the mappings, we mark the pages for removal. Later we remove the mappings, once we are in the need for new pages in the find_victim function.
Summary of changes
There are a few smaller refactors:

use Vec::leak instead of Vec::into_boxed_slice plus Box::leak
introduce a set_usage_count function instead of manually calling usage_count.store
improve comments for the valid field, they were quite terse before.

The main changes are:

introduce a slot_idx field of SlotInner which is used to access the corresponding Slot that has the usage_count field (direct pointers to the parent are hard in Rust).
reset the usage_count field in the PageWriteGuard destructor. The find_victim function will then treat this page as having zero users (as it should) and remove its mappings.
as we don't know the particular slot indices associated with the ephemeral file, we use a queue to remove their mappings from the find_victim function.

See also discussion in https://github.com/neondatabase/neon/pull/5023#discussion_r1304194495 which outlines the alternatives and design constraints, especially why we chose locks from crossbeam_channel instead of locks from std or tokio.
Note that metrics wise, the delayed execution of the remove_mappings function is not without effect: as the remove_mappings function updates the metrics, and it is executed now in a delayed fashion, metrics will show some pages as used even though they are basically unused and will be freed at the next time there is need for new pages.
Part of #4743.
Checklist before requesting a review

[x] I have performed a self-review of my code.
[ ] If it is a core feature, I have added thorough tests.
[ ] Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
[ ] If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

[ ] Do not forget to reformat commit message to not include the above checklist

My PR #5119 eliminates

PageWriteGuard for pages with valid == false that haven't been written to yet

Now looking into the EphemeralFile case.

closing in favour of #5122 (we can always reopen if that approach doesn't work).


urlopen error [SSL: CERTIFICATE_VERIFY_FAILED]

I am getting an error because of some failed validation when I follow the README
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1056)

This happends when I do  python3 -m medium_to_ghost.medium_to_ghost medium-export.zipon MacOS 10.14.4 and using Python 3.7.3

Running this command seems to work:
sudo /Applications/Python\ 3.7/Install\ Certificates.command
Make sure to specify the correct version of Python installed on your machine. This command also assumes you're using MacOS.

The response from @jehunter5811 is correct. The problem is that Python on Mac doesn't install SSL certificates by default for some reason and you have to run that command manually to install them. Otherwise no python programs can access https sites.


Warning about manual command line usage

R: @owencm
Closes #76

LGTM. Pending review from Owen before merge.

LGTM2. Thanks!


Example of working code implementation with NextJS

Hi, I just tried to make the code snippet works with NextJS.
I have as error document not find from nodetick :
error - ReferenceError: document is not defined     at de (/Users/raf/Documents/project/node_modules/@nodetoy/react-nodetoy/dist/react-nodetoy.umd.js:1:1490)
I then implemented the shader as a standalone component and import it using next/dynamic with SSR off:
Shader.tsx
import { NodeToyMaterial, NodeToyTick } from '@nodetoy/react-nodetoy';
import { data } from 'utils/shaderData';

export default function Shader() {
  return (
    <>
      <mesh>
        <boxBufferGeometry attach="geometry" args={[1, 1, 1]} />
        <NodeToyMaterial data={data} />
      </mesh>
      <NodeToyTick />
    </>
  );
}

App.tsx
import { OrbitControls } from '@react-three/drei';
import { Canvas } from '@react-three/fiber'
import dynamic from 'next/dynamic';

export const App = () => {
  const Shader = dynamic(() => import('./Shader'), { ssr: false });
  return (
    <div style={{ width: '100vw', height: '100vh' }}>
      <Canvas camera={{ fov: 75, near: 0.1, far: 1000, position: [0, 0, 2] }}>
        <ambientLight intensity={0.5} />
        <directionalLight position={[-10, -10, -5]} intensity={0.5} />
        <Shader />
        <OrbitControls />
      </Canvas>
    </div>
  );
};

This bypass the document error but trigger Error: R3F: Hooks can only be used within the Canvas component!.
Any idea on how to solve that? A nextjs implementation would be helpful, thanks 🙏

adding if (typeof window === 'undefined') return <></>; before the return component trigger a new issue Error: Hydration failed because the initial UI does not match what was rendered on the server.

Did you manage to solve this @foufrix ? Have the same issue, it also clobbers fast-refresh


describe.only runs multiple unrelated tests

When multiple unit tests have same name in describe, mocha ignores the .only setting.
Steps to reproduce:

Create test1.js and test2.js as per template below
Run mocha test1.js test2.js -R spec
Expected result

You should see only the test1 to get run

Actual result

Both test1 and test2 get run

test1.js
describe.only('Test', function () {
        it('Should run test1 only', function() {});
});

test2.js
describe('Test', function () {
        it('Should run test2', function() {});
});

This substring matching is a pain in the rear for us as well. When we mark describe as .only we'd expect that only that particular describe block will run. I understand maybe running another describe if the string matches 100%, but substring matches should be excluded IMHO.

+1, I think this issue is sabotaging my tests right now as well.

+1 this is really needs fixing. I would consider this a bug fix, the documented behaviour is:

The exclusivity feature allows you to run only the specified suite or test-case by appending .only() to the function.

I don't think prioritising upsetting someone relying on a bug over everyone else expecting documented behaviour is a good reason to not fix it.

if @mochajs/mocha can agree it's a bug, then it could go into a minor

@boneskull rather than discuss that (although I could be convinced to go put it in a minor [that sounds worrying]), how about we change #1782 to become a release of v3.0.0 in stead, merge #1632 and #1481 asap, lift the rest of milestone V3.0.0 to the next major release and ship v3.0.0? :)

This has bitten me multiple times and kind of stinks. Is there a better way we could do preprocessing of all .only() and build out a tree? The desired behavior is such that I can mark a 'top level' suite with only, then as I work through sub-level describes I can attach another only without having to remove the top-level marker.

This issue is causing problems for me because I want to describe my tests in a hierarchical structure like below, but then it is not possible to run one suite exclusively.
describe('components/Form', function() { ... })
describe('components/FormField', function() { ... })
describe('components/FormControls', function() { ... })

@PerlPong Why not restructure the specs to be something like this:
context('components', function () {
  describe('Form', function () {...});
  describe('FormField', function () {...});
  describe('FormControls', function () {...});
});

@demisx thanks for the tip, but unfortunately it does not work around this bug. If you execute the following with mocha, both suites will run:
var expect = require('chai').expect;

context('components', function() {
    describe.only('Form', function() {
        it('should be true', function () {
            expect(true).to.be.true;
        });
    });
    describe('FormField', function() {
        it('should be false', function () {
            expect(false).to.be.false;
        });
    });
});

  components
    Form
      ✓ should be true
    FormField
      ✓ should be false

@PerlPong  I see. Yeah, I believe if the spec describe string matches or starts with the string that you set .only on, it will run that test too. Ran into this inconvenience in the past myself and had to ensure my describes always start with a unique string.

Closing in favor of #1481. PR #1807 (referencing that other issues) is in the works .


"Important" is out of date

The [!IMPORTANT] part is out of date. The currently up-to-date information can be found here under important: https://docs.microsoft.com/en-us/xamarin/ios/deploy-test/app-distribution/app-store-distribution/index

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 383fce08-fe66-8a22-980f-18751b1d028f
Version Independent ID: 7a96f3a8-272a-4f0d-352f-efbb62de375b
Content: Xamarin.iOS App Distribution Overview - Xamarin
Content Source: docs/ios/deploy-test/app-distribution/index.md
Product: xamarin
Technology: xamarin-ios
GitHub Login: @lobrien
Microsoft Alias: laobri

Thanks for letting us know, and suggesting the fix! Updating the doc now.


session file save doesn't take session file names in directory into account

visit 3.4.2 beta

Why can't it look at the session file names that are already there? I think we do that for the x ray image query outputs.

to clarify, this is what I see when saving session files after opening visit in a directory that already contains session files with the default names.


Web: Searching for Character.Say gets no results

All of the struct methods in the titles, like System.Log, Character.Say, if you search any of these, they get no results. Looking the JS index, only the methods (Log, Say) are indexed, and also the structs, but there's no additional indexing of the combination.

I think the search data is currently split on words.
In the short term the best thing to do is probably to make additional words filter the results from the just word. So "System Log" would show pages that contain "System" and "Log", instead of what it does now, showing all pages which contain "System" or "Log". To be honest I'm not sure why I didn't do it that way to start with as additional filtering is probably much more useful than matching additional pages.
But the proper fix is a better index. Some decisions for the index data were to try to keep the CHM viewer usable (e.g. give scrollbar motion on the index that is actually usable).

But the proper fix is a better search index. The CHM index is generated automatically by HHC or chmcmd but the content of those varies between the two. I think HHC must be splitting words into partial words for better searching and its search index is much bigger.

Might that be a good long term goal to replace CHM with a html pages on disk instead? I think a lot of software do that today. We could have similar output for online and offline help, and users will need a browser to read offline one.
Of course I would suggest a html page redesign, because current one has limited functionality at the moment (in terms of index and table of contents).

Maybe it is the sort of thing that could be integrated into the development tools, i.e. the development tools include provide everything for editor support (LSP server, documentation server, documentation content). Then an (any) editor can request help content from the tools, and either display the content that is returned or ask the user's regular browser to open a URL, which then views the documentation page, served by a web server built into the tools.
Having the option of the data going directly back to an editor would allow the possibility that the editor could re-implement whatever type of window focus rules best suit the current editor state - I think that is probably the main argument in favour of keeping the CHM viewer, the behaviour of its window can be altered so that it co-exists nicely with windows from the main application.
Just having HTML pages on disk and nothing else is probably going to be more difficult to search.

In the short term the best thing to do is probably to make additional words filter the results from the just word. So "System Log" would show pages that contain "System" and "Log"

This is good enough, do you know if it's possible to do this using the current index? So System.Log would become "System AND Log" logic wise in the query.

Yes, it's possible. We just have to pretend the "." is an empty space.
I wanted to make a PR with some quality of life, I'll just add this change too.

It looks like when the index data is generated it does preserve a - in the middle of the word, so it might be worth trying to do the same test and adjustment of the match pattern with a ..
https://github.com/adventuregamestudio/ags-manual/blob/7083ef9d4673f141c686c52c0c433b00746083b6/lua/write_metablock.lua#L88-L100

Alright, then I'll skip turning periods into spaces.
In the meanwhile I wrapped up my proposed changes in PR #202
Not sure yet if it compiles fully. On my rig I set up only the bare minimum to produce htmls directly with pandoc without the makefile, many things don't work obviously. I should set it up on a VM with linux to try it properly.

I think this would be enough to make it work without causing too many issues:
diff --git a/lua/write_metablock.lua b/lua/write_metablock.lua
index 8bdfd90c..505fa900 100644
--- a/lua/write_metablock.lua
+++ b/lua/write_metablock.lua
@@ -90,6 +90,11 @@ function add_word(word)
     chars = chars .. '-'
   end

+   -- preserve . in the middle of the word
+  if first ~= '.' and last ~= '.' then
+    chars = chars .. '%.'
+  end
+
   -- split on non-word characters
   for w in word:gmatch('[' .. chars .. ']+') do
     if w:len() > 1 and not w:match('%d') then

...this is effectively reclassifying "." as part of a word. The will be some side effects for where existing entries get extended to include the ".", but in terms of searching for the scripting language syntax I think it should be OK.

I just realized we had never applied this patch. It's been applied, and I've also added underscore and escaped %-.
In theory that should cover all needs for keywords searching.


[Meta] textlint 12.0.0

<EMAIL_ADDRESS>collection
Breaking Changes

[x] All modules use the same version https://github.com/textlint/textlint/issues/676

In other words, the next textlint and @textlint/* will be v12.0.0

[x] textlint-tester export as default #689
[x] Drop Node.js 10 support https://github.com/textlint/textlint/issues/600
[x] Update remark #717
[x] update textlitnt-scripts@4

@textlint/markdown-to-ast
It is a markdown parser in textlint.
We have updated to<EMAIL_ADDRESS>#717 and it has many changes for Markdown AST
You can see the changes at feat(markdown-to-ast): update to remark-parse@9 #767
Features
Add FootnoteReference node
The NATO phonetic alphabet[^wiki].

[^wiki]: Read more about it on wikipedia: <http://en.wikipedia.org/wiki/NATO_phonetic_alphabet>.

Previously, It is called LinkReference, textlint@12 treat it as FootnoteReference.
Known bugs

[x] gfm parsing bug that generate broken AST: https://github.com/remarkjs/remark-gfm/issues/16

→ use workaround:  https://github.com/remarkjs/remark-gfm/issues/16#issuecomment-846357030 at https://github.com/textlint/textlint/commit/c99218ea37518e8a42bb5d283276ccf696618748
→ markdown-to-ast does not generate broken AST.

12.0.0-beta.0 #778
npm install textlint@beta

12.0.0-beta.1 #780
textlint user
npm install textlint@beta

Rule creator
npm install textlint-script@beta

I found an error on textlint-plugin-asciidoctor.

https://github.com/seikichi/textlint-plugin-asciidoctor/issues/18

However, it is not related 12.0.0 directory.
It is affected by #649 and we mitigate the effects in #783

v12.0.0-beta.2 #785
textlint user
npm install textlint@beta

textlint rule creator
npm install textlint-scripts@beta

Next will be stable releases.

#786

After release

[ ] update template repo

https://textlint.github.io/blog/2021/05/24/textlint-12.html Released 🎉

https://textlint.github.io/astexplorer/#/snippet/woXCqHBhcnNlcklEwrh0ZXh0bGludDptxINrZG93bi10by1hc3TCqMSFdHTEkGdzwoHEisSMxI7EkMSSxJRyxJbEmMSaxJzEnsSgw4DCqHbEhnNpb27EqMSqxI3Ej8SRxJPElcSXxJnEm8SdxJ90wqYxMi4wxY_CqGZpbGVuYW1lwrBzb3VyY2UudW5kZcWTbmVkwqRjb8Wlw5lNIyBUacSOZQoKVGhpcyDFuyDEi8SNLsW3LSDEj8Sgxb3Ei23Ft1tMxJBrXShoxKRwOi8vxIzFmHDFlS7FrG0pxoMgU2HEuWTFv8Sr}
Update AST Explorer and support "Save"

Complete!

→ textlint v12.0.0 released! · Discussion #789 · textlint/textlint


[zh]translate indexed-parallel-processing-static.md

relate : https://github.com/kubernetes/website/issues/29329
part 14

作业 -> Job

/approve

Thank you for your translation, it is very complete, and provide you some suggestions for code review.

after modification, I've searched '作业' in the entire doc, turns out only two: one in doc title another in paragraph title

/lgtm


pnpm db:seed is giving me a duplicate key value error of users_email_unique.

pnpm db:seed is giving me a duplicate key value error of users_email_unique. Any suggestions on what to change in the functions?
Seed process failed: PostgresError: duplicate key value violates unique constraint "users_email_unique"     at ErrorResponse (/Users/thomasjackson/Desktop/venginez-templates/next-saas-starter/node_modules/.pnpm/postgres@3.4.4/node_modules/postgres/cjs/src/connection.js:788:26)     at handle (/Users/thomasjackson/Desktop/venginez-templates/next-saas-starter/node_modules/.pnpm/postgres@3.4.4/node_modules/postgres/cjs/src/connection.js:474:6)     at TLSSocket.data (/Users/thomasjackson/Desktop/venginez-templates/next-saas-starter/node_modules/.pnpm/postgres@3.4.4/node_modules/postgres/cjs/src/connection.js:315:9)     at TLSSocket.emit (node:events:518:28)     at addChunk (node:internal/streams/readable:559:12)     at readableAddChunkPushByteMode (node:internal/streams/readable:510:3)     at Readable.push (node:internal/streams/readable:390:5)     at TLSWrap.onStreamRead (node:internal/stream_base_commons:190:23) {   severity_local: 'ERROR',   severity: 'ERROR',   code: '23505',   detail: 'Key<EMAIL_ADDRESS>already exists.',   schema_name: 'public',   table_name: 'users',   constraint_name: 'users_email_unique',   file: 'nbtinsert.c',   line: '666',   routine: '_bt_check_unique'

Have you already ran the script? You could use db:studio to view the contents and then delete that user.

ill try db studio

I just checked and theres one user of<EMAIL_ADDRESS>email. so I don't see why there's an issue with it. i am afraid to delete it because I want to use that user for testing in general.
In an attempt to delete it too I am getting this error
'update or delete on table "users" violates foreign key constraint "activity_logs_user_id_users_id_fk" on table "activity_logs"'
and its asking me to report it drizzle studios github.

is it possible theres a cookies related issues with this? can I check my database by the terminal to get more accurate info? any help would be great


Button QR-Codes umbenennen

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.

Etiketten als Begriff


typo in env var name reference

https://github.com/aws-samples/aws-iot-core-https-x509certificate-python-node/blob/6d7baf8d24bd6b190c05953f7963ae3db407d4e8/python samples/devicesimulatorhttpx509.py#L15

This has been fixed.