Negative format filter not working with Twitch

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

[X] I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

[X] I'm reporting that yt-dlp is broken on a supported site
[X] I've verified that I have updated yt-dlp to nightly or master (update instructions)
[X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
[X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[X] I've searched known issues and the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue
[X] I've read about sharing account credentials and I'm willing to share it if required

Region
United States
Provide a description that is worded well enough to be understood
When using negative filters such as vcodec!=av01 to download Twitch videos, yt-dlp reports Requested format is not available. Use --list-formats for a list of available formats. However, since this is only excluding a single codec, all others should still be considered valid choices.
Provide verbose output that clearly demonstrates the problem

[X] Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
[X] If using API, add 'verbose': True to YoutubeDL params instead
[X] Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output
$ yt-dlp -vU -f 'bestvideo[vcodec!=av01]+bestaudio' \
> --retries 'infinite' \
> --fragment-retries 'infinite' \
> --concurrent-fragments 8 \
> -o '%(upload_date>%Y%m%d)s - %(title)s.%(ext)s' https://www.twitch.tv/videos/2005982850?filter=archives
[debug] Command-line config: ['-vU', '-f', 'bestvideo[vcodec!=av01]+bestaudio', '--retries', 'infinite', '--fragment-retries', 'infinite', '--concurrent-fragments', '8', '-o', '%(upload_date>%Y%m%d)s - %(title)s.%(ext)s', 'https://www.twitch.tv/videos/2005982850?filter=archives']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version<EMAIL_ADDRESS>from yt-dlp/yt-dlp [24f827875] (pip)
[debug] Python 3.8.10 (CPython x86_64 64bit) - Linux-5.15.0-91-generic-x86_64-with-glibc2.29 (OpenSSL 1.1.1f  31 Mar 2020, glibc 2.31)
[debug] exe versions: ffmpeg 4.2.7, ffprobe 4.2.7
[debug] Optional libraries: Cryptodome-3.18.0, brotli-1.0.9, certifi-2023.05.07, mutagen-1.46.0, requests-2.31.0, secretstorage-2.3.1, sqlite3-3.31.1, urllib3-2.1.0, websockets-11.0.3
[debug] Proxy map: {}
[debug] Request Handlers: urllib, requests
[debug] Loaded 1901 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version<EMAIL_ADDRESS>from yt-dlp/yt-dlp
yt-dlp is up to date<EMAIL_ADDRESS>from yt-dlp/yt-dlp)
[twitch:vod] Extracting URL: https://www.twitch.tv/videos/2005982850?filter=archives
[twitch:vod]<PHONE_NUMBER>: Downloading stream metadata GraphQL
[twitch:vod]<PHONE_NUMBER>: Downloading video access token GraphQL
[twitch:vod]<PHONE_NUMBER>: Downloading m3u8 information
[twitch:vod]<PHONE_NUMBER>: Downloading storyboard metadata JSON
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, size, br, asr, proto, vext, aext, hasaud, source, id
ERROR: [twitch:vod] v2005982850: Requested format is not available. Use --list-formats for a list of available formats
Traceback (most recent call last):
  File "/home/USERNAME/.local/lib/python3.8/site-packages/yt_dlp/YoutubeDL.py", line 1567, in wrapper
    return func(self, *args, **kwargs)
  File "/home/USERNAME/.local/lib/python3.8/site-packages/yt_dlp/YoutubeDL.py", line 1723, in __extract_info
    return self.process_ie_result(ie_result, download, extra_info)
  File "/home/USERNAME/.local/lib/python3.8/site-packages/yt_dlp/YoutubeDL.py", line 1782, in process_ie_result
    ie_result = self.process_video_result(ie_result, download=download)
  File "/home/USERNAME/.local/lib/python3.8/site-packages/yt_dlp/YoutubeDL.py", line 2884, in process_video_result
    raise ExtractorError(
yt_dlp.utils.ExtractorError: [twitch:vod] v2005982850: Requested format is not available. Use --list-formats for a list of available formats

[info] Available formats for v2005982850:
ID         EXT   RESOLUTION FPS │   FILESIZE   TBR PROTO │ VCODEC      ACODEC     ABR MORE INFO
────────────────────────────────────────────────────────────────────────────────────────────────
sb1        mhtml 110x62       0 │                  mhtml │ images                     storyboard
sb0        mhtml 220x124      0 │                  mhtml │ images                     storyboard
Audio_Only mp4   audio only     │ ~474.18MiB  218k m3u8  │ audio only  mp4a.40.2 218k
160p30     mp4   284x160        │ ~631.02MiB  290k m3u8  │ avc1.4D000C mp4a.40.2
360p30     mp4   640x360        │ ~  1.57GiB  738k m3u8  │ avc1.4D001E mp4a.40.2
480p30     mp4   852x480        │ ~  3.13GiB 1471k m3u8  │ avc1.4D001F mp4a.40.2
720p60     mp4   1280x720       │ ~  7.31GiB 3440k m3u8  │ avc1.4D0020 mp4a.40.2
Source     mp4   1600x900       │ ~  9.27GiB 4363k m3u8  │ avc1.64002A mp4a.40.2

twitch doesn't serve video-only formats
change -f 'bestvideo[vcodec!=av01]+bestaudio' to -f 'bestvideo[vcodec!=av01]+bestaudio/best[vcodec!=av01]'

Thanks for the info, I wasn't aware you could construct the format filter in such a way. It's working for me.


saveThumb ("same movie location")

Hey there,
can u make pls the change that saveThumb (if i click it on the grid) uses the "same movie location" too if it's checked?
i tried for myself editing line 151 in ThumbGrid.js to use a spezific "saveToFolder" location but it does sadly not work -.-
greetz

Hi b1n4ryj4n
This should resolve the issue. I was working on something related so it was an easy fix.
Cheers
Jakob

Ty for the quick fix, i could not way to use this, so i compiled it myself and it works fine.
Awesome App!
Saves me a lot of time to create thumbnails.


Shareplay support

Possibly add support for shareplay in Asobi. Have to test this out beforehand.

Testing indicates that sharing your screen broadcasts audio and video just fine, there's no need for shareplay.


Quests journal multiply

3.2.4-b255
No console errors, bug occurs quite often, like 2 from 10 players time to time randomly. They run /quests journal command and when they run it again, Instead of putting journal away it creates new one completely empty. And as you cant throw it away its messy. Even i cant remove it from players inventory, except of running /clear . Any ideas?

This typically happens whenever there is either 1) a server/plugin reload, or 2) the Quests Journal's title or lore is modified in some way that the plugin cannot recognize. The former is most likely. What is preventing your players from throwing the old Journal away?

Not even OP can throw that journal away, and there was actualy no reload of any sorts, and that was the issue .

I've just added the permission "quests.admin.drop" to Quests. Any player that has this permission (including OPs) will be able to drop their Quests Journal.
Once available, use the following dev build to make use of this feature:
http://ci.ac3-servers.eu/job/Quests/283/

By the way, the above build of Quests will likely break the MythicMobsQuestModule. If need be, use this custom build to restore functionality:
https://www.dropbox.com/s/2pq695oqodtpgb1/MythicMobsQuestModule_custom.jar?dl=0

Thank you very much  @FlyingPikachu  !


Objc interop

This PR includes the changes made by @compnerd in https://github.com/apple/swift/pull/12657 to support objc interop, in addition to a patch to allow objc interop testing on non-darwin platforms.

@swift-ci Please smoke test

@swift-ci please smoke test

@swift-ci please smoke test

@swift-ci please smoke test

@swift-ci please smoke test

@swift-ci please smoke test

@swift-ci please smoke test


[stable/airflow] sql_alchemy_conn value incorrect for Airflow CLI commands

Describe the bug
When running Airflow CLI commands such as the airflow create_user command it beings up an error saying cannot use sqlite with the CeleryExecutor even when I am using the default "postgresql: enabled: true" and default "executor: CeleryExecutor".
Version of Helm and Kubernetes:
Helm: 3.2.1
Kubernetes: 1.15.7
Which chart:
stable/airflow version 7.0.1
What happened:
I have "AIRFLOW__WEBSERVER__RBAC: True" which requires me to create a user before I can log in. When accessing the shell of the "airflow-web" container and attempting to create a user via the airflow create_user cli command it brings up an error
airflow.exceptions.AirflowConfigException: error: cannot use sqlite with the CeleryExecutor

I have "postgresql" as "enabled: true" so it should be using postgresql not sqlite. This happened with any other cli command such as "airflow list_dags" as well.
After trial and error it seems that explicitly setting AIRFLOW__CORE__SQL_ALCHEMY_CONN as part of the "airflow: config" values to the default end-result value of postgresql+psycopg2://postgres:airflow@airflow-postgresql:5432/airflow allows for Airflow CLI commands such as airflow create_user to work, but this shouldn't be necessary. Airflow CLI commands should work with the default chart values.
What you expected to happen:
After running an Airflow CLI command such as "airflow list_dags" it should not error because AIRFLOW__CORE__SQL_ALCHEMY_CONN should be set as an actual environment variable of the container
How to reproduce it (as minimally and precisely as possible):

Access the shell of the "airflow-web" container in the airflow-web pod:
kubectl exec -t -I AIRFLOW-WEB_POD_NAME -c airflow-web bash
enter the command:
airflow list_dags
The following traceback and error comes up:

Traceback (most recent call last):
  File "/home/airflow/.local/bin/airflow", line 25, in <module>
    from airflow.configuration import conf
  File "/home/airflow/.local/lib/python3.6/site-packages/airflow/__init__.py", line 31, in <module>
    from airflow.utils.log.logging_mixin import LoggingMixin
  File "/home/airflow/.local/lib/python3.6/site-packages/airflow/utils/__init__.py", line 24, in <module>
    from .decorators import apply_defaults as _apply_defaults
  File "/home/airflow/.local/lib/python3.6/site-packages/airflow/utils/decorators.py", line 36, in <module>
    from airflow import settings
  File "/home/airflow/.local/lib/python3.6/site-packages/airflow/settings.py", line 37, in <module>
    from airflow.configuration import conf, AIRFLOW_HOME, WEBSERVER_CONFIG  # NOQA F401
  File "/home/airflow/.local/lib/python3.6/site-packages/airflow/configuration.py", line 622, in <module>
    conf.read(AIRFLOW_CONFIG)
  File "/home/airflow/.local/lib/python3.6/site-packages/airflow/configuration.py", line 348, in read
    self._validate()
  File "/home/airflow/.local/lib/python3.6/site-packages/airflow/configuration.py", line 204, in _validate
    self.get('core', 'executor')))
airflow.exceptions.AirflowConfigException: error: cannot use sqlite with the CeleryExecutor

Anything else we need to know:
It appears the problem is that export AIRFLOW__CORE__SQL_ALCHEMY_CONN= is set as part of the entrypoint "args" in templates/deployments-web.yaml. I think a solution would be to have this set as part of the "env" list but I don't know enough about Helm Charts to make the change myself. The same should go for AIRFLOW__CELERY__RESULT_BACKEND and AIRFLOW__CELERY__BROKER_URL as well.

@alexbegg, I have fixed this in an upcoming release

@alexbegg, this has been fixed in 7.1.0

This is still happening in the 7.1.0 and 7.5.0 releases

Looks like you folks fixed in 7.1.6

I get the same error when running -
kubectl exec -it --namespace tamagotchi-orchestration service/gita-temp1-web -- bash -c "airflow list_dags"
or airflow create_user
I upgraded to 7.1.6 but I get the same error

Yeah it works when I go straight into the shell and properly interprets the CeleryExecutor is in use, but running an exec command interprets airflow using sqlite

the workaround i did is adding to the command - source /home/airflow/airflow_env.sh  so that it will load the correct env vars

This is caused by exec not creating a login bash shell and thus not sourcing .bashrc. However, we can fix this by replacing our existing approach of using templated AIRFLOW__CORE__SQL_ALCHEMY_CONN with AIRFLOW__CORE__SQL_ALCHEMY_CONN_CMD.

This is caused bykubectl exec not creating a login bash shell and thus not sourcing .bashrc. However, we can fix this by replacing our existing approach of using templated AIRFLOW__CORE__SQL_ALCHEMY_CONN with AIRFLOW__CORE__SQL_ALCHEMY_CONN_CMD.

When would this change take place in the official release?

@sungchun12
FYI, you can just use the method documented in the README.md, and run the following to get an interactive bash shell:
# use this to run commands like: `airflow create_user`
kubectl exec \
  -it \
  --namespace airflow \
  --container airflow-scheduler \
  Deployment/airflow-scheduler \
  /bin/bash

But I will fix the environment variables at some point.

@thesuperzapper Yep, I'm able to do that as a workaround. However, it'll be nice in a simple post helm deployment script to run in a remote shell vs. interactive one once you update the environment variable!

I am having issues with the liveness prove related to this.
The variable is properly set as it prints the correct postgresql endpoint when kubectl exec -it bash into the airflow-scheduler container:
echo $AIRFLOW__CORE__SQL_ALCHEMY_CONN
But, the airflow-scheduler container keeps restarting as it fails de liveness prove. kubectl describe <airfllow-scheduler-pod> <airflow-scheduler> gives:
Warning  Unhealthy  94s (x11 over 16m)  kubelet, kube-node-0-kubelet.kube-dev.mesos  Liveness probe failed: Traceback (most recent call last):
 File "<string>", line 4, in <module>
 File "/home/airflow/.local/lib/python3.6/site-packages/airflow/__init__.py", line 31, in <module>
   from airflow.utils.log.logging_mixin import LoggingMixin
 File "/home/airflow/.local/lib/python3.6/site-packages/airflow/utils/__init__.py", line 24, in <module>
   from .decorators import apply_defaults as _apply_defaults
 File "/home/airflow/.local/lib/python3.6/site-packages/airflow/utils/decorators.py", line 36, in <module>
   from airflow import settings
 File "/home/airflow/.local/lib/python3.6/site-packages/airflow/settings.py", line 37, in <module>
   from airflow.configuration import conf, AIRFLOW_HOME, WEBSERVER_CONFIG  # NOQA F401
 File "/home/airflow/.local/lib/python3.6/site-packages/airflow/configuration.py", line 731, in <module>
   conf.read(AIRFLOW_CONFIG)
 File "/home/airflow/.local/lib/python3.6/site-packages/airflow/configuration.py", line 421, in read
   self._validate()
 File "/home/airflow/.local/lib/python3.6/site-packages/airflow/configuration.py", line 213, in _validate
   self._validate_config_dependencies()
 File "/home/airflow/.local/lib/python3.6/site-packages/airflow/configuration.py", line 247, in _validate_config_dependencies
   self.get('core', 'executor')))
airflow.exceptions.AirflowConfigException: error: cannot use sqlite with the CeleryExecutor

@Arefer you are correct, I have raised an issue for this #23804

I have now fixed this issue in 7.10.0


IUS python36u packages are deprecated

It looks like CentOS has finally added Python 3.6 to @base, and the IUS python36u* packages now point to the new base packages. Update the role to point directly to the @base packages.
This is not a breaking change for this role, but there are surprises for downstream packages that require the old IUS packages. For example, installing uwsgi-plugin-python36u fails because it cannot find python36u, which is now just python3@base (FYI, uwsgi-plugin-python36@epel can be used as a replacement).

Issue resolved by commit cbae266f588f3f703ff67a7768b9638a07e17e75.


Add pluggy extension

Description

Screencast

Checklist

[x] I read the extension guidelines
[x] I read the documentation about publishing
[x] I ran npm run build and tested this distribution build in Raycast
[x] I checked that files in the assets folder are used by the extension itself
[x] I checked that assets used by the README are placed outside of the metadata folder

Do you think it makes sense to move status to accessories instead (with a tooltip) to unclutter the left side a bit?

Yes I do, thanks for the feedback, I'll post the changes shortly!


setOnItemLongClickListener on a listview with ExpandableListItemAdapter

Hey,
I got ExpandableListItemAdapter adapter working just fine, but I was wondering if it was possible to use an onLongClickListener on an item in the list view. When I try and do it currently, nothing happens when I try and long click an item in the list view. If there is something that I'm missing, let me know!
Thanks for the help!

how can i implement long click event on expandable list????????


Remove lifetime

A mentioned in #18 the added lifetime would limit use of the API as well as cause issue with PyO3 bindings. This removes it using Vecs and to_vecs.
I do plan on using this commit as the basis for my optimization experiments which may be included depending on if this is merged or not.
Benchmark (mimalloc):

matchit
	path-tree
	w/o lifetime

insert
	59µs
	73µs
	84µs

find
	45µs
	40µs
	40µs

Benchmark (Windows' allocator):

matchit
	path-tree
	w/o lifetime

insert
	109µs
	128µs
	158µs

find
	31µs
	40µs
	39µs

The biggest performance problem when capturing parameters.
Currently, I haven't found a better way to update ranges.

matchit uses Params to improve performance.
We use smallvec.

From my profiling I did notice that each find call was an (almost) perfect triangle, means that its not taking too much time in any one check. I plan on taking a closes look using finer tracing than the initial setup I had.
I'll take a look into inserting paths, one of the ideas I had was to store the inserted paths in PathTree and have Nodes reference the path by index and use Ranges for slicing. Basically how the router works but for paths themselves.
What was the issue with capturing parameters. I did notice a second parameter check not sure if thats what you're talking about though.

What was the issue with capturing parameters.
Updates parameters array, https://github.com/viz-rs/path-tree/blob/main/src/node.rs#L284-L285

Ah those, I was wondering about those. My initial thoughts were to just place them into a single insert (eg: ranges.push(start..start + n);) using the Range enum directly instead of having to recreate it when creating the raw parameteres list. I did try it out during my first pass over the code but it didn't seem like it changed much in terms of performance.
My other idea was to have each 'leaf' Node hold all its parameters and only build the parameter list when it found a match, but it would have complicated the router ever further.

Using Range is very clean and simple.

My other idea was to have each 'leaf' Node hold all its parameters and only build the parameter list when it found a match, but it would have complicated the router ever further.

Yes we already have the path pieces that seem to do.

If it is possible to merge, assign to me. :)

I fixed the merge conflict but I am unable to assign you to the issue as I don't have the ability to do so.

@Txuritan  I have invited you to role write.

Landed in v0.5.1. Thanks.

This worked out great for the Python wrapper: https://github.com/adriangb/routrie/commit/faa4ad8bf2a10a06ed621eb1e30a5d4cceb5cc91#diff-b1a35a68f14e696205874893c07fd24fdb88882b47c23cc0e0c80a30c7d53759
It would be nice to figure out at some point how to lazily build the parameters instead of calling params() (just like you are doing in Rust) before returning the data to Python, but I suspect it would get pretty complicated again with lifetimes for probably little benefit to users.


Bump etcd image tag

Why is this PR needed?
This PR is needed to make sure the new ectd image version is being used when upgrading from v4.2.3 to v4.5.0. Note that despite the etcd version is the same the image is different as the first is based on SLE-15-SP1 and the latter on SLE-15-SP2.
Fixes SUSE/avant-garde#1924
What does this PR do?
This commit appends the revision to the etcd image reference. This is
done to force kubeadm to write a new manifest including the new image
reference when upgrading from v4.2.3 (SLE15-SP1 based) to v4.5.0
(SLE15-SP2 based) CaaSP version.
Anything else a reviewer needs to know?
Special test cases, manual steps, links to resources or anything else that could be helpful to the reviewer.
Info for QA
Fixes SUSE/avant-garde#1924, after the migration from v4.2.3 to v4.5.0 the upgraded clusters should be using the etcd image tagged with 3.4.3-rev1.
Related info
Info that can be relevant for QA:

link to other PRs that should be merged together
link to packages that should be released together
upstream issues

Status BEFORE applying the patch
How can we reproduce the issue? How can we see this issue? Please provide the steps and the prove
this issue is not fixed.
Status AFTER applying the patch
How can we validate this issue is fixed? Please provide the steps and the prove this issue is fixed.
Docs
If docs need to be updated, please add a link to a PR to https://github.com/SUSE/doc-caasp.
At the time of creating the issue, this PR can be work in progress (set its title to [WIP]),
but the documentation needs to be finalized before the PR can be merged.
Merge restrictions
(Please do not edit this)
We are in v4-maintenance phase, so we will restrict what can be merged to prevent unexpected surprises:
What can be merged (merge criteria):
    2 approvals:
        1 developer: code is fine
        1 QA: QA is fine
    there is a PR for updating documentation (or a statement that this is not needed)

I am not sure how -rev1 parses vs <IP_ADDRESS>

I am not sure how -rev1 parses vs <IP_ADDRESS>

This is a valid concern, in fact according to https://semver.org the relationship across those two tags is 3.4.3-rev1 < 3.4.3. However this is unclear to me if this is the actual comparison that is happening behind the scenes.

Exactly, this is why i was concerned.

i double checked with a quick PoC: https://github.com/evrardjp/poc-kubernetes-version-parse

Can we do 3.4.4?

Can we do 3.4.4?

I don't thing so, this would be misleading if the actual version is 3.4.3. @evrardjp do you think 3.4.3 vs <IP_ADDRESS> would do the trick? Could you verify if something like that crashes the comparison code (probably it does, this is no longer semver).

<IP_ADDRESS> gets errors in parsing, so I would say no.

Closing, this not a good solution. Probably we should rethink the three tags strategy, probably should have been +revX instead of -revX. In any case we are not considering chaning the tagging strategy now.


Installing latest version causes admin menu to dissapeer

Installed the extension via Magento connect using https://connect20.magentocommerce.com/c1b490ed2f887a5e9e7536e0a63fd69b/cloudinary+Cloudinary_Cloudinary-2.8.3
Successfully.
However, when returning to admin, the entire admin menu has vanished. Can still see the dashboard, though cant seem to access other admin pages.
(cache off, no compilation etc, obvious stuff)
Inspector reveals, the admin menu isnt just hidden, its never generated.
Removing code/community/Cloudinary/* resolves.
Using Magento ver. <IP_ADDRESS>
Any ideas?

Sorry to be clear, Removing code/community/Cloudinary/* does notresolve the issue, just stops the module from loading !!!

Hey @CosyStudios its, not something that we heard of prior to this case.
Would it be possible to share the credentials of your environment with us so we could take a closer look?
You could open a request at<EMAIL_ADDRESS>and we could check this on our end as well.

Closing this issue due to the time elapsed. Please feel free to either re-open the issue, contact our support at http://support.cloudinary.com or create a new ticket if you have any additional issues.


[BUG] Cannot Get Data Factory Pipeline Run Status

Library name and version
Microsoft.Azure.Management.DataFactory
Describe the bug
This throws an error
var run = client.Pipelines.CreateRunWithHttpMessagesAsync(_dataFactory.ResourceGroupName, _dataFactory.FactoryName, _dataFactory.StartPipelineName, parameters: parameters).Result.Body;
Expected behavior
Return the Response object instead
Actual behavior
throw an error
Microsoft.Rest.Azure.CloudException: 'Api version '2018-06-01' is not available for your subscription. Please contact customer support for assistance with this feature: https://azure.microsoft.com/support/options/'
Reproduction Steps
var provider = _tokenProviderService.GetProvider();
var token = provider.GetAccessTokenAsync("https://management.azure.com", _dataFactory.TenantId);
ServiceClientCredentials credentials = new TokenCredentials(token.Result);
DataFactoryManagementClient client = new DataFactoryManagementClient(credentials);
client.SubscriptionId = _dataFactory.SubscriptionId;
var parameters = new Dictionary<string, object>{};
var run = client.Pipelines.CreateRunWithHttpMessagesAsync(_dataFactory.ResourceGroupName, _dataFactory.FactoryName, _dataFactory.StartPipelineName, parameters: parameters).Result.Body;
Environment
No response

Thank you for your feedback.  Tagging and routing to the team member best able to assist.

Hi @jsquire thanks for your reply
Somehow it is working now, I could not figure out what happened


Cannot get my 27XXXXXXXX series logger to work

Hi everyone!
I'm trying to connect my On-Grid Solar PV system composed by:
Deye Grid connected PV Inverter (SUN-3K_-G03)
logger with serial number 27XXXXXXXX and Firmware version LSW3_15_FFFF_1.0.9E
I am a newbie so I dont know what I'm doing wrong. I followed all the steps in the guide but cannot get it working

similar situation, my module version is LSW3_15_270A_1.68 and my serial number is 27XXXXXXXX

Hi,
Similar sitution here.
My module is a Solis S3-WIFI-ST datalogger with serial number 51215146.
I do not know its firmware version.
How can I check the firmware version?
I can ping the device but I cannot log into it.
My inverter is a Solis Rhi-3.6k-48es-5g
I use Solarman version 1.4.3 with solis_hybrid.yaml (I downgraded from version 1.5.1 as I saw a post where some succeeded to resolve their connection issues with Solarman version 1.4.3 ).
I am also a newbie and I don't know what I'm doing wrong.
The integration installation via HACS went well, but the connection is 'disconneccted' and the entities are 'unknown'.
I hope anyone can help me out...
Kind regards,
Wim

My module is a Solis S3-WIFI-ST datalogger with serial number 51215146. I do not know its firmware version. How can I check the firmware version? I can ping the device but I cannot log into it.

Unfortunately the Solis S3-WIFI-ST is not a Solarman based data logger and this integration will not work for you. I suggest you look at solis-sensor (cloud polling) or learn about RS-485.

Hi Jonathan,
Thank you for your confirmation.
Indeed, in the mean time I found https://github.com/alienatedsec/solis-ha-modbus-cloud that also gives a nice overview of the differtent types of Solis dataloggers and their capabilities.
I have ordered a new DSL-W datalogger, I'll give it a try if it gets delivered (not sure if the supplier can deliver, nor if they will deliver the DSL-W as in @alienatedsec 's overview).
I'll keep you posted.
Kind regards,
Wim

@WimDeceuninck how have you downgraded? It does not let me download an older version?

Hi Stephan,
Downgrade is easy via HACS.
Select the integration.
Go to ‘redownload’.
And chose the version you want to use.
Kind regards,
Wim
Van: PCB0815 @.>
Verzonden: Friday, 21 June 2024 15:14
Aan: StephanJoubert/home_assistant_solarman @.>
CC: Willem de Zwijger @.>; Mention @.>
Onderwerp: Re: [StephanJoubert/home_assistant_solarman] Cannot get my 27XXXXXXXX series logger to work (Issue #568)
@WimDeceuninck https://github.com/WimDeceuninck  how have you downgraded? It does not let me download an older version?
—
Reply to this email directly, view it on GitHub https://github.com/StephanJoubert/home_assistant_solarman/issues/568#issuecomment-2182731259 , or unsubscribe https://github.com/notifications/unsubscribe-auth/BAPAHPO2THD5NNFO22Q5PSTZIQRIVAVCNFSM6AAAAABGR7DXP6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBSG4ZTCMRVHE .
You are receiving this because you were mentioned.  https://github.com/notifications/beacon/BAPAHPI3RWOMOGEGU6GMXOTZIQRIVA5CNFSM6AAAAABGR7DXP6WGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTUCDHK7W.gif Message ID: @.*** @.***> >
--
This email has been checked for viruses by Avast antivirus software.
www.avast.com


Carthage: Dependency "CameraEngine" has no shared framework schemes

*** Skipped building CameraEngine due to the error:
Dependency "CameraEngine" has no shared framework schemes

Any updates @remirobert ?

Probably need to make sure that .xcodeproj/xcshareddata/xcschemes is added and pushed to github.

Now scheme is shared, but It's still causes issues when it's trying to build with Carthage. Make sure you've update git tag, so Carthage will select the right commit.


Triage existing bugs on GH

We have a huge backlog of issues, many of which are outdated or irrelevant to the Light Table core project. We should prune those issues and focus on labelling the remaining ones so they can be dealt with more quickly.

Closing as we no longer need a reminder for this


Add support for printing violations in a external file

It may be a good option for the users to have a detailed HTML report so that the artifact can be shared across.

To get html report you can easily use npm axe-html-reporter pack, example of usage:
const violations = await getViolations(...) // from axe-playwright
if (violations.length > 0) {
createHtmlReport({ // from axe-html-reporter
results: {
violations: violations,
},
options: {
outputDir: resultDir,
reportFileName: ${this.testName}.html,
},
});
}
The author can use the same solution or use his own HTML parser.

@BeataKr Thanks very useful. Would you mind integrate it with the current utility internally? Would appreciate a PR

Yes, when I'll find a time for this, with pleasure :D

@abhinaba-ghosh can you have a look on https://github.com/abhinaba-ghosh/axe-playwright/pull/41
Example (working) usage:
 const axeAudit = async auditName => {
    await getAndSaveViolations(page, undefined, {
      axeOptions: {
        runOnly: {
          type: 'tag',
          values: ['wcag2a', 'wcag2aa', 'wcag21aa']
        }
      }
    },
      {
        outputDirPath: RESULTS_DIR,
        outputDir: 'accessibility',
        reportFileName: 'accessibility-audit-' +auditName.replace(/\s/g, '-') + '.html'
      })
  };

Hi,
I have started using Playwright with Axe on TypeScript. Currently I can print Reports of Axe violations through console.log. But I want to use 'axe-html-reporter'. But When Im trying it is giving me error. Im using the following code:
import { Page, expect } from '@playwright/test'
import {injectAxe, checkA11y, getViolations, reportViolations} from 'axe-playwright'
import { createHtmlReport } from 'axe-html-reporter';
import {test} from "@playwright/test"
test('exampleTest01 with Axe Injected in it', async ({page}) => {
await page.goto('https://apps.esignlive.com/login');
const violations = await getViolations(page) // from axe-playwright

if (violations.length > 0) {
createHtmlReport({ // from axe-html-reporter
results: {
violations: violations,
},
options: {
outputDir: 'resultDir',
reportFileName: 'fileName.html',
},
});
}
});
but I get the following error:
page.evaluate: TypeError: Cannot read properties of undefined (reading 'run')
   9 |     await page.goto('https://apps.esignlive.com/login');
  10 |
> 11 |     const violations = await getViolations(page) // from axe-playwright
     |                              ^
  12 |
  13 | if (violations.length > 0) {
  14 |           createHtmlReport({ // from axe-html-reporter

Do I need to initialize page and if yes then how?

I am checking this today 💪

Hello Abhinaba,
Did you happen to find any solution to it?

Apologies for the delay. I am really busy with some critical releases. Making a room for the next weekend for sure.

Hi Abhinaba,
I managed to solve with the following code
test('exampleTest01 with Axe Injected in it', async ({ page }) => {
await page.goto('webSiteAddressURL');
await injectAxe(page);
const violations = await getViolations(page) // from axe-playwright

if (violations.length > 0) {
    createHtmlReport({ // from axe-html-reporter
        results: {
            violations: violations,
        },
        options: {
            outputDir: 'resultDir',
            reportFileName: 'fileName.html',
        },
    });
}

});
Actually there was a dependency I was using older version and this"[^1.18.1]" is the correct version.

Hi @Zohairtoo that's great. Would you mind raising a PR to integrate this behaviour in the library itself?

Due to https://github.com/abhinaba-ghosh/axe-playwright/pull/209 released with 1.2.1 we can close, right?


A bit of HLinting, and testing with 8.6.4

Great lecture and nice package.
There is still room for updating the .travis.yml and updating the glambda.cabal for v2 and Cabal 3.

Glad you found this helpful. Thanks for the contribution!


Add new hook

Maybe we should add a script that generates the add hook template. Also I will release next version soon.


Use a relative path to allow tests to run from within an IDE

Hi, I tried running that test from within my IDE, PyCharm, but it failed due to the configuration location used. Would it be possible to use relative paths, so that tests run on CI or IDE's like mine, please? This PR has a tentative fix :+1: (I was interested on running just this one test, so I didn't check the other tests to see if a similar fix would be useful there.)
Thanks!
-Bruno

Thanks for your PR, this has now been fixed by https://github.com/ecmwf/aviso/pull/9 which addresses all the tests

Thanks for your PR, this has now been fixed by #9 which addresses all the tests

Thank you!!!


⚠ ClamAV has degraded performance

In 3d4d689, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 574 ms

Resolved: ClamAV performance has improved in 13ce4fe after 7 minutes.


Proposed 1.3 release

rewritten chef resources to use new custom resource syntax
updated agent startup config files to use GO_SERVER_URL instead of host and port
by default install java 8
isolated gocd_agent_autoregister_file resource
added golang agent support
use apt cookbook >= 3.0

I'll be testing this in next days before releasing.
Any feedback welcome.

FYI. We have turned on the http protocol so there is no need for apt-protocol-https anymore. As long as you are importing the GPG keys properly, the packages can be guaranteed to be untempered.

Results of the Rakefile style check:
Starting Foodcritic linting...
FC009: Resource attribute not recognised: go-cookbook/recipes/agent_windows_install.rb:21
FC009: Resource attribute not recognised: go-cookbook/recipes/server_windows_install.rb:13
FC019: Access node attributes in a consistent manner: go-cookbook/recipes/java.rb:2

Results of Rakefile unit:chefspec check:
Finished in 29.99 seconds (files took 1.66 seconds to load)
155 examples, 0 failures

ChefSpec Coverage report generated...

  Total Resources:   59
  Touched Resources: 45
  Touch Coverage:    76.27%

Untouched Resources:

  ohai[reload_passwd_for_go_user]    gocd/recipes/agent_linux_install.rb:3
  apt_package[apt-transport-https]   gocd/recipes/repository.rb:4
  directory[/mnt/big_drive]          gocd/providers/agent.rb:9
  directory[/var/log/my-go-agent]    gocd/providers/agent.rb:9
  directory[/mnt/big_drive/config]   gocd/providers/agent.rb:15
  bash[setup init.d for my-go-agent]   gocd/providers/agent.rb:21
  link[/usr/share/my-go-agent]       gocd/providers/agent.rb:29
  bash[setup init.d for go-agent]    gocd/providers/agent.rb:21
  link[/usr/share/go-agent]          gocd/providers/agent.rb:29
  ohai[reload_passwd]                gocd/recipes/server_linux_install.rb:3
  apt_package[unzip]                 gocd/recipes/server_linux_install.rb:9
  http_request[verify_go-server_comes_up]   gocd/recipes/server.rb:16
  ruby_block[publish_autoregister_key]   gocd/recipes/server.rb:23
  yum_package[unzip]                 gocd/recipes/server_linux_install.rb:9


[SPARK-7685][ML] Apply weights to different samples in Logistic Regression

In fraud detection dataset, almost all the samples are negative while only couple of them are positive. This type of high imbalanced data will bias the models toward negative resulting poor performance. In python-scikit, they provide a correction allowing users to Over-/undersample the samples of each class according to the given weights. In auto mode, selects weights inversely proportional to class frequencies in the training set. This can be done in a more efficient way by multiplying the weights into loss and gradient instead of doing actual over/undersampling in the training dataset which is very expensive.
http://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html
On the other hand, some of the training data maybe more important like the training samples from tenure users while the training samples from new users maybe less important. We should be able to provide another "weight: Double" information in the LabeledPoint to weight them differently in the learning algorithm.

Merged build triggered.

Merged build started.

Test build #39507 has started for   PR 7884 at commit e83b444.

Merged build triggered.

Merged build started.

Test build #39508 has started for   PR 7884 at commit 9ac7518.

Test build #39507 has finished for   PR 7884 at commit e83b444.

This patch fails MiMa tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Merged build triggered.

Merged build started.

Test build #39510 has started for   PR 7884 at commit 4dfc09c.

Test build #39510 has finished for   PR 7884 at commit 4dfc09c.

This patch fails Scala style tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
  final val probabilityCol: Param[String] = new Param[String](this, "probabilityCol", "Column name for predicted class conditional probabilities. Note: Not all models output well-calibrated probability estimates! These probabilities should be treated as confidences, not precise probabilities")

Merged build finished. Test FAILed.

Test build #39508 has finished for   PR 7884 at commit 9ac7518.

This patch fails MiMa tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
  final val probabilityCol: Param[String] = new Param[String](this, "probabilityCol", "Column name for predicted class conditional probabilities. Note: Not all models output well-calibrated probability estimates! These probabilities should be treated as confidences, not precise probabilities")

Merged build finished. Test FAILed.

Merged build triggered.

Merged build started.

Test build #39512 has started for   PR 7884 at commit 8bbcfe6.

Test build #39512 has finished for   PR 7884 at commit 8bbcfe6.

This patch fails Scala style tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
  final val probabilityCol: Param[String] = new Param[String](this, "probabilityCol", "Column name for predicted class conditional probabilities. Note: Not all models output well-calibrated probability estimates! These probabilities should be treated as confidences, not precise probabilities")

Merged build finished. Test FAILed.

Merged build triggered.

Merged build started.

Test build #39521 has started for   PR 7884 at commit 1e1d7a7.

Test build #39521 has finished for   PR 7884 at commit 1e1d7a7.

This patch fails Scala style tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
  final val probabilityCol: Param[String] = new Param[String](this, "probabilityCol", "Column name for predicted class conditional probabilities. Note: Not all models output well-calibrated probability estimates! These probabilities should be treated as confidences, not precise probabilities")

Merged build finished. Test FAILed.

I'll make a pass through this.  (Please let me know if I should wait b/c of ongoing updates.)

Initial thought:  I don't think the user should need 2 parameters to specify whether to use sample weights.
What if, instead, there were no default weightCol?  If the user specifies weightCol, then we use sample weights.  If weightCol is unspecified or an empty String, then we don't use sample weights.  Then we could eliminate HasWeightedSample.

@jkbradley Sounds fair. I will also add tests now. BTW, can you merge https://github.com/apache/spark/pull/7875/files which is already ready to be merged. I will have another PR to do weighted linear regression which depends on this PR. Thanks.

Also, are you adding unit tests?
Thanks for adding weighted instance support---very useful!

This is very useful for us as well. Our training samples will have exponential decay weight as time goes by. I am trying to use it now, and it seems work well. Will be nice to have similar feature in GBDT which is used in our current production. Open a JIRA, and I can work on GBDT version.

Merged build triggered.

Merged build started.

Merged build finished. Test FAILed.

Jenkins, please test again

Merged build triggered.

Merged build started.

Test build #39688 has started for   PR 7884 at commit 05328f1.

Test build #39688 has finished for   PR 7884 at commit 05328f1.

This patch fails to build.
This patch merges cleanly.
This patch adds the following public classes (experimental):
  final val probabilityCol: Param[String] = new Param[String](this, "probabilityCol", "Column name for predicted class conditional probabilities. Note: Not all models output well-calibrated probability estimates! These probabilities should be treated as confidences, not precise probabilities")

Merged build finished. Test FAILed.

Here's the link for the GBT JIRA: [https://issues.apache.org/jira/browse/SPARK-9612]

@dbtsai Earlier, I had recommended "If weightCol is unspecified or an empty String, then we don't use sample weights."  It occurred to me that we should set the default to an empty String (since Params should have defaults or be required, in general).

@jkbradley and @mengxr  I had weightCol as empty as default in my local branch. Sorry for confusion. This PR is meant to be WIP to see if we have enough time to make it in 1.5, and not intended to be merged. Thanks.

Following this PR, how about file a JIRA to implement weight for LinearRegression as well? I am thinking to use same weighting API, which will enable us to develop a general boosting algorithm with the Pipeline API. What do you think?

@rotationsymmetry Please help to create the JIRA. For LinearRegression, that should be smilier to this one. Once this one is merged, we can start to port it to LinearRegression.

@dbtsai Jira added https://issues.apache.org/jira/browse/SPARK-9642
Once logistic regression PR is merged, I will use it as the model and work on linear regression.

Merged build triggered.

Merged build started.

Test build #40013 has started for   PR 7884 at commit a0b15e6.

Test build #40013 has finished for   PR 7884 at commit a0b15e6.

This patch fails Scala style tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
  final val probabilityCol: Param[String] = new Param[String](this, "probabilityCol", "Column name for predicted class conditional probabilities. Note: Not all models output well-calibrated probability estimates! These probabilities should be treated as confidences, not precise probabilities")

Merged build finished. Test FAILed.

Merged build triggered.

Merged build started.

Merged build finished. Test FAILed.

Merged build triggered.

Merged build started.

Test build #40017 has started for   PR 7884 at commit cd4cfbc.

Test build #40017 has finished for   PR 7884 at commit cd4cfbc.

This patch fails Scala style tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
  final val probabilityCol: Param[String] = new Param[String](this, "probabilityCol", "Column name for predicted class conditional probabilities. Note: Not all models output well-calibrated probability estimates! These probabilities should be treated as confidences, not precise probabilities")

Merged build finished. Test FAILed.

Jenkins, please test again.

Merged build triggered.

Merged build started.

Test build #40020 has started for   PR 7884 at commit 8e0df52.

Merged build triggered.

Merged build started.

Test build #40021 has started for   PR 7884 at commit 68661a1.

Test build #40020 has finished for   PR 7884 at commit 8e0df52.

This patch fails MiMa tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
  final val probabilityCol: Param[String] = new Param[String](this, "probabilityCol", "Column name for predicted class conditional probabilities. Note: Not all models output well-calibrated probability estimates! These probabilities should be treated as confidences, not precise probabilities")

Merged build finished. Test FAILed.

Jenkins, retest this please

Jenkins, retest this please

Merged build triggered.

Merged build started.

Test build #40023 has started for   PR 7884 at commit 68661a1.

@jkbradley and @mengxr This PR is ready for review. Thanks.

Test build #40021 has finished for   PR 7884 at commit 68661a1.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
  final val probabilityCol: Param[String] = new Param[String](this, "probabilityCol", "Column name for predicted class conditional probabilities. Note: Not all models output well-calibrated probability estimates! These probabilities should be treated as confidences, not precise probabilities")

Merged build finished. Test FAILed.

Test build #40023 has finished for   PR 7884 at commit 68661a1.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
  final val probabilityCol: Param[String] = new Param[String](this, "probabilityCol", "Column name for predicted class conditional probabilities. Note: Not all models output well-calibrated probability estimates! These probabilities should be treated as confidences, not precise probabilities")

Merged build finished. Test FAILed.

Jenkins, retest this please

Merged build triggered.

Merged build started.

Test build #40046 has started for   PR 7884 at commit 68661a1.

Build triggered.

Build started.

Test build #40047 has started for   PR 7884 at commit 457989d.

Merged build triggered.

Merged build started.

Test build #40057 has started for   PR 7884 at commit c8b0e43.

Test build #40057 has finished for   PR 7884 at commit c8b0e43.

This patch fails to build.
This patch merges cleanly.
This patch adds the following public classes (experimental):
sealed trait LogisticRegressionTrainingSummary extends LogisticRegressionSummary
sealed trait LogisticRegressionSummary extends Serializable
  final val probabilityCol: Param[String] = new Param[String](this, "probabilityCol", "Column name for predicted class conditional probabilities. Note: Not all models output well-calibrated probability estimates! These probabilities should be treated as confidences, not precise probabilities")
final class SpecificMutableRow(val values: Array[MutableValue])
      final class SpecificRow extends $
trait BaseGenericInternalRow extends InternalRow
class GenericMutableRow(values: Array[Any]) extends MutableRow with BaseGenericInternalRow
case class StringTranslate(srcExpr: Expression, matchingExpr: Expression, replaceExpr: Expression)

Merged build finished. Test FAILed.

Jenkins, retest this please

A recent PR is causing build failures, but it's being fixed now.

Merged build triggered.

Merged build started.

Test build #40061 has started for   PR 7884 at commit c8b0e43.

Test build #40061 has finished for   PR 7884 at commit c8b0e43.

This patch fails to build.
This patch merges cleanly.
This patch adds the following public classes (experimental):
sealed trait LogisticRegressionTrainingSummary extends LogisticRegressionSummary
sealed trait LogisticRegressionSummary extends Serializable
  final val probabilityCol: Param[String] = new Param[String](this, "probabilityCol", "Column name for predicted class conditional probabilities. Note: Not all models output well-calibrated probability estimates! These probabilities should be treated as confidences, not precise probabilities")
final class SpecificMutableRow(val values: Array[MutableValue])
      final class SpecificRow extends $
trait BaseGenericInternalRow extends InternalRow
class GenericMutableRow(values: Array[Any]) extends MutableRow with BaseGenericInternalRow
case class StringTranslate(srcExpr: Expression, matchingExpr: Expression, replaceExpr: Expression)

Merged build finished. Test FAILed.

Test build #40046 has finished for   PR 7884 at commit 68661a1.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
  final val probabilityCol: Param[String] = new Param[String](this, "probabilityCol", "Column name for predicted class conditional probabilities. Note: Not all models output well-calibrated probability estimates! These probabilities should be treated as confidences, not precise probabilities")

Merged build finished. Test FAILed.

Test build #40047 has finished for   PR 7884 at commit 457989d.

This patch fails PySpark unit tests.
This patch does not merge cleanly.
This patch adds the following public classes (experimental):
  final val probabilityCol: Param[String] = new Param[String](this, "probabilityCol", "Column name for predicted class conditional probabilities. Note: Not all models output well-calibrated probability estimates! These probabilities should be treated as confidences, not precise probabilities")

Build finished. Test FAILed.

Merged build triggered.

Merged build started.

Test build #40099 has started for   PR 7884 at commit 782d701.

Test build #40099 has finished for   PR 7884 at commit 782d701.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Jenkins, retest this please

Merged build triggered.

Merged build started.

Test build #40612 has started for   PR 7884 at commit 782d701.

Test build #40612 has finished for   PR 7884 at commit 782d701.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
  final val probabilityCol: Param[String] = new Param[String](this, "probabilityCol", "Column name for predicted class conditional probabilities. Note: Not all models output well-calibrated probability estimates! These probabilities should be treated as confidences, not precise probabilities")

Merged build finished. Test FAILed.

Merged build triggered.

Merged build started.

Test build #41428 has started for   PR 7884 at commit 31b2ee8.

Test build #41428 has finished for   PR 7884 at commit 31b2ee8.

This patch fails Scala style tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
  final val probabilityCol: Param[String] = new Param[String](this, "probabilityCol", "Column name for predicted class conditional probabilities. Note: Not all models output well-calibrated probability estimates! These probabilities should be treated as confidences, not precise probabilities")

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41428/
Test FAILed.

Merged build triggered.

Merged build started.

Test build #41429 has started for   PR 7884 at commit eec6436.

Test build #41429 has finished for   PR 7884 at commit eec6436.

This patch fails to build.
This patch merges cleanly.
This patch adds the following public classes (experimental):
  final val probabilityCol: Param[String] = new Param[String](this, "probabilityCol", "Column name for predicted class conditional probabilities. Note: Not all models output well-calibrated probability estimates! These probabilities should be treated as confidences, not precise probabilities")

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41429/
Test FAILed.

Merged build triggered.

Merged build started.

Test build #41444 has started for   PR 7884 at commit 30f469f.

Test build #41444 has finished for   PR 7884 at commit 30f469f.

This patch fails Scala style tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
  final val probabilityCol: Param[String] = new Param[String](this, "probabilityCol", "Column name for predicted class conditional probabilities. Note: Not all models output well-calibrated probability estimates! These probabilities should be treated as confidences, not precise probabilities")

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41444/
Test FAILed.

Merged build triggered.

Merged build started.

Test build #41447 has started for   PR 7884 at commit 5281663.

Test build #41447 has finished for   PR 7884 at commit 5281663.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
  final val probabilityCol: Param[String] = new Param[String](this, "probabilityCol", "Column name for predicted class conditional probabilities. Note: Not all models output well-calibrated probability estimates! These probabilities should be treated as confidences, not precise probabilities")

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41447/
Test PASSed.

Merged build finished. Test PASSed.

Merged build triggered.

Merged build started.

Hello @jkbradley @mengxr, this is ready to be reviewed. Do you have a chance to have a pass on this PR? Thanks.

Test build #41454 has started for   PR 7884 at commit 3ead826.

Test build #41454 timed out     for PR 7884 at commit 3ead826     after a configured wait of 175m.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41454/
Test FAILed.

Jenkins, retest this please

Merged build triggered.

Merged build started.

Test build #41465 has started for   PR 7884 at commit 3ead826.

Test build #41465 has finished for   PR 7884 at commit 3ead826.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
  final val probabilityCol: Param[String] = new Param[String](this, "probabilityCol", "Column name for predicted class conditional probabilities. Note: Not all models output well-calibrated probability estimates! These probabilities should be treated as confidences, not precise probabilities")

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41465/
Test FAILed.

Jenkins, retest this please

Merged build triggered.

Merged build started.

Test build #41480 has started for   PR 7884 at commit 3ead826.

Test build #41480 has finished for   PR 7884 at commit 3ead826.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
  final val probabilityCol: Param[String] = new Param[String](this, "probabilityCol", "Column name for predicted class conditional probabilities. Note: Not all models output well-calibrated probability estimates! These probabilities should be treated as confidences, not precise probabilities")

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41480/
Test FAILed.

Seems Jenkins timeout a lot. Give another shot.

Jenkins, retest this please

Merged build triggered.

Merged build started.

Test build #41510 has started for   PR 7884 at commit 3ead826.

Test build #41510 has finished for   PR 7884 at commit 3ead826.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
  final val probabilityCol: Param[String] = new Param[String](this, "probabilityCol", "Column name for predicted class conditional probabilities. Note: Not all models output well-calibrated probability estimates! These probabilities should be treated as confidences, not precise probabilities")

Since instances : RDD[(Double, Vector)] is effectively processed as RDD[(Double, 1.0, Vector)] (due to default params / method aliases for add), I wonder if we can just map instances into a RDD[(Double, 1.0, Vector)] if $(weightCol).isEmpty and avoid using the Eithers throughout the code. This will add an additional pass, but I think it should be negligible compared to the optimizer time.

I think we can also map it to RDD[(Double, 1.0, Vector)] as you suggest to avoid using Either, but will result in slightly larger training set which should be negligible. I am okay with doing either way.

LGTM, I slightly prefer the RDD[(Double, 1.0, Vector)] approach but it's your call

I know Xiangrui is using RDD[(Double, 1.0, Vector)] in isotonic regression, so I don't mind as well as long as everyone is on the same page.

Just a thought I had while looking over this PR: It makes more sense to me to refer to the sample weights as sampleWeight instead of weight, since the regression coefficient vector is called weights regularly throughout the code. This was causing me some confusion and it is a bit clearer to call it sampleWeight.

@sethah  I prefer sampleWeight as well which I used in the initial version.
However, there is already code using weight for the same purpose in https://github.com/apache/spark/blob/master/mllib/src/main/scala/org/apache/spark/ml/regression/IsotonicRegression.scala#L145 , so if I change it, the api will be changed.

Ah, I missed that, my apologies. That's a bit unfortunate!

Merged build triggered.

Merged build started.

Test build #41910 has started for   PR 7884 at commit a4c3e54.

Test build #41910 has finished for   PR 7884 at commit a4c3e54.

This patch fails to build.
This patch merges cleanly.
This patch adds the following public classes (experimental):
  final val probabilityCol: Param[String] = new Param[String](this, "probabilityCol", "Column name for predicted class conditional probabilities. Note: Not all models output well-calibrated probability estimates! These probabilities should be treated as confidences, not precise probabilities")

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41910/
Test FAILed.

Merged build triggered.

Merged build started.

Test build #41911 has started for   PR 7884 at commit e94bb95.

Test build #41911 has finished for   PR 7884 at commit e94bb95.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
  final val probabilityCol: Param[String] = new Param[String](this, "probabilityCol", "Column name for predicted class conditional probabilities. Note: Not all models output well-calibrated probability estimates! These probabilities should be treated as confidences, not precise probabilities")

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41911/
Test PASSed.

weight vs. sampleWeight. I don't think sampleWeight solves the problem. I would rather change weights in linear models to coefficients instead.
Using RDD[(Double, Double, Vector)] without weights definitely introduces storage overhead, especially when the number of nonzeros per instance is small and the number of instances is very large. This happens in https://issues.apache.org/jira/browse/SPARK-10354. However, I would prefer merging the current version and test more. If we have billions of instances, the number of features are usually not that small.

@dbtsai I made one pass:)

I will prefer to change the weights in linear models to coefficients as well!

Merged build triggered.

Merged build started.

Test build #42403 has started for   PR 7884 at commit 7bcf592.

Merged build triggered.

Merged build started.

Test build #42404 has started for   PR 7884 at commit ad558e2.

Test build #42403 has finished for   PR 7884 at commit 7bcf592.

This patch fails MiMa tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
  final val probabilityCol: Param[String] = new Param[String](this, "probabilityCol", "Column name for predicted class conditional probabilities. Note: Not all models output well-calibrated probability estimates! These probabilities should be treated as confidences, not precise probabilities")

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42403/
Test FAILed.

Merged build finished. Test FAILed.

Test build #42404 has finished for   PR 7884 at commit ad558e2.

This patch fails MiMa tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
  final val probabilityCol: Param[String] = new Param[String](this, "probabilityCol", "Column name for predicted class conditional probabilities. Note: Not all models output well-calibrated probability estimates! These probabilities should be treated as confidences, not precise probabilities")

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42404/
Test FAILed.

Merged build triggered.

Merged build started.

Test build #42408 has started for   PR 7884 at commit 7f51d9b.

Test build #42408 has finished for   PR 7884 at commit 7f51d9b.

This patch fails Scala style tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
  final val probabilityCol: Param[String] = new Param[String](this, "probabilityCol", "Column name for predicted class conditional probabilities. Note: Not all models output well-calibrated probability estimates! These probabilities should be treated as confidences, not precise probabilities")

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42408/
Test FAILed.

Merged build triggered.

Merged build started.

Test build #42411 has started for   PR 7884 at commit 9acb9c9.

Test build #42411 has finished for   PR 7884 at commit 9acb9c9.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
  final val probabilityCol: Param[String] = new Param[String](this, "probabilityCol", "Column name for predicted class conditional probabilities. Note: Not all models output well-calibrated probability estimates! These probabilities should be treated as confidences, not precise probabilities")

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42411/
Test PASSed.

+1 for "coefficients"

I'll make another pass

FYI, I made https://issues.apache.org/jira/browse/SPARK-10592 for deprecating weights.

Can you please update the Scala doc to clarify:

count ignores instance weights
Add warning in training summary that it ignores the training weights currently (except for the objective trace).

Other than those small items, this looks good.

@jkbradley For count, let's do it later since it's a private API. Currently, count will return the actual number of instances, and ignores instance weights, but numNonzeros will return the weighted # of nonzeros. Let's decide the behavior of them, and modify them accordingly.

Merged build triggered.

Merged build started.

JIRA for the behavior of count and numNonZeors when we make it public. https://issues.apache.org/jira/browse/SPARK-10597

Test build #42439 has started for   PR 7884 at commit 55a28b9.

Test build #42439 has finished for   PR 7884 at commit 55a28b9.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
  final val probabilityCol: Param[String] = new Param[String](this, "probabilityCol", "Column name for predicted class conditional probabilities. Note: Not all models output well-calibrated probability estimates! These probabilities should be treated as confidences, not precise probabilities")

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42439/
Test PASSed.

Merged build triggered.

Merged build started.

Test build #42471 has started for   PR 7884 at commit a66833b.

Test build #42471 has finished for   PR 7884 at commit a66833b.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
  final val probabilityCol: Param[String] = new Param[String](this, "probabilityCol", "Column name for predicted class conditional probabilities. Note: Not all models output well-calibrated probability estimates! These probabilities should be treated as confidences, not precise probabilities")

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42471/
Test PASSed.

LGTM but there are some merge conflicts in MimaExcludes. Could you fix them? Thanks!

Merged build triggered.

Merged build started.

Test build #42497 has started for   PR 7884 at commit 8d6de99.

Test build #42497 has finished for   PR 7884 at commit 8d6de99.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
  final val probabilityCol: Param[String] = new Param[String](this, "probabilityCol", "Column name for predicted class conditional probabilities. Note: Not all models output well-calibrated probability estimates! These probabilities should be treated as confidences, not precise probabilities")

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42497/
Test FAILed.

Merged build triggered.

Merged build started.

Test build #42501 has started for   PR 7884 at commit e792b6a.

Merged build triggered.

Merged build started.

Test build #42502 has started for   PR 7884 at commit f53436a.

Test build #42502 has finished for   PR 7884 at commit f53436a.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
  final val probabilityCol: Param[String] = new Param[String](this, "probabilityCol", "Column name for predicted class conditional probabilities. Note: Not all models output well-calibrated probability estimates! These probabilities should be treated as confidences, not precise probabilities")

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42502/
Test FAILed.

Test build #42501 has finished for   PR 7884 at commit e792b6a.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
  final val probabilityCol: Param[String] = new Param[String](this, "probabilityCol", "Column name for predicted class conditional probabilities. Note: Not all models output well-calibrated probability estimates! These probabilities should be treated as confidences, not precise probabilities")

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42501/
Test FAILed.

Merged build finished. Test FAILed.

Two failed tests are from core and not related to this PR:

Unpersisting TorrentBroadcast on executors and driver in distributed mode *** FAILED *** (11 seconds, 462 milliseconds)
mesos supports killing and limiting executors *** FAILED *** (29 milliseconds)

I'm going to merge this.

Merged into master. Thanks!


Confusion on installing node packages

Hello, I was surprised that no default nodes were included in the default install:

The documentation only says this about how to install the node packages, but it isn't clear enough exactly what I should do to install them, and this may confuse other users:

Thanks, I love the idea behind this project!

Mr. Guillen,
Thank you for taking the time to give us feedback regarding Nodezator's shortcomings, it is much appreciated.
About your difficulty in finding info on how to load nodes
I'm sorry that you had to experience such inconvenience.
Your experience made it clear that we need to make some crucial resources more easy to find. For instance, we have a section in the online manual dedicated to explaining how to load nodes, and another one explaining how nodes are defined and organized into packs so they can be loaded, but it is my fault that none of those resources were shown in the form to load nodes inside the app, nor were they linked in the section of the README file that you showed us.
As you pointed out, this ends up demanding extra effort to look for the information somewhere else. The entire online manual, including the chapters linked are available in-app as well, in the menubar "Help" > "Read manual", so there's even less excuses not to make this information more discoverable.
I'll make sure such resources are very easy to find, both in the form inside the app used to load nodes and in the README file. It should not take much time, but It'll have to wait a few days or a week until I can squeeze it in between my current tasks. I'll get back to you on that when I'm finished.
About Nodezator not including node packs
Nodezator indeed doesn't include node packs in its installation. However, it comes with a lot of nodes by default already, as one can see by creating a new file and right-clicking the canvas (a popup menu appears showing a lot of different nodes that can be used).
When I created Nodezator, I wanted it to be an app that didn't force people to use specific nodes. Instead, I wanted to give people maximum freedom to create their own nodes and only load the nodes (packs) they wanted for each file.
While this gives people a lot of freedom, I do understand that this may cause Nodezator to appear a bit bare at first.
However, it is not as though this is set in stone. I'm not opposed to include a node pack or two with the default installation. Even so, wanting to include them presents its own challenges, all related to creating and choosing the right node packs for the job:

Nodezator was released less than a year ago and multi-purpose node editors aren't so popular with developers yet, so it has a small community (though thankfully it is growing). Many people still hadn't time to develop node packs they are willing to share and some are happy to use them in their own personal workflow without having to worry about publishing and maintaining them.

Apps like Nodezator make it easy to define new nodes, but designing them in the first place is the real challenge. That is, designing modular functions that go well together for a particular purpose still takes some time, effort and experimentation with real problems. Even the node packs available for download in https://gallery.nodezator.com, despite their simplicity, took some time and effort to think about real problems, which kind of operations would help solve them, and translating this all into functions to use as nodes; despite being fun work, it still takes time; and even so, those packs are far from complete; they serve as very neat examples and are useful on their own to solve some specific problems for which they were designed, but they are only, as is shown by their names, example packs.

Choosing the node packs themselves is also a challenge:  Nodezator is a multi-purpose node editor, so any node packs for any field can be created/chosen: image editing, data visualization, machine learning, text processing, you name it; based on what criteria will we choose? I mean, we could vote, but even so, won't we be tempted to add additional packs, and maybe still want to add more in the future?

Please, don't feel discouraged by the challenges I listed, they are only honest concerns, nothing more. Additionally, as I said, I'm not opposed myself to the idea of adding one or more node packs to the default installation. The prospect is quite interesting actually.
My point is just that it is too soon to think of this. I believe it is better to wait a bit more for more people to start using Nodezator, grow used to it and develop their own node packs naturally, and once we have a good number of node packs available we may start to think about picking the most used ones or design our own custom-made packs to include in Nodezator.
If after what I explained you still think we could try something soon, let me know and we can consult with the rest of the community about it, either here on github or on the discord server.
Turning this issue into a discussion
I'll also take the liberty of converting this issue into a discussion, but don't worry about it, it doesn't mean your request is any less important. It is just that in Nodezator and other Indie Python repos we use issues for things that crash the app or otherwise prevent the user from doing something that is supposed to be available. When such an issue appears, any other work is paused and all attention is given to that issue so that it can be fixed ASAP.
This measure is taken for the benefit of the users, including yourself: by doing things this way, whenever you have a real issue, it won't compete for space with other less urgent matters. Suggestions of improvements and feedback about bad experiences like this post of yours are as important as an issue, but they don't have the same urgency, which is why we turn them into discussions.

Thank you again, Mr. Guillen, for taking the time to give us feedback. Sorry for any inconvenience you may have had while using Nodezator. My goal is to work as hard as healthily possible to ensure Nodezator meets the needs of our users.
Let me know if you need anything else.
Peace.

Last but not least, thank you for your kind words in the top message:

Thanks, I love the idea behind this project!


CDI 2.0 for WildFly 12

Adapt Wildfly 12 for Weld 3.
Implement new method in WeldSecurityServices. Enable Elytron, use PrivilegedActions.
As per WELD-1914, we need other modules to be able to use Weld API module. Hence we export it from the core module.

We ran TCK tests of Weld 3 against CDI 1.2 TCK, the results can be seen here. There should only be one failure which is expected (and will appear in official TCK exclude list soon, based on CDITCK-497 and respectively CDI-545).

@jmesnil the Linux-elytron CI build failed due to build timeout but there are no visible failures. I can see that the Linux build took even longer and did not timeout. Could you please look the timeout settings and re-run that CI?

retest this please

@manovotn The elytron run timed out again. Have you tried to run it locally (using -Delytron) to confirm it passes for you?

@jmesnil not the complete test suite (e.g. without -DallTests) but otherwise I did. Is there a doc on what switches/settings do I need to run complete suit? I can try that too if needed.
Although checking on the TC log, I cannot see any test failure there. Dunno what takes so long there.

@jmesnil I ran the TS locally using mvn clean verify -DallTests -DfailIfNoTests=false -Dorg.jboss.model.test.cache.strict=true -Djboss.test.transformers.eap -Delytron (copied over from how the CI job runs).
The build and all tests passed.

retest this please

The suite now ran in time (91 min) but one test (marked as Flaky by TeamCity) failed with SocketException. The very same test succeeded in previous runs - unstable test/networking problems during execution perhaps?

retest this please

@stuartwdouglas sorry for the delay, I am currently on PTO. Rebased and replaced the util class with lambdas.
@mkouba thanks for answering the queries :)


Parse txt and xml

Info: .txt will be handled separately in a different PR later on.

I'm just not 100% sure if we shouldn't lock the test urls to newer version instead of unlocking it.
But honestly probably better to unlock since then we'd get the possible problems first instead of fiori apps.

I'm just not 100% sure if we shouldn't lock the test urls to newer version instead of unlocking it. But honestly probably better to unlock since then we'd get the possible problems first instead of fiori apps.

I just checked and seems that adding the fiori theme to the old url fixes the issue.
But in general i agree with you. Rather we facing new issues before the user facing them.
Still I would like to open up for discussion and handle that in a different PR then.


Which data is multiple images input?

Hi, which part data used in training provide multiple image inputs, and how does the image context learning achieved?

mmc4 dataset have multiple images. The in context learning is mainly achieved by mmc4 dataset

hi, do u have a preprocessed mmc4 dataset to download? like llava style format.

mmc4 cannot be changed to llava format. It is not QA. Please follow https://github.com/Efficient-Large-Model/VILA/tree/main/data_prepare and our code supports training from it.

close for now, feel free to re-open if necessary.

Hi there,
Our team has moved our VILA to a new location: NVLabs/VILA. We kindly request that you update your references to the old repository location with the new one (e.g, git clone / pipeline integration). This will ensure that you have access to the latest updates, improvements, and bug fixes for VILA project. Current repo will be archieved in one week (Jul 10th) and made private after one month (Aug 4th).
Thank you for your understanding and continued support. If your issue still exists, feel free to open a new one in the new place. We appreciate your patience and cooperation during this transition.
Best regards,
VILA Team


Couldn't establish a new session. MercuryClient$MercuryException: status: 403

Describe the bug
Since a few days, Librespot is not working anymore and instead logs a 403 authentication error.
To Reproduce
Occurs every time, on any machine.
Screenshots/Stracktraces/Logs
Librespot log:
2023-08-24 21:54:13,782 INFO  Utils:104 - Cryptography restrictions removal not needed.
2023-08-24 21:54:13,948 INFO  ZeroconfServer:472 - Zeroconf HTTP server started successfully on port 18508!
2023-08-24 21:54:15,409 INFO  Zeroconf:500 - Announced Service{alias='librespot-java', service='spotify-connect', port=18508, text={VERSION=1.0, CPath=/, Stack=SP}, addresses=[/<IP_ADDRESS>], domain='.local', protocol='tcp', host='raspberrypi4'}.
2023-08-24 21:54:19,974 TRACE ZeroconfServer:536 - Handling request: GET /?action=getInfo&version=2.7.1 HTTP/1.1, headers: {Keep-Alive=0, Connection=keep-alive, User-Agent=Spotify/121700834 OSX_ARM64/0 (Mac14,2), Host=<IP_ADDRESS>, Accept-Encoding=gzip, Content-Type=application/x-www-form-urlencoded}
2023-08-24 21:54:22,203 TRACE ZeroconfServer:536 - Handling request: GET /?action=getInfo&version=2.7.1 HTTP/1.1, headers: {Keep-Alive=0, Connection=keep-alive, User-Agent=Spotify/121700834 OSX_ARM64/0 (Mac14,2), Host=<IP_ADDRESS>, Accept-Encoding=gzip, Content-Type=application/x-www-form-urlencoded}
2023-08-24 21:54:22,270 TRACE ZeroconfServer:536 - Handling request: POST / HTTP/1.1, headers: {Keep-Alive=0, Connection=keep-alive, User-Agent=Spotify/121700834 OSX_ARM64/0 (Mac14,2), Host=<IP_ADDRESS>, Accept-Encoding=gzip, Content-Length=863, Content-Type=application/x-www-form-urlencoded}
2023-08-24 21:54:22,507 INFO  ZeroconfServer:345 - Accepted new user from <censored>. {deviceId: b457cbd5840886003282c0bc498000bb2842e13f}
2023-08-24 21:54:22,681 DEBUG TimeProvider:90 - Loaded time offset from NTP: 0ms
2023-08-24 21:54:23,376 INFO  ApResolver:99 - Loaded aps into pool: {accesspoint=[ap-gew4.spotify.com:4070, ap-gew4.spotify.com:443, ap-gew4.spotify.com:80, ap-gae2.spotify.com:4070, ap-gew1.spotify.com:443, ap-gue1.spotify.com:80], dealer=[gew4-dealer.spotify.com:443, gae2-dealer.spotify.com:443, gew1-dealer.spotify.com:443, gue1-dealer.spotify.com:443], spclient=[gew4-spclient.spotify.com:443, gae2-spclient.spotify.com:443, gew1-spclient.spotify.com:443, gue1-spclient.spotify.com:443]}
2023-08-24 21:54:23,592 INFO  Session:140 - Created new session! {deviceId: b457cbd5840886003282c0bc498000bb2842e13f, ap: ap-gae2.spotify.com:4070, proxy: false}
2023-08-24 21:54:24,518 INFO  Session:334 - Connected successfully!
2023-08-24 21:54:24,984 TRACE Session:1413 - Session.Receiver started
2023-08-24 21:54:25,047 INFO  CacheManager:95 - There are 0 cached entries.
2023-08-24 21:54:25,058 TRACE AsyncProcessor:52 - AsyncProcessor{dealer-worker} has started
2023-08-24 21:54:25,064 TRACE AsyncProcessor:52 - AsyncProcessor{event-service-sender} has started
2023-08-24 21:54:25,066 INFO  Session:1495 - Skipping SecretBlock
2023-08-24 21:54:25,067 INFO  Session:1467 - Received LicenseVersion: 0
2023-08-24 21:54:25,068 INFO  Session:1457 - Received CountryCode: DE
2023-08-24 21:54:25,070 DEBUG TokenProvider:68 - Token expired or not suitable, requesting again. {scopes: [playlist-read], oldToken: null}
2023-08-24 21:54:25,091 TRACE MercuryClient:148 - Send Mercury request, seq: 1, uri: hm://keymaster/token/authenticated?scope=playlist-read&client_id=<censored>&device_id=b457cbd5840886003282c0bc498000bb2842e13f, method: GET
2023-08-24 21:54:25,095 TRACE Session:784 - Parsed product info: {audiobook-onboarding-completed=0, remote-control=6, publish-activity=1, audio-preview-url-template=https://p.scdn.co/mp3-preview/{id}, incognito_mode_timeout=21600, ad-formats-preroll-video=0, type=premium, ab_recently_played_feature_time_filter_threshold=com.spotify.gaia=30,driving-mode=120,spotify%3Ainternal%3Astartpage=30, license-agreements=, publish-playlist=0, buffering-strategy=0, expiry=1, ab-desktop-hide-follow=0, ab-collection-hide-unavailable-albums=0, capper-profile=, payments-initial-campaign=default, profile-image-upload=1, enable-annotations=2, ab-ad-player-targeting=1, india-experience=0, video-wifi-initial-bitrate=800000, ab-mobile-running-onlymanualmode=only-manual, playlist-annotations-markup=0, prefetch-keys=1, allow-override-internal-prefs=0, prefetch-window-max=2, send-email=0, ad-formats-video-takeover=1, ab-collection-bookmark-model=1, pause-after=0, local-files-import=0, video-manifest-url=https://spclient.wg.spotify.com/manifests/v6/{type}/sources/{source_id}/options/supports_drm, ab-sugarpills-sanity-check=0, playlist-folders=1, arsenal_country=1, track-cap=0, lock-filter-explicit-content=0, audiobook-onboarding-dismissed=0, ab-mobile-running-tempo-detection=Control, is_email_verified=1, shows-collection=1, enable-annotations-read=0, addon-hifi=0, mobile-browse=0, payments-locked-state=0, employee-free-opt-in=0, shows-collection-jam=1, ab-browse-music-tuesday=1, offline=1, streaming=1, fb-info-confirmation=control, audio-quality=1, wanted-licenses=, lastfm-session=, ab-desktop-playlist-annotation-edit=1, ab-mobile-discover=0, widevine-license-url=https://spclient.wg.spotify.com/widevine-license/v1/video/license, key-caching-max-count=10000, hifi-eligible=0, ad-session-persistence=1, video-initial-bitrate=200000, ab-play-history=0, filter-explicit-content=0, mobile-payment=0, payment-state=, key-caching-max-offline-seconds=1800, ugc-abuse-report-url=https://support.spotify.com/abuse/?uri={uri}, shuffle-algorithm=1, use-pl3=0, image-url=https://i.scdn.co/image/{file_id}, use-playlist-app=0, enable-autostart=1, metadata-link-lookup-modes=0, shuffle=0, instant-search=0, instant-search-expand-sidebar=0, use-playlist-uris=0, user-profile-show-invitation-codes=0, ab-ad-requester=1, nft-disabled=1, ab-collection-union=1, key-memory-cache-mode=1:15,300, libspotify=1, high-bitrate=1, head-file-caching=1, radio=1, key-caching-auto-offline=0, explicit-content=1, video-cdn-sampling=1, ab-android-push-notifications=1, unrestricted=1, use-fb-publish-backend=2, app-developer=0, buffering-strategy-parameters=0.8:0.2:0.0:0.0:0.0:0.0:1.0:10:10:2000:10000:10485760, enable-gapless=1, ab-playlist-extender=5, enable-crossfade=1, ad-persist-reward-time=0, public-toplist=0, network-operator-premium-activation=1, video-device-blacklisted=0, collection=1, fb-grant-permission-local-render=0, backend-advised-bitrate=0, autoplay=0, catalogue=premium, storage-size-config=10240,90,500,3, ads=0, head-files-url=https://heads-ak-spotify-com.akamaized.net/head/{file_id}, ab-collection-followed-artists-only=0, browse-overview-enabled=1, on-demand=1, ap-resolve-pods=1, name=Spotify Premium, sidebar-navigation-enabled=0, loudness-levels=1:-5.0,0.0,3.0:-2.0, mobile-login=1, hifi-optin-intent=0, preferred-locale=en, license-acceptance-grace-days=30, prefetch-strategy=18, ugc-abuse-report=1, ab-watch-now=0, financial-product=pr:premium,tc:0,rt:v2_DE_default_new-family-sub-1m_0_EUR_default, ab-collection-offline-mode=0, streaming-rules=, head-files=1, capping-bar-threshold=3601, ab-new-share-flow=0, video-keyframe-url=http://keyframes-fa.cdn.spotify.com/keyframes/v1/sources/{source_id}/keyframe/heights/{height}/timestamps/{timestamp_ms}.jpg, mobile=1, is_maybe_in_social_session=0, ab-mobile-startpage=0, ab-moments-experience=0, ab-mobile-social-feed=1, ab-test-group=831, player-license=premium}
2023-08-24 21:54:25,096 INFO  Session:1495 - Skipping UnknownData_AllZeros
2023-08-24 21:54:25,097 INFO  Session:1495 - Skipping LegacyWelcome
2023-08-24 21:54:25,149 TRACE MercuryClient:191 - Handling packet, cmd: MercuryEvent, seq: -1577529805509754880, flags: 1, parts: 1
2023-08-24 21:54:25,155 DEBUG MercuryClient:226 - Couldn't dispatch Mercury event {seq: -1577529805509754880, uri: hm://pusher/v1/connections/<censored>%3D%3D, code: 200, payload: []}
2023-08-24 21:54:25,392 DEBUG Session:1471 - Received 0x10: <censored>
2023-08-24 21:54:25,596 TRACE MercuryClient:191 - Handling packet, cmd: MercuryReq, seq: 1, flags: 1, parts: 2
2023-08-24 21:54:25,598 ERROR ZeroconfServer:375 - Couldn't establish a new session.
xyz.gianlu.librespot.mercury.MercuryClient$MercuryException: status: 403
	at xyz.gianlu.librespot.mercury.MercuryClient.sendSync(MercuryClient.java:107) ~[librespot-player-1.6.3.jar:1.6.3]
	at xyz.gianlu.librespot.core.TokenProvider.getToken(TokenProvider.java:69) ~[librespot-player-1.6.3.jar:1.6.3]
	at xyz.gianlu.librespot.core.TokenProvider.get(TokenProvider.java:80) ~[librespot-player-1.6.3.jar:1.6.3]
	at xyz.gianlu.librespot.dealer.DealerClient.connect(DealerClient.java:78) ~[librespot-player-1.6.3.jar:1.6.3]
	at xyz.gianlu.librespot.core.Session.authenticate(Session.java:362) ~[librespot-player-1.6.3.jar:1.6.3]
	at xyz.gianlu.librespot.core.Session.access$600(Session.java:77) ~[librespot-player-1.6.3.jar:1.6.3]
	at xyz.gianlu.librespot.core.Session$Builder.create(Session.java:1057) ~[librespot-player-1.6.3.jar:1.6.3]
	at xyz.gianlu.librespot.ZeroconfServer.handleAddUser(ZeroconfServer.java:367) ~[librespot-player-1.6.3.jar:1.6.3]
	at xyz.gianlu.librespot.ZeroconfServer.access$200(ZeroconfServer.java:56) ~[librespot-player-1.6.3.jar:1.6.3]
	at xyz.gianlu.librespot.ZeroconfServer$HttpRunner.handleRequest(ZeroconfServer.java:499) ~[librespot-player-1.6.3.jar:1.6.3]
	at xyz.gianlu.librespot.ZeroconfServer$HttpRunner.handle(ZeroconfServer.java:574) ~[librespot-player-1.6.3.jar:1.6.3]
	at xyz.gianlu.librespot.ZeroconfServer$HttpRunner.lambda$run$1(ZeroconfServer.java:482) ~[librespot-player-1.6.3.jar:1.6.3]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641) [?:?]
	at java.lang.Thread.run(Thread.java:844) [?:?]
2023-08-24 21:54:25,801 INFO  Session:1422 - Skipping unknown command {cmd: 0x75, payload: 000000}

Version/Commit
Freshly re-downloaded jar of Player v1.6.3 & default config.toml created by the application

seems to have fixed itself, haven't changed anything on either the host device, librespot configuration nor the network though.

Just had the same thing again, probably only temporarily fixed itself.
What do to here?

I have the same error, on both 1.6.3 and on the dev branch. I can reproduce it 100% when I try to connect from Spotify Desktop (macOS). If I connect from an iOS or Android device, there is no problem.

The same issue seems to have a pending PR in librespot-the-rust-version --> https://github.com/librespot-org/librespot/pull/1220


CSGO: Support config reload

Enable config reload on go certificate server (#1493).
To reload the configuration send a SIGHUP signal to the certificate server process.
Supported:

Topology reload
PublicAddr/BindAddr reload
KeyConf reload
TrustStore reload
Customers reload

Not supported:

Public IA change
Graceful key rollover. I.e. CS does not check if the new keys are authenticated by any available certificate chain (or fetch a new chain to support the new keys).

This change is

Review status: all files reviewed at latest revision, 6 unresolved discussions.

go/cert_srv/setup.go, line 46 at r1 (raw file):
Previously, kormat (Stephen Shirley) wrote…
Doesn't this immediately break functionality?

It only breaks functionality for the short-time that the config is reloaded.
I think this is necessary.
Scenario:
Key of AS A is leaked and needs to be removed/replace from the customer mapping.
During the reload AS B updates its verifying key (which is completely legal).
If we do not block customers for this short amount of time, the update might be written after the new customers mapping has been deleted.
=> AS B will no longer be able to update its verifying key, until the config is reloaded again.
We could hand-off the customer mapping from oldCtx to newCtx. To do this properly, we either need more granular signaling (i.e. one for general update, one for customer mapping update) or need to check what has changed in the customers dir.
I think blocking the customers mapping for a short time is a lot more sane.

go/cert_srv/setup.go, line 85 at r1 (raw file):
Previously, kormat (Stephen Shirley) wrote…
What is the difference between conf and csctx? How do you decide what belongs to each? (and yeah, i just noticed we have the same structure in the BR :)

I thought about leaving leaving Store and TrustDB in the config.
But then I decided to decouple them.
Thinking about it, there is no clear distinguishing element. I guess it makes sense to leave them inside config.
My reasoning was, that Store and TrustDB are constantly evolving, but config should be more static.
(But by that logic, Customers, singer and verifier should be part of the context as well).
Also, overloading ctx seems bad.
I will change it back, if you agree.

Comments from Reviewable

Review status: 1 of 8 files reviewed at latest revision, 6 unresolved discussions.

go/cert_srv/io.go, line 51 at r1 (raw file):
Previously, kormat (Stephen Shirley) wrote…
As both of these channels will never be written to, they can be unbuffered.

Done.

go/cert_srv/io.go, line 54 at r1 (raw file):
Previously, kormat (Stephen Shirley) wrote…
This is the zero value, so it doesn't need to be specified.

Done.

go/cert_srv/setup.go, line 45 at r1 (raw file):
Previously, kormat (Stephen Shirley) wrote…
Why close the old one? The trustdb is not something that should change on a reload.

Done.

go/cert_srv/setup.go, line 46 at r1 (raw file):
Previously, Oncilla wrote…
It only breaks functionality for the short-time that the config is reloaded.
I think this is necessary.
Scenario:
Key of AS A is leaked and needs to be removed/replace from the customer mapping.
During the reload AS B updates its verifying key (which is completely legal).
If we do not block customers for this short amount of time, the update might be written after the new customers mapping has been deleted.
=> AS B will no longer be able to update its verifying key, until the config is reloaded again.
We could hand-off the customer mapping from oldCtx to newCtx. To do this properly, we either need more granular signaling (i.e. one for general update, one for customer mapping update) or need to check what has changed in the customers dir.
I think blocking the customers mapping for a short time is a lot more sane.

During refactoring, the time period where customers is blocked is reduced heavily.

go/cert_srv/setup.go, line 85 at r1 (raw file):
Previously, Oncilla wrote…
I thought about leaving leaving Store and TrustDB in the config.
But then I decided to decouple them.
Thinking about it, there is no clear distinguishing element. I guess it makes sense to leave them inside config.
My reasoning was, that Store and TrustDB are constantly evolving, but config should be more static.
(But by that logic, Customers, singer and verifier should be part of the context as well).
Also, overloading ctx seems bad.
I will change it back, if you agree.

I merged conf and ctx.

go/cert_srv/conf/customers.go, line 53 at r1 (raw file):
Previously, kormat (Stephen Shirley) wrote…
Line length.

Done.

Comments from Reviewable

Reviewed 7 of 8 files at r2.
Review status: all files reviewed at latest revision, 1 unresolved discussion.

go/cert_srv/setup.go, line 46 at r1 (raw file):
Previously, Oncilla wrote…
During refactoring, the time period where customers is blocked is reduced heavily.

Suggestion - just hand over Customers instance from old to new configs.

go/cert_srv/conf/conf.go, line 61 at r2 (raw file):

	Customers *Customers
	// ID is the element ID.
	ID string

This should be beside Topo. Probably above.

go/cert_srv/conf/conf.go, line 78 at r2 (raw file):

}

var conf atomic.Value

This shouldn't be between the type definition and its constructors/methods. I suggest moving it below.

Comments from Reviewable

Review status: 6 of 8 files reviewed at latest revision, 3 unresolved discussions.

go/cert_srv/setup.go, line 46 at r1 (raw file):
Previously, kormat (Stephen Shirley) wrote…
Suggestion - just hand over Customers instance from old to new configs.

Done.

go/cert_srv/conf/conf.go, line 61 at r2 (raw file):
Previously, kormat (Stephen Shirley) wrote…
This should be beside Topo. Probably above.

Done.

go/cert_srv/conf/conf.go, line 78 at r2 (raw file):
Previously, kormat (Stephen Shirley) wrote…
This shouldn't be between the type definition and its constructors/methods. I suggest moving it below.

Did not find suitable spot. To the bottom it goes.
Done.

Comments from Reviewable

Reviewed 2 of 2 files at r3.
Review status: all files reviewed at latest revision, 1 unresolved discussion.

Comments from Reviewable


Incorrect parameter type for validation concerns

Laravel Version: v9.10.1
PHP Version: v8.1
Database Driver & Version: Irrelevant

Description:
Given this custom validator code that was working fine before:
private const MAX_SIZE_IN_KB = 5 * 1024; // 5MB

public function validate($attributes, $value, $parameters, Validator $validator): bool
{
    return $validator->validateMax($attributes, $value, [self::MAX_SIZE_IN_KB]);
}

The change introduced 2e2e57c3670b9f09711252e2793a1734290cc54a (PR #42151) now causes PHPStan (or Psalm, for that matter) to report a type error:
 ------ -------------------------------------------------------------------
  Line   /Dir/CustomFileSizeValidator. php
 ------ -------------------------------------------------------------------
  28     Parameter #3 $parameters of method
         Illuminate\Validation\Validator::validateMax() expects array<int,
         string>, array<int, int> given.
 ------ -------------------------------------------------------------------

A workaround could be to convert self::MAX_SIZE_IN_KB to string, but I believe this should be fixed from the cause — maybe to use array<int, int|string> instead of array<int, string>.
The example above is for validateMax, but the problem may happen for other methods found in the ValidatesAttributes trait as well.
Steps To Reproduce:
N/A.

We don't consider PHPStan or Psalm errors as actual bugs. Feel free to PR something to fix anything you have in mind 👍

@driesvints Sure, I can send a PR over. I'd argue though, that this is an actual bug (documented type and actual support type don't match). What static analyzers do in this case is to make it visible.

No code will break because of this.

It doesn't take breaking code to be a bug IMHO, but I get where you're coming from. I'll see what I can do.


Problems with § in password

Hello,
we have a password given from a customer with a char for paragraph --> §
When we try to connect, we get the issue: login or passwort incorrect. i test it on your ftp server with the password and i have the same error. if i delete § from passwort in our programm and on our ftp-server it works. when i test the original password with filezilla it works too.
This is a sample of the password: kjvU§hhK785f4DiM
What can i do? the customer did not change the password.
thx
Bastian

Try one of these:
https://github.com/robinrodricks/FluentFTP#faq_utf