accelerator.gather silently fails if any item in the list is not a tensor

System Info
main

Reproduction
$ cat test.py
import torch
from accelerate import Accelerator

accelerator = Accelerator()
process_tensor = torch.tensor([accelerator.process_index], device="cuda")
gathered_tensor, i = accelerator.gather((process_tensor, 1))

run:
$ python -m torch.distributed.run --nproc_per_node=2 test.py
WARNING:__main__:
*****************************************
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
*****************************************
Traceback (most recent call last):
Traceback (most recent call last):
  File "test.py", line 6, in <module>
  File "test.py", line 6, in <module>
        gathered_tensor, i = accelerator.gather((process_tensor, 1))gathered_tensor, i = accelerator.gather((process_tensor, 1))

ValueErrorValueError: : not enough values to unpack (expected 2, got 0)not enough values to unpack (expected 2, got 0)

Expected behavior
accelerator.gather doesn't validating the incoming argument and fails with a misleading error that doesn't tell the user what's wrong.
Expecting for it to tell the user that one or more items on the list aren't tensors and not return an empty list.
Please note that it works fine on a single gpu, so the test needs to be 2 gpus or more.
Thank you.

The error is there, but it looks like torchrun (or torch.distributed.run) ignores TypeErrors... I'll change the error to a ValueError, which will provide the (more helpful) error message
ValueError: Can't apply _gpu_gather_one on object of type <class 'int'>, only of nested list/tuple/dicts of objects that satisfy is_torch_tensor.

That works, if you think the user can connect the dots as this is not the function the user calls. But hopefully they will figure it out from the traceback.
I think checking the inputs are all tensors is probably expensive, since it's the same function called again and again and this overhead will add up.
So your proposal works for me.
Thank you, Sylvain


Creating a foliage mesh ?

Hey there :)
I've been checking the examples provided by the plugin but wasn't able to find any mention of 3d foliage, and how to create them from a given static mesh.
Are they supported in the plugin yet ?
Thanks !

I would be actually interested as well, i just had the feeling that Unreal didn't ever expose any functionalities. (I might be wrong). Anyhow, it would be very useful if we generate foliage in a 3rd Party Tool and import it.

Hi, i have implemented a preliminary api: https://github.com/20tab/UnrealEnginePython/blob/master/docs/Foliage_API.md

Very nice! Thanks a lot! Do you know, when the next binaries will include those changes?

Thank you so much for the quick reply ! Really can't wait to test this one !

Binary releases are available: https://github.com/20tab/UnrealEnginePython/releases/tag/20171229


fix(api-gateway-v2): remove duplicate cookies for function URLs

Function URLs get cookies twice, in headers and the cookies array, which was causing them to duplicate. This will overwrite rather than duplicate.
Ref: https://github.com/awslabs/aws-lambda-rust-runtime/pull/586
Function URLs use the same apigatewayv2 payload otherwise.

Today function my remix app behind a function URL broke for me suddenly. It seems to me that AWS started escaping the rawQueryString. I don't know why they'd suddenly change stuff like that.
I added another commit here just for visibility; I am not sure the cookie one is needed anymore (but doesn't hurt).


Improvements to the FLAG_FAIL_FAST option

Currently the centrifugation script has a hardcoded constant called FLAG_FAIL_FAST which defaults to being set to False, but when it's set to true it will propagate errors.
As seen here: https://github.com/ooni/pipeline/pull/177#discussion_r279317109, the fact that exceptions are not obviously thrown can lead to bugs creeping up.
Moreover it's probably beneficial to at least run the CI tests with this flag on.
I think at bare minimum it should be possible to enable this flag as an argument to centrifugation, but maybe we could also revisit this pattern.
Here are some comments from @darkk which may be of help to inform a decision:

Yes, garbage-in-garbage-out.
That's done that way so the malformed measurement is still indexed and marked with exc and does not block the pipeline.
You may enable FLAG_FAIL_FAST to avoid exceptions suppression.

Storing exception as 32bit hashsum may sound like a strange idea, that's done to prevent database bloat in a pathological case as 1M of traces is not something trivial to deal with.
That idea is to use the hashsum to triage "failures" and pay attention to "popular" failures first.

FTR, the hash is computed by exc_hash().

We aren't using this pipeline anymore. Closing.


Verification missing

Why is https://github.com/efabless/caravel_user_project/tree/gfmpw-0d/verilog/dv empty?
I couldn't figure out a way to populate and use it for verification.

@mkkassem

Even after populating the dv directory, it seems that the management wrapper is referencing a caravel_power_routing which does not exist (anymore?).


compaction: improve compaction trigger

Is your feature request related to a problem? Please describe.
Currently we have the following compaction triggers:

On commit_epoch. It happens when the checkpoint process completes.
On report_compact_task. It happens when there is a status change in a compaction task.
On timer. It triggers every periodic_compaction_interval_sec.

This is sub-optimal because:

report_compact_task can happen when a compaction task is cancelled due to "no idle compactor". It is meaningless to try generating a compaction task and triggering the compaction repeatedly before compactors become available to accept compaction tasks.
Same as 1). It is meaningless to generate a compaction task on a timer trigger when compactors are not available to take tasks.
When a new compactor is added to the cluster, it needs to wait for the above triggers before it can take compaction work even though the LSM tree is already imbalance.

Describe the solution you'd like
Modify the compaction triggers to be:

On commit_epoch
On compaction task status changing from PENDING to SUCCESS/FAILED
On compactor availability changes.

The timer trigger can still be kept as a safeguard but it is not required.
Furthermore, we can unify get_compact_task and assign_compaction_task to avoid the overhead of cancelling the unassigned but generated compaction task when there is no available compactor.
Describe alternatives you've considered
No response
Additional context

On compactor availability changes.

Since we only limit number of ongoing compaction task on a compactor, for now, compactor availability changes simply mean an existing compactor finish a compaction task or a new compactor is added into the cluster.
In the future, if we adopt a more granular indicator for compactor load other than number of ongoing compaction task to limit compaction assignment, we also need to consider those indicators to see whether there is availability changes in compactor. For example, we may need to check whether we need to trigger a compaction on receiving the compaction task progress report.

Also, triggering compaction on report_compact_task can quickly flood meta log with hundreds of thousands of lines of "no idle compactor", since the wait is removed after #5240.

On compaction task status changing from PENDING to SUCCESS/FAILED

We should also consider CANCELED, because after #4496, canceled tasks maybe already assigned, and we should use compact_task_assignment to check it.

I'm also concerned about the fairness among compaction groups because when a task is completed, the next compaction task is triggered on the same group of that task. So it's possible that all the compactors will just be filled with tasks of one group, while another group seldom gets an available compactor because it can only request compaction on commit_epoch or timeout.
Currently fairness is kind of achieved, because compaction groups whose tasks get canceled due to "no idle compactor" will retry like crazy.

I'm not sure if the new policy will cause a fairness problem among compaction groups because when a task is completed, the next compaction task is triggered on the same group of that task.

Good point. Only triggering compaction on one group when a compactor becomes available to take more tasks is problematic. I feel like we need a schedule state to mark whether the scheduling should pause or resume. When we have no available compactors, schedule state is changed from NORMAL to PAUSED and compaction request in the channel will not be polled. After compactor becomes available, the schedule state is changed from PAUSED to NORMAL and compaction request will be polled in order.
Note that this logic is independent of try_send_compaction_request, meaning that is okay to only call try_send_compaction_request on the same group of the task when a task is completed.

Yes, the request channel can act as the queue. In that case, even if the task is canceled, it should still call try_send_compaction_request so that when compactors become available, the canceled group will be scheduled.

Please withhold this refactoring, especially the CompactionRequestChannel. Because the compaction testing tool which I am working on relies on existing mechanism. In short, I will use a mock channel to swallow all compaction requests in the phase of generating L0 SSTs. After that, I will use a dedicated channel between the testing tool and the CompactionScheduler to trigger compaction in a deterministic way. cc @zwang28

Please withhold this refactoring, especially the CompactionRequestChannel. Because the compaction testing tool which I am working on relies on existing mechanism. In short, I will use a mock channel to swallow all compaction requests in the phase of generating L0 SSTs. After that, I will use a dedicated channel between the testing tool and the CompactionScheduler to trigger compaction in a deterministic way. cc @zwang28

I don't think the aforementioned change involves any code change to CompactionRequestChannel. What will change is when the CompactionScheduler` chooses to poll that channel. Does your tool rely on some assumptions related to that?

It is not easy to discuss the changes I have made to the compaction scheduler here without code. I will submit a draft PR on Monday, then you can check it and see whether those changes would affect your decisions or not.

so this issue has already been completely resolved by #5405 , hasn't it?

so this issue has already been completely resolved by #5405 , hasn't it?

No, https://github.com/risingwavelabs/risingwave/pull/5405 is a quick fix to avoid huge log.

so this issue has already been completely resolved by #5405 , hasn't it?

No, #5405 is a quick fix to avoid huge log.

Which sub-optimal point remains after #5405 ?

If wait too long, compactor will not be loaded with the maximum number of tasks. If wait too short, meta node will pick tasks that will be canceled later.

If wait too long, compactor will not be loaded with the maximum number of tasks. If wait too short, meta node will pick tasks that will be canceled later.

thank you! I was incorrect.
What if replacing #5405 with waiting until compactor available?

What if replacing https://github.com/risingwavelabs/risingwave/pull/5405 with waiting until compactor available?

I think that's the ideal scenario, and the schedule state is just a way to implement "waiting until compactor available".

Actually I just took a look at the code and though about a simpler method: pick compactor then pick task. Then we can simply wait a short time (probably 1 or 2s). Because compactor picking is a relatively light-weight task (the only heavy thing about it is the compaction read lock), even if there's no idle compactor, there won't be a lot of wasted work.

Alternatively, if we revert the scheduler to pick(and hold) task and wait until available compactor, then we don't need to worry about unnecessary cancellation due to NoAvailableCompactor.
The only downside is when there is finally an available compactor, it could have been assigned a better task than the previously picked one. But I think this doesn't hurt overall performance.

otherwise, we may pick the same compactor for 100 times then assign 100 tasks to it

Currently there is only one thread "compaction scheduler" that will pick/assign/send. So I think this won't be a problem?

otherwise, we may pick the same compactor for 100 times then assign 100 tasks to it

Currently there is only one thread "compaction scheduler" that will pick/assign/send. So I think this won't be a problem?

Thanks for pointing out. Agree.

Actually I just took a look at the code and came up with a simpler (in terms of implementation) method: pick compactor then pick task. Then we can simply wait a short time (probably 1 or 2s). Because compactor picking is a relatively light-weight task (the only heavy thing about it is the compaction read lock), even if there's no idle compactor, there won't be a lot of wasted work.
A drawback of this approach is in order to make the whole pick compactor -> pick task -> assign task -> send task process atomic (otherwise, we may pick the same compactor for 100 times then assign 100 tasks to it, because "pick" is a read-only process and does not update the number of tasks assigned to that compactor), we need to include pick task into the compaction read lock, and I'm uncertain whether it's a good idea.

One more point to note, we set task pending after pick_compact_task, although make the whole process atomic, we also need to cacel the task that had been pick. Maybe we need to pay attention to the pending_task , and change the pending_status after task_assign ? But it is not very easy to modify here

As pointed out by @StrikeW, his testing tool does not modify the logic of pick_and_assign. So I will implement a schedule state-based trigger that uses some notification mechanism instead of waiting for a fixed interval.


WebDriverIO and Opera v43 : the browser does not launch

On windows 10, I'm trying to run tests on opera, using webdriverIO. My codecept.json is :
    ...
    "WebDriverIO": {
      "url": "http://www.mysite.com",
      "browser": "opera",
      "windowSize": "1152x1000",
      "show": true
      }
    ...

I'm using the latest selenium server 3.3.0 and the latest operadriver.exe 0.2.2
opera version is : 43.0.2442.1144 (PGO)
the opera browser does not even launch. (Note that IE, Chrome, Edge, Firefox correctly launch)

console output:

 > Error: The best matching driver provider org.openqa.selenium.edge.EdgeDriver can't create a new driver instance for Capabilities [{rotatable=true, marionette=false, locationContextEnabled=true, loggingPrefs=org.openqa.selenium.logging.LoggingPreferences@aa592c, browserName=opera, javascriptEnabled=true, handlesAlerts=true, requestOrigins={name=webdriverio, version=4.6.2, url=http://webdriver.io}}]
Build info: version: '3.3.0', revision: 'b526bd5', time: '2017-03-07 11:21:31 -0800'
System info: host: 'xxxxxxx', ip: 'xxx.xxx.xxx.xxx', os.name: 'Windows 10', os.arch: 'x86', os.version: '10.0', java.version: '1.8.0_121'
Driver info: driver.version: unknown

Details

CodeceptJS version: 0.5.1
NodeJS Version: 7.5.0
Operating System: Win10
WebDriverIO: 4.6.2

What do I miss ?
thanks for your help.

Please try to reproduce this issue using the webdriverio library only. It's pretty simple, just follow the instructions at http://webdriver.io
CodeceptJS is just a wrapper on top of webdriverio and I think we couldn't either break or fix this kind of things.

The opera webdriver is currently not maintained. To use the opera browser, we have to use the chrome webdriver and specify to use another binary than the default one.
I've put below a sample of my codecept.json
 "helpers": {
    "WebDriverIO": {
      "url": "http://www.mysite.com",
      "browser": "chrome",
      "windowSize": "1152x1000",
      "show": true,
      "desiredCapabilities": {
        "browsername": "opera",
        "chromeOptions": {
          "binary": "C:\\Program Files (x86)\\Opera\\43.0.2442.1144\\opera.exe"
        }
      }
    }

for OSX the binary entry may be :
"binary": "/Applications/Opera.app/Contents/MacOS/Opera"

Thanks all.


networking: change the default-restricted subnet

Previously, we were using 172.17/16, which conflicts with the default
Docker networking. Change it to 172.31/16

ok to test

CI is having some issues with this PR, I've re-submitted it as https://github.com/rkt/rkt/pull/3718.


Use new code formatting approach since ESLint deprecated all formatting rules

Since V8.53, ESLint team decided to deprecate all formatting rules. More info can be found on this official announcement.
To keep it simple, those rules will stay at least in version 9 of ESLint but will not receive any updates anymore. Those rules should be abandoned in favor of a code formatter OR replaced by the rules defined in [@stylistic/eslint-plugin-js](https://eslint.style/packages/js). This package contains all the deprecated rules and will be maintained. However, as our main dependencies (eslint-plugin-vue and eslint-config-standard) don't make use of those rules yet, it is better to wait before making the transition on our side...
This issue is derived from https://github.com/wegue-oss/wegue/pull/377 and the text above was initially provided by @sronveaux .

Just a follow-up on this one, it seems like StandardJS is kind of abandoned and that a lot of people transit to neostandard which tries to align with current StandardJS with minor changes (mainly due to plugins not yet transited to the new configuration format) which are made by the community through an open governance.
It is still in an early stage but I think we should keep an eye on it if we decide to keep aligned to StandardJS as it already makes use of the rules defined in @stylistic/eslint-plugin-js.
We should however wait for V1 to be out I think and also perhaps for Wegue to be transited to Vite if possible so eslint-plugin-vue could be removed easily.
Don't hesitate to give your opinion about this one!


Fix "@itwin/no-internal-barrel-imports" linting errors

Describe the bug
Because of this PR I created, there has been introduced a new linting error, specifically @itwin/no-internal-barrel-imports : Do not consume barrel imports within the same package. I have set the rule to "warn" for now in this base config file here so that the PR can be merged. This "warn` value should be removed and the linting errors should be fixed instead.
To Reproduce
Steps to reproduce the behavior:

Go to base of repo
Run rush lint
See all the linting problems reported as warning
Remove the line with  "@itwin/no-internal-barrel-imports": "warn", in ui/eslint.config.base.js
Fix all the linting errors

Expected behavior
No linting problems reported when running rush lint
Screenshots
Example run of rush lint:

hey, just looking at the error now.
looks like all the errors are coming from tests, and using the barrel should be ok
just suppress them in the config for all test files

hey, just looking at the error now. looks like all the errors are coming from tests, and using the barrel should be ok just suppress them in the config for all test files

Gotcha, warn or set it off entirely?

off; the rule is inapplicable here
go review the rule, can find it in the eslint plugin repo


🚸 Simplify remote SQLite synching & locking

By having only 3 points in which we synch & lock/unlock the remote SQLite file, we can simplify the code dramatically and still create a safe & performant user experience.
The one and only downside is that the user needs to manually push changes back to the cloud by calling lamin close. But given this is a known journey (git push), it shouldn't be a big deal.
The 3 points are:

init merely locks the remote sqlite file but doesn't yet upload it
load synchs changes from a remote sqlite file to the local file & locks the remote file
close updates the remote sqlite file & unlocks it

In this PR, we don't put forward the locking on init because it is heavily used in concurrent CI settings.
The really great thing here is that lamindb can be entirely free from logic around synching the cloud sqlite. It makes the code simpler and less error prone, and very performant.

So, I'm pretty happy with the user story in test-sqlite-sync notebook.
From my perspective, the only remaining thing is to have a standard naming convention for "cloud SQLite".
Given GitHub uses "remote repository" and we already talk about "remote instances", it might be more natural to consistently call things remote_sqlite rather than cloud_sqlite.
The other thing is allowing multiple lndb files in a bucket, restricting the locker just to that single lndb file.

This is the first time the _exclusion code actually gets tested, @Koncopd

https://app.codecov.io/github/laminlabs/lamindb-setup/commit/ddecc00d0a6e379d643d02030c6a270f1e4aef92
Could you think about how to get this to 100% coverage when reviewing this?


More tabs for colonists

Single screen is not enough to feel all the stats I need to see. May be add another tab for colonists as a quick workaround?

A solution to the underlying problem will be part of the planned redesign and refactor, but that'll be a while.
There won't be a hacky and confusing workaround in the meantime. It'll happen properly, even if that means that it takes some more time. I prefer doing things right the first time.


Remove LoadConfigFile on lib init

Soda already call it on RootCmd, it can be called by the user, and Connect already calls it.

You're welcome, Thank you all @stanislas-m and @mclark4386!

Thanks @sio4!

Thank you @sio4 ! Really appreciate all the help you've been giving!

@mclark4386 I didn't give a help to the others, but me and the community I belong to! Thank you so much!


chore: jsonfmt apimanagement

Ran jsonfmt -w "specification/apimanagement/**/*.json"

Can one of the admins verify this patch?

Automation for azure-sdk-for-python

Automation for azure-sdk-for-java

Automation for azure-sdk-for-js

Automation for azure-sdk-for-ruby

Automation for azure-sdk-for-go


Question about pem files

I am trying to create signed wifi mobileconfig file but I have issues with getting the .pem files. Where exactly on the apple development center can I find them? Do I need to convert .cert to pem or? I checked the guide on signing but couldn't get any further.
Any tip would be helpful
Thanks

@adelburekovic If you are running on a Mac you get them in XCode when you try to sign your first app. You can then find them in Keychain Access  with the name "Developer ID". You should know though that Apple allows any certificate that is trusted by the system to sign mobileconfig's. It may be easier to use something like Let's Encrypt to get the job done.

I’ve also used Let’s Encrypt certificates. The thing to consider though is that profile considered is valid as long as the certificate is valid and LE certs have a short life span (max 3 mo). Everything should keep working once the cert expires but checking installed profile info should start showing “unverified” or smth


[wip] network location filter

@andrewalexander you get a chance to try this out?

@kapilt Yeah it requires some integrations with ec2 still - it is not recognized as a filter, and adding it as a filter wasn't as trivial as originally expected.


HTML Editor Manager not loading saved settings

Description of bug
After saving editor settings, when revisiting the HTML Editor manager in PB the settings doesn't show up
Steps to reproduce
List the steps to reproduce the behavior:

Go to 'HTML Editor Manager'
Make some changes(e.g. select editor skins from black to moono) and save
After the save success notification refresh the whole page and go to "HTML Editor Manager" again
Changes are not made on the form(?)

Current result
It does save the settings when I check it while editing content however if i revisit the "HTML Editor Manager" again it doesn't load up the settings on the settings form which makes confusing.
Expected result
The form in "HTML Editor Manager" should load the saved settings
Error log
No error shown on Event Log/Console
Additional context
On 9.2.1 when I save settings and reload the HTML Editor Manager page again sometimes I can't get into the settings page and when editing content the editor turns black (Black skin) so not too sure if it was related with the bug #2295 before...
Affected version

[x] 9.2.2

Affected browser
I don't think it's a browser issue but I did test with the following browsers...

[x] Chrome
[x] Firefox
[ ] Safari
[ ] Internet Explorer
[x] Edge

Hi, I can verify this is an issue. It has happened to me in 9.21 and 9.2.2. Thanks for reporting this issue! I would also request that it be fixed. Thanks, Craig


Infinite scroll does not work for off-screen elements

If you imagine an instagram or twitter feed where you maybe have 10,000 pixels of elements, and you want to load more as you get to the bottom, well this doesn't work with this library. I've had a look in _distanceFromEnd and because the element is bigger than the scroll view, it automatically triggers a load. In fact, I load 20+ pages instantly without debouncing.
Is this something that might be looked at?
Cheers

I'm not spending much of my time on this library right now, if you wanted to help find a fix for the root cause or a PR I could take a look at it though.

That's okay @ide, maintaining every open source library we make can get incredibly taxing.
I managed to solve my issue by reverting back to using vanilla ListView with onEndReached.


Python int too large to convert to C long after sqllab query

When trying to query a sample dataset through sqllab, I received the error Python int too large to convert to C long. It appears the error is occuring in Superset's response processing code.
This is running commit id 040b94119 from the apache/superset git master branch.
I have the following files loaded into my Hydrolix database (interface compatible with Clickhouse):
clickhouse_types_positive_ints1.csv
clickhouse_types_positive_ints2.csv
The schema of the database describes the _uint64 column to be of type Nullable(UInt64)
I make the following query against the Hydrolix database:
SELECT _uint64  from sample_project.sample_table where _uint64 <<PHONE_NUMBER>688793473;

I receive the error Python int too large to convert to C long.
If I make a query comparing against a slightly smaller number:
SELECT _uint64  from sample_project.sample_table where _uint64 <<PHONE_NUMBER>688793472;

I get a result column of 1040 numbers and no error.
I get the following stack trace in the superset logs:
superset_app             | 2021-07-23 20:48:11,314:WARNING:superset.views.base:[SupersetError(message='Python int too large to convert to C long', error_type=<SupersetErrorType.GENERIC_DB_ENGINE_ERROR: 'GENERIC_DB_ENGINE_ERROR'>, level=<ErrorLevel.ERROR: 'error'>, extra={'engine_name': 'ClickHouse', 'issue_codes': [{'code': 1002, 'message': 'Issue 1002 - The database returned an unexpected error.'}]})]
superset_app             | Traceback (most recent call last):
superset_app             |   File "/app/superset/views/base.py", line 204, in wraps
superset_app             |     return f(self, *args, **kwargs)
superset_app             |   File "/app/superset/utils/log.py", line 241, in wrapper
superset_app             |     value = f(*args, **kwargs)
superset_app             |   File "/app/superset/views/core.py", line 2573, in sql_json
superset_app             |     return self.sql_json_exec(request.json, log_params)
superset_app             |   File "/app/superset/views/core.py", line 2774, in sql_json_exec
superset_app             |     session, rendered_query, query, expand_data, log_params
superset_app             |   File "/app/superset/views/core.py", line 2558, in _sql_json_sync
superset_app             |     [SupersetError(**params) for params in data["errors"]]
superset_app             | superset.exceptions.SupersetErrorsException: [SupersetError(message='Python int too large to convert to C long', error_type=<SupersetErrorType.GENERIC_DB_ENGINE_ERROR: 'GENERIC_DB_ENGINE_ERROR'>, level=<ErrorLevel.ERROR: 'error'>, extra={'engine
_name': 'ClickHouse', 'issue_codes': [{'code': 1002, 'message': 'Issue 1002 - The database returned an unexpected error.'}]})]

It looked like the above was swallowing an exception, so I made a small code modification to log it:
superset_app             | Traceback (most recent call last):
superset_app             |   File "/app/superset/sql_lab.py", line 459, in execute_sql_statements
superset_app             |     statement, query, user_name, session, cursor, log_params, apply_ctas,
superset_app             |   File "/app/superset/sql_lab.py", line 297, in execute_sql_statement
superset_app             |     return SupersetResultSet(data, cursor_description, db_engine_spec)
superset_app             |   File "/app/superset/result_set.py", line 109, in __init__
superset_app             |     pa_data.append(pa.array(array[column].tolist()))
superset_app             |   File "pyarrow/array.pxi", line 306, in pyarrow.lib.array
superset_app             |   File "pyarrow/array.pxi", line 39, in pyarrow.lib._sequence_to_array
superset_app             |   File "pyarrow/error.pxi", line 141, in pyarrow.lib.pyarrow_internal_check_status
superset_app             | OverflowError: Python int too large to convert to C long

For my immediate purposes I have composed this patch to avoid the error in a way that I suspect would not be portlable to different databases.
Expected results
I expect to be able to view results in Superset for a query of integers stored in my database as UInt64 that are larger than<PHONE_NUMBER>688793472.
Actual results
I see a en error, Python int too large to convert to C long, instead of results.
Screenshots

How to reproduce the bug

Go to the SQL Lab tag
Enter the query SELECT _uint64  from sample_project.sample_table where _uint64 <<PHONE_NUMBER>688793473; in the query box
click run
See error output in red

Environment
running superset via docker-compose -f docker-compose.yml up from a git checkout of version 040b94119
python and node.js versions should follow from the above info.
Checklist
Make sure to follow these steps before submitting your issue - thank you!

[x] I have checked the superset logs for python stacktraces and included it here as text if there are any.
[-] I have reproduced the issue with at least the latest released version of superset. [the issue applies to the head of the master branch]
[x] I have checked the issue tracker for the same issue and I haven't found one similar.

For the record I tried loading the same dataset into postgresql, but could not reproduce the error there with the uint64 values stored in a numeric column.

I was able to recreate the issue querying the standard clickhouse server.
I ran clickhouse from docker using docker run -d -P --name some-clickhouse-server --ulimit nofile=262144:262144 yandex/clickhouse-server
Then created a database: CREATE DATABASE IF NOT EXISTS tutorial;
Then created the simplest table I could: create table tutorial.bignum (EventTimeDateTime,bignum UInt64) ENGINE = MergeTree()　PARTITION BY toYYYYMM(EventTime)　ORDER BY (EventTime);
Then execed into the docker file ran cat | clickhouse-client -mn and loaded the data by pasting the attached sql.
clickhouse-bignum.sql.txt
After connecting Superset to the clickhouse server, I got the error by running SELECT * from tutorial.bignum in sqllab.

Reason:
>>> import pyarrow as pa
>>> pa.array([2 ** 63 - 1])
<pyarrow.lib.Int64Array object at 0x7f87edf153d0>
[
<PHONE_NUMBER>854775807
]
>>> pa.array([2 ** 63])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "pyarrow/array.pxi", line 306, in pyarrow.lib.array
  File "pyarrow/array.pxi", line 39, in pyarrow.lib._sequence_to_array
  File "pyarrow/error.pxi", line 141, in pyarrow.lib.pyarrow_internal_check_status
OverflowError: Python int too large to convert to C long
>>> pa.array([2 ** 63], type=pa.uint64())
<pyarrow.lib.UInt64Array object at 0x7f87edf15360>
[
<PHONE_NUMBER>854775808
]

This issue still exists as of 1.5.3.

Issue still exists for current version, with MySQL database.

The issue is still actual and really annoying.

For ClickHouse, you can use clickhouse-connect as the connector and it won't throw an error.  Instead, it will return the two's complement/negative value for the UInt64 column (and strings for 128/256 bit ClickHouse integer columns).  It's not a complete fix for the underlying issue -- which is the fact that Superset doesn't support UInt64 datatypes due to the PyArrow transformation, but it's workable for use cases where the UInt64 is something like a database id or the equivalent.

I'm not sure if this is still an issue in current versions of Superset (3.x). If it is, we can re-open this, or feel free to open a new issue with updated context and a reproducible case with some example data. We're no longer supporting Superset 2.x or prior, and it's been a while since this thread saw any activity, so I'm closing this as stale.


Why it does that
Briefly state why the change was necessary.
Further notes
List other areas of code that have changed but are not necessarily linked to the main feature. This could be method
signature changes, package declarations, bugs that were encountered and were fixed inline, etc.
Linked Issue(s)
Closes #205
Please be sure to take a look at the contributing guidelines and our etiquette for pull requests.

@juliapampus given that MVD has changed quite a bit in the meantime, there is probably quite a bit of rebasing work to be done. is this PR still actively worked on?


Reuse of keys in blockwise fusion can cause spurious KeyErrors on distributed cluster

Subsequent Blockwise layers are currently fused into a single layer. This reduces the number of tasks, the overhead and is very generally a good thing to do. Currently, the fused output does not generate unique key names which is a problem from a UX perspective but can also cause severe failure cases when being executed on the distributed scheduler since distributed assumes that a task key is a unique identifier for the entire task. While it is true that the data output of the fused key and the non-fused key is identical, the runspec and the local topology is intentionally very different. Specifically, a fused task, for example, may not have any dependencies while the non-fused task does have dependencies.
An example where this matters is the following (async code is not necessary but the condition is actually a bit difficult to trigger and this helps. Paste this code in a Jupyter notebook and run it a couple of times).
import asyncio
from distributed import Client, Scheduler, Worker

import dask
import time
import dask.dataframe as dd
async with (
    Scheduler() as s,
    Worker(s.address) as a,
    Worker(s.address) as b,
    Client(s.address, asynchronous=True) as c
):
    df = dask.datasets.timeseries(
        start="2000-01-01",
        end="2000-01-10",
        dtypes={"x": float, "y": float},
        freq="100 s",
    )
    out = dd.shuffle.shuffle(df, "x")
    out = out.persist()

    while not a.tasks:
        await asyncio.sleep(0.05)

    del out

    out = dd.shuffle.shuffle(df, "x")
    x, y = c.compute([df.x.size, out.x.size])
    x = await c.gather(x, y)

Note how initial shuffle is persisted and a slightly different version of this graph is computed again below but the graph below is slightly different. From what I can gather, the initial persist is fusing the keys while the latter one does not (maybe it's the other way round, I'm not sure. Either way a different issue).
This specific reproducer actually triggers (not every time) a KeyError in a workers data buffer while trying to read data.
023-01-27 16:28:34,648 - distributed.worker - ERROR - Exception during execution of task ('size-chunk-0482e0de93343089cd64837d139d9a80-49c0404470df4695b3f5aa383f11c345', 3).
Traceback (most recent call last):
  File "/Users/fjetter/workspace/distributed/distributed/worker.py", line 2366, in _prepare_args_for_execution
    data[k] = self.data[k]
  File "/Users/fjetter/workspace/distributed/distributed/spill.py", line 257, in __getitem__
    return super().__getitem__(key)
  File "/Users/fjetter/mambaforge/envs/dask-distributed-310/lib/python3.10/site-packages/zict/buffer.py", line 108, in __getitem__
    raise KeyError(key)
KeyError: "('simple-shuffle-539a650502e21de2dabd2021c9c9e684', 3)"

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/fjetter/workspace/distributed/distributed/worker.py", line 2247, in execute
    args2, kwargs2 = self._prepare_args_for_execution(ts, args, kwargs)
  File "/Users/fjetter/workspace/distributed/distributed/worker.py", line 2370, in _prepare_args_for_execution
    data[k] = Actor(type(self.state.actors[k]), self.address, k, self)
KeyError: "('simple-shuffle-539a650502e21de2dabd2021c9c9e684', 3)"

This is caused because the dependency relations between tasks are no longer accurate on the scheduler and it considers a task "ready", i.e. all dependencies in memory, too soon, causing a failure on the worker.
When validate is activated, the scheduler catches these cases earlier and raises appropriate AssertionErrors. This is not checked at runtime for performance reasons and is typically not necessary since we rely on the assumption that keys identify a task uniquely.
Apart from this artificial example, we do have internal reports about such a spurious KeyError in combination with an xgboost workload

If I understood correctly:

x is in memory on a
a receives the stimulus to compute y, which depends on x. y transitions to executing.
_before y can reach _prepare_args_for_execution, y is cancelled and x is forgotten.
before Worker.execute can complete, a receives a stimulus to execute a new task, which has the same key as y but different spec and crucially no dependency on x.
y goes into resumed state
Worker.execute terminates with KeyError, raised by _prepare_args_for_execution. If the task was still cancelled this would not be a problem as the error would be ignored.

I don't think this should be in dask/dask. This is (yet another) fragility of the cancelled state and the dask/dask graph shouldn't work around it. It should be OK to submit to the scheduler a new, different task with the same key as a released future.

Actually - I don't think you need to change the graph. You can get this error if you resubmit the same exact tasks.

x is in memory on a
a receives the stimulus to compute y, which depends on x. y transitions to executing.
before y can reach _prepare_args_for_execution, y is cancelled and x is released (which deletes it from the SpillBuffer).
client resubmits x and y
x goes in memory on b
before Worker.execute can complete on a, a receives {op: compute-task, key: y, who_has: {x: b.address}}
a transitions to resumed
Worker.execute terminates with KeyError, raised by _prepare_args_for_execution.

This requires x to go through its whole lifecycle between client, scheduler, and worker b before a has had the chance of doing even a single round of the event loop. The easiest way to achieve this is to hamstring a's event loop by spilling many GiBs worth of data.

TLDR The current system assumes strongly, in many places, that a key uniquely identifies a task, not just the output. Reusing a key for a different task violates this and distributed can fail on multiple levels.

The scheduler doesn't have the possibility to understand that the tasks are new and different. The problem has nothing to do with the Worker, this error is just surfacing there. As stated above, if validation is on, this raises in various places on the scheduler (I found three different assertion that could be triggered depending on queuing/no-queuing and some timing; here, here and I forgot where the third one was).
If you inspect closely what happens if you are executing the above script (and the del is not necessary, the del is merely there for being explicit) is that the second update-graph contains a key assign-* with a different runspec but the same key as in the first update-graph. This new state is then not properly handled by the scheduler since the scheduler assumes in many places that keys are uniquely identifying a task.
What happens here is that the scheduler knows a certain task A already that doesn't have any dependencies (first persist, w/ task fusion). Then a subsequent update-graph submits the same task but with dependencies now. This causes the previous TaskState object to be updated, i.e if task A/wout dependencies is in state ready it is updated to now have dependencies. The TaskState object after this update is now corrupt and would be caught by the scheduler validation rules but instead we're submitting this to the worker where it somehow ends up being executed.
It may be true that the worker could handle this better but the scheduler is also corrupted already.

It should be OK to submit to the scheduler a new, different task with the same key as a released future.

I believe this is where the first fallacy is hidden. In the code above, what actually happens is

update-graph (from the persist)
Client gets NPartitions future handles
update-graph (from compute; uses same keys with different spec+topologies)
and only then...
client-releases-keys for every future

i.e. the scheduler doesn't even know that the futures were released. This timing is unavoidable due to how refcounting works.
Even if you disregard this release-chain, the same issue can very very easily be constructed with a second client and different graphs.

Can either one of you motivate why blockwise fusion layers cannot generate a new key?

Agree that this is a significant problem with HLGs. The reason is the simple fact that there is no (formal/robust) mechanism to "replay" the construction of a graph after the names/properties for a subset of its tasks has been changed. That is, you can't really change the name of a Blockwise layer from "operation-c-*" to "operation-abc-*" after it has been fused with its dependencies, because you would also need to materialize the rest of the graph and manually replace all task keys beginning with "operation-c-*" to "operation-abc-*".  So, to clarify: You can change the name, but it forces graph materialization.

The reason is the simple fact that there is no (formal/robust) mechanism to "replay" the construction of a graph after the names/properties for a subset of its tasks has been changed.

So this is a limitation because there might've been an "accidental" materialization before the fusing?

So this is a limitation because there might've been an "accidental" materialization before the fusing?

Yes and no - The limitation is that there is no API to replace a layer in the HLG (e.g. hlg_graph.replace(<old-key>=new_layer)), because (1) there is no formal rule linking the output task names to the Layer name, and (2) many HLG Layers are simply wrapping already-materialized dictionaries (on purpose). Therefore, renaming the output tasks of a Blockwise Layer will often require you to materialized the entire graph.

xref https://github.com/dask/dask/issues/8635. Like @rjzamora said:

if you change the name of the layer, you also have to change the names referenced in the dependent layers. That's just not currently possible with HLG layers—we could feasibly do it to MaterializedLayers, but in general, there's no rename_input_key interface on a Layer.

xref https://github.com/dask/dask/issues/8635. Like @rjzamora said:

if you change the name of the layer, you also have to change the names referenced in the dependent layers.  That's just not currently possible with HLG layers—we could feasibly do it to MaterializedLayers, but in general, there's no rename_input_key interface on a Layer.

I'm still trying to wrap my head around it but I'm getting there.
IIUC this is mostly about layer dependencies? Best case, layer dependencies is simply exchanging a single key; Worst case would be if the layer dependent of that Blockwise layer is a MaterializedLayer in which case we'd need to walk the entire graph to rename the key. is this roughly correct?

IIUC this is mostly about layer dependencies?

Yes. The problem is that you would need to regenerate all dependent layers, and there is no clear way to do this.
Why there is no clear way to do this: There is no formal mechanism to change the dependencies for an existing Layer. The required Layer API is very limited, and does not provide a way to set the dependencies. Therefore, different logic would need to be used to change dependencies for different layers. For the MaterializedLayer case we can do this manually. However, we can't use the same logic for other layers unless we materialize them first.
Given that the logic here would be "ugly" with the existing Layer API, the ideal solution is probably to update the Layer API to include a clear mechanism for tracking dependencies. However, we have been waiting to clean up serialization. I have personally been waiting to remove the Mapping foundation as well.

Thanks, that clears things up for me.

Found a nice textbook example in the distributed tests https://github.com/dask/distributed/pull/8185/files#r1479864806

(repost from https://github.com/dask/distributed/pull/8185#issuecomment-1930341114)
Found another use case:
ddf = dd.from_pandas(df, npartitions=4)
with dask.config.set({"dataframe.shuffle.method": "p2p"}):
    ddf = ddf.set_index("a").sort_values("b")
    result = ddf.compute()

This code:

sends some keys to the scheduler on set_index,
waits for the result to return,
then sends keys to the scheduler on sort_values,
waits for the result to return,
and finally sends the computation proper with compute.

Some keys are the same, but run_spec changes; this occasionally triggers a race condition where the scheduler didn't have the time to forget those keys yet.
This is (fortuitously) not causing failures because the key is always close to its released->forgotten transition when this happens (although in other cases it may be in memory; I'm not sure).
If I add a line print(tokenize(ts.run_spec)) to _generate_taskstates, you'll get:
('assign-f0dc4405b24a6c54160e65ddfeb867eb', 0) 60dfef541bc8fe30d091a769b90d6fae
('assign-f0dc4405b24a6c54160e65ddfeb867eb', 0) 6d2d7e33c168fc1a0d0902905b811aca
('assign-f0dc4405b24a6c54160e65ddfeb867eb', 1) 4c91541c56f2890b6a7dce01aa06fab9
('assign-f0dc4405b24a6c54160e65ddfeb867eb', 1) 81485c4a5cf24f8cedf4fe5dcf84acee
('assign-f0dc4405b24a6c54160e65ddfeb867eb', 2) 96326838be2ca074533222b80e986acd
('assign-f0dc4405b24a6c54160e65ddfeb867eb', 2) f0096bf0edd773d5a48fe6f8495ac4f2
('assign-f0dc4405b24a6c54160e65ddfeb867eb', 3) 153143fb0b822dbeeb17601e883b1e45
('assign-f0dc4405b24a6c54160e65ddfeb867eb', 3) c0fb8a09ad926986969e39adf100a1a3

Hi,
I'm trying to update some of my Zarr files following a DAG execution, but some of the tasks reuse the resulting delayed arrays from others, and as they run in parallel and load the same data I think that is making Dask raise a warning mentioning this issue, I would appreciate if you could let me know if there is some workaround/alternative to be able to run my DAG without receiving the warning and without affecting the results, I let a code of the example to clarify in a better way what I'm trying to achieve.
If there is no workaround/alternative, I have the time to contribute to this issue (it would be my first in Dask, I have only contributed to Xarray).
`py
import xarray as xr
import dask.array as da
from dask.distributed import LocalCluster, Client
Change the base_path
base_path = "YOUR_PATH"
sizes = [("a", 3), ("b", 5), ("c", 2)]
def update_dataset(factor):
coords = {
dim: list(range(size))
for dim, size in sizes
}
dataset = xr.Dataset(
{
"data": xr.DataArray(
da.ones(
shape=[size for , size in sizes],
chunks=(1, 1, 1)
) * factor,
coords=coords,
dims=[dim for dim, _ in sizes]
)
}
)
path = f"{base_path}/test_zarr{factor}"
dataset.to_zarr(path, mode="w", compute=True)
return xr.open_zarr(path)
def calculate1(x, y):
path = f"{base_path}/test_zarr_calculate1"
result = x * y
result.to_zarr(path, mode="w", compute=True)
return result
def calculate2(x, y):
path = f"{base_path}/test_zarr_calculate2"
result = x / y
result.to_zarr(path, mode="w", compute=True)
return result
dag = {
"update_arr1": (update_dataset, 2),
"update_arr2": (update_dataset, 4),
"calculate1": (calculate1, "update_arr1", "update_arr2"),
"calculate2": (calculate2, "update_arr1", "update_arr2"),
"report": (lambda x, y: True, "calculate1", "calculate2")
}
with LocalCluster(
n_workers=1,
threads_per_worker=2,
memory_limit="1GB",
) as cluster:
with cluster.get_client() as client:
print(client.get(dag, "report"))
`
Warning raised:
py 2024-08-24 16:13:02,764 - distributed.scheduler - WARNING - Detected different run_specfor key 'original-open_dataset-data-c801781151f576ac3a1c261dc894c1d0' between two consecutive calls toupdate_graph`. This can cause failures and deadlocks down the line. Please ensure unique key names. If you are using a standard dask collections, consider releasing all the data before resubmitting another computation. More details and help can be found at https://github.com/dask/dask/issues/9888.
Debugging information
old task state: processing
old run_spec: (<function execute_task at 0x0000023975DFF600>, (ImplicitToExplicitIndexingAdapter(array=CopyOnWriteArray(array=LazilyIndexedArray(array=<xarray.backends.zarr.ZarrArrayWrapper object at 0x00000239772DD380>, key=BasicIndexer((slice(None, None, None), slice(None, None, None), slice(None, None, None)))))),), {})
new run_spec: (<function execute_task at 0x0000023975DFF600>, (ImplicitToExplicitIndexingAdapter(array=CopyOnWriteArray(array=LazilyIndexedArray(array=<xarray.backends.zarr.ZarrArrayWrapper object at 0x0000023977322A40>, key=BasicIndexer((slice(None, None, None), slice(None, None, None), slice(None, None, None)))))),), {})
old token: ('tuple', [('913ceb5b5beb463a9010ec0790bc30002ca34164', []), ('tuple', [('7a2b6b38794817e675b96ba80026355719825fb1', ['6693512388b528fbf156d0cc4c7b9449588c44bf'])]), ('dict', [])])
new token: ('tuple', [('913ceb5b5beb463a9010ec0790bc30002ca34164', []), ('tuple', [('7a2b6b38794817e675b96ba80026355719825fb1', ['45ef8ea51c5d50300dd28843ec098db9cbb85566'])]), ('dict', [])])
old dependencies: set()
new dependencies: set()
`

@josephnowak this looks like a bug that's been reported on the xarray tracker https://github.com/pydata/xarray/issues/9325 which was fixed in https://github.com/dask/dask/pull/11320

@fjetter Thanks for your response, I tried using Dask 2024.08.01 which in theory contains the fix that you are mentioning and I'm still getting the warning.

Hi all, I'm running dask.array.linalg.svd_compressed on a dask array, and getting an error linking to this issue, with
Detected different `run_spec` for key ('unique-chunk-9a6ff65c242a27e80238a4b391175021-fef5c953b2221b0ee5e1bf565ad89ba4', 35) between two consecutive calls to `update_graph`. This can cause failures and deadlocks down the line. Please ensure unique key names. If you are using a standard dask collections, consider releasing all the data before resubmitting another computation. More details and help can be found at https://github.com/dask/dask/issues/9888.

and
2024-11-28 09:51:12,805 - distributed.scheduler - WARNING - Detected different `run_spec` for key ('assign-b03a54a86dd939287458141b89b34755', 35) between two consecutive calls to `update_graph`. This can cause failures and deadlocks down the line. Please ensure unique key names. If you are using a standard dask collections, consider releasing all the data before resubmitting another computation. More details and help can be found at https://github.com/dask/dask/issues/9888.

and basically the same warning tens of times with different keys  (sometimes it's "assign-...", sometimes it's "unique-...", etc)
I am wondering whether that's because of a bug somewhere, or because I'm doing something kind of weird to build my dask array from a dask dataframe initially?
Context is recommender systems, and the typical user-items matrix decomposition that goes with it.
I've got a dask dataframe with user-item interaction rows, (essentially equivalent to a sparse coo matrix I believe), and when trying to turn it into a dask array (so I can use dask.array.linalg.svd_compressed on it), I had to normalise the user ids to be between 0 and n_users, but also for each chunk of the dask array I had to start from 0 again, and do the bookkeeping myself:
    def partition_to_coo(partition):
        users_from_this_partition = (partition["encoded_userid"] % division_size).values
        items_from_this_partition = partition["encoded_itemid"].values
        values = partition["value"].values
        np_array = sparse.COO(
            (values, (users_from_this_partition, items_from_this_partition)),
            shape=(max(users_from_this_partition) + 1, n_items),
        ).todense()
        return np_array

    dense_user_items_matrix = views_sorted.map_partitions(
        partition_to_coo, meta=np.ndarray(shape=(0, 0), dtype=int)
    )

Can/Should this result in the above warning?

Should I delete this & ask this in a separate github issue and just refer here?

Thanks!


Add SECURITY_CONTACTS

Closes: #19
/sig azure
Signed-off-by: Stephen Augustus<EMAIL_ADDRESS>

/assign @colemickens @khenidak

Looks fine to me but I'll wait for @khenidak in case there are more Azure folks to include.

Did this get stuck?
/retest

/lgtm
/approve

/lgtm


config: join purelind as ti-community-infra reviewer

@purelind will help TiDB community review the PR changed ti-chi-bot config.

/test pull-configs-validate-orgs

/test pull-configs-validate-orgs

/merge


adapted to appformer and kie-soup moved into kie

fixed all Jenkins release jobs to consider the move of appformer and kie-soup into kiegroup

Merge together with:
https://github.com/kiegroup/appformer/pull/109
https://github.com/kiegroup/kie-soup/pull/20

as we dont need anymore kie-soup separated with own jobs since it will be included in repository-list.txt we don't nee anymore this PR. This will be closed.


Flaky e2e test - terminal-failed-job

What happened:
Failed test kudo/harness/terminal-failed-job on a PR introducing a KEP (no code)
What you expected to happen:
no failure
How to reproduce it (as minimally and precisely as possible):
run e2e tests
Anything else we need to know?:
https://app.circleci.com/pipelines/github/kudobuilder/kudo/5320/workflows/2781ca5b-9107-4e7c-b2c2-c6ccd121b615/jobs/15615/steps
Environment:

Kubernetes version (use kubectl version):
Kudo version (use kubectl kudo version):
Operator:
operatorVersion:
Cloud provider or hardware configuration:
OS (e.g. from /etc/os-release):
Kernel (e.g. uname -a):
Install tools:
Others:

I've had a first look at this, and it doesn't look easy. It seems that for some reason k8s does not timout the second job in the e2e-test.
The failing test has two parts:

A Job that fails after 3 tries and brings the KUDO plan to a FATAL_ERROR state
A Job that times out after 60 seconds and brings the KUDO plan to a FATAL_ERROR state

The first part works, the second part does not get to the timeout. The Event log does not show the expected "Deadline Exceeded" event that would indicate that the job actually times out, so I'm not sure where the issue is. It seems that KUDO correctly detects the timeout when it happens, but k8s does not trigger the timeout.

In addition to the above analysis: I've found this suspicious line in the etcd log:
2020-09-23T12:25:12.136185294Z stderr F 2020-09-23 12:25:12.136028 W | etcdserver: read-only range request "key:\"/registry/jobs/kudo-test-smiling-eft/timeout-job\" " with result "range_response_count:1 size:3361" took too long (215.5448ms) to execute

It seems that the request takes too long to return. I've also counted how many health requests we do for the above job resulting in:
2020-09-23T12:24:50.115387918Z stderr F 2020/09/23 12:24:50 HealthUtil: job "timeout-job" still running or failed

Withing the 5 minutes of the test, we made 1196 requests so ~4 req/s. It's not that much but still somewhat excessive.

I don't think that issue fixed it :)

I think the point is that there isn't much we could do from the previous logs, and we reopen once the issue resurfaces with the improved logging.

alright, I would keep it open because people tend to ignore failures in PRs and I want to be reminded via issue but let's close it then :)

Well, I would have kept it open as well :D The close was not intentional, I hoped I could find the bug and fix it in the linked PR, but it seems to happen only rarely.
I don't mind either way - I'm pretty sure we'll find the issue again if the test flakes again :)

Happened again https://app.circleci.com/pipelines/github/kudobuilder/kudo/5564/workflows/2d82fa0a-716d-4087-9995-3a47781e19f5/jobs/16874/tests

Even with the additional logging output there's not really a reason to see why the timeout does not occour. We should probably leave this open, but I don't think there's really something we can do at this moment.


Alexa notify.alexa_media

Is not possible to use alexa_media like TTS notify
this is my error
Logger: homeassistant.components.websocket_api.http.connection
Source: custom_components/alexa_media/notify.py:195
Integration: Home Assistant WebSocket API (documentation, issues)
First occurred: 17:51:05 (5 occurrences)
Last logged: 17:53:55
[1558717512] 'NoneType' object is not iterable
Traceback (most recent call last):
File "/srv/homeassistant/lib/python3.9/site-packages/homeassistant/components/websocket_api/commands.py", line 185, in handle_call_service
await hass.services.async_call(
File "/srv/homeassistant/lib/python3.9/site-packages/homeassistant/core.py", line 1491, in async_call
task.result()
File "/srv/homeassistant/lib/python3.9/site-packages/homeassistant/core.py", line 1526, in _execute_service
await handler.job.target(service_call)
File "/srv/homeassistant/lib/python3.9/site-packages/homeassistant/components/notify/init.py", line 159, in _async_notify_message_service
await self.async_send_message(**kwargs)
File "/home/homeassistant/.homeassistant/custom_components/alexa_media/notify.py", line 195, in async_send_message
for target in targets:
TypeError: 'NoneType' object is not iterable
Thank

Your error doesn’t show any problem caused by alarmo.
Therefore I cannot help you.

@nielsfaber , I have the same error. I believe the Alarmo front-end generate an invalid object for media_player.


Change return types for functions/constructors that might fail in the future

Some functions (e.g. Shader::from_string) don't currently return Result, as they don't perform proper error handling. We should pre-emptively wrap the return types in Result so that we don't have to make another breaking change later on.
This is less of a concern for functions that currently return (), as adding a Result return will just give the user a warning.

Fixed in 0.2.


Update mkdirp version to avoid octal literal issues

Octal literals are a thing of the past, it seems. The latest patch to mkdirp removes them, so I bumped the version.
This fixes #57.

hope merge. because it don't work on strict mode.

Could we possibly have the deps pulled  in as ~ or even ^ ? Or are there actual fears that something might break here? In any case, would like to see this pulled and published. Thanks in advance! 🙇  🍻

Fixed by #66.


Remove unnecessary null pointer checks

An extra null pointer check is not needed in functions like the following.

BoostLearner
BoostLearnTask
FMatrixS

Since this do not have things to do with correctness, and explicit check was not too bad for readable purpose(hinting that we expect NULL ptr). I am closing this issue for now.

Would it be nice to improve efficiency also for this software here?

In our case, most of delete happens very infrequently (one class per usage of training). I agree that if this happens very frequently considering remove it will help. Thanks


Fixed #35236 -- Used Field attname and column attributes directly.

ticket-35236

diff --git a/django/forms/models.py b/django/forms/models.py
index cd6986d72a..4b11d5af8c 100644
--- a/django/forms/models.py
+++ b/django/forms/models.py
@@ -1140,7 +1140,7 @@ class BaseInlineFormSet(BaseModelFormSet):
         if self.fk.remote_field.field_name != self.fk.remote_field.model._meta.pk.name:
             fk_value = getattr(self.instance, self.fk.remote_field.field_name)
             fk_value = getattr(fk_value, "pk", fk_value)
-        setattr(form.instance, self.fk.get_attname(), fk_value)
+        setattr(form.instance, self.fk.attname, fk_value)
         return form

     @classmethod

:point_up: I can change it and move it forward if you like.

Sounds good to me, I won't be able to work on it until the evening otherwise.


Only Audio, no Video on iOS 10.3 while using a remote video

As the title suggests. Tested with an mp4 file hosted on AWS which plays just fine on iOS 11+

I have the same issue on Android, both with local and remote videos, both on emulator and real device. Tried with few files. Tried setting height and width in the style as some suggest. This issue seems to be brought up every now and then according to the tracker.

same issue with you
my solution:
make a separate component for Video
this component have a props: videoURL
render() {
		return (
            <View style={commonStyles.wrapperGray}>
                <Header
                    renderIcon={false}
                    title={'CF Videos'}
                    menuColor={'#3971c1'} />
                    <View style={{flex: 1, backgroundColor:'black'}}>
                        <Video source={{uri: this.props.video.urlVideo.url}}   // Can be a URL or a local file.
                            ref={(ref) =>  { this.player = ref }}                                      // Store reference
                            rate={1.0}                              // 0 is paused, 1 is normal.
                            volume={1.0}
   ...............

And now, it worked
I guess that: your component is reloaded many time, it caused your problem
I hope my solution can help you
good luck

same

This seems to be happening for specific videos. Some of mine work, and others do not (audio only). I have tried position absolute etc etc, to get the video to display, but nothing. Happens now on both iOS and android.

Can you provide a url for one of the videos that doesn't work?

Further testing, I have two videos I am testing with, both work on iOS, only 1 of the two work on Android, Im supplying a link to the video (this may expire and if so, Ill post a different link.
Here is a link to a video that fails to play on Android.
https://swingbot-video-mp4.s3.amazonaws.com/1_20121.mp4?AWSAccessKeyId=AKIAI6V4ERWWWX7B6P6A&Expires=1527126875&Signature=6DHTTXSqTYsxjNfpZeBRLnpgFUE%3D

Sorry for the late reply but I figured I was saving my videos from iOS as mov which fails to play in some devices. Extracting the mp4 from the mov and saving it onto the sever solved all the issues. There are a couple of RN libraries that can do this for you.
~ cheers
Sent from my iPhone

On May 16, 2018, at 6:58 PM, John<EMAIL_ADDRESS>wrote:
Further testing, I have two videos I am testing with, both work on iOS, only 1 of the two work on Android, Im supplying a link to the video (this may expire and if so, Ill post a different link.
Here is a link to a video that fails to play on Android.
https://swingbot-video-mp4.s3.amazonaws.com/1_20121.mp4?AWSAccessKeyId=AKIAI6V4ERWWWX7B6P6A&Expires=1527126875&Signature=6DHTTXSqTYsxjNfpZeBRLnpgFUE%3D
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.


editor help?

I have a habit of using [CTRL D] when I want to duplicate a line.  Deletes a line in your app.  Your editor uses CodeMirror, so am I assuming correct you left it at the key defautls?
If so, where should the info/link go for students to find?

What action do you propose here?
New users are unlikely to use these keybindings.  While we could potentially add a ? in the corner or something, the return seems low.

Well, all the commands are on the CodeMirror site, no need to do anything more than linking.
The first places I looked were the WiKi (which is locked, so I had to request) & the Gitter bot.  A link at one or both should suffice for those who look.  I guess a corner (?) button or [F1] help would be nice, but yes extra work might not be worth it for now.

Please keep in mind, though most of your focus have been on newbies to HTML/CSS/JS, I think you'll attract ether non-beginners like me who want the certificate & internship experience, or advanced users who will want to learn new tech.  Eg: I see your D3 & React lessons in progress; which will definitely attract experienced coders.  They'll all hope for short cuts & other conveniences, since they use them constantly.

Fair enough.  I think it's reasonable to have a link to editor info.  I'll capture this as a feature request.

Thinking about this more, best to add to Wiki &/or bot.  I agree too much work for editor GUI; unless there is more people looking for it.


Fix: hold the force uninstalling process untill the last addon been deleted

A bug generated when fluxcd addon has been enabled, then use uninstall -f
The uninstall process begin to remove the vela core controller and vela-system namespace while fluxcd addon is  been deleting. This will cause a dead clock to block whole uninstall and re-install process.
Signed-off-by: 楚岳<EMAIL_ADDRESS>Description of your changes

Fixes #
I have:

[x] Read and followed KubeVela's contribution process.
[ ] Related Docs updated properly. In a new feature or configuration option, an update to the documentation is necessary.
[x] Run make reviewable to ensure this PR is ready for review.
[x] Added backport release-x.y labels to auto-backport this PR if necessary.

How has this code been tested

Special notes for your reviewer

why we need to wait 5 second

why we need to wait 5 second

The vela-core controller is gc the resources of fluxcd addon. Should wait untill the gc process finish, then delete the vela-core controller, otherwise the fluxcd addon application will block in deleting process

can we make it shorter since there's a for-loop here?

can we make it shorter since there's a for-loop here?

Have updated with 2 minutes.

I mean 5s -> 2s, for better experience

the waiting message is
Waiting delete the fluxcd addon, timeout left 4m59.96s

LGTM


FLPATH-400: Refactor NotificationRecordController unit tests

What this PR does / why we need it:
This change is done to increase unit tests coverage
Also changed the controller code
Which issue(s) this PR fixes:
Fixes FLPATH-400
Change type

[ ] New feature
[x] Bug fix
[x] Unit tests
[ ] Integration tests
[ ] CI
[ ] Documentation
[x] Auto-generated SDK code

Impacted services

[ ] Workflow Service
[x] Notification Service

Checklist

[x] Subject and description added to both, commit and PR.
[ ] Relevant issues have been referenced.

/lgtm


Not getting Phone Number

I connect plugin and I get as I response a lot of data from phone but not the phone number. Can you tell me why?Thanks :)

Have you checked issues which are closed. See below link
https://github.com/pbakondy/cordova-plugin-sim/issues/50


Arabic And Persian Support Added To DOM PDF #4420

BUGS:
Link: https://github.com/bagisto/bagisto/issues/4420
After lots of research dompdf support can only be fixed with this package. In latest version they have given the support via composer and clean way as well.
Steps to execute this pull request

Run composer install.
Done!

@devansh-webkul
getting an exception when printing invoice

@devansh-webkul
getting an exception when printing invoice

Hi @vaishaliwebkul,
You composer package not installed properly, please remove vendor folder and re-run the command composer install.

Hi @vaishaliwebkul,
You composer package not installed properly, please remove vendor folder and re-run the command composer install.

@devansh-webkul
I already followed all the steps mentioned above but still getting this error.

@devansh-webkul
I already followed all the steps mentioned above but still getting this error.

@vaishaliwebkul,
Try composer install --no-cache and then composer dump-autoload.

@vaishaliwebkul,
Try composer install --no-cache and then composer dump-autoload.

@devansh-webkul
I have Reverified this PR with a fresh project and still getting the same error.
Environment configuration are mentioned below

composer version 2.0.2

PHP Version 7.4

@devansh-webkul
I have Reverified this PR with a fresh project and still getting the same error.
Environment configuration are mentioned below

composer version 2.0.2

PHP Version 7.4

@ghermans,
Fixed the currency symbol issue.
@vaishaliwebkul
I will try to reproduce that on the latest one also. I will update you regarding the same.

@ghermans,
Fixed the currency symbol issue.
@vaishaliwebkul
I will try to reproduce that on the latest one also. I will update you regarding the same.

There is one more issue related to Arabic in Invoice. I am using Kuwaiti Dinar in the currency which should be .د.ك , but it is opposite instead as shown in the image below.

There is one more issue related to Arabic in Invoice. I am using Kuwaiti Dinar in the currency which should be .د.ك , but it is opposite instead as shown in the image below.

Hi, @fahadkhan1740,
You can raise your issue here, as this will help all the Arabic community as well,
https://github.com/khaled-alshamaa/ar-php

Hi, @fahadkhan1740,
You can raise your issue here, as this will help all the Arabic community as well,
https://github.com/khaled-alshamaa/ar-php

There is one more issue related to Arabic in Invoice. I am using Kuwaiti Dinar in the currency which should be .د.ك , but it is opposite instead as shown in the image below.

Can you try the latest version now (i.e. using version 6.0.0 of ar-php library)? This should resolve your reported issue of د.ك.


missing dependant sourcekite tool, please configure correct "swift.path.sourcekite"

Hi,
I am unable to use this extension. I am running VS Code on macOS High Sierra, and i get the above error message. This is the output:
justinmoser/.vscode/extensions/vknabel.vscode-swift-development-environment-2.1.2/out/src/server/sourcekites.js:43
        throw err;
        ^

Error: spawn swift build -Xswiftc -framework -Xswiftc sourcekitd -Xswiftc -F -Xswiftc /Library/Developer/Toolchains/swift-latest.xctoolchain/usr/lib -Xlinker -rpath -Xlinker /Library/Developer/Toolchains/swift-latest.xctoolchain/usr/lib ENOENT
    at exports._errnoException (util.js:1050:11)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:193:32)
    at onErrorNT (internal/child_process.js:367:16)
    at _combinedTickCallback (internal/process/next_tick.js:80:11)
    at process._tickCallback (internal/process/next_tick.js:104:9)
[Error - 18:10:48] Connection to server got closed. Server will not be restarted.

I have run the build / link command as instructed on the sourcekite repo page. From my root directory (HD root):
> git clone https://github.com/jinmingjian/sourcekite.git
> cd sourcekite
> swift build -Xswiftc -framework -Xswiftc sourcekitd -Xswiftc -F -Xswiftc /Library/Developer/Toolchains/swift-latest.xctoolchain/usr/lib -Xlinker -rpath -Xlinker /Library/Developer/Toolchains/swift-latest.xctoolchain/usr/lib
and then added this to my settings.json file, under the user settings tab:
"swift.path.sourcekite": "/Users/justinmoser/sourcekite/.build/debug/sourcekite"
I also get the message:
Incorrect type. Expected "array".
in the launch.json file on this key:
"configurations": "sde.commands.debug.provideInitialConfigurations"

@JustinMoser you have installed my fork of this repository. I moved your issue to the new location.


Set custom User-Agent

It is needed for being able to access some web pages
such as YouTube Music.
The UA string is based on:
HTTP_USER_AGENT of Safari running on my environment.

Sorry to barge in on you like this.
I'm the developer of this package!
I'm thinking that being able to set User-Agent is a necessary feature for the package, and I'd like to add it to the original package.
Is this working?

Hello @akaboshinit san,
Thank you for your contact.
Your flutter_inline_webview_macos is so helpful for me.

I'm thinking that being able to set User-Agent is a necessary feature for the package, and I'd like to add it to the original package.
Is this working?

Yes, It seems to be okay on my environment.
But I'm not familiar with WKWebView, so I don't have confident whether the implementation is right way.
diff --git a/lib/main.dart b/lib/main.dart
index 019ca98..346e658 100644
--- a/lib/main.dart
+++ b/lib/main.dart
@@ -85,7 +85,9 @@ class _MyMusicAppState extends State<MyMusicApp> {
               _controller = controller;
               _controller!.loadUrl(
                   urlRequest:
+                      // URLRequest(url: Uri.parse("https://www.baka-ke.com/lab/useragent.php")));
                       URLRequest(url: Uri.parse("https://music.youtube.com")));
+
             },
           ),
         ),
diff --git a/pubspec.yaml b/pubspec.yaml
index 4a60234..c2f2d2f 100644
--- a/pubspec.yaml
+++ b/pubspec.yaml
@@ -41,7 +41,8 @@ dependencies:
   flutter_inline_webview_macos:
     git:
       url: https://github.com/m-b-t-n/flutter_inline_webview_macos
-      ref: 3288e53fe053dc7a86112ae5b6117098ff41b46d
+      ref: original
+      # ref: 3288e53fe053dc7a86112ae5b6117098ff41b46d

   # [flutter-desktop-embedding/plugins/window_size at main · google/flutter-desktop-embedding]
   # (https://github.com/google/flutter-desktop-embedding/tree/main/plugins/window_size)

Before

After

Please let me know if you have something.
Regards.


member-ordering documentation does not explain what an instance sandwich is

Bug Report

TSLint version: 5.11.0
TypeScript version: 2.9.2
Running TSLint via: CLI

Actual behavior
https://palantir.github.io/tslint/rules/member-ordering/ does not explain what "instance-sandwich" does
Expected behavior
https://palantir.github.io/tslint/rules/member-ordering/ explains what "instance-sandwich" does

I found this quite strange also, it must've been just an oversight by whoever coded it. The meaning can be found by looking in the source code in the meantime:
https://github.com/palantir/tslint/blob/master/src/rules/memberOrderingRule.ts#L62

I also want to know what an instance-sandwich is!
And I think fields-first and statics-first should also be defined in the docs. It isn't clear that they don't bring in any other rules with them besides the obvious. Does fields-first only check for fields coming first, or might it also put public before private? It should explicitly state that it does nothing more.

I made a PR for this.  How do I get someone to look at it?

member-ordering broke some code for me
I had state in my class below  getValue method
state = {
    input: '',
    search: this.getValue() ? this.getValue().name : '',
    isFetching: false,
  };

so member-ordered moved state to the top and broke my code

Please make a new issue; that appears entirely unrelated to this thread.


main menu shows GroupMS2Task, restructure visualisation menu

Looks fine to me.
Final order of modules can still be changed in the future.


Categories Data points limits to 1000

Hi
Max no of categories data points available for me in update function is 1000 only.
Kind help me how to get all the data points available in update function.
Capabilities.json is as follows.
{
    "dataRoles": [
        {
            "displayName": "Category Data",
            "name": "category",
            "kind": "Grouping"
        },
        {
            "displayName": "Measure Data",
            "name": "measure",
            "kind": "Measure"
        }
    ],
    "dataViewMappings": [
        {
            "categorical": {
                "categories": {
                    "for": {
                        "in": "category"
                    },
                    "dataReductionAlgorithm": {
                        "top": {}
                    }
                },
                "values": {
                    "select": [
                        {
                            "bind": {
                                "to": "measure"
                            }
                        }
                    ]
                }
            }
        }
    ]
}

Hello @kksevta
Power BI default limitation is 1000 data rows.
You can increase the limit up to 30000 by using this in your capabilities:
"dataReductionAlgorithm": {
    "top": {
        "count": 10000
    }
}

thanks @ignatvilesov
it worked :)


Crashes on description page

Every time I press on an anime so that I can watch it or look at the description the app just crashes, I had no problem with the original app btw.

It is impossible to help if you do not provide more information.
Like the bug report template says:
Please complete the following information:

Android version
App version
Phone model

Sorry I don't use GitHub much, I downloaded the 1.2.2 version, I'm on a Galaxy S9+, android 10

Is this still an issue in 1.6.3?

Closing due to inactivity, re-open if the issue remains.


Wrong syntax highlighting when using await using

When creating a simple file that uses the await using construct, syntax highlighting is broken below the point where it is used. For example, keywords like public are no longer displayed in dark blue, see screenshot below. This is a simple example code:
using System;
using System.Threading.Tasks;

namespace vscode_await_using_bug
{
    class Program : IAsyncDisposable
    {
        static async Task Main(string[] args)
        {
            await using (var v = new Program())
            {
                Foo();
            }
        }

        public async ValueTask DisposeAsync()
        {
            await Task.Delay(50);
            Console.WriteLine("Byebye");
        }

        public static void Foo()
        {
            Console.WriteLine("Hello World!");
        }
    }
}

Environment data
dotnet --info output:
.NET Core SDK (reflecting any global.json):
Version:   3.1.301
Commit:    7feb845744
Runtime Environment:
OS Name:     ubuntu
OS Version:  20.04
OS Platform: Linux
RID:         linux-x64
Base Path:   /usr/share/dotnet/sdk/3.1.301/
Host (useful for support):
Version: 3.1.5
Commit:  65cd789777
.NET Core SDKs installed:
3.1.301 [/usr/share/dotnet/sdk]
.NET Core runtimes installed:
Microsoft.AspNetCore.App 3.1.5 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.NETCore.App 3.1.5 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
To install additional .NET Core runtimes or SDKs:
https://aka.ms/dotnet-download
VS Code version:
1.47.1
C# Extension version:
1.22.1
OmniSharp log
Starting OmniSharp server at 7/16/2020, 5:33:04 PM
Target: /home/gidoca/src/vscode-await-using-bug
OmniSharp server started.
Path: /home/gidoca/.vscode/extensions/ms-dotnettools.csharp-1.22.1/.omnisharp/1.35.3/run
PID: 264088
    Starting OmniSharp on ubuntu 20.4 (x64)

    DotNetPath set to dotnet

    Located 1 MSBuild instance(s)
        1: StandAlone 16.4 - "/home/gidoca/.vscode/extensions/ms-dotnettools.csharp-1.22.1/.omnisharp/1.35.3/omnisharp/.msbuild/Current/Bin"

    MSBUILD_EXE_PATH environment variable set to '/home/gidoca/.vscode/extensions/ms-dotnettools.csharp-1.22.1/.omnisharp/1.35.3/omnisharp/.msbuild/Current/Bin/MSBuild.dll'

    Registered MSBuild instance: StandAlone 16.4 - "/home/gidoca/.vscode/extensions/ms-dotnettools.csharp-1.22.1/.omnisharp/1.35.3/omnisharp/.msbuild/Current/Bin"
        CscToolExe = csc.exe
        MSBuildToolsPath = /home/gidoca/.vscode/extensions/ms-dotnettools.csharp-1.22.1/.omnisharp/1.35.3/omnisharp/.msbuild/Current/Bin
        CscToolPath = /home/gidoca/.vscode/extensions/ms-dotnettools.csharp-1.22.1/.omnisharp/1.35.3/omnisharp/.msbuild/Current/Bin/Roslyn
        BypassFrameworkInstallChecks = true
        MSBuildExtensionsPath = /home/gidoca/.vscode/extensions/ms-dotnettools.csharp-1.22.1/.omnisharp/1.35.3/omnisharp/.msbuild

    Detecting Cake files in '/home/gidoca/src/vscode-await-using-bug'.

    Could not find any Cake files

    No solution files found in '/home/gidoca/src/vscode-await-using-bug'

    Detecting CSX files in '/home/gidoca/src/vscode-await-using-bug'.

    Could not find any CSX files

    Invoking Workspace Options Provider: OmniSharp.Roslyn.CSharp.Services.CSharpFormattingWorkspaceOptionsProvider, Order: 0

    Invoking Workspace Options Provider: OmniSharp.Roslyn.CSharp.Services.RenameWorkspaceOptionsProvider, Order: 100

    Invoking Workspace Options Provider: OmniSharp.Roslyn.CSharp.Services.ImplementTypeWorkspaceOptionsProvider, Order: 110

    Invoking Workspace Options Provider: OmniSharp.Roslyn.CSharp.Services.BlockStructureWorkspaceOptionsProvider, Order: 140

    Configuration finished.

    Omnisharp server running using Stdio at location '/home/gidoca/src/vscode-await-using-bug' on host 263959.

Steps to reproduce

Create a file named Program.cs with the content above.

Expected  behavior
The syntax highlight is the same as synchronous using.

Actual behavior
The syntax highlighting no longer marks keywords correctly. using is yellow, public and static are light blue.

@gidoca To workaround some of the limitations of our C# textmate theme, you can enable editor.semanticHighlighting.enabled and csharp.semanticHighlighting.enabled. It required using a theme that supports semantic highlight. You can try our Visual Studio 2019 Dark theme if you like, or you can tell VS Code that you current theme supports semanticHighlighting by following the instructions here (https://github.com/microsoft/vscode/issues/92740#issuecomment-600100176) to update your "editor.tokenColorCustomizations".

The textmate grammar has been updated to fix this problem.


Check on server if current move correct

Unit can move for number of hexes equal or less than its speed.
Unit can't move through the walls and units.


[AutoBuild] address issue 941; add new feature; add debug information

Why are these changes needed?

address #941
new feature: load JSON string config directly
add detailed debug information
slightly change the AGENT_NAME_PROMPT

Related issue number
Open #941
Checks

[x] I've made sure all auto checks have passed.

Codecov Report
Attention: 29 lines in your changes are missing coverage. Please review.

Comparison is base (9cec541) 26.63% compared to head (c2816a5) 24.13%.

Files
Patch %
Lines

autogen/agentchat/contrib/agent_builder.py
9.37%
29 Missing :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #944      +/-   ##
==========================================
- Coverage   26.63%   24.13%   -2.51%
==========================================
  Files          28       28
  Lines        3777     3800      +23
  Branches      858      865       +7
==========================================
- Hits         1006      917      -89
- Misses       2700     2760      +60
- Partials       71      123      +52

Flag
Coverage Δ

unittests
24.13% <9.37%> (-2.46%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


chore(ci): add base workflows for GH Actions

This base config is needed for Pull Requests to run C.I. at all

:tada: This PR is included in version 1.0.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


chore(slim): make the alpine image slightly more slim

the intention of the more explicit pins is to eventually point dependabot at this repository

Oh I see, you're consolidating the apk add back into one layer and updating to 3.16.0.


UE: Escaping HTML present in Tooltip and Description using XSSAPI

Description

Related Issue

Motivation and Context

How Has This Been Tested?

Screenshots (if appropriate):
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[ ] I have signed the Adobe Open Source CLA.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[ ] I have read the CONTRIBUTING document.
[ ] I have added tests to cover my changes and the overall coverage did not decrease.
[ ] All unit tests pass on CircleCi.
[ ] I ran all tests locally and they pass.

I am aware that this is a backward incompatible change as the value of the tooltip and description is changing in the JSON. What version would I have to bump?


Crash: contentLengthLong throws NoSuchMethodError for Android 6 or below.

Hi @afreakyelf
This fix is not in the latest release. Due to which there has been crashes in Android 6 or below.
Can you please provide a resolution for this?

Originally posted by @vishal-gaur-shipsy in ca4f215

to fix your trouble check this solution click
maybe this will solve your problem.


Environment variables set to empty cause silent failing of auth

Hello,
recently, we stumbled upon an interesting 'bug' that took us some time to resolve - we had set AWS_ACCESS_KEY_ID env variable set to empty, which resulted of having it in os.environ as empty string. If it's there as an empty string, it causes config resolver to use EnvProvider - even if it's clearly invalid. While i realize that this issue is caused by us, I would like to propose a small improvement to EnvProvider - which will not only check if env variables are set - but also if they're valid and log/error out when that's not the case, allowing users for quicker debugging.
If I can help with the implementation, please let me know, I would gladly take a stab at it.
Regards,
Piotr

Hm, I think this sounds reasonable but it's also arguably a breaking change / not a forwards compatible change. It might be difficult to actually define what a 'valid' credential is in all contexts. For example, the actual value set may not matter in a testing context (if the responses are stubbed, etc.). Let me run this by some of the other members and see what they think.

Hello @joguSD, thanks for checking it my issue. Yeah, I can imagine how that can be a problem, so maybe the solution would be to just check if the actual values in os.environ are truthy ? that would prevent issues with empty strings

I hit this as well.  We deploy applications to Kubernetes using Terraform.  For testing, we deploy on Minikube locally, which requires us to set the auth variables in Terraform.  HCL still doesn't have conditionals, so they're always set - even when we use the same module to deploy to a k8s cluster in AWS which is intending to use IAM.  As a result auth fails in the real deployment (since the env vars are set but are empty).
I have a modification written & deployed right now that solves this.  It does so conservatively by special-casing empty string values rather than checking truthiness.  It does this for AWS_SECRET_ACCESS_KEY, AWS_ACCESS_KEY_ID, AWS_SECURITY_TOKEN, and AWS_ACCESS_TOKEN.  I've got some tests in place for the empty string case.
I'm running the test suite right now, I should have the PR open this afternoon (EST).  Obviously merging it will be dependent not only on code quality but also on the concerns you've voiced above @joguSD .

Looks like the other AWS SDKs have the behavior of moving on to the next credential provider if the env var values are empty.  I'm inclined to accept #1687.


how to use models in controller ?

hi, i had created a model using php artisan make:module:model  command now in my controller how to make a object of it ?
$obj = new classname();
i suppose i need to add use command but i don't know how

Please take a look at the Basic Task List tutorial within Laravel's documentation:
https://laravel.com/docs/5.2/quickstart
The same concepts are used for building out modules. However, I would consider not using the module structure until you've had experience building some basic applications with Laravel on its own.


[CS] Fixes #2517 improve RemoveUselessClassCommentFixer : remove useless constructor comment

Alternative for https://github.com/symplify/symplify/pull/2518

Thank you :+1:
Failing CI is unrelated, with circular dependencies.


No RestartRound if Round Id changed

Fixes this
https://github.com/space-wizards/space-station-14/issues/5874

This should probably just check if it's already ending round at the beginning of the EndRound method.

I'm not sure it's centralized enough for that to work, without changing a lot more. That section looked messy, but I could be wrong.

The issue wasn't autoclosed because the PR desc isn't formatted properly to autoclose it.


Pinch to zoom

Very nice update using the animated library! It really is smoother and lighter!
Do you think it would be possible - just like facebook lightbox - to enable a pinch to zoom?
Have a great day

Hi, yeah it's on my todo. Thinking of maybe writing a carousel to go with it too. Just need some time to do it ;-)

Awesome news then! Could you ping me when it's up?
Cheers

Would be great to have this too

+1

:+1:

+1

+1

+1

I'm feeling the pressure haha. I've implemented this in the navigator-refactor branch.
See https://github.com/oblador/react-native-lightbox/issues/8#issuecomment-160352712 for more info.

Hi @oblador, I'm getting this to work fine on iOS but not on Android. When I click to zoom, it zooms in but doesn't pinch to zoom. It also scrolls but doesn't drag left to right to see details cut off from initial zoom. I'm using your navigator example and also cloned the navigator-refactor branch during npm install. Any help is appreciated. Thanks!

@kwokster10: I'm aware of that issue, that's because ScrollView on android doesn't support zooming. I'm not sure if I should implement zooming for android completely custom or just wait for support to land in React Native core.

Oh, interesting. Is there a workaround for this? Would using  make a difference?  Thanks!

@kwokster10: ViewPagerAndroid might help with paging the images but I don't think it'll help with zooming. Should be able to do zooming with scale transforms, but it wouldn't be as fluid as native zooming nor give you higher detail when zooming.

+1, thank you and keep up the good work, @oblador

Hello, i couldn't get to work the version from the navigator refactor branch.. I get the following error:

Element type is invalid: expected a string but got:object. Check the render method of image.ios.js

The main branch works without problems for me.

For those who can't get the navigator-refactor working for some reason but would like zooming,
Wrapping an image in a ScrollView and setting swipeToDismiss to false does the trick (assuming you prioritize zooming above swipeToDismiss)
<LightBox
        key={i}
        swipeToDismiss={false}
      >
        <ScrollView
          minimumZoomScale={1}
          maximumZoomScale={2}
          centerContent={true}
        >
          <Image
            style={styles.image}
            source={{'uri' : source}}
          >

          </Image>
        </ScrollView>
      </LightBox>

+1 .. waiting eagerly :)

+1 mee too... On the navigator refactor branch I am getting the same error as @pvlvsk

@oblador: just curious about timing, are you still working on this?

@braco: Not actively currently but I will soon need this functionality for a project I'm working on, but with NavigationExperimental so consider the navigator branch dead. Hopefully I'll be able to release something more generic for NavigationExperimental but I can't promise it right nor do I have a timeplan.

@oblador: thanks for the clarification.

Thanks @oblador! Awesome Work :)

Should it work on iOS simulator? It's not working for me on iOS simulator.

Would someone using the pinch to zoom feature on the navigator-refactor branch be willing to help me with this? http://stackoverflow.com/questions/40264486/including-a-library-from-a-github-branch-in-react-native?noredirect=1#comment67789465_40264486
Not sure what I'm doing wrong. Thanks in advance.

@oblador for zoom, consider looking at this library https://github.com/alwx/react-native-photo-view.

Is there a stable version of the pinch to zoom now?

+1 ... really need / love to have this feature :)

+1 !

+1

+1

+1

+1

+1

+1

@kobePer ,
Hey Kobeper, the above-mentioned code was working completely in ios but in android zooming is not working, can you suggest any update on top of it?
Thank you!.

+1

+1

+1

@kobePer,
the above-mentioned code is working fine on IOS devices but not on Android devices!

+1

+1

+1

+1, would be great to have this

Any updates?

@kobePer Your solution will only work for IOS any work around for android ?

+1

any update ?

Any update ?

+1


Automated Package Upgrade

:tada: This PR is included in version 25.4.56 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Fix issues with Windows

Windows KiCad is weird and awful and requires a prepackaged version of Python 2.7.
The changes are twofold: Remove the Python 3 only string formatters, and add installation instructions for an older version of YAML that supports the deprecated version of Python.

Thanks for this! Sorry I missed it until now!


Not using Tools in Langchain_Ollama causes: 'NoneType' object is not iterable

Checked other resources

[X] I added a very descriptive title to this issue.
[X] I searched the LangChain documentation with the integrated search.
[X] I used the GitHub search to find a similar question and didn't find it.
[X] I am sure that this is a bug in LangChain rather than my code.
[X] The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code
from langchain_ollama import ChatOllama
llm = ChatOllama(
model="llama3.1",
temperature=0,
disable_streaming=True
)
messages = [
(
"system",
"You are a helpful assistant that translates English to French. Translate the user sentence.",
),
("human", "I love programming."),
]
ai_msg = llm.invoke(messages)
print(ai_msg)
Error Message and Stack Trace (if applicable)
Traceback (most recent call last):
File "/home/user1/Repo/ollama-cloud/use4.py", line 16, in
ai_msg = llm.invoke(messages)
File "/home/user1/Repo/ollama-cloud/venv/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py", line 286, in invoke
self.generate_prompt(
File "/home/user1/Repo/ollama-cloud/venv/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py", line 786, in generate_prompt
return self.generate(prompt_messages, stop=stop, callbacks=callbacks, **kwargs)
File "/home/user1/Repo/ollama-cloud/venv/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py", line 643, in generate
raise e
File "/home/user1/Repo/ollama-cloud/venv/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py", line 633, in generate
self._generate_with_cache(
File "/home/user1/Repo/ollama-cloud/venv/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py", line 851, in _generate_with_cache
result = self._generate(
File "/home/user1/Repo/ollama-cloud/venv/lib/python3.10/site-packages/langchain_ollama/chat_models.py", line 648, in _generate
final_chunk = self._chat_stream_with_aggregation(
File "/home/user1/Repo/ollama-cloud/venv/lib/python3.10/site-packages/langchain_ollama/chat_models.py", line 560, in _chat_stream_with_aggregation
tool_calls=_get_tool_calls_from_response(stream_resp),
File "/home/user1/Repo/ollama-cloud/venv/lib/python3.10/site-packages/langchain_ollama/chat_models.py", line 71, in _get_tool_calls_from_response
for tc in response["message"]["tool_calls"]:
TypeError: 'NoneType' object is not iterable
Description
Trying to simple get a chat response from ChatOllama.
System Info
System Information

OS:  Linux
OS Version:  #49~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Wed Nov  6 17:42:15 UTC 2
Python Version:  3.10.12 (main, Nov  6 2024, 20:22:13) [GCC 11.4.0]

Package Information

langchain_core: 0.3.20
langchain: 0.3.7
langchain_community: 0.3.7
langsmith: 0.1.144
langchain_ollama: 0.2.0
langchain_text_splitters: 0.3.2

Optional packages not installed

langserve

Other Dependencies

aiohttp: 3.11.7
async-timeout: 4.0.3
dataclasses-json: 0.6.7
httpx: 0.27.2
httpx-sse: 0.4.0
jsonpatch: 1.33
numpy: 1.26.4
ollama: 0.4.0
orjson: 3.10.11
packaging: 24.2
pydantic: 2.10.1
pydantic-settings: 2.6.1
PyYAML: 6.0.2
requests: 2.32.3
requests-toolbelt: 1.0.0
SQLAlchemy: 2.0.35
tenacity: 9.0.0
typing-extensions: 4.12.2

we had another issue that was similar to this one, maybe try python response["message"]["tool_calls"] is not None and see if that works

Yes, that is how I would also fix the bug in the langchain implementation

Yes, that is how I would also fix the bug in the langchain implementation

did it work?

I have the same error.
change line 69 in chat_models.py helps
from
    if "message" in response:
        if "tool_calls" in response["message"]:
            for tc in response["message"]["tool_calls"]:
                tool_calls.append(

to
    if "message" in response:
        if "tool_calls" in response["message"] and response["message"]["tool_calls"]:
            for tc in response["message"]["tool_calls"]:
                tool_calls.append(

@bauerem please upgrade to latest version of langchain-ollama. It should be fix the issue.

Same error here:
Package Information

ollama 0.4.1
langchain 0.3.8
langchain-community 0.3.8
langchain-core 0.3.21
langchain-ollama 0.2.0

Code example
from langchain_ollama import ChatOllama

model = ChatOllama(model="llama3.2", temperature=0)
model.invoke("Chi è il presidente degli Stati Uniti?")

Error

TypeError                                 Traceback (most recent call last)
Cell In[34], line 4
1 from langchain_ollama import ChatOllama
3 model = ChatOllama(model="llama3.2", temperature=0)
----> 4 model.invoke("Chi è il presidente degli Stati Uniti?")
File ~/aidev/pdf-rag/.venv/lib/python3.13/site-packages/langchain_core/language_models/chat_models.py:286, in BaseChatModel.invoke(self, input, config, stop, **kwargs)
275 def invoke(
276     self,
277     input: LanguageModelInput,
(...)
281     **kwargs: Any,
282 ) -> BaseMessage:
283     config = ensure_config(config)
284     return cast(
285         ChatGeneration,
--> 286         self.generate_prompt(
287             [self._convert_input(input)],
288             stop=stop,
289             callbacks=config.get("callbacks"),
290             tags=config.get("tags"),
291             metadata=config.get("metadata"),
292             run_name=config.get("run_name"),
293             run_id=config.pop("run_id", None),
...
76                 )
77             )
78 return tool_calls
TypeError: 'NoneType' object is not iterable

@keenborder Latest version doesn't seem to have the None check.

@nourishnew yeap you are correct I have fixed it it in the PR. Hopefully in the next release it will be fixed.