[Feature/Identity] Initial interface for native authentication support

Description
This PR targets the branch opensearch-project:feature/identity)
Adding a noop implementation of an authentication manager for use tracking identity information within the OpenSearch systems.  This serves as a foundation for additional authentication and authorization systems that are in flight.  For more context on a larger design consult the draft pull request that uses an open source 3rd party authentication and authorization system.
Also see

https://github.com/opensearch-project/OpenSearch/issues/4514
https://github.com/opensearch-project/OpenSearch/issues/3846
https://github.com/opensearch-project/OpenSearch/pull/4247
https://github.com/opensearch-project/opensearch-sdk-java/blob/main/SECURITY.md

Issues Resolved

Related https://github.com/opensearch-project/OpenSearch/issues/4514

Check List

[X] New functionality includes testing.

[X] All tests pass

[X] New functionality has been documented.

[X] New functionality has javadoc added

[X] Commits are signed per the DCO using --signoff
[x] Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

I think you should put this on a feature branch and keep iterating.
For main, I believe we need at least the following:

security manager is established at all entry/start points
identity reachable in the entire codebase
noop checks for permissions everywhere

I think you should put this on a feature branch and keep iterating.

Done!
@cwperks @DarshitChanpura could you take a look? I've moved this over to our branch so we can move faster, but I'd still like make sure we are code reviewing as we go

Codecov Report

Merging #4515 (bbd60c1) into feature/identity (51a529f) will decrease coverage by 0.44%.
The diff coverage is 76.74%.

@@                  Coverage Diff                   @@
##             feature/identity    #4515      +/-   ##
======================================================
- Coverage               71.08%   70.64%   -0.45%
+ Complexity              57679    57289     -390
======================================================
  Files                    4617     4622       +5
  Lines                  275494   275531      +37
  Branches                40328    40333       +5
======================================================
- Hits                   195846   194647    -1199
- Misses                  63609    64531     +922
- Partials                16039    16353     +314

Impacted Files
Coverage Δ

.../java/org/opensearch/index/mapper/FieldMapper.java
80.74% <30.00%> (-0.71%)
:arrow_down:

...java/org/opensearch/identity/noop/NoopSubject.java
75.00% <75.00%> (ø)

.../java/org/opensearch/identity/StringPrincipal.java
90.00% <90.00%> (ø)

server/src/main/java/org/opensearch/Version.java
80.09% <100.00%> (+0.09%)
:arrow_up:

...rc/main/java/org/opensearch/identity/Identity.java
100.00% <100.00%> (ø)

.../main/java/org/opensearch/identity/Principals.java
100.00% <100.00%> (ø)

...earch/identity/noop/NoopAuthenticationManager.java
100.00% <100.00%> (ø)

server/src/main/java/org/opensearch/node/Node.java
86.37% <100.00%> (+0.04%)
:arrow_up:

...adcast/BroadcastShardOperationFailedException.java
55.55% <0.00%> (-44.45%)
:arrow_down:

.../search/aggregations/pipeline/HoltLinearModel.java
20.33% <0.00%> (-42.38%)
:arrow_down:

... and 470 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Schemas disappear after some time but not in the metadata

After some time saved (about a week), schemas disappear from postgres but metadata is kept.
In this picture, material schema is missing:

It also happened  in protobuf schema type.
PS: Please, feel free to change the title. It was difficult to explain it shortly.

Default configuration was file system storage, in a temporary directory. That will be changed to database storage.


Porting base types content from MSDN

Previously reviewed; moving in stages (59 total topics)

Open Publishing Build Service: The pull request content has been published and here are some sample preview links:

Docset 1: core-docs

Open Publishing Build Service: The pull request content has been published and here are some sample preview links:

Docset 1: core-docs

Open Publishing Build Service: The pull request content has been published and here are some sample preview links:

Docset 1: core-docs

Open Publishing Build Service: The pull request content has been published and here are some sample preview links:

Docset 1: core-docs

Open Publishing Build Service: The pull request content has been published and here are some sample preview links:

Docset 1: core-docs

Open Publishing Build Service: The pull request content has been published and here are some sample preview links:

Docset 1: core-docs

Open Publishing Build Service: The pull request content has been published and here are some sample preview links:

Docset 1: core-docs

Open Publishing Build Service: The pull request content has been published and here are some sample preview links:

Docset 1: core-docs

Last commit didn't build - closing and reopening to trigger another build.

Open Publishing Build Service: The pull request content has been published and here are some sample preview links:

Docset 1: core-docs

Open Publishing Build Service: The pull request content has been published and here are some changed files links of this time:

Docset 1 - core-docs
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.reflection.ireflect?branch=pr-en-us-1016
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.reflection.ireflectabletype?branch=pr-en-us-1016
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.reflection.localvariableinfo?branch=pr-en-us-1016
Please see 3488 more published files of this docset in report.

Status: SucceededWithWarning. Message: Warning occurred: Unable to resolve cross-reference "DecoderFallbackBuffer.Fallback"..
Open Publishing Report.

Open Publishing Build Service: The pull request content has been published and here are some changed files links of this time:

Docset 1 - core-docs
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.tuple-6?branch=pr-en-us-1016
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.tuple-7?branch=pr-en-us-1016
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.tuple-8?branch=pr-en-us-1016
Please see 3094 more published files of this docset in report.

Status: SucceededWithWarning. Message: Warning occurred: Unable to resolve cross-reference "String.Format(IFormatProvider, String, Object[])", "ICustomFormatter.Format", "String.Format(IFormatProvider, String, Object[])", "String.Format(IFormatProvider, String, Object[])", "String.Format(IFormatProvider, String, Object[])", "String.Format(IFormatProvider, String, Object[])", "ICustomFormatter.Format", "ICustomFormatter.Format", "String.Format(IFormatProvider, String, Object[])"..
Open Publishing Report.

Open Publishing Build Service: The pull request content has been published and here are some changed files links of this time:

Docset 1 - core-docs
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.threading.synchronizationlockexception?branch=pr-en-us-1016
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.threading.tasks.concurrentexclusiveschedulerpair?branch=pr-en-us-1016
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.threading.tasks.parallel?branch=pr-en-us-1016
Please see 3092 more published files of this docset in report.

Status: SucceededWithWarning. Message: Warning occurred: Invalid file link:(~/docs/standard/basetypes/backreference.md.). Referenced by file: docs/standard/basetypes/grouping.md at line: 0...
Open Publishing Report.

Open Publishing Build Service: The pull request content has been published and here are some changed files links of this time:

Docset 1 - core-docs
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.genericuriparseroptions?branch=pr-en-us-1016
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.globalization.calendar?branch=pr-en-us-1016
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.globalization.calendaralgorithmtype?branch=pr-en-us-1016
Please see 3089 more published files of this docset in report.

Status: SucceededWithWarning. Message: Warning occurred: Unable to resolve cross-reference "Enum.ToString"..
Open Publishing Report.

Open Publishing Build Service: The pull request content has been published and here are some changed files links of this time:

Docset 1 - core-docs
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.componentmodel.composition.compositioncontractmismatchexception?branch=pr-en-us-1016
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.componentmodel.composition.compositionerror?branch=pr-en-us-1016
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.componentmodel.composition.compositionexception?branch=pr-en-us-1016
Please see 3127 more published files of this docset in report.

Status: SucceededWithWarning. Message: Warning occurred: Invalid file link:(~/docs/standard/basetypes/@System.String). Referenced by file: docs/standard/basetypes/manipulatingstrings.md at line: 0...
Open Publishing Report.

Build failed - closing and reopening to trigger build

Open Publishing Build Service: The pull request content has been published and here are some changed files links of this time:

Docset 1 - core-docs
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.asynccallback?branch=pr-en-us-1016
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.attribute?branch=pr-en-us-1016
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.attributetargets?branch=pr-en-us-1016
Please see 3128 more published files of this docset in report.

Status: SucceededWithWarning. Message: Warning occurred: Invalid file link:(~/docs/standard/basetypes/System.Text.RegularExpressions.MatchCollection.Item(System.Int32)). Referenced by file: docs/standard/basetypes/objectmodel.md at line: 0...
Open Publishing Report.

Open Publishing Build Service: The pull request content has been published and here are some changed files links of this time:

Docset 1 - core-docs
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.contextstaticattribute?branch=pr-en-us-1016
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.convert?branch=pr-en-us-1016
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.converter-2?branch=pr-en-us-1016
Please see 3124 more published files of this docset in report.

Status: SucceededWithWarning. Message: Warning occurred: Invalid file link:(~/docs/standard/basetypes/System.Text.RegularExpressions.MatchCollection). Referenced by file: docs/standard/basetypes/objectmodel.md at line: 0...
Open Publishing Report.

Open Publishing Build Service: The pull request content has been published and here are some changed files links of this time:

Docset 1 - core-docs
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.collections.generic.stack-1.enumerator?branch=pr-en-us-1016
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.collections.generic.stack-1?branch=pr-en-us-1016
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.collections.generic.synchronizedcollection-1?branch=pr-en-us-1016
Please see 3387 more published files of this docset in report.

Status: SucceededWithWarning. Message: Warning occurred: Invalid file link:(~/docs/standard/basetypes/System.Text.RegularExpressions.MatchCollection). Referenced by file: docs/standard/basetypes/objectmodel.md at line: 0...
Open Publishing Report.

Open Publishing Build Service: The pull request content has been published and here are some changed files links of this time:

Docset 1 - core-docs
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.runtime.interopservices.comtypes.ienummoniker?branch=pr-en-us-1016
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.runtime.interopservices.comtypes.ienumstring?branch=pr-en-us-1016
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.runtime.interopservices.comtypes.ienumvariant?branch=pr-en-us-1016
Please see 3094 more published files of this docset in report.

Status: SucceededWithWarning. Message: Warning occurred: Invalid file link:(~/docs/standard/basetypes/9). Referenced by file: docs/standard/basetypes/quickref.md at line: 0...
Open Publishing Report.

Open Publishing Build Service: The pull request content has been published and here are some changed files links of this time:

Docset 1 - core-docs
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.data.services.client.dataservicerequest?branch=pr-en-us-1016
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.data.services.client.dataservicerequestargs?branch=pr-en-us-1016
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.data.services.client.dataservicerequestexception?branch=pr-en-us-1016
Please see 3084 more published files of this docset in report.

Status: SucceededWithWarning. Message: Warning occurred: Invalid file link:(~/docs/standard/basetypes/9). Referenced by file: docs/standard/basetypes/quickref.md at line: 0...
Open Publishing Report.

Open Publishing Build Service: The pull request content has been published and here are some changed files links of this time:

Docset 1 - core-docs
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.data.sqlclient.sqlnotificationtype?branch=pr-en-us-1016
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.data.sqlclient.sqlparameter?branch=pr-en-us-1016
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.data.sqlclient.sqlparametercollection?branch=pr-en-us-1016
Please see 3093 more published files of this docset in report.

Status: SucceededWithWarning. Message: Warning occurred: Invalid file link:(~/docs/standard/basetypes/regularexpressions). Referenced by file: docs/standard/basetypes/threadsafety.md at line: 0...
Open Publishing Report.

Open Publishing Build Service: The pull request content has been published and here are some changed files links of this time:

Docset 1 - core-docs
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.runtime.interopservices.comunregisterfunctionattribute?branch=pr-en-us-1016
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.runtime.interopservices.comvisibleattribute?branch=pr-en-us-1016
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.runtime.interopservices.connectdata?branch=pr-en-us-1016
Please see 3087 more published files of this docset in report.

Status: Succeeded.
Open Publishing Report.

Open Publishing Build Service: The pull request content has been published and here are some changed files links of this time:

Docset 1 - core-docs
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.componentmodel.dataannotations.fileextensionsattribute?branch=pr-en-us-1016
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.componentmodel.dataannotations.filteruihintattribute?branch=pr-en-us-1016
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.componentmodel.dataannotations.ivalidatableobject?branch=pr-en-us-1016
Please see 3122 more published files of this docset in report.

Status: Succeeded.
Open Publishing Report.

Open Publishing Build Service: The pull request content has been published and here are some changed files links of this time:

Docset 1 - core-docs
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.reflection.methodinfo?branch=pr-en-us-1016
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.reflection.missing?branch=pr-en-us-1016
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.reflection.module?branch=pr-en-us-1016
Please see 3106 more published files of this docset in report.

Status: Succeeded.
Open Publishing Report.

Open Publishing Build Service: The pull request content has been published and here are some changed files links of this time:

Docset 1 - core-docs
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.threading.threadstateexception?branch=pr-en-us-1016
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.threading.timeout?branch=pr-en-us-1016
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.threading.timer?branch=pr-en-us-1016
Please see 3113 more published files of this docset in report.

Status: Succeeded.
Open Publishing Report.

Open Publishing Build Service: The pull request content has been published and here are some changed files links of this time:

Docset 1 - core-docs
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.componentmodel.dataannotations.schema?branch=pr-en-us-1016
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.componentmodel.dataannotations.stringlengthattribute?branch=pr-en-us-1016
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.componentmodel.dataannotations.timestampattribute?branch=pr-en-us-1016
Please see 3114 more published files of this docset in report.

Status: Succeeded.
Open Publishing Report.

Open Publishing Build Service: The pull request content has been published and here are some changed files links of this time:

Docset 1 - core-docs
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.globalization.umalquracalendar?branch=pr-en-us-1016
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.globalization.unicodecategory?branch=pr-en-us-1016
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.globalization?branch=pr-en-us-1016
Please see 3180 more published files of this docset in report.

Status: Succeeded.
Open Publishing Report.

Open Publishing Build Service: The pull request content has been published and here are some changed files links of this time:

Docset 1 - core-docs
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.componentmodel.composition?branch=pr-en-us-1016
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.componentmodel.container?branch=pr-en-us-1016
https://stage.docs.microsoft.com/en-us/dotnet/core/api/system.componentmodel.containerfilterservice?branch=pr-en-us-1016
Please see 3118 more published files of this docset in report.

Status: Succeeded.
Open Publishing Report.

Open Publishing Build Service: The pull request content has been published and here are some changed files links of this time:

Docset 1 - core-docs
https://review.docs.microsoft.com/en-us/dotnet/articles/standard/basetypes/alternation?branch=pr-en-us-1016
https://review.docs.microsoft.com/en-us/dotnet/articles/standard/basetypes/anchors?branch=pr-en-us-1016
https://review.docs.microsoft.com/en-us/dotnet/articles/standard/basetypes/backreference?branch=pr-en-us-1016
Please see 1 more published files of this docset in report.

Status: Succeeded.
Open Publishing Report.

Open Publishing Build Service: The pull request content has been published and here are some changed files links of this time:

Docset 1 - core-docs
https://review.docs.microsoft.com/en-us/dotnet/articles/standard/basetypes/alternation?branch=pr-en-us-1016
https://review.docs.microsoft.com/en-us/dotnet/articles/standard/basetypes/anchors?branch=pr-en-us-1016
https://review.docs.microsoft.com/en-us/dotnet/articles/standard/basetypes/backreference?branch=pr-en-us-1016
Please see 1 more published files of this docset in report.

Status: Succeeded.
Open Publishing Report.

Has rendering issues that can't be fixed. Resubmitted as #1053


chore(main): release 0.7.0

:robot: I have created a release beep boop
0.7.0 (2024-08-06)
Features

support for AI Platform version 1.0.0 (05d1d8c)
update demo data (46ac618)

This PR was generated with Release Please. See documentation.

:robot: Created releases:

v0.7.0
:sunflower:


CI for Node

Creating node.js template for testing

Initializes with two players (3 ms)

Initializes with two players (3 ms)


Support DatabaseId on FirestoreDb

Google Cloud Firestore now supports multiple database, but FirestoreDb still hides the parameter in an internal method: https://github.com/googleapis/google-cloud-dotnet/blob/main/apis/Google.Cloud.Firestore/Google.Cloud.Firestore/FirestoreDb.cs#L134
The databaseId parameter should be exposed to developers.

The feature is still in preview. We will make it public when the feature is GA.

Hello @jskeet ,
is it the same for DatastoreDb[Builder]. I have :

i am trying to write to the orders db ( in datastore mode) but it still writes to (default). Can I change the db I am writing to ? or should I wait for the GA. If it's the case, the multi db is unusable for the moment.

No, Datastore support for multiple databases is GA (I believe) - but we haven't implemented it yet in the client library. It's literally being worked on at the moment.

If anyone needs to instantiate a FirestoreDb for the non-default database, you can do it with this bit of reflection code:
var firestoreClient = await new FirestoreClientBuilder().BuildAsync();
var create = typeof(FirestoreDb).GetMethod(
    "Create",
    BindingFlags.Static | BindingFlags.NonPublic | BindingFlags.Public,
    new Type[]
    {
        typeof(string),
        typeof(string),
        typeof(FirestoreClient),
        typeof(Action<string>),
        typeof(ConverterRegistry)
    });
var db = (FirestoreDb)create!.Invoke(
    null,
    new object[]
    {
        Platform.Instance().ProjectId,
        "REPLACE_WITH_YOUR_DATABASE_ID_HERE",
        firestoreClient,
        null!,
        null!,
    })!;

Correction to my previous statement: I'm afraid Datastore multi-db support is also in preview at the moment. We'll look at options for release beta versions of the libraries to expose the features, but we definitely won't release GA libraries with this in until the feature goes GA.

The 3.5.0-beta01 release for Firestore is now in progress. I hope we'll be able to release 3.5.0 early in 2024.


iOS VoiceOver Rotor support

We should make sure that Flutter applications are able to make full use of the iOS VoiceOver Rotor and that all our built-in widgets fully support the features of the VoiceOver Rotor.
https://developer.apple.com/videos/play/wwdc2020/10116/
https://support.apple.com/en-us/HT204783
https://developer.apple.com/documentation/objectivec/nsobject/1649788-accessibilitycustomrotors
https://developer.apple.com/documentation/uikit/uiaccessibilitycustomrotor

@chunhtai I seem to recall that you did some work towards this. Do you know what the status is?

No I don't recall I have done anything in this space.

Its worth mentioning that Google's TalkBack now has a feature that's quite similar. This should be implemented as well.

It might be helpful to break this request into several issues. I think this issue is too generic and cannot be implemented as described
I will start a new issue on custom actions specifically, because there is a pretty good parity between iOS and Android as far as that feature is concerned.
Unfortunately, the web is a bit more complicated, but we can handle that case separately.
Suffice it to say that adding custom actions support to Flutter would definitely help developers to make their apps more accessible.
, .

Is the rotor different from the radial context menu on iOS? We do support custom actions that - according to the docs - on iOS are surfaced in the "radial context menu" and on Android in the local context menu, see https://master-api.flutter.dev/flutter/semantics/CustomSemanticsAction-class.html.
Can someone clarify if that's exactly what this issue is asking for or specify how this is different / what is missing from our current support?


Mailhog does not register emails with newly defined paths

OS type
Linux
Drupal version
9
Codebase
Using drupal contrib module "mail system" php sender
Describe the bug
With newest versions of all containers mailhog does not register emails with newly defined paths
Solution
With this config mailhog does not work:
environment:
  SSMTP_MAILHUB: mailhog:1025
#  SSMTP_MAILHUB: opensmtpd:25
  PHP_SENDMAIL_PATH: '"/usr/bin/dos2unix -u | /usr/sbin/ssmtp -f -t"'

But when I updated evironment variables to this it worked as expected:
environment:
  SSMTP_MAILHUB: mailhog:1025
#  SSMTP_MAILHUB: opensmtpd:25
  PHP_SENDMAIL_PATH: /usr/sbin/sendmail -t -i -S

Thank you for reporting this, the problem was in ssmtp flags, it should -t -f, not -f -t


vite configuration guide

Hello, a small guide for vite with the "nwidart/laravel-modules" package.

edit package.json

Modules\User\Resources\assets\js\app.js add "import './bootstrap';".
Modules\User\Resources\assets\js\bootstrap.js

Modules\User\Resources\assets\sass\app.scss (optional variables)

Modules\User\Resources\views\layouts\master.blade.php (I recommend that you make helper, for cleaner code :) )

Modules\User\vite.config.js (I have found 4 ways, but this is the best example I think.), "Omit those in red box".

Modules\User\webpack.mix.js (Delete webpack.mix.js )
Thank you for your attention, hoping you will accept the merge. https://github.com/nWidart/laravel-modules/pull/1455
@dcblogdev @kylecmccandless

I assume your adding this bootstrap code here as a POC? Laravel has taken tailwind as its primary CSS framework and for compatibility I would suggest sticking with tailwind. Do you have a way to build all from the root vite file as to not have to run npm build in an undetermined number of modules? In a CI pipeline this would be a huge pain.
@wikigods @dcblogdev

Cool @wikigods but why images ? not a plain text ? :D


Usability issue with Kraken - no warnings for missing websocket persmissions with default API keys settings

Describe the bug
When a Kraken user creates an API key with default settings - websocket permissions are disabled. This causes the Kraken connector to not be able to reconstruct order book, and thus it cannot trade.
Currently, the Kraken connector would just give out an error logs like the following:
2024-01-25 21:19:12,916 - 111864 - hummingbot.connector.exchange.kraken.kraken_api_user_stream_data_source - ERROR - Unexpected error with Kraken WebSocket connection. Retrying after 30 seconds...
Traceback (most recent call last):
  File "/home/martin_kou/Development/hummingbot/hummingbot/connector/exchange/kraken/kraken_api_user_stream_data_source.py", line 100, in listen_for_user_stream
    self._current_auth_token = await self.get_auth_token()
  File "/home/martin_kou/Development/hummingbot/hummingbot/connector/exchange/kraken/kraken_api_user_stream_data_source.py", line 89, in get_auth_token
    return response_json["result"]["token"]
KeyError: 'result'

...

  File "hummingbot/connector/exchange_base.pyx", line 158, in hummingbot.connector.exchange_base.ExchangeBase.c_get_price
    OrderBook order_book = self.c_get_order_book(trading_pair)
  File "hummingbot/connector/exchange/kraken/kraken_exchange.pyx", line 1050, in hummingbot.connector.exchange.kraken.kraken_exchange.KrakenExchange.c_get_order_book
    raise ValueError(f"No order book exists for '{trading_pair}'.")
ValueError: No order book exists for 'BTC-USD'.

This gives the impression to users that the Kraken connector is broken, and is unable to get prices from the exchange. However, if the user enables websocket permissions on Kraken side, then the Kraken connector will work.
The websocket error messages on Kraken should tell users to check their API key's websocket permissions instead. If possible, the connect kraken command should warn the user to enable websocket permissions during API key setup so the user is aware of this requirement from the beginning.
Steps to reproduce

Set up an API key on Kraken exchange with default settings.
Connect to the Kraken exchange on Hummingbot, with the generated API key.
Create a simple pure market making strategy with any common pair on Kraken (e.g. BTC-USD).
Start the strategy.
Observe the error logs and the bot's inability to fetch prices from Kraken.

Release version
v1.24.0
Type of installation
Source
Attach required files
No response

@david-hummingbot can you modify the Kraken documentation page to inform users of the need to add WebSocket when creating API keys?

Updated docs about setting up Kraken API keys on Hummingbot


WIP: Method to find Zarr Array checksum

Fixes https://github.com/alimanfoo/zarr/issues/98
Provides a method for Zarr Array's to compute their checksum. Uses the sha1 checksum by default thanks to its speed and low probability of collisions. Though the user can change the checksum to any supported by their version of openssl (per hashlib docs) should they desire to. The hashing is done on the data in the chunk store (so encoded) and the metadata (shape, dtype, order, filtering, and attributes).

Thanks for taking a look. Happy to make tweaks. Added a couple comments inline.
As a side note, would like to use this to implement __dask_tokenize__ as well.

Re .zattrs, FWIW I think you can rely on that being small enough to pull into memory, any attribute key update requires loading the whole JSON document into memory anyway.

Re __dask_tokenize__ I was just looking at the deterministic hashing docs, but doesn't actually say what the hashing is used for. Is this to do with caching intermediate values in a dask computation?

It's used for things like constructing the name of the Dask Array created by from_array. Though I agree that should be more explicit in the docs (particularly from_array).
Really any time something calls tokenize on an object it ends up searching for __dask_tokenize__ or a function registered with normalize_token.

@jcrist can probably clarify this better than I though. 😉

Hmm...the checksums I get locally do not match with those the CIs seem to be getting. 😟
Also I'm a little surprised that the checksums of TestArrayWithPath and TestArrayWithChunkStore cases do not match the TestArray case for me locally. Wouldn't expect storage location or format to have an effect, but it seems to. 😕
Any thoughts on either of these would be appreciated.

Looks like the checksum mismatch issues are restricted to Python 2 only. Not sure of the cause yet, but could be some subtlety with how the data is represented in Python 2 as opposed to Python 3. The concerns were determined to be non-issues.

Turns out that an extra space is added right before newlines (after commas) in the JSON for things like .zarray on Python 2, but this does not occur with Python 3. Verified this is not a CR character, but actually a space (at least on my Mac).

Hm, the PY2/PY3 JSON issue is annoying. It would be ideal to get the same hexdigest on all platforms. Not sure what to suggest, will give it some thought.
Re __dask_tokenize__ I wonder how much it matters if the value is potentially expensive to compute. E.g., if it gets called every time you do da.from_array(z) and z is a large array and/or an array with relatively slow storage (e.g., S3), the delay to compute the hexdigest could make it unusable.

If you set indent=None for all calls to json.dump, does this give consistent JSON in PY2 and PY3? It's supposed to be the most compact representation. I'd be happy to sacrifice readability for consistency.

Found the magic sauce as far as Python 2/3 JSON goes. Turns out there was a subtle change in Python 3.4 to the default separator behavior (referenced below). If we just override separator to match the Python 3.4+ behavior, the issue goes away for Python 2. Pushed a commit to do exactly that. Happy to make the commit message more expressive given this subtlety.

Changed in version 3.4: Use (',', ': ') as default if indent is not None.

ref: https://docs.python.org/3/library/json.html#json.dump

Should add IDK if we want something like digest provided by hashlib algorithms, but it is probably worth considering at this point. Basically it provides the binary string equivalent to the more verbose hexdigest.

Re __dask_tokenize__ I wonder how much it matters if the value is potentially expensive to compute. E.g., if it gets called every time you do da.from_array(z) and z is a large array and/or an array with relatively slow storage (e.g., S3), the delay to compute the hexdigest could make it unusable.

So I haven't done any benchmarking yet. It would be good to do this for sure. Have access to slow storage (e.g. NFS) to do some experimenting on. Can explore that more tomorrow.
For more context, am wanting to explore caching long running computational results to disk. As part of this, I'd like the .name attribute of Dask Arrays to be consistent between runs (if no parameters or inputs have changed). So using some sort of hashing for __dask_tokenize__ would be ideal. Since Dask chose SHA1 as a default for NumPy Arrays, figured it was a safe bet here. They also try using things like CityHash and MurmurHash, which could be worth looking at if we don't mind another dependency. Though am open to other stable hash options if you have any in mind.

Looks good.

FWIW SHA1 seems fine to me.
If you'd like to add digest() for completeness that's OK by me, fine either way (I'll probably only use hexdigest()).
Re __dask_tokenize__ I think in the general case we have to assume that computing a full hash may be slow simply due to I/O, either because of very large data or slow storage or both. By slow I mean too slow for interactive work. So I'm all for supporting caching, as long as we don't interrupt the flow for the average dask user.

Ok. I'm not planning to use digest either. So let's leave it out. After all people can just call binascii.unhexlify to convert anyways.
That may be true. Though was hoping since we avoided decompression, this might be less of an issue. Planning on playing with this more tomorrow. In any event, __dask_tokenize__ is not pressing as one can still use normalize_token.register to define their own way to do this with Zarr or anything else for this matter.

Added in a changelog entry as well. This is good to go on my end.

FWIW tried running hexdigest with the default sha1 on some real data to get an idea of how long this might take. Information about the dataset and the time taken to compute the hash are included below. This was run on a Scientific Linux 7.3 cluster using NFSv3.
>>> imgs.info_items()
[('Name', '/images'),
 ('Type', 'zarr.core.Array'),
 ('Data type', 'uint16'),
 ('Shape', '(1636, 1016, 1262)'),
 ('Chunk shape', '(103, 127, 158)'),
 ('Order', 'C'),
 ('Read-only', 'True'),
 ('Compressor', "Blosc(cname='lz4', clevel=5, shuffle=SHUFFLE, blocksize=0)"),
 ('Store type', 'zarr.storage.DirectoryStore'),
 ('No. bytes', '4195332224 (3.9G)'),
 ('No. bytes stored', '2569038619 (2.4G)'),
 ('Storage ratio', '1.6'),
 ('Chunks initialized', '1024/1024')]
>>> %timeit -n 10 imgs.hexdigest()
4.66 s ± 23.6 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

Many thanks, nice feature.

Thanks. We might want to take a look at BLAKE2 down the road as that would provide the option for tree hashing, which seems very nice to use with something like Zarr.

Found some value in having a digest method as well. So put together an implementation in PR ( https://github.com/alimanfoo/zarr/pull/213 ).


Chinese translation not working for Django1.10 and after.

I found that in master branch the locale folder for Chinese named zh-Hans，it should be zh_Hans。
It take me for a whole afternoon to find out this problem....
But I see in the develop branch, this problem is also there!
Hopefully the author can update the master branch by renaming the locale folder zh-Hans. So I can get the write version from pip.
Thanks!

It's set as "zh-Hans" by transifex, and I am wondering why :thinking:
I will investigate, thanks for reporting


Kusto for "Computers Still Missing Updates" is wrong

The current sample tests for the following as means of detecting updates that are needed, however, UpdateState could also be "Installed" and would also be detected by the current where clause:
UpdateState != "Not needed" and UpdateState != "NotNeeded"
A correct way would be as follows:
let ComputersMissingUpdates3DaysAgo = Update
| where TimeGenerated between (ago(30d)..ago(1h))
| where Classification !has "Critical" and UpdateState =~ "Needed"
| summarize makeset(Computer);
Update
| where TimeGenerated > ago(1d)
| where Classification has "Critical" and UpdateState =~ "Needed"
| where Computer in (ComputersMissingUpdates3DaysAgo)
| summarize UniqueUpdatesCount = dcount(Product) by Computer, OSType

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 10099f08-04dd-b3f4-b833-3196ba54e4ef
Version Independent ID: 6596eef2-2b3a-4f69-1756-3d8ab7ae348a
Content: Azure Monitor log query examples
Content Source: articles/azure-monitor/log-query/examples.md
Service: log-analytics
GitHub Login: @bwren
Microsoft Alias: bwren

@luiscantero Thanks for your comment. The feedback has been shared with the content owner for further review. Thanks for your patience.

@luiscantero Thanks for this.  I tested it out and agree with your fix.  I'll get the article updated.

#please-close


Addition of other FeII and Stellar Templates

I am using BADASS for another project where I have to extract [OIII] kinematics from Keck KCWI-Blue data (FWHM ~0.950 A from 4600-5700A).
While the VC04 optical FeII template has been working fine for most purposes, there are some spectra in AGN-dominated spaxels where FeII emission is really strong (e.g. Ark 120/Mrk 1095), and I would like to try using different FeII templates for these bins. I have already tried using the K10 template provided and it seems to consistently return NaNs for the function value during the basinhopping step of the fitting. The K10 template seems to work fine when I use it with the MUSE data I have, which makes me think the issue could be something related to the KCWI resolution/FWHM.
I was wondering if it would be possible to implement some other optical FeII templates, such as the Boroson & Green (1992) template or even the recently published Mrk 493 STIS template in  Park et al. (2022).
In a related fashion, I would also like to know if it would be possible to add some higher-resolution stellar templates.  The default libraries seem to have lower resolutions than KCWI, so I had to manually add in the X-Shooter Library SSP Models, although in a limited capacity (solar metallicity, trimmed to 3800 A - 10000A, and re-binned to a linear scale, since by default the wavelengths are sampled in log10, but the resolution changes between the UV/OPT/IR bands). In principle, would it be possible to use additional libraries in prepare_stellar_templates, even if the templates themselves might not be sampled linearly in wavelength, and/or might have varying resolution?

At the time BADASS only supports the included FeII and stellar templates, but  other template libraries will be considered in future releases.  It is theoretically possible to add these libraries yourself in the same fashion as the current default templates, as long as they are prepared in the format that BADASS expects.  Similarly for the X-Shooter templates, it should be possible to implement these yourself, as long as they are in the format BADASS expects.  The main issues I can see however is that they are not linearly sampled as BADASS only supports templates that are linearly sampled.  The change in resolution also poses a problem.
The issues you have with NaNs in the function value occurs when the convolved template is not the same length as the input spectrum, and is caused by the resolution.  This is an ongoing problem we are trying to mitigate.  NaNs in the basinhopping solution also occurs when there are noise values <= 0, since this will cause your likelihood to be undefined.

From: rremigio @.>
Sent: Wednesday, May 18, 2022 9:20 PM
To: remingtonsexton/BADASS3 @.>
Cc: Subscribed @.***>
Subject: [remingtonsexton/BADASS3] Addition of other FeII and Stellar Templates (Issue #2)
I am using BADASS for another project where I have to extract [OIII] kinematics from the Keck KCWI-Blue data (FWHM ~0.950 A from 4600-5700A).
While the VC04 optical FeII template has been working fine for most purposes, there are some spectra in AGN-dominated spaxels where FeII emission is really strong (e.g. Ark 120/Mrk 1095), and I would like to try using different FeII templates for these bins. I have already tried using the K10 template provided and it seems to consistently return NaNs for the function value during the basinhopping step of the fitting. The K10 template seems to work fine when I use it with the MUSE data I have, which makes me think the issue could be something related to the KCWI resolution/FWHM.
I was wondering if it would be possible to implement some other optical FeII templates, such as the widely used Boroson & Green (1992) template or even the recently published Mrk 493 STIS template in Park et al. (2022)https://secure-web.cisco.com/14zzUXs-sDSTgR0-aED3pd6y1dLWohexEBxqMfDEQpGLX1hhqwi4rnnboh8uEqm_kH-zRckxLVVW3xcNLbmo0vTKChV2oeec1xCiH8_SkVh0ycxqYLDqeS9JQlzLGZ2jQoswbJnm-a26PZIJIz6_X2o0JZ__GEPp-BPyaXoUvFVlNIeosr-_RTLqXsvjEYRJf-Xng07phljoONJQRHN9uVq7NCrUkjNhfzgzYn8YUnB30f2NoecpBfgkTBcRtl5NvnOijwU0w7uX-SVKl65eTBnVJJVh_Qyp1MzPLnD0kfrkduUoVlIIeu6uHibbWvX_JIQusP1MJPR40uApCW5HM6PPNnMtsSFHx27_MKxCfFhMrjv1-XvEyau0Y7eytiScm-K46TSlSTZbtzvFylvqqlZ0Kn6LA6gf0-qsIUrgAEY1b0fVXQcgCePhPkXg5uL2mkjwidweYWaj8_FnJSSVFTg/https%3A%2F%2Fui.adsabs.harvard.edu%2Fabs%2F2022ApJS..258...38P%2Fabstract.
In a related fashion, I would also like to know if it would be possible to add some higher-resolution stellar templates. The default libraries seem to have lower resolutions than KCWI, so I had to manually add in the X-Shooter Library SSP Modelshttp://secure-web.cisco.com/1EApHf8In8dnZSQ3miGK4lBAPh7uzCphB8T6bZnJgvTtKAsFNjF_c8evx4OAAUHTf2d2Z-xYIkoosHA3Ud91hbK7ECKrM-7kcXvnf8H4LULEWhCIuFzwWz62aPxrGeahrCfeqHbJ75kDbV3ZMVc4oNWyR04mKqafOmNZ-agiUijtSdzsVr-gX0QcYMkYXBwuvOfTu_ENofU2Lg8OhbWO6Cx1EC2M6W6plOR4XshrFkt0Ur9GQX6K8ssZ-YLAqJL3--kf3TRb1DWEuqEj99WyaJeQYB5BtxXezIMPSt0tEtpUq0RJ6QxoksJ5V0HPl0qxqL4MeeJNbjqm8-KIy7Iynzsg1-sIBJ7UVEAcjUC6s600qXuparxoADyiDNbhH7_YY4y8gL5nKaAEpAY7ZlOC-MhH_K92ZauS_nhxHYoKseo4FBwpv5mnJBRl24KnnnTVY/http%3A%2F%2Fxsl.u-strasbg.fr%2Fpage_ssp_all.html, although in a limited capacity (solar metallicity, trimmed from 3800 A - 10000A, and re-binned to a linear scale, since by default the wavelengths are sampled in log10, b! ut the resolution changes between the UV/OPT/IR bands). In principle, would it be possible to use additional libraries in prepare_stellar_templates, even if the templates themselves might not be sampled linearly in wavelength, and/or might have varying resolution?
—
Reply to this email directly, view it on GitHubhttps://secure-web.cisco.com/1VQdcq-qd6AW0HHUz_tpapu0wtT81--n1mICOVpFomj4QEE9ebScbbNKY1z00yWdB-NdIH5ML6Q_ktERZF6-3JD2za64od0r5JwMFTJInKXPnxQ-FLWzPpLG6hpgCmNYHsMJZxV0tbT9cY72LUAlfZFkoeR6BB2fPCLgr9M2wcV6UXPvwyAuJaPFCLrUlssiGwe2OL5vYO7ejCgIeP1doI4-qzmDuJO2H44IuGBSYkOkrAdDuOOxBvYEBV5KwfIad-42fcks2cZCOP6ypAVXJ16pDyKrguXuSqVXYdio3O9lhlt6wsWHoGP_MYlzfJwsQnE07KJ5g-c-hL_8NCyi9raSwSfENzev9OGpnAN7o5uaBQS9FP9zT5rJlSddZXaz27j7IezE-hL7ONr95bT9bOUl8vDS4pOgIJzwat72L287RstguiV7kPpcoNME6d48B/https%3A%2F%2Fgithub.com%2Fremingtonsexton%2FBADASS3%2Fissues%2F2, or unsubscribe<https://secure-web.cisco.com/15Dlz5w8LQkHtE1JsCZhrPsEKsg0ENlCUt_uAG7cNaJZ-zfMDAFV7fkGXdbLZDVkLdPI3GAzfy_PsSRUhGsmBpWuPiM1QdW40oUIgo4ZO1i2FHqxVlo0JUcHD5xbxMylyzh6aJ4F8UMumanTVvcGgoGXw5ZBTy5dYZDdC_vVmxAx80gP_PKXe56U081E4todAvtVMx_9SUrC2g2J2BaWn5xqODndH0VMbgrgnCrd9tt4R5iRPfhlNdcQhA2bQt5Cek!
%20zNrUmFsLc3Jna5xe17TMtKufzMkfZm1lfi0MnFwVjxM34aciM64gKrbTMEAvPKykqAgxGxaiDYD1ypxmftiWkxBL6KDz-O5HZ33XwQVycA2PQtmcyHaqsIeWMabwj7lVT0qSNsrxFJR7_2AKME8zicMmPSsjHvl20DsrWYHbVv2h5AItM2BEpbddTsxWo4-/https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAF7IPQHXFTLPAHVH5CQDEPLVKWJMZANCNFSM5WKJPRDQ>.
You are receiving this because you are subscribed to this thread.Message ID: @.***>

I ended up inputting the Maraston and Stromback (2011) templates based on ELODIE (FWHM = 0.55 A, linearly sampled), which was simple.
Also was able to successfully implement Mrk 493 STIS template by adding blocks of code similar to the Veron-Cetty template (but without the need to separate broad an narrow Fe II).


Fail pip install

I created a venv with python 3.8.6 on Mac Os 10.15.7 and run pip install -r requirements.txt.
I got this message error:
ERROR: Could not find a version that satisfies the requirement PyWavelets==0.5.2
ERROR: No matching distribution found for PyWavelets==0.5.2

I believe that the problem is caused by the fact that PyWavelets==0.5.2 is very old (2017!) and the author did not make it compatible with python 3.8.
I tried with python 3.7.7 and it worked. I would therefore suggest to add in the README this information.
I created a pull request for you: https://github.com/AliaksandrSiarohin/first-order-model/pull/395

Probably it is better just remove it from requirements.

the change to requirements.txt i made to get it to work
note: i am still having issues with running the demo on cuda
imageio==2.3.0
matplotlib==3.3.4
numpy==1.20.0
pandas==1.2
python-dateutil==2.7.3
pytz==2018.5
PyYAML==5.4
scikit-image==0.18.0
scikit-learn==0.24.0
scipy==1.6.2
torch==1.8.1
torchvision==0.2.1
tqdm==4.24.0

imageio==2.9.0
matplotlib==3.4.1
numpy==1.19.5
pandas==1.2.3
python-dateutil==2.7.3
pytz==2021.1
PyYAML==5.4.1
scikit-image==0.18.1
scikit-learn==0.24.1
scipy==1.6.2
torch==1.8.1
torchvision==0.9.1
tqdm==4.24.0

the change to requirements.txt i made to get it to work
note: i am still having issues with running the demo on cuda
imageio==2.3.0
matplotlib==3.3.4
numpy==1.20.0
pandas==1.2
python-dateutil==2.7.3
pytz==2018.5
PyYAML==5.4
scikit-image==0.18.0
scikit-learn==0.24.0
scipy==1.6.2
torch==1.8.1
torchvision==0.2.1
tqdm==4.24.0

running this command solved the CUDA error. Now I'm able to run the code using CUDA. But I don't know how to embed this inside the requirements.txt file ( I'm a python noob :-()
pip3 install torch==1.8.1+cu102 torchvision==0.9.1+cu102  -f https://download.pytorch.org/whl/torch_stable.html

the change to requirements.txt i made to get it to work
note: i am still having issues with running the demo on cuda
imageio==2.3.0
matplotlib==3.3.4
numpy==1.20.0
pandas==1.2
python-dateutil==2.7.3
pytz==2018.5
PyYAML==5.4
scikit-image==0.18.0
scikit-learn==0.24.0
scipy==1.6.2
torch==1.8.1
torchvision==0.2.1
tqdm==4.24.0

running this command solved the CUDA error. Now I'm able to run the code using CUDA. But I don't know how to embed this inside the requirements.txt file ( I'm a python noob)
pip3 install torch==1.8.1+cu102 torchvision==0.9.1+cu102 -f https://download.pytorch.org/whl/torch_stable.html

Worked Thanks !


RAM was set only 8 Gb.

ubuntu-memory-2020-10-04.pptx
The computer has 128 Gb RAM, but wsl or unbutu uses 8 GB. Using stress to test wsl2-ubuntu performance, memory is totally used only 16 GB. Do we can fix the bug?
OS: Ubuntu 20.04.1 LTS on Windows 10 x86_64
Kernel: 4.19.128-microsoft-standard
Uptime: 16 mins
Packages: 643 (dpkg)
Shell: bash 5.0.17
Terminal: /dev/pts/0
CPU: Intel i9-9900K (16) @ 3.600GHz
GPU: bdf4:00:00.0 Microsoft Corporation Device 008e
Memory: 109MiB / 7958MiB

Not bug.
https://docs.microsoft.com/en-us/windows/wsl/release-notes#build-20175
https://docs.microsoft.com/en-us/windows/wsl/wsl-config#configure-global-options-with-wslconfig

Try to see the motherboard configuration for ram,i needed to set the speed of my ram sticks(16gb) each,to configuration of equal Mhz for proper working.I had the same issue of not showing the correct ram installed.but these method fixed my problem

In %USERPROFILE%\.wslconfig
[wsl2]
memory=128GB

Leaving a little headroom, so maybe memory=96GB, is not a bad idea.


Rviz2 Malformed STL error

OS: Ubuntu 20.04
ROS Distro: Galactic
Built from source or installed: source

Expected behaviour
Launching view_model.launch.py or view_robot.launch.py should show the model with correct mesh colours.
Actual behaviour
Turtlebot4 components are shown in red, Rviz logs the following error:
The STL file '*.STL' is malformed. It starts with the word 'solid', indicating that it's an ASCII STL file, but it does not contain the word 'endsolid' so it is either a malformed ASCII STL file or it is actually a binary STL file. Trying to interpret it as a binary STL file instead.
To Reproduce
Provide the steps to reproduce:
ros2 launch turtlebot4_viz view_model.launch.py description:=true
Other notes
Online solutions such as https://gist.github.com/awesomebytes/2595b1dc41831c804a4f have not worked.

Hello Roni,
How did you solved this issue?

Hello Roni,
How did you solved this issue?

Hi,
https://gist.github.com/awesomebytes/2595b1dc41831c804a4f
The solution Roni mentioned actually did work for me. I hope It does the same for you


Getting Started Example not Working with 1.0.0-alpha.10

  cd phenomic
  rm package.json
  cd examples
  cd react-app-getting-started
  npm install
  npm start

Returns the following error
/home/benjaminshafii/phenomic/examples/react-app-getting-started/Html.js:4
import * as React from "react";
^^^^^^

SyntaxError: Unexpected token import
    at createScript (vm.js:74:10)
    at Object.runInThisContext (vm.js:116:10)
    at Module._compile (module.js:588:28)
    at loader (/home/benjaminshafii/phenomic/examples/react-app-getting-started/node_modules/babel-register/lib/node.js:144:5)
    at Object.require.extensions.(anonymous function) [as .js] (/home/benjaminshafii/phenomic/examples/react-app-getting-started/node_modules/babel-register/lib/node.js:154:7)
    at Module.load (module.js:545:32)
    at tryModuleLoad (module.js:508:12)
    at Function.Module._load (module.js:500:3)
    at Module.require (module.js:568:17)
    at require (internal/module.js:11:18)

We use babel-register so we should not have this kind of issue... That's weird but it has been already reported by @ArthurClemens & @DominikGuzei on Gitter, not sure how they handled this.
Will have a look!

Ok I know what's going on. We are currently not transpiling by default modules (as we only have a  webpack bundler plugin for now). This issue is not happening on our repo since we have a babel configuration that hide the problem. I am working on a fix!

That was fast!

v1.0.0-alpha.11 is out!

Awesome, thank you @MoOx! This also fixed an issue I had experienced while trying to add my own Html.js file (not when using the example, but rather using the @phenomic/preset-react-app getting started tutorial).


Code 500 response, no further explaination

curl -X PUT -H "X-API-Key: $DATABUS_API_KEY" https://energy.databus.dbpedia.org/denis/OEP/rli_dibt_windzone/2022-11-02 --data '{"@context": "https://downloads.dbpedia.org/databus/context.jsonld", "@graph": [{"@id": "https://energy.databus.dbpedia.org/denis/OEP", "@type": "Group", "title": "OEP Group", "abstract": "OEP Group holds databus releases of OEP tables", "description": "Tables in the OEP group have been released automatically to databaus via script oep_databus_registration.py from https://github.com/LOD-GEOSS/databus-snippets additionally related metadata has been released to MOSS (https://moss.tools.dbpedia.org)."}, {"@type": "Dataset", "@id": "https://energy.databus.dbpedia.org/denis/OEP/rli_dibt_windzone/2022-11-02#Dataset", "hasVersion": "2022-11-02", "title": "Windzone map for Germany", "abstract": "none", "description": "", "license": "https://creativecommons.org/licenses/by/4.0/legalcode", "distribution": [{"@id": "https://energy.databus.dbpedia.org/denis/OEP/rli_dibt_windzone/2022-11-02#rli_dibt_windzone_.json", "@type": "Part", "file": "https://energy.databus.dbpedia.org/denis/OEP/rli_dibt_windzone/2022-11-02/rli_dibt_windzone_.json", "formatExtension": "json", "compression": "none", "downloadURL": "https://openenergy-platform.org/api/v0/schema/climate/tables/rli_dibt_windzone/rows/", "byteSize": 11430892, "sha256sum": "8b3105c3a62c61b4cab27f507606c931dc9914fe795770eeb03e4abbe2e90cbb"}]}]}'

Returns
Publishing DataId.
Publishing DataId finished with code 500.

Please check if new logging resolve this

The input went through and created a group and version on my end

Can be fixed by:

Use the https://energy.databus.dbpedia.org/api/publish endpoint
Specify the Content-Type with curl, because curl uses a weird default

So this works:
curl -X POST -H "Content-Type: application/json" -H "X-API-Key: $DATABUS_API_KEY" https://energy.databus.dbpedia.org/api/publish --data '{"@context": "https://downloads.dbpedia.org/databus/context.jsonld", "@graph": [{"@id": "https://energy.databus.dbpedia.org/denis/OEP", "@type": "Group", "title": "OEP Group", "abstract": "OEP Group holds databus releases of OEP tables", "description": "Tables in the OEP group have been released automatically to databaus via script oep_databus_registration.py from https://github.com/LOD-GEOSS/databus-snippets additionally related metadata has been released to MOSS (https://moss.tools.dbpedia.org)."}, {"@type": "Dataset", "@id": "https://energy.databus.dbpedia.org/denis/OEP/rli_dibt_windzone/2022-11-02#Dataset", "hasVersion": "2022-11-02", "title": "Windzone map for Germany", "abstract": "none", "description": "", "license": "https://creativecommons.org/licenses/by/4.0/legalcode", "distribution": [{"@id": "https://energy.databus.dbpedia.org/denis/OEP/rli_dibt_windzone/2022-11-02#rli_dibt_windzone_.json", "@type": "Part", "file": "https://energy.databus.dbpedia.org/denis/OEP/rli_dibt_windzone/2022-11-02/rli_dibt_windzone_.json", "formatExtension": "json", "compression": "none", "downloadURL": "https://openenergy-platform.org/api/v0/schema/climate/tables/rli_dibt_windzone/rows/", "byteSize": 11430892, "sha256sum": "8b3105c3a62c61b4cab27f507606c931dc9914fe795770eeb03e4abbe2e90cbb"}]}]}'


Drag and Drop reorder possible, but not seeing documentation for it.

When I use the image preview plugin I can drag and drop the order of the images in filepond. However, I am unable to find documentation on this features.
I'm using file pond for users to upload images to a gallery and the order of the gallery needs to be saved.
How do I save these images in the order the user sets in filepond using drag and drop reordering? (I'm using Laravel and Livewire)

Hello, Brentscholl! Look at this screencast here:
https://laravel-livewire.com/screencasts/s8-dragging-list
At the end there is a function that works with sorting. But the problem, as it seems to me, is not in changing the order of files when dragging, but in the order of transferring files from the client to the server. Each time, the server accepts files in a different order, even during the initial save. I also try to set up this bundle and so far it turns out poorly. If you can figure it out, please let me know.

Not sure if this is correct, but it seems to work
I added this to FilePond.setOptions()
FilePond.setOptions({
                allowMultiple: true,
                allowReorder: true,
                server: {
                    process: (fieldName, file, metadata, load, error, progress, abort, transfer, options) => {
                        @this.upload('images', file, load, error, progress)
                    },
                    revert: (filename, load) => {
                        @this.removeUpload('images', filename, load)
                    },
                },

                // Added this ----------------------------------------
                onreorderfiles(files, origin, target){
                    @this.set('images', null);
                    files.forEach(function(file) {
                        @this.upload('images', file.file);
                    });
                },
            });

This removes the previous uploaded images and reuploads with the new order.

Closed because inactive.

Hey @brentscholl, would you mind sharing your server side code for Livewire component using Pond? ;)


Specify androidx.activity:activity-compose as 1.9.0

The Compose BOM doesn't provide this.
https://developer.android.com/develop/ui/compose/bom/bom-mapping
Failed to resolve: androidx.activity:activity-compose

Feel free to consider this a bug report, close and reopen internally if simpler.


Import table definition dart in moor files

I am trying to define a table in dart file and use that table in moor files for querying. So in the moor file I imported the dart file like docs suggested. Then in database.dart, I did
@UseMoor(
  tables: [Transactions],
  include: {'./schema/transactions.moor'},
)

But this is creating duplicates in database.g.dart for Transaction like below.

If I do this in database.dart
@UseMoor(
  include: {'./schema/transactions.moor'},
)

Then I get missing implementation error.

Also is there way I can use UUID in moor file? I am mainly creating table in dart because I cannot figure out how to use .clientDefault() in moor file?

Importing a Dart file in a moor file to use its tables should work.
Unfortunately the generator can't add import statements, so you'll need to add them yourself. You have to import the file declaring Transactions into the Dart file that declares the database.

I am importing the transactions table in database generation file and also in moor file where I am writing queries for that table. But that is causing the first error. I am getting Duplicates of Transaction And TransactionCompanion(from first screenshot).

Does it work if you keep both imports, but remove the tables: [Transactions] field from @UseMoor?

That worked. Thanks for the help and an amazing library.


fix: remove unused availability zone errors

the single/multi AZ errors are no longer used since the request param has been deprecated.

@machi1990 the other references are also removed now

I think one of the connectors test might fail as they reference the removed errors. The test will need to  be updated.

I think one of the connectors test might fail as they reference the removed errors. The test will need to be updated.

This is the test in question: https://github.com/bf2fc6cc711aee1a0c2a/kas-fleet-manager/blob/47360744395aff45f165f79261135895659d445d/internal/connector/test/integration/features/connector-api.feature#L1637
and https://github.com/bf2fc6cc711aee1a0c2a/kas-fleet-manager/blob/47360744395aff45f165f79261135895659d445d/internal/connector/test/integration/features/connector-api.feature#L1693


Remove unneeded relpath() polyfill

It looks like this dates back all the way to 2007,
and the initial import of the code in commit 00f16f9.
Since Python 2.6 is the minimum supported version, and that has
os.path.relpath this is no longer needed.

Current coverage is 83.36%

Merging #213 into master will increase coverage by +0.15% as of 4057631

@@            master    #213   diff @@
======================================
  Files           21      21
  Stmts         3789    3781     -8
  Branches         0       0
  Methods          0       0
======================================
- Hit           3153    3152     -1
  Partial          0       0
+ Missed         636     629     -7

Review entire Coverage Diff as of 4057631

Powered by Codecov. Updated on successful CI builds.


aws-apigateway-dynamodb: setting "allowReadOperation" to false does not disable the read operation

Setting allowReadOperation to false does not disable the read operation.
Reproduction Steps

import { ApiGatewayToDynamoDBProps, ApiGatewayToDynamoDB } from "@aws-solutions-constructs/aws-apigateway-dynamodb";

new ApiGatewayToDynamoDB(this, 'test-api-gateway-dynamodb-default', {
  allowReadOperation: false
});

Error Log

There is no error in synthesizing the stack
Environment

CDK CLI Version      : 1.79.0
CDK Framework Version: 1.79.0
AWS Solutions Constructs Version     : 1.79.0
OS                   : Mac
Language             : TypeScript

Other

This issue seems to be with this conditional:
https://github.com/awslabs/aws-solutions-constructs/blob/8130c54616c0422b7b141b4bc9d3086becdf7a2d/source/patterns/%40aws-solutions-constructs/aws-apigateway-dynamodb/lib/index.ts#L154
Given the three input options (true/false/undefined), the conditional always will evaluate to true

This is :bug: Bug Report

JakePartusch@ This bug is fixed in v1.80.0, for proof see this

JakePartusch@ This bug is fixed in v1.80.0, for proof see this


Checkbox issue in basic form example

Thanks for those helpers it's saving a lot of time for our dev team.
However we have an issue with basic form and checkbox input. The helper add a .form-group div element which wrap .checkbox div element.
It break some advanced form building and we have to had manually a new template string for this case.
It should be awesome if your helper can improve this feature and dynamicaly add the wrapping form-group depending on the case of utilisation.

Should be fixed by commit ce0653d39721c23c780108c50ce064d795b53390. Let me know if it does not work!


import

We have the rudimentary import feature in the admin page but we should support an import that maintains custom metadata (title, description, tags) and possibly even datetime?

When I try to import I get the following error:
failed to start application on pinakes.glitch.me
This is most likely because your project has a code error.
Check your project logs, fix the error and try again.

Confirmed here. Log shows:
[2023-9-11 20:28:59] POST:/bookmark/multiadd 200 0.002ms
unable to parse url
couldn't fetch opengraph data for https://www.printables.com/@AndyPiper
couldn't fetch opengraph data for https://cults3d.com/en/users/andypiper
couldn't fetch opengraph data for https://www.thingiverse.com/andypiperuk/
couldn't fetch opengraph data for https://www.tinkercad.com/users/0z2EwxKwnmp-andy-piper
couldn't fetch opengraph data for https://thangs.com/designer/Andy%20Piper
couldn't fetch opengraph data for https://pinshape.com/users/2556538-andy-piper
couldn't fetch opengraph data for https://www.youmagine.com/andypiper/designs
couldn't fetch opengraph data for https://www.myminifactory.com/users/andypiper
file:///app/src/routes/bookmark.js:182
      title: meta.result.ogTitle,
                         ^
TypeError: Cannot read properties of undefined (reading 'ogTitle')
at file:///app/src/routes/bookmark.js:182:26
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)

So it looks like the multiadd operation is failing to fetch opengraph tags to populate the entries. I'll see what I can figure out here.

(however, this is a bug, and not directly connected to @ckolderup original issue / enhancement idea, which we should retain)

if it helps with the bug, I swapped opengraph libraries very hastily just before launch to clear up some security warnings that were being introduced and probably missed some refactoring in the process! (yet another reason I really need to get tests/checks in place) the opengraph resolution stuff should probably be extracted / reused for both single- and multi-add processes. (and if you don't get to it, I can probably fix this this week, things have just been a little hectic all of a sudden)

just submitted #91 which I would describe as a monkeypatch 🙂 and yeah, a better solution would be to run the OG resolution in one place rather than split between two locations.


issue #2 - catch case where identical rows are used in succession for the upsert

This PR applies the proposed changes as described in #2 before the change the new test case yielded the following result:
+------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------+
|                                           df1                                            |                                            df2                                            |
+------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------+
| Row(pkey=1, attr='A', cur=True, effective_date=datetime.date(2019, 1, 1), end_date=None) | Row(pkey=1, attr='A', cur=True, effective_date=datetime.date(2019, 1, 1), end_date=None) |
| Row(pkey=1, attr='A', cur=True, effective_date=datetime.date(2019, 1, 1), end_date=None) | Row(pkey=2, attr='B', cur=True, effective_date=datetime.date(2019, 1, 1), end_date=None) |
| Row(pkey=2, attr='B', cur=True, effective_date=datetime.date(2019, 1, 1), end_date=None) | Row(pkey=4, attr='D', cur=True, effective_date=datetime.date(2019, 1, 1), end_date=None) |
| Row(pkey=4, attr='D', cur=True, effective_date=datetime.date(2019, 1, 1), end_date=None) |                                            None                                           |
+------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------+

the update is equal to the existing record 1 so no action should be taken but a new record is created instead without ending the previous record with id .  with the change applied the record is untouched
+------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------+
|                                           df1                                            |                                            df2                                            |
+------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------+
| Row(pkey=1, attr='A', cur=True, effective_date=datetime.date(2019, 1, 1), end_date=None) | Row(pkey=1, attr='A', cur=True, effective_date=datetime.date(2019, 1, 1), end_date=None) |
| Row(pkey=1, attr='A', cur=True, effective_date=datetime.date(2019, 1, 1), end_date=None) | Row(pkey=2, attr='B', cur=True, effective_date=datetime.date(2019, 1, 1), end_date=None) |
| Row(pkey=2, attr='B', cur=True, effective_date=datetime.date(2019, 1, 1), end_date=None) | Row(pkey=4, attr='D', cur=True, effective_date=datetime.date(2019, 1, 1), end_date=None) ||
+------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------+

This change only accounts for the duplicate rows, there is another issue with this statement but that requires a more elaborate change and i wanted to split the issues in two seperate PR's

@dgcaron - thanks for this contribution.
@danielbeach - can you take a look and let us know if this change looks alright to you?

there is another issue with this statement but that requires a more elaborate change and i wanted to split the issues in two seperate PR's

@dgcaron - can you please make another PR with the other fixes?  I promise we'll get it reviewed and merged faster.

The changes proposed are explained #121.. fixing it properly requires a breaking change in the current call so you might want to give your thoughts before I start writing code :)


fix(kube-node-lease): remove objects created for kube-node-lease

Fixes #4547
Hey,
It looks like about 2 months ago, resources for kube-node-lease were added to karpenter:
https://github.com/aws/karpenter/pull/4453
Those changes are causing karpenter not to work on clusters without a namespace called kube-node-lease
as i see, this namespace is used for test, as it appers only in those files:
karpenter/test/suites/beta/integration/lease_garbagecollection_test.go
karpenter/test/suites/beta/integration/lease_garbagecollection_test.go

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

How do you manage node leases? Is it possible for you to create the namespace?

@njtran , should we just log (info) and continue if we don't find this namespace? We definitely shouldn't block Karpenter's operations on this.

@ellistarn yeah that'd be reasonable. @arnoldyahad Would that be sufficient for you? You could then remove the role and rolebinding from your own CI/CD as Karpenter wouldn't make the call.

I had a general question about whether having this namespace is expected for all clusters or under which circumstances we shouldn't expect to see this namespace? Is is possible to put your node leases in a different namespace?

@ellistarn @njtran first of all thanks for the quick reply.
we are using eks 1.23 (planning to upgrade to 1.24)
but there are also old clusters(1.22) that we dont plan to upgrade.
it is possible for us to do this workaround and just not use anything related to kube-node-lease but this would mean that we are divergent from karpenter and would need to do this fix after every chart upgrade.
The problem here is that users can't use newer karpenter versions (https://github.com/aws/karpenter/issues/4547) without the namespace, and if its possible for karpenter to work without it - those changes should be optional and maybe inserted only if a cluster has this namespace already.
if you are aware that this namespace is created automatically by k8s/eks in later versions and is mandatory for cluster operations please tell us and we will add it manually for now, but if its not - better not to enforce it if not needed.

if you are aware that this namespace is created automatically by k8s/eks in later versions and is mandatory for cluster operations please tell us and we will add it manually for now, but if its not - better not to enforce it if not needed.

AFAIK, I thought the feature for Node leases being stored in the kube-node-lease namespace became stable at 1.17 when this feature was introduced: https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/589-efficient-node-heartbeats. Do you know if the node lease heartbeats are being stored in another namespace? I don't believe that this behavior is specific to EKS.

@jonathan-innis thanks for replying.
i checked our clusters for leases by using:
 kubectl -n kube-system get lease

and i get information regarding all leases, so i don't know if the feature to have it in a separate namespace is "enforced" somehow and we skipped it, or its an "opt-in" to have it in a kube-node-lease dashboard.
our clusters vary from EKS-1.19 to 1.23.

@ellistarn @njtran
I confirmed with a teammate that this is not the only problem but even when removing the roles karpenter tries to get to that namespace and list resources:
{"severity":"INFO","timestamp":"2023-10-24T08:46:19.084316716Z","logger":"controller","caller":"cache/reflector.go:424","message":"k8s.io/client-go@v0.26.6/tools/cache/reflector.go:169: failed to list *v1.Lease: leases.coordination.k8s.io is for
bidden: User \"system:serviceaccount:kube-system:karpenter-controller\" cannot list resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"\n","commit":"322822a"}
{"severity":"ERROR","timestamp":"2023-10-24T08:46:19.084371722Z","logger":"controller","caller":"cache/reflector.go:140","message":"k8s.io/client-go@v0.26.6/tools/cache/reflector.go:169: Failed to watch *v1.Lease: failed to list *v1.Lease: lease
s.coordination.k8s.io is forbidden: User \"system:serviceaccount:kube-system:karpenter-controller\" cannot list resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"\n","commit":"322822a","stacktrace":"k8s
.io/client-go/tools/cache.DefaultWatchErrorHandler\n\tk8s.io/client-go@v0.26.6/tools/cache/reflector.go:140\nk8s.io/client-go/tools/cache.(*Reflector).Run.func1\n\tk8s.io/client-go@v0.26.6/tools/cache/reflector.go:224\nk8s.io/apimachinery/pkg/ut
il/wait.BackoffUntil.func1\n\tk8s.io/apimachinery@v0.26.6/pkg/util/wait/wait.go:157\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\tk8s.io/apimachinery@v0.26.6/pkg/util/wait/wait.go:158\nk8s.io/client-go/tools/cache.(*Reflector).Run\n\tk8s.io
/client-go@v0.26.6/tools/cache/reflector.go:222\nk8s.io/client-go/tools/cache.(*controller).Run.(*Group).StartWithChannel.func2\n\tk8s.io/apimachinery@v0.26.6/pkg/util/wait/wait.go:58\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\tk8
s.io/apimachinery@v0.26.6/pkg/util/wait/wait.go:75"}

might be related to:
https://github.com/aws/karpenter-core/blob/223dcd0fda49f05b3d7beebc45cdb843caaefd3c/pkg/operator/operator.go#L152
or
https://github.com/aws/karpenter-core/blob/223dcd0fda49f05b3d7beebc45cdb843caaefd3c/pkg/operator/operator.go#L152
so just removing the permissions to that namespace as i did in this PR is not enough to solve this issue.

Hey @arnoldyahad! Any updates here? We want to drive toward getting a solution to your problem here but it's still a little unclear to me what the cluster configuration looks like that's causing this namespace not to exist.


Add OCI Object Storage Bucket support

[x] I added CHANGELOG entry for this change.
[ ] Change is not relevant to the end user.

Changes
Add Oracle Cloud Infrastructure (OCI) Object Storage Bucket support
OCI Object Storage supports three different providers, default, instance principle, and raw. A lot of OCI users like to use Instance principle to security. It is very important to support instance principle for the OCI users. S3 doesn't support OCI instance principle.
Support multiple prometheus instances with a single bucket
By default, like other clients, this new Thanos OCI Object Storage client will storage the data from a single prometheus instance to the top directory of the bucket. This means each bucket can only store one prometheus instance data.
In OCI, a lot of users like to put different type of data in a single bucket.
To support multiple prometheus instances with a single bucket, a new object_base_path option is introduced in this OCI Object Storage client. If defined, the value will be used as the prefix of the prometheus data in OCI bucket.
For example, if you want to use the same bucket for 2 prometheus instances, you can define object_base_path as /path/for/prometheus1 for the first prometheus instance, and /path/for/prometheus2 for the second prometheus instance.
Three tests are included here to demo how the object_base_path work.
Verification
From original PR: Running similar OCI client against 3 prometheus instances for over 50 days without any issue.
For each prometheus instance

average 4.5GB (2GB in chunks and 2.5G index) per 2-hour prometheus block
over 50GB per day
over 2TB prometheus data uploaded to OCI Object Storage

Testing from this PR
export THANOS_TEST_OBJSTORE_SKIP=GCS,S3,AZURE,SWIFT,COS,ALIYUNOSS,BOS
// export OCI_* required variables
$ make test
go test ./...
ok  	github.com/thanos-io/objstore	(cached)
?   	github.com/thanos-io/objstore/client	[no test files]
ok  	github.com/thanos-io/objstore/clientutil	(cached)
?   	github.com/thanos-io/objstore/errutil	[no test files]
ok  	github.com/thanos-io/objstore/exthttp	(cached)
ok  	github.com/thanos-io/objstore/objtesting	123.749s
ok  	github.com/thanos-io/objstore/providers/azure	(cached)
?   	github.com/thanos-io/objstore/providers/bos	[no test files]
ok  	github.com/thanos-io/objstore/providers/cos	(cached)
ok  	github.com/thanos-io/objstore/providers/filesystem	(cached)
ok  	github.com/thanos-io/objstore/providers/gcs	(cached)
?   	github.com/thanos-io/objstore/providers/oci	[no test files]
?   	github.com/thanos-io/objstore/providers/oss	[no test files]
ok  	github.com/thanos-io/objstore/providers/s3	(cached)
ok  	github.com/thanos-io/objstore/providers/swift	(cached)
?   	github.com/thanos-io/objstore/scripts/cfggen	[no test files]
?   	github.com/thanos-io/objstore/test/e2e/e2ethanos	[no test files]
?   	github.com/thanos-io/objstore/tracing	[no test files]

See the test steps in README.md under OCI section.

Original PR: https://github.com/thanos-io/thanos/pull/4661

^^^ @kakkoyun and @yeya24 @GiedriusS @wiardvanrij @Djelibeybi

Thanks @wiardvanrij for reviewing it.
I added the changelog: https://github.com/thanos-io/objstore/pull/15/files#diff-06572a96a58dc510037d5efa622f9bec8519bc1beab13c9f251e97e657a9d4edR16

:tada:

Yea thanks @aarontams  :)

https://twitter.com/ThanosMetrics/status/1549071260415660032 :)

We haven't actually discussed the cadence of the updates of objstore among Thanos maintainers (assuming at the very least before each major release), however, if you want to immediately have this on Thanos main branch, @aarontams  you should send another PR to Thanos main repo to bump up the dependency version.


Removed duplicate code

Interesting that it failed, maybe we need to retrigger this build.

friendly reminder * :)

Oops, thanks for the patch!


FP add suppression to base for mime4j-dom #4370

Fixes Issue #4370
Description of Change
add suppression for mime4j-dom / jdom:jdom
Have test cases been added to cover the new functionality?
no

Thanks. LGTM.


Preserve redirects & messages topic

The redirect topics that were added in 2c2938f for #43 are removed in subsequent CI builds, since the  dita-ot/docs/.travis/publish.sh script removes the target output directory before generating new output.
CI builds should be adjusted to preserve these topics, along with the auto-generated DITA-messages.html topic, which was removed by CI in d26b6f0 and manually restored in 713c090.

See keep_files option:
keep_files: [DIR, FILE, ...]


TypeError: string indices must be integers

While running the package, I faced the below error all the time.
2024-07-09 15:42:42,195 - httpx - INFO - HTTP Request: GET https://www.tiktok.com/@theoriginote/live "HTTP/1.1 200 OK"
2024-07-09 15:42:42,505 - httpx - INFO - HTTP Request: GET https://webcast.tiktok.com/webcast/room/check_alive/?aid=1988&app_language=en-US&app_name=tiktok_web&browser_language=en&browser_name=Mozilla&browser_online=true&browser_platform=Win32&browser_version=5.0%20%28Windows%20NT%2010.0%3B%20Win64%3B%20x64%29%20AppleWebKit%2F537.36%20%28KHTML%2C%20like%20Gecko%29%20Chrome%2F102.0.5005.63%20Safari%2F537.36&cookie_enabled=true&cursor=&internal_ext=&device_platform=web&focus_state=true&from_page=user&history_len=4&is_fullscreen=false&is_page_visible=true&did_rule=3&fetch_rule=1&last_rtt=0&live_id=12&resp_content_type=protobuf&screen_height=1152&screen_width=2048&tz_name=Europe%2FBerlin&referer=https%3A%2F%2Fwww.tiktok.com%2F&root_referer=https%3A%2F%2Fwww.tiktok.com%2F&msToken=&version_code=180800&webcast_sdk_version=1.3.0&update_version_code=1.3.0&uuc=1&device_id=50726871387963610829&room_id=%22roomId%22%3A%227389544423330564869%22&room_ids=%22roomId%22%3A%227389544423330564869%22 "HTTP/1.1 200 OK"
Traceback (most recent call last):
  File "/home/nahid/test/tiktok-events/one/test.py", line 47, in <module>
    client.run()
  File "/home/nahid/test/tiktok-events/one/tik-venv/lib/python3.10/site-packages/TikTokLive/client/client.py", line 200, in run
    return self._asyncio_loop.run_until_complete(self.connect(**kwargs))
  File "/usr/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
    return future.result()
  File "/home/nahid/test/tiktok-events/one/tik-venv/lib/python3.10/site-packages/TikTokLive/client/client.py", line 176, in connect
    task: Task = await self.start(**kwargs)
  File "/home/nahid/test/tiktok-events/one/tik-venv/lib/python3.10/site-packages/TikTokLive/client/client.py", line 127, in start
    if fetch_live_check and not await self._web.fetch_is_live(room_id=self._room_id):
  File "/home/nahid/test/tiktok-events/one/tik-venv/lib/python3.10/site-packages/TikTokLive/client/web/routes/fetch_is_live.py", line 47, in __call__
    return (await self.fetch_is_live_room_ids(room_id))[0]
  File "/home/nahid/test/tiktok-events/one/tik-venv/lib/python3.10/site-packages/TikTokLive/client/web/routes/fetch_is_live.py", line 66, in fetch_is_live_room_ids
    return [i["alive"] for i in response_json["data"]]
  File "/home/nahid/test/tiktok-events/one/tik-venv/lib/python3.10/site-packages/TikTokLive/client/web/routes/fetch_is_live.py", line 66, in <listcomp>
    return [i["alive"] for i in response_json["data"]]
TypeError: string indices must be integers

Package Version
My version of TikTokLive is 6.0.6.

The same

I have the solution. if you still use code that has an integer type. it's best to convert it to a string first

Thanks @zufarrizal
For your observation. But that didn't help me, cause  the error occurs directly into the package's code.
TikTokLive/client/web/routes/fetch_is_live.py",line 66, in
    return [i["alive"] for i in response_json["data"]]

Fixed 6.0.7


[Feature Request] provide abstractions to allow alternative debugger hookups to use termdebug UX

Is your feature request about something that is currently impossible or hard to do? Please describe the problem.
termdebug supports GDB/MI only.  DAP exists.
Describe the solution you'd like
It would be great if the UX around termdebug could be reused for an ambitious plugin writer to support working with DAP debuggers.
Describe alternatives you've considered
vimspector and neovim's nvim-dap
Additional context
nope

Describe alternatives you've considered
vimspector and neovim's nvim-dap

Do they not count as "ambitious" enough ?

vimL only, no lua or python3 requirement

From: Ben Jackson @.>
Sent: Thursday, November 24, 2022 1:37 AM
To: vim/vim @.>
Cc: razamatan @.>; Manual @.>
Subject: Re: [vim/vim] [Feature Request] provide abstractions to allow alternative debugger hookups to use termdebug UX (Issue #11605)
Describe alternatives you've considered
vimspector and neovim's nvim-dap
Do they not count as "ambitious" enough ?
—
Reply to this email directly, view it on GitHubhttps://github.com/vim/vim/issues/11605#issuecomment-1326191161, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAD53TZ77YLCFOQTHPAO4CLWJ4ZMJANCNFSM6AAAAAASJWGP74.
You are receiving this because you are subscribed to this thread.Message ID: @.***>

From the front page DAP looks like a good idea to support.   Perhaps start with one debugger that supports it and try make it work with TermDebug?  Then plan to expand to support more.  I have no idea how much work this would be.  I can see a protocol specification, but it doesn't say how the communication is implemented.  Using JSON?
It does appear to be very MS-Windows specific, character units are UTF-16 (worst choice ever).

I can see a protocol specification, but it doesn't say how the communication is implemented. Using JSON?

It uses something like JSON-RPC over stdio. https://microsoft.github.io/debug-adapter-protocol/overview#base-protocol
basically a length header followed by a json payload.

I have no idea how much work this would be.

It's not a small amount of work, in and of itself, but the ecosystem of debug adapters and foibles is a lot of ongoing maintenance. I have been working on vimspector https://github.com/puremourning/vimspector/ for 5 years now and it implements about, say 40% of the spec. Implementing disassembly was weeks of work (elapsed, spare time) on its own.
it's also worth noting there are a number of other DAP plugins for vim that are very good.

yeah, i took a good look at vimspector (listed in original post as alternatives considered.. tyvm for it, btw), but sometimes i don't have access to a +python3 compiled vim readily.  that got me looking around and land on termdebug as a built-in thing.
it would be great if the abstractions existed since vimspector could reuse the same windowing approach and command bindings.  i'm not deep enough down the rabbit hole of how vimspector is implemented to appreciate how difficult it would be to make it work at the window/UX layer to interop with the termdebug abstractions (if such things existed).  but that's the FR.  not suggesting we put a competing DAP client out there, but allow existing DAP clients to use more of the built-in debug experience.

From: Ben Jackson @.>
Sent: Thursday, November 24, 2022 3:19 AM
To: vim/vim @.>
Cc: razamatan @.>; Manual @.>
Subject: Re: [vim/vim] [Feature Request] provide abstractions to allow alternative debugger hookups to use termdebug UX (Issue #11605)
I can see a protocol specification, but it doesn't say how the communication is implemented. Using JSON?
It uses something like JSON-RPC over stdio. https://microsoft.github.io/debug-adapter-protocol/overview#base-protocol
basically a length header followed by a json payload.
I have no idea how much work this would be.
It's not a small amount of work, in and of itself, but the ecosystem of debug adapters and foibles is a lot of ongoing maintenance. I have been working on vimspector https://github.com/puremourning/vimspector/ for 5 years now and it implements about, say 40% of the spec. Implementing disassemblyhttps://github.com/puremourning/vimspector/#disassembly was weeks of work (elapsed, spare time) on its own.
it's also worth noting there are a number of other DAP plugins for vim that are very good.
—
Reply to this email directly, view it on GitHubhttps://github.com/vim/vim/issues/11605#issuecomment-1326312260, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAD53T7RKSQH53CYB6Q2CGTWJ5FMLANCNFSM6AAAAAASJWGP74.
You are receiving this because you are subscribed to this thread.Message ID: @.***>

GDB has DAP support now. I'm curious if we can try to give an optional support from termdebug already.

I also support this. Now that termdebug is in Vim9 it should be easier to update. As a first step I would try to separate the frontend to the backend (which now only include gdb). In that way the frontend would be always the same but the backend should be easy to switch i.e. it should be easier to add other debugger support like debugpy, lldb, etc. or for a possible DAP implementation entirely in Vim9.

@jinyius Given that DAP is the main wish, could you edit the title such that such a desire is more explicit? In this way there would be higher chances that someone could pick it up. Or one could also consider a porting of nvim-dap to Vim9 if does not want to change Termdebug.


SVGs with no viewBox and width&height attribs

I need to read what the expected behavior is, but for sure something has to be rendered (there is not for now)
pointed out in https://github.com/godotengine/godot/pull/64530

Some extra info:
Here are a few examples of the SVG code extracted from fonts, it's either missing viewBox completely or it's not enclosing the image:
🍏 from Twitter Color Emoji font:
<svg xmlns="http://www.w3.org/2000/svg" id="glyph318"><g transform="translate(0 -6.75) translate(0,-1638.4) scale(56.888888888888886)"><path fill="#77B255" d="M24 7c-3 0-3 1-6 1s-3-1-6-1c-4 0-9 2-9 9 0 11 6 20 10 20 3 0 3-1 5-1s2 1 5 1c4 0 10-9 10-20 0-7.001-5-9-9-9z" /><path fill="#3E721D" d="M17.311 7.88s-1.775-4.674-6.58-6.06c-3.843-1.108-6.318.26-6.318.26s3.012 3.991 5.895 4.822c2.882.83 7.003.978 7.003.978z" /><path fill="#662113" d="M18 10a1 1 0 0 1-1-1c0-3.441 1.2-6.615 3.293-8.707a.999.999 0 1 1 1.414 1.414C19.986 3.427 19 6.085 19 9a1 1 0 0 1-1 1z" /></g></svg>

🍏 from EmojiOne Color font:
<?xml version='1.0' encoding='utf8'?>
<svg xmlns="http://www.w3.org/2000/svg" id="glyph310" viewBox="0 0 64 64"><g transform="translate(0 -9.6) translate(-0.0 -104.8576) scale(2.048)"><path d="M31.4 4.9c1.1 2.5-.5 10.9-.5 10.9s-7.5-4.4-8.6-6.9c-1.1-2.5 0-5.4 2.5-6.5 2.5-1.1 5.5 0 6.6 2.5" fill="#64892f" /><path d="M33.3 16.3h-2.6c0-4.3 2.4-11.6 12-11.6v2.6c-8.7 0-9.4 8.2-9.4 9" fill="#947151" /><path d="M32 15.8C24.7 8.4 3 8.1 3 29.3 3 42.7 18.8 62 25.7 62c3.1 0 4.2-2 6.1-2 1.8 0 2.4 1.9 6.4 1.9C45.1 62 61 42.8 61 29.3 61 8.1 39.3 8.4 32 15.8z" fill="#8cc63e" /></g></svg>

A sample SVG font rendering code, using librsvg: https://gitlab.freedesktop.org/freetype/freetype-demos/-/blob/master/src/rsvg-port.c

viewBox / width, height is only used to determine scaling factors - lines 225 ... 256
Only element with ID equal to glyph<ID> is rendered, using its actual bounding box, document viewBox / width, height are ignored - lines 303 ... 333

@bruvzg thx for your comment.
One thing is how to deal without any viewBox and viewPort information.
But the other one is to deal with examples that you gave. Please correct me if I'm wrong, but according to the svg w3c standard, the svg from EmojiOne Color font should be drawn exactly as it is - so one can see only a small part of the apple.
There is a comment in the librsvg:
"TODO: I'm not sure what the standard says about the situation if
ViewBox as well as width/height are present; however, I've
never seen that situation in real fonts."
I know what the svg standard says, doesn't it apply to fonts/emoji?
I'm not familiar with this  document - just scrolled and I didn't find any special info about viewBox in svgs. Do you know something more? Some documentation/standard, that I should check?
So probably we could treat viewbox/widht&height info in some other way for emoji, but will it be correct for all of the files?

Relevant part of OpenType specification - https://docs.microsoft.com/en-us/typography/opentype/spec/svg#coordinate-systems-and-glyph-metrics
viewBox / widht / height are used only to determine scaling, if it's not present unitsPerEm is used as a viewport size.

But the other one is to deal with examples that you gave. Please correct me if I'm wrong, but according to the svg w3c standard, the svg from EmojiOne Color font should be drawn exactly as it is - so one can see only a small part of the apple.

It should not be clipped, I guess it's valid for plain SVG, but not for OpenType.

Although the initial viewport size is the em square, the viewport must not be clipped. The  element is assumed to have a clip property value of auto, and an overflow property value of visible. A font should not specify clip or overflow properties on the  element. If clip or overflow properties are specified on the  element with any other values, they must be ignored.

@mgrudzinska close?

yes!
solved in #1318 with fix in #1344


[SPARK-6483][SQL]Improve ScalaUdf called performance.

As issue SPARK-6483 description, ScalaUdf is low performance because of calling asInstanceOf to convert per record.
With this, the performance of ScalaUdf is the same as other case.
thank @lianhuiwang for telling me how to resolve this problem.

Can one of the admins verify this patch?

Hmm, have you try what the performance gain by this change? From my understanding the bottleneck is in the function call ScalaReflection.convertToScala

Before this change, it takes 17 minutes, and now takes 5 minutes, which is the same as HiveContext + udf floor and non-udf

OK, probably we can also move the children.size match {..} out of the eval.

I mean we can do something like
val f = children.size match {
  case 1 =>
   val func = function.asInstanceOf[(Any) => Any]
   val child0 = children(0)
    (input: Row) => {
      func(ScalaReflection.convertToScala(child0.eval(input), child0.dataType)))
    }

  case 2 =>
   val func = function.asInstanceOf[(Any) => Any]
   val child0 = children(0)
   val child1 = children(1)

   (input: Row) => {
    func(ScalaReflection.convertToScala(child0.eval(input), child0.dataType))
       ScalaReflection.convertToScala(child1.eval(input), child1.dataType)))
    }
}

def eval(input: Row) = f(input)

OK, I will modify code and test again.

@chenghao-intel , I change code and test it, the result is the same as last commit  , is 5 minutes.
Please help me for checking code.

@zzcclp I will run the benchmark in my local machine, will get back soon.
@liancheng , can you trigger the unit test?

Verified the code change by the following micro-benchmark
import org.apache.spark.sql.catalyst.expressions._
import org.apache.spark.sql.types._

case class Floor(child: Expression) extends UnaryExpression with Predicate {
  override def foldable = child.foldable
  def nullable = child.nullable
  override def toString = s"Floor $child"

  override def eval(input: Row): Any = {
    child.eval(input) match {
      case null => null
      case ts: Int => ts - ts % 300
    }
  }
}

object T {
  def benchmark(count: Int, expr: Expression): Unit = {
    var i = 0
    val row = new GenericRow(Array[Any](123, 21, 42))
    val s = System.currentTimeMillis()
    while (i < count) {
      expr.eval(row)
      i += 1
    }
    val e = System.currentTimeMillis()

    println (s"${expr.getClass.getSimpleName}  -- ${e - s} ms")
  }
  def main(args: Array[String]) {
    def func(ts: Int) = ts - ts % 300
    val udf0 = ScalaUdf(func _, IntegerType, BoundReference(0, IntegerType, true) :: Nil)
    val udf1 = Floor(BoundReference(0, IntegerType, true))

    benchmark(1000000, udf0)
    benchmark(1000000, udf0)
    benchmark(1000000, udf0)

    benchmark(1000000, udf1)
    benchmark(1000000, udf1)
    benchmark(1000000, udf1)
  }
}

Without the code change it outputs
ScalaUdf  -- 1183 ms
ScalaUdf  -- 887 ms
ScalaUdf  -- 929 ms
Floor  -- 49 ms
Floor  -- 15 ms
Floor  -- 21 ms
With the code change, it outputs
ScalaUdf  -- 382 ms
ScalaUdf  -- 255 ms
ScalaUdf  -- 247 ms
Floor  -- 27 ms
Floor  -- 6 ms
Floor  -- 8 ms
Conclusions:

The code change will improve the performance of scala udf by 2-3x
Scala UDF is in very low performance compare to the built-in type of Expression.

We probably need to provide more efficient way of UDF extension interface.

ok to test

Test build #29093 has started for   PR 5154 at commit 2b7afc0.

This patch merges cleanly.

Test build #29093 has finished for   PR 5154 at commit 2b7afc0.

This patch passes all tests.

This patch merges cleanly.

This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29093/
Test PASSed.

Test build #29134 has started for   PR 5154 at commit c899e6d.

This patch does not merge cleanly.

@chenghao-intel ,I have update the code. can you take a look again. thanks.

You need to fetch the latest code and resolve the conflicts.

@zzcclp LGTM, except some small issues.

Test build #29134 has finished for   PR 5154 at commit c899e6d.

This patch fails Spark unit tests.

This patch does not merge cleanly.

This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29134/
Test FAILed.

Test build #29141 has started for   PR 5154 at commit b73836a.

This patch merges cleanly.

@SparkQA , merge again.

Test build #29142 has started for   PR 5154 at commit 0a8cdc3.

This patch merges cleanly.

@chenghao-intel , I have update the code. can you take a look again. thanks.
@AmplabJenkins , re-test

Test build #29141 has finished for   PR 5154 at commit b73836a.

This patch fails Spark unit tests.

This patch merges cleanly.

This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29141/
Test FAILed.

Test build #29142 has finished for   PR 5154 at commit 0a8cdc3.

This patch fails Spark unit tests.

This patch merges cleanly.

This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29142/
Test FAILed.

@zzcclp
children:Seq[Expression] essentially can be either List[Expression] or ArraySeq[Expression], the later doesn't support the pattern matching.
Probably you have to use the code like:
val f = children.size match {
  case 1 =>
   val func = function.asInstanceOf[(Any) => Any]
   val child0 = children(0)
    (input: Row) => {
      func(ScalaReflection.convertToScala(child0.eval(input), child0.dataType)))
    }

  case 2 =>
   val func = function.asInstanceOf[(Any) => Any]
   val child0 = children(0)
   val child1 = children(1)

   (input: Row) => {
    func(ScalaReflection.convertToScala(child0.eval(input), child0.dataType))
       ScalaReflection.convertToScala(child1.eval(input), child1.dataType)))
    }
}

@chenghao-intel , can you review again, thanks.

Test build #29152 has started for   PR 5154 at commit cc6868e.

This patch merges cleanly.

@AmplabJenkins , please re test

Test build #29152 has finished for   PR 5154 at commit cc6868e.

This patch passes all tests.

This patch merges cleanly.

This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29152/
Test PASSed.

Add a newline at the end of source file, does it need to re test?

Test build #29157 has started for   PR 5154 at commit 5ac6e09.

This patch merges cleanly.

@AmplabJenkins , please re test, thanks.

The unit test will be auto-triggered, once the code changed, you needn't say anything to @AmplabJenkins .

OK, I am new sparker, :smile:

Test build #29157 has finished for   PR 5154 at commit 5ac6e09.

This patch passes all tests.

This patch merges cleanly.

This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29157/
Test PASSed.

@zzcclp @chenghao-intel Thanks for working on this and the review comments! Merged to master.

@zzcclp Would you please set your real name on both GitHub and JIRA so that our script can put your name on the credit list of the next release? Also, it would be good if you can set your name in git config:
$ git config --global user.name "Your Name"

@liancheng , I already set my real name on my GitHub Name and JIRA Full Name. :smiley:


Automatically lock / log out of wallet after a period of time

Story
As a user
I want to be logged out automatically / have my wallet lock automatically after a period of inactivity
So that my wallet is kept secure even if I've left it open by mistake
Acceptance Criteria

When a wallet is left in a logged in / viewed stated in the app, and there has been a set period of inactivity (to be defined), I am automatically moved to the wallet listing page and effectively "signed out" of my wallet (i.e. it is now locked)
To unlock it, I go through the usual steps i.e. select from list, enter passphrase

Tasks
Note it may be worth working on this ticket and this one at the same time:
Wallet Login: Once per session#360

It requires work to be done in the backend of the wallet:
https://github.com/vegaprotocol/vega/issues/5736