METRON-2073: Create in-memory use case for enrichment with map type and flatfile summarizer

Contributor Comments
This PR adds a Stellar function for performing in-memory enrichments.  Similar to ENRICHMENT_GET, which performs lookups in HBase, this function performs lookups from an in-memory map that is loaded from HDFS.  The nosql_table, enrichment_type, and column_family parameters in ENRICHMENT_GET are similar to the ENRICHMENT_IN_MEMORY_GET path parameter in that they uniquely identify the data source.  The indicator parameter is the same (the key used to lookup an enrichment).  The new function is very similar to the OBJECT_GET function (they use the same cache abstraction) except that it performs the lookup, offers configuration specifically for in-memory enrichments, and potentially more.  The in-memory objects are lazy-loaded, meaning they are loaded the first time an ENRICHMENT_IN_MEMORY_GET is executed.
The Jira states the first pass should be a 1-time load but it wasn't difficult to expose cache expiration settings.  This allows cached enrichments to be refreshed at an interval and will support streaming enrichments.
Cache settings are applied in the following order:

Hardcoded defaults defined in constants
Top-level global config settings (used by the OBJECT_GET function)
Global config settings specific for ENRICHMENT_IN_MEMORY_GET

These settings are passed into the Guava CacheBuilder when the function is initialized.
Changes Included

An ObjectCache abstraction was created from the code in the Stellar OBJECT_GET function.  This abstraction is shared by both the OBJECT_GET and ENRICHMENT_IN_MEMORY_GET functions.  The current settings used in OBJECT_GET should be backwards compatible.
Dedicated cache settings for ENRICHMENT_IN_MEMORY_GET can be configured in the global config with the in.memory.enrichment.settings key.
Unit tests were added for new code and expanded for existing code.
Logging was added in various places to make it easier to determine when entries are being flushed and for what reason.

Testing Instructions
This has been tested in full dev with the Stellar CLI.

Spin up full dev and ensure data is flowing with no errors.
Create a enrichments.csv file with enrichment data:

key,value

Create a enrichments_updated.csv file with updated enrichment data:

key,updated value

Create a summarizer.json file that will serialize this data as a map in HDFS:

{
  "config" : {
    "columns" : {
      "key" : 0,
      "value" : 1
    },
    "state_init" : "{}",
    "state_update" : {
      "state" : "MAP_PUT(key, value, state)"
    },
    "separator" : ","
  },
  "extractor" : "CSV"
}

Use the flatfile_summarizer.sh script to create serialized files:

/usr/metron/0.7.1/bin/flatfile_summarizer.sh -i ./enrichments.csv -o ./enrichments.ser -e summarizer.json -p 1
/usr/metron/0.7.1/bin/flatfile_summarizer.sh -i ./enrichments_updated.csv -o ./enrichments_updated.ser -e summarizer.json -p 1

Put the first file in HDFS:

hdfs dfs -put enrichments.ser /tmp/enrichments.ser

Start the Stellar shell and define a field we can use for the enrichments:

[root@node1 tmp]# /usr/metron/0.7.1/bin/stellar --zookeeper node1:2181
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/metron/0.7.1/lib/metron-profiler-repl-0.7.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/<IP_ADDRESS>0-37/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Stellar, Go!
Functions are loading lazily in the background and will be unavailable until loaded fully.
{es.clustername=metron, es.ip=node1:9200, es.date.format=yyyy.MM.dd.HH, parser.error.topic=indexing, update.hbase.table=metron_update, update.hbase.cf=t, es.client.settings={}, profiler.client.period.duration=15, profiler.client.period.duration.units=MINUTES, enrichment.list.hbase.provider.impl=org.apache.metron.hbase.HTableProvider, enrichment.list.hbase.table=enrichment_list, enrichment.list.hbase.cf=t, user.settings.hbase.table=user_settings, user.settings.hbase.cf=cf, bootstrap.servers=node1:6667, source.type.field=source:type, threat.triage.score.field=threat:triage:score, enrichment.writer.batchSize=15, enrichment.writer.batchTimeout=0, profiler.writer.batchSize=15, profiler.writer.batchTimeout=0, geo.hdfs.file=/apps/metron/geo/default/GeoLite2-City.tar.gz, asn.hdfs.file=/apps/metron/asn/default/GeoLite2-ASN.tar.gz, object.cache.expiration.minutes=1, in.memory.enrichment.settings={cache.expiration=15, time.unit=SECONDS}}
[Stellar]>>> field := 'key'
key

Perform an in-memory enrichment.  There should be a slight delay (for the initial load) but subsequent calls should return immediately:

[Stellar]>>> ENRICHMENT_IN_MEMORY_GET('/tmp/enrichments.ser', field)
value

The default cache expiration is 24 hours.  Exit the Stellar CLI and set the object cache expiration to 1 minutes in the global config:

curl -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' -d '{
  ...
  "object.cache.expiration.minutes": 1
}' 'http://user:password@node1:8082/api/v1/global/config'

Note the object.cache.expiration.minutes setting is maintained for backwards compatibility.  You can also set the expiration to 1 minute with these settings:
{
  "object.cache.expiration": 1,
  "object.cache.time.unit": "MINUTES"
}

Start the Stellar CLI again and perform the enrichment again.  You should get the same result:

[root@node1 tmp]# /usr/metron/0.7.1/bin/stellar --zookeeper node1:2181
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/metron/0.7.1/lib/metron-profiler-repl-0.7.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/<IP_ADDRESS>0-37/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Stellar, Go!
Functions are loading lazily in the background and will be unavailable until loaded fully.
{es.clustername=metron, es.ip=node1:9200, es.date.format=yyyy.MM.dd.HH, parser.error.topic=indexing, update.hbase.table=metron_update, update.hbase.cf=t, es.client.settings={}, profiler.client.period.duration=15, profiler.client.period.duration.units=MINUTES, enrichment.list.hbase.provider.impl=org.apache.metron.hbase.HTableProvider, enrichment.list.hbase.table=enrichment_list, enrichment.list.hbase.cf=t, user.settings.hbase.table=user_settings, user.settings.hbase.cf=cf, bootstrap.servers=node1:6667, source.type.field=source:type, threat.triage.score.field=threat:triage:score, enrichment.writer.batchSize=15, enrichment.writer.batchTimeout=0, profiler.writer.batchSize=15, profiler.writer.batchTimeout=0, geo.hdfs.file=/apps/metron/geo/default/GeoLite2-City.tar.gz, asn.hdfs.file=/apps/metron/asn/default/GeoLite2-ASN.tar.gz, object.cache.expiration.minutes=1, in.memory.enrichment.settings={cache.expiration=15, time.unit=SECONDS}}
[Stellar]>>> field := 'key'
key
[Stellar]>>> ENRICHMENT_IN_MEMORY_GET('/tmp/enrichments.ser', field)
value

In a separate window, upload the updated enrichment file to HDFS:

hdfs dfs -put -f /tmp/enrichments_updated.ser /tmp/enrichments.ser

After a minute the value should change to the value in enrichments_updated.ser:

[Stellar]>>> ENRICHMENT_IN_MEMORY_GET('/tmp/enrichments.ser', field)
updated value

The cache expiration can also be set specifically for ENRICHMENT_IN_MEMORY_GET.  Put the previous enrichments.ser back in HDFS and change the cache settings to 15 seconds:

curl -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' -d '{
  ...
  "object.cache.expiration.minutes": 1,
  "in.memory.enrichment.settings": {
    "cache.expiration": 15,
    "time.unit": "SECONDS"
  }
}' 'http://user:password@node1:8082/api/v1/global/config'

Perform steps 9 - 11.  Now the value should be updated after 15 seconds.

Next Steps
This is intended to be a first pass and there are still some outstanding items.  I am planning on adding javadocs and a section in our READMEs describing how to do in-memory enrichments.
Are there other features people would like to see added here?  I think it could be useful to set cache settings for each object in HDFS (currently they are global to all objects and can only be changed on initialization).  This would make things more complex and require some design work.
Does anyone have suggestions for more appropriate function and setting names?
What else would you change about this?
Pull Request Checklist
Thank you for submitting a contribution to Apache Metron.
Please refer to our Development Guidelines for the complete guide to follow for contributions.
Please refer also to our Build Verification Guidelines for complete smoke testing guides.
In order to streamline the review of the contribution we ask you follow these guidelines and ask you to double check the following:
For all changes:

[x] Is there a JIRA ticket associated with this PR? If not one needs to be created at Metron Jira.
[x] Does your PR title start with METRON-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
[x] Has your PR been rebased against the latest commit within the target branch (typically master)?

For code changes:

[x] Have you included steps to reproduce the behavior or problem that is being changed or addressed?

[x] Have you included steps or a guide to how the change may be verified and tested manually?

[x] Have you ensured that the full suite of tests and checks have been executed in the root metron folder via:
mvn -q clean integration-test install && dev-utilities/build-utils/verify_licenses.sh

[x] Have you written or updated unit tests and or integration tests to verify your changes?

[x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?

[x] Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent?

For documentation related changes:

[ ] Have you ensured that format looks appropriate for the output in which it is rendered by building and verifying the site-book? If not then run the following commands and the verify changes via site-book/target/site/index.html:
cd site-book
mvn site

Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.
It is also recommended that travis-ci is set up for your personal repository such that your branches are built there before submitting a pull request.

Would it be worth changing the caching to use caffeine, as we have in other locations, given the performance gains we've seen for larger caches with caffeine?

On naming: since this is very much based on, and similar to OBJECT_GET, how about ENRICHMENT_OBJECT_GET?

The other thing worth thinking about is some guard-rails around cache size. If we load a large file, it seems like there is a good chance of blowing up the enrichment topology somewhere hard to debug with a OOM. That's probably something that needs pushing down to OBJECT_GET too to be honest.

The latest commits should address your feedback @simonellistonball.  Have you given any thought to being able to configure different cache settings for different paths/objects?

I believe I've addressed the feedback so far with the latest commit.  Let me know how it looks now and what I'm missing.

+1, nice work @merrimanr


Ionic 3 CORS issue

Hi,
I am trying to get this great package into Ionic 3 (only websocket client support I guess?)
I set up a websocket client like the example:
var websocketClient = new OSC({plugin: new OSC.WebsocketClientPlugin({host: address', port:9011})});
When running this on the ios simulator I get a cors issue:

Origin http://<IP_ADDRESS>:8100 is not allowed by Access-Control-Allow-Origin.

I tried setting a proxy which is recommended by ionic but this doesn't fix the problem
Any ideas?

CORS issues are not really related to this library, maybe this helps though: http://blog.ionic.io/handling-cors-issues-in-ionic/


Juno 1.0.0: new cask.

Juno is an IDE for the Julia language based on LightTable.

Thank you for the contribution. It will be merged in a different pull request due to some simple recommended alterations. Your contribution is still included (and still credited to you), with the appropriate modifications, when applicable.
Changes:

Facilitated future updates.
Added a small note pointing out the download host is officially different from the project’s home.
Corrected name.

Thank you.


[WIP] Build against TF2.4

Description
Update TFA to build against TF2.4
Fixes #2110

Thanks @seanpmorgan for the pointer. I think --copt="/Zc:preprocessor" will resolve the issue in Visual Studio on GitHub.Actions.

Hi, when is this PR expected to be merged?

Hi, when is this PR expected to be merged?

Stable TF 2.4 just released an hour ago, so will update PR and then we can merge

@seanpmorgan It doesn't build cause https://github.com/tensorflow/addons/pull/2238/ check https://github.com/tensorflow/addons/pull/2238/files#diff-347b0ae22433de298d1f5f70dd47844f85d0b05fc3d970ccdec0a8c6d8e4d463R66

/cc @fsx950223 Do you have a already a fix for your todo at https://github.com/tensorflow/addons/pull/2238/files#diff-347b0ae22433de298d1f5f70dd47844f85d0b05fc3d970ccdec0a8c6d8e4d463R66 ?

/cc @fsx950223 Do you have a already a fix for your todo at https://github.com/tensorflow/addons/pull/2238/files#diff-347b0ae22433de298d1f5f70dd47844f85d0b05fc3d970ccdec0a8c6d8e4d463R66 ?

I fixed it in #2291


Add Mailfence

name: Mailfence
url: https://mailfence.com/flatx/index.jsp#tool=prefs&page=accg
difficulty: easy
notes: Click "Delete my account"
domains: https://mailfence.com/

additional domain: https://kb.mailfence.com/

An entry for Mailfence already exists. (added in #425)
Should this be an update instead of a new entry?


Internal Links

https://github.com/kohheepeace/mr-pdf/issues/10#issuecomment-851393000
After adding TOC, internal links didn't work. So, the HTML parse for TOC is related.

Hey @kohheepeace did you find any solution to it? We're in the final stage and planning to use this package in production and make the PDF's live.
This fix might be very useful. :)

@kohheepeace any updates on this issue?

Since internal links are not working, is it possible to have an option to disable the generation of the TOC page? I manually disable all hyperlinks in my PDF and as such the TOC is not that useful when you cannot click any of them and I haven't found an option to skip the TOC.

kohheepeace/docusaurus-pdf#2

@kohheepeace
we are trying this workaround, did not quite get "Add id to each article: ", can you give an example?


[fix][client] ZeroQueueConsumer may not increase the permit after unloading the topic.

Fixes #15679
Master Issue: #15679
Motivation
ZeroQueueConsumer may not increase the permit after unloading the topic. because internalReceiveAsync is an async method. when it invokes increaseAvailablePermits(cnx()); the cnx may not be the new cnx.
https://github.com/apache/pulsar/blob/c642731562205261e5bc9bcba16b42df354e745c/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ZeroQueueConsumerImpl.java#L85-L94
Modifications

re-send permit when connection reconnected.

Verifying this change

[x] Make sure that the change passes the CI checks.

Documentation

[x] doc-not-needed

@mattisonchao It seems that this PR is not likely to get merged in the final release of branch-2.7? Should this be a release blocker?

@mattisonchao   hi, I move this PR to release/2.9.5, if you have any questions, please ping me. thanks.

Since we will start the RC version of 3.0.0 on 2023-04-11, I will change the label/milestone of PR who have not been merged.

The PR of type feature is deferred to 3.1.0
The PR of type fix is deferred to 3.0.1
So drag this PR to 3.0.1

As discussed on the mailing list https://lists.apache.org/thread/w4jzk27qhtosgsz7l9bmhf1t7o9mxjhp, there is no plan to release 2.9.6, so I am going to remove the release/2.9.6 label


Exif data is lost when getting an image from the photo library (iOS)

Bug Report
Problem
When opening/adding an image (HEIC or JPEG) from camera roll on iOS and destinationType in camera options is set to DestinationType.FILE_URI the image EXIF data is lost
What is expected to happen?
Exif data must be preserved.
What does actually happen?
Exif data is lost.
Information

The problem exists on iOS only. The camera is used with the below options
const options: CameraOptions = {
   destinationType: Camera.DestinationType.FILE_URI,
   sourceType: this.camera.PictureSourceType.PHOTOLIBRARY,
   encodingType: this.camera.EncodingType.JPEG,
   correctOrientation: true
};

Command or Code

N/A
Environment, Platform, Device

ionic info

Ionic:

   Ionic CLI          : 5.2.7 (/Users/alexryltsov/.nvm/versions/node/v10.16.2/lib/node_modules/ionic)
   Ionic Framework    : ionic-angular 3.9.8
   @ionic/app-scripts : 3.2.4

Cordova:

   Cordova CLI       : 9.0.0<EMAIL_ADDRESS>   Cordova Platforms : android 8.0.0, ios 5.0.1
   Cordova Plugins   : cordova-plugin-ionic-keyboard 2.1.3, cordova-plugin-ionic-webview 4.1.3, (and 25 other plugins)

Utility:

   cordova-res : 0.6.0 (update available: 0.8.1)
   native-run  : 0.2.8 (update available: 0.2.9)

System:

   Android SDK Tools : 26.1.1 (/Users/alexryltsov/Library/Android/sdk/)
   ios-sim           : 8.0.1
   NodeJS            : v10.16.2 (/Users/alexryltsov/.nvm/versions/node/v10.16.2/bin/node)
   npm               : 6.9.0
   OS                : macOS Catalina
   Xcode             : Xcode 11.2 Build version 11B52

Version information

cordova-plugin-camera 4.1.0
Checklist

[x] I searched for existing GitHub issues
[x] I updated all Cordova tooling to most recent version
[x] I included all the necessary information above

I have created a very easy to run example that demonstrates that EXIF information is lost:
https://github.com/codeconsole/exiftest
@ryaa I am unable to get your version of the plugin working with the latest version of cordova.
% cordova --version
10.0.0

  git clone https://github.com/codeconsole/exiftest
  cd exiftest
  cordova prepare
  cordova plugin remove cordova-plugin-camera
  cordova plugin add https://github.com/ryaa/cordova-plugin-camera.git
  cordova run ios
  xcrun simctl addmedia booted example_with_exif.jpeg

As soon as I try to select a photo, the app crashes (It works fine with the normal plugin)

Created a pull request with @ryaa 's changes and missing commits from master:
https://github.com/apache/cordova-plugin-camera/pull/712
Can be verified working by running the following:
% cordova --version
10.0.0

  git clone https://github.com/codeconsole/exiftest
  cd exiftest
  cordova prepare
  cordova plugin remove cordova-plugin-camera
  cordova plugin add https://github.com/codeconsole/cordova-plugin-camera.git#ryaa
  cordova run ios

Choose a photo and notice all the exif information is there.

Resolved by #712


bug: URGENT- Blocks software update - Menu disappear when navigate between multiple menus

Bug Report
Ionic version:

[x] 4.8
It blocks my next ionic update for my mobile app. I need a fast Bug Fix on this.
Current behavior:

I have a left side menu(menu 1). When I click on one of the buttons in the menu, it brings me to another page with a different menu (menu 2). I can navigate in this menu (menu 2)(what it suppose to do.) In this menu, I'm having a 'back' button to return to the last page (which contains menu 1). When I click on this button, we back to the page of the menu 1, but the menu disappear and we can't open it in any ways. When I refresh the page, the menu 1 is back and everything is Ok.
Expected behavior:

When I click on menu 2 'back' button to return to menu 1, it returns and shows the first menu correctly, then we can click on 'go to menu 2' etc.
Ionic info:

Ionic:

   Ionic CLI                     : 5.2.7 (C:\Users\yannick.jacques\AppData\Roaming\npm\node_modules\ionic)
   Ionic Framework               : @ionic/angular 4.8.0
   @angular-devkit/build-angular : 0.802.2
   @angular-devkit/schematics    : 8.2.2
   @angular/cli                  : 8.2.2
   @ionic/angular-toolkit        : 2.0.0

Utility:

   cordova-res : not installed
   native-run  : not installed

System:

   NodeJS : v12.3.1 (C:\Program Files\nodejs\node.exe)
   npm    : 6.9.0
   OS     : Windows 10

Thanks for the issue. Please try this again on Ionic 4.9.0.

Unless I've misunderstood, I believe this is the same bug I have since the end of july:

My issue : https://github.com/ionic-team/ionic/issues/18974
Bug repo: https://github.com/iamkinetic/ionic4-multiple-menus-bug
I guess @yjacques940 could confirm if this is indeed the same bug.

I've first discovered that issue when I updated to Ionic 4.7.0.  I've tried Ionic 4.9 yesterday and the issue is still there.  I've just push the ionic 4.9 update in my bug reproduction repository so you can reproduce it.
I thought i had found a workaround, but it's not working as good as I first thought, so I'm also waiting for a true fix so I can go ahead and update my applications that currently have that issue.

I've tried v4.9 and nothing has changed. I tried @iamkinetic repo, the navigation is different but the bug is the same.

@yjacques940 Thanks for the follow up. Just so I understand, are you saying this bug is the same as https://github.com/ionic-team/ionic/issues/18974?

Yes.

Ok great, thanks for the follow up. In that case I'm going to close this issue in favor of https://github.com/ionic-team/ionic/issues/18974, but I will be sure to link the two issues. I'm going to dig into this and see what I can find.
Thanks!


:sparkles:  split go releaser archives

Summary
Release file copies, and not softlink'ed one. Go release does not know how to handle those
Related issue(s)
Fixes #
Release Notes

NONE

/remove-lifecycle rotten

Why don't we reduce or plugins to a single one

That breaks down with kubectl create workspace, xref https://github.com/kcp-dev/kcp/pull/3154.
From all the commands kubectl ws is what I use by far most often. IMO it deserves to be its own command.

This stops the bleeding.
Follow-up https://github.com/kcp-dev/kcp/pull/3154.
/lgtm
/approve


Adding Crud cases for Postman testing.

refs: https://openscience.atlassian.net/browse/SVCS-419
Purpose
Add crud related and abuse case testing collections to waterbutlers Postman tests.
Summary of changes
Updated the documents to reflect added tests and clarified a few things.
Added Valid case CRUD testing
*Added Create file/folders testing
*Added Update file/folders testing
*Added Delete file/folders testing
Added Abuse case testings

Missing / in path testing
Extra / in path testing on files
Create files/folders with missing name parameter
Create file/folders with an already existing name

QA/testing notes
None, for internal use only.

Coverage remained the same at 76.68% when pulling 27cf82b8e0198075ba8e7d98f4981f1e887b8bfd on AddisonSchiller:feature/postman-tests into b787a92c180697cb8fb2f5b6c402cf7b931b339d on CenterForOpenScience:develop.

Coverage remained the same at 76.68% when pulling 27cf82b8e0198075ba8e7d98f4981f1e887b8bfd on AddisonSchiller:feature/postman-tests into b787a92c180697cb8fb2f5b6c402cf7b931b339d on CenterForOpenScience:develop.

Coverage remained the same at 76.68% when pulling 27cf82b8e0198075ba8e7d98f4981f1e887b8bfd on AddisonSchiller:feature/postman-tests into b787a92c180697cb8fb2f5b6c402cf7b931b339d on CenterForOpenScience:develop.

Coverage remained the same at 79.43% when pulling c11abe74c98312d802d2e720ba5d4b20ac59a976 on AddisonSchiller:feature/postman-tests into 70f423d1176a4d4d74eae3efcf3f79d3679811bc on CenterForOpenScience:develop.

Coverage remained the same at 79.43% when pulling c11abe74c98312d802d2e720ba5d4b20ac59a976 on AddisonSchiller:feature/postman-tests into 70f423d1176a4d4d74eae3efcf3f79d3679811bc on CenterForOpenScience:develop.

Closed in favor of [255][https://github.com/CenterForOpenScience/waterbutler/pull/255]


Added a method to split by space instead of using regex to make token…

Using regular expression with look-ahead/look-behind to split tokens is expensive. (http://stackoverflow.com/questions/9831034/poor-performance-speed-of-regex-with-lookahead)
Since the reason we use look-ahead/look-behind is to keep spaces, I replaced it with a custom function.
In my case, it reduced the time to tokenize 1000 documents from 39 seconds to 17 seconds.

Thank! This looks great. Could you please make sure mvn test succeeds?

@caniszczyk, could you please update Travis to run on Java 8? It has been broken for a while. Thanks in advance!

@nlpenguin Thanks for reminding me of the tests!
Because I changed to group continuous spaces into one Space token, it broke one test cases(and one another because it somewhat changed the order of outputs of extrace phrases function). I updated those tests and added one test to KoreanChunkerTest.scala to check space tokens.
All tests passed now.
Results :
Tests run: 86, Failures: 0, Errors: 0, Skipped: 0

LGTM

@jpark-dev  Thank you so much for your heroic contribution!!
I will publish the new version by Friday.

Thanks for the merge, and thanks for the library. :)


Adding basic TokenProviders

Define abstract class TokenProvider to define a contract for
implementations.
Create ServicePrincipalProvider which uses well-known environment
variables should they be set.
Create a "ProviderChain" which will allow a cascading search for
TokenProviders that can fulfill a request.

Closes #7
Closes #8

More context around this PR, the idea is to emulate the behavior that was added to this repository's .NET sister via this PR: https://github.com/AzureAD/microsoft-authentication-extensions-for-dotnet/pull/13

Closing old PR/issues. The "token provider chain" concept has since been implemented in Azure SDK's Azure Identity package. The MSAL EX package here specialized to become a lower-level building block that focuses on data encryption and cross-process locking.


Persistent list for previously ran commands and commands being entered.

I wish that this plugins window was similar to q: where I can see the previously ran commands and use vim keybindings. It'd also be great to see command suggestions when typing commands without having to press tab to see a list.

I wish that this plugins window was similar to q: where I can see the previously ran commands

I can't do that. You might want to check out this: cmdbuf.nvim or maybe Telescope (specifically Telescope command_history).

and use vim keybindings

There is an issue with normal. I'm working on it.

It'd also be great to see command suggestions when typing commands without having to press tab to see a list.

For this you'll need full on completion engine.

Okay


Unable to use dependency since some classes are not accessable

Hi,
Ive tried to use the maven dependency in my project,
but ended up not being able to Query anything as some classes are private..
i just ended up forking your project xD

Hey there, sorry for the rather late response. Could you tell me a bit more about this issue? I can't seem to find any private classes.

That was long ago bro i cant remember exactly but,
i added your code as a dependency, but was unable to call any event or any query..
i had to have the source code in order to make it work.. so basically its not usable, test it.. xD
anyway i'm already with some nice features on my bot after forking this. thanks a lot.

Alright, since I am unable to reproduce this issue, and you have already resolved it for yourself, I'm going to close this issue now.


[9.x] Adds Arr::join()

This PR adds join method from Collections to Arr.
$stack = ['Tailwind', 'Alpine', 'Laravel', 'Livewire'];

Arr::join($stack, ', ', ' and ');

// Tailwind, Alpine, Laravel and Livewire

Should the Collection use then Arr::join() instead?


Feature/add ts datasets

Added several datasets

Migrated datasets from main pycaret repo to this repo
Added white noise datasets to time series
Added random walk datasets to time series
Reorganized folder structure and file names

@ngupta23 Should we not also change the pointer in the get_data function? Otherwise, it will keep reading the dataset from the old location. However, the question in my mind is if we change the pointer and remove pycaret/datasets this means anybody who will use the old version of pycaret won't be able to access data using get_data. Am I thinking right?

@ngupta23 Should we not also change the pointer in the get_data function? Otherwise, it will keep reading the dataset from the old location. However, the question in my mind is if we change the pointer and remove pycaret/datasets this means anybody who will use the old version of pycaret won't be able to access data using get_data. Am I thinking right?

Yes, But first this has to be approved and moved to main branch in this repo. Only then can we remove the dataset from pycaret and change he pointer.

@ngupta23 @moezali1 It would be great if, at least for one more release, we could to keep both links in get_data, with the old one being a fallback, just in case.

More feedback from meeting

Change "info" folder to "meta"
Interface change in pycaret

# Get data
get_data("juice" )                               # by default go to "common" folder (same as today)
get_data("1", folder="time_series/white_noise")  # gets data 1.csv for time series folder

# read all data in a folder for benchmarking purposes
for i in 1:500:
    data[i] = get_data(i, folder="time_series/white_noise")

# Get details about the datasets
get_data(index=True)                                   # by default = "common" folder (slight deviation from today's interface)
get_data(index=True, folder="time_series/white_noise") # gets index for time series folder

@moezali1  are you ok with this interface change?

@ngupta23 @moezali1 It would be great if, at least for one more release, we could to keep both links in get_data, with the old one being a fallback, just in case.

I am thinking that for now, these changes will remain in the time_series branch (and pycaret-ts-alpha release) only. Once we merge time_series with main pycaret this change will automatically get incorporated in there. That way we can test this a bit before it gets incorporated into main pycaret package.

@ngupta23 @Yard1 I agree with what Antoni said. We will have to keep the datasets folder in the pycaret repo for now.  get_data function on the master branch will keep pointing to that, whereas the get_data function of the time_series branch can use the new location and when we merge that into master we will ensure that datasets doesn't get removed from the pycaret repo otherwise people using old version won't be able to access datasets.
We can later think of some kind of deprecation cycle.


Update CSV's RBI to be Ruby 3 compatible

Per the tin. Here is a doc for an updated signatures
Motivation

Ruby 3
Test plan

See included automated tests.

We have a policy of testing changes to Sorbet against Stripe's codebase before
merging them. I've kicked off a test run for the current PR.  When the build
finishes, I'll share with you whether or how it failed. Thanks!
Stripe employees can see the build results here:
→ https://go/builds/bui_NjVXZWcYxqoBy7
→ https://go/builds/bui_NjVXvDasppraii
→ https://go/builds/bui_NjVXWLuOLTbTBB

We have a policy of testing changes to Sorbet against Stripe's codebase before
merging them. I've kicked off a test run for the current PR.  When the build
finishes, I'll share with you whether or how it failed. Thanks!
Stripe employees can see the build results here:
→ https://go/builds/bui_NjmFnHymNWdQqr
→ https://go/builds/bui_NjmFZgUKuSpkH1
→ https://go/builds/bui_NjmFeXwcCQHkXC


aggreprobe function fails when use = "score"

@yangleicq it seems this may be old package versioning again. I used the latest NCTools and GeomxTools and it ran through for me.


fix issues with spaces in paths

Doesn't seem to be a way to get NODE_OPTIONS to accept -r requirements with spaces in them (prob a node core bug / missing feature), so to support paths with spaces we instead use good ol' NODE_PATH to point to a folder where we have your injection requires and the then load them as node_modules.

2.5.1

Wow, that is sad :( Should we open an issue?

Yea, I'll do it


Nextzen switchover

Overview
This PR upgrades Tangram-es and the stylesheets to run against the Nextzen tileset.
Proposed Changes

Upgrades Tangram-es to 0.9.6
Upgrades stylesheets to current latest release
Now uses the new global SDK api key sdk_api_key defined in the styles
Removes hockeyapp as its not valid anymore.
Updates SDK version to 1.2.0 for eventual release

@tallytalwar agreed! I'll work on that next. Was trying to do this with minimal changes. The zip bundles require a little more work because of how we're semi-abusing the YAML string loading to power the Stylesheet customizer code (https://github.com/nextzen/ios/blob/master/MapzenSDK/Themes.swift#L254). Also, I'm still a little hesitant to include them as straight zip files into the git repo. I'll think on it.


bump dartx dependency version

fixes #247

Looks like the tests themselves are broken. So this change should not break anything and should be merged as soon as possible.

@marcelgarus this no longer fixes for flutter beta 1.18.0-11.1.pre. https://github.com/peterlauri/hive/commit/e5b70ba8824912fe9ad335c7d9bd44aa207d6d2f would also be needed.

Thanks a lot!


Kernel: VM Crash after compile-time error

This shouldn't happen, Fasta has reported compile-time errors so we shouldn't be reading the source code. Also, it looks like this is the VM parser, not kernel that's reading the code.
FAILED: dartk-vm debug_x64 co19/Language/Classes/Constructors/Generative_Constructors/initializers_t09
Expected: Pass
Actual: Crash
Compile-time error expected.
CommandOutput[vm]:

stdout:
tests/co19/src/Language/Classes/Constructors/Generative_Constructors/initializers_t09.dart:25:9: Error: Can't use x here.
tests/co19/src/Language/Classes/Constructors/Generative_Constructors/initializers_t09.dart:26:3: Error: Expected a function body, but got 'var'.

stderr:
../../runtime/vm/object.h: 7020: error: expected: raw() != null()
Dumping native stack trace for thread 16fb
  [0x00000000009bf363] dart::Profiler::DumpStackTrace()
  [0x00000000009bf363] dart::Profiler::DumpStackTrace()
tests/co19/src/Language/Classes/Constructors/Generative_Constructors/initializers_t09.dart:26:3: Error: Expected a function body, but got 'var'.
  [0x0000000000695f91] dart::DynamicAssertionHelper::Fail(char const*, ...)
  [0x0000000000692480] Unknown symbol
  [0x00000000008b989b] dart::String::StartsWith(dart::String const&) const
  [0x00000000008ccf33] dart::Library::LookupReExport(dart::String const&, dart::ZoneGrowableArray<long>*) const
  [0x00000000008cd35d] dart::Namespace::Lookup(dart::String const&, dart::ZoneGrowableArray<long>*) const
  [0x00000000008cb595] dart::Library::LookupImportedObject(dart::String const&) const
  [0x00000000008cad74] dart::Library::ResolveName(dart::String const&) const
  [0x00000000008a3d20] dart::Library::LookupClass(dart::String const&) const
  [0x000000000069f585] dart::ClassFinalizer::ResolveClass(dart::Class const&, dart::UnresolvedClass const&)
  [0x00000000006a19a4] dart::ClassFinalizer::ResolveTypeClass(dart::Class const&, dart::Type const&)
  [0x00000000006a1ef5] dart::ClassFinalizer::ResolveType(dart::Class const&, dart::AbstractType const&)
  [0x00000000006a0da4] dart::ClassFinalizer::FinalizeType(dart::Class const&, dart::AbstractType const&, dart::ClassFinalizer::FinalizationKind, dart::ZoneGrowableHandlePtrArray<dart::AbstractType const>*)
  [0x000000000094a466] dart::Parser::ParseTypeOrFunctionType(bool, dart::ClassFinalizer::FinalizationKind)
  [0x00000000009613b2] dart::Parser::ParseTypeArguments(dart::ClassFinalizer::FinalizationKind)
  [0x00000000009818ad] dart::Parser::ParseCompoundLiteral()
  [0x0000000000944547] dart::Parser::ParsePrimary()
  [0x000000000097a02c] dart::Parser::ParsePostfixExpr()
  [0x0000000000977852] dart::Parser::ParseUnaryExpr()
  [0x000000000094e43d] dart::Parser::ParseBinaryExpr(int)
  [0x0000000000950436] dart::Parser::ParseConditionalExpr()
  [0x0000000000945fe8] dart::Parser::ParseExpr(bool, bool)
  [0x000000000095189d] dart::Parser::ParseInitializedInstanceFields(dart::Class const&, dart::LocalVariable*, dart::GrowableArray<dart::Field*>*)
  [0x00000000009539a2] dart::Parser::ParseConstructor(dart::Function const&)
  [0x000000000093e224] dart::Parser::ParseFunc(dart::Function const&, bool)
  [0x000000000093c80c] dart::Parser::ParseFunction(dart::ParsedFunction*)
  [0x0000000000795a4d] Unknown symbol
  [0x000000000078a358] Unknown symbol
  [0x0000000000799405] Unknown symbol
  [0x0000000000792a3d] Unknown symbol
  [0x00000000007925db] dart::FlowGraphInliner::Inline()
  [0x00000000006fe313] dart::CompileParsedFunctionHelper::Compile(dart::CompilationPipeline*)
  [0x0000000000700472] Unknown symbol
  [0x0000000000701439] dart::Compiler::CompileOptimizedFunction(dart::Thread*, dart::Function const&, long)
  [0x00000000007033ab] dart::BackgroundCompiler::Run()
-- End of DumpStackTrace

Command[vm]: DART_CONFIGURATION=DebugX64 out/DebugX64/dart --dfe=out/DebugX64/gen/kernel-service.dart.snapshot --no-enable-malloc-hooks --ignore-unrecognized-flags --packages=/b/build/slave/vm-kernel-linux-debug-x64-be/build/sdk/.packages /b/build/slave/vm-kernel-linux-debug-x64-be/build/sdk/tests/co19/src/Language/Classes/Constructors/Generative_Constructors/initializers_t09.dart
Took 0:00:02.053081

Short reproduction command (experimental):
    python tools/test.py -cdartk -t120 --builder-tag no_ipv6 --vm-options --no-enable-malloc-hooks co19/Language/Classes/Constructors/Generative_Constructors/initializers_t09

Seen on vm-kernel-linux-debug-x64-be build 1218.

One theory is that this is a crash in the kernel-service isolate. That would make more sense.

So far, I didn't manage to reproduce this issue locally. We didn't get a coredump, though cl/467149 enabled this now. If we see the issue again, we'll get a dump.
The issue seems to be happening in the kernel isolate (because the main isolate should not invoke dart::Parser, it's using fasta/dartk to compile the sources). From the kernel isolate, it seems that a spawned background compiler hits some issue.

We're using app-jit, right? So should this even be possible?

"app-jit" is not a precompiler: If the training run of fasta did not execute some code (e.g. compile-time error code paths) then fasta will JIT new methods (which parses methods again and compiles them), the same happens on deoptimizations.

@peter-ahe-google  Have we ever seen this problem re-appear?
I would tend to close this issue if not, since there have been so many changes in the kernel -> vm integration in the last half year: The underlying issue might long be gone and there is little I can do without a way of reproducing it.


Proposal on how to handle measured node dimensions and internals

V12 is a few weeks in alpha testing and it seems that a lot of people are still struggling with the measured node dimensions. This PR is an attempt to make it easier to understand how to access those values.
v11
In v11 users receive the measured dimensions in a dimension change event. After applying the changes, the dimensions are stored in node.width and `node.height. This caused a lot of confusion, because users assume that they can use those attributes for setting the actual dimensions (which is not the case..) when it's only used to save the measured ones. We are also using those attributes as an indicator if a node is initialized.

user passes nodes
react flow measures nodes
rf sends dimension changes
user applies dimension (node.width and node.height is set now)
node is initialized and visible

v12 alpha
In v12 we wanted to make this more clear, so we decided to add another attribute called node.computed and store the measured dimensions there. We also needed to make this change, because wanted to be able to use node.width and node.height for specifying node dimensions on user side which is necessary for SSR. In my view this solution is better than what we have in v11, but it's still a bit confusing and you need to know that measured dimensions get added after you receive the initial dimension change events.

user passes nodes
react flow measures nodes
rf sends dimension changes
user applies dimension (node.computed.width and node.computed.height is set now)
node is initialized and visible

a better solution for v12?
With this PR I am proposing another solution for the node dimensions dilemma. With this PR we are not adding the measured dimensions to the user nodes, but only store them internally. This means that users won't receive the initial dimension change event and need to get the values with new helper functions and hooks:

user passes nodes
react flow measures nodes
rf stores dimensions internally
node is initialized and visible

This PR introduces a new InternalNode type that combines the previous computed and [internalsSymbol] values in an internals attribute. The existing Node type got a cleanup and doesn't include computed and [internalsSymbol] anymore. This separation between the node types makes it more clear what to expect on library and on user side. User can access the the internals in a custom node with the useInternalNode hook:
const internalNode = useInternalNode(id);
const nodeWidth = internalNode.internals.width;

or with the getInternalNode helper function:
const { getInternalNode } = useReactFlow();

const onLayout = (direction: string) => {
  const dagreGraph = new dagre.graphlib.Graph();
  dagreGraph.setGraph();

  nodes.forEach((node) => {
    const internalNode = getInternalNode(node.id);

    dagreGraph.setNode(node.id, {
      width: internalNode?.internals.width ?? 150,
      height: internalNode?.internals.height ?? 50,
    });
  });
 ...

wdyt?
Any feedback is highly appreciated. We are not sure, if we really want to go this path, because it would introduce a new library behaviour that is way more different than the current v12 (and v11) implementation.

for my use cases, I find it usefull to have the width / height in the nodes array, because I do a lot of layout computations where parents depend on the children. (simmilar to HTML flex behaiviour).
If the dimensions will not be available in the nodes array, things will get another layer of complexity. But it's more or less negligible.
I don't know if other libraries will suffer the same issue.
On the other hand,  the proposed approch seems more straight foreward and more clear.
If I see less steps in the logic it looks like a win for me. Humans are adaptable :))
+1 for the propsed approach.

(My thoughts - I'm not a React Flow expert!)
Intuitively speaking, it does seem like computed sizes belong on the internal node because the react flow internals create/update them and the user isn't meant to set them directly. The helpers/hooks make sense to me as well 👍🏻
My only concern is what if I want to fire off a layout function as soon as the internal node sizes are computed.
From what I can tell I don't really have a way to do that. I would need to bind to the regular node/edge dimension changes, which, from what I understand aren't exactly triggered to the internal computed size update. I guess that's the part that's fuzzy for me.

It makes sense to have a seperate "InputNode" (which is currently Node)  and an "InternalNode".

Thanks for the feedback so far!
@rob-gordon Good call! I will add an onNodesMeasured callback or something like that.

What I'm a bit confused about is why width and height and handles and so on don't shouldn't just go into data for the purposes of SSR?

@ncthbrt passing node.width/node.height also affects frontend only apps. As soon as you set it, we are using it as an inline style.

I start with all nodes having position of 0,0 (and wouldn't mind just having them blank instead), and then they go through layout. In order to do a layout, I need the height and width in order to calc positions.
Right now, I do this:
  const initialized = useNodesInitialized() && !nodes.some((n) => !n.computed);
  useEffect(() => {
    if (initialized) {
      setNodes(computeLayout(GraphPipelineLayout, nodes, edges));
      // window.requestAnimationFrame(() => fitView()); // todo: not working correctly
    }
  }, [initialized]);

which seems hacky.
  const initialized = useNodesInitialized() && !nodes.some((n) => !getInternalNode(node.id)?.internals.width);

seems about the same.
Anhow, does this mean that the user nodes are never mutated?

The new approach is nice, but introduces new challenges:
a. it's very hard to implement stuff like expandParent, if we don't adjust dimensions and positions after the initial rendering.
b. it's hard to tell if a node was already measured, if we don't add that information to the user node
I really like the separation of user nodes and internal nodes but not adding the dimensions doesn't work unfortunately. So the idea is to combine the current implementation with the new ideas from this proposal. For this I will try to keep the new internal node type and add the dimensions to the user nodes again.


[BUG]: Builds fail on CI/CD

What happened?
Tailwindcss was removed in favor of NextUI, commit.
However, the imports are still present in styles/globals.css.
@tailwind base;
@tailwind components;
@tailwind utilities;

Along with that, postcss and its config and, tailwindcss config is still present, which again causes the issue.
module.exports = {
  plugins: {
    tailwindcss: {},
    autoprefixer: {},
  },
};

Which causes the CI/CD builds to fail
What browsers are you seeing the problem on?
No response
Relevant log output
Error: Cannot find module 'tailwindcss'

Import trace for requested module:
./src/styles/globals.css
./src/pages/_app.tsx

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Hi @NiketanG thanks for reporting the issue! Yes, we're in the middle of removing TailwindCSS in favour of NextUI (see #124). Moving forward we'll definitely be using NextUI for styling the website (I'm tired of changing the tech stack repeatedly & adding unnecessary complexity to the project).
So, if you would like please feel free to share a PR cleaning up any traces of TailwindCSS in the project. Don't forget to make documentation changes as well?


compile: error: shifting a negative signed value.

Please answer these questions before submitting your issue. Thanks!
What version of Go are you using (go version)?
x
What operating system and processor architecture are you using (go env)?
$ uname -a
Darwin Byrons-MacBook-Pro-3.local 15.4.0 Darwin Kernel Version 15.4.0: Fri Feb 26 22:08:05 PST 2016; root:xnu-3248.40.184~3/RELEASE_X86_64 x86_64
$ sysctl -n machdep.cpu.brand_string
Intel(R) Core(TM) i5-5287U CPU @ 2.90GHz
$ sw_vers
ProductName:   	Mac OS X
ProductVersion:	10.11.4
BuildVersion:  	15E65
GOARCH="amd64"
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOROOT="/usr/local/Cellar/go/1.7.1/libexec"
GOTOOLDIR="/usr/local/Cellar/go/1.7.1/libexec/pkg/tool/darwin_amd64"
CC="clang"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/ml/2nr7wt652n9ddq8yhtxfjzw40000gp/T/go-build066728938=/tmp/go-build -gno-record-gcc-switches -fno-common"
CXX="clang++"
CGO_ENABLED="1"

What did you do?
Attempted to build Go 1.4 from source for a certain debugger tool. Following Installing Go from source guide.
$ git clone https://go.googlesource.com/go
$ cd go
$ git checkout go1.4
$ cd src
$ ./all.bash
What did you expect to see?
A successful build.
What did you see instead?
# Building C bootstrap tool.
cmd/dist

# Building compilers and Go bootstrap tool for host, darwin/amd64.
lib9
libbio
liblink
cmd/cc
cmd/gc
cmd/6l
cmd/6a
cmd/6c
/Users/byrongrogan/git/go/src/cmd/6c/txt.c:995:28: error: shifting a negative signed value is undefined [-Werror,-Wshift-negative-value]
/Users/byrongrogan/git/go/src/cmd/6c/txt.c:1045:28: error: shifting a negative signed value is undefined [-Werror,-Wshift-negative-value]
go tool dist: FAILED: clang -Wall -Wstrict-prototypes -Wextra -Wunused -Wno-sign-compare -Wno-missing-braces -Wno-parentheses -Wno-unknown-pragmas -Wno-switch -Wno-comment -Wno-missing-field-initializers -Werror -fno-common -ggdb -pipe -Wuninitialized -O2 -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -mmacosx-version-min=10.6 -c -m64 -I /Users/byrongrogan/git/go/include -I /Users/byrongrogan/git/go/src/cmd/6c -o /Users/byrongrogan/git/go/pkg/obj/cmd/6c/txt.o /Users/byrongrogan/git/go/src/cmd/6c/txt.c

Use git checkout go1.4.3, or use a binary download of Go 1.4.

Did you just install xcode8?
Edit cmd/dist and remove the -Werror flag wherever you find it. That'll get
you unblocked.
I've heard that there will be a 1.4.4 to fix the Sierra crash issue, but
there is no ETA so I'd apply this workaround.
On Thu, 22 Sep 2016, 05:41 Byron Grogan<EMAIL_ADDRESS>wrote:

Please answer these questions before submitting your issue. Thanks!
What version of Go are you using (go version)?
x
What operating system and processor architecture are you using (go env)?
$ uname -a
Darwin Byrons-MacBook-Pro-3.local 15.4.0 Darwin Kernel Version 15.4.0: Fri
Feb 26 22:08:05 PST 2016; root:xnu-3248.40.184~3/RELEASE_X86_64 x86_64
$ sysctl -n machdep.cpu.brand_string
Intel(R) Core(TM) i5-5287U CPU @ 2.90GHz
$ sw_vers
ProductName: Mac OS X
ProductVersion: 10.11.4
BuildVersion: 15E65
GOARCH="amd64"
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOROOT="/usr/local/Cellar/go/1.7.1/libexec"
GOTOOLDIR="/usr/local/Cellar/go/1.7.1/libexec/pkg/tool/darwin_amd64"
CC="clang"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/ml/2nr7wt652n9ddq8yhtxfjzw40000gp/T/go-build066728938=/tmp/go-build -gno-record-gcc-switches -fno-common"
CXX="clang++"
CGO_ENABLED="1"
What did you do?
Attempted to build Go 1.4 from source for a certain debugger tool.
Following Installing Go from source
https://golang.org/doc/install/source guide.
$ git clone https://go.googlesource.com/go
$ cd go
$ git checkout go1.4
$ cd src
$ ./all.bash
What did you expect to see?
A successful build.
What did you see instead?
Building C bootstrap tool.
cmd/dist
Building compilers and Go bootstrap tool for host, darwin/amd64.
lib9
libbio
liblink
cmd/cc
cmd/gc
cmd/6l
cmd/6a
cmd/6c
/Users/byrongrogan/git/go/src/cmd/6c/txt.c:995:28: error: shifting a negative signed value is undefined [-Werror,-Wshift-negative-value]
/Users/byrongrogan/git/go/src/cmd/6c/txt.c:1045:28: error: shifting a negative signed value is undefined [-Werror,-Wshift-negative-value]
go tool dist: FAILED: clang -Wall -Wstrict-prototypes -Wextra -Wunused -Wno-sign-compare -Wno-missing-braces -Wno-parentheses -Wno-unknown-pragmas -Wno-switch -Wno-comment -Wno-missing-field-initializers -Werror -fno-common -ggdb -pipe -Wuninitialized -O2 -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -mmacosx-version-min=10.6 -c -m64 -I /Users/byrongrogan/git/go/include -I /Users/byrongrogan/git/go/src/cmd/6c -o /Users/byrongrogan/git/go/pkg/obj/cmd/6c/txt.o /Users/byrongrogan/git/go/src/cmd/6c/txt.c
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/golang/go/issues/17183, or mute the thread
https://github.com/notifications/unsubscribe-auth/AAAcAzPDvaPyeQ0WAJkJOSVJo5_Bw5fmks5qsYhQgaJpZM4KDMsy
.

You should try the go1.4.3 tag. It removed -Werror flag.

Could we get release-branch.go1.4 updated to go1.4.3 then? I've just been using it like I do every release branch.

Oh, also, there's no go1.4.3 tag. You maybe meant go1.4.2?

Hm, on what seems to be the latest go1.4.3, I'm not able to get this to compile with clang 8.0.0.
See https://gist.github.com/jmhodges/411d8eadb01457c909d629229e6ff69a

I think you are running into #16352.


41.2.0 gives its version as 41.1.1

See std.lua. Something appears to have gone wrong with the release process.

Huh, me too.  And no one noticed for more than 2 years! :)
It looks like the release process didn't pick up the gitignored lib/std.lua file when making the tar ball for the release branch, which left the one from the previous release showing through when unrolling it for that release.  I'm in the process of eliminating slingshot and the autotooling for release automation now that the Luke reboot can handle everything lyaml and luaposix needed from autotools, so I'm not planning to try to fix it.
In the mean time, I manually copied the proper std.lua into place on the release branch, tagged it appropriately and uploaded the amended rockspec to luarocks.
AFAICT, luarocks install stdlib behaves properly now.


CCU Firmware 2.35.x

Update to new firmware 2.35.15

Released in testing apt repository

There are some serious issue with the new CCU2 firmware using HmIP shutter actuators.
eQ-3 confirmed this issues and they will release a new firmware soon.
Because of the issues 2.35.15 firmware will not be added to the stable apt repository and I will wait for the new firmware.

A new CCU2 firmware (2.35.16) was released today by eQ-3. I released an updated piVCCU version in the testing apt repo. If there will be no new major issue, I will release that version in about 1 week in the stable apt repo, too.

Released in stable apt repository


BKCurrencyTextField formats

Hi, how we can change BKCurrencyTextField formats, for ex: 44.333 like this one, or enable only integer types: 44,000 ?

You can change format with numberFormatter property in BKCurrencyTextField.
For details, please refer to NSNumberFormatter class.
Thank you.

오후 1:20 Kanybek<EMAIL_ADDRESS>작성:

Hi, how we can change BKCurrencyTextField formats, for ex: 44.333 like this one, or enable only integer types: 44,000 ?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.


Integration with fancyBox (v1.3.4)

Hi,
I'm having issues with fancyBox integration.
When I reload gallery page everything works as expected.
When I go to gallery page using turbolink the image preview modal is not working.
I'm not getting any js errors.
Here is my application.js
document.addEventListener("page:load", function(){
  init('load');
});

document.addEventListener("page:change", function(){
  init('change');
});

$(document).ready(function() {
  init('ready');
});

function init(msg) {
  $("a.fancybox").fancybox();
  alert(msg);
}

Thanks in advance,
Marek

The solutions mentioned by @iceless58 and @antpaw not working for me.
Using Fancybox2-Rails (Fancybox 2.1.5) with Ruby 2.2.0, Rails 4.2.0 and Turbolinks 2.5.3. (cc: @kyparn)

This won"t work when using it programatically with $.fancybox.open


2.4.1

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

@baev done.

@DudaGod thanks!


What is daft best practice for resampling a timeseries?

Using Daft built-in expressions, I am able to use dt.truncate on my dataframe's timestamp column and then group by the truncated timestamps. Is this considered a best practice? Or is there another approach that scales better on large dataframes that should be used instead?
For what it's worth, Pandas offer a pandas.dataframe.resample() method, which utilizes the pandas.core.resample.Resampler class under the hood. This appears to be the near-universal recommended solution for Python-based timeseries resampling. Also scipy offers scipy.signal.
resample(). Looking at the Arrow documentation, I found methods to round timestamps, similar to Daft's dt.truncate, but nothing to resample a TimestampArray.

Hi @daveqs !
Are there use-cases in which the truncate method is insufficient? Looking at the resample documentation, I think a groupby on a truncated column should get you mostly what you need. I think we're also open to adding syntactic sugar on top of these operations if there are canonical use-cases for resampling that a higher-level API might make sense for.

Is there a optimal way to aggregate on the resampling with this, like in pandas which treats the resampling as bins.
From Pandas API:
`
Upsample the series into 30 second bins.

series.resample('3min', label='right', closed='right').sum()
2000-01-01 00:00:00     0
2000-01-01 00:03:00     6
2000-01-01 00:06:00    15
2000-01-01 00:09:00    15
Freq: 3min, dtype: int64`

Hi @daveqs !
Are there use-cases in which the truncate method is insufficient? Looking at the resample documentation, I think a groupby on a truncated column should get you mostly what you need. I think we're also open to adding syntactic sugar on top of these operations if there are canonical use-cases for resampling that a higher-level API might make sense for.

Hi @jaychia thanks for your reply.
My use case involves using Daft to transform an Iceberg table containing timestamped messages consisting of a list of signals (names and values), 1 message per row, for purposes of ML training and inference performed at regularly scheduled intervals. The transformation will yield one (resampled) timestamp per row, signals as columns. The purpose of the resampling is to ensure that I get complete rows of data covering all of my signals, since each message in my source table contains only a subset of the signals.
My transformation will be performed iteratively as a batch job, and because time behaves nicely and doesn't go backwards (no tachyons in my dataset!), I don't need to worry that any data belonging in a group arriving at time == now needs to be incorporated into groups previously transformed. Also, I know my signals in advance, so I can pass them into the daft.DataFrame.pivot method using the names argument, which should avoid the need for daft to run a global distinct operation. In other words, my use case likely won't stress Daft's ability to groupby truncated timestamps and pivot, as long as my iterations are sufficiently frequent that each batch being transformed is manageable.
That said, I'd find it helpful to know how the daft.Dataframe.groupby and daft.Dataframe.pivot methods work on a DataFrame that is partitioned. Do they get applied partition by partition, or will they be applied globally? (I would like to see them applied partition by partition to enable the operation to be distributed. If they are applied globally, I may decide to use Ray Data's map_batches in order to explicitly batch the groupby and pivot.)
As far as suggestions for syntactic sugar, I don't really care whether I write the resampling transformations out explicitly or use an API that is similar to pandas for resampling. If anything, because I'm also pivoting, writing out the transformations is helpful to see what is really happening.  Many users likely know the pandas resampling syntax already, but this transformation was pretty easy to write once i settled on the truncate/groupby method. The real advantage of adopting a resample API is so that users don't need to think about what operations are happening under the hood (definitely a double edged sword).
Here is the daft code that I wrote to perform the transformation in the description of my use case is unclear, and it runs acceptably fast on initial test cases.  I haven't tried stress testing it yet.
       '''     Schema of df is:       sender_id: string       message_id: string       timestamp: timestamp[us]       signals: list<struct<signalname: string, signalvalue: double>>     '''     resample_freq = '5 seconds'     df = df.explode('signals')     df = df.with_column('signalname', df['signals'].struct.get('signalname'))     df = df.with_column('signalvalue', df['signals'].struct.get('signalvalue'))      df = df.with_column('timestamp_trunc', df['timestamp'].dt.truncate(resample_freq))      wide_df = df.pivot(         group_by=[daft.col('timestamp_trunc'), daft.col('sender_id')],          pivot_col=daft.col('signalname'),          value_col=daft.col('signalvalue'),         agg_fn="mean",         names=features     )

This looks great @daveqs! From my reading of your code, it should do exactly what you've intended.
For your other questions:

daft.Dataframe.groupby has to be paired with a subsequent .agg call. This will perform the aggregation(s) per-group, and the outputted dataframe will be repartitioned according to the group keys.
daft.Dataframe.pivot behaves similarly if provided with the group_by argument like you have here. It will partition the dataframe by your provided groups, and perform your specified aggregation over those groups (for each signalname/signalvalue)

If they are applied globally, I may decide to use Ray Data's map_batches in order to explicitly batch the groupby and pivot.)

Could you help me understand this use-case a little better? daft.Dataframe.groupby and daft.Dataframe.pivot work on the entire dataframe. However, this is already inherently a distributed operation and you do not have to do any explicit parallelization yourself. On a high level, it looks like this:

On each partition, perform a local groupby + agg operation (on the groupby keys + pivot key)
Shuffle the data across partitions and repartition the data by the groupby + pivot keys
Now on each new partition, perform a local groupby + agg operation again (on the groupby keys + pivot key)
Now we perform a local pivot operation

Could you help me understand this use-case a little better? daft.Dataframe.groupby and daft.Dataframe.pivot work on the entire dataframe. However, this is already inherently a distributed operation and you do not have to do any explicit parallelization yourself. On a high level, it looks like this:

On each partition, perform a local groupby + agg operation (grouped by [*groupby_keys, pivot_key])

Shuffle the data across partitions and repartition the data by groupby_keys

Now on each new partition, perform a local groupby + agg operation again (grouped by [*groupby_keys, pivot_key])

Now we simply perform a local pivot operation

This is exactly what I was looking to understand, thank you!
I will follow up further on Daft slack.

Happy to help!


Partner network | BPDM service is not accessible

Current Behavior

We are not able to access the BPDM service. We tried with all the possible roles and permissions but none has worked for us. We get always 403.
Expected Behavior

We should be able to access the BPDM service with an appropriate role.
Steps To Reproduce

Check the Partner Network in the portal or try to access the BPDM service APIs.

Hi @mounirol, thanks for opening the issue. As discussed in the EF Chat: I checked this, and AFAI can say right now, there might be a mismatch between the rights and roles concept and the bpdm implementation: the implementation excepts the read_partner role https://github.com/eclipse-tractusx/bpdm/blob/v6.0.0/bpdm-pool/src/main/kotlin/org/eclipse/tractusx/bpdm/pool/config/PermissionConfigProperties.kt#L28 but according the rights and role concept this role should not be assigned https://github.com/eclipse-tractusx/portal-iam/blob/v3.0.0/docs/technical documentation/06. Roles %26 Rights Concept.md#255-bpdm-pool. I can't say if the bpdm implementation or the the R&R concept is correct, and we'll need to discuss this with @jjeroch ones she's back from vacation.

Looking into this by today

read_partner from Cl7-CX-BPDM added BPDM Pool Consumer on int, dev and rc.dev
TODO: update of template instances and documentation

@evegufy The Partner network is accessible from the portal, but it is not possible to access the BPDM APIs using the Technical user. Will this be also fixed?

@mounirol the examples which you showed me are technical users with roles for the BPDM Gate  assigned, I assume you need to create a technical user with a role that provides access to the BPDM API like BPDM Pool Consumer
cc: @jjeroch

@Sebastian-Wurm  seems like the new BPDM solution is not well known to the teams. We need a user documentation for this. Please take this up.
To be answered:

Explanation by when a customer should use technical user role "xxx" and when "xxx"
Is it really planned to create this tech user from the consumer side or is BPDM operator providing the user?

Note my expectation is the following:

App Provider (such as mounirol) need to update their app offering inside the portal marketplace and configure inside the technical user profile the permission

BPDM Sharing Output Consumer

The app provider must successfully negotiate the BPDM Access data offering in the name of the customer
As part of the successful negotiation, the app provider will receive the API endpoint to retrieve BPDM data
With the technical user of #1 the app provider can now access the BPDM endpoint and retrieve member data

...please confirm.
@evegufy in the current state of the ticket, I suggest to move the ticket to BPDM - I dont see a technical change needed on our side.

@mounirol, @jjeroch:

Permission groups are explained in the Arc42 document: https://github.com/eclipse-tractusx/bpdm/blob/main/docs/arc42/arc42-bpdm.md#permissions-by-permission-group
To access BPDM services, an EDC connection is required as described in the Arc42 document: https://github.com/eclipse-tractusx/bpdm/blob/main/docs/arc42/arc42-bpdm.md#edc-communication
So, technical users must be created by the BPDM operator. They are only used by the EDC data plane and must not be known outside of the operating environment
Consequently, app providers must not configure any technical users for accessing the BPDM services, but need to use an EDC connection on behalf of their customer / with the BPNL of their customer

@evegufy, @jjeroch:

read_partner permission is not required to get business partners from the BPDM Pool for the Portal Partner Network
read_partner_member of permission group "Cx Member" (which corresponds to CX User Portal role) must be sufficient, as only Catena-X members should be displayed by the Portal Partner Network (not the whole BPDM Pool).
Either there is a bug in the implementation of read_partner_member / isCatenaXMemberData or there are no business partners flagged with isCatenaXMemberData, if the Partner Network in the Portal shows no business partners
@nicoprow: Can you please confirm.

As far as I can see the problem is that the Portal invokes the endpoint "pool/v6/legal-entities" and receives (rightfully) a 403 since this is the endpoint to query all legal entities. Since version 6 of the API we introduced the members endpoint which returns only the Catena-X business partner member data: pool/v6/members/legal-entities/search
So from the BPDM side this behaviour is expected, but the Portal would need to integrate the members endpoint instead.

@evegufy, @jjeroch: also R&R needs to be adapted to include read_partner_member for all roles that refer to BPDM permission group "Cx Member".

@Sebastian-Wurm @nicoprow If you see a need for adjustments, I suggest you open an issue in the sig-release repo with a title like "BPDM | Consolidate Rights and Roles" and mark it with the bdpm portal and prep24.12 labels, so we can refine it for the next release, the following should be made transparent on sig-release level as well https://github.com/eclipse-tractusx/portal-iam/issues/154#issuecomment-2244464821
Changes aren't possible anymore for 24.08 as Testing is ending.
We already did the E2E Tests for 24.08 with read_partner from Cl7-CX-BPDM added to BPDM Pool Consumer.
As I didn't close the 3.0.1 of portal-iam yet, I could remove the role again, so that there appears a 403 error again, should I do that?

@evegufy: Best would be use the other endpoint, as @nicoprow described above. Either we do that in the reference implementation of the Portal or the Operating Company has to do that in their implementation of the Portal, if they want to provide the correct Partner Network functionality for Jupiter release.  The fix with read_partner is definitely wrong, so it makes sense to remove it. Wouldn't this change in portal-iam also make a general retest necessary as IAM is a cross-cutting aspect?
@maximilianong, @jjeroch any recommendations from your side?

@mounirol, @jjeroch:

Permission groups are explained in the Arc42 document: https://github.com/eclipse-tractusx/bpdm/blob/main/docs/arc42/arc42-bpdm.md#permissions-by-permission-group
To access BPDM services, an EDC connection is required as described in the Arc42 document: https://github.com/eclipse-tractusx/bpdm/blob/main/docs/arc42/arc42-bpdm.md#edc-communication
So, technical users must be created by the BPDM operator. They are only used by the EDC data plane and must not be known outside of the operating environment
Consequently, app providers must not configure any technical users for accessing the BPDM services, but need to use an EDC connection on behalf of their customer / with the BPNL of their customer

The provided url do not seem to work for me , could you please verify @Sebastian-Wurm

@MaximilianHauer the arc42 just moved folders to be compatible to the TRGs. I corrected the URLs above

Hi all,
we aligned to remove the bugfix from this release.
We will now align on how to proceed and if we can fix it before 24.12.

https://github.com/eclipse-tractusx/bpdm/blob/main/docs/architecture/08_Crosscutting_Concepts.md#edc-communication

The documentation does not fit to the tested version of 24.05. and 24.08. release. This should have been documented inside the BPDM changeLog @nicoprow as we spoke in the last release, the Orchestrator havn't been part of the release we should not document a setup in future which does not exists. In this case either the documentation must get removed or should well mention that this is just a proposal while its not yet tested or implemented.
@nicoprow & @evegufy please add this in the 24.08. release documentation of BPDM.
Any further steps to be managed with the release 24.12. as mentioned by @MaximilianHauer


Will add unicode support?

This is a great and easy use library,and will you add unicode to it?

Can you be more specific what unicode support means?

Some sqlite3 functions like 'sqlite3_open16' and so on

Let me wait until more requests come for UTF-16. BTW, are you using UTF-16 for your projects? Just for curious.

yes, our projects are UTF-16, feels bad.


Feature/limit executor passthrough

Pull Request Guidelines
Pull requests are an essential collaborative tool for modern software development.
The below list is intended to help you figure out whether your code is ready to be reviewed
and merged into ArangoDB. The overarching goal is to:

Reduce the amount of recurring defects
Reduce the impact to the other developer’s time and energy spent on defects in your code
Increase the overall autonomy and productivity of individual developers

Acceptance Checklist
The below list is not exhaustive, think thoroughly whether the provided information is sufficient.
Remove options that do not apply
Scope & Purpose
(Can you describe what functional change your PR is trying to effect?)

[ ] Bug-Fix for devel-branch (i.e. no need for backports?)
[ ] Bug-Fix for a released version (did you remember to port this to all relevant release branches?)
[ ] Strictly new functionality (i.e. a new feature / new option, no need for porting)
[ ] The behavior in this PR can be (and was) manually tested (support / qa / customers can test it)
[ ] The behaviour change can only be verified via automatic tests

Related Information
(Please reference tickets / specification etc )

[ ] There is a GitHub Issue reported by a Community User:
[ ] There is an internal planning ticket:
[ ] There is a JIRA Ticket number (In case a customer was affected / involved):
[ ] There is a design document:

Testing & Verification
(Please pick either of the following options)
This change is a trivial rework / code cleanup without any test coverage.
(or)
This change is already covered by existing tests, such as (please describe tests).
(or)
This PR adds tests that were used to verify all changes:

[ ] Added Regression Tests (Only for bug-fixes)
[ ] Added new C++ Unit Tests (Either GoogleTest or Catch-Test)

Did you add tests for a new RestHandler subclass ?
Did you add new mocks of underlying code layers to be able to verify your functionality ?
...

[ ] Added new integration tests (i.e. in shell_server / shell_server_aql)
[ ] Added new resilience tests (only if the feature is impacted by failovers)

Additionally:

[ ] There are tests in an external testing repository (i.e. node-resilience tests, chaos tests)
[ ] I ensured this code runs with ASan / TSan or other static verification tools

(Include link to Jenkins run etc)

Think about whether the new code you added is modular enough to be
easily testable by unit tests written with GTest / Catch. It is not good if your feature is so interconnected
that it prevents other people from writing their own unit gests. It should be possible
to use your code in future without extensively mocking your classes.
A bad example that required some extensive effort would be the storage engine API.

Documentation

All new Features should be accompanied by corresponding documentation.
Bugs and features should furthermore be documented in the changelog so that
developers and users have a concise overview.

[ ] Added a Changelog Entry
[ ] Added entry to Release Notes
[ ] Added a new section in the Manual
[ ] Added a new section in the http API
[ ] Added Swagger examples for the http API

CLA Note
Please note that for legal reasons we require you to sign the Contributor Agreement
before we can accept your pull requests.

http://jenkins01.arangodb.biz:8080/view/PR/job/arangodb-matrix-pr/4829/

http://jenkins01.arangodb.biz:8080/view/PR/job/arangodb-matrix-pr/4831/

http://jenkins01.arangodb.biz:8080/view/PR/job/arangodb-matrix-pr/4837/

http://jenkins01.arangodb.biz:8080/view/PR/job/arangodb-matrix-pr/4843/

Last CI run was green, except for windows compilation. Hopefully this was fixed with the last commit:
http://jenkins01.arangodb.biz:8080/view/Windows/job/arangodb-matrix-pr-windows-onlycompile/2092/

http://jenkins01.arangodb.biz:8080/view/PR/job/arangodb-matrix-pr/4893/

http://jenkins01.arangodb.biz:8080/view/PR/job/arangodb-matrix-pr/4907/


Failure to convert XYZ to RGB and back

Hi, first thanks for this great library!
I have an issue converting an XYZ value to RGB and back, it seems that for some values this works, but for others not. I could not find any range for the XYZ values in the readme (as for the other values), maybe it is related to that?.
This conversion matches (after rounding):
const startXYZ = [47.854955042890886, 29.320696241557958, 21.87902240817543]
const _rgb = convert.xyz.rgb(startXYZ);
const endXYZ = convert.rgb.xyz(_rgb) // => [ 48, 29, 22 ]

However, the following conversion is a mismatch (and quite some other value):
const startXYZ = [ 20, 87, 79 ]
const _rgb = convert.xyz.rgb(startXYZ);
const endXYZ = convert.rgb.xyz(_rgb) // => [ 48, 76, 75 ]

I have been fiddling with the color-space library as well, but that seems to have the same result (https://github.com/colorjs/color-space/issues/48). Is this a bug in the conversion algorithm?

XYZ and RGB do not have direct conversations; I'm not well versed in XYZ theory but if I recall correctly, it's an approximation and won't give you stable results when converting between the two.

Closing as this conversion is lossy in nature (not all color models can be converted cleanly between one another).


Let's Talk: Security and JSR223

JSR223 is the best tool we have for running scripts for things like Cotton Tweaker. However, its sandboxing capability is somewhat limited.  However, that still doens't prevent users from being profiled by data packs run in single-player worlds. Ultimately, the reliability of Cotton tweakers almost more than that of mods, since you can actually see what's in a script without decompiling it. However, that doesn't mean we should leave the floodgates open for whatever people want. Should we add a ClassShutter to Tweaker-run scripts, and if so, what rules should we have so that the system keeps its extensibility?

For an idea of shutter rules, does blocking out all classes except ones in the io.github.cottonmc.cotton.tweakers package and/or with the word Tweaker in the class name sound good?

Sandboxing for JSR223 - and java in general, without explicitly running a hand-rolled interpreter - is unlikely to be effective.

ah yeah, it looks like the ClassShutter from Rhino didn't make it into Nashorn/JSR223.

JSR223 has a class filter

It doesn't look like the ClassFilter is accessible to ScriptEngineManager or ScriptEngine being called. I can't find any methods in any of the related ScriptEngine code, and the Oracle docs on ClassFilter say it's specific to Nashorn, when the whole point of the tweaker system is that you can use any JSR223. The closest thing I can get access to is passing a custom ClassLoader, but I am not writing a custom class loader just for this.

I don't think this is the solution. If some script starts profiling the player in singleplayer, that's the player's concern, and he can choose to remove the script, or decide it's not a big cause to him.
By making that choice, it removes from the player.

I'm going to go ahead and close this one; we're unlikely to have any good answer to this. It's abritrary code execution. The whole surface is attack surface.


Performance improvements to Map and City Views

We noticed very slow map and city view performance when a user switches tabs.
This PR does the following.

Create separate supercluster instance for each filter type
Ensures map clustering data gets computed only initial or when show data is mutated
Cap tab list data to a fixed number (set to 25) which was the heaviest slowdown

Might wanna rebase these commits a bit :D

Related: https://artsyproduct.atlassian.net/browse/LD-338


Adding option to manage the handlers dir

Fixes #430

Can you put some tests around this to make sure the parameter works as expected and I'll merge it?

tests added @jlambert121

Thanks!


Queue.fromArn breaks the queueUrl resolution

When running localstack with docker, my SQS Queue's are configured so that their URL's are meant to look like:
$ awslocal sqs create-queue --queue-name test
{
    "QueueUrl": "http://<IP_ADDRESS>:4566/000000000000/test"
}

When I pass SQS queue's between stacks, I need to pass in the queue's arn and then fetch the queue with Queue.fromArn. When you pull a queue from Arn the queueUrl has a resolve that makes it look much different (https://sqs.${Token[TOKEN.207]}.${Token[AWS.URLSuffix.5]}/${Token[TOKEN.208]}/${Token[TOKEN.209]}). This leads to not being able to use the URL's from SQS Queues that are passed between stacks.
version: "3.5"
services:
  localstack:
    container_name: "localstack"
    image: localstack/localstack:latest
    network_mode: bridge
    ports:
      - "433:433"
      - "4566:4566"
      - "4567:4567"
      - "4571:4571"
      - "9001:9001"
    environment:
      - SERVICES=sqs, lambda, apigateway, s3, sts, ssm, sns, ses, cloudformation, iam, ec2, kms, cloudfront
      - LOCALSTACK_HOSTNAME=<IP_ADDRESS>
      - HOSTNAME=<IP_ADDRESS>
      - HOSTNAME_EXTERNAL=<IP_ADDRESS>
      - LAMBDA_EXECUTOR=docker
      - LAMBDA_REMOTE_DOCKER=true
      - DOCKER_HOST=unix:///var/run/docker.sock
      - DEFAULT_REGION=eu-west-2
      - TEST_AWS_ACCOUNT_ID=000000000000
      - DEBUG=1
    volumes:
      - "${TMPDIR:-/tmp/localstack}:/tmp/localstack"
      - "/var/run/docker.sock:/var/run/docker.sock"

import { App, Stack } from "aws-cdk-lib";
import { Construct } from "constructs";
import { Queue } from "aws-cdk-lib/aws-sqs";
import { Code, Function, Runtime } from "aws-cdk-lib/aws-lambda";

class TestStack extends Stack {
  constructor(scope: Construct, id: string) {
    super(scope, id);

    const producer = new ProducerConstruct(this, "producer");
    new ConsumerConstruct(this, "consumer", producer.queue.queueArn);
  }
}

class ProducerConstruct extends Construct {
  public readonly queue: Queue;

  constructor(scope: Construct, id: string) {
    super(scope, id);

    this.queue = new Queue(this, "Queue");
  }
}

class ConsumerConstruct extends Construct {
  constructor(scope: Construct, id: string, queueArn: string) {
    super(scope, id);

    const queue = Queue.fromQueueArn(this, "queue", queueArn);

    new Function(this, "lambda", {
      runtime: Runtime.NODEJS_14_X,
      handler: "index.handler",
      code: Code.fromAsset(""),
      environment: {
        SQS_QUEUE_URL: queue.queueUrl,
      },
    });
  }
}

const app = new App();

new TestStack(app, "stack");

From my understanding this comes from Queue.fromQueueAttributes where it defaults the queueUrl to https://sqs.${parsedArn.region}.${stack.urlSuffix}/${parsedArn.account}/${queueName} if you don't pass one in. I would assume this would need patching to match the SQS Queue Url Strategy that you set up in localstack

Hi! We just wanted to follow up to see whether your issue has been resolved. Were you able to get it working with the latest version of LocalStack? We would appreciate your feedback!

Closing this issue due to inactivity.


[AS-3677] Adding action type

The type of this PR is: TYPE

This PR resolves [CO-]
Description
Adding action type to the new events to be tracked on CMS conversations.
Relates to: https://github.com/artsy/cohesion/pull/398

PR Checklist (tick all before merging)

[ ] If I've added a new file to the tree I've exported it from the common index.ts
[ ] I've added comments with examples for any new interfaces and ensured that they're in the docs
[ ] No platform-specific terminology has been used outside of click and tap (platform is inferred by the DB storing events)

:rocket: PR was released in v4.102.0 :rocket:


Misleading async description in blog Start with Python and InfluxDB

The blog post Start with Python and InfluxDB contains a misleading typo(?) in the Write data section:

InfluxDB supports Asynchronous and Synchronous Writes, and you can specify the write type as required.

Fix:
InfluxDBClient supports asynchronous and synchronous writes, and you can specify the write type as required.
For more information about asynchronous writes, see [How to use Asyncio in `influxdb-client`](https://github.com/influxdata/influxdb-client-python#how-to-use-asyncio)

@jstirnaman We can probably transfer this issue to the influxdata/influxdata.com repo.


Remove @metamask/detect-provider

rewrite with https://github.com/MetaMask/detect-provider/blob/main/src/index.ts

It can be retained.


Timeout configure not work

Hi,
I configured the timeout as described in the documentation, but it didn't work.
Environment:

sdk
yarp version

.NET 8
2.2.0-preview.1.24266.1

My configuration and code are as follows:
{
  "ReverseProxy": {
    "Routes": {
      "test_header_route": {
        "ClusterId": "test_cluster",
        "Timeout": "00:00:01",
        // "TimeoutPolicy": "basicPolicy",
        "Match": {
          "Hosts": [
            "*"
          ],
          "Headers": [
            {
              "Name": "X-Custome-Id",
              "Values": [
                "666"
              ],
              "Mode": "ExactHeader"
            }
          ]
        }
      }
    },
    "Clusters": {
      "test_cluster": {
        "Destinations": {
          "node1": {
            "Address": "https://www.google.com",
            "Methods": [
              "POST",
              "GET"
            ]
          }
        }
      }
    }
  }
}

var builder = WebApplication.CreateBuilder(args);

builder.Services
	// .AddRequestTimeouts(opt => opt.DefaultPolicy = new Microsoft.AspNetCore.Http.Timeouts.RequestTimeoutPolicy { Timeout = TimeSpan.FromSeconds(1) }) work
        //.AddRequestTimeouts(opt => opt.AddPolicy("basicPolicy", TimeSpan.FromMilliseconds(1000))) not work
	.AddRequestTimeouts() // not work
	.AddReverseProxy()
	.LoadFromConfig(builder.Configuration.GetSection("ReverseProxy"));

var app = builder.Build();
app.UseRequestTimeouts();
app.MapReverseProxy();
await app.RunAsync();

The Timeout configuration is not working. I sent a request, and it takes about 15s to return 504 Gateway Timeout.

I can't reproduce that.
Note that request timeouts do not apply when you have a debugger attached.
Are you seeing the same behavior without one?

Hi,
I changed Timeout from 00:00:01 to 00:00:1 and it works, strange.
But when the request timed out, it returned a 400 instead of a 504.

I changed Timeout from 00:00:01 to 00:00:1 and it works, strange.

Both are parsed to the same value when reading the configuration file and as such behave exactly the same way. Are you sure you did not make any other changes?
Are you able to create a minimal runnable repro for the issue?

But when the request timed out, it returned a 400 instead of a 504.

When a timeout is hit, you're effectively terminating the request from the client as far as YARP is concerned.
If you were seeing that in a browser, it's possible that you're seeing a manufactured error the browser came up with when the request failed.
Since the timeout can occur at any point during a request/response, it's also possible that YARP already responded with a status code, but the request was canceled while we were copying data back to the client. At that point, the status code has already been sent, so it can't change.

Triage: Closing as answered. Let us know if we missed anything.

Hello,
Confirming the same behavior on the 2.2.0 release, but let me add more details.

But when the request timed out, it returned a 400 instead of a 504.

Trying to configure timeouts with ForwarderRequestConfig.ActivityTimeout on a cluster and Timeout/TimeoutPolicy on a route, but results are different: the first configuration returns 504 Gateway Timeout, second - returns 400 Bad Request. Shouldn't the results be the same?
@MihaZupan, created a repository, please have a look.

Are you able to create a minimal runnable repro for the issue?

Hello,
Confirming the same behavior on the 2.2.0 release, but let me add more details.

But when the request timed out, it returned a 400 instead of a 504.

Trying to configure timeouts with ForwarderRequestConfig.ActivityTimeout on a cluster and Timeout/TimeoutPolicy on a route, but results are different: the first configuration returns 504 Gateway Timeout, second - 400 Bad Request. Shouldn't the results be the same?
@MihaZupan, created a repository, please have a look.

Are you able to create a minimal runnable repro for the issue?

See #2662


MqttSenders and MqttListeners do not share a common MQTT client resulting in disconnect and reconnect loops when using the same clientId

❗ please do not add sensitive information in issues, you can provide extra information via email using issue number as reference ❗
Describe the issue
With MQTT a clientId is supposed to represent a device or client application and should only connect to the broker with 1 clientId. Currently a new MQTT (Java)client is being instantiated for each instance of MqttSender and MqttListener. This will result in the broker disconnecting the first MQTT client because a connection with the same clientId is already connected. Then the framework sees that the connection is lost and tries to reconnect to the broker, repeating this back-and-forth disconnect and reconnect in a endless loop.
Currently the developer is limited to only using a single MqttSender or MqttListener, or needs to use a unique clientId for each one of them. The latter is unpractical in any production setup because ACL's and credentials need to be configured on the broker for each clientId.
Reporter
M. Lenterman

To Reproduce
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error

Screenshots
If applicable, add screenshots to help explain your problem.
Configuration
Please provide the configuration of the Pipe or Receiver with the problem.

Input
Please provide an example of the input message.
Alternatively, provide a Ladybug report.

Environment
FF! 8.1.0-RC2-20240411.160355: Openforms2bpel 1.16.2
Running on ae6a4a7c57c9 using Apache Tomcat/9.0.87
Java Version: OpenJDK Runtime Environment (17.0.10+7)
Heap size: 388M, total JVM memory: 592M
Free disk space: 194GB, total disk space: 251GB
Up since: 2024-04-11 18:07:56 (17h)

Additional Environment

DBMS: [e.g. Oracle, MSSQL, MariaDB, MySql, PostgreSQL]
Browser: [e.g. Chrome, Safari, Edge, Firefox]

Additional Context
Add any other context about the problem here. (f.e. ladybug report / test adapter with larva test)

It is correct that each connection needs to have a unique clientId, but that doesn't mean each sender and/or listener needs to have a unique clientId. With MQTT a client is more or less a user/device/application. A client can subscribe to many topics and can publish messages to multiple topics.


feature request: expand 'other' in category widget

Currently it's hard to know the contents of the "Other" category in the Category widgets. It would be more useful if a user could click on Other to get a drop down, where a user could check the less common items to use in the filtering of this widget. The search option requires the user to have some knowledge of the possible categories, which can be an unfair expectation on usability.
cc @xavijam

Moving to deep-insights, where widgets are generated 💋 .

Issue moved to CartoDB/deep-insights.js #458 via ZenHub


Upgrade NuGet packages

Upgrade existing NuGet packages to latest versions within given .NET Core major version

@microsoft-github-policy-service agree


Chroniques Oubliees: version 4.04

Changes / Comments
Split capacities into title + body, body can be unfolded
Update is performed through sheetworker.
Roll20 Requests
Comments are very helpful for reviewing the code changes. Please answer the relevant questions below in your comment.

[x] Does the pull request title have the sheet name(s)? Include each sheet name.
[ ] Is this a bug fix?
[x] Does this add functional enhancements (new features or extending existing features) ?
[ ] Does this add or change functional aesthetics (such as layout or color scheme) ?
[x] If changing or removing attributes, what steps have you taken, if any, to preserve player data ?
[ ] If this is a new sheet, did you follow Building Character Sheets standards ?

If you do not know English. Please leave a comment in your native language.

Character Sheet Info Roll20 Internal Use only.


Exclude a Folder from Get-SFTPChildItem command

Hi, I am fetching a folder from remote but I wanted to exclude one nested folder from it and in my case, it is backup folder. Also, backup folder has files in it that I also want to exclude. What is the good approach for this ?
$GetDeployedFiles = Get-SFTPChildItem -SessionId $Session.SessionId -Path $remotePath -Recursive
foreach ($File in $GetDeployedFiles) {
    Write-Host $File.FullName
}

 | Where-Object {$_.FullName -match '/backup\b' }

Thanks, it worked.
One more thing, how can I create these files that I fetched from remote ?

Upload with set-sftpitemSent from my iPhoneOn Sep 20, 2022, at 8:37 AM, Gurkiran Singh @.***> wrote:
Thanks, it worked.
One more thing, how can I create these files that I fetched from remote ?
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you are subscribed to this thread.Message ID: @.***>

I am sorry, but I meant to create those files locally that I fetched from server using Get-SFTPChildItem. Can I do that using Set-SFTPItem ?

I was able to resolve this using Get-SFTPItem command
  $excludedDirectories = "backup|transfer"
  $GetDeployedFiles = Get-SFTPChildItem -SessionId $Session.SessionId -Path $remotePath | Where-Object {$_.FullName -notmatch $excludedDirectories }

  foreach ($File in $GetDeployedFiles) {
    Get-SFTPItem -SessionId $Session.SessionId -Path "$($remotePath)\$($File.Name)" -Destination "$($path)\backup" -Verbose -Force
  }


Move to Swift Tools 4.2

This is the last warning I get with danger swift now 👍
Running swift test:

And @jagreenwood too of course 😄


Getting currentRoute inside of Tab Route returns parent Route

Example:
<Router hideNavBar={true}>

            <Schema name="default" sceneConfig={Navigator.SceneConfigs.FloatFromRight} />
            <Schema name="tab" type="switch" icon={TabIcon} hideNavBar={true} />
            <Schema name="modal" sceneConfig={Navigator.SceneConfigs.FloatFromBottomAndroid} />

            <Route name="login" component={LoginWrapper} title="Login" initial={true} />
            <Route name="content">
                <Router footer={TabBar} hideNavBar={true}>
                    <Route name="stream" title="stream" component={StreamNavigator} schema="tab" initial={true}/>
                    <Route name="popular" title="popular" component={PopularContainer} schema="tab"/>
                    <Route name="notifications" title="notifications" component={NotificationsContainer} schema="tab" />
                    <Route name="user" title="user" component={UserContainer} schema="tab" />
                </Router>
            </Route>
        </Router>

When I navigate to "stream" via the TabBar, and then call Actions.currentRouter.currentRoute it returns content instead of stream
Any ideas?

fixing this would probably also solve #132

maybe #110 as well

Yes, seems the same issue. Could someone look at it? I have quite limited time nowadays...

Please check with latest version 2.1.3, there was error within TabBar component...


new: Added tests for theme formatting and logging level

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (34d1627) 92.35% compared to head (b911ad0) 94.47%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #13      +/-   ##
==========================================
+ Coverage   92.35%   94.47%   +2.11%
==========================================
  Files          22       22
  Lines        1557     1574      +17
==========================================
+ Hits         1438     1487      +49
+ Misses         99       74      -25
+ Partials       20       13       -7

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


document new publishRate trigger

Corresponding code change: https://github.com/kedacore/keda/pull/1653

Just a thought: Do we use this approach, or do we introduce a mode which defines how it should work?
Thoughts on this @zroubalik?

I had talked with @coderanger on Slack about this.
queueLength needs to be kept to prevent a breaking change from existing ScaledObject definitions. That leaves the following options:

Add a mode-like field to make queueLength really mean publish rate (that seems confusing)
Add a generic triggerValue field and a triggerType field (with queue length and publish rate possible values). This would give two ways to set a queue length trigger (queueLength: X or triggerValue: X, triggerType: QUEUE_LENGTH) which is not ideal but prevents breaking changes. Publish rate would have only one option for config (triggerValue: X, triggerType: PUBLISH_RATE)
Separate queueLength and publishRate fields that are mutually exclusive.

If this were a completely new scaler then only having the triggerValue and triggerType fields would probably be the way to go, but we've already got existing uses with the queueLength field configured.
I'm happy to align with another direction if that would be more consistent with how other scalers are set up. The one thing I don't want to do is introduce a change that would break existing users.

I was just thinking of this so that it's clear what you want to use and simplify the validation of the config without relying on assumptions. I don't like assumptions as it is not always straightforward if you are new to it or not aware of how it works without reading the code/docs carefully.
  metadata:
    mode: QueueDepth
    queueLength: '20' # Optional. Queue length target for HPA. Default: 20 messages
    # Standard config

  metadata:
    mode: MessageRate
    publishRate: '100' # Optional. Publish/sec. target on the queue for HPA. Requires http host/protocol
    # Standard config

That way, you could also use these configs:
  metadata:
    mode: QueueDepth
    # Standard config

  metadata:
    mode: MessageRate
    # Standard config

This allows people to rely on the standard value, but still, define the trigger behavior.
When using both fields, this can be tricky if you are new (and maybe didn't read the docs correctly); but if you use a flag then it is explicit. The only caveat would be, though, that if mode is not provided that it defaults to queue depth.
That said, this was just my idea; I'm happy to go with the current if everyone prefers that.

mode is just an idea, can be triggerMode as well

Using both fields is impossible, it's disjoint mode. And I'm not sure any default value for publishRate makes sense so I think this is overly verbose.

What if we kept queueLength but marked it obsolete until the next major version when it would be removed. Write out a log/event (that nobody will probably read...) to update to using the mode version.
This would leave three usage patterns:
queueLength (obsolete but supported until the next major version)
 metadata:
    queueLength: "10"

mode: QueueDepth (preferred over queueLength)
 metadata:
    mode: "QueueDepth"
    value: "10" # this could still default to 20 when omitted, if desired

mode: MessageRate
 metadata:
    mode: "MessageRate"
    value: "100" # required with `mode: "MessageRate"`

Or we could just remove queueLength from the documentation and only surface the mode: variants, but still accept the queueLength config forever as an undocumented back compat. config?

I like the deprecation suggestion!

Or we could just remove queueLength from the documentation and only surface the mode: variants, but still accept the queueLength config forever as an undocumented back compat. config?

-1 on undocumented config. Folks may have a running Keda and be trying to understand it, and having undocumented legacy flags makes that much harder. But a clear "legacy, deprecated flags" section is :+1:

@coderanger I've updated to include the deprecated queueLength in the docs but call it out as deprecated and how to migrate. It is still fully supported with the same behavior, but it is mutually exclusive to the mode/value config.

Thanks for this nice change btw, @rwkarg!