update to 0.15.1

sparse 0.15.1
Destination channel: Snowflake
Links

PKG-3507
Upstream repository
Upstream changelog/diff
Relevant dependency PRs:

setuptools_scm 8.0.4

Linter check found the following problems:
ERROR conda.cli.main_run:execute(124): `conda run conda-lint /tmp/abs_cb2k5zw80m/clone` failed. (See above for error)
The following problems have been found:
===== ERRORS =====

clone/recipe/meta.yaml:1: missing_python: For pypi packages, python should be present in the host and run sections. Missing in {}
clone/recipe/meta.yaml:1: missing_python_build_tool: Python packages require a python build tool such as setuptools.
clone/recipe/meta.yaml:1: missing_wheel: For pypi packages, wheel should be present in the host section
===== Final Report: =====
3 Errors and 0 Warnings were found

Linter check found the following problems:
ERROR conda.cli.main_run:execute(124): `conda run conda-lint /tmp/abs_845bivrwiq/clone` failed. (See above for error)
The following problems have been found:
===== ERRORS =====

clone/recipe/meta.yaml:1: missing_wheel: For pypi packages, wheel should be present in the host section
clone/recipe/meta.yaml:1: missing_python_build_tool: Python packages require a python build tool such as setuptools.
clone/recipe/meta.yaml:1: missing_python: For pypi packages, python should be present in the host and run sections. Missing in {}
===== Final Report: =====
3 Errors and 0 Warnings were found

Linter check found the following problems:
ERROR conda.cli.main_run:execute(124): `conda run conda-lint /tmp/abs_31c2hqwrlp/clone` failed. (See above for error)
The following problems have been found:
===== ERRORS =====

clone/recipe/meta.yaml:1: missing_wheel: For pypi packages, wheel should be present in the host section
clone/recipe/meta.yaml:1: missing_python_build_tool: Python packages require a python build tool such as setuptools.
clone/recipe/meta.yaml:1: missing_python: For pypi packages, python should be present in the host and run sections. Missing in {}
===== Final Report: =====
3 Errors and 0 Warnings were found


JaegerTracerCustomizer is deprecated

JaegerTracerCustomizer is mentioned in the README but is only a part of the previous and now deprecated version of this repository (https://github.com/snowdrop/opentracing-tracer/blob/master/opentracing-tracer-autoconfigure/src/main/java/me/snowdrop/opentracing/tracer/JaegerTracerCustomizer.java).
Maybe it should be removed from the README and be replaced by something else.

Thanks for pointing this out!
I'll check it in tomorrow


Upload 1.0.1 with setGsmState method

Would be great to use the setGsmState method but can't using Bintray and cloning and integrating the source into my project is a bit hard.

We're actively working on this. We're trying to move the Test Butler bintray repo from my personal account to the new LinkedIn account and keep having to go back and forth with bintray's support team. Hopefully this will be resolved soon and then we'll publish the new version.

looks like this is done:
https://github.com/linkedin/test-butler/pull/26

Yep!

Thanks for notifying 👍


update to new-compilers

Checklist

[ ] Used a fork of the feedstock to propose changes
[ ] Bumped the build number (if the version is unchanged)
[ ] Reset the build number to 0 (if the version changed)
[ ] Re-rendered with the latest conda-smithy
[ ] Ensured the license file is being packaged.

[ ] wait for 0.18 release
[ ] maybe wait for vtk8.2.0

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

@conda-forge-admin, please rerender

@conda-forge-admin, please rerender

I don't think this is the solution but I'll try it anyway:
@conda-forge-admin, please update circle

Hi! This is the friendly automated conda-forge-webservice.
I just wanted to let you know that I updated the circle-ci deploy key and followed the project.

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

thanks @chrisburr for having a look at this
not triggered builds for circle-ci is one problem, but with azure there are reported more problems:
win: LINK : warning LNK4044: unrecognized option '/lpthreads'; ignored -> no idea, help needed
linux: OPENGL_gl_LIBRARY not found -> need additional yum requirements
osx: patch not applyable -> osx.patch can be removed

@conda-forge-admin, please rerender

@conda-forge-admin, please rerender

Hi! This is the friendly automated conda-forge-webservice.
I tried to re-render for you, but it looks like there was nothing to do.

try to reopen the PR


Second (And only second!) photo after first run always fails. [Warning: Unable to load image stream.]

We have exactly the same issue with an app we are working on, so I figured lets see how others have got around it!
I just pull the repo and run it, click smile, and second click fails with log below:
[0:] Microsoft.Maui.StreamImageSourceService: Warning: Unable to load image stream.
Android.Util.AndroidRuntimeException: Only the original thread that created a view hierarchy can touch its views.
at Java.Interop.JniEnvironment.StaticMethods.CallStaticVoidMethod(JniObjectReference type, JniMethodInfo method, JniArgumentValue* args) in /Users/runner/work/1/s/xamarin-android/external/Java.Interop/src/Java.Interop/obj/Release/net7.0/JniEnvironment.g.cs:line 13250
at Java.Interop.JniPeerMembers.JniStaticMethods.InvokeVoidMethod(String encodedMember, JniArgumentValue* parameters) in /Users/runner/work/1/s/xamarin-android/external/Java.Interop/src/Java.Interop/Java.Interop/JniPeerMembers.JniStaticMethods.cs:line 97
at Microsoft.Maui.PlatformInterop.LoadImageFromStream(ImageView imageView, Stream inputStream, IImageLoaderCallback callback) in D:\a_work\1\s\src\Core\src\obj\Release
et7.0-android\generated\src\Microsoft.Maui.PlatformInterop.cs:line 443
at Microsoft.Maui.StreamImageSourceService.LoadDrawableAsync(IImageSource imageSource, ImageView imageView, CancellationToken cancellationToken) in D:\a_work\1\s\src\Core\src\ImageSources\StreamImageSourceService\StreamImageSourceService.Android.cs:line 28
--- End of managed Android.Util.AndroidRuntimeException stack trace ---
android.view.ViewRootImpl$CalledFromWrongThreadException: Only the original thread that created a view hierarchy can touch its views.
at android.view.ViewRootImpl.checkThread(ViewRootImpl.java:8798)
at android.view.ViewRootImpl.requestLayout(ViewRootImpl.java:1606)
at android.view.View.requestLayout(View.java:25390)
at android.view.View.requestLayout(View.java:25390)
at android.view.View.requestLayout(View.java:25390)
at android.view.View.requestLayout(View.java:25390)
at android.view.View.requestLayout(View.java:25390)
at android.view.View.requestLayout(View.java:25390)
at android.view.View.requestLayout(View.java:25390)
at androidx.drawerlayout.widget.DrawerLayout.requestLayout(DrawerLayout.java:1353)
at android.view.View.requestLayout(View.java:25390)
at android.view.View.requestLayout(View.java:25390)
at android.view.View.requestLayout(View.java:25390)
at android.view.View.requestLayout(View.java:25390)
at android.view.View.requestLayout(View.java:25390)
at android.view.View.requestLayout(View.java:25390)
at androidx.recyclerview.widget.RecyclerView.requestLayout(RecyclerView.java:4586)
at android.view.View.requestLayout(View.java:25390)
at android.view.View.requestLayout(View.java:25390)
at android.view.View.requestLayout(View.java:25390)
at android.view.View.requestLayout(View.java:25390)
at androidx.core.widget.NestedScrollView.requestLayout(NestedScrollView.java:2075)
at android.view.View.requestLayout(View.java:25390)
at android.view.View.requestLayout(View.java:25390)
at android.view.View.requestLayout(View.java:25390)
at android.widget.ImageView.setImageDrawable(ImageView.java:597)
at androidx.appcompat.widget.AppCompatImageView.setImageDrawable(AppCompatImageView.java:112)
at com.microsoft.maui.glide.MauiCustomViewTarget.onResourceCleared(MauiCustomViewTarget.java:30)
at com.bumptech.glide.request.target.CustomViewTarget.onLoadCleared(CustomViewTarget.java:210)
at com.bumptech.glide.request.SingleRequest.clear(SingleRequest.java:337)
at com.bumptech.glide.request.ErrorRequestCoordinator.clear(ErrorRequestCoordinator.java:48)
at com.bumptech.glide.manager.RequestTracker.clearAndRemove(RequestTracker.java:70)
at com.bumptech.glide.RequestManager.untrack(RequestManager.java:660)
at com.bumptech.glide.RequestManager.untrackOrDelegate(RequestManager.java:628)
at com.bumptech.glide.RequestManager.clear(RequestManager.java:624)
at com.bumptech.glide.RequestBuilder.into(RequestBuilder.java:811)
at com.bumptech.glide.RequestBuilder.into(RequestBuilder.java:780)
at com.bumptech.glide.RequestBuilder.into(RequestBuilder.java:771)
at com.microsoft.maui.PlatformInterop.prepare(PlatformInterop.java:229)
at com.microsoft.maui.PlatformInterop.loadInto(PlatformInterop.java:234)
at com.microsoft.maui.PlatformInterop.loadImageFromStream(PlatformInterop.java:265)
--- End of managed Android.Util.AndroidRuntimeException stack trace ---
android.view.ViewRootImpl$CalledFromWrongThreadException: Only the original thread that created a view hierarchy can touch its views.
at android.view.ViewRootImpl.checkThread(ViewRootImpl.java:8798)
at android.view.ViewRootImpl.requestLayout(ViewRootImpl.java:1606)
at android.view.View.requestLayout(View.java:25390)
at android.view.View.requestLayout(View.java:25390)
at android.view.View.requestLayout(View.java:25390)
at android.view.View.requestLayout(View.java:25390)
at android.view.View.requestLayout(View.java:25390)
at android.view.View.requestLayout(View.java:25390)
at android.view.View.requestLayout(View.java:25390)
at androidx.drawerlayout.widget.DrawerLayout.requestLayout(DrawerLayout.java:1353)
at android.view.View.requestLayout(View.java:25390)
at android.view.View.requestLayout(View.java:25390)
at android.view.View.requestLayout(View.java:25390)
at android.view.View.requestLayout(View.java:25390)
at android.view.View.requestLayout(View.java:25390)
at android.view.View.requestLayout(View.java:25390)
at androidx.recyclerview.widget.RecyclerView.requestLayout(RecyclerView.java:4586)
at android.view.View.requestLayout(View.java:25390)
at android.view.View.requestLayout(View.java:25390)
at android.view.View.requestLayout(View.java:25390)
at android.view.View.requestLayout(View.java:25390)
at androidx.core.widget.NestedScrollView.requestLayout(NestedScrollView.java:2075)
at android.view.View.requestLayout(View.java:25390)
at android.view.View.requestLayout(View.java:25390)
at android.view.View.requestLayout(View.java:25390)
at android.widget.ImageView.setImageDrawable(ImageView.java:597)
at androidx.appcompat.widget.AppCompatImageView.setImageDrawable(AppCompatImageView.java:112)
at com.microsoft.maui.glide.MauiCustomViewTarget.onResourceCleared(MauiCustomViewTarget.java:30)
at com.bumptech.glide.request.target.CustomViewTarget.onLoadCleared(CustomViewTarget.java:210)
at com.bumptech.glide.request.SingleRequest.clear(SingleRequest.java:337)
at com.bumptech.glide.request.ErrorRequestCoordinator.clear(ErrorRequestCoordinator.java:48)
at com.bumptech.glide.manager.RequestTracker.clearAndRemove(RequestTracker.java:70)
at com.bumptech.glide.RequestManager.untrack(RequestManager.java:660)
at com.bumptech.glide.RequestManager.untrackOrDelegate(RequestManager.java:628)
at com.bumptech.glide.RequestManager.clear(RequestManager.java:624)
at com.bumptech.glide.RequestBuilder.into(RequestBuilder.java:811)
at com.bumptech.glide.RequestBuilder.into(RequestBuilder.java:780)
at com.bumptech.glide.RequestBuilder.into(RequestBuilder.java:771)
at com.microsoft.maui.PlatformInterop.prepare(PlatformInterop.java:229)
at com.microsoft.maui.PlatformInterop.loadInto(PlatformInterop.java:234)
at com.microsoft.maui.PlatformInterop.loadImageFromStream(PlatformInterop.java:265)

Hi Garth,
I have no idea what is the problem with that... As you say, only the second time fails and I think that is an internal android problem (or mono droid). In the new plugging version, TakePhotoAsync works fine all times (because it use another android methods).

Yes, it is internal, and it shows a stream closing prematurely or disposing. Ill spend some time and work this one out!
From: Hector @.>
Sent: Friday, 12 May 2023 13:52
To: hjam40/Camera.MAUI @.>
Cc: Garth Connell @.>; Author @.>
Subject: Re: [hjam40/Camera.MAUI] Second (And only second!) photo after first run always fails. [Warning: Unable to load image stream.] (Issue #28)
Hi Garth,
I have no idea what is the problem with that... As you say, only the second time fails and I think that is an internal android problem (or mono droid). In the new plugging version, TakePhotoAsync works fine all times (because it use another android methods).
—
Reply to this email directly, view it on GitHubhttps://github.com/hjam40/Camera.MAUI/issues/28#issuecomment-1545622188, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ANBAMAP43EU3R4HLN7UCOBLXFYP45ANCNFSM6AAAAAAX4IJOXI.
You are receiving this because you authored the thread.Message ID<EMAIL_ADDRESS>

You might want to declare the stream variable outside the function where you are using it. ( as a class member) because of all the asynchronous work happening, the locally declared variables might be disposed before the rest of the function code completes.

@GarthConnell try put in your code a delay for 13000, before to star de camera, I had similar problem, but with this I fix it.

This issue persists unfortunately, also when using AutoSnapShot functionality from XAML. Any workaround as currently I cannot use this lib on Android.


Template spec solution update for Bitglass

Change(s):
See guidance below
Reason for Change(s):
This PR is part of template spec solution migration for solution  Bitglass.
See guidance below
Version Updated:
2.0.0
See guidance below
Testing Completed:
The link for test results is
https://microsoftapc.sharepoint.com/:w:/r/teams/SentinelCATTeam/_layouts/15/Doc.aspx?sourcedoc={08BD2EF7-5FFA-486D-AF28-86CFF40F10FF}&file=Sentinel Solution Migration -TemplateSpec - Bitglass.docx&action=default&mobileredirect=true&share=IQH3Lr0I-l9tSK8ohs_0DxD_Ab08waSW8Z5s2-etqhU6ZMc

Please rename Azure Sentinel to Microsoft Sentinel in below files


Indent with Promise chaining

// https://msdn.microsoft.com/en-us/library/windows/apps/hh700334.aspx

tempFolder.createFileAsync("tempfile.txt", Windows.Storage.CreationCollisionOption.replaceExisting)
   .then(function (tempFile) {
       // The createFileAsync call succeeded, so start the download operation.
       var downloader = new Windows.Networking.BackgroundTransfer.BackgroundDownloader();
       var transfer = downloader.createDownload(uriExample, tempFile);
        return transfer.startAsync();
   })

Copy-pasting this to Visual Studio makes this (with 2-space indent):
tempFolder.createFileAsync("tempfile.txt", Windows.Storage.CreationCollisionOption.replaceExisting)
  .then(function (tempFile) {
  // The createFileAsync call succeeded, so start the download operation.
  var downloader = new Windows.Networking.BackgroundTransfer.BackgroundDownloader();
  var transfer = downloader.createDownload(uriExample, tempFile);
  return transfer.startAsync();
})

This in fact worked in older version of TypeScript (I think the last version was 1.1), but now it's hard to make Promise chains.
// This is how I deal with Promise now
somePromise().then(() => {
}).then(() => {
});

// Not good...
somePromise().then(() => otherPromise(
  param1,
  "long-string, let's say it's a URL."
  ).then(() => {
  blah();
});

// Doesn't work
somePromise()
  .then(() => otherPromise(
  param1,
  "long-string, let's say it's a URL."
  )
  .then(() => {
  blah();
});

Can we get this better so that the MSDN example just work?

Sounds like a duplicate of #1888.


Re-enable osx_arm64 builds

          the `osx_arm64` builds are failing because they are cross-compiling and cppcheck builds something called dmake that it uses in it's build.  It appears to be cross-compiling dmake but wants to run it on the build host and that's wrong.

cmake explains that you have to do a native build first and then supply the path to your native built binaries in their docs at https://cmake.org/cmake/help/book/mastering-cmake/chapter/Cross Compiling With CMake.html#running-executables-built-in-the-project
Originally posted by @timsnyder in https://github.com/conda-forge/cppcheck-feedstock/issues/66#issuecomment-1445270658

Isuru pointed me to https://github.com/conda-forge/paraview-feedstock/blob/d4bebebea209add89221528dc524d00fd8c1bbf0/recipe/build.sh#L18-L20 in the gitter

Done in #85


Keep screen on for longer period

Screen should be kept on for approx 5 minutes while game is in progress.
There is library worth looking at https://github.com/corbt/react-native-keep-awake

Implemented this using react-native-keep-awake. Set timer to 5 minutes from last component refresh. Only works on Tracker screen.
There was problem with react-native 0.40 iOS dependencies so I've modified locally node_modules code in order to make it work. It is planned update in react-native-keep-awake module hence I'm not doing any custom module or any other modifications.


fix: filter two consecutive identical scree view events

[x] PR title and description conform to Pull Request guidelines.

Description of changes:

filter two consecutive identical scree view events

How did you test these changes?
added new test case for two consecutive identical scree view events
Documentation update required?

[x] No
[ ] Yes (Please include a PR link for the documentation update)

General Checklist

[x] Added Unit Tests

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Codecov Report
Attention: 1 lines in your changes are missing coverage. Please review.

Comparison is base (5c61666) 87.20% compared to head (d9797bd) 87.26%.

Files
Patch %
Lines

...solution/clickstream/client/ScreenRefererTool.java
66.66%
0 Missing and 1 partial :warning:

Additional details and impacted files
@@             Coverage Diff              @@
##               main      #65      +/-   ##
============================================
+ Coverage     87.20%   87.26%   +0.06%
- Complexity      454      455       +1
============================================
  Files            42       42
  Lines          1665     1665
  Branches        152      152
============================================
+ Hits           1452     1453       +1
  Misses          135      135
+ Partials         78       77       -1

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Fix and improve the c5:phpcs CLI command

Fix #9830
Move friendsofphp/php-cs-fixer from the require-dev section of /composer.json to the require-dev section of /concrete/composer.json (so that it can be used in composer-based setups)
Upgrade the code to use php-cs-fixer 3.1.0 instead of php-cs-fixer 2.16.4
Review all the fixer rules

PS: the included coding style rules are obviously just a suggestion of mine: I'm of course eager to discuss about them.
You can view all the available rules at https://mlocati.github.io/php-cs-fixer-configurator/#version:3.1
PPS: We have to use a specific version of php-cs-fixer because:

we use classes and methods marked as @internal (because https://github.com/FriendsOfPHP/PHP-CS-Fixer/issues/3126 has not been resolved yet)
@internal stuff may have breaking changes in dot releases too
PPPS: I added a custom fixer (see InlineTagFixer) because people at php-cs-fixer don't like PHP files containing mixed PHP+HTML, so they don't like fixers working with them (😵) (see https://github.com/FriendsOfPHP/PHP-CS-Fixer/pull/5272)

With the advent of our Concrete console library, which is meant to handle tasks in and around the concrete ecosystem including deployments, backups, restores, etc... would it not make more sense for this command to live there? It'd be fine to table this until another time, but I thought I'd raise the question since you're actively working on it now.

Sure! Let's continue the discussion at https://github.com/concrete5/console/issues/9

If you take more time to implement https://github.com/concrete5/console/issues/9, how about merging this PR to a temporary fix?

This PR is old and against a branch we're not using. I'm going to close this. Let's reopen this against the concrete console and work on this there, and figure out a pull request for the core phpcs command that informs the user that the command has been moved to the console class, and remove php cs fixer from the core entirely.

The PR itself looks good though - I think it's a fine start for integrating it in the console library.


Getting ValueError for unknown reason

I'm running the following keras model
input_profile = Input(shape=(23, 23, 1))
x = Conv2D(16, (3, 3), activation='relu', padding='same')(input_profile)
x = MaxPooling2D((2, 2), padding='same')(x)
x = Conv2D(8, (3, 3), activation='relu', padding='same')(x)
x = MaxPooling2D((2, 2), padding='same')(x)
x = Conv2D(8, (3, 3), activation='relu', padding='same')(x)
encoded = MaxPooling2D((2, 2), padding='same')(x)

x = Conv2D(8, (3, 3), activation='relu', padding='same')(encoded)
x = UpSampling2D((2, 2))(x)
x = Conv2D(8, (3, 3), activation='relu', padding='same')(x)
x = UpSampling2D((2, 2))(x)
x = Conv2D(16, (3, 3), activation='relu')(x)
x = UpSampling2D((2, 2))(x)
decoded = Conv2D(1, (3, 3), activation='sigmoid', padding='same')(x)

autoencoder = Model(input_profile, decoded)
autoencoder.compile(optimizer='adadelta', loss='binary_crossentropy')

X_GTEx = np.load('GTEx_X_float64.npy')
x_train = X_GTEx
x_train = np.reshape(x_train, (5207, 23, 23, 1))
from keras.callbacks import TensorBoard

autoencoder.fit(x_train, x_train,\
                epochs=50, batch_size=127,\
                shuffle=True, validation_data=(x_train, x_train),\
                callbacks=[TensorBoard(log_dir='/tmp/autoencoder')])

And running it gives me the following error:
ValueError: Error when checking target: expected conv2d_7 to have shape (20, 20, 1) but got array with shape (23, 23, 1)

Clearly, I haven't set anything with shape (20,20,1). Then what's wrong?

Hi,
You have a shape mismatch on the output, let's look at model.summary():
_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
input_1 (InputLayer)         (None, 23, 23, 1)         0
_________________________________________________________________
conv2d_1 (Conv2D)            (None, 23, 23, 16)        160
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 12, 12, 16)        0
_________________________________________________________________
conv2d_2 (Conv2D)            (None, 12, 12, 8)         1160
_________________________________________________________________
max_pooling2d_2 (MaxPooling2 (None, 6, 6, 8)           0
_________________________________________________________________
conv2d_3 (Conv2D)            (None, 6, 6, 8)           584
_________________________________________________________________
max_pooling2d_3 (MaxPooling2 (None, 3, 3, 8)           0
_________________________________________________________________
conv2d_4 (Conv2D)            (None, 3, 3, 8)           584
_________________________________________________________________
up_sampling2d_1 (UpSampling2 (None, 6, 6, 8)           0
_________________________________________________________________
conv2d_5 (Conv2D)            (None, 6, 6, 8)           584
_________________________________________________________________
up_sampling2d_2 (UpSampling2 (None, 12, 12, 8)         0
_________________________________________________________________
conv2d_6 (Conv2D)            (None, 10, 10, 16)        1168
_________________________________________________________________
up_sampling2d_3 (UpSampling2 (None, 20, 20, 16)        0
_________________________________________________________________
conv2d_7 (Conv2D)            (None, 20, 20, 1)         145
=================================================================
Total params: 4,385
Trainable params: 4,385
Non-trainable params: 0
_________________________________________________________________

Your input shape is (23, 23, 1) which is correct, however the output of the last layer, conv2d_7 is (20, 20, 1) which is incorrect.
Consider trying:
input_profile = Input(shape=(23, 23, 1))
x = Conv2D(16, (3, 3), activation='relu', padding='same')(input_profile)
x = MaxPooling2D((2, 2), padding='same')(x)
x = Conv2D(8, (3, 3), activation='relu', padding='same')(x)
x = MaxPooling2D((2, 2), padding='same')(x)
x = Conv2D(8, (3, 3), activation='relu', padding='same')(x)
encoded = MaxPooling2D((2, 2), padding='same')(x)

x = Conv2D(8, (3, 3), activation='relu', padding='same')(encoded)
x = UpSampling2D((2, 2))(x)
x = Conv2D(8, (3, 3), activation='relu', padding='same')(x)
x = UpSampling2D((2, 2))(x)
x = Conv2D(16, (3, 3), activation='relu', padding='same')(x)
x = UpSampling2D((2, 2))(x)
decoded = Conv2D(1, (2, 2), activation='sigmoid')(x)

This fixes the shape of the output to be (23, 23, 1) as desired.


fix: <<= token parse.

1. Does this PR affect any open issues?(Y/N) and add issue references (e.g. "fix #123", "re #123".):

[x] N
[ ] Y

fix: <<= token parse.

2. What is the scope of this PR (e.g. component or file name):

kclvm/lexer/src/lib.rs

3. Provide a description of the PR(e.g. more details, effects, motivations or doc link):

[ ] Affects user behaviors
[ ] Contains syntax changes
[ ] Contains variable changes
[ ] Contains experimental features
[ ] Performance regression: Consumes more CPU
[ ] Performance regression: Consumes more Memory
[x] Other

4. Are there any breaking changes?(Y/N) and describe the breaking changes(e.g. more details, motivations or doc link):

[x] N
[ ] Y

5. Are there test cases for these changes?(Y/N) select and add more details, references or doc links:

[x] Unit test
[ ] Integration test
[ ] Benchmark (add benchmark stats below)
[ ] Manual test (add detailed scripts or steps below)
[ ] Other

kclvm/lexer/src/tests.rs
kclvm/parser/src/lexer/tests.rs

6. Release note

Please refer to Release Notes Language Style Guide to write a quality release note.
None

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

3 of 3 (100.0%)  changed or added relevant lines in 3 files are covered.
31 unchanged lines in 1 file lost coverage.
Overall coverage decreased (-20.0%) to 68.02%

Files with Coverage Reduction
New Missed Lines
%

compiler_base/session/src/lib.rs
31
50.0%

Totals

Change from base Build<PHONE_NUMBER>:
-20.0%

Covered Lines:
30798

Relevant Lines:
45278

💛  - Coveralls


[Sig-scale] enable virt-operator to configure virt-handler to run more than 110 VMs per node

/kind enhancement
Motivation:
In the recent SIG-scale meeting, we are discussing to run a scale test with more than 110 VMIs per node. It is trivial to increase the max number of pods per node, but the max number of VMIs is not straightforward.
What happened:
Currently, virt-handler already have max-devices flag to increase the number of VMs per node
https://github.com/kubevirt/kubevirt/blob/0514a664151863d901d90694feb08c86df0076bb/cmd/virt-handler/virt-handler.go#L527
However, the virt-handler configuration is determined by the virt-operator that reverts all changes in the virt-handler Daemonset during the reconcile (of course, when we deploy it using the operator) .
What you expected to happen:
I expect to be able to configure virt-operator to create all virt-handler with a given max-devices number. Currently, all the virt-handler Daemonset options in virt-operator are hardcoded:
https://github.com/kubevirt/kubevirt/blob/0514a664151863d901d90694feb08c86df0076bb/pkg/virt-operator/resource/generate/components/daemonsets.go#L105

There are some discussion to use HCO annotation to patch the controllers
For example:
kubectl annotate --overwrite -n kubevirt hyperconverged kubevirt-hyperconverged kubevirt.kubevirt.io/jsonpatch='[                                                                                {
      "op":"add",
      "path":"/spec/customizeComponents/flags",
      "value": { "handler": { additionalProperties:  "--max-devices 250"} }
]'

However, kubevirtci does use HCO to deploy Kubevirt Operator

cc @rmohr @rthallisey @davidvossel @fgimenez
wdyt?

@marceloamaral yes I think we should allow setting this on the KubeVirt CR in an easy fashion.
In the meantime: HCO uses a patch mechanisms which is built into KubeVirt CR. You can define patches directly in the KubeVirt CR. See this PR for details: https://github.com/kubevirt/kubevirt/pull/3612
It is a slightly hidden feature, but I hope it helps.

can we easily detect the per node pod limit and use that as our VMI limit?

I created the PR #6098 to detect the per node pod limit and use that as our VMI limit


Update the technical description document

Imported from redmine, Issue #75177
Opened by @RickMethot on 2020-02-20
Status when imported: New
This would be a big project, but it seems worthwhile to move the technical description document into a git system, update and augment it, make it available as a living document in VLAB.
This seems like a few person-months of effort.  Rationale for bringing another person onto the team (not necessarily to do just this).  Your thoughts appreciated.
Rick

@Rick-Methot-NOAA Does ths issue seem out of date to you, in light of @nschindler-noaa 's flowchart work? Feel free to close if so.

The need still exists.  It is related to the flowchart, but the flowchart
will not have equations that describe what the code is intended to
implement.
On Thu, Mar 18, 2021 at 2:16 PM Kathryn Doering @.***>
wrote:

@Rick-Methot-NOAA https://github.com/Rick-Methot-NOAA Does ths issue
seem out of date to you, in light of @nschindler-noaa
https://github.com/nschindler-noaa 's flowchart work? Feel free to
close if so.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/nmfs-stock-synthesis/ss-documentation/issues/2#issuecomment-802305827,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABPV4ICZHCYOM7JHQ7BLNP3TEJUTVANCNFSM4TT46VBA
.

@k-doering-NOAA Thank you for the reminder on this.  Do you think this should be located within a "Technical Description" folder inside the "doc" repository or should this be a stand alone repository?  I can see pros and cons of each option.  Once we decide upon the location, I plan on creating a Rmarkdown document (rather than LaTeX) that we can begin working on.

Great!  I will create the folder and start a Rmarkdown document today so it can be incorporated into the workflow.  I suspect the completion of this document will take time but at least we will have a place to work when time permits.

OMG. This is huge news.

Don't get too excited yet since it is mostly a blank document waiting to be filled in at the moment. I am aiming to work a little bit on it every day to convert the existing technical documentation while creating space to add currently undocumented features.

Not totally blank: https://github.com/nmfs-stock-synthesis/doc/blob/main/technical_description/_main.pdf

This currently doesn't have a github action to render it, but I would be happy to add it in the future. It shouldn't take too long to set up, so feel free to ask when it is needed @chantelwetzel-noaa


About the  practical implementation of object memory

Hi, Cutie is a great work, however there was a problem annoying me which was about the actual implementation of pooling masks within object memory. In the raw paper, pooling masks was defined as below:

But in the code, it turns out that:

It seems like that the two zero conditions was actually replaced by multiplying with repeated_masks which was defined:

I wander if i got it right, look forward to your reply,thanks!

Hi, thank you for the question!
masks and inv_masks contain binary values (1 or 0) and they act as masks to the logits.sigmoid() term. You can think of the computation as

We compute condition (3) for every feature
If the feature satisfies condition (1), indicated by masks, we set it to zero by masking
If the feature satisfies condition (2), indicated by inv_masks, we set it to zero by masking

Effectively the result is the same. Feel free to ask if this isn't clear!

Hi, thank you for the question!
masks and inv_masks contain binary values (1 or 0) and they act as masks to the logits.sigmoid() term. You can think of the computation as

We compute condition (3) for every feature
If the feature satisfies condition (1), indicated by masks, we set it to zero by masking
If the feature satisfies condition (2), indicated by inv_masks, we set it to zero by masking

Effectively the result is the same. Feel free to ask if this isn't clear!

Thanks for your so quick reply! I have also considered that maybe "masks and inv_masks contain binary values (1 or 0) " , and in this way, the final result can be equal to the three conditions in paper. However,  I have found that masks was not binary tensor through debugging step by step. Further more, in order to validate it, i have also extracted the top-100 values from masks(its shape was [batchsize, object_num, 30, 54, 1])，the corresponding visualization code and top-100 elements were depicted as below:

According to values in the yellow box beyond to the png above, it seems like that the masks tensor was indeed non-binary, and thus results can not be same. Am i undertstanding it right?

Hi, thank you for the question!
masks and inv_masks contain binary values (1 or 0) and they act as masks to the logits.sigmoid() term. You can think of the computation as

We compute condition (3) for every feature
If the feature satisfies condition (1), indicated by masks, we set it to zero by masking
If the feature satisfies condition (2), indicated by inv_masks, we set it to zero by masking

Effectively the result is the same. Feel free to ask if this isn't clear!

Thanks for your so quick reply! I have also considered that maybe "masks and inv_masks contain binary values (1 or 0) " , and in this way, the final result can be equal to the three conditions in paper. However, I have found that masks was not binary tensor through debugging step by step. Further more, in order to validate it, i have also extracted the top-100 values from masks(its shape was [batchsize, object_num, 30, 54, 1])，the corresponding visualization code and top-100 elements were depicted as below:  According to values in the yellow box beyond to the png above, it seems like that the masks tensor was indeed non-binary, and thus results can not be same. Am i undertstanding it right?

Additional note: The excuted entry script was "scripting_demo.py"

Ah, it seems like the resize operation is introducing some non-binary values. They are probably inconsequential and the output would be a smooth interpolation between the conditions so I don't expect that to be a problem. Thank you for pointing this out. I think you are right.

Ah, it seems like the resize operation is introducing some non-binary values. They are probably inconsequential and the output would be a smooth interpolation between the conditions so I don't expect that to be a problem. Thank you for pointing this out. I think you are right.

Thanks for so patient reply! i have got it clearly :)


mrt2exabgp bails out with CRC error

Using http://data.ris.ripe.net/rrc00/latest-bview.gz:
root@pgen:/usr/local/etc/exabgp# /home/jlixfeld/mrtparse/examples/mrt2exabgp.py -A /home/jlixfeld/latest-bview.gz
announce route <IP_ADDRESS>/0 origin IGP as-path [45896 3356 ] community [3356:3 3356:9 3356:575 3356:2003] next-hop <IP_ADDRESS>
announce route ::/0 origin IGP as-path [29608 5511 5511 5511 5511 36925 ] med 11 community [5511:500 5511:518 5511:999 5511:5511 29608:30400] next-hop 2a01:678::2
announce route <IP_ADDRESS>/22 origin IGP as-path [45896 4826 38803 56203 ] community [19996:2001] next-hop <IP_ADDRESS>
announce route <IP_ADDRESS>/24 origin IGP as-path [45896 4826 38803 56203 ] community [19996:2001] next-hop <IP_ADDRESS>
...
...
...
Traceback (most recent call last):
File "/home/jlixfeld/mrtparse/examples/mrt2exabgp.py", line 498, in
main()
File "/home/jlixfeld/mrtparse/examples/mrt2exabgp.py", line 495, in main
conv_format(args, flags, d)
File "/home/jlixfeld/mrtparse/examples/mrt2exabgp.py", line 191, in conv_format
for m in d:
File "/usr/local/lib/python2.7/dist-packages/mrtparse-1.6-py2.7.egg/mrtparse/init.py", line 609, in next
self.unpack_data()
File "/usr/local/lib/python2.7/dist-packages/mrtparse-1.6-py2.7.egg/mrtparse/init.py", line 640, in unpack_data
data = self.f.read(self.mrt.len)
File "/usr/lib/python2.7/gzip.py", line 268, in read
self._read(readsize)
File "/usr/lib/python2.7/gzip.py", line 315, in _read
self._read_eof()
File "/usr/lib/python2.7/gzip.py", line 354, in _read_eof
hex(self.crc)))
IOError: CRC check failed 0x579c0579 != 0xd1fd5a7dL
I'm not well enough versed in Python to really be able to understand how to troubleshoot this.  Any assistance would be appreciated!

Thank you for using mrtparse.
I tried downloading "latest-bview.gz" and running mrt2exabgp.py, but there was no  error.
Since I want to know when "latest-bview.gz" you used was generated, so please tell me the result of the following commands:
# /home/jlixfeld/mrtparse/examples/print_all.py /home/jlixfeld/latest-bview.gz | head -n 10
# md5sum /home/jlixfeld/latest-bview.gz

The following results should be displayed.
% ./print_all.py /home/t2mune/Downloads/latest-bview.gz | head -n 10
---------------------------------------------------------------
MRT Header
    Timestamp:<PHONE_NUMBER>(2018-03-10 09:00:00)
    Type: 13(TABLE_DUMP_V2)
    Subtype: 1(PEER_INDEX_TABLE)
    Length: 567
PEER_INDEX_TABLE
    Collector: <IP_ADDRESS>
    View Name Length: 0
    View Name:
% md5sum /home/t2mune/Downloads/latest-bview.gz
d6bbb904b4c559d5ebc8e2569495f6d4  /home/t2mune/Downloads/latest-bview.gz

Hi,
I refreshed the copy of latest-bview.gz, and it works now.
Thank you for the pointer.

I'm glad the problem was solved.


Oh, my god, it works perfectly with all the major browsers

I have to activate compatibility mode on IE7.

Is it backward compatible, though?


Increase outTx inclusion timeout to 20 minutes

We've decided to increase EVM outtx inclusion timeout from 10 mins to 20 mins for mainnet to mitigate missing outtx tracker issue.

Mainnet label was used for issue for mainnet launch, we can remove this label. We should consider that these changes in the codebase are for all networks


Selected Row Visibility

Filter-tree and timeline-chart rows are now highlighted when selected.  Selecting a row in the timeline-chart selects the row in the filter-tree, and vice-versa.  Highlighted color matches the current color theme.
Requires timeline-chart changes to function: Selected Row Visibility - PR #209
Fixes: #772, #774
Signed-off-by: Will Yang<EMAIL_ADDRESS>

build-and-test is failing because the timeline-chart changes I made are not implemented yet.
https://github.com/eclipse-cdt-cloud/timeline-chart/pull/209

@williamsyang-work I'm going to review this and related timeline-chart patch.

It looks good to me. There seems to be build and linting error. Please fix a minor typo, build error and update the yarn.lock to include the new timeline-chart build.

The build error is caused by some network problem. I'll re-trigger build once it's back to normal. The linting error is unrelated and you need to fix it.

It looks good to me. There seems to be build and linting error. Please fix a minor typo, build error and update the yarn.lock to include the new timeline-chart build.

The build error is caused by some network problem. I'll re-trigger build once it's back to normal. The linting error is unrelated and you need to fix it.

The network issue is fixed. There are build and lint errors remaining. @williamsyang-work please update this PR to fix the issues.


Removed outdated refrence to git shell

According to this:
https://github.com/desktop/desktop/issues/340
https://github.com/desktop/desktop/issues/340#issuecomment-361716966
git shell is no longer included in the Desktop version.
It appears that the text in the item following the one I edited is auto generated. Now it says:

If you are using Git Bash, turn on ssh-agent:

The first part, "If you are using Git Bash" is now superfluous since the alternative of using the desktop app is no longer there.
This is my very first ever pull request! Please be patient with me.
Why:
Outdated/incorrect information

https://github.com/github/docs/blob/main/actions/reference/workflow-syntax-for-github-actions#permissions

I'm getting a 404 for that page.


Error: Timeout occurred while acquiring session on Google Cloud Run

Our team has been trying to run our node instances on Google Cloud Run. It works well for the first one or two hours after deployment. But, every Spanner read/write access fails with the following error after 1-2 hours of deployment:
Error: Timeout occurred while acquiring session.
at /usr/src/app/node_modules/@google-cloud/spanner/build/src/session-pool.js:719:31 at new Promise (<anonymous>)
at SessionPool._getSession (/usr/src/app/node_modules/@google-cloud/spanner/build/src/session-pool.js:718:27)
at getSession (/usr/src/app/node_modules/@google-cloud/spanner/build/src/session-pool.js:451:40)
at run (/usr/src/app/node_modules/p-queue/dist/index.js:157:104)
at PQueue._tryToStartAnother (/usr/src/app/node_modules/p-queue/dist/index.js:105:17)
at PQueue._next (/usr/src/app/node_modules/p-queue/dist/index.js:48:14)
at run (/usr/src/app/node_modules/p-queue/dist/index.js:168:22) From previous event:
at UserManager.getUsersColumns (/usr/src/app/node_modules/fam-users/dist/index.js:2012:36)
at processTicksAndRejections (internal/process/task_queues.js:93:5)

Here are the session pool options that we use for our database:
{
    acquireTimeout: 15000,
    concurrency: 100,
    idlesAfter: 5,
    max: 400,
    min: 25,
    incStep: 25,
    fail: true,
}

For debugging purpose, we log the session pool status every 30 seconds. Here is the (normal) session pool status right after deployment:
{
available: 25
borrowed: 0
currentWriteFraction: 0
isFull: false
isOpen: true
message: "USER_DB_SESSION_POOL_INFO"
numReadWaiters: 0
numWriteWaiters: 0
reads: 25
size: 25
totalPending: 0
totalWaiters: 0
writes: 0
}

Later, I found that all 25 sessions seemed to be borrowed:
{
available: 0
borrowed: 25
currentWriteFraction: 0.5
isFull: false
isOpen: true
message: "USER_DB_SESSION_POOL_INFO"
numReadWaiters: 1
numWriteWaiters: 0
reads: 0
size: 25
totalPending: 25
totalWaiters: 1
writes: 0
}

Then, the internal state of the session pool seemed to be screwed with the -23 borrowed & totalPending:
{
available: 48
borrowed: -23
currentWriteFraction: 0
isFull: false
isOpen: true
message: "USER_DB_SESSION_POOL_INFO"
numReadWaiters: 0
numWriteWaiters: 0
reads: 48
size: 25
totalPending: -23
totalWaiters: 0
writes: 0
}

Eventually, it reached the following session pool state:
{
available: 0
borrowed: 0
currentWriteFraction: 0.5
isFull: false
isOpen: true
message: "USER_DB_SESSION_POOL_INFO"
numReadWaiters: 0
numWriteWaiters: 0
reads: 0
size: 0
totalPending: 0
totalWaiters: 0
writes: 0
}

When the node instance has the session pool state with all 0s, every Spanner read/write access fails with Error: Timeout occurred while acquiring session. The issue persists until we redeploy a new instance.
Environment details

OS: ? (Google Cloud Run)
Node.js version: 12.16.1
npm version: 6.13.4
@google-cloud/spanner version: 5.15.0

Steps to reproduce

Run node instances on Google Cloud Run for a while.

@yonghoon Would you mind (temporarily) trying the following options for you test setup:
{
    idlesAfter: 500000, // This will make sure a session is (almost) never considered idle
    keepAlive: 500000, // This will practically disable the keep-alive calls for sessions
    acquireTimeout: 15000,
    concurrency: 100,
    idlesAfter: 5,
    max: 400,
    min: 25,
    incStep: 25,
    fail: true,
}

The idlesAfter and keepAlive settings above will effectively disable the pinging of sessions in the pool. My suspicion is that the problem in this case is caused by this pinging process that probably does not work very well on Cloud Run, because Cloud Run will disable the CPU of the container when there is no active request. This is the reason that running background activities are discouraged on Cloud Run.
Especially the second state that your process is logging seems to indicate that the session pool at that moment has been trying to ping all the 25 sessions in the pool, but probably with no success:
{
available: 0
borrowed: 25
currentWriteFraction: 0.5
isFull: false
isOpen: true
message: "USER_DB_SESSION_POOL_INFO"
numReadWaiters: 1
numWriteWaiters: 0
reads: 0
size: 25
totalPending: 25
totalWaiters: 1
writes: 0
}

The 25 borrowed sessions in this case is most probably not borrowed by your application, but by the keep-alive call that is executed in the background to try to keep the sessions alive. The totalPending: 25 log indicates that the pool is also trying to create 25 new sessions. This is because your application is requesting a session, and all sessions are currently in use (borrowed). The pool will then try to create a new batch of sessions, with 25 being the default batch size. This session creation also seems to take a very long time, which is what is causing the error. Exactly why that is taking such a long time in this case is not (yet) clear to me, but my suspicion is that it is caused by the failed/hanging keep-alive calls.
If changing the options to disable the pinging stops this problem from occurring, we at least know that it is caused by that. If that is the case, we could look into what a reasonable next step would be to make the pinging work on Cloud Run as well.
I don't have a direct explanation for the subsequent states that your application is logging for the session pool, but I will look into that as well.

@olavloite I will try the suggested session pool options. Thanks!

idlesAfter: 500000 & keepAlive: 500000 don't seem to resolve the issue. FYI, we have a very small amount of traffic, generated from a couple of my mobile clients, on the Google Run services. During the entire day, Google Run scaled the services up to 2 instances at max. We had a single instance for most of the day today.

The session started with normal numbers right after deployment:

{
available: 25
borrowed: 0
currentWriteFraction: 0.68
isFull: false
isOpen: true
message: "USER_DB_SESSION_POOL_INFO"
numReadWaiters: 0
numWriteWaiters: 0
reads: 8
size: 25
totalPending: 0
totalWaiters: 0
writes: 17
}

A few hours later, the numbers became abnormal.

{
available: 177
borrowed: -152
currentWriteFraction: 0.0903954802259887
isFull: false
isOpen: true
message: "USER_DB_SESSION_POOL_INFO"
numReadWaiters: 0
numWriteWaiters: 0
reads: 161
size: 25
totalPending: -152
totalWaiters: 0
writes: 16
}

Later, the numbers became a bit more crazier. ;-)

{
available: 329
borrowed: -304
currentWriteFraction: 0.0486322188449848
isFull: false
isOpen: true
message: "USER_DB_SESSION_POOL_INFO"
numReadWaiters: 0
numWriteWaiters: 0
reads: 313
size: 25
totalPending: -304
totalWaiters: 0
writes: 16
}

... the session pool numbers look weird but I haven't encountered the session acquisition timeout yet.

That is good news. Please let me know if it does occur again.

Are the negative borrowed and totalPending expected?

No, that is very strange and should not happen. I'll try to dive into that to figure out how that might happen. As far as I can tell from your statistics, the session pool still contains 25 sessions that are all available for use. The negative numbers for borrowed and totalPending are probably caused by the preparing of read/write transactions on sessions in the pool. The session pool will try to keep writeFraction sessions prepared with a read/write transaction, and while a session is being prepared, it is borrowed from the pool while a BeginTransaction RPC is executed. I think that those crazy numbers are caused by some error in that process (which is also partly a background process).
I don't know if it is possible for you, but it would be interesting to see the value of pendingPrepare next to totalPending. My suspicion is that the negative number for totalPending is caused fully by pendingPrepare (i.e. sessions that are pending being prepared with a read/write transaction, and not sessions pending creation). Would it be possible to include that in the statistics?

The session acquisition timeout still happens with the suggested session pool options. In this specific case of deployment, all the 25 borrowed ones seemed to get stuck in the pending state. The state persists forever until I redeploy services.
available: 0
borrowed: 25
currentWriteFraction: 0.5
isFull: false
isOpen: true
numReadWaiters: 0
numWriteWaiters: 0
pendingPrepare: 0
reads: 0
size: 25
totalPending: 25
totalWaiters: 0
uptime: 8241635
writes: 0

I just kicked off another round of deployment. I will update the ticket with further monitoring data tomorrow.

I encountered the session acquisition timeout again. One interesting thing about this is that the whole 25 sessions appeared to be borrowed right after the node instance was launched. There was no incoming traffic at all.
{
available: 0
borrowed: 25
currentWriteFraction: 0.5
isFull: false
isOpen: true
message: "USER_DB_SESSION_POOL_INFO (25/25) #1"
numReadWaiters: 0
numWriteWaiters: 0
pendingPrepare: 0
reads: 0
size: 25
totalPending: 25
totalWaiters: 0
uptime: 300783
writes: 0
}

The instance has been in the state for ~2.7 hours. I had 4 mobile clients hit the instance and all 20-ish requests failed due to the session acquisition timeout for ~40 seconds (2021-09-10 23:04:15.450 PDT - 2021-09-10 23:04:54.290 PDT).
"Error: Timeout occurred while acquiring session.
    at /usr/src/app/node_modules/@google-cloud/spanner/build/src/session-pool.js:719:31
    at new Promise (<anonymous>)
    at SessionPool._getSession (/usr/src/app/node_modules/@google-cloud/spanner/build/src/session-pool.js:718:27)
    at getSession (/usr/src/app/node_modules/@google-cloud/spanner/build/src/session-pool.js:451:40)
    at run (/usr/src/app/node_modules/p-queue/dist/index.js:157:104)
    at PQueue._tryToStartAnother (/usr/src/app/node_modules/p-queue/dist/index.js:105:17)
    at PQueue._next (/usr/src/app/node_modules/p-queue/dist/index.js:48:14)
    at run (/usr/src/app/node_modules/p-queue/dist/index.js:168:22)
    at runMicrotasks (<anonymous>)
    at runNextTicks (internal/process/task_queues.js:58:5)
    at processTimers (internal/timers.js:494:9)"

Eventually, the borrowed session became available somehow.
{
available: 25
borrowed: 0
currentWriteFraction: 0.04
isFull: false
isOpen: true
message: "USER_DB_SESSION_POOL_INFO (0/25) #29"
numReadWaiters: 0
numWriteWaiters: 0
pendingPrepare: 0
reads: 24
size: 25
totalPending: 0
totalWaiters: 0
uptime: 10011585
writes: 1
}

Now, all spanner calls on the instance are successful.

Thanks for releasing the fix for the negative session count. I used the latest release (https://github.com/googleapis/nodejs-spanner/releases/tag/v5.15.2) for our backend. The negative session count issue hasn't happen yet but the session timeout still happens.
All requests arriving between 2021-09-14 07:30:55.495 PDT and 2021-09-14 07:40:45.936 PDT failed due to the session acquisition timeout:

Here is the more detailed log BEFORE the session timeout happened:
{
available: 25
borrowed: 0
currentWriteFraction: 0.04
interval: 30000
isFull: false
isOpen: true
message: "SPANNER_DB_SESSION_POOL_INFO (available: 25, borrowed: 0, pending: 0, reads: 24, writes: 1) #456"
numReadWaiters: 0
numWriteWaiters: 0
pendingPrepare: 0
reads: 24
size: 25
totalPending: 0
totalWaiters: 0
uptime: 41130876
writes: 1
}

Here is the detailed log WHILE the session timeout happened (For ~10 minutes):
{
available: 0
borrowed: 25
currentWriteFraction: 0.5
interval: 30000
isFull: false
isOpen: true
message: "SPANNER_DB_SESSION_POOL_INFO (available: 0, borrowed: 25, pending: 25, reads: 0, writes: 0) #457"
numReadWaiters: 1
numWriteWaiters: 0
pendingPrepare: 0
reads: 0
size: 25
totalPending: 25
totalWaiters: 1
uptime: 41160883
writes: 0
}

Here is the detailed log AFTER the session timeout happened:
{
available: 25
borrowed: 0
currentWriteFraction: 0
interval: 30000
isFull: false
isOpen: true
message: "SPANNER_DB_SESSION_POOL_INFO (available: 25, borrowed: 0, pending: 0, reads: 25, writes: 0) #476"
numReadWaiters: 0
numWriteWaiters: 0
pendingPrepare: 0
reads: 25
size: 25
totalPending: 0
totalWaiters: 0
uptime: 41753026
writes: 0
}

@yonghoon What does interval: 30000 mean in the above logging? Is that the setting for idlesAfter and keepAlive? If so, is that the actual value in milliseconds, or is it in some other unit?

@olavloite 30000 (=30s) is the logging interval. I used the same session pool options that you suggested:
{
    idlesAfter: 500000, // This will make sure a session is (almost) never considered idle
    keepAlive: 500000, // This will practically disable the keep-alive calls for sessions
    acquireTimeout: 15000,
    concurrency: 100,
    idlesAfter: 5,
    max: 400,
    min: 25,
    incStep: 25,
    fail: true,
}

@yonghoon Cloud Run very recently introduced a new feature that should help mitigate this kind of problems: https://cloud.google.com/blog/products/serverless/cloud-run-gets-always-on-cpu-allocation
Would it be possible for you to try that option out to see if that solves this problem?

@olavloite I will try with the new allocation mode. Thanks for the suggestion!

I downgraded the priority since we haven't had an update in a while. @yonghoon let us know how your experiments went.

Closing this as we haven't heard back in a while. @yonghoon Please feel free to reopen if the problem persists.


Upgrade RxJS to TS-3.7

This should unblock Google from compiling RxJS with TS3.6, the other issues "should" be able to be fixed internally.
This should make a dent on #5047

@kolodny looks like there's one build error that needs to be resolved before I can merge this.

@kolodny looks like there's one build error that needs to be resolved before I can merge this.

Resolved.


knative/pkg: bump to release-1.7.

Changes
This just make us depend on a "released" version of knative/pkg
instead of a commit on main.
Signed-off-by: Vincent Demeester<EMAIL_ADDRESS>Submitter Checklist
As the author of this PR, please check off the items in this checklist:

[ ] Has Docs included if any changes are user facing
[ ] Has Tests included if any functionality added or changed
[x] Follows the commit message standard
[x] Meets the Tekton contributor standards (including
functionality, content, code)
[x] Has a kind label. You can add one by adding a comment on this PR that contains /kind <type>. Valid types are bug, cleanup, design, documentation, feature, flake, misc, question, tep
[ ] Release notes block below has been updated with any user facing changes (API changes, bug fixes, changes requiring upgrade notices or deprecation warnings)
[ ] Release notes contains the string "action required" if the change requires additional action from users switching to the new release

Release Notes
NONE

/kind misc

/approve


Allow scanning unpacked container filesystems

:bug: Fixes https://github.com/anchore/syft/issues/1359 :tada:
This PR is an attempt to explore how to modify the existing directory resolver to allow for resolving symlinks in an unpacked container filesystem. To do this, we add a new optional base parameter for the directory resolver that resolves all symlinks relative to this base. There are two intended use cases:

base = "/". The previous behavior, symlinks are resolved relative to the root filesystem.
base = path. Symlinks are resolved relative to the target filesystem, allowing correct behavior when scanning unpacked container filesystems on disk.

Currently, this only modifies the API surface for the source, which would probably be sufficient for my use case in buildkit-syft-scanner.
I've marked this as in-draft since I've only done some preliminary testing, but the following things probably need working out before this could be ready:

[x] Extend this to the CLI with a root-dir input type, or similar? Not quite sure on naming, I don't have any particular preference here.

Update: planned for follow-up

[x] Add tests for the new directory resolver behavior. The logic seems sound to me, but I may have missed something - thankfully, the symlink resolution seems isolated to a couple of places.

Update: tests added

[x] I've temporarily replaced filepath.EvalSymlinks with cfs.RootPath from containerd/continuity. The logic of these functions is fairly similar, however, the containerd equivalent allows bounding the result to a specified directory.
This logic is only needed for 1 of the symlink resolution components - the other is much simpler, since syft does the heavy lifting. I'm not sure how the maintainers feel about adding this as a new dependency? Maybe there's a simpler solution I'm not seeing :eyes:
Update: I can remove this dependency, since we can avoid this second instance of doing symlink resolution - we can just use the file tree that we've already indexed over, which should be more efficient anyways. I think that this is a valid use? It seems to work in the cases I tried it on.

cc @wagoodman @kzantow

Would love to the see this feature make it to the CLI. We have a use case for wanting to run syft on a root filesystem and have it work just like syft does against an existing docker image today.


Support for SD-JWT?

Any plans to support this? https://datatracker.ietf.org/doc/draft-ietf-oauth-selective-disclosure-jwt/

Hi @advatar, no plans on our end at the moment. Once the draft matures into a standard we'd be open to discuss reviewing a PR adding this if. Note though that we don't support the JWT standard currently since we're focussed on the underlying JWS/JWE structures. If you think it would still make sense to add this to this library, please feel free to open another issue or a PR with a proposal of getting this in. 🙏


Add databricks_automl to the conda env for arima and deepar

It was already passed in for Prophet. Adding for ARIMA and DeepAR, too, so we can log correctly.

Can you add a unit test

@apeforest the existing UT to log, load, and predict an ARIMA model passes, see here: https://github.com/databricks/automl/blob/branch-<IP_ADDRESS>/runtime/tests/automl_runtime/forecast/pmdarima/model_test.py#L432
I could add a test / check to see that the requirements.txt that are logged with "automl_runtime=0.2.20.x" for all 3 models?

Thanks @sabhya-db. A few nits. Otherwise, LGTM Thanks

Thanks for checking, will fix nits soon

LGTM. Do you know if it's a serverless-only issue, or should we also port this to the main branch?

@es94129
We should do this on the main branch too, an ARIMA model logged from the main branch won't be able to be served at present. For a quick hotfix you can just add "automl_runtime=xyz" to ARIMA_CONDA_ENV here: sabhya-db/ML-45532-sample-weight-2

Can you also port the changes for ARIMA to the main branch then?

will do in a separate PR


Fixed missing . in import

Duplicate of ask/celery#538 and ask/celery#540

Thanks! Closed by duplicate #540


Don't perform ASI before as if it's followed by an identifier (allow foo \n as Foo)

Fixes #11369

@dotnet-bot test this please

I don't necessarily think we should be creating a more complex syntactic edge-case here. If a user adds parentheses around their type, this suddenly becomes semantically different. I'd be fine with if you parsed this as you are now, and then issued a helpful error message though.

I think the error message we have now is clear enough.


Create Database Artifacts Using Core Data Services (CDS) for SAP HANA Cloud

Tutorials: https://developers.sap.com/tutorials/hana-cloud-cap-create-database-cds.html
Step 2 pint 2 ERROR, correct the I code snippet:
using app.interactions from '../db/interactions';
change it with
using app.interactions from '../db/src/interactions';
the path is ./db/src!!!

Thomas, can you please respond to this user's feedback?

Actually not it shouldn't be ./db/src. it should be /db.  Look back at step 1.1.  We say to create the file in ./db not ./db/src.  ./db/src is for HANA native artifacts.  The CDS files should go into the /db folder.


Readable error message for metadata schema validation

Pull Request Checklist:

[ ] Positive Test Case Written by Dev

[ ] Automated Testing

[ ] Sufficient User and Developer Documentation

[ ] Passing Jenkins Build

[ ] Peer Code review and approval

Positive Test Case

[Enter positive test case here]

METRIC
VALUE

NEW VULNERABILITIES
0

NEW BUGS
0

NEW SECURITY HOTSPOTS
0

NEW CODE SMELLS
0

NEW DUPLICATED LINES DENSITY
0.0

https://sonarqube.cuahsi-workstation.com:9000/dashboard?id=hydroshare-4294

The commit message issue number is incorrect. 4270 should be 4279


Add French translation for index.md

Hello 👋
This is a PR to translate docs/index.md to french :fr:
According to the issue: https://github.com/tiangolo/fastapi/issues/1972
Saving you some clicks:

the formatted document: here
the diff from English: here

Thanks 👽

@JulianMaurin I suggest you to create a separate MR for the initial language setup. This was done on the other languages because the setup itself doesn't need a French speaker as reviewer and it can be merged faster.

@JulianMaurin I suggest you to create a separate MR for the initial language setup. This was done on the other languages because the setup itself doesn't need a French speaker as reviewer and it can be merged faster.

Hey @Kludex, thanks for your feedback.
Here is the PR: https://github.com/tiangolo/fastapi/pull/1975

Thanks to Development - Contributing' guidelines:

Translation is correct for me ( Thx Julian ;) )
Translation can be merged! ( Thx to FastAPI Team ;) )

Thanks in advance.

Hi,
Thank you for the great work!
I've made some suggestions to the text. I hope it was the right method 😅 and that it will be useful.
There is still missing space after colon ponctuation, like for example line 36 to 42. But I think I've already made too much suggestions.
I would also be favorable to put a space after the three dots as in the expression '... de' (line 383). But I wasn't sure at 100%...
I would like to contribute from time to time. Let me know what the best method to help when reviewing the translation.
Thanks!

@jburckel

I've made some suggestions to the text. I hope it was the right method 😅 and that it will be useful.

Yes was, thanks 🙏

There is still missing space after colon ponctuation, like for example line 36 to 42. But I think I've already made too much suggestions.

✅ 93475d2914a65abe818ae84c4b0ae53a05c5d716
Validating there are no more missing withspaces using the following expressions:

before colon: [^( |https|text/-align)]:
after colon: :[^( |\/\/|15px|app|8)]

I would also be favorable to put a space after the three dots as in the expression '... de' (line 383). But I wasn't sure at 100%...

✅ 51a21a041c0bf1fa3f2ccd14de1765592e50137e
According to LeConjugeur you are right.
Validating using the following expressions:

additional withspaces before ellipsis:  \.\.\.
missing withspaces after ellipsis: \.\.\.[^( |\])]

I would like to contribute from time to time. Let me know what the best method to help when reviewing the translation.

Reviewing PRs is great! You can also take a look at the documentation and at #1972 to add you own translations :)


Slim Memory; remove hasByteBuffer() and getByteBuffer() methods

Because of https://github.com/druid-io/druid/pull/5335#issuecomment-365274611 (see the last part of the message) Memory objects are going to need to carry minimal overhead over the memory bytes themselves, at least, in comparison with ByteBuffer (in the linked message, an example API uses ByteBuffer, because it's what currently used in Druid, but it's really going to be replaced with Memory).
In order to achieve this, I think separate ResourceState objects need to be dropped. Memory is either it's own "resource state" (for Memory objects, obtained from wrap(), allocate(), or map() calls), or references to another Memory as it's "resource state" (for Memory objects, obtained from region() calls). Buffer also references to some Memory as it's resource state.
Memory has the functionality of ResourceState as package-private methods.
StepBooleans should be "inlined" to be just volatile boolean fields. (And isReadOnly shouldn't be volatile, it should be just final, because it's not changing during Memory lifetime.
Heap and off-heap Memory should also probably be separated, because off-heap needs a lot of fields, not needed by on-heap Memory: handle, valid (we change the validity state only when we unmap of deallocate direct memory, so not needed for on-heap), reference to ByteBuffer, etc. Off-heap Memory could be much beefier than on-heap, because we are not going to map or allocate small off-heap Memories.
Because of this, I think hasByteBuffer() method should be removed, because we cannot consistently support it, when wrapping on-heap ByteBuffers. Also, I think this method is useless.
Class hierarchy could look like this:
Memory
|
WritableMemory
|
BaseWritableMemoryImpl------
|                           |
WritableMemoryImpl          NonNativeEndianWritableMemoryImpl
|                           |
OffHeapWritableMemoryImpl   OffHeapNonNativeEndianWritableMemoryImpl

OffHeapWritableMemoryImpl and OffHeapNonNativeEndianWritableMemoryImpl have pretty much the same bodies, but I don't see how to avoid this repetition.

You are recommending a lot of changes here, I will have to think through them separately.

I think hasByteBuffer() method should be removed, because we cannot consistently support it, when wrapping on-heap ByteBuffers.

I'm not sure what you mean by "cannot consistently support".  If we have wrapped a ByteBuffer we MUST retain a pointer to it so that it doesn't go out of scope before we are done with it.  That being the case, providing the hasByteBuffer() is merely a convenience so that the user isn't surprised when getByteBuffer() returns null.   It parallels the ByteBuffer.hasArray() method.

I'm not sure what you mean by "cannot consistently support". Also, your discussion is about reducing memory footprint. The hasByteBuffer() method has no impact on memory footprint.

In the course of changes that I suggested, on-heap Memory won't have a ByteBuffer field, so when wrap(ByteBuffer) is called with a heap ByteBuffer, it carves the underlying byte[] object out and doesn't retain a link to the BB. So it won't be able to say, if Memory was created via wrap(BB) or wrap(byte[]).

If we have wrapped a ByteBuffer we MUST retain a pointer to it so that it doesn't go out of scope before we are done with it.

It's true for off-heap, but not on-heap ByteBuffers.

That being the case, providing the hasByteBuffer() is merely a convenience so that the user isn't surprised when getByteBuffer() returns null.

I missed that there is getByteBuffer(). I suggest to remove it too.

I created ResourceState to simplify the code both in terms of understanding, maintenance, and vastly fewer parameters being passed around.  The on-heap case requires, I think, 10 parameters. The off-heap case adds 6 more.
I am a little puzzled by your concern about space consumed by Memory when wrapping on-heap arrays.  I thought that the vast majority of use of ByteBuffer in Druid was for accessing data off-heap.  Do you use also have heavy use of on-heap ByteBuffers?

WRT the Read-only flag.
Yes, in theory, for the on-heap case, it could be final and passed to the constructor ... along with the unsafeObj_, unsafeObjHeader_, capacity_, cumBaseOffset_, memReqSvr_, valid_, regionOffset_,  byteBuf_, resourceOrder_.  The off-heap case would add nativeBaseOffset_, handle_, file_, fileOffset_, raf_, mbb_.
Responding to your next message ... why remove getByteBuffer()?  It is there for the same reason that ByteBuffer has .array().   The user may want to use Memory as a more flexible tool for manipulating data.  So you wrap(BB), pass Memory to some worker, then when you get the Memory back, you can recover the modified BB.

I am not opposed to your proposed hierarchy, but I think eliminating ResourceState will be more complex than you think.

I am a little puzzled by your concern about space consumed by Memory when wrapping on-heap arrays. I thought that the vast majority of use of ByteBuffer in Druid was for accessing data off-heap. Do you use also have heavy use of on-heap ByteBuffers?

I the first message of this thread, I linked specifically to the place, where we are going to need to allocate a lot of really small on-heap Memory objects (starting from 0-8 bytes in payload).

The user may want to use Memory as a more flexible tool for manipulating data. So you wrap(BB), pass Memory to some worker, then when you get the Memory back, you can recover the modified BB.

I thought about this. When you wrap(BB), you already have a BB. So the pattern could be
mem = wrap(BB)
doSomethingWith(mem)
use(BB); // BB is modified

This is a pretty common pattern.

And if the recipient is downstream?

Then you could pass them both downstream (doSomethingWith(mem, BB)).

Seems kind of silly if mem wraps BB, why you would have to pass both.  Nonetheless, we are arguing over just one of the fields that is just a reference.  Have you looked closely at all the others?

What about the other fields? On-heap Memory really needs only the following fields:

Object unsafeObj, long cumBaseOffset, long capacity
boolean isReadOnly
ByteBuffer buf (under debate)

Off-heap additionally needs a reference to a cleaner, mandatory reference to a BB, etc.

-unsafeObjHeader_  OK, we could eliminate this one and reference UnsafeUtil.

valid_  OK, for on-heap, this would be an internal static and not need to be passed.

memReqSvr_   This is required so that a recipient can request more memory from the memory owner.  This does happen and should not depend on whether the memory is on-heap or off-heap, as the recipient does not know and should not care.  If Druid were to actually take advantage of this, it would reclaim a whole lot of wasted memory today.   Our off-heap sketches today already can utilize this.

regionOffset_  Also required for regions within on-heap allocated spaces.  ByteBuffer has a similar field.  Note that even for on-heap, this is different from cumBaseOffset, which includes the unsafeObjHeader_

resourceOrder_  Required.  We should not assume the byte order of fields within a wrapped byte[]. Those bytes could come from anywhere.

byteBuf_ (I still think we should offer this.) also for symmetry with off-heap.

memReqSvr_ - If Druid were to actually take advantage of this, it would reclaim a whole lot of wasted memory today.

I don't see MemoryRequestServer and MemoryManager API as really usable. If you need some kind of pooling or special resource management, the requirements tend to be quite specific. And you don't really need this to be inside the memory object, you could always pass a reference to the pool along with the reference to the memory, when required (and it happens really rarely.)
regionOffset_ - I see getRegionOffset() method now, but what't an application of it? Let's remove this method too. Also, you could implement this method without the field as
return cumBaseOffset - unsafeObj == null ?
 0 : unsafe.arrayBaseOffset(unsafeObj.getClass())

resourceOrder_ - class hierarchy helps, so resource order information is extractable from the class of the Memory :)

As far as I can see, unsafeObjHeader_ is used only in toHex(). Also, I didn't benchmark it, but I suspect that calls like unsafe.arrayBaseOffset(unsafeObj.getClass()) are really fast.
Yes, valid_ needs to stay around, but only in off-heap Memories. It doesn't contradict the scheme suggested in this message: https://github.com/DataSketches/memory/issues/42#issue-300288008

unsafeObjHeader_ is used anytime cumBaseOffset_ is computed, which occurs anytime any one of regionOffset_, nativeBaseOffset_ or unsafeObjHeader_ changes, and when ResourceState is copied (for regions, duplicates).

These are also not performance-critical methods. And slowness of unsafe.arrayBaseOffset(unsafeObj.getClass()) is even not proven

Done.

Reopening because we still need slim Memory.

@leventov
Actively working on this. To do this right is requiring some major refactoring.
Nonetheless, I have a question:
Currently, the only way to create a Non-native instance of Memory is by wrapping a byte[] and specifying a non-native order, or by wrapping a BB that already happens to be non-native order.
Do you have any use-cases where you would want to start with a native instance and convert it to a non-native instance (or visa-versa), not changing whatever the resource is?  I.e., a Memory.reverseOrder()?
The only case I can think of would be if you have a mapped file of (God forbid!) mixed order!

The only case I can think of would be if you have a mapped file of (God forbid!) mixed order!

This is actually the main use-case for non-native Memory in Druid. I didn't bother to add methods like Memory.region(off, cap, ByteOrder) yet, but it should be easy.

Since you currently use MappedByteBuffers for mapping files, and they can contain mixed order regions, this means the BB.order() value is meaningless.  How do you know when and where to switch the BO of the BB and to what order?
This is probably why, in your situation, you are basically ignoring the BO state of the BB, and why, in your mind, there is only one BO state.
Our auto-setting of the implementation BO to match the BO state of the BB is not of much use to you.   But it may make sense in other environments.  This is a mess.

Essentially, what you're saying is the BO state of BB is not a reliable indicator of what the byte order is of the underlying data ... so we should ignore it and force the user to specify what is the intended order.

, this means the BB.order() value is meaningless. How do you know when and where to switch the BO of the BB and to what order?

This is probably why, in your situation, you are basically ignoring the BO state of the BB, and why, in your mind, there is only one BO state.
Our auto-setting of the implementation BO to match the BO state of the BB is not of much use to you. But it may make sense in other environments. This is a mess.
I didn't understand any of those assertions and questions.

Essentially, what you're saying is the BO state of BB is not a reliable indicator of what the byte order is of the underlying data ... so we should ignore it and force the user to specify what is the intended order.

I would say that ByteOrder is not an indicator of anything, it's a property of a Memory (or ByteBuffer) object.

The way BB is designed, you can change the BO on the fly without creating a new object.  This is why the access of multi-byte primitives from BB is slow: The accesses are byte-by-byte, whether the BO is native or not.  However, doing a duplicate() or slice() on a BB does not copy over the current state of BO, which is a bug.
In our current approach, changing the byte-order requires creating a new Memory object.  If we want to be able to change the BO on the fly, and be fast, we need to rethink the API.

I didn't propose that. I didn't say that byte order is a mutable property of Memory. It's an immutable property.

I can reduce the weight of the Memory object by about 50% (10 instance variables down to 5 for heap primitives), but it requires me to add 16 additional classes, 4 for each resource type: Memory-native, Memory-non-native, Buffer-native, and Buffer-non-native.
I am nearly done with these changes, but looking at the whole package it seems overly complex.
There is one thing we can do to vastly simplify the code:  that would be to merge the positional capabilities of Buffer into Memory and eliminate the parallel Memory/Buffer hierarchy.

The Memory hierarchy would then sit on top of "BaseBuffer" (would be renamed) and the positional logic would only be called by methods that use it.  This is how ByteBuffer works now --  positional and non-positional methods side by side in the same API.
This parallel API was originally Eric's suggestion.  But I have been suspicious ever sense that it is ultimately harder for the user and adds too much duplicate code and complexity.
asBuffer() and asMemory() go away, as well as having to keep and track an originMemory.  It would eliminate at least 14 classes in the whole package.

The only other change I would recommend is to not support both positional and non-positional array put/get methods.  I suggest we only support non-positional array methods, since it is trivial to put "position" in for the offset argument. This is different from how BB handles arrays.  But this also gives the user freedom to decide whether to update the position based on the length of the array or not, which they would do outside the array call.
The other consequence of this is that Memory would add 3 more instance variables (start, position, end).  This is very close to what ByteBuffer has now, with the note that BB is based on integers and Memory is based on longs.
What do you think?

I can reduce the weight of the Memory object by about 50% (10 instance variables down to 5 for heap primitives), but it requires me to add 16 additional classes, 4 for each resource type: Memory-native, Memory-non-native, Buffer-native, and Buffer-non-native.

See the class diagram from here: https://github.com/DataSketches/memory/issues/42#issue-300288008. 1) I don't need "slim Buffers" in Druid, so this issue focuses on Memory; 2) I don't see why direct memory and mapped memory couldn't be handled in one class. So only two new classes should be enough.

There is one thing we can do to vastly simplify the code: that would be to merge the positional capabilities of Buffer into Memory and eliminate the parallel Memory/Buffer hierarchy.

I don't like this idea. It was discussed in https://github.com/druid-io/druid/issues/3892. There should be API surface without positional (mutable) methods. So I would not touch Buffers, especially in the context of this issue.

Direct requires MemoryRequestServer and Map does not. Even if we combined these two, we are talking about 12 leaf classes instead of 16.

I don't understand why 12. I see just two: OffHeap subclasses for non-natuve and native WriteableMemoryImpls.

There are 4 instance variables that are required everywhere: capacity, regionOffset, cumBaseOffset, readOnly.  These sit at the root of the hierarchy.
The additional instance variables required depends on the resource:

Heap arrays require the unsafeObj reference.
Direct require: nativeBaseOffset, valid, memReqServer
Maps require: nativeBaseOffset, valid,
ByteBuffers require: byteBuf, nativeBaseOffset, unsafeObj

In addition, the Buffer hierarchy requires originMemory.
In order to use instance variables ONLY where they are needed the hierarchy looks like this.
----------|----------|-----------|------------------|------------|----------------
BaseState | Memory | WMemory | BaseWMemImpl | WMemImpl | HeapWMemImpl
| | | | | DirectWMemImpl

There are 4 instance variables that are required everywhere: capacity, regionOffset, cumBaseOffset, readOnly. These sit at the root of the hierarchy as finals.
The additional instance variables required depends on the resource:

Heap arrays require the unsafeObj reference. Everywhere else it is null (but no instance var holds the null).
Direct requires: nativeBaseOffset, valid, memReqServer
Maps require: nativeBaseOffset, valid,
ByteBuffers require: byteBuf, nativeBaseOffset, unsafeObj

In order to use instance variables ONLY where they are needed the hierarchy looks like this just for the Memory branch. The names are abbreviated here.

root
public abs
public abs
1st Impl
2nd Impl
3rd Impl

BaseState + 4 vars
Memory
WMem
BaseWMemImpl
WMemImpl
HeapWMemImpl + 1 var

DirectWMemImpl + 3 vars

MapWMemImpl + 2 vars

BBWMemImpl + 3 vars

WNonNative MemImpl
HeapWMemImpl + 1 var

DirectWMemImpl + 3 vars

MapWMemImpl + 2 vars

BBWMemImpl + 3 vars

The Buffer hierarchy is similar except it adds orignMemory to BaseWBufImpl and also the additional layer for the positional BaseBuffer.  Thus, 16 leafs for both.
Combining Direct and Map would mean map would carry an additional instance var that it doesn't use. Thus 3 X 4 = 12.
This will be clearer when I post the branch.

In order to use instance variables ONLY where they are needed the hierarchy looks like this just for the Memory branch.

This is not needed. I need to be slim only one flavour of Memory, that is obtained by wrapping a byte[] array, or equivalently returned from Memory.allocate(). All others need not to be slim, so they could share all the remaining fields (as well as all flavours do right now).

capacity, regionOffset, cumBaseOffset, readOnly

BTW regionOffset is a product of cumBaseOffset and the class of the unsafeObj field, so it's not needed to be stored as a field.

BTW regionOffset is a product of cumBaseOffset and the class of the unsafeObj field, so it's not needed to be stored as a field.

No it is not.  regionOffset serves the same purpose as the package-private field ByteBuffer.offset, which stores the base offset of the current view from the origin of the parent, for heap arrays and for MappedByteBuffer due to a slice().  And slice() can be based on any arbitrary byte offset based on whatever position is.  Which means it is not an even multiple of the unsafeObj object header size. So a product based on that would be wrong.
It is a bit more complex in that DirectByteBuffer uses the address field to keep track of any offset produced from a slice.  DBB also inherits from ByteBuffer, so it inherit the offset field, but doesn't use it.
In our case our nativeBaseOffset is the equivalent of address, with the exception that it is final.  If the user performs a slice() of a DBB prior to a wrap(), the nativeBaseOffset records the actual off-heap native address + the sliced offset.  Any region() operations after that record the additional offset in regionOffset.
Unlike ByteBuffer, we have the ability recover the region offset from WritableMemory objects, which is the actual offset from the root parent view of whatever the original resource was.  It turns out this is quite handy if the user needs to compute an offset of some object in a complex data structure and doesn't want to pay the cost of actually creating a region.  When reading thousands of objects from some array inside a complex data structure, not having to create a region for each one is a big performance advantage.
I don't see any shortcut that would allow us to eliminate that field.

we have the ability recover the region offset from WritableMemory objects, which is the actual offset from the root parent view of whatever the original resource was. It turns out this is quite handy if the user needs to compute an offset of some object in a complex data structure and doesn't want to pay the cost of actually creating a region.

It seems like regionOffset breaks incapsulation in some way. And I don't actually understand how does it allow to avoid creating slice objects and perform some calculations, that couldn't be done without regionOffset.

@leventov
Yes, getRegionOffset() does break encapsulation, as does getArray(), getCumulativeOffset(), and being able to cast Memory to WritableMemory. In addition, unsafeByteBuffer() and UnsafeUtil.unsafe could be misused.
Nonetheless, we have chosen to expose these methods, because the systems we want to use the Memory package in have real requirements for these capabilities, almost always for performance reasons.

The thing is that I don't see in what cases getRegionOffset() is required for efficiency of any operation. You mentioned avoiding creating slice() objects. Could you please demonstrate an example?

Sure. Look at this deserialization method
The interface requires passing in Memory .  Using getByteArray() would constitute an extra copy in passing to ByteArrayInputStream().  And in this Pig environment we know that the only backing resource for Memory is always a byte[].  Getting the region offset and a reference to the backing array is much faster.

https://github.com/DataSketches/sketches-pig/blob/53e33bfd56240b671c188a7455b3bd415a1db9b2/src/main/java/com/yahoo/sketches/pig/sampling/ArrayOfTuplesSerDe.java#L44

@leventov
All the different resource types: Heap, Direct, Map, ByteBuffer only create overhead space for just the instance variables required.
I expect to create a release in the next few days.  So please review!
This can now be closed.


feat: allow ml_feature_table as an asset type

Related to https://github.com/odpf/meteor/pull/417

LGTM, just a small concern if ml_feature_table is a good type name? cc: @sudo-suhas @ravisuhag

Do you have a suggestion @StewartJingga? In the PR for adding E2E lineage entities (https://github.com/odpf/proton/pull/193), we had discussed calling it FeatureTable instead of MLFeatureTable but chose to keep it as MLFeatureTable because 'Feature' is an overloaded word that is not clear enough.


apps wc: Propagate PSA ns labels for HNC namespaces

What this PR does / why we need it:

Propagates PSA labels for user hnc namespaces
Possible to override usernamespace PSA level (Will not be a managed HNC namespace)
Moved read-hierarchyconfigurations-role into hnc user-rbac
Removed hnc admin-rbac from admin (attached to user admins)

Which issue this PR fixes: fixes #1530
Public facing documentation PR (if applicable)

Special notes for reviewer:
Add a screenshot or an example to illustrate the proposed solution:
Checklist:

[x] Added relevant notes to WIP-CHANGELOG.md
[x] Proper commit message prefix on all commits
[ ] Updated the public facing documentation
Is this changeset backwards compatible for existing clusters? Applying:

[ ] is completely transparent, will not impact the workload in any way.
[x] requires running a migration script.
[ ] will create noticeable cluster degradation.
E.g. logs or metrics are not being collected or Kubernetes API server
will not be responding while upgrading.
[ ] requires draining and/or replacing nodes.
[ ] will change any APIs.
E.g. removes or changes any CK8S config options or Kubernetes APIs.
[ ] will break the cluster.
I.e. full cluster migration is required.

Chart checklist (pick exactly one):

[x] I upgraded no Chart.
[ ] I upgraded a Chart and determined that no migration steps are needed.
[ ] I upgraded a Chart and added migration steps.

Pipeline config (if applicable)
If you change some config options (e.g. add/rename variable or change the default value) you may need to update the config used by the pipeline in pipeline/config.

We should tie up all dependencies we now have on gatekeeper. Or make gatekeeper non optional. (Though that should be done in a separate PR.)


Having Issue in Connecting Microservice Deployment to Database Service.

I am creating a Microservice Deployment, a database Service. Struggling to connect them to each other.

Hi Aditya.
How are you trying to access the db service in your app microservice?
Ideally you have to deploy the database cluster service first and then the name of the db service can be used to access database from your app service.
say if you create a cluster service for your database service with name 'sample-database' and container port '1234', then you can access the database as http://sample-database:1234 from your app service.
you can refer this:
https://github.com/airavata-courses/vulcan/tree/feature-user-management/user-management/Kubernetes


Cannot run matchtigs

Hi! I've installed the last version of matchtigs using mamba.
Running matchtigs I obtain

$ matchtigs
--debug-print-graphtrue
'?'

That may be is expected but I cannot run --help or --version

$ matchtigs --help
assigning values to '' wasn't recognized' which wasn't expected, or isn't valid

$ matchtigs --version
Found argument 'version
matchtigs'

What I am doing wrong?
Thanks.
In case it helps

$ uname -a
Linux xxxxxxxxxxxx 4.18.0-492.el8.x86_64 #1 SMP Tue May 9 17:56:55 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Great! conda update -c conda-forge -c bioconda matchtigs fixed my problem.
Thanks!


Relax ratelimit of MangaSee

Source name
MangaSee
Source language
English
Describe your suggested feature
The loading of thumbnails in this extension is very slow (1 request in 2 seconds). Maybe it could be relaxed to 2 requests per second for the whole extension or just thumbnails? (although doing so for just thumbnails would require making changes to the Mihon source code...).
Other details
I have already tried relaxing it per source code and there do not appear to be any immidiate problems with cloudflare...
Acknowledgements

[X] I have searched the existing issues and this is a new ticket, NOT a duplicate or related to another open or closed issue.
[X] I have written a short but informative title.
[X] If this is an issue with the app itself, I should be opening an issue in the app repository.
[X] I have updated the app to version 0.15.3.
[X] I will fill out all of the requested information in this form.

Iirc the ratelimit was put in by tachi extension maintainers since MangaSee admin asked them to (since it was causing too much load on their site), otherwise they would have started blocking/breaking the extension.

I guess relaxing the ratelimit for just thumbnails (since they are small) might also lead to problems with the mangasee admin then.


Fix application logs if posible

2021-08-24T05:11:33.660977+00:00 app[worker.1]: 2021-08-24 05:11:33,660: except
2021-08-24T05:11:34.237663+00:00 app[worker.1]: 2021-08-24 05:11:34,237: except
2021-08-24T05:11:34.485379+00:00 app[worker.1]: 2021-08-24 05:11:34,485: except
2021-08-24T05:11:34.735432+00:00 app[worker.1]: 2021-08-24 05:11:34,735: except
2021-08-24T05:11:34.991372+00:00 app[worker.1]: 2021-08-24 05:11:34,991: except
2021-08-24T05:11:35.562105+00:00 app[worker.1]: 2021-08-24 05:11:35,561: except
2021-08-24T05:11:35.820503+00:00 app[worker.1]: 2021-08-24 05:11:35,820: except
2021-08-24T05:11:36.078763+00:00 app[worker.1]: 2021-08-24 05:11:36,078: except
2021-08-24T05:11:36.340188+00:00 app[worker.1]: 2021-08-24 05:11:36,340: except
2021-08-24T05:12:36.502672+00:00 app[worker.1]: 2021-08-24 05:12:36,502: except

I'm sorry for this, i'll soon fix it. Excepts getting spammed in the log means there is a network connection error between you and heroku or heroku and lichess.

Fixed. The new logs are more readable

Thank you!


Use standard list styling

Lists use this weird borderless style:

It would be better to use standard .content .boxed-list style class on these.

It would be better to use standard .content style class on these.

Of course, it's not that simple :(

Getting somewhere for the album details:

It's slightly broken though, probably due to it being a ListView and not a ListBox:

.content is now explicitly unsupported for ListViews: Misc. Boxed List Tweaks

Oh :( too bad this is not supported. I hope an alternative will be provided in the future for ListViews :/

Maybe there's a way to emulate that style with set_show_separators

My own attempts did not amount to much, I think it might be better to wait a little until adwaita stabilizes

This kinda almost works with show-separators and .card style (which is blocked by #358)

In this case, I think listbox is the proper widget. Generally we also don't have scrolling within cards unless it's something like a textview.

These used to be listboxes, but we have to manage potentially very large lists -- not so much with albums, granted, but playlists for instance!


Signin fails in certain devices (Mainly Nexus and Motorola) when user changes Langauge option in System setting from English to French or Spanish

Hi,
Sign-in fails when user changes language option in settings from English to other (Spanish or French) it throws error and never success .

Do you have any logs you can provide here?  And is this reproducible?


depend_on_referenced_packages on file_selector

I guess this warning shouldn't be happening in file_selector Flutter package, but I can't give more info on why it is happening or how to fix.

Ops. It was my fault.


[HDRP] Underwater draws through buildings

Describe the bug
Discord user Puppetmaster reported an issue where the underwater effect appears through a building:

He suggests it may have been introduced in Crest HDRP 4.1.
It might be that the surface is being culled by unity using dynamic occlusion culling. These tiles wont be added to the visible list, and therefore won't be registered to draw into the ocean mask. However I could not reproduce this effect locally.
Actually, maybe this is due to multiple water levels. I've enquired.
Versions
Unity 2019.3.9
Crest HDRP 4.1
To Reproduce
How can the developers reproduce this bug? Are you able to reproduce the issue in the example content/scenes?
Please include steps to reproduce the behaviour, with the example content ideally so we can follow the steps on our side.
Platform

Editor or standalone?
Platform (win/mac/...)

Hardware
What hardware are you seeing the issue on? GPU model is particularly useful.
Additional context
Add any other context about the problem here.

My specs:

Win 10 pro
2x Nvidia GTX 2080ti
Unity 2019.3.9f, HDRP 7.3.1
Crest 4.1 (from the Package Manager in Unity)

The Problem does only occur when I use Occlusion baking in Unity with those settings:
Smallest Occluder: 1
Smallest Hole: 0.1
Backface Threshold: 75
As soon as I clear the baked occlusion, the problem is also gone.  I made video for you.
2020-04-22_12-14-07.zip

This is same case like ours:
https://github.com/crest-ocean/crest/issues/456
And problem is not only with occlusion culling.
Small line without occlusion culling:

Big line with occlusion culling:

The underwater mask depends on a contiguous ocean surface being rendered. We've been unable to reproduce these issues ourselves. If anyone is up for it, a RenderDoc capture could be really helfpul here. If you don't know what that is I'm more than happy to explain and walk people through the steps.

After messing around i found a repro here - at runtime disable one of ocean tiles in the mid or background by setting the gameobject inactive, then put the camera looking through the island in main.unity (close-ish to the water so underwater is active):

@moosichu here is my RenderDoc capture with simplest case - cube and crest:
https://drive.google.com/open?id=1pVeoXQmxYFJBasGQ5qyrm2nKmM3-SThM
screen:

We are working on a solution to both problems. This is seprate from the horizon line issue, but both are being worked on :)

I thought this was fixed but managed to repro it, have a fix incoming.

4.3 is out with a fix for this. Please reopen if you still experience this issue

4.3 is out with a fix for this. Please reopen if you still experience this issue

is it available in the asset store?

@originalemanuel Yes