Ionic v5. iOS platform. Could not signal service com.apple.WebKit.WebContent: 113.

Good day ! Could anyone suggest please .. Ioinc 5 app crashed on iOS platform when I am downloaded a lot of files from server by angular HttpRequest ... The mobile app is working fine on Android 9,10.
Error...
2021-02-01 17:30:05.904879+0200 Parts[1305:358229] Could not signal service com.apple.WebKit.WebContent: 113: Could not find specified service
Installed platforms ios 6.1.1,
add to config.xml preference name="WKWebViewOnly" value="true"
Tested on  iOS version ipad 12.5.1 and iphone 14.4
ionic info:
Ionic:

   Ionic CLI                     : 5.4.16 (/usr/local/lib/node_modules/ionic)
   Ionic Framework               : @ionic/angular 5.5.2
   @angular-devkit/build-angular : 0.1000.8
   @angular-devkit/schematics    : 10.0.8
   @angular/cli                  : 10.0.8
   @ionic/angular-toolkit        : 2.3.3

Cordova:

   Cordova CLI       : 10.0.0
   Cordova Platforms : none
   Cordova Plugins   : no whitelisted plugins (0 plugins total)

Utility:

   cordova-res : not installed
   native-run  : not installed

System:

   ios-deploy : 1.11.3
   ios-sim    : 8.0.2
   NodeJS     : v14.15.4 (/usr/local/bin/node)
   npm        : 6.14.10
   OS         : macOS Big Sur
   Xcode      : Xcode 12.4 Build version 12D4e

Plugin list:
com-sarriaroman-photoviewer 1.2.4 "PhotoViewer"
cordova-clipboard 1.3.0 "Clipboard"
cordova-plugin-advanced-http 3.1.0 "Advanced HTTP plugin"
cordova-plugin-android-permissions 1.1.2 "Permissions"
cordova-plugin-app-version 0.1.12 "AppVersion"
cordova-plugin-background-mode 0.7.3 "BackgroundMode"
cordova-plugin-badge 0.8.8 "Badge"
cordova-plugin-compat 1.2.0 "Compat"
cordova-plugin-device 2.0.2 "Device"
cordova-plugin-file-opener2 3.0.5 "File Opener2"
cordova-plugin-file-transfer 2.0.0-dev "File Transfer"
cordova-plugin-file 6.0.2 "File"
cordova-plugin-inappbrowser 4.1.0 "InAppBrowser"
cordova-plugin-ionic-keyboard 2.2.0 "cordova-plugin-ionic-keyboard"
cordova-plugin-ionic-webview 4.2.1 "cordova-plugin-ionic-webview"
cordova-plugin-local-notification 0.9.0-beta.2 "LocalNotification"
cordova-plugin-ms-adal 0.8.1 "ADAL for Cordova"
cordova-plugin-nativestorage 2.3.2 "NativeStorage"
cordova-plugin-network-information 2.0.2 "Network Information"
cordova-plugin-printer 0.8.0 "Printer"
cordova-plugin-screen-orientation 3.0.2 "Screen Orientation"
cordova-plugin-splashscreen 5.0.2 "Splashscreen"
cordova-plugin-statusbar 2.4.2 "StatusBar"
cordova-plugin-vibration 3.1.1 "Vibration"
cordova-plugin-whitelist 1.3.3 "Whitelist"
cordova-plugin-wkwebview-engine 1.2.2-dev "Cordova WKWebView Engine"
cordova-plugin-wkwebviewxhrfix 1.0.0 "Cordova WKWebView File XHR Fix"
cordova-plugin-zip 3.1.0 "cordova-plugin-zip"
cordova-sqlite-storage 5.1.1-dev "Cordova sqlite storage plugin - cordova-sqlite-storage plugin version"
cordova.plugins.diagnostic 6.0.3 "Diagnostic"

+1
I have exactly the same issue here --
2021-02-11 13:48:08.716786+0100 App[16585:5285968] Could not signal service com.apple.WebKit.WebContent: 113: Could not find specified service
⚡  [error] - [WDS] Disconnected!

It seems I solved this problem ...

installed  cordova plugin add cordova-plugin-ionic-webview@latest
installed<EMAIL_ADDRESS>

I tried almost everything, my solution was simple, updating MacOS to the latest version, and also Xcode to the latest version, that way the error was gone, white blank screen was not happening anymore.


adding identityserver4 ref

fixes #1849

@danroth27

@brockallen @leastprivilege

:watch:

We'll have one more link by tomorrow for EF, but I can send a PR once it's done.

Should we consider just linking off directly to https://identityserver4.readthedocs.io/ instead of trying to list all of the individual quickstarts? We could make the link under security -> authentication a bit more generic: Security ASP.NET Core apps with identityServer4 or something like that.

I'll defer to @leastprivilege on that -- he'll have a stronger opinion than me.

I definitely like the broader title using "ASP.NET Core Applications" - maybe list the fundamental quickstarts (so people know what to expect) and also do a link to the our docs main page.

Which quickstarts would you consider fundamental?
I still lean towards just linking directly from security -> authentication to the IdentityServer4 docs. That way we don't have to keep our links in sync with yours and you have full control over how folks get introduced to your framework.

OK - fine with me.

So I think I am reading that the decision is to link the TOC item directly to  https://identityserver4.readthedocs.io/ instead of having a identityserver/index correct?
@danroth27

Yeah, we can just add a TOC entry to security/authentication/index.rst:
Securing ASP.NET Core apps with identityServer4 <https://identityserver4.readthedocs.io>

IdentityServer4

ok, check it.

:shipit:


Json indexing

sample queries
select person from super where JSON_MATCH(person, "addresses.street='street-1' AND name='adam-1'") limit 10

select count(*) from super where JSON_MATCH(person, "name != 'adam-1'")

select json_extract_scalar(person, '$.name', 'STRING') from super where JSON_MATCH(person, "name = 'adam-1'") limit 10

Description
submitting PR to get early feedback. This adds JSON_MATCH filter operator that makes querying json field easier and efficient.
Upgrade Notes
Does this PR prevent a zero down-time upgrade? (Assume upgrade order: Controller, Broker, Server, Minion)

[ ] Yes (Please label as backward-incompat, and complete the section below on Release Notes)

Does this PR fix a zero-downtime upgrade introduced earlier?

[ ] Yes (Please label this as backward-incompat, and complete the section below on Release Notes)

Does this PR otherwise need attention when creating release notes? Things to consider:

New configuration options
Deprecation of configurations
Signature changes to public methods/interfaces
New plugins added or old plugins removed

[ ] Yes (Please label this PR as release-notes and complete the section on Release Notes)

Release Notes
If you have tagged this as either backward-incompat or release-notes,
you MUST add text here that you would like to see appear in release notes of the
next release.
If you have a series of commits adding or enabling a feature, then
add this section only in final commit that marks the feature completed.
Refer to earlier release notes to see examples of text
Documentation
If you have introduced a new feature or configuration, please add it to the documentation as well.
See https://docs.pinot.apache.org/developers/developers-and-contributors/update-document

@yupeng9 @mayankshriv Cleaned up the PR, added the tests and a JsonIndexQuickStart on the github events data. Please take another look

Codecov Report

Merging #6216 (c0e5f33) into master (1beaab5) will decrease coverage by 21.41%.
The diff coverage is 51.88%.

@@             Coverage Diff             @@
##           master    #6216       +/-   ##
===========================================
- Coverage   66.44%   45.03%   -21.42%
===========================================
  Files        1075     1300      +225
  Lines       54773    62690     +7917
  Branches     8168     9105      +937
===========================================
- Hits        36396    28231     -8165
- Misses      15700    32130    +16430
+ Partials     2677     2329      -348

Flag
Coverage Δ

integration
45.03% <51.88%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

...ot/broker/broker/AllowAllAccessControlFactory.java
100.00% <ø> (ø)

.../helix/BrokerUserDefinedMessageHandlerFactory.java
52.83% <0.00%> (-13.84%)
:arrow_down:

...org/apache/pinot/broker/queryquota/HitCounter.java
0.00% <0.00%> (-100.00%)
:arrow_down:

...che/pinot/broker/queryquota/MaxHitRateTracker.java
0.00% <0.00%> (ø)

...ache/pinot/broker/queryquota/QueryQuotaEntity.java
0.00% <0.00%> (-50.00%)
:arrow_down:

...ker/routing/instanceselector/InstanceSelector.java
100.00% <ø> (ø)

...ceselector/StrictReplicaGroupInstanceSelector.java
0.00% <0.00%> (ø)

.../main/java/org/apache/pinot/client/Connection.java
22.22% <0.00%> (-26.62%)
:arrow_down:

...not/common/assignment/InstancePartitionsUtils.java
64.28% <ø> (-8.89%)
:arrow_down:

...common/config/tuner/NoOpTableTableConfigTuner.java
0.00% <ø> (ø)

... and 1258 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f2c37d5...c0e5f33. Read the comment docs.


Migrate loggly article

What changed, and why it matters
Updated wording, steps and removed images
Addresses #1088

Hi @encima ! I shuffled a bit the content:

Separated sections for endpoint creation and enablement
CLI examples redirecting to the cli docs (where I added the example for Loggly)
Could you please check everything still makes sense?

Hi @encima ! I shuffled a bit the content:

Separated sections for endpoint creation and enablement
CLI examples redirecting to the cli docs (where I added the example for Loggly)
Could you please check everything still makes sense?

Oooh, didn't know about Tip! Thanks, LGTM


about re_rank code

If your re_rank does not add the source domain, it is different from the re_rank version of CVPR2017 paper: Zhong Z. What you have achieved is a search in a distance matrix k-reciprocal nearest neighbors and then re_rank?

Does it mean that each picture is both a probe and a gallery?


Suggestion: Reserve glTF extension prefix

glTF documentation says:

To request a prefix, submit a GitHub issue with the name of the requested prefix and the vendor that will be using it. Maintainers, please assign the extension label to the issue.

https://github.com/KhronosGroup/glTF/commit/24e2a277dffb1f916136b8a54af440090e8fcc39
So why not submit an issue for VRM?

According to glTF Extension Registry,

Vendor extension names start with the prefix followed by an underscore, e.g., CESIUM_.

So VRM is not a valid extension name. The extension name should have been DWANGO_VRM or something (and register the DWANGO prefix). Or extras should have been used instead of an unregistered extension name.


Support running on non-local envrionment

Hi,
Thanks for maintaining this convience plugin. It would be great if I can configure the run configuration to let a particular kotest run in docker or WSL2, similar to how Junit test in Intellij can be ran in docker or WSL2 as shown below

Is this an intellij screenshot ?
On Tue, 17 May 2022 at 23:14, CXwudi @.***> wrote:

Hi,
Thanks for maintaining this convience plugin. It would be great if I can
configure the run configuration to let a particular kotest run in docker or
WSL2, similar to how Junit test in Intellij can be ran in docker or WSL2 as
shown below
[image: image]
https://user-images.githubusercontent.com/17377423/168955518-75ab4a5b-5113-48ea-8ff1-73bd4f24ed91.png
—
Reply to this email directly, view it on GitHub
https://github.com/kotest/kotest-intellij-plugin/issues/170, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAFVSGVVZIRL3QZ6AHINWRTVKRVBTANCNFSM5WHAVFOQ
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

Yes, I think this functionality appears since a 2021 version intellij

I think this is the same cause, so I didn't create a separate issue
But when I open a project stored in WSL2 and run a test by clicking the run button. it show that kotest can not run as it is not a Win32 process

On the other hand, if I right-click a folder and click run test. It works fine (but I guess it works because running test from folder let IntelliJ tries to run through JUnit 5, whereas the run button that the plugin creates is actually running the kotest through the plugin)


Start the app silent without taskbar icon

Hi, is there a way to start the application (with or without AHK) in a silent mode?
I'm fine with a tray icon, but the taskbar window (even if minimized) always kicks me out of my games when I start the application via a hotkey command.
Appreciate an answer!

Hi, I see your problem, I will look into it.

Check the 3.15 version, it should not open the window in --minimized.


Universal listings drilldown component

Moves the universal listings to a new "dropdown + drilldown" component. The component combines the generic "dropdown" with a custom theme (#11456), with a new bespoke "drilldown" component. That component could be made more generic (separate the drilldown menu interface from the filters logic as two controllers), but for now it’s just the one.
Aside from the drilldown UI, this component also manages the browser URL based on events from SwapController instances.
There are a few things that are missing from the component and will hopefully make it into the next release:

Better integration with SwapController to have:

Error handling
Loading state (currently the search form shows one but that’s not appropriate)
Resetting the form fields based on external active filters interactions

Integration with dropdown to:

Allow clicking of active filters to go straight to a specific panel within the drilldown
Resetting the drilldown when it’s "click outside" or "escape key" closed

UI

Left/right animation when switching panels
Counts within the filtering panel rather than top toggle only

Code quality

Unit tests for controller
Extract markup to separate template

Please check the following:

[x] Do the tests still pass?[^1]
[x] Does the code comply with the style guide?
[ ] For Python changes: Have you added tests to cover the new/fixed behaviour?
[ ] For front-end changes: Did you test on all of Wagtail’s supported environments?[^2]

[x] Please list the exact browser and operating system versions you tested: Chrome 122, Safari 17, Firefox 121 macOS 14
[x] Please list which assistive technologies [^3] you tested: forced colors mode, VoiceOver macOS

[ ] For new features: Has the documentation been updated accordingly?

To test this,

Go to the listings UI with and without filters predefined in the URL
Use keyboard navigation to interact with the component
Observe the interaction with active filters

I am happy to create a follow up PR (once this is merged) with a baseline set of unit tests if that helps.


DataFrame subclass type lost when assigning with unknown divisions

Describe the issue:
When assigning a new column to a dask object, it seems like the concrete subtype (e.g. geopandas.GeoDataFrame) is lost.
Minimal Complete Verifiable Example:
import dask.array
import dask.dataframe
import dask_geopandas
import geopandas
import pandas as pd

df = geopandas.GeoDataFrame({"geometry": geopandas.points_from_xy([0, 0], [0, 1])})
ddf = dask_geopandas.from_geopandas(df, npartitions=2)
ddf = ddf.clear_divisions()  # this is important

b = dask.dataframe.from_dask_array(dask.array.zeros((2,), chunks=(1, 1)), index=ddf.index)
ddf.assign(a=b).geometry.x.compute()  ## error

that raises
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
[/var/folders/x7/__bs9yvx21qbvzb17sj4qsh40000gn/T/ipykernel_95282/3433075730.py](http://<IP_ADDRESS>:8888/var/folders/x7/__bs9yvx21qbvzb17sj4qsh40000gn/T/ipykernel_95282/3433075730.py) in ?()
      8 ddf = dask_geopandas.from_geopandas(df, npartitions=2)
      9 ddf = ddf.clear_divisions()  # this is important
     10
     11 b = dask.dataframe.from_dask_array(dask.array.zeros((2,), chunks=(1, 1)), index=ddf.index)
---> 12 ddf.assign(a=b).geometry.x.compute()  ## error

[~/gh/TomAugspurger/dask-geopandas-spatial-partitioning/.direnv/python-3.12/lib/python3.12/site-packages/dask_expr/_collection.py](http://<IP_ADDRESS>:8888/lab/tree/~/gh/TomAugspurger/dask-geopandas-spatial-partitioning/.direnv/python-3.12/lib/python3.12/site-packages/dask_expr/_collection.py) in ?(self, fuse, concatenate, **kwargs)
    476         out = self
    477         if not isinstance(out, Scalar) and concatenate:
    478             out = out.repartition(npartitions=1)
    479         out = out.optimize(fuse=fuse)
--> 480         return DaskMethodsMixin.compute(out, **kwargs)

[~/gh/TomAugspurger/dask-geopandas-spatial-partitioning/.direnv/python-3.12/lib/python3.12/site-packages/dask/base.py](http://<IP_ADDRESS>:8888/lab/tree/~/gh/TomAugspurger/dask-geopandas-spatial-partitioning/.direnv/python-3.12/lib/python3.12/site-packages/dask/base.py) in ?(self, **kwargs)
    368         See Also
    369         --------
    370         dask.compute
    371         """
--> 372         (result,) = compute(self, traverse=False, **kwargs)
    373         return result

[~/gh/TomAugspurger/dask-geopandas-spatial-partitioning/.direnv/python-3.12/lib/python3.12/site-packages/dask/base.py](http://<IP_ADDRESS>:8888/lab/tree/~/gh/TomAugspurger/dask-geopandas-spatial-partitioning/.direnv/python-3.12/lib/python3.12/site-packages/dask/base.py) in ?(traverse, optimize_graph, scheduler, get, *args, **kwargs)
    656         keys.append(x.__dask_keys__())
    657         postcomputes.append(x.__dask_postcompute__())
    658
    659     with shorten_traceback():
--> 660         results = schedule(dsk, keys, **kwargs)
    661
    662     return repack([f(r, *a) for r, (f, a) in zip(results, postcomputes)])

[~/gh/TomAugspurger/dask-geopandas-spatial-partitioning/.direnv/python-3.12/lib/python3.12/site-packages/pandas/core/generic.py](http://<IP_ADDRESS>:8888/lab/tree/~/gh/TomAugspurger/dask-geopandas-spatial-partitioning/.direnv/python-3.12/lib/python3.12/site-packages/pandas/core/generic.py) in ?(self, name)
   6295             and name not in self._accessors
   6296             and self._info_axis._can_hold_identifiers_and_holds_name(name)
   6297         ):
   6298             return self[name]
-> 6299         return object.__getattribute__(self, name)

AttributeError: 'Series' object has no attribute 'x'

geopandas.GeoSeries objects automatically add .x and .y to the geometry columns. We're getting a regular pandas.Series, causing the error.
Anything else we need to know?:
Having unknown divisions does seem to be necessary. Commenting out the def = ddf.clear_divisions()  line makes the error go away. So I think we can maybe narrow the search to AssignAlign (and not Assign)
Environment:

Dask version: 2024.12.0

dask-expr from main @ d7577a2ff22d47b36a45a87dd5ff2857af81341f

Python version:

Operating System:

Install method (conda, pip, source):

The Align case triggers a shuffle under the hood which is responsible for losing the dtype.

Ah, then this might be the same root cause as https://github.com/dask/dask-expr/issues/1024.

Do you know if this used to work with the legacy backend?

It seems to:

In [1]: import dask.config
   ...:
   ...: dask.config.set(**{"dataframe.query-planning": False})
   ...:
   ...: import dask.array
   ...: import dask.dataframe
   ...: import dask_geopandas
   ...: import geopandas
   ...: import pandas as pd
   ...:
   ...: df = geopandas.GeoDataFrame({"geometry": geopandas.points_from_xy([0, 0], [0, 1])})
   ...: ddf = dask_geopandas.from_geopandas(df, npartitions=2)
   ...: # ddf = ddf.clear_divisions()  # this is important
   ...:
   ...: ddf.assign(a=1).geometry.x.compute()  # OK
   ...:
   ...: ddf.assign(a=dask.dataframe.from_dask_array(dask.array.zeros((2,), chunks=(1, 1)), index=ddf.index)).geometry.x.compute()  ## error
   ...:
/Users/tom/gh/TomAugspurger/dask-geopandas-spatial-partitioning/.direnv/python-3.12/lib/python3.12/site-packages/dask/dataframe/__init__.py:31: FutureWarning: The legacy Dask DataFrame implementation is deprecated and will be removed in a future version. Set the configuration option `dataframe.query-planning` to `True` or None to enable the new Dask Dataframe implementation and silence this warning.
  warnings.warn(
Out[1]:
0    0.0
1    0.0
dtype: float64

Ah no, that is not equivalent. The old implementation just assumes that the indices match if divisions are equal (even if they are all None), so there wasn't. shuffle triggered but you could get incorrect results with unknow divisions but different indexes.
You'd have to check
ddf.shuffle(on=ddf.columns[0]).compute()

Gotcha, that makes sense. In this case I happen to know that the indexes are identical so it would be safe to assume that the all-None divisions are OK, but I see why doing a shuffle would be needed in the general case.
Doing the shuffle does indeed lose the subclass, with or without query planning.

Yeah the assign case is interesting. We are generally checking if both dataframes originate from the same root and have the same index to avoid the shuffle. Your DataFrame clearly has a different root than the assign series, but we could definitely only check the index here. That is enough information.
The from Dask array cases is an edge case though (I think), generally the index will be different probably

Something like:
import dask_geopandas
import geopandas

df = geopandas.GeoDataFrame({"geometry": geopandas.points_from_xy([0, 0], [0, 1])})
ddf = dask_geopandas.from_geopandas(df, npartitions=2)
ddf = ddf.clear_divisions()  # this is important

ddf = ddf.assign(a=1)
ddf.assign(b=ddf.a).geometry.x.compute()  ## error

We are looking at the expression structure and can identify that left and right both have a compatible index, falling back to a trivial blockwise op.


Ability to see the attachments in "View Submissions" map view popup

I would like to see the option to config the visibility of the attachments in the "View Submissions" map view.  The user of the app can add the attachments just fine.  However when they look at the submissions in the "View Submissions" map view, there is no option to look at the attachments.  The following link is the app we are planning to use at an upcoming hackathon... http://arcg.is/1nEnWcy

I would love to see this added as well.

Here is my form: http://arcg.is/1TuGnxp

Will fix this in 4x version


Converted VG to  hierarchical

Checklist for adding MMTEB dataset
Reason for dataset addition:
VG Clustering was not hierarchical nor ClusteringFast before #656

[x] I have tested that the dataset runs with the mteb package.
[ ] I have run the following models on the task (adding the results to the pr). These can be run using the mteb run -m {model_name} -t {task_name} command.

[x] sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
[ ] intfloat/multilingual-e5-small

[ ] I have checked that the performance is neither trivial (both models gain close to perfect scores) nor random (both models gain close to random scores).
[x] If the dataset is too big (e.g. >2048 examples), considering using self.stratified_subsampling() under dataset_transform()
[x] I have filled out the metadata object in the dataset file (find documentation on it here).
[x] Run tests locally to make sure nothing is broken using make test.
[x] Run the formatter to format the code using make lint.
[ ] I have added points for my submission to the points folder using the PR number as the filename (e.g. 438.jsonl).

I can't run it for E5 as Ucloud is down today and it would take hours on my computer :')

Also added stratified subsampling code to AbsTask for multilabel problems, as this was missing.

Since the stratified subsampling doesn't exactly work as expected with multilabel data, I will just go with a random sample of 2048 entries I think.

Tests are failing because of missing datasets unrelated to my PR, what to do?


BottomNavbar has boolean for visibility

Description
Motivation
Testing Guidelines
Release Checklist

[ ] This code has unit tests
[ ] I have a plan to verify that these changes are working as expected after deploy
[ ] I have a plan for how to revert, rollback, or disable these changes if things are not working as expected

Security Checklist
This PR creates, modifies, or deletes:

[ ] API routes: API routes, parameters, or user authorization
[ ] Authentication: Authentication mechanism
[ ] Credentials: Server side credentials, or secrets in configuration / source code
[ ] Cryptography: Encryption, hashing, certificates, signatures, random numbers, etc.
[ ] User data: personal information handling, logs, error messages, etc.

Additional Notes

this was at @sabind 's request.
he wanted the bottom buttons to be hidden when we are looking at notifications.
even if this isn't wanted, i think putting a visibility boolean on the bottom buttons is a good idea and it is easy to implement

this was at @sabind 's request.
he wanted the bottom buttons to be hidden when we are looking at notifications.
even if this isn't wanted, i think putting a visibility boolean on the bottom buttons is a good idea and it is easy to implement

I think a better approach to achieving what @sabind wanted was to modify the notifications drawer component to fill the whole page like the sidebar.
The reason why I don't think it's a good idea to have a visibility attribute in the bottom panel is explained here: https://github.com/reach4help/reach4help/pull/735#pullrequestreview-430639038

this was at @sabind 's request.
he wanted the bottom buttons to be hidden when we are looking at notifications.
even if this isn't wanted, i think putting a visibility boolean on the bottom buttons is a good idea and it is easy to implement

If you open the "Notifications" you'll see a really weird issue with the bottom nav click handlers and the notifications side staying open. I mentioned it should cover the whole screen and it actually seems this will be very helpful to enhance the bottom bar with this (and solve our problem)

this was at @sabind 's request.
he wanted the bottom buttons to be hidden when we are looking at notifications.
even if this isn't wanted, i think putting a visibility boolean on the bottom buttons is a good idea and it is easy to implement

If you open the "Notifications" you'll see a really weird issue with the bottom nav click handlers and the notifications side staying open. I mentioned it should cover the whole screen and it actually seems this will be very helpful to enhance the bottom bar with this (and solve our problem)

well, my idea was to modify the notification drawer.
but this works if it works.
but @mcverter discuss this with @btuerker then.
one thing both of you have to be mindful of is how the MasterPage.ts decides which layout to use based on the definition inside the module. If this solution adds any value to that implementation then I think this is good to go

@s

this was at @sabind 's request.
he wanted the bottom buttons to be hidden when we are looking at notifications.
even if this isn't wanted, i think putting a visibility boolean on the bottom buttons is a good idea and it is easy to implement

If you open the "Notifications" you'll see a really weird issue with the bottom nav click handlers and the notifications side staying open. I mentioned it should cover the whole screen and it actually seems this will be very helpful to enhance the bottom bar with this (and solve our problem)

well, my idea was to modify the notification drawer.
but this works if it works.
but @mcverter discuss this with @btuerker then.
one thing both of you have to be mindful of is how the MasterPage.ts decides which layout to use based on the definition inside the module. If this solution adds any value to that implementation then I think this is good to go

Before this change, the BottomButtons were always open, but the MenuDrawer was covering them.  You could still see them in the DOM but they were beneath other elements.
For a container page like MasterPage.tsx, it is often good to set booleans on elements that can appear or disappear in different context.  By doing this, burhan can now use the entirety of the page for the notifications.

one thing both of you have to be mindful of is how the MasterPage.ts decides which layout to use based on the definition inside the module.

This sounds like "Dependency Inversion"
https://en.wikipedia.org/wiki/Dependency_inversion_principle
A higher container element like MasterPage.ts should never depend on its children to determine its UI

one thing both of you have to be mindful of is how the MasterPage.ts decides which layout to use based on the definition inside the module.

This sounds like "Dependency Inversion"
https://en.wikipedia.org/wiki/Dependency_inversion_principle
A higher container element like MasterPage.ts should never depend on its children to determine its UI

yea, but if we do this and make sense of this change for the future, it has to depend on the children.
but with the current implementation, it's not like that. well. I guess we can discuss this on the daily stand up call.
also, this isn't a big issue. you may have issues making sense of the code later in the future and someone might blame the codebase. But it wouldn't break the project in any way

one thing both of you have to be mindful of is how the MasterPage.ts decides which layout to use based on the definition inside the module.

This sounds like "Dependency Inversion"
https://en.wikipedia.org/wiki/Dependency_inversion_principle
A higher container element like MasterPage.ts should never depend on its children to determine its UI

yea, but if we do this and make sense of this change for the future, it has to depend on the children.
but with the current implementation, it's not like that. well. I guess we can discuss this on the daily stand up call.
also, this isn't a big issue. you may have issues making sense of the code later in the future and someone might blame the codebase. But it wouldn't break the project in any way

Why does it have to "depend on the children"?  This is an antipattern and will make the code harder to maintain.  We need to discuss these changes before anyone goes down this path

yup

one thing both of you have to be mindful of is how the MasterPage.ts decides which layout to use based on the definition inside the module.

This sounds like "Dependency Inversion"
https://en.wikipedia.org/wiki/Dependency_inversion_principle
A higher container element like MasterPage.ts should never depend on its children to determine its UI

yea, but if we do this and make sense of this change for the future, it has to depend on the children.
but with the current implementation, it's not like that. well. I guess we can discuss this on the daily stand up call.
also, this isn't a big issue. you may have issues making sense of the code later in the future and someone might blame the codebase. But it wouldn't break the project in any way

Why does it have to "depend on the children"? This is an antipattern and will make the code harder to maintain. We need to discuss these changes before anyone goes down this path

yes, you are right. what I am saying is that this change could lead to a situation like that in the future.
but in order to understand my concern fully, I think I have to explain the MaterPage.ts to you and also the layouts and I think the frontend pod/standup call would be the perfect time for me to do this

z


Final set of changes for DVM retry

Description
This PR includes some CRD changes (all fields are optional) for DVM retry. It also adds a new option in controller-config.yml to override default backoff limit in case users need to.
Check each of the following when appropriate to help reviewers verify work is complete.
Modifying an existing version

[ ] I modified operator permissions in the OLM CSV and operator.yml
[ ] I modified the operator deployment in the OLM CSV and operator.yml
[ ] I modified operand permissions in the OLM CSV and ansible role
[x] I modified CRDS in the OLM CSV and ansible role

Adding a new release version

[ ] I created a new z release directory in deploy/olm-catalog/crane-operator
[ ] I updated channels in the crane-operator.package.yaml
[ ] I created a new release directory in deploy/non-olm
[ ] I created or updated the major.minor link in deploy/non-olm
[ ] I updated the spec.skips parameter in the CSV

Related controller PR https://github.com/konveyor/mig-controller/pull/1017


Fix(UI) #8555: Tags removal issue

Describe your changes :

I worked on the following:

[x] Fixed Error, thrown while removing tags from the data asset details page
[x] Fixed bug where users were not able to remove the tags on the Dashboard Details page
[x] Changes in MlModelFeaturesList component:

Tag removal bug fix
Removed tailwind use and replaced with antd
Code optimization
Localisation changes

Type of change :

[x] Bug fix
[x] Improvement

Frontend Preview (Screenshots) :

Table Details Page:
https://user-images.githubusercontent.com/51777795/204544976-f7efff38-59c8-43c7-8af0-753bdf51d463.mov
Dashboard Details Page:
https://user-images.githubusercontent.com/51777795/204545042-0fe2f87a-c499-470a-ab07-f13e067dd8fb.mov
Ml Model Details Page:
https://user-images.githubusercontent.com/51777795/204545049-b0f57287-f80f-4bef-ad7b-2c0e50339725.mov
SourceList component:
https://user-images.githubusercontent.com/51777795/204707749-cac0457c-ce65-498c-9693-edb456b15c52.mov

Checklist:

[x] I have read the CONTRIBUTING document.
[x] I have commented on my code, particularly in hard-to-understand areas.
[x] I have added tests that prove my fix is effective or that my feature works.
[x] All new and existing tests passed.

Reviewers

Frontend: @open-metadata/ui

@chirag-madlani I've pushed the suggested changes. Can you please review it?


beacon-api voluntary exit cleanup

The beacon-api voluntary exit is probably more restrictive than it should be.
We should ideally be able to post an exit for a future epoch and have it in the pool. I'm not sure what it would mean as far as pushing the change to the network, we may have to delay publishing it.
Because keymanager-api is able to generate signed exits, we really need to have the beacon-api post be a bit more capable in handling the kinds of flows that a user is likely to want to do.
It's fair to lose the pool entries on restart potentially, but ideally being able to accept a  voluntary exit early would improve UX.  We could ideally validate other rules and just store it for future use if its submitted early (not sure about network validation rules and whether we are allowed to publish early)

closely related,
./teku voluntary-exit --epoch=204930

Current epoch is 204735.  The client errors with:

The specified epoch 204930 is greater than current epoch 204734, cannot continue.

this is basically because the CLI tool is enforcing what the api will restrict doing, rather than submitting and failing. this should also be considered when we're looking at the api.


Crashing test: Channel__history__should_retrieve_all_available_messages

When run locally with make test_iOS:
[19:05:43]: ▸     ✗ Channel__history__should_retrieve_all_available_messages, expected to have Array<ARTMessage> with count 10, got 0
[19:05:43]: ▸     ✗ Channel__history__should_retrieve_all_available_messages, expected to be true, got <false>
[19:05:43]: ▸     ✗ Channel__history__should_retrieve_all_available_messages, expected to be false, got <true>
[19:05:57]: ▸ 2020-07-01 19:05:57.777 xcodebuild[32229:2231080]  IDETestOperationsObserverDebug: Writing diagnostic log for test session to:
[19:05:57]: ▸ /Users/quintin/Library/Developer/Xcode/DerivedData/Ably-brxvaqpefvodvefhlejpuifnswap/Logs/Test/Run-Ably-iOS-Tests-2020.07.01_19-02-12-+0100.xcresult/Staging/3_Test/Diagnostics/Ably-iOS-Tests-49E0C898-21F9-46EE-8D8A-307EB9D4372C/Ably-iOS-Tests-FAE9255B-5E18-42F5-8EFD-B9F1FC32281A/Session-Ably-iOS-Tests-2020-07-01_190557-Yq5fnt.log
[19:05:57]: ▸ 2020-07-01 19:05:57.777 xcodebuild[32229:2226833] [MT] IDETestOperationsObserverDebug: (780B80C7-5463-427C-8302-8E0374B90A92) Beginning test session Ably-iOS-Tests-780B80C7-5463-427C-8302-8E0374B90A92 at 2020-07-01 19:05:57.778 with Xcode 11C505 on target <DVTiPhoneSimulator: 0x7fbe1c979460> {
[19:05:57]: ▸ 		SimDevice: iPhone XS (ABD330A8-95DB-4B93-9A43-AE2384A250F9, iOS 12.0, Booted)
[19:05:57]: ▸ } (12.0 (16A366))
[19:05:57]: ▸ 2020-07-01 19:05:57.781 xcodebuild[32229:2226833] [MT] IDETestOperationsObserverDebug: (780B80C7-5463-427C-8302-8E0374B90A92) Finished requesting crash reports. Continuing with testing.
[19:05:58]: ▸     ✗ Channel__history__should_retrieve_all_available_messages, Test crashed

┆Issue is synchronized with this Jira Bug by Unito

Closing outdated issue. The codebase changed drastically since then and all skipped tests were unskipped, see PRs for https://github.com/ably/ably-cocoa/issues/1406 and https://github.com/ably/ably-cocoa/issues/1848
If you encounter frequently failing test again, please create a new issue with a best guess why it might be happening.


Can't install addon due to missing dependencies

Netflix add-on version
Addon not installed, see description
Operative systems used
LibreELEC
Kodi version used
Kodi 19 (Matrix)
Description of the bug
I just did a clean install of LibreELEC on my Raspberry Pi 3 due to unrelated issues, and then I enabled third-party sources for the plugins and installed your repo from the provided zip file, as usual. But then when trying to install the Netflix addon, the installation fails because it is not able to satisfy the dependencies. Any idea how I could fix this?
Thank you for your help,
GTP95
Steps to reproduce the behavior
No response
Debug log - mandatory
http://ix.io/4mx7
Possible fix
No response
Additional context
No response
Screenshots
No response

I just tried again and now it worked. No idea why, I did multiple attempts before and all failed... But now it got installed.

In addition to this, maybe it's worth looking into the fact that, even after the installation was successful, somehow inputsream. adaptive didn't get installed automatically and I had to manually install it.


runtime/debug: soft memory limit

Proposal: Soft memory limit
Author: Michael Knyszek
Summary
I propose a new option for tuning the behavior of the Go garbage collector by setting a soft memory limit on the total amount of memory that Go uses.
This option comes in two flavors: a new runtime/debug function called SetMemoryLimit and a GOMEMLIMIT environment variable. In sum, the runtime will try to maintain this memory limit by limiting the size of the heap, and by returning memory to the underlying platform more aggressively. This includes with a mechanism to help mitigate garbage collection death spirals. Finally, by setting GOGC=off, the Go runtime will always grow the heap to the full memory limit.
This new option gives applications better control over their resource economy. It empowers users to:

Better utilize the memory that they already have,
Confidently decrease their memory limits, knowing Go will respect them,
Avoid unsupported forms of garbage collection tuning.

Details
Full design document found here.
Note that, for the time being, this proposal intends to supersede #44309. Frankly, I haven't been able to find a significant use-case for it, as opposed to a soft memory limit overall. If you believe you have a real-world use-case for a memory target where a memory limit with GOGC=off would not solve the same problem, please do not hesitate to post on that issue, contact me on the gophers slack, or via email at<EMAIL_ADDRESS>Please include as much detail as you can.

Afaict, the impact of memory limit is visible once the GC is CPU throttled, but not before. Would it be worth exposing the current effective GOGC as well?

@mpx I think that's an interesting idea. If GOGC is not off, then you have a very clear sign of throttling in telemetry. However, if GOGC=off I think it's harder to tell, and it gets blurry once the runtime starts bumping up against the GC CPU utilization limit, i.e. what does effective GOGC mean when the runtime is letting itself exceed the heap goal?
I think that's pretty close. Ideally we would have just one metric that could show, at-a-glance, "are you in the red, and if so, how far?"

In case you find this useful as a reference and to include in "prior art", the go-watchdog library schedules GC according to a user-defined policy. It can infer limits from the environment/host, container, and it can target a maximum heap size defined by the user. I built this library to deal with https://github.com/golang/go/issues/42805, and ever since we integrated it into https://github.com/filecoin-project/lotus, we haven't had a single OOM reported.

@rsc I believe the design is complete. I've received feedback on the design, iterated on it, and I've arrived at a point where  there aren't any major remaining comments that need to be addressed. I think the big question at the center of this proposal is whether the API benefit is worth the cost. The implementation can change and improve over time; most of the details are internal.
Personally, I think the answer is yes. I've found that mechanisms that respects users' memory limits and that give the GC the flexibility to use more of the available memory are quite popular. Where Go users implement this themselves, they're left working with tools (like runtime.GC/debug.FreeOSMemory and heap ballasts) that have some significant pitfalls. The proposal also takes steps to mitigate the most significant costs of having a new GC tuning knob.
In terms of implementation, I have some of the foundational bits up for review now that I wish to land in 1.18 (I think they're uncontroversial improvements, mostly related to the scavenger). My next step is create a complete implementation and trial it on real workloads. I suspect that a complete implementation won't land in 1.18 at this point, which is fine. It'll give me time to work out any unexpected issues with the design in practice.

No change in consensus, so accepted. 🎉
This issue now tracks the work of implementing the proposal.
— rsc for the proposal review group

The core feature has landed, but I still need to land a few new metrics to help support visibility into this.

Hey @mknyszek - first of all, thanks for the excellent work; this is great.
I wanted to share our experience thinking about enabling this in production. It works great and exactly as advertised. Some well-maintained applications have enabled it with great success, and the usage is spreading organically.
We'd ideally want to enable it for everyone by default (a vast majority of our applications have plenty of memory available), but we're currently too afraid to do this. The reason is the death spirals you called in the proposal. Applications leaking memory, with GOMEMLIMIT, can get to a significantly degraded state. Paradoxically, those applications prefer to OOM, die quickly and be restarted than to struggle for a long time. The number of applications makes avoiding leaks unfeasible.
A part of the problem (perhaps) is that we lack a good enough way of setting the right limit. We cannot set it to 98-99% of the container memory because some other applications can be running there. But, if we set it to 90%, once we hit the death spiral situation, we're in a degraded state for too long - it can take hours for OOM, and in the meantime, we are at risk of all containers of an application entering the degraded state.
Another aspect is that our containers typically don't use close to all the available CPU time. So the assumption from the gc-guide, while true, has a slightly different result in practice:

The intuition behind the 50% GC CPU limit is based on the worst-case impact on a program with ample available memory. In the case of a misconfiguration of the memory limit, where it is set too low mistakenly, the program will slow down at most by 2x, because the GC can't take more than 50% of its CPU time away.

The GC might use at most 50% of the total CPU time, but it can end up using 2-3x more CPU than the actual application work. This is "GC degradation" would be hard to explain/sell to application owners.
We're also concerned with a "degradation on failover" situation - an application that might be okay usually, in case of a sudden increase in traffic, might end up in a death spiral. And this would be precisely the time we need to avoid those.
What we're doing now is:

most high-core applications have an internal GC tuner by @cdvr1993 described here. This work predates your work but is stable.
some applications are opting in for enabling GOMEMLIMIT independently.
we'd like to enable the GC tuning on more applications - ideally with GOMEMLIMIT to reduce the amount of custom code. Since we're afraid of the death spirals, though, we've discussed building a "lightweight" version of our tuner that would watch runtime stats (perhaps #56857) that would dynamically limit the GC usage more aggressively and let applications die faster.

Hope this is useful. Again, thanks for the excellent work.

Thanks for the detailed feedback and I'm glad it's working well for your overall!
Speaking broadly, I'd love to know more about what exactly this degraded state looks like. What is the downstream effect? Latency increase? Throughput decrease? Both? If you could obtain a GODEBUG=gctrace=1 (outputs to STDERR) of this degraded state, that would be helpful in identifying what if any next steps we should take.

We'd ideally want to enable it for everyone by default (a vast majority of our applications have plenty of memory available), but we're currently too afraid to do this. The reason is the death spirals you called in the proposal. Applications leaking memory, with GOMEMLIMIT, can get to a significantly degraded state. Paradoxically, those applications prefer to OOM, die quickly and be restarted than to struggle for a long time. The number of applications makes avoiding leaks unfeasible.

Choosing to die quickly over struggling for a long time is an intentional point in the design. In these difficult situations something has to give and we chose to make that memory.
But also if the scenario here is memory leaks, it's hard to do much about that without fixing the leak. The live heap will grow and eventually even without GOMEMLIMIT you'll OOM as well. GOMEMLIMIT isn't really designed to deal with a memory leak well (generally, we consider memory leaks to be a bug in long-running applications), and yeah I can see turning it on basically turning into "well, it just gets slower before it dies, and it takes longer to die," which may be worse than not setting a memory limit at all.
As for fixing memory leaks, we're currently planning some work on improving the heap analysis situation. I hope that'll make keeping applications leak-free more feasible in the future. (#57447)
(I recognize that encountering a memory leak bug at some point is inevitable, but in general we don't expect long-running applications to run under the expectation of memory leaks. I also get that it's a huge pain these days to debug them, but we're looking into trying to make that better with heap analysis.)

A part of the problem (perhaps) is that we lack a good enough way of setting the right limit. We cannot set it to 98-99% of the container memory because some other applications can be running there. But, if we set it to 90%, once we hit the death spiral situation, we're in a degraded state for too long - it can take hours for OOM, and in the meantime, we are at risk of all containers of an application entering the degraded state.

FTR that's what the runtime/debug.SetMemoryLimit API is for and it should be safe (performance-wise) to call with a relatively high frequency. Just to be clear, is this also the memory leak scenario?
The 90% case you're describing sounds like a misconfiguration to me; if the application's live heap is really close enough to the memory limit to achieve this kind of death spiral scenario, then the intended behavior is to die after a relatively short period, but it might not if it turns out there's actually plenty of available memory. However, this cotenant situation might not be ideal for the memory limit to begin with.
As a general rule, the memory limit, when used in conjunction with GOGC=off, is not a great fit for an environment where the Go program is potentially cotenant with others, and the others don't have predictable memory usage (or the Go application can't easily respond to cotenant changes). See https://go.dev/doc/gc-guide#Suggested_uses. In this case I'd suggest slightly overcommitting the memory limit to protect against many transient spikes in memory use (in your example here, maybe 95-96%), but set GOGC to something other than off.

The GC might use at most 50% of the total CPU time, but it can end up using 2-3x more CPU than the actual application work. This is "GC degradation" would be hard to explain/sell to application owners.

I'm not sure I follow. Are you describing a situation in which your application is using say, 25% CPU utilization, and the GC is eating up 50%?

We're also concerned with a "degradation on failover" situation - an application that might be okay usually, in case of a sudden increase in traffic, might end up in a death spiral. And this would be precisely the time we need to avoid those.

(Small pedantic note, but the 50% GC CPU limiter is a mechanism to cut off the death spiral; in general a death spiral means that the GC keeps taking on more and more of the CPU load until application progress stops entirely.)
I think it depends on the load you're expecting. It's always possible to construct a load that'll cause some form of degradation, even when you're not using the memory limit (something like a tight OOM loop as the service gets restarted would be what I would expect with just GOGC).
If the memory limit is failing to degrade gracefully, then that's certainly a problem and a bug on our side (perhaps even a design flaw somewhere!). (Perhaps this risk of setting a limit too low such that you sit in the degraded state for too long instead of actually falling over can be considered something like failing to degrade gracefully, and that suggests that even 50% GC CPU is trying too hard as a default. I can believe that but I'd like to acquire more data first.)
However, without more details about the scenario in question, I'm not sure what else we can do to alleviate the concern. One idea is a backpressure mechanism (#29696), but for now I think we've decided to see what others can build since this wisdom of this space seems to have shifted a few times over the last few years (e.g. what metric should we use? Memory? CPU? Scheduling latency? A combination? If so, what combination and weighted how? Perhaps it's very application-dependent?).

What we're doing now is:

As a final note, I just want to point out that at the end of the day, the memory limit is just another tool in the toolkit. If you can make some of your applications work better without it, I don't think that necessarily means it's a failure of the memory limit (sometimes it might be, but not always). I'm not saying that you necessarily think the memory limit should be used everywhere, just wanted to leave that here for anyone who comes looking at this thread. :)

Hi @mknyszek
Regarding the 50% cpu limit... Unless we understand incorrectly it means it can use up to that CPU to avoid going over the soft limit, but for many of our applications anything more than 20% GC CPU can have a serious impact (mostly when on failover state). Currently, we dynamically change GOGC when there is memory available we tend to increase it, when there isn't we just keep decreasing it to ensure our own soft limit, but we have a minimum threshold and we allow different service owners to set their own minimum threshold. That's more or less what we are missing with Go soft limit.
We currently don't have an example using soft limit, but in the past we have had issues with GOGC being too low and this caused bigger problems than a few instances crashing due to OOM. So, based on that assumption we think the scenario would repeat with soft limit.
What would be nice is a way of modifying how much CPU the GC can take to ensure the soft limit? Or a minimum GOGC value so that service owners decide at what point they believe is better to OOM than the degradation caused to the elevated GC.
Or would you suggest is better to wait for #56857 to have a way to keep an eye on the size of live bytes, so that when it gets close to the soft limit make a decision of either eat the cost of GC or just OOM?

Thanks for the detailed feedback and I'm glad it's working well for your overall!
Speaking broadly, I'd love to know more about what exactly this degraded state looks like. What is the downstream effect? Latency increase? Throughput decrease? Both? If you could obtain a GODEBUG=gctrace=1 (outputs to STDERR) of this degraded state, that would be helpful in identifying what if any next steps we should take.

Getting the traces to work in production would be hard. We have an HTTP handler to tune GOMEMLIMIT per container, so we can experiment with that with reasonable safety. There's no way to runtime way to enable traces, right?
That being said I can perhaps try to reproduce the same situation in staging. What we have seen in production was a significant CPU time utilization increase, leading to CPU throttling, leading to both latency increase and throughput decrease.
Below is screenshot of a "slowly leaking application" (more explained below) where we enabled GOMEMLIMIT temporarily. Note the CPU utilization increased significantly more than we expected - more than 50% of GOMAXPROCS.

We'd ideally want to enable it for everyone by default (a vast majority of our applications have plenty of memory available), but we're currently too afraid to do this. The reason is the death spirals you called in the proposal. Applications leaking memory, with GOMEMLIMIT, can get to a significantly degraded state. Paradoxically, those applications prefer to OOM, die quickly and be restarted than to struggle for a long time. The number of applications makes avoiding leaks unfeasible.

Choosing to die quickly over struggling for a long time is an intentional point in the design. In these difficult situations something has to give and we chose to make that memory.
But also if the scenario here is memory leaks, it's hard to do much about that without fixing the leak. The live heap will grow and eventually even without GOMEMLIMIT you'll OOM as well. GOMEMLIMIT isn't really designed to deal with a memory leak well (generally, we consider memory leaks to be a bug in long-running applications), and yeah I can see turning it on basically turning into "well, it just gets slower before it dies, and it takes longer to die," which may be worse than not setting a memory limit at all.
As for fixing memory leaks, we're currently planning some work on improving the heap analysis situation. I hope that'll make keeping applications leak-free more feasible in the future. (#57447)
(I recognize that encountering a memory leak bug at some point is inevitable, but in general we don't expect long-running applications to run under the expectation of memory leaks. I also get that it's a huge pain these days to debug them, but we're looking into trying to make that better with heap analysis.)

So I think you might be too optimistic vs what we see in our reality here (sorry:)). We:

have applications that are leaking quick, they restart often, they need to be fixed. Those typically have higher priority, and can be diagnosed with some effort - I wouldn't actually call it pain though, profiles are typically helpful enough.
"slowly leaking memory" applications that just very slowly accumulate memory as they run. These are actually low-priority  - as long as the {release_frequency}>2-5*{time_to_oom}, fixing it will not get prioritized. Especially if some of the leaks are in gnarly bits like stat emission. This only becomes a problem during extended quiet periods - the expectation is still that the applications will crash rather than degrade.

In summary though, we strongly expect leaks to be around forever.

A part of the problem (perhaps) is that we lack a good enough way of setting the right limit. We cannot set it to 98-99% of the container memory because some other applications can be running there. But, if we set it to 90%, once we hit the death spiral situation, we're in a degraded state for too long - it can take hours for OOM, and in the meantime, we are at risk of all containers of an application entering the degraded state.

FTR that's what the runtime/debug.SetMemoryLimit API is for and it should be safe (performance-wise) to call with a relatively high frequency. Just to be clear, is this also the memory leak scenario?

Yeah, so we would need to continue running a custom tuner though, right? It also seems if we're tuning in "user-space", equivalent results can be achieved with GOGC and GOMEMLIMIT - right?

The 90% case you're describing sounds like a misconfiguration to me; if the application's live heap is really close enough to the memory limit to achieve this kind of death spiral scenario, then the intended behavior is to die after a relatively short period, but it might not if it turns out there's actually plenty of available memory. However, this cotenant situation might not be ideal for the memory limit to begin with.
As a general rule, the memory limit, when used in conjunction with GOGC=off, is not a great fit for an environment where the Go program is potentially cotenant with others, and the others don't have predictable memory usage (or the Go application can't easily respond to cotenant changes). See https://go.dev/doc/gc-guide#Suggested_uses. In this case I'd suggest slightly overcommitting the memory limit to protect against many transient spikes in memory use (in your example here, maybe 95-96%), but set GOGC to something other than off.

This is slightly more nuanced, (and perhaps offtopic) each of our containers runs with a "helper" process responsible for starting up and shipping logs and performing local health checks (it's silly. don't ask). The memory we need to reserve for it varies per application - thus, for small containers, 95% might not be enough. For larger applications, we can increase the limit, but for both cases, we'd likely still need to look at the log output dynamically.
It is not immediately clear to me how to tune the right value of GOGC combined with GOMEMLIMIT. But, more importantly, my understanding of GOMEMLIMIT is that no matter the GOGC value we can still hit the death-spiral situation.

The GC might use at most 50% of the total CPU time, but it can end up using 2-3x more CPU than the actual application work. This is "GC degradation" would be hard to explain/sell to application owners.

I'm not sure I follow. Are you describing a situation in which your application is using say, 25% CPU utilization, and the GC is eating up 50%?

Yeah, @cdvr1993  explained it in the previous comment too. If container has GOMAXPROCS=8, but utilized 3 at that time. Then we hit GOMEMLIMIT, and GC is allowed to (per our understanding) to use up to 4 cores, so GC is now using more CPU than the application. At the same time, anything above 80% CPU utilization (in our experience) results in dramatically increased latency.

We're also concerned with a "degradation on failover" situation - an application that might be okay usually, in case of a sudden increase in traffic, might end up in a death spiral. And this would be precisely the time we need to avoid those.

(Small pedantic note, but the 50% GC CPU limiter is a mechanism to cut off the death spiral; in general a death spiral means that the GC keeps taking on more and more of the CPU load until application progress stops entirely.)

Perhaps we need a different name here then:) What we've observed might not be a death spiral, but a degradation large enough to severely disrupt production. Even with the 50% limit.

I think it depends on the load you're expecting. It's always possible to construct a load that'll cause some form of degradation, even when you're not using the memory limit (something like a tight OOM loop as the service gets restarted would be what I would expect with just GOGC).

Yeah, the problem seems to occur for applications that are "mostly fine", with days between OOMs.

If the memory limit is failing to degrade gracefully, then that's certainly a problem and a bug on our side (perhaps even a design flaw somewhere!). (Perhaps this risk of setting a limit too low such that you sit in the degraded state for too long instead of actually falling over can be considered something like failing to degrade gracefully, and that suggests that even 50% GC CPU is trying too hard as a default. I can believe that but I'd like to acquire more data first.)
However, without more details about the scenario in question, I'm not sure what else we can do to alleviate the concern. One idea is a backpressure mechanism (#29696), but for now I think we've decided to see what others can build since this wisdom of this space seems to have shifted a few times over the last few years (e.g. what metric should we use? Memory? CPU? Scheduling latency? A combination? If so, what combination and weighted how? Perhaps it's very application-dependent?).

IMO it seems like what you built is "almost perfect". We just need the applications to "die faster" - the easiest changes that come to mind would be reducing the limit from 50%, to either something like 25% or a static value (2 cores?).
When I say "almost perfect" I mean it though - I suspect we could rollout the GOMEMLIMIT to 98% of our applications with great results and without a problem, but the remaining users would come after us with pitchforks. And that forces us to use the GOMEMLIMIT as an opt-in, which is very disappointing given the results we see in 98% of the applications.
Thanks for the thoughtful response!

Hey @mknyszek @cdvr1993 I raised a new issue in https://github.com/golang/go/issues/58106.

@rabbbit @mknyszek  @rsc we are facing one issue regarding the memory limit . We are setting the memory limit to 18GB in 24GB server but still GC runs very frequently and eats up 80 percent of CPU and memory used is only 4 to 5 GB max . Also memory limit is goroutine wise ? or how to set the same for whole program.
In the entry point of our application we have specified something like this :-
debug.SetMemoryLimit(int64(8* 1024 * 1024 * 1024))
Is this okay or we need to do something additional. Also where to set the optional unit as described in the documentation

@VEDANTDOKANIA Unfortunately I can't help with just the information you gave me.
Firstly, how are you determining that the GC runs very frequently, and that it uses 80 percent of CPU? That's far outside of the bounds of what the GC should allow: there's an internal limiter to 50% of available CPU (as defined by GOMAXPROCS) that will prioritize using new memory over additional CPU usage beyond that point.
Please file a new issue with more details, ideally:

Platform
Go version
The environment you're running in (if in a container or cgroup, what is the container's CPU quota?)
The STDERR output of running your program with GODEBUG=gctrace=1.

Thanks.

Also memory limit is goroutine wise ? or how to set the same for whole program.

It's for the whole Go process.

In the entry point of our application we have specified something like this :-
debug.SetMemoryLimit(int64(8* 1024 * 1024 * 1024))

That should work fine, but just so we're on the same page, that will set an 8 GiB memory limit. Note that the GC may execute very frequently (but again, still capped at roughly 50%) if this value is set smaller than the baseline memory use your program requires.

Also where to set the optional unit as described in the documentation

The optional unit is part of the GOMEMLIMIT environment variable that Go programs understand. e.g. GOMEMLIMIT=18GiB.


out of memory

XNLI
I use all translation data of xnli, examples = 5890530，Distributed Data Parallel，encountered a memory leak problem
out of memory of raw，Memory usage reaches 300g
some lib version：
seqeval
tensorboardx
tokenizer
sacremoses
pythainlp
jieba
torch==1.4.0

Hi, did you try with smaller batch sizes and using subsets of the translation data? At what point does the memory leak occur?


Partition pdf

Issue #, if available:
#297
Description of changes:
When analyzing PDFs, unstructured.partition.auto was used, however unstructured.partition.pdf capable of detailed structural analysis can be selected.
When enabling unstructured.partition.pdf , set “enable_partition_pdf”: true , in cdk.json
unstructured.partition.pdf spends a lot of time.
Implement parallel processing with multiple processes and shorten processing time by making it possible to change the container size for embedding with cdk.json
When embedding 30 PDFs of 15 to 150 pages,
It takes 7 minutes with unstructured.partition.auto and 61 minutes with unstructured.partition.pdf
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Memo: comparison when enables hi-res mode with multi processing

When embedding 30 PDFs of 15 to 150 pages,
It takes 7 minutes with unstructured.partition.auto and 61 minutes with unstructured.partition.pdf

LGTM!


Login and Register route return blank errors

See https://github.com/Lambda-School-Labs/CS10-Developer-Profiles/pull/31#discussion_r217778727 and https://github.com/Lambda-School-Labs/CS10-Developer-Profiles/pull/31#discussion_r217780638

Partially solved.
LOGIN resolved with PR Jesuarva fix router issues #38


cryptomatte not showing in render tree?

I can't find cryptomatte under the Arnold/AOV category. When I add it using a script it shows up red without ports.
@JenusL Is it working for you?

@caron Yes it's working fine here in feat/a5.3.0 branch.

i had cryptomatte installed from alShaders maybe it was messing with it. i will try a fresh build of feat/a.5.3.0 branch

Yeah that's probably why.

@caron Did you get this to work?

i removed all references to the old cryptomatte and alshaders but still doesn't show up.
while trying to figure out what was going on i noticed the cryptomatte.dll in the plugins directory wasn't being installed correctly when i was switching between the different arnold core versions. i was building with 5.3.x and then switched to 5.2.x but the cryptomatte.dll wasn't being updated (either was alembic_proc.dll). isn't scons supposed to detect this and overwrite those .dlls?
i purged the arnold plugins directory and re ran abuild install and now i can see cryptomatte in the render tree. so the issue is actually with the build system.

Hmm, yeah that's strange. Maybe we should look into that.

Could you close this @caron?

Its not exactly fixed, its a build issue.

Ok cool I'll see if I can fix it.

my guess is scons doesn't actually track those files the same way as it would the .dll files we are building in sitoa. ie. if the file exists then that is good enough.

I found this https://scons.org/faq.html#Why_is_my_directory_only_updated_the_first_time.3F

Ya, that looks like what I ran into when I researched it originally... I didn't like that idea cause that would mean polluting the Arnold API directory with this random file. Did I understand the suggested workaround correctly?

You don't need a random file if the .dll files themselves has changed. And they have changed. Otherwise you wouldn't get a crash.
I fixed it now in https://github.com/Autodesk/sitoa/pull/76/commits/c9ac9a1a5f2e88c8d5c883a2c9ad2731071bda57

That is what I thought it would do but it didn't... I see you changed the Install to pull the files instead of just the folder. That was enough for it to see the files were different then?

Yes exactly. When it looked at the folder it only checked that the folder existed.
Now it checks all the files instead and it works as expected.

Closing #69


MM-20965 - Fixing menus overflow

Summary
MM-20965 - Fixing menus overflow
Ticket Link
https://mattermost.atlassian.net/browse/MM-20965
Screenshots

Creating a new SpinWick test server using Mattermost Cloud.

Mattermost test server created! :tada:
Access here: https://mattermost-webapp-pr-4450.test.mattermost.cloud

Account Type
Username
Password

Admin
sysadmin
Sys@dmin123

User
user-1
User-1@123

Test server destroyed


[Impeller] CupertinoDatePicker might cause text on screen to glitch.

Steps to reproduce

Add CupertinoDatePicker wrapped in showCupertinoModalPopup
Try to scroll dates

Expected results
No glitches in text
Actual results
Text is randomly glitching
Code sample
Code sample
    await showCupertinoModalPopup(
      barrierDismissible: true,
      context: context,
      builder: (context) {
        return Container(
          height: 250,
          color: Colors.white,
          child: Column(
            children: [
              Flexible(
                child: CupertinoDatePicker(
                  mode: CupertinoDatePickerMode.date,
                  initialDateTime: DateTime.now(),
                  onDateTimeChanged: (date) {
                    setState(() {
                      _date = date;
                    });
                  },
                ),
              ),
            ],
          ),
        );
      },
    );

Screenshots or Video

Screenshots / Video demonstration
https://github.com/flutter/flutter/assets/15138090/671c6558-e615-4604-a09a-d3aaab73df96

Logs
Logs
https://pastebin.com/6pHNQP0v

Flutter Doctor output
Doctor output
[✓] Flutter (Channel stable, 3.19.0, on macOS 14.3.1 23D60 darwin-arm64, locale en-UA)
    • Flutter version 3.19.0 on channel stable at /Users/admin/devkit/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision bae5e49bc2 (2 weeks ago), 2024-02-13 17:46:18 -0800
    • Engine revision 04817c99c9
    • Dart version 3.3.0
    • DevTools version 2.31.1

[✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
    • Android SDK at /Users/admin/Library/Android/sdk
    • Platform android-34, build-tools 34.0.0
    • ANDROID_HOME = /Users/admin/Library/Android/sdk
    • Java binary at: /Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 17.0.7+0-17.0.7b1000.6-10550314)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 15.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 15C65
    • CocoaPods version 1.15.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2023.1)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 17.0.7+0-17.0.7b1000.6-10550314)

[✓] VS Code (version 1.86.2)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.58.0

[✓] VS Code (version 1.87.0-insider)
    • VS Code at /Applications/Visual Studio Code - Insiders.app/Contents
    • Flutter extension version 3.64.0

[✓] Connected device (5 available)
    • iPhone 15 Pro (mobile) • 42E31902-8A83-4F08-8567-7878BDCA59D5 • ios            • com.apple.CoreSimulator.SimRuntime.iOS-17-2 (simulator)
    • macOS (desktop)        • macos                                • darwin-arm64   • macOS 14.3.1 23D60 darwin-arm64
    • Chrome (web)           • chrome                               • web-javascript • Google Chrome 122.0.6261.69

[✓] Network resources
    • All expected network resources are available.

• No issues found!

hey @hexlogic  on side is working fine can you  provide the video in mp4 format and screenshots of the issue your facing rightnow

Same here with stable 3.19.0
Text in the whole page might glitch like this:

Hi @hexlogic, what device(s) are you seeing this on? How often does this bug occur? I can't seem to reproduce this on my device.

Hi @zhangtingtingx1, can you provide a complete minimal reproducible code sample so that we can investigate this issue?

Hi @hexlogic, did you experience this prior to upgrading to stable 3.19.0? How many times have you experienced this so far?
Does it come with extended usage of the app or it can come at any point at all?

Hi @hexlogic, did you experience this prior to upgrading to stable 3.19.0? How many times have you experienced this so far?
Does it come with extended usage of the app or it can come at any point at all?

Yes, it's been even upgraded to 3.19.2
I've got this or similar behaviour in 4/10 cases. It can come at any point

+1 here, I'm experiencing the same issue when I use an app on debug or release versions with Impeller enabled. I think that there's some issue with Impeller itself, because I didn't experience this behavior when I ran it with --no-enable-impeller flag
Btw, can I use manual SkSL warm-up for now? Until the problem gonna be solved

+1 here, I'm experiencing the same issue when I use an app on debug or release versions with Impeller enabled. I think that there's some issue with Impeller itself, because I didn't experience this behavior when I ran it with --no-enable-impeller flag
Btw, can I use manual SkSL warm-up for now? Until the problem gonna be solved

Yes, you're good to go. This is actually what we did on our end.

+1 here, I'm experiencing the same issue when I use an app on debug or release versions with Impeller enabled. I think that there's some issue with Impeller itself, because I didn't experience this behavior when I ran it with --no-enable-impeller flag
Btw, can I use manual SkSL warm-up for now? Until the problem gonna be solved

Yes, you're good to go. This is actually what we did on our end.

Sorry, but can you tell me your current flutter version? I'm trying to warmup shaders by pressing "M", but i'm getting this error:

Hi all, we would appreciate it if we could get a complete minimal reproducible code sample that can consistently reproduce the issue. So far I have only been able to reproduce it once with the code sample below.
Labeling for awareness for now.

code used
import 'package:flutter/cupertino.dart';
import 'package:flutter/material.dart';

void main() {
  runApp(const MyApp());
}

class MyApp extends StatelessWidget {
  const MyApp({super.key});

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        useMaterial3: true,
      ),
      home: const MyHomePage(),
    );
  }
}

class MyHomePage extends StatefulWidget {
  const MyHomePage({super.key});

  @override
  State<MyHomePage> createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {
  DateTime? _date;

  @override
  void initState() {
    super.initState();
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(),
      body: Center(
          child: SingleChildScrollView(
        child: Text('''
            Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse vitae arcu quam. Donec ut augue viverra arcu pulvinar fringilla. Sed odio ante, semper nec egestas sed, pretium eu erat. Suspendisse nisi urna, sodales vitae efficitur non, commodo in libero. Mauris dapibus urna massa, sed finibus orci dignissim ut. Donec feugiat semper ipsum, efficitur venenatis risus egestas eget. Aliquam ac mi sed est aliquam efficitur nec quis nulla. Praesent elit risus, maximus sit amet augue at, varius semper nibh. Praesent volutpat tellus mi, id congue orci tempus eget.

      Curabitur sed fringilla tellus, sed tincidunt ligula. Donec at elementum justo. Morbi vitae felis scelerisque, gravida risus tincidunt, auctor elit. Curabitur ultrices tortor id euismod facilisis. Praesent aliquam, turpis at viverra ultricies, dui diam egestas urna, sit amet lacinia lorem mauris vel orci. Duis tincidunt sed lectus eu posuere. Ut viverra massa porttitor, dignissim mi vitae, pharetra sem. Fusce quis quam sit amet felis volutpat tincidunt. Phasellus pulvinar, dolor sed sodales faucibus, mi purus interdum quam, id auctor odio felis mattis lacus. Aenean elit augue, tincidunt ut tempor vel, maximus vel quam. Donec pulvinar turpis finibus mi molestie, et bibendum nulla ultricies. Aliquam erat volutpat. Morbi semper sem purus.

      Nullam facilisis, neque et maximus scelerisque, nulla tortor dictum nisi, vel rhoncus metus est at mi. Donec ut rhoncus libero. Nam quis justo at urna egestas cursus eu non magna. Cras semper dapibus elit sit amet imperdiet. Cras in dignissim nulla. Nullam vitae malesuada velit. Sed facilisis nunc ac urna rhoncus rhoncus.

      Aenean in sollicitudin mi, in vehicula enim. Nulla facilisi. Nunc scelerisque pulvinar elit ut placerat. Ut nulla massa, pellentesque a semper eu, dignissim non ligula. Nulla vel ultrices lorem. Vestibulum sit amet iaculis nisl. Nulla ut lacus luctus, dignissim velit nec, egestas lacus. Ut euismod tortor ut purus ultricies, vitae scelerisque felis bibendum. Morbi iaculis nec turpis eu tempus. Nulla vitae ipsum nec neque interdum hendrerit id vel ante. Donec mattis volutpat viverra. Etiam id convallis lacus, at faucibus dui. Praesent enim tellus, tristique a pellentesque at, maximus eu massa. Nullam mattis porta est sed finibus. Fusce at facilisis felis, et cursus turpis. Praesent cursus leo consectetur ipsum tincidunt eleifend.

      Fusce pharetra neque ac tellus fermentum, in semper sem venenatis. In egestas mattis varius. Pellentesque vel libero non felis euismod fermentum. Donec id nulla vel eros sodales vestibulum. Vestibulum tempor orci ac elit ultrices rhoncus. Nulla sollicitudin metus orci, ultrices sodales ex dapibus sit amet. Praesent auctor tincidunt nisl, id finibus sem iaculis in. Phasellus mattis ornare orci, eu dictum nisi feugiat quis. Morbi vel nulla a libero vulputate auctor. Orci varius natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Nunc nec justo imperdiet dui mollis aliquet. Ut ante turpis, elementum pharetra vestibulum nec, rutrum id leo. Nam ornare venenatis ex, non tempus felis malesuada in. Etiam congue suscipit sollicitudin. Proin aliquam elementum lorem, ac accumsan sem interdum eget. Nunc elit leo, tempus at elit id, pellentesque gravida augue.
      '''),
      )),
      floatingActionButton: FloatingActionButton(
        onPressed: () async {
          await showCupertinoModalPopup(
            barrierDismissible: true,
            context: context,
            builder: (context) {
              return Container(
                height: 250,
                color: Colors.white,
                child: Column(
                  children: [
                    Flexible(
                      child: CupertinoDatePicker(
                        mode: CupertinoDatePickerMode.date,
                        initialDateTime: DateTime.now(),
                        onDateTimeChanged: (date) {
                          setState(() {
                            _date = date;
                          });
                        },
                      ),
                    ),
                  ],
                ),
              );
            },
          );
        },
      ),
    );
  }
}

I can reproduce this. All I need to do is open the cupertino picker and the bottom right corner will glitch slightly. This fixes itself after I interact with the picker

Its not a synchronization issue, this appears to be a partial repaint problem - at least for me anyway. I can fix this by adding:
<key>FLTDisablePartialRepaint</key>
<true />

To the Runner/Info.plist. FYI @knopp

@jonahwilliams, I don't seem to b able to reproduce this, at least not in simulator. When device are you testing this with? I assume on main? Also, just in case, could you try runing it with <key>FLTEnableImpeller</key><false/>. Partial repaint does work slightly differently with impeller and this might help narrowing it down.

If you run the example app, open the Cupertino picker, look in the bottom right corner.

@jonahwilliams, I was able to reproduce the issue. It seems that the whole picker renders wrongly with impeller, possibly out of calculated bounds too which would cause issues with partial repaint. Here's a GIF of impeller vs skia:

Update: the depth changes did not land in 3.19 so this probably isn't https://github.com/flutter/flutter/issues/144333. Instead I'd guess something like https://github.com/flutter/engine/pull/45150

I think this repro is https://github.com/flutter/flutter/issues/144333. The original post looks like a distinct issue.

Status: we're still unable to reproduce this issue. I landed a speculative revert of a related change. Without confirmation that this is or isn't helping we don't have enough information to proceed or put together a speculative cherry pick

I have a large app that I'm able to reproduce this bug on older iOS devices that are not able to run on iOS 17 (i.e. iPhone X, iPad mini 4). On newer devices I do not see this issue. Nor do I see the issue with small sample apps. My app is currently building with Flutter 3.19.5.  I'd love to test the speculative revert but there are numerous pub packages that we are using that need changes to work with current stable branch.  Would I be able to get a patched 3.19.5 engine to test with?

I could create a branch of 3.19.5 that has this fix applied, but I don't have the means to distribute binaries unless we do a release. Can you test with a local engine build @mtallenca ?

Haven't done a local engine build before - but I can do that.

The branch is here: https://github.com/flutter/engine/pull/51855 . You can either use git checkout or the gh tool to download that.

I can confirm the fix works. Tests done with release build on iOS, iPad mini 4
engine as is - app running on iPad mini 4 - https://share.icloud.com/photos/0707u9GCkVVuwSv58N7ehEzIg
engine#41855 - running on same iPad mini 4 - https://share.icloud.com/photos/0a918oBzOkf62y4QsYTGa0T-Q

It is fixed in Flutter 3.19.6
On Fri, Apr 19, 2024, 14:18 MohamedYassineDeneubourg <
@.***> wrote:

Hello, how can we fix this issue ?
—
Reply to this email directly, view it on GitHub
https://github.com/flutter/flutter/issues/144331#issuecomment-2066450455,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAESUZ5AGL44ITQVOMVLPFDY6EDRJAVCNFSM6AAAAABD6SUUOOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANRWGQ2TANBVGU
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

We've cherry picked the fix for this into the last flutter stable. Closing.


[Not Recognized Modifier] - Parsing from chat

While parsing an item from a stash or inventory you get the expected User facing version.

Parsing the same item after it's been linked in chat, from chat returns the APT version of the item and fails to recognize.

Duplicate of #71, this is a bug in the game so I'm not going to try and fix until rest of the program is in a more working state


Retrieve Indicator endpoint permissions

Story
Permissions document: https://docs.google.com/spreadsheets/d/1mWGhO-Acin5SQnfOrHm58UCY3Y965x98fk9i6-ozpq0/edit#gid=0
We need to fix permissions (see IndicatorPermissions) and add tests for /api/indicator/
Acceptance criteria
GIVEN I make a GET Retrieve query to /api/indicator/
AND I have the correct permissions
THEN the Indicator is updated

We need to clarify what permission were failing Program Admin, Org Admin etc or all on these or they are difficult to test.

I think we can close this. This issue was created before we even started implementing permissions. There is the "Permissions and Roles" document which can be revised by Karrla or Joy and mark and create new issues if some permission is missing.


fix: update the server layer extraction

Description

SquadServer regularly invokes updateLayerInformation to update the current and next layer variables, namely this.currentLayer and this.nextLayer.
This is fetched from Layers class, getLayerByName method, which filters the pulled layers this.layers by the name field.
The layers are fetched from https://raw.githubusercontent.com/Squad-Wiki/squad-wiki-pipeline-map-data/master/completed_output/_Current%20Version/finished.json

I'm actually a bit concerned that this "current version" is outdated with respect to the latest Squad version 8.0.

However, the layer name for filtering is fetched from SquadRcon class, getCurrentMap method which uses the RCON command ShowCurrentMap.  The problem here is 2-fold:

The regex used to extract the level and the layer names is broken: ^Current level is (.*), layer is (.*) Because the 2nd regex group actually matches a string like JensensRange_USA-RGF, factions USA RGF
Even if we correctly extract the "layer name", it actually corresponds to the layerid field in the pulled layers

What this PR does
Solves those 2 problems by:

Updating the regexes to extract the level and the layer ids from the RCON commands ShowCurrentMap and ShowNextMap as: ^Current level is ([^,]*), layer is ([^,]*)
Changing the matching logic to use the layerid field instead of name

I have submitted a PR that addresses duplicate Victim field that you reported: #364

Hi @Ulibos,
Any updates on testing the PR?

We are using a plugin that queries data from layers infrequently so I can't vouch for a full comprehensive test but so far I haven't seen a single failure on this part of the code.

Should we merge it then?

I'm all for merge but I'm not the one calling the shots.

There is another merge about to happen soon that will require rewrites again. Merges are on hold until then.

Force pushing. Mobile UI is preventing conversation resolution.


Recipes - Aggregate SOQL Datatable has an error

Hi!
I am going through all the recipes again in preparation for testing my fix for #99. The Aggregate SOQL Datatable tab throws an error upon load:
fetchTableCache error
Field must be grouped or aggregated: RecordTypeId Class.DataTableService.getSObjectsWithAllowedFields: line 546, column 1 Class.DataTableService.getSObjectDataFromQueryString: line 263, column 1 Class.DataTableService.getSObjectData: line 242, column 1 Class.DataTableService.getTableCache: line 226, column 1

(This is in a sandbox with a fresh install of core + recipes)

FWIW you should not use a sandbox to test recipes (they are meant for scratch orgs). However, that doesn't mean there isn't a bug for your specific sandbox. What I want to do now is determine where the source of error is coming from.
BTW, I cannot repro with a fresh scratch org and master branch, which should be the same as the latest version.
Do me a favor on this one and create a scratch org and install 1.8.0 with the 1.8.0 recipe packageId per the wiki - perhaps I have the packageIds wrong (or didnt republish the versions).
These data points will help save me time!

I just set up my first ever scratch org and installed the packages using the wiki instructions and cannot replicate the issue. How do I figure out which versions I have? I am looking at Installed Packages in setup and there are none on the list.
TY

@solo-1234 if you pushed from master branch, there is no version.
You would create a new scratch org and use the install procedure (instead of push) to see if you can replicate the issue.
Based on your comment, it sounds like I might need to push out a 1.8.1 but I would like some confirmation:

Spin new Scratch
Use package install
Confirm issue (would imply packaged code !== master)

Got it - sure! Thanks for bearing with me...

Not a problem! Happy to mentor and get extra help any way I can ;)

Do I use the sandbox or prod install links for a scratch org?

Okay in my scratch org with the packages in the wiki I do not see this error. How would I go about debugging it further in my sandbox?

Based on the error message I wonder if it doesn't like the fact that I have record types on account opportunity. I added GROUP BY RecordTypeId to both SOQLs on the page and that got rid of the errors in my sandbox.
I then copied the updated SOQLs to the scratch orgs, but it didn't like them - I get two errors:
Invalid SOQL String Invalid field: 'RecordTypeId'
Invalid SOQL String No such column 'RecordTypeId' on entity 'Account'. If you are attempting to use a custom field, be sure to append the '__c' after the custom field name. Please reference your WSDL or the describe call for the appropriate names.
Here are the SOQL strings:
SELECT COUNT(Id) FROM Account GROUP BY RecordTypeId
SELECT SUM(ExpectedRevenue) Max_Revenue, FISCAL_YEAR(CloseDate) Close_Date_Year, FISCAL_MONTH(CloseDate) Close_Date_Month, RecordTypeId FROM Opportunity GROUP BY RecordTypeId, CloseDate LIMIT 500
Not sure what the solution would be for these recipes to work in orgs with record types and in orgs without...

The original SOQL for opps is SELECT SUM(ExpectedRevenue) Max_Revenue, FISCAL_YEAR(CloseDate) Close_Date_Year, FISCAL_MONTH(CloseDate) Close_Date_Month FROM Opportunity GROUP BY CloseDate
It works fine in my scratch org.
Added an Opp Record Type (and a Sales Process which record types require) and now I get the error:
Field must be grouped or aggregated: RecordTypeId Class.DataTableService.getSObjectsWithAllowedFields: line 546, column 1 Class.DataTableService.getSObjectDataFromQueryString: line 263, column 1 Class.DataTableService.getSObjectData: line 242, column 1 Class.DataTableService.getTableCache: line 226, column 1
Modifying the SOQL to this gets rid of the error:
SELECT RecordTypeId, SUM(ExpectedRevenue) Max_Revenue, FISCAL_YEAR(CloseDate) Close_Date_Year, FISCAL_MONTH(CloseDate) Close_Date_Month, RecordTypeId FROM Opportunity GROUP BY RecordTypeId,CloseDate LIMIT 500
But that SOQL threw an error in an org without record types.
Repro steps are to take a new scratch org, create a new sales process (with any opp stages you want), and then create an opp record type. Then navigate to the Aggregate SOQL Datatable tab.
Is that enough? I don't know how to create a package to contain the sales process + record type but am open to learning.

To summarize what was discussed, the source of this issue is that when an org has record types on the object, soqldatatable will automatically add RecordTypeId to the list of queried fields (for reasons related mostly to inline editing and picklists).  With aggregate soql, if that field is included, it also needs to be added to GROUP BY.
Short term solution is to only rely on recipes working in scratch orgs (since sandboxes can vary wildly).
Longer term solution would be to adjust that feature so that the RecordTypeId field is not added to aggregate queries (which shouldn't support inline editing anyways) for the rare use cases for aggregate soql (not urgent since admins can manually add GROUP BY RecordTypeId to get it working).
@tsalb let me know please if I missed anything!

Sounds about right except no long term fix. Not worth the investment unless I get a PR.
I’ll take the action item from here to update wiki to be more explicit about where recipes are supposed to be used.

Closed by #108


Update #4

More description about being more descriptive

Walker, can you get the latest version before making this change?