Full Text Search Postgres is case insensitive by default

Bug description
Hi! I'm using Prisma Client 6.0.1 with Postgres 16 and it looks like full text search is defaulting to non-case sensitive? I was wondering if this is a bug or the expectation, because we need case-sensitive search. Thanks!
How to reproduce
Perform a search using search on a postgres server with fullTextSearchPostgres enabled on the Prisma client.
Expected behavior
I would expect the search to be case-sensitive since that's the default for other Prisma functions on Postgres.
Prisma information

generator client {
  provider        = "prisma-client-js"
  previewFeatures = ["tracing", "omitApi", "fullTextSearchPostgres"]
}

datasource db {
  provider = "postgresql"
  url      = env("DATABASE_URL")
}
...

await prisma.myModel.findFirst({
      where: {
        AND: [
          {
            myField: {
              search: "!Case"
            },
          },
          {
            myField: {
              not: null,
            },
          },
        ],
      }
    });

Environment & setup

OS: macOS
Database: PostgreSQL 16
Node.js version: 20.10.0

Prisma Version
6.0.1

Hi @roshcagra
The default behavior of full-text search in PostgreSQL is case insensitive. The behavior you're experiencing is the expected behavior for PostgreSQL full-text search, rather than a bug in Prisma. If you need case-sensitive search, you might need to use raw SQL queries to accomplish this.


Database configuration should be done via deployment, not by scoretrak itself

https://github.com/ScoreTrak/ScoreTrak/blob/d30613177a7b60b42cdb6d2d876f4c03b5ce4c9b/pkg/storage/storage.go#L72-L84

Retrieving an entry(with validation):
SELECT Properties.* FROM Teams, Hosts, Services, Properties  WHERE Hosts.ID = Services.host_id AND Services.ID = Properties.service_id  AND Properties.key = 'Username' AND Properties.service_id = 'ff783e2b-ddd2-4d24-82ee-53f58c1be716' AND Teams.id = '03302d0b-c28d-444a-9b9b-0440c3d711e3';
Updating an entry(with validation)
UPDATE Properties SET key = 'test' FROM Teams, Hosts, Services WHERE Hosts.ID = Services.host_id AND Services.ID = Properties.service_id  AND Properties.key = 'Username' AND Properties.service_id = 'ff783e2b-ddd2-4d24-82ee-53f58c1be716' AND Teams.id = '03302d0b-c28d-444a-9b9b-0440c3d711e3';

https://github.com/go-gorm/gorm/issues/4248


Interface

Docs Build status updates of commit 8990a17:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

.openpublishing.publish.config.json
:white_check_mark:Succeeded

ref102/xml/CatLibrary/Cat`2.xml
:white_check_mark:Succeeded

ref102/xml/CatLibrary/Complex`2+ComplexFlags.xml
:white_check_mark:Succeeded

ref102/xml/CatLibrary/Dog`1.xml.remove
:white_check_mark:Succeeded

ref102/xml/CatLibrary/FakeDelegate`1.xml
:white_check_mark:Succeeded

ref102/xml/CatLibrary/Tiny.xml.remove
:white_check_mark:Succeeded

ref102/xml/CatLibrary/Tom.xml
:white_check_mark:Succeeded

ref102/xml/FrameworksIndex/cat-1.0.xml
:white_check_mark:Succeeded

ref102/xml/FrameworksIndex/cat-2.0.xml
:white_check_mark:Succeeded

ref102/xml/PackageInformation/cat-1.0.json
:white_check_mark:Succeeded

ref102/xml/PackageInformation/cat-2.0.json
:white_check_mark:Succeeded

ref102/xml/VersioningTest/C.xml
:white_check_mark:Succeeded

ref102/xml/VersioningTest/I.xml
:white_check_mark:Succeeded

ref102/xml/VersioningTest/I1.xml
:white_check_mark:Succeeded

ref102/xml/VersioningTest/I2.xml
:white_check_mark:Succeeded

ref102/xml/VersioningTest/VersioningExtensions.xml
:white_check_mark:Succeeded

ref102/xml/index.xml
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel

Docs Build status updates of commit ba87ebf:
:x: Validation status: errors
Please follow instructions here which may help to resolve issue.

File
Status
Preview URL
Details

_dependentPackages/CommonPlugins/tools/SplitTOC.ps1
:x:Error

Details

_dependentPackages/CommonPlugins/tools/SplitTOC.ps1

Line 108: [Error] Error happened when executing plug-in script _dependentPackages/CommonPlugins/tools/SplitTOC.ps1 at step docset_prebuild. Please check your .openpublishing.publish.config.json?on the configuration of the plug-ins or ask for help?in Docs Support teams channel. Detailed error message: .

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
Note: Your PR may contain errors or warnings unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel

Docs Build status updates of commit b28d9c0:
:x: Validation status: errors
Please follow instructions here which may help to resolve issue.

File
Status
Preview URL
Details

_dependentPackages/CommonPlugins/tools/SplitTOC.ps1
:x:Error

Details

_dependentPackages/CommonPlugins/tools/SplitTOC.ps1

Line 108: [Error] Error happened when executing plug-in script _dependentPackages/CommonPlugins/tools/SplitTOC.ps1 at step docset_prebuild. Please check your .openpublishing.publish.config.json?on the configuration of the plug-ins or ask for help?in Docs Support teams channel. Detailed error message: System.IO.FileLoadException: Could not load file or assembly 'YamlDotNet, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=ec19458f3c15af5e' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040).

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
Note: Your PR may contain errors or warnings unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel

Docs Build status updates of commit c17b22b:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

.openpublishing.publish.config.json
:white_check_mark:Succeeded

ref102/xml/CatLibrary/Cat`2.xml
:white_check_mark:Succeeded

ref102/xml/CatLibrary/Complex`2+ComplexFlags.xml
:white_check_mark:Succeeded

ref102/xml/CatLibrary/Dog`1.xml.remove
:white_check_mark:Succeeded

ref102/xml/CatLibrary/FakeDelegate`1.xml
:white_check_mark:Succeeded

ref102/xml/CatLibrary/Tiny.xml.remove
:white_check_mark:Succeeded

ref102/xml/CatLibrary/Tom.xml
:white_check_mark:Succeeded

ref102/xml/FrameworksIndex/cat-1.0.xml
:white_check_mark:Succeeded

ref102/xml/FrameworksIndex/cat-2.0.xml
:white_check_mark:Succeeded

ref102/xml/PackageInformation/cat-1.0.json
:white_check_mark:Succeeded

ref102/xml/PackageInformation/cat-2.0.json
:white_check_mark:Succeeded

ref102/xml/VersioningTest/C.xml
:white_check_mark:Succeeded

ref102/xml/VersioningTest/I.xml
:white_check_mark:Succeeded

ref102/xml/VersioningTest/I1.xml
:white_check_mark:Succeeded

ref102/xml/VersioningTest/I2.xml
:white_check_mark:Succeeded

ref102/xml/VersioningTest/VersioningExtensions.xml
:white_check_mark:Succeeded

ref102/xml/index.xml
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel

Docs Build status updates of commit d78ad36:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

.openpublishing.publish.config.json
:white_check_mark:Succeeded

ref102/xml/CatLibrary/Cat`2.xml
:white_check_mark:Succeeded

ref102/xml/CatLibrary/Complex`2+ComplexFlags.xml
:white_check_mark:Succeeded

ref102/xml/CatLibrary/Dog`1.xml.remove
:white_check_mark:Succeeded

ref102/xml/CatLibrary/FakeDelegate`1.xml
:white_check_mark:Succeeded

ref102/xml/CatLibrary/Tiny.xml.remove
:white_check_mark:Succeeded

ref102/xml/CatLibrary/Tom.xml
:white_check_mark:Succeeded

ref102/xml/FrameworksIndex/cat-1.0.xml
:white_check_mark:Succeeded

ref102/xml/FrameworksIndex/cat-2.0.xml
:white_check_mark:Succeeded

ref102/xml/PackageInformation/cat-1.0.json
:white_check_mark:Succeeded

ref102/xml/PackageInformation/cat-2.0.json
:white_check_mark:Succeeded

ref102/xml/VersioningTest/C.xml
:white_check_mark:Succeeded

ref102/xml/VersioningTest/I.xml
:white_check_mark:Succeeded

ref102/xml/VersioningTest/I1.xml
:white_check_mark:Succeeded

ref102/xml/VersioningTest/I2.xml
:white_check_mark:Succeeded

ref102/xml/VersioningTest/VersioningExtensions.xml
:white_check_mark:Succeeded

ref102/xml/index.xml
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel


Crash

Microsoft PowerToys version
0.41.3
Running as admin

[ ] Yes

Area(s) with issue?
General
Steps to reproduce
Hitting alt+space, selecting program and then pressing enter.
Cannot be repredocued,
Version: <IP_ADDRESS>
OS Version: Microsoft Windows NT 10.0.19043.0
IntPtr Length: 8
x64: True
Date: 25/07/2021 12:06:27
Exception:
System.InvalidCastException: Unable to cast object of type 'System.Windows.Documents.Run' to type 'System.Windows.FrameworkElement'.
at PowerLauncher.MainWindow.SuggestionsList_PreviewMouseLeftButtonUp(Object sender, MouseButtonEventArgs e)
at System.Windows.Input.MouseButtonEventArgs.InvokeEventHandler(Delegate genericHandler, Object genericTarget)
at System.Windows.RoutedEventArgs.InvokeHandler(Delegate handler, Object target)
at System.Windows.EventRoute.InvokeHandlersImpl(Object source, RoutedEventArgs args, Boolean reRaised)
at System.Windows.UIElement.ReRaiseEventAs(DependencyObject sender, RoutedEventArgs args, RoutedEvent newEvent)
at System.Windows.UIElement.OnPreviewMouseUpThunk(Object sender, MouseButtonEventArgs e)
at System.Windows.Input.MouseButtonEventArgs.InvokeEventHandler(Delegate genericHandler, Object genericTarget)
at System.Windows.RoutedEventArgs.InvokeHandler(Delegate handler, Object target)
at System.Windows.EventRoute.InvokeHandlersImpl(Object source, RoutedEventArgs args, Boolean reRaised)
at System.Windows.UIElement.RaiseEventImpl(DependencyObject sender, RoutedEventArgs args)
at System.Windows.ContentElement.RaiseTrustedEvent(RoutedEventArgs args)
at System.Windows.Input.InputManager.ProcessStagingArea()
at System.Windows.Input.InputProviderSite.ReportInput(InputReport inputReport)
at System.Windows.Interop.HwndMouseInputProvider.ReportInput(IntPtr hwnd, InputMode mode, Int32 timestamp, RawMouseActions actions, Int32 x, Int32 y, Int32 wheel)
at System.Windows.Interop.HwndMouseInputProvider.FilterMessage(IntPtr hwnd, WindowMessage msg, IntPtr wParam, IntPtr lParam, Boolean& handled)
at System.Windows.Interop.HwndSource.InputFilterMessage(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam, Boolean& handled)
at MS.Win32.HwndWrapper.WndProc(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam, Boolean& handled)
at MS.Win32.HwndSubclass.DispatcherCallbackOperation(Object o)
at System.Windows.Threading.ExceptionWrapper.InternalRealCall(Delegate callback, Object args, Int32 numArgs)
at System.Windows.Threading.ExceptionWrapper.TryCatchWhen(Object source, Delegate callback, Object args, Int32 numArgs, Delegate catchHandler)
✔ Expected Behavior
Selected app to open
❌ Actual Behavior
Powertoys crashed,
Other Software
No response

Duplicate of #11898

This was fixed with the release of v0.41.4.
Please update to the latest version of PowerToys and this issue should be resolved.

Thanks Franky
I missed that,: help appreciated

Many Thanks
Robin Chappell
email sent using webmail, please accept apologies for errors and typo's.
[image: Mailtrack]
https://mailtrack.io?utm_source=gmail&utm_medium=signature&utm_campaign=signaturevirality5&
Sender
notified by
Mailtrack
https://mailtrack.io?utm_source=gmail&utm_medium=signature&utm_campaign=signaturevirality5&
25/07/21,
22:47:59
On Sun, 25 Jul 2021 at 19:12, Franky Chen @.***> wrote:

This was fixed with the release of v0.41.4
https://github.com/microsoft/PowerToys/releases/tag/v0.41.4.
Please update to the latest version of PowerToys and this issue should be
resolved.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/microsoft/PowerToys/issues/12500#issuecomment-886238650,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ASBQC6VC7Y376SMBWMOBEKLTZRHZ7ANCNFSM5A6OALNA
.


Directive blocks are not represented

I am currently implementing indentation using emacs treesit ( tree-sitter integration ), which uses this library. Everything works well except for directive blocks like
<%= if true do %>
<.some_tag_or_component>
</.some_tag_or_component>
<% end %>

which produces
(fragment [0, 0] - [4, 0]
  (directive [0, 0] - [0, 17]
    (partial_expression_value [0, 3] - [0, 14]))
  (component [1, 0] - [2, 25]
    (start_component [1, 0] - [1, 24]
      (component_name [1, 1] - [1, 23]
        (function [1, 2] - [1, 23])))
    (end_component [2, 0] - [2, 25]
      (component_name [2, 2] - [2, 24]
        (function [2, 3] - [2, 24]))))
  (directive [3, 0] - [3, 10]
    (partial_expression_value [3, 3] - [3, 7])))

Since the directive is not the parent of the component there is no simple way of checking parent for indentation.

I took a look at implementing this change in https://github.com/phoenixframework/tree-sitter-heex/commit/da2b726597e845fe6ce4012b4a575a6e33bd3c8d
I'm not sure it's possible to robustly support the grouping of directives. An example like so seems straight-forward:
<%= if true do %>
  <%= if true do %>
  <% end %>
<% end %>

Using heuristics similar to what the HEEx engine uses (i.e. looking at the first/last tokens in the directive, do or -> or end), we can tell that the first two lines are incomplete expressions and the last two end the expressions. Essentially:
(partial_expression)
  (partial_expression)
  (ending_expression)
(ending_expression)

So far this looks straightforward but there are more complicated cases like so:
<%= case x do %>
  <%= ^x -> %>X
  <%= _x -> %>Not X
<% end %>

Which is essentially:
(partial_expression)
  (partial_expression)
  (partial_expression)
(ending_expression)

Again this is still doable. When we nest them though, we're in trouble:
<%= if true do %>
  <%= case x do %>
    <%= ^x -> %>X
    <%= _x -> %>Not X
  <% end %>
<% end %>

(partial_expression)
  (partial_expression)
    (partial_expression)
    (partial_expression)
  (ending_expression)
(ending_expression)

Which partial_expressions belong to which ending_expressions?
And we can't simply use the do token to tell apart the starts of groups from partial expressions because of fns:
<%= Enum.map(things, fn _thing -> %>
  Thing
<% end) %>

Between if, case, and fns, I'm not sure we are always able to decide how to group these.
However for the purposes of indentation, I think it's only necessary to indent when you see a partial_expression and dedent when you see a ending_expression, no?

Thanks for the detailed response and explanation. Do you mind if I tinker with your change for a day or two and see if I can make it work?
The next step for heex-mode is to add better navigation which also required directives to be part of the tree, but it is not essential and if we move towards :if and :for within the tags the effort on this might not be worth the trouble.
Just to give some insight on how Emacs treesit (tree-sitter binding ) works according to my understanding.
There are 3 parts:

font-lock - simplest and just requires the ability to query and tag which matches the font

indentation - need the ability to find the node type and node parent, then indents based on an anchor and offset

(defvar heex--treesit-indent-rules
  (let ((offset heex-indent-level))
    `((heex
       ((node-is "end_tag") parent-bol 0)
       ((node-is "end_component") parent-bol 0)
       ((node-is "end_slot") parent-bol 0)
       ((parent-is "component") parent-bol ,offset)
       ((parent-is "slot") parent-bol ,offset)
       ((parent-is "tag") parent-bol ,offset)
       ))))

It is possible to indent based on siblings, but it gets more tricky as custom functions needs to exists. Adding custom functions makes it possible, but also makes it less elegant and harder to maintain.

Navigate - being able to jump up the s-expression ( parent ) tree or move forward/backwards over s-expressions ( siblings )
This is probably the most complex, but an important part of any major mode in emacs. To be able to navigate around it is required to understand the structure in more detail. I don't believe it is extremely important to get this right with directives, but will be nice in the future.

@the-mikedavis this is the same issue elixir-mode faced with a weak partial parser, but then injected artificial tokens for repeating match blocks to be able to be able to identify: https://github.com/elixir-editors/emacs-elixir/blob/master/elixir-smie.el#LL272C39-L272C39
The change to have an partial and ending_expression will make it possible to solve it in some way, but still probably not a very simple way :). It will be better, thanks.
I am not sure to what extend vscode/sublime etc.. uses or will use tree-sitter for indentation, but if this will only be an emacs thing ( and an optional dependency ) then it won't be worth spending too much time on this as it might not be a simple change to nest expressions. In the future if there is time for me to learn how tree-sitter grammar works I would like to attempt to change this. It will help with maintenance moving this complexity to this shared repo.
How would backward compatible work if we change this down the road?

I'm not sure how it works for emacs but with other tree-sitter consumers like nvim or helix, any change that removes/renames nodes is breaking: any queries that include those nodes will fail the query analysis step and be unusable. Adding an ending_expression node which was originally a partial_expression node could be seen as breaking change too I suppose.
The tree-sitter consumers that I know about lock down the versions of the tree-sitter-* repositories so that a new update doesn't automatically break things, so breaking compatibility is usually not a huge deal


[Event Request] Table 37 - Sales Line - OnGenProdPostingGroupOnBeforeValidate

Dear ALAppExtensions team,
through this issue we would like to request adding event publisher OnGenProdPostingGroupOnBeforeValidate in Table 37 Sales Line with the following signature:
[IntegrationEvent(false, false)]
local procedure OnGenProdPostingGroupOnBeforeValidate(var SalesLine: Record "Sales Line"; var xSalesLine: Record "Sales Line"; var GenProdPostingGrp: Record "Gen. Product Posting Group"; var IsHandled: Boolean);
begin
end;

which would be called as follows in the OnValidate trigger of field 75 - Gen. Prod. Posting Group:
trigger OnValidate()
var
    // >>>> New >>>>>>>>>>>>>>>>
    IsHandled: Boolean;
    // <<<< New <<<<<<<<<<<<<<<<
begin
    // >>>> New >>>>>>>>>>>>>>>>
    OnGenProdPostingGroupOnBeforeValidate(Rec, xRec, GenProdPostingGrp, IsHandled);
    if IsHandled then
        exit;
    // <<<< New <<<<<<<<<<<<<<<<
    TestJobPlanningLine;
    TestStatusOpen();
    if xRec."Gen. Prod. Posting Group" <> "Gen. Prod. Posting Group" then
        if GenProdPostingGrp.ValidateVatProdPostingGroup(GenProdPostingGrp, "Gen. Prod. Posting Group") then
            Validate("VAT Prod. Posting Group", GenProdPostingGrp."Def. VAT Prod. Posting Group");
end;

Thanks in advance!

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.
Build ID: 47021.

Hello,
we just checked and noticed that the requested parameter IsHandled is not a var parameter. Can you fix this please?

That I can. Sorry about that! It'll be with 24.0 in April, as this actually is considered a breaking change :-S


Training problem

Hello,I tried training using your model.
After one pass, it outputs this:

and the test output without data change:
Starting testing model...
Mean Squared Error on Test Data: 70.91168212890625

Is this a sign that it's working properly? Should I increase the amount of train data?

https://github.com/SunOner/mouse_net/commit/176de530502cf8ce0438cb7d8bf66d99d4f6ca3c

176de53
I tried training using the parameters you suggested,It took a long time and I don't think it was using my GPU

My CPU is occupied 60%+ for a long time,but the gpu is always at 10%+ or 20%


pixel dialogues

What feature do you want to get added on the base engine?
Pixel dialogues work for custom songs, like
corruption dead pixel in physch engine with a dialogue where it shows senpai mad saying something with that pixel dialogue thing and pixel bf typing commands with that pixel dialogue thing and also this angry sound play where senpai becomes mad in roses and same with thorns thingy but with different char dialogue thing texture and senpai dying cutscene play.
To test your sight, and reliability, please select the option of what should NOT be requested.
Stage Editor, 6K+ support, and winning icons.


Added support for watching JS files

Any updates on this?

Would this mean support for watching multiple db files?


Lack of documentation

Please, can you write some documentation for using and configuring?
At least examples of config files, and variables like host and port.
Thanks

Same here.
Copied the script into /opt/collectd_plugins/ and chmod'ed it a+x. Installed script requirements via pip install python-binary-memcached collectd.
Added this to collectd.conf:
LoadPlugin python
<Plugin python>
  ModulePath "/opt/collectd-plugins/"
  LogTraces true
  Interactive false
  Import "memcached_stat"

  <Module "memcached_stat">
    Host "memcached"
    Port 11211
    Verbose true
  </Module>
</Plugin>

The plugin runs and prints out debug messages, such as:
memcached plugin [verbose]: memcached_stats plugin: created a synthetic slab_size metric for slabid 1

But no metrics are sent (other metrics arriving fine).
@criteo: Anything else missing? TIA.


[Rn-496]: Select Inputs

It will be nice to have the inputs order by amount

moved to https://github.com/bitpay/bitpay-app/pull/242


SmartNews skill is down on Production

Yeah this is an issue with SmartNews itself: on their end they need to run a daily batch job to generate the news audio, and sometimes they don't do that, or the script fails.

Yes, I remember hearing about this.  Figured it was worth filing as a bug and assigning to someone to resolve.  Might Ed help?  Would you like me to reach out?

I'm in touch with the SmartNews team on their slack. I'll talk to Ed as well.


Batch Content Validation Issue for github.io website

Section link1,link2,link3,link4,link5:

Reason:
Format validation issue: Text exceeds page size.

Section link1,link2:

Reason:
Format validation issue: The HTML cannot be displayed correctly. There is an extra <br /> here.

Section link1:

link2,link3,link4:

Reason:
Format validation issue: Extra symbols _.

Section link1:

Reason:
Link validation issue: The link shows 404 error.

Section link1,link2,link3:

Reason:
Typing annotation issue: Missing type annotations.
@rohit-ganguly, @lmazuel, @ronniegeraghty, @cRui861, @dpwatrous for notification.

@dpwatrous please help take a look on the issue 5 in this page.

Close this issue as we'll put it into a summarzied one.


Add manifest.json for nicer mobile experience

Fixes #35
References:
https://developers.google.com/web/updates/2014/11/Support-for-installable-web-apps-with-webapp-manifest-in-chrome-38-for-Android?hl=en
https://developers.google.com/web/updates/2014/11/Support-for-theme-color-in-Chrome-39-for-Android?hl=en

I like that a lot. It's just a color, but it sure does make The Lounge feel a lot more native/integrated! 👍

Do we need a theme-color in the manifest file and client/lounge.js?

Alright, I guess I can't win then :-)
👍 for now, hopefully this will get improved over time. I'll post some screenshots shortly (got to run now) to keep a ref somewhere when someone else decides to improve this.

So, for the record, without this patch applied, we have:

First picture is in browser, others are "browser-less". Notice the black status bar (battery, connectivity, time, ...) across both modes.
Applying this patch, it works very nicely with the default theme:

Notice that the URL bar in the browser mode has changed, but so has the status bar in all modes. It is now a dark blue instead of black.
As a matter of fact, it works also very nicely on the Morning theme:

But... Not so much on Zenburn and Crypto themes:

I am using the default theme so I personally don't care too much, I know @xPaw is using Morning. Not worth reverting this at all, but good to keep in mind in case someone opens an issue for Zenburn or Crypto.


Add getProcessPIDByName to get PID by name

The idea is to include basic tooling to get process ID if injected DLL is using it.
In my case, injected dll creates a named pipe that includes PID, having a way to grab that while we're at it is better than having to use yet another library for it.

Thanks for your help
Since i just changed a lil bit of the struct of this repo your pr will not gona work
Could you please resubmit your changes based on the dev branch ^^


Typos in README.md example: :hello vs :HelloController

(On Crystal 0.22.0 and amber:master)
In the README.md example, this line did not work for me:
get "/", :hello, :world, :api
I had to make this change to make it compile:
get "/", :HelloController, :world, :api

Nevermind. It was a mistake on my part.


Research and implement an alternative to Glyphicons

 Issue by jennyhavah
Thursday Dec 14, 2017 at 07:28 GMT
Originally opened as https://github.com/mercycorps/mcta/issues/16

As it turns out, our icon font is incompatible with open source licenses -- even the paid version. Glyphicon explicitly says, "You must not include icons in open-source projects, since this license is not compatible with most of the open-source licenses."
See: See http://glyphicons.com/license/
I did a very cursory search and found Fontawesome  -- http://fontawesome.io/license/ -- fully open source and GPL friendly
GitHub also has some resources we could check out: https://github.com/topics/icon-fonts
Since we don't use tons of icons in TolaActivity, it will (hopefully?!) not be too painful to swap out our icon font service and update our icons.

Misunderstanding in the framework being used


Getting console warning when type mention (@, #) key

Description: When using the  component with the items prop, React throws a warning:
Warning: React does not recognize the items prop on a DOM element.
This happens because items is being passed to a native DOM element, which React doesn't recognize as a valid attribute.
Steps to Reproduce:
Use the  component and pass an items prop, like so:
<BeautifulMentionsPlugin items={mentionItems} />
Check the console to see the warning message:
Warning: React does not recognize the items prop on a DOM element.

Expected Behavior: React should not throw a warning, and the items prop should either be handled properly within the component or removed before being passed to any DOM elements.
Actual Behavior: The items prop is passed down to a DOM element, which causes React to throw a warning.

Hi @akashmdiu, the items, selected, and itemValue props are provided to help building custom menu items. If you don't need this customization, you can simply exclude these props when spreading the remaining props to the DOM element.
export const MenuItem = forwardRef<
  HTMLLIElement,
  BeautifulMentionsMenuItemProps
>(({ selected, item, itemValue, ...props }, ref) => (
  <li
    ref={ref}
    className={cn(
      "relative select-none ...",
      selected && "bg-accent ..."
    )}
    {...props}
  />
));


Reorder inventory items

Requested Changes

[ ] Inventory items should be displayed left-to-right and not like the current right-to-left order.
[ ] When an inventory item is used and there is more than a single item in the inventory, the other items should move as far left as possible.

@Monkey-bored could you please update your progress regarding this issue?


Add system for addition and overriding of custom data

Details
One of the most attractive features will be modding support. To that end, a system for adding and changing arbitrary data should be created. Data can take any shape, and should be stored somewhere for fast lookup. For example, one could create a new character definition.
chara "my_chara" {
    name = "my_mod.locale.chara.my_chara" // localized string
    level = 40
    // ...
}

All data will use string IDs of the form <mod name>.<namespace>.<more namespaces>. The definition above will generate an ID based on the mod it originates from, since mods are given unique names. So for a mod named my_mod, the above would generate my_mod.chara.my_chara.
In C++, this would be parsed into an HCL object and stored somewhere. Then, when the character's information is looked up:
auto definition_data = mod::s.get("my_mod.chara.my_chara");
std::cout << definition_data["name"] << " " << definition_data["level"] << std::endl;

This would replace some parts that would otherwise be baked into the C++ code and unavailable for tweaking.
Another feature the system should support is overriding of data. Players should be able to change the properties of any loaded definition like so:
override chara "my_mod.chara.my_chara" {
    level = 9999
}

These would be resolved after all the base definitions are loaded.
The character loading is only an example; all sorts of data should be editable this way. With this system in place, it should be possible to add an editing system so people can edit things directly inside the game, and many more properties that are baked into the code can be externalized.

Implemented.


Expose metrics for rig-platform and rig-operator

It should be possible to get basic request and system metrics for the services.

rig-operator has metrics exposed at :8080/metrics. We should probably add a section in the docs about this.

We could also add an option for creating a ServiceMonitor resource in the helm chart.


chore: add warn for headless: true

Warn for future behavior change of puppeteer.launch({headless: true})
The message can be suppressed by setting:
Env - PUPPETEER_LOGLEVEL=silent
Config - logLevel: 'silent'
NPM LogLevel - https://docs.npmjs.com/cli/v9/using-npm/logging
Warning:

let's mark it as a feat to make it show up in the changelog


adding header information to every message (useful for passing extra …

…auth parameters in channel)
also channels may contain wildcards such as **, so account for that when checking whether the current message is from the recursive channel

Hello @rrupsing . Thanks for the PR. At the moment I'm thoroughly looking at a large PR submitted #20 that might address some of your issues. It's taking me a little longer than expected to wrap that up. Once I do, it would be great to get your feedback on some of the things you've addressed in this PR. I think the one crash fix you have might be fixed in #20 but I do love the wildcard channel check so I definitely think we should get that in. TLDR, give me a little time but really do want your feedback and contribution once i have things wrapped up :)

@hamin this feature is possible in the other PR, am actually doing this as well but i went for the subclass model approach

@rrupsing closing the PR, I just merged #20 which includes these same changes, try out master and let me know what you think


Fix routing system so users can't access certain pages without loggin in

@NSel1727 Questions for you before I can approve this (these are my main concerns):

You are using "token", does it refer to the JWT token? or which token exactly?
In some you use "token" and others it is the ("ID"), was that intended? do you plan to have the ID everywhere later?
If as a user, I try to modify the path on the webpage to access some other student's pages or an org pages, your way blocks it, yet what would happen if I try to do that, where would I be redirected? (couldn't locate the setter method you have for this)
I checked that these changes were applied to all pages, yet can you confirm that as well?

Thank you.

I think it looks good 👍🏽
Adding on to Yohan's questions (feel free to add/correct @NSel1727 )

I believe it's JWT
It should be intended, storing ID can be useful for easy-access retrieval for other pages, since it never changes
Modifying web page path leads user back to login page (App.js file checks for role and redirects)
I went through all pages and it seems that the changes were applied throughout


Jenkins2 - TEST, PLEASE IGNORE

test to verify that jenkins ignores PRs from non-collaborators

Codecov Report

Merging #2226 into develop will increase coverage by 0.13%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           develop   #2226      +/-   ##
==========================================
+ Coverage    70.07%   70.2%   +0.13%
==========================================
  Files          689     689
  Lines        50730   51278     +548
==========================================
+ Hits         35550   36002     +452
- Misses       15180   15276      +96

Impacted Files
Coverage Δ

src/ripple/core/Stoppable.h
93.33% <0%> (-6.67%)
:arrow_down:

src/ripple/app/ledger/PendingSaves.h
93.54% <0%> (-6.46%)
:arrow_down:

src/ripple/beast/hash/xxhasher.h
66.66% <0%> (-6.07%)
:arrow_down:

src/ripple/rpc/impl/LegacyPathFind.cpp
85% <0%> (-4.48%)
:arrow_down:

src/ripple/app/tx/impl/BookTip.cpp
88% <0%> (-3.67%)
:arrow_down:

src/ripple/protocol/impl/BuildInfo.cpp
79.16% <0%> (-3.45%)
:arrow_down:

src/ripple/app/paths/impl/Steps.h
81.63% <0%> (-1.71%)
:arrow_down:

src/ripple/conditions/impl/utils.h
68% <0%> (-1.45%)
:arrow_down:

src/ripple/beast/asio/io_latency_probe.h
95.77% <0%> (-1.25%)
:arrow_down:

src/ripple/core/impl/JobQueue.cpp
86.11% <0%> (-1.22%)
:arrow_down:

... and 204 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fc64050...61ca48a. Read the comment docs.


Maintenance of this project?

Hi Elliot :wave:
It's been a while since there's been any updates to the repo (August 2021). It's a really valuable project and I would love to see it continue. Would you be interested in finding a new maintainer?
I saw that @posiczko has made some updates recent in the fork at https://github.com/posiczko/real-world-rails/

Hi Andy. I hope to be able to get back to maintaining this repo in the near future. Glad to see a fork is being given some love 👍

Hi @eliotsykes, it's been a year since your reply, do you have any updates to share?

No updates. I'm taking a peek at it this morning.


Bug: Special characters in Voucher Groupname, cannot delete role

When i generate voucher role with special characters i cannot delete the role group.
Only the codes are deleted.

The voucher group name is used in the URL, so some characters may not be used. I will include some cleansing on the input data when generating the vouchers.


Status Field of Kind: NfDeployment  Logic Addition

Status Field in Kind: NfDeployment is populated appropriately.

/approve
/lgtm


log file name doesn't change

Took codebase from : https://github.com/CoreyMSchafer/code_snippets/tree/master/Decorators
I added below snippet at the end of decorator.py and it should have created log file named as check_kw.log
However, it didn't do it. It just appended the logs created by this function to display_info.log file.
Is there anything else need to change in my_logger?
@my_logger
@my_timer
def check_kw(*args, **kwargs):
    time.sleep(1)
    print('check_kw ran with arguments ({}, {})'.format(args, kwargs))

keywordargs = {"arg3": 3, "arg2": "two", "arg1": 5}
check_kw(**keywordargs)

@shivrajjadhav733 Did you ever figure this out? I don't check GitHub issues as much as email, so in the future if you have questions it's probably best to ask there<EMAIL_ADDRESS>Or comment on the video in question


[feat] Ability for Embassy-SDK to Take Multiple Arguments

This would allow the ability to install/uninstall lots of services at once, for example

Can you list which commands you want to do this to? I don't think it makes sense to do this "globally", whatever that means.

I gave 2 examples - there may be other places where it could be useful, but those are the 2 I would like.  Service updates is probably another, although an "update all" is probably already in progress.  Globally makes no sense

Ok. Renaming to reflect. Also you're talking about embassy-cli here, not embassy-sdk.


Ability to set height of the overall interface

Hey! I love the app, but since most of the apps I use are not uwp, icons look pretty bad. I saw the previously closed issue about poor icon quality.
What about adding ability to set interface height in the config? So, if anyone didn't like the icon quality could decrease the interface height and somewhat avoid icons being so pixelated ?
Thank you for the work that you do!

quality of icons was fixed in PR 71. No need for this issue anymore


Add check_interval to example generators

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Files
Coverage Δ

src/startorch/example/circles.py
100.00% <100.00%> (+7.40%)
:arrow_up:

src/startorch/example/swissroll.py
100.00% <100.00%> (ø)

:loudspeaker: Thoughts on this report? Let us know!.


Add unwrap promise type

Solution's here: https://stackoverflow.com/questions/48944552/typescript-how-to-unwrap-remove-promise-from-a-type
Typescript discussion:
https://github.com/microsoft/TypeScript/issues/28105

We could add something like this until Typescript guys come to some agreement.
The big question is: would we want to do it only for Promise, or for all "promise-like" / "thenable" objects?
@deser you can propose a PR if you want.

I found this promise-all but on object function super useful. I even asked very similar SO question here: https://stackoverflow.com/questions/46878822/typescript-typings-for-promise-all-working-with-object
The problem is that this type is useless without it's implementation. It's very specifc and that's why I dont that adding it here makes sense. Better idea would be to create a separate package promise-all-object with correct typings.

Yeah, I also feel that typing for whole objects with "promised properties" belongs in a package that implements such a function.
What about a simpler case though? This type could potentially be useful in more contexts (name TBD):
type PromisedType<T> = T extends Promise<infer PT> ? PT : never

@quezak yeah this makes sense to me. Perhaps it could be named Promised?

Promised doesn't sound good to me, intuitively I'd think it works the other way (it wraps a type in a promise, while we want to unwrap it), that's why I went with PromisedType initially.

Why not UnwrapPromise?

Why not UnwrapPromise?

While it explains well what does the type do, I feel it doesn't fit the convention of type naming. Looking through TS's own types, and our existing ones in ts-essentials, generic types are never named like instructions, but rather descriptions of the resulting type. IMO this way feels more natural when you spell out the code in a full sentence. Say you have const p: Promise<number> and const n = await p, then you say that "the type of variable n is the promised type of Promise<number>", while for me it would feel weird to say "the type is unrap promise (...)". That's why I'm voting for PromisedType<T>, but feel free to convince me otherwise :smiley:  @krzkaczor opinions?

Okay so what do you think about:
type PromisedType<T> = T extends PromiseLike<infer PT> ? PT : never;
type AsyncOrSyncType<T> = T extends AsyncOrSync<infer PT> ? PT : never;

Naming suggestions? We could make it a pattern across whole lib... @quezak

The AsyncOrSyncType name is not 100% obvious, but I don't have any better ideas, and it's not too bad.
I like @krzkaczor's proposal, ie. that PromisedType handles all PromiseLike objects. @boenrobot the difference between the two proposed types is that it doesn't "fail" on non-promise types:
PromisedType<Promise<number>>  === number;
PromisedType<number> === never;

AsyncOrSyncType<Promise<number>> === number;
AsyncOrSyncType<number> === number;

Plot twist:  looks like this will be built-in as an awaited keyword, see PR here, it was initially scheduled for TS3.9 but was pushed back (see release notes).
Should we name the type Awaited<T> then, so it's easier to migrate later?

It's done in version 7 as awaited type


chore: update bytecode_compression.md

My suggestions regarding this section are as follows:
1. contracts use in our chain must be copied into L1 (so that the chain can be reconstructed from L1 if needed). - It should be 'contracts used in our chain'.

2. Looping over the encoded data, derived from _rawCompressedData, the corresponding chunks are pulled from the dictionary and compared to the original byte code, reverting if there is a mismatch. - Better phrasing: 'Looping over the encoded data, derived from _rawCompressedData, the corresponding chunks are retrieved from the dictionary and compared to the original bytecode. If there is a mismatch, the operation reverts'.

3. the latter of which comes from the output of the server’s compression algorithm. - Better phrasing: 'the latter of which comes from the server’s compression algorithm output.'

What ❔

Why ❔

Checklist

[ ] PR title corresponds to the body of PR (we generate changelog entries from PRs).
[ ] Tests for the changes have been added / updated.
[ ] Documentation comments have been added / updated.
[ ] Code has been formatted via zk fmt and zk lint.
[ ] Spellcheck has been run via cargo spellcheck --cfg=./spellcheck/era.cfg --code 1.

for that you'll need a local dev environment set (see https://github.com/matter-labs/zksync-era/blob/main/docs/guides/setup-dev.md). Then just run zk fmt and commit the changes! Thanks.

Closing as stale


[stable/locust] WebAuth Basic64 encoding is incorrect

When master.auth.enabled=true, master-deployment.yaml template generates base64 {username}:{password} string here.
https://github.com/deliveryhero/helm-charts/blob/master/stable/locust/templates/master-deployment.yaml#L138
It uses cat function, but it inserts white spaces between strings.
https://masterminds.github.io/sprig/strings.html#cat
Given
master
  auth:
    enabled: true
    username: "myuser"
    password: "mypassword"

BAD
value: Basic {{ cat .Values.master.auth.username ":" .Values.master.auth.password | b64enc }}
  -> it generates base64enc from "myuser : mypassword" ( white spaced )

GOOD
value: Basic {{ printf "%s:%s" .Values.master.auth.username .Values.master.auth.password | b64enc }}
  -> it generates base64enc from "myuser:mypassword" ( no white spaced )

Closing due to inactivity. Feel free to reopen if you have any updates.


Horrible performance and memory leaks in modern pages and SPFx

Category

[ ] Question
[ ] Typo
[X] Bug
[ ] Additional article idea

Expected or Desired Behavior
Modern pages and SPFx in general should work in all supported browsers. Thus it should also work without any issues in IE and Edge. Both are supported for SharePoint Online and O365 in general.
Observed Behavior
The performance especially in IE on modern pages is horrible and very bad. Also there are huge memory leaks that lead to errors within sharepoint and crashes of IE. This does not happen in Chrome!
This is a memory profile after refreshing a site with IE about 10-12 times. Also look at the time how long those few refreshes take and how slow the page is. It is a serious issue! The used memory will not get freed up. It will grow until the browser crashes/hangs or until sharepoint throws an "SPFx loadmodule not found" error. Overall, currently modern pages are totally unusable for IE and Edge users. Although it is a little bit better in Edge.

Steps to Reproduce
It is very easy to reproduce on modern pages with some content added to it. The more data and the more webparts are on the page, the worse gets the performance and memory leak. Just add content to a modern OOTB group site/communication site and add a couple of webparts. With custom webparts it will get worse, but the leak also exist just with ootb webparts.

I can just verify the issue. We have also observed the same problem and as mentioned it very easy to reproduce the issue.
The most critical part is, on enterprise level the IE is still the main browser and could not be replaced very easily. Because of these performance issues almost all our enterprise customers are rejecting to switch to the modern pages. This also kills the SharePoint transformation to modern and as a consequence all these new features.

As this is not really developer issue, rather happening apparently constantly in the pages with IE, we cannot, unfortunately, process this through this channel. Please report this from the Office 365 Admin support channels as we can only concentrate on pure developer topics. Sorry for the inconvenience and slow response time, but closing this for now as we can't handle this. Please do report these in Tenant admin process. Thx.

I think you should reopen this issue as it is directly related to SPFx. Since the OOTB Microsoft WebParts are also written using SPFx (more or less....) they also have this problem. I was just using those in my example so we don't even have to talk about issues related to custom code. But in fact, every custom webpart has this issue. So it is directly related to SPFx and it is a developer issue. Everyone that writes webparts using SPFx automatically excludes IE and Edge users. You should seriously have a look at this again, because the SPFx team is the only one that can fix this. IE Team won't do that for sure.... It does not help customers out there, if the SPFx Team is using Chrome as there no1 dev platform and don't even test there own stuff on other browsers. I am just trying to help and provide feedback. But if that is not welcome, I will stop and advise customers that are still using IE/Edge to stay on classic...

Hi @OliverZeiser - we have clear rules and processes on reporting these things. If this is caused also by oob web parts, it's not a development issue and we simply cannot report these forward from this route. There is no point re-opening the issue as that does not change that fact. We have an agreement to get all non-dev stuff routed using alternative routes and feature crews will simply ignore them if we try to route this to them from here... as this is not owned by our team.

Now - on this - you can also always read between the lines and show that the leak happens also with custom web part, which then would result opening a new issue, attaching sample solution in for the repo. That would make this a dev topic. We can't process this item as it's posted, but if @OliverZeiser, for example, has issues explicitly with the 3rd party custom ones, please report that as a new issue and we can assign that to the right people. Just guiding how to get work done on this.


perf: remove lto setting from CLI

What kind of change does this PR introduce?

[ ] Bugfix
[ ] Feature
[ ] Docs
[ ] New Binding issue #___
[ ] Code style update
[ ] Refactor
[x] Build-related changes
[ ] Other, please describe:

Does this PR introduce a breaking change?

[ ] Yes, and the changes were approved in issue #___
[x] No

Checklist

[ ] When resolving issues, they are referenced in the PR's title (e.g fix: remove a typo, closes #___, #___)
[ ] A change file is added if any packages will require a version bump due to this PR per the instructions in the readme.
[x] I have added a convincing reason for adding this feature, if necessary

Other information
This PR removes the lto=true setting for the CLI. This drastically improves the install performance (the time it takes to run cargo install tauri-cli).
Since the CLI is not doing performance-critical work and size is not important here since the cli will be compiled locally this does not have a downside it it afaik and only improves performance for rust users.
To give you an idea here are the install times on my MacBook Pro m2:

build
time

with lto
2:55.31

without lto
1:05.46

so as you can see disabling lto reduced the compile-time by 3x.

what do you think about moving the lto (and the new stuff you added in the js api) into a custom profile that inherits the release profile so that we can use that for the cargo-binstall CI?

what about binstall?

I had to use patch-package because napi-rs does not support custom profiles :(

Also @JonasKruckenberg why did you target next here instead of dev?

I had to use patch-package because napi-rs does not support custom profiles :(

I would probably look into upstreaming this to napi-rs, the maintainer is pretty responsive and would probably publish it soon after merging.
Also if I understand correctly, this change wants to improve cargo install tauri-cli which iirc will use release profile by default which doesn't have this change and users would need to cargo install tauri-cli --profile release-size-optimized which is bad DX

He's focusing on v3 so I'm not sure if he wants to patch for v2. I'll try anyway.

I had to use patch-package because napi-rs does not support custom profiles :(

I would probably look into upstreaming this to napi-rs, the maintainer is pretty responsive and would probably publish it soon after merging.
Also if I understand correctly, this change wants to improve cargo install tauri-cli which iirc will use release profile by default which doesn't have this change and users would need to cargo install tauri-cli --profile release-size-optimized which is bad DX

I think you misunderstood. We don't want users to use release-size-optimized, we want them to have faster builds. The size optimization will be used only on the cargo-binstall and NPM assets.

ah right, I was even more confused by the name. LGTM then.

@CrabNejonas we also need to rebase this branch with commit signing :cry:


Support for permissions boundary on IAM-roles

Describe the outcome you'd like
An optional value to define permissions_boundary on IAM-roles being created by AFT.
Is your feature request related to a problem you are currently experiencing? If so, please describe.
This is required to be able to deploy AFT in a master account from a reseller that enforces IAM restrictions on that specific account to ensure that they keep control over the account and hides the billing kickback details. This is done by requiring a permissions boundary to be set on all new IAM-roles. This makes the default AFT setup to fail when creating the roles unless the reseller lifts the restrictions temporarily. This involves creating a support ticket and the waiting involved. Until AWS provides functionality for resellers to maintain control over a customers master account we need this feature to be able to deploy AFT in development repeatably and without manual steps.
Additional context

Hey there, thanks for writing this up. I'll create a backlog item for this feature request, but I just have a clarification question about your use case. If I understood correctly, the request is that when new AFT deployments in new management accounts are established, those deployments have the permissions boundary applied to all roles created by AFT in that environment. Would that include the accounts that are vended by that AFT environment as well?

For us and our reseller, this only applies to the master payer account (with Organizations and Control Tower). The vended accounts (or other accounts like the AFT management account) created through Control Tower or Organizations does not have this restrictions.

Having the same problem here. As the permissions_boundary is part of aws_iam_role resource there is no way to assign a boundary that is rolled back automatically. Please allow us to pass in an iam restrictions.


Update documentation for the floating point type, expressions and elements of the DSL

[ ] Create a summary on floating point support in Theta akin to bitvectors.md
[ ] Update the description of the CFA formalism in README.md
[ ] Add some example CFAs to showcase the new expressions (also suitable for testing purposes)

I think point number one is optional. It is nice to have a long and detailed description if you have time, but the important thing is to update the CFA formalism readme (point 2). You might just list the available operators/expressions there.

Please see PR #127, I think I covered all important aspects for point 2.


UX Research - Planning & Recruitment - Scanning

Title of ticket: UX Research - Planning & Recruitment - Scanning
Description
Creating the research plan and study guide for the second round with Scanning & Tagging. And book in about 4 sessions
Dependencies
Are there any dependencies?
DOD

[x] Meeting to Review with Lise
[x] Email Team Leads for suggestions for Participants
[x] Draft Study Guide
[x] Draft Research Plan
[ ] Review Research Plan & Study Guide with Lise
[x] Book in sessions

@liseandtea @KyEggleston (not sure if you're able to attend either session to take notes)
Find documents linked below
Research Plan: https://bcgov.sharepoint.com/:w:/t/03521/EZZB3RI2ZNVCgfeXlcjUMsEBsVpRS7aCBaENscB7pa9oHw?e=7E0uPu
Study Guide: https://bcgov.sharepoint.com/:w:/t/03521/EdpmwI7gamhElge1cHi5HOoBbotHYR7pDdV6g1fTF5ykoQ?e=vmDm1R
Notes Doc: https://bcgov.sharepoint.com/:w:/t/03521/Eaef8b1-DHRFic7o9mNv_7oBDJa1Gz5lNWQe2-bvPYiOxQ?e=ewHUDV


Puppet 4 Support

Initial support for Puppet 4.  Removes errors thrown by the runtime as facter is
now a binary and doesn't have the libraries required by this module.  This removes
the dependency by performing all library routines in native ruby.  The max interface
speed code looked suspect so I've cleaned that up and ensured it works on 16.04.

Looks to work nicely...
[08/09/16 15:54:16] simon@symphony ~/src/github.com/spjmurray/puppet-ethtool $ puppet facts find `hostname -f` --modulepath .. | python -m json.tool | grep speed
        "maxspeed_enp0s25": "1000",
        "speed_tun0": "10",
        "speed_tun1": "10",
        "speed_tun2": "10",
        "speed_vboxnet0": "10",
        "speed_vboxnet1": "10",
        "speed_vboxnet2": "10",
        "speed_vboxnet3": "10",
        "speed_vboxnet4": "10",
        "speed_vboxnet5": "10",
        "speed_vboxnet6": "10",
        "speed_virbr0_nic": "10",


Flakes analysis on IBM cloud tests

/kind tests
we need some way to analyse the flakes observed as IBM Clouds tests to make tests more reliable
https://github.com/kadel/odo-tools fetches logs from https://search.ci.openshift.org, but since we stopped using Prow, this tool can be modified to fetch logs from IBM cloud.

To be worked on during DevTools week (June 2023).


docs: Using container registry for VM images

This proposal is a missing piece in combining VMs and pods into the same user workflow. We are already working out the details required to connect pods and VMs into the same overlay network. With this proposal we'll be able to converge pods and VMs into the same image repository.
Soon both the storage and network story will be the same for pods and VMs.  Nearly all the friction points involved with running VM workloads in an existing k8s cluster will be gone.

I'd argue that when using Cinder, for example (for storage), it's a better idea to use Glance over registry, due to the tight integration between the two.

To be clear. This is not a proposal to exclude something like cinder+glance. This proposal is to provide the glue necessary to use the constructs already present in a k8s cluster for vm images.

I've created a separate PR describing how we can fit the concept of ephemeral storage into our current disk model, and at a very high level mentioning how we can populate PVCs from external sources like docker repos, independently from the VM startup code flow, which I think is preferrable to baking direct knowledge of container repos into virt-handler VM startup. https://github.com/kubevirt/kubevirt/pull/311

To me the notion of this proposal is swimming a little between

I want to use the docker registry for storing my VM image and
I want to use any docker image to boot my VM

And then there are the taints of local storage and ephemeral disks.
That's at least what I read in this proposal. So let me get to them one by one:

Storing VM images in a docker registry has it's own challenges, we need logic to separate docker images form vm images.
Using docker images to boot VMs sounds reasonable, and can probably be even achieved, but I'd consider a specific disk source type in the VM API
On the ephemeral disks side, I favor @berrange approach, as it rather strengthens some primitives, like "copy modes" to build the concept.

At the bottom line - Yes, I agree that we should also see how we can consume container images, but to me it's not so critical right now, and if we do it, then I'd rather consider an explicit way in the api

At the bottom line - Yes, I agree that we should also see how we can consume container images, but to me it's not so critical right now, and if we do it, then I'd rather consider an explicit way in the api

I completely disagree about the priority here. This needs to happen now along with CNI integration.
If we want to drive adoption and gain traction in the community, we have to make KubeVirt usable. That is where we find success.
Having the ability to both store and launch VM Images from the container registry is huge on that front. It also gives us a real use case.  People who want to launch VMs backed by local storage (just like stateless containers are done) will be able to do that just by dropping KubeVirt into an existing kubernetes cluster.  We'd immediately have a path for untrusted workloads. No additional block storage need, no configuring of iscsi targets required. Just take an vm image, upload it to the registry, and post a VM manifest.
No one (that I have seen) has successfully shown VMs and Containers converged into the same user workflow... We'd have that.

I completely disagree about the priority here. This needs to happen now along with CNI integration.
If we want to drive adoption and gain traction in the community, we have to make KubeVirt usable. That is where we find success.

+1 I share that view.

From a converged workflow pov, i find enabling the clear containers style approach more compelling than storing qcow2 images inside docker images, which is a rather inefficient way of managing qcow2 images imho.

At the bottom line - Yes, I agree that we should also see how we can consume container images, but to me it's not so critical right now, and if we do it, then I'd rather consider an explicit way in the api
I completely disagree about the priority here. This needs to happen now along with CNI integration.

…

Having the ability to both store and launch VM Images from the container registry is huge on that front. It also gives us a real use case. People who want to launch VMs backed by local storage (just like stateless containers are done) will be able to do that just by dropping KubeVirt into an existing kubernetes cluster. We'd immediately have a path for untrusted workloads. No additional block storage need, no configuring of iscsi targets required. Just take an vm image, upload it to the registry, and post a VM manifest.

Yes, it would be really slick to tell this story - I do not disagree.
And IIUIC it should even work today without code changes:

Create wrapper based on iscsi target, deploy
Use iscsi disk type and reference podname.svc.cluster.local (libvirt needs to be in pod)

I do like the idea of (re)using the docker registry for VM image storage/retrieval. However, this solution looks a little to hand crafted to me (specific container, specific disk type for this).
If we can come up with a more generic story, i.e. how we can consume files in arbitrary docker images, or docker images in general, then it would look more appealing to me.
My take away from the two discussions so far is:

We obviously want to support (persistent) volumes for disk i.e. #266
Not so obvious, but it does not look unreasonable to use container images as disks as well #315

From a converged workflow pov, i find enabling the clear containers style approach more compelling than storing qcow2 images inside docker images, which is a rather inefficient way of managing qcow2 images imho.

I understand what you're saying, and just for clear containers you're probably right.
What I'm trying to get across is we have the opportunity to converge the workflows for building and deploying general purpose VMs with containers. (I know I spoke out on the untrusted workload bit, that alone isn't a strong argument and I shouldn't have brought it up specifically)
Our first iteration at this is not going to be inefficient. Yes, qcow2 in docker registry with conversion to raw at runtime is not efficient... We'll figure out a more efficient option for that in time. From the user's perspective all that is abstracted away though.  All they see is that there is a single path for containers and general purpose VM deployment in the datacenter.
It's very important that we continue this convergence path. If that means we have to sacrifice efficiency initially, then so be it. The workflow is everything.
Get the workflow right, then iterate on the underlying technology.  Simplicity then efficiency.
I believe this is the key to our success.

The problem that I don't see a way to fix the inefficiency problem - it looks inherent to the very concept of storing disk images inside docker images.
On create end you have

Create a qcow2 image
Run some "install" process to populate it
Create a docker image including the qcow2 image
Upload the docker image to the repo

If we assume we've go 2 GB of actual content, that's 2 GB of writes in step 2, and another 2GB of writes in step 3, so 4 GB in total
On the VM startup end we have

Pull down the docker image, saving locally
Extract the qcow2 file from the image
Convert the qcow2 file to whatever format we need

Each of those steps is another 2 GB of writes each, so 6 GB in total.  Compare with storing the disk images directly without having to wrap them in a docker image, we're saving 2 GB of writes at both phases. Or conversely by storing the raw contents of the disk in a docker image, and only create the qcow2 file at time ot boot, we again save 2 GB of writes on the startup side.
IOW, the only ways I see to optimize the "storing disk images inside docker registry" involve not storing disk images in the docker registry. This makes be prefer the idea of going straight to the solution of storing disk image contents in docker images, instead of the disk images themselves, or to not use the docker registry for disk images - instead have some different image registry service that conceptually works in the same manner as docker, but without having the tar.gz wrapper around the disk images

Each of those steps is another 2 GB of writes each, so 6 GB in total.

nah, it's just one write for the conversion process.

pull down container image with qcow2 file
start container which converts qcow2 file (the read only copy... see the notes on union mount filesystem) to raw.

If we don't actually make a write to the qcow2 file, it will only exist once on the local machine regardless of how many containers are accessing the file.
This isn't that much more inefficient than how people use containers today on local storage. It just seems weird at first because it is not how we're use to thinking about deploying/distributing VM images... and to reiterate, VM images in the container registry will never be the only way to interface with KubeVirt.  If someone values efficiency that we simply can't match with VMs distributed by the registery, then there will be other options for them that deviate from the container workflow.
I see optimizations coming in the form us figuring out how to use the qcow2 file stored in the container registry directly without needing the raw conversion bit.  If we find we're truly on to something here from a usability standpoint, we'll figure that part out.
In most cases, VMs are going to require more disk space than containers. That's the cost of doing business with VMs

FWIW, despite what I said, I don't really have any strong objection to implementing the docker registry concept. If combined with my proposal https://github.com/kubevirt/kubevirt/pull/311, the docker registry code would end up being entirely self-contained code, running in a self-contained POD. IOW it doesn't have any ripple effect on the rest of our code base.  My objections are more a question of what we thing will be the best "default" recommendation for image mgmt, which is really tangential to whether we implement this or not.

I do like the idea of (re)using the docker registry for VM image storage/retrieval. However, this solution looks a little to hand crafted to me (specific container, specific disk type for this).
If we can come up with a more generic story, i.e. how we can consume files in arbitrary docker images, or docker images in general, then it would look more appealing to me.

I don't think this is a problem. Users are not going to perceive the wrapper container image as something weird.
For example... Last year I was working on a node.js app. I just used the official node.js container image to package/deploy that app in.  I used the same pattern when I deployed a golang app... I found some official go v1.7 container and built my app in it.
Using that pattern... Now I have some general purpose VM I want to deploy on my cluster. I just go and use the KubeVirt container image.
From the user perspective, VMs just become things you deploy naturally into the k8s cluster like anything else.

FWIW, despite what I said, I don't really have any strong objection to implementing the docker registry concept. If combined with my proposal #311, the docker registry code would end up being entirely self-contained code, running in a self-contained POD. IOW it doesn't have any ripple effect on the rest of our code base. My objections are more a question of what we thing will be the best "default" recommendation for image mgmt, which is really tangential to whether we implement this or not.

@berrange I'm with you on this. The points of discussion are on implementation details. If we decide to support this idea, the exact way I outlined this working in this PR probably isn't what we'll actually land on.  I think you had some good points in your storage proposal that should influence this.
Given what we have now, I think it's kind of difficult for us to take this idea too much further with high level discussion.  I'll do a POC from the approach I was thinking about and we'll see how to iterate from there to come to some common agreement on next steps.

doc update to reflect implementation details in PR #325

@rmohr are we good to merge this too now that initial implementation landed?


SSR compatibility

Describe the bug
I've implemented the chat into my vue project. The chat works correctly in the client side rendering version, but in the server side rendering version I get an error ReferenceError: HTMLDivElement is not defined.
Additional context
Does this project support server side rendering ?
If so, how do you get it to work ?
Thank you

Are you using Nuxt or any other SSR framework?

We don't use nuxt, we use @akryum/vue-cli-plugin-ssr, vue-server-renderer and expressJs

Ok. I can work on SSR compatibility but could you create a minimal github repo where I can reproduce this issue?

I think we have solved this problem. One of my teammate made a plugin like this.
import Vue from 'vue'

async function initialized () {
    if (!process.browser) { return false }
    await import('vue-advanced-chat/dist/vue-advanced-chat.css')
    const VueChatWindow = await import('vue-advanced-chat')
    const ChatWindow = VueChatWindow.default

    // chat config plugin
    Vue.component('chat-window', ChatWindow)
}

initialized()

Then the chat works fine.

Looks nice, but I'm not very familiar with that.
I would be happy to accept a PR if you or your teammate could add this it inside the library so everyone can benefit from it :)

Or at least add this to the README so other people can have a solution to make the component work with SSR

Closing this as a solution is provided

This is not solution, it is looks like work around for SSR environment. Component is still not SSR compatible.

To actually make it work with SSR, you have to register it globally by adding it in plugins property and set mode to client  and also wrap the component with client-only component from Nuxt


chore(website): add some admonitions to 25.x

Summary
FIXES #12495
Test plan

@SimenB please review

please review the PR

Can you sign the CLA?

Can you sign the CLA?

Ya Sure:)
Done!

There's also a prettier error

There's also a prettier error

Can you please tell,how I can resolve it?

https://github.com/facebook/jest/blob/3c6f14bc51a857bb0baf8e6ff7af43aaf47951e9/package.json#L103

Thanks!


[FEATURE]: support update, insert and delete after with

Currently only select is supported after with (WITH clause).

Was released in<EMAIL_ADDRESS>https://github.com/drizzle-team/drizzle-orm/releases/tag/0.29.5


Configurations disableTimepicker and disableClearButton broken for Angular 1.3+

To repro (take disableClearButton as example):
Step 1: Set them to true with ngQuickDateDefaultsProvider
    app.config(function(ngQuickDateDefaultsProvider) {
      return ngQuickDateDefaultsProvider.set({
        disableClearButton: true
      });
    });

Step 2: Re-enable the clear button with attribute:
    <quick-datepicker name='myDate' ng-model='myDate' disable-clear-button='false'></quick-datepicker>

The clear button is expected, however it does not show up.
Plunker here: http://plnkr.co/edit/kXDyJ8OWdswHIIjR97Hd?p=preview
Actually the test case Given that it is configured without the timepicker and given a datepicker with timepicker re-enabled in configuration-specs.coffee fails if run with Angular 1.3+.

A bit debug and search shows it is related to Angular's break change in 1.3.0: https://github.com/angular/angular.js/commit/bdfc9c02d021e08babfbc966a007c71b4946d69d

fix(core): drop the toBoolean function
So far Angular have used the toBoolean function to decide if the parsed value is truthy. The function made more values falsy than regular JavaScript would, e.g. strings 'f' and 'no' were both treated as falsy. This creates suble bugs when backend sends a non-empty string with one of these values and something suddenly hides in the application

Thanks to lgalfaso for test ideas.

BREAKING CHANGE: values 'f', '0', 'false', 'no', 'n', '[]' are no longer treated as falsy. Only JavaScript falsy values are now treated as falsy by the expression parser; there are six of them: false, null, undefined, NaN, 0 and "".

Using isolate scope instead of reading configuration from attrs should mitigate this.

Sorry, it should remain open here.

hey @gildorwang! Great job you did there. Do you mind making PR against this repo so I can merge it the fix here ? :)


Support Pixtral Large

Mistral AI just released Pixtral Large, a 124B multimodal model built on top of Mistral Large 2.
See https://huggingface.co/mistralai/Pixtral-Large-Instruct-2411

We don't have pixtral yet, pixtral-large will probably be gated on that.

merging this issue with #6748
Thank you!!


How can i use the http api without username and password?

It is recommended that your provide a more detailed description

You cannot.


Missing variable declaration (paramErr)?

Hey there!
Thanks so much for your work on SFBAudioEngine and for providing it as open source software. I'm still trying out the project but it looks ideal for my use case.
I've tried compiling the project (after some minor tweaks which I'll address separately) and have run into an issue around a missing variable declaration.
In some places throughout the project you use:
[NSError errorWithDomain:NSOSStatusErrorDomain code:paramErr userInfo:nil];

However paramErr is undefined.
After a search in the project and on GitHub as well (https://github.com/sbooth/SFBAudioEngine/search?q=paramErr), it seems the value isn't provided anywhere.

Is this something that should be provided by the consumer of the library? If so, how?
Is this something that is missing from the project itself? If so, please suggest a fix and I'll happily open a PR.
Is this something that I've misconfigured or misunderstood? What would be useful information I can offer to help you point me in the right direction?

Thanks!

Looking into this a little further, it seems the project compiles correctly for macOS, however the missing variable errors appear only on iOS.
I can see paramErr is a constant, and by @importing CoreServices in the files where we reference these errors the problem goes away.
I'll shortly open a PR to include an import to CoreServices.
Separately to this, do you have any plans to set up CI / some automated testing for PRs? Would love to make sure the project continues to compile correctly for iOS going forward.
Thanks!

Quick update, importing CoreServices on iOS doesn't actually solve the problem. paramErr and ioErr are part of Carbon and aren't available on iOS (as far as I know).
Given that — should how would you suggest I approach fixing this?

I didn't realize that all of those (old) error codes were deprecated. I just merged a PR that removes their use.
I'd like to implement CI of some sort but I've run into problems compiling all the dependencies. I do build manually before merging PRs but in this case it seems something slipped through the cracks.


Check index once per unique index name during save embeddings.

Motivation and Context (Why the change? What's the scenario?)
I had the same problem as the submitter of this issue: https://github.com/microsoft/kernel-memory/issues/289. For every embedding during the save process, a call is made to list indexes. Using Azure AI Search, I was getting rate limit errors, i.e. Azure.RequestFailedException with the message "You are sending too many requests. Please try again later."
High level description (Approach, Design)
Made an adjustment to simply cache indexes that were already checked/created during the save embeddings process. This skips the unnecessary call to ListIndexes for every single embedding.

Thanks @spenavajr
There is also another call to client.CreateIndexAsync(...) in the method:
public async Task<(bool success, DataPipeline updatedPipeline)> SavePartitionsAsync(

I'm pretty sure we also need your fix in that method. Aas far as I understand, there is one method that only Insert the embeddings data, and another one the rest of the fields (tags, payload).
Could you add your fix to that method, please?

@microsoft-github-policy-service agree

@luismanez you are correct, I overlooked that. Thank you!

@luismanez you are correct, I overlooked that. Thank you! Updated.

Thanks again. However, looking at it again, I think we still need another change.
The exiting code iterates all the "chunks", and foreach item, iterates all the configured Memories (apparently you can configure multiple Memories, like one for Az Search, another one for Postgree, etc). If we only have one Memory, then your code will work, but if we have multiple memories, as the createdIndexes is only storing the name of the Index, when it enters in the 2nd memory item, the Index won't be created for that 2nd memory, and when it calls client.UpsertAsync will fail.
I don't see an easy solution for this, unless you move the creation before the initial foreach, but you should then to check multiple things...
@dluc Could you explain why we have multiple Memories? I don't see how to configure more than one. Initially, the KernelMemoryBuilder adds the SimpleVectorDb memory, and then, all the Extensions that I can see, like AddAzureAISearchAsMemoryDb, AddRedisAsMemoryDb, etc, register the Memory as Singleton:
.AddSingleton<IMemoryDb, RedisMemory>(); The only way seems is creating the BaseOrchestrator manually...

@luismanez that's a good point, thanks for digging in. I would assume having multiple memory databases is to simply store those memories in multiple places?
One suggestion is - we could modify what goes into the createdIndexes hash set to include a memory client prefix, e.g.
AzureAISearch::<index name> surely that should solve the problem, no?

@luismanez that's a good point, thanks for digging in. I would assume having multiple memory databases is to simply store those memories in multiple places?
One suggestion is - we could modify what goes into the createdIndexes hash set to include a memory client prefix, e.g.
AzureAISearch::<index name> surely that should solve the problem, no?

yup, but you just have an IMemoryDb object, and I haven't found any kind of id in that interface that we can use for composing something like a primaryKey...

@spenavajr might work with calling the GetType() method??

@spenavajr might work with calling the GetType() method??

Yeah, I was thinking client.GetType().Name
I will try it out & report back.

@luismanez seems to be working, check latest update - the team may want this designed differently... but it seems to solve the original issue at hand for now (I may fork this and use it for a while if not merged soon 😄)

Looks good!! Thanks a lot.
@dluc please, we need this fix otherwise we cannot work with long files when using Az Search as Memory. Thanks!

Looks good!! Thanks a lot.
@dluc please, we need this fix otherwise we cannot work with long files when using Az Search as Memory. Thanks!

someone will look into this asap, however, please note that due to some constraints, the earliest we might be able to address this could be after April 8th

@dluc Could you explain why we have multiple Memories? I don't see how to configure more than one. Initially, the KernelMemoryBuilder adds the SimpleVectorDb memory, and then, all the Extensions that I can see, like AddAzureAISearchAsMemoryDb, AddRedisAsMemoryDb, etc, register the Memory as Singleton:
.AddSingleton<IMemoryDb, RedisMemory>(); The only way seems is creating the BaseOrchestrator manually...

The intention is to allow combining different storage types that offer different cost, performance, features:
      // Vectors can be written to multiple storages, e.g. for data migration, A/B testing, etc.
      "MemoryDbTypes": [
        "SimpleVectorDb"
      ],

@dluc been super busy at work, will get on this as soon as I can!


Updates for 2.3.4 release

Docs are updated for 2.3.4 release. Please review and merge.

@rishikeshpalve Thanks!


When running on a local machine

when running it on a local machine it constantly says the env var database url unpooled is not found during prisma migration but i have rechecked that the variables exist the file is called .env

This is not recommended however you can copy the main .env file to the /packages/database folder, this should solve the problem


Reverse dependencies from /main/ to /test/

Discussed in #6814
Druid-benchmarks has a number of reverse dependencies from the main tree back to the test-tree.  This is bad practice and should be corrected.  An even larger question is why is Druid-benchmarks even in the main tree?
Specifically:
druid-benchmarks
  org.apache.druid.benchmark
    ExpressionAggregationBenchmark
      import org.apache.druid.query.expression.TestExprMacroTable; //located in test tree
    ExpressionSelectorBenchmark
      import org.apache.druid.query.expression.TestExprMacroTable; //located in test tree
  org.apache.druid.benchmark.datagen
    SegmentGenerator
      import org.apache.druid.segment.IndexBuilder; //located in test tree
      import org.apache.druid.segment.TestHelper; //located in test tree
  //located in druid-processing-0.13.0-incubating-SNAPSHOT-tests.jar

  org.apache.druid.benchmark.query
    SqlBenchmark
      import org.apache.druid.sql.calcite.util.CalciteTests; //located in test tree
      import org.apache.druid.sql.calcite.util.SpecificSegmentsQuerySegmentWalker; //located in test tree
  //located in druid-sql-0.13.0-incubating-SNAPSHOT-tests.jar

#7481 as it should fix this, and a few other issues that prevent druid from building with Eclipse.
@gianm If you have time can you take a look at it?


2 conduit-related crashes

I was in the process of finding/reporting bugs for Tech Reborn(TR) so as to properly test its multiblock machines i setup some conduits to auto-extract the results and got 2 crashes(the second one i'm not quite sure about but it occurs when i'm placing/editing conduits/conduit connections). I've also noticed an unusual side effect of when editing a conduit connection it'll attempt to place the place-able block held in your hand and does so most of the time.
I've tried backdating 3-5 versions from the latest to beta 44(endercore) & beta 74(enderio) to get no success with trying to eliminate the crashes by finding a non-crashing version.

EnderIO: beta 74+
EnderCore: beta 44+
Minecraft: 1.10.2
Forge: build 2039

https://gist.github.com/the-creamster/f0e32e5b86c5b2045bea05ed964674d1
https://gist.github.com/the-creamster/cbd5ce95cebe3dbaf462c7a170182a14

In the first one techreborn code is crashing, did you make syre it's not that code that's faulty first?
The second one looks like the single player integrated server stopped while the client wanted to connect.

The first crash i mentioned to the TR devs in a issue i had active and they said it was EIO. As for the second crash i got it after being in a world for sometime and when i attempt to rejoin a world which crashed me because of the first crash.

found it, commented there https://github.com/TechReborn/TechReborn/issues/713

Status update:
Crash 1: Still waiting for TechReborn
Crash 2: As I understand that is the client crashing after the integrated server crashed at a bad time for the client.


OutOfDirectMemoryError/large number of PooledUnsafeDirectByteBuf allocated

We're using Netty via Apache Camel (camel-netty4). The component acts as server and is only supposed to read messages from clients.
Wenn starting our system a number of clients (around 20-25) connect to the TCP server. During that process we started to experience issues with available direct memory being exhausted quickly.
[io.netty.util.internal.OutOfDirectMemoryError - failed to allocate 16777216 byte(s) of direct memory (used: 469762048, max: 477626368)]io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 16777216 byte(s) of direct memory (used: 469762048, max: 477626368) 	at io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:624) 	at io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:578) 	at io.netty.buffer.PoolArena$DirectArena.allocateDirect(PoolArena.java:686) 	at io.netty.buffer.PoolArena$DirectArena.newChunk(PoolArena.java:675) 	at io.netty.buffer.PoolArena.allocateNormal(PoolArena.java:237) 	at io.netty.buffer.PoolArena.allocate(PoolArena.java:221) 	at io.netty.buffer.PoolArena.allocate(PoolArena.java:141) 	at io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:262) 	at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:179) 	at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:170) 	at io.netty.buffer.AbstractByteBufAllocator.ioBuffer(AbstractByteBufAllocator.java:131) 	at io.netty.channel.DefaultMaxMessagesRecvByteBufAllocator$MaxMessageHandle.allocate(DefaultMaxMessagesRecvByteBufAllocator.java:73) 	at io.netty.channel.socket.nio.NioDatagramChannel.doReadMessages(NioDatagramChannel.java:242) 	at io.netty.channel.nio.AbstractNioMessageChannel$NioMessageUnsafe.read(AbstractNioMessageChannel.java:75) 	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:610) 	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:551) 	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:465) 	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:437) 	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873) 	at java.lang.Thread.run(Thread.java:745)
After tinkering with JVM and Netty parameters (e.g. "-XX:MaxDirectMemorySize=1G -Dio.netty.allocator.pageSize=8192 -Dio.netty.allocator.maxOrder=10") the whole shebang can start successfully.
However, I am very sceptical about the amount of direct memory netty allocates. When pulling a heap dump of the server application I can see that there are 63 (!) instances of PooledUnsafeDirectByteBuf held in memory.
Even after extended no-load periods none of those PooledUnsafeDirectByteBuf ever seem to be released.
Activating leak detection ("-Dio.netty.leakDetectionLevel=paranoid") yielded nothing.
Am I missing something very obvious? Is this behaviour expected? What is the recommended way to cope with this?
Netty version
4.1.5
JVM version (e.g. java -version)
Java(TM) SE Runtime Environment (build 1.8.0_121-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)
OS version (e.g. uname -a)
Linux *** 3.12.59-60.45-default #1 SMP Sat Jun 25 06:19:03 UTC 2016 (396c69d) x86_64 x86_64 x86_64 GNU/Linux

Please update to the latest Netty (4.1.8) and report back if you can still reproduce.
We retain the objects from a the PooledByteBufAllocator via the Recycler so that in itself is not necessarily an indication of a problem. You can use -Dio.netty.recycler.maxCapacityPerThread to tweak the limits or disable the Recycler.

You can disable the recycler with -Dio.netty.recycler.maxCapacity=0 -Dio.netty.recycler.maxCapacity.default=0

I can observe the same behaviour using latest Netty (4.1.8). I also tested the Recycler options, but they had no noticable effect on my problem.
To reproduce the behaviour I created a small test project.
It contains a JUnit test, which sets up a simple Netty TCP Server and then uses simple Socket connections to send message.
Running the "testMultipleClients"-Test and using some well placed breakpoints (e.g. io.netty.util.internal.PlatformDependent.incrementMemoryCounter(int)) I can determine that for each new client connection a new PooledUnsafeDirectByteBuf is reserved - up to a limit of 9 on my local machine.
So the first 9 connections each lead to a new buffer allocation, all following connections are content with that and don't reserve anything additional. This means - as far as I understand - that 9*16MB=144MB of direct memory have been allocated.
Rifling through the Netty code I couldn't quite figure out which factors/properties lead to that particular number (9) of PooledUnsafeDirectByteBuf.
In any case, in our real application scenario the number of reserved PooledUnsafeDirectByteBuf grows supposedly beyond 64 - although none of the clients put particular load onto the server (after all the error happens when the system is just starting up). Due to that rather high number of buffers, we run into the OutOfDirectMemoryError originally reported. With 1GB of MaxDirectMemory 64*16MB buffers just fit, hence the conclusion that > 64 must have been tried to have been reserved (and none freed).
What I can't figure out:

Am I up to something here or hunting ghosts?
What settings/machine specifics (# of CPU) factor into the number of reserved PooledUnsafeDirectByteBuf?
Is it ok to just tweak netty settings to get over the startup scenario - where the OutOfDirectMemoryError occurs?

To add to the last point - once all is started normally I can put quite some pressure onto the server application (fo instance using tcpkali) with a sizable number of clients (> 100) sending a sizable number of messages/second (>100) without encountering this particular issue (or any issue at all).

thanks for reporting back and the reproducer @lkoe ... we will investigate and get back to you

@lkoe this is most likely because of the default configuration of PooledByteBufAllocator which will create cores * 2 number of PoolArena each of this arena will at the end at least have 16MB of allocated direct memory.  This is done to reduce lock-condition on the arena itself but also have a "high" memory overhead depending on your memory constraint. You can overwrite this by either specify it in the constructor of PooledByteBufAllocator or by using the following system properties:
-Dio.netty.allocator.numHeapArenas=
-Dio.netty.allocator.numDirectArenas=

Hope this helps.

so，what is the conclusion? I encountered the same problem. @lkoe

Is there a tool which can print what all is allocated on Direct Memory? For My case with 1 G DirectMemory, I am getting similar exception.   I am using above settings -Dio.netty.allocator.pageSize=8192 -Dio.netty.allocator.maxOrder=10',  and when i obtain a heap dump,  I see just 30 instances ofio.netty.buffer.PooledUnsafeDirectByteBuf`.
io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 8388608 byte(s) of direct memory (used:<PHONE_NUMBER>, max:<PHONE_NUMBER>)
	at io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:506)
	at io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:460)
	at io.netty.buffer.PoolArena$DirectArena.allocateDirect(PoolArena.java:701)
	at io.netty.buffer.PoolArena$DirectArena.newChunk(PoolArena.java:690)

My machine has 32 cores.

@normanmaurer Could this issue have something to do with an unreleased slice of that incoming byteBuf?
protected void decode(ChannelHandlerContext channelHandlerContext, ByteBuf byteBuf, List<Object> list) {
    while(byteBuf.isReadable()){
         list.add(byteBuf.slice(...,...)) <- not completely released?
    }
}

@konsultaner actually you would need to call byteBuf.retainedSlice(...) here to ensure you retain the buffer as it will be released once decode completes.

We had the same issue for months now together with the async MongoDB driver and with activated SSL.
The issue only occurred while bulk operations on MongoDB driver. The JVM settings

-XX:MaxDirectMemorySize=1G -Dio.netty.allocator.pageSize=8192 -Dio.netty.allocator.maxOrder=10

finally helped. Thank you very much!
I attached my stack trace.


[Embeddingapi] Add autocase about selecting xwalkview language

-Add autocase to check XWalkView.setAcceptLanguages api
-cover the XWalkViewActivity and XWalkInitializer two ways
Impacted tests(approved): new 1, update 0, delete 0
Unit test platform: [Android]
Unit test result summary: pass 1, fail 0, block 0
https://crosswalk-project.org/jira/browse/XWALK-4350

@jennycao please help to review it.
Thanks.

LGTM


Machida3 for Python3 application code

A copy of the machida source tree to machida3 except for the wallaroo python modules.
The Celsius test can be run with machida3, there might be some encoding issues with the result out as
most of the tests use string formatting which ends up calling str() on bytes objects resulting in broken output. In order to fix the integration tests for machida3 this will have to be addressed.

Ooh exciting! I'll take a look at this later today!

Ooh exciting! I'll take a look at this later today!

Sorry, I couldn't get around to this today. I'll take a look at it on Monday, unless someone else beats me to it!
An initial skim of the code looks pretty good, though!

@caj-larsson The code changes look good, but there are a few places where integration tests (and manual tests) run into fatal errors due to typing errors.
You can grab the code for the additional tests from https://github.com/WallarooLabs/wallaroo/commit/01ec4b8baa8a7f25cf2ca95e7544a22881f6ab33
and run them as follows:
Examples/python/
cd examples/python
make test

testing/correctness/apps/
cd testing/correctness/apps/alphabet_python/
make test

make test

cd testing/correctness/apps/sequence_window_python/
make test

The tests that pass are:

examples/python/alphabet
examples/python/celsius
examples/python/market_spread
examples/python/reverse

The tests that fail are:

testing/correctness/apps/alphabet_python27
testing/correctness/apps/alphabet_python
testing/correctness/apps/sequence_window_python
examples/python/alphabet_partitioned
examples/python/word_count
examples/python/word_count_with_dynamic_keys

@caj-larsson No worries. Take your time, and thanks for the help you've given us so far! :-)

Hi @caj-larsson
We're picking this up as mainline work this week, and wanted to let you know. And we also wanted to thank you for the work you've done on this so far! It's been really helpful, and it's really heartening to see early contributions from outside the company!
However, we currently don't have access to make changes to the PR branch. As the PR creator, you can update that following these instructions: https://help.github.com/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork/ .
In the mean time, we'll work on the python3 branch in the main repo. But we'll be happy to move the updates from there back into this PR once we have access.

Oh hey, that worked!
I have:

cloned this caj-larsson/wallaroo.git
set up wallaroolabs/wallaroo.git as remote upstream
copied the python work to python3 branch for safe keeping
reset caj-larsson/wallaroo.git:master to the last common commit
fetched upstream, and rebased master
merged the upstream/python3 branch into caj-larsson/wallaroo.git:master so it includes both the original commits from here, and the new ones from the main wallaroo repo's python3 branch.
We should be able to continue working on this branch here now.

Thanks for granting me permissions on the repo, @caj-larsson!

Thanks that sounds exactly like what I expected actually!

change some machida.pony python string conversion to use UTF8 instead
of bytes

This is exactly the type of issue I was working on when my move started, there were several places I had switched to bytes incorrectly because it's sometimes hard to tell a C/pony buffer from a printable string.
Mainly these were things like labels that I missed were such when going through the parsing machine.
Personally I think we could split those big switch blocks to something a little more high level, these almost reminds me of my SoC development days haha

I did another rebase on upstream to take in improvements to CI error log and core dump capture. Hopefully this gives us core dumps from CI (in addition to the ones we can get from running the tests locally)

Last commit: all python examples pass in python3, regardless of whether they use bytes or unicode in their user code.
Next up: adding python3 resilience and topology tests

That's fantastic! I'm sorry I couldn't help more at the final stages but I realize that the closer to done things got the more assumptions I had to start making.
I'll look at the python3 project to see if there is anything else I can help with


Write tutorials to better document Concierge features

Hello World service
Hello World integration
Moderately complicated module handling multiple sub-commands (helps illustrate the use of event.arguments)
Module that passes data between different integrations

It looks like I won't be able to make the time to get on this anytime soon and I've already delayed this a lot 😞.
Should we close this issue or let someone else handle it?

Let someone else handle it. It needs to be done at some point.


Fixed issue 1173

PR for #1173

:+1: To the content.
Can you amend this to use your name instead of "connected-web-development" as the author of the commit? Thanks!

@hvanbakel I marked this do not merge just so it's easy for me to see that I finished reviewing it and it's waiting for feedback. Once you amend the commit, let me know and I can get it merged. :smile:

I performed a rebase to fix the author line. I can see from your edit in #1182 that this will not be a problem in the future.
Manually merged in c1d1d4e423fa30cd0570c36c5e14d7e613ae7a2b.