Binding and Filtering References

Hi,
First of all i would like to congratulate the devs (or the dev) for this framework.
Anyway... let's get down to business:
1 - I would like to know how to copy one object and cut all the bindings references attached to this object. I tried using some one-way binding with a child component but everytime i change the object in the child component it does affect the parent component. (even with the {{* object }} notation).
2 - There is not much reference on the "JS-side" of the components, the documentation is too focused on templating. I would suggest to maybe create a Cheat Sheet or something like this.
3 - For anyone wondering how to do "built-in" filtering in JS, here is an example:
var country = this.$options.filters.filterBy(this.countries, this.profile.country.code, 'in',['code']);

Questions and general discussion are better posted on the forums. The issue list is for bug and feature requests only.
As for cutting the references, just do a deep copy of the object. The simplest form can be JSON.parse(JSON.stringify(object)).


Support for JSX Fragment

Is there a way to use fragments with petit-dom?
export default <>

</>;

I'm trying to use petit-dom to render into a shadow-dom, and it works great, but having a single parent is not ideal because my shadow dom ends up having an unnecessary wrapper element. I know other vdom libraries support this, but I'm not sure about the tradeoffs, if any.

I just realized this is partly a duplicate of https://github.com/yelouafi/petit-dom/issues/1. As far as I know, arrays of vnodes need to be supported to support Fragment in JSX.

Fragments (and nested arrays as well) are supported in v0.4.1

Fragments (and nested arrays as well) are supported in v0.4.1

@yelouafi I see that the new implementation has abandoned the O(nd) algorithm, is there any trade-off, thank you.

@yelouafi I see that the new implementation has abandoned the O(nd) algorithm, is there any trade-off, thank you.


[[Feature Request]] Create next weekly/monthly/whatever note

Rather than having to use the Calendar plugin to create new notes, have the commands for creating next (week/month/etc) notes if one does not exist.
The commands for the current (week/month/etc) notes can create new notes, it just makes sense to be able to do the same for the next (week/month/etc).  I understand why it doesn't create notes with the 'previous' commands, however, I can set templates for these new notes but have no easy way to do so via a command, and in the case of everything but daily/weekly notes it is impossible to easily create a new one without the Calendar plugin.  It seems like this was overlooked, as most of the features to make this easy are already there, but it's just missing the final step.

In the v1.0 beta, there are now 2 separate commands for note navigation:

Open next daily note (this will open or create the note for the following day)
Jump forward to closest daily note (the old behavior)

@liamcain thanks for all your great work on this. I installed the beta and am currently testing it out. Just wanted to see if this is expected behavior.

Currently Open - Past Daily Note
The "Open next daily note" goes to the next existing note. Makes sense. No "next weekly note" available"

Currently Open - Today's Daily Note
"Open next daily note" does nothing - I would've expected it to create tomorrow's daily note. No "next weekly note" available

Currently Open - Any other Note
"Open next daily/weekly note" command is not available

I 100% agree. This is the main feature I need. Is there a PR out for this? Can I help implement it? Someone point me in the direction.


feat: add metrics for gc

Add metrics for gc(both success and failed)

/retest

ping @kzys @dmcgowan for review

/close dup with #5263


Please support trimming for self contained projects

When building a dotnet 7 self-contained project with Graph Beta SDK 5.24.0-preview, the following warning is observed:
C:\Users\{user}\.nuget\packages\microsoft.kiota.abstractions\1.1.0\lib\netstandard2.1\Microsoft.Kiota.Abstractions.dll : warning IL2104: Assembly 'Mi
crosoft.Kiota.Abstractions' produced trim warnings. For more information see https://aka.ms/dotnet-illink/libraries [C:\Users\{user}\source\repos\{project}.csproj]

Ideally, this warning would be resolved by supporting Trimming in the package - see https://learn.microsoft.com/en-us/dotnet/core/deploying/trimming/trim-warnings/il2104

Resolved via https://github.com/microsoft/kiota-abstractions-dotnet/pull/139


Fix/state

Just would like help to get these tests fixed and this should be stable:

@pavlovcik the remote test are supposed to be working with a fork url
check runner.ts for more information


Tests fails with Disposable 0.2.4

Ruby 2.2.3 and Disposable 0.2.4
/Users/acaron/.rvm/gems/ruby-2.2.3/gems/disposable-0.2.4/lib/disposable/twin/definitions.rb:1:in `<top (required)>': uninitialized constant Disposable (NameError)
	from /Users/acaron/.rvm/gems/ruby-2.2.3/gems/disposable-0.2.4/lib/disposable/twin.rb:3:in `require'
	from /Users/acaron/.rvm/gems/ruby-2.2.3/gems/disposable-0.2.4/lib/disposable/twin.rb:3:in `<top (required)>'
	from /Users/acaron/Sites/tyrant/lib/tyrant.rb:7:in `require'
	from /Users/acaron/Sites/tyrant/lib/tyrant.rb:7:in `<top (required)>'
	from /Users/acaron/Sites/tyrant/test/test_helper.rb:2:in `require'
	from /Users/acaron/Sites/tyrant/test/test_helper.rb:2:in `<top (required)>'
	from /Users/acaron/Sites/tyrant/test/authenticatable_test.rb:1:in `require'
	from /Users/acaron/Sites/tyrant/test/authenticatable_test.rb:1:in `<top (required)>'
	from /Users/acaron/.rvm/gems/ruby-2.2.3/gems/rake-10.5.0/lib/rake/rake_test_loader.rb:15:in `require'
	from /Users/acaron/.rvm/gems/ruby-2.2.3/gems/rake-10.5.0/lib/rake/rake_test_loader.rb:15:in `block in <main>'
	from /Users/acaron/.rvm/gems/ruby-2.2.3/gems/rake-10.5.0/lib/rake/rake_test_loader.rb:4:in `select'
	from /Users/acaron/.rvm/gems/ruby-2.2.3/gems/rake-10.5.0/lib/rake/rake_test_loader.rb:4:in `<main>'
rake aborted!
Command failed with status (1): [ruby -I"lib:test:lib" -I"/Users/acaron/.rvm/gems/ruby-2.2.3/gems/rake-10.5.0/lib" "/Users/acaron/.rvm/gems/ruby-2.2.3/gems/rake-10.5.0/lib/rake/rake_test_loader.rb" "test/authenticatable_test.rb" "test/session_test.rb" "test/sign_up_test.rb" "test/tyrant_test.rb" ]
/Users/acaron/.rvm/gems/ruby-2.2.3/bin/ruby_executable_hooks:15:in `eval'
/Users/acaron/.rvm/gems/ruby-2.2.3/bin/ruby_executable_hooks:15:in `<main>'
Tasks: TOP => default => test
(See full trace by running task with --trace)

Actually, it seems like it is Reform the problem. Working with 2.0.0.rc2 but not 2.1.0.


cit_magic.py does not seem to make correct sites.txt file for PI47- data example and USGS example

I am trying to get the USGS cit files working with pmag_gui. I have put the files for one site in a directory and when I run the cit_magic.py converter script with the command in the file 'command', I get files that work with pmag_gui to display the demag for the specimens. But I get the warning: "There is no geographic data for this specimen." but I thought each demag step has the geographic info available". Also demag_gui does not know what the site is. Below is a .zip of all the files for the one USGS site.
USGSSingleSiteFiles.zip
I did the same thing with the PI47- example files. I ran "conversion_scripts/cit_magic.py -f PI47-.sam" and then looked at them using demag_gui through pmag_gui. These files also give the:"There is no geographic data for this specimen." warning and does not know what the site is.
PI47files.zip
I'm not sure if the problem is in demag_gui or the cit_magic conversion. If you could take a look @Caoimhinmg that would be great. Lori @moonshoes87 might be more familiar with the demag_gui stuff.

Tested on example data sets and seems fixed.


Why does staggered execution of functions cause cold starts?

I deployed OpenWhisk on a single host. When I call both functions continuously at the same time,  cold start occurs. So why does staggered execution of functions cause cold starts?

It depends on your local configuration.
How many memory did you configure for an invoker and in which interval did you invoke each action?

The current heuristics are such that as you saturate some worker nodes, you get new worker nodes, which leads to some cold starts. To answer this question with more detail, you'd need to share your configuration and test setup.


Some widgets don't work

I installed the patched version and it seems to mostly work.  The only thing I've found that doesn't work is that some apps' widgets won't show up.  Two examples are Aqua Mail and Venmo.

From logcat:
04-02 11:49:51.495  2052  6214 W ActivityTaskManager: Permission Denial: starting Intent { act=android.appwidget.action.APPWIDGET_CONFIGURE cmp=org.kman.AquaMail/.widget.ListWidgetConfigActivity (has extras) mCallingUid=10296 } from ProcessRecord{9c50bd8 27602:net.pierrox.lightning_launcher_extreme/u0a296} (pid=27602, uid=10296) not exported from uid 10337

We're they working with the original 14.3 version? (I don't think so, but just to confirm)
That issue is due to an android change in the widget configuration. Widgets that are not exported will not work on launchers targeting older apis (like the patched version).
The debug version should work (since it targets newer apis) but the patched unfortunately will not (unless the app itself "fixes" it).
You can find a bit more info here: https://github.com/home-assistant/android/issues/2064

Original 14.3 from the play store did work.  I'll try the debug one, thanks!

@TrianguloY I'm afraid the debug version still can't create an AquaMail widget :(
And indeed, I restored my old version of 14.3 (not patched) and it can't create the widget either.  I definitely had a functioning widget on 14.3 right up until I uninstalled it to install the patched version. I can only assume that the widget continued to work because it had been there since before this restriction applied.  I even tried downgrading AquaMail's version, to no avail.
Do I need to give up on my dream of having a functioning widget?  Do you have any ideas?  My phone is rooted, if that helps.
I'm happy to help by testing, providing logcats, etc.  I'm afraid I'm not an Android dev though.

@TrianguloY I'm afraid the debug version still can't create an AquaMail widget :(

Really? That's strange. I had this bug in the past with an app that was also updated and set the exported to false, blocking new widgets.
I remember doing some tests, but I removed the app for other reasons and I don't remember finding a workaround. I've always assumed that it was due to the old api, but the debug version targets the latest so...
Is it possible for you to try a couple other launchers (other than the device default one) just to check if it's a configuration issue or generalized to non-system launchers.
The best fix would be for AquaMail to replace the exported=false to exported=true, but unless they have some sort of support...
With root you should be able to fix it...but unfortunately I don't know how. Maybe if you install LL as a system app? Maybe you have a way to change the exportability of arbitrary apps?

I did try it with another launcher (Total Launcher) and it worked fine...
I'll try reaching out to the Aqua Mail dev.  Maybe they can help.

Finally figured out how to do it.  You can use apktool to unpack the apk, and then you can edit the manifest.  Setting the activity that handles APPWIDGET_CONFIGURE to exported=true makes it so lightning can add the widget.  Then repack earth apktool, zipalign, sign, and install.
App Cloner with the Manifest Editor add-on can simplify all this.

That means patching other apks (which may not be possible in some cases) but I'm happy to hear it solved it for you.
Still, I'm going to keep this open, since the bug still persist (still don't know why other launchers can access a 'private' service...)

I saw this in the manifest, and it was exported already:
<service android:exported="true" android:label="@string/list_widget_name_short" android:name="org.kman.AquaMail.widget.ListWidget_hc$WidgetService" android:permission="android.permiss
ion.BIND_REMOTEVIEWS"/>

Maybe this is some new way of doing it?  I suppose the hc could stand for honeycomb, and lightning is doing things in a truly ancient way or something?


Genetic_backtester

incorporates changes in pull #1321
Changed launch files so they execute node directly and not thru bash.
this fixes an orphaned process problem i was seeing where the last
launched process was not getting the kill signal.
Add parameter —run_for to trade command.  this accepts an integer, and
represents how many minutes to trade before doing a clean exit.

If --run-for parameter is not given, could we the previous behavior (ie. run forever)?

When --run-for parameter is used, the following warning appears:
Deprecation warning: moment().add(period, number) is deprecated. Please use moment().add(number, period). See http://momentjs.com/guides/#/warnings/add-inverted-param/ for more info.
The following removes the warning:
var botStartTime = moment().add(so.run_for,'m')

@station384 @DeviaVir
+1 request to keep previous behavior if no --run-for or provide option to have it like -1

@narentin bug fixed by @illya13 in pull #1329


Setting bounds may not work right

Need to see if it's on initial set or on subsequent set but something seemed off a couple days ago.

Seeing the same problem. https://github.com/dabreegster/svelte-maplibre/commit/43466bb4220a6ef56bc4031b0ecd2e5dc47b3f27 is a very weird workaround. The caller explicitly setting center={undefined} doesn't seem to work. null does, but then it doesn't type-check. It might make more sense to remove the default values for center and zoom instead.
I'll try a few things out, clean it up, and send a PR


Release notes for 1.0.0-alpha.1

Tag version: 1.0.0-alpha.1
Release title: 1.0.0-alpha.1
Today, we are happy to announce the first release of Louketo. As planned some time ago the Gatekeeper repository was renamed to Louketo Proxy and now has a new home. This will enable us to build together with the OSS community a neutral and fully standard-based proxy solution with application developers in mind.
What's New in Version 1.0.0-alpha.1

Refactor of the codebase
Migration of all the documentation from the Keycloak to Louketo organization
Upgrade of some deprecated dependencies
A set of bug fixes and improvements
We also spent some time automating most of the stuff to pave the way for future releases

Changelog

3f64894 Make equality fuzzy in TestGetWithin (#631)
48c72c0 Update to user-guide.md (#632)
33c23da Use HTTP 303 for redirects, instead HTTP 307 (#627)
9fd264c Automate Louketo releases (#615)
f623dee Pass URI path upstream verbatim (#528 #483)
a0cad52 Migrate Gatekeeper documentation to Louketo repository (#610)
9079de6 GH templates for bug reports and feature requests are not working (#617)
3905412 Replace satori/go.uuid by gofrs/uuid (#611)
906a8fe Create a GitHub templates for pull-requests and new issues #522
8e8510a Run golangci-lint as part of pull request workflow #604
c500821 Migrate the codebase from Gatekeeper repository #516

@ASzc @stianst could you please review the release notes that will be published tomorrow? The changelog should be automated, but I found a bug that we need to fix in the next releases.

Will the release automation accept a version with a qualifier? I know it didn't like a pure alphabetic tag when I tested it before.
Technically if we want "OSGI format" (if that even makes sense in the context of golang), then 1.0.0-alpha.1 would be better represented as 1.0.0.alpha-1. That's from Java-land though, not sure if Go likes four-part versions.
I notice you're missing a0cad52 Migrate Gatekeeper documentation to Louketo repository (#610) from the list, but maybe that's intentional.
3f64894 Make equality fuzzy in TestGetWithin (#631) also is missing, and I see it was merged to master a few days ago.

@ASzc thanks for reviewing it Alex. 1.0.0.alpha.1 complies with the semantic version specification. So we should be good to go.
See: https://semver.org/#spec-item-9 and https://semver.org/#spec-item-11
Regards 3f64894 Make equality fuzzy in TestGetWithin (#631), it's the first item in the changelog. But please let me know if any other thing should be updated.
About a0cad52 Migrate Gatekeeper documentation to Louketo repository (#610), updated the document. Thanks for that.

Resolved and released here https://github.com/louketo/louketo-proxy/releases/tag/1.0.0-alpha.1


chore: bump tool versions

Description
Update versions of tools
How was this change tested?

make presubmit

Does this change impact docs?

[ ] Yes, PR includes docs updates
[ ] Yes, issue opened: #
[x] No

Release Note

Pull Request Test Coverage Report for Build<PHONE_NUMBER>2
Details

0 of 0   changed or added relevant lines in 0 files are covered.
2 unchanged lines in 1 file lost coverage.
Overall coverage decreased (-0.005%) to 94.229%

Files with Coverage Reduction
New Missed Lines
%

pkg/cache/unavailableofferings.go
2
95.45%

Totals

Change from base Build<PHONE_NUMBER>7:
-0.005%

Covered Lines:
37147

Relevant Lines:
39422

💛  - Coveralls


Replace i18n.t w/ tpl helper in authentication.js

Got some code for us? Awesome 🎊!
Please include a description of your change & check your PR against this list, thanks!

[x] There's a clear use-case for this code change
[x] Commit message has a short title & references relevant issues
[x] The build will pass (run yarn test and yarn lint)

More info can be found by clicking the "guidelines for contributing" link above.
I've changed i18n.t lines to tpl, according to contribution guidelines.

Thank you for the PR 🙏
I believe with the topic on our repo, merging the PR alone will allow it to be picked up. If that's not the case let me know 😸

Would be super helpful if in future you could also follow our commit guide and add the issue reference to the commit message.

Hey! So sorry about commit message, I missed it!
About Hacktoberfest it works perfectly now :) Thanks


version comparison of string values

:wave: hey there I came across this package (which looks really cool) and I noticed this line:
if thisVersion > currentVersion {
	continue
}

I was wondering how the version compare would handle this scenario:
9.up.sql
10.up.sql

given the following result when comparing the two version strings:
fmt.Println("9" > "10")
// true

have you considered parsing the version as a number?
or maybe you would consider allowing the user to customize the regular expression to ensure extra safety?  For example I could customize the up and down expressions to ensure the version is expressed as a date:
// 20210510.up.sql
regexp.MustCompile(`^([\d]{7}).up.sql$`)

or I could customize the expression to ignore the trailing description and only use the digits for comparison:
// 20210510-create-user-table.up.sql
regexp.MustCompile(`^([\d]{7})[\w-]+.up.sql$`)

Hey @ashwilliams1! Thank you for your feedback. 😊
Yes, versions are compared lexically, not numerically. This allows for descriptive versions such as 212-doing-something, and not just numbers. The easiest workaround to having low-digit version numbers is to use zero prefixes. But I would suggest doing what the built-in command does and use timestamps instead. I would be hesitant to add a new sorting functionality, for the sake of simplicity of the API.
For the latter part: I just added Before and After callback functions just yesterday. In Before, which gets the version string, you could do whatever validation you need to do and potentially error. I think that would cover your use case?

But I definitely get your point. Even the example in the README uses simple version numbers. Hmm. I'll think about what the best course of action is.

would you maybe consider something like this?
type Options struct {
+	MatchUp   *regexp.Regexp
+	MatchDown *regexp.Regexp
	After     callback
	Before    callback
	DB        *sql.DB
	FS        fs.FS
	Table     string
}

func New(opts Options) *Migrator {
+	if opts.MatchUp == nil {
+		opts.MatchUp = upMatcher
+	}
+	if opts.MatchDown == nil {
+		opts.MatchDown = downMatcher
+	}

I think this would get me half way there.

maybe add a compare function?  I could use this to parse and compare strings?
+type comparer Compare(a, b string) int

type Options struct {
+	Compare   comparer
	After     callback
	Before    callback
	DB        *sql.DB
	FS        fs.FS
	Table     string
}

func New(opts Options) *Migrator {
+	if opts.Compare == nil {
+		opts.Compare = strings.Compare
+	}

I guess it would be nice if I could somehow customize how versions are parsed and how they are compared.  I think this would allow you to keep this package relatively simple, while giving consumers of this package the flexibility they may need to tweak things.  win win :)

I was also thinking a compare function. Something like the functions used to sort in the standard lib.
I'm not 100% certain yet, but I think you have me convinced this is a good idea to add. 😉 Stay tuned.

EDIT: just a heads up that "0009" > "0010" == true so this may not solve it.

@ashwilliams1 Just saw your edit. That expression is false. 😊 Zero prefixing should work as intended. https://play.golang.org/p/X997rVOHIbg

Closing this now, may revisit if the need arises.


Change background color

Hi, what css do I need to write into my custom css atom file if I want to decrease the brightness of the grey editor background?
It think it is a really good looking theme but for me personally the background is too bright to look at it all day long. (In the risk of the grey color being part of a color composition together with the other colors. Maybe there is a bigger thought behind this?) Thanks.

The current background color is: hsl(222, 11%, 18%). To make it darker, you could change the last percentage (18%) to a lower number.. maybe something like this:
// in your `styles.less` file
.theme-one-dark-ui atom-text-editor {
  background-color: hsl(222, 11%, 12%);
}

Thanks a lot!

Am 27.03.2015 um 13:18 schrieb simurai<EMAIL_ADDRESS>The current background color is: hsl(222, 11%, 18%). To make it darker, you could change the last percentage (18%) to a lower number.. maybe something like this:
// in your styles.less file
.theme-one-dark-ui atom-text-editor {
background-color: hsl(222, 11%, 12%);
}
—
Reply to this email directly or view it on GitHub.


Add generate options

BodyStringify: Requests are controlled by Content-type

@tonybase please take a look at the failing CI.
Regarding commit messages we enforce conventional commits in order to automatically tag new versions.
I suggest squashing all commits and prefixing with feat:


unrandomize enemy shooting

Unrandomize enemy shooting so there is some sort of pattern to the shooting, but have it still look somewhat random.

Need to add flag to turn randomization on or off.


[Linux] Shadow Resolution below 2.0 breaks portraits on Re;birth2

I don't believe this is intended behavior but if Shadow Resolution is set to 1.0 portraits completely disappear, if set above 1.0 but below 2.0 they partially appear, doesn't happen in Re;birth1 with the same configuration file, unfortunately I don't have a Windows computer to test if this behavior occurs.
Shadow resolution at 1.0:

Shadow resolution 1.75:

Log file
nep.log

I don't see obvious issues with texture/viewport sizes in the log... try disabling some functionality at the bottom of the .ini file. It is indeed not an intended look.

If I disable shadow control it fixes it (or set the resolution to 2.0 or above), I tried disabling all the other options to no avail, it's not really a big deal but if it is reproducible it might be a good idea to add a note to the readme. I can test further if needed but this happens on Steam Deck and my desktop, both are AMD Mesa unsure if it's relevant

Very interesting. I tested it on Linux & Windows - but with an Nvidia GPU - and didn't see this bug.
Thanks for the heads-up, I'll add this warning later. I don't have anything with AMD GPU, so can't really guess what's going wrong. Looks like a difference in driver behaviour.


Depth render cleanup

Cleans up ininity/"camera far" treatment of the depth buffer render pass.
Instead of hardcoding infinity at 30K, we now treat infinity as camera far + 1 for the purposes of depth.

This cleans up some post processing effects that depend on infinite depth.


copy loops and gets stuck at the same files

Are you requesting a feature or reporting a bug?
bug report
If reporting a bug, is there already an issue open for this same bug?
I have searched and others have mentioned the issue but i could not see any solution.
What is the current behavior?
copies files until it gets to three specific files where it shows errors:

Error:  API call to drive.files.copy failed with error: This file cannot be copied by the authenticated user.. File: GDriveService (Copy Folder). Line: 93
Error:  API call to drive.files.copy failed with error: File not found: . File: GDriveService (Copy Folder). Line: 93
Error:  API call to drive.files.copy failed with error: File not found: . File: GDriveService (Copy Folder). Line: 93
Then the copy process freezes.  I have "continued" the process three times to the same results at the same three files.

What is the expected behavior?
continue the copy process
How would you reproduce the current behavior (if this is a bug)?
i have run the copy job over again a few times and these errors always occur on the same three files
Specify your

operating system: windows 10
browser: firefox

Any help hugely appreciated :)

Same here. If I open the file with error, ir opens fine, But for some reason the script stops with one particular file.


Change Request: Protocol Layer Description for General Guidelines

The Basics

Which design guideline is affected?

New Guideline in the "API Surface" Layer

Which target languages are affected?

All hierarchical languages.

Describe the change:

~ Must
Provide access to the underlying auto-generated REST library that provides access to 100%
of the service functionality.  In hierarchical objects, this access should be through a member
 called `Protocol`.
~

~ Must
Only include methods within the service client required to support use cases that the majority
of consumers will handle.  Defer all other use cases to either a supporting library or the Protocol
object.
~

Reasoning
In many client libraries, the actual surface that a typical application uses is minuscule and much less than the 100% functionality of the service.  Our aim should be to reduce the API surface exposed to the majority of consumers while still maintaining high fidelity with the service.
To do this, we split the client library into three portions:

A protocol layer whose only job is to interact with the REST interface of the service.  This should be auto-generated from the Swagger.
A "use case specific" convenience layer - a set of methods that provide the appropriate methods to implement common use cases.
Other methods within the convenience layer can be added on an as needed (additive) basis.

This means we will be making value judgments on the admissibility of the a use case to the convenience layer.  How likely is the use case to the common consumer?  Is it common in a back-end web service, a single-page application, or a mobile or desktop app?  What percentage of consumers are likely to need the functionality?
Common use cases should be easy.  All other use cases should be possible.
Examples:

KeyVault - getting the latest version of a secret from the secret store is a common use case.  Listing all the versions of a secret, or updating a secret is not a common use case.  The former should have a nice convenience layer - the latter probably not.
Storage - uploading and downloading a blob is a common use case.  Updating specific blocks in a block blob probably is not a common use case.
Cosmos - the typed search capabilities are probably core.  The streaming capabilities are probably not.

Copying @KrzysztofCwalina @JeffreyRichter @johanste @GorNishanov @bterlson @JonathanGiles

Proposal was rejected by the board.


Retrieving very large trees

When retrieving a tree slightly over 500 MB, I get a failure.
Note that this tree is simply the AST of a single (admittedly large) file. It is not like I am asking for a tree representing a huge codebase, so conceptually I would expect to be able to retrieve this tree.
If I specify a BODY_LIMIT of 500mb (BODY_LIMIT=500mb PGHOST=localhost npm run dev), I get this:
PayloadTooLargeError: request entity too large (received 524304384, when limit is)
    at Gunzip.onData (/Users/ftomassetti/repos/lionweb-repository/node_modules/raw-body/index.js:260:12)
    at Gunzip.emit (node:events:513:28)
    at addChunk (node:internal/streams/readable:324:12)
    at readableAddChunk (node:internal/streams/readable:297:9)
    at Readable.push (node:internal/streams/readable:234:10)
    at Zlib.processCallback (node:zlib:566:10)

This is expected.
So I just increase the BODY_LIMIT to 750mb. At this point, I get the unexpected failure:
Api.getNodes: {"ids":["tenant-strumenta___EGL-myID"]} depth 1 clientId: [object Object]
/Users/ftomassetti/repos/lionweb-repository/node_modules/raw-body/index.js:266
      buffer += decoder.write(chunk)
                        ^

RangeError: Invalid string length
    at Gunzip.onData (/Users/ftomassetti/repos/lionweb-repository/node_modules/raw-body/index.js:266:25)
    at Gunzip.emit (node:events:513:28)
    at addChunk (node:internal/streams/readable:324:12)
    at readableAddChunk (node:internal/streams/readable:297:9)
    at Readable.push (node:internal/streams/readable:234:10)
    at Zlib.processCallback (node:zlib:566:10)

The code that is compressing the response seems to fail.
This may be related to this issue https://github.com/nodejs/node/issues/35973 which says that V8 has an issue dealing with data > 512MB.
Now, even if we could get this solved somehow, I wonder if we should aim to send smaller responses and have some sort of paginated answer. Perhaps when getting a very large SerializationChunk we could send:

The first N nodes
The IDs of the remaining nodes that are not included in the response but which can be subsequently requested by the client

Something like:
// answer sent for a retrieve request
{
   data: SerializationChunk,
   remainingNodeIDs: [string] // this should be empty for "normal" requests
}

At this point, the Client could ask for the other nodes (using the existing retrieve call) and then recompose the tree on its own.
This would be a slightly more complex client, but I think that many clients could have issues dealing with responses this large.
What do you think @joswarmer and @dslmeinte ?

But how would you know beforehand whether you're requesting a large number of (large) nodes, and might want to paginate them? Or would you just always paginate with a large window–thousands of nodes, probably. Would that window always suffice? Given that there is no trivial linear ordering on nodes, what order would you use? And can order change in between paginated calls?
Just some “random” questions...

See e.g. https://www.cybertec-postgresql.com/en/pagination-problem-total-result-count/

Could we draw some inspiration from the TCP/IP protocol which essentially solves the same problem?

@ftomassetti The nice things about your proposed solution is that we don not need pagination at the server side at all, as the client can just ask for the nodes in the remainingNodeIds array.
This is interesting because pagination at the server side seem to imply (after a little bit of reading) either rerunning the entire query for each page, or storing the results of the query at the server side (with the risk of forgetting to explicitly delete these).

@ftomassetti The nice things about your proposed solution is that we don not need pagination at the server side at all, as the client can just ask for the nodes in the remainingNodeIds array.

Yes, my goal is to make this the server not having to "remember" about the query.

But how would you know beforehand whether you're requesting a large number of (large) nodes, and might want to paginate them? Or would you just always paginate with a large window–thousands of nodes, probably.

Perhaps you can specify an optional limit, with a default limit being sufficiently high

Would that window always suffice? Given that there is no trivial linear ordering on nodes, what order would you use? And can order change in between paginated calls?

Well, in the original chunk we have some order of the nodes. We can use that. The problem is then remembering that ordering, but we can delegate that to the client passing remainingNodeIds

Could we draw some inspiration from the TCP/IP protocol which essentially solves the same problem?

I am afraid I will need to refresh my memory about TCP/IP :D

just have to make sure the number of ids sent won't get too big.

That is indeed an issue. If someone would ask a crazy number of IDs, even the list of IDs alone could be too big. But in general I think that an ID is probably 1/100 or less of the size of a single node, so I think it means we should be able to serve requests that would at least 100 times bigger (something like 50GB)

Another couple of options:

Do not return the entire list of IDs to retrieve, but just IDs representing entire sub-trees missing. We could perhaps return trees up to a certain depth, and then specifying the list of IDs of the roots of the sub-trees we are not returning
We could consider using a binary format. While it is true that we compress the JSON we send out (so that the payload transmitted is not that huge) here the problem is that the original JSON is so big that we are not able to compress it. Perhaps moving to another format could reduce the payload by one or two orders of magnitude

We could consider using a binary format. While it is true that we compress the JSON we send out (so that the payload transmitted is not that huge) here the problem is that the original JSON is so big that we are not able to compress it. Perhaps moving to another format could reduce the payload by one or two orders of magnitude

This sounds like an interesting idea. Wrapping the same data structures in e.g. protobuf should not change the semantics.

If the limit is on about 50GB I don't think this is going to be a problem for a long time. I am happy to take your approach until we run into problems (or not).

“Scalability is a luxury problem.” — source unknown (maybe even me)

I'm not surprised by these findings, as compression is really good and sophisticated these days. Combining a binary format and compressing that might eke out a couple of promilles still, but I'd agree beforehand it's not worth the effort.
The only way I could see a use/need for a binary format is that deserializing from one could be optimized better than deserializing from the compressed, streaming but still chatty LionWeb JSON format.

Maybe we'd have a very easy first step: Don't pretty-print -- I'm not sure whether lw-repo does that?

Some data about the file being parsed: it has 31301 lines. So it is definitely a big file, but unfortunately, some codebases seem to have files of that size.
I can also change the way in which ASTs are converted into LionWeb nodes. Each AST node has a Position composed of a start and an end Point. We are currently creating a LionWeb node for the Position and each Point, so we are creating 4 LionWeb nodes for each AST Node. Treating Positions as primitive types in LionWeb would probably reduce the file size being parsed by almost 4. This could be an optimization advice to give to users: when possible, use primitive values and not nodes.

The aforementioned branch can serialize a model with 760.000 nodes with 500 MB of heap.
The resulting file is 705 MB with indentation, 362 MB without.

The aforementioned branch can serialize a model with 760.000 nodes with 500 MB of heap.
The resulting file is 705 MB with indentation, 362 MB without.

I am seeing the exception on the Node side: from Java, with the version in master, I can create the model, but then the Gzip interceptor on the Node side causes the explosion when trying to decompress the zip. Your changes could probably make it faster, but to solve the exception I think we need to send less data to Node

I guess you would need to apply similar optimizations on the node side:

Never store the complete JSON, neither as objects nor as string
(De-)serialize with a streaming API
Don't cache too much. For example, my implementation iterates through all nodes twice: Once to collect used languages, once to serialize nodes
I'm not sure about the repo implementation, but maybe you could go one step further: In Java, I built the full node tree in memory (as that's my "source of truth"). The repo has everything in tables, so maybe it could skip building the node tree completely and stream JSON nodes directly. Collecting the used languages might be possible with a separate SQL query -- databases tend to be pretty good at aggregating (-:

I guess you would need to apply similar optimizations on the node side:

(De-)serialize with a streaming API

Yes, but the decompression is performed through an interceptor that seems to be built-in express. The server explodes before we get a chance to serve the request with user code. I would not know how to make express run the unzip process to produce a stream and not a string.
The stacktrace is this:
    at Gunzip.onData (/Users/ftomassetti/repos/lionweb-repository/node_modules/raw-body/index.js:260:12)
    at Gunzip.emit (node:events:513:28)
    at addChunk (node:internal/streams/readable:324:12)
    at readableAddChunk (node:internal/streams/readable:297:9)
    at Readable.push (node:internal/streams/readable:234:10)
    at Zlib.processCallback (node:zlib:566:10)

I'm not sure about the repo implementation, but maybe you could go one step further: In Java, I built the full node tree in memory (as that's my "source of truth"). The repo has everything in tables, so maybe it could skip building the node tree completely and stream JSON nodes directly. Collecting the used languages might be possible with a separate SQL query -- databases tend to be pretty good at aggregating (-:

I am not sure I understand this point. On the server side I have the tree in memory, and I produce the JSON in order to send it to the server
So far the only idea that I can't think of, that is a form of streaming is this:

We know the problem is present when sending a very large number of nodes (e.g., 500,000)
We could treat all store requests below a certain threshold (e.g., 100,000 nodes) as we do know. So we would need to activate special logic only for very large store requests. As part of this special logic we would need to split them on the client side, so that the server does not explode when receiving them
If we have, for example, 500,000 nodes to send and we decide to send 100,000 nodes at the time to the server (so making 5 requests and not just one) we have a problem: the first 100,000 nodes could have children or references in the nodes we will send later. So if we just send those the server could refuse the requests as we may have references to nodes the server does not know about. I see three options here:

Option I: server aware of partial requests
We can make the server aware that this is a partial request and having it waiting to get all the parts before storing those on disk. Now that is more work for the server because it needs to keep this data in memory and possibly handle some sort of timeout to clean the memory (in case the partial requests are never completed)
Option II: hiding to the server the fact that we are sending partial requests
We could have the client split the request into single requests which are individually valid. To do so we would need to remove all pointers to nodes not yet sent. So, if I am about to send a node N1, with references or containments referring to a node N2, which I am not going to send right away, then I have to change N1 in my request, removing the pointer to N2.
Later on, when I send N2, I will need to go back and say to the server that N1 actually had a pointer to N2. This may be a bit of work, but all of the work would be on the client. The server would receive a set of requests that would be individually valid.
The drawback of this approach is that, in case N2 already exist, I would effectively delete it in between my requests. Eventually I will get the expected status on the server. Also, we may have for any reason only some of the requests go through (e.g., the client or the server die when we sent half of the requests we need to send). This could leave us in an inconsistent state
Option III: use web-sockets
Perhaps if we use web sockets we can send block of nodes and receive all of them on the server before processing the storing request

I guess you would need to apply similar optimizations on the node side:

(De-)serialize with a streaming API

Yes, but the decompression is performed through an interceptor that seems to be built-in express. The server explodes before we get a chance to serve the request with user code. I would not know how to make express run the unzip process to produce a stream and not a string.

Sorry, I don't know anything about JavaScript ecosystem or this tool, so I don't know.

I'm not sure about the repo implementation, but maybe you could go one step further: In Java, I built the full node tree in memory (as that's my "source of truth"). The repo has everything in tables, so maybe it could skip building the node tree completely and stream JSON nodes directly. Collecting the used languages might be possible with a separate SQL query -- databases tend to be pretty good at aggregating (-:

I am not sure I understand this point. On the server side I have the tree in memory, and I produce the JSON in order to send it to the server

This is specific to Jos' repository implementation. In there, we might get away with never instantiating the whole tree in memory, but only forwarding single nodes from the database to the output stream.

Branch https://github.com/LionWeb-io/lionweb-repository/tree/niko/trial-big-result contains a seriously ugly stack of hacks, but it does serve 235 MB of JSON from a node process that never takes more than 59 MB of memory.
No compression, it serves the same node 600.000 times.

Now my repository hack can also receive 700 MB sized models:
Server is running at port 3005 =========================================================
WARNING! The server is not protected by a token. It can be accessed freely. If that is NOT your intention act accordingly.
 * createPartitions request received, with body of 739658239 bytes


Metrics-server and heapster should use API proxy to connect to the node port

/kind feature
What happened:
Both heapster and metrics-server use a direct connection to the kubelet to collect metrics. This is not a secure solution as it will allow everyone in the cluster to connect to the kubelet ports. We should do a similar approach like prometheus where we use the node proxy in the kubernetes api:
https://kubernetes.default.svc:443/api/v1/nodes/<nodename>/proxy/metrics/cadvisor
What you expected to happen:
We would need to provide the ability in the upstream node proxy to connect via the api endpoints (and should not forget the RBAC permission for it

/assign

/unassign

@simonswine are you saying this would involve a change to upstream metrics-server and heapster?
I think we can just:

Make Heapster/metrics-server use secure kubelet port 10250
Make Heapster/metrics-server use service account bearer token
Enable token auth on the kubelet: --authentication-token-webhook=true

Heapster/metrics-server wouldn't go through the api server but the connections would be encrypted and mutually authenticated and we could disable the read-only kubelet port

I don't know if it's possible to prevent everyone from connecting to the kubelet ports, 10250 needs to be open for log collection/execing/port-forward AFAIK - but the above would at least force it to require credentials/RBAC permissions

This is still a valid problem. Metrics server can't scrape the masters' kubelets because of that


Add example compaction configuration

This PR adds an example configuration for the compaction plugin, affecting Suricata data.
:memo: Checklist

[n/a] All user-facing changes have changelog entries.
[n/a] The changes are reflected on vast.io, if necessary.
[X] The PR description contains instructions for the reviewer, if necessary.

:dart: Review Instructions
Please ensure that the appropriate versions of the keywords in transition (pipeline/transform etc.) are used.

I'm fine with this PR generally, but I think the file deserves some explanatory comments and the compaction directory a README. We also already bundle an example configuration with the compaction plugin, so maybe it'd be better to move the content over there rather than to duplicate it here.

so maybe it'd be better to move the content over there rather than to duplicate it here.

I think it would be nice to link this from the docs, or even slurp it in on one page entirely. From this perspective, having it here makes more sense to me.

Well, maybe I got it wrong but the idea was to have a place to keep the config we discussed and work on it together. I was unsure whether a PR to the main VAST repo would even be the right thing to do, since this is far from ready.
The compaction plugin isn't open source, is it? So it's difficult to open a PR against that ;)

Well, maybe I got it wrong but the idea was to have a place to keep the config we discussed and work on it together. I was unsure whether a PR to the main VAST repo would even be the right thing to do, since this is far from ready.

I think that's acceptable. Part of putting our documentation out there in a soon-to-be-reached state helped us a lot to see whether the concepts hold up in a bigger picture. This would be a similar approach.

The compaction plugin isn't open source, is it? So it's difficult to open a PR against that ;)

In general, the idea is to put everything in the open-source documentation, but mark the places explicitly when they required closed-source plugins. We do that already for the matcher, so this would be the same pattern here.
@dominiklohmann and I will take the discussion on how to improve this workflow elsewhere. If you have an idea, don't hold back with feedback. 🙂

@satta I'm in the process of rolling this out for testing on our internal testbed as part of the v2.2 release process.

I think you might need to remove the summarize from the plugin list since that's included now

I had to rewrite much more actually, this is the configuration we're testing now:
vast:
  plugins:
    - bundled

  db-directory: "/var/lib/vast/db"

  log-file: "/var/log/vast/server.log"
  log-rotation-threshold: 256MiB

  store-backend: segment-store
  start:
    disk-budget-high: 95
    disk-budget-low: 90
    disk-budget-check-binary: /opt/vast/dbsize.sh

  max-queries: 10
  # max-taste-partitions: 10

  console-format: "[%l] %v"
  console-sink: stderr

  enable-metrics: true
  metrics:
    self-sink:
      enable: false
      slice-size: 128
    file-sink:
      enable: true
      real-time: true
      path: "/tmp/vast-metrics.log"
    uds-sink:
      enable: true
      real-time: true
      path: "/tmp/vast-metrics.sock"
      type: "datagram"

  pipelines:
    aggregate-flows:
      - summarize:
          group-by:
            - timestamp
            - src_ip
            - dest_ip
            - dest_port
            - proto
          time-resolution: 1 hour
          aggregate:
            timestamp_min:
              min: timestamp
            timestamp_max:
              max: timestamp
            flow.pkts_toserver: sum
            flow.pkts_toclient: sum
            flow.bytes_toserver: sum
            flow.bytes_toclient: sum
            flow.start: min
            flow.end: max
            flow.alerted: any
            src_port: distinct
            community_id: distinct
      - extend:
          fields:
            event_type: flow_agg
      - rename:
          schemas:
            - from: suricata.flow
              to: suricata.flow_agg

    aggregate-dns:
      - summarize:
          group-by:
            - timestamp
            - src_ip
            - dest_ip
            - dest_port
            - proto
            - dns.rrname
            - dns.rrtype
            - dns.type
          time-resolution: 1 hour
          aggregate:
            timestamp_min:
              min: timestamp
            timestamp_max:
              max: timestamp
            src_port: distinct
            community_id: distinct
            dns.rcode: distinct
            dns.answers: distinct
      - extend:
          fields:
            event_type: dns_agg
      - rename:
          schemas:
            - from: suricata.dns
              to: suricata.dns_agg

    aggregate-snmp:
      - summarize:
          group-by:
            - timestamp
            - src_ip
            - dest_ip
            - dest_port
            - proto
            - snmp
          time-resolution: 1 hour
          aggregate:
            timestamp_min:
              min: timestamp
            timestamp_max:
              max: timestamp
            community_id: distinct
      - extend:
          fields:
            event_type: snmp_agg
      - rename:
          schemas:
            - from: suricata.snmp
              to: suricata.snmp_agg

    aggregate-smb:
      - summarize:
          group-by:
            - timestamp
            - src_ip
            - dest_ip
            - dest_port
            - proto
            - smb.session_id
          time-resolution: 1 hour
          aggregate:
            timestamp_min:
              min: timestamp
            timestamp_max:
              max: timestamp
            smb.id: max
            smb.size: sum
            smb.command: distinct
            smb.filename: distinct
            smb.disposition: distinct
            smb.share: distinct
            smb.ntlmssp.host: distinct
            smb.ntlmssp.domain: distinct
            smb.ntlmssp.user: distinct
            smb.named_pipe: distinct
            smb.share_type: distinct
      - extend:
          fields:
            event_type: smb_agg
      - rename:
          schemas:
            - from: suricata.smb
              to: suricata.smb_agg

    aggregate-http:
      - summarize:
          group-by:
            - timestamp
            - src_ip
            - dest_ip
            - dest_port
            - proto
            - http.hostname
            - http.url
            - http.http_port
            - http.http_method
          time-resolution: 1 hour
          aggregate:
            timestamp_min:
              min: timestamp
            timestamp_max:
              max: timestamp
            http.length: sum
            http.user_agent: distinct
            http.content_type: distinct
            http.referer: distinct
            http.status: distinct
      - extend:
          fields:
            event_type: http_agg
      - rename:
          schemas:
            - from: suricata.http
              to: suricata.http_agg

    aggregate-tls:
      - summarize:
          group-by:
            - timestamp
            - src_ip
            - dest_ip
            - dest_port
            - proto
            - tls.sni
            - tls.fingerprint
          time-resolution: 1 hour
          aggregate:
            timestamp_min:
              min: timestamp
            timestamp_max:
              max: timestamp
            smb.size: sum
            tls.ja3.hash: distinct
            tls.ja3s.hash: distinct
            tls.subject: sample
            tls.issuerdn: sample
            tls.serial: sample
            tls.notbefore: sample
            tls.notafter: sample
      - extend:
          fields:
            event_type: tls_agg
      - rename:
          schemas:
            - from: suricata.tls
              to: suricata.tls_agg

plugins:
  compaction:
    time:
      interval: 1 hour
      step-size: 1
      rules:
        - after: 1 day
          name: flow-reduction
          pipeline: aggregate-flows
          types:
            - suricata.flow
        - after: 1 day
          name: dns-reduction
          pipeline: aggregate-dns
          types:
            - suricata.dns
        - after: 1 day
          name: snmp-reduction
          pipeline: aggregate-snmp
          types:
            - suricata.snmp
        - after: 1 day
          name: smb-reduction
          pipeline: aggregate-smb
          types:
            - suricata.smb
        - after: 4 days
          name: http-reduction
          pipeline: aggregate-http
          types:
            - suricata.http
        - after: 4 days
          name: tls-reduction
          pipeline: aggregate-tls
          types:
            - suricata.tls

    space:
      interval: 30 min
      step-size: 1
      mode: weighted-age
      disk-budget-high: 80
      disk-budget-low: 70
      scan-binary: /opt/vast/dbsize.sh

Many thanks, I was just starting to do that after trying out the new 2.2 build. Will test the above as well 👍🏻

Note that I divided all the timings by 7 to make this easier to test, and I reduced the step sizes to 1 because the server I've rolled this out on is a bit memory-constrained.

@satta this is the configuration that we ended up using on our testbed, and it works really nicely. Can you replace the contents of the vast.yaml in the PR with it?
vast:
  pipelines:
    aggregate-flows:
      - summarize:
          group-by:
            - timestamp
            - src_ip
            - dest_ip
            - dest_port
            - proto
          time-resolution: 1 hour
          aggregate:
            timestamp_min:
              min: timestamp
            timestamp_max:
              max: timestamp
            flow.pkts_toserver: sum
            flow.pkts_toclient: sum
            flow.bytes_toserver: sum
            flow.bytes_toclient: sum
            flow.start: min
            flow.end: max
            flow.alerted: any
            src_port: distinct
            community_id: distinct
            count:
              count: event_type
      - extend:
          fields:
            event_type: flow_agg
      - rename:
          schemas:
            - from: suricata.flow
              to: suricata.flow_agg

    aggregate-dns:
      - summarize:
          group-by:
            - timestamp
            - src_ip
            - dest_ip
            - dest_port
            - proto
            - dns.rrname
            - dns.rrtype
            - dns.type
          time-resolution: 1 hour
          aggregate:
            timestamp_min:
              min: timestamp
            timestamp_max:
              max: timestamp
            src_port: distinct
            community_id: distinct
            dns.rcode: distinct
            dns.answers: distinct
            count:
              count: event_type
      - extend:
          fields:
            event_type: dns_agg
      - rename:
          schemas:
            - from: suricata.dns
              to: suricata.dns_agg

    aggregate-snmp:
      - summarize:
          group-by:
            - timestamp
            - src_ip
            - dest_ip
            - dest_port
            - proto
            - snmp
          time-resolution: 1 hour
          aggregate:
            timestamp_min:
              min: timestamp
            timestamp_max:
              max: timestamp
            community_id: distinct
            count:
              count: event_type
      - extend:
          fields:
            event_type: snmp_agg
      - rename:
          schemas:
            - from: suricata.snmp
              to: suricata.snmp_agg

    aggregate-smb:
      - summarize:
          group-by:
            - timestamp
            - src_ip
            - dest_ip
            - dest_port
            - proto
            - smb.session_id
          time-resolution: 1 hour
          aggregate:
            timestamp_min:
              min: timestamp
            timestamp_max:
              max: timestamp
            smb.id: max
            smb.size: sum
            smb.command: distinct
            smb.filename: distinct
            smb.disposition: distinct
            smb.share: distinct
            smb.ntlmssp.host: distinct
            smb.ntlmssp.domain: distinct
            smb.ntlmssp.user: distinct
            smb.named_pipe: distinct
            smb.share_type: distinct
            count:
              count: event_type
      - extend:
          fields:
            event_type: smb_agg
      - rename:
          schemas:
            - from: suricata.smb
              to: suricata.smb_agg

    aggregate-http:
      - summarize:
          group-by:
            - timestamp
            - src_ip
            - dest_ip
            - dest_port
            - proto
            - http.hostname
            - http.url
            - http.http_port
            - http.http_method
          time-resolution: 1 hour
          aggregate:
            timestamp_min:
              min: timestamp
            timestamp_max:
              max: timestamp
            http.length: sum
            http.user_agent: distinct
            http.content_type: distinct
            http.referer: distinct
            http.status: distinct
            count:
              count: event_type
      - extend:
          fields:
            event_type: http_agg
      - rename:
          schemas:
            - from: suricata.http
              to: suricata.http_agg

    aggregate-tls:
      - summarize:
          group-by:
            - timestamp
            - src_ip
            - dest_ip
            - dest_port
            - proto
            - tls.sni
            - tls.fingerprint
          time-resolution: 1 hour
          aggregate:
            timestamp_min:
              min: timestamp
            timestamp_max:
              max: timestamp
            smb.size: sum
            tls.ja3.hash: distinct
            tls.ja3s.hash: distinct
            tls.subject: sample
            tls.issuerdn: sample
            tls.serial: sample
            tls.notbefore: sample
            tls.notafter: sample
            count:
              count: event_type
      - extend:
          fields:
            event_type: tls_agg
      - rename:
          schemas:
            - from: suricata.tls
              to: suricata.tls_agg

plugins:
  compaction:
    time:
      interval: 1 day
      step-size: 3
      rules:
        - after: 1 day
          name: flow-reduction
          pipeline: aggregate-flows
          types:
            - suricata.flow
        - after: 1 day
          name: dns-reduction
          pipeline: aggregate-dns
          types:
            - suricata.dns
        - after: 1 day
          name: snmp-reduction
          pipeline: aggregate-snmp
          types:
            - suricata.snmp
        - after: 1 day
          name: smb-reduction
          pipeline: aggregate-smb
          types:
            - suricata.smb
        - after: 7 days
          name: http-reduction
          pipeline: aggregate-http
          types:
            - suricata.http
        - after: 7 days
          name: tls-reduction
          pipeline: aggregate-tls
          types:
            - suricata.tls
    space:
      interval: 1 hour
      step-size: 3
      mode: weighted-age
      disk-budget-high: 70
      disk-budget-low: 60
      scan-binary: /opt/vast/dbsize.sh
      weights:
        # Keep aggregated flow, fns, snmp and smb logs 5x longer
        - weight: 5
          types:
            - suricata.flow_agg
            - suricata.dns_agg
            - suricata.snmp_agg
            - suricata.smb_agg
        # Keep aggregated http and tls logs 10x longer
        - weight: 10
          types:
            - suricata.http_agg
            - suricata.tls_agg

Done. Thanks!

@dominiklohmann is anything holding this PR up from being merged? IIRC it works.
We could consider linking it from the docs. In this case it would be nice to somehow guarantee that the config is always valid syntax through CI.

@dominiklohmann is anything holding this PR up from being merged? IIRC it works.
We could consider linking it from the docs. In this case it would be nice to somehow guarantee that the config is always valid syntax through CI.

Yes. It's lacking documentation, we don't install it yet as an example config file (unlike the vast.yaml.example file in the repository root) and it's in the wrong place, because it documents a closed-source plugin in the open-source repository, which makes it so we cannot test whether the file is correct.
I was considering two options for this:

We move the file over to the compaction plugin and install it with an example. We also add comments inline and test it in a unit or integration test.
We inline the example into the compaction configuration as an end-to-end example.

I am currently favoring (2) as that's much less effort, and having end-to-end examples in the docs is nice anyways.

We inline the example into the compaction configuration

That would be in the docs then?

We inline the example into the compaction configuration

That would be in the (open) docs then?

Yes, I would say so.

@satta I just discussed this at length with @mavam and @lava and this is the plan we came up with:
I'll take the example configuration we had here, and move it over to the compaction plugin, which will bundle it with every installation. We will also add tests to our CI to ensure that the configuration is valid and produces the expected output.
In addition, we want to take a subset of this and use it instead of the current configuration example for compaction in our docs.
I scheduled this for myself for in ~2-3 weeks. I have a few other higher priority things on my plate until then, but we don't want this PR to stay open all that long anymore.

Cool, agreed. Do we want to close this here then?


Gledopto RGB+CT gu10 bulbs issues

Greetings! Thank you for the work you do on homebridge-hue!
I have Gledopto GU10 (Model: GL-S-007Z) bulbs tied into a gen2 hue hub. I was following the thread in issue 557 but the outcome is not clear to me.
For one, I experience the issue described (and screen-recorded) where choosing a color in the home app then choosing another color replaces the other color with the originally selected color. From my understanding, this is a not-fixable bug in the home app with bulbs that can address both CT and RGB, can someone confirm?
Otherwise, things things work - I can select the temperature or color tabs through the Home app and it successfully turns on/off the white flood lights/RGB lights within the bulb accordingly. RGB colors also work with Siri. The issue arises when trying to set white colors ("white", "warm white") with Siri. It simply sets the (much more dim) RGB channels to max instead of controlling the flood lights at all.
From my understanding, Siri uses HTML color values to set the colors and doesn't actually have any temperature logic involved - is this likely the culprit? (Do the official hue bulbs have a special way they handle this differently?)
Would there be any existing workaround for this, apart from me trying to write logic that converts specific hue/saturation values to white temperature values?
Thanks for your time!

For one, I experience the issue described (and screen-recorded) where choosing a color in the home app then choosing another color replaces the other color with the originally selected color. From my understanding, this is a not-fixable bug in the home app with bulbs that can address both CT and RGB, can someone confirm?

You mean that you try and select one of the six preset colour, but instead of setting the light colour, the preset is changed?  I don’t think that’s related to exposing both Color Temperature and Hue/Saturation.  I’ve seen this behaviour also with forceCt set to false.

From my understanding, Siri uses HTML color values to set the colors and doesn't actually have any temperature logic involved - is this likely the culprit?

For lights that only expose Hue/Saturation, the Home app translates colour temperature values to Hue/Saturation values.  When Color Temperature is (also) exposed, Home sends the colour temperature value.  That part works fine.  The problem is when homebridge-hue updates the Color Temperature (reported by the light) after you set Hue/Saturation from Home.  This is when the colours start “jumping”.  I haven’t tested Siri in this detail.

Would there be any existing workaround for this

Create a HomeKit scene to set the Color Temperature and recall that scene from Siri.  You might need to create the scene in Eve.

Do the official hue bulbs have a special way they handle this differently?

There’s three different types of Hue bulbs:

Gamut A with three channels that don’t support ct, only xy;
Gamut B with three channels that support both xy and ct.  They use different RGB LEDs optimised for mixing colour temperatures, at the expense of fewer supported colours (especially green and orange);
Gamut C with five channels, where ct drives the CW/WW channels and xy the RGB channels, similar to the GLEDOPTO RGB+CCT lights.

Only exposing Hue and Saturation makes sense for gamut B (and, of course, gamut A), since setting ct or the corresponding xy value leads to the same light state (driving the RGB LEDs to mix a colour temperature).  Imho it doesn’t make sense for gamut C, as you cannot drive the white channels from HomeKit.
Also, imho, faking colour temperature for gamut A lights makes no sense, as these lights cannot produce decent colour temperatures.

Hello,
Thank you for your response! that is useful information and interesting to know.
Mostly, I am not using the home app for any of this - I am trying to only control this all with Siri. The problems I am running into are:
1.) Setting a "warm white" color with Siri only adjusts the RGB in the bulb to simulate (poorly) a warm white color, which is also much more dim than using the white LEDs in the bulb.
2.) Setting a scene in home app and trying to invoke that with siri or the app results in the flood lights turning on for an instant, then transitioning back to a white color made by the RGB LEDs (and staying that way when trying to invoke scene again) - I think this is related to the home app not saving colors properly but swapping things around.
I have tried toying with eve but it also has issues addressing the CT aspect of the lights. (Doesn't seem to acknowledge most of the CT spectrum of the lights? Unsure).
My solution, as most of these bugs seem to be unrelated to homebridge-hue (And I mostly just post this for anyone else mostly interested in siri control for bulbs that work with hue but dont appear in homekit) is to make scenes in the hue app and create shortcuts in iOS to address those scenes. Much less modular and requires specific wording, but for my use case, it works. I think in the future I will just invest in the revised hue gu10 bulbs that just came out!
Thanks for your time! (I will leave this open for you to close in case you have any other thoughts to add).


[FEAT] Run packer watch with --allow-live flag

I'm just curious. How hard would it be to allow the packer watch command to be run with --allow-live flag? I am trying to use packer to deploy updates to a live site and I simply can't due to the fact that I'm unable to run the watch command with this flag. Would be super nice and probably really easy to do.

I guess it's a security thing, in the mean time I got past this by installing ThemeKit locally in my theme.
Then rewriting the deploy script: "deploy": "yarn packer build && yarn shopify-themekit deploy --allow-live --env ",.

config.yml in the root with the project dist folder passed in as one of the env options directory: 'dist'

You can also add the "live flag" to the env settings for each theme:
https://hayes0724.github.io/shopify-packer/docs/config/environment
{
    "development": {
      "id": "74500041118",
      "password": "ebd6ce7f27aae8cdafb8111a5b887b9",
      "store": "my-store-name.myshopify.com",
      "live": "false",
      "ignore": [
        "settings_data.json"
      ]
    }
}

I always use a dev theme and push or publish to live so I have not been able to test this.

@Eben-Hafkamp @justin-segler Just tested and both packer start and packer watch on a live theme (using env settings not flags):

Edit env live value to true
packer start or packer watch
Edit live theme 👍

{
    "development": {
      "id": "74500041118",
      "password": "ebd6ce7f27aae8cdafb8111a5b887b9",
      "store": "my-store-name.myshopify.com",
      "live": true,
      "ignore": [
        "settings_data.json"
      ]
    }
}

Awesome, I will make use of this feature. But still like having a themekit config in the root for quick liquid file deployments made straight from the dist. Handy tip for saving time compiling if that's all you need to do


Use page prefetching behaviour from Menu class for section_menu and children_menu tags

Menu model class becomes a standard object class
New MenuFromRootPage class subclasses Menu
New MenuWithMenuItems model subclasses Menu and is subclassed by MainMenu and FlatMenu
section_menu and children_menu tags create a MenuFromRootPage instance, pass them to get_sub_menu_items_for_page and add them to the context to be picked up by any subsequent sub_menu calls
Remove uppercase on labels in USE_SPECIFIC_CHOICES

Resolves #81


Typo in kubeconfig page

On the download page for kubeconfig there is a missing space between automatically and read.
It says automaticallyread.

This GitHub issue/PR is unactive since long time. Is this GitHub ISSUE/PR still needed? Please close or update it accordingly.
This reminder is autogenerated by https://github.com/MalloZup/blacktango


[WIP] Import C types from the cty crate

So this is more or less how solving https://github.com/japaric/cty/issues/13 and https://github.com/japaric/cty/issues/14 would look like. I've kept the whole history of the cty crate.
I think that I found some bugs in both libc and cty in the process.
cc @japaric @Amanieu @retep998 @alexcrichton @denzp

@gnzlbg: no appropriate reviewer found, use r? to override

I would personally pretty strongly discourage this. The libc crate is a fundamental crate in the Rust ecosystem which is designed by RFC and led by the libs team. The same does not apply to the cty crate.
Duplication may be unfortunate but it doesn't matter except for the c_void type really, otherwise all other types are just typedefs which can already unify across the two crates.

After doing this the feeling I got is that it makes more sense to keep both of these separate for now (not change libc to re-export cty), though I'd prefer to host the cty crate here so that we can use libc's CI to make sure its types are correct, both by using ctest to ensure that in the same platforms that libc's support, and maybe also by adding a build test for all targets that makes sure that both crates implement the same API on all targets that libc supports.
I don't think any change here is going to be RFC worthy. The Unsafe Code Guidelines is already documenting that some Rust types exactly match some C types (e.g. size_t, intptr_t, ptrdiff_t, int16_t, etc.), and all of that will be RFC'ed sometime this year. If that gets merged, it makes now more sense to me to just add these types to core::ffi directly.

In terms of hosting I don't think it matters too much where it lives, but I think structurally these should continue to be separate crates for the time being (although an RFC could change that)

So i'm closing this to submit a different PR that does that, don't want to lose this branch because it discovered a couple of bugs.


All pages streaming / no static?

I've noticed that when trying to use this to create a Nextjs 13 application that all pages are being built as streaming pages when using the edge runtime.
Even incredibly simple pages that should be static.
A few questions:

Is this by design?

Is it on roadmap to support caching options in the future?

For a very simple page, since it is presumably ISR, is it effectively static/delivered static anyway?

Depends where you create the page.
If it's in the /app directory it's always SSR if it's in /pages it will be the same as next 12 like you mention.

I'm not having this experience.

Opting out of the app directory
Using Next 13.1.6

If I set the entire project to use experimental-edge, all pages are streaming
If I set only relevant pages to use experimental-edge, I get 404s when deployed to Cloudflare
If you have a setup working where you use edge but have static pages, would you mind sharing it?

So toggling experimental edge seems to turn that very simple about page into a streamed page


How to run multiple instances?

I want to use load balancing, but don't know how to run multiple instance per source.

You can use inbuilt node.js cluster module and it will load balance for you.
Or simply run 2 instances of codebase on different ports.

PORT=4444 node server.js
PORT=5555 node server.js

PM2 also works well for this:
pm2 start server.js -i <number-of-workers>


[x] Have you explained what your changes do, and why they add value to the Guides?

Please note: we will close your PR without comment if you do not check the boxes above and provide ALL requested information.

The following Pull request, fix some grammatical issues and add punctuation to improve the readability of the CONTRIBUTING.md file.

gh pr checkout 2772


Cannot get the type for the node

Description:
Consider the following example
service / on new http:Listener(8080) {
    resource function get getResource(http:Caller caller, http:Request req) {
        http:Client cc = checkpanic  new("http://localhost:9090");
        cc -><cursor>
    }
}

Try to invoke the type(node) API in semantic analyzer for cc in the line of cursor and it returns empty.
Affected Versions:
slp-alpha3-snapshot

Fixed with #29288 hence closing this.


Test SigningStargateClient.sendIbcTokens

After https://github.com/cosmos/cosmjs/pull/750

This can be done in the ts-relayer by letting this function build on the CosmJS code: https://github.com/confio/ts-relayer/blob/v0.1.1/src/lib/ibcclient.ts#L1321-L1364

The implementation is released as part of 0.25.0-alpha.2 and can be tested.

@bartmacbartek Are you ok to take this issue?

I think the first step is updating ts-relayer to 0.25-alpha.2 which seems to have a number of API-breaking changes. Maybe @willclarktech wants to do the API update (as he knows what the breaking changes are) and then the Bartek could take on using the new APIs. It would be more efficient than Bartek trying to update "in the dark"

@webmaster128 OK to close this after https://github.com/confio/ts-relayer/pull/150 ?

I merged that PR, good to close from my end

You are too fast for me


implement reset + forgot flows in auth -- to create "saas starter"

it means we now have
$ rustyrails new
1. saas starter
2. stateless service (no db)
3. empty project ("ping endpoint" , no auth)

1 + 2 was done. #24


⚠ Bofrost.de has degraded performance

In c271b57, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 1647 ms

Resolved: Bofrost.de performance has improved in a61295e.


[BUG] Intermittent ValueError in test_struct_groupby_count

Describe the bug
Runs of test_struct_groupby_count intermittently throw exceptions like:
=================================== FAILURES ===================================
_ test_struct_groupby_count[Struct(not_null)(('a', Struct(('aa', Integer))))] __

self = Row(a=None), item = 'aa'

    def __getitem__(self, item):
        if isinstance(item, (int, slice)):
            return super(Row, self).__getitem__(item)
        try:
            # it will be slow when it has many fields,
            # but this will not be used in normal cases
>           idx = self.__fields__.index(item)
E           ValueError: 'aa' is not in list

Steps/Code to reproduce bug
Repeatedly running the test as  a dedicated Spark app per iteration
while (($? == 0)); do
  ((iter++))
  NUM_LOCAL_EXECS=2 TEST_PARALLEL=1 LD_LIBRARY_PATH=$CONDA_PREFIX/lib \
  SPARK_HOME=~/dist/spark-3.1.1-bin-hadoop3.2 \
  ./integration_tests/run_pyspark_from_build.sh -k test_struct_groupby_count
done

reproduces the error after a few iterations.
Running multiple iterations in the same pytest app does not reproduce the error easily unless the first iteration already hits the bug
NUM_LOCAL_EXECS=2 TEST_PARALLEL=1 LD_LIBRARY_PATH=$CONDA_PREFIX/lib \
SPARK_HOME=~/dist/spark-3.1.1-bin-hadoop3.2 \
./integration_tests/run_pyspark_from_build.sh -k test_struct_groupby_count --count=100 -x

Expected behavior
the test should work reliably
Environment details (please complete the following information)
local-cluster mode reproduces the failure

Related Spark code https://github.com/apache/spark/blob/3959f0d9879fa7fa9e8f2e8ed8c8b12003d21788/python/pyspark/sql/types.py#L1563


[QUERY] Database level  RU limit. doesn't work

Query/Question
Based on Cosmos document we can set RU limit on  database level which is in turn shared among all containers inside it.
So I manually created a database with setting through put of 400 .
But when i start my server , based on all the @Document annotations when it creates containers , each container is getting assigned by default a RU of 4000.  Which is wrong .
So how do i programatically config to limit containers RU to database level RU
I found this in JAVA V4 Sdk but how do i do it using cosmos starter package:
// Container and autoscale throughput settings
CosmosContainerProperties autoscaleContainerProperties
= new CosmosContainerProperties(containerName, "/lastName");
ThroughputProperties autoscaleThroughputProperties
= ThroughputProperties.createAutoscaledThroughput(4000); //Set autoscale max RU/s
Why is this not a Bug or a feature Request?
A clear explanation of why is this not a bug or a feature request?
Setup (please complete the following information if applicable):

OS: [e.g. iOS]
IDE : [e.g. IntelliJ]

com.microsoft.azure
azure-cosmosdb-spring-boot-starter
2.3.2

Information Checklist
Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

[x] Query Added
[x] Setup information Added

Based on my understand i figured out that @Document annotation by default sets it to 4000 RU if we dont specify anything. So how do i stop it and force it to use the RU of database level ???

As you can see , here is the code of @Document annotation
`  private Integer getRequestUnit(Class<?> domainType) {
Integer ru = Integer.parseInt(Constants.DEFAULT_REQUEST_UNIT);
final Document annotation = domainType.getAnnotation(Document.class);
    if (annotation != null && annotation.ru() != null && !annotation.ru().isEmpty()) {
        ru = Integer.parseInt(annotation.ru());
    }
    return ru;
}

`
Here it always forces to creates container with DEFAULT_REQUEST_UNIT  and there is no conf to ignore  and instead use shared RU of DB level.
Please help fixing or providing a option to do so

Thanks for filing these questions, @Abhisar.  Someone from the azure-spring team will follow up on these azure-cosmosdb-spring-boot-starter specific questions

@Abhisar  - we have these features in our backlog. You can track them here:
https://github.com/Azure/azure-sdk-for-java/issues/12711
https://github.com/Azure/azure-sdk-for-java/issues/12715

I see . Do we have an approximate timeline about when it's expected to get out ?

@Abhisar - not yet, this is still in our backlog, will update this issue when we plan to support it.

This issue has been fixed. @Abhisar , I am going to close this issue. If you still see the issue, please feel free to comment or re-open this.


buffer_not_empty condition

Thanks for picking up the baton on this.
I saw some unexpected behavior and noticed that this function might be doing the opposite of what I expected from the readme.
https://github.com/NTBBloodbath/galaxyline.nvim/blob/9e0a3a20633fdfedc8662a2ff370944d65c30836/lua/galaxyline/condition.lua#L3-L6

Hey, yeah I already fixed this (it seems that I forgot to push haha), please update and feel free to reopen this issue if you're still facing this error after the update.
Regards


flutter pod stuck at 1.0.0 forever

Steps to Reproduce

It seems that this problem (if it really is one) was discovered by multiple people, but there is no solution posted anywhere:

https://stackoverflow.com/questions/58086615/flutter-version-on-ios-stuck-at-1-0-0
https://github.com/solid-software/flutter_vlc_player/issues/143

so here's how to reproduce:

Just install most recent flutter sdk and create project

% flutter --version
Flutter 2.2.3 • channel stable • https://github.com/flutter/flutter.git
Framework • revision f4abaa0735 (6 weeks ago) • 2021-07-01 12:46:11 -0700
Engine • revision 241c87ad80
Tools • Dart 2.13.4

create a folder, enter it and run flutter create .
add an arbitrary package to pubspec.yaml, e.g. permission_handler: ^8.1.4+2
run ```pod repo list````

% pod repo list

cocoapods
- Type: git (remotes/origin/master)
- URL:  https://github.com/CocoaPods/Specs.git
- Path: /Users/arth/.cocoapods/repos/cocoapods

trunk
- Type: CDN
- URL:  https://cdn.cocoapods.org/
- Path: /Users/arth/.cocoapods/repos/trunk

2 repos

update pods with pod repo update

% pod repo update
Updating spec repo `cocoapods`
  $ /usr/bin/git -C /Users/arth/.cocoapods/repos/cocoapods fetch origin --progress
  remote: Enumerating objects: 1853, done.
  remote: Counting objects: 100% (1853/1853), done.
  remote: Compressing objects: 100% (1733/1733), done.
  remote: Total 1853 (delta 841), reused 172 (delta 71), pack-reused 0
  Receiving objects: 100% (1853/1853), 477.02 KiB | 6.53 MiB/s, done.
  Resolving deltas: 100% (841/841), completed with 21 local objects.
  From https://github.com/CocoaPods/Specs
     a37a29d61915..82bc64d1fbfe  master     -> origin/master
  $ /usr/bin/git -C /Users/arth/.cocoapods/repos/cocoapods rev-parse --abbrev-ref HEAD
  master
  $ /usr/bin/git -C /Users/arth/.cocoapods/repos/cocoapods reset --hard origin/master
  HEAD is now at 82bc64d1fbfe [Add] DatadogSDKBridge 0.4.6
Updating spec repo `trunk`
<and some stuff about beta version available>

run flutter pub get, enter the ios folder and run pod install

% pod install
Analyzing dependencies
Downloading dependencies
Installing Flutter (1.0.0)
Installing permission_handler (5.1.0+2)
Generating Pods project
Integrating client project
Pod installation complete! There are 2 dependencies from the Podfile and 2 total pods installed.
<and a lot of warnings about iOS target versions>

run pod outdated to see the following result:

% pod outdated
Updating spec repo `trunk`

CocoaPods 1.11.0.beta.2 is available.
To update use: `sudo gem install cocoapods --pre`
[!] This is a test version we'd love you to try.

For more information, see https://blog.cocoapods.org and the CHANGELOG for this version at https://github.com/CocoaPods/CocoaPods/releases/tag/1.11.0.beta.2

Analyzing dependencies
The color indicates what happens when you run `pod update`
<green>	 - Will be updated to the newest version
<blue>	 - Will be updated, but not to the newest version because of specified version in Podfile
<red>	 - Will not be updated because of specified version in Podfile

The following pod updates are available:
- Flutter 1.0.0 -> 1.0.0 (latest version 2.0.2)
<this one is red and cannot be updated!>

Expected results:
I was expecting to see Flutter bumped to 2.0.2, but it stays and remains at 1.0.0 whatever I do.
Actual results:
- Flutter 1.0.0 -> 1.0.0 (latest version 2.0.2)

Logs

no need to run it - it works, but I wonder if 2.0.2 would do good with more advanced packages and why it is not updated!

% flutter analyze
Analyzing tt...
No issues found! (ran in 6.8s)

% flutter doctor -v
[✓] Flutter (Channel stable, 2.2.3, on macOS 11.5.2 20G95 darwin-x64, locale en-US)
    • Flutter version 2.2.3 at /Users/arth/Development/AR4/TESTING/flutter
    • Framework revision f4abaa0735 (6 weeks ago), 2021-07-01 12:46:11 -0700
    • Engine revision 241c87ad80
    • Dart version 2.13.4

[✓] Android toolchain - develop for Android devices (Android SDK version 31.0.0)
    • Android SDK at /Users/arth/Library/Android/sdk
    • Platform android-31, build-tools 31.0.0
    • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version Java(TM) SE Runtime Environment (build 1.8.0_162-b12)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.5.1, Build version 12E507
    • CocoaPods version 1.10.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2020.3)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version Java(TM) SE Runtime Environment (build 1.8.0_162-b12)

[✓] Connected device (1 available)
    • Chrome (web) • chrome • web-javascript • Google Chrome 92.0.4515.131

• No issues found!

Hi @belveder79
First of all, it is not recommend to execute pod install & pod update repo  as Flutter build and Flutter run take care of these and it helps to avoid running into pod specific issues (as Flutter execute these commands itself, this has better compatibility than manual pod commands)
Flutter 1.0.0 -> 1.0.0 (latest version 2.0.2) doesn't change value in Flutter.podspec stays the same as this is just local specification, it doesn't have to change, 1.0.0 is just default value and it won't have an impact on your app build or release
Do you have any use cases where updating Flutter podspec version has any impact?

Hi @TahaTesser,
well, the problem I have is that flutter run does not work for me. It does all the compilation, and it resets some Xcode specific things (like "enable bitcode") for e.g. the Unity Flutter Widget, in case I add or remove a package dependency, while I need to disable bitcode for my Unity package on intent.
The other thing is that I think that there is a reason for the Flutter pod to be updated, so I wonder why the default will always stay at 1.0.0, when there must be a reason for an updated version of the pod !?

Hi @belveder79

well, the problem I have is that flutter run does not work for me. It does all the compilation, and it resets some Xcode specific things (like "enable bitcode") for e.g. the Unity Flutter Widget, in case I add or remove a package dependency, while I need to disable bitcode for my Unity package on intent.

If you don't execute manually  pod commands and add simply add the package you need for your app, it should run out of the box
Can you please provide complete reproducible steps for this issue?

Without additional information, we are unfortunately not sure how to resolve this issue.
We are therefore reluctantly going to close this bug for now.
Please don't hesitate to comment on the bug if you have any more information for us; we will reopen it right away!
Thanks for your contribution.
Could everyone who still has this problem please file a new issue with the exact description of what happens, logs, and the output of flutter doctor -v.
All system setups can be slightly different, so it's always better to open new issues and reference related issues.


solarish add fcntl's O_DIRECT constant.

@bors r+

:pushpin: Commit 4e59d5ee8b0532ac83ce63c9bbad5f108105ebc5 has been approved by JohnTitor
It is now in the queue for this repository.

:hourglass: Testing commit 4e59d5ee8b0532ac83ce63c9bbad5f108105ebc5 with merge cb18b837963c37a8d21732f3ca2c2096f04e6830...

:sunny: Test successful - checks-actions, checks-cirrus-freebsd-13, checks-cirrus-freebsd-14
Approved by: JohnTitor
Pushing cb18b837963c37a8d21732f3ca2c2096f04e6830 to main...


Add a Docker quickstart script for demo / test

Adding a Bash script to simplify setting up and launching local Docker containers with OpenSearch (with this plugin installed) and OpenSearch Dashboards, with the plugin configured to connect to Kendra ranking service.
Signed-off-by: Michael Froh<EMAIL_ADDRESS>Description
Bash script does the following:

Create a Dockerfile that uses the OpenSearch 2.4.0 base image and installs this plugin.
If AWS credentials are found in the specified AWS profile, the current environment, or the default profile, then they get "pushed" into the OpenSearch keystore in the Docker image.
If requested, create a Kendra Intelligent Ranking rescore execution plan and wait for it to become active.
Build the Docker image from Dockerfile and tag it (as opensearch-with-ranking-plugin).
Create a docker-compose.yml that launches two nodes:

OpenSearch based on the image we just built, with config settings pointing to a Kendra Ranking rescore execution plan.
OpenSearch-Dashboards 2.4.0

Run docker-compose up to bring up both nodes.

NOTE: I have tested this on a couple of hosts (running Linux and Mac OS) with temporary and long-lived AWS credentials. I still need to test it on an EC2 instance using instance profile credentials.
Issues Resolved
N/A
Check List

[ ] New functionality includes testing.

[ ] All tests pass

[ ] New functionality has been documented.

[ ] New functionality has javadoc added

[X] Commits are signed as per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Oops -- I marked this as "ready for review", but I want to hold off until we have a released plugin zip. The default plugin zip location is still a 404.


anyway to edit the graph in the website

thanks for the demo, but is there any way to edit the graph on the website, such as adding nodes, adding a connection, etc.

To edit the nodes and edges in the graph one would have to modify the two .csv files defined on lines 26 and 27 shown below:
edge1 = pd.read_csv('edge1.csv')
node1 = pd.read_csv('node1.csv')
The above two lines use pandas dataframe objects to read in the csv files and then display them thereafter using plotly graph objects.
Please note that there is a dedicated node/edge visualization package called cytoscape found here.

I actually read all the source code and know exactly what is happening, but I am just asking are there good way to edit the graph directly on the browser, it seems that dash doesn't have this functionality .

For the most part, you are correct in that Dash and Plotly are meant for static data that lives on the server/host. However, because Dash has a set of components that allow the user to input data, like text boxes and buttons, you could use those components to edit the edges and nodes.
For my own project, I use an instance of dcc.Textarea( ) to allow the user to input equations. Then, I parse those equations and put the symbols into nodes and edges using dash callbacks.  The relevant source is found on my application.py file. I have a dcc.Textarea( ) used and that is related to the page_1_text( ) callback.


Multi projects in same repo with their own pipeline

This isn't necessarily an issue, but more of a question. My team is currently looking at consolidating multiple projects into a single repository, while still letting them have their own pipeline. I guess you could call it a mono-repo. Although this works quite well already, we've found one thing a little bit lacking, which is how paths are handled in this scenario. We've been investing in templates to simplify the creation of pipelines and those templates generally assume that the tasks they run are run from the repository root. Obviously this fails in the face of multiple projects being within the same repository because some things are just nested one level deeper down.
I guess I was somewhat naively thinking this would just work because the pipeline definition itself would be one level deeper down as well and all would be good. Unfortunately that's not how it seems to work, at least not currently. I guess I'm wondering if there are any plans to deal with this sort of thing. For now we've worked around it by extending the templates with additional optional parameters to provide a path prefix, but I guess these problems could be solved by changing the working directory altogether. As far as I can tell there currently is no way to change the working directory at the job level. You can do it for script steps, but it doesn't look supported for tasks for example.
Are there any plans to support this in a more natural way?

In order to consolidate to fewer feedback channels, we've moved suggestions and issue reporting to Developer Community. Sorry for the delay/confusion.
There are no plans to allow changing the working directory for an entire job. Tasks which operate on files should be taking a working directory input if that's relevant to them.


Update Supported-list.md

support rabbitmq link

Coverage decreased (-0.005%) to 14.407% when pulling c800963eed30a84c4c80682a74a13495c495c77d on WithLin:patch-1 into aa9a7c443774d96385f308f2ec1f425f47719958 on apache:master.


Add Org metadata to metadata scope

As a developer I want to get the organisation metadata directly in the token or on the userinfo endpoint of the user, so I do not have to do a reqeust to the api of ZITADEL.
We already have a scope for metadata that returns the metadata of the user. Now we also have organization metadata. We should make a second scope for requesting organization metadata.
Acceptance criteria

[ ] If I send the scope urn:zitadel:iam:org:metadata the metadata of the resourceowner of the user should be included in the token in the same structure as on the user metada
[ ] If I send the scope urn:zitadel:iam:org:metadata the metadata of the resourceowner of the user should be included in the user info endpoint response when I call it

@livio-a @fforootd what should this scope return?
the metadata of the resourceowner org?
the metadata of the organisations where the user has authorizations?
Or do we even need both? And need to differ two scopes?

I think the current change should allow for the resource owner metadata to be returned.

@hifabienne was the un-assignment intended or do you still need an estimation?

It was intended. We quickly discussed the needs of customers and decided for another implementation in actions. which allows to get org metadata.


update dependies.

update dependies:

controller-runtime: v0.9.0
k8s-api: v1.21.1

also add testdata dir for test data, actually they contain empty files.
Signed-off-by: morvencao<EMAIL_ADDRESS>

/hold

@clyang82 Another look at this?

/lgtm

/unhold


update

Made some improvements and code errors

M

Employed good code practices, Made some file updates

dayson good job

😊


Linked images are not always loading

Fix was contained in https://github.com/johannesnoordanus/svg2gcode/pull/9
Without it this scenario can happen (png is in the same folder as the svg):
❯ svg2gcode --constantburn --pixelsize 0.06 --imagespeed 6000 --cuttingspeed 400 --splitfile --cuttingpower 1000 /tmp/dessin.svg /tmp/test.nc
xlink:href="dessin.png"
Image data error: neither file nor data!

I included your fix. It is a bit tedious to solve merge conflicts, but I hope I didn't make typos.
Nice updates by the way!


Fixing build errors in contrib repo

Changes
Please provide a brief description of the changes here.
Merge requirement checklist

[x] CONTRIBUTING guidelines followed
[ ] Unit tests added/updated (if applicable)
[ ] Appropriate CHANGELOG.md files updated for non-trivial, user-facing changes
[ ] Changes in public API reviewed (if applicable)

Closing it. There are too many errors to fix, beyond my domain expertise on these exporters.


Add concretize before generating Chains

Uses MCMCChains.concretize to determine type better. This fixes a bug in the infinite mixture tutorial.
I also switched us over to using import MCMCChains rather than using, since we only really use two things from that package (Chains and concretize).

I added the test. The RandomMeasures.jl tests weren't previously being run (see https://github.com/TuringLang/Turing.jl/issues/973), so I commented them out but added the infinite GMM model.

Thanks!


2022.03a Iteration Plan

SAP teams plan to address the following issues in our next iteration:

Provide a new GitHub repository to provide generation of JSON schema based on ABAP file formats' ABAP type
Close PR #307
Close PR #308
Close PR #309
Close PR #317
Close PR #319

The iteration lasts 2 weeks and ends on March 28, 2022.

Iteration Results
Closed

Provide a new GitHub repository to provide generation of JSON schema based on ABAP file formats' ABAP type; See https://github.com/sap/abap-file-formats-tools
Close PR #307
Close PR #308
Close PR #317

Open

Close PR #309
Close PR #319


reduce allocations on hot path for metrics

Description
While troubleshooting a user memory problem, and checking pprof output it was notice that we have a good amount of these allocations that could be improved.
This is not the main contributor to the memory allocation issue but a small improvement
Issue reference

Please reference the issue this PR will close: #[issue number]
Checklist
Please make sure you've  completed the relevant tasks for this PR, out of the following list:

[x] Code compiles correctly
[ ] Created/updated tests
[ ] Unit tests passing
[ ] End-to-end tests passing
[ ] Extended the documentation / Created issue in the https://github.com/dapr/docs/ repo: dapr/docs#[issue number]
[ ] Specification has been updated / Created issue in the https://github.com/dapr/docs/ repo: dapr/docs#[issue number]
[ ] Provided sample for the feature / Created issue in the https://github.com/dapr/docs/ repo: dapr/docs#[issue number]

maybe not a bad idea, the max number of elements I saw from a grep in pkg is 10. Let me run a benchmark to see if the added logic improves this without much or any performance hit

@ItalyPaleAle I did not see a big difference adding the sync.Pool as there is still another allocation that would require another kind of cache for the tag.Upsert that is adding allocations each time in https://github.com/dapr/dapr/pull/6674/files#diff-bb2a89c3078a94caa5e92dcb3e74a19afa60e05a5fb4b5fd29a52d65628fb66cR69. Maybe this can be added to an lru cache
Something else I did find a bug on grpc metrics where it is missing the appIDKey when adding the tags

@filintod good find on the metrics stuff
As for the LRU cache, i think that may be an overkill, at least for now :)

some interesting info that we should probably look for precomputed tags https://blog.howardjohn.info/posts/zero-alloc-otel/
that also hit Istio in the past

@filintod that seems a very good idea. We sure do allocate a lot of memory in hot paths due to metrics. Can you open a separate issue for that?

@ItalyPaleAle I created this issue to follow up https://github.com/dapr/dapr/issues/6680