Different tap behaviours

Hello,
Couple of UX improvements,

Right now the delete button is in the main screen taking up space. This can be moved to a swipe action. This also makes way for the URL to display nicely.
Tapping on the favicon can open the link directly in a browser.

Hey, @deepu2! They're great suggestions! Thank you for providing them! Please look forward to the very next release!

Hey thought I could give an update on this. The feature is currently under development and will be reflected to the dev branch soon. Thank you for the patience!

I thought about implementing this. And did develop an initial version. However, after creating it, it hit me. While it's a great idea to swipe to dismiss things on mobile devices, it will be a pain to swipe on a desktop, especially if the screen does not support touch. So, I will be replacing it with a card element.
It'll look like this:

Had Sitemarker been mobile only, I would have gone with your idea. However, maintaining different views for desktop and mobile requires a bit of effort. I'll keep this issue open till I push changes from dev branch to main branch for your further feedback.

Hey, @deepu2! I know it came late but the issue is now closed. The implementation follows a new UI. The code update is currently on the main branch. I will make sure a build will be made available for testing by tomorrow. Please let me know if there are any issues regarding the new builds.
Thank you for reporting this as well as your patience. I know it took a lot of time than expected, which kinda bugged me as well.


infinite scroll not enabled with pod?

Hey I see you added infinite scroll 18 days ago but I installed the pod just 3 days ago and I am not able to set picker.infiniteScrollEnabled = YES;  The property is not available and is not seen in DZNPhotoPickerController.h in the pod although I see it when I look through the example on git.

never mind I just needed to update pod. whoops


Disable WindowsAppearFaster by default

Since this seems to be causing crashes for some.
Though I can't replicate this, so maybe it will be useful for some, like myself.

Nope. Starting to crash with it now, too.


BwTreeIndex: Add visibility logic, abort and deferred delete logic, descending and limit scans

This PR addresses #357. It also adds scans that I found useful in writing #335.
Changes:

DataTable pointer is now embedded in each Block. This allows indexes to access the DataTable via only a TupleSlot. Consider removing layout_version if the schema change team doesn't need it.
BwTree scan results are checked for MVCC visibility by consulting the DataTable using TransactionContext before returning results.
index->InsertUnique() checks for MVCC visibility of conflicting keys by consulting the DataTable using TransactionContext.
index->Delete() registers a deferred commit action that then registers a deferred GC action to call delete on the key. This guarantees that the index key remains in the index as long as necessary for any other running transactions.
ScanDescending uses the BwTree's backwards iteration. It's unclear yet if this will prove to be too slow, but for now it's correct and works.
Limit scans will stop early if you want to restrict the size of your results. For example, ScanLimitAscending() with limit 1 is effectively an argmin of a range, while ScanLimitDescending() with limit 1 is effectively an argmax of a range.
Moved tests primarily concerned with CompactIntsKey and GenericKey to their own test suite.
Ported the MVCC tests that made sense to exercise the visibility logic of the BwTreeIndex.

Possibly deferred until we talk about #357.

I've updated the scans to take a TransactionContext and perform visibility checks with the DataTable. I want to write the predicate for unique key constraints while I'm poking at these files, but currently still thinking about all the criteria for that since it extends beyond visibility (need first writer wins as well).

Done with a first pass on implementation. It'll need a review at some point, but for now I'm ready to hand this off to Masha to port MVCC Data Table tests to verify that the BwTreeIndex enforces the same semantics. I brought one test over as an example (and a sanity check for myself), and we can meet up in person to discuss the implementation details a bit more.

What did I learn at school today: HyPer treats updates on indexed attributes as a delete and an insert at the table level, which solves my problem with visibility checking.

The scope of this keeps growing. I kept having to fake in tests what would eventually be the abort logic and deferred delete (handled in GC) and just rolled it into this PR using the new deferred action framework. So far it seems to work well. I'm going to wrap up the basic tests and tag this ready for review soon.

@mbutrovich When will this be merge? Our add/drop indexes project relies on this PR. I can do a review if necessary.

I was trying to avoid the API change to InitializeRawBlock, but I guess we'll live with it.


feat: unstar snippets

included feature to remove the star from the snippets. this implementation maintains the initial /add_star route (in my opinion we could rename it to /star, for clear and simple implementation).
and if the implementation isn't on point, i would be willing to update it with the correct /remove_star if asked.
fixes: #197
/claim #197

@seveibar done with playwright tests, i don;t know what's the issue with bun test check, it passes all the 34 and skips 1 locally. and i haven't changed any api routes this time.

okay, so can i create a method to get the is_starred from snippets/get_star and store it inside a variable using useEffect, and useState hooks (they'll be required now).

@RohittCodes snippets/get returns snippet.is_starred in the production api. The fake api should update to reflect this. /snippets/get_star should is not acceptable for the implementation

i've tried it, but there is no way to get star status if someone has, actually starred it, as according to the current setup, auth: "none", in snippets/get
if i change it to something like this: auth: "session",
then users who are not logged in will face issues with it.
can u just share me how is the logic setup in the prod env.

@RohittCodes you have to introduce a new auth called "optional_session" that adds ctx.auth only if there's a token

Something like this

Okhh got it.

do we need any rework on this? @seveibar


337: Update login text.

Looks like PR deployed and there are no outstanding development tickets. Marking as done.


chore(RingTheory/GradedAlgebra/HomogeneousLocalization) New simp lemmas

Also used the new simp lemmas to golf AlgebraicGeometry/ProjectiveSpectrum/*

Thanks!
bors merge


WIP: bos-tasks-whkbb: Update translations from Fedora Zanata

Update translations from Fedora Zanata

[ ] po-refresh

po-refresh in progress on bos-tasks-whkbb.
Log: http://fedorapeople.org/groups/cockpit/logs/po-refresh-446-20181126-070505/


Fix Mixpanel Readme

Mixpanel config is "token". The docs were saying "tracker".

Thanks! You are completely right


Add support for registering custom types

So I wanted this feature and I tried to implement it.
The implementation is kind of straightforward:

Instead of using the global processor's type registry, we're instantiating a new registry for each DSL instance. It damages the total RAM usage, but it enables us to configure types per-instance
Since we no longer use global registries, I've renamed Processor's type_registry to type_registry_namespace.
config has a new field now: types. I was thinking about renaming it to type_container, WDYT?
I've introduced a new class: Dry::Schema::TypeContainer, which is a simple wrapper around Dry::Types container. We need it for a couple of reasons:

Dry::Types quacks kind of like a container. Except it has registered? instead of key?. Don't want to change the dependency — it would break backwards compatibility
I don't want people to manually run container.merge(Dry::Types.container) as it exposes internal API, nor do I want them to alias registered? to key? for compatibility

So the idiomatic code would look something like this:
module MyFancyProject
  SchemaTypes = Dry::Schema::TypeContainer.new
  SchemaTypes.register('params.my_string', Types::MyString)
end

Schema = Dry::Validation.Params do
  config.types = MyFancyProject::SchemaTypes
  optional(:foo).filled(:my_string)
end

Thanks. I’ll be going through this after dry-validation 1.0.0 is released. First things first.

This is really cool, thank you. I'll make it part of 1.2.0, especially that I've fixed some issues with custom constructor types 😅


kube-config delete-data - strange message

Hi,
I tested the delete-data and even if it seems to work, as I got the doc of umount, seems somethinkg was not done correctly.
[root@kube1 ~]# kube-config delete-data
Do you want to delete all Kubernetes data about this cluster? m(ove) is default, which moves the directories to {,old}. y deletes them and n exits [M/n/y] y

Usage:
 umount [-hV]
 umount -a [options]
 umount [options] <source> | <directory>

Unmount filesystems.

Options:
 -a, --all               unmount all filesystems
 -A, --all-targets       unmount all mountpoints for the given device in the
                           current namespace
 -c, --no-canonicalize   don't canonicalize paths
 -d, --detach-loop       if mounted loop device, also free this loop device
     --fake              dry run; skip the umount(2) syscall
 -f, --force             force unmount (in case of an unreachable NFS system)
 -i, --internal-only     don't call the umount.<type> helpers
 -n, --no-mtab           don't write to /etc/mtab
 -l, --lazy              detach the filesystem now, clean up things later
 -O, --test-opts <list>  limit the set of filesystems (use with -a)
 -R, --recursive         recursively unmount a target with all its children
 -r, --read-only         in case unmounting fails, try to remount read-only
 -t, --types <list>      limit the set of filesystem types
 -v, --verbose           say what is being done

 -h, --help     display this help and exit
 -V, --version  output version information and exit

For more details see umount(8).
Deleted all Kubernetes data

Oops, I didn't think about if no containers was running...
Source:
https://github.com/luxas/kubernetes-on-arm/blob/master/sdcard/rootfs/kube-archlinux/usr/bin/kube-config.sh#L522
The reason is when containers are running: kubelet mounts a tmpfs in every container, serviceAccount secrets. So I umount them so /var/lib/kubelet may be removed.
However, when no tmpfs are mounted it just runs umount => help message.
Will fix

Should work with latest dev now. Thanks for the report. (But do not test dev right now)
I have to close issues when I fix them, otherwise I will forget which ones I have/haven't fixed :smile:


Custom webhook fails action depending on payload.

Hi there, I've faced a weird issue.
Say it we have a webhook action like this: https://gist.github.com/dennybaa/00d1936af2f441cfff15
If I issue the following commands:

curl -X POST --data "{\"message\": \"6fd781b849bb\"}" http://<IP_ADDRESS>:9101/v1/webhooks/sample
curl -X POST --data "{\"hits.rate_15m\": 0.4765927732670934, \"hits.count\": 30, \"tags\": [\"metric\"], \"@timestamp\": \"2015-04-23T13:58:28.319Z\", \"hits.rate_5m\": 0.31123263029226506, \"@version\": \"1\", \"message\": \"6fd781b849bb\", \"hits.rate_1m\": 0.08765952620175455}"

In both cases trigger is dispatched. But the action takes place only in the #1 case.

I can successfully reproduce this. 0.8.3

Thanks for the report.
Are we sure it's action not getting executed and not action getting executed and failing (e.g. echo failing because of unescaped special characters)?
In any case, I can dig in later.

I was thinking about it, but st2 passes json and it should take care of escaping of {{trigger.body}}.
As of bash it self, echo is not failing http://gyazo.com/a369b2b3ea87e9618622f802fcc978bb.
Anyway in the rulesengine log there should be an AUDIT action info about failed or executed action.

I am looking into this.

https://gist.github.com/jfryman/e932eeb74f9b740dacf1

Weird finding! If there is a '.' character in the field name, the rule doesn't work.
~/st2 git:master ❯❯❯ curl -X POST --data "{\"message\": \"6fd781b849bb\"}" http://localhost:9101/v1/webhooks/sample                                                                                ✭ ✖ ✱ ◼
{"message": "6fd781b849bb"}%                                                                                ~/st2 git:master ❯❯❯ curl -X POST --data "{\"message\": \"6fd781b849bb\", \"pi\": 3.14}" http://localhost:9101/v1/webhooks/sample                                                                                ✭ ✖ ✱ ◼
{"message": "6fd781b849bb", "pi": 3.14}%                                                                                ~/st2 git:master ❯❯❯ curl -X POST --data "{\"message\": \"6fd781b849bb\", \"pi.dot\": 3.14}" http://localhost:9101/v1/webhooks/sample                                                                                ✭ ✖ ✱ ◼
{"pi.dot": 3.14, "message": "6fd781b849bb"}%                                                                                ~/st2 git:master ❯❯❯

~/st2 git:master ❯❯❯ cat /tmp/st2.webhook_sample.out                                                                                ✭ ✖ ✱ ◼
{u'message': u'6fd781b849bb'}
{u'message': u'6fd781b849bbc'}
{u'message': u'6fd781b849bbc'}
{u'message': u'6fd781b849bbc'}
{u'message': u'6fd781b849bb'}
{u'message': u'6fd781b849bb', u'pi': 3.14}
~/st2 git:master ❯❯❯

Yeah, confirmed. Changing . to _ makes it work.
~/st2 git:master ❯❯❯ cat /tmp/st2.webhook_sample.out                                                                                ⏎ ✭ ✖ ✱ ◼
{u'message': u'6fd781b849bb'}
{u'message': u'6fd781b849bbc'}
{u'message': u'6fd781b849bbc'}
{u'message': u'6fd781b849bbc'}
{u'message': u'6fd781b849bb'}
{u'message': u'6fd781b849bb', u'pi': 3.14}
{u'hits_rate_15m': 0.4765927732670934, u'tags': [u'metric'], u'hits_rate_5m': 0.31123263029226506, u'@timestamp': u'2015-04-23T13:58:28.319Z', u'hits_count': 30, u'hits_rate_1m': 0.08765952620175455, u'message': u'6fd781b849bb', u'@version': u'1'}
~/st2 git:master ❯❯❯

We have been pickled.

Long story short, we use pickle for serializing objects for rabbitmq transport. We always knew JSON was the route to be taken but used pickle as a shortcut. In this particular case, the shortcut turned out to be disastrous. We need to switch to JSON serialization and it's an involved/major change. It has ability to break functionality. We should target this for 0.10.0 release. For now, the work around is not use _ instead of . in payload fields.

I've encountered a similar issue today when I was working on something today.
The issue is related to the payload containing dots and has nothing to do with pickle. If payload contains a dot, RulesEngine will blow up since TriggerInstance payload value can't contain dots because we don't use a special escaped dict field.
I will open a PR with fix for this shortly.

I've merged PR with a fix into master.
@dennybaa You can test the fix by downloading a new 0.10dev release from https://downloads.stackstorm.net/releases/st2/0.10dev/ when it's built (I've also already tested it myself and added a test case for it).


Write encrypted tags permission

I realized while doing some work on encrypted tags that, while we restrict which users are able to read encrypted tags, we do not restrict which users can write to them.
Depending on how you have built your automation around collins, this could have very bad consequences.
For example, if you automatically make server passwords match the password set in collins, using a configuration management tool that runs regularly like puppet or chef, an attacker that could not read the password could set it to a value they know, and then gain access or elevated permissions to your server.
This PR aims to fix this vulnerability by adding a canWriteEncryptedTags permission, that restricts which users are able to write encrypted tags.
It also renames canSeePasswords to canSeeEncryptedTags to better reflect what the permission does - you may have reasons to encrypt non-password attributes.  It does this in a backwards compatible manner, so your existing permissions configs that use canSeePasswords should continue to function.
@tumblr/collins RFR

Nice, LGTM. This should include some messaging in future release notes about backwards compatibility (or lack thereof) and provide a clear upgrade path.


Fixed bug in the URL generated by Share without code when the server is not at /

This PR is not for Extras. It fixes the generated URL to add the correct pathname for Share without code.

I don't think this will work correctly for https://code.world/haskell  You're likely to get a URL like https://code.world/haskellrun.html?...
However, it looks like just removing the slash should work.  I'll make that change instead.


create github workflow to build aarch64, i486 and armv7

Superseded by https://github.com/sailfishos-applications/daily-comics/commit/fc29c59e9ed6ac6f75168b044edf5d52096b2181


Implement document.body

This should return only the document body, as html.

Addressed with https://github.com/mojotech/dill/pull/126


Timeline events in the editor have poor readability when using a light Godot theme

The problem
Describe the bug
Many timeline events have poor readability in the editor when using a light Godot editor theme.
Punctuation is white, making it completely unreadable:

Here are all of events when viewed in light mode:

The most problematic cases involve the use of dark text with the dark backgrounds of the purple and dark green events:

Highlighted events have similarly low readability:

Buttons are darker than ideal in light mode, making them somewhat unreadable:

White icons are used in the light theme, leaving us with rather low contrast:

To Reproduce
Steps to reproduce the behavior:

Use a light Godot editor theme
View and select events in Dialogic's timeline editor

Expected behavior
n/a
Screenshots
As above
System (please complete the following information):

OS: Windows
Godot Version: 3.3.3
Dialogic Version: 1.2.5

Solutions
Workaround
n/a
Possible fixes
For the punctuation within the text event, a dark text colour should be consistently used.
For all of the other issues, there should ideally be a set of light background colours for use with light mode, and the current light icons should have dark variants. This would allow for dark text and icons to be consistently readable against the light backgrounds.
Alternatively, since the backgrounds appear to be partially transparent, they could all be changed to be completely opaque, and light text could be used against them. This would provide a consistent appearance with the existing readable dark mode.

Thanks for the great report. Only thing I can say is that the styling of selected events was already changed in 1.3Wip. They now only have a blue outline and their normal background.

Hey @Lunadyn I tried to tackle some of the described projects in the PR #502. I think the only thing missing are the darker icons. If you are satisfied with the rest of the changes, then I'd like to close this in favor of a new issue that's just about the icons. Let me know when you were able to take a look!

dark icons will be hard because we would most likely have to make them again... but if needed I can make them

yeah I know. That's why I left them out of the PR. Sadly we cannot just modulate the icons of the buttons

I know, we might need to submit that on a PR to the Godot Engine itself, so that we can modulate icons, but doubt it will be easy to get on there


feat: serialize ulids as bytes and compress layer db data

Shrinks the graph further, both by changing the Ulid serialization, and, at the expense of some extra compute, with compression, on the theory that the smaller it is, the faster we can move it through NATS.
The graph after imports is now 1,756,207 bytes vs 4,268,102 bytes on main.

bors r+

bors r+


Fortran Submodule Support Atom-IDE Outline

Dear Chris,
first of all thanks for developing and maintaining the language server and the atom ide for Fortran.
I'm frequently using the Fortran 2008 submodule construction in my code and am experiencing an issue with the outline feature of the atom ide which appears to be related to the language server. A frequently appearing part of my code looks like this
module test
implicit none
type test_type
  integer :: i
end type test_type

interface test_submodule
  module subroutine test_routine(L_input)
    integer,intent(inout) :: L_input
  end subroutine test_routine
end interface test_submodule

contains
  subroutine test_routine_2()
    ! some code
  end subroutine test_routine_2
end module test

The outline is just displaying the defined type correctly but neither the interface nor the subroutines defined in the main module. Instead the symbol for module appears. When I replace module subroutine in the interface with just subroutine everything is displayed correctly. I assume that the language server (or the ide) just detects the module statement instead of processing the whole module subroutine statement.
Thanks for looking into this!
Best,
Marco

Thanks for the report. Currently, the language server does not support submodules or some of the associated constructs (like "module subroutine" in interface statements). The issue you described with the outline should be easy enough to fix, which I will work on. However, I suspect that the interface won't be handled properly in autocompletion and other areas without improvements to handle submodules. If the code you are working with is open source I would be happy to take a look and work on improving practical submodule support as well.

I just pushed a change that should fix the outline portion of this bug. As I mentioned submodule support for other tasks will be future work, see issue #15. I will comment again on this issue once the fix has been included in a pip release.

I have published this and basic submodule support in release v0.3.6. Let me know if you run into further issues.

Outline and autocompletion work fine for me now. Thanks again for the fixes. There are two minor things which I would like to suggest for future enhancements. Both of them are mainly convenience issues.
a) Include Interfaces and procedures defined therein in the outline
b) Restrict autocompletion for variables of a certain derived type to the attributes of this type and show all possible attributes already when entering the % (I remember this worked quite well with the old autocomplete-fortran package)


Document falco syscall buffer size adjustment described in blog

What would you like to be added:
I have a Falco-related question I was wondering if anyone could answer in the documentation? Reading https://sysdig.com/blog/cve-2019-8339-falco-vulnerability/ there's a small paragraph that states:

”One way to reduce the risk of dropped system calls is by increasing the size of the shared buffer between user/kernel space. For example, increasing the size of the default kernel buffer from 8mb to 128mb (per cpu) resulted in no dropped system calls, even under the extreme workload used by the proof-of-concept program”

But I couldn't find a mention in the blog article or the Falco docs as to which kernel buffer is being described here. Does anyone know the details? I was assuming we can patch the kernel module that's built via dkms to increase the buffer, but a glance over the code didn't immediately make it obvious which buffer needed to be increased.
Why is this needed:
To reduce the number of dropped syscalls on busy/large nodes

@fntlnz RUNBPF

Thanks for this issue @dnwe !!
I think that you are referring to this configuration right?
https://github.com/falcosecurity/falco/blob/6e11e75c1522e99bbbad1967f7031538e1a9c0bf/falco.yaml#L84-L89
I think we already have some documentation for that here https://falco.org/docs/event-sources/dropped-events/
Do you see anything we could improve?
/remove-kind feature
/kind documentation
Send me an email lo at linux.com with your full name and address for the sticker! (I also accept encrypted emails if you have privacy concerns. You can get my public key here https://fntlnz.wtf/downloads/pubkey-0xD624DE73B2400EE4.asc)

@fntlnz: Those labels are not set on the issue: kind/feature

In response to this:

Thanks for this issue @dnwe !!
I think that you are referring to this configuration right?
https://github.com/falcosecurity/falco/blob/6e11e75c1522e99bbbad1967f7031538e1a9c0bf/falco.yaml#L84-L89
I think we already have some documentation for that here https://falco.org/docs/event-sources/dropped-events/
Do you see anything we could improve?
/remove-kind feature
/kind documentation
RUNBPF contest

Send me an email `lo at linux.com` with your full name and address for the sticker! (I also accept encrypted emails if you have privacy concerns. You can get my public key here https://fntlnz.wtf/downloads/pubkey-0xD624DE73B2400EE4.asc)

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@fntlnz that documents how we log when syscalls were dropped, but as per my quote from the linked blog article the original author mentioned that a user could also  increase the size of the shared buffer to prevent any syscalls from being dropped if it were sized large enough

@fntlnz: Those labels are not set on the issue: kind/feature

In response to this:

Thanks for this issue @dnwe !!
I think that you are referring to this configuration right?
https://github.com/falcosecurity/falco/blob/6e11e75c1522e99bbbad1967f7031538e1a9c0bf/falco.yaml#L84-L89
I see that we already have some documentation for that here https://falco.org/docs/event-sources/dropped-events/
Do you see anything we could improve?
/remove-kind feature
/kind documentation
RUNBPF contest

Send me an email `lo at linux.com` with your full name and address for the sticker! (I also accept encrypted emails if you have privacy concerns. You can get my public key here https://fntlnz.wtf/downloads/pubkey-0xD624DE73B2400EE4.asc)

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

I think this is the droid you are looking for: https://github.com/draios/sysdig/blob/dev/driver/ppm_ringbuffer.h#L17

Let me know if you want to pair if you get snagged compiling the driver - it took me a few tries to get everything dialed in correctly

@kris-nova perfect thanks, I'll give it a spin tomorrow and let you know

Yes @kris-nova it is!
@kris-nova and @dnwe can I propose to demo the compiling of the driver during our office hours in 2 weeks?
It will be recorded and it can be handy to our community!
Thus, in case do you agree please open an issue (with kind/debugging-hours) in the office-hours repository so we'll schedule it !

@leodido / @kris-nova I haven't quite got around to testing it yet, but I hoping I'd just be able to get away with patching the kernel module via the stable Dockerfile after the .deb has unpacked the source to /usr/src:
https://github.com/falcosecurity/falco/blob/6e11e75c1522e99bbbad1967f7031538e1a9c0bf/docker/stable/Dockerfile#L77-L87
With something like this:
diff --git a/Dockerfile b/Dockerfile
index d55ed24..1210e8d 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -44,6 +44,9 @@ RUN curl -s https://s3.amazonaws.com/download.draios.com/DRAIOS-GPG-KEY.public |
  && apt-get clean \
  && rm -rf /var/lib/apt/lists/*

+# Patch the ringbuffer in the falco kernel module to reduce dropped syscall events
+RUN sed -e '/RING_BUF_SIZE/s/8/64/' -i /usr/src/falco-*/ppm_ringbuffer.h
+
 # Change the falco config within the container to enable ISO 8601
 # output.
 RUN sed -e 's/time_format_iso_8601: false/time_format_iso_8601: true/' < /etc/falco/falco.yaml > /etc/falco/falco.yaml.new \

And let the falco-probe-loader / dkms build handle getting the compilation right for me 😎

Ah so that gives me:

Wed Sep  4 20:41:56 2019: Runtime error: error mapping the ring buffer for device /host/dev/falco0. Exiting.

Presumably because the /usr/bin/falco process needs to be (re-)built with a matching ringbuffer size

Ah so that gives me:

Wed Sep  4 20:41:56 2019: Runtime error: error mapping the ring buffer for device /host/dev/falco0. Exiting.

Presumably because the /usr/bin/falco userspace process needs to be (re-)built with a matching ringbuffer size
https://github.com/draios/sysdig/blob/ce8281b2d506114ef1ea89b904cda2baa6c1fa27/userspace/libscap/scap.c#L307-L310
and
https://github.com/draios/sysdig/blob/ce8281b2d506114ef1ea89b904cda2baa6c1fa27/userspace/libscap/scap.c#L351-L370

Yep I think thats what we need.
Just curious what is the use case here for expanding the ring buffer size? Is this in response to the kernel level components dropping syscall events?

Yes as per the first post, the linked article called it out as an option to reduce the amount of dropped syscalls. We already enabled logging and just wanted to test it out to see if we could reduce the occurrences.
I believe I got a build working, but hadn't had a chance to deploy it out yet. Will let you know

FYI we have been running for a while now with sed -e '/RING_BUF_SIZE/s/8/96/' -i /usr/src/sysdig/driver/ppm_ringbuffer.h which has reduced the number of dropped events, although we still see 1 or 2, but quite infrequently
{
  "output": "Falco internal: syscall event drop. 1 system calls dropped in last second.",
  "output_fields": {
    "ebpf_enabled": "0",
    "n_drops": "1",
    "n_drops_buffer": "1",
    "n_drops_bug": "0",
    "n_drops_pf": "0",
    "n_evts": "41188"
  },
  "priority": "Critical",
  "rule": "Falco internal: syscall event drop",
  "time": "2019-11-13T17:14:13.034211346Z"
}
{
  "output": "Falco internal: syscall event drop. 1 system calls dropped in last second.",
  "output_fields": {
    "ebpf_enabled": "0",
    "n_drops": "1",
    "n_drops_buffer": "1",
    "n_drops_bug": "0",
    "n_drops_pf": "0",
    "n_evts": "31529"
  },
  "priority": "Critical",
  "rule": "Falco internal: syscall event drop",
  "time": "2019-11-13T17:51:17.684344697Z"
}
{
  "output": "Falco internal: syscall event drop. 1 system calls dropped in last second.",
  "output_fields": {
    "ebpf_enabled": "0",
    "n_drops": "1",
    "n_drops_buffer": "1",
    "n_drops_bug": "0",
    "n_drops_pf": "0",
    "n_evts": "23159"
  },
  "priority": "Critical",
  "rule": "Falco internal: syscall event drop",
  "time": "2019-11-13T20:59:24.474584970Z"
}

From Repository Planning - Low hanging fruit: start by running in different environments with different kinds of Cpus and workloads and document suggested sizes for the ring buffer.

keeping fresh

What does here think about making the ring buffer size configurable via a flag ?
Also for everyone reading here, the ring buffer is not used for the eBPF implementation. This might also be useful information when comparing performances and to go to the root of the cause of drops.

We've never tried out the eBPF impl as the docs seemed like they suggested the kernel module was the recommended choice. Do we still get notified of drops with eBPF in the same way? I could give it a spin

@dnwe yes! drops go through the same process.
Just yesterday I made a PR documenting the eBPF installation process for multiple kinds of installations https://github.com/falcosecurity/falco-website/pull/134
Please be aware you might still have SYSDIG_BPF_PROBE if you are on < 0.18.0 - everything should be consistently set to FALCO_BPF_PROBE for 0.20.0

Is there a way to debug the dropped events? On a small VPS, we see e.g. this:
Apr  5 15:20:02 host falco: Falco internal: syscall event drop. 1 system calls dropped in last second.
Apr  5 15:20:02 host falco: 15:20:02.335890806: Critical Falco internal: syscall event drop. 1 system calls dropped in last second. (ebpf_enabled=0 n_drops=1 n_drops_buffer=1 n_drops_bug=0 n_drops_pf=0 n_evts=12297)
Apr  5 15:25:02 host falco: Falco internal: syscall event drop. 2 system calls dropped in last second.
Apr  5 15:25:02 host falco: 15:25:02.429912553: Critical Falco internal: syscall event drop. 2 system calls dropped in last second. (ebpf_enabled=0 n_drops=2 n_drops_buffer=2 n_drops_bug=0 n_drops_pf=0 n_evts=12532)
Apr  5 15:45:02 host falco: Falco internal: syscall event drop. 1 system calls dropped in last second.
Apr  5 15:45:02 host falco: 15:45:02.858715115: Critical Falco internal: syscall event drop. 1 system calls dropped in last second. (ebpf_enabled=0 n_drops=1 n_drops_buffer=1 n_drops_bug=0 n_drops_pf=0 n_evts=15544)

The n_evts counter looks like a small event count between the log messages. Any idea what happens here?

What does here think about making the ring buffer size configurable via a flag ?

@fntlnz did anything happen with this in the end? Currently we're still just maintaining our patch and building falco from source, but it'd be cool if we could just use the official docker image and provide the ringbuffer size as a parameter for the userspace app and as a module parameter for the kernel module

@dnwe Is it an option to increase the frequency of reading the buffer as to avoid having it full in the first place ? Just wondering because we are having this issue as well and we would really like to avoid having to recompile everything and spin our own version of the container!

Is it expected behavior to have dropped syscal in the log ? We consistently have 1 to 4 syscall dropped every now and then. Should we ignore there message if it's a low amount of dropped syscall ? I wonder what kind of log someone exploiting cve-2019-8339 would generate. If it would generate hundreds of syscall drop, maybe it's an OK solution to just ignore low syscall drop log message.

Are there any plans to port this workaround into the helm chart so that we can pass it as a parameter during install/upgrade?

@leodido @fntlnz @kris-nova Hey guys, if you are interested in the patch, I could bring it back to falco into the cmake patching folder (falco/cmake/modules/sysdig-repo/patch/).
Let me know what's up.

@nvanheuverzwijn +1

@emcay With our patch, you can pass this environment variable : FALCO_DRIVER_LOADER_ARGS: "--compile --module-arg ring_buf_size=134217728"
You can use our docker image ghcr.io/kronostechnologies/falco:0.24.1-18 (18 commit since patch 0.24.1) however, it is not up to date with the latest as of yet but works for sure.

@nvanheuverzwijn This should be sufficient for testing, thank you! Anyway I could take a look at the Dockerfile? Due the nature of our project I cannot blindly run software on our infra. :)

@jannis-a The fork is here https://github.com/kronostechnologies/falco

@jannis-a The fork is here https://github.com/kronostechnologies/falco

Just out of curiosity, have you noticed any improvements? If yes, could you provide a comparison?
Thanks in advance!

@nvanheuverzwijn can you port your feature upstream as soon as Sysdig merges your PR for the probe?

@danmx All of the PR are open on against every repository for falco and sysdig. I can't do much more than that, it's in the hand of the falco team.
@leogr Our initial goal was to reduce the amount of alerting due to dropped syscall. We could revert back to the old buffer size and compare the result. What would you like to see ? Are you interested in CPU/RAM performance ?

@leogr There's a major decrease in syscall drops, but the CPU usage is a little higher due to the larger buffer / less missed calls

Hey @nvanheuverzwijn and @antoinedeschenes
First of all, thanks again for your amazing work. Indeed we already started to review those PRs, though it will take a bit since we have to test it deeply.
What I'm interested in is:

% of drops decrease
CPU/RAM performance
We suspected that by increasing the ring size, the number of events processed per second decreases (possibly because it would slow a bit the whole system that could reduce the events producer performance too). See https://github.com/falcosecurity/falco/issues/669#issuecomment-635476220 for more details. Now, I'm not sure if there's a reliable way to measure this. If you can measure that too somehow it would be great ;)

PS feel free to contact me on Slack if you need any help!

I'd also be interested in performance information 😃 I hope you'll make it public

Sure, we could probably make another custom build (required to have dynamic auditing working) without the sysdig patches, easily switch between both and compare

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/falcosecurity/community.
/lifecycle stale

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/falcosecurity/community.
/lifecycle rotten

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/falcosecurity/community.
/lifecycle rotten

/remove-lifecycle rotten

/remove-lifecycle rotten

To try to answer my question of what to patch for the eBPF probe, I'm guessing it's this constant here:
https://github.com/falcosecurity/libs/blob/0acbdb729224fc626f994f5ad836009a6a271479/userspace/libscap/scap_bpf.c#L61
Would someone be able to confirm if this is correct?
Compiling falco with this patched to a larger value, it seems to have helped us reduce this from frequent 10000s calls  dropped calls to occassional 1s (but it may very well be a coincidence).

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/falcosecurity/community.
/lifecycle stale

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/falcosecurity/community.
/lifecycle rotten

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.
Provide feedback via https://github.com/falcosecurity/community.
/close

@poiana: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.
Provide feedback via https://github.com/falcosecurity/community.
/close

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

I think this is the droid you are looking for: https://github.com/draios/sysdig/blob/dev/driver/ppm_ringbuffer.h#L17

The link seems to be dead.  Does the droid still exist or did the Jawas take it away? I am running falco on a rather large and complex cluster.  We are receiving thousands of dropped syscall a day.

Ei @kolbeface this is the config we use in Falco to change the buffer size (The description is pretty detailed):
https://github.com/falcosecurity/falco/blob/1b62b5ccd1c64cd972ef0252262075cbf42a130c/falco.yaml#L807

We are receiving thousands of dropped syscall a day.

Do you mind to enable some metrics :point_down:
https://github.com/falcosecurity/falco/blob/1b62b5ccd1c64cd972ef0252262075cbf42a130c/falco.yaml#L663
to let us understand why you are facing so many drops.
Some additional questions

Which driver are you using? kmod, bpf, modern-bpf?
How many rules do you use approximatively?
Are you capturing just syscall or you are using some plugins?
Do you have the k8s metadata enrichment enabled (-k, -K)?

@Andreagit97 thanks for the reply! Forgive me, I am very new to all of this and feel a bit out of my comfort zone with this.  I inherited this project from a teammate.  I do not believe we are using a driver
        image:           registry: docker.internal-mycompany.com/docker-hub           repository: falcosecurity/falco-no-driver
I am going to experiment with expanding the buffer today.

I am going to experiment with expanding the buffer today.  If I enable the metrics rule will that create a metrics log and store it in the same place we are storing our other falco logs (S3)?

Yes :)

uhm wow! thank you for sharing!

@iohenkies: You can't reopen an issue/PR unless you authored it or you are a collaborator.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

/reopen

Do you mind to enable some metrics :point_down:
https://github.com/falcosecurity/falco/blob/1b62b5ccd1c64cd972ef0252262075cbf42a130c/falco.yaml#L663
to let us understand why you are facing so many drops.
Some additional questions

Which driver are you using? kmod, bpf, modern-bpf?
How many rules do you use approximatively?
Are you capturing just syscall or you are using some plugins?
Which Falco version are you using?

Back at work I logged in the Falcosidekick UI, I've noticed that the drops might be a bit less the last 12 hours, but as soon as I login and click around a bit:
Falco internal: syscall event drop. 852678 system calls dropped in last second

The Falco version is indeed 0.37.1.
The barely used cluster now has these settings:
driver:
  modernEbpf:
    leastPrivileged: false
    bufSizePreset: 6
    dropFailedExit: false
    cpusForEachBuffer: 2

The busy cluster has these settings:
driver:
  modernEbpf:
    leastPrivileged: false
    bufSizePreset: 8
    dropFailedExit: false
    cpusForEachBuffer: 4

These increases the memory footprint of the daemon pods significantly. Both clusters are experiencing the drops. The busy clusters seem to have more occurrences though.
What feels strange for me, is that one specific pod seems to reporting these drops over and over. Would that mean that only that node on which that pod is running on, is experiencing these problems?
I have just enabled metrics. Anything else I have to do in this regard?

ei @iohenkies thank you for the precious data! let's try to focus on just one cluster for now (the barely used one) so we can try to understand what is going on... Considering this barely used cluster:

How many nodes do you have?
Are you experiencing drops on all nodes?
How many CPUs do you have on the node that is dropping?
Can you check which are the syscalls really loaded by Falco? You need to add -o "log_level=debug" and Falco should print the list of syscalls at startup time:

Thu Feb 29 10:20:22 2024: (27) syscalls selected in total (final set): bind, clone, clone3, close, connect, dup, dup2, dup3, execve, execveat, finit_module, fork, getsockopt, init_module, link, linkat, open, openat, openat2, procexit, ptrace, sendmsg, sendto, socket, symlink, symlinkat, vfork
Thu Feb 29 10:20:22 2024: The chosen syscall buffer dimension is: 8388608 bytes (8 MBs)

Can you enable some metrics https://github.com/falcosecurity/falco/blob/1b62b5ccd1c64cd972ef0252262075cbf42a130c/falco.yaml#L790? so we can see which syscalls are dropping

Hi Andrea. Below data is from the barely used cluster.

The cluster is 12 nodes, of which 6 relevant worker nodes. Falco pods are on those
Looking at the last 24 hours I've got 10 debug entries, coming from 2 pods and therefore 2 nodes
All worker nodes have 8 CPU and 32 GB RAM
Enabled debug

falcoctl-artifact-install {"level":"INFO","msg":"Resolving dependencies ...","timestamp":"2024-02-29 09:39:06"}
falcoctl-artifact-install {"level":"INFO","msg":"Installing artifacts","refs":["ghcr.io/falcosecurity/rules/falco-rules:3"],"timestamp":"2024-02-29 09:39:07"}
falcoctl-artifact-install {"level":"INFO","msg":"Preparing to pull artifact","ref":"ghcr.io/falcosecurity/rules/falco-rules:3","timestamp":"2024-02-29 09:39:07"}
falcoctl-artifact-install {"level":"INFO","msg":"Pulling layer 1e72f9c4d8fe","timestamp":"2024-02-29 09:39:08"}
falcoctl-artifact-install {"level":"INFO","msg":"Pulling layer 2e91799fee49","timestamp":"2024-02-29 09:39:08"}
falcoctl-artifact-install {"level":"INFO","msg":"Pulling layer d4c03e000273","timestamp":"2024-02-29 09:39:08"}
falcoctl-artifact-install {"digest":"ghcr.io/falcosecurity/rules/falco-rules@sha256:d4c03e000273a0168ee3d9b3dfb2174e667b93c9bfedf399b298ed70f37d623b","level":"INFO","msg":"Verifying signature for artifact","timestamp":"2024-02-29 09:39:08"}
falcoctl-artifact-install {"level":"INFO","msg":"Signature successfully verified!","timestamp":"2024-02-29 09:39:10"}
falcoctl-artifact-install {"file":"falco_rules.yaml.tar.gz","level":"INFO","msg":"Extracting and installing artifact","timestamp":"2024-02-29 09:39:10","type":"rulesfile"}
falcoctl-artifact-install {"digest":"sha256:d4c03e000273a0168ee3d9b3dfb2174e667b93c9bfedf399b298ed70f37d623b","directory":"/rulesfiles","level":"INFO","msg":"Artifact successfully installed","name":"ghcr.io/falcosecurity/rules/falco-rules:3","timestamp":"2024-02-29 09:39:10","type":"rulesfile"}
Stream closed EOF for ams-system-falco/falco-pknrh (falcoctl-artifact-install)
falco Thu Feb 29 09:39:11 2024: Falco version: 0.37.1 (x86_64)
falco Thu Feb 29 09:39:11 2024: CLI args: /usr/bin/falco --cri /run/containerd/containerd.sock -pk
falco Thu Feb 29 09:39:11 2024: Falco initialized with configuration file: /etc/falco/falco.yaml
falco Thu Feb 29 09:39:11 2024: System info: Linux version 5.15.0-94-generic (buildd@lcy02-amd64-096) (gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024
falco Thu Feb 29 09:39:11 2024: Configured rules filenames:
falco Thu Feb 29 09:39:11 2024:    /etc/falco/rules.d
falco Thu Feb 29 09:39:11 2024: Loading rules from file /etc/falco/rules.d/rules-outbound.yaml
falco Thu Feb 29 09:39:11 2024: Watching file '/etc/falco/falco.yaml'
falco Thu Feb 29 09:39:11 2024: Watching file '/etc/falco/rules.d/rules-outbound.yaml'
falco Thu Feb 29 09:39:11 2024: Watching directory '/etc/falco/rules.d'
falco Thu Feb 29 09:39:11 2024: (3) syscalls in rules: connect, sendmsg, sendto
falco Thu Feb 29 09:39:11 2024: +(21) syscalls (Falco's state engine set of syscalls): capset, chdir, chroot, clone, clone3, close, execve, execveat, fchdir, fork, getsockopt, prctl, procexit, setgid, setpgid, setresgid, setresuid, setsid, setuid, socket, vfork
falco Thu Feb 29 09:39:11 2024: (24) syscalls selected in total (final set): capset, chdir, chroot, clone, clone3, close, connect, execve, execveat, fchdir, fork, getsockopt, prctl, procexit, sendmsg, sendto, setgid, setpgid, setresgid, setresuid, setsid, setuid, socket, vfork
falco Thu Feb 29 09:39:11 2024: The chosen syscall buffer dimension is: 33554432 bytes (32 MBs)
falco Thu Feb 29 09:39:11 2024: Starting health webserver with threadiness 8, listening on <IP_ADDRESS>:8765
falco Thu Feb 29 09:39:11 2024: Setting metrics interval to 1h, equivalent to 3600000 (ms)
falco Thu Feb 29 09:39:11 2024: Loaded event sources: syscall
falco Thu Feb 29 09:39:11 2024: Enabled event sources: syscall
falco Thu Feb 29 09:39:11 2024: Opening event source 'syscall'
falco Thu Feb 29 09:39:11 2024: Opening 'syscall' source with modern BPF probe.
falco Thu Feb 29 09:39:11 2024: One ring buffer every '2' CPUs.
falcoctl-artifact-follow {"artifact":"falco-rules:3","check every":"6h0m0s","level":"INFO","msg":"Creating follower","timestamp":"2024-02-29 09:39:11"}
falcoctl-artifact-follow {"artifact":"ghcr.io/falcosecurity/rules/falco-rules:3","level":"INFO","msg":"Starting follower","timestamp":"2024-02-29 09:39:11"}
falcoctl-artifact-follow {"followerName":"ghcr.io/falcosecurity/rules/falco-rules:3","level":"INFO","msg":"Found new artifact version","tag":"3","timestamp":"2024-02-29 09:39:12"}
falcoctl-artifact-follow {"artifactName":"ghcr.io/falcosecurity/rules/falco-rules:3","digest":"sha256:d4c03e000273a0168ee3d9b3dfb2174e667b93c9bfedf399b298ed70f37d623b","directory":"/rulesfiles","followerName":"ghcr.io/falcosecurity/rules/falco-rules:3","level":"INFO","msg":"Artifact correctly installed","timestamp":"2024-02-29 09:39:18","type":"rulesfile"}

Metrics are enabled, and I see a lot of snapshots are being made. How can I see the contents and ship them to you?

thank you!

All worker nodes have 8 CPU and 32 GB RAM

uhm so nodes are pretty small...

Metrics are enabled, and I see a lot of snapshots are being made. How can I see the contents and ship them to you?

you can create a gist https://docs.github.com/en/get-started/writing-on-github/editing-and-sharing-content-with-gists/creating-gists#creating-a-gist or attach a log file here, as you prefer

Yes, are these considered small for eBPF purposes?
Where can I find the metrics/logs though?
These are my settings, which are default, I only had to set it top true.
  metrics:
    enabled: true
    interval: 1h
    output_rule: true
    resource_utilization_enabled: true
    state_counters_enabled: true
    kernel_event_counters_enabled: true
    libbpf_stats_enabled: true
    convert_memory_to_mb: true
    include_empty_values: false

Yes, are these considered small for eBPF purposes?

Usually, we see drops on huge servers with many CPUs, like 64 or more

Where can I find the metrics/logs though?

 That is below.
{"uuid":"8c63fb8a-8a72-4f2d-a6a3-282a1484cb7c","output":"Falco metrics snapshot","priority":"Informational","rule":"Falco internal: metrics snapshot","time":"2024-03-01T08:52:34.832049033Z","source":"internal","output_fields":{"evt.source":"syscall","evt.time":1709283154832049200,"falco.cpu_usage_perc":0.7,"falco.cpu_usage_perc_total_host":8,"falco.duration_sec":79187,"falco.evts_rate_sec":2440.5,"falco.host_boot_ts":1707826372000000000,"falco.host_num_cpus":8,"falco.hostname":"falco-ctj9r","falco.kernel_release":"5.15.0-89-generic","falco.memory_pss":164,"falco.memory_rss":291,"falco.memory_used_host":3456,"falco.memory_vsz":1260,"falco.n_added_fds":4725267,"falco.n_added_threads":1275890,"falco.n_cached_fd_lookups":37505764,"falco.n_cached_thread_lookups":328899104,"falco.n_containers":8,"falco.n_failed_fd_lookups":321784597,"falco.n_failed_thread_lookups":2639213,"falco.n_fds":834,"falco.n_noncached_fd_lookups":12162611,"falco.n_noncached_thread_lookups":107208869,"falco.n_removed_fds":4632341,"falco.n_removed_threads":1264368,"falco.n_retrieve_evts_drops":635199,"falco.n_retrieved_evts":16645021,"falco.n_stored_evts":21100939,"falco.n_threads":742,"falco.num_evts":423315966,"falco.num_evts_prev":414530125,"falco.open_fds_host":5728,"falco.outputs_queue_num_drops":0,"falco.procs_running_host":1,"falco.start_ts":1709203966889463300,"falco.version":"0.37.1","scap.engine_name":"modern_bpf","scap.evts_drop_rate_sec":0,"scap.evts_rate_sec":2443.4,"scap.n_drops":260938090,"scap.n_drops_buffer_clone_fork_enter":2258,"scap.n_drops_buffer_clone_fork_exit":5011,"scap.n_drops_buffer_close_exit":131992990,"scap.n_drops_buffer_connect_enter":5336,"scap.n_drops_buffer_connect_exit":5406,"scap.n_drops_buffer_execve_enter":1716,"scap.n_drops_buffer_execve_exit":1803,"scap.n_drops_buffer_other_interest_enter":77,"scap.n_drops_buffer_other_interest_exit":82,"scap.n_drops_buffer_proc_exit":2466,"scap.n_drops_buffer_total":260938090,"scap.n_drops_perc":0,"scap.n_drops_prev":260938090,"scap.n_evts":684486142,"scap.n_evts_prev":675690047},"hostname":"falco-ctj9r","tags":null}

FWIW the below is from the busier cluster. I have 73 entries in the last hour...
{"uuid":"c7577ee9-3b61-4ca4-8db7-90028e9b492c","output":"Falco metrics snapshot","priority":"Informational","rule":"Falco internal: metrics snapshot","time":"2024-03-01T09:59:16.051883398Z","source":"internal","output_fields":{"evt.source":"syscall","evt.time":1709287156051883500,"falco.cpu_usage_perc":2.3,"falco.cpu_usage_perc_total_host":16.9,"falco.duration_sec":97188,"falco.evts_rate_sec":10293.3,"falco.host_boot_ts":1708975789000000000,"falco.host_num_cpus":8,"falco.hostname":"falco-cw5sd","falco.kernel_release":"5.15.0-94-generic","falco.memory_pss":339,"falco.memory_rss":594,"falco.memory_used_host":11212,"falco.memory_vsz":1563,"falco.n_added_fds":6865713,"falco.n_added_threads":4049849,"falco.n_cached_fd_lookups":275945062,"falco.n_cached_thread_lookups":794689053,"falco.n_containers":25,"falco.n_failed_fd_lookups":783537982,"falco.n_failed_thread_lookups":9699032,"falco.n_fds":3524,"falco.n_noncached_fd_lookups":16585029,"falco.n_noncached_thread_lookups":104767229,"falco.n_removed_fds":6730309,"falco.n_removed_threads":4165996,"falco.n_retrieve_evts_drops":1288303,"falco.n_retrieved_evts":132987187,"falco.n_stored_evts":136518908,"falco.n_threads":4928,"falco.num_evts":868931020,"falco.num_evts_prev":831876231,"falco.open_fds_host":8352,"falco.outputs_queue_num_drops":0,"falco.procs_running_host":10,"falco.start_ts":1709189968024046800,"falco.version":"0.37.1","scap.engine_name":"modern_bpf","scap.evts_drop_rate_sec":0,"scap.evts_rate_sec":10273.4,"scap.n_drops":3,"scap.n_drops_perc":0,"scap.n_drops_prev":3,"scap.n_drops_scratch_map":3,"scap.n_evts":866913417,"scap.n_evts_prev":829930107},"hostname":"falco-cw5sd","tags":null}

Ok, thank you! I can see from the metrics you sent about the barely used cluster, that you probably have traffic spikes that Falco cannot manage and so you experienced a high number of drops in this period...
{
	"uuid": "8c63fb8a-8a72-4f2d-a6a3-282a1484cb7c",
	"output": "Falco metrics snapshot",
	"priority": "Informational",
	"rule": "Falco internal: metrics snapshot",
	"time": "2024-03-01T08:52:34.832049033Z",
	"source": "internal",
	"output_fields": {
		"evt.source": "syscall",
		"evt.time":<PHONE_NUMBER>832049200,
		"falco.cpu_usage_perc": 0.7,
		"falco.cpu_usage_perc_total_host": 8,
		"falco.duration_sec": 79187,
		"falco.evts_rate_sec": 2440.5,
		"falco.host_boot_ts":<PHONE_NUMBER>000000000,
		"falco.host_num_cpus": 8,
		"falco.hostname": "falco-ctj9r",
		"falco.kernel_release": "5.15.0-89-generic",
		"falco.memory_pss": 164,
		"falco.memory_rss": 291,
		"falco.memory_used_host": 3456,
		"falco.memory_vsz": 1260,
		"falco.n_added_fds": 4725267,
		"falco.n_added_threads": 1275890,
		"falco.n_cached_fd_lookups": 37505764,
		"falco.n_cached_thread_lookups": 328899104,
		"falco.n_containers": 8,
		"falco.n_failed_fd_lookups": 321784597,
		"falco.n_failed_thread_lookups": 2639213,
		"falco.n_fds": 834,
		"falco.n_noncached_fd_lookups": 12162611,
		"falco.n_noncached_thread_lookups": 107208869,
		"falco.n_removed_fds": 4632341,
		"falco.n_removed_threads": 1264368,
		"falco.n_retrieve_evts_drops": 635199,
		"falco.n_retrieved_evts": 16645021,
		"falco.n_stored_evts": 21100939,
		"falco.n_threads": 742,
		"falco.num_evts": 423315966,
		"falco.num_evts_prev": 414530125,
		"falco.open_fds_host": 5728,
		"falco.outputs_queue_num_drops": 0,
		"falco.procs_running_host": 1,
		"falco.start_ts":<PHONE_NUMBER>889463300,
		"falco.version": "0.37.1",
		"scap.engine_name": "modern_bpf",
		"scap.evts_drop_rate_sec": 0,
		"scap.evts_rate_sec": 2443.4,
		"scap.n_drops": 260938090,
		"scap.n_drops_buffer_clone_fork_enter": 2258,
		"scap.n_drops_buffer_clone_fork_exit": 5011,
		"scap.n_drops_buffer_close_exit": 131992990,
		"scap.n_drops_buffer_connect_enter": 5336,
		"scap.n_drops_buffer_connect_exit": 5406,
		"scap.n_drops_buffer_execve_enter": 1716,
		"scap.n_drops_buffer_execve_exit": 1803,
		"scap.n_drops_buffer_other_interest_enter": 77,
		"scap.n_drops_buffer_other_interest_exit": 82,
		"scap.n_drops_buffer_proc_exit": 2466,
		"scap.n_drops_buffer_total": 260938090,
		"scap.n_drops_perc": 0,
		"scap.n_drops_prev": 260938090,
		"scap.n_evts": 684486142,
		"scap.n_evts_prev": 675690047
	},
	"hostname": "falco-ctj9r",
	"tags": null
}

falco.evts_rate_sec": 2440.5 the event rate per sec in the last hour is 2440.5, so if we extend this average metric to the whole Falco lifetime ("falco.duration_sec": 79187) we would have a total number of events equal to 2440.5*79187=193255873,5. But as you can see the total number of events is more than 3 times the one we previously computed ("scap.n_evts": 684486142).
Falco can handle throughputs of 2440.5 evt/s in fact with this throughput we have no drops "scap.evts_drop_rate_sec": 0 in the last hour. The issue is that at some point in time during the execution this throughput drastically increases and Falco cannot handle it

Unfortunately, I don't think at the moment we have a way to mitigate this, sorry... we are working on possible solutions to address these kinds of issues with a sort of kernel filtering or reducing the dependencies between syscalls....for example here I see that you need just 3 syscalls in your rules [connect, sendmsg, sendto] while almost all the drops in the stats are caused by the close syscall that you don't even need (but unfortunately you cannot disable it due to how Falco works under the hood :/) @leogr @LucaGuerra
falco Thu Feb 29 09:39:11 2024: (3) syscalls in rules: connect, sendmsg, sendto
falco Thu Feb 29 09:39:11 2024: +(21) syscalls (Falco's state engine set of syscalls): capset, chdir, chroot, clone, clone3, close, execve, execveat, fchdir, fork, getsockopt, prctl, procexit, setgid, setpgid, setresgid, setresuid, setsid, setuid, socket, vfork
falco Thu Feb 29 09:39:11 2024: (24) syscalls selected in total (final set): capset, chdir, chroot, clone, clone3, close, connect, execve, execveat, fchdir, fork, getsockopt, prctl, procexit, sendmsg, sendto, setgid, setpgid, setresgid, setresuid, setsid, setuid, socket, vfork
falco Thu Feb 29 09:39:11 2024: The chosen syscall buffer dimension is: 33554432 bytes (32 MBs)

Hey Andrea, many thanks for your elaborate answer. I hope I can bother you a last time.
It's unfortunate that this can't be changed at this time. Let me try to understand this properly.

Do these drops indeed mean that certain data is being lost? Or shouldn't I need to worry too much about it?
Does playing around with the below values do anything that might help (I don't think so actually, and the memory footprint per pod increases significantly)?

driver:
  modernEbpf:
    leastPrivileged: false
    bufSizePreset: 8
    dropFailedExit: false
    cpusForEachBuffer: 4

Seeing my data, which settings would you recommend here?

falco:
  base_syscalls:
    custom_set: [clone, clone3, fork, vfork, execve, execveat, close, socket, bind, getsockopt]
    repair: true

Should Falco be run as a Daemonset or can it function just as properly as a Deployment with less pods?

Do these drops indeed mean that certain data is being lost? Or shouldn't I need to worry too much about it?

Yeah, it means that is possible that some rules are not triggered because Falco has lost some events. It seems that events are lost only during these spikes so during the normal capture rules should trigger normally but yes we cannot know it a priori.

Does playing around with the below values do anything that might help (I don't think so actually, and the memory footprint per pod increases significantly)?

You can try something like this:
driver:
  modernEbpf:
    leastPrivileged: false
    bufSizePreset: 8
    dropFailedExit: true #this will drop failed syscalls, and maybe will remove some `close` syscalls
    cpusForEachBuffer: 2 #this will increase the memory footprint

Not sure it will really help but you can try...

Seeing my data, which settings would you recommend here?

Yes this is probably the best you can do...
falco:
  base_syscalls:
    custom_set: [clone, clone3, fork, vfork, execve, execveat, close, socket, bind, getsockopt]
    repair: true

Should Falco be run as a Daemonset or can it function just as properly as a Deployment with less pods?

Falco can be also deployed on a single node, there are no strict requirements on how to deploy it, and you are not forced to use a Daemonset.
Hope this can help :)

Many thanks Andrea, it really is greatly appreciated. I’ll pick this one up Monday, and will report back with hopefully something positive.
Only one followup question still about the Deployment instead of Daemonset: when you want to capture everything on all nodes, with eBPF and Falco, is a Daemonset then not mandatory? I.e. I’m guessing a deployment with 5 pods can’t capture everything on a 100 node cluster? It only captures events on nodes where a Falco pod is?

re @Andreagit97 comment

for example here I see that you need just 3 syscalls in your rules [connect, sendmsg, sendto] while almost all the drops in the stats are caused by the close syscall that you don't even need (but unfortunately you cannot disable it due to how Falco works under the hood :/)

Actually you have complete override control with base_syscalls when used without the repair option. The only thing you can't opt out is the procexit events from the proc scheduler exit tracepoint. https://falco.org/docs/troubleshooting/dropping/

Only one followup question still about the Deployment instead of Daemonset: when you want to capture everything on all nodes, with eBPF and Falco, is a Daemonset then not mandatory? I.e. I’m guessing a deployment with 5 pods can’t capture everything on a 100 node cluster? It only captures events on nodes where a Falco pod is?

Yes, Falco needs to run once per host / kernel instance, therefore fair to say a daemonset sort of deployment concept is mandatory, but you could of course directly install Falco once per host without using a daemonset setup . https://falco.org/docs/install-operate/deployment/

@incertum that sounds great. Would this be the droids I'm looking for (without the close)?
base_syscalls:
  custom_set: [clone, clone3, fork, vfork, execve, execveat, getsockopt, socket, connect]
  repair: false

And maybe something like this:
driver:
  modernEbpf:
    leastPrivileged: false
    bufSizePreset: 8
    dropFailedExit: true
    cpusForEachBuffer: 4

@iohenkies I am not fully caught up on all details, but I would just go for it and experiment similar to the suggestions posted here https://falco.org/docs/troubleshooting/dropping/ including the command to cross-reference the list of all syscalls that are activated based on your current configuration. In addition, I would closely monitor CPU and memory utilization https://falco.org/docs/metrics/ as you run the experiments and lastly remember base_syscalls is "use at own risk".
Finally, while we all want "perfect", I have experienced that it's beneficial to determine Service Level Agreements (SLAs) or Service Level objectives (SLOs) for your organization that should include the overall failure tolerances. I believe all time drop percentages of below 1% on very busy servers is not the worst state.

@incertum Agreed and understood.
The reason I've ended up in this thread though, is that with a pretty basic setup, on barely used cluster and nodes (each 8 CPU and 32GB RAM), syscall drops were significantly. That felt strange to be, with such a simple install.
I'll experiment some more with these settings. Thanks for all the time and effort.

@incertum that sounds great. Would this be the droids I'm looking for (without the close)?

I would discourage this. Removing the close syscall means that no one will clean the file descriptor table inside Falco, this would lead to memory leakage since you add file descriptors with socket but you don't free them with close...

dropFailedExit: true

Should allow you to save some space in the buffers because it doesn't send to userspace failed syscalls, which usually don't bring a big security value

@Andreagit97 as soon as I add the close, the syscalls start dropping (on the cluster that isn't busy at all).
I'll add dropFailedExit: true on the clusters.

@Andreagit97 as soon as I add the close, the syscalls start dropping (on the cluster that isn't busy at all).

yep, I understand it, but if you remove it, Falco won't free the memory during the execution, so it's risky to run under these conditions, Falco could crash at a certain point...
Just to be super explicit this is the code
void sinsp_parser::parse_close_exit(sinsp_evt *evt)
{
	int64_t retval;

	//
	// Extract the return value
	//
	retval = evt->get_param(0)->as<int64_t>();

	//
	// If the close() was successful, do the cleanup
	//
	if(retval >= 0)
	{
		if(evt->get_fd_info() == nullptr || evt->get_tinfo() == nullptr)
		{
			return;
		}

		//
		// a close gets canceled when the same fd is created successfully between
		// close enter and close exit.
		//
		erase_fd_params eparams;

		if(evt->get_fd_info()->m_flags & sinsp_fdinfo::FLAGS_CLOSE_CANCELED)
		{
			evt->get_fd_info()->m_flags &= ~sinsp_fdinfo::FLAGS_CLOSE_CANCELED;
			eparams.m_fd = CANCELED_FD_NUMBER;
			eparams.m_fdinfo = evt->get_tinfo()->get_fd(CANCELED_FD_NUMBER);
		}
		else
		{
			eparams.m_fd = evt->get_tinfo()->m_lastevent_fd;
			eparams.m_fdinfo = evt->get_fd_info();
		}

		//
		// Remove the fd from the different tables
		//
		eparams.m_remove_from_table = true;
		eparams.m_tinfo = evt->get_tinfo();
		eparams.m_ts = evt->get_ts();

		erase_fd(&eparams); // here we erase the fd from the table
	}

Thanks for the clarification Andrea. I've added the close. As said, unfortunately, the Falco internal: syscall event drop. 1923017 appear regularly. We'll have to test and see if this is a workable solution for us.

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.
Provide feedback via https://github.com/falcosecurity/community.
/close

@poiana: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.
Provide feedback via https://github.com/falcosecurity/community.
/close

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.


[Formula1Bridge] New bridge

Hello,
Feed on the official Formula 1 website stopped working a few days ago, which created a perfect opportunity to fix it with a new bridge. There are two things about it worth mentioning:

I am not yet sure whether the API key changes over time or stays the same - for now I have hardcoded it and will keep monitoring the website
LIMIT_MAX is set to 100 even though the API theoretically allows up to 1000, the reason is that responses longer than 7340032 bytes are not returned and instead an error message is generated

After 10 days the API key is still the same and therefore the bridge still works correctly.

gj @AxorPL !
Btw, I have just payed attention, that you are using getContents() or returnServerError() combination. I suggest not to use returnServerError as it is useless.  Discussion: https://github.com/RSS-Bridge/rss-bridge/pull/2082#pullrequestreview-644965225

Thanks, will remove it in the next commit.


Sql Server Dependencies

I forked the repo and brought it local.  When I opened it using VS2015 Ultimate, I get the following error message

This is a common dependency for building C++ projects with Visual Studio.

I've not been able to repro this. Can you delete the file and reopen the project? VS will rebuild it and you should be set.
Thanks,
Jesse

Poster has not responded with more detail in over 1 year - closing as stale.


Fix webhook commits wrong hash on HEAD reset

Use .. instead of ... with rev-list. In combination with #16282 the receiver can get the correct commit. The behaviour is now like Github.
fixes #11802

It's always hard to decide which of ".." and "..." is required.
Let's take a look at the caller hierarchy:

CommitsBetween

CommitsBeforeUntil

getCommitIDsFromRepo

CreatePushPullComment

SignMerge
pushUpdates

CommitsBetweenIDs

SyncPullMirror

I think you're right that ".." is correct for these.

CI fail is related

Codecov Report

Merging #16283 (98063b9) into main (5c80ecc) will increase coverage by 0.01%.
The diff coverage is 66.66%.

@@            Coverage Diff             @@
##             main   #16283      +/-   ##
==========================================
+ Coverage   45.40%   45.41%   +0.01%
==========================================
  Files         709      709
  Lines       83480    83483       +3
==========================================
+ Hits        37900    37917      +17
+ Misses      39500    39485      -15
- Partials     6080     6081       +1

Impacted Files
Coverage Δ

modules/git/repo_commit.go
42.60% <66.66%> (+1.73%)
:arrow_up:

services/pull/pull.go
41.99% <0.00%> (ø)

modules/ssh/ssh.go
47.02% <0.00%> (+0.87%)
:arrow_up:

modules/queue/workerpool.go
55.72% <0.00%> (+2.67%)
:arrow_up:

modules/util/timer.go
85.71% <0.00%> (+42.85%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5c80ecc...98063b9. Read the comment docs.


Bad error messages when creating Storage Service Client from connection string

Whilst creating samples, I ran into some issues working with the Storage blob client. The basic repro looks like:
from azure.storage.blob import BlobServiceClient

# Some connection string here
connection_string = ""
client = BlobServiceClient.from_connection_string(connection_string)
containers = client.list_containers()
for container in containers:
    print(container.name)

If I supply an empty string, I get this error:
ValueError: dictionary update sequence element #0 has length 1; 2 is required

An error message like "Malformed connection string" would be more useful.
If I have typo in my BlobEndpoint in my connection string, the operation hangs for a couple minutes before returning:
azure.core.exceptions.ServiceRequestError: <urllib3.connection.VerifiedHTTPSConnection object at 0x10ae29890>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known

I tried to enable logging so I could see the request/response traffic for this call, but I couldn't find guidance on how to do this.

I'm having this problem with get_service_stats method :
azure.core.exceptions.ServiceRequestError: <urllib3.connection.VerifiedHTTPSConnection object at 0x7f6e32d5d240>: Failed to establish a new connection: [Errno -2] Name or service not known
def geo(self):
        blob_service_client = BlobServiceClient.from_connection_string(self.access)
        acc = blob_service_client.get_service_stats()
        print(acc)

I could use the same object to connect , upload and download in a storage . I setup the storage as GRS ( geo-replication) , but I need to get the status of it ( Live , invalid , etc.. )

I put a logging in there , look :
# python3 main.py
Connected to Storage Account!
Request URL: 'https://rpaivatesting123-secondary.blob.core.windows.net/?restype=service&comp=stats'
Request method: 'GET'
Request headers:
    'Accept': 'application/xml'
    'x-ms-version': '2019-12-12'
    'x-ms-date': 'Wed, 11 Nov 2020 21:09:52 GMT'
    'x-ms-client-request-id': '3d73d876-2462-11eb-a223-7cb27d33114d'
    'User-Agent': 'azsdk-python-storage-blob/12.5.0 Python/3.6.9 (Linux-5.4.0-48-generic-x86_64-with-Ubuntu-18.04-bionic)'
    'Authorization': '*****'
Request body:
None
Request URL: 'https://rpaivatesting123-secondary.blob.core.windows.net/?restype=service&comp=stats'
Request method: 'GET'
Request headers:
    'Accept': 'application/xml'
    'x-ms-version': '2019-12-12'
    'x-ms-date': 'Wed, 11 Nov 2020 21:09:52 GMT'
    'x-ms-client-request-id': '3d73d876-2462-11eb-a223-7cb27d33114d'
    'User-Agent': 'azsdk-python-storage-blob/12.5.0 Python/3.6.9 (Linux-5.4.0-48-generic-x86_64-with-Ubuntu-18.04-bionic)'
    'Authorization': '*****'
Request body:
None

Request body:
None
Tuple timeout setting is deprecated
Traceback (most recent call last):
  File "main.py", line 16, in <module>
    health_check.geo()
  File "/root/sarah/DR/devops-toolbox/azure-storage-health-check/scripts/azure_storage_operations.py", line 113, in geo
    acc = bloba.get_service_stats(logging_enable=True)
  File "/usr/local/lib/python3.6/dist-packages/azure/core/tracing/decorator.py", line 83, in wrapper_use_tracer
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/azure/storage/blob/_blob_service_client.py", line 257, in get_service_stats
    timeout=timeout, use_location=LocationMode.SECONDARY, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/azure/storage/blob/_generated/operations/_service_operations.py", line 229, in get_statistics
    pipeline_response = self._client._pipeline.run(request, stream=False, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/azure/core/pipeline/_base.py", line 211, in run
    return first_node.send(pipeline_request)  # type: ignore
  File "/usr/local/lib/python3.6/dist-packages/azure/core/pipeline/_base.py", line 71, in send
    response = self.next.send(request)
  File "/usr/local/lib/python3.6/dist-packages/azure/core/pipeline/_base.py", line 71, in send
    response = self.next.send(request)
  File "/usr/local/lib/python3.6/dist-packages/azure/core/pipeline/_base.py", line 71, in send
    response = self.next.send(request)
  [Previous line repeated 5 more times]
  File "/usr/local/lib/python3.6/dist-packages/azure/core/pipeline/policies/_redirect.py", line 157, in send
    response = self.next.send(request)
  File "/usr/local/lib/python3.6/dist-packages/azure/core/pipeline/_base.py", line 71, in send
    response = self.next.send(request)
  File "/usr/local/lib/python3.6/dist-packages/azure/storage/blob/_shared/policies.py", line 543, in send
    raise err
  File "/usr/local/lib/python3.6/dist-packages/azure/storage/blob/_shared/policies.py", line 517, in send
    response = self.next.send(request)
  File "/usr/local/lib/python3.6/dist-packages/azure/core/pipeline/_base.py", line 71, in send
    response = self.next.send(request)
  File "/usr/local/lib/python3.6/dist-packages/azure/storage/blob/_shared/policies.py", line 318, in send
    response = self.next.send(request)
  File "/usr/local/lib/python3.6/dist-packages/azure/core/pipeline/_base.py", line 71, in send
    response = self.next.send(request)
  File "/usr/local/lib/python3.6/dist-packages/azure/core/pipeline/_base.py", line 71, in send
    response = self.next.send(request)
  File "/usr/local/lib/python3.6/dist-packages/azure/core/pipeline/_base.py", line 103, in send
    self._sender.send(request.http_request, **request.context.options),
  File "/usr/local/lib/python3.6/dist-packages/azure/core/pipeline/transport/_requests_basic.py", line 284, in send
    raise error
azure.core.exceptions.ServiceRequestError: <urllib3.connection.VerifiedHTTPSConnection object at 0x7f11404867b8>: Failed to establish a new connection: [Errno -2] Name or service not known

Has this issue been solved?
In case positive, what is the outcome?

I have the same issue doing blob_service_client = BlobServiceClient.from_connection_string(connect_str) the error is as below:
azure.core.exceptions.ServiceRequestError: <urllib3.connection.VerifiedHTTPSConnection object at 0x7ff02a0bd2b0>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known

Hi guys, i have the same problem, any issue ?
I m trying to connect with cosmosDB like this:
client = CosmosClient(url=endpoint, credential=key, connection_verify=False)
Error: azure.core.exceptions.ServiceRequestError: <urllib3.connection.HTTPSConnection object at 0x00000251A020AE90>: Failed to establish a new connection: [WinError 10061] nodename nor servname provided, or not known...


Fixes #2215: java.lang.IndexOutOfBoundsException: Index: 0, Size: 0 a…

…t openfoodfacts.github.scrachx.openfood.views.ContinuousScanActivity.onSuccess(ContinuousScanActivity.java:303)
Description
Added a condition to check if the environment tags list is empty or not which fixes the crash.
Related issues and discussion
#fixes #2215

Looks good to me. Thanks @balaji-ramavathu . Merging 👍


setup.cfg -> pyproject.toml

This builds properly, but it doesn't quite work on local builds; requests for shiny.js result in 404. I think it may have to do with the + character that setuptools_scm puts into the version string, e.g. 1.0.1.dev21+g4d7a2e5e.d20240821.

@wch @schloerke can either of you think of any downstream problems from shiny having version numbers that look like 1.0.1.dev27? In shinylive or something? Because this branch adds setuptools_scm and the way I've configured it, it will use version numbers like that for untagged builds.

Projects:

py-shinywidgets: ✅ ; Does not use the shiny version
py-shinyswatch: ✅ ; Uses installed shiny package and only uses bootstrap version.
shinylive assets: ✅ ; Does not use shiny version
py-shinylive: ✅ ; Currently only uses assets version and it's own package version
shinylive quarto extension: ✅ ; It only compares the shinylive assets version.

There could be possible issues in the future if we add .dev versions to py-shinylive. But in the current form it should work as long as the major and minor versions are not confused with .dev.

I may have missed something and would like @wch to add his stamp of approval. But I have confidence so far!

If we do need it on the packaging side, we can leverage packaging.version import Version for version comparison.
To remind myself... Unlike R where versions go from 1.1.0 to <IP_ADDRESS>00, python's .dev signals a dev version of the number that it's attached to. So after a 1.1.0 release, py-shiny should immediately update the version to 1.2.0.dev. As Version("1.2.0.dev0") < Version("1.2.0") is True.

TODO: Check if additional (unwanted) files are included in the sdist package bundle due to using setuptools_scm


Not published on NPM

This module is awesome. Please publish it on npm!

@rschmukler - thanks for the heads up, it should be there now.


Global mixin slowly kills Nuxt Rerender in Dev

Version
v2.11.0
Reproduction link
https://github.com/dotvirus/nuxt-global-mixin-bug
Steps to reproduce
Open the page.
Ctrl + R
Ctrl + R
Ctrl + R
... (repeat)
Watch as the render time takes longer and longer. For good measure, watch the RAM usage.
Compare performance with refreshes without global mixin:
https://github.com/dotvirus/nuxt-vuetify-typescript-express-boilerplate
What is expected ?
Page refreshes quickly again and again.
What is actually happening?
Render time increases, until Nuxt grinds to a halt.
Additional comments?
Templates & repos are using Vuetify & Typescript (with decorators).
The working repo (without mixin) works well, so I'm pretty sure at least Vuetify is not at fault here.
I may try to reproduce with vanilla JS & without Vuetify later.

This bug report is available on Nuxt community (#c10271)

https://github.com/dotvirus/nuxt-class-mixin-bug
Repo only using Nuxt Typescript and Vue Class Components with Property Decorators.
(Refresh around 5-10 times and you'll start to see the loading time increase dramatically)
Vanilla JS & Typescript without Class Components are not affected by this.


Batch awaitable

Add BatchAwaitable to wait for all stored Awaitable. SagaMonitor now uses BatchAwaitable to ensure all dispatch actions are completed and dispatch results are collected.

Codecov Report

Merging #19 into master will not change coverage.
The diff coverage is 100%.

@@          Coverage Diff          @@
##           master    #19   +/-   ##
=====================================
  Coverage     100%   100%
=====================================
  Files          41     41
  Lines         758    764    +6
=====================================
+ Hits          758    764    +6

Impacted Files
Coverage Δ

SwiftRedux/Middleware+Saga/Redux+SagaMonitor.swift
100% <100%> (ø)
:arrow_up:

SwiftRedux/Core/Redux+Awaitable.swift
100% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 90d10de...e0d8915. Read the comment docs.


go_1_14: 1.14 -> 1.14.1

Motivation for this change
New minor release https://github.com/golang/go/releases/tag/go1.14.1
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[ ] NixOS
[x] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

It was already merged into staging: #83689

hm.. I guess I missed that somehow. Thanks for noticing!


Feature: DOM mutation observer API

Expected
It would be nice to have DOM mutation observer available in bridgedotnet. It makes it possible to efficiently notice changes to document (elements added or removed, attributes changed). This powerful API permits scenarios such as: element was added deep into document tree and after it is attached one would like to call some "on element attached" code.
Actual
It currently is not built in, I've developed wrapper and tested it in my personal project.
Pull request will be sent in a while.

Thanks for the excellent pull-request. We're taking a good look at this and have tagged for possible acceptance into the 15.4 release.


Removing a video should be global (eg. regular videos/recently played/etc..)

Removing a video should be global (eg. regular videos/recently played/etc..)

Fixed via 10e85980b2088688f1ffd97af95d692918623cab


[Bug] TabView - TabIndicatorColor isn't working on load

Description
In a ContentPage, I'm using TabView. After updating to the newest Xamarin.Forms <IP_ADDRESS>3, Xamarin.Essentials 1.7.0 and Xamarin.CommunityToolkit 1.3.0-pre1, the TabIndicatorColor in TabView doesn't show up until we start changing tabs.
Steps to Reproduce

Start a new Xamarin Forms project
Update to the latest Xamarin.Forms <IP_ADDRESS>3
Add a new TabView on a page
Give the TabView a TabIndicatorColor property a color value
Run the app and not see the tab indicator

Expected Behavior
TabIndicatorColor should appear bellow the tab selected
Actual Behavior
TabIndicatorColor won't show up until starting to change tabs
Basic Information

Version with issue: 1.3.0-pre1, 1.2.0 , 1.1.0, 1.03
Last known good version: 1.0.2
IDE: Visual Studio 2019 Version 16.10.4
Platform Target Frameworks:

iOS:  14.5
Android: Android 11.0 (R)

Affected Devices: Android and iOS

Workaround
Reproduction imagery

Reproduction Link
https://github.com/NunoBem/TabViewIssue

Hey thanks for the report! Did you by any chance also test other platforms if it's happening there too?

Hi Gerald,
iOS same bug

I am having the same issue as well on iOS, but it works for me on Android
Xamarin.Forms v<IP_ADDRESS>2
XCT v1.2.0

I have the same issue only in iOS. I'm using XCT v1.2.0. Any workaround?

I fixed the issue on mine for Android, haven't checked iOS yet.
If you name your TabView component and manually set its index on load it will work. Seems to be that the tab view doesnt know what index it's on or something?
         public MyPageView()         {             InitializeComponent();             tabs.SelectedIndex = 0; // For first item selected         }

It always worked for me on Android. iOS is the one that doesn't work.

I fixed the issue on mine for Android, haven't checked iOS yet. If you name your TabView component and manually set its index on load it will work. Seems to be that the tab view doesnt know what index it's on or something?
 public MyPageView() { InitializeComponent(); tabs.SelectedIndex = 0; // For first item selected }
Edit: Sorry for formatting of code, not sure how to make it nice.

I was going through the same problem and I built this workaround. Hope it helps guys :)
`xct:TabView.TabIndicatorView

</xct:TabView.TabIndicatorView>
            <xct:TabViewItem
                x:Name="tabItemBase"
                FontSize="12"
                Text="Todos"
                TextColor="#4D4D4D"
                TextColorSelected="#4D4D4D">
                <views:TodosView />
            </xct:TabViewItem>`

@kainaalmeida It's working for me, but Animation not good :)
My Devices: Samsung s8+ and IPhone XS MAX.
Thanks for comment.

 I was going through the same problem and I built this workaround. Hope it helps guys :)
<xct:TabView.TabIndicatorView> <BoxView BackgroundColor="#0CB498" HeightRequest="3" WidthRequest="{Binding Source={x:Reference tabItemBase}, Path=Width}" /> </xct:TabView.TabIndicatorView> <xct:TabViewItem x:Name="tabItemBase" FontSize="12" Text="Todos" TextColor="#4D4D4D" TextColorSelected="#4D4D4D"> <views:TodosView /> </xct:TabViewItem>

Thanks! This worked for me. I don't see an animation issue. I tried on iPhone 12 Pro, iPhone 8 and Samsung S8.

@kainaalmeida It's working for me, but bad Animation and I decided not to use it. My Devices: Samsung s8+ and IPhone XS MAX. Thanks kai for comment 👍

@minhsu12113 Here everything is normally, I tested it on the iphone if and on the redmi note 8

 I was going through the same problem and I built this workaround. Hope it helps guys :)
<xct:TabView.TabIndicatorView> <BoxView BackgroundColor="#0CB498" HeightRequest="3" WidthRequest="{Binding Source={x:Reference tabItemBase}, Path=Width}" /> </xct:TabView.TabIndicatorView> <xct:TabViewItem x:Name="tabItemBase" FontSize="12" Text="Todos" TextColor="#4D4D4D" TextColorSelected="#4D4D4D"> <views:TodosView /> </xct:TabViewItem>

@kainaalmeida Worked for me. Thanks!


Revert "Changes to add CMD for runtime images"

Could you please review the changes?

I don't think we should merge this as-is, since it will cause the runtime images to be less feature ful lthan they are now. The same changes are part of #277 , so we should just merge them as part of that change when it is ready.


Bug: Update form doesn't load doc

Hi,
I am passing a valid doc to an update form and it won't load it. I have already posted on StackOverflow with no luck.
http://stackoverflow.com/questions/34371554/meteorjs-update-autoform-not-loading-doc
Thanks,
Paul

Found the problem. Have updated Stackoverflow.


fix(logger): reject unreasonable data

:tada: This PR is included in version 2.25.6 :tada:
The release is available on:

v2.25.6
v2.25.6
GitHub release

Your semantic-release bot :package::rocket:


[Design] Make GrantRole global privilege

Currently GrantRole is a Database scoped Privilege, i.e it can only be granted on a database or collection. But there are three privileges that are global and we need to be able to grant role without the being scoped to database.
If we send the request without the on field, wdb crashes and panics

Introduced userPrivilege category

Added userPrivilege category
grantRole in userPrivilege category
for userPrivileges, additional cascading checks are conducted if db is present
made the grantRole scope param (on) optional
added user scope


scheduler: support sharable candidate

这个 PR 实现什么功能/修复什么问题:
调度支持 public 的 host
是否需要 backport 到之前的 release 分支:

release/2.10.0

/area scheduler
/cc @swordqiu

Codecov Report

Merging #1152 into master will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff           @@
##           master   #1152   +/-   ##
======================================
  Coverage     7.9%    7.9%
======================================
  Files         364     364
  Lines       57151   57151
======================================
  Hits         4517    4517
  Misses      52236   52236
  Partials      398     398

Flag
Coverage Δ

#aFlag
7.9% <ø> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a2c6750...4257263. Read the comment docs.

/lgtm
/approve


Frontend: Cache previously accessed DAOs

At the moment, it might be a bit heavy to show a user all the DAOs that they're a part of as that query, but in the meantime, we can give a user a list of their recently-accessed DAOs in order to serve those quicker when they're in the DAO list.

User accesses the DAO list
In a top tab there's a separation between "recent" and "all" to note it for the user
Clicking "recent" would pull the cached list
Clicking "all" would give the full view of available DAOs


Fix create new target_machine for every program

Compiler#new_program instantiates a new Program instance which is used for compiling a macro run program. Some properties (such as codegen target) can be shared between multiple compilations. But the TargetMachine instance must not be shared because this would leak potentially incompatible data layouts between programs.
This patch creates a new instance of TargetMachine for every program.
See https://github.com/crystal-lang/crystal/issues/14496#issuecomment-2161174346 and previous comments for details.
Resolves (probably) #14496

What's the real difference? IIUC the original program was calling the getter, who was in turn creating an instance.


Expression that works for EF 6 can't be translated to SQL in EF Core

Describe what is not working as expected.
I have a factory that can create expression that can be translated to SQL to check if, for example, a DateTimeOffset is between the property Created and Delete to be sure I only fetch active Posts.
// x.Created and x.Publish can be public or private, doesn't matter
public static Expression<Func<Post, bool>> GenerateIsNotYetPublishedExpression(DateTimeOffset dateTimeOffset)
    => GenerateIntervalContainsExpression<Post>(dateTimeOffset, startDateExp: x => x.Created, endDateExp: x => x.Publish);

The full code can be find in my example project but here is the important bit:
/// <summary>
/// Generate an expression tree that can be used ie for creating a db query in EF to find entities within the interval.
/// </summary>
/// <typeparam name="TEntity">Entity the expression tree will be built for.</typeparam>
/// <typeparam name="TComp">Type to be compared between.</typeparam>
/// <param name="compareTo"></param>
/// <param name="startPropertyExp">Expression that should return the property to use as start.</param>
/// <param name="endPropretyExp">Expression that should return the property to use as end.</param>
/// <param name="includeStart">Include or exlude start.</param>
/// <param name="includeEnd">Include or exlude end.</param>
/// <returns>Expression tree for Linq-to-SQL</returns>
public static Expression<Func<TEntity, bool>> GenerateIntervalContainsExpression<TEntity, TComp>(
    TComp compareTo,
    Expression<Func<TEntity, TComp?>> startPropertyExp,
    Expression<Func<TEntity, TComp?>> endPropretyExp,
    bool includeStart = true,
    bool includeEnd = true)
    where TEntity : class
    where TComp : struct, IComparable<TComp>
{
    var param = Expression.Parameter(typeof(TEntity), "entity");

    Expression<Func<TComp?, TComp?, bool, bool, bool>> intervalContainsExp = (start, end, inclStart, inclEnd) =>
        (start == null || (start.Value.CompareTo(compareTo) <= 0 && inclStart) || (start.Value.CompareTo(compareTo) < 0 && !inclStart))
        && (end == null || (end.Value.CompareTo(compareTo) >= 0 && inclEnd) || (end.Value.CompareTo(compareTo) > 0 && !inclEnd));

    var args = new Dictionary<string, ConstantExpression>
    {
        [intervalContainsExp.Parameters[2].ToString()] = Expression.Constant(includeStart, typeof(bool)),
        [intervalContainsExp.Parameters[3].ToString()] = Expression.Constant(includeEnd, typeof(bool)),
    };

    var foo = Compose(intervalContainsExp, startPropertyExp, endPropretyExp, args);

    Expression<Func<bool, bool>> bar = dummy => dummy;

    return Compose(bar, foo, param);
}

private static Expression<Func<TEntity, TEntity, B>> Compose<TEntity, A, B>(
    Expression<Func<A, A, B, B, B>> f,
    Expression<Func<TEntity, A>> startExp,
    Expression<Func<TEntity, A>> endExp,
    IDictionary<string, ConstantExpression> constants)
{
    var ex = ReplaceExpressions(f.Body, f.Parameters[0], startExp.Body);
    ex = ReplaceExpressions(ex, f.Parameters[1], endExp.Body);

    var foo = Expression.Lambda<Func<TEntity, TEntity, B>>(ex, startExp.Parameters[0], endExp.Parameters[0]);

    return (Expression<Func<TEntity, TEntity, B>>)new ParametersTransformToConstantVisitor(constants)
        .Visit(foo);
}

private static Expression<Func<A, C>> Compose<A, B, C>(Expression<Func<B, C>> f, Expression<Func<A, A, B>> g, ParameterExpression param)
{
    var ex = ReplaceExpressions(f.Body, f.Parameters[0], g.Body);

    ex = new ParameterReplacer(param)
        .Visit(ex);

    return Expression.Lambda<Func<A, C>>(ex, param);
}

private static TExpr ReplaceExpressions<TExpr>(TExpr expression, Expression orig, Expression replacement)
    where TExpr : Expression
{
    var replacer = new ExpressionReplacer(orig, replacement);

    return replacer.VisitAndConvert(expression, nameof(ReplaceExpressions));
}

private class ParametersTransformToConstantVisitor : ExpressionVisitor
{
    private readonly IDictionary<string, ConstantExpression> _parameters;

    public ParametersTransformToConstantVisitor(IDictionary<string, ConstantExpression> parameters)
    {
        _parameters = parameters;
    }

    protected override Expression VisitParameter(ParameterExpression node)
        => _parameters.TryGetValue(node.Name, out var ce) ? (Expression)ce : node;

    protected override Expression VisitLambda<T>(Expression<T> node)
        => Expression.Lambda(Visit(node.Body), node.Parameters); // don't visit the parameters
}

private class ParameterReplacer : ExpressionVisitor
{
    private readonly ParameterExpression _parameter;

    internal ParameterReplacer(ParameterExpression parameter)
    {
        _parameter = parameter;
    }

    protected override Expression VisitParameter(ParameterExpression node)
        => base.VisitParameter(_parameter);
}

private class ExpressionReplacer : ExpressionVisitor
{
    private readonly Expression _from;
    private readonly Expression _to;

    public ExpressionReplacer(Expression from, Expression to)
    {
        _from = from;
        _to = to;
    }

    public override Expression Visit(Expression node)
        => node == _from
            ? _to
            : base.Visit(node);
}

The reason behind this is that I use NodaTime. Therefore I need to have private backing fields/properties that I map to the database instead of my public properties when using EF 6. Since it's a bit of a PITA to write the same code over and over again I can just use the factory and point it to which property is the start of the interval and which is the end of the interval. The code has been working well and EF 6 can use it to translate my queries to proper SQL, all fine and dandy! However, EF Core can't translate the expression to SQL but has to use client side evaluation:
Exception message: The LINQ expression 'where ((((Convert([entity].Created) == null) OrElse (([entity].Created.CompareTo(__compareTo_0) <= 0) AndAlso True)) OrElse False) AndAlso ((([entity].Deleted == null) OrElse False) OrElse ((Convert([entity].Deleted).CompareTo(__compareTo_0) > 0) AndAlso True)))' could not be translated and will be evaluated locally.'.

Notice that the error above is when using Nullable<DateTimeOffset>, which I'm not using in my example but either way they fail.
Steps to reproduce
Clone and run example project once in Visual Studio to create a database on SQL Server 2017 Express LocalDB. Then run the tests. Get_All_Unpublished_Posts_Not_Working should be failing and every other should pass. The code is a bit (well, a lot) messy, partially because I didn't know that DateTimeOffset didn't work properly for SQLite so I had to move to SQL Server when I already thought I was finished. Enough with my excuses...
Further technical details
EF Core version: 2.2.4
Database Provider: Microsoft.EntityFrameworkCore.SqlServer
Operating system: Windows 10 1803
IDE: Visual Studio 2019 16.1.3

I upgraded repro project shared to 3.1 and all tests are passing now.

Thanks @smitpatel!


super_select accepts_new documentation needs controller logic example.

The documentation at https://bullettrain.co/docs/field-partials/super-select describes how do configure the field partial, but doesn't provide an example of how to process the payload in the controller.

We can use the join-model Super Scaffolding example as a starting point for this documentation. No need to duplicate it into these docs, just reference it as prerequisite knowledge.
The implementation in process_params of app/controllers/account/projects_controller.rb would look like this:
    strong_params[:tag_ids] = create_models_if_new(strong_params[:tag_ids]) do |name|
      @team.projects_tags.find_or_create_by(name: name)
    end

We should also an example of creating values on the fly for super_select fields that aren't multiple: true (e.g. with create_model_if_new instead of create_models_if_new) For this, we could use the belongs_to Super Select example as a starting point, showing them an example where the user provides an email address and we create a membership and invitation on the fly.
Let's create this as an independent markdown document and link to it under Advanced Examples at the bottom of https://github.com/bullet-train-co/bullet_train-core/blob/main/bullet_train/docs/field-partials/super-select.md .

Sounds good! Will tackle this next.

We now have info about how to handle things server side in this section of the docs:
https://bullettrain.co/docs/field-partials/super-select#accepting-new-entries

Looks like @pascallaliberte added those in https://github.com/bullet-train-co/bullet_train-core/pull/591
I'm going to close this issue.