Control filebeat from hiera, not conditional logic

This PR removes conditional logic for controlling filebeat stuff:
if $::domain =~ /^.*\.(dev|integration\.publishing\.service)\.gov\.uk/ {
  # blah
}

and replaces it with a govuk_beat::enable hiera value, which is disabled by default.
The idea is that we want to add more filebeat::prospectors but we don't want to add a conditional every time we want to add a prospector.
This should be a pure refactoring -- there shouldn't be any behaviour changes here.
I've copied the filebeat::prospector commit message here too for visibility:
collect prospectors in the filebeat class
We only want filebeat::prospector instances to have an effect on nodes
which actually have filebeat installed.
Previously, this was achieved in nginx::log by having an if
statement conditional on $::domain.  However, I don't really want to
duplicate that conditional logic everywhere we might want to have a
prospector.
This commit makes filebeat::prospector create virtual file resources,
which only get realized in the filebeat class.  This means we can
remove the conditional logic from where we declare prospectors.
(For this, I take advantage of puppet's automatic tagging: the file
created in filebeat::prospector is automatically tagged with
filebeat::prospector).
We use virtual resources elsewhere with things like @collectd::plugin,
so there is prior art for this kind of thing.
I wasn't sure whether to make the file resource virtual, or the
filebeat::prospector resource.

This looks good, but wondering whether making the filebeat::prospector resource virtual makes more sense when reading the code?

@surminus yeah I think you're probably right.  Happy to do that

@surminus updated


SSKeychain doesn't retain/restore its data when app gets updated from iTunes

First thanks for having such library :+1: .
I have been using this open source library for storing the data securely in my iOS app. Yesterday, I face an issue ,SSKeychain wasn't able to retain its data when I updated my app from v1.0 to v2.0 from iTunes.
Here is the cope snippet :
NSString *retrieveuuid = [SSKeychain<EMAIL_ADDRESS>account:@"AppName"];
if (retrieveuuid == nil) {
    NSString *uuid  = [self createNewUUID];

    //Store the password in Keychain
    NSError *error = nil;
    [SSKeychain setPassword:uuid<EMAIL_ADDRESS>account:@"AppName" error:&error];

    if ([error code] == SSKeychainErrorNotFound) {
        NSLog(@"ID not found");
    }
}

Over here, I generated a unique device string and used the keychain to store the same and the app heavily depends on unique string since from iOS5 to iOS7 there are lots of transformations done by Apple in concerned to Unique Identifier, as the methods got deprecated.
So, is this something that keychain won't be able to retain its values/identifier, when the app gets updated from Apple OR am I missing out at some point. Please help out if its possible to store the Identifier permanently in device, irrelevant of Installing, uninstalling ,reset and updating the app.
Thanks,

This isn't an issue with SSKeychain. I'd recommend searching for "Keychain Item Accessibility Constants" in the documentation.


Database tests

@creativepsyco Please check and merge.

@creativepsyco I squashed the commits.

Can you rebase with fossasia:master and push again? The travis test won't work if the PR cannot be merged (or is in a merge-able) state


[stable/verdaccio] Upgrade version, add logo and fix documentation

upgrade Verdaccio to 2.7.0
add logo, keywords and additional source
fix documentation

/ok-to-test

/lgtm


Storage should reject read requests for non-existent org-bucket pairs

I was writing some tests for task run logs reading from the system bucket, and when I used a deliberately bad organization ID, against a real storage engine and query controller, I was surprised to see that the query returned successfully (albeit with an empty result set).
Reading from a non-existent org-bucket pair is roughly equivalent to reading from a non-existent DB in 1.x. The read should be rejected with a useful error message.

I'm not sure whether this is still an issue.


"error:1408F10B:SSL routines:ssl3_get_record:wrong version number" on https monitoring

⚠ Please verify that this bug has NOT been raised before.

[X] I checked and didn't find similar issue

🛡 Security Policy

[X] I agree to have read this project Security Policy

📝 Describe your problem
Monitoring the URL test2.suitec.de works without problems in http, but throws following error on https:
"write EPROTO<PHONE_NUMBER>80416:error:1408F10B:SSL routines:ssl3_get_record:wrong version number:../deps/openssl/openssl/ssl/record/ssl3_record.c:332:"
The screenshot shows me switching between the http and https version multiple times

Visiting the https website in the browser works without problems
A quick google search yielded me the possibility of http being provided on the https port, however trying to enforce http on port 443 returns an error and calling the https site did not seem to leave any trace of http in the wireshark logs
The Server behind the URL is running behind a proxy, however I think it to be unlikely that that is the cause of the problem, seeing as test1.suitec.de is running behind the same proxy and has no problems being monitored on both http and https.
In theory I could just monitor the status of the server itself in http and that would solve most of my problems, however then I wouldn't be notified if for some reason the renewal of certificates failed, so I would really like to get this running properly.
Any help would be appreciated and I am open to questions
🐻 Uptime-Kuma Version
1.17.1
💻 Operating System and Arch
Ubuntu 22.04 LTS
🌐 Browser
Google Chrome Version 103.0.5060.114 & Safari Version 15.3
🐋 Docker Version
Docker version 20.10.12, build 20.10.12-0ubuntu4 & docker-compose version 1.29.2, build unknown
🟩 NodeJS Version
No response

Was it randomly happened?
I suggest that you should setup another Uptime Kuma instance in a different location, see if the issue is still existing.

Hey, I'm sorry for the lack of response.
I have by now tried it in the Sandbox environment and there the Address was perfectly reachable under https. I will have to move my installation soon anyways, so I have the hopes that that will fix my issue. If not I'll ask here again.
Thank you for the help :)

I had the same issue.
And it happens randomly
Version: 1.18.0


haskell.packages.ghc7103.cheapskate is broken

Issue description
cheapskate is broken on GHC 7.10.3
Steps to reproduce
$ nix-build '<nixpkgs>' -A haskell.packages.ghc7103.cheapskate
these derivations will be built:
  /nix/store/hrni0zn7l9sjgdz06f6i3s9diplirr15-cheapskate-<IP_ADDRESS>.drv
building path(s) _/nix/store/2p78794zsi493c170550zx7qx2ngsmdx-cheapskate-<IP_ADDRESS>_
setupCompilerEnvironmentPhase
Build with /nix/store/ciz5l5c2v8xwvcp1fgmvlc2rwmw3mz93-ghc-7.10.3.
unpacking sources
unpacking source archive /nix/store/8jj1y68d02a15v0w5fizbay0kkyyj8ba-cheapskate-<IP_ADDRESS>.tar.gz
source root is cheapskate-<IP_ADDRESS>
setting SOURCE_DATE_EPOCH to timestamp<PHONE_NUMBER> of file cheapskate-<IP_ADDRESS>/Setup.hs
patching sources
Replace Cabal file with edited version from http://hackage.haskell.org/package/cheapskate-<IP_ADDRESS>/revision/1.cabal.
compileBuildDriverPhase
setupCompileFlags: -package-db=/tmp/nix-build-cheapskate-<IP_ADDRESS>.drv-0/package.conf.d -j48 -threaded
[1 of 1] Compiling Main             ( Setup.hs, /tmp/nix-build-cheapskate-<IP_ADDRESS>.drv-0/Main.o )
Linking Setup ...
configuring
configureFlags: --verbose --prefix=/nix/store/2p78794zsi493c170550zx7qx2ngsmdx-cheapskate-<IP_ADDRESS> --libdir=$prefix/lib/$compiler --libsubdir=$pkgid --with-gcc=gcc --package-db=/tmp/nix-build-cheapskate-<IP_ADDRESS>.drv-0/package.conf.d --ghc-option=-optl=-Wl,-rpath=/nix/store/2p78794zsi493c170550zx7qx2ngsmdx-cheapskate-<IP_ADDRESS>/lib/ghc-7.10.3/cheapskate-<IP_ADDRESS> --enable-split-objs --disable-library-profiling --disable-executable-profiling --enable-shared --enable-library-vanilla --enable-executable-dynamic --enable-tests
Configuring cheapskate-<IP_ADDRESS>...
Setup: At least the following dependencies are missing:
aeson -any, http-types -any, wai >=0.3, wai-extra -any
builder for _/nix/store/hrni0zn7l9sjgdz06f6i3s9diplirr15-cheapskate-<IP_ADDRESS>.drv_ failed with exit code 1
error: build of _/nix/store/hrni0zn7l9sjgdz06f6i3s9diplirr15-cheapskate-<IP_ADDRESS>.drv_ failed

Technical details

System: (Ubuntu 14.04.5 LTS)
Nix version: (1.11.4)
Nixpkgs version: (16.09beta430.c4469ed)

@peti, @abbradar: perhaps related to this issue https://github.com/haskell/cabal/issues/2598

Forgot to mention - it builds with GHC 8.0.1

Thanks for looking into this!


⚠ Turnos 24x7 has degraded performance

In 0c2b914, Turnos 24x7 (https://reservas-dev.alternativasinteligentes.com/) experienced degraded performance:

HTTP code: 200
Response time: 1191 ms

Resolved: Turnos 24x7 performance has improved in 7461b51.


[SPARK-48118][SQL] Support SPARK_SQL_LEGACY_CREATE_HIVE_TABLE env variable

What changes were proposed in this pull request?
This PR aims to support SPARK_SQL_LEGACY_CREATE_HIVE_TABLE env variable to provide users an easier migration path.
Why are the changes needed?
Like SPARK_ANSI_SQL_MODE for spark.sql.ansi.enabled, the platform providers can control the default value of spark.sql.legacy.createHiveTableByDefault configuration.
Does this PR introduce any user-facing change?
No.
How was this patch tested?
Pass the CIs.
Was this patch authored or co-authored using generative AI tooling?
No.

cc @cloud-fan

Could you review this SQL PR too for Apache Spark 4.0.0-preview, @viirya ?

Thank you, @viirya !

Merged to master.


Integration of Atlantic Service and HTTP Client Infrastructure

Summary
This PR introduces the Atlantic service integration along with reusable HTTP client infrastructure components.
Changes

Integrated Atlantic service for proving capabilities
Implemented a generic HTTP client for reusability across services
Created a request builder utility for standardized request construction

Technical Details

The generic HTTP client provides helper functions that can be reused for future HTTP client implementations
Request builder utility simplifies and standardizes the process of creating HTTP requests
Atlantic service integration leverages the new HTTP infrastructure

Testing
Test also added for the atlantic service

coverage: 70.035% (+0.7%) from 69.327%
when pulling 185de15f99c8e1d904d906aecb75aea54605508f on atlantic_refactor
into 803be5ff50f99be52afa37e3d795a2d10934cbe2 on main.


How to use calendar ?

which files js and css implement to use the calendar ?
how to have a link on left sidebar ?
Thanks

This is not AdminThemeBundle-specific. Please read AdminLTE's documentation to learn more
Create a page, put the calendar in it, and add a menu item leading to your page's route - follow AdminThemeBundle documentation to learn more about adding items to your menu.


how would i add serving static files with return file(filename) in the posts endpoint to return images for the posts

Returning a file is not a great idea. The post should return image URLs, which I guess you can generate with url_for(). Then for development Flask serves these images, but in production you have a file server that gets these before they get to Flask. For example nginx.


ListBox deselects selected item when calling Move() on bound ObservableCollection

Bug Description
When a ListBox.ItemSource is bound to an ObservableCollection, and an item is selected, if Move() is called on the underlying item in the collection, then the ListBox selection is cleared. This does not happen if the selected item is not the one that Move() is called on, even though its position may change.
To Reproduce
The following example contains a ListBox bound to an ObservableCollection. The button calls Move() to move the first item to the end of the list. Note that when the first item is selected and subsequently moved, it is deselected. Selecting any other item and calling Move() causes the item's position to change, but it stays selected.
XAML:
<UserControl xmlns="https://github.com/avaloniaui"
             xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
             xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
             xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
             xmlns:vm="clr-namespace:Bug.ViewModels"
             mc:Ignorable="d" d:DesignWidth="200" d:DesignHeight="450"
             Width="200" Height="450"
             x:Class="Bug.Views.MainView"
             x:DataType="vm:MainViewModel">
  <DockPanel>
    <Button Content="Move" DockPanel.Dock="Right" Command="{Binding MoveCommand}"/>
    <ListBox ItemsSource="{Binding Items}">
      <ListBox.ItemTemplate>
        <DataTemplate>
          <Label Content="{Binding}" />
        </DataTemplate>
      </ListBox.ItemTemplate>
    </ListBox>
  </DockPanel>
</UserControl>

ViewModel:
public class MainViewModel : ViewModelBase
{
    public ObservableCollection<string> Items { get; set; } = new ObservableCollection<string>()
    {
        "Item 1",
        "Item 2",
        "Item 3",
        "Item 4",
        "Item 5"
    };

    public ICommand MoveCommand { get; set; }

    public MainViewModel()
    {
        MoveCommand = ReactiveCommand.Create(() =>
        {
            Items.Move(0, Items.Count - 1);
        });
    }
}

Expected Behavior
The ListBox's selected item should remain selected after calling ObservableCollection.Move() on the underlying item.
Avalonia Version
11.1.0-rc2
OS
Windows

This also happens with Dynamic Data.


Feature/day21

Codecov Report

Merging #24 (4242fcc) into master (cb10cef) will decrease coverage by 0.09%.
The diff coverage is 92.18%.

:exclamation: Current head 4242fcc differs from pull request most recent head c09cb48. Consider uploading reports for the commit c09cb48 to get more accurate results

@@            Coverage Diff             @@
##           master      #24      +/-   ##
==========================================
- Coverage   93.44%   93.35%   -0.10%
==========================================
  Files          20       21       +1
  Lines        1664     1792     +128
==========================================
+ Hits         1555     1673     +118
- Misses        109      119      +10

Impacted Files
Coverage Δ

day21/src/main.rs
92.18% <92.18%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cb10cef...c09cb48. Read the comment docs.


Allow serializing underlying NSError objects, closes #10506

Explain the motivation for making this change. What existing problem does the pull request solve?
See https://github.com/facebook/react-native/issues/10506. A native NSError with NSUnderlyingErrorKey set causes a JSON stringify error from the websocket dispatcher if remote debugging is enabled.
Test plan (required)
I'm not familiar with the react native testing framework. Happy to add a test for this if someone can point me to where this part of the codebase is exercised :)
I did some spot checks with nil user dictionaries and nil underlying errors here. The case that this solves is testable using https://github.com/superseriouscompany/react-native-error-repro, specifically:
NSError *underlyingError = [NSError errorWithDomain:@"underlyingDomain" code:421 userInfo:nil];
NSError *err = [NSError errorWithDomain:@"domain" code:68 userInfo:@{@"NSUnderlyingError": underlyingError}];

reject(@"foo", @"bar", err);

It's a little odd but the unit tests for this area of the code are in Examples/UIExplorer/UIExplorerUnitTests/ . It would be great to add one! - just a simple one that ensures this one use case works.

Thanks for pointing me in the right direction @lacker, added a unit test. Let me know what else is needed.

@facebook-github-bot shipit


Warnings on file upload

When uploading a file, the following warnings are logged
[WARN] 14-Oct-2018 20:22:51.223 | synapticloop.b2.response.B2AuthorizeAccountResponse - Found an unexpected key of 'allowed' in JSON that is not mapped to a field, with value '{"bucketName":null,"capabilities":["listKeys","writeKeys","deleteKeys","listBuckets","writeBuckets","deleteBuckets","listFiles","readFiles","shareFiles","writeFiles","deleteFiles"],"namePrefix":null,"bucketId":null}'.
[WARN] 14-Oct-2018 20:22:51.223 | synapticloop.b2.response.B2AuthorizeAccountResponse - Found an unexpected key of 'recommendedPartSize' in JSON that is not mapped to a field, with value '100000000'.
[WARN] 14-Oct-2018 20:22:51.224 | synapticloop.b2.response.B2AuthorizeAccountResponse - Found an unexpected key of 'absoluteMinimumPartSize' in JSON that is not mapped to a field, with value '5000000'.

Is it possible to have a setting/property not to warn on missed keys.

I believe these are new keys that aren't being accounted for by this library :(

Indeed I had a look at the code and the "allowed" field isn't listed. The other two seem to be among expected fields - https://github.com/synapticloop/backblaze-b2-java-api/blob/master/src/main/java/synapticloop/b2/response/B2AuthorizeAccountResponse.java. Not sure why they are being included.

Just realised that I'm using the 2.2.0 jar whose code may be different from the latest. That would explain the why.


Dynamic MPI support

In the context of NEURON wheel integration of CoreNEURON we need to implement dynamic MPI capabilities.
Before diving into implementation details, a few ideas to bare in mind:

would it be possible to capitalize on the libnrnmpi library?
if yes, for the short term we can detect is CoreNEURON is a submodule and use libnrnmpi; on the longer term, we can then see about implementing dynamic MPI in CoreNEURON or maybe extract libnrnmpi in another repo ?

would it be possible to capitalize on the libnrnmpi library?

I think the answer is yes. But it is not clear to me how independent the data part has to be when NEURON and CoreNEURON coexist in direct mode. In direct mode, I think NEURON can initialize MPI and CoreNEURON just use that instance. Ie. the
MPI_Comm nrnmpi_comm;

Something to note is that a script, mkdynam.sh, transforms the nrnmpidec.h list of nrnmpi wrappers for the desired MPI functions
into the helper files, nrnmpi_dynam_wrappers.inc, nrnmpi_dynam.h, nrnmpi_dynam_cinc.  And nrnmpi_dynam.cpp is responsible
for dlopening the correct libmpi and then, based on the libmpi implementation, dlopening the correct libnrnmpi.

@alexsavulescu : this ticket would require some analysis of current code usage in CoreNEURON. We can have discussion about this in next month / after summer break.

Tagging here @sergiorg-hpc as well as he has recent experience with dynamic library loading. Next week can discuss some of the technical details.

I pushed a sketch of how I think this could be handled to https://bbpgitlab.epfl.ch/hpc/user/coreneuron-dynamic-mpi-sandbox (https://godbolt.org/z/erY4Y6f9K is a single-file no-dependency version of the same).

Some pointers/next steps:

The nrnmpi_... functions that are used in the CoreNEURON code are declared here and defined here.
In CoreNEURON these should be replaced with instances of the mpi_function<...> helper in the coreneuron-dynamic-mpi-sandbox example above. These should be callable exactly as before, i.e. no changes should be needed at the call sites: https://github.com/BlueBrain/CoreNeuron/blob/db8477c90fe33f6c7f7025482e81ac0972ed12d0/coreneuron/utils/memory_utils.cpp#L82-L84
With dynamic MPI disabled in CoreNEURON (i.e. taking the 2nd branch of https://bbpgitlab.epfl.ch/hpc/user/coreneuron-dynamic-mpi-sandbox/-/blob/aab81a461c3a1d69ac4aaa760ee1eec04068ab23/include/coreneuron/mpi_utils.h#L49-58), nothing should break. Implementations like this are compiled/linked into other CoreNEURON code, -lmpi is used.
With dynamic MPI enabled in CoreNEURON, we need to compile a separate shared library containing the implementations like this. This shared library is linked with -lmpi, but CoreNEURON/special/... do not link against it. It should be dynamically loaded if --mpi is passed, the instances of mpi_function<...> will use it to initialise their function pointers to the implementations.

For this ticket, we can ignore the possibility of having multiple MPI implementations. Later, when CoreNEURON and NEURON are more tightly integrated, we can inherit that logic from NEURON.
cc: @pramodk @alexsavulescu @alkino


Fix race condtion when mutliple local endpoints are created

There is a race condition when the worker nodes get restarted one by one. More than one submariner endpoint is created in one cluster as two gateways tries to be the leader. One yields later and this triggers a endpoint delete event. This causes routes to be cleared in both local and remote cluster. Later an endpoint update even comes for the active gateway, and we are using the update event in the fix

More than one submariner endpoint is created in one cluster as two gateways tries to be the leader. One yields later and this triggers a endpoint delete event.

Can you explain this in more detail? Have you observed a scenario where 2 gateways are simultaneously the leader?
Whatever the exact scenario/sequence, I wonder if https://github.com/submariner-io/submariner/pull/2763 might alleviate it.

More than one submariner endpoint is created in one cluster as two gateways tries to be the leader. One yields later and this triggers a endpoint delete event.

Can you explain this in more detail? Have you observed a scenario where 2 gateways are simultaneously the leader?
Whatever the exact scenario/sequence, I wonder if #2763 might alleviate it.

When nodes are restarted, there are disruption in api server connection and during this time,  two leaders are present simultaneously and two local endpoints are created. Later one of them yields as it is not able to get the gateway lock.
I think https://github.com/submariner-io/submariner/pull/2763 will not fix it. @sridhargaddam  any thoughts?

When nodes are restarted, there are disruption in api server connection and during this time, two leaders are present simultaneously and two local endpoints are created. Later one of them yields as it is not able to get the gateway lock.

Are you saying that there are 2 local endpoints present in the cluster at the same time? When the gateway acquires the leader lock, it first deletes any other endpoints before creating its own. Also it doesn't delete its endpoint if it loses the leader lock and/or exits. I'm trying to see a scenario where 2 local endpoints coexist simultaneously. I think we need to understand this.
If the API server isn't available then it shouldn't be able to acquire the leader lock nor create endpoints as those operations require access to the API server.
What could happen is if the first gateway acquires the lock and creates an endpoint then later loses connection to the API server and can't renew the lock after 5 sec (RenewDeadline) and exits. Then the second gateway acquires the lock after 10 sec (LeaseDuration`) and creates another endpoint. But this is a fail-over scenario, ie there's aren't 2 leaders present simultaneously.

When nodes are restarted, there are disruption in api server connection and during this time, two leaders are present simultaneously and two local endpoints are created. Later one of them yields as it is not able to get the gateway lock.

Are you saying that there are 2 local endpoints present in the cluster at the same time? When the gateway acquires the leader lock, it first deletes any other endpoints before creating its own. Also it doesn't delete its endpoint if it loses the leader lock and/or exits. I'm trying to see a scenario where 2 local endpoints coexist simultaneously. The whole point of using leader election is to prevent 2 gateway controllers from running critical code simultaneously. If there's a hole in this process then I think we need to understand this. I'm wary of just trying to "band-aid" it in the route-agent.
If the API server isn't available then it shouldn't be able to acquire the leader lock nor create endpoints as those operations require access to the API server.
What could happen is if the first gateway acquires the lock and creates an endpoint then later loses connection to the API server and can't renew the lock after 5 sec (RenewDeadline) and exits. Then the second gateway acquires the lock after 10 sec (LeaseDuration) and creates another endpoint. But this is a fail-over scenario, ie there's aren't 2 leaders present simultaneously.
Can you provide a detailed step-by-step sequence of the scenario and any log messages?

This is the log from node where gateway got elected and later yielded
2023-10-17T10:23:08.621Z INF ..-source/app/main.go:429 main                 Using namespace "submariner-operator" for the leader election lock I1017 10:23:08.621704       1 leaderelection.go:245] attempting to acquire leader lease submariner-operator/submariner-gateway-lock... I1017 10:23:08.731886       1 leaderelection.go:255] successfully acquired lease submariner-operator/submariner-gateway-lock 2023-10-17T10:23:08.731Z DBG ..ols/record/event.go:298 main                 Event(v1.ObjectReference{Kind:"Lease", Namespace:"submariner-operator", Name:"submariner-gateway-lock", UID:"25695380-3b5e-4125-8a16-e5ea46cd37cc", APIVersion:"coordination.k8s.io/v1", ResourceVersion:"5520789", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' compute-1-submariner-gateway became leader 2023-10-17T10:23:08.732Z INF ..reswan/libreswan.go:127 libreswan            Using NATT UDP port 4500 2023-10-17T10:23:08.732Z INF ..gine/cableengine.go:104 CableEngine          CableEngine controller started, driver: "libreswan" 2023-10-17T10:23:08.732Z INF ..public_ip_watcher.go:62 Endpoint             Starting the public IP watcher. 2023-10-17T10:23:08.732Z INF ..r/datastoresyncer.go:73 DSSyncer             Starting the datastore syncer 2023-10-17T10:23:08.733Z INF ..ers/tunnel/tunnel.go:40 Tunnel               Starting the tunnel controller 2023-10-17T10:23:08.834Z INF ..er/healthchecker.go:110 HealthChecker        CableEngine HealthChecker started with PingInterval: 1, MaxPacketLossCount: 5 2023-10-17T10:23:08.835Z INF ..thchecker/pinger.go:106 HealthChecker        Starting pinger for IP "<IP_ADDRESS>" 2023-10-17T10:23:08.835Z INF ..er/healthchecker.go:159 HealthChecker        CableEngine HealthChecker started pinger for CableName: "submariner-cable-dnd-prsurve-dev-2-10-1-114-105" with HealthCheckIP "<IP_ADDRESS>" 2023-10-17T10:23:08.863Z INF ..ery/natdiscovery.go:162 NAT                  Starting NAT discovery for endpoint "submariner-cable-dnd-prsurve-dev-2-10-1-114-105" 2023-10-17T10:23:09.008Z INF .._update_federator.go:72 Federator            local -> broker: Updated Cluster "submariner-broker/dnd-prsurve-dev-1"  2023-10-17T10:23:09.032Z INF .._update_federator.go:72 Federator            broker -> local: Updated Cluster "submariner-operator/dnd-prsurve-dev-2"  2023-10-17T10:23:09.061Z INF ../datastoresyncer.go:241 DSSyncer             Ensuring we are the only endpoint active for this cluster 2023-10-17T10:23:09.062Z INF ../datastoresyncer.go:316 DSSyncer             Creating local submariner Cluster: types.SubmarinerCluster{ID:"dnd-prsurve-dev-1", Spec:v1.ClusterSpec{ClusterID:"dnd-prsurve-dev-1", ColorCodes:[]string{"blue"}, ServiceCIDR:[]string{"<IP_ADDRESS>/16"}, ClusterCIDR:[]string{"<IP_ADDRESS>/14"}, GlobalCIDR:[]string{"<IP_ADDRESS>/16"}}}  2023-10-17T10:23:09.095Z INF .._update_federator.go:72 Federator            broker -> local: Updated Cluster "submariner-operator/dnd-prsurve-dev-1"  2023-10-17T10:23:09.095Z INF ../datastoresyncer.go:329 DSSyncer             Creating local submariner Endpoint: types.SubmarinerEndpoint{Spec:v1.EndpointSpec{ClusterID:"dnd-prsurve-dev-1", CableName:"submariner-cable-dnd-prsurve-dev-1-10-1-114-39", HealthCheckIP:"", Hostname:"compute-1", Subnets:[]string{"<IP_ADDRESS>/16"}, PrivateIP:"<IP_ADDRESS>", PublicIP:"<IP_ADDRESS>", NATEnabled:false, Backend:"libreswan", BackendConfig:map[string]string{"natt-discovery-port":"4490", "preferred-server":"false", "udp-port":"4500"}}}  2023-10-17T10:23:09.103Z INF .._update_federator.go:72 Federator            local -> broker: Updated Endpoint "submariner-broker/dnd-prsurve-dev-1-submariner-cable-dnd-prsurve-dev-1-10-1-114-39"  2023-10-17T10:23:09.143Z INF .._update_federator.go:72 Federator            broker -> local: Updated Endpoint "submariner-operator/dnd-prsurve-dev-1-submariner-cable-dnd-prsurve-dev-1-10-1-114-39"  2023-10-17T10:23:09.158Z INF .._update_federator.go:72 Federator            broker -> local: Updated Endpoint "submariner-operator/dnd-prsurve-dev-2-submariner-cable-dnd-prsurve-dev-2-10-1-114-105"  2023-10-17T10:23:09.217Z INF .._update_federator.go:72 Federator            local -> broker: Updated Endpoint "submariner-broker/dnd-prsurve-dev-1-submariner-cable-dnd-prsurve-dev-1-10-1-114-39"  2023-10-17T10:23:09.285Z INF ../datastoresyncer.go:105 DSSyncer             Datastore syncer started 2023-10-17T10:23:09.288Z INF ..ncer/node_handler.go:73 DSSyncer             Updating the endpoint HealthCheckIP to globalIP "<IP_ADDRESS>" 2023-10-17T10:23:09.288Z INF ../datastoresyncer.go:329 DSSyncer             Creating local submariner Endpoint: types.SubmarinerEndpoint{Spec:v1.EndpointSpec{ClusterID:"dnd-prsurve-dev-1", CableName:"submariner-cable-dnd-prsurve-dev-1-10-1-114-39", HealthCheckIP:"<IP_ADDRESS>", Hostname:"compute-1", Subnets:[]string{"<IP_ADDRESS>/16"}, PrivateIP:"<IP_ADDRESS>", PublicIP:"<IP_ADDRESS>", NATEnabled:false, Backend:"libreswan", BackendConfig:map[string]string{"natt-discovery-port":"4490", "preferred-server":"false", "udp-port":"4500"}}}  2023-10-17T10:23:09.302Z DBG ..ry/request_handle.go:52 NAT                  Received request from <IP_ADDRESS>:4490 - REQUEST_NUMBER: 0x3f5a538c62273a99, SENDER: "submariner-cable-dnd-prsurve-dev-2-10-1-114-105", RECEIVER: "submariner-cable-dnd-prsurve-dev-1-10-1-114-39" 2023-10-17T10:23:09.303Z DBG ..y/request_handle.go:117 NAT                  Sending response to <IP_ADDRESS>:4490 - REQUEST_NUMBER: 0x3f5a538c62273a99, RESPONSE: OK, SENDER: "submariner-cable-dnd-prsurve-dev-1-10-1-114-39", RECEIVER: "submariner-cable-dnd-prsurve-dev-2-10-1-114-105" 2023-10-17T10:23:09.323Z INF .._update_federator.go:72 Federator            broker -> local: Updated Endpoint "submariner-operator/dnd-prsurve-dev-1-submariner-cable-dnd-prsurve-dev-1-10-1-114-39"  2023-10-17T10:23:09.375Z INF .._update_federator.go:72 Federator            local -> broker: Updated Endpoint "submariner-broker/dnd-prsurve-dev-1-submariner-cable-dnd-prsurve-dev-1-10-1-114-39"  2023-10-17T10:23:09.523Z DBG ..ery/request_send.go:115 NAT                  Sending request - REQUEST_NUMBER: 0xc089e199fd91e6c6, SENDER: "submariner-cable-dnd-prsurve-dev-1-10-1-114-39", RECEIVER: "submariner-cable-dnd-prsurve-dev-2-10-1-114-105", USING_SRC: <IP_ADDRESS>:4490, USING_DST: <IP_ADDRESS>:4490 2023-10-17T10:23:09.523Z DBG ..ery/request_send.go:115 NAT                  Sending request - REQUEST_NUMBER: 0xc089e199fd91e6c7, SENDER: "submariner-cable-dnd-prsurve-dev-1-10-1-114-39", RECEIVER: "submariner-cable-dnd-prsurve-dev-2-10-1-114-105", USING_SRC: <IP_ADDRESS>:4490, USING_DST: <IP_ADDRESS>:4490 2023-10-17T10:23:09.548Z DBG ..y/response_handle.go:31 NAT                  Received response from <IP_ADDRESS>:4490 - REQUEST_NUMBER: 0xc089e199fd91e6c6, RESPONSE: OK, SENDER: "submariner-cable-dnd-prsurve-dev-2-10-1-114-105", RECEIVER: "submariner-cable-dnd-prsurve-dev-1-10-1-114-39" 2023-10-17T10:23:09.548Z DBG ../remote_endpoint.go:184 NAT                  selected private IP "<IP_ADDRESS>" for endpoint "submariner-cable-dnd-prsurve-dev-2-10-1-114-105" 2023-10-17T10:23:09.548Z INF ..gine/cableengine.go:195 CableEngine          Installing Endpoint cable "submariner-cable-dnd-prsurve-dev-2-10-1-114-105" 2023-10-17T10:23:09.548Z INF ..reswan/libreswan.go:579 libreswan            Starting Pluto 002 listening for IKE messages 002 forgetting secrets 002 loading secrets from "/etc/ipsec.secrets" 002 loading secrets from "/etc/ipsec.d/submariner.secrets" 2023-10-17T10:23:10.159Z INF ..reswan/libreswan.go:371 libreswan            Creating connection(s) for {"metadata":{"name":"dnd-prsurve-dev-2-submariner-cable-dnd-prsurve-dev-2-10-1-114-105","namespace":"submariner-operator","uid":"ed71ddf8-3c47-4d21-ac4f-e94d3d0245b1","resourceVersion":"98223","generation":2,"creationTimestamp":"2023-10-13T11:14:34Z","labels":{"submariner-io/clusterID":"dnd-prsurve-dev-2"},"managedFields":[{"manager":"submariner-gateway","operation":"Update","apiVersion":"submariner.io/v1","time":"2023-10-13T11:14:35Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:labels":{".":{},"f:submariner-io/clusterID":{}}},"f:spec":{".":{},"f:backend":{},"f:backend_config":{".":{},"f:natt-discovery-port":{},"f:preferred-server":{},"f:udp-port":{}},"f:cable_name":{},"f:cluster_id":{},"f:healthCheckIP":{},"f:hostname":{},"f:nat_enabled":{},"f:private_ip":{},"f:public_ip":{},"f:subnets":{}}}}]},"spec":{"cluster_id":"dnd-prsurve-dev-2","cable_name":"submariner-cable-dnd-prsurve-dev-2-10-1-114-105","healthCheckIP":"<IP_ADDRESS>","hostname":"compute-0","subnets":["<IP_ADDRESS>/16"],"private_ip":"<IP_ADDRESS>","public_ip":"<IP_ADDRESS>","nat_enabled":false,"backend":"libreswan","backend_config":{"natt-discovery-port":"4490","preferred-server":"false","udp-port":"4500"}}} in bi-directional mode 2023-10-17T10:23:10.159Z INF ..reswan/libreswan.go:435 libreswan            Executing whack with args: [--psk --encrypt --name submariner-cable-dnd-prsurve-dev-2-10-1-114-105-0-0 --id <IP_ADDRESS> --host <IP_ADDRESS> --client <IP_ADDRESS>/16 --ikeport 4500 --to --id <IP_ADDRESS> --host <IP_ADDRESS> --client <IP_ADDRESS>/16 --ikeport 4500 --dpdaction=hold --dpddelay 30] 002 "submariner-cable-dnd-prsurve-dev-2-10-1-114-105-0-0": added IKEv2 connection 181 "submariner-cable-dnd-prsurve-dev-2-10-1-114-105-0-0" #1: initiating IKEv2 connection 2023-10-17T10:23:10.215Z INF ..gine/cableengine.go:202 CableEngine          Successfully ins
And this is the other cluster.
2023-10-17T13:00:53.875Z INF ..-source/app/main.go:411 main                 Gateway leader election config values: main.leaderConfig{LeaseDuration:10, RenewDeadline:5, RetryPeriod:2} 2023-10-17T13:00:53.875Z INF ..-source/app/main.go:429 main                 Using namespace "submariner-operator" for the leader election lock I1017 13:00:53.875555       1 leaderelection.go:245] attempting to acquire leader lease submariner-operator/submariner-gateway-lock... 2023-10-17T13:00:55.755Z DBG ..ry/request_handle.go:52 NAT                  Received request from <IP_ADDRESS>:4490 - REQUEST_NUMBER: 0xd9a9855a8b5d99d, SENDER: "submariner-cable-dnd-prsurve-dev-1-10-1-114-39", RECEIVER: "submariner-cable-dnd-prsurve-dev-2-10-1-114-75" 2023-10-17T13:00:55.755Z DBG ..y/request_handle.go:117 NAT                  Sending response to <IP_ADDRESS>:4490 - REQUEST_NUMBER: 0xd9a9855a8b5d99d, RESPONSE: OK, SENDER: "submariner-cable-dnd-prsurve-dev-2-10-1-114-75", RECEIVER: "submariner-cable-dnd-prsurve-dev-1-10-1-114-39" I1017 13:01:32.027737       1 leaderelection.go:255] successfully acquired lease submariner-operator/submariner-gateway-lock 2023-10-17T13:01:32.027Z DBG ..ols/record/event.go:298 main                 Event(v1.ObjectReference{Kind:"Lease", Namespace:"submariner-operator", Name:"submariner-gateway-lock", UID:"5f585cbc-cb75-40a8-9e55-f5b2cc30a62b", APIVersion:"coordination.k8s.io/v1", ResourceVersion:"5819855", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' compute-2-submariner-gateway became leader 2023-10-17T13:01:32.028Z INF ..reswan/libreswan.go:127 libreswan            Using NATT UDP port 4500 2023-10-17T13:01:32.030Z INF ..gine/cableengine.go:104 CableEngine          CableEngine controller started, driver: "libreswan" 2023-10-17T13:01:32.030Z INF ..public_ip_watcher.go:62 Endpoint             Starting the public IP watcher. 2023-10-17T13:01:32.030Z INF ..ers/tunnel/tunnel.go:40 Tunnel               Starting the tunnel controller 2023-10-17T13:01:32.031Z INF ..r/datastoresyncer.go:73 DSSyncer             Starting the datastore syncer 2023-10-17T13:01:32.132Z INF ..er/healthchecker.go:110 HealthChecker        CableEngine HealthChecker started with PingInterval: 1, MaxPacketLossCount: 5 2023-10-17T13:01:32.134Z INF ..thchecker/pinger.go:106 HealthChecker        Starting pinger for IP "<IP_ADDRESS>" 2023-10-17T13:01:32.135Z INF ..er/healthchecker.go:159 HealthChecker        CableEngine HealthChecker started pinger for CableName: "submariner-cable-dnd-prsurve-dev-1-10-1-114-39" with HealthCheckIP "<IP_ADDRESS>" 2023-10-17T13:01:32.165Z INF ..ery/natdiscovery.go:162 NAT                  Starting NAT discovery for endpoint "submariner-cable-dnd-prsurve-dev-1-10-1-114-39" 2023-10-17T13:01:32.365Z INF .._update_federator.go:72 Federator            local -> broker: Updated Cluster "submariner-broker/dnd-prsurve-dev-2"  2023-10-17T13:01:32.419Z INF ../datastoresyncer.go:241 DSSyncer             Ensuring we are the only endpoint active for this cluster 2023-10-17T13:01:32.420Z INF ../datastoresyncer.go:316 DSSyncer             Creating local submariner Cluster: types.SubmarinerCluster{ID:"dnd-prsurve-dev-2", Spec:v1.ClusterSpec{ClusterID:"dnd-prsurve-dev-2", ColorCodes:[]string{"blue"}, ServiceCIDR:[]string{"<IP_ADDRESS>/16"}, ClusterCIDR:[]string{"<IP_ADDRESS>/14"}, GlobalCIDR:[]string{"<IP_ADDRESS>/16"}}}  2023-10-17T13:01:32.431Z INF .._update_federator.go:72 Federator            broker -> local: Updated Cluster "submariner-operator/dnd-prsurve-dev-2"  2023-10-17T13:01:32.432Z INF ../datastoresyncer.go:329 DSSyncer             Creating local submariner Endpoint: types.SubmarinerEndpoint{Spec:v1.EndpointSpec{ClusterID:"dnd-prsurve-dev-2", CableName:"submariner-cable-dnd-prsurve-dev-2-10-1-114-75", HealthCheckIP:"", Hostname:"compute-2", Subnets:[]string{"<IP_ADDRESS>/16"}, PrivateIP:"<IP_ADDRESS>", PublicIP:"<IP_ADDRESS>", NATEnabled:false, Backend:"libreswan", BackendConfig:map[string]string{"natt-discovery-port":"4490", "preferred-server":"false", "udp-port":"4500"}}}  2023-10-17T13:01:32.446Z INF .._update_federator.go:72 Federator            broker -> local: Updated Endpoint "submariner-operator/dnd-prsurve-dev-2-submariner-cable-dnd-prsurve-dev-2-10-1-114-75"  2023-10-17T13:01:32.461Z INF .._update_federator.go:72 Federator            local -> broker: Updated Endpoint "submariner-broker/dnd-prsurve-dev-2-submariner-cable-dnd-prsurve-dev-2-10-1-114-75"  2023-10-17T13:01:32.511Z INF .._update_federator.go:72 Federator            local -> broker: Updated Endpoint "submariner-broker/dnd-prsurve-dev-2-submariner-cable-dnd-prsurve-dev-2-10-1-114-75"  2023-10-17T13:01:32.576Z INF ../datastoresyncer.go:105 DSSyncer             Datastore syncer started 2023-10-17T13:01:32.587Z INF ..ncer/node_handler.go:73 DSSyncer             Updating the endpoint HealthCheckIP to globalIP "<IP_ADDRESS>" 2023-10-17T13:01:32.588Z INF ../datastoresyncer.go:329 DSSyncer             Creating local submariner Endpoint: types.SubmarinerEndpoint{Spec:v1.EndpointSpec{ClusterID:"dnd-prsurve-dev-2", CableName:"submariner-cable-dnd-prsurve-dev-2-10-1-114-75", HealthCheckIP:"<IP_ADDRESS>", Hostname:"compute-2", Subnets:[]string{"<IP_ADDRESS>/16"}, PrivateIP:"<IP_ADDRESS>", PublicIP:"<IP_ADDRESS>", NATEnabled:false, Backend:"libreswan", BackendConfig:map[string]string{"natt-discovery-port":"4490", "preferred-server":"false", "udp-port":"4500"}}}  2023-10-17T13:01:32.627Z INF .._update_federator.go:72 Federator            broker -> local: Updated Endpoint "submariner-operator/dnd-prsurve-dev-2-submariner-cable-dnd-prsurve-dev-2-10-1-114-75"  2023-10-17T13:01:32.680Z INF .._update_federator.go:72 Federator            local -> broker: Updated Endpoint "submariner-broker/dnd-prsurve-dev-2-submariner-cable-dnd-prsurve-dev-2-10-1-114-75"  2023-10-17T13:01:32.777Z DBG ..ry/request_handle.go:52 NAT                  Received request from <IP_ADDRESS>:4490 - REQUEST_NUMBER: 0xd9a9855a8b5d99f, SENDER: "submariner-cable-dnd-prsurve-dev-1-10-1-114-39", RECEIVER: "submariner-cable-dnd-prsurve-dev-2-10-1-114-75" 2023-10-17T13:01:32.777Z DBG ..y/request_handle.go:117 NAT                  Sending response to <IP_ADDRESS>:4490 - REQUEST_NUMBER: 0xd9a9855a8b5d99f, RESPONSE: OK, SENDER: "submariner-cable-dnd-prsurve-dev-2-10-1-114-75", RECEIVER: "submariner-cable-dnd-prsurve-dev-1-10-1-114-39" 2023-10-17T13:01:32.869Z DBG ..ery/request_send.go:115 NAT                  Sending request - REQUEST_NUMBER: 0x13a6e38575ce4e52, SENDER: "submariner-cable-dnd-prsurve-dev-2-10-1-114-75", RECEIVER: "submariner-cable-dnd-prsurve-dev-1-10-1-114-39", USING_SRC: <IP_ADDRESS>:4490, USING_DST: <IP_ADDRESS>:4490 2023-10-17T13:01:32.869Z DBG ..ery/request_send.go:115 NAT                  Sending request - REQUEST_NUMBER: 0x13a6e38575ce4e53, SENDER: "submariner-cable-dnd-prsurve-dev-2-10-1-114-75", RECEIVER: "submariner-cable-dnd-prsurve-dev-1-10-1-114-39", USING_SRC: <IP_ADDRESS>:4490, USING_DST: <IP_ADDRESS>:4490 2023-10-17T13:01:32.894Z DBG ..y/response_handle.go:31 NAT                  Received response from <IP_ADDRESS>:4490 - REQUEST_NUMBER: 0x13a6e38575ce4e52, RESPONSE: OK, SENDER: "submariner-cable-dnd-prsurve-dev-1-10-1-114-39", RECEIVER: "submariner-cable-dnd-prsurve-dev-2-10-1-114-75" 2023-10-17T13:01:32.895Z DBG ../remote_endpoint.go:184 NAT                  selected private IP "<IP_ADDRESS>" for endpoint "submariner-cable-dnd-prsurve-dev-1-10-1-114-39" 2023-10-17T13:01:32.895Z INF ..gine/cableengine.go:195 CableEngine          Installing Endpoint cable "submariner-cable-dnd-prsurve-dev-1-10-1-114-39" 2023-10-17T13:01:32.895Z INF ..reswan/libreswan.go:579 libreswan            Starting Pluto 002 listening for IKE messages 002 Kernel supports NIC esp-hw-offload 002 adding UDP interface tun0 <IP_ADDRESS>:500 002 adding UDP interface tun0 <IP_ADDRESS>:4500 002 adding UDP interface ens192 <IP_ADDRESS>:500 002 adding UDP interface ens192 <IP_ADDRESS>:4500 002 adding UDP interface lo <IP_ADDRESS>:500 002 adding UDP interface lo <IP_ADDRESS>:4500 002 adding UDP interface lo [::1]:500 002 adding UDP interface lo [::1]:4500 002 loading secrets from "/etc/ipsec.secrets" 002 loading secrets from "/etc/ipsec.d/submariner.secrets" 2023-10-17T13:01:33.426Z INF ..reswan/libreswan.go:371 libreswan            Creating connection(s) for {"metadata":{"name":"dnd-prsurve-dev-1-submariner-cable-dnd-prsurve-dev-1-10-1-114-39","namespace":"submariner-operator","uid":"c7551a0d-e0e8-4fae-af5c-1a47f29fe5e4","resourceVersion":"5579847","generation":6,"creationTimestamp":"2023-10-16T13:40:38Z","labels":{"submariner-io/clusterID":"dnd-prsurve-dev-1"},"managedFields":[{"manager":"submariner-gateway","operation":"Update","apiVersion":"submariner.io/v1","time":"2023-10-17T10:23:45Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:labels":{".":{},"f:submariner-io/clusterID":{}}},"f:spec":{".":{},"f:backend":{},"f:backend_config":{".":{},"f:natt-discovery-port":{},"f:preferred-server":{},"f:udp-port":{}},"f:cable_name":{},"f:cluster_id":{},"f:healthCheckIP":{},"f:hostname":{},"f:nat_enabled":{},"f:private_ip":{},"f:public_ip":{},"f:subnets":{}}}}]},"spec":{"cluster_id":"dnd-prsurve-dev-1","cable_name":"submariner-cable-dnd-prsurve-dev-1-10-1-114-39","healthCheckIP":"<IP_ADDRESS>","hostname":"compute-1","subnets":["<IP_ADDRESS>/16"],"private_ip":"<IP_ADDRESS>","public_ip":"<IP_ADDRESS>","nat_enabled":false,"backend":"libreswan","backend_config":{"natt-discovery-port":"4490","preferred-server":"false","udp-port":"4500"}}} in bi-directional mode 2023-10-17T13:01:33.426Z INF ..reswan/libreswan.go:435 libreswan            Executing whack with args: [--psk --encrypt --name submariner-cable-dnd-prsurve-dev-1-10-1-114-39-0-0 --id <IP_ADDRESS> --host <IP_ADDRESS> --client <IP_ADDRESS>/16 --ikeport 4500 --to --id <IP_ADDRESS> --host <IP_ADDRESS> --client <IP_ADDRESS>/16 --ikeport 4500 --dpdaction=hold --dpddelay 30] 002 "submariner-cable-dnd-prsurve-dev-1-10-1-114-39-0-0": added IKEv2 connection 181 "submariner-cable-dnd-prsurve-dev-1-10-1-114-39-0-0" #1: initiating IKEv2 connection 2023-10-17T13:01:33.488Z INF ..gine/cableengine.go:202 CableEngine          Successfully installed Endpoint cable "submariner-cable-dnd-prsurve-dev-1-10-1-114-39" with remote IP <IP_ADDRESS> 2023-10-17T13:01:33.527Z DBG ..reswan/libreswan.go:255 libreswan            Connection "submariner-cable-dnd-prsurve-dev-1-10-1-114-39-0-0" not found in active connections obtained from whack: map[], map[] 2023-10-17T13:01:33.527Z DBG ..reswan/libreswan.go:269 libreswan            Connection "submariner-cable-dnd-prsurve-dev-1-10-1-114-39" not found in active connections obtained from whack: map[], map[] 2023-10-17T13:01:34.997Z INF .._update_federator.go:72 Federator            broker -> local: Updated Endpoint "submariner-operator/dnd-prsurve-dev-1-submariner-cable-dnd-prsurve-dev-1-10-1-114-39"  2023-10-17T13:01:35.008Z INF .._update_federator.go:72 Federator            broker -> local: Updated Cluster "submariner-operator/dnd-prsurve-dev-1"  2023-10-17T13:01:38.135Z ERR ..thchecker/pinger.go:161 HealthChecker        Failed to successfully ping the remote endpoint IP "<IP_ADDRESS>" error="more than 5 packets lost" 2023-10-17T13:01:38.173Z INF ..thchecker/pinger.go:177 HealthChecker        Ping to remote endpoint IP "<IP_ADDRESS>" is successful

I see where cluster "dnd-prsurve-dev-1" acquired the leader lock
 I1017 10:23:08.731886       1 leaderelection.go:255] successfully acquired lease submariner-operator/submariner-gateway-lock
But it didn't lose the lock, at least not in that log snippet. How many gateways were configured on cluster "dnd-prsurve-dev-1"?

This is related to https://issues.redhat.com/browse/ACM-8274. There were 3 gateways on cluster 2. gateway-f6pm7 got the leader lock first, created EP <IP_ADDRESS> and then lost the lock:
I1017 13:00:12.587547    1 leaderelection.go:255] successfully acquired lease submariner-operator/submariner-gateway-lock
E1017 13:00:26.130636    1 leaderelection.go:327] error retrieving resource lock submariner-operator/submariner-gateway-lock: Get “[https://<IP_ADDRESS>:443/apis/coordination.k8s.io/v1/namespaces/submariner-operator/leases/submariner-gateway-lock](https://<IP_ADDRESS>/apis/coordination.k8s.io/v1/namespaces/submariner-operator/leases/submariner-gateway-lock)”: context deadline exceeded

Then gateway-8647c got the leader lock and deleted EP <IP_ADDRESS> and created EP <IP_ADDRESS>:
I1017 13:00:30.961641    1 leaderelection.go:255] successfully acquired lease submariner-operator/submariner-gateway-lock
[90m2023-10-17T13:00:31.292Z[0m [32mINF[0m ../datastoresyncer.go:265 DSSyncer             Successfully deleted existing submariner Endpoint “dnd-prsurve-dev-2-submariner-cable-dnd-prsurve-dev-2-10-1-114-75”
[90m2023-10-17T13:00:31.311Z[0m [32mINF[0m ../datastoresyncer.go:329 DSSyncer             Creating local submariner Endpoint: types.SubmarinerEndpoint{Spec:v1.EndpointSpec{ClusterID:“dnd-prsurve-dev-2", CableName:“submariner-cable-dnd-prsurve-dev-2-10-1-114-99”, HealthCheckIP:“”, Hostname:“compute-1”, Subnets:[]string{“<IP_ADDRESS>/16"}, PrivateIP:“<IP_ADDRESS>”, PublicIP:“<IP_ADDRESS>", NATEnabled:false, Backend:“libreswan”, BackendConfig:map[string]string{“natt-discovery-port”:“4490”, “preferred-server”:“false”, “udp-port”:“4500”}}}

But then gateway-f6pm7 recreated EP <IP_ADDRESS> after it lost leadership and before it exited:
[90m2023-10-17T13:00:32.610Z[0m [32mINF[0m ../datastoresyncer.go:329 DSSyncer             Creating local submariner Endpoint: types.SubmarinerEndpoint{Spec:v1.EndpointSpec{ClusterID:“dnd-prsurve-dev-2”, CableName:“submariner-cable-dnd-prsurve-dev-2-10-1-114-75", HealthCheckIP:“<IP_ADDRESS>”, Hostname:“compute-2", Subnets:[]string{“<IP_ADDRESS>/16”}, PrivateIP:“<IP_ADDRESS>", PublicIP:“<IP_ADDRESS>”, NATEnabled:false, Backend:“libreswan”, BackendConfig:map[string]string{“natt-discovery-port”:“4490", “preferred-server”:“false”, “udp-port”:“4500"}}}
[9:38](https://redhat-internal.slack.com/archives/D052PTTFS0P/p1698370689903849)
[90m2023-10-17T13:00:33.054Z[0m [1m[31mFTL[0m[0m ..al/pkg/log/logger.go:67 main                 Leader election lost, shutting down

That’s odd. After leader election was lost it should’ve exited quickly but did not for 7 sec. And it should not have recreated the EP.

I'll push a PR to fix the EP create/update issue.

That’s odd. After leader election was lost it should’ve exited quickly but did not for 7 sec. And it should not have recreated the EP.

Right, this is exactly what was happening.

It does a create or update operation but we just want update in this case.

AFAIR, we were getting update event itself for the endpoint?

AFAIR, we were getting update event itself for the endpoint?

Not sure I follow...

That’s odd. After leader election was lost it should’ve exited quickly but did not for 7 sec. And it should not have recreated the EP.

Right, this is exactly what was happening.

It does a create or update operation but we just want update in this case.

AFAIR, we were getting update event itself for the endpoint?

I think since we call create or update, when the new gateway node comes up and try to create the endpoint , it comes as an update event, since the endpoint already exists.

AFAIR, we were getting update event itself for the endpoint?

Not sure I follow...

Saw your PR, its clear now.


Fix config file parsing

Remove serde-hex since it's no longer compatible with the new toml version
Include the hex crate
Add functions for serializing and deserializing
Add test for serialization and deserialization methods

Not sure if this is the best way, but the file needed to be backward compatible.

maybe you also want to adapt test_parse_u16_hex

maybe you also want to adapt test_parse_u16_hex

Ups, forgot to push the changes after updating it. Thanks!


Added Users to TableQueries

Describe your changes :

Worked on adding users to tableQueries

Type of change :

[x] Improvement

Frontend Preview (Screenshots) :
For frontend related change, please link screenshots of your changes preview! Optional for backend related changes.

Checklist:

[x] I have read the CONTRIBUTING document.
[x] I have performed a self-review of my own.
[x] I have tagged my reviewers below.
[ ] I have commented on my code, particularly in hard-to-understand areas.
[ ] My changes generate no new warnings.
[ ] I have added tests that prove my fix is effective or that my feature works.
[x] All new and existing tests passed.

Reviewers

@harshach @Sachin-chaurasiya @darth-coder00

@pmbrull can you check if this is the right approach? I have modified the code according to the comments

Also, do not hesitate to reach out if there are any doubts about the code or if you think something can be improved, always happy to discuss :)

Flagged the security hotspot as safe, as only affects sample data. No further action needed


Update openra to 20180218

After making all changes to the cask:

[x] brew cask audit --download {{cask_file}} is error-free.
[x] brew cask style --fix {{cask_file}} left no offenses.
[x] The commit message includes the cask’s name and version.

@EricFromCanada Please remove the extra commit.

It seems the script doesn't first check if you're on the master branch…


Allow StructureSpecific in XML writer

If I'm understanding the DataMessage section in the XML writer code it will always produce "Generic" output. Would it be possible to allow "StructureSpecific" as an alternative?
If so, and you can summarize what needs to happen, I would be happy to submit a pull request. I tried locally just by changing "Generic" to "StructureSpecific" in the line linked above, but that does not alter the inner structure. For example, the output still has "SeriesKey" elements, which to my knowledge is not in StructureSpecific.

Certainly possible and a desirable enhancement, but non-trivial.
The summary: the "inner structure" is generated by the rest of the code in the function you linked to. Every call to writer.recurse() within that function results in a call to a different method in the same file. For instance:
https://github.com/khaeru/sdmx/blob/3e4cbc1bed655151b7ef19a3ed7fc3ecf311c0b4/sdmx/writer/xml.py#L130-L131
In this case, ds is an instance of model.DataSet. So the writer finds another method that accepts this type of object:
https://github.com/khaeru/sdmx/blob/3e4cbc1bed655151b7ef19a3ed7fc3ecf311c0b4/sdmx/writer/xml.py#L541-L542
…and calls that, which in turn recurses for the constituent classes like SeriesKey, Observation, etc. Each method returns lxml Element objects, that are assembled (e.g. elem.append()) into the complete XML document.
In order to output structure-specific SDMX-ML, every such method would need to be extended to (a) check whether generic or structure-specific output is to be produced and (b) if the latter, generate the XML elements and attributes that the standard requires for structure-specific data.
(The experiment you mention only changed the name of one particular generated XML element—because it did not alter any of the other code that generates the elements and attributes, the output does not change.)
A PR for this would also need appropriate extensions to the test suite.

@khaeru Thanks, the makes sense. Do you think the Generic and StructureSpecific are pretty much the entirety of the possible formats? Or should this be done in a way that allows for other formats in the future?

Coding a standards-compliant implementation will require careful attention to the actual standards documents, which will answer that and other questions 🙂 The SDMX website seems to be down at this moment (?!), but here's an archived version: https://web.archive.org/web/20200912164322/https://sdmx.org/?page_id=5008 —§3A is the one you want to consult.
For the moment, if each of the affected functions in sdmx.writer.xml contains an if/else block with just 2 clauses, that's fine from my point of view. If it became a performance concern (cf. #49) or the code ends up being too complex, then we could try a pattern of using one set of functions for generic data but a distinct set of functions for structure-specific data. Let's cross that bridge if we come to it.


add custom auth

Firebase's custom authentication plugins use the secret of the Firebase app in question to generate custom tokens. Exposing a Firebase's secret in a client-side application means that it becomes discoverable by anyone why uses that application. And when someone has the secret of your Firebase, they have unrestricted access to your Firebase database. That is why you should never generate custom tokens in a client-side application.


People get stuck when respawning on panel B on Crash Site in fastball

like caused by https://github.com/R2Northstar/NorthstarMods/commit/050e707f4762b976791c0934b88779e038f70ade
When people respawn at panel B on Crash Site, they get stuck in place without being able to move. This happened on my Fastball Randomiser server which is using the newest version of _gamemode_fastball.gnut

Screenshots

Should be fixed with #171

#171 has been merged so this issue should be fixed for now.


Add support for tolerations

The django-initializer pod had no support for setting tolerations previously. This PR should fix that and add the ability to specify tolerations right in the values.yml file,  such as:
[...]
initializer:
  affinity: ...
  nodeSelector: ...
  tolerations: ...

[...]

Thanks, can you reggae against the 'dev' branch please?

thanks, but seems to contain some phantom commits :-)

@valentijnscholten I am unsure how those commits sneaked up in there... Is it because i made my changes on the master branch and i am trying to merge into dev?
How should i go about removing them from this PR?

easier to make a new PR

Fair enough. Will close this one and make new PR.


[Feature] Add gradient and mario themed tooltip component

Description
Two new tooltip components in the library

Gradient Tooltip
Mario themed tooltip

Screenshots

Checklist

[X] I have checked the existing issues.
[X] I follow Contributing Guidelines of this project.
[X] I am a GSSOC'24 contributor.
[ ] I am a IWOC'24 contributor.
[ ] I am a JWOC'24 contributor.
[ ] I am a SWOC'24 contributor.

Code of Conduct

[X] I agree to follow this project's "Code of Conduct".

@Rakesh9100 Please assign me this issue.

@Rakesh9100 Please assign me this issue.

@hars-21 go forward
I am assigning u
As admin is busy for some days
Do your task and provide your PR asap.
Then admin merge it with the proper labels .

@Rakesh9100 Please assign this issue to me! @saurabhsingh720 @

@hars-21 go ahead bro

Ok

this is issue is solved right?

No, I am working on it.


increase map limit

Amazed I didn't hit this earlier:
# grep 128 src/map.cpp
  int max_entries = 128;

# bpftrace -e 'profile:hz:999 { @[cpu, nsecs] = count(); } interval:s:1 { exit() }' | wc -l
    131

That should be way more.

This is a dup of #464 which was categorised as an enhancement - obviously it's a very serious usability issue.

ok thanks, I want to change one of these bug titles to capture the follow-up work: we need to warn when the map key limit is reached.

I think just do it under which ever one you want and close the other one. The important thing is that you're fixing it which is great!


Failed to send sensor data while connecting to Golioth

Due to the configuration setting CONFIG_LTE_AUTO_INIT_AND_CONNECT=n, it's possible that the firmware can start taking sensor readings before the device gets a chance to connect to Golioth. When it tries to send these sensor readings to Golioth, the following error occurs:
[00:00:23.665,557] <err> app_work: Failed to send sensor data to Golioth: -115

Similarly, it's possible the firmware will try to write to LightDB state before the client is connected, producing the following error:
[00:00:22.586,975] <err> app_state: Unable to write to LightDB State: -115

Ideally we would check that the client is connected before trying to send data to Golioth, but for now there doesn't appear to be a good way to check that the client is connected in the Zephyr SDK.
Per slack convo with @szczys:

We should just let it try to send and have the errors thrown in the terminal. There is really no harm to this, we want things running so that the faceplate is getting sensor readings at power-up and doesn't need to wait for  the network.
AFAIK there isn't a great way to test for a golioth connection in the Zephyr repo, but the Firmware SDK repo has that and we will soon be merging the Zephyr port into that so this should be a short-term issue.

This was resolved in https://github.com/golioth/reference-design-air-quality/commit/11a4b625e4296230b869893cef077876dabd9736


‘fClient’ was not declared in this scope

Compiling denarius-qt v<IP_ADDRESS> in Debian produced the following errors:

src/txdb-bdb.cpp:36:13: error: ‘fClient’ was not declared in this scope
assert(!fClient);

src/txdb-bdb.cpp:226:49: error: ‘GetArgInt’ was not declared in this scope
int nCheckLevel = GetArgInt("-checklevel", 1);

src/txdb-bdb.cpp:404:24: error: ‘class CBlockIndex’ has no member named ‘hashProofOfStake’
pindexNew->hashProofOfStake = diskindex.hashProofOfStake;

I had no problem compiling the earlier version <IP_ADDRESS>.

For v<IP_ADDRESS> If you are compiling the QT you must add the flag USE_LEVELDB=1 to the qmake command, like the below. This is fixed in the future v<IP_ADDRESS>, it will compile in the future by default with the TXDB LevelDB.
qmake "USE_LEVELDB=1" "USE_UPNP=1" "USE_QRCODE=1" denarius-qt.pro


https://github.com/YiqiDh/DllInjector互赞

https://github.com/YiqiDh/DllInjector
回复必赞

https://github.com/theajack/cnchar
已star

https://github.com/theajack/cnchar
已star
已star

https://github.com/theajack/cnchar
已star

已

https://github.com/theajack/cnchar
已star

已star


how does pos ethereum prevent bribery for late block proposal?

TLDR: There exist a financial incentive for the execution payload to be proposed later than the anticipated time window for it. We worry that this might lead to favoring larger validators, encourage attester bribing infrastructure and might contribute to making consensus attacks described in recent papers more likely.
Context:
The beacon chain has slots. A slot is currently every 12s. Within this slot exists an execution chain block as well as consensus objects. While there is theoretically (12s-epsilon) time to put the beacon block together within the slot, there are actually sub-slots within this slot that are softly enforced by consensus clients. These sub-slots are necessary in order to account for propagation time (incl. network delays) to attesters and for the proper construction of a beacon block.
Afaik - an execution chain block is supposed to be proposed in the first 4s of the 12s slot. This gives 8s for it to be voted on by attesters and in MEV-Boost, leaves enough for the body to be revealed by the end of the slot. For eg. see waitToOneThirdOrValidBlock() in Prysm's client here: https://github.com/prysmaticlabs/prysm/blob/8c8f1bb9c18ebbc42241848a328ed275b12587e5/validator/client/attest.go#L250 (thanks @terencechain )
Worry 1: Late execution block proposals:
In theory, the proposer has until 12s-epsilon to submit a beacon block. This means the execution chain block could be constructed maybe at 11s rather than 4s in. This time difference means the builder would have an additional 7s to construct a block. In those 7s, new transactions could’ve come in and made the block more profitable.
The proposer and builders now have an incentive to delay the block submission as much as possible.
This is problematic because:

a proposer that controls a lot of validators can inherently force them to vote later (by modifying their consensus client source codes) vs a solo validator. This means there is an advantage to controlling a lot of validators that are well-connected to the p2p network.
another flavor of this problem is that there might exist an incentive to bribe attesters in order to delay their attestations.

Questions:

on average, how much more profitable are blocks after X seconds from Flashbots data? This will help us understand how big of an advantage would larger validators get.
what is the technical complexity of either bribing attesters and making sure the attesters attest later than expected?
what is the financial risk encountered by doing this (later attestations get less rewards + there is a risk of missing the slot afaik which would incur a slashing penalty)? can we surface and calculate the expected value of doing this?
Given this information, how expensive would it be to bribe attesters?

The good news is that we have numbers to quantify this, in particular, Flashbots finds that on average, each additional second of waiting is worth 0.034 eth in additional miner payment (incl. 1559 tips). The outlier numbers we have (top 0.1% of blocks excluding tips - only bundle payments) are around 1 eth/s (thx @metachris and @Ruteri ). These numbers generally seem too small for this worry to be relevant.
The bad news is that this quantification is imperfect. In particular, if someone can delay their ultimate proposal to a few seconds later, they are able to do cross-domain arbitrage with low-latency domains more effectively. Priced in, this might be worth large sums of money.
To-do:

[ ] poll relevant parties (searchers who run these type of strategies) to understand what this could amount to for them.

Worry 2: Reorgs
Two papers (https://arxiv.org/pdf/2203.01315.pdf and https://eprint.iacr.org/2021/1413.pdf) were recently released detailing 5 attacks on PoS consensus (co-authored by @casparschwa @barnabemonnot et.al). One attack in particular relies on the timing of a block proposal.
We worry wether the financial incentives outlined above exert an upwards pressure on the likeliness of such a timing attack happening in the wild. I understand these attacks have been fixed and suspect this worry can be easily alleviated by @casparschwa sharing how these attacks were fixed.
We open the discussion here and look forward to your questions and comments :)!

Worry 1: Late execution block proposals:
Thanks for bringing up this issue on timeliness of block proposals. In short, yes it's a real problem. However, it's a known problem for which we have a solution in mind already. The solution is (block, slot)-voting, it just needs some details figured out. In other words, this is a known issue with the protocol and active exploit of this issue will lead to accelerated mitigation.
In this HackMD you'll find a more detailed write-up of the problem, what this means for proposers/builders and what solutions could look like (there are more ideas!). Here's the tldr:

A block proposer can release as late as 11s into the slot and still end up canonical. The only constraint is that the next block proposer hears the late block so that they build on top of it (as opposed to the late block's parent).
However, if a proposer chooses to release very late virtually no attesters will vote for their late block (because they will have attested already). This opens the late block up to reorg attacks. The next proposer can use something called Proposer LMD Score Boosting (essentially as if some validators had already attested to this block) to reorg out a preceeding that has less votes accumulated than the size of the boost. This effectively introduces a soft-cap for the block releasing deadline around the 4s mark (which is the time when hoenst validators attest, if they havent heard a valid block already).
Both builders and proposers are interested in colluding since more tx mempool listening time implies more MEV, which is good for both parties involved. The questions is whether they can collude without trusting each other? And I think the answer is yes: Builders can simply send new bids as time goes by and the proposer just picks the best whenever they decide to finally build and release their block. This is good in so far that there is no advantage to being a large, reputable staker (builders trust you) vs being a solo staker (builders don’t trust you).

Worry 2: Reorgs
The three attacks mentioned in our paper are fixed by the aforementioned Proposer LMD Score Boosting. One of the attacks in the other paper by Neu et al. is not an attack on PoS Ethereum. The other one is fixed by removing equiovcating validators from fork choice consideration. I am happy to go into more detail on this here or elsewhere, but didn't want to go beyond scope in this comment.
Some other comments
There is actually one more "sub-slot":

At the beginning of the slot a block proposer is supposed to propose a block.
Committee members are supposed to attest when they hear a valid block or 4s into the slot -- whatever comes first.
Aggregators are supposed to release their attestation aggregates 8s into slot.

This means that if the aggregators don't hear your attestations some time before the 8s mark they will likely not include them, meaning you're incentivized to release them before the 8s mark (if you want them to be included asap).

A proposer that controls a lot of validators can inherently force them to vote later (by modifying their consensus client source codes) vs a solo validator. This means there is an advantage to controlling a lot of validators that are well-connected to the p2p network.

Both solo and multi-validators can modify the source code. It is not clear to me how there is an advantage only available to one or the other. Controlling lots of validators does not mean you control lots of nodes on the p2p layer. You can, but typically you'll use the same beacon node that handles the p2p stuff. And again, both a solo- and multi-validator could run multiple p2p nodes.
One advantage that someone controlling many validators has: They are more likely to also control an aggregator in the relevant (committee, slot)-pair in question. This means a large enough adversary could potentially circumvent the 8s deadline without foregoing inclusion rewards.

And finally, one question for you: Rather than an average value for what each additional second of waiting is worth, do you have data on the distribution of this?
I wanted to post this already, but I'll think more about some of the other questions mentioned. Let me know, if there are follow-up questions etc.

However, if a proposer chooses to release very late virtually no attesters will vote for their late block (because they will have attested already). This opens the late block up to reorg attacks.

To expand on this.
If proposer of slot N is regularly willing to be "dishonest" (deviating from spec and normal software) and wait well past the 4s mark, then it is reasonable to believe the proposer of slot N+1 will (also be dishonest) utilize their proposer boost to re-org that proposer (by building on N's parent) and to steal MEV from proposer at slot N.
So it is not actually clear that waiting near or past the 4s mark is a viable strategy to get more MEV due to this strategy being attackable by the next slot proposer.

Beyond that. A general note here on this security issue. Mentioned by @casparschwa but worth being very clear what's at stake here.
Ethereum PoS has a number of known upgrades in the pipeline. Some are feature enhancements (e.g. sharded data) while some are security enhancements (e.g. proof of execution, DAS, etc). While we (researchers, developers, community) would like to prioritize shipping features, if known security issues become active targets, security enhancements will have to be prioritized
Manipulating the release of blocks between 4 and 11s in a slot is a known degenerate case with a proposed fix (called (block, slot) fork choice). This is explicitly against the "honest" spec and would (1) fairly attributable and (2) seen as nefarious behavior. In the event that flashbots or some other mechanism begins to exploit this issue to shift the timing equillibrium, it will accelerate the priority of shipping (block, slot) at the expense of important (desired?) scalability features.

The good news is that we have numbers to quantify this, in particular, Flashbots finds that on average, each additional second of waiting is worth 0.034 eth

I'm curious about how much the honest block that builds on top of the late block would degrade and whether that's something we can quantify. For example, say at slot N, a dishonest proposer releases block at 11s and gains X more MEV. Would it be fair to estimate that at slot N+1, the honest proposer releases block at 0s and loses X MEV that it would have gained? Assuming both proposers use mev-boost and connect to different relays or same relay (would that make a big difference?)

There's also an active prototype by the lighthouse team to combat this: https://github.com/sigp/lighthouse/pull/2860
If it goes well, it'll likely land itself into the honest validator spec and all clients will implement it.

Hey everyone! Thanks a lot for you comments here, and thanks in particular to @casparschwa and @fradamt for the conversations offline and the detailed answers above. They've been critical in writing this reply!
TLDR:

there are some mitigations for late block proposals including Proposer LMD score boosting, and in the future (block,slot) voting.

however none of them protect against a degenerate case where the network without needing coordination pushes block proposal and attestation times further and further.
this is an issue to further study, as well as to potentially monitor for by setting up infrastructure to measure time at which blocks are proposed within a slot and to call out any misbehavior publicly.

other tangential research questions to study are: retroactive rewards for timely block proposals, quantifying how much money can be made from delaying block proposals, mev oracles as a scaling factor to help retroactive timely block proposal rewards and whether deterministic block times are a fundamental flaw of PoS Ethereum.

One misunderstanding corrected privately that I'll relay here is an attestation of slot n can only get included in the block from slot n+1 (the earliest).
You'll find below further details on all of the above:
Mitigations
Currently, there is no network-wide mitigation for late block proposals that's implemented and all a proposer has to do is to make sure that the next proposer hears their block (block n) by the time they start constructing their block (block n+1). This means proposers today can propose a block very late in their slot.
One immediate mitigation that's implemented in some clients is Proposer LMD score boosting. This was implemented for another issue than late block proposals but turns out to be helpful for our issue too. In a nutshell, this would allow the next proposer to re-org the previous one out if they proposed their block too late.
How does it work?
If you don't accumulate any fork choice weight because you’re releasing your block so late that the entire attester committee doesn’t see it in time, then you’re prone to be reorged out by the next proposer. If you want to be reorg-resistant then you need to release it in such a way that your late block accumulates at least PROPOSER_BOOST votes, which brings you back to around a 4s deadline. One outstanding question here is how to set the boost value correctly. Read more about it here
An advanced version of that mitigation that avoids relying on the proposer but instead on the attester committee is (block,slot) voting. This currently has a rough spec but if i understand correctly needs more work before being implemented. In (block,slot) voting, committee members can vote on an empty slot (which they weren't able to previously), meaning the empty slot now accumulates fork choice weight making it harder for a late block to become canonical. This enshrines the 4s rule in-protocol, without needing proposer LMD score boost.
Finally, there are other mitigation discussions at an experimental stage like retroactive timely rewards for block proposers, effectively changing the components of the block proposer reward to include timeliness. One problem with this is that it is only effective if this reward is higher than the reward from a proposer stands to make from 'late MEV' which makes it problematic since MEV rewards can shift while this parameters may be static.
You can learn more about these mitigations in this document from @casparschwa here: (Un)-Timeliness in PoS Ethereum
Some additional comments:

On @casparschwa 's point about the potentiality of relays enforcing proposal deadlines on builders: I agree with your intuition. It would make such relays less competitive. We've already seen similar dynamics happen today among different relays on PoW Ethereum.

On @terencechain's point, and also @djrtwo's point: absolutely!

if this is a network-wide behavior, large players (large wrt to stake ownership) would be stealing from one another.
this is easily detectible, the one who is constantly spoiling the pot should be able to get told off.
on the other hand, if it's a solo validator it's totally fine because they're a proposer so rarely that such a behaviour wouldn't have a material impact unless it becomes endemic.

how would you be able to detect this behavior?

you could detect it by the value of the proposer's MEV and the content of their block. if their block includes txs at the 4s mark and beyond then you know they were still constructing it then. this would require a node setup to benchmark when txs are received in the mempool and some statistical analysis.

one heuristic that can work to mitigate edge cases of this first one would be that if it's a big validator who's constantly late, it's worth asking why.

meta-question: should monitoring infrastructure be set-up for this/should this be part of the monitoring infrastructure on the network? (tagging @elopio, @epheph and @taarushv here)

So, can we close this issue?
I don't think so! There is one oustanding concern worth discussing:
Even if proposer-boost is implemented network-wide, and even with (block,slot) voting, there is still an incentive for proposers to propose as late as they can within the boundaries given to them (ie. as close as possible to the 4s mark).
Because the block would be proposed around the 4s deadline, some attesters would likely miss it as they vote on the latest head they've seen. This would affect their total rewards since voting on timely head gives 14/64 of total validator rewards (so roughly 10% of their reward).
The ideal strategy for attesters is that if they're protected from 'proposer_boost', they might as well wait 0.5s later to make sure they're in the group that sees the proposer slot. The future proposers now might push their proposal to 4.5s rather than 4s. You can imagine this game being iterated, pushing block proposal deadlines further and further without any network-wide coordination needed. (Read more about this in here by @fradamt https://notes.ethereum.org/HO8UXG4rSdmH4Sv16iWKuw?view#Shifting-timing-equilbria-of-block-releasing-amp-attestations )
There is more complexity to this problem to think about such as:

if the network doesn't move in unison when they push this limit, it becomes quite complex to think about attestations and things like timing attacks.
there are more network-dynamics to consider, like:

solo validators having to monitor the network to figure out that this behavior is happening,
solo validators potentially not being able to catch up because they would constantly have to download a client that's incentive compatible (ie. with updated deadlines)
the advantage of large validators who would likely control an attestation aggregator and are able to play around the aggregator deadline that's at 8s in the slot which solo validators wouldn't be able to

If the network does move in unison doing this behavior, should we expect a drift over time of block proposals and attestations?
So where does that leave us?

There are multiple solutions to mitigate late block proposals. Yay!
This behavior is detectible and can be addressed publicly, where players doing this can be called out.
However, none of these solutions are silver bullets and there is still an underexplored worrisome degenerate case.

Further research
Quantifying MEV:
Having a better idea of the amount of MEV players stand to make would allow us to think about these mitigation solutions better. In particular it would help us think about the usefulness of retroactive timely rewards for block proposals. Generally, estimating what isn't in the numbers which I mentioned above would be helpful although seemingly difficult to achieve. (ie. quantifying cross-domain MEV for later block proposals)
Understanding the degenerate case better:
A bunch of questions were listed above as part of understanding the dynamics of the iterated game better.
It's also worth asking the question, why is it bad if blocks are proposed later? I'm asking this question because I'd like to be able to point to something written to explain why a validator/validator pool who's acting under economically rational assumption shouldn't do this. From all our conversation above, the answer seems to be:

it would significantly complicate reasoning about balancing attacks.
it is likely useless if the network does it in unison (it just delays slots in perpetuity)
it may enforce higher hardware & software requirements on validators and less tolerance on network delays and propagation. This means the network becomes less accessible to players that don't have the best connections or professional set-ups. This goes against the ethos of the system we're building.
it is unwanted behavior that if noticed, will become a higher priority to solve and as @djrtwo mentioned, will be prioritized against efforts that stand to make Ethereum better by adding new features to it.

Against deterministic block times?
I wonder if this is simply a fundamental flaw of chains with deterministic block times, and that current PoW Ethereum's variable block times is actually better for decentralization. If you start from the assumption that users will always want to push the limit of how far they can wait to propose a block because of economic incentives, variable block times introduce uncertainty which was helpful to prevent this behavior. While I don't have more thoughts here at the moment, I wonder if ethereum will be the first big test of a sophisticated (wrt to MEV activity) PoS network with deterministic block times and whether there may be conclusions to draw on the experiment. I also know of efforts like Algorand's PoS chain that mimics PoW-like probabilistic leader election that may be worth studying further.

We are starting data collection for mev rewards around our relay in https://github.com/flashbots/flashbots-data-transparency/issues/9 and https://github.com/flashbots/flashbots-data-transparency/issues/6


resizeListener function binding

Currently the resizeListener you define in your view model must be a function variable.
resizeListener = function(width, height) {
    console.log("width=" + width);
    console.log("height=" + height);
  }

Unfortunately, if you make it a member function:
resizeListener (width, height) {
    console.log("width=" + width);
    console.log("height=" + height);
  }

when it is called it has the incorrect context (not the view-model).
I don't really know how to fix this, so suggestions welcome.

can now use aurelia's own event binding to call any method without it needing to encapsulate this


Fixing the docs

This PR:

Fixes many bugs in the docs (especially in the utilities)
Fixes the math display in the 4th notebook
Adds the flag -W to the doc tests, which should fail if any warning is raised

This PR addresses
#111 #106

@vishal-ph, the PR is ready for review!


GetPartOutValueFromPageAsync for >$1k sets returns just thousands as single digit

Well, maybe an example will show what I mean:

If the part out value is above $1,000.00 then the code cuts hundreds, and it's returned as $1
Example sets with the issue:
10307
75313

Thanks for contributing ;D

#23


feat(upgade) upgrade Ubuntu version to focal

CI failed here; see https://travis-ci.com/github/Kong/kong-vagrant/builds/180712458#L1208
both postgres and cassandra were not installed/started

do not merge this yet! Postgres is broken

getting
hijss-MacBook:kong-vagrant thijs$ vagrant ssh
*******************************************************************************
 WARNING: The Kong source in /kong has latest tag 2.0.4-62-g358d5bbb
          whilst this vagrant box was build against version 2.1.3.
          Please make sure the checked-out version in /kong matches the
          binaries of 2.1.3.
*******************************************************************************
vagrant@ubuntu1904:~$ kong version
/usr/local/openresty/nginx/sbin/nginx: /lib/x86_64-linux-gnu/libcrypt.so.1: version `XCRYPT_2.0' not found (required by /usr/local/openresty/nginx/sbin/nginx)
vagrant@ubuntu1904:~$

also on this branch, so that is not caused by the postgres fix, so going to merge that one.


gh workflow support, DocC support, gh pages support, local dev / test…

… support
This PR covers tickets: SWCLFW-[17, 18, and 19]
Github Workflows (devops & automation)

Github workflows added for: - Swift build & test using CouchDB instance via Docker - Automatic pull and config of latest CouchDB - Swift build and test operations - Automatic CouchDB container cleanup - Building and deployment of framework documentation static website to repository's Github Pages

Developer support & automating for Github workflow testing:

projectroot/scripts/local contains automation scripts
support for act added to test gh workflows locally
automated run script for build-test using act added to scripts/local/gh-workflows
local docker env config & cleanup scripts in scripts/local/docker

General Project Updates:

README.md updated with new docs, cleanup, re-org, and clearer statements on support and legal
PR Template updated with minor changes
Issue template updated language and examples
.gitignore updated to ignore docs/ output folder path

merging to test new gh workflow to build and deploy docs


UofT-DSI | SQL - Homework 4

What changes are you trying to make? (e.g. Adding or removing code, refactoring existing code, adding reports)
Adding SQL queries for the first time to meet the requirements of Homework 4.
What did you learn from the changes you have made?
I learned I needed a more recent version of SQLite 3 installed to make REGEXP work.
Was there another approach you were thinking about making? If so, what approach(es) were you thinking of?
Yes, for more complicated queries usually there is more than one way to achieve the same results. For the last question of Homework 4, I explored a couple more options.
Were there any challenges? If so, what issue(s) did you face? How did you overcome it?
Yes, it took me some additional effort to get the REGEXP operator working.
Also, I bumped into a bug (not sure if it's in SQLite or DB Browser for SQLite) that forced me to use -- instead of /* and */' in the comments, otherwise I would get an error. Odd.
How were these changes tested?
Manual inspection of the result sets.
A reference to a related issue in your repository (if applicable)
Checklist

[x] I can confirm that my changes are working as intended

HW4 marked.


Synchronize concurrent tasks

Hi here Marco from coverlet https://github.com/tonerdo/coverlet
One of most requested feature is "Solution wide coverage".
To do that we need to sync/communicate with concurrent running task https://github.com/tonerdo/coverlet/tree/master/src/coverlet.msbuild.tasks
Is there a build-in mechanism?
If I run for instance dotnet test /p:CollectCoverage=true MergeWith.sln (https://github.com/tonerdo/coverlet/issues/598#issuecomment-551174529) I see overlapping task instances and concurrent tests.
We should sync CoverageResultTask  https://github.com/tonerdo/coverlet/blob/master/src/coverlet.msbuild.tasks/CoverageResultTask.cs#L73 and wait all tests and merge results.
Is there a way/pattern to do this?Or we need to implement custom sync/interprocess communication(we need a unique identifier to group task of same sln build)?
cc: @tonerdo

There is no built-in mechanism for this. In addition, people rolling their own locks have a long, sad history of causing confusing, hard-to-debug build breaks and hangs. So please consider whether there's any way you can redesign to avoid the need for the synchronization.

There is no built-in mechanism for this. In addition, people rolling their own locks have a long, sad history of causing confusing, hard-to-debug build breaks and hangs. So please consider whether there's any way you can redesign to avoid the need for the synchronization.

Eh...it's also my concern 😞 but users complain about the fact that msbuild works only with "project" and there are some tool/fields that need to be grouped to "solution" to be useful.
Users want only 1 coverage report file but we can run task only "project based" and also runs in parallel
Thank's for quick response @rainersigwald if you've some link on docs or some project that tackled this problem let me know.


Discussions repository Feature

I think that turn on the Discussion repository feature could be useful in order to provide a space to discuss topics other than issues or enhancements requests but always related to the use of this project.
Architecture
It would be a good idea to have a space where talk about architectural questions related to AWS Lambdas implementation with Swift using this project or how-to implement AWS Lambda Layers and the software design implications that it brings.
As UIKit framework teams did, the gave us the framework and said...

Hey guys, the best choice to work with this framework is following the MVC architecture pattern

We can start thinking about different architectural patterns that match with the nature of Lambdas and its usage with Swift, for example Hexagonal architecture, or event the MVC architecture.
Design Patterns
Another point of discuss is recommend set of software patterns to solve common projects goals, for example the use of Lambdas as AWS API Gateway backaend (defining a handler per endpoint and each related operation follows the Command pattern, or something like that...)
Sample Project
We can define some kind of sample project that serves as a PoC for all of this stuffs and future ones, as the projects used in the WWDC to show the different features, and include it in the Example folder of this project.
Conclusion
All of those points could be useful to define the way to work with AWS Lambdas using Swift at different projects scale, from the simple one, receiving an event from AWS SQS, to the complex one, where a mix of various API Gateway and SQS events are receivied and have a shared Domain and Data layers in a AWS Lambda Layer, storing data in a AWS DynamoDB instance

thanks for bringing this up @fitomad  there are a few spots where we can discuss such things:
swift forums: https://forums.swift.org/c/related-projects/aws-lambda-runtime/63
community slack: https://swift-open-source.slack.com/archives/C014TGU0GTE

Thanks for the info.
I didn't know the Slack channel (I hope giphy integration was disabled 🤪)
I'll take a look to forums and Slack
I think we can mark as closed this issue.

Hi @tomerd
Just one question about Slack community channel. Does It works under invitation process?
Thanks!

Here's an invite link, anyone is free to generate such invite link: https://join.slack.com/t/swift-open-source/shared_invite/zt-1pdcdvaqp-Nvh1QMRifpp_Js1kt5UjPQ

Thanks @ktoso!


Wrong regexes in youtube-regex-filters.txt

Hi there,
Pi-Hole v5.7 is unable to use the following regexes:
^[\w-.]?google(?:adservices|apis|syndication|usercontent)?.com$
^[\w-.]?l.google.com$
The error message is - Invalid character range.
Could you fix them please?

Same problem here

I have redone a lot of my regex filters so they should work much better. Thanks for alerting me.


fix split branches

fix #42

@tamasgal feel free to test it. the logic now is that we fully expand split-ed branch:
if user asks for V and there exists V/blah, we replace V with V/blah automatically, so users gets back all branches fully splitted

Thanks Jerry, I will have a look. I also have a WIP branch with some custom interpretation logic, let's see...

CI error is irrelevant.


ARROW-15721: [Docs][FlightSQL] Add Flight/Flight SQL to subprojects

Adds a Flight SQL documentation page that we can link to from the website.

I would like to add sequence diagrams, how do we feel about something like sphinxcontrib-mermaid?

CC @jduo if you'd like to take a look

Thanks for the suggestions!

I would like to add sequence diagrams, how do we feel about something like sphinxcontrib-mermaid?

Looks interesting. @jorisvandenbossche What do you think?

@jduo @pitrou any other comments here? I think we can use ARROW-14698 to expand the documentation once this lands


NATIVE not accessible

I can't seem to use Endianness::NATIVE, the rust compiler complains with "variant not found in Endianness"
For example:
140 |         let event_size = <NostrEvent as Writable<Endianness::NATIVE>>::bytes_needed(event)?;
    |                                                              ^^^^^^ variant not found in `Endianness`

I think the const may need to be defined outside of the Enum to be accessible.

Nevermind. I can't put a const in that position anyways.


Add AZ failure: EKS

See rudpot/scenario-az branch for existing work

@rudpot , I will pick this up.


experimentalEditContextEnabled breaks markdown.editor.pasteUrlAsFormattedLink.enabled

Does this issue occur when all extensions are disabled?: Yes

I can repro with the latest insiders and Stable builds:
Version: 1.95.0-insider
Commit: c85acdad6b679973c79b01a38afa91448c705c1c
Date: 2024-10-18T07:31:27.077Z
Browser: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Code-Insiders/1.95.0-insider Chrome/128.0.6613.162 Electron/32.1.2 Safari/537.36

Steps to Reproduce:

This is today's Insiders build, no extensions, no custom settings
Create a markdown file and paste a URL: the paste as... picker is shown as expected
Add/Enable "editor.experimentalEditContextEnabled": true
Paste the link again: this time the paste as... picker is not shown and it is not possible to force it with other keyboard shortcuts
Pasting an image in the markdown file is also broken the same way: the image is simply not pasted, nothing happens

One future alternative would be to switch to use the browser async clipboard api however this does not have the same functionality as the events right now. For example we currently  can't write out important mime types such as text/uri-list. Until those limitations are addressed I don't think we can switch to use it
Maybe we could try to create a paste on a hidden textarea so that we still get a real event? Here's what I'm thinking:

cmd+v is detected as a paste by our keybindings
We create a text area that is not visible to the user. This textarea also would also be outside of the edit context
We use  execCommand to trigger a real paste event on that hidden textarea
We then dispatch this paste event back to any listeners. The paste event object should remain valid for the synchronous dispatch of the event

Hi @mjbvz thank you for the suggestion I have the following PR which fixes this issue : https://github.com/microsoft/vscode/pull/232806. The PR however introduces a new method getTextAreaDomNode into the ICodeEditor interface to solve this. I am not quite sure this is a good idea, and I wanted to ask you for a review.

The paste problem is fixed bug I think this fix introduced a different problem, see #233012


Treat .msix files the same as .appx

Description
This change ensures that .msix files are treated exactly the same as .appx files, auto-detecting the UAP framework.
Related issue
Fixes #2476, #2296

@MSLukeWest does this look okay to you?


Function parameters confusing in inplace_copy and inplace_lerp

def inplace_copy(src: Tensor, tgt: Tensor, *, auto_move_device = False):
    if auto_move_device:
        tgt = tgt.to(src.device)

    src.copy_(tgt)``

It seems tgt is being copied into src, which is confusing because I read it as src = source and tgt = target. Similarly for inplace_lerp. Is this desirable?

@sheshanshag hey! yea you are right, that is a bit confusing
made the change


Assignment 4

Mingda Chen mc4084

Labs 3 and 4 accepted.
Nice work, although you should be able to get better performance on lab 3.