Secure intra-cluster communication with TLS

Right now components within a cluster (etcd, for example) are accessed over non-encrypted channels.  Ideally this would be secured.
This is an offshoot of #128.

This depends on #3168 to get the certs in place.

I think this will be entirely? done once #3168 is in place -- kubelets only talk to api server and will be secure.  Ditto kubeproxy.  Nodes won't use clustered-salt for setup, but rather per-node salt (for all providers?).  I propose if there are remaining work items, we fork those to specific issues tracking them.

I created a new cluster on GCE using default settings to do an audit of our current state of intra-cluster communication security.
I started by looking through the open ports (using netstat -an).
The master is listening on a variety of TCP ports, but many of these (7001, 10250, 10251, 10252, 2380, 8080) are only bound to the loopback address and thus wouldn't result in any communication traversing the network. Two ports (7080 and 6443) were open to the internal GCE network (10.240.x.x). And a handful of ports (22, 443, 4001, 4194) were open on all interfaces. As far as securing communications goes, ssh (22), https (443) and the read/write server on the master (6443) are all ok. There is no need for etcd (4001) or cAdvisor (4194) on the master to be reachable from elsewhere in the cluster and once we have secured master/node communication (#3168) we should be able to upgrade the read-only server on the master (7080) from http to https.
Each node is listening for tcp connections on ports 22 (ssh), 4194 (cAdvisor), 10250 (kubelet), and 10249 (kube-proxy healthz port) and all of these endpoints are open on all interfaces. The kubelet is currently contacted over http but will be upgraded to https as part of #3168. cAdvisor and kube-proxy are both serving http and should either be upgraded to https or only bind to localhost (if they don't need to be contacted remotely).
In addition to the open ports, I looked at the active network connections on the various VMs.
The nodes had established connections to the master on port 6443 (read-write) and 7080 (read-only) -- these communications should both be secured. All machines had connections to <IP_ADDRESS> on port 80 which is the local metadata server and doesn't leave the local machine (and is thus secure) -- this also isn't applicable for non-GCE deployments. The master had established connections to each node VM on port 10250 (kubelet) but I didn't see any connections to cAdvisor or the kube-proxy.
I also saw connections to two container IP addresses -- <IP_ADDRESS>:8086 (influx grafana controller) and <IP_ADDRESS>:9200 (elastic search logging controller) -- both of which are serving http connections. We are also sending udp packets to the cluster DNS service on port 53.

Fixing port 7080 and kubernetes-ro
Fix kube-proxy to use 6443 instead of 7080: #5917
Fix things that use kubernetes-ro service and KUBERNETES_RO env vars, including kube2sky, elasticsearch, and several examples. #5921

I've split the remaining work out into smaller issues:

etcd on the master should only be accessible from localhost: #5962
kube-proxy healthz port should only be accessible from localhost: #5963
influx grafana controller should be reachable securely: #5964
elastic search logging controller should be reachable securely: #5965
cAdvisor should only be accessible from localhost: #5967
cluster dns should be reachable securely: #5968

so I'm going to close this overarching issue.


[Email] Add gmail support

Description
[Email] Add gmail support
Add support for gmail login, show, send and forward.
Related Issue
Gmail support #349
P0: (Email) Google support #52
Testing Steps

Add the following in your appsettings and create google OAuth Connection Settings in your web app bot resource.
"googleAppName": "",
"googleClientId": "",
"googleClientSecret": "",
"googleScopes": "https://mail.google.com/ https://www.googleapis.com/auth/contacts",

Login with google account

Try show emails/ send emails/ forward emails/ reply emails.

@DingmaomaoBJTU - appsettings.json has been updated with an instrumentation key - can you remove please. Otherwise changes look great.


JavaScript capstone project: Add likes functionality

In this PR, the following were implemented

Post a like to the API
get likes from API
Display likes on the UI

@SoniaKarungi  Working functional code.
✅ Use of ES6 syntax.
✅ Use of ES6 modules.
✅ Use of callbacks and promises.
Ready for merge 😎

Thanks @jmoseka


Problem if called from a navigation controller that is embedded in a navigation controller.

I'm using iPhone 6S with iOS 9.1.
To replicate:

Open Demo project,
Select main storyboard
Select controller,
Embed in navigation controller.
Embed in Tabbar controller.
Run app
Only the "recent images" from the horizontal scroll work, take photo and select from library don't work,

I changed how the UIImagePickerController was presented.
- (void)selectFromLibraryWasPressed
{
    UIImagePickerController *picker = [[UIImagePickerController alloc] init];
    picker.delegate = self;
    picker.sourceType = UIImagePickerControllerSourceTypePhotoLibrary;
    picker.mediaTypes = [[NSArray alloc] initWithObjects:(NSString *)kUTTypeImage, nil];

    [self presentViewController:picker animated:YES completion:nil];
}

Subtle change here is to present the picker on self, and not on self.navigationcontroller


radial gauge seems to be broken.

In my project, if I don't update or reset (destroy and create again) within 5 minutes and 30 seconds with 1 sec update interval,
radial gauge stops working. I'm using version 2.1.7 in google chrome.
And this can be easily reproduced on canvas-gauge example site (https://canvas-gauges.com/documentation/examples/)
Here, using blue compass radial gauge, if you let it run for some minutes, it stops updating.

canvas-gauge is a great product but it seems like nobody is working on this project anymore.
maybe it's time for someone with time to fork this project?

Thanks for your collaboration, @peakhunt
Link: https://github.com/peakhunt/canvas-gauges


When I start TQVaultae 3.1.0 a message appears

When I start TQVaultae 3.1.0 a message appears:
"Could not load Text DB".

I have TQAE Atlamntis v.2.7 - current version on Steam.
What is the answer? Why do you publish the wrong version with the description that it allegedly works with TQAE Atlantis 2.7?
Originally posted by @Trejanus in https://github.com/EtienneLamoureux/TQVaultAE/issues/170#issuecomment-505625469

Hello, thanks for reporting your problem.
Please note that the TQVaultAE community of volunteer developers do not claim anything, as per the licence of this product. The current release of the product is an alpha release, which may contain known bugs, as per our disclaimer in the installation section.
Kindly wait for the upcoming release for a fix.

It is obvious. I only reported a problem. I do not think I'm the first to face this problem. I am waiting for a solution and I thank the authors for their work so far :-)

You can reproduce the error by manually entering an invalid game path. However, that's not a bug, but simply a wrong config. Solution: reinstall TQVaultAE (or manually edit the config, I guess)

Unless, of course, that message is used for other errors as well.

Check if the fix documented in our FAQ under "Q. Error Loading Resources. This may be caused by a bad language or game path setting." fixes your problem.


Dm extension branch

WDM with increased functionality with HMF to include filter parameters, using MAR from CDM, and interpolation between HMF WDM tables.

I think just one thing left to do here at the try/except that tries to set MassFunctionWDM. Can you make sure the if block containing this line only executes if have_hmf==True? Otherwise it will fail when hmf is not installed, which applies to the test suite (subject to change), but in general trying to keep dependencies optional if at all possible.

Okay I added that have_hmf dependency. Hopefully it works now!

Codecov Report

Merging #31 (7928c1f) into main (64c2784) will decrease coverage by 0.00%.
The diff coverage is 30.00%.

@@            Coverage Diff             @@
##             main      #31      +/-   ##
==========================================
- Coverage   54.61%   54.60%   -0.01%
==========================================
  Files         112      112
  Lines       22208    22211       +3
==========================================
+ Hits        12128    12129       +1
- Misses      10080    10082       +2

Impacted Files
Coverage Δ

ares/populations/Halo.py
76.11% <ø> (ø)

ares/util/SetDefaultParameterValues.py
97.41% <ø> (ø)

ares/physics/HaloMassFunction.py
46.98% <30.00%> (+0.08%)
:arrow_up:

ares/analysis/TurningPoints.py
72.65% <0.00%> (-0.79%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 64c2784...7928c1f. Read the comment docs.


No more requirement to be installed on OS X.

This stops NPM from refusing to install ios-sim on an OS other than OS X.
Right now I have a project where ios-sim is a dev-dependency, but developers on Linux can't even npm install the project now, because package.json's os field is preventing NPM from considering it a successful installation. I think the timing of that decision should be at runtime, not install time. Consider also that there may be people who want to install dependencies on Windows or Linux, and package the whole project, before using it on OSX.
The fact that there is no C/C++/ObjC code whatsoever in ios-sim shows that this can be installed on any platform without a problem.

+1

Merged and published as 5.0.4

Much appreciated! :)


How can i use "Referer" while uploading a multimedia file

I need to add custom header while uploading the multimedia files.
I used ObjectMetaData for applying my header, but it not working.  Let me know how can i add my custom header Refer.

@SkyNite See Object Key and Metadata. Referer isn't a system defined metadata. However you can add it as custom defined metadata which is prefixed with 'x-amz-meta'.

@fosterzhang : great thanks, will check and update.

@SkyNite two issues:

ObjectMetadata.setHeader is internal use only.
With user metadata, x-amz-meta is prepended by the SDK automatically. Simply use "Referer".

See ObjectMetadata for more information.


chore(main): release remote-backend 1.0.0

:robot: I have created a release beep boop
1.0.0 (2022-08-27)
Features

update release type (9564ec2)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/devhalos/nihil-infra/releases/tag/remote-backend-v1.0.0 :sunflower:


belindas-closet_sprint01_issue43_forgot-password

Resolves Issue 43
As a developer, I did the following:

Created a new screen for when a user forgets their password and clicks on text 'Forgot Password' from login screen
The user can enter their email address and press submit
The user can press the 'back arrow' to return to the Login screen

See Image of 'Forgot Password' screen below:

We also want to include unit tests if and when possible to make sure we're getting full coverage of our code. This will simulate a real world environment where tests are extremely important.*
--No Tests Yet--
Unit Test Example

I need help creating a test for this PR. I am unsure what one would look like in this scenario.

Just a minor thing:
When clicking the back arrow from the Forgot Password Page, it should return to the Login Page instead of the Home Page.
Great work

Just a minor thing: When clicking the back arrow from the Forgot Password Page, it should return to the Login Page instead of the Home Page. Great work

@tinpham5614
I just updated the route of the back arrow to return to Login and not Home. Thank you for catching!


Optimised db schema

In this pull request (PR), several optimizations and improvements have been made to the schema:

Introducing the roles table (using ENUM) and removing unnecessary tables:

role ENUM('admin', 'member', 'guest') NOT NULL

The week table is deemed unnecessary as it can be managed by query with different timesatamps

created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP

Introduction of a task assignment section to replace the permission section in the schema.

A trigger has been added to update the total contributions automatically:

DELIMITER //
CREATE TRIGGER update_total_contributions AFTER INSERT ON contributions
FOR EACH ROW
BEGIN
    UPDATE users
    SET total_contributions = total_contributions + 1
    WHERE user_id = NEW.user_id;
END;
//
DELIMITER ;
total_contributions INT DEFAULT 0

These changes optimize the schema by reducing redundancy and improving efficiency.
Here is an ER diagram for better demonstration:

@soniyaprasad77 please move css and js files to "public/" folder.
With regard to changes to contributions table, I am not sure if it is good idea. Please explain in more detail.
Think of a situation, a user shared solutions to say week w1 and task t1 in lang l1, lang l2.
Then a day later the same user shared solutions to week w1, task t2 in lang l1.
How would store this information?

The approach I've implemented resembles the one used for displaying user ages. Instead of storing the age directly in the database, we store the user's birth date. Then, at query time, we calculate the age by subtracting the birth date from the current date. Similarly, for the four scenarios you've provided regarding Task 1, we store the timestamp created_at. This allows us to identify the week of the task.
When a user submits their contribution to Task 1, we store the task ID and the timestamp of the contribution. This enables us to track how much time has elapsed since the task was created. With this schema, there should be no issue with users submitting multiple contributions for the same task. However, I haven't yet implemented the language part, but that will be addressed in the next pull request.

@soniyaprasad77, in real world, user sometimes submit contribution outside of the allotted week. For example, in the week 100, a user might decide to submit solutions to week 80 he missed for some reason.
The system should still consider solution belongs to week 80, although it was submitted in week 100. How would you handle this?
Let's keep the discussion on.

Certainly! I agree that it is a good case and I don't anticipate any issues. I haven't created any constraints that would prevent users from contributing late.
I would like to review my schema with the user flow architecture and provide an image in the attachment. This will help to explain the schema in greater detail.

@soniyaprasad77 lets get this sorted quickly.
Let me explain you what we want and you suggest a solution to this:

Admin creates 2 tasks (T1 and T2) each week and make these tasks public on every Monday morning.
Member/Guest picks the task (T1 or T2 or both), solves it and submits the solution.

We allow member/guest to solve any past tasks (T1 or T2 or both) if they want.
So we want to be able tag the solution to past tasks (T1 or T2 or both) of past week and not the time they submit the solution.
So when we want contributions report for any member/guest we should be able to identify week, task  and language.
There should be mechanism to identify which week belongs to a month.
So when we want monthly/yearly contributions report then we can easily get that.
task -> belongs -> week -> belongs -> month -> belongs -> year


Add more details to tweet in infinite stream view

Is your feature request related to a problem? Please describe.
Not all tweet meta data is shown, e.g. fav count, creation date are missing from the view.
Describe the solution you'd like
Add the relevant meta data to the view and add configuration possibility if they should be shown or not.

Without displaying retweeted tweets it will not be desirable to show favorites and retweet count.
This leaves the creation time in e.g. a natural formatting (e.g. 13 minutes ago)


Better interop with jdk proxy

Suggested by @trustin
https://github.com/line/armeria/pull/2496#issuecomment-597107331
https://docs.oracle.com/javase/8/docs/technotes/guides/net/proxies.html

Auto-detect the default proxy settings from the system properties
Add ProxyConfigSelector which is similar to ProxySelector, hopefully with a way to adapt an existing ProxySelector implementation into our ProxyConfigSelector.

Closing this since the feature was added


Update the Readme.md for Xamarin app

We need to update the Readme to ensure how a person new to the project can use use the Xamarin portion of the project in both development and deployment aspect

Will suggest if we create a separate ReadeMe.md file within the visage-xfapp folder as parent ReadMe.md is pre-populated with significant content, instead we can simply link to the Xamarin specific ReadMe.md from parent ReadMe.md.

In that case, lets create another wiki for both yours and @maniSbindra work. I will create another issue where I update the Readme.

@indcoder that makes sense. I will create Wiki specific for the Xamarin app.


Add optional startup delay (worker sleep)

Use case: For a demo, Telegraf startup needs to be delayed until all SRL nodes are up and running. Time dependencies are a bit of a hack, but it can help to workaround issues
Explicit depends-on (with optional delay) would be nice too, but this one is an easy add-on

Hi @karimra
I wonder if you have some nice ideas how to overcome the following limitation of nodes/links creation scheduling:
right now we get to links creation only after all the nodes have been created, so if we have delayed a creation of a certain node in a lab (as done in this PR), we won't start creating any links, even for the nodes that haven't been delayed and ready to accept links

It should be doable this way:

Use two maps of type map[string]struct{}, the first tracks created nodes createdNodes, the second tracks scheduled links scheduledLinks

The Node deploy method adds the name of the created node to createdNodes when it's done.

A separate goroutine loops through the links( c.Links ), checks if both nodes that make up that link are created ( i.e exist in the above map)

If both exist and the link is not in scheduledLinks , add the link name to scheduledLinks and send the link to the linkChan to be created

If one of the above conditions is not met, continue to next link.

The link scheduling goroutine should return when len(c.Links) == len(scheduledLinks)

Since the two maps will be accessed by multiple goroutines, there should be a mutex to control that

I think this whole thing is too complicated, why not create 2 labs where the second one contains the nodes that need to be delayed, wrap the whole thing in a script with a sleep statement.
For this use case, there is no need to create links to telegraf, only mgmt.

@karimra what do you say to the brute force approach I took in ce1fe7e?
the first idea that came to my mind is to return the waitgroups from nodes creation and wait for them after the scheduling of the links
when the links are about to be created, I copy a map of links and go over it checking if the NSPath of both endpoints of the links is not empty (that is denotes if a node was created by that time)

Is NSPath always populated? even in the case of mgmt-net and bridge nodes ?

Indeed, that may cause trouble as the tests show.
What if we are a field to the NodeConfig such as "deploymentStatus" and we will set it to "created" in the end of createNodes function?
And then we will check if this node is created when we schedule links to linkschan?

Indeed, that may cause trouble as the tests show.
What if we add a field to the NodeConfig such as "deploymentStatus" and we will set it to "created" in the end of createNodes function?
And then we will check if this node is created when we schedule links to linkschan?

that works as well, replacing the createdNodes map

Looks like it works nicely with docker but something is off with containerd and only with generate command. Need to look deeper :(

@steiler as a containerd expert, do you want to have a peek as to why the tests for generate command have an endless loop without break when we check links?


[SPARK-41359][SQL] Use PhysicalDataType instead of DataType in UnsafeRow

What changes were proposed in this pull request?
The main change of this pr is refactor UnsafeRow#isMutable  and UnsafeRow#isFixedLength  method to use PhysicalDataType instead of DataType.
Why are the changes needed?
Simplify type match.
Does this PR introduce any user-facing change?
No
How was this patch tested?
Existing UnsafeRowSuite

@cloud-fan

@cloud-fan


Use CAPI machine name for node name and add validation on failuredomain name

Issue #, if available:
Since failuredomain name can be used as label value for node, an validation is added to confirm failuredomain name is a valid label value:

must be 63 characters or less (can be empty),

unless empty, must begin and end with an alphanumeric character ([a-z0-9A-Z]),

could contain dashes (-), underscores (_), dots (.), and alphanumerics between.

Use CAPI machine name as node name so that they matches.
Description of changes:
Original cluster yaml definition allows node label for control plane and worker node, to support label with a dynamic value (failure domain), we use ds.meta_data.failuredomain as value in eksa cluster yaml. Note: we have to remove {{ }} since '{' is not an allowed char in label value.
Specify CAPI machine name as node name when kubelet registers node.
Testing (if applicable):
Unit testing, manual testing.
  labels:
    beta.kubernetes.io/arch: amd64
    beta.kubernetes.io/os: linux
    cluster.x-k8s.io/failure-domain: az1
    kubernetes.io/arch: amd64
    kubernetes.io/hostname: wanyufe-hkwh4
    kubernetes.io/os: linux
    node-role.kubernetes.io/control-plane: ""
    node-role.kubernetes.io/master: ""
    node.kubernetes.io/exclude-from-external-load-balancers: ""
  name: wanyufe-hkwh4

➜  eks-main git:(main) ✗ k get machines -A
NAMESPACE     NAME                            CLUSTER   NODENAME                        PROVIDERID                                           PHASE     AGE    VERSION
eksa-system   wanyufe-etcd-lkbj8              wanyufe                                   cloudstack:///eb75d397-00cc-44b2-a47b-681f58e86bd2   Running   3m3s
eksa-system   wanyufe-hkwh4                   wanyufe   wanyufe-hkwh4                   cloudstack:///c53402a6-57ff-43a8-8dd7-e8292295814f   Running   3m3s   v1.21.13-eks-1-21-17
eksa-system   wanyufe-md-0-784765b4f6-s2tg5   wanyufe   wanyufe-md-0-784765b4f6-s2tg5   cloudstack:///9a53e6c5-6b23-4706-8dda-3e975a1a4ef7   Running   3m3s   v1.21.13-eks-1-21-17

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

/cherrypick release-0.12

/approve

/retest


Fix 'site.cfg' location in Travis testing.

It turns out that all tests using iris-test-data have been skipping for a while
-- as in all the visible log messages "SKIP: Tests(s) require external data."
-- e.g. : https://travis-ci.org/SciTools/iris/jobs/341527576#L3747
This is in fact since https://github.com/SciTools/iris/commit/8a327b4f0faf4d150b72508262eaa6b5a31690f8,
However, I'm bafflled as to why

it seem not to be the Iris code changes at that point
it isn't the newer version of setuptools

The existing code in .travis.yml seems definitely wrong, because it creates site.cfg in the local directory, which is not the iris installation.
I'm currently at a loss to explain how iris.config previously managed to find the test data ??
Anyway,
proposed here is to put the 'site.cfg' into the installation, which should fix it.
NOTE:
This won't fix all the existing test failures,  due to the failure of iris.tests.check_graphic.
In fact, there will be a whole lot more of them ...

Take a look @bjlittle ?
Depressingly I think this is now "correct" = loads more errors !
Most of those are about ImageHash though, which is a known problem ...

The doc_dir is still referenced

In that case this is the "wrong" approach, and #2978 is the "right" one.


FedX federation fails with query evaluation exception when evaluating zero length property path

Current Behavior
We found that the FedX query engine currently fails with the following exception when the query contains a zero length path (e.g. ?myClass rdfs:subClassOf* ?parentClass)
The exact exception message is:

Federation Strategy does not support org.openrdf.query.algebra.evaluation.TripleSource#getStatements. If you encounter this exception, please report it.

This is because the federation engine falls back to the DefaultEvaluationStrategy implementation for property paths, which in case of zero-path is not tailored to the use in federation.
Note that path with length minimum 1 (i.e. the + operator) are working fine.
Expected Behavior
The federation engine should support correct semantics of zero-length path
Steps To Reproduce
Can be reproduced with a query like (assuming data being present)
SELECT * WHERE {
    ?subClass (rdfs:subClassOf*) ?myClass

Version
4.3.x
Are you interested in contributing a solution yourself?
Yes
Anything else?
We would like to see this addressed in 4.3.x and eventually in in the 5.0 release

@aschwarte10 We have also encountered this issue recently on the 4.3.9 branch and carried out some initial debugging.
I'll provide what we have as additional input.
On a single fedx repository, the following was inserted:
INSERT DATA {
  GRAPH ex:graph {
    :a foaf:knows :b .
    :a a foaf:Thing .
    :b foaf:friend :c .
    :b a foaf:Thing .
    :a foaf:knows :c .
    :d foaf:friend :e .
    :d a foaf:Thing .
    :e foaf:knows :f .
    :e a foaf:Thing .
    :f foaf:knows :e .
    :f a foaf:Thing .
    :f foaf:name "test" .
    :a foaf:homepage :h .
  }
}

Queries
All of these are wrapped in SELECT * { } .

Query
Result
Comments

?s (foaf:knows|foaf:friend)+ ?o . ?s foaf:knows+ ?o .
🟢
ArbitraryLengthPath -> NUnion -> StatementSourcePattern

?s (foaf:knows|foaf:friend)* ?o .
🔴  Runtime exception
NUnion -> (ZeroLengthPath + 2x StatementSourcePattern)

?s (foaf:knows|foaf:friend)? ?o .
🔴  Runtime exception
ArbitraryLengthPath -> NUnion -> StatementSourcePattern (Failure against ZeroLengthPathIteration as above)

?s a foaf:Thing; (foaf:knows|foaf:friend)* ?o . ?s a foaf:Thing; (foaf:knows|foaf:friend)? ?o . ?s (foaf:knows|foaf:friend)* ?o; a foaf:Thing . ?s (foaf:knows|foaf:friend)? ?o; a foaf:Thing .
🟡  Sometimes completes, sometimes appears to cause evaluation deadlock and provided query timeout is hit.
NJoin of simple pattern before arbitrary length one appears to use different query evaluation path. It doesn’t appear to matter whether the simple pattern precedes or follows the prop path. In addition DISTINCT is generated for these which appears not be fully supported.

(Apologies for the lack of separation of patterns in the table above)
In addition, we have not observed any issues with property paths which do not use the / (i.e. alternative path). The alternative path (as can be seen from the above comments) is translated into a NUnion of each possible combination.
In fact, the following queries also trigger the issue, just like the alternative path modifier:
SELECT * {
  {
    ?s foaf:knows* ?o .
  } UNION {
    ?s foaf:friend* ?o .
  }
}

SELECT * {
    ?s foaf:knows+ ?o .
}

How to detect

? - NUnion containing ZeroLengthPath (need to check what pre-optimised query looks like)

Or maybe any ZeroLengthPath ?

* - ArbitraryLengthPath (of zero or more presumably)

@vtermanis


go fmt config.go

Hello again @pefoley2 thanks for catching this. I wonder when this changed.
LGTM, merging it in!


[FLINK-3667] delay connection to JobManager until job execution

lazily initialize ActorSystem
make sure it is not created before job execution
print connection information on the CLI

Please provide some context for this PR, the referenced JIRA is marked as closed.

there are also some test failures in flink-clients:
testManualOptionsOverridesConfig(org.apache.flink.client.CliFrontendAddressConfigurationTest)  Time elapsed: 0.031 sec  <<< FAILURE!
org.junit.ComparisonFailure: expected:<[ip-10-221-130-22.ec2.internal]> but was:<[<IP_ADDRESS>]>
	at org.junit.Assert.assertEquals(Assert.java:115)
	at org.junit.Assert.assertEquals(Assert.java:144)
	at org.apache.flink.client.CliFrontendTestUtils.checkJobManagerAddress(CliFrontendTestUtils.java:110)
	at org.apache.flink.client.CliFrontendAddressConfigurationTest.testManualOptionsOverridesConfig(CliFrontendAddressConfigurationTest.java:99)

Sorry, didn't see your comments. The test failures should be resolved (I ran mvn verify).
The changes delay the creation of the ActorSystem and thus any blocking actions until we have executed the user jar.


sealos 4.0 老版本卸载安装 kubernetes:v1.24.2 无法启动

安装程序日志
2022-07-05 11:49:34 [EROR] Applied to cluster error: failed to init init master0 failed, error: failed to execute command(kubeadm init --config=/var/lib/sealos/data/default/etc/kubeadm-init.yaml --skip-certificate-key-print --skip-token-print -v 0 --ignore-preflight-errors=SystemVerification) on host(<IP_ADDRESS>:22): output(W0705 11:45:15.060301   33945 initconfiguration.go:120] Usage of CRI endpoints without URL scheme is deprecated and can cause kubelet errors in the future. Automatically prepending scheme "unix" to the "criSocket" with value "/run/containerd/containerd.sock". Please update your configuration!
[init] Using Kubernetes version: v1.24.2
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Using existing kubeconfig file: "/etc/kubernetes/admin.conf"
[kubeconfig] Using existing kubeconfig file: "/etc/kubernetes/kubelet.conf"
W0705 11:45:34.491781   33945 kubeconfig.go:249] a kubeconfig file "/etc/kubernetes/controller-manager.conf" exists already but has an unexpected API Server URL: expected: https://<IP_ADDRESS>:6443, got: https://apiserver.cluster.local:6443
[kubeconfig] Using existing kubeconfig file: "/etc/kubernetes/controller-manager.conf"
W0705 11:45:34.669181   33945 kubeconfig.go:249] a kubeconfig file "/etc/kubernetes/scheduler.conf" exists already but has an unexpected API Server URL: expected: https://<IP_ADDRESS>:6443, got: https://apiserver.cluster.local:6443
[kubeconfig] Using existing kubeconfig file: "/etc/kubernetes/scheduler.conf"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all running Kubernetes containers by using crictl:
- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock logs CONTAINERID'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher), error(Process exited with status 1). Please clean and reinstall
containerd日志
月 05 11:49:36 k8s.<IP_ADDRESS> containerd[32343]: time="2022-07-05T11:49:36.280114877+08:00" level=error msg="PullImage "k8s.gcr.io/kube-scheduler:v1.24.2" failed" error="failed to pull and unpack image "k8s.gcr.io/kube-scheduler:v1.24.2": failed to resolve reference "k8s.gcr.io/kube-scheduler:v1.24.2": failed to do request: Head "https://k8s.gcr.io/v2/kube-scheduler/manifests/v1.24.2\": dial tcp <IP_ADDRESS>:443: i/o timeout"
7月 05 11:50:06 k8s.<IP_ADDRESS> containerd[32343]: time="2022-07-05T11:50:06.282947711+08:00" level=error msg="PullImage "k8s.gcr.io/etcd:3.5.3-0" failed" error="failed to pull and unpack image "k8s.gcr.io/etcd:3.5.3-0": failed to resolve reference "k8s.gcr.io/etcd:3.5.3-0": failed to do request: Head "https://k8s.gcr.io/v2/etcd/manifests/3.5.3-0\": dial tcp <IP_ADDRESS>:443: i/o timeout"
7月 05 11:50:36 k8s.<IP_ADDRESS> containerd[32343]: time="2022-07-05T11:50:36.285500460+08:00" level=error msg="PullImage "k8s.gcr.io/kube-controller-manager:v1.24.2" failed" error="failed to pull and unpack image "k8s.gcr.io/kube-controller-manager:v1.24.2": failed to resolve reference "k8s.gcr.io/kube-controller-manager:v1.24.2": failed to do request: Head "https://k8s.gcr.io/v2/kube-controller-manager/manifests/v1.24.2\": dial tcp <IP_ADDRESS>:443: i/o timeout"
7月 05 11:51:06 k8s.<IP_ADDRESS> containerd[32343]: time="2022-07-05T11:51:06.288345405+08:00" level=error msg="PullImage "k8s.gcr.io/kube-apiserver:v1.24.2" failed" error="failed to pull and unpack image "k8s.gcr.io/kube-apiserver:v1.24.2": failed to resolve reference "k8s.gcr.io/kube-apiserver:v1.24.2": failed to do request: Head "https://k8s.gcr.io/v2/kube-apiserver/manifests/v1.24.2\": dial tcp <IP_ADDRESS>:443: i/o timeout"
7月 05 11:51:36 k8s.<IP_ADDRESS> containerd[32343]: time="2022-07-05T11:51:36.291146322+08:00" level=error msg="PullImage "k8s.gcr.io/kube-scheduler:v1.24.2" failed" error="failed to pull and unpack image "k8s.gcr.io/kube-scheduler:v1.24.2": failed to resolve reference "k8s.gcr.io/kube-scheduler:v1.24.2": failed to do request: Head "https://k8s.gcr.io/v2/kube-scheduler/manifests/v1.24.2\": dial tcp <IP_ADDRESS>:443: i/o timeout"
containerd 无容器启动
crictl ps -a
CONTAINER           IMAGE               CREATED             STATE               NAME                ATTEMPT             POD ID

'systemctl status kubelet'
'journalctl -xeu kubelet'

老版本是多少的版本

7月 05 14:34:30 k8s.<IP_ADDRESS> containerd[14575]: time="2022-07-05T14:34:30.370124062+08:00" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.devmapper" error="devmapper not configured"
7月 05 14:34:30 k8s.<IP_ADDRESS> containerd[14575]: time="2022-07-05T14:34:30.370995675+08:00" level=warning msg="could not use snapshotter devmapper in metadata plugin" error="devmapper not configured"
7月 05 14:34:30 k8s.<IP_ADDRESS> containerd[14575]: time="2022-07-05T14:34:30.376286161+08:00" level=error msg="failed to initialize a tracing processor "otlp"" error="no OpenTelemetry endpoint: skip plugin"
7月 05 14:34:30 k8s.<IP_ADDRESS> containerd[14575]: time="2022-07-05T14:34:30.378922794+08:00" level=error msg="failed to load cni during init, please check CRI plugin status before setting up network for pods" error="cni config load failed: no network config found in /etc/cni/net.d: cni plugin not initialized: failed to load cni config"

老版本是sealos:3.0
k8s:1.19.16
kubelet 日志
7月 05 14:39:55 k8s.<IP_ADDRESS> kubelet[24301]: E0705 14:39:55.601656   24301 kubelet.go:2424] "Error getting node" err="node "k8s.<IP_ADDRESS>" not found"
7月 05 14:39:55 k8s.<IP_ADDRESS> kubelet[24301]: E0705 14:39:55.633149   24301 certificate_manager.go:471] kubernetes.io/kube-apiserver-client-kubelet: Failed while requesting a signed certificate from the control plane: cannot create certificate signing request: Post "https://apiserver.cluster.local:6443/apis/certificates.k8s.io/v1/certificatesigningrequests": dial tcp <IP_ADDRESS>:6443: connect: connection refused
7月 05 14:39:55 k8s.<IP_ADDRESS> kubelet[24301]: E0705 14:39:55.702006   24301 kubelet.go:2424] "Error getting node" err="node "k8s.<IP_ADDRESS>" not found"
7月 05 14:39:55 k8s.<IP_ADDRESS> kubelet[24301]: E0705 14:39:55.762622   24301 controller.go:144] failed to ensure lease exists, will retry in 6.4s, error: Get "https://apiserver.cluster.local:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/k8s.<IP_ADDRESS>?timeout=10s": dial tcp <IP_ADDRESS>:6443: connect: connection refused
7月 05 14:39:55 k8s.<IP_ADDRESS> kubelet[24301]: E0705 14:39:55.802787   24301 kubelet.go:2424] "Error getting node" err="node "k8s.<IP_ADDRESS>" not found"
7月 05 14:39:55 k8s.<IP_ADDRESS> kubelet[24301]: E0705 14:39:55.903568   24301 kubelet.go:2424] "Error getting node" err="node "k8s.<IP_ADDRESS>" not found"
7月 05 14:39:55 k8s.<IP_ADDRESS> kubelet[24301]: I0705 14:39:55.957535   24301 kubelet_node_status.go:70] "Attempting to register node" node="k8s.<IP_ADDRESS>"
7月 05 14:39:55 k8s.<IP_ADDRESS> kubelet[24301]: E0705 14:39:55.958133   24301 kubelet_node_status.go:92] "Unable to register node with API server" err="Post "https://apiserver.cluster.local:6443/api/v1/nodes\": dial tcp <IP_ADDRESS>:6443: connect: connection refused" node="k8s.<IP_ADDRESS>"
7月 05 14:39:56 k8s.<IP_ADDRESS> kubelet[24301]: E0705 14:39:56.003963   24301 kubelet.go:2424] "Error getting node" err="node "k8s.<IP_ADDRESS>" not found"
7月 05 14:39:56 k8s.<IP_ADDRESS> kubelet[24301]: E0705 14:39:56.104661   24301 kubelet.go:2424] "Error getting node" err="node "k8s.<IP_ADDRESS>" not found"

把kubelet打全一点，这些日志都不是关键信息


CLI builds on ubuntu do not successfully compile and execute

When building a developer setup on a clean Ubuntu VM (not the packaged installer for pyrsia)
Steps taken/Steps to reproduce

Start from a clean build of ubuntu's latest OS
Download latest mainline branch "main" from Pyrsia
execute cargo build --workspace
executing cargo build --workspace after adding openssl dev and clang dev packages with apt install works after multiple calls to cargo build.
After cli successfully builds execute DEV_MODE=on RUST_LOG="pyrsia=debug,info"  ./pyrsia_node -H <IP_ADDRESS> -p 7888 -L /ip4/<IP_ADDRESS>/tcp/44001

Expected Behavior
Successful build of the software.
Successful starting of the CLI.
Experienced Instance of the bug
Had to install undocumented dependencies, clang and openssl
cli crashed with the following log
$ DEV_MODE=on RUST_LOG="pyrsia=debug,info"  ./pyrsia_node -H <IP_ADDRESS> -p 7888 -L /ip4/<IP_ADDRESS>/tcp/44001
DEBUG pyrsia_node > Parse CLI arguments
DEBUG pyrsia_node > Create p2p components
WARN  pyrsia::util::keypair_util > Failed to persist newly generated keypair: Os { code: 20, kind: NotADirectory, message: "Not a directory" }
DEBUG pyrsia_node                > Start p2p event loop
DEBUG pyrsia_node                > Setup HTTP server
DEBUG pyrsia_node                > Pyrsia Docker Node will bind to host = <IP_ADDRESS>, port = 7888
DEBUG pyrsia_node                > Setup HTTP routing
DEBUG pyrsia_node                > Setup HTTP server
INFO  pyrsia_node                > Pyrsia Docker Node will start running on <IP_ADDRESS>:7888
DEBUG pyrsia_node                > Start p2p components
DEBUG pyrsia::network::client    > p2p::Client::listen "/ip4/<IP_ADDRESS>/tcp/44001"
DEBUG pyrsia_node                > Provide local artifacts
ERROR pyrsia::node_manager::handlers > Error initializing Artifact Manager Directory, error is: Failed to create artifact manager directory in dev mode
thread 'main' panicked at 'Error initializing Artifact Manager Directory, error is: Failed to create artifact manager directory in dev mode', src/node_manager/handlers.rs:78:13
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace
INFO  pyrsia::network::event_loop    > Local node is listening on "/ip4/<IP_ADDRESS>/tcp/44001/p2p/12D3KooWDTZxwB8AWjkfcTaVwvyUAqep1ZRBDL8fUi5oec5NKV2S"
INFO  pyrsia::network::event_loop    > Local node is listening on "/ip4/<IP_ADDRESS>/tcp/44001/p2p/12D3KooWDTZxwB8AWjkfcTaVwvyUAqep1ZRBDL8fUi5oec5NKV2S"
INFO  pyrsia::network::event_loop    > Local node is listening on "/ip4/<IP_ADDRESS>/tcp/44001/p2p/12D3KooWDTZxwB8AWjkfcTaVwvyUAqep1ZRBDL8fUi5oec5NKV2S"
ERROR pyrsia::node_manager::handlers > Initialization of Artifact Manager Directory panicked!

In order to build pyrsia_cli on a Ubuntu machine from the workspace we have a dependency on openssl and clang.
When installing pyrsia_cli from the installer - there is not dependency on openssl. Maybe there is a dependency on clang
When pyrsia_cli uses https it will need openssl in the future.
This nuance has not documented clearly.

There are 3 issues noticed above. Split them into 3 tickets before trying to fix them.

#515  Issue about Clang

@MitaliBo can you please look at the startup issue  when you get a chance, @mseabornIBM brought it up at standup
ERROR pyrsia::node_manager::handlers > Error initializing Artifact Manager Directory, error is: Failed to create artifact manager directory in dev mode

This issue has been patched!


Add LORGD2_VALIDATOR as a validator

I am Neil, a self-taught crypto enthusiast. I've been in this space since 2019, building mastenodes, validators and participating in the community of projects. I participated to Celestia Testnet, Aptos Testnet, Sui Testnet, Massa Testnet, Ziesha Testnet, Nolus Testnet, Shardem Betanet. I participated to Namada Trusted set up too in December 2022.
Discord : Neil#7985
Twitter : @weineil_

https://discord.com/channels/833618405537218590/1062301142727524383/1073123044068962346


Pro - New Feature Template does not have a default value for symbol set or populate tags from icon

Some issues/questions @BobBooth and I had after using the New Feature Template form with the DictionaryRenderer

Tags - Should the New Feature template dialog be able to populate the tags from the Stylx icon tags?

Feature Class default value - when I create a new feature template for a layer/feature class that does not have a subdomain, but does have a default value - the default value is not getting set on the "New Template" form.

I just wanted to make sure this was the intended behavior (seems like the default feature value should be getting set here, it gets set it you create/edit  a new feature) - is there any way to set/control this?

@psanchez1327 - can you please enlighten us?

I'm going to close this issue if there are no objections.
The Military Overlay template as well as all the sample projects at https://github.com/Esri/military-features-data/tree/v.next/data/mil2525d do have templates included with them - and most will start with one of them.
The auto-populate tags from style tags is a good idea though - it would really save us time creating template -  @Dbarnes1 @BobBooth we may want to write get an enhancement CR written for just that one


Starting two different stubs in the same port

Question
We have a hybrid kind of project model where multiple services runs on same port .
Now say that my consumer C1 depends on P1(producer 1 ) and P2(producer 2) and both runs on the same port .
Is there any way I can configure on the client side , so that both are loaded on the same port ?

How can they run on the same port? What you can do is to run a single WireMock instance on one port, but load it with stubs from 2 different producers. If the URLs don't overlap then things should work fine.
But IMO you could have 2 separate client contract tests, both of which have the http client's port configurable. I don't think you should combine those tests into one.

Thanks for the reply .
How can they run on the same port?  - This is because ours is a mixed architecture and multiple projects are bundled as one fatty jar and deployed to a target port .
Have a follow up doubt too ..
If Can i run the wiremock on the same server port as the producers are also loaded . Do you have some example which I can refer if this is possible .

I have a problem with understanding what you write. Why not do sth like this (pseudocode):

class Client {

    private final int producerAPort;
    private final int producerBPort;

    Client(int producerAPort, int producerBPort) { this.producerAPort = producerAPort; this.producerBPort = producerBPort; }

    void callProducers() {
    // here you have your logic
    }

}

Configuration
@Configuration
class ClientConfig {

    @Value("${producera.port:8765}") int producerAPort;
    @Value("${producerb.port:8765}") int producerBPort;

    @Bean Client client() {
        return new Client(this.producerAPort, this.producerBPort);
    }
}

Now in your contract tests you do the following:
// all the annotations you need
// notice random ports
@AutoConfigureStubRunner(..., ids="...producer-a", "...producer-b")
public class ClientContractTest {

    @Autowired StubFinder finder;

    int producerAPort;
    int producerBPort;

    @Before
    public void setup(){
        this.producerAPort = this.finder.findStubUrl("...producer-a").getPort();
        this.producerBPort = this.finder.findStubUrl("...producer-b").getPort();
    }

    @Test
    public void shouldWorkForTwoProducers() {
        Client client = new Client(this.producerAPort, this.producerBPort);

        // test whatever you need
    }

}

Does it make sense?

Yes .. Thank you so much ... Will try out and get back in case of any issues .

No problem


On Launch triggered multiple times

firebaseMessaging.configure(
        onMessage: (Map<String, dynamic> message) async {
          final notification = message['data'];
          print("onMessage");
          await Navigator.of(context, rootNavigator: true).push(
              MaterialPageRoute(builder: (context) => Rating('onMessage')));
          print('cek onMessage body  = ${notification['name']} raw = $message');
        },
        onResume: (Map<String, dynamic> message) async {
          await print('cek onResume firebase = $message');
          await Navigator.of(context, rootNavigator: true).push(
              MaterialPageRoute(builder: (context) => Rating('onResume')));
        },
        onLaunch: (Map<String, dynamic> message) async {
          if (!_isConfigured) {
            print("onLaunch");
            print('cek onLaunch firebase = $message');
            await Navigator.of(context, rootNavigator: true).push(
                MaterialPageRoute(builder: (context) => Rating('onLaunch')));

            _isConfigured = true;
          }
        },
        onBackgroundMessage: myBackgroundMessageHandler);

on launch triggered again when we close the application with back button

Hi @joharputr
Can you please provide your flutter doctor -v your flutter run --verbose and a complete reproducible minimal code sample
Thank you

I had the same issue, but I noticed that I trigger configure twice, I fix that and it's working fine.

hi @TahaTesser
[√] Flutter (Channel stable, 1.20.4, on Microsoft Windows [Version 10.0.18363.1082], locale id-ID)
• Flutter version 1.20.4 at D:\master\flutter
• Framework revision fba99f6cf9 (11 days ago), 2020-09-14 15:32:52 -0700
• Engine revision d1bc06f032
• Dart version 2.9.2
[√] Android toolchain - develop for Android devices (Android SDK version 29.0.3)
• Android SDK at D:\master\sdk_android\commandlinetools-win-6514223_latest
• Platform android-29, build-tools 29.0.3
• ANDROID_HOME = D:\master\sdk_android\commandlinetools-win-6514223_latest
• Java binary at: C:\Program Files\Android\Android Studio\jre\bin\java
• Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b01)
• All Android licenses accepted.
[√] Android Studio (version 4.0)
• Android Studio at C:\Program Files\Android\Android Studio
• Flutter plugin version 48.0.2
• Dart plugin version 193.7361
• Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b01)
[√] VS Code (version 1.46.1)
• VS Code at C:\Users\ASUS\AppData\Local\Programs\Microsoft VS Code
• Flutter extension version 3.12.2
[√] Connected device (1 available)
• RMX1821 (mobile) • Y5ZD8LMFHYF6I7JF • android-arm64 • Android 10 (API 29)
• No issues found!

@danielwalczak you call configure twice in init state?

Yes I have it in initState, but the way I setup splash screen -> home screen redirect did trigger configure twice, When I fixed this issue, everything was ok. I thought you might have similar issue.

Hi @joharputr
Can you please provide complete flutter run --verbose with onunch trigger multiple times e and a complete reproducible minimal code sample
Thank you


Derive peer name from product_uuid

Fixes #901.

There is a gotcha here for upgrades: weave stop; <upgrade>; weave launch will change the peer identity. This is bad since it prevents reclaiming of container IPs.
The obvious fix is that instead of deriving the peer name from the machine id, we continue to derive it from the bridge mac, and instead derive the bridge mac from the machine id (replacing the random generation). That's what I was going to do initially but it's less clean. Deriving MACs from permanent IDs is dodgy since it will result in clashes with any other program doing the same (though I suppose we could get around that by hashing the machine-id with a weave-specific salt). It also prevents us from gaining 2 bits (the bridge MAC must have the 'multicast' bit clear and the 'locally administered' bit set).
I haven't got any better idea though. Thoughts?

Would also fix #894, I think.

Why is CircleCI failing? I suspect all hosts have the same machine-id. The docker info reported by the test runs for each of the hosts shows that the docker daemon has the same ID (YTMJ:U24E:PHCW:JR6D:HP3T:KP5N:XN7J:SNOR:BKGZ:25OB:WIOI:VWJP) on all of them, and while the daemon ID is not derived from the machine-id (afaict), this nevertheless indicates that these hosts are all clones.
This is problematic, and I cannot think of a fix.
Presumably some docker tools, e.g. swarm, will be unhappy with multiple daemons having the same ID, so it's not just a problem for weave.

@tomwilkie is there anything peculiar about how the circle hosts were created?
If this "all the hosts look the same" issue is something that users can easily run into then this whole approach I am pursuing here is dead :(

Nitpicking: they're "GCE hosts" that we happen to create by running a script (weave/test/gce.sh) on CircleCI.

This (lack of unique machine-id) is problematic, and I cannot think of a fix.

I switched to using the BIOS ID, which seems to work.

Copied from https://github.com/weaveworks/weave/issues/901#issuecomment-170854398

I am really worried that some common way of creating "machines" via some cloning/copying/whatever mechanism might result in ID duplicates.

I think before we merge this we need to ensure that snapshots/clones (or whatever they're called in the respective vendor's terminology) don't duplicate the product_uuid:

[ ] VirtualBox
[x] DigitalOcean
[ ] AWS
[ ] Azure
[ ] GCE

@errordeveloper can you help with that?

@rade does this not add a mandatory requirement for weave to be run as root, even if your user can access docker without?

That requirement is already there. weave --local checks that you are running as root, and weaveexec is run with --privileged for a reason.

D'oh, yes of course.

I'm not sure why we haven't listed the popular hypervisors that people are likely to use, and I found this one note out in the wild wild web.
One useful note regarding VMWare. So whether you moved or copied a VM on your machine, it will still appear to be unique. It's the same machine if you haven't moved the files.
I'm intending to confirm what's the case with KVM exactly, whilst at it.
So I'm suggesting to append the following to our list:

[X] VMWare (tested using AppCatalyst, should apply to all products)
[ ] KVM
[ ] VirtualBox

Also, testing these would give us a better coverage for smaller providers that are not covered above, as those are quite likely to run on either VMWare or KVM.

The systemd-detect-virt tool may be of some interest here also.

On VMWare:
> docker run -ti alpine cat /sys/class/dmi/id/product_uuid
564D74BB-2119-E8E8-10D0-77D1195A51A2
> sudo cat /sys/class/dmi/id/product_uuid
564D74BB-2119-E8E8-10D0-77D1195A51A2

On EC2:
> docker run -ti alpine cat /sys/class/dmi/id/product_uuid
EC209BC9-2CD3-41AB-1D79-171974C47FBC
> sudo cat /sys/class/dmi/id/product_uuid
EC209BC9-2CD3-41AB-1D79-171974C47FBC

Also, regarding /sys/hypervisor/uuid  on EC2, these days it appears identical to /sys/class/dmi/id/product_uuid, only lowercase.
> cat /sys/hypervisor/uuid
ec27dad0-2b18-1ef6-9650-42cdb21f0495
> sudo cat /sys/class/dmi/id/product_uuid
EC27DAD0-2B18-1EF6-9650-42CDB21F0495

I have checked, and cloning an instance into an image and starting a new instance from that did result in a new UUID. Also, rebooting instance from within itself resulted in UUID persistence and rebooting it via 'Stop' and 'Start' in the AWS console still resulted in UUID persistence, which @awh said wasn't the case in DO. I haven't waited very long between stopping and starting it again, so not sure if it could potentially end-up on a different physical host or how exactly that works... In any case, please do not that these UUIDs begin with 'EC2', it's kind of amusing.

So I'm suggesting to append VMWare and KVM to our list.

Have added these to the list in @awh's comment, so we keep just one list.

I've tested with VirtualBox (5.0.12) on my ubuntu machine, cloning our host1 vagrant test VMs.
When I first start the VM right after cloning, it has the same product_uuid as the parent. This persists even across reboots. However, when "power off" the VM and restart it, it gets a fresh product_uuid. And this then remains unchanged even across further power-offs.
I tried this with both "Full" and "Linked" clones, and with and without telling it to re-set the MAC addresses. Curiously, when I tell it to change MACs, the MAC doesn't actually change until the first "power off", and then remains the same. It's almost as if the cloning operation remains half-finished until the clone is powered off. Perhaps this is normal behaviour.
The clones aren't going to be of much use if they all have the same MACs, so I don't think the fact that the UUID also remains the same should concern us.

@rade those finding are interesting. Personally, I'm not very surprised about VirtualBox doing what you have observed.

On GCE rebooting and power cycling persists the UUID. Cloning results in a new UUID.

On Azure rebooting and power cycling persists the UUID. I'm still looking for the clone button....

So in Azure you can only clone a "classic" VM, and there is no obvious way to do something similar in the new Resource Manager world. Cloned "classic" VM does get a new UUID, as expected.

I tested KVM: an Ubuntu guest automagically got a new product_uuid which survived a reboot, and cloning the guest got a different product_uuid.
An Alpine guest under KVM got no product_uuid unless the -uuid flag was used on the command-line to start the VM.
So I would say this is working as desired under KVM.

Having restarts work reliably for us is very important. Being able to reclaim IPs is as well. It's not necessarily equally important for all of our machines & all of our services, but there are cases where it very much matters to us.
It looks to me like the only issue is having the host peer ID be the same each time weave runs.  I do appreciate the attempts to make weave autoconfigure in most cases. But in some environments autoconfig isn't going to be reliable enough, and one of the cool things with weave is being able to deploy across a heterogeneous environment.
When launching weave, it would be great if we could simply specify the peer ID. This would also provide administrators with an easy way to bring up a replacement machine after a node has permanently failed.

When launching weave, it would be great if we could simply specify the peer ID.

You can already do that. weave launch --name d2:e1:4d:cc:92:15.

Thanks @rade , that's awesome. It wasn't clear to me scanning through the weave script or looking at it's help output.

@rade When launching with --name, are there any restrictions on the name? Will any (psuedo-)random mac-like string work, or are there other considerations? In random_mac(), care is taken so that the multicast bit is clear and locally administered bit is set. Can that be ignored, or should that be taken into consideration?

For what it's worth, here is the process which ends up running in the container: "/home/weave/weaver --port 6783 --name RANDOM_MAC_HERE --nickname HOSTNAME_HERE --datapath datapath --ipalloc-range IPALLOC_RANGE_HERE--dns-effective-listen-address <IP_ADDRESS> --dns-listen-address <IP_ADDRESS>:53 --http-addr <IP_ADDRESS>:6784 --name NAME_I_SPECIFIED_HERE --ipalloc-default-subnet DEFAULT_SUBNET_HERE REMOTE_PEERS_HERE"
It seems to come up with the correct peer ID (NAME_I_SPECIFIED_HERE), but I note that the --name flag is being passed twice.

Will any (pseudo-)random mac-like string work?

Yes.

the --name flag is being passed twice.

That's fine. Last one wins, which is why we don't need any special code in the script for this and a few other options (e.g. --nickname).

With 1.4.4 I have noticed this after rebooting some of my instances in EC2:
root@ip-172-20-0-93:~# weave status ipam
9e:32:45:f9:af:d8(ip-172-20-0-93)        32768 IPs (03.1% of total)
e6:19:24:41:11:36(ip-172-20-0-92)        65535 IPs (06.2% of total)
82:c8:e0:d9:db:87(ip-172-20-0-76)        65537 IPs (06.3% of total)
0e:b5:e1:a7:63:ad(ip-172-20-0-91)        65535 IPs (06.2% of total)
4e:09:9f:f2:81:4e(ip-172-20-0-223)       65535 IPs (06.2% of total)
3a:ba:ee:58:8c:35(ip-172-20-0-223)      131074 IPs (12.5% of total) - unreachable!
a2:43:08:c6:20:cb(ip-172-20-0-91)        65535 IPs (06.2% of total) - unreachable!
9e:f3:0c:4c:8a:79(ip-172-20-0-167)      131072 IPs (12.5% of total) - unreachable!
e2:3f:01:58:21:11(ip-172-20-0-166)      131072 IPs (12.5% of total) - unreachable!
ca:0b:2f:c1:6f:94(ip-172-20-0-168)      131072 IPs (12.5% of total) - unreachable!
f2:69:4f:49:49:46(ip-172-20-0-152)       65538 IPs (06.3% of total)
de:99:02:c8:9d:d0(ip-172-20-0-223)       32767 IPs (03.1% of total) - unreachable!
22:82:0f:f7:cd:f4(ip-172-20-0-88)        65536 IPs (06.2% of total)

Am I correct in assuming that this change should fix it?


Is maintenance stopped

It's almost a year since it was updated

Hi<EMAIL_ADDRESS>It might look like it's stopped but no, I do still watch this project and work on issues.
I've had fairly limited time for a while so haven't been able to work a lot on adding features.
If/when that changes there's a bunch of things I'd like to get in there.

Already have an addon where changes to devices can be described in protobufs, this with things like , waiting , re-trying and chaining things together .. Makes it easier to build up libraries of common operations.
Multi platform support.. With other terminal libraries out that handle multiple  platforms expect could change over to one of those and get the multiplatform for free.
Re-factor, this project was written quite a while ago, both my way to code and Go best practices has changed since , would be nice to get the code up to current standards.

PRs are always welcome of-course.


tests/dns: add tests for task 7018 - v2

Previous PR: https://github.com/OISF/suricata-verify/pull/1897/files
Describe changes:

differentiate flow checks for 6.0.x, since tcp.*_max_regions are from 7.0.x onwards
rebase

Ticket
Redmine ticket:
https://redmine.openinfosecfoundation.org/issues/7018

Suricata PR already merged xD

Feedback finally incorporated with https://github.com/OISF/suricata-verify/pull/1967


Query::all() with iterator result

What steps will reproduce the problem?
Yii 2.x (and also Yii 1.x) Query::all() result is very limited.
Main problems:

loading all record it once

larger memory requirement
Many records require batch and similar solutions

does not support scanning

If result change to Iterator class:

ActiveRecord populate only in sequence, only at one per time
store only raw attributes array
support scan based database
rewind disabling option: less memory, allowing scan by infinity
include batch and each functionality
ActiveRecord populate error is skippable
optional meta data

eg. google cloud bigquery add more meta fields to result (example: processed bytes, next page token, cache life)

What is the expected result?
example: Console process read normally, and support infiniti of records.
$iterator = $query->all();
if($iterator->databaseSpecificMetadaData /* limited result */) {

}
foreach($iterator as $key => $model) {
  // loop:
  // 1. next
  // 2. ActiveRecord popuplate - if required
  // 3. use model
  // next loop
}

What do you get instead?
example Console process require batch reading
Additional info
example scanning from Google Cloud Bigtable.

Q
A

Version
1.x 2.x

PHP version
-

Operating system
-

Suggest
class Query {
  public function all() {
    return new QueryDataProvider($this);
  }
  public function one() {
    $dp = new QueryDataProvider($this);
    return $dp->first();
  }

  public function count() {
    $dp = new QueryDataProvider($this);
    return $dp->count();
  }
}

class QueryDataProvider extends DataProvider implements \Iterator { // or best practive DataProvider is \Iterator (this may already be true.)
  protected $query;
  protected $rowCache;   // raw data cache (attributes only)
  function __counstruct($query) {
     $this->query = $query;
  }
  public function count() {
    return $this->query->(yii2)count();
  }
  public function rewind() {
    if($disabledrewind && $rowCache !== null) {
       throw new NotSupportedException('rewind disabled!');	// For a simpler example
    }
    ...
    $this->next();
  }

  public function valid() {
    return sizeof($this->rowCache);
  }

  public function next() {
    if($this->rowCache === null) {
      $this->rowCache = $this->query->(yii2)asArray()->(yii2)all();
    } else {
      array_shift($this->rowCache);   // rewind disabled, drop previous elements
    }
  }

  public function key() {
    reset($this->rowCache);
    return key($this->rowCache);
  }

  public function current() {
    $row = reset($this->rowCache);
    // ActiveRecord::populate($row) if ActiveQuery (populate on the fly each element)
    return $row;
  }

}

Try this
$dataReader = $query->createCommand()->query();

foreach ($dataReader as $row) {
   ...
}

Also QueryDataReader can be found in yiisoft/data-db package
https://github.com/yiisoft/data-db/blob/master/src/AbstractQueryDataReader.php


Lockers (all pushables?) can be pushed through walls

I've found lockers can be pushed through walls in many instances, mostly up and down but up is easier.
The best way to reproduce is to try to push one through the boundary of two wall tiles.
I've yet to be able to do it with water/welding tanks or crates, so it's either lockers are thin enough to make it through or there's some snowflake behavior on them.
Video: https://cdn.discordapp.com/attachments/675078881425752124/731116771310043146/2020-07-10_12-50-43.mp4

It's also pretty easy to push them onto a table.

Turns out this is a duplicate of https://github.com/space-wizards/space-station-14/issues/1327, only it's visible at the server default of 30 tps.

This is a lot easier to replicate now.

Tried for a while to replicate on current master, looks to be only present in very uncommon situations. (Really really bad TPS which there's already and issue for #1327 ) Or when there's like 50 pushables in the same area. Closing.


show-open-prs-of-forks includes unrelated forks

Visit one of your (source) repos where you opened PRs
Fork it
Open it
Notice: your existing PRs are counted in this feature

In my case, I have this dummy fork https://github.com/fregante/refined-github but all of PRs are directly on the source (https://github.com/sindresorhus/refined-github)

cc @loilo

Fair point. I'll take care of it. 🙂

Oof. From what I can tell, GitHub has no search filter in place to limit results to a certain head repo.
This creates two problems:

We can't determine the exact number of open PRs without fetching a lot more data from the API than we do now. (At least I think we can't. There may be a possibility to utilize the GraphQL API in a way which can filter for certain nested node values. But since I have no considerable knowledge of GraphQL, I can't tell whether or not this is possible.)
We also can't reliably link to a list of open PRs since that link is entirely composed of issue search filters.

I'm not sure how to handle this.
We could solve part of it by checking if the user has write access not only to the forked repo but also to the source repo. This should be solvable in one API request and could be cached for a long time.

If the user has no write access to the source repo, we should be fine keeping the current process in place.
If the user has write access to the source repo (as in your reproduction setup), we may consider fetching individual pull requests from the API and filtering them by head repo on the client side. We then could forego linking to a list of PRs and just tell the user about the fact that there are open PRs.

What do you think @fregante?

Here: https://developer.github.com/v4/explorer/
We'd have to fetch all your PR info, not just the first one, but it's not a lot of data.
query {
		search(
			first: 100,
			type: ISSUE,
			query: "repo:sindresorhus/refined-github is:pr is:open author:fregante"
		) {
			nodes {
				... on PullRequest {
					baseRepository {
						nameWithOwner
					}
					number
				}
			}
		}
}

{
  "data": {
    "search": {
      "nodes": [
        {
          "baseRepository": {
            "nameWithOwner": "sindresorhus/refined-github"
          },
          "number": 2925
        }
      ]
    }
  }
}

Then something like:
const prs = search.nodes.map(pr => pr.baseRepository.nameWithOwner === getRepoURL());

Yes, I had tried it out in the explorer, that's basically how I imagined it to be implemented. The "lot" of data was more in comparison with before, and it grows linearly with number of PRs.
I think this would be a reasonable implementation. In the (probably a bit absurd) case that somebody has > 100 open PRs, I'd bail out of counting.
How'd you deal with the link problem?

I don’t expect anyone to have a considerable amount of PRs open from a fork, they’re distant outliers.
As for the link, there’s no direct fix, but:

I’d expect people to open PR from the fork OR the source, not at the same time (generally)
if they do have mixed repos, our pr-branches feature will help them understand which is which

Alternatively, I think that the search accepts searching by number and logical operators. If that works, we could link to the search issue:2 OR issue:30 OR issue:40

Doesn't seem to be possible though. 🙁

If some one wants to send a PR I tested https://github.com/sindresorhus/refined-github/issues/2934#issuecomment-604146813 and it looks fine to me. (I changed it to headRepository because I think that is correct)
https://github.com/yakov116/refined-github/commit/b698cf3f3e8f2f5d38e815f4eedbd2cb9aa47888
@@ -17,24 +17,28 @@ const countPRs = cache.function(async (forkedRepo: string): Promise<[number, num
 	// This allows to link to the PR directly if only one is found
 	const {search} = await api.v4(`
 		search(
-			first: 1,
+			first: 100,
 			type: ISSUE,
 			query: "repo:${forkedRepo} is:pr is:open author:${getUsername()}"
 		) {
-			issueCount
 			nodes {
 				... on PullRequest {
 					number
+					headRepository {
+						nameWithOwner
+					}
 				}
 			}
 		}
 	`);

-	if (search.issueCount === 1) {
-		return [1, search.nodes[0].number];
+	const prs = search.nodes.filter((pr: AnyObject) => pr.headRepository.nameWithOwner.toLowerCase() === getRepoURL());
+
+	if (prs.length === 1) {
+		return [1, prs[0].number];
 	}

-	return [search.issueCount];
+	return [prs.length];
 }, {
 	maxAge: 1 / 2, // Stale after 12 hours
 	staleWhileRevalidate: 2,


[Fix #1078] Fix false negative for Rails/LexicallyScopedActionFilter

Resolves https://github.com/rubocop/rubocop-rails/issues/1078
This PR fixes the edge case where the class/module body does not define any methods at all.

Before submitting the PR make sure the following are checked:

[x] The PR relates to only one subject with a clear title and description in grammatically correct, complete sentences.
[x] Wrote good commit messages.
[x] Commit message starts with [Fix #issue-number] (if the related issue exists).
[x] Feature branch is up-to-date with master (if not - rebase it).
[x] Squashed related commits together.
[x] Added tests.
[x] Ran bundle exec rake default. It executes all tests and runs RuboCop on its own code.
[x] Added an entry (file) to the changelog folder named {change_type}_{change_description}.md if the new code introduces user-observable changes. See changelog entry format for details.
[ ] If this is a new cop, consider making a corresponding update to the Rails Style Guide.

Thanks!


Fix repr of Resolvers

There is a "," missing between module and subdirs in the ResourceResolver.__repr__.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>6
Details

0 of 0   changed or added relevant lines in 0 files are covered.
1 unchanged line in 1 file lost coverage.
Overall coverage decreased (-0.2%) to 95.0%

Files with Coverage Reduction
New Missed Lines
%

resources.py
1
95.28%

Totals

Change from base Build<PHONE_NUMBER>0:
-0.2%

Covered Lines:
532

Relevant Lines:
560

💛  - Coveralls


Please add some example code for SVAR-FCI.

Request from a user. -JR

I added a simple SVAR-FCI example to run_continuous.py. Please ask for elaboration if this is not clear enough to use.


some special arguments do not work fine in channel.QueueDeclare

I tried demo code below:
package main

import (
    "github.com/streadway/amqp"

    "os"
    "fmt"
    "time"
)

func main() {
    url := os.Getenv("CLOUDAMQP_URL")
    if url == "" {
        url = "amqp://localhost"
    }
    connection, _ := amqp.Dial(url)
    defer connection.Close()
    go func(con *amqp.Connection) {
        channel, _ := connection.Channel()
        defer channel.Close()
        durable, exclusive := false, false
        autoDelete, noWait := true, true
        q, _ := channel.QueueDeclare("queueName", durable, autoDelete, exclusive, noWait, nil)
        channel.QueueBind(q.Name, "#", "amq.topic", false, nil)
        autoAck, exclusive, noLocal, noWait := false, false, false, false
        messages, _ := channel.Consume(q.Name, "", autoAck, exclusive, noLocal, noWait, nil)
        multiAck := false
        for msg := range messages {
            fmt.Println("Body:", string(msg.Body), "Timestamp:", msg.Timestamp)
            msg.Ack(multiAck)
        }
    }(connection)

    go func(con *amqp.Connection) {
        timer := time.NewTicker(1 * time.Second)
        channel, _ := connection.Channel()

        for t := range timer.C {
            msg := amqp.Publishing{
                DeliveryMode: 1,
                Timestamp:    t,
                ContentType:  "text/plain",
                Body:         []byte("Hello world"),
            }
            mandatory, immediate := false, false
            channel.Publish("amq.topic", "ping", mandatory, immediate, msg)
        }
    }(connection)

    select {}
}

and what a weird question was it didn't work when the first argument 'name' of channel.QueueDeclare value was "queueName", the others like "queueN","queueNam" would be fine.
When the 'name' arg value is "queueName", call channel.QueueDeclare will catch an error:
Exception (406) Reason: "PRECONDITION_FAILED - inequivalent arg 'auto_delete' for queue 'queueName' in vhost 'vhost': received 'true' but current is 'false'

The error contains an answer: you are trying to re-declare a queue using a different set of properties. Either use the properties that match or delete the queue and re-declare it again. There is nothing specific to this client.

Yes, you are right. My made mistake.


[docs-infra] Improve the heading buttons positioning

[x] I have followed (at least) the PR section of the contributing guide.

Was again just browsing the docs looking for improvement opportunities and realized that the anchor button for the heading was again misaligned (this seems to happen whenever changes happen to the heading styles). 😭 Figured that using an inline-flex for the button itself and other flex-related properties would ensure the alignment instead of an arbitrary line height ⎯ similarly with the heading itself instead of absolute positioning the two.
Not fully aware of whether these changes have unintended consequences, so... let me know if this works out!
https://deploy-preview-38428--material-ui.netlify.app/experiments/docs/headers/

I see 3 regressions compared to HEAD:

Collapsed anchor link look

https://github.com/mui/material-ui/assets/3165635/c75117cb-bb3c-4379-a4b9-7bd01a19d2a2

Vertical layout shift on hover

https://github.com/mui/material-ui/assets/3165635/f5bf4b9c-5fde-449c-a7cb-f356f8602cd7

Unstable layout on hover

https://github.com/mui/material-ui/assets/3165635/6fefd910-1611-4924-82e0-3678b60a4043

I wonder if the solution to issues 1 & 3 listed above isn't just making the anchor button visible by default on mobile? Both of them are related to hovering on mobile breakpoints, which is not really a thing...

@alexfauquette whenever you have the time, let me know if this makes sense! :)

Point 3, is still present no?
https://github.com/mui/material-ui/assets/3165635/78729567-7170-4da3-bbbf-6728a13a3e66
https://deploy-preview-38428--material-ui.netlify.app/material-ui/getting-started/faq/#legacy-i-have-several-instances-of-styles-on-the-page

@oliviertassinari Uhm, yes... 😅 Big titles get affected. I'll work on it today, given this has been merged ⎯ otherwise, we can just revert this PR. Taking advantage though, any way you visualize we can solve this, with flex?


proposal: testing: report energy consumption figures

I propose to report energy consumption figures in Go benchmarking machinery. This will allow Go developers to have some estimate of energy consumption of their programs as it's been shown that data movement accounts for 62.7% of energy usage in consumer devices. With energy factors becoming more and more important it's proved that impact of high-level code on energy is orders of magnitude higher than low level/OS management etc.
With given allocated bytes in memory and ns/op I believe energy figures can be derived easily, however, architecture in use could have significant impact on these figures.
For instance, the following snippets show two different implementations where the first one uses variables locally and the second uses channels instead of variables. In terms of performance, the first one beats the second one by a factor of ~1000x (using Go's benchmarking tool). However, the second one avoids allocation of big data arrays and exploits serialisation (less random access), therefore I assume it must show better energy figures for real-world benchmarks:
func WithVariable() (int, int) {

	a := 2 + 3
	b := a + 3
	c := a + 5
	return b, c
}

func WithoutVariable() (int, int) {

	a := make(chan int, 2)
	a <- 2 + 3
	a <- 2 + 3
	return (<-a + 3), (<-a + 5)
}

I can't even imagine the second one showing better energy figures. Actual CPU time is a fairly solid proxy for energy consumption.

I agree with @seebs .  I do not know of any reliable energy measurement that significantly differs from CPU time.  And if there were one, it could differ based on the smallest details of the specific processor implementation and external memory and system bus, etc., details that the Go standard library does not currently check and that may not even be available.
So I think this proposal can not be adopted without a clear understanding and explanation of how to implement it.

This also depends on the target machine's power saving strategy. For example, most recent Android devices will try to go into "deep sleep" as often as possible, so any app or process that wakes up the device regularly is going to use more battery. On the other hand, laptops don't generally do deep sleep, but they do spin their fans and throttle when heat increases due to CPU usage spikes.
Seems to me like the best way to measure energy consumption is to actually run your program for a while under realistic conditions and measure the amount of energy drawn.

@mvdan 'run your program for a while' isn't that what go test -run=x -bench=. dose?
@ianlancetaylor agreed, as I mentioned in the proposal it's gonna be architecture dependent. What I would like to see is see rough energy figures. The reported performance figures aren't accurate either, are they? it all depends how the OS schedules the underlying threads..
@seebs the above snippet is just an example to show how serialisation via channels vs. random access could affect performance and energy. The effect could be not as bad as 1000x when big arrays are compared..

I can't even imagine the second one showing better energy figures. Actual CPU time is a fairly solid proxy for energy consumption.

The Energy equation: Energy (J) = Power (W) x Time(s). Yes, reducing time implies a reduction in the energy consumed but the Power variable is not a constant. I have seen works that show time and energy could be in opposite direction too.

I would expect that in general, channels are going to be much more expensive than direct operations -- they have synchronization overhead, etcetera.
What I think you're getting at is a general contrast between holding large amounts of memory to operate on, contrasted with streaming operations which operate on only a small amount of data at a time, thus, use less storage?
I think it might be useful-ish, where possible, to expose performance data based on CPU performance counters or OS-level measurements of CPU time actually spent executing a process, as opposed to wall-clock time. Energy estimation might be much, much, harder -- we don't actually have particularly solid information on things like relative energy consumption of individual instructions in multicore systems, or a way to get it, apart from overall measurements of power consumption when running various workloads.
In practice, I think as long as you measure total CPU time used by the process, you're going to be fairly close to an accurate measure of approximate energy consumption. (Less so if, say, a GPU is also involved -- offloading things to a GPU might make runtime faster but cost significant energy to do so.)
See also: https://github.com/aclements/perflock, and perf.

@Mahdi89 The reported performance figures are as accurate as the OS provides.  Dependencies on thread scheduling are an issue with how repeatable the numbers are, not how accurate they are.  And we have tooling to deal with repeatability, such as https://godoc.org/golang.org/x/perf/cmd/benchstat .
But unlike performance, which we can measure directly using tools provided by the OS and the CPU, I do not know of any way to measure energy.  My comments about CPU details were intended to point out that estimating energy use is infeasable.  So if we can't measure it, and we can't estimate it, I don't see what useful information we can provide.a

This also depends on the target machine's power saving strategy.

Both iOS and macOS prefer to sleep when possible. To that end, they offer APIs for approximate timers (“do this every five minutes or so”) so that they can coalesce timers, wake up, do lots of work, and go back to sleep. It might be interesting to investigate Go support for those, particularly for iOS.
But to repeat Ian, I don’t see how to measure power consumption even moderately reliably such that it could be reported at the end of a benchmark.

@ianlancetaylor we might not be able to measure it but I think we can estimate it, e.g. using an analytical model for the Power (W) variable based on allocated memory, ns/op and number of memory operations vs. number of non-memory operations. Next, we already have the Time(s) variable to draw total energy.
@josharian how many power saving strategies are out there? aggressive, moderate and non I assume what developer would want to know.
@seebs exactly, by streaming data less storage will be needed.

@Mahdi89 I do not believe that an estimate would be useful, for various reasons mentioned above by various people.  If you want to propose measuring allocated memory, ns/op, and number of memory operations, propose that.  Let's not try to pretend that we are measuring energy use when we are measuring something else.

Amount of memory in use by a single program has very little effect on energy consumption. Amount of  memory physically installed affects power consumption, amount of memory allocated doesn't directly matter much; we can't take memory out when it's not in use, and i don't think any current systems can power-save individual blocks of memory. So if you're on a machine with 8GB of memory, it draws about the same amount of power no matter how much of it is "allocated".

@ianlancetaylor I don't think if that's a pretension, it's more of an approximation that requires to get as accurate as possible by collecting more and more parameters. These need to be observed by the current library over time I believe. It should give the programmer a sense to have better notion of the energy consumed by his/her program.

@seebs regarding execution time I just did this simplistic experiment to show how the concept of using channels gets interesting when dealing with bigger arrays: https://github.com/Mahdi89/vartest

Can you clarify why you think that memory usage is lower for make(chan int, 10000) than for make([]int, 10000)?

Okay, looking over this, I think I understand. The underlying contention is that reduced memory usage would result in reduced energy consumption. I don't think that's generally true for specific computers. Reducing overall memory footprint of everything on a system might permit you to build systems with less memory in them, reducing energy consumption a bit, but reducing memory usage for individual apps is unlikely to significantly impact energy consumption in most cases. (There's some edge cases, like memory usage which causes a system to start swapping.)
But there's a secondary issue, which is that you keep using buffered channels to illustrate your contention. But a buffered channel still has that large storage space. So all you're doing is replacing simpler operations with more complex operations -- you aren't even reducing the amount of storage in use.
I also think you're seriously underestimating the cost of channels compared to the underlying operations. They're cheap by standards of concurrency systems, they're still very expensive by comparison to the underlying computations.

May I also suggest that, while example A may use more watt hours than example B, that's not a very interesting observation? If B is 1000 times slower, it is not a better solution from an energy efficiency perspective unless it also uses 1000 times less power. To illustrate this, consider that this implementation likely uses less power than both versions originally posted:
func SleepyVariable() (int, int) {
	a := 2 + 3
	time.Sleep(1*time.Second)
	b := a + 3
	time.Sleep(1*time.Second)
	c := a + 5
	time.Sleep(1*time.Second)
	return b, c
}

It's also relatively more expensive because you still have to power the machine to do basically nothing most of the time. To that end, I see the more interesting measurement to be computation performed per watt hour. If you use instructions retired as a proxy for watt hour usage, then what you should be looking for is maximizing instructions per cycle efficiency. That will give you the most cost-effective implementation.

@seebs you're right, that would lead to systems with less memory in them. e.g. In FPGA based systems buffered channels get mapped onto on-chip FIFOs, so data remains local to the computation and data movement from/to off-chip memories (e.g DRAM) gets significantly reduced.
There is another aspect to using channels vs variables/arrays: random access patterns. Channels force processes to communicate serially. That alleviates the degree of randomness, and hence, data movement across the system. Something similar to storage-less or 'hot-potato' communication: https://users.ece.cmu.edu/~omutlu/pub/bless_isca09.pdf

@dhobsd I agree with this: 'If B is 1000 times slower, it is not a better solution from an energy efficiency perspective unless it also uses 1000 times less power'. Now, why do I think B could be more power efficient than A: I believe channels utilise on-chip memories (caches) better than random access arrays allocated on off-chip memories (DRAM) as they force serial communication they should have better cache utilisation therefore lesser data movement will be required when dealing with bigger data. This idea stems from the fact that in consumer devices almost two/third of the total energy usage is due to data movement.
By the way, this gap of 1000x starts to reduce when considering bigger data: https://github.com/Mahdi89/vartest

Package testing reports what the CPU or operating system can tell us.
If the CPU or operating system can start giving us accurate energy estimates
(for what happened during a two-second benchmark)
then great, let's have a discussion about how to report that number.
If not, there does not seem to be a compelling reason for us to try
to develop our own attempts at an accurate estimate. (They won't be accurate!)

@rsc agreed, our estimates won't be accurate enough, however as a developer I'd like to see some estimates to have an idea what implications of my data structure use is on energy. For instance, I've realised people use channel based implementation free-lists to keep objects around and avoid GC. So I assume channels are cache friendlier and they can reduce data movement across the system (which is the main source for energy consumption).

@Mahdi89 It seems to me like you're basing your hypothesis on pure guesses. In practice, channels are a bad example, as they're expensive compared to simpler structures like slices or arrays.
The proposal has been declined, so I'd suggest you continue the discussion elsewhere, like golang-nuts. If you'd like to open another proposal or bug in the future, perhaps try to gather pieces of code which reliably differ in CPU cost and energy cost. You'll need hard evidence if you want a proposal this ambitious to be considered.


As a user, I want to mark a completed task as incomplete because I realised that it is not fully completed

Complete undoneCommand to support this feature


License

Hi I was wondering what license this code is under?

Hey, I second this, as I would like to use your nice color picker in one of my open-source projects.
@rshest could you please clarify the license? Thanks!

Done, sorry for only getting to this now.

Thanks!


(PUP-9915) service provider: sync defaultfor :debian

Make the systemd defaultfor the opposite of debian defaultfor. This
should avoid the need for updates with every future Debian release.

Can one of the admins verify this patch?

CLA signed by all contributors.

jenkins please test this on debian-8-amd64,debian-8-i386,debian-9-amd64,debian-9-i386

Hi @zeha,
Thanks for your contribution! I triggered an acceptance test run with debian hosts just to be safe.
When everything passes, please change the base branch and rebase your commit on 5.5.x, so we would have this improvement in all releases.
Thanks!

@GabrielNagy I can't really look at the jenkins error, can you take a look?

jenkins please test this on debian8-64a,debian8-32a,debian9-64a,debian9-32a

@GabrielNagy I can't really look at the jenkins error, can you take a look?

Sorry, I triggered the tests with the wrong target strings :worried:
Should run fine now.

jenkins please test this on debian8-64a,debian8-32a,debian9-64a,debian9-32a

Looks good!
@zeha, please rebase this on 5.5.x so we can merge it. Thanks!

Thanks. @GabrielNagy I've rebased this on top of 6.0.x.

jenkins please test this on debian8-64a,debian8-32a,debian9-64a,debian9-32a


Check if FD_SETSIZE is already defined before trying to define it

If FD_SETSIZE is already set, undefine it before trying to define it
(mutes a windows warning in gitian log)

@mitchellcash want to test this on mingw (can't imagine it would cause anything but just for good measure)?

@IngCr3at1on the build was fine so I'm happy for this get in.


Error: social media icon are not updated

###update the social icons
In the footer section there are social media icon but the twitter logo is not updated
Here's the image of it:

Please assign this to me and also add label

@Antima2004 This issue is already open and assigned to me under Issue #634. Please go through the existing Issues before submitting your issue. You can close this issue and find a new one if you want.

I have to close this issue or i can do @sanjay-kv

yes you can do

Ok
Thanks a lots

Hey
@sanjay-kv
Its already updated, twitter logo how can i add??


Claims - Use singular naming for individual objects in GetClaims operation

When examining operations that return lists. For example, the GetClaims operation returns an object named "ClaimListDetails," which contains a list of objects also named "ClaimDetails." I suggest changing the name of the individual claim object to "ClaimDetail" for the following reasons:
Clarity and readability: Using singular names for individual objects makes the code more readable and easier to understand. The singular name "ClaimDetail" clearly communicates that it represents a single claim detail, whereas "ClaimDetails" could be misinterpreted as a collection of claim details.
Consistency and best practices: It is a common convention in most programming languages, including C#, to use singular names for classes or objects that represent single instances. Following this convention makes your code more consistent with other projects and libraries, which helps other developers quickly understand your code.
Logical separation: Using singular names for individual objects and plural names for collections or containers allows for a logical separation between the two concepts. This distinction helps developers understand how your code is structured and makes it easier to identify the relationships between objects and collections.
In summary, I suggest changing the name of the individual claim object to "ClaimDetail" to improve code readability, align with programming best practices and conventions, and help logically separate individual objects from collections.
The same issue applies to "ClaimTemplateList" and "ClaimTemplateListDetails.

Meeting TN-FMÞ-VH7-16 suggested taking a closer look at the IOBWS3.0.yaml to use as a comparison, where it seems that accountdetails is found in similar usage.

No need to Change


Clarification on Inference Implementation in VOC (inference.py)

Hi,
Thank you for the fantastic work and for making the code available. I have a question regarding the inference.py script in the VOC implementation.
Based on the paper, you mentioned that the soft segmentation map is generated by the ensemble of the mean teachers (Equation 4 in the paper) during inference. However, in the current implementation, it seems that only the prediction from Teacher 1 is being used. Below is the relevant code snippet from inference.py:
with torch.no_grad():
    prediction = model.module.decoder1(model.module.encoder1(data),
                                       data_shape=[data.shape[-2], data.shape[-1]],
                                       req_feature=True)

prediction = torch.nn.functional.interpolate(prediction, size=(H, W),
                                             mode='bilinear', align_corners=True)

Shouldn't the inference consider the predictions from both teachers to align with the ensemble approach described in the paper? Or am I misunderstanding something?
I would appreciate your clarification on this. Thanks

Hi @codevisioner
Sorry for the late reply.
One of the reviewers for this paper raised concerns about the fairness of the comparison, stating that utilising two networks to produce an ensemble result is not fair compared to other works, especially under the sliding evaluation scenario. Therefore, the results for the Cityscape dataset are based on a single teacher. Please note that with the ensemble results of the two teachers, I believe you will achieve a better mIoU, and you are welcome to use them if you wish.
Thanks for your interest of the work.
Cheers,
Yuyuan


Update geniaaz.md contributions

Update as of 19/10

Codecov Report

Merging #144 into master will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master     #144   +/-   ##
=========================================
  Coverage     60.14%   60.14%
  Complexity      629      629
=========================================
  Files           111      111
  Lines          2271     2271
  Branches        280      280
=========================================
  Hits           1366     1366
  Misses          831      831
  Partials         74       74

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 10b4417...d254ada. Read the comment docs.


[INLONG-9508][Manager] Add Iceberg field type mapping strategy to improve usability

[INLONG-9508][Manager] Add Iceberg field type mapping strategy to improve usability

Fixes #9508

Motivation
Inlong Iceberg ExtractNode or LoadNode currently does not support some special field types, add Iceberg field type mapping strategy to mapping to the corresponding type to run correctly

@chestnut-c please fix UT

@chestnut-c please fix UT

ok

close this PR, Commit it  again later


parsing blob files, possible with this lib?

Hello. Dealing with some hisilicon chips. they generally initalize a number of registers using a sort of binary
csv file with the following format:
struct regentry {
    uint32_t reg_addr; // address of the register
    uint32_t value; // value to be written or read
    uint32_t delay; // amount of nop's to execute to delay
    uint32_t attr; // read or write and other flags
};

I've been putting together a svd file for the soc for use with the PyCortexMDebug project, and was hoping
this may allow for me to use said svd file to parse out the actions taken instead of manually figuring the bitfields.
for reference:
00000000  80 00 01 12 00 00 00 00  00 00 00 00 fd 00 00 00  |................|
00000010  e0 01 01 12 00 00 00 00  00 00 00 00 00 00 55 00  |..............U.|
00000020  00 00 00 00 00 00 00 00  64 00 00 00 00 00 00 00  |........d.......|
00000030  10 00 01 12 55 55 d5 12  00 00 00 00 fd 00 00 00  |....UU..........|
00000040  14 00 01 12 2d 10 00 00  00 00 00 00 fd 00 00 00  |....-...........|
00000050  00 00 00 00 00 00 00 00  64 00 00 00 00 00 00 00  |........d.......|
00000060  e8 01 01 12 0f 00 00 00  00 00 00 00 00 00 1d 00  |................|
00000070  80 00 01 12 9b 05 00 00  00 00 00 00 fd 00 00 00  |................|
00000080  e0 01 01 12 9b 05 00 00  00 00 00 00 00 00 55 00  |..............U.|
00000090  00 00 00 00 00 00 00 00  64 00 00 00 00 00 00 00  |........d.......|

Note these values are encoded in little endian.

This is possible. Had a project,where I got a list of addresses for my MCU where I needed to translate those addresses back to a readable name.
The general outline is to first iterate over all peripherals, collect their memory range and if your address lies within that range, iterate over all of the peripheral's registers (by adding their offset to the peripheral's base). If you find a match there, than your address belongs to the peripheral and relates to the register you found.
There are ways to improve this general algorithm by building a search tree over those values you have to check, but the general idea stays the same.

Thanks @BenBE. I'm not much of a pythonist, do you think you could provide a small snippet as
an example of how one would do this?

Awright. I've gotten to the point where I have a reg address (Peripheral base address + register address) and
a SVDRegister object, and value read/written, along with bitrange, in a context. Now I just need to figure out
how to parse out the bitrange of the value into fields and their enumerated values, if possible. hrm.


Support for cpf validation

Support to validate CPF fields (Brazilian personal registration number).

Codecov Report

Merging #27 into master will decrease coverage by 4.02%.
The diff coverage is 78.26%.

@@            Coverage Diff            @@
##             master   #27      +/-   ##
=========================================
- Coverage     94.02%   90%   -4.03%
- Complexity       38    51      +13
=========================================
  Files            10    11       +1
  Lines            67    90      +23
  Branches          9    18       +9
=========================================
+ Hits             63    81      +18
  Misses            3     3
- Partials          1     6       +5

Impacted Files
Coverage Δ
Complexity Δ

...c/main/java/convalida/validators/CpfValidator.java
78.26% <78.26%> (ø)
13 <13> (?)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3a95796...c13dc1c. Read the comment docs.


Gbou/add grouping column features in table component without column group in solution prod 10430

Before creating a PR, please check that:

[ ] Code is clean
[ ] Tests are still working (cypress tests and yarn test)
[ ] Changes don't cause new errors or warnings in browser console
[ ] Documentation is up-to-date
[ ] Breaking changes are clearly identified with conventional commits

@Guismus could you add a commit to update the documentation file doc/scenarioParametersConfiguration.md (after section Columns definition) and provide a description of this new feature with some examples?


BIGTOP-3066: Bump tez to 0.9.1

Bump to 0.9.1 for 1.3 release.
Change-Id: I9ee682bdf17dde6f768d64538776643ae4d4dcbf
Signed-off-by: Yuqi Gu<EMAIL_ADDRESS>

+1