Add godbolt.ms as a remote compiler explorer!

Whee!

As discussed on Slack (So it does not go forggoten):
It's worth switching every compiler we have running on wine to MS. To keep old links working, the ids on MS's side have to either be the same, or be added as aliases.
Ideally we would end up getting the new compilers + a 1 for ~1 instance of the versions we do currently support.

@partouf yeah, and the people at Microsoft who are working on this now would have to add MSVC 15.0 and 15.4, if they wanted to replace the two VS2017 compilers. How would they alias the old names to the new names, tho?

@ubsan Compilers have an alias property (for example: https://github.com/mattgodbolt/compiler-explorer/blob/master/etc/config/c%2B%2B.amazon.properties#L374) that can be set, and any compiler referring to the alias will find it.

We are working on getting https up on the Microsoft hosted site, I will let you know when there is an update.  It would be helpful if we could talk to someone about how it is handled on your site because this is all new to us compiler devs.  Would @mattgodbolt be the best one to talk about that?
Also yes to your privacy policy.  It would probably be good to add a link to http://privacy.microsoft.com and explain what it is doing.  Would you like us (Microsoft) to update that, or can it be added to this PR?

We are working on getting https up on the Microsoft hosted site, I will let you know when there is an update. It would be helpful if we could talk to someone about how it is handled on your site because this is all new to us compiler devs. Would @mattgodbolt be the best one to talk about that?

As far as I know, the node webserver is not set up to handle https (the library we use can do it, but we don't use it). I think Matt has setup an Amazon/Cloudfront proxy thing to handle https requests and forward them to the node webserver.
I'm not sure what you use, but within Azure there are options to do the same https://docs.microsoft.com/en-us/azure/application-gateway/create-ssl-portal (I've never tried this)

Also yes to your privacy policy. It would probably be good to add a link to http://privacy.microsoft.com and explain what it is doing. Would you like us (Microsoft) to update that, or can it be added to this PR?

Also yes to your privacy policy. It would probably be good to add a link to http://privacy.microsoft.com and explain what it is doing. Would you like us (Microsoft) to update that, or can it be added to this PR?

We can add it to this PR too. Be aware changing the PP will cause popups to re-confirm it when we merge. See also #1109 to maybe consider mitigating this.
Thanks again for your help on this :)

Hi!
Seems like https is now available on the MS link! Yay!
I'll hold off merging this until I get the OK from MS: both on the wording for the PP change and the "ok" to actually link the sites!
I guess I'll need to know who my point of contact is if I need to change anything on the MS side of things too.
Thanks! Exciting times!

After this is merged, I'd like to have the properties file from the MS instance in the rep, or a separate fork if needed. Would make it easier to communicate the compiler aliasing, changes to CE that affect the MS side and future binary support.

You can use me as the contact at Microsoft (philc@microsoft.com), you also have Dale's contact information from our SSL email chain.
You have the OK on our side to link the sites.  I will need to get back to you on the privacy policy though.

Thanks: have emailed you both. Hope to get this merged soon (esp once #1125 is merged too)

I have updated the PP with the wording we discussed over email, and if you wish to have it changed of course we will. I'm going to merge this but will continue to discuss the other issues, namely:

knowing which version of CE is on www.godbolt.ms
knowing the properties file (per @partouf's comment)
also we'll need to think at some point in the future about how the libraries one can pick could work.

Thanks again for your work on this! I'm super excited and hope to roll this out this evening!


Dev Spaces Content Validation Issue for github.io website

Section Links:

https://azuresdkdocs.blob.core.windows.net/$web/python/azure-mgmt-devspaces/1.0.0b3/index.html

Reason:
Format validation Issue: Extra symbol _.

Section Link:

https://azuresdkdocs.blob.core.windows.net/$web/python/azure-mgmt-devspaces/1.0.0b3/genindex.html

Reason:
Format validation Issue: Text exceed the page size.

Section Link:

https://azuresdkdocs.blob.core.windows.net/$web/python/azure-mgmt-devspaces/1.0.0b3/azure.mgmt.devspaces.operations.html

Reason:
Format validation Issue: Extra symbol -.

Section Link:

https://azuresdkdocs.blob.core.windows.net/$web/python/azure-mgmt-devspaces/1.0.0b3/azure.mgmt.devspaces.models.html

Reason:
Typing annotation Issue: Missing type annotation for parameter value.

Section Link:

https://azuresdkdocs.blob.core.windows.net/$web/python/azure-mgmt-devspaces/1.0.0b3/azure.mgmt.devspaces.operations.html

Reason:
Typing annotation Issue: Reason: Please unify the format, update to *args:Any, **kwargs:Any or *args, **kwargs.
@rohit-ganguly , @lmazuel for notification.

Close this issue as we'll put it into a summarzied one.


Support for content-encoding gzip and deflate

self-explanatory

+100

closed by 8bcac5f

there is still an issue in 8bcac5f, Content-encoding: deflate has to use zlib.inflate instead zlib.gunzip.


Manage Database Connections Across Entire System

MMS is maintaining 2 database connections per project.  On a system with hundreds of projects, this becomes a problem.  This is an attempt to enforce some kind of limitation on the number of idle connections without sacrificing too much performance.

There will most likely be a couple conflicts once the release branch is merged into develop. This PR will have to wait for the release.

Ok, let's just fix the formatting and it looks good to me.

Requested changes completed.  Thanks!


[New Port Request] libtcc

Library name : libtcc
Library description : The libtcc library enables you to use TCC as a backend for dynamic code generation. The idea consists in giving a C string containing the program you want to compile directly to libtcc. Then you can access to any global symbol (function or variable) defined.
Source URL : http://download.savannah.gnu.org/releases/tinycc/
Source GIT Repository URL :-
https://repo.or.cz/tinycc.git
Project homepage : https://bellard.org/tcc/
Anything else that is useful to know when adding : latest version is 0.9.27 & it is also the last version.

is it still developed?

[Note: I am no longer working on TCC. Check the mailing list to get up to date information.]
~cite https://bellard.org/tcc/

I think that this would be appropiate for a custom registry since we had the discussion in the past:
https://github.com/microsoft/vcpkg/pull/17540#issuecomment-829462086
https://github.com/microsoft/vcpkg/pull/17540#issuecomment-829608222

But libTCC is not experimental, it's been there for long time and last update was in 2020, and it works fine, it mostly supports c99 and is stable enough to be a vcpkg port.
It's open source so chances are that someone may decide to maintain it in future.

@Tanishq-Banyal yeah thats true.
Just discovered the git repository https://repo.or.cz/tinycc.git which indicates an active maintenance from the community.

Just discovered the git repository https://repo.or.cz/tinycc.git which indicates an active maintenance from the community.

Yeah I didn't knew about it either since he didn't mentioned the repository on his website.
Seems like only "the owner bellard" has stopped maintaining, Community commits are still coming everyday.

Any progress ?

So I don't have time for that right now. Either someone else will do it, or you try to create the port yourself.
Some good starting points:
https://github.com/microsoft/vcpkg/blob/master/docs/examples/packaging-github-repos.md
Since tinycc's git repo isn't on github, you need this one instead of vcpkg_from_github vcpkg_from_git with the tag (https://repo.or.cz/tinycc.git/refs  => tags/release_0_9_27)

If you wish to add this port in the future, you can reopen this issue, but we're closing it for now, consider making a pull request instead.


Fix sentry logging

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.0 out of 3 committers have signed the CLA.:x: Rogier Goede:x: Dragnipur:x: boldpawnRogier Goede seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


[opentitantool] Add Skeleton of Ti50 host emulation driver

The Ti50 team will compile our code to either OpenTitan, to past
Google Titan chips, or to a third target which is x86 code running
inside a number of Linux processes.
The team hopes to be able to run some of the same tests across all
three targets, ideally using the same sequence of OpenTitanTool
invocations.
This patch creates an empty Transport implementation, for the Ti50
team to later populate.  I am posting this pull request in part to
rubber stamp the approach of putting some Ti50-specific code of little
use to other teams into the OpenTitan repository.
Signed-off-by: Jes B. Klinke<EMAIL_ADDRESS>

I will put this on hold, until we have a sketch of a design for the socket protocol.

This has been superseded by the proxy protocol outlined in #10889, which in addition to Ti50 host emulation, will also be used for Verilator simulation, and allowing persistent "sessions" with USB debuggers that do not take nicely to the many disconnections and reconnections from individual invocations of OpenTitan tool.


Remove specific version from NuGet package link

@L-E-son : Thanks for your contribution! The author(s) have been notified to review your proposed change.

#label:"aq-pr-triaged"

#sign-off

@SnehaGunda: I'm sorry - only the author of this article, @timsander1, can sign off on your changes. But we do have an exception process - if you are on the Microsoft content or product team for this product area, you can ask the PR review team to review and merge it by sending mail to the techdocprs alias.

#sign-off

@cosmosdbnotebooks: I'm sorry - only the author of this article, @timsander1, can sign off on your changes. But we do have an exception process - if you are on the Microsoft content or product team for this product area, you can ask the PR review team to review and merge it by sending mail to the techdocprs alias.


Mutual TLS - Skip client certificate validation

Description:
Is it possible to request a client certificate during the (mutual) TLS handshake that:

If no certificate is presented, the requests is forwarded to the upstream services
If given by the client, the certificate is not validated at all by the Gateway. Simply the certificate information is forwarded to the upstream services using XFCC headers.

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: ClientTrafficPolicy
metadata:
  name: enable-mtls
spec:
  targetRefs:
  - group: gateway.networking.k8s.io
    kind: Gateway
    name: eg
  tls:
    maxVersion: "1.2"
    clientValidation:
      optional: true
      caCertificateRefs:
      - kind: "Secret"
        group: ""
        name: "my-ca-secret"
  headers:
    xForwardedClientCert:
      mode: AppendForward
      certDetailsToAdd:
      - Chain

If I've understood the documentation correctly, it's only possible configuring optional mutual TLS, that is:

If no certificate is presented, the request is forwarded to the upstream services
If given by the client, the certificate is ALWAYS validated at all by the Gateway using the given caCertificateRefs .

In fact, I've seen that if caCertificateRefs is not correcly specified, theCertificate Request is not present in the TLS handshake. Would it be possible perphaps if  caCertificateRefs is null / not set, that simply no client certificate validation is performed?

caCertificateRefs not being set can be perceived as a misconfiguration, and not intent, in the future we plan on return 500 for these cases https://github.com/envoyproxy/gateway/issues/4153

For sure I need to set optional to true, but I also need to define caCertificateRefs with a valid secret so that the server will send the client certificate request in the handshake, but it will then add the Distinguished Name of the CA certificate in the certificate request.
The TLS specification states the following (https://datatracker.ietf.org/doc/html/rfc5246#section-7.4.4):

certificate_authorities

    A list of the distinguished names [[X501](https://datatracker.ietf.org/doc/html/rfc5246#ref-X501)] of acceptable
    certificate_authorities, represented in DER-encoded format.  These
    distinguished names may specify a desired distinguished name for a
    root CA or for a subordinate CA; thus, this message can be used to
    describe known roots as well as a desired authorization space.  If
    the certificate_authorities list is empty, then the client MAY
    send any certificate of the appropriate ClientCertificateType,
    unless there is some external arrangement to the contrary.

I'm interested in this part of the specification because some HTTP clients (or other protocols that support TLS) refuse sending a client certificate because they check against the Distinguished Names provided by the handshake.

If  the certificate_authorities list is empty, then the client MAYsend any certificate of the appropriate ClientCertificateType,        unless there is some external arrangement to the contrary.

How can achieve the server sending the Certificate Request in the TLS handsahak with an empty list of Distinguished Names?
I was able to do this with Envoy right away, so I think it should be possible with Envoy Gateway.

@haritzsaiz which field in envoy are you referring to ?

reopening this issue, to make a decision on the definition of optional, and should it also accommodate untrusted certs / certs thats fail validation

Some users may want to have the following setup: Verify a cert if presented and require a different auth option otherwise. The other auth method may be implemented by the backend server, an ext-auth server, patching EG config to apply validation matching to certain paths or a lua script inspecting the validation status etc.
I think (not sure) that envoy doesn't distinguish between successful validation and acceptance of an untrusted cert:
https://github.com/envoyproxy/envoy/blob/1b1bb4716ced9bd9dc84f87ea46bc5cb9c469da9/source/common/tls/cert_validator/default_validator.cc#L237
So, if optional would now also mean that non-verified and verified certificates are treated the same, the above setup would not be possible.
I'm +1 for this feature in general, to allow users to onboard MTLS gradually, but maybe it requires a dedicated knob.

I think it's reasonable to support this as Envoy already supports it, and EG should distinguish between these two options because they have different semantics:

optional: EG allows requests without client cert, or requests with a client cert and it should be validated by EG
passthrough-client-cert: client certs are required, and they'll not be validated by EG

prefer waiting a little longer to hear more similar use cases for this features from other users


Use Noto font for screen titles

This PR replaces the obsolete title font with the (larger) Noto font. The difference in rendered size made a few adjustments necessary.
All subtitles should be at the same vertical position.
I guess we could remove the two obsolete fonts default and title now.
Fixes #2029

Refer to this link for build results (access rights to CI server needed):
http://jenkins.terasology.org/job/TerasologyPRs/325/
Hooray Jenkins reported success with all tests good!

What about this block?
https://github.com/MovingBlocks/Terasology/blob/develop/engine/src/main/resources/assets/skins/mainmenu.skin#L44-L47
It is the one other spot I see "font": "title" although it looks like a plain "title" finds a lot of other hits too. But I'm not sure if those mean something else?
I'd be happy to delete the old fonts as well, am just not sure how to verify them not being in use in modules (possible candidates: InGameHelp, JoshariasSurvival, ManualLabor - pinging @Josharias) - I guess I might notice either logged errors failing to find assets or the text looking weird if even present?

Good catch! I updated the font as well. It is fairly large for those headings though:

Maybe we should add functionality to render it smaller than it actually is (or create a medium-size font for those subcaptions).

Refer to this link for build results (access rights to CI server needed):
http://jenkins.terasology.org/job/TerasologyPRs/326/
Hooray Jenkins reported success with all tests good!

Thanks, submitted #2042 as a follow-up so we don't forget :-)


Add example config and spark import.

I'd rather we focus on the spark-packages method for installation

OK, updated.

Looks good! Merging


Browser crashes when all values of Bar graph are equal to 1

Code

Javascript
var data9 = {
    labels: jQuery("#MostLovedEdible").data('labels'),
    series: [jQuery("#MostLovedEdible").data('values')]
};
new Chartist.Bar('#MostLovedEdible', data9,options);

It may sound stupid , but in some cases when you generate the data dynamically this may happen.

I am having the same problem, but I've noticed 2 things.
1 - It crashes only when the onlyInteger option has been set to true
2 - It crashes only when the series is filled with 0 or 1. If any element in the series is bigger than 1, everything is fine.
This example crashes unless you change one of the elements to be > 1, or set to false the onlyInteger
new Chartist.Bar('.ct-chart', {
labels: [1, 2, 3, 4, 5, 6, 7, 8],
series: [
[1, 1, 1, 1, 1, 1, 1, 1]
]
}, {
axisY: {
onlyInteger: true
}
});

Reason: Chartist.rho jumps into infinite loop when 'num' = 1

Workaround: set low/high bounds.

This is a duplicate of #318 and was fixed with 0d66139b6e7b334f120c2e0590c3cf5b1a8882ec


BAEL-2368 convert string array to string

Tests that show converting with different methods.

Can you put this in its own package within  the com.baeldung folder? Typically, we are trying to follow that folder structure


Support grayscale feature for non-properties namespaces(#4316)

What's the purpose of this PR
Support grayscale feature for non-properties namespaces
Which issue(s) this PR fixes:
Fixes #4316
Brief changelog

Non-properties namespace displays grayscale buttons.
Modify src/main/resources/static/views/component/namespace-panel-master-tab.html
Non-properties namespace grayscale version displays the configuration Tab and hides the new configuration button.
Modify src/main/resources/static/views/component/namespace-panel-branch-tab.html
When the non-properties namespace is modified, the key and type cannot be modified and the comment is hidden.
Modify src/main/resources/static/views/component/item-modal.html
Modify the grayscale version of the Modify configuration dialog, showing modified configuration instead of new configuration.
Modify src/main/resources/static/scripts/controller/config/ConfigNamespaceController.js

Follow this checklist to help us incorporate your contribution quickly and easily:

[x] Read the Contributing Guide before making this pull request.
[x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
[ ] Write necessary unit tests to verify the code.
[ ] Run mvn clean test to make sure this pull request doesn't break anything.
[ ] Update the CHANGES log.

recheck

我已阅读CLA文件，并特此签署CLA

recheck


Re-enabling pausing on inactive window

What is your question?
I don't know how or why it happened, but I know it wasn't there before.
Each time I tab out of PE (any unmodified build), it just stays active for some reason.
The reason might be, that I somehow got a crash whilst playing the Mic Of Time Mod (which used PE), which also reset ALL of (regular) PE scores and achievements I had (again)
Anyways, is there any way, to enable it again? It like stays even persistent after a restart of the computer

unless i'm interpreting this wrong probably just change FlxG's autoPause variable I think that's what controls it


Merge create link menu options

Merges the currently separated menu options "Create link to coordinates" and "Create link to URL" into one option with 2 tabs.
Note: The name "canvasLink" is just temporary, you might wanna change it if you feel like doing so. I also haven't done much testing, so maybe I've missed out on something, let me know if I did!

oops kkosty got there way first
well i randomly got bored and remembered this but didn't remember kkosty already did it :p


Drill down is not working for business glossary

Describe the bug
ldh v0.8.x service was deployed and data was ingested. The entries can be seen on mysql and elastic search but on UI business glossary is showing empty
To Reproduce
Steps to reproduce the behavior:

Go to datahub home page
click on business glossary
you will see no data message

Expected behavior
On click on business glossary tile populated data should be visible
Screenshots

Closed via #2788


Need to be able to specify which .nycrc file to use

Expected Behavior
In Istanbul I could use this CLI parameter:
--config=test/.istanbul-client.yml
to specify the exact version of the config file to use for this run.
Observed Behavior
I can not seem to do the same for NYC.
We have different runs of NYC for our Node-side code and for our browser-side code and want to keep they separated. But to do that we would need to have two different .nycrc files and the ability to specify which one to use.
It seems that we only have the ability to have a single .nycrc file and it must be called .nycrc
If there a way to add the --config option into NYC similar to what was available in Istanbul?
nyc --config=test/.nycrc-client.yml

Is this done through --nycrc-path?

Yes, that is the answer.

@intervalia awesome sounds like this did the trick for you? sorry for the discrepancy.


Usage: usage generated for destroyed VMs with no backups

Description
Fixes: #4990
When a VM associated with a backup offering is destroyed/expunged, the backup offering isn't unassigned, and despite the VM having no backups present, backup usage is generated. This PR prevent usage record generation when there are no backups present for a VM with a backup offering associated to it. This is done by ensuring that usage event for backups is generated only when a  the backup size > 0

Types of changes

[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] New feature (non-breaking change which adds functionality)
[X] Bug fix (non-breaking change which fixes an issue)
[ ] Enhancement (improves an existing feature and functionality)
[ ] Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity
Bug Severity

[ ] BLOCKER
[ ] Critical
[X] Major
[ ] Minor
[ ] Trivial

Screenshots (if appropriate):
How Has This Been Tested:

@blueorangutan package

@Pearl1594 a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: :heavy_check_mark: centos7 :heavy_check_mark: centos8 :heavy_check_mark: debian.  I'll keep you posted as I make progress.

Packaging result: :heavy_check_mark: centos7 :heavy_check_mark: centos8 :heavy_check_mark: debian. SL-JID 33

@blueorangutan test

@Pearl1594 a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

Trillian test result (tid-697)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 33549 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5017-t697-kvm-centos7.zip
Smoke tests completed. 87 look OK, 0 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

@olivierlemasle can you review/test this PR? Thanks

Thanks @GutoVeronezi for the review, I've addressed your comments.

@blueorangutan package

@Pearl1594 a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: :heavy_check_mark: centos7 :heavy_check_mark: centos8 :heavy_check_mark: debian. SL-JID 47

@olivierlemasle Thanks for the review.
With respect to your concern regarding usage_backup records not being marked as removed for destroyed VMs (with no backups) - in commit d3de10e47c8cfd5dbe155c42b7b715a1e6ed5ec1, usage records for VM's that are destroyed with no backups are prevented by disassociating the backup offering from the VM - while this seemed appropriate while testing against Dummy backup Provider, on discussion with @rhtyd it was understood that since veeam, an external entity, handles the backup job, though we may unassign  the  offering from the VM from ACS end, thus preventing further generation of usage records, it may still be active on the Provider (Veeam) end - in case a recurring schedule has been set up, which MAY have an adverse effect. Hence, the current solution has been proposed which will prevent usage record generation if the backup size is 0.
I don't think we would lose usage records if usage.stats.job.aggregation.range < backup.framework.sync.interval, as I had tested (with the dummy backup provider) with usage.stats.job.aggregation.range = 2mins and backup.framework.sync.interval = 5mins.

Testing it today :eyes:

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: :heavy_check_mark: centos7 :heavy_check_mark: centos8 :heavy_check_mark: debian. SL-JID 69

@blueorangutan test

@Pearl1594 a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

Trillian test result (tid-714)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 47713 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5017-t714-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_routers_network_ops.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Smoke tests completed. 84 look OK, 3 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

test_08_deploy_and_upgrade_kubernetes_ha_cluster
Failure
145.55
test_kubernetes_clusters.py

test_01_RVR_Network_FW_PF_SSH_default_routes_egress_true
Failure
304.22
test_routers_network_ops.py

test_02_RVR_Network_FW_PF_SSH_default_routes_egress_false
Failure
305.06
test_routers_network_ops.py

test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL
Failure
468.82
test_vpc_redundant.py

test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers
Failure
371.52
test_vpc_redundant.py

test_05_rvpc_multi_tiers
Failure
433.58
test_vpc_redundant.py

test_05_rvpc_multi_tiers
Error
458.46
test_vpc_redundant.py

As I expected, this causes a regression when the usage aggregation range is greater than the backup sync interval.
@Pearl1594, you wrote:

I don't think we would lose usage records if usage.stats.job.aggregation.range < backup.framework.sync.interval, as I had tested (with the dummy backup provider) with usage.stats.job.aggregation.range = 2mins and backup.framework.sync.interval = 5mins.

Perhaps you did missed converting the backup sync interval to seconds?
On my side, I did a test on this PR with:

usage.stats.job.aggregation.range = 5 (5 minutes)
backup.framework.sync.interval = 600 (10 minutes)

I have a (non-deleted, running) VM, with a backup (using the dummy B&R provider).
Here is the usage records for type=28:

Backup usages (click to open)
{
  "listusagerecordsresponse": {
    "count": 6,
    "usagerecord": [
      {
        "account": "admin",
        "accountid": "b4d9535b-bc01-11eb-a756-02000b4c0001",
        "domainid": "9654cd27-bc01-11eb-a756-02000b4c0001",
        "domain": "ROOT",
        "zoneid": "06fb3b55-38c6-4f44-a5a2-ec1a167413a6",
        "description": "Backup usage of size 0.0000 GiB for VM test-ole (195bb990-63a1-4805-87d5-f9d6a2f4122d) and backup offering Silver (0be952ea-b9ab-42b0-ab40-99cece5acc25, user ad-hoc/scheduled backup allowed: true)",
        "usage": "0.0000 GiB",
        "usagetype": 28,
        "rawusage": "1000",
        "virtualmachineid": "195bb990-63a1-4805-87d5-f9d6a2f4122d",
        "name": "i-2-3-VM",
        "usageid": "195bb990-63a1-4805-87d5-f9d6a2f4122d",
        "startdate": "2021-05-23'T'20:30:00+00:00",
        "enddate": "2021-05-23'T'20:35:00+00:00",
        "tags": []
      },
      {
        "account": "admin",
        "accountid": "b4d9535b-bc01-11eb-a756-02000b4c0001",
        "domainid": "9654cd27-bc01-11eb-a756-02000b4c0001",
        "domain": "ROOT",
        "zoneid": "06fb3b55-38c6-4f44-a5a2-ec1a167413a6",
        "description": "Backup usage of size 0.0000 GiB for VM test-ole (195bb990-63a1-4805-87d5-f9d6a2f4122d) and backup offering Silver (0be952ea-b9ab-42b0-ab40-99cece5acc25, user ad-hoc/scheduled backup allowed: true)",
        "usage": "0.0000 GiB",
        "usagetype": 28,
        "rawusage": "1000",
        "virtualmachineid": "195bb990-63a1-4805-87d5-f9d6a2f4122d",
        "name": "i-2-3-VM",
        "usageid": "195bb990-63a1-4805-87d5-f9d6a2f4122d",
        "startdate": "2021-05-23'T'20:40:00+00:00",
        "enddate": "2021-05-23'T'20:45:00+00:00",
        "tags": []
      },
      {
        "account": "admin",
        "accountid": "b4d9535b-bc01-11eb-a756-02000b4c0001",
        "domainid": "9654cd27-bc01-11eb-a756-02000b4c0001",
        "domain": "ROOT",
        "zoneid": "06fb3b55-38c6-4f44-a5a2-ec1a167413a6",
        "description": "Backup usage of size 0.0000 GiB for VM test-ole (195bb990-63a1-4805-87d5-f9d6a2f4122d) and backup offering Silver (0be952ea-b9ab-42b0-ab40-99cece5acc25, user ad-hoc/scheduled backup allowed: true)",
        "usage": "0.0000 GiB",
        "usagetype": 28,
        "rawusage": "1000",
        "virtualmachineid": "195bb990-63a1-4805-87d5-f9d6a2f4122d",
        "name": "i-2-3-VM",
        "usageid": "195bb990-63a1-4805-87d5-f9d6a2f4122d",
        "startdate": "2021-05-23'T'20:50:00+00:00",
        "enddate": "2021-05-23'T'20:55:00+00:00",
        "tags": []
      },
      {
        "account": "admin",
        "accountid": "b4d9535b-bc01-11eb-a756-02000b4c0001",
        "domainid": "9654cd27-bc01-11eb-a756-02000b4c0001",
        "domain": "ROOT",
        "zoneid": "06fb3b55-38c6-4f44-a5a2-ec1a167413a6",
        "description": "Backup usage of size 0.0000 GiB for VM test-ole (195bb990-63a1-4805-87d5-f9d6a2f4122d) and backup offering Silver (0be952ea-b9ab-42b0-ab40-99cece5acc25, user ad-hoc/scheduled backup allowed: true)",
        "usage": "0.0000 GiB",
        "usagetype": 28,
        "rawusage": "1000",
        "virtualmachineid": "195bb990-63a1-4805-87d5-f9d6a2f4122d",
        "name": "i-2-3-VM",
        "usageid": "195bb990-63a1-4805-87d5-f9d6a2f4122d",
        "startdate": "2021-05-23'T'21:00:00+00:00",
        "enddate": "2021-05-23'T'21:05:00+00:00",
        "tags": []
      },
      {
        "account": "admin",
        "accountid": "b4d9535b-bc01-11eb-a756-02000b4c0001",
        "domainid": "9654cd27-bc01-11eb-a756-02000b4c0001",
        "domain": "ROOT",
        "zoneid": "06fb3b55-38c6-4f44-a5a2-ec1a167413a6",
        "description": "Backup usage of size 0.0000 GiB for VM test-ole (195bb990-63a1-4805-87d5-f9d6a2f4122d) and backup offering Silver (0be952ea-b9ab-42b0-ab40-99cece5acc25, user ad-hoc/scheduled backup allowed: true)",
        "usage": "0.0000 GiB",
        "usagetype": 28,
        "rawusage": "1000",
        "virtualmachineid": "195bb990-63a1-4805-87d5-f9d6a2f4122d",
        "name": "i-2-3-VM",
        "usageid": "195bb990-63a1-4805-87d5-f9d6a2f4122d",
        "startdate": "2021-05-23'T'21:10:00+00:00",
        "enddate": "2021-05-23'T'21:15:00+00:00",
        "tags": []
      },
      {
        "account": "admin",
        "accountid": "b4d9535b-bc01-11eb-a756-02000b4c0001",
        "domainid": "9654cd27-bc01-11eb-a756-02000b4c0001",
        "domain": "ROOT",
        "zoneid": "06fb3b55-38c6-4f44-a5a2-ec1a167413a6",
        "description": "Backup usage of size 0.0000 GiB for VM test-ole (195bb990-63a1-4805-87d5-f9d6a2f4122d) and backup offering Silver (0be952ea-b9ab-42b0-ab40-99cece5acc25, user ad-hoc/scheduled backup allowed: true)",
        "usage": "0.0000 GiB",
        "usagetype": 28,
        "rawusage": "1000",
        "virtualmachineid": "195bb990-63a1-4805-87d5-f9d6a2f4122d",
        "name": "i-2-3-VM",
        "usageid": "195bb990-63a1-4805-87d5-f9d6a2f4122d",
        "startdate": "2021-05-23'T'21:20:00+00:00",
        "enddate": "2021-05-23'T'21:25:00+00:00",
        "tags": []
      }
    ]
  }
}

As you can see, we have usage for 20:30 to 20:35, then 20:40 to 20:45, etc., but one usage record over 2 is missing.
The cause is that the usage server generates an usage record for a 5-minutes period only if there is at least one event on that period, and the events are generated every 10 minutes:
+---------------------+---------------------+------------------------------------------------------+------+
| created             | type                | resource_name                                        | size |
+---------------------+---------------------+------------------------------------------------------+------+
| 2021-05-23 20:34:56 | BACKUP.USAGE.METRIC | Backup-test-ole-195bb990-63a1-4805-87d5-f9d6a2f4122d | 1000 |
| 2021-05-23 20:44:56 | BACKUP.USAGE.METRIC | Backup-test-ole-195bb990-63a1-4805-87d5-f9d6a2f4122d | 1000 |
| 2021-05-23 20:54:56 | BACKUP.USAGE.METRIC | Backup-test-ole-195bb990-63a1-4805-87d5-f9d6a2f4122d | 1000 |
| 2021-05-23 21:04:56 | BACKUP.USAGE.METRIC | Backup-test-ole-195bb990-63a1-4805-87d5-f9d6a2f4122d | 1000 |
| 2021-05-23 21:14:56 | BACKUP.USAGE.METRIC | Backup-test-ole-195bb990-63a1-4805-87d5-f9d6a2f4122d | 1000 |
| 2021-05-23 21:24:56 | BACKUP.USAGE.METRIC | Backup-test-ole-195bb990-63a1-4805-87d5-f9d6a2f4122d | 1000 |
+---------------------+---------------------+------------------------------------------------------+------+

So, when the usage aggregation range is greater than the backup sync interval, we're missing some usage records when the VM is still existing. However, when the VM destroyed, the usages stop as expected.

@olivierlemasle can the acceptable solution be as simple as leave it as is because backups are first class citizen? As per your bug report, at least 1 backup is created/exists for the VM so ideally the usage records should continue to be created until that VM's backup is removed by the user explicitly. However, if even after removing the backups if it does create usage records then it's a real bug. (didn't check just speculating here)
The only edge case is when VM is expunged with no backups but possible a backup external job such as for Veeam, in which case the solution can be simply to check and unassign the backup offering for the VM which has no backups (i.e. delete the Veeam job external which won't actually work anyway as the VM which was assigned to it does not exist anymore in Vmware). What do you think @Pearl1594 @olivierlemasle ?

However, if even after removing the backups if it does create usage records then it's a real bug. (didn't check just speculating here)

Yes @rhtyd That's the issue this PR tries to fix. Currently, when you delete a VM which has backups, THEN you delete the backups, usages for the backups keep being generated.

@blueorangutan package

@Pearl1594 a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@olivierlemasle @rhtyd I've changed the approach to addressing the issue; the fix now takes care of the following:

When a VM is expunged and has no backups but is assigned  to a backup offering, we remove the VM from the backup offering
A cleanup process has been added to remove the backup offering from VMs that are expunged and have no backups to prevent further generation of usage records
When the last backup of an expunged VM is deleted (this is for dummy provider), we remove the VM from the backup offering - preventing further generation of usage records.

Packaging result: :heavy_check_mark: centos7 :heavy_check_mark: centos8 :heavy_check_mark: debian. SL-JID 109

@blueorangutan test

@Pearl1594 a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@olivierlemasle can you test the changes now? We're trying to keep the changes simple and cover all cases as Pearl has described.

@blueorangutan package

@Pearl1594 a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: :heavy_check_mark: centos7 :heavy_check_mark: centos8 :heavy_check_mark: debian.  I'll keep you posted as I make progress.

Packaging result: :heavy_check_mark: centos7 :heavy_check_mark: centos8 :heavy_check_mark: debian. SL-JID 124

@blueorangutan test

@Pearl1594 a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

LGTM thanks for replying @Pearl1594
@shwstppr can you review as well

Test was not necessary changes between last two rounds won't make difference. I'll merge this and kick smoketests on health check PR.

Trillian test result (tid-794)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 32555 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5017-t794-kvm-centos7.zip
Smoke tests completed. 87 look OK, 0 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File


[Feature,Doc] QValue refactoring and QNet + RNN tuto

@matteobettini @smorad
I have drafted a tutorial. It's very rough but it trains!
I have refactored the way we work with LSTMs.
The idea is to have module that has 3 in keys (an input and the hidden keys) and 3 output (one value and 2 hidden keys).
Questions for you:

We could by default use the same key but the problem here is that we want to write the "next" hidden state in tensordict["next", "hidden"] or smth like that.
Is it too complex? I could make it work with just ["hidden0", "hidden1"] though, but on the torchrl side it will be less clean. If you look at the example, the in_keys are  ["hidden0", "hidden1"] (current) and the out are  [("next", "hidden0"), ("next", "hidden1")].
Any comment on the use of "is_init"?
The current implementation allows you to use the same lstm for executing the policy and for batched exection of the lstm. The default is step-by-step. I think the docstring is informative regarding this feature. Does this make sense? The switch from one behaviour to the other is a bit clunky but specifying this in the constructor doen't really solve our problem as you'll need to call the constructor twice (which I think is worse than calling a method to convert the module).
One thing it cannot do is this: if your loop is a bit more complicated (e.g. it takes the previous action + current observation as input) and if you want to backprop through the chain of events it will not work as intended, but for these kind of things you'll need a custom loop anyway I guess

Note: there is still some magic that needs to be done when we pass 2 consecutive trajectories to the RNN, but it can be handled without too much trouble (I think)
@matteobettini I will do that with padding. Don't tell me, I know!

EDIT: Problem solved, feature implemented!!

We can't escape from padding in rnns right?

We can't escape from padding in rnns right?

There is the packed_sequence artifact but i'm not entirely sure that it solves the problem.
My implementation of padding / unpadding is 2x faster than other torch tools btw

Awesome, really looking forward to this!
I think it makes sense to write the recurrent state to next. It just might make things tricky with when/who is calling step_mdp. Are there any footguns here, e.g. collector calling step_mdp and reversing state, (next, state) before placing them into the replay buffer?
Would it make sense to use td[(prev, done)] instead of is_init? Perhaps there are some edge cases I haven't considered, or maybe this is tricky to implement.
Opinion: I think set_temporal_mode could be a little confusing. Is this different than inference (collector) mode, or just a synonym for inference? If they are indeed the same, maybe it makes sense to have if self.eval: ... in the module forward pass instead of an explicit time-batch/single-step mode.
I would really prefer that packed_sequence is not used here. It's brought me nothing but pain.

Awesome, really looking forward to this!
I think it makes sense to write the recurrent state to next. It just might make things tricky with when/who is calling step_mdp. Are there any footguns here, e.g. collector calling step_mdp and reversing state, (next, state) before placing them into the replay buffer?

If you want to store every h_{t-1}, s_t, a_t, h_t, s_{t+1} following the nomenclature above, this will correspond to
TensorDict(
    fields={
        hidden: Tensor(shape=torch.Size([4, 16]), device=cpu, dtype=torch.int64, is_shared=False),
        action: Tensor(shape=torch.Size([4, 1]), device=cpu, dtype=torch.int64, is_shared=False),
        observation: Tensor(shape=torch.Size([4, 2]), device=cpu, dtype=torch.bool, is_shared=False),
        next: TensorDict(
            fields={
                hidden: Tensor(shape=torch.Size([4, 16]), device=cpu, dtype=torch.int64, is_shared=False),
                observation: Tensor(shape=torch.Size([4, 2]), device=cpu, dtype=torch.bool, is_shared=False),
            batch_size=torch.Size([4]),
            device=cpu,
            is_shared=False),
    batch_size=torch.Size([4]),
    device=cpu,
    is_shared=False)

Opinion: I think set_temporal_mode could be a little confusing. Is this different than inference (collector) mode, or just a synonym for inference? If they are indeed the same, maybe it makes sense to have if self.training: ... in the module forward pass instead of an explicit time-batch/single-step mode.

I agree it's not ideal.
First, we had a similar discussion here if your want to have a look.
First, let's put the cards on the table. There are various things that are related but not exactly identical:

Train / eval mode
Explorative / exploitative mode
temporal / static mode (eg example here)
grad on / off mode (eg example here)
When doing inference and collecting data for training you will be in
eval + explorative + static + grad off
When testing the policy in "real" setting you will be
eval + exploitative + static + grad off
When training the policy you will be in
train + exploitative* + temporal** + grad on***

not always true, sometimes you want to sample the action, sometimes not
** not always true: in dreamer you simulate trajectories during training that are fully reparametrized, and you're in static mode bc the RNN feeds itself with the action that results from the previous call.
*** depending on the loss (policy / critic / model in MBRL) you may want to keep some ops out of the graph or just detach some variables (not talking about meta-RL!)

Gradient and exploration are currently handled via "exploration_type" and the torch "grad_mode" environment variables.
Train and eval are handled via the module attribute. I hesitate for temporal, but I can see both cases where you may want to use static / temporal at both training and inference time. I thought that giving the user full power over what happens internally was the easiest option to avoid any kind of confusion.
If it's about naming i'm happy to reconsider the name oc. I'm actually looking forward to have more input on this from the community: what's the best way of approaching all of these "modes" that -- to me -- seem very algorithm-dependant.
Just think about SARSA vs QMax, where the action selection is completely different. We can easily do that via "exploration_type", but it's neither related to eval/train, gradient off/on or else.

I would really prefer that packed_sequence is not used here. It's brought me nothing but pain.

Got it, it's a headache on our side too to let's just not use it.

That tensordict format looks good to me.
Yeah that's a good point with the modes. In that case it's better to err on the side of flexibility than simplicity. I think perhaps a small table in the docs mapping each of these modes to a 1-2 sentence description would help clear up any ambiguities. I do like the way exploration etc. is currently handled. Is set_recurrent_mode more specific than set_temporal_mode? Could either be batch_time or sequential_time.
nit: hidden/state are super overloaded terms. I think we should be explicit with recurrent_state_h, recurrent_state_c or rstate_h, rstate_c, which will help new users when they are grepping through the codebase.


Need to add "cam_ts_done" variable to example YAML File

Currently if one want to skip time series generation, but still compute the climo files, then one needs to add the variable:
cam_ts_done: True
To their config (YAML) file.  However, the example baseline config file does not contain this option, so for the new user its existence may not be known, and they may end up having a failed run.  Thus this option should be added to the example baseline config file, with enough written in the comments to hopefully make its use self-explanatory.

Thanks @nusbaume , that variable was just what I needed.
I'm trying to do the same for the baseline case too, although it is gate-kept by some different logic. I will keep working on it tomorrow unless you have a quick fix... If I do get it to work, can I try a pull request?

Hi @bitterbark
Glad to hear that worked!  I think something may be wrong with your baseline setup, given the error you sent earlier via email.  There might be a quick fix, but I would need to see the YAML file itself.  If you can email it to me, or post it in a reply here, then I can quickly look it over to see if anything is wrong (beyond what the validator can do, which is basically just check for syntax errors).
Otherwise, if it turns out to be a real bug and you find a fix for it, then definitely feel free to create a PR, and we'll make sure to get the fix merged.  Thanks for offering!

Sure, it is on the cgd machines:
/home/bundy/diagnostics/CAM_diagnostics/CAM_diagnostics/config_dani_test.yaml
I was aiming for the lightest possible test--running it on existing time series files. (Which is what I did months ago, using the same files from your previous example config file). Glad to have help but I don't want to burden you with it!
BTW if you have example model data you could point me to, I could try to do it the standard way. I think we should plan to distribute something like that for users to test the system is working before they introduce the other potential source of error that will be using their own data.

Thanks for pointing me to the YAML file!  I think I found the error (if correct it is a bug on our end, whoops).
In the file lib/CamDiag.py, there should be a block of code starting on line 242 that looks like this:
         if not self.__basic_info['compare_obs']:
            try:
                if config['diag_cam_baseline_climo']['calc_cam_climo']:

                    #If so, then add CAM baseline climatology info to object:
                    self.__cam_bl_climo_info = read_config_obj(config, 'diag_cam_baseline_climo')

                    #Expand CAM baseline climo info variable strings:
                    self.__expand_references(self.__cam_climo_info, config_search_dict)
            except:
                raise KeyError("'calc_bl_cam_climo' in 'diag_cam_baseline_climo' not found in config file.  Please see 'config_example.yaml'.")

Can you replace that code block with just:
        if not self.__basic_info['compare_obs']:
            #If so, then add CAM baseline climatology info to object:
            self.__cam_bl_climo_info = read_config_obj(config, 'diag_cam_baseline_climo')

            #Expand CAM baseline climo info variable strings:
            self.__expand_references(self.__cam_climo_info, config_search_dict)

and let me know if it works?  Thanks again!

Thanks, it works. I'd like to try a pull request if that's okay. No worries if you'd rather just do it yourself.


Raspberry Pi images

Is your feature request related to a problem? Please describe.
I really wish to have the .img files for Raspberry Pi's, like RancherOS have, or some kind of installer for SDCards.
K3S declares:
5. Minimal to no OS dependencies (just a sane kernel and cgroup mounts needed)...

would be much much lighter than installing on top of Debian or Ubuntu, also not bloated with systemd and other stuff which are not used by k3s.
Describe the solution you'd like
I think, some stuff can be taken from RancherOS repository to achieve that feature.

I second the request.
I have three Raspberry PI 4 reproachfully staring at me from my desk for not having installed K3os

@TheAifam5 @rharmonson

I think, some stuff can be taken from RancherOS repository to achieve that feature.

I thought this also. The discussion in how do rancheros and k3os differ? https://github.com/rancher/k3os/issues/201 helped me see how the two OS are different.
Please be sure to :+1: your preferred Raspberry Pi in What ARM boards would the community like supported? https://github.com/rancher/k3os/issues/65.

I have three Raspberry PI 4 reproachfully staring at me from my desk for not having installed K3os

You may find the discussion in Arm ISO is not bootable for Raspberry Pi 3 https://github.com/rancher/k3os/issues/111 helpful, in particular https://github.com/rancher/k3os/issues/111#issuecomment-548001732.

I spent a bit of time looking into if there was a way to create an image that had u-boot which could boot k3os via grub2 via uefi from the arm64 iso (u-boot supports uefi).
I kinda got it booting, however I do believe there was something weird with the kernel or something as things didn't work out.

For this to work though, I guess the kernel used in k3os would need to be ubuntu's HWE kernel, which I know there's a PR for, or there would "need" to be a way to bring your own kernel for cases where you need a custom kernel to support your board.

HWE?
New term for me, so here is a link for others that may not be familiar with Ubuntu's "HardWare Enabled" kernel.
https://askubuntu.com/questions/248914/what-is-hardware-enablement-hwe

Stumbled onto this but not tested, yet. It may be something warranting exploration. Thoughts?
https://github.com/pftf/RPi4

Stumbled onto this but not tested, yet. It may be something warranting exploration. Thoughts?
https://github.com/pftf/RPi4

I've seen that too, I'd suggest we start with looking into u-boot as that also has an uefi interface but also works on multiple boards and not just the rpi's.
If there's a problem getting u-boot working the way we need it on rpis that project might be a good alternative.

I agree with your suggestion.
I found a u-boot and raspberry specific article which I will spend time
reviewing this weekend. This is not my area of my expertise, but will take
a stab at it in hopes I can contribute.
On Sat, Dec 14, 2019 at 10:02 AM Simon Fridlund<EMAIL_ADDRESS>wrote:

Stumbled onto this but not tested, yet. It may be something warranting
exploration. Thoughts?
https://github.com/pftf/RPi4
I've seen that too, I'd suggest we start with looking into u-boot as that
also has an uefi interface but also works on multiple boards and not just
the rpi's.
If there's a problem getting u-boot working the way we need it on rpis
that project might be a good alternative.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/rancher/k3os/issues/309?email_source=notifications&email_token=AB4Y3LSXUNINX2FVUYIEWIDQYUNS5A5CNFSM4JTIQBQKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEG4H2NI#issuecomment-565738805,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AB4Y3LWWD2GLAHOBSVUDBGDQYUNS5ANCNFSM4JTIQBQA
.

So far I've gotten u-boot up and running on some of my rpis, I'm able to load k3os iso via usb stick. I'm now looking into trying to get the same iso to load from a second partition on the sdcard.

So far I've gotten u-boot up and running on some of my rpis, I'm able to load k3os iso via usb stick. I'm now looking into trying to get the same iso to load from a second partition on the sdcard.

Any luck with this?

I created this repository to generate k3os images for the Raspberry Pi and another arm64 device: https://github.com/sgielen/picl-k3os-image-generator

@sgielen, thank you!
I reviewed your repository and it looks very promising. I do have a question regarding build-image.sh. Reviewing your code, I assume I would just update the line 78 with the URL from https://github.com/raspberrypi/firmware/releases when future releases occur?
78: if [ "$IMAGE_TYPE" = "raspberrypi" ]; then dl_dep raspberrypi-firmware.tar.gz https://github.com/raspberrypi/firmware/archive/1.20190925.tar.gz
As soon as I complete a proof of concept with RPi4s and Ceph, I will explore your work more completely.
Again, thank you!

@rharmonson Yes, although currently you'd also need to remove the existing file to get it to download the new one. I'm considering fixing that by letting it check against the expected SHA sum or something like that.
Feel free to try it and submit a pull request if you find it works for you :-)

Thank you and I may very well give it a go.

@sgielen Hi there, I just bought 4 RPi 4 to make my own cluster as well and I'd like to install k3os on them. The objective is to run a web application developed using the MEAN stack. Do you think that would work or should I use a light raspian as anyone does?

I'm just afraid to waste many hours to eventually realize that what I'm trying is way to "hacky" and then go back to a more standard approach

@vnelis I feel ya. My Docker needs are simple, and I am loving swarm on RancherOS.

@sgielen Hi there, I just bought 4 RPi 4 to make my own cluster as well and I'd like to install k3os on them. The objective is to run a web application developed using the MEAN stack. Do you think that would work or should I use a light raspian as anyone does?

Hi @vnelis! I would think that if the application isn't too heavy, and you are willing to invest some time in getting it running on Kubernetes, it will probably work well. Keep in mind that it's possible the latency is higher than with modern x86 systems.
In order to get the most out of a cluster setup like this (if only in terms of fun), I'd suggest looking into highly available storage like OpenEBS (which, with some work, seems to work well on arm64) and use a system like MetalLB to give your Kubernetes Service objects IPs in your network that don't correspond to a single node. This way, you eliminate SPOFs within your cluster, allowing you to perform maintenance on single nodes at a time without bringing down your application.

@sgielen do you have any instructions on setting up OpenEBS on Raspberry Pi? I've tried settings up cStor backend, but every time i try to use PVs i get defunct zrepl processes in cstor pools pods requiring reboot -fn on the affected nodes.

generally, arm32 build of all components is desirable (kernel, initrd, etc)

I still can't believe that 2yrs later that there are still no official images for RPI.

The rancher-os has not been updated for ages and looks like they remake everything from scratch. Maybe afterwards would be much easier to setup k3s - will see :)

They deprecated RancherOS a while ago. (probably in favor of k3os)

They deprecated RancherOS a while ago. (probably in favor of k3os)

oh but what about the v2-test branch: https://github.com/rancher/os/tree/v2-test

spicy


Public DNS-SD API for browsing, resolving, and publishing services

Currently the various bits of CUPS, PAPPL, etc. have a conditional mess of code that use the Avahi or mDNSResponder APIs to do DNS-SD discovery and sharing. This continues to be a major pain, and CUPS should have a common API that developers can use to access whatever the local API/service is for this. Goals:

Support the Avahi, mDNSResponder, and Win32 APIs, with potential for others (systemd/D-Bus) in the future
Support asynchronous browsing, resolution, and service publication with support for sub-types, domains, and LOC+TXT records
Provide callbacks as needed
Provide helper functions for renaming, getting the mDNS hostname, and getting/setting a list of DNS-SD domains

[dnssd d921af68f] Initial skeleton of new DNS-SD API (Issue #19)

The proposed API manages all DNS-SD requests/registrations on a separate thread represented by the cups_dnssd_t context structure/pointer. Deleting the DNS-SD context also deletes all pending requests/registrations. The semantics of the service API reflect Avahi's peculiar handling of services as part of a named group, but we can simulate that with mDNSResponder and WinDNS.
Create/destroy DNS-SD context:
void  cupsDNSSDDelete(cups_dnssd_t *dns);
cups_dnssd_t *cupsDNSSDNew(cups_dnssd_error_cb_t error_cb, void *cb_data);

Create/destroy browsing request:
void  cupsDNSSDBrowseDelete(cups_dnssd_browse_t *browser);
cups_dnssd_browse_t *cupsDNSSDBrowseNew(cups_dnssd_t *dns, uint32_t if_index, const char *types, const char *domains, cups_dnssd_browse_cb_t browse_cb, void *cb_data);

Create/destroy query request:
void  cupsDNSSDQueryDelete(cups_dnssd_query_t *query);
cups_dnssd_query_t *cupsDNSSDQueryNew(cups_dnssd_t *dns, uint32_t if_index, const char *types, const char *domains, cups_dnssd_query_cb_t query_cb, void *cb_data);

Create/restore resolve request:
void  cupsDNSSDResolveDelete(cups_dnssd_resolve_t *res);
cups_dnssd_resolve_t *cupsDNSSDResolveNew(cups_dnssd_t *dns, uint32_t if_index, const char *types, const char *domains, cups_dnssd_resolve_cb_t resolve_cb, void *cb_data);

Create/destroy named services. Each service has a name and one or more instances:
bool  cupsDNSSDServiceAdd(cups_dnssd_service_t *service, uint32_t if_index, const char *types, const char *domains, const char *host, uint16_t port, size_t num_txt, cups_option_t *txt);
void  cupsDNSSDServiceDelete(cups_dnssd_service_t *service);
cups_dnssd_service_t *cupsDNSSDServiceNew(cups_dnssd_t *dns, const char *name, cups_dnssd_service_cb_t cb, void *cb_data);
bool  cupsDNSSDServicePublish(cups_dnssd_service_t *service);
bool  cupsDNSSDServiceSetLocation(cups_dnssd_service_t *service, const char *geo_uri);

Mostly done, just have the Windows DNS-SD APIs to support.

Created #29 to track usage of the Windows DNS-SD APIs. Closing this one.


Doesn't work with paths that have spaces or special characters

This one simply can't be navigated to with bcd

Still not working, cd errors with "too many arguments"


Make the health check timeout for balancer member configurable

Atfer httpd upgrade to version 55, the health check timeout has been set to be equal to the request timeout.
We have lost the capability to define a timeout for the health check that is different from that of the request.
Our worker request timeout is set to 60s (time needed to treat a request) while its health check should be quick (5s).
The need for a short health check is to allow httpd to quickly detect suspended workers (that do not receive hc) and set their health status to HcFl so that requests do not get routed to them.
The goal of this pull request is to make the timeout of the health check configurable by balancer member.

Hello @ylavic, I really appreciate any insights or feedback you might have about this PR.

Hello @icing, as this PR will be closed soon, I really appreciate any insights or feedback you might have about it.

@Mrida sorry, I am not really familiar with that part of the code. I recommend contacting the dev mailing list to get more attention at<EMAIL_ADDRESS>

Hello @trawick, I really appreciate any input you might have about this pr.

UP ! We need this change !


Search sidebar UI

Description
Moves the search UI in to the sidebar.

Adds simpler set of components for the search controls offering a lighter UI style

These components will eventually be used across the app, but for now are only used in search to reduce the scope of this changeset

Adds lucide-react for open source icons
Tweak some of the colors, like secondary text and border colors
Replace useFetchNode() with useAddToGraph() that takes any number of nodes or edges
Removes Carousel and Swiper since it was only used be the old search UI
Removes drag and drop logic since that was a part of the old search UI

It's possible users may ask for this back, but I want to see if we can get away with not having it

Validation

Related Issues

Check List

[x] I confirm that my contribution is made under the terms of the Apache 2.0
license.
[x] I have run pnpm checks to ensure code compiles and meets standards.
[x] I have run pnpm test to check if all tests are passing.
[x] I have covered new added functionality with unit tests if necessary.
[ ] I have added an entry in the Changelog.md.

I just found an issue. The empty state for the graph needs to have the messaging updated to reflect the move to the sidebar.

Fixed


Conflicts with React Router 1.0.0-rc1

I'm adding React Router for server side rendering, while I have React Mini Router for hashbang usage within a non-indexable client-only app. The issue is that React Router passes history object internally to every component it renders. React Mini Router uses this value to go itself to non-hash mode, which means when React Router renders the component React Mini Router will never go into hash mode.
Thus I think the check for history property should be explicit true value; or a more descriptive prop name should be used instead, because history={true} doesn't really tell what is going to be true; with React Router history prop is more understandable as you need to create a history object and pass that.

If you can submit a pull request that would be appreciated. Using two different routers in the same application isn't a use case that I need for any of the projects that use this library.

FYI I'm now refactoring codebase to only use React Router; quite a bit of more work to do and there will be some issues as I can't focus on every issue as I have to get things to production asap, but the result is cleaner overall.


2023.12 support

I am implementing 2023.12 support here. Note that some things, like the changes to dlpack, will not be wrapped and will require upstream support.
Rather than focusing on one library at a time, my plan is to implement the most important functions first for each library, and potentially do a release with partial support. Once full support is implemented I will update __array_api_version__. So if anyone has any functions that they'd like for me to prioritize please note them here. I am going to start with the functions used by scipy at https://github.com/scipy/scipy/blob/main/scipy/_lib/_array_api.py.
Here is the checklist of functions from https://github.com/data-apis/array-api-tests/issues/249:
Here is the list of changes https://data-apis.org/array-api/latest/changelog.html#v2023-12
New Functions

[ ] __array_namespace_info__ (i.e., the inspection API)
[ ] clip
[x] copysign
[ ] cumulative_sum
[ ] hypot
[x] maximum
[x] minimum
[ ] moveaxis
[ ] repeat
[ ] searchsorted
[ ] signbit
[ ] tile
[ ] unstack

API Changes

[ ] device keyword added to astype
[ ] ~from_dlpack requires exceptions in some cases~ (won't be wrapped)
[ ] ~copy and device keywords to from_dlpack~ (won't be wrapped)
[ ] sum, prod, trace output dtype change (https://github.com/data-apis/array-api-tests/issues/234)

Other Changes

[ ] tensordot should support negative axes

@lucascolley @mdhaber could you expand on what backend is missing minimum:
https://github.com/scipy/scipy/blob/fffcaa8c58b6909d22063c29e4f9593059a40230/scipy/_lib/_array_api.py#L478-L480
As far as I can tell, it's in numpy, torch, cupy, jax, and dask.array, so should already work. Is the issue that it's missing from array-api-strict?

Yes, probably implemented it because it was not in array-api-strict. I don't remember whether there were any other incompatibilities.

OK. I have 2023 strict support ready to go https://github.com/data-apis/array-api-strict/pull/35 (except for some annoying CI issues). I'll get a release out so you can use it.

Thanks!

searchsorted has also come up in SciPy, but personally happy to wait until full support.

I've just released array-api-strict with 2023.12 support. For now, it must be enabled manually using the new flags features https://data-apis.org/array-api-strict/api.html#module-array_api_strict._flags but once I'm confident there are no issues with it I'll make a release making 2023.12 the default in array-api-strict.

Merging what I have so far so I can do a release with partial support. Work on 2023.12 support will continue in another PR.


Edit landing page terminology section

I've only made minor edits, since I think our approach is already nicely described.

@rutemartins - I quickly did a git diff and it seems okay with just a couple of wordsmithing changes to the terminology paragraph.  Go ahead and push this individual file to the shimi branch.


Suggestion: Make the fuzzy finder classes exposed/exported for other plugins to use

I've been working on some Atom plugins recently and I wanted a fuzzy-finder that I could insert into different places as wanted, however the code here didn't lend itself too openly to that.
Would you accept changes that made it easier for other plugins to gain access to classes such as ProjectView, FuzzyFinderView, and BufferView?
My current idea is that these could either be exported in the package's main module as-is, and potentially additional constructors to aid creating them to avoid the recent issue where the constructors now require the metricsReporter to function.
Adding these changes into the upstream would allow metrics reporting to be done in use cases outside the scope of this plugin, which may or may not be desirable.
Two examples I think are clear:
switch-header-source
https://github.com/dschwen/switch-header-source/issues/36
Using the require(module.path + /lib/X.js hack/workaround to gain access to the Fuzzy Finder, would be easier if this plugin provided access or even easier if this plugin provided a factory function for getting new instances of a properly constructed FuzzyFinderView. (with metrics linked, etc)
There's also a lot of other calls to this packages functions which would be massively simplified
CodeRibbon
A plugin I'm working on is using a fuzzy-finder as the background element for Panes which don't have any items in them. This allows users to keep the tree view closed more often and still easily see a list of recent / active files in the project.
In my case I extended some classes from this plugin (also using the require(modulepath + file) hack) to get rid of the use of metricsReporter since I didn't have access to that variable when constructing new FuzzyFinderViews.
Extending the FuzzyFinderView: https://github.com/utk-se/CodeRibbon/blob/9ad197cf915a75cd62cdc929e3a564863581896f/lib/fuzzyfinder/fuzzy-finder-view.js
Using a fuzzy finder instance per pane: https://github.com/utk-se/CodeRibbon/blob/9ad197cf915a75cd62cdc929e3a564863581896f/lib/code-ribbon-patch.js#L359-L369
others
There are probably some other packages I don't know about that also either are currently using classes from this plugin, or would greatly benefit from being able to use a fuzzy finder that's already well known and tested.
I'm willing to make a PR with whatever changes we'd think would best solve this problem - but I'm still unsure of a few things:

Should we expose/export the metricsReporter instance that fuzzy-finder creates, in order to allow use of the exported class constructors?
Should we provide generator/factory functions that return prebuilt and initialized instances of ProjectView, FuzzyFinderView, and/or BufferView?
If we do make factory functions for those classes, should I make createProjectView() and friends use those new factory functions or leave them as is?

It seems like what you're asking for could potentially be provided through services? Please let me know if you think that's the case.

I believe it could, although I don't know how I'd make such a clear API for it like for example status-bar has.
I can't immediately image what I'd pass via something like consumeFuzzyFinder, the only thing that I instinctively think of would be to pass those 4 views that people want access to, and maybe the metricsReporter.
Perhaps if I were to go with the factory route, consumeFuzzyFinder(factory_func) would give the consumer a function to call to generate new instances of ProjectView, FuzzyFinderView, etc?
function fuzzyFinderFactory ( type: string ) {
  switch (type) {
    return new instance of that type;
  }
}

// consumer has:
consumeFuzzyFinderFactory(ff_factory_function) {
  my_plugins_fuzzy_finder = ff_factory_function('fuzzy-finder');
}

I guess the reason I didn't immediately go this route is because I thought:

Switching construction of the finders to a factory seemed a bit overkill for this plugin
Other plugin developers (like me) probably want to be able to create their own classes that extend from FuzzyFinderView, etc

My preconception is that it's easier to use a package's a.packages.loadPackage('fuzzy-finder').mainModule.SomeBaseClass or const {FuzzyFinderView} = require('fuzzy-finder') to extend new classes from, although I'm not nearly as experienced as some other Atom developers are.

Thanks for the feedback!
We don't feel that directly exposing the classes is the right way to go with this. Services are the mechanism that we created for Atom packages to depend on each other. This allows them to be versioned so that we don't have to worry about API mismatches. So we would be open to a well-written pull request that implements a service that other Atom packages could depend on.
Because this isn't something that we're going to work on ourselves, especially as described, we're going to go ahead and close this issue.
Thanks again for the great and detailed feedback!


Large lists causes variable explorer to bog down Jupyter session

I have a large list of results from a Scipy equation solver with each element being of type scipy.integrate._ivp.ivp.OdeResult. I've noticed that when I have a large results list, my Jupyter session becomes incredibly slow. From sys.getsizeof, I can see that the lists is relatively small (~103 to 104 bytes). Whereas, a pandas dataframe that I have is much larger (~107) and has not caused the same issue. I've used timers to determine the code itself is still running at the expected speed. It's only the interaction with the Jupyter session that is slow. After disabling the variable explorer, my interaction with Jupyter returns to normal. I suspect the issue has to do with the way the variable explorer is probing the variables. I would guess that accessing the list of results is inefficient and hence when the variable explorer constantly probes it, it slows everything down. In fact, when I call the list of results, it takes much longer to return the list as compared to the larger pandas dataframe mentioned earlier.

Is my intuition correct as to why the session slows down?
Would it be possible to add the ability to quickly enable and disable the variable explorer? Or at least have the ability to increase the amount of time between variable probes? As far as I can tell, the only way to fix the issue for me is to disable the variable explorer and then rebuild.

Thanks @lckr and others for a great extension!
A solution for this problem would be big, as it seems to still be common. I've largely had to stop using it, but would love to be able to pick it back up if the performance improved.
https://github.com/jupyter/notebook/issues/3303
https://github.com/jupyter/notebook/issues/3224#issuecomment-382300098


fix(esbuild): update tsconfck to fix bug that could cause a deadlock

where parsing a tsconfig with references that extend it again.
See https://github.com/dominikg/tsconfck/issues/154

Description

Additional context

What is the purpose of this pull request?

[ ] Bug fix
[ ] New Feature
[ ] Documentation update
[ ] Other

Before submitting the PR, please make sure you do the following

[ ] Read the Contributing Guidelines, especially the Pull Request Guidelines.
[ ] Check that there isn't already a PR that solves the problem the same way to avoid creating a duplicate.
[ ] Provide a description in this PR that addresses what the PR is solving, or reference the issue that it solves (e.g. fixes #123).
[ ] Update the corresponding documentation if needed.
[ ] Ideally, include relevant tests that fail without this PR but pass with it.

/ecosystem-ci run


[MAJOR] js: Move to webpack, pug from r.js, jade

Introduction
As already discussed in https://github.com/posativ/isso/issues/754, try to make Isso more maintainable by facilitating modern testing practices.
This PR is a bit massive, but trying to make pug or other updated deps play nice with RequireJs just for the sake of a clean history is not something I'm prepared to do.
So, here goes:

js: Move to webpack, pug from r.js, jade
In the interest of maintainability and integration with other testing tools, get rid of RequireJS (sometimes also called r.js or requirejs)
This necessitates the following changes:

Removing the almond (AMD) module loader (superseded by webpack internals)
Removing requirejs-text for loading static resources (superseded by webpack asset/source loader)
Removing jade templating engine (superseded by pug, which represents the original jade project that had to be renamed due to trademark issues)
pug v2+ no longer accepts 'mystr-{#foo.bar}'-style variables inside attributes, so *.pug templates need to be changed accordingly (mystr' + foo.bar)
Note: templates have been moved to templates/ directory and renamed from .jade to .pug.
jade-runtime and accompanying shims (superseded by simply pug-runtime)
Move .svg files inside own dir, simplify loading via webpack's asset/source
Add webpack config
Add build scripts to package.json
Adjust Makefile to use webpack via npm run
Emit JS source map files from webpack (eases debugging)
.gitignore JS source map files

While RequireJS utilized the following syntax:
define(["app/foo"], function(foo) {
  var bar = doSomeStuff...
  return bar;
})

Jest and other testing libraries need the NodeJS standard CommonJS style:
const foo = require("app/foo");
var bar = doSomeStuff...
module.exports = { bar };

Note: ES6/ECMAScript2015-style import has not been used since it would need transpilation via e.g. babel to be understood by Jest.
The RequireJS-style wrapping of all code inside define({...}) caused everything to be indented by at least one level.
To keep the changes in this commit manageable, indentation will be removed in a later commit.
isso: js: treewide: De-indent one level
Since switching from RequireJS to CommonJS syntax, the define() wrapper is no longer needed and as such, the the code formerly inside parentheses can be de-indented by one level.
isso: js: i18: RequireJS to module.exports
(Just an extension of previous commit, to avoid touching too many files at once)
demo: Use dev version of embed.js
This makes it easier to debug when testing. The demo shouldn't be user-facing, but either way, the performance penalty should be outweighed by debugging benefits.
isso: js: Reduce deps for app/lib/
(Just a minor commit to make the following unit tests easier)

Verifying
You can verify this commit by checking out this PR:
git clone https://github.com/posativ/isso isso
cd isso/
git fetch origin pull/800/head:pull-800
git checkout pull-800

And then installing everything as documented in docs: Install from Source.
Then, either embed Isso into your site or use the demo at localhost:8080/demo

A note for people from the future: You can use git blame --ignore-rev=<sha> to "ignore" the de-indent commits. See A better git blame.
Note to myself: We can probably ditch the templates altogether and simply use plain html with javascript string interpolation. That way, we lose pug, pug-loader and pug-runtime, which would make the bundle significantly smaller.

Pinging @stefangehn, @blatinier, @Lucas-C and anyone else interested in moving this project forward.

I've created a package for easier testing:
isso-0.12.5-webpack.tar.gz

This is great! Awesome job on moving everything to ES6 modules & webpack

Thank you for reviewing!  Thanks for keeping me honest ;) I'll document as I have written in individual responses inline.

Maybe this should be merged as part of #801 as we would have the added confidence of the new tests?

While waiting for the test suite to be adequate might sound like a sensible strategy for keeping bugs out, it'll also mean keeping up a massive amount of rebase work and effectively blocking any contribs to the client (or doing a lot of merge conflict work).
So I'll be selfish and merge this webpack/pug change soon-ish, before tests are done. My "justification" for this is that once 0.12.6 is released, we can afford to be open for integrating more major changes.


Add g_ prefix to file_operations and mountpt_operations

Summary
Continue the work https://github.com/apache/nuttx/pull/9033:

fs: Add g_ prefix for all global file_operations instances
fs: Add g_ prefix for all global mountpt_operations instances

Impact
code refactor only
Testing
CI

@xiaoxiang781216
What is the motivation for this change?
I find confusing to tag a static const with g_ as they are only passed to register_driver() and never reference them outside the module.

No, it reference outside the source code.

Does the CS need better constraints? https://nuttx.apache.org/docs/latest/contributing/coding_style.html#global-variables
Please ask for consensus on the mailing list for this change.

I find confusing to tag a static const with g_ as they are only passed to register_driver() and never reference them outside the module.

The coding standard has always required that static const begin with the prefix g_  .  All code that I am aware is like this.  The code of this PR did not follow the coding standard.
That is the way it has always been and I hope we do not change the coding standard and introduce such an inconsistency in the naming pattern.  g_ indicates the scope.  "static const" is irrelevant in this regard.


SeqN Editor Test

Closes #1284
I added a SeqN Editor playwrite test.

@duranb I went through and removed a lot of the this.updatePage()
I need to have it when we are switching back and forth from the sequence form page and the sequence editor page or the test will not be able to see the HTML elements


Fixed logout unknown command when no user login

Pull Request Test Coverage Report for Build 256

0 of 3 (0.0%)  changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.08%) to 82.87%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/main/java/seedu/address/logic/parser/DefaultParser.java
0
1
0.0%

src/main/java/seedu/address/logic/parser/exceptions/InvalidLogoutException.java
0
2
0.0%

Totals

Change from base Build 255:
-0.08%

Covered Lines:
2564

Relevant Lines:
3094

💛  - Coveralls


new features: add distributed scheduling support.

new features: add distributed scheduling support.

same as #3732


Cannot call macro defined in the same source file

Scala Version: 3.0.0
Scalatest Version: 3.2.9
Minimized code
MainMacro.scala
import scala.quoted.*

case class Wrapper[T](t: T) extends MainMacro[T]

trait MainMacro[U] { this: Wrapper[U] =>
  inline def showTypeRepr: String = ${ MainMacro.showTypeReprImpl[U]}
}

object MainMacro {
  def showTypeReprImpl[U: Type](using Quotes): Expr[String] = {
      import quotes.reflect.*

      val tpe = TypeRepr.of[U]

      '{${Expr(tpe.toString)}}
  }
}

Main.scala
object TestData {
  case class Person(name: String, age: Int)
}

object Test1 extends App {
  import TestData.*

    println(Wrapper(Person("a", 1)).showTypeRepr)
}

Test1.scala
import org.scalatest.flatspec.AnyFlatSpec
import org.scalatest.matchers.should.Matchers

object TestData {
    case class Person(name: String, age: Int)

}
class Test1 extends AnyFlatSpec with Matchers {
  import TestData.*

  it should "xyz" in {
       Wrapper(Person("a", 1)).showTypeRepr
  }
}

Output

sbt clean run - as expected

TypeRef(ThisType(TypeRef(ThisType(TypeRef(NoPrefix,module class <empty>)),module class TestData$)),class Person)

sbt clean test  - error

[error] 12 |       Wrapper(Person("a", 1)).showTypeRepr
[error]    |       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[error]    |       Cannot call macro class Person defined in the same source file
[error]    | This location contains code that was inlined from Test1.scala:12

It may be related to https://github.com/lampepfl/dotty/issues/12498

@mbore @bvenners I did a quick test on this one, the problem can be reproduced without ScalaTest, you may change your example Test1.scala to just:
object TestData {
    case class Person(name: String, age: Int)

}
class Test1 {

  import TestData.*

  inline def in(fun: => Any): Any = fun

  in {
    Wrapper(Person("a", 1)).showTypeRepr
  }
}

and try to compile, you should see the same error.  I think the problem is in Scala 3's inline function, fyi we have another inline function related report here (with different error):
https://github.com/scalatest/scalatest/issues/2030
I asked a question in the Dotty gitter, but so far no clue about the cause yet, I'll see if I should file an issue to Dotty's issue tracker tomorrow.

We initially reported this in dotty (https://github.com/lampepfl/dotty/issues/12498), but it turns out it doesn't reproduce on dotty master. However, the also found a variant which manifests itself only sometimes, and consistently when used with ScalaTest - hence this issue. The answer is somewhere in the triangle dotty-sbt-scalatest :)

@mbore @adamw Starting my day here to work out an example project that can reproduce the problem without ScalaTest:
https://github.com/cheeseng/scala3-inline-problem
Note that if this object TestData in https://github.com/cheeseng/scala3-inline-problem/blob/main/src/test/scala/Test1.scala#L1 is moved into a separate file, the error does go away.  But if the following:
object TestData {
  case class Person(name: String, age: Int)
}

object Test1 extends App {
  import TestData.*

    println(Wrapper(Person("a", 1)).showTypeRepr)
}

is suppose to work, I don't see why the example does not work.

@mbore @adamw I updated the example to use Scala 3.0.1, but unfortunately still has the same problem.

@mbore @adamw I updated the example to use Scala 3.0.2, but unfortunately still has the same problem.  It can be reproduced in:
https://github.com/cheeseng/scala3-inline-problem

@mbore @adamw This seems to be fixed now in Scala 3.1.2, can you verify, and if it does, kindly close the issue.
Thanks.

I've just tried to run this example with Scala 3.1.2 and it seems to be still broken... did you add any flags?

@mbore Nope I didn't, I am trying this with https://github.com/cheeseng/scala3-inline-problem , if you use 3.1.1 you'll see the error, but using 3.1.2 seems to fix it, are you running the same project?

This still exists for me. Unfortunately, this prevents deriving things like Codecs on companion objects.
  case class Person(
    name: String,
    age: Int
  )

  object Person {
    val schema = DeriveCodec.gen[Person]
  }

@mbore @kitlangton Not sure how I got the 3.1.2 without error, I check again today and found that scala 3.1.3 still has the same problem, I updated the example repo https://github.com/cheeseng/scala3-inline-problem , I'll ping in dotty's issue also.

Any updates on this? Its scala 3.3.0 now???


docs: Fix mailbox k_mbox_msg.tx_block documentation

Replace old tx_block.pool_id to tx_block.data;
Fix code examples.

Signed-off-by: Rodrigo Peixoto<EMAIL_ADDRESS>Signed-off-by: Rodrigo Peixoto<EMAIL_ADDRESS>Signed-off-by: Anas Nashif<EMAIL_ADDRESS>

Codecov Report

Merging #8387 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #8387   +/-   ##
=======================================
  Coverage   64.61%   64.61%
=======================================
  Files         423      423
  Lines       40293    40293
  Branches     6801     6801
=======================================
  Hits        26034    26034
  Misses      11126    11126
  Partials     3133     3133

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6c60abb...285ea76. Read the comment docs.


Document Path Redirects and Rewrites (GEP 726)

This GEP was implemented but has not been documented yet.

/assign

/unassign @robscott

The todo list here:

[ ] Take the GEP (https://gateway-api.sigs.k8s.io/geps/gep-726/) and write some documentation that explains how to use the feature. This probably needs to be a separate Guide under the guides heading (stored in /site-src/v1alpha2/guides).
[ ] #1124 will mean that there will be slight schema changes to any examples.

/help-wanted


kafka: add support for metadata request in mesh-filter

Commit Message: kafka: add support for metadata request in mesh-filter
Additional Description: Add another request handler - metadata requests are used by Kafka clients to figure out where Kafka servers actually reside (first client connects to any server, sends metadata asking who owns partitions x y z and then connects there). Given that we want Envoy to act as Kafka server (as we want whole traffic to pass thru it), we need to pretend that we are Kafka server - this is handled in MetadataRequestHolder::computeAnswer.
Risk Level: Low
Testing: unit tests
Docs Changes: N/A
Release Notes: N/A
Platform Specific Features: N/A

/wait

/wait

/wait

/assign @mattklein123

/wait


Since updates, no longer uploads location to server

Since the app updated like 2-3 updates ago, it no longer uploads my location to my private server. I says send failed. I have had my IT department look into it and they cannot get the app to work. Is their a way to get the previous version or way to walk me through what changed and make it work with my server?

Send failed means that either app hasn't received a reply from the server or there was some issue and server returned error in response. You need to check server logs to see what happened.


Just a heads up

This is not really an issue with the software and you can close this issue right away. Just wanted to point out that the README.md lists the URL for 0.0.2-alpha under "Quickstart using a packaged release" while the URL points to 0.0.3-alpha under "Quickstart using a pre-built chroot".

Thanks! I've gone ahead and updated it

Nice. Thank you for your work with this, exactly what I need :)


Use  in keymapping to prevent flicker on some UIs

Using : in the mapping causes some UIs to glitch when repeating a command (.) on a textobject. <cmd> prevents that.

Thank you!


UpdateStatus Request fails (doesn't use Auth-Token)

I wondered, why we always get something like:
...
07:44:49 java.io.FileNotFoundException: {"message":"Not Found","documentation_url":"https://developer.github.com/v3"}
07:44:49 	at org.kohsuke.github.Requester.handleApiError(Requester.java:590)
07:44:49 	at org.kohsuke.github.Requester._to(Requester.java:267)
07:44:49 	at org.kohsuke.github.Requester.to(Requester.java:208)
07:44:49 	at org.kohsuke.github.GHRepository.createCommitStatus(GHRepository.java:871)
07:44:49 	at org.jenkinsci.plugins.ghprb.extensions.status.GhprbSimpleStatus.createCommitStatus(GhprbSimpleStatus.java:273)
07:44:49 	at org.jenkinsci.plugins.ghprb.extensions.status.GhprbSimpleStatus.onBuildStart(GhprbSimpleStatus.java:186)
07:44:49 	at org.jenkinsci.plugins.ghprb.GhprbBuilds.onStarted(GhprbBuilds.java:132)
...

This is Jenkins  2.7 with ghprb-1.32.4 and github-api-1.75.
Digging a little bit deeper revealed, that the Authorization header incl. Token (which has repo:status included) is not send in status update and probably therefore it fails.

Correction: 'uc.getRequestProperties()' doesn't return all RequestProperty entries actually being set - it hides some, like 'Authorization'. However, setting the logger for 'sun.net.www.protocol.http.HttpURLConnection' to 'ALL' causes it to show all headers, and there we can see, that the 'Authorization' header is used as expected. So the bug is probably on the github server side?
POST /repos/foo/foo/statuses/00f19478fb4a5e508d9e53a702283bbd25c5cc73 HTTP/1.1
Authorization: token<PHONE_NUMBER>abcdef0123456789abcdef01234567
Accept-Encoding: gzip
Content-type: application/x-www-form-urlencoded
User-Agent: Java/1.8.0_45-internal
Host: api.github.com
Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2
Connection: keep-alive
Content-Length: 122

{target_url=http://come.home:8080/job/foo-PRs/29/, description=Build started sha1 is merged., state=pending}

HTTP/1.1 404 Not Found
Server: GitHub.com
Date: Wed, 15 Jun 2016 18:16:39 GMT
Content-Type: application/json; charset=utf-8
Transfer-Encoding: chunked
Status: 404 Not Found
X-RateLimit-Limit: 5000
X-RateLimit-Remaining: 4951
X-RateLimit-Reset:<PHONE_NUMBER>
X-OAuth-Scopes: public_repo, read:org, read:repo_hook, repo:status, user:email
X-Accepted-OAuth-Scopes:
X-GitHub-Media-Type: github.v3
Access-Control-Expose-Headers: ETag, Link, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval
Access-Control-Allow-Origin: *
Content-Security-Policy: default-src 'none'
Strict-Transport-Security: max-age=31536000; includeSubdomains; preload
X-Content-Type-Options: nosniff
X-Frame-Options: deny
X-XSS-Protection: 1; mode=block
Content-Encoding: gzip
X-GitHub-Request-Id: 8D2C185A:1C0C2:19CC89AE:57619B87

This looks like a credentials issue

Well-written issue but the 404 implies to me that it's an issue with credentials either with the token being used, or with the repo. I'm closing for now as I do a sweep of older issues.

@benpatterson @jelmd I just got this exact error and opened an issue for it, however I ended up fixing it by renaming the repository and then changing it back to the original value, so it seemed to in fact be on the Github side of things.
More details here: https://issues.jenkins-ci.org/browse/JENKINS-40359


FlowExports floods log file with confusing logs

Describe the bug

While investigating https://github.com/antrea-io/antrea/issues/5565, I found there are massive logs generated by flow exporter and the content doesn't look right.
W1002 13:52:27.801284       1 connections.go:129] Could not retrieve the Service info from antrea-agent-proxier for the serviceStr: **.**.**.**:8126/TCP
W1002 13:52:32.747359       1 connections.go:129] Could not retrieve the Service info from antrea-agent-proxier for the serviceStr: **.**.**.**:8126/TCP
W1002 13:52:42.758149       1 connections.go:129] Could not retrieve the Service info from antrea-agent-proxier for the serviceStr: **.**.**.**:8126/TCP
W1002 13:52:45.332383       1 connections.go:129] Could not retrieve the Service info from antrea-agent-proxier for the serviceStr: **.**.**.**:8126/TCP
W1002 13:52:52.764561       1 connections.go:129] Could not retrieve the Service info from antrea-agent-proxier for the serviceStr: **.**.**.**:8126/TCP
W1002 13:53:02.775244       1 connections.go:129] Could not retrieve the Service info from antrea-agent-proxier for the serviceStr: **.**.**.**:8126/TCP
W1002 13:53:04.557143       1 connections.go:129] Could not retrieve the Service info from antrea-agent-proxier for the serviceStr: **.**.**.**:8126/TCP
W1002 13:53:12.786529       1 connections.go:129] Could not retrieve the Service info from antrea-agent-proxier for the serviceStr: **.**.**.**:8126/TCP
W1002 13:53:22.788296       1 connections.go:129] Could not retrieve the Service info from antrea-agent-proxier for the serviceStr: **.**.**.**:8126/TCP
W1002 13:53:32.796952       1 connections.go:129] Could not retrieve the Service info from antrea-agent-proxier for the serviceStr: **.**.**.**:8126/TCP
W1002 13:53:37.747412       1 connections.go:129] Could not retrieve the Service info from antrea-agent-proxier for the serviceStr: **.**.**.**:8126/TCP
W1002 13:53:47.755555       1 connections.go:129] Could not retrieve the Service info from antrea-agent-proxier for the serviceStr: **.**.**.**:8126/TCP

The masked IP was a Node IP, and the port was definitely not a NodePort, so trying to getting the Service by this IP and port is destined to fail.
W1005 02:33:43.104906       1 connections.go:112] Cannot map any of the IP <IP_ADDRESS> or **.**.**.** to a local Pod
W1005 02:33:43.104975       1 connections.go:112] Cannot map any of the IP <IP_ADDRESS> or **.**.**.** to a local Pod
W1005 02:33:43.104986       1 connections.go:112] Cannot map any of the IP <IP_ADDRESS> or **.**.**.** to a local Pod
W1005 02:33:43.105031       1 connections.go:112] Cannot map any of the IP <IP_ADDRESS> or **.**.**.** to a local Pod
W1005 02:33:43.105072       1 connections.go:112] Cannot map any of the IP <IP_ADDRESS> or **.**.**.** to a local Pod
W1005 02:33:43.105088       1 connections.go:112] Cannot map any of the IP <IP_ADDRESS> or **.**.**.** to a local Pod
W1005 02:33:43.105101       1 connections.go:112] Cannot map any of the IP <IP_ADDRESS> or **.**.**.** to a local Pod
W1005 02:33:43.105112       1 connections.go:112] Cannot map any of the IP <IP_ADDRESS> or **.**.**.** to a local Pod

The masked IP was an external IP, I think this is a flow from a remote Node to an external IP via the Egress IP on this Node.
Expected

FlowExporter shouldn't flood log files
The logs shouldn't be confusing.

Versions:
Please provide the following information:

Antrea version (Docker image tag).
v1.11.3 but issue may exist all existing releases.

Does the masked IP originally look like this (...)?

Does the masked IP originally look like this (...)?

Sorry, I don't get it. You mean whether there are parentheses? I only masked the numbers, there was no parentheses. I believe the mismatch is not caused by extra characters, otherwise the whole feature wouldn't work at all.

I thought they were asterisks.
I'm confused about the scenarios. So the masked IP in two pictures are different?
For the first picture, we want to figure out why the Node IP is treated as a service?
For the second picture, does the antrea-agent on the Node have all the connections via the Egress IP on that Node?
<IP_ADDRESS>, <IP_ADDRESS>...these IPs should be the Pod IP in other Nodes?  Since the Flow Exporter will only do fillPodInfo once for each unique connection, I think the logs in the second picture are reasonable?
What's the expected behavior we want? Increase the log-level for these logs?

I thought they were asterisks.
I'm confused about the scenarios. So the masked IP in two pictures are different?

I replaced the IPs with asterisks to avoid leaking users' node IP and public IP they are using.
The two masked IPs are different. I have indicated the first one is a Node IP and the second one is an external IP.

For the first picture, we want to figure out why the Node IP is treated as a service?

I have explained the queried IP is definitely not a Service IP, writing such warnings to log files is useless and arouses unnecessary concern when user look at it and when debugging real issue, and it causes useful logs to be evicted. My expectation is this log shouldn't be there at all. Of course the reason why they exist is because the Node IP is mistreated as service by some code or there is perhaps no attempt to check whether the flow belongs to pod-to-service at all.

For the second picture, does the antrea-agent on the Node have all the connections via the Egress IP on that Node?
<IP_ADDRESS>, <IP_ADDRESS>...these IPs should be the Pod IP in other Nodes?

These Pod IPs belongs to Pods on other Nodes.

Since the Flow Exporter will only do fillPodInfo once for each unique connection, I think the logs in the second picture are reasonable?
What's the expected behavior we want? Increase the log-level for these logs?

Why it even needs to log this when it's impossible to find the pod information for a flow between a remote Pod and external IP? The expectation is to not log error/warning when everything works as expected, and not log useless INFO. RemotePod-to-External is not the only case that non Pod information can be found, there are other cases like External-to-LoadBalancerIP/NodePort which selects a remote Pod.

I think the logs in the second picture are reasonable?

Note that if they are reasonable, we will have nothing to debug when troubleshooting real issue as it could generate hundreds of lines of such log if you look at the timestamp of the second section. Actually it has caused the log has nothing useful when investigating https://github.com/antrea-io/antrea/issues/5565.


Added DRF PDF

What does this PR do?
Add resource(s) | Remove resource(s) | Add info | Improve repo
For resources
Description
Why is this valuable (or not)?
How do we know it's really free?
For book lists, is it a book? For course lists, is it a course? etc.
Checklist:

[ ] Read our contributing guidelines.
[ ] Search for duplicates.
[ ] Include author(s) and platform where appropriate.
[ ] Put lists in alphabetical order, correct spacing.
[ ] Add needed indications (PDF, access notes, under construction).
[ ] Used an informative name for this pull request.

Follow-up

Check the status of GitHub Actions and resolve any reported warnings!

I have made the necessary changes please merge it now

Please add the "Compiled from StackOverflow documentation" attribution we use for goalkicker/riptutorial books

Duplicate of #7220