Parser error with SVG datauri inlining

I hate to open an unclear/ambiguous issue, but I've been unable to track down what's actually going wrong here.
With the following CSS, simply running css.parse() is blowing up:

Error: undefined:1:249: property missing ':'

It seems to only be occurring when there are multiple rules on one line (i.e. minified) that contain the string "charset", only when the SVG is URL encoded. I've stripped it down as much as I can to get it to a readable size (original file: http://www.shopify.com/assets2/application-bf82c00b0b5b610626f416a6ccc3d6aa.css).
Any thoughts on what might be going on here? I'll try to take a closer look in the next few days but figured I'd open the issue in the meantime. Thanks!
.thing1{background-image:url("data:image/svg+xml;charset=utf8,%3Csvg%3E%3Cg%3E%3Crect%20transform=%22matrix(0.7071%20-0.7071%200.7071%200.7071%20-9.5016%2021.7357)%22/%3E%3C/g%3E%3C/svg%3E");}.thing2{background-image:url("data:image/svg+xml;charset=utf8,%3Csvg%3E%3C/svg%3E");}

The underlying issues (at least I'm pretty sure) is that the parsing for a value stops at the closing ) of matrix(, inside the quoted string. This is because as soon as it finds an open ( it searches for the closest matching ) (see this line).
A shorter example that reproduces the issue (I'm pretty sure it is the same issue, just a slightly different error) is this: .a{ b: url(")"); } .c { d: url(";"); }. Remove the ) in ")" and it parses fine.

Actually you can reproduce the exact same error message (property missing ':') with this: .a { b: url(")"); d: url(";a"); }.

Hi,
Is there any advance on this issue ?
I encounter the same problem with SVG inlined which I have reduced to the following "conditions" :
It seems to occur when there's more than one call to an SVG in an inlined CSS.

2 SVG calls with inlined CSS -> it doesn't work

.foo{background-image:url("data:image/svg+xml;charset=utf8,%3Csvg%20version%3D%221.1%22%20xmlns%3D%22http%3A%2F%2Fwww.w3.org%2F2000%2Fsvg%22%20viewBox%3D%220 0 13 9%22%20preserveAspectRatio%3D%22none%22%3E%0A%09%3Cpath%20fill%3D%22rgba(5,178,189,.99)%22%20d%3D%22M8.53 0L7.19 1.352l2.18 2.193H0v1.912h9.37L7.19 7.65l1.34 1.352 4.47-4.5z%22%20%2F%3E%0A%3C%2Fsvg%3E")}.bar{background-image:url("data:image/svg+xml;charset=utf8,%3Csvg%20version%3D%221.1%22%20xmlns%3D%22http%3A%2F%2Fwww.w3.org%2F2000%2Fsvg%22%20viewBox%3D%220 0 25 21%22%20preserveAspectRatio%3D%22none%22%3E%0A%09%3Cpath%20fill%3D%22rgba(5,46,73,.99)%22%20d%3D%22M7.5,15.6l-4.8-4.9L0,13.4l6.4,6.5l0,0L7.5,21l18-18.3L22.8,0L7.5,15.6z%22%20%2F%3E%0A%3C%2Fsvg%3E")}

Same as before, but with a new line between the 2 SVG calls -> it works

.foo{background-image:url("data:image/svg+xml;charset=utf8,%3Csvg%20version%3D%221.1%22%20xmlns%3D%22http%3A%2F%2Fwww.w3.org%2F2000%2Fsvg%22%20viewBox%3D%220 0 13 9%22%20preserveAspectRatio%3D%22none%22%3E%0A%09%3Cpath%20fill%3D%22rgba(5,178,189,.99)%22%20d%3D%22M8.53 0L7.19 1.352l2.18 2.193H0v1.912h9.37L7.19 7.65l1.34 1.352 4.47-4.5z%22%20%2F%3E%0A%3C%2Fsvg%3E")}
.bar{background-image:url("data:image/svg+xml;charset=utf8,%3Csvg%20version%3D%221.1%22%20xmlns%3D%22http%3A%2F%2Fwww.w3.org%2F2000%2Fsvg%22%20viewBox%3D%220 0 25 21%22%20preserveAspectRatio%3D%22none%22%3E%0A%09%3Cpath%20fill%3D%22rgba(5,46,73,.99)%22%20d%3D%22M7.5,15.6l-4.8-4.9L0,13.4l6.4,6.5l0,0L7.5,21l18-18.3L22.8,0L7.5,15.6z%22%20%2F%3E%0A%3C%2Fsvg%3E")}

1 SVG call -> it works

.foo{background-image:url("data:image/svg+xml;charset=utf8,%3Csvg%20version%3D%221.1%22%20xmlns%3D%22http%3A%2F%2Fwww.w3.org%2F2000%2Fsvg%22%20viewBox%3D%220 0 13 9%22%20preserveAspectRatio%3D%22none%22%3E%0A%09%3Cpath%20fill%3D%22rgba(5,178,189,.99)%22%20d%3D%22M8.53 0L7.19 1.352l2.18 2.193H0v1.912h9.37L7.19 7.65l1.34 1.352 4.47-4.5z%22%20%2F%3E%0A%3C%2Fsvg%3E")}

I got this issue while using purifycss and used postcss-base64 to encode any inlined SVG into base64 as a workaround until https://github.com/reworkcss/css/pull/110 is merged.

New editor styles transformation approach using PostCSS is now merged (many thanks @zaguiini)!


test(html/parser): Flatten tests to make git faster

Description:
This PR flattens html tests to make git faster

Seems like html tests are quite complex.
cc @alexander-akait How can I flatten this?
Performance difference of git status -z -u is about 0.45 sec vs 0.4 sec

File count is fine. Only directory count matters

WIP on improvements

Closing in favor of https://github.com/swc-project/swc/pull/6158


list_activity_events is not considering user_id_filter and activity_filter filter.

Describe the bug
list_activity_events is not considering user_id_filter and activity_filter filter.
df_activity_events = labs.admin.list_activity_events(
start_time =  "2024-10-19T10:00:00",
end_time =  "2024-10-19T10:10:00",
activity_filter="GetWorkspaces",
user_id_filter="xyz@abc.com",)
Expected behavior
it should return only one activity for and only for given user. But it is returning many activities and for all the user in the given time window.

Fixed in 0.8.4.


feat: 1.0 Release

PR summary
This PR contains the changes that will be merged into "main" to produce the 1.0 release of the validator.  I'm opening the PR now as a draft so that we can identify any merge conflicts as we're preparing the 1.0-prelease branch.
PR Checklist
General checklist
Please make sure that your PR fulfills the following requirements:

[ ] The commit message follows the Angular Commit Message Guidelines.
[ ] Tests for the changes have been added (for bug fixes / features)
[ ] Docs have been added / updated (for bug fixes / features)
[ ] Dependencies have been updated as needed
[ ] .secrets.baseline updated as needed?

Checklist for adding a new validation rule:

[ ] Added new validation rule definition (packages/ruleset/src/rules/*.js, index.js)
[ ] If necessary, added new validation rule implementation (packages/ruleset/src/functions/*.js, updated index.js)
[ ] Added new rule to default configuration (packages/ruleset/src/ibm-oas.js)
[ ] Added tests for new rule (packages/ruleset/test/*.test.js)
[ ] Added docs for new rule (docs/ibm-cloud-rules.md)

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.2 out of 3 committers have signed the CLA.:white_check_mark: padamstx:white_check_mark: dpopp07:x: semantic-release-botYou have signed the CLA already but the status is still pending? Let us recheck it.

:tada: This PR is included in version 1.0.0 :tada:
The release is available on npm package (@latest dist-tag)
Your semantic-release bot :package::rocket:

:tada: This PR is included in version 1.0.0 :tada:
The release is available on npm package (@latest dist-tag)
Your semantic-release bot :package::rocket:


Should each Dockerfile be moved to the corresponding language server repository?

That way the deployment process from the perspective of the language server author would only require committing a change to their Dockerfile in their repository (and maybe cutting a release).
Unless there is a process watching for releases and pushing images to https://hub.docker.com/r/sourcegraph/, language server authors will need their own Docker Hub account and set up automated pushes.
There are probably other tradeoffs to take into consideration 💭

This issue has been mostly resolved by the addition of Renovate in #98


implement odm naming_strategy option in ODM

similar to doctrine orm naming strategy:
naming_strategy: doctrine.orm.naming_strategy.underscore
http://elnur.pro/how-to-enable-the-underscore-naming-strategy-for-doctrine-orm-in-symfony/
let's implement the stuff but in ODM
:+1:

Closing this: I believe it's better to explicitly map the field name. Making this a config option would allow userland code to change it, which can then cause serious issues (as has happened to me in an ORM project).


chore(release): v1.6.0 🎉

:robot: I have created a release beep boop
1.6.0 (2024-12-26)
Features

terms: use uni-time glossary for duration term (3268ee6)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/ehmpathy/simple-on-disk-cache/releases/tag/v1.6.0 :sunflower:


User defined keras callbacks

The current KerasBaseTask class cannot be flexibly changed Keras callbacks by users. We need to modify the class so that the callbacks can be changed flexibly.

Implemented https://github.com/UTokyo-ICEPP/multiml/commit/f0b1670afb31a8750a7060d0aa998cceef855f90


Formatting Issue

The time selection is going 100% across the screen and repeating times across the day. I really liked the app when it worked before this change that happened just this week on the herokuapp.

I added a screen cap to show off the issue.

Thank you for the fix!

Hey @CarlJKashnier. Has the issue resolved? I don't know what's causing the app to act that way.
I haven't had any time to work on LetsMeet, if you want to give it a shot and work on the codebase - feel free to do so!

@AkiraLaine - It is fixed, thank you.

This can be closed.


Rednose and config

I've added rednose to python 3 dependencies too (that fixed the tests).
Thanks!


'ERR_UNHANDLED_REJECTION'

Hello, I just started learning programming, please tell me what the problem is?
MintyTON % yarn start
yarn run v1.22.19
$ tsc --skipLibCheck && node dist/app.js
Started uploading images to IPFS...
node:internal/process/promises:289
triggerUncaughtException(err, true /* fromPromise */);
^
[UnhandledPromiseRejection: This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). The promise rejected with the reason "#".] {
code: 'ERR_UNHANDLED_REJECTION'
}
Node.js v20.5.1
error Command failed with exit code 1.

`import dotenv from "dotenv";
import { toNano } from "ton-core";
import { readdir } from "fs/promises";
import { openWallet } from "./utils";
import { waitSeqno } from "./delay";
import { NftCollection } from "./contracts/NftCollection";
import { NftItem } from "./contracts/NftItem";
import { updateMetadataFiles, uploadFolderToIPFS } from "./metadata";
import { GetGemsSaleData, NftSale } from "./contracts/NftSale";
import { NftMarketplace } from "./contracts/NftMarketplace";
dotenv.config();
async function init() {
const metadataFolderPath = "./data/metadata/";
const imagesFolderPath = "./data/images/";
const wallet = await openWallet(process.env.MNEMONIC!.split(" "), true);
console.log("Started uploading images to IPFS...");
const imagesIpfsHash = await uploadFolderToIPFS(imagesFolderPath);
console.log(
Successfully uploaded the pictures to ipfs: https://gateway.pinata.cloud/ipfs/${imagesIpfsHash}
);
console.log("Started uploading metadata files to IPFS...");
await updateMetadataFiles(metadataFolderPath, imagesIpfsHash);
const metadataIpfsHash = await uploadFolderToIPFS(metadataFolderPath);
console.log(
Successfully uploaded the metadata to ipfs: https://gateway.pinata.cloud/ipfs/${metadataIpfsHash}
);
console.log("Start deploy of nft collection...");
const collectionData = {
ownerAddress: wallet.contract.address,
royaltyPercent: 0.05, // 0.05 = 5%
royaltyAddress: wallet.contract.address,
nextItemIndex: 0,
collectionContentUrl: ipfs://${metadataIpfsHash}/collection.json,
commonContentUrl: ipfs://${metadataIpfsHash}/,
};
const collection = new NftCollection(collectionData);
let seqno = await collection.deploy(wallet);
console.log(Collection deployed: ${collection.address});
await waitSeqno(seqno, wallet);
// Deploy nft items
const files = await readdir(metadataFolderPath);
files.pop();
let index = 0;
seqno = await collection.topUpBalance(wallet, files.length);
await waitSeqno(seqno, wallet);
console.log(Balance top-upped);
for (const file of files) {
console.log(Start deploy of ${index + 1} NFT);
const mintParams = {
queryId: 0,
itemOwnerAddress: wallet.contract.address,
itemIndex: index,
amount: toNano("0.05"),
commonContentUrl: file,
};
const nftItem = new NftItem(collection);
seqno = await nftItem.deploy(wallet, mintParams);
console.log(Successfully deployed ${index + 1} NFT);
await waitSeqno(seqno, wallet);
index++;
}
console.log("Start deploy of new marketplace  ");
const marketplace = new NftMarketplace(wallet.contract.address);
seqno = await marketplace.deploy(wallet);
await waitSeqno(seqno, wallet);
console.log("Successfully deployed new marketplace");
const nftToSaleAddress = await NftItem.getAddressByIndex(collection.address, 0);
const saleData: GetGemsSaleData = {
isComplete: false,
createdAt: Math.ceil(Date.now() / 1000),
marketplaceAddress: marketplace.address,
nftAddress: nftToSaleAddress,
nftOwnerAddress: null,
fullPrice: toNano("10"),
marketplaceFeeAddress: wallet.contract.address,
marketplaceFee: toNano("1"),
royaltyAddress: wallet.contract.address,
royaltyAmount: toNano("0.5"),
};
const nftSaleContract = new NftSale(saleData);
seqno = await nftSaleContract.deploy(wallet);
await waitSeqno(seqno, wallet);
await NftItem.transfer(wallet, nftToSaleAddress, nftSaleContract.address);
}
void init();`


Fix Scene Background Parser for SkyBox

Please check if the PR fulfills these requirements

[x] The commit message follows our guidelines
[ ] Tests for the changes have been added (for bug fixes / features)
[ ] Docs have been added / updated (for bug fixes / features)

fix: https://github.com/oasis-engine/editor/issues/96

Codecov Report

Merging #361 (aa3717e) into main (6a3706d) will increase coverage by 0.15%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##             main     #361      +/-   ##
==========================================
+ Coverage   44.83%   44.98%   +0.15%
==========================================
  Files         314      314
  Lines       17484    17422      -62
  Branches     2340     2327      -13
==========================================
- Hits         7839     7838       -1
+ Misses       9645     9584      -61

Impacted Files
Coverage Δ

packages/loader/src/scene-loader/SceneManager.ts
0.00% <0.00%> (ø)

packages/draco/src/decoder.ts
0.00% <0.00%> (-0.92%)
:arrow_down:

packages/loader/src/scene-loader/Oasis.ts
0.00% <0.00%> (ø)

...ackages/loader/src/scene-loader/ResourceManager.ts
0.00% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6a3706d...aa3717e. Read the comment docs.


[question] Sending messages to topics

Does this library support sending messages to topics?
https://firebase.google.com/docs/cloud-messaging/android/topic-messaging

instead of opening this issue you could have simply make a simple search for the word "topic" in this module's landing page and discover that it does. would probably take you less time.

I only see subscribing / unsubscribing. Is there a way to also SEND a message to a topic? @bardu @atlanteh

It supports registering to topics. You can't send notification messages from any client sdk as you need to suply the server key which should only be known to you.


Fix #911 Add Eclipse Che Installer

test this

Codecov Report

Merging #913 into master will decrease coverage by 0.34%.
The diff coverage is 95.83%.

@@            Coverage Diff             @@
##           master     #913      +/-   ##
==========================================
- Coverage   95.49%   95.15%   -0.35%
==========================================
  Files          39       40       +1
  Lines        3443     3655     +212
  Branches      295      521     +226
==========================================
+ Hits         3288     3478     +190
- Misses        155      167      +12
- Partials        0       10      +10

Impacted Files
Coverage Δ

browser/model/che.js
89.74% <95.83%> (ø)

browser/services/credentialManager.js
96.87% <0%> (-3.13%)
:arrow_down:

browser/pages/location/controller.js
92.3% <0%> (-2.86%)
:arrow_down:

browser/pages/install/controller.js
78.84% <0%> (-2.75%)
:arrow_down:

browser/main.js
98.48% <0%> (-1.52%)
:arrow_down:

browser/model/devstudio.js
96.8% <0%> (-0.97%)
:arrow_down:

browser/services/platform.js
99.11% <0%> (-0.89%)
:arrow_down:

browser/model/cdk.js
99.21% <0%> (-0.79%)
:arrow_down:

browser/model/helpers/downloader.js
95.27% <0%> (-0.63%)
:arrow_down:

browser/services/data.js
97.06% <0%> (-0.59%)
:arrow_down:

... and 27 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 379f202...2f71e89. Read the comment docs.


Artifact Hub Improvements

Overview
Artifact Hub is an open source project for finding, installing and publishing Kubernetes packages. The OpenFeature Operator (OFO) is available on Artifact Hub but the configuration is basic. Improvements should be made to increase its discoverability.
Requirements

Searching for "openfeature" or "open feature" should return the OFO
A readme should be included on the landing page
A change log to should included
The OpenFeature logo should be used
A "verified publish" badge should be added
An "official" badge should be added (if possible)

Resources

https://artifacthub.io/docs/topics/repositories/helm-charts/
https://artifacthub.io/docs/topics/annotations/helm/

@beeme1mr  This requires an account to be setup for the OpenFeature org on Artefact Hub, otherwise there will be an association with a personal account (this is the current state)


Show the offline message only once on Course Listing

Show the offline message only once on Course Listing and not when user returns back from previous screens.
@rohan-dhamal-clarice @aleffert - Please review
JIRA: https://openedx.atlassian.net/browse/MOB-1579

:+1:

:+1:


test: parity roundtripping

Motivation
#496
Solution
PR Checklist

[x] Added Tests
[ ] Added Documentation
[ ] Breaking changes

CI failure is unrelated.


I want to know location x/y?

I get JSON data by 'Format.JSON', but I don't know what json means, please!

There is help for you
https://www.json.org/json-en.html


Eservice database changes

This PR introduces functionality for eservice database. The database manager is a separate class providing basic db functionality. The data itself is stored as json. CLI is provided via commands such as pdo-create-eservicedb, pdo-update-eservicedb etc. Documentation can be found in eservice/docs/.
A new run-tests-with-database.sh shell script has been added to built/tools to show the usage of the database for pdo test scripts.
Signed-off-by: prakashngit<EMAIL_ADDRESS>

Too many conflicts while squashing, will fix and PR again


Which branch should I use for version 93.0.4577.82?

I've found both branch is updated for version 93.0.4577.82,
and branch v92.0.4515.131 is newer than branch master.
Which branch should I use for version 93.0.4577.82?
I've found two differences for patches in the two branches:

windows-fix-building-without-safebrowsing.patch are different
There is patches/my in branch master but not in branch v92.0.4515.131

BTW, I've tried to upgrade to 93.0.4577.82 before I saw your upgrade, it's not easy for me, thanks.

v92.0.4515.131 should be used for v93. Appearently using the version number as a branch name is a bad idea. However, there is an open PR so I cannot rename the branch until the PR is closed/merged.
The only difference between windows-fix-building-without-safebrowsing.patch is that the v92 branch contains a fix for the download UI.
patches/my contains patches that I'd like to have, but they may not be for everyone.

v94.0.4606.71 is only in the master branch, has download UI been fixed in v94.0.4606.71?

Yes, but you might want to remove patches in patches/my if you intend to use the master branch.


Page refresh doesn't work

When refreshing the page, the path is not recognized.
You can only access it using the UI.

Works on the dev machine, but not in Azure.


The SQL can't run on distributed mode

What type of bug is this?
Unexpected error
What subsystems are affected?
Frontend
What happened?
The following sql can't run on distributed mode, but works in standalone:
CREATE TABLE IF NOT EXISTS timestamp (t TIMESTAMP, ts TIMESTAMP time index DEFAULT current_timestamp());

INSERT INTO timestamp VALUES ('2008-01-01 00:00:01', 1), (NULL, 2), ('2007-01-01 00:00:01', 3), ('2008-02-01 00:00:01', 4), ('2008-01-02 00:00:01', 5), ('2008-01-01 10:00:00', 6), ('2008-01-01 00:10:00', 7), ('2008-01-01 00:00:10', 8);

SELECT t-t FROM timestamp;

The error:
 Error: 3001(EngineExecuteQuery), No field named timestamp.t. Valid fields are "timestamp.t - timestamp.t

What operating system did you use?
Any OS
Relevant log output and stack trace
No response
How can we reproduce the bug?
As described above.

Wait for https://github.com/apache/arrow-datafusion/pull/7224

Wait for apache/arrow-datafusion#7224

It was merged, when will we update datafusion?

Wait for apache/arrow-datafusion#7224

It was merged, when will we update datafusion?

Recently. I have another unmerged patch

It's resolved after https://github.com/GreptimeTeam/greptimedb/pull/2659
MySQL [(none)]> CREATE TABLE IF NOT EXISTS timestamp (t TIMESTAMP, ts TIMESTAMP time index DEFAULT current_timestamp());
Query OK, 0 rows affected (0.002 sec)

MySQL [(none)]> INSERT INTO timestamp VALUES ('2008-01-01 00:00:01', 1), (NULL, 2), ('2007-01-01 00:00:01', 3), ('2008-02-01 00:00:01', 4), ('2008-01-02 00:00:01', 5), ('2008-01-01 10:00:00', 6), ('2008-01-01 00:10:00', 7), ('2008-01-01 00:00:10', 8);
Query OK, 8 rows affected (0.001 sec)

MySQL [(none)]> SELECT t-t FROM timestamp;
+---------------------------+
| timestamp.t - timestamp.t |
+---------------------------+
| 00:00:00                  |
| NULL                      |
| 00:00:00                  |
| 00:00:00                  |
| 00:00:00                  |
| 00:00:00                  |
| 00:00:00                  |
| 00:00:00                  |
+---------------------------+
8 rows in set (0.002 sec)


In windows zig exe source path not encapsulated with " causing failure when a space is present in file path.

relevant when pressing "run" or "debug"

+1 on this, every single MacOS user who updated to newest version will have this crash because of the MacOS path: /Users/<username>/Library/Application Support/Code/User/globalStorage/ziglang.vscode-zig/zig/macos-aarch64-0 .13.0/zig


Spectra don't appear to work

The spectra do not seem to take in data under any condition. Using both resource explorer and getspectrum -v3, I see no data ever recorded to any spectrum. However, they are clearly connected within the firmware diagram. I added diagnostic information to the oscilloscope on the status of the charge integrator. I may have fixed this but haven't had a chance to test it yet.

The spectra apparently didn't have any memory assigned to them. However, because I have been informed they aren't supposed to be read using the generated library, I've removed them instead of working further on the getspectrum program.


too short variables in constuctor

Hello. My structure initializing in constructor with options is marked as "too short".
What do you think about option to skip this case.
struct st {
  p1 bool
  p2 bool
  ....
}

type Option func(*st)

func New() st {
   s := st{
     // set default values
   }

   for _, o := range oo {
      o(&s)
   }

   return s
}

In this case, a comment //nolint:varnamelen should be used.


Manual install fastlane on mac

I want to install manually fastlane and have it in custom folder, then just PATH to direct to its bin
but when running fastlane --version
/System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require': cannot load such file -- fastlane/cli_tools_distributor (LoadError)
	from /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
	from /Users/marius/dev/fastlane-2.216.0/bin/fastlane:12:in `<main>'

my zsh path setup
export PATH=$HOME/dev/fastlane-2.216.0/bin:$PATH

Not sure if this is possible or do i just need to install via Brew ?

My guess it's related to using the default Ruby on your Mac and permissions. I would recommend not directly working with the default Ruby on macOS.


Document (and/or debug) launching rkt in rkt

(Posted at the request of @alban ): There are several use cases for launching rkt containers inside of other rkt containers. While this is possible, there are potential pitfalls and configuration requirements for doing this successfully. It would be great if those pitfalls and requirements were documented, or if running rkt in rkt was explicitly supported or not supported by the project.

Just tested it:
I start the outer rkt with:
sudo rkt --dns=<IP_ADDRESS> --insecure-options=image  run --interactive docker://debian

Then, I start the inner rkt with:
apt-get update
apt-get install wget
wget https://github.com/coreos/rkt/releases/download/v1.0.0/rkt-v1.0.0.tar.gz
tar xf rkt-v1.0.0.tar.gz
cd rkt-v1.0.0
./rkt run --interactive --insecure-options=image --no-overlay --net=host docker://busybox

It works with the following limitations:

I used --net=host in the inner pod because I didn't give CAP_NET_ADMIN to the outer pod.
I used --no-overlay in the inner pod because otherwise I get #2045.

didn't make it due to OCI/Fest activity

from #2167:

Sooner than later we'll run kubernetes's kubelet with stage1-fly, which will itself spawn rkt pods. Here are my first thoughts of what we need to investigate, mainly which paths to mount into the stage1-fly environment.
Eventually, the kubelet will spawn rkt pods, and I think we need a couple things for rkt to work properly in rkt fly. Here are some thoughts.
We need /etc/rkt for various rkt and CNI configurations. We have possible conflicts with CNI if the kubelet and the host both execute rkt, so we need at least /var/lib/cni for the host-local allocator.
Then there might be issues with GCing. If /var/lib/rkt will not be mounted the pods started by the kubelet are not going to be GCd by the host's GC timer.
Note that it won't be mounted recursively and will not have overlayfs mounts etc. from the host's pods but that, but I'm not 100% sure about the consequences this will have if rkt GC would be called by the kubelet and only find the empty pod directories. @jonboulle @vcaputo maybe one of you can foresee this?
Where will the rkt API service be running? This is going to be tricky if the kubelet and the host both run pods and don't share resources examined by the API service. @yifan-gu

Moving to next milestone

If we mount /sys without the cgroup fs in /sys/fs/cgroup with https://github.com/coreos/rkt/pull/2680, we will need to document how to mount the cgroup fs in this issue.

moving to next milestone, didn't make it due to other activities

@s-urbaniak will be focusing on architecture documentation for 1.10

sorry, same fate as https://github.com/coreos/rkt/issues/2115#issuecomment-230443598 :-(

One possible use case for the documentation: https://github.com/blablacar/dgr/issues/170
/cc @nyodas

This feature is currently blocked by https://github.com/coreos/rkt/issues/2874, because currently it is not possible to mount /sys/fs/cgroup in the pod.

This feature is currently blocked by https://github.com/coreos/rkt/issues/2874, because currently it is not possible to mount /sys/fs/cgroup in the pod.

Instead of bind-mounting /sys/fs/cgroup from the host into the outer rkt, I think that a better way would be to run a shell script in the outer rkt that mount manually the cgroup filesystems (new mounts instead of bind-mounts).
The benefits would be:

it would not break when cgroup namespaces become a reality (coming up with https://github.com/systemd/systemd/pull/3589 and https://github.com/coreos/rkt/issues/1757)
if we use network inside the outer rkt and need /sys/class/net/ for some reason, it will point to the correct network namespace.

See how it can be done:
https://github.com/jpetazzo/dind/blob/master/wrapdocker

Having a working rkt-in-rkt story is unfortunately a pretty long story. We can start a "privileged" pod using:
$ sudo rkt run \
  --volume rkt,kind=host,source=/usr/bin/rkt \
  --volume usr-lib-rkt,kind=host,source=/usr/lib/rkt \
  --volume etc-rkt,kind=host,source=/etc/rkt \
  --mount volume=rkt,target=/usr/bin/rkt \
  --mount volume=usr-lib-rkt,target=/usr/lib/rkt \
  --mount volume=etc-rkt,target=/etc/rkt \
  --insecure-options=image \
  docker://progrium/busybox \
  --dns=<IP_ADDRESS> \
  --interactive \
  --exec=/bin/bash \
  --cap-retain=CAP_SYS_ADMIN,CAP_SYS_CHROOT

After further experimentation, and as @alban explained above, the general strategy is to establish new cgroups mounts in this "privileged outer pod. But currently we get this:
$ mount -t sysfs sysfs /sys
mount: mounting sysfs on /sys failed: Device or resource busy

This is because /sys/fs is marked as a inaccessible directory in https://github.com/coreos/rkt/blob/master/stage1/init/common/pod.go#L1087.
We need to need to make additional tricks to remount sysfs in the pod:
mkdir /sys2
mount -t sysfs sysfs /sys2
mount --bind /sys2/fs /sys/fs
umount /sys2

Note: This remount "trick" won't work any more, once seccomp will be introduced. Now we can finally mount cgroups:
CGROUP=/sys/fs/cgroup
mount -n -t tmpfs -o uid=0,gid=0,mode=0755 cgroup $CGROUP

for SUBSYS in $(cut -d: -f2 /proc/1/cgroup)
do
        [ -d $CGROUP/$SUBSYS ] || mkdir $CGROUP/$SUBSYS
        mountpoint -q $CGROUP/$SUBSYS ||
                mount -n -t cgroup -o $SUBSYS cgroup $CGROUP/$SUBSYS
done

After having this setup, I still get an error (@iaguis, @alban ideas?) when trying to run an inner pod (reproducible using a progrium/busybox, and ubuntu outer pod):
$ opkg-install ca-certificates
$ rkt run --debug --insecure-options=image --interactive docker://busybox --exec=/bin/sh
image: using image from local store for image name coreos.com/rkt/stage1-coreos:1.10.1
image: using image from local store for url docker://busybox
run:
  └─error rendering tree store
    └─cannot render aci
      └─error extracting ACI
        └─extracttar error: exit status 1, output: error extracting tar

Summary:

The above described "workaround" to remount /sys/fs does not seem to be a viable recommendation to document a canonical rkt-in-rkt documentation.

The corrupt cas directory needs investigation.

We need a way to "opt-out" of security features to fully idiomatically enable rkt-in-rkt. The current consensus is to have a --privileged[=false] flag to implement the above without specifying separate capability, seccomp, innaccessiable path settings.

Starting from systemd v231 /proc/sys/net will be mounted read-write [2] if the container is in a user network namespaces though so I'll retest once systemd v231 lands in rkt v1.14.0.
[1] https://github.com/systemd/systemd/blob/v230/src/nspawn/nspawn-mount.c#L302
[2] https://github.com/systemd/systemd/blob/v231/src/nspawn/nspawn-mount.c#L302

bumping to next release for having a complete network story.

short update: systemd v231 still doesn't help unfortunately. system-nspawn remounts /proc/sys/net iff --private-network was being passed which we don't. We should probably pass an env var to systemd-nspawn signalling we are already in a network namespace similar as it is done here https://github.com/systemd/systemd/commit/0c582db0c6ec3330953270cf0324038cb6615617.
/cc @alban is this is an approach worth pursuing upstream in systemd-nspawn?

@s-urbaniak I guess yes... they seem open to more env var (https://github.com/systemd/systemd/pull/4018#discussion_r76971862)
But instead of an env var about network namespace, it could be an env var about disabling custom paths protections in general.

The outstanding issue (having /proc/sys read-write) is tracked in #3245

https://github.com/coreos/rkt/issues/3245 is still WIP, hence bumping.

rkt-in-rkt basically works using #3389, and https://github.com/systemd/systemd/pull/4395 when invoked as follows:
$ sudo rkt run \
  --volume rkt,kind=host,source=/usr/bin/rkt \
  --volume stage1,kind=host,source=/usr/lib/rkt/stage1-images/stage1-coreos.aci \
  --volume etc-rkt,kind=host,source=/etc/rkt \
  --mount volume=rkt,target=/usr/bin/rkt \
  --mount volume=stage1,target=/usr/lib/rkt/stage1-images/stage1-coreos.aci \
  --mount volume=etc-rkt,target=/etc/rkt \
  --insecure-options=all-run,image \
  docker://progrium/busybox \
  --net=host \
  --dns=<IP_ADDRESS> \
  --interactive \
  --exec=/bin/sh

/ # opkg-install --force-depends iptables ca-certificates
/ # rkt run --no-overlay --dns=<IP_ADDRESS> --insecure-options=image --interactive docker://progrium/busybox --exec /bin/sh
Downloading sha256:dfda3e01f2b [=============================]   243 KB / 243 KB
Downloading sha256:00cf8b9f3d2 [=============================]     220 B / 220 B
Downloading sha256:3aaade50789 [=============================]     247 B / 247 B
Downloading sha256:7ff999a2256 [=============================] 2.53 KB / 2.53 KB
Downloading sha256:a3ed95caeb0 [=============================]       32 B / 32 B
Downloading sha256:a3ed95caeb0 [=============================]       32 B / 32 B
Downloading sha256:77c6c00e8b6 [=============================] 2.17 MB / 2.17 MB
Downloading sha256:d2ba336f2e4 [=============================] 33.7 KB / 33.7 KB
run: group "rkt" not found, will use default gid when rendering images
stage1: warning: error setting journal ACLs, you'll need root to read the pod journal: group "rkt" not found
/ # ping www.google.de
PING www.google.de (<IP_ADDRESS>): 56 data bytes
64 bytes from <IP_ADDRESS>: seq=0 ttl=59 time=29.820 ms

It reveals the following process tree:
bash
 \_ sudo ./build-rkt-1.19.0+git/target/bin/rkt run --debug --stage1-path=/home/sur/src
     \_ stage1/rootfs/lib64/ld-linux-x86-64.so.2 stage1/rootfs/usr/bin/systemd-nspawn
         \_ /usr/lib/systemd/systemd --default-standard-output=tty
             \_ /usr/lib/systemd/systemd-journald
             \_ /bin/sh
                 \_ stage1/rootfs/lib64/ld-linux-x86-64.so.2 stage1/rootfs/usr/bin/sys
                     \_ /usr/lib/systemd/systemd --default-standard-output=tty --log-t
                         \_ /usr/lib/systemd/systemd-journald
                         \_ /bin/sh

Overlay in the inner container doesn't work though (needs investigation).

The proc, and sys mounts in the container now are as follows:
/ # /bin/mount | grep -e 'on \/sys \|on \/proc'
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
tmpfs on /proc/sys/kernel/random/boot_id type tmpfs (ro,nosuid,nodev,mode=755)
tmpfs on /proc/kmsg type tmpfs (rw,nosuid,nodev,mode=755)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)

When running the container with overlay (aka overlayfs-over-overlayfs), I get the following error which needs investigation:
$ sudo rkt run \
  --volume rkt,kind=host,source=/usr/bin/rkt \
  --volume stage1,kind=host,source=/usr/lib/rkt/stage1-images/stage1-coreos.aci \
  --volume etc-rkt,kind=host,source=/etc/rkt \
  --mount volume=rkt,target=/usr/bin/rkt \
  --mount volume=stage1,target=/usr/lib/rkt/stage1-images/stage1-coreos.aci \
  --mount volume=etc-rkt,target=/etc/rkt \
  --insecure-options=all-run,image \
  docker://progrium/busybox \
  --net=host \
  --dns=<IP_ADDRESS> \
  --interactive \
  --exec=/bin/sh

/ # opkg-install --force-depends iptables ca-certificates
/ # rkt run --no-overlay --dns=<IP_ADDRESS> --insecure-options=image --interactive docker://progrium/busybox --exec /bin/sh
/ # rkt run --debug --dns=<IP_ADDRESS> --insecure-options=image --interactive docker://progrium/busybox --exec /bin/sh
image: using image from file /usr/bin/stage1-src.aci
image: using image from local store for url docker://progrium/busybox
stage0: Preparing stage1
stage0: Writing image manifest
stage0: Loading image sha512-cb2b26ae3f8bbf71c1380b6a2b6f599dcfce023722812e89e6f6bfb1553b90f4
stage0: Writing image manifest
stage0: Writing pod manifest
run: group "rkt" not found, will use default gid when rendering images
stage0: Setting up stage1
stage0: error setting up stage1
  └─error rendering overlay filesystem
    └─problem mounting overlay filesystem
      └─error mounting overlay with options 'lowerdir=/var/lib/rkt/cas/tree/deps-sha512-c4824b4fb40d4c2b1b5f44e24d2f89adb0a0e01d9ee542255f7fee455734c22a/rootfs,upperdir=/var/lib/rkt/pods/run/b50f6362-600a-400b-b71e-1149fcb0b39b/overlay/deps-sha512-c4824b4fb40d4c2b1b5f44e24d2f89adb0a0e01d9ee542255f7fee455734c22a/upper,workdir=/var/lib/rkt/pods/run/b50f6362-600a-400b-b71e-1149fcb0b39b/overlay/deps-sha512-c4824b4fb40d4c2b1b5f44e24d2f89adb0a0e01d9ee542255f7fee455734c22a/work' and dest '/var/lib/rkt/pods/run/b50f6362-600a-400b-b71e-1149fcb0b39b/stage1/rootfs'
        └─invalid argument

Regarding the overlay-over-overlay problem, I see the following kernel log entry:
[78743.001372] overlayfs: filesystem on '/var/lib/rkt/pods/run/3a821735-5c22-42f4-a3cf-b9eaa8586c2b/overlay/deps-sha512-cc290423b58b10408609a9a52bc11036255d945d2b8c7dbfac3ac1a193f747d3/upper' not supported as upperdir

The kernel docs https://www.kernel.org/doc/Documentation/filesystems/overlayfs.txt say:
The upper filesystem will normally be writable and if it
is it must support the creation of trusted.* extended attributes, and
must provide valid d_type in readdir responses, so NFS is not suitable.

Since / in the outer container is mounted on an overlay filesystem the above constraint is not satisfied.
There are several possibilities which work:

Start the outer container using the --no-overlay option. This causes / to be mounted natively.
Mount a host volume on a non-overlay filesystem as /var/lib/rkt to the outer container.
Start the inner container using the --no-overlay option.

@s-urbaniak this is on a modern kernel? thought it was fixed in https://github.com/coreos/rkt/issues/1537 (which we call out in README.md)

@jonboulle this is kernel 4.8.8 (running Arch)

i.e. the test from https://github.com/coreos/rkt/issues/1537#issuecomment-145478517 fails with the above message.

Note that this is another error message "not supported as upperdir" vs. "No such device or address"

OK, I suggest closing this out and adding a caveat + follow-up issue for nested overlay (also README note might need a tweak?)

Should running rkt in a rkt container work in 1.19.0 with --no-overlay, now?  I'm still running into a couple of different problems when trying it on CentOS 7 with kernel 4.8.9 (mlkernel from elrepo).  I looked at the issues closed on the v1.20.0 milestone for actual code changes related to running rkt in rkt, but nothing stood out to indicate this shouldn't now work with 1.19.0.
Based on @s-urbaniak's examples above (mainly specifying --no-overlay for both containers):
[root@localhost ~]# rkt run --no-overlay --volume rkt,kind=host,source=/usr/bin/rkt --volume stage1,kind=host,source=/usr/lib/rkt --volume etc-rkt,kind=host,source=/etc/rkt  --mount volume=rkt,target=/usr/bin/rkt   --mount volume=stage1,target=/usr/lib/rkt   --mount volume=etc-rkt,target=/etc/rkt   --insecure-options=all   docker://ubuntu   --net=host   --dns=<IP_ADDRESS>   --interactive   --exec=/bin/bash
root@rkt-faa1536d-ff94-44ad-ab3d-9847d87d1ce3:/# rkt run --debug --no-overlay --insecure-options=all   docker://progrium/busybox   --net=host   --dns=<IP_ADDRESS>   --interactive   --exec=/bin/sh
image: using image from file /usr/lib/rkt/stage1-images/stage1-coreos.aci
image: remote fetching from URL "docker://progrium/busybox"
image: fetching image from docker://progrium/busybox
Downloading sha256:a3ed95caeb0 [=============================]       32 B / 32 B
Downloading sha256:a3ed95caeb0 [=============================]       32 B / 32 B
Downloading sha256:3aaade50789 [=============================]     247 B / 247 B
Downloading sha256:d2ba336f2e4 [=============================] 33.7 KB / 33.7 KB
Downloading sha256:dfda3e01f2b [=============================]   243 KB / 243 KB
Downloading sha256:00cf8b9f3d2 [=============================]     220 B / 220 B
Downloading sha256:7ff999a2256 [=============================] 2.53 KB / 2.53 KB
Downloading sha256:77c6c00e8b6 [=============================] 2.17 MB / 2.17 MB
stage0: Preparing stage1
stage0: Writing image manifest
stage0: Loading image sha512-e350475af8d32ff7973ce7c4940ed6e0abf4c5c8f3da48c3b72b872e0d8516ae
stage0: Writing image manifest
stage0: Writing pod manifest
run: group "rkt" not found, will use default gid when rendering images
stage0: Setting up stage1
stage0: Wrote filesystem to /var/lib/rkt/pods/run/eade84e6-47ce-4993-a30d-160223543b2d
stage0: Pivoting to filesystem /var/lib/rkt/pods/run/eade84e6-47ce-4993-a30d-160223543b2d
stage0: Execing [/var/lib/rkt/pods/run/eade84e6-47ce-4993-a30d-160223543b2d/stage1/rootfs/init --debug --net=host --interactive --local-config=/etc/rkt --dns-conf-mode=resolv=stage0,hosts=default --disable-capabilities-restriction --disable-paths --disable-seccomp eade84e6-47ce-4993-a30d-160223543b2d]
stage1: warning: error setting journal ACLs, you'll need root to read the pod journal
  └─group "rkt" not found
stage1: error getting container subcgroup
  └─could not determine if we're running from a unit file
    └─error calling sd_pid_get_owner_uid: no medium found
root@rkt-faa1536d-ff94-44ad-ab3d-9847d87d1ce3:/# echo $?
254

If I use progrium/busybox as the outer container I get a different error:
[root@localhost ~]# rkt run --no-overlay --volume rkt,kind=host,source=/usr/bin/rkt --volume stage1,kind=host,source=/usr/lib/rkt --volume etc-rkt,kind=host,source=/etc/rkt  --mount volume=rkt,target=/usr/bin/rkt   --mount volume=stage1,target=/usr/lib/rkt   --mount volume=etc-rkt,target=/etc/rkt   --insecure-options=all   docker://progrium/busybox   --net=host   --dns=<IP_ADDRESS>   --interactive   --exec=/bin/sh
/ # rkt run --debug --no-overlay --insecure-options=all   docker://progrium/busybox   --net=host   --dns=<IP_ADDRESS>   --interactive   --exec=/bin/sh
image: using image from file /usr/lib/rkt/stage1-images/stage1-coreos.aci
image: remote fetching from URL "docker://progrium/busybox"
image: fetching image from docker://progrium/busybox
Downloading sha256:a3ed95caeb0 [=============================]       32 B / 32 B
Downloading sha256:a3ed95caeb0 [=============================]       32 B / 32 B
Downloading sha256:dfda3e01f2b [=============================]   243 KB / 243 KB
Downloading sha256:3aaade50789 [=============================]     247 B / 247 B
Downloading sha256:d2ba336f2e4 [=============================] 33.7 KB / 33.7 KB
Downloading sha256:7ff999a2256 [=============================] 2.53 KB / 2.53 KB
Downloading sha256:77c6c00e8b6 [=============================] 2.17 MB / 2.17 MB
Downloading sha256:00cf8b9f3d2 [=============================]     220 B / 220 B
stage0: Preparing stage1
stage0: Writing image manifest
stage0: Loading image sha512-e350475af8d32ff7973ce7c4940ed6e0abf4c5c8f3da48c3b72b872e0d8516ae
stage0: Writing image manifest
stage0: Writing pod manifest
run: group "rkt" not found, will use default gid when rendering images
stage0: Setting up stage1
stage0: Wrote filesystem to /var/lib/rkt/pods/run/c385b98a-6f4c-4adb-96e4-8e2a06fc5956
stage0: Pivoting to filesystem /var/lib/rkt/pods/run/c385b98a-6f4c-4adb-96e4-8e2a06fc5956
stage0: Execing [/var/lib/rkt/pods/run/c385b98a-6f4c-4adb-96e4-8e2a06fc5956/stage1/rootfs/init --debug --net=host --interactive --local-config=/etc/rkt --dns-conf-mode=resolv=stage0,hosts=default --disable-capabilities-restriction --disable-paths --disable-seccomp c385b98a-6f4c-4adb-96e4-8e2a06fc5956]
stage1: warning: error setting journal ACLs, you'll need root to read the pod journal
  └─group "rkt" not found
stage1: error getting container subcgroup
  └─could not determine if we're running from a unit file
    └─unable to open a handle to the library
/ # echo $?
254

@blalor The corresponding PR didn't land yet in master, it will land in v1.21.0.

@s-urbaniak but this issue is against 1.20.0?

@jonboulle whoops, you are right, let me reassign this issue to v1.21.0, it is still pointing to the initial target release, and I will also create a follow-up issue regarding overlay.


header X-Content-Type-Options: nosniff not customizable

Hi,
I noted that header X-Content-Type-Options: nosniff is present in Shinyproxy responses.
That's nice, but I also add this header on proxy, therefore I would like to turn it off.
Unfortunately, I don't see an option for this.
BTW: it is a little connected to #254, according to it this header was added silently in version 2.4.x. Maybe we could add some switch for it in application.yaml? BTW, I would love to have also some other headers like CSP, X-XSS-Protection, Cache-Control, etc. customizable.
Or maybe I miss something in configuration?
Thanks,
Jakub

Hi
It's currently indeed not possible to tweak these headers. What you could do is to configure the proxy to remove the X-Content-Type-Options header. For example for nginx, you can use:
proxy_hide_header X-Content-Type-Options;

Nevertheless, I understand that it would be useful to configure the headers a bit more.
Do you think it would make sense to have one property that disables all headers set by ShinyProxy and the another property in ShinyProxy where you can specify all your customs headers? For example:
proxy:
  disable_security_headers: true
  add_response_headers:
   - "myHeader: myValue:
   - etc

BTW if you would end up with a working CSP policy (or any other useful header), could you please share this with us? I did not have the time to dig into how those headers could secure ShinyProxy, though it could be very interesting to have these.

Hi, I guess this issue affected my work, I've changed the default html, so I use French character, but I'm having issue with the response because shiny proxy is using the default header  Content-Language: en-US

Hi, today we release ShinyProxy 3.0.0, in which it is now possible to customize these headers, please have a look at the documentation: https://shinyproxy.io/documentation/configuration/#disable-no-sniff-header


replace eval with safer literal_eval

When looking through the examples I noticed quite a few are using eval, this might be potentially unsafe if someone runs it let's say on an embeddings file from someone else, could execute arbitrary python code hidden there. I am suggesting replacement with literal_eval, which is also used in other examples in the repo.
Also VSCode seems to insist on making some formatting changes that I haven't figured out how to stop. If this is an issue please feel free to reject.

Thanks. This looks safer. Have you checked these examples? Otherwise I'll find time to check this week before merging.


Replace Magnific Popup with Photoswipe (same author, better touch handling)

I would be interested to work together on this issue, if you think it's a good way to go for this project.
It's also, not requiring jquery (FYI)
On the website of Maginific Popup the author claims:

If you wish to add touch-swipe support, check my article on the Smashing Magazine, or new PhotoSwipe script.

Sure, if you want to submit a PR with PhotoSwipe as a drop-in replacement for Magnific Popup I'd be happy to look it over.
The version of MP the theme ships with is outdated too, so it would be nice to move to something lighter and with better touch support.


v-bind="$attrs" in <component> does not render as is attribute

Version
3.2.31
Reproduction link
sfc.vuejs.org/
Steps to reproduce

Add is attribute in custom component

What is expected?
component with v-bind="$attrs" is correctly parse as is $attrs.
What is actually happening?
component is rendered like a web component.
is attribute needs to be binded explicitly.

I don't know if it is a bug or working as intended, but I do expect that all $attrs are forwarded and binded to the component instance and rendered as is tag attribute.
I'm trying to create a base component that works as a foundation for others components that don't know the context when it will be used, so I cannot make is explicit at this level.
To make it work I have to bind explicitly the is attribute.
npm ls:
├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>└──<EMAIL_ADDRESS>

is attribute is needed(compile time) for <component>  to be considered a dynamic component.
<script setup>
  defineProps(['is'])
</script>

<template>
  <pre>$attrs: {{ JSON.stringify($attrs) }}</pre>
  <component v-bind="$attrs" :is=$props.is>
    <slot />
  </component>
</template>


Values for Server 2019

Please update for filter values for Server 2019 family

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 7937ded2-6bd7-e34e-08a8-5e7f6e9ffe4c
Version Independent ID: b4313161-be46-059c-6a6c-1af2a90252f6
Content: Create WMI Filters for the GPO (Windows 10)
Content Source: windows/security/threat-protection/windows-firewall/create-wmi-filters-for-the-gpo.md
Product: w10
Technology: windows
GitHub Login: @Dansimp
Microsoft Alias: dansimp

Good point. I can only offer to create a PR to add the information, but I am not familiar or experienced with this topic, so I would need to have the information pointed out to me. (Looking forward to any follow-up from the Microsoft Docs team members.)

@officedocsbot assign @jvsam

Thank you for your feedback @pyoungberg. Although, looking at the theme, the deployment guide, including all the procedures used in it are intended for Windows 10 & Windows Server 2016 OS.
What do you think @illfated? Will adding 2019 information—if we have the information—fit since the content will deviate from the applicable OS?
Anyway, there's an interesting discussion in the Tech Community regarding Windows Server 2019 WMI Filters or Security Group Filtering, with sample filter but for Windows Server 2019 Domain Controllers only. Found a lot of external resources like this, though we cannot guarantee the accuracy. There is a TechNet article but it does not have 2019 specifically, but have sample strings that target Desktop OS and 64 Bit.  It helps to confirm.

Given that the Windows Server 2019 WMI Filters content is very different (or quite different) from the Windows Server 2016 WMI Filters content, I prefer to see what the document author would say about this. I do not know enough about this content to tell if this page is the right place to add Windows Server 2019 WMI Filters content or if it belongs on a separate page for Windows Server 2019.

Haven't tested as the documentation out there was spotty. Was hoping for an authoritative answer. I'd assume you could just post or link to a list of Windows 10 releases w/ version numbers we the users could sort out the rest.

Thank you for your feedback, I hope it will be useful enough for the maintainers of this page & topic.
@Dansimp : Could we have a comment from you regarding future plans or actions regarding documentation of Windows Server 2019 WMI Filters for GPO?

Thanks @pyoungberg is this what you are looking for?
Thanks @illfated. @Dansimp can you provide some insights on this one please?

Thanks @pyoungberg is this what you are looking for?
Thanks @illfated. @Dansimp can you provide some insights on this one please?

Yup thanks I can make a filter with that!

That's great @pyoungberg, any update? If you can share your filters here, it would help other users who are also looking for the same information. Looking forward to hearing from you.

I don't have a need for this anymore in my env so I wasn't able to test, but if I did write one it'd probably be like this:
`#Windows 2016 LTSB
SELECT * from Win32_OperatingSystem WHERE Version > "10.0.00000" AND Version < "10.0.17763" AND ProductType="1"
#Windows 2019 LTSB
SELECT * from Win32_OperatingSystem WHERE Version > "10.0.17762" AND ProductType="1"`

Thank you @pyoungberg for sharing that information. I am sure other users who'll stumble upon this thread—which is available on the feedback section of the doc content page—will find it useful.
We will now close this issue and thank you again for your feedback and for being part of the Microsoft Docs community!

@officedocsbot close


Message Lock Expired

Description
I am writing a python consumer to consume messages from service bus queue. I am running it in a while loop such that it consumes some messages and then process it and then repeatedly do the same. It works fine locally however, when I try to run it in Azure VM it throws the following error:
I tried to debug the issue and it is able to connect to the service bus and only fails when the message is marked complete. There is no lag between processing messages, all happens pretty much in seconds.
Actual Behavior
consumer    | Traceback (most recent call last):
consumer    |   File "consume_messages.py", line 14, in <module>
consumer    |     message.complete()
consumer    |   File "/consumer/.local/lib/python3.6/site-packages/azure/servicebus/common/message.py", line 316, in complete
consumer    |     self._is_live('complete')
consumer    |   File "/consumer/.local/lib/python3.6/site-packages/azure/servicebus/common/message.py", line 95, in _is_live
consumer    |     raise MessageLockExpired(inner_exception=self.auto_renew_error)
consumer    | azure.servicebus.common.errors.MessageLockExpired: Message lock expired
consumer    | Traceback (most recent call last):
consumer    |   File "consume_messages.py", line 14, in <module>
consumer    |     message.complete()
consumer    |   File "/consumer/.local/lib/python3.6/site-packages/azure/servicebus/common/message.py", line 316, in complete
consumer    |     self._is_live('complete')
consumer    |   File "/consumer/.local/lib/python3.6/site-packages/azure/servicebus/common/message.py", line 95, in _is_live
consumer    |     raise MessageLockExpired(inner_exception=self.auto_renew_error)
consumer    | azure.servicebus.common.errors.MessageLockExpired: Message lock expired

Expected Behavior
The consumer should start consuming the messages and mark it complete in the VM

@SeanFeldman thanks for forwarding and @Jainit25 thanks for reporting. //cc: @KieranBrantnerMagee and @mayurid could you take a look at this

Thanks everyone for jumping into the conversation. I further debugged and found out that the following logic is failing in message.py being the root cause of the issue.
if self.locked_until and self.locked_until <= datetime.datetime.now():
    return True

On the VM I printed the locked_until time and the current time and found out that the locked_until time is less than the time when the message was read. For example, if I am reading/marking the message as complete at 10:00, the locked_until time is 9:45 which makes no sense. So I compared the VM time which is 15 mins ahead of actual UTC time.
On my local machine this error is not occurring as my machine is accurate with the EST time. The reason why it was passing locally and failing in the VM.
I want to ask for recommendations on fixing this issue now as we now know the root cause.

Time from the server is always in UTC. The servers on Azure might have a slight clock skew, but it should not be as big as 30 seconds. If it's a custom VM, perhaps time should be synchronized.
I'm no Python expert, but datetime.datetime.now(): looks suspicious. Unless it's a UTC time. Perhaps it should be datetime.now(timezone.utc)?

datetime.datetime.now(): is the logic from Service Bus python package when we import packages using from azure.servicebus import Message
I believe the datetime.now() picks up the system time. Since, the clock is skewed it is getting the time 15 mins prior from the current time. So the logic of service bus message.complete() is failing.

I believe the datetime.now() picks up the system time.

In that case it could be the issue. The time should be always UTC and if the SDK is testing it against a non-UTC time, based on where the location/time zone, the condition will fail even the lock wasn't technically lost. I'm also starting to question this guarding statement. Rather send the message to the broker and if the time has elapsed, the broker will reject the completion, resulting in lock lost exception anyways.

When we run the command timedatectl, we saw a mismatch between the RTC and Universal time.
This is to update that the issue has been fixed by matching the RTC time to the Universal time when we update the chronyd.conf file to add the following command.
refclock PHC /dev/ptp0 poll 3 dpoll -2 offset 0
After updating the configuration we need to run the following command in the terminal to restart chronyd
systemctl restart chronyd
For more documentation please refer: https://docs.microsoft.com/en-us/azure/virtual-machines/linux/time-sync

Hey folks,
Glad you were able to address the clock skew as a workaround, Jainit.
Thanks as well for pointing this issue our way Sean, and for the assistance.  I'd agree that the non-UTC lock comparison is suspicious, but when obtaining the time from the message annotation we also extract the timestamp into local time, so the comparison is consistent, if surprising.  We've taken steps to address this for the upcoming GA version of this library (Currently in preview, 7.0.0b2), times are now all UTC.
Regarding the guard, since your suggestion seemed on-point, I spoke with some of the folks with prior knowledge of the SDK and discovered that its (the guard) existence was in response to the service not giving useful feedback if the message to be settled was already expired; see here.  Now, this was a while back, and begs the question how the other languages have addressed this in the protocol layer, so I'll be resurrecting that discussion to try and get a better answer and potentially improve this scenario.
Closing this issue now as the immediate problem appears to be resolved; as always don't hesitate to reopen and/or reach out if any further questions or discussion would be helpful.


Update Kotlin to 1.2.50

Because why not? It was released 12 hours ago and we still haven't updated!

Codecov Report

Merging #245 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff            @@
##             master    #245   +/-   ##
========================================
  Coverage      78.1%   78.1%
  Complexity      467     467
========================================
  Files            59      59
  Lines          1338    1338
  Branches        219     219
========================================
  Hits           1045    1045
  Misses          201     201
  Partials         92      92

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6ba3aaf...ee04a84. Read the comment docs.


Seems like the labels on the LB's are reversed

Looking at the diagram on this page, it seems like the lb labels are reversed.
The "front" LB is labeled ILB, while the LBs in the tiers behind it are labeled "LB".
My understanding is that in Azure, we typically call la LB with a public IP a "Load Balancer", while a LB with a private IP is called an "Internal Load Balancer" or "ILB".   See:  https://docs.microsoft.com/en-us/azure/load-balancer/components#frontend-ip-configurations
Hence I believe the labels on this chart are reversed.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 2442849c-9e87-56b2-dffd-5c8089e8132c
Version Independent ID: 828da465-186b-f819-9575-ee51e8c83306
Content: Multi-tier web application built for HA/DR - Azure Example Scenarios
Content Source: docs/example-scenario/infrastructure/multi-tier-app-disaster-recovery.md
Service: architecture-center
Sub-service: example-scenario
GitHub Login: @sujayt
Microsoft Alias: pnp

Thanks for reporting - this issue is under review.  This is a Microsoft Internal DevOps Bug ID AB#332004

@larryclaman - our apologies for the long delay on this, and thanks for this contribution.
The diagram has been updated and will republish tomorrow morning, Pacific time.
#please-close


Problems with IDF 2.1.1?

This won't be a very useful post I'm sorry.  I found many problems began appearing after updating to IDF 2.1.1.  Has anyone else tried it?

It has support for idf 3.0 and higher


[BUG] Filter disabled: goal and guages test is not stable

What is the bug?
I'm running this test and target 1.2 OpenSearchDashboard, and failed with "Expected to find content: '/^7,544$/' within the element"
The following test failed which looking for the value 7,544, but our page is 7565, and this bug comment showing below, can we make this test stable, what kind of situation about "sometimes", and how we fixes the "sometimes" situation. Thanks
  it('Filter disabled: goal and guages', () => {
    // Goal label should be 7,544, and the gauge label should be 39.958%%
    // Inconsistency: original code says that the goal label should have "7,544",
    // but sometimes the goal displays "7,565". It may have been related to a
    // data loading issue.
    commonUI.checkValuesExistInComponent('svg > g > g > text.chart-label', [
      '7,544',
      '39.958%',
    ]);
  });

dashboard filtering
Adding and removing filters from a dashboard
disabling a filter unfilters the data on
Filter disabled: goal and guages:
AssertionError: Timed out retrying: Expected to find content: '/^7,544$/' within the element: [ <text.chart-label>, 1 more... ] but never did.

How can one reproduce the bug?
Steps to reproduce the behavior:
npx cypress run --spec "cypress/integration/core-opensearch-dashboards/vanilla-opensearch-dashboards/*.js" --env "openSearchUrl={our 1.2 endpoint}" --config "baseUrl={our1.2 endpoint}"
What is the expected behavior?
Filter disabled: goal and guages should pass
What is your host/environment?

OS: IOS
Version 12.2

Do you have any screenshots?
Yes

Do you have any additional context?
N/A

We can skip it for now https://www.tabnine.com/code/javascript/functions/jest/It/skip


Spell check: underlines instead of highlighting?

I started using spell check in vim recently and they seem to underline instead of highlight. Is that set in  your vimrc? And how can I change it to the default / highlighting?
Also I it seems to miss some basic typos (e.g. ompiling for compiling). What might be the source of this?

For me it's not even underlining. Can someone comment on this. I can provide my my_config.vim file if that is necessary. @amix

Please see the SpellBad/SpellCap configuration:
E.g.,
SpellBad       xxx ctermbg=9 gui=undercurl guisp=Red


Issue while compiling

Warning:Dependency com.github.ZieIony:MaterialRecents:d2aec46f48 on project app resolves to an APK archive which is not supported as a compilation dependency. File: C:\Users~.gradle\caches\modules-2\files-2.1\com.github.ZieIony\MaterialRecents\d2aec46f48\494f96d345471f38b6f3f4c1c42347a2b42f8303\MaterialRecents-d2aec46f48.apk

It's an app, not a library. If you wish, you can copy RecentsList.java to your project and use it. If you prefer a library, this widget is a part of Carbon (https://github.com/ZieIony/Carbon)


How to change IP address of the Nuki Bridge in the configuration?

Hi Konstantin,
my internet provider sent me a new router.
How can I change the IP address of the Nuki Bridge in the Nuki NG integration?
Kind regards,
Stefan

you should better set a fixed IP for bridge in the router's DHCP settings, otherwise it can change anytime again

On the same boat...

The integration should in my view automatically rediscover the bridge and reconfigure the IP without further user interaction. If the auto discovery method of HA are not appropriate for Nuki, an option flow could be used to trigger a new lookup.


Initial music library support

I plan on doing more with this, but for now the functionality is there.

the queue option functions well but could use some styling.. i only know dirty html :)


refGene.txt hq19  - file turncated

joni@mpi1:~/ASIAT/DNA$ bedToBam -i /home/joni/ASIAT/DNA/56001801066357A.snp.vcf -g /mpi3/S5/annovar/humandb/refGene.txt > kani1snp.bam
joni@mpi1:~/ASIAT/DNA$ samtools view kani1snp.bam
[main_samview] truncated file.
joni@mpi1:~/ASIAT/DNA$
It work's hq18 database but not hq19 downloaded
http://hgdownload.cse.ucsc.edu/goldenpath/hg19/database/refGene.txt.gz
as I understand it's .bed format ?

wget  http://hgdownload.cse.ucsc.edu/goldenPath/hg19/database/knownGene.txt.gz
joni@mpi1:~/ASIAT/DNA$ bedtools sort -i /mpi3/S5/annovar/humandb/knownGene.txt > /mpi3/S5/annovar/humandb/knownGene.sorted.txt
Error: malformed GFF entry at line 79482. Start was greater than end. Exiting.
joni@mpi1:~/ASIAT/DNA$

Ubuntu 18.04
Version:   v2.26.0

Version:   v2.29.2-25-g3dd63c46
joni@mpi1:~/ASIAT/DNA$ /mpi3/S5/bedtools2/bin/bedtools sort -i /mpi3/S5/annovar/humandb/knownGene.txt > /mpi3/S5/annovar/humandb/knownGene.sorted.txt
Error: malformed GFF entry at line 79482. Coordinate detected that is < 1. Exiting.

I am confused about what your problem is. What file?

refGene.txt is not in BED format. But you can convert it to BED: https://github.com/ken0-1n/refgenetxttobed


Fix Create Checkout Call

Description of PR that completes issue here...
Changes

Moved the post params and pass the [] for query param to make it workable

Checklist

[x] Docs are complying/updated
[x] Tests are passing

Please revert commits 36ca593 and 4d7183f.

It will fix #14


How to create a private key

Any help is appreciated. I can't find a way to generate a private key from a SYNC wallet. All I can get is the keystore.
I'm trying to use connex repl. which requires the private key to import a wallet.
How do I get this private key?

Try Keystore.decrypt(keystore, 'your password') https://github.com/vechain/thor-devkit.js/blob/master/src/keystore.ts#L35

Thanks, this solves the issue


Inconsistency in requirements tables XML notation

We're not consistent with our use of XML notation in the requirements tables, for example CSIP3 uses mets/@TYPE while CSIP4 denotes @CONTENTTYPESPECIFICATION.

Handled


Selected Date display format

Please describe the feature you would like to request.
I would like a way to control the format of the date that is displayed on the date picker after a date has been selected
What is the use-case or motivation for this proposal?
We want to standardize the way dates are displayed in our current system but the datepicker does not allow us to change the display of the selected date (which does not match our system standard)
Is there anything else I should know?
No

https://github.com/dalelotts/angular-bootstrap-datetimepicker#display-of-year-month-day-hour-ampm-and-minute-formats

I suppose that I should have given more context here, I apologize. Our system is using the date time picker as both a date picker and a date-time picker. We want to standardize the dates but we also want to disable the time display components where the min view is day (no time selection required).
Unfortunately we have date-time pickers and date pickers on the same page, so changing locale settings between pages is not all that viable.

It's difficult to visualize what you are trying to accomplish.

I created a basic Stackblitz example to demonstrate this. I included descriptions by the fields to try explain what I was saying earlier
https://stackblitz.com/edit/angular-ivy-s1otuu

Thank you! I understand now.

I'm swamped with work at the moment, so I don't have time to fork and update your example.
However, I'm 99% sure there are no code changes needed for this.  Move the code into a separate component and use the element level dependency injector to supply the format you want.

Thank you for the reply 😄
I did not know that you can override dependency tokens tokens in a non-global way.
I will fork my own StackBlitz and make the changes and post it here so that anyone else who finds the issue has a solution

I created a fork from my StackBlitz which I implemented the fix as suggested.
Thanks again
https://stackblitz.com/edit/angular-ivy-2lghv4

I'm glad it worked!
If you have time and interest, a PR to the demo project to add an example showing how to implement and test this would be great.


fix #198 Engagements is now nicely aligned

Current coverage is 45.74%

Merging #203 into master will not affect coverage as of 93b608b

@@            master    #203   diff @@
======================================
  Files           21      21
  Stmts          341     341
  Branches         0       0
  Methods          0       0
======================================
  Hit            156     156
  Partial          0       0
  Missed         185     185

Review entire Coverage Diff as of 93b608b

Powered by Codecov. Updated on successful CI builds.


Some browser examples overlap slides, find a way to fix this automatically

e.g.
https://angular-presentation.firebaseapp.com/dependency-injection/exercise-pre

related to #17

Closing this, guess this should be a problem of whoever creates the content.


error execution phase preflight: couldn't validate the identity of the API Server: Get "https://<IP_ADDRESS>:8443..."

Details
What steps did you take and what happened:
Trying out the updated Ansible installation playbook today on a freshly imaged 20.04 four node pi4B cluster. Hit this failure mode when running env ANSIBLE_CONFIG=ansible/ansible.cfg ansible-playbook ansible/playbooks/all.yml:
TASK [cluster : join | add node to cluster] ******************************************************************************************************************************************
Sunday 14 February 2021  10:42:28 -0800 (0:00:00.793)       0:09:52.749 *******
fatal: [blackberry]: FAILED! => changed=true
  cmd:
  - kubeadm
  - join
  - --config
  - /etc/kubernetes/kubeadm-join.yaml
  delta: '0:05:08.846494'
  end: '2021-02-14 18:47:38.301132'
  msg: non-zero return code
  rc: 1
  start: '2021-02-14 18:42:29.454638'
  stderr: |2-
            [WARNING SystemVerification]: missing optional cgroups: hugetlb
    error execution phase preflight: couldn't validate the identity of the API Server: Get "https://<IP_ADDRESS>:8443/api/v1/namespaces/kube-public/configmaps/cluster-info?timeout=10s": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
    To see the stack trace of this error execute with --v=5 or higher

What did you expect to happen:
All playbooks run to completion with a 4 node k8s cluster started up successfully.
Anything else you would like to add:
I searched for <IP_ADDRESS> in the local clone. Found it inside the the old variable section of ansible/group_vars/cluster.yml...
## Old variables
####
# Role - keepalived
####
keepalived_vip: <IP_ADDRESS>

Note: Miscellaneous information that will assist in solving the issue.

Additional Information:

Note: Anything to give further context to the bug report.

Update:
I figured I'd set the keepalived_vip value in the old variables section of cluster.yml to the same value I'm using in master.yml and try again. First I ran the nuke playbook on the existing deploy. Running all.yml failed again, this time because it found config inside /etc/kubernetes/manifests/.  Maybe a problem with nuke here.

Update:
I reimaged all the pi's back to back ubuntu per the first part of your install doc. I ran all.yaml install again. This time I have keepalived_vip value in the old variables section of cluster.yml to the same value I'm using in master.yml. The install moved past the keepalived_vip problem caused the old variables setting. Hit another failure mode:
TASK [cluster : join | add node to cluster] ******************************************************************************************************************************************
Sunday 14 February 2021  12:17:44 -0800 (0:00:00.919)       0:09:29.725 *******
changed: [blackberry]
fatal: [strawberry]: FAILED! => changed=true
  cmd:
  - kubeadm
  - join
  - --config
  - /etc/kubernetes/kubeadm-join.yaml
  delta: '0:04:12.506743'
  end: '2021-02-14 20:21:57.663006'
  msg: non-zero return code
  rc: 1
  start: '2021-02-14 20:17:45.156263'
  stderr: |2-
            [WARNING SystemVerification]: missing optional cgroups: hugetlb
    error execution phase kubelet-start: error uploading crisocket: timed out waiting for the condition
    To see the stack trace of this error execute with --v=5 or higher
  stderr_lines: <omitted>
  stdout: |-
    [preflight] Running pre-flight checks
    [preflight] Reading configuration from the cluster...
    [preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
    [preflight] Running pre-flight checks before initializing the new control plane instance
    [preflight] Pulling images required for setting up a Kubernetes cluster
    [preflight] This might take a minute or two, depending on the speed of your internet connection
    [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
    [download-certs] Downloading the certificates in Secret "kubeadm-certs" in the "kube-system" Namespace
    [certs] Using certificateDir folder "/etc/kubernetes/pki"
    [certs] Generating "etcd/peer" certificate and key
    [certs] etcd/peer serving cert is signed for DNS names [localhost strawberry] and IPs [<IP_ADDRESS> <IP_ADDRESS> ::1]
    [certs] Generating "apiserver-etcd-client" certificate and key
    [certs] Generating "etcd/server" certificate and key
    [certs] etcd/server serving cert is signed for DNS names [localhost strawberry] and IPs [<IP_ADDRESS> <IP_ADDRESS> ::1 <IP_ADDRESS>]
    [certs] Generating "etcd/healthcheck-client" certificate and key
    [certs] Generating "front-proxy-client" certificate and key
    [certs] Generating "apiserver-kubelet-client" certificate and key
    [certs] Generating "apiserver" certificate and key
    [certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local strawberry] and IPs [<IP_ADDRESS> <IP_ADDRESS> <IP_ADDRESS>]
    [certs] Valid certificates and keys now exist in "/etc/kubernetes/pki"
    [certs] Using the existing "sa" key
    [kubeconfig] Generating kubeconfig files
    [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
    [endpoint] WARNING: port specified in controlPlaneEndpoint overrides bindPort in the controlplane address
    [kubeconfig] Writing "admin.conf" kubeconfig file
    [endpoint] WARNING: port specified in controlPlaneEndpoint overrides bindPort in the controlplane address
    [kubeconfig] Writing "controller-manager.conf" kubeconfig file
    [endpoint] WARNING: port specified in controlPlaneEndpoint overrides bindPort in the controlplane address
    [kubeconfig] Writing "scheduler.conf" kubeconfig file
    [control-plane] Using manifest folder "/etc/kubernetes/manifests"
    [control-plane] Creating static Pod manifest for "kube-apiserver"
    [control-plane] Creating static Pod manifest for "kube-controller-manager"
    [control-plane] Creating static Pod manifest for "kube-scheduler"
    [check-etcd] Checking that the etcd cluster is healthy
    [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
    [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
    [kubelet-start] Starting the kubelet
    [kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...
    [kubelet-check] Initial timeout of 40s passed.
  stdout_lines: <omitted>

After about 10 minutes of no output on stdout the play continued, completing a group cni and storage tasks, then completing the plays:
TASK [cni : preflight checks] ********************************************************************************************************************************************************
Sunday 14 February 2021  12:28:29 -0800 (0:10:45.179)       0:20:14.905 *******
included: /Users/chris/GH/raspbernetes/k8s-cluster-installation/ansible/roles/cni/tasks/pre_checks.yml for hackberry

TASK [cni : validate variable : cni_plugin] ******************************************************************************************************************************************
Sunday 14 February 2021  12:28:29 -0800 (0:00:00.099)       0:20:15.004 *******
ok: [hackberry] => changed=false
  msg: All assertions passed

TASK [cni : validate variable : cni_main_master] *************************************************************************************************************************************
Sunday 14 February 2021  12:28:30 -0800 (0:00:00.119)       0:20:15.124 *******
ok: [hackberry] => changed=false
  msg: All assertions passed

TASK [cni : validate plugin support] *************************************************************************************************************************************************
Sunday 14 February 2021  12:28:30 -0800 (0:00:00.115)       0:20:15.239 *******
ok: [hackberry] => changed=false
  msg: All assertions passed

TASK [cni : validate pod subnet is used when using flannel] **************************************************************************************************************************
Sunday 14 February 2021  12:28:30 -0800 (0:00:00.161)       0:20:15.401 *******
skipping: [hackberry]

TASK [cni : validate pod subnet is used when using cilium] ***************************************************************************************************************************
Sunday 14 February 2021  12:28:30 -0800 (0:00:00.117)       0:20:15.518 *******
skipping: [hackberry]

TASK [cni : setup calico container network interface (cni)] **************************************************************************************************************************
Sunday 14 February 2021  12:28:30 -0800 (0:00:00.117)       0:20:15.635 *******
skipping: [blueberry]
skipping: [blackberry]
included: /Users/chris/GH/raspbernetes/k8s-cluster-installation/ansible/roles/cni/tasks/calico.yml for hackberry

TASK [cni : applying calico] *********************************************************************************************************************************************************
Sunday 14 February 2021  12:28:30 -0800 (0:00:00.202)       0:20:15.838 *******
changed: [hackberry]

TASK [storage : preflight checks] ****************************************************************************************************************************************************
Sunday 14 February 2021  12:28:41 -0800 (0:00:10.349)       0:20:26.188 *******
included: /Users/chris/GH/raspbernetes/k8s-cluster-installation/ansible/roles/storage/tasks/pre_checks.yml for hackberry

TASK [storage : check os_family support] *********************************************************************************************************************************************
Sunday 14 February 2021  12:28:41 -0800 (0:00:00.093)       0:20:26.281 *******
ok: [hackberry] => changed=false
  msg: All assertions passed

TASK [storage : validate variable : openebs_enabled] *********************************************************************************************************************************
Sunday 14 February 2021  12:28:41 -0800 (0:00:00.146)       0:20:26.428 *******
ok: [hackberry] => changed=false
  msg: All assertions passed

TASK [storage : include family specific tasks] ***************************************************************************************************************************************
Sunday 14 February 2021  12:28:41 -0800 (0:00:00.104)       0:20:26.532 *******
skipping: [hackberry]
skipping: [blueberry]
skipping: [blackberry]

PLAY RECAP ***************************************************************************************************************************************************************************
<IP_ADDRESS>                  : ok=1    changed=0    unreachable=0    failed=0    skipped=1    rescued=0    ignored=0
blackberry                 : ok=67   changed=38   unreachable=0    failed=0    skipped=14   rescued=0    ignored=0
blueberry                  : ok=85   changed=45   unreachable=0    failed=0    skipped=18   rescued=0    ignored=0
hackberry                  : ok=130  changed=51   unreachable=0    failed=0    skipped=22   rescued=0    ignored=0
strawberry                 : ok=86   changed=44   unreachable=0    failed=1    skipped=18   rescued=0    ignored=0

Sunday 14 February 2021  12:28:41 -0800 (0:00:00.180)       0:20:26.712 *******
===============================================================================
cluster : join | add node to cluster ---------------------------------------------------------------------------------------------------------------------------------------- 645.18s
cluster : initialize | execute kubeadm init on first control plane node ----------------------------------------------------------------------------------------------------- 141.82s
common : reboot hosts -------------------------------------------------------------------------------------------------------------------------------------------------------- 63.43s
kubernetes : install kubernetes packages (3/4) ------------------------------------------------------------------------------------------------------------------------------- 41.31s
keepalived : debian : install keepalived ------------------------------------------------------------------------------------------------------------------------------------- 36.21s
haproxy : debian : install haproxy ------------------------------------------------------------------------------------------------------------------------------------------- 28.68s
cri : containerd | install package from apt repository ----------------------------------------------------------------------------------------------------------------------- 21.78s
common : install common Kubernetes ansible module ---------------------------------------------------------------------------------------------------------------------------- 20.33s
common : install common packages --------------------------------------------------------------------------------------------------------------------------------------------- 20.03s
kubernetes : install helm package (3/3) -------------------------------------------------------------------------------------------------------------------------------------- 19.10s
kubernetes : adding apt repository for kubernetes (2/4) ---------------------------------------------------------------------------------------------------------------------- 17.17s
cri : containerd | ensure docker.io apt repository is enabled ---------------------------------------------------------------------------------------------------------------- 16.62s
kubernetes : adding apt repository for helm (2/3) ---------------------------------------------------------------------------------------------------------------------------- 16.31s
cni : applying calico -------------------------------------------------------------------------------------------------------------------------------------------------------- 10.35s
Gathering Facts --------------------------------------------------------------------------------------------------------------------------------------------------------------- 9.31s
Gathering Facts --------------------------------------------------------------------------------------------------------------------------------------------------------------- 8.38s
Gathering Facts --------------------------------------------------------------------------------------------------------------------------------------------------------------- 6.81s
cri : crictl | unarchive crictl binary ---------------------------------------------------------------------------------------------------------------------------------------- 3.98s
common : apt-get upgrade ------------------------------------------------------------------------------------------------------------------------------------------------------ 3.55s
kubernetes : add apt signing key for kubernetes (1/4) ------------------------------------------------------------------------------------------------------------------------- 3.49s

Here is cluster node status at this point:
> kubectl get nodes -o wide --kubeconfig ./k8s-config.yaml
NAME         STATUS     ROLES                  AGE   VERSION   INTERNAL-IP    EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION     CONTAINER-RUNTIME
blackberry   NotReady   <none>                 11m   v1.20.2   <IP_ADDRESS>   <none>        Ubuntu 20.04.2 LTS   5.4.0-1028-raspi   containerd://1.4.3
blueberry    NotReady   control-plane,master   10m   v1.20.2   <IP_ADDRESS>   <none>        Ubuntu 20.04.2 LTS   5.4.0-1028-raspi   containerd://1.4.3
hackberry    NotReady   control-plane,master   12m   v1.20.2   <IP_ADDRESS>   <none>        Ubuntu 20.04.2 LTS   5.4.0-1028-raspi   containerd://1.4.3
strawberry   NotReady   <none>                 72s   v1.20.2   <IP_ADDRESS>   <none>        Ubuntu 20.04.2 LTS   5.4.0-1028-raspi   containerd://1.4.3

I decide to run the all.yml play again. This time with no fatal errors. However, my node status shows only two masters (not three as I spec'd in the inventory):
> kubectl get nodes -o wide --kubeconfig ansible/playbooks/output/k8s-config.yaml
NAME         STATUS   ROLES                  AGE   VERSION   INTERNAL-IP    EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION     CONTAINER-RUNTIME
blackberry   Ready    <none>                 28m   v1.20.2   <IP_ADDRESS>   <none>        Ubuntu 20.04.2 LTS   5.4.0-1028-raspi   containerd://1.4.3
blueberry    Ready    control-plane,master   26m   v1.20.2   <IP_ADDRESS>   <none>        Ubuntu 20.04.2 LTS   5.4.0-1028-raspi   containerd://1.4.3
hackberry    Ready    control-plane,master   28m   v1.20.2   <IP_ADDRESS>   <none>        Ubuntu 20.04.2 LTS   5.4.0-1028-raspi   containerd://1.4.3
strawberry   Ready    <none>                 17m   v1.20.2   <IP_ADDRESS>   <none>        Ubuntu 20.04.2 LTS   5.4.0-1028-raspi   containerd://1.4.3

More cluster state details re above. Note that one of my declared masters (strawberry) is not running etcd or kube-apiserver.
hackberry (master):
pi@hackberry:~$ sudo crictl ps
CONTAINER           IMAGE               CREATED             STATE               NAME                      ATTEMPT             POD ID
9a81d4f033456       35af1df8416d5       30 minutes ago      Running             calico-node               0                   63f1cc1b988cd
a9db1cfcdd76c       95d99817fc335       30 minutes ago      Running             kube-apiserver            7                   f420ec7436438
968b95dbae0c9       05b738aa1bc63       34 minutes ago      Running             etcd                      2                   321b103cca468
b2227ea8e2d97       3a1a2b528610a       40 minutes ago      Running             kube-controller-manager   1                   d47890e1ba157
7a7753851d9ab       60d957e44ec8a       40 minutes ago      Running             kube-scheduler            1                   f625fffc45bbf
aeb39c7ac3efd       788e63d07298d       42 minutes ago      Running             kube-proxy                0                   ab9d064b9295e

blueberry (master):
pi@blueberry:~$ sudo crictl ps
CONTAINER           IMAGE               CREATED             STATE               NAME                      ATTEMPT             POD ID
ac54a7b16f731       3a1a2b528610a       30 minutes ago      Running             kube-controller-manager   1                   9b185ac5454fd
5a22a7d8528a3       35af1df8416d5       30 minutes ago      Running             calico-node               0                   58fa61fc45d9a
d0054c698ebf0       05b738aa1bc63       32 minutes ago      Running             etcd                      0                   e48cc0575773d
4099be2608d81       95d99817fc335       32 minutes ago      Running             kube-apiserver            6                   be8677296d508
4ba5a75ffb0a6       60d957e44ec8a       40 minutes ago      Running             kube-scheduler            0                   48dda9bf2a9e5
e0306a62e0e89       788e63d07298d       40 minutes ago      Running             kube-proxy                0                   69a4c4384e835

strawberry (intended master):
pi@strawberry:~$ sudo crictl ps
CONTAINER           IMAGE               CREATED             STATE               NAME                      ATTEMPT             POD ID
6405e63c02be4       35af1df8416d5       30 minutes ago      Running             calico-node               0                   034286a83df90
0e6c98f8a89df       788e63d07298d       31 minutes ago      Running             kube-proxy                0                   0740ce290c2ff
30ba49df68752       60d957e44ec8a       40 minutes ago      Running             kube-scheduler            0                   2dd66e00bd2be
4632943f39249       3a1a2b528610a       40 minutes ago      Running             kube-controller-manager   0                   e32207bafed57

blackberry (worker):
pi@blackberry:~$ sudo crictl ps
CONTAINER           IMAGE               CREATED             STATE               NAME                      ATTEMPT             POD ID
2c6e7b609bacc       acf4bc146ed19       29 minutes ago      Running             calico-kube-controllers   0                   8888ae3c57ffc
3afbd3e2d83b7       db91994f4ee8f       30 minutes ago      Running             coredns                   0                   f5955392080c5
21f385bc19c1d       db91994f4ee8f       30 minutes ago      Running             coredns                   0                   15e8ea135da08
c88c330892f02       35af1df8416d5       30 minutes ago      Running             calico-node               0                   b4dd709f7fd86
4eb84272d3f4d       788e63d07298d       42 minutes ago      Running             kube-proxy                0                   2cae842481891

In case this helps ^^:
> cat ansible/inventory
[all]
hackberry hostname=hackberry.almond.lan ansible_host=<IP_ADDRESS> ansible_user=pi
strawberry hostname=strawberry.almond.lan ansible_host=<IP_ADDRESS> ansible_user=pi
blueberry hostname=blueberry.almond.lan ansible_host=<IP_ADDRESS> ansible_user=pi
blackberry hostname=blackberry.almond.lan ansible_host=<IP_ADDRESS> ansible_user=pi

[cluster:children]
controlplane
nodes

[controlplane]
hackberry
strawberry
blueberry

[nodes]
blackberry

[docker_cache]
#registry hostname=registry ansible_host=<IP_ADDRESS> ansible_user=pi

; These entires are here for backward compatibility as we transition away from the old names.
[k8s:children]
masters
workers

[masters]
hackberry
strawberry
blueberry

[workers]
blackberry

## Old variables
####
# Role - keepalived
####
keepalived_vip: <IP_ADDRESS>

Ok, so this looks like testing leftovers snuck into the "defaults" - keepalived_vip should normally be an empty string, like it is in the master.yml.
The second issue is an odd one - the API server just doesn't seem to respond properly when another control-plane node attempts to join. There's a check that's built into the playbook to ensure the API server is available prior to attempting to join, sometimes it catches the delay, sometimes it doesn't.
If you execute the nuke playbook; ansible-playbook playbooks/nuke.yml, it will reset the nodes to a state prior to creating the cluster. Executing the all playbook from this state should succeed, please let us know if it is successful.


Connecting kviklet to AWS RDS PostgreSQL in Private Subnet

Hi,
We have an AWS RDS PostgreSQL database hosted in a private subnet within one AWS VPC account. We would like to set up a connection to this RDS instance using Kviklet. However, our setup involves the RDS instance and the Kviklet service being in different AWS accounts VPC .
Could you please confirm if this is possible? If yes, could you guide us on how to configure the connection under these circumstances?
Thank you for your assistance.

Hey vjmax,
I'm not an expert on AWS networking but as far as I know two different VPCs can't talk to each other by default. You'll need some sort of network setup to make the routing work e.g. vpc peering:
https://docs.aws.amazon.com/whitepapers/latest/aws-vpc-connectivity-options/amazon-vpc-to-amazon-vpc-connectivity-options.html
The more typical setup would be to have a public subnet in which you host the Kviklet container and a private subnet in which your RDS instance is hosted. This then enables you to make Kviklet reachable (e.g. via a load balancer, vpn or whatever else you'd like to use) without having the database directly exposed through the same means.

In general that's a valid request but there is a bunch of different way of how you could "vpn/tunnel" to your database and supporting a reasonable subset would be quite a lot of effort.
For now I think the safer way would be to hide kviklet and the DB behind a safe tunnel/vpn that acts as a second line of defense. You can still configure subnets in AWS in a way so that only requests from the instance itself can reach the database. e.g. this is similar to a setup I have worked with before.


Added gitGood.

Description
I have added the soucre code for the react app. Also, I have added my name and project links to contributors.json.
Fixes #[47]
Dependencies
react, react-dom, react-router-dom, react-terminal, react-use, react-icons, @primer/octicons-react, @splinetool/react-spline.
Future Improvements
In the main gitGood repo there are two issues open. Currently I'm working on those issues to increase the interactivity and engagement of the tutorial.
Mentions
NA
Screenshots of relevant screens

Developer's checklist

[X] My PR follows the style guidelines of this project
[X] I have performed a self-check on my work

If changes are made in the code:

[X] I have followed the coding guidelines
[X] My changes in code generate no new warnings
[ ] My changes are breaking another fix/feature of the project
[X] I have added test cases to show that my feature works
[X] I have added relevant screenshots in my PR

Great project 🎉
Happy Contribution 😄


Scanning Error after update

I have just pulled the latest release. Since then, I am getting a Scanning Error after the login.
[        search] [  ERROR] Scanning error @ ConnectionError: HTTPSConnectionPool(host='android.clients.google.com', port=443): Max retries exceeded with url: /auth (Caused by NewConnectionError('<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x00000000040D3D68>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed',))

After removing the -st option, I was able to log in.

How's that done?


Allow For Customization of a Dates Event Box

It would be nice to have a prop that allows for the passing in of a component that controls the style of each calendar day box similar to the renderDay prop on the monthly calendar.

Good idea, i will try to make a pr for this in the next couple days!

Awesome!!! Thank you.

So, I want to keep going with the current abstraction. Which means all I did in my PR was split out the monthly day component so that it can be replaced: https://github.com/zackify/react-calendar/pull/4 check it out.
Essentially this would let you make MyMonthlyBody.js in your codebase, and it would look like this:
export function MonthlyDay<DayData>({ renderDay }) {
  let { day, events } = useMonthlyBody<DayData>();
  let dayNumber = format(day, 'd');

  return (
    <div
      key={day.toISOString()}
      aria-label={`Events for day ${dayNumber}`}
      className="h-48 p-2 border-b-2 border-r-2"
    >

{dayNumber}

{format(day, 'EEEE')}

        {renderDay(events)}

  );
}

You can then fully change any of the default code to your desire. The goal of this library is to mostly drop in to your app by making "MyCalendar" and composing everything. And be able to swap out any portion with your own code. Just import the required hooks for the data.

I have released this as 0.1.0!


Windows build fails due to standalone micro runtime

Attempting to build on Windows fails with config.cmake options [set(USE_CUDA ON), set(USE_LLVM ON)]. The build process followed instructions @ https://docs.tvm.ai/install/from_source.html
By default the micro runtime is built, however this fails due to incompatibility with Windows.
For example, see unix-specific library export macro definitions:

"TVM_MICRO_RUNTIME_API_API" - include/tvm/runtime/micro/standalone/utvm_runtime.h
"TVM_MICRO_RUNTIME_API_BACKEND_API" - src/runtime/micro/standalone/utvm_graph_runtime.h

(Note, further incompatibilities exist after resolving these definitions, however I did not debug further)
I was able to successfully build by disabling compilation of the micro runtime [config.cmake option set(USE_MICRO_STANDALONE_RUNTIME ON->OFF)]. While this resolved the problem for me, I am unsure if this runtime is a general requirement.

Given that the micro runtime is only works under unix, perhaps it makes sense to disable ti by default, but explicitly enable it in ci, cc @ajtulloch

https://github.com/dmlc/tvm/pull/4240


adjusting heron client logging in runtime

my understanding is that today heron clients dont allow the log level to be specified at run time and this is prebaked in compile time.
can we consider a PR to allow supplying the log level and configuration of the logger in run time? i will be happy to submit a PR

If you pass --verbose flag on command line, logging level is set to logging.DBEUG, otherwise it is logging.INFO.

or do you mean you want more granularity? like command-line argument --verbose=LEVEL?

My understanding is to change the log level of instance logs in run time. Let set the log level of an instance from INFO to DEBUG to investigate an issue.