failed to run custom build command for cairo-sys-rs v0.3.1

Hi,
I am using Ubuntu 16.04 x64.
I followed the documentation at http://gtk-rs.org/ to build my first rust+gtk program: I added the three needed lines in Cargo.toml. However 'cargo build' gives me an error:
   Compiling gio v0.1.0
   Compiling c_vec v1.0.12
   Compiling gtk v0.1.0
   Compiling gdk-sys v0.3.1
   Compiling atk-sys v0.3.1
   Compiling gdk-pixbuf-sys v0.3.1
   Compiling cairo-rs v0.1.0
   Compiling gdk-pixbuf v0.1.0
   Compiling gdk v0.5.0
   Compiling bitflags v0.5.0
   Compiling cairo-sys-rs v0.3.1
failed to run custom build command for `cairo-sys-rs v0.3.1`
Process didn't exit successfully: `/home/hadrien/rust/dkalc/target/debug/build/cairo-sys-rs-f59614cd267c1996/build-script-build` (exit code: 1)
--- stderr
`"pkg-config" "--libs" "--cflags" "cairo >= 1.10"` did not exit successfully: exit code: 1
--- stdout
Package cairo was not found in the pkg-config search path.
Perhaps you should add the directory containing `cairo.pc'
to the PKG_CONFIG_PATH environment variable
No package 'cairo' found

Is there something missing in the documentation? I printed $PKG_CONFIG_PATH but it is empty.

Do you have libgtk-3-dev installed as suggested here?

Do you have libgtk-3-dev installed as suggested here

Thank you gkoz, it worked. I got confused between the documentation on the home page and the documentation at http://gtk-rs.org/docs/ which is more complete.


Support i686-pc-windows-msvc

This crate cannot be compiled for i686-pc-windows-msvc until https://github.com/rust-lang-nursery/rust-bindgen/issues/541 is resolved.

rust-lang-nursery/rust-bindgen#541 is fixed on nightly, so maybe this can be fixed now? It would require Nightly Rust for a while, then Nightly or Beta Rust, but that's better than the current state.

The problem was the name mangling of the c function, it doesn't use the thiscall ABI, so it will work on stable too.
I just wait for the next release of bindgen :)


Error in IE11

When I load the comments (latest remark master branch ) in IE11 I get an error in the console:
Object doesn't support this action
var i=new Promise(function(t,n){o=r[e]=[t,n]});
File: remark.js, Line: 1, Column: 629

The comments just show the loader.
I don't have this problem in any other browser I've tested.
Does remark still support IE11?
I tried using the pollyfill.io default pollyfill which contains the promises pollyfill, but the error still occurs.
EDIT: If I load remark in IE11 on the demo page at https://remark42.com/demo/ the comments also fail to load and the console show an error:
Unhandled promise rejection Error: Loading chunk 5 failed.
(error: https://remark42.com/demo/vendors~polyfills.js)
"Unhandled promise rejection"
[object Error]{description: "Loading chu...", message: "Loading chu...", name: "Error", request: "https://rem...", stack: "Error: Load...", type: "error"}

You may already be aware of that error, but I thought I'd mention it as it's also a promise error like the error I have in my code.

I'm now loading the remark42 backend from a subdomain  for my live server, but now I'm getting a different error in IE11:
  | This content cannot be displayed in a frame
  | To help protect the security of information you enter into this website, the publisher of this content does not allow it to be displayed in a frame.
  | What you can try:
  | Open this content in a new window |   | Open this content in a new window
  | Open this content in a new window

Any plans on getting remark working with IE11, or is that browser not supported anymore?

I'm not sure. From one hand it is about 5% of the market, from the over - even MS is saying you shouldn't use IE 11 unless you have a very specific need. Unless the fix is really trivial I'd say nay. @Reeywhaar - what say you?

I'm going to check everything on old ios versions (9.3), as for IE, well I'd be glad to support, I'll see what I can do

@leem, I have tested yesterday and #313 was merged. Can you check please, if there any issues in ie11? I suspect there can be some related to proxying, because ie doesn't work with dev setup, dev server proxied request constantly gets ECONNRESET.

Sorry for the delay, I was away from computer. I've tested remark in ie11, but are getting the same error.
Object doesn't support this action
var i=new Promise(function(t,n){o=r[e]=[t,n]});
File: remark.js, Line: 1, Column: 629

To make sure I was using updated remark  I completely removed the remark container and image from Docker and reinstalled the Git repo then ran docker-compose pull && docker-compose up -d.

Just noticed however the remark demo page is now working with ie11. Maybe I've installed the remark update incorrectly?

@Reeywhaar, I'm still getting the Object doesn't support this action error.
Here's how the backend looks.
Any ideas?
server {

    listen 443 ssl http2;

    server_name comments.{{ domain_name }};
    ssl_certificate      /etc/letsencrypt/live/{{ domain_name }}/fullchain.pem;
    ssl_certificate_key  /etc/letsencrypt/live/{{ domain_name }}/privkey.pem;
    ssl_session_cache shared:SSL:40m;
    ssl_session_timeout 21h;
    ssl_session_tickets off;
    ssl_buffer_size 4k;

    # Enable server-side protection against BEAST attacks
    ssl_protocols TLSv1.2;
    ssl_prefer_server_ciphers on;
    ssl_ciphers "ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384";
ssl_dhparam /etc/ssl/certs/dh4096.pem;
    ssl_ecdh_curve secp521r1:secp384r1;

    # Enable OCSP stapling
    # ref. http://blog.mozilla.org/security/2013/07/29/ocsp-stapling-in-firefox
    ssl_stapling on;
    ssl_stapling_verify on;
    ssl_trusted_certificate /etc/letsencrypt/live/{{ domain_name }}/fullchain.pem;

    resolver <IP_ADDRESS> <IP_ADDRESS> [2606:4700:4700::1111] [2606:4700:4700::1001] valid=300s; # Cloudflare
    resolver_timeout 5s;

       # ie11 support headers https://github.com/umputun/remark/pull/313#issue-272179992
      add_header Expires "Sun, 01 Jan 2019 00:00:00 GMT";
      add_header Cache-Control "no-store, no-cache, must-revalidate";
      add_header Pragma "no-cache";

      # ref: https://developer.mozilla.org/en-US/docs/Security/HTTP_Strict_Transport_Security
      # add_header Strict-Transport-Security "max-age=31536000; includeSubDomains; preload always;";
      add_header Strict-Transport-Security "max-age=0; includeSubDomains";

      # See https://securityheaders.io/ for referer-options:
      add_header Referrer-Policy no-referrer-when-downgrade;

      # X-Frame-Options is to prevent from clickJacking attack
      add_header X-Frame-Options SAMEORIGIN;

       #  disable content-type sniffing on some browsers.
      add_header X-Content-Type-Options nosniff;

      # This header enables the Cross-site scripting (XSS) filter
      add_header X-XSS-Protection "1; mode=block";

location / {
         add_header Expires "Sun, 01 Jan 2019 00:00:00 GMT";
         add_header Cache-Control "no-store, no-cache, must-revalidate";
         add_header Pragma "no-cache";

         proxy_redirect          off;
         proxy_set_header        X-Real-IP $remote_addr;
         proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
         proxy_set_header        Host $http_host;
         # proxy_pass            http://remark42:8080/;
         proxy_pass              http://<IP_ADDRESS>:8080/;
     }
  }

I've fixed this now. The comments are working in ie11. Thanks :)
In case your wondering, the problem was caused by me having the image in docker-compose.yml set to latest when it needs to be set to master.
change image: umputun/remark42:latest to image: umputun/remark42:master.

cool, closing.


Add remote uri source option

Closing this because Lottie Android intentionally have networking built in because every app has a unique networking stack and I don't want to add any transitive deps to lottie.


Consider renaming body to payload?

By request from #redux on reactiflux...
Admittedly this is pedantic, would you consider renaming body to payload? I thought the term payload was used in Facebook's original Flux guides, but I didn't see it. So perhaps other implementations use payload? I know that you had commented that Flummox uses body. It probably doesn't matter much, but body makes me think that I'm dealing with a web response.

As pointed out by raydeo on #redux, payload is found all over this page: https://facebook.github.io/flux/docs/dispatcher.html

payload on that page refers to the action itself. But I'm cool with renaming with payload. It's clearer that way.


TEST Update docs

This PR will test lgtm auto being revoked and lgtm merge method

/lgtm

/remove lgtm

/lgtm

/lgtm


Better error messaging when pop-ups get blocked

We need to generate more helpful error messages when pop-up windows are blocked by the browser. Some Funcunit tests are failing with an obscure error (Uncaught TypeError: Object [object HTMLDocument] has no method 'getAttributeNode', http://jmvc.local/funcunit/syn/synthetic.js:332) when an attempt is made to focus an invisible document, because it was a blocked pop-up.

I'm confused why this error isn't being thrown in open.js:
FuncUnit.win = window.open(url, "funcunit",  "height=1000,toolbar=yes,status=yes,width="+width/2+",left="+width/2);
if(!FuncUnit.win){
throw "Could not open a popup window.  Your popup blocker is probably on.  Please turn it off and try again";
}

http://stackoverflow.com/questions/668286/detect-blocked-popup-in-chrome


Fix clippy, ignore font load test

I've removed unused imports, prefixed an unused variable with an underscore and didn't do anything to an unused pub function. Also fixed
warning: panic message is not a string literal
 --> src/error_handling.rs:5:12
  |
5 |     panic!(explanation.to_string());
  |            ^^^^^^^^^^^^^^^^^^^^^^^
  |
  = note: `#[warn(non_fmt_panic)]` on by default
  = note: this is no longer accepted in Rust 2021
help: add a "{}" format string to Display the message
  |
5 |     panic!("{}", explanation.to_string());

After this, there's only the one warning about the unused function ( build_neovide_command in src/bridge/mod.rs )

I ignored the test_load test in src/renderer/fonts/font_loader.rs because it fails when I don't have DejaVu Serif on linux. Also, I had to prepend an underscore to build_neovide_command in src/bridge/mod.rs to make clippy pass.
All the pre-commit hooks work now, for me

I don't know if testing for DejaVu is a good assumption, only 20 packages listed on https://archlinux.org/packages/extra/any/ttf-dejavu/ require it explicitly, out of which 9 are testing dependencies and only 11 need it at runtime

I'm learning rust, so apologies if it is overreaching :sweat_smile:

It would probably be a good idea to find a better default don't to test against rather than remove the test altogether.

I don't know if one exists

I reviewed the PR and generally think it looks good. I added a couple of thoughts. Thanks so much for the contribution!

@Kethku can you take a look?

I've not yet understood when clippy marks things as warnings/errors

Clean now!


Adding SUSE 15 support

Description
This PR adds support for SUSE Leap 15.2+ as a KVM hypervisor and Management / Usage server on Cloudstack
Related docs link https://github.com/apache/cloudstack-documentation/pull/224
Types of changes

[ ] Breaking change (fix or feature that would cause existing functionality to change)
[x] New feature (non-breaking change which adds functionality)
[ ] Bug fix (non-breaking change which fixes an issue)
[ ] Enhancement (improves an existing feature and functionality)
[ ] Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale

[x] Major
[ ] Minor

Screenshots (if appropriate):

How Has This Been Tested?
https://github.com/apache/cloudstack/pull/5110#issuecomment-868888524

Hi @davidjumani
we are also working on SUSE 15 support (for now focused on common and agent).
Maybe we can combine our efforts (see our progress here NDBSGMS/cloudstack#2)

Hi @davidjumani
we are also working on SUSE 15 support (for now focused on common and agent).
Maybe we can combine our efforts (see our progress here NDBSGMS#2)
Additional we will also add SUSE15 support to cloudstack-rpm-builder (see NDBSGMS/cloudstack-rpm-builder#1)

Hi @mib1185 that sounds like a good idea, So far opensuse is working as an agent, I've tested it against basic / adv zones and smoke tests are passing. Right now I'm testing it with ovs and dpdk. It will be great if you can review / update stuff that needs changes

Thanks @mib1185 I'll add the changes and run a round of tests

Maybe a more generell discussion, but we call it already only SUSE support, since upcoming openSUSE Leap 15.3 and SLES 15.3 will be official 100% compatible to each other (but from our observations already 15.2 is compatible in scope of needed dependencies for CloudStack) and those "SUSE support" will include openSUSE Leap and SLES. Or should we still differ between "openSUSE Leap" and "SUSE Linux Enterprise Server" support in CloudStack?

SLES

Good point, I'll rename it to suse

Maybe a more generell discussion, but we call it already only SUSE support, since upcoming openSUSE Leap 15.3 and SLES 15.3 will be official 100% compatible to each other (but from our observations already 15.2 is compatible in scope of needed dependencies for CloudStack) and those "SUSE support" will include openSUSE Leap and SLES. Or should we still differ between "openSUSE Leap" and "SUSE Linux Enterprise Server" support in CloudStack?

Good point, I'll rename it to suse

@blueorangutan package

@davidjumani a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: :heavy_multiplication_x: centos7 :heavy_multiplication_x: centos8 :heavy_check_mark: debian. SL-JID 238

@blueorangutan package

Packaging result: :heavy_check_mark: centos7 :heavy_check_mark: centos8 :heavy_check_mark: debian. SL-JID 239

@blueorangutan package

@davidjumani a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: :heavy_check_mark: centos7 :heavy_check_mark: centos8 :heavy_check_mark: debian. SL-JID 245

@mib1185 Can you test it with DPDK? I'm unable to in my environment
@GabrielBrascher Once this is ready would you be able to test suse + CEPH ?

@mib1185 Can you test it with DPDK? I'm unable to in my environment

I'm sorry, but our current (very limited) test and also our productive environment uses linux bridges, so honestly there is no experience with OVS nor DPDK on our side 😕
In addition, since we are currently in evaluation of <IP_ADDRESS>, we are very limited in our capacity for the next weeks.

Packaging result: :heavy_check_mark: centos7 :heavy_check_mark: centos8 :heavy_check_mark: debian. SL-JID 261

@blueorangutan package

@davidjumani a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: :heavy_check_mark: centos7 :heavy_check_mark: centos8 :heavy_check_mark: debian.  I'll keep you posted as I make progress.

Packaging result: :heavy_check_mark: centos7 :heavy_check_mark: centos8 :heavy_check_mark: debian.  I'll keep you posted as I make progress.

Packaging result: :heavy_check_mark: centos7 :heavy_check_mark: centos8 :heavy_check_mark: debian.  Thanks

@blueorangutan package

@davidjumani a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: :heavy_check_mark: centos7 :heavy_check_mark: centos8 :heavy_check_mark: debian. SL-JID 388

@blueorangutan package

Packaging result: :heavy_check_mark: centos7 :heavy_check_mark: centos8 :heavy_check_mark: debian. SL-JID 393

@blueorangutan package

@davidjumani a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: :heavy_multiplication_x: centos7 :heavy_check_mark: centos8 :heavy_check_mark: debian. SL-JID 395

@blueorangutan package

@davidjumani a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: :heavy_check_mark: centos7 :heavy_check_mark: centos8 :heavy_check_mark: debian. SL-JID 396

@blueorangutan test suse15 kvm-suse15

@davidjumani a Trillian-Jenkins test job (suse15 mgmt + kvm-suse15) has been kicked to run smoke tests

Packaging result: :heavy_check_mark: centos7 :heavy_check_mark: centos8 :heavy_check_mark: debian :heavy_check_mark: suse. SL-JID 414

@blueorangutan package

@davidjumani a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: :heavy_check_mark: centos7 :heavy_check_mark: centos8 :heavy_check_mark: debian :heavy_check_mark: suse. SL-JID 416

Trillian test result (tid-1140)
Environment: kvm-suse15 (x2), Advanced Networking with Mgmt server s15
Total time taken: 42506 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5110-t1140-kvm-suse15.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Smoke tests completed. 87 look OK, 1 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

test_07_deploy_kubernetes_ha_cluster
Failure
3608.06
test_kubernetes_clusters.py

test_08_deploy_and_upgrade_kubernetes_ha_cluster
Failure
0.07
test_kubernetes_clusters.py

test_09_delete_kubernetes_ha_cluster
Failure
0.05
test_kubernetes_clusters.py

ContextSuite context=TestKubernetesCluster>:teardown
Error
102.64
test_kubernetes_clusters.py

@blueorangutan package

@davidjumani a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: :heavy_check_mark: centos7 :heavy_check_mark: centos8 :heavy_check_mark: debian :heavy_check_mark: suse. SL-JID 434

@blueorangutan test suse15 kvm-suse15

@davidjumani a Trillian-Jenkins test job (suse15 mgmt + kvm-suse15) has been kicked to run smoke tests

Trillian test result (tid-1178)
Environment: kvm-suse15 (x2), Advanced Networking with Mgmt server s15
Total time taken: 44704 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5110-t1178-kvm-suse15.zip
Intermittent failure detected: /marvin/tests/smoke/test_internal_lb.py
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_routers_network_ops.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Smoke tests completed. 86 look OK, 2 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

test_02_RVR_Network_FW_PF_SSH_default_routes_egress_false
Failure
347.54
test_routers_network_ops.py

test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers
Failure
470.48
test_vpc_redundant.py

@blueorangutan package

@davidjumani a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 458

@blueorangutan test suse15 kvm-suse15

@davidjumani a Trillian-Jenkins test job (suse15 mgmt + kvm-suse15) has been kicked to run smoke tests

Trillian test result (tid-1195)
Environment: kvm-suse15 (x2), Advanced Networking with Mgmt server s15
Total time taken: 36500 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5110-t1195-kvm-suse15.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_host_maintenance.py
Intermittent failure detected: /marvin/tests/smoke/test_hostha_kvm.py
Smoke tests completed. 87 look OK, 1 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

test_hostha_enable_ha_when_host_disabled
Error
5.79
test_hostha_kvm.py

test_hostha_enable_ha_when_host_in_maintenance
Error
304.75
test_hostha_kvm.py

@davidjumani please have a look at https://github.com/apache/cloudstack/pull/5110#discussion_r663463852 ... except this small comment - great work :+1:

@blueorangutan package

@davidjumani a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 492

@blueorangutan package

@davidjumani a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@blueorangutan package

@davidjumani a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 494

@blueorangutan test suse15 kvm-suse15

@davidjumani a Trillian-Jenkins test job (suse15 mgmt + kvm-suse15) has been kicked to run smoke tests

Trillian test result (tid-1217)
Environment: kvm-suse15 (x2), Advanced Networking with Mgmt server s15
Total time taken: 34671 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5110-t1217-kvm-suse15.zip
Intermittent failure detected: /marvin/tests/smoke/test_host_maintenance.py
Intermittent failure detected: /marvin/tests/smoke/test_hostha_kvm.py
Smoke tests completed. 87 look OK, 1 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

test_hostha_enable_ha_when_host_disabled
Error
3.58
test_hostha_kvm.py

test_hostha_enable_ha_when_host_in_maintenance
Error
302.71
test_hostha_kvm.py

@blueorangutan package

@davidjumani a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 504

@blueorangutan test suse15 kvm-suse15

@andrijapanicsb a Trillian-Jenkins test job (suse15 mgmt + kvm-suse15) has been kicked to run smoke tests

Feature LGTM, here is what have been tested:
Smoke test results (Suse mgmt + suse KVM):
88 look ok, 0 have errors
Manual tests performed:
console proxy on system and regular VMs - OK
deploy a VM from a default template, uploaded template and ISO image - OK
VM operations: reboot, stop/start on another host, reinstall, create/revert snapshot, move to another host, attach/detach ISO, recover, expunge - OK
volume operations: resize, migrate, destroy/recover, attach disk, create/revert snapshot, create template from snapshot - OK
storages: add/remove primary and secondary storage, mark secondary storage read-only and back - OK
system VMs operations: start/stop, reboot, migrate, get diag. data, run diagnostics, destroy - OK
outbound traffic: add/remove egress rules to VM network to allow outbound traffic - OK
inbound traffic: add/remove firewall and port-forwarding rules to allow inbound connections - OK
usage server: make sure the usage service is gathering usage data - OK
roles: deploy a VM, create a network as regular user - OK
VM network operations: add/remove network to VM, add/remove secondary IP address, change IP address, set default NIC root - OK
hosts: add/remove, enable/disable a host, enable/disable maintenance mode - OK
Basic zone with shared storage - Zone deployment + basic VM / template / network / storage lifecycle - OK
Basic zone with local storage - Zone deployment + basic VM / template / network / storage lifecycle - OK
Advanced zone without SG and with shared storage - 100% Smoke Tests - OK
Advanced zone without SG and with local storage - Smoke Tests - OK
Other combinations: VLAN, VXLAN, config drive - OK
Storage: Local, NFS - OK
Test Matrix :  (Trillian combinations)

C7 MS + Suse KVM Agent - OK
Suse MS + Suse KVM Agent - OK
Suse MS + C7 KVM Agent - OK
Suse MS + VMware 6.7u3 / 7.0 - OK / OK
Suse MS + XCPNG 8.2 / LTS - OK

Support for packaging and testing with BO (should be quick if jenkins jobs exist) - OK
Upstream docs with updates to installation and QIG sections - OK

@blueorangutan package

@vladimirpetrov a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 760

@blueorangutan test suse15 kvm-suse15

@andrijapanicsb a Trillian-Jenkins test job (suse15 mgmt + kvm-suse15) has been kicked to run smoke tests

@vladimirpetrov - where are the results for this:
_Smoke test results (Suse mgmt + suse KVM):
88 look ok, 0 have errors
C7 MS + Suse KVM Agent - OK
Suse MS + Suse KVM Agent - OK
Suse MS + C7 KVM Agent - OK
Suse MS + VMware 6.7u3 / 7.0 - OK / OK
Suse MS + XCPNG 8.2 / LTS - OK_

i.e. why are not visible here, publicly? (ping me offline if needed) - thx

Trillian test result (tid-1486)
Environment: kvm-suse15 (x2), Advanced Networking with Mgmt server s15
Total time taken: 45775 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5110-t1486-kvm-suse15.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_vm_life_cycle.py
Intermittent failure detected: /marvin/tests/smoke/test_host_maintenance.py
Intermittent failure detected: /marvin/tests/smoke/test_hostha_kvm.py
Smoke tests completed. 87 look OK, 2 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

test_01_migrate_VM_and_root_volume
Error
71.32
test_vm_life_cycle.py

test_02_migrate_VM_with_two_data_disks
Error
48.15
test_vm_life_cycle.py

test_hostha_enable_ha_when_host_disabled
Error
0.65
test_hostha_kvm.py

test_hostha_enable_ha_when_host_in_maintenance
Error
304.10
test_hostha_kvm.py

@davidjumani may I kindly ask you to check the failures please. cc @vladimirpetrov

@andrijapanicsb Env related, can be ignored but will kick them again

@blueorangutan package

@davidjumani a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 782

@blueorangutan test suse15 kvm-suse15

@davidjumani a Trillian-Jenkins test job (suse15 mgmt + kvm-suse15) has been kicked to run smoke tests

@vladimirpetrov - where are the results for this:
_Smoke test results (Suse mgmt + suse KVM):
88 look ok, 0 have errors
C7 MS + Suse KVM Agent - OK
Suse MS + Suse KVM Agent - OK
Suse MS + C7 KVM Agent - OK
Suse MS + VMware 6.7u3 / 7.0 - OK / OK
Suse MS + XCPNG 8.2 / LTS - OK_

i.e. why are not visible here, publicly? (ping me offline if needed) - thx

tests were run "in house" for specific reasons - but are LGTM:
_Build #1305 - FR140-opensuse-smoketest (Jul 8, 2021 2:58:54 PM)
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Smoke tests completed. 88 look OK, 0 have error(s)
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---_

I would be happy to see the last run that @davidjumani initiated, show now/acceptable failures, and then LGTM this.

last tests failed.
@blueorangutan test suse15 kvm-suse15

@andrijapanicsb a Trillian-Jenkins test job (suse15 mgmt + kvm-suse15) has been kicked to run smoke tests

Trillian test result (tid-1539)
Environment: kvm-suse15 (x2), Advanced Networking with Mgmt server s15
Total time taken: 59103 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5110-t1539-kvm-suse15.zip
Intermittent failure detected: /marvin/tests/smoke/test_internal_lb.py
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_privategw_acl.py
Intermittent failure detected: /marvin/tests/smoke/test_routers_network_ops.py
Intermittent failure detected: /marvin/tests/smoke/test_vm_life_cycle.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermittent failure detected: /marvin/tests/smoke/test_host_maintenance.py
Intermittent failure detected: /marvin/tests/smoke/test_hostha_kvm.py
Smoke tests completed. 83 look OK, 6 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

test_07_deploy_kubernetes_ha_cluster
Failure
3631.12
test_kubernetes_clusters.py

test_08_deploy_and_upgrade_kubernetes_ha_cluster
Failure
0.06
test_kubernetes_clusters.py

test_09_delete_kubernetes_ha_cluster
Failure
0.04
test_kubernetes_clusters.py

ContextSuite context=TestKubernetesCluster>:teardown
Error
40.88
test_kubernetes_clusters.py

test_04_rvpc_privategw_static_routes
Failure
522.65
test_privategw_acl.py

test_01_isolate_network_FW_PF_default_routes_egress_true
Failure
121.38
test_routers_network_ops.py

test_01_migrate_VM_and_root_volume
Error
81.84
test_vm_life_cycle.py

test_02_migrate_VM_with_two_data_disks
Error
56.51
test_vm_life_cycle.py

test_10_attachAndDetach_iso
Error
12.35
test_vm_life_cycle.py

test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL
Failure
559.50
test_vpc_redundant.py

test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers
Failure
509.19
test_vpc_redundant.py

test_hostha_enable_ha_when_host_in_maintenance
Error
301.77
test_hostha_kvm.py

@blueorangutan test suse15 kvm-suse15

@andrijapanicsb a Trillian-Jenkins test job (suse15 mgmt + kvm-suse15) has been kicked to run smoke tests

@blueorangutan test suse15 kvm-suse15

@rhtyd a Trillian-Jenkins test job (suse15 mgmt + kvm-suse15) has been kicked to run smoke tests

LGTM based on manual and automatic tests.

automatic tests are failing miserably, so we cant LGMT-it until they pass with same "score" as the master branch (I understand this is local lab issue - but other won't understand/know this, by browsing the PR test results)
No LGTM until tests show no errors (or just the known ones)

@blueorangutan test suse15 kvm-suse15

@rhtyd a Trillian-Jenkins test job (suse15 mgmt + kvm-suse15) has been kicked to run smoke tests

Trillian test result (tid-1637)
Environment: kvm-suse15 (x2), Advanced Networking with Mgmt server s15
Total time taken: 35879 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5110-t1637-kvm-suse15.zip
Intermittent failure detected: /marvin/tests/smoke/test_vm_life_cycle.py
Intermittent failure detected: /marvin/tests/smoke/test_host_maintenance.py
Intermittent failure detected: /marvin/tests/smoke/test_hostha_kvm.py
Smoke tests completed. 87 look OK, 2 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

test_01_migrate_VM_and_root_volume
Error
70.47
test_vm_life_cycle.py

test_02_migrate_VM_with_two_data_disks
Error
48.24
test_vm_life_cycle.py

test_hostha_enable_ha_when_host_in_maintenance
Error
304.85
test_hostha_kvm.py

@blueorangutan test suse15 kvm-suse15

@rhtyd a Trillian-Jenkins test job (suse15 mgmt + kvm-suse15) has been kicked to run smoke tests

Trillian test result (tid-1639)
Environment: kvm-suse15 (x2), Advanced Networking with Mgmt server s15
Total time taken: 38931 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5110-t1639-kvm-suse15.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_router_dhcphosts.py
Intermittent failure detected: /marvin/tests/smoke/test_vm_life_cycle.py
Smoke tests completed. 87 look OK, 2 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

ContextSuite context=TestKubernetesCluster>:teardown
Error
72.02
test_kubernetes_clusters.py

test_01_migrate_VM_and_root_volume
Error
71.76
test_vm_life_cycle.py

test_02_migrate_VM_with_two_data_disks
Error
49.22
test_vm_life_cycle.py

@davidjumani would kind confirming that the 3 failed tests are due to e.g. DNS resolution issue on the lab where the tests are running, i.e. NOT related to this PR - so that we can merge it?
I can't Approve the request  (myself) until we have clear understanding on the root cause of tests failures
Thanks

@blueorangutan package

@davidjumani a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_multiplication_x: suse15. SL-JID 900

@blueorangutan package

@davidjumani a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 901

@blueorangutan test suse15 kvm-suse15

@davidjumani a Trillian-Jenkins test job (suse15 mgmt + kvm-suse15) has been kicked to run smoke tests

@blueorangutan package

@davidjumani a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 905

@blueorangutan test suse15 kvm-suse15

@davidjumani a Trillian-Jenkins test job (suse15 mgmt + kvm-suse15) has been kicked to run smoke tests

Trillian test result (tid-1687)
Environment: kvm-suse15 (x2), Advanced Networking with Mgmt server s15
Total time taken: 38335 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5110-t1687-kvm-suse15.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Smoke tests completed. 88 look OK, 1 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

ContextSuite context=TestKubernetesCluster>:teardown
Error
69.91
test_kubernetes_clusters.py

Trillian test result (tid-1692)
Environment: kvm-suse15 (x2), Advanced Networking with Mgmt server s15
Total time taken: 41385 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5110-t1692-kvm-suse15.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_vm_life_cycle.py
Intermittent failure detected: /marvin/tests/smoke/test_host_maintenance.py
Intermittent failure detected: /marvin/tests/smoke/test_hostha_kvm.py
Smoke tests completed. 88 look OK, 1 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

test_hostha_enable_ha_when_host_disabled
Error
2.61
test_hostha_kvm.py

test_hostha_enable_ha_when_host_in_maintenance
Error
304.84
test_hostha_kvm.py

Env related issues, good to merge


Enhance Kasan

Summary

Add software tag kasan
Fixed a series of kasan detection issues

Impact
No
Testing
No

remove "Revert" commits in list

Error: /home/runner/work/nuttx/nuttx/nuttx/mm/kasan/hook.c:409:5: error: Mixed case identifier found
Error: /home/runner/work/nuttx/nuttx/nuttx/mm/kasan/hook.c:414:5: error: Mixed case identifier found This cannot be solved because the hwasan instrumentation function itself is case-sensitive.

Check / check (pull_request) Error: /home/runner/work/nuttx/nuttx/nuttx/mm/kasan/hook.c:409:5: error: Mixed case identifier found Error: /home/runner/work/nuttx/nuttx/nuttx/mm/kasan/hook.c:414:5: error: Mixed case identifier found This cannot be solved because the hwasan instrumentation function itself is case-sensitive.

let's create a patch add the whitelist to tools/nxstyle.c

Check / check (pull_request) Error: /home/runner/work/nuttx/nuttx/nuttx/mm/kasan/hook.c:409:5: error: Mixed case identifier found Error: /home/runner/work/nuttx/nuttx/nuttx/mm/kasan/hook.c:414:5: error: Mixed case identifier found This cannot be solved because the hwasan instrumentation function itself is case-sensitive.

let's create a patch add the whitelist to tools/nxstyle.c. BTW, what's about documentation fail?

The last time the patch that added document was merged into the community, it might not have been detected.


Patch 1

Very minor updates to the README file

We should be targeting release/0.0.1 branch. I'll change the base branch.


Series astype new style

Add support of literal values for dtype
Unskip test_series_astype_literal_dtype1 test
Add test for errors='ignore' option

Hello @Vyacheslav-Smirnov! Thanks for opening this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file hpat/datatypes/hpat_pandas_series_functions.py:

Line 796:9: E129 visually indented line with same indent as next logical line
Line 829:5: E303 too many blank lines (2)
Line 830:9: E129 visually indented line with same indent as next logical line
Line 840:13: E722 do not use bare 'except'
Line 841:21: E261 at least two spaces before inline comment
Line 851:9: E722 do not use bare 'except'
Line 852:17: E261 at least two spaces before inline comment


Add a config option to include tests in the on-save syntax checking

Add an option to include tests in the on-save syntax checking handler. Without it tests are ignored e.g.:

With this feature enabled, the same code is checked correctly:

looks good!
Could you include a change to the documentation here: https://github.com/rust-lang/sublime-rust#syntax-checking ?
Thanks

Updated the README as suggested and rebased on current master.

@tnajdek is this ready to be merged now then?

sorry @Jayflux, I'm traveling and wasn't able to adapt the code, however it seems @ehuss has done just that in #149, which makes this PR obsolete.


[CERT-TEST-FAILURE] [TC-SEAR-1.3] - Python Test Failure

Feature Area
Other
Test Case
TC-SEAR-1.3
Reproduction steps

Command used to launch the DUT : ./chip-rvc-app
Command used to run the python script : python3 TC_SEAR_1_3.py --commissioning-method on-network --discriminator 3840 --passcode 20202021 --PICS connectedhomeip/examples/rvc-app/rvc-common/pics/rvc-app-pics-values  --app-pid <PID> --storage-path admin_storage.json --endpoint 1 --paa-trust-store-path ../../credentials/development/paa-root-certs/

While executing the TC-SEAR-1.3 In Test step 9 SelectArea command requires a list data type for the NewArea argument. In line no 128 valid_area is read from the PIXIT.SEAR.VALID_AREAS and it is used as a NewArea argument for the  SelectArea command hence it is failed with below error

[MatterTest] 09-02 12:25:19.350 INFO ***** Test Step 9 : Send SelectAreas command with NewAreas(7)
[MatterTest] 09-02 12:25:19.352 INFO Found an existing secure session to [1:0000000012344321]!
[MatterTest] 09-02 12:25:19.354 ERROR Exception occurred in test_TC_SEAR_1_3.
Traceback (most recent call last):
File "/home/ubuntu/cntr/connectedhomeip/no/lib/python3.10/site-packages/mobly/base_test.py", line 783, in exec_one_test
test_method()
File "/home/ubuntu/cntr/connectedhomeip/src/python_testing/matter_testing_support.py", line 1954, in async_runner
return _async_runner(body, self, *args, **kwargs)
File "/home/ubuntu/cntr/connectedhomeip/src/python_testing/matter_testing_support.py", line 1942, in _async_runner
return asyncio.run(runner_with_timeout)
File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/usr/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
return future.result()
File "/usr/lib/python3.10/asyncio/tasks.py", line 445, in wait_for
return fut.result()
File "/home/ubuntu/cntr/connectedhomeip/src/python_testing/TC_SEAR_1_3.py", line 129, in test_TC_SEAR_1_3
await self.send_cmd_select_areas_expect_response(step=9, new_areas=valid_areas, expected_response=Clusters.ServiceArea.Enums.SelectAreasStatus.kInvalidInMode)
File "/home/ubuntu/cntr/connectedhomeip/src/python_testing/TC_SEAR_1_3.py", line 69, in send_cmd_select_areas_expect_response
ret = await self.send_single_cmd(cmd=Clusters.Objects.ServiceArea.Commands.SelectAreas(newAreas=new_areas),
File "/home/ubuntu/cntr/connectedhomeip/src/python_testing/matter_testing_support.py", line 1217, in send_single_cmd
result = await dev_ctrl.SendCommand(nodeid=node_id, endpoint=endpoint, payload=cmd, timedRequestTimeoutMs=timedRequestTimeoutMs,
File "/home/ubuntu/cntr/connectedhomeip/no/lib/python3.10/site-packages/chip/ChipDeviceCtrl.py", line 1205, in SendCommand
res = await ClusterCommand.SendCommand(
File "/home/ubuntu/cntr/connectedhomeip/no/lib/python3.10/site-packages/chip/clusters/Command.py", line 317, in SendCommand
payloadTLV = payload.ToTLV()
File "/home/ubuntu/cntr/connectedhomeip/no/lib/python3.10/site-packages/chip/clusters/ClusterObjects.py", line 195, in ToTLV
return self.descriptor.DictToTLV(asdict(self))
File "/home/ubuntu/cntr/connectedhomeip/no/lib/python3.10/site-packages/chip/clusters/ClusterObjects.py", line 189, in DictToTLV
self.DictToTLVWithWriter('', None, data, tlvwriter)
File "/home/ubuntu/cntr/connectedhomeip/no/lib/python3.10/site-packages/chip/clusters/ClusterObjects.py", line 184, in DictToTLVWithWriter
_field.PutFieldToTLV(_field.Tag, val, writer, debugPath + f'.{_field.Label}')
File "/home/ubuntu/cntr/connectedhomeip/no/lib/python3.10/site-packages/chip/clusters/ClusterObjects.py", line 96, in PutFieldToTLV
self._PutSingleElementToTLV(
File "/home/ubuntu/cntr/connectedhomeip/no/lib/python3.10/site-packages/chip/clusters/ClusterObjects.py", line 59, in _PutSingleElementToTLV
if issubclass(elementType, ClusterObject):
TypeError: issubclass() arg 1 must be a class
[MatterTest] 09-02 12:25:19.363 INFO 		***** Test Failure :
[MatterTest] 09-02 12:25:19.363 INFO Finished test in 747620.0ms
[MatterTest] 09-02 12:25:19.364 ERROR

Test test_TC_SEAR_1_3 failed for the following reason:
issubclass() arg 1 must be a class

File "/home/ubuntu/cntr/connectedhomeip/no/lib/python3.10/site-packages/chip/clusters/ClusterObjects.py", line 59, in _PutSingleElementToTLV
TypeError: issubclass() arg 1 must be a class

Hence python script needs to be updated with the valid_areas data type to the list object. It has to be updated from valid_areas = self.matter_test_config.global_test_params['PIXIT.SEAR.VALID_AREAS'] to valid_areas = [self.matter_test_config.global_test_params['PIXIT.SEAR.VALID_AREAS']].
Bug prevalence
Whenever we do
GitHub hash of the SDK that was being used
708558881bf6e5b0dac3c09489124907927b6cd2
Platform
raspi
Anything else?
Same Issue with Test steps 11, 12, 13, 15
Python Script References:

https://github.com/project-chip/connectedhomeip/blob/master/src/python_testing/TC_SEAR_1_3.py

Reference Log:
TC-SEAR-1.3.txt

We should --json-arg PIXIT.SEAR.VALID_AREAS:'[7, 1234567]' argument in python script run command. Issue is resolved. Hence closing it.


Minor fixes for the lmp installer qemu instructions

Readiness

[ ] Merge (pending reviews)
[ ] Merge after date or event
[ ] Draft

Overview
Why merge this PR? What does it solve?
Checklist
Optional. Add a 'x' to steps taken.
You can fill this out after opening the PR. "Did I..."

[ ] Run spelling and grammar check, preferably with linter.
[ ] Avoid changing any header associated with a link/reference.
[ ] Step through instructions (or ask someone to do so).
[ ] Review for wordiness
[ ] Match tone and style of page/section.
[ ] Run make linkcheck.
[ ] View HTML in a browser to check rendering.
[ ] Use semantic newlines.
[ ] follow best practices for commits.

[ ] Descriptive title written in the imperative.
[ ] Include brief overview of QA steps taken.
[ ] Mention any related issues numbers.
[ ] End message with sign off/DCO line (-s, --signoff).
[ ] Sign commit with your gpg key (-S, --gpg-sign).
[ ] Squash commits if needed.

[ ] Request PR review by a technical writer and at least one peer.

Comments
Any thing else that a maintainer/reviewer should know.
This could include potential issues, rational for approach, etc.

Docs for 23b0c480277e8549786b427f1c68665879557dc7 are browsable at: https://ci.foundries.io/projects/fio-docs/builds/1687/docs/artifacts/html/index.html

Changes look fine, and I can merge when ready. Please remember to fill out the PR Template for all doc PRs. (I may add an option for small changes where most of the checklist does not apply in the near future.) The readiness section is particularly helpful. Also all commits should be both signed and signed-off to avoid having the merge blocked and requiring bypassing branch protections.


📝 Documentation: Long-term project vision

Bug Report Checklist

[X] I have pulled the latest main branch of the repository.
[X] I have searched for related issues and found none that matched my issue.

Overview
create-typescript-app has come a long way in the last two (!) years! It started as a small template for me to unify my disparate repository config files. Now it's a full project with >500 stars, repeat community contributors, and offshoot projects to help it work smoothly. I love this. 💖
I plan on continuing to prioritize create-typescript-app over at least the next year. I see its progress as evolving through at least three distinct stages:

(~2022) Personal template: used just in my own personal repos
(~2023) Shared template: Adding in more comprehensive docs, options, and generally more flexibility so folks other than me can use it on their repos too
(~2024) Shared engine: Splitting out a new package so that other templates like this one can be made

My hope is that in the next year or so, folks will be able to make their own shared templates that mix-and-match pieces of tooling. For example, a GitHub Actions flavor of create-typescript-app might use all the same pieces as this one except it'd swap out the builder from tsup to web-ext. See also #1175.
I don't know exactly how this would look - but I am excited to find out 😄.
Filing this issue to track placing a slightly more solidified form of this explanation in the docs.
Additional Info
No response

OK, I've done a good amount of think, and I'm pretty sure I've understood how this should work. I think there are roughly five layers that'll need to be made:

🏷 Inputs: read in data from the creation context
🧱 Blocks: each individual dev tooling piece, optionally with data from 🏷 inputs
b. 🪪 Metadata: signals output from the block that can be used in other blocks
b. 🧹 Migrations: descriptions of how to clean up from previous versions
🧰 Addons: extensions that provide additional input to a block, optionally with data from 🏷 inputs
🎁 Presets: configurable groups of 🧱 blocks and 🧰 addons
💝 create: the end-user runtime that receives all that info and creates or updates a repository

I'm thinking the 💝 create package will be what manages those tooling layers.
Pieces of each will be published as packages that you can then pull into your project.
Any customizations done in a package will need to be applied as options to a project pulls in.
This will solve the issue of applying changes after migration (#1184).
On top of all that will be end-user templates such as ➕ create-typescript-app.
They'd include any specific logic for building tools.
🏷 Inputs
🏷 Inputs will be small metadata-driven functions that provide any data needed to inform 🧱 blocks and 🧰 addons later.
These may include:

Files on disk, such as raw text or parsed .json
Sending network requests
The user's npm whoami

💝 create will manage providing 🏷 inputs with a runtime context containing a file system, network fetcher, and shell runner.
For example, an 🏷 input that retrieves the current running time:
import { createInput } from "@create/input";

export const inputJSONFile = createInput({
  produce: () => performance.now(),
});

Note that 🧱 blocks and 🧰 addons won't be required to use 🏷 inputs to source data.
Doing so just makes that data easier to mock out in tests later on.
🏷 Input 📥 Options
🏷 Inputs will need to be reusable and able to take in 📥 options.
They'll describe those options as the properties of a Zod object schema.
That will let them validate provided values and infer types from an options property in their context.
For example, an 🏷 input that retrieves JSON data from a file on disk using the provided virtual file system:
import { createInput } from "@create/input";
import { z } from "zod";

export const inputJSONFile = createInput({
  options: {
    fileName: z.string(),
  },
  async produce({ fs, options }) {
    try {
      return JSON.parse((await fs.readFile(options.fileName)).toString());
    } catch {
      return undefined;
    }
  },
});

Later on, 🧱 blocks and 🧰 addons that use the input will be able to provide those options.
🏷 Input 🧪 Testing
The create ecosystem will include testing utilities that provide mock data to an 🏷 input under test.
For example, testing the previous inputJSONFile:
import { createMockInputContext } from "@create/input-tester";
import { inputJSONFile } from "./inputJSONFile.ts";

describe("inputJSONFile", () => {
  it("returns package data when the file on disk contains valid JSON", () => {
    const expected = { name: "mock-package" };
    const context = createMockInputContext({
      files: {
        "package.json": JSON.stringify(expected),
      },
      options: {
        fileName: "package.json",
      },
    });

    const actual = await inputJSONFile(context);

    expect(actual).toEqual(expected);
  });
});

🏷 Input Composition
🏷 Inputs should be composable: meaning each can take data from other inputs.
💝 create will include a take function in contexts that calls another 🏷 input with the current context.
For example, an 🏷 input that determines the npm username based on either npm whoami or package.json 🏷 inputs:
import { createInput } from "@create/input";
import { inputJSONFile } from "@example/input-json-data";
import { inputNpmWhoami } from "@example/input-npm-whoami";

export const inputNpmUsername = createInput({
  async produce({ fs, take }) {
    return (
      (await take(inputNpmWhoami)) ??
      (await take(inputJSONFile, { fileName: "package.json" })).author
    );
  },
});

🧱 Blocks
The main logic for template contents will be stored in 🧱 blocks.
Each will define its shape of 🏷 inputs, user-provided options, and resultant outputs.
Resultant outputs will be passed to create to be merged with other 🧱 blocks' outputs and applied.
Outputs may include:

Cleanup scripts to run after setup
Files to create or modify on disk
Network requests to the GitHub API
Packages to install

For example, a 🧱 block that adds a .nvmrc file:
import { createBlock } from "@create/block";

export const blockNvmrc = createBlock({
  async produce() {
    return {
      files: {
        ".nvmrc": "20.12.2",
      },
    };
  },
});

The create ecosystem will include testing utilities that provide mock data to a 🧱 block under test:
import { createMockBlockContext } from "@create/block-tester";
import { blockNvmrc } from "./blockNvmrc.ts";

describe("blockNvmrc", () => {
  it("returns an .nvmrc", () => {
    const context = createMockInputContext();

    const actual = await blockNvmrc(context);

    expect(actual).toEqual({ ".nvmrc": "20.12.2" });
  });
});

🧱 Blocks and 🏷 Inputs
Blocks can take in data from 🏷 inputs.
💝 create will handle lazily evaluating 🏷 inputs and retrieving user-provided inputs.
They'll receive the same take function in their context that executes an 🏷 input.
For example, a 🧱 block that adds all-contributors recognition using a JSON file 🏷 input:
import { BlockContext, BlockOutput } from "@create/block";
import { formatYml } from "format-yml"; // todo: make package

export const blockAllContributors = createBlock({
  async produce({ take }) {
    const existing = await take(inputJSONFile, {
      fileName: "package.json",
    });

    return {
      files: {
        ".all-contributorsrc": JSON.parse({
          // ...
          contributors: existing?.contributors ?? [],
          // ...
        }),
        ".github": {
          workflows: {
            "contributors.yml": formatYml({
              // ...
              name: "Contributors",
              // ...
            }),
          },
        },
      },
    };
  },
});

🧱 Block 📥 Options
🧱 Blocks may be configurable with user options similar to 🏷 inputs.
They will define them as the properties for a Zod object schema and then receive them in their context.
For example, a 🧱 block that adds Prettier formatting with optional Prettier options:
import { createBlock } from "@create/block";
import prettier from "prettier";
import { prettierSchema } from "zod-prettier-schema"; // todo: make package
import { z } from "zod";

export const blockPrettier = createBlock({
  options: {
    config: prettierSchema.optional(),
  },
  async produce({ options }) {
    return {
      files: {
        ".prettierrc.json":
          options.config &&
          JSON.stringify({
            $schema: "http://json.schemastore.org/prettierrc",
            ...config,
          }),
      },
      packages: {
        devDependencies: ["prettier"],
      },
      scripts: {
        format: "prettier .",
      },
    };
  },
});

🧱 Block 📥 options will then be testable with the same mock context utilities as before:
import { createMockBlockContext } from "@create/block-tester";
import { blockPrettier } from "./blockPrettier.ts";

describe("blockPrettier", () => {
  it("creates a .prettierrc.json when provided options", () => {
    const prettierConfig = {
      useTabs: true,
    };
    const context = createMockInputContext({
      options: {
        config: prettierConfig,
      },
    });

    const actual = await blockPrettier(context);

    expect(actual).toEqual({
      files: {
        ".prettierrc.json": JSON.stringify({
          $schema: "http://json.schemastore.org/prettierrc",
          ...prettierConfig,
        }),
      },
      packages: {
        devDependencies: ["prettier"],
      },
      scripts: {
        format: "prettier .",
      },
    });
  });
});

🧱 Block 🪪 Metadata
🧱 Blocks should be able to signal added metadata on the system that other blocks will need to handle.
They can do so by returning properties in a metadata object.
Metadata may include:

documentation: A Record<string, string> of docs entries to add to .md file(s)
files: An array of objects containing glob: string and type: FileType of Config, Source, or Test

Over time, this may need to encompass more metadata, such as whether files are auto-generated

For example, this Vitest 🧱 block indicates that there can now be src/**/*.test.* test files, as documented in .github/DEVELOPMENT.md:
import { BlockOutput, FileType } from "@create/block";

export function blockVitest(): BlockOutput {
  return {
    files: {
      "vitest.config.ts": `import { defineConfig } from "vitest/config"; ...`,
    },
    metadata: {
      documentation: {
        ".github/DEVELOPMENT.md": `## Testing ...`,
      },
      files: [{ glob: "src/**/*.test.*", type: FileType.Test }],
    },
  };
}

In order to use 🪪 metadata provided by other blocks, block outputs can each be provided as a function.
That function will be called with an object containing all previously generated 🪪 metadata.
For example, this Tsup 🧱 block reacts to 🪪 metadata to exclude test files from its entry:
import { BlockContext, BlockOutput, FileType } from "@create/block";

export function blockTsup(): BlockOutput {
  return {
    fs: ({ metadata }: BlockContext) => {
      return {
        "tsup.config.ts": `import { defineConfig } from "tsup";
          // ...
          entry: [${JSON.stringify([
            "src/**/*.ts",
            ...metadata.files
              .filter(file.type === FileType.Test)
              .map((file) => file.glob),
          ])}],
          // ...
        `,
      };
    },
  };
}

In other words, 🧱 blocks will be executed in two phases:

An initial, metadata-less phase that can produce outputs and metadata
A second, metadata-provided phase that can produce more outputs

It would be nice to figure out a way to simplify them into one phase, while still allowing 🪪 metadata to be dependent on 📥 options.
A future design iteration might figure that out.
🧱 Block 🧹 Migrations
🧱 Blocks should be able to describe how to bump from previous versions to the current.
Those descriptions will be stored as 🧹 migrations detailing the actions to take to migrate from previous versions.
For example, a 🧱 block adding in Knip that switches from knip.jsonc to knip.json:
import { BlockContext, BlockOutput } from "@create/block";

export function blockKnip({ fs }: BlockKnip): BlockOutput {
  return {
    files: {
      "knip.json": JSON.stringify({
        $schema<EMAIL_ADDRESS>      }),
    },
    migrations: [
      {
        name: "Rename knip.jsonc to knip.json",
        run: async () => {
          try {
            await fs.rename("knip.jsonc", "knip.json");
          } catch {
            // Ignore failures if knip.jsonc doesn't exist
          }
        },
      },
    ],
  };
}

Migrations will allow create to be run in an idempotent --migrate mode that can keep a repository up-to-date automatically.
🧰 Addons
There will often be times when sets of 🧱 block options would be useful to package together.
For example, many packages consuming an ESLint 🧱 block might want to add on JSDoc linting rules.
Reusable generators for 📥 options will be available as 🧰 addons.
Their produced 📥 options will then be merged together by 💝 create and then passed to 🧱 blocks at runtime.
For example, a JSDoc linting 🧰 addon for a rudimentary ESLint linting 🧱 block with options for adding plugins:
import { createAddon } from "@create/addon";
import { blockESLint, BlockESLintOptions } from "@example/block-eslint";

export const addonESLintJSDoc = createAddon({
  produce(): AddonOutput<BlockESLintOptions> {
    return {
      options: {
        configs: [`jsdoc.configs["flat/recommended-typescript-error"]`],
        imports: [`import jsdoc from "eslint-plugin-jsdoc"`],
        rules: {
          "jsdoc/informative-docs": "error",
          "jsdoc/lines-before-block": "off",
        },
      },
    };
  },
});

Options produced by 🧰 addons will be merged together by ... spreading, both for arrays and objects.
The create ecosystem will include testing utilities that provide mock data to an 🧰 addon under test:
import { createMockAddonContext } from "@create/addon-tester";
import { addonESLintJSDoc } from "./addonESLintJSDoc.ts";

describe("addonESLintJSDoc", () => {
  it("returns configs, imports, and rules", () => {
    const context = createMockAddonContext();

    const actual = await addonESLintJSDoc(context);

    expect(actual).toEqual({
      options: {
        configs: [`jsdoc.configs["flat/recommended-typescript-error"]`],
        imports: [`import jsdoc from "eslint-plugin-jsdoc"`],
        rules: {
          "jsdoc/informative-docs": "error",
          "jsdoc/lines-before-block": "off",
        },
      },
    });
  });
});

🧰 Addon 📥 Options
🧰 Addons may be configurable with user options similar to 🏷 inputs and 🧱 blocks.
They should be able to describe their options as the properties for a Zod object schema, then infer types for their context.
For example, a Perfectionist linting 🧰 addon for a rudimentary ESLint linting 🧱 block with options for partitioning objects:
import { createAddon } from "@create/addon";
import { blockESLint, BlockESLintOptions } from "@example/block-eslint";
import { z } from "zod";

export const addonESLintPerfectionist = createAddon({
  options: {
    partitionByComment: z.boolean(),
  },
  produce({ options }): AddonOutput<BlockESLintOptions> {
    return {
      options: {
        configs: [`perfectionist.configs["recommended-natural"]`],
        imports: `import perfectionist from "eslint-plugin-perfectionist"`,
        rules: options.partitionByComment && {
          "perfectionist/sort-objects": [
            "error",
            {
              order: "asc",
              partitionByComment: true,
              type: "natural",
            },
          ],
        },
      },
    };
  },
});

🧰 Addon 📥 options will then be testable with the same mock context utilities as before:
import { createMockAddonContext } from "@create/addon-tester";
import { addonESLintPerfectionist } from "./addonESLintPerfectionist.ts";

describe("addonESLintPerfectionist", () => {
  it("includes perfectionist/sort-objects configuration when options.partitionByComment is provided", () => {
    const context = createMockAddonContext({
      options: {
        partitionByComment: true,
      },
    });

    const actual = await addonESLintPerfectionist(context);

    expect(actual).toEqual({
      options: {
        configs: [`perfectionist.configs["recommended-natural"]`],
        imports: `import perfectionist from "eslint-plugin-perfectionist"`,
        rules: {
          "perfectionist/sort-objects": [
            "error",
            {
              order: "asc",
              partitionByComment: true,
              type: "natural",
            },
          ],
        },
      },
    });
  });
});

🎁 Presets
Users won't want to manually configure 🧱 blocks and 🧰 addons in all of their projects.
🎁 Presets that configure broadly used or organization-wide configurations will help share setups.
For example, a 🎁 preset that configures ESLint, README.md with logo, and Vitest 🧱 blocks with JSONC linting, JSDoc linting, and test linting 🧰 addons:
import { createPreset } from "@create/preset";
import { blockESLint } from "@example/block-eslint";
import { blockReadme } from "@example/block-readme";
import { blockVitest } from "@example/block-vitest";

export const myPreset = createPreset({
  produce() {
    return [
      blockESLint({
        addons: [addonESLintJSDoc(), addonESLintJSONC(), addonESLintVitest()],
      }),
      blockReadme({
        logo: "./docs/my-logo.png",
      }),
      blockVitest(),
    ];
  },
});

🎁 Preset 📥 Options
🎁 Presets will need to be able to take in options.
As with previous layers, they'll describe their options as the properties for a Zod object schema.
For example, a 🎁 preset that takes in keywords and forwards them to a package.json 🧱 block:
import { createPreset } from "@create/preset";
import { blockPackageJson } from "@example/block-package-json";
import { z } from "zod";

export const myPreset = createPreset({
  options: {
    keywords: z.array(z.string()),
  },
  produce({ options }) {
    return [
      blockPackageJson({
        keywords: options.keywords,
      }),
    ];
  },
});

🎁 Preset 📄 Documentation
For example, the scaffolding of a 🧱 block that generates documentation for a preset from its entry point:
import { createBlock } from "@create/block";
import { z } from "zod";

createBlock({
  options: {
    entry: z.string().default("./src/index.ts"),
  },
  produce({ options }) {
    return {
      metadata: {
        documentation: {
          "README.md": `## Preset Options ...`,
        },
      },
    };
  },
});

Template Repositories
Users may opt to keep a GitHub template repository storing a canonical representation of their template.
The template can reference that repository's locator.
Projects created from the template can then be created from the template.
For example, a preset referencing a GitHub repository:
import { createPreset } from "@create/preset";

export const myTemplatePreset = createPreset({
  repository: "https://github.com/owner/repository",
  produce() {
    // ...
  },
});

This is necessary for including the "generated from" notice on repositories for a template.
The repository containing a preset might be built with a different preset.
For example, a repository containing presets for different native app builders might itself use a general TypeScript preset.
💝 create
Internally, create will:

Initialize shared context: the built-in options, file system, network fetcher, and shell runner
Run 🧱 blocks in order with their portions of the context

File, network, and shell operations are stored so they can be run later
Migrations are also stored to be run later
Any metadata are stored and merged internally

Run any stored migrations
Run delayed portions of 🧱 blocks in order that required metadata
Run all stored file, network, and shell operations

There may need to be options provided for changing when pieces run.
For example, there may be migrations that depend on being run before stored file operations.
💝 create CLIs
Initializing a new repository can be done by running create on the CLI.
Zod arguments will be automatically converted to Node.js parseArgs args.
Using the earlier myPreset example as a package named my-create-preset:
npx create --preset my-create-preset --preset-option-keywords abc --preset-option-keywords def

The result of running the CLI will be a repository that's ready to be developed on immediately.
💝 create Configuration
Users will alternately be able to set up the 🧱 blocks and 🧰 addons in a file like create.config.ts.
They will have the user default-export calling a createConfig function with an array of blocks.
For example, a small project that only configures one TypeScript 🧱 block to have a specific compiler option:
// create.config.ts
import { createConfig } from "@create/config";
import { blockTsc } from "@example/block-tsc";

export default createConfig([
  blockTsc({
    compilerOptions: {
      target: "ES2024",
    },
  }),
]);

A more realistic example would be this equivalent to the create-typescript-app "common" base with a logo and bin using a dedicated 🎁 preset:
// create.config.ts
import { createConfig } from "@create/config";
import { presetTypeScriptPackageCommon } from "@example/preset-typescript-package-common";

export default createConfig(
  presetTypeScriptPackageCommon({
    bin: "./bin/index.js",
    readme: {
      logo: "./docs/my-logo.png",
    },
  })
);

Running a command like npx create will detect the create.config.ts and re-run 💝 create for the repository.
Any 🧹 migrations will clean up out-of-date files.
💝 create CLI Prompts
It's common for template builders to include a CLI prompt for options.
💝 create will provide a dedicated CLI package that prompts users for options based on the Zod options schema for a 🎁 preset.
For example, given a 🎁 preset that describes its name and other documentation:
import { createPreset } from "@create/preset";

export const myPreset = createPreset({
  documentation: {
    name: "My Preset",
  },
  options: {
    access: z
      .union([z.literal("public"), z.literal("private")])
      .default("public"),
    description: z.string(),
  },
  produce() {
    /* ... */
  },
});

...a 💝 create CLI would be able to prompt a running user for each of those options:
npx create-my-preset

Let's ✨ create ✨ a repository for you based on My Preset!

> Enter a value for access.
  Allowed values: "public", "private" (default: "public")
  ...

> Enter a value for description:
  ...

> Would you like to make a create.config.ts file to pull in template updates later?
  y/n

Future versions of 💝 create could provide hooks to customize those CLIs, such as adding more documentation options in createPreset.
💝 create Monorepo Support
Adding explicit handling for monorepos is not something I plan for a v1 of 💝 create.
I'll want to have experience maintaining a few more of my own monorepos before seriously investigating what that would look like.
This does not block end-users from writing monorepo-tailored 🧱 blocks or 🎁 presets.
They can always write two versions of their logic for the ones that need it, such as:

@example/block-tsc
@example/block-tsc-references

Alternately, individual packages can always configure 💝 create tooling on their own.
➕ create-typescript-app
💝 create will be a general engine.
It won't have any specific 🧱 blocks or 🎁 presets built-in.
Instead, external packages such as ➕ create-typescript-app will take on the responsibility of creating their own framework-/library-specific 🧱 blocks and 🎁 presets.
For example, a non-exhaustive list of ➕ create-typescript-app packages might look like:

🧱 Blocks:

@create-typescript/block-all-contributors
@create-typescript/block-compliance
@create-typescript/block-contributing
@create-typescript/block-cspell
@create-typescript/block-eslint
@create-typescript/block-github-alt-text
@create-typescript/block-husky
@create-typescript/block-knip
@create-typescript/block-markdownlint
@create-typescript/block-package-json
@create-typescript/block-pnpm
@create-typescript/block-prettier
@create-typescript/block-license-mit
@create-typescript/block-readme
@create-typescript/block-release-it
@create-typescript/block-renovate
@create-typescript/block-tsc
@create-typescript/block-tsup
@create-typescript/block-vitest

🧰 Addons:

@create-typescript/addon-all-contributors-auto-action
@create-typescript/addon-eslint-comments
@create-typescript/addon-eslint-jsdoc
@create-typescript/addon-eslint-jsonc
@create-typescript/addon-eslint-eslint
@create-typescript/addon-eslint-md
@create-typescript/addon-eslint-regexp
@create-typescript/addon-eslint-perfectionist
@create-typescript/addon-eslint-vitest
@create-typescript/addon-markdownlint-sentences-per-line
@create-typescript/addon-pnpm-dedupe
@create-typescript/addon-prettier-plugin-curly
@create-typescript/addon-prettier-plugin-sh
@create-typescript/addon-prettier-plugin-packagejson
@create-typescript/addon-tsup-bin
@create-typescript/addon-vitest-console-fail-test
@create-typescript/addon-vitest-coverage

🎁 Presets:

@create-typescript/preset-minimal
@create-typescript/preset-common
@create-typescript/preset-everything

The 🎁 presets will be configurable with 📥 options to swap out pieces as needed for repositories.
For example, some repositories will want to swap out the Tsup 🧱 block for a different builder.
Over time, @create-typescript will encompass all common TypeScript package types from repositories I (Josh) use.
That will include browser extensions, GitHub actions, and web frameworks such as Astro.

OK, I'm pretty sure I've figured out how this should roughly work. I think there are five layers that'll need to be made:

🏷 Inputs: read in data from the creation context
🧱 Blocks: each individual dev tooling piece, optionally with data from 🏷 inputs
b. 🪪 Metadata: signals output from the block that can be used in other blocks
b. 🧹 Migrations: descriptions of how to clean up from previous versions
🧰 Addons: extensions that provide additional input to a block, optionally with data from 🏷 inputs
🎁 Presets: configurable groups of 🧱 blocks and 🧰 addons
💝 create: the end-user runtime that receives all that info and creates or updates a repository

I'm thinking the 💝 create package will be what manages those tooling layers. Pieces of each will be published as packages that you can then pull into your project. Any customizations done in a package will need to be applied as options to a project pulls in. This will solve the issue of applying changes after migration (#1184).
On top of all that will be end-user templates such as ➕ create-typescript-app. They'd include the actual 🧱 blocks, 🧰 addons, and 🎁 presets end-users will tell 💝 create to build their repositories with.
🏷 Inputs
🏷 Inputs will be small metadata-driven functions that provide any data needed to inform 🧱 blocks and 🧰 addons later.
These may include:

Files on disk, such as raw text or parsed .json
Sending network requests
The user's npm whoami

💝 create will manage providing 🏷 inputs with a runtime context containing a file system, network fetcher, and shell runner.
For example, an 🏷 input that retrieves the current running time:
import { createInput } from "@create/input";

export const inputJSONFile = createInput({
  produce: () => performance.now(),
});

Note that 🧱 blocks and 🧰 addons won't be required to use 🏷 inputs to source data. Doing so just makes that data easier to mock out in tests later on.
🏷 Input 📥 Options
🏷 Inputs will need to be reusable and able to take in 📥 options. They'll describe those options as the properties of a Zod object schema. That will let them validate provided values and infer types from an options property in their context.
For example, an 🏷 input that retrieves JSON data from a file on disk using the provided virtual file system:
import { createInput } from "@create/input";
import { z } from "zod";

export const inputJSONFile = createInput({
  options: {
    fileName: z.string(),
  },
  async produce({ fs, options }) {
    try {
      return JSON.parse((await fs.readFile(options.fileName)).toString());
    } catch {
      return undefined;
    }
  },
});

Later on, 🧱 blocks and 🧰 addons that use the input will be able to provide those options.
🏷 Input 🧪 Testing
The create ecosystem will include testing utilities that provide mock data to an 🏷 input under test.
For example, testing the previous inputJSONFile:
import { createMockInputContext } from "@create/input-tester";
import { inputJSONFile } from "./inputJSONFile.ts";

describe("inputJSONFile", () => {
  it("returns package data when the file on disk contains valid JSON", () => {
    const expected = { name: "mock-package" };
    const context = createMockInputContext({
      files: {
        "package.json": JSON.stringify(expected),
      },
      options: {
        fileName: "package.json",
      },
    });

    const actual = await inputJSONFile(context);

    expect(actual).toEqual(expected);
  });
});

🏷 Input Composition
🏷 Inputs should be composable: meaning each can take data from other inputs. 💝 create will include a take function in contexts that calls another 🏷 input with the current context.
For example, an 🏷 input that determines the npm username based on either npm whoami or package.json 🏷 inputs:
import { createInput } from "@create/input";
import { inputJSONFile } from "@example/input-json-data";
import { inputNpmWhoami } from "@example/input-npm-whoami";

export const inputNpmUsername = createInput({
  async produce({ fs, take }) {
    return (
      (await take(inputNpmWhoami)) ??
      (await take(inputJSONFile, { fileName: "package.json" })).author
    );
  },
});

🧱 Blocks
The main logic for template contents will be stored in 🧱 blocks. Each will define its shape of 🏷 inputs, user-provided options, and resultant outputs.
Resultant outputs will be passed to create to be merged with other 🧱 blocks' outputs and applied. Outputs may include:

Cleanup scripts to run after setup
Files to create or modify on disk
Network requests to the GitHub API
Packages to install

For example, a 🧱 block that adds a .nvmrc file:
import { createBlock } from "@create/block";

export const blockNvmrc = createBlock({
  async produce() {
    return {
      files: {
        ".nvmrc": "20.12.2",
      },
    };
  },
});

The create ecosystem will include testing utilities that provide mock data to a 🧱 block under test:
import { createMockBlockContext } from "@create/block-tester";
import { blockNvmrc } from "./blockNvmrc.ts";

describe("blockNvmrc", () => {
  it("returns an .nvmrc", () => {
    const context = createMockInputContext();

    const actual = await blockNvmrc(context);

    expect(actual).toEqual({ ".nvmrc": "20.12.2" });
  });
});

🧱 Blocks and 🏷 Inputs
Blocks can take in data from 🏷 inputs. 💝 create will handle lazily evaluating 🏷 inputs and retrieving user-provided inputs. They'll receive the same take function in their context that executes an 🏷 input.
For example, a 🧱 block that adds all-contributors recognition using a JSON file 🏷 input:
import { BlockContext, BlockOutput } from "@create/block";
import { formatYml } from "format-yml"; // todo: make package

export const blockAllContributors = createBlock({
  async produce({ take }) {
    const existing = await take(inputJSONFile, {
      fileName: "package.json",
    });

    return {
      files: {
        ".all-contributorsrc": JSON.parse({
          // ...
          contributors: existing?.contributors ?? [],
          // ...
        }),
        ".github": {
          workflows: {
            "contributors.yml": formatYml({
              // ...
              name: "Contributors",
              // ...
            }),
          },
        },
      },
    };
  },
});

🧱 Block 📥 Options
🧱 Blocks may be configurable with user options similar to 🏷 inputs. They will define them as the properties for a Zod object schema and then receive them in their context.
For example, a 🧱 block that adds Prettier formatting with optional Prettier options:
import { createBlock } from "@create/block";
import prettier from "prettier";
import { prettierSchema } from "zod-prettier-schema"; // todo: make package
import { z } from "zod";

export const blockPrettier = createBlock({
  options: {
    config: prettierSchema.optional(),
  },
  async produce({ options }) {
    return {
      files: {
        ".prettierrc.json":
          options.config &&
          JSON.stringify({
            $schema: "http://json.schemastore.org/prettierrc",
            ...config,
          }),
      },
      packages: {
        devDependencies: ["prettier"],
      },
      scripts: {
        format: "prettier .",
      },
    };
  },
});

🧱 Block 📥 options will then be testable with the same mock context utilities as before:
import { createMockBlockContext } from "@create/block-tester";
import { blockPrettier } from "./blockPrettier.ts";

describe("blockPrettier", () => {
  it("creates a .prettierrc.json when provided options", () => {
    const prettierConfig = {
      useTabs: true,
    };
    const context = createMockInputContext({
      options: {
        config: prettierConfig,
      },
    });

    const actual = await blockPrettier(context);

    expect(actual).toEqual({
      files: {
        ".prettierrc.json": JSON.stringify({
          $schema: "http://json.schemastore.org/prettierrc",
          ...prettierConfig,
        }),
      },
      packages: {
        devDependencies: ["prettier"],
      },
      scripts: {
        format: "prettier .",
      },
    });
  });
});

🧱 Block 🪪 Metadata
🧱 Blocks should be able to signal added metadata on the system that other blocks will need to handle. They can do so by returning properties in a metadata object.
Metadata may include:

documentation: A Record<string, string> of docs entries to add to .md file(s)
files: An array of objects containing glob: string and type: FileType of Config, Source, or Test

Over time, this may need to encompass more metadata, such as whether files are auto-generated

For example, this Vitest 🧱 block indicates that there can now be src/**/*.test.* test files, as documented in .github/DEVELOPMENT.md:
import { BlockOutput, FileType } from "@create/block";

export function blockVitest(): BlockOutput {
  return {
    files: {
      "vitest.config.ts": `import { defineConfig } from "vitest/config"; ...`,
    },
    metadata: {
      documentation: {
        ".github/DEVELOPMENT.md": `## Testing ...`,
      },
      files: [{ glob: "src/**/*.test.*", type: FileType.Test }],
    },
  };
}

In order to use 🪪 metadata provided by other blocks, block outputs can each be provided as a function.
That function will be called with an object containing all previously generated 🪪 metadata.
For example, this Tsup 🧱 block reacts to 🪪 metadata to exclude test files from its entry:
import { BlockContext, BlockOutput, FileType } from "@create/block";

export function blockTsup(): BlockOutput {
  return {
    fs: ({ metadata }: BlockContext) => {
      return {
        "tsup.config.ts": `import { defineConfig } from "tsup";
          // ...
          entry: [${JSON.stringify([
            "src/**/*.ts",
            ...metadata.files
              .filter(file.type === FileType.Test)
              .map((file) => file.glob),
          ])}],
          // ...
        `,
      };
    },
  };
}

In other words, 🧱 blocks will be executed in two phases:

An initial, metadata-less phase that can produce outputs and metadata
A second, metadata-provided phase that can produce more outputs

It would be nice to figure out a way to simplify them into one phase, while still allowing 🪪 metadata to be dependent on 📥 options. A future design iteration might figure that out.
🧱 Block 🧹 Migrations
🧱 Blocks should be able to describe how to bump from previous versions to the current. Those descriptions will be stored as 🧹 migrations detailing the actions to take to migrate from previous versions.
For example, a 🧱 block adding in Knip that switches from knip.jsonc to knip.json:
import { BlockContext, BlockOutput } from "@create/block";

export function blockKnip({ fs }: BlockKnip): BlockOutput {
  return {
    files: {
      "knip.json": JSON.stringify({
        $schema<EMAIL_ADDRESS>      }),
    },
    migrations: [
      {
        name: "Rename knip.jsonc to knip.json",
        run: async () => {
          try {
            await fs.rename("knip.jsonc", "knip.json");
          } catch {
            // Ignore failures if knip.jsonc doesn't exist
          }
        },
      },
    ],
  };
}

Migrations will allow create to be run in an idempotent --migrate mode that can keep a repository up-to-date automatically.
🧰 Addons
There will often be times when sets of 🧱 block options would be useful to package together. For example, many packages consuming an ESLint 🧱 block might want to add on JSDoc linting rules.
Reusable generators for 📥 options will be available as 🧰 addons. Their produced 📥 options will then be merged together by 💝 create and then passed to 🧱 blocks at runtime.
For example, a JSDoc linting 🧰 addon for a rudimentary ESLint linting 🧱 block with options for adding plugins:
import { createAddon } from "@create/addon";
import { blockESLint, BlockESLintOptions } from "@example/block-eslint";

export const addonESLintJSDoc = createAddon({
  produce(): AddonOutput<BlockESLintOptions> {
    return {
      options: {
        configs: [`jsdoc.configs["flat/recommended-typescript-error"]`],
        imports: [`import jsdoc from "eslint-plugin-jsdoc"`],
        rules: {
          "jsdoc/informative-docs": "error",
          "jsdoc/lines-before-block": "off",
        },
      },
    };
  },
});

Options produced by 🧰 addons will be merged together by ... spreading, both for arrays and objects.
The create ecosystem will include testing utilities that provide mock data to an 🧰 addon under test:
import { createMockAddonContext } from "@create/addon-tester";
import { addonESLintJSDoc } from "./addonESLintJSDoc.ts";

describe("addonESLintJSDoc", () => {
  it("returns configs, imports, and rules", () => {
    const context = createMockAddonContext();

    const actual = await addonESLintJSDoc(context);

    expect(actual).toEqual({
      options: {
        configs: [`jsdoc.configs["flat/recommended-typescript-error"]`],
        imports: [`import jsdoc from "eslint-plugin-jsdoc"`],
        rules: {
          "jsdoc/informative-docs": "error",
          "jsdoc/lines-before-block": "off",
        },
      },
    });
  });
});

🧰 Addon 📥 Options
🧰 Addons may be configurable with user options similar to 🏷 inputs and 🧱 blocks. They should be able to describe their options as the properties for a Zod object schema, then infer types for their context.
For example, a Perfectionist linting 🧰 addon for a rudimentary ESLint linting 🧱 block with options for partitioning objects:
import { createAddon } from "@create/addon";
import { blockESLint, BlockESLintOptions } from "@example/block-eslint";
import { z } from "zod";

export const addonESLintPerfectionist = createAddon({
  options: {
    partitionByComment: z.boolean(),
  },
  produce({ options }): AddonOutput<BlockESLintOptions> {
    return {
      options: {
        configs: [`perfectionist.configs["recommended-natural"]`],
        imports: `import perfectionist from "eslint-plugin-perfectionist"`,
        rules: options.partitionByComment && {
          "perfectionist/sort-objects": [
            "error",
            {
              order: "asc",
              partitionByComment: true,
              type: "natural",
            },
          ],
        },
      },
    };
  },
});

🧰 Addon 📥 options will then be testable with the same mock context utilities as before:
import { createMockAddonContext } from "@create/addon-tester";
import { addonESLintPerfectionist } from "./addonESLintPerfectionist.ts";

describe("addonESLintPerfectionist", () => {
  it("includes perfectionist/sort-objects configuration when options.partitionByComment is provided", () => {
    const context = createMockAddonContext({
      options: {
        partitionByComment: true,
      },
    });

    const actual = await addonESLintPerfectionist(context);

    expect(actual).toEqual({
      options: {
        configs: [`perfectionist.configs["recommended-natural"]`],
        imports: `import perfectionist from "eslint-plugin-perfectionist"`,
        rules: {
          "perfectionist/sort-objects": [
            "error",
            {
              order: "asc",
              partitionByComment: true,
              type: "natural",
            },
          ],
        },
      },
    });
  });
});

🎁 Presets
Users won't want to manually configure 🧱 blocks and 🧰 addons in all of their projects. 🎁 Presets that configure broadly used or organization-wide configurations will help share setups.
For example, a 🎁 preset that configures ESLint, README.md with logo, and Vitest 🧱 blocks with JSONC linting, JSDoc linting, and test linting 🧰 addons:
import { createPreset } from "@create/preset";
import { blockESLint } from "@example/block-eslint";
import { blockReadme } from "@example/block-readme";
import { blockVitest } from "@example/block-vitest";

export const myPreset = createPreset({
  produce() {
    return [
      blockESLint({
        addons: [addonESLintJSDoc(), addonESLintJSONC(), addonESLintVitest()],
      }),
      blockReadme({
        logo: "./docs/my-logo.png",
      }),
      blockVitest(),
    ];
  },
});

🎁 Preset 📥 Options
🎁 Presets will need to be able to take in options. As with previous layers, they'll describe their options as the properties for a Zod object schema.
For example, a 🎁 preset that takes in keywords and forwards them to a package.json 🧱 block:
import { createPreset } from "@create/preset";
import { blockPackageJson } from "@example/block-package-json";
import { z } from "zod";

export const myPreset = createPreset({
  options: {
    keywords: z.array(z.string()),
  },
  produce({ options }) {
    return [
      blockPackageJson({
        keywords: options.keywords,
      }),
    ];
  },
});

🎁 Preset 📄 Documentation
For example, the scaffolding of a 🧱 block that generates documentation for a preset from its entry point:
import { createBlock } from "@create/block";
import { z } from "zod";

createBlock({
  options: {
    entry: z.string().default("./src/index.ts"),
  },
  produce({ options }) {
    return {
      metadata: {
        documentation: {
          "README.md": `## Preset Options ...`,
        },
      },
    };
  },
});

Template Repositories
Users may opt to keep a GitHub template repository storing a canonical representation of their template. The template can reference that repository's locator. Projects created from the template can then be created from the template.
For example, a preset referencing a GitHub repository:
import { createPreset } from "@create/preset";

export const myTemplatePreset = createPreset({
  repository: "https://github.com/owner/repository",
  produce() {
    // ...
  },
});

This is necessary for including the "generated from" notice on repositories for a template. The repository containing a preset might be built with a different preset. For example, a repository containing presets for different native app builders might itself use a general TypeScript preset.
💝 create
Internally, create will:

Initialize shared context: the built-in options, file system, network fetcher, and shell runner
Run 🧱 blocks in order with their portions of the context

File, network, and shell operations are stored so they can be run later
Migrations are also stored to be run later
Any metadata are stored and merged internally

Run any stored migrations
Run delayed portions of 🧱 blocks in order that required metadata
Run all stored file, network, and shell operations

There may need to be options provided for changing when pieces run. For example, there may be migrations that depend on being run before stored file operations.
💝 create CLIs
Initializing a new repository can be done by running create on the CLI. Zod arguments will be automatically converted to Node.js parseArgs args.
Using the earlier myPreset example as a package named my-create-preset:
npx create --preset my-create-preset --preset-option-keywords abc --preset-option-keywords def

The result of running the CLI will be a repository that's ready to be developed on immediately.
💝 create Configuration
Users will alternately be able to set up the 🧱 blocks and 🧰 addons in a file like create.config.ts. They will have the user default-export calling a createConfig function with an array of blocks.
For example, a small project that only configures one TypeScript 🧱 block to have a specific compiler option:
// create.config.ts
import { createConfig } from "@create/config";
import { blockTsc } from "@example/block-tsc";

export default createConfig([
  blockTsc({
    compilerOptions: {
      target: "ES2024",
    },
  }),
]);

A more realistic example would be this equivalent to the create-typescript-app "common" base with a logo and bin using a dedicated 🎁 preset:
// create.config.ts
import { createConfig } from "@create/config";
import { presetTypeScriptPackageCommon } from "@example/preset-typescript-package-common";

export default createConfig(
  presetTypeScriptPackageCommon({
    bin: "./bin/index.js",
    readme: {
      logo: "./docs/my-logo.png",
    },
  })
);

Running a command like npx create will detect the create.config.ts and re-run 💝 create for the repository. Any 🧹 migrations will clean up out-of-date files.
💝 create CLI Prompts
It's common for template builders to include a CLI prompt for options. 💝 create will provide a dedicated CLI package that prompts users for options based on the Zod options schema for a 🎁 preset.
For example, given a 🎁 preset that describes its name and other documentation:
import { createPreset } from "@create/preset";

export const myPreset = createPreset({
  documentation: {
    name: "My Preset",
  },
  options: {
    access: z
      .union([z.literal("public"), z.literal("private")])
      .default("public"),
    description: z.string(),
  },
  produce() {
    /* ... */
  },
});

...a 💝 create CLI would be able to prompt a running user for each of those options:
npx create-my-preset

Let's ✨ create ✨ a repository for you based on My Preset!

> Enter a value for access.
  Allowed values: "public", "private" (default: "public")
  ...

> Enter a value for description:
  ...

> Would you like to make a create.config.ts file to pull in template updates later?
  y/n

Future versions of 💝 create could provide hooks to customize those CLIs, such as adding more documentation options in createPreset.
💝 create Monorepo Support
Adding explicit handling for monorepos is not something I plan for a v1 of 💝 create. I'll want to have experience maintaining a few more of my own monorepos before seriously investigating what that would look like.
This does not block end-users from writing monorepo-tailored 🧱 blocks or 🎁 presets. They can always write two versions of their logic for the ones that need it, such as:

@example/block-tsc
@example/block-tsc-references

Alternately, individual packages can always configure 💝 create tooling on their own.
➕ create-typescript-app
💝 create will be a general engine.
It won't have any specific 🧱 blocks or 🎁 presets built-in.
Instead, external packages such as ➕ create-typescript-app will take on the responsibility of creating their own framework-/library-specific 🧱 blocks and 🎁 presets.
For example, a non-exhaustive list of ➕ create-typescript-app packages might look like:

🧱 Blocks:

@create-typescript/block-all-contributors
@create-typescript/block-compliance
@create-typescript/block-contributing
@create-typescript/block-cspell
@create-typescript/block-eslint
@create-typescript/block-github-alt-text
@create-typescript/block-husky
@create-typescript/block-knip
@create-typescript/block-markdownlint
@create-typescript/block-package-json
@create-typescript/block-pnpm
@create-typescript/block-prettier
@create-typescript/block-license-mit
@create-typescript/block-readme
@create-typescript/block-release-it
@create-typescript/block-renovate
@create-typescript/block-tsc
@create-typescript/block-tsup
@create-typescript/block-vitest

🧰 Addons:

@create-typescript/addon-all-contributors-auto-action
@create-typescript/addon-eslint-comments
@create-typescript/addon-eslint-jsdoc
@create-typescript/addon-eslint-jsonc
@create-typescript/addon-eslint-eslint
@create-typescript/addon-eslint-md
@create-typescript/addon-eslint-regexp
@create-typescript/addon-eslint-perfectionist
@create-typescript/addon-eslint-vitest
@create-typescript/addon-markdownlint-sentences-per-line
@create-typescript/addon-pnpm-dedupe
@create-typescript/addon-prettier-plugin-curly
@create-typescript/addon-prettier-plugin-sh
@create-typescript/addon-prettier-plugin-packagejson
@create-typescript/addon-tsup-bin
@create-typescript/addon-vitest-console-fail-test
@create-typescript/addon-vitest-coverage

🎁 Presets:

@create-typescript/preset-minimal
@create-typescript/preset-common
@create-typescript/preset-everything

The 🎁 presets will be configurable with 📥 options to swap out pieces as needed for repositories. For example, some repositories will want to swap out the Tsup 🧱 block for a different builder.
Over time, @create-typescript will encompass all common TypeScript package types from repositories I (Josh) use. That will include browser extensions, GitHub actions, and web frameworks such as Astro.

I need time to digest this, but off the bat this looks beautiful and well thought out. Your emoji game is on point ❤

For example, a 🧱 block that adds a .nvmrc file:

This is amazing! Read through core parts to get an idea on things. DAMN!

cool, looks like a similar approach like https://github.com/projen/projen

I will respond to ☝ soon - finishing up some drafts!
In the meantime, @DonIsaac pointed me to https://nx.dev/features/generate-code. I'll comment on the differences with that too!

OK! In order...

https://github.com/projen/projen

Projen is really interesting, thanks for pointing me at it @JohannesKonings! I think it's a step in the right direction from Yeomen, similar to my vision for create. There are a few significant points I think we differ on:

It has templates implemented using classes and a class per project type, which is not the architecture I'd choose for configuration.

For example, additions to templates are done with project methods such as project.addDeps('axios').
Files are generated with class instances such as new JsonFile(project, 'src/config.json', {.

It imposes a much tighter grip on repositories than I'd want create to by default:

package.json tasks are expected to be managed by projen, e.g. "build": "npx projen build". In general, projen builds in a core concept of "tasks" that I don't want to couple to repo generation.
I wouldn't want to bake in project-specific concepts such as "Bundling" to the core engine, even if they are common to many projects.

I don't see in the docs or vision several features that I think are core to what create needs: migration, inference of options from existing files, granular block-level testing (not just project-level)

Ultimately, even though Projen looks great and has an active team behind it, I don't think it's a match for what I'm looking to use in create-typescript-app. It seems like a much better fit for teams rather than small open source project Its focus seems to be more around integration and a comprehensive core engine. My leaning is towards having an extensible core engine focused on web projects. I could be very wrong here 😅 but, next point...
Another ding against using Projen is that we're just very different maintenance crews. Projen is still 0.x and it would take a lot of time for me to ramp up on it & get integrated with the team. Pragmatically speaking, it'd be easier for me to 'tunnel vision' on building my own thing that focuses on my use case. Then later on, if I discover why another project is better, the core building blocks of create & create-typescript-app built on it should be moveable.

https://nx.dev/features/generate-code

I took a dive and this isn't what I'm looking for here. From https://nx.dev/extending-nx/recipes/local-generators, generators "automate many tasks you regularly perform as part of your development workflow" - but that's generally scoped within a repository. They're not targeted to composable, reusable templates.

I put a very rough starting sketch of a create-based CTA here: https://github.com/JoshuaKGoldberg/create-typescript-app/tree/create. It's super early stage and I'll have to iterate on it a bunch to get its files output the same as the template as-is. But I'm excited that the core engine is starting to shape up!
What'll likely happen next is:

I'll iterate on the core create engine and branch here until there's feature parity on file generation
I'll add APIs to create for just the files portion, thereby allowing...
create-typescript-app to use create's file APIs to generate files
Repeat steps 2-3 for each of the remaining parts of create-typescript-app
Eventually, create-typescript-app will be mostly a thin shell around create - at which point I'll be able to switch it over entirely

OK! #1670 is merged, and adds a CTA_CREATE_ENGINE=true flag that switches the files creation portion of CTA to blocks using the https://github.com/JoshuaKGoldberg/create engine. The engine's design went through quite a few iterations after https://github.com/JoshuaKGoldberg/create-typescript-app/issues/1181#issuecomment-2284797207. I think they simplified into something cleaner:

There's no more delineation between user-defined 🧱 blocks and 🧰 addons: everything is just a 🧱 block now.
There's no more concept of "delayed" 🧱 blocks. Instead, 🧱 blocks are re-run with any additional 📥 options: now, amusingly, called 📥 addons. The engine intelligently knows how to merge 📥 addons together and detect when the 📥 addons for a 🧱 block have changed.
Testing is done with dedicated functions like testBlock and testInput. That's a single step for tests, instead of two steps to create a mock context and then call the construct in question.
🪪 Metadata is completely gone. Everything is done using 🧱 block 📥 options. This allows the engine to remain agnostic: instead of, say, hardcoding the concept of "documentation" to Record<string, string>, an individual project can define its own documentation 🧱 block.
Instead of 🎁 Preset-specific 📥 options, there is a new concept of :basecamp: bases. A :basecamp: base defines the shared options used by 🧱 blocks.
🎁 Presets define their 🧱 blocks as a straightforward array, like blocks: [blockTsup, blockTypeScript]. This forces the 🧱 blocks to receive any important settings from their :basecamp: base.

https://create-josh.vercel.app -> has a description of the engine. It doesn't use emojis, and is built on Astro Starlight, so I hope it's a bit more comprehensible than this issue. 😂
I'm really, really pleased with this. I think the leaner architecture is really cool and will make projects like CTA intellectually fun to work on.
Closing this issue out as resolved by #1670 because my personal focus will now be migrating the rest of CTA's internals onto create. That will necessitate building features into create, so I'm tracking things in both repos:

Here: milestone/6 Create Engine Onboarding
There: create/milestone/1 create-typescript-app Feature Parity

I'll file issues soon in both milestones to track all the work that needs to be done. Very exciting! 🥳