Cluster with apiEndpoints set fails to come up

Bringing up a new cluster up on latest master ae4956992dcea09a3b6f0973a2822f4875d1859c does not seem to work for config with apiEndpoints from https://github.com/kubernetes-incubator/kube-aws/pull/468, the workers fail to join the cluster.
With this config all works fine as before and the cluster comes up:
externalDNSName: k8s-dev-v1.REDACTED
createRecordSet: true
recordSetTTL: 60
#hostedZone: ""
hostedZoneId: REDACTED

But switching to this config where everything else is identical the workers fail to become "healthy":
#externalDNSName: k8s-dev-v1.REDACTED
#createRecordSet: true
#recordSetTTL: 60
#hostedZone: ""
#hostedZoneId: REDACTED

apiEndpoints:
  - name: versioned
    createRecordSet: true
    dnsName: k8s-dev-v1.REDACTED
    hostedZone:
      id: REDACTED
    recordSetTTL: 60
  - name: unversioned
    dnsName: k8s-dev.REDACTED

worker:
  apiEndpointName: versioned

The control plane stack ended up with resource APIEndpointVersionedELB and the update there update was complete pending cleanup. It was just the worker stack that failed to update.
In case anyone ends up here also, currently it seems impossible to kube-aws update a cluster when moving to apiEndpoints without updating the endpoints to a disjointed set. This is because CloudFormation tries to add a Route 53 RecordSet with the same value but different resource Logical ID. Workaround is to ensure endpoint DNS names do not contain existing externalDNSName.

Checking the worker that had to be rolled back (before it disappeared) reveals at least these errors:
Apr 11 02:34:28 ip-10-0-24-56.eu-west-1.compute.internal kubelet-wrapper[2164]: panic: runtime error: invalid memory address or nil pointer dereference
Apr 11 02:34:28 ip-10-0-24-56.eu-west-1.compute.internal kubelet-wrapper[2164]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x969c25]
Apr 11 02:34:28 ip-10-0-24-56.eu-west-1.compute.internal kubelet-wrapper[2164]: goroutine 1 [running]:
Apr 11 02:34:28 ip-10-0-24-56.eu-west-1.compute.internal kubelet-wrapper[2164]: panic(0x3a34940, 0xc420018030)
Apr 11 02:34:28 ip-10-0-24-56.eu-west-1.compute.internal kubelet-wrapper[2164]:         /usr/local/go/src/runtime/panic.go:500 +0x1a1
Apr 11 02:34:28 ip-10-0-24-56.eu-west-1.compute.internal kubelet-wrapper[2164]: k8s.io/kubernetes/vendor/k8s.io/client-go/kubernetes.(*Clientset).Events(0x0, 0x0, 0x0, 0x0, 0x0)
Apr 11 02:34:28 ip-10-0-24-56.eu-west-1.compute.internal kubelet-wrapper[2164]:         <autogenerated>:25 +0x5
Apr 11 02:34:28 ip-10-0-24-56.eu-west-1.compute.internal kubelet-wrapper[2164]: k8s.io/kubernetes/cmd/kubelet/app.makeEventRecorder(0xc420cdb200, 0xc420736780, 0xc42104e8d0, 0x28)
Apr 11 02:34:28 ip-10-0-24-56.eu-west-1.compute.internal kubelet-wrapper[2164]:         /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubelet/app/server.go:348 +0x1ba
Apr 11 02:34:28 ip-10-0-24-56.eu-west-1.compute.internal kubelet-wrapper[2164]: k8s.io/kubernetes/cmd/kubelet/app.run(0xc420cdb200, 0xc420736780, 0x498068, 0xc420e3d900)
Apr 11 02:34:28 ip-10-0-24-56.eu-west-1.compute.internal kubelet-wrapper[2164]:         /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubelet/app/server.go:505 +0x2c2
Apr 11 02:34:28 ip-10-0-24-56.eu-west-1.compute.internal kubelet-wrapper[2164]: k8s.io/kubernetes/cmd/kubelet/app.Run(0xc420cdb200, 0x0, 0xc420e3d910, 0xc420e3d920)
Apr 11 02:34:28 ip-10-0-24-56.eu-west-1.compute.internal kubelet-wrapper[2164]:         /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubelet/app/server.go:285 +0x39
Apr 11 02:34:28 ip-10-0-24-56.eu-west-1.compute.internal kubelet-wrapper[2164]: main.NewKubelet.func1(0xc4208050e0, 0xc420719320, 0x0, 0x11, 0x0, 0x0)
Apr 11 02:34:28 ip-10-0-24-56.eu-west-1.compute.internal kubelet-wrapper[2164]:         /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/hyperkube/kubelet.go:38 +0x33
Apr 11 02:34:28 ip-10-0-24-56.eu-west-1.compute.internal kubelet-wrapper[2164]: main.(*HyperKube).Run(0xc42005b440, 0xc42000c130, 0x11, 0x11, 0x0, 0x0)
Apr 11 02:34:28 ip-10-0-24-56.eu-west-1.compute.internal kubelet-wrapper[2164]:         /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/hyperkube/hyperkube.go:179 +0x468
Apr 11 02:34:28 ip-10-0-24-56.eu-west-1.compute.internal kubelet-wrapper[2164]: main.(*HyperKube).RunToExit(0xc42005b440, 0xc42000c120, 0x12, 0x12)
Apr 11 02:34:28 ip-10-0-24-56.eu-west-1.compute.internal kubelet-wrapper[2164]:         /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/hyperkube/hyperkube.go:189 +0x61
Apr 11 02:34:28 ip-10-0-24-56.eu-west-1.compute.internal kubelet-wrapper[2164]: main.main()
Apr 11 02:34:28 ip-10-0-24-56.eu-west-1.compute.internal kubelet-wrapper[2164]:         /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/hyperkube/main.go:47 +0x853
Apr 11 02:34:28 ip-10-0-24-56.eu-west-1.compute.internal systemd[1]: kubelet.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Apr 11 02:34:28 ip-10-0-24-56.eu-west-1.compute.internal systemd[1]: kubelet.service: Unit entered failed state.
Apr 11 02:34:28 ip-10-0-24-56.eu-west-1.compute.internal systemd[1]: kubelet.service: Failed with result 'exit-code'.
Apr 11 02:34:38 ip-10-0-24-56.eu-west-1.compute.internal systemd[1]: kubelet.service: Service hold-off time over, scheduling restart.
Apr 11 02:34:38 ip-10-0-24-56.eu-west-1.compute.internal systemd[1]: Stopped kubelet.service.

Apr 11 02:34:42 ip-10-0-24-56.eu-west-1.compute.internal systemd[1144]: Failed to open directory /run/user/500/systemd/transient/var-lib-rkt-pods-run-60ca7ae9\x2d4043\x2d4d6c\x2da194\x2d892ac60b8a7c-stage1-rootfs-opt-stage2-hyperkube-rootfs-var-lib-rkt-pods-run-bf768a84\x2dc2fc\x2d49d4\x2d99da\x2de8a273d0638d-stage1-rootfs-opt-stage2-hyperkube-rootfs-usr-lib-os\x2drelease.mount.wants: File name too long
Apr 11 02:34:42 ip-10-0-24-56.eu-west-1.compute.internal systemd[1144]: Failed to open directory /run/user/500/systemd/transient/var-lib-rkt-pods-run-60ca7ae9\x2d4043\x2d4d6c\x2da194\x2d892ac60b8a7c-stage1-rootfs-opt-stage2-hyperkube-rootfs-var-lib-rkt-pods-run-bf768a84\x2dc2fc\x2d49d4\x2d99da\x2de8a273d0638d-stage1-rootfs-opt-stage2-hyperkube-rootfs-usr-lib-os\x2drelease.mount.requires: File name too long
Apr 11 02:34:42 ip-10-0-24-56.eu-west-1.compute.internal systemd[1144]: Failed to open directory /etc/systemd/user/var-lib-rkt-pods-run-60ca7ae9\x2d4043\x2d4d6c\x2da194\x2d892ac60b8a7c-stage1-rootfs-opt-stage2-hyperkube-rootfs-var-lib-rkt-pods-run-bf768a84\x2dc2fc\x2d49d4\x2d99da\x2de8a273d0638d-stage1-rootfs-opt-stage2-hyperkube-rootfs-usr-lib-os\x2drelease.mount.wants: File name too long
Apr 11 02:34:42 ip-10-0-24-56.eu-west-1.compute.internal systemd[1144]: Failed to open directory /etc/systemd/user/var-lib-rkt-pods-run-60ca7ae9\x2d4043\x2d4d6c\x2da194\x2d892ac60b8a7c-stage1-rootfs-opt-stage2-hyperkube-rootfs-var-lib-rkt-pods-run-bf768a84\x2dc2fc\x2d49d4\x2d99da\x2de8a273d0638d-stage1-rootfs-opt-stage2-hyperkube-rootfs-usr-lib-os\x2drelease.mount.requires: File name too long

Apr 11 02:35:11 ip-10-0-24-56.eu-west-1.compute.internal systemd[1144]: Failed to set up mount unit: Invalid argument
Apr 11 02:35:11 ip-10-0-24-56.eu-west-1.compute.internal systemd[1144]: Failed to set up mount unit: Invalid argument
Apr 11 02:35:11 ip-10-0-24-56.eu-west-1.compute.internal systemd[1]: Failed to set up mount unit: Invalid argument
Apr 11 02:35:11 ip-10-0-24-56.eu-west-1.compute.internal systemd[1]: Failed to set up mount unit: Invalid argument
Apr 11 02:35:11 ip-10-0-24-56.eu-west-1.compute.internal systemd[1144]: Failed to set up mount unit: Invalid argument
Apr 11 02:35:11 ip-10-0-24-56.eu-west-1.compute.internal systemd[1]: Failed to set up mount unit: Invalid argument
Apr 11 02:35:11 ip-10-0-24-56.eu-west-1.compute.internal systemd[1144]: Failed to set up mount unit: Invalid argument
Apr 11 02:35:11 ip-10-0-24-56.eu-west-1.compute.internal systemd[1144]: Failed to set up mount unit: Invalid argument
Apr 11 02:35:11 ip-10-0-24-56.eu-west-1.compute.internal systemd[1144]: Failed to set up mount unit: Invalid argument
Apr 11 02:35:11 ip-10-0-24-56.eu-west-1.compute.internal systemd[1144]: Failed to set up mount unit: Invalid argument
Apr 11 02:35:11 ip-10-0-24-56.eu-west-1.compute.internal systemd[1144]: Failed to set up mount unit: Invalid argument

panic in kubelet IMHO should be reported to kubernetes

@redbaron OK, I can report assuming I can give them enough details. All I have right now is the above though.

@c-knowles Thanks for the report.
It definitely seems like my tests were not sufficient and a terrible bug that the various refs throughout kubeconfig and cloud-config-worker become empty when there're 2 or more API endpoints defined in cluster.yaml.
I've already finished the fix in my local branch and I'll update the master branch shortly after more careful testings.

@mumoshu OK, I had noticed the kubeconfig issue somewhere else but not related the two things. Is there anything we can to about switching from default to apiEndpoints not working with kube-aws update? Perhaps I just have to name one of the endpoints default?

@c-knowles I'm still a bit unsure about your case.
So, in other words, you'd like to name your former record set and the newer record set both created for k8s-dev-v1.REDACTED the same in a cloudformation stack? So that kube-aws update can be used to migrate your cluster initially created with externalDNSName, to a newer version with no externalDNSName but apiEndpoints?

@mumoshu exactly yes. Since the validation ensures I cannot use both the old and new settings together the typical path to using this will almost certainly be:
Previous config:
externalDNSName: original-domain

New config:
apiEndpoints:
  - name: original
    dnsName: original-domain
  - name: additional
    dnsName: additional-domain

But doing this causes kube-aws update to not work due to the reason above.

This issue was automatically closed via #529 but please feel free to reopen and add more GitHub issues if necessary.

@c-knowles I'm not very happy to say something like this but will you be ok with the current situation as:

I believe the path mentioned in your comment is already supported as long as kube-aws up and kube-aws update with the respective configuration is executed with the same version of kube-aws
And there's already a general guidance that we recommend users to recreate clusters when upgrading kube-aws

?
Please feel free to point out if I'm missing other aspects of the use-case.

@mumoshu I tried again today but the path I mentioned doesn't work even with the same version of kube-aws.
Steps to reproduce:

Create a cluster using externalDNSName
Migrate to apiEndpoints, at least one of the endpoints will be the same as the old externalDNSName (this is the typical migration path)
Run kube-aws update
CloudFormation tries to create a resource APIEndpoint<API ENDPOINT NAME>ELBRecordSet but the DNS name is already taken by APIEndpointDefaultELBRecordSet from the previous version of the same stack

This is the exact error:
12:25:40 UTC+0800	CREATE_FAILED	AWS::Route53::RecordSet	APIEndpoint<REDACTED>ELBRecordSet	Tried to create resource record set [name='<REDACTED>', type='CNAME'] but it already exists

I can workaround this, my question is just is there anything we could do to ease migration? e.g. make one of the endpoints the "default" (and name the resource APIEndpointDefaultELBRecordSet)
One workaround without creating a whole new cluster is manually change the existing Route 53 entry and just add a single character so the original name can be created again by the stack.

I've updated a dev cluster today with the changes on master and everything seems good this time around, workers all joining correctly. I'm wondering if I should report the panic to kubernetes, I believe the config we were passing was likely to https://:443 which makes very little sense but probably should return a validation error.

@c-knowles Thanks for the detailed feedback!
Making one of endpoints named default would work.
However, If we only intend to support the upgrade path from externalDNSName to apiEndpoints within the same kube-aws version, perhaps we'd better deprecate externalDNSName and pre-populate the first item of apiEndpoints in cluster.yaml, rather than the set of old externalDNSName, createRecordSet, etc?

Hmmm, I'm not sure why it worked for you this time. I guess we made no change around it since then.
Regarding the panic, I think it would be a low-priority thing as I believe there're relatively a small amount of users face the issue. However, the bug is a bug - submitting an issue for it would be very helpful!

@mumoshu I'm all for deprecation if it simplifies kube-aws. In particular I'm not a fan of many layers of configuration/defaulting because I think it's sometimes hard to determine what the value will end up being. So I would agree it's probably better to deprecated externalDNSName and just populate first items of apiEndpoints.
As for why it worked this time, sorry for the confusion as I just meant your fixes on master. I still performed my workaround for the Route 53 RecordSet otherwise the stack always fails to update, I didn't want to waste the time to create a whole new stack just to change one DNS entry 😆

I found the flag related to the panic above is deprecated flag so I filed a separate issue to sort that out in kube-aws. Not sure if switching to non-deprecated flags will still have the same problem but since it's a small case I'm not going to pursue it further right now.


Text composables: Android Basics with Compose

URL of codelab
https://developer.android.com/codelabs/basic-android-kotlin-compose-text-composables?hl=es-419&continue=https%3A%2F%2Fdeveloper.android.com%2Fcourses%2Fpathways%2Fandroid-basics-compose-unit-1-pathway-3%3Fhl%3Des-419%23codelab-https%3A%2F%2Fdeveloper.android.com%2Fcodelabs%2Fbasic-android-kotlin-compose-text-composables#10
In which task and step of the codelab can this issue be found?
class MainActivity : ComponentActivity() {
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContent {
HappyBirthdayTheme {
// A surface container using the 'background' color from the theme
Surface(
modifier = Modifier.fillMaxSize(),
color = MaterialTheme.colors.background
) {
BirthdayGreetingWithText( message = "Happy Birthday Sam!", from = "- from Emma")
}
}
}
}
}
Describe the problem
color = MaterialTheme.colors.background  .... colors=unresolved reference
Steps to reproduce?

Go to...line 16 import com.example.happybirthday.ui.theme.HappyBirthdayTheme  unresolved reference com
line 26  color = MaterialTheme.colors.background  unresolved reference coors
Click on...com.  and  colors
See error...

Versions
Android Studio version: Android Studio Flamingo | 2022.2.1 Patch 1
API version of the emulator:  Pixel_3a_API_33:x86:64
Additional information

Hi @Jaseren
Thank you for writing in about this issue.
The lesson wording, screenshots, and code are all being worked on to address this issue.
The corrected content will be republished again in the near future.
Hope you are enjoying the course!
Regards.


fix pod access svc ip failed

fix pod access svc ip failed

[ ] Make sure you have followed Kube-OVN Code Style.

What type of this PR
Examples of user facing changes:

Features
Bug fixes
Docs
Tests

Which issue(s) this PR fixes:
Fixes #3347
WHAT

🤖 Generated by Copilot at ae9db5c
This pull request updates the ACL actions to support host network policy and fixes a module dependency issue for Kubernetes 1.22. It modifies the files pkg/ovs/ovn-nb-acl.go and go.mod.

🤖 Generated by Copilot at ae9db5c

To fix a k8s issue with slices
We use a replace directive, which suffices
To avoid a breakage
In the API usage
And keep our go.mod free of vices

HOW

🤖 Generated by Copilot at ae9db5c

Override the k8s.io/endpointslice module version to fix compatibility with Kubernetes 1.22 (link)
Change the ACL actions from allow-stateless to allow-related to enable stateful traffic between pods and nodes (link, link)

We need a testcase to cover this

We need a testcase to cover this

1

@changluyi , I have faced the same issue #3347 , waiting for this fix as well. Could you please let us know if this fix will be available in 1.12.0, 1.12.1 and 1.12.2 as well.

@Kovendan03 we will publish a new release 1.12.3 this week to contain this fix


Upgrage 2.15.03 to 2.19.03

Hello
How to install the latest version of Phoscon?
My current version is 2.15.03
http://deconz.dresden-elektronik.de/debian/stable/ offers version 2.19.03

Docker pull


Testnet 135 candidate

Candidate for next testnet
Changes

properly close msgserver on exit (#2539)
add context to msgserver logging
#2490
#2479
#2433
try a range of layers for active set (for hare eligibility) (in case there is one bad layer)

This is no longer relevant as we've retired the 0.1 testnet line


Search Engine Optimization

SEO
Added Search Engine Optimization recourses

This does not have anything to do with design. Design resources only.. thanks


I believe this is a bug for webdriver_manager. I wanted to make sure it wasn't anything from Pylenium's side before dismissing it.
If you can work around the issue by downgrading webdriver_manager, then this bug belongs in their repo

I posted an issue on their repo too, I wanted to do that initially but ended up short on time : https://github.com/SergeyPirogov/webdriver_manager/issues/412

I encountered the same issue.

the fix would be to limit the dependency to before 3.8.1 for webderiver_manager

But isn't this a bug that should be reported to webdriver_manager?

It's possible it's a bug from webdriver_manager, but it's also possible it comes from unexpected use here in pylenium, so idk where's the source of that bug
And idk if downgrading is the only fix, i didn't get much time to investigate tbh

I encountered the same problem webdriver-manager 3.8.0

Downgrade a major version, not minor please. 3.7.x

The workaround is to update "version": "latest" to "version": "" in `pylenium.json.
This problem is probably related to SergeyPirogov/webdriver_manager#386.

Ran into this as well, ended up hacking around by hardcoding the download temporarily.  Version:"" would have been simpler ;)

Had this issue too. @chingc your workaround worked like a charm for me. Thankyou very much.

Finally getting to this myself. I updated webdriver-manager to v3.8.4 and set "version": "" and it worked! This is great news 🎉
For now, I will set "version": "" as the default for pylenium.json and push it up to fix this. Thanks to all of you for helping 🙏🏽

With the https://github.com/SergeyPirogov/webdriver_manager/releases/tag/v3.8.6 release I had again to specify the version as "latest" in the json for it to work.


tsc --build / Project References Feedback & Discussion

Continuation of #3469 "Medium-sized Projects" which has grown too large for GitHub to display comfortably
Please use this thread for feedback, general discussion, questions, or requests for help / "is this a bug" discussions.
Active bugs / PRs in this area:

https://github.com/Microsoft/TypeScript-Handbook/pull/788 Handbook page

Possible next steps:

[ ] (none yet; @andy-ms) Find unloaded downstream projects during rename
[ ] #25376 Infer project references from monorepo tooling package.json files
[ ] #25562 Terse mode output for tsc -b
[ ] (none yet; @weswigham ?) Intermediate build outputs to temp folders
[ ] (none; stretch) Parallelization / worker processes under tsc -b

Other interesting links:

https://github.com/RyanCavanaugh/learn-a - sample project references repo using lerna
https://github.com/RyanCavanaugh/project-references-outfile - sample project references repo using --outFile
https://github.com/Microsoft/TypeScript/issues/3469#issuecomment-341317069 Ryan's outline of what we're doing/did and why
https://github.com/Microsoft/TypeScript/issues/3469#issuecomment-400439520 pre-draft of handbook page
https://github.com/RyanCavanaugh/learn-a/pull/3/files learn-a repo using yarn workspaces!

A couple of things:

watch doesn't currently output an initial build. It waits until the first file change. This differs to non build mode.
How is build order determined? I found I had to order the project references in dependency order to get the build to complete first time without error

@timfish thanks! Re 1 - PR up at #25610. For the other issue, can you sketch out the files you have? The order is supposed to be a simple topological sort that should always yield a correct ordering. I'll try to repro locally in the meantime based on what you've described

@RyanCavanaugh Were you all able to land the cross-project rename for the RC?

@EisenbergEffect we have "upstream" renames (renaming a usage in client affects declarations in shared) but not "downstream" (renames in shared affecting uses in server and client) yet

Gotcha. Any ideas is you'll be able to get the downstream rename in for the final release? or have you already determined that it needs to push out to 3.1 or beyond?

It looks like we can probably get downstream renames for loaded projects in by 3.0 final. Detecting which other projects on your computer need to be loaded to do an "exhaustive downstream rename" is looking dicey - we haven't found any mechanisms that would let us short-circuit that work when the symbol being renamed is exported from the current file.

I am using the API to build Typescript projects, including createWatchProgram and createWatchCompilerHost.

Are these updated to use the new project references and 2) do I need to do anything else in my code other than update the tsconfig.json?

If you're hosting the compiler API, you don't need to do anything new for project references; everything should work as-is. If you want to take advantage of the new --build mode features, the entry point is createSolutionBuilder and you'll need to provide a BuildHost and CompilerHost along with the file timestamp APIs on the latter.
To migrate to using project references in your code itself, updating your tsconfig.json to a) add references and b) add composite: true may be sufficient - if not, you'll see errors informing you what else needs to happen. Writing a comprehensive migration guide has been a difficult task because project setups are so varied - I'm hoping we can find migrations from early adopters as something to point people to, but it's hard to give guidance without seeing specific build layouts.

@RyanCavanaugh ref: Build order.
I'll try to get a repo reproducing this although it won't be until next week.
If I'm just adding all references to a root tsconfig, it should work out the dependency tree from that?

@newtack the compiler source code itself (src/compiler/sys.ts for timestamp APIs and src/compiler/tsc.ts & tsbuild.ts) are good references.
@timfish

If I'm just adding all references to a root tsconfig, it should work out the dependency tree from that?

Correct

Trying this out with the 3.0 RC.

declarationMaps
We've also added support for declaration source maps.
If you enable --declarationMap, you'll be able to use editor features like "Go to Definition" and Rename to transparently navigate and edit code across project boundaries in supported editors.

I tried using this but for some reason "Go to Definition" didn't seem to work. I'm probably doing something wrong, but filed an issue with a minimal reproduction case here: #25662
Will this also handle "Find All References"? If not, is there any way that could be supported?

Based on @RyanCavanaugh and @rosskevin's work to demonstrate Project References within a Lerna-based monorepo (https://github.com/RyanCavanaugh/learn-a, https://github.com/RyanCavanaugh/learn-a/pull/3, https://github.com/Microsoft/TypeScript/issues/3469#issuecomment-401446098), I've forked an made another not-for-merging branch of this repo which uses both Yarn workspaces and Babel for transpilation: https://github.com/RyanCavanaugh/learn-a/pull/4
Hopefully this example helps others learn! I've been working on a monorepo reminiscent of this, and now (I think) I have a better idea of how to configure Project References with it.

I was hoping to get some advice on how to handle path resolutions. My directory currently looks something like this:
- client/
  - assets/
  - views/
  - src/
  - tsconfig.json
  - webpack.config.js
- shared/
  - src/
  - tsconfig.json
- server/
  - src/
  - tsconfig.json

Im struggling to reference the shared directory. I was surprised to learn that the "paths" section along with baseURL doesn't allow for path aliasing like webpack does. So now I'm not sure how to reference the shared directory at all.
For example, the following doesn't work:
// server/src/index.ts
import { mySharedFunc } from "../../shared/src"

// This doesn't work either. "shared" is still shared after compilation
import { mySharedFunc } from "shared"

Because the project compiles down to:
- dist/
  - client/
    - bundle.js
  - shared/
     index.js
  - server/
    - index.js

As you can see, it is invalid to reference the src dir as it doesn't exist after compilation.
Should I restructure my repo somehow? Is it possible to get my path aliasing to work?

I've updated my folder structure to have client, server, and shared under one src directory. This resolved my issue.

@RyanCavanaugh any luck with my code? https://github.com/Microsoft/TypeScript/issues/3469#issuecomment-403978372

I assumed that with "composite": true, "declaration": true would always be forced. However this only seems to apply with tsc --build.
I found this out while doing a normal build (tsc) and not seeing expected definition files.
Is this expected behaviour? It seems counter-intuitive to me.

I found myself using the same technique as the references system in our temporary custom incremental build system (comparing d.ts files) to decide what to rebuild in a non incremental scenario.
However, after finding strange results, I've discovered that the typescript compiler (3.0-RC) do NOT generate deterministically d.ts files.
Indeed, when Unions are involded, the displayed order of those seems to depend on the nodeJS run, which makes to system detect false positives.
ex: {         status: "400";         body: KoStatus;     } | {         status: "200";         body: OkStatus;     }
vs:
{         status: "200";         body: OkStatus;     } | {         status: "400";         body: KoStatus;     }
I think you may have found yourselves those issues and wonder if that's a know/accepted limitation between runs or if I should open an issue (have not found an existing one yet).

Trying it out on the codebase I'm working on, I ran into a lot of Output file '.../_.d.ts' has not been built from source file '.../_.ts' errors. I managed to strip down a repro to almost nothing, and found that when rootDir and outDir are involved as well as a project reference, source files within an individual project that has a dependency project would be compiled in alphabetical order.
The consequence is that an A.ts that imports from B.ts will fail with Output file '.../B.d.ts' has not been built from source file '.../B.ts' but if I rename A.ts to C.ts everything works, as does removing the project reference or not using root/outDir. The problem occurs in 3.0.0-rc as well as next right now.
Repro project: repro.zip

I just tested TS 3 RC project references in a typical npm package setup with included tests. The tests were set up as a TS project referencing the source code TS project.
I surprisingly found that project references do not adhere to the outDir compilerOption of referenced or referencing projects, and that the relative paths for module imports in the compiled referencing project's JS files therefore are wrong; paths point to the TS folder, not the compiled output folder.
A somewhat simplified view of my PoC:

src/
(contains implementation code, TS project REFERENCED by tests)
dist/
(compiled output from src/ - using the outDir compilerOption)
test/
(tests for implementation code in src/, REFERENCING project of src/)

Expected behavior:
Relative paths of module imports in the compiled code for the referencing project should be calculated from:

relative path from outDir compilerOption of referenced project
relative path from outDir compilerOption of referencing project

I put up a small repository to demonstrate the problem:
https://github.com/josundt/tsprojects-outdirproblem

Is <TypeScriptBuildMode> msbuild parameter pllaned for 3.0.1? It's not working in 3.0.0, tsc.exe still runs with --project, even if it's set to true.

@RyanCavanaugh This project thing looks really cool so I decided to give it a try. It hasn't been going so well for me so I thought I'd share my experiences and the code-base in case it's helpful for you guys.
Basically, I have the following folders in my project.

back-end - A Node.JS back-end. References code from shared.
front-end - An Angular 2 front-end. References code from shared.
shared - Vanilla JS code. Doesn't reference any of the other folders.
specs - Some tests written in typescript. References back-end, front-end and shared.

My previous set-up (TypeScript v2.x) was to have a tsconfig.json in the root with the most "forgiving" settings and that would give my IDE all the intellisense etc and cross project renames. I then had a compile.json in back-end (which was a tsconfig file in disguise), and another compile.json in the front-end (same concept). The reason for these files is that I wanted to only compile front-end code (for webpack) or back-end code (for ts-node) separately from the rest.  Needless to say, they wouldn't have any issues picking up the code that I referenced in the shared folder as the paths were all relative etc. This all worked perfectly fine for my work flow and I've been super happy with it (except about a year and a half ago the Angular CLI didn't like it at all, but I'm doing webpack manually and haven't retried since...)
So, I upgraded TypeScript to the @next version and switched over to this whole tsconfig per folder thing and removed the tsconfig.json that was at the root. I'm using the latest version of the VSCode Insiders build, but it doesn't really seem to work at all. In the sense that when I'm looking at some back-end code that references something in the shared folder, it says that thing simply doesn't exist.
The ts-node for my back-end only works if I compile the shared folder and emit to disk, something that I'd rather not have to do because my back-end compiles down to ES6 and my front-end down to ES5. Although I guess it wouldn't be so bad if it emitted declarations only?
Anyway, I've checked it all in, would you mind taking a look and seeing what you think? I wouldn't mind if you wanted to use the repo (or part of it) as some kind of example project. Thanks!
Here's the link: https://github.com/SMH110/Pizza-website/tree/TS_Projects
The previous working setup not using the new projects functionality is at this commit: https://github.com/SMH110/Pizza-website/commit/9548bdd2917439f66e787dcc41e8f99e267e6fce

Hi @RyanCavanaugh! I'm very excited to start using this feature at Canva, so have been having a play. I have run into one bug.
When a project contains a reference to project in a ancestor folder, includes source files must be listed in dependency order.
Sample project: https://github.com/WearyMonkey/ts3-file-order-bug
e.g.
{
  "compilerOptions": {
    "composite": true,
    "declaration": true

  },
  "references": [
    { "path": ".." }
  ],
  "files": [
    "./imports.ts",
    "./exports.ts"
  ]
}

Where imports.ts is:
import { foo } from './exports';

and exports.ts is:
export const foo = 'foo';

Fails with the error
error TS6305: Output file '/Users/toby/dev/ts3_bug/proj1/proj2/exports.d.ts' has not been built from source file '/Users/toby/dev/ts3_bug/proj1/proj2/exports.ts'.

Changing the order of files to:
  "files": [
    "./exports.ts",
    "./imports.ts"
  ]

Succeeds compilation as expected.
I've tried on 3.0-rc, 3.0.1-insiders.20180726, and 3.1.0-dev
Some more general feedback on my experience so far:

OMG so fast.. can't wait!
The relationship between rootDir, outDir and how that affects cross project * imports is quite confusing, and I was only able to find the correct (I think) configuration via trial and error.
I often get has not been built from source file error, and theres no help to find the cause, forcing me to fix via trial and error.

Just as a heads up for everyone: @RyanCavanaugh recently became a father! 🎉 👶 🍼 🎉
He'll be out for a while, but in the meantime if you are running into anything, I'd encourage you to file separate issues and the team will try to take a look. Thanks all!

Could you please provide us sample project for composite projects with sharing typings folder? I try several ways. But nothing work.

This feature looks like holy-grail we were looking for from the very beginning of TS: Since compiling every single .ts into single .js (1:1) was too fragmented (and would had resulted in enormous <link..> tags), we opted for All:1 (single .js file), all our pages use single <link..> that load one jumbo .js file, but browser is getting a lot of code that is not used by specific page (e.g. editor of "Contact" does obtain code for editor of "Email").
Now we will be able to compile shared .ts files into one .js and specific .ts files into another .js file, but still limiting them to reasonable count (like five).
Am I right, that now we can have within single .csproj file (WebAPI) multiple .js targets (accompanied by d.ts) so we can modularize our project ?
I would compare it to analogy: Multitude of .cs files are compiled into DLLs. Up till now we had only one option: to compile all into single EXE.
Do I underestand it correctly ?

I try to create composite project for testing this feature. But this feature still need a lot of implement in term of tooling.
In VS Code insider build, I can use renaming tool to rename "add" function in source project. However, I cannot go to definition of reference function in test project. It navigate to definition file instead.
In VS2017 15.8 Preview 5, it didn't change exported function in test project after I rename "add" function in source project and it also show definition file instead of read source code.
src/utils.ts
export function add(x: number, y: number) {
    return x + y;
}

export function remove(x: number, y: number) {
    return x - y;
}

export const defaultSettings: AppPlatform.Setting = {
    width: 50,
    height: 50
};

test/mainTest.ts
import { add, defaultSettings } from "../src/utils";

export function testAdd(){
    const result = add(5, 4);

    return result;
}

export function getDefaultSetting() {
    return defaultSettings;
}

https://github.com/Soul-Master/typescript3-composite-project

I just tried out the composite project feature.
It all compiles, and runs with ts-node successfully.
However, when I try to "Go to Definition (F12)" with VS Code, it just says "No definition found".
Hovering my cursor over a function of the imported project gives the correct type declarations, though.
TS 3.0.1

VS Code:
Version: 1.25.1
Commit: 1dfc5e557209371715f655691b1235b6b26a06be
Date: 2018-07-11T15:40:20.190Z
Electron: 1.7.12
Chrome: 58.0.3029.110
Node.js: 7.9.0
V8: 5.8.283.38
Architecture: x64

For anyone combing this thread looking for sample setups, Aurelia vNext is now in full swing 😄 You can find our monorepo here: https://github.com/aurelia/aurelia It's setup with TypeScript 3 projects, Lerna 3, Mocha/Chai/Sinon/Karma for unit tests and CircleCI 2.0 for continuous integration.
Big thanks to the TypeScript team for all their hard work. Aurelia vNext is fully committed to TS now and we're loving the new productivity and contribution-pleasant setup we are able to achieve with this new combination of tools. Keep it up!

Feedback 1: I'd like tsc -b -w to show what has been rebuilt in the CLI (similar to tsc -p -w). Currently it just silently rebuilds. -v is too noisy.
Feedback 2: Can you have an official example (as in, GitHub project) that illustrates how one should structure a project, linked from the handbook? Nobody is going find the example projects you are linking in this issue.
Feedback 3: The blog post has a misleading comment // Needed for project references. for composite, when composite is only required for referenced project.
Feedback 4: The blog post does not include --watch in the --build mode section (IMO it's the most useful one). This is inconsistent with the handbook.

Has anybody else had any issues with auto-importing from a project reference? VSCode always wants to import from my dist folder instead of the src folder.
Example:
// Auto imports from here
import { MyFunc } from "../../../../dist/shared";
// Instead of here
import { MyFunc } from "../../../shared";

@drew-y I'm pretty sure this is the intention with --build mode. Because you're essentially referencing a separate TypeScript project (shared in this case), you must import the built output of it.

@ryanstaniforth That does make sense. And I now realize that "../../../../dist/shared" technically works. However "../../../shared" is still preferable, at the very least is terms of aesthetics. So it would be nice if there was still some sort of option to prefer that directory. If possible.

@drew-y I see where you're coming from, I first assumed I should import from the source directory. If that was the case though, what would be the point of TypeScript ensuring dependencies are compiled first. There must be a reason that we're not aware of for this design decision.

@ryanstaniforth Agreed. Although technically, in my case, "../../../shared" is an import from the dist directory.
At least for me, my dist directory directly mirrors my src directory. I.E.:
src/
  client/
  server/
  shared/
dist/
  client/
  server/
  shared/

@drew-y Oh I see, so you actually only have a single project, i.e. only one tsconfig.json? In that case I see you're problem, it shouldn't favour dist.

Is there an example of using this with a multirepo? My development environment is working great while working across projects but each one is broken into separate repos and NPM packages. When I try to build them via CI, the references aren't pulled down so it complains and refuses to build.
It would be great if there was an option that lets me fallback to normal resolution (via node_modules/), but I don't think there is one.
Or is this really only useful for monorepos?
Thank you for the fantastic feature.

I just tried VSCode 1.26.0 with my composite project. With TypeScript 3.0, nothing works. I cannot go to definition or rename method in all referenced files.
However, it works fine when I switch TypeScript to 2.7.1 for unknown reason.

https://github.com/Soul-Master/typescript3-composite-project

I've never seen the need for build before / after hooks for TypeScript until now, but after successfully getting project references working in my project I'm wondering if they're worth considering.
I use NPM scripts to automatically bump the patch version of my packages every time I build, e.g.:
"build": "npm version patch --no-git-tag-version && tsc"

But of course I'm not the one building these packages any more, the TypeScript build mode is.
Some useful hooks for this scenario would be preBuild and postBuild which only run if something has actually changed in the project (which presumably --build can distinguish up front).

If I have project A with a reference to project B and I change a single file in project B, I am seeing the following:
tsc "touches" every file in project B outDir.
tsc "touches" every file in project A outDir after that.
In a big project with two different nodemon processes being watched, this is causing some mayhem. For some reason I actually end up in a loop of restarts (debugging that led me to realize the above), but even beyond that, wouldn't this make a lot more sense and be a lot more gentle?
tsc outputs just the changed file in project B to outDir
tsc does nothing in project A

Having serious problems with only a single depth of referenced projects in vscode:
I am using outDir with a common build folder that mirrors my source folder.
However, vscode seems entirely dependent on the .d.ts files that are generated by tsc -b -w running in the background.

One issue this causes is that often vscode will have a completely wrong code comment: i.e. when I mouse-over in the editor, its showing a comment for the wrong element (as if the internal source mapping typescript is using for the current file is old).

Another issue is that the type system seems to break quickly: i.e. I can reload vscode and all will be well and all the comments will be correct. Then as soon as I type anthing in the file, some of the type information breaks (some of the types from the dependent project will go to any etc.). This happens even when I can F12 directly to the .d.ts file in the build folder (however, I did notice that although the .d.ts code is correct, the color scheme is wrong for the broken type.)

Anyway, at this point typescript 3.0 references seem to work ok with tsc but vscode is not stable enough to use it.

Another issue: It seems the tsc -b -w will not build child (referenced) projects if the main project has an error. I would expect the build tree to execute the dependencies first (and generate their outputs) even if the main project has a problem that prevents the build (assuming the tsconfig references are correct). However, it doesn't seem to be working like this.
With vscode dependent on the outputs of the referenced projects, it is not possible to get code comments and helps to work without manually building the referenced projects.

Found the problem and added a bug report: https://github.com/Microsoft/TypeScript/issues/26703

I attempted to use project references in a monorepo, so I don't know if this should be here or the other discussion. I made a fork of learn-a project https://github.com/panjiesw/learn-a to better describe what I found.
One of monorepo pattern using lerna + NPM is to reference subprojects inter dependencies using NPM's own "package": "file:<location>" instead of their version in package.json. The learn-a fork above have been modified to this structure. Some modified preview:
// root package.json
{
  "dependencies": {
    "@ryancavanaugh/pkg1": "file:packages/pkg1", // direct sub-package reference
    "@ryancavanaugh/pkg2": "file:packages/pkg2", // direct sub-package reference
    "@ryancavanaugh/pkg3": "file:packages/pkg3" // direct sub-package reference
  },
  "devDependencies": {
    "lerna": "^3.2.1",
    "typescript": "^3.0.3"
  }
}

// pkg2 package.json
{
  //<... unmodified stuffs ...>

  "dependencies": {
    "@ryancavanaugh/pkg1": "file:../pkg1" // direct inter sub-package reference
  }
}

This way, even without --hoist option in lerna bootstrap, NPM will hoist all the sub-package dependencies in the root's node_modules. Combine this with the newly addition of TypeScript build mode for project reference, we can have a powerful build orchestration from the root package only, instead of repeating build instruction in every package.
There are issues though, in VSCode v1.26.1, the auto-import and refactoring experience is still not ideal for me. For sub-package that has "references" to other package in their tsconfig (pkg2 and pkg3), somehow, the auto-import always use absolute path, even for something (say, interface) in a sibling .ts file. Even if my "typescript.preferences.importModuleSpecifier" setting is set to "auto" or "relative".
// pkg2/src/baz/interface.ts
export interface Baz {
  message: string;
}

// pkg2/src/baz/index.ts
import { Baz } from '@ryancavanaugh/pkg2/src/baz/interface'; // auto-imported

export default function baz(): Baz {
  return { name: 'baz', message: 'Hello Baz' };
}

This doesn't happen in pkg1 which doesn't reference other sub-package
// pkg1/src/bar/interface.ts
export interface Bar {
  message: string;
}

// pkg1/src/bar/index.ts
import { Bar } from './interface'; // correct auto-import

export default function bar(): Bar {
  return {name: 'bar', message: 'Hello World!'};
}

@RyanCavanaugh I'm pretty sure I'm missing something, or it's a bug, but project references seem not to work when using esnext.  Reproduction based on learn-a yarn workspaces here: #26819.  Result is a dreaded error TS2307: Cannot find module, and removing the module/target from compiler options makes the sample compilable again.

I filed another issue for direct file imports not working in #26823. I will try to back out that difference from the esnext reproduction to see if #26823 is the root cause.
It seems quite unworkable if true that with project references you can only resolve from the index, not direct to a file e.g.
// import * as p1 from '@ryancavanaugh/pkg1'; // works
import { fn } from '@ryancavanaugh/pkg1/foo'; // does not work

export function fn4() {
  // p1.fn();
  fn();
}

@rosskevin The reason you only can resolve from the index is because tsc imports the built files when using project references. This means that your import must be import { fn } from '@ryancavanaugh/pkg1/lib/foo' because the tsconfig for pkg1 has "outDir": "lib" and the package.json has "typings": "lib/index.d.ts". The import would work as you expected if outDir was unset and typings was set to just index.d.ts.
Also, the same problem would occur when importing any package that was built out of source unless a prepackage script changed the location of the build files. In that case the package.json would also have to be changed so that the "main" and "typings" keys did not reference the build directory.

That may be true @WhiteAbeLincoln, but I did not try unsetting outDir because build artifacts would pollute the src directory, and that isn't something that is acceptable to us.

Yeah, I can understand that. The only solution I can think of is to use your current workaround with paths, or use the —listEmittedFiles option and have a script that will parse that output and delete the emissions in the src directory

On Sep 18, 2018, at 08:34, Kevin Ross<EMAIL_ADDRESS>wrote:
That may be true @WhiteAbeLincoln, but I did not try unsetting outDir because build artifacts would pollute the src directory, and that isn't something that is acceptable to us.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

Should tsc -b --watch rebuild projects incrementally?
Checked now (typescript@3.1.rc), on any change in one file tsc building the whole project and all dependent projects.
Tested with two parallel tsc --watch processes - works much faster, rebuilds only changed file.
So, is incremental rebuild not implemented yet or it's a bug?

I think that typescript Types signature exported in d.ts files are not equivalent between nodejs runs (the order of union members can change for example), that may explain some strange rebuild behaviours.

I'm using references and @WearyMonkey's comment helped me, the referenced projects compile file, but the project I'm pointing to aka tsc --build src/server/tsconfig.json --watch was failing to build itself! I added the problematic file to the files:

and now it works! so basically, "include": ["./**/*"] does not work with --build
I tried the dev and rc builds.

I tried to use project references with projects that use simple namespaces. The structure looks like that:
a/
 src/
    a1.ts :  namespace test { export class A { public attr = E.A; } }
    a2.d.ts :  declare namespace test { const enum E { A=1, B=2 } }
 tsconfig.json :
		{
			"compilerOptions": {
				"target": "es5", "types": [], "declaration": true, "composite": true,
				"outDir": "build","declarationDir": "def", "rootDir": "src",
				"declarationMap": true, "sourceMap": true, "noEmitOnError": true,
			}
		}
b/
  src/
   b1.ts : namespace test { let b = new B(); b.attr = E.A; let a = new A(); a.attr = E.B; }
   b2.ts : namespace test { export class B { public attr = E.B; } }
tsconfig.json
		{
			"compilerOptions": {
				"target": "es5", "types": [], "declaration": true, "composite": true,
				"outDir": "build", "declarationDir": "def", "rootDir": "src",
				"declarationMap": true, "sourceMap": true, "noEmitOnError": true,
			},
			"references": [{ "path": "../a" } ]
		}

When I compile, I have the following errors:
>node ..\node_modules\typescript\lib\tsc.js -version
Version 3.2.0-dev.20180926
>node ..\node_modules\typescript\lib\tsc.js  -b b -v
[09:30:24] Projects in this build:
    * a/tsconfig.json
    * b/tsconfig.json

[09:30:24] Project 'a/tsconfig.json' is out of date because output file 'a/build/a1.js' does not exist

[09:30:24] Building project 'C:/testnext/test/a/tsconfig.json'...

[09:30:25] Project 'b/tsconfig.json' is out of date because output file 'b/build/b1.js' does not exist

[09:30:25] Building project 'C:/testnext/test/b/tsconfig.json'...

b/src/b1.ts:4:10 - error TS2304: Cannot find name 'E'.

4 b.attr = E.A;
           ~

b/src/b1.ts:5:14 - error TS2552: Cannot find name 'A'. Did you mean 'a'?

5  let a = new A();
               ~

  b/src/b1.ts:5:6
    5  let a = new A();
           ~
    'a' is declared here.

b/src/b1.ts:6:11 - error TS2304: Cannot find name 'E'.

6  a.attr = E.B;
            ~

b/src/b2.ts:5:16 - error TS2304: Cannot find name 'E'.

5  public attr = E.B;
                 ~

Does this works as expected? Is there a way to make it work? For the moment, I'm using a single project with all sub projects embedded. Problem is, we can reference b classes from a project.

@RyanCavanaugh I think Im experiencing the same issue as @goloveychuk , where using --watch with --build doesnt seem to incrementally build projects; when a change is detected in any of the projects, only that project is rebuilt, but the entire project is rebuilt, rather than just the changes. --verbose doesnt have any info on this; I'm assuming that this is the case because a recompile upon a change using --build --watch takes about the same amount of time as a full compile of that individual project, 7 seconds, but when running tsc --watch without --build on that project, file changes are recompiled in less than a second

I'm seeing the same thing as @goloveychuk and @yonilerner. I tested on a small projekt that just takes 3 seconds to build but I'm seeing that tsc -w -b takes 3 seconds for the initial build and then also 3 seconds for every file change while  tsc -w -p takes 3 seconds for the initial build and then 0 seconds for file changes:
$ tsc -b -w --preserveWatchOutput packages/pkg1

15:44:42] Starting compilation in watch mode...

[15:44:45] Found 0 errors. Watching for file changes.

[15:44:48] File change detected. Starting incremental compilation...

[15:44:50] Found 0 errors. Watching for file changes.

[15:45:12] File change detected. Starting incremental compilation...

[15:45:15] Found 0 errors. Watching for file changes.

vs
$ tsc -w --preserveWatchOutput -p packages/pkg1
[15:45:29] Starting compilation in watch mode...

[15:45:33] Found 0 errors. Watching for file changes.

[15:45:36] File change detected. Starting incremental compilation...

[15:45:36] Found 0 errors. Watching for file changes.

[15:45:38] File change detected. Starting incremental compilation...

[15:45:38] Found 0 errors. Watching for file changes.

Looking for som clarification if this is the intended behaviour of tsc -b -w or if there is a plan to make it  behave like tsc -w -p?

@yonilerner @jonaskello sounds like a bug, though we've had some recent improvements in this area as well that may have addressed it. If you have a repo we can use to test on, we can investigate?

We do not yet do incremental build at program level in tsbuild watch mode (Like tsc -w) and it’s a todo yet to be addressed and that’s the reason you see that much time to build

@sheetalkamat Ah ok, that would explain it. Thanks for the info. Do you have an idea of when that will be implemented?

@sheetalkamat Nice to know that this feature is coming :-). Currently it is not really possible to use tsc -b -w during development for larger projects because it is so much slower than tsc -w -p or ts-loader. Of course I'm also interested to know the roadmap for implementing this.

Hey,
We've run into an issue with project references that we can't quite figure out. I'll try to explain the scenario, it's not entirely straightforward.
These are the relevant parts of our monorepo project structure:
platform/
    client/
        ui/
            tsconfig.json
    shared/
        libs/
            ui/
                vuex-extensions/
                    tsconfig.json
    web/
        thing/
            tsconfig.json

client/ui and web/thing both reference shared/libs/ui/vuex-extensions. All three projects depend on the npm module vuex. The vuex-extensions project contains some vuex helper decorators.
When we import the vuex-extensions shared library into the client/ui project or the web/thing project those projects stop successfully compiling, due to the following reason:
import Vuex from "vuex";
import { createDataStore } from "@shared/vuex-extensions/DataModule";

Adding the import statement for the vuex-extensions changes the import Vuex from "vuex" module resolution. Instead of importing from the "local" node_modules, "vuex" is resolved to the shared lib. And it seems as if the ambient type declarations present in the vuex typings aren't applied in this scenario.
See following screenshots:

We've found a work-around by forcing the typescript compiler to use the correct path for vuex by adding it to the paths section of our tsconfig:
"paths": {
	"@shared/localization-store": [
		"../../shared/libs/ui/localization-store"
	],
	"@shared/vuex-extensions/*": [
		"../../shared/libs/ui/vuex-extensions/dist/*"
	],
	"vuex": ["node_modules/vuex"]
},

I'm not sure if this is a bug in the module resolution, intended behavior, incorrect usage or all three at once. Any help on the matter would be appreciated. Thanks!

@ciddan you can use --traceResolution to see what is going on with the resolution in both cases for further investigating

@yonilerner @jonaskello the --watch behavior change is likely to come in 3.3

I'm also struggling to get project references working as expected with a react lerna monorepo project. The idea is to have a publishable component library as used by app1 and app2 but all in one monorepo and share build tools.
Here's a working example: https://github.com/axelnormand/react-typescript-monorepo
I couldn't get it to work in VS Code without also adding that paths bit in the tsconfigs like @ciddan mentioned.
As also mentioned up thread, VS Code can't compile it out the box as the parent projects need the generated d.ts files, so looking forward to future versions when you don't need declaration and declarationMap on.
My solution for VS Code was to use tasks.json to call compile:watch npm target (which just calls tsc -b). Then when VS Code starts up you have to press ctrl+shift+b to get it going:
"tasks": [
    {
      "type": "npm",
      "script": "compile:watch",
      "problemMatcher": ["$tsc-watch"],
      "isBackground": true,
      "group": {
        "kind": "build",
        "isDefault": true
      }
    }
  ]

So my actual question is why do i need paths as well as references?
Paths is then also needed to plugin in to ts-jest settings
ps Thanks for all the great typescript work so far though as love developing react/react native with it.

Really love project references. Been working to learn and update the pnpjs project to use them. Things are mostly working but I have run into a snag and I am not sure if I'm holding it right.
I have a set of projects that all reference each other (non-recursively) and I am using paths in tsconfig along with project references. That all works IF I specify all the files in an include pattern as outlined above. Should though referenced project's include/files be auto included in the build? Why would I want/need to list the files from referenced projects?

I get "xxx is not in project file list" error when enabling composite:
tsconfig.json
{
  "compilerOptions": { "composite": true, ... },
  "files": [ "src/index.ts" ]
}

Before enabling, this would work as tsc will follow the dependencies in src/index.ts.
After enabling, all files referenced throws an error:
error TS6307: File 'xxx' is not in project file list. Projects must list all files or use an 'include' pattern.

If I have to include src, this causes all test files next to the source files included in the build.
Is this a bug? If so, should I file it separately? 🌷

@unional from what I understand, this is intentional; not a bug:

All implementation files must be matched by an include pattern or listed in the files array. If this constraint is violated, tsc will inform you which files weren’t specified

(from https://www.typescriptlang.org/docs/handbook/project-references.html#composite)
I believe this constraint helps tsc figure out the project dependency tree more quickly/easily. You might find include more convenient than files, and it works best when projects are grouped into folders.

Right now we only use declaration maps to jump to source files instead of declarations. Its working as intended at moment but could change in future. #25844 will handle out of date declaration maps to not crash.

There is a caveat using project references.
Imagine monorepo setup with packages A and B where A is dependent on B. In tsconfig of A there is a reference to B.
Now when i want to use a different version of B in A (that version will be downloaded from npm) i get in a situation where A references version of B that is located in monorepo but actually uses version of B that is downloaded from npm. This can cause many troubles by providing wrong typings info.

@yonilerner @jonaskello the --watch behavior change is likely to come in 3.3

Great! The current behavior is especially annoying when you want to run unit tests in watch mode.
Because tsc --build --watch in combination with project references compiles every file again (instead of just the changed ones) and updates the timestamp on every compiled output file. That leads to running the watched unit tests multiple times because every project in my project references gets updater one after another.

It seems quite unworkable if true that with project references you can only resolve from the index, not direct to a file e.g.

Just wanted to share that this seems like a good thing that you can't use things that weren't made "public." I think of it as an enforcement of no-submodule-imports from ts-lint: https://github.com/palantir/tslint/issues/3051
The ability to prevent coupling future proofs a team like mine without having to resort to separate repos or microservices with brittle CI/CD pipelines.
Anyway, I'm really looking forward to these features. Great work @RyanCavanaugh and the team! TypeScript makes coming to work every day an absolute joy! :)

How to make tsc --build always emit output files, whether or not there is an error?
It is different from tsc at this point.
Tried to set noEmitOnError: false, but seems not work

One big blocker we still have for build mode is the requirement that declarations are turned on.
In our monorepo we have multiple "toplevel" packages (several backends, several frontends) that are not libraries, and they share multiple dependencies.
The backends and frontends cannot have declarations turned on. It would be a huge amount of unnecessary busy work to do it, to eliminate a bunch of TS4058 s i.e.
Return type of exported function has or is using name 'X' from external module "/path/to/external/module" but cannot be named.

This in turn means we cannot add them to the composite project. As a result, they somehow have to be built and watched separately from "build mode" which complicates the whole setup and negates the benefits for large chunks of code.

With the latest version v3.3.3 we still have the issue https://github.com/Microsoft/TypeScript/issues/25600#issuecomment-439401123
tsc --build --watch builds every project (all files within that project) where I changed a single file instead of just that one file. Is that now the expected behaviour?

Using this in Jest now while migrating from Flow to TypeScript. My only real complaint constructive feedback is that it's slow (emitDeclarationOnly currently takes a full minute on my beefy 15" 2018 MBP, and we're not done migrating so it'll keep increasing) so it'd be really nice if TSC provided some sort of progress meter. Some sort of 213123/123123123123 thing that updated whilst it was building would be awesome (similar to progress in webpack or yarn).
Incremental builds are fast (and watch works great), though!

Seems like 7b290fdbd4e892379c3ab3ed46d7b41f15a31a2b introduced this behavior to update all last modified timestamps of all output files of a project even if the contents didn’t change. I don’t quite understand the explanation in the commit message and why this is necessary.
Running tsc --build -w --listEmittedFiles confirms this assumption, after changing a single file it lists all files within the project as being emitted.

Enforcing "noEmitOnError" brings a lot of inconvenience, is this caveats temporary?
E.g. we have to create separate tsconfig files for incremental building that disables options like noUnusedLocals and noUnusedParameters.
Also this makes refactoring harder because we are expecting a lot of intermediate errors. And a dependant project will be out of sync with its dependency unless we fix all the errors in the dependency first.

Seems like 7b290fd introduced this behavior to update all last modified timestamps of all output files of a project even if the contents didn’t change. I don’t quite understand the explanation in the commit message and why this is necessary.
Running tsc --build -w --listEmittedFiles confirms this assumption, after changing a single file it lists all files within the project as being emitted.

No explanation about this folks?

update all last modified timestamps of all output files of a project even if the contents didn’t change

Let's say the following sequence occurs on a solution containing projects core and ui, where ui depends on core:

1 PM Clean build
2 PM Modify core implementation (without modifying types)
3 PM Build the entire solution with tsc --build
4 PM Run tsc --build ui

Now, the 4 PM build should do nothing, because at 3 PM we built the entire solution, and nothing has changed.
The question is, how does that happen?
The timestamps on ui, naively, would indicate that its outputs are older than its inputs, because the 1 PM ui output is older than the changed 2 PM core build output upon which it depends.
But when we did the 2 PM build, we realized that ui is "still up to date" because core's types didn't change. At this point we effectively choose to do an instant free build by moving ui's timestamps forward.
It's identical to us doing a real build of ui, it just happens that we already know what the outputs will be because we have realized the upstream inputs are unchanged.

@RyanCavanaugh I think the point is moreso that we shouldn't be updating the timestamps - while doing so makes our recordkeeping easier, it causes naive tooling watching those files to trigger when they don't necessarily need to (since the contents haven't changed).

@weswigham tsc -b depends on timestamps and is soley based on timestamps of inputs and outputs so not updating it is just wrong.

@sheetalkamat How so? Doesn't it just mean that rather than relying solely on a file's mtime, we should be looking at the mtime(s) of the project(s) it depends on? If those projects mtime's are newer, then we determined we didn't need to reemit during the last build, and thus still don't need to?

@weswigham What if you cancelled your build after project reference was done building. Now on next turn you don't know if you skipped build or were not even gotten chance to determine if build is needed?

Mmmm... you could avoid that issue if you transactionally update all outputs at once (or nothing). Eg, you output to a cache dir, and only once done do you then rename the cache dir to the output dir. Or you can write a lockfile that you only delete when fully done.

I think the point is moreso that we shouldn't be updating the timestamps - while doing so makes our recordkeeping easier, it causes naive tooling watching those files to trigger when they don't necessarily need to (since the contents haven't changed).

The fact of relying on timestamps is irrelevant to the downstream tool impact scenario. If our compiler could execute in 1ms and build any project instantly, a solution build would absolutely write these files unconditionally, even in the presence of unchanged output file contents. We are now sometimes able to do exactly that.

If those projects mtime's are newer, then we determined we didn't need to reemit during the last build, and thus still don't need to?

All of the timestamp-update-only code is dependent on noticing that the .d.ts content of an upstream project ended up being identical to what was already on disk while we were emitting it. It's not possible to replicate this behavior between invocations by examining timestamps without a sidecar status file, which was something we explicitly scoped out of non-incremental build.

I'm running into issues when converting our current code base to project references, because transpiled imports don't respect the outDir of the referenced projects. In this issue this has been declared as "working as intended" but I don't see us moving anywhere near composite projects as long as we can't get them working reliably.
I created an example repo that demonstrates our issue.
There are two projects in it a and b. a is supposed to build the source code in a way that a/index.js can be run with node. Since the build output does not rewrite imports and does not restructure the build output similar to how non-composite projects are resolved (i.e. re-creating the folder structure to make imports work) a/index.js can not be executed.
What is the use case for these project references? I was hoping that they would enable us to reference various TypeScript projects without worrying where they are located.

Can we possibly get an option to emit on error for project references?
Now everyday when I write code, whose intermediate state has unused errors 95% of the time, I can't run or debug the code because nothing is emitted.
http://www.typescriptlang.org/docs/handbook/project-references.html#caveats

if one of your out-of-date dependencies had a new error, you’d only see it once because a subsequent build would skip building the now up-to-date project.

Can't you mark that up-to-date project with some other flag to indicate it's not error free, and rebuild it on change?
My day-to-day pain is if I'm working on a project a that depends on b, c, d, even if I'm not making changes to b/c/d that would cause the above scenario, a never emits when I'm writing code. I literally had to run both tsc -b and tsc -p to get project a to emit.

Can we add a option to should the build progress
like:  (12/50) abc.ts compiling...

A limitation we have recently discovered: "Find all references" and "Rename symbol" don't work across referenced projects, see https://github.com/microsoft/TypeScript/issues/30823. Posting the link here to make the other issue easier to find for others.
I apologize for a possibly unwanted notification.

When specifying references, if the references do not resolve to a folder or tsconfig, the process to terminate with error.
{
  "references": [{
    "path": "not-exist-location"
  }]
}

Currently, the build will succeed.
This makes identifying the root cause of https://github.com/kulshekhar/ts-jest/issues/766 very hard to do.
(I'm suspecting there are some relative path issues for ts-jest running in a monorepo)

I have a codebase with 34 sub-projects and I've noticed that there are times where watch will claim to have re-checked and recompiled code but the previous errors still show up.

tsc -b -w /*sub-projects-here*/
tsc complains about error at file X line Y
Fix problem
tsc detects change
tsc claims to have re-checked, re-emitted
tsc STILL complains about error at file X line Y
Keep making changes and scratching head until tired
Ctrl+C to close tsc
Delete output folder
tsc -b /*sub-projects-here*/
Builds successfully

This case happens pretty often to me.
And constantly rebuilding isn't really feasible because a complete rebuild takes 9 minutes.

I still don't understand why the project references functionality is requiring me to build .d.ts files that aren't actually used for anything else just to get tsc -b to work for other packages in a working monorepo setup, when type inference in VS Code works fine without ever creating those .d.ts files. I feel like there's some total disconnect here in intended functionality that I just don't get.

hi, I'm using Program in a monorepo. I got the error d.tshas not been built from source file. Are there any ways to solve it. I would love to make my Program emit behavior to be similar to tsc -b.

If you're hosting the compiler API, you don't need to do anything new for project references; everything should work as-is. If you want to take advantage of the new --build mode features, the entry point is createSolutionBuilder and you'll need to provide a BuildHost and CompilerHost along with the file timestamp APIs on the latter.
To migrate to using project references in your code itself, updating your tsconfig.json to a) add references and b) add composite: true may be sufficient - if not, you'll see errors informing you what else needs to happen. Writing a comprehensive migration guide has been a difficult task because project setups are so varied - I'm hoping we can find migrations from early adopters as something to point people to, but it's hard to give guidance without seeing specific build layouts.

@RyanCavanaugh I see a few issues with the current typing of createSolutionBuilder
function createSolutionBuilder<T extends BuilderProgram>(host: SolutionBuilderHost<T>, rootNames: readonly string[], defaultOptions: BuildOptions): SolutionBuilder<T>;

The name rootNames input is very misleading. This input should be something like tsConfigPaths. Because this works:

const solution = ts.createSolutionBuilder(solutionBuilderHost, ['./tsconfig.json'], {
   incremental: true,
})

but this doesn't:
const solution = ts.createSolutionBuilder(solutionBuilderHost, compilerOptions.fileNames, {
   incremental: true,
})

defaultOptions with interface BuildOptions which has the last property is

[option: string]: CompilerOptionsValue | undefined;

This is unclear what else can be the valid build options

Just to check whether what I'm seeing is expected... I tried converting my medium-size (20k lines) project to use references and tsc -b -w, on the assumption that it'd help the compiler do fast incremental (watch) builds. However, it ended up significantly slowing rebuilds when I edit one of the modules that a lot of other modules depend on—it looks like it unconditionally rebuilds all the dependents in that case, leading to tens-of-seconds rebuilds in some cases. This was a rather big regression compared to just running tsc -w without references (where rebuilds are sub-second). Is that just how this feature works, or am I doing something wrong?

@RyanCavanaugh no offense but is anyone using project references in production? I am trying to setup a monorepo with this feature and I just don't get there. I looked at many examples but it's just not working. Now, I am hanging at the same bug described earlier: https://github.com/microsoft/TypeScript/issues/25600#issuecomment-406932881
It's quite a tedious and frustrating experience I am not used to TS

@desmap I've had so many problems with a monorepo it's loony. Nearly constant, all sorts of issues referencing projects, with intellisense. You have to really, really, spend time and hunt down a lot of weird advice, and at best you have it half working.
Wish I'd saved a few more links to help you out. Only thing on the top of my head is look for the "multiple inheritance" tsconfig structure blog post, and typescript-monorepo-toolkit helps manage refs.

@RyanCavanaugh has anyone suggested that TypeScript should provide project information of build error? Occasionally we need to know which referenced project, during build, is causing the build error due to different tsconfig.json.
A common scenario:
We have browser projects and Node.js projects using some third-party modules that requires some extra types to work. Most of the projects are configured correctly, but some of them are misconfigured. Now it's not that straightforward to find out the misconfigured ones.

@desmap  I'm doing this in multiple of my projects, and I don't have much issue. Here are some of these projects:
https://github.com/unional/standard-log
https://github.com/mocktomata/mocktomata

Leaving some constructive feedback: I spent most of the day trying to set up monorepo using yarn workspaces ad tsc- b and finally gave up and went back to the normal build mode. Here's what I was trying to achieve:
3 packages, each with src and tsconfig.json compiling into */build directories

server: No dependents
frontend: No dependents
common: referred to by server and frontend

(I was trying to use yarn workspaces, as in RyanCavanaugh/learn-a#3)
The first wall I hit was when trying to do cross-package imports using import { X } from '@project/common/x'. (Which I think is a very common approach). I gave up after seeing #10866 et al, not wanting to add another build dep just for this.
Since TS won't rewrite the imports, I resigned myself to import { X } from '@project/common/build/x'. But then that led to TS6305: Output file '/common/build/x.d.ts' has not been built from source file '/common/src/x.ts'. I could get tsc happy by instead using  import { X } from '@project/common/src/x', but of course that failed at runtime and it was back to square one. (I suspect this is related to #25864)
Curiously, even without project references "go to definition" is able to take me to the source file in common from call sites in server. Not sure if this is due to source-maps + symlinks interacting fortuitously, but that solves one major goal from this effort.
Incremental builds are still challenging, but we just run 3 different build processes for now. It's going to get painful as we add more packages though since we need to manually watch all build directories for nodemon to reload the dev server after any changes.

Some projects extensively use relative imports. These imports unambiguously resolve to another file on disk. Paths like ../../core/utils/otherMod would be common to find, though flatter directory structures are usually preferred in these repos.
... While the directory structure implies a project stucture, it's not necessarily definitive.

-- https://github.com/microsoft/TypeScript/issues/3469#issuecomment-341317069
The emergent disappointment of compilerOptions.paths seems to me to be motivated by this same above scenario that the Project References proposal acknowledged.  I don't think the proposal was aiming to directly address the implication of project structure but this is something we are still in desperate search for.
Typescript has enabled (for some) the development of larger codebases than ever before practical.  Now that we're using javascript ecosystem for really serious applications, we're running into problems that don't have nice answers yet.  We HATE having to publish multiple packages and npm link nonsense, but we like modularity.
I'm not exactly sure why we think we need it, but we do think we need to make clear in the code & import statements that a module boundary exists here.  Relative paths are ambiguous in that regard, ugly, and relative - which makes refactoring or copy & paste harder since relative paths have to be adjusted.
If paths can't solve this problem because ...

We are not going to implement features, even under a commandline flag, that imply changing JS semantics by rewriting it during emit

... then is there some other way to make it obvious - even in situations without semantic highlighting support - that this reference is a project reference?
Even if we use paths to alias for tsc, and then we use monorepo package linking or whatever to ALSO ensure that the files will be resolvable (though differently) at runtime, it still doesn't help with the auto-import statements emitting relative paths.  There ought to be exactly one correct and normalized specifier that developers and the language service can agree on.
I guess this doesn't need to be fully solved by typescript, but it seems to me that it will require some kind of support from typescript to make it smooth & robust.
Is there a way to decouple it in an acceptable way that will continue to allow typescript to empower ever larger codebases?

Some projects extensively use relative imports. These imports unambiguously resolve to another file on disk. Paths like ../../core/utils/otherMod would be common to find, though flatter directory structures are usually preferred in these repos.
... While the directory structure implies a project stucture, it's not necessarily definitive.

-- https://github.com/microsoft/TypeScript/issues/3469#issuecomment-341317069
The emergent disappointment of compilerOptions.paths seems to me to be motivated by this same above scenario that the Project References proposal acknowledged.  I don't think the proposal was aiming to directly address the implication of project structure but this is something we are still in desperate search for.
Typescript has enabled (for some) the development of larger codebases than ever before practical.  Now that we're using javascript ecosystem for really serious applications, we're running into problems that don't have nice answers yet.  We HATE having to publish multiple packages and npm link nonsense, but we like modularity.
I'm not exactly sure why we think we need it, but we do think we need to make clear in the code & import statements that a module boundary exists here.  Relative paths are ambiguous in that regard, ugly, and relative - which makes refactoring or copy & paste harder since relative paths have to be adjusted.
If paths can't solve this problem because ...

We are not going to implement features, even under a commandline flag, that imply changing JS semantics by rewriting it during emit

... then is there some other way to make it obvious - even in situations without semantic highlighting support - that this reference is a project reference?
Even if we use paths to alias for tsc, and then we use monorepo package linking or whatever to ALSO ensure that the files will be resolvable (though differently) at runtime, it still doesn't help with the auto-import statements emitting relative paths.  There ought to be exactly one correct and normalized specifier that developers and the language service can agree on.
I guess this doesn't need to be fully solved by typescript, but it seems to me that it will require some kind of support from typescript to make it smooth & robust.
Is there a way to decouple it in an acceptable way that will continue to allow typescript to empower ever larger codebases?

With tsc build & project refs I'm having a hard time implementing something that doesn't double the build time in large web repo. Below is a simpler model showing the majority of imports / shared code between the two are just interfaces that help define constructor parameters, JSON responses, etc.
tsconfig.json (project refs to ModuleA & B)
ModuleA
 -a.ts (imports ITestInterface from "../moduleB/a")
 -tsconfig.json
ModuleB
 -a.ts (references classes exported by b & c in ModuleB)
 -b.ts
 -c.ts
 -tsconfig.json

The --diagnostics & --traceResolution show me that ModuleA/a.ts scans and pulls in all files within ModuleB & doesn't just stop at the imported interface from ModuleB/a.ts. Wonder why tsc doesn't just stop at that point when building ModuleA.
Think I'll need to force the build process to run two tsc processes in parallel. Or just go back to one tsconfig & share npm dependencies (tricky).

--build doesn't work with path aliases (or specifically the tools that do the path remapping). Typically I do this with an NPM postbuild hook, which obviously doesn't fire. I am considering hacky alternatives atm, and I'm not enjoying the prospect at all.

I just spent a while debugging an issue that goes away 100% when using typescript 4.6.3. The problem versions were 4.5.5 and 4.4.4. Written up at https://gist.github.com/SgtPooki/b7a35516e0d0e423cac0f94a03e3a96f just in case it's helpful to anyone.

So it seems like I have to add paths despite a package being installed as a dependency via pnpm workspaces for it to be correctly picked up as a project reference. Is this correct? I seem to have another project with Yarn 1.x there I didn't need to do this... Is this a problem specific to pnpm?

Haven't written here in a while but thought I'd chime where I lended at (as I was originally subscribed to this issue and I'm unsubscribing here)
tsc --build has not been useful / mised the mark for us. We expected something that would let us rely on package.json to build packages in a monorepo. Despite the feature requests, that didn't happen and it didn't look like it was going to happen. After a brief detour with Bazel, we finally found TurboRepo (https://turborepo.com/) which solved the build optimisation issues for us. As a result, we no longer have any use of --build

I'm also using turbo and not using tsc -b anymore.
However, project reference is still needed to make IDE happy and able to update language server without rebuild. I have tried using paths as a replacement of references but could not get the same experience out of it.

We have an issue in our project, and we'd love to hear your thoughts, and how we could fix it. The codebase is large, and we have many circular dependencies between referenced projects. The only way we can go around it at the moment (till we move things around to fix things) is to add the files that are needed in both tsconfig.include projects but this has a downside, when TSC builds both projects (or more), there are the same files in multiple referenced projects, and when ts server crashes (which it would due to the size of the project), we get a bunch of errors that the files are not built from the source.
Not sure how things work under the hood but what's your suggestion to fix this?

@aminpaks I think it is unavoidable to fix the circular dependencies first. This can be done by extracting common code into higher-order modules or by using something like the register pattern.

@aminpaks I think it is unavoidable to fix the circular dependencies first. This can be done by extracting common code into higher-order modules or by using something like the register pattern.

Right. I agree, and we're working on fixing the circular dependency issue.

@Bessonov in your experience how much incremental build sped up vscode for your team?

@aminpaks
Do you really mean VSCode? I found VSCode to be very unreliable for error checking because it only checks opened files. However, I believe this is due to optimizations, and I haven't looked into it for a long time.
I haven't used project references and incremental build for years, and my experience may no longer be relevant. But we encountered all types of bugs. Ultimately, we switched to webpack, ts-loader, and fork-ts-checker-webpack-plugin, and it was the only setup that worked more or less reliably. The chain is still extreme slow. Now, I am waiting for the stabilization of Bun workspaces to test it with plain tsc --build --incremental --watch + project references for error checking.
By the way, you may be interested in https://github.com/microsoft/TypeScript/issues/33685. Found it by accident :smile:

@aminpaks Do you really mean VSCode? I found VSCode to be very unreliable for error checking because it only checks opened files. However, I believe this is due to optimizations, and I haven't looked into it for a long time.
I haven't used project references and incremental build for years, and my experience may no longer be relevant. But we encountered all types of bugs. Ultimately, we switched to webpack, ts-loader, and fork-ts-checker-webpack-plugin, and it was the only setup that worked more or less reliably. The chain is still extreme slow. Now, I am waiting for the stabilization of Bun workspaces to test it with plain tsc --build --incremental --watch + project references for error checking.

We're tying to improve the DX, and are focused on making the types in our projects faster. Overall I find the incremental build not working well in our project. We have issue in our tsconfigs that I'm addressing, and then removing the circular dependencies but I'm not sure that'll solve the issue we're seeing with incremental build in our projects.
We still see the "Configure excludes" in vscode, and ts server crashes. I was expecting when I open a file in vscode but I'm not touching it it wouldn't rebuild it, but it seems that's not the case. ts server loads all the files, and eventually crashes due to too many files being loaded.

By the way, you may be interested in #33685. Found it by accident 😄

Yeah that's great if happens, found that the other day.

Also something that we discovered is that we can't exclude test files from referenced projects but they contribute a large amount of time to compilation time.
For example Project A that depends on Project B, it's true that Project A can work without ProjectB's test files, but we couldn't find anyway to exclude them from the projects.

Also something that we discovered is that we can't exclude test files from referenced projects but they contribute a large amount of time to compilation time
That you can do, by referencing the tsconfig directly:

// project a tsconfig.build.json
{
  "references": [{
    "path": "../package-b/tsconfig.build.json"
  }]
}

Does anyone know how to extend jest in referenced projects. We have some custom extensions but I can't add them in tsconfig.include as it's gonna be added in all referenced projects, and it breaks the incremental build. I don't wanna add them to each test either, is there a way to make it available in globals, and just add the extensions in the jest setup?

@RyanCavanaugh
Here are our thoughts when we used the project references at Shopify:
Configuring composite projects is not hard but it adds an overhead, and it becomes difficult when we try to split projects without moving files. On the other hand the referenced projects still break for the entry points that imports all files. For some reasons TS compiler imports files as soon as it finds an import statement even though they are excluded in the tsconfig.exclude, I think this should be a compilation error instead of magically making it work. For example we have a project entry A that acts as the foundation, and imports all the other sections of the app, let's say 28 composite projects, as soon as we open any file from within the project A, TS eagerly imports all files from 28 projects. This means TS compiler has to load all files in these projects, parse them, generate type declaration, and run type check on them. This process is extremely slow, in our case it takes about 2 minutes.
We think a different approach might work better for the type checking in vscode. You folks know the TS compiler inside out, and you have enough context to tell if this idea is doable. What if we keep the d.ts files with the source code, and commit them to the repository, then the type checker reads them from the file system, and on another process we run the type declaration emitter, if there is any mismatch between the source, and the type declaration the compiler can issue an error. This allows to speed up the initial load, allow a better editor DX, and also we might be able to make this process multi-threaded.

For some reasons TS compiler imports files as soon as it finds an import statement even though they are excluded in the tsconfig.exclude

This is working as intended and is documented quite prominently: https://www.typescriptlang.org/tsconfig#exclude

Important: exclude only changes which files are included as a result of the include setting. A file specified by exclude can still become part of your codebase due to an import statement in your code, a types inclusion, a /// <reference directive, or being specified in the files list.

It is not a mechanism that prevents a file from being included in the codebase - it simply changes what the include setting finds.

Differently said: exclude only filters include, nothing more.

Thanks for the feedback!

For some reasons TS compiler imports files as soon as it finds an import statement even though they are excluded in the tsconfig.exclude

exclude is only a filter on include, the docs are very explicit about this

TS eagerly imports all files from 28 projects. This means TS compiler has to load all files in these projects, parse them, generate type declaration, and run type check on them

It sounds like you're talking about in the editor? You can turn on the disableSourceOfProjectReferenceRedirect option to disable this loading.

What if we keep the d.ts files with the source code, and commit them to the repository, then the type checker reads them from the file system...

This is more or less what composite projects / project references do already, especially under incremental. It sounds like you might have some configuration issues based on what you've provided.

Thanks for your quick feedback.

exclude is only a filter on include, the docs are very explicit about this

True, but this behaviour easily leads to misconfiguration, incorrect, and unintended results. It would make our jobs much easier to detect the root cause by the compiler.

This is more or less what composite projects / project references do already, especially under incremental. It sounds like you might have some configuration issues based on what you've provided.

That's possible, we recently started looking into our configs. Is there a way for us to warm up the TS Server (maybe generate the d.ts files so we reduce the initial load times in vscode?

@RyanCavanaugh

It sounds like you're talking about in the editor? You can turn on the disableSourceOfProjectReferenceRedirect option to disable this loading.

Yes, we're working on improving our developer experience, and reduce the vscode memory consumption. With our project, vscode very much is unusable. Getting IntelliSense, or code-completion in a file takes up to a minute in some places, and eventually the TS server crashes.
I tried turning on disableSourceOfProjectReferenceRedirect setting, and the result is odd but the memory consumption is much better. When a file is opened I see lots of Output file 'built/path/to/file.d.ts' has not been built from 'path/to/file.ts' ts(6305) error on imports from referenced projects.
Not sure how it's supposed to work but I was expecting that the compiler builds the referenced projects for the imported files (in the background), and then free the memory, specially cause these imports are not being actively open, or used directly in the editor. Would you confirm is that error the intended behaviour?

disableSourceOfProjectReferenceRedirect options means that instead of picking up source from the referenced project we pick built d.ts files just like we do during compilation. This means that you would need to keep the output files of referenced project upto date. We do not do background in memory build for projects

@sheetalkamat
Thanks for the reply,
Is the expectation to run the TSC in a watch mode to compile the referenced projects separately? I don't know how efficient that would be. It seems none of the available options works well for our project.

I'm using a npm workspace setup, where my local workspaces are symlinked to the root node_modules. An interesting behavior I notice when using project reference in this setup is tsc doesn't seem to find the .d.ts declaration of a referenced project properly:
references: [
  {path: '../path/to/packageA/tsconfig.json'}
]

// importing a JS file
import {A} from '@my-scope/packageA/src/SomeJSModule'
// complains: can't not find the declaration file for ..., you can install '@types/my_scope_packageA'

However, the error goes away if I import this js file using a relative path.
The workaround I come up with is just using path to map the absolute imports to relative:
paths: {
  "@my-scope/packageA/*": ["../path/to/packageA/*"]
}

@chbdetta that would seem to indicate something isn't set up right with the symlinks. In general for a monorepo workspace, tsc -p even without project references should be resolving to packageA's .d.ts

Also you can use --traceResolution to see where typescript is looking for files to resolve

@RyanCavanaugh @sheetalkamat Here's a reproducing repo: https://github.com/chbdetta/ts-project-reference-playground

@chbdetta I don't get any errors running that

@RyanCavanaugh I left a @ts-expect-error on the error line

@chbdetta issue there is that js files discovered with node_modules differ from how ts are treated. The js file take into account maxNodeModuleJsDepth which defaults to 0 in the tsconfig, You would want to set that to different value to include js file. The current setup will give you error even if project B did not have reference to project A

@sheetalkamat does increasing maxNodeModuleJsDepth have some performance impact? And will it change the type resolution for non-workspace packages (e.g. it'll try to infer from a .js file of a non-workspace package instead of asking for installing a @type/* package?)

https://www.typescriptlang.org/tsconfig#maxNodeModuleJsDepth

Hey all! I tried searching through GitHub issues to see if there's a reason this wouldn't work or hasn't been done, but I've got a project with 8 libs and 3 apps using project references. It follows this basic structure:
tsconfig.base.json
workspaces/
  apps/
    app-1/
      tsconfig.json
    app-2/
      tsconfig.json
    .../
  libs/
    lib-1/
      tsconfig.json
    lib-2/
      tsconfig.json
    .../

There's a little more complexity, but that's the basic idea. However, I have a ton of repetition between all my tsconfig.json files that basically all look like this:
{
  "extends": ["../../../tsconfig.base.json"],
  "references": [
    {
      "path": "../../libs/lib-1"
    },
    {
      "path": "../../libs/lib-2"
    },
    // ...
  ]
}

It would be super nice if we could use a wildcard/glob in situations like this, where I could simply have all of my tsconfig.json files look like this:
{
  "extends": ["../../../tsconfig.base.json"],
  "references": ["../../libs/*"]
}

It also makes it way easier to add new libs/apps in the future to the monorepo. Also, I did try this and it does indeed not work:
error TS5083: Cannot read file '/Users/ebrouwer.dev/source/taxbit/belt/workspaces/libs/*'.

I'm happy to try and write a PR if people think this is possible and useful! Also happy to make a separate issue for this specifically, but this seemed like a fairly active ticket and the right place to start the discussion. Thanks!

@eabrouwer3 yes, a separate issue on that would be great. Thanks!

https://github.com/microsoft/TypeScript/issues/56279 in case anyone's interested.

--clean confuses me. It only deletes files that would be emitted, but that seems almost exactly the opposite of what would be useful. How can I clean out "leftover" output files after renaming or removing an input file, without trashing the incremental build behavior?

How can I clean out "leftover" output files after renaming or removing an input file

Run --clean before deleting the file. Once you've removed the input file, tsc has no idea whether it's a file you made by hand, or a file from a previous invocation.

Run --clean before deleting the file. Once you've removed the input file, tsc has no idea whether it's a file you made by hand, or a file from a previous invocation.

I agree with @raihle - I'm not sure how it is useful. On the other hand, cleaning up stale files inside a CI pipeline with cached build artifacts could speed up the pipeline without worrying about leftovers. Perhaps something like .tscleanignore to let tsc know about files created manually? (I'm not sure if I have any, except .tsbuildinfo inside the build folder.)

We're really not interested in breaking into jail with a feature that can delete files that aren't ones we would have normally overwritten anyway. The last thing I want to deal with is "tsc deleted all my source code because I misconfigured it, why would you even ship this??"  There are many, many, many dev tools capable of creating outputs that don't have built-in means to delete those outputs, and tsc isn't particularly unique here. Maybe shipping --clean in the first place was a mistake but I don't want to spend good money after bad if that's the case.

Thanks for taking the time to share your thoughts, @RyanCavanaugh! It's truly appreciated!
Personally, I can follow the 'rm'-way. The challenge lies in matching the configuration of tsc, such as rootDir and outDir, and different artifacts like `*.(js|mjs|d.ts|d.mts|js.map|mjs.map|...).

I'm working on improving TS config for Vite templates, and I've come to the conclusion that using project references with files: [] in the main config and no composite is the best setup.
There is one issue is that a lot of people are used to run tsc or tsc --noEmit at the root of the project, and this silently does nothing. Is there plan to:

make it an error in that case with "You probably want tsc -b"
default to build mode when the tsconfig is clearly made to only be used with it


Complete almost all tests for RS Connect board

Adding small fixes to NodeJS callbacks and update RSConnect board with last changes.
TODO: there is a bug with custom metadata test on RSConnect and some problems with versions.

Going to merge this one to avoid future merge conflicts.


How to use custom inputs?

Is it possible to test out any of the tasks on custom input?
Say, for the Game24 task, I don't wish to use any of the records in the index but want to feed a set of 4 numbers of my choice. How to execute that?

Simplest hack:
from tot.methods.bfs import solve
from tot.tasks.game24 import Game24Task

args = argparse.Namespace(backend='gpt-4', temperature=0.7, task='game24', naive_run=False, prompt_sample=None, method_generate='propose', method_evaluate='value', method_select='greedy', n_generate_sample=1, n_evaluate_sample=3, n_select_sample=5)

task = Game24Task()
task.data.append("1 2 3 4")
ys, infos = solve(args, task, -1)
print(ys[0])