Memory Leak on ridiculously simple repo

You guys do an awesome job and we all appreciate it! 🎉
🐛 Bug Report
On a work project we discovered a memory leak choking our CI machines. Going down the rabbit hole, I was able to recreate the memory leak using Jest alone.
Running many test files causes a memory leak.

I tried a number of solutions from https://github.com/facebook/jest/issues/7311 but to no avail. I couldn't find any solutions in the other memory related issues, and this seems like the most trivial repro I could find.
Workaround :'(
We run tests with --expose-gc flag and adding this to each test file:
afterAll(() => {
  global.gc && global.gc()
})

To Reproduce
Steps to reproduce the behavior:
git clone git@github.com:javinor/jest-memory-leak.git
cd jest-memory-leak
npm i
npm t

Expected behavior
Each test file should take the same amount of memory (give or take)
Link to repl or repo (highly encouraged)
https://github.com/javinor/jest-memory-leak
Run npx envinfo --preset jest
Paste the results here:
System:
    OS: macOS High Sierra 10.13.6
    CPU: (4) x64 Intel(R) Core(TM) i7-5557U CPU @ 3.10GHz
  Binaries:
    Node: 10.15.0 - ~/.nvm/versions/node/v10.15.0/bin/node
    Yarn: 1.12.3 - /usr/local/bin/yarn
    npm: 6.4.1 - ~/.nvm/versions/node/v10.15.0/bin/npm
  npmPackages:
    jest: ^24.1.0 => 24.1.0

I found this out recently but you can use the Chrome console to debug Node scripts! You can try using the Chrome console to profile Jest while it's running to try and dig into the issue.
I believe the command is: node --inspect ./node_modules/.bin/jest --watch -i. When running, open Chrome and go to about:inspect. You should then see the running Node script.

Do I understand correctly that using the workaround to force GC runs makes the heap size remain constant? In that case it's not really a memory leak, just v8 deciding not to run the GC because there is enough memory available. If I try running the repro with 50MB heap size
node --max_old_space_size=50 node_modules/.bin/jest --logHeapUsage --runInBand --config=jest.config.js

the tests still complete successfully, supporting this assumption.

@milesj I ran through some memory dumps, but couldn't make much sense of them, I'm not too experienced with pursuing leaks and I didn't want to point in the wrong direction without something solid to count on.
@jeysal you are right of course! The thing is our tests freeze in the middle of running since (I assume and could be wrong) we run out of memory. After spending a lot of time trying to figure this out, I found https://github.com/facebook/jest/issues/7274. It seemed to me from the discussion that the behaviour I encountered here is not intended. wdyt @SimenB ?

Bueller?

My tests are also leaking massively on CI but the exact same setup locally doesn't really leak (much at least).
It's so bad, I'm considering disabling tests on CI until I can make sense of what the difference is beside the OS. ):

Hey guys!
I simplified the memory leak case to a single file which runs tautological tests and eventually throws an exception due to a memory leak.
I'm not sure how to move forward with this... help? @SimenB @jeysal @milesj
<EMAIL_ADDRESS>test /Users/javinor/workspace/jest-memory-leak
> jest --logHeapUsage --runInBand --config=jest.config.js

 RUNS  __test__/repeat100.test.js

<--- Last few GCs --->

[82597:0x103800000]    79124 ms: Mark-sweep 1259.1 (1442.2) -> 1240.0 (1436.2) MB, 438.3 / 0.0 ms  (average mu = 0.310, current mu = 0.304) allocation failure scavenge might not succeed
[82597:0x103800000]    79197 ms: Scavenge 1255.7 (1436.2) -> 1242.6 (1438.2) MB, 15.9 / 0.0 ms  (average mu = 0.310, current mu = 0.304) allocation failure
[82597:0x103800000]    79256 ms: Scavenge 1256.5 (1438.2) -> 1244.7 (1441.7) MB, 11.0 / 0.0 ms  (average mu = 0.310, current mu = 0.304) allocation failure

<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 0x2440d84dbe3d]
Security context: 0x30802859e6e9 <JSObject>
    1: prepareStackTrace [0x3080c9459f79] [/Users/javinor/workspace/garbage/jest_leak/node_modules/source-map-support/source-map-support.js:~388] [pc=0x2440d854b545](this=0x308094d07b59 <JSFunction Error (sfi = 0x308018616e01)>,error=0x3080e18623f9 <Error map = 0x3080f3c619f9>,stack=0x3080e1862e91 <JSArray[23]>)
    2: InternalFrame [pc: 0x2440d848ee75]
    ...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0x10003c597 node::Abort() [/Users/javinor/.nvm/versions/node/v10.15.3/bin/node]
 2: 0x10003c7a1 node::OnFatalError(char const*, char const*) [/Users/javinor/.nvm/versions/node/v10.15.3/bin/node]
 3: 0x1001ad575 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/Users/javinor/.nvm/versions/node/v10.15.3/bin/node]
 4: 0x100579242 v8::internal::Heap::FatalProcessOutOfMemory(char const*) [/Users/javinor/.nvm/versions/node/v10.15.3/bin/node]
 5: 0x10057bd15 v8::internal::Heap::CheckIneffectiveMarkCompact(unsigned long, double) [/Users/javinor/.nvm/versions/node/v10.15.3/bin/node]
 6: 0x100577bbf v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/Users/javinor/.nvm/versions/node/v10.15.3/bin/node]
 7: 0x100575d94 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/Users/javinor/.nvm/versions/node/v10.15.3/bin/node]
 8: 0x10058262c v8::internal::Heap::AllocateRawWithLigthRetry(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/Users/javinor/.nvm/versions/node/v10.15.3/bin/node]
 9: 0x1005826af v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/Users/javinor/.nvm/versions/node/v10.15.3/bin/node]
10: 0x100551ff4 v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationSpace) [/Users/javinor/.nvm/versions/node/v10.15.3/bin/node]
11: 0x1007da044 v8::internal::Runtime_AllocateInNewSpace(int, v8::internal::Object**, v8::internal::Isolate*) [/Users/javinor/.nvm/versions/node/v10.15.3/bin/node]
12: 0x2440d84dbe3d
Abort trap: 6

Similar here, jest + ts-jest, simple tests get over 1GB of memory and eventually crash.

crashes for us too

@javinor For a test file containing a ridiculous number of tests, I'm not sure there's much we can do, we have to keep the test objects around until the test file is finished - this is the heap while the tests are running:

Hundreds of MBs of test objects, so for the case shown in this repro we can't really help either :/

FYI @scotthovestadt is currently working on holistically improving memory efficiency of Jest, so improvements are coming (some of them in the next minor version).

I wonder, why isn't it possible for Jest to spawn a process for each test file, which will guarantee that memory will be freed? Ok, it can be slower, of course, but in my case - it's much better to be slower rather than get a crash from out-of-memory and be blocked to use Jest alltogether...
Maybe an option? Or a separate "runner" (not sure if I understand architecture and terminology right)?
Is it architecturally possible?
Or, will Node-experimental-workers solve it?..

I've made a few improvements to memory in the next release:
#8234
#8233
I have a future plan to improve memory in a couple of ways:

by not holding onto test results at all, which are currently held until the end. related: #8242
limiting number of modules that can exist in the require cache

The problem with your suggestion of just spawning a new worker for each test is that it would be very slow. A better suggestion along the same lines would be to monitor the memory usage of the processes and auto-restart them at some threshold. I have some concerns about that in general, I'd rather always fix memory leaks than paper them over, but if a PR did that I would accept it.
Let me know if the release next week helps with the problems you've been experiencing.

@scotthovestadt thanks for the info! I'll definitely check with the next release.
My actual issue is reported here: https://github.com/facebook/jest/issues/8247
And there I'm dealing with much bigger leaks, +30Mb per each test file (~150 files in my project).

Thanks for the responses guys!
I think I can break this down to different two problems:

Memory consumption grows the more test files I have
Memory consumption grows the more tests I have per file

We're running thousands of tests, each creating a relatively big setup so we get bitten twice.
The original screenshot showing the consumption growing from test file to test file, hinting to a link between tests - I have a few guesses as to why this happens, but nothing solid yet.
The exception I referred to later, as far as I can tell, really has to do with what @jeysal pointed out - having a large number of tests in the file. In our case, we have only hundreds of tests but with a very large setup. I'll try to provide a better reproduction of this.
I'll update after the next release, when I get to poke around a bit more and see the additional fixes in action. Thanks guys!

There must be something else wrong because I'm currently using Jest v23.6 and everything works fine, no memory leaks, no anything.
If I upgrade to latest Jest then the memory leaks start to happen, but only on the GiLab CI runner. Works fine locally.

New release is out: https://github.com/facebook/jest/releases/tag/v24.6.0

Meh, it's still leaking in my setup ):
Running with gitlab-runner 11.8.0 (4745a6f3)
  on the-mill-ec2-gitlab-runner GvW8cLGR
Using Docker executor with image cypress/base:8 ...
Using locally found image version due to if-not-present pull policy
Using docker image sha256:2ad7a04a3a1d41fac2585d88e2dbd19ed3df9c57df2b27276c30e4f317c2c8e3 for cypress/base:8 ...
section_start:1554216455:prepare_script
Running on runner-GvW8cLGR-project-10463077-concurrent-0 via runner-gvw8clgr-gitlab-runner-1554212599-0c63c2c8...
section_end:1554216456:prepare_script
section_start:1554216456:get_sources
Cloning repository...
Cloning into '/builds/the-mill/lambda-gaming'...
Checking out a592f4d2 as update-fe-bo-packages...
Skipping Git submodules setup
section_end:1554216459:get_sources
section_start:1554216459:restore_cache
section_end:1554216460:restore_cache
section_start:1554216460:download_artifacts
Downloading artifacts for nodeModulesFe (189107055)...
Downloading artifacts from coordinator... ok        id=189107055 responseStatus=200 OK token=iQz-3io7
section_end:1554216481:download_artifacts
section_start:1554216481:build_script
$ cd ./packages/frontends
$ node --version
v8.12.0
$ node --expose-gc ./node_modules/.bin/ng test --ci --code-coverage=false --run-in-band
PASS apps/whitelabel/src/app/app.component.spec.ts (195 MB heap size)
  AppComponent
    ✓ should create the app (87ms)

Test Suites: 1 passed, 1 total
Tests:       1 passed, 1 total
Snapshots:   0 total
Time:        5.168s
Ran all test suites.

PASS libs/api/src/lib/api.module.spec.ts (364 MB heap size)
  ApiModule
    ✓ should create (69ms)

Test Suites: 1 passed, 1 total
Tests:       1 passed, 1 total
Snapshots:   0 total
Time:        3.283s
Ran all test suites.

PASS libs/layout/src/lib/layout/layout.component.spec.ts (6.025s, 1181 MB heap size)
FAIL libs/layout/src/lib/sidebar/sidebar.component.spec.ts
  ● Test suite failed to run

    TypeScript diagnostics (customize using `[jest-config].globals.ts-jest.diagnostics` option):
    libs/layout/src/lib/sidebar/sidebar.component.spec.ts:26:7 - error TS2739: Type '{ __typename: "NavList"; classes: string[]; navListItems: undefined[]; }' is missing the following properties from type 'NavList': horizontal, id

    26       navList: {
             ~~~~~~~

      libs/layout/src/lib/sidebar/sidebar.graphql.ts:8:3
        8   navList: NavList;
            ~~~~~~~
        The expected type comes from property 'navList' which is declared here on type 'Sidebar'

PASS libs/layout/src/lib/header/header.component.spec.ts (1213 MB heap size)
PASS libs/layout/src/lib/footer/footer.component.spec.ts (1194 MB heap size)
PASS libs/layout/src/lib/layout.module.spec.ts (5.055s, 1212 MB heap size)

Test Suites: 1 failed, 4 passed, 5 total
Tests:       7 passed, 7 total
Snapshots:   3 passed, 3 total
Time:        19.157s
Ran all test suites.

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

<--- Last few GCs --->

[10:0x2510510]   136242 ms: Mark-sweep 1362.8 (1460.0) -> 1362.8 (1460.0) MB, 1827.9 / 0.1 ms  allocation failure GC in old space requested
[10:0x2510510]   137760 ms: Mark-sweep 1362.8 (1460.0) -> 1362.8 (1429.0) MB, 1518.2 / 0.1 ms  last resort GC in old space requested
[10:0x2510510]   139285 ms: Mark-sweep 1362.8 (1429.0) -> 1362.8 (1429.0) MB, 1524.6 / 0.1 ms  last resort GC in old space requested

<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0xb1cf5fa5879 <JSObject>
    1: stringSlice(aka stringSlice) [buffer.js:560] [bytecode=0x1e161b35dd29 offset=94](this=0x11a4d26822d1 <undefined>,buf=0x20928062d71 <Uint8Array map = 0x35a39b242941>,encoding=0xb1cf5fb5409 <String[4]: utf8>,start=0,end=1097377)
    2: toString [buffer.js:633] [bytecode=0x1e161b35d979 offset=145](this=0x20928062d71 <Uint8Array map = 0x35a39b242941>,encoding=0xb1cf5fb5409 <String[4]: utf8>,st...

 1: node::Abort() [ng test --ci --code-coverage=false --run-in-band]
 2: 0x8cbf4c [ng test --ci --code-coverage=false --run-in-band]
 3: v8::Utils::ReportOOMFailure(char const*, bool) [ng test --ci --code-coverage=false --run-in-band]
 4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [ng test --ci --code-coverage=false --run-in-band]
 5: v8::internal::Factory::NewRawTwoByteString(int, v8::internal::PretenureFlag) [ng test --ci --code-coverage=false --run-in-band]
 6: v8::internal::Factory::NewStringFromUtf8(v8::internal::Vector<char const>, v8::internal::PretenureFlag) [ng test --ci --code-coverage=false --run-in-band]
 7: v8::String::NewFromUtf8(v8::Isolate*, char const*, v8::NewStringType, int) [ng test --ci --code-coverage=false --run-in-band]
 8: node::StringBytes::Encode(v8::Isolate*, char const*, unsigned long, node::encoding, v8::Local<v8::Value>*) [ng test --ci --code-coverage=false --run-in-band]
 9: 0x8e4a76 [ng test --ci --code-coverage=false --run-in-band]
10: v8::internal::FunctionCallbackArguments::Call(void (*)(v8::FunctionCallbackInfo<v8::Value> const&)) [ng test --ci --code-coverage=false --run-in-band]
11: 0xb0bbec [ng test --ci --code-coverage=false --run-in-band]
12: v8::internal::Builtin_HandleApiCall(int, v8::internal::Object**, v8::internal::Isolate*) [ng test --ci --code-coverage=false --run-in-band]
13: 0x220bb08042fd
/bin/bash: line 90:    10 Aborted                 (core dumped) node --expose-gc ./node_modules/.bin/ng test --ci --code-coverage=false --run-in-band
section_end:1554216631:build_script
section_start:1554216631:after_script
section_end:1554216633:after_script
section_start:1554216633:upload_artifacts_on_failure
Uploading artifacts...
WARNING: ./packages/frontends/junit/*.xml: no matching files
ERROR: No files to upload
section_end:1554216634:upload_artifacts_on_failure
ERROR: Job failed: exit code 1

After updating the 24.6.0, we are seeing the same issue running our CI tests. When logging the heap usage, we see an increase of memory usage after each test file.

This should help: https://github.com/facebook/jest/pull/8282
Will be released soon.

How soon? )':

For those reading along at home, this went out in 24.8.0.

The problem with your suggestion of just spawning a new worker for each test is that it would be very slow.

This would also be a huge breaking change.

If I upgrade to latest Jest then the memory leaks start to happen, but only on the GiLab CI runner. Works fine locally.

Also encounter out of memory issue with jest + ts-jest:
https://circleci.com/gh/unional/standard-log/382
Originally I thought it was some circular dependency on my source code, but may be jest related.
So throwing this into the mix. 🌷

@unional if you're on Circle, make sure maxWorkers isn't higher than the CPU allotted you by Circle.

@Supernats thanks. I think I did have that during the failure, currently I'm running it with --runInBand 😢
But it still fail once in a while:
https://circleci.com/gh/unional/standard-log/448

I have Jest 24.8.0 and #8282 doesn't seem to help. Also --runInBand only helps a bit (4 GB instead of 10 GB 😮).
Pleaaaaaaase fix this ...

Yes, following this thread for long since it still fails for us and in ~10% of the cases runs with "out of memory" for CircleCI 2Gb RAM instances.

Found an article for how to use heap snapshot to debug jest memory leak here: https://chanind.github.io/javascript/2019/10/12/jest-tests-memory-leak.html I tried to use the same method to debug but didn't find the root cause.
Even global.gc() does not help for me, still seeing heap size keeps growing for each test.

Found an article for how to use heap snapshot to debug jest memory leak here: https://chanind.github.io/javascript/2019/10/12/jest-tests-memory-leak.html I tried to use the same method to debug but didn't find the root cause.
Even global.gc() does not help for me, still seeing heap size keeps growing for each test.

Very useful, thanks!

I have a very similar report: https://github.com/jakutis/ava-vs-jest

It seems graceful-fs has a potential memory leak issue, I mocked graceful-fs in my jest environment (with the trick suggested here https://github.com/facebook/jest/issues/6399#issuecomment-399888409), then the issue is resolved.

Is there any progress on this ? I still encouter this problem even with the most simple test suites (with and without ts-jest)

Having same issue with ts-jest, the graceful-fs tip didn't work for me

i have also repro'd this with https://github.com/omgoshjosh/jest-mem-leak-investigation

strangely, running node --expose-gc ./node_modules/.bin/jest --runInBand --logHeapUsage "fixes" the issue but running it with npx jest --runInBand --logHeapUsage or ./node_modules/.bin/jest --runInBand --logHeapUsage produces a memory leak

jest-memory-leak-demo
I've been experiencing memory leaks due to this library and it has made it unusable on one of the projects I'm working on. I've reproduced this in jest-memory-leak-demo, which only has jest as a dependency. I've reproduced this on macOS and within a Docker container using the node:14.3.0 image.
npx envinfo --preset jest

  System:
    OS: macOS 10.15.5
    CPU: (8) x64 Intel(R) Core(TM) i7-8569U CPU @ 2.80GHz
  Binaries:
    Node: 14.3.0 - /usr/local/bin/node
    Yarn: 1.22.4 - ~/.yarn/bin/yarn
    npm: 6.13.7 - ~/.npm-global/bin/npm
  npmPackages:
    jest: ^26.0.1 => 26.0.1

In jest-memory-leak-demo, there are 50 test files with the following script:
it('asserts 1 is 1', () => {
  for (let i = 0; i < 1000; i++) {
    expect(1).toBe(1);
  }
});

Running a test yields 348 MB and 216 MB heap sizes in macOS and Docker, respectively.
However, when I run with node's gc exposed:
node --expose-gc ./node_modules/.bin/jest --logHeapUsage --runInBand

it yields 38 MB and 36 MB heap sizes in macOS and Docker, respectively.
Looking into the code, I see that jest-leak-detector is conditionally constructed based on the config. So if I don't run jest with --detectLeaks, I expect exposing the gc to have no effect. I searched jest's dependencies to see if any package is abusing the gc, but I could not find any.
Is there any progress on this issue?

This is a dummy post to report this issue is still present and makes TDD harder so I'll look forward to any solution

strangely, running node --expose-gc ./node_modules/.bin/jest --runInBand --logHeapUsage "fixes" the issue but running it with npx jest --runInBand --logHeapUsage or ./node_modules/.bin/jest --runInBand --logHeapUsage produces a memory leak
[...]

We are experiencing the same issue even for empty test files.
I am starting to think that either --logHeapUsage is not working properly or that by using --expose-gc only the reports are changing.

It is surprising that forcing GC seems to fix the issue for us as well (FYI, the jest test runner calls gc() if available when --logHeapUsage is enabled, hence the behavior @omgoshjosh saw). I'm assuming that actually leaked references wouldn't get marked by a forced GC any differently than they would during a normally scheduled GC. Forcing GC in test hooks is also a very poor workaround, because it can slow things down tremendously.
All in all, very peculiar behavior.

having the same issues here as well

+1

+1

+1, same issue

a few colleagues who are on a mac don't seem to be able to replicate this bug. could it be linux specific?

a few colleagues who are on a mac don't seem to be able to replicate this bug. could it be linux specific?

Definitely not, I'm also on OSX, and it happens left and right.

a few colleagues who are on a mac don't seem to be able to replicate this bug. could it be linux specific?

I noticed a significant difference in the heap size on OSX vs Docker node image after exposing the node GC. While the heap kept around ~400MB on the OSX they still climbed to 1300MB in the Docker container. Without exposing the GC the difference is negligible. So there might be some difference in how the GC works on different platforms.

Same here, my CI crashes all time

I have the same issue. CI crashes and is leaking memory.

Quick Recap

Good - Running tests with jest@23 works as expected - heap size oscillates but goes down to original - it appears like the GC is succeeding to collect all memory allocated during the tests
Bad - Running tests with jest@24+ including<EMAIL_ADDRESS>- heap size continues to grow over time, oscillating, but doesn't seem to go down to the initial size - My assumption is that there's a memory leak preventing GC for freeing up all the memory
I took the screenshots after running @jaredjj3's example repo - see this comment)

@SimenB help? 🙏
Running with jest@23

Running with jest@26

For those wanting to get their CI pipeline going with jest@26, I found a workaround that works for me. (this issue comment helped, combined with this explanation). I increased the maximum oldspace on node, and although the leak persists, my CI pipeline seems to be doing better/passing. Here my package.json input:
 "test-pipeline": "node --max-old-space-size=4096 ./node_modules/.bin/jest --runInBand --forceExit --logHeapUsage --bail",
What else I tried and scraped together from a few other issues:

exposed garbage collector (did nothing) (i.e. node --expose-gc ./node_modules/...) && used the afterEach fix from above
inspected the port where my my server was running (from here, increasing heap seemed invisible to the inspector)
patched graceful-fs with this. Probably taken from this issue, but it did nothing

Hey guys, my team also encounter this issue, and we would like to share our solution.
Firstly, we need to understand that Node.js will find its best time to swipe unused memory by its own garbage collection algorithm. If we don’t configure it, Node.js will do their own way.
 If we call global.gc(), node.js will swipe all known unused memory.

--max-old-space-size=xxx : If we add this flag while running node.js. We are asking node.js to swipe all known unused memory if unused memory has reach xxx MB.

Secondly, I think we have 2 types of memory leak issue.

Type 1: Node.js knows that there are some unused memory, but he think that doesn’t matter his process, so he didn’t swipe unused memory.  However, he ran out of all memory on his container / device.

Type 2: Node.js doest NOT know that there exists some unused memory.

For Type 1, it’s easier to solve. We can use --expose-gc flag and run global.gc() in each test to swipe unused memory.
Or, we can add --max-old-space-size=xxx to remind node.js to swipe all known unused memory once it reached limit.
Before adding --max-old-space-size=1024:
 PASS  src/xx/x1.test.js (118 MB heap size)
 PASS  src/xx/x2.test.js (140 MB heap size)
 ...
 PASS  src/xx/x30.test.js (1736 MB heap size)
 PASS  src/xx/x31.test.js (1746 MB heap size)
...

After adding --max-old-space-size=1024:
 PASS  src/xx/x1.test.js (118 MB heap size)
 PASS  src/xx/x2.test.js (140 MB heap size)
 ...
 PASS  src/xx/x20.test.js (893 MB heap size)
 PASS  src/xx/x21.test.js (916 MB heap size)

// -> (everytime it reachs 1024MB, it will swipe ununsed memory)

 PASS  src/xx/x22.test.js (382 MB heap size)
 ...

(Note: if we specify lower size, it will of course use less unused memory, but more frequent to swipe it)
For Type 2, we may need to investigate where memory leaks happened. This would be more difficult.
Because in our team, we only have Type I issue, so our solution is adding --max-old-space-size=1024 to Node.js

Finally, I would like to explain why --expose-gc works in previous comment.
Because in Jest source code, if we add --logHeapUsage to Jest, Jest will call global.gc() if gc exists. In other words, if we add --logHeapUsage to Jest and add --expose-gc to Node.js, in current version of Jest, it will force Node.js to swipe all known unused memory for each run of test.
However, I don’t really think adding --logHeapUsage and --expose-gc to solve this issue is a good solution. Because it’s more like we “accidentally” solve it.

Note:
--runInBand: ask Jest to run all tests sequentially. By default, Jest will run tests in parallel by several workers.
--logHeapUsage: log heap memory usage in each test.

Something that might be helpful for those debugging a seeming memory leak in you Jests tests:
Node's default memory limit applies seperately to each worker, so make sure that the total memory available > number of workers * the memory limit.
When the combined memory limit of all of the workers is greater than the available memory, Node will not realized that it needs to run GC, and the memory usage will climb until it OOM's.
Setting the memory limit correctly causes Node to run GC much more often.
For us, the effect was dramatic. When we had the --max-old-space-size=4096 and three workers on a 6GB machine, memory usage increased to over 3gb per worker and eventually OOM'd. Once we set it to 2gb, memory usage stayed below 1gb per worker, and the OOM's went away.

I believe I may be experiencing the same problem as reported in this thread.

 PASS  src/pages/Web/Account/_helpers/__tests__/SettingsContainer.test.js (49 MB heap size)
 PASS  src/pages/Web/Account/OrganizationInfo/__tests__/OrgNameSection.test.js (47 MB heap size)
 PASS  src/components/Modal/Modal.test.js (47 MB heap size)
 PASS  src/pages/Web/Account/_helpers/__tests__/SettingsAvailability.test.js (47 MB heap size)

experiencing the exact same issue with jest+ts-jest on a nestjs project
event the simplest test is reporting 500mb of heap size.
 PASS  tier5-test/one.get.spec.ts (7.304 s, 596 MB heap size)

describe('one', () => {

  it('one', async () => {
    expect(2).toEqual(2);
  });
});

Here are my findings.
Did a study on 4 of our apps, made a benchmark with the following commands.

Case
Command

A
NODE_ENV=ci node node_modules/.bin/jest --coverage --ci --runInBand --logHeapUsage

B
NODE_ENV=ci node --expose-gc node_modules/.bin/jest --coverage --ci --runInBand --logHeapUsage

C
NODE_ENV=ci node --expose-gc ./node_modules/.bin/jest --logHeapUsage

D
NODE_ENV=ci node node_modules/.bin/jest --coverage --ci --logHeapUsage

Cross apps
Max Heap

Average Heap

App1
Max Heap and Average Heap

Heap Chronology

App2
Max Heap and Average Heap

Heap Chronology

App3
Max Heap and Average Heap

Heap Chronology

App4
Max Heap and Average Heap

Heap Chronology

Rank of appearance of the highest heap

Hope this helps or re-kindles the debate.

I'm having this problem too, and notably it only happened when I upgraded to Webpack 5. Prior to that, tests were running just fine.

Did a heap snapshot for my test suite and noticed that the majority of the memory was being used by strings that store entire source files, often the same string multiple times!

The size of these strings continue to grow as jest scans and compiles more source files.  Does it appear that jest or babel holds onto the references for the source files (for caching maybe) and never clears them?

Did a heap snapshot for my test suite and noticed that the majority of the memory was being used by strings that store entire source files, often the same string multiple times!

The size of these strings continue to grow as jest scans and compiles more source files. Does it appear that jest or babel holds onto the references for the source files (for caching maybe) and never clears them?

I have found the same thing with Jest, but I have the feeling this is normal. Still, it does beg the question of whether these strings are cleared once jest has finished using them
However, what I've discovered not too long ago, is that importing data from .json files caused my "memory leak", or rather, a "memory overflow". Once I stuck the JSON files into js files (the dat aof which now sits inside some variable), and exported that variable, heap usage had been significantly reduced during testing, enough that our pipeline didn't crash anymore.

We at renovate solved the major issues by correctly disable nock after each test and run jest with node --expose-gc node_modules/jest/bin/jest.js --logHeapUsage

We also noticed that there were a ton of strings containing modules that were eating up memory. It all went away when we set memory limits correctly: https://github.com/facebook/jest/issues/7874#issuecomment-744561779
Setting memory limits correctly is effectively the same as forcing GC with --logHeapUsage, as node will run GC automatically to keep from going over the allowed memory.

When I --logHeapUsage --runInBand on my test suite with about 1500+ tests, the memory keeps climbing from ~100MB to ~600MB. The growth seems to be in these strings, and array of such strings (source).
It is apparent that the number of compiled files will grow as jest moves further in the test suite — but if this doesn't get GC'ed, I don't have a way to separate real memory leaks from increases due to more modules stored in memory.
On running a reduced version my test suite (only about ~10 tests), I was able to narrow down on this behaviour —
TEST A, TEST B, TEST C, TEST D
I was observing TEST B. Without doing anything —
TEST A (135 MB heap size)
TEST B (150 MB heap size)
TEST C (155 MB heap size)
TEST D (157 MB heap size)

If I reduce the number of imports TEST B is making, and replace them with stubs —
TEST A (135 MB heap size)
TEST B (130 MB heap size). <-- Memory falls!
TEST C (140 MB heap size)
TEST D (147 MB heap size)

v27 seems to leak more memory. The test of my project never encountered OOM on v26, but it was killed on v27.

We at renovate solved the major issues by correctly disable nock after each test and run jest with node --expose-gc node_modules/jest/bin/jest.js --logHeapUsage

I went trough the pull requests in your repo but can you give some more information on this? I have been having issues with large heap memory and flaky tests in CI only and I use nock. Any help is apreciated.

You nedd  nock.restore(); in afterAll
https://github.com/renovatebot/renovate/blob/394f0bb7416ff6031bf7eb14498a85f00a6305df/test/http-mock.ts#L106-L121

I'm being affected by this as well. Our application is a react app with typescript.
I have tried NODE_OPTIONS=--max_old_space_size=2048, jest.useFakeTimers();, removing all import *, --clear-cache, with and without --coverage. To no avail.
Confusingly when I run the chrome devtools to look at the memory heap, it seems to think the heap is only 25mb? It's over 3000mb so I'm not sure what that is about. There are a crapload of strings with code though.
I'm wondering if it's some interaction with the version of jest, node, typescript, react, babel, or eslint?
I'm using:
jest 27.2.4
node v16.4.0
typescript 4.4.3
eslint 7.32.0
react 16.4.2

This works for me: https://github.com/kulshekhar/ts-jest/issues/1967#issuecomment-834090822
Add this to jest.config.js
globals: {
    'ts-jest': {
      isolatedModules: true
    }
  }

In the end what really improved our pipeline was fiddling with the
maxworkers option to figure our how many would make the fastest yet most
stable pipeline. The memory leak thingy did not seem to stabilize and
reduce pipeline times as much as manually finding the right figure for
maxworkers actually did.
Le sam. 9 oct. 2021 à 10:37, Huy Tran @.***> a écrit :

This works for me: kulshekhar/ts-jest#1967 (comment)
https://github.com/kulshekhar/ts-jest/issues/1967#issuecomment-834090822
Add this to jest.config.js
globals: {
'ts-jest': {
isolatedModules: true
}
}
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/facebook/jest/issues/7874#issuecomment-939239534, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AEEIWZWW5MRCPY72EDYKMXDUF7PJ7ANCNFSM4GW5MRBQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

Did a heap snapshot for my test suite and noticed that the majority of the memory was being used by strings that store entire source files, often the same string multiple times!

The size of these strings continue to grow as jest scans and compiles more source files. Does it appear that jest or babel holds onto the references for the source files (for caching maybe) and never clears them?

+1, we are also running into this issue. Jest is using over 5GB per worker for us. Our heap snapshots show the same thing as above. Any updates would be greatly appreciated. We cannot scale jest with this issue.

I've observed today two unexplained behaviour.

There's too much memory usage even when disabling code transforms and cleaning jest's cache
When using --forceExit or --detectOpenHandles (or a combination of both), the memory usage drops from 1.4GB to roughly 300MB

I don't know if this specific to our codebase or if the memory leak issue is tied to tests that somehow don't really finish/cleanup properly (which detectOpenHandles or forceExit somehow fix)

I'm seeing this same leak, with heap snapshots that look remarkably similar to those posted by @pastelsky above.  I'm at Jest 27.0.6.  Tried moving ahead to 27.4.3 and back to 26.6.3, to no avail.  Tried a few things mentioned in this issue, without affect.

TLDR - The memory leak stops at version 22.4.4 and ends at version 23.0.0-alpha.1. The following steps are for: (1) people to assert/refute this and then (2) find the offending commit causing the memory leak and increased memory usage.
In https://github.com/facebook/jest/issues/7874#issuecomment-639874717, I mentioned that I created a repo jest-memory-leak-demo to make this issue easier to reproduce in local and Docker environments.
I took it a step further and decided to find the version that the memory leak started to show. I did this by listing all the versions returned from yarn info jest. Next, I manually performed a binary search to find the versions where version i does not produce a memory leak and version i + 1 does produce a memory leak.
Here is my env info (I purposely excluded the npmPackages since the version was the variable in my experiment):
npx envinfo --preset jest

  System:
    OS: macOS 12.0.1
    CPU: (10) arm64 Apple M1 Max
  Binaries:
    Node: 17.1.0 - ~/.nvm/versions/node/v17.1.0/bin/node
    Yarn: 1.22.17 - ~/.nvm/versions/node/v17.1.0/bin/yarn
    npm: 8.1.2 - ~/.nvm/versions/node/v17.1.0/bin/npm

The key finding is in jest-memory-leak-demo/versions.txt#L165-L169. You can see several iterations of the binary search throughout the file. I did one commit per iteration, so you also can examine the commit history starting at edc0567ad4710ba1be2bf2f745a7d5d87242afc4.
The following steps are for the community to validate these initial findings and ultimately use the same approach to find the offending commit causing the memory leaks. Here's a StackOverflow post that will help: "How to get the nth commit since the first commit?".
It would also be great if someone can write a script to do this in jest-memory-leak-demo. The most challenging part of doing this is programming memory leak detection. A list of versions can be found by running yarn info jest. I don't have time to do this at the moment.

NOTE: I was not very scientific about defining what versions produce a memory leak and what versions don't. First of all, I should have used the yarn docker-test command to reproduce the results on other machines, but I just wanted to get an answer as fast as possible. Second, for each version, I should have run the test command >30 times and then aggregated the results. If you decide to reproduce this in the way I did it, YMMV.

NOTE: For earlier versions, I had to add the following to my package.json:
"jest": {
 "testURL": "http://localhost/"
}

If I didn't, I got the following error:
SecurityError: localStorage is not available for opaque origins

At first, I was diligent in removing this if it was not needed, but then I got lazy after iteration eight or so and just kept it. I don't >know if this affected the results.

I have tested the reproduction with jest 24, jest 27 and jest 28 beta:

Version
--runInBand
min heap size
max heap size

24.9.0
true
53 MB
259 MB

24.9.0
false
47 MB
61 MB

27.5.1
true
36 MB
71 MB

27.5.1
false
26 MB
30 MB

28.0.0-alpha.6
true
38 MB
73 MB

28.0.0-alpha.6
false
27 MB
36 MB

(All tested on node.js v14.15.3)
I think in general the leak has become less of an issue, but the discrepancy between --runInBand=true and --runInBand=false suggests that there is still an issue. It has become much smaller though.
See also:
#12142 (leak when using --runInBand)
#10467 (duplicate of this issue)
#7311 (leak when using --runInBand)
#6399 (leak when using --runInBand)
As for the cause, from other issues relating to leaks, I suspect that there are multiple issues playing a role. For example:
#6738 [Memory Leak] on module loading
#6814 Jest leaks memory from required modules with closures over imports
#8984 jests async wrapper leaks memory
#9697 Memory leak related to require (might be a duplicate of / has common cause with 6738?)
#10550 Module caching memory leak
And #8832 could either be another --runInBand issue or a require / cache leak.
There are also leak issues with coverage, JSDOM and enzyme #9980 has some discussion about that. And #5837 is directly about the --coverage option.

As for triaging memory leaks, there needs to be some minimum info:

NodeJS version.
Leak stats with and without --runInBand
Is coverage used? If so, does the leak persist without code coverage?

And then it would be nice to find out if the scenario is one of the require and cache leaks. Figuring that out is probably a little involved unless those leaks get fixed.

Ah, thank you so much @StringEpsilon! Is there still a leak now (i.e. forever growing, never reset) in the reproduction on the OP using v27? If not, I think we should close this and encourage new issues with reproductions as you say.
Note that 2 changes in v27 very much impacts this - we swapped default test environment from jsdom to node, and default test runner from jasmine to circus. So direct comparison between versions might not be perfectly valid.

If it's a single test file it always runs in band

@radcapitalist
I am sorry, my intention was only to report the progress that was made and to figure out the overall situation with the ~30 tickets. I was not trying to suggest that the various leaks are not an issue anymore. In fact, if anything, my testing suggests that --runInBand still has a large confounding effect on whatever leaks may occur.
As for the node 16 issue, there is a workaround via patch-package, see this comment https://github.com/facebook/jest/issues/11956#issuecomment-1011310131 and following.

@StringEpsilon No need to apologize whatsoever!  @SimenB suggested closing this after your comment and that just worried me a little :-).
We are working on splitting up our test files more to see if it helps us.
And thanks very much for the patch-package tip, I will look into it!
Eric

@SimenB I have drafted a meta issue: https://gist.github.com/StringEpsilon/6c8e687b47e0096acea9345f8035455f

One thing I came over when going through the list of issues was this comment: https://github.com/facebook/jest/issues/7311#issuecomment-578729020
So I tried out with this quick and dirty diff locally:
diff --git i/packages/jest-leak-detector/src/index.ts w/packages/jest-leak-detector/src/index.ts
index 0ec0280104..6500ad067f 100644
--- i/packages/jest-leak-detector/src/index.ts
+++ w/packages/jest-leak-detector/src/index.ts
@@ -50,7 +50,7 @@ export default class LeakDetector {
   }

   async isLeaking(): Promise<boolean> {
-    this._runGarbageCollector();
+    runGarbageCollector();

     // wait some ticks to allow GC to run properly, see https://github.com/nodejs/node/issues/34636#issuecomment-669366235
     for (let i = 0; i < 10; i++) {
@@ -59,18 +59,18 @@ export default class LeakDetector {

     return this._isReferenceBeingHeld;
   }
+}

-  private _runGarbageCollector() {
-    // @ts-expect-error
-    const isGarbageCollectorHidden = globalThis.gc == null;
+export function runGarbageCollector(): void {
+  // @ts-expect-error
+  const isGarbageCollectorHidden = globalThis.gc == null;

-    // GC is usually hidden, so we have to expose it before running.
-    setFlagsFromString('--expose-gc');
-    runInNewContext('gc')();
+  // GC is usually hidden, so we have to expose it before running.
+  setFlagsFromString('--expose-gc');
+  runInNewContext('gc')();

-    // The GC was not initially exposed, so let's hide it again.
-    if (isGarbageCollectorHidden) {
-      setFlagsFromString('--no-expose-gc');
-    }
+  // The GC was not initially exposed, so let's hide it again.
+  if (isGarbageCollectorHidden) {
+    setFlagsFromString('--no-expose-gc');
   }
 }
diff --git i/packages/jest-runner/src/runTest.ts w/packages/jest-runner/src/runTest.ts
index dfa50645bf..5e45f06b1b 100644
--- i/packages/jest-runner/src/runTest.ts
+++ w/packages/jest-runner/src/runTest.ts
@@ -22,7 +22,7 @@ import type {TestFileEvent, TestResult} from '@jest/test-result';
 import {createScriptTransformer} from '@jest/transform';
 import type {Config} from '@jest/types';
 import * as docblock from 'jest-docblock';
-import LeakDetector from 'jest-leak-detector';
+import LeakDetector, {runGarbageCollector} from 'jest-leak-detector';
 import {formatExecError} from 'jest-message-util';
 import Resolver, {resolveTestEnvironment} from 'jest-resolve';
 import type RuntimeClass from 'jest-runtime';
@@ -382,6 +382,11 @@ export default async function runTest(
     // Resolve leak detector, outside the "runTestInternal" closure.
     result.leaks = await leakDetector.isLeaking();
   } else {
+    if (process.env.DO_IT) {
+      // Run GC even if leak detector is disabled
+      runGarbageCollector();
+    }
+
     result.leaks = false;
   }

So if running after every test file, this gives about a 10% perf degradation for jest pretty-format in this repo.
$ hyperfine 'node packages/jest/bin/jest.js pretty-format' 'node packages/jest/bin/jest.js pretty-format -i' 'DO_IT=yes node packages/jest/bin/jest.js pretty-format' 'DO_IT=yes node packages/jest/bin/jest.js pretty-format -i'
Benchmark 1: node packages/jest/bin/jest.js pretty-format
  Time (mean ± σ):      2.391 s ±  0.088 s    [User: 2.418 s, System: 0.392 s]
  Range (min … max):    2.273 s …  2.574 s    10 runs

Benchmark 2: node packages/jest/bin/jest.js pretty-format -i
  Time (mean ± σ):      2.315 s ±  0.060 s    [User: 2.381 s, System: 0.385 s]
  Range (min … max):    2.229 s …  2.416 s    10 runs

Benchmark 3: DO_IT=yes node packages/jest/bin/jest.js pretty-format
  Time (mean ± σ):      2.513 s ±  0.101 s    [User: 2.966 s, System: 0.397 s]
  Range (min … max):    2.413 s …  2.746 s    10 runs

Benchmark 4: DO_IT=yes node packages/jest/bin/jest.js pretty-format -i
  Time (mean ± σ):      2.581 s ±  0.179 s    [User: 2.981 s, System: 0.403 s]
  Range (min … max):    2.423 s …  3.032 s    10 runs

Summary
  'node packages/jest/bin/jest.js pretty-format -i' ran
    1.03 ± 0.05 times faster than 'node packages/jest/bin/jest.js pretty-format'
    1.09 ± 0.05 times faster than 'DO_IT=yes node packages/jest/bin/jest.js pretty-format'
    1.11 ± 0.08 times faster than 'DO_IT=yes node packages/jest/bin/jest.js pretty-format -i'

However, this also stabilizes memory usage in the same way --detect-leaks does.
So it might be worth playing with this (e.g. after every 5 test files instead of every single one?). Thoughts?

I'll reopen (didn't take long!) since I'm closing most other issues and pointing back here 🙂

Hi,
We experiments the same issue about memory leaks with our Angular 13 app
We try the --detect-leaks as suggested above and it seems to work but only with node 14 (14.19.1)
npx jest-heap-graph "ng test --run-in-band --log-heap-usage --detect-leaks"
Here the heap graph for 14.19.1 :
--- n: 127 ---
     287.00 ┤                  ╭╮                  ╭╮╭╮           ╭╮
     285.00 ┤      ╭╮ ╭╮ ╭╮    ││╭╮ ╭╮  ╭╮      ╭╮ ││││      ╭╮  ╭╯│      ╭╮  ╭╮      ╭╮
     283.00 ┤      ││ │╰╮││    ││││ │╰╮╭╯│  ╭╮ ╭╯│ ││││  ╭╮  ││╭╮│ ╰╮╭╮   ││ ╭╯│ ╭╮ ╭╮││        ╭╮
     281.00 ┼╮ ╭─╮ ││ │ │││  ╭╮││││ │ ││ │  ││ │ │ │╰╯╰╮╭╯│  │││││  │││  ╭╯│╭╯ │ ││ ││││   ╭╮   ││
     279.00 ┤│ │ │ ││ │ │││  │╰╯│││╭╯ ╰╯ ╰╮ │╰╮│ │ │   ╰╯ ╰╮ │╰╯╰╯  ││╰╮ │ ││  ╰╮││ ││││ ╭╮││╭╮ ││
     277.00 ┤│ │ │ ││ │ │││╭╮│  ││││      │ │ ││ │╭╯       │ │      ╰╯ ╰─╯ ╰╯   ╰╯│╭╯││╰─╯││╰╯╰─╯╰────
     275.00 ┤│ │ │╭╯╰╮│ ││││││  ││││      ╰╮│ ││ ╰╯        │ │                    ╰╯ ╰╯   ╰╯
     273.00 ┤│╭╯ ││  ││ ╰╯││││  ││╰╯       ╰╯ ││           │ │
     271.00 ┤││  ││  ││   ││││  ││            ││           │╭╯
     269.00 ┤││  ╰╯  ╰╯   ││╰╯  ╰╯            ╰╯           ││
     267.00 ┤╰╯           ╰╯                               ╰╯

Here the heap graph for 16.14.2 :
--- n: 126 ---
    2212.00 ┤                                                                                ╭─────
    2074.50 ┤                                                                          ╭────────╯
    1937.00 ┤                                                                ╭─────────╯
    1799.50 ┤                                                        ╭───────╯
    1662.00 ┤                                               ╭────────╯
    1524.50 ┤                                      ╭────────╯
    1387.00 ┤                             ╭────────╯
    1249.50 ┤                    ╭────────╯
    1112.00 ┤            ╭───────╯
     974.50 ┤   ╭────────╯
     837.00 ┼───╯

npx envinfo --preset jest

  System:
    OS: Windows 10 10.0.19044
    CPU: (12) x64 11th Gen Intel(R) Core(TM) i5-11500H @ 2.90GHz
  Binaries:
    Node: 16.14.2 - C:\Program Files\nodejs\node.EXE
    Yarn: 1.22.18 - ~\workspace\WEBTI\fe-webti\node_modules\.bin\yarn.CMD
    npm: 8.5.0 - C:\Program Files\nodejs\npm.CMD
  npmPackages:
    jest: ^27.3.1 => 27.5.1

We also tried the coverageProvider set to babel as here https://github.com/facebook/jest/issues/11956#issuecomment-1112561068 no change
We guess our tests may are not perfectly well written but there still leaks

I use Jest for integration testing, it can be complicated to find the source of the memory leak (maybe related to a graceful teardown problem in my test suite, not a Jest issue).
I use this workaround to avoid OOM using matrix on Github actions.
name: Backend

on:
  pull_request:
    branches:
      - master
    types: [opened, synchronize, reopened, unlabeled]

jobs:
  buildAndLint:
    timeout-minutes: 10
    runs-on: ubuntu-latest

    strategy:
      matrix:
        node-version: [16.x]

    steps:
      - uses: actions/checkout@v2
      - run: yarn build && yarn lint
  Test:
    needs: [buildAndLint]
    timeout-minutes: 30
    runs-on: ubuntu-latest

    strategy:
      matrix:
        node-version: [16.x]
        # Each app folder test will be run in parallel using the matrix
        folder: [adapters, auth, cloud, customSchema, miscTest, schemas, utils]

    steps:
      - uses: actions/checkout@v2
      # Improve this to use github artifact
      - run: yarn build:fast
      # Jest will only run test from the folder
      - run: yarn test ${{ matrix.folder }}
        env:
          NODE_ENV: TEST

This script could be improved to upload each LCOV result in a final job and then merge all coverage results into one using nyc merge see: https://stackoverflow.com/questions/62560224/jest-how-to-merge-coverage-reports-from-different-jest-test-runs

As @UnleashSpirit mentioned, downgrading to node 16.10 fixed the memory issue with Jest.

@j0k3r it works:)

Wow - this one got us too ... after scouring the internet and finding this ... reverting to 16.10 fixed our build too (in gitlab). Here's hoping there's a longer-term solution, but many thanks for the suggestion!

This looks like it'll drag on forever as it doesn't appear to be getting picked up on the node/v8 side. Is there anything that can be done in order to escalate this?

And we're already starting to run into problems with libs requiring a specific min LTS version of node now (as per my linked issue above). This is getting painful to work around without dodgy --ignore-engines for CI.

It seems there is a suggested fix for Jest as per this comment: https://bugs.chromium.org/p/v8/issues/detail?id=12198#c20
Hopefully this makes more sense to someone in the Jest team ... is this something that could be persued?

Did Victor's suggested workaround work for Node? Updating from above, it would be to change https://source.chromium.org/chromium/chromium/src/+/main:v8/src/heap/heap.h;l=1460;drc=de8943e4326892f4e584a938b31dab0f14c39980;bpv=1;bpt=1 to remove the is_current_gc_forced_ check.
In general it's my understanding that --exposed-gc is primarily a testing feature and shouldn't be depended upon in production. Is it not possible to remove forced GCs from how jest runs?

Same issue here too. Downgrading to node 16.10 fixed the memory issue with Jest.

Same issue for me. Downgrading to node 16.10 fixed the memory leak with Jest. I was seeing 3.4GB heap sizes with node 16.24, down to ~400MB with node 16.10.

Looking for some forward momentum here as well; CI pipeline is locked at 6.10 and explicitly calls node --expose-gc ./node_modules/.bin/jest to reduce heap sizes from ~3699MB to 610MB on two repositories.
Previous chromium issue linked by @mbyrne00 is labeled as wont-fix, which is concerning to me with Node 18 becoming LTS in October.
Is there any solution in mind?

The chromium issue points back to node and jest and how they are implementing certain features. As one user on the Chromium issue thread pointed out, this is the closest thing to an attempt to resolve the issue so hopefully it gets legs: https://github.com/facebook/jest/pull/12205/files#diff-c0d5b59e96fdc7ffc98405e8afb46d525505bc7b1c24916b5c8482de5a186c00R1359-R1373

Just FYI, the interesting news on the node / v8 upstream bug can be found at #11956 - where some mitigations for jest 29 and a workaround (albeit a very slow one) for jest 27 were discussed just 4 days ago.
As far as I understood @SimenB's intention to reopen this issue, it is NOT to discuss known causes of memory leaks, but to track the overall state of jest leakages outside of identified and known issues.

I've run some tests considering various configurations. Hope it helps someone.

node version
node args
jest args
custom behavior
time (seconds)
heap (mb)

16.10
--expose-gc --no-compilation-cache
--maxWorkers 1
afterAll(global.gc) + force options.serial to false on jest-runner
303
45

16.18
--expose-gc --no-compilation-cache
--maxWorkers 1
afterAll(global.gc) + force options.serial to false on jest-runner
325
47

16.10
--expose-gc --no-compilation-cache
--maxWorkers 2
-
236
64

16.18
--expose-gc --no-compilation-cache
--maxWorkers 2
-
167
67

16.10
--expose-gc
--maxWorkers 1
afterAll(global.gc) + force options.serial to false on jest-runner
234
82

16.10
--expose-gc
--maxWorkers 2
-
155
96

16.10
--expose-gc --no-compilation-cache
--runInBand --detectLeaks
afterAll(global.gc)
313
159

16.10
--expose-gc --no-compilation-cache
--runInBand --detectLeaks
-
307
160

16.10
--expose-gc --no-compilation-cache
--runInBand
-
313
160

16.10
--expose-gc --no-compilation-cache
--maxWorkers 1
-
333
160

16.10
--expose-gc --no-compilation-cache
--runInBand --detectLeaks
afterEach(global.gc)
397
160

16.18
--expose-gc --no-compilation-cache
--runInBand --detectLeaks
afterAll(global.gc)
281
164

16.18
--expose-gc --no-compilation-cache
--runInBand --detectLeaks
afterEach(global.gc)
298
164

16.18
--expose-gc --no-compilation-cache
--maxWorkers 1
-
287
165

16.18
--expose-gc --no-compilation-cache
--runInBand --detectLeaks
-
300
165

16.18
--expose-gc --no-compilation-cache
--runInBand
-
337
165

16.10
--expose-gc
--runInBand --detectLeaks
-
258
199

16.10
--expose-gc
--runInBand
-
247
201

16.10
--expose-gc
--maxWorkers 2
-
286
201

16.10
--expose-gc
--runInBand --detectLeaks
afterAll(global.gc)
256
202

16.10
--expose-gc
--runInBand --detectLeaks
afterEach(global.gc)
309
206

16.10

--runInBand
-
261
629

16.18
--expose-gc
--maxWorkers 2
-
277
899

16.18
--no-compilation-cache
--runInBand
-
297
907

16.18

--runInBand
-
281
1055

16.18
--expose-gc
--runInBand
-
347
1262

16.18
--expose-gc
--maxWorkers 1
afterAll(global.gc) + force options.serial to false on jest-runner
337
1380

Test Suites: 3 skipped, 31 passed, 31 of 34 total
Tests:       20 skipped, 49 todo, 171 passed, 240 total
Snapshots:   0 total

* Running with Jest 29.2.2 on a bitbucket pipeline container using node official docker images

From a preliminary run or two, it looks to me like going back to 16.10 is resolving these errors for us as well.
Is there any more clarity on why this is, or what a real fix might look like?

Just spend about 2 days figuring out how to overcome this, until I discovered #11956.
TLDR; regression introduced in node 16.11, fixed in 21.1.
Backporting Bugfix for node 18 seems to be on the way, see
https://github.com/jestjs/jest/issues/11956#issuecomment-1824288913

Just spend about 2 days figuring out how to overcome this, until I discovered #11956.
TLDR; regression introduced in node 16.11, fixed in 21.1. Backporting Bugfix for node 18 seems to be on the way, see #11956 (comment)

In case anyone stumbles across this and wants a simple solution, node 20.10.0 contains a fix for this.
Upgrading to node 20.10.0 solved this for me 🎉

Reading the linked issue, it says '21.1', but might as well being already backported to '20.x', leaving '18.x' to be waiting for a fix🤷.
For our team, switching our ci-builds to '21.x' did the deal, even if this might introduce runtime-confusion 😉.

I'm currently memory leaks in Jest (after #11956 solution), so I was interested in this case.
I followed these steps:

Clone the reproduction repo.
rm package-lock.json (outdated npm version)
npm install
Clone Jest main and build it
Add .nvmrc with 20.14.0
Edit runTest.ts in jest-runner to point to the project own jest-environment-node (instead of the default target repo)
Run jest with this script in jest-cli:

node ./bin/jest.js -c /repos/jest-memory-leak/jest.config.js --runInBand --logHeapUsage

The initial execution of the 30 test files barely reached 100MB at the last file, so I added more duplicates, totaling in 651 files.

Process finished with exit code 0

I think it's safe to conclude that the initial issue is resolved. It's not to say that there aren't memory leaks, but only that they are not reproducible using a simple repo.
For information on my progress with the other leaks, see #15215.

adding workerIdleMemoryLimit: '1024MB', to the jest.config.js file resolved the issue for me.

Do you know if we have an update on this? I face this problem while upgrading to Expo version 50 as we have a significant test set from which I have tests with a considerable heap size.
Example: VerifyMobileNumber.resendCode.test.tsx (9.593 s, 2355 MB heap size)
Current setup:
NodeJS: v20.16.0
Jest: 29.7.0
Jest-expo: 50.0.4
expo: 50.0.20
My current concerns:

workerIdleMemoryLimit is ignored with the heap size. Why is this? Could we limit memory usage in general
Not many resources to explore the root cause of the heap size

We run our tests in a node with shared resources.
These settings seemed to have helped

workerThreads=true (didn't see measurable improvement from this, without workerIdleMemoryLimit)
workerIdleMemoryLimit=500mb (this kept memory in check, we had suits going well over 1gb)
--max-semi-space-size=64 (I saw a 7% improvement locally on a suite, so assumed to be improved in CI too)

This setting essentially prevents node from doing as many small GC events, by giving more room for garbage. The trade-off is that there will be more/slower major GC events, but I think that by setting a workerIdleMemoryLimit you're already setting yourself up for plenty of major GC events.

@EduardoAC  We have ~3100 test files.

~400 files (13%) use async
only 6 use useFakeTimers.

I know run time is not really entirely related to memory, but our most problematic/flaky tests are component tests (react testing library) with lots of async calls, and a few that have large jest.setTimeout(60 * 1000) values.


Add custom operator protocols on the modules

Right now, minimal defines all the grpc protocols for all the modules in system.proto (peers, blockchain, txpool...). We should pass to each module the reference to the grpc server and let each module define his own system.proto spec.

This was closed by @ferranbt a while ago 👍


Add an optional meta field

Hi,
Is it possible to request a meta field for link state? would be nice to be able to show links in UP/DOWN, i'm pulling data from NMS to generate topology so would be able to push this too and display nicely on the map.
P.S: is there a way of offsetting / aligning interface description to middle of the link?
Thank you.

Hi,
inet-henge copies class attribute of node and link to SVG DOM, which means you can change link appearance with CSS.
# .json

"links": [
  { "source": "POP01-bb01", "target": "POP01-bb02",
    "class": "fault",
    "meta": { "foo":  "bar"}
  },
...

<style>
    .link.fault {
        stroke-dasharray: 5, 5;
    }
    .link text .foo {
        font-size: 10px;
    }
</style>

  diagram.init('loopback', 'interface', 'description', 'type', 'foo');

P.S. Add meta to link then call .init('foo') to show something at the middle of the link.

See the source code of this demo for details.


Update mir-algorithm dependency

mir-algorithm has been updated and now no longer works with run.dlang.org.
I get the message
Unresolvable dependencies to package mir-algorithm:
onlineapp ~master depends on mir-algorithm ~>3.7.32

when running the code:
/+dub.sdl:
dependency "mir-algorithm" version="~>3.7.32"
+/

import mir.ndslice.slice: sliced;

void main() {
    auto x = [3.0, 1, 5, 0, 2].sliced;
}

Heads up: @9il

It will be updated somehow after a while

Yep, the current thing looks like they have update the package, but not the website.
dependency "mir-algorithm" version="*" always works

@9il Nice trick!
Should I close this then if it will be updated automatically?

The site should be updated in lockstep, so it is a bug.


Preview callback will unregister DisplayListener

I see that you will unregister the DisplayListener here in this line. In my test, the first onUpdate call will remove the viewFinder and then add it again just like what the code does, but it will unregister the listener right away, so when I try to rotate the device it will never be called.
How this supposes to be?
P/S If I try to register the listener in "onViewAttached..." it will be fine.

The logic inside of AutoFitPreviewBuilder definitely needs work. I'll look into it, but I think that you are correct. Let's keep this issue open until I can spend some time to dig deeper.


Integrate invenio-administration

Add possibility to CRUD banners via the Administration web UI
Notes discussed on demo presentation:
Edit view

[x] * change the Active widget to checkbox/slider
[x] * change the Message input to be multiline
[x] * category field: make the helper text more clear, the categories match to colors/styles... not very clear for the user (they will not know what color to expect?)
[x] * reorder the fields: message first, then category, then URL path, then start/end, then active (or suggestions) - moved to a task https://github.com/inveniosoftware/invenio-administration/issues/155

List view

[x] * make the Message col much larger, it is the most important
[x] * remove Created/Updated/Category/Message from sorting
[x] * remove column Created
[x] * fix the "loading" gif position

missing invenio-banners PR

The PR in invenio-banner has not been merged yet, let's not close the issue until then to avoid losing track @anikachurilova


Unknown operator when using syntastic

Hi @kstep, here you have the info of the bug I told you about. I just have to :w on any recent opened file and I'll get these errors in the blink of an eye. I am using neovim but this is also reproducible with vim. I am using vim-scala as well, which provides Scala highlighting and a lot of other useful stuff (it uses syntastic as well).
Screenshot:

@jvican is this on a separate build of ensime-vim? I just tried repping this on the f-p project and didn't have any issues, despite using Syntastic and vim-scala as well.
Here's my :SyntasticInfo:
Syntastic version: 3.7.0-99 (Vim 704, Darwin)
Info for filetype: scala
Global mode: active
Filetype scala is active
The current file will be checked automatically
Available checkers: ensime fsc scalac
Currently enabled checkers: fsc scalac ensime

could you share yours as well?

Mine:
Syntastic version: 3.7.0-101 (Vim 704, Linux)
Info for filetype: scala
Global mode: active
Filetype scala is active
The current file will be checked automatically
Available checkers: ensime fsc scalac
Currently enabled checkers: fsc scalac

The only difference is that ensime is not listed in Currently enabled checkers.

I actually manually added ensime to my checkers right before posting that
output, so it seems that our setups are identical, yet I don't see any
issues.
Are you running python 2.7 or 3?
On Mar 19, 2016 10:19 AM, "Jorge"<EMAIL_ADDRESS>wrote:

Mine:
Syntastic version: 3.7.0-101 (Vim 704, Linux)
Info for filetype: scala
Global mode: active
Filetype scala is active
The current file will be checked automatically
Available checkers: ensime fsc scalac
Currently enabled checkers: fsc scalac
The only difference is that ensime is not listed in Currently enabled
checkers.
—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
https://github.com/ensime/ensime-vim/issues/200#issuecomment-198710425

You have to do it on the branch feature/add-map-flatmap. But anyway, I just found the error. The problem comes from the server, since I receive these messages (note the weird parameters and, especially, the -1s which are the one causing the error since syntastic expects positive numbers.)
Message:
2016-03-19 19:29:15.519715: unqueue: result received {
  "payload": {
    "typehint": "NewScalaNotesEvent",
    "isFull": false,
    "notes": [{
      "beg": -1,
      "line": 1,
      "col": 0,
      "end": -1,
      "file": "/data/rw/code/scala/f-p/core/src/main/scala/fp/SiloRef.scala",
      "msg": "Unused import",
      "severity": {
        "typehint": "NoteWarn"
      }
    }]
  }
}

I'll be submitting a PR to fix this in some hours. Also, I think I just found the reason why vim gets blocked when exiting.

@fommil is this a new bug in ensime-server?


Reduce regex intersections

Had some discussion about intersecting DFAs here, refer to this repo:
https://github.com/tjjfvi/3FA


The code ran incorrectly

In the initial code, some variable types cannot be correctly identified, for example, Map cannot be detected in some hpp files, how to solve this problem, hope to answer

Please ensure your development environment is correct. You can check if the problem is about the compiler(e.g. g++) or the IDE(e.g. VS Code).
Perhaps you can check the CMakeLists.txt in the code/final folder.


Node.js  Loaders Team Meeting 2021-09-03

Time
UTC Fri 03-Sep-2021 17:00 (05:00 PM):

Timezone
Date/Time

US / Pacific
Fri 03-Sep-2021 10:00 (10:00 AM)

US / Mountain
Fri 03-Sep-2021 11:00 (11:00 AM)

US / Central
Fri 03-Sep-2021 12:00 (12:00 PM)

US / Eastern
Fri 03-Sep-2021 13:00 (01:00 PM)

EU / Western
Fri 03-Sep-2021 18:00 (06:00 PM)

EU / Central
Fri 03-Sep-2021 19:00 (07:00 PM)

EU / Eastern
Fri 03-Sep-2021 20:00 (08:00 PM)

Moscow
Fri 03-Sep-2021 20:00 (08:00 PM)

Chennai
Fri 03-Sep-2021 22:30 (10:30 PM)

Hangzhou
Sat 04-Sep-2021 01:00 (01:00 AM)

Tokyo
Sat 04-Sep-2021 02:00 (02:00 AM)

Sydney
Sat 04-Sep-2021 03:00 (03:00 AM)

Or in your local time:

https://www.timeanddate.com/worldclock/fixedtime.html?msg=Node.js+Foundation+Loaders Team+Meeting+2021-09-03&iso=20210903T1700
or https://www.wolframalpha.com/input/?i=05PM+UTC%2C+Sep+03%2C+2021+in+local+time

Links

Minutes Google Doc: https://docs.google.com/document/d/1sMuJDnUegsD1xJcJnpWcYRk1JakYSEYmbX1p7q4Ajd0/edit

Agenda
Extracted from loaders-agenda labelled issues and pull requests from the nodejs org prior to the meeting.
nodejs/node

esm: consolidate ESM Loader methods #37468

Invited

Loaders team: @nodejs/loaders

Observers/Guests
Notes
The agenda comes from issues labelled with loaders-agenda across all of the repositories in the nodejs org. Please label any additional issues that should be on the agenda before the meeting starts.
Joining the meeting

link for participants: https://zoom.us/j/97506450082

For those who just want to watch: https://www.youtube.com/c/nodejs+foundation/live

@bmeck @GeoffreyBooth I ended up spending every night this week addressing feedback and fighting CI timeouts to nodejs/node#37468, so I have not had time to prep for tomorrow's meeting (or even refresh on the two proposals). It looks like that would be the only item on the agenda; if so, I suggest we cancel this week's and pick up at the next.

I can’t attend at the earlier time either, so let’s cancel.
We should perhaps find a better permanent time; I can open an issue to discuss that in a bit.


Support show icon when has an external monitor

Since the new Macbook Pro 14 inches has a notch, the menu bar space is limited, I only want menu icons to show when plugging an external monitor, when I unplugged it, the icon should be hidden, so it will save menu bar space.
So I introduced a new menu Icon option: Only if has an external monitor.

Hey @Demogorgon314 - I think a similar option is already present in the app. If you select Only if at least one slider is present and disable brightness slider for built-in displays by unticking Enable for Apple branded and built-in displays as well then you should achieve something similar, right?

(note: you are checking against OtherDisplay so in effect the setting you added should have exactly the same result as the approach I described above - external Apple branded displays are AppleDisplay in the app).

Hey @Demogorgon314 - I think a similar option is already present in the app. If you select Only if at least one slider is present and disable brightness slider for built-in displays by unticking Enable for Apple branded and built-in displays as well then you should achieve something similar, right?

Oh, that is true, my mistake, thanks for telling me.

Hey @Demogorgon314 - I think a similar option is already present in the app. If you select Only if at least one slider is present and disable brightness slider for built-in displays by unticking Enable for Apple branded and built-in displays as well then you should achieve something similar, right?

Oh, that is true, my mistake, thanks for telling me.

No problem. But as I see you are good at swift so if you'd like to contribute anything else don't hesitate! Thank you! :)

Hey @Demogorgon314 - I think a similar option is already present in the app. If you select Only if at least one slider is present and disable brightness slider for built-in displays by unticking Enable for Apple branded and built-in displays as well then you should achieve something similar, right?

Oh, that is true, my mistake, thanks for telling me.

No problem. But as I see you are good at swift so if you'd like to contribute anything else don't hesitate! Thank you! :)

Actually, this is my first write the swift code. Please forgive me if there is any mistake and point out my mistake. Thank you! : )


Reword glossary entry for ReplicaSet

Explain what a ReplicaSet is, rather than what a ReplicationController isn't.

Makes sense to me! (And is accurate wrt to how RS' work within the context of a deployment)
/lgtm
/approve


aya-bpf: why BPF_MAP_TYPE_ARRAY doesn't have set() method?

in aya, Array has set() method
https://github.com/aya-rs/aya/blob/main/aya/src/maps/array/array.rs#L115-L125
...but aya-bpf Array doesn't. why?
https://github.com/aya-rs/aya/blob/main/bpf/aya-bpf/src/maps/array.rs#L19-L60
I thought BPF_MAP_TYPE_ARRAY is mutable from bpf side. am I missing something?
https://prototype-kernel.readthedocs.io/en/latest/bpf/ebpf_maps_types.html

It's indeed mutable. I already have a PR adding get_ptr and get_mut_ptr methods. I can also add a set/insert method there.

@vadorovsky looks like we still don't have get_mut?

@vadorovsky looks like we still don't have get_mut?

It has get_ptr_mut as the other map do.
https://github.com/aya-rs/aya/blob/33baf7ef2203a27c69ff640d17fb382fca870c5c/bpf/aya-bpf/src/maps/array.rs#L61-L64
The question is whether we want to have set() method, to give people better API then using raw pointers. I can work on that.


[openssl] Fix arm-neon-android asm

Follow-up from #33890, tentative arm-neon-android asm patch.
Patch submitted upstream: https://github.com/openssl/openssl/pull/22181, tagged "severity: fips change".

CC @ankurvdev

Tagging reviewed because the change seems OK to me, upstream seems responsive, so I want to remember to check every day...

I poked upstream again today

@tom-cosgrove-arm of upstream says:

@BillyONeal I don't think the final fix will look like this, but not sure exactly what it should be at the moment. This is now top of my OpenSSL list, though
(The reason I don't think the final fix will look like this is at a minimum the duplication, and I think the fix will need something in arm-xlate.pl too)

so I dropped reviewed for now.

It can't go into OpenSSL without some Perl so that even arm64-vms is happy :smile:

@dg0yt Can you solve the conflicts?

Based on upstream's comment in https://github.com/openssl/openssl/pull/22181#issuecomment-2185322378 I'm going to take this.

Thanks for the fix and sorry for the runaround with upstream!


feat(aws): Add MFA flag if try to assume role in AWS

Context
Currently you cannot assume roles in AWS if the roles are using MFA
Description
This PoC adds "--mfa" flag which asks user for MFA ARN + Code to authenticate
License
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

I added the test and strip() to the input. I have a couple of questions:

Where do you update the documentation?
Do you want the flag to be --mfa, or would you like it to be more specific to the role assumption? For example --role-mfa?

I added the test and strip() to the input. I have a couple of questions:

Awesome!

Where do you update the documentation?

The tutorial for the Role assumption in Boto3 is here https://github.com/prowler-cloud/prowler/blob/master/docs/tutorials/aws/role-assumption.md

Do you want the flag to be --mfa, or would you like it to be more specific to the role assumption? For example --role-mfa?

Good question, we could use --role-mfa to be more specific. Please change the references in the code and tests.
Thank you for your contribution!


Adding block device support

Adding support for Block VolumeMode in addition to FileSystem.
Block devices are created as files mounted on loopback.
Updating scripts to take in VolumeMode as first argument and path as second argument.

@tryggvil Can you take a look at https://github.com/rancher/local-path-provisioner/pull/127 ? That approach looks good.

Closing the PR, it seems pending for a while.
@tryggvil feel free to contribute again if #127 not enough for you. Thanks.


Mozilla location services on the map

There's a beautiful dataset that Mozilla publicly gives as a part of Mozilla Location Services
https://location.services.mozilla.com/downloads
Can we do something about this as a possible data source into Loklak ? This issue is just a question on which we could brainstorm and see what we could do, the data sample looks as follows

Awesome! Let's put this on the map. Which milestone does that fit in?

Closing this due to inactivity. Currently it is not a focus, I believe.


Documentation for cloning a specific folder in git to work on PR

Just a note but we need to update the documentation for the community on how to pull the repo and specify a folder.  So if they want to work on bootstrap specifically they don't have to pull the entire repo down.  Is this possible ?

They must pull all the repo

Ok sounds good thanks for clarifying


Configure Docker Environment and Project Structure

Should configure proper Docker env and project structure.

Use https://github.com/joyofpw/docker/blob/master/docker/processwire/Dockerfile
As base


Turkish Typo Fixes

Turkish language ISO code is "TR" not "TUR". Also there are some typo mistakes in Turkish translation.

Were you able to run with "tr"? In someway it always route to /tr even if you select "en". This is why I used "tur".

It automatically detects and redirects from the browser language. To turn it off, localeDetection must be set to false in the i18n setting. This situation works in the other languages as well. As another option, the English language address can be changed to /en in dropdown.

O see, thanks for explanation and typo fix 🙏


ci.yaml validation check in presubmits hangs for many hours.

An example of such a hang (that is still running for 3hours and 14mins as of this writing) is here. This is blocking commits to the engine.
Attempts to debug the failure are hampered by the "View more details on flutter-dashboard" link pointing to a the Flutter dashboard root page.

/cc @CaseyHillers do you mind help take a look?

This is catching true issues preventing a PR from landing without tests. In general, you should add an empty commit to retrigger the tests.
Lowering priority as other recent PRs have checks (https://github.com/flutter/engine/pull/27015). Feel free to raise again if I missed anything
A general solution to work around this is flutter/infra should have a vacuum cron job to detect when checks are idle for too long, and retry them.

This is catching true issues preventing a PR from landing without tests.

That this catches true issues isn't the .... issue. Right now it hangs running for many hours with no logs or guidance on what went wrong.

@CaseyHillers can we close this one or is there pending work?

This issue still occurs. The next planned work is to allow retrying the check

Retrying the check has been enabled. Closing this as this issue hasn't been occurring.


get error when enters 'sudo sift install'

after I enter sudo sift install i get a error like this

how do i solve this

Hi @wilsonLam2002 , can you confirm that you have internet access on that machine? If so, you can use the new Cast installer which will be the replacement for the sift CLI you're currently using.
Visit the link above and download the release for your system and architecture, then install it using sudo dpkg -i <downloaded_file>.
Once installed, you can use the command: sudo cast install sift. If you want to use a different mode (ie: server), you would modify it with sudo cast install --mode server sift.
If you give that a shot, it should work. Let us know how that goes and we can go from there.

thanks for the info, but after i enter sudo dpkg -i command it returns dpkg: error: archive 'downloaded file name' is not a regular file. did i download the wrong file?

No you likely didn't. When I put <downloaded_file> that's a placeholder for whatever the actual name is of the file you downloaded, since I don't know the name of the file you chose to download. Try it again with the actual file name.

@wilsonLam2002 The file name should end in .deb so you'll have to include that in the command:
sudo dpkg -i cast_v0.14.0_linux_amd64.deb
And make sure you run the command in the directory where you downloaded the file.

thank you very much it worked

i've just finish installing, but shouldn't it be login in to sansforensics?

Hi @wilsonLam2002 , the username should be your username, and it should be set up with all of the options which come with SIFT if the install completed successfully. Right now, only the VM has the default user of sansforensics.

Hi @wilsonLam2002 , did the last answer I provided resolve your issue / question?

Hi @wilsonLam2002 , did the last answer I provided resolve your issue / question?

Oh yes, thanks for the help

I'll close this page.

Hahaha


Cap on weekly DKP

Within Essential, DKP that can be earned per week, is capped to 20 DKP (10 DKP per evening), even if a person would be on the bench for a 3rd night. In TBC, we foresee the possibility of having a cap of 20 DKP even if a person raids 3 nights (2 evenings of Kara, 1 for Gruul).
Could we make it so that that weekly DKP gain is capped to a certain amount?
Example:

Bob gets 10 DKP on Wednesday for joining Kara, first couple bosses
Bob gets 10 DKP on Thursday for joining Gruul.
Bob would get 10 DKP on Monday for joining Kara continuation, but that would be >20 DKP, so he doesn't actually get the DKP added.

The big advantages of this weekly cap are:
A. The DKP-awarder doesn't have to check for every individual in the raid whether he got DKP-to-cap already. Can simply award the same amount of DKP to everybody in the raid.
B. When applying DKP for Bench, it doesn't matter if you accidentally also give to somebody who was also in the raid.

And because of the way the addon works, if the 2nd entry "Bob gets 10 DKP on Thursday for joining Gruul" is ignored from the calculation, Bob's weekly total would still be 20 DKP, because the 3rd entry ("Monday Kara continuation") would then 'become active' (no longer affected by the cap).

For clarification: the weekly cap would be per raid week (so starting on Wednesday in the EU).

Week will start on weekly reset.
It will be based on GetServerTime() thus linux epoch.
We will do a week number calculation based on reset time offsets from GMT
543600 EU
486000 US


IpType validation moved to value method

CHANGES update missing

ack

commited to 0.46 with https://github.com/crate/crate/commit/ca1127ed1dd1262d213fcc0a79c8900006caf1de


Local backend and minor fixes

This PR

adds a local phonebook (json file) backend that takes precedence over the online directories.
fixes incorrect counts in the response message (in case of multiple matching results)
adds a production WSGI server
starts using a phonenumber library for parsing and formatting
returns also the phone number in the hm field

Hi @rueckix, thanks for your PR, and merry christmas!
I do not have the Gigaset base station in active use any more, so I won't work on this in the future. If you want to take over the maintenance of this project, I could add a link to your repo.

Hi @rueckix, thanks for your PR, and merry christmas!
I do not have the Gigaset base station in active use any more, so I won't work on this in the future. If you want to take over the maintenance of this project, I could add a link to your repo.

Merci. Happy Holidays!
Sure, I could take over the maintenance via my fork. In the meantime, thanks for the suggestions and hints! This will further improve the code.


wrong number of arguments (2 for 1) in site.sass

When the compass-rails gem is installed in my Rails 4.0.4 application I get wrong number of arguments (2 for 1) on all of my sass files. When removing the gem, the site works properly.

I've just installed this:
gem "compass-rails", github: "Compass/compass-rails", branch: "master"
and it works.
found it her https://github.com/Compass/compass-rails/issues/195

+1 @Pjero

+1, using compass-rails on master fix the issue with Rails 4.1.8 and sass-rails 5.0.1

Use version 2.0.2 it has what you need.

After trying to upgrade to sass-rails 5.0.1 I started receiving the error when using image_tag in my erb. Didn't throw any exceptions when compiling the SASS file.
Went through a lot of combinations from this thread but was able to fix the issue by just downgrading sass-rails to 4.0.3.

I've got the same problem after bundle update.
I don't know what the exactly problems is, but the following combination fixed it.
* compass-rails (2.0.4)
* sass-rails (5.0.1)
* sprockets (2.12.3)
* sprockets-rails (2.2.4)

You can specify gem versions in your Gemfile like:
gem 'sass-rails', '~> 5.0'
gem 'compass-rails', '~> 2.0.4'

See: Ruby's Pessimistic Operator
Hope it helps.

Thank you guys so much. I had a similar problem and @Pjero worked for me. THANK YOU!

+1 @Pjero

@kakipo thank you!
working combination of gem versions
gem 'sass-rails', '~> 5.0'
gem 'compass-rails', '~> 2.0.4'

@kakipo thank you!
working combination of gem versions:
gem 'sass-rails', '~> 5.0'
gem 'compass-rails', '~> 2.0.4'


Integrate expressions/functions with DSC

PR Summary

Removed resourceId() regex and replaced with a function
Add execution of expressions to get/set/test resource properties
This integration allows use of concat() and base64() functions within a configuration
Change from HashMap to serde Map for resource properties so don't need to transform from one type to the other due to nested properties
Expressions are evaluated only for resource properties at invocation time
NOTE: when converting Value to string, you have to use as_str() instead of to_string() as the latter includes outer quotes

PR Context
Partially implements https://github.com/PowerShell/DSC/issues/57

The test failure is due to not handling nested types in expression invocation so fixing that.

The remaining failure is due to properties that get returned may look like an expression so if you pipe that back to dsc, it fails.  I think the right thing to do here is if it's retrieved via config, then it needs to escape it if it starts and ends with brackets.


Angular 5 issue

I'm submitting a ...  (check one with "x")
[x] bug report => search github for a similar issue or PR before submitting
[ ] feature request
[ ] support request => Please do not submit support request here, post on Stackoverflow or Gitter

Current behavior
After upgrading to angular 5 and also after updating version to ^11.0.0 i am getting below run time error when i try with  ng serve -aot, with ng serve it is working fine as expected
webpack-internal:///../../../core/esm5/core.js:1546 ERROR Error: Uncaught (in promise): TypeError: Cannot read property 'subscribe' of undefined
TypeError: Cannot read property 'subscribe' of undefined
Expected behavior
It should work without any error
Reproduction of the problem
update angular version to 5 and ngx-datatable version to  11.0.0 and run the angular project with ng serve -aot command.
What is the motivation / use case for changing the behavior?

Please tell us about your environment:

Windows 7
visual studio code

Table version: 0.8.x
11.0.0

Angular version: 2.0.x
5.0

Browser: [all | Chrome XX | Firefox XX | IE XX | Safari XX | Mobile Chrome XX | Android X.X Web Browser | iOS XX Safari | iOS XX UIWebView | iOS XX WKWebView ]
Chrome latest

Language: [all | TypeScript X.X | ES6/7 | ES5]
typescript 2.4.0

Sounds like a RxJS/TypeScript error.
After upgrading to Angular 5 and angular-cli 1.5, I was not able to build until I updated TypeScript from 2.4 to 2.5
So try TypeScript 2.5.3 (latest) and see if it works.

Latest Typescript is 2.6.1

The same issue: ERROR Error: Uncaught (in promise): TypeError: Cannot read property 'subscribe' of undefined. TypeError: Cannot read property 'subscribe' of undefined
version: 11.0.0
Angular: 5.0.0
cli: 1.5.0
node: v8.7.0
typescript: 2.5.0

I have same error

What is your rx version?

I have opened issue with Angular since it seems more relevant there: https://github.com/angular/angular/issues/20240

I had the same issue.
Cloning the repo and including the typescript sources themselves fixed it. It seems that the build is to blame here.

@solnat Can you elaborate on steps, and more importantly can you create pull request with change so that this fix becomes part of next minor version.

@arsyed04 There was no code changes needed for me. A working fix was to do
import {NgxDatatableModule} from '@swimlane/ngx-datatable/src';
instead of
import {NgxDatatableModule} from '@swimlane/ngx-datatable';
Building in AoT straight from the typescript source instead of importing the dist fixes the issue. I have no idea why though, didn't bother to look into the build process of this library.

Confirmed, AOT only issue.
If you're using angular-cli, this can be reproduced in development mode with: ng serve --aot
The exception is raised from this code.
At the line with: instance[output.propName].subscribe(...
function createDirectiveInstance(view, def) {
    // components can see other private services, other directives can't.
    var /** @type {?} */ allowPrivateServices = (def.flags & 32768 /* Component */) > 0;
    // directives are always eager and classes!
    var /** @type {?} */ instance = createClass(view, /** @type {?} */ ((def.parent)), allowPrivateServices, /** @type {?} */ ((def.provider)).value, /** @type {?} */ ((def.provider)).deps);
    if (def.outputs.length) {
        for (var /** @type {?} */ i = 0; i < def.outputs.length; i++) {
            var /** @type {?} */ output = def.outputs[i];
            var /** @type {?} */ subscription = instance[/** @type {?} */ ((output.propName))].subscribe(eventHandlerClosure(view, /** @type {?} */ ((def.parent)).nodeIndex, output.eventName)); /** @type {?} */
            ((view.disposables))[def.outputIndex + i] = subscription.unsubscribe.bind(subscription);
        }
    }
    return instance;
}

For me, it happens with propName='scroll'
instance: DatatableComponent
output: {
  eventName: "scroll"
  propName: "scroll"
  target: null
  type: 1
}

instance['scroll'] === undefined,  which is the source of this error.

I suspect the issue is here:
src/components/body/scroller.component.ts
  ngOnInit(): void {
    // manual bind so we don't always listen
    if (this.scrollbarV || this.scrollbarH) {
      this.parentElement = this.element.parentElement.parentElement;
      this.onScrollListener = this.renderer.listen(
        this.parentElement, 'scroll', this.onScrolled.bind(this));
    }
  }

I don't know what renderer.listen() does without doing more research.
Does anyone see how this breaks with Angular 5?

@arlowhite renderer.listen() is required to do virtual scrolling. Without it, the entire list would have to be pre-rendered, and that's not scalable.

I think they removed the need for doing the renderer hooking like this in ng5. I need to confirm though.

@solnat 's trick got me past the subscribe error.
import {NgxDatatableModule} from '@swimlane/ngx-datatable/src';
Although I had to fix some TypeScript errors (implicit any and not all code paths returning a value).
 Otherwise things like @ViewChild(DatatableComponent) will break.

In that situation, your project is compiling the typescript more than likely thus it working. The packaging of aot files must have been busted with the upgrade.

Looks like Material is having this issue too - https://github.com/angular/material2/issues/8229#issuecomment-342502556

Still get the subscribe error with 11.0.2

@arlowhite - published 11.0.3 with your PR.

@solnat import {NgxDatatableModule} from '@swimlane/ngx-datatable/src' works only with AoT. in normal build it gives error. And import {NgxDatatableModule} from '@swimlane/ngx-datatable' works only in normal build with AOT it gives error.
I am using 11.0.3.
Any solution?

Hi, same problem here, cannot build with AOT that receive "TypeError: Cannot read property 'subscribe' of undefined", but if i change the import works with AOT but does not work without...

If you're having an AOT error, be sure to update to the latest version. It looks like that is addressed in it.

Hello,
Any update on this? Can anyone please ind out the solution for this? Its urgent !!!

The solution mentioned earlier working well to use for now.

@bopm so for you Is it working for AOT as well as non AOT  build?

we downgraded to 10.4.0.

@rsaulo i am using angular 5. so will it work for that version?

We too, it's working, following our current setup :
Angular CLI: 1.6.0-beta.0
Node: 9.0.0
OS: darwin x64
Angular: 5.0.0
... animations, common, compiler, compiler-cli, core, forms
... http, platform-browser, platform-browser-dynamic
... platform-server, router
@angular/cdk: 5.0.0-rc0
@angular/cli: 1.6.0-beta.0
@angular/flex-layout: 2.0.0-beta.10
@angular/material: 5.0.0-rc0
@angular-devkit/build-optimizer: 0.0.32
@angular-devkit/core: 0.0.20
@angular-devkit/schematics: 0.0.35
@ngtools/json-schema: 1.1.0
@ngtools/webpack: 1.9.0-beta.0
@schematics/angular: 0.1.2
typescript: 2.4.2
webpack: 3.8.1

@rsaulo Yes, it's working...

11.0.3 with AOT is NOT working. ('subscribe' of undefined)
Angular 5.0.1 and angular-cli 1.5.0
This only affects AOT builds. Please double-check that you are building with --aot and aren't still using the import from 'ngx-datatable/src' work-around.
I also updated to Angular CLI: 1.6.0-beta.0, but it didn't help.

Sadly, it's not working without aot with that workaround. So downgraded too.

@bopm import { } from 'ngx-datatable/src' only works when using AOT, so if you use this work-around, you need to use --aot all the time, even during development. i.e. ng serve --aot

i hava the same issue with angular 5 and ngx-datatable ^11.0.3
with --aot must with import {NgxDatatableModule} from '@swimlane/ngx-datatable/src';
without --aot must with import {NgxDatatableModule} from '@swimlane/ngx-datatable';

The only solution for now is downgrading to 10.4.0.

@arlowhite - can you make me a demo project to test w/?

@amcdnl Here's a new angular-cli 1.5.0 project that reproduces the error:
https://github.com/arlowhite/simple-ngx-datatable
Just run:

npm install
npm start

npm start is configured in package.json to run in AOT mode: ng serve --aot

I have the same problem, my project is broken when compiled with aot

Same issue. :S

+1 for aot not working (Angular 5.0.1)

Issue persists with Angular 5.0.1.

Huge thanks for the pointer to the /src workaround, that got me back up and running having been scratching my head for a while over this. I take the point about the /src workaround not being so handy though as it makes non AOT development super slow.
The workaround I have in place is to modify the tsconfig-aot.json file (in the ngx-datatable project) as described here https://github.com/angular/angular/issues/19587
Having made the above change, recompiled ngx-datatable and reference that version instead I can use AOT and non AOT without the /src.
It sounds like the problem is to do with the TS version not being fully supported by Angular 5 as yet. The referenced solution is probably non ideal and should be backed out once Angular 5 supports TS 2.5+

Please try 11.0.4

Both regular and aot compilations working on Angular 5.0.1 ;)
Thanks a lot for a fix!

Yes, it works! Thanks!

It works, thanks!

Success.  Thank you

Working. Thanks.

Thank you @amcdnl for the fix. I am closing this issue as it is resolved.

@amcdnl thanks


Feat: install fontbm

The Moddable SDK uses the fontbm tool to allow projects to incorporate TrueType / OpenType fonts at arbitrary sizes. The fontbm tool is installed separately from the Moddable SDK and made available by the $FONTBM environment variable.
As an experiment to learn a bit more about xs-dev. I implemented a naive fontbm command to install fontbm on macOS. Despite limited experience with Node and this being my first encounter with Gluegun, it was reasonably straightforward to implement.
A few questions:

Should this be its own command or an option for setup? It isn't a development platform so setup seemed a bit awkward
If it is a separate command, is it reasonable to put the Mac, Windows, and Linux support all in a single source file?
Is there a better way to check that FreeType is installed other than the hardcoded path used here? (It feels fragile)
What other obvious things did I get wrong? ;)

Thanks for the quick and thorough feedback.

I would like to include this feature under the setup command, but I agree, it doesn't make sense as a "device" or target platform. Maybe a --tool flag would be better. Or if fontbm is considered a project dependency, maybe collecting third-party dependencies under an install command makes sense: xs-dev install fontbm

I think I prefer setup with --tool since most of what this does is just like what setup does (pull repo, check dependencies, build, set-up environment variables). I'm not sure how to do that. Would you prefer that I take a shot at that or maybe you should set it up the way you intend?
which freetype-config seems like the best way to check for FreeType. I'll do that.

I have been separating OS-specific commands, i.e. installing dependencies, and keeping agnostic commands, i.e. git clone in the same file.

 (ESP32 Windows set-up, by contrast, is so different than macOS that your approach makes perfect sense). But, I'm happy to follow your lead.

It's interesting that fontbm has prebuilt binaries for Linux and Windows, so xs-dev could reach for those compiled releases instead of building manually. It would be great to include pre-built macos binaries in those releases too.

I deliberately set aside the topic of prebuilt binaries here. Dealing with prebuilt Moddable SDK binaries (and others, like fontbm) seems like it might take some more thought.

should a specific tagged release by cloned instead of pulling the master branch? This could ensure only confirmed working versions of fontbm are installed, same as the ESP tooling versions.

I noticed that in some of your code. As usual, the tradeoff is between benefiting from improvements automatically and getting broken by them by surprise. ;) Is that a case-by-case choice, or is it considered best-practice to use a tag?

adding a success message at the end will help let users know how to continue after the install is complete

Good point. Will do.

I think I prefer setup with --tool since most of what this does is just like what setup does (pull repo, check dependencies, build, set-up environment variables). I'm not sure how to do that. Would you prefer that I take a shot at that or maybe you should set it up the way you intend?

You should be able to keep most of what you have as the run function and move it into a module within the src/toolbox/setup directory. Instead of getting a toolbox argument to the function, the built-in helpers can be imported from the gluegun package.
In the src/commands/setup.ts module, look for a tool field in the setup command's parameter options: https://github.com/HipsterBrown/xs-dev/blob/main/src/commands/setup.ts#L16
If that tool field exists with a value of fontbm, then your fontbm module should be called similar to this.

This is probably fine, especially since it seems like the dependency installation (cmake) is the main difference.

I deliberately set aside the topic of prebuilt binaries here. Dealing with prebuilt Moddable SDK binaries (and others, like fontbm) seems like it might take some more thought.

That's fair. It is a larger conversation that should be considered separately.

As usual, the tradeoff is between benefiting from improvements automatically and getting broken by them by surprise. ;) Is that a case-by-case choice, or is it considered best-practice to use a tag?

I mostly followed the guidance of the Moddable docs whenever a specific branch was mentioned. And I did run into a breaking change when using the main branch of the wasm setup, which led to this PR: https://github.com/Moddable-OpenSource/moddable/pull/786
If we can depend on a tagged version, I would prefer it so we can safely upgrade as needed.

I think I've implemented everything above (other than Linux support).
fontbm is now installed with xs-dev setup --tool fontbm. The source moved to toolbox/setup and the setup command is modified to support --tool. I imagine that eventually you would want a list of tools there rather than a special case for fontbm, similar to how devices is handled.

I imagine that eventually you would want a list of tools there rather than a special case for fontbm, similar to how devices is handled.

Yes, exactly.
Can you create a pull request with your changes? Looking forward to the first external contribution. 😊

This feature has been released in version 0.5.0: https://github.com/HipsterBrown/xs-dev/blob/main/CHANGELOG.md#050
Thanks again for the contribution! 👏 🎉


added documentation for using Web Font Loader as CommonJS module

It's not documented anywhere and I had to search for it to come across #226 (and then I had to guess/search the package name on npm too)
To save others some time it should be documented somewhere.

This is a good idea. One small thing needs to be fixed though, the package name is incorrect.

I guess that was some copy/paste mistake

I'll merge it in, thanks a lot!


Added options for live and screen sleep

In this PR I've added two options to hide time and progress bar which is called "isLive" and another one to allow keep device screen on while in full screen (allowedScreenSleep).
Issues: #64 #44

Looks great, thanks!