v1.0.0: deploy examples to github pages.

@gioboa will be very useful to add extra steps to our github action to deploy examples to github pages.
This article covers it:
https://javascript.plainenglish.io/deploying-a-vite-app-to-github-pages-using-github-actions-a-step-by-step-guide-ff63d93e4da2
we just need to repeat it twice. Once for host and once for remote.

I can deploy the examples on my CloudFlare account too. I'm wondering if it is better to have these examples in the main repo with all the other module-federation examples

probably best is to keep here, as thanks to you it is easier to maintain
them.
On Thu, 15 Aug 2024, 13:06 Giorgio Boa, @.***> wrote:

I can deploy the examples on my CloudFlare account too. I'm wondering if
it is better to have these examples in the main repo with all the other
module-federation examples
—
Reply to this email directly, view it on GitHub
https://github.com/module-federation/vite/issues/57#issuecomment-2291150874,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAI5PLBW6AFX5Y33VCGWVIDZRSKWDAVCNFSM6AAAAABMSCLAQGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOJRGE2TAOBXGQ
.
You are receiving this because you authored the thread.Message ID:
@.***>

@husayt what are the benefits to have the examples online?

since they are static bundles, no node server is needed. the benefit is that they will help a lot to understand what this project does. Otherwise, one has to check out, install, and run locally just to try them. Examples are also playingvthe role of the documentation for us currently
btw, @gioboa, great article you posted on this effort, and thanks for mentioning me. Seems you did not have my name and just put username. It's Huseyn Guliyev if you want to have a full name there.

Yep, thanks 😂
I think it's time consuming to maintain all the examples with a high quality. we can grab few screenshots for all of them and deploy the most complicated and complete one. With this approach we will reduce the effort in long term.

Absolutely, one deployed example will be enough at this stage

Let's deploy this one 👍

Do you have anything deployed yet? I can't for the life of me make this work in a deployed app, it would be nice to have an app that works deployed and code for it, so we can just compare what's different.

Do you have anything deployed yet? I can't for the life of me make this work in a deployed app, it would be nice to have an app that works deployed and code for it, so we can just compare what's different.

Found an example that will actually run from the built assets folder, it's the official one as well, so my bad I didn't start with it:
https://github.com/gioboa/react-microfrontend-demo?tab=readme-ov-file


Could not find a version that satisfies the requirement spleeter

hi, I use macos and system version is 10.15.4.
I use pip or pip3 both shows the error,
ERROR: Could not find a version that satisfies the requirement spleeter (from versions: none)
ERROR: No matching distribution found for spleeter.
so... any solution? or it has to be done within conda?
thank you!

Hi @Esmee-Zhang
Please fill up a complete bug report with more details. Are you using python 3.8 ?

i have the same issue, running Manjaro on kernel 5.6 with python 3.8 installed. othe packages upgrade without a problem using pip.

I am also getting the same error, using python3.8.2, could that be the issue, on 7.5 it worked.

We are not supporting Python 3.8, please downgrade or use a virtualenv.


forge ingots

so, there's a problem with the ores where when you hover over some/most of the weapons same with the Spartan shields any answers.

Those are modded materials and that text means that there's no mod adding the corresponding ingot, try to instal a mod with those(immersive engineering for example) to see if the text dissapears


runtime: OOM when runing in sparse available memory system

What version of Go are you using (go version)?

$ go version
go version go1.11 linux/amd64

Does this issue reproduce with the latest release?
YES
What operating system and processor architecture are you using (go env)?
go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/data/golang"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go1.11"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go1.11/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build473055106=/tmp/go-build -gno-record-gcc-switches"

What did you do?

When I use  go1.11 to compile elastic metricbeat with tags v7.0.0，the binary file would run out of memory when the system available memory is sparse.
The system memory useage is like this:

free -m
              total        used        free      shared  buff/cache   available
Mem:           1868        1686          14          97         167          43
Swap:             0           0           0

What did you expect to see?
The system  available memory is sparse, But Metricbeat memory usage would less then 10MB when it started. So It could be started.
What did you see instead?
It can't start instead of oom.
The oom runtime stack logs are:

fatal error: runtime: out of memory

runtime stack:
runtime.throw(0x283d1da, 0x16)
        /usr/local/go1.11/src/runtime/panic.go:608 +0x72 fp=0x7ffc2108ab90 sp=0x7ffc2108ab60 pc=0x11123e2
runtime.sysMap(0xc000000000, 0x4000000, 0x3e26558)
        /usr/local/go1.11/src/runtime/mem_linux.go:156 +0xc7 fp=0x7ffc2108abd0 sp=0x7ffc2108ab90 pc=0x10fd687
runtime.(*mheap).sysAlloc(0x3e0c560, 0x4000000, 0x0, 0x0)
        /usr/local/go1.11/src/runtime/malloc.go:619 +0x1c7 fp=0x7ffc2108ac58 sp=0x7ffc2108abd0 pc=0x10f14d7
runtime.(*mheap).grow(0x3e0c560, 0x1, 0x0)
        /usr/local/go1.11/src/runtime/mheap.go:920 +0x42 fp=0x7ffc2108acb0 sp=0x7ffc2108ac58 pc=0x11099c2
runtime.(*mheap).allocSpanLocked(0x3e0c560, 0x1, 0x3e26568, 0x0)
        /usr/local/go1.11/src/runtime/mheap.go:848 +0x337 fp=0x7ffc2108acf0 sp=0x7ffc2108acb0 pc=0x1109847
runtime.(*mheap).alloc_m(0x3e0c560, 0x1, 0x2a, 0x0)
        /usr/local/go1.11/src/runtime/mheap.go:692 +0x119 fp=0x7ffc2108ad30 sp=0x7ffc2108acf0 pc=0x1109059
runtime.(*mheap).alloc.func1()
        /usr/local/go1.11/src/runtime/mheap.go:759 +0x4c fp=0x7ffc2108ad68 sp=0x7ffc2108ad30 pc=0x113ef1c
runtime.(*mheap).alloc(0x3e0c560, 0x1, 0x101002a, 0x7ffc2108add0)
        /usr/local/go1.11/src/runtime/mheap.go:758 +0x8a fp=0x7ffc2108adb8 sp=0x7ffc2108ad68 pc=0x11092fa
runtime.(*mcentral).grow(0x3e0e318, 0x0)
        /usr/local/go1.11/src/runtime/mcentral.go:232 +0x94 fp=0x7ffc2108ae00 sp=0x7ffc2108adb8 pc=0x10fd084
runtime.(*mcentral).cacheSpan(0x3e0e318, 0x0)
        /usr/local/go1.11/src/runtime/mcentral.go:106 +0x2f8 fp=0x7ffc2108ae48 sp=0x7ffc2108ae00 pc=0x10fcbd8
runtime.(*mcache).refill(0x7f237641b000, 0x220000002a)
        /usr/local/go1.11/src/runtime/mcache.go:122 +0x95 fp=0x7ffc2108ae78 sp=0x7ffc2108ae48 pc=0x10fc795
runtime.(*mcache).nextFree.func1()
        /usr/local/go1.11/src/runtime/malloc.go:749 +0x32 fp=0x7ffc2108ae98 sp=0x7ffc2108ae78 pc=0x113e332
runtime.(*mcache).nextFree(0x7f237641b000, 0x3e2652a, 0x4000, 0x7f237641b000, 0x7ffc2108af58)
        /usr/local/go1.11/src/runtime/malloc.go:748 +0xb6 fp=0x7ffc2108aef0 sp=0x7ffc2108ae98 pc=0x10f1b86
runtime.mallocgc(0x180, 0x2807a20, 0x7ffc2108af01, 0x7f237641f000)
        /usr/local/go1.11/src/runtime/malloc.go:903 +0x793 fp=0x7ffc2108af90 sp=0x7ffc2108aef0 pc=0x10f24d3
runtime.newobject(0x2807a20, 0x3e265c0)
        /usr/local/go1.11/src/runtime/malloc.go:1032 +0x38 fp=0x7ffc2108afc0 sp=0x7ffc2108af90 pc=0x10f28b8
runtime.malg(0x7f2300008000, 0x7f237641b000)
        /usr/local/go1.11/src/runtime/proc.go:3285 +0x31 fp=0x7ffc2108b000 sp=0x7ffc2108afc0 pc=0x111c661
runtime.mpreinit(0x3e06840)
        /usr/local/go1.11/src/runtime/os_linux.go:311 +0x29 fp=0x7ffc2108b020 sp=0x7ffc2108b000 pc=0x1110819
runtime.mcommoninit(0x3e06840)
        /usr/local/go1.11/src/runtime/proc.go:624 +0xc1 fp=0x7ffc2108b058 sp=0x7ffc2108b020 pc=0x1115f71
runtime.schedinit()
        /usr/local/go1.11/src/runtime/proc.go:546 +0x89 fp=0x7ffc2108b0c0 sp=0x7ffc2108b058 pc=0x1115c39
runtime.rt0_go(0x7ffc2108b1c8, 0x1, 0x7ffc2108b1c8, 0x0, 0x7f2375b95c05, 0x2000000000, 0x7ffc2108b1c8, 0x100000000, 0x1140e50, 0x0, ...)
        /usr/local/go1.11/src/runtime/asm_amd64.s:195 +0x11a fp=0x7ffc2108b0c8 sp=0x7ffc2108b0c0 pc=0x1140f7a

BUT
When I use go1.10.8 to build the same elastic metricbeat code, It started without oom and the  system available memory is also about 40 MB left.
Is there any breaking changes start from go version 1.11?

Go 1.11 is no longer supported, http://golang.org/s/release. Can you please re test with Go 1.14.

@davecheney Thanks for your reply,I have tested with the latest release Go1.14.4. the result is same.
The runtime stack logs are:

fatal error: runtime: out of memory

runtime stack:
runtime.throw(0x26d9370, 0x16)
        /usr/local/go1.14.4/src/runtime/panic.go:1116 +0x72 fp=0x7fff2e91e338 sp=0x7fff2e91e308 pc=0x1034092
runtime.sysMap(0xc000000000, 0x4000000, 0x3afca58)
        /usr/local/go1.14.4/src/runtime/mem_linux.go:169 +0xc5 fp=0x7fff2e91e378 sp=0x7fff2e91e338 pc=0x1017d55
runtime.(*mheap).sysAlloc(0x3ae7920, 0x400000, 0x0, 0x0)
        /usr/local/go1.14.4/src/runtime/malloc.go:715 +0x1cd fp=0x7fff2e91e420 sp=0x7fff2e91e378 pc=0x100b17d
runtime.(*mheap).grow(0x3ae7920, 0x1, 0x0)
        /usr/local/go1.14.4/src/runtime/mheap.go:1286 +0x11c fp=0x7fff2e91e488 sp=0x7fff2e91e420 pc=0x102600c
runtime.(*mheap).allocSpan(0x3ae7920, 0x1, 0xd352a00, 0x3afca68, 0x1)
        /usr/local/go1.14.4/src/runtime/mheap.go:1124 +0x6a0 fp=0x7fff2e91e508 sp=0x7fff2e91e488 pc=0x1025d50
runtime.(*mheap).alloc.func1()
        /usr/local/go1.14.4/src/runtime/mheap.go:871 +0x64 fp=0x7fff2e91e560 sp=0x7fff2e91e508 pc=0x10630e4
runtime.(*mheap).alloc(0x3ae7920, 0x1, 0x12a, 0x1)
        /usr/local/go1.14.4/src/runtime/mheap.go:865 +0x81 fp=0x7fff2e91e5b0 sp=0x7fff2e91e560 pc=0x10252b1
runtime.(*mcentral).grow(0x3af87f8, 0x0)
        /usr/local/go1.14.4/src/runtime/mcentral.go:255 +0x79 fp=0x7fff2e91e5f0 sp=0x7fff2e91e5b0 pc=0x1017779
runtime.(*mcentral).cacheSpan(0x3af87f8, 0x0)
        /usr/local/go1.14.4/src/runtime/mcentral.go:106 +0x2bc fp=0x7fff2e91e638 sp=0x7fff2e91e5f0 pc=0x10172ac
runtime.(*mcache).refill(0x7f8695204108, 0x2a)
        /usr/local/go1.14.4/src/runtime/mcache.go:138 +0x85 fp=0x7fff2e91e658 sp=0x7fff2e91e638 pc=0x1016d95
runtime.(*mcache).nextFree(0x7f8695204108, 0x7f869520412a, 0xfffffffffffffff8, 0x7fff2e91e6e0, 0x7fff2e91e708)
        /usr/local/go1.14.4/src/runtime/malloc.go:868 +0x87 fp=0x7fff2e91e690 sp=0x7fff2e91e658 pc=0x100b9a7
runtime.mallocgc(0x180, 0x26a4580, 0x7fff2e91e701, 0x100cb42)
        /usr/local/go1.14.4/src/runtime/malloc.go:1036 +0x793 fp=0x7fff2e91e730 sp=0x7fff2e91e690 pc=0x100c2e3
runtime.newobject(0x26a4580, 0x4000)
        /usr/local/go1.14.4/src/runtime/malloc.go:1165 +0x38 fp=0x7fff2e91e760 sp=0x7fff2e91e730 pc=0x100c6d8
runtime.malg(0x8000, 0x3afcac0)
        /usr/local/go1.14.4/src/runtime/proc.go:3360 +0x31 fp=0x7fff2e91e7a0 sp=0x7fff2e91e760 pc=0x103f3a1
runtime.mpreinit(0x3acfc40)
        /usr/local/go1.14.4/src/runtime/os_linux.go:339 +0x2d fp=0x7fff2e91e7c0 sp=0x7fff2e91e7a0 pc=0x103119d
runtime.mcommoninit(0x3acfc40)
        /usr/local/go1.14.4/src/runtime/proc.go:630 +0x108 fp=0x7fff2e91e808 sp=0x7fff2e91e7c0 pc=0x1038898
runtime.schedinit()
        /usr/local/go1.14.4/src/runtime/proc.go:547 +0x95 fp=0x7fff2e91e860 sp=0x7fff2e91e808 pc=0x10384b5
runtime.rt0_go(0x7fff2e91e968, 0x1, 0x7fff2e91e968, 0x0, 0x7f869497ec05, 0x2000000000, 0x7fff2e91e968, 0x100000000, 0x1065850, 0x0, ...)
        /usr/local/go1.14.4/src/runtime/asm_amd64.s:214 +0x125 fp=0x7fff2e91e868 sp=0x7fff2e91e860 pc=0x1065985

Thank you for confirming that the problem occurs with Go 1.14.
As an experiment, are you able to add a small amount of swap to the system, and verify if the problem continues to occur?

@davecheney After created about 100MB swap spaces, and set swappiness=0 to avoid swapping whenever possible,it can run successfully without oom.
Here is system memory usage:

free -m
              total        used        free      shared  buff/cache   available
Mem:           1868        1772          15          23          80          36
Swap:           102         102           0

ping @mknyszek

As of Go 1.11, heap arenas are mapped in 64 MiB increments on linux/amd64 (golang.org/cl/85887). Looking at the stack trace, it looks there's a failure to map the first arena (as read-write). In theory this shouldn't happen with default settings on Linux because we don't immediately go and touch all of those 64 MiB, i.e we map it but we don't use it unless we actually need it. Given that it's the actual mmap call that's failing, it looks like the OS is just telling us we can't map that memory for whatever reason.
One reason this could be is overcommit rules, or virtual address space limits. Can you show me the output of
$ cat /proc/sys/vm/overcommit_memory
$ cat /proc/sys/vm/overcommit_ratio
$ ulimit -v

on your system? Also, what version of Linux are you running on?
A work-around here could be to compile for GOARCH=386 where we use 4 MiB arenas; you should be able to map at least 2 arenas before Linux thinks you're out of memory. It will likely mean slower code overall (bit intrinsics are missing for example, and they're used on a bunch of critical paths in the runtime, also 32-bit registers instead of 64-bit) but your code should work.

@mknyszek Thanks for your detailed explanation.
The outpout of system configs are as follows:

$ cat /proc/sys/vm/overcommit_memory
0
$ cat /proc/sys/vm/overcommit_ratio
50
$ ulimit -v
unlimited

The version of Linux is 3.10.107
It looks like overcommit rules forbidded mapping the 64 MiB heap arenas.
As an experiment,when setting overcommit_memory to 1, It worked without OOM.
The work-around to compile for GOARCH=386 is also worked,as your explanation It will likely mean slower code overall,may be there is another better choice.
The 64 MiB heap arenas is hardcode or it changeable via a compiler flag?

@gopherbot remove WaitingForInfo

run a helloworld net/http on mips32 it will ask for 800MB VSS, if set overcommit_memory to 2(i.e. no overcommit), the system will immediately become unusable (can't fork: out of memory) when I run 20+ of them in parallel. Comparatively I can much more C/c++ http processes in parallel, each of them only takes 4MB or so for VSS instead of 800MB.
Can we at least make golang configurable for this huge VSS by default? so golang can run well on resource restricted linux systems? tinygo is for MCU and I'm talking about embedded boards with 64MB/128M/256M RAM here. embedded systems sometimes set overcommit_memory to 2 and swappiness to 0 to make sure the app can run without OOM for a very long time in the field.

run a helloworld net/http on mips32 it will ask for 800MB VSS, if set overcommit_memory to 2(i.e. no overcommit), the system will immediately become unusable (can't fork: out of memory) when I run 20+ of them in parallel. Comparatively I can much more C/c++ http processes in parallel, each of them only takes 4MB or so for VSS instead of 800MB.
Can we at least make golang configurable for this huge VSS by default? so golang can run well on resource restricted linux systems? tinygo is for MCU and I'm talking about embedded boards with 64MB/128M/256M RAM here. embedded systems sometimes set overcommit_memory to 2 and swappiness to 0 to make sure the app can run without OOM for a very long time in the field.

@ianlancetaylor isn't there a proposal for what @laoshaw describes?

@ianlancetaylor isn't there a proposal for what @laoshaw describes?

Maybe, but I couldn't find it.
See also #38010.

Maybe, but I couldn't find it.
See also #38010.

@laoshaw to suggest a configuration feature, file a new issue named "proposal: [summary]" that describes the problem and how you'd like to solve it.

@laoshaw to suggest a configuration feature, file a new issue named "proposal: [summary]" that describes the problem and how you'd like to solve it.

I actually don't know how to write a good proposal yet, still new to golang, just did some quick comparison to see if it's feasible to shift future devel to golang, the embedded boards I mentioned actually is probably as widely deployed as golang's microservice installation base  in the cloud. All I want is to have reasonable VSS and set it super high only when you're on a resource rich server environment and absolutely necessary for that. The default value should not that huge at least. Or let me pass a runtime env flag to adjust my own VSS size that can overwrite whatever Golang feels the best? I tried 'ulimit -v' and it does not work either.

I actually don't know how to write a good proposal yet, still new to golang, just did some quick comparison to see if it's feasible to shift future devel to golang, the embedded boards I mentioned actually is probably as widely deployed as golang's microservice installation base  in the cloud. All I want is to have reasonable VSS and set it super high only when you're on a resource rich server environment and absolutely necessary for that. The default value should not that huge at least. Or let me pass a runtime env flag to adjust my own VSS size that can overwrite whatever Golang feels the best? I tried 'ulimit -v' and it does not work either.

Pls at least file a separate bug report describing your problem.

Pls at least file a separate bug report describing your problem.

yes it's filed here: https://github.com/golang/go/issues/43699

yes it's filed here: https://github.com/golang/go/issues/43699


[Webpack 5]: cache leak with incremental compilation

Bug report
What is the current behavior?
We recently upgraded from webpack4 to webpack5 and are experiencing memory leaks when using webpack5 caching.
We have the following caching policy defined:
  "cache": {
    "cacheDirectory": "/home/admin/src/coda/.webpack-cache",
    "type": "filesystem",
    "store": "pack",
    "name": "baseApp",
    "idleTimeoutForInitialStore": 0
  }

When we launch webpack devserver the VM usage climbs to about 2.5GB and then stabilizes.   The bundle outputs in about 70s.     When we mutate a source file, this correctly triggers a recompile, and memory doubles before hitting our max cap of 6GB.    The process shortly OOM's after that.
We've tried various options for the filesystem cache including maxAge, maxGeneration, etc. all with no difference. I also tried removing contentHashs and setting output: {clean: true} to no avail.   The only thing that does help is disabling caching completely (cache=false) but that causes a horrible regression in incremental compilation times.
Here's a snapshot of our running webpack5 config:
{
  "context": "/home/admin/src/coda",
  "devServer": {
    "stats": "errors-only"
  },
  "watchOptions": {
    "ignored": [
      "/home/admin/src/coda/home"
    ]
  },
  "devtool": "cheap-module-source-map",
  "output": {
    "crossOriginLoading": "anonymous",
    "devtoolModuleFilenameTemplate": "[resource-path]",
    "devtoolFallbackModuleFilenameTemplate": "[resource-path]?[contenthash]",
    "path": "/home/admin/src/coda/artifacts/browser-app",
    "filename": "[name].[chunkhash].entry.js",
    "chunkFilename": "[name].[contenthash].chunk.js",
    "publicPath": "/dev/cdn/assets/"
  },
  "resolve": {
    "symlinks": false,
    "fallback": {
      "path": "path-browserify",
      "child_process": false,
      "tls": false,
      "fs": false,
      "net": false,
      "http": "stream-http",
      "https": "https-browserify",
      "stream": "stream-browserify",
      "crypto": "crypto-browserify",
      "os": "os-browserify/browser",
      "zlib": "browserify-zlib"
    },
    "alias": {
      "webpack-intermediate-assets": "/home/admin/src/coda/build/webpack-intermediate-assets",
      "@kr-modules": "/home/admin/src/coda/modules"
    },
    "extensions": [
      ".ts",
      ".tsx",
      ".js",
      ".json"
    ],
    "modules": [
      "/home/admin/src/coda/node_modules"
    ],
    "unsafeCache": true
  },
  "stats": {
    "modules": false,
    "children": false
  },
  "cache": {
    "cacheDirectory": "/home/admin/src/coda/.webpack-cache",
    "type": "filesystem",
    "store": "pack",
    "name": "baseApp",
    "idleTimeoutForInitialStore": 0
  },
  "entry": {
    "browser": "@kr-modules/browser/app/entrypoint",
    "external-form": "@kr-modules/browser/external-form/entrypoint"
  },
  "mode": "development",
  "module": {
    "rules": [
      {
        "exclude": [
          {}
        ],
        "test": {},
        "use": [
          {
            "loader": "cache-loader",
            "options": {
              "cacheDirectory": "/home/admin/src/coda/.cache-loader/baseApp"
            }
          },
          {
            "loader": "thread-loader",
            "options": {
              "poolTimeout": null,
              "workers": 3,
              "name": "ts-pool"
            }
          },
          {
            "loader": "ts-loader",
            "options": {
              "configFile": "tsconfig.dev.json",
              "transpileOnly": true,
              "happyPackMode": true,
              "onlyCompileBundledFiles": true,
              "experimentalFileCaching": true,
              "compilerOptions": {
                "module": "esnext"
              }
            }
          }
        ]
      },
      {
        "test": {},
        "exclude": [
          {}
        ],
        "use": [
          "/home/admin/src/coda/node_modules/mini-css-extract-plugin/dist/loader.js",
          {
            "loader": "css-loader",
            "options": {
              "modules": {
                "localIdentName": "[name]--[local]--[hash:base64:8]",
                "mode": "global"
              },
              "sourceMap": true,
              "url": false,
              "esModule": true
            }
          },
          {
            "loader": "postcss-loader",
            "options": {
              "sourceMap": true
            }
          },
          {
            "loader": "less-loader",
            "options": {
              "sourceMap": true,
              "lessOptions": {
                "modifyVars": {
                  "asset-url-prefix": "'/dev/cdn/assets/795f0f96cc72'",
                  "root-asset-url-prefix": "'/dev/cdn'"
                }
              }
            }
          }
        ]
      },
      {
        "test": {},
        "use": {
          "loader": "file-loader",
          "options": {
            "name": "img/[name].[contenthash].[ext]"
          }
        }
      },
      {
        "test": {},
        "loader": "null-loader"
      }
    ]
  },
  "plugins": [
    {
      "resourceRegExp": {},
      "newContentRegExp": {}
    },
    {
      "definitions": {
        "process.env.NODE_ENV": "\"development\"",
        "process.env.LOAD_DOCUMENTATION": "false"
      }
    },
    {},
    {
      "paths": [
        {},
        {}
      ]
    },
    {
      "options": {
        "basePath": "",
        "fileName": "../server/baseApp.manifest.json",
        "filter": null,
        "map": null,
        "publicPath": null,
        "removeKeyHash": {},
        "sort": null,
        "transformExtensions": {},
        "useEntryKeys": false,
        "writeToFileEmit": false
      }
    },
    {
      "options": {
        "filename": "[name].[contenthash].css",
        "ignoreOrder": false,
        "chunkFilename": "[name].[contenthash].chunk.css"
      }
    },
    {
      "options": {
        "filter": {},
        "allow": "(Apache-2.0 OR BSD-2-Clause OR BSD-3-Clause OR ISC OR MIT OR OFL-1.1 OR Unlicense OR WTFPL OR W3C OR Zlib)",
        "ignore": [
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>        ],
        "override": {
          "<EMAIL_ADDRESS>{
            "licenseName": "Unlicense"
          }
          // shortened...
        },
        "emitError": true,
        "outputFilename": "OpenSourceSoftwareLicenses-baseApp.json"
      }
    },
    {
      "options": {},
      "timeEventData": {},
      "smpPluginAdded": false
    },
    {
      "definitions": {}
    },
    {
      "definitions": {
        "Buffer": [
          "buffer",
          "Buffer"
        ],
        "process": "process/browser"
      }
    },
    {
      "options": {
        "resourceRegExp": {}
      }
    }
  ],
  "optimization": {
    "minimize": false,
    "concatenateModules": false,
    "minimizer": [
      // TerserPlugin
      {
        "options": {
          "test": {},
          "extractComments": true,
          "cache": true,
          "parallel": 8,
          "terserOptions": {
            "ecma": 2018,
            "compress": {
              "ecma": 2018,
              "keep_classnames": true,
              "unsafe": true,
              "unsafe_arrows": false,
              "unsafe_math": false
            }
          }
        }
      },
      // OptimizeCSSAssetsPlugin
      {
        "pluginDescriptor": {
          "name": "OptimizeCssAssetsWebpackPlugin"
        },
        "options": {
          "assetProcessors": [
            {
              "phase": "compilation.optimize-chunk-assets",
              "regExp": {}
            }
          ],
          "assetNameRegExp": {},
          "cssProcessorOptions": {
            "map": {
              "annotationPrefix": "",
              "inline": false
            }
          },
          "cssProcessorPluginOptions": {}
        },
        "phaseAssetProcessors": {
          "compilation.optimize-chunk-assets": [
            {
              "phase": "compilation.optimize-chunk-assets",
              "regExp": {}
            }
          ],
          "compilation.optimize-assets": [],
          "emit": []
        },
        "deleteAssetsMap": {}
      }
    ]
  }
}

Here's sample output during the compile:
2021-04-13-17:59:20 0|app-webpack     | [baseApp] Change in modules/global-style/colors.less; rebuilding…
2021-04-13-18:00:33 0|app-webpack     |  SMP  ⏱
2021-04-13-18:00:33 0|app-webpack     | General output time took 1 min, 12.86 secs
2021-04-13-18:00:33 0|app-webpack     |  SMP  ⏱  Loaders
2021-04-13-18:00:33 0|app-webpack     | mini-css-extract-plugin, and
2021-04-13-18:00:33 0|app-webpack     | css-loader, and
2021-04-13-18:00:33 0|app-webpack     | postcss-loader, and
2021-04-13-18:00:33 0|app-webpack     | less-loader took 1 min, 5.029 secs
2021-04-13-18:00:33 0|app-webpack     |   module count = 750
2021-04-13-18:00:33 0|app-webpack     | css-loader, and
2021-04-13-18:00:33 0|app-webpack     | postcss-loader, and
2021-04-13-18:00:33 0|app-webpack     | less-loader took 1 min, 4.92 secs
2021-04-13-18:00:33 0|app-webpack     |   module count = 750
2021-04-13-18:00:33 0|app-webpack     | assets by status 10.8 MiB [cached] 6 assets
2021-04-13-18:00:33 0|app-webpack     | assets by status 43.7 MiB [emitted]
2021-04-13-18:00:33 0|app-webpack     |   assets by info 43.5 MiB [immutable]
2021-04-13-18:00:33 0|app-webpack     |     assets by path *.js 41.8 MiB
2021-04-13-18:00:33 0|app-webpack     |       asset browser.6762f8d648ff98edcf15.entry.js 23.1 MiB [emitted] [immutable] (name: browser) 1 related asset
2021-04-13-18:00:33 0|app-webpack     |       asset external-form.61557166b57a93b7fb7f.entry.js 18.7 MiB [emitted] [immutable] (name: external-form) 1 related asset
2021-04-13-18:00:33 0|app-webpack     |     assets by path *.css 1.65 MiB
2021-04-13-18:00:33 0|app-webpack     |       asset browser.e5ff57d15328ceb4a662.css 1000 KiB [emitted] [immutable] (name: browser) 1 related asset
2021-04-13-18:00:33 0|app-webpack     |       asset external-form.31948c4ab1b96ab265b5.css 689 KiB [emitted] [immutable] (name: external-form) 1 related asset
2021-04-13-18:00:33 0|app-webpack     |   asset OpenSourceSoftwareLicenses-baseApp.json 215 KiB [emitted]
2021-04-13-18:00:33 0|app-webpack     |   asset ../server/baseApp.manifest.json 2.79 KiB [emitted]
2021-04-13-18:00:33 0|app-webpack     | Entrypoint browser 24.1 MiB (20.5 MiB) = browser.e5ff57d15328ceb4a662.css 1000 KiB browser.6762f8d648ff98edcf15.entry.js 23.1 MiB 2 auxiliary assets
2021-04-13-18:00:33 0|app-webpack     | Entrypoint external-form 19.4 MiB (17.1 MiB) = external-form.31948c4ab1b96ab265b5.css 689 KiB external-form.61557166b57a93b7fb7f.entry.js 18.7 MiB 2 auxiliary assets
2021-04-13-18:00:33 0|app-webpack     | webpack 5.32.0 compiled successfully in 72866 ms

Versioning information:
webpack: 5.32.0
webpack-cli: 4.5.0
webpack-dev-server: ^.11.2
I noticed similar issues reported in https://github.com/webpack/webpack/issues/12947.    The proposal there was to use a memory cache and set output.clean.   I've tried that with {type: "memory", maxGenerations: 1} and output.clean, as well as stripping contentHashes. I still see the issue.
Anything else I can do to help narrow down the underlying issue?   Any workaround that would
maintain caching without the OOM?
Thanks,
Nigel.
If the current behavior is a bug, please provide the steps to reproduce.
I'm just running:
webpack-cli -w --color --env dev--config=...

and then mutate a source file.
What is the expected behavior?
The memory working-set should be stable and not grow without bounds.
Other relevant information:
webpack version: 5.32.0
Node.js version: 14.16.1
Operating System: MacOS (or Linux Buster)
Additional tools:

Sounds like a memory leak somewhere. Difficult to find without access to the repo or a memory profile.
But I think the huge memory usage originates from mini-css-extract-plugin and we have an PR open to add an experimental largely more memory efficient mode.
I'm looking for testers and your case seems like a good fit.

@nigel-codaio Maybe you can provide example of repo, so we can investigate memory leak

@sokra -- looks like https://github.com/webpack-contrib/mini-css-extract-plugin/pull/737 was released with v1.5.0.   I'll try pulling this in to see if it helps.
@alexander-akait -- I haven't had luck trimming this down but should be able to set aside time to investigate this week.

I tried the latest mini-css-extract-plugin (v1.5.0) with today's webpack5 release (5.34.0) and still hit the issue.    @sokra it definitely appears related to the mini-css-extract plugin as editing non-less files doesn't exhibit the memory leak.
Sharing the full project is challenging as this is a commercial codebase.   Are there any guides or instructions on how I might perform memory profile?  Any best practices to follow?  I'm game to dig in but don't know how to start.  Thanks!

@nigel-codaio Maybe you can provide full configuration when memory leak?

Small notes - OptimizeCssAssetsWebpackPlugin is deprecated and potential leaking, please use https://github.com/webpack-contrib/css-minimizer-webpack-plugin/. What is terser-webpack-plugin version? You don't need cache-loader there big bugs and leaks, please use cache.type: 'filesystem' (now we have built-in cache
)

maybe fixed by https://github.com/webpack/webpack/pull/13184

Hi @sokra and @alexander-akait -- thanks for the suggestions.
I made the following changes today:

Migrated to css-minimizer-webpack-plugin
Removed cache-loader -- we are already using the webpack5 filecache.   I'd previously noticed some benefits of the cache-loader with typescript but these were relatively small.
Bumped our main app bundle memory from 4GB to 6GB

With these changes in place, things appear to be working better.     @sokra I also pulled in https://github.com/webpack/webpack/pull/13184 and tried bundling with that.  This also gives some improvement and appears to have a lower memory cap than without it.
What's the recommendation of filecache vs. memory cache?   Our production builds don't use any form of caching currently so I'm mostly focused on webpack devserver behavior with an eye on incremental compilation cost.  Is there a discussion anywhere on the tradeoffs between the caching approaches, and a best-practice example on the optimal config for dev?
Thanks.

You can use the filesystem cache for production and development. The filesystem cache includes a memory cache to avoid reading/writing to disk too often.
You usually don't need to change any of the advanced settings for the cache. The defaults should work fine.

@sokra The filesystem cache includes a memory cache or you mean cache: { type: 'memory' } ?

cache: { type: "filesystem" }

@sokra so you mean even though we write cache type as filesystem there will be things stored in memory as well?

Yes, memory cache is faster. So for incremental builds that's faster. But cache items only stay for a few generations in the memory cache. See maxMemoryGenerations.
You could disable that with maxMemoryGenerations: 0, where cache items are only stored in memory until they are serialized to disk. After that they need to be read from disk on access.

@sokra The memory consumption is stays low if I give cache type as 'memory' but when I give it as filesystem the memory usage is very high and the process crashes in just a few minutes.
I'm sticking with cache type memory for now.

@vijaybritto which webpack version?

5.35.1

@nigel-codaio I was working with enhancing our cache hit ratio for webpack on our CI servers, it turns out turning off unsafeCache solved a leak that happens in the cache folder when the cache is set to filesystem.
With unsafeCache enabled, webpack would regularly not use parts of the cache, and it would keep generating large number of files continuously  which would fill up the hard drive.
i am using webpack version 5.35.1, and turning off unsafeCache greatly improved the consistency and file storage consumption on my webpack builds

@vijaybritto Can you try latest version?

@vijaybritto can you share your original webpack config?

@nigel-codaio The problem still exists?

@alexander-akait @sokra we observe the similar issue with our application. Memory consumption with cache: { type: "memory" } is around 2GB but with filesystem cache it grows dramatically (7-8GB). This basically makes filesystem cache unusable in our setup, my personal laptop starts memory swapping aggressively which degrades performance.
I was able to make a memory snapshot with Chrome, here is a screenshot. Lmk what other info you need, I can probably share the entire snapshot if you give me your email.

@kanoshin What is webpack version?

@alexander-akait 5.38.1

@kanoshin we need entire snapshot, please do it not only once, 1-2-5-10 runners, you can put it here, because I don't think it contains something sensitivity

I also tried setting maxMemoryGenerations: 0 and experimental option from mini-css-extract-plugin but it didn't help. We have a codebase with about 9k typescript & less files.

Feel free to ping me when you prepare more snapshots, maybe leaking in non official plugins, you can test it locally

@alexander-akait attaching the snapshots. It's almost impossible to take the snapshot in our larger app, so I decided to use a smaller one but it still shows a very disproportional memory usage comparing to the codebase size. This app is around 223MB of source code with node_modules but initial memory usage is 3x of that amount and then it keeps growing between rebuilds.
What I'm trying to say is that it's not just a memory leak I'm reporting but most importantly is that simply the initial memory consumption for larger apps makes filesystem cache unusable for larger codebases.
Google drive with zip archive: https://drive.google.com/file/d/1_XR27NpTQnYcOnFZFowqJqcB1FEbe1g0/view?usp=sharing.

@kanoshin thanks for these profiles. That were very useful. I needed to implement some custom tooling to analyse them, since the devtools show WeakMap references very badly.
Anyway I found two problems:

The less-loader keeps hold on the last webpack compilation the processed less files.

There is a weird reference of some Compilations in v8 feedback vectors:

I hope 273bd8f (#13580) will fix that.
I also found some places where memory usage could be improved a bit by getting rid of same unused data, but these where not leaks...

@kanoshin I released the 2. change as https://github.com/webpack/webpack/releases/tag/v5.39.1 Maybe you could test with that version again and capture new profiles (In this case a single one after 20 rebuilds would be enough). It's difficult to see from the profile if this is was the only problem...

If you want to "workaround" for problem 1, you can remove this line: https://github.com/webpack-contrib/less-loader/blob/465ffc4052642d799bb29a85056517db31ee1bf5/src/utils.js#L198

I think we can do major release for less-loader and drop it

@kanoshin Please try https://github.com/webpack-contrib/less-loader/releases/tag/v10.0.0

Please update https://github.com/webpack-contrib/mini-css-extract-plugin/releases/tag/v1.6.1, many memory leaks were fixed

@sokra @alexander-akait I made the same measurements using updated less-loader and mini-css-extract-plugin. Looks like memory consumption improved in absolute numbers but I still see steady growth from snapshot to snapshot. This time I used heapdump package for taking snapshots.
There are still references to Compilation between the snapshots:

Here are the snapshots: https://drive.google.com/file/d/1QPk9Qf5dBDt_YDUhZcEn4p9bTOVwbQeW/view?usp=sharing

hmm... interesting... Could you try to run gc() before doing the heapdump? (--expose-gc)

@sokra sure, just did that again. Here are two snapshots one after 1 rebuild and second one after ~20 rebuilds across 4-5 different files (TS, LESS, CSS) https://drive.google.com/file/d/1p0v_sdl65CqaaS1PY2WU2iSWzfk-Cn7X/view?usp=sharing.

Ok it took me a full day to implement tooling for analysing heapsnapshots, but I think I created something that works. Using the devtools it's next to impossible to follow the retainer chain when WeakMaps are involved. The tricky thing with WeakMaps it that you would need to follow the key retainer and the WeakMap retainer while skipping over cycles where the key is held by something the value references.
Anyway here is a dump of retainers of the 12 Compilation objects in your heapsnapshot:
(It's still not easy to read and you need to know a lot about v8 to understand it. A normal leak is a simple chain. When WeakMaps are involved you will see multiple chains.)
synthetic (GC roots) @3
 [13] = synthetic (Global handles) @29
  internal 4492 = processImmediate() in node:internal/timers @6541
   internal context = object system / Context @3758169
    internal previous = object system / Context @3203705
     context timerListMap = object {  } @3203647
      [60000] = object TimersList @5832419
       ._idleNext = object Timeout @7284333
        ._onTimeout = () @11585955
         internal context = object system / Context @1657275
          context compiler = object Compiler @600341
           ._lastCompilation = object Compilation @5653795

This is the last active compilation, which is held by the Compiler. That's fine.

synthetic (GC roots) @3
 [13] = synthetic (Global handles) @29
  internal 4355 / DevTools console = object { dir, dirxml, profile, profileEnd, clear, table, keys, values, debug, undebug, ... } @6363
   .require = require() in node:internal/modules/cjs/helpers @4210341
    .cache = object { /Users/kir...le-call.js, /Users/kir...s/index.js, /Users/kir...hortOut.js, /Users/kir...terator.js, /Users/kir...i/index.js, /Users/kir...oopHook.js, /Users/kir...ailHook.js, /Users/kir...b/pkcs1.js, /Users/kir...ject-dp.js, /Users/kir...p/index.js, ... } @169023
     ./Users/kirilla/repos/st/app/Clients/Web/node_modules/less/lib/less/plugin-manager.js = object Module @591753
      internal 2 / part of key (Module @591753) -> value (Object @1942751) pair in WeakMap (table @3687911) = object { __esModule, default } @1942751
synthetic (GC roots) @3
 [13] = synthetic (Global handles) @29
  internal 4516 = () in node:internal/process/esm_loader @6589
   internal context = object system / Context @3004645
    context ESMLoader = object Loader @3004659
     .cjsCache = object WeakMap @4793971
      internal table = array  @3687911
       internal 1235 / part of key (Module @591753) -> value (Object @1942751) pair in WeakMap (table @3687911) = object { __esModule, default } @1942751
        .default = PluginManagerFactory() in /Users/kirilla/repos/st/app/Clients/Web/node_modules/less/lib/less/plugin-manager.js @1940823
         internal context = object system / Context @1754891
          context pm = object PluginManager @8405329
           .webpackLoaderContext = object { getOptions, emitWarning, emitError, getLogger, resolve, getResolve, emitFile, addBuildDependency, utils, rootContext, ... } @8725371
            .getOptions = getOptions() in /Users/kirilla/repos/st/app/Clients/Web/node_modules/webpack/lib/NormalModule.js @9361177
             internal context = object system / Context @9361179
              context compilation = object Compilation @7747467

This is leak number 1. Again (or still) related to the less-loader. While we removed our own reference from the less object, less itself seem to leak the last plugin manager used, which references our loader context and so the Compilation.
maybe we can remove webpackLoaderContext from pluginManager again after finishing compilation? cc @alexander-akait
But it also makes sense to fix the pm leak in less: https://github.com/less/less.js/blob/c5181800980f0a24dfaa5fe71fba813a6a80542a/packages/less/src/less/plugin-manager.js#L158 cc @matthew-dean

synthetic (GC roots) @3
 [13] = synthetic (Global handles) @29
  internal 4355 / DevTools console = object { dir, dirxml, profile, profileEnd, clear, table, keys, values, debug, undebug, ... } @6363
   .require = require() in node:internal/modules/cjs/helpers @4210341
    .cache = object { /Users/kir...le-call.js, /Users/kir...s/index.js, /Users/kir...hortOut.js, /Users/kir...terator.js, /Users/kir...i/index.js, /Users/kir...oopHook.js, /Users/kir...ailHook.js, /Users/kir...b/pkcs1.js, /Users/kir...ject-dp.js, /Users/kir...p/index.js, ... } @169023
     ./Users/kirilla/repos/st/app/Clients/Web/node_modules/webpack/lib/ChunkGraph.js = object Module @2512045
      internal 2 / part of key (Module @2512045) -> value (ChunkGraph @2640331) pair in WeakMap (table @3687911) = ChunkGraph() in /Users/kirilla/repos/st/app/Clients/Web/node_modules/webpack/lib/ChunkGraph.js @2640331
synthetic (GC roots) @3
 [13] = synthetic (Global handles) @29
  internal 4516 = () in node:internal/process/esm_loader @6589
   internal context = object system / Context @3004645
    context ESMLoader = object Loader @3004659
     .cjsCache = object WeakMap @4793971
      internal table = array  @3687911
       internal 7718 / part of key (Module @2512045) -> value (ChunkGraph @2640331) pair in WeakMap (table @3687911) = ChunkGraph() in /Users/kirilla/repos/st/app/Clients/Web/node_modules/webpack/lib/ChunkGraph.js @2640331
        internal context = object system / Context @3437037
         internal previous = object system / Context @3417781
          context chunkGraphForChunkMap = object WeakMap @4947027
           internal table = array  @990375
            internal 9 / part of key (Chunk @990383) -> value (ChunkGraph @990385) pair in WeakMap (table @990375) = object ChunkGraph @990385
synthetic (GC roots) @3
 [13] = synthetic (Global handles) @29
  internal 4492 = processImmediate() in node:internal/timers @6541
   internal context = object system / Context @3758169
    internal previous = object system / Context @3203705
     context timerListMap = object {  } @3203647
      [60000] = object TimersList @5832419
       ._idleNext = object Timeout @7284333
        ._onTimeout = () @11585955
         internal context = object system / Context @1657275
          context compiler = object Compiler @600341
           internal 39 / part of key (Compiler @600341) -> value (PersistentChildCompilerSingletonPlugin @601845) pair in WeakMap (table @601787) = object PersistentChildCompilerSingletonPlugin @601845
synthetic (GC roots) @3
 [13] = synthetic (Global handles) @29
  internal 4355 / DevTools console = object { dir, dirxml, profile, profileEnd, clear, table, keys, values, debug, undebug, ... } @6363
   .require = require() in node:internal/modules/cjs/helpers @4210341
    .cache = object { /Users/kir...le-call.js, /Users/kir...s/index.js, /Users/kir...hortOut.js, /Users/kir...terator.js, /Users/kir...i/index.js, /Users/kir...oopHook.js, /Users/kir...ailHook.js, /Users/kir...b/pkcs1.js, /Users/kir...ject-dp.js, /Users/kir...p/index.js, ... } @169023
     ./Users/kirilla/repos/st/app/Clients/Web/node_modules/html-webpack-plugin/lib/cached-child-compiler.js = object Module @2512555
      .exports = object { CachedChil...ompilation } @3688233
       .CachedChildCompilation = CachedChildCompilation() in /Users/kirilla/repos/st/app/Clients/Web/node_modules/html-webpack-plugin/lib/cached-child-compiler.js @601335
        internal context = object system / Context @601299
         context compilerMap = object WeakMap @601305
          internal table = array  @601787
           internal 3 / part of key (Compiler @600341) -> value (PersistentChildCompilerSingletonPlugin @601845) pair in WeakMap (table @601787) = object PersistentChildCompilerSingletonPlugin @601845
            .compilationState = object { isCompiling, isVerifyingCache, entries, compiledEntries, compilationResult, mainCompilationHash } @7146865
             .compilationResult = object { compiledEntries, dependencies, mainCompilationHash } @3496709
              .compiledEntries = object { /Users/kir.../index.ejs } @3554355
               ./Users/kirilla/repos/st/app/Clients/Web/node_modules/html-webpack-plugin/lib/loader.js!/Users/kirilla/repos/st/app/Clients/Web/packages/auth/app/index.ejs = object { content, hash, entry } @2857067
                .entry = object Chunk @990383
                 internal 20 / part of key (Chunk @990383) -> value (ChunkGraph @990385) pair in WeakMap (table @990375) = object ChunkGraph @990385
                  ._cacheChunkGraphModuleKey2 = object PublicPathRuntimeModule @2341369
                   .compilation = object Compilation @1652005

This is leak number 2. Two layers of WeakMaps are involved here. There is probably no way to find that in the normal tooling.
We can clean it up a bit by assuming that the module scoped variable chunkGraphForChunkMap in webpack/lib/ChunkGraph.js, and the Compiler @600341 are globals:
chunkGraphForChunkMap = object WeakMap @4947027
 internal table = array  @990375
  internal 9 / part of key (Chunk @990383) -> value (ChunkGraph @990385) pair in WeakMap (table @990375) = object ChunkGraph @990385
object Compiler @600341
 internal 39 / part of key (Compiler @600341) -> value (PersistentChildCompilerSingletonPlugin @601845) pair in WeakMap (table @601787) = object PersistentChildCompilerSingletonPlugin @601845
/Users/kirilla/repos/st/app/Clients/Web/node_modules/html-webpack-plugin/lib/cached-child-compiler.js = object Module @2512555
 .exports = object { CachedChil...ompilation } @3688233
  .CachedChildCompilation = CachedChildCompilation() in /Users/kirilla/repos/st/app/Clients/Web/node_modules/html-webpack-plugin/lib/cached-child-compiler.js @601335
   internal context = object system / Context @601299
    context compilerMap = object WeakMap @601305
     internal table = array  @601787
      internal 3 / part of key (Compiler @600341) -> value (PersistentChildCompilerSingletonPlugin @601845) pair in WeakMap (table @601787) = object PersistentChildCompilerSingletonPlugin @601845
       .compilationState = object { isCompiling, isVerifyingCache, entries, compiledEntries, compilationResult, mainCompilationHash } @7146865
        .compilationResult = object { compiledEntries, dependencies, mainCompilationHash } @3496709
         .compiledEntries = object { /Users/kir.../index.ejs } @3554355
          ./Users/kirilla/repos/st/app/Clients/Web/node_modules/html-webpack-plugin/lib/loader.js!/Users/kirilla/repos/st/app/Clients/Web/packages/auth/app/index.ejs = object { content, hash, entry } @2857067
           .entry = object Chunk @990383
            internal 20 / part of key (Chunk @990383) -> value (ChunkGraph @990385) pair in WeakMap (table @990375) = object ChunkGraph @990385
             ._cacheChunkGraphModuleKey2 = object PublicPathRuntimeModule @2341369
              .compilation = object Compilation @1652005

So the html-webpack-plugin keeps a reference the (child) Compilation used to compute the html files.
Not sure if this is intendend. It seem to leak it because it references the Chunk @990383 via compiledEntries.*.entry cc @jantimon

object Compilation @1652005
 .compiler = object Compiler @3351893
  .parentCompilation = object Compilation @609611

Compilation @1652005 is the compilation from the html-webpack-plugin. As it's a child compilation is has a reference to the parent Compilation.
This makes the reference to Compilation @1652005 expensive.

Now it gets dirty. The following 8 Compilations are referenced by the Compilation @609611 from above (which seem to be the first Compilation):
synthetic (GC roots) @3
 [13] = synthetic (Global handles) @29
  internal 4355 / DevTools console = object { dir, dirxml, profile, profileEnd, clear, table, keys, values, debug, undebug, ... } @6363
   .require = require() in node:internal/modules/cjs/helpers @4210341
    .cache = object { /Users/kir...le-call.js, /Users/kir...s/index.js, /Users/kir...hortOut.js, /Users/kir...terator.js, /Users/kir...i/index.js, /Users/kir...oopHook.js, /Users/kir...ailHook.js, /Users/kir...b/pkcs1.js, /Users/kir...ject-dp.js, /Users/kir...p/index.js, ... } @169023
     ./Users/kirilla/repos/st/app/Clients/Web/node_modules/webpack/lib/NormalModuleFactory.js = object Module @2512527
      .exports = NormalModuleFactory() in /Users/kirilla/repos/st/app/Clients/Web/node_modules/webpack/lib/NormalModuleFactory.js @3350351
       internal context = object system / Context @1661983
        context unsafeCacheDependencies = object WeakMap @3523185
         internal table = array  @3524939
          internal 21778 / part of key (HarmonyImportSideEffectDependency @1741453) -> value (Object @4567883) pair in WeakMap (table @3524939) = object { module, fileDependencies, missingDependencies, contextDependencies } @4567883
object Compilation @609611
 .chunkGraph = object ChunkGraph @990597
  .moduleGraph = object ModuleGraph @946751
   ._dependencyMap = object Map @1656927
    internal table = array  @1656953
     internal 49021 = object HarmonyImportSideEffectDependency @1741453
      internal 13 / part of key (HarmonyImportSideEffectDependency @1741453) -> value (Object @4567883) pair in WeakMap (table @3524939) = object { module, fileDependencies, missingDependencies, contextDependencies } @4567883
       .module = object NormalModule /Users/kirilla/repos/st/app/Clients/Web/node_modules/query-string/index.js @1361323
        .parser = object JavascriptParser @6141493
         .hooks = object { evaluateTypeof, evaluate, evaluateIdentifier, evaluateDe...Identifier, evaluateCa...sionMember, isPure, preStatement, blockPreStatement, statement, statementIf, ... } @5786111
          .callMemberChain = object HookMap @5786183
           ._map = object Map @10164927
            internal table = array  @7039523
             internal 9 = object Hook @7039527
              .taps = object Array @10244077
               [0] = object { type, fn, name } @6142947
                .fn = () in /Users/kirilla/repos/st/app/Clients/Web/node_modules/webpack/lib/dependencies/CommonJsExportsParserPlugin.js @6142949
                 internal context = object system / Context @8392167
                  context this = object CommonJsExportsParserPlugin @10247099
                   .moduleGraph = object ModuleGraph @5282155
                    ._cacheModuleGraphModuleKey2 = object JsonpChunkLoadingRuntimeModule @5383121
                     .compilation = object Compilation @6140385

This sounds like a problem in webpack. It should not have this line: .parser = object JavascriptParser @6141493.
Usually the parser reference should be removed from the Modules once that Compilation is done, but that doesn't seem to be the case for all modules.
Investigating...

Let's call it a "cache". It doesn't leak a infinite amount of memory, only the latest compilation used for html generation.
Maybe we can reduce the memory usage a bit by not referencing Chunk objects from the compilation (Chunks will hold on the whole Compilation and parent Compilation, because you can reach other chunks from there and modules, and for backward-compat reasons, etc.)

I also can't see any leaks here. But we can definitely try to improve the cache size :)
Maybe I should explain why this cache was introduced at all:
Previously the html-webpack-plugin child compiler would compile on every watch run even if no relevant file was changed.
With the new filesnapshot api it was finally possible to run a child compiler only if it's files change

@jantimon Why we need to keep whole compilation, maybe we can cache only related stuff?

@kanoshin Can you test again, please update webpack https://github.com/webpack/webpack/releases/tag/v5.42.0 and less-loader https://github.com/webpack-contrib/less-loader/releases/tag/v10.0.1

I've tried it. I see no memory growth when modifying both TS and LESS modules. We had +100Mb per modification of all files in package once. Right now it is gone :+1: However, when I modify LESS files there's an occasinal error in the debugger. It crashes the process. This is it:
VM3690 AssetGenerator.js:268 Uncaught TypeError: Cannot read property 'dataUrl' of undefined
    at AssetGenerator.getTypes (/home/strider/Projects/st/app/Clients/Web/node_modules/webpack/lib/asset/AssetGenerator.js:268:24)
    at NormalModule.getSourceTypes (/home/strider/Projects/st/app/Clients/Web/node_modules/webpack/lib/NormalModule.js:1091:39)
    at NormalModule.cleanupForCache (/home/strider/Projects/st/app/Clients/Web/node_modules/webpack/lib/NormalModule.js:362:45)
    at NormalModuleFactory.cleanupForCache (/home/strider/Projects/st/app/Clients/Web/node_modules/webpack/lib/NormalModuleFactory.js:670:11)
    at Compiler._cleanupLastNormalModuleFactory (/home/strider/Projects/st/app/Clients/Web/node_modules/webpack/lib/Compiler.js:383:34)
    at Compiler.createNormalModuleFactory (/home/strider/Projects/st/app/Clients/Web/node_modules/webpack/lib/Compiler.js:1049:8)
    at Compiler.newCompilationParams (/home/strider/Projects/st/app/Clients/Web/node_modules/webpack/lib/Compiler.js:1071:30)
    at Compiler.compile (/home/strider/Projects/st/app/Clients/Web/node_modules/webpack/lib/Compiler.js:1082:23)
    at /home/strider/Projects/st/app/Clients/Web/node_modules/webpack/lib/Watching.js:188:19
    at Hook.eval [as callAsync] (eval at create (/home/strider/Projects/st/app/Clients/Web/node_modules/webpack/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:24:1)

I remember no such error in the previous tests (before today's fixes). May it be caused by the fixes?

Yes, regression, please wait, we will fix it in near future

@jantimon Why we need to keep whole compilation, maybe we can cache only related stuff?
yes that's what I meant by we should try to decrease the cache size 😄

Right now the following is cached:
https://github.com/jantimon/html-webpack-plugin/blob/8f8f7c53c4e4f822020d6da9de0304f8c23de08f/lib/cached-child-compiler.js#L31-L37
{
  dependencies: FileDependencies,
  compiledEntries: {[entryName: string]: ChildCompilationResultEntry}
}

ChildCompilationResultEntry is coming from: https://github.com/jantimon/html-webpack-plugin/blob/8f8f7c53c4e4f822020d6da9de0304f8c23de08f/lib/child-compiler.js#L189-L192
{
            content: templateSource,
            hash: childCompilation.hash || 'XXXX',
            entry: entries[entryIndex]
}

I assume that entries might keep the entire compilation in memory or I missed something

@jackfranklin yes
chunkGraphForChunkMap = object WeakMap @4947027
 internal table = array  @990375
  internal 9 / part of key (Chunk @990383) -> value (ChunkGraph @990385) pair in WeakMap (table @990375) = object ChunkGraph @990385
object Compiler @600341
 internal 39 / part of key (Compiler @600341) -> value (PersistentChildCompilerSingletonPlugin @601845) pair in WeakMap (table @601787) = object PersistentChildCompilerSingletonPlugin @601845
/Users/kirilla/repos/st/app/Clients/Web/node_modules/html-webpack-plugin/lib/cached-child-compiler.js = object Module @2512555
 .exports = object { CachedChil...ompilation } @3688233
  .CachedChildCompilation = CachedChildCompilation() in /Users/kirilla/repos/st/app/Clients/Web/node_modules/html-webpack-plugin/lib/cached-child-compiler.js @601335
   internal context = object system / Context @601299
    context compilerMap = object WeakMap @601305
     internal table = array  @601787
      internal 3 / part of key (Compiler @600341) -> value (PersistentChildCompilerSingletonPlugin @601845) pair in WeakMap (table @601787) = object PersistentChildCompilerSingletonPlugin @601845
       .compilationState = object { isCompiling, isVerifyingCache, entries, compiledEntries, compilationResult, mainCompilationHash } @7146865
        .compilationResult = object { compiledEntries, dependencies, mainCompilationHash } @3496709
         .compiledEntries = object { /Users/kir.../index.ejs } @3554355
          ./Users/kirilla/repos/st/app/Clients/Web/node_modules/html-webpack-plugin/lib/loader.js!/Users/kirilla/repos/st/app/Clients/Web/packages/auth/app/index.ejs = object { content, hash, entry } @2857067
           .entry = object Chunk @990383
            internal 20 / part of key (Chunk @990383) -> value (ChunkGraph @990385) pair in WeakMap (table @990375) = object ChunkGraph @990385
             ._cacheChunkGraphModuleKey2 = object PublicPathRuntimeModule @2341369
              .compilation = object Compilation @1652005

above trace where they stored

I see some ModuleGraphConnection objects in the delta. I've also detected them in the previous tests.

@StreetStrider It can be any non official plugin, please provide profiling

@alexander-akait
https://drive.google.com/file/d/14C4JDlYMssApdpXl5OLPvWYCQCcbKXQC/view

thanks I will look soon

I open-sourced a tool to easily analyse these heapsnapshots: https://github.com/sokra/heapdump-analyser
NODE_OPTIONS=--max_old_space_size=16000 npx heapdump-analyser /path/to/file.heapsnapshot Compilation

This shows the retainer graphs for all Compilation objects within the heapsnapshot.

@StreetStrider I can't see a leak in your profiles...

@sokra thanks, I was not exactly sure. Increasing in delta does not mean leak directly, but the numbers seemed suspicious to me.
How to prepare the third profile? Usually I take 1st profile, then make some actions, then take 2nd. Do I need to just add another round to that process?

Another round. Then view the 3rd profile with Objects allocated between 1 and 2 selected.

Hi folks, we've discovered this same memory leak and I can reproduce it with a vanilla Webpack/HtmlWebpackPlugin setup, repo here: https://github.com/helloitsjoe/webpack-memory-leak
I wrote details in the README, and some of this may be restating points above, but here's a summary:
In HtmlWebpackPlugin, htmlWebpackPluginHooksMap (WeakMap) adds a compilation every time the template HTML file index.template.html is modified. These WeakMap entries are never garbage collected, but I haven't been able to determine why. My one lead is that moduleGraphForModuleMap (WeakMap in Webpack) keys are holding onto references to the Compilation objects.
Here are some relevant screenshots, also in the readme, let me know if I can help more or provide more info:

Heap snapshot

We can see that each key in that WeakMap has a reference to the compilation, which has a reference to the HtmlWebpackPlugin compiler:

WeakMap

I've opened https://github.com/webpack/webpack/issues/17851 to track this and belive I have a fix, I will have a PR ready shortly.


Re-exporting a type should consider about 'isolatedModules' flag

Current Behaviour
Current version of ctix will re-export a default exported type using this schema:
//in T.ts
type T = string;
export default T;

//in index.ts generated by ctix
export { default as T} from './T';

which will encounter following compile error when  '--isolatedModules' flag is set:
TS1205 [ERROR]: Re-exporting a type when the '--isolatedModules' flag is provided requires using 'export type'.

Motivation
Since '--isolatedModules' flag is required by powerful typescript compilers (esbuild for example), it will be great to consider about 'isolatedModules'.
Reference
Typescript Handbook

@KotoriK Hi,
I agree importance of isolatedModules option!
But I need a some time because how can determine export type or export in ctix.
I will try to found everyone understandable and agree soludtion.
Thank you!

@KotoriK Hi,
ctix support isolatedModules option version after 1.2.2
Thank you :)


operators [R] jaeger (1.11.0 1.11.1 1.12.0 1.12.1 1.13.0 1.13.1 1.14.0 1.15.0 1.16.0 1.17.0 1.17.1 1.18.0 1.18.1 1.19.0 1.20.0 1.21.3 1.22.1 1.24.0 1.25.0 1.26.0 1.28.0 1.29.1 1.8.2)

Add v1.29.1 and convert previous versions to new bundle format.
Signed-off-by: Robert Kukura<EMAIL_ADDRESS>Thanks submitting your Operator. Please check below list before you create your Pull Request.
New Submissions

[ ] Are you familiar with our contribution guidelines?
[ ] Have you packaged and deployed your Operator for Operator Framework?

[ ] Have you tested your Operator in all supported installation modes?
[ ] Have you considered whether you want use semantic versioning order?
[ ] Is your submission signed?
[ ] Is operator icon set?

Updates to existing Operators

[x] Did you create a ci.yaml file according to the update instructions?
[x] Is your new CSV pointing to the previous version with the replaces property if you chose replaces-mode via the updateGraph property in ci.yaml?
[x] Is your new CSV referenced in the appropriate channel defined in the package.yaml or annotations.yaml ?
[x] Have you tested an update to your Operator when deployed via OLM?
[x] Is your submission signed?

Your submission should not

[x] Modify more than one operator
[x] Modify an Operator you don't own
[x] Rename an operator - please remove and add with a different name instead
[x] Modify any files outside the above mentioned folders
[x] Contain more than one commit. Please squash your commits.

Operator Description must contain (in order)

[x] Description about the managed Application and where to find more information
[x] Features and capabilities of your Operator and how to use it
[x] Any manual steps about potential pre-requisites for using your Operator

Operator Metadata should contain

[x] Human readable name and 1-liner description about your Operator
[x] Valid category name1
[x] One of the pre-defined capability levels2
[x] Links to the maintainer, source code and documentation
[x] Example templates for all Custom Resource Definitions intended to be used
[x] A quadratic logo

Remember that you can preview your CSV here.
--
1 If you feel your Operator does not fit any of the pre-defined categories, file an issue against this repo and explain your need
2 For more information see here

@rkukura   Value of metadata.annotations.containerImage not used in any container, please check.
The operator is using some image which must have exactly the same value as metadata.annotations.containerImage.
Please check both values and make them the same.

@rkukura Value of metadata.annotations.containerImage not used in any container, please check. The operator is using some image which must have exactly the same value as metadata.annotations.containerImage. Please check both values and make them the same.

This isn't making sense to me. Unless I'm mistaken, this metadata.annotations.containerImage is from the bundle's CSV and identifies the image for OLM to deploy in a container as the operator. This image does seem to exist. What would cause some different image, or no image at all, to be deployed instead? Or does "used in any container" mean something different than being deployed in a container by OLM?
BTW, 1.8.2 is a very old version, and both the CSV and CRD have changed significantly in subsequent versions. I'm tempted to simply remove 1.8.2 and see if the newer versions pass CI.

Unable to pull quay.io/jaegertracing/jaeger-operator, please check permissions or path.

@rkukura, it has failed on api-server resource not found installing ServiceMonitor jaeger-operator-metrics-monitor: GroupVersionKind monitoring.coreos.com/v1, Kind=ServiceMonitor not found on the cluster

/retest

/unhold

/merge possible

/merge possible

/merge possible


Null Injector Error - [TreeDragDirective -> TreeDraggedElement]

I'm submitting a...

[ ] Regression (a behavior that used to work and stopped working in a new release)
[x ] Bug report
[ ] Feature request
[ ] Documentation issue or request

Upgrading from version 8.3 to 10.0.0
In 8.3.0 simply adding forRoot() to the TreeModule import solved this issue.
In 10.0.0 there is no static forRoot() method
Expected behavior:
The Drag Directive and Drag Element should be imported with the module and no errors should occur
Minimal reproduction of the problem with instructions:
Include drag and drop functionality in tree
Upgrade from 8.3.0 and change paths where used to '@circlon/angular-tree-component'
Load tree component
Version of affected browser(s),operating system(s), npm, node and angular-tree-component:
    Win10 (Version 10.0.17763 Build 17763)
    npm 6.13.4
Node.js v12.14.1
Chrome: Version 85.0.4183.102

Hi @Eddie541, are you importing the TreeDragDirective or TreeDraggedElement into your project? Can you share some example what you do with the tree?
You just need to import the TreeModule, there is no need to include .forRoot() anymore.

I'm just now circling back to this.
Thanks for the response.
I can't show full code but here are some details and information including a screenshot (may not be able to post) of the error:
// In a 'common module' our tree component
// line 44 is here -> <tree-root #treeRoot
[(state)]="state"
[nodes]="dataLoaded$ | async"
[options]="options"
				 // in component.ts file

@ViewChild('treeRoot', { static: true })
public treeRoot: TreeComponent;
// common module import
import {TreeModule} from '@circlon/angular-tree-component';
@NgModule({
imports: [
...
TreeModule,
...
]
Also The app builds and runs, even in prod but running tests that have the tree included fail with many errors like this:
ERROR in ../node_modules/@circlon/angular-tree-component/lib/components/tree-node-collection.component.d.ts:5:9 - error TS1086: An accessor cannot be declared in an ambient context.
5     get nodes(): any;
~~~~~
It's my understanding is that this is either a typescript or angular version mismatch. I tried updating Typescript (to 4.0.5 - no luck) but I can't yet upgrade to Angular 9 if that is what is required. We are on the 8.2.0 version of Angular.
I've removed node_modules cache verified and re-installed everything but still no luck

This is an issue for me as well, a blocking issue for upgrading my app from angular 8 to angular 9.
I have tried the above steps of moving from 8.3.0 to 9.0.1, 9.0.2, and 9.0.3
I had to add "skipLibCheck": true,  to my tsconfig.json just to get the project to compile and even on trees where I am not using or leveraging the dragging functionality the TreeDraggedElement is causing the entire component to fail.

Version 9.0.3 is made for Angular 9. If you are still on Angular 8 and switched the Tree to Version 9 that probably will not work correctly due to the mayor changes between Angular 8 and 9.
If you have those issues still after beeing on Angular 9 and Version 9 of the tree, then I would need a bit more information. I have build a simple demo project in Angular 9 and that works fine including the unit tests. But it's just a simple project, to recreate your use cases I need some help.

Moving to Angular 9 resolved the issue with TreeDraggedElement for me.

Moving to Angular 9 resolved the issue with TreeDraggedElement for me.


add ssl to dev keycloak

i created an A record to point dev-sso.etenlab.com to the ip address of the keycloak server.
use lets encrypt to create and install an ssl cert. when you need domain validation, send me the codes and i'll add them.
https://letsencrypt.org/

http://dev-sso.etenlab.com/


Release 0.1.2?

There should be a new version of devtools now that reek is reenabled and upgraded: https://github.com/mbj/devtools/commit/ddef8c1914a3671ff46f9bbe48cf1960a3d449c7

Closed in https://github.com/mbj/devtools/commit/9d32f46467ea887649b773c7bb26b61394bd66c2


remove unused dependencies

removed unused dependency from the admin-ui

Codecov Report

Merging #387 into master will not change coverage.
The diff coverage is 0%.

@@           Coverage Diff           @@
##           master     #387   +/-   ##
=======================================
  Coverage   37.98%   37.98%
=======================================
  Files          20       20
  Lines        1677     1677
=======================================
  Hits          637      637
  Misses       1040     1040

Impacted Files
Coverage Δ

app/scripts/app.js
8.25% <ø> (ø)
:white_check_mark:

app/scripts/controllers/feed.js
4.3% <0%> (ø)
:white_check_mark:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 97fe47f...223ea3f. Read the comment docs.


Send email when someone like your project

[ ] Send segment data by impact-graph
[ ] Create autopilot journey for  project liked event

I'm not sure if we should send emails when a project receives a like - this can make lots of spam. Gonna icebox this one.

Yeah we disbled the email notifier for like on notification center


GitPod fails

When I tried to create GitPod environment, I got this error message.
error[E0599]: no method named `get_notify_fd` found for struct `ScmpFilterContext` in the current scope
   --> crates/libcontainer/src/seccomp/mod.rs:270:17
    |
270 |             ctx.get_notify_fd()
    |                 ^^^^^^^^^^^^^ method not found in `ScmpFilterContext`

For more information about this error, try `rustc --explain E0599`.
error: could not compile `libcontainer` (lib) due to previous error
error: Recipe `youki-release` failed on line 19 with exit code 101

Sorry for confusing you, but we haven't supported gitpod. We are supporting CodeSpace instead of it.
https://github.com/containers/youki/tree/main/.devcontainer

We should remove gitpod's links. Please still open until it is completed.


Ctrl+Shift+L does not work when in editor

I have tested to use the Ctrl+Shift+L inside the editor to get the keybindings but it seems that its overwritten or does not work.
If I however select Database Navigator then the keybinding does work.
OS: windows 10
1709 (OS Build 16299.371)

on the other hand I can see the keyboard shortcut change in the menu upon select.
So I wonder if this is intentional and in that case, is there documentation over how this all works with the keyboard shortcuts?

Currently Ctrl+Shift+L is bound to "Convert to lowercase" command.
But maybe we don't need it, maybe "Convert to uppercase" (Ctrl+Shift+U) should work as a case toggle.

In fact there are other default Eclipse shortcuts for upper/lower case commands. These explicit keybidnings were made for very old DBeaver version.
I'll remove them as they doesn't make much sense now.

thank you! and very nice job on dbeaver. :)


chore: relicense to Apache-2

@npmccallum I think I did what was asked, but I'm not sure I understand what is meant by removing copyright assessment. Is that the authors line in Cargo.toml? Should I change it to "Enarx Project"?

@npmccallum gentle nudge

I don't have a maintainer voice here, but from the CCC threads and the comments in the PR, it looks like this is ready to merge.
Nit: I don't think this is material but you may have inadvertently dropped a linefeed in most of the mod.rs files.
cc: @npmccallum


DS-479 - bump OAS 2.x to 3.x

Changes

upgraded OAS from 2.x to 3.x (for both schema based and schema free data services)
reordered the list of API's
fixed issues with schema

The yaml generating logic is being moved to ds-service-manager hence these changes are not necessary


Documentation editor: error on saving documentation

Expected behavior
In the jaffle-shop project I updated some documentation in the new documentation editor and clicked the "Save documentation" button to save the newly edited fields to the yml file located in the same folder:
Actual behavior
An error occured with the following error-Message:
Could not save documentation to \c:\Users\XNLXD03x\projects\jaffle_shop-main\models\staging\schema.yml: Error: ENOENT: no such file or directory, open '\c:\Users\XNLXD03x\projects\jaffle_shop-main\models\staging\schema.yml'
Steps To Reproduce
Run the jaffle-shop project on a Windows Server, edit the documentation in the documentation editor and click Save.
Log output/Screenshots

Output from Developer Tools:

Operating System
Windows Server 2019
dbt version
1.6.1
dbt Adapter
dbt-oracle
dbt Power User version
0.17.3
Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

Any thoughts on what is going on here and how I might get around this issue?
Appreciate any help with this in advance!

Fixed by https://github.com/innoverio/vscode-dbt-power-user/commit/bc345513b87e7de57b115f8317d0b390eae7c51f

@sandhein New version with the fix has been release. Please make sure you update the extension to 0.17.4

@sandhein circling back on this, please let us know if this is still an issue.

I am closing this issue for now. Please re-open if the problem persists.

@anandgupta42: Thank you for the fix, works perfectly now:-).
Sorry for the late answer, I have been in holiday a couple of days.


Test splitting

From 12 minutes back 5 minutes

@setu4993 really need your help caching the installs.
Idea:

We have a seperate action that caches the installs everday, then we use artifacts or a docker image to load the installs.
Then we seperate the linting from the splitted tests.
This should get us back down to 6 minutes hopefully

We have a seperate action that caches the installs everday, then we use artifacts or a docker image to load the installs.

I think just using the plain GH Actions' pip cache setup is a good start. Let's see how much better it makes things.

Then we seperate the linting from the splitted tests.

Agreed! We started doing that for our internal repos and would highly advise moving lint / format steps out of test steps. Those have a separate set of dependencies (dev dependencies) that are unrelated to the primary project.

Codecov Report

Merging #579 (8e7716c) into main (6ef9748) will decrease coverage by 0.04%.
The diff coverage is n/a.

:exclamation: Current head 8e7716c differs from pull request most recent head 64d4c65. Consider uploading reports for the commit 64d4c65 to get more accurate results

@@            Coverage Diff             @@
##             main     #579      +/-   ##
==========================================
- Coverage   85.25%   85.22%   -0.04%
==========================================
  Files         157      157
  Lines       12202    12200       -2
==========================================
- Hits        10403    10397       -6
- Misses       1799     1803       +4

see 4 files with indirect coverage changes
:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Launch option handleSIGHUP doesn't work for child processes (chromium instances)

Steps to reproduce
Tell us about your environment:

Puppeteer version: 1.11.0
Platform / OS version: Linux Mint 19 (Ubuntu 18.04 LTS)
Node.js version: 11.9.0

What steps will reproduce the problem?

Make a script that launches puppeteer with the handleSIGHUP: false option
1.1 The script will spawn some chrome processes

Send the signal to the script with kill -s SIGHUP 31249
2.2 The browser process (PID 31260) won't close, as said in the docs
Send the signal directly to the browser with kill -s SIGHUP 31260
3.3 The browser closes, while the script remains open

What is the expected result?
I expected all the chromium processes to ignore SIGHUP with the handleSIGHUP option.
Maybe I misunderstood the docs, or how much control Puppeteer has over the browser.
What happens instead?
Since the browser closes when the signal is sent directly to the process, features like nodemon's graceful reloading don't work, since the browser closes instantly when the signal is sent to all the processes in the tree.
Is it possible to configure the chrome instances spawned from the script to also ignore SIGHUP?

Is it possible to configure the chrome instances spawned from the script to also ignore SIGHUP?

@matheusb-comp No, I don't think this is possible. We only have control on the SIGHUP handling for node.js process.

Since the browser closes when the signal is sent directly to the process, features like nodemon's graceful reloading don't work, since the browser closes instantly when the signal is sent to all the processes in the tree.

Any chance nodemon can be configured to send sighup to the root process only? It's weird it tries to bring down the whole process subtree - children might not and should not know about nodemon existance.

Any chance nodemon can be configured to send sighup to the root process only?

Not that I know of, and I believe it can't be done, by what I looked at the code.
Anyway, thanks for


Open badge links in new tab/page

Summary

Setting target=_blank on badges to open links in a new page.
Fixing header margin issue
Adding badge to style guide
Minor consistency edit for keeping style definitions in our .less files and not in having it directly in an HTML tag

Reason

Badge link targeting issue was reported by Rick.
Have been running into a weird margin issue on headers, turned out it was due to a hacky workaround to have clickable anchor links. The workaround is no longer necessary, since the issue has long been fixed, and all we had left was this weird margin.

Testing
https://deploy-preview-3949--kongdocs.netlify.app/hub/kong-inc/mocking/ - click on badges, links should open in new tab/window.
https://deploy-preview-3949--kongdocs.netlify.app/contributing/markdown-rules/#badges style guide entry

This is unfortunately an existing known issue that isn't caused by anything in this PR. Check a live page:


Local lobby required for sv_cheats 1

If trying to use the script from another server than from local server, an error after using sv_cheats 1 occurs: "Change can be only made by the owner of the server" or something along the lines of that.
Could this be fixed, or at least added to the guide that you need to set up a local lobby?

I don't believe that can be fixed; most of the methods under the hood that do things like give the bots gold are flagged by the engine as cheats and won't work unless sv_cheats is enabled.
That said, the lobby host can tick the ENABLE CHEATS box in the lobby settings, and that should apply regardless of the server location (I suspect.  We always host locally, so I could be totally wrong here).
Apologies for the late reply!

If I understand, sv_cheats toggles the cheats which can be used by console, while "Enabled cheats" are the cheats in chat. I am not sure, how can I make a workaround. Guess I have to have a good computer and internet connection.
Anyway, if you still play this mode, could you figure that out? Thanks a lot.

Since I'm doing github maintenance, I'll just close this. I feel there's nothing to be done about this.


Fix GHA CI after they upgraded nodejs

They upgraded to a version that requires a glibc newer than is in the pre-2023 ASWF containers.

Merging immediately because this does fix the CI.


Mutable Estimators

This is an attempt to provide estimators that mutate instead of being immutable (like most dask estimators). After writing this, I no longer think this is a good idea, it makes things very tricky to reason about. This PR is just for discussion.
Currently the following works:
import numpy as np
from dask import compute
from sklearn import linear_model, decomposition, datasets, pipeline
from sklearn.base import BaseEstimator
from dklearn.pipeline import Pipeline
from dklearn.grid_search import GridSearchCV

logistic = linear_model.LogisticRegression()
pca = decomposition.PCA()

# Create a dask backed pipeline. All operations on this happen lazily.
pipe = Pipeline(steps=[('pca', pca), ('logistic', logistic)])

# dask estimators are also sklearn estimators
assert isinstance(pipe, BaseEstimator)
# This means that `clone`, etc... work just fine

digits = datasets.load_digits()
X_digits = digits.data
y_digits = digits.target

# All of these run lazily, but match the sklearn api
pipe.fit(X_digits, y_digits)
# just for the demo, don't really test on the training set :)
predicted = pipe.predict(X_digits)
score = pipe.score(X_digits, y_digits)

# Computing many things at once
pipe2, predicted, score = compute(pipe, predicted, score)

# Computed estimators are still dask objects
assert isinstance(pipe2, Pipeline)
# But they're not the same as the pre-compute object (no mutation from
# computation)
assert pipe is not pipe2

# The `to_sklearn` method converts back to sklearn objects
pipe3 = pipe.to_sklearn()
assert isinstance(pipe3, pipeline.Pipeline)
assert not isinstance(pipe3, Pipeline)  # not a dask pipeline

# GridSearchCV and RandomizedSearchCV are also implemented. These don't
# currently happen lazily, due to the need for `refit`.
n_components = [20, 40, 64]
Cs = np.logspace(-4, 4, 3)
grid = dict(pca__n_components=n_components,
            logistic__C=Cs)

search = GridSearchCV(pipe, grid)
search.fit(X_digits, y_digits)
assert isinstance(search.best_estimator_, Pipeline)

Good parts have been pulled into master. Closing.


Implement natural copy/paste with tables

Fix #80
This implements a smarter copy/paste behavior with tables. Here are how each case is handled:

Copying the content of a single cell into another cell
→ The content of the first cell is pasted inside the second cell
Copying the content of a single cell outside the table
→  Just the content of the cell is pasted (not the table)
Copying some content into a cell
→ The content is inserted inside the cell
Copying multiple cells somewhere else inside the table
→ The copied fragment of table is patched at the given position, overwritting cells and adding rows and columns if necessary.
Copying multiple cells outside the table
→ A new table is pasted, containing the copied cells.

This behavior is very much like the one from Google Docs and Dropbox paper.

I have deployed the new behavior here: https://gitbookio.github.io/slate-edit-table/


fix: Fix support for @auth on fields with no @model

Description of changes
Fix "missing datasource" issue reported in #2133, due to regression in support for @auth directives on root types.
Do not merge this until we resolve the correct branch -- this might involve reverting main to the post-release state so we can apply the hotfix, unless we want to craft a new release automation to apply to a different branch.
Issue #, if available
#2133
Description of how you validated changes

Added new tests to cover V2 transformer support for this case
Rerunning existing E2E tests to ensure no breakage

Checklist

[x] PR description included
[x] yarn test passes
[x] Tests are changed or added

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

LGTM. If the hot fix is to be merged to release branch and released there, I suggest a backup for the main branch before you make force push on main. After the hot fix is merged back to main, the previous not released commits can be cherry-picked from the backup branch.

No force-push necessary: I experimented with what I believe would be the commit flows, and everything merges cleanly.
That said, I'll definitely be making a backup of main, just to be on the safe side!


Local-only temporary write mode

Tell us more about this new feature.
Some software will write temporary files in the same location as the data source it is reading. Typically these are index files or caches to speed up subsequent access. Since these files are temporary and may be specific to the reader's operation they do not need to be persisted back to the data store. So this "local write" mode would allow writing to the mount point, but the files would only exist locally.
This could be confusing and anti-pattern enough to be a horrible idea, with the ideal solution to fix the readers to work with the idea of read-only file systems and always write temporary files to a local location. However for legacy or closed code this could be useful.

Super interesting idea. It would definitely have to be opt-in, with some loud warnings that "files will never be persisted", but I can see how it would be useful. We'll look into this one.

Might be possible with overlayfs or aufs? Haven't tried them myself, though.

We have looked into this one and we have decided to not prioritize this feature this year, according to Mountpoint’s core mission and tenets.

Mapping these requirements to Mountpoint’s core mission and tenets led us to not prioritize this feature this year


⚠ Bofrost.es has degraded performance

In aa45f58, Bofrost.es (https://www.bofrost.es) experienced degraded performance:

HTTP code: 200
Response time: 1659 ms

Resolved: Bofrost.es performance has improved in 55a4990 after 7 minutes.


Region coordinates not centered on Google Maps

Is this a bug report?
Yes
Have you read the Installation Instructions?
Yes
Environment
"react-native": "0.53.0",
"react": "16.2.0",
"react-native-maps": "0.20.1",

Steps to Reproduce

Render a Marker at the coordinates received from onRegionChangeComplete
Add provider='google'
Zoom out on map as much as possible.
Notice how the marker is not in the center of the screen.

Expected Behavior

I expect the marker to be in the center on both apple maps and google maps. In other words:
I expect the map to return the center coordinate in onRegionChangeComplete and onRegionChange

Actual Behavior
On apple maps (without provider="google" on iOS devices) the returned coordinate from onRegionChangeComplete/onRegionChange is always centered in the screen (as expected). Using Google maps it is not centered while zooming out. The error/offset gets unnoticeable while zoomed in or at the equator.
Apple maps vs Google maps:
Purple marker indicates actual center of map. Red marker indicates coordinates received.

Reproducible Demo

import React, { Component } from 'react';
import { Dimensions, View, StyleSheet } from 'react-native';
import MapView, { Marker } from 'react-native-maps';

export default class App extends Component {

  state = {
    region: {
      latitude: -100,
      longitude: 0,
      latitudeDelta: 10,
      longitudeDelta: 10,
    },
    marker: {
      latitude: 0,
      longitude: -100,
    }
  };

  onRegionChange = (region) => {
    this.setState({
      region,
      marker: {
        latitude: region.latitude,
        longitude: region.longitude,
      }
    });
  }

  render() {
    return (
      <View style={{ flex: 1 }}>
        <MapView
          onRegionChangeComplete={this.onRegionChange}
          style={{ flex: 1 }}
          initialRegion={this.state.region}
          provider='google'
        >
          <Marker coordinate={this.state.marker} />
        </MapView>
        <View style={styles.dot} />
      </View>
    );
  }
}

const { height, width } = Dimensions.get('window');
const dotSize = 10;

const styles = StyleSheet.create({
  dot: {
    width: dotSize,
    height: dotSize,
    position: 'absolute',
    top: (height - dotSize) / 2,
    backgroundColor: 'blue',
    left: (width - dotSize) / 2
  }
});

I am also  interested to find a workaround to get the actual center coordinate if anyone has an idea. Thanks!
I have tried a workaround using coordinateForPoint to check the center coordinate at each regionChange but I can't successfully get it to work with google maps on iOS.

+1 i noticed this behavior sporadically

Oof, same issue here, having a hard time debugging this one.
Odd how the mapPadding prop seems to compound the problem, essentially shifting the region passed to onRegionChange 🤔
Let me know of any ideas/workarounds!

Notices such behavior too

Experiencing this behaviour, too:
when zooming in through <MapView>'s animateToRegion() method (triggered by a button), I create a new zoomToRegion object by leaving the current longitude and latitude values untouched while dividing the deltas by two. Same effect: the region moves into the direction of the equator.
I visualized my region and center point on http://geojson.io, and even there the center point seems out of center. My gut feeling tells me this is due to mercator projection distorting latitude. Have a look at
https://de.wikipedia.org/wiki/Mercator-Projektion#/media/File:Mercator-proj.png
Imagine a rectangle close to the pole and take its calculated latitude center point. It's visually off-center. Citing Wikipedia: "the Mercator projection distorts the size of objects as the latitude increases from the Equator to the poles"
Maybe a coordinateForPoint() (https://github.com/react-community/react-native-maps/blob/master/docs/mapview.md#methods) could do the trick to grab a "visual" center point? Will try.

coordinateForPoint() is currently broken https://github.com/react-community/react-native-maps/issues/2578
Working workaround demo here: https://snack.expo.io/@hennzen/zoom-to-visual-center
See README.md for details

i am still facing this problem. Has anyone found a new solution? This is very important for me

If anyone else ends up here, had problems with this as well. Try adding paddingAdjustmentBehavior="never" to MapView (says it's default in docs but doesn't seem that way?)

If anyone else ends up here, had problems with this as well. Try adding paddingAdjustmentBehavior="never" to MapView (says it's default in docs but doesn't seem that way?)

It is working well on Android, but on iOS you have to add paddingAdjustmentBehavior="never" to make it work.


Cleanly track 15-25 interesting datasets on noms.io

For the Noms OSS launch, we tracked a handful of datasets on demo.noms.io, but there are some problems:

The datasets we picked are not necessarily very useful to anyone, we were choosing more to push what noms could do
They are not generally very clean data

Most of the types are strings. We cannot type them better for a few reasons:

Noms doesn't support optional types and most of these datasets have columns that are nullable
The data is frequently formatted in a way that the csv parser doesn't understand, e.g., using Y/N for boolean columns
The data uses datatypes we don't have good representations for (e.g., date, time) in Noms currently

(minor, but annoying) Column names that have spaces or other special characters come out looking weird because of "Q-escaping"

We only pull in data from csv, where many interesting datasets are in other formats

This bug is to pick 15-25 interesting, valuable datasets, and track them with high quality on noms.io.

(WIP)
Dataset Candidates:

Popular on data.gov

National Stock Number Extract

Popular on NYC open data

Popular on Chicago open data

Popular on SF open data

Frequently requested on Stack Exchange Open Data

Matches Socrata's description of what users find valuable

Software-community related

Hacker News corpus
Reddit corpus (huge!)

Setting sights lower: proggit corpus

Stack Overflow database (they publish it regularly, don't know if historical snapshots are available)

Random other things that seem high value

Weather (@zane?)


Add max_identifier_length and remove its own table_alias_length

Refer https://github.com/rails/rails/pull/28339
Steps to reproduce:
$ git clone https://github.com/rails/rails.git
$ cd rails/activerecord
$ git checkout 5-1-stable
$ bundle install
$ bundle exec rake test:jdbcpostgresql

Actual result without this pull request
$ bundle exec rake test:jdbcpostgresql
... snip ...
Using jdbcpostgresql
NameError: undefined local variable or method `max_identifier_length' for #<ActiveRecord::ConnectionAdapters::PostgreSQLAdapter:0x1f66d8e1>
           new_column_from_field at /home/yahonda/git/rails/activerecord/lib/active_record/connection_adapters/postgresql/schema_statements.rb:163
                block in columns at /home/yahonda/git/rails/activerecord/lib/active_record/connection_adapters/abstract_adapter.rb:167
                             map at org/jruby/RubyArray.java:2486
                         columns at /home/yahonda/git/rails/activerecord/lib/active_record/connection_adapters/abstract_adapter.rb:166
                         columns at /home/yahonda/git/rails/activerecord/lib/active_record/connection_adapters/schema_cache.rb:67
                    columns_hash at /home/yahonda/git/rails/activerecord/lib/active_record/connection_adapters/schema_cache.rb:73
                    load_schema! at /home/yahonda/git/rails/activerecord/lib/active_record/model_schema.rb:471
                    load_schema! at /home/yahonda/git/rails/activerecord/lib/active_record/attributes.rb:233
                    load_schema! at /home/yahonda/git/rails/activerecord/lib/active_record/attribute_decorators.rb:50
            block in load_schema at /home/yahonda/git/rails/activerecord/lib/active_record/model_schema.rb:464
                 mon_synchronize at /home/yahonda/.rbenv/versions/jruby-<IP_ADDRESS>/lib/ruby/stdlib/monitor.rb:214
                     load_schema at /home/yahonda/git/rails/activerecord/lib/active_record/model_schema.rb:461
                 attribute_types at /home/yahonda/git/rails/activerecord/lib/active_record/model_schema.rb:353
              type_for_attribute at /home/yahonda/git/rails/activerecord/lib/active_record/model_schema.rb:376
                            type at /home/yahonda/git/rails/activerecord/lib/active_record/table_metadata.rb:32
                build_bind_param at /home/yahonda/git/rails/activerecord/lib/active_record/relation/predicate_builder.rb:168
  block in create_binds_for_hash at /home/yahonda/git/rails/activerecord/lib/active_record/relation/predicate_builder.rb:114
                            each at org/jruby/RubyHash.java:1343
           create_binds_for_hash at /home/yahonda/git/rails/activerecord/lib/active_record/relation/predicate_builder.rb:89
                    create_binds at /home/yahonda/git/rails/activerecord/lib/active_record/relation/predicate_builder.rb:34
                           build at /home/yahonda/git/rails/activerecord/lib/active_record/relation/where_clause_factory.rb:21
                          where! at /home/yahonda/git/rails/activerecord/lib/active_record/relation/query_methods.rb:612
                           where at /home/yahonda/git/rails/activerecord/lib/active_record/relation/query_methods.rb:605
                         find_by at /home/yahonda/git/rails/activerecord/lib/active_record/relation/finder_methods.rb:78
           find_or_initialize_by at /home/yahonda/git/rails/activerecord/lib/active_record/relation.rb:233
           find_or_initialize_by at /home/yahonda/git/rails/activerecord/lib/active_record/querying.rb:6
                             []= at /home/yahonda/git/rails/activerecord/lib/active_record/internal_metadata.rb:18
                          define at /home/yahonda/git/rails/activerecord/lib/active_record/schema.rb:56
                          define at /home/yahonda/git/rails/activerecord/lib/active_record/schema.rb:44
                          <main> at /home/yahonda/git/rails/activerecord/test/schema/schema.rb:1
                            load at org/jruby/RubyKernel.java:973
                 block in (root) at /home/yahonda/git/rails/activesupport/lib/active_support/dependencies.rb:1
                            load at /home/yahonda/git/rails/activesupport/lib/active_support/dependencies.rb:286
                 load_dependency at /home/yahonda/git/rails/activesupport/lib/active_support/dependencies.rb:258
                            load at /home/yahonda/git/rails/activesupport/lib/active_support/dependencies.rb:286
                          <main> at /home/yahonda/git/rails/activerecord/test/cases/helper.rb:139
                         require at org/jruby/RubyKernel.java:955
                          <main> at /home/yahonda/git/rails/activerecord/test/cases/helper.rb:148
                         require at org/jruby/RubyKernel.java:955
                 block in (root) at /home/yahonda/git/rails/activerecord/test/cases/batches_test.rb:1
                          select at org/jruby/RubyArray.java:2565
                          <main> at /home/yahonda/git/rails/activerecord/test/cases/batches_test.rb:1
rake aborted!

With this pull request
5004 runs, 13992 assertions, 6 failures, 29 errors, 6 skips

looks good to me, what do you think @rdubya ?

Looks good to me as long as we are ok with dropping support for Postgres 7-. I assume its old enough we probably don't need to support it anymore, specially since AR only officially supports 9.1+ I believe. @kares @enebo ?

Thanks for the discussion. I'd prefer AR-JDBC supports the same database and same range of database versions as Rails does. I mean if Rails 5.1 supports PostgreSQL 9.1 or higher, AR-JDBC also supports PostgreSQL 9.1 or higher.
Rails itself does not support AWS Redshift database and there is another 3rd party gem called https://github.com/ConsultingMD/activerecord5-redshift-adapter If AR-JDBC had plenty resources to support databases more than Rails does it would be nice to consider them but I do not think there is.

the point is it would be just minor differences compared to postgresql adapter (due similar JDBC layer)
... but yeah - we do not care at this point. it was mostly to feed some historical perspectives into the box

@yahonda thanks for the quick fix :green_heart: ... we shall get CI into shape in a week or two

Thank you.


Contact Page added

Have you created this PR under SWOC? @vishal0316

Owner

yes

SWOC contribution starts from 1st Jan 2024 @vishal0316
So closing this PR for now! Good Luck!!

SWOC contribution starts from 1st Jan 2024 @vishal0316
So closing this PR for now! Good Luck!!

So is this countable or not?

No
Issue will get assigned from 1st jan 2024 and then only you can create your PR @vishal0316