⚡ Performance: Project service doesn't cache all fs.realpath

Acknowledgement

[X] I acknowledge that issues using this template may be closed without further explanation at the maintainer's discretion.

Comment
Acknowledgement

[x] I acknowledge that issues using this template may be closed without further explanation at the maintainer's discretion.

Comment
🔎 Search Terms
project service fs realpath realpathSync
🙁 Actual behavior
When using typescript-eslint's parserOptions.projectService, type checking APIs switch from the traditional manual TypeScript ts.Program approach to the editor-style ts.ProjectService. We're observing excess calls to the realpath function on some paths in node_modules/ - up to 3-4 for some paths (!).
🙂 Expected behavior
There should be no uncached realpath calls, I'd think?
As a draft, I added a basic caching Map to realpath and ran a before & after comparison with hyperfine (--runs 50). The results showed a ~0.5-2.5% improvement in lint time:

Variant
Measurement
User Time

Baseline
3.153 s ± 0.039 s
4.403 s

Caching
3.073 s ± 0.048 s
4.377 s

diff patch to switch to the Caching variant...
diff --git a/node_modules/typescript/lib/typescript.js b/node_modules/typescript/lib/typescript.js
index 4baad59..e53476d 100644
--- a/node_modules/typescript/lib/typescript.js
+++ b/node_modules/typescript/lib/typescript.js
@@ -13,6 +13,8 @@ See the Apache Version 2.0 License for specific language governing permissions
 and limitations under the License.
 ***************************************************************************** */

+var realpathCache = new Map();
+
 var ts = {}; ((module) => {
 "use strict";
 var __defProp = Object.defineProperty;
@@ -8798,6 +8800,15 @@ var sys = (() => {
       return path.length < 260 ? _fs.realpathSync.native(path) : _fs.realpathSync(path);
     }
     function realpath(path) {
+      const cached = realpathCache.get(path);
+      if (cached) {
+        return cached;
+      }
+      const result = realpathWorker(path);
+      realpathCache.set(path, result);
+      return result;
+    }
+    function realpathWorker(path) {
       try {
         return fsRealpath(path);
       } catch {

Additional information about the issue
On the typescript-eslint side:

https://github.com/typescript-eslint/performance is the performance repository showing how to create these profiles.

https://github.com/typescript-eslint/performance/blob/73e32fdd3138c479d9e275305aa18e62e88d57be/README.md#comparison-project-service-uncached-file-system-path-reads has the measurements for this specific investigation.

https://github.com/typescript-eslint/typescript-eslint/pull/6754 is the PR that first added usage of the project service in typescript-eslint.

These are the top 10 most common paths called by realpath...
4<EMAIL_ADDRESS>4 /Users/josh/repos/performance/node_modules/prettier
3 /Users/josh/repos/performance/cases/files-1024-layout-even-singlerun-true-types-service
3 /Users/josh/repos/performance/cases/files-1024-layout-even-singlerun-true-types-service/src
3 /Users/josh/repos/performance/cases/files-1024-layout-even-singlerun-true-types-service/src/example0
3 /Users/josh/repos/performance/cases/files-1024-layout-even-singlerun-true-types-service/src/example1
3 /Users/josh/repos/performance/cases/files-1024-layout-even-singlerun-true-types-service/src/example10
3 /Users/josh/repos/performance/cases/files-1024-layout-even-singlerun-true-types-service/src/example10/nested1
3 /Users/josh/repos/performance/cases/files-1024-layout-even-singlerun-true-types-service/src/example10/nested10
3 /Users/josh/repos/performance/cases/files-1024-layout-even-singlerun-true-types-service/src/example10/nested10/nested1

Here's an example call stack from the most common one...
Error
    at Object.realpath (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:8807:33)
    at host.compilerHost.realpath (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:127075:128)
    at realPath (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:44380:35)
    at getOriginalAndResolvedFileName (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:43301:28)
    at resolveTypeReferenceDirective (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:43389:74)
    at Object.resolve (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:123707:42)
    at resolveNamesWithLocalCache (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:129425:29)
    at Object.resolveTypeReferenceDirectiveReferences (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:129493:12)
    at ConfiguredProject2.resolveTypeReferenceDirectiveReferences (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:182460:33)
    at resolveTypeReferenceDirectiveNamesWorker (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:124519:20)
    at resolveTypeReferenceDirectiveNamesReusingOldState (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:124672:14)
    at processTypeReferenceDirectives (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125992:156)
    at findSourceFileWorker (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125881:9)
    at findSourceFile (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125737:20)
    at processImportedModules (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:126134:11)
    at findSourceFileWorker (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125886:7)
    at findSourceFile (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125737:20)
    at processImportedModules (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:126134:11)
    at findSourceFileWorker (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125886:7)
    at findSourceFile (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125737:20)
    at processImportedModules (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:126134:11)
    at findSourceFileWorker (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125886:7)
    at findSourceFile (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125737:20)
    at /Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125686:22
    at getSourceFileFromReferenceWorker (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125657:26)
    at processSourceFile (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125684:5)
    at /Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125978:7
    at forEach (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:2389:22)
    at processReferencedFiles (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125977:5)
    at findSourceFileWorker (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125880:9)
    at findSourceFile (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125737:20)
    at /Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125686:22
    at getSourceFileFromReferenceWorker (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125657:26)
    at processSourceFile (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125684:5)
    at processTypeReferenceDirectiveWorker (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:126015:7)
    at processTypeReferenceDirective (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:126007:5)
    at processTypeReferenceDirectives (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:126001:7)
    at findSourceFileWorker (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125881:9)
    at findSourceFile (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125737:20)
    at /Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125686:22
    at getSourceFileFromReferenceWorker (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125657:26)
    at processSourceFile (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125684:5)
    at processTypeReferenceDirectiveWorker (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:126015:7)
    at processTypeReferenceDirective (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:126007:5)
    at createProgram (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:124224:9)
    at synchronizeHostDataWorker (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:148948:15)
    at synchronizeHostData (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:148844:7)
    at Object.getProgram (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:149020:5)
    at ConfiguredProject2.updateGraphWorker (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:183144:41)
    at ConfiguredProject2.updateGraph (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:183000:32)
    at ConfiguredProject2.updateGraph (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:184302:24)
    at updateWithTriggerFile (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:184785:11)
    at _ProjectService.reloadConfiguredProject (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:186401:5)
    at ConfiguredProject2.updateGraph (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:184298:29)
    at updateWithTriggerFile (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:184785:11)
    at updateConfiguredProject (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:184793:9)
    at _ProjectService.findCreateOrReloadConfiguredProject (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:187262:44)
    at _ProjectService.tryFindDefaultConfiguredProjectForOpenScriptInfo (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:187286:25)
    at _ProjectService.tryFindDefaultConfiguredProjectAndLoadAncestorsForOpenScriptInfo (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:187332:25)
    at _ProjectService.assignProjectToOpenedScriptInfo (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:187217:27)
    at _ProjectService.openClientFileWithNormalizedPath (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:187411:48)
    at _ProjectService.openClientFile (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:187120:17)
    at useProgramFromProjectService (/Users/josh/repos/performance/node_modules/@typescript-eslint/typescript-estree/dist/useProgramFromProjectService.js:61:28)
    at getProgramAndAST (/Users/josh/repos/performance/node_modules/@typescript-eslint/typescript-estree/dist/parser.js:44:100)
    at parseAndGenerateServices (/Users/josh/repos/performance/node_modules/@typescript-eslint/typescript-estree/dist/parser.js:155:11)
    at Object.parseForESLint (/Users/josh/repos/performance/node_modules/@typescript-eslint/parser/dist/parser.js:101:80)
    at Object.parse (/Users/josh/repos/performance/node_modules/eslint/lib/languages/js/index.js:186:26)
    at parse (/Users/josh/repos/performance/node_modules/eslint/lib/linter/linter.js:931:29)
    at Linter._verifyWithFlatConfigArrayAndWithoutProcessors (/Users/josh/repos/performance/node_modules/eslint/lib/linter/linter.js:1696:33)
    at Linter._verifyWithFlatConfigArray (/Users/josh/repos/performance/node_modules/eslint/lib/linter/linter.js:2062:21)
    at Linter.verify (/Users/josh/repos/performance/node_modules/eslint/lib/linter/linter.js:1528:61)
    at Linter.verifyAndFix (/Users/josh/repos/performance/node_modules/eslint/lib/linter/linter.js:2299:29)
    at verifyText (/Users/josh/repos/performance/node_modules/eslint/lib/eslint/eslint.js:498:48)
    at /Users/josh/repos/performance/node_modules/eslint/lib/eslint/eslint.js:939:40
    at async Promise.all (index 1)
    at async ESLint.lintFiles (/Users/josh/repos/performance/node_modules/eslint/lib/eslint/eslint.js:880:25)
    at async Object.execute (/Users/josh/repos/performance/node_modules/eslint/lib/cli.js:521:23)
    at async main (/Users/josh/repos/performance/node_modules/eslint/bin/eslint.js:153:22)

This looks very similar to #59338. My instinct is that it's the same root cause. Filing separately for visibility into the issue & just in case they're not actually the same.

Will dig in and check but note that we already cache the realPath but right now its only within project itself. It is not shared across the projects (because cache invalidating etc right now does not work across projects and is part of ongoing work at #55968) . So if there are multiple projects resolving same location that need to be opened, then it would be working as intended. But will need to look at all the stacks and debug and see.


Fix(CollectiveOverview): Remove credit card confirmation activity from timeline

#9909  👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @gustavlrsn and the rest of your teammates on  Graphite


cmd/compile: confusing errors when forgetting an import for an anonymous field

What version of Go are you using (go version)?
go version devel +a0e7f12771 Sat Sep 29 10:07:58 2018 +0000 linux/amd64

Does this issue reproduce with the latest release?
Yes - checked 1.11.
What operating system and processor architecture are you using (go env)?
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/mvdan/go/cache"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/mvdan/go/land:/home/mvdan/go"
GOPROXY=""
GORACE=""
GOROOT="/home/mvdan/tip"
GOTMPDIR=""
GOTOOLDIR="/home/mvdan/tip/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build472648887=/tmp/go-build -gno-record-gcc-switches"

What did you do?
Forget to add an import for an anonymous struct field.
What did you expect to see?
What I see when the field isn't anonymous:
$ cat f.go
package p

type t struct {
        F sync.Mutex
}
$ go build f.go
# command-line-arguments
./f.go:4:4: undefined: sync

What did you see instead?
A confusing error about sync, and a redundant error about Mutex:
$ cat f.go
package p

type t struct {
        sync.Mutex
}
$ go build f.go
# command-line-arguments
./f.go:4:2: sync is not a package
./f.go:4:2: undefined: Mutex

/cc @griesemer

Indeed. Could use a better error message.

I would like to give it a try, may anyone give me some pointers?

FWIW, go/types reports: "undeclared name: sync" which seems clearer.
@hellozee Not sure if this is a great "starter issue" in the compiler as it seems pretty central to how name lookup works - but I'm just guessing; haven't looked into this yet. But basically 1) you'd have to find which part of the code reports "sync is not a package" and why it's not complaining about sync not being defined instead. And 2) mark the respective node with Broke or SetDiag so that we don't get a 2nd error.

@griesemer sure

@griesemer the files in question would be src/cmd/compile/internal/gc/noder.go , packname method of the noder structure. And src/cmd/compile/internal/gc/typecheck.go, typecheck1 function.
Am I going in the right direction?

@hellozee Yes, but it seems like this issue requires a bit more thought since the problem appears during "noding". It may be possible to avoid this with some restructuring, but that wouldn't be a "starter issue".

Ohh better to leave it to experts then, thanks for heads up though @griesemer


Copilot autocomplete is ignored when there are trailing lines in a document, and no characters present on the line of the suggestion. The tab key inserts spaces instead

Check for existing issues

[X] Completed

Describe the bug / provide steps to reproduce it
When pressing tab to autocomplete a suggestion given by copilot, the editor will accept the copilot suggestion if there are no trailing lines in the document. However, if a trailing line exists, the copilot suggestion is ignored, and tabs are inserted. This is unexpected since it should accept the copilot suggestion but does not.
This only happens when no text is entered and a copilot suggestion is available anyway such as when closing brackets, etc. If text etc. is entered and subsequently a copilot suggestion exists, pressing tab has the expected behavior of autocompleting the copilot suggestion, even if trailing lines are present.
The attached videos show Rust but I was able to replicate this in JS and TS too, so it seems to be language agnostic I think.
Environment
Zed: v0.97.5 (stable)
OS: macOS 13.4.1
Memory: 64 GiB
Architecture: aarch64
If applicable, add mockups / screenshots to help explain present your vision of the feature
The first video shows the unexpected behavior. Spaces are inserted despite a copilot suggestion being present.
https://github.com/zed-industries/community/assets/10237081/17f27674-c9f4-49ea-b357-5b288c73497e
These two show the expected behavior. Whenever a copilot suggestion exists, pressing tab should insert that. This occurs when trailing lines do not exist, or at least 1 char is present on the line where copilot suggestions are being inserted.
https://github.com/zed-industries/community/assets/10237081/c7727663-3edd-4c4b-bc86-49934d6c02a5
https://github.com/zed-industries/community/assets/10237081/b13e2dd9-fb07-46fa-92cb-9e9c07ff208d
If applicable, attach your ~/Library/Logs/Zed/Zed.log file to this issue.
If you only need the most recent lines, you can run the zed: open log command palette action to see the last 1000.
Tailing the logs while trying to get it to accept the autocomplete suggestion shows no new output. Still the last 1000 lines of logs are attached below.
zed-log.txt

Thanks for the detailed report.

i ran into this behavior while trying to use tab to accept a copilot suggestion for a case statement in a .js file. here’s what is looked like for me:
https://github.com/zed-industries/community/assets/517889/8681a899-caa5-4768-8033-36eb88e93d3c


Provide way to access underlying CameraController

Is your feature request related to a problem? Please describe.
The specific use case I'm looking to solve is to know when the camera tap-to-focus has completed. The current tap-to-focus functionality does not hook into CameraController#getTapToFocusState and seems to rely on an arbitrary delay to consider the focusing "done".
While this should theoretically be independently solvable, the more general "issue" here is the availability of the data but a restrction on access. Since CameraController is the interface to the Camera, it would be nice if that was directly exposed instead of needing native support within the library for a specific feature.
Describe the solution you'd like
It would be nice if the controller property in CameraState was publicly accessible (currently it is declared as internal)
The KDoc for the property in this library even says itself (emphasis mine): Main controller from CameraX. **useful in cases that haven't been release some feature yet**.
And yet it cannot be useful if it is not marked as public

Hey @vanshg !
I misunderstood it. The onFocus lambda is intended for the focusTapContent in the composable, not for CameraController#getTapToFocusState. Maybe I'll need to rename and refactor this later 🤔
Nevertheless, I appreciate your pull request and I'll accept it. I'm going to work on releasing another version ASAP.
Thanks a lot!


MacOS support

I have only tested it on an arm64 based Mac.

Related to https://github.com/microsoft/vcpkg/pull/31203

Thanks for your addition, MacOSX is not a supported platform for the full stack, maybe it compiles by accident, but not tested at all, ACE/TAO are also not tested on MacOSX.

Hmm... Then we could either add a message notifying the user that MacOS is not a supported platform or just leave this PR open, so that people who want to build it for MacOS, may patch it themselves.
I might add Android and IOS as well, but for that, I want to refactor the vcpkg port to support cross-compiling and support the tools (ridlc, brix11, ..., ...) properly. I think it would take the pain out of building ACE/TAO and axcioma. brix11 already simplifies the process a whole lot, but something like vcpkg goes one step further.

Okay, I ran the tests. At least on an arm64 Mac, there are several failed tests, which, as far as I remember, was not the case on a x64_86 machine a few years ago. I will take care of them over time.

See https://www.taox11.org/scoreboard.html and https://www.axcioma.org/scoreboard.html for the platforms we officially support, on the supported platforms all TAOX11 unit tests do succeed. For ARM64 mac also cross platform testing is something that must be done, I have read multiple times that people had problems with interoperability between ARM64 mac and Intel.

For ARM mac there is a PR on ACE/TAO master that seems to be pretty important, see https://github.com/DOCGroup/ACE_TAO/pull/2142

First, thank you for this patch!
I have tested it on my iMack (i386):
But it  Compiles only with c++std=c++17!
I was wondering about the empty DYLD_LIBRARY_PATH?
cat configure.log
export ACE_ROOT=/Users/clausklein/Workspace/cpp/axcioma/ACE/ACE
export DYLD_LIBRARY_PATH=
export LD_LIBRARY_PATH=/Users/clausklein/Workspace/cpp/axcioma/lib:/Users/clausklein/Workspace/cpp/axcioma/ACE/ACE/lib:
export MPC_BASE=/Users/clausklein/Workspace/cpp/axcioma/taox11/bin/MPC
export MPC_ROOT=/Users/clausklein/Workspace/cpp/axcioma/ACE/MPC
export PATH=/Users/clausklein/Workspace/cpp/axcioma/bin:/Users/clausklein/Workspace/cpp/axcioma/taox11/bin:/usr/local/opt/llvm/bin:/usr/local/opt/llvm/bin://Users/clausklein/perl5/bin:/usr/local/opt/net-snmp/bin:/usr/local/opt/coreutils/libexec/gnubin:/usr/local/libexec/bin:/usr/local/opt/curl/bin:/usr/local/opt/libxml2/bin:/usr/local/opt/libxslt/bin:/usr/local/opt/expat/bin:/usr/local/opt/net-snmp/bin:/Users/clausklein/Library/Python/3.9/bin:/usr/local/opt/python/libexec/bin:/usr/local/opt/sqlite/bin:/usr/local/bin:/usr/local/sbin:/Users/clausklein/scripts:/Users/clausklein/bin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/texbin:/usr/local/MacGPG2/bin:/opt/X11/bin:/Library/Apple/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin
export RIDL_BE_PATH=:/Users/clausklein/Workspace/cpp/axcioma/taox11
export RIDL_BE_SELECT=c++11
export RIDL_ROOT=/Users/clausklein/Workspace/cpp/axcioma/ridl/lib
export TAOX11_ROOT=/Users/clausklein/Workspace/cpp/axcioma/taox11
export TAO_ROOT=/Users/clausklein/Workspace/cpp/axcioma/ACE/TAO
export X11_BASE_ROOT=/Users/clausklein/Workspace/cpp/axcioma
========================= config.h =========================
#define ACE_HAS_VERSIONED_NAMESPACE 1
#define ACE_MONITOR_FRAMEWORK 0
#include "ace/config-macosx.h"
=================== platform_macros.GNU ====================
crossbuild=0
ssl=0
bzip2=0
zlib=0
valgrind=0
valgrind_keep_debuginfo=0
acetaompc=1
cross=0
debug=0
inline=1
optimize=1
buildbits=64
include $(ACE_ROOT)/include/makeinclude/platform_macosx.GNU
c++std=c++17
===================== default.features =====================
crossbuild=0
ssl=0
bzip2=0
zlib=0
valgrind=0
valgrind_keep_debuginfo=0
acetaompc=1
cross=0
========================= MPC.cfg ==========================
includes = $MPC_ROOT/config, $ACE_ROOT/bin/MakeProjectCreator/config, $TAOX11_ROOT/bin/MPC/config
dynamic_types = $MPC_ROOT, $TAO_ROOT/MPC, $ACE_ROOT/bin/MakeProjectCreator, $TAOX11_ROOT/bin/MPC
main_functions = cplusplus:ACE_TMAIN
========================= .ridlrc ==========================
{
  "backend": "c++11",
  "be_path": [
    "taox11"
  ]
}
======================== .brix11rc =========================
{
  "project_type": "gnuautobuild",
  "project_compiler": "gnu",
  "brix_paths": [
    "taox11"
  ],
  "user_environment": {
    "ACE_ROOT": "/Users/clausklein/Workspace/cpp/axcioma/ACE/ACE",
    "TAO_ROOT": "/Users/clausklein/Workspace/cpp/axcioma/ACE/TAO",
    "MPC_ROOT": "/Users/clausklein/Workspace/cpp/axcioma/ACE/MPC",
    "DYLD_LIBRARY_PATH": "${DYLD_LIBRARY_PATH}"
  },
  "crossbuild": false,
  "target_platform": "Darwin"
}
====================== workspace.mwc =======================
workspace {
  $(ACE_ROOT)/ace
  $(ACE_ROOT)/apps/gperf/src
  $(TAO_ROOT)/TAO_IDL
  $(TAO_ROOT)/tao
  $(TAO_ROOT)/orbsvcs/orbsvcs/CosNaming.mpc
  $(TAO_ROOT)/orbsvcs/orbsvcs/CosNaming_Serv.mpc
  $(TAO_ROOT)/orbsvcs/orbsvcs/CosNaming_Skel.mpc
  $(TAO_ROOT)/orbsvcs/Naming_Service
  $(TAO_ROOT)/orbsvcs/orbsvcs/Svc_Utils.mpc
  $(TAO_ROOT)/orbsvcs/tests/Simple_Naming
  $(TAO_ROOT)/utils/catior/catior.mpc
  $(TAOX11_ROOT)/tao/x11
  $(TAOX11_ROOT)/orbsvcs/orbsvcs
}
============================================================
Test config: MACOSX FIXED_BUGS_ONLY ACETAOMPC

bash-5.2$ builddriver bin/brix11 make
builddriver executing: 'bin/brix11 make'
Compilation SUCCEED in 379.300847 seconds
Number of warnings: 16
WarningErrorEntry(path='WARNING: (g++) file: /Users/clausklein/Workspace/cpp/axcioma/ACE/ACE/ace/OS_NS_stdlib.inl line: 265 - /Users/clausklein/Workspace/cpp/axcioma/ACE/ACE/ace/OS_NS_stdlib.inl', lineno='265', severity='warning', message="'mktemp' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of mktemp(3), it is highly recommended that you use mkstemp(3) instead. [-Wdeprecated-declarations]", column='12')
WarningErrorEntry(path='INFO: (g++) file: /Users/clausklein/Workspace/cpp/axcioma/ACE/ACE/ace/OS_NS_stdlib.inl', lineno='265', severity='warning', message="'mktemp' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of mktemp(3), it is highly recommended that you use mkstemp(3) instead. [-Wdeprecated-declarations]", column=None)
WarningErrorEntry(path='INFO: (g++) file: i/Users/clausklein/Workspace/cpp/axcioma/ACE/ACE/ace/OS_NS_stdio.inle  - l1045riecom', lineno='15', severity='warning', message="m'vsprintf' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use vsnprintf(3) instead. [-Wdeprecated-declarations]", column=None)
WarningErrorEntry(path='110 | namespace TAOX11_../../tao/x11/WStringSeqC.h', lineno='115', severity='warning', message='nested namespace definition is a C++17 extension; define each namespace separately [-Wc++17-extensions]', column=None)
WarningErrorEntry(path='WARNING: (g++) file: ../../tao/x11/DoubleSeqC.h', lineno='115', severity='warning', message='nested namespace definition is a C++17 extension; define each namespace separately [-Wc++17-extensions]', column=None)
WarningErrorEntry(path='WARNING: (g++) file: ../../tao/x11/ServicesC.h line: 432 - ../../tao/x11/WCharSeqC.h', lineno='115', severity='warning', message='nested namespace definition is a C++17 extension; define each namespace separately [-Wc++17-extensions]', column='27')
WarningErrorEntry(path='WARNING: (g++) file: ../../tao/x11/Typecode_typesC.h line: 250 - ../../tao/x11/LongDoubleSeqC.hIn file included from', lineno='115', severity='warning', message='/Users/clausklein/Workspace/cpp/axcioma/taox11/tao/x11/orb_registry.cpp  :15:', column='27')
WarningErrorEntry(path='WARNING: (g++) file: ../../tao/x11/StringSeqC.h line: 110 -   182 | ../../tao/x11/BooleanSeqC.h', lineno='110', severity='warning', message='nested namespace definition is a C++17 extension; define each namespace separately [-Wc++17-extensions]', column='27')
WarningErrorEntry(path='WARNING: (g++) file: ../../tao/x11/CurrentC.h line: 186 - ../../tao/x11/BooleanSeqC.h', lineno='115', severity='warning', message='nested namespace definition is a C++17 extension; define each namespace separately [-Wc++17-extensions]', column='27')
WarningErrorEntry(path='WARNING: (g++) file: ../../tao/x11/TimeBaseC.h', lineno='312', severity='warning', message='nested namespace definition is a C++17 extension; define each namespace separately [-Wc++17-extensions]', column='27')
WarningErrorEntry(path='WARNING: (g++) file: ../../../tao/x11/Typecode_typesC.h line: 250 - ../../../tao/x11/Messaging_SyncScopeC.h', lineno='84', severity='warning', message='nested namespace definition is a C++17 extension; define each namespace separately [-Wc++17-extensions]', column='27')
WarningErrorEntry(path='WARNING: (g++) file: ../../../tao/x11/ShortSeqC.h', lineno='58', severity='warning', message='nested namespace definition is a C++17 extension; define each namespace separately [-Wc++17-extensions]', column=None)
WarningErrorEntry(path='WARNING: (g++) file: ../../../tao/x11/anytypecode/VisibilityC.h line: 65 - ../../../tao/x11/OctetSeqC.h', lineno='115', severity='warning', message='nested namespace definition is a C++17 extension; define each namespace separately [-Wc++17-extensions]', column='27')
WarningErrorEntry(path='914 | namespac../../../tao/x11/portable_server/RequestProcessingPolicyC.he TA:OX29811_N', lineno='27', severity='warning', message='nested namespace definition is a C++17 extension; define each namespace separately [-Wc++17-extensions]', column=None)
WarningErrorEntry(path='../../../tao/x11/DoubleSeqC.h', lineno='115', severity='warning', message='nested namespace definition is a C++17 extension; define each namespace separately [-Wc++17-extensions]', column=None)
WarningErrorEntry(path='WARNING: (g++) file: ../../../tao/x11/pi/PIForwardRequestC.h line: 162 - ../../../tao/x11/ServicesC.h', lineno='367', severity='warning', message='nested namespace definition is a C++17 extension; define each namespace separately [-Wc++17-extensions]', column='27')
For full log, please open: /var/folders/wb/ckvxxgls5db7qyhqq4y5_l1c0000gq/T/build-vn9vjl32.log
bash-5.2$

With the last ACE/TAO release the is still an compile error on OSX:
bash-5.2$ ./cmake-install.sh
+ rm -rf /Users/clausklein/Workspace/cpp/axcioma/build /Users/clausklein/Workspace/cpp/axcioma/install
+ export LD_LIBRARY_PATH=/Users/clausklein/Workspace/cpp/axcioma/stagedir/lib:/usr/local/lib:/usr/lib
+ LD_LIBRARY_PATH=/Users/clausklein/Workspace/cpp/axcioma/stagedir/lib:/usr/local/lib:/usr/lib
+ export DYLD_LIBRARY_PATH=/Users/clausklein/Workspace/cpp/axcioma/stagedir/lib:/usr/local/lib:/usr/lib
+ DYLD_LIBRARY_PATH=/Users/clausklein/Workspace/cpp/axcioma/stagedir/lib:/usr/local/lib:/usr/lib
+ cmake -S . -B /Users/clausklein/Workspace/cpp/axcioma/build -G Ninja -D CMAKE_CXX_COMPILER_LAUNCHER=/usr/local/bin/ccache -D CMAKE_BUILD_TYPE=Release -D CMAKE_SKIP_BUILD_RPATH=OFF -D CMAKE_INSTALL_RPATH=/Users/clausklein/Workspace/cpp/axcioma/stagedir/lib -D BUILD_WITH_INSTALL_NAME_DIR=OFF -D CMAKE_BUILD_WITH_INSTALL_RPATH=ON -D CMAKE_INSTALL_RPATH_USE_LINK_PATH=ON -D CMAKE_STAGING_PREFIX=/Users/clausklein/Workspace/cpp/axcioma/install -D CMAKE_PREFIX_PATH=/Users/clausklein/Workspace/cpp/axcioma/stagedir -D CMAKE_CXX_STANDARD=17 -D BUILD_SHARED_LIBS=ON -Wdev -Wdeprecated
-- The CXX compiler identification is AppleClang <IP_ADDRESS>00040
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Git: /usr/local/bin/git (found version "2.42.0")
-- CPM: Adding package<EMAIL_ADDRESS>(v1.11.0 at /Users/clausklein/.cache/CPM/packageproject.cmake/fc402989ee6e628476b3d8661f8f96774d89df74)
-- CPM: Adding package Callback_Quoter@0 (6e4b1f50e70a2ee8e224bf1b176a915053f8e8d6 at /Users/clausklein/.cache/CPM/callback_quoter/512066631536c9c8aa26f34d5fd54c93052f620f)
-- Configuring done (6.0s)
-- Generating done (0.1s)
CMake Warning:
  Manually-specified variables were not used by the project:

    BUILD_WITH_INSTALL_NAME_DIR

-- Build files have been written to: /Users/clausklein/Workspace/cpp/axcioma/build
+ cmake --build /Users/clausklein/Workspace/cpp/axcioma/build --target all
[14/22] Building CXX object _deps/callback_quoter-build/CMakeFiles/idl2cpp.dir/NotifierC.cpp.o
FAILED: _deps/callback_quoter-build/CMakeFiles/idl2cpp.dir/NotifierC.cpp.o
/usr/local/bin/ccache /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -DACE_HAS_CUSTOM_EXPORT_MACROS=0 -DACE_HAS_IPV6 -DACE_HAS_VERSIONED_NAMESPACE=1 -D__ACE_INLINE__ -I/Users/clausklein/Workspace/cpp/axcioma/build/_deps/callback_quoter-build -I/Users/clausklein/Workspace/cpp/axcioma/taox11 -I/Users/clausklein/Workspace/cpp/axcioma/taox11/orbsvcs -I/Users/clausklein/Workspace/cpp/axcioma/ACE/ACE -I/Users/clausklein/Workspace/cpp/axcioma/ACE/TAO -I/Users/clausklein/Workspace/cpp/axcioma/ACE/TAO/orbsvcs -O3 -DNDEBUG -std=gnu++17 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.0.sdk -mmacosx-version-min=13.6 -fdelayed-template-parsing -Wno-deprecated-declarations -MMD -MT _deps/callback_quoter-build/CMakeFiles/idl2cpp.dir/NotifierC.cpp.o -MF _deps/callback_quoter-build/CMakeFiles/idl2cpp.dir/NotifierC.cpp.o.d -o _deps/callback_quoter-build/CMakeFiles/idl2cpp.dir/NotifierC.cpp.o -c /Users/clausklein/Workspace/cpp/axcioma/build/_deps/callback_quoter-build/NotifierC.cpp
In file included from /Users/clausklein/Workspace/cpp/axcioma/build/_deps/callback_quoter-build/NotifierC.cpp:14:
In file included from /Users/clausklein/Workspace/cpp/axcioma/ACE/TAO/tao/AnyTypeCode/Any_Impl_T.h:80:
In file included from /Users/clausklein/Workspace/cpp/axcioma/ACE/TAO/tao/AnyTypeCode/Any_Impl_T.cpp:5:
In file included from /Users/clausklein/Workspace/cpp/axcioma/ACE/TAO/tao/AnyTypeCode/Any_Unknown_IDL_Type.h:17:
In file included from /Users/clausklein/Workspace/cpp/axcioma/ACE/ACE/ace/Refcounted_Auto_Ptr.h:16:
/Users/clausklein/Workspace/cpp/axcioma/ACE/ACE/ace/Auto_Ptr.h:73:12: error: no member named 'auto_ptr' in namespace 'std'
using std::auto_ptr;
      ~~~~~^
1 error generated.
[15/22] Building CXX object _deps/callback_quoter-build/CMakeFiles/idl2cpp.dir/ConsumerC.cpp.o
FAILED: _deps/callback_quoter-build/CMakeFiles/idl2cpp.dir/ConsumerC.cpp.o
/usr/local/bin/ccache /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -DACE_HAS_CUSTOM_EXPORT_MACROS=0 -DACE_HAS_IPV6 -DACE_HAS_VERSIONED_NAMESPACE=1 -D__ACE_INLINE__ -I/Users/clausklein/Workspace/cpp/axcioma/build/_deps/callback_quoter-build -I/Users/clausklein/Workspace/cpp/axcioma/taox11 -I/Users/clausklein/Workspace/cpp/axcioma/taox11/orbsvcs -I/Users/clausklein/Workspace/cpp/axcioma/ACE/ACE -I/Users/clausklein/Workspace/cpp/axcioma/ACE/TAO -I/Users/clausklein/Workspace/cpp/axcioma/ACE/TAO/orbsvcs -O3 -DNDEBUG -std=gnu++17 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.0.sdk -mmacosx-version-min=13.6 -fdelayed-template-parsing -Wno-deprecated-declarations -MMD -MT _deps/callback_quoter-build/CMakeFiles/idl2cpp.dir/ConsumerC.cpp.o -MF _deps/callback_quoter-build/CMakeFiles/idl2cpp.dir/ConsumerC.cpp.o.d -o _deps/callback_quoter-build/CMakeFiles/idl2cpp.dir/ConsumerC.cpp.o -c /Users/clausklein/Workspace/cpp/axcioma/build/_deps/callback_quoter-build/ConsumerC.cpp
In file included from /Users/clausklein/Workspace/cpp/axcioma/build/_deps/callback_quoter-build/ConsumerC.cpp:14:
In file included from /Users/clausklein/Workspace/cpp/axcioma/ACE/TAO/tao/AnyTypeCode/Any_Impl_T.h:80:
In file included from /Users/clausklein/Workspace/cpp/axcioma/ACE/TAO/tao/AnyTypeCode/Any_Impl_T.cpp:5:
In file included from /Users/clausklein/Workspace/cpp/axcioma/ACE/TAO/tao/AnyTypeCode/Any_Unknown_IDL_Type.h:17:
In file included from /Users/clausklein/Workspace/cpp/axcioma/ACE/ACE/ace/Refcounted_Auto_Ptr.h:16:
/Users/clausklein/Workspace/cpp/axcioma/ACE/ACE/ace/Auto_Ptr.h:73:12: error: no member named 'auto_ptr' in namespace 'std'
using std::auto_ptr;
      ~~~~~^
1 error generated.
[16/22] Building CXX object _deps/callback_quoter-build/CMakeFiles/idl2cpp.dir/ConsumerS.cpp.o
FAILED: _deps/callback_quoter-build/CMakeFiles/idl2cpp.dir/ConsumerS.cpp.o
/usr/local/bin/ccache /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -DACE_HAS_CUSTOM_EXPORT_MACROS=0 -DACE_HAS_IPV6 -DACE_HAS_VERSIONED_NAMESPACE=1 -D__ACE_INLINE__ -I/Users/clausklein/Workspace/cpp/axcioma/build/_deps/callback_quoter-build -I/Users/clausklein/Workspace/cpp/axcioma/taox11 -I/Users/clausklein/Workspace/cpp/axcioma/taox11/orbsvcs -I/Users/clausklein/Workspace/cpp/axcioma/ACE/ACE -I/Users/clausklein/Workspace/cpp/axcioma/ACE/TAO -I/Users/clausklein/Workspace/cpp/axcioma/ACE/TAO/orbsvcs -O3 -DNDEBUG -std=gnu++17 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.0.sdk -mmacosx-version-min=13.6 -fdelayed-template-parsing -Wno-deprecated-declarations -MMD -MT _deps/callback_quoter-build/CMakeFiles/idl2cpp.dir/ConsumerS.cpp.o -MF _deps/callback_quoter-build/CMakeFiles/idl2cpp.dir/ConsumerS.cpp.o.d -o _deps/callback_quoter-build/CMakeFiles/idl2cpp.dir/ConsumerS.cpp.o -c /Users/clausklein/Workspace/cpp/axcioma/build/_deps/callback_quoter-build/ConsumerS.cpp
In file included from /Users/clausklein/Workspace/cpp/axcioma/build/_deps/callback_quoter-build/ConsumerS.cpp:14:
In file included from /Users/clausklein/Workspace/cpp/axcioma/ACE/TAO/tao/ORB_Core.h:17:
In file included from /Users/clausklein/Workspace/cpp/axcioma/ACE/TAO/tao/Resource_Factory.h:26:
/Users/clausklein/Workspace/cpp/axcioma/ACE/ACE/ace/Auto_Ptr.h:73:12: error: no member named 'auto_ptr' in namespace 'std'
using std::auto_ptr;
      ~~~~~^
1 error generated.
[17/22] Building CXX object _deps/callback_quoter-build/CMakeFiles/idl2cpp.dir/NotifierS.cpp.o
FAILED: _deps/callback_quoter-build/CMakeFiles/idl2cpp.dir/NotifierS.cpp.o
/usr/local/bin/ccache /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -DACE_HAS_CUSTOM_EXPORT_MACROS=0 -DACE_HAS_IPV6 -DACE_HAS_VERSIONED_NAMESPACE=1 -D__ACE_INLINE__ -I/Users/clausklein/Workspace/cpp/axcioma/build/_deps/callback_quoter-build -I/Users/clausklein/Workspace/cpp/axcioma/taox11 -I/Users/clausklein/Workspace/cpp/axcioma/taox11/orbsvcs -I/Users/clausklein/Workspace/cpp/axcioma/ACE/ACE -I/Users/clausklein/Workspace/cpp/axcioma/ACE/TAO -I/Users/clausklein/Workspace/cpp/axcioma/ACE/TAO/orbsvcs -O3 -DNDEBUG -std=gnu++17 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.0.sdk -mmacosx-version-min=13.6 -fdelayed-template-parsing -Wno-deprecated-declarations -MMD -MT _deps/callback_quoter-build/CMakeFiles/idl2cpp.dir/NotifierS.cpp.o -MF _deps/callback_quoter-build/CMakeFiles/idl2cpp.dir/NotifierS.cpp.o.d -o _deps/callback_quoter-build/CMakeFiles/idl2cpp.dir/NotifierS.cpp.o -c /Users/clausklein/Workspace/cpp/axcioma/build/_deps/callback_quoter-build/NotifierS.cpp
In file included from /Users/clausklein/Workspace/cpp/axcioma/build/_deps/callback_quoter-build/NotifierS.cpp:14:
In file included from /Users/clausklein/Workspace/cpp/axcioma/ACE/TAO/tao/ORB_Core.h:17:
In file included from /Users/clausklein/Workspace/cpp/axcioma/ACE/TAO/tao/Resource_Factory.h:26:
/Users/clausklein/Workspace/cpp/axcioma/ACE/ACE/ace/Auto_Ptr.h:73:12: error: no member named 'auto_ptr' in namespace 'std'
using std::auto_ptr;
      ~~~~~^
1 error generated.
[18/22] Building CXX object _deps/callback_quoter-build/CMakeFiles/supplier.dir/supplier.cpp.o
ninja: build stopped: subcommand failed.
bash-5.2$
``


PostgreSQL Compatibility Page

Updating the porting from postgres page to become a postgres compatibility page

This change is

Online preview: http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/b349bae49a2de0d152641d71dbfd320e221f89f7/

Online preview: http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/4b8cecc8423a96cb14b5697c3875324b418879c8/
Edited pages:

releases/v19.2.0-rc.2.md
releases/v19.2.0.md
v19.2/migrate-from-oracle.md
v19.2/migration-overview.md
v19.2/selection-queries.md


bug in file attachment under manual journal in accountants

issue not fixed! @saranya13-tech please recheck

issue fixed!


onDismiss called immediately

The tests are failing for me but this is the test:
Create a new Crouton and check that the onDismiss function is not called before the timeout has expired.
it('should not call onDismiss directly', function() {
    var dismissCalled = 0;
    var crouton = TestUtils.renderIntoDocument(
      Crouton({
        id: 123456,
        message: 'data.message',
        type: 'errorClassName',
        hidden: false,
        onDismiss: function() {
          dismissCalled++;
        }
      }))

      expect(dismissCalled, 0)
  })

At the moment this test would fail because index.js::changeState is called from componentWillMount where this line is executed:
this.dismiss();

Solutions?

Uncomment the line? The dismiss should be called (eventually) due the setTimeout
Do not call changeState from componentWillMount but instead use getInitialState and getDefaultProps for the 'startup' logic

@Laoujin I have remove this.dismiss() , I think this will works with you and already publish to npm.
You can upgrade and try.


Update effectRenderer.ts

CC @sebavan

Please make sure to label your PR with "bug", "new feature" or "breaking change" label(s).
To prevent this PR from going to the changelog marked it with the "skip changelog" label.

Snapshot stored with reference name:
refs/pull/15657/merge
Test environment:
https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/15657/merge/index.html
To test a playground add it to the URL, for example:
https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/15657/merge/index.html#WGZLGJ#4600
Links to test babylon tools with this snapshot:
https://playground.babylonjs.com/?snapshot=refs/pull/15657/merge
https://sandbox.babylonjs.com/?snapshot=refs/pull/15657/merge
https://gui.babylonjs.com/?snapshot=refs/pull/15657/merge
https://nme.babylonjs.com/?snapshot=refs/pull/15657/merge
To test the snapshot in the playground with a playground ID add it after the snapshot query string:
https://playground.babylonjs.com/?snapshot=refs/pull/15657/merge#BCU1XR#0

Visualization tests for WebGPU (Experimental)
https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/15657/merge/testResults/webgpuplaywright/index.html

WebGL2 visualization test reporter:
https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/15657/merge/testResults/webgl2playwright/index.html


Properly Implement Featured

Is this feature Currently Implemented in any way with the upstream API Server

[X] Yes
[ ] No

Describe the solution you'd like
To accomplish the Featured Packages, as suggested by @Spiker985, we can every week run a task, that looks at every packages download change over the week. And whichever has the highest change is used on the featured page. Maybe trunicated to 30 items. Then once the week is over this is checked again. This can be accomplished by each time its run take a snapshot of the download count, maybe under download_count_snapshot which contains the download count at that time, for it to be looked at again the next week. This new download_count_snapshot will need to be added when a package is installed, and pruned during show, but otherwise should be relatively easy to implement.

I would probably use two fields, prev_download_count and current_download_count or something similar
That way you just cycle current -> prev, and fill current. Bonus points if you have a calculated field for the delta, and you can use that with a weighting system to produce a mean value for use in the featured page.
Though I haven't looked at how you're storing your data, so the 3 vs 1 field may be overkill

@Spiker985 currently a download count is stored, which is constant tick up or down depending if the package is installed or uninstalled. So while it may not be needed for a current_download_count we could store the calculated delta, but absolutely as we were talking about on the Discord Server, I do like the weighing system you're mentioning here. Although not entirely sure what youre meaning about the mean value from the weighting system. Are you meaning to basically figure out the normal distribution compared to all packages, and take the mean and up as the featured packages?

More or less.
Take this as an example:
Package A has 0 downloads
Package B has 200 downloads
If package A gets 10 downloads, bringing the total to 10, the delta is 10, but the percent of change is 1000%
If package B gets 100 downloads, bringing the total to 300, the delta is 100, but the percent of change is 50%
Furthermore, if you take the delta, divided by the time the data occured, you get an average over time.
Package A, 10 downloads over 7 days, is an average of 1.43 downloads a day.
Package B, 100 downloads over 7 days, is an average of 14.3 downloads a day.
So you can see that depending on how you want to calculate the popularity of downloads, you may need to weigh the results.

Thanks for taking the time to explain it thoroughly, and this does raise a fantastic point about what should be valued.
And the count of what truly is more popular I feel would be looking at the average downloads a day. Since that may be the most agnostic way to look at it. But some more research may need to be done to determine the best method

Since version 2.x will not occur on this repo, and instead will occur on the main Pulsar-Edit repo, I'll close this issue, and add it to the roadmap of the Backend on Pulsar.


Sync project from bitbucket

Summary by CodeRabbit

New Feature: Introduced SharedPreferencesHelper and DeviceUtil classes to manage shared preferences and retrieve device information, enhancing the app's ability to remember user preferences and adapt to different devices.
Refactor: Simplified the BrgWidgetEventBus and SignalrConnectionManager classes, improving code readability and maintainability.
Refactor: Enhanced the NetworkManager class to include additional headers for better network requests handling.
Bug Fix: Updated the BrgError class to allow nullable errorCode and message properties, preventing potential crashes when these values are not available.
Style: Updated the theme and UI elements of the main Flutter application for a more engaging user experience.

Review suggestions will be implemented later


Bugfix/ssh key path

Codecov Report

Merging #67 (1a86761) into master (b737b36) will decrease coverage by 0.21%.
The diff coverage is 66.66%.

@@            Coverage Diff             @@
##           master      #67      +/-   ##
==========================================
- Coverage   62.91%   62.69%   -0.22%
==========================================
  Files          34       34
  Lines        4845     4847       +2
==========================================
- Hits         3048     3039       -9
- Misses       1797     1808      +11

Impacted Files
Coverage Δ

turbo_tunnel/ssh.py
55.36% <50.00%> (-0.60%)
:arrow_down:

turbo_tunnel/__init__.py
88.23% <100.00%> (ø)

tests/__init__.py
80.00% <0.00%> (-20.00%)
:arrow_down:

turbo_tunnel/utils.py
73.68% <0.00%> (-1.39%)
:arrow_down:

tests/test_ssh.py
99.09% <0.00%> (-0.91%)
:arrow_down:

turbo_tunnel/icmp.py
70.66% <0.00%> (-0.11%)
:arrow_down:

:mega: Codecov can now indicate which changes are the most critical in Pull Requests. Learn more


Path Generation, Sprite Shape, some handy functions

I've added some new features and a few handy functions. Here's a list:
PathGenerator.cs
Generates a 2D path over time, and creates a sprite shape that matches it, as well as a collider.
PathGenerator_Road.cs
Same as above, except it generates a road mesh using Sebastian's road mesh creator.
PathToSpriteShape.cs:
A static class used as a tool to create a Sprite Shape spline from a path. To use sprite shapes, add the 2D SpriteShape package from the Package Manager.
Use the int UpdateSpriteShape(SpriteShapeController controller, PathCreator pathCreator, int j) function.  Pretty accurate for open paths. Tangents of the last point aren't correct on closed paths though, and have to be adjusted manually at the moment.
PathCollider.cs
A simple script to create an edge collider from a path and set a PhysicsMaterial2D to it.
Functions added in VertexPath.cs:
float CalculatePercentByPosition(Vector3 position) - calculates a rough path percentage/time based on the distance to the path.
int GetNearestVertex(Vector3 position) - Returns the index of the nearest vertex to a given position.
Vector3 GetNearestVertexPosition(Vector3 position) Returns the position of the nearest vertex.
Vector3 GetDirection(int index, EndOfPathInstruction endOfPathInstruction = EndOfPathInstruction.Loop) - returns direction from index, from the tangents array.
Vector3[] anchorTangents array, has tangent data for the anchor points of the bezier path.
float GetUniformSpeed(float speed) - returns speed as a percentage of the length of the path, allowing objects of paths that have different lengths to travel at the same speed.
Additions to VertexPathUtility.cs
Added List<Vector3> anchorTangents in PathSplitData . It saves the tangent vectors of the anchor points.
Changes in BezierPath.cs:
Made points list public.
I've also added a few test scenes and test scripts.

Objects moving at the same speed on different paths, with sprite shapes generated along the paths

Hi, thanks for contributing. I can't review some of this at the moment though due to the use of Odin Inspector (which I don't own and don't want as a dependency for this tool). The SpriteShapeController script also appears to be missing.

Heya! No worries.
The use of Odin is only in a test script (ConvertToSpriteShape) to call a method from the editor, so it can be ignored if needed, or I can try to create a normal editor or remove it. You can use the static PathToSpriteShape class without it.
The Sprite Shape Controller is from the 2D Sprite Shape package from Unity. What I've done is create a way to convert between your paths and Sprite Shape splines, so that you can use the sprite shape system, which isn't as robust when it comes to using in real time scripts IMO.
In any case, thank you for this package! I'm working on recreating a game jam game I did last year using it at the moment, and it's going great!

Hi, thanks for contributing. I can't review some of this at the moment though due to the use of Odin Inspector (which I don't own and don't want as a dependency for this tool). The SpriteShapeController script also appears to be missing.

I have removed the file that uses Odin from the repo. I don't know if you'd want the sprite shape stuff as it does use an external package, but I thought it'd be a nice thing to have. Maybe an add-on?
If you have some time, could you look at what I've added in the core classes and see if there's anything I could have done better? Or if there's anything else you need/want me to try and implement. I'm still not as familiar with it as I'd like, and I'm adding things that could be useful or that I think I'll need for my project. Thanks!

Great idea with the GetNearestVertex methods. I've changed the implementation a little because I think its more useful to get the actual nearest point on the path, rather than nearest vertex. The sprite stuff is cool, I will have to think about how to incorporate that at some point without requiring the sprite package.

I'll close this for now and separate the path generation script from the sprite shape/road mesh stuff so it can be used alone if you want to include it in the examples.

I've changed the implementation a little because I think its more useful to get the actual nearest point on the path, rather than nearest vertex.

Yeah. that's true! Good idea

The sprite stuff is cool, I will have to think about how to incorporate that at some point without requiring the sprite package.

I'll separate it from the rest of this and put it (and any other external package support I might add) into a different branch, so that my master will not require dependencies. Is it ok if I make other pull requests in the future?

Pull requests are very welcome, thanks :)


refactor: Checkbox

Overview
Refactors Checkbox to use shared system props. Check out the DarkMode GridForm here

PR Checklist

[ ] Related to designs:
[x] Related to JIRA ticket: gm-157
[x] I have run this code to verify it works
[x] This PR includes unit tests for the code change

📬Published Alpha Packages:@codecademy/gamut-kit@0.4.59-alpha.65cb1b.0@codecademy/gamut-labs@19.15.2-alpha.65cb1b.0@codecademy/gamut@34.13.2-alpha.65cb1b.0<EMAIL_ADDRESS>
🚀 Styleguide deploy preview ready!https://60de09cd2dec5d009998c01e--gamut-preview.netlify.appDeploy Logs

📬Published Alpha Packages:@codecademy/gamut-kit@0.4.59-alpha.0be9a2.0@codecademy/gamut-labs@19.15.2-alpha.0be9a2.0@codecademy/gamut@34.13.2-alpha.0be9a2.0<EMAIL_ADDRESS>
🚀 Styleguide deploy preview ready!https://60ec72af3d6d3f40c95700f7--gamut-preview.netlify.appDeploy Logs

📬Published Alpha Packages:@codecademy/gamut-kit@0.4.59-alpha.23ca17.0@codecademy/gamut-labs@19.15.2-alpha.23ca17.0@codecademy/gamut@34.13.2-alpha.23ca17.0<EMAIL_ADDRESS>
🚀 Styleguide deploy preview ready!https://60eca4dcc2415108e45c3a8f--gamut-preview.netlify.appDeploy Logs

📬Published Alpha Packages:@codecademy/gamut-kit@0.4.59-alpha.877372.0@codecademy/gamut-labs@19.15.2-alpha.877372.0@codecademy/gamut@34.13.2-alpha.877372.0<EMAIL_ADDRESS>
🚀 Styleguide deploy preview ready!https://60eca688fdf4f1009dafbe8f--gamut-preview.netlify.appDeploy Logs

📬Published Alpha Packages:@codecademy/gamut-kit@0.4.59-alpha.1bf759.0@codecademy/gamut-labs@19.15.2-alpha.1bf759.0@codecademy/gamut@34.13.2-alpha.1bf759.0<EMAIL_ADDRESS>
🚀 Styleguide deploy preview ready!https://60eca9927f8b5c0c3abd6807--gamut-preview.netlify.appDeploy Logs

📬Published Alpha Packages:@codecademy/gamut-kit@0.4.59-alpha.90b1aa.0@codecademy/gamut-labs@19.15.2-alpha.90b1aa.0@codecademy/gamut@34.13.2-alpha.90b1aa.0<EMAIL_ADDRESS>
🚀 Styleguide deploy preview ready!https://60eddd0b46370b36a6e79246--gamut-preview.netlify.appDeploy Logs

📬Published Alpha Packages:@codecademy/gamut-kit@0.4.65-alpha.a662af.0@codecademy/gamut-labs@19.16.3-alpha.a662af.0@codecademy/gamut@34.14.4-alpha.a662af.0<EMAIL_ADDRESS>
🚀 Styleguide deploy preview ready!https://60f1eb007e71970aadadbe2b--gamut-preview.netlify.appDeploy Logs

📬Published Alpha Packages:@codecademy/gamut-kit@0.4.65-alpha.f95b84.0@codecademy/gamut-labs@19.16.3-alpha.f95b84.0@codecademy/gamut@34.14.4-alpha.f95b84.0<EMAIL_ADDRESS>
🚀 Styleguide deploy preview ready!https://60f1ffbacf06352c61211678--gamut-preview.netlify.appDeploy Logs

📬Published Alpha Packages:@codecademy/gamut-kit@0.4.65-alpha.ba606a.0@codecademy/gamut-labs@19.16.3-alpha.ba606a.0@codecademy/gamut@34.14.4-alpha.ba606a.0<EMAIL_ADDRESS>
🚀 Styleguide deploy preview ready!https://60f5b98cc0d5dd5d84aa53d4--gamut-preview.netlify.appDeploy Logs

📬Published Alpha Packages:@codecademy/gamut-kit@0.4.65-alpha.43145a.0@codecademy/gamut-labs@19.16.3-alpha.43145a.0@codecademy/gamut@34.14.4-alpha.43145a.0<EMAIL_ADDRESS>
🚀 Styleguide deploy preview ready!https://60f5c944e6f72c08ab7d6cbe--gamut-preview.netlify.appDeploy Logs

📬Published Alpha Packages:@codecademy/gamut-kit@0.4.69-alpha.004322.0@codecademy/gamut-labs@19.16.7-alpha.004322.0@codecademy/gamut@35.0.2-alpha.004322.0<EMAIL_ADDRESS>
🚀 Styleguide deploy preview ready!https://60f5d56c7e1a3e0e6b65eafb--gamut-preview.netlify.appDeploy Logs

📬Published Alpha Packages:@codecademy/gamut-kit@0.4.69-alpha.0dba71.0@codecademy/gamut-labs@19.16.7-alpha.0dba71.0@codecademy/gamut@35.0.2-alpha.0dba71.0<EMAIL_ADDRESS>
🚀 Styleguide deploy preview ready!https://60f83fb13d354921b159413c--gamut-preview.netlify.appDeploy Logs

📬Published Alpha Packages:@codecademy/gamut-kit@0.4.69-alpha.e96dc7.0@codecademy/gamut-labs@19.16.7-alpha.e96dc7.0@codecademy/gamut@35.0.2-alpha.e96dc7.0<EMAIL_ADDRESS>
📬Published Alpha Packages:@codecademy/gamut-kit@0.4.69-alpha.06d917.0@codecademy/gamut-labs@19.16.7-alpha.06d917.0@codecademy/gamut@35.0.2-alpha.06d917.0<EMAIL_ADDRESS>
🚀 Styleguide deploy preview ready!https://60f88e4c6f28394292e70892--gamut-preview.netlify.appDeploy Logs

📬Published Alpha Packages:@codecademy/gamut-kit@0.4.70-alpha.0f0409.0@codecademy/gamut-labs@19.16.8-alpha.0f0409.0@codecademy/gamut@35.1.1-alpha.0f0409.0<EMAIL_ADDRESS>
🚀 Styleguide deploy preview ready!https://60f899d3f263c44f07d80c00--gamut-preview.netlify.appDeploy Logs


Fix "callback-return" issue in app/models/User.js

Expected return with your callback function.
https://codeclimate.com/github/subwaymatch/vocapin-server/app/models/User.js#issue_58db70499ead1e0001000015

Fixed via 83b640d77d88d50797da79c100055d1c9e94ee1e


CharacterCount doesn't render or initialise properly if maxlength or maxwords is not passed to the macro

What
It's a two part thing:
First is the HTML part, before JS kicks in: if omitting the maxlength or maxwords option when calling the govukCharacterCount macro, the HTML rendered displays a "You can enter up to undefined characters" message.

Second is the JavaScript part: When initialising the component with a JavaScript maxlength or maxwords config, the message disappears, but doesn't get replace by a count of the remaining character.

This will be problematic for people looking to use the JavaScript configuration to set a blanket setting accross their service. For that to work, we need to render the component without a data-maxwords or data-maxlength attribute and fix the HTML part.
And then once that's through, we need to make sure the component initialises OK with a missing length.
Who needs to work on this
Developers
Who needs to review this
Developers
Done when

[ ] HTML render shows nothing after the field it neither maxwords or maxlength is specified
[x] JavaScript initialises properly when running over a HTML generated by the macro without maxwords or maxlength

This happens with the current version of govuk-frontend, in Firefox. Latest version's JavaScript handles things OK in terms of replacing the message 🥳
However the macro still outputs an untidy text and we could either hide it as I initially suggester (using govuk-visually-hidden like the JS does), or replace the default to "You can enter a limited number of characters".

Yeah, unfortunately I think this has been around a while – see also #1725. I think it'd be nice to fix this in v4.4.0, but I don't think this is a regression.

That was just something I noticed while preparing my demo yesterday juggling between the 4.3.1 and main versions, but I prefered flagging to keep track of it.
For the JavaScript bit, thinking about it, without any default for maxlength removing the message is probably fine. We can't really set a default anyway for that maxlength as a "generic number of charaters to limit a textarea" feels quite arbitrary.
Regarding the HTML from the macro, thanks for linking to the old issue. I think with the introduction of fallbackHintText for that component, that may solve the final user experience when the field is initialised without a maximum, letting developers provide a custom message..

As a quick bit of background, we did some changes to the character counter functionality earlier this year to fix bugs and improve the screen reader experience in a number of browsers and technologies (see PR #2577 for the long read).
Consequentially, the counter is a bit of a complex web of visible (sometimes) and hidden (sometimes) elements. It may be best to avoid any solution that involves changing the visibility of elements, as this may have repercussions on those past changes, requiring some extensive re-testing!

Cheers, that's some handy stuff to be aware. I had noticed the separate visible and debounced screen-reader feedback in the JavaScript side. I think that part should be left alone indeed: no maxlength, no maxwords -> no extra feedback if you don't set one when configuring JavaScript sounds a fine way to do.
Regarding a HTML cleanup, I think there's all the options for whatever people want to do:

fallbackHintText lets you set some text
countMessage.classes lets you add a govuk-visually-hidden on the .govuk-character-count__message element, similarly to what the JS would do when it kicks in if you really want an empty area. You probably don't, though, to avoid a layout shift when the JS kicks in so we may not want to shout too much about that one. And if you never show a count message, might as well just use a plain textarea 😄


Features/interior header menu

new nav
Waiting for some spanish from senior lora

http://ci-deploys.cid-labs.com/cid-harvard/atlas-subnational-frontend/features/interior-header-menu/213/

updated with translations

heyo, is this okay to merge?

I was getting a lot of navigation highlighting bugs before, but can't reproduce now

I pushed those fixes before i left.
Greg Shapiro wrote:

I was getting a lot of navigation highlighting bugs before, but can't
reproduce now
—
Reply to this email directly or view it on GitHub
https://github.com/cid-harvard/atlas-subnational-frontend/pull/183#issuecomment-160653757.

missed that, is that what 61dc6af2 accomplished?
Greg Shapiro
617-943-9870
<EMAIL_ADDRESS>On Mon, Nov 30, 2015 at 10:00 AM, Quinn Lee<EMAIL_ADDRESS>wrote:

I pushed those fixes before i left.
Greg Shapiro wrote:

I was getting a lot of navigation highlighting bugs before, but can't
reproduce now
—
Reply to this email directly or view it on GitHub
<
https://github.com/cid-harvard/atlas-subnational-frontend/pull/183#issuecomment-160653757
.

—
Reply to this email directly or view it on GitHub
https://github.com/cid-harvard/atlas-subnational-frontend/pull/183#issuecomment-160653901
.

yup

The only issue I have is that the text overlaps a bunch in spanish mode. (A little bit for locations, slightly worse for industries, pretty bad for products)

Could that be logged as a layout bug and made into a bugfix for next sprint? The vertical version of the header layout will be in that bucket of work anyways, which will resolve a lot of that overlap.

well... I don't want to do the work twice, but I also don't want to merge anything to beta with text overlap issues in the happy case. It costs too much in the first impression. I'd settle for calling the navigation story done and waiting for the layout update before merging this up.
I should have brought it up earlier, but in general you should design for Spanish as the primary UI, and English as the secondary.

Actually the correct approach is to design for all the localized languages to work — but in this case working out the distinction between what is done technically vs what is done visually seems like where we're getting hung up here.

breadcrumbs for locations

ok do it to it. :ship:


Item "Pocket Emporium" can accumulate more than 5 charges

Reported by a player

In both screenshots I can see that "Pocket Emporium" doesn't go above 5 charges. Wrong report?

Closing because I couldn't reproduce the bug and the screenshots don't demonstrate the bug.


Suspense loader not retriggered if query is invalidated

Describe the bug
If a query is invalidated because data needs to be refetched, the React Suspense loading state is not also re-triggered.
Expected Behavior
I would expect the fallback to retrigger when:

The query is invalidated. A loader is appropriate because the data is invalid, and refetching must occur.
The refetch function is called

I would not expect the fallback to retrigger if:

The query is "refetched" by setting the refetch prop, as polling may be wanted in the background without disrupting the current screen.

To Reproduce
See codepen here:
https://codesandbox.io/s/charming-joliot-nm1pt?file=/src/index.js

This may be related to #1059?

Its not exactly the same issue, but it is related because we need to explicitly define the behavior of suspense in several scenarios, including this one. Closing this issue in favor of #1059


Acceptance Criteria 2

GIVEN theme buttons
WHEN I click a button
THEN the color theme of the timer changes
THEN there is a ‘glow’ on the theme selected
THEN the theme is saved locally

THEN there is a ‘glow’ on the theme selected

THEN the theme is saved locally in localStorage or client memory


Localisation overhaul

Changes proposed in #3154 . Requires game side update.
Highlights:

The name LocalisableStringDescriptor may be too long. Remove the term "descriptor"?
I'll write new code with NRT enabled.
For different constructing scenarios, use FromXXX instead of new to point out.
Keep the implicit conversion from string for not localisable text. Remove the implicit conversion to string to reduce misusages (For example, == was bound to (string)left == (string)right previously). ToString() is expected to give a good result.
I don't expect a great amount of not localisable text in the game that's ready for ship, thus not optimizing for them.
Interpolated string that's converted into System.FormattableString does have more overhead, but the language team is planning to improve it. Tolerate it since text formatting shouldn't be so hot in game.

I'll readily admit I have little to no sense of the target scale of localisation work that will have to be made in the end, so I might very well be wrong, but all this feels too aggressively and prematurely optimised. I'll defer to the opinions of the core team since they have experience with localising stable.

prematurely optimised

I admit this.
However, given the architecture, localisation changes will also involve plain text, especially quickly updated numbers. We should be careful to not introduce significant overhead to those.

On the topic of cleaning up LocalisableString... I'm not against pre-optimising it. However I wonder if the following structure could work instead:
LocalisableString:
    LocalisationType Type;
    string Contents;
    ILocalisationData Data;

TranslatableString : ILocalisationData
    string Key;
    string Fallback;

    implicit operator LocalisableString(TranslatableString str) => new LocalisableString
    {
        Type = LocalisationType.Translation,
        Content = Fallback,
        Data = this
    }

LocalisationEngine
    GetBindable(LocalisableString str)
        switch (str.Type)
            case LocalisationType.Translation:
                data = (TranslatableString)str.Data
                // Use data for key lookup

Usage:
new SpriteText
{
    Text = new TranslatableString("my_key", "fallback string goes here")
}

SpriteText.Text == "fallback string goes here" (via "Content")

My current implementation:
28 bytes struct on every scenario
smoogipoo's idea:
plain text: 8 bytes struct + pre-allocated string
formattable: 8 bytes struct + allocated 24 bytes
unicode: 8 bytes struct + allocated 16 bytes
Yeah don't be afraid of allocation. Big structs can cause other problems.

Not sure how to interpret that finding. I'm not optimising for allocation, I'm optimising for readability of code while an improvement to the underlying mechanism hasn't been implemented.
As far as allocations are concerned, the data will always be heap-allocated. Essentially the struct is going to be 8 (str-ptr) + 8 (data-ptr) = 16 bytes in all cases.
And the heap allocation is fine because in the cases where data is used it's not going to be changed often. One exception is maybe localisation + formatting but at the same time you also have to be conscious of formatting too many times too, so I don't think it's a point of concern for us.
The only thing that needs optimisation is the plain text scenario, for which two pointers is the absolute best you can do.

I'm not optimising for allocation, I'm optimising for readability of code while an improvement to the underlying mechanism hasn't been implemented in the language.

I was manually simulating the discriminated union feature. Complex simulation should only be used when it's getting a significant benefit, which is not the case here. The DU simulation's overhead is more on plain texts.

Well TextFlowContainer is blocking OsuCheckBox...

Yep I was fine with this last time. I think my only remaining consideration was breaking changes.

I've been busy for a long time and won't be available until later this year. You can make changes on this based on your decision.

Thanks for the heads up! Luckily you left this PR in a very good state.

While applying changes osu-side for this from scratch (just to get a feel for how large the breaking changes are) I noticed that the order of parameters for RomanisableString is opposite of what we used to have for LocalisedString. I'm not hugely against this, but was this an intentional change?

I've pushed some fixes to actually make SpriteText work (not sure if this regressed somewhere along the way. @smoogipoo need a review on this since it seems like it would be quite easy to break something.

I noticed that the order of parameters for RomanisableString is opposite of what we used to have for LocalisedString.

Having the original "source/unromanised" text go first and the romanised variant go second feels slightly more correct to me, but I wouldn't have personally cared much if it was the other way around. Seeing that you already seem to have gone through game-side usages and flipped the order I'd probably leave that as it is now.

@peppy I don't know the reasoning for the change in order, but I'm fine with it being this way.

The SpriteText changes look fine to me. I'll leave an approval with your merge.


Pin r-base (CONDA_R) to major.minor version instead of full version.

This has been discussed in conda-forge/r-base-feedstock#16. The benefit is that
we don't have to rebuild everything for every released patch version.
R does not explicitly say that it follows semantic versioning, however the codebase
gives strong evidence for this:

In the sources (base/R/library.R), they have an error if a package was build before 3.0.0, and a warning if a package was built with a newer version than the current. No warning in case of an older version.
The function update.packages() from r-base, which is recommended to be executed after updating R and copying over packages to the new version (as e.g. done by installr), has an option checkBuilt, which, when set to true rebuilds packages if and only if the package has been built under an older major.minor version. Not under a changed patch version.

In summary, all built-in facilities are designed to take care for major.minor updates, implying that major.minor.patch updates are safe and need no rebuild.
ping @jakirkham, @mingwandroid, @ocefpaf, @bgruening.

Before merging, @jakirkham please let me know if you agree with this, since it will have an effect on all R-packages in conda-forge.

Implications and recommendations mean nothing when a DLL refuses to load due to missing symbols.

@mingwandroid but this is more a bug, isn't it? This can also happen for other software and we patch these usually and talk with upstream? We try to get an official statement from some R devs about this.

I'd be interested to hear what upstream has to say about ABI compatibility, yes. What I've read so far has been very vague. But you need to also realise that changing a single configure parameter or updating to a new version of a dependency if it's static will also change the ABI of the built R.dll and once that happens, stuff you built stands a good chance of not loading or worse, crashing (e.g. if a C function got some new parameters).
ABI compatibility is just not something you can assume. You either use tools to guarantee it (which we do not have) or you play it very safe.

@mingwandroid and you mean this is different for R than for other software we have in conda-forge?

Yes, clearly. R libraries that involve compilation are dynamically loaded plugins that import the core R dlls, this is far from typical. Only Python is similar and it has a very fixed ABI (that they only break occasionally => currently if you build a library using Python 3.6.1 it will not load on Python 3.6.0 for example due to https://bugs.python.org/issue29943).
This is an example of how exacting binary compatibility is C++ is somewhat easier in some ways but in C, when a parameter has been added to a C function, it's signature does not change so the calling (old) code will happily pack parameters into the stack incorrectly. This will manifest as a very hard to debug crash rather than the relatively friendly "Failed to load shared object".

So that means that the Python ABI has been broken in 3.6.1? But Conda is also pinning only major.minor of Python, right? Isn't this a very similar issue now, or am I getting this wrong?

This is indicative that pinning only to major.minor is risky even on projects that promise to carefully curate their ABI. In this case we were able to apply a backwards compat. fix that I'll make a PR to python-feedstock for, but any non-pure Python packages built on 3.6.1 until that is merged will not run on 3.6.0.
I'm pointing out the danger here. If/when I rebase default's R on top of conda-forge's I will continue to pin to the patch level, a policy which has served me very well.

Of course, I agree that it is always more secure to pin the exact version or even the build number. However, I don't think this is more risky in R than it is in Python. To validate this, I have evaluated the R NEWS since 2010. Interestingly, they mention if packages need to be rebuild (see below). What we can see is that such recommendations never occurred in patch releases, in 7 years. Of course, it could be that the ABI has been broken without mentioning it in the release notes. But the notes are very extensive, and I think it is reasonable to put a tiny bit of trust into the R community.
CHANGES IN R-devel
CHANGES IN R 3.4.0 patched
CHANGES IN R 3.4.0
    Packages which register native routines for .C or .Fortran need to be re-installed for this version (unless installed with R-devel SVN revision r72375 or later).
    (C-level Native routine registration.) The deprecated styles member of the R_CMethodDef and R_FortranMethodDef structures has been removed. Packages using these will need to be re-installed for R 3.4.0.
CHANGES IN R 3.3.3
CHANGES IN R 3.3.2
CHANGES IN R 3.3.1
CHANGES IN R 3.3.0
    (Windows) R is now built using gcc 4.9.3. This build will require recompilation of at least those packages that include C++ code, and possibly others. A build of R-devel using the older toolchain will be temporarily available for comparison purposes.
CHANGES IN R 3.2.5
CHANGES IN R 3.2.4
CHANGES IN R 3.2.3
CHANGES IN R 3.2.2
CHANGES IN R 3.2.1
CHANGES IN R 3.2.0
    Entry points rbinom_mu, rnbinom_mu and rmultinom are remapped (by default) to Rf_rbinom_mu etc. This requires packages using them to be re-installed.
CHANGES IN R 3.1.3
CHANGES IN R 3.1.2
CHANGES IN R 3.1.1
CHANGES IN R 3.1.0
CHANGES IN R 3.0.3
CHANGES IN R 3.0.2
CHANGES IN R 3.0.1
CHANGES IN R 3.0.0
    C entry points R_qsort and R_qsort_I now have start and end as size_t to allow them to work with longer vectors on 64-bit platforms. Code using them should be recompiled.
CHANGES IN R VERSION 2.15.3
CHANGES IN R VERSION 2.15.2
CHANGES IN R VERSION 2.15.1
CHANGES IN R VERSION 2.15.0
CHANGES IN R VERSION 2.14.2
CHANGES IN R VERSION 2.14.1
CHANGES IN R VERSION 2.14.0
    All packages must have a namespace, and one is created on installation if not supplied in the sources. This means that any package without a namespace must be re-installed under this version of R (but previously-installed data-only packages without R code can still be used).
CHANGES IN R VERSION 2.13.2
CHANGES IN R VERSION 2.13.1
CHANGES IN R VERSION 2.13.0
    Installed packages with zip-ed data sets can still be used, but a warning that they should be re-installed will be given.
CHANGES IN R VERSION 2.12.2 patched
CHANGES IN R VERSION 2.12.2
CHANGES IN R VERSION 2.12.1
CHANGES IN R VERSION 2.12.0
CHANGES IN R VERSION 2.11.1 patched
CHANGES IN R VERSION 2.11.1
CHANGES IN R VERSION 2.11.0
    The R grammar now prevents using the argument name in signatures of S4 methods for $ and $<-, since they will always be called with a character string value for name. The implicit S4 generic functions have been changed to reflect this: packages which included name in the signature of their methods need to be updated and re-installed.
CHANGES IN R VERSION 2.10.1 patched
CHANGES IN R VERSION 2.10.1
CHANGES IN R VERSION 2.10.0
        Text, HTML and latex help and examples for packages installed under the new system are converted on-demand from stored parsed Rd files. (Conversions stored in packages installed under R < 2.10.0 are used if no parsed Rd files are found. It is recommended that such packages be re-installed.)

I used the following command:
cat r-news r-news-old | grep -P "(CHANGES IN|re-installed|recomp|re-comp)" | sed "s/\.\.\/\.\.\/html\/logo.jpg //" | sed "s/\[R logo\] //" > r-breakages

This is indicative that pinning only to major.minor is risky even on projects that promise to carefully curate their ABI. In this case we were able to apply a backwards compat. fix that I'll make a PR to python-feedstock for, but any non-pure Python packages built on 3.6.1 until that is merged will not run on 3.6.0 (and then they'll need to be rebuilt).

I understand that and I can appreciate both approaches. At the end is higher stability vs slightly lower maintainability cost. To be honest I am more inclined to use this approach an if we break things, we can revert it.

I'm pointing out the danger here. If/when I rebase default's R on top of conda-forge's I will continue to pin to the patch level, a policy which has served me very well.

Indeed. defaults is another beast and I, as a user, do expect that kind of care with the packages. However, this PR in conda-build-all won't affect defaults at all. The conda-forge R recipe can, and should, be as close as possible to defaults to allow for a better interoperability between defaults and conda-forge.

Reading some text file is not the same thing as listing the exports of a shared library and studying the header for changes to any functions. The R NEWS file clearly says nothing about the ABI of any static libraries that also got linked into r-base (though I was somewhat careful to avoid that esp. on Windows). I do not have time to do this at present but when I do, I will study these details against all the R versions I have personally built, using either [Dependency Walker](http://www.dependencywalker.com/) [1] or the [ABI Compliance Checker](https://lvc.github.io/abi-compliance-checker/) [2].
You need to consider both the costs of and what happens when pinning to x.y.z vs pinning to x.z in the worst case scenario (however unlikely you think that is).
The cost of pinning to x.y.z is that the CI systems will rebuild every package when a new .z release comes out. The worst case scenario is that nothing goes wrong.
The cost of pinning to x.y is less building on CI systems and less packages on anaconda.org. The worst case scenario is that if the ABI breaks you are forced to remove all the earlier packages that have overly loose pinning (as they'll remain installed even after you've tightened up your pinning and built new packages).
It is my opinion that you really do not want to pin (pun intended) into a corner here.

I think it is reasonable to put a tiny bit of trust into the R community.

It is not to do with this because conda packages link to other conda packages and are updated out of lock-step with what happens at R. Stop trying to make me look bad when all I'm doing is applying my experience building software distributions, namely MSYS2 and the R side of the Anaconda Distribution. I should also note that I am looking out for the best interests of conda-forge here and not raising objections for the fun of it.

 I apologize if it sounded like that. I really appreciate your opinion and expertise and I actually enjoy this fruitful discussion.

Let me summarize our insights so far:

We now have strong evidence that R only broke the ABI in the major.minor versions of the last 7 years.
And we know that this is not sufficient because statically linked libraries that change their version while updating to a new patch release could still break the ABI.

Now, my question would be: isn't this the same with Python? And can't we solve this by pinning those statically linked libraries in our r-base recipe and only update the pinning when we reach a new major version?

Ok, in the PR conda-forge/r-base-feedstock#16 I have now pinned all library dependencies. It looks like none of them is static, but I did it just to be sure. Given this and that R itself did not break ABI in patch versions in the last 7 years, I would vote for the following:

merge this PR
once it has landed in conda-forge, merge the PR that updates to 3.4.

Am I right that conda-smithy will then trigger updates for all R-related feedstocks that pin to R 3.4?
@conda-forge/core @ocefpaf @sodre @bgruening can we come to a decision?

Little (human) benefit (only CI time), high risk and very high fixing costs (removing underpinned packages) == don't do it.

Sorry, maybe I am still missing something here, but if you look at my summary, where is the high risk?
Another benefit is storage.

If we cannot come to a consensus here, I have another possible strategy:
We only package the second patch version of each R release (i.e. 3.3.2, 3.4.2, 3.5.2, ...). This would have the same effect in terms of maintenance, and it would not increase the risk at all.

Your pinning is too rigid. The MSYS2 packages adopt a different pinning scheme where msys2-epoch package is used to en-masse update them. This is due to it being a rolling distro where only the latest versions of each package (at a given point in time) are expected to work together correctly.
I've had enough of trying to convince you though, it's not worth it. I'll leave it for others to decide whether to follow this foolhardy course of action.

The recent Python 3.6.1 ABI debacle should be, IMHO, enough for anyone to just admit "ah, ok, that was a very close call, we should not risk it.". Luckily 3.6.1 was caught early and there was a way to back out the change (albeit at the cost of losing the bug-fix that was the root cause of the ABI breakage, if that bug-fix had been for a vulnerability we'd not have been able to back it out).

Every update of a pinning that occurs in so many recipes is a huge effort. There is always breakage, not working urls and so on. I know that because I just did the Python 3.6 migration in bioconda. Hence the proposed strategy of not doing every release. However, this is up to conda-forge, for which I cannot decide. This is what Bioconda will do if we decide to not change the pinning scheme for R in conda-forge.
The Python breakage is indeed worrying. If I don't understand you completely wrong, wouldn't the consequence be that Python should be pinned to the patch version as well? Will there be such a change for Python in the default channel? Will all pinning be down to the patch version in the future?

@mingwandroid please don't feel offended! We all appreciate your hard work here, your experience and that you try to help.
I guess the main problem we would like to solve as a broader Conda community is time and space.
CI re-builds take time, not just compute time, also our time to fix the mentioned issues from Johannes. With a bigger community we can maybe tackle this, we still need travis, circle-ci and Co to help us.
One other problem of smaller communities is space. Continuum generously provided us with some additional quota but also this is not unlimited and we need to take this into account. Not sure about conda-forge here.
So what we wanted to try here is improve this on the R side as we spend way to much time in rebuild R packages and they take so much space in comparison to Python packages. We tried and ask people and all said it is enough to pin against minor version. However, if your experience is different can we (anaconda, conda-forge, bioconda) agree that we all build at least against one minor version .2 as Johannes proposed? Then we all can decide on our own if we can afford the effort to build also against all other minor versions. I guess BioConda would then skip .0 and .1 releases and try to explain this to people waiting eagerly for there new packages.
Thanks again!

If I don't understand you completely wrong, wouldn't the consequence be that Python should be pinned to the patch version as well.

Python pinning is currently x.x, there's quite a lot of hardcoded stuff in the build system that creates conda defaults around that (but conda-build 3 should allow us to break free from that as it allows us to generalize all pinning). It's always been the case (AFAIUI) that x.x is what people would call the 'correct' pinning for Python and that x.x.x is overkill. CPython was careless with the 3.5.3 (bytecode magic number bump) and 3.6.1 (ABI change) releases and suddenly bad hacks were needed to allow x.x pinning to continue for both 3.5.x and 3.6.x, hacks that, had the situation been just a little bit different, would not have been possible and instead mass pulling of packages and rebuilding with more exact pinning would have been necessary (and try to imagine the amount of work and level of panic that would have caused). So despite what you've read/heard and the checks you've done into R pinning (which I note did not involved looking at the actual exports of the core R shared libraries on each of the 3 systems and the exact function prototypes in the headers between a range of minor versions) it is far safer to pin to x.y.z.
The overall concern for me is that we're trying hard to gain as much compatibility (binary and pinning) across the entire *conda ecosystem so that people can mix and match channels more freely and in an arbitrary order of preference. To do that we can't really all do our own thing in these regards.
I hope things like CI time and storage have solutions that do not involve large risks and continued incompatibility.

Thanks @mingwandroid. That makes sense. If it is a general trend to have stricter pinning, I totally agree that we should not go into the opposite direction with R. If conda-forge is fine, I would then close this PR at the beginning of next week. @jakirkham @ocefpaf @sodre, maybe we can decide instead on a common update strategy in conda-forge and bioconda? x.x.1 or x.x.2?

maybe we can decide instead on a common update strategy in conda-forge and bioconda? x.x.1 or x.x.2?

I understand all the points made by @mingwandroid and I do believe they are all very sound for high stability and maintainability, specially for defaults. However, I also understand your reasons for trying the approach in this PR.
Maybe we need a live meeting to discuss this? Or maybe we can just play safe and follow @mingwandroid advices.

I just caught up with the entire thread and I understand the reason for the PR. As long as the cost is only compute and storage, I rather play it safe and pin at the revision level. The ideal scenario is to rebuild at every revision, but we need to start somewhere. Here is an idea to consider:

Pin every revision greater than or equal to 2. I eye-balled R NEWS, that looks like a happy medium for the 3.x.x series. This tries to address the msys2-epoch issue raised by @mingwandroid.
Pin a .0 or .1 revision that is not superseded after Z days/months. This guarantees we don't wait indefinitely for a .2 revision.  And gives R users a no later date when they can expect a .0 or .1 release to be available across conda-forge, bioconda.

This has been a really valuable thread. Just to say, I appreciate the work you put into the PR @johanneskoester. Given the healthy conversation I'd like to get to the point where conda-build-all isn't the tool that decides on the pinning scheme - I think that is here/coming in conda-build, but honestly I'm a little out-of-the-loop on that.
Given the risk vs cost + the fact that this should eventually move down to conda-build, I'm going to close this as a wontfix.
For the record, I don't really have a strong feeling towards the actual trade-offs - I just feel like it isn't the place of conda-build-all to force any particular scheme.

Thanks, I agree with what you say.


RHIDP-4567-1 - Provide the ability to load a custom Backstage theme from a dynamic plugin

RHIDP-4567-1 - Provide the ability to load a custom Backstage theme from a dynamic plugin
(Replaces PR #781)

IMPORTANT: Do Not Merge - To be merged by Docs Team Only
Version(s):
1.4
Add the relevant labels to the Pull Request.
Issue:
RHIDP-4567

Approvals in PR #781

/cherrypick release-1.4


Issue with gullistick and Gamma fix

I noticed that the gamma fix and Gullistick don’t work well together. Sometimes the Y-Axis doesn’t center itself, and I have to move the sticks a bit for them to center again. I tried the three fixes from Gamma, but I had to revert to the stock configuration to get them to center correctly again. However, that brings back the snapping issue. Has anyone else experienced this?

Am in the same boat, the sticks start off fine, but have issues returning to centre


Allow for blank lines within filter blocks

Attempts to resolve #3.
I'm OK with Regular Expressions but not familiar with Oniguruma syntax or with Textmate-style syntax files, so I have no idea if what I'm doing here is correct.
My first stab just changed the matched line to ^\1[^\s]+$, to say it indents as far as the beginning of the filter block (^(\s*):ruby e.g.) but no farther and has something other than spaces on it – that is, it's not just a blank line. But what was happening was that the %div in the example at #3 was ending up white, because it matched that line.
So instead I'm using a lookahead, which just matches the start of a new line but not any of the text on that line.
It does the trick for my use case, but I honestly have no idea if it's a reliable overall solution. Definitely open to suggestions etc.

Not quite there. This looks for a line that is indented as deep as the filter block head (:ruby) but not shallower:

@karuna I'm not sure this should've been merged? Haven't had a chance to investigate but the comment above seems to indicate this isn't a full solution

@bensaufley I checked the result and it's working, might be broken in other places. But now that it's been published, can you make a new pull request if you have the chance to update it? Thanks.


Incorrect stack trace shown

Thanks for making this package.
Currently, when I raise an issue from within my task, the stack trace only shows this package's code:

This is with the following task:
def adder(x, y):
    if x > 50:
        raise(ValueError("a is greater than 50"))

I would expect the stack trace to involve my own code

Hi @jesseduffield,
You can set "backtrace" field to the number of lines you want to keep (default is 5).
Example:
job_1 = Job(jobtype='foo', backtrace=10)

That's interesting: whether I set that value or not, I'm still only seeing the same three lines as in the above screenshot.
It seems like there's two issues at play:

the original exception is excluded
newlines in the stack trace aren't being split on

debug log output:
 06:25:12 - FaktoryConsumer - WARNING - Task (job 5094c6d188b14bb19d04a4a953ded426) raised <class 'ValueError'>: a is greater than 70
 06:25:12 - FaktoryClient - DEBUG - C: FETCH default

 --- Logging error ---
 pebble.common.RemoteTraceback: Traceback (most recent call last):
   File "/app/venv/lib/python3.11/site-packages/pebble/common.py", line 175, in process_execute
     return Result(SUCCESS, function(*args, **kwargs))
                            ^^^^^^^^^^^^^^^^^^^^^^^^^
   File "/app/core/tasks/adder.py", line 11, in adder
     raise(ValueError("a is greater than 70"))
 ValueError: a is greater than 70

 The above exception was the direct cause of the following exception:

 Traceback (most recent call last):
   File "/app/venv/lib/python3.11/site-packages/pyfaktory/consumer.py", line 139, in task_done
     result = future.result()
              ^^^^^^^^^^^^^^^
   File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 449, in result
     return self.__get_result()
            ^^^^^^^^^^^^^^^^^^^
   File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
     raise self._exception
 ValueError: a is greater than 70

 During handling of the above exception, another exception occurred:

 Traceback (most recent call last):
   File "/usr/local/lib/python3.11/logging/__init__.py", line 1110, in emit
     msg = self.format(record)
           ^^^^^^^^^^^^^^^^^^^
   File "/usr/local/lib/python3.11/logging/__init__.py", line 953, in format
     return fmt.format(record)
            ^^^^^^^^^^^^^^^^^^
   File "/app/venv/lib/python3.11/site-packages/coloredlogs/__init__.py", line 1140, in format
     return logging.Formatter.format(self, record)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   File "/usr/local/lib/python3.11/logging/__init__.py", line 687, in format
     record.message = record.getMessage()
                      ^^^^^^^^^^^^^^^^^^^
   File "/usr/local/lib/python3.11/logging/__init__.py", line 377, in getMessage
     msg = msg % self.args
           ~~~~^~~~~~~~~~~
 TypeError: not all arguments converted during string formatting
 Call stack:
   File "/usr/local/lib/python3.11/threading.py", line 995, in _bootstrap
     self._bootstrap_inner()
   File "/usr/local/lib/python3.11/threading.py", line 1038, in _bootstrap_inner
     self.run()
   File "/usr/local/lib/python3.11/threading.py", line 975, in run
     self._target(*self._args, **self._kwargs)
   File "/app/venv/lib/python3.11/site-packages/pebble/pool/process.py", line 192, in message_manager_loop
     pool_manager.process_next_message(SLEEP_UNIT)
   File "/app/venv/lib/python3.11/site-packages/pebble/pool/process.py", line 229, in process_next_message
     self.task_manager.task_done(message.task, message.result)
   File "/app/venv/lib/python3.11/site-packages/pebble/pool/process.py", line 310, in task_done
     task.future.set_exception(result.value)
   File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 565, in set_exception
     self._invoke_callbacks()
   File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 340, in _invoke_callbacks
     callback(self)
   File "/app/venv/lib/python3.11/site-packages/pyfaktory/consumer.py", line 152, in task_done
     self.logger.debug(
 Message: 'Task (job 5094c6d188b14bb19d04a4a953ded426) backtrace: '
 Arguments: (['  File "/app/venv/lib/python3.11/site-packages/pyfaktory/consumer.py", line 139, in task_done\n    result = future.result()\n             ^^^^^^^^^^^^^^^\n', '  File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 449, in result\n    return self.__get_result()\n

In the above output, the part that really matters to me is:
   File "/app/core/tasks/adder.py", line 11, in adder
     raise(ValueError("a is greater than 70"))
 ValueError: a is greater than 70

But that does not appear.
Looks like one of the issues is:
            self.logger.debug(
                f"Task (job {future.job_id}) backtrace: ",
                traceback.format_tb(err_traceback),
            )

which should be:
            self.logger.debug(
                f"Task (job {future.job_id}) backtrace: {traceback.format_tb(err_traceback)}",
            )

This is the cause of the exception 'TypeError: not all arguments converted during string formatting'.
As for the newlines, that appears to be the default behaviour of the traceback module, but given how sidekiq renders newlines (i.e. just converts them to spaces), I would split the array based on those newlines (which would mean that the default backtrace of 5 would indeed lead to 5 lines being shown in sidekiq, as opposed to the current 3)

Thanks @jesseduffield for the description of the problem. I can see now. Would you be willing to investigate and make a PR to fix it?


Move binaries from resources to external repo (or other place)

Author the user story for this feature
As a Developer, I want to move all binaries from main/resources and test/resources to external repo or other place (based on the research of task #849) so I can have a flank repository smaller.
Is your feature request related to a problem? Please describe.
External binaries should not be part of source code, to have quicker checkout and to keep repository size as small as possible.
Describe the solution you'd like
The solution should be chosen based on the research of task #849

The integration task might influence this: https://github.com/Flank/flank/issues/1072

Since binaries directory size is around 18,2 MB and the content inside is not changing so often, we could use GitHub lfs which has suitable storage for us.
What is more, these files are only needed for Linux and windows.
WDYT?

Based on slack discussion

bandwidth for lfs on github is expensive
outages are caused when the bandwidth is is exceeded
there's flakiness associated with lfs which make the builds unstable
the recommended approach is to publish these as a package (either to github packages) or a content store like GCS or github releases (which is backed by S3)