CRA: Tests fail when Jest configuration is provided

Hi,
I was wondering if it's possible to use this adapter with a typescript/react project. Here are some random facts related:

tests are found in test.tsxfiles
it's a create-react-app based app (has implications on how accessible certain settings are)
jest is found in node_modules/.bin/jest
no other jest plugins are installed/active

I just tried whether it'd work out of the box. But there I only get "Error while loading test - click to show" with the output:
'Usage: test.js [--config=] [TestPathPattern]\n'  (including the a list of options).
Sorry, I'm new to VSCode plugin configurations ... so maybe it's all there and I just don't see it.

Create react app is supported, but may be broken at the moment. I just merged a fix for basic repos.  I can have a look at CRA apps to see if there is a problem with them too.

A brand new TS CRA app created using:
yarn create react-app cra-typescript --template typescript

Renders the following output:

So if you are not using the latest, or have modified anything, then let me know.  Note that you can access the extension logs by switching to the output window and selecting the output from Jest Test Explorer log.   You will need to turn on Jest Test Explorer > Logpanel setting to achieve this.

Thanks for your feedback and for testing it!
I'm running version 0.7.4 of the extension. (re-installed both the extension and the dependency). Sadly I ran into the same error. My thought was that maybe the testRegext pattern is missing in the jest configuration in package.json. But CRA does not allow for me to override this value (-> second part of the log)
Here's the log output:
[2020-10-01 08:00:52.349] [INFO] Loading Jest tests...
[2020-10-01 08:00:52.351] [INFO] Loading Jest settings from /Users/tobias/Documents/<path_to_project>/package.json...
[2020-10-01 08:00:54.266] [ERROR] Error loading tests 'Usage: test.js [--config=<pathToConfigFile>] [TestPathPattern]\n' +
  '\n' +
  'Options:\n' +
  '  --help, -h                    Show help                              [boolean]\n' +
  '  --version, -v                 Print the version and exit             [boolean]\n' +
  '  --all                         The opposite of `onlyChanged`. If `onlyChanged`\n' +
  '                                is set by default, running jest with `--all`\n' +
  '                                will force Jest to run all tests instead of\n' +
  '                                running only tests related to changed files.\n' +
  '                                                                       [boolean]\n' +
  '  --automock                    Automock all files by default.         [boolean]\n' +
  '  --bail, -b                    Exit the test suite immediately after `n` number\n' +
  '                                of failing tests.                      [boolean]\n' +
  '  --browser                     Respect the "browser" field in package.json when\n' +
  '                                resolving modules. Some packages export\n' +
  '                                different versions based on whether they are\n' +
  '                                operating in node.js or a browser.     [boolean]\n' +
  '  --cache                       Whether to use the transform cache. Disable the\n' +
  '                                cache using --no-cache.                [boolean]\n' +
  '  --cacheDirectory              The directory where Jest should store its cached\n' +
  '                                dependency information.                 [string]\n' +
  '  --changedFilesWithAncestor    Runs tests related to the current changes and\n' +
  '                                the changes made in the last commit. Behaves\n' +
  '                                similarly to `--onlyChanged`.          [boolean]\n' +
  '  --changedSince                Runs tests related to the changes since the\n' +
  '                                provided branch. If the current branch has\n' +
  '                                diverged from the given branch, then only\n' +
  '                                changes made locally will be tested. Behaves\n' +
  '                                similarly to `--onlyChanged`.           [string]\n' +
  '  --ci                          Whether to run Jest in continuous integration\n' +
  '                                (CI) mode. This option is on by default in most\n' +
  '                                popular CI environments. It will  prevent\n' +
  '                                snapshots from being written unless explicitly\n' +
  '                                requested.             [boolean] [default: true]\n' +
  '  --clearCache                  Clears the configured Jest cache directory and\n' +
  '                                then exits. Default directory can be found by\n' +
  '                                calling jest --showConfig              [boolean]\n' +
  '  --clearMocks                  Automatically clear mock calls and instances\n' +
  '                                between every test. Equivalent to calling\n' +
  '                                jest.clearAllMocks() between each test.[boolean]\n' +
  '  --collectCoverage             Alias for --coverage.                  [boolean]\n' +
  '  --collectCoverageFrom         A glob pattern relative to <rootDir> matching\n' +
  '                                the files that coverage info needs to be\n' +
  '                                collected from.                         [string]\n' +
  '  --collectCoverageOnlyFrom     Explicit list of paths coverage will be\n' +
  '                                restricted to.                           [array]\n' +
  '  --color                       Forces test results output color highlighting\n' +
  '                                (even if stdout is not a TTY). Set to false if\n' +
  '                                you would like to have no colors.      [boolean]\n' +
  '  --colors                      Alias for `--color`.                   [boolean]\n' +
  '  --config, -c                  The path to a jest config file specifying how to\n' +
  '                                find and execute tests. If no rootDir is set in\n' +
  '                                the config, the directory containing the config\n' +
  '                                file is assumed to be the rootDir for the\n' +
  '                                project.This can also be a JSON encoded value\n' +
  '                                which Jest will use as configuration.   [string]\n' +
  '  --coverage                    Indicates that test coverage information should\n' +
  '                                be collected and reported in the output.\n' +
  '                                                                       [boolean]\n' +
  '  --coverageDirectory           The directory where Jest should output its\n' +
  '                                coverage files.                         [string]\n' +
  '  --coveragePathIgnorePatterns  An array of regexp pattern strings that are\n' +
  '                                matched against all file paths before executing\n' +
  '                                the test. If the file pathmatches any of the\n' +
  '                                patterns, coverage information will be skipped.\n' +
  '                                                                         [array]\n' +
  '  --coverageReporters           A list of reporter names that Jest uses when\n' +
  '                                writing coverage reports. Any istanbul reporter\n' +
  '                                can be used.                             [array]\n' +
  '  --coverageThreshold           A JSON string with which will be used to\n' +
  '                                configure minimum threshold enforcement for\n' +
  '                                coverage results                        [string]\n' +
  '  --debug                       Print debugging info about your jest config.\n' +
  '                                                                       [boolean]\n' +
  '  --detectLeaks                 **EXPERIMENTAL**: Detect memory leaks in tests.\n' +
  '                                After executing a test, it will try to garbage\n' +
  '                                collect the global object used, and fail if it\n' +
  '                                was leaked            [boolean] [default: false]\n' +
  '  --detectOpenHandles           Print out remaining open handles preventing Jest\n' +
  '                                from exiting at the end of a test run. Implies\n' +
  '                                `runInBand`.          [boolean] [default: false]\n' +
  '  --env                         The test environment used for all tests. This\n' +
  '                                can point to any file or node module. Examples:\n' +
  '                                `jsdom`, `node` or `path/to/my-environment.js`\n' +
  '                                                                        [string]\n' +
  '  --errorOnDeprecated           Make calling deprecated APIs throw helpful error\n' +
  '                                messages.             [boolean] [default: false]\n' +
  '  --expand, -e                  Use this flag to show full diffs instead of a\n' +
  '                                patch.                                 [boolean]\n' +
  '  --filter                      Path to a module exporting a filtering function.\n' +
  '                                This method receives a list of tests which can\n' +
  '                                be manipulated to exclude tests from running.\n' +
  '                                Especially useful when used in conjunction with\n' +
  '                                a testing infrastructure to filter known broken\n' +
  '                                tests.                                  [string]\n' +
  '  --findRelatedTests            Find related tests for a list of source files\n' +
  '                                that were passed in as arguments. Useful for\n' +
  '                                pre-commit hook integration to run the minimal\n' +
  '                                amount of tests necessary.             [boolean]\n' +
  '  --forceExit                   Force Jest to exit after all tests have\n' +
  '                                completed running. This is useful when resources\n' +
  '                                set up by test code cannot be adequately cleaned\n' +
  '                                up.                                    [boolean]\n' +
  '  --globalSetup                 The path to a module that runs before All Tests.\n' +
  '                                                                        [string]\n' +
  '  --globalTeardown              The path to a module that runs after '
[2020-10-01 08:00:54.267] [INFO] Finished loading Jest tests.
[2020-10-01 08:01:37.624] [INFO] Loading Jest tests...
[2020-10-01 08:01:37.627] [INFO] Loading Jest settings from /Users/tobias/Documents/<path_to_project>/package.json...
[2020-10-01 08:01:39.365] [ERROR] Error loading tests '\n' +
  'Out of the box, Create React App only supports overriding these Jest options:\n' +
  '\n' +
  '  • clearMocks\n' +
  '  • collectCoverageFrom\n' +
  '  • coveragePathIgnorePatterns\n' +
  '  • coverageReporters\n' +
  '  • coverageThreshold\n' +
  '  • displayName\n' +
  '  • extraGlobals\n' +
  '  • globalSetup\n' +
  '  • globalTeardown\n' +
  '  • moduleNameMapper\n' +
  '  • resetMocks\n' +
  '  • resetModules\n' +
  '  • restoreMocks\n' +
  '  • snapshotSerializers\n' +
  '  • transform\n' +
  '  • transformIgnorePatterns\n' +
  '  • watchPathIgnorePatterns.\n' +
  '\n' +
  'These options in your package.json Jest configuration are not currently supported by Create React App:\n' +
  '\n' +
  '  • testRegex\n' +
  '\n' +
  'If you wish to override other Jest options, you need to eject from the default setup. You can do so by running npm run eject but remember that this is a one-way operation. You may also file an issue with Create React App to discuss supporting more options out of the box.\n' +
  '\n'
[2020-10-01 08:01:39.365] [INFO] Finished loading Jest tests.

I asked a colleague (same project) to confirm the error. He's getting the very same output.

Update:
The difference to a vanilla CRA project is this entry in the package.json:
"jest": {
    "collectCoverageFrom": [
      "src/**/*.+(ts|tsx)",
      "!src/**/*.d.ts",
      "!src/index.tsx"
    ],
    "coverageThreshold": {
      "global": {
        "branches": 0,
        "functions": 0,
        "lines": 0,
        "statements": 0
      }
    },
    "coverageReporters": [
      "text"
    ]
  }

Without this override of the jest config it works.

Awesome, I think I have the repro now.  Seems that CRA apps that specify a Jest config (in package.json at least) fail.  I'll have to debug further to understand what is going on.

Having the same issue right now. Is there something I can check to help solve the issue?

same issue here.

I can confirm that having those supported overrides for CRA in package.json is the issue in reading the tests.

I have the same issue, out of the box I can run test but not debug and any config overrides crash the test explorer

It works when I put the configurations in jest: { } block in the package.json file, but won't work after I move the config into <rootDir>/jest.config.js


Reduce commits by bot on the master branch

Description
The Creative Commons bot makes a very large number of commits to the master branch to keep issues synchronised between GitHub organisation and the issues databag in the repository. This should be reduced as it quickly leads to forks and clones going out of date. Every other copy of the repository goes stale almost every hour.
Reproduction

Fork the repository and wait for a couple of hours.
Your repository is now several commits behind creativecommons:master.

Expectation
The commits should either be infrequent, or atomically small, or on a different branch. One of the simplest ways to do that is to eliminate the updatedAt field from the data, as even changes to that field (which happen very frequently) lead to new commits. A more long term approach would be to setup serverless functions on Netlify/Vercel to proxy authenticated requests to GitHub on behalf of the client as in this example.
Resolution

[ ] I would be interested in resolving this bug.

Is this issue ready for work? If so, I can take it on.
Will a naive solution work for now until we can get the plan for the more long term approach solidified?

I think yes, this is a very pressing issue. Please just make sure we're not using the updatedAt value anywhere before we get rid of it.

Okie dokie, I can do this in the next couple days.


Sessions Details : Display a button to jump to the logs with Charger and Date prefiled

It has a dependency with issue #367

Dups


Support for react 19

Impressive library :)
I am using react 19 rc. npm install gives me below error
npm error Conflicting peer dependency<EMAIL_ADDRESS>npm error node_modules/react
npm error   peer<EMAIL_ADDRESS>|| ^17.0.0 || ^18.0.0" from<EMAIL_ADDRESS>npm error   node_modules/react-json-view-lite
npm error<EMAIL_ADDRESS>from the root project

When I force install it, I get some runtime issues. I am guessing there is some api difference.

Hi,
I haven't tried React 19 yet :) I have plans to remove React 16 and 17 from supported versions and add 19 in the 2.x.x version of the library. I consider to have major version upgrade after I merge this PR: https://github.com/AnyRoad/react-json-view-lite/pull/32 , probably this or next week.


Nebra RESET pins and SPI

Wondering what the reset pins for the SPI interfaces in the Nebra Outdoor model?
I see for the 1st it is 38, but using util_chip_id I cant seem to find the other.
Thanks!

Reset pins are 38 and 39 .. duh


Unable to Compile Headless Chromium r609904

Since version 1.11.0, I no longer seem to be able to compile the bundled headless Chromium in AWS.
Would be great to get some guidance on what changed, if anything.
Steps to Reproduce
Tell us about your environment:

Puppeteer: 1.11.0 (Chromium 72.0.3618.0 - r609904)
Platform / OS: Amazon Linux AMI 2<IP_ADDRESS>70930 x86_64 HVM GP2

What steps will reproduce the problem?
When using the Ansible playbook available here, that replicates the official Linux build instructions, and compiles headless Chromium statically (for compatibility with AWS Lambda), the Compiling Headless Chromium step fails (but only for r609904, previous revisions compile just fine).
For reference, the following GN arguments are set:
    - name: Creating Headless Chromium Configuration
      copy:
        content: |
          import("//build/args/headless.gn")
          enable_nacl = false
          is_component_build = false
          is_debug = false
          remove_webcore_debug_symbols = true
          symbol_level = 0
        dest: /srv/source/chromium/src/out/Headless/args.gn

And then ninja is called like so:
    - name: Compiling Headless Chromium
      shell: |
        autoninja -C out/Headless headless_shell
      args:
        chdir: /srv/source/chromium/src

What is the expected result?
Compilation should succeed.
What happens instead?
Compilation fails with the following error:
# autoninja -C out/Headless headless_shell
/srv/source/depot_tools/ninja -C out/Headless headless_shell
ninja: Entering directory `out/Headless'
[95/22176] ACTION //mojo/public/js:bindings_lite(//build/toolchain/linux:clang_x64)
FAILED: gen/mojo/public/js/mojo_bindings_lite.js
python ../../third_party/closure_compiler/js_binary.py --compiler ../../third_party/closure_compiler/compiler/compiler.jar --output gen/mojo/public/js/mojo_bindings_lite.js --deps gen/mojo/public/js/bindings_lite_sources.js_library gen/mojo/public/js/control_message_bindings_lite.js_library --sources ../../mojo/public/js/bindings_lite.js --flags compilation_level=ADVANCED_OPTIMIZATIONS generate_exports --externs ../../third_party/closure_compiler/externs/chrome.js ../../third_party/closure_compiler/externs/polymer-1.0.js ../../third_party/closure_compiler/externs/mojo_core.js ../../third_party/closure_compiler/externs/pending.js
Exception in thread "main" java.lang.UnsupportedClassVersionError: com/google/javascript/jscomp/CommandLineRunner : Unsupported major.minor version 52.0
	at java.lang.ClassLoader.defineClass1(Native Method)
	at java.lang.ClassLoader.defineClass(ClassLoader.java:808)
	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
	at java.net.URLClassLoader.defineClass(URLClassLoader.java:442)
	at java.net.URLClassLoader.access$100(URLClassLoader.java:64)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:354)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:348)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:347)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:430)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:323)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:363)
	at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:482)

[132/22176] CXX obj/services/audio/public/mojom/constants/constants.mojom.o
ninja: build stopped: subcommand failed.

I noticed that r609904 points to commit 99a174b7b1d13e53dbb2cd3aab84ba589c862d86.
Assuming the file on chrome/VERSION is correct, it should be version 72.0.3617.0 and not 72.0.3618.0?

Solved.

For anyone finding this later, the original issue described above ("... Unsupported major.minor version 52.0")  is because the Amazon Linux AMI defaults to Java 1.7, and the Chromium build tools pick that up as a part of the build process.  They'll even install Java 1.7 as a part of the process if it isn't there.  But the build won't work with 1.7.
You can change the default by using update-alternatives:
$ sudo update-alternatives --config java

There are 3 programs which provide 'java'.

  Selection    Command
-----------------------------------------------
   1           /usr/lib/jvm/jre-1.6.0-openjdk.x86_64/bin/java
   2           /usr/lib/jvm/jre-1.8.0-openjdk.x86_64/bin/java
*+ 3           /usr/lib/jvm/jre-1.7.0-openjdk.x86_64/bin/java

Enter to keep the current selection[+], or type selection number: 2
$


Make Github pipeline more robust by increasing HTTP ttl and adding a retry

Make Github pipeline more robust by increasing HTTP ttl and adding a retry.
Signed-off-by: Santiago Pericasgeertsen<EMAIL_ADDRESS>

For what is worth, there were no retries needed for this PR


Where to add a Google API key for geocoder?

Hi, have used Bootleaf for years and this is a really great step forward on something that was already good!
Hopefully a simple request, but I can't seem to find an answer for myself!  Google's Geocoder now needs an API key so I was wondering where I need to add this?  I suspect I could hard code it into Control.Geocoder.js but I don't really want to do this!
I did try
	"leafletGeocoder": {
		//https://github.com/perliedman/leaflet-control-geocoder
		"key": "AIza..........................................UGiCk",
		"collapsed": false,
		"position": "bottomleft",
		"placeholder": "Search for a location",
		"type": "Google", // OpenStreetMap, Google, ArcGIS
		//"suffix": "Australia" // optional keyword to append to every search
	},

but that doesn't seem to pass the key in the request.
Thanks

Hi James,
I'm really glad that you find it useful!
I had to make some changes to the leaflet geocoder in order to get it to append the Google Maps API key. I also added this option to the Bootleaf config.js file. If you fetch the latest version of this repo you should see the changes:
"leafletGeocoder": {
  "type": "Google",
  "key": "AIzaS....sbW_E", // when using the Google geocoder, include your Google Maps API key
},

  // SL. This section wasn't working in the original code so I modified it to suit the Google Maps key setting in the Bootleaf config file
  if (this._key && this._key.key) {
    params.key = this._key.key;
}

Works a treat, thanks for such a prompt response


Doesn't see anything to be built

nixpkgs-review v2.7.0:
$ nixpkgs-review pr 180271
$ git -c fetch.prune=false fetch --no-tags --force https://github.com/NixOS/nixpkgs master:refs/nixpkgs-review/0 pull/180271/head:refs/nixpkgs-review/1
remote: Enumerating objects: 9, done.
remote: Counting objects: 100% (9/9), done.
remote: Compressing objects: 100% (8/8), done.
remote: Total 9 (delta 0), reused 4 (delta 0), pack-reused 0
Unpacking objects: 100% (9/9), 228.22 KiB | 2.30 MiB/s, done.
From https://github.com/NixOS/nixpkgs
 + 273dce6c32f...d31f0743f07 refs/pull/180271/head -> refs/nixpkgs-review/1  (forced update)
$ git worktree add /Users/winter/.cache/nixpkgs-review/pr-180271/nixpkgs 23ee5ac46ad942208b82f3631d891cdb60bb8409
Preparing worktree (detached HEAD 23ee5ac46ad)
Updating files: 100% (30982/30982), done.
HEAD is now at 23ee5ac46ad vscode-extensions.piousdeer.adwaita-theme: init at 1.0.7
$ git merge --no-commit --no-ff d31f0743f07a884db41e29db738f33350b2f1d98
Automatic merge went well; stopped before committing as requested
Nothing to be built.

Link to currently reviewing PR:
https://github.com/NixOS/nixpkgs/pull/180271

$ /nix/store/9s055bnn4l671f7vlsyaifsbr5z64jij-nix-2.9.1/bin/nix-shell /Users/winter/.cache/nixpkgs-review/pr-180271/shell.nix

This is on a PR which adds a new package (NixOS/nixpkgs#180271), and I've also seen it happen on a PR that updates an existing package (NixOS/nixpkgs#180227).

Looks like it's OfBorg support that's causing it not to build.


localCache entries are never getting cleared from memory in pLocal mode

OrientDB version: 2.0.5
We are running OrientDB in pLocal mode as part of our webapp in tomcat.
We use timkerpop api's to access and traverse the graph. As we do that we see entries getting into the local cache. We see that these entries are never getting cleared even though there is no external references to the records apart from OrientGraph.
What would be the reason for these not getting cleared even after GC runs?
In the documentation, we see two configurations - cache.level1.size and cache.level2.size that can limit the cache entries. There is nothing in OGlobalConfiguration to set these values .How do we set these configurations?

How do you know they are not cleared? Have you tried 2.1.2 ?

No we are on 2.0.5 and cannot upgrade right now.

Sorry but still how do you know that they are not cleared ?

Our performance team ran some tests and collected metrics using Eclipse MAT. The results were collected after 2 hours and 12 hours. We noticed that the "graph" object was holding on to a lot of object references.
In case you are interested I can send you a private mail with the results.

Yes,
If you do not mind, but I think the most important is that, according to my
opinion, you use graph object in wrong way.
Seems you open it and keep open till application shutdown.
But you should use graph factory and pool instead.
You may consider graph object as a connection, you open it at the start of
the method and close at the end (by calling of shutdown method).
Acquiring of graph object instance using graph factory and pool is very
cheap.
Is my assumption about usage of graph object in your project correct ?
On Thu, Sep 24, 2015 at 11:27 AM Shaji Gopal<EMAIL_ADDRESS>wrote:

Our performance team ran some tests and collected metrics using Eclipse
MAT. The results were collected after 2 hours and 12 hours. We noticed that
the "graph" object was holding on to a lot of object references.
In case you are interested I can send you a private mail with the results.
—
Reply to this email directly or view it on GitHub
https://github.com/orientechnologies/orientdb/issues/4781#issuecomment-142855555
.

Luca,
You are right we create the graph by calling  new OrientGraph() and we shut it down only during application shutdown. We will try using the graphFactory instead.

Hi ,
I am Andrey ,
Any way I am looking forward for your feedback :-)
On Tue, Sep 29, 2015, 10:58 Shaji Gopal<EMAIL_ADDRESS>wrote:

Luca,
You are right we create the graph by calling new OrientGraph() and we shut
it down only during application shutdown. We will try using the
graphFactory instead.
—
Reply to this email directly or view it on GitHub
https://github.com/orientechnologies/orientdb/issues/4781#issuecomment-143977594
.

Do not forget to setup pool in graph factory . It is very important .
On Tue, Sep 29, 2015, 18:21 Andrey Lomakin<EMAIL_ADDRESS>wrote:

Hi ,
I am Andrey ,
Any way I am looking forward for your feedback :-)
On Tue, Sep 29, 2015, 10:58 Shaji Gopal<EMAIL_ADDRESS>wrote:

Luca,
You are right we create the graph by calling new OrientGraph() and we
shut it down only during application shutdown. We will try using the
graphFactory instead.
—
Reply to this email directly or view it on GitHub
https://github.com/orientechnologies/orientdb/issues/4781#issuecomment-143977594
.

The issue has been closed because no response has been received after a while. In case the issue is still valid, please reopen it or write a comment that explains why it's still valid and a way to reproduce it. Thank you.


Support multiple camera streaming, not limit to 3

Fixed in v5.15.5, see https://github.com/ossrs/oryx/commit/c25f81b8e4e591252e80a179a24b0d5fe179d19d


Actions DOR has taken and consequences Screen is Confusing

Screen id dbd-01a review and allow to edit actions and hardships comments is vague about what the user should be editing the paragraph for. It seems like it's getting sent to a form in the package, but who's reading it is strange. Might also want to just show the editable portion, so there's not a block of text.

This is never used anywhere in the assembled document, so I'm just gonna not include it in the interview


MINIFICPP-1507 convert InputStream::read to size_t

Converting write will be done in a followup PR to keep the PR size managable. Errors are now reported as static_cast<size_t>(-1).

Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
For all changes:

[x] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?

[x] Does your PR title start with MINIFICPP-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.

[x] Has your PR been rebased against the latest commit within the target branch (typically main)?

[x] Is your initial contribution a single, squashed commit?

For code changes:

[x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[x] If applicable, have you updated the LICENSE file?
[x] If applicable, have you updated the NOTICE file?

For documentation related changes:

[x] Have you ensured that format looks appropriate for the output in which it is rendered?

Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible.

Manually cancelled github actions CI runs triggered by the push event after every job was successful from the pull_request event.


Feature Request: Add ReactJS Support

Description:
I would like to request the addition of ReactJS support to the project generated by StartEase. React is a popular JavaScript library for building user interfaces, and it would be great to have it as an option when scaffolding projects.

I'll like to work on this with your permission.

i've assigned this to you . this is a top priority issue so please try get it done in time .


Debug try.jupyter.org

It's broken in IHaskell.

I returned to https://try.jupyter.org/ after some months and it stay broken. What happens?

It's still broken.
Sadly I have not yet gotten a chance to fix it.
(If anyone wants to jump in and figure out why, they are welcome to do so! You can find the Dockerfile in the tmpnb repository and play with it.)

Thanks I will take a look at it

How I can add the IHaskell kernel to it?

The Dockerfile used for tmpnb is here: https://github.com/jupyter/docker-demo-images/blob/master/Dockerfile
If you were to fix the Dockerfile it would work
IHaskell is here
It's an old version that uses cabal instead of Stack for installation, but I'm not sure why it doesn't work. a first step would be to at least get this Dockerfile set up and fix the installation in the Dockerfile to a recent IHaskell using stack to install it

I'll try and see if I can fix this.


SPARK-18093][SQL] Fix default value test in SQLConfSuite to work rega…

…rdless of warehouse dir's existence
What changes were proposed in this pull request?
Appending a trailing slash, if there already isn't one for the
sake comparison of the two paths. It doesn't take away from
the essence of the check, but removes any potential mismatch
due to lack of trailing slash.
How was this patch tested?
Ran unit tests and they passed.

Test build #67514 has started for PR 15623 at commit dea1f33.

Test build #67514 has finished for PR 15623 at commit dea1f33.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67514/
Test PASSed.

Thanks @srowen, indeed, it could be stripping the trailing slash too. I am ok either way too. Let me know if you have strong preference to the other one, otherwise, this should be ok.
Thanks again!

Test build #67535 has started for PR 15623 at commit 707f1ee.

LGTM will merge once tests pass.

Test build #67535 has finished for PR 15623 at commit 707f1ee.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67535/
Test PASSed.

Merging to master / 2.0.


Login codes can be claimed more than once

Simple fix, add is_claimed attribute while creating the code.
On verification if the code is correct,  set that value to true. Do a PUT with the user_id and the timestamp instead of just marking the latest code as claimed. Also add a attribute_not_exists flag on create incase a code is requested in the same millisecond.. somehow.

Fixed #50


Core: BlobReadChannel limit parameter doesn't behave as documented in ReadChannel

Environment details

OS type and version: Ubuntu 20.04
Java version: 17
version(s): 2.9.3

Any additional information below
The limit parameter in ReadChannel is documented as the following:

Limit the maximum number of bytes available to be read from this channel. If the limit is larger than the actual size of the content this will have no material impact.
NOTE:Implementers are not required to return a new instance from this method, however they are allowed to. Users of this method should always use the instance returned from this method.
Default Implementation:By default, this method will simply return this.
Params:
limit – the maximum number of bytes to limit this channel to
Returns:
The instance of channel which will respect the limit.

However, when trying to use the BlobReadChannel to read a range of bytes from object storage, limit is used as end-exclusive:
final int toRead =           Math.toIntExact(Math.min(limit - position, Math.max(byteBuffer.remaining(), chunkSize)));
If you look in documentation it is also stated explicitly:
https://cloud.google.com/storage/docs/samples/storage-download-byte-range
Is this unintended behaviour or a documentation issue?
Thanks!

When performing a range get of an object, the Range header actually specifies the offsets inclusive. So in this case, the intent is that we're limiting a length on number of bytes rather than an end-offset. We automatically translate from length to end-offset for the request, which gives the appearance of end-exclusive.
When running the below code sample, you can see in the further logs Range: bytes=5-34 which will yield a response containing 30 bytes.
  public static void main(String[] args) {
    Storage s = StorageOptions.newBuilder().build().getService();
    BlobInfo obj = BlobInfo.newBuilder(bucket, "obj/62").build();
    String base62 = Streams.concat(
        IntStream.rangeClosed('0', '9'),
        IntStream.rangeClosed('A', 'Z'),
        IntStream.rangeClosed('a', 'z')
    )
        .mapToObj(i -> (char) i)
        .map(Object::toString)
        .collect(Collectors.joining(""));
    Blob blob = s.create(obj, base62.getBytes(StandardCharsets.UTF_8), BlobTargetOption.doesNotExist());
    Long size = blob.getSize();
    System.out.println("size = " + size);

    try (ReadChannel r = s.reader(obj.getBlobId()).limit(35)) {
      r.seek(5);
      ByteStreams.copy(Channels.newInputStream(r), System.out);
    } catch (Exception e) {
      e.printStackTrace(System.err);
    }
  }

2022-07-18 13:28:12,782 INFO  [main] com.google.api.client.http.HttpTransport - -------------- REQUEST  --------------
GET https://storage.googleapis.com/download/storage/v1/b/my_bucket/o/obj%2F62?alt=media
Accept-Encoding: gzip
Authorization: <Not Logged>
Range: bytes=5-34
User-Agent: gcloud-java/ Google-API-Java-Client/1.34.1 Google-HTTP-Java-Client/1.41.8 (gzip)
x-goog-api-client: gl-java/1.8.0 gdcl/1.34.1 linux/5.17.11

2022-07-18 13:28:12,850 INFO  [main] com.google.api.client.http.HttpTransport - -------------- RESPONSE --------------
HTTP/1.1 206 Partial Content
Content-Range: bytes 5-34/62
Pragma: no-cache
X-Goog-Metageneration: 1
Date: Mon, 18 Jul 2022 17:28:12 GMT
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
ETag: CJPOgYr7gvkCEAE=
Content-Disposition: attachment
Vary: X-Origin
Vary: Origin
Expires: Mon, 01 Jan 1990 00:00:00 GMT

2022-07-18 13:28:12,851 INFO  [main] com.google.api.client.http.HttpTransport - Total: 30 bytes
2022-07-18 13:28:12,851 INFO  [main] com.google.api.client.http.HttpTransport - 56789ABCDEFGHIJKLMNOPQRSTUVWXY
56789ABCDEFGHIJKLMNOPQRSTUVWXY

@BenWhitehead First of all thank you for the response and explanation 😀
Second, doesn't this mean that BlobReadChannel is not doing what the ReadChannel interface says it should regarding the limit parameter?
Sounds like ReadChannel treats it as number of bytes to read

Do you feel like the following modification clarifies and matches better with the behavior?
-   * Limit the maximum number of bytes available to be read from this channel. If the limit is
-   * larger than the actual size of the content this will have no material impact.
+   * Limit the maximum number of bytes to be read from the objects content, counting from the
+   * beginning of the object, which will be available to read from this channel. If the limit
+   * is larger than the actual size of the content this will have no material impact.
+   *
+   * If used in conjunction with {@link #seek(long)} the total number of returned bytes from
+   * this channel will be reduced by the number of bytes specified to seek.

@BenWhitehead That's better I think


SDMMD_ServiceReceiveMessage always returns kAMDSendMessageError when called multiple times

Sample code provided here https://github.com/mountsi/sample1
After calling SDMMD_ServiceReceiveMessage multiple times (in my case either 147 times on an ipad 4 and 120 times on an iphone 5) the method always returns kAMDSendMessageError instead of kAMDSuccess. These counts are reproducible.

I was able to reproduce this behavior, checking the device's syslog at the same time I am noticing it print:
lockdownd [26] <Error>: Could not set socket option SO_OPPORTUNISTIC: Invalid argument on each request, and then kernel [0] <Notice>: proc: table is full after it starts returning kAMDErrorError from the device. This error code only returns when the socket times out, so I'm going to guess that the command you are issuing needs another message to close the session or otherwise terminate the service.

I agree.
The worse problem here is that in this state all memory on the iOS device is used and iOS on the device might crash anytime.
I did not find anything in the framework that looks like it's closing something that isn't closed with a SDMMD_AMDeviceDisconnect.
I'm not into the framework too deep but I assume a SDMMD_AMDeviceDisconnect should close any connection to the device?
I also noticed that closing the application will free up the used memory on the device instantly (close all existing connections), this confirms that with every call an connection is opened but not closed

Closing a connection with a device will not stop service connections. My suggestion would be to stop polling for devices and starting new services each time your timer fires and continue to use the same device object.

Saving the socket object returned by SDMMD_TranslateConnectionToSocket and reuse it every time needed I found out the most practical solution for me.
Does something speak against this?

no, you should be holding onto the connection object. SDMMD_TranslateConnectionToSocket exists just as a means of convenience to access the internal socket, you need to keep the connection object around to close it properly.

Hey, i saw you pop into the irc channel to ask about how to release the SDMMD_AMConnectionRef object, simply call CFRelease() on the object and it will close the existing connection. Since connection objects are single-use there isn't a point of keeping it around without a valid socket.

This should do the trick, thanks!
Keeping the connection object and using it later solves the issue.


[vite:vue] failed to resolve "extends":"./.nuxt/tsconfig.json"

Installing this module causes the following error with nuxt prepare:
 ERROR  [vite:vue] failed to resolve "extends":"./.nuxt/tsconfig.json" in C:\Users\ThatTonybo\Desktop\chat-app\web\tsconfig.json                                                                                14:18:58
file: C:/Users/ThatTonybo/Desktop/chat-app/web/node_modules/nuxt-icon/dist/runtime/Icon.vue

Does it still fail after running yarn dev?


💡 [FEATURE] - Build a single script in the home directory for development and production run

Start Date
No response
Implementation PR
No response
Reference Issues
No response
Summary
There is package.json file with prettier in the home directory which runs on npm run format and npm run format:fix
Similarly, create 2 files in the home directory called as dev.js and prod.js
and then add the command in package.json to run them with npm run dev and npm run build respectively
Those files respectively should build all the installation steps with node child process to run it and also log in the terminal of all the output. Also, add relevant error handling.
Make use of only one terminal window.
For eg: For the dev environment, once you add npm run dev
it should run those commands, and where you need to run both the backend and frontend.
For every backend command, add BACKEND: <the command> and use chalk package to color the BACKEND word.
for the output of the backend ( logs ), add BACKEND: <the terminal output> and similarly use chalk package with different color.
Similarly for the frontend as well, with 2 more chalk colors.
Basic Example

Drawbacks

Unresolved questions
No response

Multiple folks can work on this issue. Let me know if you want to be assigned to this issue!

Subject: Intrested in contributing to  Issue #[9]
Hello devarshishimpi ,
I am intrested in working on this issue and I think its greate idea to create a script which could start client and server at the same time, I could use it in my further projects too!


STREAM-20 Update scripts command to support ALTER TABLE ADD COLUMN

Implement script command
added template for ALTER TABLE ALTER COLUMN

Looking really good, very close to merging.

NIce work!


How can I upgrade orionjs:froala to froala v2?

So I'm in the process of migrating orionjs:froala to Froala V2. I've made some progress but I'm stuck in some parts. Inside the froala.js file
https://github.com/orionjs/orion/blob/master/packages/froala/froala.js
I changed the initialization of the editor from editable to froalaEditor.
https://gist.github.com/ecuanaso/f809adbe0c6a1a63903f
This change helped me show the Froala v2 in my orion backend panel->
http://postimg.org/image/qo1yob18h/
However when I try to write content and or upload images inside the froala editor, I receive an error 'Editor should be initialized before calling the html.get method.'

So my question, what else inside the orion:froala package do I have to change to make it work with froala v2?

When I get this up and running how do I configure the image_manager to receive images from my server? If you're not sure what the image_manager is here's a screenshot.

http://s25.postimg.org/enn1ncnpb/Screen_Shot_2015_10_05_at_8_56_18_AM.png
and here's the link to the tutorial on how to server side integration, however it's in php.
https://www.froala.com/wysiwyg-editor/v2.0/docs/server-integrations/php-image-manager
I'm trying to find the material way to connect the image manager with meteor.

I started a PR with the update to version 2:
https://github.com/orionjs/orion/pull/355

Hey, macrozone. I was able to upgrade to v2, there's still some bugs I have to iron out though. Right now, I'm working on integrating my server to to show images from the images I upload to the filesystem show on the image-manager.
The image-manager right now shows images from the froala server ->
http://s25.postimg.org/enn1ncnpb/Screen_Shot_2015_10_05_at_8_56_18_AM.png


Nested types not supported by @DependencyClient

Description
@DependencyClient
struct SomeFailingClient {
  var endpoint: () async throws -> InternalValue

  struct InternalValue {
    var value: Bool
  }
}

extension SomeFailingClient.InternalValue {
}

Checklist

[X] I have determined whether this bug is also reproducible in a vanilla SwiftUI project.
[ ] If possible, I've reproduced the issue using the main branch of this package.
[X] This issue hasn't been addressed in an existing GitHub issue or discussion.

Expected behavior
No response
Actual behavior
No response
Steps to reproduce
No response
Dependencies version information
1.1.0
Destination operating system
iOS 17
Xcode version information
No response
Swift Compiler version information
swift-driver version: 1.87.1 Apple Swift version 5.9 (swiftlang-<IP_ADDRESS>.108 clang-15<IP_ADDRESS>)
Target: arm64-apple-macosx14.0

Hi @malonehedges, this is a bug in the compiler unfortunately. We have it documented in TCA, but perhaps we need to document it in this repo too:
https://pointfreeco.github.io/swift-composable-architecture/main/documentation/composablearchitecture/reducer()#Circular-reference-errors
As this is not an issue with the library I am going to convert it to a discussion.


Add stats about duration and padding proportion

This PR includes average utterance duration (utt_duration) and padding proportion (utt_pad_proportion) in tensorboard logs.

What is the sample output?

What is the sample output?

2022-07-20 17:59:30,363 INFO [train.py:775] (0/2) Epoch 2, batch 1600, loss[loss=0.2137, simple_loss=0.3831, pruned_loss=0.2222, over 7195.00 frames; utt_duration=1600, utt_pad_proportion=0.007582, over 18.00 u
tterances.], tot_loss[loss=0.2397, simple_loss=0.4049, pruned_loss=1.027, over 1465428.24 frames; utt_duration=1233, utt_pad_proportion=0.06145, over 4760.53 utterances.], batch size: 18, lr: 2.70e-03
2022-07-20 17:59:30,364 INFO [train.py:775] (1/2) Epoch 2, batch 1600, loss[loss=0.2039, simple_loss=0.3653, pruned_loss=0.2125, over 7188.00 frames; utt_duration=1598, utt_pad_proportion=0.006629, over 18.00 u
tterances.], tot_loss[loss=0.2384, simple_loss=0.4028, pruned_loss=1.025, over 1465413.17 frames; utt_duration=1251, utt_pad_proportion=0.05533, over 4691.62 utterances.], batch size: 18, lr: 2.70e-03
2022-07-20 17:59:30,365 INFO [train.py:795] (1/2) Computing validation loss
2022-07-20 17:59:30,365 INFO [train.py:795] (0/2) Computing validation loss
2022-07-20 17:59:46,551 INFO [train.py:804] (1/2) Epoch 2, validation: loss=0.4018, simple_loss=0.3997, pruned_loss=0.2019, over 944034.00 frames; utt_duration=679.8, utt_pad_proportion=0.1322, over 5567.00 utt
erances.


[Test] Add ml singleshot example

Commits to be reviewed in this PR
[Test] Add ml singleshot example
This patch adds integrated test to show and test ml singleshot.
Self evaluation:

Build test: [X]Passed [ ]Failed [ ]Skipped
Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee<EMAIL_ADDRESS>

:octocat: cibot: @zhoonit, A builder checker could not be completed because one of the checkers is not completed. In order to find out a reason, please go to http://nnstreamer.mooo.com/nntrainer/ci/repo-workers/pr-checker/1701-202111102120530.85571098327637-3aec758444db220f1767e3237ec3d0ebcbd81ac6/.


Bug: vim-lsp fails to parse headers sent from volar when using @volar/vue-language-server with --stdio flag

For example, when the user don't set volar.config.js, volar v0.33.0 or later calls console.log for logging. But, when using volar with --stdio, texts output from console.log also included to messages sent to vim-lsp. Then, it breaks parsing function in vim-lsp and I cannnot use volar... After I replaced console.log to console.warn, which means the message goes to stderr, both volar and vim-lsp look work fine.
https://github.com/johnsoncodehk/volar/blob/3852ee85c8d7a8e20e0bca084d2b73cf11491f93/packages/vue-language-service/src/languageService.ts#L519
Also, following code could cause same problem.
https://github.com/johnsoncodehk/volar/blob/271cdc7c3ad9f93725b422ce7e4086775b09929b/packages/shared/src/ts.ts#L142

Thanks for the report! Don't know that logging had big impact. 😅

@johnsoncodehk If just replacing console.log to console.warn is good solution for this project, I'll make pull request for this. Does it look good to you? If no, I'll just wait for the fix.
Anyway, I'm very happy since now I could share the problem with you 😄  Thank you!

Of course, nice to have a PR for this. :)


DS refactoring offers - Stage 1

Removed IOfferRepository from controller
Added SaaSKitContext to controller to inti OfferService
OfferService now provides all Offers
OfferService has test contructor with IRepositories to facilitate mocks or stubs.
OfferController no longer references IOfferAttributes repo.

Adddresses but not close #375

I am closing this pull request to go back and ensure no changes to migration files.


chore(proc): Update repo's settings

Description
Support the Java 11 and Java 17 runtimes by requiring all changes to pass checks.

Self-approving.


seemingly no way to confirm registration for a site

Perhaps I'm missing something -- but it seems that only the passport section of the registration API's payload is exposed to terraform. There is no way to specify the token. How can one tell which registration it will confirm?

@mattsawyer77: After registration acceptance the id of the object will be the registration id it has accepted.

Token is used to determine for which tenant the volterra CE software will create a registration request
Once the registration request is created, user uses volterra_registration_approval resource to approve it in which the user gives cluster_name information which is nothing but the site name information and hostname of the specific node
If you have multiple nodes in a volterra site then there are multiple registration request which will be generated so user need to accept those registrations


Mechanism for reflecting tuples into GraphQL

Currently we don't support reflecting tuples into GraphQL because GraphQL does not support heterogeneous lists and because reflecting named tuples as an "Object" with fields presents a problem of partial fetching.
I have an idea of how tuples can be generically reflected, but this would make them read-only w.r.t. GraphQL. So it's not the best solution, but it would at least address those usages where read-only approach is acceptable.
The idea is to reflect all tuples as a GraphQL Object. For unnamed tuples the fields would be given some generic numbered name like field0, field1, etc. This gets around the issue of not supporting heterogeneous lists. However, the objects in GraphQL don't have to be fetched completely (unlike the underlying tuples, which are a single unit), so we need a mechanism for addressing that. The solution I propose is to expose the tuple property as <prop_name>_tuple or some similar mangled name. This means that the <prop_name>_tuple can be an ad-hoc computable of any type we like in the underlying EdgeQL query, so we can reshape the actual tuple in any way.
Consider this example:
type Foo {
    # some really fancy tuple with non-trivial structure
    property data ->
        tuple<int64, str, array<float64>, tuple<a: int64, b: float64>>
}

The GraphQL query would look something like this:
query {
  Foo(
    filter: {data_tuple: {field0: {eq: 42}}}
  ) {
    data_tuple {
      field0
      field2
      field3 {
        b
      }
    }
  }
}

And the resulting EdgeQL would be something like:
SELECT Foo {
    data_tuple := (
        field0 := .data.0,
        field2 := .data.2,
        field3 := (a := .data.3.a)
    )
} FILTER .data.0 = 42;

We wouldn't support any operations on the fields inside the tuple (such as filtering), so that we could still maintain the integrity of the tuple data. So each field is optional to fetch, but cannot be altered in any other way when fetching.
@1st1, @elprans what are your thoughts on this?

Actually, I realized we can reflect a tuple field insert by making the input object for it have all its fields and non-nullable, so that they must be specified. So then inserting any tuple structure becomes an all-or-nothing proposition in GraphQL.

This seems reasonable to me.  Not a high-priority for now, though.

However, the objects in GraphQL don't have to be fetched completely (unlike the underlying tuples, which are a single unit), so we need a mechanism for addressing that.

I'm not sure I agree this is a problem. Users can always fetch the elements of the tuple they want by saying { prop { _0 _1 _2 } }. If they want to fetch all of them they can always create a computable / view that would just cast the tuple to json.


Switch on enum of one member not considered exhaustive

TypeScript Version: 2.1.4
Code:
// --strictNullChecks

enum Enum1 {
    A,
    B
}

enum Enum2 {
    A
}

interface Foo {
    readonly kind: Enum1
}

interface Bar {
    readonly kind: Enum2
}

let f = (x: Foo): number => {
    switch (x.kind) {
        case Enum1.A:
            return 1;
        case Enum1.B:
            return 2;
    }
}

let g = (x: Bar): number => {
    switch (x.kind) {
        case Enum2.A:
            return 1;
    }
}

Expected behaviour (as far as I know):
Compiles without error
Actual behaviour:
Function g does not compile, with error TS2366: Function lacks ending return statement and return type does not include 'undefined'..
Without strict null checks, it compiles without error.

Is it possible to get information where error occurred? Right now there is no file, line information about that.
I just install fresh windows installation with VS2017RC and get this error. I would like to add default statement to switches (they already should be there, but apparently I miss one), but there are simply to many files to check it.

I have a related issue also with enums with one value, you can directly copy this to the microsoft typescript playground:
No error as expected for this snippet (2 values in enum)
enum Rating {
  Like = 1,
  Dislike
};

let a: Rating;

switch (a) {
  case Rating.Like:
    break;

  case Rating.Dislike:
    break;

  default:
    let s: never = a;
}

On the other hand, drop it down to one value in the enum and you get the error:

Type 'Rating' is not assignable to type 'never'.

enum Rating {
  Like = 1
};

let a: Rating;

switch (a) {
  case Rating.Like:
    break;

  default:
    let s: never = a; // error here
}

Examples in this issue seems to be working as expected in latest.


Add a reverse mapping from role to namespaces

Hi!
Is your feature request related to a problem? Please describe.
I'm working on KoP, and I'm currently implementing security on it. I'm currently struggling to list all topics according to a Pulsar role.
Part of the Kafka handshake is called the Metadata request:

[...] Kafka brokers can answer a metadata request that describes the current state of the cluster: what topics there are, which partitions those topics have, which broker is the leader for those partitions, and the host and port information for these brokers.

Most of the client implementation are expecting to retrieve all available topics by sending a MetadataRequest with an empty list of topics. In the context of authorization, I can't list all topics related to a Pulsar Role because I can't reverse from a role the associated tenants and namespaces.
Describe the solution you'd like
I would like to add a new internal function that, given a role, is answering the full list of associated namespaces.
I think this type of code should be exposed within the admin API but maybe there is another(and better!) location like the NamespaceService?

Thanks @PierreZ . This would also be useful for admin and other protocol implementations.

Thanks @jiazhai, where do you think I should put the implementation? I am hesitating between:

PulsarAuthorizationProvider
NamespacesBase
NamespaceService

I think that most AuthZ implementation won't be able to fulfill that info though

I digged through the code, roles are stored within ZK as a serialized version of the policies class. Which means that it is  PulsarAuthorizationProvider which is responsible for this. I'm wondering if we should add a new method to the interface to be sure that all AuthorizationProvider can reverse a role to their associated namespaces. What do you think?

roles are stored within ZK as a serialized version of the policies class.

That's true for the default implementation, though the AuthZ backend can be (and is in real-life) also pluggable with different implementations.

'm wondering if we should add a new method to the interface to be sure that all AuthorizationProvider can reverse a role to their associated namespaces. What do you think?

That would be requiring to do a scan of the metadata to search for namespaces in which a role has access to. Or maintaining a different mapping, with the trouble of keeping that in sink with the source of truth.

That would be requiring to do a scan of the metadata to search for namespaces in which a role has access to. Or maintaining a different mapping, with the trouble of keeping that in sink with the source of truth.

True, I was thinking about implementing your first proposal. Something that is looping through znodes, and check if auth_policies.namespace_auth.containsKey(role) for the default implementation.

Or maintaining a different mapping, with the trouble of keeping that in sink with the source of truth.

I agree with you, not sure it's worth it for now and for a first implementation.

That's true for the default implementation, though the AuthZ backend can be (and is in real-life) also pluggable with different implementations.

That means that I don't have many options regarding the location of where that piece of code should be: it must be hold by the AuthorizationProvider.

I think that most AuthZ implementation won't be able to fulfill that info though

I must admit that I saw only one implementation within the codebase, so I might underestimate this particular point.

True, I was thinking about implementing your first proposal. Something that is looping through znodes, and check if auth_policies.namespace_auth.containsKey(role) for the default implementation.

That might be expensive if called frequently from clients in any case with a non-trivial number of namespaces.

I must admit that I saw only one implementation within the codebase, so I might underestimate this particular point.

Yes, there's just one implementation in the codebase, the default where ACLs are stored with the namespace policies on ZK, though there are others using different authorization options, including :
* Use AuthZ information stored in the AuthN credentials (eg: JWT token, or TLS certificate, also containing which resources this token has access to.

* Third party ACLs provider (eg: Okta, Auth0, Vault, etc.. )

Thanks a lot @merlimat for all your insights 😄 How this issue should be adressed in your opinion?

I'm facing the same situation while developing pulsar-flink-connector.
My scene is as follows:
I was developing a Pulsar catalog in Flink and mapping tenant/namespace to database,topic to table. Therefore, for each role, I need to list all topics it has access to, and users could fire queries based on his/her tables.

I cheated for my part, I'm injecting the tenant/namespace in the user field of the SASL_PLAIN to find topics. Leaving the issue opened as the main problem is still here, like for @yjshen.

Closed as stale. Please open a new issue if it's still relevant to the maintained versions.


modify.Propogate()

promote or demote attributes, or
disspate, disperse, proposgate, promote, radiate, spread
modify.Propogate(entities, attribute_names, enum_from, enum_to, enum_original, enum_method)
attribute_names = name, list_of_names, null
modify.Propogate(my_ents, "area", "polygon", "collection", "sum")
modify.Propogate(my_ents, "rgb", "vertex", "neighbours", "average")
this does not affect the original attribute
if you ant to del the attribute, just set it to null
when promoting numeric attributes, the user needs to specify the method of aggregation
method = max, min, average, mode, median, sum

implemented as modify.pushAttribs()


Add user enabled check for saml

Problem:
When attempting to login with a user which is not enabled no error is
presented in the UI.
Solution:
Add a check on the user for enabled and redirect with an error.
Previously this was being caught on the token so the UI would not
show/redirect an error.
Issue: rancher/rancher/issues/10646

LGTM


CSPM 23.9.1 Release Updates

Description

Motivation and Context

How Has This Been Tested?

Screenshots (if appropriate)

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)

Checklist

[ ] I have updated the documentation accordingly.
[ ] I have read the CONTRIBUTING document.
[ ] I have added tests to cover my changes if appropriate.
[ ] All new and existing tests passed.

https://redlock.atlassian.net/browse/RLP-112256
https://redlock.atlassian.net/browse/RLP-108057
https://redlock.atlassian.net/browse/RLP-110518


Add object part iterators

This will be useful for s3 <-> gs sync functionality.
depends on #42 #43 #44

Codecov Report

Merging #45 into master will increase coverage by 0.14%.
The diff coverage is 88.37%.

@@            Coverage Diff             @@
##           master      #45      +/-   ##
==========================================
+ Coverage   86.62%   86.76%   +0.14%
==========================================
  Files           8        8
  Lines         299      340      +41
==========================================
+ Hits          259      295      +36
- Misses         40       45       +5

Impacted Files
Coverage Δ

ssds/blobstore/__init__.py
64.00% <50.00%> (-6.59%)
:arrow_down:

ssds/blobstore/s3.py
88.23% <95.45%> (+1.81%)
:arrow_up:

ssds/blobstore/gs.py
96.05% <100.00%> (+0.74%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 138a140...3eb5269. Read the comment docs.


HCMPRE-169 code review comments

Summary by CodeRabbit

New Features

Added a new dependency to improve modularity and capabilities of the health services.
Introduced new constants for better error handling related to invalid plan IDs.
New method to count PlanConfiguration entities based on search criteria.

Improvements

Enhanced tenant ID handling in validation logic for better clarity and maintainability.
Simplified response generation in PlanConfigController for creating plan configurations.

Documentation

Improved code documentation and comments for better understanding of methods and their functionalities.

Refactor

Refactored multiple services to utilize modern Java practices, improving code readability and maintainability.

Bug Fixes

Fixed potential inconsistencies in error message formatting.

@coderabbitai review

https://digit-discuss.atlassian.net/browse/HCMPRE-169
This is the ticket @kavi-egov


Add repositories

automatically rebase when behind
raise concurrency

Change-type: patch

@balena-ci I self-certify!


@Ajax.beginform is missing in MVC 6

notice that ajax.beginform is missing from MVC 6 , why ?  when I try to write ajax.beginform in a view its seems ajax object is missing. Any help ?
If I write ajax form like below then it works fine but “Onsuccess” event and so many other parameters are missing in it. See below
 <form asp-controller="PreRegistration" asp-action="Index"  data- ajax="true"

 data-method="post"  data-ajax-mode="replace" data-ajax-update="#cooldiv">

            FirstName:<input asp-for="FirstName" />

                           <input type="submit" value="Save" />

    </form>

Duplicate of https://github.com/aspnet/Mvc/issues/2015


drush si config_installer fails: file_put_contents(sites/default/settings.php): failed to open stream: Permission denied install.inc:337

When I run drush si config_installer inside the container, it fails with a permission error. I followed the "Usage" instructions in the readme. All files (including sites/default/settings.php) are owned by my user id and group id and have -rwxrwxr-x permissions, while the container runs as uid 1000. I cannot work around this by specifying a different --user option to docker-compose exec, as only the drupal user (uid 1000) has access to the Drupal directory /var/www/drupal/.
$ docker-compose exec --user drupal drupal bash
🐳 drupal@decoupled.backend.docker.amazee.io:~/public_html/backend/web (master *=)$ drush si config_installer -y --account-name=admin --account-pass=admin
You are about to DROP all tables in your 'drupal' database. Do you want to continue? (y/n): y
Starting Drupal installation. This takes a while. Consider using the --notify global option.                                                                                [ok]
file_put_contents(sites/default/settings.php): failed to open stream: Permission denied install.inc:337                                                                                [warning]
Exception: Failed to modify sites/default/settings.php. Verify the file permissions. in /var/www/drupal/public_html/backend/web/core/includes/install.inc:338                                                                            [error]
Stack trace:
#0 /var/www/drupal/public_html/backend/web/profiles/contrib/config_installer/src/Form/SiteConfigureForm.php(121): drupal_rewrite_settings(Array)
#1 [internal function]: Drupal\config_installer\Form\SiteConfigureForm->buildForm(Array, Object(Drupal\Core\Form\FormState), Array)
#2 /var/www/drupal/public_html/backend/web/core/lib/Drupal/Core/Form/FormBuilder.php(514): call_user_func_array(Array, Array)
#3 /var/www/drupal/public_html/backend/web/core/lib/Drupal/Core/Form/FormBuilder.php(472): Drupal\Core\Form\FormBuilder->retrieveForm('config_installe...', Object(Drupal\Core\Form\FormState))
#4 /var/www/drupal/public_html/backend/web/core/includes/install.core.inc(909): Drupal\Core\Form\FormBuilder->submitForm('Drupal\\config_i...', Object(Drupal\Core\Form\FormState))
#5 /var/www/drupal/public_html/backend/web/core/includes/install.core.inc(584): install_get_form('Drupal\\config_i...', Array)
#6 /var/www/drupal/public_html/backend/web/core/includes/install.core.inc(540): install_run_task(Array, Array)
#7 /var/www/drupal/public_html/backend/web/core/includes/install.core.inc(117): install_run_tasks(Array)
#8 /var/www/drupal/public_html/backend/vendor/drush/drush/includes/drush.inc(726): install_drupal(Object(Composer\Autoload\ClassLoader), Array)
#9 /var/www/drupal/public_html/backend/vendor/drush/drush/includes/drush.inc(711): drush_call_user_func_array('install_drupal', Array)
#10 /var/www/drupal/public_html/backend/vendor/drush/drush/commands/core/drupal/site_install.inc(82): drush_op('install_drupal', Object(Composer\Autoload\ClassLoader), Array)
#11 /var/www/drupal/public_html/backend/vendor/drush/drush/commands/core/site_install.drush.inc(255): drush_core_site_install_version('config_installe...', Array)
#12 /var/www/drupal/public_html/backend/vendor/drush/drush/includes/command.inc(422): drush_core_site_install('config_installe...')
#13 /var/www/drupal/public_html/backend/vendor/drush/drush/includes/command.inc(231): _drush_invoke_hooks(Array, Array)
#14 /var/www/drupal/public_html/backend/vendor/drush/drush/includes/command.inc(199): drush_command('config_installe...')
#15 /var/www/drupal/public_html/backend/vendor/drush/drush/lib/Drush/Boot/BaseBoot.php(67): drush_dispatch(Array)
#16 /var/www/drupal/public_html/backend/vendor/drush/drush/includes/preflight.inc(66): Drush\Boot\BaseBoot->bootstrap_and_dispatch()
#17 /var/www/drupal/public_html/backend/vendor/drush/drush/drush.php(12): drush_main()
#18 {main}

Sounds like you installed Druipal before once and are ttrying to reinstall now. In that case, you need to set the permissions of the settings.php file back to writable on the host system so the installer can touch it.


Visualizations?

Is there any support for viewing visualizations from this kernel in jupyter?

plotly-scala or vegas ought to work with the upcoming version (currently on the develop branch).


Invisible glyph bounds at wrong positions in PDF

Environment

Tesseract Version: Tesseract Open Source OCR Engine v<IP_ADDRESS>90623 with Leptonica
(downloaded from https://digi.bib.uni-mannheim.de/tesseract/ )
Platform: W7 64 bit 6.1.7601

Call:
"C:\Program Files\Tesseract-OCR\tesseract" scan.tif scan-ocr pdf
Current Behavior:
text bounds are not identical to visible glyphs in Adobe Reader. Example:

Expected Behavior:
text bounds should be identical to visible glyphs in Adobe Reader. In the graphic, the blue color should cover the "n".
Suggested Fix:
I suspect that the /W array is missing in the font dictionary:

So Adobe will use the /DW 500 entry (screenshot from PDF 32000 specification):

scan-ocr.pdf
scan.tif.zip

Interesting suggestion. If correct, why would it show up as an n - 1
problem in highlighting?

Sorry, I don't understand what you mean. My argument is that the highlights widths don't match. Adobe gets these from the font data, and widths are different in a proportional font. And it isn't just the "n". When trying to highlight the "I" it looks like this:

The glyphless font deliberately uses equal width for every character. I
stretch the the word using Tz in the PDF to make it fit. So I expect word
highlighting to look correct, but not character highlighting within a word.
This design was chosen to maximize compatibility across all the scripts
supported by Tesseract while minimizing complexity.

I had a look with the glyph contour display of PDFBox and there it matches the word bounds:

So maybe Adobe is to blame, but users will of course see this differently :-(

I think I found a bit more... "Introduction" has 12 characters but looks like this in the PDF content stream:
1 0 0 1 77.76 738.16 Tm /f-0-0 11 Tf 107.076 Tz [ <0049006E00740072006F00640075006300740069006F006E0020> ] TJ
this is 13 characters. The last one is a space. This space is over the final "n".

When removing "3 Tr" so that the "invisible" font gets visible, it looks like this:

This is really 13 characters. For some reason, Adobe doesn't want to mark the final space.

I just see that the PDFBox screenshot shows it too: "ISO" has 4 characters, "32000" has 6 characters.
Maybe the original idea was to put the space there for text extraction? However it isn't needed, good text extractors "imagine" the space from the position differences.
If the space character is needed, then it should be positioned over the actual space.

https://github.com/tesseract-ocr/tesseract/issues/1900

Thanks, after reading that one, I think this issue is also somewhat duplicate of https://github.com/jbarlow83/OCRmyPDF/issues/450 .

You should check the bounding box of the whole word 'Introduction' with the hocr format. Does it also end before the last glyph?

Tesseract's recognizer just finds words, and doesn't tell us anything about
spaces. Which makes sense: how would an OCR program know if there is one
space, two spaces, etc? We add the space in during PDF generation to help
some viewer with copy-paste; otherwise it is common for words to run
together. Apple's viewer is notorious for this. I'm a little reluctant to
put a space outside the word bounding box - there is no guarantee there
will be room for it, and I don't really want the PDF output module to get
into the layout analysis game. One possibility might be to play with the
font such that U+0020  gets zero (or non-zero) width, while every other
character maintains the same fixed width we've always had. Then adjust the
Tz word stretch appropriately.
https://github.com/tesseract-ocr/tesseract/blob/master/src/api/pdfrenderer.cpp#L471
I haven't touched the font in a while, so not sure how easy it is to make a
change like this. If you want to play with this yourself, I recommend using
the program "ttx" from fonttools to transform the font into an XML file.
Edit the file, then transform it back. I have a feeling it won't be trivial
but it might be possible. See also the design discussion at the top of
pdfrenderer.cpp, which explains how everything works.

Yeah I understand that this feature was implemented to "help" low quality text extractors.
How about making the feature configurable for PDF? IMHO the majority user expectation is whatever Adobe does, that is the gold standard.
Zero width space also sounds like an interesting idea to explore. You probably have to add appropriate /W entries.
(The reason I created this issue: we're using a commercial OCR tool on a project that grows fast. The OCR is fine, but licensing is a pain, it doesn't use all CPU cores, and the logging is almost non existent, the whole thing is a black box, so I was thinking about replacing it with tesseract, but before we discuss this with the client I need to be sure the client would be satisfied and that its clients too)

@amitdo The bounding box is correct:

      Introduction

Adobe Acrobat is not as popular as it used to be 10 years ago.
Default PDF viewers:

Windows 10 - Chromium based Edge - Pdfium
macOS - Preview
Chromium / Chrome / Edge - Pdfium
Firefox - pdf.js
Linux - Evince/Okular (Poppler)

So most users will use the OS/browser's built-in PDF viewers, which is not Adobe's viewer.
I tested your pdf file with Evince (poppler) and with chromium (pdfium). The words bounding boxes look very good.
The best solution is to find a method that will work on all these viewers, without a special parameter for specific viewer.

I tested your pdf file with  and with Chromium (pdfium), Firefox (pdf.js) Evince (poppler).
The words bounding boxes look very good when the page is viewed with pdfium/pdf.js.
Poppler suffers from the same issue you raised above combined with a 'zebra effect'.

With PDF.js on firefox, double click marks the whole word, when I mark the final "n", I get a space.
With Chrome, double click shows the same effect than with Adobe Reader.
With MS Edge, same effect than with PDF.js.

I took a look at the code. It looks like one can pretty easily remap U+0020
to an alternate glyph in the cidtogmap. It's been five years since the last
significant change, and my memory is terrible, but I I'm confident we
currently map everything down to a single "glyph" in the font. That
slightly misleading code at line 549 of pdfrenderer.cpp is just filling out
the 2 byte entries one byte at a time.
So then there's the question of adding a another glyph to the font. The
design notes from Ken say we've got an unused glyph at index 0. Unused
because it gives heartburn to the Adobe parser. And then one at index one
which is used everywhere. It's not quite trivial, but I don't yet see any
reason we can't add another entry at index 2 that is identical or near to
the entry in index 1. This means tranforming the font to xml using ttx from
fonttools, doing some careful copy pasting, transforming it back, and
hoping nothing too scary happens.
Next there is the question of assigning  the zero width (or near zero
width) to just that new entry. As of right now, I'm not sure exactly how to
do that. But I think Tilman's suggestion of adding a /W array to the
/CIDFont dictionary is the first thing to try. (Currently line 526 in
pdfrenderer.cpp). There's probably spot inside the font as well to specify
width, that we'll want to also set, for consistency, compatibility, and
minimal confusion.
Finally, I already mentioned that the bounding box stretch can be computed
without considering the U+0020, which is basically removing line 471 from
pdfrenderer.cpp.  After that - if it works at all - then just compatibility
testing with various renderers.
I really don't know if this will work or not, but there's a chance, and
it's my best suggestion for what to try. Might make sense to contact Ken
Sharp and see if he has an opinion on the topic. Tilman, I know it's a lot
of work but if you want to try this, you will probably get it done
significantly faster than me. (Unlike 5 years ago, my day job does not
currently intersect with PDF. That doesn't totally stop me, but it does
slow things down quite a lot.)

Thanks for the nice comment; my problem is that I haven't done C/C++ for almost 10 years except maintenance of my existing software. I don't even have a dev system up that supports current language standards so I would have to install / understand / learn that first. However I'll keep it this issue in mind when I have more time at work.

The C++ part @jbreiden suggested is just

Then adjust the
Tz word stretch appropriately.

That seem to be easy. I probably can do the necessary changes.
But before updating the code we need to change the font as Jeff suggested. For this task C++ skills are not needed. Jeff suggested to use a Python based program to export the font to xml (?), so you can edit it in a text editor.

Oh, there is that part

I took a look at the code. It looks like one can pretty easily remap U+0020
to an alternate glyph in the cidtogmap.

Maybe it easy, but I don't know how to do this part.
Still, you can play with the font...

@jbarlow83, maybe you can help us here.

I'll spend a little time right now and see what I can do.

I tried the simplest thing possible, leaving the font alone and trying to
use that glyph at index 0. I expected Adobe Reader to completely choke, and
Pdfium/Chrome to work great. Instead, my ancient copy of Adobe Reader 9.5.5
(e.g. the one for Linux) works fine. However, Pdfium/Chrome is highlighting
beyond the end of the word. That's what you would expect if Pdfium was
ignoring the zero width on index 0.
--- pdfrenderer.cpp.orig 2019-07-07 08:23:24.000000000 -0700
+++ pdfrenderer.cpp 2020-02-09 11:18:40.578544848 -0800
@@ -535,6 +536,7 @@
"  /Subtype /CIDFontType2\n"
"  /Type /Font\n"
"  /DW " << (1000 / kCharWidth) << "\n"

"  /W [ 0 [0 500] ]\n"
">>\n"
"endobj\n";
AppendPDFObject(stream.str().c_str());
@@ -546,6 +548,8 @@
for (int i = 0; i < kCIDToGIDMapSize; i++) {
cidtogidmap[i] = (i % 2) ? 1 : 0;
}
const int kSpaceCID = 20;
cidtogidmap[kSpaceCID * 2 + 1] = 0;
size_t len;
unsigned char *comp = zlibCompress(cidtogidmap.get(), kCIDToGIDMapSize,
&len);
stream.str("");

Alternative: (tesseract hocr) + (hocr-pdf (https://github.com/ImageProcessing-ElectronicPublications/hocr-tools)).

Tried modifying the font to add a specific entry for U+0200. Same results, Adobe good, pdfium bad. This is the point where I pause, and people take a look for mistakes. If nobody finds anything, the next step is probably asking for help. That's Ken Sharp about the overall approach & especially the font, and Pdfium folks to help debug why the /W entry does not appear to be honored.
--- pdfrenderer.cpp.orig	2019-07-07 08:23:24.000000000 -0700
+++ pdfrenderer.cpp	2020-02-09 12:00:57.961541649 -0800
@@ -468,7 +468,6 @@
     } while (!res_it->Empty(RIL_BLOCK) && !res_it->IsAtBeginningOf(RIL_WORD));
     if (res_it->IsAtBeginningOf(RIL_WORD)) {
       pdf_word += "0020";
-      pdf_word_len++;
     }
     if (word_length > 0 && pdf_word_len > 0) {
       double h_stretch =
@@ -535,6 +536,7 @@
     "  /Subtype /CIDFontType2\n"
     "  /Type /Font\n"
     "  /DW " << (1000 / kCharWidth) << "\n"
+    "  /W [ 1 [500 1] ]\n"
     ">>\n"
     "endobj\n";
   AppendPDFObject(stream.str().c_str());
@@ -544,8 +546,11 @@
   const std::unique_ptr<unsigned char[]> cidtogidmap(
       new unsigned char[kCIDToGIDMapSize]);
   for (int i = 0; i < kCIDToGIDMapSize; i++) {
-    cidtogidmap[i] = (i % 2) ? 1 : 0;
+    cidtogidmap[i] = (i % 2) ? 0x01 : 0x00;
   }
+  const int kSpaceCID = 20;
+  cidtogidmap[kSpaceCID * 2] = 0x00;
+  cidtogidmap[kSpaceCID * 2 + 1] = 0x02;
   size_t len;
   unsigned char *comp = zlibCompress(cidtogidmap.get(), kCIDToGIDMapSize, &len);
   stream.str("");

debug.pdf
font.zip

@amitdo I will look.
I'd consider using a separate Tz for the trailing space rather than modifying the font.
1.0 Tz [ <0049006E00740072006F00640075006300740069006F006E> ] 0.001 Tz [ <0020> ] TJ
Seems like it would be simpler and less reliant on fonts being parsed.

The /W entry as it is now

means CID 1 has a width of 500, CID 2 has a width of 1. I assume that all others have default width (500). If you wanted to change the width of space, then you should have done something for CID 32.

You are correct. Result works on both Acroread & Pdfium. File attached and ready for compatibility testing. If nobody finds trouble, I'm comfortable submitting. This variant makes no changes to the font, and sets the width of space to zero.
--- pdfrenderer.cpp.orig	2019-07-07 08:23:24.000000000 -0700
+++ pdfrenderer.cpp	2020-02-09 13:26:33.743553816 -0800
@@ -468,7 +468,6 @@
     } while (!res_it->Empty(RIL_BLOCK) && !res_it->IsAtBeginningOf(RIL_WORD));
     if (res_it->IsAtBeginningOf(RIL_WORD)) {
       pdf_word += "0020";
-      pdf_word_len++;
     }
     if (word_length > 0 && pdf_word_len > 0) {
       double h_stretch =
@@ -535,6 +536,7 @@
     "  /Subtype /CIDFontType2\n"
     "  /Type /Font\n"
     "  /DW " << (1000 / kCharWidth) << "\n"
+    "  /W [ 32 [0] ]\n"
     ">>\n"
     "endobj\n";
   AppendPDFObject(stream.str().c_str());
@@ -544,8 +546,11 @@
   const std::unique_ptr<unsigned char[]> cidtogidmap(
       new unsigned char[kCIDToGIDMapSize]);
   for (int i = 0; i < kCIDToGIDMapSize; i++) {
-    cidtogidmap[i] = (i % 2) ? 1 : 0;
+    cidtogidmap[i] = (i % 2) ? 0x01 : 0x00;
   }
+  const int kSpaceCID = 0x0020;
+  cidtogidmap[kSpaceCID * 2] = 0x00;
+  cidtogidmap[kSpaceCID * 2 + 1] = 0x00;
   size_t len;
   unsigned char *comp = zlibCompress(cidtogidmap.get(), kCIDToGIDMapSize, &len);
   stream.str("");

testme1.pdf

@jbarlow83 The problem with hidden Arial is coverage. Tesseract supports the entire basic multilingual plane and beyond. The glyphless font is equally happy with Cherokee and English.

Chromium, Evince - look good.
Firefox - no effect, the issue still exist.

I just tested with "testme1.pdf" and double clicking "Introduction" on firefox works fine. But the same for "digital" highlights at the wrong place, even more for "enable" but this may be a firefox bug. I think I reported such a bug myself a long time ago. PDFBox shows correct glyph bounds.
However PDFBox reports a warning "No glyph for code 32 (CID 0020) in font GlyphLessFont", which didn't happen with the original file. But this is just a minor inconvenience.

Firefox has never worked well with Tesseract PDF. Does this change make it
worse?
https://github.com/mozilla/pdf.js/issues/6509
https://github.com/mozilla/pdf.js/issues/6863
I suppose we should also check samples in vertical Japanese, right to left
Andrabic, and bidirectional to see if there are any regressions. Plus PDF
parsers running on non Linux operating systems.

Firefox has never worked well with Tesseract PDF. Does this change make it
worse?

No, at least not with the tested document.

@jbreiden, what's the status of this? Should we apply your patch?

@stweil,
Please do some tests with this patch on macOS with a few images.
When testing the PDF file produced by Tesseract with a PDF viewer, the tester should test:

"Select all" (menu/button/keyboard)

copy the selected area and paste it to a text editor. Compare the result to tesseract's txt output.

Do a mouse double click on one word. Repeat this action on other words.
Select  by mouse click and drug multiple lines.
Search a word, search two words (word<space>word).

When testing the PDF file produced by Tesseract with a PDF viewer, the tester should test:

"Select all" (menu/button/keyboard)

copy the selected area and paste it to a text editor. Compare the result to tesseract's txt output.

Do a mouse double click on one word. Repeat this action on other words.
Select  by mouse click and drug multiple lines.
Search a word, search two words (word<space>word).

@jbarlow83,
Maybe you can help us here by doing some tests with this patch on macOS with a few images?

@amitdo looking for a solution to this problem I found this issue, so I applied the patch above and there it is, the solution is working. All the tests you described are positive, at least on the couple of documents I checked it on - simple document as well as form with tricky formatting. In all of the cases the text is extracted correctly. The way the text is overlayed is better than without the patch. It's not perfect though, I found couple of places where it's still misses the glyph boundaries (mostly when around letters i), but overall it's a huge improvement.

@bbqf
Thanks for reporting.
We still didn't hear from Mac users. How does this patch works with macOS Preview?

We still didn't hear from Mac users. How well does this patch work with macOS Preview?

This patch unfortunately does not improve results on macOS Preview (Preview 10.1, macOS 10.14.6). Assuming I compared the right files. I did not apply the patch.
Visually, it's better:
Without patch (scan-ocr.pdf):

With patch (testme1.pdf):

However it removes spaces from the copy and paste text:
Without patch (scan-ocr.pdf):
programming language, PDF is based ona structured binary file format that is optimized for high performance in interactive viewing. PDF also includes objects, such as annotations and hypertext links, that are not part of the page content itself but are useful for interactive viewing and document interchange.

With patch (testme1.pdf):
programminglanguage,PDFisbasedona structuredbinaryfileformatthatisoptimizedforhighperformance in interactive viewing. PDF also includes objects, such as annotations and hypertext links, that are not part of thepagecontentitselfbutareusefulforinteractiveviewinganddocumentinterchange.

I compared the previously uploaded files indicated above without applying the patch.

Definitely looks like one-off bug.

Maybe it does like the zero width space, and it will honor 1 unit width.

Hi, do we have any update on this issue?

I'd love to contribute and finally get the fix released, but I have no access to Mac, and as I reported earlier, the fix works for me on Windows. Is there a cross-platform way to test it? I am fine with Linux/Docker/VMs but I can't help with Mac.

@bbqf It's possible to set up VM for macOS guest on Windows or Linux. e.g. https://www.makeuseof.com/tag/macos-windows-10-virtual-machine/
I can often be persuaded to test new files and I have access to all platforms.

@jbarlow83,
https://github.com/tesseract-ocr/tesseract/issues/2879#issuecomment-583892594
Can you please try to implement your suggestion and test it?

Hello!
I would like to implement the fix for this problem, since we feed the PDFs Tesseract generates into Poppler, It's not a problem if it breaks behaviour on other PDF renderers.
But I don't want to maintain a fork of Tesseract and have to compile it myself. So my idea was to extract the essence of the fix and apply them after the fact, to the PDFs that Tesseract generates. However I am not having success. Perhaps someone can advise me on exactly the mutation I need to carry out on the PDF in order to benefit from this fix?
I have tried: (code examples are in rust)

removing the space (U+0020) at the end of every word

for operation in &mut content.operations {
    match operation.operator.as_ref() {
        "Tj" | "TJ" => {
            for operand in operation.operands.iter_mut() {
                match operand {
                    Object::Array(ref mut arr) => {
                        for obj in arr {
                            let obj = obj.as_str_mut().unwrap();
                            if obj[obj.len() - 2..] == [0, 32] {
                                obj.truncate(obj.len() - 2);
                            }
                        }
                    }
                    _ => {}
                }
            }
        }
        _ => {}
    }
}

setting the width of the space character in all the fonts to 0

let fonts = doc
    .objects
    .iter()
    .filter_map(|(id, obj)| (obj.type_name().ok() == Some("Font")).then_some(id.to_owned()))
    .collect::<Vec<_>>();
for font in fonts {
    if let Ok(font) = doc.get_dictionary_mut(font) {
        let _32 = Object::Integer(32);
        let _0 = Object::Array(vec![Object::Integer(0)]);
        font.set(b"W".to_vec(), Object::Array(vec![_32, _0]));
    }
}

(admittedly, i'm not sure what this part of the diff is doing)
-    cidtogidmap[i] = (i % 2) ? 1 : 0;
+    cidtogidmap[i] = (i % 2) ? 0x01 : 0x00;
   }
+  const int kSpaceCID = 0x0020;
+  cidtogidmap[kSpaceCID * 2] = 0x00;
+  cidtogidmap[kSpaceCID * 2 + 1] = 0x00;

Neither options result in any visible difference in the PDF for me. Can anyone advise?

At this point I believe improvements would come from having Tesseract generate tagged PDFs with structural markup that indicate word boundaries. @arifd that would implementing section 14.8 of the PDF RM.

Any chance that this issue will be fixed sometimes after 3 years?

Try this file.
scan.pdf
If word boundaries look "kinda" ok, I'll commit this one-off fix.
Pdf viewers do not highlight space, so we should not add it to the word.
index 6b9e248d..fb4bcd2f 100644
--- a/src/api/pdfrenderer.cpp
+++ b/src/api/pdfrenderer.cpp
@@ -466,7 +466,6 @@ char *TessPDFRenderer::GetPDFTextObjects(TessBaseAPI *api, double width, double
     } while (!res_it->Empty(RIL_BLOCK) && !res_it->IsAtBeginningOf(RIL_WORD));
     if (res_it->IsAtBeginningOf(RIL_WORD)) {
       pdf_word += "0020";
-      pdf_word_len++;
     }
     if (word_length > 0 && pdf_word_len > 0) {
       double h_stretch = kCharWidth * prec(100.0 * word_length / (fontsize * pdf_word_len));

In my opinion it looks good now.

This patch was rejected before. See  #3139.

It works significantly better to output the word and space separately, and use horizontally scaling to calculate the width of the space so it falls exactly between the end of the current of the word and beginning of the next word.
The "words mixed together" issue happens because the actual position of the space will overlap the word boxes instead of being between them, especially if the word is particularly wide or narrow (wwwwwww vs iiiiii). So the PDF renderers are acccurately reporting what they "see".
I implemented this in OCRmyPDF's hOCR based renderer. I won't have time to add it to Tesseract for a few months but that is how to move forward.
The next other thing to do, I believe, is add a double width character and negative displacement character to the GlyphlessFont, to better handle Asian and RTL scripts respectively.

Let's continue the discussion that started in
https://github.com/tesseract-ocr/tesseract/issues/3673#issuecomment-1986862566.
See the following comments.
CC: @stweil, @zdenop, @egorpugin

Different viewer behavior means that someone is correct and others are not.

No. It means that the pdf format is very complex and the spec itself is not clear even for pdf experts.
Also, every pdf viewer use its own 'clever guesses' techniques  for some features of the format. This is very relevant here.

I don't like this patch, but if @stweil will explicitly say he has no issue with it, I will stop talking about it.

Why dont you like it?
Do we have a better patch?

Why dont you like it?

Because it is not good for Apple's Preview. Evince also has some issues with it.

It does not matter.
Patch fixes incorrect word length (+1 extra). Like word 'a' had length of 2.

For now, a better alternative is to keep the status quo (the code before the latest applied patch). Although the text selection looks somewhat ugly (off by one), copy and paste and search functionality work better in Apple's Preview and column selection works better in Evince.


Buried treasure maps

Spark link
https://spark.lucko.me/Ca7y18HA0o
Description of issue
Opening a buried treasure causes a massive lag, and taking the map in the inventory make me unable to join the server
Purpur version
[20:14:15 INFO]: Checking version, please wait...
[20:14:16 INFO]: Current: git-Purpur-2062 (MC: 1.20.1)*

You are running the latest version

Agreements

[X] I am running the latest version of Purpur available from https://purpurmc.org/downloads.
[X] I have searched for and ensured there isn't already an open issue regarding this.
[ ] I ticked all the boxes without actually reading them
[X] My version of Minecraft is supported by Purpur.

Other

This is my error while connecting to the server

This is, unfortunately, a vanilla issue. There are options withing both paper and purpur to disable or limit treasure searching.
Paper has an option to only look for already found chests, and to disable cartographers from generating treasure maps. Purpur has an option for disabling dolphins from searching for treasure.

Oh i see


Update CODEOWNERS

Add @Azure/azure-sdk-write-identity as an owner of /sdk/identity

/check-enforcer override


nova spawns should not return exception when no spawns have occurred.

To test, delete all your docker containers and then do a bin/start_dev_env.sh.  After the server starts, log in to Goldstone and go to the "compute" metrics page.

:+1:
after initial bin/wipe_docker.sh and page rendering:

immediately after a nova boot command:

within a minute:


add gateway api user stories kni kep

Adds the Gateway API user story that was an action item from today's zoom sync.

cc @dprotaso (this is where we're collaborating on https://github.com/kubernetes/enhancements/pull/4477, if you'd like to expand or add more related user stories)


Change references to schema id and from_schema slot in YAML source

The main YAML source file (currently: https://github.com/GenomicsStandardsConsortium/mixs/blob/main/src/mixs_6_2_for_merge/schema/mixs_6_2_for_merge.yaml) has:

id: https://w3id.org/mixs-6-2-rc
from_schema: https://w3id.org/mixs-6-2-rc

This needs to be changed so we're using https://w3id.org/mixs
Note: This will further require modifications to the .htaccess file on w3id.org (https://github.com/perma-id/w3id.org/blob/master/mixs/.htaccess) so that the patterns know where to look for files/artifacts in the new directory structure.

All appearances of the regular expression mixs.6.2, across the repo, have been replaced with plain mixs

https://w3id.org/mixs redirects to the documentation index page, https://genomicsstandardsconsortium.github.io/mixs/

I recommend removing all references to the never-implemented "vocab endpoint"

https://github.com/perma-id/w3id.org/pull/3650/files

Because the (expandable) schema source for from_schema is says it is a read-only slot, I am removing all of those assertions.


Show pacman package as the manager on Arch

Show pacman as the package manaer instead of yay

No. Most people would rather see their AUR manager.

But isn't yay just a wrapper around pacman? It's not a true package manager

But isn't yay just a wrapper around pacman? It's not a true package manager

yay is a package manager helper. If you wanna see pacman instead, feel free to modify your local copy. There's also metapackagemanagers and other helpers on here as well such as pmm, paru, pacstall, port, etc.


Bluetooth: Host: No space to store CCC when GATT client loses bond and CCC Lazy Loading is enabled

Describe the bug
When previously bonded GATT client reconnects and CONFIG_BT_SETTINGS_CCC_LAZY_LOADING is enabled, CCC configuration is loaded from Settings. If GATT client had lost bond and CONFIG_BT_GATT_AUTO_SEC_REQ is disabled, GATT client can stay connected without reestablishing security. However, when this GATT client tries to write CCC on previously used Attribute it can't do that and following log message appears: LOG_WRN("No space to store CCC cfg");. It happens because the find_ccc_cfg function in subsys/bluetooth/host/gatt.c returns error. That's because the existing CCC for the GATT client can't be overwritten as the security is not reestablished (as expected) and also there is no space to store new CCC configuration, because cfg array length in struct _bt_gatt_ccc is set to CONFIG_BT_MAX_CONN which is often equal to 1. When CONFIG_BT_SETTINGS_CCC_LAZY_LOADING is disabled, the cfg array length is set to CONFIG_BT_MAX_PAIRED + CONFIG_BT_MAX_CONN and the issue doesn't replicate, because there is enough space to store new CCC configuration.
To Reproduce
Steps to reproduce the behavior:

Build and flash zephyr/samples/bluetooth/peripheral sample with BT_GATT_AUTO_SEC_REQ disabled.
Pair with the sample from your Bluetooth central (e.g. phone). The phone automatically writes Service Changed CCC. You can also manually write some others CCC (e.g. Battery Service) using e.g. nRF Connect app.
Unpair your phone.
Reconnect your phone without pairing (you can do that via nRF Connect app)
Write Service Changed CCC or some other CCC that you have written in step 2.
See that the CCC write fails with warning message: No space to store CCC cfg.

You can verify that it doesn't reproduce if CONFIG_BT_SETTINGS_CCC_LAZY_LOADING is disabled.
Expected behavior
The CCC write should succeed regardless of CONFIG_BT_SETTINGS_CCC_LAZY_LOADING being enabled or no. The phone doesn't want to overwrite the existing CCC. It wants to write new one as it has lost bond.
Impact
Google Fast Pair Service protocol (https://developers.google.com/nearby/fast-pair/specifications/introduction) requires using GATT before bonding. Because of that, if a Phone loses bond, it is unable to bond again using Fast Pair, because it is unable to write CCC when CONFIG_BT_SETTINGS_CCC_LAZY_LOADING is enabled.
Environment (please complete the following information):

OS: Linux
Commit SHA: b37ce40878ec52295afe6350d17712d60eb6639a (recent one)

I agree that the same result should be expected regardless of CONFIG_BT_SETTINGS_CCC_LAZY_LOADING.

Might be fixed by https://github.com/zephyrproject-rtos/zephyr/pull/62985

Might be fixed by #62985

Yes, it solves this issue, because now CCC cfg can be overwritten by peer that doesn't reestablish security which was not allowed earlier.


way to use any other gateway without using default gateway in the computer

Is there any option to use another gateway for the proxy server without using the default gateway in windows
thanks

Yes, you can set any upstream proxy... you don't have to use the windows proxy settings. Moreover you can use different upsteream proxy for each request (for example by url or for load balancing reason)

thanks for the quick reply
can you please direct me to sample code
thanks and best regards

You can find the sample code here:
https://github.com/justcoding121/Titanium-Web-Proxy/blob/master/examples/Titanium.Web.Proxy.Examples.Basic/ProxyTestController.cs

Thanks for your reply
i try like this
var proxyServer = new ProxyServer();
proxyServer.UpStreamHttpProxy = new ExternalProxy("<IP_ADDRESS>", 808);
proxyServer.UpStreamHttpsProxy = new ExternalProxy("<IP_ADDRESS>", 808);
its works fine for all traffic but if i want external proxy for specific sites for eg: facebook.com
how to implement that
i try OnRequest method like this
if (e.HttpClient.Request.RequestUri.AbsoluteUri.Contains("facebook.com"))
{
e.CustomUpStreamProxy = new ExternalProxy("<IP_ADDRESS>", 808);
}
then its not working is there any option to do that
thank you very much
and best regards

Is your code at least called?

yes its called i try in debug mode and object e.CustomUpStreamProxy has new value as well


Update README.md

had to sudo two things

Thanks, but perhaps you missed the note:
“ Note: In the rest of the note, a command prefixed with # means that it must be executed in superuser mode. The $ prefix means it should be executed in regular user mode.”