lib.types: attrsWith named placeholder

Description of changes

Set placeholder="name" to configure the <name> placeholder for the docs. This is particularly useful when "<name>" doesn't make sense or when dealing with nested attrsOf submodule. Which would yield "<name>.<name>"

Usage example
mkOption {
  type = types.attrsWith {
    elemType = types.str;
    placeholder = "userName";
  };
  default = "root";
}

Context

Closes https://github.com/NixOS/nixpkgs/issues/295872
Closes #193311

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Apart from that everything should behave the same.

Seems like a wasted opportunity actually. The type could accept a function from name to type so you really have a name, that you can even feed into the submodule specialArgs using a different name name, etc.
Anyway, I guess what I'm getting at is that the attrs types should be factored into a single more capable function, because this property is not mutually exclusive with the other property about laziness (attrsOf/lazyAttrsOf).
See also

https://github.com/NixOS/nixpkgs/issues/295872

I'll add that to the description as well, because this would close that.

Seems like a wasted opportunity actually. The type could accept a function from name to type so you really have a name, that you can even feed into the submodule specialArgs using a different name name, etc.

So before i start factoring this, and make sure the checks pass.
Do you mean something like this?
    # Interface
    # elemTypeFn :: String -> OptionType
    namedAttrsOf = attrName: elemTypeFn: mkOptionType rec {
     # ...
    }

 # Complex Usage with submodule taking a function itself
 # I choose username as concrete name name here.
 # Couldn't make up a nice name for `namedAttr` but `name` is probably bad, since it already used in types.submodule in a different way.
   mkOption {
      type = namedAttrsOf "username" (namedAttr: submoduleWith {
          specialArgs = {
             inherit namedAttr;
          };
          modules = [
           # ... other modules receiving the name
          ];
        }
      );
   };

I was thinking something along the lines of
attrsWith {
  name = "username";
  itemType = submoduleWith { modules = [ <...> ]; };
  # or, perhaps later:
  # itemTypeFunction = name: submoduleWith { modules = f name; specialArgs = g name; };
  # and perhaps later:
  # lazy = true;
};

This is more extensible and will let us cover lazyAttrsOf as well.
Also, instead of attrsWith we could do dict, because a submodule value is also an attrset but very different. (See also the issue and the defintiion of dictionary in https://nix.dev/manual/nix/2.24/development/json-guideline)

@infinisil Just added some little documentation and fixed the missing functor attributes (wrapped, type), that got removed from my previous commit. Nixos manual should be able to build.

https://github.com/NixOS/nixpkgs/pull/344216#discussion_r1781965008 really needs to be added as a test, because those wrapped and type attributes aren't needed and actually break it :sweat_smile:

#344216 (comment) really needs to be added as tests, because those wrapped and type attributes aren't needed and actually break it 😅. lib/tests/modules.sh would be a good entry-point for testing that.

Is it breaking? I ran only the test that you gave me and they where all fine. I'll run the other ones lib/tests/modules.sh  as well. Shouldn't those run with CI because all jobs passed as well?

I mean that in the comment I linked, I showed you two additional test cases that should be added to modules.sh in this PR

@infinisil I am somehow still stuck with the name merging.
Previously attrsOf used:
functor = (defaultFunctor name) // { wrapped = elemType; };
Note that
wrapped = elemType;
Then in defaultTypeMerge we got this pice of code:
...
    else if (f.wrapped != null && f'.wrapped != null) && (wrapped != null)
       then f.type wrapped
    # value types
    else if (f.payload != null && f'.payload != null) && (payload != null)
       then f.type payload

This means if i omit wrapped then f.type  is called with payload and f.binOp is executed to merge the name.
But if wrapped is omited, the manual wont build. (Not sure why and if this is a bug in the manual)
Alternatively if i add wrapped as before - then f.type  is called with elemType and the name is not merged - Because f.wrapped.typeMerge f'.wrapped.functor is executed to merge wrapped and not payload

@infinisil what do you think about this. I have this  questions still in mind:

[ ] Should <name> be configured via parameter  name ? Because in the linked Issue everyone seemed convince that it shouldn't but it seems fine once implemented?
[ ] Should defaultTypeMerge return early?
[ ] Is it a required that the functor still needs wrapped?

Should <name> be configured via parameter  name ?

This should be considered in the context of also having e.g. https://github.com/NixOS/nixpkgs/pull/218812/files, which is a laterally related but independent parameter. This is somewhat of a red herring though, because we have good reasons to prefer a different name name for other reasons.
Picking good names makes a big difference in the UX, as they're used over and over.
"Name" is not great because it can refer to any of:

the default presentation of the placeholder in docs and diagnostics, <name>
the module argument by the name of name
now also: the setting that controls the placeholder
potentially: the setting that controls the attribute name of the module argument

This means we have to often qualify the name of name, which requires more thinking than just having separate terms.
It'd be easier to just say label or placeholder

separate term, no need to decide whether to qualify it when talking or thinking about it
describes the purpose
unambiguous (especially placeholder)

sometimes labels are key-value pairs, like https://github.com/NixOS/nixpkgs/issues/305741

Compare:

"name" is the default name
"name" is the default label
"name" is the default placeholder

Compare:

an attribute set type constructor has a name for documentation

carries too much weight, implying that they are unique in some imaginary namespace, which is not the case

an attribute set type constructor has a label for documentation

similar to "name" but weaker implication

an attribute set type constructor has a placeholder for documentation

I think placeholder is the best out of these three.
As for the other two points, I think @infinisil has more experience with type merging.

@infinisil If we remove wrapped from the functor then the following error hapens when building the nixos manual:
error:
       … while calling anonymous lambda

         at /nix/store/n9qgjvaqaipfi7y5skc0r1l27y487axh-nixos/lib/eval-cacheable-options.nix:1:1:

            1| { libPath
             | ^
            2| , pkgsLibPath

       … from call site

         at /nix/store/n9qgjvaqaipfi7y5skc0r1l27y487axh-nixos/lib/make-options-doc/default.nix:130:17:

          129|   filteredOpts = lib.filter (opt: opt.visible && !opt.internal) transformedOpts;
          130|   optionsList = lib.flip map filteredOpts
             |                 ^
          131|    (opt: opt

       … while calling 'flip'

         at /nix/store/cr32l4lf8qjmx7vnfkfqqgxx7dfdvx7k-lib/trivial.nix:317:16:

          316|   */
          317|   flip = f: a: b: f b a;
             |                ^
          318|

       … from call site

         at /nix/store/n9qgjvaqaipfi7y5skc0r1l27y487axh-nixos/lib/make-options-doc/default.nix:127:13:

          126| let
          127|   rawOpts = lib.optionAttrSetToDocList options;
             |             ^
          128|   transformedOpts = map transformOptions rawOpts;

       … while calling 'optionAttrSetToDocList''

         at /nix/store/cr32l4lf8qjmx7vnfkfqqgxx7dfdvx7k-lib/options.nix:320:32:

          319|
          320|   optionAttrSetToDocList' = _: options:
             |                                ^
          321|     concatMap (opt:

       … while calling anonymous lambda

         at /nix/store/cr32l4lf8qjmx7vnfkfqqgxx7dfdvx7k-lib/options.nix:321:16:

          320|   optionAttrSetToDocList' = _: options:
          321|     concatMap (opt:
             |                ^
          322|       let

       … from call site

         at /nix/store/cr32l4lf8qjmx7vnfkfqqgxx7dfdvx7k-lib/options.nix:358:26:

          357|         # builtins.trace opt.loc
          358|         [ docOption ] ++ optionals subOptionsVisible subOptions) (collect isOption options);
             |                          ^
          359|

       … while calling 'optionals'

         at /nix/store/cr32l4lf8qjmx7vnfkfqqgxx7dfdvx7k-lib/lists.nix:820:5:

          819|     cond:
          820|     elems: if cond then elems else [];
             |     ^
          821|

       … from call site

         at /nix/store/cr32l4lf8qjmx7vnfkfqqgxx7dfdvx7k-lib/options.nix:353:31:

          352|           let ss = opt.type.getSubOptions opt.loc;
          353|           in if ss != {} then optionAttrSetToDocList' opt.loc ss else [];
             |                               ^
          354|         subOptionsVisible = docOption.visible && opt.visible or null != "shallow";

       … while calling 'optionAttrSetToDocList''

         at /nix/store/cr32l4lf8qjmx7vnfkfqqgxx7dfdvx7k-lib/options.nix:320:32:

          319|
          320|   optionAttrSetToDocList' = _: options:
             |                                ^
          321|     concatMap (opt:

       … from call site

         at /nix/store/cr32l4lf8qjmx7vnfkfqqgxx7dfdvx7k-lib/options.nix:358:67:

          357|         # builtins.trace opt.loc
          358|         [ docOption ] ++ optionals subOptionsVisible subOptions) (collect isOption options);
             |                                                                   ^
          359|

       … while calling 'collect'

         at /nix/store/cr32l4lf8qjmx7vnfkfqqgxx7dfdvx7k-lib/attrsets.nix:867:5:

          866|     pred:
          867|     attrs:
             |     ^
          868|     if pred attrs then

       … while calling 'collect'

         at /nix/store/cr32l4lf8qjmx7vnfkfqqgxx7dfdvx7k-lib/attrsets.nix:867:5:

          866|     pred:
          867|     attrs:
             |     ^
          868|     if pred attrs then

       … from call site

         at /nix/store/cr32l4lf8qjmx7vnfkfqqgxx7dfdvx7k-lib/attrsets.nix:868:8:

          867|     attrs:
          868|     if pred attrs then
             |        ^
          869|       [ attrs ]

       … while calling 'isType'

         at /nix/store/cr32l4lf8qjmx7vnfkfqqgxx7dfdvx7k-lib/types.nix:76:18:

           75| rec {
           76|   isType = type: x: (x._type or "") == type;
             |                  ^
           77|

       … from call site

         at /nix/store/cr32l4lf8qjmx7vnfkfqqgxx7dfdvx7k-lib/types.nix:931:32:

          930|             # is just to avoid conflicts with potential options from the submodule
          931|             _freeformOptions = freeformType.getSubOptions prefix;
             |                                ^
          932|           };

       … while calling 'getSubOptions'

         at /nix/store/cr32l4lf8qjmx7vnfkfqqgxx7dfdvx7k-lib/types.nix:618:23:

          617|       emptyValue = { value = {}; };
          618|       getSubOptions = prefix: elemType.getSubOptions (prefix ++ ["<${placeholder}>"]);
             |                       ^
          619|       getSubModules = elemType.getSubModules;

       error: value is null while a set was expected
Cacheable portion of option doc build failed.
Usually this means that an option attribute that ends up in documentation (eg `default` or `description`) depends on the restricted module arguments `config` or `pkgs`.

Rebuild your configuration with `--show-trace` to find the offending location. Remove the references to restricted arguments (eg by escaping their antiquotations or adding a `defaultText`) or disable the sandboxed build for the failing module by setting `meta.buildDocsInSandbox = false`.

error: builder for '/nix/store/0w0qq6927cvy20qmlzdr5ydsxr91f42a-lazy-options.json.drv' failed with exit code 1;
       last 20 log lines:
       >           930|             # is just to avoid conflicts with potential options from the submodule
       >           931|             _freeformOptions = freeformType.getSubOptions prefix;
       >              |                                ^
       >           932|           };
       >
       >        … while calling 'getSubOptions'
       >
       >          at /nix/store/cr32l4lf8qjmx7vnfkfqqgxx7dfdvx7k-lib/types.nix:618:23:
       >
       >           617|       emptyValue = { value = {}; };
       >           618|       getSubOptions = prefix: elemType.getSubOptions (prefix ++ ["<${placeholder}>"]);
       >              |                       ^
       >           619|       getSubModules = elemType.getSubModules;
       >
       >        error: value is null while a set was expected
       > Cacheable portion of option doc build failed.
       > Usually this means that an option attribute that ends up in documentation (eg `default` or `description`) depends on the restricted module arguments `config` or `pkgs`.
       >
       > Rebuild your configuration with `--show-trace` to find the offending location. Remove the references to restricted arguments (eg by escaping their antiquotations or adding a `defaultText`) or disable the sandboxed build for the failing module by setting `meta.buildDocsInSandbox = false`.
       >
       For full logs, run 'nix log /nix/store/0w0qq6927cvy20qmlzdr5ydsxr91f42a-lazy-options.json.drv'.
error: 1 dependencies of derivation '/nix/store/bqpgk6sy8f9kbx385hhbjpl4c3ylfnsj-options.json.drv' failed to build
error: 1 dependencies of derivation '/nix/store/rajbr0l6pky8s4rmc61g6i0fqdphkv2f-nixos-manual-html.drv' failed to build

Could we perhaps merge the changes that are mere refactors first? Unfortunately the first commit already introduces a new feature.
Just having a very basic attrsWith { lazy? } merged would simplify the diff and unblock #351888.

@roberth i just looked if we could split out the name part. But the problem with payload and wrapped would still exist with "only" lazy. Because this is something that requires payload for merging as well.
So we dont really save that much only one type merge operation.

If we merge this one before it solves the issue with wrapped and payload.
https://github.com/NixOS/nixpkgs/pull/350906
Which would clean those changes

@infinisil I just found why omitting functor.wrapped breaks the manual:
https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/services/mail/public-inbox.nix#L10
Seems like this line was added in https://github.com/NixOS/nixpkgs/pull/104457
And unfortunately depends on the internal structure of the attrsOf type.

@roberth @infinisil #354738 introduced attrsWith itself.
I'll rebase this; So the scope is to only allow customizing the placeholder itself.

Cool, thank you!


[WFLY-7040] Elytron HTTP mechanisms tests

https://issues.jboss.org/browse/WFLY-7040
Green Digest-SHA-256 test require https://github.com/wildfly-security/wildfly-elytron/pull/1072 - upgrade to wildfly-elytron 1.2.0.Beta12 (after it will be tagged)

@stuartwdouglas Can you please check this one when you get back?

Some of new tests are duplicity to existing ones running with -Delytron flag. What is advantage to have them separate? Is this strategy general recommendation?

@mchoma yes, as discussed with @darranl, we need it covered in -DallTests:

I don't want us to be relying on the -Delytron tests as that is not executed when we run a -DallTests so we actually end up with a large part of essential testing missing.
Relying on CI is not free so to get test results it either takes longer to test at the time of tagging as now multiple test runs are needed or we risk tagging and then the tag needing to be recreated due to a CI failure,
IMO we should consider -Delytron as supplementary so we need tests in the elytron module so we have something we can add to as we add new features
Relying on -Delytron the next issue is when we want to modify tests for Elytron bugs and features that are not compatible with PicketBox - we are not going to be fixing PicketBox even if PicketBox specific issues are found unless they are serious enough.

(But as I has checked it, I admit that covered cases are (except of DIGEST-SHA256) already covered by -Delytron )

@hkalina ping me when you are happy with your changes and I'll remove the fixme label

@kabir  I am ok with changes now.
@olukas Are you ok with that? (especially the tearDown?)

@olukas fixed

Fixed problem with paths on Windows, but the ConstantPermissionMapperTestCase fails when SPNEGO setup tasks fails before - will troubleshoot yet (but regarding to shaky history not related to this PR).

Retest this please


Max/min Problem in C++

Have you read the Contributing Guidelines on Pull Requests?
Yes
Description
You must create an array of length K from elements of arr such that its unfairness is minimized.
Checklist

[Yes] I've read the contribution guidelines.
[Yes] I've checked the issue list before deciding what to submit.
[Yes] I've edited the README.md and link to my code.

Related Issues or Pull Requests
#6230

@ankitaggarwal23 I have done the changes

You have like 17 files changed. What i would recommend is start with a fresh branch whenever you are working on a new issue/pr


CI Image is deprecated

The image we have selected to use in CircleCI has been deprecated in line with https://discuss.circleci.com/t/linux-image-deprecations-and-eol-for-2024/50177
We should update the image.

Once the image is updated, we see different behaviour in Linux AMD64 and Linux ARM64.
On amd64
$ docker run --rm -it --memory=512Mb ubuntu:22.04 bash
# root@95e8a477bb2c:/# cat /sys/fs/cgroup/cpu.max
# max 100000

On arm64
$ docker run --rm -it --memory=512Mb ubuntu:22.04 bash
# root@f5bdb9945390:/# cat /sys/fs/cgroup/cpu.max
# cat: /sys/fs/cgroup/cpu.max: No such file or directory

This is currently pending a response from CircleCI support.


Syntax error on flowtype in react native module

Do you want to request a feature or report a bug?
I want to report a bug
What is the current behavior?
`  FAIL  components/presenters/appointments/AppointmentDebrief.native.test.js
● Test suite failed to run
/home/main/Documents/veyo-frontend/commons/node_modules/react-native/Libraries/BatchedBridge/BatchedBridgedModules/NativeModules.js:17
function normalizePrefix(moduleName: string): string {
                                   ^
SyntaxError: Unexpected token :

  at transformAndBuildScript (node_modules/jest-runtime/build/transform.js:284:10)

`
If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal repository on GitHub that we can npm install and npm test.
create a npm module in which you put some component.native.js code.
create an unit test named component.native.test.js code
use this jest configuration
{   "testRegex": "(\\.(?:native|android|ios)\\.test\\.js)$",   "testPathIgnorePatterns": [     "\\.web\\.test\\.js"   ],   "preprocessorIgnorePatterns": [     "/node_modules(?!\\/(redux\\-request\\-state|redux\\-data\\-fetching))/"   ],   "scriptPreprocessor": "<rootDir>/node_modules/react-native/jestSupport/preprocessor.js",   "setupFiles": ["<rootDir>/node_modules/react-native/jestSupport/env.js"],   "preset": "jest-react-native" } use the babelrc{ "presets": ["react-native"] }
What is the expected behavior?
to not throw a syntax error
Run Jest again with --debug and provide the full configuration it prints. Please mention your node and npm version and operating system.
`jest version = 16.0.1
test framework = jasmine2
config = {
"haste": {
"defaultPlatform": "ios",
"platforms": [
"android",
"ios"
],
"providesModuleNodeModules": [
"react",
"react-native"
]
},
"moduleNameMapper": [
[
"^image![a-zA-Z0-9$-]+$",
"GlobalImageStub"
],
[
"^[./a-zA-Z0-9$-]+\.(bmp|gif|jpg|jpeg|png|psd|svg|webp)$",
"RelativeImageStub"
]
],
"modulePathIgnorePatterns": [
"/home/main/Documents/veyo-frontend/commons/node_modules/react-native/Libraries/react-native/",
"/home/main/Documents/veyo-frontend/commons/node_modules/react-native/packager/"
],
"preprocessorIgnorePatterns": [
"/node_modules(?!\/(redux\-request\-state|redux\-data\-fetching))/"
],
"setupFiles": [
"/home/main/Documents/veyo-frontend/commons/node_modules/jest-react-native/build/setup.js",
"/home/main/Documents/veyo-frontend/commons/node_modules/react-native/jestSupport/env.js"
],
"testEnvironment": "/home/main/Documents/veyo-frontend/commons/node_modules/jest-environment-node/build/index.js",
"testRegex": "(\.(?:native|android|ios)\.test\.js)$",
"testPathIgnorePatterns": [
"\.web\.test\.js"
],
"scriptPreprocessor": "/home/main/Documents/veyo-frontend/commons/node_modules/react-native/jestSupport/preprocessor.js",
"preset": "jest-react-native",
"rootDir": "/home/main/Documents/veyo-frontend/commons",
"name": "-home-main-Documents-veyo-frontend-commons",
"testRunner": "/home/main/Documents/veyo-frontend/commons/node_modules/jest-jasmine2/build/index.js",
"automock": false,
"bail": false,
"browser": false,
"cacheDirectory": "/tmp/jest",
"clearMocks": false,
"coveragePathIgnorePatterns": [
"/node_modules/"
],
"coverageReporters": [
"json",
"text",
"lcov",
"clover"
],
"globals": {},
"mocksPattern": "mocks",
"moduleDirectories": [
"node_modules"
],
"moduleFileExtensions": [
"js",
"json",
"jsx",
"node"
],
"noStackTrace": false,
"notify": false,
"resetModules": false,
"testPathDirs": [
"/home/main/Documents/veyo-frontend/commons"
],
"testURL": "about:blank",
"timers": "real",
"useStderr": false,
"verbose": null,
"watch": false,
"cache": false,
"watchman": true,
"testcheckOptions": {
"times": 100,
"maxSize": 200
}
}
jest-haste-map: duplicate manual mock found:
Module name: nullthrows
Dupicate Mock path: /home/main/Documents/veyo-frontend/commons/node_modules/react-native/node_modules/fbjs/lib/mocks/nullthrows.js
This warning is caused by two manual mock files with the same file name.
Jest will use the mock file found in:
/home/main/Documents/veyo-frontend/commons/node_modules/react-native/node_modules/fbjs/lib/mocks/nullthrows.js
Please delete one of the following two files:
/home/main/Documents/veyo-frontend/commons/node_modules/react/node_modules/fbjs/lib/mocks/nullthrows.js
/home/main/Documents/veyo-frontend/commons/node_modules/react-native/node_modules/fbjs/lib/mocks/nullthrows.js
jest-haste-map: duplicate manual mock found:
Module name: fetchWithRetries
Dupicate Mock path: /home/main/Documents/veyo-frontend/commons/node_modules/react-native/node_modules/fbjs/lib/mocks/fetchWithRetries.js
This warning is caused by two manual mock files with the same file name.
Jest will use the mock file found in:
/home/main/Documents/veyo-frontend/commons/node_modules/react-native/node_modules/fbjs/lib/mocks/fetchWithRetries.js
Please delete one of the following two files:
/home/main/Documents/veyo-frontend/commons/node_modules/react/node_modules/fbjs/lib/mocks/fetchWithRetries.js
/home/main/Documents/veyo-frontend/commons/node_modules/react-native/node_modules/fbjs/lib/mocks/fetchWithRetries.js
jest-haste-map: duplicate manual mock found:
Module name: base62
Dupicate Mock path: /home/main/Documents/veyo-frontend/commons/node_modules/react-native/node_modules/fbjs/lib/mocks/base62.js
This warning is caused by two manual mock files with the same file name.
Jest will use the mock file found in:
/home/main/Documents/veyo-frontend/commons/node_modules/react-native/node_modules/fbjs/lib/mocks/base62.js
Please delete one of the following two files:
/home/main/Documents/veyo-frontend/commons/node_modules/react/node_modules/fbjs/lib/mocks/base62.js
/home/main/Documents/veyo-frontend/commons/node_modules/react-native/node_modules/fbjs/lib/mocks/base62.js
jest-haste-map: duplicate manual mock found:
Module name: crc32
Dupicate Mock path: /home/main/Documents/veyo-frontend/commons/node_modules/react-native/node_modules/fbjs/lib/mocks/crc32.js
This warning is caused by two manual mock files with the same file name.
Jest will use the mock file found in:
/home/main/Documents/veyo-frontend/commons/node_modules/react-native/node_modules/fbjs/lib/mocks/crc32.js
Please delete one of the following two files:
/home/main/Documents/veyo-frontend/commons/node_modules/react/node_modules/fbjs/lib/mocks/crc32.js
/home/main/Documents/veyo-frontend/commons/node_modules/react-native/node_modules/fbjs/lib/mocks/crc32.js
jest-haste-map: duplicate manual mock found:
Module name: fetch
Dupicate Mock path: /home/main/Documents/veyo-frontend/commons/node_modules/react-native/node_modules/fbjs/lib/mocks/fetch.js
This warning is caused by two manual mock files with the same file name.
Jest will use the mock file found in:
/home/main/Documents/veyo-frontend/commons/node_modules/react-native/node_modules/fbjs/lib/mocks/fetch.js
Please delete one of the following two files:
/home/main/Documents/veyo-frontend/commons/node_modules/react/node_modules/fbjs/lib/mocks/fetch.js
/home/main/Documents/veyo-frontend/commons/node_modules/react-native/node_modules/fbjs/lib/mocks/fetch.js
jest-haste-map: duplicate manual mock found:
Module name: ErrorUtils
Dupicate Mock path: /home/main/Documents/veyo-frontend/commons/node_modules/react-native/node_modules/fbjs/lib/mocks/ErrorUtils.js
This warning is caused by two manual mock files with the same file name.
Jest will use the mock file found in:
/home/main/Documents/veyo-frontend/commons/node_modules/react-native/node_modules/fbjs/lib/mocks/ErrorUtils.js
Please delete one of the following two files:
/home/main/Documents/veyo-frontend/commons/node_modules/react/node_modules/fbjs/lib/mocks/ErrorUtils.js
/home/main/Documents/veyo-frontend/commons/node_modules/react-native/node_modules/fbjs/lib/mocks/ErrorUtils.js
jest-haste-map: duplicate manual mock found:
Module name: ErrorUtils
Dupicate Mock path: /home/main/Documents/veyo-frontend/commons/node_modules/react-native/Libraries/Utilities/mocks/ErrorUtils.js
This warning is caused by two manual mock files with the same file name.
Jest will use the mock file found in:
/home/main/Documents/veyo-frontend/commons/node_modules/react-native/Libraries/Utilities/mocks/ErrorUtils.js
Please delete one of the following two files:
/home/main/Documents/veyo-frontend/commons/node_modules/react-native/node_modules/fbjs/lib/mocks/ErrorUtils.js
/home/main/Documents/veyo-frontend/commons/node_modules/react-native/Libraries/Utilities/mocks/ErrorUtils.js
jest-haste-map: @providesModule naming collision:
Duplicate module name: fbjs
Paths: /home/main/Documents/veyo-frontend/commons/node_modules/react-native/node_modules/fbjs/package.json collides with /home/main/Documents/veyo-frontend/commons/node_modules/react/node_modules/fbjs/package.json
This warning is caused by a @providesModule declaration with the same name across two different files.
jest-haste-map: @providesModule naming collision:
Duplicate module name: immutable
Paths: /home/main/Documents/veyo-frontend/commons/node_modules/react-native/node_modules/immutable/package.json collides with /home/main/Documents/veyo-frontend/commons/node_modules/react-native/node_modules/fbjs/node_modules/immutable/package.json
This warning is caused by a @providesModule declaration with the same name across two different files.
jest-haste-map: @providesModule naming collision:
Duplicate module name: core-js
Paths: /home/main/Documents/veyo-frontend/commons/node_modules/react/node_modules/core-js/package.json collides with /home/main/Documents/veyo-frontend/commons/node_modules/react-native/node_modules/fbjs/node_modules/core-js/package.json
This warning is caused by a @providesModule declaration with the same name across two different files.
FAIL  components/presenters/appointments/AppointmentDebrief.native.test.js
● Test suite failed to run
/home/main/Documents/veyo-frontend/commons/node_modules/react-native/Libraries/BatchedBridge/BatchedBridgedModules/NativeModules.js:17
function normalizePrefix(moduleName: string): string {
                                   ^
SyntaxError: Unexpected token :

  at transformAndBuildScript (node_modules/jest-runtime/build/transform.js:284:10)

Test Suites: 1 failed, 1 total
Tests:       0 total
Snapshots:   0 total
Time:        0.03s
Ran all test suites.
`

Did you have setting jest config on package.json ?
as this https://github.com/g0v/react-native-sample/blob/master/package.json#L20

preprocessorIgnorePatterns overwrites the configuration from the preset. You'll need to exclude react-native from it, as well:
"preprocessorIgnorePatterns": [ "/node_modules(?!\\/(react-native|redux-request-state|redux-data-fetching))/" ]

Do you have a .babelrc file with the following preset?
{
  "presets": ["react-native"]
}


Project Feedback!

<img alt="+1" title="+1" src="/images/emoji/unicode/1f44d.png"}" style="vertical-align:middle" width="20" height="20" /> Congrats on completing the third project!
I have provided a detailed Project 3 Feedback Guide here which covers the most common issues with this submitted project. Read through the feedback guide point-by-point to determine how you could improve your submission.
Let us know if you have any other thoughts or questions about this assignment. Hopefully by now you feel pretty comfortable with all the major pieces to basic Android apps (Views, Controllers, Models, Authentication, API Communication, Preferences, ActionBar, et al) and see how they all fit together.

👍 Congrats on completing the fourth project!
I have provided a detailed Project 4 Feedback Guide here which covers the most common issues with this submitted project. Read through the feedback guide point-by-point to determine how you could improve your submission.
Let us know if you have any other thoughts or questions about this assignment. Hopefully by now you feel pretty comfortable with all the major pieces to basic Android apps (Views, Controllers, Models, Authentication, API Communication, Preferences, ActionBar, et al) and see how they all fit together.
/cc @codepathreview


ReferenceError: document is not defined

I updated from 5.1.0 to 6.0.0 and am now getting the error
ReferenceError: document is not defined

during server side rendering in a NextJS app.

Screenfull is meant to run in a browser. The previous version did indeed polyfill document, but I no longer think that's the correct place to handle it. If you want to run a browser-based library o the server, it's up to you to polyfill the required globals.

I updated from 5.1.0 to 6.0.0 and am now getting the error
ReferenceError: document is not defined

during server side rendering in a NextJS app.

Dear @RyanWarner
I'm meet this problem,
Can you help me fix this problem, Please.

@tranmanhhung I solved the problem by downgrading back to version 5.1.0.

@tranmanhhung I solved the problem by downgrading back to version 5.1.0.

I just found solved this problem by..
const nativeAPI = (() => {
const unprefixedMethods = methodMap[0];
const returnValue = {};
for (const methodList of methodMap) {
    const exitFullscreenMethod = methodList?.[1];
    if (typeof window === 'object') {
        if (exitFullscreenMethod in document) {
            for (const [index, method] of methodList.entries()) {
                returnValue[unprefixedMethods[index]] = method;
            }

            return returnValue;
        }
    }
}
return false;

})();
line 56 added if (typeof window === 'object') {


Add USB Prorotyping Mode

Emulator compilation is broken, and I'm not going to merge until that's fixed. I think you'll probably preprocess out some USB stuff rather than stubbing it.
In general I'd like to see more comments (doxygen and inline), and enums (rather than magic numbers) just to have a better idea of what's going on. I also reserve the right to astyle or clang-format this at some point.

What are the odds this works on windows too?

This will work on Windows, but I don't plan to add support for it until weekend after this coming one.

Emulator compilation is broken, and I'm not going to merge until that's fixed. I think you'll probably preprocess out some USB stuff rather than stubbing it.
In general I'd like to see more comments (doxygen and inline), and enums (rather than magic numbers) just to have a better idea of what's going on. I also reserve the right to astyle or clang-format this at some point.

Can do.

Emulator compilation is still broken:
main/advanced_usb_control.c:5:10: fatal error: esp_partition.h: No such file or directory
    5 | #include "esp_partition.h"
      |          ^~~~~~~~~~~~~~~~~

Would it be easier for me to merge, then fix it myself?

Emulator compilation is still broken:
main/advanced_usb_control.c:5:10: fatal error: esp_partition.h: No such file or directory
    5 | #include "esp_partition.h"
      |          ^~~~~~~~~~~~~~~~~

Would it be easier for me to merge, then fix it myself?

I am not getting this error. Where do you get it?  I'm not including main/advanced_usb_control.c in my build tree for the emu?


Can we use plugins with AnalyticsNode?

I haven't seen much discussion about the AnalyticsNode portion of the repo, but it seems to support sending Analytics events from a server.
Is there any way to use the same plugins functionality that is used by AnalyticsBrowser? If not, is there any way to add middleware to AnalyticsNode?

@chrisradek
Thank you for the quick reply and the thorough response!
Should we also use identifies as normal with AnalyticsNode? I believe the old analytics-node library required a userID to be sent top-level in track events in order to associate events with a user.

AnalyticsNode won't automatically assign the userID to all track events, so you'll still need to set that yourself to associate events with a user.

@chrisradek
Where would that be set? As a top-level property? Or somewhere in the options of a track event? I see the the anonymousId property inside of the "Options" type, but don't see an actual userId in there.

@kylegwalsh Sorry it looks like I was mistaken, you actually can call identify and it will populate the track calls. The risk there is that if you have multiple identify and track calls being invoked concurrently then a track call could have an unexpected userId associated with it. (This is more common when writing a service in node, if you have an application with a single user then it would be fine to rely on that behavior).
It's not currently typed correctly, but you can pass the userId alongside the anonymousId as part of the track Options. We'll ned to update the typings on our end but it will work currently.

@chrisradek
Perfect, the latter method is actually more convenient for me anyway! I will make sure to tag the userId in the options whenever I trigger a track call.
Thanks for your help.


Ability to customise customer check when creating an Invoice

Hello,
I have an application which allows payment for effectively two different products; membership and registration fees. Unfortunately two different Stripe customer IDs were generated when someone bought both of these products. This has now been fixed but older transactions exist that were created in this manner.
Is there a way of customising this condition (I want to further check on the customer's email address)? Or a way of disabling this check? Without this I cannot provide access to invoices for both products.

https://github.com/laravel/cashier/pull/801/files#diff-2e90a7f804b6b4d2e661f5d0bf080075R46

https://github.com/laravel/cashier/blob/1d2c7a701b659a2646dd211a7277eda06f35efbf/src/Invoice.php#L44-L46

Unfortunately not. That would be solved with this issue: https://github.com/laravel/cashier/issues/545

@driesvints: would it be a good idea to warn the user (or at least mention in the docs) that if they call createAsStripeCustomer() and a Stripe Customer ID already exists (auth()->user()->stripe_id) Cashier will silently overwrite what was previously there as per the code below:
https://github.com/laravel/cashier/blob/3c485612cfd22feaa550d99e78ade39f7c57b2f8/src/Billable.php#L741
Now that the new check is in place (if ($owner->stripe_id !== $invoice->customer)) any past invoices (if createAsStripeCustomer() was used) cannot be retrieved as the customer IDs won't match.
If a developer cares about displaying past invoices they should use createOrGetStripeCustomer() instead to re-use an existing Stripe Customer ID should it exist.

Actually, the problem here is that the createAsStripeCustomer method shouldn't be callable when a stripe_id is already set. I'll re-open this and try to fix this by throwing an exception if it's already set.

My only worry with throwing an exception at the point a stripe_id is already set and createAsStripeCustomer() is called relates to the following scenario.
When a customer makes their first ever transaction it is perfectly acceptable for a developer to use createAsStripeCustomer(). However, it's only when a customer makes a subsequent transaction that this new exception will be thrown when an existing stripe_id is found. This means a developer could have code in production that may only throw an exception if a customer makes more than one transaction.
Perhaps advocating the use of createOrGetStripeCustomer() would be a good idea too?

@u01jmg3 in that scenario it's not wanted to constantly call createAsStripeCustomer for every transaction you make as you'll end up with numerous customers in Stripe which can't be traced back to that one user anymore. It's probably a bug in their app then.
I indeed now notice that createOrGetStripeCustomer isn't documented. I'll get that fixed as well.
Thanks for reporting this btw 👍

as you'll end up with numerous customers in Stripe which can't be traced back to that one user anymore

Exactly - not good

I indeed now notice that createOrGetStripeCustomer isn't documented. I'll get that fixed as well.

Perfect 😄


Implement websocket client WsClient

https://curl.se/docs/websocket.html
https://curl.se/libcurl/c/curl_ws_recv.html
https://curl.se/libcurl/c/curl_ws_send.html
https://curl.se/libcurl/c/curl_ws_meta.html

Is anyone working on it? I think I can give it a try this week.

Great!

I've implemented this in the linked PR, however there are some issues which I think are related to libcurl itself, and I've submitted an issue for in the curl repository.


Error "Request Failed, status 403/503" when viewing tasks on iPadOs 16.1.2

Describe the bug
The pending tasks are not rendering in my view when I call the plugin on my iPad. I get an error saying "Request Failed, status" followed by either 403 or 503. This issue has been occurring for me for several weeks.
To Reproduce
Steps to reproduce the behavior:
Call the plugin on an iPad
View the pending tasks
Expected behavior
I expect the pending tasks to render in my view when I call the plugin on my iPad, as they do on my Windows version.
Desktop (please complete the following information):
Plugin version: 1.10.1
iPadOS version: 16.2.2 (occurs on 15 as well)
Obsidian Version: 1.4.1
Additional context
I have already updated my API key and tried the example filters, even the simplest ones, but the issue persists.

Hey! I don't see that error anymore! Apparently it's already fixed :)!


Modify FDSN logic to acknowledge multiple channels

There was a hardcoded assumption in get_inventory_from_df that only one valid value of channel instance should be added to the list of returned channels from a station.
This is not only false in the case of wildcards (e.g. F, Q), it is also false when a channel has been broken into multiple runs.
CAS04 is a good example of this.
See mth5 issue #157, and aurora 277 for more details.
No guarantee this fixes everything, and the logic in get_inventory_from_df should be cleaned up, but at a minimum, I confirm that with the minor change committed at the start of this branch:

[x]  there are no longer any channels with zero filters in the CAS04 channel summary.
[x] The CAS04 processing against reference REV06 produces TFs that are highly similar to those archived in SPUD.

Codecov Report
Patch coverage: 34.64% and project coverage change: -0.37% :warning:

Comparison is base (1d3de90) 58.31% compared to head (afc39f0) 57.95%.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
Additional details and impacted files
@@            Coverage Diff             @@
##               fc     #158      +/-   ##
==========================================
- Coverage   58.31%   57.95%   -0.37%
==========================================
  Files         134      135       +1
  Lines       14207    14398     +191
==========================================
+ Hits         8285     8344      +59
- Misses       5922     6054     +132

Files Changed
Coverage Δ

tests/version_1/test_make_mth5_ofr.py
25.17% <25.17%> (ø)

mth5/clients/fdsn.py
39.70% <47.66%> (+2.03%)
:arrow_up:

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Where is the metrics-macros crate?

The readme says it is hosted in this repo but I cant find it https://crates.io/crates/metrics-macros

Ah, figured it out https://github.com/metrics-rs/metrics/pull/386
Maybe the crate should have a deprecation message.

I pushed a final version -- v0.7.1 -- that updated the README to mark the crate as no longer being used, and explains how to find the corresponding source code (in this repository) for a given version.


Query metadata (size and modified time) on MTP files?

Hi, I have some file syncing code in Rust that currently only works with the local filesystem, but I'd like to add support for MTP devices and your library seems like a great candidate for that.  My syncing code requires traversing to compare directory hierarchies and then copying files based on differences, all of which your library seems to support nicely.  But it also requires getting the file size and modified timestamp of files to decide which need to be transferred and which don't.  On the local filesystem I get this metadata from std::fs::Metadata (https://doc.rust-lang.org/std/fs/struct.Metadata.html).  I don't see a way with your library to get that info from files on a MTP device--or am I missing it?
To be fair, it's not clear to me how to get that information from the underlying API (https://learn.microsoft.com/en-us/windows/win32/wpd_sdk/retrieving-properties-for-a-single-object) but I'm new to MTP so I'm probably missing that too.
Any advice?  Thanks!

Ah, my bad, now I see Object::get_object_properties (https://github.com/daladim/winmtp/blob/master/src/object/mod.rs#L68C109-L68C130), so if I just dig through the resulting IPortableDeviceValues I suspect I'll find what I need.
If that's correct, please feel free to close this issue.  Sorry for the noise.

Ah, my bad, now I see Object::get_object_properties (https://github.com/daladim/winmtp/blob/master/src/object/mod.rs#L68C109-L68C130), so if I just dig through the resulting IPortableDeviceValues I suspect I'll find what I need.
If that's correct, please feel free to close this issue.  Sorry for the noise.

Hello, yes I think get_object_properties is indeed what you're looking for.
Note that this is a very recent addition to this lib, and it has not been released to crates.io yet. I'll try to release version 0.2.2 tonight. In the meantime, you can always use [patches.crates-io] in your Cargo.toml to fetch winmtp from Github

winmtp 0.3.0 has just been released!
You can now easily get file sizes using e.g. https://github.com/daladim/winmtp/blob/4f9b9e83b7616f524883a6fe95f91ada7ae0be6b/tests/file_access.rs#L112
I haven't dug into file dates. I may be wrong, but I suppose you'll have to query an int that happens to be a FILETIME


Allow users to delete their own profile

Right now, people email in to request for their profiles to be deleted. Sometimes there's more back-and-forth required when we have to confirm that they actually indeed own the account.
It would be nice to have a delete-account flow where users can do this themselves (securely too!)

We already have a method for deleting a User, so this will mainly be UI/UX work.


Is there a way to get the bank name from the bank code or IBAN or BIC?

I know the question is a bit out of scope, but is there a way to get the bank name from the IBAN, BIC or bank code?
I see schwifty nicely gets the bank code, but couldn't find an API or module to get the bank name.

>>> from schwifty import BIC
>>> bic = BIC("GENODEM1GLS")
>>> bic.bank_name
'GLS Gemeinschaftsbank'

or
>>> from schwifty import IBAN
>>> iban = IBAN("DE90100400000123456789")
>>> iban.bic.bank_name
'Commerzbank, Filiale Berlin 1'

Wonderful, thanks. Missed that in the documentation, sorry!


feat: Verify function against contract class id in private kernel

Work in progress
Updates the private kernel circuit to verify that the function being executed belongs to the contract class id in the address preimage.

Introduces ContractClassId as a type in noir-protocol-circuits
Updates contract class computation to remove public functions root since they'll be eventually dropped
Updates PrivateCallData input to private kernel to include preimages for address and contract class
Introduces WrappedFieldLike type for codegen so all wrapped field types can be converted from fields
Introduces an immutable simple MerkleTree type in circuits.js so we don't pass raw Fr[]s around
Sets vk_hash to zero in ts-land since it's mocked to zero in noir-land
Replaces pxe contract_tree with a private_functions_tree
Updates noir protocol circuits fixture data and builder

Fixes #4056

Benchmark results
Metrics with a significant change:

note_history_trial_decrypting_time_in_ms (5): 157 (+93%)
note_trial_decrypting_time_in_ms (8): 109 (+154%)
note_trial_decrypting_time_in_ms (32): 91.2 (+116%)

Detailed results
All benchmarks are run on txs on the Benchmarking contract on the repository. Each tx consists of a batch call  to create_note and increment_balance, which guarantees that each tx has a private call, a nested private call, a public call, and a nested public call, as well as an emitted private note, an unencrypted log, and public storage read and write.
This benchmark source data is available in JSON format on S3 here.
Values are compared against data from master at commit 83686598 and shown if the difference exceeds 1%.
L2 block published to L1
Each column represents the number of txs on an L2 block published to L1.

Metric
8 txs
32 txs
128 txs

l1_rollup_calldata_size_in_bytes
45,412
179,524
716,068

l1_rollup_calldata_gas
221,524
865,996
3,446,656

l1_rollup_execution_gas
314,011
983,736
3,666,313

l2_block_processing_time_in_ms
1,160 (-1%)
4,360
17,242

note_successful_decrypting_time_in_ms
299 (-11%)
962 (-6%)
3,602 (-5%)

note_trial_decrypting_time_in_ms
:warning: 109 (+154%)
:warning: 91.2 (+116%)
145 (+1%)

l2_block_building_time_in_ms
18,268 (-1%)
72,012
289,127

l2_block_rollup_simulation_time_in_ms
13,030 (-1%)
51,724
208,061

l2_block_public_tx_process_time_in_ms
5,144 (-1%)
20,212 (-1%)
80,826 (-1%)

L2 chain processing
Each column represents the number of blocks on the L2 chain where each block has 16 txs.

Metric
5 blocks
10 blocks

node_history_sync_time_in_ms
13,890 (+1%)
26,323 (-1%)

note_history_successful_decrypting_time_in_ms
2,602 (+11%)
4,661

note_history_trial_decrypting_time_in_ms
:warning: 157 (+93%)
161 (-7%)

node_database_size_in_bytes
17,322,064
32,964,688 (-1%)

pxe_database_size_in_bytes
29,923
59,478

Circuits stats
Stats on running time and I/O sizes collected for every circuit run across all benchmarks.

Circuit
circuit_simulation_time_in_ms
circuit_input_size_in_bytes
circuit_output_size_in_bytes

private-kernel-init
309 (-3%)
44,481 (-1%)
26,193

private-kernel-ordering
195 (+1%)
44,513
14,929

base-rollup
1,407
128,970
881

root-rollup
82.7
4,088
677

private-kernel-inner
412 (-3%)
71,640 (-1%)
26,193

public-kernel-private-input
243
32,615
26,193

public-kernel-non-first-iteration
242
32,657
26,193

merge-rollup
7.65 (-6%)
2,608
881

Tree insertion stats
The duration to insert a fixed batch of leaves into each tree type.

Metric
0 deployed contracts
1 deployed contracts

tx_size_in_bytes
15,575
37,972

Transaction processing duration by data writes.

Metric
0 new commitments
1 new commitments

tx_pxe_processing_time_ms
554 (-5%)
1,375

Metric
1 public data writes

tx_sequencer_processing_time_ms
631


biggerMiterBox transform problem

mandibleCurve has a crescent direction from the starting point to the endPoint.
If mandiblePlanes are created in decrescent order the biggerMiterBoxes won't transform well (like in the picture)

Workaround till the bug is fixed: create the mandible planes in crecent order.
Follow the picture.

This was solved here https://github.com/SlicerIGT/SlicerBoneReconstructionPlanner/commit/17270cbde815910a60b9c74a52335220215f252d


doc3d dataset 403 forbidden

Hello, thank you for contributing such an excellent work. Excuse me, I have already applied for a username and password of doc3d dataset. I tried to download the doc3d dataset, but it kept reporting 403 forbidden when downloading it. Is there a problem with the server? When will the server be repaired? thanks

我也遇到相同问题

Sorry I just realized this issue. Unfortunately the data server in our lab has been down indefinitely...I believe many guys in the lab are also affected...I am not sure if the server could be repaired...If I get an update from the server admin, I will post here.
Sorry for the inconvenience :(


Failing multi-cluster metric test

I get an error when running the tests/test_metrics.py::test_cluster_metric() test on Python 3.8.13:
_________________________________________________ test_cluster_metric _________________________________________________

evaluator = <synthgauge.evaluate.Evaluator object at 0x000002F4544ACB80>

    def test_cluster_metric(evaluator):
        """Check clustering metric."""
        evaluator.add_metric('multi_clustered_MSD', random_state=24)
        evaluator.evaluate()
>       assert evaluator.metric_results == pytest.approx({
            'multi_clustered_MSD': 0.00291989251686431
        })
E       AssertionError: assert {'multi_clust...5805624295994} == approx({'mult...31 ± 2.9e-09})
E         comparison failed. Mismatched elements: 1 / 1:
E         Max absolute difference: 0.0002763119544347105
E         Max relative difference: 0.09463086495096182
E         Index               | Obtained              | Expected
E         multi_clustered_MSD | 0.0026435805624295994 | 0.00291989251686431 ± 2.9e-09

tests\test_metrics.py:83: AssertionError

It's off by about 0.0003 which isn't the end of the world.
@TomJamesW doesn't see this for a Python 3.6 install but can with Python 3.8. We should check the observed values are the same between our environments.
I get consistent results (0.00264...) which suggests that there isn't some PRNG bleeding and probably has to do with some internal changes to another package. One of:

pytest: unlikely but should check changes to pytest.approx() parameter defaults
scikit-learn: this is where the heavy lifting is done for clustering and data generation

These packages are also the only ones with major version differences between Tom's 3.6 env and my 3.8 env.

My current conda environment is set up as follows:
name: sg-dev
channels:
  <the usual artifactory channels>
dependencies:
  - asttokens=2.0.5
  - backcall=0.2.0
  - bzip2=1.0.8
  - ca-certificates=2<IP_ADDRESS>
  - colorama=0.4.4
  - debugpy=1.6.0
  - decorator=5.1.1
  - executing=0.8.3
  - ipykernel=6.13.1
  - ipython=8.4.0
  - jedi=0.18.1
  - jupyter_client=6.1.12
  - jupyter_core=4.10.0
  - libffi=3.4.2
  - libzlib=1.2.12
  - matplotlib-inline=0.1.3
  - nest-asyncio=1.5.5
  - openssl=3.0.3
  - packaging=21.3
  - parso=0.8.3
  - pickleshare=0.7.5
  - pip=22.1.2
  - prompt-toolkit=3.0.29
  - psutil=5.9.1
  - pure_eval=0.2.2
  - pygments=2.12.0
  - pyparsing=3.0.9
  - python=3.8.13
  - python-dateutil=2.8.2
  - python_abi=3.8
  - pywin32=303
  - pyzmq=22.3.0
  - setuptools=62.3.2
  - six=1.16.0
  - sqlite=3.38.5
  - stack_data=0.2.0
  - tk=8.6.12
  - tornado=6.1
  - traitlets=5.2.2.post1
  - ucrt=10.0.20348.0
  - vc=14.2
  - vs2015_runtime=14.29.30037
  - wcwidth=0.2.5
  - wheel=0.37.1
  - xz=5.2.5
  - pip:
    - alabaster==0.7.12
    - astroid==2.11.5
    - atomicwrites==1.4.0
    - attrs==21.4.0
    - babel==2.10.1
    - certifi==2<IP_ADDRESS>
    - cfgv==3.3.1
    - charset-normalizer==2.0.12
    - coverage==6.4.1
    - cycler==0.11.0
    - distlib==0.3.4
    - docutils==0.17.1
    - filelock==3.7.1
    - flake8==4.0.1
    - fonttools==4.33.3
    - identify==2.5.1
    - idna==3.3
    - imagesize==1.3.0
    - iniconfig==1.1.1
    - jinja2==3.1.2
    - joblib==1.1.0
    - kiwisolver==1.4.2
    - kmodes==0.12.1
    - lazy-object-proxy==1.7.1
    - markupsafe==2.1.1
    - matplotlib==3.5.2
    - mccabe==0.6.1
    - nodeenv==1.6.0
    - numpy==1.22.4
    - pandas==1.4.2
    - pillow==9.1.1
    - platformdirs==2.5.2
    - pluggy==1.0.0
    - pre-commit==2.19.0
    - py==1.11.0
    - pycodestyle==2.8.0
    - pyflakes==2.4.0
    - pyqt5==5.15.6
    - pyqt5-qt5==5.15.2
    - pyqt5-sip==12.10.1
    - pytest==7.1.2
    - pytest-cov==3.0.0
    - pytz==2022.1
    - pyyaml==6.0
    - requests==2.27.1
    - scikit-learn==1.1.1
    - scipy==1.8.1
    - seaborn==0.11.0
    - snowballstemmer==2.2.0
    - sphinx==4.3.2
    - sphinx-autoapi==1.8.4
    - sphinx-rtd-theme==1.0.0
    - sphinxcontrib-applehelp==1.0.2
    - sphinxcontrib-devhelp==1.0.2
    - sphinxcontrib-htmlhelp==2.0.0
    - sphinxcontrib-jsmath==1.0.1
    - sphinxcontrib-qthelp==1.0.3
    - sphinxcontrib-serializinghtml==1.1.5
    - threadpoolctl==3.1.0
    - toml==0.10.2
    - tomli==2.0.1
    - typing-extensions==4.2.0
    - unidecode==1.3.4
    - urllib3==1.26.9
    - virtualenv==20.14.1
    - wrapt==1.14.1

For future reference, I got this output into an environment file with the following command in PowerShell:
PS > conda env export -n sg-dev --no-builds | findstr -v "prefix" > environment.yml

I looked over the release notes and comparisons between our environments for scikit-learn, numpy,  and kmodes. The only thing that really stands out is the switch in default algorithm for sklearn.cluster.KMeans.
In scikit-learn~=0, the algorithm of choice was Elkan's, and now it's Lloyd's, which is more efficient. According to the release notes, they should converge to identical results.
Rather than toying with the maximum iterations and worrying about rounding errors, we can fix algorithm="elkan" in the KMeans instantiation.
I'll open a PR now.


[TiCDC] refine the log print about redo apply command

Is your feature request related to a problem?
when I run redo apply command, I saw this logs printed:
[2022/07/01 18:58:57.445 +08:00] [INFO] [helper.go:52] ["init log"] [file=] [level=info]
[2022/07/01 18:58:57.451 +08:00] [INFO] [redo.go:104] ["apply redo log starts"] [checkpointTs=434286918047891468] [resolvedTs=434286922307731459]
[2022/07/01 18:58:57.451 +08:00] [INFO] [sink.go:171] ["succeed to parse parameter from sink uri"] [protocol=] [txnAtomicity=table]
[2022/07/01 18:58:57.479 +08:00] [INFO] [mysql_params.go:315] ["sink uri is configured"] [dsn="normal:******@tcp(<IP_ADDRESS>:3306)/?interpolateParams=true&multiStatements=true&allow_auto_random_explicit_insert=1&charset=utf8mb4&readTimeout=2m&sql_mode=%22NO_ENGINE_SUBSTITUTION%2CIGNORE_SPACE%2CONLY_FULL_GROUP_BY%2CALLOW_INVALID_DATES%2CNO_AUTO_VALUE_ON_ZERO%22&tidb_placement_mode=%22ignore%22&tidb_txn_mode=optimistic&time_zone=%22UTC%22&timeout=2m&transaction_isolation=%22READ-COMMITTED%22&writeTimeout=2m"]
[2022/07/01 18:58:57.481 +08:00] [INFO] [mysql.go:167] ["Start mysql sink"]
[2022/07/01 18:58:58.483 +08:00] [INFO] [mysql.go:519] ["clean up table max resolved ts in MySQL sink"] [tableID=71] [resolvedTs=434286922307731459]
[2022/07/01 18:58:58.483 +08:00] [INFO] [mysql.go:524] ["clean up table checkpoint ts in MySQL sink"] [tableID=71] [checkpointTs=434286922307731459]
Apply redo log successfully

but I still don't know what position to apply to.
Describe the feature you'd like
refine the note printed by redo apply, like:
Apply redo log successfully, the downstream database is already in TSO(434286922307731459)

Describe alternatives you've considered
No response
Teachability, Documentation, Adoption, Migration Strategy
No response

Should we also print human-readable timestamp in the log？

/assign @zhaoxinyu

/unassign @asddongmen

Should we also print human-readable timestamp in the log？

good comment


The text in Inputs is cut off

The text is cutoff :

Any development on this?

this would be that you line height is too high for the height of the input box,
lower your line height or make your input box taller

also next time try providing code so that its easier for people to debug


Unicode char as delimiter won't use C engine

I have the following code:
dfEL = pd.read_csv(IN_PATH, delimiter='\\u00A7', encoding='utf-8')
which I've also tried with other ways of writing the delimiter, e.g.:
dfEL = pd.read_csv(IN_PATH, delimiter='§', encoding='utf-8')
These other methods don't work, and generate a UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc2 in position 10: unexpected end of data.
The first method, though, won't use the C regex engine:
dfEL = pd.read_csv(IN_PATH, delimiter='\\u00A7', encoding='utf-8')
C:\Anaconda3\lib\site-packages\ipykernel\__main__.py:1: ParserWarning: Falling back to the 'python' engine because the 'c' engine does not support regex separators (separators > 1 char and different from '\s+' are interpreted as regex); you can avoid this warning by specifying engine='python'.
  if __name__ == '__main__':

Shouldn't this only be considered one character and still use (the I presume faster) C engine?
Sample data - there's a lot of messiness in the rightmost column, which is why an unusual separator was used:
foo§1457431587429§$request_details.bar
foo§1457431587429§$request_details.foo.bar
foo§1457431587429§$request_details.foo.foo.bar
foo§1457431587429§null
foo§1457431587429§null
foo§1457431587429§null
foo§1457431587429§$request_type
foo§1457431587429§$additional_details
foo§1457431587429§$Generic_Params.success_msg_folder+$request_type.action+'_'+$request_type.object
§1459972605829§$Path
§1459972605829§$Name
§1441995198746§$original.original
§1441995198746§null

pd.show_versions()

INSTALLED VERSIONS
------------------
commit: None
python: 3.5.1.final.0
python-bits: 64
OS: Windows
OS-release: 7
machine: AMD64
processor: Intel64 Family 6 Model 61 Stepping 4, GenuineIntel
byteorder: little
LC_ALL: None
LANG: None

pandas: 0.18.1
nose: 1.3.7
pip: 8.1.1
setuptools: 20.3
Cython: 0.23.4
numpy: 1.11.1
scipy: 0.18.0
statsmodels: 0.6.1
xarray: None
IPython: 4.1.2
sphinx: 1.3.1
patsy: 0.4.0
dateutil: 2.5.1
pytz: 2016.2
blosc: None
bottleneck: 1.0.0
tables: 3.2.2
numexpr: 2.6.1
matplotlib: 1.5.1
openpyxl: 2.3.2
xlrd: 0.9.4
xlwt: 1.0.0
xlsxwriter: 0.8.4
lxml: 3.6.0
bs4: 4.4.1
html5lib: None
httplib2: 0.9.2
apiclient: 1.5.0
sqlalchemy: 1.0.12
pymysql: None
psycopg2: None
jinja2: 2.8
boto: 2.39.0
pandas_datareader: None

Copy-pastable example (python3)
import pandas as pd
from io import StringIO
s = "a§b\n1§2\n3§4"

pd.read_csv(StringIO(s), sep='§')

By the way, for you first example, I think you want delimiter='\u00A7' (you had an extra backslash).

Apologies for not including a cut and paste example.
Actually, I do need the delimiter with two backslashes. In fact, your copy-and-paste example doesn't work for me as written:
import pandas as pd
from io import StringIO
s = "a§b\n1§2\n3§4"

pd.read_csv(StringIO(s), sep='§')

---------------------------------------------------------------------------
UnicodeDecodeError                        Traceback (most recent call last)
<ipython-input-1-43a4e6b0419c> in <module>()
      3 s = "a§b\n1§2\n3§4"
      4
----> 5 pd.read_csv(StringIO(s), sep='§')
      6

<cutting>

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc2 in position 1: unexpected end of data

With the doubled form:
import pandas as pd
from io import StringIO
s = "a§b\n1§2\n3§4"

pd.read_csv(StringIO(s), sep='\\u00A7')

C:\Anaconda3\lib\site-packages\ipykernel\__main__.py:5: ParserWarning: Falling back to the 'python' engine because the 'c' engine does not support regex separators (separators > 1 char and different from '\s+' are interpreted as regex); you can avoid this warning by specifying engine='python'.
Out[2]:
   a  b
0  1  2
1  3  4

import pandas as pd
from io import StringIO
s = "a§b\n1§2\n3§4"

And with single form:
pd.read_csv(StringIO(s), sep='\u00A7')

---------------------------------------------------------------------------
UnicodeDecodeError                        Traceback (most recent call last)
<ipython-input-3-e789e47e41b2> in <module>()
      3 s = "a§b\n1§2\n3§4"
      4
----> 5 pd.read_csv(StringIO(s), sep='\u00A7')
      6

<cutting>

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc2 in position 1: unexpected end of data

That's odd, I wonder if we escape something improperly... Compare these two:
In [22]: '\u00A7'
Out[22]: '§'

In [24]: '\\u00A7'
Out[24]: '\\u00A7'

When you have the double \, the first backslash escapes the second,  so Out[24] is the literal string \\u00A7.

So those work:
'\u00A7'
Out[2]: '§'

'\\u00A7'
Out[3]: '\\u00A7'

Right - I don't think I've seen this behavior with unicode outside of pandas, but I rarely venture into unicode.

The reason for the error is that the data is getting encoded as utf-8 (see here), which destroys the delimiter in the data:
>>> data = "a§b\n1§2\n3§4"
>>> data.encode('utf-8')
b'a\xc2\xa7b\n1\xc2\xa72\n3\xc2\xa74'

However, ord(§) == 167, which is \xa7.  This causes the data to split improperly with the first element of the header being a\xc2, which leads to the error that you're seeing.
Now technically, we should be splittng by \xc2\xa7, but the C engine doesn't support splitting of that kind (we only support single character splitting).
In the long run, the solution would be to somehow support multi-char delimiters (tricky since we parse byte by byte with the C engine).  In the short-term, I think we should check the separator to see if it would be a multi-char when encoded, and if so, raise an error.   Thoughts?

Trying to detect such separators and raising an informative message sounds fine IMO


Add alternative method to get authentication code

Link to the RainbowCrack plugin which lets you recover your own Audible activation data (activation_bytes) offline.

I specifically didn't add this project due to the offline rainbow table crack. I want to avoid any possible suggestion that AAXtoMP3 could be used to remove DRM from unlawfully obtained Audible books that the user doesn't own.
I've looked at the project, and it seems to be a good one, but I don't want to link it here.

That makes sense.
For anyone who finds this issue, I found tables way easier to set up in order to get my activation code than audible-activator.


[WIP] Refactor planner API

This is a WIP PR to initiate discussion on introducing planner API.
Summary

Add planner classes

Planner: Base class
ConcretePlanner : Planner: Base class for non-meta planners (e.g., SnapPlanner). Open to a better name.
MetaPlanner

Add problem classes (for the planning methods in prpy): This is to encapsulate different sets of parameters per planning problems.
The snap planner is refactored using the new API. Other planners will be refactored as well once we nail down the planner API.

Before creating a pull request

[ ] Document new methods and classes
[x] Format code with make format

Before merging a pull request

[x] Set version target by selecting a milestone on the right side
[ ] Summarize this change in CHANGELOG.md
[x] Add unit test(s) for this change

Thanks to @jslee02 for putting together this draft. It's a great start and is very helpful for driving the discussion.
This is actually quite a bit more sophisticated than what I had in mind. I wrote up my original thoughts from back when we started Aikido below. I intend this to be the starting point of further discussion - I am not trying to prescribe this as the only solution. I am very interested in hearing what drove the design decisions in this pull request, since the devil is often in the details of the implementation.

I was originally envisioning four "layers" to the planning interface:

The planner itself, which may comply with no standard interface. The goal of this API is to expose the full functionality of the planner while (a) a providing a clean API, (b) using standard Aikido types wherever possible, and (c) introducing minimal overhead. Often, this will be as simple as a free function. All of the planning functions in Aikido today fall into this layer, e.g. planOMPL(), planCRRTConnect(), and computeParabolicTiming().

A planner interface for each type of query. Instead of having a singular Planner interface, there would be a separate interface for each type of query. We would define a new interface if: (a) the query is common in our application code (e.g. MAGI actions) and (b) more than one planner can solve the query, i.e. it is possible to implement more than one adapter class as outlined in (3) below.

An adapter class, that uses (1) to solve queries specified in a standard format. Any arguments that are not part of the planner interface, e.g. planner-specific resolution parameters, should be specified by the adapter's constructor. Often, this involves converting the query into the type of constraints that the planner accepts. For example, an adapter class could solve a "plan to a goal configuration" query by calling planOMPL() with a FiniteSampleable goal constraint.

A meta-planner implements an adapter class using another planner interface, i.e. it converts from one planner interface to another. For example, a meta-planner may convert an arbitrary PlanToConfigurations planner to a PlanToConfiguration planner by calling PlanToConfigurations on a list that contains the singular goal configuration.

If we need a meta planner that operates on multiple types of planning queries, e.g. by calling multiple planners in sequence, then we can implement that as a template class. The details here are are a little hairy, so we need to think carefully about how we define the planner interfaces to make this possible.

These are the general principles that drove this design:

It should always be possible to access the full power of a planner. Implementing an interface should never reduce a planner's functionality.
An interface only has value if all implementations of the interface can be used interchangeably (sometimes called the Liskov substitution principle).
Every planning query has a different interface, so it's not possible to use them interchangeably.
It's useful to enforce the type on arguments and return values of planning functions at compile time.
Trying to implement a generic Planner base class requires bypassing most of the C++ type system. If that is necessary, you should consider writing that code in Python instead. :winking:

Looking at the current implementation in this pull request, I think there are a few great ideas that we could add to the mix:

Problem classes provide a nice mechanism for defining arguments. Inheritance could come in handy here once we have a wider variety of planner interfaces. For example, we could write helper code for a lot of boilerplate StateSpace, DistanceMetric, etc initialization. It also is a nice workaround for C++'s lack of named initialization lists.
Result classes provide a nice mechanism for annotating the output of planners with planner-specific information. This has many of the same advantages as Outcome did in https://github.com/personalrobotics/aikido/pull/266.

Here is a more concrete example in C++-like pseudocode:
// 1. Planner
trajectory::InterpolatedUniquePtr planOMPL(
  const statespace::StateSpace::State* _start,
  const statespace::StateSpace::State* _goal,
  statespace::StateSpacePtr _stateSpace,
  statespace::InterpolatorPtr _interpolator,
  distance::DistanceMetricPtr _dmetric,
  constraint::SampleablePtr _sampler,
  constraint::TestablePtr _validityConstraint,
  constraint::TestablePtr _boundsConstraint,
  constraint::ProjectablePtr _boundsProjector,
  double _maxPlanTime,
  double _maxDistanceBtwValidityChecks);

// 2. Planner Interfaces
class FromConfigurationToConfigurationPlanner
{
public:
  virtual trajectory::TrajectoryUniquePtr planFromConfigurationToConfiguration(
    const statespace::StateSpacePtr& stateSpace,
    const statespace::StateSpace::State* startState,
    const statespace::StateSpace::State* goalState) = 0;

};

class FromConfigurationToConfigurationsPlanner
{
public:
  virtual trajectory::TrajectoryUniquePtr planFromConfigurationToConfigurations(
    const statespace::StateSpacePtr& stateSpace,
    const statespace::StateSpace::State* startState,
    std::vector<const statespace::StateSpace::State*> goalStates) = 0;
};

// 3. Adapter Interfaces
class OMPLFromConfigurationsToConfigurationsPlanner
  : public virtual FromConfigurationToConfigurationsPlanner
{
public:
  virtual trajectory::TrajectoryUniquePtr planFromConfigurationToConfigurations(
    const statespace::StateSpacePtr& stateSpace,
    const statespace::StateSpace::State* startState,
    std::vector<const statespace::StateSpace::State*> goalStates)
  {
    return planOMPL(...);
  }
};

// 4. Meta-Planner
class FromConfigurationsToConfigurations_to_FromConfigurationToConfiguration
  : public virtual FromConfigurationToConfigurationPlanner
{
public:
  explicit FromConfigurationsToConfigurations_to_FromConfigurationToConfiguration(
      std::unique_ptr<OMPLFromConfigurationsToConfigurationsPlanner> delegatePlanner)
    : mDelegatePlanner{std::move(delegatePlanner)}
  {
  }

  virtual trajectory::TrajectoryUniquePtr planFromConfigurationToConfiguration(
    const statespace::StateSpacePtr& stateSpace,
    const statespace::StateSpace::State* startState,
    const statespace::StateSpace::State* goalState)
  {
    return mDelegatePlanner->planFromConfigurationToConfigurations(
      stateSpace, startState, {goalState});
  }
};

@mkoval Thanks for the suggestion. I think the major differences between our designs are:
(1) Having base planner class (i.e., Planner) vs. not
(2) Having all the solvable planning methods in a planner vs. having them as separate classes
More specifically, I propose having a planner class and putting all the functions of the solvable planning problems (methods) into the planner class. So we might only have like SnapPlanner, VectorFieldPlanner, OmplPlanner, TrajOptPlanner, and so on; and multiple planning functions will be in each class. In your design, we would have more classes which are the permutation of planners and planning problems, and single planning function in each class.
Here are some thoughts on possible advantages of this PR's implementation that I believe:
Having planning functions in a specific planner class would be able to share reusable functionality and data in the same class. I believe those planning functions in a planner class have a similar fashion of solving the problems. For example, TrajOptPlanner would solve all the problems solving optimization problems. It would be still possible with your design by having an additional class(es) and composing it into the necessary adaptor classes, but I think this could introduce unnecessary complex (I could be biased).
Another possible advantage is that the base planner class can provide useful interface such as canSolve(problem), solve(problem), and getSolvableProblems(). These functions allow you to get the capability of the planner. This functionality would be able to be achieved with your design by adding them to meta-planner, but my proposal would enable to implement meta-planners straightforwardly with the virtual functions of the base planner class.
I'm not sure how to incorporate Problem classes into your design because it seems the planner interfaces are the problem classes to me. Probably we have different definitions of Problem classes in each mind? :sweat_smile:

I think your summary of the design differences is accurate. 👍

In your design, we would have more classes which are the permutation of planners and planning problems, and [a] single planning function in each class.

This is not strictly required. If the implementation of multiple planning interfaces for a planner are similar, then you can implement them in the same class. However, I do think that this will end up being relatively rare in practice - I think an adapter class is often the better way to go.

So we might only have like SnapPlanner, VectorFieldPlanner, OmplPlanner, TrajOptPlanner, and so on; and multiple planning functions will be in each class.

This is the approach that we used in PrPy. We quickly discovered that it is difficult to handle planner-specific parameters. Multiple planners may implement the same planning method (e.g. PlanToConfiguration), but take different parameters. For example: some planners require time limits, others require heuristics, etc.
With my proposed API, planner-specific parameters (e.g. time limits) are passed into the adapter class' constructor. The planning method itself always take only the arguments defined by the planner interface(s) that it implements.
This is critical when planners are used via dependency injection. For example, a MAGI action can take an std::unique_ptr<PlanToEndEffectorOffset> without caring which specific planner it is using or what extra **kwargs it needs to pass to it.

Having planning functions in a specific planner class would be able to share reusable functionality and data in the same class. [...] It would be still possible with your design by having an additional class(es) and composing it into the necessary adaptor classes, but I think this could introduce unnecessary complex (I could be biased).

This is exactly how I propose to solve the re-use problem. Nothing about defining a variety of interfaces prohibits us from defining helper functions and/or classes. I suspect that the resulting code would end up being more reusable and easier to unit test, since it separates the helper code from the implementation of the planner. (This is largely my gut reaction, so I don't want to start a holy war. 😬)

Another possible advantage is that the base planner class can provide useful interface such as canSolve(problem), solve(problem), and getSolvableProblems(). These functions allow you to get the capability of the planner.

This is a neat idea. This seems most useful for allowing a MetaPlanner to have the upmost flexibility in deciding which planner to apply to a problem. The disadvantage is that you lose a lot of nice type safety enforced by the C++ type system. My approach is much more rigid because the list of queries is specified as C++ types at compile time, but has that type safety.
While it has a lot of potential, I am not sure how useful this level of generality will be in practice today. We have relatively small sets of planners and types of planning queries. However, I do entirely understand the appeal of this - I believe this is actually pretty similar to the design that @cdellin and the NREC folks used for motion planning on CHIMP in the DRC.

In general, I was trying to follow the philosophy that Aikido is a toolbox of small, reusable pieces of code that can stitched together to solve a variety of planning problems. I really like the idea of adapter classes because it is so DRY. Uou can write (and test!) a piece of code once, rather than repeating it once per planner.
For example, any planner that implements PlanFromConfigurationsToConfigurations would automatically get PlanToConfigurationFromConfigurations, PlanFromConfigurationsToConfiguration, and PlanToConfigurationToConfiguration for free. You could also convert any planner that implements PlanToConfiguration to PlanToTSR by sampling from the TSR and calling PlanToConfiguration (this is what TSRPlanner does in PrPy).
This is similar to how we defined NewtonsMethodProjectable once, rather than re-implementing a rootfinder every time we need to do constraint projection.

I am interested in hearing your thoughts on this @jslee02 - I'm open to be convinced otherwise. 😈

At a meta-level, we seem to be discussing the relative merits of composition over inheritance. @mkoval and @jslee02 are way better at discussing this than I am, so I'll defer to them, except point out that composition often enables more flexibility whereas inheritance often enables more readability.
I want to emphasize one comment that @mkoval made that I think is critical for any path we pursue:

This is the approach that we used in PrPy. We quickly discovered that it is difficult to handle planner-specific parameters. Multiple planners may implement the same planning method (e.g. PlanToConfiguration), but take different parameters. For example: some planners require time limits, others require heuristics, etc.

This has annoyed me [maybe us all] no end. Although I am a fan of having some structure in what a planner can do [and I think the PlanTo* structure we came up with in prpy is really fantastic], I always felt like the way it was implemented in prpy really really straightjacketed us in accessing the planners. I suppose one way to address this is how scipy and much of MATLAB address this is via an open-ended options dictionary. But then that demands a lot of documentation and can easily go very wrong.

Hi all, thanks for the great suggestions. I think both designs are flexible and capable to handle the variety of problems we have. In terms of readability, I slightly prefer @jslee02 's proposal. However, I think there's one major advantage of @mkoval 's proposal that may be a bit complicated to handle in @jslee02 's current design:

With my proposed API, planner-specific parameters (e.g. time limits) are passed into the adapter class' constructor. The planning method itself always take only the arguments defined by the planner interface(s) that it implements. This is critical when planners are used via dependency injection. For example, a MAGI action can take an std::unique_ptr without caring which specific planner it is using or what extra **kwargs it needs to pass to it.

These are in @jslee02 's version problem x planner-specific parameters, so it cannot be passed in at the constructor of @jslee02 's planner (e.g. OMPLPlanner). @jslee02 and I had a brief discussion on how to handle this, e.g. via DART's Aspect functionality, which (to my best understanding) is a sophisticated version of an open-ended options-dictionary.  The amount of documentation may not be too different from what we may end up doing with @mkoval 's design, but given that we have to take such an approach, I think being able to rely on C++'s type safety is more appealing.

This is the approach that we used in PrPy. We quickly discovered that it is difficult to handle planner-specific parameters. Multiple planners may implement the same planning method (e.g. PlanToConfiguration), but take different parameters. For example: some planners require time limits, others require heuristics, etc.

As @gilwoolee mentioned, we could use DART's aspect mechanism (of @mxgrey) to achieve that. In our case, [~]Problem will be Composites, and each additional planner specific parameters will be in Aspects, which will be defined in each planner. As so, the planner can use the aspect when it's added to the problem; otherwise, use the defaults. I believe Aspect-Component is designed to handle this kind of problems.

(@mkoval's comment) In general, I was trying to follow the philosophy that Aikido is a toolbox of small, reusable pieces of code that can stitched together to solve a variety of planning problems. I really like the idea of adapter classes because it is so DRY. Uou can write (and test!) a piece of code once, rather than repeating it once per planner.
(@siddhss5's comment) At a meta-level, we seem to be discussing the relative merits of composition over inheritance. @mkoval and @jslee02 are way better at discussing this than I am, so I'll defer to them, except point out that composition often enables more flexibility whereas inheritance often enables more readability.

I also prefer DRY, and it can be achieved in my design as well. [X]Problem class could have reusable common functionalities, which are the X planning problem specific. I believe the problem classes are the same with the planner interface in your pseudo code in that sense; except (1) the planning functions are defined in planner classes in my design, and (2) problem classes are derived as adapter planners for each planning methods in your design.
Overall, I believe both designs don't have any major technical issue to achieve certain flexibility we want to obtain, but I think my design would be more straightforward to implement the planner capability checking functionality (e.g., canSolve(problem), solve(problem), and getSolvableProblems()).
I think it would be helpful to compare and improve the design ideas here (or in gist). Let me write down sample code soon. :)

@brianhou, @gilwoolee, and I discussed further, and I think we can reach a compromise. Essentially, it would be based on your Planner Interfaces and incorporate Problem classes and canSolve/getSolveableProblems into that.
Here is a high-level picture I have:
Summary

Define planner interfaces per planning problems
Define adapter interfaces
Pass Problem classes to the planner interfaces
Add canSolve and getSolveableProblems to Robot class
Make a concrete robot class (e.g., Herb) inherit Robot class and planner interfaces that the robot want to support

Please let me know what you guys think or what is not clear. If this sounds good, I will make the changes in this PR.

I've revised the planner API according to the discussion (+ ideas came up with while implementing it). Let me know what do you think. Here is a rough summary of the changes.
Note on the Changes

Kept Planner class to provide canPlan, getPlannableProblems, and plan, which takes the base problem class
Added PlannerForSingleProblem : Planner which is a base class of the planner interface classes. It is templatized for a planning problem and implements all the virtual function of Planner class.
Added PlannerForMultiProblem : Planner which is the base class of concrete planner class that can solve more than one planning problem
Revised snap planner (i.e., SnapConfigurationToConfigurationPlanner) to inherit PlannerForSingleProblem
Renamed planning problem classes in this fashion: <start_type>To<goal_type>. For example, ConfigurationToConfiguration
Renamed planner classes in the same fashion of planning problem but with '-Planner` appendix.

@mkoval @psigen Could you guys take a look the latest changes if it makes sense when you have a chance? :smile: Thanks!

I have revised this PR based on the online and offline discussions. @brianhou, @gilwoolee, @aditya-vk Second round of review, please. :smile:

The build failures on macOS should be fixed once #395 is merged.

Thanks for the changes! Merging this now 🎉


App-specific upload paths for images

Now the upload path consists from three parts: CKEDITOR_UPLOAD_PATH, optional date path //, optional user path (id or other field).
What I need is more flexibility in paths:

Some optional path suffix to set the application-specific paths, like uploads/news/, uploads/events/, uploads/gallery/, etc.

More flexible path ending, to set paths like // or ///.

Is it possible to do it via some settings, or, if not, could you implements these options in the new package versions?

Maybe you can use the CKEDITOR_FILENAME_GENERATOR setting for this. If not you could write and contribute a solution yourself?

Thank you very much for the idea. If it is possible to generate file names together with path, it is just what I need! I will try it and write here.


[Question] - How to Identify / distinguish between a regular SAML 2.0 and AD FS SAML2.0

From your docs, I can see this line:
lowercase_urlencoding - Defaults to false. ADFS users should set this to true.
The question is, is there a way to differentiate between customers that are using SAML 2.0 Azure ( for example ) and SAML 2.0 AD FS?

You only need to enable this, if you experience issues while validating the Signatures of the LogoutReqquest/LogoutResponses generated by ADFS. If you don't experience any issue, you don't need to enable it.

Hi @pitbulk
Understood, the reason I am asking it is because I need both methods.
Now adfs is not working (Azure adfs)
Can you please also check it on your end?


[Key Vault] Correctly handle token refreshes in AD FS

Description
Resolves https://github.com/Azure/azure-sdk-for-python/issues/33621. https://github.com/Azure/azure-sdk-for-python/pull/30195 partially fixed authentication with KV in AD FS, but the issue crops back up when tokens are refreshed -- the refresh interaction wasn't updated to exclude a tenant ID from token requests in AD FS.
This updates tests to test the refresh scenario, and I verified that the updated tests failed before the code changes in this PR.
All SDK Contribution checklist:

[x] The pull request does not introduce [breaking changes]
[x] CHANGELOG is updated for new features, bug fixes or other significant changes.
[x] I have read the contribution guidelines.

General Guidelines and Best Practices

[x] Title of the pull request is clear and informative.
[x] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

[x] Pull request includes test coverage for the included changes.

API change check
APIView has identified API level changes in this PR and created following API reviews.
azure-keyvault-administration

Thanks, looks good to me! Not sure why the bot is saying there are API changes in this PR.

e.g. https://apiview.dev/Assemblies/Review/7643acce229a47c8a2d146efc30638fb/8b88868d7fcc41179195b974254e924d?diffRevisionId=7d803f0851c746538a6193c1a0689b24&doc=False&diffOnly=True


[ERR_MODULE_NOT_FOUND]: Cannot find module '/Users/me/code/forte/opus/node_modules/next/navigation.js'

Context
What's your version of next-usequerystate?
 "next-usequerystate": "^1.13.0",

Next.js information (obtained by running next info):
 Operating System:
      Platform: darwin
      Arch: x64
      Version: Darwin Kernel Version 22.6.0: Fri Sep 15 13:39:52 PDT 2023; root:xnu-87<IP_ADDRESS>0.8~1/RELEASE_X86_64
    Binaries:
      Node: 18.12.1
      npm: 8.19.2
      Yarn: 1.22.19
      pnpm: N/A
    Relevant packages:
      next: 12.2.2
      eslint-config-next: 12.2.2
      react: 18.2.0
      react-dom: 18.2.0

Are you using:

❌ The app router
✅ The pages router

Description
We wanted to give next-usequerystate another try today because it seems like what we want.
But I wonder if something about our setup (e.g. 12.2.2) is insufficient.
I get this error when I try to browse to a page using this library.
 error - Error [ERR_MODULE_NOT_FOUND]: Cannot find module '/Users/me/code/forte/opus/node_modules/next/navigation.js' imported from /Users/me/code/forte/opus/node_modules/next-usequerystate/dist/index.js
[next]     at new NodeError (node:internal/errors:393:5)
[next]     at finalizeResolution (node:internal/modules/esm/resolve:323:11)
[next]     at moduleResolve (node:internal/modules/esm/resolve:916:10)
[next]     at defaultResolve (node:internal/modules/esm/resolve:1124:11)
[next]     at nextResolve (node:internal/modules/esm/loader:163:28)
[next]     at ESMLoader.resolve (node:internal/modules/esm/loader:841:30)
[next]     at ESMLoader.getModuleJob (node:internal/modules/esm/loader:424:18)
[next]     at ModuleWrap.<anonymous> (node:internal/modules/esm/module_job:76:40)

Thanks.
Reproduction

Please provide minimal reproduction.
And you cannot use next/navigation in pages router.

@ryan-walsh-forte I'm afraid you can only go so high as<EMAIL_ADDRESS>under Next.js 12, due to 1.8+ using the imports of the new navigation router (which works on both pages and app directories).

@franky47 sending a pr soon

@Talent30 @franky47 Thank you so much!


Added tools to the Learning Design category

Checklist

[x] I made something good today 💐.
[x] I put links in the correct format: [Tool](link) — description.
[x] My description starts with the lower-case letter and ends with the full stop.
[x] I put the tool in the alphabetical order.

Optional

[x] I added the appropriate labels: free, open-source or mac.
[x] I put the tool only to one category.
[x] I followed Contribution Guidelines.

In reference to issue #131, this pull request to add in an initial set of tools for the new Tools for Learning Design category.

@ericalli thanks for the suggestion to add Learning Design section and for adding all these nice resources.


add CustomEvent polyfill

Fixes the issue with IE and the prerender code.

I just realized there were places I forgot to condition on if (CustomEvent). But this works too so :+1:


miniziti configurable timeout

If you're intentionally normalizing all wait periods to a single global value, then this PR looks good to me.


Make Plato stats available by serving the static files

Plato creates static files which can be served with a static file server to easily view.
Same can be done with coverage files and protractor screen shots.
It would be nice to wire up BrowserSync notifications to display links that take you to these sites.

This can be done easily with browser-sync cli.


[cxx-interop] Apply typedef -> enum patch to method param types as well.

Basically just applying https://github.com/apple/swift/pull/42431 to ObjC method param types.

@swift-ci please test.

@swift-ci Please Test Source Compatibility

Windows test failures are unrelated. Merging.


PAW3902: align mode change logic with the spec

The spec specifies that the mode change condition should be met for 10
consecutive frames before changing to the next mode.
The spec (and comment) says that "PAW3902JF should not operate with Shutter < 0x01F4 in Mode 2" -
however the if condition checked the reverse condition

Signed-off-by: Koby Aizer<EMAIL_ADDRESS>

Looks good, do you have any test data you can share? https://logs.px4.io/

Looks good, do you have any test data you can share? https://logs.px4.io/

I'll try to create some tomorrow, it did fly but unfortunately without logging

These changes look correct (and similar to something I had started), let's merge and revisit the reset separately. Thanks!

Cool, just for the sake of completeness - we had troubles with the setup today. However, my teammates did find some old log in low light with those changes: https://review.px4.io/plot_app?log=1f024625-bfaa-4a55-be17-d4ecf1a60f95
I still see some toggling there (there is an added print of each mode change), however it is much less than without this change.


add test with balance output and at least 2 species controlling density

Add  a test that splits a flow + transport simulation in subdomains. There should be at least 2 species controlling density.
At least one of the partitions should not be at the model boundary, but internal. The transport model should compute balance, and merged balance results should be compared.
Note (20-2-'24):
The scope of this issue has been reduced. Balance output checking will be done in a follow up issue. This issue now only focusses on making a multispecies model with variable density, partitioning it  and checking concentration results.

merged to feature branch


Unsupported definition list syntax in GFM output

When GitHub-flavored Markdown is generated via the markdown_github transformation type, the Markdown files use PHP Markdown Extra's Definition List syntax for DITA definition lists, but this syntax is unfortunately not supported in GitHub-flavored Markdown.
On GitHub, the leading colons in definitions are rendered literally and terms do not appear in bold:
Term 1
:   Definition 1

Pandoc's alternative tilde syntax might be preferable to preserve a visual indication/indentation of definitions:
Term 1
 ~  Definition 1

Alternatively, GFM output could simply render terms in bold and definitions as common paragraphs below:
**Term 1**
Definition 1

The “Markdown DITA” format generated via the markdown transformation type should continue to use the definition list syntax to enable generation of definition lists in DITA content from MDD Markdown files.

One alternative could be
*   **Term 2**

    Definition 2a

    Definition 2b
```

This clearly marks where the list ends.


Investigate ConnectDisconnect test problem.

ConnectDisconnect test in main_test.go currently hangs indefinitely if running with all other tests.
By initial investigation I found that grpc server is not shutting down. Current assumption is that there is open connection somewhere which is not being closed. Since we use grpc.GracefulShutdown which waits for all connection to be resolved, and changing it to Shutdown which drops all connections resolves issue. Suspected is noise.skademlia.connLoop which is goroutine which cannot be shutdown externally and it finalizes itself only after certain amount of time.

Problem was in UpdateParams test which changed configuration for all follow up tests, and connect-disconnect one was the only one.

Fixed in #355


Rename update to async_update for consistency

SSIA

Does this need any changes in Core?

Does this need any changes in Core?

I still have to look but I don’t think so.

Shouldn't need any changes in core: homeassistant/components/zha/entity.py#L137-L141 (and don't see update() being called anywhere else either)

thx. I just double checked as well and I agree.


Feature Request: Ability to always see Cyclomatic Complexity/Maintainability Index

I feel it would be nice to see the Cyclomatic Complexity and Maintainability Index for a any procedure regardless of whether it creates a warning or not.  It would help when doing a code review and want to know what the measures are without having to adjust the LinterCop.json settings.
Could you show it like GitLens shows the commit info?

Could you provide it in the hover like a warning does?  Maybe as an optional "info" or "hint" level entry?

That already extists. Its rule LC0009 You just need to set it to info in your ruleset. Its disabled by default

Cool thanks.
Going to send another one 😛


The CI of cpp tests is broken

Describe the bug
Recently I found a bug in C++ client that was introduced from #11492 (I'll fix it soon). However, the point is, the related C++ unit tests should fail but the CI passed. See https://github.com/apache/pulsar/runs/3188401433:
     176 ms: ./main AuthPluginToken.testTokenWithHttpUrl (try #1)
      89 ms: ./main AuthPluginToken.testTokenWithHttpUrl (try #2)
      35 ms: ./main BasicEndToEndTest.testV2TopicHttp (try #1)
      27 ms: ./main BasicEndToEndTest.testV2TopicHttp (try #2)
      27 ms: ./main BasicEndToEndTest.testHandlerReconnectionLogic (try #1)
      25 ms: ./main BasicEndToEndTest.testHandlerReconnectionLogic (try #2)
    1519 ms: ./main ClientTest.testConnectTimeout (try #1)
    1519 ms: ./main ClientTest.testConnectTimeout (try #2)
      28 ms: ./main ProtobufNativeSchemaTest.testSchemaJson (try #1)
      34 ms: ./main ProtobufNativeSchemaTest.testSchemaJson (try #2)
     107 ms: ./main ProtobufNativeSchemaTest.testAutoCreateSchema (try #1)
      69 ms: ./main ProtobufNativeSchemaTest.testSchemaIncompatibility (try #1)
      86 ms: ./main ProtobufNativeSchemaTest.testAutoCreateSchema (try #2)
      97 ms: ./main ProtobufNativeSchemaTest.testSchemaIncompatibility (try #2)
      91 ms: ./main ProtobufNativeSchemaTest.testEndToEnd (try #1)
      82 ms: ./main ProtobufNativeSchemaTest.testEndToEnd (try #2)
    9244 ms: ./main KeySharedConsumerTest.testOrderingKeyPriority (try #1)
    1059 ms: ./main ConsumerTest.testBatchUnAckedMessageTracker (try #1)
     121 ms: ./main ClientDeduplicationTest.testProducerDeduplication (try #1)
      73 ms: ./main ClientDeduplicationTest.testProducerDeduplication (try #2)
      40 ms: ./main AuthPluginTest.testTlsDetectHttps (try #1)
      40 ms: ./main AuthPluginTest.testTlsDetectHttps (try #2)

It looks like the current CI of cpp tests allow the failed tests and bugs could be introduced with CI passed.
To Reproduce
Expected behavior
CI should fail if tests still failed after retrying.
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):

OS: [e.g. iOS]

Additional context
Add any other context about the problem here.

I found the broken CI was introduced from https://github.com/apache/pulsar/pull/10309, do you know why the CI passed even the tests failed? @lhotari
BTW, I'm going to fix these failed tests ASAP.

It's fixed by #11575


Add create-react-class dependency

It appears this forked library added create-react-class functionality but without specifying it as a dependency. A react-redux update in PocketDerm appears to be failing because react-redux no longer specifies create-react-class as a dependency, and so in the update PocketDerm no longer has it, causing use of this library to fail.
This PR should fix that. Going to test it now.

Merging in after fixing build issues in PocketDerm.


AttributeError

Encounter the error while running code block 8.

This error is caused by an upgrade to Pandas that removed this function. We will address this in a future update, but from now the toolbox requires you use pandas version .24.
you can do this by adding !pip install pandas==0.24.2 --user to your notebook to version lock pandas