Allow more constructs to work as type guards for unknown

Search Terms
unknown type guard
Related: https://github.com/Microsoft/TypeScript/pull/24439#issuecomment-394185089, https://github.com/Microsoft/TypeScript/issues/25172
Suggestion
Currently, only a very limited set of type guards are able to narrow the new unknown type:

Array.isArray (because it is a typeguard for arg is any[]) and probably some more in the lib files
instanceof
self-written typeguards

However to make working with unknown types less awkward, I'd like to see a couple of other constructs being able to narrow the unknown type:
let x: unknown;
// Direct equality should narrow to the type we compare to
x === "1"; // should narrow x to string or the literal "1" type, similar for other types aswell

// All these should narrow x to {prop: any}
"prop" in x;
x.prop != null;
x.prop !== undefined;
typeof x.prop !== "undefined";

// typeof should work on properties of the unknown variable
typeof x.prop === "string"; // should narrow x to {prop: string}

Use Cases
Make unknown easier to work with!
// before, very verbose!
const x: unknown = undefined!;

function hasProp1(x: any): x is {prop1: any} {
	return "prop1" in x;
}
function hasProp2(x: any): x is {prop2: any} {
	return "prop2" in x;
}
// imagine doing this for many more properties

if (hasProp1(x)) {
	x.prop1;
	if (hasProp2(x)) {
		x.prop2;
	}
}

// ===========

// after, much more concise and less overhead
const x: unknown = undefined!;
if ("prop1" in x) {
	x.prop1;
	if ("prop2" in x) {
		x.prop2;
	}
}

Checklist
My suggestion meets these guidelines:

[x] This wouldn't be a breaking change in existing TypeScript / JavaScript code
[x] This wouldn't change the runtime behavior of existing JavaScript code
[x] This could be implemented without emitting different JS based on the types of the expressions
[x] This isn't a runtime feature (e.g. new expression-level syntax)

The use of in is covered by #21732.  Shall we add the other checks for a property (comparison to undefined, typeof) to #21732 and call this a duplicate of #21732 + #25172?

Sure, why not!

And one more thing:
let foo: unknown;
if (typeof foo === "object") {
	// foo should probably be narrowed to {[prop: string]: unknown} here
}

At the least, at typeof foo === "object" it should narrow to object. Currently remains unknown and the following fails:
function f(u: unknown): object { return typeof u === "object" ? u : {}; }

Similar requests in https://github.com/Microsoft/TypeScript/issues/10715, https://github.com/Microsoft/TypeScript/issues/25172, and https://github.com/Microsoft/TypeScript/issues/21732

I was hoping unknown would let me have type-safe data-loading, e.g.:
interface SomeValue { a: string, b?: number, c: 'left' | 'right' }
function readValue(id: string): SomeValue {
  const u: unknown = await someDataSource(id);
  if (
    typeof u !== 'object' ||
    u === null ||
    typeof u.a !== 'string' ||
    b in u && typeof u.b !== 'number' ||
    u.c !== 'left' && u.c !== 'right'
  ) {
    throw new Error(util.format('Invalid value with id %O from some data source: %O', id, u));
  }

  return value; // TS is checking that the checks above actually verify the return type here
}

This to me would be a better match to TS for what #26078 wants, but I wouldn't complain about adding quick-fixes to add the missing checks!
(remember that typeof u === "object" should actually narrow to object | null - Thanks javascript!)

I would like it if type guards with unknown worked a little more like this.
let x: unknown

if (typeof x === 'object' && x !== null && 'foo' in x && typeof x.foo === 'string') {
  /* x is promoted to {foo: string} here */
}

I think the type promotion ought to work like so, if at all possible.

typeof unknown === 'object' -> object | null
(object | null) !== null -> object
'foo' in object -> {foo: unknown}
typeof {foo: unknown}.foo === 'string' -> {foo: string}

I realize I'm a bit late, but you might be interested in https://github.com/gcanti/io-ts -- provides a nice way to generate your necessary typeguards, though might be a bit heavy handed for the common usecase (and thus probably still worth considering this issue)

Thanks for the suggestion, but that's probably not relevant to the discussion.

I'd also like to add that unknownValue instanceof Array should really be refined to unknown[], not any[] as is the current behavior. I'm also not getting warnings about implicit any when I do that.

Is there any update regarding this issue? I would love to use the unknown type, but at this point it's just too verbose to narrow it down to bigger objects. This proposal would make it a lot easier.

Until this is fixed, this is a helper that can be used to make it easier to write manual type guards for unknown types that you expect to be nested objects:
export function isUnknownObject(x: unknown): x is { [key in PropertyKey]: unknown } {
  return x !== null && typeof x === 'object';
}

Example usage:
function example(x: unknown) {
    if (isUnknownObject(x) && isUnknownObject(x.prop) && typeof x.prop.subProp === 'string') {
        console.log(x.prop.subProp);
    } else {
        console.log('Could not find subProp');
    }
}

example({
    prop: {
        subProp: 'test',
    }
});
example({});

For more complicated use cases, using something like https://github.com/gcanti/io-ts is probably a better option than writing the type checks manually, but isUnknownObject can be useful for simple cases.

@butchler Similar to what you suggested (infect i started with that)
function isAssumedType<T = Record<string, unknown>>(x: unknown): x is Partial<T> {
	return x !== null && typeof x === "object";
}

// usage
if (isAssumedType<CommandCreator>(arg) && arg.execute && arg.host) {
    return true;
}

The main difference is that arg will be partially typed so when you do checks with props they are bound to the interface so you can F2 rename safely and will also be updated

isAssumedType<CommandCreator>(arg)

This is effectively the same as a type assertion (i.e. arg as Partial<CommandCreator>, but unlike a type assertion it does not use an explicit as keyword and it implicitly changes the type of arg in the following expressions.
Type assertions are fine and have to be used sometimes, but personally I would avoid using something like isAssumedType because 1) it is less explicit so other people reading the code might not realize a type assertion is being made and 2) it makes it very easy and convenient to use type assertions, which is probably a bad thing because type assertions should generally be avoided when possible.

Yes, naming is not the best I agree (but whatever, you can call it as you want), and to be honest I only use it private in file along with some type guards so my main intention is usage in guards so far.
As you suggested arg as Partial<CommandCreator> doesnt work inlined within the condition e.g.

Whereas with my suggestion works as following:

So again, the main benefit is that F2 rename works (which is a huge plus imo)
Anyway just wanted to share.

if (typeof x === 'object' && x !== null && 'foo' in x && typeof x.foo === 'string') {
/* x is promoted to {foo: string} here */
}

typeof x.foo doesn't work -> Property 'foo' does not exist on type 'object'.

if (typeof x === 'object' && x !== null && 'foo' in x && typeof x.foo === 'string') {
/* x is promoted to {foo: string} here */
}

typeof x.foo doesn't work -> Property 'foo' does not exist on type 'object'.

if (typeof x === 'object' && x !== null && 'foo' in x && typeof x.foo === 'string') {
/* x is promoted to {foo: string} here */
}

typeof x.foo doesn't work -> Property 'foo' does not exist on type 'object'.

That's an example of something I would like to work, but currently doesn't.

if (typeof x === 'object' && x !== null && 'foo' in x && typeof x.foo === 'string') {
/* x is promoted to {foo: string} here */
}

typeof x.foo doesn't work -> Property 'foo' does not exist on type 'object'.

That's an example of something I would like to work, but currently doesn't.

if (typeof x === 'object' && x !== null && 'foo' in x && typeof x.foo === 'string') {
/* x is promoted to {foo: string} here */
}

typeof x.foo doesn't work -> Property 'foo' does not exist on type 'object'.

That's an example of something I would like to work, but currently doesn't.

Maybe something like this works.. It does the job for me.. Here 'result' is of type unknown. So narrowing down to the required type using user defined type guard seems to be the sane type-safe solution. Or are there are other possibilities as well?
assert(
isOfType(result) && result.state,
"Either object is unknown or  the property 'state' doesn't exist"
)
function isOfType(object: unknown): object is T {
return object !== null && typeof object === 'object'
}

if (typeof x === 'object' && x !== null && 'foo' in x && typeof x.foo === 'string') {
/* x is promoted to {foo: string} here */
}

typeof x.foo doesn't work -> Property 'foo' does not exist on type 'object'.

That's an example of something I would like to work, but currently doesn't.

Maybe something like this works.. It does the job for me.. Here 'result' is of type unknown. So narrowing down to the required type using user defined type guard seems to be the sane type-safe solution. Or are there are other possibilities as well?
assert(
isOfType(result) && result.state,
"Either object is unknown or  the property 'state' doesn't exist"
)
function isOfType(object: unknown): object is T {
return object !== null && typeof object === 'object'
}

Would like to see this implemented.
function isTest(arg: unknown): arg is Test {
  return (typeof arg === "object") && (arg !== null) &&
    ("quest" in arg) && (typeof arg.quest === "string");
}

interface Test {
  quest: string;
}

Typescript still complains for the above code even though we check for all the cases before accessing the quest property.
Using any instead of unknown takes away the benefits of using TS in the first place.

I would expect this to work as well with an unknown err, though perhaps there's a case I haven't thought of which would cause failure: if (err?.message) fnThatExpectsTruthyValue(err.message)

The lack of better/easier type guards for unknown type is why I'm not using it in try...catch blocks as well. I should be able to do simply
catch (e: unknown) {
  console.log(e?.message);
}

I don't want to overcomplicate my code just because unknown is so hard to work with.

Crosslinking to #27706

Adding this kind of type of guards to Arrays, Maps and Sets could be done quite easily, Objects are be a bit more complicated.
const inp = process.env.INPUT as unknown

const arr = ['a', 'b', 'c'] as const
if (arr.includes(inp)) {
  console.log(inp)
  //          ^? "a" | "b" | "c"
}

const obj = {'a':1, 'b':2, 'c':3} as const
if (obj.hasOwnProperty(inp)) {
  console.log(inp)
  //          ^? "a" | "b" | "c"
}

const map = new Map([['a',1], ['b', 2], ['c', 3]] as const)
if (map.has(inp)) {
  console.log(inp)
  //          ^? "a" | "b" | "c"
}

const set = new Set(['a', 'b', 'c'] as const)
if (set.has(inp)) {
  console.log(inp)
  //          ^? "a" | "b" | "c"
}


Lifecycle issue: metadata is not changed once lifecycle operation happened

Is this a BUG REPORT or FEATURE REQUEST?:

Uncomment only one, leave it on its own line:
/kind bug

What happened:
Cannot download object after data lifecycle task happend:

What you expected to happen:
Can download object after data lifecycle task happend
How to reproduce it (as minimally and precisely as possible):

create lifecycle configuration with rule of migrating after expire
upload object
change time of object
download object after lifecycle operation is finished.

Anything else we need to know?:
Environment:

Gelato(release/branch) version:
OS (e.g. from /etc/os-release):
Kernel (e.g. uname -a):
Install tools:
Others:

Fixed by 839


Split extra tasks from standalone tests

What does this PR do?
Small refactor required for #11098
Splits the script run_standalone_tests.sh into two scripts:

The batched standalone tests
Additional tasks that need to run outside pytest

Before submitting

[x] Was this discussed/approved via a GitHub issue? (not for typos and docs)
[x] Did you read the contributor guideline, Pull Request section?
[x] Did you make sure your PR does only one thing, instead of bundling different changes together?
[x] Did you make sure to update the documentation with your changes? (if necessary)
[x] Did you write any new necessary tests? (not for typos and docs)
[x] Did you verify new and existing tests pass locally with your changes?
[x] Did you update the CHANGELOG? (not for typos, docs, test updates, or internal minor changes/refactorings)

PR review
Anyone in the community is free to review the PR once the tests have passed.
Before you start reviewing make sure you have read Review guidelines. In short, see the following bullet-list:

[x] Is this pull request ready for review? (if not, please submit in draft mode)
[x] Check that all items from Before submitting are resolved
[x] Make sure the title is self-explanatory and the description concisely explains the PR
[x] Add labels and milestones (and optionally projects) to the PR so it can be classified

Did you have fun?
I made sure I had fun coding 🙃
cc @carmocca @akihironitta @borda @justusschock @awaelchli @rohitgr7

We use the refactor label for code refactors (this is just a CI change) and the tests label for direct changes to the tests (again, this is just a CI change). Removed them

#11098 was merged before, but I would still merge the small improvements here.


API lowercases setup key group names

Describe the problem
When creating setup keys through the API, group names are lowercased, but are still case sensitive -- title cased group names are not found.
To Reproduce
When I POST to the endpoint /api/setup-keys with the following payload
{
  "name": "Testing key",
  "type": "one-off",
  "expires_in": 86400,
  "revoked": false,
  "auto_groups": [
    "Foobars"
  ],
  "usage_limit": 1
}

I get this error
{"message":"group foobars doesn't exist","code":404}

But I can create a setup key with "Foobars" through the UI.
Expected behavior
The setup key should be properly created

Hello @clarkmcc the auto_groups expect group IDs instead of names. You can get that with
GET /API/groups

Ah, thanks


Wofost72_PP insensitive to irrigation compared to Wofost72_WLP_FD

Allard's feedback:

It is a result of the flexibility of the event mechanism.
Wofost72_PP runs a simulation for the potential production level, as such there is no handler that will react to irrigation events.
However, that does not impede you from sending irrigation events. But given that there is no handler, there is no code that will actually look at the contents of the event so nothing happens.
Only in case of the Wofost72_WLP_FD there is an event handler that looks at the irrigation event content, so the contents must be specified as a float.


Review token ordering for V3 functions

AFAIU in V3 tokens are no longer sorted by address but instead by the order of tokens being passed when the pool is created.
In V3 weighted pool creation we are sorting the tokens by address here before creating the pool.
This could perhaps lead to unexpected behaviour for users? Is there anywhere else we're doing similar?

This affects add liquidity implementation as well. I pinged SC team to discuss the best approach here.

After discussing this with SC team, we decided to keep V2 pattern and apply token sorting to V3 as well, so I'm closing this task as it's no longer needed.


Example with koa-router doesn't work

"koa-router": "^7.3.0"
this is version, app.use can work, but router not work!!!

I got the same problem.
...
app.use(KoaBodyParser())
router.all('/api/file', KoaBody(), async ctx => {
  ctx.body = {
    api: true,
    type: ctx.method,
    query: ctx.query,
    body: ctx.request.body,
    files: ctx.request.body.files
  }
})

"koa":"2.5.1"
"koa-router": "7.4.0"
"koa-body":"2.6.0"

Can one of you provide a single file that reproduces the exact issue? Is there an error?

@MarkHerhold
My problem is that when the Content-type  is application/x-www-form-urlencoded it does not respond to the.
code:
app.use(KoaBodyParser())

router.all('/api/file', KoaBody(), async ctx => {
  ctx.body = {
    api: true,
    type: ctx.method,
    query: ctx.query,
    body: ctx.request.body,
    files: ctx.request.body.files
  }
})

When the code is app.use(KoaBodyParser())  othor router it dose not work !

@MarkHerhold
const router = new Router() router.use('/upload', KoaBody({ multipart: true }), async ctx => {   console.log(ctx.request.body); })
or use router.post and etc, it not work, but use app.use, encountered the same problem @damianb, other routers await to loading.
use koa-router, maybe router.use parameters is error, and not run next() other middleware

the same with me ,the result of  console.log(ctx.request.body.files)  is undefined , i have already to upload a picture.

how to resolve this problem ??

@Chen-WeiZhen files was moved to ctx.request.files in the latest version.

Whet I try to use koa-body in router, I get this exception:
error:  message=next is not a function, stack=TypeError: next is not a function
at /srv/.../node_modules/koa-body/index.js:110:14
at process._tickCallback (internal/process/next_tick.js:68:7)

in app level it works fine. My params:
koaBody({
    json: false,
    text: false,
})

@kaimiyang I'm still not exactly sure what the issue you are facing is. My speculation is that you aren't correctly initializing your router before trying to use it.
This minimal example works for me:
var log       = console.log,
    Koa       = require('koa'),
    app       = new Koa(),
    router    = require('koa-router'),
    koaBody   = require('../index'),
    port      = process.env.PORT || 4291
    host      = process.env.HOST || 'http://localhost';

/*!
 * Accepts only urlencoded and json bodies.
 */
router.post('/upload', koaBody(), async (ctx) => {
    console.log(ctx.request.body);
  }
);

app.use(router.routes());
app.listen(port);

log('Test with executing this commands:');
log('curl -i %s:%s/post/users -d "user=admin"', host, port);
log('Press CTRL+C to stop...');

@gugu please open a separate issue and include your version of both koa-body and koa-router.
Both of you, please see the koa-router example here.


Unexpected keyword with Polars from_dict

I get a type error coming from the following stack:
File "/Users/Library/Python/3.9/lib/python/site-packages/img2table/tables/processing/bordered_tables/lines.py", line 133, in remove_word_lines
df_lines = (pl.from_dicts(dicts=[line.dict for line in lines])
TypeError: from_dicts() got an unexpected keyword argument 'dicts'
Seems like the keyword may need to be data but have not seen anyone else have this issue?

Hi,
Seems like polars realeased a new update with some breaking changes. I'm gonna check it and make relevant modifications.

Amazing thanks!


Build failure: linuxKernel.packages.linux_xanmod_latest.ddcci-driver

Steps To Reproduce
Steps to reproduce the behavior:

build linuxKernel.packages.linux_xanmod_latest.ddcci-driver or linuxKernel.packages.linux_xanmod_stable.ddcci-driver

Build log
$ nix log /nix/store/bql6fb6gjwns4zhfq7nllv21a82lch06-ddcci-driver-6.4.1-0.4.3.drv
warning: The interpretation of store paths arguments ending in `.drv` recently changed. If this command is now failing try again with '/nix/store/bql6fb6gjwns4zhfq7nllv21a82lch06-ddcci-driver-6.4.1-0.4.3.drv^*'
@nix { "action": "setPhase", "phase": "unpackPhase" }
unpacking sources
unpacking source archive /nix/store/09wv22sw8v39n7msdkcvdh41vgz079wv-source
source root is source
@nix { "action": "setPhase", "phase": "patchPhase" }
patching sources
applying patch /nix/store/z8xd0s8k917agi98wprma324khzq4f4h-kernel-6.2-6.3.patch
patching file ddcci/ddcci.c
patching file dkms.conf
patching file include/linux/ddcci.h
@nix { "action": "setPhase", "phase": "updateAutotoolsGnuConfigScriptsPhase" }
updateAutotoolsGnuConfigScriptsPhase
@nix { "action": "setPhase", "phase": "configurePhase" }
configuring
no configure script, doing nothing
@nix { "action": "setPhase", "phase": "buildPhase" }
building
build flags: SHELL=/nix/store/a7f7xfp9wyghf44yv6l6fv9dfw492hd3-bash-5.2-p15/bin/bash O=\$\(buildRoot\) CC=/nix/store/x7n44lfys59k5ajj9w1fkxw5391cnn5v-gcc-wrapper-12.3.0/bin/cc HOSTCC=/nix/store/x7n44lfys59k5ajj9w1fkxw5391cnn5v-gcc-wrappe>
make -C "ddcci"
make[1]: Entering directory '/build/source/ddcci'
make -C "/nix/store/gmjjd5x694gm6avwni04l7dhd6xx7ddy-linux-6.4.1-dev/lib/modules/6.4.1-xanmod1/build" M=/build/source/ddcci modules
  CC [M]  /build/source/ddcci/ddcci.o
/build/source/ddcci/ddcci.c:43:34: error: macro "DEFINE_SEMAPHORE" requires 2 arguments, but only 1 given
   43 | static DEFINE_SEMAPHORE(core_lock);
      |                                  ^
In file included from /nix/store/gmjjd5x694gm6avwni04l7dhd6xx7ddy-linux-6.4.1-dev/lib/modules/6.4.1-xanmod1/source/include/linux/fs.h:25,
                 from /build/source/ddcci/ddcci.c:19:
/nix/store/gmjjd5x694gm6avwni04l7dhd6xx7ddy-linux-6.4.1-dev/lib/modules/6.4.1-xanmod1/source/include/linux/semaphore.h:34: note: macro "DEFINE_SEMAPHORE" defined here
   34 | #define DEFINE_SEMAPHORE(_name, _n)     \
      |
/build/source/ddcci/ddcci.c:43:8: error: type defaults to 'int' in declaration of 'DEFINE_SEMAPHORE' [-Werror=implicit-int]
   43 | static DEFINE_SEMAPHORE(core_lock);
      |        ^~~~~~~~~~~~~~~~
/build/source/ddcci/ddcci.c: In function 'ddcci_device_release':
/build/source/ddcci/ddcci.c:998:23: error: 'core_lock' undeclared (first use in this function); did you mean 'file_lock'?
  998 |                 down(&core_lock);
      |                       ^~~~~~~~~
      |                       file_lock
/build/source/ddcci/ddcci.c:998:23: note: each undeclared identifier is reported only once for each function it appears in
/build/source/ddcci/ddcci.c: In function 'ddcci_detect_device':
/build/source/ddcci/ddcci.c:1584:15: error: 'core_lock' undeclared (first use in this function); did you mean 'file_lock'?
 1584 |         down(&core_lock);
      |               ^~~~~~~~~
      |               file_lock
/build/source/ddcci/ddcci.c: At top level:
/build/source/ddcci/ddcci.c:43:8: warning: 'DEFINE_SEMAPHORE' defined but not used [-Wunused-variable]
   43 | static DEFINE_SEMAPHORE(core_lock);
      |        ^~~~~~~~~~~~~~~~
cc1: some warnings being treated as errors
make[3]: *** [/nix/store/gmjjd5x694gm6avwni04l7dhd6xx7ddy-linux-6.4.1-dev/lib/modules/6.4.1-xanmod1/source/scripts/Makefile.build:252: /build/source/ddcci/ddcci.o] Error 1
make[2]: *** [/nix/store/gmjjd5x694gm6avwni04l7dhd6xx7ddy-linux-6.4.1-dev/lib/modules/6.4.1-xanmod1/source/Makefile:2181: /build/source/ddcci] Error 2
make[1]: *** [Makefile:38: ddcci.ko] Error 2
make[1]: Leaving directory '/build/source/ddcci'
make: *** [Makefile:28: ddcci] Error 2

Additional context
Built from config in file
Notify maintainers
I apologize but there is no maintainer listed so I notify the last few contributors:
@erdnaxe @ElvishJerricco @sweenu
Metadata
 $ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 6.4.1-xanmod1, NixOS, 23.11 (Tapir), 23.11.20230722.12303c6`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.15.1`
 - nixpkgs: `/nix/store/gxw5rlp29swdpgz26vhaqzy21b9pxrim-source`

mr is available so it's a matter of fetching and patching

mr is available so it's a matter of fetching and patching

Hello, how do you apply a patch to a kernel module ?
I’ve tried
  nixpkgs.overlays = [
    (self: super: {
      linuxPackages_zen = super.linuxPackages_zen.extend (kself: ksuper: {
        ddcci_driver = ksuper.ddcci-driver.overrideAttrs (oldAttrs: {
          patches =
            (oldAttrs.patches or [])
            ++ [
              (super.fetchpatch {
                url = "https://gitlab.com/ddcci-driver-linux/ddcci-driver-linux/-/merge_requests/13.patch";
                hash = lib.fakeHash;
              })
            ];
        });
      });
    })
  ];

but can’t get it to work (it doesn’t change the package nor fetch the patch). I’ve searched for examples or tutorials but haven’t found any :confused:

This is what's working for me. (Also I wonder why it takes more than 2 months to merge this little change):
boot = {

    kernelPackages = pkgs.linuxPackages_xanmod_latest;

    extraModulePackages = with config.boot.kernelPackages; [
      (
        ddcci-driver.overrideAttrs (o: {
          patches = (o.patches or [ ]) ++ [
            ./ddcci_semaphore_fix.patch
          ];
        })
      )
    ];

    kernelModules = [ "ddcci" ];

I excluded irrelevant configurations.
I know you asked for a overlays but I wonder if that's really necessary for one time use.

Thank you @Mikilio  !

I know you asked for a overlays but I wonder if that's really necessary for one time use.

I wanted to write an overlay to avoid modifying my main config and adding it in my patch directory. But your solution will do it for now so I can use ddcci again 🎉

EDIT: for debugging I'd recommend reading what nixpkgs.overlay actually does and what the default value for nixpkgs.pkgs is

Thanks for the suggestion


Question for the support of dhg for heterogeneous hypergraph

I am wondering whether dhg supports hypergraph datasets with heterogeneous graphs, which means that for each hyperedge, there are several types of entities.

Absolutely, a heterogeneous hyperedge will have a more powerful modeling ability. It is a good direction for DHG's future work. Can you provide more materials? Some related links like code repositories, papers, and datasets of existing heterogeneous structures?

Thanks! I have no ability to offer materials of such characteristics, and looking forward to your findings!
------------------ 原始邮件 ------------------
发件人:                                                                                "iMoonLab/DeepHypergraph"                                                                                @.>;
发送时间: 2022年9月19日(星期一) 晚上7:08
@.>;
<EMAIL_ADDRESS>主题: Re: [iMoonLab/DeepHypergraph] Question for the support of dhg for heterogeneous hypergraph  (Issue #9)
Absolutely, a heterogeneous hyperedge will have a more powerful modeling ability. It is a good direction for DHG's future work. Can you provide more materials? Some related links like code repositories, papers, and datasets of existing heterogeneous structures?
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.Message ID: @.***>

There was a study on node classification task ,which proposed a graph neural network-based representation learning framework for heterogeneous hypergraphs. This work has been published in WSDM，named Heterogeneous Hypergraph Embedding for Graph Classification. It may give you some reference.


[Bug]

Summary
A clear and concise description of what the bug is, and what you expected to happen.
How to reproduce

Go to '...'
Click on '...'
Scroll down to '...'
See error

Please include screenshots.
Additional context
Add any other context about the problem here.

This might've been opened in error, so closing this out.


oxmysql docs outdated

Hi, my commit with fixes is not on page, someone need to build that.

Okay, so I'm dumb.
If it's possible, I would suggest create redirect from
https://overextended.github.io/oxmysql/
to
https://overextended.github.io/docs/oxmysql/Usage/
Or remove old docs for oxmysql.


LocktimeSoftware.NetLimiter4 version <IP_ADDRESS>

[x] Have you signed the Contributor License Agreement?
[x] Have you checked that there aren't other open pull requests for the same manifest update/change?

[x] Does your manifest conform to the 1.0 schema?

Note: <path> is the name of the directory containing the manifest you're submitting.

Microsoft Reviewers: Open in CodeFlow

Hopefully the failed validation is not caused by the CMD window which installer opens to install service.

Great it failed again and I don't now if it was once again error 1603 which I'm unable to reproduce in my VM.

@ItzLevvie Well automated testing once again failed probably on error 1603. So it looks like I have to contact Windows Package Manager team.

I'm adding here screen recording of successful installation on freshly installed VM.
https://user-images.githubusercontent.com/37749100/124811681-02671b00-df63-11eb-8fc1-19510d01aa66.mp4

Still failing in Sandbox with exit code 1603
--> Installing WinGet

                                                                                Tip: you can type 'Update-EnvironmentVariables' to update your environment variables, such as after installing a new software.                                                                                --> Installing the Manifest <IP_ADDRESS>
Found NetLimiter [LocktimeSoftware.NetLimiter4]
This application is licensed to you by its owner.
Microsoft is not responsible for, nor does it grant any licenses to, third-party packages.
Downloading https://www.netlimiter.com/files/download/nl4/netlimiter-<IP_ADDRESS>.exe
  ██████████████████████████████  7.19 MB / 7.19 MB
Successfully verified installer hash
Starting package install...
Installer failed with exit code: 1603

--> Refreshing environment variables

PS C:\Users\WDAGUtilityAccount\Desktop\winget-pkgs>

Still failing in Sandbox with exit code 1603
--> Installing WinGet

                                                                                Tip: you can type 'Update-EnvironmentVariables' to update your environment variables, such as after installing a new software.                                                                                --> Installing the Manifest <IP_ADDRESS>
Found NetLimiter [LocktimeSoftware.NetLimiter4]
This application is licensed to you by its owner.
Microsoft is not responsible for, nor does it grant any licenses to, third-party packages.
Downloading https://www.netlimiter.com/files/download/nl4/netlimiter-<IP_ADDRESS>.exe
  ██████████████████████████████  7.19 MB / 7.19 MB
Successfully verified installer hash
Starting package install...
Installer failed with exit code: 1603

--> Refreshing environment variables

PS C:\Users\WDAGUtilityAccount\Desktop\winget-pkgs>

@OfficialEsco well it looks I have to install windows setbox and try it in it because in VMs I don't have this behavior.

@OfficialEsco well it looks I have to install windows setbox and try it in it because in VMs I don't have this behavior.

Yeah for some reason Sandbox catches some of the bottom cases, however not all the time, i have no idea what the MS Team means by Clean VM when we also use bare bone VMs AND Sandbox where it still successes 🤷♂
I've also found some programs which require certain devices to be connected (gets passed trough to VM), but this shouldn't the issue here..
Anyways i found the issue
Just tested the Exe in interactive mode to see if there was anything that could cause the issue, nothing, however it installed VC++2015-2019 x64 and x86, i then uninstalled NetLimiter but not the VC++2015-2019 and it successes
--> Installing WinGet

                                                                                Tip: you can type 'Update-EnvironmentVariables' to update your environment variables, such as after installing a new software.                                                                                PS C:\Users\WDAGUtilityAccount\Desktop\winget-pkgs> winget install -m .\manifests\l\LocktimeSoftware\NetLimiter4\<IP_ADDRESS>\                       Found NetLimiter [LocktimeSoftware.NetLimiter4]
This application is licensed to you by its owner.
Microsoft is not responsible for, nor does it grant any licenses to, third-party packages.
Downloading https://www.netlimiter.com/files/download/nl4/netlimiter-<IP_ADDRESS>.exe
  ██████████████████████████████  7.19 MB / 7.19 MB
Successfully verified installer hash
Starting package install...
Successfully installed
PS C:\Users\WDAGUtilityAccount\Desktop\winget-pkgs>

I then uninstalled NetLimiter and VC++2015-2019
PS C:\Users\WDAGUtilityAccount\Desktop\winget-pkgs> winget install -m .\manifests\l\LocktimeSoftware\NetLimiter4\<IP_ADDRESS>\
Found NetLimiter [LocktimeSoftware.NetLimiter4]
This application is licensed to you by its owner.
Microsoft is not responsible for, nor does it grant any licenses to, third-party packages.
Downloading https://www.netlimiter.com/files/download/nl4/netlimiter-<IP_ADDRESS>.exe
  ██████████████████████████████  7.19 MB / 7.19 MB
Successfully verified installer hash
Starting package install...
Installer failed with exit code: 1603

Well I didn't think of that that failed installation might be caused by not installed VC++ 2015-2019. I on website of app developer there is no information that it's dependent on VC++ 2015-2019.

Well I didn't think of that that failed installation might be caused by not installed VC++ 2015-2019. I on website of app developer there is no information that it's dependent on VC++ 2015-2019.

Well you have the answer to this, is VC++2015-2019 x64 and x86 installed on your VM? Does it still work if you uninstall VC++2015-2019?
The EXE clearly installs VC++2015-2019 x64 and x86, it might not be dependent on it, but it probably tries to install it but fails.

VM in my screen recording is fresh VM from iso downloaded from Microsoft

VM in my screen recording is fresh VM from iso downloaded from Microsoft

take a Screenshot of Control Panel before and after Install please =)

OK I'll do from my main computer after I come home after work.

@OfficialEsco Well I did fresh install of Windows 10 in VM and it looks like MS started to include VC++ 2015-2019 directly in Windows.

@OfficialEsco Well I did fresh install of Windows 10 in VM and it looks like MS started to include VC++ 2015-2019 directly in Windows.

That most be a VMware thing, because VC++2015-2019 is never preinstalled on any of the PC's i've installed on, and my Virtualbox server with the same ISO also don't have VC++2015-2019
🤷♂

@OfficialEsco Well I did fresh install of Windows 10 in VM and it looks like MS started to include VC++ 2015-2019 directly in Windows.

That most be a VMware thing, because VC++2015-2019 is never preinstalled on any of the PC's i've installed on, and my Virtualbox server with the same ISO also don't have VC++2015-2019
🤷♂

So probably VMware Tools did it

So that's probably why installation was successful. So yeah there is VC++ dependency.

@OfficialEsco You were right VC++ was installed by VMware Tools

So I installed manually netlimiter on VM without VMware tools so no VC++ and installer automatically downloaded VC++ so yeah all along it was caused by VC++ dependency. But I thought there aren't any because a always had it installed in my VM.

So thank you for your help because I would probably never notice that VMware Tools installs VC++ automatically.

@OfficialEsco Can you mark this PR that it's waiting for dependency support please?

You'll have to ask @denelon; wait until Monday or so.

Please wait until Monday or so.

OK, no problem.

Well at least I tried to test if dependencies will work in insider version of winget. I'll probably change this PR to draft.


Update: 1.11.2

Pls update your mod for Forge 1.11.2`

The 1.11 version seems to work just fine in 1.11.2


Fix frame/contracts compatibility issues and re-include it in our runtime

I will handle this.

We're going to disable Ink! contracts for now, therefore this issue is closing.


fs: re-evaluating native module sources is not supported. If you are using the graceful-fs module, please update it to a more recent version.

when i run i get the following warning

Duplicate issue. Not upgrading.

@phated what is the original issue can you merge this with that so i can end up with some solution

@NikosEfthias https://github.com/gulpjs/gulp/issues?utf8=✓&q=is%3Aissue+is%3Aopen+graceful-fs


Unprotected pull from Docker Registry V2

I have V1 registry with nginx reverse proxy. My configuration allows docker pull without login and for any push login is needed. This all is working great till I started migrating to V2. With migration to V2 version of registry, i found below issues

GET requests also need to be protected for docker login to work
If I try to do a pull with every HTTP method protected for V2 registry and not logging in, the pull falls back to v1 as it gets 401 from /v2/ call instead of prompting for login. Image gets pulled from V2 registry perfectly if i do docker login manually and then do a pull.

Docker Registry Version - 2.1.1
Docker Version - 1.7.1
NGINX Version -  1.8.0
Anyone else facing similar issues with migration?

The V2 registry works somewhat differently from v1.  The recommended setup for putting a V2 registry behind nginx is detailed here:
https://github.com/docker/distribution/blob/master/docs/nginx.md

I have the registry running. As I am using same nginx instance for V1 and V2, the pull request falls back to v1 when v2 call returns 401 instead if prompting for login credentials. The image does exist in v2 registry. As GET/HEAD calls cannot be unprotected for V2 it makes every single call fall back to V1 unless user manually runs docker login before running the pull command.
Please let me know if this explains the scenario better.

I ran into the same thing, any one has any workaround or ideas for the question asked by @gauravbhandare

Until the client is made smart enough to tell the difference between 401 and 404, my solution is for Nginx to add this header:
        add_header 'Docker-Distribution-Api-Version' 'registry/2.0' always;

After auth, if you inspect the HTTP traffic, you'll see this header twice, because the registry adds it. It was too much effort to get Nginx to ensure it's only there once.

@mrwacky42 Dude you saved my life. About 4 days of very little sleep my private docker registry is up and running. If anyone wants an AWS cloudformation stack that is minimal and will cost $7/month to run your own docker registry let me know.

I am closing this issue.  If the header requirements are not clearly enough stated in the linked document, we always welcome PRs to improve clarity.
@ballerabdude : please refrain from advertising your business in our github issue tracker.

@RichardScothern i don't understand why the issue would be closed ? I feel this issue is still open, as pointed out by @gauravbhandare, i am running into the same thing. Having said that, this is not a documentation issue, but something that isn't working in the docker-registry v2, the way it uses GET/HEAD requests for authentication while doing a PUSH image. Is there a solution/workaround for this.!

@RichardScothern - I think he meant to say he's got a CF stack to spin up a registry that will cost $END_USER $7/month to run on AWS, not that he's providing that as a service.
As far as a solution, I think it's teaching the docker client that 401 is not a good reason to fall back to v1..

@mrwacky42 : it's off topic for the issue at hand.
@mv012004 did you follow the instructions in the linked document?

@RichardScothern i tried the NGINX configuration as mentioned by the link you provided. The same did not work as well and same issue persists. Its more because of the way Docker client handles the responses from V2 registry and falls back to V1. My current setup is NGINX fronted for Registry V1 and allows unprotected pulls but prompts for login for push requests. When i have Registry V2 also behind the same NGINX, since V2 has no way to unprotect only pull requests, if i protect pull for V2 then instead of prompting for login it falls back to V1 registry. Here is the complete nginx conf that I am using and works perfectly fine with Docker Registry V1. For V2, i face the issue mentioned in the issue description.
#user  nginx;
worker_processes  2;

error_log  /nginx01/error.log warn;

events {
    worker_connections  1024;
}

http {
    include       mime.types;
    default_type  application/octet-stream;
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /nginx01/access.log main;
    sendfile        on;
    keepalive_timeout  65;

    upstream docker-registry {
        server <Docker Registry V1 Primary host:port>;
        server <Docker Registry V1 Backup host:port> backup;

    }
    upstream docker-registry-v2 {
        server <Docker Registry V2 Primary host:port>;
        server <Docker Registry V2 Backup host:port> backup;
    }

    ldap_server ldapserver {
        url "<LDAP URL>";
        binddn "<BINDDN>";
        binddn_passwd "<BINDDN_PASSWORD>";
        group_attribute member;
        group_attribute_is_dn on;
        satisfy any;
        require valid_user;
    }
    server {
        listen       <PORT>;
        proxy_set_header  Host              $http_host;   # required for docker client's sake
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Real-IP $remote_addr;

        proxy_set_header X-Forwarded-Proto https;
        proxy_set_header X-Original-URI $request_uri;

        client_max_body_size 0; # disable any limits to avoid HTTP 413 for large image uploads

        # required to avoid HTTP 411: see Issue #1486 (https://github.com/dotcloud/docker/issues/1486)
        chunked_transfer_encoding on;

    location / {
        location /v1 {
            limit_except GET {
                auth_ldap "Restricted Space";
                auth_ldap_servers ldapserver;
            }

            proxy_pass http://docker-registry;
        }
                location /v2 {
                    add_header Docker-Distribution-Api-Version 'registry/2.0' always;
                        limit_except GET {
                                auth_ldap "Restricted Space";
                                auth_ldap_servers ldapserver;
                        }
                        proxy_pass http://docker-registry-v2;
                }
        }

        error_page  404              /errors/error.html;

        # redirect server error pages to the static page /50x.html
        #
        #error_page   500 502 503 504  /50x.html;
        #location = /50x.html {
        #    root   html;
        #}
    }
}

I did some more debugging and i guess the issue is because of the way Docker client initiates the calls for V1 and V2 registry versions
For V1, if user does a docker push, the first call is made with PUT/POST for /v1/repositories/xxx and as the same is protected the client gets 401 back and is prompted for login whereas for V2 the first call is GET /v2/ which returns 200 as my config has unprotected pull and the docker client considers that it can start pushing manifests and the PUT fails with 401. If first call is PUT/POST for docker push i guess everything will work as expected with Docker Registry V2 as well.
My overall intention is to have unprotected pulls so that anyone can run a container using the image from repository but any push needs to be protected so that only authorized users are pushing the upgrades.
Hope this gives better understanding.

@gauravbhandare - I scanned through the logs and observed the exact same behavior. I need to secure the push with authentication/authorization but want to allow everyone to pull images irrespective of doing docker login prior to pull, which makes sense and fits the same case as pulling images from Docker hub.

Little more debugging seems to point that the /auth on docker daemon is having some issue when the registry is configured with unprotected GET and returns 200 for /v2/ during docker login command execution.
Logs from Docker daemon
DEBU[0411] Calling POST /auth
INFO[0411] POST /v1.20/auth
DEBU[0411] hostDir: /etc/docker/certs.d/<REGISTRY_HOST:PORT>
DEBU[0411] pinging registry endpoint https://<REGISTRY_HOST:PORT>/v0/
DEBU[0411] attempting v2 ping for registry endpoint https://<REGISTRY_HOST:PORT>/v2/
DEBU[0412] attempting v2 login to registry endpoint https://<REGISTRY_HOST:PORT>/v2/
ERRO[0412] Handler for POST /auth returned error: no successful auth challenge for https://<REGISTRY_HOST:PORT>/v2/ - errors: []
ERRO[0412] HTTP Error                                    err=no successful auth challenge for https://<REGISTRY_HOST:PORT>/v2/ - errors: [] statusCode=500

Logs from the access log for NGINX fronting Registry V2
<IP> - - [17/Oct/2015:18:27:02 -0400] "GET /v2/ HTTP/1.1" 200 2 "-" "docker/1.8.2 go/go1.4.2 git- os/linux arch/amd64" "-"

So even if the registry is returning 200, the handling in auth.go seems to give 500 internal error even before the "trying "basic" auth challenge with params" log entry.
When the registry is protected for GET as well, the client logs show clean login.
DEBU[0368] Calling POST /auth
INFO[0368] POST /v1.20/auth
DEBU[0368] hostDir: /etc/docker/certs.d/<REGISTRY_HOST:PORT>
DEBU[0368] pinging registry endpoint https://<REGISTRY_HOST:PORT>/v0/
DEBU[0368] attempting v2 ping for registry endpoint https://<REGISTRY_HOST:PORT>/v2/
DEBU[0368] attempting v2 login to registry endpoint https://<REGISTRY_HOST:PORT>/v2/
DEBU[0368] trying "basic" auth challenge with params map[realm:Docker Registry - Restricted Space v2 scope:]

Trying to pinpoint the issue but hoping for some expect to take a look as well.

This needs adding new POST end point in Registry like V1 had /v1/users and using the same for login on client side instead of /v2/

This is a serious issue, with the docker client and probably need to be opened under docker-client however this needs to be taken look at, i am not sure why there is no activity on this issue. Can some of the devs look at this.
The issue seems to be with the docker login call doing a GET which is unprotected and since the GETs are allowed it assumed login successful even though the login has never happened, this needs to be fixed :|

@mv012004 it's a serious issue for us too: https://github.com/zalando-stups/pierone/issues/57

So, if i understand correctly, docker right now can not do unauthenticated pulls from registry and have authentication enable only in the push, correct? it must be "auth enable" for everything or "auth disabled" for everything.
As a workaround, one can create 2 sites configs, one for GET and HEAD requests (all other requests are denied) and unauthenticated. The other site config, using a different dns, allows every request and is always request authentication. Both of course share the same registry backend. This allows one public registry where only authenticated users can push, via a different address. Example:
dockerhub-ro.example.net - GET and HEAD without auth
dockerhub-rw.example.net - everything with auth
Not perfect but may help some people, specially since this thick had almost 1 year without any activity.
As a long term fix, docker should be fixed to be able to authenticate both PULL and PUSH requests independently of the other

@danielmotaleite yes, we are using exactly this workaround with two DNS domains pointing to the same registry and the server changing its behavior depending on Host header:

docker pull registry.opensource.zalan.do/stups/openjdk:latest
docker push registry-write.opensource.zalan.do/stups/openjdk:someversion

Hello @danielmotaleite,
Can you tell me how to use this workaround ?
I don't use nginx. I create two registry (2 different addresses / same registry backend) :

1 registry read-only mode, no authentication
1 registry RW, basic authentication

Do you think this method can cause problems ? Conflict between the 2 registry.
Thansk,
Best regards

@SpeedBlack what i have done is having ONE registry, being publish via 2 reverse-proxy, using 2 DNS
you say you have  "2 registry", but then say "same registry backend" , so i assume that the first reference, the "2 registry" is really a "reverse proxy"... i use nginx for reverse proxy but almost all reverse proxy servers that support authentication would work (haproxy, apache, nginx, etc)
If the registry backend is really the same, that is fine, that is the root solution from this workaround, it's just being presented in 2 different ways

Hello @danielmotaleite,
Ok, thanks ! After my tests it seems to work properly.
About the configuration, I do not know if I should use the same value for "REGISTRY_HTTP_SECRET" option.
In the documentation, it's written about "REGISTRY_HTTP_SECRET" :

A random piece of data used to sign state that may be stored with the client to protect against tampering. For production environments you should generate a random piece of data using a cryptographically secure random generator. If you omit the secret, the registry will automatically generate a secret when it starts. If you are building a cluster of registries behind a load balancer, you MUST ensure the secret is the same for all registries.

have you any idea ?
Thanks !

You need to set the REGISTRY_HTTP_SECRET when you have several backend servers working in a cluster. Having only one backend with multiple load-balancer sites will not need this.

Yes, I have installed 2 registries and both using the same storage:
Docker registry read only :
docker run -d
-p 5001:5000
–restart=always
–name registry-ro
-e REGISTRY_HTTP_HOST=registry-ro.example.com
-v /registry/TEST_RO_RW/:/var/lib/registry:ro
.....
.....
registry:2

Docker registry Read-write :
docker run -d
-p 5002:5000
–restart=always
–name registry-rw
-e REGISTRY_HTTP_HOST=registry-rw.example.com
-v /registry/TEST_RO_RW/:/var/lib/registry
.....
.....
registry:2

You write:

My problem with your setup is what happens when someone is uploading something and at same time, someone is reading the same image, but i have not enough knowledge about how registry works to guess. Try asking the developers and/or test your self

I tried multiple scenarios and it seems ok but can you explain to me exactly ?
I don't understand very well and I would like to trie.
Thanks a lot !

this way is not good idea because pull / push use url/imagename:version so we need layer7 routing .so nginx maybe do thant. or the private registry software has a option. but I don't know


add scatter plots for AOD AAOD at AERONET sites

This PR closes #571

Hi Yan @yfenganl and Chris @golaz, I have integrated the AODVIS and AODABS diagnostics in e3sm_diags. Let me know if you'd like to change or add anything. Thanks!
Here is an example output: https://portal.nersc.gov/cfs/e3sm/zhang40/tests/test_aeronet/viewer/aerosol_aeronet/index.html

@tomvothecoder thank you for reviewing and the suggestions. I will merge the PR once all tests pass.


Updated started-hello.md

I have changed two just in the file with appropriate alternative
Issue #184

Changes

Changed two just present in the file

Flags

Related Issues

Issue #
Pull Request #
This is my commit message

Signed-off-by: snehit31<EMAIL_ADDRESS>

Duplicate of #211


fix(docs): update docs for autocomplete to show correct type on classNames prop

Closes #3216
📝 Description
Components that classNames props had the wrong type specified on their documentation page. This fixes for all those components.
⛳ Current behavior (updates)

Please describe the current behavior that you are modifying

🚀 New behavior

Please describe the behavior or changes this PR adds

💣 Is this a breaking change (Yes/No):
No
📝 Additional Information
There are a lot of files but all have similar kind of change. Changed Record<UnionType, string> to Partial<Record<UnionType, string>> for each component classNames type.

Summary by CodeRabbit

New Features

Enhanced the Autocomplete component with new interactive elements, including a dropdown popover and a trigger button for improved usability.

@wingkwong please check and let me know if this works now. I will update for the rest of the components.

Cool. Will update with pointers and open a new PR. Thanks for the feedback.

@aayush-goyal actually you can just update in your branch and push. This PR will be updated automatically. Please don't close the PR and recreate a new one. This is hard to manage as reviewers need to check the previous comments and also the issue will link several closed PRs.

@aayush-goyal Any update?

Hi, running busy with office work. ETA - 2 weeks otherwise you can transfer this to other contributors. Thanks and sorry for the late reply.

@aayush-goyal are you still working on this PR?

No, please assign it to someone else.

I'll handle it.


ADLS behind a firewall

Could you please provide an example how to enable access to a firewalled ADLS Gen2 from Data Factory v2? I’ve managed to enable Polybase connectivity to the storage account using Managed Identity but the same approach does not seem to work for Data Factory.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 16b565fd-5303-cfcb-73f7-e2fd54a9c3a1
Version Independent ID: bcbd827a-09e5-3990-2a42-1a0d35cc9415
Content: Copy data to or from Azure Data Lake Storage Gen2 using Data Factory
Content Source: articles/data-factory/connector-azure-data-lake-storage.md
Service: data-factory
GitHub Login: @linda33wj
Microsoft Alias: jingwang

@mrphinche  Thanks for reaching out. At this time we are reviewing your request and will get back to you soon.

@mrphinche Since the Data Factory isn't treated as a trusted Microsoft Service yet, and if you want to access your ADF to connect to ADLS Gen2, please use a Self Hosted Integration Runtime in the Linked Service that connects to ADLS Gen2 account. Below are the things to be considered.

The Self Hosted IR should be hosted on a Static IP.
Then the static IP address has to be added to the firewall enabled ADLS Gen2 account. To do this go to "Firewalls and virtual networks" then to "Allow access from selected networks" and under "Firewall" section add the Static IP address in which Self Hosted IR is hosted and "save" the settings.

Now in your ADF pipeline, while creating the Linked Service, select Self Hosted IR hosted on Static IP, and ADLS Gen2 account in which your Static IP is added. Then click on test connection (this test connection should be successful).

Finally, once your linked service test connection is successful, then you can use it in your ADF pipeline for data integration to and from ADLS Gen2.
Currently we do not have a specific document which clearly show how to copy data to or from Firewall Enabled Azure Data Lake Storage Gen2 using Azure Data Factory.
Please feel free to provide your idea/thought about having a document for this specific scenario (copy data to or from Firewall Enabled Azure Data Lake Storage Gen2 using Azure Data Factor) in User voice forum: Feedback forum, which will allow the community to upvote and for the product team to include into their plans.
Hope this helps....

@KranthiPakala-MSFT, thank you for the detailed explanation. I have finally succeeded after adding the virtual network that the IR VM is attached to to the firewall rules ... it seems it would work even without a static IP..

@mrphinche Glad to know that your issue is resolved. Yes, it will work adding virtual network of IR VM to the firewall rules We really appreciate for bringing this to our notice and sharing your solution. This will help others a lot.
We will now proceed to close this thread. If there are further questions regarding this matter, please tag me in your reply. We will gladly continue the discussion and we will reopen the issue.
Thank you!

@KranthiPakala-MSFT - I have similar issue. I have to use AzureIR. I have ADLS Gen2 and firewall settings with VNET but MS trusted services access checked. Now, i am trying to access it from data factory but test connection is failed. I was recommended to get Data Factory Managed Identity and give it role based access as Blob storage contributor on ADLS but it doesnt find guid as showed in the picture.

@sharmamanish260979 Thanks for reaching out. Could you please give a try using your Data Factory name in the search instead of Managed Identity Application ID (Guid)?

@sharmamanish260979 Here is a MSDN thread which might be helpful in your case.
https://social.msdn.microsoft.com/Forums/en-US/ea386f5e-2d5d-4f9c-8ead-28e6d2aad374/storage-accoung-v2-access-with-firewall-vnet-to-data-factory-v2?forum=AzureDataFactory
Hope this helps.
Thanks.


fix(cmd/rpc): Rework all methods that take variadic params

Unfortunately, go-openrpc can't handle methods that accept variadic params. We should rework all these methods to accept the Options struct(as was proposed in https://github.com/celestiaorg/celestia-node/pull/2630)

We should aim to fix variadics so that we have non-breaking way to add Options struct to each method

We don't have any methods that use variadic params on our RPC so I think this can be closed

No, we need want variadics for non-breaking options

@Wondertan Can we start working on this?

This is a tricky one, but I believe it is valuable, so I would happily review it

@Wondertan is there any specific endpoints need to support non-breaking options ? From my review, might be state package or anything else in mind?


allowing platform specification through flag

allows platform specification through new flag --platform
if not platform defaults to local (empty string)
we should use --platform linux/amd64 for csci350 (not sure which one for csci104)

Thanks for looking it over! Committed your suggestion :) -- really enjoyed working on this though, it's a great tool you made!

Great, thanks! I'll need to update the README there once I minor version bump this to v0.3.7 so this gets built into the latest release.


(android) Fix issue #525

Platforms affected
Android
Motivation and Context
This solves issue https://github.com/apache/cordova-plugin-file/issues/525 which is a serious problem.
Closes apache/cordova-plugin-file#525
Description
This implements the fix described in https://github.com/apache/cordova-plugin-file/issues/525#issue-1265882820
Testing
I tested locally, pushed to production. Approximately 1000 installs working with no issues.
Checklist

[ ] I've run the tests to see all new and existing tests pass
[ ] I added automated test coverage as appropriate for this change
[ ] Commit is prefixed with (platform) if this change only applies to one platform (e.g. (android))
[x] If this Pull Request resolves an issue, I linked to the issue in the text above (and used the correct keyword to close issues using keywords)
[x] I've updated the documentation if necessary

Was this tested against both use cases of using either http/https scheme and the file protocol?

Is there any way for me to help push this forward?
I haven't tested this with https, only with "default" capacitor configurations which is http and localhost.

Please, fix this ASAP, because it's blocking some new projects


[100] Fixed FindBugs DM_DEFAULT_ENCODING

I agree that UTF-8 is probably the best character set to use. I just wonder if there are places using some other set, like Charset.defaultCharset() ? At some point we may want to consider investigating that.


Kerberos Support for On-prem Kerberized Oracle databases

Describe the Feature
I am trying to use this adapter to connect to my Oracle database which only supports kerberos for authentication and username/password is not supported. Can we please add this feature to the adapter or show me how to leverage this if already implemented ?
Describe alternatives you've considered
No response
Who will this benefit?
All users of a kerberized Oracle database
Anything else?
No response

We are investigating this. We use python-oracledb as the underlying database driver. So, any external authentication supports needs to be supported by the driver

As I understand python-oracle db does support kerberos auth in thick mode

@aadishsanghvi
Could you let me know how do you connect to Oracle Database using SQLPlus and Kerberos Authentication ? I am interested in understanding the SQL net configuration used. Also, have you tried the thick mode ?
I am trying to setup Kerberos instance to test the setup but that might take a few days.

I am not sure about sql plus. but I have an oracle instance which supports only kerberos authentication, I was able to connect to it using cx_Oracle which is also a python library very similar to python-oracledb, this requires an instant client which has a sqlnet.ora file configuration pointing to the location of your krb5.conf file and the keberos credential cache file, then you use the drivers provided by the cx_oracle and define your instance's dsn etc etc.

@aadishsanghvi
I know of cx_Oracle and python-oracledb. Could you share the code using cx_Oracle to connect using Kerberos ?

Hi @aadishsanghvi
We are working on it. The plan is to make this a part of dbt-oracle==1.9.0 which we plan to release in January 2025.


support for parsing single event in EPCIS JSON document

Hi @Echsecutor,
As per latest updates in openapi.yml, added support to parse single event contained epcis json document.

Hi @Echsecutor, Just a friendly reminder, please take a look at this PR and let me know if you have any review comments.

Hi @Echsecutor, Just a friendly reminder, please take a look at this PR and let me know if you have any review comments.

Hi @RalphTro, This branch can also be merged as it has no impact on any existing feature and it will be a useful feature to support a single EPCIS event in JSON format.

Dear @dakbhavesh ,
Thanks! I just had a look into this and it worked when I run our script via the command line tool for the event you added ("python3 epcis_event_hash_generator/main.py -p tests/examples/epcisDocWithSingleEvent.json")
I noticed the same bug your other PR is about (i.e. that eventID is to be ignored). Will have a look into this soon as well.
eventType=ObjectEventeventTime=2005-04-04T02:33:31.116ZeventTimeZoneOffset=-06:00epcListepc=urn:epc:id:sgtin:0614141.107346.2017epc=urn:epc:id:sgtin:0614141.107346.2018action=OBSERVEbizStep=https://ns.gs1.org/voc/Bizstep-shippingdisposition=https://ns.gs1.org/voc/Disp-in_transitreadPointid=urn:epc:id:sgln:0614141.07346.1234bizTransactionListbizTransaction=http://transaction.acme.com/po/12345678type=https://ns.gs1.org/voc/BTT-poeventID=_:event1

Dear @RalphTro, I have fixed the trivial issue. Just needed to update the hash and prehash for a single event example.  Pushed changed directly to master as it was pretty straightforward.

@dakbhavesh sorry for not looking into this due to time constraints. @RalphTro many thanks for taking over!


Copy definition of placement new to <wil/wistd_memory.h>

Elsewhere in WIL, placement new is used regardless of whether <new> is included or not. Since we can't safely #include <new>, we copy its definition of placement new (and placement new[]) into <wil/wisdt_memory.h>

Nevermind. This library is more complex than I thought!


how do i declare global mappings ?

Hi,
I like to prevent inlining packages imported like "react" and "@wordpress/components" BUT reference them from global variables.
An example: import { Button } from ''@wordpress/components'; should be transpiled to const { Button } = windows.wp.components;
microbundle allows "something like that" to declare globals as commandline option and also a way to do control at least inlining via convention : https://github.com/developit/microbundle/wiki/How-Microbundle-decides-which-dependencies-to-bundle
How can I do this with nanobundle ?
Kind regards,
Lars

Hi @lgersman !
Since v0.0.18, nanobundle has "standalone mode", which tries to inline all dependencies. In standalone mode, you can specify explicitly which dependency will not be inlined.
I'm not sure this option and microbundle's option you mentioned is met your requirement.
Can you provide some esbuild or rollup config you use for it?

Hello @cometkim,
the reason for my question is that i would like to use your nanobundle to bundle Wordpress Gutenberg blocks. These blocks consists of Javascript running in the browser inside the Gutenberg Editor.
The Gutenberg Editor provides React and many of the @wordpress/* packages via global variable windows.wp (Example: import { Button } from ''@wordpress/components'; is available in the browser via const { Button } = windows.wp.components;).
That means that these packages dont need to be bundled but transpiled to access the import using global variables (like window.React and window.wp.components).
in Rollup you would declare that like this :
const config = {
    input: 'foo.js,
    output: {
      file: `dist/foo.js`,
      format: 'iife',
      globals: {
          react: 'React',
          '@wordpress/components': 'wp.components'
      },
      ....
    },

The transpilation is also possible using esbuild (see here : https://github.com/lgersman/ix-article-esbuild-2021/blob/develop/bin/esbuild-bundle.mjs#L64)

Ok, I understand your requirement.
I'm trying to keep the config of nanobundle as simple as possible. I could also explore a good place for additional config per dependency in package.json (e.g. peerDependencyMeta), but I think import maps would be a cleaner way to do this.
vendor/wp-components.js
export const {
  Button,
} = window.wp.components;

import-maps.json or package.json
{
  "imports": {
    "@wordpress/components": "./vendor/wp-components"
  }
}

Good morning,
I understand your reasons (simplicity !) but some packages may expose dozens of exports ... which will result in a huge amount of import map code.
Don't you think there is a better way to expose globals ?

Of course, simplicity is important, but there is another big problem here.
Adding personal opinions without esbuild config already existing or well-known standards is against the goal of the nanobundle.
Maybe it makes sense to provide an escape hatch that allows custom plugins for other cases.

Sounds good.
Hyeseong Kim @.***> schrieb am Do., 23. Dez. 2021,
15:49:

Of course, simplicity is important, but there is another big problem here.
Adding personal opinions without esbuild config already existing or
well-known standards is against the goal of the nanobundle.
Maybe it makes sense to provide an escape hatch that allows custom plugins
for other cases.
—
Reply to this email directly, view it on GitHub
https://github.com/cometkim/nanobundle/issues/12#issuecomment-1000353004,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAT5NY5YXC4MTPCV2ETJR3USMZGNANCNFSM5KG2R7ZQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you were mentioned.Message ID:
@.***>


feat: clickable url for azd template show

The command azd template list gt the feature of clickable repository paths in release 1.4.2. This PR brings this feature also to the command azd template show xyz transfering the logic implemented in #2845 to the show command.
To avoid code duplication the formatting logic is trenafered to the path.go file which looks like a good fit for this kind of logic.
The sequence of the output of the template show command was not changed. If this should be aligned with the template list command, just comment and I add a commit

/azp run azure-dev - cli

Tested locally:

/azp run azure-dev cli

/azp run azure-dev - cli


chore: delete internal/manifest

internal/maniefst only holds the transitive scanning code for Maven pom.xml which is now moved to internal/scalibrextract/language/java/pomxmlnet

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 67.13%. Comparing base (faad98e) to head (7b71505).
Report is 5 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1462      +/-   ##
==========================================
- Coverage   67.23%   67.13%   -0.11%
==========================================
  Files         192      192
  Lines       18169    18161       -8
==========================================
- Hits        12216    12192      -24
- Misses       5308     5313       +5
- Partials      645      656      +11

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


DSND-2371: Implement handling of top level annotations

Describe the changes

Top level annotations are persisted using the entity_id whereas child annotations use the parent_entity_id.
Top level annotations also have more top and data level fields set compared to child annotations.

Related Jira tickets
DSND-2371
Developer check list
General

[ ] Is the PR as small as it can be?
[ ] Has the Jira ticket been updated with any relevant comments?
[ ] Is the README up to date?
[ ] Has the POM been updated for the latest versions of dependencies?
[ ] Has the code been double-checked against main?
[ ] Does the code adhere standards in this repository, including SonarQube checks?

Testing

[ ] Do the code changes have unit and integration tests with code coverage > 80%?
[ ] Has the code been tested locally and/or passed the API Karate tests on Tilt?
[ ] Have mandatory manual test cases been run?
[ ] Are extra Karate tests required?
[ ] Are all the test data resources up to date?

Release preparation
Where possible, add links to the respective Jira tickets when an item is checked

[ ] Have these changes been included in a release ticket?
[ ] Are changes required to the docker-chs-development deployment or test data?
[ ] Are any changes required to the environment added to deployment repositories?

Notes to the tester
Add any testing hints or instructions here.

CI: No security warnings found

CI: Security warnings found!


LruCache locks don't work

The locks are released within the if-statement before executing any of the logic that should be locked
code: https://github.com/Syndica/sig/blob/69b9a8e871698371902cb1b60a1d3f046a502c4d/src/common/lru.zig#L124C17-L124C41
something like this should work:
if (kind == .locking) self.mux.lock();
defer if (kind == .locking) self.mux.unlock();

Ah yes defer is block scoped in Zig, not function scoped. Can you push a PR/fix?


Fix network ci/allinone

rebuild

Ansible <IP_ADDRESS> change notes says "Added a version check for jinja2-2.9, which does not fully work with Ansible currently.", pinning Ansible back to <IP_ADDRESS> will unblock people until upstream Ansible works again with Jinja.

centos branch is now pinned so you should be able to rebase and retest.

retest

retest

retest

retest

retest

retest

ok to test


Update metrics docs

The metrics docs currently are outdated compared with the scaffold project.
Following some recommendations.

Rename General metrics -> Configuring controller-runtime metrics
Rename Custom resource specific metrics -> Configuring metrics about custom resources
Move and rename Expose Custom Metrics -> Subsection of Configuring controller-runtime metrics called Defining custom metrics
New section called Exposing metrics, that includes snippets for service creation and service monitor creation, and includes the Garbage collection section since that is referring to the service that gets created.

/remove-lifecycle stale.

Closed in the bug triaged - We decide that it will probably be affected by the integration with SDK and then, these docs will be updated anyway.


enable the use of iaas-specific kernels

Use IAAS_KERNEL environment variable to identify the use of, and label for, iaas specific kernels (e.g. aws)
This somewhat replicates how fips was handled in the bionic branch, but with an attempt to be more generic.

@toabctl Is this PR ready for review? asking because there currently still is active discussion.

Thanks! @rthill91


Driver does not terminate on CTRL+C

When pressing CTRL+C to terminate the driver, the driver stops publishing messages but does not shutdown properly.
Ubuntu 20.04, official apt package for ROS Noetic, v1.06

Sometimes CTRL+C needs to be pressed multiple times, and even then, the shutdown can take some time. Could you perhaps try with v1.0.8 and confirm that it keeps reoccurring?


Order of the atoms in the supercell in the FORCE_CONSTANTS format

Dear developers,
What is the order of atoms that phonopy expects for the FORCE_CONSTANTS format?
This is relevant for atoms in the supercell in the full format, how are the coordinates of atoms in the supercell decided?
I am trying to interface my code (https://github.com/SSCHAcode/CellConstructor) with phonopy, however I struggle to understand how to generate the FORCE_CONSTANTS file properly in supercells.
My simple wrapper works correctly if the supercell is 1,1,1, however, it fails in bigger supercells, suggesting that I am missing the correct assignment of atoms in the supercell.
From the guide https://phonopy.github.io/phonopy/input-files.html#force-constants-and-force-constants-hdf5 it is not very clear, especially as, as far as I understand, the structure files only contain the information about the primitive cell.
All the best,
Lorenzo

I don't see what you mean about the order of atoms.

The correspondence between the atomic index and the actual position of the atom in the supercell:
Let us say the primitive cell has 5 atoms, and we have a 2x2x2 supercell. in the Full format we have
Full format: (n_satom, n_satom, 3, 3)
n_satoms = 40
However, atom index 7, what atom is?  how are atoms ordered with respect to the supercells?
How do atom index i correspond to the atom in the primitive cells? How are supercells ordered?

Let us have a list of atoms in primitive cell, [1, 2, 3, 4]. Then we want to create its 2x2x2 supercell. Then does what you want to know is something like [1, 2, 3, 4] * 8 or [1] * 8 + [2] * 8 + [3] * 8 + [4] * 8, or something else? Have you read Sec. 3 of this paper https://iopscience.iop.org/article/10.1088/1361-648X/acd831?

If you start with supercell and give a transformation matrix to the primitive cell, phonopy is designed to work for arbitrary order of atoms in supercell. In case the order of atoms in primitive cell matters, the order is determined as follows. Given a list of atoms in supercell, there is a mapping of atoms in supercell to those in primitive cell by lattice translations. The firstly found atom among translationally equivalent atoms in supercell is appended to the list of atoms in primitive cell.

Now, if the supercell is constructed by non-diagonal matrix, the story is not very simple. Could you define your question more robustly?


Turkish Characters Support

I developed an Unreal Engine 5 XML creation and parser plugin based on TinyXml2 and you can access it from
https://github.com/FF-Plugins-Docs/FF_TinyXml2
But when I want to get an XML to a string with these codes
Open XML = https://github.com/FF-Plugins-Docs/FF_TinyXml2/blob/main/Source/FF_TinyXml2/Private/FF_TinyXml2BPLibrary.cpp#L32
Print XML = https://github.com/FF-Plugins-Docs/FF_TinyXml2/blob/main/Source/FF_TinyXml2/Private/FF_TinyXml2BPLibrary.cpp#L144
my Turkish characters comes broken.
I saw an issue like this and maybe it is related with my problem.
https://github.com/leethomason/tinyxml2/issues/944

TInyXML-2 really doesn't have any language / local support. It tries to be UTF-8 pure. Without a specific example of the UTF-8 getting munged up, there's not much I can do.


azure ml studio experiements published to azure ai gallery are no longer loading

I saved off each of my experiments, so that I could load them into a different azure machine learning studio workspace.  When I try and visit any of their unadvertised gallery links I see the experiment title come up but then there is a spinning progress wheel that just keeps spinning and never finishes.  Same result using edge and chrome.  Is this a known new issue as I wasn't having this issue a week or two back with this process for migrating an experiment from one workspace to another?
https://gallery.azure.ai/Home/Author?authorId=3C2A952C5E2841C814CFCD5C90FA0CC6D203FA239201F2378092FF6FCF75A5B2
https://gallery.cortanaintelligence.com/Experiment/ob1-capstone-project-v3
https://gallery.cortanaintelligence.com/Experiment/ob1-capstone-project-v3-Predictive-Exp
https://gallery.cortanaintelligence.com/Experiment/ob1-capstone-project-v3b-Predictive-Exp
https://gallery.cortanaintelligence.com/Experiment/ob1-capstone-project-v2
https://gallery.cortanaintelligence.com/Experiment/ob1-capstone-project-v2-Predictive-Exp
https://gallery.cortanaintelligence.com/Experiment/ob1-capstone-project
https://gallery.cortanaintelligence.com/Experiment/ob1-capstone-project-Predictive-Exp

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: b1f9dd8c-f7f7-1c47-33b5-fa8ac6c0858b
Version Independent ID: 5f12856e-1caa-89b0-0552-06bf8faa35d2
Content: Overview of Azure Batch AI service
Content Source: articles/batch-ai/overview.md
Service: batch-ai
GitHub Login: @alexsuttonms
Microsoft Alias: asutton

@myusrn Glad to see you! Thank you very much for reporting the issue here. We are currently investigating and will update you shortly.

@myusrn Thanks for the feedback! I have assigned the issue to the content author to investigate further and update the document as appropriate.
@jpconnock This is a product issue when try to open above experiments. Can you please help and check it? And also please update the document as necessary. Thanks a lot!

Hi @YutongTie-MSFT & @jpconnock
Any updates on this issue?  I just retried today and still repro'ng the same issue.
I'm looking to use this to move training and predictive experiments from a azure ml studio workspace I want to delete over to one I'm looking to keep.
I noticed that the "experiments" view in the one workspace I want to move these items to now has a "copy to workspace" option on actions bar across the bottom that would allow me to do what I want w/o having to use the publish to gallery followed by open into new workspace from gallery process.  The problem is that "copy to workspace" option isn't showing up in the workspace I want to pull these experiments out of.  Any insights as to why that is?

@YutongTie-MSFT - Is this issue reproducible? This is not a documentation issue but a product issue that should be resolved on the forum.
@myusrn - sorry for the long delay. this issue was mis-categorized and slipped through the cracks.

@myusrn Could you please send an email to me at<EMAIL_ADDRESS>We would like to work with you if you are still suffering from this issue. Thank you.

I sent email to<EMAIL_ADDRESS>address as requested above.  This issue still repros for me with the links I provided above but never heard in this issues exchange if clicking any of those links repros the issue for you.

@YutongTie-MSFT - Thanks for helping @myusrn directly.
@myusrn - You can also post questions like these in the Azure Machine Learning Forum .
#please-close


Notification type "OPSGenie" - Unable to send notifications via a proxy

⚠ Please verify that this bug has NOT been raised before.

[X] I checked and didn't find similar issue

🛡 Security Policy

[X] I agree to have read this project Security Policy

Description
Hello,
With the notification integration type "OPSGenie", it's impossible to transmite notification through a proxy because the integration is not able to talk correctly to http proxy.
We see direct requests "POST https://api.eu.opsgenie.com/v2/alerts" in the logs, this is not supported.
The integration has to first open a tunnel using CONNECT method: e.g.: "CONNECT api.eu.opsgenie.com:443 HTTP/1.0"
A "curl" command is automatically do this, for example.
Please, could you add this possibility ? We use Kuma behind a proxy
Thank you in advance
👟 Reproduction steps
Configure the notification integration "OPSGenie" and use is through a proxy
👀 Expected behavior
The notification integration "OPSGenie" should be able to send notification through a proxy.
😓 Actual Behavior
Error 503 when we test to send a notification because the proxy refuse the connection
🐻 Uptime-Kuma Version
1.23.3
💻 Operating System and Arch
Portainer integration
🌐 Browser
Not relevant
🐋 Docker Version
No response
🟩 NodeJS Version
No response
📝 Relevant log output
No response

Not a bug, but a feature request.
None of the notification providers use the proxy settings
If you'd like to contribute such a feature see our contribution guide

Duplicate of #616


Back Translate 3Di Tokesn to PDB Format

Hello, I have a few questions which might be very basic to others.

Can we convert the 3Di tokens produced by foldseek into the actual 3D structure format like PDB files?

How can I compare two sequences of predicted and true 3Di tokens? Can I use something like the TM Score for that? I am working on a model to predict 3Di tokens from amino acids and am searching for the best metric to evaluate the model.

I would be very greatful to whom can answer my questions.

Hi,
1.) no, from what I know this is not possible, yet.
2.) TM-score requires 3D structures for 3D-alignment and distance computation. So if you only have predicted 3Di, this won't be possible. For comparing true/predicted 3Di, I usually used simple classification accuracy. If you want to make it a bit more complicated, you can also compute alignment of true/predicted 3Di using the 3Di-substitution matrix provided by Foldseek.


⚠ Processors has degraded performance

In ad27240, Processors ($STATUS_URL) experienced degraded performance:

HTTP code: 200
Response time: 48 ms

Resolved: Processors performance has improved in d5441fc.


Deprecated<EMAIL_ADDRESS>

We should not need this or we would need to find an alternative?

#249

done

I don't think it's resolved?
Also I think we could at least consider removing it as I'm not aware of many usage?

actually it was just not published 😅  will create another discussion about removing it


design improvements for tiny style

the corner radius for the (badge) background were off, bothered me 😆
have a lovely day 💕

before:

after:


Add unit tests for more helper funcs

🔧 Changes
This PR adds unit tests for the helper funcs of internal/cli/users_roles.go and internal/cli/custom_domains.go.
Also, the logic of the interactive auth0 users roles picker helpers was refactored for the sake of testability and less code duplication. This allowed unit-testing code that previously was not testable.
🔬 Testing
The refactoring of the auth0 users roles picker helpers was tested manually in addition to the unit tests that were added.
auth0 users roles add
https://user-images.githubusercontent.com/5055789/231333315-5d617010-cc2b-488f-81ac-f1f35ff824c6.mov
auth0 users roles remove
https://user-images.githubusercontent.com/5055789/231333465-d52a35d5-5751-4bd3-9441-f5596fff1b2d.mov
📝 Checklist

[X] All new/changed/fixed functionality is covered by tests (or N/A)
[X] I have added documentation for all new/changed functionality (or N/A)

Codecov Report
Patch coverage: 87.80% and project coverage change: +0.81 :tada:

Comparison is base (b1fc4cb) 68.62% compared to head (5424989) 69.44%.

:exclamation: Current head 5424989 differs from pull request most recent head eea68bb. Consider uploading reports for the commit eea68bb to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #728      +/-   ##
==========================================
+ Coverage   68.62%   69.44%   +0.81%
==========================================
  Files          87       87
  Lines       11239    11228      -11
==========================================
+ Hits         7713     7797      +84
+ Misses       3014     2919      -95
  Partials      512      512

Impacted Files
Coverage Δ

internal/cli/users_roles.go
75.87% <87.80%> (+26.50%)
:arrow_up:

... and 2 files with indirect coverage changes
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


[requires.io] dependency update on 1.0.0 branch

Leaving this as a reminder to drop requires.io and double-check that pyup.io integration is working


wip: POC of checksums using bindesc

This is a proof of concept demonstrating how binary descriptors could be used to store a checksum/build-id of the executable.
This was first requested here: #51532.
Essentially the steps taken are:

Create binary descriptors for the checksums, filling them with zeros.
Compile normally, until the final zephyr.elf is linked.
Run a script on the ELF, that collects all the raw data into an intermediate image.
Scan the symbols to see what hashes the user is interested in.
Calculate those hashes over the intermediate image, and modify the symbols inside zephyr.elf to those values.
Continue regularly and create bin, hex and uf2 files.

Another important note is that the rom_start section is not included in the checksum calculation, because that section contains the binary descriptors. This ensures that the build time is not included in the calculation, producing consistent checksums on consecutive builds.
This was tested on native_posix and rpi_pico.
I'm submitting this as a draft, as there are still some questions to be answered:

Should we modify zephyr.elf directly, or create a new ELF file?
Should the build system itself set the checksums, or should it be done by the user like west sign?
Does the method I use to calculate the hash capture everything that should be checksum-ed, and exclude everything that shouldn't? How portable is this? Should we just use objcopy?

Example output:
APP_VERSION_STRING "1.0.0"
BUILD_DATE_TIME_STRING "2023/10/01 17:22:43"
C_COMPILER_NAME "GNU"
C_COMPILER_VERSION "12.2.0"
CHECKSUM_MD5_BYTES b'&\\\r\x9d-|-\x10\xa2\xb1VC6Y\xf4;'
CHECKSUM_MD5_STRING "265c0d9d2d7c2d10a2b156433659f43b"
KERNEL_VERSION_MAJOR 3
KERNEL_VERSION_STRING "3.4.99"
0x2003 b'\x01\x02\x03\x04'
0x0002 5
0x1001 "Hello world!"

Extra feature: added the option to make the descriptors space of constant size, giving the descriptors room to change their sizes without affecting the checksum. For example, these two images have the same checksum, even though the my_string descriptor is of different lengths:
CHECKSUM_MD5_STRING "9a4e796a0530ae18d28099122c849ce9"
0x1001 "Hello world!"

CHECKSUM_MD5_STRING "9a4e796a0530ae18d28099122c849ce9"
0x1001 "Hello world!!!!!"


Pin FileCheck To A Version

I'm informed that SwiftPM's dependency resolution is not a fan of projects with transitive dependencies that both depend on top-of-tree projects.  Experiment with pinning to a minimum version of FileCheck to see if we can't work that out.

⛵


Q: can this be used for a generic JDBC driver?

Thank you

Yes.
Most thoroughly used with Oracle and Vertica databases. Unit tests however are written for MySQL as that is more available.

Thank you @japettyjohn
So glad I've found you! :)
Hoping to be able to test it in the next month. Cool.

closing it :) TX!


Embed files associated with the document as a whole

Resolves #4411
Part of #2499
This PR implements file embeddings associated with the document as a whole. The attachments do not have visible annotations. It also adds an option for PDF/A-3 to allow arbitrary file embeddings with PDF/A compliance. I will gladly move this to a second PR if preferred.
The embeddings are part of a new pdf module:
This will embed the file without a description using the file path for the name annotation
#pdf.embed("text.txt")

We can also set the name and description of the embedding
#pdf.embed("text.txt", name: "file.txt", description: "An embedded file")

Todos:

[ ] Support embedding bytes

#embed is causing pagebreaks for me when using before #set page("a4") or after #pagebreak(weak: true).

Reproduction
#embed.decode("bill-reports.pdf", "test.txt")
#set page("a4")
= header
#pagebreak(weak: true)
#embed("bill-reports.typ")

Hope that's the right place for this comment

@sirchnik the implementation was changed with some pointers from LaurenzV on Discord. The layouting issues should be fixed now.
One thing I haven't managed so far, but I still plan to do, is to move the embed methods into a PDF scope (so it's #pdf.embed(test.txt) and #pdf.embed.decode("some file content")).

I would prefer to drop the decode variant. We plan to deprecate the decode functions and instead allow passing bytes directly to the individual functions instead.

From the proposal it's not completely clear yet, if and how it would be possible to link to these files. Is it just possible to add a label to the #pdf.embed call? Or is this not possible from a PDF point of view anyway?

Embedded file streams can be included in PDF 1.7 in two different ways:

associated with the whole file (this is what this PR implements)
associated with a specific location in the PDF with a File Attachment Annotation (<IP_ADDRESS> in ISO 32000-1)

The second would give you a clickable name, description and icon in the PDF file to download the attachment. These annotations could be added in a follow-up PR and point to the same embedded file streams that this PR adds to the PDF export.

@laurmaedje Would it be OK to keep the decode variant here and leave a replacement for all the decode methods to a follow-up PR?

I guess we can do that. Is the PR ready for review?

I will add back the decode variant then (in a couple hours) :+1:
Apart from that the PR is ready, yes.

@laurmaedje everything is pushed; ready for review.

Here are some initial comments, for now mostly from a code perspective rather than a PDF perspective.

Thank you! All comments are addressed via commit and/or response.

All comments are addressed; ready for review.


Words per row and Word count improper when we have multiple text columns

This is the result I got for titanic dataset and it looks improper.

Oh no! This is not good. Will add to list to fix.


Add types for Afterpay/Clearpay payment method

It would be awesome to have defined types for the Afterpay/Clearpay payment method. 😁

v2.0.1 includes these new types<EMAIL_ADDRESS>


Missing gpiod_line_request_input_flags

Without support for gpiod_line_request_input_flags (which perhaps should be an additional argument in the requestInputMode) its not possible to set a pull up or pull down on the line.
This is the only thing preventing this library from being almost a drop-in replacement (that I have noticed) for the capabilities offered by most sysfs based APIs.

Any plans for this?

feature should be available on 0.4.0 Release, anyone able to test it please provide feedback. here.

https://github.com/sombriks/node-libgpiod/pull/21

Nice! I can certainly test it this week.

From a usability / API perspective I would have that method called "requestInputFlags" as it does not set the mode of the GPIO as requestInputMode as does.
Other than that it seems to work as designed.

From a usability / API perspective I would have that method called "requestInputFlags" as it does not set the mode of the GPIO as requestInputMode as does.
Other than that it seems to work as designed.

That's nice!
we can enhance the API design in the upcoming 1.x version. it's done for now.


Error when using Evolu with Vite

I am trying to use Evolu with Vite and I am getting the following errors:

In dev server: loading dep (outdated optimize dep) when loading DbWorker.worker.js
When trying to build:  Invalid value "iife" for option "output.format" - UMD and IIFE output formats are not supported for code-splitting builds.

You can replicate the error here (just run npm run dev / npm run build in the repo):
https://github.com/kaladivo/evolu-vite-error

Hi, thank you for the report. I planned to add a Vite example anyway, so I will fix it soon.

Done.
https://github.com/evoluhq/evolu/tree/main/examples/vite

Not yet. Only the dev is working. The build still has to be fixed.

Related https://github.com/evoluhq/evolu/issues/262


What is the alleged security problem addressed in the README

The somewhat clickbait header in the README appears to claim that symmetric JWS with HMAC/SHA-256 possesses a security problem. I am curious what the authors claim to be the problem here exactly as it is not elaborated in the text whatsoever  and the RFC yield said signature method a svalid.

Hi @marwalsch, thanks for your question.
I'll address some of them:

According to NIST HMAC/SHA-256 should have at least a 64 bit key
According to NIST You need to rotate keys at least every year. Althought the best practices says to roll it every 3 months.
You should use a Random number generator to generate the key. So a SUPER SECRET STRING at appsettings isn't recommended.
You should keep the key in a safe place.
You shouldn't share the crypto keys with another API's to validate a JWT. So HMAC is the worst choice for distributed applications

Its not a problem with algorithm itself. Í challenge you to try to find a good example at Google, the vast majority of examples are teaching to create JWT at dotnet without any care about the aspects of what really matters: The key.

Thanks for the response, a few thoughts on that:

The given sample actually does not reveal the secret's le length. AFAIK regarding secret keys for HMAC & SHA256:

exceeding the block size's length of 512 is discouraged as the secret itself is hashed;
the generally recommended size is equal to the hash output length, i.e. 256 (which is generally enough);
-> 64 bit seems to offer too little entropy, and if you refer to bytes, 'at least' would be not beneficial (see bulletpoint 1).

Agreed, but the sample does not necessarily imply that Settings.Secret is not rotated (might be an overridden getter).

We also don't know whether the key was generated through strong pseudo-random algorithms beforehand.

Agreed, Settings.Secret might be in-memory or an abstracted key store itself though, again the sample does not reveal that.

There is absolutely nothing wrong with using HS256 to sign tokens if there is no third party required to validate but merely the issuer itself. I think it's actually to be preferred in such an environment as sloppy implementation of RS256 validation possesses huge pitfalls like erroneously accepting tokens signed with the public key (it's also a lot slower).

These are just my 2cents. I generally agree with your concerns, but I think the given image is quite misleading as it might imply to some (as myself) that HMAC & SHA-256 or its given implementation in .NET possesses major flaws  causing security issues, which is not the case.

@marwalsch thanks for your comments!
tl;dr
In a way we'll disagree that image is misleading in a sense to alert about a .NET flaw. The image direct point to some aspects of code which leads to the points described above. So developers are getting bad examples and they clearly don't know about. But I'm not a guy who dies with my convictions and ideas. If you have other idea or suggestion about how to alert developers about the risks of generating a Key in the same way that everyone is teaching at first page of Google, please share and I'll be glad to change.
The long version:
My fight is against the bad examples about generate JWT through internet. The image is kind of alert for those who got an example about generate a JWT at Google.
There is no example in Microsoft docs about generating a JWT at ASP.NET Core. (Or maybe I didn't search too deep). So every developer who got stuck on it will search at Google. And I'll share the first page of results and some links, no one tells the risks of the key or generate a key using RNGCryptoServiceProvider.
Examples:

ASP.NET Core Web API - Creating And Validating JWT (JSON Web Token) - March, 2020
Creating And Validating JWT Tokens In ASP.NET Core - Jan 2020
Asp Net Core 5 Rest API Authentication with JWT Step by Step - Jan 2021

Every blog post in this image don't mention anything about key and how to generate it.

My fight is against the bad examples about generate JWT through internet.

I noticed the lack of disclaimers about bad practices applied for the sake of brevity in otherwise verbose blogs myself. Though my general assumption in good faith was people being aware that minimal examples shouldn't be placed in production without considering all the mentioned aspects such as algorithms, secret storage, etc; and I salute your attempt to make people reconsider.
Thus my suggestions would be:

Remove the hints in the image referencing "string" and "Hmac" as these are, from a stand-alone point of view, no indicators of insecurity;

replace them and perhaps the snipped with something that directly addresses the actual concerns, namely primarily the points 2-4 mentioned beforehand (1 and 5 are still debatable if you ask me).

add a chapter to the readme explaining what exactly is wrong with the given example (essentially exactly what you did in your first comment on my issue). While I think it's right and necessary to address any security flaws, it might not be the right way to just offer a solution but rather explain what exactly the problems are in the first place.


2.0 - Developers should be able to build protos within installing using setup.py

When build_protos.py was moved into a library, the ability to just build the protos was lost. This functionality is needed for local development and testing. I recommend adding a new command to setup.py:
...
class BuildProtosCommand(install):
"""Post installation command."""
def run(self):
    build_protos()

cmdclass={
    'install': PostInstallCommand,
    'build_protos': BuildProtosCommand,
},

Resolved in #1050


Empty/Too much Expressions

What is the correct output for "if":[]? I would assume false, but some interpretations say null. Same for {"if":[false, "apple", false, "banana", false, "carrot"]} (too much expressions)

Both cases I'd deem to be undefined behaviour. The “2*n + 1”-operands use of if exists in JsonLogic, but I wouldn't be happy to overload if with when/switch functionality.

Shall we keep it on nil/null then?

Returning nil/null would make it defined behaviour.
To be sure: you mean: what's the evaluation result of an if where the guard (the 1st argument) evaluates to the empty array []?

Yes. I mean "if": [] and "all":[], "some":[] All kind of empty expressions return currently null/nil.

An if on a non-truthy/falsy guard should be undefined behaviour, to avoid weird corner cases.
all and some are not in CertLogic, but can be implemented using a reduce. A reduce on an empty main operand evaluates to the initial operand.

@SchulzeStTSI Is this enough of an answer? / Can I close the issue?

@dslmeinte Yes we can close it. JSON Logic Swift passes all CertLogic Unittests now.