Multiple exports of "X" found, only the last one will be kept

Attempting to unpack/unminify some webpacked code (Ref):
⇒ cd ./unpacked/_next/static/chunks

⇒ npx @wakaru/unpacker pages/_app.js -o ./pages/_app-unpacked/
Generated 213 modules from pages/_app.js to pages/_app-unpacked (24,049ms)

⇒ npx @wakaru/unminify ./pages/_app-unpacked/* -o ./pages/_app-unminified
# ..snip..

I got a bunch of warnings(?) similar to the following while attempting to unminify; though it's not clear if these warnings relate to the module listed before them, or after them:
• Transforming pages/_app-unminified/pages/_app-unpacked/module-10538.js (361.8ms)
Multiple exports of "default" found, only the last one will be kept
• Transforming pages/_app-unminified/pages/_app-unpacked/module-10608.js (64.1ms)

.. snip..

• Transforming pages/_app-unminified/pages/_app-unpacked/module-138.js (153.3ms)
Multiple exports of "getUserCacheKey" found, only the last one will be kept
• Transforming pages/_app-unminified/pages/_app-unpacked/module-13957.js (290.8ms)

etc

The 'multiple exports' log seems to happen in unminify's replaceWithExportDeclaration:
https://github.com/pionxzh/wakaru/blob/e2ff1ba7e455b1f31a869e1bcefb9bab3fa71964/packages/unminify/src/transformations/un-esm.ts#L401-L407
The 'Transforming' log seems to happen in unminify's CLI, and appears after the call to runDefaultTransformation (which calls runTransformations, which uses transformationMap, which includes unEsm, which contains transformAST, which calls transformExport, which calls replaceWithExportDeclaration)
This implies that the warning before the module applies to that module..

Full list of the different warnings + modules that caused them
Multiple exports of "default" found, only the last one will be kept
• Transforming pages/_app-unminified/pages/_app-unpacked/module-10608.js (64.1ms)

Multiple exports of "getUserCacheKey" found, only the last one will be kept
• Transforming pages/_app-unminified/pages/_app-unpacked/module-13957.js (290.8ms)

Multiple exports of "default" found, only the last one will be kept
• Transforming pages/_app-unminified/pages/_app-unpacked/module-16861.js (163.4ms)

Multiple exports of "default" found, only the last one will be kept
• Transforming pages/_app-unminified/pages/_app-unpacked/module-18458.js (212.8ms)

Multiple exports of "default" found, only the last one will be kept
• Transforming pages/_app-unminified/pages/_app-unpacked/module-21121.js (147.7ms)

Multiple exports of "default" found, only the last one will be kept
• Transforming pages/_app-unminified/pages/_app-unpacked/module-21508.js (55.2ms)

Multiple exports of "Base64" found, only the last one will be kept
• Transforming pages/_app-unminified/pages/_app-unpacked/module-27046.js (283.1ms)

Multiple exports of "default" found, only the last one will be kept
• Transforming pages/_app-unminified/pages/_app-unpacked/module-36386.js (97.5ms)

Multiple exports of "checkCookies" found, only the last one will be kept
• Transforming pages/_app-unminified/pages/_app-unpacked/module-44544.js (892.6ms)

Multiple exports of "STORAGE_PREFIX" found, only the last one will be kept
• Transforming pages/_app-unminified/pages/_app-unpacked/module-49787.js (94.5ms)

Multiple exports of "default" found, only the last one will be kept
• Transforming pages/_app-unminified/pages/_app-unpacked/module-50201.js (2,190.6ms)

Multiple exports of "ExceptionEndpoint" found, only the last one will be kept
• Transforming pages/_app-unminified/pages/_app-unpacked/module-5421.js (1,131.6ms)

Multiple exports of "StatsigInvalidArgumentError" found, only the last one will be kept
• Transforming pages/_app-unminified/pages/_app-unpacked/module-5502.js (238.5ms)

Multiple exports of "default" found, only the last one will be kept
• Transforming pages/_app-unminified/pages/_app-unpacked/module-60925.js (160.4ms)

Multiple exports of "default" found, only the last one will be kept
• Transforming pages/_app-unminified/pages/_app-unpacked/module-65860.js (93.2ms)

Multiple exports of "default" found, only the last one will be kept
• Transforming pages/_app-unminified/pages/_app-unpacked/module-6631.js (202.8ms)

Multiple exports of "default" found, only the last one will be kept
• Transforming pages/_app-unminified/pages/_app-unpacked/module-67523.js (105.4ms)

Multiple exports of "default" found, only the last one will be kept
• Transforming pages/_app-unminified/pages/_app-unpacked/module-67860.js (75.8ms)

Multiple exports of "default" found, only the last one will be kept
• Transforming pages/_app-unminified/pages/_app-unpacked/module-69270.js (89.2ms)

Multiple exports of "default" found, only the last one will be kept
• Transforming pages/_app-unminified/pages/_app-unpacked/module-70479.js (123.6ms)

Multiple exports of "default" found, only the last one will be kept
• Transforming pages/_app-unminified/pages/_app-unpacked/module-71083.js (576.5ms)

Multiple exports of "default" found, only the last one will be kept
• Transforming pages/_app-unminified/pages/_app-unpacked/module-74861.js (51.2ms)

Multiple exports of "default" found, only the last one will be kept
• Transforming pages/_app-unminified/pages/_app-unpacked/module-76226.js (123.2ms)

Multiple exports of "default" found, only the last one will be kept
• Transforming pages/_app-unminified/pages/_app-unpacked/module-77415.js (107.2ms)

Multiple exports of "UnsupportedStrategy" found, only the last one will be kept
• Transforming pages/_app-unminified/pages/_app-unpacked/module-80456.js (2,691.6ms)

Multiple exports of "default" found, only the last one will be kept
• Transforming pages/_app-unminified/pages/_app-unpacked/module-84019.js (130.3ms)

Multiple exports of "default" found, only the last one will be kept
• Transforming pages/_app-unminified/pages/_app-unpacked/module-8629.js (83.7ms)

Multiple exports of "default" found, only the last one will be kept
• Transforming pages/_app-unminified/pages/_app-unpacked/module-91921.js (75ms)

Multiple exports of "default" found, only the last one will be kept
• Transforming pages/_app-unminified/pages/_app-unpacked/module-94355.js (54.4ms)

Multiple exports of "default" found, only the last one will be kept
• Transforming pages/_app-unminified/pages/_app-unpacked/module-98421.js (65.1ms)

Multiple exports of "difference" found, only the last one will be kept
• Transforming pages/_app-unminified/pages/_app-unpacked/module-98480.js (80.3ms)

I haven't yet looked through the individual modules that caused warnings, but I wanted to open this issue as a placeholder for adding any more info about them that seems relevant as I do.

Multiple exports of "default" found, only the last one will be kept
• Transforming pages/_app-unminified/pages/_app-unpacked/module-10608.js (64.1ms)

This is the code within module-10608.js that is triggering the warning:
function B() {
  throw TypeError(
    "Invalid attempt to destructure non-iterable instance.\nIn order to be iterable, non-array objects must have a [Symbol.iterator]() method."
  );
}
(module.exports = B),
  (module.exports.__esModule = !0),
  (module.exports.default = module.exports);

Other modules with similar patterns to this:

pages/_app-unminified/pages/_app-unpacked/module-16861.js

// ..snip..
(module.exports = B),
  (module.exports.__esModule = !0),
  (module.exports.default = module.exports);

pages/_app-unminified/pages/_app-unpacked/module-18458.js

Mostly the same, but maybe slightly different

pages/_app-unminified/pages/_app-unpacked/module-21121.js
pages/_app-unminified/pages/_app-unpacked/module-21508.js
pages/_app-unminified/pages/_app-unpacked/module-36386.js
pages/_app-unminified/pages/_app-unpacked/module-60925.js
pages/_app-unminified/pages/_app-unpacked/module-65860.js
pages/_app-unminified/pages/_app-unpacked/module-67523.js
pages/_app-unminified/pages/_app-unpacked/module-67860.js
pages/_app-unminified/pages/_app-unpacked/module-70479.js
pages/_app-unminified/pages/_app-unpacked/module-84019.js
pages/_app-unminified/pages/_app-unpacked/module-8629.js
pages/_app-unminified/pages/_app-unpacked/module-91921.js
pages/_app-unminified/pages/_app-unpacked/module-94355.js
pages/_app-unminified/pages/_app-unpacked/module-98421.js
pages/_app-unminified/pages/_app-unpacked/module-98480.js
etc

Multiple exports of "getUserCacheKey" found, only the last one will be kept
• Transforming pages/_app-unminified/pages/_app-unpacked/module-13957.js (290.8ms)

module-13957.js has the following:
"use strict";;
Object.defineProperty(exports, "__esModule", { value: !0 }),
  (exports.getUserCacheKey = exports.getHashValue = exports.SimpleHash = void 0);

// ..snip..

(exports.SimpleHash = J), (exports.getHashValue = X), (exports.getUserCacheKey = Q);

pages/_app-unminified/pages/_app-unpacked/module-27046.js
"use strict";;
Object.defineProperty(exports, "__esModule", { value: !0 }),
  (exports.Base64 = void 0),
  (exports.Base64 = {
      // ..snip..
      return exports.Base64._encodeBinary(G);
    },
  });

pages/_app-unminified/pages/_app-unpacked/module-5502.js
// ..snip..

Object.defineProperty(exports, "__esModule", { value: !0 }),
  (exports.StatsigInvalidArgumentError = exports.StatsigUninitializedError = void 0);

// ..snip..

exports.StatsigInvalidArgumentError = V;

Multiple exports of "STORAGE_PREFIX" found, only the last one will be kept
• Transforming pages/_app-unminified/pages/_app-unpacked/module-49787.js (94.5ms)

This seems incorrect?
export const OVERRIDES_STORE_KEY =
  "STATSIG_LOCAL_STORAGE_INTERNAL_STORE_OVERRIDES_V3";
export const STICKY_DEVICE_EXPERIMENTS_KEY =
  "STATSIG_LOCAL_STORAGE_STICKY_DEVICE_EXPERIMENTS";
export const INTERNAL_STORE_KEY = "STATSIG_LOCAL_STORAGE_INTERNAL_STORE_V4";
export const STATSIG_STABLE_ID_KEY = "STATSIG_LOCAL_STORAGE_STABLE_ID";
export const STATSIG_LOCAL_STORAGE_LOGGING_REQUEST_KEY =
  "STATSIG_LOCAL_STORAGE_LOGGING_REQUEST";

export const LOCAL_STORAGE_KEYS = {
  STATSIG_LOCAL_STORAGE_STABLE_ID: true,
  STATSIG_LOCAL_STORAGE_INTERNAL_STORE_V4: true,
  STATSIG_LOCAL_STORAGE_STICKY_DEVICE_EXPERIMENTS: true,
  STATSIG_LOCAL_STORAGE_INTERNAL_STORE_OVERRIDES_V3: true,
  STATSIG_LOCAL_STORAGE_LOGGING_REQUEST: true,
};

export const STORAGE_PREFIX = "STATSIG_LOCAL_STORAGE";

I didn't look too closely at these remaining ones, some of them may fit into similar patterns as above, but they also might be unique cases:
Multiple exports of "checkCookies" found, only the last one will be kept
• Transforming pages/_app-unminified/pages/_app-unpacked/module-44544.js (892.6ms)

Multiple exports of "default" found, only the last one will be kept
• Transforming pages/_app-unminified/pages/_app-unpacked/module-50201.js (2,190.6ms)

Multiple exports of "ExceptionEndpoint" found, only the last one will be kept
• Transforming pages/_app-unminified/pages/_app-unpacked/module-5421.js (1,131.6ms)

Multiple exports of "default" found, only the last one will be kept
• Transforming pages/_app-unminified/pages/_app-unpacked/module-6631.js (202.8ms)

Multiple exports of "default" found, only the last one will be kept
• Transforming pages/_app-unminified/pages/_app-unpacked/module-69270.js (89.2ms)

Multiple exports of "default" found, only the last one will be kept
• Transforming pages/_app-unminified/pages/_app-unpacked/module-71083.js (576.5ms)

Multiple exports of "default" found, only the last one will be kept
• Transforming pages/_app-unminified/pages/_app-unpacked/module-74861.js (51.2ms)

Multiple exports of "default" found, only the last one will be kept
• Transforming pages/_app-unminified/pages/_app-unpacked/module-76226.js (123.2ms)

Multiple exports of "default" found, only the last one will be kept
• Transforming pages/_app-unminified/pages/_app-unpacked/module-77415.js (107.2ms)

Multiple exports of "UnsupportedStrategy" found, only the last one will be kept
• Transforming pages/_app-unminified/pages/_app-unpacked/module-80456.js (2,691.6ms)

This is kind of expected because some bundlers will assign void 0 to the exports.[target] first, transforming them can be unsafe. But it would be good if we could detect this pattern and silence the warnings as they can be considered safe.

Why the example of STORAGE_PREFIX is incorrect to you?

But it would be good if we could detect this pattern and silence the warnings as they can be considered safe.

@pionxzh Yeah, that was sort of my thoughts in collecting/grouping all of the warnings; as it seems like a lot of them follow a general pattern that would probably be safe to silence the warning for.

Why the example of STORAGE_PREFIX is incorrect to you?

@pionxzh I originally thought that I was looking at/provided the unpacked (but not unminimised) code above, because it didn't seem to have any duplicate identifiers; but seems that was actually the unminimised code (just checked again).
Looking at the actual unpacked code, it seems to be another case of the void 0 duplicate pattern

module-49787.js
Unpacked:
"use strict";
Object.defineProperty(exports, "__esModule", { value: !0 }),
  (exports.STORAGE_PREFIX =
    exports.LOCAL_STORAGE_KEYS =
    exports.STATSIG_LOCAL_STORAGE_LOGGING_REQUEST_KEY =
    exports.STATSIG_STABLE_ID_KEY =
    exports.INTERNAL_STORE_KEY =
    exports.STICKY_DEVICE_EXPERIMENTS_KEY =
    exports.OVERRIDES_STORE_KEY =
      void 0),
  (exports.OVERRIDES_STORE_KEY =
    "STATSIG_LOCAL_STORAGE_INTERNAL_STORE_OVERRIDES_V3"),
  (exports.STICKY_DEVICE_EXPERIMENTS_KEY =
    "STATSIG_LOCAL_STORAGE_STICKY_DEVICE_EXPERIMENTS"),
  (exports.INTERNAL_STORE_KEY = "STATSIG_LOCAL_STORAGE_INTERNAL_STORE_V4"),
  (exports.STATSIG_STABLE_ID_KEY = "STATSIG_LOCAL_STORAGE_STABLE_ID"),
  (exports.STATSIG_LOCAL_STORAGE_LOGGING_REQUEST_KEY =
    "STATSIG_LOCAL_STORAGE_LOGGING_REQUEST"),
  (exports.LOCAL_STORAGE_KEYS = {
    STATSIG_LOCAL_STORAGE_STABLE_ID: !0,
    STATSIG_LOCAL_STORAGE_INTERNAL_STORE_V4: !0,
    STATSIG_LOCAL_STORAGE_STICKY_DEVICE_EXPERIMENTS: !0,
    STATSIG_LOCAL_STORAGE_INTERNAL_STORE_OVERRIDES_V3: !0,
    STATSIG_LOCAL_STORAGE_LOGGING_REQUEST: !0,
  }),
  (exports.STORAGE_PREFIX = "STATSIG_LOCAL_STORAGE");

Unminimised:
export const OVERRIDES_STORE_KEY =
  "STATSIG_LOCAL_STORAGE_INTERNAL_STORE_OVERRIDES_V3";
export const STICKY_DEVICE_EXPERIMENTS_KEY =
  "STATSIG_LOCAL_STORAGE_STICKY_DEVICE_EXPERIMENTS";
export const INTERNAL_STORE_KEY = "STATSIG_LOCAL_STORAGE_INTERNAL_STORE_V4";
export const STATSIG_STABLE_ID_KEY = "STATSIG_LOCAL_STORAGE_STABLE_ID";
export const STATSIG_LOCAL_STORAGE_LOGGING_REQUEST_KEY =
  "STATSIG_LOCAL_STORAGE_LOGGING_REQUEST";

export const LOCAL_STORAGE_KEYS = {
  STATSIG_LOCAL_STORAGE_STABLE_ID: true,
  STATSIG_LOCAL_STORAGE_INTERNAL_STORE_V4: true,
  STATSIG_LOCAL_STORAGE_STICKY_DEVICE_EXPERIMENTS: true,
  STATSIG_LOCAL_STORAGE_INTERNAL_STORE_OVERRIDES_V3: true,
  STATSIG_LOCAL_STORAGE_LOGGING_REQUEST: true,
};

export const STORAGE_PREFIX = "STATSIG_LOCAL_STORAGE";

Reopen, there are still some false alerts that can be improved.


[DO NOT MERGE] CI test for PR

Checking if CI works for pull requests. DO NOT MERGE.

CI does not seem to work if the PR is from a forked repository, as they cannot access the GitHub secrets in main repo.


Update blade.indexerpostprocessor to fix errors on server startup

Hi Tibi,
The liferay-blade-samples project: blade.indexerpostprocessor currently throws NullPointerException after a server restart.
This pull request is fixing that error, could you please review the solution?
Many thanks,
Peter

Closing as per https://issues.liferay.com/browse/LRDOCS-2893?focusedCommentId=1033219&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-1033219


support for canvas upload

Upload to canvas
canvas has their own metadata not in gradescope which, presumably, needs to match for upload.  we'll ask that user's download a canvas gradebook to get this metadata as input.  Call should be similar to:
gradescope_mean.to_canvas grade_full.csv canvas.csv

build a new csv from grade_full.csv, match "SID" to "SIS User ID"
give output
3.  warn if canvas has a missing student not in grade_full
4. note if grade_full has student not in canvas

CRN & term from command line
output to xls directly
update canvas.md with instructions on uploading ot multiple sections

also, print some warning about canvas's default 2 decimal precision which rounds.  (really frustrating at end of semester when our grading policy doesn't round ... )

an earlier comment requests xls for canvas, this was a typo.  its banner which needs an xls file (see #11) we won't support xls here


Add missing quote in README.md

[x] All commits have been signed for the Developer Certificate of Origin. See https://github.com/chef-cookbooks/community_cookbook_documentation/blob/master/CONTRIBUTING.MD

Thanks

Thanks


Clean up plural handling of ONE vs. =1, easier way to include the howMany parameter

ICU plurals have two separate cases for =1 and ONE, where the former means precisely 1 and the second is anything singular, e.g. 21 in Russian. And =0 is similar.
Right now we have only a "one" named parameter on Intl.plural. This is by default emitted as an =1 into the translation file, but is used for any singular case. So there are a couple of issues.

Provide the ability to specify both. But we can't use "1" or "=1" as a parameter name, so a good name is tricky.
Backward compatibility for existing messages.
Fix up documentation - in particular messages of the ONE form should include the howMany parameter in the text.

Related to this is that using the howMany parameter is awkward. If you just use it directly in an interpolation then it just uses the toString(), it's not formatted according to the locale rules. Since it's an integer and typically small this is usually not too bad. But for locales using non-Latin digits it will be a problem. And for large numbers the grouping separators will be wrong.  Imagine a "There are 1.234 new messages"
It's possible we could do some magic where Intl.plural does the formatting and passes the string as an extra parameter, and generates corresponding code. That wouldn't work for the non-generated case, but if we assume without loss of generality that developers are fine with en_US formatting it might not be too bad. e.g.
e.g.
plural(howMany, a, b, c, ...) {
var formattedHowMany = ...
return _plural(howMany, formattedHowMany, a, b, c);
}
Or alternatively we treat it like we do select with enums, where we always get the string format of the parameter and use that to match. It would mean that the plural logic for choosing would have to work with the locale-specific format, which is ugly, but less intrusive in code generation and it would work uniformly.

I have a clever plan, which may even work.
I think we can generate code in the translations that will format the number as a string and use that as the parameter, but pass the number down to the selection logic. In the non-translated fallback case you can just interpolate the parameter, and it will print as an integer in the default locale. If we assume that's US English  or at least that the number formatting of int.toString() matches the developer's locale, then that will work. And even in the case of a fallback translation it will do it properly. Something like
foo(n, other) => Intl.plural(n, one: 'There are $n', ...

... and in generated code...

m123(somePrivateName, other) {
   var n = formatCorrectlyForLocale(somePrivateName);
   return Intl.plural(somePrivateName, one: 'There are $n', ...
}

So we don't modify the user's strings or variable names. They still interpolate the variable 'n', it just isn't the original parameter in the generated code.

Closing in favor of #411 .


Duplicate changes of items that didn't change via source redfish

Hello!
I'm using script to collect information every 2 hours from redfish source (HPe servers) to JSON files and then run a netbox-sync that using multiple configs (2 vCenters + redfish) to add info to NetBox:
15 */2 * * * /opt/check_redfish/redfish-sync.sh >/dev/null 2>&1
23 */2 * * * /opt/netbox-sync/.venv/bin/python3 /opt/netbox-sync/netbox-sync.py -c /opt/netbox-sync/redfish-config.ini /opt/netbox-sync/vcenter-new-config.ini /opt/netbox-sync/vcenter-old-config.ini >/dev/null 2>&1
In change log I see a lot of changes for hardware, but inside those changes only one thing that changes is added a duplicate tag
to object:

Config of redfish source (redfish-config.ini):
[common]
log_level = DEBUG
log_to_file = True
log_file = /opt/netbox-sync/logs/netbox_sync.log

[netbox]
api_token = XXXX
host_fqdn = netbox.localdomain.local
port = 443
validate_tls_certs = False
prune_enabled = True
prune_delay_in_days = 7
timeout = 30
max_retry_attempts = 4
use_caching = True
cache_directory_location = cache

[source/redfish-sclabs]
enabled = True
type = check_redfish
inventory_file_path = /opt/check_redfish/inventory/
permitted_subnets = <IP_ADDRESS>/24
overwrite_power_supply_name = True
overwrite_power_supply_attributes = True

Any ideas why we have duplicates of tags and how to prevent that? I'm trying to clean over 6 million records from change log :D

Hey @koznov,
thank you for the bug report. Good question. Just wondering:

any reason why you have split your config into multiple files?
why it shows Source: redfish SCLABS in the screenshot and redfish-sclabs in the config example?
is a the same tag added on each run?
can you run the script with -l DEBUG2 option and send me an example where NetBox gets updated with the same tag twice in on request. That might clarify things.

Thank you

Hey Ricardo,
Thanks for answering.

any reason why you have split your config into multiple files?
Only for having different configuration and easier to make changes in every file. I can join them together for easier debugging if you want :-)
why it shows Source: redfish-SCLABS in the screenshot and redfish-sclabs in the config example?
That's a good question and I cannot answer on it. Config that is used for redfish sync has string [source/redfish-sclabs]:

is a the same tag added on each run?
Yes, all changes that are made related to redfish has the same pattern with adding the same tag to objects with Type Inventory Item (RAM, PSU, NIC, etc) - all objects that are taken from json files of servers.

can you run the script with -l DEBUG2 option and send me an example where NetBox gets updated with the same tag twice in on request. That might clarify things.
Sure, attaching full log of command:
/opt/netbox-sync/.venv/bin/python3 /opt/netbox-sync/netbox-sync.py -c /opt/netbox-sync/redfish-config.ini /opt/netbox-sync/vcenter-sclabs-config.ini /opt/netbox-sync/vcenter-norislab-config.ini -l DEBUG2 ><EMAIL_ADDRESS>

issue-363.log

Maybe should I try to clear cache folder and then try again to sync, and check after that again?

Hi,
I checked your log and it's rather strange and should not happen. Also curios why the source tag character case in name changes. Maybe you would like to try out the latest development version.

No luck,
Cloned development branch, copied configs, and run once and then again - same result. Logs are attached.
issue-363.log

I suppose that duplicating also related not only to Redfish, but to vCenter source as well:

Adding also configs for analyzing:
akoznov@netbox:~$ egrep -v '^;|^$' /opt/netbox-sync/redfish-config.ini
[common]
log_level = DEBUG
log_to_file = True
log_file = /opt/netbox-sync/logs/netbox_sync.log
[netbox]
api_token = XXX
host_fqdn = netbox.sclabs.cloud
port = 443
validate_tls_certs = False
prune_enabled = True
prune_delay_in_days = 7
timeout = 30
max_retry_attempts = 4
use_caching = True
cache_directory_location = cache
[source/redfish-sclabs]
enabled = True
type = check_redfish
inventory_file_path = /opt/check_redfish/inventory/
permitted_subnets = <IP_ADDRESS>/24
overwrite_power_supply_name = True
overwrite_power_supply_attributes = True
akoznov@netbox:~$ egrep -v '^;|^$' /opt/netbox-sync/vcenter-sclabs-config.ini
[common]
log_level = DEBUG
log_to_file = True
log_file = /opt/netbox-sync/logs/netbox_sync.log
[netbox]
api_token = XXX
host_fqdn = netbox.sclabs.cloud
port = 443
validate_tls_certs = False
prune_enabled = True
prune_delay_in_days = 7
timeout = 30
max_retry_attempts = 4
use_caching = True
cache_directory_location = cache
[source/vcenter-sclabs]
enabled = True
type = vmware
host_fqdn = vc01.sclabs.cloud
port = 443
username =<EMAIL_ADDRESS>password = XXX
validate_tls_certs = False
permitted_subnets = <IP_ADDRESS>/20, <IP_ADDRESS>/19, <IP_ADDRESS>/22, <IP_ADDRESS>/24, <IP_ADDRESS>/24, <IP_ADDRESS>/22, <IP_ADDRESS>/22, <IP_ADDRESS>/16, <IP_ADDRESS>/20, <IP_ADDRESS>/22
cluster_site_relation = Noris/.* = NBG6ITA
cluster_tenant_relation = .* = Noris Lab
host_role_relation = .* = Hypervisor
vm_role_relation = .* = VM
match_host_by_serial = True
collect_hardware_asset_tag = True
dns_name_lookup = True
custom_dns_servers = <IP_ADDRESS>, <IP_ADDRESS>
set_primary_ip = when-undefined
skip_vm_comments = False
skip_vm_templates = True
skip_offline_vms = False
skip_srm_placeholder_vms = False
strip_host_domain_name = False
strip_vm_domain_name = False
cluster_tag_source = object
host_tag_source = object, cluster
vm_tag_source = object, cluster
sync_custom_attributes = True
host_custom_object_attributes = summary.runtime.bootTime
vm_custom_object_attributes = config.uuid
sync_vm_dummy_interfaces = True
disable_vlan_sync = False
track_vm_host = False
overwrite_device_interface_name = True
host_management_interface_match = management, mgmt
ip_tenant_inheritance_order = device, prefix
sync_vm_interface_mtu = True
akoznov@netbox:~$ egrep -v '^;|^$' /opt/netbox-sync/vcenter-norislab-config.ini
[common]
log_level = DEBUG
log_to_file = True
log_file = /opt/netbox-sync/logs/netbox_sync.log
[netbox]
api_token = XXX
host_fqdn = netbox.sclabs.cloud
port = 443
validate_tls_certs = False
prune_enabled = True
prune_delay_in_days = 7
timeout = 30
max_retry_attempts = 4
use_caching = True
cache_directory_location = cache
[source/vcenter-norislab]
enabled = True
type = vmware
host_fqdn = vc-1.vdi.sclabs.net
port = 443
username =<EMAIL_ADDRESS>password = XXX
validate_tls_certs = False
permitted_subnets = <IP_ADDRESS>/12, <IP_ADDRESS>/8, <IP_ADDRESS>/16, fd00::/8, <IP_ADDRESS>/16, fd00:bade:fee:1::/64
cluster_site_relation = Noris/.* = NBG6ITA
cluster_tenant_relation = .* = Noris Lab
host_role_relation = .* = Hypervisor
vm_role_relation = .* = VM
match_host_by_serial = True
collect_hardware_asset_tag = True
dns_name_lookup = True
custom_dns_servers = <IP_ADDRESS>, <IP_ADDRESS>
set_primary_ip = when-undefined
skip_vm_comments = False
skip_vm_templates = True
skip_offline_vms = False
skip_srm_placeholder_vms = False
strip_host_domain_name = False
strip_vm_domain_name = False
cluster_tag_source = object
host_tag_source = object, cluster
vm_tag_source = object, cluster
sync_custom_attributes = True
host_custom_object_attributes = summary.runtime.bootTime
vm_custom_object_attributes = config.uuid
sync_vm_dummy_interfaces = True
disable_vlan_sync = False
track_vm_host = False
overwrite_device_interface_name = True
host_management_interface_match = management, mgmt
ip_tenant_inheritance_order = device, prefix
sync_vm_interface_mtu = True

Now I'm manually deleting all Tags that created by netbox-sync, clearing the cache and try to resync couple of the times. Come back with some feedback.

After removing tags, clearing the cache and rerunning sync a couple of the times, only the changed items are changing inside NetBox. Thanks!

awesome


Bug fix broken document links

data/document_exclusion_list.json - Updated the json file to attach the iati-identifier with the project codes.

helpers/project_helpers.rb - Updated the code to stop slicing the projectid.

Coverage decreased (-0.06%) to 18.986% when pulling 6f7114e914b058910d9535511bf0e41f812d05bb on bug-fix-broken-document-links into ec7de4a52b40edd0e34ae2f3340f3b5552950965 on master.


Proposal: General conventions for spatial derivatives

Your idea
Hello @bids-standard/maintainers, @bids-standard/steering  & everyone,
we, @francopestilli, @arokem, @effigies, @oesteban and @PeerHerholz, would like to submit a proposal concerning spatial derivatives. We hope to engage in fruitful discussions with you all and further refine our proposal. If you have any questions, please don't hesitate to post them as well.
Abstract
In this issue, we propose a general principle for developing BIDS extension proposals for derivative data. The goal is to establish consensus so that parts of BEPs that propose terms in line with this proposal will be considered accepted in principle. The proposal is to ask for feedback from the community, provide a timeline for the discussion, and settle on a decision-making process. At the end of the timeline, we request a decision be reached. The proposal is RECOMMENDED, not REQUIRED, in that BEPs would be allowed to deviate when deemed necessary.
Problem statement
In working through BEPs 12 and 16, we have identified a repeated pattern in generating derivatives within several imaging modalities' workflows where:

We require a reference map that is used to encode spatial features and parameters. There is an antecedent of this in BIDS with BEP23 (see below). In that BEP, the proposed naming takes the pattern _<suffix>ref (e.g., _boldref, _dwiref, etc.), and that solution has been suggested as a possibility in issue #1532 of the spec repository.

We have derived data that are no longer of the same type as the original, but for which we would like to keep the notion of the modality from which this was derived while also signaling that it is derived (i.e., non-raw).

Proposal
Introduce a new suffix pattern : _<suffix>map, where <suffix> is a BIDS suffix used in the raw data (e.g., dwi or bold). For example, the proposed pattern produces the suffices _dwimap or _boldmap.
Motivation
Many users are not equipped to understand fine distinctions between different classes of derivatives (e.g., those that are produced by a model fit and a direct computation)
This suffix pattern provides context through the concatenation of a raw data suffix and the word "map", which implies that the file still contains spatially contiguous information (in contrast to tabular/"tidy" data, with each row representing a brain region, for example).
Precedents and interactions with other BEPs
BEP 23: PET Derivatives
BEP 23 has introduced "maps" that correspond to the conventions introduced by BEP 001 (qMRI), such as T1map, T2map, etc. The following maps were introduced:

RDmap (receptor density map)
BPmap (binding potential map)
GEmap (genetic expression map)

These generally will be distributed as mean/standard-deviation pairs, for example: sub-01_stat-mean_desc-5HT_RDmap.nii.gz/sub-01_stat-std_desc-5HT_RDmap.nii.gz.
BEP 12: Functional MRI derivatives
BEP 12 proposes a collection of summary statistics, including mean, standard deviation, temporal SNR, regional homogeneity, etc. Following the example of BEP 23, it has adopted the proposal.

<source_entities>_stat-<mean|std|...>_boldmap.nii.gz

BEP 16: diffusion-weighted imaging derivatives
:warning: We discuss this option as a considered alternative below
The current writing of the proposal follows the alternative listed below, where model fit and model-derived parameters are described:

<source_entities>_model.<extension>
<source_entities>_mdp.<extension>

This pattern is, in principle, more generalizable across the other ongoing BEPs and Derivatives in general:

A data process might have generated primary parameters that are either 3D (x,y,z) or 4D (x,y,z,v). These parameters might be of help for further data analysis or data interpretation, and ultimately the data end user. Examples include "statistics" such as mean, std, etc., or model derivatives, such as DTI FA.

At the same time, the process might have generated secondary parameters. These are not strictly necessary for further processing or data interpretation, but they can be potentially useful to interpret the outputs of the data process, to track history of the processing, for reproducibility and ultimately for debugging purposes of the developer/modeler of the code.

BEP 39: dimensionality reduction-based networks
The current version of the proposal uses a comparable pattern as outlined for BEP16:

<source_entities>_mdp.<extension>
<source_entities>_mfp.<extension>

Alternatives Considered

Suffixes that distinguish between model-fit and model-derived parameters. This alternative is implemented in the current state of BEP16 and BEP39. We assess this option should be deemed rejectable for the following reasons:

This distinction does not seem useful for end users as there are no antecedents of previous adoption by some neuroimaging sub-community, who may or may not care about understanding the distinction.
The distinction between model-fit and model-derived parameters is not always clear. To take one example, the eigenvalues and eigenvectors of the DTI tensor model could be seen as fit or derived. The utility of this high-level distinction is undermined if every such case is either left to the determination of the tool developer or requires an explicit declaration in the spec.
For BIDS purposes, it is more important to state what something is than how it was derived.
This is not something that any currently existing software does.

For the word that modifies the <suffix>, the following options have been considered

tensor : this was deemed rejectable because while the fancy Google branding has run with it, it still means something in physics.
array : all non-scalar data may be considered an array, but it lacks the association with spatial meaning
image : this was deemed rejectable because in its common usage in neuroimaging software, it implies raw data (e.g., boldimage would most likely be read as an image containing BOLD data)

Allowing each BEP to create separate suffixes that provide a good match to the use-case in that BEP. This is the status quo and was deemed rejectable to make both decision making and technical implementation simpler because it provides a reference rule for future implementations and avoids the proliferation of suffixes.

Decision making
As outlined above, we propose a two-stage decision-making process within a set timeline to reach a consensus. Furthermore, we aim to evaluate the feasibility of this process concerning other BIDS-related discussions, ie community-driven/guided decision-making.
Stage 1
In the first stage, comments from the entire community are solicited and discussed. We suggest a time period of 2 weeks, starting the day after the proposal was initially circulated/posted.
Stage 2
In the second stage, voting on the provided/proposed options (based on the Stage 1 outcomes) will take place. Here, we also suggest a time period of 2 weeks, starting the day after Stage 1 was finished.
After this time, this proposal will become part of the standard operating procedures of BIDS and be referenced in BEP development guidelines.

Tagging a few people that I hope they can pitchin:
@tsalo @dlevitas @yarikoptic @adelavega @jdkent @gkiar @rwblair @guiomar @dorahermes @KirstieJane @bendhouseart @cmaumet @sappelhoff @choldgraf

personally, I am against it as this would create a plethora of suffixes Xmap of all sorts, when meas- can take care of a lot of that ; I let @mnoergaard elaborate for BEP23

@CPernet Can you give an example? It seems to produce a maximum of 5 suffixes (boldmap, cbfmap, dwimap, aslmap, petmap) that I can think of, and this would not be a requirement that they be used. It would say that, if a somewhat generic image derivative is needed for that modality, that would be an acceptable suffix.
In fact, this currently has reduced the number of suffixes for bold derivatives, as many have been moved into the stat- entity. (Which could be changed to meas- if that turns out to be the consensus.)

For BEP23, we now put the outcome measure e.g. binding potential (BP) into meas-BP, so there would be no need for it in the suffix. Furthermore, because this data belongs to PET data, it would always be placed inside a pet directory in the derivatives (i.e. derivatives/pipelinex/sub-XX/ses-XX/pet/), thus making the pet in map slightly redundant. Currently for BEP23, the notion of maps has not been implemented yet (we previously talked about molmap, mimap, etc), mostly because it was solved by other entities.

What suffix are you using for RDmaps now, if it's meas-RD_? It appears to be missing from the bep.

@effigies

petmap is no good and mimap was discussed and preferred molecular imaging map because it serves a larger community than PET (but not used)
cbfmap, ok but let's then add cbvmap
dwimap really? diffusion of what? if we use modelling like NODDI you have three outputs, again using meas- makes more sense
etc ...

@effigies right now we don't have a suffix in line with the PET examples in BEP38 (https://docs.google.com/document/d/1RxW4cARr3-EiBEcXjLpSIVidvnUSHE7yJCUY91i5TfM/edit#heading=h.4k1noo90gelw). If a suffix is required, then I think the use of petmap is the most reasonable one.

some PET examples:
sub-1_seg-DKT_hemi-left_stat-mean_meas-BPND.nii
sub-1_seg-DKT_hemi-left_stat-mean_meas-SUV.nii

Suffixes have been a required part of BIDS files up to this point. I guess I'll go comment on that thread; I don't understand what's being gained by removing suffixes.

' I don't understand what's being gained by removing suffixes' I'm not suggesting removing those that have been useful, I'm saying that it is not always needed, like we don't see the utility in PET

@effigies was confused because the examples didn't have suffixes before the edit in the comment.
That said, the rationale behind _*map is to say this originally was derived from a given modality (in the sense of BIDS suffix) but now it may represent something completely different, so it's not that modality anymore.
In a strict sense, sub-1_seg-DKT_hemi-left_stat-mean_meas-BPND_pet.nii is not PET anymore, so saying sub-1_seg-DKT_hemi-left_stat-mean_meas-BPND_PETmap.nii signals the fact that this file is directly linked to the original PET dataset, but it is not PET data (as it comes from the scanner, as per BIDS raw definitions) anymore.
Re: datatype folder - some other folders have been proposed, and it is not unimaginable we'll hit a use case where you may need a PETmap outside the pet/ folder.

reasonable argument

Regarding the proposal of "*_<modality>map.<ext>", I would consider the example:

... BEP 001 (qMRI), such as T1map, T2map, etc.

to provide an argument against. Those existing suffices serve to distinguish eg. a non-quantitative image that is merely weighted by T1, ie. "T1w", from an image whose intensities reflect the quantitative parameter T1, ie. "map of T1" or "T1map". With something like "*_dwimap.<ext>", it's a quantitative map, but the first half of the suffix now encodes the modality involved (which is already indicated in the filesystem location) rather than the actual parameter encoded. So it's using the same suffix construction pattern, but the building blocks are different, which I would have thought would cause confusion. Strawmanning the philosophy for a moment, a BIDS App quantifying relaxation tissue properties would be expected to produce "_anatmap.<ext>", yet is promoted to having more specific suffices compared to other modalities only because of BEP001.

RE distinction between "model fit" and "model-derived" (regardless of what particular terms or suffices might be involved):

This distinction does not seem useful for end users ...

Perhaps not currently, but it depends on how forward-thinking we want to be about it. In part, this statement may be expressed from the perspective of "I am a human, and I am reading filesystem names and trying to understand content". Personally I think that with derivatives, especially complex intermediate ones, machine interpretability and robust definition for utilisation in downstream analysis take precedence, whereas for analysis endpoints obviously human readability are again important.
Where I saw a prospect for benefit in the distinction:

If either:

Re-running a tool to generate additional derivatives (as came up recently in #1583); or
Using the output of one tool that has performed a model fit as the input to another tool that does something with that model fit;

, then it is only the set of parameters that encode the model fit that are necessary to perform those calculations.

There may be metadata that describe a given model, and parameters that dictate the way in which that model was fit to the input data. Some such metadata should be applicable to those parameters that encode the model fit to the data, but should not be applicable to parameters that are further downstream derivatives of such, whether because they are superseded by something else or because such information should be relegated to provenance.
This is easy enough if the data encoding the fitting of the model to the data are expressed in a single data file: just put the corresponding metadata in the paired sidecar. Where it gets tricky is in situations where such model fit data must by necessity be split across multiple data files. In BEP016 this happens for anything more complex than a tensor or single-tissue CSD. There, what the distinction between model fit and model-derived hypothetically provides is the ability to utilise the inheritance principle to define such metadata just once, and have it be applicable to all data files that encode direct results of the model fit, but not be applicable to anything that is a subsequent derivative calculation of such. Unfortunately this plan doesn't work out-of-the-box because of the limitations of the inheritance principle (#1003).

The distinction between model-fit and model-derived parameters is not always clear. To take one example, the eigenvalues and eigenvectors of the DTI tensor model could be seen as fit or derived.

I would be interested if anyone can provide an example of a software that performs a direct optimisation of the eigenvalues and eigenvectors, rather than optimising the tensor coefficients and then performing an eigendecomposition of the result. In the absence of such I would state that these are unambiguously model-derived. It is the six coefficients of the symmetric rank-2 tensor that are the model fit parameters. I would hope (perhaps naively) that the presentation of such data in this way would be a net insight over time.
Maybe part of the problem here is a lack of consensus of the criteria by which that distinction is made. In my own formulation, this is not to do with what may be yielded by any given existing software tools, but with:

The mathematical optimisation of the inverse problem that is defined by any given model;
Directed dependencies, ie. you can calculate FA from the set of tensor coefficients, but you can't calculate the set of tensor coefficients from FA.

This is not something that any currently existing software does.

Depends on what you mean by "does". It would not be fair to base such an argument on eg. BIDS Apps; while there's insight to be gained from what BIDS Apps developers have encountered and the solutions they've come up with, BIDS Derivatives should IMO not be determined entirely based on the non-conformant derivatives currently being generated by BIDS Apps. Outside of BIDS Apps, in MRtrix3 land this is exactly what we do. dwi2tensor fits the tensor coefficients to the DWI data, tensor2metric derives parametric maps from this fit. Same for CSD, though more complex: dwi2fod performs the spherical deconvolution method, but from there there's all sorts of derivatives that can be computed using many different commands. We strive for modularity a la Unix philosophy. This might be a source of the different perspectives: BIDS Apps are moreso about pushing one button and getting a dense dataset of results. But I can foresee a future where the derivatives of BIDS Apps are used extensively as inputs to other BIDS Apps (technically it's already happening, just not in a way recognised by the standard / official BIDS Apps interface), and there a more modular approach is going to shine.

For the word that modifies the <suffix>, the following options have been considered

I had previously considered simply "parameter" / "param". That's entirely generic, and should apply reasonably to a broad spectrum of data types beyond images. But it's a little unwieldy. And I was dealing with data relating to fitting of models, and so wanted "model" to fit in there somewhere, but then what results from fitting a model to the data are themselves "parameters"... :face_with_head_bandage:  eventually "mfp" and "mdp" arose as a way of getting both in there concisely and yielding the distinction I required for inheritance as above.
As I've stated previously, I'm not married to that proposal; I'm just yet to figure out / be presented with anything that I see less downsides to.

Regarding the proposal of "*_<modality>map.<ext>", I would consider the example:

... BEP 001 (qMRI), such as T1map, T2map, etc.

to provide an argument against. Those existing suffices serve to distinguish eg. a non-quantitative image that is merely weighted by T1, ie. "T1w", from an image whose intensities reflect the quantitative parameter T1, ie. "map of T1" or "T1map". With something like "*_dwimap.<ext>", it's a quantitative map, but the first half of the suffix now encodes the modality involved (which is already indicated in the filesystem location) rather than the actual parameter encoded. So it's using the same suffix construction pattern, but the building blocks are different, which I would have thought would cause confusion.

Except for the final comment about confusion, this is a statement of a fact.
Regarding the confusion, I find this proposal less confusing (since these _*map suffices will clearly be always in a derivatives' folder as opposed to the quantitative counterpart) than any other alternative, as described in the proposal. Regarding BEP001, it is possible that clarifications are pushed upstream instead of downstream, and avoid _*map to indicate the quantitative nature of the contrast _qT2 would be as informative as _T2map, while freeing space for a broader meaning in the derivatives folder. MP2RAGE images were originally encoded as T1w images and the transition to new suffices was not traumatic, even if the BEP has been accepted already.

Re: model derived/fit
The following discussion should not be here, but rather in the main location where this is being discussed. Please let me know what is the target and I'll happily move the discussion there.

Perhaps not currently, but it depends on how forward-thinking we want to be about it. In part, this statement may be expressed from the perspective of "I am a human, and I am reading filesystem names and trying to understand content". Personally I think that with derivatives, especially complex intermediate ones, machine interpretability and robust definition for utilisation in downstream analysis take precedence, whereas for analysis endpoints obviously human readability are again important.

BIDS has been (to great success) pushed by the 80/20 principle. Attempting to capture such complexity with the filesystem organization falls in the 20% of the rule. If being able to represent the difference between derived and fit parameters is necessary, a better solution to BIDS such as NIDM should be considered.

I would be interested if anyone can provide an example of a software that performs a direct optimisation of the eigenvalues and eigenvectors, rather than optimising the tensor coefficients and then performing an eigendecomposition of the result.

BIDS should not suggest/support/indicate how things are implemented. Probably such a software doesn't exist but the point does not support a more favorable view of the proposal. The relationship between BIDS and practice should be the opposite - if there is a software that explicitly differentiates derived/fit in the naming at the output (therefore disallowing the user to choose their preference) because allowing the flexibility in naming would endanger the safe consumption of the results in further processing, then BIDS should maybe consider it. This is what is meant by no software "does" that (hereby the definition of "does" is complete).

hi Folks, thanks for the comments! This a critical discussion that perhaps boils down to the following question: "What level of details (parameters, models, output, etc.) will we need in the future when generating derivatives?" If we avoid answering this important question now, we should be able to accept the current proposal. MRTrix3.0 should be able to generate derivative maps compatible with the current proposal.
@Lestropie is correct in saying that currently, we are not extremely modular in BIDS. My suggestion here is to avoid discussing modularity and the type and quantity of parameters but instead discuss those in the context of BIDS 2.0 and BIDS Provenance BEP (which should be the one providing provenance of the parameters used to generate a dataset.
In other words, I am suggesting separating (A) the data generated (derivative)[to be addressed here]  from (B) the model and parameters used [more appropriate to be discussed in the context of provenance and/or even BIDS 2.0]

I have been convinced by @oesteban argument above -- re _*map is not that modality anymore (a transform), but the issue are all the current _map which kinds of conflicts

Hello! @robertoostenveld asked that I point out this document: https://docs.google.com/document/d/1JtTu5u7XTkWxxnCIH6sxGajGn1qG_syJ-p14aejpk3E/edit, and asked that we consider how the principles laid out in that document relate to this proposal. In particular, one prinicple (not very explicitly laid out in the document, imho) is to try to use entities more and suffixes less. @robertoostenveld identified that as a potential issue with the present proposal being discussed in this issue. Maybe this principle should also be made more explicit in the document.
I will just weight in that this is an objection similar to the one raised by @CPernet above, but as pointed out in previous comments (e.g., https://github.com/bids-standard/bids-specification/issues/1602#issuecomment-1698959331) it creates a rather limited number of additional suffixes, so might be in line with the principles.

That said, the rationale behind _*map is to say this originally was derived from a given modality

So if I were to do a 2nd-level statistical analysis on 1st-level contrasts from bold data, would I get a _boldmapmap?

So if I were to do a 2nd-level statistical analysis on 1st-level contrasts from bold data, would I get a _boldmapmap?

Why not _statmap as proposed in #887?

First, on the specific proposal:

if I were to do a 2nd-level statistical analysis on 1st-level contrasts from bold data, would I get a _boldmapmap?

GLM outputs are not currently in anybody's proposal. As noted by @tsalo, _statmap is being used for that purpose in some projects until some BEP is accepted that would require a different suffix.
The goal of this proposal is more along the lines of first-order (or close enough to make little difference), voxelwise derivatives from a variety of modalities. These could be MD or FA in diffusion, cortical thickness in structural, regional homogeneity in functional. There just isn't a good term for all of these, so we could split them into a large number of suffixes to say "These must be treated as different measures", but that's excessive. The things they have in common are 1) being voxelwise derivatives; 2) generally being an input to some further processing, not an end goal (although they could be in some contexts!).
I am not married to _boldmap; it just seemed to cover the case as well as anything else I could think of and it seemed to have some precedent in other BEPs, and I'm happy to consider alternatives. I'd even be happy to step out of the discussion, let everybody else consider the alternatives and I will just implement it when you're done.

On the proposal process:
I'm concerned though that we are letting the perfect be the enemy of the good, and the desire many of us have to establish some principles that will apply in all cases is halting progress. This proposal is specifically worded not to be a binding decision on all future BEPs, but as a good-enough solution to a proximal problem seen in multiple BEPs. What I would like to see, and what I heard from Franco and Peer that they would like to see, is some decision that we can go back to our BEPs and implement and not have to relitigate this separately in each BEP and again when it's time for community review.
And I want to stress that we are not asking the steering group to join the debate and then decide for us. The steering group as the final authority might need to decide on contentious spec-level decisions, but the thing that needs steering is the community, not the individual decisions in each BEP. What we need is a way to establish community consensus without waiting to the end of the BEP process, and I think the steering group can play a role in creating that process and giving it legitimacy. If, at the end of that process the community consensus is clear, I would hope the steering group would announce that consensus, regardless of the consensus (or lack thereof) within the steering group itself.

The goal of this proposal is more along the lines of first-order (or close enough to make little difference), voxelwise derivatives from a variety of modalities.

This fairly nicely seeds a different perspective by which I wanted to look at this one.
Firstly, not sure if it's intentional, but to me, for a broad-reaching proposal like this I wouldn't be constraining to specifically voxel-wise maps. Any format that defines a spatial embedding of data can have stored within it data corresponding to any parameter. I don't see that "map" applies to voxels but not to eg. vertices or electrodes.
On one hand that could be a good thing; but conversely it leads into my concern. It's not clear what the scope / limiting principle of the proposal is; where "map" is no longer applicable and something else is either preferable or required. If the result of computing some statistic across volumes is a "map", and a voxel-wise parameter estimated from a biophysical model is a "map", then what are the criteria by which something is not a "map"?
I'll derive an example of this from BEP016, since it's where most of my thinking on the topic has been.
For things such as MD or FA calculated from the tensor model fit, I can absolutely see the appeal of those being broadcast to users as "maps". Indeed even something like NODDI's kappa parameter, more commonly transformed into the Orientation Dispersion Index (ODI), which is directly optimised as part of the model fitting procedure, is entirely reasonably sold as a "map". A key attribute that these have in common is that they are scalars per voxel. However in BEP016 there are may more of what are currently called data representations (not sold on that name), which reflect the fact that in diffusion MRI we have orientationally-dependent information per voxel, and there are many different mathematical forms of such. So the question is: should all of these simply be "_dwimap.*"? For things like tensors or FODs, there's probably still some reasonable sense to it, though it feels less intuitive. Where I think it struggles is something like, say, fixel azimuth-inclination angles. My own subjective sense is that "map" conveys with it a certain expectation of compatibility with trivial user interaction / visualisation, whereas anything for which interpretation of the data is more complex than just "an ND block of data within a spatial embedding" feels like the linguistic correspondence with "map" is more strained.
Linking back to the whole MDP / MFP thing, my observation was that often times we'll fit a complex orientationally-dependent model and then derive some scalar parameter from it since it's better for visualisation / analysis than the whole model fit result, but conversely, there are some models for which there is a scalar parameter that is genuinely a component of the model fitting procedure (eg. NODDI's kappa). For that reason I moved the distinction between scalar & more complex data representations to metadata (also some require additional metadata for interpretation, so not the only reason). But the suffices needed to be compatible with the situation also. I figured that all of these are "parameters" so went with that; but I also saw some merit in the fit vs. derived distinction, hence the acronyms. By erasing the fit / derived distinction, and reverting from "parameters", "map" re-escalates the scalar-vs-more-complex confound. It may be the case that it's acceptable to most to go "map" across the board, I just think it's worth elucidating.

For the word that modifies the <suffix>, the following options have been considered

For BIDS purposes, it is more important to state what something is than how it was derived

Extending your list by one would be "scalar". This is potentially reduced in scope compared to "map" (depending on what you think the scope of such should be), but to me is more precise in terms of both its scope and "stating what something is". It's kind of TRX-like, where for streamlines tractography data "data per vertex", "data per streamline" and "data per group (of streamlines)" are split at the filesystem level.
IN BEP016-land, this would involve giving each "data representation" its own suffix (eg. "dec", "tensor", "sh", "polar", "3vector"; up for debate). This change alone would not resolve the complex inheritance problem, as some of these representations would still have compulsory metadata fields. But the number of new suffices required would be < 10, which is far less than giving every possible derivative image contrast its own suffix.

I do not have much of a position about what to do with non-scalar "maps". The closest that BEP12 comes to that is per-voxel regressors or spatiotemporal decompositions, and we have given them different suffixes (_timeseries and _{mixing,components}, though I do need to catch up with what's going on with BEP39 for the latter...). I don't see a problem with having a different suffix for per-voxel tensors, if we're really stretching the concept of "map" too far. If we end up with 5-7 new suffixes that cover all of the cases well enough, that seems like a reasonable outcome for a BEP, to my mind.
And I do agree that I would extend map to per-vertex scalars as well. I would defer to tractography experts about what to call per-streamline or per-group metrics.
I'm not 100% sure you're proposing _dwiscalar and _boldscalar intead of _*map, but in case you are, _boldscalar would indicate to me a single value, not a single value per-voxel. Though perhaps  _boldscalar.nii.gz would make it clear enough that I've probably got 3D data. It still reads weirdly to me, so I prefer map, but I'm not 100% opposed.

I'm not 100% sure you're proposing _dwiscalar and _boldscalar intead of _*map

Probably moreso "_scalar" rather than "_<modality>scalar", as per interleaved discussion in this thread.
Doesn't resolve the issues with "map" regarding applicability / scope (including to data for which more specific suffices already exist), and it'd be a sizeable strategic sidestep in terms of what attributes determine suffices. But it's nevertheless an option that was historically considered in this context.

Alternatively, we could try massaging "map" a little. I mentioned I'd contemplated "parameter". In the context of statistical outputs, "_spm" would be applicable for statistical parametric maps, even if there's a software package that took its name from that acronym. And obviously we want something that applies to things that aren't statistical derivatives. Can "_parmap (parametric map)" work? It removes ambiguities around alternative meanings of "map", and to me subjectively it lessens the issue of data called "map" being trivially user navigable; a complex model fit output is still a spatial "map" of that "parameter", just there's no guarantee of any given parameter being human-readable. There's precedent for contraction & concatenation to form suffices; it immediately reminds me of BEP017 "_relmat".

It may seem trivial discussing even the name, but we don't want to reiterate the 'atlas' naming issue.
@effigies is however right, I think we should 1. Agree on some naming now (map, parmap, whatever) 2. Let the BEPs work with that naming for a while so we are moving forward 3. Re-evaluate see how that works for everyone.

Why not _statmap as proposed in https://github.com/bids-standard/bids-specification/issues/887?

As that violates the previously suggested principle about the suffix being derived as _<previousmodalitysuffix>map. And why would at the 1st level the modality _stat then be preferred over _boldmap?
(Please note that I do not care specifically about this example, but about principles and long-term consequences for BIDS file naming).
I agree with Cyril that this seems to lead to a plethora of suffixes.  If we need many "names" to describe something, that is the reality and fine with me, but I think that the value in <entity>-<value> is more appropriate for an open-ended list of "names". I worry about maintenance consequences of an ever-growing list of suffixes for the validator, but also for other tools that ingest BIDS data. And for the spec itself, since every possible suffix would need to be documented in the specification. The only way I see to avoid that is by not having any constraints on the suffix (hence the plethora), but I don't consider that attractive.

About the letter of the proposal

Probably moreso "_scalar" rather than "_<modality>scalar", as per interleaved discussion in this thread.

_scalar would be very inconsistent with BIDS raw - with that reasoning, all MRI modalities would fall under a generic _mri suffix and taking it further, most of imaging modalities would just be _reconst as are typically the outcome of some reconstruction algorithm.
Setting the specificity of suffixes to very low levels voids them of usefulness. The datatype/ folder organization does not help a lot, as it is (another) arbitrary partition of the data and in derivatives it may evolve to new datatypes (e.g., conn/) that could, eventually, contain _*map s.
Another interesting feature of _*map is that I may have a <rootsuffix>map.json file at the top of the derivatives to give uniform and fast access to the metadata shared across those, as opposed to a meas-<label>_<suffix>.json that has other problems:

if <suffix> were a modality (as @CPernet discussed above) then we cannot have a <suffix>.json and several meas-<label>_<suffix>.json as siblings, forcing us to use e.g., desc-preproc_<suffix>.json for derivatives that are a conversion of the original dataset more ready for modeling.
if <suffix> were just _scalar then scalar.json does not really say anything.

Regarding _<rootsuffix>scalar vs _<rootsuffix>map I'm with @effigies, and would add that map tells me this is data (dimension typically 1 but could be a vector, or complex) distributed on some manifold (typically grid, but could be point set or surface). _<rootsuffix>scalar tells me little about how the scalar is sampled and also restricts the dimension of the feature to 1.

So if I were to do a 2nd-level statistical analysis on 1st-level contrasts from bold data, would I get a _boldmapmap?

@robertoostenveld can you point me at where the proposal above could be interpreted this way? To be honest, this logic sounded to me like this: _if I concatenate two bold runs, would I get a _boldbold ?

I agree with Cyril that this seems to lead to a plethora of suffixes.

I'm not really sure why more suffixes could be a problem. This issue emerged back in 2017 when we were having the early conversations about derivatives in the BIDS workshops, and I still don't see why it is bad per se.
These and other recent discussions are leading me to think that BIDS Derivatives is a really bad idea, and there's no alternative to NIDM and solutions as such where the interpretation of the structure is self-contained.
About the spirit of the proposal
As @effigies has mentioned above (https://github.com/bids-standard/bids-specification/issues/1602#issuecomment-1710796785) there is also an intent of unlocking the many decision processes currently opened within BIDS, trying to move these decisions back to the community rather than them be taken by the steering committee. The steering committee is afforded that purview, but the success of BIDS comes from bringing the community together, so before just deciding, the committee should search for community consensus before making any decision. This proposal aims to give our community an instrument for that.
Please note that above this point, there is @effigies statement above and above that the proposal itself. I'm certain both text represent the spirit of the proposal much better than this attempt.

'a suffix is a value without it's entity'
for instance we could have had modality-fmriBOLD but instead we have * _bold.*, modality-anatT1w is just _T1w., then came about the possible modality-qT1 which instead became T1map.  --> we need a simple governing rule for suffixes otherwise stick to entities
TBH I'm really conflicted here, on the one hand this proposal seems useful, i.e. it allows moving forward with stuff (even if one can argue about what name to use), one the other it also raises an issue on how we want to use suffixes in general (which might become a long discussion and thus blocks progress in those BEPs)
PS: my opinion not a reflection of the steering group

robertoostenveld can you point me at where the proposal above could be interpreted this way

You write _*map where * suggests that it is a wildcard and hence can be anything. You also write _<rootsuffix>map, where root refers to the root of the shared data. That is not guaranteed to be the raw data. If I share 1st level _boldmap with you, and you do the 2nd level analysis, then boldmap is your root, resulting in boldmapmap according to the logic described above. Or does the BIDS specification define somewhere what "root" means other than how I take it here?
It might have been different, had you written _<rawsuffix>map, although in that case it would not be able to distinguish 1st and 2nd level. As https://docs.google.com/document/d/1JtTu5u7XTkWxxnCIH6sxGajGn1qG_syJ-p14aejpk3E/edit details, derivatives and raw datasets as a whole only differ in their predecessor (respectively a BIDS or a non-BIDS source). Hence, my 1st level data that I share with you could have be constructed from a non-BIDS source, which makes it "raw". Hence I don't see _<rawsuffix>map being well-defined either.

I'm not really sure why more suffixes could be a problem.

I see the plethora of suffixes as a problem because the specification needs to be explicit about them, which means every extra suffix needs a BIDS version bump, and every suffix needs to be incorporated in all tooling. Values represented in <entity>-<value> allow either rigidity (limited list) or flexibility (anything goes as long as the format is consistent) and are IMHO at the moment better dealt with in the specification (here and here). As far as I know, we don't have a human-readable overview of all suffixes. We do have this list that is part of the schema. FYI, we now have 103 suffixes, some of which of the format *map.
If the proposal were to now add a limited number of well-defined _xxxmap and _yyymap suffixes rather than a wildcard _*map, I don't see the problem so much.

You write _*map

Yes, but this is not written in the proposal. I asked about the proposal. My elaboration later on with _*map was intended to communicate with @CPernet, and it seemed to work toward that effect.
So the question remains: where in the proposal is this suggested?

If the proposal were to now add a limited number of well-defined _xxxmap and _yyymap suffixes rather than a wildcard _*map, I don't see the problem so much.

I believe the proposal does not give space for the user to arbitrarily create new suffixes.

I see the plethora of suffixes as a problem because the specification needs to be explicit about them, which means every extra suffix needs a BIDS version bump, and every suffix needs to be incorporated in all tooling. Values represented in <entity>-<value> allow either rigidity (limited list) or flexibility (anything goes as long as the format is consistent) and are IMHO at the moment better dealt with in the specification (here and here). As far as I know, we don't have a human-readable overview of all suffixes. We do have this list that is part of the schema. FYI, we now have 103 suffixes, some of which are of the format *map.

BIDS should try to offer a compromise between flexibility and user-friendliness. If flexibility is afforded by means of entity-label pairs, user-friendliness goes rapidly down because it implements a strategy for users to keep doing exactly what they were doing, just it now looks "bidsy". If that is intended, I would prefer everyone keeps naming things the way they like but make the data NIDM compliant.
The suffix and the prefix are traditionally the most relevant parts of names in practice. Making unspecific suffixes will inevitably lead to discussing a strong ordering criteria for entities, because humans will automatically drop the unspecific suffix from the filename and focus on the next bit.
And again, let me repeat that the spirit aspect of the proposal is also very relevant.

So the question remains: where in the proposal is this suggested?

Here it states: "Introduce a new suffix pattern : _map, where  is a BIDS suffix used in the raw data (e.g., dwi or bold). For example, the proposed pattern produces the suffices _dwimap or _boldmap."
This proposal makes _boldmap a suffix, and hence by recursion _boldmapmap also becomes a possible suffix. Unless recursion is excluded (which it is not in the proposal), this leads to an infinite number of suffixes.

Despite I disagree recursion can be implied, further down it says:

This suffix pattern provides context through the concatenation of a raw data suffix and the word "map"

So, the only situation where you could get a mapmap is if the original raw data suffix had map in it such as those explained above.
This is probably something to consider when developing the derivatives of those particular suffixes, and this proposal is also made as a recommendation. It could have some language anticipating this case and excluding existing raw suffixes ending in map from it.
Finally, as a redux, this proposal could have brought into the light that currently existing "map" suffixes may require some discussion and allow alternatives that allow the "map" extension in derivatives, if "mapmap" is sufficiently annoying (it is a little to me, tbh).

I would like to see a clearer definition of _*map.ext added to the proposal such that I can better understand the implications. In my current understanding the suffixes determine what type of metadata should be present with a file and restricts which extensions are allowed. For example, the suffixes _bold.ext, _eeg.ext, _meg.ext and _ieeg.ext indicate which .json and .tsv files MUST be present. I don't understand yet whether how that will be applied in the general _*map.ext. Say I create an _ieegmap.ext, I want to know which metadata I should save for interpretation and how I should, for example, save the type of units contained in this file.

What type of metadata and extension go with an _*map.ext? Will this be defined in each extension proposal?

Is _*map.ext intended to be restricted to nifti volumes with values of a certain quantity and units? Can I even use this proposal for electrophysiology data? Would this proposal allow me to save bold responses/values around iEEG electrodes and create a _boldmap.ext, but these data would be of an electrophysiology data type (maybe this example does not fall in the 80/20 rule, but trying to come up with a potentially confusing example on purpose to help clarify/restrict the proposal).

@dorahermes I think finding counter-examples that would discourage applying this proposal would be very useful. We did not spend so much time on that, so I agree it's worth attempting. @francopestilli, @arokem, @PeerHerholz and @effigies please correct me if I say something imprecise about the proposal:

What type of metadata and extension go with an _*map.ext? Will this be defined in each extension proposal?

This particular proposal does not prescribe extensions for _<suffix>map (please, let's avoid the star notation _*map as it seems more confusing than useful).

2. Is _*map.ext intended to be restricted to nifti volumes with values of a certain quantity and units? Can I even use this proposal for electrophysiology data?

It's not restricted to NIfTI (e.g., I can see it used with GIFTI and with CIFTI), nor does it impose any other metadata, such as units. Yes, this proposal wants to remain sufficiently flexible so other BEPs can easily adopt or ignore it if it doesn't apply.

Just as a note, I am not trying to discourage this proposal in general, but some edits and clarifications would be very helpful. Specifically, I would like to see more clearly described how each BEP using _<suffix>map (may/should/must) specify the metadata and extensions allowed for the _<suffix>map .

I am not trying to discourage this proposal in general

Apologies if my response can be interpreted in this way. I meant that finding counter-examples is indeed a good idea in this particular case.

how each BEP using _<suffix>map (may/should/must) specify the metadata and extensions allowed for the _<suffix>map .

I believe the proposal is very open-ended, so I don't think the goal was to limit what metadata and extensions should/must be used. As said above, I agree the proposal could be more explicit about what may be used.
Perhaps the most direct examples could come from what should not be done with this proposal:

Name a derivative image&meta corresponding to an original image (e.g., say some RSfMRI _bold.ext) that remains "the same modality". For example, after head-motion realignment, our RSfMRI timeseries is still a _bold.ext and not a _boldmap.ext

@dorahermes do you see specific issues? Do you think you can provide an example?

The specific issue I have is that the proposal section is too vague. I would suggest adding something like what I put between []:
"Introduce a new suffix pattern : _map, where  is a BIDS suffix used in the raw data (e.g., dwi or bold). For example, the proposed pattern produces the suffices _dwimap or _boldmap. [BEPs may use this suffix pattern under the conditions specified below and MUST specify the extension and metadata that are required with the suffix.]"
@francopestilli would it be possible to add a section with 'Conditions under which BEPs may and may not use the suffix pattern?'
For examples, would e.g. tmap or betamap be allowed under this pattern? Betamap seems particularly underspecified and could refer to a statistic OR a beta oscillation map of some kind without further specification.

Hi @dorahermes what about something like the following?
Introduce a new suffix pattern: _<suffix>map, where suffix is a BIDS suffix used in the raw data (e.g., dwi or bold). For example, the proposed pattern produces the suffices _dwimap or _boldmap. BEPs may use this suffix pattern under the conditions specified below and MUST specify the extension and metadata that are required with the suffix.
(1) The file descriptor does fall under one of the generic derivatives descriptors.
(2) No other descriptor exists in the BIDS spec. For example, statsmap cannot be used, because it is already being used, or soon to be, for a different specification.
@oesteban @effigies @arokem @robertoostenveld @CPernet
I asked @PeerHerholz to edit the original post

FYI 'we' rather not use _petmap because many similar results can be obtained from other sources than pet, and _mimap would be preferred (molecular imaging map) -- we'll have to think if similar _*map applies across electrophys

@francopestilli That sounds good to me.
@CPernet I suppose I consider mimap "in the spirit" of this proposal. Again, the point is not to force any other BEPs to adopt this convention precisely. It's to get a rough consensus that it's good enough to at least use across a couple BEPs and, if others find it good enough within their own BEP communities, then those of us on the outside won't nitpick it when it comes time for broad review.
Speaking as someone with no electrophys experience, I did not really anticipate that they would use it, as it doesn't really seem to fit the shape of their data. That said, I could see a route to a _<X>map that would follow this logic: Suppose you project your EEG data onto the cortical surface and end up with _hemi-<LR>_space-fsnative_eeg.func.gii files. Then, if you follow the lead of BEP012 in its current (unmerged) state, you might describe the temporal mean of that as _hemi-<LR>_space-fsnative_stat-mean_eegmap.func.gii. Not that this is necessarily a useful thing to do...

totally agree @effigies chris that is there just FYI and for not being stuck on  _<modality>map  ;  @oesteban convinced me it was useful so I have no real concern but those raised about limiting somehow to not see an explosion of suffixes

@francopestilli that sounds good, thank you!

I am fine with this as a proposal to the BIDS extensions guidelines as documented on https://github.com/bids-standard/bids-extensions.

The originally-posited discussion period is now over, and we would like to move to a vote of support for this proposal, as indicated in the OP. Please indicate your support by adding a :+1: to this comment or your objection by adding a :-1:. If the yays exceed the nays, we can close this and move on to create a PR that will add the language that has been settled through this discussion on to the bids extensions guidelines. The voting on this will close in 2 weeks, October 12th at 9am PT.

A reminder that voting on this issue will close in two days.

OK. Voting is now closed and the proposal is accepted.
@PeerHerholz : when you get a chance, could you please work to incorporate this in https://github.com/bids-standard/bids-extensions? I'll post an issue linking back here, so we can keep track of this.


TLS attempted over port 53 when enabling tls-upstream

Hello Unbound folks! I'm experimenting with DNS resolvers for the first time and came across an issue with DNS over TLS.
Describe the bug
Certain configurations cause unbound to try to connect to upstreams and/or authoritative nameservers with TLS over port 53, which is not supported by servers nor the RFC. Here's an example on the mailing list of someone else seeing this behaviour. It should probably be connecting over port 853 by default, and never over 53 if using TLS as a transport.
I ran into this issue while trying to make unbound use DoT to connect to authoritative nameservers while recursing, which I now realize is probably not yet supported. I configured my server to operate as a recursive resolver, and then set tls-upstream hoping it might also cause recursive lookups to go over DoT if available. To my delight it did attempt to use DoT! But on the wrong port, and with no fallback to plaintext DNS. I'm guessing it's not supported and that my configuration is wrong, but it would be nice if the server didn't fail in such a strange way.
To reproduce
Steps to reproduce the behavior:

Follow the Getting Started guide.
In the configuration file, set tls-upstream: yes.
Don't configure any upstreams or forwarding.
Run unbound and make a query.

Expected behavior
Unbound could either:

connect to the upstream or authoritative nameservers using TLS on port 853
Ignore upstream-tls when recursing
Maybe throw an error about the config

System

Unbound version: 1.19.0 from Homebrew
OS: Mac OS 14
unbound -V output: onfigure line: --prefix=/opt/homebrew/Cellar/unbound/1.19.0 --sysconfdir=/opt/homebrew/etc --enable-event-api --enable-tfo-client --enable-tfo-server --with-libevent=/opt/homebrew/opt/libevent --with-libnghttp2=/opt/homebrew/opt/libnghttp2 --with-ssl=/opt/homebrew/opt/openssl@3 --with-libexpat=/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk/usr
Linked libs: libevent 2.1.12-stable (it uses kqueue), OpenSSL 3.2.1 30 Jan 2024
Linked modules: dns64 respip validator iterator
TCP Fastopen feature available

Additional information

unbound.conf

server:
	username: "_unbound"
	root-hints: ""
	hide-identity: no
	hide-version: no
	hide-trustanchor: no
	qname-minimisation: yes
	auto-trust-anchor-file: "/opt/homebrew/etc/unbound/root.key"
	tls-upstream: yes
	tls-system-cert: yes
	http-notls-downstream: yes

Hi Philippe,
You are right that DoT is not common to authoritatives yet.
The manpage in Unbound does warn that this option is global for all upstreams and useful in tunneling scenarios. If you know of specific upstreams that do support DoT you can explicitly configure them in forward and/or stub configuration sections as described in the manpage.
So I see this as not an issue for Unbound.
I believe what you would like to see is support for unilater probing; a draft that describes a mechanism for a resolver to actively check for upstream DoT/DoQ support. This is not yet in Unbound but there are plans (not concrete) to include it in a future version.

I see now, I had misunderstood that paragraph to mean that it would contact all nameservers with tls over port 853, and you are correct that a probing mechanism like the one in that draft is what I had in mind. I will close this issue and wait for a future version. Thanks for the reply!


website examples data format?

Hi, what is the binary format used for the examples with 858.4K  scatterplot-data?
To add some more context: I am trying to compare performance aspects between various options of using large binary data, and would love to learn about the format deck.gl website examples use, how this format is converted into layer data etc.

no love? :(

@serdaroquai The compression system used for those examples was a hack done as part of the early deck.gl 3.0 release almost three years ago to ensure the website would load on mobile. It is part of the website code in deck and was never intended to be used by apps.
We haven't touched this for a long time and it would require some research on our side to refresh ourselves on the details, not sure it is worth it.
In addition, while we haven't really discussed and agreed on this yet, my take is that with the next-gen binary support planned for v8.0, we should phase out this compression scheme completely in our next major release, further reducing our appetitive for digging into this now.
If you keep poking into the website folder and the worker code there and get a bit further have more specific questions feel free to ask again, perhaps we could answer them with less effort on our side.


ngc exits with base64 is not a function

Bug Report or Feature Request (mark with an x)
- [X] bug report -> please search issues before submitting
- [ ] feature request

Versions.

node --version: v4.4.7
npm --version: 4.4.1
Repro steps.

Download the attached sample
Open console (cmd in our case) and navigate to the base directory (where the package.json is located)
run npm install
run "node_modules/.bin/ngc" -p src/tsconfig-aot.json

The log given by the failure.

Mention any other details that might be useful.
quickstart_aot.zip

I might be because you have a non supported version of Node.
As per README:

Both the CLI and generated project have dependencies that require Node 6.9.0 or higher, together with NPM 3 or higher.

It works with the current node version. Thanks!


added arithmetic shift operations

part 2 of fix #44

Codecov Report

Merging #47 into master will not change coverage.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##            master       #47   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files           43        45    +2
  Lines          311       321   +10
  Branches        49        49
=========================================
+ Hits           311       321   +10

Impacted Files
Coverage Δ

source/bits/ashiftl.ts
100.00% <100.00%> (ø)

source/bits/ashiftr.ts
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 97ff2e6...228e163. Read the comment docs.

oops, I swapped left shift with right shift. Fixed it a few minutes ago.

@0xflotus I merged the circular shift one as it was obvious how it should behave on arbitrary lengths of bits (doesn’t modify length). However, I’m really not sure about this one and if it’s more reasonable to keep the length or to decrease/increase it by amount


Rewrite the design system

Create animation.scss and typography.scss
Change the design of tips.
Change the design and animation of dropdowns.
Change the font to Inter font.
Add a background to selected items in the sidebar.

Thank you! It looks nice :)


data-grid loader allows interacting with grid rows & clr-dg-action-bar.

Select one ...  (check one with "x")
[x] bug
[ ] feature request
[ ] enhancement

Expected behavior
When loader is being shown, data-grid should not allow :

User to interact with action bar buttons
Expanding/collapsing rows.
Focussing input fields present in the grid rows after expanding it.

Actual behavior
With loader enabled, user is able to do following:

Click action bar buttons
Expand or collapse the rows.
Focus on input fields present in the expanded grid row.

Reproduction of behavior
https://plnkr.co/edit/XobaJiLJE2AaBGXEsysT?p=preview
Environment details

Angular version: 5.1.3

Clarity version: 0.10.20

OS and version:  MacOs Sierra

Browser: Chrome 62.0.3202.94 (Official Build) (64-bit)

Any progress on this bug? Which release we are targeting this fix ?

When a dagagrid is loading, datagrid should not allow:
• User to interact with action bar buttons.
• Focusing/expanding/collapsing rows.
• Focusing input fields present in the grid rows after expanding it.

Partially fixed by #2670. It prevents any kind of click actions, but interactive elements can still be accessed via keyboard navigation while the datagrid is loading.

The workaround here is to disable your action bar buttons since they exist outside of the Datagrid and Clarity can't easily manage this for you. We would consider a contribution to address this issue if someone is able to work out a solution. As we build Clarity Core implementations, we expect that this issue won’t be an issue or will be configurable on the application side. To help us clean up our backlog, we are going to close this with a functional workaround available and suggest you follow updates for Clarity Core for enhancements that can support your use case with Clarity Core components.


[New Extension] Add timezone-converter extension

Description
It's a handy timezone converter tool, that converts any timezone to any timezone.
Screencast
https://user-images.githubusercontent.com/3612098/219062404-a239fc46-f376-4a6a-938b-9655b6b5f73d.mov
Checklist

[x] I read the extension guidelines
[x] I read the documentation about publishing
[x] I ran npm run build and tested this distribution build in Raycast
[x] I checked that files in the assets folder are used by the extension itself
[x] I checked that assets used by the README are placed outside of the metadata folder

Hi 👋
Thanks for your contribution 🔥
I changed the code a bit; could you check the corrections out?
It feels like some timezones are missing, but I can't be sure 😄
Would it make sense to add the time diff after the entries in the list?

Hi @pernielsentikaer

I changed the code a bit; could you check the corrections out?

Good catches, thank you!

It feels like some timezones are missing, but I can't be sure

I'm using ECMAScript Internationalization API.

Would it make sense to add the time diff after the entries in the list?

That's a great ideia, what about like this?

It looks awesome @joaom182 🔥  I added a few details - can you give it another spin?

Great, I like the confirm action and the indication with colors of a destructive action in the menu! 🔥
I've changed the Colors.Red to Colors.Orange on the time diff tag item, to have a more neutral communication to not indicate something good/bad.
Also there is something odd happening with the list component 🤔
https://user-images.githubusercontent.com/3612098/219658885-40d3d33d-8ffb-4b83-948d-c8ebc4d43e8e.mov

Also there is something odd happening with the list component 🤔

It's a bug in Raycast; you didn't do anything wrong.
We can wait a few days and see if we can fix it before the next release or just release and hope it's fixed soon
What do you think?

@pernielsentikaer Let me remove that time diff for now, because that is what is causing the line break on the tags list. Once the issue is fixed, I bring that feature back.
Let's do it just for the time being to keep a consistent experience.
Does that sound good to you?

this issue should have been fixed in 1.57.0. Mind checking?

@sxn sure! I'll check it this weekend.


undefined method `banner' for main:Object (NoMethodError) on calling from command line

Hi,
I'm getting the following error when I try to use cobweb from command line. Here is the full stack trace:
#<Gem::Specification name=sidekiq version=4.1.4>
/Users/gustavo/.rvm/gems/ruby-2.3.1@site-shift/gems/cobweb-1.1.0/bin/cobweb:13:in `block in <top (required)>': undefined method `banner' for main:Object (NoMethodError)
	from /Users/gustavo/.rvm/gems/ruby-2.3.1@site-shift/gems/slop-4.3.0/lib/slop/options.rb:33:in `initialize'
	from /Users/gustavo/.rvm/gems/ruby-2.3.1@site-shift/gems/slop-4.3.0/lib/slop.rb:23:in `new'
	from /Users/gustavo/.rvm/gems/ruby-2.3.1@site-shift/gems/slop-4.3.0/lib/slop.rb:23:in `parse'
	from /Users/gustavo/.rvm/gems/ruby-2.3.1@site-shift/gems/cobweb-1.1.0/bin/cobweb:12:in `<top (required)>'
	from /Users/gustavo/.rvm/gems/ruby-2.3.1@site-shift/bin/cobweb:23:in `load'
	from /Users/gustavo/.rvm/gems/ruby-2.3.1@site-shift/bin/cobweb:23:in `<main>'
	from /Users/gustavo/.rvm/gems/ruby-2.3.1@site-shift/bin/ruby_executable_hooks:15:in `eval'
	from /Users/gustavo/.rvm/gems/ruby-2.3.1@site-shift/bin/ruby_executable_hooks:15:in `<main>'

The error occurs when I simply call cobweb --help.
I'm using:

Sidekiq 4.1.4
Redis Server 2.8.19

Is there anything I'm missing?
Thanks in advance,
Gustavo

Hi,
What version of the slop gem are you using?  There were quite a few breaking changes when they moved to v4 and above.  Should be around v 3.6.0 for the current cobweb code.
Stewart.


Sub-namspaces for Yaml mappings

I have a LOT of Yaml mappings, and I have had issues trying to "sub-namespace" them using the Yaml driver.
I am using the mongodb-odm-bundle, so if this is a question better suited for that project, please let me know.
My mappings are currently like this:
Acme\Document\Apple
Acme\Document\Orange

But I need to sub-namespace like this:
Acme\Document\Fruit\Apple
Acme\Document\Fruit\Apple

Nothing I try using Yaml seems to work.
Eg.
# Acme\Resources\config\doctrine\Apple.mongodb.yml
Acme\Document\Fruit\Apple:
    type: document
    fields:
        .....

Results in an invalid mapping error:

Invalid mapping file 'Acme.Apple.mongodb.yml' for class 'Acme\Apple'.

Hoping someone can point out what I'm doing wrong, or to let me know if sub-namspacing documents in Yaml is not possible.

Answered my own question after a little more tinkering:
Naming the mapping file with the sub-namespace seems to solve it:
Eg.

Acme\Fruit\Apple mapping file should be named Acme/Resources/config/doctrine/Fruit.Apple.mongodb.yml


[FEAT]: Client link to Advertising Statistics

Description
Login to Advertising Statistics
is not accessible from the frontend.
This link can be added to the theme's footer
since a landing page exists for advertizers.
Additional Information
No response

Wrap the link with condition if banners enabled !


Fosstodon has been down for at least several hours to this moment

Maybe it is time to post something to the Hub about the current outage?

Its up, but very slow. It was like this a few days ago, too.


Fix systemd main pid handling when running with --log

Fixes regression introduced in  https://github.com/k3s-io/k3s/pull/4115, reported by @kralicky
Proposed Changes
Using MAINPID breaks systemd's exit detection, as it stops watching the
original pid, but is unable to watch the new pid as it is not a child
of systemd itself. The best we can do is just notify when execing the child
process.
We also need to consolidate forking into a single place so that we don't
end up with multiple levels of child processes if both redirecting log
output and reaping child processes.
Types of Changes
bugfix
Verification

start k3s in docker without the docker --init flag; note that it does not leave behind zombie containerd-shim processes.
start k3s as systemd service with the --log flag; note that logs are written to the specified log file.
start k3s as systemd service with the --log flag; note that it is restarted properly when killed.

Linked Issues

#4189

User-Facing Change
Fixed an issue that caused k3s to not be restarted by systemd when started with the `--log` flag.

Further Comments

Codecov Report

Merging #4190 (ac3759f) into master (ac7a8d8) will increase coverage by 0.00%.
The diff coverage is 0.00%.

@@           Coverage Diff           @@
##           master    #4190   +/-   ##
=======================================
  Coverage   11.46%   11.47%
=======================================
  Files         134      135    +1
  Lines        9242     9237    -5
=======================================
  Hits         1060     1060
+ Misses       7956     7951    -5
  Partials      226      226

Flag
Coverage Δ

inttests
0.67% <0.00%> (+<0.01%)
:arrow_up:

unittests
11.12% <0.00%> (+<0.01%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

pkg/cli/agent/agent.go
0.00% <0.00%> (ø)

pkg/cli/cmds/init_linux.go
0.00% <0.00%> (ø)

pkg/cli/cmds/log.go
0.00% <0.00%> (ø)

pkg/cli/cmds/log_linux.go
0.00% <0.00%> (ø)

pkg/cli/server/server.go
0.00% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ac7a8d8...ac3759f. Read the comment docs.


Download the data (VOC, Cityscapes) and pre-trained models from [OneDrive link] is not found

Download the data (VOC, Cityscapes) and pre-trained models from [OneDrive link] is not found

I have also encountered this problem

Sorry, there is a little problem with my OneDrive account. I have updated the links. @yjq767579182 @KKaiYH

thank you very much

I hope this email finds you well. I am writing to inform you about an issue I encountered with the data dataset that was provided.
Upon attempting to access the data dataset, I discovered that it appears to be corrupt or damaged. Unfortunately, this prevents me from properly utilizing the dataset for my current project. I have made several attempts to extract and utilize the data, but all of them have been unsuccessful.
yaojiaqi
@.***

------------------ 原始邮件 ------------------
发件人:                                                                                "charlesCXK/TorchSemiSeg"                                                                                @.>;
发送时间: 2023年5月21日(星期天) 晚上10:10
@.>;
<EMAIL_ADDRESS>主题: Re: [charlesCXK/TorchSemiSeg] Download the data (VOC, Cityscapes) and pre-trained models from [OneDrive link] is not found (Issue #85)
Closed #85 as completed.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you were mentioned.Message ID: @.***>

Since 2021, I haven't made any modifications to the data. Could it be due to network issues? @yjq767579182


Document Types aren't passed down through World Collection + DirectoryCollectionMixin

game.actors.get() fails to return an Actor; instead, it's unsure about the proper document type. This is because of a variety of TODOs and un-implemented pass throughs with the mixin.

Fixed as of 88da04e4c5ade6beaf7db386393d8e77e9abe714.


Fix debug location is some Ownership utils

Insert point may have invalid LocationKind. Use auto-generated loc.

@swift-ci smoke test


[BUG] Fixed incorrect bounding enforcement

Reference Issues/PRs
#1810

What does this implement/fix? Explain your changes.
This PR changes the way in which MSM, TWE, ERP and EDR initialise their cost matrix. The reason for this change it to make them compatible with the bounding matrix format.

Specifically MSM, TWE, ERP and EDR now all initialise their cost matrix to infinite. This makes them compatible with how the bounding matrix is enforced in DTW (and similar distances).
Does your contribution introduce a new dependency? If yes, which one?

Any other comments?
The test values I updated is the test was using a window (and now the bounding is fixed the values have changed).
See issue for further details and explanation of the bug.

PR checklist

For all contributions

[x] I've added myself to the list of contributors. Alternatively, you can use the @all-contributors bot to do this for you.
[x] The PR title starts with either [ENH], [MNT], [DOC], [BUG], [REF], [DEP] or [GOV] indicating whether the PR topic is related to enhancement, maintenance, documentation, bugs, refactoring, deprecation or governance.

For new estimators and functions

[x] I've added the estimator to the online API documentation.
[x] (OPTIONAL) I've added myself as a __maintainer__ at the top of relevant files and want to be contacted regarding its maintenance. Unmaintained files may be removed. This is for the full file, and you should not add yourself if you are just making minor changes or do not want to help maintain its contents.

For developers with write access

[x] (OPTIONAL) I've updated aeon's CODEOWNERS to receive notifications about future changes to these files.

Nice ! @chrisholder could you add an example of before and after (for initializations only) ? to keep track of the changes, can be here or on the related issue

ah never mind i saw you already did put a code snippet on the issue :)