Implement VMLA linking and the rest of the TargetBackend linker interface

To clean up the IR and reduce startup cost it'd be nice to link all VMLA executables together. This will require completing the TargetBackend linking support, which means solving how executables are selected based on backend. Right now the command buffer conversion is performing executable lookup but ideally we'd insert hal.device.switch ops based on what the linker remaps things to - or make that part of a data set provided to the target backend during recordDispatch.
We could enable this by moving the executable lookup and descriptor set management into the recordDispatch. We'd then need to rely on command buffer-specific optimizations to CSE(ish) things (like deduping two blocks that dispatch the same way, etc), which is something we want anyway. In the short term it would mean inserting duplicate work in the case of multi-device builds, but that seems acceptable as for models like resnet50 we just want to see it for a single device at a time and this would clean things up a ton.
From there, linking in the VMLA case becomes just globbing all targets/entry points together and there's no requirement that entry points have the same interfaces so we can also move all those in. There should then be a single executable with N entry points. Any symbol deduplication can happen within this module and entry points can be combined, which is nice.
This is needed for the LLVM AOT case so proving it out now with VMLA will make that significantly easier.

One of the details to figure out here is how to support multiple interfaces within a single executable, so we can link across executables with different interfaces. (If we actually need to do that - the savings seem worth it?)
I think this relates to

We could enable this by moving the executable lookup and descriptor set management into the recordDispatch.

We currently declare a single hal.interface op per executable (independent of the number of target backends in use): https://github.com/google/iree/blob/73b3e1e9e9897982c6d7e4f1172473086c7931b6/iree/compiler/Dialect/HAL/Transforms/MaterializeInterfaces.cpp#L370-L376
I think that can stay, since the mapping from Flow::ExecutableOp -> HAL::ExecutableOp is straightforward.
During other IR transformations, I was thinking we could support multiple interfaces by listing them side-by-side and using entry points to associate them with translated functions:
  hal.executable @main_ex_dispatch_0 attributes {sym_visibility = "private"} {
    hal.interface @legacy_io_0 {
      hal.interface.binding @arg0, set=0, binding=0, type="StorageBuffer", access="Read"
      hal.interface.binding @arg1, set=0, binding=1, type="StorageBuffer", access="Read"
      hal.interface.binding @ret0, set=0, binding=2, type="StorageBuffer", access="Write|Discard"
    }
    hal.interface @legacy_io_1 {
      hal.interface.binding @arg0, set=0, binding=0, type="StorageBuffer", access="Read"
      hal.interface.binding @ret0, set=0, binding=1, type="StorageBuffer", access="Write|Discard"
    }
    hal.executable.target @vmla, filter="vmla" {
      hal.executable.entry_point @main_ex_dispatch_0 attributes {interface = @legacy_io_0, ordinal = 0 : i32, signature = (tensor<1x1xf32>, tensor<1x1xf32>) -> tensor<1x1xf32>}
      hal.executable.entry_point @main_ex_dispatch_1 attributes {interface = @legacy_io_1, ordinal = 0 : i32, signature = (tensor<1x1xf32>) -> tensor<1x1xf32>}
      ...

However, a few steps later, after linking will happen, we create ExecutableCachePrepareOps that require a "layout", derived from the single interfaceOp: https://github.com/google/iree/blob/73b3e1e9e9897982c6d7e4f1172473086c7931b6/iree/compiler/Dialect/HAL/Transforms/MaterializeResourceCaches.cpp#L234-L247
that goes all the way down to the runtime HAL: https://github.com/google/iree/blob/ef7f00260e0c4f5f1edda1832a9cd915be0060b3/iree/hal/api.cc#L1972-L1974
though the layout is only used by the Vulkan backend: https://github.com/google/iree/blob/ef7f00260e0c4f5f1edda1832a9cd915be0060b3/iree/hal/vulkan/pipeline_cache.cc#L52-L59
so now I'm wondering:

Does it still make sense to link executables with different interfaces together?
If so, what changes should be made to the HAL (particularly the Vulkan HAL)?

Another option could be to enable linking between executables with different interfaces in VMLA and LLVM backends, but not in Vulkan. It isn't immediately clear to me though how we'd implement that cleanly:

Would the base ExecutableCache::PrepareExecutable still take an ExecutableLayout parameter?
How would these restrictions be enforced in the IR? Just adding special logic to VulkanSPIRVTarget's implementation of TargetBackend::linkExecutables that only links between compatible executables would work as a start, but relaxing restrictions on HAL::ExecutableOp and other related changes may have some ripple effects.

Discussed with @benvanik and there's a hack we can use:

Change ExecutableOp::getInterfaceOp to ExecutableOp::getFirstInterfaceOp
In the VulkanSPIRV target and Vulkan HAL, continue using a single interface per executable and continue to pass the layout down to executable caching / pipeline creation
In other targets / HALs, support multiple interfaces per executable, and be sure to not introduce a dependency on the layout during executable caching

It's a sharp edge, and continuing gap between GPU and CPU backends, but it will unblock linking on CPU backends.
Longer term, we can rework how executables are cached (to batch them and make them async). The detail to consider here for Vulkan is that we create a vkpipeline, which is shader + pipeline state + pipeline layout + entry point (see VkComputePipelineCreateInfo?), unlike metal and d3d which don't require the entry point/layout to be tied together in the cache.

Question: for Vulkan/SPIR-V side, shouldn't we merge all hal.interface ops into one by reassigning the set/binding numbers? This will make sure we have one pipeline layout for the whole combined module and we just need to create one pipeline object.

we can do that to an extent, but we wouldn't want to pass 800 shaders as a single pipeline (I don't think, anyway) - so we could link together executables with similar interfaces or lifetime, but likely not the same as with llvm/vmla where everything can be in a single executable.
interfaces -> pipeline layouts are already deduped at runtime

pass 800 shaders as a single pipeline

You mean as a single shader module? For a compute pipeline I think we just specify one entry point from the shader module, but can certainly use a pipeline layout that has binding slots for an aggregation of entry points.
I'm asking because we have @KareemErgawy helping us right now to implement a SPIR-V combiner, so we will see SPIR-V "linking" utilities in MLIR working soon. :)

You need one pipeline per entry point and that's where all the work happens. So one shader module with 100 entry points means creating 100 pipelines, and as such the benefits of keeping things in the same shader module are trickier to balance. If the SPIR-V combiner can get us code deduplication (where the size of one shader module of 100 entry points is less than the size of 100 individual shader modules with one entry point) then it's worth it. That'll mean outlining/sharing blocks/etc.

Yup, certainly. There is no way to trim down the number of pipeline objects. My previous comment is more on that by combining all the shader modules into one and remap the descriptor set/binding numbers, we can reduce the number of descriptor set and pipeline layout objects. But I understand there is a balance to dance as we can see small number of maxDescriptorSetStorageBuffers.

If the SPIR-V combiner can get us code deduplication

That's the goal! :) But it may take some time to come.

Linking in VMLA is "done", with some room for further cleanup and optimizations. I'll close this and we can follow up in other issues.
I'll look at https://github.com/google/iree/issues/3137 next, and also update this comment: https://github.com/google/iree/blob/0a225f92503f493eba5c1c93aa153a2b44d5ed1b/iree/compiler/Dialect/HAL/Target/TargetBackend.h#L283-L289


websocket communication error undefined

Hi, help me with some advice. deconz is installed in docker. The internal port and the external port are different from each other. How can I install an external port? (Internal is automatically selected).
config:
"platforms": [
{
"name": "Hue",
"excludeSensorTypes": [
"ZLLSwitch",
"ZGPSwitch",
"ZLLPresence",
"ZLLLightLevel",
"ZLLTemperature",
"ZHAHumidity",
"ZHAPressure",
"CLIP"
],
"forceEveWeather": true,
"hosts": [
"<IP_ADDRESS>:6681"
],
"hueDimmerRepeat": true,
"hueMotionTemperatureHistory": true,
"lights": true,
"nativeHomeKitLights": true,
"nativeHomeKitSensors": true,
"nupnp": true,
"resource": true,
"sensors": true,
"wallSwitch": true,
"platform": "Hue"
}
Error:
[8/23/2019, 7:00:09 AM] [Hue] deconz: websocket communication error undefined on ws://<IP_ADDRESS>:443/

How can I install an external port?

You cannot.  Homebridge-hue connects to the websocket port that deCONZ reports; there’s no configuration option to change that.

The internal port and the external port are different from each other.

Why on Earth would you want to do that?  You can specify which ports deCONZ uses through command line parameters when starting deCONZ.  Both for the websocket port as well as for the API port.


docs: expand QL book entry on monotonic aggregates

Related to https://github.com/Semmle/ql/pull/2829 (and partially fixes https://github.com/github/product-documentation/issues/2188).
I've moved Monotonic aggregates in QL into the QL language handbook, so that we can remove the 'Advanced QL' section altogether before we move any of the CodeQL docs.
@alexet: I've tried to simplify the content into couple of bullet points, but I've left in the full example for the recursive use. Please can you review the changes?
After this is merged I'll open a separate PR to do the final bits of tidying up of Advanced QL.

Preview: https://jenkins.internal.semmle.com/job/Docs/job/Generate-Sphinx/1280/artifact/target/sphinx/ql-handbook/expressions.html#monotonic-aggregates

@alexet over to you for a technical review please


Modal covers status bar but leaves gap at bottom on some devices

Modal covers status bar but leaves gap at bottom on some devices

modal下面有个空白区，高度和statusbar同样高

@lijianyu1985 Can you tell me the specific model number?

Invariant Violation: Invariant Violation: requireNativeComponent: "RCTTranslucentModalHostView" was not found in the UIManager.
Expo 34
Android

hi, see that you have no feedback, I will close the issue first, if there are still problems, please provide the code and more information


feat: adds support for non-submodule repos

Feature request from #3
This PR adds support for non-submodule repos by parsing through the directories to find .git.
The following default extension's configs were added to support this feature:
find_subdirectories = {
    enabled = false,
    depth = 1
}

Next steps

[ ] Add documentation in README.md
[ ] Tested/Reviewed by a user
[ ] Remove print() lines

Found an issue while testing, taking note here to refer to later when I have some time to look at it.
When find_subdirectories is enabled and on a neo-tree window I receive the below error:
E5108: Error executing lua: ...odules.nvim/lua/telescope/_extensions/git_submodules.lua:21: attempt to index a nil value
stack traceback:
	...odules.nvim/lua/telescope/_extensions/git_submodules.lua:21: in function 'open_git_tool'
	...odules.nvim/lua/telescope/_extensions/git_submodules.lua:209: in function 'run_replace_or_original'
	...re/nvim/lazy/telescope.nvim/lua/telescope/actions/mt.lua:65: in function 'key_func'
	...hare/nvim/lazy/telescope.nvim/lua/telescope/mappings.lua:269: in function <...hare/nvim/lazy/telescope.nvim/lua/telescope/mappings.lua:268>


App directory [lang] paths return 404 from NextJS v13.4.13

What version of this package are you using?
v2.5.3
What operating system, Node.js, and npm version?
MacOS, v18.7.0, 8.15.0
What happened?
Navigating to app directory pages with the [lang] path parameter returns 404 when NextJS is upgraded to 13.4.13 or later.
I have created a simple reproduction of the issue in this repo. I simply used the NextJS official example for next-translate. When I first started it up it was working, but then once I upgraded NextJS and restarted the server, it started getting 404. I tried different versions of NextJS until I found that v13.4.12 works but v13.4.13 and later do not.
What did you expect to happen?
Page loads successfully.
Are you willing to submit a pull request to fix this bug?
I am motivated to get this fixed quickly, so I would be willing to help but I am not familiar with the internals of next-translate or next. With some guidance I would be happy to do the heavy lifting.
There are quite a lot of changes from 13.4.12 to 13.4.13, so it's not immediately clear why this broke.

There were a couple changes around basePath, maybe that's a good place to look.

https://github.com/vercel/next.js/pull/53174
https://github.com/vercel/next.js/pull/53189

next.config.js
setting i18n to undefined worked for me, idk if this has any drawbacks
const nextTranslate = require("next-translate-plugin");
const config = nextTranslate({});
config.i18n = undefined;

module.exports = config;

next.config.js
setting i18n to undefined worked for me, idk if this has any drawbacks
const nextTranslate = require("next-translate-plugin");
const config = nextTranslate({});
config.i18n = undefined;

module.exports = config;

I can confirm this removes the issue, it also helps with https://github.com/aralroca/next-translate/issues/1141, but, as you've said, I don't know what the drawbacks could be.

setting i18n to undefined worked for me, idk if this has any drawbacks

Nice find, this seems like a good workaround if only app directory is used. The i18n attribute is still used for the pages directory. In our case, we have an existing application on pages and are trying to migrate it to app directory incrementally, so we need it working in both.

The problem is Next.js normalizes nextUrl.pathname by removing the locale when i18n config is present. But you're right, the config needs to be present in the case of incremental migration, so I think this is something that needs to be solved in their end.

I actually found a bug on the NextJS side that may be the cause of this: https://github.com/vercel/next.js/issues/53724

Setting i18n to undefined solved my problem but now I see that the locale in the request.nextUrl is removed by nextJS, any work arounds for this?

Same here. Setting i18n to undefined fixes the issue but I would love a more permanent and less hacky solution

The i18n is there to support migration from pages to app, i18n is required for pages, but not for app. Feel free for now to remove it. Probably we are going to drop the support for the migration from pages to app BECAUSE NEXT.JS is not supporting this migration, there are a lot of issues to have the i18n in app router, when this i18n should be used only in pages without interfering to app router... :/

Hi everyone.
config.i18n = undefined; solved the issue.
But now i'm getting errors during production build while nextjs is generating static pages:
Error occurred prerendering page "/500".
RangeError: Incorrect locale information provided

Error occurred prerendering page "/404".
RangeError: Incorrect locale information provided

Export encountered errors on following paths:
        /_error: /404
        /_error: /500

I tried to bring back i18n to next config (removed line config.i18n = undefined;) -> got previous issue but next build completed without errors.
I'm not using pages folder so would like to continue using config.i18n = undefined; because it makes application behaviour more predictable.
For more context: i don't have custom error pages. Errors occurred in default nextjs error pages.

Hi everyone. config.i18n = undefined; solved the issue.
But now i'm getting errors during production build while nextjs is generating static pages:
Error occurred prerendering page "/500".
RangeError: Incorrect locale information provided

Error occurred prerendering page "/404".
RangeError: Incorrect locale information provided

Export encountered errors on following paths:
        /_error: /404
        /_error: /500

I tried to bring back i18n to next config (removed line config.i18n = undefined;) -> got previous issue but next build completed without errors.
I'm not using pages folder so would like to continue using config.i18n = undefined; because it makes application behaviour more predictable. For more context: i don't have custom error pages. Errors occurred in default nextjs error pages.

ah...
checked new release (2.6.2) and it contains this fix.
thank you.

okay, so I am on 2.6.2 and using only the app router. I am not removing i18n from the next.config.js object anymore, but then I get the following error:
Error occurred prerendering page "/en/dashboard/products". Read more: https://nextjs.org/docs/messages/prerender-error
Error: The provided export path '/dashboard/products' doesn't match the '/[lang]/dashboard/products' page.
Read more: https://nextjs.org/docs/messages/export-path-mismatch
    at getParams (/Users/bjoernrave/projects/valooba-mono/apps/pds/node_modules/next/dist/export/helpers/get-params.js:27:15)
    at exportPageImpl (/Users/bjoernrave/projects/valooba-mono/apps/pds/node_modules/next/dist/export/worker.js:86:47)
    at /Users/bjoernrave/projects/valooba-mono/apps/pds/node_modules/next/dist/export/worker.js:214:22
    at Span.traceAsyncFn (/Users/bjoernrave/projects/valooba-mono/apps/pds/node_modules/next/dist/trace/trace.js:105:26)
    at Object.exportPage (/Users/bjoernrave/projects/valooba-mono/apps/pds/node_modules/next/dist/export/worker.js:213:41)
    at execFunction (/Users/bjoernrave/projects/valooba-mono/apps/pds/node_modules/next/dist/compiled/jest-worker/processChild.js:1:2764)
    at execHelper (/Users/bjoernrave/projects/valooba-mono/apps/pds/node_modules/next/dist/compiled/jest-worker/processChild.js:1:2422)
    at execMethod (/Users/bjoernrave/projects/valooba-mono/apps/pds/node_modules/next/dist/compiled/jest-worker/processChild.js:1:2510)
    at process.messageListener (/Users/bjoernrave/projects/valooba-mono/apps/pds/node_modules/next/dist/compiled/jest-worker/processChild.js:1:1228)
 ✓ Generating static pages (24/24)

> Export encountered errors on following paths:
        /[lang]/dashboard/order-history/page: /de/dashboard/order-history
        /[lang]/dashboard/order-history/page: /en/dashboard/order-history
        /[lang]/dashboard/production/page: /de/dashboard/production
        /[lang]/dashboard/production/page: /en/dashboard/production
        /[lang]/dashboard/products/page: /de/dashboard/products
        /[lang]/dashboard/products/page: /en/dashboard/products
        /[lang]/dashboard/service-calls/page: /de/dashboard/service-calls
        /[lang]/dashboard/service-calls/page: /en/dashboard/service-calls
        /[lang]/dashboard/settings/page: /de/dashboard/settings
        /[lang]/dashboard/settings/page: /en/dashboard/settings
        /[lang]/dashboard/table-plan/page: /de/dashboard/table-plan
        /[lang]/dashboard/table-plan/page: /en/dashboard/table-plan
        /[lang]/dashboard/time-tracking/page: /de/dashboard/time-tracking
        /[lang]/dashboard/time-tracking/page: /en/dashboard/time-tracking
        /[lang]/page: /de
        /[lang]/page: /en
        /_error: /404
        /_error: /500
        /_error: /de/404
        /_error: /de/500
        /_error: /en/404
        /_error: /en/500

Anyone has the same problem or knows how to fix?

Hi, config.i18n = undefined; solved my issue too.
It was hard to find this fix because I wasn't aware that the error started with version 13.4.13. (I upgraded from 13.4.12 to 14.0.0)
Should we update the README with this fix or wait for a better one?
Thanks!

I'm using pages router and I have a redirect issue on the newer next.js version.
I use this as a temporary solution:
const config = nextTranslate(nextConfig);
config.i18n.localeDetection = false;

module.exports = config;

If it also works for you, it's atleast better than setting i18n to undefined I guess?

Can you tried in 3.0.0-canary.1 version? (both, next-translate & next-translate-plugin). Thanks

One thing that wasn't noted, or maybe it's just me, @abuu-u's solution causes fast refresh to break, I am forced to disable translations while developing if I want fast refresh to work


Make Firebase Storage compile and work in independent watch app

I added a simple use case of Storage that apple watch is able to load image successfully.

Please add a watchOS build test to the matrix in https://github.com/firebase/firebase-ios-sdk/blob/master/.github/workflows/storage.yml

I wonder if we need to bump up both FirebaseStorage and FirebaseAuthInterop's versions

It's some kind of test config problem.
In the log:
[!] CocoaPods could not find compatible versions for pod "FirebaseAuthInterop":
  In Podfile:
    FirebaseAuth (from `/Users/travis/build/firebase/firebase-ios-sdk/FirebaseAuth.podspec`) was resolved to 6.4.3, which depends on
      FirebaseAuthInterop (~> 1.0)
    FirebaseStorage (from `/Users/travis/build/firebase/firebase-ios-sdk/FirebaseStorage.podspec`) was resolved to 3.6.0, which depends on
      FirebaseAuthInterop (~> 1.1)
Specs satisfying the `FirebaseAuthInterop (~> 1.1), FirebaseAuthInterop (~> 1.0)` dependency were found, but they required a higher minimum deployment target.

I'll be able to help on this later this week

@doudounan is working on a fix of the pod lib lint error in watchOS introduced in #4944. It's strange it only got caught up in the Storage pod lib lint. @mikehaney24

This is a different problem.   The Interop pods should be in the list here - https://github.com/firebase/firebase-ios-sdk/blob/master/scripts/localize_podfile.swift#L27


Why bn_feat rather than feat is used in evaluation when feature before or after BN is used in training?

https://github.com/JDAI-CV/fast-reid/blob/a5550d77254faca06be4b4794c5ed6dcfa7da0bc/fastreid/modeling/heads/reduction_head.py#L55
https://github.com/JDAI-CV/fast-reid/blob/a5550d77254faca06be4b4794c5ed6dcfa7da0bc/fastreid/modeling/heads/reduction_head.py#L63
Hello, I don't know much about ReID. Can anyone help me figure out why feature adopted in training and evaluation is different?

Experimental results show that performance is the best in this way.

Experimental results show that performance is the best in this way.

Thanks for your reply. If I want to distill feature from teacher, I should use the feature before BN. Right?

You can try both features before BN and after BN to see which one can get a good result.
Actually, we try to distill model with feature between teacher and student model, but cannot get a good result. Our best result can be got by kl loss from classification logits between teacher and student model.

@L1aoXingyu Great thanks for sharing your experience. I will try KL loss next. Yesterday, I distilled features before BN by directly using L2 loss, it seems to work fine.


Update tag-driven-release machinery to the state of the art

Ala scala-java8-compat

https://github.com/scala/scala-java8-compat/blob/master/admin/README.md

this happened (6afb291eea3dfadb889a3ea622d73457c2d4e3ef)


Sort out request flags

At the moment the request flags are always being set to: 0x00000020
This means 'Implicit /uncached':
http://owfs.org/index.php?page=owserver-flag-word
It may be useful to be able to choose the flags to set.

I changed the request flags in 39d5fbb5ca680111f6250089cde8667208ae8bc7 to request | alias | uncached.
It would be good to be able to make this configurable.


One core to rule them all

Documentation Is:

[ ] Missing
[ ] Needed
[ ] Confusing
[X] Not Sure?

Explanation
In the readme we can read the below:
2. With this generic structure, everyone can create his own modules and share them with others. Therefore there is only one core (to govern all of them).
Proposal
The lord of the cores would require this to be written as proposed:
2. With this generic structure, everyone can create his own modules and share them with others. Therefore there is only one core (to rule them all).

Hello @Xavier-Brinon,
Thanks, I love it! I changed it in the readme and the documentation:

https://github.com/leon-ai/leon/commit/c74dda4cb9acc78de143ae01fdc6b4ef0a5ec3ef
https://github.com/leon-ai/docs.getleon.ai/commit/11b0aa5e4aa0c932361344357fe72391c0c782fb


player.markers.add doesn't work

I'm working on a project where i need to add markers from a json array. However adding markers dinamycly is not possible. This is my code:
player.markers({
    markers: []
});

if(bookmarks_array.length !== 0){
    bookmarks_array.forEach(function(bookmark) {
        console.log(bookmark);

        var color_bookmark = "";

        if(bookmark.public == true){
            color_bookmark = "green";
        }else{
            color_bookmark = "blue";
        }

        player.markers.add({
            markers: [{
                time: bookmark.postion,
                text: bookmark.content,
                class: color_bookmark
            }]
        });

    });
}

The error i'm getting is:

VIDEOJS: ERROR: TypeError: Cannot read property 'time' of undefined
at Object.time (/js/video-js/videojs-markers/videojs.markers.js:17:22)
at e.onTimeUpdate (/js/video-js/videojs-markers/videojs.markers.js:251:45)
at HTMLDivElement.r (/js/video-js/video.min.js:2:378)
at HTMLDivElement.U.r.dispatcher.r.dispatcher (/js/video-js/video.min.js:1:6923)
at X (/js/video-js/video.min.js:1:7602)
at e.trigger (/js/video-js/video.min.js:2:2969)
at e.so.(anonymous function) [as handleTechTimeupdate_] (/js/video-js/video.min.js:6:17654)
at e.r (/js/video-js/video.min.js:2:378)
at HTMLVideoElement.r (/js/video-js/video.min.js:2:378)
at HTMLVideoElement.U.r.dispatcher.r.dispatcher (/js/video-js/video.min.js:1:6923)

Sometimes i also get the error that the function "player.markers.add" does not exists. Any suggestions at what i'm doing wrong?

You are sending an object with markers property as an array to player.markers.add function. You should just send the array containing the marker descriptors.

As @ikovic  suggested, for the add function, you only need to pass in an array of marker objects. so something like
player.markers.add([{time: ..., text: ..., class: ...}]);


Feature References Request

Do you have any plan to support references?
Example:
{
  "references": [
    {
      "name": "RtValidateReqDto",
      "subject": "fip.common.RtValidateReqDto",
      "version": 1
    },
    {
      "name": "RtResDto",
      "subject": "fip.common.RtResDto",
      "version": 1
    }
  ],
  "schema":"{\"namespace\":\"fip.common\",\"name\":\"RtValidateDto\",\"type\":\"record\",\"fields\":[{\"name\":\"err\",\"type\":[\"null\",\"string\"],\"default\":null},{\"name\":\"req\",\"type\":[\"null\",\"RtValidateReqDto\"],\"default\":null},{\"name\":\"res\",\"type\":[\"null\",\"RtResDto\"],\"default\":null}]}",
  "schemaType":"AVRO"
}

Another attempt at references:  https://github.com/kafkajs/confluent-schema-registry/pull/121
Can we get one of these merged?

Hi Folks. I would love for this project to support Protocol Buffer references (imports). I have written up a PR that implements the first step of this support over in https://github.com/kafkajs/confluent-schema-registry/pull/135
I have just updated the description to explain the context, how it works, and what's still missing (as far as I know). I would greatly appreciate if one of you would take a look and tell me if it makes sense, or if there are parts that are not easy to follow 👍

Perhaps we can team up and create a little task force for all of us who are interested in adding support for references :)


Allow default to be set on the objectType

It would be nice to be able to set a default rule on the object type with overrides on each field that defines one.

Partially fixed by #59, but it breaks the typing and I still have no idea why.

@lewebsimple Not sure how good you are with advanced typescript but I could use a hand on that one if you have time.
I put most details in there: https://github.com/graphql-nexus/nexus/issues/568
The issue is quite easy to reproduce and I could not find a good fix for it.

@Sytten hmmm I'm no TypeScript wizard so I compared your code to the official fieldAuthorizePlugin which should be similar in scope.
At first glance, the corresponding types FieldShieldResolver and FieldAuthorizeResolver differ in a significant way.. while FieldAuthorizeResolver has a (function) signature of (root, args, ctx, info) => MaybePromise<boolean | Error>, yours is of type { resolve: (root, args, ctx, info, options) => Promise<ShieldRuleResult> }.
So maybe using the .resolve type instead of the (parent) FieldShieldResolver could fix that ?
Also, I fail to see the difference between ShieldRule and ShieldRuleFunction.

It also occurs to me that the generated type differ from what you state in the other issue:
This is what you wrote:
interface NexusGenPluginTypeConfig<TypeName extends string> {
  shield?: ShieldRule<TypeName, any>
}

This is what I get in my project:
interface NexusGenPluginTypeConfig<TypeName extends string> {
  shield?: ObjectTypeShieldResolver<TypeName>;
}

where ObjectTypeShieldResolver<TypeName> corresponds to ShieldRule<TypeName, never>
Should it be any instead of never ?

Yeah I tried to simplify it, the ObjectTypeShieldResolver is just an alias type that plugins are supposed to use:
https://github.com/Sytten/nexus-shield/blob/master/src/plugin.ts#L48
Its import process that is a bit strange.
For some reason TS cant infer the real type and falls back to string which backs populate to the object and messes up the whole typing.

Did you base your code off of the official fieldAuthorize plugin ?

Similar yes, but I reused the class based system of graphql-shield. I did my best on the TS trickery (some of it is explained at the end of the README), but this last problem I really have no clue. I would need better AST debugging tooling to understand why the compiler cant infer the parameter on object but can on the resolvers.

does it still not work?
objectType.shield seems to not trigger:
export const Post = objectType({
  name: 'Post',
  definition(t) {
    t.model.author()
    t.model.title()
    t.model.sourceId()
    t.model.content()
    t.model.createdAt()
    t.model.published()
    t.model.authorId()
  },
  shield: {
    resolve: async (root, args, ctx, info, options) => {
      console.log('here')

      return false
    }
  }
})

console: empty

Im getting this error when using t.field, t.string works fine

I am not really sure why the typing is broken. I don't have a lot of time to work that, so if it is important for you I would suggest you take a look at the code (will review PRs).

Just a note for later, with the new in/out keyword for typescript there is probably a way to force the compiler to understand that the type used is an in and it should not override the assignee typing.


flipped base and counter currency for poloniex functions

see https://github.com/dutu/cryptox/issues/4

Thank you for reporting the issue and PR


Add perturbing (me) as a CIP editor

With this PR I would like to put myself forward in the role as a CIP editor. Given this PR, I would love to fill these big shoes and contribute to the continuation of the CIP process as it is now.
A big thanks to both @KtorZ and @SebastienGllmt for their effort and, above all, their great technical minds.
Regarding my competence for this role, I have contributed to various CIP's and even co-authored some myself. I hope others recognize this and have faith in my ability to execute this role successfully :smiley:

I vouch for this guy, for what its worth. 😇😅


Filter parameter and its filtering syntax - no result for LIKE

Hello,
"The Filter parameter uses the same filtering syntax that is used by the Where-Object cmdlet."
First of all, think you for this switch, it is a time-saver. My search with "Where-Object" runs a little over 5 mins, whereas the Filter is finished in under 30s!
This is all great, but it only works for -EQ for me. -LIKE is not as successful
Get-CsOnlineUser | Where-Object LineURI -Like "*$Number*"
This returns an Object as my $Number is specified as part the LineURI string.
Get-CsOnlineUser -Filter { LineURI -like "*$Number*" }
This does not return anything, just finishes immediately.
Is -LIKE implemented for Filtering?
I had a quick look at Get-AzureADUser, and there the Filter is implemented as an oData v3.0 filter statement (which does not have LIKE, sadly...)
Thank you,
David

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 7008ec10-a056-d8bb-bb4e-b4690a84adfd
Version Independent ID: 2dcb78d9-7f36-208e-0989-184905d73011
Content: Get-CsOnlineUser (SkypeForBusiness)
Content Source: skype/skype-ps/skype/Get-CsOnlineUser.md
Service: skypeforbusiness-powershell
GitHub Login: @tomkau
Microsoft Alias: tomkau

Hello @DEberhardt how are you?
This was discussed recently, please see here, that is why Example 6 was added.
Thank you for your feedback!

Hello,
thank you. Example 6 is helpful when it comes to values in variables, though I do think this worked for me in my script. Will test :)
However, the LIKE operator was never mentioned anywhere. Will try output when adhering to the example and let you know my findings.
Thanks,
David

@DEberhardt Ok David, let me know how it goes!

Success! - Example 6 also holds for the Like operator :+1:
$filterString = 'LineURI -like "*{0}*"' -f $Number
Get-CsOnlineUser -Filter $filterString | Select UserprincipalName,LineURI
Cheers!

Thanks for returning and sharing your results @DEberhardt David!


potential race condition prevents 'unix.rm' from deleting directory

Raised in #205 by @scott314159, there seems to be a race condition during the finalization command of the Pyaflowa preprocessing module where the main job tries to delete the log file directory but returns OSError: [Errno 39] Directory not empty , which traces back from shutil.rmtree.
This seems to be a know issue with shutil.rmtree, see e.g., https://stackoverflow.com/questions/11228079/python-remove-directory-error-file-exists that happens on NFS file systems where file locks are present.
My thinking is that this is a race condition where the file locks cannot be removed before the rm command tries to delete. One potential try is to put a try-except clause in unix.rm that waits a few seconds if it hits this OSError, to give the filesystem some time to remove it's lock.
https://github.com/adjtomo/seisflows/blob/d7661ee32c8deef7574a89d31290a96453324777/seisflows/tools/unix.py#L197-L205

Thanks.  Once I get to picking apart the scripts I will consider this.
As an aside, I am not sure why log files would be removed as part of the process?  If the information in the log file is potentially useful it should not be deleted so the user can review the logs if needed.  If the log files are not useful then why write them?  I know that you are just calling Pyaflowa as part of the seisflows scripts so you don’t necessarily have control over the behavior.
From: Bryant Chow @.>
Sent: Sunday, March 31, 2024 12:13 PM
To: adjtomo/seisflows @.>
Cc: scott314159 @.>; Mention @.>
Subject: [adjtomo/seisflows] potential race condition prevents 'unix.rm' from deleting directory (Issue #206)
Raised in #205 https://github.com/adjtomo/seisflows/issues/205  by @scott314159 https://github.com/scott314159 , there seems to be a race condition during the finalization command of the Pyaflowa preprocessing module where the main job tries to delete the log file directory but returns OSError: [Errno 39] Directory not empty , which traces back from shutil.rmtree.
This seems to be a know issue with shutil.rmtree, see e.g., https://stackoverflow.com/questions/11228079/python-remove-directory-error-file-exists that happens on NFS file systems where file locks are present.
My thinking is that this is a race condition where the file locks cannot be removed before the rm command tries to delete. One potential try is to put a try-except clause in unix.rm that waits a few seconds if it hits this OSError, to give the filesystem some time to remove it's lock.
—
Reply to this email directly, view it on GitHub https://github.com/adjtomo/seisflows/issues/206 , or unsubscribe https://github.com/notifications/unsubscribe-auth/BHNHRJTH4WA2L73OI5DDNH3Y3BG2NAVCNFSM6AAAAABFQTI6L6VHI2DSMVQWIX3LMV43ASLTON2WKOZSGIYTOMJRGY4TEMQ .
You are receiving this because you were mentioned.  https://github.com/notifications/beacon/BHNHRJTU2MOWFRRIBCHDUTDY3BG2NA5CNFSM6AAAAABFQTI6L6WGG33NNVSW45C7OR4XAZNFJFZXG5LFVJRW63LNMVXHIX3JMTHIIJUE7I.gif Message ID: @.*** @.***> >

Hey @scott314159, again, sorry for the long pause in communication here! Yeah the log message removal is intended to reduce file count, as for large-scale problems there may be (hundreds of) thousands of files created which may stress file systems, but I agree there is usually useful information there. The parameter export_log_files is meant to make it a User decision to save these files by copying them to a more permanent location, and is set True by default.


Windows 6.5 version - Wallet freezes on start.

This issue tracker is only for technical issues related to ALQO.
General ALQO questions and/or support requests and are best directed to the ALQO Discord.
Describe the issue
Locked Wallet freezes on start. Have tried on two different machines. See screenshot.
Can you reliably reproduce the issue?
If so, please list the steps to reproduce below:

Expected behavior
Wallet should sync.
Actual behavior
Wallet works find the 5-7 seconds, then freezes.
Screenshots.

What version of ALQO are you using?
Windows 6.5 version
Machine specs:

OS: Windows
CPU: Intel(R) Core(TM) i5-2450 CPU@ 2.5GHz
RAM: 8Gb
Disk size: 1TB
Disk Type (HD/SDD): SDD

Any extra information that might be useful in the debugging process.
This is normally the contents of a debug.log, db.log or config.log file. Raw text or a link to a pastebin type site are preferred.

hi @meraki10 , can you please provide more details please, like the last section of your debug log and a complete screenshot.

Hey, so this is what i did. I remember there were issues syncing locked wallets in 6.4 version also. So i just deleted my wallet.dat file and restarted the wallet. And voila, it fully got synchronized. I closed and restarted the wallet several times and it ran just fine.
After having tested that the newly created wallet is working fine, I deleted and replaced it with my backup wallet.dat which is pass protected. And as soon as i start the wallet(although i was just able to see my balance), it freezes again.

I figured that it might be my wallet file that is corrupted, so once again delted the pass protected wallet file and re ran the alqo wallet hoping that the wallet should run just fine as it did before when i had deleted my wallet file. However this time again it freezes on start and i keep getting warning to keep checking my pc's date(which is update as per internet time).

Happy to announce i was able to resolve this by following steps:

Delete original wallet.dat file(take backup for later use).
Reset time settings - Both time and time zone(disable and re enable it).
Rerun wallet, a new wallet file is created and wallet syncs properly.
Replace new wallet.dat with my backup file.

PS- For some reason the above process only worked when i created a new wallet.dat file. In first attempt i tried with my original wallet file but wallet would not sync.


Make sure that coverage is over 90% in codecov by refactoring

Describe the bug
Current coverage is at 87%.
Affected Component(s)

[X] Back-End (Database, Microservices, Containers, etc)
[X] Front-End (Flask, API, etc)
[ ] Docs
[ ] Build System

Additional context
https://codecov.io/gh/FORTH-ICS-INSPIRE/artemis

Coverage hit 90%, this can stop being an issue and try to keep it as high in the future.


Allow a low memory option for index construction

This splits out the internals of of the NNdescent loop to either use or skip the use of the tried cache depending on whether the user selects a low_memory option which defaults to False.

Thanks for the suggestions!


Python3 need to be rebuilt after final system clang is built

When compiling Cython, Build fails:
error: command 'x86_64-pc-linux-musl-clang' failed: No such file or directory

Perhaps Python3 should either be rebuilt after final system clang OR compiled for llvmtools.

And if building graphene:
FAILED: src/Graphene-1.0.gir
env PKG_CONFIG_PATH=/opt/Xorg/lib/pkgconfig:/opt/Xorg/share/pkgconfig:/src/graphene-1.10.8/OUT/meson-uninstalled /usr/bin/g-ir-scanner --quiet --no-lib
tool --namespace=Graphene --nsversion=1.0 --warn-all --output src/Graphene-1.0.gir --c-include=graphene-gobject.h '--identifier-filter-cmd=/usr/bin/pyt
hon3 /src/graphene-1.10.8/src/identfilter.py' --accept-unprefixed --quiet --warn-all -DGRAPHENE_COMPILATION -I/src/graphene-1.10.8/src -I/src/graphene-
1.10.8/OUT/src -I/src/graphene-1.10.8/include/. -I/src/graphene-1.10.8/OUT/include/. --filelist=/src/graphene-1.10.8/OUT/src/libgraphene-1.0.so.0.1000.
8.p/Graphene_1.0_gir_filelist --include=GObject-2.0 --symbol-prefix=graphene --identifier-prefix=Graphene --pkg-export=graphene-gobject-1.0 --cflags-be
gin -D_GNU_SOURCE -I/src/graphene-1.10.8/include/. -I/src/graphene-1.10.8/OUT/include/. -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I/usr/incl
ude/gobject-introspection-1.0 --cflags-end -I/src/graphene-1.10.8/include/. -I/src/graphene-1.10.8/OUT/include/. --add-include-path=/usr/share/gir-1.0
--add-include-path=/src/graphene-1.10.8/include/. --add-include-path=/src/graphene-1.10.8/OUT/include/. -L/src/graphene-1.10.8/OUT/src --library graphe
ne-1.0 --extra-library=m --extra-library=gobject-2.0 --extra-library=glib-2.0 --extra-library=girepository-1.0 --sources-top-dirs /src/graphene-1.10.8/
 --sources-top-dirs /src/graphene-1.10.8/OUT/
Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/setuptools/_distutils/spawn.py", line 57, in spawn
    proc = subprocess.Popen(cmd, env=env)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/subprocess.py", line 1024, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib/python3.11/subprocess.py", line 1901, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'x86_64-pc-linux-musl-clang'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/setuptools/_distutils/unixccompiler.py", line 179, in preprocess
    self.spawn(pp_args)
  File "/usr/lib/python3.11/site-packages/setuptools/_distutils/ccompiler.py", line 1007, in spawn
    spawn(cmd, dry_run=self.dry_run, **kwargs)
  File "/usr/lib/python3.11/site-packages/setuptools/_distutils/spawn.py", line 63, in spawn
    raise DistutilsExecError(
distutils.errors.DistutilsExecError: command 'x86_64-pc-linux-musl-clang' failed: No such file or directory

Build succeeds if a link is made:
sudo ln -sv clang-15 /usr/bin/x86_64-pc-linux-musl-clang

Also intltool wasnt built for final system

python3 built for stage1 toolchain, via commit 9bca49d79657eead622d9ced2066ce1e721e7b43

For LLVM-15.0.6 build [on i686], python3 [for final system] build locks up under chroot. Last build command seen:
clang -shared build/temp.linux-i686-3.11/sources/Python-3.11.3/Modules/_ctypes/_ctypes.o build/temp.linux-i686-3.11/sources/Python-3.11.3/Modules/_ctypes/callbacks.o build/temp.linux-i686-3.11/sources/Python-3.11.3/Modules/_ctypes/callproc.o build/temp.linux-i686-3.11/sources/Python-3.11.3/Modules/_ctypes/cfield.o build/temp.linux-i686-3.11/sources/Python-3.11.3/Modules/_ctypes/stgdict.o -L. -L/usr/local/lib -lffi -ldl -o build/lib.linux-i686-3.11/_ctypes.cpython-311-i386-linux-musl.so

I'm not sure if its an issue with the hardware OR with my build instructions.
I check if my amd64 build has the same issue

Fixed. Python3 now built after final system LLVM

Fixed. Python3 now built after final system LLVM

What was the problem then?

The original problem (cannot use python3 from llvmtools as final system python3... was hoping to build python3 once for both llvmtools and final system) was resolved by building python once more.
I'll open another issue for the 32bit X86 build

Issue for python 32-bit build stalling under chroot: https://github.com/dslm4515/CMLFS/issues/102


Better documentation and error messages when namerd io.l5d.thriftNameInterpreter misconfigured

I ran into these issues setting up a simple linkerd + namerd configuration.
I made the mistake of omitting the io.l5d.thriftNameInterpreter interpreter from namerd's config.yml file.  In the linkerd config.yml, I created a io.l5d.namerd interpreter which pointed to the namerd io.l5d.httpController port.
This is what I observed:

linkerd would not resolve anything.  The only error was com.twitter.finagle.RequestTimeoutException: exceeded 10.seconds to unspecified while dyn binding /http/1.1/GET/web. Remote Info: Not Available which is not very informative.  I wasted some time trying to figure out whether linkerd could even talk to namerd.  Simply stating that no response was received from namerd or that the response was in the wrong format would have been helpful.

linkerd's dtab and /delegator screens in the admin interface failed to load.  No useful error was given.

namerd showed no logging output even though I assume it had received an incorrectly formatted request on it's httpController port.  Also, there was no warning about a thriftController not being configured.  I would expect a warning since the primary use case is to be paired with linkerd.

AFAIK, there is no mention in the documentation that the thriftController must be present to talk to linkerd.

I've been looking into linkerd + namerd combination too and ran into the same issues.
I might add that I think in general the documentation is very unclear, and mostly not up to date with the implementation, and that the logging of both services is extremely bad. I case something doesn't work, the logs give no direction as to what the issue can be.

Hi @brendanyounger!  Sorry that you got tripped up on this.  This is a really common error so any suggestions on how to make this more user friendly would be appreciated.  I'll give you some background on each of your points.

When linkerd sends a thrift request to namerd's http interface, namerd does not respond.  I'm not sure exactly why this is the case, perhaps at the protocol level an HTTP message is indistinguishable from an incomplete thrift request?  Normally we could just set a timeout on this request and fail it if namerd doesn't respond within the timeout but this interface uses long-polling where namerd isn't expected to respond until the data changes.  This makes setting a timeout impossible.  We could investigate exactly why namerd can't fail the request immediately when it receives a message of the wrong protocol but in the medium-term we want to move away from this thrift long-polling interface in favor of gPRC anyway (https://github.com/BuoyantIO/linkerd/issues/842).
Adding a timeout on the delegator screen, on the other hand, makes total sense.  Issue is here: https://github.com/BuoyantIO/linkerd/issues/525.  PRs welcome :)
I don't think a warning when the thrift namer interface is absent is appropriate because the other interfaces (http and eventually gRPC) can be used for naming (if the right interpreter is configured).
The docs mention that briefly here https://linkerd.io/config/0.8.5/linkerd/index.html#namerd "The io.l5d.namerd interpreter uses namerd’s long-poll thrift interface and the io.l5d.namerd.http interpreter uses namerd’s HTTP streaming interface."  But I agree that this is pretty deeply buried and not 100% clear.  Suggestions and/or PRs for how to make this more clear are greatly appreciated.

Thanks for your interest in the project!


Dataset Viewer issue for vishnun/NLP-KnowledgeGraph

Link
https://huggingface.co/datasets/vishnun/NLP-KnowledgeGraph
Description
The dataset viewer is not working for dataset vishnun/NLP-KnowledgeGraph.
Error details:
Error code:   ConfigNamesError
Exception:    OSError
Message:      [Errno 116] Stale file handle
Traceback:    The previous step failed, the error is copied to this step:  kind='/config-names' dataset='vishnun/NLP-KnowledgeGraph' config=None split=None---Traceback (most recent call last):
                File "/src/services/worker/src/worker/job_runners/config_names.py", line 99, in compute_config_names_response
                  for config in sorted(get_dataset_config_names(path=dataset, use_auth_token=use_auth_token))
                File "/src/services/worker/.venv/lib/python3.9/site-packages/datasets/inspect.py", line 323, in get_dataset_config_names
                  dataset_module = dataset_module_factory(
                File "/src/services/worker/.venv/lib/python3.9/site-packages/datasets/load.py", line 1215, in dataset_module_factory
                  raise e1 from None
                File "/src/services/worker/.venv/lib/python3.9/site-packages/datasets/load.py", line 1192, in dataset_module_factory
                  return HubDatasetModuleFactoryWithoutScript(
                File "/src/services/worker/.venv/lib/python3.9/site-packages/datasets/load.py", line 825, in get_module
                  dataset_readme_path = cached_path(
                File "/src/services/worker/.venv/lib/python3.9/site-packages/datasets/utils/file_utils.py", line 183, in cached_path
                  output_path = get_from_cache(
                File "/src/services/worker/.venv/lib/python3.9/site-packages/datasets/utils/file_utils.py", line 611, in get_from_cache
                  http_get(
                File "/usr/local/lib/python3.9/tempfile.py", line 496, in __exit__
                  result = self.file.__exit__(exc, value, tb)
              OSError: [Errno 116] Stale file handle

Thanks for reporting, @MangoFF.
We are having issues with the datasets-serve.
We'll keep you informed once the issues are fixed.

The issues with the datasets-server were fixed and I have forced the refresh of your viewer.
It works as expected now:


fix inferrability of cpu shmem arrays

fixes #83

@jkozdon Is this also a issue on GPU code?

I didn't see a performance degradation on the GPU, so I assume not. But will check....

There does not seem to be any problems in cuda.

If there were, would the gpu code have even run?


feat(HMS-3390): Delete blueprint button

This is a work in progress for now, as there is still a small glitch caused by some error. I did not discover the reason for that yet. But feel free to comment if this is going the right direction. :+1:

Thank you both @lucasgarfield and @amirfefer, for the review, I love it! I addressed your comments and rebased on 63f4059.

I do not think that related images should be deleted (we would have to edit the backend endpoint IMO). I thought we agreed on keeping related images.

You're  probably right, I just wanted to double check, the images anyway stored  just temporally


Show gomaxprocs on status page

It'd be helpful to have access to golang runtime information somewhere.

What other Go runtime information would be relevant, other than gomaxprocs?

GOGC I guess.

Prometheus stats like ingestion rate, chunk counts and time series would be useful.

I noone is working on it, can i take it?

Sure.

@brian-brazil What's best way to access stats like chunk and time series counts?
Some of them are available as a metric in DefaultGatherer, but to access them we would need to search them by name.
Ingestion rate would need to be calculated, but based on which metric?

You'd need to add an API to get them from the TSDB. Ingestion rate will be a bit tricky, as that changes over time.

@brian-brazil Is this ok? https://github.com/serathius/prometheus/commit/0873cc83482ac15a99bdf9e83e911715e818134e
To config reload and wal corruptions described https://github.com/prometheus/prometheus/issues/1481 I would need to add Status to WAL interface and also pass it from main.go

It feels dirty to use metrics like that.

I agree. Do you know other less hacky method?

Maybe track it more directly in the TSDB?

Yes we could track metric value in separate variable and always update it when metric is updated. It creates second source of truth, and would be hard to avoid breaking it in future.
Dirty part can be hidden in function ToFloat64 implemented like in https://github.com/prometheus/client_golang/issues/58

I added other runtime information to status page using ToFloat64 here https://github.com/serathius/prometheus/commit/a8018eb0070c77989b255ee93cb9055f22ed66e8#diff-3acf7247bfcbc5bf815ae1f421d34342R24

@beorn7 What's your opinion on using metrics as shown in the above code example for use elsewhere?

It's tempting to use metrics for that. However, one of the reasons for not providing "Getters" on metric types is to avoid encouraging exactly this usage pattern.
My take so far was the following: If I have a need to track some value for use outside of metrics collection by Prometheus, I track that value completely outside of the Prometheus metrics library code. I then use https://godoc.org/github.com/prometheus/client_golang/prometheus#NewGaugeFunc or https://godoc.org/github.com/prometheus/client_golang/prometheus#NewCounterFunc to funnel that separately tracked value into the Prometheus metrics collection. Let's say (imaginary example!) I need to track the number of chunks because I want to throttle ingestion if it passes a certain threshold. I should not funnel that value through the Prometheus library. This would just be needlessly complex including a roundtrip of converting an int into a float and back. Sure, the Prometheus library conveniently wraps atomic increments and decrements, but that's really one line with the atomic package anyway. Note that this approach avoids the duplicated sources of truth mentioned in https://github.com/prometheus/prometheus/issues/1468#issuecomment-371069294 as the one source of truth is the separately tracked value, which is merely mirrored into the Prometheus universe at collection time with the GaugeFunc or CounterFunc.
The use case for this issue is perhaps slightly different as you don't need those numbers for some internal use but to display a status page, which – arguably – is a curated and nicer looking human-readable version of /metrics. However, if we really want to go down that path of considering all those status information “just metrics”, we should not pollute the interface of the tsdb.DB by adding a Status method. We should realize that tsdb.DB already has a way to expose metrics: https://godoc.org/github.com/prometheus/tsdb#Open takes a prometheus.Registerer, and that's how metrics are communicated by the tsdb.DB. That means, we can call the Gather method of the registry we have passed in as a Registerer whenever we render the status page, and then populate status.ConfigStatus from the resulting []*dto.MetricFamily slice. That will be a bit fiddly at this one code location (thanks to the clunky protobuf representation in Go), but at least you would not need to propagate code changes all the way down to the tsdb package.

That makes sense. There'll be other non-TSDB metrics we want (and also calculating some rates) so working from the default registry is probably what makes sense here.

Using Gather was my first idea, but I skipped it because it's more complicated and also relies on reversing logic of construction metric name. So still it will need to have knowledge about name and namespace of metric and rely on it not changing. In the worst case something in chain changes the resulting name, and we have empty status page.
But if it does not pose a problem, we can go with it.

There's no namespaces in the proto, it's the same structure and data as what you're working with via WriteTo.

Note that the Gather method returns MetricFamilies, while the Write writes into a dto.Metric. There is a longer history and (partially flawed) rationale behind this distinction, but in any case, what you get with Gather has all the names in it.


fix(share/getters): short-circuit on empty root in ShrexGetter

Should fix issue where GetEDS hangs on empty data root
To test run:
celestia share get-eds “$(celestia header get-by-height 500 --node.store ~/.celestia-light-mocha-4 | jq .result)” --node.store ~/.celestia-light-mocha-4
with and without the PR

So I tested this on LN, and I never triggered hitting. I found though a weird behaviour. It's guaranteed that LN gets an empty block on the first time with slight delay, but it always stuck on the further tries.

Note, we need to add empty block test case to our Getter test suite

Hmm, a swamp test of a LN calling twice resolves both times and logs hitting - I wonder what is different between swamp and the real call. Are you sure the node was running the right build?

I'lol double check tmrw @distractedm1nd

It is logging hit for me every time, and doesn't hang ever even on multiple runs 🤔
But on main, it retrieves after the first time, logging
2023-10-16T14:05:26.436+0200	ERROR	header/p2p	p2p/exchange.go:166	verifying head received from tracked peer	{"tracked peer": "12D3KooWS7CW7y1VRuHif4nJqLyUyUtHaCFmJDEfTd9wMpgFucNk", "height": 290816, "err": "header verification failed: known header: '290816' <= current '290816'"}
2023-10-16T14:05:26.437+0200	ERROR	header/p2p	p2p/exchange.go:166	verifying head received from tracked peer	{"tracked peer": "12D3KooWHHUVwvXgdxg4k2GtfSA6yB6iDJ1fBYnvBU9LhSMzoi2A", "height": 290816, "err": "header verification failed: known header: '290816' <= current '290816'"}
2023-10-16T14:05:26.842+0200	ERROR	header/p2p	p2p/exchange.go:166	verifying head received from tracked peer	{"tracked peer": "12D3KooWQNp5N12C19cDvhDSAVLnW6tYjthZs1abi3mpXpSqwkpJ", "height": 290816, "err": "header verification failed: known header: '290816' <= current '290816'"}
2023-10-16T14:05:26.853+0200	ERROR	header/p2p	p2p/exchange.go:166	verifying head received from tracked peer	{"tracked peer": "12D3KooWHEU9efXKk5PeCyWyNVz2YYwW8bzxv25vjtfiJN6fLpTE", "height": 290816, "err": "header verification failed: known header: '290816' <= current '290816'"}

And then fails the second time.
So confirmed that this is fixing the issue

Codecov Report

Merging #2846 (617f823) into main (a52e2de) will increase coverage by 0.15%.
Report is 5 commits behind head on main.
The diff coverage is 25.00%.

@@            Coverage Diff             @@
##             main    #2846      +/-   ##
==========================================
+ Coverage   51.30%   51.46%   +0.15%
==========================================
  Files         168      167       -1
  Lines       10744    10773      +29
==========================================
+ Hits         5512     5544      +32
+ Misses       4750     4727      -23
- Partials      482      502      +20

Files
Coverage Δ

share/getters/shrex.go
65.55% <25.00%> (-0.93%)
:arrow_down:

... and 16 files with indirect coverage changes


Query param causing asyncData and fetch to be called on parent component

Version
v2.11.0
Reproduction link
https://codesandbox.io/s/stoic-tdd-ysdwi?fontsize=14&hidenavigation=1&theme=dark
Steps to reproduce
If you are watching the console tab and click around the players it works fine.
As soon as you press the addQuery button, then click on a player, the asyncData and fetch methods of the parent component are hit
What is expected ?
The asyncData and fetch methods should not be hit
What is actually happening?
AsyncData and fetch are hit
Additional comments?
Are there any workarounds to prevent this?

This bug report is available on Nuxt community (#c10352)

Possibly related? https://github.com/nuxt/nuxt.js/issues/6998

It still seems to be an issue:
https://codesandbox.io/s/stoic-tdd-ysdwi?fontsize=14&hidenavigation=1&theme=dark


update sidebar, translate welcome and overview pages

Errors in CONTRIBUTING.md

[404] https://rooch.network/learn/introduction | Failed: Network error: Not Found


[Refactor Flask II] Break up top-level

Rebase and splinter of #796 specific to offloading various endpoints from the main app.py into distinct namespaces

Well, for reasons that are currently beyond me, our attempt last week to simplify the serving of paths to neurosift has been causing issues so I reverted to the previous structure and we can tackle it in a follow-up
If I go for the flask_restx.Resource structure inside a separate namespace (and adjust all fetches to that new namespace accordingly) I get

whereas it should normally look like


Does AutoFixture support .NET Core 3.1 and .NET 5?

Autofixture has .NET platforms compatibility table
for
.NET Framework | .NET Standard
What about .netcore 3.1 and .net 5.0?
AutoMoqDataAttribute does not work there!!

@rathoregee this question has already been answered, on several occasions here and on SO.
The short answer is, AutoFixture is compatible with .NET Core, .NET 5 and should remain compatible with future versions.
If you have encountered a bug in one of the libraries please raise an issue and provide a minimum reproducible example.
Closing the issue.

@rathoregee I use regularly AutoMoq in a project running on .NET 5 (and testing .NET 6) and I don't have any issue. As a quick suggestion, make sure to also have a direct reference to Moq so that you are sure you're importing the latest version.

For example i cannot use like this
public class AutoMoqDataAttribute : AutoDataAttribute
{
public AutoMoqDataAttribute()
: base(new Fixture()
.Customize(new AutoMoqCustomization()))
{
}
}
It ask to use add         [System.Obsolete]
using AutoFixture;
using AutoFixture.AutoMoq;
using AutoFixture.Xunit2;
namespace PdfUploder.Tests.UnitTests
{
public class AutoMoqDataAttribute : AutoDataAttribute
{
[System.Obsolete]
public AutoMoqDataAttribute()
: base(new Fixture()
.Customize(new AutoMoqCustomization()))
{
}
}
}

@rathoregee it tells you the constructor is obsolete because that particular overload of the constructor is obsolete, and you should probably use another constructor overload. This has nothing to do with the target .NET framework version.
The recommended constructor overload is AutoMoqCustomization(Func<IFixture> factory). This means your custom AutoData attribute should look like this:
using AutoFixture;
using AutoFixture.AutoMoq;
using AutoFixture.Xunit2;

namespace PdfUploder.Tests.UnitTests
{
    public class AutoMoqDataAttribute : AutoDataAttribute
    {
        public AutoMoqDataAttribute()
            : base(() => new Fixture()
                .Customize(new AutoMoqCustomization()))
        {
        }
    }
}

OK... now i am using like this
using AutoFixture;
using AutoFixture.AutoMoq;
using AutoFixture.Xunit2;
using System.Linq;
namespace PdfUploder.Tests.UnitTests
{
public class AutoMoqDataAttribute : AutoDataAttribute
{
public AutoMoqDataAttribute() : base(() =>
{
var fixture = new Fixture().Customize(new CompositeCustomization(
new AutoMoqCustomization(),
new SupportMutableValueTypesCustomization()));
        fixture.Behaviors.OfType<ThrowingRecursionBehavior>().ToList().ForEach(b => fixture.Behaviors.Remove(b));
        fixture.Behaviors.Add(new OmitOnRecursionBehavior());

        return fixture;
    })
    {
    }
}

}
but IFormFile all propeties are coming NULL what the point ?

string ContentDisposition { get; }
string ContentType { get; }
string FileName { get; }
My understadnig it should provide an prepared object
when you use the attr..
like
[Theory(DisplayName = "Valid pdf upload casev0 "), AutoMoqData]
can you provide any working code or example to solve IFormFile issue or to create a object from it. thanks for help so far

@rathoregee behind the covers IFormFile is generated as a mock, and by default Moq does not generate any values for the members.
If you want AutoFixture to generate the member values you should configure the AutoMoqCustomization to generate member data for the members by setting the property ConfigureMembers to true.
new AutoMoqCustomization { ConfigureMembers = true  }


How to update all models

Do I have tun run ollama pull <model name> for each model downloaded? Is there a more automatic way to update all models at once?

That'll be a nice feature, but as it stands now, shouldn't be hard to do yourself with a single command like below
ollama list | awk 'NR>1 {print $1}' | xargs -I {} sh -c 'echo "Updating model: {}"; ollama pull {}; echo "--"' && echo "All models updated."

Here's a little PowerShell script to do the same thing, if you're on Windows or have it installed on your OS. Note that you may need to update the URI if you're hosting on a different port/server (I personally am using an NGINX reverse proxy)
(Invoke-RestMethod http://localhost:11434/api/tags).Models.Name.ForEach{ ollama pull $_ }
To perform a dry-run of the command, simply add quotes around "ollama pull $_" to print the command to the terminal instead of executing it.

Hi @misaligar, it looks like this issue is quite similar to what's been reported in #2586. We understand the current workaround isn't ideal, but please know we're actively seeking a more effective solution. We'll keep the community updated as soon as we make progress that improves the situation. For now, I'll close this issue, but we're aware of this issue and working to resolve this.

hope this will be merged soon:
https://github.com/ollama/ollama/pull/2179

I have a little zsh alias that adds this functionality you might find useful:
# wrap the ollama command, if the parameter is pull with no other parameters pull all models
function ollama() {
  if [[ $1 == "pull" ]] && [[ $# -eq 1 ]]; then
    echo "pulling all models..."
    ollama list | awk '$1 !~ /^registry.local/ {print $1}' | while read -r model; do
      echo "Pulling $model"
      ollama pull "$model"
    done
  else
    command ollama "$@"
  fi
}

On Windows, PowerShell users can use this:
$models = ollama list | Select-Object -Skip 1 | ForEach-Object { $_.Split()[0] } foreach ($model in $models) {     Write-Output "Updating model: $model"     ollama pull $model     Write-Output "--" } Write-Output "All models updated."

On Windows, PowerShell users can use this:
$models = ollama list | Select-Object -Skip 1 | ForEach-Object { $_.Split()[0] }
foreach ($model in $models) {
    Write-Output "Updating model: $model"
    ollama pull $model
    Write-Output "--"
}
Write-Output "All models updated."

Thank you! That didn't work for me and I came up with the following:
function updateModels {
    $models = ollama list | Select-Object -Skip 1 | ForEach-Object {
        if ($_ -notmatch '^NAME\s+') {
            if ($_ -match '(\S+)\s+') {
                $Matches[1]
            }
        }
    }

    if ($models.Count -eq 0) {
        Write-Output "No models found to update."
        return
    }

    foreach ($model in $models) {
        Write-Output "Updating model: $model"
        ollama pull $model
        Write-Output "--"
    }
    Write-Output "All models updated."
}


fix: onAfterBuild hook async not work

Summary
Fix onAfterBuild hook async not work
Related Links
https://github.com/web-infra-dev/rspack/pull/6058
Checklist

[x] Tests updated (or not required).
[ ] Documentation updated (or not required).

!canary

!canary


How to do global segmentation？

Thank you for your model.！I would like to know how your model performs global segmentation. Can the author open source the inference code for global segmentation in the future?

Thank you for your model.！I would like to know how your model performs global segmentation. Can the author open source the inference code for global segmentation in the future?

Hi,  @xxk123456
Have you implemented this feature yet? I have the same requirement.

@BIT-DYN
Hi，I also have this requirement，have you achieve it？


os-wireguard: wireguard-go uses /dev/tunX,  new openvpn-server config fails

[✓] I have read the contributing guide lines at https://github.com/opnsense/plugins/blob/master/CONTRIBUTING.md
[✓] I have searched the existing issues, open and closed, and I'm convinced that mine is new.
[✓] The title contains the plugin to which this issue belongs
11 openvpn-server Instances and 2 wireguard instances are running on the Firewall. When i create a new openvpn-server instance, this instance is unreachable.
To Reproduce:
1.) Install OPNsense and install os-wireguard
2.) under VPN:OpenVPN:Servers 11 instances
3.) under VPN:WireGuard create and enable 2 local instances
4.) reboot OPNsense
5.) create a new VPN:OpenVPN:Servers instance
6.) In Webgui the new instance themes find, but openvpn-server ist unreachable
Logs while creating openvpn-instance 12:

<11>1 2022-10-24T11:07:06+02:00 rtr.lan php 60786 - [meta sequenceId="1"] /vpn_openvpn_server.php: The command '/sbin/ifconfig 'tun12' name 'ovpns12'' returned exit code '1', the output was 'ifconfig: interface tun12 does not exist'
<11>1 2022-10-24T11:07:06+02:00 rtr.lan php 60786 - [meta sequenceId="2"] /vpn_openvpn_server.php: The command '/sbin/ifconfig 'ovpns12' group openvpn' returned exit code '1', the output was 'ifconfig: interface ovpns12 does not exist'
<11>1 2022-10-24T11:07:06+02:00 rtr.lan php 60786 - [meta sequenceId="3"] /vpn_openvpn_server.php: The command '/sbin/route -q delete <IP_ADDRESS>' returned exit code '1', the output was 'route: route has not been found'
<13>1 2022-10-24T11:07:06+02:00 rtr.lan configctl 38867 - [meta sequenceId="4"] event @<PHONE_NUMBER>.04 msg: Oct 24 11:07:06 rtr.lan config[60786]: [2022-10-24T 11:07:06+02:00][INFO] config-event: new_config /conf/backup/config-1666602425.8506.xml
<13>1 2022-10-24T11:07:06+02:00 rtr.lan configctl 38867 - [meta sequenceId="5"] event @<PHONE_NUMBER>.04 exec: system event config_changed
<11>1 2022-10-24T11:07:16+02:00 rtr.lan php 60786 - [meta sequenceId="6"] /vpn_openvpn_server.php: OpenVPN server 12 instance start timed out.
<11>1 2022-10-24T11:07:17+02:00 rtr.lan opnsense 14122 - [meta sequenceId="7"] /usr/local/etc/rc.filter_configure: ROUTING: keeping current default gateway '<IP_ADDRESS>'

Expected behavior:
A working openvpn-server instance
Problem seems to be a already used /dev/tun device. openvpn tries to allocate the next free /dev/tun12 device:

root@rtr:~ # head /var/etc/openvpn/server12.conf
dev ovpns12
verb 1
dev-type tun
dev-node /dev/tun12
writepid /var/run/openvpn_server12.pid

This device is already in use by wireguard-go:

root@rtr:~ # lsof /dev/tun12
COMMAND     PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
wireguard 80096 root    3u  VCHR  0,137      0t0  137 /dev/tun12

just to verify:

root@rtr:~ # ./lsof -p 80096
COMMAND     PID USER   FD     TYPE             DEVICE SIZE/OFF   NODE NAME
wireguard 80096 root  txt     VREG               0,99  2523136   5132 /usr/local/bin/wireguard-go (/dev/ufs/OPNsense_Nano)
wireguard 80096 root  cwd     VDIR               0,99      512      2 / (/dev/ufs/OPNsense_Nano)
wireguard 80096 root  rtd     VDIR               0,99      512      2 / (/dev/ufs/OPNsense_Nano)
wireguard 80096 root    0r    VCHR               0,27      0t0     27 /dev/null (devfs)
wireguard 80096 root    1r    VCHR               0,27      0t0     27 /dev/null (devfs)
wireguard 80096 root    2r    VCHR               0,27      0t0     27 /dev/null (devfs)
wireguard 80096 root    3u    VCHR              0,137      0t0    137 /dev/tun12 (devfs)
wireguard 80096 root    5u  KQUEUE 0xfffff800587d7200                 count=0, state=0x2
wireguard 80096 root    6u    PIPE 0xfffff80058618000    16384        ->0xfffff80058618160
wireguard 80096 root    7u    PIPE 0xfffff80058618160     4096        ->0xfffff80058618000
wireguard 80096 root    8u     rte                         0t0        no protocol control block
wireguard 80096 root    9u    unix 0xfffff8001e2a1b10      0t0        /var/run/wireguard/wg2.sock
wireguard 80096 root   10u     rte                         0t0        no protocol control block
wireguard 80096 root   11u  KQUEUE 0xfffff800587d7300                 count=0, state=0x2
wireguard 80096 root   12r    VDIR               0,99      512 507483 /var/run/wireguard (/dev/ufs/OPNsense_Nano)
wireguard 80096 root   13u    IPv4 0xfffff8001e659d90      0t0    UDP *:47001
wireguard 80096 root   14u    IPv6 0xfffff8001e0c69b0      0t0    UDP *:47001

try to start on cli:

root@rtr:~ # /usr/local/sbin/openvpn --config /var/etc/openvpn/server12.conf
2022-10-24 13:12:48 Cipher negotiation is disabled since neither P2MP client nor server mode is enabled
2022-10-24 13:12:48 OpenVPN 2.5.7 amd64-portbld-freebsd13.1 [SSL (OpenSSL)] [LZO] [LZ4] [MH/RECVDA] [AEAD] built on Jul 25 2022
2022-10-24 13:12:48 library versions: OpenSSL 1.1.1q  5 Jul 2022, LZO 2.10
2022-10-24 13:12:48 NOTE: the current --script-security setting may allow this configuration to call user-defined scripts
2022-10-24 13:12:48 Cannot open TUN/TAP dev /dev/tun12: Device busy (errno=16)
2022-10-24 13:12:48 Exiting due to fatal error

I'm new to OPNsense, so I don't know the boot process in detail. Just to guess what happens:

openvpn 1-11 starts and use /dev/tun1 to /dev/tun11
wireguard-go use first /dev/tun0
the next wireguard-go instance use /dev/tun12
openvpn don't know anything about the blocked /dev/tun12 device and tries to use it.

Environment
OPNsense 22.7.6 (amd64, OpenSSL).
AMD GX-412TC SOC
os-wireguard 1.12

Due to both OpenVPN and WireGuard using "tun" interfaces the implementation limitation is that /dev/tunX nodes are static so more than one service using "tun" will eventually break. You can perhaps use wireguard-kmod and sidestep this. Otherwise there isn't anything we intend to do about it.
Cheers,
Franco

Did a test on two other OPNsense firewalls. wireguard-kmod works very well. No tun conflicts any more. Thank you!


Trouble installing

I have trouble installing this addon.
1) Go to AssetLib tab and search for GodotXUnit and Install project. Don't enable yet.

Done.
2) Include these scripts to your mono project csproj:
- addons/GodotXUnit/GodotTestRunner.cs
- addons/GodotXUnit/Plugin.cs
- addons/GodotXUnit/XUnitDock.cs

Done.
- execute this command on the cli at the root of your project:
3) nuget restore ./addons/GodotXUnit/GodotXUnitApi/GodotXUnitApi.csproj

Done
4) Add this reference to your main project:
./addons/GodotXUnit/GodotXUnitApi/GodotXUnitApi.csproj

Couldn't do this from Rider's GUI, I had to manually edit my project's .csproj:
  <ItemGroup>
    <ProjectReference Include="addons/GodotXUnit/GodotXUnitApi/GodotXUnitApi.csproj" />
  </ItemGroup>

Which then results in the error when trying to build:
Metadata file 'addons/GodotXUnit/GodotXUnitApi/bin/Debug/GodotXUnitApi.dll' could not be found

Help me, fledware, you're my only hope.

That likely means that the GodotXUnitApi project isn't being built. have you tried just building the GodotXUnitApi project? if that doesnt get you anywhere, can you paste the main .csproj?
also note, this plugin hasn't been tested with godot 3.2.3.

I'm on .NET Standard 2.1 and Godot 3.2.3.
I've been playing around with GodotXUnit for couple of days now, but there's just no way I can get it running. I guess I'll just lay back for a while and wait until you're ready to update it to .NET Standard.

I've tested it with the different .Net backends and it works. The issue is with the new godot build for c# in general. I can't get any multi csproj solutions to build with Godot.NET.Sdk/3.2.3.
a good work around would be to just entirely remove the addons/GodotXUnit/GodotXUnitApi/GodotXUnitApi.csproj file and project reference and just include all of the cs files into the root project. a working example can be found here: https://github.com/fledware/GodotXUnit/tree/hack-example
couple things to note:

there is only one csproj
targeting unit test projects will likely not work
if you do not follow the note in this file, GodotXUnit will not be able to find tests with the GodotFact attribute: addons/GodotXUnit/GodotXUnitApi/GodotFactAttribute.cs

@Pilvinen did this work out for you?

No. I spent two days on playing around with it trying out all kinds of things, but eventually just gave up.
I think the instructions could be more detailed and specific. But I'm not sure if that would have helped me because I'm on Godot 3.2.3, .NET Standard 2.1, and C# 8.0.

it is working with godot 3.2.3 in the branch i showed.
I'm going to close this and do a writeup in the readme on the new godot build issue.


Add 'content_type' & 'charset' properties to aiohttp.ClientResponse

What do these changes do?
Extends aiohttp.ClientResponse with content_type and charset properties like in aiohttp.web.Request.
Are there changes in behavior for the user?
Backwards compatible, nothing old was changed, added two new properties.
Related issue number
#1349
Checklist

[x] I think the code is well written
[x] Unit tests for the changes exist
[x] Documentation reflects the changes
[x] Add yourself to CONTRIBUTORS.txt

The format is <Name> <Surname>.
Please keep alphabetical order, the file is sorted by names.

[x] Add a new entry to CHANGES.rst

Choose any open position to avoid merge conflicts with other PRs.
Add a link to the issue you are fixing (if any) using #issue_number format at the end of changelog message. Use Pull Request number if there are no issues for PR or PR covers the issue only partially.

Current coverage is 98.81% (diff: 100%)

Merging #1352 into master will increase coverage by <.01%

@@             master      #1352   diff @@
==========================================
  Files            29         29
  Lines          6760       6774    +14
  Methods           0          0
  Messages          0          0
  Branches       1117       1119     +2
==========================================
+ Hits           6680       6694    +14
  Misses           36         36
  Partials         44         44

Powered by Codecov. Last update 71a22e6...52bd2f0

Just move HeadersMixin into helpers.py and inherit ClientResponse from it.

Implemented.

Thanks!


Questions on the split files

Hi,
Some datasets like 3RScan, ARKitScenes, HM3D, and MultiScan consist of multiple split files, each denoting different subsets. Specifically, they have three types of split files: {train/val}_split_before_filter.txt, {train/val}_split_non_overlap.txt, and {train/val}_split.txt. Could you clarify the distinctions among these split files?
Furthermore, in ProcThor, there are {train/val}_split_all.txt as well as {train/val}_split.txt. What is the difference between these files?
Thanks!

Hi,
Sorry for the confusion. The {train/val}_split_non_overlap.txt is meant for the per-dataset experiment in the early stage of the project. Later we merge them to create a larger training set, i.e., train_split_before_filter.txt. In our final experiments, we use {train/val}_split.txt for all the datasets by filtering out 'problematic' scenes, e.g., too large, too small, or multi-floor. In short words,

Use {train/val}_split.txt to reproduce the experiments in the paper. The val.txt for MultiScan is the SceneVerse-val. There is no val.txt for ARKitScenes, HM3D, 3RScan because we are not validating on them.
The train_split_before_filter.txt = train_split_non_overlap.txt + train_split_non_overlap.txt.
train.txt + filter_out scenes = train_split_before_filter.txt

We may clear out the unnecessary files if people find them confusing in a later version.
Best,

Thank you for the reply! It resolved most of my confusion except train_split_all.txt. What is the difference between train_split_all.txt and train_split.txt in ProcThor?

Yes, you are right. And I have updated the previous comment accordingly.

I see. Thank you very much for kind reply :)


NetworkManager issues when configuring CRC

I'm following the instructions to install CodeReady Containers (https://ibm.github.io/spm-kubernetes/prereq/openshift/codeready-containers/#creating-a-crc-project).  I'm using an RHEL VM, and when I attempt the "crc start" command I get the following error regarding NetworkManager:
[vnc@bluffing1 ~]$ crc start
INFO Checking if running as non-root
INFO Checking if podman remote executable is cached
INFO Checking if admin-helper executable is cached
INFO Checking minimum RAM requirements
INFO Checking if Virtualization is enabled
INFO Checking if KVM is enabled
INFO Checking if libvirt is installed
INFO Checking if user is part of libvirt group
INFO Checking if libvirt daemon is running
INFO Checking if a supported libvirt version is installed
INFO Checking if crc-driver-libvirt is installed
INFO Checking if systemd-networkd is running
INFO Checking if NetworkManager is installed
INFO Checking if NetworkManager service is running
INFO Checking if /etc/NetworkManager/conf.d/crc-nm-dnsmasq.conf exists
File not found: /etc/NetworkManager/conf.d/crc-nm-dnsmasq.conf: stat /etc/NetworkManager/conf.d/crc-nm-dnsmasq.conf: no such file or directory

Can someone advise how /etc/NetworkManager/conf.d/crc-nm-dnsmasq.conf file should be configured?
There is some info here but I assume that is just an example? Not sure what values I should put

For the record, I tried both the values shown from the RedHat Troubleshooting page as well as my own IP address, but in either case I get File has unexpected content: /etc/NetworkManager/conf.d/crc-nm-dnsmasq.conf error.
It looks like this:
server=/crc.testing/<IP_ADDRESS>
server=/apps-crc.testing/<IP_ADDRESS>

For the record, I tried both the values shown from the RedHat Troubleshooting page as well as my own IP address, but in either case I get File has unexpected content: /etc/NetworkManager/conf.d/crc-nm-dnsmasq.conf error.
It looks like this:
server=/crc.testing/<IP_ADDRESS>
server=/apps-crc.testing/<IP_ADDRESS>

@kdombros Did you run crc setup? That command was supposed to configure the missing NetworkManager files.

@kdombros Did you run crc setup? That command was supposed to configure the missing NetworkManager files.

I did run that command - but now I think I understand what went wrong:  after that step, I discovered the NetworkManager was disabled, so I think the set up was incomplete. I did a crc delete and recreated and now that problem is fixed. That's the Good news!
Now the bad news: I'm getting this error from crc start:
Error creating machine: Error creating the VM: Error creating machine: Error in driver during machine creation: host is not running: getting ip during machine start
LMK if you can offer advise on how to figure out what is wrong... I'll take a Google of it now

I did run that command - but now I think I understand what went wrong:  after that step, I discovered the NetworkManager was disabled, so I think the set up was incomplete. I did a crc delete and recreated and now that problem is fixed. That's the Good news!
Now the bad news: I'm getting this error from crc start:
Error creating machine: Error creating the VM: Error creating machine: Error in driver during machine creation: host is not running: getting ip during machine start
LMK if you can offer advise on how to figure out what is wrong... I'll take a Google of it now

Just a hunch, but could you run lscpu and check the Vendor ID and search the flags at the bottom of the output for vmx.
Only the hosts with Intel CPUs in that environment have support for nested virtualisation enabled.
If your VM is running on one of the AMD nodes, you will have to reprovision in SVL.
If you'd like a shortcut, one of the other teams published an Ansible playbook that will do the whole setup for you, although with an older version of CRC (according to the ReadMe).

Just a hunch, but could you run lscpu and check the Vendor ID and search the flags at the bottom of the output for vmx.
Only the hosts with Intel CPUs in that environment have support for nested virtualisation enabled.
If your VM is running on one of the AMD nodes, you will have to reprovision in SVL.
If you'd like a shortcut, one of the other teams published an Ansible playbook that will do the whole setup for you, although with an older version of CRC (according to the ReadMe).

Yea I always provision at SVL so looks like I'm good on that hunch:
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                4
On-line CPU(s) list:   0-3
Thread(s) per core:    1
Core(s) per socket:    1
Socket(s):             4
NUMA node(s):          1
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 61
Model name:            Intel Core Processor (Broadwell, IBRS)
Stepping:              2
CPU MHz:               2300.000
BogoMIPS:              4600.00
Virtualization:        VT-x
Hypervisor vendor:     KVM
Virtualization type:   full
L1d cache:             32K
L1i cache:             32K
L2 cache:              4096K
NUMA node0 CPU(s):     0-3
Flags:                 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl eagerfpu pni pclmulqdq vmx ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single ssbd ibrs ibpb tpr_shadow vnmi flexpriority ept vpid fsgsbase bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx avx512f rdseed adx smap avx512cd xsaveopt arat md_clear spec_ctrl

Yea I always provision at SVL so looks like I'm good on that hunch:
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                4
On-line CPU(s) list:   0-3
Thread(s) per core:    1
Core(s) per socket:    1
Socket(s):             4
NUMA node(s):          1
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 61
Model name:            Intel Core Processor (Broadwell, IBRS)
Stepping:              2
CPU MHz:               2300.000
BogoMIPS:              4600.00
Virtualization:        VT-x
Hypervisor vendor:     KVM
Virtualization type:   full
L1d cache:             32K
L1i cache:             32K
L2 cache:              4096K
NUMA node0 CPU(s):     0-3
Flags:                 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl eagerfpu pni pclmulqdq vmx ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single ssbd ibrs ibpb tpr_shadow vnmi flexpriority ept vpid fsgsbase bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx avx512f rdseed adx smap avx512cd xsaveopt arat md_clear spec_ctrl

This looks like the same error as reported in https://github.com/code-ready/crc/issues/1213
Running CRC in a VM is not officially supported, though in that case upgrading to a later version of RHEL helped.

This looks like the same error as reported in https://github.com/code-ready/crc/issues/1213
Running CRC in a VM is not officially supported, though in that case upgrading to a later version of RHEL helped.

Hey Andrey - I think you determined the Hypervisor on my RHEL VM is based on Ubuntu?  So one way or another I'm trying to  use CRC on an unsupported platform right? We should close this issue off / I'll re-visit CRC on a cleaner VM, etc.

Indeed - the issue here appears to be an older hypervisor, which is not something that we can control.
I provisioned 4 new VMs (RHEL 7.8/7.9/8.3, and Ubuntu 20.04) on the same platform and was able to setup CRC in all cases, with the core difference being newer hosts.
Closing.


docker build failed

docker build -t azure_opc_pub https://github.com/Azure/iot-edge-opc-publisher
Sending build context to Docker daemon  145.4kBs://github.com/Azure/iot-edge-opc-publisher  143.8kB
Error response from daemon: Dockerfile parse error line 7: unknown instruction: <!DOCTYPE

Use: docker build -t azure_opc_pub https://github.com/Azure/iot-edge-opc-publisher.git


Upgrade to latest imgui?

An upgrade to latest imgui would be great. Currently pyimgui is using 1.65, which was released in Sep 2018. The latest release is currently 1.76

Yes! The newer versions have tabs which are much needed.

I actually forked this project to get the latest Docking branch. It would be great to have it here, so I don't have to maintain the fork. The ImGui upgrade was painless though.

Will take this up right after the 1.2.0 release and it is currently building.

I did update to imgui 1.77 on separate imgui-upstream-v1.77-branch branch that can be previewed in #162 PR

@swistakm One other possible reason to update to 1.77 is that it will make wrapping, for instance, implot possible.

Is it possible to integrate IMGUI docking branch?
IMGUI maintains a separate branch(docking) which has a feature for auto layout and docking just like Visual Studio.

Hello @swistakm,
Would it be possible to have the binaries on the branch so that we can pip install -U<EMAIL_ADDRESS>without downloading the 5Go of Microsoft Visual C++ Build Tools ?
Pretty please :}

branch is working well so far
imgui.INPUT_TEXT_READ_ONLY is now imgui.INPUT_TEXTREAD_ONLY - so that makes it non backwards compatible. Is this a bug or intentional, are there other non backwards compatible changes I should be aware of?

I'm also seeing this quite often:
  File "imgui/core.pyx", line 1865, in imgui.core.new_frame
imgui.core.ImGuiError: ImGui assertion error ((g.IO.DeltaTime > 0.0f || g.FrameCount == 0) && "Need a positive DeltaTime!") at imgui-cpp/imgui.cpp:6736

looks to be an imgui problem, so maybe 1.77 isn't the best release to pick or the submodule reference needs to be updated?

INPUT_TEXT_ENTER_RETURNS_TRUE has also become INPUT_TEXTENTER_RETURNS_TRUE so pretty sure there's a bug there

Moved to work off of https://github.com/swistakm/pyimgui/pull/192 - INPUT_TEXTENTER_RETURNS_TRUE and INPUT_TEXTREAD_ONLY are no longer an issue.


Only show locations from current note in embedded map

Is there a way to write a query (maybe using path) that makes an embedded map show only the locations from the note it is embedded in?
90% of the time I simply want to add a map at the end of the note showing all inline geo-locations but that means to update the query to match the path of the current note everytime.

See the suggestion here: https://github.com/esm7/obsidian-map-view/issues/140
I believe that together with the Templater plugin, you can easily set up what you're looking for.

Yes, that's certainly a workaround.
For future reference, here is the tenplate which I use for now.
```mapview
{"name":"Default","mapZoom":8,"centerLat":38.58252615935333,"centerLng":-99.83745527819484,"query":"path:\"<% tp.file.path(true) %>\" ","chosenMapSource":0,"embeddedHeight":800}
```

In a future realease it would be nice to have the possibility to write a query like "query":"thisNote" to achieve the same result.
Thanks for helping out and this useful plugin in general!

Thanks for sharing!
Whether Map View should implement this internally is something I'm in a dilemma about.
On one hand, this is surely a useful feature, and having it built into Map View would make it cleaner and in some edge cases also better (e.g. if you copy the snippet to a different note it would work as expected).
On the other hand, every extra new feature adds a bit more stuff to maintain from that point onward, while at the same time there is a plugin that specializes in perfecting 90% of this behavior (and provides a much richer syntax than what I'd probably take upon myself to maintain).
If while working with this workaround you reach useful insights, I'd love to hear.

I understand and have no idea how difficult it is to implement a function like that.
The big issue I see with using templater is that the query is static after the template is inserted.
That means moving or even renaming the note breaks the map. Which is not obvious for inexperienced users and in general the expected/learned behaviour in Obsidian is that editing notes does not break things.
Also I think it's a quite natural and common use case to create a note with a few locations and show them in a map without collecting locations from other, unrelated notes. Personally I even thought that was the default use case for the plugin when I started using it. It was quite cumbersome at the begining to learn how to query the current note.
But again, I understand your reasoning and really don't know how hard it is to maintain.

Would like to see support for this as well, as it seems to a very common use-case. As stated above, if note is renamed, the embedded map will break.

Using dataviewjs this can be achieved as follows...
This particular example expects location of the note to be set in front view to center the map around this location.
let self = dv.current();
let filePath = self.file.path;
let map = mapview 			{"name":"Default","embeddedHeight":400,"mapZoom":13,"centerLat":+self['location'][0]+,"centerLng": + self['location'][1] + ,"query": "path:\\" + filePath + \\"","chosenMapSource":0}\n;
dv.paragraph( '' + map + '');

Added a $filename$ template parameter in the just-released version 4.0.0 :balloon:

A side effect of $filename$ (which makes sense based on your comments in the docs), is that map view seems to interpret the embedded code block as having changed, so the "Save" button is always displayed.

That's a side effect I did not consider, and it sounds like enough of an annoyance to convince me to work on a more comprehensive implementation.


Add web-time

according to https://github.com/rhaiscript/rhai/issues/891, since instant is no longer maintained, we should use web-time to replace it. Actually I have built WASM to make sure it works in it, but it seems that I don't find unit tests for WASM.

It is failing the no-std build... Can you check?
Also I wonder if it is possible to write tests for WASM...

Also I wonder if it is possible to write tests for WASM...

I'm not sure if I can write tests for WASM, but I'll try it. BTW, I have fixed the no-std build.

If we add web-time with optional = true  then we can't pass the cargo build --target wasm32-unknown-unknown --no-default-features compile; Otherwise if we set optional = false, we can't pass cargo build --manifest-path=no_std/no_std_test/Cargo.toml --profile unix  compile. It seems like there isn't perfect way to solve it.

Why is the wasm build failing without default features?

[target.'cfg(not(feature = "no_time"))'.dependencies]
Why did you change it to no_time instead? Before it is pulled in only for wasm which doesnt have Instant.

[target.'cfg(not(feature = "no_time"))'.dependencies]
Why did you change it to no_time instead? Before it is pulled in only for wasm which doesnt have Instant.

You are correct. I have changed it to target.'cfg(all(target_family = "wasm", not(feature = "no_time")))'.dependencies, but it still fails the tests. It appears that when we add web-time as dependencies, it fails the no_std tests. I'm not certain if it's because of the implementation of web-time.

It shoudnt... no-std is not supposed to pull in web-time, then it shouldnt be affected.
If it still fails that means web-time is still being brought in somehow...

I'm pretty sure the issue is related to once_cell and web_time. I've created an example to reproduce the problem. Can run cargo c --features time or cargo c to see the issue. However, it works correctly if I switch from web_time to instant. I'm not sure why this is happening.

Is it possible that web-time does not support no-std?

It's not supported to be pulled if I don't add features time in this, but it still failed.

It's not supported to be pulled if I don't add features time in this, but it still failed.

If only one dependency crate in the entire tree depends on std, then stdlib will be pulled in, causing compilation error in no-std.

It turn out that feature = ... in target.'cfg(...)'.dependencies is not supported for selecting dependencies and will not work as expected. So actually rhai will always pull instant whenever it's wasm or not. But web-time don't support to work in no_std environment, so now we can't use web_time to replace instant.

It turn out that feature = ... in target.'cfg(...)'.dependencies is not supported for selecting dependencies and will not work as expected. So actually rhai will always pull instant whenever it's wasm or not. But web-time don't support to work in no_std environment, so now we can't use web_time to replace instant.

I wonder if setting to resolver = 2 will fix that issue. https://blog.rust-lang.org/2021/03/25/Rust-1.51.0.html#cargos-new-feature-resolver


Disable approve button until PPO is ready for approval

Disable approve button until PPO is ready for approval

Fix the test with proper login check for mobile view then resubmit PR


Cmd-click should open in Safari without switching

I don't know if this is a bug or feature, but Cmd-clicking on an article title should open the Safari window behind NNW. This is a common action in macOS.
Feature: An option for right arrow from an article to do the same as above, i.e., open the article in Safari behind NNW. Has to be an option or accidental arrowing would be frustrating to those who hadn't selected the option.
Safari should of course be taken to mean default browser.
Keep up the good work.

There's an option in the preferences for opening in the background; Command-click should probably invert this setting.

More of an FYI - may be Apple's or Google's problem instead.  I use the "Open in background in browser" preference.  That works fine for Safari and Firefox, but if Chrome is set as my default web browser then command-clicking a link does not open in the background and takes me directly to the link in Chrome.
MacOS: 10.15.5
Chrome: Version 83.0.4103.116 (Official Build) (64-bit)

Chrome bug.

It’s a bug in Chrome, and I don’t know of any way to work around it.

Sounds good, thank you!

This is also the case with Firefox 87.
When I'm using Newsboat in the terminal, I'm able to open articles from my feeds without bringing the application to the foreground with open -ga.
I'm not very familiar with macOS development, so I'm not sure if there is an equivilant in Swift when opening an application from another.

This has been implemented, so I'm closing this one out.


Webhook for Harmony not being saved

I'm able to define a Harmony hub with IP via web interface, but it will not save the an entered webhook URL.

Same here, it just puts blank there

I'm encountering this same defect.  I enter a webhook URL, but it just shows a blank and does not store it in the config file.

I assumee this in 4.5.0?

Yes, 4.5.0, at least for me.

Me too

Add me to that list v4.5.0

I should really finish my implementations......

Is there a work-around by directly editing the .ini file and, if so, what is the format?
Thanks!

Yes, in the habridge.config file after the  name and ip of the harmony, add this ',"webhook":"http://<IP_ADDRESS>/activity/${activity.id}/${activity.label} OR http://hook?a=${activity.label}"'

Thanks. Is there anyway to decrypt config for direct manipulation

Th only part of the config that is encrypted are the security settings and passwords. If you want to write a lot of code to decrypt with the key you give, have at it, but it is encrypted for a reason.....

I just want to batch update the IP of devices but can't see a way