[WIP] Plugin System

This MR adds a plugin system allowing users to more tightly integrate with the hyprland codebase.
TODO:
Expand API:

[x] Native hyprctl-like calls without IPC
[x] Notifications
[x] Function Hooks (Possibly in a further MR)
[x] API methods for decos
[x] API methods for config options

Fix bugs:

[ ] SEGV protection only works once (?)

Docs:

[ ] Wiki MR for new pages (for devs & end users)

Examples:

[x] Add a CMake workflow Makefile for the example plugin

I invite everyone to take a look at the src/plugins/PluginAPI.hpp header and suggest any further API changes / additions.

This looks really useful, especially the ability to create custom layouts - would it be possible to create custom decorations and put them on windows (like dwindle's old group system)?

technically already is, but yeah an API call for API stability would be nice. True.

@fufexan nix/meson ci fails (cuz new submodule)

@Merlin04 custom decoration API added, as a POC the examplePlugin will now draw double borders.

@sp1ritCS would you know how to make Hyprland link with udis86 in Meson? The Arch Meson build fails because the linker can't find udis86.

@fufexan
diff --git a/meson.build b/meson.build
index efc2dd5..c6d5568 100644
--- a/meson.build
+++ b/meson.build
@@ -45,7 +45,7 @@ xcb_dep = dependency('xcb', required: get_option('xwayland'))

 cmake = import('cmake')
 udis = cmake.subproject('udis86')
-udis86 = udis.dependency('udis86')
+udis86 = udis.dependency('libudis86')

 if get_option('xwayland').enabled() and not have_xwlr
 	error('Cannot enable Xwayland in Hyprland: wlroots has been built without Xwayland support')

can meson build the example plugin already?
btw, I kinda fail to understand the use for having udis86 to jump to hooks. This might be dumb, but what's wrong with just storing a function pointers that will be executed when a particular hook fires?

Thanks @sp1ritCS! I was clearly too dumb do read the 3 letters in the project's prefix.

@fufexan we also should solve the problem with hyprland headers.
What I thought is just making an envvar for the plugin example makefile like HYPRLAND_DIR.
You mentioned using pkgconfig
Any thoughts?

For FreeBSD apply b0373a69...e1ca475c. Runtime untested as Clang need more fixes.
examplePlugin errors with Clang
main.cpp:58:61: error: cannot cast from type 'void (CCompositor::*)(CWindow *, wlr_surface *)' to pointer type 'void *'
    g_pFocusHook = HyprlandAPI::createFunctionHook(PHANDLE, (void*)&CCompositor::focusWindow, (void*)&hkFocusWindow);
                                                            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
customDecoration.cpp:36:24: error: type 'double' cannot be narrowed to 'int' in initializer list [-Wc++11-narrowing]
    wlr_box fullBox = {m_vLastWindowPos.x - *PBORDERSIZE, m_vLastWindowPos.y - *PBORDERSIZE, m_vLastWindowSize.x + 2.0 * *PBORDERSIZE, m_vLastWindowSize.y + 2.0 * *PBORDERSIZE};
                       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
customDecoration.cpp:36:24: note: insert an explicit cast to silence this issue
    wlr_box fullBox = {m_vLastWindowPos.x - *PBORDERSIZE, m_vLastWindowPos.y - *PBORDERSIZE, m_vLastWindowSize.x + 2.0 * *PBORDERSIZE, m_vLastWindowSize.y + 2.0 * *PBORDERSIZE};
                       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                       static_cast<int>(                )
customDecoration.cpp:36:59: error: type 'double' cannot be narrowed to 'int' in initializer list [-Wc++11-narrowing]
    wlr_box fullBox = {m_vLastWindowPos.x - *PBORDERSIZE, m_vLastWindowPos.y - *PBORDERSIZE, m_vLastWindowSize.x + 2.0 * *PBORDERSIZE, m_vLastWindowSize.y + 2.0 * *PBORDERSIZE};
                                                          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
customDecoration.cpp:36:59: note: insert an explicit cast to silence this issue
    wlr_box fullBox = {m_vLastWindowPos.x - *PBORDERSIZE, m_vLastWindowPos.y - *PBORDERSIZE, m_vLastWindowSize.x + 2.0 * *PBORDERSIZE, m_vLastWindowSize.y + 2.0 * *PBORDERSIZE};
                                                          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                                                          static_cast<int>(                )
customDecoration.cpp:36:94: error: type 'double' cannot be narrowed to 'int' in initializer list [-Wc++11-narrowing]
    wlr_box fullBox = {m_vLastWindowPos.x - *PBORDERSIZE, m_vLastWindowPos.y - *PBORDERSIZE, m_vLastWindowSize.x + 2.0 * *PBORDERSIZE, m_vLastWindowSize.y + 2.0 * *PBORDERSIZE};
                                                                                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
customDecoration.cpp:36:94: note: insert an explicit cast to silence this issue
    wlr_box fullBox = {m_vLastWindowPos.x - *PBORDERSIZE, m_vLastWindowPos.y - *PBORDERSIZE, m_vLastWindowSize.x + 2.0 * *PBORDERSIZE, m_vLastWindowSize.y + 2.0 * *PBORDERSIZE};
                                                                                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                                                                                static_cast<int>(                       )
customDecoration.cpp:36:136: error: type 'double' cannot be narrowed to 'int' in initializer list [-Wc++11-narrowing]
    wlr_box fullBox = {m_vLastWindowPos.x - *PBORDERSIZE, m_vLastWindowPos.y - *PBORDERSIZE, m_vLastWindowSize.x + 2.0 * *PBORDERSIZE, m_vLastWindowSize.y + 2.0 * *PBORDERSIZE};
                                                                                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
customDecoration.cpp:36:136: note: insert an explicit cast to silence this issue
    wlr_box fullBox = {m_vLastWindowPos.x - *PBORDERSIZE, m_vLastWindowPos.y - *PBORDERSIZE, m_vLastWindowSize.x + 2.0 * *PBORDERSIZE, m_vLastWindowSize.y + 2.0 * *PBORDERSIZE};
                                                                                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                                                                                static_cast<int>(                       )
customDecoration.cpp:70:19: error: type 'double' cannot be narrowed to 'int' in initializer list [-Wc++11-narrowing]
    wlr_box dm = {m_vLastWindowPos.x - m_seExtents.topLeft.x, m_vLastWindowPos.y - m_seExtents.topLeft.y, m_vLastWindowSize.x + m_seExtents.topLeft.x + m_seExtents.bottomRight.x,
                  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
customDecoration.cpp:70:19: note: insert an explicit cast to silence this issue
    wlr_box dm = {m_vLastWindowPos.x - m_seExtents.topLeft.x, m_vLastWindowPos.y - m_seExtents.topLeft.y, m_vLastWindowSize.x + m_seExtents.topLeft.x + m_seExtents.bottomRight.x,
                  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                  static_cast<int>(                         )
customDecoration.cpp:70:63: error: type 'double' cannot be narrowed to 'int' in initializer list [-Wc++11-narrowing]
    wlr_box dm = {m_vLastWindowPos.x - m_seExtents.topLeft.x, m_vLastWindowPos.y - m_seExtents.topLeft.y, m_vLastWindowSize.x + m_seExtents.topLeft.x + m_seExtents.bottomRight.x,
                                                              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
customDecoration.cpp:70:63: note: insert an explicit cast to silence this issue
    wlr_box dm = {m_vLastWindowPos.x - m_seExtents.topLeft.x, m_vLastWindowPos.y - m_seExtents.topLeft.y, m_vLastWindowSize.x + m_seExtents.topLeft.x + m_seExtents.bottomRight.x,
                                                              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                                                              static_cast<int>(                         )
customDecoration.cpp:70:107: error: type 'double' cannot be narrowed to 'int' in initializer list [-Wc++11-narrowing]
    wlr_box dm = {m_vLastWindowPos.x - m_seExtents.topLeft.x, m_vLastWindowPos.y - m_seExtents.topLeft.y, m_vLastWindowSize.x + m_seExtents.topLeft.x + m_seExtents.bottomRight.x,
                                                                                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
customDecoration.cpp:70:107: note: insert an explicit cast to silence this issue
    wlr_box dm = {m_vLastWindowPos.x - m_seExtents.topLeft.x, m_vLastWindowPos.y - m_seExtents.topLeft.y, m_vLastWindowSize.x + m_seExtents.topLeft.x + m_seExtents.bottomRight.x,
                                                                                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                                                                                static_cast<int>(                                                      )
customDecoration.cpp:71:19: error: type 'double' cannot be narrowed to 'int' in initializer list [-Wc++11-narrowing]
                  m_seExtents.topLeft.y};
                  ^~~~~~~~~~~~~~~~~~~~~
customDecoration.cpp:71:19: note: insert an explicit cast to silence this issue
                  m_seExtents.topLeft.y};
                  ^~~~~~~~~~~~~~~~~~~~~
                  static_cast<int>(    )

@jbeich why include_type: system? it's a subproject.

regarding clang errors:
casts, yeah sure, annoying clang.
main.cpp:58:61: error: cannot cast from type 'void (CCompositor::*)(CWindow *, wlr_surface *)' to pointer type 'void *'
    g_pFocusHook = HyprlandAPI::createFunctionHook(PHANDLE, (void*)&CCompositor::focusWindow, (void*)&hkFocusWindow);

tho what? Why is that not allowed excuse me?

why include_type: system? it's a subproject.

2ff0bb6e documents the error in commit message. Hyprland cannot fix errors in subprojects. udis86 is written in C, so it doesn't need to conform to C++ standards.
https://gcc.gnu.org/onlinedocs/cpp/System-Headers.html

That's an easy fix. We do that for wlr already.
#define register
#include <udis86.h>
#undef register

#define register
#include <udis86.h>
#undef register

I confirm, this works fine as an alternative to 2ff0bb6e.

thanks! BSD patches applied.
I have not done anything with the void* cast as it is (IMHO) completely valid, dk why clang complains.

fixado.

This might be dumb, but what's wrong with just storing a function pointers that will be executed when a particular hook fires?

it is, don't worry.

I'd request that you elaborate a bit more on this, given that this way excludes anything that isn't x86_64 (hyprland doesn't build on ia86 for a while now, however everything else [aarch64 in particular] still works).

hyprland doesn't build on ia86 for a while now

Do you mean i386 aka IA-32? If so Hyprland v0.22.0beta builds fine and goes back at least to v0.18.0beta (the first version I've packaged).

hyprland doesn't build on ia86 for a while now

Do you mean i386 aka IA-32? If so Hyprland v0.22.0beta builds fine and goes back at least to v0.18.0beta (the first version I've packaged).

In my particular case it's i586, but yes that is the arch I ment. I've took a look at it and it seems to be the vulkan backend of wlroots that fails building, not hyprland directly.
Given that there isn't any configuration or wlroots in the log, am I assuming correctly, that you patch hyprland to link against wlroots provided by the OS?

seems to be the vulkan backend of wlroots that fails building

I can reproduce, see wlroots!4029.

patch hyprland to link against wlroots provided by the OS?

Yep but system wlroots was built with Vulkan enabled even if Hyprland uses its own OpenGL renderer instead.

I'd request that you elaborate a bit more on this, given that this way excludes anything that isn't x86_64 (hyprland doesn't build on ia86 for a while now, however everything else [aarch64 in particular] still works).

Hyprland will build just fine with any other arch, but function hooks will not be placed if the arch is not AMD64:
https://github.com/hyprwm/Hyprland/blob/55b66bacb89781e7679a75291dab1f07d052f75c/src/plugins/HookSystem.cpp#L45-L47

Hyprland will build just fine with any other arch, but function hooks will not be placed if the arch is not AMD64

yes, I'm aware of that. I'd just like to know what hyprland accomplishes using inline asm that couldn't be done in C, thus making the other arches "second class" as plugins won't work.

I can reproduce, see wlroots!4029.

great, added this patch and it seems to build on i586 again; thanks. (ideally someone updates the wlroots submodule before the next release, so I can drop that patch again)

yes, I'm aware of that. I'd just like to know what hyprland accomplishes using inline asm that couldn't be done in C, thus making the other arches "second class" as plugins won't work.

You cannot do function hooks without going into assembly.

I have not done anything with the void* cast as it is (IMHO) completely valid, dk why clang complains.

IIRC casting function pointers can only be cast to other function pointers, on a few architectures function pointers can be a different size than data pointers. Then again, the hooks system wouldn't work on those architectures anyways, and if you're going to eventually cast it to uints anyways it really doesn't matter.

This PR is nearing the state where I am happy with writing docs for it and merging. Are there any suggestions / comments / issues left?

@jbeich had to revert your (uint64_t*) casts as they were making me segfault (arch, x86_64)

Function hooks seem done to me. I have 3 different hooks for different types of functions (public member, private member and public non-member) and all seem to work alright.
idk if jbeich has a different idea for a BSD fix, but in the meantime I'll slowly get going on the wiki mr.

Yeah the meson patch no longer applies. I'll fix it in a sec.

On FreeBSD (with GCC 12) #include ".../IHyprLayout.hpp" brings references to symbols not directly used by the plugin. How does it work on Linux?
Nevermind, I think, this is what ASM stuff in CFunctionHook::hook() is for.

Unsure what you mean, but I can't answer why your plugin fails to load.

I'll be waiting for final comments on the plugin system. If no further issues / comments arise till tomorrow, I will be merging.

I'll be waiting for final comments on the plugin system. If no further issues / comments arise till tomorrow, I will be merging.

I haven't looked into API much yet, would it be possible to replace the entire Hyprland config with a custom plugin somewhat like in DWM or Awesome?

technically, yes.
Would you want to do it? No.

Since there are no further comments, I am merging. Thanks everyone who contributed by discussion or code!


alacritty: --add-rpath instead of --set-rpath

Description of changes
As reported by @blucoat in https://github.com/NixOS/nixpkgs/issues/219213 alacritty's RPATH is missing many of the libraries which it links to, including for example glibc.
The problem was diagnosed by @kchibisov as being caused by alacritty's use of --set-rpath (which completely replaces the rpath) instead of --add-rpath (which adds additional entries to the rpath):
https://github.com/NixOS/nixpkgs/issues/219213#issuecomment-1507655801
This commit implements @kchibisov's idea to change --set-rpath to --add-rpath:
https://github.com/NixOS/nixpkgs/issues/219213#issuecomment-1507691822
Closes #219213
Things done

Built on platform(s)

[x] x86_64-linux
[x] aarch64-linux

[x] Tested basic functionality of all binary files (usually in ./result/bin/)
[x] Fits CONTRIBUTING.md.

It looks like we have 537 occurrences of --set-rpath in nixpkgs :frowning_face:
Some of those are part of --set-rpath $(patchelf --print-rpath), which is okay, but most of them are probably causing problems similar to this one.  Ugh.

Maybe our fixupPhase should fail if any binaries in the outpath have unresolved DT_NEEDED entries in their ELF structures.  There would be a mechanism to override this, of course, if needed.

Acked-by: Kirill Chibisov<EMAIL_ADDRESS>


Dunnikri/add tcl to qsys tags

Fixes the following issues in ASE simulation flow involving Qsys systems:

Avoid manipulating file system hierarchy when copying over qsys systems.

When copying over Qsys systems, generate_ase_environment.py adds a suffix (qsys_sim/target_dir_/*) to the original directory string which breaks Qsys systems that rely on relative paths to locate instantiated IP/other qsys systems.

Sub directories under the location where .qsys files are located are not copied over in generate_ase_environment, breaking generation.

Qsys systems that instantiate other IPs require their sources (*_hw.tcl format) in the simulation directory. Adds support in rtl_src_config and generate_ase_environment to copy over TCL sources.

TCL files should be specified in filelist.txt,  with the tag “SI:”. rtl_src_config emits TCL with other simulation sources. They get filtered and copied into the simulation directory.

Coverage remained the same at 66.89% when pulling f533642a3d921a260f946885796f728ae90f0ed3 on dunnikri/add_tcl_to_qsys_tags into 227c1907c961140f73dd3f3f8d566c57b03fd081 on master.

The existing suffix (.tcl) is effectively a tag, too. I think your change already causes .tcl files to be emitted by rtl_src_config only when --tcl is specified, right? If so, you've already eliminated them from Quartus builds and SI: is redundant (unnecessary). You just need to gather the .tcl files using --tcl in generate_ase_environment. You're basically doing that already and, I suspect, don't need the extra check for the suffix in that script.

Re "I think your change already causes .tcl files to be emitted by rtl_src_config only when --tcl is specified, right? If so, you've already eliminated them from Quartus builds"
Not really. All files except the ones with "SI:" tag got added to QII compile flow. Here's what I tried
Emit sources and Quartus/simulator includes
for c in cfg:
if(tag SI: exists)
print(sim file)
elif(tag QII: exists)
print(QII file)
else
if(opts.sim)
print(sim file that exists in sim tag map)
elif(opts.tcl) // causes QII compile to pick up the TCL
print(tcl file)
....
I didnt see a switch to explicitly select the QII compile flow. i,e QII compile is implied by the absence of --sim switch. Did I miss something?

No, you're right. I forgot that .tcl files could be QII files outside of Qsys, so have to be included in the QII group. You've defined the category of SI:*.tcl to be files that need to be handled specially for simulation, which is fine.
I still think you can simplify the new generate_ase_environment code, taking advantage of the fact that rtl_src_config will filter out just the tcl files for you.

Ok. Let me look at it.

@michael-adler @OPAE/ase can you approve now? All travis checks passed.

Thanks Michael and Tracy.


Sending ascii art via irc clients

Or more usefully, how can I compose a multi-line code block via the irc interface and send it?

Not possible, irc doesn't has something like multiline. You can receive ascii art from slack or mattermost though :)

Well, yes, but could you fake it in some way?
What if the ircd buffered the input for something like half a second and if it received a new line inside that period it added it to the buffer and reset the timer? Then if you're pasting/outputting lines programatically you would end up with a multiline post when you stopped sending while the buffer delay would be low enough to not be noticeable for just normal interactive chatting.

Note that since irc throtles messages (by default you can only send 1 message per second, the server will just not process your next message) and you don't want to overflow your buffer on the server (servers have a limited receive buffer, and if you fill it up completly, you get disconnected from the ircd) most clients have a rate limiting in how fast they send out messages to the ircd
This means that if you make matterircd wait "like half a second" you will not solve your problem for most clients, as they will never send 2 messages within the same second...

Bitlbee has an option paste_buffer and paste_buffer_delay where it will join messages coming in in short time, also the flodding delay in the clients can be configured (irc.server_default.anti_flood_prio_high for weechat for example).

I've just commited support for this in master.
Matterircd will buffer up to 500ms automatically and combine those messages in 1 message.
So pasting ascii-art (and other stuff) now works.
Some changes I had to do for irssi
/set paste_join_multiline OFF
/set cmd_queue_speed 10msec
/set cmds_max_at_once 50
/set paste_detect_time 0msecs
/set paste_verify_line_count 50


[VCQ][DevDiv_VSO][VSCode][{0} Would you like to try debugging anyway?: placeholder]

MS PM
Daniel Ye

MOR PM
Jan Baltus (Moravia IT)

String Resource ID
0; "extensions/ms-vscode.js-debug//src/targets/node/nodeBinaryProvider.outOfDate"

Source String
{0} Would you like to try debugging anyway?

Subtenant
VSCode

Source Control Link

Filename
vscode-extensions\ms-vscode.js-debug.xlf

Question
What will be placeholder replaced by? Considered by now as an independent entry.

Additional details

Core Team Response

That message is shown in a dialog when version detection fails. The placeholder is replaced by the original (already-translated) error message, which is usually runtime.node.outdated:
The Node version in "{0}" is outdated (version {1}), we require at least Node 8.x.


Multitasking solutions

We need to figure out how the app will handle multitasking.

Do we need to solve for it? Is this a real use case?
What methods are available?

@kyliefk iOS does not currently allow us to run ARKit while multitasking, so at a bare minimum the experience would need to be limited to touch only. I'm in the "not a real use case camp" and would vote to close this


*: update to latest versions

For https://github.com/coreos/dbtester/issues/309.

Also, given the GC latency improvements in 1.8 Go, are we going to use 1.8 etcd and consul binaries (if available) for benchmarks.

@ronin13 Yeah actually next tests would be consul 0.8+ (with go1.8), etcd 3.2+ (with go1.8).


Is helm mirror still maintained

@ioolkos
The version has remained at 1.12.3 , The latest version is 1.12.6


Channel mapping on Core Audio devices (MacOS)

Hello!
Is it possible to create channel maps for Core Audio devices so that audio signal can be directed to specific channels of a multi-channel interface? I'm currently using a Node.js port of Web Audio API (web-audio-engine) in order to send unique audio to specific channel pairs on a device with 10 outputs, but so far I can only send the audio to two channels without the audio glitching and reporting output underflows. I can successfully accomplish this in-browser by selecting the audio interface as the default output device for the system and routing the audio to the correct channels in my code, but the same routing does not work properly when selecting this device through naudiodon.
The feature in questions is PaMacCore_SetupChannelMap in pa__mac__core.h. @gschmottlach-xse mentioned something similar when referring to his ASIO port of naudiodon, but because the currently supported host APIs present Windows devices in pairs of two (Line 1/2, Line 3/4, etc.), there is a suitable workaround for me to handle the lack of ASIO-specific channel selection features.

If you end up implementing ASIO support in the refactored baseline then please consider fully supporting the ASIO host-specific features/APIs such as channel selectors and retrieving channel names. I did that in my implementation and these APIs are invaluable when you're dealing with ASIO input/outputs with 8 or more available channels and only need access to a subset of those channels. It's easy to forget these ASIO specific APIs. Good luck with your implementation . . . I'd love to see it properly integrated into the baseline!

Please let me know if there's a way to access this functionality via this wrapper, or if you might have any ideas on how to go about achieving similar results. Thanks for your help, and great work on this project!

@psunderr did you have any luck with this?
I attempted Web Audio API to target specific device channels (Eg: Audio Interface channels 3/4) but had no luck.

@Stoyvo unfortunately not, but if you give me more details of your setup (OS, Web Audio library, audio device, etc.), I can let you know if we're having the same issue. You can email me directly at the address in my profile so we don't clutter up this issue.


[bug] Error Switching Networks (canary build)

Is there an existing issue for this?

[X] I have searched the existing issues

RainbowKit Version
0.12.15-canary.3
wagmi Version
0.12.16
Current Behavior
Network switches are erroring and not being pushed to the user's wallet.
Expected Behavior
Network switch request is pushed to the wallet.
Steps To Reproduce

Connect a wallet through walletConnectWallet
Attempt to switch networks through the RainbowKit network selector
Error is logged and switch request is never pushed to the wallet

Link to Minimal Reproducible Example (CodeSandbox, StackBlitz, etc.)
https://web3-j1ied10oz-dylandesrosier.vercel.app/
Anything else?
I know this is a canary build. If these issues are known and will be fixed on release then just lmk! We're looking to just bump v0.12.x to retain WalletConnect support at PoolTogether to avoid the full migration to viem with legacy code.
Repo with code to reproduce:
https://github.com/dylandesrosier/web3-app/tree/walletconnect-v2
Wallet connection code:
https://github.com/dylandesrosier/web3-app/blob/walletconnect-v2/src/components/WalletConnectionProvider.tsx
Error when connecting a wallet:

Error when switching wallets with Trust Wallet:

Not entirely relevant but this was an error on first load (before clearing local storage) after migrating from "@rainbow-me/rainbowkit": "^0.11.0" and "wagmi": "^0.11.5":

@dylandesrosier This is an issue in earlier versions of Wagmi. It was fixed in v1.0.3, and unfortunately the only way to pull in those changes is to get up-to-date with the Wagmi v1 track.

I am told the WalletConnect team will also fix those Error: no matching key errors fired from their library very soon 🤞

@DanielSinclair that looks like a relatively simple change. I see that there's some extra data there but is there no equivalent check that could be done in Wagmi v0?

@dylandesrosier It is internal to Wagmi, so you could patch against the pre-1.0 equivalent
https://github.com/wagmi-dev/wagmi/blob/2b43670b4a04fc65a093a581b9e30d05ff441f27/packages/core/src/actions/providers/watchProvider.ts#L10-L17

Relayed to the Wagmi team to see if we can cherry-pick to 0.12.x

I tried around 10 days ago and it was working fine, and I tried again today and it was not popping up the confirmation in the wallet. I am building a library on top of viem and I am experiencing a similar issue while calling walletClient.switchChain() which ideally should popup the confirmation in the wallet but it was not popping up, just like we are encountering an issue with the rainbowkit. Now, rainbowkit is also using viem bts, thus i think its an issue with viem

@Nazeeh21 Which wallet are you looking at? WalletConnect v2 support is unfortunately unequal right now. Rainbow Wallet would be one of the better test wallets as we have maintaind a close relationship with WalletConnect through this transition.


A few build issues

I encountered some issues getting this library installed and building reliably. They may have been specific to my system, but I'll document them here for posterity and in case anyone else runs into similar snags.
I needed to remove the --no-sys-headers flag to get it to find stddef.h
in /script/d_tree_sitter.d
-  const headers = " --ignore-system-paths --no-sys-headers " ~ prebuiltHeaders;
+  const headers = " --ignore-system-paths " ~ prebuiltHeaders;

and in dub.sdl I needed to adjust my lflags
-lflags "./src/build/libtree_sitter.a"
+lflags "$PACKAGE_DIR/src/d_tree_sitter/build/libtree_sitter.a"

Are you on MacOS? Because I had trouble building the project on Mac. We can try your changes inside CI, and merge them.

This was on (Arch) Linux. I'm sure I'll end up debugging this on MacOS at some point though.

Build on MacOS is now added to CI. You might try on Arch Linux and let me know.

This fix is applied in #8


Fast field column index based field presence.

We recently introduced field presence by indexing a special field presence field.
This however comes at a cost. That field presence field is redundant with the docvalue column index if the field is a docvalue.
In that case, we should not populate the special field presence field and instead return a DocSet built using the DocSet leveraging the ColumnIndex.

unassigning

@fulmicoton  What does reserved mean?

Should we add this as a new query to tantivy first and then hook it up in quickwit or do you want to keep everything in quickwit? In case the initial presence-field based implementation, this query was translated into existing term query. But I don't see a way to return this special DocSet without introducing a new query and I don't see an issue for such query in tantivy. Hence my question.


Recognize "rlib" and "cdylib" crates

Addresses #828 and https://github.com/lunaryorn/emacs-module.rs/issues/4 and probably numerous others.

Thank you!


TokenIssuer.#implicit_grant_with_creds redir_url is fixed

The implicit_grant_with_creds always fixes the redir_ui to point to cloud foundry (https://uaa.cloudfoundry.com/redirect/#{@client_id}). This causes problems if uaa is deployed outside.

@yogeshsajanikar We have stopped active development on cf-uaa-lib, and are planning to build a replacement for it. I will close this issue now.


chore: add a zzz-release binary that update the version of crates

It updates the versions, create a commit, and show which packages need to be published in what order.

This does not follow best practices, sorry. I can split some functions so it becomes easier to read.

Let's have in the readme a section that tells what we should cargo run when releasing


Configure component once, use many times

Discussed in https://github.com/streamsync-cloud/streamsync/discussions/195

Originally posted by abeka14 January 11, 2024
Hi all.
StreamSync is exactly what i was looking for. Thank you for developers. I want to build an app with several pages, and I want to navigate to those pages through buttons in left sidebar. StreamSync allows to use sidebar, but it puts it inside particular page. So, it is gone when you open a new page. Are there any ways to make a sidebar available in all pages? Thank you in advance!

@raaymax This is the Reuse component we talked about, cool feature, would use the new options for Text fields that you developed.
If you ever need a break from frontend testing there's this ;)


WebCmdlets should support retry

suggested design
Invoke-WebRequest ... -RetryCount <int> -RetryInterval <int-seconds>

This one was on my personal wish list!

As there is already a pull request #5760 for this issue, I'd like to add my thoughts with regards to this new feature.
Many Web Services recommend that clients should implement a retry strategy to cope with intermittent failures of a web service, but also not to overload the service with too many subsequent requests (e.g. use Exponential Backof). But there are also other use cases for retries, where a client wants to regularly check if a Website comes online (retry until Status Code not 404), if a resource was modified (retry until Status Code not 304). Some responses may indicate, that the request rate should be reduced (e.g. AWS uses Status Code 503 "Slow Down"), which should be considered by a retry strategy.
In my opinion, PowerShell should allow to configure different strategies for retries. The following are some of my thoughts and points taken from the discussion in PR #5760:

Parameter -Retry could indicate that the WebCmdlet should retry the request with a default, conservative, retry strategy
Parameter -RetryCount could indicate how many retries should be made
Parameter -RetryIntervalSec could indicate the interval between different request
Parameter -RetryBackofStrategy could indicate if linear or exponential backof strategy should be used
Parameter -RetryStatusCodes could contain a list of Status Codes for which the request is retried
Parameters could also be set on WebSession in PrepareSession() and then have this area of code retrieve the values from WebSession. It would mean extending WebRequestSession to include new Properties

In my opinion this is such an important feature, that it requires more discussion and an agreement on how it should be implemented, before work is continued on the PR.

I agree that we should consider the design more thoroughly before work continues in the PR. At present, users can roll their own retry and backoff strategies, so it is not urgent that we include this feature. However, including the wrong feature may make us a bad citizen and promote poor behavior.  I definitely don't want that.
/cc @adityapatwardhan @SteveL-MSFT

I'll just add a couple of thoughts:

it seems that there's enough discussion to warrant a RFC
by default, we should borrow from curl making the assumption they've already spent time going through design discussion unless there's something that we can improve upon
there doesn't need to be a single PR to address all the requests, we can do them in stages as long as subsequent PRs build upon previous PRs and don't introduce confusion or compatibility breaks


How do you test SPM.

SPM uses Lora, do you save it as a Unet in pt format?

Yes. The tested model checkpoints can be found in Google Drive.
More details about how to download them through one command can be found in AdvUnlearn's Github.


test MixedL21Norm

Currently MixedL21Norm has some raise an error

Traceback (most recent call last):
  File "C:\Users\ofn77899\Documents\Projects\CCPi\GitHub\hackathon\block\Wrappers\Python\test\test_functions.py", line 340, in test_mixedL12Norm
    f_no_scaled.proximal_conjugate(U,1,out=z3)
  File "C:\Users\ofn77899\Documents\Projects\CCPi\GitHub\hackathon\block\Wrappers\Python\ccpi\optimisation\functions\MixedL21Norm.py", line 114, in proximal_conjugate
    res = (sum(tmp).sqrt()).maximum(1.0)
AttributeError: 'int' object has no attribute 'sqrt'

see #231

WIP in pdhg_fix branch

I think this can be closed as the test in question pass now.


New command: Export Microsoft Teams messages for the given user or team

Usage
m365 teams message export [options]
Description
Export Microsoft Teams chat messages for a given user, or a team.
Options

Option
Description

--userId [userId]
The Id of the user whose messages are to be exported. Specify either userId, userName or teamId

--userName [userName]
The UPN of the user whose messages are to be exported. Specify either userId, userName or teamId.

--teamId [teamId]
The Id of the team of which the messages are to be exported. Specify either userId, userName or teamId.

--fromDateTime[fromDateTime]
The start of time range to query

--toDateTime[toDateTime]
The end of time range to query

-l, --licenseModel [licenseModel]
License requirements are applicable on the protected Teams export API. Allowed values are modelA, modelB, evaluation. The default is evaluation. Read more about the license requirements.

-a, --withAttachments
When specified, the command will also download all chat attachments. Use folderPath as well to set the rootfolder.

-p, --folderPath [folderPath]
The folder path to save downloaded attachments to when using withAttachments.

Remarks
Microsoft Teams APIs in Microsoft Graph that access sensitive data are considered protected APIs. Export APIs require that you have additional validation, beyond permissions and consent, before you can use them. To request access to these protected APIs, complete the request form
Additional Information
https://docs.microsoft.com/en-us/microsoftteams/export-teams-content#how-to-access-teams-export-apis
Prerequisites to access this API
https://docs.microsoft.com/en-us/microsoftteams/export-teams-content#prerequisites-to-access-teams-export-apis

@pnp/cli-for-microsoft-365-maintainers here is a new command for review of spec, please let me know if this is okay and then we can put it in the backlog and label it appropriately. Thanks 😊

@pnp/cli-for-microsoft-365-maintainers here is a new command for review of spec, please let me know if this is okay and then we can put it in the backlog and label it appropriately. Thanks 😊

Thanks for the suggestion.

I wouldn't call the command export. As it retrieves all items, we should name it Get all Microsoft Teams messages for the given user
To make the command easier to use, I'd suggest that we allow specifying user's UPN rather than ID, which users would need to retrieve separately first
Are filter, pageSize and pageNumber meant to be required? If not, let's denote them as optional using [] rather than <>
What can be specified in the filter?
What is the default behavior if pageSize and pageNumber are not specified?

Thanks for the suggestion.

I wouldn't call the command export. As it retrieves all items, we should name it Get all Microsoft Teams messages for the given user
To make the command easier to use, I'd suggest that we allow specifying user's UPN rather than ID, which users would need to retrieve separately first
Are filter, pageSize and pageNumber meant to be required? If not, let's denote them as optional using [] rather than <>
What can be specified in the filter?
What is the default behavior if pageSize and pageNumber are not specified?

Good catch on point # 3! thanks @waldekmastykarz.
In the filter they can specify the date range or messageType etc. See sample response here
{
"id": "string (identifier)",
"replyToId": "string (identifier)",
"from"<EMAIL_ADDRESS>"microsoft.graph.identitySet"},
"etag": "string",
"messageType": "string",
"createdDateTime": "string (timestamp)",
"lastModifiedDateTime": "string (timestamp)",
"deletedDateTime": "string (timestamp)",
"subject": "string",
"from": {
                "application": null,
                "device": null,
                "conversation": null,
                "user": {
                    "id": "string (identifier)",
                    "displayName": "string",
                    "userIdentityType": "aadUser"                }
            },
"body"<EMAIL_ADDRESS>"microsoft.graph.itemBody"},
"summary": "string",
"chatId": "string (identifier)"
"attachments"<EMAIL_ADDRESS>"microsoft.graph.chatMessageAttachment"}\],
"mentions"<EMAIL_ADDRESS>"microsoft.graph.chatMessageMention"}\],
"importance": "string",
"locale": "string",
}

The API gives back @odata.nextlink in case there are multiple values similar to other commands or this sort.

Good catch on point # 3! thanks @waldekmastykarz.
In the filter they can specify the date range or messageType etc. See sample response here
{
"id": "string (identifier)",
"replyToId": "string (identifier)",
"from"<EMAIL_ADDRESS>"microsoft.graph.identitySet"},
"etag": "string",
"messageType": "string",
"createdDateTime": "string (timestamp)",
"lastModifiedDateTime": "string (timestamp)",
"deletedDateTime": "string (timestamp)",
"subject": "string",
"from": {
                "application": null,
                "device": null,
                "conversation": null,
                "user": {
                    "id": "string (identifier)",
                    "displayName": "string",
                    "userIdentityType": "aadUser"                }
            },
"body"<EMAIL_ADDRESS>"microsoft.graph.itemBody"},
"summary": "string",
"chatId": "string (identifier)"
"attachments"<EMAIL_ADDRESS>"microsoft.graph.chatMessageAttachment"}\],
"mentions"<EMAIL_ADDRESS>"microsoft.graph.chatMessageMention"}\],
"importance": "string",
"locale": "string",
}

The API gives back @odata.nextlink in case there are multiple values similar to other commands or this sort.

Hi,
not sure if we will be able to use this command at a large scale. I fell into this already a couple of times with some other interesting endpoints.

Microsoft Teams APIs in Microsoft Graph that access sensitive data are considered protected APIs. Export APIs require that you have additional validation, beyond permissions and consent, before you can use them. To request access to these protected APIs, complete the request form.

The command will throw a 401 exception until you got an approval from Microsoft and your AppId has been enabled.
@waldekmastykarz I do not believe that MS will enable our PnP CLI app with these permissions due to security reasons. I tried to get the approval already a couple of times but failed in the past.
Is this something we want to clarify before? Maybe have some discussions with the Microsoft responsible parties to see how we can proceed here?

Hi,
not sure if we will be able to use this command at a large scale. I fell into this already a couple of times with some other interesting endpoints.

Microsoft Teams APIs in Microsoft Graph that access sensitive data are considered protected APIs. Export APIs require that you have additional validation, beyond permissions and consent, before you can use them. To request access to these protected APIs, complete the request form.

The command will throw a 401 exception until you got an approval from Microsoft and your AppId has been enabled.
@waldekmastykarz I do not believe that MS will enable our PnP CLI app with these permissions due to security reasons. I tried to get the approval already a couple of times but failed in the past.
Is this something we want to clarify before? Maybe have some discussions with the Microsoft responsible parties to see how we can proceed here?

@rabwill rather than having a catch-all filter property, I'd suggest we introduce specific options for the specific values so that we can apply validation to it.
@plamber I wouldn't expect getting clearance for our app, especially as it doesn't support app-only auth. Instead, users would get clearance for their own app and use it in combination with CLI. In the past, I requested and got approval so the process is working given proper justification. I agree that the required clearance will likely limit the usage of this command. Perhaps if we can describe a number of usage scenarios and the justification, it could help users get approvals and use this command.

@rabwill rather than having a catch-all filter property, I'd suggest we introduce specific options for the specific values so that we can apply validation to it.
@plamber I wouldn't expect getting clearance for our app, especially as it doesn't support app-only auth. Instead, users would get clearance for their own app and use it in combination with CLI. In the past, I requested and got approval so the process is working given proper justification. I agree that the required clearance will likely limit the usage of this command. Perhaps if we can describe a number of usage scenarios and the justification, it could help users get approvals and use this command.

@waldekmastykarz so just the date range to start off now, and we can add any as we please?

@waldekmastykarz so just the date range to start off now, and we can add any as we please?

@rabwill is date the most common filter?

@rabwill is date the most common filter?

Since it is already mentioned as a filter example in the doco, perhaps it's a good start? ☺

Since it is already mentioned as a filter example in the doco, perhaps it's a good start? ☺

Good point 👍

Good point 👍

Shall we move this to backlog then? Do we need more discussion @plamber @waldekmastykarz ?

Shall we move this to backlog then? Do we need more discussion @plamber @waldekmastykarz ?

I think we're good to go 👍

I think we're good to go 👍

I would like to take this

Opening up due to lack of activity

We do have a command for listing messages nowadays
https://pnp.github.io/cli-microsoft365/cmd/teams/chat/chat-message-list/
How would this relate? Should we maybe review the specs?

Following this spec, this command would use the Teams export APIs which are different than the Chat message list APIs.

For example teams chat message export?

Oh wait, this is across all chats for the user. Never mind. Great spec!

On...the...other...hand 🤗
The teams export api exposes two endpoints according to the docs, one for user chat messages and the other for channel messages.
If we where to add both as commands, we'd run into our existing channel message list.
To align it with our existing commands we could have two new commands:
teams chat message export (current issue)
teams channel message export (to be created as issue)
This way they would align better and it would be clearer what the commands do.
I'd also vote for an additional option:

Option
Description

'--licenseModel [licenseModel]
The license model to use when executing the command. Accepted values: `A

Great suggestion @martinlingstuyl! What if we'd combine them both under teams message export? We already have a message namespace in the CLI. Would that make sense or are the chat message and channel message APIs so different that it wouldn't make sense to combine them under message?

I think the only thing that's different is the URL. So it would be possible to combine them.
Now you're saying it, we do not have teams channel message list, I was speaking too quickly. We have teams message list. All the teams message <verb> commands refer to channel messages.
You might debate whether it is logical to put a command in there that can export both channel and chat messages. On the other hand, the teams chat message namespace is all about the current user. While teams message is not really about the current user.
I think I like the idea. We do need different options:

Option
Description

--userId [userId]
The Id of the user whose messages are to be exported. Specify either userId or channelId.

--channelId [channelId]
The Id of the channel of which the messages are to be exported. Specify either userId or channelId.

I'd say we drop userName from the specs.

@martinlingstuyl I agree. Since we already have the m365 teams message list with --channelId option I would leave it and include a new option which is --userId. So I totally agree and feel it the same way as you described 👍.
@pnp/cli-for-microsoft-365-maintainers  any other comments?
maybe we could open this up?... and maaaybe even as "good first issue" ?

As spoken about above, We should update the spec a bit first:
teams message export [options]

I updated it, what about this?

@martinlingstuyl weren't we suppose to drop the userName option for now?

Well, we're using both options in some other places as well, so I think we can use them here.
But before we open it up: we need some additional research here:
https://devblogs.microsoft.com/microsoft365dev/announcing-general-availability-of-microsoft-graph-export-api-for-microsoft-teams-messages/
We might need teamId instead or next to channelId.
And we certainly need a licenseModel option as well. We discussed that earlier and it is applicable on this api.

Also: what about retrieving chat-attachments:

Message Attachments: Export APIs include the links to the attachments that are sent as part of messages. Using Export APIs you can retrieve the files attached in the messages.

We could add something of a switch to trigger behavior that downloads the attachments.

Ok, I updated the specs a bit, adding the licenseModel option and teamId.
What do you guys think of auto exporting the attachments as well, (using --includeAttachments, and --filePath options) or should we leave that to the user, as the attachment urls are provided?

I also did a check: it seems the endpoint getAllMessages can only be called on team level, for all channels. It seems to be unavailable on channel level.
I need to do a better test, but it seems we can drop channelId

good research @martinlingstuyl 👍.
I think the --includeAttachments option would be ok and not exporting them by default.

Any thoughts from other @pnp/cli-for-microsoft-365-maintainers about the attachments?

Looking at the different purpose and functionality (and caveats) of the export and the regular list API, I wonder if we should combine both commands after all or if we should rather keep them separate. My gut feeling says to keep them separate, because using the export API requires you to have a specific AAD app, which has been granted special API access and a license, whereas you don't need either for the regular list command, which isn't meant btw to be used at scale.

@waldekmastykarz this is a very valid comment 😯.
@martinlingstuyl what do you think about making two separate commands? And maybe separate issues?

We already have teams chat message list, so we'd just need one new command for export, which this issue is for, correct?

Correct,
We have teams chat message list.
The export should be a separate command.
I'm doubting though if we should rename it to teams chat message export. It would be more consistent.
What about the question around attachments. What do you think of that?

I'm doubting though if we should name this command teams chat message export. Instead of teams message export. It would be more consistent with the other command.

teams message export seems more suiting indeed. Looking at the docs, it seems like you can't export messages from a specific chat, so adding chat in the command's name would be misleading.

What about the question around attachments. What do you think of that?

How about --withAttachments?

Ok, Check the updated specs. How about that?

I'd suggest we rename --filePath to --folderPath, since we want people to point to a folder rather than a file

Do we need the pageSize and pageNumber options?

Do we need the pageSize and pageNumber options?

We don't!

I'd suggest we rename --filePath to --folderPath, since we want people to point to a folder rather than a file

🤦♂ OMC (Cussing for nerds 2.0: oh my code)

@martinlingstuyl looks ok to me 👍

@waldekmastykarz any comments?
it has been a long run 🙂 @martinlingstuyl should we open it up ?

Would love to work on this one.

🧹time!
All yours @MathijsVerbeeck!

Just some feedback regarding this:
I've written the code, but am still struggling with something.
When retrieving the messages, we will get a JSON of the message with an attachment body like this:

If we would then use --withAttachments, we would have to download this file. This cannot be done by directly using the contentUrl, but we could do this by using getFileByServerRelativePath.
I'm just still struggling with the inconsistency on how we would define our url, as, theoretically, it is possible to refer to a file on the root site collection, with would make the weburl hard to compose.
Not sure if any of you have any suggestions regarding this?

I'm just still struggling with the inconsistency on how we would define our url, as, theoretically, it is possible to refer to a file on the root site collection, with would make the weburl hard to compose.

Not sure I understand the problem. If we split the URL into authority (protocol + domain) and the server-relative path, wouldn't that be enough to build a URL like https://contoso.sharepoint.com/_api/<the API call to get the URL by server-relative URL>?

@waldekmastykarz But if, let's say, I have a file under /sites/testsite, I cannot retrieve this file using https://contoso.sharepoint.com/_api/web/getfilebyserverrelativepath(DecodedUrl='/sites/testsite/shared documents/file.docx'), as this will throw an error file does not exist. You have to do this specifically on the site collection. So the question is: How would I know what the sitecollection of which to retrieve the files would be.

So the challenge is to find site collection URL from the full file URL? If so, it's vaguely familiar and I'm pretty sure we implemented it somewhere. Not sure though if it was for SharePoint or Graph APIs, but I'm pretty sure we have a solution for it in our code base.

Exactly. I will have a look then 😄 Thanks for the feedback!

Check out getGraphSiteInfoFromFullUrl in file convert pdf. Not sure if it's an exact match but it could help you get started.


optional ability to configure handoff wait timeout on realtime tasks

this PR introduces "handoffConditionTimeout" property in realtime tuning configure for users to be able to configure a timeout on the handoff wait condition (by default, old behavior of waiting forever is retained).
we need this because we have had many instances of data loss (including one today) where coordinator/historical were not able to load new segments for hours for a systematic failure and new realtime tasks failed to start because old tasks were taking up all worker slots and never finishing. when new realtime tasks can't start, then realtime pipeline loses all the data it has.
if new tasks could start then there wouldn't have been any data loss because tasks were able to create segments, copy them to hdfs and update db.

@himanshug could you include a test? It should be possible to add one in RealtimeIndexTaskTest, as that already has some tests for handoff.

@gianm addressed all comments and added teset for handoff timeout.

:+1: after travis

:+1: it would be good to append PR number in commit message to make it simple to track issues.


docs: use files directory instead of config directory

The website documentation still mentions using the config/files directory, but the bluebuild client gives this warning:  "WARN  => Use of the ./config directory is deprecated. Please move your non-recipe files into ./files".
Also including clarification for the following would be helpful:

Is simply renaming 'config' directory to 'files'  is all that's needed?
For the files module, would they be placed in files/files or just /files?

@notnotno It currently does mostly work, since the preliminary work is done in BlueBuild CLI regarding this.
Only gschema-overrides module won't work, which has hardcoded path to config/.
But we also changed files module to conform to this new files/ location, so it can use any location of choice in files/ instead of being hardcoded to files/files/ in your case.
It took some time until we fully investigated & decided on how this transition from config/ to files/ will finally look.
Hence why we didn't make docs about this yet.

This issue seems to just have been forgotten to be closed. See https://blue-build.org/blog/config-files/


nvim outputs 'Plug' randomly and unpredictably

Example: Just wanted to type check and got che<Plug>CocRefreshck.
This happens quite often when you type normally  but it is hard to reproduce. But I found a way: If you smash the buttons fgh like fghfghfghfghffhgfhgf you get hgjhgjhgggjhgjh<Plug>CocRefresh<  Plug>CocRefresh<Plug>CocRefreshg<Plug>CocRefresh<Plug>CocRefresh<Plug>CocRefreshjh<Plug>CocRefreshg<Plug>CocRefresh<Plug>CocRefresh<Plug>CocRefresh<Plug>CocRefreshjh<Plug>CocRefresh<Plug>CocRefresh<Plug>CocRe  freshg<Plug>CocRefresh<Plug>CocRefresh<Plug>CocRefreshjh<Plug>CocRefresh<Plug>CocRefresh<Plug>CocRefresh<Plug>CocRefreshgh<Plug>CocRefresh<Plug>CocRefresh<Plug>CocRefresh<Plug>CocRefresh<Plug>CocRefresh<Plug>  CocRefreshjh<Plug>CocRefresh<Plug>CocRefresh<Plug>CocRefresh<Plug>CocRefresh<Plug>CocRefreshg<Plug>CocRefres
The tsserver log shows:
Info 251  [18:39:58.668] request:                                                                                {"seq":25,"type":"request","command":"updateOpen","arguments":{"changedFiles":[{"fileName":"/MYJSFILE.js","textChanges":[{"newText":"hgjhgjhgggjhgjh<Plug>CocRefresh<  Plug>CocRefresh<Plug>CocRefreshg<Plug>CocRefresh<Plug>CocRefresh<Plug>CocRefreshjh<Plug>CocRefreshg<Plug>CocRefresh<Plug>CocRefresh<Plug>CocRefresh<Plug>CocRefreshjh<Plug>CocRefresh<Plug>CocRefresh<Plug>CocRe  freshg<Plug>CocRefresh<Plug>CocRefresh<Plug>CocRefreshjh<Plug>CocRefresh<Plug>CocRefresh<Plug>CocRefresh<Plug>CocRefreshgh<Plug>CocRefresh<Plug>CocRefresh<Plug>CocRefresh<Plug>CocRefresh<Plug>CocRefresh<Plug>  CocRefreshjh<Plug>CocRefresh<Plug>CocRefresh<Plug>CocRefresh<Plug>CocRefresh<Plug>CocRefreshg<Plug>CocRefres","start":{"line":1,"offset":38},"end":{"line":1,"offset":38}}]}]}}                                   Info 252  [18:39:58.669] Project '/dev/null/inferredProject1*' (Inferred) 0                                                                                Info 252  [18:39:58.672] >Files (1473)
I had some extensions installed but deinstalled all and tested them isolated and without any extension this bug doesn't come but once I installed coc-tsserver it appears.
I have latest node v12.6.0 and the latest nvim nightly v0.4.0-1326-g34c6e5bea
installed coc extensions:
    1 + coc-tsserver 1.3.13 ~/.config/coc/extensions/node_modules/coc-tsserver

my nvim coc config (there is config for other extensions from before the test):
 call coc#add_extension('coc-tsserver')                                                                                map <leader>cc :CocConfig<CR>                                                                                map <leader>cm :CocList marketplace<CR>
  map <leader>cl :CocList<CR>
  set signcolumn=yes

  " Use tab for trigger completion with characters ahead and navigate.
  " Use command ':verbose imap <tab>' to make sure tab is not mapped by other plugin.
  inoremap <silent><expr> <TAB>
    \ pumvisible() ? coc#_select_confirm() :
    \ coc#expandableOrJumpable() ? "\<C-r>=coc#rpc#request('doKeymap', ['snippets-expand-jump',''])\<CR>" :
    \ <SID>check_back_space() ? "\<TAB>" :
    \ coc#refresh()
  inoremap <expr><S-TAB> pumvisible() ? "\<C-p>" : "\<C-h>"

  function! s:check_back_space() abort
    let col = col('.') - 1
    return !col || getline('.')[col - 1]  =~# '\s'
  endfunction

  let g:coc_snippet_next = '<tab>'

  " Use <c-space> to trigger completion.
  inoremap <silent><expr> <c-space> coc#refresh()

  " Remap keys for gotos
  nmap <silent> gd <Plug>(coc-definition)
  nmap <silent> gy <Plug>(coc-type-definition)
  nmap <silent> gi <Plug>(coc-implementation)
  nmap <silent> gr <Plug>(coc-references)

  " Use K to show documentation in preview window
  nnoremap <silent> K :call <SID>show_documentation()<CR>

  function! s:show_documentation()
    if (index(['vim','help'], &filetype) >= 0)
      execute 'h '.expand('<cword>')
    else
      call CocAction('doHover')
    endif
  endfunction

  " Highlight symbol under cursor on CursorHold
  autocmd CursorHold * silent call CocActionAsync('highlight')

  " coc-smartf
  nmap f <Plug>(coc-smartf-forward)
  nmap F <Plug>(coc-smartf-backward)
  nmap ; <Plug>(coc-smartf-repeat)
  nmap , <Plug>(coc-smartf-repeat-opposite)

  augroup Smartf
    autocmd User SmartfEnter :hi Conceal guifg=#000000 guibg=#1fffd2
    autocmd User SmartfLeave :hi Conceal guifg=#00ff00
  augroup end

my coc config:
{
"coc.preferences.formatOnType": true,
"signature.enable": true,
"tsserver.npm": "/home/name/.nvm/versions/node/v12.6.0",
"tsserver.formatOnType": false,
"typescript.showUnused": false,
"javascript.preferences.quoteStyle": "single",
"javascript.validate.enable": false,
"smartf.timeout": 1000,
"suggest.noselect": false
}

I found the reason: i had to remove the plugin vim-paste-easy


Test very large display

I added a new example that provides a very large display (1.1 million photos from the Mars rovers). The metaData.jsonp file is >250MB. It's here on GitHub: https://github.com/hafen/trelliscope-examples3/tree/main/mars
I gzipped metaData.jsonp so that it would be easier to commit so you probably need to decompress that before testing.
This provides a good example to test large displays. It takes a long time to load the metadata. On Chrome it works fine but on Safari I get this:

It might be nice to add a loading spinner. I'm also wondering if in cases like this I should provide a subset of the metadata for the first 10-20 panels so that we can at least show something while the rest is loading. Any thoughts?

hmmm i think the ideal solution would be to load it asynchronously especially to lay the foundation in the event a backend is ever implemented. i think this will be good to bring up tomorrow in our meeting too.

this is no longer an issue when we looked into the jsonp to js file and speeding up load times


Website Logo

Make a sketch for our logo, you guys can comment below under the Website logo issue

What will be our logo fellas?


Merge Jetpack Compose 1.5.0

Merge JC 1.5.0 commit

hold on, the build fails on floatValue

hold on, the build fails on floatValue

Ready for review


#19: fix extra indentation in add_to_chain beats.

A long, long time ago, before ScreenPy was split up, and before we had The Narrator... the little extra indent on chained Actions helped to show which Actions were part of the chain.
ScreenPy has grown up, and out, and now we've got some more sophisticated systems in place to log things much nicer. Those extra indents are now weird anomalies! Granted, weird anomalies that aren't easy to see, but annoying to those who notice (cough @bandophahita cough).
This PR removes that extra little indentation, since it's no longer needed!

heh @MarcelWilson approved from his work account again, that silly rascal!


Root relative outputs relative URL in single page application

Hello, I noticed the root-relative option https://github.com/webpack/css-loader#root-relative-urls doesn't work well if you have a single page application that uses history.pushState.
Example:
I have the following structure:
/
--/src
  --/assets/images/hello.png
  --/css/style.css
--/build
  -- (here is where files go after compilation)

And I have an hello.css file with the following content:
#hello {
  background-image: url("/assets/images/hello.png")
}

Now, my config for CSS and PNG files will be something like:
{test: /\.css$/, loader: 'style-loader!css-loader?root=..}
{test: /\.(png)$/, loader: 'file-loader?name=[path][name].[ext]?[hash]&context=./src'}

This results in the output-ed css being (notice the missing / before the path):
#hello {
  background-image: url("assets/images/hello.png")
}

Which will work well as long as I access my application at /, but will fail for example if I share an URL like /hello/world because then the background image will be searched at /hello/world/assets/images/hello.png.
Should absoulte paths be translated? If I don't specify a root, then the file-loader doesn't kick in so I won't have my assets during build.
I see 2 options:

When you don't specify a root, css-loader requires the files anyway, just doesn't change the url.
If you specify a root, the loader is used but the path isn't translated.

What do you think?

I also see 2 options:

provide a output.publicPath. i. e. "/". It's prefixed all generated urls.
use a <base> tag in your HTML to set to correct base path.

That's great! Thank you a lot :)


Material Image loading error

For #2561. Print error to the console when a material image fails to load. This message will get sent to the Sandcastle console. The downside is that both a 404 and this message is printed to the dev console.

We don't need this.


Icon request: Battle.net

URL of the site or app (required): https://battle.net/
Logo resources for the icon (optional, but recommended): https://www.dropbox.com/s/148sfcn76ipwqje/Battlenet.svg?dl=0

Use Blizzard icon (Blizzard has rebranded the Battle.net to "Blizzard Battle.net")
https://github.com/krisu5/aegis-icons/blob/master/PNG/Blizzard.png


about CEQ metric

Hi, excited to see such an interesting new dataset. There are some questions about the evaluation metric CEQ.
the CEQ score has a significant positive relationship with the human evaluation results. but I didn't find the corresponding section in the code，nor in the leaderboardleaderboard.
where can i find the CEQ metric.

Hi, excited to see such an interesting new dataset. There are some questions about the evaluation metric CEQ. the CEQ score has a significant positive relationship with the human evaluation results. but I didn't find the corresponding section in the code，nor in the leaderboardleaderboard. where can i find the CEQ metric.

Thank you for your interest in our dataset. We will upload the evaluation script and relevant data of CEQ metric in the next few days.

Hi, thanks for your exciting work!
Where can I find the CEQ metric now? I still can't find the corresponding script or results.

Sorry about that, we will upload the code and data by next week.

Sorry for the delay.
We have uploaded the CEQ metric script, refer to https://github.com/Waste-Wood/e-CARE/tree/main/CEQ
Thanks for following our work.


Ensure all error pages show the process bitness

https://github.com/aspnet/AspNetCore/tree/master/src/Shared/ErrorPage should be updated to show process bitness.

We think most of them are good but @shirhatti is concerned about the Hosting ErrorPage.


[UI] Settings inconsistent with the UI,dashboard icon disappearing

Current Behavior
The UI of the settings page is not consistent with the UI also the dashboard icon was disappearing with the click.
Desired Behavior
The UI must be consistent for all pages.
Screenshots/Logs

![Screenshot from 2023-01-31 01-12-31](https://user-images.githubusercontent.com/99184676/215579504-c0a7c0e5-d96f-48c5-8c91-65a2b9401

a7d.png)
Environment

Browser: Chrome Safari Firefox
Host OS: Mac Linux Windows
Meshery Server Version: stable-v
Meshery Client Version: stable-v
Platform: Docker Kubernetes

Contributor Guides and Handbook

🎨 Wireframes and designs for Meshery UI in Figma
🖥 Contributing to Meshery UI
🙋🏾🙋🏼 Questions: Discussion Forum and Community Slack

@leecalcote Can you assign this one to me??

Thank you for calling this out, @partik03


gmtpy

Just to mention that this library here exists since around 7years by @emolch
https://github.com/emolch/gmtpy
Examples and usage here:
http://emolch.github.io/gmtpy/
Before you restart doing something that has been done already?
Maybe this can be brought together?

@hvasbath I'm well aware of the other GMT python attempts: https://genericmappingtools.github.io/gmt-python/design.html#previous-work
I'm taking a very different approach to the wrapper and would probably have to rewrite most of any of those libraries. Plus, they are made for GMT5. I'm working with Paul Wessel to make GMT6 and this library is a being developed side-by-side with it.


Setting a trigger for SSL bug that set EnablePopUps=false automatically.

Description of bug
DNN bug with SSL sites and POP UPs.
Since it seems like you have to deactivate POP UPs for SSL Signed pages in DNN, This customer needs a trigger that actually set EnablePopUps=false automatically.
The fact is that it has to work only with the SSL pages that the customer sets up. For the other pages, which are not signed with SSL, the POP UPs should work as usual.
Steps to reproduce
Sign with SSL a web page

Try to do any action that triggers a pop up. i.e. Click on login or registration on top right

Current result
Mixed content error to load unsecure http resource
Expected result
It should load popup without mixed content error
Affected version

[x] 9.2.2
[x] 9.2.1
[x] 9.2
[x] 9.1.1

imo the correct fix would be supporting popups on SSL pages.

Hmm, any insight on that popup bug with ssl enabled, I have a clean 9.4.0 RC with ssl and popups work without warnings, I also have some other older sites with ssl and the login does not show in a popup. Sooo was this fixed but the PR not linked to this or is there more to reproduce this issue on recent versions. @Mohtshum can you reproduce this behaviour on a clean 9.4.0 RC install ?

Closing this issue due to a lack of verification.  If someone sees this is still a valid issue, please reply and we'll re-open.


Activation of the Python extension

Issue Type: Bug
When I startup VS Code I get this in a workspace that exclusively contains markdown files. This is the only window I have open. Why?

Extension version: 2021.2.636928669
VS Code version: Code - Insiders 1.55.0-insider (Universal) (9b2ee7fb7d970b9a628fdb3545bdd01d67078d16, 2021-03-11T05:15:23.749Z)
OS version: Darwin arm64 20.3.0

System Info

Item
Value

CPUs
undefined

GPU Status
2d_canvas: enabledgpu_compositing: enabledmetal: disabled_offmultiple_raster_threads: enabled_onoop_rasterization: enabledopengl: enabled_onprotected_video_decode: unavailable_offrasterization: enabledskia_renderer: disabled_off_okvideo_decode: enabledwebgl: enabledwebgl2: enabled

Load (avg)
2, 3, 2

Memory (System)
16.00GB (0.28GB free)

Process Argv
--enable-proposed-api lszomoru.lszomoru-proposed-api-sample --crash-reporter-id ef10dfea-3602-4c74-ba1a-109ce7da7825

Screen Reader
no

VM
0%

A/B Experiments
vsliv695:30137379
vsins829:30139715
vsliv368:30146709
vsreu685:30147344
python383:30185418
pythonvspyt602:30263608
vspor879:30202332
vspor708:30202333
vspor363:30204092
vstry244:30244315
pythonvsdeb440:30224570
pythonvsded773:30223139
pythonvspyt875:30259475
pythonvsnew554:30265444
pythontb:30258533
vscoresta800:30265947
vspre833:30267464

This also happens btw for an empty window:

Our activation events are explicitly tied to python commands, language selection, or Python-specific files:
https://github.com/microsoft/vscode-python/blob/f85d5912f89da029a3f4fffcc2d46a387f70cff1/package.json#L51-L94
The only thing that changed recently is we trigger on more Python-related files:

pyproject.toml
Pipfile
setup.py
requirements.txt
manage.py
app.py

Otherwise our activationEvents have not changed in 4 months and that was only for registering a command.
Only other thing involving start-up is we added "requiresWorkspaceTrust": "onStart" recently, but our understanding is that doesn't force activation. Is there a way to see in the logs why an extension was activated?

I realized this is because of https://github.com/DonJayamanne/typescript-notebook/issues/4.

The real trouble here is the dependency from the Jupyter extension to the Python extension.  I.e. when Jupyter activates, Python activates although, as in this case, there is no reason for Python to be active.

The long-term plan is to break the hard dependency and make it optional. I suspect Jupyter already has an issue open for this, but in case they don't I'll transfer your issue over to there.


Manager: do not use functions that return widgets

This is bad practice, see:

https://stackoverflow.com/questions/53234825/what-is-the-difference-between-functions-and-classes-to-create-r- eusable-widgets/53234826#53234826
https://github.com/flutter/flutter/issues/19269

What we should do:

either codegen, for example with https://pub.dev/packages/functional_widget
just convert them to stateless/stateful widgets
use a Builder widget

@fzyzcjy I can do this, which would you prefer?

Hmm we are using mobx and thus Observers. Observer is like Builder, and thus IMHO is not slow to do so.

While that is true, there are a TON of functions like
Widget _buildBody(BuildContext context) {
over the code.
It is bad practice as you can see.

Yes I agree. I originally did that because of the simplicity it gives, since you know, convenient_test_manager runs on a desktop, thus this slowdown is never noticeable for humans ;)
Happy to hear that you like to contribute to the codebase! Maybe we can prioritize to work on more critical features first. For example, to have some new feature.

Hey!
I am currently working on the dark theme PR because a lot of colors were hardcoded.
If you give permission, I'd like to start a bigger project to refactor the UI a bit, for example

no functions like this
no hardcoded colors
make it more responsive (see #321 )

I will try to make the diff as minimal as possible.


Fix segment tree

I found critical bug in my implementation. Now it fixed and I checked that by some performance tests. As an apology I added some more illustrations in description :smile: . Also there is mistake in main readme: it's not that segment tree, but this

Cool. I will probably merge it tomorrow or the day after (busy right now).


Correct setup link

The current link to https://kubernetes.io/docs/setup/ presently ends up
redirecting to https://kubernetes.io/docs/setup/. This corrects the link
to what seems to be the correct endpoint.
Signed-off-by: Nathan LeClaire<EMAIL_ADDRESS>

This change is

CLA signed

lgtm
thanks


[Bug] GTCE EU setting does not apply to turbines

I noticed the setting to enable or disable GTCE EU support, and was very happy to discover that I could have a reason for players to use both the fusion reactor from GTCE and the various nuclear power options from NuclearCraft, without needing to balance them (too hard) against each other.
Unfortunately, having constructed a turbine, I seem to still be able to extract power from the NuclearCraft machine and into a GTCE battery buffer.

The cable does not visually connect, but it still transfers power into my buffer below.

This is probably true for all blocks which output energy - will fix!

<3
I made a quick test with the Voltaic Pile, and it did not seem to have the same bug, for what it's worth!

I could not recreate that 'lack of bug' with the voltaic piles, i.e. it was also broken for them, but anyways, it's now fixed ;)

Huzzah! Thank you for your great work, and happy holidays! :)

And to you too!


Emissary 404's request with host:port in Host Header

Describe the bug
Prometheus scrape GET request uses host with port to grab metrics from an endpoint (example-server.com:443/metrics). Emissary does not match the host/authority with port to Mapping with host set to (example-server.com). Same issue as https://github.com/emissary-ingress/emissary/issues/2276
To Reproduce
Steps to reproduce the behavior:

Install Emissary v2.0.0ea with helm
Deploy AmbassadorListener, AmbassadorHost, AmbassadorMapping, TLSContext
Enable scrape in Prometheus config to host behind Emissary like prometheus-internal.example.com
Curl with Host curl https://prometheus-internal.example.com -H'host:prometheus-internal.example.com:443'
Emissary responds with 404

> GET / HTTP/1.1
> Host:prometheus-internal.example.com:443
> User-Agent: curl/7.77.0
> Accept: */*
< HTTP/1.1 404 Not Found
< date: Wed, 21 Jul 2021 18:40:55 GMT
< server: envoy

Emissary logs confirm

Example scrape from Prometheus:
ACCESS [2021-07-21T18:21:43.936Z] "GET / HTTP/1.1" 404 NR 0 0 0 - "<IP_ADDRESS>" "Prometheus/2.25.2" "5d11d9bc-2242-427d-9b57-a032a8beda16" "prometheus-internal.example.com:443" "-"
Example curl:
ACCESS [2021-07-21T19:14:05.737Z] "GET / HTTP/1.1" 404 NR 0 0 0 - "<IP_ADDRESS>" "curl/7.77.0" "b94a32ee-d5bd-4940-8e4a-cb3476acd3f5" "prometheus-internal.example.com:443" "-"

Expected behavior
Emissary to route correctly by handling host:port in host/authority header
Versions (please complete the following information):

Ambassador: v2.0ea
Kubernetes environment: k3s (1.20.7) , EKS (1.19)
Version: 2.0.0-dev.383+g00c1c766b

Additional context
Attempted work around mentioned in https://github.com/emissary-ingress/emissary/issues/2276, none worked except for the Lua script mentioned here. This is fine for testing, but not a solution for multi-cluster deployment.

Is there any update on it ?

We're stuck on this as well.
As mentioned in the description, the only thing that has worked is the Lua script.
Running emissary:2.3.2.
I'm running Ambassador on debug and the strip_matching_host_port config documented in https://www.getambassador.io/docs/edge-stack/latest/howtos/grpc#mappings-with-hosts doesn't seem to do absolutely anything.
I see the :authority header has the 443 port regardless of whether the config is true or false in the conn_manager_impl request headers. I imagine that it has to do with the part where it says "This only applies if the port matches the underlying Envoy listener port." but it's extremely unclear to what this is referring to.
Do you have an example of what that underlying Envoy listener port refers to?


Fix key links in README.md (except Contributing)

Fix main repo and docs links on the top.
NOTE: the "Contributing" link is still broken, because I have no idea where it should point to.

Ah, thank you for this. Was too busy to realize links were broken. As for the CONTRIBUTING file, you can get it from here and adapt it to Adorad, if you'd like.


Release tmt-1.29.0

Follow the steps below to create a new major or minor release:

[ ] Run the full test coverage using tmt -c how=full run
[ ] Use git log --oneline --no-decorate x.y-1.. to generate the changelog
[x] Update README with new contributors since the last release
[ ] Add a Release tmt-x.y.0 commit with the specfile update
[ ] Create a pull request with the commit, ensure tests pass, merge it

Release a new package to Fedora and EPEL repositories:

[ ] Move the fedora branch to point to the new release
[ ] Tag the commit with x.y.0, push tags git push --tags
[ ] Create a new github release__ based on the tag above
[ ] Check Fedora pull requests__, make sure tests pass and merge

Finally, if everything went well:

[ ] Close the corresponding release milestone
[ ] Once the copr build__ is completed, move the quay branch
to point to the release commit as well to build fresh container
images.

Handle manually what did not went well:

If the automation triggered by publishing the new github release
was not successful, publish the fresh code to the pypi__
repository manually using make wheel && make upload
If there was a problem with creating Fedora pull requests, you
can trigger them manually using /packit propose-downstream
in any open issue.
If running packit propose-downstream__ from your laptop make
sure that the post-upstream-clone action is disabled in
.packit.yaml to prevent bumping the devel version.

@psss Attempt in 1.30 release notes, improvements appreciated.

@psss Attempt in 1.30 release notes, improvements appreciated.

Done in 0e66e267. Please, have a look if everything's ok.

@psss Looks good. thanks


Fix typo in ci.yaml

Recommend that you merge this. It'll be reading the PR triggers from the target branch, not the PR so even though you've tweaked the triggers in this change, they aren't being evaluated.

@mitchdenny I'm merging the PR but do you see any error with the ci.yaml in the target branch though? I thought v4* would work too? David and I have spent hours on this and a little help here will be appreciated.

After merging this PR I created another PR at https://github.com/Azure/autorest.java/pull/551. As you can see, with this version of ci.yaml there builds are still not running.


Include comic title in Intent.EXTRA_TEXT

Messengers don't use EXTRA_SUBJECT at all, so I added the comic title to
Intent.EXTRA_TEXT.
I also bumped the gradle plugin to fix the build in AS 3.4.1.
Signed-off-by: mueller-ma<EMAIL_ADDRESS>

Nice :)


Give measurements a meaningful name

Something like 'Spectrometer Name - Sequence Name - Timestamp'

Fixed with ae32a95205948eb0bcff8a742e1d8c2668e846d6

Upsi wrong issue.


Importing a new city, howto, what information is needed?

Hey, I found https://github.com/dabreegster/abstreet/issues/119 this issue and its quite long so i wanted to ask in general if there is a checklist of howto or what to do to import a new map. I'm interested in Salzburg, for example, its not that big but it should be really good mapped because there are a lot of mappers around (including me). So it may be a good example as well?
What additional information would be needed?

Hi! https://dabreegster.github.io/abstreet/howto/new_city.html has the instructions. If you're just trying things out, the one-shot command should be fine. If you'd like to collaborate more and get Salzburg running well in A/B Street, then it'd be easier to add it as a supported city, so we can iterate on it more quickly.
If you have trouble running the one-shot commands, let me know and I can do it. The most useful input to start is a bounding polygon for the area you're interested in. You can get a sense from the existing maps in the game about how big this should be; maybe it's the entire area, or maybe it's better to divide into a few pieces (which can overlap).

So, i was partly succesfull. I downloaded a small part via osm.org and then mistakenly opened the .osm file with JOSM. Decided to download via JOSM the rest but it seems the rest is just blacked out with some buildings in there. I guess the JOSM files differ from the direct export somehow?

Starting File (change from txt to osm)
map.txt
File with additional data via JOSM cannot be uploaded because its above 28MB but the only difference i can see is export via page is like <member type="way" ref="102141199" role=""/> and JOSM is <member type='way' ref='102141199' role='' />
Another thing: Via JOSM it would be possible to reduce the downloaded data with an Overpass, if you can tell what is definitely not needed it could be excluded when downloading. Or mention that JOSM is not suited for providing data for the game.
Whatever, Downloaded via https://download.bbbike.org/ Salzburg and imported it with the importer.exe --oneshot=Salzburg.osm and it needed quite much more time than my failed JOSM approach. But it seemed the are covered by this download is basically huge (the map file is now 394MB) and i will try to make it way smaller.
Another thing i noticed right now - my wrong import hat some driving buses around, the bbbox download not - is it possible theree are some routes missing - or how can i find out more about this?

Decided to download via JOSM the rest but it seems the rest is just blacked out with some buildings in there

When you omit --oneshot_clip, then the map boundary is taken from the <bounds minlat="..." ..../> tag in the file. My guess is that JOSM doesn't update this properly when you tell it to download a new area with an existing file. Anything outside the boundary gets clipped out, except parking lots.

Via JOSM it would be possible to reduce the downloaded data with an Overpass, if you can tell what is definitely not needed it could be excluded when downloading.

That is a possibility, but I'm not sure the complexity is worth it. What A/B Street will extract from the .osm could change over time. The main thing that increases .osm file size and isn't needed are the version, timestamp, changeset, author metadata on all of the objects. I think I've seen some OSM tools that support filtering that out.

it needed quite much more time than my failed JOSM approach

Yup, large areas take longer to import. It's not documented yet, but you can also pass --skip_ch to avoid an expensive importing step that builds contraction hierarchies, which speed up route-finding later.

my wrong import hat some driving buses around, the bbbox download not

Extracting the bus routes and building the paths for them is actually broken right now; I need to finish #190 still. A few routes might get imported, but it doesn't have much to do with your choice of .osm extract.

What information is now needed to populate the city?

The most interesting information would be a travel demand model, which tells you how many people travel between different buildings (or parcels of land, or small regions -- depending on the agency producing the model), when they leave, how likely they are to drive vs bike vs walk vs take transit. It's been quite hard to find public data for this. An intermediate that would help is some sort of census, showing how many people live in different chunks of the city, and ideally some attributes about how many of them own cars, take transit, etc. Even without any of this, you can still start traffic on the map by changing the "Traffic" dialog at the top to "trips from home to work" -- this will just infer residences vs businesses from OSM tags.
One way to use A/B Street to find things to map is by looking for street parking. A few roads are tagged (this is from the parking layer, press l and then p):

https://dabreegster.github.io/abstreet/howto/map_parking.html has some info on this.
I imported the entire bbike area and glanced at it. Two things jump out that're bugs in A/B Street I could work on:

Kapuzinerberg and other large parks aren't being picked up. Looks like https://github.com/dabreegster/abstreet/blob/1e0d8536a4cc51a363c068dd1f8f4a3971097426/convert_osm/src/extract.rs#L522 should map landuse=forest to AreaType::Park, just to render it green.

Residenzplatz uses pedestrian areas. A/B Street doesn't support these yet, sadly. So you'll probably notice all of the buildings have driveways/paths connecting to the nearest road, and that the paths just cut right through other buildings.

I can fix 1) later this week. Next steps depend on your goals. Are you looking to use A/B Street as Q/A testing to find parts of Salzburg that have lanes, street parking, turn restrictions, etc mis-tagged? Are you interested in exploring some proposal to pedestrianize some roads or make new bus-only lanes or something like that?

Closing this issue for now, because I don't think there's anything left to do. Feel free to reopen if you have some more steps in mind!

What information is now needed to populate the city?

The most interesting information would be a travel demand model, which tells you how many people travel between different buildings (or parcels of land, or small regions -- depending on the agency producing the model), when they leave, how likely they are to drive vs bike vs walk vs take transit. It's been quite hard to find public data for this.

Can you give me a hint on what to search for? I found https://www.researchgate.net/publication/326175780_Travel_Demand_Modelling_of_Touristic_Trips_in_the_Province_of_Salzburg this but this is for the surrounding and not for the city itself. It seems like something like this will be provided via https://www.bmk.gv.at/themen/verkehrsplanung/verkehrsmodell.html but that would need some more time.

An intermediate that would help is some sort of census, showing how many people live in different chunks of the city, and ideally some attributes about how many of them own cars, take transit, etc. Even without any of this, you can still start traffic on the map by changing the "Traffic" dialog at the top to "trips from home to work" -- this will just infer residences vs businesses from OSM tags.

At least the first part is possible via https://www.stadt-salzburg.at/statistik-bevoelkerung/ and "Bevölkerung nach Statistischen Zählbezirken" - https://www.stadt-salzburg.at/fileadmin/landingpages/statistik/einwohner/2020_bevoelkerung_nach_zaehlbezirken.pdf but how do i need to provide this information to abstreet? I may even ask the officials for it.
If i can acquire at least the how many use transit, cars, bike and so on, would it be possible to get Salzburg into abstreet or just if somewhat accurate travel demand data is available?

Can you give me a hint on what to search for?

I've only managed to find public data in Seattle, so I'm not sure what to search for. "Travel demand model", "origin/destination table", and "activity model" are some terms.

this but this is for the surrounding and not for the city itself

If I understood the abstract, " This paper covers the integration of tourists in an already existing macroscopic transport model of Salzburg." maybe suggests there's already a model of the city itself. If you can get in touch with the authors, they might be able to help.

but how do i need to provide this information to abstreet?

There's no standard format yet. We started something similar for Berlin -- see https://github.com/dabreegster/abstreet/blob/master/importer/src/berlin.rs. There's one file with a boundary polygon for each planning region, and another CSV file that describes number of people per region, and some code to randomly distribute N people over however many buildings happen to be in that polygon. I haven't run the table PDF through a translator yet, but I'm guessing things like 24 -Äußeres Nonntal / Freisaal are areas. Do these have polygons to bound them?

If i can acquire at least the how many use transit, cars, bike and so on, would it be possible to get Salzburg into abstreet or just if somewhat accurate travel demand data is available?

It recently got a little easier for me to manage more cities in abst, so we could add it in now, and I can maintain the map as the binary format changes. We can keep working on finding input for the travel demand model. How about to start, could you send a geojson.io polygon, or maybe even a few covering different parts of the city? Chopping it up is useful for smaller file sizes. I chopped up Paris arbitrarilyish like this, for example:

I haven't run the table PDF through a translator yet, but I'm guessing things like 24 -Äußeres Nonntal / Freisaal are areas. Do these have polygons to bound them?

Yes, you are correct, these are areas. There is Nonntal for example https://www.openstreetmap.org/relation/7489976
Guess thats what you mean? I can gather it together so it all fits the polygon.

It recently got a little easier for me to manage more cities in abst, so we could add it in now, and I can maintain the map as the binary format changes.

That would be awesome

We can keep working on finding input for the travel demand model. How about to start, could you send a geojson.io polygon, or maybe even a few covering different parts of the city? Chopping it up is useful for smaller file sizes.

Will do and look it up for you. And yes, i will chop it up, its pretty easy for salzburg as there is the river, the Autobahn and the border to chop it up nicely. Not much entry-points as well. Tell me if this works or if it should overlap more
yellow - North Salzburg
green - East Salzburg
teal - South Salzburg
purple - West Salzburg
{
  "type": "FeatureCollection",
  "features": [
    {
      "type": "Feature",
      "properties": {
        "stroke": "#555555",
        "stroke-width": 2,
        "stroke-opacity": 1,
        "fill": "#ffff80",
        "fill-opacity": 0.5
      },
      "geometry": {
        "type": "Polygon",
        "coordinates": [
          [
            [
              13.02952766418457,
              47.82923348731677
            ],
            [
              13.035364151000977,
              47.81891816173151
            ],
            [
              13.036651611328125,
              47.81050302635426
            ],
            [
              13.036479949951172,
              47.80779375424168
            ],
            [
              13.038883209228514,
              47.80358545562179
            ],
            [
              13.043603897094727,
              47.80099112906964
            ],
            [
              13.0642032623291,
              47.80779375424168
            ],
            [
              13.07192802429199,
              47.81672805230014
            ],
            [
              13.072786331176758,
              47.819148694192286
            ],
            [
              13.059568405151367,
              47.83211449691779
            ],
            [
              13.038454055786133,
              47.830904492365704
            ],
            [
              13.02952766418457,
              47.82923348731677
            ]
          ]
        ]
      }
    },
    {
      "type": "Feature",
      "properties": {
        "stroke": "#555555",
        "stroke-width": 2,
        "stroke-opacity": 1,
        "fill": "#80ff80",
        "fill-opacity": 0.5
      },
      "geometry": {
        "type": "Polygon",
        "coordinates": [
          [
            [
              13.045148849487305,
              47.800702862566204
            ],
            [
              13.053388595581055,
              47.79839667295524
            ],
            [
              13.061628341674805,
              47.794245273679884
            ],
            [
              13.07046890258789,
              47.79666696356607
            ],
            [
              13.081712722778319,
              47.80052990189636
            ],
            [
              13.08222770690918,
              47.81482604021226
            ],
            [
              13.068580627441406,
              47.8142496591576
            ],
            [
              13.042402267456055,
              47.80179826676978
            ],
            [
              13.045148849487305,
              47.800702862566204
            ]
          ]
        ]
      }
    },
    {
      "type": "Feature",
      "properties": {
        "stroke": "#555555",
        "stroke-width": 2,
        "stroke-opacity": 1,
        "fill": "#80ffff",
        "fill-opacity": 0.5
      },
      "geometry": {
        "type": "Polygon",
        "coordinates": [
          [
            [
              13.036994934082031,
              47.80289364787613
            ],
            [
              13.029356002807617,
              47.80127939397355
            ],
            [
              13.020172119140625,
              47.77475232696755
            ],
            [
              13.041114807128906,
              47.77261803978678
            ],
            [
              13.044204711914062,
              47.77411781830723
            ],
            [
              13.060083389282227,
              47.775732916031856
            ],
            [
              13.075790405273438,
              47.77752100142527
            ],
            [
              13.075962066650389,
              47.78219280527603
            ],
            [
              13.074760437011719,
              47.785364778461314
            ],
            [
              13.067636489868164,
              47.79084318534558
            ],
            [
              13.056328296661377,
              47.79705615319922
            ],
            [
              13.052723407745361,
              47.798771435733016
            ],
            [
              13.043968677520752,
              47.801019955632015
            ],
            [
              13.040385246276855,
              47.80257656624696
            ],
            [
              13.036994934082031,
              47.80289364787613
            ]
          ]
        ]
      }
    },
    {
      "type": "Feature",
      "properties": {
        "stroke": "#555555",
        "stroke-width": 2,
        "stroke-opacity": 1,
        "fill": "#ff80ff",
        "fill-opacity": 0.5
      },
      "geometry": {
        "type": "Polygon",
        "coordinates": [
          [
            [
              13.002233505249023,
              47.85360178173402
            ],
            [
              13.001375198364258,
              47.84156308579322
            ],
            [
              12.989187240600584,
              47.83355494174212
            ],
            [
              12.983779907226562,
              47.828714888603244
            ],
            [
              12.972965240478516,
              47.8116558652381
            ],
            [
              12.96180725097656,
              47.79839667295524
            ],
            [
              12.95107841491699,
              47.791592817160414
            ],
            [
              12.948246002197264,
              47.78709486407255
            ],
            [
              12.943696975708008,
              47.78697952682338
            ],
            [
              12.938375473022461,
              47.784269027789286
            ],
            [
              12.93966293334961,
              47.78075093529741
            ],
            [
              12.938804626464844,
              47.77809779004784
            ],
            [
              12.934856414794922,
              47.77665580649395
            ],
            [
              12.929878234863281,
              47.77382940272034
            ],
            [
              12.926273345947266,
              47.76725309331018
            ],
            [
              12.928504943847656,
              47.76529157687734
            ],
            [
              12.967815399169922,
              47.76736847432615
            ],
            [
              12.990045547485352,
              47.775444509403115
            ],
            [
              13.020343780517576,
              47.77342561821119
            ],
            [
              13.022918701171875,
              47.776482765780024
            ],
            [
              13.03105115890503,
              47.800601968912105
            ],
            [
              13.04051399230957,
              47.80207211421171
            ],
            [
              13.037552833557129,
              47.805559939554506
            ],
            [
              13.036072254180908,
              47.808629608585626
            ],
            [
              13.03675889968872,
              47.81426406876196
            ],
            [
              13.035686016082764,
              47.81888934510192
            ],
            [
              13.029892444610596,
              47.829060621654705
            ],
            [
              13.002233505249023,
              47.85360178173402
            ]
          ]
        ]
      }
    }
  ]
}

Thanks for the boundaries; they work great! If you're running from git and pull the latest commit, you can use the "load map" control in the game, go to "download more cities", and bring in Salzburg.
@michaelkirk and I are in the process of generalizing the traffic demand generation in #424. The tables you found can ultimately get turned into CensusAreas and plugged into this new approach. It might be a few more days or weeks of churn to get the new approach working. In the meantime, you can use the "home to work" pattern to try things out.

Ah, i'll just wait for the next update :)
So, now i will try to get information about the travel demand data and i will wait for your information on how i can(or should) input the data about how much people live in what district in Salzburg. (and later the travel-demand-model)


Proposal: example.c.wren files

Instead of using something like generate_builtins in upstream, I'd rather do something like have filename convention to translate files into the C forms, just by wrapping the files. These would be normal wren files, but they would be compiled into a C file, and then compiled along with the interpreter, similar to how the builtins are, but much more scalable.

I did this, but I'm not happy with it. It makes building complicated.


Fix fix shrink_memtable_by_agg should also update _row_in_blocks

Proposed changes
#28660
#28536
Issue Number: close #xxx

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall

run buildall


Is there any way to get the size of the TFRecordDataset?

Please make sure that this is a feature request. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:feature_template
System information

TensorFlow version (you are using): tensorflow-2.0.0-alpha0
Are you willing to contribute it (Yes/No): Yes

Describe the feature and the current behavior/state.
I get some tfrecord files from someone. when I try to train the network with tf.keras.Model.fit(), it needs to pass a parameter "steps_per_train", but I don't know the size and how many data was saved in the tfrecord files. I have read the document of tensorflow2.0 but I find nothing help, so is there any way to get the size of the TFRecordDataset? Thank u
Will this change the current api? How?
No, it won't change the current api. It may add an attribute with a "get" function.
Who will benefit with this feature?
Every who need to know the size of tfrecord dataset
Any Other info.

No, the only way to find out the cardinality of TFRecordDataset is to enumerate all of its elements.