Implement tagfiles provider

Closes #446
I chose to implement a separate provider (tagfiles) instead of improving proj_tags because the behavior & logic is too different and it's going to end up messy to try to keep both behaviors in a single provider.
I had some trouble understanding the logic of how vimscript & rust talk to each other so there are some hacks in this PR, I'll clean up before it's merged. It's also my first project with rust so the compiler and myself have been fighting all night. If you have any advice or see anything wrong with the rust code don't hesitate to point it out.
Tasks:

[x] Make the query only match in the tag name, not the whole line
[x] Ensure paths are shortened & truncated correctly
[x] Cleanup vim-rust communication
[x] Handle jumping to tag correctly, currently there might be conflicts if different tags have the same name
[x] Decide if we want to display tag's kind (either as text or as icon)
[x] Consider paths relative to tag file
[x] Shorten rendered paths if necessary
[x] Test parser correctness
[x] (?) FZY matches exact strings incorrectly
[x] Remove unrelated commits (refactoring)
[ ] Options: filter by language (aka file extension)

I've also been doing some gentle refactoring on the vimscript side, and I may need to refactor a bit the rust side to decouple line searching and line rendering.

When I start searching in a 30mb tags file, after entering the first key, CPU load jumps to 100% and stays there for quite a while. The input becomes unresponsive and the chars show up one after one. The delay is several seconds.

Got it, we probably need to be more lazy in the data exchange, or implement paging. I'll build some ginourmous tag files and check how to improve the performance before merging.

I tried the latest commit, it's much better!

Yes, the async workflow was already implemented I just didn't put it in place correctly because I don't know my way around the codebase yet :angel:
This PR is mostly implemented, I still need to finish the handling of paths relative to their tagfile but if you use a tagfile at ./tags it should be working fine. Please test it and report any bugs that you find.
I've implemented many hacks due to the rigidity of the vim-rust communication. First one is that instead of making the filtering algorithm take an extracter function to get the slice of text it needs to filter like I had envisionned, I simply made the TAGS_RE regex handle tags, proj_tags and tagfiles :/
The second one is that I need to send back to vim different informations: the rendered line, the file of the tag, and the pattern of the tag (to locate it within the file). However, the current protocol is limited to sending back a JSON object like { "lines": [...}, "indices": [...] }. I didn't want to touch it because the architecture is very coupled and it needed more refactoring that I cared to do. So I just slapped a :::FILENAME:::PATTERN at the end of the rendered line and I split(line, ':::') it when I need the info back :laughing: I use the syntax to conceal the :::... part. I'm not proud but the first objective is to make it work.

 It isn't supposed to fail because the input of tag_name_only is controlled by us. I've simply made it return the whole string if it fails because it's hard to debug which of the entries (that you didn't see) caused this panic. So if duringa tag search you see more than the tag name being searched in (highlighted), please take a screenshot of that and report it so I can fix it :)

Awesome, could you drop me a copy of that tag file here?

So I've finished implementing path handling, which means that paths in tagfiles are now considered relative to where the tagfile is, which means that jumps work as expected. Any number of tagfiles can be fed to maple.
I've also added a global --cwd option that is passed to maple. This is the current directory in vim. It is used so maple can shorten paths to make them relative to where the user is. If that fails, paths are made relative to the user home directory (with a ~). If that fails, they're rendered as absolute.

(test with tags coming from different files, with a mix of asbolute & relative paths; all is well)
@hasufell I've haven't received any news :/ If you missed my comment above I requested if you could send me that tagfile that fails so I can debug the issue you're experiencing.
@liuchengxu There is one detail still bothering me (well for this PR), I've seen there is thic concept of "forerunner" in place, and even though I get the general idea of what we might want to do with that, the way tagfiles is setup (used proj_tags as a template), the forerunner process doesn't seem to do work that is used by the other processes, well at least when it's a huuuuuge (500mb) tagfile. It just keeps running and doesn't seem useful. I've also seen many different return values (g:__clap_forerunner_tempfile, g:__clap_forerunner_result, and nothing, which starts another maple) and it seems a bit much. I was wondering how you had envisionned all this and why there are so many options. If you would be able to explain a bit how it's supposed to work and how it could be improved I would like it very much :)

I've haven't received any news :/

Sorry, was swamped with other stuff. I just uploaded a file

The forerunner job is spawned for collecting the whole results eagerly if source of provider is actually a shell command like fd --type f that needs to be executed in the terminal.
The forerunner job returns:

the list of total lines if it's less 100000. https://github.com/liuchengxu/vim-clap/blob/b1f4e1d400209c3c97898b9b4caf577f93943923/crates/maple_cli/src/cmd/exec.rs#L25
if the number of results lines is larger than 100000, they will be wrote to a cache file and vim will receive the cached file name. Next time when user inputs some query, clap will directly do the filtering on the cached file instead of spawning source cmd again.

You are right, it's unnecessary to spawn the foreruner job for tagfiles provider as the whole results are already there, we just need to do the filtering on the tagfiles. It's more like blines provider that querys against the current buffer which is in fact a file. You can implement init property in tagfiles provider explicitly instead of using the default impl so that forerunner won't be started.
https://github.com/liuchengxu/vim-clap/blob/b1f4e1d400209c3c97898b9b4caf577f93943923/autoload/clap/provider/blines.vim#L45
One improvement for forerunner job I can think of is to stop it properly. Currently the whole output will be cached in the memory that could be problem in some corner cases that you runs a crzay command, in which case we should terminate the forerunner job in time on the Rust side.
https://github.com/liuchengxu/vim-clap/blob/b1f4e1d400209c3c97898b9b4caf577f93943923/crates/maple_cli/src/light_command.rs#L161-L172
@romgrk Not sure if I make myself clear. Let me know if you have any questions.

Ok, so there are still a few things that need to be tested but basically the initial implementation is complete. The problem is I'm not satisfied with how things are. :(
Currently, how it works is as follow: when the user inputs a query, read the whole tag files, score each entry and return those that match. Whenever the query is updated, scratch the previously running process and start anew. It seems a bit brutal. Maybe I haven't understood some things correctly, but is this how the other similar providers are running?
I've been trying out the tagfiles provider in repos like the neovim source, with 24,000 entries, and it takes a few seconds before anything shows up. I would have expected things to be more performant. And the thing is I've tested the original C fzy on the command line, and when I search for something in that same 24,000 entries tagfile the search is, as far as my brain can tell, instant. This is what I expect from a fuzzy finder, not the current behavior. I will be investigating further how to improve because it's obvious that something is wrong with the current implementation.

@hasufell I couldn't find the entries listed in your screenshot in the tagfile you sent me :/ Could you check to include some tags that reproduce the problem illustrated? (eg I've cat tags | grep PACKAGE_VERSION but didn't find anything)

I've been trying out the tagfiles provider in repos like the neovim source, with 24,000 entries, and it takes a few seconds before anything shows up

It's probably because you are running some loop in vimscript, basically all the heavy jobs should be done in Rust, and vim only  receives the final results without doing any processing but displaying them. I can take a look later, blines provider is a better template for tagfiles.

Yes, there is probably something looping. I'll look into it in the next days.

It could be a natural disk latency, not some vimscript loop. Just saying.

Lol well that wasn't hard to find.

Ok ok, so I think I'm mostly done now.
As indicated by my last comment, performance issues were due to me trying to use (complex) regexes as string parsers which is always a bad idea. Hopefully I'll remeber now. However, to fix that issue I had to introduce something a bit hacky; the part of maple that fetches items can only provide a list of String to the filterer, without additional metadata. To know where to parse, I've introduced a separator token (::::) that marks starts/ends of locations in the strink. The tag_name_only parser now uses at first. As both tagfiles and proj_tag go through the same branches, I've updated proj_tags to use it as well. I don't like it, but the other solution is to try to refactor maple so that fetching & filtering can exchange metadata and I have been fighting with rustc way too much right now.
The problem with tag_name_only previously is that tag names, even if in theory they shouldn't contain spaces, contains spaces in practice (even universal-ctags does it), quite unsuprinsigly. The only character that I'm sure can't be in the tag name is the tab character. Tag names can be surprising. I even worry that Haskell operators could be ::::. I'm sure it's possible but let's ignore that possibility for now.
I have also implement a more robust tag line parser. Most of the fields are pretty easy to parse, but the tagaddress is the most problematic one because it can contain escaped characters, so we need to do real parsing instead of just finding start/end indexes with tabs. I've added many tests coming from different tag tools: universal-ctags, scss-ctags, and hasktags (@hasufell I couldn't install codex and if you see my comment above your tagfile didn't seem to contain the invalid lines...)
There is one detail that I've left out, maple's tagfiles subcommand doesn't handle the --languages subcommand, which would be nice to handle. Usually I know for which language I'm looking a tag for and that can narrow the search. OTOH I'm low on energy, and maple is performant again due to the fix above, so I haven't felt annoyed with speed even in 25k+ tags directories. I'd maybe keep it asa future improvements.
Comments welcome.

the part of maple that fetches items can only provide a list of String to the filterer, without additional metadata.

That can be a good direction.

introduced a separator token (::::) that marks starts/ends of locations in the strink.

Hmm, not a fan of ::::. But I'm also not sure if the way of vista.vim is robust enough https://github.com/liuchengxu/vista.vim/blob/bcb50336188074c9cfc1aad4b463f4c0e698a68c/autoload/vista/parser/ctags.vim#L91-L92

" {tagname}<Tab>{tagfile}<Tab>{tagaddress}[;"<Tab>{tagfield}..]
" {tagname}<Tab>{tagfile}<Tab>{tagaddress};"<Tab>{kind}{scope}

Basically only tagaddress is not easy to parse, maybe we can simply parse out tagname and tagfile and then deal with the rest fields. @masatake Any tips for parsing these fields from the tagfile?

That can be a good direction.

Yes that's how I would envision things for the future but it will require a substantial refactoring. During this PR I at some point started such refactoring but I ended up lost in a confusion of types (because it's my first rust project). Not sure how long that may take.

Hmm, not a fan of ::::

Me neither. But I couldn't see any intermediary steps between that and the refactor. I'm open to propositions.

" {tagname}{tagfile}{tagaddress}[;"{tagfield}..]
" {tagname}{tagfile}{tagaddress};"{kind}{scope}
Basically only tagaddress is not easy to parse, maybe we can simply parse out tagname and tagfile and then deal with the rest fields.

For your 2 examples, as long as tagaddress is valid, my parse will work (I'll test it to be sure). There is quite a bit of variation in what characters come after tagadress, but actually all we need to do is to first parse the valid address, then skip any non-tab characters that come at the end.  IMHO the tag parser is acceptable as it is, it can always be adjusted as bugs are reported.

I'm bad in English. So I read only the part of this pull request. I will answer happily if you summarize your issue and open an issue for summarized one at universal-ctags/ctags repo. In addition I don't know vimscript at all. I'm an Emacs user.
" {tagname}<Tab>{tagfile}<Tab>{tagaddress}[;"<Tab>{tagfield}..]
" {tagname}<Tab>{tagfile}<Tab>{tagaddress};"<Tab>{kind}{scope}

Basically only tagaddress is not easy to parse, maybe we can simply parse out tagname and tagfile and then deal with the rest fields. @masatake Any tips for parsing these fields from the tagfile?

My first question is "do you need the value of {tagaddress}"?
If you don't need it, you should not generate it.
With -n option you can suppress it:
$ cat foo.c
cat foo.c
int
main(void)
{
	return 0;
}
$ ./ctags -o - foo.c
./ctags -o - foo.c
main	foo.c	/^main(void)$/;"	f	typeref:typename:int
$ ./ctags -o - -n foo.c
./ctags -o - -n foo.c
main	foo.c	2;"	f	typeref:typename:int

The regex pattern for the tagaddress becomes simple: [0-9]+.
If you need it and if you need precise result, you must parse  {tagaddress} carefully.
See https://docs.ctags.io/en/latest/man/tags.5.html.
If you can accept false-positive-alike result, you can use following pattern:
([^\t]+)\t([^\t]+)(.*)\$/(;"\t(.*)$)?

Here
\1 -> tagname, \2->tagfile, \3$/"->tagaddress, \4->rest fields.

@hasufell I couldn't install codex and if you see my comment above your tagfile didn't seem to contain the invalid lines...

@romgrk There are no invalid lines. My point was that it searches both the filepath and the identifier. But I haven't tested the latest commit.

I tried the latest commit and it doesn't really work for me. I don't even know where it searches, but it can't find regular identifiers. When I use :ts <ident> then I get correct results though, so the tags file is correct.

I will answer happily if you summarize your issue and open an issue for summarized one at universal-ctags/ctags repo.

Sure, just opened https://github.com/universal-ctags/ctags/issues/2683 :)

If you can accept false-positive-alike result, you can use following pattern:
([^\t]+)\t([^\t]+)(.)$/(;"\t(.)$)?
Here
\1 -> tagname, \2->tagfile, \3$/"->tagaddress, \4->rest fields.

We need the tagaddress and the regex won't work for us as we don't support just universal-ctags. The current parse accepts the following. I've reviewed the man page for tags, and IMHO our parser is complete enough. Here are the different address that are accepted by the parser:

pattern such as /function main()/ plus any non-tab chars (such as ;" or [;")
pattern such as /^\tfunction main() \/\/ comment$/ plus any non-tab chars
line numbers such as 83 plus any non-tab chars
One thing in the man page not accepted by the parser would be multiple search pattern such as:
/struct foo/;/int x/;
I think accepting multiple search patterns add unnecessary complexity, and I haven't seen anyone generating such addresses in practice. It's not that much more work but IMHO not necessary. Let me know what you think.

@hasufell Could you add the run it with the following path? It will place the search command (including tagfiles paths) in your clipboard. You could also run the command directly in your terminal to check the output. I'd be happy if you could report any issue that you find.

patch.txt

diff --git a/autoload/clap/provider/tagfiles.vim b/autoload/clap/provider/tagfiles.vim
index 9e1344c..741d634 100644
--- a/autoload/clap/provider/tagfiles.vim
+++ b/autoload/clap/provider/tagfiles.vim
@@ -53,6 +53,7 @@ function! s:update_query()
   let args += ['tagfiles', g:clap.input.get()]
   let args += map(tagfiles(), {_, f -> '--files=' . f})
   let cmd = call(function('clap#maple#build_cmd'), args)
+  let @+ = cmd
   call clap#filter#async#dyn#start_directly(cmd)
 endfunc

Sorry about the huge commit it's not self-consistent otherwise. Here are the changes:

Sources now return a iterator over SourceItem { display: String, filter: Option<String> } instead of String. The optional filter field is filled with what needs to be matched in, and display represents the rendered line. For now only tag providers use filter
There were cyclic dependencies problems so I extracted commonly re-used types out of printer and into their own crate, types. Which I noticed after that it conflicted with stdio_server's types which has been renamed server_types.
I've done it as simply as I could because I still have some trouble with Rust borrow-checker. There are a few XXX comments in places where I think cloning should be avoidable but am not sure how to do so. (ping @ImmemorConsultrixContrarie ?)
The refactor feels very good. I was able to eliminate the tag_name_only function all its friends could probably be eliminated ruthlessly.

line numbers such as 83 plus any non-tab chars

There is a case violating the above assumption (in u-ctags output).
$ cat /tmp/bar.c
int main (void) {	return 0; } /* -> \treturn */
$ ./ctags --excmd=combine -o - /tmp/bar.c
main	/tmp/bar.c	0;/^int main (void) {	return 0; } \/* -> \\treturn *\/$/;"	f	typeref:typename:int

The refactor feels very good. I was able to eliminate the tag_name_only function all its friends could probably be eliminated ruthlessly.

@romgrk I like the direction 👍 , it's definitely making vim-clap extensible, but can we extract these refactorings into some smaller PRs to make the review easier? For example, We can make the types refactoring a seperate PR, and then we can first experiment the tagfile parser in the current proj_tags, which can be another PR. After that, we can focus on the new tagfiles provider integration.

@romgrk I like the direction +1 , it's definitely making vim-clap extensible, but can we extract these refactorings into some smaller PRs to make the review easier?

I'd be happy to but as far as I could see it was less optimal.

For example, We can make the types refactoring a seperate PR, and then we can first experiment the tagfile parser in the current proj_tags, which can be another PR. After that, we can focus on the new tagfiles provider integration.

To experiment in both tagfiles and proj_tags we needed a way to match the tag_name without the :::: separator. This was impossible other than the last commit refactoring. The previous implementation (regex matching) wouldn't have worked because tag names can contain whitespaces, digits and colons, all of which would have confused tag_name_only.
The types refactoring was necessary to the above refactoring because the types were defined in the printer crate but they were needed in all crates. This refactoring imho simple enough, it was a copy-paste of these lines into their own crate. The server_types refactor is unfortunate though, but happened mid-way through after renaming lots of printer:: to types::. I can reverse the server_types refactor if you insist and have a good name for my types crates. This name needs to reflect the love I have for it.
Separating this PR in different PRs also has disadvantages, for examples proj_tags and tagfiles would need different extractor, which means we would need to keep tag_name_only and also create more applier functions in matcher's line_splitter.rs, and also I think creating another LineSplitter symbol throughout the codebase.
Well I hope my arguments for why it's not desirable to split this PR have convinced you, otherwise let me know I will review for how it could be split. Also if your hesitations are partly due to the robustness of the parse I have no problem improving it, adding tests, and researching the different tag formats. I will address the u-ctags issue mentioned above.

@romgrk @ImmemorConsultrixContrarie I have made a PR #583 about the fetching&filtering metadata direction, I'm also not the fan of SourceItem way. When we want to display or match something special, use a special matcher then. Feel free to leave any comments.

Already saw :) Left a comment directly on that PR, we can continue the discussion there for the matchers architecture.

Not sure if you still have some interest in this PR, it's probably a pain to resolve the conflicts now -_-, you might want to start over on top of the latest master. I'm happy to help push it forward if you still have a plan to work on it. @romgrk

Not sure if you still have some interest in this PR, it's probably a pain to resolve the conflicts now -_-, you might want to start over on top of the latest master. I'm happy to help push it forward if you still have a plan to work on it. @romgrk

TBH I've been using coc-clap for symbol-jumping lately, and I am low on time these days :/
I'll close it for now as it would indeed be too much work to resolve conflicts, but we can always use some of it later if the tagfiles is implemented.

@romgrk Totally understand it. From my point of view, proj_tags is actually already fast enough.

That's a pretty bad sign for the contribution workflow to let a PR like this go stale. I'm still stuck on this branch.

Unfortunate but open source development is limited by the time and interest of contributors :| You're welcome to take over the PR.

Unfortunate but open source development is limited by the time and interest of contributors :| You're welcome to take over the PR.

I would, but I'm having health issues that limit the time I can code. Plugins that improve ergonomics are key for me.
What I meant with contribution workflow is this: On any of my projects, I'd never introduce major changes to master while a PR as big as this is still open. I'd either ping the author of the PR until it's done or take over the remaining work.
Avoiding major merge conflicts is up to the maintainer of the repository, imo, not to contributors. It's hard to have contributors in open source at all. Difficulties with getting things merged just scares them off.


Added support for PyErr_WriteUnraisable

Fixes #2884

@mitsuhiko thanks again for this - I'll give it a quick rebase & fixup this morning so that hopefully it passes CI and merges. 👍

bors r+

I've rebased and rewritten the test to not use py.run.

bors r+


Embulk stop on dupplicate record but not on wrong column data

I have following pattern when import csv data to Postgresql
Pattern#1 Record insert dupplicate
=> Embulk show me error and all records rollback
Pattern#2 Record has wrong data type, Example I set character "A" in a numeric column
=>  Embulk show nothing and all records inserted, the wrong data column is set to NULL value
Question:
Are there any way to continue the process in case of Pattern#1? I mean it's will ignore the dupplicate records and continue with the new one?
I see the the stop_on_invalid_record is false by default, but seem it's only relate data type issue, not constraint problem.

Hello, @Ryo51289
You need to read an issue template before post. Does this question relate to the core?
I can insert the following duplicate entry(multiple id 2 entry) with the following configuration.
in:
  type: file
  path_prefix: hoge/csv/sample_
  parser:
    charset: UTF-8
    newline: LF
    type: csv
    delimiter: ','
    quote: '"'
    escape: '"'
    null_string: 'NULL'
    trim_if_not_quoted: false
    skip_header_lines: 1
    allow_extra_columns: false
    allow_optional_columns: false
    columns:
    - {name: id, type: long}
    - {name: account, type: long}
    - {name: time, type: timestamp, format: '%Y-%m-%d %H:%M:%S'}
    - {name: purchase, type: timestamp, format: '%Y%m%d'}
    - {name: comment, type: string}
#out: {type: stdout}
out:
  type: postgresql
  host: localhost
  user: user
  password: pass
  table: dup_test
  database: embulk_test
  mode: insert

1,32864,2015-01-27 19:23:49,20150127,embulk
2,14824,2015-01-27 19:01:23,20150127,embulk jruby
2,14824,2015-01-27 19:01:23,20150127,embulk jruby
3,27559,2015-01-28 02:20:02,20150128,"Embulk ""csv"" parser plugin"
4,11270,2015-01-29 11:54:36,20150129,NULL

embulk_test=# select * from dup_test;
 id | account |          time          |        purchase        |          comment
----+---------+------------------------+------------------------+----------------------------
  1 |   32864 | 2015-01-28 04:23:49+09 | 2015-01-27 09:00:00+09 | embulk
  2 |   14824 | 2015-01-28 04:01:23+09 | 2015-01-27 09:00:00+09 | embulk jruby
  2 |   14824 | 2015-01-28 04:01:23+09 | 2015-01-27 09:00:00+09 | embulk jruby
  3 |   27559 | 2015-01-28 11:20:02+09 | 2015-01-28 09:00:00+09 | Embulk "csv" parser plugin
  4 |   11270 | 2015-01-29 20:54:36+09 | 2015-01-29 09:00:00+09 |
(5 rows)

embulk: 0.9.23
java: 1.8.0_251
OS: macOS 10.15.7
embulk-output-postgresql: 0.9.0

@hiroyuki-sato
Thank for your confirmation.
Sorry about mistake about explaination.
My case is dupplicate error by unique contrainst. Your example have no error because no contrainst setting, I think.
I will tried with mode:'replace' to see what happen.

Wel, this should be mention in output-


Removes any remnants of TieredSmt

Follow-up to: https://github.com/0xPolygonMiden/crypto/pull/277

@bobbinth fixed the file given today's new clippy. Can you review that too?


Unity3D, Very high GC memory allocation 32kb/frame when using unity RayCast

Hi. There is high memory allocation, when using "unity RayCast" mode and Occlusion method set to Partial in Steam audio source.

I think it happens because of passing delegate to native dll.

The Unity ray tracer's Physics.Raycast function results in GC allocation every time rays are traced. Starting with Unity 2017.1, you can use the function Physics.RaycastNonAlloc to trace rays without causing any GC allocation. If you are using Unity 2017.1 or later, you can try modifying the file Assets/SteamAudio/RayTracer.cs and replacing all calls to Physics.Raycast with calls to Physics.RaycastNonAlloc. Let us know if this helps resolve the issue.

didn't help i even commented out Physics.Raycast all together returning false.
i think its because when SteamAudio engine calls AnyHit it creates those arrays float[] rayOrigin, float[] rayDirection, to avoid GC you could use structs or just passing x,y,z floats directly
  public static void AnyHit(float[] rayOrigin,
                                  float[] rayDirection,
                                  float minDistance,
                                  float maxDistance,
                                  ref int hitExists,
                                  IntPtr userData)

i uploaded demo scene, https://drive.google.com/file/d/1sda9FFsX6WJekeQYyw8hiuBlA6lB-obd/view?usp=drivesdk
Steps i did  to reproduce

Add SteamAudioListener, SteamAudioCustomSettings, Change RayTrace to unity
Add AudioSource with SteamAudioSource audio, enable Direct Sound Occluson, and set Occlusion Method to partial with radius 10
Run and open profiler

So I was able to repro the issue, and it turns out there are three places where GC allocations happen:

Physics.Raycast. This can be fixed by switching to Physics.RaycastNonAlloc, as I mentioned previously.
.NET marshaling of the float[] rayOrigin and float[] rayDirection parameters to AnyHit, as you mentioned. Unfortunately, this can't be replaced with structs or raw xyz values because the signature must match the C API callback. Instead, the workaround is to accept these parameters as IntPtr instead, and get their values without allocating anything by using Marshal.Copy to copy into a pre-allocated buffer.
Querying the current scene's LayerMask involves a call to GameObject.FindObjectOfType. This can be worked around by caching the value until the scene changes.

These three workarounds have been implemented and it looks like there are no more GC allocations in AnyHit. These changes will be part of the next release of Steam Audio.
Meanwhile, I'm attaching the changed files so you can drop them into your project as a temporary measure if needed. Place these in Assets/SteamAudio in your Unity project.
SteamAudio.zip
Let me know if this resolves the issue, or if you are still seeing GC allocations.

Thanks it works! no GC allocations anymore.

The latest release, Steam Audio 2.0-beta.18, contains a fix for this issue.


Add memory optimizations to default configuration files.

Also, remove some undefs from mbed TLS config to avoid compilation error on platforms that require those.

OK.
I will then document these, instead of adding them to the default.
Looks like, we cannot force these as a default, some platforms will still break. There is no portable way to force mbed TLS to be in minimal config.


Implement ack and nack methods, add javadoc and tests

This PR implements ack and nack methods, adds javadoc and unit tests. System tests will come as soon as pull methods are implemented.

Coverage increased (+0.08%) to 84.374% when pulling 264755c1efae7c88c7a689dfbdda9ca212197cda on mziccard:pubsub-ack-nack into b872399fe0c8a444cc5b52a89fd3baff9cd18aab on GoogleCloudPlatform:pubsub-alpha.

Once I understand (or get feedback from JJ or Arie)  the javadoc story for the repo, I expect to give quite a bit of feedback.
The code and the testing look ok, but I want to spend a few minutes at an earlier time of day re-reviewing to make sure that I believe your tests actually cover things and are helpful.

I think so far we were careful to follow the correct Javadoc guidlines for structure (spaces, tag ordering) and rules (punctuation, case,..). As for the descriptions, I think the focus was on clarity which in some
cases made it less concise. You can find in both the JDK and Guava examples to match either style.
I don't have strong opinion about it, however if you think clarity is not effected or even improved by making it shorter than I think you should go for it.
BTW, Looking at the generated Javadoc it looks like @param descriptions are not enabled.

BTW, Looking at the generated Javadoc it looks like @param descriptions are not enabled.

Arie, what do you mean with this?

Arie, what do you mean with this?

I was wrong. I sampled few places (just to see how things look like per earlier discussion) and unfortunately each of these did not specify @param...
E.g. this and this and this and more...

Our internal guidelines specify that it should be a fragment, not a sentence and that it's ok to not use @param if you mention the value in the paragraph.  It also specifies fewer words (ie. fragments) are better as long as it's clear.
So, I'm going to suggest a few re-wordings to be simpler - we won't go through and change existing stuff, but I think being concise is better for our users.

Not to be picky but let me clarify. Fragments are sequences of words that look like a sentence but are not. The difference from a sentence is that fragments are allowed to contain no independent clauses. An independent clause is a sequence of words that contains a subject and a verb and makes sense by itself (extrapolated from the context).
Acknowledges the given messages for the provided subscription. is already a fragment (it's missing the subject). @lesv suggestion (Acknowledges given messages for provided subscription.) simply removes the articles but does not affect its fragment nature.
For what concerns @param descriptions, according to Oracle's guide, they should start with a noun (hence the suggested @param subscription whose messages are being acknowledged is wrong), possibly preceded by an article.
I don't mind applying changes (even project-wide) if that improve docs readability I just doubt that removing some articles or conjunctions would help users.

Yep - I agree - It's what I've been pondering.   I do dislike several mentions of the same thing for the future.  LGTM

Thanks for the pass @lesv


ObsidianOpen Can't Start a New Session

🐛 Describe the bug
When I try to run :ObsidianOpen on a file, Obsidian will throw an error with the exception in the picture below.

However, when I run :ObsidianOpen on a file while an existing Obsidian session is running, it works fine. I am on EndeavourOS running the AUR version of Obsidian (sourced from AppImage). Does this seem like a system problem, an Obsidian problem, or an obsidian.nvim problem? Is there a way to circumvent this easily, i.e. try to start an Obsidian session in the specified vault upon failure and then re-run the obsidian bin command with the file as input?
Versions
lazy-lock.json:
{
  "LuaSnip": { "branch": "master", "commit": "82108e7e31cc6fc223cc5df5cae6d89f70bb199f" },
  "cmp-nvim-lsp": { "branch": "main", "commit": "5af77f54de1b16c34b23cba810150689a3a90312" },
  "cmp_luasnip": { "branch": "master", "commit": "05a9ab28b53f71d1aece421ef32fee2cb857a843" },
  "diffview.nvim": { "branch": "main", "commit": "3dc498c9777fe79156f3d32dddd483b8b3dbd95f" },
  "friendly-snippets": { "branch": "main", "commit": "69a2c1675b66e002799f5eef803b87a12f593049" },
  "gitsigns.nvim": { "branch": "main", "commit": "3e6e91b09f0468c32d3b96dcacf4b947f037ce25" },
  "lazy.nvim": { "branch": "main", "commit": "96584866b9c5e998cbae300594d0ccfd0c464627" },
  "lualine.nvim": { "branch": "master", "commit": "566b7036f717f3d676362742630518a47f132fff" },
  "mason-lspconfig.nvim": { "branch": "main", "commit": "0989bdf4fdf7b5aa4c74131d7ffccc3f399ac788" },
  "mason.nvim": { "branch": "main", "commit": "e110bc3be1a7309617cecd77bfe4bf86ba1b8134" },
  "neodev.nvim": { "branch": "main", "commit": "be8d4d4cab6c13c6a572269c9d6a63774baba9a0" },
  "neogit": { "branch": "master", "commit": "760d6d74c328ce2a38b02677bf5161999b992da2" },
  "nvim": { "branch": "main", "commit": "5e36ca599f4aa41bdd87fbf2c5aae4397ac55074" },
  "nvim-cmp": { "branch": "main", "commit": "538e37ba87284942c1d76ed38dd497e54e65b891" },
  "nvim-lspconfig": { "branch": "master", "commit": "796394fd19fb878e8dbc4fd1e9c9c186ed07a5f4" },
  "nvim-notify": { "branch": "master", "commit": "1576123bff3bed67bc673a3076e591abfe5d8ca9" },
  "nvim-tree.lua": { "branch": "master", "commit": "f1b3e6a7eb92da492bd693257367d9256839ed3d" },
  "nvim-treesitter": { "branch": "master", "commit": "8cd2b230174efbf7b5d9f49fe2f90bda6b5eb16e" },
  "nvim-web-devicons": { "branch": "master", "commit": "db0c864375c198cacc171ff373e76bfce2a85045" },
  "obsidian.nvim": { "branch": "main", "commit": "dbd81af68896a6276001dd9f65c7cb965b411338" },
  "plenary.nvim": { "branch": "master", "commit": "55d9fe89e33efd26f532ef20223e5f9430c8b0c0" },
  "telescope-fzf-native.nvim": { "branch": "main", "commit": "6c921ca12321edaa773e324ef64ea301a1d0da62" },
  "telescope.nvim": { "branch": "master", "commit": "d90956833d7c27e73c621a61f20b29fdb7122709" }
}

For reference, I tried running the bin command outside of obsidian.nvim with the proper URL and got the following result:
2024-01-10 11:50:39 Loading main app package /tmp/.mount_obsidiuya4F3/resources/obsidian.asar
Received callback URL obsidian://open?vault=ThoughtPool&file=Inbox.md
2024-01-10 11:50:59 Checking for update using Github
2024-01-10 11:51:05 Success.
2024-01-10 11:51:05 Latest version is 1.5.3 (insider)
2024-01-10 11:51:05 App is up to date.

The app opened correctly to "Inbox.md". I did have to disable the "Open Daily Note on Startup" option for this to work, although there's probably a work-around with the obsidian command for that (I'm just not aware of it).

@manoliptram sorry I have no idea. Seems like an issue with the Obsidian app but I'm just guessing here. If you find a shell command that works let me know. Maybe we need to adjust our command or at least add the option to override it.

@manoliptram sorry I have no idea. Seems like an issue with the Obsidian app but I'm just guessing here. If you find a shell command that works let me know. Maybe we need to adjust our command or at least add the option to override it.

I was running into the same error when I tried this plugin and ended up writing my own open command. A couple differences I noticed after skimming this plugin's source code just now:

I use vim.uri.encode and obsidian.nvim uses a custom url_encode util
I use vim.fn.jobstart to create a detached process, obsidian.nvim uses custom async utils
I use the path URL param and this plugin uses vault and file params

Unfortunately, I'm not sure what is causing the issue since I'm not familiar with the codebase. Here is my code for reference, hopefully it will help someone figure it out.
local M = {}

--- Open a file in the Obsidian GUI app. Accepts an absolute or relative
--- filepath as an argument, otherwise uses the current buffer.
--- If the file is not found, $NOTES is prepended to the argument. Setting
--- $NOTES to a vault directory allows you to open notes from anywhere on your
--- filesystem without having to specify an absolute path.
--- @param event table  A user command's callback argument. See :h nvim_create_user_command
--- @example - assumes ~/work/project/thing.md doesn't exist
---   :let $NOTES="~/some/path/to/notes"
---   :pwd                    -> ~/work/project
---   :ObsidianOpen thing.md  -> opens ~/some/path/to/notes/thing.md
function M.obsidian_open(event)
  local absolute_filepath

  if event.args ~= "" then
    if vim.fn.filereadable(event.args) == 1 then
      absolute_filepath = vim.fn.fnamemodify(event.args, ":p")
    else
      absolute_filepath = vim.fn.fnamemodify(("%s/%s"):format(vim.env.NOTES, event.args), ":p")

      if vim.fn.filereadable(absolute_filepath) == 0 then
        vim.api.nvim_err_writeln(
          ("File not found at: '%s' or '%s'"):format(event.args, absolute_filepath)
        )
        return
      end
    end
  else
    absolute_filepath = vim.fn.expand "%:p"
  end

  -- https://help.obsidian.md/Advanced+topics/Using+Obsidian+URI#Shorthand%20formats
  local uri = ("obsidian://open?path=%s"):format(vim.uri_encode(absolute_filepath))

  if vim.fn.exists "*netrw#BrowseX" then
    vim.fn["netrw#BrowseX"](uri, 0)
  elseif vim.fn.executable "xdg-open" == 1 then
    vim.fn.jobstart("xdg-open " .. uri, { detach = true })
  elseif vim.fn.executable "open" == 1 then
    vim.fn.jobstart("open " .. uri, { detach = true })
  end
end

vim.api.nvim_create_user_command("OO", M.obsidian_open, {
  complete = "file",
  nargs = "?",
  desc = "Open a note in the Obsidian GUI app",
})

return M

I did some testing and it isn't related to the util.urlencode or using vault/file vs path. It must be related to how the job is created; somewhere in the async module.


Fix server shutdown if conmon-rs got manually killed

What type of PR is this?
/kind bug
What this PR does / why we need it:
We should not fail in case conmon-rs got killed by another instance, because this would trigger an error in sandbox removal in CRI-O and therefore makes it impossible to remove the sandbox. We now double check if the PID still exists before going into the wait loop.
Which issue(s) this PR fixes:

None
Special notes for your reviewer:
None
Does this PR introduce a user-facing change?

Fixed server shutdown error if PID does not exist any more.

Codecov Report

Merging #717 (543ac09) into main (92f7abf) will increase coverage by 0.53%.
The diff coverage is n/a.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #717      +/-   ##
==========================================
+ Coverage   32.91%   33.45%   +0.53%
==========================================
  Files          13       13
  Lines        1124     1124
  Branches      416      415       -1
==========================================
+ Hits          370      376       +6
+ Misses        474      466       -8
- Partials      280      282       +2

/lgtm


Batchload gpx to collecting events

Basic GPX batch load, as described in Issue #699.

Coverage increased (+0.1%) to 72.101% when pulling 66f77690171fb008654f28c15483e10b5edd07ea on batchload_GPX_to_collecting_events into f731f4dc2af975d42f30d12065d2a8be1f834278 on development.


TopN query error seems incorrect - "Must have an AggregatorFactory"

When I run a TopN query, I get an error that I must give an "AggregatorFactory" for a particular metric.
I'm using the standard "longMax" aggregation in my query. (see query below)
Why am I getting this error?
Error:
{"error": "Instantiation of [simple type, class io.druid.query.topn.TopNQuery] value failed: Must have an AggregatorFactory or PostAggregator for metric[purchase_amount], gave[[LongSumAggregatorFactory{fieldName='purchase_amount', expression='null', name='max_purchase_amount'}]] and [[]]"}

Query:
{
  "queryType": "topN",
  "dataSource": "my_records",
  "dimension": "my_unique_id",
  "threshold": 1,
  "metric": "purchase_amount",
  "aggregations": [
    { "type": "longMax", "name": "max_purchase_amount", "fieldName": "purchase_amount" }
  ],
  "intervals": ["2018-01-06T03:07:48.482Z/2018-01-06T03:08:21.523Z"],
  "granularity": {
    "type": "duration",
    "duration": 330.41,
    "origin": "2018-01-06T03:07:48.482Z"
  },
  "filter": {
    "type": "and",
    "fields": [
      { "type": "selector", "dimension": "first_name", "value": "Don" },
      { "type": "selector", "dimension": "last_name", "value": "Pinkus" }
    ]
  }
}

Had the same issue. Seems like in the aggregations the name and fieldName should be the actual metric_name ... weird and seems like a bug.
"aggregations": [
{ "type": "longMax", "name": "max_purchase_amount", "fieldName": "purchase_amount" }
],
Should be :
"aggregations": [
{ "type": "longMax", "name": "purchase_amount", "fieldName": "purchase_amount" }
],


The magic number 0.43

The prediction was done on which testing dataset to reach an accuracy of 0.43 ?  I mean, was the prediction done on the blind data set with SHANKLE well or NEWBY well or any other data?

Please use the printed article as the intended view of performance, not the notebooks. So the blind well is SHANKLE and the F1 score is 0.43. I will try to make the notebooks consistent today.

Reopening because, while b9f2b15fcf4b56bed54e5b69246f431df497426b has resolve the mismatch between the article nd the notebook, I cannot reproduce the F1 score in the article.
Right now, it's 0.39 (cf 0.43 in the article). Hoping I just missed something when I merged the final version of the notebook. Maybe @brendonhall can check the notebook and make sure it's the same as his. I'm thinking the main differences could be the test/train random seed, and the values of the hyperparameters c and gamma.

See this closed issue by @CannedGeo #10 Training_data.csv?

Hi everyone, sorry for being late to the party on this.  The problem is that the main notebook and the article are a little out of sync.  I streamlined the narrative of the notebook quite a bit for the article.  One of the things I did was create a dataset for the article that only had a complete set of well logs.  facies_vectors.csv is the original training set from the website I obtained the data from.  I removed the vectors that don't have a PE value, and saved that dataset has training_data.csv.  That is what I used for the article, and I have changed the notebook to match.
Second, in the article I used test_size=0.1 when splitting the training and cross validation sets.  The notebook had a value of 0.2, so I have changed this to match.
After these changes, I'm getting an overall F1 of 0.42 for the classifier.  Not quite what is reported in the article.  When I run the pure article code again I'm also getting 0.42.  Perhaps there has been some change in the ordering/randomization of the data?  Maybe some libraries have been updated? It's Halloween???

Awesome, thanks @brendonhall.
As far as I'm concerned, 0.42 is the same as 0.43. I am very confident that someone will get a better fit than 0.43 soon anyway, if they haven't already. So if this isn't a non-issue already, I think it will be soon.
@thanish Thank you again for raising this. I'll close it now. The notebook is a good reflection of the 'base case'. We can of course re-open if need be, I trust you will let me know. Cheers!


fix(config): tweak z-trm6 options

This is mainly a cosmetic fix for HA.
These predefined options, but since we allow manual entry Home-Assistant doesn´t list the options.

We should also consider if we should use the the more technical sensor descriptions.
Home-Assistant has some override code that switches the temp sensor depending on operation mode.
Example: https://github.com/home-assistant/core/blob/dev/homeassistant/components/zwave_js/discovery.py#L548
Did a quick discussion on this topic in zwave_dev but, but there was no clear recommendation. As the current layout is more in line with the new simple stile of zwave-js, it might be better to keep todays code.
The only benefit in HA would be that we could reuse some code, but it´s not much.
Basically  if param2 should be changed to something like this.
		{
			"#": "2",
			"label": "Sensor Mode",
			"valueSize": 1,
			"minValue": 0,
			"maxValue": 5,
			"defaultValue": 1,
			"allowManualEntry": false,
			"options": [
				{
					"label": "F-Mode, Floor",
					"value": 0
				},
				{
					"label": "A-Mode, Internal",
					"value": 1
				},
.........
				{
					"label": "PWR-Mode, Power regulator",
					"value": 5
				}
			]
		},


Login test fails on Travis CI; passes locally

Get Travis CI to run the rake/rspec tests

...so, the first problem was that Gemfile and Gemfile.lock were out-of-sync so we ran "bundle install" and committed the resulting Gemfile.lock
Then, Travis CI complained about not finding rake, so that was added to the Gemfile.
Now, we need a MongoDB instance. Hopefully this is the last cog that needs fixing.

The Travis CI documentation also mentions:

A few users have reported that MongoDB does not accept connections when from the build script.
The issue is intermittent, and the only reliable way to avoid it is to inject an artificial wait before making the first connection

If the problem starts to rear its head, the following is the recommended fix:
before_script:
  - sleep 15

Okay, now Travis CI does actually run the RSpec tests, but fails because it's unable to log in.
Perhaps it isn't sending back the session ID cookie so after the login, the application attempts to redirect the user to (most likely) the login page.
Failures:

  1) Helpdesk actually logs in

     Failure/Error: expect(last_response.status).to eq 200

       expected: 200
            got: 302

       (compared using ==)

     # ./spec/app_spec.rb:30:in `block (2 levels) in <top (required)>'

Finished in 0.15656 seconds (files took 0.46839 seconds to load)
4 examples, 1 failure

Failed examples:
rspec ./spec/app_spec.rb:25 # Helpdesk actually logs in

Travis CI is easy to setup because when you tell it that the language of your project is Ruby, it tries to work its 'magic'. Although it's likely from a pre-Docker era, the yml config is simple. There's probably something specific to the environment they are building that is causing the test to fail.
When looking at alternatives, I noticed that AppVeyor is better suited for .NET projects.
Any chance Oracle's Wercker can pick up the Docker files and run a build?
TeamCity is a JetBrains product. They build good stuff, so I've got to give this a try. Does anyone have some experience that they can share?
CodeShip uses Docker and has a yml config to specify the commands to run for performing a test.

Doh! Travis CI never gets the default user because it is created on calling "/", but none of the tests call "/" so the DB is always empty!
Adding a call to "/" seems to have fixed the issue.


Consider using a 3rd-party package for forms?

Right now the setup of form fields is array based.
I think this will be limiting, since it is hard to customize form fields or add app specific logic and layout to how form (fields) look and work.
I thinks this is a well maintained form package :
https://github.com/adamwathan/form
For the rest of this BackPack idea: I think its great!

Yes, @cristiantone and some other people have pushed toward using LaravelCollective's Form.
I'm open to the idea but... honestly, I see very little benefits except for the fact that it looks cleaner than pure HTML + PHP. The way I see it:

using no form package is limitless,
using a package has limits and when you reach that limit, you fall back to HTML+PHP.

So what's the point of using it in the package, then?
Also, choosing one form package will somehow force developers to use the same one - and maybe they prefer a different one... Your thoughts?

After careful consideration when running through all fields in the 2.1 update, I've concluded that using a Form helper would do more harm than good, at least in the field types.
Backpack\CRUD 2.1 will still ship with LaravelCollective's Form, which is used in a few places where there are forms, but not for the actual field types, since it's way easier to understand and modify them when NOT using forms, and for this package I believe that's more important than pretty code.


Two button layout for menus

Is your feature request related to a problem? Please describe.
Provide support for menus with only two buttons.
Confirm that the bug is in the embedded library
Yes, embedded issue.
Describe the solution you'd like
two button support

See https://github.com/davetcc/tcMenu/issues/370


Explain more detailed the Async-Part of Impersonation-Section

Section: Impersonation
Sentence: RunImpersonated doesn't support asynchronous operations and shouldn't be used for complex scenarios.
Please explain more detailed what async means in this context. Does it only mean that the impersonated delegate is forbidden to run async code or does it mean that two independent threads should not start impersonation at same time.
Second Szenario
Please let me explain what I mean with the second szenario:
My Asp.Core app gets two user requests (I think the requests are handled by two different threads). At some point both requests would impersonate at same time but on two different threads. Is this meant with "asynchronous operations" or not?
First Szenario
For the first szenario I have a technical question:
What is the exactly problem when running an async task within the impersonated delegate.
a) Is it because the async task would not run within impersonation context
b) or because the sync delegate close the impersonation before async has finished?

By the way, it would be helpfull if the documentation of WindowsIdentity.RunImpersonated would reference about the asp.net-core behavior or link this page (https://docs.microsoft.com/en-us/dotnet/api/system.security.principal.windowsidentity.runimpersonated).

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 401ad9a0-9e51-80a2-5846-82e9790d7257
Version Independent ID: fbc36c26-9992-1f4c-66d3-02f898ee7ec4
Content: Configure Windows Authentication in ASP.NET Core
Content Source: aspnetcore/security/authentication/windowsauth.md
Product: aspnet-core
Technology: aspnetcore-security
GitHub Login: @scottaddie
Microsoft Alias: riande

Thank you for contacting us. Due to a lack of activity on this discussion issue we're closing it in an effort to keep our backlog clean. If you believe there is a concern related to the ASP.NET Core which hasn't been addressed, please file a new issue.


OSX dotnet packaging issue (".dylib" vs ".so")

Hi,
I have tested the dotnet package of the release SimpleITK-2.2.0rc2.post79, and found out that I need to rename the native library from ".so" to ".dylib" in order for the dotnet runtime to find it. I wonder if the csharp test is run using mono, maybe, and then that could be the culprit. I'm using the (now standard) dotnet package from microsoft, which is mit-licensed and multi-platform, and can also cross-compile packages.
Apart from this bug, I've been able to cross-compile from any to any of win,linux, and osx with no problem a test program. That's amazing!
To reproduce the osx bug one should do the following steps on a mac:

install the dotnet sdk from microsoft https://dotnet.microsoft.com/en-us/download

clone this repository https://github.com/vincenzoml/SimpleITK-FSharp-Example

cd to the repository and run dotnet run

Actually I cannot execute the above three steps as I cannot install dotnet on the only osx account that I have. So what I did (but it should make no difference) is:

clone the repository on a machine with dotnet installed

run dotnet publish -r osx-x64

copy the two files in bin/Debug/net6.0/osx-x64/ to the target osx machine

The SimpleITK version and ITK version should be printed (note that msbuild will download the zip file from the simpleitk release page automatically), however that does not happen. Here is the problem
vincenzo@maxima ~ % ./testSimpleITK
Unhandled exception. System.TypeInitializationException: The type initializer for 'itk.simple.SimpleITKPINVOKE' threw an exception.
 ---> System.TypeInitializationException: The type initializer for 'SWIGExceptionHelper' threw an exception.
 ---> System.DllNotFoundException: Unable to load shared library 'libSimpleITKCSharpNative' or one of its dependencies. In order to help diagnose loading problems, consider setting the DYLD_PRINT_LIBRARIES environment variable: dlopen(liblibSimpleITKCSharpNative, 1): image not found
   at itk.simple.SimpleITKPINVOKE.SWIGExceptionHelper.SWIGRegisterExceptionCallbacks_SimpleITK(ExceptionDelegate applicationDelegate, ExceptionDelegate arithmeticDelegate, ExceptionDelegate divideByZeroDelegate, ExceptionDelegate indexOutOfRangeDelegate, ExceptionDelegate invalidCastDelegate, ExceptionDelegate invalidOperationDelegate, ExceptionDelegate ioDelegate, ExceptionDelegate nullReferenceDelegate, ExceptionDelegate outOfMemoryDelegate, ExceptionDelegate overflowDelegate, ExceptionDelegate systemExceptionDelegate)
   at itk.simple.SimpleITKPINVOKE.SWIGExceptionHelper..cctor()
   --- End of inner exception stack trace ---
   at itk.simple.SimpleITKPINVOKE.SWIGExceptionHelper..ctor()
   at itk.simple.SimpleITKPINVOKE..cctor()
   --- End of inner exception stack trace ---
   at itk.simple.SimpleITKPINVOKE.Version_VersionString()
   at itk.simple.Version.VersionString()
   at <StartupCode$testSimpleITK>.$Program.main@() in /mnt/c/Users/vince/data/local/repos/SimpleITK-FSharp-Example/Program.fs:line 1
zsh: abort      ./testSimpleITK

Setting the DYLD_PRINT_LIBRARIES environment variable prints a lot more information which however does not clarify the issue any further.
Renaming the native library from .so to .dylib on the target machine solves the problem.

Just for completeness, in a new branch of that repository I added a csharp test and can confirm that the bug does not depend on the choice of the language, that is, is also present in the csharp version.

You are correct that we are building and testing the the Mac CSharp libraries with mono.

I found out that it's actually a bug in mono, inherited from glib, at least according from one answer to this question:
https://stackoverflow.com/questions/47225350/how-to-access-native-apis-from-mono-command-line-app
So probably "the right way" is to rename the .so to .dylib before zipping. The "very right way" would be to use dotnet instead of mono, as dotnet is converging on all platforms. As far as I understand, mono will become much less relevant with the release of .net 7 which is now at preview 3.

See here https://www.mono-project.com/docs/advanced/pinvoke/ for a more official statement on the osx bug in mono.

Here is a PR (#1637) which updates the CMake code to produce a dylib library on OSX. It works locally, and on the current azure devops CI. It's not clear when mono was updated to support dylib, but it seems to currently support dylibs.

Are you completely sure it's OK to still use mono, and not dotnet? I am unsure whether mono is going to be outdated soon.

It's true, all managed dlls are binary compatible between mono and dotnet, this includes the managed part of simpleITK, and compiled programs. There is something fishy here though: even if all dlls should be compatible, they are binary different between the different versions of the simpleITK binary (one per platform).
Ideally, there should be a way to produce just ONE dll (maybe the windows one would be the ideal candidate) and then just the native library for each platform.
Right now I don't know more about this aspect, and am referencing the managed library by path depending on the architecture just to be sure. That has pitfalls though, for instance I need to set RuntimeIdentifier in the project file even if the user did not want to do that.

Awaiting the nightly binaries but where will I find them? Just on the releases page?

Awaiting the nightly binaries but where will I find them? Just on the releases page?

The nightly binaries are published here: https://github.com/SimpleITK/SimpleITK/releases/tag/latest

It's true, all managed dlls are binary compatible between mono and dotnet, this includes the managed part of simpleITK, and compiled programs. There is something fishy here though: even if the three managed dlls of the three simpleITK packages should be compatible, they are binary different between the different versions of the simpleITK binary (one per platform).

The thing todo would be to example the SWIG generated cs files to determine what the difference it between the platforms in the source code. Binary difference could be from the source code or the compiler generate code, or even just some meta information encoded by the compiler ( paths?, os?, versions?).

Welcome to the SimpleITK community.
This looks terrific! It looks like the perfect way to use SimpleITK in the .Net environment. Glad to hear the new fix for OS X worked.
The best place for this would be adding it to the main SimpleITK repository in the "Examples" directory. Would you be interested in contributing the quickstarts in a PR?

I want to try to keep the repository up-to-date with the latest simpleitk version. How do I track the new releases (nightly for the moment, and then stable from when stable will distribute the binaries)?

SimpleITK 2.2rc3 will be published in the next couple of week, after a couple build issues are resolved. That should be a version publish in the Github Releases that can be used in this quickstart/example. The nightly version changes whenever there are changes in the repository. This also deleted the old nightly version so that the old binaries are no longer available. This enables manual downloads and development with latest features but makes it difficult to depend or publish items which depend on the nightly builds.

Eventually I will create a nuget package, or find someone who can do it (unless the simpleitk organization itself wants to take the burden), but for the moment I want to try to keep the repository up-to-date with the latest simpleitk version. How do I track the new releases (nightly for the moment, and then stable from when stable will distribute the binaries)?

This is another terrific contribution. It would be great to publish an official SimpleITK nugget package directly from the SimpleITK project. Integrating some scripts and processes into the release process to also publish nuggets is certainly something we can accommodate. How to build the nugget, providing code to do so, testing and what changes to the current building and packages are all welcomed contributions.

@vincenzoml  The SimpleITK v2.2rc3 release contains the fixed OS X CSharp package. The quick start example you wrote can be updated to reference this release tag.
The published packages RC package resolve this issue.

Hi, sorry for not replying for quite some time. So: it would be great to contribute to the examples section of the simpleitk site. Is it managed in github? I don't have time right now, I am fully overbooked until july, but then will do my best to submit a PR.
Thanks for the release, will test as soon as possible (unfortunately maybe in july too).


Fix Slider not moving in short distances.

Description
Slider currently relies in HorizontalDragGestureRecognizer which currently relies in kTouchSlop to detect if user tapped or scrolled. Issue is, you always want a Slider to be scrollable. It makes no sense for a Slider that is in position 0.3 to only begin scrolling in position 0.4. This also causes a bad user experience (as per the GIF below).
I decided to modify HorizontalDragGestureRecognizer and VerticalDragGestureRecognizer to receive an optional touchSlop parameter which defaults to kTouchSlop, so that there is no behaviour or breaking change. This way, any user or widget might specify their own touchSlop value, which is what I did with the Slider. I am open to discuss other approaches or modifying my implementation.
Related Issues

https://github.com/flutter/flutter/issues/40310
Tests
How do I even test this?
Checklist
Before you create this PR confirm that it meets all requirements listed below by checking the relevant checkboxes ([x]). This will ensure a smooth and quick review process.

[x] I read the [Contributor Guide] and followed the process outlined there for submitting PRs.
[x] I signed the [CLA].
[x] I read and followed the [Flutter Style Guide], including [Features we expect every widget to implement].
[x] I updated/added relevant documentation (doc comments with ///).
[x] All existing and new tests are passing.
[x] The analyzer (flutter analyze --flutter-repo) does not report any problems on my PR.
[x] I am willing to follow-up on review comments in a timely manner.

Breaking Change
Does your PR require Flutter developers to manually update their apps to accommodate your change?

[ ] Yes, this is a breaking change (Please read [Handling breaking changes]). Replace this with a link to the e-mail where you asked for input on this proposed change.
[x] No, this is not a breaking change.

Could someone please take a look?

This is definitely an issue that this of fixes. I've seen it happen multiple times as well as the stuttery or stopped or clipped motion while actively changing the slider value automatically as a video plays.

This should only be visible if you can also scroll vertically. If there's no competing vertical drag gesture (or some other competing gesture, e.g. a higher-level tap gesture detector), it should start dragging immediately.
Unless I'm misunderstanding the issue?

I'm going to close this PR because before we can determine if it is the right fix we need to figure out what broke.


Before and After Retry, and Logging

I'd like to log retries similar to what is done in the tenacity documentation.
https://tenacity.readthedocs.io/en/latest/#before-and-after-retry-and-logging

ENRICO!?!?!? You here!? How long it's been? 20 years? 😅
Anyhow, I think this is subproblem of #21 but an important addition: there should be a plug before and after a retry.


Add from_relative to Body?

There was some discussion about adding a from_relative method to the Body class on a recently closed PR thread: https://github.com/poliastro/poliastro/pull/153
Is that something that there's some interest in seeing in a new PR?

You're right @jzmiller1, I oversaw that. Please go ahead :) I think this comment summarizes more or less what you wanted, right?
https://github.com/poliastro/poliastro/pull/153#issuecomment-284218548

Yes, something like that should work well.  I'll put a PR for it together in the next week unless you already had the code for it somewhere?

I don't have it, so please do. Also, I encourage you to join our chat, so we can guide you more quickly:
https://riot.im/app/#/room/#poliastro:matrix.org

This was closed in #235.


Crash / Failure When Handling MediaServicesReset on iOS

Hi,
I apologize in advance if it ends up that StackOverflow is the appropriate place to post this - however I have searched the codebase to the best of my ability and I cannot find a reference to the iOS MediaService notifications / events and given that by "default" usage of AudioKit I experience a crash, I felt it appropriate to post an issue to GitHub.  Happy to change if needed, just let me know.
So, the issue I am seeing is that when I trigger a Reset Media Services from the Settings -> Developer menu on an iOS device, the next subsequent AKPlayer.play() call will fail with an assertion exception that looks something like:
... required condition is false: _engine->IsRunning() Terminating app due to uncaught exception 'com.apple.coreaudio.avfaudio', reason: 'required condition is false: _engine->IsRunning()'
I poked around a bit in AudioKit+StartStop.swift and it seems that there is some code to handle other similar events (route / configuration changes); fair enough, maybe I just need to do something similar in my own AVAudioSessionMediaServicesWereReset notification observer.
However, I can't seem to come up with a combination of calls that will properly reset the AudioKit engine without that "restart" failing separately (or crashing, depending on how I call things) with the following logs:
2018-08-21 16:15:12.634808-0700 Drivetime[7967:2553564] [aurioc] 921: failed: -50 (enable 2, outf< 2 ch,  44100 Hz, Float32, non-inter> inf< 2 ch,      0 Hz, Float32, non-inter>)
2018-08-21 16:15:12.672142-0700 Drivetime[7967:2553564] [avae] AVAEInternal.h:103:_AVAE_CheckNoErr: [AVAudioEngineGraph.mm:1227:Initialize: (err = PerformCommand(*outputNode, kAUInitialize, NULL, 0)): error -50
2018-08-21 16:15:12.672700-0700 Drivetime[7967:2553564] *** Terminating app due to uncaught exception 'com.apple.coreaudio.avfaudio', reason: 'error -50'

Based on my (limited) understanding of AVAudioEngine et al., it seems like the internal engine initialization on my "second start" / "restart" is considered to be passing bad parameters of some sort to the system.
One operating theory I have is that perhaps the existence of the finalMixer internally from the previous engine is breaking things when calling start a second time (I've seen multiple suggestions / references elsewhere that in the event of a media services reset all nodes etc. should be completely dropped and re-created).
However, this appears to be set to internal so I can't test out resetting the finalMixer myself without building the library from code / scratch (not ideal for me but I can if necessary).
Anyways, to make a long story short - I feel like I'm missing something obvious here and I was hoping maybe you could help me understand why a media services reset doesn't appear to be handled like route / configuration changes already are, if these crashes / issues are expected as a result of a media services reset in the current codebase, and what I should do to handle them properly myself while using AudioKit.
Finally, for context, here are the two most relevant examples of what I've tried doing to handle this so far (without success) - keep in mind I am dispatching the call to this function to the main queue from the MediaServicesReset observer based on the example I saw in AudioKit+StartStop.swift:
func resetAudioKit() {
    do {
      // I've tried just calling start directly (and also calling engine.start() directly) as that's more or less how I see it handled in AudioKit+StartStop.swift
      mixer = AKMixer()
      AudioKit.output = mixer
      try AudioKit.start()  // I have also tried calling AudioKit.engine.start() directly as well.
    } catch {
      print("AudioKit Reset ERROR!")
    }
  }

func resetAudioKit() {
    do {
      // If we've already setup before, I have also tried stopping AudioKit first before starting it.
      if mixer != nil {
        try AudioKit.stop()  // I have also tried calling try AudioKit.engine.stop() directly as well.
      }
      mixer = AKMixer()
      AudioKit.output = mixer
      try AudioKit.start()  // I have also tried calling AudioKit.engine.start() directly as well.
    } catch {
      print("AudioKit Reset ERROR!")
    }
  }

Definitely seems like it could get more attention in StackOverflow, but would need a tl;dr too. :). Basically it would be great if all our issues were actionable and I'm not really sure what to do here. Could this be broken up into smaller issues?

@aure thanks for the quick response!
I guess one first step / TL;DR version of this might be - as far as I can tell in my local testing, if I create an AKMixer as AudioKit.output and then connect AKPlayer nodes to them over time, AudioKit will not handle a MediaServicesReset gracefully and the app using it will crash on the next AKPlayer.play().
Separate from all the other details / questions above, is that a concise enough version for the team to verify / reproduce and go from there?

Verification and reproduction is made easier if you could supply a sample project. A lot of times we'll be given what people feel is the relevant code, but then to start, setup, and prove a project is time consuming.  Could you share a simple project that reproduces the problem?

Hi @aure,
I'll try to put together my own example project I can send that is super clean for this specific case, but in the meantime, if it's of any value, it appears some of the existing examples in the AudioKit repository that include interactivity ("play this sound now") etc. will break as-is if you hit the Reset Media Services button from the Developer menu on a physical device.  I'll see if I can find a specific good existing example with a write up of steps to send you to make it super easy.
In the meantime, I went ahead and integrated the source code version of AudioKit to play around a bit more and also did some more research.
Based on Apple's audio presentation at WWDC: http://devstreaming.apple.com/videos/wwdc/2016/507n0zrhzxdzmg20zcl/507/507_delivering_an_exceptional_audio_experience.pdf?dl=1 (slide 58), I do think that there is some additional plumbing needed to handle this event cleanly.  They specifically mention:
Need to destroy and recreate objects
• AVAudioEngine, queues, remote I/Os, players, etc

And I believe it is the lack of this which is causing a crash when this event occurs.
I implemented a basic test function in AudioKit+StartStop to test this:
    @objc open static func reset() throws {
        if engine.isRunning {
            try AKTry {
                engine.stop()
            }
        }

        let temp = output
        engine.reset()
        engine = AVAudioEngine()
        finalMixer = AKMixer()
        output = temp

        try AKTry {
            try start()
        }
    }

And it does indeed prevent the crash in question, but because I am not sure how to clean up all the other nodes (for example previous AKPlayer's and all of their sub-nodes), subsequent crashes still occur (e.g. the dealloc of the old AVAudioEngine ends up triggering AKPlayer's completion callback, which in turn then goes and looks for the currentFrame and causes a crash because the old AVAudioEngine is now nil and triggers an assert...
Anyhow - I'll see if I can find a very specific example / set of steps I can send to allow a quick repro on your side.  Just thought I'd share my ongoing findings in the meantime as well.

Hi, did you solve the problem?
Similar crash sometimes occurs when you try play() for AKPlayer (Fatal Exception: com.apple.coreaudio.avfaudio player started when engine not running)

Hi @extnous and @aure
Sorry for the extreme delay in my response.
We've finally gotten around to tackling this for our own app / use case and have come up with a potential solution, which partially involves some AudioKit changes.
I've gone ahead and submitted a PR (https://github.com/AudioKit/AudioKit/pull/1522) - feel free to use it if it's helpful (or at least reference it if you come across this later as a user experiencing this issue, even if it isn't something worth merging).
Thanks!


Wrong order when compose up crash nginx.

Hello ;)
Sometimes nginx is executed first and then have no configuration from nginx-gen.
Also its not possible to use "depends_on" directive cause it have volumes_from and then we have:
ERROR: Circular dependency between nginx and nginx-gen
Starting simple-site Starting nginx Starting nginx-gen Starting letsencrypt-nginx-proxy-companion Attaching to simple-site, nginx, nginx-gen, letsencrypt-nginx-proxy-companion nginx                             | 2016/07/20 14:19:34 [emerg] 1#1: open() "/etc/nginx/vhost.d/default" failed (2: No such file or directory) in /etc/nginx/conf.d/default.conf:56 nginx                             | nginx: [emerg] open() "/etc/nginx/vhost.d/default" failed (2: No such file or directory) in /etc/nginx/conf.d/default.conf:56 nginx-gen                         | 2016/07/20 14:19:34 Generated '/etc/nginx/conf.d/default.conf' from 2 containers nginx-gen                         | 2016/07/20 14:19:34 Sending container 'nginx' signal '1' nginx-gen                         | 2016/07/20 14:19:34 Error sending signal to container: API error (500): Cannot kill container nginx: Container 2c79e66156352b697fd5020efd2147484783f234a477da238741bea3477ef117 is not running nginx-gen                         | 2016/07/20 14:19:34 Watching docker events nginx exited with code 1 nginx-gen                         | 2016/07/20 14:19:34 Contents of /etc/nginx/conf.d/default.conf did not change. Skipping notification '' nginx-gen                         | 2016/07/20 14:19:34 Received event start for container 147eda37160f letsencrypt-nginx-proxy-companion | 2016/07/20 14:19:34 Contents of /app/letsencrypt_service_data did not change. Skipping notification '/app/update_certs' letsencrypt-nginx-proxy-companion | 2016/07/20 14:19:34 Watching docker events letsencrypt-nginx-proxy-companion | 2016/07/20 14:19:34 Contents of /app/letsencrypt_service_data did not change. Skipping notification '/app/update_certs' letsencrypt-nginx-proxy-companion | Reloading nginx proxy (using separate container nginx-gen)... nginx-gen                         | 2016/07/20 14:19:35 Received signal: hangup nginx-gen                         | 2016/07/20 14:19:35 Received signal: hangup letsencrypt-nginx-proxy-companion | Creating/renewal mysite.com certificates... (mysite.com) nginx-gen                         | 2016/07/20 14:19:35 Generated '/etc/nginx/conf.d/default.conf' from 2 containers nginx-gen                         | 2016/07/20 14:19:35 Sending container 'nginx' signal '1' nginx-gen                         | 2016/07/20 14:19:35 Error sending signal to container: API error (500): Cannot kill container nginx: Container 2c79e66156352b697fd5020efd2147484783f234a477da238741bea3477ef117 is not running letsencrypt-nginx-proxy-companion | 2016-07-20 14:19:35,291:INFO:simp_le:1383: Certificates already exist and renewal is not necessary, exiting with status code 1. letsencrypt-nginx-proxy-companion | Sleep for 3600s nginx-gen                         | 2016/07/20 14:19:39 Debounce minTimer fired nginx-gen                         | 2016/07/20 14:19:39 Contents of /etc/nginx/conf.d/default.conf did not change. Skipping notification ''

I believe this got fixed by docker-gen.


drupal 0.3.2

Updates the Drupal to version 8.1.10

@k8s-bot ok to test

@k8s-bot test this

I signed it!

LGTM!


feat: measure latency of HTTP events

Why
As a single page app owner, I rely on AJAX requests to fetch data in the background so that I may present a different view without refreshing. In order to insist on the highest standard, I need observability into the performance of Fetch and Xhr requests, which is best measured by latency.
Changes

Merged with #412, which includes breaking change of enabling trackAllRequests by default
Fetch and Xhr Plugins use Date.now() timestamps to measure latency
Schema change to HttpEvent, includes required fields startTime and duration
Timestamps are shared between Trace and Http Events
Performance API was NOT used in this PR because PerformanceResourceTiming entries and monkey patched HttpEvents can only be mapped by request URL, which will have collisions on repeated requests. Collisions can be resolved by creation order, but a stress test is needed to investigate to what extent incorrect mappings actually occur. The PRT implementation will be released later as apart of another feature, once tested.

I have rebased release/2.0.x onto this PR so that #412 is not longer included in these changes

In fix: remove redundant trace latency tests, I realized that unit tests to check if trace events capture latency are not needed. start_time and end_time are required on trace events, so previous unit tests already provide enough coverage.

question: In the PR description, you mention that this is merged with #412. Will we still need the other PR or will we close that in favor of having this PR include all changes?

Yeah we'll keep these separate! #412 has been merged into release/2.0.x. I've rebased those changes onto this PR, so that #417 will only contain the latency changse.
When we are ready to release 2.0, we will do a merge fast forward of 2.0 onto main.

note: I see the commit history is a bit long, please make sure we squash before merging!

💀


Meshery: New Istio vetter for namespaces ignored by auto-injection

Current Behavior
The Istio sidecar injector will ignore the following namespaces even if they carry the Istio-injection=enabled label:
NamespaceSystem
NamespacePublic
as described here - https://github.com/istio/istio/blob/13cb3422ae47f95dfbc829a327724a56929cab3b/pilot/pkg/kube/inject/initializer.go#L28
Desired Behavior
A new vetter for Istio should flag anytime these namespaces have the istio-injection annotation enabled.

Hi @leecalcote! I would like to work on this issue.

Awesome. That would be great, @supra08

@supra08 as you familiarize with this code base, you'll want to look to the layer5io/meshery-istio repo.

@leecalcote can i work in this issue??

@crustyy I would love for you to, however, we need to put a design spec in place first in order to support the creation of new vetters. I'll add the design-needed label to this issue.

Hi @crustyy, This issue is still open. Would you like to still work on it?

hey @leecalcote @iambami can u guide how to move ahead with this issue....means how to approach?


NullPointerException at org.telegram.tl.StreamingUtils.writeTLObject(StreamingUtils.java:192)

With great effor I builded working (almost) api library, and it even sent me sms code (I used telegram-bot code). But after I write this sms code I got the following exception:
Exception in thread "Scheduller#1613213400" java.lang.NullPointerException 	at org.telegram.tl.StreamingUtils.writeTLObject(StreamingUtils.java:192) 	at org.telegram.api.requests.TLRequestMessagesSendMessage.serializeBody(Unknown Source)	at org.telegram.api.requests.TLRequestMessagesSendMessage.serializeBody(Unknown Source) 	at org.telegram.tl.TLObject.serialize(TLObject.java:46) 	at org.telegram.tl.StreamingUtils.writeTLObject(StreamingUtils.java:192) 	at org.telegram.tl.StreamingUtils.writeTLMethod(StreamingUtils.java:203) 	at org.telegram.api.requests.TLRequestInvokeWithLayer11.serializeBody(Unknown Source) 	at org.telegram.tl.TLObject.serialize(TLObject.java:46) 	at org.telegram.tl.TLObject.serialize(TLObject.java:34) 	at org.telegram.mtproto.schedule.Scheduller.actualPackages(Scheduller.java:388) 	at org.telegram.mtproto.schedule.Scheduller.doSchedule(Scheduller.java:406) 	at org.telegram.mtproto.MTProto$SchedullerThread.run(MTProto.java:704) 	at org.telegram.api.requests.TLRequestMessagesSendMessage.serializeBody(Unknown Source)
I think it can't serialize auth data. What can I do with it? Anybody faced the same problem? Thanks for answers.

 But I can tell that finally I gave up on this library, there was too many problems


How to request installation of WhiteSource integration on a given GH repository

Acceptance Criteria

As FINOS Project Team member, I want to be able to request activation of WhiteSource integration for GitHub for a given GitHub repository

Tasks

[x] Document and socialise<EMAIL_ADDRESS>as support queue per GitHub specific requests

This is contained in the following page on the FINOS wiki.

https://finosfoundation.atlassian.net/wiki/spaces/FDX/pages/1129283585/WhiteSource+for+GitHub.com#WhiteSourceforGitHub.com-EnableWhiteSourcescanning


the error like 'on block transaction failed due to a bad allocation'

When syncing blocks ''on block transaction failed due to a bad allocation' got a error!

Try to increase your chain state. In your config.ini set chain-state-db-size-mb = to a higher value. The default value 1024 MB might not be enough

@Lomect
How did you solve it?


Frontend error handling and logging

Audit frontend code for error paths and ensure errors are A) caught, B) handled gracefully by the UI, C) reported to the server, and D) tracked somewhere on the server along with server errors.

Now that we have our own frontend server, I think the best way to do this is just send a custom request to our frontend server and have the server log it. This requires no libraries and is pretty straightforward. The frontend already has a logger class that's being used and the environment variables are set up for different environments: https://github.com/SatcherInstitute/health-equity-tracker/blob/897e88c590f20481a7b2bbc2fc35ad321cea752a/frontend/src/utils/Logger.ts#L27
It just needs to have the implementation changed to send the error data to the server. The server will need to add a new endpoint:
https://github.com/SatcherInstitute/health-equity-tracker/blob/master/frontend_server/server.js
That handles the error reports and logs them using a Cloud Run library: https://github.com/SatcherInstitute/health-equity-tracker/issues/29#issuecomment-755546166

Moving to backlog for now as per discussion, we should figure this out soon after launch though


Update starter-notebook.ipynb

EDA: Feature distribution/Data Visualization

I think we should consider dropping the scatter plots. It is difficult to make any meaning out of it.


Do you think it's fine to merge these two incredibly complex branches? Especially considering how many changes were made on Main


Account line type summary view.

A basic view and/or rest api to get the following data.
total (sum) of values by:
property by tax_year by tax_type.
The function might be something like this, but using sql not for loops.
# sudo code
tax_type_summary_by_year(parcel_id, tax_year, tax_type=all):
    select account rows where parcel_id== parcel_id and tax_year=tax_year
    for each tax_type, sum all entries
        return tax_type_sum

add new endpoint: /api/v1/account/tax_type_summary
we can use query parameters to filter by: tax_year, min_tax_year, max_tax_year and property__parid
examples:
/api/v1/account/tax_type_summary?tax_year=2015
/api/v1/account/tax_type_summary?min_tax_year=2013
/api/v1/account/tax_type_summary?max_tax_year=2016
/api/v1/account/tax_type_summary?property_parid=M035100


Enhancement: work with raw values

Hi @jschirrmacher
I have several sources in my MQTT, that don't deliver JSON, but single (raw) values (e.g. vzlogger). These are not processed in the current version. I enhanced the MQTTClient.ts so it's detected whether the input is JSON or a raw value. Frankly, I'm not a typescript programmer, but I got some AI help, so it might look clumsy. But hey: it works!
To define a raw source, I use the following in config.yaml:
 - name: electric_meter_bezug
    description: The kWh from the EVU
    type: counter
    topic: vzlogger/data/chn0/raw
    path: value
  - name: electric_meter_einspeisung
    description: The kWh to the EVU
    type: counter
    topic: vzlogger/data/chn1/raw
    path: value
  - name: electric_meter_actual
    description: The kWh actual measured
    type: gauge
    topic: vzlogger/data/chn2/raw
    path: value

I went through GIT hell and was not able to create a PR based on your last PR, so please find the proposed code here
import * as actualMqtt from "mqtt"
import { type MqttClient } from "mqtt"
import { flatten } from "useful-typescript-functions"
import { metrics } from "./MetricsModel"
import { type BaseType, Configuration } from "./types"

function unique<T>(list: T[]) {
  return [...new Set(list)]
}

export interface MQTT {
  connect(url: string): MqttClient
}

interface Logger {
  info: (msg: string | object) => void
  error: (msg: string | object) => void
  debug: (msg: string | object) => void
}

const regExStr = "\\d{4}-\\d{2}-\\d{2}T\\d{2}:\\d{2}:\\d{2}Z"
const dateTimeRegEx = new RegExp(regExStr)
const invalidDateTimeRegEx = new RegExp('(?<!")(' + regExStr + ')(?!")', "g")

export default function (
  config: Configuration,
  mqtt: MQTT = actualMqtt,
  logger: Logger = console
) {
  const user = config.MQTT_USER
  const pwd = config.MQTT_PASSWORD
  const brokerName = config.MQTT_BROKER ?? "localhost"
  const port = config.MQTT_PORT ?? "1883"
  const auth = user && pwd ? `${user}:${pwd}@` : ""

  const client = mqtt.connect(`mqtt://${auth}${brokerName}:${port}`)
  const topics = unique(config.metrics.map((metric) => metric.topic))

  client.on("connect", () => {
    logger.info(`Connection to mqtt://${brokerName}:${port} established`)

    topics.forEach((topic) => {
      client.subscribe(topic, (err) => {
        if (err) {
          logger.error(
            `Subscription of topic '${topic}' failed: ${JSON.stringify(err)}`
          )
        } else {
          logger.info(`Topic '${topic}' subscribed, waiting for data...`)
        }
      })
    })
  })

  client.on("message", (topic, message) => {
    const rawData = message.toString().replaceAll(invalidDateTimeRegEx, '"$1"')

    let jsonData: Record<string, unknown> | null = null

    try {
      // Check if the message is JSON
      if (rawData.startsWith("{") && rawData.endsWith("}")) {
        jsonData = JSON.parse(rawData, (_, value) => {
          if (typeof value === "string" && value.match(dateTimeRegEx)) {
            return new Date(value)
          }
          return value
        })
      } else {
        // If it's a single value, wrap it in a JSON object with a default key
        jsonData = { value: rawData }
      }

      // Flatten the JSON object
      const values = flatten(jsonData)
      logger.debug({ topic, values })

      // Store the parsed values in the metrics
      metrics[topic] = values as Record<string, BaseType>
    } catch (error) {
      logger.error({
        error: (error as Error).message,
        rawData,
        topic,
      })
    }
  })

  client.on("error", (error) => logger.error({ error }))

  client.on("disconnect", () =>
    logger.info(`mqtt://${brokerName}:${port} disconnected`)
  )
}

Thank you for providing this enhancement! I'll take that as a starting point for a new PR, but will need a few days.

I checked the current behavior and found that when I set the path value of a config entry to an empty string (""), it already accepts single raw values, since they are a valid JSON value, both with a numeric or a string value.
The only difference is, that the key of the metric is an empty string instead of the proposed "value".
If this is sufficient for you, I would prefer not to increase the complexity of the adapter.
I've created a PR #126 containing a test that checks with a configuration like this for reference.


[Fix Merge conflicts]Record perf contexts to the write procedure (#227)

Record perf contexts to the write procedure

Signed-off-by: Yilin Chen<EMAIL_ADDRESS>
change perf context to use Duration and add observe_since

Signed-off-by: Yilin Chen<EMAIL_ADDRESS>
add a bit more comments about perf context collecting in write

Signed-off-by: Yilin Chen<EMAIL_ADDRESS>
Address some comments, add tests and changelog

Signed-off-by: Yilin Chen<EMAIL_ADDRESS>
address comment

Signed-off-by: Yilin Chen<EMAIL_ADDRESS>
address more comments

Signed-off-by: Yilin Chen<EMAIL_ADDRESS>

https://github.com/LykxSassinator/raft-engine/blob/f3c268bb954f63f2809f84a940141db4419b2c44/src/env/default.rs#L47


[US] Update connected peers counter

Estimated story points: 5
Description

As a user I want to see how many peers are currently connected to me.

Acceptance criteria

Once a peer joins the counter should increment, if one disconnects it should decrement.

time: 3h


Refactor email templates and implement text improvements

What
Improve Email Builders to centralise email template terminology using i18n as well as avoid duplication and make it easier to change (more details in the commit message).
Update our email templates with the changes suggested by our content designers:

Add headers for page description, change note and time updated at
Change email subject
Add opening line to the body
Reorder parts of the footer (to avoid inconsistent placement of manage and unsubscribe links)
Remove unsubscribe link from immediate emails
Update text for manage your subscriptions link
Update permission reminder text and formatting
Remove ‘do not reply’ and update feedback part formatting
Remove line between content changes and footer

Why
These changes will make email-alert-api easier to maintain and reduce the amount of developer effort needed when future changes are required. It should also stop email for different frequencies getting out of sync with each other.
Our user research showed some points of confusion in the understanding of content within email.
These pattern-wide fixes we can make to improve email alerts across GOV.UK.
Trello card: Refactor templates in email-alert-api
Trello card: Make text improvements to GOV.UK email templates

I had a chat with Leena (content designer) and amended commit 7e8293ab8566a58adf7c1207287ceaff27279191 to only remove the line and space before the footer from digest email. There is a new commit to add a line after opening line for immediate emails.


CLI development set up is not helpful

CLI development set up is not helpful - It is not clear what tools to install and build the project and how to debug the project.
https://github.com/microsoft/vscode/blob/main/cli/CONTRIBUTING.md

"building the CLI" is correct (and works once you install Rust). The other steps are outdated. Notice you marked this as "important", please feel free to DM me if you need help setting it up


feat(app): Wrong pipette alert in tip probe

overview
This PR addresses #861, with two adjustments to the requirements listed in the ticket:

App will direct users to the robot settings page to change pipettes rather than incorporating the change modal in the calibration page
Due to various concerns in both the app and the API, the app will not prevent the user from proceeding to tip probe

The information available to the app to determine an "incorrect" pipette is not completely reliable
Accordingly, I'm not comfortable blocking tip probe
See #1260 for some of the story, also RPC is making this worse

changelog

feat(components): Add/fix children/styling for InstrumentInfo and AlertItem
feat(app): Add alert in tip probe if pipette is incorrect

review requests
Standard code review + robot testing.  Follow the test plan in #861 by messing around with existing protocols in data
@IanLondon this PR touches AlertItem and IntrumentInfo in components lib so please make sure I didn't mess with PD.
Styling isn't very accurate to screens, but after talking to @pantslakz, this page is kind of in flux. I elected to use as much of what we already have (in terms of components) without too much concern for getting it "right" in anticipation of a future UI pass.

:cactus::cactus::cactus: