in_tail: Fix for unexpected file close after logs rotate in update_wa…

Which issue(s) this PR fixes:
Partially fixes #3614 (follow_inode true case)
What this PR does / why we need it:
Thank you for reading to the great developer's!
This PR is fixed for unexpected file close after logs rotate in fluentd v1.16.1.
I found the #3614 's reproducer.
Before applying this patch, fluentd cause unexpected file close after logs rotate every hour on my reproducer.
After applying this patch, fluentd does not cause it on my reproducer.
I have been running testing a long time.
Docs Changes:
Release Note:

Thanks for your contribution!
DCO check is failed: https://github.com/fluent/fluentd/pull/4185/checks?check_run_id=13780778605

Commit sha: 8f44700, Author: KatuyaKawakami, Committer: KatuyaKawakami; Can not find "KatuyaKawakami<EMAIL_ADDRESS>in ["Katuya Kawakami<EMAIL_ADDRESS>"Masaki Hatada<EMAIL_ADDRESS>
Could you fix your name in the Author in your commit message?
- KatuyaKawakami
+ Katuya Kawakami

Could you fix your name in the Author in your commit message?

It's resolved, thanks!

Fix #3614

Could you add partially or something similar to this line in your commit message & the first comment of this PR?
As I described in #3614, it doesn't seem fix the issue on follow_inode false case.

It seems that CI stalls on all platforms.

Fix #3614

Could you add partially or something similar to this line in your commit message & the first comment of this PR? As I described in #3614, it doesn't seem fix the issue on follow_inode false case.

Thank you for commenting.
Katsuya will handle your comment next week.
By the way, please let me explain why we didn't take care of follow_inode false case.
To fix this unexpected file close issue, we have to modify the tail plugin to create a tail list with hash values generated by inode numbers.
But users who use follow_inode false might want the plugin to create a tail list with hash values generated by file names.
That's why we hesitated to take care of follow_inode false case in this commit.
In our understanding, follow_inode false is just for keeping the compatibility with old verions of the tail plugin.
https://docs.fluentd.org/input/tail#follow_inodes explains that follow_inode false can cause "read rotated files duplicately" problem.
So every user should use follow_inode true unless there is some particular reason.
Please let me know if our understanding is wrong.
We can change our code if needed.

It seems that CI stalls on all platforms.

$ bundle install
$ bundle exec rake test TEST=test/plugin/test_in_tail.rb TESTOPTS="-v"
...
TailInputTest:
  test_EACCES:																.: (0.513357)
...
  group rules line limit resolution:
    test: valid:															.: (0.003195)
  inode_processing:
    test_should_close_watcher_after_rotate_wait:

The cause is test_should_close_watcher_after_rotate_wait doesn't follow the in_tail's modification.
In addition, your patch seems to be missing a fix.
diff --git a/lib/fluent/plugin/in_tail.rb b/lib/fluent/plugin/in_tail.rb
index 71d1be9b..27ee48da 100644
--- a/lib/fluent/plugin/in_tail.rb
+++ b/lib/fluent/plugin/in_tail.rb
@@ -443,7 +443,11 @@ module Fluent::Plugin
         return
       end

-      @tails[path] = tw
+      if @follow_inodes
+        @tails[target_info.ino] = tw
+      else
+        @tails[path] = tw
+      end
       tw.on_notify
     end

diff --git a/test/plugin/test_in_tail.rb b/test/plugin/test_in_tail.rb
index b1e28fc9..37fbf950 100644
--- a/test/plugin/test_in_tail.rb
+++ b/test/plugin/test_in_tail.rb
@@ -2156,13 +2156,13 @@ class TailInputTest < Test::Unit::TestCase
       target_info =<EMAIL_ADDRESS>       mock.proxy(Fluent::Plugin::TailInput::TailWatcher).new(target_info, anything, anything, true, true, anything, nil, anything, anything).once
       d.run(shutdown: false)
-      assert d.instance.instance_variable_get(:@tails)[target_info.path]
+      assert d.instance.instance_variable_get(:@tails)[target_info.ino]

       Timecop.travel(now + 10) do
         d.instance.instance_eval do
-          sleep 0.1 until @tails[target_info.path] == nil
+          sleep 0.1 until @tails[target_info.ino] == nil
         end
-        assert_nil d.instance.instance_variable_get(:@tails)[target_info.path]
+        assert_nil d.instance.instance_variable_get(:@tails)[target_info.ino]
       end
       d.instance_shutdown
     end

Even though with above fix, some more other tests are failed.
We need to fix these tests too.

@ashie
Thank you for your comment and analysis.
I'm digging it.

In our understanding, follow_inode false is just for keeping the compatibility with old verions of the tail plugin.

Basically your recognition is reasonable, we should make follow_inode true default when it's proofed enough. But I don't think it's proofed enough. It's introduced at v1.12 and we experienced some fatal issues in v1.12 - v1.15.

By the way, please let me explain why we didn't take care of follow_inode false case.

Even though in follow_inode case unexpected stall isn't acceptable.
Although follow_inode true is intended to resolve log duplication issue originally, modification for follow_inode feature seems introduce stall issue as in #3614. Such reports are increased after we merge this feature.

@ashie

Could you add partially or something similar to this line in your commit message & the first comment of this PR?

Sure, I rewrite commit message and comment of this PR.

Even though with above fix, some more other tests are failed.
We need to fix these tests too.

I revised and rewrite the codes.
All tests has passed locally.
Could you review this if all test passed in the github workflow?

Thanks for this fix! I'm seeing this too.

To me, this looks like a reasonable fix to address the problem of one scenario (https://github.com/fluent/fluentd/issues/3614#issuecomment-1578977365).
Though I think @ashie has a better grasp of the concerns of this fix, my concern is this code.
https://github.com/fluent/fluentd/blob/84b1b49a654121a57f16ab4f02cfac7656fc79e6/lib/fluent/plugin/in_tail.rb#L525
Why do we need this deletion?

https://github.com/fluent/fluentd/blob/84b1b49a654121a57f16ab4f02cfac7656fc79e6/lib/fluent/plugin/in_tail.rb#L525
Why do we need this deletion?

Hmm? Perhaps I'm confusing something...

Why do we need this deletion?

Whey the key for @tails is path, old TailWatcher can be dropped just by @tails[path] = tw.
But when the key is inode, old one cannot by @tails[inode] = true, need to drop it explicitly.
rotated_target_info.ino != new_ino here.

Why do we need this deletion?

Whey the key for @tails is path, old TailWatcher can be dropped just by @tails[path] = tw. But when the key is inode, old one cannot be dropped by @tails[inode] = true, need to drop it explicitly. rotated_target_info.ino != new_ino here.

Yes, ashie-san is correct.
When using path as a key of "@tail", the key would be the same between a rotated log file and a new log file.
When using inode as a key, the key would be different between two files. So we need to delete old "@tail" entry here.

Why do we need this deletion?

Whey the key for @tails is path, old TailWatcher can be dropped just by @tails[path] = tw. But when the key is inode, old one cannot be dropped by @tails[inode] = true, need to drop it explicitly. rotated_target_info.ino != new_ino here.

Thanks. I can imagine that, but something seems inconsistent to me...
The current implementation of managing TailWatcher by the paths may not be wrong even if follow_inodes is enabled.
For TailWatcher, the member @path and @ino are both constant.
So we have to recreate it when one of those needs to be updated.
Currently, TailWatchers are managed by linking the paths.
Basically, it has nothing to do with follow_inodes.
It should be symmetrical whether we use the paths or the inodes as the keys for @tails.
So, I think something is wrong here with the need for deletion.

If we use the inodes for the keys of @tails, shouldn't we recreate the watchers by the update of the paths for each inode?
Otherwise, it's not symmetrical.
(I don't think it has to be symmetrical if it doesn't seem to have any unintended effects, though)

Sorry, maybe I'm saying something wrong...
I will reconsider this asymmetry.

Now I understand the cause of this asymmetry.
It is caused by the watcher getting a new inode from the watcher's (old) path to determine the update.

I have still some concerns about changing the key...

Now I understand the cause of this asymmetry. It is caused by the watcher getting a new inode from the watcher's (old) path to determine the update.

Given this, it does not seem wrong to manage watchers by using the paths as the keys, regardless of follow_inodes.
The current update process of update_watcher seems not wrong to me.
It seems to me that the problem exists in refresh_watchers.

Given this, it does not seem wrong to manage watchers by using the paths as the keys, regardless of follow_inodes.

You are right, collecting TailWatchers is done by path by refresh_watcher (according to path setting in <source> section), so managing by path isn't wrong regardless of follow_inodes.
I think changing the key is just for comprehensibility of the code.

Sorry, I couldn't understand what is "asymmetry".
Please let me explain my idea.
For example, let's consider the case of that fluentd watches "/var/log/0.log".
Now the log file was rotated. An old "/var/log/0.log" uses inodeA and a new "/var/log/0.log" uses inodeB.
When using a path as a key of @tails,  both of old and new TailWatcher objects would be stored to the same @tails["/var/log/0.log"].
On the other hand, when using an inode as a key of @tails, the old one would be stored to @tails[inodeA] but the new one would be stored to @tails[inodeB].
So @tails[inodeA] would be no longer used after "/var/log/0.log" got rotated. That's why we remove @tails[inodeA] entry.
It doesn't mean that we dispose the old TailWatcher object here. It would be disposed by the following line.
      detach_watcher_after_rotate_wait(rotated_tw, pe.read_inode) if rotated_tw

The old TailWatcher would be disposed in the same line even in follow_inodes false.
So I think that it's symmetry.

Given this, it does not seem wrong to manage watchers by using the paths as the keys, regardless of follow_inodes.
The current update process of update_watcher seems not wrong to me.

To change the key from path to inode is the most important point of this fix.
We believe that the issue is caused by using the same @tails entry even after the rotation.
As what #4190 explains, @tails["/var/log/0.log"] might have been already updated to the new "/var/log/0.log" by refresh_watchers(). In that case, update_watcher() would close the new one wrongly.
The path is the same even after the rotation, but the inode is changed.
So a different @tails entry would be used after the rotation if we use inode as a key. We can prevent wrongly closing by that.

I think changing the key is just for comprehensibility of the code.

Our idea is to use inode as a key of all functions of in_tail.rb when follow_inodes true.
The plugin would be no longer work if we changed the key only in some of functions.

Thanks, everyone.
I am wondering if the key of @tails is not a fundamental part of this problem.
I'm still not sure, but I commented in https://github.com/fluent/fluentd/issues/3614#issuecomment-1580130292 on what I thought was the possible alternative solution.

To change the key from path to inode is the most important point of this fix.
We believe that the issue is caused by using the same @tails entry even after the rotation. As what #4190 explains, @tails["/var/log/0.log"] might have been already updated to the new "/var/log/0.log" by refresh_watchers(). In that case, update_watcher() would close the new one wrongly.

I think changing the key is just for comprehensibility of the code.

Our idea is to use inode as a key of all functions of in_tail.rb when follow_inodes true. The plugin would be no longer work if we changed the key only in some of functions.

Ah, sorry, just for comprehensibility of the code is misleading.
We understand what's wrong and why this PR can fix the issue.
We are talking about whether there are more ideal solution or not.
Because tail watchers are collected according to path setting in <source> section, managing by path isn't wrong ideally.
(It causes this issue in actual though).
We think there are some other ideas to resolve it without changing the key, #4191 is one of such idea.

Sorry, I couldn't understand what is "asymmetry".

Sorry, I just wanted to say that path as the key is reasonable for the current structure, regardless of follow_inodes.
A: The current implementation:

Key of @tails: path
Update of TailWatcher is processed based on the path

Based on the path, it sees the change of ino.

B: This PR's implementation:

Key of @tails: ino
Update of TailWatcher is processed based on the path

Based on the path, it sees the change of ino.

C:  State that I feel is symmetrical to A. (In this case, we don't need to worry about the deletion.)

Key of @tails: ino
Update of TailWatcher is processed based on the ino

Based on the ino, it sees the change of path.

The important point is that the current TailWatcher is updated based on its path by TimerTrigger or StateWatcher, and we don't want to change it.
B will certainly solve #3614 problem partially, but if we only consider the update process of TailWatcher (update_watcher), the current A is reasonable.

Thanks to this PR and #4191, I am starting to see problems that I didn't understand before.
Thank you very much. I would like to think about it some more.

Hi @ashie san, @daipom san,
Thank you for the explanations.
Maybe I got your idea.

The important point is that the current TailWatcher is updated based on its path by TimerTrigger or StateWatcher, and we don't want to change it.

I couldn't find a code that fluentd passes path to TimerTrigger.
But at least fluentd passes path to StatWatcher.
StatWatcher is the child class of Coolio::StatWatcher. We cannot modify Coolio::StatWatcher to use inode instead of path.
I imagine that's why you guys want to keep path as the key of @tails.
But I feel that it's not weird even if we use inode as the key even though StatWatcher still uses path.
In my understanding:

StatWatcher is used just for detecting the update of a log file specified by path.
tw.on_notify() will be called if StatWatcher detected the update. It calls @rotate_handler.on_notify(stat) then check if inode got changed.
@on_rotate would be called if inode change was detected.

To change the key of @tails would have no impact for the above.
on_rotate() cares inode. So I feel that it's natural that @tails is also managed by inode.

We think there are some other ideas to resolve it without changing the key, #4191 is one of such idea.

Currently I haven't found any negative side-effect of #4191.
I agree with that #4191 is good if fluentd community wants a small fix.
(I also think that #4191's if @follow_inodes && tw.ino != ino might be able to be changed just to if tw.ino != ino, even though I haven't chekced it deeply)
But #4191 is a last catch fix as @garyzjq san said.
In our opinion, to use path as the key of @tails even in follow_inodes true case can reduce the maintainability of this plugin.
The difference of the behavior of follow_inodes true and follow_inodes false is big, I think.
For example:

The same entry of pos file is updated even after rotation in follow_inodes false
But a new entry is created in follow_inodes true case.

In follow_inodes false case:
/var/log/pods/aaaa/bbbb/0.log<PHONE_NUMBER>36aab7<PHONE_NUMBER>60571f <= the same entry is updated even after rotation

In follow_inodes true case:
/var/log/pods/aaaa/bbbb/0.log      ffffffffffffffff<PHONE_NUMBER>60571f
/var/log/pods/aaaa/bbbb/0.log      ffffffffffffffff<PHONE_NUMBER>605732
/var/log/pods/aaaa/bbbb/0.log<PHONE_NUMBER>36aab7<PHONE_NUMBER>60571f <= A new entry is added after rotation

To match the behavior of @tails to the above is good to avoid confusion, I think.
Of course I can understand that a big code change isn't preferred to avoid regressions.
So It's still ok if the community doesn't choose our fix. It's a difficult decision...

@masaki-hatada Thanks for considering my concern!

The important point is that the current TailWatcher is updated based on its path by TimerTrigger or StateWatcher, and we don't want to change it.

I couldn't find a code that fluentd passes path to TimerTrigger.
But at least fluentd passes path to StatWatcher.

This is about here (TailWatcher::on_notify).
https://github.com/fluent/fluentd/blob/dcc05e59e038ffef16ee2b510d3d9e34b23bbdfe/lib/fluent/plugin/in_tail.rb#L807-L815
TailWatcher uses path as a constant value and takes a new inode of the path.

Maybe @ashie isn't as hesitant to change the key as I am. :smile:

@daipom san,
Thank you for the reply.

This is about here (TailWatcher::on_notify).
https://github.com/fluent/fluentd/blob/dcc05e59e038ffef16ee2b510d3d9e34b23bbdfe/lib/fluent/plugin/in_tail.rb#L807-L815
TailWatcher uses path as a constant value and takes a new inode of the path.

I feel that to use path here is good in any case, because there is no os systemcall which requires inode instead of path to check the stat.
Even if such a systemcall existed, we would become unable to detect the rotation by that. (The old inode would still point to the old log file even after the file got renamed)
It's ideal that to get the latest stat of a file by path then check whether its inode got updated or not.

Maybe @ashie isn't as hesitant to change the key as I am.

Ok!
I will wait the decision of him.
I or @kattz-kawa can update our code if needed.
(Adding more comments to better understanding, etc)

Yes. I don't want to change it, as I said before. (Sorry for my poor explanation.)

No, your explanation was good. ^^;

Based on this fact (path is the constant value for the watcher), it seems natural to manage the watchers by its' paths as the keys.

What I want to say is that we don't need to match the key of @tails and the above.
For example:

expand_paths() uses inode as a key of hash when follow_inode true case.
https://github.com/fluent/fluentd/blob/fbc3c2d2e2875cfea71fb36e4648dd2e7e5d33b9/lib/fluent/plugin/in_tail.rb#L342
But expand_paths() still uses path for detecting new files.
https://github.com/fluent/fluentd/blob/fbc3c2d2e2875cfea71fb36e4648dd2e7e5d33b9/lib/fluent/plugin/in_tail.rb#L289

Can we change expand_paths() as to always use inode instead of path?
It's impossible, because to detect new files can be done only with path.
We can say the same thing for Fluent::FileWrapper.stat(@path) and @tails.
Fluent::FileWrapper.stat(@path) cannot be changed to use inode. But there is no relation between it and the key of @tails.
We can also say that inode should be used for @tails since expand_paths() uses inode for hash.
Although it depends on the way of thinking.

As @kattz-kawa says in https://github.com/fluent/fluentd/issues/3614#issuecomment-1573422980, I am concerned this fix may cause a leak.
I will share more details later.
I feel the need to make the related test codes...

Use inodes as the key of @tails
Common condition

follow_inodes is enabled.
Files: file.log (inode-A)

@tails[inode-A] => Watcher(file.log, inode-A)

Rotation!

file.log is moved to file.log.1.
A new file is created as file.log.

Files: file.log (inode-B), file.log.1 (inode-A)
refresh_watcher and update_watcher can be processed at the same time...

If refresh_watcher is proceessed before update_watcher

refresh_watcher

Find file.log (inode-B) as a new target.
@tails[inode-A] => Watcher(file.log, inode-A) Not changed
@tails[inode-B] => Watcher(file.log.1, inode-A) Newly added

update_watcher

Watcher(file.log, inode-A) will call it's update_watcher

This will update the inode of file.log to inode-B.
Detach and delete @tails[inode-A] (Watcher(file.log, inode-A))

It will read the remained data of file.log.1 (inode-A) since the io_handler is not reopened.
(really?? What about open_on_every_update?)

Overwrite @tails[inode-B] by Watcher(file.log, inode-B) WRONG

@tails[inode-B] already has a watcher. We can't overwrite it here.

Result:

@tails[inode-B] => Watcher(file.log, inode-B)
Watcher(file.log.1, inode-A) remains, but it does not exist in @tails. LEAK

Certainly, we can trace the new current file, but there seems to be a leak of the watcher.

For comparison.
Use paths as the key of @tails
Common condition

follow_inodes is enabled.
Files: file.log (inode-A)

@tails[file.log] => Watcher(file.log, inode-A)

Rotation!

file.log is moved to file.log.1.
A new file is created as file.log.

Files: file.log (inode-B), file.log.1 (inode-A)
refresh_watcher and update_watcher can be processed at the same time...

If refresh_watcher is processed before update_watcher

refresh_watcher

Find file.log (inode-B) as a new target.
Overwrite @tails[file.log] by Watcher(file.log, inode-B)

WRONG @tails[file.log] already has a watcher. We can't overwrite it here.

Result:

@tails[file.log] => Watcher(file.log, inode-B)
Watcher(file.log, inode-A) remains, but it does not exist in @tails. !!LEAK!!

update_watcher

Watcher(file.log, inode-A) will call it's update_watcher

This will update the inode of file.log to inode-B.
Detach and delete @tails[file.log] (Watcher(file.log, inode-B)) !!WRONG!!

This stops tracking the new file!!

Overwrite @tails[file.log] by Watcher(file.log, inode-B) !!Meaningless!!

@tails[file.log] already has the completely same watcher.

Result:

@tails[file.log] => Watcher(file.log, inode-B)

Once detached, so it is no longer read?

Watcher(file.log, inode-A) still remains without detached.

I thought of those simulations on the table.
I want to check the actual behavior later. (In addition, I want to create the test codes...)

Watcher(file.log.1, inode-A) remains, but it does not exist in @tails. !!LEAK!!

I think the watcher will be an orphan.
I still don't know if this will lead to some kind of resource leak or not.
In https://github.com/fluent/fluentd/issues/3614#issuecomment-1578261598, @ashie says

they will be scheduled to free by rotate_watcher in this case.

@daipom san,
Thank you for the explanation.

Use inodes as the key of @tails
Common condition

follow_inodes is enabled.

Files: file.log (inode-A)

@tails[inode-A] => Watcher(file.log, inode-A)

Rotation!

file.log is moved to file.log.1.
A new file is created as file.log.

Files: file.log (inode-B), file.log.1 (inode-A)

refresh_watcher and update_watcher can be processed at the same time...

If refresh_watcher is processed before update_watcher

refresh_watcher

Find file.log (inode-B) as a new target.
@tails[inode-A] => Watcher(file.log, inode-A) Not changed

In this case, the watcher would be detached by stop_watchers() called from refresh_workers(), I think.

@tails[inode-B] => Watcher(file.log.1, inode-A) Newly added

update_watcher

Watcher(file.log, inode-A) will call it's update_watcher

This will update the inode of file.log to inode-B.

Detach and delete @tails[inode-A] (Watcher(file.log, inode-A))

It will read the remained data of file.log.1 (inode-A) since the io_handler is not reopened.
(really?? What about open_on_every_update?)

Overwrite @tails[inode-B] by Watcher(file.log, inode-B) !!WRONG!!

@tails[inode-B] already has a watcher. We can't overwrite it here.

Please correct me if I was wrong, but maybe the following "if" statement is for avoiding this.
https://github.com/fluent/fluentd/blob/84b1b49a654121a57f16ab4f02cfac7656fc79e6/lib/fluent/plugin/in_tail.rb#L521
In my understanding, "new_position_entry.read_inode == 0" means that the new log file hasn't been found by refresh_watchers() yet.
update_watcher() updates @tails[inode-B] only in this case.
I also had though that the current implementation of update_watcher() is very complicated, but I has hesitated to modify this since it would be a very big change...

@masaki-hatada
Sorry for my late response.

In my understanding, "new_position_entry.read_inode == 0" means that the new log file hasn't been found by refresh_watchers() yet.

Thanks!
I will fix the comments!

Sorry for my incorrect description.
I updated the following 2 comments.

https://github.com/fluent/fluentd/pull/4185#issuecomment-1580767216
https://github.com/fluent/fluentd/pull/4185#issuecomment-1580772174

I ran #4207 test_updateTW_after_refreshTW test with this fix.
The test fails because the expected value does not match this modification, but that is not the problem with this PR.
This PR certainly solves one of in_tail's problems.
<{:position_entries=>
  [["/home/daipom/work/fluentd/fluentd/test/plugin/../tmp/tail/876881f23793c42b295c/tail.txt",
    "ffffffffffffffff",
    45351307],
   ["/home/daipom/work/fluentd/fluentd/test/plugin/../tmp/tail/876881f23793c42b295c/tail.txt",
    "0000000000000016",
    45351310]],
 :record_values=>["file1 log1", "file1 log2", "file2 log1", "file2 log2"],
 :tail_watcher_inodes=>[45351307, 45351310, 45351307],
 :tail_watcher_io_handler_opened_statuses=>[false, false, false],
 :tail_watcher_paths=>
  ["/home/daipom/work/fluentd/fluentd/test/plugin/../tmp/tail/876881f23793c42b295c/tail.txt",
   "/home/daipom/work/fluentd/fluentd/test/plugin/../tmp/tail/876881f23793c42b295c/tail.txt",
   "/home/daipom/work/fluentd/fluentd/test/plugin/../tmp/tail/876881f23793c42b295c/tail.txt1"]}> expected but was
<{:position_entries=>
  [["/home/daipom/work/fluentd/fluentd/test/plugin/../tmp/tail/876881f23793c42b295c/tail.txt",
    "0000000000000016",
    45351307],
   ["/home/daipom/work/fluentd/fluentd/test/plugin/../tmp/tail/876881f23793c42b295c/tail.txt",
    "0000000000000016",
    45351310]],
 :record_values=>["file1 log1", "file1 log2", "file2 log1", "file2 log2"],
 :tail_watcher_inodes=>[45351307, 45351310],
 :tail_watcher_io_handler_opened_statuses=>[false, false],
 :tail_watcher_paths=>
  ["/home/daipom/work/fluentd/fluentd/test/plugin/../tmp/tail/876881f23793c42b295c/tail.txt",
   "/home/daipom/work/fluentd/fluentd/test/plugin/../tmp/tail/876881f23793c42b295c/tail.txt"]}>

Based on the information I have learned from this PR and #4191, I have created a PR.

#4208

If there is no problem with #4208, I think #4208 would be a more direct solution.

Sorry for my incorrect description. I updated the following 2 comments.

Thank you for the update!

in_tail: Fix for unexpected file close after logs rotate in update_wa… #4185 (comment)

I have a comment about this.

update_watcher

Watcher(file.log, inode-A) will call it's update_watcher

This will try to update the inode of file.log to inode-B and try to delete @tails[inode-A] (Watcher(file.log, inode-A)).

However, the update and deletion is not executed because inode-B is already found (new_position_entry.read_inode != 0)

Only detach

Overwrite @tails[inode-B] by Watcher(file.log, inode-B) !!WRONG!!

@tails[inode-B] already has a watcher. We can't overwrite it here.

If my understanding is correct, "Overwrite @tails[inode-B] by Watcher(file.log, inode-B)" won't happen in "new_position_entry.read_inode != 0" case.
(Maybe you just forgot to remove this line, I think)
https://github.com/fluent/fluentd/blob/84b1b49a654121a57f16ab4f02cfac7656fc79e6/lib/fluent/plugin/in_tail.rb#L515-L529

@masaki-hatada Thanks!  I'm sorry. I forgot to remove it.
I have updated it again!

@kattz-kawa @masaki-hatada I have pushed the commit to solve the conflicts.
After all, I think we can change the key to solve some problems in #4221.

Hmm, there is a failure case that is concerned with this :thinking:
https://github.com/fluent/fluentd/actions/runs/5408475806/jobs/9827580198?pr=4185
Failure: test_updateTW_before_refreshTW_and_detach_after_refreshTW(TailInputTest::Update watchers for rotation with follow_inodes)
/home/runner/work/fluentd/fluentd/test/plugin/test_in_tail.rb:2814:in `test_updateTW_before_refreshTW_and_detach_after_refreshTW'
     2811:         end
     2812:       end
     2813:
  => 2814:       assert_equal(
     2815:         {
     2816:           record_values: ["file1 log1", "file1 log2", "file2 log1", "file2 log2"],
     2817:           tail_watcher_paths<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><{:position_entries=>
  [["/home/runner/work/fluentd/fluentd/test/plugin/../tmp/tail/94e63360cbdd8f447f72/tail.txt",
    "ffffffffffffffff",
    1114610],
   ["/home/runner/work/fluentd/fluentd/test/plugin/../tmp/tail/94e63360cbdd8f447f72/tail.txt",
    "0000000000000016",
    1114614]],
 :record_values=>["file1 log1", "file1 log2", "file2 log1", "file2 log2"],
 :tail_watcher_inodes=>[1114610, 1114614, 1114610],
 :tail_watcher_io_handler_opened_statuses=>[false, false, false],
 :tail_watcher_paths=>
  ["/home/runner/work/fluentd/fluentd/test/plugin/../tmp/tail/94e63360cbdd8f447f72/tail.txt",
   "/home/runner/work/fluentd/fluentd/test/plugin/../tmp/tail/94e63360cbdd8f447f72/tail.txt",
   "/home/runner/work/fluentd/fluentd/test/plugin/../tmp/tail/94e63360cbdd8f447f72/tail.txt1"]}> expected but was
<{:position_entries=>
  [["/home/runner/work/fluentd/fluentd/test/plugin/../tmp/tail/94e63360cbdd8f447f72/tail.txt",
    "0000000000000016",
    1114610],
   ["/home/runner/work/fluentd/fluentd/test/plugin/../tmp/tail/94e63360cbdd8f447f72/tail.txt",
    "0000000000000016",
    1114614]],
 :record_values=>["file1 log1", "file1 log2", "file2 log1", "file2 log2"],
 :tail_watcher_inodes=>[1114610, 1114614, 1114610],
 :tail_watcher_io_handler_opened_statuses=>[true, false, false],
 :tail_watcher_paths=>
  ["/home/runner/work/fluentd/fluentd/test/plugin/../tmp/tail/94e63360cbdd8f447f72/tail.txt",
   "/home/runner/work/fluentd/fluentd/test/plugin/../tmp/tail/94e63360cbdd8f447f72/tail.txt",
   "/home/runner/work/fluentd/fluentd/test/plugin/../tmp/tail/94e63360cbdd8f447f72/tail.txt1"]}>

diff:
  {:position_entries=>
    [["/home/runner/work/fluentd/fluentd/test/plugin/../tmp/tail/94e63360cbdd8f447f72/tail.txt",
?     "ffffffffffffffff",
?<PHONE_NUMBER>000016
?      ????????????????
      1114610],
     ["/home/runner/work/fluentd/fluentd/test/plugin/../tmp/tail/94e63360cbdd8f447f72/tail.txt",
      "0000000000000016",
      1114614]],
   :record_values=>["file1 log1", "file1 log2", "file2 log1", "file2 log2"],
   :tail_watcher_inodes=>[1114610, 1114614, 1114610],
?  :tail_watcher_io_handler_opened_statuses=>[true, false, false, false],
   :tail_watcher_paths=>
    ["/home/runner/work/fluentd/fluentd/test/plugin/../tmp/tail/94e63360cbdd8f447f72/tail.txt",
     "/home/runner/work/fluentd/fluentd/test/plugin/../tmp/tail/94e63360cbdd8f447f72/tail.txt",
     "/home/runner/work/fluentd/fluentd/test/plugin/../tmp/tail/94e63360cbdd8f447f72/tail.txt1"]}

Hmm, there is a failure case that is concerned with this 🤔
https://github.com/fluent/fluentd/actions/runs/5408475806/jobs/9827580198?pr=4185

Oh, somehow, the first TailWatcher is not detached and not unwatched.
I've tried several times locally and still can't reproduce it.
We should check if there is a possibility that the results change depending on the timing...
Anyway, some handlers not being closed is a problem...

The following "path" should be changed to "key" or something for avoiding confusion. (We didn't notice this when posting the patch in past...)
https://github.com/fluent/fluentd/blob/719cdee2fe1235c7ef7bf69f3e708f3bf25c12ff/lib/fluent/plugin/in_tail.rb#L491-L492
Thank you for rebasing our patch. We will recheck this and write here if we found any issue.

The following "path" should be changed to "key" or something for avoiding confusion. (We didn't notice this when posting the patch in past...)
https://github.com/fluent/fluentd/blob/719cdee2fe1235c7ef7bf69f3e708f3bf25c12ff/lib/fluent/plugin/in_tail.rb#L491-L492

Thanks!
Fixed: https://github.com/fluent/fluentd/compare/719cdee2fe1235c7ef7bf69f3e708f3bf25c12ff..0915daf405ca75d9695fbd0094643b6365bfb365

Thank you for rebasing our patch. We will recheck this and write here if we found any issue.

Thanks! If you find any issue, feel free to fix it!

Hi @daipom san, @ashie san,
Just FYI.
Currently @kattz-kawa is also running https://github.com/fluent/fluentd/pull/4185#issuecomment-1612437497 in our local env, but we haven't faced the issue.

Thanks for your check! I want to conclude by the end of this week.
I will consider this point: https://github.com/fluent/fluentd/pull/4185#discussion_r1246115818

I'm also confusing why this commit can solve #4221 .
I leave a memo.
Concern1: Really is it ok if update_watcher() detaches rotated log files?
I had thought only the case that fluentd monitors only current log files.
(Because this is the configuration we use for our environment)
  path /test/fluentd/input4/test.log

Rotated log files can be detached in update_watcher() in this case.
But how about the case that fluentd monitors also rotated log files?
  path /test/fluentd/input4/test.log*

Rotated log files should continue to be monitored even after they got renamed. (e.g., test.log => test.log.1)
So my question is "Is it ok if update_watcher() detaches rotated log files in that case?".
If the answer is "No", we need to leave the detaching task to stop_watchers() called in refresh_watchers() instead of update_watcher().
Concern2: Why cannot stop_watchers() work when follow_inodes is true?
The following line mentions "When follow_inodes is true, it's not cleaned up by refresh_watcher".
https://github.com/fluent/fluentd/blob/155bdcabfc61180c0880a47725952e1db5fea49a/lib/fluent/plugin/in_tail.rb#L518
Maybe it means that stop_watchers() isn't called by refresh_watchers() when follow_inodes is true.
I considered why it isn't called.

expand_paths() creates a hash list. The key will be inode when follow_inodes is true.
https://github.com/fluent/fluentd/blob/155bdcabfc61180c0880a47725952e1db5fea49a/lib/fluent/plugin/in_tail.rb#L342
=> If inode-1 of a deleted file(log-A) was reused with another new log file(log-B), log-B's TailWatcher will be stored in to its hash[inode-1].
existence_path() also creates a hash list. The key will be inode when follow_inodes is true.
https://github.com/fluent/fluentd/blob/155bdcabfc61180c0880a47725952e1db5fea49a/lib/fluent/plugin/in_tail.rb#L357
=> log-A's TailWatcher will be stored into its hash[inode-1]
Compare existence_path's hash list and expand_paths's hash list to create unwatched_hash
https://github.com/fluent/fluentd/blob/155bdcabfc61180c0880a47725952e1db5fea49a/lib/fluent/plugin/in_tail.rb#L380
=> log-A's TailWatcher won't be added to unwatched_hash since we can find the same inode-1 key from expand_paths's hash list. But the entry of expand_paths's hash list is for log-B. This is not for log-A.
==> Cannot call stop_watchers() for log-A.
==> Cannot call start_watchers() for log-B due to the same reason.

So we need to detach log-A's TailWatcher in update_watcher() instead of stop_watchers().
(refresh_watchers() becomes able to call start_watchers() for log-B after the detaching)
How can we solve the conflict between Concern1 and 2?
We can ignore Concern1 if the expected behavior of the tail plugin is that TailWatcher object should be recreated every time the log file path got changed.
However, to avoid reading the same log twice, we should tell the previous position to the new TailWatcher object.
But where is the code to do that?
Isn't the plan B of https://github.com/fluent/fluentd/pull/4221#discussion_r1246007375 still needed even after we use inode as a key?

Thanks for your considering it!

Concern1: Really is it ok if update_watcher() detaches rotated log files?

I think It is OK.
Newly created TailWatchers will tail it.

TailWatcher-A (test.log (current))
Rotation

test.log -> test.log.1
new test.log

TailWatcher-A detaches test.log and newly creates TailWatcher-A' (test.log)
test.log.1 is newly found in refresh_watcher and a new TailWatcher-B (test.log.1) is created, but it doesn't tail it due to the pos file.
Rotation again

test.log.1 -> test.log.2
test.log -> test.log.1
new test.log

TailWatcher-A' detaches test.log and newly creates TailWatcher-A'' (test.log)
TailWatcher-B detaches test.log.1 and newly creates TailWatcher-B' (test.log.1)
test.log.2 is newly found in refresh_watcher ...
...

Concern2: Why cannot stop_watchers() work when follow_inodes is true?

It is mostly true.
This is because refresh_watcher cannot recognize that the inode is disappeared.

As you say,  refresh_watcher recognize the disappeared targets by comparing @tails and the existing file lists.
However, update_watcher overwrites @tails[path], so the information about the old inode of the path is lost from @tails.
So, refresh_watcher can't find the old inode from @tails, and can't recognize that the inode is disappeared.

This happens because the key of @tails is path.
If it is inode, then that overwriting doesn't occur, and refresh_watcher can unwatch the disappeared inode by rotation.

However, to avoid reading the same log twice, we should tell the previous position to the new TailWatcher object.
But where is the code to do that?

These lines load the position from the pos file.
If the previous position is recorded, it is loaded here and passed to the new TailWatcher.
https://github.com/fluent/fluentd/blob/e20697e12003e2c993ccf6f1c0bac190010b121b/lib/fluent/plugin/in_tail.rb#L435
https://github.com/fluent/fluentd/blob/e20697e12003e2c993ccf6f1c0bac190010b121b/lib/fluent/plugin/in_tail.rb#L509

@daipom san,
Thank you for your explanation!

Concern2: Why cannot stop_watchers() work when follow_inodes is true?

Your assumptions seem mostly correct.
This is because refresh_watcher cannot recognize that the inode is disappeared.

As you say, refresh_watcher recognize the disappeared targets by comparing @tails and the existing file lists.
However, update_watcher overwrites @tails[path], so the information about the old inode of the path is lost from @tails.
So, refresh_watcher can't find the old inode from @tails, and can't recognize that the inode is disappeared.

This happens because the key of @tails is path. If it is inode, then that overwriting doesn't occur, and refresh_watcher can unwatch the disappeared inode by rotation.

I was worried if @tails[inode] can be still overwritten by another log file due to inode re-use while fluentd still has a TailWatcher of the old log file.
But I noticed that it cannot happen, since inode won't be freed until fluentd closes the old file by update_watcher()^^;
So my concern was cleared.

However, to avoid reading the same log twice, we should tell the previous position to the new TailWatcher object.
But where is the code to do that?

These lines load the position from the pos file. If the previous position is recorded, it is loaded here and passed to the new TailWatcher.
https://github.com/fluent/fluentd/blob/e20697e12003e2c993ccf6f1c0bac190010b121b/lib/fluent/plugin/in_tail.rb#L435
https://github.com/fluent/fluentd/blob/e20697e12003e2c993ccf6f1c0bac190010b121b/lib/fluent/plugin/in_tail.rb#L509

Ok. We can refer to the previous position by using inode because we set follow_inodes to true.
My current concern is that the previous position might have been cleared by the following line.
https://github.com/fluent/fluentd/blob/e20697e12003e2c993ccf6f1c0bac190010b121b/lib/fluent/plugin/in_tail/position_file.rb#L58
The unwatch() function will be called when detach_watcher_after_rotate_wait() is called with tail_watcher.unwatched = true.
The current code sets tail_watcher.unwatched = true when follow_inodes is true.
https://github.com/fluent/fluentd/blob/e20697e12003e2c993ccf6f1c0bac190010b121b/lib/fluent/plugin/in_tail.rb#L504-L521

TailWatcher-A detaches test.log and newly creates TailWatcher-A' (test.log)
test.log.1 is newly found in refresh_watcher and a new TailWatcher-B (test.log.1) is created, but it doesn't tail it due to the pos file.

Isn't the entry of the pos file cleared before test.log.1 is newly found?

My current concern is that the previous position might have been cleared by the following line.
The unwatch() function will be called when detach_watcher_after_rotate_wait() is called with tail_watcher.unwatched = true. The current code sets tail_watcher.unwatched = true when follow_inodes is true.

TailWatcher-A detaches test.log and newly creates TailWatcher-A' (test.log)
test.log.1 is newly found in refresh_watcher and a new TailWatcher-B (test.log.1) is created, but it doesn't tail it due to the pos file.

Isn't the entry of the pos file cleared before test.log.1 is newly found?

Yes!
The PR #4221 is to fix this problem!

Instead of #4221, I would like to fix it as follows:

Change the key of @tails to inode (This PR)
Stop unwatching in update_watcher (Future PR)

Thank you too for all your help!
If you have any concerns or anything you've noticed, please let me know!

https://github.com/fluent/fluentd/pull/4185#discussion_r1246115818
I'm about the correct specifications.
I think the detached TailWathcer must be surely removed from @tails.
If not removing it from @tails, then it must not be detached.
We need to add the fix.

Instead of #4221, I would like to fix it as follows:

Change the key of @tails to inode (This PR)
Stop unwatching in update_watcher (Future PR)

Ok. I got that Both of PRs are required for fixing #4221 . Thank you for your explanation!

Sorry, I said I wanted to stop watching in update_wathcer in the future PR, but now I think we should do it in this PR.
It is not good to separate these fixes since using inodes for the key directly makes it unnecessary.

I fixed the concerns. I think this is ideal behavior.
I will check the actual behavior more.

Sorry, there was an easy mistake in the testcases.
Fixed: https://github.com/fluent/fluentd/compare/bc130634eebc03c45d8ab1170f8e5a162110d311..ce898dd572d5745663abd456486169d772daa8aa

Sorry, one test is still failing...
I will fix it.

This is because refresh_watcher cannot recognize that the inode is disappeared.
* As you say,  `refresh_watcher` recognize the disappeared targets by comparing `@tails` and the existing file lists.

* However, `update_watcher` overwrites `@tails[path]`, so the information about the old inode of the path is lost from `@tails`.

* So, `refresh_watcher` can't find the old inode from `@tails`, and can't recognize that the inode is disappeared.

This happens because the key of @tails is path.
If it is inode, then that overwriting doesn't occur, and refresh_watcher can unwatch the disappeared inode by rotation.

I'm sorry again 😢
By using inode for the key, the overwriting of @tails does not occur anymore.
However, we remove TailWatcher from @tails, so the information of old inode is lost from @tails and refresh_watcher can not recognize the inode is disappeared after all... 😢
https://github.com/fluent/fluentd/blob/ce898dd572d5745663abd456486169d772daa8aa/lib/fluent/plugin/in_tail.rb#L518

I believe the current logic is the ideal behavior for update_watcher if using inode for the key.
However, the problem of unwatching still remains.

The following is my idea, although there might be still some overlooks...

Use inode as a key of @tails if follow_inodes = true.
Always use path as a key of hash in existence_path() and expand_paths() regardless of follow_inodes is true or not.
Don't make update_watcher() to call detach_watcher_after_rotate_wait() if follow_inodes is true. Instead, update the path of its TailWatcher.
Make start_watchers() to create a new TailWatcher only when if the same inode doesn't exist in @tails  if follow_inodes is true.

The base idea of this is "Don't recreate TailWatcher even if log files got rotated. Keep using the same TailWatchers until fluentd stopped monitoring them".

@kattz-kawa @masaki-hatada I have pushed the commit to solve the conflicts.
After all, I think we can change the key to solve some problems in #4221.

I thought this PR could nicely solve the current problem of unwatching (#4221), but it was wrong.
The problem of unwatching should be fixed in another PR.

I'm very sorry for rebasing this branch again and again.
Now this PR is for simply changing the key.
I want to consider whether to include this PR in the next version while considering the remaining in_tail issues.
After all, the only change I made is to ensure that the old TailWatcher is removed from @tails. (https://github.com/fluent/fluentd/pull/4185#discussion_r1246115818)
https://github.com/fluent/fluentd/blob/45b193fbdb264d8ff41832e56aa7227e010ac17c/lib/fluent/plugin/in_tail.rb#L518
(It was done in if new_position_entry.read_inode == 0 block originally, but I moved it outside the if block)

@masaki-hatada @kattz-kawa I'm very sorry for troubling you all.
Thanks for your suggestion to solve the remaining problem of unwatching: https://github.com/fluent/fluentd/pull/4185#issuecomment-1630218546
Maybe that problem should be handled separately from this PR.
I'll take this idea into account and see what I can do about this problem.

For now we don't have plan changing the key from path to inode, so I close this.
In the future we may consider it again, we'll refer this pull request again at that time.
Thanks for your contribution!


Line editing and command history

It would be good to have a bash like command history browsable via up/down keys.

First just start using readline(), which adds shell-like like input line-editing. In a second step start using the readline lib's history features too.

Using readline() has the additional advantage that the lexer always works from string buffers supplied by our custom YY_INPUT macro, instead of files. This paves the way for multiple input files and user defined functions.


Add relationship between PersistentVolumes and Pods

Currently we do not show the user which Pods are using a given Persistent Volume.
In this PR we work towards supplying that information to the user.
BZ: https://bugzilla.redhat.com/show_bug.cgi?id=1402329
cc: @zakiva @simon3z

@simon3z please note that right now we only support a relationship between a pod and a persistent volume which is issued using a persistent volume claim.
Also, there will be a separate PR for the UI in the future.

miq-bot add label wip

miq-bot add_label wip

@miq-bot add_label wip

Was surprised to see alias_attribute used for associations, but looks like what people are doing.
alias_attribute :container_groups, :parents

:+1:

@zakiva PTAL.

@nimrodshn please add tests.

@simon3z Added tests.

@nimrodshn Are you going to add the other direction too? i.e. from a Container Group to the Persistent Volumes it uses?

@zakiva AFAIK we are not adding the opposite direction..
Just refactored tests please take a look.

@zakiva AFAIK we are not adding the opposite direction..

@nimrodshn Considering the discussion in this BZ: https://bugzilla.redhat.com/show_bug.cgi?id=1435235, we would like to enhance the UI usability of Volumes. Specifically, I think it will be nice to have here (and in the UI) the other direction too (Pod => Persistent Volumes).
cc @simon3z

@simon3z PTAL..

@zakiva I will make the opposite connection Pods->PV in a separate PR.
cc @simon3z

@moolitayer @zakiva @zeari can you review/approve?

Important note
The order of this merge is:

First, the back-end.
Second, the tests (this PR)
Third, the UI (this PR)

@nimrodshn any reason not to move the tests from https://github.com/ManageIQ/manageiq-providers-kubernetes/pull/3 to this PR under the parent container group namespace (app/models/container_group.rb)?
It's not good the tests and code live in different repositories (e.g some one tells you to fix something, you break the tests and don't know about it until the PR is merged, then no one else can merge anything till that is fixed etc.)

@moolitayer PTAL.

nit comment, LGTM :+1:
cc @moolitayer

cc @simon3z

LGTM :+1:
@miq-bot assign Fryguy
@miq-bot add_label fine/yes
@miq-bot add_label blocker
cc @Fryguy @chessbyte @blomquisg

I have to admit, I'm a little concerned when I see us introducing parent (which will probably be an parent_id type N+1), and it accesses it with a :through.
Let's still merge this, but, just want to put that out there.

@blomquisg can you please review merge this blocker?

I have to admit, I'm a little concerned when I see us introducing parent (which will probably be an parent_id type N+1), and it accesses it with a :through.
Let's still merge this, but, just want to put that out there.

@kbrock you have any suggestions around this?  I'm fine merging for now, but if you have a potential performance concern, maybe you could write up an issue to capture it?  I don't want it lost.

Thanks @blomquisg
@kbrock Iet me know what we can do to help. Generally PersistentVolume|ContainerVolume models have given us some grief since they have been added

@kbrock, I second @moolitayer on that; let me know what we can do to fix any concerns and issues.

@nimrodshn @moolitayer
The ancestry gem has been our knee jerk solution to represent parent.
When we want a parent/child relationship to span multiple classes (e.g.: ems/folders/vms) then we have been using relationships.
While I like to avoid relationships wherever possible, we are basically doing the same thing as we do with vmware trees. So that may be the right way of representing these trees.

Fine backport details:
$ git log -1
commit a2a4b0e1bb0c6a2d3aa57ad78541dc0a16cc39b5
Author: Greg Blomquist<EMAIL_ADDRESS>Date:   Wed Apr 26 12:59:32 2017 -0400

    Merge pull request #14231 from nimrodshn/add_pvc_relationship

    Add relationship between PersistentVolumes and Pods
    (cherry picked from commit cf353cb36771a508a1993699c1e0727dfbfcd923)

    https://bugzilla.redhat.com/show_bug.cgi?id=1447049


[codemod] Fix codemod crash on MuiDivider property

Closes https://github.com/mui/material-ui/issues/43109
Avoid accessing non-existing properties in the @mui/codemod.

Hey @Janpot, thanks for the report and PR. I pushed a refactor to use the findComponentDefaultProps util, let me know what you think.

Looks good 👍


Collector 'Getting Started' shell commands do not work in Powershell

This line does not work for me in Powershell:
./bin/otelcorecol_* --config ./examples/local/otel-config.yaml
Error message: "./bin/otelcorecol_* : The term './bin/otelcorecol_*' is not recognized as the name of a cmdlet, function, script file,
or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try
again.
At line:1 char:1

./bin/otelcorecol_* --config ./examples/local/otel-config.yaml

  + CategoryInfo          : ObjectNotFound: (./bin/otelcorecol_*:String) [], CommandNotFoundException
  + FullyQualifiedErrorId : CommandNotFoundException"

Thanks -- there is a note under those instructions that say the commands only work in bash.

Actually, it did not work neither in bash shell, but I realized that it was partly about the lack of make in Windows, so I downloaded the Windows version of it. However, it still does not work neither with git bash: "makefile:153: *** multiple target patterns.  Stop.", neither with PS or cmd. Powershell tells bit more: "FIND: Parameter format not correct
Access denied -" - which I assume means that there is no FIND on Windows, which is not a surprise, but this part of documentation should be about Windows. Am I missing something?

That part of the documentation isn't about running the collector on Windows specifically, it's supposed to be general instructions for running it locally.

@geff10 @austinlparker can I work on this, updating the note section ?

Go for it

@austinlparker thank you very much !!. Could you please assign this to me.

@austinlparker do you want me to update the note section or we need to define the steps for the windows users?

@rahuldimri are you still up for this?


GIGACHAD BOYFRIEND LOTS OF MONEY LOTS OF SKIBIDI RIZZ

THIS IS OUR SONG
https://github.com/endingwithali/boyfriendlist/assets/55166195/e7f34c88-59d9-4918-9118-a4d9691099c6

Late to the party. entry denied


Populate PackageSignals for obfuscation detection data

See comment to https://github.com/ossf/package-analysis/pull/486

Will not use for now, see #521


[DS-4084] - Relationships should link to Items via REST API

This commit is the first step to support https://jira.duraspace.org/browse/DS-4084
Using the commits in this PR, the HAL links are included in the relationship REST. The leftId and rightId properties have not yet been removed to avoid breaking the Angular UI.
In the next step, we'll implement a PR in Angular to use the HAL links. For testing purposes, it's probably best this PR is merged prior to the Angular PR
Once this PR and the Angular PR are merged, we'll create one additional REST PR to remove the leftId and rightId properties
You can see this PR in action at e.g. https://dspace7-entities.atmire.com/rest/#https://dspace7-entities.atmire.com/rest/api/core/relationships/530

The Travis CI build failed.
But I don't think it's related with this PR. Please check the error message:
Tests run: 3, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 143.553 sec - in org.dspace.app.rest.IdentifierRestControllerIT
Running org.dspace.app.rest.CommunityRestRepositoryIT

No output has been received in the last 10m0s, this potentially indicates a stalled build or something wrong with the build itself.
Check the details on how to adjust your build configuration on: https://docs.travis-ci.com/user/common-build-problems/#Build-times-out-because-no-output-was-received

The build has been terminated

@benbosman : If you are willing to rebase all configurable_entities PRs on the latest code, I've just merged #2286  to that branch.  This PR should hopefully fix the Travis CI oddities for all recent PRs (including this one).

The Travis failure is indeed not due to a problem in this PR. The code in this PR doesn't cause any errors when performing the tests locally.
I will verify whether I can rebase the branch because Raf is on leave

@tdonohue Thank you for the fix to Travis. The build now succeeds as well


Ontology update with relative path

Changes in ontology, documentation and diagram to include the description of relative paths and reflect in the diagram changes from core in rml:logicalTarget.

There seems to be something wrong? I only have an update of the diagram in the PR. The last commits seems to undo the changes in the ontology files?

ups I may have done sth wrong in the process... Some months ago I updated some stuff but didn't make the PR, let me check and re-do it


Sets crash answer tab

The answer coalescer takes individual answers and groups them together - that produces sets.  When trying to view one of these  answers with sets, clicking the answer tab crashes the viewer.  The debugger shows
Uncaught TypeError: store.getSetNodes is not a function

This should be fixed in version 0.2.0.


Allow updating container fields without sending the whole JSON

Users want to update parts of the container specification, without having to resend the whole container JSON. For example in order to update the Docker image tag, without having to include all the portMappings, etc.
Quotes from the DCOS Community Slack:

If I send the JSON for container with only the image property then it drops everything unless it has a default.

And:

hello - let’s say I have an application deployed with marathon referencing a docker image with the v1.0 tag. Is it possible with a simple http call update the docker image tag(and therefore deploying the new version of the application) without modifying the rest of the application? Of do I need to update the whole json manifest? Thanks

Note: This issue has been migrated to https://jira.mesosphere.com/browse/MARATHON-3199. For more information see https://groups.google.com/forum/#!topic/marathon-framework/khtvf-ifnp8.

Note: This issue has been migrated to https://jira.mesosphere.com/browse/MARATHON-3199. For more information see https://groups.google.com/forum/#!topic/marathon-framework/khtvf-ifnp8.


UI: Non visible elements should be hidden

If a Revery component has focus but then is no longer rendered, the focus attribute is still populated by the now invisible element. If an element is no longer visible it should also not be focused.

Ah, interesting! I think there are two cases to consider here:

Case 1: Focused node was removed from the node tree
Case 2: Focused node is not visible (ie, opacity = 0.0)

I think in Case 1, we'd want to relinquish focus, but not necessarily Case 2.

@bryphe does rendering empty i.e. React.listToElement([]) remove the node from the tree?, re 2 I agree the behaviour in the case isn't 100% clear or necessarily what a user would be expecting

@bryphe does rendering empty i.e. React.listToElement([]) remove the node from the tree?

If there was a node rendering the component, and then something triggered a state change to cause React.empty/React.listToElement([]) to be rendered - the previous element should be removed 👍

re 2 I agree the behaviour in the case isn't 100% clear or necessarily what a user would be expecting

I know some web apps actually depend on this behavior - there were some that would 'hijack' the input by having an invisible textbox follow the UI (it was better for perf to have a single  input element than many across the page, etc) - at least in the older web days 😄

re 2 I agree the behaviour in the case isn't 100% clear or necessarily what a user would be expecting

I know some web apps actually depend on this behavior - there were some that would 'hijack' the input by having an invisible textbox follow the UI

Is that the best proof for that assumption?
With the right accessibility model, that type of hackery would hopefully be unnecessary
example for arguments sake

if we assume a tree-like model with a cursor to represent the navigable elements, then when an element is removed yield focus to nearest parent
if we assume a flat list of navigable elements, then when an element is remove we decrement the cursor to the nearest usable element

I've can't count how many times I'v had to code around browsers requiring element requires a minimum of 0.00001 opacity, and height/width > 0 to be rendered. There seems some precedent at least for some level render optimizations
For instance, we focus is on a menu list which is then collapsed, do we really expect the focus to remain on the now dismissed menu item? Should we only return focus when the menu explicitly yields control back? That feels a bit cumbersome


Fix file path in call to clean_file

The call to i18n_tool transifex --pull fails during the call to the
clean_translated_locales function because the path to the *.po files
is incorrect. In dirname.files('*.po'), the result already includes
the path relative to the dirname, so it is incorrect to perform an
additional joinpath.

Thanks for the pull request, @regisb! I've created OSPR-3877 to keep track of it in JIRA. JIRA is a place for product owners to prioritize feature reviews by the engineering development teams.
Feel free to add as much of the following information to the ticket:

supporting documentation
edx-code email threads
timeline information ("this must be merged by XX date", and why that is)
partner information ("this is a course on edx.org")
any other information that can help Product understand the context for the PR

All technical communication about the code itself will still be done via the GitHub pull request interface. As a reminder, our process documentation is here.

Coverage remained the same at 72.432% when pulling 542fe2fc80acff8bbf5e07a02958e89baca8fe41 on overhangio:overhangio/fix-clean-file into fff15374a18e8823b2db680b9f9f6fc72e20c29d on edx:master.

@regisb Thank you for your contribution. @nedbat Can you review this please?

@regisb 🎉 Your pull request was merged!
Please take a moment to answer a two question survey so we can improve your experience in the future.


Convert generate API to async

Account generation is currently sync, keeping connection open until response. It could mean that client can shut down the connection after http conn alive timeout, and never get the account creation ack.
For automatic creation of accounts and the volatility of tx confirmation times, we should make the generate API asynchronous like the rest of POST requests.

This should be fixed when we do #1362


New version of cassandra-sys-rs required to build cleanly on stable

Currently, one of this project's dependencies does not build on stable Rust, so this does not either.
To fix, merge https://github.com/tupshin/cassandra-sys-rs/pull/6, publish the new version, and update this crate to use the new version.

As demonstrated by https://travis-ci.org/tupshin/cassandra-rs/builds/184235343, this has now been fixed. Thanks.


Roomdesc not displayed correctly

Issue #1:
window:door_stonesec connects window:rm_stonewalled (loc1) and window:rm_secret (loc2) and has both roomdesc and rooomdesc2, but at loc2 roomdesc is displayed
I see this:
This low secret passage leads south to dark room filled with shadowy pillars and
ancient mystery.  Small niches cut into the north wall allow access up to the ceiling.
The other walls, though sound, show some cracks.

Should be this: A hole in the ceiling leads &+Cup.&*
The door is actuated by window:stat_statue as below:

   The statue is of marble and depicts a mighty sorcerer, though its features have been disfigured by the vandalism of passing adventurers.

import sys
statue = MUD.getPhysical("window:door_stonesec")
st_loc1 = statue.getCurLoc()
st_loc2 = statue.getCurLoc2()
if (statue.getDoorState() == "open"):
st_loc1.sendMsg("The statue slides over and covers the hole.\n")
st_loc2.sendMsg("There is a grinding of stone as the hole is covered.\n")
statue.setDoorState("closed")
else:
st_loc1.sendMsg("The statue slides aside revealing a hole &+Cdown&.\n")
st_loc2.sendMsg("A hole leading &+Cup& appears in the ceiling.\n")
statue.setDoorState("open")
sys.exit(1)

 A huge marble &+Bstatue&* of a sorcerer stands here, beside a hole leading &+Cdown&*.
 A huge marble &+Bstatue&* of a sorcerer stands proudly against the north wall.
 A hole in the ceiling leads &+Cup.&*
 There is a &+Bhole&* in the ceiling covered by some sort of stone panel.

  You see nothing special.

  The hole is big enough to climb through, if the stone panel were moved.

No error messages for these issues either  in the process terminal or in mud.log

Fixed.


[Bug]: creating a tag on rebirth gives you the ability to press enter on packs/filters and play them as songs, crashing the game

Is there an existing issue for this?

[X] I have searched the existing issues

Contact Details
No response
Version Info
Latest available release
What operating system are you seeing the problem on?
No response
Bug Behavior
upon creating a tag you can enter the currently hovered element in the songwheel, regardless of if it's a song, pack or filter
also carries over to the search menu, which does the above when pressing enter instead of searching
Expected Behavior
No response
Reproduction Steps
No response
Anything else?
No response

done f2678fe1992ab83a3dd6c38395acfc57793cc9d4


MAIN-B-18863-hotfix

MAIN
This PR is for a hotfix for MAIN PR.  Bug was recently discovered where Prime Simulator was throwing 500 Internal Server error when attempting to create shipments. This issue is directly related to recent PR. This feature added a trigger (migrations/app/schema/20240402192009_add_shipment_locator_and_shipment_seq_num.up.sql). ) to update a new count (shipment_seq_num)column on moves table. This count column  is used to calculate a  new unique index (shipment_locator)column on mto_shipments table to allow system to uniquely id a shipment using the locator code + counter. For example, if a move has 3 shipments the new unique column for the respective shipments will have DUMMYCODE-01,  DUMMYCODE-02, DUMMYCODE-03 where DUMMYCODE is the the locator code for the move. The bug occurs whenever a newly created shipment is added. The insert of new shipment record executes a db trigger to update the count column on moves table and it uses that counter to  set shipment_locator column. The problem arises  during a secondary service call to update move data is executed  afterwards where it OVERRIDES the correct count value on moves table because the update uses stale data. This then causes the system to fall behind in the sequencing number for the next new shipment, thus creating a unique index constraint database error showing a 500 INTERNAL SERVER ERROR.
B-18862
Prime updates
B-18863
Office updates
B-18864
Rest API updates
INT PR
Summary
This encompasses B-18862, B-18863, and B-18864. Due to heavy overlap in required changes for the changes to be functional and for tests to pass they were combined.
All three tasks are centered around adding pickupAddress, destinationAddress, secondaryPickupAddress, and secondaryDestinationAddress to PPM shipments.
B-18864 (Micheal) encompasses the changes to the backend to support the new fields for both the office and prime api.
B-18863 (Cory) is specific to the Office UI.
B-18863 (Cory) is specific to the prime view within the Office UI.
Is there anything you would like reviewers to give additional scrutiny?
this article explains more about the approach used.
Verification Steps for the Author
These are to be checked by the author.

[ ] Tested in the Experimental environment (for changes to containers, app startup, or connection to data stores)
[ ] Have the Agility acceptance criteria been met for this change?

Verification Steps for Reviewers
These are to be checked by a reviewer.

[ ] Has the branch been pulled in and checked out?
[ ] Have the BL acceptance criteria been met for this change?
[ ] Was the CircleCI build successful?
[ ] Has the code been reviewed from a standards and best practices point of view?

Setup to Run the Code

Instructions for starting storybook
Instructions for starting the MilMove application
Instructions for running tests

How to test
Prime UI:
Create (Please verify this work both with and without secondary addresses):

Access the Office UI
Login as a Prime Simulator
Select a move
Select 'Create Shipment'
Fill in the fields for a ppm shipment
Submit the ppm
Verify the ppm now exists within the move

Update (Please verify this work both with and without secondary addresses):

Access the Office UI
Login as a Prime Simulator
Select a move
Select 'Update Shipment' for a PPM from the move (or create one)
Update fields
Submit the updates
Verify the ppm updates are displayed in the move

Office UI
Create (Please verify this work both with and without secondary addresses):

Access the Office UI
Login as a Service Counselor
Select a move
Select 'Add a new Shipment'
Fill in the fields for a ppm shipment
Submit the ppm
Verify the ppm now exists within the move

Update (Please verify this work both with and without secondary addresses):

Access the Office UI
Login as a Service Counselor
Select a move
Select 'Edit Shipment' for a PPM from the move (or create one)
Update fields
Submit the updates
Verify the ppm updates are displayed in the move

Frontend

[ ] There are no aXe warnings for UI.
[ ] This works in Supported Browsers and their phone views (Chrome, Firefox, Edge).
[ ] There are no new console errors in the browser devtools.
[ ] There are no new console errors in the test output.
[ ] If this PR adds a new component to Storybook, it ensures the component is fully responsive, OR if it is intentionally not, a wrapping div using the officeApp class or custom min-width styling is used to hide any states the would not be visible to the user.
[ ] This change meets the standards for Section 508 compliance.

Backend

[ ] Code follows the guidelines for Logging.
[ ] The requirements listed in Querying the Database Safely have been satisfied.

Database
Any new migrations/schema changes:

[ ] Follows our guidelines for Zero-Downtime Deploys.
[ ] Have been communicated to #g-database.
[ ] Secure migrations have been tested following the instructions in our docs.

Screenshots

is this safe to merge- after successful pre-build
?


[Network] Fix issue with auto-resolving LB child resources

Fix #4090.
Converts the LB subresource test to ScenarioTest, breaks it up into smaller chunks, and adds tests to prevent this feature from regression.

This checklist is used to make sure that common guidelines for a pull request are followed.
General Guidelines

[X] The PR has modified HISTORY.rst with an appropriate description of the change (see Modifying change log).

Command Guidelines

[X] Each command and parameter has a meaningful description.
[X] Each new command has a test.

(see Authoring Command Modules)

Codecov Report

Merging #4092 into master will increase coverage by 0.02%.
The diff coverage is 61.53%.

@@            Coverage Diff             @@
##           master    #4092      +/-   ##
==========================================
+ Coverage   70.49%   70.52%   +0.02%
==========================================
  Files         437      437
  Lines       28147    28151       +4
  Branches     4318     4318
==========================================
+ Hits        19843    19854      +11
+ Misses       7015     7009       -6
+ Partials     1289     1288       -1

Impacted Files
Coverage Δ

...etwork/azure/cli/command_modules/network/custom.py
56.33% <61.53%> (+0.56%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 76be766...70d50c4. Read the comment docs.


Accomplishment feature available to work on ?

Is someone working on accomplishment feature ?
if not than i would like to add it bc i somwhat implemented it in my site

png)
also what do u mean by overview ?

Hi @HenzelMoras it is not done yet. It would be great if you can send a PR.

also what do u mean by overview ?

A summary of what you did/learn on this course.

Hi @hossainemruz  okay on it.
i followed readme for contributing
which folder im supposed to edit
hugo-toha.github.io/themes/toha/{files-to-edit}

?

Yes.

okay thank you

Added on   #262


chore(deps): update terraform terraform-ibm-modules/event-notifications/ibm to v1.15.3

This PR contains the following updates:

Package
Type
Update
Change

terraform-ibm-modules/event-notifications/ibm (source)
module
patch
1.15.1 -> 1.15.3

Release Notes

terraform-ibm-modules/terraform-ibm-event-notifications (terraform-ibm-modules/event-notifications/ibm)
v1.15.3
Compare Source
Bug Fixes

deps: update terraform ibm to latest for the deployable architecture solution (#340) (84561e3)

v1.15.2
Compare Source
Bug Fixes

deps: update terraform-module (#344) (b55ed6c)

Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
👻 Immortal: This PR will be recreated if closed unmerged. Get config help if that's undesired.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

/run pipeline

/run pipeline

/run pipeline


[RFC] Create Easy Actor Interface via WASI and stdio

Summary
This RFC proposes a new bi-directional communication mechanism between WebAssembly guest modules (actors) and the wasmCloud host runtime that involves information exchange across captured/managed stdio pipes.
Rationale
The current wasmbus protocol requires the wasmCloud host to implement eight functions and the guest module to implement 1 and make extensive use of the functions exported by the host. Performing a single operation involves a complicated dance of sending numbers back and forth across the guest/host boundary and manually manipulating and extracting bytes to and from the module's linear memory.
The current protocol is also fairly inefficient. The most costly thing we can do in a WebAssembly host is cross the guest/host boundary, and we do that multiple times per invocation right now in order to maintain statelessness and optimize for reading and writing "just numbers".
While this protocol is platform and language agnostic, and any wasm32-targeting language should be able to implement it, the barrier to entry could be considered too high, with too much friction involved in someone creating a new actor SDK language binding.
The proposition is that instead of the complicated suite of "pointer hinting" functions we have now, a simpler exchange of data could be facilitated just by leveraging the stdio (pipes) capabilities of WASI. This would allow us to read and write entire blobs of data without having to manually manipulate pointers or linear memory (the WASI support in the underlying engine would take care of this for us).
Components and Implementation
There are two main components to the implementation of this. The first would be modifying the host such that it sends invocations to the actors (guest modules) via stdio, and reads data back from the modules via stdio. The following pseudocode describes the processing loops that would exist in the host and in the actors. The processing loop for actors would likely be hidden and simplified by the various language-specific SDKs.
This proposal does not suggest making any changes to the RPC traffic carried across the lattice via NATS. This is only for local communication between a host and actor.
Comm Loop (Host)
The host is responsible for plucking inbound messages off the wire (NATS) destined for a given actor. The invocation data is then sent to the actor and a loop is then started. This loop reads data from the actor's stdout until the data is a invocation response. Invocations pulled from stdout are processed by the host (e.g. by sending to the appropriate capability provider)
inv = deserialize(msg) // e.g. from NATS
stdio.write inv(msg)
do
   resp = stdio.read
   if resp is inv
      process(resp)
until resp is inv-r

Comm Loop (Guest)
The guest communications loop involves an infinite loop of waiting for new data to come in on stdin. Once new data (an invocation) does arrive, then the guest does whatever work is necessary. If the guest needs to make host calls (send invocations to actors or providers), then the guest will write an invocation to stdout and get an invocation response in return.
When the guest is finally finished processing the instigating invocation, it will return an invocation response to the host, signifying that it's done with its work.
while msg = stdio.read do
    <do work>
    stdio.write inv
    result = stdio.read inv-r
    stdio.write inv
    result = stdio.read inv-r
    <do work>
    stdio.write inv-r
end

Sample Exchange - Key-Value Counter
The following describes (roughly) the stdio message exchange pattern as it would be implemented in one of our more commonly used examples - the key value counter.
sequenceDiagram
    Host->>+Guest: Inv (HttpServer.HandleRequest)
    Guest->>+Host: Inv (KeyValue.Get)
    Host->>-Guest: Inv-R
    Guest->>+Host: Inv (KeyValue.Set)
    Host->>-Guest: Inv-R
    Guest->>-Host: Inv-R

Message Exchange Protocol
The reasoning behind this particular protocol is we want to make it simple and easy to read, generate, and parse. We do not want to require anyone to use things like protobufs or any other heavyweight format for exchanging data (remember that the format of the raw payload remains opaque to the host, and can be msgpack or whatever is dictated by the contract).
The other main requirement is since we're using stdio, we need to ensure that we don't accidentally treat some token in the middle of encoded data as a newline. This is why we use a preamble line that contains the payload size, allowing us to read a stream of bytes without applying meaning to newlines.
Transmitting an Invocation
While we have an Invocation struct already used for the NATS-based/lattice RPC transmission, we don't actually need that structure for this use case. For example, we don't need the hash, claims, anti-forgery token, etc. Instead we can use a 2-line stanza optimized for stdio where we can either read to the end of a line or read a fixed number of bytes from the buffer.
To tell the other side of a call that we're sending an invocation, we can use the following format:
INV [target] [contract] [link] <operation> <payload_size>\n
<raw binary payload>

The target, contract, and link fields are all optional and only needed when sending an invocation destined for a provider. When sending to an actor, the target field can be either the public key or the call alias (the same way it works today). The important mandatory fields from the first line are the operation and payload_size.
Transmitting an Invocation Response
Sending an invocation response to the other side of this exchange involves the following stanza. Here we send a numeric result code (0 for failure, 1 for success, the same as we do today). If the code indicates failure, the binary payload is assumed to be a UTF-8 encoded string containing a description of the error. If the code indicates success, then the payload is treated as the opaque blob it should be and is delivered accordingly.
INV-R <result_code> <payload_size>\n
<raw binary payload>

How do wasm runtimes support WASI api libraries without incurring the memory mangling overhead?
Are pipes a standard supported by multiple wasm runtimes?

Can we specify this protocol using wit and https://github.com/bytecodealliance/wit-bindgen? That way we can support running with guest languages out of the box, and then add codegen support when we can place it on our roadmap

this protocol doesn't need to be specified with anything, let alone wit. We're sending opaque blobs here, so we don't actually need code generation for this protocol. Further, the preamble line is just ASCII text, further eliminating the need for overhead like wit.
WASI gets us running with guest languages out of the box, not wit or wit-bindgen.

WASI gets us running with guest languages out of the box, not wit or wit-bindgen.

Okay great, I just wanted to have the simplest path forward to support additional languages. I'm mostly interested in adopting community Wasm standards wherever it makes sense and I think getting the little helper functions to parse the preamble line written in various languages is probably the only step we'd need

@autodidaddict this work has been supplanted by Wasm components right? Any reservation to closing this issue?

Correct this can close


refactor: #619

Closes https://github.com/openfga/openfga/issues/619
Reviewers: please click on this

This is directly related to https://github.com/openfga/openfga/pull/618.


Skipping keys when iohook is enabled on mac

I am using iohook in an electon app, its like an upwork tracker... I am listening to keyup event and when I start the tracker, my keys starts to skip... its very random and less intense.
Expected Behavior
When iohook is listing, I should be able to use the system normal. I should be able to type normal.
Current Behavior
When iohook is listening to keyup, my keys skip as I type.
Possible Solution
IDK
Steps to Reproduce (for bugs)

Start listing to keyup in an electron app.
start typing anywhere...

Context
It works find in linux and windows. only happen on mac. Have tested it on multiple macbooks.
Your Environment
abi: electron-76
"electron": "8.0.2",
"iohook": "^0.6.5",

I am having the same issue. It will catch the first key and then it will not catch any more.

Similar issue here. Only allowing 1 key down event. (currently running MacOS Mojave 10.14.6)
var ioHook = require("iohook")

ioHook.on('keydown', function (event) {
    console.log(event.keycode)

});

ioHook.start();

Same issue in a Nodejs app
Code :
"use strict";

const ioHook = require("iohook");

ioHook.on("keyup", (e) => {
  console.log(e);
});

ioHook.start(true);

Output :
→ node index.js
hook_run [1121]: Accessibility API is enabled.
hook_run [1172]: CGEventTapCreate Successful.
hook_run [1178]: CFMachPortCreateRunLoopSource successful.
hook_run [1183]: CFRunLoopGetCurrent successful.
hook_run [1196]: CFRunLoopObserverCreate successful.
dispatch_event [108]: Dispatching event type 1.
hook_enable [257]: Thread Result: (0).
process_key_released [464]: Key 0X1C released. (0X24)
dispatch_event [108]: Dispatching event type 5.
{
  shiftKey: false,
  altKey: false,
  ctrlKey: false,
  metaKey: false,
  keycode: 28,
  rawcode: 36,
  type: 'keyup'
}
process_key_pressed [367]: Key 0X20 pressed. (0X2)
dispatch_event [108]: Dispatching event type 4.
process_key_pressed [379]: Using dispatch_sync_f for key typed events.
ddddddd

Is this still an issue for anyone?

Just had this exact issue running macOS Big Sur 11.1
`const ioHook = require('iohook');
ioHook.on("keydown",function(keyPress){
console.log(Keycode: ${keyPress.keycode})
});
ioHook.start();`
Only output the key code once.

I'm also having the problem of only logging the first keystroke on a mac, has anyone solved it please? The very bad news is that I'm going to consider using jnativehook for java


Read permissions for provider

How should the permissions be setup to allow read-only access to a record provided by a custom listener?
I setup a repository to replicate the problem: https://github.com/geecu/deepstream.io-test-permission
When the client tries to read a record, it sends a CR action to the server. The server doesn't have the record, so it treats it as a C action, denying the client's access to the record, so the request never reaches the listener.

It just struck me: the client can have create permission on the record, without the write permission and then in works as expected.
It's a bit counter-intuitive to have the user create rights when all you need is read a record, even though the documentation can certainly be interpreted like that (create triggered when a record is requested for the first time) - it's quite subtle, IMO :)
Anyway, this issue can be closed now.


Remove temporary version override after update

Version overrides aren't temporary, they're left in the metadata. Thanks for keeping an eye on it though.


How to get an Object from api instead of array

Hi,
Thanks for this great tutorial. really you help me a lot to learn news ideas specially RxJava and Dagger 2.
I am new to Android and Java but i think, it would be great if you use Retrofit for api calss also Parcelables.
in my case, i have api response comes in objects not in array. so, what changes should i do to receive objects?
Thanks a lot

Hey @Fshamri this is not meant to be a forum, please post your question on Stackoverflow or similar.
Closing as not an issue for this project.


Intensity parsed incorrectly

I was adding tests from the METAR examples in AC 00-45G and the AIM and found that the following METAR results in an assertion when testing the weather group:
SPECI KMSN 012312Z 13021G26KT 1SM -FZRAPL BR OVC011 00/M02 A2988 RMK AO2 PK WND 14031/2255 SFC VIS 1 1/2 P0005 $
Assert failed. Expected:<light> Actual:<moderate>

This is an error in the weather group regex, which only allows FZ to be combined with one of FG, DZ or RA. The substring 'RAPL' gets parsed since that is the first and "best" match.


Istio should exit immediately during shutdown if envoy exits

Describe the feature request
If envoy exits after istio-proxy receives SIGTERM (perhaps because someone sends /quitquitquit to it), then istio-proxy should exit immediately.
Currently it will get stuck in Agent.terminate.

If exitOnZeroActiveConnections is true, then it will keep polling the number of active connections. Since this will fail, it will return -1 all the time, which will cause it to loop infinitely until the pod termination grace period expires.
If exitOnZeroActiveConnections is false, then it will pointlessly keep waiting until the terminationDrainDuration expires.

To fix this, it must pull from a.statusCh and take appropriate action.
If this is fixed, then use cases like ingress-nginx that cannot rely on exitOnZeroActiveConnections can at least issue /quitquitquit just before exiting to abort the terminationDrainDuration sleep prematurely.
Describe alternatives you've considered
None
Affected product area (please put an X in all that apply)
[ ] Docs
[ ] Installation
[x] Networking
[ ] Performance and Scalability
[ ] Extensions and Telemetry
[ ] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure
Affected features (please put an X in all that apply)
[ ] Multi Cluster
[ ] Virtual Machine
[ ] Multi Control Plane
Additional context

Are you asking if /quitquitquit should forcefully exit? If yes - I think that is fair given that it is currently only used in Jobs. But if it is used else where, we can also add /quitquitquit?shutdownImmediate .

No, I'm saying right now there is arguably a bug if envoy exits for any reason while istio-proxy is in the middle of shutdown (in Agent.terminate).
Also, I think you may have misunderstood. I am referring to the envoy endpoint. From what I understand, istio-proxy itself also has a /quitquitquit endpoint that causes it to send SIGTERM to itself, but that is separate.

I get to understand the issue, it is a potential race here, let me file a fix

@ramaraochavali It seems the istio-policy-bot needs you to do something else.

no stale

@howardjohn This is not stale.

no stale

Is anyone working on this?


Create stub files for all exercises

We have decided to require all file-based tracks to provide stubs for their exercises.
The lack of stub file generates an unnecessary pain point within Exercism, contributing a significant proportion of support requests, making things more complex for our students, and hindering our ability to automatically run test-suites and provide automated analysis of solutions.
We believe that it’s essential to understand error messages, know how to use an IDE, and create files. However, getting this right as you’re just getting used to a language can be a frustrating distraction, as it can often require a lot of knowledge that tends to seep in over time. At the start, it can be challenging to google for all of these details: what file extension to use, what needs to be included, etc. Getting people up to speed with these things are not Exercism’s focus, and we’ve decided that we are better served by removing this source of confusion, letting people get on with actually solving the exercises.
The original discussion for this is at https://github.com/exercism/discussions/issues/238.
Therefore, we’d like this track to provide a stub file for each exercise.

If this track already provides stub files for all exercises, please close this issue.
If this track already has an open issue for creating stubs, then my apologies. Please close one as a duplicate.
Otherwise, please respond to this issue with useful details about what needs to be done to complete this task in this track so that people who are not familiar with the track may easily contribute.

I can gladly help with that.
Currently we do have only stub files for Hello World and Hamming. Would it be correct in assuming, that we provide just empty files for the other exercises? IMO providing a valid function signature would be provided to much. Is that correct?

I think you're right that for PHP all we need is the empty file.

Sorry about the confusion with various referencing commits. I had troubles placing a new line at the end of the file. The last one is the valid one.


Update travis file to not include sudo

See this update from travis for more information:
https://docs.travis-ci.com/user/migrating-from-legacy/?utm_source=legacy-notice&utm_medium=banner&utm_campaign=legacy-upgrade

And we're not using sudo anywhere?

And we're not using sudo anywhere?

Didn't find it anywhere, no.

This test itself ran using container based infrastructure, so the pull request serves as a verification that this should work.


Error raised on merg of 2 operations

I am not sure what are the conditions which raised the error, but here is what I know.
When I merged two operations which both had "unknown  type" (but don't know if it is real "unknown type" or just "no type", an error was raised by the browser (didn't note the error, sorry).

Thank you for opening a ticket. We might need more information or STR here :p

I know that... But I couldn't get the exact conditions when I oppened the ticket. I'll try to reproduce on my dev instance.

OK. I didn't reproduce, but on reading the code, I understood what happened:

Merge 2 operations with no operation type set.
Here: https://github.com/bnjbvr/kresus/blob/incoming/client/store.js#L575
The type is not set to the unknownTypeID, then the error was raised.

Good catch. I'll take care of fixing this; we shouldn't just apply the unknown operation type afterwards, but make sure that creating an operation does it for us. Also, there's a discrepancy between the naming of types, type and operationTypeId, which is at fault here.

Fixed by https://github.com/bnjbvr/kresus/commit/c6441b3e0990fad7a556b845983bcb136375e9cf .
If you have the opportunity, can you test and confirm, please?


[Test] Support nested services

Changes
Test PR with custom OpenAPI spec

Codecov Report
Attention: Patch coverage is 47.45763% with 31 lines in your changes are missing coverage. Please review.

Project coverage is 57.92%. Comparing base (8537cd3) to head (68bb5cc).
Report is 1 commits behind head on main.

Files
Patch %
Lines

databricks/sdk/service/settings.py
47.45%
31 Missing :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #558      +/-   ##
==========================================
+ Coverage   57.91%   57.92%   +0.01%
==========================================
  Files          45       45
  Lines       28291    28312      +21
==========================================
+ Hits        16385    16401      +16
- Misses      11906    11911       +5

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


changed around dependencies

i think this is what the problem was - i used save-dev when i should've used save with npm...? -_- my bad

lgtm


Fix permissions issues with S6 Overlay 3.x

What this PR does

It utilizes s6-setuidgid to set proper permissions for webuser (important for PHP commands + Laravel)
I also cleaned up a few other permission related things

Things to test

[ ] Ensure the queue works well. I don't know enough about the app to test it
[ ] Ensure the /app/config.yml works well. I had to require "root" to copy this over

Great work @alexjustesen!
My last PR had a ton of improvements and suggestions, but then I realized I was on the wrong friggen branch 😆
When I switched to the correct branch, you already had almost all of my suggestions done 🥳
Keep up the great work!

Hell yeah @jaydrogers, thanks a ton for looking into this and glad I could help (somewhat) commit to improving the base images too!


chore(deps): upgrade runtime dependencies

Upgrades project dependencies. See details in workflow run.

Automatically created by projen via the "upgrade-runtime-dependencies-2.x" workflow

⚪ Backport skipped
The pull request was not backported as there were no branches to backport to. If this is a mistake, please apply the desired version labels or run the backport tool manually.
Manual backport
To create the backport manually run:
backport --pr 2854

Questions ?
Please refer to the Backport tool documentation